Utilising affordable smartphones and open-source time-lapse photography for pollinator image collection and annotation

Valentin Ștefan; Aspen Workman; Jared C. Cobain; Demetra Rakosy; Tiffany M. Knight

doi:10.26786/1920-7603(2025)778

Authors

Valentin Ștefan Helmholtz Centre for Environmental Research - UFZ, Department of Species Interaction Ecology, Halle (Saale), Germany. https://orcid.org/0000-0002-4757-8008
Aspen Workman Helmholtz Centre for Environmental Research - UFZ, Department of Species Interaction Ecology, Halle (Saale), Germany. https://orcid.org/0009-0001-0504-2205
Jared C. Cobain Helmholtz Centre for Environmental Research - UFZ, Department of Species Interaction Ecology, Halle (Saale), Germany.
Demetra Rakosy Helmholtz Centre for Environmental Research - UFZ, Department of Species Interaction Ecology, Halle (Saale), Germany.
Tiffany M. Knight

DOI:

https://doi.org/10.26786/1920-7603(2025)778

Keywords:

smartphones, plant-pollinator interactions, time-lapse photography, monitoring, image observation, arthropod identification

Abstract

Monitoring plant-pollinator interactions is crucial for understanding the factors influencing these relationships across space and time. Traditional methods in pollination ecology are resource-intensive, while time-lapse photography offers potential for non-destructive and automated complementary techniques. However, accurate identification of pollinators at finer taxonomic levels (i.e., genus or species) requires high enough image quality. This study assessed the feasibility of using a smartphone setup to capture time-lapse images of arthropods visiting flowers and evaluated whether these images offered sufficient resolution for arthropod identification by taxonomists. Smartphones were positioned above target flowers from various plant species in urban green areas around Leipzig and Halle, Germany. We present proportions of arthropod identifications (instances) at different taxonomic levels (order, family, genus, species) based on visible features in the images as interpreted by taxonomists. We document whether limitations stem from the automated setup (e.g., fixed positioning preventing capture of distinguishing features despite high image resolution) or from low image quality. Recommendations are provided to address these challenges. Our results indicate that 89.81% of all Hymenoptera instances were identified to family level, 84.56% of pollinator family instances to genus level, and only 25.35% to species level. We were less able to identify Dipterans to finer taxonomic levels, with nearly 50% of instances not identifiable to family level, and only 26.18% and 15.19% identified to genus and species levels. This was due to their small size and the more challenging features needed for identification (e.g., in the wing veins). Advancing smartphone technology, along with their accessibility, affordability, and user-friendliness, offers a promising option for coarse-level pollinator monitoring.

References

Alison J, Alexander JM, Diaz Zeugin N, Dupont YL, Iseli E, Mann HMR, Høye TT (2022) Moths complement bumblebee pollination of red clover: a case for day-and-night insect surveillance. Biology Letters 18:20220187. DOI: https://doi.org/10.1098/rsbl.2022.0187

Amarathunga DC, Grundy J, Parry H, Dorin A (2021) Methods of insect image capture and classification: A Systematic literature review. Smart Agricultural Technology 1:100023. DOI: https://doi.org/10.1016/j.atech.2021.100023

Barlow SE, O’Neill MA (2020) Technological advances in field studies of pollinator ecology and the future of e-ecology. Current Opinion in Insect Science 38:15–25. DOI: https://doi.org/10.1016/j.cois.2020.01.008

Beery S, Van Horn G, Perona P (2018) Recognition in Terra Incognita. In: Proceedings of the European Conference on Computer Vision (ECCV).pp 456–473. [online] URL: https://openaccess.thecvf.com/content_ECCV_2018/html/Beery_Recognition_in_Terra_ECCV_2018_paper.html (accessed 6 December 2023).

Bhuiyan T, Carney RM, Chellappan S (2022) Artificial intelligence versus natural selection: Using computer vision techniques to classify bees and bee mimics. iScience 25:104924. DOI: https://doi.org/10.1016/j.isci.2022.104924

Bjerge K, Alison J, Dyrmann M, Frigaard CE, Mann HM, Høye TT (2023) Accurate detection and identification of insects from camera trap images with deep learning. PLOS Sustainability and Transformation 2:e0000051. DOI: https://doi.org/10.1371/journal.pstr.0000051

Bjerge K, Frigaard CE, Karstoft H (2023) Object Detection of Small Insects in Time-Lapse Camera Recordings. Sensors 23:7242. DOI: https://doi.org/10.3390/s23167242

Bjerge K, Geissmann Q, Alison J, Mann HMR, Høye TT, Dyrmann M, Karstoft H (2023) Hierarchical classification of insects with multitask learning and anomaly detection. Ecological Informatics 77:102278. DOI: https://doi.org/10.1016/j.ecoinf.2023.102278

Bjerge K, Mann HMR, Høye TT (2022) Real‐time insect tracking and monitoring with computer vision and deep learning. Remote Sensing in Ecology and Conservation 8:315–327. DOI: https://doi.org/10.1002/rse2.245

Carolan JC, Murray TE, Fitzpatrick Ú, Crossley J, Schmidt H, Cederberg B, McNally L, Paxton RJ, Williams PH, Brown MJF (2012) Colour Patterns Do Not Diagnose Species: Quantitative Evaluation of a DNA Barcoded Cryptic Bumblebee Complex. PLoS ONE 7:e29251. DOI: https://doi.org/10.1371/journal.pone.0029251

Clarin B ‐Markus, Bitzilekis E, Siemers BM, Goerlitz HR (2014) Personal messages reduce vandalism and theft of unattended scientific equipment. Methods in Ecology and Evolution 5:125–131. DOI: https://doi.org/10.1111/2041-210X.12132

Coral (2020) A development board to quickly prototype on-device ML products. Scale from prototype to production with a removable system-on-module (SoM). [online] URL: https://www.coral.ai/products/dev-board (accessed 6 December 2023).

Creedy TJ, Norman H, Tang CQ, Qing Chin K, Andujar C, Arribas P, O’Connor RS, Carvell C, Notton DG, Vogler AP (2020) A validated workflow for rapid taxonomic assignment and monitoring of a national fauna of bees (Apiformes) using high throughput DNA barcoding. Molecular Ecology Resources 20:40–53. DOI: https://doi.org/10.1111/1755-0998.13056

Donovan T, Balantic C, Katz J, Massar M, Knutson R, Duh K, Jones P, Epstein K, Lacasse-Roger J, Dias J (2021) Remote ecological monitoring with smartphones and Tasker. Journal of Fish and Wildlife Management 12:163–173. DOI: https://doi.org/10.3996/JFWM-20-071

Droissart V, Azandi L, Onguene ER, Savignac M, Smith TB, Deblauwe V (2021) PICT: A low‐cost, modular, open‐source camera trap system to study plant–insect interactions. Methods in Ecology and Evolution 12:1389–1396. DOI: https://doi.org/10.1111/2041-210X.13618

Dutta A, Zisserman A (2019) The VIA Annotation Software for Images, Audio and Video. In: Proceedings of the 27th ACM International Conference on Multimedia. ACM, Nice France, pp 2276–2279. DOI: https://doi.org/10.1145/3343031.3350535

Edwards J, Smith GP, McEntee MHF (2015) Long-term time-lapse video provides near complete records of floral visitation. Journal of Pollination Ecology 16:91–100. DOI: https://doi.org/10.26786/1920-7603(2015)16

Harman M (2023) Open Camera. [online] URL: https://sourceforge.net/projects/opencamera/ (accessed 4 December 2023).

Høye TT, Ärje J, Bjerge K, Hansen OLP, Iosifidis A, Leese F, Mann HMR, Meissner K, Melvad C, Raitoharju J (2021) Deep learning and computer vision will transform entomology. Proceedings of the National Academy of Sciences 118:e2002545117. DOI: https://doi.org/10.1073/pnas.2002545117

Kohlberg AB, Myers CR, Figueroa LL (2024) From buzzes to bytes: A systematic review of automated bioacoustics models used to detect, classify and monitor insects. Journal of Applied Ecology:1365-2664.14630. DOI: https://doi.org/10.1111/1365-2664.14630

Lahoz-Monfort JJ, Magrath MJL (2021) A Comprehensive Overview of Technologies for Species and Habitat Monitoring and Conservation. BioScience 71:1038–1062. DOI: https://doi.org/10.1093/biosci/biab073

Lortie CJ, Budden A, Reid A (2012) From birds to bees: applying video observation techniques to invertebrate pollinators. Journal of Pollination Ecology 6:125–128. DOI: https://doi.org/10.26786/1920-7603(2011)20

Mäder P, Boho D, Rzanny M, Seeland M, Wittich HC, Deggelmann A, Wäldchen J (2021) The Flora Incognita app – Interactive plant species identification. Methods in Ecology and Evolution 12:1335–1342. DOI: https://doi.org/10.1111/2041-210X.13611

Martineau M, Conte D, Raveaux R, Arnault I, Munier D, Venturini G (2017) A survey on image-based insect classification. Pattern Recognition 65:273–284. DOI: https://doi.org/10.1016/j.patcog.2016.12.020

Mcelveen D, Meyer R t (2020) An Effective and Affordable Camera Trap for Monitoring Flower-visiting Butterflies in Sandhills: with Implications for the Frosted Elfin (Callophrys irus). Journal of Pollination Ecology 26:12–15. DOI: https://doi.org/10.26786/1920-7603(2020)573

Meek PD, Ballard GA, Sparkes J, Robinson M, Nesbitt B, Fleming PJS (2019) Camera trap theft and vandalism: occurrence, cost, prevention and implications for wildlife research and management Rowcliffe M, Caravaggi A (eds). Remote Sensing in Ecology and Conservation 5:160–168. DOI: https://doi.org/10.1002/rse2.96

Mertens JEJ, Brisson L, Janeček Š, Klomberg Y, Maicher V, Sáfián S, Delabye S, Potocký P, Kobe IN, Pyrcz T, Tropek R (2021) Elevational and seasonal patterns of butterflies and hawkmoths in plant-pollinator networks in tropical rainforests of Mount Cameroon. Scientific Reports 11:9710. DOI: https://doi.org/10.1038/s41598-021-89012-x

Montero‐Castaño A, Koch JBU, Lindsay TT, Love B, Mola JM, Newman K, Sharkey JK (2022) Pursuing best practices for minimizing wild bee captures to support biological research. Conservation Science and Practice 4:e12734. DOI: https://doi.org/10.1111/csp2.12734

Motivans Švara E, Ştefan V, Sossai E, Feldmann R, Aguilon DJ, Bontsutsnaja A, E‐Vojtkó A, Kilian IC, Lang P, Mõtlep M, Prangel E, Viljur M, Knight TM, Neuenkamp L (2021) Effects of different types of low‐intensity management on plant‐pollinator interactions in Estonian grasslands. Ecology and Evolution 11:16909–16926. DOI: https://doi.org/10.1002/ece3.8325

Nagai M, Higuchi Y, Ishikawa Y, Guo W, Fukatsu T, Baba YG, Takada MB (2022) Periodically taken photographs reveal the effect of pollinator insects on seed set in lotus flowers. Scientific Reports 12:11051. DOI: https://doi.org/10.1038/s41598-022-15090-0

Nagano Y (2023) Changes in pollinators’ flower visits and activities potentially drive a diurnal turnover of plant‐pollinator interactions. Ecological Entomology 48:650–657. DOI: https://doi.org/10.1111/een.13262

Naqvi Q, Wolff PJ, Molano‐Flores B, Sperry JH (2022) Camera traps are an effective tool for monitoring insect–plant interactions. Ecology and Evolution 12:e8962. DOI: https://doi.org/10.1002/ece3.8962

NVIDIA (2019) Jetson Nano. [online] URL: https://developer.nvidia.com/embedded/jetson-nano (accessed 6 December 2023).

Ollerton J, Winfree R, Tarrant S (2011) How many flowering plants are pollinated by animals? Oikos 120:321–326. DOI: https://doi.org/10.1111/j.1600-0706.2010.18644.x

Pegoraro L, Hidalgo O, Leitch IJ, Pellicer J, Barlow SE (2020) Automated video monitoring of insect pollinators in the field. Emerging Topics in Life Sciences 4:87–97. DOI: https://doi.org/10.1042/ETLS20190074

Quicke DJL (1997) Parasitic wasps, 1st ed. Chapman & Hall, London ; New York.

Rakosy D, Ashman T, Zoller L, Stanley A, Knight TM (2023) Integration of historic collections can shed light on patterns of change in plant–pollinator interactions and pollination service. Functional Ecology 37:218–233. DOI: https://doi.org/10.1111/1365-2435.14211

Rakosy D, Motivans E, Ştefan V, Nowak A, Świerszcz S, Feldmann R, Kühn E, Geppert C, Venkataraman N, Sobieraj-Betlińska A, Grossmann A, Rojek W, Pochrząst K, Cielniak M, Gathof AK, Baumann K, Knight TM (2022) Intensive grazing alters the diversity, composition and structure of plant-pollinator interaction networks in Central European grasslands. PLOS ONE 17:e0263576. DOI: https://doi.org/10.1371/journal.pone.0263576

Ratnayake A, Yasin HM, Ghani Naim A, Abas PE (2023) Classification of Subspecies of Honey Bees using Convolutional Neural Network. In: 2023 6th International Conference on Applied Computational Intelligence in Information Systems (ACIIS). IEEE, Bandar Seri Begawan, Brunei Darussalam, pp 1–6. DOI: https://doi.org/10.1109/ACIIS59385.2023.10367282

Ratnayake MN, Amarathunga DC, Zaman A, Dyer AG, Dorin A (2023) Spatial monitoring and insect behavioural analysis using computer vision for precision pollination. International Journal of Computer Vision 131:591–606. DOI: https://doi.org/10.1007/s11263-022-01715-4

Ratnayake MN, Dyer AG, Dorin A (2021) Tracking individual honeybees among wildflower clusters with computer vision-facilitated pollinator monitoring. PLOS ONE 16:e0239504. DOI: https://doi.org/10.1371/journal.pone.0239504

Rodrigues BN, Boscolo D (2020) Do bipartite binary antagonistic and mutualistic networks have different responses to the taxonomic resolution of nodes? Ecological Entomology 45:709–717. DOI: https://doi.org/10.1111/een.12844

Ruczyński I, Hałat Z, Zegarek M, Borowik T, Dechmann DKN (2020) Camera transects as a method to monitor high temporal and spatial ephemerality of flying nocturnal insects. Methods in Ecology and Evolution 11:294–302. DOI: https://doi.org/10.1111/2041-210X.13339

Rydhmer K, Bick E, Still L, Strand A, Luciano R, Helmreich S, Beck BD, Grønne C, Malmros L, Poulsen K, Elbæk F, Brydegaard M, Lemmich J, Nikolajsen T (2022) Automating insect monitoring using unsupervised near-infrared sensors. Scientific Reports 12:2603. DOI: https://doi.org/10.1038/s41598-022-06439-6

Sittinger M, Uhler J, Pink M, Herz A (2024) Insect detect: An open-source DIY camera trap for automated insect monitoring. PLOS ONE 19:e0295474. DOI: https://doi.org/10.1371/journal.pone.0295474

Smith C, Rendón A, Barahona R, Moya W (2021) Consequences of the high abundance of Bombus terrestris on the pollination of Vicia faba. Journal of Pollination Ecology 29:258–272. DOI: https://doi.org/10.26786/1920-7603(2021)630

Spiesman BJ, Gratton C, Hatfield RG, Hsu WH, Jepsen S, McCornack B, Patel K, Wang G (2021) Assessing the potential for deep learning and computer vision to identify bumble bee species from images. Scientific reports 11:1–10. DOI: https://doi.org/10.1038/s41598-021-87210-1

Ssymank A, Doczkal D, Rennwald K, Dziock F (2011) Rote Liste und Gesamtartenliste der Schwebfliegen (Diptera: Syrphidae) Deutschlands. In: Rote Liste gefährdeter Tiere, Pflanzen und Pilze Deutschlands. Band 3: Wirbellose Tiere (Teil 1). Bundesamt für Naturschutz, Bonn [online] URL: https://www.rote-liste-zentrum.de/en/Schwebfliegen-Diptera-Syrphidae-1756.html (accessed 30 November 2023).

Stark T, Ştefan V, Wurm M, Spanier R, Taubenböck H, Knight TM (2023) YOLO object detection models can locate and classify broad groups of flower-visiting arthropods in images. Scientific Reports 13:16364. DOI: https://doi.org/10.1038/s41598-023-43482-3

Steen R (2017) Diel activity, frequency and visit duration of pollinators in focal plants: in situ automatic camera monitoring and data processing. Methods in Ecology and Evolution 8:203–213. DOI: https://doi.org/10.1111/2041-210X.12654

Steen R, Thorsdatter Orvedal Aase AL (2011) Portable digital video surveillance system for monitoring flower-visiting bumblebees. Journal of Pollination Ecology:90–94. DOI: https://doi.org/10.26786/1920-7603(2011)15

Ștefan V (2022) boxcel v2.0 - Integrate Excel with Python for visualizing images with their corresponding bounding boxes for object detection annotation workflows.

Van Horn G, Mac Aodha O, Song Y, Cui Y, Sun C, Shepard A, Adam H, Perona P, Belongie S (2018) The iNaturalist species classification and detection dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).pp 8769–8778. [online] URL: https://openaccess.thecvf.com/content_cvpr_2018/html/Van_Horn_The_INaturalist_Species_CVPR_2018_paper.html (accessed 6 December 2023). DOI: https://doi.org/10.1109/CVPR.2018.00914

van Klink R, August T, Bas Y, Bodesheim P, Bonn A, Fossøy F, Høye TT, Jongejans E, Menz MHM, Miraldo A, Roslin T, Roy HE, Ruczyński I, Schigel D, Schäffler L, Sheard JK, Svenningsen C, Tschan GF, Wäldchen J, Zizka VMA, Åström J, Bowler DE (2022) Emerging technologies revolutionise insect ecology and monitoring. Trends in Ecology & Evolution:S0169534722001343. DOI: https://doi.org/10.1016/j.tree.2022.06.001

Whitfield JB (1998) Phylogeny and evolution of host-parasitoid interactions in Hymenoptera. Annual Review of Entomology 43:129–151. DOI: https://doi.org/10.1146/annurev.ento.43.1.129

Wills BD, Landis DA (2018) The role of ants in north temperate grasslands: a review. Oecologia 186:323–338. DOI: https://doi.org/10.1007/s00442-017-4007-0