Computer vision approaches, such as deep learning, potentially offer a range of benefits to entomology, particularly for the image-based identification of taxa. An experiment was conducted to gauge the ability of a convolution neural network (CNN) to identify genera of Braconidae from images of forewings. A deep learning CNN was trained via transfer learning from a small set of 488 images for 57 genera. Three-fold cross-validation achieved an accuracy of 96.7%, thus demonstrating that identification to genus using forewings is highly predictive. Further work is needed to increase both the coverage to species level and the number of images available.

Braconidae, computer vision, diagnositics, identification, model

Insect populations are challenging to study. One of the main problems is the identification of species, particularly in hyper diverse groups such as Hymenoptera, and because knowledge of biodiversity around the world is uneven (Amano and Sutherland 2013; Hoye et al. 2020). However, advances in computer vision approaches provide potential new solutions to this global challenge (Hoye et al. 2020; Greeff et al. 2022). Computer vision approaches, such as machine learning and deep learning, are currently influencing a wide range of scientific disciplines but are only relatively recently being applied to entomology (Boer and Vos 2018; Marques et al. 2018; Hansen et al. 2019).

Recent studies on image-based insect identification are showing that deep learning models can extract features from images and learn to differentiate species to an accuracy approaching, or exceeding, human expertise (Valan et al. 2019; Hoye et al. 2020). For example, over half of British ground beetles (Carabidae) can be identified to species, and 74% to genus using convolutional neural networks trained on an image set of over 19,000 images (Hansen et al. 2019). Boer and Vos (2018) used over 10,000 images from AntWeb (www.antweb.org) (to classify ants at species level based on dorsal, head, and profile images. Accuracy of identification was between 62–92% for species and 79–95% for genus, depending on different configurations of the models. Marques et al. (2018) also examined the classification of ants, and achieved an accuracy of 80–90%, demonstrating that high confidence and robustness in ant genera identification can be achieved.

Further to identification and diagnostics, the use of images is also being combined with additional automation and/or robotics to undertake sampling in the field, routine laboratory sample processing, or extracting data from images (Ärje et al. 2020). For example, Bjerge et al. (2021) have developed an automated light trap to monitor moths and identify the species using computer vision-based tracking and deep learning. An automated field trial over 48 nights captured more than 250,000 images, an average of 5675 images per night, with a high validation score for the identification of the 8 most common moth species. Machine learning methods have been used to automate the extraction of data on insect herbivore damage from plant specimens in museums, including the ability to identify different types of herbivores (Meineke and Davies 2018).

In this paper, we test the ability of a convolutional neural network to classify genera of Braconidae that are present in New Zealand using images of the forewing.

A total of 488 wings were used representing 57 genera. Results from cross-validation gave an overall accuracy of 96.7% (472/488; Table 1). Of the 16 misclassified images, 14 images had low confidence scores (<0.9), indicating the network struggled to classify them (Table 2), many of these are from the subfamily Microgastrinae.

This small experiment demonstrated that forewings appear to be highly predictive of genus level identifications. The model accuracy is particularly impressive given the very small number of images. Often hundreds or even thousands of images are needed to build these models. For example, Hansen et al. (2019) had a set of over 19,000 images for ground beetles (Carabidae), and Boer and Vos (2018) used over 10,000 images from AntWeb. We suggest our trial was successful because the forewing morphology (veins/cells) are already recognised as key diagnostic characters for Braconidae, and the images of a forewing are quite simple with considerably less ‘noise’ than dorsal and lateral habitus images of an insect body (Valan et al. 2019).

Two main questions need to be addressed in future work. Firstly, how well does only one species (or morphospecies) represent a genus. Several of the genera above are monotypic, and for some genera the forewing morphology will differ very little between species, but for genera with higher species diversity this condition is unlikely to hold. However, this was an initial trial of the technology, and as the number of species-level image sets increases then genus-level identification becomes less relevant. Secondly, how well will the model perform when additional species or genera are added. An increase in the number of ‘classes’ (taxa) will likely increase the morphological variability in the dataset, perhaps affecting model accuracy and consequently needing more source images to overcome (Greener et al. 2021). A related issue is the level of image standardisation required. The images used in this study were all photographed and processed with the same equipment setup; adding subsequent imagery (either for the same taxa or novel ones) has the potential to cause the model to focus on spurious photographic differences during training. Similarly, the model has only been tested on images held out from the same set; how well it performs on other image sets (for the same taxa) needs to be tested to determine how well the model ‘transfers’ to novel image sets and whether further refinement of the training process is required, such as more aggressive image pre-processing and augmentation or the inclusion of more images.

Machine learning tools, particularly convolutional neural networks (CNNs), are fast becoming a valuable tool for the identification of insects (Valan et al. 2019; Hoye et al. 2020). Identifications are a vital part of making insects visible and accessible (Greeff et al. 2022). An increase in the number and level of taxa being identified offers many benefits, including accelerating the discovery and increasing the awareness of a greater proportion of biodiversity, providing informed information for applications such as other academic research, conservation, and biosecurity, and may free time for more research tasks.

At present, the major hurdle is the shortage of images (Valan et al. 2019; Greeff et al. 2022), particularly many images of the same species, rather than just one representative photo for a publication. Digitization efforts are underway in many countries that involve taking images of specimens, and large image libraries are available such as those on iNaturalist and for specific taxa (e.g., Antweb, www.antweb.org), however, these will not always cover, or be suitable for, every taxonomic group. Although the above model has been built for use in New Zealand, from a distinct set of genera of which several are endemic, the images from this project could be used for Braconidae in other countries or regions, albeit with very careful interpretation of the results. Consequently, it is vital that researchers facilitate sharing and exchange of their images (Valan et al. 2019), and that collaborative and user-friendly software be developed (Greeff et al. 2022).

Funded by the Ministry of Business, Innovation and Employment (MBIE) through the Strategic Science Investment Fund (SSIF) for Nationally Significant Collections and Databases (NSCDs) at Manaaki Whenua-Landcare Research (MWLR) via the Biota Portfolio (BIO) within the Research Priority Area (RPA) for Collections and Databases. Thanks to S. Malysheva for slide mounting the wings. Many thanks to the taxonomic experts who have identified specimens and worked on the New Zealand Braconid fauna over the years, especially to Sergey Belokobylkij, Donald Quicke, and Jose Fernandez-Triana.

Funded by the Ministry of Business, Innovation and Employment (MBIE) through the Strategic Science Investment Funding (SSIF) for Nationally Significant Collections and Databases.