Research Article |
Corresponding author: Thomas Eltz ( thomas.eltz@rub.de ) Academic editor: Michael Ohl
© 2022 Fernanda Herrera-Mesías, Christopher Bause, Sophie Ogan, Hannah Burger, Manfred Ayasse, Alexander M. Weigand, Thomas Eltz.
This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Citation:
Herrera-Mesías F, Bause C, Ogan S, Burger H, Ayasse M, Weigand AM, Eltz T (2022) Double-blind validation of alternative wild bee identification techniques: DNA metabarcoding and in vivo determination in the field. Journal of Hymenoptera Research 93: 189-214. https://doi.org/10.3897/jhr.93.86723
|
Over the past few decades, several investigations around the globe have reported alarming declines in the abundance and diversity of bee species. The success of effective conservation strategies targeting these important pollinators relies heavily on accurate biodiversity assessments. The shortage of taxonomic experts and the escalation of the ongoing biodiversity crisis call for the development of alternative identification tools to implement efficient monitoring programs. The validation of such techniques is crucial to ensure that they provide results comparable to those of traditional morphotaxonomy. Here we performed two double-blind experiments to evaluate the accuracy of a pair of new techniques used for wild bee identification: DNA metabarcoding and in vivo identification in the field. The methods were tested on sets of wild bees from Germany and their results compared against evaluations done by panels of bee experts using traditional morphotaxonomy. On average the congruency of species identification between metabarcoding and morphotaxonomy was 88.98% across samples (N = 10), while in vivo identification and morphotaxonomy were 91.81% congruent (N = 7) for bees considered feasible for in vivo identification in the field. Traditional morphotaxonomy showed similar congruencies when compared to itself: 93.65% in the metabarcoding study and 92.96% in the in vivo study. Overall, these results support both new methods as viable alternatives to traditional microscopy-based assessment, with neither method being error-free. Metabarcoding provides a suitable option to analyze large numbers of specimens in the absence of highly trained taxonomic experts, while in vivo identification is recommended for repeated long-term monitoring, and when working in areas where the sampling of individuals could threaten local populations of endangered wild bee species. Further research is still needed to explore the potential of both techniques for conservation management and wildlife monitoring, as well as to overcome their current limitations as taxonomic tools.
Apiformes, conservation, molecular taxonomic tools, morphotaxonomy, non-lethal identification
Wild bees (Hymenoptera, Anthophila) are insect pollinators that are both ecologically important and of remarkable economic interest (
The recent decline of wild bees and other major insect groups in several regions of the world has become a matter of global concern among conservation biologists and the general public (
To preserve wild bee biodiversity, conservation initiatives adapted to the habitat requirements of local bee communities must be implemented (
Despite its importance, reliable taxonomic information is rather incomplete in several regions of the world. Even in Central Europe, the population trend of most wild bee species remains unknown (
It is a common procedure in wild bee monitoring to collect adult specimens in the field via active methods such as targeted sweep netting, or passive sampling using devices like pan traps or vane traps (
The accuracy of PIN relies strongly on the experience of the taxonomist because it can be extraordinarily complex, as diagnostic traits can vary substantially between regions, localities, or even within local populations. Traits, especially coloration and vestiture, can even vary for a given individual bee over the flight season (
DNA-based monitoring methods and molecular identification pipelines have great potential to assist PIN in wild bee inventorying (
Despite their advantages, metabarcoding approaches are not free of technical limitations and flaws. Several investigations have reported that it is generally not possible to retrieve taxon abundance data because final read numbers are heavily affected by species amplification efficiency (i.e. primer bias;
In the present study we test the accuracy of a customized metabarcoding pipeline (“DNA”) incorporating a voucher-saving work-flow targeting Central European wild bees (Herrera-Mesías et al. submitted).
Both PIN and DNA metabarcoding of bulk samples, are invasive techniques in the sense that they remove specimens from the population, thereby reducing local population size and potentially endangering local population survival. Only very few studies are dealing with effects of such lethal sampling methods on population development. Even though
Thus, for this study, double-blind experiments were performed to evaluate the accuracy of two alternative taxonomic identification techniques used on wild bees, DNA metabarcoding of bulk samples (“DNA”), and in vivo identification (“IVI”). We compared the output of both methods against the evaluation of a panel of wild bee experts to determine similarities and discrepancies between the new approaches and traditional morphotaxonomy based on dry-pinned specimens (“PIN”).
To evaluate the metabarcoding pipeline described in Herrera-Mesías et al. (submitted) a total of 230 wild bee specimens were used. The samples were collected by S.O. and a field assistant using hand nets during 10 sampling events from 27 April to 22 July in 2020 in 7 different sites distributed across the Federal State of Rhineland-Palatinate (Germany). The netted bees were killed with ethyl acetate and immediately stored under cool conditions. From the end of the field day until the pinning of the individuals, all samples were stored frozen to prevent possible degradation of the DNA. Bees were pinned (males with genitals pulled out) and labeled by the end of the field season. For DNA extraction one complete midleg of each individual was removed using fire-sterilized tweezers and transferred to 2 mL Eppendorf tubes. After processing the bees of a sampling event, all surfaces and tools, i.e., tweezers, were sterilized to exclude cross contamination. The legs were pooled per sampling event, the pooled samples labeled with integers 1 through 10 by S.O. and shipped to the Zoology Department of the Musée national d’histoire naturelle Luxembourg, where further molecular analysis (“DNA”) was conducted by F.H-M. and A.W. without specific knowledge of sites or specimens.
The pinned voucher specimens were shipped to two internationally recognized wild bee experts, both with over 15 years of experience in wild bee faunistics and taxonomy, who were asked to identify them to species level (“PIN”). Both experts consented the use of their identifications for the double-blind evaluation of the metabarcoding approach. During the laboratory analysis, the team processing the pooled leg samples had no access to the voucher specimens nor any of their metadata information or the evaluations done by the experts. The wild bee experts never met each other, and their taxon lists were handled by a third party (T.E.) until the DNA pipeline output was completed. The voucher specimens are deposited in the MNHNL invertebrate dry collection for long-term storage and curation (MNHNL127130-127359).
For the metabarcoding pipeline, a two-step PCR protocol using fusion primers based on
To increase the data robustness and the probability of detecting low biomass specimens, a PCR replicate strategy was followed. Two replicates of each sample plus one positive control (i.e. a mock community of known wild bee community composition) were included in the final setup. The success of both PCR replicates was verified by electrophoresis and their amplicons were purified with a NucleoSpin Gel and PCR Clean-up kit (Macherey-NagelTM). The DNA concentrations of the purified products were measured and equimolarly pooled into the final library (27.42 μl, 48.47 ng/μl). The clean library was sequenced on one lane of an Illumina MiSeq System (2x250 bp) at the Luxembourg Centre for Systems Biomedicine (Belval, Luxembourg).
The resulting DNA metabarcoding sequence data was processed using the JAMP R package (https://github.com/VascoElbrecht/JAMP), with the settings and supplementary tools described in Herrera-Mesías et al. (submitted). Taxonomic sorting was performed by comparing the resulting OTU fasta files against sequences stored in the Barcode of Life Data system (BOLD;
The resulting data were pruned using TaxonTableTools (
To maintain double-blindness between DNA and PIN, the curated table was sent to T.E. who cross-tabulated identification results for each sample for a first comparison. Only then were the results made available to the rest of the team for numerical analysis. To allow comparison among the output of both approaches, the curated taxon list was transformed into a presence/absence table and combined with the results of the morphological approach.
To test the accuracy of in vivo determination of wild bees in the field, one of the authors (C.B.) accompanied bee monitorers during wild bee surveys within the “BienABest” project. Surveys took place from April to September 2020 at nine different sites throughout Germany and were conducted by a total of seven trained bee monitorers, whose experience in bee faunistics and taxonomy varied from some to many years. The monitorers used a reduced-impact monitoring method that includes in vivo identification (IVI) of encountered wild bees along variable transect walks (
The 210 bees to be included in the laboratory evaluation of IVI were killed with ethyl acetate or by freezing, and pinned by C.B. Furthermore, genitalia of male specimens were extracted and fixed outside the metasoma if required for species identification. The pinned specimens were re-labeled with a unique number code to omit information about date, locality or any other detail that would violate the anonymity of the monitorers.
The pinned bees were first identified by one internationally recognized wild bee expert with many years of experience in bee faunistics, morphotaxonomy and systematics (EXP data set) who worked under the knowledge that the identifications would later be used for IVI evaluation. Consecutively the specimens were sent to four other recognized wild bee experts, with several to many years of experience in bee morphotaxonomy, for independent identification (PIN). These experts were paid at rates typical for freelance work and were also aware that their work was part of a scientific investigation. To reconcile all discrepancies of identifications between the EXP data set and PIN, these were consecutively discussed in detail with the respective PIN-experts. Based on these discussions, and taking into account COI barcodes of two critical bee individuals (see Suppl. material
For data analysis, the whole data set of wild bee IDs was divided into seven bee sets, each representing the identifications made by one individual monitorer, enabling us to analyze discrepancies between IVI and PIN across monitorers, and to contrast them with the discrepancy among PIN identifications for the same sets of bees. Additionally, a comparison to the consensus list showed the percentage of correctly identified bees per IVI and PIN expert.
To further analyze the congruency and discrepancy of identification within and among DNA and PIN, and within and among IVI and PIN, we calculated Bray-Curtis similarities based on presence/absence taxon tables (DNA evaluation) or quantitative taxon tables (IVI evaluation) using the PRIMER-E software (version 6.1.6;
After trimming and quality filtering, 2,874,629 high quality reads from the original 4,395,456 read pairs were retained (Short Read Archive bioproject: PRJNA876388). About 67.8% of the 1,447,238 original unassigned reads corresponded to PhiX. A total of 17.27% of the original 278 OTUs detected in the dataset were discarded after filtering based on a 0.01% read abundance threshold, remaining 230 OTUs for further analysis. 480 chimeras were discarded as well during clustering. After comparison against the BOLD systems database and replicate consistency analysis with Taxon Table Tools,146 OTU consistently found across replicates were preliminary identified as Hymenoptera taxa to various levels of taxonomic resolution (Suppl. material
The number of taxonomic units detected by DNA in individual samples varied between 11 and 22. All the species intentionally pooled in the mock community sample (positive control) were detected. From the ten samples considered in the analysis, only one (S2) presented a perfect congruence between the metabarcoding results (“DNA”) and the evaluations of both taxonomic wild bee experts (“PIN1” and “PIN6”), based on the values of the Bray-Curtis index and the visual analysis of the dendrogram (Fig.
Across samples, the average congruency between the DNA and PIN (“PINav”) was 88.98% (Table
Percentage congruency of taxon lists resulting from DNA and PIN across samples. DNA x PIN1, DNA x PIN6 and PIN1 x PIN6: Percentages are calculated based only on the wild bee taxa detected by the methods considered in each pairwise comparison. DNA x PINav: Average of pairwise congruency between DNA and both PIN experts. Mean congruency across samples and standard deviations (SD) are also given. N = number of bee individuals in each set.
Bee Set (=Sample) | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Comparison | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | Mean congruency (%) | SD |
N=25 | N=14 | N=21 | N=28 | N=30 | N=27 | N=17 | N=14 | N=27 | N=27 | |||
DNA x PIN1 | 93.33 | 100.00 | 66.67 | 90.48 | 95.24 | 93.33 | 100.00 | 90.91 | 100.00 | 90.00 | 92.00 | 9.75 |
DNA x PIN6 | 93.33 | 100.00 | 66.67 | 77.27 | 86.36 | 81.25 | 92.86 | 90.91 | 90.00 | 80.95 | 85.96 | 9.63 |
DNA x PINav | 93.33 | 100.00 | 66.67 | 83.87 | 90.80 | 87.29 | 96.43 | 90.91 | 95.00 | 85.48 | 88.98 | 9.30 |
PIN1 x PIN6 | 100.00 | 100.00 | 100.00 | 85.71 | 90.48 | 87.50 | 92.86 | 100.00 | 90.00 | 90.00 | 93.65 | 5.77 |
The total sample size of evaluated bees was reduced from originally 210 bees to 208 bees due to critical damage in two specimens caused by repeated shipping. The number of identified bees per monitorer/bee set varied from 19 to 46 bee individuals. Fig.
Dendrogram based on Bray-Curtis similarity index to illustrate congruency of wild bee identification among IVI and PIN experts. X-axis shows index values expressed as percentage Bray-Curtis similarity. PIN experts 1 to 4 are the same among the seven bee sets, whereas the IVI expert is different for each bee set.
Averaged across bee sets, there was a taxon list congruency between IVI and PIN of 91.81%. PIN results among themselves showed an average taxon list congruency of 92.96% (Table
Percentage congruency of taxon lists resulting from IVI and PIN in comparison to each other and a curated consensus list (CON) for each of the seven bee sets. IVI x PIN: Average of pairwise congruencies between one IVI expert and the four PIN experts. PIN x PIN: Average of pairwise congruencies between the four PIN experts. IVI x CON: Congruency between one IVI expert and the consensus list. PIN x CON: Average of congruencies between each of the four PIN experts and the consensus list. Grand means and standard deviations (SD) across bee sets are also given. N = number of bee individuals in each set.
Bee Set (=Sample) | |||||||||
---|---|---|---|---|---|---|---|---|---|
Comparison | S1 | S2 | S3 | S4 | S5 | S6 | S7 | Mean congruency (%) | SD |
N=21 | N=29 | N=39 | N=19 | N=24 | N=46 | N=30 | |||
IVI x PIN | 86.90 | 92.24 | 80.77 | 98.68 | 94.79 | 93.48 | 95.83 | 91.81 | 5.62 |
PIN x PIN | 85.71 | 91.38 | 90.60 | 97.37 | 97.92 | 93.84 | 93.89 | 92.96 | 3.90 |
IVI x CON | 95.24 | 96.55 | 84.62 | 100.00 | 91.67 | 97.83 | 100.00 | 95.13 | 5.06 |
PIN x CON | 90.48 | 93.97 | 94.87 | 98.68 | 96.88 | 95.65 | 95.83 | 95.19 | 2.38 |
The performed double-blind validations demonstrated that error rates of the evaluated novel methods were of a similar (low) order of magnitude as compared to traditional morphotaxonomy, suggesting they represent valid alternatives for wild bee monitoring. In addition, we found that neither of the methods, traditional pinning, in vivo identification or DNA metabarcoding, were error free. In the following we shed light on the types of errors that occurred and discuss strengths and weaknesses of the respective methods. To our knowledge, this is the first double-blind study to evaluate per-sample accuracy of wild bee identification within and across methods. Even if previous studies have compared the congruency of diverse identification techniques used in wild monitoring against traditional morphotaxonomic outcomes (
The overall congruency found between the metabarcoding pipeline (DNA) and morphological identification results (PIN) on a per-sample basis analysis (88.98% mean congruency) agrees well with previous findings reported by
Despite the high overall similarity of the results obtained by DNA and PIN in our study, 26 cases of disagreement were present (Suppl. material
Further research on cryptic diversity following a similar approach would contribute to reach final conclusions regarding the status of similarly challenging species complexes, such as the Halictus simplex-complex. Although our dataset pooled species within this complex into one entity for the overall comparisons, DNA was able to precisely identify H. langobardicus regardless of the sex of the individual, whereas PIN was only able to assign a species-level annotation to males (Suppl. material
Given that the genetic results of controversial species complexes involve an additional level of analysis (
Another factor potentially affecting the congruency of metabarcoding results with morphological analysis is environmental contamination. For example, in seven cases the pipeline detected additional wild bee species to the ones reported by the taxonomic experts. Five false positive detections were found in S3 (Bombus lapidarius, Bombus pascuorum, Andrena cineraria, Chelostoma florisomne and Dasypoda hirtipes), one in S5 (Halictus confusus) and one in S8 (Melecta luctuosa). The additional species in S3 correspond to easily identifiable wild bees and three of them were completely absent in the whole wild bee set, which means that they cannot have been overlooked by PIN. Most likely, DNA traces from an outside source are likely responsible for these additional findings. Carry-over DNA from other specimens in the field, the sampling containers, or from specimen handling before DNA extraction represents a more likely explanation than cross-contamination in the laboratory as no other bees were being processed within the laboratory premises at the time of the double-blind experiment. The same situation may explain the presence of H. confusus in S5 and of M. luctuosa in S8. Tag-switching as an alternative explanation for the false positive results of species generally present in the overall data set seems unlikely, as tag combinations with high Levenshtein distances (=>3) were chosen to avoid the artificial generation of existing tag combinations given the sequencing platform used (
False positives and false negatives are known drawbacks affecting taxonomic assessment results originating from PCR-based high throughput sequencing techniques, potentially leading to taxonomic biases such as “biodiversity inflation” (
Finally, three false negatives were also found in the metabarcoding dataset (Sphecodes gibbus in S1, Lasioglossum pauxillum in S3 and Melecta albifrons in S6). In this case, insufficient sequencing depth seems a more likely explanation than obscurity due to primer bias, as all missing species show low primer-template mismatch with the selected primer pair (Herrera-Mesías et al. submitted). In the experiment, the sequencing run produced fewer overall read numbers than the ones reported by similar works (13.8 million reads in
Despite the lack of a perfect match with the expert evaluations, the results of the DNA metabarcoding pipeline are similar enough to be advised as a viable alternative to microscopy-based assessment, especially when considering its high congruency to the PIN1 results. Moreover, this approach offers several advantages for broad-scale assessments in the context of conservation biology projects, when large quantities of wild bees may be challenging and costly to identify (
Finally, DNA metabarcoding presents a crucial limitation for wild bee monitoring purposes, as it should only be used for qualitative assessment. An alternative molecular, cost-effective but specimen-based solution allowing qualitative results can be offered by high-throughput or next-generation sequencing DNA barcoding (Creedy et al. 2019;
We found that IVI of bee individuals considered feasible for alive determination in the field by the monitorer led to similar rates of correct identification as PIN, i.e., 95% as judged post-hoc based on the curated consensus list (CON). This may seem surprising, because IVI took place in the field without a dissecting microscope. For a better understanding of the results, it is necessary to look more closely at the different error sources that led to incongruencies between the expert identifications.
First, biased expectations appeared to have caused misidentifications especially in IVI, where monitorers had knowledge of local bee communities from previous visits. This kind of mistake seems to have generated several cases of incorrect bumblebee identification. For example, in case of BBV86 and BBV98 (see Suppl. material
In contrast, PIN appears to have been more susceptible to mistakes like misplaced entries in excel sheets or mix-up of specimens. Such errors were suggested by unlikely misidentifications as in BBV42, a worker bumblebee Bombus lapidarius that had been identified as Halictus subauratus, a bee that could not be more different. In addition, some errors arose from biases in the used identification keys or reference collections. For example, the popular (and generally very good) identification key for bumblebees by
Due to the design of the study there might be a number of intrinsic biases that could have increased the accuracy of IVI relative to PIN. First, IVI experts had a free choice regarding which of the encountered bees they considered feasible for IVI (during evaluated monitorings approximately 90% of individuals were considered feasible for IVI, a rate that corresponds well with IVI rates during regular BienABest monitorings; BienABest project, unpublished results). Thus, they could directly influence the sample of bees/identifications that was being evaluated. In addition, IVI experts were very aware of being evaluated, and were constantly reminded of the fact by the presence of C.B. who collected their IVI bees. PIN experts, while also having been informed that their results will be used in a double-blind evaluation, did not work under close observation. This discrepancy in experienced scrutiny could have led to different likelihoods of careless mistakes.
The relationship between the amount of experience of the expert and the accuracy of identification results is less than clear. All experts included in this study (IVI and PIN, also for the DNA comparison) were recognized experts of bee morphotaxonomy with at least some years, but mostly many years, of experience. If there was a difference at all, the amount of experience was slightly higher and less variable among PIN than among IVI experts. The IVI monitorer considered least experienced did indeed deliver the least accurate identification result of only 84.6% in comparison to the consensus list. However, the respective bee set (S3) was also the one that had the lowest congruency among PIN experts (90.6%), suggesting that the set was difficult.
In general, IVI as conducted within the BienABest project yielded accurate identifications in nineteen out of twenty bees (95%). It needs to be emphasized that such accuracy can only be achieved by highly trained experts, a resource that is in short supply (
There is a controversial debate on whether the use of IVI is in fact necessary and desirable for wild bee monitoring (
IVI and most DNA metabarcoding approaches relying on bulk samples might have another disadvantage, as both strategies usually do not deposit extensive reference material. A reference collection for future comparison is often a legal requirement or at least important to judge about spatio-temporal patterns of individual species in times of changing taxonomies, e.g. within species complexes. Voucher specimens are also relevant in case upcoming taxonomic methods require biomaterial or morphometric data to address open taxonomic questions, or for educational purposes (Lister et al. 2011;
In the metabarcoding setup here applied, DNA was extracted from individual legs while the rest of the voucher specimens were archived in the invertebrate collection of the MNHNL. Although this led to an increase in the hands-on-times and costs per sample, it preserves specimens for future conservation studies (
Regarding IVI, additional documentation could be provided by depositing high-quality images taken from live bees confined in observation jars, as is currently done by some experts. However, this requires appropriate equipment and imposes substantial additional effort during field work. Also, there currently exists no general depository for digital specimens of wild bees.
To our best knowledge, this is the first study to compare the accuracy of alternative taxonomic tools against morphology-based identifications using a double-blind approach. Both DNA metabarcoding and in vivo determination in the field presented high overall congruency of their identification results with a traditional microscopy-based assessment performed by morphotaxonomic experts. These results validate the use of these alternative assessment techniques in conservation projects targeting wild bees of Central Europe. The metabarcoding pipeline is recommended for the qualitative analysis of large samples in the absence of taxonomic experts, and for resolving morphotaxonomic problems. However, strategies that boost data robustness are highly advised to control the effect of potential environmental contaminations, false positives, and false negatives. Moreover, metabarcoding data should not be used on its own to estimate quantitative population parameters due to biases in PCR amplification. On the other hand, in vivo identification can be used for quantitative assessment. It is advised for long-term monitoring, especially in fragile ecosystems with vulnerable bee populations. It is susceptible to misidentification due to preconceptions and potentially constrained by the experience and availability of monitorers. By concept, in vivo identification results in no or fewer deposited reference specimens so that the detection of rare and particularly noteworthy species may be difficult to validate. Generally, all techniques rely heavily on the availability of reference materials such as barcode sequences, voucher specimens, or reference images. Further efforts are needed to address this issue, thus filling the gap of information needed to refine the detection capacity of alternative identification techniques.
The authors would like to thank the anonymous monitorers and wild bee experts that contributed with their evaluations to the morphological identifications of the wild bee sets. We also thank Stéphanie Lippert, Amanda Luttringer and Balint Andrasi from the Zoology department at the Musée national d’histoire naturelle Luxembourg (MNHNL) for their collaboration with the laboratory work. We would also like to thank Rashi Halder from the Luxembourg Centre for Systems Biomedicine in Belval for her collaboration regarding the high-throughput sequencing. Financial support was received under the Bauer and Stemmler foundations programme “FORSCHUNGSGEIST! Next Generation Sequencing in der Ökosystemforschung”, from the Deutsche Bundesstiftung Umwelt (DBU), Ruhr-Universität Bochum, and from the BienABest project.
Supplementary file 1
Data type: Docx file.
Explanation note: Curations made to merge taxonomic lists resulting from DNA and PIN (metabarcoding pipeline part).
Supplementary file 2
Data type: Docx file.
Explanation note: Barcoding of two critical specimens for the consensus list (CON).
Supplementary file 3
Data type: Xlsx file.
Explanation note: Raw Hymenoptera metabarcoding data with number of sequence reads as well as information on taxon curation and individual tagging combinations.
Supplementary file 4
Data type: Docx file.
Explanation note: Identification results of DNA and PIN.
Supplementary file 5
Data type: Docx file.
Explanation note: Identification results of IVI and PIN experts per bee specimen compared to the CONsensus list.