TY - JOUR
T1 - Immunoassays are not immune to errors
T2 - Examples from two studies of steroid output from freshwater trout farms
AU - Ellis, Tim
AU - Margiotta-Casaluci, Luigi
AU - Pottinger, Tom G.
AU - Morris, Steve
AU - Reese, R. Allan
AU - Sumpter, John P.
AU - Scott, Alexander P.
N1 - Funding Information:
This work was funded by the UK’s Department for Environment, Food and Rural Affairs ( Defra ) under project CB0427 Accuracy of methods of sex steroid determination. Thanks to: the farmers and land-owners for their cooperation and discussions; Dr Colin Waring and staff at the University of Portsmouth (School of Biological Sciences, Institute of Marine Sciences Laboratories) for practical assistance; Richard Williams (Centre for Ecology and Hydrology) and the National River Flow Archive http://www.ceh.ac.uk/data/nrfa/data/retrievals.html ) for supplying river flow data; John Elliott (Environment Agency, EA) for provision of maps of the R Test river system; the EA National Requests Team for supplying water quality data for EA sampling points at the field sites; and Ioanna Katsiadaki for critical review. The literature examples of errors in assay results were simply from previous incidental encounters. Any references to products or services do not imply or constitute endorsement or recommendation.
Funding Information:
This work was funded by the UK's Department for Environment, Food and Rural Affairs (Defra) under project CB0427 Accuracy of methods of sex steroid determination. Thanks to: the farmers and land-owners for their cooperation and discussions; Dr Colin Waring and staff at the University of Portsmouth (School of Biological Sciences, Institute of Marine Sciences Laboratories) for practical assistance; Richard Williams (Centre for Ecology and Hydrology) and the National River Flow Archive http://www.ceh.ac.uk/data/nrfa/data/retrievals.html) for supplying river flow data; John Elliott (Environment Agency, EA) for provision of maps of the R Test river system; the EA National Requests Team for supplying water quality data for EA sampling points at the field sites; and Ioanna Katsiadaki for critical review. The literature examples of errors in assay results were simply from previous incidental encounters. Any references to products or services do not imply or constitute endorsement or recommendation.
Publisher Copyright:
© 2019
PY - 2020/1/1
Y1 - 2020/1/1
N2 - A “reproducibility crisis” is widespread across scientific disciplines, where results and conclusions of studies are not supported by subsequent investigation. Here we provide a steroid immunoassay example where human errors generated unreproducible results and conclusions. Our study was triggered by a scientific report citing abnormally high concentrations (means of 4–79 ng L−1) of three natural sex steroids [11-ketotestosterone (11-KT), testosterone (T) and oestradiol (E2)] in water samples collected from two UK rivers over 4 years (2002–6). Furthermore, the data suggested that trout farms were a major source because reported steroid concentrations were 1.3–6 times higher downstream than upstream. We hypothesised that the reported levels were erroneous due to substances co-extracted from the water causing matrix effects (i.e. “false positives”) during measurement by enzyme-linked immunoassay (EIA). Thus, in collaboration with three other groups (including the one that had conducted the 2002–6 study), we carried out field sampling and assaying to examine this hypothesis. Water samples were collected in 2010 from the same sites and prepared for assay using an analogous method [C18 solid phase extraction (SPE) followed by extract clean-up with aminopropyl SPE]. Additional quality control (“spiked” and “blank”) samples were processed. Water extracts were assayed for steroids using radioimmunoassay (RIA) as well as EIA. Although there were statistically significant differences between EIA and RIA (and laboratories), there was no indication of matrix effects in the EIAs. Both the EIAs and RIAs (uncorrected for recovery) measured all three natural steroids at <0.6 ng L−1 in all river water samples, indicating that the trout farms were not a significant source of natural steroids. The differences between the two studies were considerable: E2 and T concentrations were ca. 100-fold lower and 11-KT ca. 1000-fold lower than those reported in the 2002–6 study. In the absence of evidence for any marked changes in husbandry practice (e.g. stock, diet) or environmental conditions (e.g. water flow rate) between the study periods, we concluded that calculation errors were probably made in the first (2002–6) study associated with confusion between extract and water sample concentrations. The second (2010) study also had several identified examples of calculation error (use of an incorrect standard curve; extrapolation below the minimum standard; confusion of assay dilutions during result work-up; failure to correct for loss during extraction) and an example of sample contamination. Similar and further errors have been noted in other studies. It must be recognised that assays do not provide absolute measurements and are prone to a variety of errors, so published steroid levels should be viewed with caution until independently confirmed.
AB - A “reproducibility crisis” is widespread across scientific disciplines, where results and conclusions of studies are not supported by subsequent investigation. Here we provide a steroid immunoassay example where human errors generated unreproducible results and conclusions. Our study was triggered by a scientific report citing abnormally high concentrations (means of 4–79 ng L−1) of three natural sex steroids [11-ketotestosterone (11-KT), testosterone (T) and oestradiol (E2)] in water samples collected from two UK rivers over 4 years (2002–6). Furthermore, the data suggested that trout farms were a major source because reported steroid concentrations were 1.3–6 times higher downstream than upstream. We hypothesised that the reported levels were erroneous due to substances co-extracted from the water causing matrix effects (i.e. “false positives”) during measurement by enzyme-linked immunoassay (EIA). Thus, in collaboration with three other groups (including the one that had conducted the 2002–6 study), we carried out field sampling and assaying to examine this hypothesis. Water samples were collected in 2010 from the same sites and prepared for assay using an analogous method [C18 solid phase extraction (SPE) followed by extract clean-up with aminopropyl SPE]. Additional quality control (“spiked” and “blank”) samples were processed. Water extracts were assayed for steroids using radioimmunoassay (RIA) as well as EIA. Although there were statistically significant differences between EIA and RIA (and laboratories), there was no indication of matrix effects in the EIAs. Both the EIAs and RIAs (uncorrected for recovery) measured all three natural steroids at <0.6 ng L−1 in all river water samples, indicating that the trout farms were not a significant source of natural steroids. The differences between the two studies were considerable: E2 and T concentrations were ca. 100-fold lower and 11-KT ca. 1000-fold lower than those reported in the 2002–6 study. In the absence of evidence for any marked changes in husbandry practice (e.g. stock, diet) or environmental conditions (e.g. water flow rate) between the study periods, we concluded that calculation errors were probably made in the first (2002–6) study associated with confusion between extract and water sample concentrations. The second (2010) study also had several identified examples of calculation error (use of an incorrect standard curve; extrapolation below the minimum standard; confusion of assay dilutions during result work-up; failure to correct for loss during extraction) and an example of sample contamination. Similar and further errors have been noted in other studies. It must be recognised that assays do not provide absolute measurements and are prone to a variety of errors, so published steroid levels should be viewed with caution until independently confirmed.
KW - Calculation error
KW - Enzyme-linked immunoassay
KW - Extraction
KW - Radioimmunoassay
KW - Reproducibility crisis
KW - Steroid
UR - http://www.scopus.com/inward/record.url?scp=85072276859&partnerID=8YFLogxK
U2 - 10.1016/j.ygcen.2019.113226
DO - 10.1016/j.ygcen.2019.113226
M3 - Article
C2 - 31374286
AN - SCOPUS:85072276859
SN - 0016-6480
VL - 285
JO - General and Comparative Endocrinology
JF - General and Comparative Endocrinology
M1 - 113226
ER -
