Potential Reporting Bias in Neuroimaging Studies of Sex Differences

Sean P. David; Florian Naudet; Jennifer Laude; Joaquim Radua; Paolo Fusar-Poli; Isabella Chu; Marcia L. Stefanick; John P. A. Ioannidis

doi:10.1038/s41598-018-23976-1

Potential Reporting Bias in Neuroimaging Studies of Sex Differences

Sean P. David, Florian Naudet, Jennifer Laude, Joaquim Radua, Paolo Fusar-Poli, Isabella Chu, Marcia L. Stefanick, John P. A. Ioannidis

Research output: Contribution to journal › Article › peer-review

65 Citations (Scopus)

193 Downloads (Pure)

Abstract

Numerous functional magnetic resonance imaging (fMRI) studies have reported sex differences. To empirically evaluate for evidence of excessive significance bias in this literature, we searched for published fMRI studies of human brain to evaluate sex differences, regardless of the topic investigated, in Medline and Scopus over 10 years. We analyzed the prevalence of conclusions in favor of sex differences and the correlation between study sample sizes and number of significant foci identified. In the absence of bias, larger studies (better powered) should identify a larger number of significant foci. Across 179 papers, median sample size was n = 32 (interquartile range 23-47.5). A median of 5 foci related to sex differences were reported (interquartile range, 2-9.5). Few articles (n = 2) had titles focused on no differences or on similarities (n = 3) between sexes. Overall, 158 papers (88%) reached “positive” conclusions in their abstract and presented some foci related to sex differences. There was no statistically significant relationship between sample size and the number of foci (−0.048% increase for every 10 participants, p = 0.63). The extremely high prevalence of “positive” results and the lack of the expected relationship between sample size and the number of discovered foci reflect probable reporting bias and excess significance bias in this literature.

Original language	English
Article number	6082
Journal	Scientific Reports
Volume	8
Issue number	1
Early online date	17 Apr 2018
DOIs	https://doi.org/10.1038/s41598-018-23976-1
Publication status	Published - 2018

Access to Document

10.1038/s41598-018-23976-1Licence: CC BY

Potential Reporting Bias in_DAVID_Publishedonline17Paril2018_GOLD VoR (CC BY)Final published version, 1.27 MBLicence: CC BY

Cite this

@article{e8547a9cabcf43f8a2557607950f7367,

title = "Potential Reporting Bias in Neuroimaging Studies of Sex Differences",

abstract = "Numerous functional magnetic resonance imaging (fMRI) studies have reported sex differences. To empirically evaluate for evidence of excessive significance bias in this literature, we searched for published fMRI studies of human brain to evaluate sex differences, regardless of the topic investigated, in Medline and Scopus over 10 years. We analyzed the prevalence of conclusions in favor of sex differences and the correlation between study sample sizes and number of significant foci identified. In the absence of bias, larger studies (better powered) should identify a larger number of significant foci. Across 179 papers, median sample size was n = 32 (interquartile range 23-47.5). A median of 5 foci related to sex differences were reported (interquartile range, 2-9.5). Few articles (n = 2) had titles focused on no differences or on similarities (n = 3) between sexes. Overall, 158 papers (88%) reached “positive” conclusions in their abstract and presented some foci related to sex differences. There was no statistically significant relationship between sample size and the number of foci (−0.048% increase for every 10 participants, p = 0.63). The extremely high prevalence of “positive” results and the lack of the expected relationship between sample size and the number of discovered foci reflect probable reporting bias and excess significance bias in this literature.",

author = "David, {Sean P.} and Florian Naudet and Jennifer Laude and Joaquim Radua and Paolo Fusar-Poli and Isabella Chu and Stefanick, {Marcia L.} and Ioannidis, {John P. A.}",

year = "2018",

doi = "10.1038/s41598-018-23976-1",

language = "English",

volume = "8",

journal = "Scientific Reports",

issn = "2045-2322",

publisher = "Nature Publishing Group",

number = "1",

}

TY - JOUR

T1 - Potential Reporting Bias in Neuroimaging Studies of Sex Differences

AU - David, Sean P.

AU - Naudet, Florian

AU - Laude, Jennifer

AU - Radua, Joaquim

AU - Fusar-Poli, Paolo

AU - Chu, Isabella

AU - Stefanick, Marcia L.

AU - Ioannidis, John P. A.

PY - 2018

Y1 - 2018

N2 - Numerous functional magnetic resonance imaging (fMRI) studies have reported sex differences. To empirically evaluate for evidence of excessive significance bias in this literature, we searched for published fMRI studies of human brain to evaluate sex differences, regardless of the topic investigated, in Medline and Scopus over 10 years. We analyzed the prevalence of conclusions in favor of sex differences and the correlation between study sample sizes and number of significant foci identified. In the absence of bias, larger studies (better powered) should identify a larger number of significant foci. Across 179 papers, median sample size was n = 32 (interquartile range 23-47.5). A median of 5 foci related to sex differences were reported (interquartile range, 2-9.5). Few articles (n = 2) had titles focused on no differences or on similarities (n = 3) between sexes. Overall, 158 papers (88%) reached “positive” conclusions in their abstract and presented some foci related to sex differences. There was no statistically significant relationship between sample size and the number of foci (−0.048% increase for every 10 participants, p = 0.63). The extremely high prevalence of “positive” results and the lack of the expected relationship between sample size and the number of discovered foci reflect probable reporting bias and excess significance bias in this literature.

AB - Numerous functional magnetic resonance imaging (fMRI) studies have reported sex differences. To empirically evaluate for evidence of excessive significance bias in this literature, we searched for published fMRI studies of human brain to evaluate sex differences, regardless of the topic investigated, in Medline and Scopus over 10 years. We analyzed the prevalence of conclusions in favor of sex differences and the correlation between study sample sizes and number of significant foci identified. In the absence of bias, larger studies (better powered) should identify a larger number of significant foci. Across 179 papers, median sample size was n = 32 (interquartile range 23-47.5). A median of 5 foci related to sex differences were reported (interquartile range, 2-9.5). Few articles (n = 2) had titles focused on no differences or on similarities (n = 3) between sexes. Overall, 158 papers (88%) reached “positive” conclusions in their abstract and presented some foci related to sex differences. There was no statistically significant relationship between sample size and the number of foci (−0.048% increase for every 10 participants, p = 0.63). The extremely high prevalence of “positive” results and the lack of the expected relationship between sample size and the number of discovered foci reflect probable reporting bias and excess significance bias in this literature.

U2 - 10.1038/s41598-018-23976-1

DO - 10.1038/s41598-018-23976-1

M3 - Article

SN - 2045-2322

VL - 8

JO - Scientific Reports

JF - Scientific Reports

IS - 1

M1 - 6082

ER -

Potential Reporting Bias in Neuroimaging Studies of Sex Differences

Abstract

Access to Document

Fingerprint

Cite this