Decisive Evidence on a Smaller-Than-YouThink Phenomenon: Revisiting the "1-in-X'' Effect on Subjective Medical Probabilities

TY - JOUR

T1 - Decisive Evidence on a Smaller-Than-YouThink Phenomenon

T2 - Revisiting the "1-in-X'' Effect on Subjective Medical Probabilities

AU - Sirota, Miroslav

AU - Juanchich, Marie

AU - Kostopoulou, Olga

AU - Hanak, Robert

PY - 2014/5

Y1 - 2014/5

N2 - Accurate perception of medical probabilities communicated to patients is a cornerstone of informed decision making. People, however, are prone to biases in probability perception. Recently, Pighin and others extended the list of such biases with evidence that 1-in-X ratios (e.g., 1 in 12) led to greater perceived probability and worry about health outcomes than N-in-X*N ratios (e.g., 10 in 120). Subsequently, the recommendation was to avoid using 1-in-X ratios when communicating probabilistic information to patients. To warrant such a recommendation, we conducted 5 well-powered replications and synthesized the available data. We found that 3 out of the 5 replications yielded statistically nonsignificant findings. In addition, our results showed that the 1-in-X effect was not moderated by numeracy, cognitive reflection, age, or gender. To quantify the evidence for the effect, we conducted a Bayes factor meta-analysis and a traditional meta-analysis of our 5 studies and those of Pighin and others (11 comparisons, N = 1131). The meta-analytical Bayes factor, which allowed assessment of the evidence for the null hypothesis, was very low, providing decisive evidence to support the existence of the 1-in-X effect. The traditional meta-analysis showed that the overall effect was significant (Hedges' g = 0.42, 95% CI 0.29-0.54). Overall, we provide decisive evidence for the existence of the 1-in-X effect but suggest that it is smaller than previously estimated. Theoretical and practical implications are discussed.

AB - Accurate perception of medical probabilities communicated to patients is a cornerstone of informed decision making. People, however, are prone to biases in probability perception. Recently, Pighin and others extended the list of such biases with evidence that 1-in-X ratios (e.g., 1 in 12) led to greater perceived probability and worry about health outcomes than N-in-X*N ratios (e.g., 10 in 120). Subsequently, the recommendation was to avoid using 1-in-X ratios when communicating probabilistic information to patients. To warrant such a recommendation, we conducted 5 well-powered replications and synthesized the available data. We found that 3 out of the 5 replications yielded statistically nonsignificant findings. In addition, our results showed that the 1-in-X effect was not moderated by numeracy, cognitive reflection, age, or gender. To quantify the evidence for the effect, we conducted a Bayes factor meta-analysis and a traditional meta-analysis of our 5 studies and those of Pighin and others (11 comparisons, N = 1131). The meta-analytical Bayes factor, which allowed assessment of the evidence for the null hypothesis, was very low, providing decisive evidence to support the existence of the 1-in-X effect. The traditional meta-analysis showed that the overall effect was significant (Hedges' g = 0.42, 95% CI 0.29-0.54). Overall, we provide decisive evidence for the existence of the 1-in-X effect but suggest that it is smaller than previously estimated. Theoretical and practical implications are discussed.

KW - "1-in-X" effect

KW - subjective probability

KW - probability perception

KW - meta-analysis

KW - Bayes factor meta-analysis

KW - COGNITIVE REFLECTION

KW - RISK COMMUNICATION

KW - RATIO BIAS

KW - EXPERIMENTAL-PSYCHOLOGY

KW - BAYES FACTOR

KW - T TESTS

KW - NUMERACY

KW - JUDGMENTS

KW - COMPREHENSION

KW - DENOMINATOR

U2 - 10.1177/0272989X13514776

DO - 10.1177/0272989X13514776

M3 - Article

SN - 0272-989X

VL - 34

SP - 419

EP - 429

JO - Medical Decision Making

JF - Medical Decision Making

IS - 4

ER -