Comparing what to what, on what scale? The impact of item comparisons and reference points in communicating risk and uncertainty

TY - JOUR

T1 - Comparing what to what, on what scale? The impact of item comparisons and reference points in communicating risk and uncertainty

AU - Rakow, Tim

AU - Blackshaw, Emily

AU - Pagel, Christina

AU - Spiegelhalter, David S

PY - 2018/10

Y1 - 2018/10

N2 - The mandated public availability of individual hospital's audit data for children's heart surgery in the UK creates a challenging scenario for communicating these complex and sensitive data to diverse audiences. On the basis of this scenario, we conducted three experiments with the aim of understanding how best to help lay people understand these data and the practical goal of improving the public presentation of these data. The experiments compared different outcome measures for displaying the survival rate (percentage scale versus the ratio of the predicted/observed rates) and presentation formats (individual hospital versus all hospitals shown) for outcomes data presented relative to prediction intervals generated by a risk model that adjusts for case mix. Our data highlight how easily inappropriate comparisons can influence evaluations of complex data: for instance, both a survival ratio of 1 and the presence of other hospitals seemingly provided reference points that resulted in inappropriately harsh evaluations of some hospitals. By drawing on evaluability theory, we demonstrate how to enhance people's understanding of these complex data while also discouraging inappropriate comparisons, which has implications for communicating risk and uncertainty and for choice architecture design in a range of contexts.

AB - The mandated public availability of individual hospital's audit data for children's heart surgery in the UK creates a challenging scenario for communicating these complex and sensitive data to diverse audiences. On the basis of this scenario, we conducted three experiments with the aim of understanding how best to help lay people understand these data and the practical goal of improving the public presentation of these data. The experiments compared different outcome measures for displaying the survival rate (percentage scale versus the ratio of the predicted/observed rates) and presentation formats (individual hospital versus all hospitals shown) for outcomes data presented relative to prediction intervals generated by a risk model that adjusts for case mix. Our data highlight how easily inappropriate comparisons can influence evaluations of complex data: for instance, both a survival ratio of 1 and the presence of other hospitals seemingly provided reference points that resulted in inappropriately harsh evaluations of some hospitals. By drawing on evaluability theory, we demonstrate how to enhance people's understanding of these complex data while also discouraging inappropriate comparisons, which has implications for communicating risk and uncertainty and for choice architecture design in a range of contexts.

KW - Risk communication

KW - Uncertainty

KW - Prediction intervals

KW - Reference-dependent evaluation

KW - Evaluability

U2 - 10.1002/bdm.2073

DO - 10.1002/bdm.2073

M3 - Article

SN - 0894-3257

VL - 31

SP - 547

EP - 561

JO - Journal of Behavioral Decision Making

JF - Journal of Behavioral Decision Making

IS - 4

ER -