Distinguishing differences in construct from differences in response style: gsem for item response theory models with anchoring vignettes

Andrew Pickles; Matt Bluett-Duncan; Helen Sharp; Silia Vitoratou

doi:10.1177/1536867X241297920

Distinguishing differences in construct from differences in response style: gsem for item response theory models with anchoring vignettes

Andrew Pickles, Matt Bluett-Duncan, Helen Sharp, Silia Vitoratou

Research output: Contribution to journal › Article › peer-review

Abstract

Item response theory models allow estimation of participant and group-mean trait scores from responses to a set of items, but estimates can be biased when participants vary in their response style. We illustrate models fit in gsem that can account for such response style differences by comparing self-report with their ratings of anchoring vignettes—descriptions of other individuals displaying different levels of the trait. Simulation results from standard item response theory, mean bias, random bias, and free-threshold models are illustrated. We show that unbiased estimates can be recovered when the vignettes rated depend on the participants’ own self-rating or are even rated by a different sample, substantially broadening their scope of application.

Original language	English
Pages (from-to)	666-686
Number of pages	21
Journal	STATA JOURNAL
Volume	24
Issue number	4
DOIs	https://doi.org/10.1177/1536867X241297920
Publication status	Published - Dec 2024

Keywords

anchoring vignettes
cross-culture calibration
gsem
IRT
item response theory
response bias
st0760

Access to Document

10.1177/1536867X241297920

Cite this

@article{e9fcc178d0ba4635a3c8d9f6d0302664,

title = "Distinguishing differences in construct from differences in response style: gsem for item response theory models with anchoring vignettes",

abstract = "Item response theory models allow estimation of participant and group-mean trait scores from responses to a set of items, but estimates can be biased when participants vary in their response style. We illustrate models fit in gsem that can account for such response style differences by comparing self-report with their ratings of anchoring vignettes—descriptions of other individuals displaying different levels of the trait. Simulation results from standard item response theory, mean bias, random bias, and free-threshold models are illustrated. We show that unbiased estimates can be recovered when the vignettes rated depend on the participants{\textquoteright} own self-rating or are even rated by a different sample, substantially broadening their scope of application.",

keywords = "anchoring vignettes, cross-culture calibration, gsem, IRT, item response theory, response bias, st0760",

author = "Andrew Pickles and Matt Bluett-Duncan and Helen Sharp and Silia Vitoratou",

note = "Publisher Copyright: {\textcopyright} 2024 StataCorp LLC.",

year = "2024",

month = dec,

doi = "10.1177/1536867X241297920",

language = "English",

volume = "24",

pages = "666--686",

journal = "STATA JOURNAL",

issn = "1536-867X",

publisher = "DPC Nederland",

number = "4",

}

TY - JOUR

T1 - Distinguishing differences in construct from differences in response style

T2 - gsem for item response theory models with anchoring vignettes

AU - Pickles, Andrew

AU - Bluett-Duncan, Matt

AU - Sharp, Helen

AU - Vitoratou, Silia

PY - 2024/12

Y1 - 2024/12

N2 - Item response theory models allow estimation of participant and group-mean trait scores from responses to a set of items, but estimates can be biased when participants vary in their response style. We illustrate models fit in gsem that can account for such response style differences by comparing self-report with their ratings of anchoring vignettes—descriptions of other individuals displaying different levels of the trait. Simulation results from standard item response theory, mean bias, random bias, and free-threshold models are illustrated. We show that unbiased estimates can be recovered when the vignettes rated depend on the participants’ own self-rating or are even rated by a different sample, substantially broadening their scope of application.

AB - Item response theory models allow estimation of participant and group-mean trait scores from responses to a set of items, but estimates can be biased when participants vary in their response style. We illustrate models fit in gsem that can account for such response style differences by comparing self-report with their ratings of anchoring vignettes—descriptions of other individuals displaying different levels of the trait. Simulation results from standard item response theory, mean bias, random bias, and free-threshold models are illustrated. We show that unbiased estimates can be recovered when the vignettes rated depend on the participants’ own self-rating or are even rated by a different sample, substantially broadening their scope of application.

KW - anchoring vignettes

KW - cross-culture calibration

KW - gsem

KW - IRT

KW - item response theory

KW - response bias

KW - st0760

UR - http://www.scopus.com/inward/record.url?scp=85213855012&partnerID=8YFLogxK

U2 - 10.1177/1536867X241297920

DO - 10.1177/1536867X241297920

M3 - Article

AN - SCOPUS:85213855012

SN - 1536-867X

VL - 24

SP - 666

EP - 686

JO - STATA JOURNAL

JF - STATA JOURNAL

IS - 4

ER -

Distinguishing differences in construct from differences in response style: gsem for item response theory models with anchoring vignettes

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this