TY - JOUR
T1 - The internal structure of situational judgement tests reflects candidate main effects
T2 - Not dimensions or situations
AU - Jackson, Duncan J. R.
AU - LoPilato, Alexander C.
AU - Hughes, Dan
AU - Guenole, Nigel
AU - Shalfrooshan, Ali
PY - 2017/3/1
Y1 - 2017/3/1
N2 - Despite their popularity and capacity to predict performance, there is no clear consensus on the internal measurement characteristics of situational judgement tests (SJTs). Contemporary propositions in the literature focus on treating SJTs as methods, as measures of dimensions, or as measures of situational responses. However, empirical evidence relating to the internal structure of SJT scores is lacking. Using generalizability theory, we decomposed multiple sources of variance for three different SJTs used with different samples of job candidates (N1 = 2,320; N2 = 989; N3 = 7,934). Results consistently indicated that (1) the vast majority of reliable observed score variance reflected SJT‐specific candidate main effects, analogous to a general judgement factor, and that (2) the contribution of dimensions and situations to reliable SJT variance was, in relative terms, negligible. These findings do not align neatly with any of the proposals in the contemporary literature; however, they do suggest an internal structure for SJTs.
AB - Despite their popularity and capacity to predict performance, there is no clear consensus on the internal measurement characteristics of situational judgement tests (SJTs). Contemporary propositions in the literature focus on treating SJTs as methods, as measures of dimensions, or as measures of situational responses. However, empirical evidence relating to the internal structure of SJT scores is lacking. Using generalizability theory, we decomposed multiple sources of variance for three different SJTs used with different samples of job candidates (N1 = 2,320; N2 = 989; N3 = 7,934). Results consistently indicated that (1) the vast majority of reliable observed score variance reflected SJT‐specific candidate main effects, analogous to a general judgement factor, and that (2) the contribution of dimensions and situations to reliable SJT variance was, in relative terms, negligible. These findings do not align neatly with any of the proposals in the contemporary literature; however, they do suggest an internal structure for SJTs.
U2 - 10.1111/joop.12151
DO - 10.1111/joop.12151
M3 - Article
SN - 0963-1798
VL - 90
SP - 1
EP - 27
JO - JOURNAL OF OCCUPATIONAL AND ORGANIZATIONAL PSYCHOLOGY
JF - JOURNAL OF OCCUPATIONAL AND ORGANIZATIONAL PSYCHOLOGY
IS - 1
ER -