Beyond Rating Accuracy: Unpacking Frame-of-Reference Assessor Training Effectiveness

C. Allen Gorman; Duncan Jackson; John Meriac; Joseph Himmler

Beyond Rating Accuracy: Unpacking Frame-of-Reference Assessor Training Effectiveness

C. Allen Gorman^*, Duncan Jackson, John Meriac, Joseph Himmler

^*Corresponding author for this work

Human Resource Management

Research output: Contribution to journal › Article › peer-review

Abstract

Evidence from previous research suggests that frame-of-reference training is effective at improving assessor ratings in many organizational settings. Yet no research has presented a thorough examination of systematic sources of variance (assessor-related effects, evaluation settings, and measurement design features) that might influence training effectiveness. Using a factorial ANOVA and variance components analyses on a database of four studies of frame-of-reference assessor training, we found that (a) training is most effective at identifying low levels of performance and (b) the setting of the training makes little difference with respect to training effectiveness. We also show evidence of the importance of rater training as a key determinant of the quality of performance ratings in general. Implications for frame-of-reference training theory and practice are discussed.

Original language	English
Journal	Industrial and Organizational Psychology
Publication status	Accepted/In press - 7 Sept 2023

Cite this

@article{0a5350ff07124030945f89d6d306b804,

title = "Beyond Rating Accuracy: Unpacking Frame-of-Reference Assessor Training Effectiveness",

abstract = "Evidence from previous research suggests that frame-of-reference training is effective at improving assessor ratings in many organizational settings. Yet no research has presented a thorough examination of systematic sources of variance (assessor-related effects, evaluation settings, and measurement design features) that might influence training effectiveness. Using a factorial ANOVA and variance components analyses on a database of four studies of frame-of-reference assessor training, we found that (a) training is most effective at identifying low levels of performance and (b) the setting of the training makes little difference with respect to training effectiveness. We also show evidence of the importance of rater training as a key determinant of the quality of performance ratings in general. Implications for frame-of-reference training theory and practice are discussed.",

author = "Gorman, {C. Allen} and Duncan Jackson and John Meriac and Joseph Himmler",

year = "2023",

month = sep,

day = "7",

language = "English",

journal = "Industrial and Organizational Psychology",

issn = "1754-9426",

publisher = "Cambridge University Press",

}

TY - JOUR

T1 - Beyond Rating Accuracy: Unpacking Frame-of-Reference Assessor Training Effectiveness

AU - Gorman, C. Allen

AU - Jackson, Duncan

AU - Meriac, John

AU - Himmler, Joseph

PY - 2023/9/7

Y1 - 2023/9/7

N2 - Evidence from previous research suggests that frame-of-reference training is effective at improving assessor ratings in many organizational settings. Yet no research has presented a thorough examination of systematic sources of variance (assessor-related effects, evaluation settings, and measurement design features) that might influence training effectiveness. Using a factorial ANOVA and variance components analyses on a database of four studies of frame-of-reference assessor training, we found that (a) training is most effective at identifying low levels of performance and (b) the setting of the training makes little difference with respect to training effectiveness. We also show evidence of the importance of rater training as a key determinant of the quality of performance ratings in general. Implications for frame-of-reference training theory and practice are discussed.

AB - Evidence from previous research suggests that frame-of-reference training is effective at improving assessor ratings in many organizational settings. Yet no research has presented a thorough examination of systematic sources of variance (assessor-related effects, evaluation settings, and measurement design features) that might influence training effectiveness. Using a factorial ANOVA and variance components analyses on a database of four studies of frame-of-reference assessor training, we found that (a) training is most effective at identifying low levels of performance and (b) the setting of the training makes little difference with respect to training effectiveness. We also show evidence of the importance of rater training as a key determinant of the quality of performance ratings in general. Implications for frame-of-reference training theory and practice are discussed.

M3 - Article

SN - 1754-9426

JO - Industrial and Organizational Psychology

JF - Industrial and Organizational Psychology

ER -

Beyond Rating Accuracy: Unpacking Frame-of-Reference Assessor Training Effectiveness

Abstract

Fingerprint

Cite this