Estimating the Reliability of Nontechnical Skills in Medical Teams

Publication date: Available online 16 January 2017
Source:Journal of Surgical Education
Author(s): Ipek Mete, Michael T. Brannick
ObjectiveTo assess the reliability of measurement of assessments of nontechnical skills in medical teams.BackgroundThere are several scales for assessing nontechnical skills in medical teams. In this critique, we describe the reliability of such assessments and discuss factors that may influence the obtained estimate of reliability for such scales.MethodA review of the literature yielded 28 reliability estimates, mostly intraclass correlations. We analyzed the methods of scale development and reliability estimation for factors that are likely to influence the resulting estimate, including aspects of the judges, the targets, what is observed, and the kind of reliability coefficient.ResultsTypical reliability was approximately 0.7, but estimates were quite varied in magnitude, from quite low (0.3) to quite high (0.9). Thus, highly reliable assessment is possible, but rarely achieved. Factors that are likely to influence the magnitude of reliability include the number of assessors, the participants' range in skill level, and the observational process. Most studies do not present enough information to make the estimates comparable from one study to the next.ConclusionsAlthough nontechnical skills can be measured reliably under some circumstances, it is not clear from the existing literature that one scale provides more reliable data than another. We recommend several practices that would help readers to make informed decisions about which scales are most useful for their purposes.



http://ift.tt/2iHbw3M