Interrater and intrarater agreement on heart murmurs

Abstract

Objective: To investigate interrater and intrarater agreement between physicians and medical students on heart sound classification from audio recordings, and factors predicting agreement with a reference classification.<p> <p>Design: Intra- and interrater agreement study. <p>Subjects: Seventeen GPs and eight cardiologists from Norway and the Netherlands, eight medical students from Norway. <p>Main outcome measures: Proportion of agreement and kappa coefficients for intrarater agreement and agreement with a reference classification. <p>Results: The proportion of intrarater agreement on the presence of any murmur was 83% on average, with a median kappa of 0.64 (range k ¼ 0.09–0.86) for all raters, and 0.65, 0.69, and 0.61 for GPs, cardiologist, and medical students, respectively. <p>Results: The proportion of agreement with the reference on any murmur was 81% on average, with a median kappa of 0.67 (range 0.29–0.90) for all raters, and 0.65, 0.69, and 0.51 for GPs, cardiologists, and medical students, respectively. Results: Distinct murmur, more than five years of clinical practice, and cardiology specialty were most strongly associated with the agreement, with ORs of 2.41 (95% CI 1.63–3.58), 2.19 (1.58–3.04), and 2.53 (1.46–4.41), respectively. <p>Conclusion: We observed fair but variable agreement with a reference on heart murmurs, and physician experience and specialty, as well as murmur intensity, were the factors most strongly associated with agreement.