Get the Story Right
Interrater Reliability of an Oral Case Presentation Rating tool in a Pediatric Clerkship Lewin L et al. Teaching and Learning in Medicine 2013; 25(1):31-38.
Reviewed by Christy Peterson
What was the study question?
Is the inter-rater reliability of an oral case presentation rating tool - that includes elements of clinical reasoning - high enough to use for assessment of 3rd year medical students' oral case presentations?
How was the study done? A written oral case presentation rating tool was developed, piloted, and edited by faculty at the University of Maryland School of Medicine using recorded case presentations. Three faculty raters were trained to use the form and then rated prerecorded student oral case presentations. Responses were evaluated for correlation.
What were the results?
Fifteen students submitted recorded oral presentations. The interclass correlation coefficient was 0.9 when taken as a whole. This number indicates a good degree of correlation between reviewer responses. There were a few individual items with discordance. Lowest correlation values were found for provision of a general statement at the beginning of the physical exam; inclusion of only active issues in the past medical history, family and social histories and review of systems; and inclusion of an appropriate diagnostic/therapeutic plan.
What are the implications of these findings?
This validated tool is a welcome addition to the medical educator's armamentarium for use in evaluating students' oral presentations. It seems to me that any tool is helpful and using one that has some data behind it to support interrater concordance is even better.
Editor's note: The authors cite an earlier study (Haber and Lingard, 2001) describing the differences between students' and experienced physicians' understandings of the purpose of oral case presentations. From a student's perspective, the purpose of the oral presentation is to organize the large amount of data collected about the patient. Experienced clinicians view oral presentations as a flexible method of telling a story that leads the listener to a particular conclusion. Aside from being a validated and potentially useful tool to evaluate students' oral case presentations, the rating tool used in this study can help close the gap between those two perspectives (RR).