Council on Medical Student Education in Pediatrics

COMSEP Logo

Search This Site

COMSEP 2010 Albuquerque Meeting

INTER-RATER RELIABILITY OF AN ORAL CASE PRESENTATION RATING TOOL

Authors:
Linda O. Lewin, MD, Department of Pediatrics, University of Maryland, Baltimore, MD

Background:  Effective oral communication is a core skill of physicians.  Medical students must learn not only how to collect data about their patients and synthesize that data into an appropriate assessment and plan; they must also learn how to communicate their thoughts clearly to others involved in the care of the patient.  There are few validated tools for assessing medical students’ ability to give effective oral case presentations, and those that are described focus on the mechanical aspects of placing all the data into the right sections, without attention to the clinical reasoning processes that are needed to craft a presentation that gives the listener the correct subset of data to support the proposed differential diagnosis.  The Oral Presentation Tool (OPT) was created to allow educators to rate medical students’ patient presentations and encompasses the content, organization, and clinical reasoning involved in creating those presentations. Objectives: The objective of this study was to measure the inter-rater reliability of the OPT. Methods:  Third year medical students on their pediatrics clerkship between April and June, 2009 were asked to participate.  Each student recorded one full new inpatient oral presentation.  Presentations were rated by three trained raters using the OPT.  The ratings were reviewed by the author to generate a list of analyses to be performed.  Statistical analysis of ratings using Krippendorff’s Alpha are pending. Results:  15 students agreed to participate. Preliminary review showed that there was moderate agreement among raters with regard to sub-sections of the presentation and overall performance, but significant variance on several individual items.  The current statistical analysis will quantify how well the raters agreed on individual items, in sub-sections, and on overall presentation skills of the students. Discussion:  Preliminary results suggest that some revision of the tool is needed.  The quantitative results will pinpoint the areas that need attention, and ultimately lead to a tool that will reliably rate student oral case presentations.