In Summary… The development and preliminary validation of a rubric to assess medical students’ written summary statements in virtual patient cases. Smith S et al. Academic Medicine 2015
The development and preliminary validation of a rubric to assess medical students’ written summary statements in virtual patient cases. Smith S et al. Academic Medicine 2015
Reviewed by Elizabeth Lee
What was the study question?
Could a scoring rubric be developed and validated to assess medical students' written summary statements of virtual patient cases?
How was the study done?
Four physicians independently coded 50 summary statements of virtual patient cases, and created their own assessment rubric. Through an iterative process of comparing coding structures and scoring, the physicians created a final assessment rubric and applied it to 60 additional summary statements to establish initial content validity of the rubric.
What were the results?
The final rubric included five components: factual accuracy, appropriate narrowing of the differential diagnosis, transformation of information (ie. using the word ‘tachycardic’ if the patient has a HR of 180), use of semantic qualifiers, and global rating. Interrater reliability for the entire rubric was acceptable. Internal consistency of the rubric was acceptable. Content analysis of the physicians’ comments indicated differences in the application of the rubric when assigning the global rating. However, all four physicians agreed that the most important element of the rubric was the inclusion of information that accurately shaped the student’s understanding of the differential diagnosis.
What are the implications of these findings?
This is the first published approach for assessing medical students’ summary statements. The rubric holds promise as a tool for faculty to provide feedback to students in order to improve their clinical reasoning skills. The rubric could also be used as a self-assessment tool for students as they work toward mastery. Further work needs to be done to use the rubric as a scoring tool for higher-stakes assessment.
Editor’s note: The results of this well done study, by our own COMSEP members, provides guidance on teaching students the aspects that “count” on a summary statement. So often, it is hard for experienced teachers to break down a given task into its component parts - the results of this study give clinical teachers a framework for teaching, and assessing, summary statements (SLB).