Journal Club

In-training assessment; qualitative study of effects on supervision and feedback in an undergraduate clinical rotation. Daelmans, HEM, Overmeer, RM, Vander Hem-Stokroos, HH et al. Medical Education 2006; 40:51-58.

Reviewed by Margaret Golden, SUNY Downstate College of Medicine

"In-training assessment" (ITA) goes to the heart of the LCME mandate for "timely formative feedback." As used in this study, ITA is a term of art for a specific program of "systematic observation, feedback, and documentation of student's performance during clinical training," which these authors had shown to be a feasible and reliable method to document frequent assessments during the day-to-day work routine by many different assessors.

This study was done the year ITA was instituted for the medicine clerkship at a school in the Netherlands. The authors were interested in how the program translated from paper into action, and was based on semi-structured interviews with a sample of students (9/30), residents (9/13) and attendings (9/12). Standard procedures were used for transcribing, coding, and reviewing the interviews, which each lasted 30-45 minutes.

The findings were not surprising, but discouraging nonetheless. In spite of a carefully designed form specifying which competencies to assess in which kinds of encounters, very little feedback was reported by either the students or the assessors. Although the authors did not report about the actual documented assessments, they do report that borderline or failing assessments were almost never given. In the interviews, on the other hand, the assessors indicated that they had occasion to give borderline/fail assessments, but chose not to do so. One reason they forebore to give true but painful assessments is that they would not be the ones to follow up with the student, and they were not sure of how remediation would come about. Certainly faculty improvement might have improved outcomes.

Valid assessment of clinical skills remains the Holy Grail for clerkship directors. This study suggests that in addition to other parameters those assessing the learners need to have a long enough time with a learner to feel comfortable conveying negative assessments/feedback, and that both the learner and the evaluator need to know in advance how learner deficiencies will be remediated.

(Comment: Feedback remains central to improving medical student performance. This study confirms that despite our best efforts feedback is lacking and made to "taste good" and be "less filling." - Bill Raszka)