Kernan WN, Holmboe E, O'Connor PG. Assessing the Teaching Behaviors of Ambulatory Care Preceptors. Acad Med. 2004; 79:1088 - 1094 Reviewed by John S. Venglarcik, III
Kernan WN, Holmboe E, O'Connor PG. Assessing the Teaching Behaviors of Ambulatory Care Preceptors. Acad Med. 2004; 79:1088 - 1094
Reviewed by John S. Venglarcik, III
Kernan and colleagues have done all of us a favor by providing a mechanism to assess the teaching behavior of ambulatory preceptors in a reliable fashion. In recent years there has been a growing concern regarding the clinical skills instruction our students receive in medical school. The authors developed an instrument for measuring teaching behavior in an ambulatory setting then used the instrument to assess teaching behavior in their institution. The first step was to develop an instrument that they called the Teaching Encounter Card (TEC). The TEC included eight teaching behaviors: allowing student to complete the history and physical, hear full results of student evaluation, observe student performing part of physical exam, hear student assessment before giving one's own, ask questions beyond assessment and plan, hold discussion away from patient, observe student interviewing patient, allow student to do visit closure and provide feedback to student. The TEC was validated with third year students on the ambulatory portion of the internal medicine clerkship at Yale. Both the faculty and students received instructions on use of the card. Standardized patients and faculty scripts were used to complete the validation process.
In a field application, the sensitivity of the TEC was at least 80% for six of the behaviors (range 64% -100%) and specificity was at least 80% for all nine behaviors and 100% for five. For the full trial 20 students completed the TEC for 17 preceptors and 270 teaching encounters. The preceptors used six of the behaviors in over 70% of the encounters. The three exceptions were: observing student interview patient (68%), providing feedback (52%) and observing student perform part of the physical examination (24%). There was greater variability for behaviors that were used less often by the preceptors (e.g. observing students perform part of the physical examination).
Accuracy was particularly high for five behaviors: allowing student to complete the history and physical, observing student performing part of physical exam, hearing student assessment before giving one's own, holding discussion away from patient and provide feedback to student. The authors expressed concern that students failed to recognize the other behaviors in the preceptors. The authors concluded that the TEC provided valid information about the behavior of physicians who teach ambulatory care internal medicine. The TEC was well accepted by students and faculty. The ultimate utility of the card could be confirmed by its effect on faculty teaching behavior.
Comment: The authors undertook a difficult task in attempting to quantify ambulatory teaching behavior. What I liked the most, and what makes this article interesting, is that the authors validated the tool (TEC) before they used it to assess teaching behavior, making this a powerful study. The tool was simple, uncomplicated and easy to use (another study, on a smaller scale, used a PDA). Although I thought the procedure was correct, the relatively small number of observations (11 students engaged in only two different teaching encounters) may account for the wide variability of some values. However, this is a minor criticism as the sensitivity and specificity were both high. The authors also pointed out that the students themselves may not recognize some of the teaching behaviors and suggested that a way to improve sensitivity would be to train students to recognize certain teaching behaviors.
This is a good effort to quantify certain teaching behaviors and, hopefully, provide physicians with opportunities for faculty development. I am not certain about the utility of the entire TEC as designed for pediatrics. Some of the teaching behaviors might need some modification if applied to pediatrics or even be substituted by different behaviors. Any modification would require validation but perhaps we should consider making a similar attempt in pediatric ambulatory care.
(As John mentions, this study is well done, including validation of their tool. It is also an important topic - how to provide more specific feedback to preceptors. Given the large number of preceptors we interact with and the generosity of our volunteer faculty this relatively simple method seems to be ripe for application. Would something like this help you give better feedback to your preceptors? As John questions, do you think we would need to change any of the behaviors for pediatrics? Would you preceptors welcome this? Karen Marcdante)