September 2023

You must be an angel--or is it a devil?

Seehusen DA, Kleinheksel AJ, Huang H, Harrison Z, Ledford CJW. The Power of One Word to Paint a Halo or a Horn: Demonstrating the Halo Effect in Learner Handover and Subsequent Evaluation. Acad Med. 2023 Aug 1;98(8):929-933. https://dx.doi.org/10.1097/ACM.0000000000005161

Reviewed by Srividya Naganathan

What was the study question?
Does learner (trainee) handover/labeling influence medical educators’ evaluations?

How was the study done?
A voluntary experiment was conducted at a national conference of family medicine educators, who were all involved in trainee education and evaluation. All the participants (n=70) viewed a video of a simulated patient encounter and evaluated 10 items on a paper form. The video participants were randomized to be labeled as ‘above-average’ (halo) or ‘below-average’ (horn) family medicine residents. The videos were also randomized to include either a male or female actor playing the resident, but otherwise the performances were scripted to be identical .  The videos were endorsed by the medical school faculty for authenticity of the patient encounter.

What were the results?
There was a significant difference noted on the evaluation scores between the 2 groups with participants rating higher scores for the ‘halo’ or’ above- average’ label (15.07-15.13) as compared to scores for the ‘horn’ or ‘below-average’ label (12.05-12.92). Furthermore, the medical educator's years of experience was negatively correlated with the observation score (p<.001). There was no difference noted in the scores with respect to learner gender.

How can I apply this to my work in education?
Although this was an experimental study, it raises concern regarding unconscious bias in the evaluators when presented with learner handover. Medical educators should be aware of the ‘halo’ and ‘horn’ effect when hearing general comments about trainees or handover from other educators.  This may influence the educator’s perceptions and influence evaluations through unconscious bias. Faculty development sessions and training may help to recognize bias and ways to mitigate them.

Editor’s Note: The halo and horn effect have been documented in psychology for more than a century.  What this study adds is their power to skew the evaluations of even experienced educators with no prior knowledge or assumptions about a learner.  One can argue about forward feeding in feedback and formative assessment, but it clearly should have no role in summative evaluation. (JG)

---

Grades? MCAT Scores? Participation in athletics??? 

Anderson KG, Lemos J, Pickell S, Stave C, Sgroi M. Athletes in medicine: A systematic review of performance of athletes in medicine. Med Educ. 2023;57(9):807-819. https://dx.doi.org/10.1111/medu.15033

Reviewed by Daniel Herchline

What was the study question?
Is participation in athletics a predictor of success in medical school and beyond?

How was the study done?
The authors performed a systematic review of the literature using PRISMA guidelines, aiming to identify studies that assessed the relationship between performance metrics of students, residents, and physicians with prior experiences in athletics. The inclusion criteria for studies included quantitative assessments of medical students, residents, or attending physicians, used prior participation in athletics as a predictor or explanatory variable, and were limited to studies in the US and Canada.

What were the results?
The authors identified 18 studies that fit the inclusion criteria, 14 (78%) of which evaluated medical students. Of the included studies, 16 (89%) found that athletes performed significantly better than non-athletes in realms including academic medical school performance, surgical skills performance, and rates of burnout. The authors do note that the overall results comparing the relationship between athletic experiences and academic performance in medical school are inconclusive as several studies had mixed findings surrounding standardized test scores, clerkship grades, and OSCE scores.

How can I apply this to my work in education?
With the paradigm shifts occurring in medical school recruitment and admissions, many schools are opting to use a more holistic approach when considering applicants. Admissions committees may consider involvement in athletics as a predictor of success in medical school given the parallels that exist between sport and medicine (teamwork, growth-oriented feedback, and time-management, among others). Although the findings of this study suggest that prior athletes may be less prone to burnout and have higher levels of grit and uncertainty tolerance, this reviewer suggests that caution be taken in trying to select individuals with these predispositions rather than mitigating the underlying drivers during training and beyond.

Editor’s Note: It is not surprising that the characteristics of a successful athlete are those that would also help a student or resident be successful: response to feedback, growth mindset, commitment, resilience, time management, team player. While history of high-level athletics may be a desirable attribute of potential medical school applicants, I think there needs to be caution in considering use of this as a metric for admission as there are significant time and financial costs associated with training/competing at a high level that may not be accessible to a diverse population. (KFo)

---

A New Hope? R2C2 to the rescue! 

Lockyer J, Lee-Krueger R, Armson H. et al. Application of the R2C2 Model to In-the-Moment Feedback and Coaching. Academic Medicine 2023;98(9):1062-1068. https://dx.doi.org/10.1097/ACM.0000000000005237

Reviewed by Kyra Len

What was the study question?
Is the R2C2 (relationship, reaction, content, coaching) model useful for in-the-moment feedback in the clinical environment?

How was it done?
This was a qualitative multi-institutional study that included 5 institutions in Canada and the United States.  Three learner-preceptor dyads were recruited from each site. Site leaders trained the dyads in the R2C2 process. The training was 1 hour and included education on experiential approaches, videos and role playing. After training, the learner-preceptor dyads precepted in the outpatient and inpatient clinical environments. During follow up interviews (1-2 days and 2-3 weeks later), feedback was collected about the perceptions during each of the R2C2 phases and suggestions for improvement. Transcripts were coded for themes.

What were the results?
Fifteen dyads consisted of preceptor-students and preceptor-residents. Coaching and feedback sessions ranged from 4-22 minutes (median = 12). While building relationships, exploring reactions and reflections, and confirming content were done consistently, coaching for change and co-creating an action plan were not consistently done.

In the relationship phase, the most important aspects from the learner's perspective were respect, interest in their learning, and trust.  Successful reactions and reflections occurred when the learner had an opportunity to reflect without lengthy explanations from the preceptor. Questions such as “How was that experience for you?” and “Did anything surprise you” helped drive the learner to reflect. Confirming content was not always done, but those who followed the R2C2 model transitioned the conversation to ask their learner for more specific content. As an example, one preceptor said “If you wanted to work on a couple of things between now and the next time we have a chat, what might they be?” Coaching and co-creating was often not executed properly if at all. Preceptors could not fully execute the coaching components including action plans and follow up. Those who completed this step seemed to have more experience with R2C2 or coaching. A coaching example: “What are some strategies you might want to incorporate to help hone your skills in?”

How can I apply this to my work in education?
With proper training and practice, the R2C2 model for in the moment feedback can be a useful tool for faculty to help medical students reflect and develop a concrete action plan for improvement.

Editor’s Note: This article provides a nice framework on how educators can approach their learner dyads and where educators have the most difficulty in applying the framework. What is left to be known though is the effectiveness of this framework for performance as a clinician. (AKP)