COMSEP Journal Club
January 2021
Editors: Karen Forbes, Randy Rockney, and Jon Gold
Student life without honors grades
Seligman L, Abdullahi A, Teherani A, Hauer K. From Grading to Assessment for Learning: A Qualitative Study of Student Perceptions Surrounding Elimination of Core Clerkship Grades and Enhanced Formative Feedback. Teaching and Learning in Medicine. 24 Nov 2020. https://doi.org/10.1080/10401334.2020.1847654
Reviewed by Ben Grebber and Molly Rideout
What was the study question?
How does elimination of tiered grading in core clerkships with increased focus on formative feedback affect motivation, clerkship learning experiences, and wellbeing?
How was the study done?
Researchers interviewed 18 medical students during their core clerkships at the University of California, San Francisco following the transition to pass/fail grading in January of 2019. Traditional grading components were supplemented with twice weekly formative feedback from supervisors. Five of the student participants had been participating in core clerkships prior to the change and thus had experience with both grading structures, and 13 participants had experienced only the pass-fail structure. Using semi-structured interviews, participants were asked about motivation, wellbeing, learning behaviors, team dynamics, feedback, and student and supervisor attitudes regarding assessment changes. Interview responses were analyzed using themed coding of responses.
What were the results?
Researchers identified three major themes from their surveys: student engagement in clerkships, wellbeing, and recognition of the learning context. Regarding engagement, the researchers reported students felt more motivation to learn without pressure to receive a grade, with some participants reporting their priorities transitioned from exam study toward patient care. Students also described agency in their learning with more freedom to direct their learning to areas such as seeking feedback and practicing critical reflection. Moreover, researchers found a pass/fail curriculum facilitated collaborative relationships among all co-workers, including peers and supervisors. In terms of recognition of overall learning context, students felt they were graded more fairly in the pass/fail structure given the inherent variability of the clerkship. However, some students wondered how they would be differentiated from peers with the new grading system.
What are the implications of these findings?
The data from this pilot study suggests a transition to pass/fail grading with increased formative feedback in core clerkships improves student involvement in patient care and participation during the clerkship. The students described feelings of control, a greater engagement in learning, intrinsic motivation, and improved wellbeing. Based on these findings, a larger study including participants from multiple institutions could be developed. In addition, investigation of the association between pass/fail grading on performance during residency would be instructive.
Editor’s note: While small, the results of this qualitative study align with studies of similar changes in pre-clerkship curricula and other educational literature. One potential future pitfall—as USMLE Step 1 moves to pass/fail next year, student anxiety may be transferred to USMLE Step 2 CK, potentially interfering with students’ ability to make choices that support wellness. (JG)
Student Mental Health
An exploration of medical student attitudes towards disclosure of mental illness
Fletcher I, Castle M, Scarpa A, Myers O, Lawrence E.
Medical Education Online, 25:1, 1727713, https://dx.doi.org/10.1080/10872981.2020.1727713
Reviewed by Srividya Naganathan
What was the study question?
Would medical students disclose mental illness on residency and licensure applications?
How was the study done?
An anonymous electronic survey was administered to all students at University of New Mexico School of Medicine. The survey questions addressed: (1) Presence of prior mental illness; (2) Change in mental health during medical school; (3) Willingness to use mental health services; and (4) Disclosure on State board applications. The authors also asked about disclosure on residency applications, something not asked currently. Students unwilling to disclose mental health issues were asked to identify reasons for non-disclosure.
What were the results?
Of the 349 students surveyed, the response rate was 50%. 36% of respondents identified mental health problems prior to starting medical school. 47% felt their mental health worsened during medical school (p<0.001). 34% reported being likely to seek mental health services while 23% reported they would be unlikely to do so. Almost half of the students reported that they would not disclose mental health problems on State Board applications and, if asked, 63% reported that they would not disclose mental health problems on residency match applications. Fear of stigmatization was reported to be the reason for non-disclosure on licensure and residency applications in 49.7% and 65.7% respectively.
What are the implications of these findings?
This study demonstrates that fear of disclosing mental health problems by medical students is an-ongoing issue and problems tend to worsen during medical school. Fear of being stigmatized and concern about the residency match are perceived as barriers to seeking appropriate services. Unfortunately, this problem is not limited to students as rising rates of depression, burnout and mental health problems plague all physicians.
Editor’s note: When I edit clinical evaluators’ narrative comments about students, I routinely eliminate profanity and suggestions that the student seek attention for mental health issues. Obviously, an evaluator’s suggestion for the need for such attention embedded in a high stakes evaluation is not the best approach to identifying and addressing mental health concerns of students. This study, if validated, does indicate that something needs to be done because over a third of medical students came to medical school with mental health issues and the medical school experience can make the situation worse. Endeavors like student wellness initiatives and robust counseling and advising efforts should be encouraged. The biggest problem, as highlighted by this study, is the stigmatization of mental health concerns. All of medicine, and society in general, needs help on that front. (RR)
I can guess who wrote that!
Idiosyncrasy in Assessment Comments: Do Faculty Have Distinct Writing Styles When Completing In-Training Evaluation Reports?
Ginsburg S, Gingerich A, Kogan JR, Watling CJ, Eva KW. Acad Med. 2020 Nov;95(11):S81-S88. https://doi.org/10.1097/ACM.0000000000003643.
Reviewed By: Anna Chen and Angela Punnett
What was the study question?
Do faculty have distinct writing styles when completing in-training evaluation reports that affect how their comments are received?
How was the study done?
In-training evaluation report (ITER) comment data from 2012 to 2015 was compiled from four large North American Internal Medicine training programs. Each dataset was analyzed independently to reflect the distinct ITER forms used, comprising a total of 23 to 142 faculty providing 459 to 2666 assessments on 161 to 989 trainees. The Linguistic Index and Word Count (LIWC) was used to sort language along different dimensions, including word count, word length, sentence length, and word categories. Data were further analyzed using four summary variables (analytical thinking, clout, authenticity, and emotional tone) and two additional variables that considered presence of “red flag” words. Generalizability theory analyzed whether individual faculty could be differentiated. Associations between writing style and faculty or trainee characteristics were sought.
What were the results?
Faculty evaluators had distinct writing styles that were stable across different trainee assessments. The most stable distinguishing features of faculty writing styles included word count, sentence length, and use of long words. Faculty were also moderately distinguishable by use of analytical thinking words and clout words, which reflected formality/logic, and expertise/confidence, respectively. Faculty’s language use was not related to demographics, such as rank, gender, role, or division, nor was it related to resident demographics, such as gender or training year.
What are the implications?
Written evaluations are valued in both undergraduate and postgraduate medical trainee assessments. While narrative comments provide a rich source of evaluation detail, they are susceptible to idiosyncrasies in faculty writing styles, which may reflect the faculty member’s individuality more than the trainee’s abilities. These differences in writing style may lead to misinterpreted trainee evaluations, particularly when read by those not immersed in the culture of the organization and/or not familiar with the evaluator.
Editor’s Comments: One thing I found interesting was this study’s discussion about how writing style dimensions such as word count, longer words and writing with more authority (“clout”) impact the credibility of faculty comments as perceived by residents, program directors and competence committees. This study suggests those dimensions are linked more to the individual faculty than to creditability per se. I think we have a lot to learn in terms of understanding writing styles and the interpretation of narrative comments on learner assessments. (KFo)
A moment of gratitude.
As we look forward to 2021, we want to take a moment to express our appreciation for the efforts of all our contributors during the past year. It has been a challenging year for everyone, and our reviewers took the time to highlight some of the most important educational research of the year. We hope you learned as much from their reviews as we did.
Here are our reviewers this past year:
| Alcorta, Christa | |
| Beck Dallaghan, Gary | |
| Bhaloo, Zafira | |
| Blatman, Zachary | |
| Blatt, Carly | |
| Cappello, Elizabeth | |
| Chen, Anna | |
| Dietrick, Barbara | |
| Donner, Julia | |
| Friedman, Suzanne | |
| Goplerud, Dana | |
| Haight, Michele | |
| Herchline, Daniel | |
| Holland, Joanna | |
| Leschchiner, Alina | |
| Mcrae, Ashlyn | |
| Meller, Janet | |
| Meyers, Nicole | |
| Nabower, Aleisha | |
| Naganathan, Srividya | |
| Noak, Chris | |
| Pahwa, Amit | |
| Punnett, Angela | |
| Rideout, Molly | |
| Rudnick, Melanie | |
| Sanaz, Devlin | |
| Schulman, Jacob | |
| Siegel, Rebecca | |
| Van Opstal, Elizabeth | |
| Ye, Linlei | |
| Young, Kimberly |