Hello COMSEP!
Today is National Velociraptor Day. Hope to never meet one….
If you’re feeling inspired after a great meeting in Cincinnati, one way to contribute is by writing a review for the COMSEP Journal Club Feel free to reach out to any of us if you’re interested—no prior experience necessary.
Enjoy,
Amit, Karen and Jon
More on the effects of delaying USMLE Step 1
Does Delaying the United States Medical Licensing Examination Step 1 to After Clerkships Affect Student Performance on Clerkship Subject Examinations?, Jurich D, Daniel M,. Hauer KE, Seibert C, Chandran L, Pock AR, Fazio SB, Fleming A, Santen SA. Teaching and Learning in Medicine, (2021) 33:4, 366-381. https://doi.org/10.1080/10401334.2020.1860063
Reviewed by Sanghamitra Misra
What was the study question?
Is student performance on Clinical Subject Exams (CSEs) affected by delaying Step 1 until completion of the core clerkships?
How was this study done? Retrospective analysis was conducted of CSE scores for students from 8 schools that moved Step 1 after core clerkships between 2012 and 2016. Quantification of the effect was the primary analysis. The secondary analyses were to (1) understand if clerkship order impacted CSE performance and (2) compare numbers of students scoring in the lowest percentiles (below the national 5th percentile) before and after the curricular change.
What were the results? In the 8 schools that delayed Step 1 until after the clerkships, collectively, the students had statistically significantly lower performance on four CSEs (Medicine, Neurology, Pediatrics, and Surgery) but not on Obstetrics/Gynecology or Psychiatry. Comparing performance 3 years pre- and post-Step 1 change, differences across all clerkships ranged from 0.3 to −2.0 points, with an average difference of −1.1. CSE performance in clerkships taken early in the sequence was more affected by the curricular change, and differences gradually disappeared with subsequent examinations. Medicine and Neurology showed the largest average differences. There was a slightly higher chance of scoring below the national fifth percentile in 4 of the clinical subject exams (Medicine, Neurology, Pediatrics, and Psychiatry) when Step 1 was delayed until after clerkships.
What are the implications? In this study of 8 schools, delaying Step 1 until after core clerkships had a very small impact on CSE scores overall, with decreased scores for exams early in the clerkship sequence and an increased number of students below the fifth percentile. However, such small score differences have minor effects on clerkship grades. The authors concluded that the overall size of the effect is unlikely to be educationally meaningful and that delaying Step 1 is non-inferior.
Overall, delaying Step 1 is reasonable, but schools should consider creating strategies to address CSE performance and Step 1 preparation during the clerkships.
Editor’s Note: This study may be reassuring for those who currently use CSE’s and are considering delaying Step 1 until after clerkships. Of course, the choice to use CSE’s and how much grade weight they carry is one that clerkships can make. Not included in this study is the effect of the recent change of USMLE Step 1 to pass/fail (see below). (JG)
Supporting students facing academic difficulties is difficult
Understanding the Impact of Academic Difficulties Among Medical Students: A Scoping Review
David Kirtchuk, Geoffrey Wells, Tom Levett, Clare Castledine, Richard de Visser. Medical Educ. 2022; 56: 262-269. https://dx.doi.org/10.1111/medu.14624
Reviewed by Melissa Held
What was the study question?
What is known about borderline/failing medical students’ experiences of academic struggle, learning style, teaching and remediation?
How was it done?
This scoping review of the literature explored students’ experience of academic difficulties in relation to academic factors, psychological factors, and social/contextual factors. Two authors screened articles (n=1084), removed duplicates, and assessed full text articles for eligibility. Eight articles were included in the final review. A data extraction form was created to include demographics, participant characteristics, student experiences, identified themes and quality control methods. Thematic analysis was conducted using Braun and Clarke’s six-step approach.
What were the results?
Thematic analysis found three main themes:
(1) Identity preservation: Failure forced students to reflect on their self-perception as being academically successful and they often felt peers saw them as a failure. Isolation from others was common and students didn’t want to be seen as “weak” by peers. Lower achieving students were unable to adapt study styles and believed that effort alone will be rewarded; this inability to self-reflect made remediation more difficult as this group often externalized factors responsible for their failures. (2) Medical schools’ dual roles: An interesting theme emerged of school mistrust. Schools were seen as both “judge and jury” and were positioned as a gatekeeper to success within medical school even though intended to be a support resource.
(3) Coping strategies and external pressures: Social isolation was the result of self-identity preservation and lack of access to support. Students often found it easier to approach peers than faculty. External pressures included financial pressures, relationships, mental health and other personal difficulties. High achievers often used these as motivators to focus their studies, whereas low achievers often attributed their failures to these external factors.
What are the implications?
It is important to understand the complex, varied and individual ways in which students respond to their own academic struggles. Most of these struggles cannot be predicted in the admissions process but occur during medical school training. For many students, it is their first time failing and it can have a profound impact on their self-identity and be a barrier to seeking help. Some students noted it was helpful to reflect on their own experiences; this may be something that can be offered to struggling students to challenge their current approach to learning and make shifts in studying or behaviors. Near peers may also help students in a different way that makes them feel more comfortable and make the process feel less punitive or stigmatizing.
Editor’s Comments: Despite medical school programs designed to support struggling learners, it seems a “one-size fits all” approach may not be the best approach, however individualized remediation is highly resource intensive. Further, how to shift the common perception of remediation as being “punitive” is a challenge that requires further exploration. (KFo)
Pass/Fail Grading is Less Stressful for Students…Isn’t it?
Exploring Students’ Perspectives on Well-Being and the Change of United States Medical Licensing Examination Step 1 to Pass/Fail. Nicole M. Mott, Jacquelyn B. Kercheval & Michelle Daniel (2021) Teaching and Learning in Medicine, 33:4, 355-365, https://dx.doi.org/10.1080/10401334.2021.1899929
Reviewed by Lauren K. Kahl
What was the study question?
What is the perspective of first- and second-year medical students on the NBME decision to change USMLE Step 1 to pass/fail, and how will this decision impact their well-being?
How was it done?
Semi-structured focus groups were conducted from May to June 2020 with first- and second-year medical students who had not yet taken Step 1. Purposive and theoretical sampling was used to select students from six medical schools in a variety of locations and reflecting varying degrees of prestige. The interviews aimed at understanding student perspectives in four main areas: 1) Student experience after the announcement to change Step 1 to pass/fail, 2) How this may affect their approach to residency applications, 3) How this impacts student well-being, and 4) Other implications of the grading change. An inductive and iterative constructivist approach was used to identify themes and analyze results. Differences in coding were discussed between authors to achieve consensus.
What were the results?
Thirty students, four-to-six from each of the six institutions, participated in the focus groups. Seventy- three percent were first-year students. Fifty percent were female, 47% were male, and 1 was non- binary. Fifty-seven percent were Caucasian, 27% were Asian, 10% were African American, and 7% were Hispanic or Latino/a. Students were concerned that the shift would lead to increased emphasis on non- academic or non-clinical aspects of medical school, such as research or extracurricular pursuits. While the change may alleviate short-term anxiety around studying for the exam, they feared it would lead to more longitudinal stress and exacerbate stress around taking Step 2. Students were conflicted on how the change would impact bias and disparities in medical school. A small number of students showed optimism that over time, the change to pass/fail may have a positive impact on student well-being and possibly lead to a more holistic evaluation of students.
What are the implications?
Any major change comes with questions, uncertainties, and unexpected consequences. Only time will tell what the true impact of the change will be. However, steps can be taken to improve medical student well-being, especially surrounding their assessment and residency applications. Potential examples include transparency about grading mechanisms and how residency programs evaluate applicants and steps to reduce disparities among medical schools and the students they enroll. A larger, more longitudinal study is needed to evaluate the impact on students of this grading change.
Editor’s note: While this study contains a small number of students, the study locations represented the different types of medical students and the authors used sampling techniques to get a representative sample. The study reaffirms what we know about the move to binary reporting of grades. There is a shift on the items used to differentiate students. The article did make me realize that schools with resources have the ability to provide those other items (e.g. research years, access to faculty doing world- renowned research). (AKP)