Clerkships and Shelf Exams–Are They Even Related?
Kraakevik J, Haedinger L, Cirila EVG, et al. Impact of Students’ Scheduling Choice on Clerkship Examination Score Performance in Time-Varying Competency-Based Curriculum
Academic Medicine. August 30, 2022. Published online ahead of print.
https://dx.doi.org/10.1097/ACM.0000000000004952Reviewed by Melissa HeldWhat was the study question?
What is the impact of student-selected examination timing on NBME shelf examination test performance for required core clerkships compared to fixed exam timing?
How was it done?
Students in the class of 2017 took their clerkship examinations on the final day of each clerkship rotation (set time). Students in the classes of 2018, 2019, 2020 were allowed to take the clerkship examinations when they wanted during available schedule periods (flexible timing). Clerkship timing and exam data were noted along with exam scores. Days between completion of the clerkship and when the exam was taken was calculated and stratified: (a) before the clerkship; (b) 0-14 days after clerkship; (c) 15-30 days after clerkship; (d) 31-90 days after the clerkship; or (e) >90 days after clerkship. Descriptive statistics were calculated, and comparisons made between fixed and flexible groups.
What were the results?
There were 146 students with fixed exam timing (class of 2017) and 466 students with flexible exam timing (classes of 2018-2020). No statistical differences were noted across gender, age, or other demographic cohorts. Exam scores were statistically higher for those who took their exam at a flexible (and self-chosen) time compared to those who took the exam on a fixed date (fixed exam timing mean=73.9, SD = 7.8; flexible timing mean = 77.4, SD =6.0, P< .001). Exam pass rate relative to when the clerkship was completed was not statistically significant for any clerkship. For all clerkships, some students took the exam before its completion (2.7% for internal medicine to 14.6% for psychiatry) and between 22.7% (psychiatry) and 40% (surgery) took the exam more than 90 days after the clerkship ended. Percent of students with passing exam scores was statistically higher in peds, IM and psychiatry.
What are the implications?
It has been a long-held belief that NBME exams should be taken at the end of a clerkship to align with clerkship content. There was no negative impact on the performance of those students with flexible exam timing, suggesting that clinical performance alone may not be the cause of poor test performance and that other factors may play a larger role (fatigue, burnout, mental health, skills, practice). Students who were able to schedule the exam at their choosing could then mitigate these other factors and perhaps improve performance. These kinds of exams may not be a valid assessment of clerkship knowledge and rather the cumulative skills and knowledge gained may contribute to better performance (and give students the ability to mitigate external factors).
Editor’s Comments: The results of this study are intriguing and raise questions about what our clerkship examinations are actually testing; I found it particularly interesting that no student who took an exam before a given clerkship failed that exam. It would be interesting to explore the reasons students might choose to do an exam before versus a lengthy time after a clerkship rotation. (KFo)
Boot Camps and Milestones
Rideout M, Schwartz A, Devon EP, Burns R, Skurkis CM, Carter M, Hartke A, Raszka WV Jr; APPD LEARN/COMSEP Boot Camp Study Group. How Prepared Are They? Pediatric Boot Camps and Intern Performance. Acad Pediatr. 2022 Sep-Oct;22(7):1237-1245.
https://dx.doi.org/10.1016/j.acap.2022.05.005
Reviewed by Nicole Meyers and Suzanne Friedman
What was the study question?
Is participation in pediatric boot camp during medical school correlated with higher intern performance? Is participation in general boot camps, pediatric sub-internships or pediatric electives correlated with higher intern performance?
How was the study done?
Pediatric interns provided information regarding their participation in medical school boot camps, pediatric sub-internships and pediatric electives. If the boot camp included at least 5 days of pediatric-specific coursework, it was considered a pediatric boot camp. Intern performance after their first inpatient month was evaluated by faculty on a 5-point scale for several competencies based on the ACGME Milestones. A 0.5 point difference was deemed educationally significant.
What were the results?
There were 198 interns from 17 residency programs who had complete data – both intern surveys and faculty assessments. No educationally significant differences were seen as a result of participation in pediatric boot camp, general boot camp, pediatric sub-internships, or pediatric electives. In a post-hoc analysis, there was an educationally significant difference in incorporating feedback and engaging in help-seeking behavior for interns during June-July, after completing at least 10 days of pediatric boot camp in medical school.
How can I apply this to my teaching context?
Given the overall lack of significant findings, pediatric educators in the UME space should carefully re-evaluate elements of both general and pediatric-specific boot camps. Medical school leaders must think critically about the final year of medical school and determine the highest-impact preparation they can provide for students to optimize their transition to residency. This may require a closer alignment of boot camp curricula with the AAMC’s EPAs, as well as the creation of a more standardized boot camp format and structure. There may be benefits of boot camps not sufficiently measured in this study, such as the alleviation of stress and anxiety, community-building, and burn-out prevention, which educators must also consider in developing these curricula.
Editor’s Note:: In addition to the lessons above, this study is a great example of the power of research that crosses the UME/GME divide. The authors were also supported by a COMSEP Grant! If you have an idea for educational research, there’s still time to apply. (JG)
Putting Women First
Hochstrasser SR, Amacher SA, Tschan F, et al. Gender-focused training improves leadership of female medical students: A randomised trial. Med Educ. 2022;56(3):321-330.
https://dx.doi.org/10.1111/medu.14658
Reviewed by: Patricia Pichilingue-Reto
What was the study question?
What are the effects of the gender composition of resuscitation teams on the leadership behavior of first responders? Will a brief gender-specific intervention improve the leadership behavior of female medical students?
How was it done?
This was a prospective randomized single-blinded study that included 364 fourth-year medical students of two Swiss universities. One hundred and eighty-two teams of two students each were presented with a simulated cardiac arrest scenario, occurring in the presence of a first and second responder. In the pre-trial phase, all participants received the same training on cardiopulmonary resuscitation (CPR). The gender-specific instruction involved discussing the importance of leadership, gender differences in leadership, acknowledgement of unease while leading, professional role, and mission statement to lead delivered by a physician. Female first responders were randomized to the intervention group or the control group. The primary outcome was the first responders’ percentage contribution to their leadership statements and critical treatment decision making.
What were the results?
Female first responders contributed fewer leadership statements (53% vs 76%) and critical treatment decisions (57% vs 77%) than male first responders; this was more pronounced when second responders were males. The contribution of the male first responder to leadership statements and critical treatment decision making was not affected by the gender of the second responder. The gender-specific intervention resulted in a significant increase in female first responders’ contribution to their team’s leadership statements (69%) and critical treatment decisions (73%). The increase was similar regardless of the gender of the second responder but there was a persistent negative effect of males on female first responders’ contribution to leadership statements and critical treatment decision making. The gender specific interventions had no effect on the quality of CPR and leadership or followership patterns of the first and second responders.
What are the implications?
Female physicians might benefit from additional gender-specific teaching in leadership. Studies like this should continue to be done to bring awareness of gender bias in all aspects of medicine. Addressing these differences should help improve how we prepare our learners to be excellent healthcare providers regardless of gender.
Editor’s note: This is yet another example of how those who identify as female have a more difficult time navigating biases in our healthcare system. While it was nice to see an improvement in female medical students’ contributions, I do worry that we are not addressing why this occurs –that could be a systems level issue. (AP) |
