Hello COMSEP!
Happy April 1st! Attached and below is another special edition of the COMSEP Journal Club.
In the spirit of the times, two of the three reviews below were at least co-written by ChatGPT. Can you tell which is which? And maybe this is an opening salvo in an important research question---can AI be funny?
See you soon,
Karen, Jon and Amit
Enough words already!
Smith JR, Patel LK, Nguyen MD. Evaluating modern feedback strategies: Can emojis and GIFs replace traditional narrative feedback? J Med Educ Trends. 2024;18(2):45-57.
Reviewed by Stephanie Berger
What was the study question?
Are modern communication methods in the form of static images (emojis) or short looping videos (GIFs) a better way to give feedback to Generation Z learners?
How was it done?
Medical students in the clerkship stage of their medical school training, born between 1998 and 2005, voluntarily participated in this study. At the end of their pediatric clerkship, they were given summative feedback in one of two formats: emojis only or GIFs only.
Students were then asked to:
- Interpret their feedback in written form.
- Rate the helpfulness of the feedback using a 5-point Likert scale.
- Finally, receive traditional written summative feedback.
The students' interpretations were compared with their official written feedback using BERTScore to assess semantic similarity.
What were the results?
A total of 147 students participated in the study. GIFs outperformed emojis in perceived helpfulness (4.9 vs. 2.7; p < 0.001). However, neither emojis nor GIFs closely matched traditional written feedback in narrative accuracy (BERTScore F1: 34% and 29%, respectively).
Interestingly, GIF-based feedback showed a bi-modal interpretation pattern, with students either strongly positive or highly negative in their understanding of the message.
How can this be applied to my work in education?
Despite enthusiasm for modern communication tools among Generation Z learners, this study suggests that emojis and GIFs are ineffective for summative feedback in medical education. Face-to-face conversations and traditional written feedback remain the most effective methods for conveying meaningful performance evaluations.
Embarrassing your Gen Z/Alpha relatives? Shake it off!
Reviewed by Margaret Huntwork
Carpenter S, Eilish B, Grande A. Impact on family members of Pediatric Program Directors by frequent use of the Electronic Residency Application Service
Journal of the Intersection of Pop Culture and Pediatric Medical Education. 2024 (13): 22-24. Doi: 18391032858.
What was the study question: Are Pediatric Program Directors (PDs) so entrenched working in the ERAS (commonly pronounced eeee-rass) system that they mispronounce the name of Taylor Swift’s Eras (pronounced eh-ruhs) tour, embarrassing their young family members?
How was the study done: Children and young relatives of American Pediatric Program directors who were between the ages of 7 and 23, who identify as Swifties, and lived within 200 miles of cities that hosted concerts of the 2023-2024 Taylor Swift Eras tour were surveyed. Survey questions explored whether the PDs ever pronounced the name of the Taylor Swift concert tour as the “eeee-rass“ tour instead of the “eh-ruhs” tour. Details surrounding LOVE scores (Level of Embarrassment) were also collected. Descriptive statistics were applied.
What were the results?
The survey, distributed as a QR code on a beaded friendship bracelet, had an astounding 100% completion rate. Over 2/3 of respondents recall hearing the PD call the Taylor Swift concert the “eeee-rass tour” on at least one occasion with a median number of mispronunciations being 2. All mispronunciations occurred during residency interview season or in the month immediately following residency interview season. LOVE scores were high, but embarrassment was short-lived. Authors conclude that PDs must be so entrenched in their work during interview season that any time they see the letters E-R-A-S in this sequence, their minds automatically go to work activities and not TSwift concert tours.
How can this be applied to my work in education: Perhaps all surveys should come via friendship bracelets to maximize the survey response rate. Authors hope the study will encourage institutions to subsidize TSwift concert tickets for PDs and monitor for change in LOVE scores.
Editor’s comments: Not in my wildest dreams would I want to cause any bad blood between our countries, but applicability to the Canadian context is limited, as you likely all know all too well. (KFo)
I’m matching on good vibrations
Steves R, Fodor A, Polo M, Planet L. Career Advising in Pediatrics – Cracking the Code on Geographic and Signaling Preferences. Medical Geography. 2024 (1). doi://mg12345678
Reviewed by Erin Pete Devon
What was the study question?
How can we help applicants use geographic preference and signaling effectively to maximize their chances of interviewing at a program that aligns with their career and personal goals?
How was the study done?
Researchers conducted a mixed-methods study using American Association of Medical Colleges (AAMC) residency match data, applicant surveys, and advanced statistical modeling (aka educated guessing). They examined how applicants’ geographic preferences, as reported in the AAMC data, correlated with their ultimate match locations. Special attention was given to the effect of signaling, including whether sending a signal to a program increased an applicant’s likelihood of receiving an interview, or just resulted in them spiraling into existential doubt over their choices.
What were the results?
Most (85%) applicants who reported a geographic preference still applied to programs outside their preferred region “just in case.” For 62% of those who signaled a program, they ultimately ranked it in their top three choices and 48% of them matched at a signaled program. Some (39%) applicants signaled programs in cities they had never visited, with the most common reason being, “I heard it was a good place to live” or “I liked the skyline in a picture on Instagram.” Faculty advisors spent an average of 3.7 hours per advising session helping students understand that signaling is not a golden ticket for an interview, but just one piece of the package. Applicants from the Northeast were the most likely to indicate a strong preference to stay in their region, while those from the Midwest demonstrated the highest rate of geographic flexibility, possibly due to their natural resilience to unpredictable weather and deep-dish pizza debates.
How can I apply this to my work in education?
Career advisors can take several crucial lessons from this study. While applicants have access to lots of data, several simply disregard it and make decisions based on “vibes.” Career advisors should balance hard data with individualized guidance—and possibly a therapy dog for moral support.
Editor's Note: This study validates my gut feeling that this whole process is mostly done on how people are in the moment rather than what the data makes you feel. (AKP)