July 2019

posted in: 2019 Journal Clubs | 0

A standardized letter for residency selection

Bajwa NM, Yudkowsky R, Belli D, Vu NV, Park YS.  Validity evidence for a residency admissions standardized assessment letter for pediatrics. Teach Learn Med 2018; 30(2):173-183.

https://dx.doi.org/10.1080/10401334.2017.1367297

Reviewed by Gary Beck Dallaghan

 

 

What was the study question?

Can a standardized Assessment Letter for PediatricS (ALPS) based on competencies desired for a pediatric resident--professional integrity, scientific curiosity, patient management skills, autonomy and organization, teamwork/collaboration, and communication skills-- be developed and validated?

How was the study done?

Candidates for the University of Geneva Pediatrics Residency Program were interviewed by 2 faculty members using structured questions which were scored and independently gave each candidate a global rating. Other considerations for the final rank included exam scores, CV, prior publications, and ALPS score.

Validity evidence included content (CanMEDS roles and pediatric specific competencies were used to build a blueprint for ALPS), internal structure (Cronbach’s alpha for internal consistency reliability along with generalizability and decision studies), response process (intraclass correlation was used for interrater reliability), relations to other variables (association between ALPS and interview scores, global rating scores, and admissions decisions), and consequences (multiple logistic regression model with admissions decision as the dependent variable).

What were the results?

For those 71 (of 73) applicants in this study, 142 ALPS were available for the analyses.  Response process resulted in fair to moderate ALPS rater agreement.  Applicant variance accounted for 28% of the ALPS scores, suggesting ALPS differentiates candidates.  The decision study suggested a minimum of 4 ALPS forms are needed to have a reliable G-coefficient.  ALPS scores correlated with interview, global scores, and acceptance decisions.  However, ALPS was not a significant predictor of admissions decisions in the logistic regression model.

What are the implications?

Having a standardized letter of evaluation differentiated candidates, even with faculty using the higher end of the rating scales.  ALPS offers a structured evaluation of candidate performance that could be helpful for faculty who are not accustomed to writing letters.

Editor’s note:  At last a means to avoid the Lake Woebegone phenomenon with letters of recommendation for pediatric residency applicants. (RR)

 

How learners think about  competency-based medical education

Perceptions of competency-based medical education from medical student discussion forums.

Dehmoobad Sharifabadi A, Clarkin C, & Doja, A. (2019). Medical Education, 53(7), 666–676.

https://doi.org/10.1111/medu.13803

Reviewed by Daniel Herchline

What was the study question?

What are trainees’ perceptions of competency-based medical education (CBME)?

How was the study done?

The authors conducted a qualitative analysis of two online medical student/resident discussion forums, Premed 101 (Canadian) and Student Doctor Network (American), to identify themes present in discussions regarding CBME. The authors examined the data through the theoretical lens of social constructivism and identified emergent themes using inductive content analysis.

 

What were the results?

The authors reviewed a total of 3623 posts spanning 70 discussion threads between the two forums. While the comments were posted anonymously, the authors noted that some participants self-identified as medical students or residents. The overarching themes identified included effect of CBME on length of residency, effect of CBME on post-residency endeavors, the subjective nature of evaluations, CBME promoting trainee-centered education, terminating a resident’s position, the advantages of CBME, and gaining learning experience versus simply checking boxes. Although the American-based and Canadian-based forums shared similarities in the discussion topics, there were some notable differences in prioritization of topics which highlighted potential contrasting geo-cultural perspectives.

 

What are the implications?

The successful implementation of CBME will likely depend, at least partly, on trainees’ perceptions of its benefits and challenges. While trainees express multiple concerns about the transition to CBME, they are also cognizant of its potential upside. Regardless of their opinions, trainees should be made primary stakeholders in discussions involving CBME. Furthermore, they should be given as much clarity as possible regarding the impact of CBME on length of training, assessment and evaluation, and remediation.

Editor’s Note: While much of the discussion in these forums focused on residents, the same issues are likely to arise with medical students as medical schools move toward a competency-based framework.  This early window affords us the opportunity to avoid some of the mistakes we might otherwise make. (JG)

 

What’s being done about wellness?

Medical School Strategies to Address Student Well-Being: A National Survey.

Dyrbye LN, Sciolla AF, Dekhtyar M et al. Academic Medicine. 2019;94(6):861-868.

https://doi.org/10.1097/ACM.0000000000002611

Reviewed by Michele Haight

What was the study question?

What are the current strategies being implemented in US Medical Schools to promote medical student well-being?

How was the study done?

Thirty-two MD and DO granting US Medical schools which participate in the American Medical Association’s (AMA) “Accelerating Change in Medical Education Consortium” were invited via e-mail to participate in a 2016 survey about their student well-being programs, activities, curricula, etc. Multiple individuals from each school with knowledge about student wellness participated in the survey. Conflicting information from survey participants was further probed by the authors and any discrepancies were resolved through consensus.

What were the results?

Of the twenty-seven (84%) consortium medical schools that participated in the survey, sixteen (59%) reported having a student well-being curriculum. Frequency of curriculum delivery varied, occurring monthly (75%), quarterly (6%) or yearly (19%). Student attendance policies also varied, with 13% having mandatory activities, 31% being optional and 56 % had a combination of optional and mandatory activities. The survey identified the following well-being domains: Emotional/Spiritual, Physical, Financial, Social. Activities included mindfulness meditation training; supporting student-defined organizations; encouraging physical activity through organized events; lectures related to finances; and hosting of social events. Many schools underscored the necessity of having a wide assortment of social activities that appeal to diverse groups of students.  Only 22% of respondents reported that they had developed student wellness competencies. The most common evaluation strategies for the wellness curriculum were student participation rates (96%) and student satisfaction rates as reported on surveys or course evaluations (81%). Most (82%) of the respondents reported that an individual or group of individuals were charged with student wellness oversight most of whom had dedicated time for well-being activities. Budgets for student well-being programs ranged from $0-35,000/yr. with an average of $2963/yr.

What are the implications of these findings?

Distress in medical school is an indicator of distress in residency training. Therefore, student wellness programs need to be a multi-dimensional, institutionally resourced, and strategically integrated component of the overall UME curriculum.

Editor’s Comments:  As we continue to explore ways to promote student wellness, we must consider how to rigorously evaluate the impact of these interventions. This study demonstrates there are many approaches, but we must determine what strategies are truly valuable and make a difference to student wellness, and to ensure resources are allocated to those most impactful interventions (KFo).

Getting from spoonfed to self-feeding on clinical rotations

Designing faculty development: lessons learnt from a qualitative interpretivist study exploring students’ expectations and experiences of clinical teaching.

Blitz J, de Villiers M, van Schalkwyk S. BMC Medical Education 2019; 19:49.

https://doi.org/10.1186/s12909-019-1480-7

Reviewed by: Eliza Pope & Angela Punnett

What was the study question?

How do senior medical students’ experiences and expectations of clinical teaching impact how they learn during clinical rotations?

How was the study done?

This qualitative study is the first of four analyses of clinical teaching, conducted by medical educators at Stellenbosch University in South Africa to inform the strengthening of faculty development initiatives for clinical teachers. Using cconvenience sampling, 23 medical students in their final clinical rotation participated in three scheduled focus groups. These groups explored students’ definitions of clinical teaching, beneficial and detrimental experiences, and suggestions for improvement. Transcripts of these meetings were analyzed for major themes, using an interpretivist approach.

What were the results?

The majority of themes tied to a broader concept of tension between students’ expectations and experiences of clinical teaching. These tensions spanned from frustrations due to lack of concrete answers and structure, to not being included or given responsibility in a clinical team, to challenges in requesting to be taught in a clinical setting. The researchers also examined how students responded to these frustrations, highlighting both an acceptance of the discomfort in clinical learning, but also a realization that learning was seldom handed to them, and that as learners, they needed to develop their own sense of agency. Notably however, students felt that despite this need, they had little agency to adapt clinical experiences to meet their learning needs. This stemmed largely from uncertainty, a lack of relationship with clinicians, and a lack of clarity regarding a given physician’s role in teaching.

What are the implications?

The authors suggest that faculty development initiatives should include strategies to help clinician educators and by extension, resident teachers that (1) promote the personal agency of learners and (2) offer engagement experiences to them through clinical participation. In particular, students valued caring for patients independently, formulating a diagnosis and management plan, and subsequently consolidating this plan with input from physicians.

Editor’s Comments: I suspect most of us have experienced the disconnect between students’ and our own perceptions of what constitutes teaching and learning in the clinical environment along with students’ oft relayed desire to be spoon fed the “answers”.  It makes me wonder what more we could/should be doing to help our learners take ownership of their learning BEFORE they get to the wards and clinics (KFo).