COMSEP Journal Club
December 2020
Editors: Jon Gold, Karen Forbes, and Randy Rockney
Grading and Its Meaning
Are we measuring what matters? How student and clerkship characteristics influence clinical grading. Ingrams MA, Pearman JL, Estrada CA, Zinski A, Williams W. Academic Medicine Published ahead of print. DOI: 10.1097/ACM.0000000000003616
Reviewed by Janet Meller
What was the study question?
What evaluation components predict honors and does the relative importance of components differ among disciplines?
How was the study done?
Clinical evaluations of 3rd year students rotating through IM, Surgery and Pediatrics from July 2015-June 2017 at a single medical school were examined. The evaluation tool consists of 19 items of which 12 address clinical work. Each of the latter has 4 milestone-based descriptors. Evaluations were examined using exploratory factor analysis, internal reliability, bivariate analyses between honors and 4 items: contact time, evaluator, the results of the 12 items above and the recommendation by evaluator for pass/honors.
What were the results?
3,947 evaluations were completed: 35.6% IM, 37.2% surgery and 27.2% pediatrics. 38.4% were honors: 46.9% in IM; 35.8% in surgery and 30.7% in pediatrics (p.0.001). Mean contact time increased likelihood of honors. Residents and evaluators who observed H&Ps were more likely to recommend honors. The most highly valued categories were: knowledge application, clinical reasoning and response to feedback (IM); presentation skills, record keeping and team interaction (Surgery); and knowledge application, presentation skills and dependability (Pediatrics).
What are the implications?
Clinical evaluations play a large role in assessment of medical students and hence residency selection. It is important to more fully understand their structural and student-level components. In this study, evaluators prioritized the cognitive domain over interpersonal skills. Also, length of contact between learner and evaluator is important especially because most clerkships don’t allow for prolonged contact between students and faculty. This study did not address the value of other important skills like ownership, motivation and curiosity. Demonstrating that residents are more likely to award honors indicates more training in evaluation is necessary.
Editor’s note: This is one more effort in our never-ending search for the holy grail of clinical evaluation. Clinical evaluation is made even more difficult, as shown in this study, by the students’ oftentimes brief and fragmented clinical experiences. Other studies have noted that residents and younger physicians tend to be more generous with their honors grades. The reasons for that phenomenon would be an interesting topic for study. (RR)
Traditional Lecture Versus Online Module ? The answer is….
Effectiveness of Modalities to Teach Evidence Based Medicine to Pediatric Clerkship Students: A Randomized Controlled Trial. Hadvani T et al. Academic Pediatrics; 2020 in press. https://doi.org/10.1016/j.acap.2020.09.012
Reviewed by: Nicole Meyers and Suzanne Friedman
What was the study question?
How does a traditional didactic session (TDS) compare to a self-paced multimedia module (SPM) for teaching EBM to pediatric clerkship students in terms of effectiveness, knowledge, attitudes, confidence and skills related to EBM.
How was the study done?
This randomized controlled trial was conducted at a Baylor College of Medicine-affiliated children’s hospital. Students on the inpatient component of their pediatrics clerkship rotation were cluster randomized into two groups: TDS versus SPM. The TDS and SPM content included topics such as developing a PICO question and conducting literature searches. The TDS group received a lecture using a PowerPoint presentation, while the SPM group participated in an interactive, multimedia online module. Students in both groups submitted a Critically Appraised Topic (CAT) form that was scored using a validated tool to assess their EBM skills; this was considered the study’s primary outcome. Secondary outcomes (knowledge, attitudes, confidence and accessing evidence) were measured using surveys administered pre-intervention, immediately post-intervention and 3 months later.
What were the results?
There were no significant differences in CAT scores between students in the TDS versus SPM group. Additionally, no differences were seen in secondary outcomes including attitudes between the two groups across time points. There was improvement in these domains from pre- to immediately post-educational intervention in both groups. Improvements in knowledge and confidence were sustained at 3 months post-intervention for both groups; however, accessing evidence did not improve and attitudes declined at this later timepoint.
What are the implications?
This study demonstrates the efficacy of both an in-person didactic and a self-administered online module for teaching EBM to medical students. Educators should consider the resources available at each of their institutions in deciding the optimal approach to EBM teaching, as an SPM requires more upfront development, yet a TDS can be challenging to sustain. The stagnation in seeking evidence and decline in attitudes toward EBM at the 3-month timepoint underscores the need for longitudinal EBM curricula. This requires coordination across clerkships to ensure EBM is consistently emphasized throughout the clinical years of medical school.
Editor’s comments: Although the educational outcomes of both methods were comparable, there is always the concern that online resources may not be used by all learners. Of note in this study, all students assigned to the traditional didactic session attended the session, whereas there was no monitoring of how many students completed the self-paced multimedia module, or how engaged they were in participation. Since online resources are particularly relevant during current times, given the huge time commitment in developing such a resource, it would likely be helpful to have a system to track a learner’s progress or at minimum completion of the module. (KFo)
Interdisciplinary Teamwork
“I didn’t know they had such a key role.” Impact of medical education curriculum change on medical student interactions with nurses: A qualitative exploratory study of student perceptions. Samuriwo, R, Laws, E, Webb, K, Bullock, A. Advances in Health Sciences Education. 2020; 25: 75 – 93. https://doi.org/10.1007/s10459-019-09906-4
Reviewed by Cristina Alcorta and Randy Rockney
What was the study question?
Can opportunities for medical students to learn from allied professionals affect perceptions of interactions with nurses?
How was the study done?
This qualitative study employed semi-structured, narrative interviews with medical students from two different curricula: traditional and integrated. The school had transitioned from its traditional curricula, in which medical students had a 1-week nursing placement during second year, to an integrated curriculum providing new opportunities for medical students to learn from students and professionals from a range of disciplines (pharmacology, nursing, etc). A consensus was reached on the main themes and subthemes.
What were the results?
Three main themes were identified: 1) Medical Curriculum 2) Understanding Nurses’ Roles 3) Relationships and Teamwork. Subthemes included topics like Interprofessional Education, Maternal nurturing, Hierarchy & Culture, Conflict and frustration. Participants endorsed that the interprofessional curriculum had increased awareness of nursing roles and responsibilities, enabling them to challenge pre-existing stereotypes. In addition, nurses were recognized as medical educators who supported participants to develop competence. Participants described a perceived difference in nurses’ attitudes towards male and female students, showing a less-positive demeanor towards women.
What are the implications of these findings?
This study calls attention to early clinical exposure in the development of interprofessional relationships. Participants identified barriers to team-working: poor understanding of nurses’ educational and professional roles and stereotypical views about the relative knowledge and abilities of nurses and doctors. Of note, there was significant commentary on gender roles influencing respect and professionalism. Facilitating development of meaningful interprofessional relationships can challenge gaps or misconceptions in knowledge, attitudes or perceptions to facilitate effective teamwork. This, in turn, may minimize adverse patient outcomes during post-doctoral training. Overall, this study lends consideration to whether interprofessional education should evolve from optional to mandatory.
Editor’s note: One of the first principles I learned as a medical student over 40 years ago is that when a nurse asks you to jump you should ask how high. As an intern and a resident, nurses saved my skin lots of times…especially in the NICU. Medicine is and should be a team sport and awareness of the knowledge and skills of one’s teammates, and the respect that should follow, is best for all parties. Formal interprofessional training can hopefully jumpstart that lifelong commitment to teamwork. (RR)
Lockdown age: can medical students see patients from home?
Medical students consulting from home: A qualitative evaluation of a tool for maintaining student exposure to patients during lockdown.Darnton R , Lopez T, Anil M , Ferdinand J, Jenkins M(2020). Medical Teacher. 2020; https://dx.DOI.org/10.1080/0142159X.2020.1829576
Reviewed by Alina Leschiner
What was the study question?
Can face-to-face patient encounters be sufficiently substituted with remote consultations in medical education in a way that is feasible, acceptable and effective?
How was the study done?
Following COVID lockdown, second-year medical students in Cambridge England participated in a pilot experience in which students attended 3 remote clinics via video or phone from their own home; the supervising doctor was based in a primary care center or their own home; the patient was based in their own home.
A sample of 13 students and 10 clinical supervisors participated in post-experience semi-structured interviews. Transcripts of the interviews were recorded and then analyzed for themes.
What were the results?
Overall, some students viewed remote consulting was seen as a crucial skill that was an essential part of medical practice, while others saw at as a second-best experience when in-person learning was unavailable. The difference depended on whether they felt able to connect with patients and practice their clinical skills.
The main perceived advantages of remote consultations included reduced travel time and less dead time for students along with access to fragile and complex patients. Students and supervisors reported that the fact that they were on camera forced all participants to be more alert and attentive during the encounters.
The main perceived disadvantages mentioned were difficulties with internet connections, inability for supervisors to give a full-rounded feedback to students secondary to limited time for informal communication, and fear of being interrupted during the encounter. Supervisors reported that selecting the right patients for students to make it a valuable experience was ta time-consuming process. They also reported that the dynamics of three-way communication were more challenging remotely than in person.
What are the implications?
Remote patient consultations conducted by medical students under supervision from home may be acceptable and educationally valuable. They require a thorough patient selection, reliable internet connection and a comfortable communication platform along with reasonable amount of time for patient discussions. With those corrections remote patient encounters could be used as an efficient substitute for in person clinical experience in the times of pandemics.
Editor’s note: Some of the challenges described are the same ones that practicing clinicians have conducting telehealth visits. Others are unique to the educational experience. Pandemic or no, it is likely that telehealth will be part of healthcare’s future and learning how to do it effectively is probably a worthwhile investment. (JG)