Council on Medical Student Education in Pediatrics

COMSEP Logo

Search This Site

COMSEP 2012 Indianapolis Meeting

Poster Presentation:


Do You See What I See? Faculty and student assessments of directly observed encounters

Authors:
Miriam B Schechter, MD - Albert Einstein College of Medicine /Children's Hospital at Montefiore; Jeffrey R Avner, MD - Albert Einstein College of Medicine /Children's Hospital at Montefiore; Allison N Schechter -

Background: Direct observation of trainees during patient encounters is an important method for evaluating clinical skills. Self-assessment helps students develop a realistic view of their strengths and weaknesses.  One measure of the accuracy of student self-assessment is the agreement with faculty assessments. Use of a relative ranking scale has been shown to enhance correlation between student and faculty assessments.

Objective: To determine the correlation of pediatric clerks’ self-assessments with faculty assessments of a directly observed clinical encounter, using a relative ranking scale, and to identify skills judged by each group as those requiring the most and least work, to help guide educational efforts.

Design/Methods: A directly observed clinical encounter is a requirement during our pediatrics clerkship. For this formative assessment, the attending uses a detailed standardized checklist of clinical skills. To encourage self-assessment, we added the use of a relative ranking scale, with which the student and faculty member independently rank skills from those needing the most work to those needing the least. Six general areas from the checklist are ranked: opening skills, history-taking skills, interpersonal skills, PE-approach, PE-technique, and patient education. The checklist and rankings are discussed immediately after the encounter.  Correlation of rankings for 160 student-faculty pairs, was calculated using the Spearman rank order correlation coefficient. The top areas felt to require the least and the most work were identified.

Results: There was a positive correlation between student-attending pair rankings, with a mean ρ=0.57 (SD 0.40; 95%CI 0.51-0.64). Students ranked PE-technique or patient education as the area needing the most work (42% and 33% of students respectively), as did faculty (for 31% and 44% of students respectively). Students and faculty chose opening skills or interpersonal skills as needing the least work (41% and 40% by students respectively, 37% and 38% by faculty).

Conclusion:  A relative ranking scale to evaluate a directly observed pediatric encounter showed good correlation between student self-assessments and faculty assessments. Both generally agreed on areas needing the most work and those performed best. This information will assist students and faculty in formulating learning plans.