Lindsey Tilt, MD - Morgan Stanley Children's Hospital of New York Presbyterian-Columbia University; Andrew Mutnick, MD - Morgan Stanley Children's Hospital of New York Presbyterian-Columbia University; Robert Best, MD - Morgan Stanley Children's Hospital of New York Presbyterian-Columbia University; Martin Pusic, MD, PhD - Morgan Stanley Children's Hospital of New York Presbyterian-Columbia University
Background: Outpatient teaching models such as the One-Minute Preceptor direct trainees to independently commit to clinical decisions before the preceptor gives feedback. Comparing trainee decisions to those of the preceptor could yield insight into trainee decision-making.
Objective: To determine whether Trainee-Preceptor decision concordance, operationalized as a rating scale form, reflects trainee clinical decision-making ability to a measurable extent.
Design/Methods: We recruited a convenience sample of residents and students rotating in the PED. A 2-sided form had the same 7 clinical decisions on each side: whether to perform blood, urine, spinal fluid tests, imaging, IV fluids, antibiotics, or a consult. The rating choices were: Definitely Not, Probably Not, Probably Would or Definitely Would. Trainees rated each decision after seeing a patient, but before presenting to the preceptor, who, after evaluating the patient, rated the same 7 decisions on the 2nd side of the form. The preceptor also indicated the most relevant decision (MRD) for that patient.
We examined the validity of the technique using hypothesis testing; we posited that residents would have a higher degree of concordance with the preceptor than would medical students. This was tested using dichotomized analyses (accuracy, kappa) and ROC curves with the preceptor decision as the gold standard.
Results: 31 students completed 130 forms (Median 4 forms; IQR 2,6) and 23 residents completed 206 (6; IQR 3,12). Preceptors included 24 attendings and 3 fellows (9; IQR 4, 21). Students were concordant with preceptors in 70% (k=0.38) of MRD while residents agreed in 79.6% (p=0.045), k=0.59. ROC analysis revealed significant differences between students and residents in the AUC for the MRD (0.84 vs 0.72; p=0.03).
Conclusions: This measure of trainee-preceptor concordance requires further research but may eventually allow for assessment of trainee clinical decision-making. It also has the pedagogical advantage of promoting independent trainee decision-making.