Council on Medical Student Education in Pediatrics

COMSEP Logo

Search This Site

Journal Club

Hatala R, Issenberg SB, Knssen B1, ColeG, Bacchus CM and Scalese, RJ. Assessing the Relationship between Cardiac Physical Examination Technique and Accurate Bedside Diagnosis during an Objective Structured Clinical Examination (OSCE) Academic Medicine 2007; 82: S26-29.


Hatala R, Issenberg SB, Knssen B1, ColeG, Bacchus CM and Scalese, RJ. Assessing the Relationship between Cardiac Physical Examination Technique and Accurate Bedside Diagnosis during an Objective Structured Clinical Examination (OSCE) Academic Medicine 2007; 82: S26-29.

Reviewed by Margaret Golden

The use of standardized patients and simulators to assess learners' competence in performing physical examinations has become the rule rather than the exception. Although much has been written about the reliability of such methodologies, real questions lurk about the validity of such measures for the criterion that really matters: will this candidate find the physical signs pertinent to a patient's diagnosis?

Most OSCEs as well as the National Board CSE-whether they involve living, breathing human beings or breathing but not living simulators---are scored by an observer who noles which physical exam maneuvers were performed and whether they were performed correctly. While simulators can be programmed to have a variety of physical findings. most standardized patients have few if any findings. Hence the post-encounter measures of whether the learner came to the "right" diagnosis rely primarily on reasoning from data in the history, not data from the physical exam. Yet these encounters are used to rate competence in both history taking and physical examination. The tn.cit assumption is Ihat correct technique will lead to recognition of pertinent physical findings-if they were to be present. These authors used a fairly elaborate study design to test this assumption with respect to the cardiac exam. They compared each examinee's performance and diagnostic accuracy on standardized patients (SPs), real patients (RPs) with cardiac findings, and a cardiac simulator (CPS).

The examinees were internists who had recently passed the RCPSC (Canadian) Internal Medicine exam. Each examinee was rat&! at each station by two examiners, all of whom had previoos experience as examiners for the RCPSC 1M exam. Real patients with stable physical findings of aortic stenosis, mitral stenosis, and mitral regurgitation were first identified. Non-diagnostic presenting scenarios and audio simulations were then prepared for the SP's and the patient simulators, to match the findings of the real patients. As [ understand it, the examinee first examined the SP. hearing the actor's nonnal heart sounds, then listened to a recording of the "test" sounds for that precordial location. The examinees were rated on their technique and on their diagnostic accuracy. and were also given a global assessment of competence.

The authors found a surprising low inter-rater reliability on physical exam technique, of .59 for the RP stations, up to .76 (or the simulators. The correlation between examination technique and diagnostic accuracy--lheir main outcome measure-was only in the range ofO.29-O.39-highest for the real patients, lowest for the simulators.

The authors discuss reasons for the lower than expected inter-rater reliability on technique, which might have masked a stronger association between good technique and "getting the diagnosis." But overall, they conclude thai "technique may be an insufficient surrogate for the assessment of accurate bedside diagnosis." 1n other words, there is an 800 pound goriHa in the room: we are measuring technique usually very reliably-but we are not measuring how effectively examinees are detecting and interpreting findings-which is what matters to our patients.

We can certainly use our creativity to come up with more sophisticated simulations, and we will. I think. these results have another important take home message: we---dinical educators-eannot be replaced by computers. We know, when we work with students or residents, if they are "getting it" or not. even if they go through all the right moves. We cannot abdicate our responsibility for taking time to observe our trainees in action.

Ed Note: This is an excellent summary and call to action by our reviewer Margaret Golden . The problem of technique being a poor surrogate for diagnostic accuracy is even more compounded in pediatrics where so many of our patients with physical exam findings are, by nature of their age or development. not suitable for use in an examination setting. S8

Return to Journal Club