Council on Medical Student Education in Pediatrics


Search This Site

Journal Club

One cannot be assessed by a few questionnaires alone
Multisource feedback in evaluating the performance of doctors:  The example of the UK General Medical Council patient and colleague questionnaires.  Wright C et al.  Academic Medicine 2012;87:1668-1678.

Reviewed by Maria Alcocer

What was the study question?
UK physicians are required to undergo five-year cycle reviews for revalidation of their General Medical Council (GMC) Licensure.  A component of this review now includes a multisource feedback (MSF) Patient Questionnaire (PQ) and Colleague Questionnaire (CQ), which the physician is expected to collect, reflect on, and use to inform their further professional development where appropriate as part of their revalidation appraisal process. 

This study sought to examine how suitable the PQ and CQ tools are in terms of suitability for use in the evaluation of doctors’ professional performance prior to their mandatory use.

How was the study done?
UK Doctors from a variety of National Health Service and private-sector organizations were invited to voluntarily participate.  Doctors use the PQ as an exit survey for 45 consecutive patients and nominated up to 20 colleagues (10 medical, 10 non-medical) to provide feedback using the CQ.

The CQs & PQs were developed by the GMC based on “Good Medical Practice” principles and included items related to performance, global assessment items and a free-text comment area.  Changes over time were evaluated by having respondents rate performance on two occasions as a test/retest.

What were the results?
Although ratings were highly skewed toward favorable impressions of doctor performance, statistical analysis found that overall the PQ & CQ surveys were acceptable and met reliability and validity standards as part of a larger workplace formative feedback assessment. 

What are the implications of these findings?
Although this study was voluntary and contained several potentials for positively skewed results, multisource feedback can be a valuable tool in the overall formative evaluation, self-assessment and reflection of a medical provider. 

In the UK, the surveys are not currently validated against directly observed measures of practice, skills or knowledge.  It is worthwhile to consider incorporating MSF surveys along with these components particularly during medical school and post-graduate training when professional attitudes and identities are being formed. 

Editor’s note: This is a large scale study of the psychometric properties of two physician rating forms. They concluded that each physician required feedback from at least 34 patients and 15 colleagues to make the findings valid and reliable. It provides a note of caution to those of us who would like to incorporate multi-source feedback into our evaluations of our students (LL).

Return to Journal Club