Power to the people: taking the assessment of physician performance outside the profession

Vahé A. Kazandjian, PhD, MPH

CMAJ 1999;161:44-5

• Letter: Should we preach parsimony for health care?
• Education: Assessment of physician performance in Alberta: the Physician Achievement Review

On the eve of a new millennium, one of these, medicine, is taking on a direct social responsibility, that of demonstrating to the communities it serves that any care given is not only appropriate (as valued by clinicians) but also parsimoniously provided and responsive to society's needs. Such is the backdrop for many of the performance evaluation initiatives in North America,^1,2,3,4 Europe⁵ and Australia.⁶ At the core of these initiatives and wrapped within the principles of quality and accountability is the expectation of improvement. Indeed, documentation of better delivery of care, with emphasis on compassion for patients, is what various stakeholders, including patients, regulators, insurers and government, are requesting. The question is, are these constituents asking about the outcomes of care or about the processes by which that care is delivered? Is the medical profession expected to deliver the best care or to achieve the best outcomes? Ironically, although the answer to these questions is neither straightforward nor universal, it must be determined, because good processes do not always result in good outcomes.

The Physician Achievement Review as a means of reviewing performance

In this issue, Dr. William Hall and associates describe a singularly noteworthy initiative of the College of Physicians and Surgeons of Alberta (page 52). The Physician Achievement Review (PAR) is a systematic, ongoing, structured program to assess selected dimensions of physicians' performance. The ultimate goal of this initiative is to improve medical practice through peer review of comparative performance profiles by the college's Physician Performance Committee.

The authors specify that the name of the program, Physician Achievement Review, was selected "... to denote a supportive purpose and the goals of describing professional accomplishment and improving practice." The developers of the PAR program have addressed these goals by identifying a typology of the attributes of the performance of medical care, characterizing these attributes as essential, important, desirable or irrelevant. Most important, the assessments done within the context of the PAR program are not made by the physician alone, but by patients, medical peers, consultants and other health care providers. It is perhaps in this vision of who should evaluate performance that the PAR distinguishes itself from other similar initiatives. Although there have been attempts to have physicians' performance evaluated by peers alone, the developers of the PAR have recognized the importance of accountability to a larger group which, eventually, will decide if the ultimate social service, health care, has been provided caringly, parsimoniously and appropriately. But what is "performance"?

The definition of performance

When dealing with performance measurement systems, such as the PAR, it is important to distinguish between performance and quality. Whatever the eventual strategy, all measurement systems are designed to identify variation. Because variation has no intrinsic value, the distinction between performance and quality becomes one that might be used to build consensus among stakeholders or, if misused, might seriously threaten an accountability initiative. In fact, at least one study using US national statistics reported that high variation may be associated with highly appropriate care and that, conversely, low variation may be associated with highly inappropriate care.⁷ Thus, I would propose the following operational definitions aimed at unifying the concepts of performance and quality: Performance is the objective description of activities toward a well-stated goal. Quality is the value placed upon that performance.

I suggest that, in health care, placing a value on performance — evaluating performance — is best carried out at the local level, where the care is provided. The rationale is that the value system in place there, which will be based on the expectations of local residents, will be reflected in the evaluation of performance. Thus, when the PAR is tested locally in Alberta, it will be affected by local practice styles and expectations. For example, rural patients may have different expectations than urban patients and may evaluate the same type and level of care differently. As a result, the validity of the PAR initiative in contributing to the improvement of performance may not be fully tested if the initiative is not applied in different environments.

Could the experience of Alberta's PAR program be expanded nationally? Although it is too early to proceed in that direction, a number of structural characteristics of the PAR program suggest that a larger initiative could be undertaken. First, the program is based on a comparative analysis of performance. Indeed, physician performance profiles are systematically quantified through rates, and trend analysis is planned over time, across physicians. Second, it seems to be reliable in documenting variation and the potential sources of and reasons for such variation. Third, its framework is flexible and can thus accommodate changing philosophy, science and the relation between process and outcome. The first two characteristics pave the way toward generalization of the PAR to other jurisdictions, while the third needs further consideration.

Indeed, although the framework is flexible and amenable to change, it may be challenged by an "uneven" evolution in clinical practice and philosophy across regions or groups of providers. Thus, the generalizability of the PAR may be stepwise rather than simultaneous across regions.

In contrast, the first 2 structural characteristics of the PAR are fashioned following more readily available experiences from the health care research and epidemiological fields.

For clinicians who are exposed to research about and reporting on appropriateness of care, the interpretation of what variation really means when they are dealing with individual patients is a complex question. It is not easy to translate a group observation into a recommendation on how to manage an individual patient. The College of Physicians and Surgeons of Alberta, by accounting for the expectations of people other than physicians in the assessment tools of the PAR, has shown significant vision and strategic courage. However, it seems absolutely necessary that clinicians be involved in the evaluation of the variation, since the interpretation of performance profiles, comparative analyses and temporal patterns will require an intimate understanding of the relation between patients' characteristics (including their expectations) and decisions about the care path, which consists of clinical and patient need management dimensions. The PAR initiative is a thoughtful attempt to use past initiatives as guides rather than exclusive models. As such, we may be seeing a groundbreaking initiative by the College of Physicians and Surgeons of Alberta.

Competing interests: None declared.

Send a letter to the editor Envoyez une lettre à la rédaction

Reprint requests to: Dr. Vahé A. Kazandjian, Center for Performance Sciences, Inc., 6820 Deerpath Rd., Elkridge MD 21075-6200; fax 410 379-9558; vkazandjian@cpsciences.com

1. Kazandjian VA, Lied T. Caesarean section rates: effects of participation in a performance measurement project. Jt Comm J Qual Improv 1998;24(4);187-96. [MEDLINE]
2. Vayda E. A comparison of surgical rates in Canada and in England and Wales. New Engl J Med 1973;289:1224-1229. [MEDLINE]
3. Vayda E, Morison M, Anderson GD. Surgical rates in the Canadian provinces, 1968 to 1972. Can J Surg 1976;19:235-42. [MEDLINE]
4. Stockwell H, Vayda E. Variations in surgery in Ontario. Med Care 1979;17(4):390-6. [MEDLINE]
5. Donaldson L, Donald I. Approaches to quality improvement in the British National Health Service. In: Kazandjian VA, editor. The epidemiology of quality. Gaithersburg, Md.: ASPEN; 1995. p. 197-220.
6. Boyce N. Quality in Australian health care: moving the goalpost on a roughly paved road to improvement. In: Kazandjian VA, editor. Effectiveness of CQI in health care: stories from a global perspective. Milwaukee, Wis.: American Society for Quality Control; 1997. p. 110-27.
7. Chassin MR, Brook RH, Park RE, Keesey J, Fink A, Kosecoff J, et al. Variations in the use of medical and surgical services by the Medicare population. N Engl J Med 1986;314:285-90. [MEDLINE]

© 1999 Canadian Medical Association