This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: https://info.ahrq.gov. Let us know the nature of the problem, the Web address of what you want, and your contact information.
Please go to www.ahrq.gov for current information.
AQA Invitational Meeting Summary
Report of the Performance Measurement Workgroup
Kevin Weiss, American College of Physicians
Kevin Weiss opened his remarks with a comment about the workgroup's process.
He said that the workgroup tries to discuss a document or a concept until
there is little left to discuss. As a result, when the workgroup votes on
an item there are generally only a handful of abstentions (usually because
of a conflict of interest, because a person did not have time to review the
materials, or because a person was not comfortable speaking for his or her
organization on the issue) and a handful of dissenting votes. He added that
there are usually about 65-70 people on each call.
Weiss thanked the members of his workgroup, and noted that they have accomplished
much. These accomplishments include approval of the Ambulatory Care Starter
Set (26 measures), cardiology and cardiac surgery measures, and development
and approval of a document on principles of efficiency measurement. In addition,
he said the workgroup has held six cost-of-care physician education WebEx
seminars and two meetings with specialty societies to review measure development
Next, Weiss outlined his workgroup's meeting objectives:
- Review and endorse the registry principles document.
- Review and endorse the Guide for the Selection of Performance Measures
for Medical Subspecialty Care.
- Review and endorse the Surgery/Procedure Workgroup Mission and Principles.
- Review and endorse the quality metrics for dermatology, radiology/neurology,
rheumatology/clinical endocrinology, ophthalmology, surgery, and orthopedic
- Review and endorse the CAHPS® Clinician and Group Survey,
including composite measures.
- Review and endorse the candidate list of conditions for cost-of-care
- Review and discuss work in progress.
Finalizing Principles and Guidance Documents
Weiss started with Principles in the Use of Registries for Enhancing
Quality of Care Through Performance Measurement. He noted that the
document has been revised to clarify that the principles refer to registries
that are created for the purpose of performance measurement.
There was no discussion.
Motion: To review and endorse the Principles in the Use
of Registries for Enhancing Quality of Care through Performance Measurement.
Result: The motion was adopted unanimously.
Next, Weiss turned to the Guide for the Selection of
Performance Measures for Medical Subspecialty Care. The purpose of
the document is to refine the general guidelines to reflect what should
be included for subspecialty care.
A participant opened the discussion with a comment about adding language
addressing the needs of consumers and purchasers. He also commented on guideline
4 ("Measures should consider the importance of good comprehensive care
for a condition even if only one or two of the measures will ultimately be
used for public reporting. The complete set will be useful for physician
practice quality improvement efforts."), saying that he believed language
should be added that says the set "should be important and relevant
to consumers and purchasers." He also suggested adding language that
says the "collection of measures should be feasible."
Weiss asked for the sense of the group about adding the proposed language
to each of the proposed amendments. In each case, there was consensus on
doing this (with only a few dissenting votes).
One participant asked whether the amended phrases were redundant because
a broad, general principles document already exists. Another participant
also referred to the broader principles document. Weiss said he would check
the general principles document to see whether there was existing language
to cover these points, and rework this document if necessary.
A second participant noted that he did not really care if the language was
redundant because it bears repeating that measures must be feasible and relevant
to purchasers and consumers.
One participant wondered whether the language in guideline 6 ("Measures
of appropriateness of care should be utilized, whenever possible")
should be changed to say "whenever appropriate." Another person
said he was fine with the concept, but preferred to say "whenever relevant."
Another participant stressed that it is important to make sure from the
outset to create measures that are relevant to consumers. In response, someone
else suggested that the solution is to identify for whom a measure is being
We need to create a roadmap for better health care, said one participant,
and the specialty society she represents has limited resources. She noted
that if her organization has to prioritize between a measure that will make
a big difference in quality of care for patients and a measure that will
emphasize quality to the consumers, then it will opt in favor of the measure
that physicians believe will improve quality of care.
Carolyn Clancy said that the issue of capacity across all physician societies
is one that the AQA Steering Committee has been discussing at length. She
said that while there are clearly measures that physicians recognize as important
to quality, it is important to recognize that what consumers understand is
a fast-moving target.
One participant stressed that involving consumer advocacy organizations
in the process is very important.
Another person suggested that the document under discussion and the AQA
Surgery/Procedures Workgroup Mission and Principles should be harmonized.
Referring to the latter document, he said that language should be added
to principle 4 ("The measure must address one of the IOM's six dimensions
of quality care [safe, effective, patient-centered, timely, efficient,
and equitable].") to say that the measure "collectively should
try to address all six elements."
Motion: To review and endorse the Guidelines for the
Selection of Performance Measures for Medical Subspecialty Care, as amended (to address feasibility
Result: The motion was adopted.
Next, a member of the workgroup discussed the Surgery/Procedure Workgroup
Mission and Principles, and asked for discussion on whether to include
language in principle 4 regarding "collectively try[ing] to address
all six elements."
One participant noted that the overarching principles criteria include
feasibility. Another person said that it is important to acknowledge that
there are two separate tracks that will need to be brought together at some
A third person said that while the initial steps to provide measurement
are useful, there is also pressure to produce measures that consumers and
purchasers can use. He suggested that the health plans have expertise in
this area and can help the AQA in this regard. The immediate concern is whether
the measures being proposed are ones for which there is no way that health
plans can collect the needed information (e.g., he said that it is not possible
to capture whether or not someone receives an antibiotic after surgery).
There has to be a way to move more quickly than we are moving with the current
measure set, he concluded.
One participant pointed to two critical elements that need to be considered:
the feasibility (cost) of collecting and analyzing data, and the fact that
the two tracks are not equally weighted and may or may not intersect. From
a health care point of view, said the participant, we have to see what will
be best for the consumer. He added that it was important to come to terms
with an explicit bias in favor of consumers/patients and improving health
care for them.
Another person noted that everything that comes before the AQA includes
technical specifications and said that he thought some of the concerns were,
in fact, being addressed.
We have given a lot of thought to implementation of these measures, said
one participant. He said that there will have to be work done on the measures
to add in a cost-of-care component, adding that there is a need to look at
new Category 2 codes. Another person said that it is necessary but not sufficient
that codes be created and assigned to these measures. He also raised a related
question: What is the path by which all physicians will use these codes,
and how do we ensure that all payers will use them? This, he said, stands
in the way of implementation.
One participant stressed that health care is a vital part of the entire
economy, affecting people and jobs. Business, he said, is looking to the
AQA to bring forward a set of measures to drive efficiency and quality for
purchasers and consumers of health care.
Kevin Weiss noted that the proposed language says that the measure "collectively should try
to address all six elements," not that it must. We need to
work to get where we want to be, he said, but, most importantly, we must
keep moving as quickly as possible.
Finally, a participant noted that some measures are fully specified down
to how to measure at an individual physician level with administrative data
that we could use tomorrow, and some measures will be relatively feasible
(and can be coded and reported). For others, however, it will take more time
to get physicians to use other codes, and the path to implementation will
be more difficult.
Motion: To review and endorse the Surgery/Procedure Workgroup Mission
Result: The motion was adopted.
Proposed New Quality Measures for Adoption
Kevin Weiss discussed new quality measures for:
- Dermatology (from the American Academy of Dermatology/Physician Consortium
for Performance Improvement)
- Rheumatology/clinical endocrinology (from the American College of Rheumatology/
American Academy of Clinical Endocrinology)
- Radiology/neurology (from the American College of Radiology/American
Academy of Neurology)
- Ophthalmology (from the American Academy of Ophthalmology)
- Perioperative surgery (from the American College of Surgeons, in collaboration
with the Surgical Quality Alliance)
- Orthopedic surgery (from the American Academy of Orthopedic Surgeons)
Weiss noted that some of the proposed measures are very simple and others
are far more complex. He said the differences reflected where different specialty
societies are in terms of their ability to develop measures. The differences
also reflect the recognition that there are some very real gaps in health
care quality, he said. Weiss noted that while some of the measures are basic,
they come from a sense that physicians are not getting something right and
that this is unacceptable. He added that his workgroup had made it clear
that basic measures are a good first step, but that they could not be a means
of avoiding more difficult issues. The specialty societies understand this,
A representative from the American Academy of Dermatology briefly outlined
the three proposed measures for physicians caring for patients with a current
diagnosis of melanoma or a history of cutaneous melanoma. This outline was
followed by a discussion of whether or not the measures are too simple to
One participant said he could not support the first two measures (addressing
patient history and the complete physical skin examination) because there
should be a level of basic competency. He also argued that, from an equity
point of view, there needs to be a level playing field regarding performance
Another person, however, saw the matter differently. She said that it was
tragic that the two measures were necessary, and it points to the fact that
the quality chasm is greater than the medical community has generally acknowledged.
We must look at and close that chasm to get to better quality, she argued.
She also urged participants to look at "always" events, and that
having measures means that plans and networks can look at these and know
they are being done. To presume these are being done is not fair to patients,
she said. Another participant thanked the American Academy of Dermatology
for bringing forward an issue of major concern and said there is an urgency
Another participant noted that it is important that the AQA see itself
as improving the field of medicine and pushing the envelope. On the other
hand, she said, she preferred to wait on approval until the American Academy
of Dermatology can present a more comprehensive set of measures.
One participant said that when the measures were first brought before the
workgroup she thought they were too basic. However, she said, there is a
documented gap in care, and we need to address it and improve patient care.
She added that not all measures adopted have to be used for pay for performance;
some, she said, may just reflect a basic level of care. A second participant
echoed the previous comments, adding that she thought perhaps one solution
is to label the measures as first phase, temporary, or provisional.
This would signal that these are only a first step, she said, and send a
message that this measure set is not commensurate with the other sets of
measures that the ACA has endorsed.
There also was discussion about the language "any physician," and
whether the measures are meant for dermatologists or physicians in general.
The answer is dermatologists. A question was raised about whether these measures
have been coded (yes), and about to whom they apply. The answer is that the
measures apply to patients seen within a practice within a calendar year.
Wrapping up the discussion, Weiss noted that the performance measurement
workgroup had held pretty much the same discussion. Ultimately, the workgroup
voted with only one abstention to move the measures forward, he said, recognizing
that they are very basic and that if we close this gap then we have done
well for improving care.
Motion: To approve the proposed dermatology measures.
Result: The motion was adopted with a handful of dissenting votes and a few abstentions.
Weiss noted that he recognized that many voting yes were doing so on the
understanding that these are very basic measures—and that the American
Academy of Dermatology needs to move forward rapidly with additional measures.
Next, Weiss turned to the measures for rheumatology/endocrinology. The five
measures are intended for physicians who are (1) treating patients aged 50
years and older with a hip, spine, or radial fracture or (2) managing the
ongoing care of patients with a diagnosis of osteoporosis. There was no discussion.
Motion: To approve proposed rheumatology measures 2-5.
Result: The motion was adopted with one dissenting vote and one abstention.
The neurology measures (1-6) are designed for any physician caring for patients
with a diagnosis of stroke or transient ischemic attack in the hospital setting.
The radiology measures (7-8) are designed for radiologists and other physicians
reading the imaging studies of patients with a diagnosis of stroke or transient
ischemic attack in the hospital setting.
There was a question about coding. The answer was that this has not happened
yet but is in the works. Another participant noted that coding specifications
were sent out during the public comment period. There was also a comment
about the "any physician" language, seeking clarification that
it refers to neurologists/radiologists and not to all physicians.
Motion: To approve proposed neurology measures 1-6.
Result: The motion was adopted.
Motion: To approve the proposed measurement set for radiologists (measures
Result: The motion was adopted.
The eight measures proposed by the American Academy of Ophthalmology are
intended for ophthalmologists caring for patients age 18 years and older
with primary open-angle glaucoma, age-related macular degeneration, cataracts,
and diabetic retinopathy.
Echoing the earlier comments on the dermatology standards, there was again
a discussion about whether some of the measures are too basic. Like the earlier
discussion, the comments revolved around whether the bar was set too low
or whether it was important to address core competencies and documented gaps
in care, no matter how basic. A couple of participants suggested that perhaps
these measures should be labeled a starter set, and one participant summed
up the discussion by noting that there is a large quantity of data suggesting
that collecting data in and of itself improves performance. Thus even the
most basic standards are important, he said, and he supports the proposed
In concluding the discussion, Weiss proposed that a short synopsis of the
discussion be placed on the AQA Web site along with these measures in advance
of a more formal AQA discussion on the matter.
Motion: To approve the proposed measurement set for ophthalmology.
Result: The motion was adopted.
These measures apply to physicians caring for patients undergoing a surgical
procedure, as specified in each measure.
There was a suggestion to extend the use of prophylactic antibiotics to
children. The decision was no, because the use in children was quite different
and needed a separate guideline.
Motion: To approve the proposed starter set for surgery.
Result: The motion was adopted.
The proposed measurement set applies to all physicians treating patients
age 50 years and older with a hip, spine or radial fracture or managing the
ongoing care of patients diagnosed with osteoporosis.
Motion: To approve orthopedic surgery measures 1 and 5.
Result: The motion was adopted.
CAHPS® Clinical and Group Survey
Next, Weiss discussed the proposed CAHPS® Clinical and Group Survey.
He noted that the first step was to approve the measure as a whole. The next
step, he said, was to figure out how to translate the items into composite
There was a question about the status of the assessment tool in the National
Quality Forum (NQF). Weiss replied that a final recommendation from the NQF
Steering Committee was due the following week, after which the measure would
be put out for public comment.
There was also a question from a participant about how the CAHPS® Clinical
and Group Survey was being presented to this forum. Was it being presented
as a consumer, experience-of-care transparency tool? In response, Weiss
noted that it was meant for implementation in consumer and physician reporting
Motion: To adopt the CAHPS® ambulatory composite
measures that can be used with the survey.
Result: The measure was adopted.
Turning to the candidate list of 20 conditions as a starter set for cost-of-care
measures work, Weiss started with a brief review of the process to date.
He noted that much of the urgency on quality has come from the recognition
that there is not always an efficient use of care. Thus, he said, the AQA
came to the conclusion that it has to look at both efficiency measures and
The cost-of-care subgroup has looked at:
- Drilling down a framework on cost-of-care measures.
- Actionable moments in the physician community to address cost of care.
Referring to the 20-member candidate list of conditions, Weiss said that
20 was not a magical number, but rather that the subgroup drew a line at
a point where at least half of the people participating felt that these items
were of the highest priority.
Weiss stressed that the conditions are not measures. Rather, he said, they
are procedures for which AQA will try to press for application to measure
development. Weiss added that in compiling the list the subgroup looked at
whether this set of procedures reflects both the human lifespan and a broad
range of physical issues.
Opening the discussion, one participant expressed concern that the AQA was
biting off too much at once and would be better off with a smaller group
of conditions that could be moved forward more quickly. He also expressed
concern that some conditions had practice guidelines of varying qualities.
A second participant expressed concern about number 19 (spine: lumbar), suggesting
that it is too broad.
One participant said that he was concerned about where the discussion was
headed. He noted that the real goal is to find the true cost-of-care drivers,
and noted that it does not necessarily have anything to do with the efficiency
with which one person manages an episode. Instead, it is about the number
of episodes. Another participant expressed concern that several of the conditions
and procedures on the list (including gastroesophageal reflux disease, hip
fracture, and sinusitis) did not have a quality measure that has been approved
by NQF, AQA, or any other organization. She urged that they be removed from
the list until quality measures have been approved.
One person said he was concerned about having a patient-centered denominator
rather than a physician-centered one (i.e., cost of care per patient rather
than per practice). In response, Weiss said that addressing this question
is an essential next step.
One participant expressed concern that the cost-of-care subgroup was very
small and not broadly open to stakeholders. She urged the AQA to table the
discussion until there was more discussion (and a broader discussion) at
the workgroup level.
On a separate issue, the same participant said that the list was ambitious
and could lead to a significant amount of work. It should be narrowed down
to areas where there are already robust quality measures in place, she said.
Another participant suggested adding a section that lists specialty cost
of care (e.g., dermatology cost of care), endorses some type of episode grouping
(to get at continuum of care issues), and addresses unit grouping.
Yet another participant said that dealing with the needs of patients may
involve higher costs in certain populations or at various points in order
to lower the overall costs to the system. So what is the measurement: The
cost to the system of a diabetic patient? The cost of the entire system?
He added that it is important to address the needs of the uninsured. Finally
a participant, also citing the high percentage of uninsured Americans, said
that tabling the measure was not an option.
Motion: To approve the Cost-of-Care Proposed Starter Set of Conditions
Result: The motion was adopted, with a number of people dissenting.
that the issues raised would be addressed by his workgroup.
Next, Weiss addressed the Cost-of-Care Measurement Discussion Paper.
He noted that the paper merely offered a cost-of-care framework, and said
the draft document would need to come back as a formal document. He asked
whether people would be comfortable in moving the draft document forward.
Motion: To approve the Performance Measurement Workgroup Cost-of-Care
Measurement Discussion Paper as a draft working document.
Result: The motion was adopted.
Work in Progress
Weiss noted that ad hoc workgroups in the areas of test measures, individual
v. physician group analysis, and appropriateness criteria have been established.
Wrapping up the discussion, Weiss noted that the workgroup on performance
measurement has much left to do. The agenda moving forward, he said, includes
continuing to review measures submitted by medical and surgical specialty
groups, defining and developing principles for test measures, and establishing
a subgroup on special populations.