Considerations in Public Reporting of the AHRQ QIs

 

By Dr. Shoshanna Sofaer, D.P.H.

Baruch College

 

An increasing number of organizations are issuing comparative reports to the public on hospital quality, using all or some of the AHRQ Quality Indicators. Others are planning to follow suit soon. This article identifies, based on research and practical experience, what “reporters” need to consider and know as they embark on this ambitious venture. There are four key questions to be addressed:

 
  • Who are the expected audience(s) for your report?
  • For what purpose(s) is each audience likely to use your report?
  • How do you design a report that will be understood and interpreted appropriately?
  • How do you disseminate and promote your report so that people want to find it and can in fact find it easily.

 

Who’s your audience? How might they use the report?

 

The issue of audiences and uses go together. As a public report, likely uses include increasing the overall “transparency” or “accountability” of the health care system, a particular concern of policy makers and policy advocates. Individual members of the public  may also use the report for various purposes, including seeing how “their” hospital is doing compared to others in the community, region or state; as part of a conscious “activated consumer” process of selecting a high quality facility; or as a focal point of conversations with family, friends and personal physicians. Purchasers might use these reports to identify quality problems and push for improvements or to reward high performers with higher reimbursements or better access to patients. Last but certainly not least, there is considerable evidence that an effective public report increases how much hospitals themselves use quality measurement to guide and drive quality improvement.

 

Elements of Effective Report Design

 

The bottom lines of effective report design is that members of the public should find that the report is trustworthy, addresses aspects of hospital performance that matter to them, makes it easy to see variations in performance and is easy (particularly in the context of a website) to navigate.

 

Report designers have to realize that the public at large shares neither their deep understanding of the meaning of various quality measures nor their need to know the details of how measures and scores were constructed. Reports need to make explicit what measures mean and why they are important.

 

There are a lot of AHRQ QIs. We are just beginning to learn which resonate with the public and which do not; this issue should be considered in deciding which QIs to report. Research evidence shows that when people don’t understand a measure, they don’t think it is important. So don’t just give folks the technical name of an indicator; in plain English, explain the measure, why it’s important and what it does (and does not) show about hospital performance.

 

Reports need to provide access to the technical details but avoid forcing people to master them before getting to the “beef” of the content. This relates to another important feature of effective reports – they don’t overwhelm people with too much data. Evidence from cognitive science shows people can keep only five ideas, plus or minus two, in their short-term memory at one time. So don’t set up your report so people have to slog through the scores of 20 hospitals on 30 measures on a single page or “layer” of your report.

 

How you decide to “score” your findings is both a technical and often a political decision. But remember that if your scoring method makes everyone look alike (especially if they are really not at all alike) the public will lose interest. Just as important is to present scores so they are easy to interpret. Since one major purpose of reports is to help people make comparisons, comparative scoring has been very popular in comparative quality reports. Even if people don’t see all the nuances of why one hospital gets one star and another gets three, they do “get” that three stars means better performance. Recent evidence indicates that “word icons,” i.e. actually putting the word “Average” or “Better than Average” into a chart, results in more people accurately interpreting results. Bar graphs are effective to show “absolute” scores, which are often critical to providing meaningful “anchors” for comparative ratings. Finally, how you order the ratings can be key. Rank ordering of ratings has been demonstrated to be more “evaluable” and also a very effective “spur” to quality improvement efforts.

 

Disseminating and Promoting Comparative Quality Reports

 

If people don’t know a report is available, they won’t find it. If they don’t find it, they can’t look at it; if they don’t look at it, they won’t use it. Think about issues of dissemination and promotion from Day One and recognize that resources will be needed to promote the availability of the report. The field of social marketing has brought great practical wisdom to efforts to change health behaviors. It can and should also be used in efforts to encourage the public to look at and use comparative quality reports.

 

Where to learn more

 

The full set of slides on which this article is based can be accessed at http://www.qualityindicators.ahrq.gov/usermeeting_presentations_2005.htm. Sponsors of comparative health care quality reports can also take advantage of AHRQ’s “Talking Quality” website (http://www.talkingquality.gov), a “soup to nuts” resource. A brand new resource from the California Health Care Foundation is “Consumers in Health Care: The Burden of Choice” which distills the latest research on consumer decision-making, explores the methods consumers use to make choices, and looks at what influences affect consumer action. This report by Shaller Associates can be found at www.chcf.org.

 

Supplement to AHRQ Quality Indicators Newsletter, Volume 1, Number 3, December 2005.