Quality Indicator Development

FAQ questions How are the Quality Indicators codes determined?

Date Published: February 28, 2011
Date Updated: -

Overall, administrative data have the advantage of being populated by professional coders that use a common set of practices and guidelines, which brings some uniformity to the data that may be lacking in clinical data abstracted from medical records or recorded in electronic medical records (EMR) systems.

The Quality Indicators software compiles hospital inpatient administrative data that provides demographics on the patient and the provider, diagnosis codes, procedure codes, and information about the admission, payer, and discharge. The Quality Indicators software has been maintained to be backwards compatible and validly handle ICD-9-CM diagnosis and procedure codes in effect from 1994.

The Quality Indicators measures are a constant work in progress. They are continually being revised in response to new research or validation efforts, NQF recommendations, or user feedback. If you have a question regarding a coding change, then consult the change logs for the relevant Quality Indicators PQI, IQI, PSI, PDI. The change log documents all coding changes that occur. Also consider the technical specifications and examine which cases match each denominator to determine why each case was flagged.

  ▲

Do you consider do not resuscitate (DNR)?

Date Published: February 28, 2011
Date Updated: June 1, 2012

The AHRQ QI do not currently consider do not resuscitate (DNR) as either a denominator exclusion or covariate in the risk-adjustment. We are however currently evaluating three relatively recent data elements related to hospice, palliative care and DNR. First, the UB-04 data element Point of Origin added a data value of “F” (Transfer from a Hospice Facility) in January 2010. Second, an ICD-9-CM diagnosis code V49.86 (Do not resuscitate status) was added October 1, 2010. Finally, the UB-04 data element Condition Code has a data value “P1” for “a DNR order was written at the time of or within the first 24 hours of the patient’s admission to the hospital and is clearly documented in the patient’s medical record”. The availability of HCUP data for 2010 will allow us to evaluate empirically one or more of these potential data elements alone or in combination as either an exclusion or covariate. As with any other potential patient characteristic, the empirical evaluation will focus on whether the characteristic is a mediator (and therefore a covariate) or moderator (and therefore a stratification or exclusion) of the quality of care.

Although there is currently an ICD-9-CM diagnosis code (V66.7) for encounter for palliative care, the AHRQ QI do not incorporate the code in QI calculations because it does not specifically identify hospice care, can be applied at any time during a hospitalization (e.g., several weeks or months after admission), and is not yet reliably reported. See AHA Coding Clinic for ICD-9-CM, 3Q 2008, Volume 25(32):13-14: "This code may be reported for any terminally ill patient who receives palliative care, regardless as to when the decision is made. There is no time limit or minimum for the use of this code assignment." See also AHA Coding Clinic for ICD-9-CM, 1Q 1998, Volume 15(1):11: "Terms such as comfort care, end-of-life care, and hospice care are all synonymous with palliative care. These, or similar terms, need to be written in the record to support the use of code V66.7."

In order for us to consider use of the V66.7 code, the coding guidance will need to be clarified or 5th digits must be included. We encourage professional societies with interest in this code to submit proposals to clarify the guidance and/or the creation of additional, more specific codes.


  ▲

How are the codes used in the AHRQ QI?

Date Published: February 28, 2011
Date Updated: -

The denominator exclusion includes all of the codes in the numerator definition. For example, if any of the codes used in the numerator are in the principal diagnosis field, then the case is excluded from the denominator. Similarly, if any of the codes used in the numerator are in a secondary diagnosis field and are present on admission, then the case is excluded from the denominator. A patient meeting criteria for multiple measures will be included in each measure's denominator.

  • In order to qualify as a coded condition an incident must be:
    • A condition or event that is not an EXPECTED or an inherent part of the procedure.
    • Clinically significant as determined by increased morbidity or additional care (including observation and diagnostic tests).

We list codes explicitly and not imply that additional digits are included. For example, the PQIs denote diagnosis codes as 3 or 4 digits, so we do not accept codes with 5 digits. The SAS® formats have the definitive list of codes if there is some question about a particular code.

Any indicator that uses a population denominator (from US Census) should use the patient FIPS code. Otherwise there might be cases in the numerator that are not included in the denominator.

  ▲

How do I learn about MDC, DRG, and groupers?

Date Published: February 28, 2011
Date Updated: June 28, 2011
  • MDC (major diagnostic category) and DRG (diagnostic related group) crosswalk is available here http://www.cms.gov/AcuteInpatientPPS/01_overview.asp#TopOfPageDRG TopOfPageDRG and MDC are used primarily in the inclusion rules for the covariate definitions. You should run a Center for Medicare and Medicaid Services (CMS) DRG grouper on your data to get DRG and MDC prior to running the AHRQ Quality Indicators (more detail is available on the software page for SAS or WinQI.
  • The DRG used is the CMS DRG (Version 24 and before) or the MS-DRG (Version 25 and later) from the Centers for Medicare and Medicaid Services. There is no crosswalk between the APR-DRG and the CMS-DRG or MS-DRG. However, the software to calculate the CMS-DRG may be purchased from the National Technical Information Service.
  • In most cases, MDCs are a straightforward aggregation of DRGs. For both SAS and WinQI, the software will make these aggregations automatically if the MDC is not already on the dataset. However, there are some DRGs that are not unique to an MDC; in these cases the MDC may only be assigned by the grouper.
  • We use the 3M APR-DRGs (All Patient Refined DRGs) with the risk-of-mortality subclass in the risk adjustment for the IQI. We use a modified CMS DRG (collapsing with and without comorbid conditions or complications, or CC) and the AHRQ comorbidity software for the PSI. We use the modified CMS DRG (collapsing with and without comorbid conditions or complications, or CC) or AHRQ CCS software for the PDI. DRGs are not used in the PQI.
  • You cannot dummy code the DRG codes and get correct results out of the comorbidity software, because many of the comorbidities are defined in terms of DRGs.
  • “Limited license grouper” means that not all of the APR-DRGs are included: only those that are actually used in the AHRQ Quality Indicators risk adjustment. So only about 30% of the discharges would have an APR-DRG and a risk-of-mortality subclass assigned. With the limited license grouper, the remaining discharges would have an APR-DRG value of zero. If you have your own APR-DRG grouper license, you can add the APR-DRG and risk-of-mortality data element to your input data file. 3M can offer technical support on how to use their grouper product. Once these data elements have been added to your input data file, you can use the AHRQ Quality Indicators software to load your data, and in the data element mapping step, map these data elements to your input data file data element. The software will then use the data elements from your input data file rather than using the limited license grouper.

Each year, DRG codes are updated-usually with new codes. The DRG version refers to the version in effect for a particular discharge record. The main purpose of DRG Version in the software is to distinguish CMS-DRG codes from Medicare Severity Diagnosis Related Group (MS-DRG) codes. Previous versions used discharge year/discharge quarter, where anything equal to or later than October 1, 2007 was considered MS-DRG. Not all hospitals had MS-DRG codes available after 2007/4 (i.e., discharge year/discharge quarter), so we changed the entry process to match HCUP where DRG versions 25-28 (as of October 2011) expect the DRG column data to be MS-DRG codes. Anything codes less than 25 (20-24) are considered CMS-DRG codes.

  ▲

How does the Low-Mortality DRG indicator work?

Date Published: February 28, 2011
Date Updated: -

Conceptually, the rationale for the Low-Mortality DRG indicator is that the deaths that are flagged belong to DRGs with very low mortality rates (i.e., less than 0.5%). In other words, death is not an expected outcome for these DRGs. Cases that are flagged are considered never events (events that should not occur) like transfusion reaction and foreign body left in during procedure. Therefore, patients that belong to one DRG do not have a higher risk of death than patients in another DRG, and one might argue that risk adjustment is not appropriate. However, in the real world the lines between never events and potentially preventable events are not so clearly drawn. The mortality rate for some DRGs is higher than for other DRGs. In addition, users have expressed a preference for risk-adjustment and risk-adjustment was a requirement for NQF endorsement, so the indicator is in fact risk-adjusted.

  ▲

Do you use E-codes?

Date Published: February 28, 2011
Date Updated: -

Yes. A few of the indicators use E-code in the numerator, denominator or exclusion specifications. E-codes have different coding requirements than other ICD-9-CM codes, which require that a complication be coded only if it was unexpected and changed the course of care. Because national guidelines for E-codes do not require that a condition be an unexpected aspect of a procedure or disease in order to receive an additional code (i.e., an E-code), many minor and anticipated complications may be coded using these E-codes. Although several clinical panels have endorsed the concept of the indicator, in practice the types of cases identified are often not the type of complication originally envisioned by the panels during Quality Indicators measure development.

  ▲

Do you use CPT codes?

Date Published: February 28, 2011
Date Updated: -

We do not support CPT (Current Procedural Terminology) codes for use with the AHRQ Quality Indicators that use inpatient administrative data, though we are considering their use in emergency department PSIs and PQIs.

  ▲

How do I interpret the composites?

Date Published: February 28, 2011
Date Updated: June 27, 2011

The provider-level composite measure is the weighted average of the smoothed rates of a set of indicators. The other rates are not part of the composite measure. Area-level composites measures, however, are calculated in the same manner as normal indicators. The reliability-adjusted rates are the same as the smoothed rates. There is additional information in the composite user guides ( PQI, IQI, PSI, PDI.) An increase in the composite is interpreted as an increase in the observed rate of the hospital relative to the expected rate. Examining each indicator can help determine what is driving the overall rate.

  ▲

Why are PQI #02 and PQI# 09 excluded from the composite?

Date Published: February 28, 2011
Date Updated: -

Both of these indicators have discharge based denominators, rather than population denominators.

  ▲ Why isn’t 518.5 included in Postoperative Respiratory Failure?

Date Published: June 24, 2011
Date Updated: -

518.5 is a non-specific code, which includes traumatic respiratory failure, as well as respiratory distress, wet lung syndrome and idiopathic respiratory insufficiency for instance. We originally included this code in the software, but it was removed after chart review studies found it had a very high false positive rate.

However, we have since proposed changes to the ICD-9-CM system to increase the specificity of the codes and this proposal was discussed at the recent Coding and Maintenance Committee meeting (see page 30 of http://www.cdc.gov/nchs/data/icd9/TopicpacketforMarch2011_HA1.pdf). It is likely that this proposal or a very similar solution will be adopted and we will be able to further modify the indicator definition to capture more cases without sacrificing the specificity.

  ▲

How does PSI #12 Postoperative Pulmonary Embolism or Deep Vein Thrombosis Rate distinguish preoperative versus postoperative?

Date Published: September 12, 2012
Date Updated: -

We are currently considering a proposal to rename the indicator PSI #12 Perioperative Pulmonary Embolism or Deep Vein Thrombosis Rate to reflect the inclusion in the numerator of both peri-operative and post-operative events. However, the current AHRQ PSI 12 logic cannot distinguish between preoperative (but hospital-acquired) and peri- or post-operative deep vein thromboses in surgical patients. In three separate validation studies on this indicator, 11%, 11%, and 24% of the confirmed events initially presented before the index surgical procedure. Since the advent of "present on admission" coding and more specific ICD-9-CM codes for venous thromboses, these preoperative cases have become one of the leading causes of "false positive" cases of PSI 12. Of course, labeling of these cases as "false positive" is based on the current or proposed title of the indicator ("postoperative deep vein thrombosis..." or "peri-operative deep vein thrombosis...") but would not apply if the indicator were re-titled "hospital-acquired deep vein thrombosis..." as some have recommended.

The AHRQ QI team has considered, and will continue to consider, approaches to screen out these cases, such as excluding records in which the index surgical procedure is clearly delayed (as in the case described). The practical problem with these approaches is that they cannot distinguish cases in which the delay was under the control of the hospital and cases in which the delay was due to the natural progression of the patient's illness. For example, we find that many patients admitted for hip fractures and other acute orthopedic injuries do not receive definitive surgery until four or more days into their hospital stay, and they remain "uncovered" with thromboprophylaxis during this critical period. In this setting, a preoperative DVT may be considered a potentially preventable complication related to surgical delay, and would be appropriately captured by PSI 12.


  ▲

What logic was used in the development of PSI #4 Death Rate among Surgical Inpatients with Serious Treatable Complications?

Date Published: September 12, 2012
Date Updated: -

The logic of PSI #4 is detailed in the QI Development report; however, the measure focus is on the progression from complication to death (and the hospital’s ability to influence that progression); whether the hospital was responsible for causing the complication in this admission, a prior admission, or not at all is not material to that focus. Silber et al's more recent analysis (2007) also supports having as broad a denominator definition as possible, such as including conditions that were present on admission. In fact, Silber prefers an even broader denominator definition that would include every patient who dies in the hospital.

Absent any empirical evidence that restricting the denominator to conditions that arose during the same hospital stay would increase the validity of the indicator for assessing hospital quality, we have chosen to retain fidelity with the original concept of "failure to rescue", as it was developed by Silber et al. (1992) and adapted by Needleman et al. (2002). This rationale is explicitly discussed in Needleman's editorial (2007).

While the above discusses the rationale for the inclusion of conditions present on admission in the denominator, there is also a consideration of using the present on admission status in risk adjustment. At the present time, analysis and consideration is being given to the use of POA status as a covariate for a future version of the measure (that is, whether or not the condition that qualified the case for the denominator was POA).


Some seminal articles influencing the development of this measure are:
  • Silber JH, Williams SV, Krakauer H, Schwartz JS. Hospital and patient characteristics associated with death after surgery. A study of adverse occurrence and failure to rescue. Med Care 1992;30(7):615-29.
  • Needleman J, Buerhaus P, Mattke S, Stewart M, Zelevinsky K. Nurse-staffing levels and the quality of care in hospitals. N Engl J Med 2002;346(22):1715-22.
  • Needleman, J. & Buerhaus, P. (2007). Failure-to-Rescue: Comparing Definitions to Measure Quality of Care. Medical Care, 45(4).
  • Silber, J., Romano, P., Rosen, A., Wang, Y., Even-Shoshan, O., & Volpp, K. (2007). Failure-to-Rescue Comparing Definitions to Measure Quality of Care. Medical Care, 45(4).
  • Additionally supporting resources:
    • Horwitz, L., Joanne, C., Cerese, J. and Krumholz, H. (2007). Failure to Rescue: Validation of an Algorithm Using Administrative Data, Medical Care, 45(4).
    • McDonald, K., Davies, S., Geppert, J., & Romano, P. (2007). Why Rescue the Administrative Data Version of the “Failure to Rescue” Quality Indicator. Medical Care, 45(4).

  ▲