The Quality Indicators software is designed to read hospital administrative discharge data that generally conforms to HCUP specifications, in which each hospitalization is reported on a single record. The Quality Indicators are possible because many hospitals and health organizations collect data that have a common data elements and common data values.
Each record must conform to the specifications listed in the Data Elements and Coding Conventions appearing in the Software Instruction for SAS and WinQI available on the AHRQ Quality Indicators website. With only a few exceptions, the same specifications apply to all four modules. They all require diagnosis codes and procedure codes in ICD-9-CM format, in which the codes are in character format with significant leading zeroes and trailing blanks. Other variables include sex, age, admission type, discharge status, and diagnostic related group (DRG). This last variable (DRG) generally requires a grouper program, available through Centers for Medicare and Medicaid Services (CMS) or a third party vendor.
Once you get your data into the proper format, you will be able to calculate provider-based (i.e., a specific hospital) observed rates because the numerator and the denominator for these are both drawn from your input data. However, area-based rates are available that use Census data from the county in which the patients in your hospital reside If this population is not meaningful for you, you may want to create your own alternate population file for more relevant denominators, as follows:
The software assembles a list of FIPS codes for the counties in which patients reside in hospitals with relevant cases , and uses the FIPS codes in both a population file we provide and in the your data. If your client base is drawn from a wider area and Census data are not relevant, then you can construct a comparable file to use, assuming it contains a comparable data structure and uses the correct coding conventions. The file must contain a record for each unique combination of state, county (FIPS codes), sex, age group (5 year bands), race, and population estimates for the years 1995 through 2012. Your alternate population file must contain these same data elements and coding conventions, but can use a code other than a FIPS code to represent your alternate geographic entities, so long as these codes can be matched to the PSTCO field in your input discharge data.
The Population File (POP95T12.TXT) must replicate the following format:
| Field | Variable | Column Position | Format | Codes |
|---|---|---|---|---|
| 1 | State | 1-2 | Zero Filled Numeric | FIPS Code |
| 2 | County | 3-5 | Zero Filled Numeric | FIPS Code |
| 3 | Sex | 7 | Numeric | 1=Male, 2=Female |
| 4 | Age Group | 9-10 | Numeric | 1=0-4 years 2=5-9 years 3=10-14 years 4=15-17 years 5=18-24 years 6=25-29 years 7=30-34 years 8=35-39 years 9=40-44 years 10=45-49 years 11=50-54 years 12=55-59 years 13=60-64 years 14=65-69 years 15=70-74 years 16=75-79 years 17=80-84 years 18=85+ years |
| 5 | Race | 12 | Numeric | 1=White, 2=Black,
3=Hispanic, 4=Asian & PI, 5=Amer. Indian, 6=Other |
| 6 | 1995 Population | 13-19 | Numeric | Integer Totals |
| 7 | 1996 Population | 20-26 | Numeric | |
| 8 | 1997 Population | 27-33 | Numeric | |
| 9 | 1998 Population | 34-40 | Numeric | |
| 10 | 1999 Population | 41-47 | Numeric | |
| 11 | 2000 Population | 48-54 | Numeric | |
| 12 | 2001 Population | 55-61 | Numeric | |
| 13 | 2002 Population | 62-68 | Numeric | |
| 14 | 2003 Population | 69-75 | Numeric | |
| 15 | 2004 Population | 76-82 | Numeric | |
| 16 | 2005 Population | 83-89 | Numeric | |
| 17 | 2006 Population | 90-96 | Numeric | |
| 18 | 2007 Population | 97-103 | Numeric | |
| 19 | 2008 Population | 104-110 | Numeric | |
| 20 | 2009 Population | 111-117 | Numeric | |
| 21 | 2010 Population | 118-124 | Numeric | |
| 22 | 2011 Population | 125-131 | Numeric | |
| 23 | 2012 Population | 132-138 | Numeric |
The Quality Indicators Windows Application is designed to run as a single-user application. Two or more users are unable to share a database. The application is only available in a SAS® and WinQI version for a Microsoft Operating system. In general, the ease-of-use and case level analysis capabilities of the Windows software are geared toward the needs of hospitals, and the open-source flexibility of the SAS software is geared toward researchers.
We do not recommend trying to modify the programming code because the code is complex and the stratification logic is embedded throughout the program. If the user decides to modify the programming code, then we do not provide support on the modified program.
▲At this time there is no way to run WinQI command line in silent mode. The internal code is attached to the screen display code. We are moving in the direction of separating the display from the code.
▲The installation instructions can be found here for SAS or WinQI.
Why won't the software work?A few common reasons that the software may not work correctly are:
By saving your mapping (*.qim) file, you will be able to easily point WinQI to the same mapping file during the data load of your original data when recalling the original data within WinQI on a separate computer.
▲The following example illustrates the calculation and interpretation of Quality Indicators rates. An average provider rate of 0.001051, as provided by the software without the scale option, means that the average rate of hospitals with at least 1 case in the denominator was 1.05 per 1,000 or 0.1%. If the standard deviation on that average provider rate is 0.50 per 1,000, then approximately 2/3 of hospitals had rates between 0.55 and 1.60 per 1,000 (i.e., Average & Standard deviation) or rates between 0.055% and 0.16%. The population rate of 1.05 means that the average rate in the reference population (i.e., all discharges in the data file) was 1.05 per 1,000. Please note that the interpretation is based on how the rate is scaled (e.g., per 1,000 or 100,000).
Risk adjustment is highly specific to each indicator. The indicators themselves are subject to extensive validation and expert panel review and are the products of an extensive process. They are designed to measure specific events (the numerator) for specific populations that are at risk (the denominator). Risk adjustment calculations and parameters used by the syntax are the product of a lengthy process that applies the syntax to a large national file of discharges and uses logistic regression analysis to calculate the risk-adjusting coefficients. The risk factors vary from indicator to indicator, as do the coefficients.
The expected rate and risk-adjusted rate are actually two separate concepts. A risk-adjusted rate is the rate the hospital would have if it had an average case mix. In other words, it holds the hospital's performance on the Quality Indicators constant and compares that to an average case mix. This is in contrast to an expected rate that holds the hospital's case mix constant and calculates the rate expected if the hospital performed at an average level. The expected rate is the rate that you would expect if your performance is the same as the national sample. It is the rate that the whole set of US hospitals would perform if they all had the same demographics and case severity as your hospital.
Race is only used as an optional stratification for the user. Race does not apply to any of the criteria used to define indicator numerators or denominators, and is not used as a factor in risk adjustment. We only offer the option of stratifying our results according to the numeric values encountered in your data. If your codes do not conform to the specifications listed above, some output will be mislabeled. No calculations are affected other than stratifying results by the values in your data.
Where do I learn about present on admission (POA)?This website offers a webinar (slides and transcript dated May 12 and 14, 2010) and white paper about the use of POA: http://www.qualityindicators.ahrq.gov/Resources
Generally, the smoothed and risk-adjusted rates are very similar. If your interest is how your hospital or group of hospitals performs at a given time compared to a standardized case mix or standardized reference population, then you should use the risk-adjusted rate. If you are interested in how your facilities are most likely to do over time or in the future, you should rely on the smoothed rates.
▲Several rates can be used for comparison:
The expected rates are calculated from the appropriate coefficients for age group, gender, and other risk factors (e.g., the IQI uses APR-DRG, risk of mortality and severity group, while the PSI uses DRG group and comorbidity). The coefficients are calculated from hospital discharges collected from the NIS. Low expected rates may be because the sample is very different from the NIS sample or truncated in some way, or because the APR-DRG and other variables generated by the limited license grouper have not been included or generated correctly by the user.
▲Risk adjusted rates are calculated as O/E * P, or the observed rate divided by the expected rate, times the population rate. The population rate is based on the entire population, not a sub-group, so when stratification is selected that confounds variables used in risk adjustment, the syntax presents only the O/E ratios.
▲Calculate your observed rate by dividing your numerator by your denominator and multiplying the quotient by 1000. We recommend using per 100,000 for counties or states; the state denominator will obviously be larger, but the numerator can be expected to be larger as well, so that both levels of measurement will use the same scale.
▲We provide the software to users for use with their own hospital discharge data, so the responsibility for identifying outliers in their data really lies with them, and our software does not do this automatically. Secondly, there is no standard way to identify outliers when you are dealing with what is essentially a dichotomy, at least at the individual hospital record level, and it is especially difficult when you are dealing with relatively rare events as many of our quality indicators are. For a continuous variable measured among a large sample of records, you might indicate three or four standard deviations to constitute an outlier, but it is not really appropriate for rates.
▲The rationale for the confidence intervals (CIs) is that one is using information from a past time period to inform current decisions, so the uncertainty in those decisions is reflected in the CIs. The CI for the risk adjusted rate is
Risk adjusted rate & standard error (SE) * 1.96
where
SE = (population mean/expected rate)*(1/population)*sqrt(expected rate variance).
The method we use for the CIs is David W. Hosmer, Stanly Lemeshow. Confidence interval estimates of an index of quality performance based on logistic regression models. Statistics in Medicine, Volume 14, Issue 19, pages 2161-2172 (October 1995).
The computation for the CI for the smoothed rate is located here.
Do all indicators have confidence intervals?Indicators reported as counts do not have CIs. Risk-adjusted rates of zero have CIs because they are rounded to zero, while the observed rates are exactly zero and therefore don't have CIs. The measures that are risk adjusted are included in the covariate tables PQI, IQI, PSI, PDI.
▲If the CIs overlap, then there is no statistical difference; however, if they don't overlap, then there is significant statistical difference. If the CI is above the population rate then the outcome of interest is significantly higher than expected. However, if the CI is below the population rate then the outcome of interest is significantly lower than expected. It is up to you to determine if the statistical difference is clinically meaningful.
▲The software expects that the DRG or ICD-9 code on any given discharge record is valid for the fiscal year of the discharge date. The software is designed to be backwards compatible with previous fiscal year versions.
Our software ONLY accepts three common data formats: Text (comma separated values), Microsoft Access®, and Microsoft Excel®. Two key formatting issues are that each row of data represents a separate discharge record, and each column of data represents a single variable for all discharges.
▲You can report one quarter of data for provider rates; the only caution is that the relatively low frequency of events means that with the shorter time interval the rates may fluctuate from quarter to quarter (more so than when reporting annual rates). But that fluctuation will be accounted for in the confidence interval (in other words, the CI for a quarter of data is wider than the CI for a year of data). However, for area rates, given the use of Census data for the denominators, which assume one year of data, it is necessary to perform a proportional adjustment.
▲Your input data need to follow our software Data Elements and Coding Conventions described in the software documentation (SAS or WinQI). Remember that even if you are using the Nationwide Inpatient Sample (NIS), you need to format the data for use with the AHRQ Quality Indicators software.
▲You may load only one file at a time. The previously used file will be replaced by the new one you are uploading.
▲SAS
To print a detailed report, set the macro %LET PRINT = 1 in the control file by looking for a banner marked “indicate if records should be printed at the end of each program”.
WinQi
When you select either the area rates or the provider rates, you will see a succession of menus to guide you through all the necessary selections and options. Both have menus to select the indicator, select date ranges (optional), select stratifiers (optional), and additional options like risk-adjusted and smoothed rates. Provider indicators also have a menu to select hospitals (optional) and composite rates (optional). If you select a single stratifier (e.g., county for area rates or hospital for provider rates), the observed denominator will be in the fourth column from the left, following the indicator (column 1), the hospital (column 2), and the observed numerator (column 3). Of course, if you stratify on two variables, say hospital and gender, then these values will fill the second and third column, with the observed numerator and denominator appearing in the fourth and fifth columns, respectively. Note that the denominator will apply to that particular combination of stratifier values, with an overall total just before the next indicator begins.
The quality indicators are risk-adjusted by specific variables such as age, gender, age and gender interaction (PQI, IQI, PSI and PDI), APR-DRG (IQI Only), DRG (PSI and PDI only), comorbidities (PSI and PDI only), and severity (IQI only). You can, however, stratify the risk-adjusted rates by variables such as hospital, race, and pay category. You could stratify by another variable such as physician identifier with the Windows application by mapping this variable to one of the custom stratifiers and then selecting it in the Provider Report Wizard strata screen. The user might be able to do this with the SAS software by treating (renaming) your physician identifier as the hospital identifier, but only for provider rates if this option is available in the data and provides useful information. There is likely to be more bias due to unobserved patient characteristics at the physician level, and the physician-level rates will be less reliable (i.e., will have more statistical noise).
▲In SAS, each program's output file is used as input by the next program, e.g., PSSASP2.SAS and PSSASP3.SAS use the output file from PSSASP1.SAS. But the output files can also be used for special purposes by the user, since they contain the flag variables like TPPS06 that can be used for additional research or tabulations. Beginning in Version 4.3, the determination of whether a case is in the outcome of interest (TPPS06=1) or the population at risk (TPPS06=0) complicated by the use of POA data in the P2 and P3 programs to exclude discharges that are present on admission. However, in general if the indicator flag variable contains a 1, then the case is in the numerator. If the variable contains a 0, then the case is in the denominator but not in the numerator. For provider rates the denominator equals the number of valid records, or the number with either a 0 or a 1. If the variable contains the SAS missing code ("."), then the cases was excluded from the indicator-the case did not meet the inclusion criteria or met one of the exclusion criteria.
To include all original variables plus flag variables in the output, the user must edit the "KEEP" phrase in the DATA statement in the P1 SAS program. Note that the record count in the output file will not be exactly equal to the record count in the input file because cases with missing age or sex (and some other data elements) are deleted. The record count will only be the same if these data elements are not missing.
In WinQI, all of the quality indicator logic is executed at once, for indicators selected by the user, and the rates are output with the reporting wizard. The features of the reporting wizard are discussed in the WinQI software documentation.
▲The patient-level report displays results for a single record (single patient's discharge record) while the area rate report (rates with Census data in the denominator) and the provider rate report (rates with subsets of discharges in the denominator) present summary statistics on groups of discharges, depending upon the stratification you select.
▲A useful interpretation concept is to consider the AHRQ Quality Indicators on a continuum from a "safety" indicator to a "quality" indicator, where the distinction is the degree of preventability. Safety indicators (which would include the PSI and safety related PDI) are on the more preventable end of the continuum, while quality indicators (which would include the PQI, IQI and quality related PDI) are on the less preventable end. For safety indicators, the relevant performance benchmark is zero regardless of whether your rate is above or below average. For quality indicators, the relevant performance benchmark is the "best" performing hospital or area, which may be above zero, depending on the inherent risk of the outcome for the particular condition or procedure. Empirically, safety indicators have more of a skewed distribution, with most hospitals having very few events and a few hospitals having more events; Quality Indicators tend to have more of a normal distribution.
We maintain and update the AHRQ Quality Indicators software, which uses common data points (generally following HCUP conventions) in hospital administrative discharge data to generate standardized measures of hospital care quality and safety. Many states and agencies have adopted the Quality Indicators for public reporting, but our user base also includes providers and researchers who use the software for various reasons. We are continuously enhancing the indicators and appreciate user feedback.
▲