Software

FAQ questions What do I need to use your software?

Date Published: February 28, 2011
Date Updated: March 29, 2013

The Quality Indicators software is designed to read hospital administrative discharge data that generally conforms to HCUP specifications, in which each hospitalization is reported on a single record. The Quality Indicators are possible because many hospitals and health organizations collect data that have common data elements and common data values.

Each record must conform to the specifications listed in the Data Elements and Coding Conventions appearing in the Software Instruction for SAS and WinQI available on the AHRQ Quality Indicators website. With only a few exceptions, the same specifications apply to all four modules. They all require diagnosis codes and procedure codes in ICD-9-CM format, in which the codes are in character format with significant leading zeroes and trailing blanks. Other variables include sex, age, admission type, discharge status, and diagnostic related group (DRG). This last variable (DRG) generally requires a grouper program, available through Centers for Medicare and Medicaid Services (CMS) or a third-party vendor.

Once you get your data into the proper format, you will be able to calculate provider-based (i.e., a specific hospital) observed rates because the numerator and the denominator for these are both drawn from your input data. However, area-based rates are available that use Census data from the county in which the patients in your hospital reside. If this population is not meaningful for you, you may want to create your own alternate population file for more relevant denominators, as follows:

The software uses state and county FIPS codes to link the counties in which patients reside to a population file provided with the software. If your client base is drawn from a wider area and Census data are not relevant, you can construct an alternate population file, assuming it contains a comparable data structure and uses the same coding conventions. The file must contain a record for each unique combination of state, county (FIPS codes), sex, age group (5 year bands), race, and population estimates for the years 1995 through 2013. While your alternate population file must contain these same data elements and coding conventions, you can use a unique identifier other than a FIPS state or county code to represent your alternate geographic entities, so long as the combination of these two codes can be matched to the PSTCO field in your input discharge data.

The Population File (POP95T13.TXT) must replicate the following format:

FieldVariableColumn PositionFormatCodes
1State1-2Zero Filled NumericFIPS Code
2County3-5Zero Filled NumericFIPS Code
3Sex7Numeric1=Male, 2=Female
4Age Group9-10Numeric1=0-4 years
2=5-9 years
3=10-14 years
4=15-17 years
5=18-24 years
6=25-29 years
7=30-34 years
8=35-39 years
9=40-44 years
10=45-49 years
11=50-54 years
12=55-59 years
13=60-64 years
14=65-69 years
15=70-74 years
16=75-79 years
17=80-84 years
18=85+ years
5Race12Numeric1=White, 2=Black, 3=Hispanic,
4=Asian & PI,
5=Amer. Indian,
6=Other
61995 Population13-19NumericInteger Totals
71996 Population20-26Numeric
81997 Population27-33Numeric
91998 Population34-40Numeric
101999 Population41-47Numeric
112000 Population48-54Numeric
122001 Population55-61Numeric
132002 Population62-68Numeric
142003 Population69-75Numeric
152004 Population76-82Numeric
162005 Population83-89Numeric
172006 Population90-96Numeric
182007 Population97-103Numeric
192008 Population104-110Numeric
202009 Population111-117Numeric
212010 Population118-124Numeric
222011 Population125-131Numeric
232012 Population132-138Numeric
242013 Population139-145Numeric
  ▲

What are the capabilities of the software?

Date Published: February 28, 2011
Date Updated: -

The Quality Indicators Windows Application is designed to run as a single-user application. Two or more users are unable to share a database. The application is only available in a SAS® and WinQI version for a Microsoft Operating system. In general, the ease-of-use and case level analysis capabilities of the Windows software are geared toward the needs of hospitals, and the open-source flexibility of the SAS software is geared toward researchers.

We do not recommend trying to modify the programming code because the code is complex and the stratification logic is embedded throughout the program. If the user decides to modify the programming code, then we do not provide support on the modified program.

  ▲

Can I run WinQI in silent mode?

Date Published: February 28, 2011
Date Updated: -

At this time there is no way to run WinQI command line in silent mode. The internal code is attached to the screen display code. We are moving in the direction of separating the display from the code.

  ▲

How do I install the software?

Date Published: February 28, 2011
Date Updated: -

The installation instructions can be found here for SAS or WinQI.

  ▲

Why won't the software work?

Date Published: February 28, 2011
Date Updated: March 29, 2013

A few common reasons that the WinQI software may not work correctly are:

  • Did you check the SQL server?
    • If your disk volume is compressed the SQL server will not work properly. Use Windows® Internet Explorer® to look in Program Files/Microsoft® SQL Server®/AHRQQI Data to see if any files are blue instead of black. If so, follow the directions found in the WinQI software documentation.
  • Have you changed your computer's network domain since you installed it?
    • The database server name should be the name of that server/computer, so please try changing the local name to the actual server name via the program options in AHRQ Quality Indicators software. If this doesn't work, then you will need to reinstall the application. During the database install, specify the actual server/computer name for the database server.
  • Did you format the input data correctly?
    • The most frequently cited user Quality Indicators software error is not following the required format exactly.

A few common reasons that the SAS QI software may not work correctly are:

  • Are you using SAS Enterprise Guide?
    • The SAS QI software is not intended for use with the Enterprise Guide application because certain elements of the code utilize the “X” command, which does not work in Enterprise Guide. The user must have access to Base SAS (including SAS/STAT and SAS/ACCESS) in order to run the SAS QI software.
  • Did you properly modify the default path names and settings in the SAS programs?
    • Comments are included in the CONTROL programs for each module to indicate which lines must be modified by the user. All references to “C:\Pathname\” in the SAS programs must be replaced with valid system paths to folders containing the referenced files. Once the CONTROL program has been modified, the path names must be changed in all other SAS programs within the same module to properly reference the location of the CONTROL program..
  • Did you run the SAS programs in the proper order?
    • Please consult the SAS Software Instructions or the comments near the top of each CONTROL program to determine the proper run order for the SAS programs in each module. If you intend to use APR-DRG values in the IQI module, the Limited License APR-DRG Grouper must be run before running any programs in the IQI module.
  • Are you using the appropriate version of the Limited License APR-DRG Grouper (IQI only)?
    • If you have SAS installed on a 32-bit Windows operating system (e.g., Windows XP), you need to download and run the 32-bit version of the Limited License APR-DRG Grouper.
    • If you are using a 64-bit Windows operating system, you may be using 64-bit or 32-bit SAS, and the version of the Limited License APR-DRG Grouper must match your SAS installation. You can determine the version of SAS installed by selecting Start → All Programs → SAS → SAS 9.x.
      • If you see “SAS 9.x (32) (English)” in the menu that is displayed, you have 32-bit SAS installed. You will need to download and run the 32-bit version of the Limited License APR-DRG Grouper.
      • If you see “SAS 9.x (English)” in the menu that is displayed, you have 64-bit SAS installed. You will need to download and run the 64-bit version of the Limited License APR-DRG Grouper.
  • Are you using the appropriate version of the Prediction Module software?
    • Unlike the Limited License APR-DRG Grouper, the Prediction Module architecture must match that of your operating system, not your SAS installation. If you are using a 32-bit Windows operating system, you must download and install the 32-bit Prediction Module. If you have a 64-bit Windows operating system, you must install the 64-bit version of the Prediction Module.
  ▲

Can I transfer to a different computer?

Date Published: February 28, 2011
Date Updated: March 30, 2012

By saving your mapping (*.qim) file, you will be able to easily point WinQI to the same mapping file during the data load of your original data when recalling the original data within WinQI on a separate computer.

  ▲

What do the rates mean, and how are they calculated?

Date Published: February 28, 2011
Date Updated: -

The following example illustrates the calculation and interpretation of Quality Indicators rates. An average provider rate of 0.001051, as provided by the software without the scale option, means that the average rate of hospitals with at least 1 case in the denominator was 1.05 per 1,000 or 0.1%. If the standard deviation on that average provider rate is 0.50 per 1,000, then approximately 2/3 of hospitals had rates between 0.55 and 1.60 per 1,000 (i.e., Average & Standard deviation) or rates between 0.055% and 0.16%. The population rate of 1.05 means that the average rate in the reference population (i.e., all discharges in the data file) was 1.05 per 1,000. Please note that the interpretation is based on how the rate is scaled (e.g., per 1,000 or 100,000).

Risk adjustment is highly specific to each indicator. The indicators themselves are subject to extensive validation and expert panel review and are the products of an extensive process. They are designed to measure specific events (the numerator) for specific populations that are at risk (the denominator). Risk adjustment calculations and parameters used by the syntax are the product of a lengthy process that applies the syntax to a large national file of discharges and uses logistic regression analysis to calculate the risk-adjusting coefficients. The risk factors vary from indicator to indicator, as do the coefficients.

The expected rate and risk-adjusted rate are actually two separate concepts. A risk-adjusted rate is the rate the hospital would have if it had an average case mix. In other words, it holds the hospital's performance on the Quality Indicators constant and compares that to an average case mix. This is in contrast to an expected rate that holds the hospital's case mix constant and calculates the rate expected if the hospital performed at an average level. The expected rate is the rate that you would expect if your performance is the same as the national sample. It is the rate that the whole set of US hospitals would perform if they all had the same demographics and case severity as your hospital.

  • Expected value = discharge level outcome based on discharge level data (age, sex, etc.) calculated with a normative population experience
  • Risk-Adjusted value = aggregate provider (hospital) value adjusted for a normative population experience (above) and hospital.
  ▲

Can I use race in risk adjustment?

Date Published: February 28, 2011
Date Updated: -

Race is only used as an optional stratification for the user. Race does not apply to any of the criteria used to define indicator numerators or denominators, and is not used as a factor in risk adjustment. We only offer the option of stratifying our results according to the numeric values encountered in your data. If your codes do not conform to the specifications listed above, some output will be mislabeled. No calculations are affected other than stratifying results by the values in your data.

  ▲

Where do I learn about present on admission (POA)?

Date Published: February 28, 2011
Date Updated: -

This website offers a webinar (slides and transcript dated May 12 and 14, 2010) and white paper about the use of POA: http://www.qualityindicators.ahrq.gov/Resources

  • Cases may be excluded for a relevant principal diagnosis or secondary diagnosis that is POA.
  • Cases flagged as POA are assigned a weight of zero (0) and are not dropped from the denominator. When the rates are computed, the weight of zero (0) means that they are not counted in the rates.
  ▲

How do smoothed and risk-adjusted rates differ?

Date Published: February 28, 2011
Date Updated: -

Generally, the smoothed and risk-adjusted rates are very similar. If your interest is how your hospital or group of hospitals performs at a given time compared to a standardized case mix or standardized reference population, then you should use the risk-adjusted rate. If you are interested in how your facilities are most likely to do over time or in the future, you should rely on the smoothed rates.

  ▲

Why do my rates look low?

Date Published: February 28, 2011
Date Updated: -

The expected rates are calculated from the appropriate coefficients for age group, gender, and other risk factors (e.g., the IQI uses APR-DRG, risk of mortality and severity group, while the PSI uses DRG group and comorbidity). The coefficients are calculated from hospital discharges collected from the NIS. Low expected rates may be because the sample is very different from the NIS sample or truncated in some way, or because the APR-DRG and other variables generated by the limited license grouper have not been included or generated correctly by the user.

  ▲

What are the formulas used?

Date Published: February 28, 2011
Date Updated: -

Risk adjusted rates are calculated as O/E * P, or the observed rate divided by the expected rate, times the population rate. The population rate is based on the entire population, not a sub-group, so when stratification is selected that confounds variables used in risk adjustment, the syntax presents only the O/E ratios.

  ▲

How do I calculate my rate per 1,000?

Date Published: February 28, 2011
Date Updated: -

Calculate your observed rate by dividing your numerator by your denominator and multiplying the quotient by 1000. We recommend using per 100,000 for counties or states; the state denominator will obviously be larger, but the numerator can be expected to be larger as well, so that both levels of measurement will use the same scale.

  ▲

How are outliers handled?

Date Published: February 28, 2011
Date Updated: -

We provide the software to users for use with their own hospital discharge data, so the responsibility for identifying outliers in their data really lies with them, and our software does not do this automatically. Secondly, there is no standard way to identify outliers when you are dealing with what is essentially a dichotomy, at least at the individual hospital record level, and it is especially difficult when you are dealing with relatively rare events as many of our quality indicators are. For a continuous variable measured among a large sample of records, you might indicate three or four standard deviations to constitute an outlier, but it is not really appropriate for rates.

  ▲

How are confidence intervals determined?

Date Published: February 28, 2011
Date Updated: -

The rationale for the confidence intervals (CIs) is that one is using information from a past time period to inform current decisions, so the uncertainty in those decisions is reflected in the CIs. The CI for the risk adjusted rate is

Risk adjusted rate & standard error (SE) * 1.96

where

SE = (population mean/expected rate)*(1/population)*sqrt(expected rate variance).

The method we use for the CIs is David W. Hosmer, Stanly Lemeshow. Confidence interval estimates of an index of quality performance based on logistic regression models. Statistics in Medicine, Volume 14, Issue 19, pages 2161-2172 (October 1995).

The computation for the CI for the smoothed rate is located here.

  ▲

Do all indicators have confidence intervals?

Date Published: February 28, 2011
Date Updated: -

Indicators reported as counts do not have CIs. Risk-adjusted rates of zero have CIs because they are rounded to zero, while the observed rates are exactly zero and therefore don't have CIs. The measures that are risk adjusted are included in the covariate tables PQI, IQI, PSI, PDI.

  ▲

How do I know if there is a significant difference?

Date Published: February 28, 2011
Date Updated: -

If the CIs overlap, then there is no statistical difference; however, if they don't overlap, then there is significant statistical difference. If the CI is above the population rate then the outcome of interest is significantly higher than expected. However, if the CI is below the population rate then the outcome of interest is significantly lower than expected. It is up to you to determine if the statistical difference is clinically meaningful.

  ▲

Will my data work with the program?

Date Published: February 28, 2011
Date Updated: -

The software expects that the DRG or ICD-9 code on any given discharge record is valid for the fiscal year of the discharge date. The software is designed to be backwards compatible with previous fiscal year versions.

Our software ONLY accepts three common data formats: Text (comma separated values), Microsoft Access®, and Microsoft Excel®. Two key formatting issues are that each row of data represents a separate discharge record, and each column of data represents a single variable for all discharges.

  ▲

Can I use less than one year's worth of data?

Date Published: February 28, 2011
Date Updated: -

You can report one quarter of data for provider rates; the only caution is that the relatively low frequency of events means that with the shorter time interval the rates may fluctuate from quarter to quarter (more so than when reporting annual rates). But that fluctuation will be accounted for in the confidence interval (in other words, the CI for a quarter of data is wider than the CI for a year of data). However, for area rates, given the use of Census data for the denominators, which assume one year of data, it is necessary to perform a proportional adjustment.

  ▲

How do I upload my data?

Date Published: February 28, 2011
Date Updated: -

Your input data need to follow our software Data Elements and Coding Conventions described in the software documentation (SAS or WinQI). Remember that even if you are using the Nationwide Inpatient Sample (NIS), you need to format the data for use with the AHRQ Quality Indicators software.

  ▲

Can I upload multiple data files?

Date Published: February 28, 2011
Date Updated: -

You may load only one file at a time. The previously used file will be replaced by the new one you are uploading.

  ▲

How do I generate reports?

Date Published: February 28, 2011
Date Updated: March 30, 2012

SAS
To print a detailed report, set the macro %LET PRINT = 1 in the control file by looking for a banner marked “indicate if records should be printed at the end of each program”.

WinQi
When you select either the area rates or the provider rates, you will see a succession of menus to guide you through all the necessary selections and options. Both have menus to select the indicator, select date ranges (optional), select stratifiers (optional), and additional options like risk-adjusted and smoothed rates. Provider indicators also have a menu to select hospitals (optional) and composite rates (optional). If you select a single stratifier (e.g., county for area rates or hospital for provider rates), the observed denominator will be in the fourth column from the left, following the indicator (column 1), the hospital (column 2), and the observed numerator (column 3). Of course, if you stratify on two variables, say hospital and gender, then these values will fill the second and third column, with the observed numerator and denominator appearing in the fourth and fifth columns, respectively. Note that the denominator will apply to that particular combination of stratifier values, with an overall total just before the next indicator begins.

  ▲

Can I generate individual or custom reports?

Date Published: February 28, 2011
Date Updated: March 29, 2013

The quality indicators are risk-adjusted by specific variables such as age, gender, age and gender interaction (PQI, IQI, PSI and PDI), APR-DRG (IQI Only), DRG (PSI and PDI only), comorbidities (PSI and PDI only), and severity (IQI only). You can, however, stratify the risk-adjusted rates by variables such as hospital, metro area or by county, age categories, gender, race, and pay category. You could stratify by another variable such as physician identifier with the Windows application by mapping this variable to one of the custom stratifiers and then selecting it in the Provider Report Wizard strata screen. The user might be able to do this with the SAS software by treating (renaming) your physician identifier as the hospital identifier, but only for provider rates if this option is available in the data and provides useful information. There is likely to be more bias due to unobserved patient characteristics at the physician level, and the physician-level rates will be less reliable (i.e., will have more statistical noise).

  ▲

How are the output data generated?

Date Published: February 28, 2011
Date Updated: March 29, 2013

In SAS, each program's output file is used as input by the next program, e.g., PSSASP2.SAS and PSSASP3.SAS use the output file from the SAS program: PSSASP1.SAS. But the output files can also be used for special purposes by the user, since they contain the flag variables like TPPS06 that can be used for additional research or tabulations. Beginning in Version 4.3, the determination of whether a case is in the outcome of interest (TPPS06=1) or the population at risk (TPPS06=0) was complicated by the use of POA data in the P2 and P3 programs to exclude discharges that are present on admission. However, in general if the indicator flag variable contains a 1, then the case is in the numerator. If the variable contains a 0, then the case is in the denominator but not in the numerator. For provider rates the denominator equals the number of valid records, or the number with either a 0 or a 1. If the variable contains the SAS missing code ("."), then the case was excluded from the indicator because the case did not meet the inclusion criteria or met one of the exclusion criteria.

To include all original variables plus flag variables in the output, the user must edit the "KEEP" phrase in the DATA statement in the P1 SAS program. Note that the record count in the output file will not be exactly equal to the record count in the input file because cases with missing age or sex (and some other data elements) are deleted. The record count will only be the same if these data elements are not missing.

In WinQI, all of the quality indicator logic is executed at once, for indicators selected by the user, and the rates are output with the reporting wizard. The features of the reporting wizard are discussed in the WinQI software documentation.

  ▲

What is the difference between Area and Provider reports?

Date Published: February 28, 2011
Date Updated: -

The patient-level report displays results for a single record (single patient's discharge record) while the area rate report (rates with Census data in the denominator) and the provider rate report (rates with subsets of discharges in the denominator) present summary statistics on groups of discharges, depending upon the stratification you select.

  ▲

How do I interpret my reports?

Date Published: February 28, 2011
Date Updated: June 1, 2012

A newly available resource is the QI Toolkit, which can be used to help your hospital understand the QI for use in quality improvement and patient safety. The QI Toolkit section on Assessing Indicator Rates Using Trends and Benchmarks provides information on comparing and reporting on the QI

  ▲

How do I determine the Observed – denominator and numerator (and Observed rate) for the PDI and PSI indicators that use the Markov Chain Monte Carlo (MCMC) Risk Adjustment Model?

Date Published: March 29, 2013
Date Updated:

When exporting data (specifically PSI and PDI discharge records) from WinQI version 4.4 there are a few additional steps required to derive the observed numerator and denominator (and further, the observed rate) for a given indicator. To determine the Observed Numerator, please multiply the indicator value (PSI10) by the PSI10_wtdNum value and sum these by any strata of interest (e.g. Hospital ID). For the Observed Denominator, sum the values in the PSI10_weight field. These values can then be used to generate the Observed Rate by dividing the Observed Numerator / Observed Denominator (i.e. sum of (PSI10*PSI10_wtdNum) / sum of (PSI10_weight)).

Please note:

Not all PSI and PDI indicators are affected by this. The following PDI and PSI indicators are those that you should keep in mind when interpreting WinQI’s discharge output: NQI1, NQI2, NQI3, PDI1, PDI2, PDI5, PDI6, PDI8, PDI9, PDI10, PDI11, PDI12, PSI2, PSI3, PSI4, PSI6, PSI7, PSI8, PSI9, PSI10, PSI11, PSI12, PSI13, PSI14, and PSI15.

These indicators are also listed in table B1, Appendix B of the document titled Estimating Risk-Adjustment Models Incorporating Data on Present on Admission. Those indicators with an “X” in the “Measure Specifications” column use POA in their technical specifications or during flagging/exclusion.

  ▲

Does this dialog box indicate a problem during installation?

Date Published: May 16, 2013
Date Updated:

The screen shot below provides an example of the kind of dialog box that users might see during installation of the AHRQ QI software.

Sample Installation Screenshot

Click the “Close” button on the dialog box and the installation will continue. This dialog box does not indicate a problem with the installation. Different components of the QI software target different versions of the Visual C++ Redistributable Library. And the dialog box simply notifies the user that an additional version of the library is being installed. This will not cause a problem with the execution of the QI software or any other applications installed on the user’s computer.

  ▲