Pressure ulcer development is an important outcome measure in long-term care [11, 12] because it is a common, preventable, and potentially serious adverse event that may substantially affect health and survival [13-15]. The Department of Veterans Affairs currently uses its administrative databases to calculate and disseminate facility-level rates of pressure ulcer development for its long-term care facilities. Facilities with high rates are encouraged to examine their practices and implement preventive interventions. However, these incidence rates do not consider the case mix of residents of individual facilities. Thus, facilities cannot determine whether a high rate is caused by poor quality of care or by care of a frailer and more functionally impaired population.

We used Veterans Affairs administrative files to examine the effect of case-mix adjustment in comparing facility-level rates for pressure ulcers. Specifically, we 1) identified clinical and functional status variables that predict pressure ulcer development in residents of Veterans Affairs long-term care institutions; 2) evaluated the performance properties of a prediction model based on these factors; 3) used this model to measure differences in case mix among Veterans Affairs long-term care facilities; and 4) examined whether case-mix adjustment results in perceptions about the performance of individual facilities that differ from perceptions based on unadjusted rates. Our study shows the way in which existing administrative data can be used to monitor case-mix-adjusted outcomes for long-term care.

Methods

Top Methods Results Discussion Author & Article Info References

Database

We used the Department of Veterans Affairs Patient Assessment File, which was developed for case-mix-based reimbursements for long-term care on the basis of Resource Utilization Groups [16]. The Patient Assessment File contains demographic, diagnostic, and functional information on all patients receiving long-term care. Data are collected by local staff at each facility semiannually on 1 April and 1 October of each year. The form is also completed at the time of admission or transfer to one of these units. If the date of admission is within 1 week of the semiannual evaluation, only one entry is likely to be made. Assessments are guided by an instruction manual.

The Patient Assessment File records whether a patient had a pressure ulcer on the evaluation date and, if so, the stage of the largest ulcer. Pressure ulcers are staged on a scale of 1 to 5 as follows: stage 1, erythematous skin; stage 2, a superficial layer of broken or blistered skin; stage 3, involvement of subcutaneous tissues; stage 4, extension to muscle or bone; and stage 5, stage 4 ulcers for which no treatment plan is documented in the medical record. For our study, we combined stages 4 and 5.

Additional information is present in specific sections of the Patient Assessment File. The administrative data section includes sex, year of birth, whether the assessment was for a new admission or was one of the semiannual assessments, the treating facility, and the bed section. The bed section may be either nursing home or intermediate medicine; residents of the latter have various needs, including short-course rehabilitation and long-term interventions more intensive than those provided in a typical nursing home. The medical treatments section of the Patient Assessment File records whether residents are receiving specific interventions, such as oxygen therapy, respiratory care, tube feedings, care of non-pressure ulcer wounds, dialysis, chemotherapy, and radiation therapy. The medical events section identifies specific conditions present during the previous 4 weeks, including dehydration, internal bleeding, stasis ulcer, and terminal illness. The selected diagnoses section documents whether residents have multiple sclerosis, urinary tract infection, hemiplegia, or quadriplegia.

The activities of daily living section, consisting of data on eating, mobility, transfer, and toileting, has been fully described elsewhere [17]. In brief, each activity of daily living is rated on a scale of 1 to 5; 1 indicates no need for supervision or physical assistance, and 5 indicates complete dependence. For toileting, however, we reversed scores 4 (patient has bowel or bladder incontinence and is not taken to a toilet) and 5 (patient has bowel or bladder incontinence but is taken to a toilet every 2 to 4 hours), because we believed that a score of 4 indicated greater risk for the development of pressure ulcers.

Other sections of the Patient Assessment File used in our study were those on behaviors (presence of verbal disruption, physical aggression, disruptive behavior, and hallucinations) and specialized services (receipt of rehabilitation services, including physical, occupational, corrective, manual arts, and educational therapies).

Study Samples

If consecutive entries for the same person are linked within the database, patients may be followed for changes in health status. We developed two such samples for our study, one for model derivation and the other for model validation. We created the derivation sample by merging three consecutive 6-month periods from the Patient Assessment File: the periods ending 1 April 1992, 1 October 1992, and 1 April 1993. Within each period, each newly admitted or continuing care patient was eligible to contribute a case to the derivation sample only if he or she had not had a stage 2 or greater pressure ulcer at the initial evaluation and remained institutionalized as of the semiannual assessment that ended the period. Thus, cases in the sample were followed for different amounts of time, to a maximum of 6 months. For patients with eligible data in more than one of the three periods, one period was selected randomly; no patient contributed more than one case to the derivation sample.

The validation sample drew cases from the single 6-month period ending 1 October 1993. The included cases had no ulcer at initial assessment and remained institutionalized as of 1 October 1993.

Selection of Study Variables

Because stage 1 ulcers often do not advance to a more serious lesion with appropriate interventions [15], we followed the usual practice of not counting them as outcome events [18-20]. Thus, the outcome event was defined as a new stage 2 or greater pressure ulcer at the period-ending evaluation.

Potential predictors of pressure ulcer development were identified from the Patient Assessment File. We evaluated each variable in the file for evidence from the literature showing an association with pressure ulcer development or a clinically plausible association between that variable and known risk factors such as immobility, poor nutrition, and medical illness [13-15]. Only variables with such evidence were included in subsequent analyses. Because stage 1 ulcers are associated with the development of larger ulcers [21], we included the presence of stage 1 ulcers as a potential predictor. Additionally, because the clinical literature has suggested that recency of admission may be an important predictor [22-24], we captured this variable by dividing patients into four groups. We defined cases in group 1 as new admissions institutionalized for less than 2 months between their initial and follow-up assessments; group 2, for 2 to less than 4 months; and group 3, for 4 to less than 6 months. Group 4 consisted of continuing care patients who had been institutionalized for a full 6 months between assessments.

We also examined whether predictors of pressure ulcer development differ between patients in intermediate medicine and those in nursing homes and whether incontinence is a stronger predictor in patients with greater immobility. However, because interaction terms were weakly associated with pressure ulcer development, we have not included them in the analyses presented here.

Predictors of Pressure Ulcer Development

We identified predictors of pressure ulcer development from the derivation sample. We did bivariate testing, including chi-square tests for ordinal variables and t-tests for continuous variables, to identify candidate variables associated with pressure ulcers. We also examined variables with more than two levels of risk to ensure that increasing levels of that variable were associated with a greater risk for pressure ulcer development. Variables significantly (P < 0.1) associated with pressure ulcers in the derivation sample were entered into a logistic regression model to identify independent predictors (P < 0.05). Analyses were done using the Statistical Analysis System, Version 6 (SAS Institute, Cary, North Carolina).

Evaluation of Model Performance

We evaluated the performance of the logistic model first in the derivation sample and then by applying it (using the same ß coefficients) to the validation sample. Each sample was divided into deciles of increasing predicted risk for pressure ulcer development. Within each decile, expected rates of pressure ulcer development were calculated from the logistic model and were compared with the observed rates. We tested model calibration using a Hosmer-Lemeshow goodness-of-fit statistic applied to the deciles of risk to determine whether to reject the null hypothesis that the model is well calibrated [25]. We measured model discrimination—the ability to assign a higher probability of risk among patients developing an ulcer—using the area under the receiver-operating characteristic curve, that is, the c-statistic [26].

Description of Facility Case Mix

Facility case mix was defined as the expected rate of pressure ulcer development and was calculated for each long-term care facility with at least 100 patients in the validation sample. The expected rate at a facility is the average predicted probability of the development of pressure ulcers among its cases, as computed by applying the model to each patient's characteristics at the initial evaluation.

Effect of the Model on Facility Ratings

We determined observed rates of pressure ulcer development for each facility and calculated the ratio of observed to expected rates. We then calculated an adjusted facility incidence rate by multiplying this ratio by the mean of the rates observed at all facilities. This definition standardizes rates so that a facility observed to have 10% more pressure ulcers than expected on the basis of its case mix will have an adjusted rate that is 10% higher than the mean observed rate of all facilities.

We assessed the effect of the model on facility ratings in three ways. First, we looked for instances in which case-mix adjustment resulted in a different interpretation about an individual facility's performance than arose with the use of unadjusted rates. Second, we examined the risk-adjusted ranks of facilities that had the 15 highest unadjusted rates (top 20%). Finally, we used adjusted and unadjusted rates to determine which facilities would be high outliers, defined as being two standard deviations above the mean.

Results

Top Methods Results Discussion Author & Article Info References

Study Samples

The sample used to derive predictors of pressure ulcer development consisted of 31 150 residents of long-term care facilities; the validation sample consisted of 17 946 cases. We constructed the derivation sample by randomly selecting one eligible time interval from as many as three available for each patient; the validation sample used all eligible cases from a single period. As a result, some differences in resident characteristics were seen between the two study samples (Table 1).

Predictors of Pressure Ulcer Development

A new pressure ulcer developed in 1350 residents from the derivation sample (4.3%). On logistic modeling, 11 of 31 initial-state factors were significantly (P < 0.05) associated with the future development of pressure ulcers (Table 2): dependence in transferring, dependence in mobility, dependence in toileting, presence of a stasis ulcer, receipt of care for a non-pressure ulcer wound in the previous 4 weeks, presence of a stage 1 ulcer, presence of a terminal illness, institutionalization for 2 to 6 months, urinary tract infection, residence in intermediate medicine, and number of specialized services being received. Factors entered into the logistic model but not independently associated with pressure ulcer development include age, quadriplegia, multiple sclerosis, and dependence in eating.

Model Performance

Performance properties of the model are described in Table 3. For both the derivation and validation samples, expected rates of pressure ulcer development within each decile of model-predicted risk were similar to the observed rates; a 15-fold difference in risk was seen between cases in the lowest deciles and those in the highest deciles. Model calibration was confirmed with the Hosmer-Lemeshow statistic (derivation sample chi-square equals 13.0; P = 0.11; validation sample chi-square equals 11.6; P = 0.17). For model discrimination, the c-statistic was 0.75 in the derivation sample and 0.76 in the validation sample.

Facility Case Mix

Of the 143 Veterans Affairs long-term care facilities in the validation sample, we used the 74 with at least 100 patients in facility-specific analyses. The mean number of patients per facility was 196 (range, 100 to 421 patients). Facility case mix, as measured by the expected rate of pressure ulcer development, varied more than threefold between the highest and lowest facility. Expected rates by facility ranged from 1.9% to 6.3% (mean, 3.4%). Analysis of variance confirmed that these 74 facilities differed in their average level of predicted risk (P < 0.001).

Effect of the Model on Facility Ratings

Observed facility rates for pressure ulcer development in the 74 facilities ranged from 0% to 10.9% (mean ±SD, 3.3% ± 2.4%). Case-mix-adjusted rates for these facilities ranged from 0% to 13.6% (mean, 3.2% ± 2.3%).

Adjustment for case mix made a difference for some facilities. For example, one pair of facilities had nearly identical observed rates (1.8% and 1.9%), but after adjustment for case mix, the percentage of patients developing an ulcer was 0.9% at one facility and 3.0% at the other. An additional two facilities (observed rates, 5.7% and 5.0%) had case-mix-adjusted rates of 3.3% and 6.5%, respectively. The two facilities with the highest unadjusted rates (10.9% and 10.3%) looked different after adjustment (13.6% and 6.4%, respectively).

The relative rankings of facilities are also affected by case-mix adjustment. Case-mix-adjusted ranks of the facilities with the 15 highest observed rates (the top 20%) are listed in Table 4. The facilities whose unadjusted ranks were 8 and 11 ranked 17 and 32, respectively, with the use of the adjusted rates. For facility K, this corresponds to a decrease from the ninth to the sixth decile. The facility ranked 14 moved to fourth place after adjustment for case mix.

Different facilities may therefore be targeted as having quality of care problems, depending on whether unadjusted or adjusted rates are used. For example, among the five facilities identified as high outliers on the basis of an observed rate greater than 2 standard deviations above the mean (range, 8.7% to 10.9%), only two remained as outliers after adjusted rates were used. Among seven facilities that had been targeted as being in the top 10% on the basis of unadjusted rates, only five remained as outliers after case-mix adjustment.

Discussion

Top Methods Results Discussion Author & Article Info References

Concerns over the quality of long-term care are widespread [11, 12, 27]. The Institute of Medicine has endorsed the use of outcome measures for describing the performance of nursing homes [11]. Regulatory organizations may use these ratings to identify and punish facilities with worse-than-expected performance. Increased reimbursements may also be directed toward facilities meeting certain standards [27]. Nursing homes that meet quality standards use this in their marketing, and consumers may select nursing homes on the basis of such ratings [28]. Accurate information on health outcomes is also important in quality improvement efforts. Central to quality improvement is understanding sources of variability in performance and providing performance feedback to individual facilities [5]. Our study shows that adjustment for case mix can be important in reporting and interpreting facility-specific rates.

The routine use of data on pressure ulcers has been widely advocated as a measure of quality of long-term care [11, 12]. The development of a pressure ulcer is a potentially serious adverse event that harms health and reduces survival for residents of long-term care facilities [13, 14]. Many pressure ulcers are preventable, and guidelines for their prevention have been widely disseminated [15]. Furthermore, pressure ulcer development is often attributed to poor quality of care and is associated with additional adverse outcomes [29]. The Department of Veterans Affairs has endorsed the use of pressure ulcers as an outcome measure and sends information on pressure ulcer incidence rates to all facilities semiannually.

We examined the effect of case-mix adjustment in interpreting ratings of long-term care facilities. Central to case-mix adjustment is a model that can distinguish among cases whose level of risk differs. Although several tools for predicting pressure ulcers have been described [15], these instruments are based on clinical judgments about the factors and weights that may be appropriate and were not prospectively derived from observations on existing patients. Empirically derived models, more generally, can outperform judgment-based prediction rules [30]. In deriving a model, we have conducted the largest cohort study to date [19, 21, 22, 31], involving more than 30 000 residents of long-term care facilities, to identify predictors of pressure ulcer development and to quantify their effect. The resulting model performs well in cross-validated tests of discrimination and calibration. The value for the c-statistic was equal or superior to values we have obtained with other pressure ulcer prediction rules and populations [32].

With the use of this model, expected rates of pressure ulcer development at Veterans Affairs long-term care facilities varied more than threefold, from 1.9% to 6.3%. This finding shows that there are important differences among facilities in case mix and suggests that the use of unadjusted rates may be misleading. For some individual facilities, case-mix-adjusted rates provide a different perception of performance than do unadjusted rates. The selection of outliers is also affected by case-mix adjustment.

Although all facilities are always encouraged to reduce the number of pressure ulcers, comparisons based on unadjusted rates both unfairly target good facilities with complex residents and engender complacency among providers who are not doing as well as could be expected with residents at low risk. Facility-level data are only useful if they help to show how well facilities perform compared with the level at which we expect them to perform, according to a realistic standard set by peer performance with similar patients. Thus, case-mix adjustment is essential to accurately measuring and understanding variations in performance, key principles of quality improvement.

Our study provides several additional insights. We have shown that administrative databases can be used to study risk-adjusted outcomes in long-term care. Although risk-adjusted models have been used extensively with administrative data in the hospital setting [6], such use is rare in long-term care [10]. Previous studies using administrative data have relied on summary information at the facility level, such as number of patients with pressure ulcers or functional impairments; these studies did not relate characteristics of individual patients to outcomes [10]. Little risk adjustment was possible, and the prevalence, rather than the incidence, of conditions such as pressure ulcers was used to measure quality. As large data sets based on Resource Utilization Groups (used in our study) and the Minimum Data Set [33] become increasingly available in long-term care, patient-level modeling to control for case mix should become more common. Although our prediction model needs to be validated for use elsewhere, the finding that case mix is important is probably generalizable to more heterogeneous, non-Veterans Affairs settings.

Our study also identifies predictors of pressure ulcer development. Previous studies [19, 21, 22, 31] have been considerably smaller and have had less power to simultaneously consider the effect of several variables. Our results confirm the importance of dependence in transferring and mobility as major predictors. Specific diagnoses associated with immobility, such as coma, multiple sclerosis, and quadriplegia, were not independent predictors. As noted in other studies, incontinence was a weak predictor, but we could not differentiate between patients with fecal incontinence and those with urinary incontinence. For other variables, such as wound care, stasis ulcer, and urinary tract infection, the links to pressure ulcer development remain unclear and warrant further study.

Several limitations of our study should be noted. We included patients only if they resided on a long-term care unit on specific dates of the calendar year; patients who were discharged or died before an evaluation date had no period-ending assessment and thus could not be included in such an analysis. However, these patients were excluded equally from all facilities and should not have affected our results.

We calculated incidence rates by comparing cross-sectional data obtained from two time points. Because additional pressure ulcers would have developed and healed during the intervening period, our rates are underestimates. However, the same underestimating occurs throughout the database and probably would not have affected our overall study conclusions.

Accuracy of administrative data is always a concern. Facilities may have incentives to either under- or over-report pressure ulcers. Under-reporting would result in an improved report on quality of care; facilities that over-report would have higher reimbursements on the basis of case mix. Facilities could then "game" the system to achieve specific goals. Studies evaluating the accuracy of the Patient Assessment File have been limited. One study that used a Pearson correlation coefficient as a measure of agreement for two raters reported values greater than 0.90 [34]. Further studies of both reliability and validity should be done.

Characteristics of cases included in our derivation and validation samples were somewhat different. However, our model predicted risk for pressure ulcer equally well in the validation data and in the derivation data, suggesting that it can be successfully applied to similar, new databases.

Differences in observed facility rates of pressure ulcers are caused by random variation around the true underlying rate of pressure ulcers, as well as by differences in case mix or quality of care [35]. For demonstration, we have chosen a definition of outliers that may be suboptimal for the actual comparison of facilities. More complicated techniques for ranking facilities and for determining outliers should be considered when summary statistics such as rates of pressure ulcers are reported [36, 37]. Our conclusions on the importance of case mix are not altered, however, because we chose a consistent definition of outliers for comparing adjusted and raw data. Experience with hospital mortality rates shows that combining data from several periods can also reduce the effect of random variation [35]. This issue warrants further study with pressure ulcers as well as with other potential indicators of quality in long-term care.

Finally, although our predictive model performed well, additional information on other factors thought to be associated with pressure ulcer development was not available. These factors include measures of acute illness and whether adequate nutritional support was being provided [14]. Inclusion of such factors might improve model performance and change how individual facilities are ranked.

Facility ratings have been intensely studied in the hospital setting. Here, we have examined another important setting: long-term care. We found that large differences in case mix exist among Veterans Affairs long-term care facilities and that adjustment for case mix is important in interpreting facility incidence rates for pressure ulcers. These findings suggest that even preliminary judgments about facility quality should rely on adjusted rates and that the use of such rates should become part of standard practice. Finally, we have shown that administrative data can be used to examine the quality of long-term care. We recommend further applications of this approach.

Drs. Ash and Moskowitz: Section of General Internal Medicine, 720 Harrison Avenue, Suite 1108, Boston University Medical Center, Boston, MA 02118.

Mr. Halpern: Office of Quality Management, Department of Veterans Affairs, 810 Vermont Avenue, Washington, DC 20420.