The distribution of cost data is often skewed because a small percentage of patients invariably incur extremely high costs relative to most patients. The skewed distribution of cost data is often log-normal; that is, the log-transformed cost data follow a normal distribution [1-3, 5-7]. For the comparison of mean costs between two groups, three commonly used methods are the parametric Student t-test on untransformed costs, the parametric Student t-test on log-transformed costs, and the nonparametric Wilcoxon test.

These methods have some limitations in the analysis of skewed cost data. The use of parametric Student t-tests on untransformed costs is based on the assumption that cost is approximately normally distributed; this is especially important in small to moderate samples. Nonparametric Wilcoxon tests are appropriate for testing the equality of two-sample means only when the shape (and thus the variance) of the distribution of cost is the same in both groups. The use of parametric Student t-tests on log-transformed costs assumes that cost has a log-normal distribution. However, testing the equality of means in the log-scale is equivalent to testing the equality of means in the original scale only if variances in the log-scale are equal. The impact of violating these assumptions when applying these common methods to cost data has not been assessed.

To overcome these limitations, Zhou and coworkers [8] proposed a Z-score method for comparing the means of log-normally distributed costs between two groups. The purpose of this paper is to examine the effect of this approach on the results and conclusions of previously published studies that compare costs. We identified articles recently published in the medical literature and describe how they analyzed their skewed cost data. Wherever enough information was given, we reanalyzed the data using the Z-score method and observed whether the conclusions were altered. Finally, we make some recommendations about the use of the various tests.

Methods

Top Methods Results Discussion Author & Article Info References

Summary of the Z-Score Method

When comparing costs between two groups, health services and clinical researchers are often interested in whether mean costs are the same between the two groups. If we denote the mean costs of two groups as M₁ and M₂, our null hypothesis of interest is H₀: M₁ – M₂ = 0.

Assume that log-transformed costs in group 1 and group 2 are normally distributed with means micro₁ and micro₂, respectively, and variances sigma₁² and sigma₂², respectively. Because log M₁ = micro₁ + sigma₁² and log M₂ = micro₂ + sigma₂² (Appendix 1), H₀ is equivalent to the following:

H₀: (micro₂ + sigma₂²/2) –(micro₁ + sigma₁²/2) = 0.

The new test [8] is a Z-score based on an estimate of (micro₂ + sigma₂²/2) –(micro₁ + sigma₁²/2) divided by its SE; computation of the Z-score is described in Appendix 2.

The Z-score method is appropriate for testing H₀ if the cost data are log-normally distributed. It does not ignore the skewness of cost data (as the t-test on untransformed data does), nor does it require equal variances in the original scale (as the Wilcoxon test does) or in the log-scale (as the t-test in the log-scale does). Appendix 1 gives more detail about why the t-test on log-transformed costs is testing a null hypothesis, H₀ *: micro₁ – micro₂ = 0, which is different from H₀ unless sigma₁ = sigma₂.

In a simulation study, assuming that the cost data are log-normal, Zhou and colleagues [8] showed that when variances in the log-scale are unequal, the Wilcoxon test and the t-tests on both the log-scale and the original scale are all subject to incorrect type I error rates, but the Z-score method has accurate type I error rates and has adequate power. When variances in the log-scale are equal (that is, when H₀ and H₀ * are equivalent), the performance of the Z-score method is similar to that of the Wilcoxon test and the t-tests on both the log-scale and the original scale. Thus, the Z-score method is always appropriate for testing the equal mean costs of two log-normal samples, regardless of the equality or inequality of variances in the log-scale.

Identification of Articles

To study the statistical methods used in the literature, we did a MEDLINE search to identify articles published between January 1991 and January 1996 that had the MeSH heading "costs and cost analysis" (n = 2333). Narrowing the focus to "hospital costs" (n = 414), we then identified articles that were either in the "statistical and numerical" subgroup or articles for which hospital costs were the major focus (n = 146). After we eliminated articles not written in English, commentaries, letters, or editorials, 118 articles remained and were reviewed.

For each article, we recorded sample sizes, descriptive statistics, and the types of analyses conducted. We also recorded whether the article was only descriptive or inferential and whether cost data were transformed (logarithmically or in another way). If they were, we recorded whether justification for the transformation was provided. For articles in which statistical analysis was done and sufficient detail was provided, we reanalyzed the data by using the method of Zhou and colleagues [8].

Results

Top Methods Results Discussion Author & Article Info References

As the Figure 1 shows, 69 articles (58.5%) were descriptive only and did not include statistical inferences. Most of the other 49 articles described more than one statistical test, regression analysis, or both.

View larger version (18K): [in this window] [in a new window]   Figure 1. Summary of statistical methods used in published articles. The numbers for each type of analysis total more than the number of articles because several articles included more than one analysis. IN = natural logarithm.

Logarithmic transformation of cost data was more likely to be done when regression analysis was used. Of the regression analyses done on costs in 21 articles, 11 used the natural logarithm of costs as the dependent variable; 1 used the Cox semiparametric model to account for censored data; and the other 9 used cost as the dependent variable. The most frequent justification for transforming data was skewness of cost data. When t-test results, Wilcoxon test results, or CIs were presented, the analyses had been done most frequently on untransformed data. Of the 36 articles that described two-sample tests, only 4 transformed the data to the natural logarithm of cost: 3 for two-sample t-tests and 1 for a paired t-test. Untransformed data were used in the other analyses. These analyses included Wilcoxon tests to compare two groups (12 cases), two-sample t-tests (22 cases), and a paired t-test (1 case). All four one-sample (t-distribution) CIs were calculated on untransformed cost data.

Eleven articles [9-19] included two-sample tests and contained enough information to allow us to reanalyze the data using the Z-score method. These 11 articles described a total of 23 Wilcoxon tests and 24 t-tests. The Table 1 shows the P values reported in the articles and the P values derived by using the new Z-score method. In some cases, the changes are dramatic and could affect interpretation of the findings. For example, one test (test 16.9) had a P value of 0.16 reported in the article, but the Z-score method produced a P value of 0.001.

If a P value less than 0.05 indicates a statistically significant result, six results (three Wilcoxon test results and three t-test results) were changed enough on reanalysis to alter some conclusions. Specifically, one Wilcoxon test (test 17.1) showed statistically significant results in the article, but reanalysis showed nonsignificant results; two Wilcoxon tests (tests 16.6 and 16.9) showed nonsignificant results in the article, but reanalysis showed statistical significance. For those articles that used t-tests on untransformed cost data, two statistically significant results (on tests 18.1 and 19.1) became nonsignificant in reanalysis; one nonsignificant result (on test 12.2) became statistically significant.

Discussion

Top Methods Results Discussion Author & Article Info References

We used the mean to measure the central tendency of skewed cost data. Other measures, such as the median, may be more appropriate for descriptive purposes. However, the bottom line for policy-makers is total cost, which can be derived only from the mean. In our review of the literature, we found that for two-sample comparisons, most investigators used t-tests on untransformed costs or nonparametric Wilcoxon tests; some also used t-tests on log-transformed costs. All three methods have limitations. The validity of the t-test on untransformed costs requires the normality of the cost data, except for large samples. The Wilcoxon test requires equal variances in the original scale, and it is very sensitive to skewness in data [20]. Although the t-test of log-transformed costs adjusts for skewness, it tests the wrong null hypothesis unless variances in the log-scale are equal. The Z-score method [8] was specifically developed to test equality of means in two log-normal samples and explicitly adjusts for skewness in cost data. Despite the limitations of the t-test of untransformed costs and the Wilcoxon test, reanalysis done by using the Z-score method did not change the conclusions of most studies. However, it did change some results, and the direction of change cannot be predicted.

Neither the newly proposed Z-score method nor the Wilcoxon test provides an immediate CI for the difference in log-normal means. Appropriate CIs are available only for the one-sample, log-normal mean [21]. Using the Z-score method may lead to P values that differ from those produced with traditional tests. However, until methods are developed to determine whether there is also an appreciable shift in precision (CIs), the importance of the shift in P value cannot be fully interpreted.

For testing the equality of mean costs between two log-normally distributed samples, we make the following recommendation: For large samples (1000), the t-test on untransformed costs would be appropriate. For small to moderate samples, if the variances in the log-scale are equal, either the t-test on log-transformed costs or the Wilcoxon test could be used; otherwise, the Z-score method should be used.

If cost data cannot be approximated by a log-normal distribution, we recommend the use of the nonparametric bootstrap approach [8]. If it is necessary to adjust for covariates in assessing differences in the means of costs, a commonly used approach is a linear regression model on untransformed costs or log-transformed costs. Alternatively, the standard Cox semiparametric regression model has also been used to analyze cost data by treating costs as potentially right-censored failure times [22]. The validity of these regression methods for cost data needs to be studied. Finally, if cost data contain many zero costs, we cannot perform logarithms on cost data. We are working on extending our Z-score method to address this situation.

In summary, the availability of large administrative and clinical databases has facilitated studies of health care costs in the literature. The methods commonly used for cost analysis have some limitations that could affect the conclusions of some studies. The use of appropriate statistical methods are important in cost analysis; informed decisions depend on them.

Appendix 1: Why H0 and H0 * Might Not Be the Same

Let X_i be the health care cost for the ith patient in the first group, and let Y_i be the health care cost for the ith patient in the second group. Let their corresponding means be M₁ and M₂, respectively. Then, our null hypothesis of interest is whether the mean costs are equal between the two groups. This hypothesis can be expressed as H₀: M₁ = M₂. Assume that the logarithms of X_i and Y_i are normally distributed: (Equation 1)

(1)

The t-test of log-transformed cost data is actually testing the null hypothesis H₀ * that the means of log-transformed costs are equal in the two groups. This null hypothesis can be expressed as H₀ *: micro₁ = micro₂. The logarithm of the mean of cost data depends not only on the mean of log-transformed cost data but also on the variance of log-transformed cost data. This relation can be expressed by: (Equation 2) where k = 1,2. These formulas were derived from the fact that the mean of a log-normal distribution is exp(micro_k + sigma_k²/2). For a more detailed explanation, see Johnson and Kotz [23].

(2)

Thus, the original hypothesis can be written as follows: (Equation 3)

(3)

From this relation, we see that if the variances of log-transformed cost data are unequal (sigma₁ sigma₂), the null hypothesis H₀ (that logarithms of mean costs in the two groups are the same) differs from the null hypothesis H₀ * (that means of log-transformed cost in the two groups are the same). Specifically, if sigma₁ sigma₂, then even after the t-test of the log-transformed costs accepts the hypothesis H₀ * (micro₁ = micro₂), we could still reject the hypothesis H₀. If sigma₁ < sigma₂, even after the t-test of the log-transformed costs concludes that the mean of the log-transformed cost in group 1 is greater than that in group 2 (micro₁ > micro₂), we could still accept the hypothesis H₀ of equal mean costs; if sigma₁ > sigma₂, even after the t-test of the log-transformed costs concludes that micro₁ < micro₂, we could still accept the hypothesis H₀.

Appendix 2: Definition of the Z-Score Test Statistic

Let X_i be the health care cost for the ith patient in the first group with n₁ patients, and let Y_i be the health care cost for the ith patient in the second group with n₂ patients. Define (Equation 4) The Z-score test statistic proposed by Zhou and coworkers [8] is (Equation 5) The P value of the test is easily obtained by looking up a standard normal distribution table for the Z statistic.

(4)

(5)