The Effect of Including C-Reactive Protein in Cardiovascular Risk Prediction Models for Women

  1. Nancy R. Cook, ScD;
  2. Julie E. Buring, ScD; and
  3. Paul M Ridker, MD
  1. From Brigham and Women's Hospital, Harvard Medical School, and Harvard School of Public Health, Boston, Massachusetts.

    Abstract

    Background: While high-sensitivity C-reactive protein (hsCRP) is an independent predictor of cardiovascular risk, global risk prediction models incorporating hsCRP have not been developed for clinical use.

    Objective: To develop and compare global cardiovascular risk prediction models with and without hsCRP.

    Design: Observational cohort study.

    Setting: U.S. female health professionals.

    Participants: Initially healthy nondiabetic women age 45 years and older participating in the Women's Health Study and followed an average of 10 years.

    Measurements: Incident cardiovascular events (myocardial infarction, stroke, coronary revascularization, and cardiovascular death).

    Results: High-sensitivity CRP made a relative contribution to global risk at least as large as that provided by total, high-density lipoprotein (HDL), and low-density lipoprotein (LDL) cholesterol individually, but less than that provided by age, smoking, and blood pressure. All global measures of fit improved when hsCRP was included, with likelihood-based measures demonstrating strong preference for models that include hsCRP. With use of 10-year risk categories of 0% to less than 5%, 5% to less than 10%, 10% to less than 20%, and 20% or greater, risk prediction was more accurate in models that included hsCRP, particularly for risk between 5% and 20%. Among women initially classified with risks of 5% to less than 10% and 10% to less than 20% according to the Adult Treatment Panel III covariables, 21% and 19%, respectively, were reclassified into more accurate risk categories. Although addition of hsCRP had minimal effect on the c-statistic (a measure of model discrimination) once age, smoking, and blood pressure were accounted for, the effect was nonetheless greater than that of total, LDL, or HDL cholesterol, suggesting that the c-statistic may be insensitive in evaluating risk prediction models.

    Limitations: Data were available only for women.

    Conclusions: A global risk prediction model that includes hsCRP improves cardiovascular risk classification in women, particularly among those with a 10-year risk of 5% to 20%. In models that include age, blood pressure, and smoking status, hsCRP improves prediction at least as much as do lipid measures.

    Editors' Notes

    Context

    • The value of adding high-sensitivity C-reactive protein (hsCRP) to a global risk assessment model is unknown.

    Contribution

    • The authors used the Women's Health Study, a nationwide cohort of 15048 initially healthy women, to develop a cardiovascular disease (CVD) risk prediction model using hsCRP and Framingham risk model predictors. While hsCRP improved overall model fit, the clinical utility of hsCRP in terms of reclassification was most substantial for those with a 5% or greater 10-year risk based on traditional risk factors.

    Cautions

    • The study does not address the clinical value of lowering hsCRP level.

    Implications

    • In this largely low-risk population, adding hsCRP to the Framingham model reclassified patients into groups that better reflected their actual CVD risk. This effect was most clinically relevant for those at intermediate risk.

    —The Editors

    The Framingham risk model (1) is used extensively for detecting risk for coronary heart disease and has been adapted by the Adult Treatment Panel III (ATP III) of the National Cholesterol Education Program (2). The traditional risk factors included are strong predictors of cardiovascular risk, and the model has been validated in several populations (3). However, despite the model's success, up to 20% of all coronary events occur in the absence of any major risk factor (4, 5). In addition, most individuals who do not develop coronary heart disease have at least 1 clinically elevated Framingham risk factor (6).

    Given these modest levels of sensitivity and specificity, research over the past decade has focused on novel blood-based atherosclerotic risk factors that, like cholesterol, can be inexpensively obtained and interpreted in the primary care setting. One of the most promising of these is high-sensitivity C-reactive protein (hsCRP), a biomarker of inflammation that has consistently been shown to predict incident myocardial infarction, stroke, and cardiovascular death among apparently healthy men and women after adjustment for all components of the Framingham risk score (7-16). Blood levels of hsCRP also correlate with hypofibrinolysis and abnormal glucose metabolism, and thus reflect pathophysiologic processes that are related to vascular occlusion but are not easily measured with traditional risk factors (17-20). On that basis, in 2003 the Centers for Disease Control and Prevention and the American Heart Association published the first set of guidelines to endorse the use of hsCRP as a potential adjunct to traditional risk factor screening (21).

    Despite these data, no simple clinical algorithm that includes Framingham covariables and hsCRP has been developed, and thus it has not been possible to determine whether individuals might be more accurately classified if hsCRP were added to global risk prediction models for major cardiovascular disease (CVD), including myocardial infarction, coronary revascularization, stroke, and cardiovascular death (22).

    Methods

    We compared the clinical utility of global cardiovascular risk prediction models based on Framingham covariables with and without hsCRP among participants in the Women's Health Study (WHS) (23-25), a large-scale, nationwide cohort of U.S. women age 45 years and older who were free of CVD and cancer at study entry. Women were followed annually for the development of CVD, with an average follow-up of 10 years. All reported CVD outcomes, including myocardial infarction, ischemic stroke, coronary revascularization procedures, and deaths from cardiovascular causes, were adjudicated by an end points committee after medical record review. All study participants provided written informed consent, and the study protocol was approved by the institutional review board of Brigham and Women's Hospital in Boston, Massachusetts.

    Baseline blood samples were assayed for C-reactive protein with a validated, high-sensitivity assay (Denka Seiken, Tokyo, Japan) and for total, high-density lipoprotein (HDL), and low-density lipoprotein (LDL) cholesterol with direct-measurement assays (Roche Diagnostics, Basel, Switzerland). Women who were diabetic at baseline were excluded from predictive modeling because ATP III labeling considers diabetes to be a risk equivalent for coronary heart disease (2). In parallel with guidelines established for lipid evaluation (26), models were initially fitted in a derivation cohort limited to women not taking hormone replacement therapy at baseline (n = 15048 with data on all variables) and were then applied to all nondiabetic women (n = 26927) for clinical risk prediction.

    Development of Risk Prediction Models

    Models were fitted by using Cox proportional hazards models (27), restricting predictors to components of the Framingham risk score (including either total or LDL cholesterol as well as HDL cholesterol), and adding hsCRP. To determine the functional form used for each predictor, we examined spline plots and fitted power functions to determine the best fit for each variable. When fit was similar, we chose the simplest form, usually a linear term or log transformation, especially when supported by previous data from existing ATP III algorithms.

    To model blood pressure more fully, we included a nonlinear term for systolic blood pressure. As in the Framingham score, we included antihypertensive medication use and considered its interaction with blood pressure. At baseline in the WHS, use of cholesterol-lowering medications was rare, was not composed of statins, and was not statistically significant in these models. As also in the Framingham score, current but not past smoking was included. We considered interactions of all predictors with smoking and age, particularly on the basis of inclusion in the ATP III risk score. To enhance model simplicity, these were not included when they were only of marginal statistical significance. Of interest, when body mass index was added to the final model, it was not a statistically significant predictor of CVD in these data, an observation consistent with its absence from the Framingham risk score.

    In addition to comparing the performance of the final model derived from the WHS data with hsCRP to the WHS model without hsCRP, we also compared the performance of the formal ATP III model (2), with and without hsCRP (Appendix). The ATP III model includes terms for the natural logarithms of age, of total and HDL cholesterol, and of systolic blood pressure and was developed for the prediction of “hard” coronary heart disease events, including myocardial infarction and death from coronary heart disease. Total CVD, including ischemic stroke as well as revascularization procedures, was used as the end point in these analyses. To be conservative and allow the best possible fit for all traditional covariables, we recalculated β-coefficients for the ATP III model in the WHS data before evaluating any additive effects of hsCRP.

    Measures of Model Fit

    The primary means of comparing predictive models based on Framingham covariables with and without hsCRP was the Bayes information criterion, a likelihood-based measure that adds a penalty for model complexity (28, 29). Lower values indicate better fit. Because of its common use in the medical literature, we also computed the c-index (28), or concordance probability, which is a generalization of the c-statistic, or the area under the receiver-operating characteristic curve (30), that allows for censored data. For these analyses, the c-index was computed and adjusted for optimism due to overfitting with bootstrap sampling (31) using the Design library in S-PLUS software (Insightful Corp., Seattle, Washington) (28).

    For comparison, we also computed several other measures of global model fit (provided in the Appendix), including other likelihood-based measures such as model weights for the Bayes information criterion, which provide an estimate of the posterior probability of each model given the set of candidate models considered (29, 32); the Akaike information criterion and its corresponding model weights (32); and Nagelkerke's generalized model R2(33, 34). We computed the D-statistic of Royston and Sauerbrei (35), based on the separation of survival curves by predictor variables, and again adjusted for optimism. Differences in statistics between nested models were tested with a 1-sided test using bootstrap sampling (31). We also calculated the Brier score (28), which directly compares the observed outcomes with the fitted probabilities.

    To assess model calibration, or how closely the predicted probabilities reflect actual risk, observed risk was calculated on the basis of 8 years of follow-up (available for all participants) and was extrapolated to 10 years for display purposes. We computed the Hosmer–Lemeshow calibration statistic (36) comparing observed and predicted risk using 10 categories based on 2–percentage point increments in predicted risk, ranging from less than 2% to 18% or greater. We also computed this statistic using decile categories of predicted probabilities.

    To address clinical utility, we directly compared predicted risk estimates that are based on models using Framingham covariables with and without hsCRP with actual risk that was observed during study follow-up among all 26927 women for whom data were available. We used weighted κ statistics (37) to compare the predicted probabilities with and without hsCRP. To approximate clinical criteria commonly used in current treatment guidelines (2, 38), we grouped the predicted probabilities into 10-year risk categories of 0% to less than 5%, 5% to less than 10%, 10% to less than 20%, and 20% or greater.

    Finally, to address the generalizability of the final WHS risk prediction model with hsCRP, we calibrated the predicted probabilities to observed risk in the Framingham Heart Study, using a limited-access data set available from the National Heart, Lung, and Blood Institute at http://www.nhlbi.nih.gov/resources/deca/default.htm. Mean values among women age 47 years and older at the 10th Framingham examination were evaluated. Mean HDL cholesterol level at the 10th examination was estimated on the basis of age and HDL cholesterol level at the 15th examination. The population mean hsCRP value was estimated from a model derived from the WHS, including the other risk factors plus body mass index. The average 10-year risk for major CVD in the Framingham data, including myocardial infarction, stroke, and cardiovascular death, was estimated by using a product-limit estimator. The projected 10-year risk from the WHS models was calibrated to the 10-year rate of cardiovascular outcomes among women in the Framingham data (Appendix).

    Role of the Funding Sources

    This work was supported by grants from the Donald W. Reynolds Foundation, the Leducq Foundation, the Doris Duke Charitable Foundation, and the National Institutes of Health. The funding agencies had no role in the design, conduct, or reporting of the study or in the decision to submit the manuscript for publication.

    Results

    Among the 15048 women used for model development, the mean age was 54 years (SD, 8); 1841 women (12%) were current smokers. A total of 2227 women (15%) had blood pressure of 140/90 mm Hg or higher, and 1802 women (12%) were taking antihypertensive medication at baseline. Median lipid values were as follows: total cholesterol level, 5.3 mmol/L (interquartile range, 4.7 to 6.1 mmol/L) [206 mg/dL (interquartile range, 181 to 234 mg/dL)] LDL cholesterol level, 3.2 mmol/L (interquartile range, 2.6 to 3.8 mmol/L) [124 mg/dL (interquartile range, 102 to 147 mg/dL)] and HDL cholesterol level, 1.3 mmol/L (interquartile range, 1.1 to 1.5 mmol/L) [49 mg/dL (interquartile range, 42 to 59 mg/dL)]. The median hsCRP level was 1.5 mg/L (interquartile range, 0.6 to 3.4 mg/L). Over the mean 10-year follow-up, 390 women developed CVD, including 116 myocardial infarctions, 100 ischemic strokes, 217 coronary revascularization procedures, and 65 deaths due to cardiovascular cause. Some women experienced more than 1 of these events.

    The best-fitting global cardiovascular risk prediction model in these data included all Framingham covariables and hsCRP (Table 1). Figure 1 shows that after adjustment for all Framingham variables, there was a log-linear relationship between hsCRP and risk for CVD. While risk estimates based on the model coefficients given in Table 1 provide the best estimates, for clinical utility we also calculated a point scoring system (39) for cardiovascular risk analogous to that used by the National Cholesterol Education Program (Figure 2). The absolute risk estimates in Figure 2 are calibrated to the overall incidence in the Framingham Heart Study sample to enhance generalizability.

    Table 1. Best-Fitting Global Cardiovascular Risk Prediction Model among the Model Derivation Cohort of 15048 Women from the Women's Health Study
    Figure 1.   Risk estimates are provided on a natural log scale and were derived from a Cox regression model using a flexible spline curve. Dotted lines represent 95% CIs.
    View larger version:
      Figure 1.   Risk estimates are provided on a natural log scale and were derived from a Cox regression model using a flexible spline curve. Dotted lines represent 95% CIs. Relative risk (RR) of future cardiovascular events according to baseline high-sensitivity C-reactive protein (hsCRP) levels in the model derivation cohort (n= 15048), adjusted for Framingham covariables.
      Figure 2. This scoring system is intended as an illustration only. CVD = cardiovascular disease; HDL = high-density lipoprotein; RR = relative risk; SBP = systolic blood pressure. To convert cholesterol values to mmol/L, multiply by 0.02586.
      View larger version:
        Figure 2. This scoring system is intended as an illustration only. CVD = cardiovascular disease; HDL = high-density lipoprotein; RR = relative risk; SBP = systolic blood pressure. To convert cholesterol values to mmol/L, multiply by 0.02586. Cardiovascular point scoring system for women based on Framingham covariables and high-sensitivity C-reactive protein (hsCRP).

        Relative Contributions to Global Risk of Age, Blood Pressure, Smoking, Lipids, and hsCRP

        Table 2 presents a comparison of the relative contributions to global risk made by each individual Framingham covariate and by hsCRP. Age is the strongest predictor of risk in these data, leading to a high likelihood ratio and a c-index of 0.73. After adjustment for age, likelihood ratio statistics demonstrated the strongest improvement in fit for systolic blood pressure, followed in descending order by hsCRP; current smoking; and HDL, total, and LDL cholesterol. In models that included age, systolic blood pressure, and current smoking (the 3 strongest predictors in aggregate), likelihood ratio statistics similarly demonstrated the strongest fit for hsCRP, followed in order by HDL, total, and LDL cholesterol.

        Table 2. Relative Contribution of Individual Framingham Covariables and High-Sensitivity C-Reactive Protein to Global Cardiovascular Risk

        Table 2 also demonstrates the relative insensitivity of the c-index as a method of determining model fit. For example, while the likelihood ratio statistic was able to rank the relative contributions of systolic blood pressure, hsCRP, smoking, and HDL cholesterol in the age-adjusted analyses, the c-index had little ability to distinguish among any of these (all values between 0.75 and 0.77). Similarly, in the analyses adjusted for age, systolic blood pressure, and smoking, the likelihood ratio statistic indicated that either hsCRP or HDL cholesterol improved fit more than did total or LDL cholesterol, yet the c-index was again unable to distinguish between these measures (all values 0.80).

        Discrimination and Calibration in Models with and without hsCRP

        For both the ATP III model and the final WHS model, the likelihood ratio test for the inclusion of hsCRP was highly significant (P< 0.001). Of note, the Bayes information criterion indicated a strong preference for the inclusion of hsCRP (Table 1) after adjustment for adding a variable. This suggests that the model including hsCRP provided better fit, even after adjustment for the increase in number of predictors. Similarly, models that included hsCRP demonstrated better calibration (higher P value for calibration), while models without hsCRP had larger deviations between the observed and predicted probabilities in the higher-risk categories (Figure 3). By contrast, the c-index again showed minimal ability to detect differences in model fit. As shown in the Appendix Table, all global measures of fit showed improvement when hsCRP was added to prediction models based on Framingham covariables alone.

        Figure 3. The model that includes hsCRP shows closer agreement between observed and model-based predicted risk. WHS = Women's Health Study.
        View larger version:
          Figure 3. The model that includes hsCRP shows closer agreement between observed and model-based predicted risk. WHS = Women's Health Study. Calibration curves for risk prediction models without (top) and with (bottom) high-sensitivity C-reactive protein (hsCRP) in the model.
          Appendix Table. Comparison of Discrimination and Calibration for Global Risk Prediction Models with and without High-Sensitivity C-Reactive Protein

          Clinical Risk Classification and Accuracy

          To better compare model performance within clinical categories, we classified all nondiabetic women (n = 26927) into 4 risk groups defined by the ATP III categories of 10-year risk for CVD of 0% to less than 5%, 5% to less than 10%, 10% to less than 20%, and 20% or greater. We then compared the WHS models with and without hsCRP by cross-classifying expected risks and comparing these to the observed proportions of events in each group. While there was general agreement between these classifications (weighted κ= 0.86), the predicted risk categories changed substantially with the addition of hsCRP for women with at least a 5% 10-year risk according to only the Framingham risk variables (Table 3). Specifically, more than 20% of all participants with intermediate risk were reclassified with the addition of hsCRP; among those originally classified as having 5% to less than 10% risk, 12% moved down a category in risk and 10% moved up. Among those originally classified as having 10% to less than 20% risk, 19% were reclassified: 14% to a lower and 5% to a higher category. Among those at high risk (≥20% risk), 14% were reclassified into a lower-risk category. By contrast, among those with less than 5% risk according to Framingham covariables, only 2% were reclassified. Thus, overall in this low-risk cohort, with 88% of women in the lowest risk group, 4% were reclassified. However, this overall percentage depends heavily on the underlying risk and would be substantially greater in an older or higher-risk population. For comparison, according to ATP III risk categories based on National Cholesterol Education Program β-coefficients rather than those derived from the WHS, among women originally classified as having less than 5%, 5% to less than 10%, 10% to less than 20%, and 20% or greater 10-year risk, 4%, 38%, 42%, and 20%, respectively, were reclassified in the WHS model that included hsCRP.

          Table 3. Observed and Expected Risks among all 26927 Nondiabetic Women in the Women's Health Study Using the Final Global Risk Prediction Model with and without High-Sensitivity C-Reactive Protein

          Of note, for women reclassified, the models that included hsCRP also estimated the actual risk more accurately. For example, among those classified in the 5% to less than 10% category according to the model without hsCRP, the 12% reclassified to the lower-risk category actually experienced only a 2% risk, and the 10% reclassified to a higher-risk category actually experienced a higher 15% risk (Table 3). The addition of hsCRP also led to better calibration for women initially classified into the other 3 groups.

          Discussion

          The Framingham risk score provides a useful measure of risk stratification for coronary heart disease and has been valuable in clinical practice. Whether it can be improved by including other simple and inexpensive measures has not previously been determined. We fitted predictive models for major CVD, including myocardial infarction, coronary revascularization, stroke, and cardiovascular death, using traditional Framingham predictors, with and without the addition of hsCRP. As shown, all global measures of predictive accuracy were improved in prediction models that included hsCRP. In particular, likelihood-based measures showed a strong preference for the models with hsCRP, and the Bayes information criterion weights, corresponding to the posterior model probabilities, strongly supported the addition of hsCRP. As also shown in these data, the relative contribution to global risk made by hsCRP was at least as large as that made by total, HDL, or LDL cholesterol. The added predictive value of including hsCRP was most evident among those at 5% or greater 10-year risk.

          While all the likelihood-based measures of fit improved when hsCRP was added, the c-index, or generalized c-statistic, changed little with the addition of any blood-based risk factor (including total, HDL, and LDL cholesterol) once we accounted for age, smoking, and blood pressure. In fact, the increase in the c-index associated with hsCRP was greater than that for any of the lipids. This observation underscores the limitations of the c-index, or area under the receiver-operating characteristic curve, as a method for determining model fit, despite its continued popular use in the medical literature. The c-index is particularly suited to retrospective case–control studies, in which the actual outcome probabilities cannot be estimated (40). Since it is based exclusively on ranks, however, it measures only how well the predicted values can rank-order the responses. It may not be as sensitive as the likelihood function in choosing between models, especially when the models are strong (28). This may be particularly true in settings where many individuals fall into low-risk groups, as is true of cardiovascular risk detection in the general population as well as in the WHS. In the current data, the c-index could not distinguish between blood pressure, smoking, or any of the measured blood predictors, even though the likelihood ratio statistics clearly ordered variable contributions, with systolic blood pressure being the strongest predictor after age. In fact, hsCRP was a stronger individual predictor than any of the lipid measures in these data, including HDL cholesterol, its closest competitor. Thus, reliance solely on the c-statistic for model development could erroneously lead to the exclusion of lipids as well as hsCRP from risk prediction models.

          Accuracy, or the predictive ability of a model, has 2 major components, discrimination and calibration (28). The c-statistic is a measure of discrimination, or the ability to separate 2 groups, such as case-patients and controls. Particularly in a prospective study, calibration, or how well the predicted probabilities reflect actual risk, is another aspect of accuracy not captured by the c-statistic. A model could discriminate well but lack even internal calibration if the fitted scores do not reflect the true probability of an event. The predicted probability given the risk factors, or the post-test probability, can be more useful clinically in assessing future cardiovascular risk than sensitivity or specificity, on which the c-statistic is based. Put another way, as Moons and Harrell have stated (41), sensitivity and specificity have no direct diagnostic meaning; for the patient, the issue is not the risk for having a positive test result, but the risk for developing the disease. In our data, the model that included hsCRP was better calibrated and was a better predictor of the probability of disease.

          Clinically, the inclusion of hsCRP in global prediction models more accurately predicted true cardiovascular risk in these data. While it did not strongly affect estimated risk among women originally at very low risk, hsCRP had a substantial effect among women at 5% or higher 10-year risk. Among those classified as having 5% to less than 20% risk, the addition of hsCRP reclassified about 20% of women into more accurate risk strata. This would suggest that an effective clinical strategy to improve risk prediction might be to evaluate hsCRP among women with at least 5% predicted risk based on traditional risk factors.

          One of the primary goals of any risk prediction algorithm for CVD is to identify individuals at increased risk who will benefit from aggressive lifestyle changes, including dietary moderation, exercise, and smoking cessation, all of which reduce hsCRP levels in addition to decreasing cardiovascular risk. Thus, knowledge of hsCRP level and more precise global risk estimation may help motivate patients to improve adherence to therapeutic lifestyle changes, as currently advocated in the ATP III guidelines (2). Ten-year risks are also commonly estimated to make therapeutic decisions with regard to lipid-lowering therapy, particularly with statin agents. Previous research demonstrates that statin agents decrease hsCRP levels in a manner largely independent of LDL cholesterol, and the efficacy of statin therapy is linked in part to underlying hsCRP levels (42-46). As shown in our data, approximately 20% of individuals at “intermediate risk” had their risk estimates substantively increased or decreased with improved accuracy when hsCRP was added to the Framingham covariables. Thus, the use of an hsCRP-modified Framingham risk score also has the potential to help more patients and physicians to better direct the use of preventive statin therapy to appropriate risk groups.

          Limitations of our analysis merit consideration. First, our analysis is limited to women, and thus care must be taken before these data are generalized to men. However, data from several other large cohorts of men have found hsCRP to predict risk independently of the traditional Framingham covariables (7, 9-12, 14, 16, 47). Second, our data are based on a single determination of hsCRP instead of 2 measures, as is currently recommended (21). This limitation would tend to increase variability in our measures of hsCRP and thus lead, if anything, to an underestimation of true effects. Third, in contrast to LDL cholesterol, there remains no evidence to date that reducing hsCRP level itself will reduce cardiovascular risk. As such, although these data show that in women the addition of hsCRP can improve global risk prediction models, they should not be construed as implying a direct benefit of hsCRP reduction, an issue now being evaluated in several clinical trials. Finally, we evaluated only hsCRP in this study and recognize the possibility that other biomarkers of inflammation, hemostasis, and thrombosis may become available in the future. In this regard, we believe that the methods developed here to address incremental value for risk screening will be of use as other novel biomarkers emerge.

          AppendixGraphic

          Computation of 10-Year Risk

          The 10-year risk for cardiovascular disease calibrated to the Framingham population may be estimated for each individual woman using the β-coefficients in Table 1. First, multiply each woman's risk factor x by the appropriate coefficient in Table 1 and sum these (= Σβ × x). The risk may then be computed from the following equation:

          Risk = 1 − (0.903)exp(Σβ × x − 8.795)

          Additional Measures of Model Fit

          The Appendix Table presents additional measures comparing the best-fitting models in the WHS data with and without hsCRP, as well as the ATP III model with coefficients refitted to the WHS data, with and without hsCRP. As indicated, all global measures of fit showed a preference for the models that included hsCRP. The measures are as follows:

          1. The likelihood ratio chi-square provides a global test of model fit. It is a function of the degrees of freedom, or number of terms in the model. The difference between chi-square values provides a test of the model improvement with hsCRP (P< 0.0001 for both the WHS and ATP III models).

          2. The Bayes information criterion is a function of the log likelihood but adds a penalty for added variables based on the sample size (28). It is not influenced by the number of predictors, so models can thus be compared directly. Lower values reflect better fit, suggesting improvement with the addition of hsCRP.

          3. The Bayes information criterion weight provides an estimate of the posterior probability of each model given the set of candidate models considered (29, 32). The weights suggest a much higher probability that the WHS model that includes hsCRP is correct.

          4. The Akaike information criterion is a function of the log likelihood that adds a penalty of 2 for each added variable (32), less extreme than the penalty used in the Bayes information criterion. Lower values are better, again suggesting improvement with hsCRP.

          5. The Akaike information criterion weights reflect the relative likelihood of a model given the data and the set of models (32). These weights display a clear preference for the models with hsCRP.

          6. Nagelkerke's generalized model R2(33, 34) is a measure of the fraction of the−2 log likelihood explained by the predictors, analogous to the percentage of variance explained in a linear model. It is adjusted to a range of 0 to 1 and is higher for models with hsCRP, both in the original data and after adjustment for optimism using the bootstrap (31, 48).

          7. The D-statistic of Royston and Sauerbrei (35) measures the separation of survival curves across levels of the predictor variables, analogous to distance between Kaplan–Meier curves. This is higher for models that included hsCRP, even after adjustment for optimism, suggesting better prediction for these models.

          8. The Brier score (28) computes the sum of squared differences between the observed outcome and the fitted probability. It is lower for models that included hsCRP, indicating that the predicted probabilities are closer to the observed outcomes.

          9. The c-index represents the area under the receiver-operating characteristic curve (30), allowing for censored data. This is a measure of discrimination based on ranks and is similar but slightly higher for models that included hsCRP, even after adjustment for optimism. The c-statistic is the probability that, for a randomly selected pair of subjects, one diseased and the other nondiseased, the person with disease will have the higher estimated disease probability according to the model.

          10. The Hosmer–Lemeshow calibration statistic (36) classifies predicted probabilities into categories and compares the mean predicted probability with the observed risk within each category. A P value representing a significant difference indicates a lack of fit. When decile categories are used, the predicted probability is less than 5% for the first 9 of 10 categories. Calibration is adequate for all models that use this measure and is somewhat better for models without hsCRP. The calibration statistic based on risk percentage compares observed and predicted risk by using 10 categories based on 2–percentage point increments in predicted risk, from 0% to 2% risk to 18% or greater risk. This statistic indicates significant deviation of observed and predicted values in models without hsCRP, suggesting a lack of fit in higher-risk categories.

          Article and Author Information

          • Grant Support: By grants from the Donald W. Reynolds Foundation (Las Vegas, Nevada), the Leducq Foundation (Paris, France), and the Doris Duke Charitable Foundation (New York). The overall Women's Health Study cohort is supported by grants HL-43851 and CA-47988 from the National Heart, Lung, and Blood Institute and the National Cancer Institute (both in Bethesda, Maryland).

          • Potential Financial Conflicts of Interest: Honoraria: P.M. Ridker (Dade Behring); Grants received: P.M. Ridker (Reynolds Foundation, Leducq Foundation, Doris Duke Foundation, National Heart, Lung, and Blood Institute, National Cancer Institute, American Heart Association, Dade Behring, AstraZeneca, Novartis, Sanofi-Aventis). Dr. Ridker is listed as a co-inventor on patents held by the Brigham and Women's Hospital that relate to the use of inflammatory biomarkers in cardiovascular disease.

          • Requests for Single Reprints: Nancy R. Cook, ScD, Division of Preventive Medicine, Brigham and Women's Hospital, 900 Commonwealth Avenue East, Boston, MA 02215; e-mail, ncook{at}rics.bwh.harvard.edu.

          • Current Author Addresses: Drs. Cook, Buring, and Ridker: Division of Preventive Medicine, Brigham and Women's Hospital, 900 Commonwealth Avenue East, Boston, MA 02215.

          • Author Contributions: Conception and design: N.R. Cook, P.M. Ridker.

          • Analysis and interpretation of the data: N.R. Cook, P.M. Ridker.

          • Drafting of the article: N.R. Cook, P.M. Ridker.

          • Critical revision of the article for important intellectual content: N.R. Cook, J.E. Buring, P.M. Ridker.

          • Final approval of the article: N.R. Cook, P.M. Ridker.

          • Statistical expertise: N.R. Cook.

          • Obtaining of funding: P.M. Ridker.

          • Collection and assembly of data: J.E. Buring.

          References

          1. 1.
          2. 2.
          3. 3.
          4. 4.
          5. 5.
          6. 6.
          7. 7.
          8. 8.
          9. 9.
          10. 10.
          11. 11.
          12. 12.
          13. 13.
          14. 14.
          15. 15.
          16. 16.
          17. 17.
          18. 18.
          19. 19.
          20. 20.
          21. 21.
          22. 22.
          23. 23.
          24. 24.
          25. 25.
          26. 26.
          27. 27.
          28. 28.
          29. 29.
          30. 30.
          31. 31.
          32. 32.
          33. 33.
          34. 34.
          35. 35.
          36. 36.
          37. 37.
          38. 38.
          39. 39.
          40. 40.
          41. 41.
          42. 42.
          43. 43.
          44. 44.
          45. 45.
          46. 46.
          47. 47.
          48. 48.

          Responses to this article

          Summary for Patients

          « Previous | Next Article »Table of Contents