The rule developed by Wells' group included the summation of the number of diagnostic variables present (0 to 8), minus 2 if another diagnosis was just as or more likely to explain the presented symptoms or signs (Table 1). Accordingly, a score of —2 indicated the lowest risk for presence of DVT and 8 the highest. We refer to this version of the Wells rule as the original Wells rule (5, 21, 22). Using this original scoring rule, Wells and colleagues then categorized the 593 patients into 3 risk groups: low (score 0), medium (score of 1 or 2), or high (score 3). Patients in the lowest-risk group did not undergo a second ultrasonography. We term this latter version of the Wells rule the categorized Wells rule.

In a recently introduced version of the Wells rule (13), patients with a Wells score of 1 or less and a normal result on D-dimer testing were defined to be at very low risk for DVT. Leg ultrasonography could safely be omitted in these patients. This version of the rule further decreased the number of unnecessary ultrasonographies in the lowest-risk group without increasing the number of missed cases of DVT. Other researchers, however, recently suggested using a Wells score of 0 or less plus a normal result on D-dimer testing to categorize a patient as being at very low risk and thus not requiring further work-up (9).

External Validation Study in Primary Care Patients

From 1 January 2002 to 1 March 2003, we prospectively identified consecutive adult (age > 18 years) patients who visited their primary care physician and were clinically suspected to have DVT. The suspicion of DVT was based on swelling, redness, or pain in the legs that had been present for 30 days or less. Patients in whom pulmonary embolism was also suspected were excluded. We conducted the study in a large, circumscribed geographic region of The Netherlands that included 3 nonteaching hospitals with a catchment area of 110 primary care physicians and about 250 000 inhabitants. Just before the start of the study, the participating general practitioners received detailed instructions by mail on how to use the necessary questionnaires, including the Wells rule. They also attended a specially organized conference and workshop on the logistics of the study and received a newsletter that included information about the progress of the project 4 times yearly. The diagnostic work-up, including study forms for documentation of patient information and referral of patients for further work-up, was integrated into the regular work-up of primary care patients. All 110 primary care physicians in the catchment area participated in the study. On average, 9 to 10 patients with suspected DVT per physician per year were included. The study area, participating general practitioners, and study patients can be considered representative of the country.

After informed consent was obtained, the primary care physician systematically documented information on the patient's history and physical examination by using a standard form on which the items and possible answers were specified. Patient history included sex, presence of previous DVT, family history of DVT, history of cancer (active cancer in the past 6 months), immobilization for more than 3 days, recent surgery (within the past 4 weeks), and duration of the 3 main symptoms (a painful, red, or swollen leg). Physical examination included the presence of tenderness along the deep venous system, distention of collateral superficial (nonvaricose) veins, pitting edema, swelling of the affected limb, and a difference between the circumference of the 2 calves.

The 9 items of the Wells rule (Table 1) were also included in the standard form for patient history and physical examination. All of the items were measured according to the original Wells rule (5, 21, 22). The last item of the rule—presence of an alternative diagnosis—has never been unambiguously defined and often causes controversy among users of the rule (23). In our study, physicians were asked to give their own assessment of the patient's probability of having DVT by using a score of 1 to indicate high probability of DVT, no alternative diagnosis likely; 2 to indicate moderate probability of DVT, alternative diagnosis possible; or 3 to indicate low probability of DVT, alternative diagnosis certain. To tailor the judgment of the physician on this item, 7 common alternative diagnoses for patients with suspected DVT were provided on the study form. If a low or moderate probability was assigned to a patient, we subtracted 2 points from the Wells score in the analysis. The items of the Wells score were documented only for the purposes of our study and were not used to determine a patient's further diagnostic work-up.

After history taking and physical examination, all patients were referred to the hospital for D-dimer testing and leg ultrasonography (24). Venous blood was drawn from the anterior cubital vein and analyzed by using established methods according to the manufacturer's recommendations. The analyses were performed with 1 of 2 highly sensitive D-dimer tests: an enzyme-linked immunosorbent assay (VIDAS, bioMérieux, Lyon, France) or a quantitative latex assay method (Tinaquant, Roche, Mannheim, Germany), depending on the laboratory routine of the participating hospital. Both assays have minimal risk for missing patients with DVT (25, 26). In line with the available guidelines and previous studies, the result of D-dimer testing was considered abnormal if the assay yielded a D-dimer level of 500 ng/mL or greater (26, 27).

Finally, as a reference test to determine the true presence or absence of DVT (3, 28), each patient underwent real-time, B-mode compression ultrasonography of the lower extremities by using a 5- to 7.5-MHz linear-array sonographic scanner (system V, GE Healthcare, Milwaukee, Wisconsin). Color duplex imaging was used to identify venous vessels and venous flow patterns and was repeated 7 days later in patients with a normal result on ultrasonography. Proximal DVT was considered present if results of 1 of the 2 ultrasonographic tests were abnormal. The ultrasonogram was considered abnormal when the common femoral vein, the superficial femoral vein, or the popliteal vein up to the trifurcation was not completely compressible. Obstruction of the iliac veins was tested with color duplex imaging (2, 29, 30). The person who performed the imaging was blinded to patient history, findings on physical examination, and results of D-dimer testing.

The study protocol was approved by the Medical Ethical Committee of the University Medical Center Utrecht, Utrecht, the Netherlands.

Statistical Analysis

We validated the categorized Wells rule by quantifying per score the overall proportion of patients and the actual prevalence of DVT. The sensitivity, specificity, negative predictive value, and likelihood ratio for a negative test result (negative likelihood ratio = [1–sensitivity]/specificity), with corresponding 95% CIs, were manually calculated. Because ruling out DVT is the main purpose of both the Wells rule and the D-dimer assay, the positive predictive value and the likelihood ratio for a positive test result are not presented. For similar reasons, only the diagnostic accuracy variables for the threshold between the low Wells score and the medium and high scores are presented.

The above analysis was repeated for the Wells rule in combination with results of D-dimer testing, both as originally proposed by Wells and colleagues (13) (DVT is ruled out if score is 1 and D-dimer assay result is normal) and as proposed by others (9) (DVT is ruled out if score is 0 and D-dimer assay result is normal).

If data on a patient were missing, the research physician contacted the patient's primary care physician immediately after the study forms were received. Nonetheless, 1 or more study variables was missing for 127 patients (45 patients had a missing value on 1 or more of the 9 items of the Wells rule). The missing data per study variable ranged from only 0.4% to 4% (the lowest was 5 missing responses for "paralysis or recent immobilization of the lower extremity," and the highest was 52 missing responses for "thrombosis in family"). We imputed the missing values of all study variables by using a linear regression method with the addition of a random error term (SPSS software, version 12.0 for Windows, SPSS, Inc., Chicago, Illinois) (17, 31, 32). We also performed a complete case analysis. Because this yielded similar results to those obtained after imputation, only the results after imputation are presented.

Role of the Funding Sources

The funding sources had no role in the design, conduct, or reporting of the study or in the decision to submit the manuscript for publication.

Results

Top Editors' Notes Methods Results Discussion Author & Article Info References

Of the 1326 patients considered for inclusion, 31 were excluded because the physician did not supply sufficient information about fulfillment of the inclusion and exclusion criteria (Figure). Table 2 shows the characteristics of the 1295 included patients. Deep venous thrombosis was diagnosed in 289 patients (prevalence, 22%); it was diagnosed on the first ultrasonogram in 284 patients and on the second ultrasonogram obtained 1 week later in 5 patients. Results of D-dimer testing were normal in 403 patients (31%) and abnormal in 892 (69%).

View larger version (41K): [in this window] [in a new window]   Figure. Study protocol and outcome. Results of the clinical score as estimated by the Wells rule, compression ultrasonography, and D-dimer testing are given. DVT = deep venous thrombosis.

The Wells score ranged from –2 points (61 patients) to 8 points (1 patient); the median score was 1. Table 3 shows results of use of the categorized Wells rule without findings on the D-dimer assay. Five hundred seven (39%) patients were at low risk for DVT (score 0), 321 (25%) were at medium risk (score of 1 or 2), and 467 (36%) were at high risk (score 3). The observed prevalence of DVT was 12.0% (61 of 507 patients) among low-risk patients, 16.5% (53 of 321 patients) among medium-risk patients, and 37.5% (175 of 467 patients) among high-risk patients. When a threshold was introduced between the lowest score and the other 2 scores, the sensitivity was 78.9%, specificity was 44.3%, negative predictive value was 88.0%, and negative likelihood ratio was 0.48.

Combining the Wells score with results of D-dimer testing (Table 3) increased diagnostic accuracy and the ability of the rule to exclude DVT. Two hundred twenty-two patients (17% of all patients) at lowest risk for DVT (score 0) had a normal result on D-dimer testing. Five of these 222 patients had DVT (prevalence, 2.3%). The sensitivity increased to 98.3%, but the specificity was halved. The negative predictive value also increased to 97.8%, and the negative likelihood ratio decreased to 0.08. If a Wells score of 1 or less was used in combination with a normal result on D-dimer testing to categorize patients, 21% (instead of 17%) of the patients were categorized as being at very low risk. The prevalence of missed cases of DVT, however, increased to 2.9%.

Discussion

Top Editors' Notes Methods Results Discussion Author & Article Info References

We validated the diagnostic decision rule for DVT with and without D-dimer testing, as developed in secondary care outpatients by Wells and colleagues, in a primary care setting. Compared with the original studies (5, 21, 22), the accuracy of the Wells rule without D-dimer testing in our sample was worse than expected. In particular, the overall proportion of patients in the lowest risk group (Wells score 0)—the most important group in which to correctly rule out DVT—was relatively low (39% instead of 55%), and the prevalence of DVT in that group was unacceptably high (12.0% instead of 3%). Sensitivity, specificity, and negative predictive value were substantially lower than in the original study. Applying the Wells rule in combination with D-dimer testing to avoid ultrasonography decreased the percentage of missed DVT cases from 12.0% to 2.9% (sensitivity, 97.2%). However, this percentage of missed cases is too high (10-12, 33). In addition, the overall proportion of patients in the lowest-risk group decreased from 39% to 21%, which largely outweighs the benefits of a lower percentage of missed DVT cases. Use of other score thresholds of the Wells rule in combination with a normal result on D-dimer testing to categorize patients as being at very low risk for DVT did not improve diagnostic accuracy variables (Table 4).

Prediction rules derived from a particular data set are commonly overfitted and yield results or risk estimations that are too optimistic when applied to other groups (17-20, 34, 35). One important reason for the poor performance of the Wells rule in our sample is the difference in settings (secondary care outpatients versus primary care patients), which may have led to spectrum bias (15, 34-36). The distribution of patient characteristics in our sample differed from that of the original study (Table 2). We included more elderly persons, women, and patients who had undergone previous surgery and fewer patients with active cancer. Also, secondary care patients are often preselected by primary care physicians, which often influences the prevalence of the disease under study and the distribution and diagnostic values of signs and symptoms (37). The prevalence of DVT in our study was slightly (6%) higher than that in the original study by Wells and colleagues. We think that the slightly higher prevalence is most likely due to a difference in the definition and selection of patients in whom DVT was suspected.

As discussed elsewhere (23), a more general reason for the poor performance of the Wells rule in other samples is the potential for interobserver variability for the last item (Table 1), "presence of an alternative diagnosis at least as likely as that of DVT." Scoring of this item is based on subjective criteria and determined by the skills of the physician. In our study, this item was scored by 110 primary care physicians in their daily practice, rather than by a few trained research physicians as in the study by Wells and colleagues.

Recently, Fancher and associates (33) published a systematic review of the combination of D-dimer testing and clinical probability of DVT according to the Wells rule in 11 studies of secondary care outpatients. A detailed definition of secondary care outpatients was not given. The pooled results showed a prevalence of venous thromboembolism in the lowest risk group of 0.4% to 0.5%, depending on the used D-dimer assay. These percentages are lower than those that we found and those in the original studies by Wells and colleagues (Table 3). The results of use of the Wells rule without D-dimer test results were not reported.

To further enhance the clinical meaning of our study and the inferences on the generalizability of the Wells rule across different groups of patients in whom DVT is suspected, we compared our results not only with this review and the original studies by Wells and colleagues but also to a validation study by Kraaijenhagen and coworkers, who investigated secondary care outpatients referred by their primary care physician (9). This latter study is large (1756 patients) and provided adequate numbers of patients in the different risk categories. The sample, however, consisted of referred outpatients. The sample studied by Kraaijenhagen and coworkers (9) included the same percentage of patients with cancer as that studied by Wells and colleagues (5), but as in our study, Kraaijenhagen and coworkers included more elderly persons, women, and patients who had undergone surgery (Table 2). Wells and colleagues excluded patients with a previous diagnosis of DVT, whereas Kraaijenhagen and coworkers excluded only patients with a previous diagnosis of DVT but no normalization of ultrasonographic findings.

Table 3 shows the accuracy of the Wells rule as applied to the sample studied by Kraaijenhagen and coworkers. The proportion of missed cases of DVT in the lowest risk group (7.9%) was lower than that in our study but is still unacceptably high. Combining the Wells score at a threshold of 0 with a normal result on D-dimer testing decreased the proportion of missed cases to 1.8%, which was a fraction lower than that which we observed (2.3%) but is still twice as high as that found by Wells and colleagues (0.9%) (13). Of note, Wells and colleagues used a threshold score of 1 or less (Table 3). In summary, the Wells rule did not adequately rule out DVT in patients in the study by Kraaijenhagen and coworkers (9), even though the rule was applied to secondary care outpatients. The rule did, however, perform better than in our study of primary care patients in whom DVT was suspected (Table 4).

Our study had some methodologic problems and limitations. First, besides the potential for interobserver variation, we used a 3-item question to score the last variable of the Wells rule ("presence of an alternative diagnosis at least as likely as that of DVT") instead of a 1-item question as used by Wells and colleagues. Although our measurement seems more precise, it could have resulted in a different performance of the Wells rule in our sample. Our percentage of patients with an alternative diagnosis is similar to those of other studies but could not be compared with that of the original study by Wells and colleagues because they did not provide this information (8, 9). Second, we used a different reference standard than did Wells and colleagues. They used a single leg ultrasonography and a 3-month follow-up visit (documentation of symptomatic DVT within 3 months) as the reference standard in the lowest-risk group, whereas we used repeated ultrasonography without follow-up. Hence, it could be argued that in the study by Wells and colleagues, the lowest-risk group included patients with a negative initial ultrasonogram and negative follow-up who actually had DVT that would have been detected during repeated ultrasonography. Alternatively, we did not have 3-month follow-up data on our patients and may have missed some cases of DVT. However, if we had such follow-up data, the number of missed cases of DVT in the lowest-risk group would probably be higher than in the present results (Table 2), worsening the performance of the Wells rule in primary care. Third, the limited performance of the categorized Wells rule could be due to the inclusion of patients with previous DVT; these patients were excluded in the study that led to development of the categorized Wells rule (5). To enhance comparability, we therefore validated the categorized Wells rule in patients with a first DVT only (987 patients). The prevalence of DVT in each risk group did not change: 12.2% in the low-risk group, 16.3% in the medium-risk group, and 39.6% in the high-risk group.

Even though the difference in settings would be expected to compromise the accuracy of the Wells rule in our sample, the fact remains that even in combination with a normal result on D-dimer testing, the rule cannot be generalized to all patients in whom DVT is suspected, whether in secondary or primary care. The question then arises whether the Wells rule in combination with D-dimer testing can be used by general practitioners to determine which patients with suspected DVT should be referred to secondary care and which could be safely kept under their own surveillance. On the basis of our findings, we believe that an unambiguous answer cannot yet be given. Investigations, such as that by Wells and colleagues (5, 13), are needed in primary care patients to develop a specific diagnostic rule to exclude DVT in these patients. Such a study is in progress (38). Furthermore, we recently showed that use of clinical findings only is insufficient to reach a decision on patient referral and concluded that in primary care, D-dimer testing seems necessary to safely exclude DVT without further work-up (39).

In conclusion, we found that the Wells rule, alone or in combination with D-dimer testing, does not guarantee accurate estimation of risk in primary care patients in whom DVT is suspected. Because of the apparent differences between primary and secondary care, a diagnostic rule combining patient history, physical examination, and D-dimer assay findings that has been developed using only primary care patients is of more value. Only in this way will physicians be able to safely exclude DVT in primary care patients.