Investigators at the National Institutes of Health (NIH) began to recruit patients with the Cushing syndrome in the early 1980s to explore the diagnostic usefulness of newly available corticotropin-releasing hormone. They have now studied more than 500 patients for several weeks each, have subjected them to multiple diagnostic test procedures with contemporary adrenal steroid and ACTH assays, and have amassed an enormous and unique database on this uncommon endocrine disorder. Having popularized petrosal sinus sampling [6], they now seek noninvasive, alternative differential diagnostic procedures. In this issue, Avgerinos and colleagues [7] compare the metyrapone stimulation [8] and high-dose dexamethasone suppression [2] tests for the differential diagnosis of the ACTH-dependent Cushing syndrome. They confirm that each test is useful and that both are better than either alone. Although this is reassuring, I have concerns about this latest in the series of articles resulting from this clinical investigation project, as I have had about some others [9, 10].

One concern is possible patient selection bias. The authors mention several sources of potential bias, and others that we cannot eliminate or even evaluate also exist. We simply do not know how accurately the metyrapone test (which they propose we do with the conventional 8 a.m. to 8 a.m. urine collection period [7], although this was not used in this study) will predict the site of ACTH secretion in our next patient with the Cushing syndrome. It did not serve the authors as well as they claim because the patient who ultimately proved to have primary adrenal disease met their criterion for an increase in the plasma 11-deoxycortisol level of greater than 400 times the baseline value [7]. The authors eliminated this patient from their analysis because, in retrospect, he did not have the ACTH-dependent Cushing syndrome. However, they thought he did when the test was done, and diagnostic tests are useful only prospectively. What were his plasma ACTH values, his response to corticotropin-releasing hormone, and the results of inferior petrosal sinus sampling? How did this patient navigate through so many 100%-accurate tests without being correctly classified?

My major concern, however, is the mathematical approach to diagnosis described in this series of articles, that is, the continuing search for the one test that with proper cut-off criteria will lead to the correct classification of every patient with Cushing disease and the ectopic ACTH syndrome. We do not need sophisticated statistical analysis to calculate the criterion for 100% specificity of this test for Cushing disease: We can just draw a horizontal line above the highest value of the 15 patients with the ectopic ACTH syndrome in Avgerinos and colleagues' Figure 2 [7]. The problem with seeking a diagnostic accuracy of 100% in every test is shown by the record of the NIH group with regard to the high-dose dexamethasone suppression test, among others: First, sensitivity decreased [5, 11], and now stricter criteria are recommended to maintain a specificity of 100% [7]. If the goal is to exclude all patients with the ectopic ACTH syndrome, the criterion is determined by the most extreme outlier in the total series of patients. As series size increases, the criterion can do nothing but increase and the sensitivity nothing but decrease.

The same is true of the design of diagnostic tests. These investigators reported that, when corticotropin-releasing hormone was injected 2 hours after the final dexamethasone dose of a low-dose suppression test, the plasma cortisol response successfully distinguished 39 patients with Cushing disease from 19 patients with the pseudo-Cushing syndrome 15 minutes after corticotropin-releasing hormone was administered, but not 5, 30, 45, or 60 minutes after [12]. Will this hold true for 390 patients with Cushing disease and 190 patients with the pseudo-Cushing syndrome? I doubt it.

While striving for perfection, why not enlist the patient's help? In this study, patients were excluded because of incomplete urine collections defined as a "coefficient of variation of creatinine greater than 30%" (a phrase deleted from the final version of the manuscript), but a level of 30% or less was acceptable. Consider a woman with Cushing disease whose excretion of 17-hydroxycorticosteroids actually increases 80% during administration of metyrapone and a man with the ectopic ACTH syndrome whose excretion increases by 60%, in both of whom the criterion for an increase in 17-hydroxycorticosteroid excretion of greater than 70% was met [7]. Assume that their steroid excretion, like their creatinine excretion, remains constant day and night and from one day to the next. (Steroid excretion does not remain constant, and the results could be either more or less erroneous than these). The day before metyrapone is given, the woman starts her urine collection by adding to her empty bottle 3 hours of urine that had collected in her bladder rather than discarding it. On the next day she inadvertently discards 3 hours of urine. She has a variation in creatinine excretion of 25% and a factitious increase in urine 17-hydroxycorticosteroid excretion of 40% and is misdiagnosed as having the ectopic ACTH syndrome—unless she is among the 38% of patients with Cushing disease whose plasma levels of 11-deoxycortisol increases more than 400-fold. The man adds the urine from the last 3 hours of the first day to the start of the collection of the second day. He has a variation in creatinine excretion of 25% and a factitious increase in urine excretion of 17-hydroxycorticosteroids of 106% and is misdiagnosed as having Cushing disease—unless his plasma level of 11-deoxycortisol increases more than 400-fold. If he were included in the NIH series, his results would also increase the cut-off criterion and decrease test sensitivity.

Carefully instructed patients [13] collect sequential 24-hour urine specimens that vary no more than 10% from day to day. When told how important this is to ensuring correct diagnosis, they often do even better. Creatinine can be measured before steroids are assayed, and the collection can be repeated if the variance is too great. Invalid test criteria, unnecessary tests, incorrect diagnoses, and inappropriate surgery are the potential results of not doing so. A combination of inaccurate urine collection and laboratory error, not the novel dissociation of glucocorticoid up-regulatory and down-regulatory modulation of ACTH production by pituitary adenomas proposed by Avgerinos and colleagues, most likely accounts for the disparity between some of their metyrapone and dexamethasone test results and may partly explain why two measures for each test were needed and why two tests were better than one.

Adrenal function in the Cushing syndrome blurs into the normal range, hormone secretion is often erratic, and the degree of hypothalamic-pituitary suppression and the responsiveness of nonpituitary tumors to normal regulatory factors vary. Thus, it is unlikely that a single test can, with mathematical precision, lead to the diagnosis of the Cushing syndrome or establish its cause. Each cause of the Cushing syndrome has its own unique pathophysiology. The endocrinologist is constantly aware of this and subjects the patient to the least number of hypothalamic-pituitary-adrenal function tests necessary to obtain results that, when considered together, define the underlying cause.

The authors have proposed a contemporary diagnostic approach to the Cushing syndrome, as have myself and others [14, 15], but no one has a database as extensive as that amassed by the NIH. What could best justify the enormous expense and effort invested in this large clinical investigative project is a matrix analysis of each individual patient's diagnostic track to predict which combination of tests (including nonendocrine tests such as chest computed tomography and magnetic resonance imaging) done in what order and using which cut-off criteria are necessary to establish the correct diagnosis in all but the most unusual patient. Let us not begrudge the authors' missing a single patient with primary adrenal disease. Perfection is an elusive goal.