When Should a New Test Become the Current Reference Standard?

2 December 2008 | Volume 149 Issue 11 | Pages 816-821

The evaluation of claims that a new diagnostic test is better than the current gold standard test is hindered by the lack of a perfect reference judge. However, this problem may be sidestepped by focusing on the clinical consequences of the decision rather than on estimation of accuracy. Consequences can be assessed by use of a "fair umpire" test that is not perfect yet can discriminate between disease and nondisease cases and is not biased in favor of 1 test.

This article discusses 3 principles to aid judgments about the value of new tests. First, the consequences are best examined in cases with disagreement between the current and new tests. Second, resolving these disagreements requires a fair, but not necessarily perfect, umpire test. Finally, umpire tests include consequences, such as prognosis and response to treatment, as well as causal exposures and other test results.

Author and Article Information

From the University of Oxford, Oxford, United Kingdom; University of Sydney, Sydney, New South Wales, Australia; and University of Birmingham, Edgbaston, Birmingham, United Kingdom.

Acknowledgment: The authors thank Gordon Guyatt, Ajit Lalvani, Sally Lord, Jenny Doust, and Chris Hyde for their helpful comments on drafts.

Grant Support: In part by funding from a UK National Institute for Health Research program grant and from the Australian National Health and Medical Research Council Program grant 402764 to the Screening and Test Evaluation Program.

Potential Financial Conflicts of Interest: None disclosed.

Requests for Single Reprints: Paul Glasziou, MB, BS, PhD, Centre for Evidence-Based Medicine, Department of Primary Health Care, University of Oxford, Oxford OX3 7LF, United Kingdom; e-mail, paul.glasziou{at}dphpc.ox.ac.uk.

Current Author Addresses: Dr. Glasziou: Centre for Evidence-Based Medicine, Department of Primary Health Care, University of Oxford, Oxford OX3 7LF, United Kingdom.

Dr. Irwig: Screening and Test Evaluation Program, School of Public Health, and University of Sydney, Sydney, New South Wales 2006, Australia.

Dr. Deeks: Unit of Public Health, Epidemiology, and Biostatistics; University of Birmingham, Edgbaston, Birmingham B15 2TT, United Kingdom.

Read all Rapid Responses

Forget about the gold standard

Yvo M Smulders, et al.

Annals Online, 22 Dec 2008 [Full text]

Keep the reference standard, but add spectrum

Paul Glasziou, et al.

Annals Online, 29 Jan 2009 [Full text]