The Reliability of Medical Record Review for Estimating the Frequency of Medical Mistakes

Summaries for Patients are a service provided by Annals to help patients better understand the complicated and often mystifying language of modern medicine.

Summaries for Patients are presented for informational purposes only. These summaries are not a substitute for advice from your own medical provider. If you have questions about this material, or need medical advice about your own health or situation, please contact your physician. The summaries may be reproduced for not-for-profit educational purposes only. Any other uses must be approved by the American College of Physicians-American Society of Internal Medicine.

The summary below is from the full report titled "The Reliability of Medical Record Review for Estimating Adverse Event Rates." It is in the 4 June 2002 issue of Annals of Internal Medicine (volume 136, pages 812-816). The authors are EJ Thomas, SR Lipsitz, DM Studdert, and TA Brennan.

What is the problem and what is known about it so far?

In 1999, the U.S. Institute of Medicine reported that mistakes in U.S. hospitals are alarmingly frequent. This report relied on studies that used doctors and nurses to review patients' medical records to identify mistakes. In these studies, the doctors and nurses looked at the medical records to first decide whether a patient had experienced a complication. Second, they had to decide whether the complication was related to medical care. Third, they needed to decide whether the complication was an expected side effect or was caused by someone's mistake. Judging whether someone made a mistake can be difficult when you have only the information in the patient medical record.

Why did the researchers do this particular study?

To learn more about the reliability of estimates of medical mistakes determined by medical record review.

What was studied?

500 medical records of patients who had been hospitalized in Utah or Colorado in 1992.

How was the study done?

Three separate doctors reviewed each medical record according to the same set of instructions. The researchers then calculated how frequent mistakes were among the 500 patient records using different definitions. The definitions varied according to the number of reviewers and their level of certainty required to count a complication as a mistake.

What did the researchers find?

The estimated number of mistakes ranged from fewer than 1% (1 in 100) to more than 32% [32 in 100], depending on the rules set by the researchers. The following changes in the review process markedly decreased the estimates of mistakes: 1) increasing the number of reviewers from one to three and 2) requiring reviewers to be highly confident that a complication was due to a mistake.

What were the limitations of the study?

This study involved only three reviewers. The results might vary with a different set of reviewers. Moreover, there was no way to know for certain that a mistake had actually occurred. The number of mistakes found in these records is not an estimate of a patient's risk for being injured in a hospital.

What are the implications of the study?

It is difficult to determine from a review of patient charts whether medical mistakes have occurred. The estimates reported by the Institute of Medicine could be inaccurate.