A New Bedside Test of Cognition for Patients with HIV Infection

Methods

Patients

Consecutive inpatients admitted to the Osler 8 AIDS service at Johns Hopkins Hospital during a 2-month period were asked to participate in the study. Seven outpatients from the Johns Hopkins Moore Clinic were also tested. Patients who were in acute distress, recovering from surgery or invasive procedures, or terminally ill were excluded. Patients were only tested if they were alert and judged not to be delirious. Sixty-two patients completed the testing. Approximately 40 patients were excluded by the ward research coordinator because they were seriously ill; these patients were more likely to be delirious or moribund, or both, than were those who participated. Approximately five patients declined to participate; these patients did not differ systematically from the participants.

Testing

The Mini-Mental State Exam (MMSE) is a brief screening test of cognition that measures orientation, memory, concentration, language, and praxis [14]. The Trailmaking Test, parts A and B [15], requires persons to draw a line connecting circles in a specified sequence. Part A assesses psychomotor speed and sequencing ability; part B assesses the ability to switch between two categories.

The Mental Alternation Test requires alternation between numbers and letters. Patients were told that the researchers were interested in testing their thinking and memory, that they might find some questions difficult and others easy, and that their best effort would be appreciated. Patients are asked to count to 20, say the alphabet, and then alternate between the numbers and letters in the following fashion: 1-A, 2-B, 3-C . Progressing from the most recent number or letter to the next letter or number in the sequence is one alternation. The number of correct alternations in 30 seconds, discounting any errors, determines the score. The maximum score is 52 points.

The MMSE was administered first, followed by either the Trailmaking Test or the Mental Alternation Test, the order being alternated between successive patients to control for any learning effect. Patients were tested in the late morning or early afternoon, usually at 10 a.m. or 2 p.m.

The tests were administered by a psychiatrist and an internist to the first 25 patients and by the internist to the remaining participants. The MMSE required between 5 and 10 minutes for administration; the Trailmaking Test, parts A and B, required from 7 to 15 minutes, whereas the Mental Alternation Test was administered in less than 1 minute.

Data Analysis

To determine the relation between the performance on the Mental Alternation Test and the better standardized tests, we calculated correlation coefficients among scores on the MMSE; the Trailmaking Test, part B; and the Mental Alternation Test. To find the utility of using performance on the Mental Alternation Test to predict cognitive impairment as measured by scores on the MMSE and the Trailmaking Test, part B, we calculated the sensitivity and specificity of various Mental Alternation Test cutoff values in identifying abnormal performance on the MMSE and the Trailmaking Test, part B, based on population norms [16, 17]. A chi-square analysis was done to determine the likelihood that patients with an abnormal Mental Alternation Test score would have abnormal scores on the MMSE and the Trailmaking Test, part B. A general linear model analysis of the MMSE score was conducted to determine whether the Mental Alternation Test score was a significant source of variance when age, sex, educational level, intravenous drug use, homosexual activity, and performance on the Trailmaking Test, parts A and B, were included in the model.

A single examiner (KSH) administered the Mental Alternation Test twice in the same day to a series of 20 patients to determine test-retest reliability. Two investigators (BNJ, KSH) independently scored a tape recording of 15 persons performing the Mental Alternation Test, and inter-rater reliability was calculated.

Results

Patient characteristics are shown in Table 1. The Mental Alternation Test scores ranged from 0 to 35 (mean SD, 16.8 8.5). Scores on the MMSE ranged from 12 to 30 (mean, 25.0 4.0). Scores on the Trailmaking Test, part A, ranged from 20 to 300 s (mean, 72.8 68.8 s). The mean score on the Trailmaking Test, part B, was 238 199 s; scores ranged from 50 to 600 s (10 patients unable to complete the test were assigned a null value of 600 s). The Mental Alternation Test score correlated significantly at the P < 0.001 level with the MMSE score (r = 0.68); the Trailmaking Test, part A, score (r = 0.53);and the Trailmaking Test, part B, score (r = 0.54).A scatterplot of Mental Alternation Test scores and MMSE scores is shown in Figure 1.

View larger version:

• In this window
• In a new window

Figure 1. Scatterplot of Mini-Mental State Exam (MMSE) scores versus Mental Alternation Test scores.

A general linear model for variation fitted with covariants showed only the Trailmaking Test, part B, and the Mental Alternation Test to be significant covariates at the P < 0.05 significance level. These two covariants accounted for 61% of the total variation in the MMSE score.

Reliability as measured by test-retest correlation gave a Pearson correlation coefficient of 0.80. The Pearson correlation coefficient for inter-rater reliability was 0.85. Using cutoff scores of 15/14 on the Mental Alternation Test to divide performance into normal and abnormal, we found that agreement between raters was high (, 0.84).

A receiver operating curve is a graphic presentation of one test's sensitivity and false-positive rate (1 specificity) along a range of thresholds or cutoff scores for identifying a case, that is, abnormality as defined by some other measure [18, 19]. In our study, a case was defined as a patient with an abnormal MMSE or Trailmaking Test score. Having established a significant correlation between the Mental Alternation Test and the MMSE and Trailmaking Test, we then wished to know how to use the new test of cognition to identify such cases (that is, what constituted normal and abnormal Mental Alternation Test scores). The MMSE scores were recoded as normal if 24 or above and abnormal if lower based on established norms [16]. Trailmaking Test, part B, scores were recoded as abnormal if greater than 140 s, a value 2 standard deviations above the average score for an age-appropriate sample [18]. The appropriateness of different cutoff values for defining the abnormal Mental Alternation score was determined by calculating the sensitivity and specificity of each cut-off score for identifying cases with abnormal scores on the MMSE and Trailmaking Test, part B. The most appropriate value will have a position on the receiver operating curve in the uppermost left-hand corner, corresponding to a high sensitivity and low false-positive rate (high specificity). A Mental Alternation Test cutoff score of 15 yielded the best results for the MMSE (sensitivity, 95% [95% CI, 90% to 100%]; specificity, 79% [CI, 69% to 89%]) (Figure 2, Table 2) and for the Trailmaking test, part B (sensitivity, 78% [CI, 68% to 88%]; specificity, 93% [CI, 90% to 100%]). Patients making fewer than 15 alternations in 30 seconds were significantly more likely to have abnormal scores on the MMSE (P < 0.0001) and the Trailmaking test, part B (P < 0.0001).

View larger version:

• In this window
• In a new window

Figure 2. The usefulness of different cut-off values for defining abnormal performance on the Mental Alternation Test was determined by calculating the sensitivity and specificity of each cut-off score for abnormal Mini-Mental State Exam (MMSE) performance (MMSE score < 24). Receiver operating curve.

Discussion

The Mental Alternation Test is a verbal test of cognition that can be easily and quickly administered to patients at risk for cognitive impairment. The Mental Alternation Test had high sensitivity and specificity for abnormal performance on the MMSE and the Trailmaking Test, part B; patients making fewer than 15 correct alternations in 30 seconds were significantly more likely to have an abnormal MMSE score.

Most participants in this study were inpatients, many of whom had advanced-stage AIDS. They had several potential causes for cognitive impairment, many of which are shared with the general hospital population. The success of the Mental Alternation Test in this population suggests it may also be useful in the general hospital population. The impairments detected in our study participants cannot be assumed to be caused by HIV infection alone, and the utility of the Mental Alternation Test in ambulatory, asymptomatic persons cannot be determined from our study. However, our sample included persons who scored normally on the MMSE, indicating that not all persons in our sample had gross cognitive impairment as detectable by the MMSE.

The Mental Alternation Test was designed as a screening tool with high sensitivity for cognitive impairment and lower specificity, because patients with false-positive results will be identified later by more extensive testing. The Mental Alternation Test does not, by itself, identify the cause of cognitive impairment. Poor performance on the Mental Alternation Test could result from a preexisting condition such as subnormal intelligence; a new medical condition such as encephalitis; or poor concentration due to sleep deprivation, medication side effects, or pain. In our study, there was no indication that the test specifically identified patients with the HIV dementia complex but only that it can be used to identify cognitive impairment in patients with HIV infection. However, because the Mental Alternation Test is timed, it may offer an advantage over the MMSE in detecting HIV-related cognitive impairment; the MMSE has no timed element and may be insensitive to slowed mentation and psychomotor speed.

The advantages of the Mental Alternation Test include its rapid administration, which requires no materials except a timepiece. It can be used in visually impaired patients or in those who have a reduced ability to use paper and pencil because of weakness or the presence of technical equipment such as monitors and intravenous lines. Additional experience with the Mental Alternation test will provide normative and validation data for use in other populations.

The Mental Alternation Test can help to raise and clarify diagnostic issues. The identification of cognitive impairment is an important part of the assessment of behavioral complications in patients with HIV infection; such complications include inappropriate actions on an inpatient ward and noncompliance as an outpatient. There is evidence that management of the patient is improved through cognitive screening [20]. Because of the frequency of cognitive impairment in patients with HIV infection, administration of the Mental Alternation Test represents an efficient use of the clinician's time. We recommend that patients with HIV infection have a quantitative cognitive assessment.

Contact Dr. Teng for a record form and more information about the Mental Alternation Test.