Patient Education Materials about the Treatment of Early-Stage Prostate Cancer: A Critical Review

Methods

Development of Review Criteria

We developed the criteria for evaluating the content of patient education materials by first examining the empirical and theoretical literature on informed consent and decision aids. For a primary-level analysis, we adopted the Cochrane definition of balanced presentation: 1) Are all options presented (including, if appropriate, watchful waiting)? and 2) Are potential harms as well as potential benefits presented (4)? We used the Cochrane criteria to eliminate all patient education materials that 1) did not discuss the 4 standard prostate cancer treatments (watchful waiting, radical prostatectomy, radiation therapy, and hormone therapy), 2) discussed only cancer in general, or 3) discussed only prostate cancer screening.

In a second-level detailed analysis, we applied previously developed generic criteria specific to prostate cancer (5). Additional items to reflect patient needs were obtained through a literature review and through review of our criteria from a local prostate cancer support group and from prostate cancer experts (urologists, oncologists, nurses, researchers, and activists) from the Michigan Cancer Consortium Prostate Cancer Action Committee.

Four of the authors independently developed common definitions and coded test documents. An initial list of 85 criteria was reduced to 54 criteria by consensus and was operationalized (Tables 1, 2, and 3). A weighting system was considered but was not implemented because no gold standard could be identified from the literature on decision aids or risk communication. Similarly, the Cochrane criteria provide an empirically validated set of criteria for the second-level detailed analysis.

In previous work, we have shown that decision aids can produce decisions that better reflect patient preferences (6, 7). The parallel criteria selected for prostate cancer are the categories of treatment outcomes and side effects. Traditional patient education elements of description of the anatomy and disease progression were also included.

Identification of Patient Education Materials

Print Materials, Videotapes, and CD-ROMs

To identify these materials (identified between September and December 2001), we asked national organizations (including patient advocacy groups, government organizations, pharmaceutical companies, insurance companies, health maintenance organizations, universities, and comprehensive cancer centers) for their materials. To be included, all materials must be widely available to the public at no cost.

Internet Search Strategy

We first reviewed Web sites of prominent organizations (including all of those identified during the print material, videotape, and CD-ROM search). Second, we reviewed Web sites of pharmaceutical companies that had received approval from the U.S. Food and Drug Administration to produce prostate cancer drugs.

Third, a “naive patient with prostate cancer” strategy used an open (broad-based) search strategy with Google and Yahoo, which located more than 300 000 Web sites. We reviewed at least the first 100 links provided and discontinued our search when relevance decreased. We did not evaluate Web sites composed of links to other Web sites or duplicated print materials previously reviewed (for example, the Web site of the National Cancer Institute).

The initial review set of 546 materials was reduced by 502 materials after applying the Cochrane criteria for all standard treatment options to be presented (see the Appendix Table, for list of print materials not eligible for review). The remaining 44 materials (19 print materials, 19 Web sites, 4 videotapes, and 2 CD-ROMs) underwent a formal content and quality review (Table 4).

Content Review

Three coding teams composed of 2 coders were each responsible for reviewing one third of the materials. Thus, each material was independently scored by 2 coders. Criteria were rated as present if they were mentioned in the text, even if briefly (8). Paired ratings were compared, and differences were resolved by a second review and, in rare occasions, by a third reviewer. Percentage agreement achieved by the teams ranged from 91% to 94%.

Quality Review

The quality review evaluated 1) the accuracy of the information contained in the patient education material, 2) whether presentation of treatment options was balanced, and 3) whether the information was comprehensible to the average reader. An experienced clinician with previous prostate cancer research experience performed the accuracy and balance review. The clinician did not participate in the content review and was blind to content scores. Accuracy criteria were developed from the prostate cancer literature on the natural history of the disease and the efficacy and side effects of treatments. The estimates of balance were subjective, graded from minimal to marked bias toward any particular treatment method.

To further evaluate the best materials identified through the simple content inclusion criteria, a health literacy expert performed an extensive “plain-language” review on the top 5 print materials and top 5 Web sites. To determine these top-rated materials, we developed a scoring system that identified how many of the 54 essential criteria each piece of patient education material contained but did not prioritize further. Those materials that contained inaccurate information or were strongly biased toward a particular treatment were excluded from consideration. Only 1 of the print materials was eliminated from the top rating because of presence of bias. The review assessed characteristics of text and design that affect reading ease and comprehension, incorporating the widely used Suitability Assessment of Materials system (9). Criteria include 1) readability; 2) amount and organization of content; 3) writing style as it affects literacy demands; 4) graphics, layout, and typography; 5) evidence of learning stimulation; and 6) cultural appropriateness. Each criterion was evaluated according to specific subcharacteristics rated on a 0- to 2-point scale (0 = unsuitable; 1 = adequate; 2 = superior; or not applicable). Final scores were calculated as percentages based on a denominator of 44 possible points. The 0- to 2-point scale was then translated into grade level. Web sites were evaluated by using similar criteria.

Updating the Materials

Several Web sites changed during the 4-month review period. The review described later in this paper reflects assessment of Web sites between 1 September 2001 and 14 December 2001. In April 2003, we rescored the top 5 and bottom 5 Web sites and brochures to determine whether they had changed substantially since our review.

Role of the Funding Source

The funding source was not involved in the design, conduct, or reporting of the study or in the decision to submit the manuscript for publication.

Results

Tables 1, 2, and 3 show the proportion of criteria (in consolidated form) scored as present in the 44 patient education materials. Although videotape and the CD-ROM materials are reported in Tables 1, 2, and 3, discussion in the paper is restricted to print materials and Web sites because these are the tools most available to public audiences. We found some differences while rescoring the top 5 and bottom 5 print materials and Web sites, but they were small and did not change the rankings of the materials.

Content Review

Disease Process

Most patient education materials included basic information on prostate anatomy and physiology (95% and 80% of print materials and Web sites, respectively). Most print materials and Web sites also discussed prostate cancer staging (100% and 95%, respectively) and grading (74% and 84%, respectively). (Content analysis for individual documents is available from the authors.)

Treatment Information

To meet patient information needs, patient education materials should 1) describe each procedure in full detail, 2) address the psychological effect of treatment, and 3) distinguish between temporary and permanent outcomes.

Many materials did not include detail about patient treatment experience. For instance, approximately 50% did not inform patients about the need for hospitalization (after radical prostatectomy) and only 53% of print materials and 21% of Web sites discussed the need for catheterization after a radical prostatectomy. Many materials also did not include complete information on side effects. Although most materials listed incontinence and impotence as side effects of treatments, far fewer acknowledged the risk for bowel disorders or death. Only 42% of print materials and 53% of Web sites discussed the likelihood of pain, nausea, or fatigue as a result of radiation therapy. Few materials differentiated between permanent and temporary side effects, particularly for radical prostatectomy or hormone therapy. The patient education materials avoided discussing negative outcomes, such as side effects, emotional discomfort, or death.

Participation in Decision Making

Most patient education materials (84% of print materials and 68% of Web sites) explicitly encouraged patients to be active decision makers. Support of shared decision making came in several forms, from simply stating that patients should talk with their physicians about their preferences and concerns to providing patients with questions for their physicians.

True decision aids are designed not only to help patients understand the probable benefits and risks of treatment options but also to help patients consider the value they place on the benefits versus the risks (10). No material we reviewed qualified as a decision aid. The materials were adequate for describing treatment options but did not provide sufficient information to actively assist patients in their decision making.

Describing Strength of Evidence

Patients must understand that no clinical trial has revealed any differences in 10- to 15-year all-cause mortality across treatments (“Identify medical controversy about best treatment” in Table 3). This has important consequences in light of recent findings that urologists recommend radical prostatectomies to their patients, whereas radiation oncologists recommend radiation (11). Only 37% of print materials and 37% of Web sites described the lack of conclusive evidence.

Inclusion of Quantitative or Graphic Information

Quantitative information about the likelihood of side effects, years of being disease-free (12), and recurrence rates associated with each procedure is highly relevant to patients' decision making. None of the materials discussed recurrence or success rates; fewer than 50% of materials included any type of numeric or rate information. Furthermore, of all the patient education materials, only 1 Web site included graphical information.

Quality Review

Accuracy and Balance of Print Materials

No print patient education materials we evaluated had clinically significant misstatements, although some references were out of date because of the publication date. We found only one case of clinically significant imbalance in the treatment descriptions. However, a general bias was toward active treatment that minimized the role of watchful waiting. In addition, the likelihood and impact of side effects were minimized.

Accuracy and Balance of Web Sites

All sites reviewed were accurate, but some erred by omission, most typically de-emphasizing side effects of therapy. Several sites presented so little of the key content that they generated somewhat misleading impressions.

Although we did not find a clinically significant imbalance in treatment descriptions in any Web site, we again found a bias toward active treatment and minimizing the likelihood and effect of side effects.

Plain-Language Evaluation

The 5 top-rated print materials and 5 top-rated Web sites, as judged by the content review, received a health plain-language review. The readability level for all but 1 of the materials, based on the Suitability Assessment of Materials system and the Fry readability formula, was above the ninth-grade level. Although typical for health information, this level of reading difficulty is above the average reading ability of U.S. adults. This suggests that upwards of half the population would have difficulty comprehending the material. As shown by the results of the 1992 National Adult Literacy Survey (13), certain groups are most likely to have limited literacy skills—older adults, some minority groups, and adults with chronic health problems. These groups are most in need of information about prostate cancer.

Beyond these high grade-level scores, patient education materials were largely written in passive, third-person, clinical language. Pages were text-dense and lacked design elements to guide the reader. Most materials also lacked visual appeal or illustrations to add human interest and reinforce key points. Overall, these materials did not follow guidelines for plain-language materials (24).

Discussion

Patients are increasingly being encouraged to actively participate in treatment decisions for localized prostate cancer. Furthermore, patients report wanting to be involved in these decisions. In a recent survey, fewer than 2% of men recently receiving a diagnosis of prostate cancer wanted only their physicians to decide what type of treatment they should receive (14). Previous research has shown that most physicians rarely inform patients of the risks and benefits associated with all standard treatment options (8).

We evaluated whether the implementation of patient education materials reflects the emerging standards of patient decision aids. Specifically, do they contain the information necessary for patients to make informed decisions? Is the information accurate, balanced, and understandable? In what ways can materials be improved? Our review of the most thorough patient education materials in the public domain shows that although they provide a large volume of information, these materials lack several important content elements. Much of the material did not thoroughly describe patients' probable experience with treatments (such as hospitalization and catheterization) or side effects of treatments. Some materials were misleading and contained overly optimistic portrayals of the likelihood of side effects.

Although most patient education materials encouraged shared decision making, there is still room for improvement. Merely suggesting that shared decision making is beneficial is not enough; rather, patients must be provided with guidance on how to engage in this process. This assistance can be provided both directly and indirectly. A direct method is to provide a list of questions to discuss with their physician. This can also help direct their attention to the most important aspects of the clinical problem. A less direct method is to model the decision-making process by having prostate cancer survivors “speak” (for example, through the use of testimonials) in the materials.

No patient education material could be classified as a “decision aid” (15, 16). Decision aids have mitigated problems that patients may have with 1) knowledge of risks and benefits, 2) perceived pressure from other people, or 3) limited skills or power in sharing decision making with health professionals (15). Thus, patient education materials must be improved to highlight the tradeoffs that patients must make when choosing a treatment.

The paucity of quantitative information found in the patient education materials may be due to the difficulty inherent in outcomes that are highly variable and hard to apply to a specific patient. Another factor may be that the creators of health education materials are aware that many people have difficulty understanding quantitative information (17) and are concerned that presenting numeric information may decrease readership. However, studies have consistently found that people value quantitative assessment of the risks and benefits in treatment information (18) and, if information is presented appropriately, can use this effectively in their decision making (19, 20). Innumeracy has predicted differences in patient treatment decisions. For example, in a study of mammography screening, women who had low numeric ability made different screening evaluations than did more highly numerate women (21). Currently, simple numbers out of 100 for rates of dying and experiencing adverse events seem to be most widely understood. Ideally, these materials would include more than just numeric data but also graphical representations of the quantitative information (for example, bar graphs, line graphs, and pictographs) because past research has shown the advantages of providing this information (12, 22) (Fagerlin A, Wang C, Ubel P. Reducing the influence of anecdotal reasoning on people's health care decisions: Is a picture worth a thousand statistics? Submitted for publication).

Our health plain-language review showed that, with the exception of 1 piece of print material, the readability levels exceeded the general reading abilities of the average U.S. adult. In addition, the materials generally did not reflect the best graphic design, organization, writing style, and appearance of consumer-friendly patient education materials (23, 24).

Our study has several limitations. One limitation is restricting our search to publicly available materials. We could not obtain materials from insurance companies or health maintenance organizations that were available only to participating members. However, materials from national cancer and urologic organizations seem to represent many frequently used publications. On the other hand, narrowing the sample to materials that included all standard treatment options does not allow us to comment on materials designed to explain or promote a single treatment option. We also chose not to review books because they have a larger scope. Specifically, we were interested in evaluating clinical and informed-consent tools. While some books address these issues, most books do not. In addition, we did not review materials written in languages other than English. Future research would benefit by reviewing books and materials written in other languages. Our study is also limited by the fact that 1 reviewer assessed the accuracy, balance, and readability. Other reviewers may have come to different conclusions about these criteria. Finally, our content criteria focus on informed decision making. Other aspects of health education may require a different evaluation template. The criteria we used were derived from theoretical work in decision analysis and risk communication as applied in decisions aids (3). Ideally, these prostate cancer–specific criteria should be independently validated to determine their importance in producing a decision consistent with patient values. This empirical validation would create a weighted set of criteria reflecting the factors most relevant to informed, preference-sensitive decision making. However, our simple, summed criteria provide an initial assessment.

Conclusion

Patients should be encouraged to participate in their prostate cancer decisions. Several organizations and individuals have developed tools that aim to improve men's ability to make informed decisions. Unfortunately, the patient education materials that are currently available do not meet these goals completely. They typically do not contain complete information about prostate cancer treatments and their accompanying side effects, do not contain adequate quantitative information, and are written at reading levels that exceed those of the average U.S. adult. If men are to truly make informed decisions about their prostate cancer, better tools for delivering information about the disease and treatment options must be developed.