Assessing screening quality in the CDC's Colorectal Cancer Screening Demonstration Program
Marion R. Nadel PhD, MPH,
Division of Cancer Prevention and Control, Centers for Disease Control and Prevention, Atlanta, Georgia
Corresponding author: Marion R. Nadel, PhD, Division of Cancer Prevention and Control, Centers for Disease Control and Prevention, 4770 Buford Highway, Mailstop F-76, Atlanta, GA 30341; Fax: (770) 488-4286; email@example.com
The articles in this supplement were commissioned based on participation in evaluating the Centers for Disease Control and Prevention-funded Colorectal Cancer Screening Demonstration Program.
The opinions or views expressed in this supplement are those of the authors and do not necessarily reflect the opinions or recommendations of the journal editors, the American Cancer Society, John Wiley & Sons Inc, or the Centers for Disease Control and Prevention.
We gratefully acknowledge William Helsel, William Kammerer, William Howe, and Tanner Rockwell of Information Management Services Inc for providing analytical and data management support.
Gaps in screening quality in community practice have been well documented. The authors examined recommended indicators of screening quality in the Centers for Disease Control and Prevention's Colorectal Cancer Screening Demonstration Program (CRCSDP), which provided colorectal cancer screening and diagnostic services between 2005 and 2009 for asymptomatic, low-income, underinsured, or uninsured individuals at 5 sites around the United States.
For each client screened in the CRCSDP, a standardized set of colorectal cancer clinical data elements was collected. Data regarding client age, screening history, risk level, screening test indication, results, and recommendation for the next test were analyzed. For colonoscopies, data were analyzed regarding whether the cecum was reached, bowel preparation was adequate, and identified lesions were completely removed.
Overall, 53% of the fecal occult blood tests (FOBTs) (2295 tests) distributed were completed and returned. At the 2 sites with adequate numbers of FOBTs, 77% and 97%, respectively, of clients with positive results received follow-up colonoscopies. Site-specific cecal intubation rates ranged from 90% to 98%. Adenoma detection rates were 32% for men and 21% for women. For approximately one-third of colonoscopies, the recommended interval to the next test was shorter than recommended by national guidelines. At some sites, endoscopists failed to report on the adequacy of bowel preparation and completeness of polyp removal.
Screening for colorectal cancer can decrease the incidence of and death from this disease and is recommended in clinical practice guidelines.[1-6] To achieve the maximum benefit with minimal harm, screening must be implemented appropriately, with adequate attention to quality assurance. Problems with the implementation of colorectal cancer screening in clinical practice have been well documented for all of the recommended screening methods, including the 2 most commonly used options: fecal occult blood test (FOBT) and colonoscopy.[7-12]
The collection of specimens for FOBT in the clinician's office at the time of digital rectal examination rather than by the patient at home is a common practice, even though the in-office test has extremely low sensitivity and is not recommended for colorectal cancer screening.[8-10] Achieving high return rates when patients are given home tests is most likely the greatest challenge to the successful implementation of an FOBT screening program. Although FOBT can only be effective as a screening test when positive results are followed by complete examination of the colon and removal of identified polyps or cancers, a lack of follow-up of positive FOBTs has been documented in numerous local studies and national surveys. Approximately one-third of individuals who reported having had a positive FOBT in the National Health Interview Surveys conducted in both 2000 and 2005 reported no follow-up to their positive tests.
Wide variations in the quality of performance and documentation of colonoscopy, the most commonly used colorectal cancer screening method, have been described and are a growing concern in the gastroenterology and public health communities.[11, 12] Numerous studies have found a wide range in the rates of adenoma detection among endoscopists. The adequacy of procedure reports has been shown to vary, with some endoscopists failing to report critical elements.[13, 14] Recommendations for rescreening or surveillance intervals until the next test are often not in compliance with national guidelines.[15, 16] To reduce the variation in test performance and improve outcomes, it has been recommended that every colonoscopy practice institute a continuous quality improvement process in which quality indicators are routinely monitored and deficient performance corrected.
In this study, we examined data related to the quality of screening services with FOBT and colonoscopy from the 5 sites that participated in the Centers for Disease Control and Prevention (CDC)'s Colorectal Cancer Screening Demonstration Program (CRCSDP) between 2005 and 2009. The quality indicators assessed included return rate and appropriate follow-up of FOBT, cecal intubation rates, adenoma detection rates, reporting adequacy and recommended rescreening and surveillance intervals for colonoscopy.
MATERIALS AND METHODS
The CDC-funded CRCSDP provided screening, diagnostic, and surveillance services for colorectal cancer for asymptomatic, low-income individuals who were underinsured or uninsured in Baltimore, Maryland (Baltimore City); St. Louis, Missouri (St. Louis); the state of Nebraska (Nebraska); King, Clallam, and Jefferson counties in Washington State (Greater Seattle); and Suffolk County in New York (Suffolk County, NY) between 2005 and 2009. A detailed description of the program is provided elsewhere.[17, 18]
St. Louis, Nebraska, and Greater Seattle initially offered primary screening with guaiac-based FOBT for individuals at average risk of colorectal cancer and follow-up colonoscopy for those with positive FOBTs; individuals at an increased risk of colorectal cancer because of family history were offered colonoscopy for primary screening. Later in the program, these sites allowed individuals at average risk to undergo colonoscopy for primary screening. In Baltimore City and Suffolk County, NY, the primary screening test was colonoscopy for all clients, regardless of whether they were of average or increased risk. At all 5 sites, individuals with a personal history of colorectal cancer or adenomas underwent colonoscopy for surveillance.
For each client screened in the CRCSDP, a standardized set of colorectal cancer clinical data elements (CCDEs) was collected and provided to the CDC. The CCDEs included information regarding a client's age, personal history of colorectal polyps or cancer, and history of screening, which could be either self-reported or taken from the client's medical record. The CCDEs also included information concerning whether the client was considered to be at an increased risk because of a family history of colorectal cancer; each site was allowed to define its own criteria for increased risk. For each screening and diagnostic test provided as part of the CRCSDP, test date, test indication, results, and recommendations for the next test were collected. The data described herein are from the CCDEs provided to the CDC at the end of the CRCSDP. Our analysis included an examination of the first screening test obtained by each client for which a result was reported. The data we report on colonoscopy include procedures performed either as the primary screening test; as follow-up to positive FOBTs; or for surveillance after a diagnosis of cancer or adenoma, except where indicated.
For each colonoscopy, the CCDEs included information regarding whether the cecum was reached, whether the bowel preparation was considered adequate by the clinician performing the procedure, and the clinician's recommendation for which test the client should have next and when.
For each polyp or lesion identified, information was collected concerning the size, location, method and completeness of removal, and histology. This information was obtained by staff at each site from the endoscopy and pathology reports. Because endoscopy reporting was not standardized for the CRCSDP, site staff occasionally needed to convert the terms found in reports (eg, to describe bowel preparation quality) to fit the categories specified in the CCDEs.
We considered clients to be at average risk of colorectal cancer if they did not report any personal history of colorectal cancer or adenomas and were not considered by staff at the site to be at an increased risk because of family history. We categorized each colonoscopy as complete or incomplete; incomplete colonoscopies were defined as those for which the cecum was not reached, bowel preparation was inadequate, and/or identified polyps or lesions were not completely removed.
We computed the cecal intubation rate as the percentage of colonoscopies in which the cecum was reached. We computed the adenoma detection rate as the percentage of colonoscopies in which at least 1 adenoma was reported; we excluded clients who reported having undergone colonoscopy before receiving services from the CRCSDP because adenomas are less prevalent among people who have undergone a previous colonoscopy. Because adenoma prevalence varies by sex and age, we computed sex-specific adenoma detection rates for clients aged ≥ 50 years to allow comparison with published rates. We limited our assessment of clinician recommendation for the next test to those clients at average risk because individuals at increased risk may require testing at shorter intervals. For our analysis of recommendations for rescreening or surveillance intervals, we did not include data from Baltimore City (n = 462 clients) because its policy on data entry for this variable limited the recommended interval to program guidelines; recommendations that deviated from these guidelines were recorded elsewhere and not included in the CCDEs.
Fecal Occult Blood Test
Overall, 53% of the FOBTs that were distributed to clients were completed and returned. Percentages ranged from 47% of clients in Nebraska (1319 of 2813 clients) to 63% of clients in Greater Seattle (909 of 1447 clients) and 63% of clients in St. Louis (67 of 107 clients). Of all the clients who underwent FOBT, 1.9% were considered to be at an increased risk of colorectal cancer. St. Louis is not included in further FOBT analyses because the number of FOBTs collected was small.
Both Nebraska and Greater Seattle used high-sensitivity guaiac tests, such as the Hemoccult SENSA test (Beckman Coulter, Brea, Calif). In Nebraska, slides were developed in a central laboratory; in Greater Seattle, they were developed in community clinics. Of the 1319 clients in Nebraska who had at least 1 FOBT, 75 (5.7%) had a positive result on the first test; 73 (97%) of these clients received a follow-up colonoscopy in the CRCSDP. Of the 909 clients in Greater Seattle who underwent FOBT, 154 (17%) had a positive first test, 118 of whom (77%) received a follow-up colonoscopy. For 19 of the clients who did not receive a follow-up colonoscopy (50%) in the CRCSDP, the reason was patient refusal.
Approximately one-fifth of clients with a positive FOBT in either Nebraska or Greater Seattle underwent a colonoscopy within 60 days of the positive FOBT; approximately two-thirds underwent a colonoscopy within 120 days (Fig. 1).
A total of 3215 individuals underwent colonoscopy in the CRCSDP, either as a primary screening test (n = 2935), to follow up a positive FOBT (n = 210), or for surveillance in those clients with a personal history of colorectal cancer or adenomas (n = 70). Several indicators of quality are presented in Table 1 for initial colonoscopies received in the CRCSDP.
Table 1. Colonoscopy Quality Indicatorsa by Site in the CDC's Colorectal Cancer Screening Demonstration Program, 2005 Through 2009 (n=3215)
Suffolk County, NY
Abbreviation: CDC, Centers for Disease Control and Prevention.
Initial Colorectal Cancer Screening Demonstration Program colonoscopy received per client.
Cecal intubation rates ranged from 89.7% in St. Louis to 97.8% in Nebraska. At 4 of the 5 sites, the data indicated that the quality of bowel preparation was adequate for at least 90% of examinations. In St. Louis, 65.5% of examinations were reported as having adequate bowel preparation, 17.1% as inadequate, and 17.4% did not have information regarding bowel preparation quality. The completeness of polyp removal was not reported for 11.5% of colonoscopies in St. Louis and 11.6% in Greater Seattle. Overall, 10.5% of colonoscopies were considered incomplete either because the cecum was not reached, bowel preparation was inadequate, and/or all polyps or lesions were not completely removed. Percentages ranged from 4.5% in Nebraska to 22.3% in St. Louis (data not shown).
Adenoma detection rates were computed for first-ever colonoscopies performed either as a primary screening test or as a diagnostic test after a positive FOBT for clients aged ≥ 50 years (Table 2). Adenoma detection rates were higher overall for men (32.2%) than for women (21.1%), and ranged from 24.2% in Baltimore City to 41.5% in Greater Seattle for men and from 16.6% in Baltimore City to 26.8% in Greater Seattle for women. When colonoscopies in which the cecum was not reached and/or the bowel preparation was not adequate were excluded, adenoma detection rates changed only slightly (data not shown). Adenoma detection rates were higher for clients at an increased risk because of personal or family history (39.3% for men and 25.6% for women) compared with those at average risk (31.1% for men and 19.6% for women). The numbers of clients at an increased risk at each site were too small, especially for women, to allow for the calculation of meaningful site-specific adenoma detection rates. The adenoma detection rate for clients in Nebraska (n = 69) and Greater Seattle (n = 107) who underwent a colonoscopy to follow up a positive FOBT was approximately 20%.
Table 2. Number and Percentagea of Clients with ≥1 Adenoma Detected on First-Ever Colonoscopyb Among Clients Aged ≥50 Years in the CDC's Colorectal Cancer Screening Demonstration Program, 2005 Through 2009 (n=2693)
Suffolk County, NY
Abbreviation: CDC, Centers for Disease Control and Prevention.
The percentage of clients with ≥1 adenoma detected is known as the adenoma detection rate.
Includes colonoscopies performed as the primary screening test (all sites) or to follow up on a positive fecal occult blood test (Nebraska and Greater Seattle). Clients who reported undergoing colonoscopy before enrollment were excluded.
Rescreening and Surveillance Recommendations After Colonoscopy
A total of 1606 clients at average risk underwent complete first colonoscopies in the CRCSDP in St. Louis; Nebraska; Suffolk County, NY; or Greater Seattle, either as their primary screening test or to follow up positive FOBTs. Sixty-three (3.9%) clients were excluded because their data regarding screening outcome or recommended interval to the next test were not complete. Of the 1543 clients with complete data, 2 were diagnosed with cancer. The recommended interval to the next test for the other clients is shown in Table 3; 4 clients who had serrated adenoma(s) as their most severe finding are not included in Table 3 because to the best of our knowledge, surveillance recommendations for this finding were not specified in guidelines published at that time. For comparison, intervals recommended in national screening and surveillance guidelines are provided in Table 4.19
Table 3. Recommended Interval to Next Test, Based on Results of the Initial Colonoscopy in the CDC's Colorectal Cancer Screening Demonstration Program, 2005 Through 2009 (n=1537)a
Interval to Next Colonoscopy
>1 to 2 Years
>3 to <5 Years
>5 to <10 Years
Abbreviation: CDC, Centers for Disease Control and Prevention.
Includes clients at average risk who underwent an initial colonoscopy as a primary screening test or to follow up a positive fecal occult blood test (FOBT). Data from Baltimore City were not included. Clients whose initial colonoscopy was incomplete were excluded. Seventy-six cases were excluded because of incomplete data and 6 cases because of findings of serrated adenoma or cancer. The numbers in bold type represent intervals that adhere to national guidelines, available from Winawer S, Zauber AG, Fletcher RH, et al. Guidelines for colonoscopy surveillance after polypectomy: a consensus update of the US Multi-Society Task Force on Colorectal Cancer and the American Cancer Society. Gastroenterology. 2006;130:1872-1885.19
FOBT recommended instead of colonoscopy for 2 cases.
FOBT recommended instead of colonoscopy for 1 case.
FOBT recommended instead of colonoscopy for 1 case.
1-2 tubular adenomas measuring <1 cm with no high-grade dysplasia or villous histology
3-10 adenomas, or at least 1 adenoma measuring ≥1 cm or at least 1 adenoma with high-grade dysplasia or villous histology
Table 4. Recommended Surveillance Intervals After Colonoscopy in Patients at Average Riska
Interval to Next Colonoscopy, Years
Adapted from Winawer S, Zauber AG, Fletcher RH, et al. Guidelines for colonoscopy surveillance after polypectomy: a consensus update of the US Multi-Society Task Force on Colorectal Cancer and the American Cancer Society. Gastroenterology. 2006;130:1872-1885. These recommendations assume that the baseline colonoscopy reached the cecum, bowel preparation was adequate, and all polyps identified were completely removed.
Since the article by Winawer et al was published, evidence has been growing that large or multiple hyperplastic polyps in the proximal colon may require earlier follow-up.
Of the clients in whom no polyps were found, 70% were told to return in 10 years and 26% were told to return in 5 years to < 10 years. For clients whose colonoscopies found only hyperplastic or other nonadenomatous polyps (eg, inflammatory, hamartomatous, etc), 35% were told to return in 10 years, 48% to return in 5 years to < 10 years, and 17% to return in ≤ 3 years. Among those clients with the most severe finding of 1 to 2 tubular adenomas measuring < 1 cm, 67% were told to return in 5 years and 30% were told to return in ≤ 3 years. Of clients with at least 1 adenoma measuring ≥ 1 cm or with villous features or high-grade dysplasia or at least 3 tubular adenomas of any size, 64% were told to return in 3 years and another 26% were told to return sooner.
Overall, 65% of the recommendations we analyzed adhered to published national guidelines, with the level of agreement varying by site. For clients in whom no polyps were found, 61% in Nebraska were told to return in 10 years, whereas 71% to 75% received this recommendation in St. Louis; Suffolk County, NY; and Greater Seattle (data not shown). The numbers of clients with other screening outcomes were too small at most sites to allow for meaningful comparisons.
To maximize the benefit of screening, programs should try not only to increase the number of eligible individuals who are screened, but they also should monitor the quality of screening and ensure that recommended clinical standards are being met. In this study, we examined indicators of the quality of screening provided at the 5 sites participating in the CDC-funded CRCSDP from 2005 to 2009. For colonoscopy, cecal intubation rates and adenoma detection rates (which are considered to be important quality indicators), reached target levels suggested by expert groups.[11, 20] However, recommended rescreening or surveillance intervals after colonoscopy were not in agreement with national guidelines for more than one-third of clients. Our study also identified a need to improve the documentation of important elements in the colonoscopy report.
FOBT is a noninvasive, low-risk option for screening, and has been demonstrated to reduce deaths in randomized controlled trials.[1-4] However, for this test to be effective, the following objectives must be met: patients must return their test kits, be rescreened on a regular basis, and receive follow-up colonoscopy if they have positive FOBTs. Meeting these requirements proved to be a formidable challenge for the CRCSDP. Although the CRCSDP was an organized program with established policies and oversight, return rates indicated substantial room for improvement, as did follow-up of positive tests at 1 site. Rescreening rates are addressed in another article in this supplement.
Both Greater Seattle and Nebraska used the higher sensitivity version of the original guaiac-based test, such as Hemoccult SENSA. The high-sensitivity test is recommended over the original test in current screening guidelines.[5, 6] The positivity rate was surprisingly high in Greater Seattle and may be associated with a failure to identify and exclude symptomatic patients (eg, those with rectal bleeding or hemorrhoids) or the failure of clients to restrict their diet for testing. Dietary restriction is more important for the high-sensitivity test than for standard guaiac-based tests, but is not necessary for the fecal immunochemical tests because they are specific for human globin. In Greater Seattle, tests were developed and interpreted in the local clinics rather than at a laboratory. Quality control policies should be in place that ensure a standard process for developing and interpreting guaiac slides.
During the past decade, the use of colonoscopy, especially for screening, has increased dramatically.[22, 23] At the same time, concerns over the quality of performance of this procedure and variations in performance among endoscopists have intensified. In an effort to improve quality, recommendations for indicators that should be measured as part of ongoing quality improvement, along with target levels for selected indicators, have been developed and are published elsewhere.[11, 12, 20, 21] For an endoscopist to effectively examine the entire mucosal surface of the colon, the bowel must be adequately cleaned and the cecum must be reached. Therefore, adequate bowel preparation and cecal intubation are key quality indicators. When either is not achieved, or when polyps are not removed completely, the colonoscopy must be repeated at a shorter interval, thereby increasing cost, burden, and risk to the patient. Cecal intubation, adequacy of bowel preparation, completeness of polyp removal, and polyp descriptors such as size are critical to determining the appropriate surveillance interval. Therefore, explicit documentation of these elements by the endoscopist in the colonoscopy report is considered an important indicator of colonoscopy that meets high-quality standards.[20, 21] We found incomplete documentation of bowel preparation quality and of completeness of polyp removal at some CRCSDP sites. Deficiencies in the documentation of important elements also have been found in other settings, including those with electronic reporting systems.[13, 14]
One key reporting element, the quality of bowel preparation, can be particularly problematic. This measure is subjective and endoscopists commonly use terms such as “excellent,” “good,” “fair,” and “poor” to describe it. In clinical practice, these terms do not have standardized definitions. The US Multi-Society Task Force on Colorectal Cancer (USMSTF) and the Quality Assurance Task Group of the National Colorectal Cancer Roundtable have recommended that bowel preparation be rated as “adequate” or “inadequate” to detect lesions measuring > 5 mm.[11, 21] In the CRCSDP, endoscopists used their usual report formats and terminology and site staff often had to classify bowel preparation quality as adequate or inadequate on the basis of the descriptors used by the endoscopists. Different site staff may have interpreted the terms differently when categorizing this data element, accounting for some of the differences in the percentage of examinations that were reported to be of adequate quality. It has been recommended that if bowel preparation is found to be inadequate in > 10% of examinations in a particular clinical practice, then preparation protocols and patient instructions should be assessed, and remedial steps should be taken to improve preparation quality.
Cecal intubation was achieved in at least 94% of examinations at 4 of the 5 sites; at the 5th site, the rate was 90%. Although relatively low rates of cecal intubation have been reported in some studies,[24-26] rates > 90% are commonly achieved, and many studies of screening colonoscopies have reported rates of ≥ 97%. The USMSTF has suggested quality improvement targets of ≥ 90% for all examinations and ≥ 95% for screening examinations for cecal intubation rates that are adjusted by excluding procedures aborted because of inadequate bowel preparation or severe colitis.
Probably the most important quality indicator for colonoscopy is the actual prevalence of adenomas detected. Most studies comparing the performance of multiple endoscopists have identified significant disparities in the rate at which polyps or adenomas are identified.[11, 12] The importance of the adenoma detection rate as a quality indicator was clearly demonstrated in a recent analysis of data from a population-based colonoscopy screening program in Poland, in which adenoma detection rates were found to be inversely related to the risk of interval colorectal cancer after screening colonoscopy. The adenoma detection rate is a function of the quality of bowel preparation, the examination technique, and the demographics of the patient population. The USMSTF has recommended that adenoma detection rates for people aged ≥ 50 years who receive first-time screening examinations should be ≥ 25% for men and ≥ 15% for women. In the United Kingdom, the National Health Service Bowel Cancer Screening Programme recently set its standard for the adenoma detection rate at ≥ 35%. Adenoma detection rates for the CRCSDP met the targets set by the USMSTF at all sites and, overall, were higher for clients at increased risk. A limitation of our assessment by risk level was that screening history, personal history of adenomas, and family history were self-reported by clients and therefore may not be accurate.
In 2006, consensus guidelines for surveillance after polypectomy were jointly published by the USMSTF and the American Cancer Society in an attempt to optimize the use of colonoscopic resources by shifting resources from unnecessarily intensive surveillance of low-risk polyps to screening (Table 4). Of the clients at average risk who underwent a complete colonoscopy in the CRCSDP either as a primary screening test or as a follow-up test to a positive FOBT, 35% were told to return sooner than recommended by national guidelines. For colonoscopies that found only hyperplastic polyps, some of the apparent disagreement with the 2006 national guidelines might be explained by growing concern that large or multiple hyperplastic polyps may require earlier follow-up. Updated guidelines published since the CRCSDP for the first time provide recommendations for surveillance of serrated polyps. A limitation in our analysis of rescreening and surveillance recommendations was that we do not know if there were other non–neoplasia-related findings that may have influenced the clinician. We also did not have access to the detailed findings for each site. There were situations, for example, that did not fall neatly into any of the outcome categories specified in the CCDEs and therefore site staff had to make subjective decisions.
Deviations from recommended surveillance intervals have been documented in clinical practice. When surveyed, many clinicians indicate that they choose shorter surveillance intervals than recommended in clinical practice guidelines. Surveillance that occurs too frequently provides little or no benefit while exposing patients to the risk of complications, increasing costs (measured both in financial and human terms), and wasting resources that could be better used for primary screening. Analysis of the actual use of surveillance colonoscopy by clinicians in 9 areas across the United States that are participating in the Prostate, Lung, Colorectal, and Ovarian Cancer Screening Trial showed both significant overuse of colonoscopy among participants without adenomas or those with low-risk adenomas and substantial underuse among participants with advanced adenomas.
We were not able to assess quality indicators by endoscopist or clinical practice. Ideally, quality indicators should be measured at the level of the endoscopist or, at least, the practice because problematic performance by individual clinicians can easily be masked when data from large numbers of clinicians are combined. We encourage sites to monitor quality indicators by endoscopist whenever possible. In an accompanying article in this supplement, quality indicators are assessed at the endoscopist level at 1 of the CRCSDP sites.
Because the CRCSDP was an organized program with established policies and oversight, we would expect compliance with rescreening and surveillance recommendations, as well as with including recommended elements in the endoscopy report, to be lower in general clinical practice.
In the CDC's current Colorectal Cancer Control Program, the CDC and site staff monitor quality indicators on a regular basis. We encourage screening programs to raise awareness among their clinical communities about the need for the routine monitoring of quality indicators as part of an ongoing quality improvement system.
The Colorectal Cancer Screening Demonstration Program evaluated in this supplement was funded by the Centers for Disease Control and Prevention Funding Opportunity Number RFA AA030.