The Prostate Cancer Prevention Trial (PCPT) Risk Calculator is a widely used prediction tool for aiding decisions about biopsy for prostate cancer. This study hypothesized that recently reported differences between predictions from the model and findings from other cohorts were due to how prostate-specific antigen (PSA) was entered into the statistical model, and to the inclusion of protocol end-of-study biopsies for which there was no clinical indication.
Data was obtained from the 5088 PCPT participants and was used to construct the PCPT Risk Calculator. The relationship between PSA and the risk of a positive biopsy was modeled by using locally-weighted regression (lowess), an empirical estimate of actual risks observed which does not depend on a statistical model. Risks were estimated with and without the 3514 end-of-study biopsies.
For PSA levels above biopsy thresholds (∼4 ng/mL), the PCPT Risk Calculator greatly overestimated actual empirical risks (eg, 44% versus 26% at 5 ng/mL). The change in risk with increasing PSA was less among for-cause biopsies compared with the end-of-study biopsies (P = .001). Risk of high-grade disease was overestimated at PSA level of ≥ 6 ng/mL.
Prostate-specific antigen (PSA) is an imperfect marker for prostate cancer and, as such, urologists need to consider carefully other indications for biopsy. These include the digital rectal examination (DRE); the patient's age, race, and family history; prior negative biopsy; and free-to-total PSA ratio. The problem for the clinician is how to integrate separate items of information in order to make a decision about biopsy.
One promising approach is to include different risk factors in a statistical risk prediction model, the output of which is a predicted probability of prostate cancer. The Prostate Cancer Prevention Trial (PCPT) Risk Calculator for prostate cancer, and a separate calculator for high-grade (Gleason grade ≥ 7) disease, were developed men in the placebo arm of the PCPT who underwent biopsy at or before the end of 7 years of annual PSA or DRE screens. Because the PCPT required an end-of-study biopsy regardless of PSA or DRE results, the PCPT Risk Calculators are not subject to ascertainment bias, which can occur when only participants with clinical indications for biopsy are actually biopsied. The PCPT Risk Calculator appears to be relatively widely used in clinical practice. In early 2013, the calculator had recorded close to 125,000 Web views, and several external validation studies have supported its value.[2-4]
The Prostate Biopsy Collaborative Group (PBCG) was established in 2009 to combine and harmonize data from different prostate biopsy cohorts, and then apply standardized statistical analyses to each data set. The collaboration includes 10 separate prostate biopsy cohorts with a total of 25,772 biopsies and 8503 cancers. Analyses of this data set suggests the PCPT Risk Calculator gives higher than expected estimates of risk of prostate cancer for men with elevated PSAs. For example, the probability of cancer for a man with a PSA level of 8 ng/mL is approximately 30% for most cohorts, with a range of 24% to 48%. In contrast, the average risk for a man with PSA of 8 ng/mL from the PCPT Risk Calculator is nearly 55%. Furthermore, the PCPT involved repeated screening, an attribute which in other cohorts led to a flattening of the risk curve. For instance, among the population of men undergoing annual screening with a PSA below a biopsy threshold of 3 ng/mL one year and above this threshold the next, the risk of cancer did not markedly increase in the PSA range spanning 3 to 10 ng/mL. This most likely occurred because prostate cancer is relatively slow-growing, so that different levels of moderately elevated PSA simply constitute “noise” from benign disease. The PCPT Risk Calculator, conversely, indicates a steadily rising risk as PSA increases.
In this study, we reanalyzed data from the PCPT in an attempt to understand the discrepancies between the PCPT Risk Calculators and data from the PBCG. Specifically, we addressed 2 hypotheses to explain these discrepancies: the statistical technique for modeling PSA for the PCPT Risk Calculator and the inclusion of both “for-cause” and “end-of-study” biopsies in the data set.
MATERIALS AND METHODS
De-identified data from the PCPT were obtained from the study investigators and reanalyzed. The cohort comprised 5273 men in the control arm of the study who underwent biopsy. Excluded from the analysis were 182 men who did not have a PSA measurement within 6 months prior to biopsy and 3 men with PSA values greater than 100 ng/mL. The final cohort of 5088 comprised 1574 who were indicated for biopsy due to an abnormal DRE or elevated PSA ≥ 4 (“for-cause biopsy”) and 3514 men who underwent biopsy at the end of the study without a specific indication. We excluded data from repeat biopsy, with all analyses performed on each patient's first biopsy; in contrast, the PCPT Risk Calculators were based on the last biopsy for each participant and included prior negative biopsy as a predictor.
Risk of prostate cancer at a given level of PSA was derived from the PCPT Risk Calculator using the formulas available at the PCPT Risk Calculators Web site. The models requires as inputs PSA, DRE, first-degree family history of prostate cancer, and history of a prior negative biopsy; for the high-grade model, race and age are additional inputs and family history is dropped. PSA is transformed by the natural logarithm before inclusion in the logistic regression model underlying the calculator. To calculate risk for an average patient, we calculated the mean of each predictor and calculated the risk using the PCPT Risk Calculator at each level of PSA but with all other predictors at the mean. For binary predictors, we used the proportion of patients with that characteristic. As a conservative approach, we also calculated the minimum risk for a first biopsy from the PCPT Risk Calculator, by calculating the probability for a 60-year-old Caucasian male with normal DRE and no family history.
To model the relationship between PSA and prostate cancer risk, we used locally weighted scatterplot smoothing (lowess), which is a model-free method for estimating actual risks according to a single predictor, in this case PSA. In brief, the method uses regression to estimate the risk of prostate cancer at a given PSA level x, using data only from patients with PSA levels near to x. Patients with PSA levels closer to x are given higher weight than patients with PSAs further away. Multiple regressions are fit and then linked together. This allows the regression line to curve in order to fit the data. The resulting process can be thought of in terms of an average moving-window estimate of risk for each unique PSA value. The lowess estimates of risk presented in this article were estimated using an 80% bandwidth, meaning that 80% of the data are used in estimating risk at each PSA value. We varied the 80% bandwidth from 40% to 90%, and the results were nearly identical, suggesting that our findings are not sensitive to the choice of bandwidth. These analyses were repeated for the endpoint of high-grade prostate cancer, defined as a Gleason score 7 or higher. To determine whether change in risk of cancer with increasing PSA level differed between for-cause and end-of-study biopsies, we used logistic regression with PSA (entered as a linear term), reason for biopsy and their interaction term as predictors. Analyses were conducted using Stata, version 12.0 (Stata Corporation, College Station, Tex).
Characteristics of the data set used for analysis are shown in Table 1. As expected, men with positive biopsy were older, had higher PSA levels, and were more likely to be of African descent or have a family history of prostate cancer. It is also apparent, however, that differences between groups are slight. There is an apparently counterintuitive finding for DRE, with a higher prevalence of DRE positive in men without cancer among the for-cause biopsies. This is because men undergo biopsy for either positive DRE or elevated PSA level, and elevated PSA is more strongly associated with biopsy outcome than DRE.
Estimates are median (quartiles) or numbers and frequency.
Comparing positive and negative biopsy from for-cause and end-of-study biopsies combined. P values calculated using the rank sum test for continuous variables and chi-square test for categorical data.
Abbreviations: DRE, digital rectal examination; PSA, prostate-specific antigen.
Age at biopsy, y
69 (65, 73)
70 (65, 74)
67 (63, 72)
68 (64, 73)
0.9 (0.6, 1.5)
1.1 (0.8, 1.6)
3.0 (1.2, 4.7)
4.3 (2.1, 4.9)
First-degree family history of prostate cancer
Biopsy Gleason grade
Figure 1 compares the smoothed empirical risks in the PCPT cohort against predicted PCPT Risk Calculator risks of prostate cancer according to PSA level. Corresponding numerical data are shown in Table 2. It is clear that although estimates are reasonably close at low PSA levels, the PCPT Risk Calculator overestimates risk for elevated PSA. At a PSA of 8 ng/mL, for example, the empirical risk of prostate cancer on biopsy is 29% compared to 54% for the PCPT model. Even the minimum possible risk from PCPT, 49%, greatly exceeds the observed risk.
Table 2. Risk of Cancer or High Grade Cancer (Gleason Score 7+) for a Given Prostate-Specific Antigen (PSA) Level Comparing Empirical Curve Fitting to the Prostate Cancer Prevention Trial (PCPT) Risk Calculator
Empirical Curve Fitting
Average PCPT Risk
Minimum PCPT Risk
Risk of cancer
Risk of high-grade disease
The results for high-grade cancer are shown in Figure 2 and Table 2. Although there is again evidence that risk is overestimated at elevated PSA, differences are less extreme. For a PSA of 8 ng/mL, risks are 12%, 15%, and 17% for observed risks, the PCPT minimum risk and PCPT average risk, respectively.
Figure 3 shows the results of lowess curve fitting including only “for-cause” biopsies (n = 1574). As hypothesized, the observed prostate cancer risk curve flattens with increasing PSA, with essentially no increases in risk above the biopsy threshold of 4 ng/mL. For example, risk changes from 16% to 29% for PSA of 1 ng/mL compared with 4 ng/mL; the comparable increase in risk for a similar 3 ng/mL difference in PSA between 4 and 7 ng/mL is zero. This effect is attenuated for high-grade cancer. Risk does increase for PSA levels above 4 ng/mL, although at a much reduced rate compared to PSA levels below 4. The change in risk with increasing PSA was significantly less among for-cause biopsies compared with the end-of-study biopsies (P = .001).
The PCPT Risk Calculator overestimated risk of prostate cancer when applied to a subcohort of the first biopsies for select PCPT participants on the placebo arm of the PCPT. The differences in risks are of clear clinical relevance in the PSA ranges where biopsy is normally considered: at a PSA of 5 ng/mL, for example, one might imagine that a patient is at least ambivalent about biopsy at 24% risk; it is hard to believe that any patient would refuse biopsy if given close to 50:50 chance of cancer. One plausible explanation is in terms of the statistical modeling technique used to generate the PCPT model. The log-linear approach, used in the original PCPT model, assumes that risk increases by a fixed amount for a given increase in the log of PSA, regardless of the PSA level. Most PCPT participants had low PSA (71% below 2 ng/mL, 85% below 4 ng/mL), and risk increases relatively steeply with PSA when PSA is low. The assumption that risk continues to increase identically for PSA values above 4 ng/mL leads to an overestimation of risk when PSA is high.
Curve fitting restricted to “for-cause” biopsies replicated prior findings that, for men undergoing intensive screening, risk does not rise markedly as PSA increases above the biopsy threshold. Although this finding makes clinical and biological sense, it is not widely appreciated in clinical practice. Indeed, the concept of PSA velocity, which has been advocated despite clear evidence of lack of benefit,[7, 8] suggests quite the contrary.
Use of curve fitting confirms that the risks in the PCPT were highly comparable to other clinical trial cohorts. That is, when lowess methods were applied to both PCPT and PBCG cohorts, results were similar, suggesting that previously reported differences in estimates are attributable to differences in statistical methods. For example, the prostate cancer risk for a PSA of 8 ng/mL in the PCPT was 30%; in the European randomized trial cohorts, risks ranged from 31% to 34% compared with 40% to 48% in US clinical cohorts. The most reasonable hypothesis to explain the difference between research and clinical settings is that clinical work-up by a urologist after a raised PSA is effective at discriminating benign from malignant causes of PSA elevations. In clinical practice, a urologist is unlikely to biopsy a man immediately on the basis of a moderately elevated PSA, but would likely recommend that PSA test be repeated after a few weeks. The urologist will likely also conduct an examination and take a history, looking for signs and symptoms of benign disease. Analysis of the relationship between PSA and risk in such clinical cohorts leads to higher estimates of risk. To explain this effect, take 100 men with a PSA of 8 ng/mL. In one cohort, such as the PCPT or European Randomised Study of Screening for Prostate Cancer, the men are referred for immediate biopsy according to study protocol and approximately 30 are found to have cancer. This would give a risk of 30% for a PSA level of 8 ng/mL. Now let us assume that these 100 men are referred for clinical work-up. If 25 have obvious indications of benign disease, or have PSA below biopsy thresholds on repeat testing, then 75 would be biopsied, with 30 cancers found, giving a risk of 40%.
This effect may explain why the PCPT Risk Calculator has sometimes demonstrated good calibration. For example, the PCPT Risk Calculator was well-calibrated when applied to patients referred for prostate biopsy by urologists at 1 of 5 US urology clinical practice sites and those biopsied at major Canadian academic centers. This is an interesting example of “being right for the wrong reason”: the PCPT Risk Calculator overestimates risk due to its modeling approach, but the clinical cohort has higher risk than a research cohort, leading to approximately accurate estimates.
In conclusion, revision of the PCPT Risk Calculator, indeed, any proposed prostate cancer risk calculator, should take into account the nonlinear association between PSA and risk. Although it is often thought that this can be achieved through the use of log transformation, logs are not a flexible approach to modeling, because the increase in risk for a given increase in the log of PSA is fixed. Alternatives such as polynomials or splines might instead be explored. Future models might also consider the impact of both recent screening and clinical work-up on risk. Clinicians should be cautious about applying the PCPT Risk Calculator in clinical practice, particularly if it is used to guide decisions in the absence of clinical work-up.
This investigation was supported in part by the National Cancer Institute (NCI), Division of Cancer Prevention, grant CA37429, a NCI Cancer Center support grant to the University of Texas Health Science Center, San Antonio, CTRC (grant 5P30 CA0541474-18). Supported in part by funds from David H. Koch provided through the Prostate Cancer Foundation, the Sidney Kimmel Center for Prostate and Urologic Cancers, and P50-CA92629 SPORE grant from the NCI to Dr. P. T. Scardino.