Tools to improve clinical trial design in urothelial cancer


  • Allison M. Deal MS,

    1. Biostatistics and Clinical Data Management Core, University of North Carolina at Chapel Hill Lineberger Comprehensive Cancer Center, Chapel Hill, North Carolina
    Search for more papers by this author
  • Matthew I. Milowsky MD

    Corresponding author
    1. Department of Medicine, Division of Hematology and Medical Oncology, University of North Carolina at Chapel Hill Lineberger Comprehensive Cancer Center, Chapel Hill, North Carolina
    • Corresponding author: Matthew I. Milowsky, MD, Division of Hematology and Oncology, University of North Carolina at Chapel Hill, third Floor Physician's Office Building, 170 Manning Drive, Chapel Hill, NC 27599; Fax: (919) 966-6735;

    Search for more papers by this author


Two reports in this issue describe a nomogram for predicting survival and a landmark analysis demonstrating the relation between 6-month and 9-month progression-free status and overall survival in patients with metastatic urothelial cancer. These 2 important tools will be used to improve future clinical trial design in patients with advanced urothelial cancer.

Metastatic urothelial cancer is a devastating disease in which there have been no therapeutic advances in over 30 years. Cisplatin-based chemotherapy is associated with a survival benefit; however, the median survival for patients is only 14 months, and <20% remain alive at 5 years. This lack of progress in urothelial cancer is related in part to the historic difficulty in conducting and completing large, randomized clinical trials with power to evaluate survival outcomes. In addition, smaller phase 2 trials that were designed based on historic controls have been problematic because of differences in pretreatment patient characteristics. In the 2 articles in this issue of Cancer that accompany this editorial, Galsky and colleagues present a nomogram for predicting survival and examine the relation between 6-month and 9-month progression-free status and overall survival (OS) in patients with metastatic urothelial cancer who received treatment with cisplatin-based chemotherapy.[1, 2] It is an exciting time in oncology with the development of novel agents targeting unique molecular pathways in cancer. An understanding of the biology of urothelial cancer with the elucidation of new targets and molecular subtypes has already led to trials evaluating novel therapeutics in patients with advanced disease. Although these 2 reports do not directly advance the field with a positive clinical trial of a novel therapeutic, they provide critical tools for the pretreatment stratification of patients and for the development of studies with intermediate endpoints.

Prognostic models to predict outcome have been developed successfully in many malignancies, including the incorporation of these tools for patient selection and pretreatment stratification into clinical trials. Bajorin et al developed the Memorial Sloan-Kettering Cancer Center (MSKCC) risk-score model for predicting long-term survival in patients with metastatic urothelial cancer based on the hypothesis that controlling for pretreatment patient characteristics would explain in part the variability in reported survival and would guide future trial design and interpretation.[3] In a multivariable regression analysis of 203 patients with unresectable or metastatic urothelial cancer who received treatment with methotrexate, vinblastine, doxorubicin, and cisplatin (M-VAC), 2 factors—a Karnofsky performance status (KPS) <80% and visceral (lung, liver, or bone) metastases—had independent prognostic significance. The median survival for patients in that study who had zero, 1 or 2 risk factors were 33 months, 13.4 months, and 9.3 months, respectively (P = .0001). That model has been validated in 2 randomized phase 3 clinical trials in patients receiving cisplatin-based chemotherapy.[4, 5]

Galsky and colleagues set out to improve on prognostic modeling in advanced urothelial cancer with the development of a point-based nomogram for predicting median, 1-year, 2-year, and 5-year survival in patients who received first-line cisplatin-based chemotherapy. By using individual data pooled from 384 patients enrolled on 7 phase 2 and 3 trials evaluating first-line, cisplatin-based chemotherapy in patients with metastatic urothelial cancer, a nomogram was constructed that included 5 variables: Eastern Cooperative Oncology Group performance status (0, 1, or 2), tumor site (bladder vs other), the number of visceral metastases (0, 1, 2, or 3), lymph node metastases (yes/no), and the white blood cell count (above normal limits or not). The nomogram was subjected to both internal and external validation.

A second prognostic nomogram for predicting the survival of patients with metastatic urothelial cancer who received cisplatin-based chemotherapy was recently published.[6] Individual patient data from 308 patients who were treated on 7 trials were pooled, resulting in a nomogram that included 4 pretreatment variables to predict median, 1-year, 2-year, and 5-year survival: visceral metastases, albumin, performance status, and hemoglobin. The nomogram was subjected to internal validation with a bootstrap-corrected concordance index (c-index) of 0.67 and was externally validated (using data from Cancer and Leukemia Group B Trial 90102, a phase 2 trial of cisplatin, gemcitabine, and gefitinib) with a c-index of 0.63. This model also demonstrated improved performance compared with the MSKCC risk-score model.

Nomograms have been successfully incorporated into clinical trials and also have been used to inform clinical decision making in multiple cancer types, in large part because of the user-friendly interface, which allows for their use during clinical encounters for patient counseling. Although it is easy to use, the statistical framework for these models requires close examination to ensure proper development and use. Iasonos et al provides a methodological approach for building, interpreting, and using nomograms with a recommendation for an 8-step approach to reviewing nomograms to ensure no significant methodological flaws exist.[7] These following assessment guidelines may be applied to the Galsky et al nomogram:

Data Population—Eligibility Criteria

Although the nomogram was derived from clinical trial data and not from a population-based cohort, its multicenter nature (derived from 8 clinical trials) and its consistent eligibility criteria (ie, unresectable and/or metastatic urothelial cancer and only first-line, cisplatin-based chemotherapy) are generalizable to patients with metastatic urothelial cancer, for whom cisplatin-based chemotherapy represents a standard first-line strategy.

Clinical Outcome

Survival is the appropriate endpoint for metastatic urothelial cancer, and the ability to provide survival probabilities at 1 year, 2 years, and 5 years is more tangible for providers and, most important, for patients.

Assess All Variables Under Consideration

The authors state that the list of variables was “based on published reports and authors' experience.” This is appropriate for nomogram development; however, additional variables, such as laboratory values (hemoglobin and albumin), that were included in the Apolo et al nomogram may have been considered as well. On the basis of the Harrell guidelines,[8] approximately 10 to 15 events are needed for each variable that is included in a time-to-event model; thus, given the large sample size and the number of events (384 patients with 68% deaths = 262 events), the model could have included more covariates, possibly improving the fit. However, improved model fit in the training data set may not translate into a model that has better prediction ability. The trade-off is a less user friendly nomogram and possibly one that is harder to validate in external data sets given the additional data that would be needed.


The authors fulfill the criteria for reporting estimates of effects and confidence intervals of effect measures, and they provide calibration plots demonstrating that the predictions differ only minimally from actual outcomes.

Is the Underlying Final Model Clear and Clinically Useful?

The variables needed for the nomogram are assessed in a standard fashion for these patients and are clearly categorized. Although it has been demonstrated previously that the presence of lymph node metastases and having a primary site other than the bladder are associated with inferior outcomes, they are not significant in the final model, which the authors appropriately acknowledge as possibly because of sample size.

Are the Hazard Ratios Clinically Interpretable?

Although they are not all statistically significant, all hazard ratio estimates go in the direction one would expect and have levels of magnitude on par with their level of severity.

How Did They Validate the Model?

The authors performed both internal and external validation with 186 patients from a phase 3 trial that enrolled between 1997 and 2002 and compared M-VAC with docetaxel and cisplatin. C-indices, which quantify the level of concordance between predicted probabilities and the actual chance of having the event of interest,[7] of 0.639 and 0.626 are reported in the uncorrected and bootstrapped corrected internal validation. A c-index of 0.634 was observed in the external validation. The authors acknowledge that these concordance indices suggest only moderate discrimination and represent a “starting point.”

Is the Model Better Than Existing Models?

The Galsky et al model was not directly compared with the MSKCC risk-score model, in which the Apolo et al nomogram demonstrated improved performance. Compared with the Apolo et al nomogram, the Galsky et al nomogram has 2 of the same variables (performance status and visceral metastases) with c-indices of 0.67 and 0.63, respectively. It is possible that a nomogram with all of the variables included would perform best.

Overall, the Galsky et al nomogram is well designed and represents an excellent step in the development of future nomograms with additional and/or novel prognostic variables that may lead to further improvements in performance.

Galsky et al use the same data set to evaluate the relation between 6-month and 9-month progression status and OS in patients with metastatic urothelial cancer who receive first-line cisplatin-based chemotherapy. Time-to-event endpoints provide advantages over the response rate, particularly in the screening of novel agents with mechanisms of action that are associated not with response but with prolonged stabilization of disease. The authors perform a landmark analysis for progression at 6 months and 9 months after treatment initiation in the 364 patients included in their cohort. In the landmark analysis, the median OS for those who progressed at 6 months was 3.87 months compared with 15.06 months for those without progression (P < .0001), and the median OS for those who progressed at 9 months was 5.65 months compared with 21.39 months for those without progression (P < .0001). These findings were externally validated on the 186-patient cohort treated on the phase 3 trial comparing M-VAC and docetaxel plus cisplatin. The authors conclude that progression status at 6 months and 9 months predicted OS and have the potential to serve as endpoints in phase 2 trials of novel regimens.

Use of the landmark method was originally proposed to eliminate the bias in comparing outcomes between responders and nonresponders.[9] Because group assignment (responder or not) is determined during the course of follow-up, and not before treatment initiation, using this method biases the results in favor of responders. Investigators often misinterpret significant differences in OS as evidence that response causes longer survival, when, instead, response is more likely a marker of patients who have pretreatment characteristics that favor survival. In the landmark method, patients are still grouped according to response, but response is evaluated at a single time: the landmark time. Patients who go off protocol before the landmark time are not included in the analysis, and response status is determined at the landmark time regardless of subsequent changes in tumor response.

Similar to grouping patients based on response, grouping them based on progression-free status at 6 months or 9 months cannot be determined before treatment initiation. To evaluate the association between progression status at 6 months or 9 months and OS using the landmark method, OS time must be calculated starting at the landmark time. Thus, as dictated by the landmark method, patients who died before the landmark time were excluded from the analysis, and the authors demonstrate that progression-free status at 6 months and 9 months is associated significantly with longer OS from that time point.

These 2 important articles provide the much needed tools for better clinical trial design that incorporates nomograms for pretreatment stratification and important intermediate endpoints to provide an early “go/no-go” signal. Although additional work will be required, Galsky et al have provided the foundation for clinical trials that will be designed to move the field forward.


No specific funding was disclosed.


The authors made no disclosures.