Personalized risk‐based screening for diabetic retinopathy: A multivariate approach versus the use of stratification rules

Aims To evaluate our proposed multivariate approach to identify patients who will develop sight‐threatening diabetic retinopathy (STDR) within a 1‐year screen interval, and explore the impact of simple stratification rules on prediction. Materials and methods A 7‐year dataset (2009‐2016) from people with diabetes (PWD) was analysed using a novel multivariate longitudinal discriminant approach. Level of diabetic retinopathy, assessed from routine digital screening photographs of both eyes, was jointly modelled using clinical data collected over time. Simple stratification rules based on retinopathy level were also applied and compared with the multivariate discriminant approach. Results Data from 13 103 PWD (49 520 screening episodes) were analysed. The multivariate approach accurately predicted whether patients developed STDR or not within 1 year from the time of prediction in 84.0% of patients (95% confidence interval [CI] 80.4‐89.7), compared with 56.7% (95% CI 55.5‐58.0) and 79.7% (95% CI 78.8‐80.6) achieved by the two stratification rules. While the stratification rules detected up to 95.2% (95% CI 92.2‐97.6) of the STDR cases (sensitivity) only 55.6% (95% CI 54.5‐56.7) of patients who did not develop STDR were correctly identified (specificity), compared with 85.4% (95% CI 80.4‐89.7%) and 84.0% (95% CI 80.7‐87.6%), respectively, achieved by the multivariate risk model. Conclusions Accurate prediction of progression to STDR in PWD can be achieved using a multivariate risk model whilst also maintaining desirable specificity. While simple stratification rules can achieve good levels of sensitivity, the present study indicates that their lower specificity (high false‐positive rate) would therefore necessitate a greater frequency of eye examinations.

progression, such as duration of diabetes, glycated haemoglobin (HbA1c) and type of diabetes, has been widely reported. 3,8,9,[19][20][21][22][23] Given the low annual incidence rate of STDR in the population with diabetes in developed countries (<3%), personalized risk-based screening should offer a cost-effective approach to reduce the economic burden on health systems without compromising efficacy. 8,9,24 Identifying STDR early is important, not just to enable prompt and effective treatment and thus maintenance of vision, but also to allow cost-effective screening intervals tailored to patients' needs. The potential benefit of a personalized screening approach (where patients at higher risk are screened more often than those at low risk) depends on the accuracy and validity of the predictive model used and on the associated costs. Data collected from a patient over time (longitudinal data) that capture changes in clinical markers could be used to improve the accuracy of a predictive model. In the present study, the individual trajectories of the clinical profiles in PWD were used to develop and validate a predictive model for STDR.
We jointly modelled demographic and clinical data to characterize the baseline level of retinopathy and changes in level of retinopathy over time. We applied a risk-based longitudinal multivariate approach that enables the identification of patients who will develop STDR within 1 year from the time of prediction. In the United Kingdom and elsewhere, there is a current debate as to whether a simple rule based on the patient's level of retinopathy alone or alternative multivariate clinical models should be used to determine risk-based screening intervals for STDR. [21][22][23][24][25][26][27] Risk stratification for development of STDR based on just the results of two screening episodes has been proposed by Stratton et al. 25 They observed that the annual rate of progression to STDR was 0.7% for patients with no DR at two consecutive annual digital photographic screenings, 1.9% for patients with no DR in either eye at first screening but mild non-proliferative DR (NPDR)/background DR (BDR) in just one eye at second screening, and 11% for patients with mild NPDR/BDR in both eyes at both screenings. In the present study we compare the overall accuracy of our multivariate model with two simple risk stratification rules, including the rule by Stratton et al., which has been agreed by the UK National Screening Committee to be introduced in England within the next few years. 26 2 | MATERIAL AND METHODS in a purpose-built data warehouse. Patients were offered annual screening for DR according to national recommendations. 28 When patients did not attend their first appointment for screening they were offered a second appointment (usually within 6 weeks of the first appointment Patients with STDR at the start of the prediction period were excluded from the analysis. For the purposes of the present analysis, the values of the time-dependent clinical variables closest to the time of the screen episodes (ie, annual screening episodes) were used. A complete case analysis, similar to that used by Scanlon et al., 9 was followed, and screening visits for which model covariates were not available were excluded from the final model.

| Model development and statistical analysis
We have recently developed a multivariate discriminant approach, which can be used to predict the future status of a patient using their clinical history. 29,30 In the present study, we applied this statistical approach to estimate the risk that a patient would develop STDR in either/both eyes within a 1-year period, and this was achieved by using the demographic and longitudinal primary care data to jointly model the changes in level of retinopathy over time for both eyes ( Figure S1). Our approach is based on the following rationale: two longitudinal models are generated using part of the dataset (training dataset), one for each of two possible prognostic groups (patients who develop STDR and patients who do not develop STDR within 1 year).
These two models focus on modelling the progression from no DR to mild NPDR/BDR, making use of the biochemical and demographic records of the patients. The status of a new patient is then predicted depending on which of the two models the new patient's clinical profile is statistically closest to. 29 This statistical approach calculates the risk of a new patient developing STDR within 1 year from the time of prediction, and this risk can be updated each time new data become available for the patient.
The transition from no DR to mild NPDR/BDR was modelled using a bivariate generalized linear mixed-effects model that takes into account the correlation between measurements at different time points for the same patient. It is bivariate because it captures the measurements from both the right and left eye in a single model. The grading in each eye was considered as a binary longitudinal variable, with 0 representing no DR and 1 denoting mild NPDR/BDR. Correlation between repeated measurements for a patient and between retinopathy grading in each eye was modelled using random (patientspecific) intercept terms in the mixed model.  29,30 Penalized expected deviance alongside a forward selection approach, in combination with clinical judgment, was applied to identify the relevant demographic and clinical risk factors that influence changes in retinopathy level over time. Models were compared using penalized expected deviance, which penalizes for model complexity and is suitable for complex hierarchical models. 31 As a result of the stochastic nature of the Markov chain Monte Carlo model, it is possible that different random starting values (seeds) generate slightly different models. Hence, in order to check the stability of the model, the process was generated for multiple seeds. Two training datasets, involving data from 70% of patients in each of the two prognostic groups, were used to build the model and data from the remaining 30% were used to test the predictive accuracy of the model. Training and test sets were randomly generated 100 times and the results were averaged. The statistical analyses were performed in R version 3.0.2 using the package mixAK. 32 To allow for the fact that patients have been observed for different lengths of follow-up period, time since first screening was included as a covariate in the longitudinal models. All patients had been followed up for a minimum of 2 years. To develop the models, we considered the period of time from the start of their observations up until the point 1 year before their final visit (in order to be able to predict the clinical status 1 year after). For patients who developed STDR, the final visit was defined as the time at which STDR was detected (ie, data beyond STDR detection were ignored). For patients who did not develop STDR the final visit was the last recorded visit; therefore, group memberships (whether or not the patient developed STDR within 1 year of the prediction visit) were known for all patients.
The fitted mixed models, one for each prognostic group, were used in a longitudinal discriminant analysis to predict the likelihood that a new patient would/would not develop STDR within 1 year. In particular, the likelihood of the new patient's data coming from each of the two mixed models was assessed and then weighted by the prevalence of each group to give a probability of developing STDR within 1 year. If this probability was greater than a threshold (chosen through analysis of a receiver-operating characteristic [ROC] curve), then the patient was classified as developing STDR within 1 year and otherwise they were classified as non-STDR.
The two prognostic models were subsequently used to predict for a new patient (test set) the likelihood of developing/not-developing STDR within 1 year. Intuitively, the patient is linked to the group with the model the new patient's profile is closer to. The level of accuracy of the multivariate approach was assessed using the area under the ROC curve and its 95% confidence interval (CI). We also assessed the values of sensitivity (percentage of patients among those who truly developed STDR, who were correctly identified by the model), specificity (percentage of patients among those who did not develop STDR,

| Simple stratification rules
Annual screening for DR has been adopted by several national screening programmes. 27,28,33 Risk-based stratified screening intervals are likely to be introduced in a number of countries within the next few years in order to cope with the imminent significant rise in the number of PWD. We explored with our data the overall accuracy of prediction using simple stratification rules based on retinopathy level alone to identify low-and high-risk patients (the latter group consisting of patients who are likely to develop STDR within 1 year). Ideally, lowrisk patients could be offered 2-year screening intervals or longer intervals. We defined sensitivity as the percentage of patients, out of the patients who developed STDR within 1 year, who are correctly predicted by the rule (and therefore allocated to annual screening intervals). Specificity was defined as the percentage of patients of those who did not develop STDR within 1 year, who were correctly predicted by the rule as not developing STDR within 1 year (and therefore were allocated to biennial screening intervals). We also calculated the reduction in the number of screening episodes achieved by simple stratification rules when compared with the currently recommended annual screening.
Subgroup analyses were conducted to further explore the effect of diabetes type on the classification performance.  Table 1. Screening visits for which model covariates were not available were excluded from the final model (12%).

| RESULTS
Compared with the non-STDR group, we found that patients who developed STDR during the follow-up period were more likely to be men, to be younger, to have type 1 diabetes, a longer disease duration and a higher HbA1c level, and were more likely to have missed screening appointments.
As expected, the majority of patients who developed STDR (74%) within 1-year screen interval exhibited mild NPDR/BDR in both eyes at their previous screen visit (prediction visit); only a small percentage of patients (12%) who developed STDR showed no DR in either eye at their previous screen visit. This trend was reversed for patients who did not develop STDR (79% showed no DR in both eyes and only 8% showed mild NPDR/BDR in both eyes at the time of prediction).
As expected, the risk factors for the progression from no DR to mild NPDR/BDR showed similar odds ratios (ORs) for the right and left eyes, with similar interpretation (data from both eyes were jointly modelled; the full model specification is given in Table S1). For simplicity, we report one OR for each risk factor. Ethnicity and estimated glomerular filtration rate (eGFR) were not included in the model because of the high rates of missing values observed (18% and 36%, respectively) and the lack of ethnic representation (predominantly white).   Table S1) and a remaining group with a lower initial risk of progression (although in both cases patients' initial risk of progression was much higher than in the "no STDR" group). In the group of patients who did not develop STDR, just over a third of the patients belonged to a group with a very low initial risk of progression from no DR to mild NPDR/BDR (weight = 36.2%), with the remaining patients having a higher initial risk. The model takes into account the correlation between the right and left eye outcomes through the covariance matrices.

| Accuracy of the multivariate discriminant tool
The level of accuracy of the multivariate approach shown by Figure 1 indicates

| Comparison with simple stratification rules
When we assessed the overall predictive accuracy of the simple stratifi-

| DISCUSSION
We are heading towards personalized medicine, whereby patient management can be tailored based on individual risk of disease or response to treatment. In particular, the predicted risk of developing STDR can be used to recommend personalized screening intervals.
Annual screening is currently adopted in many national screening programmes at a considerable cost to health services. 9 Longer screening intervals for those at low risk of developing STDR have already been introduced, for example, in Iceland, using a model that accounts for the level of retinopathy as well as other clinical information, including type and duration of diabetes, HbA1c level and blood pressure. [34][35][36] Additional models have recently been proposed in the literature to tailor screening intervals for DR. 8,37 Longer screening intervals for patients with a low risk of developing STDR are expected to be implemented in England and Wales within the next few years to cope with the increased economic burden triggered by the steady increase in disease prevalence and the lack of extra funding for screening. Riskbased approaches are timely in that they could be used to identify patients at high risk so that they can be closely monitored and treated earlier, while the majority of patients at low risk can be screened less often, allowing the optimization of limited health resources.
In the present paper, we report the results of a recently developed multivariate longitudinal approach 29,30 to predict the risk that a given patient with diabetes will develop STDR within 1 year. The model shows high levels of classification accuracy (sensitivity and specificity were 85.4% and 84.0%, respectively). The AUC was 0.90, which is higher than the AUCs previously reported in Scanlon et al. 9 and Aspelund et al. 35 of 0.79 and 0.76, respectively. A similar AUC (0.90) was reported in Eleuteri et al. 8 and although the specificity reported by these authors was 90%, the level of sensitivity was much lower than the sensitivity achieved with our multivariate model (67% vs. 85.4%).
There are a number of advantages to our approach. From a methodological point of view, the approach is robust against misspecification of the distribution of the random effects term, which is a term that takes into account the correlation between measurements at different time points in the model. 29 The approach has the potential to develop dynamic models with which the risk can be recalculated every time new data from the patient become available. 29,30 All 92 practices approached within the Liverpool area agreed to participate, which demonstrates the screening coverage data.
Limitations of this study include the possible misclassification in level of retinopathy during grading, the fact that the costs of misclassification were not considered (which differ between STDR and non-STDR misclassification) and the fact that only internal validation was conducted. We acknowledge that a comprehensive assessment of the model's predictive performance would require external validation using data from a different cohort; for example, the predictive accuracy of our model needs to be validated for different ethnic groups.
Our dataset, which included predominantly white patients, did not include representative samples from different ethnic groups. The differences in performance between our approach and the stratification rules must be explored using different cohorts to confirm the reproducibility of our findings.
Several multivariate regression models have been proposed over the years to predict DR risk. 9,15,16,35,36,38 In these models, retinopathy   We jointly modelled clinical data and retinopathy to predict STDR accurately. A substantial body of evidence suggests that changes in the values of certain risk factors has a beneficial effect on outcomes in diabetic retinal diseases. 3,27,41,42 The multivariate predictive model we have developed uses baseline clinical data to model changes in DR (transitions among the states no DR and mild NPDR/BDR in either eye). We conclude that long-term progression of DR is driven by the patient's overall clinical profile with respect to diabetes control and that a risk prediction model using systemic risk factor data, as well as retinopathy level, may offer a better trade-off between achieving an acceptable sensitivity, while also keeping a desirable specificity.