World-leading machine learning (ML) technology developed at the University of Cambridge promises powerful new prediction and analytical tools to support the clinical care of individuals with the life-limiting condition. The work is being presented at the North American Cystic Fibrosis Conference 2020, running online from 7 to 23 October.
The three research projects highlighted below were led by the Director of the Cambridge Centre for AI in Medicine (CCAIM), Professor Mihaela van der Schaar, who also runs the Van der Schaar Laboratory, and Professor Andres Floto, Co-Director of CCAIM and Principal Investigator for the Floto Lab, all at the University of Cambridge.
Cystic Fibrosis (CF) is fertile ground for developing ML methods, in part because of the UK Cystic Fibrosis Registry – an extensive database that covers 99% of the UK’s CF population. The CF Registry holds metadata for each CF patient, including demographic information, CFTR genotype, disease-related measures including infection data, comorbidities and complications, lung function, weight, intravenous antibiotics usage, medications, transplantations and deaths. Each of the following projects made use of the registry, for which we thank its sponsor and manager, the Cystic Fibrosis Trust.
Project 1: Predicting Clinical Trajectories in CF through Deep Learning-Based Temporal Clustering
The progression of CF is highly variable between individuals, and hard to predict. The identification of subgroups of individuals with similar patterns of progression can provide insight into how the disease develops, while also informing personalised care-management strategies.
Fig 1. To predict the probable clinical outcome for a new patient, they can be clustered by temporal phenotype with many past patients who were most similar in terms of their disease trajectory. In this example, at time t1 the patient’s history of clinical observations is most similar to previous patients in phenotype cluster 1, who typically experienced outcome B. However, the patient’s disease trajectory continued to evolve dynamically and at time t2 the patient was in cluster 2, in which outcome A was more likely.
Experiment 1: True class is available and tractable
Set-up: Next-year development of three comorbidities: diabetes, ABPA, and intestinal obstruction.
The novel approach achieved highly significant (P < 0.001) improvements in both clustering and prediction performance over existing methods.
Experiment 2: True class is not available or intractable
Set-up: Next-year development of 22 comorbidities
Again using CF registry data, the team used a Recurrent Neural Network to assign individual CF patients to clusters that shared similar future outcomes of interest, specifically the development of a range of comorbidities within the next year. The approach updates temporal phenotype (cluster classification) as new observations are collected over time (see Fig 2). Once again, this novel method achieved improvements in both clustering and prediction performance compared with existing clustering methods, which do not incorporate future clinical outcomes of interest.
Fig 2. Patient A has cystic fibrosis, which can result in a host of comorbidities. The Recurrent Neural Network (RNN) applied to Patient A’s early data places him in Cluster 1, with very low risk of developing asthma or liver disease. After two years and further clinical data collected, the RNN moves him to Cluster 9, with a high probability of developing asthma within a year, which then occurs. At year 5, his condition has progressed again, and the RNN now judges him to be in Cluster 3, with a high chance of developing liver disease, which also occurs within a year of prognosis.
This deep learning approach to temporal clustering of individuals with CF represents a huge opportunity to understand not only the pathogenesis of CF, but also the progression of CF at the level of the individual, and allows for the personalisation of conversations between patients and their clinicians about health, prognosis and prevention.
Authors: Changhee Lee, R. Andres Floto, Mihaela van der Schaar.
Find the NACFC 2020 research poster here.
Also see paper: “Temporal Phenotyping using Deep Predictive Clustering of Disease Progression”
Authors: Changhee Lee, Mihaela van der Schaar
Conference: International Conference on Machine Learning (ICML) 2020
Project 2: AI-Based Hypothesis Testing for Individuals with Cystic Fibrosis
This new ML technology can explore the robustness of hypothesised patient subgroups based on groups of disease trajectories. The rate of lung function decline and treatment responses differ between people with CF – between men and women, for example. Robust methods that could identify subgroups with differing lung-function trajectories (or responses to treatments such as new drugs) would not only deliver enormous insights into the pathophysiology of CF but also provide opportunities for personalised therapy.
Designing tests to better understand the structure underlying the different variables related to Cystic Fibrosis is challenging because the available data are longitudinal, sometimes irregularly and sparsely sampled and with some degree of uncertainty. Existing tests largely assume data to be fixed dimensional vectors and observed without noise, but these assumptions coerce the data into convenient forms, invalidating any guarantees of such tests. This new ML-based hypothesis test, called Robust Maximum Mean Discrepancy (RMMD), can test the differences between clinical trajectories based on irregularly sampled time-series data (see Fig 3).
Fig 3. Hypothesis testing with lung function trajectories of CF patients: The left panel shows the observed measurement of lung function over time for four patients. Each trajectory is interpreted as a distribution of potential FEV1 measurements over time (middle panel). The problem then becomes comparing sets of distributions rather than individual measurements. For example, one may compare two groups of patients (for example, the blue and red groups in the right-hand panel), to determine whether there are significant differences in patient trajectories.
The researchers took retrospective longitudinal data from the UK Cystic Fibrosis Registry to systematically test for differences in lung function decline. They demonstrate gains in power (i.e. the proportion of correctly rejected differences in two synthetically generated populations) of up to 50% compared with tests that do not capture the uncertainty between observations. Synthetic data were used to quantitatively evaluate the test because the ground-truth aetiology of CF is often unknown.
Fig 4. Robust Maximum Mean Discrepancy (RMMD) outperforms competing technologies in every experiment (while showing good control of type I error).
Then using real CF data, the method identified – in a principled, solely data-driven manner – several important subgroups which had significant differences in lung function trajectory. These subgroups were defined by gender, by their smoking status, the presence (or not) of specific genetic mutations, and the presence (or not) of the Pseudomonas aeruginosa bacterium.
What is more, in CF and beyond, these tests could be used to quantify the evidence for a large number of scientific questions – for example, is blood pressure over time associated with lung function over time? This method will also enable the automatic discovery of significant and previously unknown relationships in healthcare data.
Find the NACFC 2020 research poster here.
Authors: Alexis Bellot, R. Andres Floto, Mihaela van der Schaar.
Project 3: Predicting Clinical Trajectories in CF using Automatic Longitudinal Selection of Deep Learning Models
A big challenge in making medical predictions is that the optimal ML model to apply to the data often varies over time, due to changes in the population affected, disease characteristics, environmental factors, new interventions and treatments, at both the individual and population level.
The Cambridge researchers tackled this challenge using a novel method called Stepwise Model Selection via Deep Kernel Learning (SMS-DKL). The system automatically and dynamically selects the best model at each data timestep, as more patient data accumulates. SMS-DKL consists of a Bayesian algorithm that optimises model selection in a stepwise fashion, treating the performance at data timestep as its own black box function. This results in a multiple black-box function optimisation problem, which is solved jointly and efficiently by exploiting correlations among black-box functions using deep kernel learning.
The team applied this SMS-DKL approach retrospectively to longitudinal data in the UK CF registry to optimise the model selection for predictions based on 90 time-series variables (e.g. treatments, comorbidities, infections), focusing on three key clinical outcomes of particular interest for individuals with CF: 1-year mortality, the development of allergic bronchopulmonary aspergillosis (ABPA) and E. Coli infection.
SMS-DKL outperformed existing model selection methods using standard performance metrics (the area under the precision-recall curve (AUPRC)). SMS-DKL also outperformed existing model selection methods in terms of its speed in finding the optimal model.
These results show that the already-superior predictive power of ML models can be reliably optimised in practise.
Find the NACFC 2020 research poster here.
Authors: Yao Zhang, R. Andres Floto, Mihaela van der Schaar.
How pioneering Machine Learning can help to future-proof clinical trials in the era of COVID-19
