World-leading AI technology developed at the University of Cambridge by the directors of the Cambridge Centre for AI in Medicine and their colleagues offers a glimpse of the future of precision medicine, and unprecedented predictive power to clinicians caring for individuals with the life-limiting condition.
Accurately predicting how an individual’s chronic illness is going to progress is critical to delivering better-personalised, precision medicine. Only with such insight can a clinician and patient plan optimal treatment strategies for intervention and mitigation. Yet there is an enormous challenge in accurately predicting the clinical trajectories of people for chronic health conditions such as cystic fibrosis, cancer, cardiovascular disease and Alzheimer’s disease.
There are many reasons for this. One is that the trajectory of such complex diseases varies enormously between individuals, which is further exacerbated by individual differences in the development of complications and comorbidities.
Another aspect to address is the varied nature of healthcare data itself. An enormous amount of data is generated when a person develops a chronic illness and begins a long-term interaction with healthcare services. It is a complex mass of “static” information – such as sex, genomic measures and blood type – and time-series data such as ongoing clinical measures, test results and so on. These can add up to hundreds of clinical variables per patient, many of them changing over time. To complicate matters further, different data is collected, at different times, for different patients.
“Prediction problems in healthcare are fiendishly complex,” says Professor Mihaela van der Schaar, Director of the Cambridge Centre for AI in Medicine (CCAIM). “Even machine learning approaches, which deal in complexity, struggle to deliver meaningful benefits to patients and clinicians, and to medical science more broadly. Off-the-shelf ML solutions, so useful in many arenas, simply do not cut it in predictive medicine.”
Unlock this complexity, however, and enormous healthcare gains await. That is why several teams led by Professor van der Schaar have developed a rapidly evolving suite of world-class machine learning (ML) approaches and tools that have successfully overcome many of the challenges. These tools have the power harness the complexity of healthcare data and turn it into actionable, patient-specific clinical insights. At every step of their technological development, these novel approaches and tools have proven to be best in their class, superior not only to standard medical predictive tools but also to competing ML approaches.
Applying novel ML in cystic fibrosis
A useful way to illustrate the progress made by these Cambridge researchers is to zoom in on their work with a specific condition – cystic fibrosis. “Cystic fibrosis is an excellent exemplar of a hard-to-treat, chronic condition,” says the Co-Director of CCAIM, Andres Floto, Professor of Respiratory Biology at the University of Cambridge and Research Director of the Cambridge Centre for Lung Infection at Papworth Hospital. “It is often unclear how the disease will progress in a given individual over time, and there are multiple, competing complications that need preventative or mitigating interventions.”
CF is a genetic condition that affects a number of organs, but primarily the lungs, where it leads to progressive respiratory failure and premature death. In 2019, the median age of the 114 people with CF who died in the UK was 31. Only about half of the people born in the UK with CF in 2019 are likely to live beyond the age of 49. The management of CF involves a punishing daily burden of self-administered care. Clinical insights gained through machine learning could reduce this burden and increase longevity through increasingly personalised treatment and intervention choices, accurate clinical predictions and accelerated medical discovery.
In just two years, researchers led by professors van der Schaar and Floto have developed technology that has moved from producing ML-based predictions of lung failure in CF patients using a snapshot of patient data – itself a remarkable improvement on the previous state of the art – to dynamic predictions of individual disease trajectories, predictions of competing health risks and comorbidities, “temporal clustering” with past patients, and much more.
It is astonishing progress in a very short time, and it reveals the power of ML methods to tackle the remaining mysteries of common chronic illnesses and provide highly precise predictions of patient-specific health outcomes of unprecedented accuracy. While we take a whistle-stop tour of six of these new ML tools and their application to CF data, bear in mind that such techniques can be readily applied to other chronic diseases (and indeed have already been applied in Alzheimer’s, cardiovascular disease and several forms of cancer). Therein lies their true power.
The UK Cystic Fibrosis Registry
Cystic Fibrosis is fertile ground to explore ML methods, in part because of the UK Cystic Fibrosis Registry. It is an extensive database, managed by the UK Cystic Fibrosis Trust, that covers 99% of the UK’s CF population. It holds both static and time-series data for each CF patient, including demographic information, CFTR genotype, disease-related measures including infection data, comorbidities and complications, lung function, weight, intravenous antibiotics usage, medications, transplantations and deaths.
“Almost everyone with cystic fibrosis in the UK entrusts the Registry to hold their patient data, which is then used to ensure the best care for all people with the condition,” says Dr Janet Allen, the Director of Strategic Innovation at the Cystic Fibrosis Trust. “What’s exciting is that the approaches developed by Professor van der Schaar take this to a completely new level, developing tools to harness the complexity of the CF data. Turning such data into medical understanding is a key priority for the future of personalised healthcare.”
Each of the following six studies – the latter three of which were presented at the North American Cystic Fibrosis Conference in October 2020 – made use of the registry, for which we thank the UK Cystic Fibrosis Trust. The ongoing CF-related work of Professor van der Schaar’s team is now funded through a grant from the Cystic Fibrosis Foundation in the US.
The decline in lung function in individuals with CF is closely monitored by clinicians: when the volume of air a patient is able to exhale – a measure called forced expiratory volume (FEV1) – falls below 30% of healthy levels, this is a traditional threshold that indicates it may be time consider a lung transplant referral. But this is a complex illness, and there are many other important considerations, which makes the decision to refer an extremely challenging one. Occasionally, patients are referred too early, meaning that not only that they might be risking a major operation before it becomes imperative, but also that donor lungs are not always going to the people who need them most. What if the data from over 100 patient variables could all be factored into making a prediction of the patient’s future mortality risk so that decisions to refer are better informed?
The researchers applied AutoPrognosis to CF patients’ data. AutoPrognosis is a state-of-the-art machine learning method for prognostic scoring (see figure 1) developed by Professor van der Schaar and Ahmed Alaa of the University of California, Los Angeles.
AutoPrognosis constructed a machine learning pipeline – automatically choosing the optimal ML modelling to apply to the data – to predict 3-year mortality based on a static “snapshot” of the patient data. This is a useful prediction timeline because it takes several years for a lung transplant to occur following a referral and subsequent listing for a transplant. The modelling was conducted using baseline data from adult patients in 2012 – the most recent cohort for which 3-year mortality data was available. Rather than taking a handful of factors into consideration, AutoPrognosis was fed 115 variables for each patient. It was an agnostic, data-driven approach for discovering risk factors.
Figure 1. Given a data set and an output of clinical interest – in the case of CF, a prediction of 3-year mortality – AutoPrognosis uses Bayesian optimisation to select the best machine learning pipeline to apply to the data. The pipelines include choices for imputation, feature processing, classification and calibration.
In work published in Nature Scientific Reports, AutoPrognosis outperformed all existing models, achieving a 35% improvement in accuracy over traditional methods for clinical referral policy. Not only that, while it confirmed FEV1 as the primary variable in determining suitability for transplant, it also revealed that variables related to gas exchange (oxygenation) in the lungs were also important factors. This connection was previously unspecified.
It is a cause for celebration that ML approaches can significantly improve prognostic forecasting and support optimised referral for transplant, but we are just getting started.
Paper: “Prognostication and Risk Factors for Cystic Fibrosis via Automated Machine Learning”
Authors: Ahmed Alaa, Mihaela van der Schaar
Publication: Nature Scientific Reports, July 2018, DOI:10.1038/s41598-018-29523-2
Making a one-off prediction based on static data is very useful but limited in scope. After all, a patient’s disease develops over time; it has a trajectory. The ability to simultaneously forecast multiple clinical variables – including how quickly lung function will decline or the likelihood of developing CF-related diabetes or liver disease – would deliver enormous benefits to individuals with CF and their clinical carers. Such technology would allow for focused screening or planned interventions.
Conventional machine learning typically uses Markovian models to predict a patient’s future state by looking solely at their current state. This is a simplistic approach. For example, even if two CF patients are at a similar stage of disease progression today, the lung function of one of these patients may be declining at twice the rate as that of the other – something Morkovian models miss. Because a patient’s medical history influences their future clinical trajectory, the researchers integrated this historical clinical information into deep-learning ML architecture using a Recurrent Neural Network (RNN), which dynamically updates its memory state over time as new patient data becomes available via their electronic health record.
The model uses an attention mechanism to observe the patient’s clinical history and map it to weights that determine how much influence the patient’s previous disease states and clinical events have on the future course of their disease. These attention weights can be used to explain the relationships between the patient’s previous clinical events and their disease progression – excellent news for clinicians aiming to piece together and explain their patient’s clinical narrative (see figure 2). What’s more, this “attentive state-space model” can also maintain superior predictive accuracy for future clinical outcomes for a specific patient.
Figure 2. This diagram illustrates (using synthetic data), how a specific CF patient’s condition develops over time. By the patient’s latest visit (3 December 2016), they have already developed cirrhosis of the liver and diabetes, had a lung transplant and then developed kidney disease, and their FEV1 is 54%. The Attentive State-Space Model judges this person to be in rapid decline, with a 32% chance of dying within 12 months (and a likelihood of developing asthma two years in the future). At the bottom right, the model illustrates the relative contribution to the mortality risk of the comorbidities and lung transplant.
Using data from a cohort of patients enrolled in the CF Registry, the model learned for itself (in an unsupervised way) a representation of three stages of CF progression, where each stage corresponded to a clinically distinguishable phenotype of disease activity. For example, the FEV1 biomarker suggested by the model for ML-derived stage-three disease matched the cut-off values of FEV1 used by current medical guidelines for potentially referring critically ill patients for a lung transplant. The model also identified the risks of various comorbidities – such as diabetes, asthma and depression – in each of the three stages. This shows how the model’s unsupervised learning can be translated into rich, clinically actionable information to support decision-making.
To test the model’s predictive power, the team used it to sequentially predict the 1-year risk of four comorbidities and one type of infection (Pseudomonas aeruginosa) common in the CF population. The model’s predictions on all five tests were more accurate than existing ML models tailored and trained for these specific prediction tasks. A fantastic achievement.
It is because attentive state-space models combine the interpretational benefits of probabilistic models and the predictive strength of deep learning that we envision them being put to use in large-scale disease phenotyping and for supporting clinical decision-making.
Paper: “Attentive State-Space Modeling of Disease Progression”
Authors: Ahmed Alaa, Mihaela van der Schaar
Conference: Neural Information Processing Systems (NeurIPS) 2019
The most common cause of death in an individual with CF is respiratory failure, but some patients may in fact be at greater risk of death from some other cause. It is important that clinicians and patients be aware of these likelihoods when developing treatment plans.
So building on the previous work with the RNN and using the data and variables drawn from the CF registry, the next goal was to develop a best-in-class dynamic survival-analysis tool. The Cambridge researchers called it Dynamic-DeepHit.
The new model assessed two competing risks: death from respiratory failure, and death from all other causes. Dynamic-DeepHit learned the complex relationships between the patient’s clinical trajectories and survival probabilities directly (see figure 3). In other words, unlike existing works in statistics, this ML method was able to learn data-driven associations between the longitudinal data and the various associated risks – it needed no prior assumptions about underlying disease and time-to-event processes. In addition, the system updates its predictions when new data become available.
Figure 3. Dynamic-DeepHit learns from the spectrum of time-series data to predict death from respiratory failure and other causes. The example variables illustrated on the left are body-mass index (BMI), forced expiratory volume (FEV1%) and the number of days spent receiving invasive antibiotics (ABX).
When the team compared Dynamic-DeepHit’s results with other prediction methods, including joint models and landmarking models based on Cox Proportional Hazards and Random Survival Forests, they found its discrimination between the two competing risks beat the best benchmark, providing a significant improvement in discriminating individual risks of different forms of health failures due to cystic fibrosis.
In addition, the analysis incorporated post-processing statistics that enabled the team to identify the patient variables, and the timing of medical events, that had particularly significant impacts on the patient’s future health outcomes. The features that most significantly affected patients’ risk of dying were FEV1% predicted, the days they spent (in hospital or at home) receiving invasive antibiotic treatments, BMI and oxygen therapies.
Paper: “Dynamic-DeepHit: A Deep Learning Approach for Dynamic Survival Analysis With Competing Risks Based on Longitudinal Data”
Authors: Changhee Lee, Jinsung Yoon and Mihaela can der Schaar
Publication: IEEE Transactions on Biomedical Engineering, January 2020, DOI: 10.1109/TBME.2019.2909027.
This is where our best-in-class medical ML technology offers increasing patient benefit and where medical discovery can really start to accelerate. The progression of CF is highly variable between individuals, and therefore hard to predict. The identification of subgroups of individuals with similar patterns of progression could provide insight into how the disease develops, while also informing more patient-specific management strategies. Patient clustering approaches in general are valued across the entire spectrum of chronic medical conditions, of course. The difference is, ML offers super-specific clustering (see figure 4).
Figure 4. To predict the probable clinical outcome for a new patient, they can be clustered with many past patients who were most similar in terms of their characteristics and disease trajectory (i.e. their matching “temporal phenotypes”). In this example, at time t1 the patient was most similar to previous patients in phenotype cluster 1, who typically experienced outcome B. However, the patient’s disease trajectory continued to evolve dynamically and at time t2 they were in cluster 2, in which outcome A was more likely.
Again using CF registry data, the team used a Recurrent Neural Network-based architecture to assign individual CF patients to clusters that shared similar future outcomes of interest, specifically the development of comorbidities within the next year (see figure 5). Once again, this novel method achieved improvements in both clustering and prediction performance compared with conventional clustering methods, which do not incorporate future clinical outcomes of interest.
Figure 5. Patient A has cystic fibrosis, which can result in a host of comorbidities. The Recurrent Neural Network applied to Patient A’s early data places him in Cluster 1, with very low risk of developing asthma or liver disease. After two years, further clinical data result in his being moved to Cluster 9, with a high probability of developing asthma within a year, which then occurs. At year 5, his condition has progressed still further, and the RNN now judges him to be in Cluster 3, with a high chance of developing liver disease, which also occurs within a year of prognosis.
The upshot here is that the deep learning approach to temporal clustering of individuals with CF represents a huge opportunity to understand not only the pathogenesis of CF, but also the progression of CF at the level of the individual. This is precision medicine that enables an increase in the specificity of conversations between patients and their clinicians about health, prognosis and prevention.
NACFC 2020 research poster: “Temporal Phenotyping using Deep Predictive Clustering of Disease Progression”
Authors: Changhee Lee, R. Andres Floto, Mihaela van der Schaar.
Also see paper: “Temporal Phenotyping using Deep Predictive Clustering of Disease Progression”
Authors: Changhee Lee, Mihaela van der Schaar
Conference: International Conference on Machine Learning (ICML) 2020
Here is more novel ML technology, which can explore the robustness of hypothesised patient subgroups based on their disease trajectories. The rate of lung function decline and treatment responses differ between people with CF, such as between men and women. But does the rate of decline differ between patients taking a new drug treatment, for example?
Robust methods that could identify subgroups with differing lung function trajectories or responses to treatments would not only deliver significant insights into the pathophysiology of CF but also provide opportunities for greater precision when tailoring patient-specific therapies.
This is a recent problem in medical ML: while standard hypothesis testing can be used to define such significant differences, most tests are restricted to a snapshot of data and cannot test the difference between trajectories that are based on irregularly sampled time-series data. So the researchers developed an ML-based hypothesis test to do just that (see figure 6).
Figure 6. Hypothesis testing with lung function trajectories of CF patients. The left panel shows the irregularly sampled measurements of lung function for four patients. Each trajectory is interpreted as a distribution of potential FEV1 measurements over time (middle panel). The problem then becomes comparing sets of distributions rather than individual measurements. For example, one may compare two groups of patients (the blue and red groups in the right-hand panel), to determine whether there are significant differences in patient trajectories.
The researchers took retrospective longitudinal data from the UK Cystic Fibrosis Registry to systematically test for differences in lung function decline. They demonstrate gains in power – i.e. the proportion of correctly rejected differences in two synthetically generated populations – of up to 50% compared with tests that do not capture the uncertainty between observations. (Synthetic data was used to quantitatively evaluate the test because the ground-truth aetiology of CF is often unknown.)
Then using real CF data, the method identified – in a solely data-driven manner – several important subgroups which had significant differences in lung function trajectory. These subgroups were defined by gender, by their smoking status, the presence (or not) of specific genetic mutations, and the presence (or not) of the Pseudomonas aeruginosa bacterium.
In CF and beyond, these tests could be used to quantify the evidence for a large number of scientific questions – for example, is blood pressure over time associated with lung function over time? This method will enable the automatic discovery of significant and previously unknown relationships in healthcare data.
NACFC 2020 research poster: “AI-Based Hypothesis Testing for Individuals Cystic Fibrosis”
Authors: Alexis Bellot, R. Andres Floto, Mihaela van der Schaar.
A big challenge in making medical predictions is that the optimal ML model to apply to the data often varies over time, due to changes in the population affected, disease characteristics, environmental factors, new interventions and treatments, at both the individual and population level. Medicine is a “messy” business, and such irregularities are common, so the best ML model to use may change over time. This is an elephant in the room that is rarely addressed.
The Cambridge researchers tackled this elephant head on, using a novel method called Stepwise Model Selection via Deep Kernel Learning (SMS-DKL). The system automatically and dynamically selects the best model at each data timestep, as more patient data accumulates.
The team applied this SMS-DKL approach retrospectively to longitudinal data in the UK CF registry to optimise the model selection for predictions based on 90 time-series patient variables, focusing on three key clinical outcomes of particular interest for individuals with CF: 1-year mortality, the development of allergic bronchopulmonary aspergillosis (ABPA) and E. Coli infection. SMS-DKL outperformed existing model selection methods not only using standard performance metrics but also in terms of its speed in finding the optimal ML model.
What this demonstrates is that the already-superior predictive power of ML models can be reliably optimised, automatically, in practise.
NACFC 2020 research poster: “Predicting Clinical Trajectories in CF using Automatic Longitudinal Selection of Deep Learning Models”
Authors: Yao Zhang, R. Andres Floto, Mihaela van der Schaar.
Looking to the future
Taken together, this suite of new tools offers tremendous potential benefit to everyone in the CF ecosystem, from patients to clinicians and medical researchers. And this benefit could extend far beyond the UK. Data registries for CF exist in most Western countries, including the US, Canada, and across Europe. Applying these tools to the data in these other registries will significantly enhance their predictive power and multiply the benefit to patients, Dr Allen notes.
That’s because, while the UK CF Registry currently collects data on over 10,600 people with cystic fibrosis, with data going back to 1996, the US and Canadian registries represent a combined CF population of over 35,000 people, with data stretching back to the 1960s. While the UK CF Registry has recently moved from annual-only to more frequent data collection, the US and Canadian registries have been recording multiple visits per year for some time.
“Our medical ML technology has matured rapidly, and it is ready to be deployed,” says Professor van der Schaar. “The time has come to bring its clear benefits to the individuals who need it most – in this case, the people living with cystic fibrosis. This means collaborating further with clinicians and increasing our engagement with wider healthcare systems and with data guardians beyond the UK.”
Another reason that this line of research is game-changing is that the ML techniques that CCAIM researchers have successfully applied to CF data can be straightforwardly reconfigured to tackle the static and time-series data associated with any number of common chronic conditions, to the great potential benefit of patients, clinicians and medical science in general. For example, technologies to predict clinical trajectories, developed by researchers led by Professor van der Schaar, have already been applied to conditions as diverse as Alzheimer’s, cardiovascular disease and several forms of cancer, all with exciting results.
Machine learning technologies have proven to be adept at predicting the clinical trajectories of people with long-term health conditions, and innovation will continue at pace. The patient-centred revolution in precision healthcare is beginning, by enabling and empowering clinicians and researchers to extract greater value from the growing availability of healthcare data. Improvements beckon in every aspect of healthcare, from prevention and diagnosis to prognosis, treatment and patient monitoring.
The challenge ahead is to realise the potential of these remarkable tools by making them available to clinicians and hospitals around the world, where they can quickly help to improve and save the lives of countless people living with chronic illness.
By Sean O’Neill (Communications Lead, CCAIM)
Thank you for reading. We will leave you with a diagram summarising the key aspects of the groundbreaking research covered in this article, and showing how it feeds into the coming revolution in precision medicine.
Additional reading:
Paper: “Opportunities for Machine Learning to Transform Care for People with Cystic Fibrosis”
Authors: Mahed Abroshan, Ahmed M. Alaa, Oli Rayner, Mihaela van der Schaar
Journal of Cystic Fibrosis 19, 2020. DOI: 10.1016/j.jcf.2020.01.002
Advances in machine learning for cystic fibrosis revealed at the North American CF Conference 2020
