News | OJ Watson

AI for epidemic modelling: Nature 2025 and tool development

Mon, 16 Feb 2026 00:00:00 +0000

I recently contributed to Nature (2025) on artificial intelligence for infectious disease epidemic modelling. The paper reviews how modern AI methods can support epidemic forecasting, surveillance, intervention planning, and model acceleration while still needing careful public-health interpretation.

Paper: https://doi.org/10.1038/s41586-024-08564-w
Related tool: emidm
Related research section: AI-enabled epidemic modelling

Alongside this, my group continues work on emulator-based approaches that improve calibration speed and scenario exploration for policy-facing analyses. The aim is practical: make complex infectious disease models easier to interrogate under uncertainty, without losing sight of the assumptions that make those models useful.

Humanitarian modelling: forecast evaluation and policy translation

Mon, 16 Feb 2026 00:00:00 +0000

A continuing focus is improving how epidemic forecasts are evaluated and interpreted for policy decisions in resource-constrained and crisis contexts.

medRxiv preprint: https://doi.org/10.1101/2025.08.11.25333414
SocArXiv perspective: https://doi.org/10.31235/osf.io/tcjqs_v1
Related tools: vrcmort and vpdsus

These outputs sit alongside ongoing work on mortality estimation, vaccine-preventable disease risk, cholera anticipatory action, and the translation of model evidence into humanitarian decision-making.

See the Humanitarian evidence for humanitarian operations section for context.

Lancet Global Health 2024: impact of the 100 Days Mission

Mon, 16 Feb 2026 00:00:00 +0000

Our Lancet Global Health (2024) study quantified the expected impact of accelerating vaccine availability under the 100 Days Mission framework.

Paper: https://doi.org/10.1016/S2214-109X(24)00286-9
Related prior work: Global impact of first-year COVID-19 vaccination (Lancet ID, 2022)
Related tools: squire, nimue, vece

This work estimates how much faster vaccine availability could change pandemic outcomes, and where the benefits are most likely to depend on realistic delivery and access constraints. It is a useful link between methodological modelling and the operational planning questions that determine whether vaccines reach people in time.

This sits within the Vaccine impact and preparedness strand of my research.

Nature Medicine 2025: global pfhrp2/3 deletion risk

Mon, 16 Feb 2026 00:00:00 +0000

A key recent output is our Nature Medicine (2025) paper on the global risk of selection and spread of Plasmodium falciparum pfhrp2/3 deletions.

Paper: https://doi.org/10.1038/S41591-025-03974-3
Related modelling background: eLife 2017 pfhrp2 deletion modelling
Related software: hrp2malaRia and hrpup

This work builds on several years of modelling, surveillance, and policy-facing analysis around malaria rapid diagnostic test performance. The central question is an operational one: where and when might pfhrp2/3 deletions undermine diagnosis, and how should surveillance adapt?

The paper links directly to my broader Malaria research and WHO-facing policy support activities.

Introduction to squire

Mon, 04 Jan 2021 08:38:52 -0800

This is an overview of the squire package, which is a deterministic model of COVID-19 transmission. The squire model was developed as part of the Imperial College COVID-19 Response Team and has been key in a number of our COVID-19 projects and publications to date, including:

This is a brief overview of the model, which is fully described at https://github.com/mrc-ide/squire

Overview

squire is a package enabling users to quickly and easily generate calibrated estimates of SARS-CoV-2 epidemic trajectories under different control scenarios. It consists of the following:

An age-structured SEIR model incorporating explicit passage through healthcare settings and explicit progression through disease severity stages.
The ability to calibrate the model to different epidemic start-dates based on available death data.
Simulate the impacts of different control interventions (including general social distancing, specific shielding of elderly populations, and more stringent suppression strategies).

If you are new to squire, the best place to start is below, where we detail how to install the package, how to set up the model, and how to run it with and without control interventions.

Model Structure

Overall Structure

squire uses an age-structured SEIR model, with the infectious class divided into different stages reflecting progression through different disease severity pathways. These compartments are:
* S = Susceptibles
* E = Exposed (Latent Infection)
* I_Mild = Mild Infections (Not Requiring Hospitalisation)
* I_Case = Infections Requiring Hospitalisation
* I_Hospital = Hospitalised (Requires Hospital Bed)
* I_ICU = ICU (Requires ICU Bed)
* I_Rec = Recovering from ICU Stay (Requires Hospital Bed)
* R = Recovered
* D = Dead

Decision Trees for Healthcare Capacity

Given initial inputs of hospital/ICU bed capacity and the average time cases spend in hospital, the model dynamically tracks available hospital and ICU beds over time.

Individuals newly requiring hospitalisation (either a hospital or ICU bed) are then assigned to either receive care (if the relevant bed is available) or not (if maximum capacity would be exceeded otherwise). Whether or not an individual receives the required care modifies their probability of dying.

Installation

squire utilises the package odin to generate the model. odin implements a high-level language for implementing mathematical models and can be installed by running the following command:

install.packages("odin")

The model generated using odin is written in C and so you will require a compiler to install dependencies for the package and to build any models with odin. Windows users should install Rtools. See the relevant section in R-admin for advice. Be sure to select the “edit PATH” checkbox during installation or the tools will not be found.

The function odin::can_compile() will check if it is able to compile things, but by the time you install the package that will probably have been satisfied.

After installation of odin, ensure you have the devtools package installed by running the following:

install.packages("devtools")

Then install the squire package directly from GitHub by running:

devtools::install_github("mrc-ide/squire")

If you have any problems installing then please raise an issue on the squire GitHub.

If everything has installed correctly, we then need to load the package:

library(squire)

Getting Started

Running the Model (Unmitigated)

1. Running the model using baseline parameters and no control interventions

The full model is referred to as the explicit_SEEIR model, with hospital pathways explicitly exploring whether individuals will require a general hospital bed providing oxygen or an ICU bed that provides ventilation.

To run the model we need to provide at least one of the following arguments:

country
population and contact_matrix_set

If the country is provided, the population and contact_matrix_set will be generated (if not also specified) using the demographics and matrices specified in the global report.

To run the model by providing the country we use run_explicit_SEEIR_model():

r <- run_explicit_SEEIR_model(country = "Afghanistan", replicates = 5)

The returned object is a squire_simulation object, which is a list of two ojects:

output - model output
parameters - model parameters

squire_simulation objects can be plotted as follows:

plot(r)
#> Warning in plot.squire_simulation(r): Summary statistic estimated from <10
#> replicates
#> Warning in plot.squire_simulation(r): Confidence bounds estimated from <10
#> replicates
#> Warning: It is deprecated to specify `guide = FALSE` to remove a guide. Please
#> use `guide = "none"` instead.

This plot will plot each of the compartments of the model output. We can also plot specific compartments using the var_select argument that can be passed to plot(). Arguments passed to var_select must be one of the variables in the plot above.

plot(r, var_select = c("E", "IMild"))
#> Warning in plot.squire_simulation(r, var_select = c("E", "IMild")): Summary
#> statistic estimated from <10 replicates
#> Warning in plot.squire_simulation(r, var_select = c("E", "IMild")): Confidence
#> bounds estimated from <10 replicates
#> Warning: It is deprecated to specify `guide = FALSE` to remove a guide. Please
#> use `guide = "none"` instead.

Or, you can specify one of deaths, infections, hospital_occupancy, ICU_occupancy, hospital_demand or ICU_demand, and plot these summary metrics that represent the combination of a number of different compartment e.g:

plot(r, var_select = "deaths")
#> Warning in plot.squire_simulation(r, var_select = "deaths"): Summary statistic
#> estimated from <10 replicates
#> Warning in plot.squire_simulation(r, var_select = "deaths"): Confidence bounds
#> estimated from <10 replicates
#> Warning: It is deprecated to specify `guide = FALSE` to remove a guide. Please
#> use `guide = "none"` instead.

All of the plotting above makes uses of the squire function format_output which provides you with a means of manipulating and managing the output from a run_explicit_SEEIR_model call. Using it you can specify the model outputs (e.g. compartments) you want, as well as whether you want that output aggregated over age or not. Here we extract the latent compartment (E). The data columns correspond to the compartment name (compartment), timestep (t), model run number (replicate) and the model output (y).


output <- format_output(r, var_select = "E")
head(output)
#> # A tibble: 6 × 4
#> replicate compartment t y
#> <dbl> <chr> <dbl> <dbl>
#> 1 1 E 0.1 20
#> 2 1 E 0.2 20
#> 3 1 E 0.3 20
#> 4 1 E 0.4 20
#> 5 1 E 0.5 20
#> 6 1 E 0.6 20

If we wanted age-disaggregated data, we could set reduce_age to FALSE which will generate the same dataframe as before, but with an additional column indicating the age-group.


output <- format_output(r, var_select = "E", reduce_age = FALSE)
head(output)
#> # A tibble: 6 × 5
#> replicate age_group compartment t y
#> <dbl> <int> <chr> <dbl> <dbl>
#> 1 1 1 E 0.1 0
#> 2 1 1 E 0.2 0
#> 3 1 1 E 0.3 0
#> 4 1 1 E 0.4 0
#> 5 1 1 E 0.5 0
#> 6 1 1 E 0.6 0

2. Changing parameters in the model.

The model has a number of parameters for setting the R0, demography, contact matrices, the durations of each compartment and the health care outcomes and healthcare availability. In addition, the initial state of the population can be changed as well as simulation parameters, such as the number of replicates, length of simulation and the timestep. For a full list of model inputs, please see the function documentation

For example, changing the initial R0 (default = 3), number of replicates ( default = 10), simulation length (default = 365 days) and time step (default = 0.5 days), as well as setting the population and contact matrix manually:


# Get the population
pop <- get_population("United Kingdom")
population <- pop$n
# Get the mixing matrix
contact_matrix <- get_mixing_matrix("United Kingdom")
# run the model
r <- run_explicit_SEEIR_model(population = population,
contact_matrix_set = contact_matrix,
R0 = 2.5,
time_period = 200,
dt = 0.1,
replicates = 5)
plot(r)
#> Warning in plot.squire_simulation(r): Summary statistic estimated from <10
#> replicates
#> Warning in plot.squire_simulation(r): Confidence bounds estimated from <10
#> replicates
#> Warning: It is deprecated to specify `guide = FALSE` to remove a guide. Please
#> use `guide = "none"` instead.

We can also change the R0 and contact matrix at set time points, to reflect changing behaviour resulting from interventions. For example to set an 80% reduction in the contact matrix after 100 days :


# run the model
r <- run_explicit_SEEIR_model(population = population,
tt_contact_matrix = c(0, 100),
contact_matrix_set = list(contact_matrix,
contact_matrix*0.2),
R0 = 2.5,
time_period = 200,
dt = 0.1,
replicates = 5)
plot(r, var_select = "infections")
#> Warning in plot.squire_simulation(r, var_select = "infections"): Summary
#> statistic estimated from <10 replicates
#> Warning in plot.squire_simulation(r, var_select = "infections"): Confidence
#> bounds estimated from <10 replicates
#> Warning: It is deprecated to specify `guide = FALSE` to remove a guide. Please
#> use `guide = "none"` instead.

where n_E2_I is the daily number of new infections.

To show an 80% reduction after 80 days but only maintained for 40 days :


# run the model
r <- run_explicit_SEEIR_model(population = population,
tt_contact_matrix = c(0, 80, 120),
contact_matrix_set = list(contact_matrix,
contact_matrix*0.2,
contact_matrix),
R0 = 2.5,
time_period = 220,
dt = 0.1,
replicates = 5)
plot(r, var_select = "infections")
#> Warning in plot.squire_simulation(r, var_select = "infections"): Summary
#> statistic estimated from <10 replicates
#> Warning in plot.squire_simulation(r, var_select = "infections"): Confidence
#> bounds estimated from <10 replicates
#> Warning: It is deprecated to specify `guide = FALSE` to remove a guide. Please
#> use `guide = "none"` instead.

Alternatively, we could set a changing R0, which falls below 1 after 80 days:


# run the model
r <- run_explicit_SEEIR_model(population = population,
contact_matrix_set = contact_matrix,
tt_R0 = c(0, 80),
R0 = c(2.5, 0.9),
time_period = 200,
dt = 0.1,
replicates = 5)
plot(r, var_select = "infections")
#> Warning in plot.squire_simulation(r, var_select = "infections"): Summary
#> statistic estimated from <10 replicates
#> Warning in plot.squire_simulation(r, var_select = "infections"): Confidence
#> bounds estimated from <10 replicates
#> Warning: It is deprecated to specify `guide = FALSE` to remove a guide. Please
#> use `guide = "none"` instead.

The model in squire also allows you to alter healthcare capacity. Default values for the arguments hosp_bed_capacity and ICU_capacity are taken from the World Bank and a systematic review of the literature. However, you can also specify your own:


library(patchwork)
r <- run_explicit_SEEIR_model(population = population,
contact_matrix_set = contact_matrix,
R0 = 2.5,
time_period = 200,
dt = 0.1,
replicates = 15,
hosp_bed_capacity = 1000,
ICU_bed_capacity = 100)
c <- plot(r, var_select = "hospital_occupancy")
d <- plot(r, var_select = "ICU_occupancy")
c / d
#> Warning: It is deprecated to specify `guide = FALSE` to remove a guide. Please use `guide = "none"` instead.
#> It is deprecated to specify `guide = FALSE` to remove a guide. Please use `guide = "none"` instead.

3. Extracting and Plotting Relevant Outputs

Whilst the above provides quick and easy ways to plot relevant outputs, we recognise users might want to play around with the data themselves. This can be done with the format_output function.

library(ggplot2)
library(patchwork)
library(dplyr)
#>
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#>
#> filter, lag
#> The following objects are masked from 'package:base':
#>
#> intersect, setdiff, setequal, union
x <- run_explicit_SEEIR_model(country = "Afghanistan", hosp_bed_capacity = 500, ICU_bed_capacity = 200)
deaths <- format_output(x = x, var_select = "deaths") %>%
mutate(replicate = factor(replicate))
a <- ggplot(deaths, aes(x = t, y = y, col = replicate)) +
geom_line() + ylab("Daily Deaths")
infections <- format_output(x = x, var_select = "infections") %>%
mutate(replicate = factor(replicate))
b <- ggplot(infections, aes(x = t, y = y, col = replicate)) +
geom_line() + ylab("Daily Infections")
hosp_bed <- format_output(x = x, var_select = "hospital_occupancy") %>%
mutate(replicate = factor(replicate))
c <- ggplot(hosp_bed, aes(x = t, y = y, col = replicate)) +
geom_line() + ylab("Hospital Bed Occupancy")
ICU_bed <- format_output(x = x, var_select = "ICU_occupancy") %>%
mutate(replicate = factor(replicate))
d <- ggplot(ICU_bed, aes(x = t, y = y, col = replicate)) +
geom_line() + ylab("ICU Bed Occupancy")
z <- a + b + c + d +
plot_layout(guides = 'collect')
z

Hospital bed occupancy fluctuates very slightly above the user-specified number of hospital beds available (+1 in the above example). This is due to individuals being discharged from ICU beds into general hospital beds when general hospital bed occupancy is already at maximum capacity.

We assume that these individuals newly discharged from the ICU would in reality be prioritised to receive general hospital beds and so give them a general hospital bed. Whilst this is now shown in the above panel of hospital bed occupancy, this means that less-sick individuals in general hospital beds would be discharged slightly early to make space. We assume that for these small number of the healthiest individuals in hospital, there is no excess mortality associated with this early discharge.

Future work will explore a range of bed-prioritisation structures to consider how results might be shaped by who gets a bed and when.

4. Calibrating the Model to Observed Deaths Data

The model can be simply calibrated to time series of deaths reported in settings. This can be done using the calibrate function. For example, let’s use the time series of deaths in Algeria up to the 22nd April 2020:

df <- read.csv(squire:::squire_file("extdata/example_DZA.csv"), stringsAsFactors = FALSE)
head(df)
#> X date deaths cases
#> 1 1 2020-02-26T00:00:00Z 0 1
#> 2 2 2020-03-01T00:00:00Z 0 2
#> 3 3 2020-03-04T00:00:00Z 0 2
#> 4 4 2020-03-05T00:00:00Z 0 7
#> 5 5 2020-03-06T00:00:00Z 0 5
#> 6 6 2020-03-09T00:00:00Z 0 3

We can calibrate the model to this data. This calibration will infer the most likely start date of the epidemic and the R0 at the start of the epidemic. Calibration occurs by scanning across a range of possible R0 values and start dates. Fitting works using a particle filter, which can be parallelised using future::plan(future::multiprocess()) before running calibrate.

# set up for parallelisation
future::plan(future::multiprocess())
# Fit model
out <- calibrate(
data = df,
R0_min = 2,
R0_max = 4,
R0_step = 0.5,
first_start_date = "2020-02-10",
last_start_date = "2020-02-22",
day_step = 4,
replicates = 10,
n_particles = 20,
country = "Algeria"
)

calibrate returns the same output as run_explicit_SEEIR_model, with the first three elements in out being the simulation outputs, the model and model parameters. Note that simulation replicates are aligned to the maximum date in the data provided, and as a result we can use the same plotting functions as before:

plot(out, "deaths", date_0 = max(df$date), x_var = "date")
#> Warning: It is deprecated to specify `guide = FALSE` to remove a guide. Please
#> use `guide = "none"` instead.

With default parameters, calibrate will simulate up to the maximum date in the data provided. The fit to this data can be shown using the plotting function and specifying particle_fit to be TRUE

plot(out, particle_fit = TRUE)

We can see that this is not the best fit, capturing only the beginning of the data points. We can also diagnose the fitting further by looking at the scan_results of the grid search.

plot(out$scan_results)

This grid shows the negative log likeihood for each parameter pair. We can also plot the related probability for each pair as follows:

plot(out$scan_results, what = "probability")

The reason for the poor fits to the data shown earlier is because Algeria has implemented interventions prior to today. These can also be incorporated into calibrate. For example, we can grab the assumed changes to transmission based on government intervention for Algeria.

interventions <- read.csv(squire:::squire_file("extdata/example_DZA_intervention.csv"))
int_unique <- squire:::interventions_unique(interventions)
int_unique
#> $dates_change
#> [1] "2020-03-12" "2020-03-18" "2020-03-22" "2020-03-23"
#>
#> $change
#> [1] 0.900 0.850 0.550 0.225

We can then provide these to calibrate as the dates (date_R0_change) and relative reductions to R0 (R0_change). We will also specify for model fits to be continued for 14 days into the future with forceast = 14:

out <- calibrate(
data = df,
R0_min = 2,
R0_max = 4,
R0_step = 0.5,
first_start_date = "2020-02-10",
last_start_date = "2020-02-22",
day_step = 4,
replicates = 10,
n_particles = 20,
forecast = 14,
R0_change = int_unique$change,
date_R0_change = int_unique$dates_change,
country = "Algeria"
)

Let’s see if that is any better.

plot(out, particle_fit = TRUE)

That is a much better fit.

Any parameter that you could provide to run_explicit_SEEIR_model can be passed to calibrate. This includes time varying arguments such as contact_matrix_set, ICU_bed_capacity and hosp_bed_capacity. To incorporate these into model fitting correctly, the date at which these change must be provided (similarly to how date_R0_change was provided above) using date_ICU_bed_capacity_change, date_ICU_bed_capacity_change and date_hosp_bed_capacity_change respectively. In addition, the user must provide a baseline value for these, i.e. the contact matrix and bed capacity at the beginning of the epidemic:

out <- calibrate(
data = df,
R0_min = 2,
R0_max = 4,
R0_step = 0.5,
first_start_date = "2020-02-10",
last_start_date = "2020-02-22",
day_step = 4,
replicates = 10,
n_particles = 20,
forecast = 14,
R0_change = int_unique$change,
date_R0_change = int_unique$dates_change,
baseline_contact_matrix = get_mixing_matrix("Algeria"),
contact_matrix_set = list(get_mixing_matrix("Algeria")*0.9),
date_contact_matrix_set_change = "2020-03-16",
baseline_hosp_bed_capacity = squire:::get_hosp_bed_capacity("Algeria"),
hosp_bed_capacity = squire:::get_hosp_bed_capacity("Algeria")*c(1.1,1.2),
date_hosp_bed_capacity_change = c("2020-04-02", "2020-04-08"),
baseline_ICU_bed_capacity = squire:::get_ICU_bed_capacity("Algeria"),
ICU_bed_capacity = squire:::get_ICU_bed_capacity("Algeria")*c(1.05),
date_ICU_bed_capacity_change = c("2020-04-10"),
country = "Algeria"
)

(N.B. Given the potentially long running time for the grid search, the model state is returned at the end of every day rather than every time step (provided by dt). As a result model outputs such as n_E2_I (number of infections in a time step) and delta_D (number of deaths in a time step) reflect the outputs in the last time step rather than in all the time steps in the last day. The plotting functions and format_output provided in squire handle this correctly and work out the correct number for these in the last day.)

5. Conducting scenario projections to fitted data

Once you have calibrated the model to death data, it is possible to then implement mitigation strategies forwards in time from the point of calibration. This is achieved using the projections function. For example, let’s consider a country that as of 28th April 2020 is yet to experience an epidemic take off but may be about to. Guinea reported its first COVID-19 death on 16th April 2020 and has reported 7 deaths by the 28th April 2020 according to data from the European CDC.

First let’s calibrate to this data with no interventions in place (there are likely some interventions in place but nothing major, such as a lockdown, has been implemented by 2020-04-28) and simulate forward for 180 days:

df <- read.csv(squire:::squire_file("extdata/example_GIN.csv"), stringsAsFactors = FALSE)
out <- calibrate(
data = df,
R0_min = 2.5,
R0_max = 4,
R0_step = 0.5,
first_start_date = "2020-03-10",
last_start_date = "2020-03-25",
day_step = 5,
replicates = 10,
n_particles = 20,
forecast = 180,
country = "Guinea"
)

Firstly, let’s plot the fit up to the current day

plot(out, particle_fit = TRUE) +
ggplot2::xlim(as.Date(c("2020-04-10","2020-04-28"))) +
ggplot2::ylim(c(0,10))

The fit is good and captures the stuttering chains at the beginning of the epidemic. However, if we plot the forecasted deaths we can see the epidemic is likely to take off:

plot(out, "deaths")
#> Warning: It is deprecated to specify `guide = FALSE` to remove a guide. Please
#> use `guide = "none"` instead.

We can now use the output of calibrate to explore different scenario projections using projections. For example, to contast this unmitigated epidemic against a mitigation scenario with a 50% reduction in R0 today and a further 30% in 2 weeks time:

# create our projections
p <- projections(r = out, R0_change = c(0.5, 0.2), tt_R0 = c(0, 14))

The output generated from projections is the same class and structure as from calibrate and can be plotted against the unmitigated scenario using projection_plotting:

ggproj <- projection_plotting(r_list = list(out,p),
scenarios = c("Unmitigated","Mitigation"),
var_select = c("ICU_occupancy", "ICU_demand"),
add_parms_to_scenarios = TRUE,ci = FALSE,summarise = TRUE)
# and lets add in the ICU capacity
ggproj + ggplot2::geom_hline(yintercept = out$parameters$ICU_bed_capacity)
#> Warning: It is deprecated to specify `guide = FALSE` to remove a guide. Please
#> use `guide = "none"` instead.

We can see above that the intervention introduced is nearly sufficient to prevent ICU demand (solid line red) from exceeding the supply, whereas in the unmitigated strategy this did occur.

We can also model changing interventions by changing the contact matrix over time as well as the availability of ICU and hospital beds. E.g. decreasing contacts by 75% in a week before relaxing it to 80% in 30 days time, while increasing hospital and ICU beds by 200% in 30 days time. (N.B. We can turn off the automatic scenario parameter labelling with add_parms_to_scenarios = FALSE):

# create our projections
p <- projections(r = out,
contact_matrix_set_change = c(1, 0.25, 0.8),
tt_contact_matrix = c(0, 7, 30),
hosp_bed_capacity_change = c(1, 3),
tt_hosp_beds = c(0, 30),
ICU_bed_capacity_change = c(1, 3),
tt_ICU_beds = c(0,30))
projection_plotting(r_list = list(out, p),
scenarios = c("Unmitigated","Mitigation"),
var_select = c("ICU_occupancy", "ICU_demand","deaths"),
add_parms_to_scenarios = FALSE,
ci = FALSE,summarise = TRUE) +
ggplot2::geom_hline(yintercept = out$parameters$ICU_bed_capacity)
#> Warning: It is deprecated to specify `guide = FALSE` to remove a guide. Please
#> use `guide = "none"` instead.

Above we can see that the increase in bed causes the mitigation strategy to plateau at a higher bed occupancy, however, not high enough to meet the demand. However, the impacts to the contact matrix and beds have caused both a shift and a decrease in the daily deaths, which in turn causes a noticable decrease in total deaths:

projection_plotting(r_list = list(out,p),
scenarios = c("Unmitigated","Mitigation"),
var_select = c("D"),
add_parms_to_scenarios = FALSE,
ci = FALSE,summarise = TRUE)
#> Warning: It is deprecated to specify `guide = FALSE` to remove a guide. Please
#> use `guide = "none"` instead.

Rather than using relative changes to the interventions, we can provide absolute measures. For example, to change the R0 to 2 today and add 500 ICU beds in 40 days time:

# what is the current capacity
icu <- tail(out$parameters$ICU_bed_capacity,1)
# create our projections
p <- projections(r = out,
R0 = 2,
tt_R0 = 0,
ICU_bed_capacity = c(icu, icu + 500),
tt_ICU_beds = c(0,40))
projection_plotting(r_list = list(out,p),
scenarios = c("Unmitigated","Mitigation"),
var_select = c("ICU_occupancy", "ICU_demand","deaths"),
add_parms_to_scenarios = FALSE,
ci = FALSE,summarise = TRUE)
#> Warning: It is deprecated to specify `guide = FALSE` to remove a guide. Please
#> use `guide = "none"` instead.

Lastly, in order to run simulations for longer than the number set in calibrate(forecast = x), we can use the argument time_period to set the numbers of days that projections should simulate for. For example, we could redo our calibration and have it only run up the current day:

df <- read.csv(squire:::squire_file("extdata/example_GIN.csv"), stringsAsFactors = FALSE)
out <- calibrate(
data = df,
R0_min = 2.5,
R0_max = 4,
R0_step = 0.5,
first_start_date = "2020-03-10",
last_start_date = "2020-03-25",
day_step = 5,
replicates = 10,
n_particles = 20,
forecast = 0,
country = "Guinea"
)
plot(out, particle_fit = TRUE)

We can now use this to project 90 days forwards with the current level of interventions, as well as a projection of 90 days with a 50% reduction in R0:

p_no_change <- projections(out, time_period = 90)
p_change <- projections(out, R0_change = 0.5, tt_R0 = 0, time_period = 90)
projection_plotting(list(p_no_change, p_change),
scenarios = c("No Change", "50% R0"),
add_parms_to_scenarios = FALSE,
var_select = "deaths",
date_0 = max(df$date),
x_var = "date")

Interacting with The Demographic and Health Surveys (DHS) Program data

Tue, 29 Jan 2019 00:00:00 +0000

This is a copy of my article for ROpenSci detailing the rdhs package:

There seem to be a lot of ways to write about your R package, and rather than have to decide on what to focus on I thought I’d write a little bit about everything. To begin with I thought it best to describe what problem rdhs tries to solve, why it was developed and how I came to be involved in this project. I then give a brief overview of what the package can do, before continuing to describe how writing my first proper package and the rOpenSci review process was. Lastly I wanted to share a couple of things that I learnt along the way. These are not very clever or difficult things, but rather things that were difficult to Google, which now I think about it should probably be the best metric for a difficult problem.

Motivation

What is the DHS Program

The Demographic and Health Survey (DHS) Program has collected and disseminated population survey data from over 90 countries for over 30 years. This amounts to over 400 surveys that give representative data on health indicators, which in many countries provides the key data that mark progress towards targets such as the Sustainable Development Goals (SDGs). In addition, DHS survey data has been used to inform health policy such as detailing trends in child mortality¹ and characterising the distribution of malaria control interventions in Africa in order to map the burden of malaria².

This is all to the say that the DHS provides really useful data. However, although standard health indicators are routinely published in the survey final reports that are published by the DHS program, much of the value of the DHS data is derived from the ability to download and analyse the raw datasets for subgroup analysis, pooled multi-country analysis, and extended research studies.

This where I got involved, in trying to create a tool that helped enable researchers to quickly gain access to the raw data sets.

How I got involved

I am fortunate enough to be a PhD student in a really large department at Imperial College London, which means that I get the opportunity to be involved in many projects that are outside the scope of my actual PhD. The “downside” of that is sometimes you get given “code monkey” jobs as the bottom rung of the monkey ladder. And so, a few months into my PhD (Nov 2016), I was given the job of downloading data on malaria test results from the DHS program that was going to be used by some collaborators. At the time I was very happy to be involved, however, I was apprehensive to spend too long on the job as I didn’t know how much time to be spending on side projects vs my PhD (something I still don’t know with 6 months to go). This combined with only having a year or so’s experience writing R meant that the code I wrote to do the job was a bunch of scrappy scripts that required manually downloading the datasets before parsing them with these R scripts. Dirty but it got the job done.

Some time passed, and another collaborator wanted some different data collated from the DHS program. At this point, I had 6 more months familiarity with R and knew a bit more so I started writing it as an R package. However, it was still messy and it required manually downloading the datasets first, but I was happy with it and again it wasn’t a major project of mine. This would have been probably where the project ended if I hadn’t had a conversation (Sept 2017) in the tea room (prompted solely by the presence of free biscuits) with the other main author of rdhs, Jeff Eaton.

We got chatting, and realised we both had a bunch of scripts for doing bits of the analysis pipeline. We also realised that we had both had numerous requests for data sets from the DHS program at which point we thought it would be best to do something properly. I had also at this point been keen to start using testhat within my work as I had been told it would save me time in the future, and up till that point I hadn’t found a good case to get to grips with it (mainly writing code on my own, that was never very big and was only used by myself). And so we started writing rdhs, which was accepted by rOpenSci and CRAN in December 2018.

Package overview

Disclaimer: The following section (the API and Dataset Downloads headings) is an overiew of the Introduction Vignette. If you want a longer introduction to the package then head there, otherwise carry on and eventually you will get to my ramblings about the package development process.

Most of the functionality of rdhs can be roughly summarised in the 5 main steps that are involved from wanting to get data on x to having a curated data set created from survey data from multiple surveys. These steps involve:

Accessing standard survey indicators through the DHS API.
Using the API to identifying the surveys and datasets relevant to your particular analysis, i.e. the ones that ask questions related to your topic of interest.
Downloading survey datasets from the DHS website.
Loading the datasets and associated metadata into R.
Extracting variables and combining datasets for pooled multi-survey analyses.

We will quickly cover these 5 main steps, with the first 2 showing how rdhs functions as an API client and the last 3 points showing how rdhs can be used to download raw data sets from the DHS website. Before we have a look at these, let’s first load rdhs:

library(rdhs)

API

1. Access standard indicator data via the API

The DHS program has published an API that gives access to 12 different data sets. Each API endpoint represents one of the 12 data sets (e.g. https://api.dhsprogram.com/rest/dhs/tags), and can be accessed using the dhs_<>() functions. For more information about this see the DHS API website.

One of those functions, dhs_data(), interacts with the the published set of standard health indicator data calculated by the DHS. For example, to find out the trends in antimalarial use in Africa, and see if perhaps antimalarial prescription has decreased after rapid diagnostic tests were introduced (assumed 2010).

# Make an api request
resp <- dhs_data(indicatorIds = "ML_FEVT_C_AML", surveyYearStart = 2010,breakdown = "subnational")
# filter it to 12 countries for space
countries <- c("Angola","Ghana","Kenya","Liberia",
"Madagascar","Mali","Malawi","Nigeria",
"Rwanda","Sierra Leone","Senegal","Tanzania")
# and plot the results
library(ggplot2)
ggplot(resp[resp$CountryName %in% countries,],
aes(x=SurveyYear,y=Value,colour=CountryName)) +
geom_point() +
geom_smooth(method = "glm") +
theme(axis.text.x = element_text(angle = 90, vjust = .5)) +
ylab(resp$Indicator[1]) +
facet_wrap(~CountryName,ncol = 6)

2. Identify surveys relevant for further analysis

You may, however, wish to do more nuanced analysis than the API allows. The following 4 sections detail a very basic example of how to quickly identify, download and extract datasets you are interested in.

Let’s say we want to get all DHS survey data from the Democratic Republic of Congo and Tanzania in the last 5 years (since 2013), which covers the use of rapid diagnostic tests for malaria (“RDT” below). To begin we’ll interact with the DHS API to identify our datasets.

## make a call with no arguments
sc <- dhs_survey_characteristics()
sc[grepl("Malaria", sc$SurveyCharacteristicName), ]

## SurveyCharacteristicID SurveyCharacteristicName
## 57 96 Malaria - DBS
## 58 90 Malaria - Microscopy
## 59 89 Malaria - RDT
## 60 57 Malaria module
## 61 8 Malaria/bednet questions

There are 87 different survey characteristics, with one specific survey characteristic for malaria rapid diagnostic tests (RDT). In this example we will use this to find the surveys that include this characteristic. (There are other ways to find the datasets with the API and other options to control how to filter the API, which are explored here)

# lets find all the surveys that fit our search criteria
survs <- dhs_surveys(surveyCharacteristicIds = 89,
countryIds = c("CD","TZ"),
surveyType = "DHS",
surveyYearStart = 2013)
# and lastly use this to find the datasets we will want to download
# and let's download the flat files (.dat) datasets
datasets <- dhs_datasets(surveyIds = survs$SurveyId,
fileFormat = "flat",
fileType = "PR")
str(datasets)

## 'data.frame': 2 obs. of 13 variables:
## $ FileFormat : chr "Flat ASCII data (.dat)" "Flat ASCII data (.dat)"
## $ FileSize : int 6595349 6622102
## $ DatasetType : chr "Survey Datasets" "Survey Datasets"
## $ SurveyNum : int 421 485
## $ SurveyId : chr "CD2013DHS" "TZ2015DHS"
## $ FileType : chr "Household Member Recode" "Household Member Recode"
## $ FileDateLastModified: chr "September, 19 2016 09:58:23" "August, 07 2018 17:36:25"
## $ SurveyYearLabel : chr "2013-14" "2015-16"
## $ SurveyType : chr "DHS" "DHS"
## $ SurveyYear : int 2013 2015
## $ DHS_CountryCode : chr "CD" "TZ"
## $ FileName : chr "CDPR61FL.ZIP" "TZPR7AFL.ZIP"
## $ CountryName : chr "Congo Democratic Republic" "Tanzania"

We can now use this to download our datasets for further analysis.

Dataset Downloads

3. Download survey datasets

To be able to download survey datasets from the DHS website, we need to set up an account through the DHS website to enable you to request access to the datasets. Instructions on how to do this can be found here.

Once we have created an account, we set up our credentials using the function set_rdhs_config(). See the Introduction Vignette for more clarity about the various options for setting up your config.

## set up your credentials
set_rdhs_config(email = "rdhs.tester@gmail.com",
project = "Testing Malaria Investigations",
cache_path = "project_one",
config_path = "~/.rdhs.json",
data_frame = "data.table::as.data.table",
global = TRUE)

We can now download the data sets we identified earlier from the API, using get_datasets:

# download datasets
downloads <- get_datasets(datasets$FileName)

4. Load datasets and associated metadata into R

We can now examine what it is we have actually downloaded, by reading in one of these datasets:

# read in our dataset
cdpr <- readRDS(downloads$CDPR61FL)

The dataset returned here contains all the survey questions within the dataset.
The dataset is by default stored as a labelled class from the haven package.

If we want to get the data dictionary for this dataset, we can use the function get_variable_labels:

# let's look at the variable_names
head(get_variable_labels(cdpr))

## variable description
## 1 hhid Case Identification
## 2 hvidx Line number
## 3 hv000 Country code and phase
## 4 hv001 Cluster number
## 5 hv002 Household number
## 6 hv003 Respondent's line number (answering Household questionnaire)

The default behaviour for the function get_datasets was to download the datasets, read them in, and save the resultant data.frame as a .rds object within the cache directory. It also creates the data dictionary and caches this for you, which allows us to quickly query for particular variables or variable_labels:

# rapid diagnostic test search
questions <- search_variable_labels(datasets$FileName, search_terms = "malaria rapid test")

Or if we know what variables we want, we can identify which surveys include these:

# and grab the questions from this now utilising the survey variables
questions <- search_variables(datasets$FileName, variables = c("hv024","hml35"))
head(questions)

## variable description dataset_filename
## 1 hv024 Province CDPR61FL
## 2 hml35 Result of malaria rapid test CDPR61FL
## 3 hv024 Region TZPR7AFL
## 4 hml35 Result of malaria rapid test TZPR7AFL
## dataset_path
## 1 /home/oj/GoogleDrive/AcademicWork/Imperial/git/rdhs/paper/project_one/datasets/CDPR61FL.rds
## 2 /home/oj/GoogleDrive/AcademicWork/Imperial/git/rdhs/paper/project_one/datasets/CDPR61FL.rds
## 3 /home/oj/GoogleDrive/AcademicWork/Imperial/git/rdhs/paper/project_one/datasets/TZPR7AFL.rds
## 4 /home/oj/GoogleDrive/AcademicWork/Imperial/git/rdhs/paper/project_one/datasets/TZPR7AFL.rds
## survey_id
## 1 CD2013DHS
## 2 CD2013DHS
## 3 TZ2015DHS
## 4 TZ2015DHS

More information about download options and querying the survey questions can be found here

5. Extract variables and combine datasets

To extract our data we pass our questions object to the function extract_dhs, which will create a list with each dataset and its extracted data as a data.frame.

# extract the data and add geographic information too
extract <- extract_dhs(questions, add_geo = FALSE)

The resultant extract is a list, with a new element for each different dataset that you have extracted. We can now combine our two data frames for further analysis using the rdhs package function rbind_labelled():

# first let's bind our first extraction, without the hv024
extract_bound <- rbind_labelled(extract)

## Warning in rbind_labelled(extract): Some variables have non-matching value labels: hv024.
## Inheriting labels from first data frame with labels.

The thrown warning has shown us that hv024 did not have matching labels between the two lists, and the labels from the first list have been used. hv024 stores the regions for these 2 countries, and we probably want to keep all the labels, which we can do by using the labels argument:

# lets try concatenating the hv024
better_bound <- rbind_labelled(extract, labels = list("hv024"="concatenate"))

We could also specify new labels for a variable. For example, imagine the two datasets encoded their rapid diagnostic test responses differently, with the first one as c("No","Yes") and the other as c("Negative","Positive"). We can choose to relabel these, e.g. as c("NegativeTest","PositiveTest"):

# lets try concatenating the hv024 and providing new labels
better_bound <- rbind_labelled(
extract,
labels = list("hv024"="concatenate",
"hml35"=c("NegativeTest"=0, "PositiveTest"=1))
)
# and our new label
head(attr(better_bound$hml35,"labels"))

## NegativeTest PositiveTest
## 0 1

For more information about controlling how to extract data from your downloaded sections, see the last section in the introduction vignette.

We now have managed to go from our initial request for data about the use of rapid diagnostic tests for malaria to a finalised data set that we can use going forwards for any downstream analysis (and hopefully it didn’t take that long to do it!). This data set includes survey responses from multiple surveys within one data frame, which in this case includes data from Tanzania and the Democratic Republic of Congo. However, it would be easy to extend our earlier API query to include more countries. For example if we had not limited our search to these 2 countries, the same code as above would have returned data from over 200,000 individuals across 21 countries. Similarly if we wanted to include more survey responses, we could have provided different search terms to search_variables or search_variable_labels. By widening our search terms, and including more datasets within the search we can easily create data sets that can be used to answer important global health questions such as:

Which malaria RDTs are performing worse in low malaria prevalence regoions?
What is the link between HIV prevalence and wealth?
How far apart should births occur to minmise childhood mortality?

Ramblings after my first completed package

Clichéd but the process of actually writing a package, and all that entailed, was a real highlight. I had made R packages before, but I had never done everything that a good R package should have (tests, effective continuous integration, full documentation, a pkgdown website, contribution and code of conduct guides, and so on). One particular highlight for me was actually having the opportunity to work on a code base with someone else in a collaborative way. I work in a large collaborative group, however, this has not translated as much to working on the same set of code with someone. As a result I’ve never had to properly learn how to use git outside of clone, commit and push, nor had I made use of much of the useful aspects of GitHub. So learning how to correctly use branches in git and realising that helpful comments are actually helpful (eventually) was really great. With this in mind I wanted to thank Jeff Eaton again for taking on this project. He definitely helped drive it over the finish line, and it was nice to have a glimpse at what working as a developer would look like if I decide to leave pure academia.

There were also a few things that before I started writing rdhs I knew I would have to figure out but I didn’t have a clue where to start, and for which repeated googling didn’t eventually help with. Fortunately, I work in the same department as Rich FitzJohn, so it was great having someone to point me in the right direction. The following are three of the things that I genuinely had no idea how to do before, so I thought I’d share them here (and so I can remind myself in the future):

1. Logging into a website from R

The DHS website has a download manager that you can use to select surveys you want to download, and it will auto generate a list of URLs in a text file. When I saw this, I thought this would be great for creating a database of what data sets and the URLs a user’s login details can give them, which can then be cached so that rdhs knows whether you can download a data set or not. The only problem is, that to download those data sets you need to be logged in, and you also need to be logged in to get to the download manager. For me, I didn’t know how to translate being “logged in” into R code, or even what that looked like. But turns out it wasn’t too bad after being shown by Rich where to start looking.

To know where to look I opened up Chrome and went to developer tools. From there I opened up the Network Tab, which then records the information being sent to the URL. So to know what information is required to login I simply logged in as normal, and then inspected what appeared in the network tab’s Headers Tab. This then showed me what the needed Request URL was, and what information was being submitted in the Form Data at the bottom of this tab.

I could then use this information to log in from with R using an httr::POST request:

# authentication page
terms <- "https://dhsprogram.com/data/dataset_admin/login_main.cfm"
# create a temporary file
tf <- tempfile(fileext = ".txt")
# set the username and password
values <- list(
UserName = your_email,
UserPass = your_password,
Submitted = 1,
UserType = 2
)
# log in.
message("Logging into DHS website...")
z <- httr::POST(terms, body = values) %>% handle_api_response(to_json = FALSE)

To me, this seemed really cool, and then meant I could do the same style of steps to get to the Download Manager webpage and then tick all the check boxes in the page to generate the URL with all the download links in.

2. Caching API results from a changing API

We wanted to be able to cache a user’s API request for them locally when designing rdhs. We felt this was important as it would reduce the burden on the API itself, as well as enable researchers who were without internet (e.g. currently working in the field), the ability to still access previous API requests. However, designing something neat that would be easy to respond to changes in the API version would I thought be outside my skill set.

Again, enter Rich and this time with his package storr. This was a lifesaver, and created an easy infrastructure for storing API responses in a key-value store. I could then use the specific API URL as the key and the response as the value. Initially I thought I would have to keep saving the response with explicit names (e.g. the URL), but storr handles all this for you, and also then helps get around having too long file names if your API request is very long for example.

To respond to changes in the API, my solution was perhaps not the neatest, but I simply kept a record of the date you last made an API request and compared it to the API’s data updates endpoint. If I could see any recent changes, I then could clear all the API requests cached. This would made a lot simpler using the namespaces options in storr, which meant that I was able to keep all API cached data in one place, which could then be easily deleted on mass.

3. Tests, Travis & Authentication

The last thing caused me the most amount of headaches. How do I write tests that require authentication and can use travis for continuous integration. Initially, I made a dummy account with the DHS website for this, but realised that sharing the credentials of an account with access to just dummy data sets would not enable me to test the weird edge cases that started popping up related to certain data sets. The first solution that I used for a few months was to set up environment variables within travis itself, which could then be used to create a valid set of credentials.

This worked, however, it meant that I would have to write a lot of the rdhs functionality to use environment variables that were the user’s email and password, which felt wrong and quite clunky. All I wanted was to pass to Travis a valid set of login credentials that would then be used within the tests, much in the same way that a user would. To do this I had to learn a bit more about what the .travis.yml document could actually be used for, because to begin with I had only been using it to specify the software language.

Again, Rich pointed me to using sodium to create an encrypted version of a valid login credentials:

# read in a key from a local file
key <- sodium::hash(charToRaw(readLines("scripts/key.txt")))
# create a tat with all the necessary login credentials
zip("rdhs.json.tar",files=c("rdhs.json", "tests/testthat/rdhs.json"))
# read this tar in as binary data
dat <- readBin("rdhs.json.tar",raw(),file.size("rdhs.json.tar"))
# encrypt the data using sodium and our key before saving it
enc <- sodium::data_encrypt(msg = dat,key = key)
saveRDS(enc,"rdhs.json.tar.enc")

This encrypted copy could be included in the GitHub repository, and I could set up the key as a Travis environment variable to decrypt it. This decryption step could then be written within my .travis.yml file, and would mean that all my tests had access to my login credentials in a secure way.

Options to Contribute

There are a few things that would be great to add in the future.

Adding a suite of tools for doing spatial mapping. A lot of the time, people want to know what the prevalence of x is either at a fine spatial scale, or grouped at administrative/county/state levels. rdhs helps provide the tools to get geolocated measures of x, and I think it would be a great next step to add a suite of mapping tools. It would be great if they could be used to either create a mesh through these points (probably using INLA), or calculate survey weighted means at requested spatial scales or match them to a provided SpatialPolygons object. Related to this is it would be good to also link in the Spatial Data Repository from the DHS, so that users can easily download shape files for their analyses (issue #71).
Not related to any specific issues, but it would be good to have a clearer set of downstream analysis pipelines. One example is a package in development by Jeff Eaton called demogsurv, which is used to calculate common demographic indicators from household survey data, including child mortality, adult mortality, and fertility. This is just one example, but over time there will be a number of bespoke analysis tools down the line, and so it would be nice to begin a collection/grouping of these tools (possibly as a wiki or similar).
It would be nice to have a way to manually add sources of survey data. At the moment the pipeline for downloading raw data sets used the DHS API a lot, however, what if you had some survey data (either locally or shared at a URL) that you wanted to bring into your analysis pipeline. Something similar to this is done for the model_datasets within rdhs, which is a set of dummy data sets that the DHS hosts online but are not included in their API.

Acknowledgements and Final Thoughts

Firstly, I want to thank Anna Krystalli for handling the review, and for being incredibly patient throughout, especially at the end as we were fixing the last authentication bug. Also many thanks to Lucy McGowan and Duncan Gillespie for taking the time to review the package and for their input, which led to lots of improvements (and also linking the add_line function from httr was seriously helpful, and I’ve used that function in lots of other my other work now). I also wanted to more broadly thank the review process as a whole. Having the option to discuss the package and needed solutions with the reviewers within a GitHub issues system is fantastic. It made the process personal and was substantially improved over review processes I have had at academic journals. Lastly, another big thank you Jeff Eaton and Rich FitzJohn, and also to the infectious disease epidemiology department at Imperial for providing a lot of really helpful ginuea pig testing of the numerous iterations of rdhs.

Silva, Romesh. 2012. “Child Mortality Estimation: Consistency of Under-Five Mortality Rate Estimates Using Full Birth Histories and Summary Birth Histories.” PLoS Medicine 9: e1001296. doi:[10.1371/journal.pmed.1001296](https://doi.org/10.1371/journal.pmed.1001296).↩︎
Bhatt, S, D J Weiss, E Cameron, D Bisanzio, B Mappin, U Dalrymple, K E Battle, et al. 2015. “The effect of malaria control on Plasmodium falciparum in Africa between 2000 and 2015.”Nature 526: 207–11. doi:[10.1038/nature15535](https://doi.org/10.1038/nature15535).↩︎