Part 1 -- Modelling the coronavirus: why all public health models are not the same

Scientific Models Making the News Headlines

As most readers have probably seen, in the last couple weeks the issue of how to model the coronavirus has made headline news. In particular, two scientific public health models have been highlighted amongst an increadible outpouring of modelling research. The first is the simulation model by Ferguson and colleagues at Imperial College London; and the second is the complex network model by Alessandro Vespignani and colleagues at Northeastern University in the States. 

The impact these public health models have made on how governments are thinking about such things as containment, mitigation, suppression and eradication has been pronounced. And, what is likewise significant is that these models come from the computational and complexity sciences, which signals within the scientific community (in particular epidemiology and public health) a major advance in how we are dealing with the global complexity we are confronted with today -- which is an incredible accomplishment for the modelling community. Making sense of these advances and how they inform policy is therefore a key task.

BLOG POST 1 of N

In response to these advances, the current post is meant to be the first of several addressing the complex challenges of modelling the coronavirus as a public health issue. It is also about clarifying for a wider audience how and why such modelling is important, as well as the value and power of complex systems thinking and computational modelling for public health policy. Still, this does not mean modelling will answer all of our questions; nor does it mean that all models are equally useful! 

The Models We Use Impact the Decisions We Make

Image from Brockmann Lab http://rocs.hu-berlin.de/

In addition to the challenges of converting science into policy recommendations, the coronavirus has made very clear the recommendations made by policy evaluators and politicians are very much determined by the models upon which they rely -- be they economic, scientific, political. And, as we have seen with the controversial public health recommendations in the UK, USA and elsewhere in the world (circa March 2020), while some models are very good at simulating COVID-19 and making reasonably useful predictions, others are significantly problematic and lacking. In other words, even scientific models can be lacking. (See for example, Tracking an epidemic requires computer models. But what if those models are wrong?)

1. Some COVID-19 models are better than others

Case in point are the significant errors made by many public health models. For example, many of the current models are far too simplistic, Many also fail to model human interactions and the complex networks in which we live, or how societies, social networks or populations react to the policies being implemented to help them.

Also, even if interactive and dynamic, many are country specific; that is, they fail to see their society and population as part of a wider and highly complex global system and ignore that we live in a global network society, including its major transportation routes. As a result, as the World Health Organisation has been pleading for the past three months, their interventions are not coordinated with other countries; nor do they consider much beyond their own borders. Given this view, they assume that outbreaks in other parts of the world are not their problem; or that such 'problems' only happen in other countries. 

In turn, they also misunderstand collective psychology and how quickly ‘panic’ or ‘rebellion’ can set in -- the hording of toilet paper is a good example; or people failing to follow health behaviours guidelines; or how culture impacts health practices, as in Italy versus Singapore for example.

And, realtive to the issue of time and societal reaction, they fail to forecast the knock-on effects of their policies. Knock-on effects include:

• Unexpected drawbacks -- for example, how social isolation impacts different segements of the population differently; such as those who provide critical services, the elderly, children, the working poor; and those with pre-exisiting health problems.
• Perverse effects -- for example, school closings lead to kids being cared for by elderly grandparents, who are now made more vulnerable to catching the virus. Or, poor people who cannot pay the rent because the pub or restaurant at which they work is closed.
• Rebound effects -- for example, many of the current measures being employed may result in a rebound of COVID-19 this autumn 2020. For example, the simulation model developed by Ferguson and colleagues at Imperial College (which has helped to shift how the UK is addressing COVID-19) shows that some approaches to containment will lead to a resurgence sometime around late October/early November.

2. Cases, configurations and COVID-19 transmission

The other major challenge is that many of these models do not think about disease transmissions from a case-based configurational perspective. Regardless of the method used, a case-based configurational perspective is anchored in four core arguments that deeply resonate with the majority of computational methods used today. First, the case (which in this case is someone with COVID-19) and its trajectory across time/space are the focus of study, not the individual variables or attributes of which it is comprised. Second, cases and their trajectories are treated as composites (profiles), comprised of an interdependent, interconnected sets of variables, factors or attributes. Third, the relationships and social interactions amongst cases are also important, as are the hierarchical social contexts/systems in which these relationships take place. And, finally, cases and their relationships and trajectories are the methodological equivalent of complex systems – that is, they are emergent, self-organizing, nonlinear, dynamic, network-like, etc – and therefore should be studied as such. 

Given that many public health models do not embrace this approach, they often struggle to demonstrate the differential impact that the spread of COVID-19 will have on different populations and subgroups. 

For epidemiologists, different populations and sub-groups -- be they communities, towns, cities, regions, different socioeconomic groups, particular health vulnerable groups, etc -- differ from one another, based on differences in their respective public health profiles, which includes such key social factors as age, gender, ethnicity, residence, income, education, health behaviours, quality of healthcare and public health system (e.g., access to respirators, hospital beds, quarantine centres). 

Again, by profiles we also mean how these different sets of social factors intersect one another to account for different health outcomes and wellbeing of these different groups. For example, poor parents living in urban environments where they need to work cannot afford to self-quarantine as easily as a privileged professor like myself living on a university campus. As such, the policies needed for my health differ from the needs of others. For example, for working parents, it might be how and when we partially close schools so the kids of these working parents can still attend school and be safe? Or, how can these parents be given a way to afford school closings and self-quarantine?

And to make the point one more time, for those unfamiliar with a configurational approach, it is the methodological backbone of just about all smart technology and machine learning, which tries to develop profiles on how different groups or communities of people shop, surf the web, travel, use social media, etc, in order to better target interventions specific to these groups, be it new things to buy, news to read, music to listen to, health behaviours to change, etc. Various names for this approach are data mining, data science, digital social science, and big data -- see Mark Carrigan's blog.

3. Data, data and more data

The other major problem with many of these models is that they lack good data. Now, to be fair, that is presently a problem for all modelling, as the coronavirus presents a bit of a different epidemiological scenario, particularly the absence of current human immunity and also no current vaccines.

Another data issue that it is not clear is what the "reproduction number" (R_o) should be -- which is defined as how many people each infected person can infect if the transmission of the virus is not hampered by quarantines, face masks, or other factors.

Another major data challenge is COVID-19's incubation time, which is how long it takes for the virus to cause symptoms. (For more on this point, see this Science Magazine article).

Still, all models are only as good as the data on which they are fine-tuned and validated. Which gets even more tricky if one is thinking about data trajectories and trends across time; as well as about sampling and representation in terms of the sub-groups and communities in a country or region of the world and, in turn, their different respective configurations of factors. In short, things get complex very fast.

(NOTE: For those interested in the details, here is a supplementary overview of the 2006 model that Ferguson and colleagues used to develop their current COVID-19 simulation of the UK. It goes into considerable detail about the importance of data for modelling. Also, for an overview of the importance of data for initializing, calibrating and validating a model, see http://jasss.soc.surrey.ac.uk/19/1/9.html. Also, see https://ieeexplore.ieee.org/document/8732881)

4. Managing pandemics such as COVID-19 versus eradicating them

The final major issue is one of perspective. Is the goal to eradicate or manage COVID-19? For example, the total quarantine of a population is mostly focused on trying to stop the coronavirus in the short-term. What happens, though, when a country tries to go back to normal? Will the virus return? This is a major concern for China and Italy right now (circa March 2020).

For complexity scientists, such a decision has everything to do with, ... well, ... complexity.

 

The more complex the problem, the more likely it is best treated as what we call a nexus issue or wicked problem. Such problems are wicked because the configuration of factors is far too challenging and intersects with too many different domains of our lives, to effectively solve once and for all. For example, quarantining the population can collapse the economy; keeping the economy going leads to people becoming sick; developing herd immunity for a country creates very high and unacceptable mortality rates; however, not doing so could lead to even higher mortality rates, and so forth. As such, complexity scientists opt for a systems mapping approach for wicked and nexus problem, with the goal of managing the issue, and with an eye to all of the issues we discussed above, including knock-on effects.

NOTE: Managing a pandemic still allows for periodic suppression of a virus such as COVID-19. But that does not mean the public health problem has gone away or will not return or evolve sufficiently to harm again. Even Ferguson and colleagues at Imperial College London, who developed the simulation being most widely explored and used by the UK government end their report with the following caveated conclusion:

We therefore conclude that epidemic suppression is the only viable strategy at the current time. The social and economic effects of the measures which are needed to achieve this policy goal will be profound. Many countries have adopted such measures already, but even those countries at an earlier stage of their epidemic (such as the UK) will need to do so imminently.

Our analysis informs the evaluation of both the nature of the measures required to suppress COVID-19 and the likely duration that these measures will need to be in place. Results in this paper have informed policymaking in the UK and other countries in the last weeks. However, we emphasise that is not at all certain that suppression will succeed long term; no public health intervention with such disruptive effects on society has been previously attempted for such a long duration of time. How populations and societies will respond remains unclear.

All of which takes us to the last major point.


5. While some models are better than others, all models are wrong!
At the end of the day, no matter how incredible the model, it is somehow or in some way wrong. That is the truth of it. All models are underdetermined by their evidence. As such, we always need to be sober in our usage of scientific and public health modelling. While scientific models can help us make decisions, as the complexity they seek to model increases, they are ultimately predictions of what might happen, not necessarily what will happen. Change the data or some of the parameters and the outcome it predicts can be different. It is a harsh reality of modelling complexity.

As one of my professors once said in response to the question, why can we put a person on the moon but we cannot solve poverty? The professor answers, because poverty is more complex!

MODELLING COVID-19: So, what does an effective approach to simulation look like?

So, given the above concerns, what does an effective model of COVID-19 look like? That will be the focus of my next two blogs. To demonstrate, I will review the simulation model by Ferguson and colleagues at Imperial College London; and then the complex network model by Alessandro Vespignani and colleagues at Northeastern University in the States.