Infectious Disease Modelling: What's The Track Record?

The London Times has a fascinating story on the tiny, incestuous world of infectious disease modelers. It begins with the two modeling outfits in Great Britain: Oxford University and Imperial College, London. We have all learned about Imperial as the source of the two wildly conflicting estimates that the British government has relied on. Now an Oxford professor is questioning Imperial’s model:

The first public signs of academic tensions over Imperial’s domination of the debate came when Sunetra Gupta, professor of theoretical epidemiology at Oxford University, published a paper suggesting that some of Imperial’s key assumptions could be wrong.

It turns out there is some history here. Oxford had the dominant modeling group until Sir Roy Anderson left Oxford for Imperial twenty years ago. He left Oxford because he opposed a colleague’s promotion to a permanent professorship on the ground (apparently false) that she was only being considered because she slept with a member of the panel. That colleague was…Sunetra Gupta. As noted, the disease modeling world is small.

The Times quotes Gupta:

I decided to publish and speak out because the response to this pandemic is having a huge effect on the lives of vulnerable people with a profound cost and it seems irresponsible that we should proceed without considering alternative models. Imperial has a long history of involvement with government and its epidemiological models can have huge importance and translational impact but it’s tricky to use them to forecast what’s going to happen. We need to also consider alternatives.

That is consistent with our repeated call for transparency in epidemiological models, so that we can understand what assumptions are driving the extraordinary actions being taken by many governments. But the Times goes on to ask an important question that I have not previously seen raised: do disease modelers have a track record that deserves to inspire the remarkable confidence that is being reposed in them?

Woolhouse was also working with Anderson when mad cow disease spread from cattle into humans in the 1980s and 1990s and the government asked Oxford to help calculate the scale of the infection. This led to the cull of 4.4 million cattle, which suppressed the disease.
By the time foot-and-mouth disease (FMD) struck in 2001, however, Anderson’s clash with Gupta had seen him move to Imperial. … Oxford was in effect sidelined and it was from Imperial that Ferguson and Anderson dominated the government response to foot and mouth.
That response, involving the slaughter of more than 11 million sheep and cattle at a cost of more than £8bn was based entirely on modelling and remains hugely controversial — with many believing the modellers got it wrong. They were modelling a fast-moving epidemic with little accurate data. A subsequent government inquiry was damning of the general approach and its conclusions may be relevant to the current crisis. It said: “The FMD epidemic in UK in 2001 was the first situation in which models were developed in the ‘heat’ of an epidemic and used to guide control policy . . . analyses of the field data, suggest that the culling policy may not have been necessary to control the epidemic, as was suggested by the models produced within the first month of the epidemic. If so it must be concluded that the models supporting this decision were inherently invalid.”
The Imperial modellers’ next big public challenge came eight years later when swine flu swept the world — fortunately killing few Britons because older people tended to be immune and younger ones were strong enough to fight it off. Britain was, however, left with 34 million doses of unused and expensive vaccines. Again there was an inquiry — which concluded that ministers had once again treated modellers as “astrologers”, asking them to provide detailed forecasts when they had too little data.
“Modelling did not provide early answers,” it concluded. “The major difficulty with producing accurate models was the lack of a relatively accurate idea of the total number of cases . . . This is not to reject the use of models, but to understand their limitations: modellers are not ‘court astrologers’.”

All of that sounds familiar. I conclude with the famous words of Richard Feynman, one of the 20th centuries most eminent scientists: “Science is the belief in the ignorance of experts.”

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.

Infectious Disease Modeling: What’s the Track Record?