For anyone who has been following the modeling projections for the spread of COVID-19, there was an apparent bombshell in the last few days: The Imperial College London model that was released roughly two weeks ago — and which generated massive media and policy maker attention — was updated.
Based on more data, the modelers were increasingly confident that their worst-case scenario, which they previously characterized as “unlikely,” was even less likely to occur. So they revised their projections for infections, hospitalizations and deaths substantially downward.
The Wall Street Journal published an editorial on March 27, “Worst-Case Coronavirus Science,” and it is a must read for anyone interested in understanding the likely trajectory of the pandemic.
The Journal made four points that warrant elaboration:
We need to update our projections when the situation changes.
In California, our COVID-19 situation is going to change two times in the next three weeks, and everyone needs to recognize this. The simplest way to think about this is as follows:
COVID-19 is hitting California in three waves. The first wave corresponds to the roughly unchecked spread of the virus before March 8, which is when state and local governments started issuing escalating public health orders to slow the spread.
The second wave corresponds to the period between March 8 and March 19, when Gov. Gavin Newsom’s stay-at-home order was issued. Although the virus will have been spreading rapidly during this phase, it should have been at a sub-maximum rate because of the public health measures in place.
Finally, there is Wave 3, which corresponds to the period after the stay-at-home order was issued.
The key question is when is each of these waves going to hit? Because the virus has a 14-day incubation period and our testing results are not reported immediately, each wave has a potentially 14-21-plus-day lifespan after its last calendar day.
I have used 19 days in my models. This means that the Wave 1 trajectory could continue to appear in our data on confirmed cases, hospitalizations, etc. as late as March 29. In other words, there is a decent chance that some of the new cases reported over the weekend originated before we undertook any public health measures.
The data from Wave 2 could appear in our data as late as April 8. This means that the full benefits from the stay-at-home order will likely not be visible in the data until roughly April 9, although there will be clues in the data long before then.
If the decentralized public health measures taken before March 19 were effective, we should start seeing that impact reflected in the data this week. Please note that this will not mean a decrease in cases or hospitalizations. The numbers will continue to increase, but at a slower rate of acceleration.
That is what we need to look for in the data. We should then see another change when the full impact of the stay-at-home order is reflected in the data, which I am estimating to occur on April 9.
“If we hope to neutralize a pandemic we don’t fully understand, we need to encourage a culture in which scientists feel able to adapt and clarify with new evidence. Scientists would help themselves if, in explaining their findings, they would be more candid about the assumptions and variables.”
As someone with a public policy and modeling background, I could not agree more. This is why I explicitly list every assumption and variable upon which my model for California health-care capacity depends, and note that it will be updated as more and better data become available. I expect to release an updated model early this week.
I would add a key corollary principle: Publishing models for peer review would also improve our ability to create more precise models to guide policy makers. This is why I have repeatedly called on the state to publish the model it is using to predict the spread of COVID-19 in California and the health-care system capacity required to handle that spread.
The initial California predictions were crafted at roughly the same time as the Imperial College model, and they likely reflected an equally unlikely worst-case scenario. They need to be tested and updated based on actual data, just as the Imperial College model was.
“Instead of a presentation of what we know and don’t, too often the focus (of news media reporting) has been … sensationalizing.”
Here too, I could not agree more. Most of this alarmist reporting stems from an inability or unwillingness in the news media to do the math required to present information about the spread of the virus in the correct context.
Here are three examples of how this dynamic manifests itself:
1. The “Surge” or “Spike” in Cases Each Day
The trajectory of a spreading virus can range from an increase in daily cases of 1x/day (a totally flattened curve) to > 1.3x/day (an unchecked virus). We are nowhere near averaging 1.3x/day in California. This is already clear from the data.
We also have not seen a surge in cases. This chart below explains the differences in virus trajectories and compares them to California over the six days ending March 26:
The blue line represents the uncontrolled growth of COVID-19. The red line represents constant growth (i.e., no spike or surge) at a rapid, but sub-maximal rate. The green line represents the growth in daily cases in California over the past six days. The purple line represents a curve that is flattening.
As you can see, there has not been any surge or spike in our daily confirmed cases, except (perhaps — see below) from Day 5 to Day 6, despite many headlines to the contrary over the past six days. Overall, the California line tracks the constant rate of growth line fairly closely.
2. But wait a minute. Shouldn’t I be worried about that big jump from Day 5 to Day 6 (March 26)?
The answer is: “Not yet, and probably not.”
I am unwilling to read too much into one day of reporting because the number of tests performed each day varies, there is a variable lag in reporting test results, and California has a backlog of test results that have not been reported (and may be disproportionately negative).
Under these facts, the cases confirmed today could include tests that were administered 48 hours ago, 120 hours ago or 168 hours ago (or more, conceivably). And we could get a batch of results today that is much larger or smaller than the day before.
So, we cannot compare daily confirmed cases with prior days on a truly apples-to-apples basis to figure out if the situation is worsening.
And remember: The number of new daily cases does not always increase. It actually decreased in California on March 15, 18 and 20.
But no one declared the spread of the virus to be in decline based on those three days, viewed in isolation from the larger context. So why are we declaring “surges” and “spikes” when the opposite occurs on a given day? We shouldn’t.
One way to try to address this lag and variability in the data would be to report cases as a moving three-, four- or five-day average. This would, in theory, smooth out the curve and produce a more accurate snapshot of the spread of the virus over time.
But it would not be perfect, and a moving average based on many days could be misleading as well; it could overly smooth the curve and delay the recognition of important changes.
So I calculated the moving three-day average increase in daily cases for California from March 12-26 in order to reduce the impact of dramatic one-day differences without overly smoothing the curve.
Here’s what the curve looks like, including our seeming spike on March 26:
There are three important characteristics of this curve that everyone should understand:
» It looks more like an EKG — the data are messy (as we expect given all of the reporting challenges noted above).
» The “surge” on March 19 wasn’t actually a surge (and the most recent spike may not prove to be either; we’ll know in one more day).
» Critically, the trend line is fairly constant. This means that when we look at trailing three-day averages, the public measures put in place before the stay-at-home order may have capped the rate of growth near 1.2x/day. (This is just another way of depicting the red line from the first chart — a constant rate of growth).
This should become much clearer in the data over the next few days (depending on how California clears its backlog). The more important question is whether the curve has been flattened at a rate of growth that the health0care system can handle. I believe the data will show that it has been by roughly April 9.
3. What would a flattened curve look like in California?
In a very simplified form, my March 26 Model projects the following curve for California daily hospital admissions from March 24 to April 16:
This shows daily hospital admissions rising for several days, then dropping slightly as the full impact of the stay-at-home order is achieved, and then leveling off as long as the order remains in place. Note: My curve for daily confirmed new cases would be identical (but with different numbers).
Here is the critical point to understand from this curve: With public health measures in place, we can manage a certain (reasonably flat) level of new COVID-19 hospital admissions each day for an indefinite period of time (this is how the health-care system functions in normal periods).
This is because at a certain point in the controlled spread of the virus, the number of new hospital cases each day will roughly correspond to the number of discharges (and sadly, deaths) each day.
The point of the stay-at-home order is to ensure we are at (or below, ideally) this number of daily hospital admissions (which by my calculations, is well above 588 new admissions/day).
Using the predicted rate of daily growth in hospital admissions, the curve for my March 26 Model would look something like this:
This curve reflects a constant rate of growth before the stay-at-home order begins to take effect (as noted above). When it does, the rate of growth slows to its lowest level under the order. From that point forward, the rate of growth is flat so long as the stay-at-home order is maintained. Note: I don’t expect the curve in either chart to be completely flat. Instead, I expect it to look like an EKG hovering very closely around the flattened trend line.
These are the types of curves we should be reporting to inform the public regarding the trajectory of the outbreak, rather than graphs that simply show the inexorable (and seemingly steep) rise in total confirmed cases (the vast majority of which turn into confirmed recoveries).
“In the battle to save lives and address the scourge of COVID-19, good information is paramount.”
I could not agree more. This is why I have called on the government to collect and publish the data we need to accurately model the course of the virus and our health-care system capacity, assess whether the stay-at-home order has worked and, if so, whether we have less-disruptive public health measures we could employ to control the growth of the virus over the mid-to-long term.
Conclusions
As I have said, models are only as good as the assumptions and data on which they are built. I would encourage everyone to examine these issues carefully before accepting a model as reasonable, including the (unpublished) model California has used to predict the spread of the virus and health-care system capacity in the state.
Please do not be overly alarmed by the reporting on any particular day. I would suggest looking at the trends based on a moving three-day average. I may even start reporting the data this way to make it easier for people to determine the actual trajectory of the coronavirus.
Lastly, ignore the count of total cases. It does not tell you what you need to know about the spread of the virus or its likely impact on the health-care system. Instead, focus on the rate of growth in daily confirmed new cases or hospital admissions (if we get that data) (using a moving three-day average).
If it is constant or trending downward toward 1x, then we are on the right track. Right now, it looks like we are.
Read critically. Look at sustained trends, not daily headlines. Stay well.
— Brian Goebel served as a senior official in the Treasury and Homeland Security departments following 9/11. Since 2005, he has founded successful consulting and analytics firms serving governments around the globe; launched 2040 Matters, a nonpartisan public policy blog dedicated to restoring the American Dream for younger Americans; and was elected to the Montecito Water District Board of Directors in 2018. Click here for previous columns. The opinions expressed are his own.