Communities across the globe are trying to understand what percent of their population has been exposed to COVID-19 by searching random samples of residents for antibodies against the virus.
Los Angeles County, the most populous county in the U.S., 863 people in an antibody study. New York tested 3,000. A Boston study looked at just 200 people passing a certain street corner. But one sparsely populated county in the Rocky Mountains has done a more thorough job than any of the others. It tested about 70% of its population -- almost 5,500 people. And the results are now in.
San Miguel County in Colorado teamed up with a biotech company (founded by a couple of its residents) to offer a COVID-19 antibody test to all residents who wanted one. It took about a month to get results back, but they show that about 30 people .
鈥淥nly 0.5% of our population tested positive for the antibodies,鈥� said Susan Lilly, a spokesperson with the county. Even if you add in 80 more people -- those who were 鈥渂orderline,鈥� meaning there鈥檚 a chance they were in the early stages of producing antibodies -- that鈥檚 still less than 2%.
鈥淚n a lot of ways it鈥檚 very good news because we haven鈥檛 had the health impact or devastation that so many people and so many areas have,鈥� said Lilly. 鈥淥n the other hand, it can be considered bad news if you say, 鈥榃ell, gee, we haven't been hit. Yet.鈥欌��
This fits with the picture emerging as regions search their residents鈥� blood for antibodies. Antibody surveys in other places have also found evidence of low prevalence, with exceptions like New York City (21% positive) and Heinsberg, Germany (14% positive).
Public health professionals were crossing their fingers that antibody studies would reveal lots of people who鈥檇 already been infected and could return to regular life with their newfound immune armor. But, as The Atlantic last month, 鈥渋f the first results from antibody surveys, also known as 鈥榮erosurveys,鈥� in the U.S. are anything to judge by, simply not enough people are immune. Too many Americans are still vulnerable to COVID-19 infection for these tests to be the 鈥榞ame changer鈥� that many were hoping for.鈥�
鈥淵ou would have hoped for 45% or even 60% positive,鈥� Mark Perkins, a diagnostics expert at the World Health Organization, told in late April. 鈥淭hat would mean that there is lots of silent transmission, and a lot of immunity in the population. It now looks like, sadly, that鈥檚 not true. Even the high numbers are relatively small.鈥�
Dr. Ajay Sethi, an infectious disease epidemiologist at the University of Wisconsin, Madison, says the San Miguel County numbers are, 鈥渋n a lot of ways, expected鈥� due to results from other places. But what鈥檚 different about San Miguel County鈥檚 antibody testing is its scale.
For example, in Santa Clara County, California, researchers were for their methods of recruiting volunteers and extrapolating their findings to the overall population, which some said led to an overestimate of the percent infected.
鈥淭here's always a question of how representative the sample that was studied is of the entire population," Sethi said. "Sampling matters, especially in the context of shelter-in-place policies, where so many people are not even leaving their homes in order to be at a supermarket to get tested or being selected for testing. So you have to, you know, take those results with a grain of salt. The San Miguel County study, though not very many people came back positive, what was interesting about that was two-thirds of the county was tested. And that's a very large sample. I think that's pretty representative.鈥�
Scientists say there are a lot of caveats with antibody testing -- it鈥檚 still unclear to what extent antibodies against COVID-19 mean immunity against COVID-19, and if so, for how long. And antibody test results can be difficult to interpret, with the possibility for both false positives and false negatives. A lot of that has to do with statistics, rather than with the performance of a specific test.
"It is kind of a strange thing," Dr. H. Gilbert Welch, who studies issues with tests and screening at Brigham and Women's Hospital, . "An antibody test is much more likely to be wrong in a population with very little COVID exposure."
And, unfortunately, it means that even a bigger, more representative sample doesn鈥檛 guarantee better results.
Say the same test was used on two towns of equal size, the only difference being that one had a worse outbreak than the other. As created by NPR shows, in the town where 20% of people had recovered from COVID-19, each person who got the test would only have a 4% chance of being told they had antibodies against the virus, when in fact they didn鈥檛. In the town where only 1% had recovered from COVID-19, the chance of false positives would rise to 52%. (ProPublica has illustrating this issue).
A lot of this has to do with two somewhat slippery measures of a test鈥檚 reliability: sensitivity and specificity.
Ideally, the perfect antibody test would identify all positives (also known as 鈥�100% specific鈥�) and wouldn鈥檛 misidentify any negatives (also known as 鈥�100% sensitive鈥�). COVAXX, the company that did the San Miguel County antibody tests, claims has such performance. from March from its parent company, United Biomedical, said the sensitivity finding was based on trials in which the test successfully identified COVID-19 antibodies in the blood of an undisclosed number of patients who were confirmed to have COVID-19 by other tests. It said the specificity finding was based on trials in which the test successfully came back 鈥渘egative鈥� after testing 900 blood samples collected from patients before the COVID-19 outbreak started. (That鈥檚 well above the FDA to test at least 75 negative samples.)
But Dr. Lee Riley, a professor of epidemiology and infectious diseases at University of California, Berkeley School of Public Health, said such perfect performance is so far mythical. 鈥淭here鈥檚 no such antibody test. It just doesn鈥檛 exist,鈥� he said.
COVAXX, the company that makes the test, said it was unable to comment on performance, following recently updated guidance from the Food and Drug Administration, and said the company has submitted documentation to the FDA for review.
Assuming 100% specificity is impossible, Riley said, say the test is actually 99% specific, meaning that 1% of those tested will be false positives. Of the 5,500 people tested in San Miguel, 55 people should come back as false positives. The total number of positives, Riley explained, should be 55 plus the number of actual positives, so the fact that San Miguel County came back with just 29 positives is a red flag. 鈥淚 would not rely on the results of that test,鈥� he said.
That鈥檚 not to say this test is any different from others. As the New York Times , lots of tests likely boast better performance than in reality. A group of scientists checked 14 different antibody tests (which did not include the one used in San Miguel County). They found that only one test was 100% specific, meaning all the people it identified as positive were, indeed, positive. Two others were 99% specific, meaning only 1% of people tested were given a falsely positive result. But the three tests with high specificity didn鈥檛 have high sensitivity, at best missing 10% of the people who were, in fact, positive. Initially the FDA wasn鈥檛 inspecting COVID-19 antibody tests the way it normally would in non-pandemic times, but after enough concerns arose about their validity, the agency started companies to submit data about their tests to the agency for review.
If all of this seems confusing, you are not alone. As pointed out, two things are clear about antibody studies so far: 鈥淭he first: Whether the study was conducted in California or Denmark, in the Netherlands or Germany, most have shown the virus has not yet infected a big portion of the screened population 鈥� The second thing: The world still has no idea how to interpret the import of any of these studies.鈥�
However, as Riley pointed out, despite those drawbacks, antibody testing in San Miguel County could still prove useful to gauge whether the disease is spreading or not. 鈥淚f they repeat the same study and they find the prevalence hasn鈥檛 changed, that鈥檚 a good sign that the virus is not spreading,鈥� he said.
A Florida county is doing just that, randomly testing hundreds of people for antibodies, which showed that over at least two weeks in April, the infection rate remained at 6%. According to a county , 鈥渢he fact that this number held steady implies that our physical distancing is working and that together, we are 鈥榝lattening the curve.鈥欌��
In Colorado, San Miguel County is just coming out of a stay-at-home order. It plans to offer the antibody test a second time at some point in the future.
This story was produced by the Mountain West News Bureau, a collaboration between Wyoming Public Media, Boise State Public Radio in Idaho, KUNR in Nevada, the O'Connor Center for the Rocky Mountain West in Montana, KUNC in Colorado, 九色网 in New Mexico, with support from affiliate stations across the region. Funding for the Mountain West News Bureau is provided in part by the .
Do you have questions about COVID-19? How has this crisis affected you? Our reporters would love to hear from you. You can submit your question or share your story .
Copyright 2020
