Just How User-Friendly Are Current Rapid At-Home COVID-19 Test Kits?

Nearly a dozen at-home COVID-19 antigen test kits have received Emergency Use Authorization (EUA) from the US Food and Drug Administration. And that begs a crucial question: Just how user-friendly are these at-home tests? A new report from health care not-for-profit ECRI is the first to study the usability of current at-home products. Regrettably, the findings are not terribly encouraging.

At-home testing and the pandemic

At-home tests can be a win-win. In addition to providing rapid results—often in under 10 minutes—home antigen tests are more convenient for consumers who do not fancy the prospect of waiting in long lines at testing centers. At-home rapid tests also reduce the burden on laboratories that are currently overwhelmed with back orders to test patients displaying COVID-19 symptoms. The fly in the ointment are the risks that come with relying on untrained patients to collect and test their own samples. At-home COVID-19 tests are performed on tissue samples collected using a long swab. While throat swabs are allowed in some countries, at-home COVID-19 test kits in the US are cleared only for nasal collection. Ideally, these samples should be collected by a trained health care professional and tested by a CLIA-approved laboratory. In addition to posing health hazards (see the related item below), self-collection and testing increases the risk of inaccurate test results.

FDA Warns Against Using At-Home COVID-19 Kit Swabs in the Throat The FDA sent a Twitter message to consumers who use at-home rapid antigen COVID-19 tests: Use the swabs that come in the kit in your nose and not your throat. The Jan. 7 warning comes in response to reports that some people are using the swabs intended to collect nasal samples from their throats and posting the results on social media with the hashtag #SwabYourThroat. “The FDA advises that COVID-19 tests should be used as authorized, including following their instructions for use regarding obtaining the sample for testing,” an FDA spokesperson told CNN. “The FDA has noted safety concerns regarding self-collection of throat swabs, as they are more complicated than nasal swabs—and if used incorrectly, can cause harm to the patient," the spokesperson added. "The CDC recommends that throat swabs be collected by a trained health care provider."

The current consensus among policy makers, scientists, and health care providers is that these risks are acceptable given the current exigencies of the pandemic. In December 2020, Australian company Ellume became the first to receive FDA authorization for an over-the-counter, at-home COVID-19 test. Since then, the agency has granted EUA for at-home products from Abbott, Becton Dickinson, Quidel, Access Bio, and, most recently, Amazon, Roche, and Siemens Healthineers. At-home products have also been the centerpiece of the Biden COVID-19 testing strategy. The president has purchased and is in the process of distributing over $3 billion worth of free rapid at-home tests to individuals who request them. It has also ordered private insurers to cover the costs of at-home testing, effective Jan. 15.

ECRI Evaluates At-Home COVID-19 Test Usability

To be effective, at-home tests must be user-friendly. “If an at-home test is complicated, if it's cumbersome, it can dramatically increase the error rate,” said Marcus Schabacker, president and CEO of ECRI, in the report. “If you mix up how to do the test, you might get a false positive or more importantly, a false negative test.” The ECRI study is the first to evaluate the usability of currently available at-home COVID-19 tests. ECRI used a System Usability Scale (SUS) measuring perceived ease of use based on a questionnaire in which individuals who use the product assign a score of between 1 and 5 to indicate the extent of their agreement (with 5 indicating the highest level of agreement) with a general statement about the product’s usability. Aggregate scores range from 0 (Worst Imaginable) to 100 (Best Imaginable).Scale adapted from: Aaron Bangor, Philip T. Kortum & James T. Miller (2008)An Empirical Evaluation of the System Usability Scale, International Journal of Human–Computer Interaction, 24:6, 574-594, DOI: 10.1080/10447310802205776 via ECRI

Study Results

None of the seven products that ECRI evaluated received a SUS score in the Excellent range:

• The three highest scoring kits (On/Go, CareStart, Flowflex) kits fell a bit short of Excellent with a Very Good rating;
• Three kits (QuickVue, BinaxNOW, InteliSwab) achieved Good scores; and
• The BD Veritor kit was rated as Marginally Acceptable.

When asked to indicate which of the seven tests they found the most difficult to use:

• 67 percent listed BD Veritor;
• 22 percent listed BinaxNOW; and
• 11 percent listed the InteliSwab test.

ECRI Usability Comparison of Rapid At-Home COVID-19 Test Kits

Product	Manufacturer	Usability Rating	User Comments
On/Go	AccessBio	82.9 Very Good	+ Good written instructions + Cassette allows for handling without touching test strip - App allows for photographing but doesn’t interpret result
CareStart	AccessBio	80.8 Very Good	+ Good written instructions + Cassette allows for handling without touching test strip - App allows for photographing but doesn’t interpret result
Flowflex	ACON Laboratories	79.5 Very Good	+ Good written instructions + Cassette allows for handling without touching test strip + Test result easy to interpret - Hard to attach dropper tip into buffer vial - QR code for video instructions only linked to printed instructions
QuickVue	Quidel	75.6 Good	+ Clear printed instructions - Hard to remove cap from reagent tube - Some packaging hard to open
BinaxNOW	Abbott Laboratories	73.3 Good	+ Clear and easy instructions - Complicated to perform - Hard to drip 6 small drops into tiny readout card - Relatively small font instructions
InteliSwab	OraSure Technologies	73.3 Good	+ No need to dip reagent into tester + Test result easy to interpret + QR-linked instructional video easy to follow - 30-minute test readout too long to wait - Hard to remove cap from reagent vial - Tapered flat swab part of test device is uncomfortable - 15 nasal swirls required vs 5 for most other tests
BD Veritor	Becton Dickinson	51.8 OK (Marginally Acceptable)	+ App video guidance helpful + App enables forwarding test result to other parties - No written test instructions - No picture of what positive or negative test looks like—must rely on app’s interpretation

In addition to the lackluster usability scores, ECRI noted that none of the seven tests came with directions in languages other than English or Spanish, important factors in considering health equity.