SARS-CoV-2 infection primarily begins in the upper respiratory tract and may progress to the lower respiratory tract, other tissues, and the nervous and cardiovascular systems, which may lead to cytokine storm, pneumonia, ARDS, neurological injury2-18 and cognitive deficits5,10, cardiovascular complications19-25, DNA damage26-29, organ failure, and death. Even mild untreated infections may result in persistent cognitive deficits30—the spike protein binds to fibrin leading to fibrinolysis-resistant blood clots, thromboinflammation, and neuropathology. Minimizing replication as early as possible is recommended.

SARS-CoV-2 infection and replication involves the complex interplay of 500+ host and viral proteins and other factorsA,31-38, providing many therapeutic targets for which many existing compounds have known activity. Scientists have predicted that over 12,000 compounds may reduce COVID-19 risk39, either by directly minimizing infection or replication, by supporting immune system function, or by minimizing secondary complications.

We analyze all significant controlled studies of azithromycin for COVID-19. Search methods, inclusion criteria, effect extraction criteria (more serious outcomes have priority), all individual study data, PRISMA answers, and statistical methods are detailed in Appendix 1. We present random-effects meta-analysis results for all studies, studies within each treatment stage, individual outcomes, peer-reviewed studies, Randomized Controlled Trials (RCTs), and higher quality studies.

Fig. 3 shows stages of possible treatment for COVID-19. Prophylaxis refers to regularly taking medication before becoming sick, in order to prevent or minimize infection. Early treatment refers to treatment immediately or soon after symptoms appear, while late treatment refers to more delayed treatment.

RCTs help to make study groups more similar and can provide a higher level of evidence, however they are subject to many biases54, and analysis of double-blind RCTs has identified extreme levels of bias55. For COVID-19, the overhead may delay treatment, dramatically compromising efficacy; they may encourage monotherapy for simplicity at the cost of efficacy which may rely on combined or synergistic effects; the participants that sign up may not reflect real world usage or the population that benefits most in terms of age, comorbidities, severity of illness, or other factors; standard of care may be compromised and unable to evolve quickly based on emerging research for new diseases; errors may be made in randomization and medication delivery; and investigators may have hidden agendas or vested interests influencing design, operation, analysis, reporting, and the potential for fraud. All of these biases have been observed with COVID-19 RCTs. There is no guarantee that a specific RCT provides a higher level of evidence.

RCTs are expensive and many RCTs are funded by pharmaceutical companies or other organizations with conflicts of interest, for example governments that previously denied treatment with the study drug. For COVID-19, this creates an incentive to show efficacy for patented commercial products, and an incentive to show a lack of efficacy for inexpensive treatments. The bias is expected to be significant, for example Als-Nielsen et al. analyzed 370 RCTs from Cochrane reviews, showing that trials funded by for-profit organizations were 5 times more likely to recommend the experimental drug compared with those funded by nonprofit organizations. Bekelman et al. and Lundh et al. show that industry-sponsored studies are more likely to be favorable. For COVID-19, some major philanthropic organizations are largely funded by investments with extreme conflicts of interest for and against specific COVID-19 interventions.

High quality RCTs for novel acute diseases are more challenging, with increased ethical issues due to the urgency of treatment, increased risk due to enrollment delays, and more difficult design with a rapidly evolving evidence base. For COVID-19, the most common site of initial infection is the upper respiratory tract. Immediate treatment is likely to be most successful and may prevent or slow progression to other parts of the body. For a non-prophylaxis RCT, it makes sense to provide treatment in advance and instruct patients to use it immediately on symptoms, just as some governments have done by providing medication kits in advance. Unfortunately, no RCTs have been done in this way. Every treatment RCT to date involves delayed treatment. Among the 227 treatments we have analyzed, 67% of RCTs involve very late treatment 5+ days after onset. No non-prophylaxis COVID-19 RCTs match the potential real-world use of early treatments. They may more accurately represent results for treatments that require visiting a medical facility, e.g., those requiring intravenous administration.

RCTs have a bias against finding an effect for interventions that are widely available—patients that believe they need the intervention are more likely to decline participation and take the intervention. RCTs for azithromycin are more likely to enroll low-risk participants that do not need treatment to recover, making the results less applicable to clinical practice. This bias is likely to be greater for widely known treatments, and may be greater when the risk of a serious outcome is overstated. This bias does not apply to the typical pharmaceutical trial of a new drug that is otherwise unavailable.

Evidence shows that observational studies can also provide reliable results. Concato et al. found that well-designed observational studies do not systematically overestimate the magnitude of the effects of treatment compared to RCTs. Anglemyer et al. analyzed reviews comparing RCTs to observational studies and found little evidence for significant differences in effect estimates.

We performed a similar analysis across the 227 treatments we cover, showing no significant difference in the results of RCTs compared to observational studies, RR 0.97 [0.91‑1.03]44. Similar results are found for all low-cost treatments, RR 0.98 [0.90‑1.06]. High-cost treatments show a non-significant trend towards RCTs showing greater efficacy, RR 0.92 [0.84‑1.02]. Details can be found in the supplementary data.

Lee et al. showed that only 14% of the guidelines of the Infectious Diseases Society of America were based on RCTs. Evaluation of studies relies on an understanding of the study and potential biases. Limitations in an RCT can outweigh the benefits, for example excessive dosages, excessive treatment delays, or remote survey bias may have a greater effect on results. Ethical issues may also prevent running RCTs for known effective treatments. For more on issues with RCTs see63,64.

Concato et al. report a paradoxical finding—RCT results had higher variability, and only RCTs were found to sometimes report significant results the opposite of the overall result. The same trend is seen for the most popular (most politicized) COVID-19 treatments—considering all statistically significant results reported in studies, RCTs are slightly more likely to report a result in the opposite direction. In other words, for these COVID-19 treatments and for the topics covered by Concato et al., assuming causality from a single study is more likely to result in an incorrect conclusion for RCTs.

Increased risk of inconsistent results for RCTs suggests higher prevalence of bias, which may arise due to many issues including design bias, conflicts of interest, treatment differences by physicians aware of allocation, attrition bias, ascertainment bias, randomization failures, errors, or fraud.

Currently, 59 of the treatments we analyze show statistically significant efficacy or harm, defined as ≥10% decreased risk or >0% increased risk from ≥3 studies. Of these, 54% have been confirmed in RCTs, with a mean delay of 7.8 months (62% with 8.7 months delay for low-cost treatments). The remaining treatments either have no RCTs, or the point estimate is consistent.

Neither observational studies nor RCTs prove causation—any study can be flawed or fraudulent. We need much more, for example a combination of results from many independent teams, detailed understanding of each study, knowledge of conflicts/team reliability, dose-response relationships, delay-response relationships, logical results across outcomes, or details consistent with preclinical expectations.

All studies must be evaluated individually. RCTs for a given medication and disease may be more reliable, however they may also be less reliable. For off-patent medications, very high conflict of interest trials may be more likely to be RCTs, and more likely to be large trials that dominate meta-analyses.