SARS-CoV-2 infection primarily begins in the upper respiratory tract and may progress to the lower respiratory tract, other tissues, and the nervous and cardiovascular systems, which may lead to cytokine storm, pneumonia, ARDS, neurological injury2-18 and cognitive deficits5,10, cardiovascular complications19-25, DNA damage26-29, organ failure, and death. Even mild untreated infections may result in persistent cognitive deficits30—the spike protein binds to fibrin leading to fibrinolysis-resistant blood clots, thromboinflammation, and neuropathology. Minimizing replication as early as possible is recommended.

SARS-CoV-2 infection and replication involves the complex interplay of 500+ host and viral proteins and other factorsA,31-38, providing many therapeutic targets for which many existing compounds have known activity. Scientists have predicted that over 12,000 compounds may reduce COVID-19 risk39, either by directly minimizing infection or replication, by supporting immune system function, or by minimizing secondary complications.

We analyze all significant controlled studies of lopinavir/ritonavir for COVID-19. Search methods, inclusion criteria, effect extraction criteria (more serious outcomes have priority), all individual study data, PRISMA answers, and statistical methods are detailed in Appendix 1. We present random-effects meta-analysis results for all studies, studies within each treatment stage, individual outcomes, peer-reviewed studies, Randomized Controlled Trials (RCTs), and higher quality studies.

Fig. 3 shows stages of possible treatment for COVID-19. Prophylaxis refers to regularly taking medication before becoming sick, in order to prevent or minimize infection. Early treatment refers to treatment immediately or soon after symptoms appear, while late treatment refers to more delayed treatment.

RCTs help to make study groups more similar and can provide a higher level of evidence, however they are subject to many biases42, and analysis of double-blind RCTs has identified extreme levels of bias43. For COVID-19, the overhead may delay treatment, dramatically compromising efficacy; they may encourage monotherapy for simplicity at the cost of efficacy which may rely on combined or synergistic effects; the participants that sign up may not reflect real world usage or the population that benefits most in terms of age, comorbidities, severity of illness, or other factors; standard of care may be compromised and unable to evolve quickly based on emerging research for new diseases; errors may be made in randomization and medication delivery; and investigators may have hidden agendas or vested interests influencing design, operation, analysis, reporting, and the potential for fraud. All of these biases have been observed with COVID-19 RCTs. There is no guarantee that a specific RCT provides a higher level of evidence.

RCTs are expensive and many RCTs are funded by pharmaceutical companies or other organizations with conflicts of interest, for example governments that previously denied treatment with the study drug. For COVID-19, this creates an incentive to show efficacy for patented commercial products, and an incentive to show a lack of efficacy for inexpensive treatments. The bias is expected to be significant, for example Als-Nielsen et al. analyzed 370 RCTs from Cochrane reviews, showing that trials funded by for-profit organizations were 5 times more likely to recommend the experimental drug compared with those funded by nonprofit organizations. Bekelman et al. and Lundh et al. show that industry-sponsored studies are more likely to be favorable. For COVID-19, some major philanthropic organizations are largely funded by investments with extreme conflicts of interest for and against specific COVID-19 interventions.

High quality RCTs for novel acute diseases are more challenging, with increased ethical issues due to the urgency of treatment, increased risk due to enrollment delays, and more difficult design with a rapidly evolving evidence base. For COVID-19, the most common site of initial infection is the upper respiratory tract. Immediate treatment is likely to be most successful and may prevent or slow progression to other parts of the body. For a non-prophylaxis RCT, it makes sense to provide treatment in advance and instruct patients to use it immediately on symptoms, just as some governments have done by providing medication kits in advance. Unfortunately, no RCTs have been done in this way. Every treatment RCT to date involves delayed treatment. Among the 227 treatments we have analyzed, 67% of RCTs involve very late treatment 5+ days after onset. No non-prophylaxis COVID-19 RCTs match the potential real-world use of early treatments. They may more accurately represent results for treatments that require visiting a medical facility, e.g., those requiring intravenous administration.

Concato et al. report a paradoxical finding—RCT results had higher variability, and only RCTs were found to sometimes report significant results the opposite of the overall result. The same trend is seen for the most popular (most politicized) COVID-19 treatments—considering all statistically significant results reported in studies, RCTs are slightly more likely to report a result in the opposite direction. In other words, for these COVID-19 treatments and for the topics covered by Concato et al., assuming causality from a single study is more likely to result in an incorrect conclusion for RCTs.

Increased risk of inconsistent results for RCTs suggests higher prevalence of bias, which may arise due to many issues including design bias, conflicts of interest, treatment differences by physicians aware of allocation, attrition bias, ascertainment bias, randomization failures, errors, or fraud.

Currently, 59 of the treatments we analyze show statistically significant efficacy or harm, defined as ≥10% decreased risk or >0% increased risk from ≥3 studies. Of these, 54% have been confirmed in RCTs, with a mean delay of 7.8 months (62% with 8.7 months delay for low-cost treatments). The remaining treatments either have no RCTs, or the point estimate is consistent.

Neither observational studies nor RCTs prove causation—any study can be flawed or fraudulent. We need much more, for example a combination of results from many independent teams, detailed understanding of each study, knowledge of conflicts/team reliability, dose-response relationships, delay-response relationships, logical results across outcomes, or details consistent with preclinical expectations.

All studies must be evaluated individually. RCTs for a given medication and disease may be more reliable, however they may also be less reliable. For off-patent medications, very high conflict of interest trials may be more likely to be RCTs, and more likely to be large trials that dominate meta-analyses.

The time between infection or the onset of symptoms and treatment may critically affect how well a treatment works. For example an antiviral may be very effective when used early but may not be effective in late stage disease, and may even be harmful. Oseltamivir, for example, is generally only considered effective for influenza when used within 0-36 or 0-48 hours52,53. Baloxavir marboxil studies for influenza also show that treatment delay is critical — Ikematsu et al. report an 86% reduction in cases for post-exposure prophylaxis, Hayden et al. show a 33 hour reduction in the time to alleviation of symptoms for treatment within 24 hours and a reduction of 13 hours for treatment within 24-48 hours, and Kumar et al. report only 2.5 hours improvement for inpatient treatment.

Fig. 19 shows a mixed-effects meta-regression for efficacy as a function of treatment delay in COVID-19 studies from 227 treatments, showing that efficacy declines rapidly with treatment delay. Early treatment is critical for COVID-19.

Details of the patient population including age and comorbidities may critically affect how well a treatment works. For example, many COVID-19 studies with relatively young low-comorbidity patients show all patients recovering quickly with or without treatment. In such cases, there is little room for an effective treatment to improve results, for example as in López-Medina et al.

Efficacy may depend critically on the distribution of SARS-CoV-2 variants encountered by patients. Risk varies significantly across variants58, for example the Gamma variant shows significantly different characteristics59-62. Different mechanisms of action may be more or less effective depending on variants, for example the degree to which TMPRSS2 contributes to viral entry can differ across variants63,64.

Effectiveness may depend strongly on the dosage and treatment regimen.

The quality of medications may vary significantly between manufacturers and production batches, which may significantly affect efficacy and safety. Williams et al. analyze ivermectin from 11 different sources, showing highly variable antiparasitic efficacy across different manufacturers. Xu et al. analyze a treatment from two different manufacturers, showing 9 different impurities, with significantly different concentrations for each manufacturer.

The use of other treatments may significantly affect outcomes, including supplements, other medications, or other interventions such as prone positioning. Treatments may be synergistic67-91, therefore efficacy may depend strongly on combined treatments.

Across all studies there is a strong association between different outcomes, for example improved recovery is strongly associated with lower mortality. However, efficacy may differ depending on the effect measured, for example a treatment may be more effective against secondary complications and have minimal effect on viral clearance.

The distribution of studies will alter the outcome of a meta-analysis. Consider a simplified example where everything is equal except for the treatment delay, and effectiveness decreases to zero or below with increasing delay. If there are many studies using very late treatment, the outcome may be negative, even though early treatment is very effective. All meta-analyses combine heterogeneous studies, varying in population, variants, and potentially all factors above, and therefore may obscure efficacy by including studies where treatment is less effective. Generally, we expect the estimated effect size from meta-analysis to be less than that for the optimal case. Looking at all studies is valuable for providing an overview of all research, important to avoid cherry-picking, and informative when a positive result is found despite combining less-optimal situations. However, the resulting estimate does not apply to specific cases such as early treatment in high-risk populations. While we present results for all studies, we also present treatment time and individual outcome analyses, which may be more informative for specific use cases.

For COVID-19, delay in clinical results translates into additional death and morbidity, as well as additional economic and societal damage. Combining the results of studies reporting different outcomes is required. There may be no mortality in a trial with low-risk patients, however a reduction in severity or improved viral clearance may translate into lower mortality in a high-risk population. Different studies may report lower severity, improved recovery, and lower mortality, and the significance may be very high when combining the results. "The studies reported different outcomes" is not a good reason for disregarding results. Pooling the results of studies reporting different outcomes allows us to use more of the available information. Logically we should, and do, use additional information when evaluating treatments—for example dose-response and treatment delay-response relationships provide additional evidence of efficacy that is considered when reviewing the evidence for a treatment.

We present both specific outcome and pooled analyses. In order to combine the results of studies reporting different outcomes we use the most serious outcome reported in each study, based on the thesis that improvement in the most serious outcome provides comparable measures of efficacy for a treatment. A critical advantage of this approach is simplicity and transparency. There are many other ways to combine evidence for different outcomes, along with additional evidence such as dose-response relationships, however these increase complexity.

Trials with high-risk patients may be restricted due to ethics for treatments that are known or expected to be effective, and they increase difficulty for recruiting. Using less severe outcomes as a proxy for more serious outcomes allows faster and safer collection of evidence.

Currently, 59 of the treatments we analyze show statistically significant efficacy or harm, defined as ≥10% decreased risk or >0% increased risk from ≥3 studies. 85% of these have been confirmed with one or more specific outcomes, with a mean delay of 4.6 months. When restricting to RCTs only, 53% of treatments showing statistically significant efficacy/harm with pooled effects have been confirmed with one or more specific outcomes, with a mean delay of 7.5 months. Fig. 23 shows when treatments were found effective during the pandemic. Pooled outcomes often resulted in earlier detection of efficacy.

Pooled analysis could hide efficacy, for example a treatment that is beneficial for late stage patients but has no effect on viral clearance may show no efficacy if most studies only examine viral clearance. In practice, it is rare for a non-antiviral treatment to report viral clearance and to not report clinical outcomes; and in practice other sources of heterogeneity such as differences in treatment delay are more likely to hide efficacy.

Analysis validates the use of pooled effects and shows significantly faster detection of efficacy on average. However, as with all meta-analyses, it is important to review the different studies included. We also present individual outcome analyses, which may be more informative for specific use cases.

Publishing is often biased towards positive results. Trials with patented drugs may have a financial conflict of interest that results in positive studies being more likely to be published, or bias towards more positive results. For example with molnupiravir, trials with negative results remain unpublished to date (CTRI/2021/05/033864 and CTRI/2021/08/0354242). For lopinavir/ritonavir, there is currently not enough data to evaluate publication bias with high confidence.

Funnel plots have traditionally been used for analyzing publication bias. This is invalid for COVID-19 acute treatment trials — the underlying assumptions are invalid, which we can demonstrate with a simple example. Consider a set of hypothetical perfect trials with no bias. Fig. 24 plot A shows a funnel plot for a simulation of 80 perfect trials, with random group sizes, and each patient's outcome randomly sampled (10% control event probability, and a 30% effect size for treatment). Analysis shows no asymmetry (p > 0.05). In plot B, we add a single typical variation in COVID-19 treatment trials — treatment delay. Consider that efficacy varies from 90% for treatment within 24 hours, reducing to 10% when treatment is delayed 3 days. In plot B, each trial's treatment delay is randomly selected. Analysis now shows highly significant asymmetry, p < 0.0001, with six variants of Egger's test all showing p < 0.0593-100. Note that these tests fail even though treatment delay is uniformly distributed. In reality treatment delay is more complex — each trial has a different distribution of delays across patients, and the distribution across trials may be biased (e.g., late treatment trials may be more common). Similarly, many other variations in trials may produce asymmetry, including dose, administration, duration of treatment, differences in SOC, comorbidities, age, variants, and bias in design, implementation, analysis, and reporting.

Summary statistics from meta-analysis necessarily lose information. As with all meta-analyses, studies are heterogeneous, with differences in treatment delay, treatment regimen, patient demographics, variants, conflicts of interest, standard of care, and other factors. We provide analyses for specific outcomes and by treatment delay, and we aim to identify key characteristics in the forest plots and summaries. Results should be viewed in the context of study characteristics.

Some analyses classify treatment based on early or late administration, as done here, while others distinguish between mild, moderate, and severe cases. Viral load does not indicate degree of symptoms — for example patients may have a high viral load while being asymptomatic. With regard to treatments that have antiviral properties, timing of treatment is critical — late administration may be less helpful regardless of severity.

Details of treatment delay per patient is often not available. For example, a study may treat 90% of patients relatively early, but the events driving the outcome may come from 10% of patients treated very late. Our 5 day cutoff for early treatment may be too conservative, 5 days may be too late in many cases.

Comparison across treatments is confounded by differences in the studies performed, for example dose, variants, and conflicts of interest. Trials with conflicts of interest may use designs better suited to the preferred outcome.

In some cases, the most serious outcome has very few events, resulting in lower confidence results being used in pooled analysis, however the method is simpler and more transparent. This is less critical as the number of studies increases. Restriction to outcomes with sufficient power may be beneficial in pooled analysis and improve accuracy when there are few studies, however we maintain our pre-specified method to avoid any retrospective changes.

Studies show that combinations of treatments can be highly synergistic and may result in many times greater efficacy than individual treatments alone67-91. Therefore standard of care may be critical and benefits may diminish or disappear if standard of care does not include certain treatments.

This real-time analysis is constantly updated based on submissions. Accuracy benefits from widespread review and submission of updates and corrections from reviewers. Less popular treatments may receive fewer reviews.

No treatment or intervention is 100% available and effective for all current and future variants. Efficacy may vary significantly with different variants and within different populations. All treatments have potential side effects. Propensity to experience side effects may be predicted in advance by qualified physicians. We do not provide medical advice. Before taking any medication, consult a qualified physician who can compare all options, provide personalized advice, and provide details of risks and benefits based on individual medical history and situations.

1 of the 18 studies compare against other treatments, which may reduce the effect seen.

SARS-CoV-2 infection and replication involves a complex interplay of 500+ host and viral proteins and other factors31-38, providing many therapeutic targets. Over 12,000 compounds have been predicted to reduce COVID-19 risk39, either by directly minimizing infection or replication, by supporting immune system function, or by minimizing secondary complications. Fig. 25 shows an overview of the results for lopinavir/ritonavir in the context of multiple COVID-19 treatments, and Fig. 26 shows a plot of efficacy vs. cost for COVID-19 treatments.