Table 1 summarizes the results for all stages combined, with different exclusions, and for specific outcomes. Table 2 shows results by treatment stage. Figure 3, 4, 5, 6, 7, 8, 9, 10, 11, and 12 show forest plots for random effects meta-analysis of all studies with pooled effects, mortality results, ventilation, ICU admission, hospitalization, progression, recovery, cases, viral clearance, and peer reviewed studies.

To avoid bias in the selection of studies, we analyze all non-retracted studies. Here we show the results after excluding studies with major issues likely to alter results, non-standard studies, and studies where very minimal detail is currently available. Our bias evaluation is based on analysis of each study and identifying when there is a significant chance that limitations will substantially change the outcome of the study. We believe this can be more valuable than checklist-based approaches such as Cochrane GRADE, which may underemphasize serious issues not captured in the checklists, overemphasize issues unlikely to alter outcomes in specific cases (for example, lack of blinding for an objective mortality outcome, or certain specifics of randomization with a very large effect size), or be easily influenced by potential bias. However, they can also be very high quality.

The studies excluded are as below. Figure 17 shows a forest plot for random effects meta-analysis of all studies after exclusions.

[Di Pierro], unadjusted differences between groups.

[Holt], significant unadjusted confounding possible.

[Veterini], the observered difference in duration could be caused by the baseline difference in Ct values.

Heterogeneity in COVID-19 studies arises from many factors including:

The time between infection or the onset of symptoms and treatment may critically affect how well a treatment works. For example an antiviral may be very effective when used early but may not be effective in late stage disease, and may even be harmful. Oseltamivir, for example, is generally only considered effective for influenza when used within 0-36 or 0-48 hours [McLean, Treanor]. Baloxavir studies for influenza also show that treatment delay is critical — [Ikematsu] report an 86% reduction in cases for post-exposure prophylaxis, [Hayden] show a 33 hour reduction in the time to alleviation of symptoms for treatment within 24 hours and a reduction of 13 hours for treatment within 24-48 hours, and [Kumar] report only 2.5 hours improvement for inpatient treatment.

Figure 18 shows a mixed-effects meta-regression for efficacy as a function of treatment delay in COVID-19 studies from 51 treatments, showing that efficacy declines rapidly with treatment delay. Early treatment is critical for COVID-19.

Details of the patient population including age and comorbidities may critically affect how well a treatment works. For example, many COVID-19 studies with relatively young low-comorbidity patients show all patients recovering quickly with or without treatment. In such cases, there is little room for an effective treatment to improve results (as in [López-Medina]).

Efficacy may differ significantly depending on the effect measured, for example a treatment may be very effective at reducing mortality, but less effective at minimizing cases or hospitalization. Or a treatment may have no effect on viral clearance while still being effective at reducing mortality.

There are many different variants of SARS-CoV-2 and efficacy may depend critically on the distribution of variants encountered by the patients in a study. For example, the Gamma variant shows significantly different characteristics [Faria, Karita, Nonaka, Zavascki]. Different mechanisms of action may be more or less effective depending on variants, for example the viral entry process for the omicron variant has moved towards TMPRSS2-independent fusion, suggesting that TMPRSS2 inhibitors may be less effective [Peacock, Willett].

Effectiveness may depend strongly on the dosage and treatment regimen.

The use of other treatments may significantly affect outcomes, including anything from supplements, other medications, or other kinds of treatment such as prone positioning.

The quality of medications may vary significantly between manufacturers and production batches, which may significantly affect efficacy and safety. [Williams] analyze ivermectin from 11 different sources, showing highly variable antiparasitic efficacy across different manufacturers. [Xu] analyze a treatment from two different manufacturers, showing 9 different impurities, with significantly different concentrations for each manufacturer. Non-prescription supplements may show very wide variations in quality [Crawford, Crighton].

We present both pooled analyses and specific outcome analyses. Notably, pooled analysis often results in earlier detection of efficacy as shown in Figure 19. For many COVID-19 treatments, a reduction in mortality logically follows from a reduction in hospitalization, which follows from a reduction in symptomatic cases, etc. An antiviral tested with a low-risk population may report zero mortality in both arms, however a reduction in severity and improved viral clearance may translate into lower mortality among a high-risk population, and including these results in pooled analysis allows faster detection of efficacy. Trials with high-risk patients may also be restricted due to ethical concerns for treatments that are known or expected to be effective.

Pooled analysis enables using more of the available information. While there is much more information available, for example dose-response relationships, the advantage of the method used here is simplicity and transparency. Note that pooled analysis could hide efficacy, for example a treatment that is beneficial for late stage patients but has no effect on viral replication or early stage disease could show no efficacy in pooled analysis if most studies only examine viral clearance. While we present pooled results, we also present individual outcome analyses, which may be more informative for specific use cases.

Currently, 36 of the treatments we analyze show statistically significant efficacy or harm, defined as ≥10% decreased risk or >0% increased risk from ≥3 studies. 97% of treatments showing statistically significant efficacy/harm with pooled effects have been confirmed with one or more specific outcomes, with a mean delay of 3.1 months. When restricting to RCTs only, 55% of treatments showing statistically significant efficacy/harm with pooled effects have been confirmed with one or more specific outcomes, with a mean delay of 2.9 months.

The distribution of studies will alter the outcome of a meta analysis. Consider a simplified example where everything is equal except for the treatment delay, and effectiveness decreases to zero or below with increasing delay. If there are many studies using very late treatment, the outcome may be negative, even though early treatment is very effective. This may have a greater effect than pooling different outcomes such as mortality and hospitalization. For example a treatment may have 50% efficacy for mortality but only 40% for hospitalization when used within 48 hours. However efficacy could be 0% when used late.

All meta analyses combine heterogeneous studies, varying in population, variants, and potentially all factors above, and therefore may obscure efficacy by including studies where treatment is less effective. Generally, we expect the estimated effect size from meta analysis to be less than that for the optimal case. Looking at all studies is valuable for providing an overview of all research, important to avoid cherry-picking, and informative when a positive result is found despite combining less-optimal situations. However, the resulting estimate does not apply to specific cases such as early treatment in high-risk populations. While we present results for all studies, we also present treatment time and individual outcome analyses, which may be more informative for specific use cases.

Publishing is often biased towards positive results, however evidence suggests that there may be a negative bias for inexpensive treatments for COVID-19. Both negative and positive results are very important for COVID-19, media in many countries prioritizes negative results for inexpensive treatments (inverting the typical incentive for scientists that value media recognition), and there are many reports of difficulty publishing positive results [Boulware, Meeus, Meneguesso]. For probiotics, there is currently not enough data to evaluate publication bias with high confidence.

One method to evaluate bias is to compare prospective vs. retrospective studies. Prospective studies are more likely to be published regardless of the result, while retrospective studies are more likely to exhibit bias. For example, researchers may perform preliminary analysis with minimal effort and the results may influence their decision to continue. Retrospective studies also provide more opportunities for the specifics of data extraction and adjustments to influence results.

Figure 20 shows a scatter plot of results for prospective and retrospective studies. 67% of retrospective studies report a statistically significant positive effect for one or more outcomes, compared to 38% of prospective studies, consistent with a bias toward publishing positive results. The median effect size for retrospective studies is 40% improvement, compared to 35% for prospective studies, suggesting a potential bias towards publishing results showing higher efficacy.

Funnel plots have traditionally been used for analyzing publication bias. This is invalid for COVID-19 acute treatment trials — the underlying assumptions are invalid, which we can demonstrate with a simple example. Consider a set of hypothetical perfect trials with no bias. Figure 21 plot A shows a funnel plot for a simulation of 80 perfect trials, with random group sizes, and each patient's outcome randomly sampled (10% control event probability, and a 30% effect size for treatment). Analysis shows no asymmetry (p > 0.05). In plot B, we add a single typical variation in COVID-19 treatment trials — treatment delay. Consider that efficacy varies from 90% for treatment within 24 hours, reducing to 10% when treatment is delayed 3 days. In plot B, each trial's treatment delay is randomly selected. Analysis now shows highly significant asymmetry, p < 0.0001, with six variants of Egger's test all showing p < 0.05 [Egger, Harbord, Macaskill, Moreno, Peters, Rothstein, Rücker, Stanley]. Note that these tests fail even though treatment delay is uniformly distributed. In reality treatment delay is more complex — each trial has a different distribution of delays across patients, and the distribution across trials may be biased (e.g., late treatment trials may be more common). Similarly, many other variations in trials may produce asymmetry, including dose, administration, duration of treatment, differences in SOC, comorbidities, age, variants, and bias in design, implementation, analysis, and reporting.

Pharmaceutical drug trials often have conflicts of interest whereby sponsors or trial staff have a financial interest in the outcome being positive. Probiotics for COVID-19 lack this because they are generally inexpensive and widely available. In contrast, most COVID-19 probiotics trials have been run by physicians on the front lines with the primary goal of finding the best methods to save human lives and minimize the collateral damage caused by COVID-19. While pharmaceutical companies are careful to run trials under optimal conditions (for example, restricting patients to those most likely to benefit, only including patients that can be treated soon after onset when necessary, and ensuring accurate dosing), not all probiotics trials represent the optimal conditions for efficacy.

Summary statistics from meta analysis necessarily lose information. As with all meta analyses, studies are heterogeneous, with differences in treatment delay, treatment regimen, patient demographics, variants, conflicts of interest, standard of care, and other factors. We provide analyses by specific outcomes and by treatment delay, and we aim to identify key characteristics in the forest plots and summaries. Results should be viewed in the context of study characteristics.

Some analyses classify treatment based on early or late administration, as done here, while others distinguish between mild, moderate, and severe cases. Viral load does not indicate degree of symptoms — for example patients may have a high viral load while being asymptomatic. With regard to treatments that have antiviral properties, timing of treatment is critical — late administration may be less helpful regardless of severity.

Details of treatment delay per patient is often not available. For example, a study may treat 90% of patients relatively early, but the events driving the outcome may come from 10% of patients treated very late. Our 5 day cutoff for early treatment may be too conservative, 5 days may be too late in many cases.

Comparison across treatments is confounded by differences in the studies performed, for example dose, variants, and conflicts of interest. Trials affiliated with special interests may use designs better suited to the preferred outcome.

In some cases, the most serious outcome has very few events, resulting in lower confidence results being used in pooled analysis, however the method is simpler and more transparent. This is less critical as the number of studies increases. Restriction to outcomes with sufficient power may be beneficial in pooled analysis and improve accuracy when there are few studies, however we maintain our pre-specified method to avoid any retrospective changes.

Studies show that combinations of treatments can be highly synergistic and may result in many times greater efficacy than individual treatments alone [Alsaidi, Andreani, Biancatelli, De Forni, Gasmi, Jeffreys, Jitobaom, Jitobaom (B), Ostrov, Thairu]. Therefore standard of care may be critical and benefits may diminish or disappear if standard of care does not include certain treatments.

This real-time analysis is constantly updated based on submissions. Accuracy benefits from widespread review and submission of updates and corrections from reviewers. Less popular treatments may receive fewer reviews.

No treatment, vaccine, or intervention is 100% available and effective for all current and future variants. Efficacy may vary significantly with different variants and within different populations. All treatments have potential side effects. Propensity to experience side effects may be predicted in advance by qualified physicians. We do not provide medical advice. Before taking any medication, consult a qualified physician who can compare all options, provide personalized advice, and provide details of risks and benefits based on individual medical history and situations.

1 of 22 studies combine treatments. The results of probiotics alone may differ. 1 of 11 RCTs use combined treatment. [Neris Almeida Viana] present another meta analysis for probiotics, showing significant improvement for one or more of recovery.

Statistically significant improvements are seen for mortality, hospitalization, progression, and recovery. 11 studies from 11 independent teams in 8 different countries show statistically significant improvements in isolation (6 for the most serious outcome). Meta analysis using the most serious outcome reported shows 25% [15‑33%] improvement. Results are similar for Randomized Controlled Trials, similar after exclusions, and similar for peer-reviewed studies. Better results are seen with early treatment. Results are robust — in exclusion sensitivity analysis 20 of 22 studies must be excluded to avoid finding statistically significant efficacy in pooled analysis.

The immune effects of probiotics are strain-specific and studies use different strains.

[Ahanchian] Small RCT 60 healthcare workers in Iran, showing lower cases with treatment but without statistical significance. Once daily oral synbiotic capsule (Lactocare®) containing 1 billion CFU L. (Lactobacillus) casei, L. rhamnosus, Streptococcus thermophilus, Bifidobacterium breve, L. acidophilus, Bifidobacterium infantis, L. bulgaricus, and Fructooligosacharide. IRCT-20101020004976N6.

Submit Corrections or Updates.

[Catinean] Retrospective 60 patients in Romania taking probiotics and 60 matched controls, showing faster symptom resolution with the use of probiotics. Spore-based probiotic containing five strains of Bacillus.

Submit Corrections or Updates.

[Ceccarelli] Prospective analysis of 69 severe COVID-19 patients requiring non-invasive oxygen therapy, 40 treated with probiotic formulation SLAB51, showing lower oxygen requirements and higher blood levels of pO2, O2Hb and SaO2 with treatment. Authors suggest that enzymes in SLAB51 could reduce oxygen requirements in intestinal cells, resulting in more oxygen available for other organs.

Submit Corrections or Updates.

[Ceccarelli (B)] Retrospective 200 severe condition hospitalized patients in Italy, 88 treated with probiotic Sivomixx, showing lower mortality with treatment.

Submit Corrections or Updates.

[d'Ettorre] Retrospective 70 hospitalized patients in Italy, 28 treated with probiotic Sivomixx, showing lower risk of respiratory failure and faster recovery with treatment.

Submit Corrections or Updates.

[Di Pierro (B)] Interim report on an RCT for prophylactic treatment with S. salivarius K12, showing significantly lower cases with treatment. Only patients with symptoms or known positive contacts were tested. Trial identification/registration details are not provided.

Submit Corrections or Updates.

[Di Pierro] RCT 50 hospitalized patients in Pakistan, 25 treated with S. salivarius K12, showing lower mortality with treatment, without statistical significance. There were more patients with higher oxygen requirements at baseline in the control group - 18 vs. 6 with O2 ≥ 8 L/min.

Submit Corrections or Updates.

[Gutiérrez-Castrellón] RCT 293 outpatients in Mexico, 147 treated with a probiotic composed of three L. plantarum strains (KABP022, KABP023 and KABP033) and one P. acidilacti strain (KABP021), showing improved recovery with treatment. There were no hospitalizations or deaths. NCT04517422.

Submit Corrections or Updates.

[Haran] RCT 350 COVID+ outpatients in the USA, 174 treated with prebiotic KB109 (a microbiome metabolic therapy candidate), showing lower combined hospitalization, ER, and urgent care visits with treatment. NCT04414124.

Submit Corrections or Updates.

[Holt] Prospective survey-based study with 15,227 people in the UK, showing lower risk of COVID-19 cases with vitamin A, vitamin D, zinc, selenium, probiotics, and inhaled corticosteroids; and higher risk with metformin and vitamin C. Statistical significance was not reached for any of these. Except for vitamin D, the results for treatments we follow were only adjusted for age, sex, duration of participation, and test frequency. NCT04330599. COVIDENCE UK.

Submit Corrections or Updates.

[Ivashkin] RCT 200 patients, 99 treated with a probiotic (Lacticaseibacillus rhamnosus PDV 1705, Bifidobacterium bifidum PDV 0903, Bifidobacterium longum subsp. infantis PDV 1911, and Bifidobacterium longum subsp. longum PDV 2301). There was no significant difference in mortality or recovery time, however benefits were seen for diarrhea. NCT04854941.

Submit Corrections or Updates.

[Li] Retrospective 311 severe condition hospitalized patients in China, 123 treated with probiotics, showing slower viral clearance and recovery with treatment. Authors note that probiotics were able to moderate immunity and decrease the incidence of secondary infections.

Submit Corrections or Updates.

[Louca] Survey analysis of dietary supplements showing probiotic usage associated with lower incidence of COVID-19. These results are for PCR+ cases only, they do not reflect potential benefits for reducing the severity of cases. A number of biases could affect the results, for example users of the app may not be representative of the general population, and people experiencing symptoms may be more likely to install and use the app.

Submit Corrections or Updates.

[Navarro-López] RCT with 24 probiotics and 15 control patients in Spain, showing lower overall symptoms and lower digestive symptoms with treatment. Kluyveromyces marxianus B0399 plus lactobacillus rhamnosus CECT 30579.

Submit Corrections or Updates.

[Rodriguez-Blanque] Prophylaxis RCT with 127 probiotics and 128 control healthcare workers in Spain, showing no significant difference in cases. There were only 4 cases. Severity information by arm is not provided. L. coryniformis K8 CECT 5711.

Treatment may help sustain the immune response to vaccination - in the subgroup of subjects for whom more than 81 days had passed since they received the first dose, IgG levels were significantly higher in the treatment group. Patients that started probiotic consumption before the first vaccine dose also reported significantly fewer side effects.

Submit Corrections or Updates.

[Saviano] RCT 80 COVID-19 interstitial pneumonia patients in Italy, 40 treated with probiotics, showing significantly reduced gut inflammatory markers with treatment, and lower ICU admission and mortality, without statistical significance. Bifidobacterium lactis LA 304, lactobacillus salivarius LA 302, and lactobacillus acidophilus LA 201 bid for 10 days.

Submit Corrections or Updates.