RCT vs. observational studies for COVID-19

For COVID-19, across the 212 treatments we analyze, there is no difference in results between RCTs and observational studies, RR 0.98 [0.91‑1.04]. Restricting to, or waiting for, RCTs provided no benefit. Observational studies for COVID-19 do not systematically overestimate or underestimate efficacy when compared with RCTs, and they provided confirmation of efficacy 8+ months faster1.
Two additional large analyses also find no significant advantage for RCTs over observational studies2,3. Concato et al. also report a paradoxical finding—RCT results had higher variability, and only RCTs were found to sometimes report significant results the opposite of the overall result. The same trend is seen for the most popular (most politicized) COVID-19 treatments—considering all statistically significant results reported in studies, RCTs are slightly more likely to report a result in the opposite direction. In other words, assuming causality from a single study would have a higher error rate for RCTs.
Neither observational studies nor RCTs prove causation—any study can be flawed or fraudulent. We need much more, for example a combination of results from many independent teams, detailed understanding of each study, knowledge of conflicts/team reliability, dose-response relationships, delay-response relationships, logical results across outcomes, details consistent with preclinical expectations, etc.
RCTs aim to equalize study groups, but add their own biases. For acute diseases with strong benefits for earlier treatment, the typical increased treatment delay adds a major confounding factor. RCTs may also be more likely to have funding or staff with conflicts of interest. Both RCTs and observational studies span the bias spectrum, from minimal to extreme. Studies must be evaluated individually. While obsevational studies require greater expertise to analyze, the advantages in practicality, cost, time, and ethics are substantial.
0 0.5 1 1.5 2 Low-cost 0.98 [0.90-1.07] RR CI High-profit 0.93 [0.85-1.03] All treatments 0.98 [0.91-1.04] COVID-19 RCT vs. observationalresults from 6,000+ studies c19early.org January 2026 RCTs showhigher efficacy RCTs showlower efficacy
Treatment Total Test for overall effect: z = -0.77 ( P = .44 ) Test for subgroup differences: χ 1 2 = 0.63 ( P = .43 ) Low-cost treatments High-cost treatments Total Total Test for overall effect: z = -0.38 ( P = .70 ) Test for overall effect: z = -1.40 ( P = .16 ) Acetaminophen Alkalinization Andrographolide Antiandrogen Antihistamine H1RA Artemisinin Aspirin Azvudine Budesonide Camostat Cannabidiol Chlorhexidine Chlorpheniramine Colchicine Curcumin Dexamethasone Famotidine Favipiravir Fluvoxamine Hydrogen Peroxide Hydroxychloroquine Ibuprofen Indomethacin Ivermectin Lactoferrin Lopinavir/ritonavir Mebendazole Melatonin Metformin Montelukast N-acetylcysteine NaCl Nafamostat Nigella Sativa Nitazoxanide Nitric Oxide Phthalocyanine Povidone-Iodine Probiotics Quercetin Resveratrol Selenium Sodium Bicarbonate Spironolactone Sunlight Thermotherapy Vitamin A Vitamin B12 Vitamin B9 Vitamin C Vitamin D Vitamin K Zinc Amubarvimab/romlusevimab Bamlanivimab/etesevimab Bebtelovimab Casirivimab/imdevimab Convalescent Plasma Deuremidevir Ensitrelvir Molnupiravir Paxlovid Regdanvimab Remdesivir SA58 Sotrovimab Tixagevimab/cilgavimab Tocilizumab RR (95% CI) 0.98 [0.91; 1.04] 0.98 [0.90; 1.07] 0.93 [0.85; 1.03] 1.19 [0.84; 1.68] 1.33 [0.72; 2.45] 0.58 [0.28; 1.21] 0.52 [0.30; 0.89] 0.84 [0.32; 2.19] 0.77 [0.26; 2.28] 1.03 [0.90; 1.18] 0.85 [0.50; 1.46] 0.92 [0.46; 1.84] 1.72 [0.56; 5.30] 0.59 [0.04; 9.06] 0.55 [0.27; 1.12] 0.82 [0.40; 1.68] 1.26 [0.88; 1.80] 0.88 [0.56; 1.38] 0.77 [0.44; 1.35] 0.86 [0.60; 1.24] 0.93 [0.71; 1.22] 1.16 [0.57; 2.36] 0.69 [0.11; 4.28] 1.13 [0.92; 1.37] 0.47 [0.22; 1.03] 6.86 [0.62; 76.31] 1.39 [0.84; 2.31] 2.32 [1.05; 5.13] 0.92 [0.63; 1.33] 0.31 [0.02; 4.88] 1.48 [0.87; 2.53] 0.84 [0.50; 1.40] 1.10 [0.33; 3.60] 0.93 [0.62; 1.39] 0.98 [0.54; 1.78] 0.69 [0.14; 3.47] 0.98 [0.44; 2.21] 5.84 [2.07; 16.52] 0.72 [0.32; 1.59] 1.35 [0.41; 4.30] 0.93 [0.56; 1.54] 0.92 [0.66; 1.29] 10.71 [0.46; 250.39] 0.99 [0.10; 11.72] 1.01 [0.07; 14.61] 2.37 [0.90; 6.22] 0.80 [0.31; 2.04] 1.15 [0.62; 2.14] 0.93 [0.19; 4.55] 0.76 [0.32; 1.82] 0.35 [0.03; 3.83] 0.10 [0.03; 0.39] 1.02 [0.81; 1.28] 1.02 [0.79; 1.32] 3.49 [0.13; 93.95] 0.83 [0.54; 1.28] 0.42 [0.01; 15.37] 1.30 [0.43; 3.97] 2.69 [0.31; 23.17] 0.79 [0.33; 1.90] 0.85 [0.72; 1.02] 0.37 [0.01; 10.38] 0.68 [0.35; 1.32] 0.90 [0.65; 1.26] 1.08 [0.86; 1.36] 2.09 [0.19; 22.80] 0.90 [0.72; 1.13] 0.47 [0.06; 3.35] 1.18 [0.58; 1.97] 1.10 [0.56; 2.18] 0.98 [0.74; 1.30] 0.1 0.2 0.5 1 2 5 10 Ratio of RCT RR to observational RR RCTs show higher efficacy RCTs show lower efficacy
References