Extreme COI, impossible data, blinding broken, randomization/blinding failure, uncorrected errors, protocol violations, no response from authors, refusal to release data
Blinding broken.
Leaked documents show that blinding was broken, with interim results available not only within the team, but shared externally with a group of 90+ people, many from other ivermectin trials, in meetings organized by Dr. Andrew Hill. Unreleased leaked emails also indicate sharing with higher-up individuals at the NIH. As below, protocol changes were made mid-trial to add additional bias against ivermectin
doyourownresearch.substack.com (B).
Blinding failure.
Ivermectin/placebo blinding was done by assigning a letter to each group that was only known to the pharmacist. If a patient received a 3-dose treatment, investigators immediately know that the patient is more likely to be in the treatment group than the control group, because 3-dose placebo was relatively rare (~46% from PP). If a patient received non-3-day treatment, investigators immediately know that the patient is not an ivermectin treatment patient. Moreover, by observing the frequency of allocations, investigators can easily determine which letter corresponds to active ivermectin 3-day treatment, thereby removing all blinding. For example, consider 3-dose-ivermectin and 3-dose-placebo being identified by the letters G and K. If allocations to date have been G:11 and K:20, there is a very high probability that K is ivermectin. Note that this blinding failure is only obvious because the journal required the authors to restrict to the 3-day placebo group. Also note that it would have been trivial to avoid if desired, for example by using a unique identifier for all medication bottles. Note that there may be additional reasons for blinding failure, for example the paper specifies identically shaped bottles, but does not appear to specify identical appearance tablets , and patients receiving a placebo injection.
Patient counts for reported period impossible.
Authors claim the ivermectin and control patients were all from on or after March 23, 2021, however independent analyses of the enrollment graph (contained in this presentation
dcricollab.dcri.duke.edu) require including patients prior to this date to reach the reported numbers
doyourownresearch.substack.com (D), longhaulwiki.com. The enrollment graph shows much higher enrollment to ivermectin near the start of the trial. The only way that the number of placebo patients can be the same as the number of treatment patients is if placebo patients were taken from an earlier period
doyourownresearch.substack.com (E), Marinos (B), which creates a nonconcurrent control group
nejm.org and substantial
confounding by time as below.
Placebo counts vs. fluvoxamine not possible.
The IVM placebo arm has 679 patients and the FLV arm has 756. The 679 should be shared between the arms, with 77 extra patients for FLV. For FLV, there were 34 placebo patients requiring mechanical ventilation, for IVM there was only 25, indicating that 9 of the extra 77 placebo patients for FLV had mechanical ventilation, a much higher percentage during a period that had lower deaths and CFR (and included vaccinated patients). Placebo all-cause hospitalization shows 95/679 for IVM and 99/756 for FLV, i.e., only 4 of the extra 77 patients were hospitalized, but the paper reports an additional 9 patients with mechanical ventilation.
Impossible baseline data.
Analysis of baseline data by Marinos shows conflicting data and points to data being changed to hide randomization failure and incomparable groups. For details see
twitter.com (E).
Randomization violation, significant confounding by time.
Unequal randomization, significant
confounding by time. The trial reports 1:1:1:1 randomization, however independent analysis shows much higher enrollment in the ivermectin treatment arm towards the start of the trial
c19ivm.org, longhaulwiki.com. This introduces very significant
confounding by time due to the major change in the distribution of variants.
Zavascki show dramatically higher mortality for Gamma vs non-Gamma variants (28 day mortality from symptom onset aHR 4.73 [1.15-19.41]). Many more patients were randomized to ivermectin vs. placebo in the first few weeks, for example the first week shows 82 ivermectin vs. 28 placebo patients, 2.9x higher. The period of excess ivermectin enrollment coincides closely with a period of significantly higher deaths and CFR in Brazil.
![]()
Data pledge violation, unavailable over 900 days from protocol, over 700 days from publication.
The trial registration states that data was to be available at termination and upon request
clinicaltrials.gov, however authors have not responded to a request for the data. Even funders of the trial have been unable to access the data
odysee.com (B). Requests can be sent to thetogethertrial@gmail.com, let us know the outcome. For a detailed timeline of requests and the denial to release data see
Scheim (appendix A).
DSMC not independent.
Reviewer 1 of the protocol notes that the DSMC is not independent
gatesopenresearch.org. Prof. Thorlund is Vice President of the contract research organisation (CRO, Cytel), professor at the sponsoring university, and an author of the protocol. Dr. Häggström is an employee of the CRO.
doyourownresearch.substack.com (D), twitter.com (F) reveals many other conflicts. Prof. Thorlund has written >100 papers with Prof. Mills. Prof. Singh has written 29 papers with Prof. Mills. Prof. Orbinski has written 9 papers with Prof. Mills. The first version of the web site showed Prof. Mills and Prof. Thorlund as joint leads. Emails pointed to a company MTEK Sciences, founded by Prof. Mills and Prof. Thorlund (MTEK is hypothesized to stand for Mills, Thorlund, Edward, Kristian). MTEK received grants from the Gates Foundation. MTEK also employed Dr. Häggström. MTEK was acquired by Cytel in 2019. Dr. Häggström works for the Gates Foundation. Two members of the DSMC have published a paper with members of a well known anti-ivermectin research group
Thorlund and Dr. Hill, whose meta analysis has reports of external influence
c19ivm.org (B), c19ivm.org (C), . The trial protocol reports that
"an independent DSMC will be established, composed of scientists of unrivalled reputation and expertise, without involvement with this research protocol."
Extreme conflicts of interest.
Disclosed conflicts of interest include: Pfizer, Merck, Bill & Melinda Gates Foundation, Australian Government, Medicines Development for Global Health, Novaquest, Regeneron, Astrazeneca, Daichi Sankyo, Commonwealth Science and Research Organization, and Card Research. Many conflicts of interest appear unreported. For example, Unitaid is a sponsor
Harper, togethertrial.com (B).
Three conflicting death counts.
In the original paper, Table 3 shows 21 and 24 deaths, while Table S6 shows 20 and 25
. In Table 3, death and grade 5 events showed the same 21/24 numbers, but different effect sizes, with 0.81 being closer to the 20/25 counts and the previously reported number. This is consistent with one death being moved between arms after manuscript generation, but not updated in Table S6 or the Table 3 AE RR. This cannot be explained by the safety population excluding patients with zero doses because the AE control deaths are higher. In email, a co-principal investigator suggested that the discrepancy was due to one being COVID-19 deaths and the other being all-cause deaths
stevekirsch.substack.com. That explanation does not fit the data because one arm increases while the other arm decreases. Both co-principal investigators report in the paper that
"they had full access to all the trial data and vouch for the accuracy and completeness of the data and for the fidelity of the trial to the protocol." A third set of death counts, 20 and 24, with RR 0.84, was presented by a co-principal investigator on Mar 18, 2022
vimeo.com (B). In total, 4 different death relative risks have been presented: Mar 18, 2022 presentation: 0.84, Mar 30 paper: 0.88 (T3), 0.81 (T3 AE), and 0.80 (TS6, presented as 20 and 25 only without group sizes). 6 days after publication, the paper was updated, with no information given on what was changed. In this version, a "respiratory, thoracic and mediastinal disorders" death was removed from the control arm and an "infections and infestations" death was added to the ivermectin arm. The paper still indicates RR 0.81 for death AE.
Placebo adverse event conflicts with fluvoxamine arm.
For FLV, there were 11 grade 1 AEs, for IVM there were 12, with 77 less patients. For FLV, there were 50 grade 3 AEs, for IVM there were also 50, meaning the 77 extra patients had 0% grade 3 AEs vs. an expected 7.4%. For FLV, there were 54 CKD patients according to Figure 3 and eTable1 (2 according to Table 1). For IVM there was 5.
Conflicting adverse event counts.
Table 3 and Table S6 adverse event counts do not match for any grade, e.g., grade 1/2 in Table S6 shows 82 for IVM, while Table 3 shows 65 . The Apr 5 update changed the grade 5 events without explanation, however the other grades remain conflicting.
3-day dosing patients before March 23 missing.
The co-principal investigator wrote on March 6 that 3-day dosing was being administered, and that the clinicaltrials entry was out of date at that time
twitter.com (J). This earlier start of the 3-dose arm would resolve an oustanding major inconsistency. Analysis of the trial randomization shows that reaching the 3-day placebo count requires patients from March 4
doyourownresearch.substack.com (F), and it would be logical for the 3-day placebo and 3-day active arms to have started on the same day. This reinforces existing concerns as to which patients were included in the analysis, and adds additional questions regarding what happened to the patients prior to March 23, and if patients were treated prior to ethics approval. Ethics approval for the dose change was received on March 21 according to the paper
twitter.com (K), with the regulator document dated March 15
twitter.com (L).
Multiple false statements by investigators.
There has been multiple false statements by investigators raising questions about their ethics and the reliability of their work
pierrekory.substack.com.
Investigators not responding to concerns.
After details of major data errors and protocol violations became known, investigators appear to have stopped responding to all researchers regarding serious concerns with the trial
pierrekory.substack.com, twitter.com (M) (and have still not responded to us).
ICODA reports never having the data.
Investigators report that the data is available via ICODA:
"The final trial dataset will be accessible by written request to the study principal investigators (G Reis or EJ Mills). There are no contractual agreements to limit access to final trial data. All data collected by the TOGETHER Trial will be shared with the International COVID-19 Data Alliance". Not only has there been no reports of successful access to the data, but an ICODA manager reports that they have never had the data
t.me.
Indications of ivermectin use in the placebo arm.
Recent ivermectin use was not in the exclusion criteria, however community use was widespread. Ivermectin was available OTC, was recommended by the government for COVID-19, and had nine times higher sales
twitter.com (N). Authors claim they ensured patients did not use ivermectin via "extensive screening", but do not explain why this was not an exclusion criterion, or how this unwritten exclusion was ensured even though there is extensive missing data related to written exclusion criteria. Similar unwritten exclusions were not mentioned for other arms
, a primary investigator previously stated such an exclusion should not be an issue
twitter.com (P), and it is not mentioned in the interview sheets
osf.io. After publication, a co-principal investigator reportedly wrote that
"even if some patients did access IVM, the fact that it is blinded should still maintain balance", which is incorrect, placebo patients taking ivermectin are expected to improve, treatment patients that already have significant tissue distributions may have positive, neutral, or negative responses to additional treatment. Further, there are contradictory reports of how patients with prior ivermectin use were handled, and there is an indication that patients who had taken, or "were likely to take" ivermectin for COVID-19 were re-allocated to fluvoxamine or placebo
doyourownresearch.substack.com (G).
Conflicting and inconsistent PP/ITT groups.
Conflicting and inconsistent decreases in PP vs. ITT groups between different tables and between the treatment and control arms
Scheim.
Refusal to release per-protocol mortality and hospitalization data.
Refusal to release per-protocol mortality and hospitalization results although they were reported for fluvoxamine and their importance was emphasized by coauthor Dr. Boulware
Scheim.
Conflicting descriptions of the placebo group.
There are multiple conflicting statements regarding the placebo group
Scheim.
Failure of NEJM to publish letters reporting errors and issues.
NEJM has refused to publish any of the many letters submitted by scientists reporting errors and critical issues with the trial. For details see
Scheim.
Placebo tablets may not match treatment tablets.
Authors do not specify the appearance of the placebo tablets, suggesting that they may not match the treatment tablets, providing an additional reason for blinding failure. A Brazilian investigator reports that, at the time of the trial, there was only one likely placebo manufacturer, and they reportedly did not receive a request to produce identical placebo tablets . They also report that compounded ivermectin in Brazil is considered unreliable.
Viral load protocol violation, high Ct may hide efficacy.
The protocol has change in viral load as an outcome, however only viral clearance is reported, and without any details (for example, using a high Ct value would have limited relevance). Notably, the same trial did report viral load results for a commercial drug where one of the principal investigators is a founder of the company
dcricollab.dcri.duke.edu (B).
162 control patients missing onset data vs. 1 for peginterferon lambda.
162 control patients are missing time from onset (Figure 2), however for peginterferon lambda (which overlaps with ivermectin in time) authors claim only one control patient is missing time from onset
dcricollab.dcri.duke.edu (B) (page 25).
Contradictory inclusion/exclusion of vaccinated patients, changes, and confounding.
The trial changed from including vaccinated patients to excluding them on Mar 21, 2021
clinicaltrials.gov (B), twitter.com (Q), and on Jul 5 the exclusion was changed to specify >14 days. As discussed, meeting the reported placebo counts likely requires taking placebo patients from the earlier period, which has significant confounding due to variant changes. The vaccine inclusion change adds additional confounding, which also favors the placebo group. The original vaccine inclusion criterion is shown in both the protocol and the clinicaltrials.gov record
clinicaltrials.gov (C), togethertrial.com (C). Note that the paper, master protocol, Brazilian protocol, and trial registration report contradictory information on vaccine inclusion/exclusion and changes over time
.
Metformin/fluvoxamine conclusions opposite of COVID-OUT, but matching earlier studies on each team.
The Together trial and COVID-OUT both tested metformin and fluvoxamine. Notably, they came to opposite conclusions. In Together, authors found efficacy for fluvoxamine, but the metformin results were so negative that the trial was terminated early. In COVID-OUT it was the opposite, authors (although not the journal editor) found efficacy for metformin, while the fluvoxamine results were so negative that the trial was terminated early
twitter.com (S). Note that the Together authors include researchers that found fluvoxamine effective in earlier studies, while the COVID-OUT authors include researchers that found metformin effective in earlier studies.
Author claims results from 628 researchers should be censored for false information.
62 studies by 628 scientists report statistically significant positive results for ivermectin treatment of COVID-19
c19ivm.org (D). One author claimed that a report of positive results is "disinformation" and distributed a request to report and censor the author
, , twitter.com (V). While discussion is warranted for all studies, a call for censorship of results is extreme and raises questions. Author provides no basis for the results of the 628 scientists being wrong and warranting of censorship, and there is no indication that author has even read most of the studies. Author cherry-picked two of 102 studies, (COVID-OUT and ACTIV-6
Bramante, Naggie, both very high COI studies with an extensive list of issues and very delayed treatment) and claimed that
"no benefit of ivermectin was observed" . In addition to ignoring the 62 studies reporting statistically significant positive results, ACTIV-6
c19ivm.org (E) reported a posterior probability that ivermectin is effective of 99%, 98%, and 97% for mean time unwell, clinical progression @14 days, and clinical progression @7 days (even though none of the pre-specified primary outcomes were reported, and noting that these preprint results were changed without explanation), and COVID-OUT showed 61% lower hospitalization with ivermectin vs. placebo (not including metformin), although this was not reported.
Paper silently modified without notification or explanation.
Apr 5: The paper was silently updated, with no indication or explanation of the changes. Changes include: age range, placebo description, per-protocol count, and death counts (as above). May 5: The paper was silently updated again. A new summary notes that authors attempted to screen for previous ivermectin use, contradicting both the discussion section, where authors claim they ensured no use for COVID-19, and the exclusion criteria and interview forms, which do not specify ivermectin use.
Team selected dose below what they believe is required.
Dr. Craig Rayner, a senior investigator on the trial, previously published research indicating that a higher dose is required
sciencedirect.com, raising the question of why the dose and fasting administration was chosen, for both the single day and 3 day dose regimens. Clinical equipoise
en.wikipedia.org raises ethical questions about running a trial where the invesigators do not believe the chosen dose is effective.
Krolewiecki show an antiviral effect only with plasma concentrations above 160ng/mL. Figure S5 shows that the authors expected the mean concentration to be well below this level
. Dosage requirements are likely to vary significantly depending on many factors including the variant encountered, time of administration, mode of administration, patient genetics, concomitant medications, SOC, and the distribution of the infection in different tissues. However, the dose used is far below what is recommended by clinicians for post-infection treatment with the Gamma variant — about 2.5 - 6.5x lower, depending on the recommendation and which estimate of fasting/fed administration is used. The trial used fasting administration, however Merck's product information reports that
"administration of 30mg ivermectin following a highfat meal resulted in an approximate 2.5-fold increase in bioavailability relative to administration of 30mg ivermectin in the fasted state." merck.com.
Viral clearance results mismatch.
In Figure S3, the numbers in the table do not match the graph, for example day 0 to day 3 for ivermectin shows a 30% decline in viral detection in the table, while the graph shows only a 4% decline
twitter.com (Y). Table 3 results do not match either of these results. For example at day 3, table 3 reports 7.4% clearance for ivermectin. In Table 3, about 15% more patients have viral clearance results in the control arm, suggesting likely
confounding by time. Authors provide no details on which patients were tested in each arm, however the confounding may be very significant. Notably, a July 2021 change adds viral load testing for the IV arms
twitter.com (Z), suggesting that placebo viral results may have a significant number of patients from a very different time period
twitter.com (Z) where patient characteristics
twitter.com (AA), viral variants, and outcomes may be very different.
FDA concerns regarding trial conduct for peginterferon lambda.
The peginterferon lambda arm of this trial generated positive results, however the FDA denied a request for a pre-EUA meeting, citing concerns about the conduct of the trial
ir.eigerbio.com. Reportedly, the
"FDA will not consider an EUA application based on results generated from the TOGETHER study" ir.eigerbio.com. Details of the FDA's concerns were not provided.
Co-principal investigator agrees with political need to demonstrate poor efficacy.
Frank Harrell commented that "..the question of whether this study was stopped too early in light of the political ramifications of needing to demonstrate that the efficacy is really unimpressive.. really could be raised..", and co-principal investigator Prof. Ed Mills responded
"I totally agree with Frank" vimeo.com.
Private comments on efficacy contradict public comments.
Co-principal investigator Prof. Ed Mills said the following in private communication:
"There is a clear signal that IVM works in COVID patients.. that would be significant if more patients were added.. you will hear me retract previous statements where I had been previously negative" stevekirsch.substack.com, while public comments are contradictory, e.g.
“There was no indication that ivermectin is clinically useful”.
Mid-trial change adds patients showing lower efficacy.
Authors modified the inclusion criteria mid-trial to allow adding more low-risk patients <50
twitter.com (AB). Comparison with the metformin placebo arm shows many more patients <50 were included in the later part of the trial. Notably, the primary outcome shows much higher efficacy for patients ≥50, which would be statistically significant if there was about 50% more patients ≥50.
Funding from Gates Foundation.
Analysis suggests funding of the ivermectin arm in part by the Gates Foundation by way of Certara and Prof. Rayner
doyourownresearch.substack.com (J), in addition to previously disclosed funding of the overall trial. Authors list only FastGrants and Rainwater in the ivermectin paper. The Gates Foundation was cited as a funder on the web site until Sep 13, 2021
web.archive.org (after completion of the trial). Correspondence regarding fluvoxamine and fluoxetine notes that Certara support for the TOGETHER study was funded by the Gates Foundation
thelancet.com.
Conflicting reports on trial end date.
There are conflicting reports on when enrollment into the ivermectin and fluvoxamine arms ended
doyourownresearch.substack.com (K), with the master protocol showing the end as July 26 and the web site confirming an end in July. However, the papers report August 5 and 6.
Expression of concern.
An expression of concern was posted for the metformin arm in 2024
thelancet.com (B).
Side-effect prevalence consistent with treatment error.
The side effects (e.g., gastrointestinal side effects were lower in the ivermectin arm) suggest that many ivermectin patients may not have received authentic ivermectin, or that placebo patients may have taken ivermectin. For comparison, there was a 3.6 times greater incidence of diarrhea in the treatment arm in
Lim.
Screening to treatment delay unknown.
Most Together Trial master protocols show an additional day delay in already late treatment for most patients. The Aug 5, 2021 protocol published with the metformin paper
ars.els-cdn.com, shows treatment administration one day after screening, baseline, and randomization (Table 2, schedule of study activities). This can also be found in the protocol dated Mar 11, 2021
drive.google.com. The protocol attached to the ivermectin paper, dated Feb 15, 2021, shows a different schedule, stating that the treatment
should be administered on the same day of randomization. There is no explanation of when this change was made, how the overlapping metformin and ivermectin arms could use different schedules, or how this change was implemented (there are many tasks in the screening and baseline visits). There is no reporting for how many patients received treatment on the same day. The form for the first treatment visit asks if there were clinical events including >6hr ER visits since the baseline visit, which would not be possible if this visit was immediately after randomization. Time of first treatment was recorded
osf.io, but no information has been reported. According to
Forrest, WhatsApp messaging and video was used for recruitment, raising the question of how medication was delivered in cases where recruitment was done online. The Brazilian versions of the protocol do not match the master protocol, with all showing administration on the same day. The first Brazilian version of the protocol stated the drug
"must be administered on the same day", however in March 2021 and later versions, this was changed to
"should be administered on the same day".
![]()
317 unknown onset patients, onset required for inclusion.
Figure 2 shows that the time from onset was unknown for 317 patients, however time from onset is required for the inclusion criteria.
Unknown onset patients treated late.
After imputation, the percentage of patients in the late treatment subgroup went from 46% to 56%. 87% of the unknown patients were predicted to be in the late group. This is reasonable and expected — patients that do not recall when the onset was are more likely to have had onset further in the past. What is not clear is how these patients could be enrolled in the trial, how many of these patients had onset >7 days, how this very late 317 patient subgroup could show much greater efficacy as above, and why authors did not report this result, analyze this in greater detail, or recommend further research.
Conflicting comorbidity counts.
The companion fluvoxamine arm ran from Jan 20 to Aug 5, 2021, while this trial ran from March 23 to Aug 6, 2021 — most control patients should be shared, with an additional 10% for fluvoxamine from the earlier start. The Aug 6 presentation, which has a date of 9:38am Aug 6 local trial time , shows 678 placebo patients, indicating that either 0 or 1 placebo patients were randomized on Aug 6. Zero patients should have been randomized on Aug 6, because authors cannot add patients after unblinding. The fluvoxamine control arm shows 16/756 control patients with asthma. The ivermectin control arm has a subset of these patients (679), but shows a much higher prevalence of asthma (60 patients). This might be possible due to imputation if there was a very high percentage of missing data, however imputation does not appear to be a good explanation. For example, placebo CKD goes from 2 to 5 (FLV->IVM). First, it is not logical to impute CKD on patients based on the other variables. Second, the protocol specifies imputation only with up to 20% missing data, making it unlikely that imputation would add 150% of CKD patients. Third, the degree of change between FLV and IVM varies dramatically, with IVM reporting 666%, 275%, 150%, and 43% more patients for CPD, asthma, CKD, and CCD, without any clear explanation for similar differences in the percentage of missing data (all were collected on the same interview form).
Unexplained >6 month delay.
The paper was delayed over 6 months with no explanation. The companion fluvoxamine arm, completed at the same time, was published Aug 23, 2021. The very long delay, high profile of the topic, and other issues above
raise questions. The paper was submitted to NEJM in Sep 2021
vimeo.com (C). COI forms suggest that additional authors were added after submission and the corresponding author changed from Prof. Mills to Dr. Rayner
, whose conflicts include Pfizer, Merck, the Gates Foundation, and the Australian Goverment.
Placebo patients receiving an injection not blinded.
One-day placebo patients, which study documents repeatedly report as being included in the placebo group, would include those receiving a single injection, and are therefore not blinded
Scheim.
Major imputation error.
In the paper authors use imputation in Table 1 but not in Figure 2. Authors also released a version of Figure 2 with imputation
togethertrial.com (D), where the numbers for age and BMI now match the imputed numbers in Table 1. However, the time from onset numbers are very different, with the treatment arm showing 302 patients for 0-3 days, and the imputed version of Figure 2 showing 367
.
Single dose results missing.
Results for the single dose ivermectin arm have not been reported.
Incorrect conclusion.
The conclusion states that ivermectin "did not result in a lower incidence of [hospitalization] or of [ER observation >6hr]". This is incorrect, hospitalization was 17% lower, which is not statistically significant with the sample size and typical statistical analysis. For the Bayesian analysis the authors use, the ITT probability of superiority for ivermectin was 79.4%, which is a positive result, the opposite of the conclusion.
Missing age information.
According to Figure 2, 98 patients are missing age information .
Mid-trial protocol changes.
There were several mid-trial protocol changes on July 5, 2021
clinicaltrials.gov (D). The number of patients for viral load analysis was reduced, only for the ivermectin arm. All-cause, cardiovascular, and respiratory death outcomes were deleted (all-cause was reported). Exclusions were modified to allow enrolling patients vaccinated within the last 14 days. Inclusion criteria were modified to allow enrolling healthy young people — the criterion
"fever >38C at baseline" was added, allowing enrollment independent of increased risk.
COI: designed by Cytel.
The trial was designed by Cytel, a company that helps pharmaceutical companies get approval and that works very closely with Pfizer
cytel.com, cytel.com (B). Cytel's software and services are used by the top 30 pharmaceutical companies
cytel.com (C).
COI: co-principal investigator works for Cytel and the Gates Foundation.
A co-principal investigator works for Cytel and the Gates Foundation
empendium.com:
"The majority of the time I work for a company called Cytel, where I design clinical trials, predominantly for the Bill & Melinda Gates Foundation".
COI: pharma funding and denial of independent investigator.
Reportedly, the first author's center is funded by pharmaceutical companies, and independent investigators tried to participate in the trial but were denied
odysee.com (C).
COI: Gates Foundation.
The Gates Foundation is a founding partner of GAVI, which took out Google ads telling people not to use ivermectin
, and a major funder of Unitaid, which may have modified the results of the Hill meta analysis in a way that prevented adoption
c19ivm.org (B), c19ivm.org (C), .
COI: Certara.
One of the senior investigators was Dr. Craig Rayner, President of Integrated Drug Development at Certara - another company with a similar mission to MMS Holdings. They state on their website that:
"Since 2014, our customers have received over 90% of new drug and biologic approvals by the FDA." One of their clients is Pfizer
certara.com.
COI: MMS Holdings.
The trial is associated with MMS Holdings
dcricollab.dcri.duke.edu, whose mission includes helping pharmaceutical companies get approval and designing scientific studies that help them get approval. One of their clients is Pfizer
mmsholdings.com.
COI: analysis company works closely with Pfizer.
All analyses were done by Cytel. Cytel is a statistical modelling company that helps pharmaceutical companies get approval — they work very closely with Pfizer
cytel.com (B). Cytel's software and services are used by the top 30 pharmaceutical companies
cytel.com (C).
COI: Fast Grants funding closely related to companies that censored scientific research.
Fast Grants funding is closely related to tech companies that censored ivermectin and other early treatment research. These companies have a strong conflict of interest in not admitting that their censorship was harmful
drtesslawrie.substack.com, fastgrants.org.
Unexpected differences in missing data.
Age is unknown for 98 patients, however according to Figure 2, BMI is missing for only 11 patients, smoking status is unknown for only 2 patients, lung disease is unknown for only one patient, and cardiovascular disease is known for all patients.
Out of funding claim contradicted by funder.
A co-principal investigator has reported that the trial was stopped because they ran out of funding, however this is contradicted by the Rainwater Foundation, which reported that they would have given more money to finish the trial if the investigators had asked
pierrekory.substack.com (B).
Misrepresentation of dosing recommendation.
Investigators have misrepresented an email from the FLCCC regarding recommended dosing
pierrekory.substack.com (B).
Unknown onset patients show statistically significanct efficacy.
For the known time since onset subgroups, both groups show worse results than the overall results , with the missing 317 patients showing significant efficacy RR 0.51, p = 0.02 (compared to 1.00 and 1.14 for known patients).
Mean delay likely excluding unknown onset.
The reported mean number of days from symptoms to randomization probably only includes known onset patients and therefore is likely to significantly underestimate the actual average (in addition to not including the time between randomization and treatment).
Per-protocol placebo much more effective.
The 3-dose per-protocol placebo group shows greater efficacy
Marinos (B), twitter.com (AF). This could be consistent with placebo patients accidently receiving treatment.
Multiple conflicting randomization protocols.
twitter.com (AG) reviewed the randomization protocol, finding three different algorithms, and conflicting versions in the papers.
Dominated by Gamma variant with different characteristics, no discussion.
The trial took place in an area of Brazil and time when the Gamma variant was dominant. Brazilian clinicians report that this variant is much more virulent, and that significantly higher dosage and/or earlier treatment is required, as may be expected for variants where the peak viral load is significantly higher and/or reached earlier
Faria, Nonaka.
Incorrect dose reporting, patients at higher risk due to BMI may have received lower per kg doses, and show lower efficacy.
The paper reports 400μg/kg for 3 days, however the protocol indicates that this was only up to 90kg, meaning that the dose received for higher-risk high BMI patients was even further reduced from dosage which is already far below clinician recommendations for the dominant variant
twitter.com (AH). 50% of patients had BMI ≥30. Much greater efficacy was seen in the low BMI subgroup (RR 0.77 vs 0.98).
Conflicting target enrollment and reasons for termination.
There are conflicting target enrollment numbers. The protocol showed 800 patients per arm as of Mar 21, 2021 (after the trial started)
static1.squarespace.com, twitter.com (AI), the co-principal investigator reported 800 per arm in an interview published June 14, 2021
halifaxexaminer.ca, and the protocol changed to 681 on June 22
static1.squarespace.com (B). However, the trial record from Jan indicates 2724 (681*4) patients
clinicaltrials.gov (C), suggesting that the 800 goal was later, and was kept for fluvoxamine but reverted for ivermectin. The fluvoxamine arm which started two months earlier was terminated at the same time, and was terminated due to superiority
Reis after 741/756 patients. Note that Gamma was declining significantly around the termination point, which likely favors improved efficacy if the trial continued, given the late treatment and dosage used. The co-principal investigator reports three different reasons for stopping the trial
twitter.com (AJ): a) because they ran out of money, b) because third parties were not supportive, and c) it was done by the DSMC and was out of their control.
Primary outcome subject to bias, selected after single dose arm.
The subjective "emergency room visit for >6 hours" criterion shows higher risk (RR 1.16), while hospitalization is lower (RR 0.83 all-cause, RR 0.84 COVID-19). The primary outcome results were set on March 21, 2021, after the single dose ivermectin arm. Given the known public biases of some investigators, this may have been specifically chosen to reduce efficacy. Authors claim that the 6hr threshold did not include waiting time, however the emergency visit form has no mention of waiting time, only recording presentation and discharge times
osf.io.
Reported terminated due to futility, but threshold not reached.
The trial was reportedly terminated due to futility
twitter.com (AK), however the futility thresholds were 20%, 40% and 60%, and all published probabilities are >60% (ITT 79.4%). Additionally, the fluvoxamine arm did not have the higher 60% threshold, only using 40%. Note the DSMC was not independent as below.
Subgroup analysis protocol violations.
The presented subgroup analysis is inconsistent with plans and with the fluvoxamine paper, including not presenting pre-specified subgroups, presenting subgroups that were not pre-specified, presenting different subgroups to the contemporary fluvoxamine paper, and modifying subgroup definitions
twitter.com (AL).
Many pre-specified outcomes missing.
Many outcomes specified in the protocol appear to be missing, including the co-primary outcome of COVID-19 mortality (only all-cause mortality is provided, specific AE details not provided), time to clinical failure, days with respiratory symptoms, mortality due to pulmonary complications, cardiovascular mortality, COVID-19 symptom scale assessment, WHO clinical worsening scale assessment, and 14 day mortality.
Single-dose recruiting continued after change.
The trial had requested moving to 3-dose treatment by Feb 15/19, when only 19 patients had been recruited, however the trial continued recruiting an additional 59 patients to single dose treatment
twitter.com (AM).
Funding list incorrect, missing Gates Foundation and Unitaid.
The paper does not include the Bill and Melinda Gates Foundation or Unitaid as funders, however the protocol shows the Gates Foundation
gatesopenresearch.org (B) and the web site shows Unitaid
togethertrial.com (B).
Statistical analysis plan dated after trial start.
The statistical analysis plan appears to be dated after the trial started
twitter.com (AN).
Imputation protocol violation.
The protocol specifies multiple imputation with up to 20% of missing data, however imputation was done with time from symptom onset, which has >23% missing data
twitter.com (AO).
Expected analyses missing.
Authors do not provide time from onset analysis for either mortality or hospitalization, only the combined measure including the ER visits where anomalous results are seen. Authors do not provide per-protocol or mITT results for mortality or hospitalization. Per-protocol mortality results were provided for the companion fluvoxamine trial.
Conflicting reasons for dose change.
Conflicting reasons have been given for the change from 1-day to 3-day dosing. In email from March 6, the co-principal investigator says the change was "based on emerging trials from Andrew Hill's synthesis"
twitter.com (AP). The paper says the change was made "on the basis of feedback from advocacy groups". Neither of these match the report that the dosing change was made at the request of one of the trial funders
pierrekory.substack.com.
Details of placebo unspecified.
The placebo appears to be unspecified in the paper and protocol. The initial trial announcement indicated the placebo was vitamin C
cytel.com, which would be an active treatment according to the
results of 70 studies (mortality RR 0.81 [0.73-0.91]). The metformin arm reports using talc, however fluvoxamine and ivermectin do not appear to report details of the placebo, which could potentially be different, for example based on manufacturer limitations for matching active treatment tablets.
Antigen test requirement.
The protocol indicates that patients with a negative test may be included if they become positive a few days later, potentially resulting in a long unreported delay between randomization and treatment, depending on how investigators interpreted the protocol. The requirement for a positive antigen test excludes the possibility of early treatment in many cases - tests have very high false negative rates in the early stages of infection, and symptoms may appear before the test becomes positive.
Bayesian probability of superiority, featured for FLV, hidden in appendix.
The bayesian probability of superiority figure, featured in the main paper for FLV, MET, HCQ, was hidden in the appendix for IVM
twitter.com (AQ).
Two different per-protocol counts.
Figure 1 shows 228 per-protocol for the control arm, while Table 2 shows 288. This was modified in the Apr 5 update without explanation.
Source of ivermectin unspecified (fluvoxamine source specified).
Authors do not specify the source of the ivermectin used in the trial, whereas they do specify the source for the fluvoxamine arm (Luvox, Abbott). Depending on the source, Ivermectin has been reported to be of unreliable quality in Brazil.
No confirmation of placebo manufacturing.
A local Brazilian investigator reports that, at the time of the trial, there was only one likely placebo manufacturer, and they reportedly did not receive a request to produce identical placebo tablets . They also report that compounded ivermectin in Brazil is considered unreliable. The protocol reports that "the study medication used will come from pharmaceutical plants that hold a commercial authorization for their production, already approved by ANVISA."
This trial uses a soft primary outcome, easily subject to bias
and event inflation in both arms (e.g., observe >6 hours independent of
indication). There is also an unusual inclusion criteria: "patients with
expected hospital stays of <= 5 days". This is similar to "patients less
likely to need treatment beyond SOC to recover", and would make it very easy
to reduce the effect seen. This is not in either of the published
protocols.
The local Brazilian investigator also reports that nitazoxanide
was tested in the same location, however very few patients reportedly
experienced urine discoloration, while all are expected to experience this
side effect. They also suggest that 6-hour observation is a poor choice
because it is almost impossible to stay less than 6 hours in Brazil.
This study is excluded in the after exclusion results of meta
analysis:
multiple anomalies as per detailed analysis.
