Teams build methods to forecast the results of forthcoming randomised trials from their pre-analysis plans. Each forecast is scored against the real result once the trial is resolved.
Decades of investment in randomised trials have generated a rich evidence base – thousands of completed RCTs on what works in development policy.
Any single trial, read on its own terms, captures only a fraction of the information the corpus contains. Taken together, the literature exhibits systematic structure – regularities in how treatment effects vary across intervention types, populations, and contexts – that only becomes legible at scale. To the extent this holds, each new trial contributes not merely an increment to the literature, but to an advancement in the field's collective predictive capacity. This pilot is built around one empirical question:
To what degree can current statistical, machine learning, and AI systems accurately forecast the direction and magnitude of treatment effects prior to trial completion?
The steps from entry to scoring.
$300,000 in prizes for the pilot.
There are two divisions teams can participate in.
Open to any method or toolchain, with no restrictions on the models or tools used. Submissions are judged solely on the accuracy and calibration of their forecasts.
Open to any model, but a winning submission must be reproducible. The competitor provides a self-contained package that regenerates the full pipeline. Forecasts are scored on the same scale as the open division.
A separate track from the prizes. Ten grants of $20,000 support teams in building and testing their approach while they compete. A grant does not affect prize eligibility.
The tournament asks an open empirical question: how well can current statistical, machine learning, and AI systems forecast the results of social-science experiments it has not seen, and what allows it to do so? There is little evidence on either question today.
Forecasts will be submitted before trials resolve and scored against the real result under a pre-registered rule, producing a first measurement of forecasting accuracy and of which methods work. Those results bear on two questions at once: what AI systems can do, and how far the experimental evidence base can be used predictively. The aim is to learn whether this is possible, and how.
The tournament is open to anyone who wants to forecast experimental results, whether entering alone or as a team of any size.
Entrants may come from any background, including but not limited to academic researchers, industry ML teams, independent forecasters, and evidence-synthesis teams. All are equally eligible, and no specific disciplinary background is required. Which kind of approach forecasts best is itself one of the open questions the pilot is designed to answer.
Entry is open globally, except where participation or receipt of prize funds is restricted by applicable law.
If a participant is an author of a forthcoming trial, or otherwise in a position to know its result before it is reported, they are requested to disclose this at registration. Such conflicts are assessed case by case.
For each trial in the prediction set, participants receive its pre-analysis plan together with a set of standardised forecasting questions. The questions fix what is being predicted and keep it consistent across participants.
The questions ask for the trial's main treatment effects – the average effect of the intervention on its primary outcomes, in standardised units. Participants forecast each one, with their uncertainty, before the trial resolves.
Each forecast is scored with a proper scoring rule – a Normal log-likelihood. Because participants submit a full distribution rather than a point estimate, the rule rewards two things at once: how close their central estimate is to the true effect, and how honestly they expressed their uncertainty.
The rule rewards accuracy and calibration together: a confident but incorrect forecast is penalised more heavily than an appropriately uncertain one, so overconfidence is costly and guessing is not rewarded.
This depends on the division.
In the open division, there are no restrictions on the models or tools used. Any approach is permitted, and submissions are judged solely on the accuracy and calibration of their forecasts.
In the reproducible division, any model may also be used, but a winning submission must be reproducible: the participant supplies a self-contained package that regenerates its forecasts end to end from the materials provided, without relying on data of unknown provenance. This package is not required to enter – only on winning, when it is reviewed. A submission that does not pass review is not eligible for the reproducible prize. The full criteria are set out in the participant terms.
No. The trials in the prediction set are unpublished at the time forecasts are made: their results have not been reported in any journal, working paper, or public dataset, and the submission window closes before the trials resolve. Any study that resolves earlier than expected, or whose results are found to have leaked, is removed from the prediction set.
All participants receive a curated training corpus of open-access development and social-science studies to learn from. The rules on what additional external data and sources may be used will be set out in the participant terms before submissions open.
Many entrants will use frontier AI models that are already pre-trained on large amounts of public text, and the tournament does not restrict or audit what those models were trained on. In the reproducible division, a winning submission must additionally be able to regenerate its features using only the materials provided and other publicly available sources, not data of unknown provenance – see the prize divisions below.
There are two prizes, each worth $150,000: an open prize and a reproducible prize.
Every submission is scored in a single pool under the same rule; the division affects which prize a submission is eligible for, not how it is scored. Open submissions face no methodological restrictions. Reproducible submissions additionally meet the reproducibility standard (see "Can participants use any AI model or tool?" above), which makes them eligible for the reproducible prize.
Because scoring is shared, a submission that meets the reproducible standard competes for both prizes. If it is ranked top overall, it wins both – $300,000.
Entrants whose method can meet the reproducibility standard should enter reproducible; meeting it adds a second prize without changing how forecasts are scored. Those who cannot meet it, or who would rather not constrain their approach, should enter open.
The research grants are 10 awards of $20,000 each, intended to help teams build and test their forecasting approach. A grant does not affect prize eligibility: grant recipients compete for the main prize on the same terms as everyone else, so a grant is support, not a substitute.
Applications can be submitted through a dedicated form [going live soon]. Grants are awarded on the strength of the proposed method and the team's capacity to carry it out. Across the funded teams, the aim is to support a range of approaches.
Participants retain ownership of their methods. The tournament does not take ownership of the approaches or code that participants develop.
Submitted forecasts are private and are not published.
As a condition of claiming a prize, a winning submission is released open source, with documentation of its method, so that others can learn from and build on it.
Yes. The tournament is a research contribution as well as a competition, and the pilot's findings will be published.
Winning methods are also published. As a condition of claiming a prize, winners submit documentation of their approach shortly after results are announced, and winning submissions are released open source. The exact timeline and format are set out in the participant guidelines.
The submission portal opens in September 2026; registration is not yet open. In the meantime, registering your interest through the Register button provides ongoing updates as the tournament develops – announcements, information sessions, and the call for forecasts when submissions open.
Yes. This is the first round, and further rounds are intended to follow.
Register to receive the call for forecasts, links to our webinars and information sessions.
We'll be in touch as the tournament firms up. Thanks for your interest.