How submissions are scored
Four metrics combine into one composite score. Challenge 1 ranks on that score; Challenge 2 ranks on how much of it you keep when you move to a new machine.
What we measure
Flux accuracy
Coefficient of determination of the predicted flux map over all grid points, timesteps, and test shots. The primary signal of reconstruction quality.
Predicted scalars
Mean R² of the only two scalars you submit — q95 q₉₅ (edge safety factor) How many times a magnetic field line goes the long way round the torus for each time it goes the short way, measured near the plasma edge. It is a stability margin — hence "safety". One of the two scalars you submit (it depends on the toroidal field function, which a flux map cannot contain). and βN β_N (normalized beta) How much plasma pressure the magnetic field is holding, normalized so machines of different sizes can be compared. One of the two scalars you submit (it depends on the pressure profile, which a flux map cannot contain). . They are the two equilibrium parameters that a flux map cannot contain (they need the toroidal field function and the pressure profile), so they remain genuine prediction targets.
Boundary alignment
Symmetric Hausdorff distance Hausdorff distance The worst-case gap between two shapes: how far you must travel from the least-well-matched point on one contour to reach the other. Used to compare predicted and true plasma boundaries. between the predicted and true LCFS LCFS Last Closed Flux Surface — the outermost magnetic surface that closes on itself without hitting a wall. In practice: the edge of the plasma. contours, normalized by the true LCFS major radius. Smaller is better. You do not submit a contour — the scorer extracts it from your flux map.
ψ-derived scalars
Mean agreement of seven scalars the scorer derives from your submitted flux map — Raxis, Zaxis, elongation κ, triangularity δtop/δbot, volume V, and li l_i (internal inductance) A measure of how peaked the electric current is across the plasma — high l_i means current concentrated in the core. You do not submit it: the scorer derives l_i from your flux map (part of the Consistency term). — scored against the same derivation on the true flux. A scalar can only be earned by a flux map that implies it.
No scalar is skipped because of boundary type. The corpus is
diverted-only, so the
limited-plasma carve-out that used to exclude the shape terms is gone. A frame leaves a scalar's
average only when the ground-truth derivation itself fails on it — a degenerate frame with
no closed flux surface, say. If your flux map fails to yield a scalar the truth does have,
that is not a free pass: the scorer substitutes the mean and you take the loss. A perfect flux map
scores D_LCFS = 0 and Consistency = 1 by construction, because the same
published code runs on your flux and on the true flux.
One score per challenge
S_model S_model The composite score that decides Challenge 1: S = 0.55·R²ψ + 0.15·R²_q95,βN + 0.10·(1 − D_LCFS) + 0.20·Consistency. It blends flux-map accuracy, the two predicted scalars, boundary alignment, and the consistency of the eight ψ-derived scalars into one number in [0, 1]. Higher is better.
S = 0.55·R²ψ + 0.15·R²_q95,βN + 0.10·(1 − D_LCFS) + 0.20·Consistency
Highest composite S_model on the hidden DIII-D test set.
G_ratio G_ratio The score that decides Challenge 2: G = S_model(MAST) / S_model(DIII-D) — the fraction of your DIII-D performance you keep when moving zero-shot to MAST. 1.0 would mean perfect transfer. It needs predictions for both machines: a DIII-D-only entry scores 0.
G = S_model(MAST) / S_model(DIII-D)
Highest G_ratio among entries reaching R²ψ > 0.6 on DIII-D.
S_model is a weighted blend in [0, 1] — R² terms are clipped at 0 and
D_LCFS at 1. G_ratio is the fraction of your DIII-D performance retained under
zero-shot Zero-shot Applying a model to a new machine with no training data from it at all — no fine-tuning, no examples. MAST has zero training shots on purpose; that is the whole of Challenge 2. transfer, so values near 1 mean near-complete transfer. Because it is a
ratio of the two, Challenge 2 requires predictions for both machines: a
DIII-D-only entry scores G_ratio = 0.
Submission format
What the scorer expects
.npz per machine, keyed by shot and EFIT timestamp: your flux maps plus
q95 and betaN — nothing else. The boundary, axis, shape, volume and
li are all derived from your flux map by the scorer.
Scoring runs CPU-only, well within the platform limit.
The manifest
manifest.json travels with your predictions. It is descriptive only — the scorer
locates your predictions by filename, so keep the two .npz named exactly
as the config names.
Development, then a blind final
Phase 1 — development
Jul 27 – Oct 18, 2026Phase 2 — final (blind)
Oct 19 – Oct 26, 2026Rules & eligibility
- 1 Each team joins the competition on Codabench with a single valid contact email; aliases that already exist as solo participants are deactivated. Registering with the organizers is a separate step and is not required to be scored.
- 2 External public datasets (other tokamak archives, OMFIT-produced equilibrium tables) and publicly available pre-trained vision or scientific foundation models are permitted, with explicit disclosure in the methods report.
- 3 No restriction is placed on programming language or framework.
- 4 All train/test splits must respect shot-level boundaries.
- 5 Prize candidates must supply code and a pinned environment specification that reproduces their submitted predictions, so the organizers can independently rebuild and re-score them.
- 6 Top three teams in each award category must release source code under an OSI-approved licence and submit a 1–2 page methods report before prizes are paid.
- 7 Organizing-team members with access to the hidden ground truth are excluded from prize eligibility.
- 8 All participants follow the NeurIPS Code of Conduct.