Watch the model solve.
Inspect each puzzle, its generated reasoning and program, and the grids checked by the evaluator.
Program execution diagnostics — separate from grid accuracy
— · Programs passing demonstrations and query checks
This checks the generated Python program. It is not the grid score or a repair-run score. The model’s written grid is scored independently.
Success means every final predicted query grid matches its expected output exactly, including dimensions and every cell. Program execution is recorded separately to support verification and repair; it does not determine grid correctness. Live results are incomplete: faster responses arrive first, so early percentages can be biased. Only checkpoint 3493 appears here. Expected answers are visible to you but hidden from the model and its tools. Full saved text appears when a response completes; this view does not stream unfinished tokens.
Baseline generation and scoring first. New repair runs are deferred; earlier repair-pilot results remain available.
Per-family scores
| Family | Scored | Grid correct | Program checks |
|---|
Select a family
Completed responses will appear here as evaluation proceeds.