Scaling Lab — the inference-compute frontier
Sweep the test-time compute budget and watch every strategy trace its own scaling curve on the Game-of-24 benchmark. This is the diagram behind Scaling LLM Test-Time Compute Optimally (Snell et al., 2024) — but live and interactive.
Best-of-N sampling with an Outcome Reward Model
Sample N independent attempts at the same puzzle, score each with an ORM, keep the best. Compare three aggregators: Best-of-N, Majority (self-consistency), and Weighted-vote (soft-max-over-reward).
Chain-of-Thought explorer · self-consistency voting
Watch a single chain of reasoning unfold step by step — or sample K chains and vote on the final answer (Wang et al. 2023, "Self-Consistency Improves CoT").
Tree-of-Thoughts · BFS / DFS / Beam search with value-guided pruning
The Yao et al. (2023) classic, reproduced live. Each node is a partial Game-of-24 state (remaining numbers after an operation). A simulated LLM-judge scores every frontier node with a value in [0,1]; the search keeps only the top-b at each ply.
MCTS reasoner with a Process Reward Model
The o1/R1/Claude-Mythos flavour. Each rollout expands a node by UCB1 = q̂ + c·√(ln N / n), simulates to a terminal, and backpropagates the PRM-averaged step reward. Watch the tree grow — the highest-q branch glows green.
Self-Refine loop · propose → critique → revise
Madaan et al. (2023): a single model iterates on its own output using a self-generated critique. We plot the reward trajectory and mark the reward-hacking / over-refinement regime.
Problem Arena · six strategies race on one puzzle
A head-to-head runoff. All strategies run on the same puzzle under the same compute budget; first to solve wins. Bars show time-to-solution and total tokens.
The mathematics of test-time compute
1. Inference scaling law (Snell et al. 2024, Brown et al. 2024)
For a fixed pre-trained model, the probability of solving a problem with compute budget C empirically follows a power-log frontier:
pass(C) ≈ 1 − exp(−α · Cβ)
with β ≈ 0.3–0.6 depending on strategy. Critically: there exists a compute-optimal strategy per problem difficulty tier — greedy for easy, MCTS+PRM for hard. Below ~10⁴ inference FLOPs all strategies are compute-wasteful; above ~10⁸ the marginal gain collapses.
2. UCB1 selection rule
Inside MCTS we descend by maximizing
UCB(s,a) = q̂(s,a) + c · √( ln N(s) / n(s,a) )
where q̂ is the running mean reward, n the visits, and c the exploration constant (√2 classical). In reasoning MCTS, q̂ is replaced by a blend λ · PRM + (1−λ) · rollout_reward.
3. Outcome vs Process Reward Models
4. Self-consistency (Wang et al. 2023)
Sample K independent chains, return the modal answer. Works because many reasoning tasks admit multiple paths to the same correct answer but different incorrect answers diverge. Empirically > 20 pp lift over greedy on GSM8K at K = 40.
5. Why Game-of-24?
Introduced as a ToT benchmark by Yao et al. 2023 because (a) its search tree has a manageable branching factor, (b) ground-truth validation is trivial, and (c) GPT-4-greedy fails on most instances (pass@1 ≈ 7%). That makes it the perfect domain for comparing test-time strategies: the signal is loud.
Further reading
- Snell, Lee, Xu, Kumar — Scaling LLM Test-Time Compute Optimally, 2024 (arXiv:2408.03314)
- Yao et al. — Tree of Thoughts: Deliberate Problem Solving with LLMs, 2023 (arXiv:2305.10601)
- Wang et al. — Self-Consistency Improves Chain of Thought Reasoning, ICLR 2023
- Lightman et al. — Let's Verify Step by Step, OpenAI 2023 (arXiv:2305.20050)
- Madaan et al. — Self-Refine: Iterative Refinement with Self-Feedback, NeurIPS 2023
- DeepSeek-AI — DeepSeek-R1, 2025 · OpenAI — o1 system card, 2024