PR #1394 explicitly says its implementation builds on PR #1218.
source PR ↗CodeSOTA · Slayer Research Graph · Genesis Dataset 001Source pinned at f5c079314c48
Imported research programme · OpenAI Parameter Golf
GitHub recorded the runs. Slayer reconstructs the research.
A versioned graph of the question, competing hypotheses, merged experiments, declared ancestry, technique transfer, evidence and cumulative findings behind a 16 MB language model.
- Imported rows
- 48
- Best pinned score
- 1.0565 BPB
- Baseline
- 1.2244 BPB
- Curated graph edges
- 14
01 · Research contract
One question. Fixed constraints. Comparable evidence.
How do we minimize language-model bits per byte under a 16 MB artifact budget?
- Artifact
- ≤ 16,000,000 bytes
- Training
- ≤ 10 minutes
- Evaluation
- FineWeb validation
- Metric
- Bits per byte ↓
The complete submitted artifact, not a parameter-count estimate.
Measured on 8×H100 SXM.
A separate evaluation budget preserves a common target.
Tokenizer-agnostic; lower is better.
02 · Frontier progression
Five scores. Five inspectable experiment records.
This sequence describes movement of the visible frontier. It is deliberately not rendered as a single ancestry chain: leaderboard adjacency is not evidence of code descent.
- 01PR #1394
SP8192 + GPTQ Embeddings + Depth Recurrence + SDClip
On PR #1394: SP8192, GPTQ embeddings, looped layers 4-5, MuonEq-R, and std-based GPTQ clipping
1.08563BPB · 5 seeds - 02PR #1413
SP8192 + QK-Gain 5 + Legal Score-First TTT
On PR #1413: QK-Gain 5.0 + legal score-first TTT on the PR #1394 SP8192 stack
1.08279BPB · 3 seeds - 03PR #1493
SP8192 + 3-Layer Recurrence + Parallel Residuals + Legal TTT
On PR #1493: 3-layer recurrence, parallel residuals, QK-Gain 5.25, and legal score-first TTT on the PR #1394 stack
1.081BPB · 3 seeds - 04PR #1514
SP8192 + Muon 0.97 + Legal Score-First TTT
On PR #1514: SP8192 with Muon 0.97 and legal score-first TTT; 3-seed sweep beats #1493 (p=0.020)
1.07983BPB · 3 seeds - 05PR #1530
VarLen Attention + Fused MLP + Doc-Independent Legal TTT
On PR #1530: variable-length FA3 attention, fused Triton MLP, grouped small-parameter all-reduces, and doc-independent score-first LoRA TTT
1.07336BPB · 3 seeds
03 · Typed lineage
The relation is part of the evidence.
Edges below are curated from explicit statements in the source PRs. A comparison is not silently promoted to ancestry.
PR #1394 attributes the looped-layer depth-recurrence idea to PR #1204.
source PR ↗PR #1394 attributes row-normalized Muon to PR #1217.
source PR ↗The official leaderboard describes PR #1413 as QK-Gain 5.0 and legal score-first TTT on the PR #1394 SP8192 stack.
source PR ↗The official leaderboard describes PR #1493 as three-layer recurrence and parallel residuals on the PR #1394 stack.
source PR ↗PR #1493 uses the QK-gain and legal score-first TTT technique line represented by PR #1413; it does not declare PR #1413 as its sole code parent.
source PR ↗PR #1514 names PR #1413 as its direct predecessor and adds Muon 0.97 plus a causal token n-gram tilt.
source PR ↗PR #1514 identifies PR #1394 as the underlying SP8192 base stack.
source PR ↗The official leaderboard reports that the PR #1514 three-seed sweep beats PR #1493; it does not claim PR #1493 as the implementation parent.
source PR ↗PR #1530 says it applies its earlier changes to the recent PR #1523 record implementation.
source PR ↗PR #1530 says it incorporated the orthogonal improvements from PR #1552 before submission.
source PR ↗PR #1530 ports its variable-length attention, fused MLP and document-independent LoRA TTT changes from the author's earlier PR #1354.
source PR ↗PR #1530 reports its result against the contemporary PR #1529 frontier.
source PR ↗PR #1530 references PR #1493 as an earlier score and technique comparison, not as its direct parent.
source PR ↗04 · Imported evidence
The complete pinned leaderboard.
48 accepted rows imported from the official README at commit f5c079314c48. Scores remain source observations; CodeSOTA does not label them independent reproductions.
| Rank | Run | BPB ↓ | Author | Date | Source |
|---|---|---|---|---|---|
| 1 | Calib32 Token-Only N-gram + AsymLogit StackOn PR #2135: pre-cutoff PR #2130 architecture rerun on clean canonical CaseOps data with GPTQCALIBRATIONBATCHES=32; 3-seed mean 1.05651 under grace policy (p=0.014 vs PR #2014) | 1.0565 | codemath3000 | 2026-05-01 | PR #2135 ↗ |
| 2 | Progressive Context Growth + Short-Doc Score-First TTTOn PR #2014: PR #1855/#1953 CaseOps stack with progressive context growth to 3k plus short-doc score-first TTT on the AWQ-lite/AsymLogit lineage; 3-seed mean 1.05759 (p=0.011 vs PR #1953) | 1.0576 | simonbissonnette | 2026-04-30 | PR #2014 ↗ |
| 3 | Long-Context No-Q/V TTT + QK-Gain 5.25On PR #1953: PR #1945 V21 base with 2560 eval/TTT context, no-Q/V TTT mask, TTT LR 0.75, and QKGAININIT=5.25; 3-seed mean 1.05855 (p=0.063 vs PR #1945 V21 v2) | 1.0586 | andrewbaggio1 | 2026-04-30 | PR #1953 ↗ |
| 4 | AWQ-Lite GPTQ + AsymLogit on PR1855 StackOn PR #1945 commit 70067534: PR #1855 stack plus PR #1908 AWQ-lite mixed GPTQ and PR #1923 AsymLogit; V21 v2 3-seed mean 1.05943 after strict seed-42 rerun (p=0.034 vs PR #1855) | 1.0594 | alertcat | 2026-04-29 | PR #1945 ↗ |
| 5 | BOS-Fixed SmearGate + LQER + SparseAttnGate + 9-Hparam StackOn PR #1855: BOS-fixed #1797-derived stack with LQER, PR #1787 SparseAttnGate/PolarNS/FusedCE base, per-group lrzip compression, and 9 greedy hyperparameter overrides; submitted 3-seed mean 1.06108 with broader reproduction support (p=0.188 vs PR #1868 latest rerun) | 1.0611 | codemath3000 | 2026-04-27 | PR #1855 ↗ |
| 6 | BOS-Fixed SmearGate + LQER Asymmetric + PR1787 SparseAttn + Phased TTTOn PR #1851 with 3-seed compliance-rerun support from PR #1868: BOS-boundary fix from PR #1851 applied to dexhunter's PR #1797 SmearGate + LQER stack, using the PR #1787 SparseAttnGate/PolarNS/FusedCE base plus CaseOps and phased score-first TTT | 1.0614 | aquariouseworkman | 2026-04-27 | PR #1851 ↗ |
| 7 | PR1736 + PolarNS + MINLR + SparseAttnGate + FusedCE + Warm-A TTTOn PR #1787: PR #1736 CaseOps stack plus Polar Express Newton-Schulz coefficients, MINLR=0.1, SparseAttnGate, fused softcapped CE, and PR #1767-style warm-start-A TTT | 1.0634 | nprime06 | 2026-04-23 | PR #1787 ↗ |
| 8 | CaseOps + MLPClip12 + SmearGate/LoRA-TTTOn PR #1769: CaseOps stack with SmearGate/LoRA-TTT refinements and MLPClip12; 5-seed mean improves the accepted CaseOps frontier (p=0.063 vs #1736) | 1.0645 | dexhunter | 2026-04-22 | PR #1769 ↗ |
| 9 | SP8192 + CaseOps + GatedAttn + QuantGate + Loop45 + Phased TTTOn PR #1736: adopts romeerp's lossless CaseOps transform from PR #1729 with byte-sidecar BPB accounting, then adds gated attention and quant-gate scaling on the PR #1530 SP8192 phased-TTT stack | 1.0655 | dexhunter | 2026-04-19 | PR #1736 ↗ |
| 10 | CaseOps Tokenizer + Tapered WD + Phased TTTOn PR #1729: lossless CaseOps bijective case transform with validation byte sidecars, plus mild late Muon weight-decay taper on the PR #1626 legal phased-TTT stack | 1.0678 | romeerp | 2026-04-19 | PR #1729 ↗ |
| 11 | SmearGate + Attention Output Gate + Legal TTTOn PR #1667: SmearGate, attention output gate, depth recurrence, parallel residuals, QK-Gain 5.25, quantization, and score-first TTT | 1.0714 | MarioPaerle | 2026-04-16 | PR #1667 ↗ |
| 12 | VarLen Attention + Fused MLP + Multi-Phase Global SGD TTTOn PR #1626: VarLen attention, fused MLP, multi-phase global SGD TTT, trimmed GPTQ, MLR 0.026, int7 embeddings, and adaptive clip | 1.0719 | dexhunter | 2026-04-14 | PR #1626 ↗ |
| 13 | VarLenAttn + PhasingTTTOn PR #1610: #1530-style VarLen/fused stack plus phased TTT over already-scored validation chunks | 1.0728 | romeerp | 2026-04-13 | PR #1610 ↗ |
| 14 | VarLen Attention + Fused MLP + Doc-Independent Legal TTTOn PR #1530: variable-length FA3 attention, fused Triton MLP, grouped small-parameter all-reduces, and doc-independent score-first LoRA TTT | 1.0734 | samacqua | 2026-04-11 | PR #1530 ↗ |
| 15 | Improved Parallel Residuals + CUTLASS EVT + Legal TTTOn PR #1529: PR #1523 SP8192 baseline with fuller two-lane parallel residual routing, PARALLELRESIDUALSTART=8, inline CUTLASS EVT/Triton fused kernels, and legal score-first TTT; corrected 3-seed mean after GPTQ reserve/seed fix (p=0.001 vs #1514) | 1.0758 | msisovic | 2026-04-11 | PR #1529 ↗ |
| 16 | SP8192 + Muon 0.97 + Legal Score-First TTTOn PR #1514: SP8192 with Muon 0.97 and legal score-first TTT; 3-seed sweep beats #1493 (p=0.020) | 1.0798 | dexhunter | 2026-04-09 | PR #1514 ↗ |
| 17 | SP8192 + 3-Layer Recurrence + Parallel Residuals + Legal TTTOn PR #1493: 3-layer recurrence, parallel residuals, QK-Gain 5.25, and legal score-first TTT on the PR #1394 stack | 1.081 | bigbag | 2026-04-09 | PR #1493 ↗ |
| 18 | SP8192 + Parallel Residuals + Score-First TTTOn PR #1477: parallel residuals on the PR #1413 SP8192 + legal score-first TTT stack | 1.0822 | aryanbhosale | 2026-04-08 | PR #1477 ↗ |
| 19 | SP8192 + QK-Gain 5 + Legal Score-First TTTOn PR #1413: QK-Gain 5.0 + legal score-first TTT on the PR #1394 SP8192 stack | 1.0828 | dexhunter | 2026-04-06 | PR #1413 ↗ |
| 20 | SP8192 + Parallel Residuals + Hessian-Aware SDClipOn PR #1412: parallel residuals, Hessian-aware SDClip, and progressive recurrence on the PR #1394 stack | 1.0835 | Robby Sneiderman | 2026-04-06 | PR #1412 ↗ |
| 21 | SP8192 + GPTQ Embeddings + Depth Recurrence + SDClipOn PR #1394: SP8192, GPTQ embeddings, looped layers 4-5, MuonEq-R, and std-based GPTQ clipping | 1.0856 | Kevin Clark | 2026-04-05 | PR #1394 ↗ |
| 22 | SP4096 + Depth Recurrence + Parallel Residuals + MuonEq-ROn PR #1334: SP4096 + depth recurrence + parallel residuals + MuonEq-R + QK-Gain 5.0 | 1.0897 | aryanbhosale | 2026-04-04 | PR #1334 ↗ |
| 23 | MuonEq-R + Depth Recurrence + WD=0.090 + All-Int6 GPTQOn PR #1285: MuonEq-R + layers 4-5 recurrence + higher weight decay + all-int6 GPTQ | 1.0912 | dexhunter | 2026-04-03 | PR #1285 ↗ |
| 24 | 4096-Vocab + Larger Model + High WD + SimplificationsOn PR #1218: SP4096 + 4x MLP + high weight decay, with TTT, hash embeddings, SmearGate, and value residuals removed | 1.0979 | Kevin Clark | 2026-04-01 | PR #1218 ↗ |
| 25 | Parallel Residuals + Mini Depth RecurrenceOn PR #1204: mini recurrence on layers 4-5 + parallel attention/MLP residual lanes + AR self-generated GPTQ calibration | 1.1063 | Marko Sisovic | 2026-03-31 | PR #1204 ↗ |
| 26 | RascalOn PR #1120: XSA-all + Parallel Muon + coprime loader + Bigram2048/RoPE16 + SWA/late QAT without GPTQ | 1.1099 | newjordan | 2026-03-30 | PR #1120 ↗ |
| 27 | Coprime-Stride Loader + Full GPTQ + XSA-allOn PR #1060: coprime multi-shard loader + Full Hessian GPTQ + XSA on all layers + BigramHash(2816x112) | 1.1122 | dexhunter | 2026-03-29 | PR #1060 ↗ |
| 28 | 11L AR Self-Gen GPTQ + XSAOn PR #1019: Self-Generated GPTQ Calibration Data + all-layer XSA on the PR #549 stack | 1.1147 | abaybektursun | 2026-03-25 | PR #1019 ↗ |
| 29 | LeakyReLU² + Legal Score-First TTT + Parallel MuonOn PR #549: LeakyReLU(0.5)^2 + TTT + Parallel Muon on the PR #414 stack | 1.1194 | abaybektursun | 2026-03-23 | PR #549 ↗ |
| 30 | 11L EMA + GPTQ-lite + warmdown3500On PR #374: GPTQ-lite clip search + EMA, plus warmdown3500 and QAT@0.15 | 1.1228 | signalrush | 2026-03-22 | PR #374 ↗ |
| 31 | 11L Partial RoPE + LN Scale + EMA + XSA4On PR #287: Partial RoPE (16/64) + layerwise LN scale | 1.1248 | jfprincz | 2026-03-21 | PR #287 ↗ |
| 32 | 11L XSA4 + EMA + Int6 MLP3xOn PR #198: XSA on the last 4 layers + EMA replacing SWA | 1.1271 | jfprincz | 2026-03-20 | PR #198 ↗ |
| 33 | 11L Efficient Partial XSAOn PR #198: Efficient Partial XSA on the deepest 3 layers | 1.1307 | unnir | 2026-03-20 | PR #198 ↗ |
| 34 | 10L Int5-MLP + BigramHash(10240)10 layers, mixed int5/int6 quantization, BigramHash(10240), SWA(0.4), WD=0.04 | 1.1428 | thwu1 | 2026-03-20 | source ↗ |
| 35 | Int6 MLP3x + SmearGate + BigramHash3x MLP + SmearGate + BigramHash + OrthoInit + Muon WD + SWA | 1.1458 | Raahil Shah | 2026-03-20 | source ↗ |
| 36 | 11L MLP3x + Int6 QAT11 layers, 3x MLP, int6 QAT, zstd-22, WD=0.04, sliding eval | 1.1502 | aruniyer | 2026-03-20 | source ↗ |
| 37 | SmearGate + OrthoInit + Muon WDSmearGate + BigramHash + 3x MLP + int6 STE QAT + sliding eval | 1.1556 | aquariouseworkman | 2026-03-19 | source ↗ |
| 38 | Ternary Quantization73.7M params quantized to 1 0 -1 + misc arch changes | 1.157 | Ciprian-Florin Ifrim | 2026-03-24 | source ↗ |
| 39 | 10L Int6 QAT + Zstd MLP2.6x10 layers, int6 QAT + zstd-22, MLP 1344, Muon 0.99, sliding eval | 1.1586 | yahya010 | 2026-03-19 | source ↗ |
| 40 | Mixed Quant + Sliding Window EvalInt6 block weights + int8 embeddings + 3x MLP + sliding eval | 1.163 | aquariouseworkman | 2026-03-19 | source ↗ |
| 41 | Muon WD + 10 layerIncludes prev. wins + Spectral embed init + resid mix | 1.1748 | notapplica | 2026-03-19 | source ↗ |
| 42 | Sliding Window EvalSliding window evaluation at stride=64, increasing context for eval | 1.1925 | Matthew Li | 2026-03-19 | source ↗ |
| 43 | Lora TTTTest-time training with LORAs | 1.1928 | samacqua | 2026-03-19 | source ↗ |
| 44 | 4k seq length4k seq length + better hypers | 1.2014 | Spokane Way | 2026-03-19 | source ↗ |
| 45 | 2048 seq length2048 seq length (train + val) | 1.206 | Spokane Way | 2026-03-18 | source ↗ |
| 46 | int6 mixed precision10 layers, mixed int8/int6 | 1.2147 | Nan Liu | 2026-03-18 | source ↗ |
| 47 | fp16 EmbedFP16 Tied Embedding + LR/Warmdown Tuning | 1.2197 | Renier Velazco | 2026-03-18 | source ↗ |
| 48 | Naive Baseline9layer 512dim 1024vocab TiedEmbeddings 4 KV heads | 1.2244 | Baseline | 2026-03-18 | source ↗ |
05 · Provenance
Refreshable, attributable, inspectable.
The checked-in dataset can be regenerated with npm run research:import-parameter-golf. The importer resolves the official leaderboard, repository snapshot and selected PR evidence, then records the exact upstream commit in the artifact.
- Repository snapshot
- openai/parameter-golf ↗
f5c079314c4877fbb0af378c0abade5a8ca33d3a - Repository activity observed
- 5,177 stars · 3,284 forks · 2026-08-20
- Independent reruns
- Not inferred from merge status. They require their own evidence record.
- Post-challenge review
- OpenAI review ↗