PR #1493 · SP8192 + 3-Layer Recurrence + Parallel Residuals + Legal TTT
The combined three-layer recurrence, parallel residual routing, QK-gain and legal TTT variant improves the PR #1394 base stack.
Method before metrics.
On PR #1493: 3-layer recurrence, parallel residuals, QK-Gain 5.25, and legal score-first TTT on the PR #1394 stack Declared changes include: SP8192 + 3-layer depth recurrence (L3-5) + parallel residuals (L7+) + QK-Gain 5.25 + legal score-first TTT No SLOT, no pre-quant TTT, no n-gram cache, no ETLB — fully compliant [x] Eval (sliding + TTT) under 600s (~500s actual) [x] Score-first TTT: compliant with Issue #1017 conditions 1-4
- Hypothesis
- H-PG-1493 · The combined three-layer recurrence, parallel residual routing, QK-gain and legal TTT variant improves the PR #1394 base stack.
- Derived from
- PR #1394 · SP8192 + GPTQ Embeddings + Depth Recurrence + SDClip
experiment:E-PG-1394 - Uses technique from
- PR #1413 · SP8192 + QK-Gain 5 + Legal Score-First TTT
experiment:E-PG-1413 - Compared with
- PR #1514 · SP8192 + Muon 0.97 + Legal Score-First TTT
experiment:E-PG-1514 - Compared with
- PR #1530 · VarLen Attention + Fused MLP + Doc-Independent Legal TTT
experiment:E-PG-1530
The comparison contract.
A reproduction matches this protocol. A fork changes it and declares the deviation.
- model Family
- Parameter Golf language model
- dataset
- FineWeb validation
- benchmarks
- FineWeb validation BPB
- controlled Variables
- 16,000,000-byte artifact cap · 600-second training budget · 8×H100 SXM evaluation hardware · tokenizer-agnostic BPB
- independent Variable
- combined implementation changes declared in the source PR
- seeds
- 42 · 314 · 999
- hardware
- 8×H100 SXM
- pr number
- 1,493
- source pr
- https://github.com/openai/parameter-golf/pull/1493
- source commit
- 857de4755ef003134e4d3130ca792ff54861c577
- source dataset commit
- f5c079314c4877fbb0af378c0abade5a8ca33d3a
- seed count
- 3
- metric direction
- lower_is_better
- artifact limit bytes
- 16,000,000
- training budget seconds
- 600
- declared changes
- SP8192 + 3-layer depth recurrence (L3-5) + parallel residuals (L7+) + QK-Gain 5.25 + legal score-first TTT · No SLOT, no pre-quant TTT, no n-gram cache, no ETLB — fully compliant · [x] Eval (sliding + TTT) under 600s (~500s actual) · [x] Score-first TTT: compliant with Issue #1017 conditions 1-4 · [x] No SLOT, no pre-quant TTT, no ETLB, no n-gram cache
- declared relations
- [object Object] · [object Object]
What this experiment produced or evaluated.
- Produced
- SP8192 + 3-Layer Recurrence + Parallel Residuals + Legal TTT ↗
external-run:ER-PG-1493 - Uses
- benchmark:fineweb-validation-bpb
Change the evidence, not just the discussion.
Start locally, inspect the portable manifest, and sign in only if you choose to publish.
Attach a run to this protocol.
Record the measurement and its provenance. New evidence is unverified and pending review until CodeSOTA checks it.
Stable, portable, attributable.
- Stable ID
- E-PG-1493
- Visibility
- public
- Created
- 2026-04-09
- Updated
- 2026-08-20