Phase 6
Populated context, KV placement, and reference comparisons.
Read complete source ↗ Evidence / performance
The measurement.
And its limits.
Selected reported results, not a universal speed claim. Every row below belongs to a specific model, workload, and decoding configuration.
Measured machine A large-RAM server.
A large-RAM server.
An 8 GiB GPU.
- CPU2× Xeon Gold 6130 · 32C / 64T
- Memory376 GiB DDR4 · dual NUMA
- GPUGeForce GTX 1070 · 8 GiB
- PlatformLinux · AVX-512 capable
Reading these results
Source: Phase 6.6.1 tables in the report snapshot at 140f6f7. Engine change:
c7c8c79; earlier Phase 6.6: ed7a5db. GEXEC active, CPU expert
path, max draft 4, greedy generation. Qwen cross-window parity runs use GUILD_IQ_MT_MIN=1.
These are decode tokens per second, not end-to-end request throughput. Ranges combine reported development runs and revisions; they are not confidence intervals or paired median/dispersion estimates.
GEXEC artifacts were regenerated during tuning. Exact per-row calibration identity is not established here, limiting controlled cross-revision comparisons.
01 / Selected development results The workload changes
The workload changes
the outcome.
| Model / quantization | Workload | Prompt / gen | Normal | Fixed strategy | Adaptive | Source |
|---|---|---|---|---|---|---|
Ornith 1.5 35B Q4_K_M | W2 · Summarization | 256 / 128 | 50.27 | Lookup · 82.19 | 80.08–80.84 | §8.3 A ↗ |
Ornith 1.5 35B Q4_K_M | W7 · Creative prose | 256 / 128 | 49.75 | Lookup · 48.84 | 49.10–49.18 | §8.3 A ↗ |
Qwen3.8-Flash-Next UD-IQ4_XS | W3 · JSON schema | 256 / 64 | 18.33 | MTP · 23.43 | 23.52 | §8.3 B ↗ |
Qwen3.8-Flash-Next UD-IQ4_XS | W8 · Mathematical reasoning | 256 / 64 | 18.40 | MTP · 23.94 | 23.68 | §8.3 B ↗ |
Ornith 1.5 35B / W2 · Summarization
Normal (T=1) 50.27 tok/s
Fixed (Lookup) 82.19 tok/s
Adaptive (Phase 6.6.1) 80.08–80.84 tok/s
Favorable input/output overlap. Fixed lookup is faster in the reported results.
Ornith 1.5 35B / W7 · Creative prose
Normal (T=1) 49.75 tok/s
Fixed (Lookup) 48.84 tok/s
Adaptive (Phase 6.6.1) 49.10–49.18 tok/s
Adaptive remains below normal. Fallback does not guarantee zero regression.
Qwen3.8-Flash-Next / W3 · JSON schema
Normal (T=1) 18.33 tok/s
Fixed (MTP) 23.43 tok/s
Adaptive (Phase 6.6.1) 23.52 tok/s
Development result; the small difference from fixed MTP is not a demonstrated statistically significant win.
Qwen3.8-Flash-Next / W8 · Mathematical reasoning
Normal (T=1) 18.40 tok/s
Fixed (MTP) 23.94 tok/s
Adaptive (Phase 6.6.1) 23.68 tok/s
Phase 6.6 adaptive was 24.31 tok/s. This Phase 6.6.1 result regresses versus that revision and fixed MTP.
Held-out counterexample: Ornith H4 (Riemann tensor) reports 50.16 tok/s normal,
42.07 fixed lookup, and 47.92 adaptive. Adaptive is below normal by about 4.5%. The summary table
does not provide complete per-run configuration; treat it as a reported observation rather than
a reproducible paired estimate.
02 / Boundaries of the evidence What these numbers
What these numbers
do not establish.
80+ tok/s Ornith results rely on favorable speculative workloads with substantial reusable text. They are not general chat throughput, all-model performance, or measurements on a small-RAM consumer desktop.
Offline calibration measures CPU dispatch and lookup/retrieval microbenchmarks. Full verification curves, GPU/context costs, and concurrent-session scaling still include estimates. A doubled CPU job count is not a concurrent-session benchmark.
The website does not assert successful Phase 6.6.1 sanitizer, complete CUDA CTest, seeded-sampling, or concurrency validation without retained evidence. The engine report is available as a historical source, with those limits noted.
Ryzen 5 5600X / RTX 5070 results remain pending actual hardware measurement.
03 / Source archive
Read the engineering.
Phase 6.5
CPU expert kernels, prefill, and end-to-end optimization.
Read complete source ↗Phase 6.6 / 6.6.1
Adaptive decoding, calibration, reported workloads, and held-out cases.
Read complete source ↗