Skip to main content

Framework baselines (permanent harness, 2026-07)

Environment: M5 Max 18-core / 128 GB (dev laptop), quiet machine, Rust 1.96.1, release build. Harness: benchmarks/ binaries; raw JSON in benchmarks/results/. Reproduce per the usage headers in each binary.

Framework ceiling

pipeline_synthetic: generator source → real chain (filter, zero-work) → real sink pool → null writer; 256 B payloads, 2 shards, 2 I/O threads, 64 KiB chunks, 65,536-row batches, 5 ms linger.

Framework ceiling: records/s by pipeline threads
Higher is better
Zero-work records saturate the fixed egress side (2 shard workers), so the throughput peak is at 2 threads — this is a ceiling harness, not a scaling proof.
2 threads (highlighted)other arms
Framework ceiling: records/s by pipeline threads — Higher is better1 thread: 39.4M/s; 2 threads: 74.6M/s; 4 threads: 62.8M/s; 8 threads: 67.1M/s1 thread2 threads4 threads8 threads1 thread: 39.4M/s39.4M/s2 threads: 74.6M/s74.6M/s4 threads: 62.8M/s62.8M/s8 threads: 67.1M/s67.1M/s
Data table
VariantValue95% CIn
1 thread39,391,160.234049454 records/s
2 threads74,570,651.21641488 records/s
4 threads62,844,306.02613395 records/s
8 threads67,114,340.48164862 records/s
Apple M5 Max · commit 63084c56a8 · 2026-07-10
Per-thread throughput
Higher is better
Per-thread records/s falls as threads contend for the fixed egress side.
1 thread (highlighted)other arms
Per-thread throughput — Higher is better1 thread: 39.4M/s; 2 threads: 37.3M/s; 4 threads: 15.7M/s; 8 threads: 8.4M/s1 thread2 threads4 threads8 threads1 thread: 39.4M/s39.4M/s2 threads: 37.3M/s37.3M/s4 threads: 15.7M/s15.7M/s8 threads: 8.4M/s8.4M/s
Data table
VariantValue95% CIn
1 thread39,391,160.234049454 records/s
2 threads37,285,325.60820744 records/s
4 threads15,711,076.506533487 records/s
8 threads8,389,292.560206078 records/s
Apple M5 Max · commit 63084c56a8 · 2026-07-10

Interpretation: ~25.4 ns/record full-framework cost at one thread (lane poll, ack issue/resolve, chain, encode, chunk, queue handoff, batching, checkpoint commits — the pure chain alone is ~9 ns). Zero-work records saturate the fixed egress side (2 shard workers), so throughput peaks at 2 threads and 4 and 8 threads both regress — this is a ceiling harness, not a scaling proof; with real per-record work the pipeline threads dominate and the sink side stops being the limiter. The exact per-arm figures are in each chart's data table — these ceiling arms are a single run each, so they carry no 95% CI (CIs appear only where a benchmark records repeated reps).

The accounting correction records how an earlier revision of these figures read as rows exceeding records by ~10%, and the conservation assertion that now guards it.

⚠ Finding for tuning documentation

With the default sink batch config (500k rows) and an unthrottled source, the 256 MiB in-flight budget fills in tens of milliseconds while a batch needs hundreds of thousands of rows to seal — per-thread pause controllers then sit in 500 ms min_pause cycles and throughput collapses ~24× (measured 2.5M rec/s in that state, hand-recorded during the tuning investigation — no committed record). Real sources rarely outrun the budget this way, but the interaction (batch-seal thresholds vs budget watermarks vs min_pause) deserves a documented sizing rule; recorded for the hardening pass.