Skip to main content

Zero-copy seam (design validation, 2026-07)

This study is a divan micro-bench; it emits no machine-readable Report records, so its hand-recorded tables are kept as prose.

Environment: Apple M-series (dev laptop), Rust 1.96.1, synthetic parse → map → filter → flat_map → shard-encode chain, batch 512, ~5,340 records/drive. Source: seam prototype (branch worktree-agent-a162b0ef2e0a29f4e).

Variantns/recordThroughputAllocations
Borrowed payloads, static chain~4025.2M rec/s0/record (1 Arc per batch, hard-asserted)
Borrowed, dyn boundary per batch~43 (+9%)23M rec/s0/record
Borrowed, dyn per stage~43 on 3 stages; scales with stages, blocks vectorization0/record
Owned payloads, static chain~148 (3.7× slower)6.8M rec/s38,512/iteration

Conclusions: zero-copy justifies the design; erase once per batch, never per stage; the +9% dyn-per-batch delta amortizes to ~1–2% at realistic per-record cost.

Production chain (post-merge, crates/etl-core/benches/chain.rs, batch 512 payloads → 1,536 records via flat_map, full metrics accumulators on): borrowed ~9.2 ns/record, 109M records/s, 0 allocations/record (5/iter fixed); owned equivalent ~23.5 ns/record with one allocation per record — 2.5× slower on small payloads, consistent with the spike's 3.7× at larger sizes. The counting-allocator integration test (crates/etl-core/tests/chain_alloc.rs) hard-fails if per-iteration allocations scale with record count.