Sinks
A sink writes records out: the framework batches, shards, retries, and
acknowledges them, and only commits a source watermark once the sink confirms
the data behind it is durable. Configure one with a single-key sink: section
(or a sinks: map for multiple
destinations). All sinks share the framework's
batch / inflight / retry pool settings.
| Sink | Config tag | Summary |
|---|---|---|
| Kafka | sink: { kafka: ... } | Producer on rdkafka; forced acks=all + idempotence, per-batch delivery-report acknowledgement, one topic per sink. |
| ClickHouse | sink: { clickhouse: ... } | Sharded RowBinary/Native insert; end-to-end wait_end_of_query acks, dedup tokens, circuit-breaker replica rotation. |
The Memory / Capture test connector also acts as a sink
(sink: { capture: {} }); it is a dual-role dev aid documented at the
connectors root, not here.
Writing your own sink is a small trait pair (RowEncoder + ShardWriter) —
see Custom sinks.