Skip to main content

Sinks

A sink writes records out: the framework batches, shards, retries, and acknowledges them, and only commits a source watermark once the sink confirms the data behind it is durable. Configure one with a single-key sink: section (or a sinks: map for multiple destinations). All sinks share the framework's batch / inflight / retry pool settings.

SinkConfig tagSummary
Kafkasink: { kafka: ... }Producer on rdkafka; forced acks=all + idempotence, per-batch delivery-report acknowledgement, one topic per sink.
ClickHousesink: { clickhouse: ... }Sharded RowBinary/Native insert; end-to-end wait_end_of_query acks, dedup tokens, circuit-breaker replica rotation.

The Memory / Capture test connector also acts as a sink (sink: { capture: {} }); it is a dual-role dev aid documented at the connectors root, not here.

Writing your own sink is a small trait pair (RowEncoder + ShardWriter) — see Custom sinks.