Expand description
Error taxonomy and per-stage error policies.
Three classes of failure exist in a pipeline (see docs/DESIGN.md):
retryable (transient I/O — handled by the sink retry layer),
record-level (a bad payload or failed user code — subject to
ErrorPolicy), and fatal (invariant violations — the pipeline
stops). Record-level policies are deliberately limited to Skip and
Fail; every skip is surfaced through metrics.
Structs§
- Fatal
Error - An unrecoverable pipeline failure: an invariant was violated or a
Fail-policy stage tripped. The pipeline transitions toFailed, the partition watermarks stop advancing, and the process exits non-zero.
Enums§
- Deser
Error - A payload could not be deserialized.
- Error
Class - Broad classification used in metrics labels (
error_type) and by the retry layer. - Error
Policy - What to do when a record fails in a stage.
- Sink
Error - A sink failed to write a batch.
- Source
Error - A source failed to poll, commit, or manage its assignment.