pub trait Source: Send {
type Lane: SourceLane;
// Required methods
fn open(&mut self, ctx: SourceCtx) -> Result<(), SourceError>;
fn poll_events(
&mut self,
timeout: Duration,
) -> Result<SourceEvent<Self::Lane>, SourceError>;
fn commit(
&mut self,
watermarks: &[(PartitionId, i64)],
) -> Result<(), SourceError>;
// Provided methods
fn flush_commits(&mut self) -> Result<(), SourceError> { ... }
fn pause(&mut self, lanes: &[LaneId]) -> Result<(), SourceError> { ... }
fn resume(&mut self, lanes: &[LaneId]) -> Result<(), SourceError> { ... }
}Expand description
Control plane of a source. Driven by the runtime’s controller from a single thread; lanes run on pipeline threads.
Required Associated Types§
Sourcetype Lane: SourceLane
type Lane: SourceLane
The lane type this source produces.
Required Methods§
Sourcefn open(&mut self, ctx: SourceCtx) -> Result<(), SourceError>
fn open(&mut self, ctx: SourceCtx) -> Result<(), SourceError>
Connect and prepare. Called once before any other method.
Sourcefn poll_events(
&mut self,
timeout: Duration,
) -> Result<SourceEvent<Self::Lane>, SourceError>
fn poll_events( &mut self, timeout: Duration, ) -> Result<SourceEvent<Self::Lane>, SourceError>
Service control-plane work (rebalance callbacks, statistics) and
return the next event, waiting at most timeout. Must be called
regularly regardless of backpressure state.
Sourcefn commit(
&mut self,
watermarks: &[(PartitionId, i64)],
) -> Result<(), SourceError>
fn commit( &mut self, watermarks: &[(PartitionId, i64)], ) -> Result<(), SourceError>
Store per-partition committable positions (each is the offset one past the last acknowledged record). Positions are durable per the source’s own policy (e.g. interval auto-commit of stored offsets).
Provided Methods§
Sourcefn flush_commits(&mut self) -> Result<(), SourceError>
fn flush_commits(&mut self) -> Result<(), SourceError>
Synchronously flush stored positions (shutdown, revocation).