Interface IConsumerGroupPositioner
Moves this gateway pod's Redis consumer group for a topic to the stream's tail right before the gateway opens its Dapr subscription for that topic, so a new subscription only receives messages published after it opened.
public interface IConsumerGroupPositioner
Remarks
Dapr's Redis pub/sub uses one consumer group per sidecar
(consumerID: {podName}, see the helm pubsub component) and the group
outlives the subscription. When the gateway reopens a topic after its last
caller left, Dapr resumes the group at its last-delivered-id; a group
it doesn't have yet it creates at the stream's start ("0", hard-coded
in components-contrib, with no per-subscription override). Both hand the new
subscriber whatever is still in the stream, and Dapr's reclaim loop
redelivers the group's unacked entries too. streamTTL doesn't bound
this between runs: it trims only when something publishes to the topic.
Observed live: a Binance order-book worker created at 15:31 was fed
11:16 WebSocket diffs and spent its first minutes on hours-old backlog.
The policy is no catch-up at all: whatever was published while this pod had no subscription for the topic is not delivered, including during the brief gap of a worker reload.
Methods
AckAbandonedPendingAsync(IReadOnlyCollection<string>, TimeSpan, Func<string, RetainedEntry, bool>?, CancellationToken)
Ack this pod's consumer-group entries on topics that have
been pending at least minIdle, oldest first, each offered to
redeliver (topic, entry as stored) before its ack. Returns how
many were acked. Never throws; failures are logged.
Task<int> AckAbandonedPendingAsync(IReadOnlyCollection<string> topics, TimeSpan minIdle, Func<string, RetainedEntry, bool>? redeliver, CancellationToken cancellationToken)
Parameters
topicsIReadOnlyCollection<string>minIdleTimeSpanredeliverFunc<string, RetainedEntry, bool>cancellationTokenCancellationToken
Returns
Remarks
The gateway's handler always answers Dapr Success right after fanning out,
so a live entry is pending for microseconds. An entry pending for seconds was
taken by a closed subscription's reader, which on a busy topic grabs one
message before any sweep can wake it (see UnblockSidecarReadersAsync(CancellationToken)).
Left alone, Dapr's reclaim loop redelivers it 60s late (observed: order-book
events 74-86s old). An entry published after the topic's open subscription
started is a live message the closed reader stole from it, so
redeliver hands it to that subscription (2-7s late, not
lost: common#28); an older one is dropped, consistent with subscriptions
starting at the stream tail with no catch-up.
PositionAtTailAsync(string, CancellationToken)
Position the group at the tail and ack its pending entries up to there. Returns the stream id it was positioned at: every entry after it was published for the new subscription (null when this backend cannot say). Throws when that fails, so the subscription fails rather than replaying silently.
Task<string?> PositionAtTailAsync(string topic, CancellationToken cancellationToken)
Parameters
topicstringcancellationTokenCancellationToken
Returns
ReadSinceAsync(string, DateTimeOffset, int, long, CancellationToken)
The entries topic's stream still retains from
since on (the stream's own clock), oldest first; null when
this backend cannot read a stream's history. Stops reading once it holds more
than max entries or more than maxBytes
payload bytes, so the caller sees the overrun without the whole stream in
memory. Reads only: no consumer group moves. For SubscribeRequest.replay_since.
Task<IReadOnlyList<RetainedEntry>?> ReadSinceAsync(string topic, DateTimeOffset since, int max, long maxBytes, CancellationToken cancellationToken)
Parameters
topicstringsinceDateTimeOffsetmaxintmaxByteslongcancellationTokenCancellationToken
Returns
UnblockSidecarReadersAsync(CancellationToken)
Wake this pod's sidecar readers parked in a blocking XREADGROUP
(CLIENT UNBLOCK, timeout mode: the read returns empty, no message
is consumed). A reader left behind by a closed subscription then sees its
cancelled context and exits; a live reader just reads again. Returns how
many were woken. Never throws; failures are logged.
Task<int> UnblockSidecarReadersAsync(CancellationToken cancellationToken)
Parameters
cancellationTokenCancellationToken
Returns
Remarks
daprd cancels a closed streaming subscription's component context, but the
Redis component's reader is parked in XREADGROUP ... BLOCK 0 with no
socket deadline, and go-redis doesn't interrupt it. So it stays parked until
the topic's next message, which it then takes into the consumer group's
pending list and abandons unacked. The live subscriber on the same pod gets
that message only once the reclaim loop picks it up (60s later). Reproduced on
daprd 1.18.1, 1.18.4 and 1.19.0-rc.1, with both Dapr.Messaging and a raw
gRPC call: one leftover reader per closed subscription. Observed live: 40
leftover readers on one gateway sidecar delivered order-book events 30-100s
late, starved its Redis pool (publish circuit breaker open), and stole
diffs from workers.