Class ProviderSelectingConsumerGroupPositioner
Picks the positioner for the pub/sub backend actually loaded, by the Dapr
component's type, the first time it is needed (the sidecar is up by then).
Only pubsub.redis has one; other backends start subscriptions as
their component configures (e.g. Kafka initialOffset: newest).
public sealed class ProviderSelectingConsumerGroupPositioner : IConsumerGroupPositioner
- Inheritance
-
ProviderSelectingConsumerGroupPositioner
- Implements
- Inherited Members
Constructors
ProviderSelectingConsumerGroupPositioner(Func<CancellationToken, Task<string?>>, Func<IConsumerGroupPositioner?>, ILogger)
Picks the positioner for the pub/sub backend actually loaded, by the Dapr
component's type, the first time it is needed (the sidecar is up by then).
Only pubsub.redis has one; other backends start subscriptions as
their component configures (e.g. Kafka initialOffset: newest).
public ProviderSelectingConsumerGroupPositioner(Func<CancellationToken, Task<string?>> componentType, Func<IConsumerGroupPositioner?> createRedis, ILogger logger)
Parameters
componentTypeFunc<CancellationToken, Task<string>>createRedisFunc<IConsumerGroupPositioner>loggerILogger
Fields
RedisType
The Dapr component type with a positioner.
public const string RedisType = "pubsub.redis"
Field Value
Methods
AckAbandonedPendingAsync(IReadOnlyCollection<string>, TimeSpan, Func<string, RetainedEntry, bool>?, CancellationToken)
Ack this pod's consumer-group entries on topics that have
been pending at least minIdle, oldest first, each offered to
redeliver (topic, entry as stored) before its ack. Returns how
many were acked. Never throws; failures are logged.
public Task<int> AckAbandonedPendingAsync(IReadOnlyCollection<string> topics, TimeSpan minIdle, Func<string, RetainedEntry, bool>? redeliver, CancellationToken cancellationToken)
Parameters
topicsIReadOnlyCollection<string>minIdleTimeSpanredeliverFunc<string, RetainedEntry, bool>cancellationTokenCancellationToken
Returns
Remarks
The gateway's handler always answers Dapr Success right after fanning out,
so a live entry is pending for microseconds. An entry pending for seconds was
taken by a closed subscription's reader, which on a busy topic grabs one
message before any sweep can wake it (see UnblockSidecarReadersAsync(CancellationToken)).
Left alone, Dapr's reclaim loop redelivers it 60s late (observed: order-book
events 74-86s old). An entry published after the topic's open subscription
started is a live message the closed reader stole from it, so
redeliver hands it to that subscription (2-7s late, not
lost: common#28); an older one is dropped, consistent with subscriptions
starting at the stream tail with no catch-up.
PositionAtTailAsync(string, CancellationToken)
Position the group at the tail and ack its pending entries up to there. Returns the stream id it was positioned at: every entry after it was published for the new subscription (null when this backend cannot say). Throws when that fails, so the subscription fails rather than replaying silently.
public Task<string?> PositionAtTailAsync(string topic, CancellationToken cancellationToken)
Parameters
topicstringcancellationTokenCancellationToken
Returns
ReadSinceAsync(string, DateTimeOffset, int, long, CancellationToken)
The entries topic's stream still retains from
since on (the stream's own clock), oldest first; null when
this backend cannot read a stream's history. Stops reading once it holds more
than max entries or more than maxBytes
payload bytes, so the caller sees the overrun without the whole stream in
memory. Reads only: no consumer group moves. For SubscribeRequest.replay_since.
public Task<IReadOnlyList<RetainedEntry>?> ReadSinceAsync(string topic, DateTimeOffset since, int max, long maxBytes, CancellationToken cancellationToken)
Parameters
topicstringsinceDateTimeOffsetmaxintmaxByteslongcancellationTokenCancellationToken
Returns
UnblockSidecarReadersAsync(CancellationToken)
Wake this pod's sidecar readers parked in a blocking XREADGROUP
(CLIENT UNBLOCK, timeout mode: the read returns empty, no message
is consumed). A reader left behind by a closed subscription then sees its
cancelled context and exits; a live reader just reads again. Returns how
many were woken. Never throws; failures are logged.
public Task<int> UnblockSidecarReadersAsync(CancellationToken cancellationToken)
Parameters
cancellationTokenCancellationToken
Returns
Remarks
daprd cancels a closed streaming subscription's component context, but the
Redis component's reader is parked in XREADGROUP ... BLOCK 0 with no
socket deadline, and go-redis doesn't interrupt it. So it stays parked until
the topic's next message, which it then takes into the consumer group's
pending list and abandons unacked. The live subscriber on the same pod gets
that message only once the reclaim loop picks it up (60s later). Reproduced on
daprd 1.18.1, 1.18.4 and 1.19.0-rc.1, with both Dapr.Messaging and a raw
gRPC call: one leftover reader per closed subscription. Observed live: 40
leftover readers on one gateway sidecar delivered order-book events 30-100s
late, starved its Redis pool (publish circuit breaker open), and stole
diffs from workers.