Messaging and Queues
Published on
Messaging decouples producers and consumers in time and rate. It can absorb bursts and isolate availability, while introducing backlog, duplicate delivery, ordering, schema, and operational concerns.
Define the contract
- event or command meaning, schema owner, and compatibility rules;
- partition or routing key and the ordering boundary it creates;
- delivery acknowledgement and redelivery behavior;
- retention, replay, and deletion requirements;
- consumer idempotency and deduplication strategy;
- retry limits, delay, poison-message handling, and dead-letter ownership.
“Exactly once” is usually scoped to a particular broker or transaction boundary. End-to-end effects still require idempotency, unique operation identifiers, transactional outbox or equivalent coordination, and reconciliation.
Operate backlog
Measure oldest-message age, arrival and processing rates, retry volume, partition skew, consumer lag, failures, and dead-letter growth. Queue length alone is ambiguous because acceptable length depends on throughput and service objectives.
Apply backpressure before backlog exceeds recovery capacity. Scale consumers only while downstream systems can absorb the work. Preserve replay safety and data retention when changing schemas or consumer behavior.
Choose a work queue, pub/sub broker, or durable event log from delivery, ordering, replay, throughput, latency, routing, and ownership requirements—not from product category alone.