What is a message queue?
A checkout that calls the warehouse directly goes down the moment the warehouse does. A queue between them means the order still saves.
Message queue
task queue
A message queue is a service that stores messages sent by one system until another system is ready to read them. The sender writes a message and moves on; it does not wait for the message to be handled.
Call an API directly and both sides have to be up, fast, and available at the same instant. A queue removes that coupling. The producer drops a message and returns immediately. A consumer picks it up when it has capacity, whether that is fifty milliseconds later or five minutes later.
That gap is the whole point. It is also the whole cost. Direct calls give you an answer right away, or a clear failure right away. A queue gives you neither. Processing becomes eventual, so the sender has to stop assuming the work is already done by the time it moves on.
It buys three things, at one price
A sale event, a bulk import, or a spike in sign-ups can push traffic to ten times its normal rate for a few minutes. A direct API call has no room to absorb that. A queue does: messages pile up, and the consumer works through them at whatever pace it can sustain. Nothing gets rejected.
The second thing a queue buys is a safe place to retry. A downstream service that times out for thirty seconds does not lose the message; it stays in the queue until that service answers again. Retrying a direct call is harder. The caller must hold the request open and try again itself. That fails the moment the caller restarts or the user closes the tab.
The third is order. Most queues process messages roughly in the sequence they arrived, within a given group or partition. That matters when a later message depends on an earlier one, like an inventory update that has to land before the sale it caused.
- 01Amazon SQS, Google Pub/Sub, RabbitMQ, and Kafka are the queues you will meet most often; they differ in ordering guarantees and throughput, not in the basic contract above.
- 02A queue does not remove the need for the consumer to keep up on average. It only buys time during a burst, not unlimited capacity forever.
- 03Ordering usually applies within a partition or group, not across the whole queue. Two unrelated messages can still process out of sequence.
- ProduceSender writes a message and moves on.
- StoreMessage sits in the queue.
- ConsumeA worker picks it up when free.
- ProcessThe actual work runs.
- AcknowledgeConsumer confirms it finished.
- Retry or dropNo ack means it comes back.
The sender's job ends at step one. Everything after that happens on the consumer's schedule, not the sender's.
Common questions
01When should I use a queue instead of calling an API directly?
Use a queue when the sender should not wait for the work to finish, or when traffic arrives in bursts. A direct call is simpler. It works fine when both sides are fast and reliable, and the caller genuinely needs an immediate answer.
02Does a message queue guarantee a message is processed exactly once?
Most queues guarantee at-least-once delivery by default. A message can be processed more than once if the consumer crashes after doing the work but before acknowledging it. Exactly-once processing needs extra design, usually by making the work safe to repeat rather than by relying on the queue alone.
03Where does the message go if the consumer is down?
The message waits in the queue. Nothing is lost, as long as the queue's retention window is longer than the outage. Most managed queues keep unprocessed messages for a set number of days, then drop them. A very long outage can still lose work if nobody extends that window or drains the backlog first.
04Is a message queue the same as a task queue?
The terms are used interchangeably in most conversations, and task queue is a common alternate name for the same idea. Some frameworks reserve task queue for background jobs triggered from application code. Message queue covers the broader case: any system passing messages between services.
