RabbitMQ Tutorial for Beginners: Exchanges, Queues, and Message Patterns

Overview

RabbitMQ is a mature message broker that implements the AMQP protocol. It routes messages between producers and consumers through exchanges and queues, and it excels at task distribution, work queues, and reliable delivery. This tutorial explains the core concepts and walks through working code.

When to Use RabbitMQ vs Kafka

Core Concepts

Exchange Types

Running RabbitMQ Locally

docker run -d --name rabbitmq \
  -p 5672:5672 \
  -p 15672:15672 \
  rabbitmq:3-management

Open http://localhost:15672 and log in with guest/guest. The management plugin gives you a full view of exchanges, queues, connections, and message rates.

The official Docker image is documented in the RabbitMQ Docker Hub page.

Step 1: Send a Message with Python

pip install pika

import pika

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()

channel.queue_declare(queue='tasks', durable=True)

channel.basic_publish(
    exchange='',
    routing_key='tasks',
    body='Process order 12345',
    properties=pika.BasicProperties(delivery_mode=2),  # persistent
)

print("Message sent")
connection.close()

An empty exchange name uses the default exchange, which routes directly to the queue named by the routing key.

Step 2: Consume Messages

import pika
import time

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()

channel.queue_declare(queue='tasks', durable=True)
channel.basic_qos(prefetch_count=1)

def callback(ch, method, properties, body):
    print(f"Received: {body.decode()}")
    time.sleep(2)   # simulate work
    ch.basic_ack(delivery_tag=method.delivery_tag)

channel.basic_consume(queue='tasks', on_message_callback=callback)
print("Waiting for messages")
channel.start_consuming()

Acknowledgment and Durability

RabbitMQ removes a message from a queue only after the consumer acknowledges it. If the consumer crashes before acking, the message is redelivered.

For end-to-end durability you need all three: a durable queue, a persistent message, and an acknowledgment.

Work Queue Pattern

Multiple consumers reading from the same queue distribute the workload. This is the classic "competing consumers" pattern.

# Start multiple worker processes
python worker.py &
python worker.py &
python worker.py &

# Then publish many tasks
python producer.py

RabbitMQ round-robins messages across available consumers. With prefetch_count=1, a slow worker does not accumulate a backlog.

Publish/Subscribe with Fanout

# Producer
channel.exchange_declare(exchange='logs', exchange_type='fanout')
channel.basic_publish(exchange='logs', routing_key='', body='Broadcast message')

# Consumer
result = channel.queue_declare(queue='', exclusive=True)
queue_name = result.method.queue
channel.queue_bind(exchange='logs', queue=queue_name)

The empty queue name lets the broker generate a unique name. exclusive=True deletes the queue when the consumer disconnects.

Topic Exchange Routing

channel.exchange_declare(exchange='logs', exchange_type='topic')

# Publish with a structured routing key
channel.basic_publish(exchange='logs', routing_key='auth.error', body='...')
channel.basic_publish(exchange='logs', routing_key='api.info', body='...')

# Subscribe to all auth messages
channel.queue_bind(exchange='logs', queue=q, routing_key='auth.*')

# Subscribe to all errors regardless of service
channel.queue_bind(exchange='logs', queue=q2, routing_key='*.error')

* matches exactly one word; # matches zero or more words.

Dead Letter Queues

channel.queue_declare(
    queue='tasks',
    durable=True,
    arguments={
        'x-dead-letter-exchange': 'dlx',
        'x-message-ttl': 60000,
        'x-max-length': 10000,
    },
)

Messages are routed to the dead-letter exchange when they are rejected, expire, or exceed the queue length. This is essential for diagnosing poison messages that repeatedly fail.

Monitoring and Operations

• Enable the rabbitmq_management plugin for a web UI.
• Use rabbitmqctl list_queues name messages consumers to inspect from the CLI.
• Monitor queue depth; a growing backlog means consumers cannot keep up.
• Set a max queue length to prevent unbounded memory growth.
• Use a cluster with mirrored or quorum queues for high availability.

Common Mistakes