RabbitMQ is the invisible engine behind Magento 2's asynchronous work: bulk REST operations, async endpoints, order emails, product alerts, inventory reservation cleanup and B2B quote and shared-catalog updates all travel through message queues. The storefront can be perfectly fast while the queues silently back up — orders confirmed, but emails arriving an hour late, bulk operations stuck on "processing", and inventory_reservation rows growing because the cleanup consumer never caught up. This guide is about the layer most performance audits skip: the broker and the consumers that drain it. You will learn how to read RabbitMQ's diagnostics, where the real bottlenecks hide (usually the consumers, sometimes the broker, rarely both), and a concrete tuning playbook that scales from a single node to a high-volume store. How Magento uses RabbitMQ Magento defines queues through three XML files per module: queue_topology.xml (exchanges, queues, bindings), queue_consumer.xml (which consumer processes which queue, with what handler) and communication.xml (which topic routes to which handler). Messages published under a topic land in the broker, and a consumer process picks them up, calls the handler, and acknowledges the message. Everything is opt-in per queue: you can run consumers per queue yourself with bin/magento queue:consumers:start, and heavy modules (async order processing, B2B, inventory) add their own consumers. If a consumer is never started, its queue simply fills up. That is the most common "RabbitMQ problem" on real stores: not a broker issue, a missing or under-provisioned consumer. See the async operations and message queues overview for the full architecture. Diagnose before you tune Never guess where the backlog is. RabbitMQ ships everything you need on the node itself: # Per-queue depth and consumer state — your primary view rabbitmqctl list_queues name messages messages_ready messages_unacknowledged consumers # Who is connected and how many channels they hold rabbitmqctl list_connections name channel_max client_properties # Per-consumer state: which queue, how many messages it holds rabbitmqctl list_consumers # Broker health: memory, disk, alarms, file descriptors rabbitmq-diagnostics memory rabbitmq-diagnostics alarms rabbitmqctl status On the Magento side, get the list of consumers and confirm which are actually running: bin/magento queue:consumers:list ps aux | grep queue:consumers Read the numbers like this: messages_ready high while consumers > 0: the handlers are too slow or there are too few consumers. Add workers or optimize the handler — more on both below. messages_unacknowledged high: consumers are holding messages mid-processing, crashed mid-message, or stuck in a redelivery loop. Check consumer logs for repeated exceptions and look at handler duration. Memory or disk alarm active: the broker is throttling publishers and consumers. Fix the broker first (see below); tuning consumers while the broker is in flow-control is pointless. Consumers = 0 on a queue that should always be drained: the consumer died and nothing restarted it — a process-manager problem, not a tuning problem. Also correlate with the database. A queue backlog usually shows up as side effects: inventory_reservation growth when the cleanup consumer lags (see the inventory reservation deep dive), or an exploding bulk/operation table when async.operations.all is stuck (see sales order performance). Tune the consumers — the biggest win 1. Decide: wait or exit By default a Magento consumer exits when the queue is empty or after --max-messages. Frequent exit/restart churns connections and delays message processing. Set consumers_wait_for_messages in app/etc/env.php so consumers stay alive and wait for new messages: 'queue' => [ 'consumers_wait_for_messages' => 1 ], 2. Run more than one process per queue A single consumer is usually single-threaded PHP. Throughput scales with worker count until you saturate the handler's real bottleneck (database writes, API calls, filesystem). Run several processes for the queues that matter: bin/magento queue:consumers:start async.operations.all --max-messages=10000 & bin/magento queue:consumers:start async.operations.all --max-messages=10000 & bin/magento queue:consumers:start async.operations.all --max-messages=10000 & For long-lived setups, declare parallelism in app/etc/env.php instead of juggling background processes: 'queue' => [ 'consumers_wait_for_messages' => 1, 'consumers' => [ 'async.operations.all' => ['multiple_processes' => 3] ] ], Magento 2.4.6+ goes further with declarative consumer settings in app/etc/consumers.xml, where you can cap runtime per process per consumer: <config> <consumer name="async.operations.all" maxMessages="10000" maxExecutionTime="3600" maxIdleTime="60" multipleProcesses="3"/> </config> On the command line the same caps exist since 2.4.6 as --max-execution-time and --max-idle-time, next to the classic --max-messages and --batch-size. Pick your poison; the declarative file survives deploys and is reviewable, which is why we prefer it. 3. Cap messages per process — memory control Queue consumers are long-running PHP, and long-running PHP accumulates memory: object managers, singletons and collection instances that never get released. A consumer that runs for days will quietly grow from 120 MB to a gigabyte before crashing — we covered the mechanics in memory leaks in cron jobs & queue consumers. The standard fix is the bounded worker: --max-messages (and --max-execution-time) so each process restarts before memory becomes a problem. 10000 messages or 3600 seconds per process is a sane starting point; watch RSS over a week and tighten until restarts cost less than the leaks. 4. Respect ordering and idempotency Multiple consumers on one queue mean messages can be processed out of order and retried. Most Magento queues tolerate that (bulk operations, emails, reservations), but the handlers must be idempotent: a message redelivered after a crash (redelivered=true) must not double-apply. If you have a queue where order truly matters, run a single process for it and accept the throughput ceiling. 5. Stop poison-message loops A handler that always throws (bad payload, missing product, persistent API error) gets the message rejected and redelivered forever — the consumer logs the same exception in a tight loop, hammering the broker and the database while messages_unacknowledged climbs. Two proven fixes: make the handler safe against bad input (validate, catch, log-and-skip), and add a dead-letter exchange in queue_topology.xml so repeated failures land in a quarantine queue you can inspect instead of looping. Tune the broker Right-size the node RabbitMQ is memory- and I/O-hungry. Give it its own VM or container; do not share a disk with MySQL, and never let the broker swap. For high-volume stores, 4 GB+ RAM and SSDs are a floor, not a luxury — the load testing & capacity planning guide has a method for finding how much of this you actually need. Memory watermark — the alarm everyone hits The broker blocks publishers (flow control) when memory exceeds vm_memory_high_watermark, 0.4 (40% of RAM) by default. On a box with 8 GB that means flow control starts at 3.2 GB — which on stores with deep queues, many consumers and default channel buffers happens sooner than you think. Tune it deliberately, not as a knee-jerk: # rabbitmq.conf vm_memory_high_watermark.absolute = 6GB disk_free_limit.absolute = 2GB The same logic applies to disk: disk_free_limit defaults to 50 MB, which is far too little for a busy store — a full disk freezes everything. Set an absolute limit you can live with. File descriptors and connection limits Every consumer connection, channel and queued message can eat file descriptors. The classic failure is consumers that cannot reconnect because the broker hit its FD ceiling. Raise the system ulimit (65535 or higher) and verify with rabbitmqctl status — the File descriptor count/limit line. If your consumers each hold multiple channels, also check channel_max. Heartbeats RabbitMQ's default heartbeat is 60 seconds. Magento's AMQP client can run with heartbeat 0 (disabled), which means network equipment can silently kill idle consumer connections — the queue looks drained, the consumer looks alive, and nobody is doing any work. Configure heartbeat explicitly in the AMQP section of app/etc/env.php (for example 'heartbeat' => 60) and monitor list_connections for the timeout column so churn is visible. Quorum queues for durability, classic for throughput Magento's default topology uses classic queues, which are the fastest choice for a single-node broker. If you run RabbitMQ in a cluster, know that mirrored classic queues are deprecated and removed in RabbitMQ 4.0 — plan for quorum queues (RabbitMQ 3.8+), which replicate and can survive node loss. Quorum queues cost write amplification, so use them for the critical queues (order processing, B2B) and keep classic queues for throwaway work. Run consumers properly in production The consumers_runner cron job Magento ships is fine for keeping a low-volume store's consumers alive, but for anything serious use a process manager. systemd or supervisord gives you autostart, autorestart with proper backoff, startsecs crash detection, logging, and resource limits per consumer: [program:magento_async_operations] command=php /var/www/magento/bin/magento queue:consumers:start async.operations.all --max-messages=10000 user=www-data autostart=true autorestart=true startsecs=10 numprocs=3 process_name=%(program_name)s_%(process_num)02d redirect_stderr=true Notes from real deployments: Give consumers their own PHP-FPM pool settings equivalent — they compete with web traffic for CPU and MySQL. On a busy store, cap worker counts so a backlog drain doesn't starve the storefront; see PHP-FPM tuning for the same calculation applied to web workers. When a backlog builds up (flash sale, failed consumer overnight), add temporary workers rather than restarting the broker. Draining a 200k-message queue is a throughput problem: workers, not magic. Alert on what matters: queue depth and message age exceeding a threshold, messages_unacknowledged growth, memory/disk alarms, and consumers = 0 for critical queues. Every alert needs a runbook action, or it becomes noise you ignore. Test under load before raising worker counts in production. Email sends and API calls have their own ceilings — blasting 10 workers at a slow SMTP relay makes the backlog worse, not better. The email performance guide shows the same pattern for outbound sending. Playbook summary Measure first: rabbitmqctl list_queues for ready/unacked depth, rabbitmq-diagnostics alarms, queue:consumers:list against ps aux. Set consumers_wait_for_messages=1 so consumers stop exit/restart churn. Scale consumers: multiple_processes per queue in env.php, or consumers.xml (2.4.6+) with maxMessages/maxExecutionTime/maxIdleTime. Bound memory: --max-messages/--max-execution-time caps so processes restart before leaking. Kill poison loops: validate and catch in handlers; dead-letter queue for repeated failures. Right-size the broker: dedicated node, explicit vm_memory_high_watermark and disk_free_limit, raised file descriptors, configured heartbeats. Run under a process manager with alerting on depth, age, unacked, alarms and zero-consumer states. Test under load before scaling workers, and watch the downstream bottleneck (DB, SMTP, APIs), not just queue depth. A healthy RabbitMQ setup is boring: queues near zero, consumers stable for weeks, no alarms. If yours is exciting, work through the diagnosis first — the backlog is almost always a consumer problem wearing a broker costume.