[Other] Recommended number of connections when using multiple independent consumers (Camel + Spring-AMQP)

[clude86]

Community Support Policy

• I have read RabbitMQ's Community Support Policy

RabbitMQ version used

4.1.x

How is RabbitMQ deployed?

Community Docker image

Steps to reproduce the behavior in question

Hello,

I have a question regarding RabbitMQ best practices for the number of client connections when using many independent consumers.

In my application (Java / Spring Boot), I am using Apache Camel with the 'spring-rabbitmq' component, which internally uses Spring-AMQP. By default, Spring-AMQP gives all listener containers the same ConnectionFactory, and therefore all consumers share a single TCP connection and a single underlying executor thread pool (default size: availableProcessors * 2 threads)

In our setup, several independent Camel routes (each consuming from its own queue with prefetch: 1 and manual ackn) start to block each other because they share the same ConnectionFactory and the same thread pool. This leads to consumer stops even though queues are independent. This happens in specific situations (network outage, connection problem with an external system) where the camel route may need some time to complete, which leeds to the mentioned blocking scenario.
 
If we instead create one ConnectionFactory per route (effectively giving each consumer route its own dedicated TCP connection), this could potentially resolve the issue, as each consumer would then be isolated.

Current RabbitMQ infrastructure overview:

• Queues: 90
• Connections:
  Currently 1 shared connection for all Camel routes
  All consumers share same Spring-AMQP ConnectionFactory and executor

My questions:

1. Why are there so few threads to consume messages via a connection? How should my application can be efficient/non-blocking when only a few threads are 'working for me'?

2. Is it acceptable and recommended from RabbitMQ's perspective to use multiple TCP connections (e.g. one per consumer or per rabbitmq queue) if isolation is required?

3. The official documentation recommends separating publisher and consumer connections to avoid flow-control interactions. Does this recommendation extend to separating multiple independent consumers into multiple connections?

4. Are there any drawbacks or risks we should consider when an application maintains 90 connections to the RabbitMQ node (each long-lived and used by exactly one consumer)?

5. Is there any rule of thumb on when to prefer:
         • one shared connection with multiple channels vs.
         • multiple connections with separate channels?
          

6. Is there any comprehensive documentation about usage of connections, channels and multithreading in typical business scenarios? I didn't find anything useful, expecially concerning the multithreading/blocking topic.

Any guidance from RabbitMQ maintainers would be greatly appreciated.
 
 
Thanks!

[acogoluegnes]

If we instead create one ConnectionFactory per route (effectively giving each consumer route its own dedicated TCP connection), this could potentially resolve the issue, as each consumer would then be isolated.

A connection factory creates connections, so you can have a 1-1 connection-to-consumer mapping with the same connection factory.

A connection factory has a shared executor used to dispatch messages to consumer. It uses it for each connection it creates, unless you create a connection with ConnectionFactory#newConnection(ExecutorService). There is a default value for this shared executor, but you can set your own. So there's plenty of solution to tweak the dispatching.

1. Why are there so few threads to consume messages via a connection? How should my application can be efficient/non-blocking when only a few threads are 'working for me'?

I guess you are using defaults, they are reasonable for most applications, but you may have to tweak your setup.

2. Is it acceptable and recommended from RabbitMQ's perspective to use multiple TCP connections (e.g. one per consumer or per rabbitmq queue) if isolation is required?

It is perfectly fine to do this. I can't say it is "recommended" because it depends on the workload.

3. The official documentation recommends separating publisher and consumer connections to avoid flow-control interactions. Does this recommendation extend to separating multiple independent consumers into multiple connections?

No.

4. Are there any drawbacks or risks we should consider when an application maintains 90 connections to the RabbitMQ node (each long-lived and used by exactly one consumer)?

It is a limited number of connections, it should be fine on most hardware, unless you run thousands of instances of this application.

5. Is there any rule of thumb on when to prefer:
   • one shared connection with multiple channels vs.
   • multiple connections with separate channels?

Channels are more lightweight and they exist to multiplex a connection. I can't answer precisely to this question, it depends on the workload.

6. Is there any comprehensive documentation about usage of connections, channels and multithreading in typical business scenarios? I didn't find anything useful, expecially concerning the multithreading/blocking topic.

There is the official documentation.

[michaelklishin]

Is there any comprehensive documentation about usage of connections, channels

• Connections guide
• Channels guide
• Tuning for a Large Number of Connections
• Dealing with High Connection Churn (and why connection churn is a condition to avoid)

We cannot really know how many connections or connections per channel your specific system would need. Avoid connection churn, channel churn, and make sure to read the above doc guides (or, in the case of the Networking guide, those two specific sections).