Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows compatibility, HAProxy latest version, recoverability, durable storage, cluster size to 5 #6

Open
wants to merge 13 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .env
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
RABBITMQ_DEFAULT_USER=guest
RABBITMQ_DEFAULT_PASS=guest
RABBITMQ_DEFAULT_VHOST=/
RABBITMQ_DEFAULT_VHOST=/
RABBITMQ_ERLANG_COOKIE=12345
1 change: 0 additions & 1 deletion .erlang.cookie

This file was deleted.

1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
*.sh text eol=lf
13 changes: 8 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@ Setup a RabbitMQ Cluster environment on your device using the pure [RabbitMQ](ht

## Quick start

```
```con
docker compose up
```

Open http://localhost:15672 to login RabbitMQ dashboard.
Open <http://localhost:15672> to login RabbitMQ dashboard.

> Username: `guest`
> Password: `guest`
Expand All @@ -24,13 +24,15 @@ Open http://localhost:15672 to login RabbitMQ dashboard.

### `docker-compose.yml`

Docker [compose](https://docs.docker.com/compose/compose-file/) config file, including 3 RabbitMQ service cluster and a HAProxy.
Docker [compose](https://docs.docker.com/compose/compose-file/) config file, including 5 RabbitMQ service cluster and a HAProxy.

| Service | Description |
| ----------- | ------------------------- |
| `rabbitmq1` | RabbitMQ (cluster) |
| `rabbitmq2` | RabbitMQ (cluster member) |
| `rabbitmq3` | RabbitMQ (cluster member) |
| `rabbitmq4` | RabbitMQ (cluster member) |
| `rabbitmq5` | RabbitMQ (cluster member) |
| `haproxy` | Load Balancer |

#### Default expose ports
Expand All @@ -39,18 +41,19 @@ Docker [compose](https://docs.docker.com/compose/compose-file/) config file, inc
| ----------------- | --------------------------------------------------- |
| `localhost:5672` | AMQP 0-9-1 and AMQP 1.0 clients |
| `localhost:15672` | HTTP API clients, management UI and `rabbitmqadmin` |
| `localhost:1936` | HAProxy statistics |

### `.env`

| Name | Default |
| ------------------------ | ------- |
| `RABBITMQ_DEFAULT_USER` | guest |
| `RABBITMQ_DEFAULT_PASS` | guest |
| `RABBITMQ_DEFAULT_VHOST` | / |
| `RABBITMQ_ERLANG_COOKIE` | 12345 |

### `.erlang.cookie`

Put your custom [Erlang Cookie](https://www.rabbitmq.com/clustering.html#erlang-cookie) inside this file (default: `12345`) for the nodes in cluster communicate with each other.
Put your custom [Erlang Cookie](https://www.rabbitmq.com/clustering.html#erlang-cookie) value in the `RABBITMQ_ERLANG_COOKIE` environment variable file (`.env`).

### `haproxy.cfg`

Expand Down
5 changes: 5 additions & 0 deletions advanced.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
[
%% five replicas by default, only makes sense for nine node clusters
{rabbit, [{quorum_cluster_size, 5},
{quorum_commands_soft_limit, 512}]}
].
17 changes: 6 additions & 11 deletions cluster-entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,20 @@

set -e

# Change .erlang.cookie permission
chmod 400 /var/lib/rabbitmq/.erlang.cookie

# Get hostname from enviromant variable
HOSTNAME=`env hostname`
echo "Starting RabbitMQ Server For host: " $HOSTNAME
/usr/local/bin/docker-entrypoint.sh rabbitmq-server &
sleep 5
rabbitmqctl wait /var/lib/rabbitmq/mnesia/rabbit\@$HOSTNAME.pid

if [ -z "$JOIN_CLUSTER_HOST" ]; then
/usr/local/bin/docker-entrypoint.sh rabbitmq-server &
sleep 5
rabbitmqctl wait /var/lib/rabbitmq/mnesia/rabbit\@$HOSTNAME.pid
else
/usr/local/bin/docker-entrypoint.sh rabbitmq-server -detached
sleep 5
if [ -n "$JOIN_CLUSTER_HOST" ]; then
echo "Starting RabbitMQ Server For host: " $HOSTNAME
rabbitmqctl wait /var/lib/rabbitmq/mnesia/rabbit\@$HOSTNAME.pid
rabbitmqctl stop_app
rabbitmqctl join_cluster rabbit@$JOIN_CLUSTER_HOST
rabbitmqctl start_app
fi

# Keep foreground process active ...
tail -f /dev/null
sleep infinity
82 changes: 77 additions & 5 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,43 +1,97 @@
version: '3'
services:
rabbitmq1:
image: rabbitmq:3-management
restart: unless-stopped
hostname: rabbitmq1
environment:
- RABBITMQ_DEFAULT_USER=${RABBITMQ_DEFAULT_USER}
- RABBITMQ_DEFAULT_PASS=${RABBITMQ_DEFAULT_PASS}
- RABBITMQ_DEFAULT_VHOST=${RABBITMQ_DEFAULT_VHOST}
- RABBITMQ_ERLANG_COOKIE=${RABBITMQ_ERLANG_COOKIE}
volumes:
- ./.erlang.cookie:/var/lib/rabbitmq/.erlang.cookie
- ./advanced.config:/etc/rabbitmq/advanced.config
- ./cluster-entrypoint.sh:/usr/local/bin/cluster-entrypoint.sh
- rabbitmq1-data:/var/lib/rabbitmq
- rabbitmq1-logs:/var/log/rabbitmq
entrypoint: /usr/local/bin/cluster-entrypoint.sh
networks:
- rabbitmq

rabbitmq2:
image: rabbitmq:3-management
restart: unless-stopped
hostname: rabbitmq2
depends_on:
- rabbitmq1
environment:
- JOIN_CLUSTER_HOST=rabbitmq1
- RABBITMQ_ERLANG_COOKIE=${RABBITMQ_ERLANG_COOKIE}
volumes:
- ./.erlang.cookie:/var/lib/rabbitmq/.erlang.cookie
- ./advanced.config:/etc/rabbitmq/advanced.config
- ./cluster-entrypoint.sh:/usr/local/bin/cluster-entrypoint.sh
- rabbitmq2-data:/var/lib/rabbitmq
- rabbitmq2-logs:/var/log/rabbitmq
entrypoint: /usr/local/bin/cluster-entrypoint.sh
networks:
- rabbitmq

rabbitmq3:
image: rabbitmq:3-management
restart: unless-stopped
hostname: rabbitmq3
depends_on:
- rabbitmq1
environment:
- JOIN_CLUSTER_HOST=rabbitmq1
- RABBITMQ_ERLANG_COOKIE=${RABBITMQ_ERLANG_COOKIE}
volumes:
- ./.erlang.cookie:/var/lib/rabbitmq/.erlang.cookie
- ./advanced.config:/etc/rabbitmq/advanced.config
- ./cluster-entrypoint.sh:/usr/local/bin/cluster-entrypoint.sh
- rabbitmq3-data:/var/lib/rabbitmq
- rabbitmq3-logs:/var/log/rabbitmq
entrypoint: /usr/local/bin/cluster-entrypoint.sh
networks:
- rabbitmq

rabbitmq4:
image: rabbitmq:3-management
restart: unless-stopped
hostname: rabbitmq4
depends_on:
- rabbitmq1
environment:
- JOIN_CLUSTER_HOST=rabbitmq1
- RABBITMQ_ERLANG_COOKIE=${RABBITMQ_ERLANG_COOKIE}
volumes:
- ./advanced.config:/etc/rabbitmq/advanced.config
- ./cluster-entrypoint.sh:/usr/local/bin/cluster-entrypoint.sh
- rabbitmq4-data:/var/lib/rabbitmq
- rabbitmq4-logs:/var/log/rabbitmq
entrypoint: /usr/local/bin/cluster-entrypoint.sh
networks:
- rabbitmq

rabbitmq5:
image: rabbitmq:3-management
restart: unless-stopped
hostname: rabbitmq5
depends_on:
- rabbitmq1
environment:
- JOIN_CLUSTER_HOST=rabbitmq1
- RABBITMQ_ERLANG_COOKIE=${RABBITMQ_ERLANG_COOKIE}
volumes:
- ./advanced.config:/etc/rabbitmq/advanced.config
- ./cluster-entrypoint.sh:/usr/local/bin/cluster-entrypoint.sh
- rabbitmq5-data:/var/lib/rabbitmq
- rabbitmq5-logs:/var/log/rabbitmq
entrypoint: /usr/local/bin/cluster-entrypoint.sh
networks:
- rabbitmq

haproxy:
image: haproxy:1.7
image: haproxy:3.0
restart: unless-stopped
volumes:
- ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
depends_on:
Expand All @@ -47,3 +101,21 @@ services:
ports:
- 15672:15672
- 5672:5672
- 1936:1936
networks:
- rabbitmq

networks:
rabbitmq:

volumes:
rabbitmq1-data:
rabbitmq1-logs:
rabbitmq2-data:
rabbitmq2-logs:
rabbitmq3-data:
rabbitmq3-logs:
rabbitmq4-data:
rabbitmq4-logs:
rabbitmq5-data:
rabbitmq5-logs:
18 changes: 11 additions & 7 deletions haproxy.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
global
log 127.0.0.1 local1
log stdout format raw local0
maxconn 4096

defaults
Expand Down Expand Up @@ -28,9 +28,11 @@ listen rabbitmq
timeout client 3h
timeout server 3h
option clitcpka
server rabbitmq1 rabbitmq1:5672 check inter 5s rise 2 fall 3
server rabbitmq2 rabbitmq2:5672 check inter 5s rise 2 fall 3
server rabbitmq3 rabbitmq3:5672 check inter 5s rise 2 fall 3
server rabbitmq1 rabbitmq1:5672 check inter 200ms rise 1 fall 1
server rabbitmq2 rabbitmq2:5672 check inter 200ms rise 1 fall 1
server rabbitmq3 rabbitmq3:5672 check inter 200ms rise 1 fall 1
server rabbitmq4 rabbitmq4:5672 check inter 200ms rise 1 fall 1
server rabbitmq5 rabbitmq5:5672 check inter 200ms rise 1 fall 1

listen mgmt
bind *:15672
Expand All @@ -39,6 +41,8 @@ listen mgmt
timeout client 3h
timeout server 3h
option clitcpka
server rabbitmq1 rabbitmq1:15672 check inter 5s rise 2 fall 3
server rabbitmq2 rabbitmq2:15672 check inter 5s rise 2 fall 3
server rabbitmq3 rabbitmq3:15672 check inter 5s rise 2 fall 3
server rabbitmq1 rabbitmq1:15672 check inter 200ms rise 1 fall 1
server rabbitmq2 rabbitmq2:15672 check inter 200ms rise 1 fall 1
server rabbitmq3 rabbitmq3:15672 check inter 200ms rise 1 fall 1
server rabbitmq4 rabbitmq4:15672 check inter 200ms rise 1 fall 1
server rabbitmq5 rabbitmq5:15672 check inter 200ms rise 1 fall 1