Kannisto.org

TLS-secured RabbitMQ as Docker container

The main purpose of the blog posts is to persist some instructions I have written for myself. However, I'm happy if someone else finds these beneficial too.

DISCLAIMER. The content is provided as is. Absolutely no warranty of any kind.

– Petri Kannisto

DISCLAIMER. This guide may contain practices that differ from what tough Docker professionals do.

Introduction

This guide shows how to setup a TLS-secured RabbitMQ as a Docker container. The motivation came as it it appeared somewhat difficult to upgrade an existing RabbitMQ installation. The instructions provided by RabbitMQ seemed insufficient to me as the actual upgrade part just said "upgrade" with no further elaboration. There is also a strong dependency with the Erlang version, and you cannot upgrade Erlang without upgrading RabbitMQ and vice versa. Meanwhile, you should also upgrade the operating system sometimes. Therefore, Docker helps in upgrading, as the service can be rebuilt from a newer base image as needed.

To set up RabbitMQ without Docker, see this post.

After completing this setup, you have a RabbitMQ server with the following features:

This guide assumes the following:

While the setup is straightforward, multiple steps are necessary. I had to do some googling to figure out what to do.

Alternatives

There is already a Github project that generates self-signed certificates for RabbitMQ running in Docker (see https://github.com/roboconf/rabbitmq-with-ssl-in-docker ). I wanted to document the process on my own and also to use pre-generated certificates.

Furthermore, instead of configuring RabbitMQ to directly use TLS, you could set up a termination proxy. This would simplify the configuration. See https://en.wikipedia.org/wiki/TLS_termination_proxy .

RabbitMQ and Docker files

First, this section contains some explanation. Then, the actual files will come.

Dockerfile, Docker Compose file, and other setup

I chose to split the creation of my RabbitMQ service into a Dockerfile and Docker Compose file. At least the Compose file is unnecessary, but it enables you to write a shorter start command when exposing ports.

Regarding this part, the difficulty comes from first letting RabbitMQ to start and only then configuring the users. This can be achieved with a shell script that waits for the service to start; see https://stackoverflow.com/questions/30747469/how-to-add-initial-users-when-starting-a-rabbitmq-docker-container

The format of the configuration file has changed from the older versions. You can see some examples here: https://github.com/rabbitmq/rabbitmq-server/blob/v3.8.x/deps/rabbit/docs/rabbitmq.conf.example . This page is the respective RabbitMQ documentation: https://www.rabbitmq.com/configure.html .

Set up logging

The default RabbitMQ image provides its log output to Docker output streams. This differs from the basic RabbitMQ that uses a log file.

The basic way to access the output from a container is to call docker logs, but this is problematic if the container fails. The logs command doesn't work when the container is stopped. There are another means to access logs from a stopped container, but if you accidentally remove the container, the logs are gone. Furthermore, the logs will be cleared each time the container starts. Thus, to guarantee the availability of logs in case of a problem, another approach is necessary.

You could set up another logging scheme with Docker, but I wanted to save time as these were unfamiliar to me. I wanted to stick with a simple file-based approach.

Fortunately, the logging behavior of RabbitMQ can be configured. I had to google for long to find how this is controlled in the most recent Docker image. Even adding an explicit configuration item wasn't enough but I had to also remove the default config, as my config didn't seem to override the default. In the end, I chose to put logs into the subfolder rmqlogs located in the same folder as the Docker Compose file. This would happen with a bind mount in the compose file.

The files

This guide assumes the following directory structure:

rabbitmq-tls [DIR]
- certs [DIR]
  - rootca.pem (root CA file)
  - server.key (private key for the server)
  - server.pem (server certificate)
- rmqlogs [DIR]
- compose.yaml.yml
- Dockerfile
- init.sh
- rabbitmq.conf

compose.yaml

services:
  rabbitmq:
    build: .
    ports:
      - "5671:5671"
    volumes:
      - "./rmqlogs:/home/rmqlogs"

Dockerfile

# Not using "latest" in case a newer version would break something
FROM rabbitmq:3.10

# This explicit definition is required to monitor start in the init script later
ENV RABBITMQ_PID_FILE /var/lib/rabbitmq/mnesia/rabbitmq

# Set environment variable for log location
ENV RABBITMQ_LOG_BASE /home/rmqlogs

# Create logs folder and remove the default config to enable file logging.
# 'rabbitmq.conf' did not seem to override the default that logs only to console.
RUN mkdir -p /home/rmqlogs \
 && rm /etc/rabbitmq/conf.d/10-defaults.conf

# Update package repos
RUN apt-get update

# Copy key and certificate files
COPY cert /home/cert

# Copy config file
COPY rabbitmq.conf /etc/rabbitmq/rabbitmq.conf

# Copy init script
RUN mkdir -p /home/myhome
COPY init.sh /home/myhome
RUN chmod +x /home/myhome/init.sh

# Expose ports and initialize
EXPOSE 5671
CMD ["/home/myhome/init.sh"]

init.sh

#!/bin/sh

# This first part is run only after the server has been started up!
# Monitoring for the PID file to appear before setting up the users.
# See https://stackoverflow.com/questions/30747469/how-to-add-initial-users-when-starting-a-rabbitmq-docker-container
( rabbitmqctl wait --timeout 60 $RABBITMQ_PID_FILE ; \
rabbitmqctl change_password guest SomePasswordNotIntendedToBeUsed ; \
rabbitmqctl add_user localadmin MyAdminPassword 2>/dev/null ; \
rabbitmqctl add_user testuser MyGoodPassword 2>/dev/null ; \
rabbitmqctl set_user_tags localadmin administrator ; \
rabbitmqctl set_permissions localadmin ".*" ".*" ".*" ; \
rabbitmqctl set_permissions testuser "^amq.*|^testuser.*" "^amq.*|^testuser.*" "^amq.*|^testuser.*" ) &

# Start!
rabbitmq-server

rabbitmq.conf

# Enable connection encryption with TLS
listeners.ssl.default = 5671

# Disable non-encrypted connections
listeners.tcp = none

# Only enable the most recent TLS versions due to vulnerabilities in the older ones
ssl_options.versions.1 = tlsv1.3
ssl_options.versions.2 = tlsv1.2

# Certificate and key file locations
ssl_options.cacertfile = /home/cert/rootca.pem
ssl_options.certfile   = /home/cert/server.pem
ssl_options.keyfile    = /home/cert/server.key

# These users can only connect locally. 'guest' is the default user.
loopback_users.guest = true
loopback_users.localadmin = true

# Log to file, not console (this defaults to true in RabbitMQ Docker image)
log.console = false

Set up automatic updates for Docker images

For security, you want to set up automatic updates. I chose an approach that's easy and simple but not necessary scalable. Anyway, it works, checking regularly if the RabbitMQ Docker image has been updated.

First, create a shell script to run the update. This will be called refresh_do.sh. Please note that the image name will differ from rabbitmq-tls_rabbitmq at least if the root folder of your Docker files has a different name.

#!/bin/bash
echo ------------------------------------------------------
echo Docker image refresh starting at $(date)
docker compose stop
docker compose rm -f
docker image rm rabbitmq-tls_rabbitmq
docker compose up -d
echo Docker image refresh finished at $(date)
echo ------------------------------------------------------

Next, create another script to choose the correct working directory and redirect the output into a file called refresh_log.txt. The script is called refresh.sh and should be located in the same folder as the previous script.

#!/bin/bash
# cd to the dir of this script
cd "$(dirname "$0")";
# Echo stdout and stderr to file
/bin/bash refresh_do.sh &>> refresh_log.txt

You should have the following files in the end. You can choose the location, but the two scripts should be in the same directory.

Remember to make the shell scripts executable:

chmod +x refresh.sh
chmod +x refresh_do.sh

It's also a good idea to try now if the refresh script works. Once you run this, the related logs should be generated in refresh_log.txt

./refresh.sh

Finally, create a Cron task to automatically run the script every night. Give crontab -e in the terminal. This will open a text editor that lets you schedule tasks. The line below shows how to run the update every night at 3 a.m. Please note that this will cause an outage!

0 3 * * * /path/to/your/script/refresh.sh

Alternatively, to run only every Monday at 3 a.m.:

0 3 * * mon /path/to/your/script/refresh.sh

Cron will keep running even after you log out. See https://unix.stackexchange.com/questions/197615/does-a-job-scheduled-in-crontab-run-even-when-i-log-out

Verify TLS connectivity

If you want to enable trying if the certificates work even locally without DNS, you must edit the hosts file. Let us assume your certificate has mydomain.com as the domain. To experiment locally, you'd add the following line into /etc/hosts:

127.0.0.1 mydomain.com

Once you are done with the experiments, remember to remove this entry from the hosts file!

Once I had the message bus running, I used trivial NodeJs clients to see if the messages travel as expected (see the code below). Make sure the username and password match the ones you use!

If you want to verify that the server only accepts TLS-enabled connections, edit the URLs to contain amqp instead amqps and 5672 for the port (default). With these values, the connection should fail.

Data producer

var amqp = require("amqplib/callback_api");

console.log("Producer");
console.log("Setting up AMQP connection...");

var amqpOpts = {};

// Setting up an AQMP listener
amqp.connect('amqps://testuser:MyGoodPassword@mydomain.com:5671', amqpOpts, function(err0, conn)
{
    // Topics:
    // https://www.rabbitmq.com/tutorials/tutorial-five-javascript.html

    console.log("Creating AMQP channel...");

    conn.createChannel(function(err1, channel)
    {
        var exchangeName = 'testuser';
        var exchangeType = "topic";
        var topicKey = "testuser.topic1";
        var msg = new Date().toISOString();

        channel.assertExchange(exchangeName, exchangeType, {durable: false});

        channel.publish(exchangeName, topicKey, Buffer.from(msg));
        console.log("--- Sent to %s: '%s'", topicKey, msg);
    });

    // TODO: How to close the connection properly?
});

Data consumer

var amqp = require("amqplib/callback_api");

console.log("Consumer");
console.log("Setting up AMQP connection...");

var amqpOpts = {};

// Setting up an AQMP listener
amqp.connect('amqps://testuser:MyGoodPassword@mydomain.com:5671', amqpOpts, function(err0, conn)
{
    // Topics:
    // https://www.rabbitmq.com/tutorials/tutorial-five-javascript.html

    console.log("Creating AMQP channel...");

    conn.createChannel(function(err1, channel)
    {
        var exchangeName = 'testuser';
        var exchangeType = "topic";
        var topicKey = "testuser.topic1";

        channel.assertExchange(exchangeName, exchangeType, {durable: false});

        channel.assertQueue('', {exclusive: true}, function(err2, q)
        {
            // Binding to a queue
            channel.bindQueue(q.queue, exchangeName, topicKey);

            console.log('--- Waiting for messages. Press CTRL+C to exit.');

            channel.consume(q.queue, function(msg)
            {
                // Printing the schedule
                console.log("Received:" + msg.content.toString());
            },
            {noAck: true});
        });
    });

    // TODO: How to close the connection properly?
});