HAProxy Docker Compose files

Docker Compose files to spin up an instance of HAProxy.

How to run

Add a COMPOSE_ENV file and save its location as a shell variable along with the location where this repo lives, here for example /opt/containers/haproxy plus all other variables. At env/fqdn_context.env.example you'll find an example environment file.

When everything's ready start HAProxy with Docker Compose, otherwise head down to Initial setup first.

Environment

export COMPOSE_DIR='/opt/containers/haproxy'
export COMPOSE_CTX='ux_vilnius'
export COMPOSE_PROJECT='haproxy-'"${COMPOSE_CTX}"
export COMPOSE_FILE="${COMPOSE_DIR}"'/docker-compose.yml'
export COMPOSE_ENV=<add accordingly>

Context

On your deployment machine create the necessary Docker context to connect to and control the Docker daemon on whatever target host you'll be using, for example:

docker context create fully.qualified.domain.name --docker 'host=ssh://root@fully.qualified.domain.name'

Start

docker --context 'fully.qualified.domain.name' compose --project-name "${COMPOSE_PROJECT}" --file "${COMPOSE_FILE}" --env-file "${COMPOSE_ENV}" up --detach

Reload

HAProxy lends itself well to graceful reloads in a similar fashion to Nginx. Considering a running HAProxy container named containername check config file syntax:

docker exec -i containername haproxy -c -f /usr/local/etc/haproxy/haproxy.cfg

Reload HAProxy

docker kill --signal SIGHUP containername

Initial setup

We're assuming you run Docker Compose workloads with ZFS-based bind mounts. ZFS management, creating a zpool and setting adequate properties for its datasets is out of scope of this document.

Datasets

Create ZFS datasets.

• Parent dateset

  zfs create -o mountpoint=/opt/docker-data 'zpool/docker-data'
  

• Container-specific dataset

  zfs create -p 'zpool/docker-data/haproxy-'"${COMPOSE_CTX}"'/haproxy/config'
  

• Create a subdir for certificates

  mkdir -p '/opt/docker-data/haproxy-'"${COMPOSE_CTX}"'/haproxy/config/certs'
  

A change in file or directory ownership is typically not needed. HAProxy runs by default as user ID 99 and group ID 99, it has no problem accessing a bind-mounted file owned by root:root with typical 0644 (rw-r--r--) permissions.

Additional files

Place the following files on target server. Use the directory structure at build-context as a guide, specifically at docker-data.

build-context/
├── docker-data
│   └── config
│       ├── certs
│       │   ├── example.com.pem
│       │   ├── example.com.pem.key
│       │   ├── example.net.pem
│       │   └── example.net.pem.key
│       ├── haproxy.cfg
│       └── hosts.map
├── Dockerfile
└── extras

In our example config (see next section HAProxy config example) each certificate is stored in what HAProxy calls a cert bundle: one complete PEM-formatted plain text SSL certificate chain file something.pem plus its matching key named something.pem.key. The chain file from top to bottom contains domain certificate, optionally intermediate certificate and lastly Certificate Authority certificate. With this naming scheme HAProxy finds keys and matching certs automatically.

When done head back up to How to run.

HAProxy config example

Proper config for your HAProxy use case is beyond the scope of this document. The example build-context/docker-data/config/haproxy.cfg file is a courtesy config to get you started. Your real world file will likely turn out differently.

Here's a quick rundown of what you see in the example file.

Section global

global
    daemon
    maxconn                         200
    log                             stdout format raw local0
    ssl-default-bind-ciphersuites   TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
    ssl-default-bind-options        prefer-client-ciphers no-tls-tickets ssl-min-ver TLSv1.3
    ssl-default-server-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
    ssl-default-server-options      no-tls-tickets ssl-min-ver TLSv1.3

We follow recommendations from Mozilla's SSL Configuration Generator for SSL settings. The container outputs logging data to stdout. We're setting a global connection limit of maxconn 200. Keeping some limit in place makes HAProxy behavior predictable, it doesn't have to be this low. This argument goes hand in hand with the ulimits directive in docker-compose.yml.

...
    ...
        ulimits:
            nproc: ${ULIMIT_NPROC-65535}
            nofile:
                soft: ${ULIMIT_NPROC-65535}
                hard: ${ULIMIT_NPROC-65535}

If you don't specify a sensible limit HAProxy will try and allocate whatever your platform allows.

Sections defaults

defaults generic_defaults
    log                             global
    maxconn                         50

    # Irrelevant on frontends
    timeout                         connect 5s
    timeout                         server  5s

    # Irrelevant on backends
    timeout                         client 30s

defaults http_defaults from generic_defaults
    mode                            http
    option                          forwardfor
    option                          httplog

Set generic defaults and supplement them with HTTP-specific defaults.

Statistics

frontend fe_stats from http_defaults
    bind                            :"${STATS_PORT-61000}"
    stats                           enable
    stats                           uri /
    stats                           refresh 10s
    acl ips_allowed                 src 10.10.10.0/24 10.10.1.0/24
    http-request                    deny if !ips_allowed

Enable statistics output. Access to this endpoint is permitted only from acl ips_allowed which can be for example network prefixes in your local network.

A proxy example

frontend fe_https-proxy from http_defaults
    bind                            :80
    bind                            :443 ssl strict-sni crt /usr/local/etc/haproxy/certs alpn h2,http/1.1
    redirect                        scheme https code 301 if !{ ssl_fc }
    http-response                   set-header Strict-Transport-Security max-age=63072000
    use_backend                     %[req.hdr(host),lower,map_dom(/usr/local/etc/haproxy/hosts.map,be_no-match)]

We use a hosts.map file to map HTTP host header information for incoming requests to the backend that should serve them. Check out build-context/docker-data/config/hosts.map for an example file.

Also HTTP traffic gets redirected to HTTPS and we specify more best practices suggested by Mozilla's SSL Configuration Generator.

backend be_example.net from http_defaults
    compression                     algo gzip
    compression                     type text/plain text/css
    option                          httpchk
    http-check                      send meth HEAD uri /status hdr Host example.net hdr User-Agent "${HEALTH_CHECK_USER_AGENT-HAProxy} health check"
    server                          example.net example.net:8080 check inter 10s fall 3 rise 3

In our backend example we do HTTP health checks by sending a HEAD request every 10 seconds. When doing so we identify ourselves as an HAProxy instance so the downstream application can recognize legitimate health checks in its logs.

Note that both frontend and backend inherit settings from the http_defaults section defined earlier.

No match

backend be_no-match from http_defaults
    http-request                    deny deny_status 404

To requests without a match we serve an HTTP status code 404.