200 lines
8.0 KiB
Markdown
200 lines
8.0 KiB
Markdown
# HAProxy Docker Compose files
|
|
|
|
Docker Compose files to spin up an instance of HAProxy.
|
|
|
|
# How to run
|
|
|
|
Add a `COMPOSE_ENV` file and save its location as a shell variable along with the location where this repo lives, here for example `/opt/containers/haproxy` plus all other variables. At [env/fqdn_context.env.example](env/fqdn_context.env.example) you'll find an example environment file.
|
|
|
|
When everything's ready start HAProxy with Docker Compose, otherwise head down to [Initial setup](#initial-setup) first.
|
|
|
|
## Environment
|
|
```
|
|
export COMPOSE_DIR='/opt/containers/haproxy'
|
|
export COMPOSE_CTX='ux_vilnius'
|
|
export COMPOSE_PROJECT='haproxy-'"${COMPOSE_CTX}"
|
|
export COMPOSE_FILE="${COMPOSE_DIR}"'/docker-compose.yml'
|
|
export COMPOSE_ENV=<add accordingly>
|
|
```
|
|
|
|
## Context
|
|
|
|
On your deployment machine create the necessary Docker context to connect to and control the Docker daemon on whatever target host you'll be using, for example:
|
|
```
|
|
docker context create fully.qualified.domain.name --docker 'host=ssh://root@fully.qualified.domain.name'
|
|
```
|
|
|
|
## Start
|
|
|
|
```
|
|
docker --context 'fully.qualified.domain.name' compose --project-name "${COMPOSE_PROJECT}" --file "${COMPOSE_FILE}" --env-file "${COMPOSE_ENV}" up --detach
|
|
```
|
|
|
|
## Reload
|
|
|
|
HAProxy lends itself well to graceful reloads in a similar fashion to Nginx. Considering a running HAProxy container named `containername` check config file syntax:
|
|
|
|
```
|
|
docker exec -i containername haproxy -c -f /usr/local/etc/haproxy/haproxy.cfg
|
|
```
|
|
|
|
Reload HAProxy
|
|
|
|
```
|
|
docker kill --signal SIGHUP containername
|
|
```
|
|
|
|
# Initial setup
|
|
|
|
We're assuming you run Docker Compose workloads with ZFS-based bind mounts. ZFS management, creating a zpool and setting adequate properties for its datasets is out of scope of this document.
|
|
|
|
## Datasets
|
|
|
|
Create ZFS datasets.
|
|
|
|
* Parent dateset
|
|
```
|
|
zfs create -o mountpoint=/opt/docker-data 'zpool/docker-data'
|
|
```
|
|
|
|
* Container-specific dataset
|
|
```
|
|
zfs create -p 'zpool/docker-data/haproxy-'"${COMPOSE_CTX}"'/haproxy/config'
|
|
```
|
|
|
|
* Create a subdir for certificates
|
|
```
|
|
mkdir -p '/opt/docker-data/haproxy-'"${COMPOSE_CTX}"'/haproxy/config/certs'
|
|
```
|
|
|
|
A change in file or directory ownership is typically not needed. HAProxy runs by default as user ID 99 and group ID 99, it has no problem accessing a bind-mounted file owned by `root:root` with typical `0644` (`rw-r--r--`) permissions.
|
|
|
|
## Additional files
|
|
|
|
Place the following files on target server. Use the directory structure at [build-context](build-context) as a guide, specifically at `docker-data`.
|
|
|
|
```
|
|
build-context/
|
|
├── docker-data
|
|
│ └── config
|
|
│ ├── certs
|
|
│ │ ├── example.com.pem
|
|
│ │ ├── example.com.pem.key
|
|
│ │ ├── example.net.pem
|
|
│ │ └── example.net.pem.key
|
|
│ ├── haproxy.cfg
|
|
│ └── hosts.map
|
|
├── Dockerfile
|
|
└── extras
|
|
```
|
|
|
|
In our example config (see next section [HAProxy config example](#haproxy-config-example)) each certificate is stored in what HAProxy calls a cert bundle: one complete PEM-formatted plain text SSL certificate chain file `something.pem` plus its matching key named `something.pem.key`. The chain file from top to bottom contains domain certificate, optionally intermediate certificate and lastly Certificate Authority certificate. With this naming scheme HAProxy finds keys and matching certs automatically.
|
|
|
|
When done head back up to [How to run](#how-to-run).
|
|
|
|
# HAProxy config example
|
|
|
|
Proper config for your HAProxy use case is beyond the scope of this document. The example [build-context/docker-data/config/haproxy.cfg](build-context/docker-data/config/haproxy.cfg) file is a courtesy config to get you started. Your real world file will likely turn out differently.
|
|
|
|
Here's a quick rundown of what you see in the example file.
|
|
|
|
## Section `global`
|
|
|
|
```
|
|
global
|
|
daemon
|
|
maxconn 200
|
|
log stdout format raw local0
|
|
ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
|
|
ssl-default-bind-options prefer-client-ciphers no-tls-tickets ssl-min-ver TLSv1.3
|
|
ssl-default-server-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
|
|
ssl-default-server-options no-tls-tickets ssl-min-ver TLSv1.3
|
|
```
|
|
|
|
We follow recommendations from [Mozilla's SSL Configuration Generator](https://ssl-config.mozilla.org/) for SSL settings. The container outputs logging data to `stdout`. We're setting a global connection limit of `maxconn 200`. Keeping some limit in place makes HAProxy behavior predictable, it doesn't have to be this low. This argument goes hand in hand with the `ulimits` directive in [docker-compose.yml](docker-compose.yml).
|
|
|
|
```
|
|
...
|
|
...
|
|
ulimits:
|
|
nproc: ${ULIMIT_NPROC-65535}
|
|
nofile:
|
|
soft: ${ULIMIT_NPROC-65535}
|
|
hard: ${ULIMIT_NPROC-65535}
|
|
```
|
|
|
|
If you don't specify a sensible limit HAProxy will try and allocate whatever your platform allows.
|
|
|
|
## Sections `defaults`
|
|
|
|
```
|
|
defaults generic_defaults
|
|
log global
|
|
maxconn 50
|
|
|
|
# Irrelevant on frontends
|
|
timeout connect 5s
|
|
timeout server 5s
|
|
|
|
# Irrelevant on backends
|
|
timeout client 30s
|
|
|
|
defaults http_defaults from generic_defaults
|
|
mode http
|
|
option forwardfor
|
|
option httplog
|
|
```
|
|
|
|
Set generic defaults and supplement them with HTTP-specific defaults.
|
|
|
|
## Statistics
|
|
|
|
```
|
|
frontend fe_stats from http_defaults
|
|
bind :"${STATS_PORT-61000}"
|
|
stats enable
|
|
stats uri /
|
|
stats refresh 10s
|
|
acl ips_allowed src 10.10.10.0/24 10.10.1.0/24
|
|
http-request deny if !ips_allowed
|
|
```
|
|
|
|
Enable statistics output. Access to this endpoint is permitted only from `acl ips_allowed` which can be for example network prefixes in your local network.
|
|
|
|
## A proxy example
|
|
|
|
```
|
|
frontend fe_https-proxy from http_defaults
|
|
bind :80
|
|
bind :443 ssl strict-sni crt /usr/local/etc/haproxy/certs alpn h2,http/1.1
|
|
redirect scheme https code 301 if !{ ssl_fc }
|
|
http-response set-header Strict-Transport-Security max-age=63072000
|
|
use_backend %[req.hdr(host),lower,map_dom(/usr/local/etc/haproxy/hosts.map,be_no-match)]
|
|
```
|
|
|
|
We use a `hosts.map` file to map HTTP `host` header information for incoming requests to the backend that should serve them. Check out [build-context/docker-data/config/hosts.map](build-context/docker-data/config/hosts.map) for an example file.
|
|
|
|
Also HTTP traffic gets redirected to HTTPS and we specify more best practices suggested by [Mozilla's SSL Configuration Generator](https://ssl-config.mozilla.org/).
|
|
|
|
```
|
|
backend be_example.net from http_defaults
|
|
compression algo gzip
|
|
compression type text/plain text/css
|
|
option httpchk
|
|
http-check send meth HEAD uri /status hdr Host example.net hdr User-Agent "${HEALTH_CHECK_USER_AGENT-HAProxy} health check"
|
|
server example.net example.net:8080 check inter 10s fall 3 rise 3
|
|
```
|
|
|
|
In our backend example we do HTTP health checks by sending a `HEAD` request every 10 seconds. When doing so we identify ourselves as an HAProxy instance so the downstream application can recognize legitimate health checks in its logs.
|
|
|
|
Note that both `frontend` and `backend` inherit settings from the `http_defaults` section defined earlier.
|
|
|
|
## No match
|
|
|
|
```
|
|
backend be_no-match from http_defaults
|
|
http-request deny deny_status 404
|
|
```
|
|
|
|
To requests without a match we serve an HTTP status code 404.
|