diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..723ef36 --- /dev/null +++ b/.gitignore @@ -0,0 +1 @@ +.idea \ No newline at end of file diff --git a/README.md b/README.md index 7d1b0e1..8c3ea10 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,125 @@ -# paperless-ngx +# paperless-ngx Docker Compose files +Docker Compose files to spin up an instance of paperless-ngx. + +# How to run + +Add a `COMPOSE_ENV` file and save its location as a shell variable along with the location where this repo lives, here for example `/opt/containers/paperless_ngx` plus all other variables. At [env/fqdn_context.env.example](env/fqdn_context.env.example) you'll find an example environment file. + +When everything's ready start paperless-ngx with Docker Compose, otherwise head down to [Initial setup](#initial-setup) first. + +## Environment + +We're assuming that the upstream Git repo [github.com/paperless-ngx/paperless-ngx](https://github.com/paperless-ngx/paperless-ngx) is checked out locally. In our example we're assuming it's living at `/opt/git/github.com/paperless-ngx/paperless-ngx/tags/latest`. The repo contains multiple Docker Compose files in [docker/compose](https://github.com/paperless-ngx/paperless-ngx/tree/dev/docker/compose) dir, we'll be using one of them as our baseline. + +``` +export UPSTREAM_REPO_DIR='/opt/git/github.com/paperless-ngx/paperless-ngx/tags/latest' +export UPSTREAM_COMPOSE_FILE="${UPSTREAM_REPO_DIR%/}"'/docker/compose/docker-compose.postgres-tika.yml' +export COMPOSE_CTX='ux_vilnius' +export COMPOSE_PROJECT='paperless_ngx-'"${COMPOSE_CTX}" +export COMPOSE_PROJECT_DIR='/opt/containers/paperless_ngx' +export COMPOSE_OVERRIDE="${COMPOSE_PROJECT_DIR%/}"'/compose.override.yaml' +export COMPOSE_COMMON_SETTINGS="${COMPOSE_PROJECT_DIR%/}"'/common-settings.yaml' +export COMPOSE_ENV= +export PAPERLESS_NGX_VERSION='2.14.7' +``` + +## Prep upstream Git repo + +``` +git -C "${UPSTREAM_REPO_DIR}" reset --hard origin +git -C "${UPSTREAM_REPO_DIR}" checkout dev +git -C "${UPSTREAM_REPO_DIR}" pull +git -C "${UPSTREAM_REPO_DIR}" checkout 'v'"${PAPERLESS_NGX_VERSION}" +``` + +## Context + +On your deployment machine create the necessary Docker context to connect to and control the Docker daemon on whatever target host you'll be using, for example: +``` +docker context create fully.qualified.domain.name --docker 'host=ssh://root@fully.qualified.domain.name' +``` + +## Pull + +Pull images from Docker Hub verbatim. + +``` +docker compose --project-name "${COMPOSE_PROJECT}" --file "${UPSTREAM_COMPOSE_FILE}" --file "${COMPOSE_OVERRIDE}" --env-file "${COMPOSE_ENV}" pull +``` + +## Copy to target + +Copy images to target Docker host, that is assuming you deploy to a machine that itself has no network route to reach Docker Hub or your private registry of choice. Copying in its simplest form involves a local `docker save` and a remote `docker load`. Consider the helper mini-project [quico.space/Quico/copy-docker](https://quico.space/Quico/copy-docker) where [copy-docker.sh](https://quico.space/Quico/copy-docker/src/branch/main/copy-docker.sh) allows the following workflow: + +``` +while IFS= read -u 10 -r image; do + copy-docker "${image}" fully.qualified.domain.name +done 10< <(docker compose --project-name "${COMPOSE_PROJECT}" --file "${UPSTREAM_COMPOSE_FILE}" --file "${COMPOSE_OVERRIDE}" --env-file "${COMPOSE_ENV}" config | grep -Pi -- 'image:' | awk '{print $2}' | sort | uniq) +``` + +## Start + +``` +docker --context 'fully.qualified.domain.name' compose --project-name "${COMPOSE_PROJECT}" --file "${UPSTREAM_COMPOSE_FILE}" --file "${COMPOSE_OVERRIDE}" --env-file "${COMPOSE_ENV}" up --detach +``` + +# Initial setup + +We're assuming you run Docker Compose workloads with ZFS-based bind mounts. ZFS management, creating a zpool and setting adequate properties for its datasets is out of scope of this document. + +## Datasets + +Create ZFS datasets and set permissions as needed. + +* Parent dateset + ``` + export "$(grep -Pi -- '^CONTEXT=' "${COMPOSE_ENV}")" + zfs create -o canmount=off zpool/data/opt + zfs create -o mountpoint=/opt/docker-data zpool/data/opt/docker-data + ``` + +* Container-specific datasets + ``` + zfs create -p 'zpool/data/opt/docker-data/paperless_ngx-'"${COMPOSE_CTX}"'/broker/data' + zfs create -p 'zpool/data/opt/docker-data/paperless_ngx-'"${COMPOSE_CTX}"'/db/data' + zfs create -p 'zpool/data/opt/docker-data/paperless_ngx-'"${COMPOSE_CTX}"'/webserver/consume' + zfs create -p 'zpool/data/opt/docker-data/paperless_ngx-'"${COMPOSE_CTX}"'/webserver/data' + zfs create -p 'zpool/data/opt/docker-data/paperless_ngx-'"${COMPOSE_CTX}"'/webserver/export' + zfs create -p 'zpool/data/opt/docker-data/paperless_ngx-'"${COMPOSE_CTX}"'/webserver/media' + ``` + +* Change ownership + ``` + chown -R 999:999 '/opt/docker-data/paperless_ngx-'"${COMPOSE_CTX}"'/'{'broker','db'}'/'* + chown -R 1000:1000 '/opt/docker-data/paperless_ngx-'"${COMPOSE_CTX}"'/webserver/'* + ``` + +## Additional files + +No additional files are needed to get started. + +Head back up to [How to run](#how-to-run). + +# Development + +## Conventional commits + +This project uses [Conventional Commits](https://www.conventionalcommits.org/) for its commit messages. + +### Commit types + +Commit _types_ besides `fix` and `feat` are: + +- `refactor`: Keeping functionality while streamlining or otherwise improving function flow +- `docs`: Documentation for project or components + +### Commit scopes + +The following _scopes_ are known for this project. A Conventional Commits commit message may optionally use one of the following scopes or none: + +- `paperless_ngx`: A change to how the `paperless_ngx` service component works +- `build`: Build-related changes such as `Dockerfile` fixes and features. +- `mount`: Volume or bind mount-related changes. +- `net`: Networking, IP addressing, routing changes +- `meta`: Affects the project's repo layout, file names etc. diff --git a/common-settings.yaml b/common-settings.yaml new file mode 100644 index 0000000..19d917f --- /dev/null +++ b/common-settings.yaml @@ -0,0 +1,11 @@ +services: + common-settings: + environment: + TZ: "${TIMEZONE:-Etc/UTC}" + logging: + driver: "json-file" + options: + max-size: "10m" + max-file: "10" + compress: "true" + restart: "${RESTARTPOLICY:-always}" diff --git a/compose.override.yaml b/compose.override.yaml new file mode 100644 index 0000000..2d2a55c --- /dev/null +++ b/compose.override.yaml @@ -0,0 +1,72 @@ +services: + broker: + container_name: "paperless_ngx-broker-${CONTEXT}" + networks: + paperless_ngx-default: + extends: + file: "${COMPOSE_COMMON_SETTINGS}" + service: 'common-settings' + volumes: + - "/opt/docker-data/paperless_ngx-${CONTEXT}/broker/data:/data" + db: + image: "docker.io/library/postgres:${PGSQL_VERSION}" + container_name: "paperless_ngx-db-${CONTEXT}" + networks: + paperless_ngx-default: + extends: + file: "${COMPOSE_COMMON_SETTINGS}" + service: 'common-settings' + volumes: + - "/opt/docker-data/paperless_ngx-${CONTEXT}/db/data:/var/lib/postgresql/data" + gotenberg: + container_name: "paperless_ngx-gotenberg-${CONTEXT}" + networks: + paperless_ngx-default: + extends: + file: "${COMPOSE_COMMON_SETTINGS}" + service: 'common-settings' + tika: + container_name: "paperless_ngx-tika-${CONTEXT}" + networks: + paperless_ngx-default: + extends: + file: "${COMPOSE_COMMON_SETTINGS}" + service: 'common-settings' + webserver: + image: "ghcr.io/paperless-ngx/paperless-ngx:${PAPERLESS_NGX_VERSION}" + container_name: "paperless_ngx-webserver-${CONTEXT}" + networks: + paperless_ngx-default: + ports: !override + - "${WEBSERVER_VIP}:${WEBSERVER_PORT}:8000" + extends: + file: "${COMPOSE_COMMON_SETTINGS}" + service: 'common-settings' + volumes: + - "/opt/docker-data/paperless_ngx-${CONTEXT}/webserver/data:/usr/src/paperless/data" + - "/opt/docker-data/paperless_ngx-${CONTEXT}/webserver/media:/usr/src/paperless/media" + - "/opt/docker-data/paperless_ngx-${CONTEXT}/webserver/export:/usr/src/paperless/export" + - "/opt/docker-data/paperless_ngx-${CONTEXT}/webserver/consume:/usr/src/paperless/consume" + environment: + PAPERLESS_OCR_LANGUAGE: "${PAPERLESS_OCR_LANGUAGE}" + PAPERLESS_OCR_LANGUAGES: "${PAPERLESS_OCR_LANGUAGES}" + PAPERLESS_SECRET_KEY: "${PAPERLESS_SECRET_KEY}" + PAPERLESS_TIME_ZONE: "${PAPERLESS_TIME_ZONE}" + PAPERLESS_URL: "${PAPERLESS_URL}" + PAPERLESS_EMAIL_HOST: "${PAPERLESS_EMAIL_HOST}" + PAPERLESS_EMAIL_HOST_PASSWORD: "${PAPERLESS_EMAIL_HOST_PASSWORD}" + PAPERLESS_EMAIL_HOST_USER: "${PAPERLESS_EMAIL_HOST_USER}" + PAPERLESS_EMAIL_PORT: "${PAPERLESS_EMAIL_PORT}" + PAPERLESS_EMAIL_USE_SSL: "${PAPERLESS_EMAIL_USE_SSL}" + PAPERLESS_OCR_CLEAN: "${PAPERLESS_OCR_CLEAN}" +volumes: !reset [] +networks: !override + paperless_ngx-default: + name: paperless_ngx-${CONTEXT} + driver: bridge + driver_opts: + com.docker.network.enable_ipv6: "false" + ipam: + driver: default + config: + - subnet: ${SUBNET} diff --git a/env/fqdn_context.env.example b/env/fqdn_context.env.example new file mode 100644 index 0000000..62eaa17 --- /dev/null +++ b/env/fqdn_context.env.example @@ -0,0 +1,24 @@ +CONTEXT=ux_vilnius +PAPERLESS_EMAIL_HOST=smtp.example.com +PAPERLESS_EMAIL_HOST_PASSWORD=my-email-password +PAPERLESS_EMAIL_HOST_USER=mail@example.com +PAPERLESS_EMAIL_PORT=465 +PAPERLESS_EMAIL_USE_SSL=true +PAPERLESS_OCR_CLEAN=clean-final +PAPERLESS_OCR_LANGUAGE=swe +PAPERLESS_OCR_LANGUAGES=jpn jpn-vert +PAPERLESS_SECRET_KEY=my-secret-key +PAPERLESS_TIME_ZONE=Europe/San_Marino +PAPERLESS_URL=https://fully.qualified.domain.name +PGSQL_VERSION=16 +SUBNET=172.30.95.0/24 +TIMEZONE=Europe/San_Marino +WEBSERVER_PORT=61000 +WEBSERVER_VIP=10.10.10.1 + +# Other available defaults +# USERMAP_UID=1000 +# USERMAP_GID=1000 +# PAPERLESS_FORCE_SCRIPT_NAME=/PATHPREFIX +# PAPERLESS_STATIC_URL=/PATHPREFIX/static/ # trailing slash required +# RESTARTPOLICY=always \ No newline at end of file