148 lines
7.4 KiB
Markdown
148 lines
7.4 KiB
Markdown
# Spin it up
|
|
|
|
## Defaults
|
|
|
|
We use the [upstream github.com/paperless-ngx/paperless-ngx](https://github.com/paperless-ngx/paperless-ngx) repo. Assuming you have this checked out at `/opt/git/github.com/paperless-ngx/paperless-ngx/branches/main`. The `COMPOSE_CTX` (`CTX` as in context) is a uniq identifier to differentiate one instance from another. This can be for example `hr_juba` to indicate that this particular instance runs on behalf of the Human Resources department based in Juba in South Sudan in Africa.
|
|
|
|
Examples assume we're using the `docker-compose.postgres-tika.yml` file which spins up PostgreSQL as its database and also provides [Apache Tika](https://tika.apache.org/) and [Gotenberg PDF API](https://gotenberg.dev/docs/about).
|
|
|
|
## Environment variables
|
|
|
|
* Set env vars
|
|
```
|
|
export COMPOSE_DIR='/opt/git/github.com/paperless-ngx/paperless-ngx/branches/main/docker/compose'
|
|
export COMPOSE_CTX=''"${COMPOSE_CTX}"''
|
|
export COMPOSE_PROJECT_NAME='paperless_ngx-'"${COMPOSE_CTX}"
|
|
export COMPOSE_FILE="${COMPOSE_DIR%/}"'/docker-compose.postgres-tika.yml'
|
|
export COMPOSE_ENV_FILE=<set accordingly>
|
|
```
|
|
|
|
Also check out the example env file at [env/fully.qualified.domain.name_ctx.env.example](env/fully.qualified.domain.name_ctx.env.example).
|
|
|
|
* Start containers
|
|
```
|
|
docker compose --project-name "${COMPOSE_PROJECT_NAME}" --file "${COMPOSE_FILE}" --env-file "${COMPOSE_ENV_FILE}" up --detach
|
|
```
|
|
|
|
# Prep work
|
|
|
|
## Data sets
|
|
|
|
To get started from scratch create your ZFS datasets and set permissions as needed for `paperless-ngx`.
|
|
|
|
* Parent dateset
|
|
```
|
|
zfs create -o mountpoint=/opt/docker-data 'zpool/docker-data'
|
|
```
|
|
|
|
* Container-specific datasets
|
|
```
|
|
zfs create -p 'zpool/docker-data/paperless_ngx-'"${COMPOSE_CTX}"'/broker/data'
|
|
zfs create -p 'zpool/docker-data/paperless_ngx-'"${COMPOSE_CTX}"'/db/data'
|
|
zfs create -p 'zpool/docker-data/paperless_ngx-'"${COMPOSE_CTX}"'/webserver/data'
|
|
zfs create -p 'zpool/docker-data/paperless_ngx-'"${COMPOSE_CTX}"'/webserver/media'
|
|
zfs create -p 'zpool/docker-data/paperless_ngx-'"${COMPOSE_CTX}"'/webserver/export'
|
|
zfs create -p 'zpool/docker-data/paperless_ngx-'"${COMPOSE_CTX}"'/webserver/consume'
|
|
```
|
|
|
|
* Change ownership for all webserver-related dirs
|
|
```
|
|
chown -R 1000:1000 'zpool/docker-data/paperless_ngx-'"${COMPOSE_CTX}"'/webserver'
|
|
```
|
|
|
|
## Apply patch
|
|
|
|
Identify yourself to the local `paperless-ngx` repo. Obviously substitute your own name. An e-mail address is optional here. You don't want to contribute upstream, you just want to locally apply a patch file.
|
|
```
|
|
git -C '/opt/git/github.com/paperless-ngx/paperless-ngx/branches/main' config user.name "hygienic-books"
|
|
git -C '/opt/git/github.com/paperless-ngx/paperless-ngx/branches/main' config user.email ""
|
|
```
|
|
|
|
Apply `paperless_ngx.patch` to Docker Compose file. We use the `docker-compose.postgres-tika.yml` Compose file. Assuming this repo lives at `/opt/containers/paperless_ngx`:
|
|
```
|
|
git -C '/opt/git/github.com/paperless-ngx/paperless-ngx/branches/main' am '/opt/containers/paperless_ngx/paperless_ngx.patch'
|
|
|
|
# Output will be:
|
|
Applying: refactor(compose): 4 spaces indentation
|
|
Applying: refactor(compose): Harmonize restart and logging settigs
|
|
Applying: refactor(compose): Replace static exposed port with environment variable
|
|
Applying: refactor(compose): Harmonize container names
|
|
Applying: refactor(compose): Replace named volumes with bind mounts
|
|
...
|
|
```
|
|
|
|
And then back up to [Environment variables](#environment-variables).
|
|
|
|
# Upgrade an existing repo
|
|
|
|
Check [Prep work](#prep-work) for first time steps. On consecutive upgrades proceed as follows.
|
|
|
|
## Revert unpushed local changes
|
|
|
|
Return repo state to exactly the upstream repo's original branch state throwing away the commits you added.
|
|
```
|
|
git -C '/opt/git/github.com/paperless-ngx/paperless-ngx/branches/main' reset --hard origin
|
|
```
|
|
|
|
Switch to `main` branch, get newest commits from upstream
|
|
```
|
|
git -C '/opt/git/github.com/paperless-ngx/paperless-ngx/branches/main' checkout main
|
|
git -C '/opt/git/github.com/paperless-ngx/paperless-ngx/branches/main' pull
|
|
```
|
|
|
|
Pick and checkout new tag
|
|
```
|
|
while IFS= read -r; do commitDate=$(grep -Pio '^.+?(?=[[:space:]])' <<< "${REPLY}"); commitDate=$(date --date='@'"${commitDate}" +%F-%H%M%S); tagRef="$(cut -d $'\t' -f2 <<< "${REPLY}")"; tagName="$(grep -Pio '(?<=refs/tags/)[^\r\n\f]+' <<<"${tagRef}")"; commitHash="$(git rev-list -n 1 "${tagRef}")"; echo "${commitDate} ${commitHash} ${tagName}"; done < <(git for-each-ref --sort=v:refname --format='%(*creatordate:raw)%00%(creatordate:raw)%00%(refname)' refs/tags | awk -F"\0" 'BEGIN {ORS=""} $1 == "" {print $2} $1 != "" {print $1} {print "\t"$3"\n"}')
|
|
|
|
# Output goes like:
|
|
...
|
|
2023-04-27-161244 864e242ed9c454585e236b0c20ccae0927b4c9b2 v1.14.1
|
|
2023-04-27-195703 356c26ce848ca5301156a33e9ea75a10255f404b v1.14.2
|
|
2023-05-03-155437 4353646b3ac5805f7c582599b784a2fc246b3700 v1.14.3
|
|
2023-05-04-164855 ec4814a76e88efa81387316d8c42afc7e220fcbe v1.14.4
|
|
2023-05-15-170859 3e129763c799a7141e5ecd04862c0160caeeef5b v1.14.5
|
|
...
|
|
|
|
git -C '/opt/git/github.com/paperless-ngx/paperless-ngx/branches/main' checkout 'tags/vx.y.z'
|
|
```
|
|
|
|
Lastly [apply patch](#apply-patch). If patch does not apply cleanly read on in the next section [Create new patch](#create-new-patch) to find out how to fix your patch.
|
|
|
|
# Create new patch
|
|
|
|
## Add your changes as commits
|
|
|
|
With `paperless-ngx` repo checked out at `/opt/git/github.com/paperless-ngx/paperless-ngx/branches/main` get the it into a state with which you're happy then
|
|
```
|
|
git -C '/opt/git/github.com/paperless-ngx/paperless-ngx/branches/main' format-patch 31b7e26f6c4d7111f4f4957996efb9f7a5d06cb9^..64beae08ffe8f9a65208e2567919fd75564b95c6 --stdout > '/opt/containers/paperless_ngx/paperless_ngx.patch'
|
|
```
|
|
|
|
Where the first commit hash is our first commit and the other commit hash is our last commit. Note the caret (`^`) right after the first commit hash.
|
|
|
|
## Investigation
|
|
|
|
You may have to try and find out how a known good base commit differs from a newer one in case the newer one does no longer cleanly accept the patch.
|
|
|
|
Get commit hashes from both affected tags, e.g.
|
|
```
|
|
while IFS= read -r; do commitDate=$(grep -Pio '^.+?(?=[[:space:]])' <<< "${REPLY}"); commitDate=$(date --date='@'"${commitDate}" +%F-%H%M%S); tagRef="$(cut -d $'\t' -f2 <<< "${REPLY}")"; tagName="$(grep -Pio '(?<=refs/tags/)[^\r\n\f]+' <<<"${tagRef}")"; commitHash="$(git rev-list -n 1 "${tagRef}")"; echo "${commitDate} ${commitHash} ${tagName}"; done < <(git for-each-ref --sort=v:refname --format='%(*creatordate:raw)%00%(creatordate:raw)%00%(refname)' refs/tags | awk -F"\0" 'BEGIN {ORS=""} $1 == "" {print $2} $1 != "" {print $1} {print "\t"$3"\n"}')
|
|
|
|
# Output goes like:
|
|
...
|
|
2023-04-27-161244 864e242ed9c454585e236b0c20ccae0927b4c9b2 v1.14.1
|
|
2023-04-27-195703 356c26ce848ca5301156a33e9ea75a10255f404b v1.14.2
|
|
2023-05-03-155437 4353646b3ac5805f7c582599b784a2fc246b3700 v1.14.3
|
|
2023-05-04-164855 ec4814a76e88efa81387316d8c42afc7e220fcbe v1.14.4
|
|
2023-05-15-170859 3e129763c799a7141e5ecd04862c0160caeeef5b v1.14.5
|
|
...
|
|
```
|
|
|
|
Diff them
|
|
```
|
|
git -C '/opt/git/github.com/paperless-ngx/paperless-ngx/branches/main' diff ec4814a76e88efa81387316d8c42afc7e220fcbe 3e129763c799a7141e5ecd04862c0160caeeef5b 'docker/compose/docker-compose.postgres-tika.yml'
|
|
```
|
|
|
|
Output will be empty in case no difference exists in `docker/compose/docker-compose.postgres-tika.yml` between both commit hashes.
|
|
|
|
Commit your updated patch file into _this_ repo. With a new working patch in hand head back up to [Upgrade an existing repo](#upgrade-an-existing-repo).
|