Compare commits

..

67 Commits

Author SHA1 Message Date
15d67f2e3f Merge pull request '26-install-zfs-only-when-needed' (#27) from 26-install-zfs-only-when-needed into main
Reviewed-on: #27
2025-01-17 20:11:11 +00:00
6fb13a097c Merge remote-tracking branch 'origin/26-install-zfs-only-when-needed' into 26-install-zfs-only-when-needed 2025-01-17 21:09:40 +01:00
64c66cbd0f fix(zfs): Install ZFS only when needed (#26)
In situations where this script runs on alternative Arch Linux live
CD ISOs such as github.com/stevleibelt/arch-linux-live-cd-iso-with
-zfs we may not have to insall ZFS kernel modules. Test if the 'zfs'
module is loaded and skip installation if yes.
2025-01-17 21:09:19 +01:00
1036ce9c5b fix(zfs): Install ZFS only when needed (#26)
In situations where this script runs on alternative Arch Linux live
CD ISOs such as github.com/stevleibelt/arch-linux-live-cd-iso-with
-zfs we may not have to insall ZFS kernel modules. Test if the 'zfs'
module is loaded and skip installation if yes.
2025-01-17 21:08:44 +01:00
3ae8b53616 build(meta): Don't commit .idea JetBrains dir (#26) 2025-01-17 21:05:53 +01:00
2adac7b94b Add a 2G loop device for emergency swap 2025-01-02 01:00:06 +01:00
9525976fe2 For now no pacman upgrade in ISO 2024-12-28 18:27:29 +01:00
624d278971 For now no pacman upgrade in ISO 2024-12-28 18:21:04 +01:00
08f33c33c7 refactor(os): Retry pacman download on fail 2024-12-28 16:33:13 +01:00
eadbbea2fd Merge pull request '19-define-os-ip' (#24) from 19-define-os-ip into main
Reviewed-on: #24
2023-11-12 03:26:36 +00:00
d7d41eec5b refactor(os): Homogenous single quotes (#19) 2023-11-12 03:32:27 +01:00
6fe5d59108 refactor(os): remove unneeded line break (#19) 2023-11-12 03:31:31 +01:00
96b860e0db refactor(os): Update interactive questionnaire gif (#19) 2023-11-12 03:31:02 +01:00
d467dd6610 refactor(os): Phrasing (#19) 2023-11-11 04:35:41 +01:00
8ed5849596 refactor(os): Phrasing (#19) 2023-11-11 04:26:11 +01:00
5f6a5bfdac refactor(os): No need to hide kernel version string (#19) 2023-11-11 04:20:29 +01:00
47321313a9 refactor(os): Fix function numbering (#19) 2023-11-11 04:07:44 +01:00
c344aac77a refactor(os): Fix comment spacing (#19) 2023-11-11 04:06:30 +01:00
fd2c08a18a refactor(os): Always enable NTP (#19) 2023-11-11 04:05:21 +01:00
d74fc097eb refactor(os): Be specific with systemd unit names (#19) 2023-11-11 04:04:58 +01:00
acc6b2c721 feat(os): Reuse or set SSH pub keys in OS (#19) 2023-11-11 04:04:14 +01:00
d380fc4b6d fix(os): No need to double-quote in a heredoc (#19) 2023-11-11 04:03:09 +01:00
d95d43a22b feat(os): Downgrade kernel when requested (#19) 2023-11-11 04:02:30 +01:00
557db12e8d fix(zbm): ZBM 2.2.2 no longer needs nor has its own stub loader file (#19) 2023-11-11 04:01:59 +01:00
eaf83163b6 refactor(os): Typo (#19) 2023-11-11 04:01:28 +01:00
257648c99d feat(os): Optionally force kernel downgrade for ZFS compatibility (#19) 2023-11-11 04:01:14 +01:00
3903498d40 feat(iso): Modularize network unit file content (#21) 2023-11-11 01:20:32 +01:00
59beb10404 feat(iso): Write OS IPs, gateway, NTP into env file (#21) 2023-11-11 01:19:56 +01:00
d99bb94c52 feat(iso): In setup questionnaire ask for NTP servers (#21) 2023-11-11 01:19:24 +01:00
c9f979a0ad feat(iso): In setup questionnaire ask for OS IP and gateway (#21) 2023-11-11 01:19:01 +01:00
db3c705376 refactor(iso): echo consistently with single quotes (#21) 2023-11-11 01:18:19 +01:00
d5e219dbf5 feat(iso): Add bash-only CIDR calculator function (#21) 2023-11-11 01:17:14 +01:00
bd586e4c67 fix(zfs): Fix multi-line bash command (#21) 2023-11-11 01:16:37 +01:00
d2108276a0 Merge pull request '21-assist-in-env-setup' (#23) from 21-assist-in-env-setup into main
Reviewed-on: #23
2023-11-06 00:39:32 +00:00
ed441299bc fix(iso): Asking for an SSH pub key is mandatory with SSH daemon (#21) 2023-11-06 00:48:40 +01:00
37cafc1f20 fix(iso): Don't reassign settings_file locally (#21) 2023-11-06 00:25:47 +01:00
1bc09b7f8b refactor(os): Install paru-bin instead of paru (#21)
We straight up install paru-bi via its PKGBUILD from
AUR, we skip the additional step we used to do
where we first installed paru from its GitHub
project. This saves time and most importantly
scarce RAM that Rust otherwise needs during
paru compilation.

On systems with little RAM (as in 4 GiB) paru's
compilation process would sometimes fail when
/etc/makepkg.conf when its MAKEFLAGS was
set to use many cores e.g. via "-j$(nproc --ignore 1)".

Without the manual paru compilation step we now
don't need the ability anymore to
--replace-conflicting packages.
2023-11-05 23:29:54 +01:00
1b94e7e3b8 refactor(zbm): Ask for rerun (#21) 2023-11-05 16:57:50 +01:00
e39d60cb00 refactor(zbm): Exit after setting up vars (#21) 2023-11-05 16:55:06 +01:00
055f970f43 docs(zbm): Render asciinema gif with larger typeface (#21) 2023-11-05 16:41:15 +01:00
16e67c8b28 docs(zbm): Typo (#21) 2023-11-05 16:30:48 +01:00
4fb7a91703 docs(zbm): Link to giv instead of mp4 (#21) 2023-11-05 16:30:16 +01:00
2b5d4b4ec8 docs(zbm): Testing HTML image tag in Markdown (#21) 2023-11-05 16:28:17 +01:00
b885bde3c6 docs(zbm): Explain network device naming (#21) 2023-11-05 16:23:00 +01:00
316aa56a55 docs(zbm): Explain interactive questionnaire (#21) 2023-11-05 16:04:57 +01:00
39039ce3fd docs(zbm): No idea if IPv6 works (#21) 2023-11-05 04:53:52 +01:00
1183f3f025 refactor(iso): Rename .gitignore file (#21) 2023-11-05 04:53:17 +01:00
be76d6b0f8 feat(meta): Ignore loca settings file while testing (#21) 2023-11-05 04:52:21 +01:00
67ea72de51 feat(iso): Add setup function to generate answer file (#21) 2023-11-05 04:51:32 +01:00
9f3ada2a36 feat(iso): Parse arguments (#21) 2023-11-05 04:50:31 +01:00
28414af039 refactor(iso): Globally define name of settings file (#21) 2023-11-05 04:49:32 +01:00
58da096eab docs(meta): Typo 2023-11-05 02:37:50 +01:00
5f57698e7d Merge pull request '6-add-ssh-to-bootloader' (#20) from 6-add-ssh-to-bootloader into main
Reviewed-on: #20
2023-11-05 01:35:45 +00:00
2b982be84e docs(meta): Bits and pieces updated (#6) 2023-11-05 02:35:03 +01:00
782168c1b1 docs(zbm): Nudge user to work with Dropbear keepalives (#6) 2023-11-05 02:10:33 +01:00
5cf4b2c325 docs(zbm): Let's not overpromise, the command stanza at first glance seems trickier than anticipated (#6) 2023-11-04 01:16:39 +01:00
a2cb784cb3 feat(zbm): In Syslinux/extlinux mode add KCL (#6) 2023-11-04 01:08:39 +01:00
119be2b876 docs(zbm): Give an example ip= KCL (#6) 2023-11-04 00:46:44 +01:00
9560677352 fix(zbm): Link to dropbear patch hooks (#6) 2023-11-04 00:41:37 +01:00
52c09fc93f docs(zbm): Explain Syslinux KCL (#6) 2023-11-04 00:37:30 +01:00
1181432add docs(iso): Explain sed in_array_remove_n mechanism (#6) 2023-11-04 00:32:16 +01:00
e3025883fa docs(iso): Explain sed insert_n_at_the_end mechanism (#6) 2023-11-04 00:32:01 +01:00
79feaed5ac docs(iso): Explain sed insert_n_before_m mechanism (#6) 2023-11-04 00:20:39 +01:00
7338924c82 docs(zbm): Explain networking and SSH (#6) 2023-11-04 00:08:48 +01:00
924925e08e docs(iso): Do a pass over how we explain passwords for local accounts (#6) 2023-11-04 00:03:42 +01:00
3030eb0f2d docs(iso): Use either an env options file or shell exports (#6) 2023-11-04 00:01:29 +01:00
305e4191f1 docs(zfs): Correctly umount after maintenance (#6) 2023-11-04 00:00:03 +01:00
3 changed files with 855 additions and 155 deletions

2
.gitignore vendored Normal file
View File

@@ -0,0 +1,2 @@
archzbm_settings.env
.idea

207
README.md
View File

@@ -82,7 +82,7 @@ NAME SIZE FSTYPE PARTTYPE PARTTYPENAME PTTYPE
# Partition naming # Partition naming
Since this script works with UEFI and legacy BIOS mode we'll be addressing both disk layout schemes with umbrella terms for better readability: "The zpool partition" will be GPT `BF00` partition and MBR `bf` partition. You'll parse the text accordingly. "The boot partition" will be GPT `EF00` partition as well as the MBR `83` partition. Since this script works with UEFI and legacy BIOS mode we'll be addressing both disk layout schemes with umbrella terms for the rest of this document for better readability: "The zpool partition" will be GPT `BF00` partition and MBR `bf` partition. You'll parse the text accordingly. "The boot partition" will be GPT `EF00` partition as well as the MBR `83` partition.
# ZFS dataset layout # ZFS dataset layout
@@ -99,6 +99,40 @@ The script will create a single ZFS zpool `zpool` on the zpool partition with da
## Options ## Options
The following options can be given either by exporting them as shell variables prior to script execution or in a file named `archzbm_settings.env` that lives in your current working directory where you're about to execute the script. You can walk yourself through an interactive questionnaire that helps create a valid `archzbm_settings.env` file. Check out [Command line setup help](#command-line-setup-help) for details on the questionnaire.
If you instead want to define settings yourself with an `archzbm_settings.env` file its file format is identical to shell variable assignments of the form `VAR=value` or `VAR='value'`.
If `./archzbm_settings.env` exists the script will `source` its content and `export` all variables for use in future steps.
In cases where a variable is both exported prior to script execution and specified in `archzbm_settings.env` the latter will override the former.
Known options are as follows.
### Kernel downgrade
By default we install newest `linux` and `linux-headers` packages into a `chroot`. Once we're in that `chroot` we then install newest [AUR zfs-dkms package](https://aur.archlinux.org/packages/zfs-dkms). You may want to override `linux` and `linux-headers` versions to ensure you end up with a compatible mix between them and `zfs-dkms`.
For example:
```
export ARCHZBM_KERNEL_VER=6.5.9.arch2
```
In our `chroot` this will trigger execution of:
```
downgrade --ala-only 'linux=6.5.9.arch2' 'linux-headers=6.5.9.arch2' --ignore always
```
Where `downgrade` is the [AUR downgrade package](https://aur.archlinux.org/packages/downgrade). This will downgrade `linux` and `linux-headers` and will add a setting to your `/etc/pacman.conf`:
```
[options]
IgnorePkg = linux linux-headers
```
Setting `ARCHZBM_KERNEL_VER` to an empty string `''` or keeping it undefined are both valid and will retain newest versions instead of downgrading.
Also read [Kernel selection](#kernel-selection) for details.
### Compression ### Compression
By default we create a zpool with ZFS property `compression=on`. If the `lz4_compress` pool feature is active this will by default enable `compression=lz4`. See `man 7 zfsprops` for example in ZFS 2.1.9 for details. See `zpool get feature@lz4_compress <pool>` to check this feature's status on your `<pool>`. By default we create a zpool with ZFS property `compression=on`. If the `lz4_compress` pool feature is active this will by default enable `compression=lz4`. See `man 7 zfsprops` for example in ZFS 2.1.9 for details. See `zpool get feature@lz4_compress <pool>` to check this feature's status on your `<pool>`.
@@ -119,24 +153,134 @@ export ARCHZBM_ZFSPROPS_NO_ENCRYPTION=yup
### Passwords ### Passwords
By default both the zpool password and the account password for `root` are literally `password`. While you can certainly change these after initial system setup you can also optionally set these passwords in a settings file named `archzbm_settings.env` that lives in your current working directory where you're about to execute the script. File format is identical to shell variable assignments of the form `VAR=value` or `VAR='value'`. By default both the zpool password and the account password for `root` are literally `password`. While you can certainly change these after initial system setup (see [Password change](#password-change)) you can also optionally set passwords before script execution as follows:
If `./archzbm_settings.env` exists the script will `source` its content and `export` all variables for use in future steps. Only known variables are:
``` ```
ARCHZBM_ZPOOL_PASSWORD='a fancy password' ARCHZBM_ZPOOL_PASSWORD='a fancy password'
ARCHZBM_ROOT_PASSWORD='t0psecr3t!' ARCHZBM_ROOT_PASSWORD='t0psecr3t!'
``` ```
> While the `root` password may be weak and `chpasswd` won't care do make sure to set a zpool password that meets ZFS' complexity rules. Per `man 7 zfsprops` section `keyformat` the only requirement is a length "between 8 and 512 bytes" (as in minimum 8 characters). If you pick a password that's too weak ZFS will reject zpool creation and very ungracefully derail the rest of this script. The script doesn't check what you're setting. > While the `root` password is allowed to be weak and `chpasswd` won't care do make sure to set a zpool password that meets ZFS' complexity rules. Per `man 7 zfsprops` section `keyformat` the only requirement is a length "between 8 and 512 bytes" (as in minimum 8 characters). If you pick a password that's too weak ZFS will reject zpool creation and very ungracefully derail the rest of this script. The script doesn't check what you're setting.
The script does create a second user named `build` but doesn't set a password on account creation. As such no password variable can be set for it in `./archzbm_settings.env`. It's intended as a helper for system setup tasks such as `sudo -u build paru -S <package>` where an account password is irrelevant since `root` can always `sudo` whatever it wants. You will not be able to log in to the `build` account yourself although you certainly could set a password for it. Instead we suggest you create a proper user account for yourself. Your newly installed Arch Linux comes with an `/etc/motd` greeting that summarizes this as: The script does create a second user named `build` but doesn't set a password on account creation. It's intended as a helper for system setup tasks such as `sudo -u build paru -S <package>` where an account password is irrelevant since `root` can always `sudo` whatever it wants. You will not be able to log in to the `build` account yourself although you certainly could set a password for it. Instead we suggest you create a proper user account for yourself. Your newly installed Arch Linux comes with an `/etc/motd` greeting that summarizes this as:
``` ```
useradd --create-home --shell /bin/bash --user-group --groups wheel <user> useradd --create-home --shell /bin/bash --user-group --groups wheel <user>
passwd <user> passwd <user>
``` ```
### Networking
By default the script configures plain ZFSBootMenu without networking nor an SSH server. If you're interested in SSH-ing into your ZFSBootMenu boot loader you're going to want to specify some of the following variables.
#### IP address
> IPv6 addresses are untested. Script has been confirmed working with IPv4 addresses.
```
ARCHZBM_NET_CLIENT_IP=''
ARCHZBM_NET_SERVER_IP=''
ARCHZBM_NET_GATEWAY_IP=''
ARCHZBM_NET_NETMASK=''
ARCHZBM_NET_HOSTNAME=''
ARCHZBM_NET_DEVICE=''
ARCHZBM_NET_AUTOCONF=''
```
By default none of the variables are set to any value and no networking will be available in ZFSBootMenu. If you want networking as in an IP address bound to a network interface set at least one of these variables or one of the [SSH](#ssh) variables listed further down. Setting one or more `ARCHZBM_NET_*` variables to an empty string is valid. If at least one variable is given either from this paragraph or from [SSH](#ssh) we're assuming that you want networking. Unspecified values and values set to the empty string `''` use defaults.
For networking we rely on the [mkinitcpio-nfs-utils](https://archlinux.org/packages/core/x86_64/mkinitcpio-nfs-utils/) package with its `net` hook. Please refer to its [initcpio-install-net](https://gitlab.archlinux.org/archlinux/packaging/packages/mkinitcpio-nfs-utils/-/blob/main/initcpio-install-net) script file for usage hints on above variables. The hook implements a subset of the [ip Kernel Command Line argument](https://docs.kernel.org/admin-guide/nfs/nfsroot.html).
Mapping between `net` hook field names and our shell variables is straightforward. Fields 8, 9 and 10 (DNS and NTP server addresses) from the official `ip` docs are unsupported in `net` hook. As such our hook has a total of 7 fields available for you to configure.
```
+-------------+------------------------+
| net hook | This script |
+-------------+------------------------+
| <client-ip> | ARCHZBM_NET_CLIENT_IP |
| <server-ip> | ARCHZBM_NET_SERVER_IP |
| <gw-ip> | ARCHZBM_NET_GATEWAY_IP |
| <netmask> | ARCHZBM_NET_NETMASK |
| <hostname> | ARCHZBM_NET_HOSTNAME |
| <device> | ARCHZBM_NET_DEVICE |
| <autoconf> | ARCHZBM_NET_AUTOCONF |
+-------------+------------------------+
```
A valid example with a few fields populated may look like so:
```
ARCHZBM_NET_CLIENT_IP='10.10.10.2'
ARCHZBM_NET_GATEWAY_IP='10.10.10.1'
ARCHZBM_NET_NETMASK='255.255.255.0'
ARCHZBM_NET_DEVICE='eth0'
ARCHZBM_NET_AUTOCONF='none'
```
Note that in this example `ARCHZBM_NET_SERVER_IP` and `ARCHZBM_NET_HOSTNAME` are left unassigned.
It'll add the following `ip=` instruction to your Kernel Command Line:
```
ip=10.10.10.2::10.10.10.1:255.255.255.0::eth0:none
```
This is also valid and will configure `eth0` via DHCP:
```
ARCHZBM_NET_DEVICE='eth0'
ARCHZBM_NET_AUTOCONF='dhcp'
```
> In ZFSBootMenu the device names that go into `ARCHZBM_NET_DEVICE` are raw unchanged kernel device names such as `eth0`. If you're unsure which device name to use in your Arch Linux live CD ISO image check `dmesg` output. During boot typically a kernel module will first assign the raw kernel device name then later `systemd` will enforce [Predictable Network Interface Names](https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/).
>
> In `dmesg | grep` on a physical PC with an MSI B550-A Pro mainboard from 2020 that comes with one onboard Realtek RTL8111H network adapter governed by the Realtek RTL-8169 Gigabit Ethernet driver from the `r8169` kernel module you will for example see:
> ```
> # dmesg -T | grep eth
> [time] r8169 0000:2a:00.0 eth0: RTL8168h/8111h, 04:7c:16:00:01:02, XID 541, IRQ 95
> [time] r8169 0000:2a:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]
> [time] r8169 0000:2a:00.0 enp42s0: renamed from eth0
> ```
>
> Notice how a Predictable Network Interface Name comes in on line 3. What *__you__* need here is the `eth0` part.
#### SSH
If you want networking indicated by the fact that at least one of the `ARCHZBM_NET_*` variables is set or one of the `ARCHZBM_SSH_*` vars we assume that you want an SSH daemon as well. This comes in the form of a `dropbear` daemon with minimal configurability. Use the following variables to define Dropbear's behavior.
```
ARCHZBM_SSH_PORT='22'
ARCHZBM_SSH_KEEPALIVE_INTVL='1'
ARCHZBM_SSH_AUTH_KEYS=''
```
In `ARCHZBM_SSH_PORT` you specify Dropbear's listening port, this defaults to `22` if unconfigured or set to an empty string. With `ARCHZBM_SSH_KEEPALIVE_INTVL` you define at which interval Dropbear will send keepalive messages to an SSH client through the SSH connection. This defaults to `1` as in every `1` second a keepalive message is sent. Per [man 8 dropbear](https://man.archlinux.org/man/extra/dropbear/dropbear.8.en) a value of `0` disables Dropbear sending keepalive messages. We suggest to leave this on and to keep the interval short, see [SSH in ZFSBootMenu](#ssh-in-zfsbootmenu) for how to work with this.
Dropbear in this setup only supports key-based authentication, no password-based authentication. The value from `ARCHZBM_SSH_AUTH_KEYS` will be converted to a list of public SSH keys allowed to SSH into Dropbear as its default `root` user while ZFSBootMenu is running. The format of `ARCHZBM_SSH_AUTH_KEYS` is a single line where `authorized_keys` entries are split with double-commas:
```
ssh-rsa Eahajei8,,ssh-ed25519 kaeD0mas ...
```
This syntax crutch allows you to use the full range of Dropbear-supported `authorized_keys` stanzas, see [man 8 dropbear](https://man.archlinux.org/man/extra/dropbear/dropbear.8.en) for what's available. Whether or not this is useful to you is another topic :) At least the functionality for stanzas is there by separating values in `ARCHZBM_SSH_AUTH_KEYS` with double-commas.
## Command line setup help
An interactive questionnaire can guide you through settings and goes like this:
![Command line setup questionnaire](https://i.imgur.com/OXG75GH.gif)
To do the questionnaire yourself start this script with the `setup` argument:
```
export SCRIPT_URL='https://quico.space/quico-os-setup/arch-zbm/raw/branch/main/setup.sh' && curl -s "${SCRIPT_URL}" | bash -s -- setup
```
When done rerun it without that argument:
```
export SCRIPT_URL='https://quico.space/quico-os-setup/arch-zbm/raw/branch/main/setup.sh' && curl -s "${SCRIPT_URL}" | bash
```
# Steps # Steps
The script takes the following installation steps. The script takes the following installation steps.
@@ -151,11 +295,49 @@ The script takes the following installation steps.
1. Configure boot method 1. Configure boot method
- Either an EFI image with EFI boot order entries on a UEFI machine - Either an EFI image with EFI boot order entries on a UEFI machine
- Or Syslinux with `extlinux` for a legacy BIOS computer - Or Syslinux with `extlinux` for a legacy BIOS computer
1. If requested by user enable SSH in ZFSBootMenu. We then also add:
- [quico.space/quico-os-setup/mkinitcpio-dropbear-pacman-hook](https://quico.space/quico-os-setup/mkinitcpio-dropbear-pacman-hook)
1. Add `pacman` hooks to keep ZFSBootMenu images (and `extlinux`) updated 1. Add `pacman` hooks to keep ZFSBootMenu images (and `extlinux`) updated
- [quico.space/quico-os-setup/zbm-regen-pacman-hook](https://quico.space/quico-os-setup/zbm-regen-pacman-hook) - [quico.space/quico-os-setup/zbm-regen-pacman-hook](https://quico.space/quico-os-setup/zbm-regen-pacman-hook)
- [quico.space/quico-os-setup/zbm-syslinux-pacman-hook](https://quico.space/quico-os-setup/zbm-syslinux-pacman-hook) - [quico.space/quico-os-setup/zbm-syslinux-pacman-hook](https://quico.space/quico-os-setup/zbm-syslinux-pacman-hook)
1. Exit into Arch Linux live CD ISO image shell for you to `reboot` and frolick 1. Exit into Arch Linux live CD ISO image shell for you to `reboot` and frolick
# SSH in ZFSBootMenu
Per [SSH](#ssh) and [Networking](#networking) this script will optionally add a Dropbear SSH daemon to ZFSBootMenu. While the mechanism of SSH-ing into a server isn't particularly noteworthy we humbly suggest that in this particular use case you let your SSH client listen for keepalive messages from the server.
```
ssh -o ServerAliveInterval=3 -o ServerAliveCountMax=0 root@<addr> -p <port>
```
A typical workflow with Dropbear is for you to SSH into it, issue `zfs` or `zfsbootmenu` commands and allow the Arch Linux boot process to commence. As soon as you're done Dropbear will terminate as ZFSBootMenu hands control off to your operating system's kernel. Without your client listening to keepalive messages it may not realize that the connection's gone for quite some time until you harshly interrupt it.
The server defaults to sending keepalive messages to your client every second.
With `-o ServerAliveInterval=3` you instruct your client to send an are-your-still-there message to the server if your client ever stops getting keepalive messages from the server for 3 seconds. The server defaults to sending 1 keepalive ping per second so even on a somewhat lossy connection we can reasonably expect to get one message through to us within 3 seconds.
When it comes to the point that your SSH client sends an are-your-still-there message it expects a near-realtime response. It will accept `-o ServerAliveCountMax=0` failures from the server to comply.
This effectively configures your SSH client to remain connected even through somewhat lossy hops to the Dropbear daemon; and to cleanly disconnect 3 seconds and some change after you've executed whatever you needed to do in ZFSBootMenu.
# Kernel selection
This script compiles ZFS via Arch Linux' [Dynamic Kernel Module Support](https://wiki.archlinux.org/title/Dynamic_Kernel_Module_Support) (DKMS). Not all kernels allow for successful compilation, in some instances a particularly recent kernel version may change APIs to such a degree that ZFS compilation simply fails.
We strongly suggest to that you:
- Firstly, refer to a resource such as the [Arch Linux Archive package version list](https://archive.archlinux.org/packages/l/linux/) to find out what newest kernel version this script will install.
- Secondly, research if newest [AUR zfs-dkms package](https://aur.archlinux.org/packages/zfs-dkms) is compatible with that kernel. Two reasonable points of contact are AUR and its comments section for `zfs-dkms` where users quickly report issues; and the [github.com/openzfs/zfs issues list](https://github.com/openzfs/zfs/issues).
An example for this is that `linux-6.6.1.arch1-1-x86_64` came out on Wednesday, November 8, 2023 at a time when newest `zfs-dkms` package version [was 2.2.0](https://aur.archlinux.org/cgit/aur.git/commit/?h=zfs-dkms&id=da1b6372c57b16f2781a7fda2b95971bb392c5ee) which did not compile against `linux` 6.6.x.
You'd then set for example:
```
export ARCHZBM_KERNEL_VER=6.5.9.arch2
```
Where any 6.5.x version is known to work well with `zfs-dkms`. See also [Kernel downgrade](#kernel-downgrade) for details on how to configure this.
# Flavor choices # Flavor choices
We make the following opinionated flavor choices. Feel free to change them to your liking. We make the following opinionated flavor choices. Feel free to change them to your liking.
@@ -173,14 +355,14 @@ We make the following opinionated flavor choices. Feel free to change them to yo
After installation you're going to want to at least touch these points in your new Arch Linux install: After installation you're going to want to at least touch these points in your new Arch Linux install:
- Package manager hook: `pacman` does not have a hook to do ZFS snapshots - Package manager hook: `pacman` does not have a hook to do ZFS snapshots
- See [this GitHub gist](https://gist.github.com/Soulsuke/6a7d1f09f7fef968a2f32e0ff32a5c4c#file-arch_on_zfs-txt-L238) and [zfs-snapshotter.bash](https://github.com/Soulsuke/arch-zfs-tools/blob/master/zfs-snapshotter.bash) for inspiration - See [quico.space/quico-os-setup/zfs-pacman-hook](https://quico.space/quico-os-setup/zfs-pacman-hook/src/branch/1-get-base-version-going) for an example you may want to install
- Hostname: Installation chose a pseudo-randomly generated 8-character string with `pwgen` - Hostname: Installation chose a pseudo-randomly generated 8-character string with `pwgen`
- Check `hostnamectl set-hostname <hostname>` - Check `hostnamectl set-hostname <hostname>`
- Unprivileged user accounts: The OS was installed with `root` and unprivileged `build` users - Unprivileged user accounts: The OS was installed with `root` and unprivileged `build` users
- Unless you had a settings file per [Passwords](#passwords) you're going to want to change passwords now: - Unless you had a settings file or exported shell env vars per [Passwords](#passwords) you're going to want to change passwords now:
- ZFS: The password for all datasets underneath `zpool` is `password`. - ZFS: The password for all datasets underneath `zpool` is `password`.
- Local `root` account: The local `root` account's password is `password`. - Local `root` account: The local `root` account's password is `password`.
- Arch User Repository (AUR) helper: We installed [paru](https://github.com/Morganamilo/paru) as our AUR helper, we installed from GitHub via `makepkg -si` then replaced itself with its [paru-bin](https://aur.archlinux.org/packages/paru-bin) version from AUR. - Arch User Repository (AUR) helper: We installed [paru](https://github.com/Morganamilo/paru) as our AUR helper, we installed from AUR as [paru-bin](https://aur.archlinux.org/packages/paru-bin).
- In `/etc/systemd/network/50-wired.network` instead of a DHCP-based network config you can get a static one. The DHCP-based one for reference looks like: - In `/etc/systemd/network/50-wired.network` instead of a DHCP-based network config you can get a static one. The DHCP-based one for reference looks like:
``` ```
... ...
@@ -278,7 +460,7 @@ In order to generate a new master key after you've changed your user key as ment
--large-block \ --large-block \
--compressed \ --compressed \
'zpool/root/archlinux-sxu@rekey' | \ 'zpool/root/archlinux-sxu@rekey' | \
\
zfs receive \ zfs receive \
-Fvu \ -Fvu \
-o 'encryption=on' \ -o 'encryption=on' \
@@ -498,7 +680,12 @@ arch-chroot /mnt /bin/bash
When done exit `chroot` and cleanly remove your work: When done exit `chroot` and cleanly remove your work:
``` ```
# UEFI system ...
umount /mnt/efi umount /mnt/efi
# ... or legacy BIOS system
umount /mnt/boot/syslinux
zfs umount -a zfs umount -a
zpool export zpool zpool export zpool
``` ```

801
setup.sh
View File

@@ -19,9 +19,10 @@ exec 3>&1
declare this_script_url declare this_script_url
this_script_url="${SCRIPT_URL:?}" this_script_url="${SCRIPT_URL:?}"
declare zpool_name zfs_arch_dataset_name declare zpool_name zfs_arch_dataset_name settings_file
zpool_name='zpool' zpool_name='zpool'
zfs_arch_dataset_name='archlinux' zfs_arch_dataset_name='archlinux'
settings_file='archzbm_settings.env'
declare -A partition_types declare -A partition_types
partition_types[gpt_zfs]='6a85cf4d-1dd2-11b2-99a6-080020736631' partition_types[gpt_zfs]='6a85cf4d-1dd2-11b2-99a6-080020736631'
@@ -35,6 +36,363 @@ trap '[ "$?" -ne 77 ] || exit 77' ERR
declare zpool_drive efi_drive boot_drive part_schema declare zpool_drive efi_drive boot_drive part_schema
function calculate_prefix_from_netmask () {
# https://stackoverflow.com/a/50419919
c='0'
x='0'"$(printf -- '%o' ${1//./ })"
while [ "${x}" -gt '0' ]; do
# Modulo then bitwise-shift x and store as new x
let c+="$(( x % 2 ))" 'x>>=1'
done
printf -- '%s' '/'"${c}";
}
function setup_env_vars () {
printf -- '%s\n' \
'We will go over a series of questions to create an answer file with' \
'options you want the script to use.' \
'' \
'Current working directory is:'\
"$(pwd)" \
'' \
'After we'"'"'re done answer file will be written to current working dir:' \
'./'"${settings_file}" \
'' \
'Press <Ctrl>+C to abort this process. No answer file will' \
'be written to ./'"${settings_file}"' if you abort the script.' \
'' \
'When done rerun the same command you just did without '"'"'setup'"'"' argument.' \
''
read -u3 -n 1 -s -r -p "Press any key to begin questionnaire"
echo
echo '----------------------------------------'
echo
read -u3 -p 'Please type kernel version to use, leave empty for latest, confirm with <Enter>: ' ARCHZBM_KERNEL_VER
echo
echo 'Do you want compressed datasets?'
select arg_compressed in 'Compressed' 'Uncompressed'; do
case "${arg_compressed}" in
'Compressed')
break
;;
'Uncompressed')
ARCHZBM_ZFSPROPS_NO_COMPRESSION='true'
break
;;
esac
done <&3 && echo
echo 'Do you want encrypted datasets?'
select arg_encrypted in 'Encrypted' 'Unencrypted'; do
case "${arg_encrypted}" in
'Encrypted')
break
;;
'Unencrypted')
ARCHZBM_ZFSPROPS_NO_ENCRYPTION='true'
break
;;
esac
done <&3 && echo
if [[ "${arg_encrypted}" = 'Encrypted' ]]; then
echo 'Do you want a custom dataset decryption password?'
select arg_custom_dataset_pw in 'Yes' 'No (use '"'"'password'"'"')'; do
case "${arg_custom_dataset_pw}" in
'Yes')
want_custom_dataset_pw='true'
break
;;
'No (use '"'"'password'"'"')')
break
;;
esac
done <&3 && echo
if [[ "${want_custom_dataset_pw}" ]]; then
read -u3 -p 'Please type password, confirm with <Enter>: ' -s ARCHZBM_ZPOOL_PASSWORD
echo
echo
fi
fi
echo 'Do you want a custom '"'"'root'"'"' user password?'
select arg_custom_root_pw in 'Yes' 'No (use '"'"'password'"'"')'; do
case "${arg_custom_root_pw}" in
'Yes')
want_custom_root_pw='true'
break
;;
'No (use '"'"'password'"'"')')
break
;;
esac
done <&3 && echo
if [[ "${want_custom_root_pw}" ]]; then
read -u3 -p 'Please type password, confirm with <Enter>: ' -s ARCHZBM_ROOT_PASSWORD
echo
echo
fi
echo 'Do you want an SSH daemon in ZFSBootMenu?'
select arg_ssh_in_zbm in "Yes" "No"; do
case "${arg_ssh_in_zbm}" in
Yes)
want_ssh_in_zbm='true'
break
;;
No)
break
;;
esac
done <&3 && echo
if [[ "${want_ssh_in_zbm}" ]]; then
echo 'How do you want to assign an IP address in ZFSBootMenu?'
select arg_ip_autoconf_method in 'Statically' 'Dynamically, DHCP' 'Dynamically, BOOTP' 'Dynamically, RARP'; do
case "${arg_ip_autoconf_method}" in
'Statically')
ARCHZBM_NET_AUTOCONF='none'
break
;;
'Dynamically, DHCP')
ARCHZBM_NET_AUTOCONF='dhcp'
break
;;
'Dynamically, BOOTP')
ARCHZBM_NET_AUTOCONF='bootp'
break
;;
'Dynamically, RARP')
ARCHZBM_NET_AUTOCONF='rarp'
break
;;
esac
done <&3 && echo
read -u3 -p 'Which device to configure (eth0 is a good guess): ' ARCHZBM_NET_DEVICE
echo
if [[ "${arg_ip_autoconf_method}" = 'Statically' ]]; then
read -u3 -p 'Interface IP address: ' ARCHZBM_NET_CLIENT_IP
echo
read -u3 -p 'Netmask: ' ARCHZBM_NET_NETMASK
echo
read -u3 -p 'Gateway IP address: ' ARCHZBM_NET_GATEWAY_IP
echo
fi
echo 'Do you want a custom SSH listening port?'
select arg_custom_ssh_port in 'Yes (let me specify)' 'No (keep port 22)'; do
case "${arg_custom_ssh_port}" in
'Yes (let me specify)')
want_custom_ssh_port='true'
break
;;
'No (keep port 22)')
break
;;
esac
done <&3 && echo
if [[ "${want_custom_ssh_port}" ]]; then
read -u3 -p 'Please type SSH port, confirm with <Enter>: ' ARCHZBM_SSH_PORT
echo
fi
echo 'Do you want the SSH daemon to use a custom keepalive send interval?'
select arg_custom_ssh_keepalive_intvl in 'Yes (let me specify)' 'No (keep 1)'; do
case "${arg_custom_ssh_keepalive_intvl}" in
'Yes (let me specify)')
want_custom_keepalive_intvl='true'
break
;;
'No (keep 1)')
break
;;
esac
done <&3 && echo
if [[ "${want_custom_keepalive_intvl}" ]]; then
read -u3 -p 'Please type server keepalive send interval, confirm with <Enter>: ' ARCHZBM_SSH_KEEPALIVE_INTVL
echo
fi
read -u3 -p 'Please type SSH pub keys on one line separated by double-commas (,,) and confirm with <Enter>: ' ARCHZBM_SSH_AUTH_KEYS
echo
fi
if [[ "${want_ssh_in_zbm}" ]]; then
echo 'Do you want to define operating system'"'"'s IP address?'
select arg_os_ip in 'Yes (let me specify)' 'Yes (use ZBM addresses)' 'No (DHCP is fine)'; do
case "${arg_os_ip}" in
'Yes (let me specify)')
want_custom_ip_in_os='true'
want_dns_and_ntp='true'
break
;;
'Yes (use ZBM addresses)')
ARCHZBM_OS_CLIENT_IP="${ARCHZBM_NET_CLIENT_IP}"
ARCHZBM_NET_CLIENT_IP_PREFIX="$(calculate_prefix_from_netmask "${ARCHZBM_NET_NETMASK}")"
ARCHZBM_OS_CLIENT_IP+="${ARCHZBM_NET_CLIENT_IP_PREFIX}"
ARCHZBM_OS_GATEWAY_IP="${ARCHZBM_NET_GATEWAY_IP}"
want_dns_and_ntp='true'
break
;;
'No (DHCP is fine)')
break
;;
esac
done <&3 && echo
if [[ "${want_custom_ip_in_os}" ]]; then
read -u3 -p 'Interface IP address with CIDR prefix (a.b.c.d/nn): ' ARCHZBM_OS_CLIENT_IP
echo
read -u3 -p 'Gateway IP address: ' ARCHZBM_OS_GATEWAY_IP
echo
fi
else
echo 'Do you want to define operating system'"'"'s IP address?'
select arg_os_ip in 'Yes (let me specify)' 'No (DHCP is fine)'; do
case "${arg_os_ip}" in
'Yes (let me specify)')
want_own_ip_in_os='true'
want_dns_and_ntp='true'
break
;;
'No (DHCP is fine)')
break
;;
esac
done <&3 && echo
if [[ "${want_own_ip_in_os}" ]]; then
read -u3 -p 'Interface IP address with CIDR prefix (a.b.c.d/nn): ' ARCHZBM_OS_CLIENT_IP
echo
read -u3 -p 'Gateway IP address: ' ARCHZBM_OS_GATEWAY_IP
echo
fi
fi
if [[ "${want_ssh_in_zbm}" ]]; then
echo 'Do you want to define OS '"'"'root'"'"' user'"'"'s SSH pub key?'
select arg_root_pub_keys in 'Yes (let me specify)' 'Yes (use ZBM pub keys)' 'No (don'"'"'t enable sshd.service)'; do
case "${arg_root_pub_keys}" in
'Yes (let me specify)')
want_custom_pub_keys_in_os='true'
break
;;
'Yes (use ZBM pub keys)')
ARCHZBM_OS_SSH_AUTH_KEYS="${ARCHZBM_SSH_AUTH_KEYS}"
break
;;
'No (don'"'"'t enable sshd.service)')
break
;;
esac
done <&3 && echo
if [[ "${want_custom_pub_keys_in_os}" ]]; then
read -u3 -p 'Please type SSH pub keys on one line separated by double-commas (,,) and confirm with <Enter>: ' ARCHZBM_OS_SSH_AUTH_KEYS
echo
fi
else
echo 'Do you want to define OS root user'"'"'s SSH pub key?'
select arg_root_pub_keys in 'Yes (let me specify)' 'No (don'"'"'t enable sshd.service)'; do
case "${arg_root_pub_keys}" in
'Yes (let me specify)')
want_own_pub_key_in_os='true'
break
;;
'No (don'"'"'t enable sshd.service)')
break
;;
esac
done <&3 && echo
if [[ "${want_own_pub_key_in_os}" ]]; then
read -u3 -p 'Please type SSH pub keys on one line separated by double-commas (,,) and confirm with <Enter>: ' ARCHZBM_OS_SSH_AUTH_KEYS
echo
fi
fi
if [[ "${want_dns_and_ntp}" ]]; then
read -u3 -p 'Specify one or more comma-separated DNS IPs: ' ARCHZBM_OS_DNS_IP
echo
echo 'Do you want to override Arch Linux'"'"' NTP servers?'
select arg_custom_ntp in 'Yes' 'No'; do
case "${arg_custom_ntp}" in
'Yes')
want_own_ntp='true'
break
;;
'No')
break
;;
esac
done <&3 && echo
if [[ "${want_own_ntp}" ]]; then
read -u3 -p 'Specify one or more comma-separated NTP hostnames or IPs: ' ARCHZBM_OS_NTP_IP
echo
fi
fi
for env_var in 'ARCHZBM_KERNEL_VER' 'ARCHZBM_ZFSPROPS_NO_COMPRESSION' 'ARCHZBM_ZFSPROPS_NO_ENCRYPTION' 'ARCHZBM_ZPOOL_PASSWORD' 'ARCHZBM_ROOT_PASSWORD' 'ARCHZBM_NET_AUTOCONF' 'ARCHZBM_NET_DEVICE' 'ARCHZBM_NET_CLIENT_IP' 'ARCHZBM_NET_NETMASK' 'ARCHZBM_NET_GATEWAY_IP' 'ARCHZBM_SSH_PORT' 'ARCHZBM_SSH_KEEPALIVE_INTVL' 'ARCHZBM_SSH_AUTH_KEYS' 'ARCHZBM_OS_CLIENT_IP' 'ARCHZBM_OS_GATEWAY_IP' 'ARCHZBM_OS_SSH_AUTH_KEYS' 'ARCHZBM_OS_DNS_IP' 'ARCHZBM_OS_NTP_IP'; do
if [[ "${!env_var}" ]]; then
printf -- '%s='"'"'%s'"'"'\n' \
"${env_var}" "${!env_var}" \
>> "${settings_file}"
fi
done
printf -- '%s\n' \
'Done, please rerun script now with just' \
'... | bash' \
'so without the '"'"'setup'"'"' argument'
exit 77
}
function arg_parse () {
[[ "${1}" ]] && while :; do
case "${1}" in
-[[:alnum:]]*)
>&3 printf -- '%s\n' \
'Short options '"'${1}'"' detected. Only known option is' \
'literal string '"'"'setup'"'"'. No idea what '"'${1}'"' is.' \
'Exiting ...'
exit 77
;;
--*)
>&3 printf -- '%s\n' \
'Long-form option '"'${1}'"' detected. Only known option is' \
'literal string '"'"'setup'"'"'. No idea what '"'${1}'"' is.' \
'Exiting ...'
exit 77
;;
setup)
setup_env_vars
return
;;
*)
>&3 printf -- '%s\n' \
'Argument '"'${1}'"' detected. Only known option is literal' \
'string '"'"'setup'"'"'. No idea what '"'${1}'"' is.' \
'Exiting ...'
exit 77
;;
esac
done
}
function we_are_changerooted () { function we_are_changerooted () {
if [ "$(stat -c '%d:%i' '/')" != "$(stat -c '%d:%i' '/proc/1/root/.')" ]; then if [ "$(stat -c '%d:%i' '/')" != "$(stat -c '%d:%i' '/proc/1/root/.')" ]; then
return 0 return 0
@@ -71,19 +429,34 @@ function update_pacman_db () {
systemctl start reflector systemctl start reflector
# In an ISO and for the minimal number of packages we need we do not # In an ISO and for the minimal number of packages we need we do not
# care about partial upgrades # care about partial upgrades
pacman -Syyuu --noconfirm #
# Are we better off not attempting an upgrade inside the ISO?
# Let's try and find out.
# while ! pacman -Syyuu --needed --noconfirm --downloadonly; do
# sleep 5
# done
# pacman -Syyuu --needed --noconfirm
pacman -Syy
} }
function install_pkgs () { function install_pkgs () {
#1.5 #1.5
printf -- '%s\n' 'Installing packages ...' printf -- '%s\n' 'Installing packages ...'
while ! pacman -S --needed --noconfirm --downloadonly "${@}"; do
sleep 5
done
pacman -S --needed --noconfirm "${@}" pacman -S --needed --noconfirm "${@}"
} }
function install_zfs () { function install_zfs () {
#1.6 #1.6
declare reset_colors='\033[0m' declare reset_colors='\033[0m'
curl -s 'https://raw.githubusercontent.com/eoli3n/archiso-zfs/master/init' | bash if modinfo 'zfs' &>/dev/null; then
>&3 printf -- '%s\n' \
'ZFS kernel module is loaded, no need to install ...'
else
curl -s 'https://raw.githubusercontent.com/eoli3n/archiso-zfs/master/init' | bash
fi
printf -- "${reset_colors}" printf -- "${reset_colors}"
} }
@@ -328,9 +701,8 @@ function export_pool () {
function load_settings_file () { function load_settings_file () {
#1.8 #1.8
local working_dir settings_file settings_abs local working_dir settings_abs
working_dir="$(pwd)" working_dir="$(pwd)"
settings_file='archzbm_settings.env'
settings_abs="${working_dir}"'/'"${settings_file}" settings_abs="${working_dir}"'/'"${settings_file}"
if [[ -r "${settings_abs}" ]]; then if [[ -r "${settings_abs}" ]]; then
set -a set -a
@@ -406,27 +778,29 @@ function install_archlinux () {
#1.12 #1.12
pacman_dl_parallel pacman_dl_parallel
pacman_dont_check_space pacman_dont_check_space
pacstrap /mnt \ while ! pacstrap /mnt \
base \ base \
base-devel \ base-devel \
linux \ linux \
linux-headers \ linux-headers \
linux-firmware \ linux-firmware \
amd-ucode \ amd-ucode \
efibootmgr \ efibootmgr \
vim \ vim \
git \ git \
iwd \ iwd \
networkmanager \ networkmanager \
network-manager-applet \ network-manager-applet \
dialog \ dialog \
os-prober \ os-prober \
reflector \ reflector \
bluez \ bluez \
bluez-utils \ bluez-utils \
man-db \ man-db \
xdg-utils \ xdg-utils \
xdg-user-dirs xdg-user-dirs; do
sleep 5
done
} }
function gen_fstab () { function gen_fstab () {
@@ -468,6 +842,29 @@ function in_file_in_array_insert_n_before_m () {
arg_array="${2:?}" arg_array="${2:?}"
arg_string="${3:?}" arg_string="${3:?}"
arg_precede="${4:?}" arg_precede="${4:?}"
# Look for a line that contains in this order
# - String "${arg_array}" and a equals sign (=), assign capture group \1
# - Followed by as few characters as possible followed by either an
# opening parenthesis or a space character, assign capture group \2
# - String "${arg_precede}", capture as capture group \3
# - Followed by either a closing parenthesis or a space which are then
# followed by as many non-line break characters as possible, capture
# as capture group \2
#
# For following example text we're assuming that:
# - "${arg_array}" equals 'HOOKS'
# - "${arg_precede}" equals 'filesystems'
# - "${arg_string}" equals 'zfs'
#
# This finds a 'HOOKS=' array definition that contains the string
# 'filesystems' either at the beginning of the 'HOOKS=(...)' opening
# parenthesis, at the very end or somewhere in the middle where it may
# be preceded or followed by one or more space characters. It saves
# 'HOOKS=', it saves whatever precedes 'filesystems' and 'filesystems'
# itself plus whatever comes after 'filesystems' until end of line. It
# lastly inserts 'zfs' and a space character right in front of
# 'filesystems'.
sed -ri \ sed -ri \
-e 's'$'\x1''('"${arg_array}"'=)(.*?[( ])('"${arg_precede}"')([) ][^\r\n\f]*)'$'\x1''\1\2'"${arg_string}"' \3\4'$'\x1''g' \ -e 's'$'\x1''('"${arg_array}"'=)(.*?[( ])('"${arg_precede}"')([) ][^\r\n\f]*)'$'\x1''\1\2'"${arg_string}"' \3\4'$'\x1''g' \
"${arg_file}" "${arg_file}"
@@ -478,6 +875,21 @@ function in_file_in_array_insert_n_at_the_end () {
arg_file="${1:?}" arg_file="${1:?}"
arg_array="${2:?}" arg_array="${2:?}"
arg_string="${3:?}" arg_string="${3:?}"
# Look for end of array, insert "${arg_string}" right before
#
# For following example text we're assuming that:
# - "${arg_array}" equals 'HOOKS'
# - "${arg_string}" equals 'net'
#
# This:
# - Finds a 'HOOKS=' array definition, saves it as \1
# - Finds as many non-closing parenthesis characters as possible, so all
# characters until just before the final ')' character of the line and
# saves this as \2
# - Finds the closing parenthesis character plus all non-line break
# characters until end of line that come after it and saves this as \3
# - Adds a space character and 'net' after \2
sed -ri \ sed -ri \
-e 's'$'\x1''('"${arg_array}"'=)([^)]*)(\)[^\r\n\f]*)'$'\x1''\1\2 '"${arg_string}"'\3'$'\x1''g' \ -e 's'$'\x1''('"${arg_array}"'=)([^)]*)(\)[^\r\n\f]*)'$'\x1''\1\2 '"${arg_string}"'\3'$'\x1''g' \
"${arg_file}" "${arg_file}"
@@ -488,6 +900,23 @@ function in_file_in_array_remove_n () {
arg_file="${1:?}" arg_file="${1:?}"
arg_array="${2:?}" arg_array="${2:?}"
arg_string="${3:?}" arg_string="${3:?}"
# Look for any line that contains "${arg_string}", delete that string
#
# For following example text we're assuming that:
# - "${arg_array}" equals 'HOOKS'
# - "${arg_string}" equals 'fsck'
#
# First -e expression removes 'fsck' wherever it is defined as the one
# and only element of any HOOKS=(fsck) should such a line exist.
#
# Second -e expression finds string 'fsck' where it's preceded by space
# character(s) and followed by either space character(s) or a closing
# parenthesis.
#
# Third -e expression finds string 'fsck' where it's preceded by space
# character(s) or an opening parenthesis and followed space
# character(s).
sed -ri \ sed -ri \
-e 's'$'\x1''((\()('"${arg_string}"')(\)))'$'\x1''\2\4'$'\x1''g' \ -e 's'$'\x1''((\()('"${arg_string}"')(\)))'$'\x1''\2\4'$'\x1''g' \
-e 's'$'\x1''('"${arg_array}"'=.*?)([[:space:]]+'"${arg_string}"')([[:space:]]+|\))'$'\x1''\1\3'$'\x1''g' \ -e 's'$'\x1''('"${arg_array}"'=.*?)([[:space:]]+'"${arg_string}"')([[:space:]]+|\))'$'\x1''\1\3'$'\x1''g' \
@@ -622,56 +1051,17 @@ function get_aur_helper () {
usermod --append --groups 'wheel' 'build' usermod --append --groups 'wheel' 'build'
printf -- '%s\n' '%wheel ALL=(ALL:ALL) NOPASSWD: ALL' > '/etc/sudoers.d/10-wheel-group-no-passwd-prompt' printf -- '%s\n' '%wheel ALL=(ALL:ALL) NOPASSWD: ALL' > '/etc/sudoers.d/10-wheel-group-no-passwd-prompt'
pushd /tmp pushd /tmp
git clone 'https://aur.archlinux.org/paru.git' git clone https://aur.archlinux.org/paru-bin.git
chown -R 'build:' 'paru' chown -R 'build:' 'paru-bin'
pushd 'paru' pushd 'paru-bin'
sudo --user 'build' makepkg -si --noconfirm sudo --user 'build' makepkg -si --noconfirm
popd popd
rm -rf 'paru' rm -rf 'paru-bin'
popd popd
} }
function paru_install () { function paru_install () {
declare -a paru_install_packages sudo --user build paru -S --noconfirm "${@}"
[[ "${1}" ]] && while :; do
case "${1}" in
-[[:alnum:]]*)
>&3 printf -- '%s\n' \
'Short-form argument '"'${1}'"' not supported for function '"'${FUNCNAME[0]}()'"'. Only known accepted argument' \
'is '"'"'--replace-conflicting'"'"' without a value given. Exiting ...'
exit 77
;;
--replace-conflicting)
pacman_force_yes='true'
shift
continue
;;
--*)
>&3 printf -- '%s\n' \
'Long-form argument '"'${1}'"' not supported for function '"'${FUNCNAME[0]}()'"'. Only known accepted argument' \
'is '"'"'--replace-conflicting'"'"' without a value given. Exiting ...'
exit 77
;;
'')
# All arguments processed
break
;;
*)
paru_install_packages+=("${1}")
shift
;;
esac
done || {
>&3 printf -- '%s\n' \
'No argument '"'${1}'"' given for function '"'${FUNCNAME[0]}'"'. Exiting ...'
exit 77
}
if [[ "${pacman_force_yes}" ]]; then
yes 'y' | sudo --user 'build' paru -S "${paru_install_packages[@]}"
unset -v pacman_force_yes
else
sudo --user 'build' paru -S --noconfirm "${paru_install_packages[@]}"
fi
} }
function configure_syslinux () { function configure_syslinux () {
@@ -719,7 +1109,6 @@ EFI:
ImageDir: /efi/EFI/ZBM ImageDir: /efi/EFI/ZBM
Versions: false Versions: false
Enabled: true Enabled: true
Stub: /usr/share/zfsbootmenu/stubs/linuxx64.efi.stub/linuxx64.efi.stub # workaround: https://github.com/zbm-dev/zfsbootmenu/discussions/501
Kernel: Kernel:
CommandLine: ro loglevel=0 zbm.import_policy=hostid CommandLine: ro loglevel=0 zbm.import_policy=hostid
Prefix: vmlinuz Prefix: vmlinuz
@@ -741,7 +1130,11 @@ EOF
fi fi
# Up here maybe 'ro quiet' instead of 'ro'. This is ZFSBootMenu's kernel # Up here maybe 'ro quiet' instead of 'ro'. This is ZFSBootMenu's kernel
# command line. # command line. In MBR/Syslinux/extlinux mode /it/ must pass arguments to
# ZFSBootMenu's kernel so check /boot/syslinux/syslinux.cfg for how we start
# ZFSBootMenu in this mode. The .Kernel.CommandLine in
# /etc/zfsbootmenu/config.yaml is irrelevant for us in MBR/Syslinux/extlinux
# mode.
# Assign cmdline for final kernel start. This is our Arch Linx kernel # Assign cmdline for final kernel start. This is our Arch Linx kernel
# command line. # command line.
@@ -751,6 +1144,8 @@ EOF
function get_dropbear_hooks () { function get_dropbear_hooks () {
mkdir -p '/opt/git/quico.space/quico-os-setup/mkinitcpio-dropbear-pacman-hook/branches/main' mkdir -p '/opt/git/quico.space/quico-os-setup/mkinitcpio-dropbear-pacman-hook/branches/main'
git -C '/opt/git/quico.space/quico-os-setup/mkinitcpio-dropbear-pacman-hook/branches/main' clone 'https://quico.space/quico-os-setup/mkinitcpio-dropbear-pacman-hook.git' . git -C '/opt/git/quico.space/quico-os-setup/mkinitcpio-dropbear-pacman-hook/branches/main' clone 'https://quico.space/quico-os-setup/mkinitcpio-dropbear-pacman-hook.git' .
ln -s '/opt/git/quico.space/quico-os-setup/mkinitcpio-dropbear-pacman-hook/branches/main/pacman-mkinitcpio-dropbear-hook.sh' '/usr/local/bin/pacman-mkinitcpio-dropbear-hook'
ln -s '/opt/git/quico.space/quico-os-setup/mkinitcpio-dropbear-pacman-hook/branches/main/pacman-mkinitcpio-dropbear-install.sh' '/usr/local/bin/pacman-mkinitcpio-dropbear-install'
ln -s '/opt/git/quico.space/quico-os-setup/mkinitcpio-dropbear-pacman-hook/branches/main/pacman-mkinitcpio-dropbear-hook.hook' '/usr/share/libalpm/hooks/pacman-mkinitcpio-dropbear-hook.hook' ln -s '/opt/git/quico.space/quico-os-setup/mkinitcpio-dropbear-pacman-hook/branches/main/pacman-mkinitcpio-dropbear-hook.hook' '/usr/share/libalpm/hooks/pacman-mkinitcpio-dropbear-hook.hook'
ln -s '/opt/git/quico.space/quico-os-setup/mkinitcpio-dropbear-pacman-hook/branches/main/pacman-mkinitcpio-dropbear-install.hook' '/usr/share/libalpm/hooks/pacman-mkinitcpio-dropbear-install.hook' ln -s '/opt/git/quico.space/quico-os-setup/mkinitcpio-dropbear-pacman-hook/branches/main/pacman-mkinitcpio-dropbear-install.hook' '/usr/share/libalpm/hooks/pacman-mkinitcpio-dropbear-install.hook'
} }
@@ -771,43 +1166,70 @@ function customize_dropbear_hooks () {
fi fi
} }
function set_unique_ip_in_syslinux_kcl () {
local zbm_config default_ip
zbm_config="${1:?}"
default_ip="${2:?}"
# First -e expression removes first looks for lines that contain
# 'APPEND' plus a space character and only in those lines removes all
# occurrences of ' ip=' followed by as many non-space characters as
# possible. This removes whatever 'ip=' definition already was present.
#
# Second -e expression similarly looks for lines that contain 'APPEND'
# plus a space character then at their end inserts a space plus our
# desired new 'ip=' definition. This puts in place the 'ip=' we want.
sed -ri \
-e \\$'\x1''APPEND '$'\x1 s'$'\x1'' ip=([^[:space:]]*)'$'\x1'''$'\x1''gi' \
-e \\$'\x1''APPEND '$'\x1 s'$'\x1''$'$'\x1'' '"${default_ip}"''$'\x1''gi' \
"${zbm_config}"
}
function ensure_ip_in_kcl () { function ensure_ip_in_kcl () {
local zbm_config kcl_length kcl_string default_ip ip_addr_found new_kcl first_kcl_elem local default_ip
local -a kcl
paru_install 'go-yq'
zbm_config='/etc/zfsbootmenu/config.yaml'
kcl_length="$(yq '.Kernel.CommandLine | length' "${zbm_config}")"
if [[ "${kcl_length}" -eq '0' ]]; then
>&3 printf -- '%s\n' \
'No .Kernel.CommandLine YAML element with content found in '"${zbm_config}"'. Exiting ...'
exit 77
else
kcl_string="$(yq '.Kernel.CommandLine' "${zbm_config}")"
fi
default_ip='ip='"${ARCHZBM_NET_CLIENT_IP}"':'"${ARCHZBM_NET_SERVER_IP}"':'"${ARCHZBM_NET_GATEWAY_IP}"':'"${ARCHZBM_NET_NETMASK}"':'"${ARCHZBM_NET_HOSTNAME}"':'"${ARCHZBM_NET_DEVICE}"':'"${ARCHZBM_NET_AUTOCONF}" default_ip='ip='"${ARCHZBM_NET_CLIENT_IP}"':'"${ARCHZBM_NET_SERVER_IP}"':'"${ARCHZBM_NET_GATEWAY_IP}"':'"${ARCHZBM_NET_NETMASK}"':'"${ARCHZBM_NET_HOSTNAME}"':'"${ARCHZBM_NET_DEVICE}"':'"${ARCHZBM_NET_AUTOCONF}"
mapfile -t kcl < <(<<<"${kcl_string}" tr ' ' '\n' | sed '/^$/d')
for kcl_elem in "${!kcl[@]}"; do if [[ "${part_schema}" = 'gpt' ]]; then
if grep -Piq -- 'ip=' <<<"${kcl[$kcl_elem]}"; then local zbm_config kcl_length kcl_string ip_addr_found new_kcl first_kcl_elem
ip_addr_found='true' local -a kcl
kcl["${kcl_elem}"]="${default_ip}" paru_install 'go-yq'
fi
done zbm_config='/etc/zfsbootmenu/config.yaml'
if [[ ! "${ip_addr_found}" ]]; then kcl_length="$(yq '.Kernel.CommandLine | length' "${zbm_config}")"
kcl+=("${default_ip}") if [[ "${kcl_length}" -eq '0' ]]; then
fi >&3 printf -- '%s\n' \
new_kcl='' 'No .Kernel.CommandLine YAML element with content found in '"${zbm_config}"'. Exiting ...'
first_kcl_elem='true' exit 77
for kcl_elem in "${kcl[@]}"; do
if [[ ! "${first_kcl_elem}" ]]; then
new_kcl+=' '"${kcl_elem}"
else else
new_kcl+="${kcl_elem}" kcl_string="$(yq '.Kernel.CommandLine' "${zbm_config}")"
unset -v first_kcl_elem
fi fi
done
yq -i '.Kernel.CommandLine = "'"${new_kcl}"'"' "${zbm_config}" mapfile -t kcl < <(<<<"${kcl_string}" tr ' ' '\n' | sed '/^$/d')
for kcl_elem in "${!kcl[@]}"; do
if grep -Piq -- 'ip=' <<<"${kcl[$kcl_elem]}"; then
ip_addr_found='true'
kcl["${kcl_elem}"]="${default_ip}"
fi
done
if [[ ! "${ip_addr_found}" ]]; then
kcl+=("${default_ip}")
fi
new_kcl=''
first_kcl_elem='true'
for kcl_elem in "${kcl[@]}"; do
if [[ ! "${first_kcl_elem}" ]]; then
new_kcl+=' '"${kcl_elem}"
else
new_kcl+="${kcl_elem}"
unset -v first_kcl_elem
fi
done
yq -i '.Kernel.CommandLine = "'"${new_kcl}"'"' "${zbm_config}"
else
local zbm_config
zbm_config='/boot/syslinux/syslinux.cfg'
set_unique_ip_in_syslinux_kcl "${zbm_config}" "${default_ip}"
fi
} }
function set_pub_keys () { function set_pub_keys () {
@@ -883,6 +1305,10 @@ function get_disks_with_one_efipart () {
function install_os_in_chroot () { function install_os_in_chroot () {
#2.2 #2.2
dd if='/dev/zero' of='/swapfile' bs='1M' count='2048'
losetup '/dev/loop9' '/swapfile'
mkswap '/dev/loop9'
swapon '/dev/loop9'
### Reinit keyring ### Reinit keyring
# As keyring is initialized at boot, and copied to the install dir with pacstrap, and ntp is running # As keyring is initialized at boot, and copied to the install dir with pacstrap, and ntp is running
# Time changed after keyring initialization, it leads to malfunction # Time changed after keyring initialization, it leads to malfunction
@@ -890,6 +1316,9 @@ function install_os_in_chroot () {
rm -rf '/etc/pacman.d/gnupg' rm -rf '/etc/pacman.d/gnupg'
pacman-key --init pacman-key --init
pacman-key --populate archlinux pacman-key --populate archlinux
while ! pacman -S archlinux-keyring --noconfirm --downloadonly; do
sleep 5
done
pacman -S archlinux-keyring --noconfirm pacman -S archlinux-keyring --noconfirm
locale-gen locale-gen
@@ -900,7 +1329,13 @@ function install_os_in_chroot () {
unleash_makepkg #2.5 unleash_makepkg #2.5
add_motd_getting_started_msg #2.6 add_motd_getting_started_msg #2.6
get_aur_helper #2.7 get_aur_helper #2.7
paru_install --replace-conflicting 'paru-bin' if [[ "${ARCHZBM_KERNEL_VER}" ]]; then
paru_install 'downgrade'
yes | downgrade --ala-only \
'linux='"${ARCHZBM_KERNEL_VER}" \
'linux-headers='"${ARCHZBM_KERNEL_VER}" \
--ignore always
fi
paru_install 'zfs-dkms' 'zfs-utils' 'jq' paru_install 'zfs-dkms' 'zfs-utils' 'jq'
hwclock --systohc hwclock --systohc
mkinitcpio -P mkinitcpio -P
@@ -920,6 +1355,9 @@ function install_os_in_chroot () {
add_syslinux_pacman_hook add_syslinux_pacman_hook
fi fi
add_zbm_pacman_hook add_zbm_pacman_hook
swapoff '/dev/loop9'
losetup -d '/dev/loop9'
rm '/swapfile'
} }
function set_root_pw () { function set_root_pw () {
@@ -935,11 +1373,54 @@ function set_root_pw () {
function configure_networking () { function configure_networking () {
#3.3 #3.3
local -a dns_addresses ntp_addresses
# Begin network unit file with a default top section
cat > '/mnt/etc/systemd/network/50-wired.network' <<"EOF" cat > '/mnt/etc/systemd/network/50-wired.network' <<"EOF"
[Match] [Match]
Name=en* Name=en*
[Network] [Network]
EOF
# Decide on what comes next in network unit file
if [[ "${ARCHZBM_OS_CLIENT_IP}" ]] || \
[[ "${ARCHZBM_OS_GATEWAY_IP}" ]] || \
[[ "${ARCHZBM_OS_DNS_IP}" ]] || \
[[ "${ARCHZBM_OS_NTP_IP}" ]]; then
cat >> '/mnt/etc/systemd/network/50-wired.network' <<EOF
Address=${ARCHZBM_OS_CLIENT_IP}
Gateway=${ARCHZBM_OS_GATEWAY_IP}
EOF
if [[ "${ARCHZBM_OS_DNS_IP}" ]]; then
mapfile -t dns_addresses < <(<<<"${ARCHZBM_OS_DNS_IP}" tr ',' '\n' | sed '/^$/d')
else
dns_addresses+=('8.8.8.8')
dns_addresses+=('8.8.4.4')
fi
for dns_addr in "${dns_addresses[@]}"; do
cat >> '/mnt/etc/systemd/network/50-wired.network' <<EOF
DNS=${dns_addr}
EOF
done
if [[ "${ARCHZBM_OS_NTP_IP}" ]]; then
mapfile -t ntp_addresses < <(<<<"${ARCHZBM_OS_NTP_IP}" tr ',' '\n' | sed '/^$/d')
for ntp_addr in "${ntp_addresses[@]}"; do
cat >> '/mnt/etc/systemd/network/50-wired.network' <<EOF
NTP=${ntp_addr}
EOF
done
fi
cat >> '/mnt/etc/systemd/network/50-wired.network' <<"EOF"
IPForward=yes
Domains=~.
EOF
else
cat >> '/mnt/etc/systemd/network/50-wired.network' <<"EOF"
DHCP=ipv4 DHCP=ipv4
IPForward=yes IPForward=yes
@@ -947,33 +1428,56 @@ IPForward=yes
UseDNS=yes UseDNS=yes
RouteMetric=10 RouteMetric=10
EOF EOF
fi
systemctl enable 'systemd-networkd' --root='/mnt' systemctl enable 'systemd-networkd' --root='/mnt'
systemctl disable 'systemd-networkd-wait-online' --root='/mnt' systemctl disable 'systemd-networkd-wait-online' --root='/mnt'
} }
function configure_dns () { function configure_sshd () {
#3.4 #3.4
local pub_key_line
cat >> '/mnt/etc/ssh/sshd_config.d/40-defaults.conf' <<"EOF"
PasswordAuthentication no
PermitRootLogin yes
EOF
while IFS= read -r pub_key_line; do
printf -- '%s\n' "${pub_key_line}" >> '/mnt/root/.ssh/authorized_keys'
done < <(<<<"${ARCHZBM_OS_SSH_AUTH_KEYS}" sed -r -e 's/,,/\n/g')
systemctl enable 'sshd.service' --root='/mnt'
}
function configure_dns () {
#3.5
rm '/mnt/etc/resolv.conf' rm '/mnt/etc/resolv.conf'
ln -s '/run/systemd/resolve/stub-resolv.conf' '/mnt/etc/resolv.conf' ln -s '/run/systemd/resolve/stub-resolv.conf' '/mnt/etc/resolv.conf'
# Optionally you may want /etc/systemd/network/50-wired.network to use # Optionally you may want /etc/systemd/network/50-wired.network to use
# UseDNS=no and hardcode DNS server(s) here: # UseDNS=no and hardcode DNS server(s) here:
# sed -i 's/^#DNS=.*/DNS=1.1.1.1/' /mnt/etc/systemd/resolved.conf # sed -i 's/^#DNS=.*/DNS=1.1.1.1/' /mnt/etc/systemd/resolved.conf
systemctl enable 'systemd-resolved' --root='/mnt' systemctl enable 'systemd-resolved.service' --root='/mnt'
}
function configure_ntp () {
#3.6
systemctl enable 'systemd-timesyncd.service' --root='/mnt'
} }
function configure_reflector () { function configure_reflector () {
#3.5 #3.7
systemctl enable 'reflector.service' 'reflector.timer' --root='/mnt' systemctl enable 'reflector.service' 'reflector.timer' --root='/mnt'
} }
function configure_zfs () { function configure_zfs () {
#3.6 #3.8
systemctl enable 'zfs-import-cache' 'zfs-mount' 'zfs-import.target' 'zfs.target' --root='/mnt' systemctl enable 'zfs-import-cache.service' 'zfs-mount.service' 'zfs-import.target' 'zfs.target' --root='/mnt'
} }
function configure_zfs_mount_gen () { function configure_zfs_mount_gen () {
#3.7 #3.9
mkdir -p '/mnt/etc/zfs/zfs-list.cache' mkdir -p '/mnt/etc/zfs/zfs-list.cache'
touch '/mnt/etc/zfs/zfs-list.cache/'"${zpool_name}" touch '/mnt/etc/zfs/zfs-list.cache/'"${zpool_name}"
zfs list -H -o name,mountpoint,canmount,atime,relatime,devices,exec,readonly,setuid,nbmand | sed 's/\/mnt//' > '/mnt/etc/zfs/zfs-list.cache/'"${zpool_name}" zfs list -H -o name,mountpoint,canmount,atime,relatime,devices,exec,readonly,setuid,nbmand | sed 's/\/mnt//' > '/mnt/etc/zfs/zfs-list.cache/'"${zpool_name}"
@@ -981,7 +1485,7 @@ function configure_zfs_mount_gen () {
} }
function set_new_uefi_boot_entries () { function set_new_uefi_boot_entries () {
#3.8 #3.10
declare -a uefi_images declare -a uefi_images
mapfile -t uefi_images < \ mapfile -t uefi_images < \
<(find '/mnt/efi/EFI/ZBM' -type f -iname '*.efi' -print0 | \ <(find '/mnt/efi/EFI/ZBM' -type f -iname '*.efi' -print0 | \
@@ -1024,7 +1528,7 @@ function set_new_uefi_boot_entries () {
} }
function umount_all () { function umount_all () {
#3.9 #3.11
if [[ "${part_schema}" = 'mbr' ]]; then if [[ "${part_schema}" = 'mbr' ]]; then
umount '/mnt/boot/syslinux' umount '/mnt/boot/syslinux'
else else
@@ -1036,44 +1540,51 @@ function umount_all () {
function finalize_os_setup () { function finalize_os_setup () {
#3.1 #3.1
set_root_pw #3.2 set_root_pw #3.2
configure_networking #3.3 configure_networking #3.3
configure_dns #3.4 if [[ "${ARCHZBM_OS_SSH_AUTH_KEYS}" ]]; then
configure_reflector #3.5 configure_sshd #3.4
configure_zfs #3.6
configure_zfs_mount_gen #3.7
if [[ "${part_schema}" = 'gpt' ]]; then
set_new_uefi_boot_entries #3.8
fi fi
umount_all #3.9 configure_dns #3.5
configure_ntp #3.6
configure_reflector #3.7
configure_zfs #3.8
configure_zfs_mount_gen #3.9
if [[ "${part_schema}" = 'gpt' ]]; then
set_new_uefi_boot_entries #3.10
fi
umount_all #3.11
} }
function main () { function main () {
if [[ "${#@}" -gt '0' ]]; then
arg_parse "${@}"
fi
if we_are_changerooted; then if we_are_changerooted; then
install_os_in_chroot #2.2 install_os_in_chroot #2.2
else else
no_kernel_update_in_iso #1.1 no_kernel_update_in_iso #1.1
set_ntp #1.2 set_ntp #1.2
resize_cow_space #1.3 resize_cow_space #1.3
update_pacman_db #1.4 update_pacman_db #1.4
install_pkgs 'jq' #1.5 install_pkgs 'jq' #1.5
install_zfs #1.6 install_zfs #1.6
uefi_or_bios #1.7 uefi_or_bios #1.7
load_settings_file #1.8 load_settings_file #1.8
setup_zpool #1.9 setup_zpool #1.9
mount_system #1.10 mount_system #1.10
copy_zpool_cache #1.11 copy_zpool_cache #1.11
install_archlinux #1.12 install_archlinux #1.12
gen_fstab #1.13 gen_fstab #1.13
set_hostname #1.14 set_hostname #1.14
set_locale #1.15 set_locale #1.15
add_zfs_hook_to_initramfs #1.16 add_zfs_hook_to_initramfs #1.16
set_initramfs_build_list #1.17 set_initramfs_build_list #1.17
add_zfs_files_to_new_os #1.18 add_zfs_files_to_new_os #1.18
enter_chroot #2.1 enter_chroot #2.1
# We're done in chroot # We're done in chroot
finalize_os_setup #3.1 finalize_os_setup #3.1
fi fi
} }
main main "${@}"