quico-os-setup/arch-zbm
1
0
Fork 0
Code Issues 3 Pull Requests Packages Projects Releases Wiki Activity

Merge pull request '5-auto-regen-zbm' (#13) from 5-auto-regen-zbm into main

Browse Source 
Reviewed-on: #13
This commit is contained in:
hygienic-books 2023-10-27 01:09:08 +00:00
commit bb8c052f8b
3 changed files with 64 additions and 225 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

68
README.md
View File

@ -12,7 +12,7 @@ On a UEFI system ensure these conditions are met. See [How to prep](#how-to-prep
- One GPT-partitioned disk
- Arch Linux live CD ISO image sees exactly one partition with partition type code `BF00` ("Solaris root")
- Arch Linux live CD ISO image sees exactly one partition with partition type code `EF00` ("EFI system partition")
- Arch Linux live CD ISO image sees exactly one partition with partition type code `EF00` ("EFI System Partition")
- The `EF00` EFI partition is mountable, in practical terms this usually only means it has a file system.
- No ZFS zpool exists
@ -21,16 +21,22 @@ On a UEFI system ensure these conditions are met. See [How to prep](#how-to-prep
If you are instead running a legacy BIOS machine ensure these conditions are met. See [How to prep](#how-to-prep) for details on how to meet these conditions.
- One MBR-partitioned disk
- Arch Linux live CD ISO image sees exactly one partition with partition type code `BF` ("Solaris root")
- Arch Linux live CD ISO image sees exactly one partition with partition type code `bf` ("Solaris root")
- Arch Linux live CD ISO image sees exactly one partition with partition type code `83` ("Linux")
- The `83` Linux partition is mountable, in practical terms this usually only means it has a file system.
- No ZFS zpool exists
Neither with a UEFI nor legacy BIOS system are any of these conditions a requirement from ZFSBootMenu. We're just setting requirements to easily identify if you intend to do a UEFI or a legacy BIOS install. Subsequently the script has no logic to detect UEFI or legacy BIOS mode, that's legwork left to the reader :) The Internet seems to agree that a good quick check is to see if your Arch Linux live CD ISO image has directory `/sys/firmware/efi`.
```
[ -d /sys/firmware/efi ] && echo 'Likely a UEFI system' || echo 'Probably a legacy BIOS system'
```
If you're unsure nothing's stopping you from just giving it a go with a best guess and if that fails you know you guessed wrong.
## How to prep
### UEFI
On a blank example disk `/dev/sda` you can fulfill the UEFI requirements (One `EF00` partition with a file system plus one `BF00` partition) for example like so:
On a blank example disk `/dev/sda` you can fulfill the UEFI requirements (one `EF00` partition with a file system plus one `BF00` partition) for example like so:
```
sgdisk --new '1::+512M' --new '2' --typecode '1:EF00' --typecode '2:BF00' /dev/sda
mkfs.vfat /dev/sda1
@ -39,29 +45,27 @@ mkfs.vfat /dev/sda1
>
> `--new '2'`: Create partition number `2`. Both field number 2, the start sector, and field number 3, the end sector, are unspecified, there's no field separator `:`. Field number 2 will be the first free sector - in this case right after partition 1 - and field number 3 will be end of disk. Thus partition `2` will fill the remaining free disk space.
>
> `--typecode '1:EF00'`: Partition 1 gets partition type code `EF00`, an EFI system partition.
> `--typecode '1:EF00'`: Partition 1 gets partition type code `EF00`, an EFI System Partition.
>
> `--typecode '2:BF00'`: Partition 2 gets partition type code `BF00`, a Solaris root partition.
The result will be something like this at which point you can start the `setup.sh` script, see [How to run this?](#how-to-run-this) below for more details.
```
# lsblk --paths
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
/dev/loop0 7:0 0 685.5M 1 loop /run/archiso/airootfs
/dev/sr0 11:0 1 808.3M 0 rom /run/archiso/bootmnt
/dev/sda 202:0 0 10G 0 disk
├─/dev/sda1 202:1 0 512M 0 part
└─/dev/sda2 202:2 0 9.5G 0 part
# lsblk --paths --output 'NAME,SIZE,FSTYPE,PARTTYPE,PARTTYPENAME,PTTYPE' /dev/sda
NAME SIZE FSTYPE PARTTYPE PARTTYPENAME PTTYPE
/dev/sda 10G gpt
├─/dev/sda1 512M vfat c12a7328-f81f-11d2-ba4b-00a0c93ec93b EFI System gpt
└─/dev/sda2 9.5G 6a85cf4d-1dd2-11b2-99a6-080020736631 Solaris root gpt
```
### Legacy BIOS
For a legacy BIOS machine you'll be using a master boot record (MBR) on your disk.
For a legacy BIOS machine you'll be using a Master Boot Record (MBR) on your disk.
```
printf -- '%s\n' 'label: dos' 'start=1MiB, size=512MiB, type=83, bootable' 'start=513MiB, size=+, type=bf' | sfdisk /dev/sda
mkfs.vfat /dev/sda1
```
> `label: dos`: Create the following partition layout in a master boot record.
> `label: dos`: Create the following partition layout in a Master Boot Record.
>
> `start=1MiB, size=512MiB, type=83, bootable`: Partition 1 begins 1 Mebibyte after disk start and is 512 Mebibyte in size. We're setting its bootable flag and setting partition type code `83` ("Linux").
>
@ -76,13 +80,15 @@ NAME SIZE FSTYPE PARTTYPE PARTTYPENAME PTTYPE
└─/dev/sda2 9.5G 0xbf Solaris dos
```
## ZFS dataset layout
# Partition naming
The script will create a single ZFS zpool `zpool` on the `BF00` partition with dataset child `zpool/root` which itself has one child `zpool/root/archlinux`, that's where Arch Linux gets installed. Parallel to `zpool/root` it'll create `zpool/data` with a `zpool/data/home` child dataset that gets mounted at `/home`.
Since this script works with UEFI and legacy BIOS mode we'll be addressing both disk layout schemes with umbrella terms for better readability: "The zpool partition" will be GPT `BF00` partition and MBR `bf` partition. You'll parse the text accordingly. "The boot partition" will be GPT `EF00` partition as well as the MBR `83` partition.
The script will use the `EF00` partition to install a ZFSBootMenu EFI executable if `efibootmgr` says that no such `ZFSBootMenu` entry exists. If ZFSBootMenu gets added to the EFI partition it'll become primary boot option.
# ZFS dataset layout
## How to run this?
The script will create a single ZFS zpool `zpool` on the zpool partition with dataset child `zpool/root` which itself has one child `zpool/root/archlinux`, that's where Arch Linux gets installed. Parallel to `zpool/root` it'll create `zpool/data` with a `zpool/data/home` child dataset that gets mounted at `/home`.
# How to run this?
- Boot an Arch Linux live CD ISO image
- Run:
@ -91,9 +97,9 @@ The script will use the `EF00` partition to install a ZFSBootMenu EFI executable
```
During execution the script will call itself when it changes into its `chroot`, that's why we `export SCRIPT_URL`. Feel free to update `"${SCRIPT_URL}"` with whatever branch or revision you want to use from [quico.space/quico-os-setup/arch-zbm](https://quico.space/quico-os-setup/arch-zbm). Typically `.../branch/main/setup.sh` as shown above is what you want.
### Options
## Options
#### Compression
### Compression
By default we create a zpool with ZFS property `compression=on`. If the `lz4_compress` pool feature is active this will by default enable `compression=lz4`. See `man 7 zfsprops` for example in ZFS 2.1.9 for details. See `zpool get feature@lz4_compress <pool>` to check this feature's status on your `<pool>`.
@ -102,7 +108,7 @@ To get a zpool with uncompressed datasets export the shell variable `ARCHZBM_ZFS
export ARCHZBM_ZFSPROPS_NO_COMPRESSION=yesplease
```
#### Encryption
### Encryption
By default we encrypt the zpool with ZFS property `encryption=on`. In ZFS 2.1.9 this defaults to `encryption=aes-256-gcm`.
@ -111,19 +117,23 @@ To get a zpool with unencrypted datasets export the shell variable `ARCHZBM_ZFSP
export ARCHZBM_ZFSPROPS_NO_ENCRYPTION=yup
```
## Steps
# Steps
The script takes the following installation steps.
1. Install ZFS tools and kernel module with [github.com/eoli3n/archiso-zfs](https://github.com/eoli3n/archiso-zfs)
1. Create one ZFS zpool on top of `BF00` partition, encrypted and compressed datasets, password `password`
1. Create one ZFS zpool on top of zpool partition, encrypted and compressed datasets, password `password`
1. _See paragraphs [Compression](#compression)/[Encryption](#encryption) to optionally disable properties_
1. Create dataset for Arch Linux and `/home`
1. Install Arch Linux into pool
1. Add ZFSBootMenu to `EF00` partition if it doesn't exist already
1. Add ZFSBootMenu to boot partition
1. Configure boot method
- Either an EFI image with EFI boot order entries on a UEFI machine
- Or Syslinux with `extlinux` for a legacy BIOS computer
1. Add `pacman` hooks to keep ZFSBootMenu images (and `extlinux`) updated
1. Exit into Arch Linux live CD ISO image shell for you to `reboot` and frolick
## Flavor choices
# Flavor choices
We make the following opinionated flavor choices. Feel free to change them to your liking.
@ -135,7 +145,7 @@ We make the following opinionated flavor choices. Feel free to change them to yo
- Timezone is `Etc/UTC`
- Check `timedatectl set-timezone <tzdata-zone>`
## Post-run manual steps
# Post-run manual steps
After installation you're going to want to at least touch these points in your new Arch Linux install:
@ -452,7 +462,13 @@ zpool import zpool -d /dev/disk/by-partuuid -R /mnt -f -N
zfs load-key -L prompt zpool
zfs mount zpool/root/archlinux
zfs mount -a
# UEFI system ...
mount /dev/sda1 /mnt/efi
# ... or legacy BIOS system
mount /dev/sda1 /mnt/boot/syslinux
arch-chroot /mnt /bin/bash
```
@ -503,7 +519,7 @@ Explanation:
zpool/root none off no
zpool/root/archlinux /mnt noauto yes <-- Now mounted
```
- We lastly mount our EFI system partition (ESP), in this example it's living at `/dev/sda1` so adjust this path accordingly.
- We lastly mount our EFI System Partition (ESP), in this example it's living at `/dev/sda1` so adjust this path accordingly.
```
# df -hTP

44
setup.sh
View File

@ -18,7 +18,6 @@ exec 3>&1
declare this_script_url
this_script_url="${SCRIPT_URL:?}"
postconf_hook="$(dirname "${this_script_url}")"'/zbm_set_new_uefi_boot_entries.sh'
declare zpool_name zfs_arch_dataset_name
zpool_name='zpool'
@ -221,7 +220,7 @@ function select_part () {
if [[ ! "${parts}" ]]; then
case "${part_type}" in
efi)
part_type_human_readable='EFI system partition (ESP) with partition type code EF00'
part_type_human_readable='EFI System Partition (ESP) with partition type code EF00'
;;
zfs)
if [[ "${part_schema}" = 'mbr' ]]; then
@ -658,25 +657,22 @@ function configure_zfsbootmenu () {
paru_install 'zfsbootmenu'
if [[ "${part_schema}" = 'gpt' ]]; then
mkdir -p '/etc/zfsbootmenu/posthooks.d'
cat > '/etc/zfsbootmenu/config.yaml' <<EOF
Global:
ManageImages: true
BootMountPoint: /efi
InitCPIO: true
PostHooksDir: /etc/zfsbootmenu/posthooks.d
Components:
Enabled: false
EFI:
ImageDir: /efi/EFI/ZBM
Versions: false
Enabled: true
Stub: /etc/zfsbootmenu/stub-loader.d/linuxx64.efi.stub
Stub: /usr/share/zfsbootmenu/stubs/linuxx64.efi.stub/linuxx64.efi.stub # workaround: https://github.com/zbm-dev/zfsbootmenu/discussions/501
Kernel:
CommandLine: ro loglevel=0 zbm.import_policy=hostid
Prefix: vmlinuz
EOF
get_known_good_stub_loader
else
configure_syslinux
cat > '/etc/zfsbootmenu/config.yaml' <<EOF
@ -701,12 +697,18 @@ EOF
zfs set org.zfsbootmenu:commandline='rw nowatchdog rd.vconsole.keymap=de-latin1' "${zpool_name}"'/root/'"${zfs_arch_dataset_name}"
}
function get_known_good_stub_loader () {
local known_good_stub_loader local_stub_loader_abs
known_good_stub_loader='https://github.com/zbm-dev/zfsbootmenu/raw/master/testing/stubs/linuxx64.efi.stub'
local local_stub_loader_abs='/etc/zfsbootmenu/stub-loader.d/linuxx64.efi.stub'
mkdir -p "$(dirname "${local_stub_loader_abs}")"
curl --silent --location "${known_good_stub_loader}" --output "${local_stub_loader_abs}"
function add_syslinux_pacman_hook () {
mkdir -p '/opt/git/quico.space/quico-os-setup/zbm-syslinux-pacman-hook/branches/main'
git -C '/opt/git/quico.space/quico-os-setup/zbm-syslinux-pacman-hook/branches/main' clone 'https://quico.space/quico-os-setup/zbm-syslinux-pacman-hook.git' .
chmod +x '/opt/git/quico.space/quico-os-setup/zbm-syslinux-pacman-hook/branches/main/pacman-zbm-syslinux-regen.sh'
ln -s '/opt/git/quico.space/quico-os-setup/zbm-syslinux-pacman-hook/branches/main/pacman-zbm-syslinux-regen.sh' '/usr/local/bin/pacman-zbm-syslinux-regen'
ln -s '/opt/git/quico.space/quico-os-setup/zbm-syslinux-pacman-hook/branches/main/pacman-zbm-syslinux-regen.hook' '/usr/share/libalpm/hooks/pacman-zbm-syslinux-regen.hook'
}
function add_zbm_pacman_hook () {
mkdir -p '/opt/git/quico.space/quico-os-setup/zbm-regen-pacman-hook/branches/main'
git -C '/opt/git/quico.space/quico-os-setup/zbm-regen-pacman-hook/branches/main' clone 'https://quico.space/quico-os-setup/zbm-regen-pacman-hook.git' .
ln -s '/opt/git/quico.space/quico-os-setup/zbm-regen-pacman-hook/branches/main/pacman-zbm-image-regen.hook' '/usr/share/libalpm/hooks/pacman-zbm-image-regen.hook'
}
function get_disks_with_one_efipart () {
@ -752,6 +754,12 @@ function install_os_in_chroot () {
# Yes, we do this twice so we immediately get a functional backup file
generate-zbm
if [[ "${part_schema}" = 'mbr' ]]; then
paru_install 'rsync'
add_syslinux_pacman_hook
fi
add_zbm_pacman_hook
}
function set_root_pw () {
@ -849,15 +857,8 @@ function set_new_uefi_boot_entries () {
fi
}
function insert_zbm_postconf_hook () {
#3.9
declare postconf_target_abs='/mnt/etc/zfsbootmenu/posthooks.d/'"$(basename "${postconf_hook}")"
curl --silent --location "${postconf_hook}" --output "${postconf_target_abs}"
chmod +x "${postconf_target_abs}"
}
function umount_all () {
#3.10
#3.9
if [[ "${part_schema}" = 'mbr' ]]; then
umount '/mnt/boot/syslinux'
else
@ -877,9 +878,8 @@ function finalize_os_setup () {
configure_zfs_mount_gen #3.7
if [[ "${part_schema}" = 'gpt' ]]; then
set_new_uefi_boot_entries #3.8
insert_zbm_postconf_hook #3.9
fi
umount_all #3.10
umount_all #3.9
}
function main () {

177
zbm_set_new_uefi_boot_entries.sh
View File

@ -1,177 +0,0 @@
#!/bin/bash
# Whatever comes in on file descriptor (FD) 3 gets redirected to where file
# descriptor 1 is pointing. File descriptor 1 points to stdout so when we
# output-redirect something into FD 3 it shows up on stdout. We can use this
# to produce arbitrary logging output inside a subshell like so:
#
# function my_func () {
# some_command "${1:?}"
# >&3 echo 'A log message'
# }
#
# var="$(my_func arg_1)"
#
# Here "${var}" will only capture the output of some_command "${1:?}". It
# will not capture 'echo' which will instead show up on our stdout/FD 1.
exec 3>&1
# https://unix.stackexchange.com/a/48550
set -E
trap '[ "$?" -ne 77 ] || exit 77' ERR
function get_partitions () {
declare partitions_json
partitions_json="$(lsblk --output PATH,PARTTYPE --json --tree)" || return 1
printf -- '%s' "${partitions_json}"
return 0
}
function get_parts () {
local zfs_install_drive
declare parttype parts
parttype="${1:?}"
zfs_install_drive="${2:-}"
case "${parttype}" in
zfs)
parts="$(get_partitions | jq --raw-output '.[][] | select(.children | length > 0) | .children[] | select(.parttype=="6a85cf4d-1dd2-11b2-99a6-080020736631") | .path')" || exit 1
;;
efi)
parts="$(get_partitions | jq --raw-output '.[][] | select(.children | length > 0) | select(.path=="'"${zfs_install_drive:?}"'") | .children[] | select(.parttype=="c12a7328-f81f-11d2-ba4b-00a0c93ec93b") | .path')" || exit 1
;;
*)
>&3 printf -- '%s\n' 'Unknown partition type '"'"'"${parttype}"'"'"', exiting ...'
exit 77
;;
esac
printf -- '%s' "${parts}"
return 0
}
function we_have_exactly_one_part () {
local parttype parts_list parts_count
parttype="${1:?}"
parts_list="${2:?}"
parts_count="$(wc -l <<<"${parts_list}")"
if [[ "$(wc -c <<<"${parts_list}")" -gt '1' ]]; then
case "${parts_count}" in
0)
>&3 printf -- '%s\n' 'No '"${parttype^^}"' partition found. Exiting ...'
exit 77
;;
1)
return 0
;;
*)
return 1
;;
esac
>&3 printf -- '%s\n' 'Partition list does not look valid. Cowardly exiting ...'
exit 77
fi
}
function select_part () {
local parts enriched_parts enriched_parts_count part_number part_type zfs_install_drive
declare part
part_type="${1:?}" # 'efi' or 'zfs'
zfs_install_drive="${2:-}"
if [[ "${zfs_install_drive}" ]]; then
# This is intended to find correct EFI partition
parts="$(get_parts "${part_type}" "${zfs_install_drive}")"
else
parts="$(get_parts "${part_type}")"
fi
if [[ ! "${parts}" ]]; then
case "${part_type}" in
efi)
part_type_human_readable='EFI system partition (ESP) with partition type code EF00'
;;
zfs)
part_type_human_readable='ZFS zpool partition with partition type code BF00'
;;
esac
>&3 printf -- '%s\n' \
'It looks as if there is no '"${part_type_human_readable}" \
'on any of the disks. Is this a chroot? Exiting ...'
exit 77
fi
if we_have_exactly_one_part "${part_type}" "${parts}"; then
part="${parts}"
else
>&3 printf -- '%s\n' 'More than one '"${part_type^^}"' partition to pick for installation. Cowardly exiting ...'
exit 77
fi
printf -- '%s' "${part}"
return 0
}
function get_part_parent () {
local child_partition parent_partition
child_partition="${1:?}"
parent_partition="$(get_partitions | jq --raw-output '.[][] | select(.children | length > 0) | select(.children[].path=="'"${child_partition:?}"'") | .path')"
printf -- '%s' "${parent_partition}"
return 0
}
function get_disks_with_one_efipart () {
local disks_with_one_efipart
# Find disks that have exactly one EFI partition and where that EFI
# partition is partition number 1. We expect exactly one disk to meet
# these criteria. Anything else and we bail.
disks_with_one_efipart="$(lsblk --output PATH,PARTTYPE --json --tree | jq --raw-output '.[][] | select(.children | length > 0) | select( any (.children[]; (.path | test("^[^[:digit:]]+1")) and (.parttype=="c12a7328-f81f-11d2-ba4b-00a0c93ec93b")) and ([select(.children[].parttype=="c12a7328-f81f-11d2-ba4b-00a0c93ec93b")] | length == 1) ) | .path')"
if [[ "$(wc -l <<<"${disks_with_one_efipart}")" -eq '1' ]] && [[ "$(wc -c <<<"${disks_with_one_efipart}")" -gt '1' ]]; then
printf -- '%s' "${disks_with_one_efipart}"
return 0
fi
return 1
}
function set_new_uefi_boot_entries () {
declare -a uefi_images
mapfile -t uefi_images < \
<(find '/efi/EFI/ZBM' -type f -iname '*.efi' -print0 | \
xargs -0 --no-run-if-empty --max-args '1' stat -c '%Y %n' | \
sort -V | \
awk '{print $2}')
zpool_drive="$(select_part 'zfs')"
zfs_parent="$(get_part_parent "${zpool_drive:?}")"
efi_drive="${zfs_parent}"
if efibootmgr | grep -Piq -- 'ZFSBootMenu'; then
local -a old_uefi_entries
mapfile -t old_uefi_entries < \
<(efibootmgr | \
grep -Pio -- '(?<=^Boot)[^\*[:space:]]+(?=\*? ZFSBootMenu)')
for old_uefi_entry in "${old_uefi_entries[@]}"; do
efibootmgr --bootnum "${old_uefi_entry}" --delete-bootnum &>/dev/null && {
>&3 printf -- '%s\n' \
'EFI boot entry '"${old_uefi_entry}"' deleted.'
}
done
fi
if ! efibootmgr | grep -Piq -- 'ZFSBootMenu'; then
local efi_disks_list
efi_disks_list="$(get_disks_with_one_efipart)"
if grep -Piq -- '^'"${efi_drive}"'$' <<<"${efi_disks_list}"; then
for uefi_image in "${uefi_images[@]}"; do
uefi_image_version="$(basename "${uefi_image%%.EFI}")"
uefi_image_inverted="${uefi_image#/efi}"
uefi_image_inverted="${uefi_image_inverted//\//\\}"
efibootmgr --disk "${efi_drive}" \
--part 1 \
--create \
--label 'ZFSBootMenu '"${uefi_image_version}" \
--loader "${uefi_image_inverted}" &>/dev/null && {
>&3 printf -- '%s\n' \
'EFI boot entry ZFSBootMenu '"${uefi_image_version}"' added.'
}
done
fi
fi
}
set_new_uefi_boot_entries