On each target machine we're storing the Git repo that has our unit
files underneath '/opt/git'. Prior to this commit we used to place
symlinks from '/etc/systemd/system' into the appropriate '/opt/git'
subdir. On most machines this worked fine, however, on some we
encountered a race condition on system start.
Sometimes '/opt/git' is not the same file system as '/'. If that's
the case chances are that systemd initializes the system and fails to
access a unit file at '/opt/git' because that file system is just
barely not yet visible early in the boot process.
For a timer unit for example this could result in enabling the unit
and upon reboot seeing that the unit no longer existed in systemd's
world view e.g. via 'systemctl list-timers' even though the symlink
at '/etc/systemd/system' still pointed to an '/opt/git' subdir when
inspected manually seconds after boot. journalctl, however, would
clearly confirm that at system initialization the symlink target was
inaccessible.
We could fiddle around with delaying boot until '/opt/git' and its
descendants are visible but the sane solution is to just not rely on
a separate file system for important stuff such as unit files. We now
copy unit files to '/etc/systemd/system' instead of symlinking them.