[Cc Ben as he gave feedback to the last iteration, Luca as he wanted
something actionable]

Hi folks

As I abondened the last try and also learned some new things in the
meantime, I'd like to discuss another try at re-organizing how Debian
does boot loaders and initramfs. This mail mostly tries to get the goals
and requirements straight.

Please provide feedback. Also for missing stuff.

Regards,
Bastian

## Goals

- Setup complete boot entries from packaged and generated files
- Support dumb file systems for /boot by default, so boot loaders can
drop complex file system support.
- Re-create stuff in /boot from scratch
- Remove symlink handling from kernel package
- Single entry point for packages and admins, aka no tool specific
"update-initramfs" anymore

## Requirements

- Read package files in /usr
- Uses observed state, not changes provided by maintainer scripts
- Dumb writes to /boot. No rename, no sym, hard links. Can use ref
links if possible.
- Generate initramfs if needed, creates new entry if re-generated
- Keep older stuff (like previous kernel, initramfs) for a short while
- Possible to support multiple targets, like grub, zipl, flash-kernel
- Multiple inputs, plain kernel, UKI, also in one version
- Combinations of inputs, Xen+Linux
- Completely outside of kernel package
- Backward compatible support for stuff packaged in /boot
- Use config for kernel command line

## Open questions

- How to select default entry if supported, just sort by version and use
newest? This also works somewhat in BLS.
- How to interface with boot loaders, just absorb all knowledge about
config and only run install tools if necessary (like with zipl as
block map based loader)? grub might get support for BLS, at least
there exists a patch somewhere, that will make it easier, and we can
just iterate over config files defining one entry each.

## Prior works

- Current Debian: change based, overwrites by dpkg in /boot, versioned
by ABI
- systemd install-kernel: only BLS as target, which nothing used by
default in Debian can read
More?

## File system layout

Some initial ideas about how stuff could look.

### Boot file system (/boot)

This file system might be shared, so everything is somewhat referenced
to the machine id. This should be somewhat compatible with BLS (type
1).

* /boot/$machineid/
* ./grub/: config snippets, so we can do "no overwrite"

### Distribution file system (/usr)

* /usr/lib/boot/$package(_$modifier)/
* ./data: raw data for item
* ./metadata: info about item in undetermined format

--
I'm a soldier, not a diplomat. I can only tell the truth.
-- Kirk, "Errand of Mercy", stardate 3198.9

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Tue, 2022-11-01 at 21:29 +0100, Bastian Blank wrote:

[Cc Ben as he gave feedback to the last iteration, Luca as he wanted something actionable]

Hi folks

As I abondened the last try and also learned some new things in the
meantime, I'd like to discuss another try at re-organizing how Debian
does boot loaders and initramfs.  This mail mostly tries to get the
goals
and requirements straight.

Please provide feedback.  Also for missing stuff.

Regards,
Bastian

Thank you, this is great.

## Goals

- Setup complete boot entries from packaged and generated files
- Support dumb file systems for /boot by default, so boot loaders can
  drop complex file system support.
- Re-create stuff in /boot from scratch
- Remove symlink handling from kernel package
- Single entry point for packages and admins, aka no tool specific
  "update-initramfs" anymore

Could you clarify what you mean by "single entry point" here? It's the
only point I can't quite decode. A trigger?

I would like to suggestion this as an additional, explicit goal, rather
than implicit:

End result should be fully compatible with the BLS (for the readers: https://uapi-group.org/specifications/specs/boot_loader_specification/
)

Given we are doing something new, it's very well worth to be fully
compatible with cross-distro standards so that we can leverage existing toolchains.

## Requirements

- Read package files in /usr
- Uses observed state, not changes provided by maintainer scripts
- Dumb writes to /boot. No rename, no sym, hard links. Can use ref
  links if possible.
- Generate initramfs if needed, creates new entry if re-generated
- Keep older stuff (like previous kernel, initramfs) for a short
while
- Possible to support multiple targets, like grub, zipl, flash-kernel
- Multiple inputs, plain kernel, UKI, also in one version
- Combinations of inputs, Xen+Linux
- Completely outside of kernel package
- Backward compatible support for stuff packaged in /boot
- Use config for kernel command line

## Open questions

- How to select default entry if supported, just sort by version and
use
  newest?  This also works somewhat in BLS.

Yes, we should follow BLS on this, so that end result is predictable, well-defined and doesn't vary wildly from other distros.

- How to interface with boot loaders, just absorb all knowledge about
  config and only run install tools if necessary (like with zipl as
  block map based loader)?  grub might get support for BLS, at least
  there exists a patch somewhere, that will make it easier, and we
can
  just iterate over config files defining one entry each.

Self-described and auto-discoverable images should be the primary mean.
For reference, the patch that adds support for the BLS to Grub is here:

https://github.com/osteffenrh/grub2/commit/d0c402c96159423242cf7b612773126ccc11a83b

UKIs will be proposed for Fedora, this should land as part of that work
and do a lot of the heavy lifting for us:

https://fedoraproject.org/wiki/Changes/Unified_Kernel_Support_Phase_1

In fact, if Grub can do UKIs, do we even need Type 1 entries (separate
textual config files) for anything at that point?

## Prior works

- Current Debian: change based, overwrites by dpkg in /boot,
versioned
  by ABI
- systemd install-kernel: only BLS as target, which nothing used by
  default in Debian can read
More?

For reference, Debian images can be built using dracut + sd-boot + kernel-install, images built with mkosi work like that:

https://github.com/systemd/mkosi

## File system layout

Some initial ideas about how stuff could look.

### Boot file system (/boot)

This file system might be shared, so everything is somewhat
referenced
to the machine id.  This should be somewhat compatible with BLS (type
1).

This just dropped talking about lots of these concepts:

https://0pointer.net/blog/linux-boot-partitions.html

* /boot/$machineid/
  * ./grub/: config snippets, so we can do "no overwrite"

### Distribution file system (/usr)

* /usr/lib/boot/$package(_$modifier)/
  * ./data: raw data for item
  * ./metadata: info about item in undetermined format

What would 'metadata' be in this context?

--
Kind regards,
Luca Boccassi

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEErCSqx93EIPGOymuRKGv37813JB4FAmNnwV4ACgkQKGv37813 JB7Bww//SAfqfm4NuvZY3hQjo8t13bAyPGz0hLmKdD5JIzFOO3Xi2WNFcvh7+vCF IV5lctT7NMkCFW6p7aFAH86sMnhGlqEQvCIbZ/2tPCeKhELw7eSzW3O9/WFIqSdD y5EyccnSgA8EpAIRO34sH7gfY+0FUd0zkAHbW0jtkNGwp0xXXwW1BacmuoKGXyIJ sj/R4wBHH1JluzijWHcWoRtPdAgC3S4DKD3fAnXNGyr0DGgdvaUn8+1tOsZz8Vpj vA04XvUwaH04e2W/VXZ9X9jONvBBQvs8eRsRGotjM+X1Hz6QGXy/Zpg9xDMk7TAO DGGW7pEk1Jz/D4lcTuLn3g4x5lRfLfRUxQ5jsjn/8l7euhtX0o7PCxmOm1UStJDM +lW+w0RkMcasWt6XqQAIFRdkdl1hC+fveBeW3USKn4zLu+EVuzqiZvpSXWyhQZ9S vV9mwHoqncqeqTxw1clzoDoSLKIURERmiuqbaM3DhkKa8C1ZBGHYTMe6GGoA8kFF hdcSW2NnLeDD+UUtXh0R6dqmxhDL1SqOn/SVsXM+HbPDeTKoaCA88K+ZFX4g2LOh FQUou121MSuSfIfCqKnB7x/MtdUouZ54XwxdobbH/NOfozGZ52dR0YcWrlB5EhCm PJ+8/77R6m1yTyhfiPYOMniVPmGcUkSls6uSO1H9kUVA5tAb8YI=
=1/Lt
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Having written a similar inquiry not too long ago, I welcome this discussion, thanks! :)

On Tue, Nov 01, 2022 at 09:29:07PM +0100, Bastian Blank wrote:

## Prior works

[..]
- systemd install-kernel: only BLS as target, which nothing used by
default in Debian can read
[..]

To maybe clarify this a bit: While kernel-install does target BLS primarily, it has support for differing layouts via the KERNEL_INSTALL_LAYOUT environment variable (set via layout= in {/usr/lib,/etc}/kernel/install.conf).

kernel-install scripts are also just that, executables with two CLI entry points
(add and remove) and a set of fixed environment variables that they
receive. Currently all of this is written in POSIX shell (compatible with
dash) - they can be made to do everything.

The BLS type 1 loader entries (an example can be found in the BLS [1]) can in the worst case be read with a line of AWK or shell, but Unified Kernel Images (UKIs / BLS type 2) are fully self-contained and can do without any sidecar config files.

## File system layout

Some initial ideas about how stuff could look.

### Boot file system (/boot)

This file system might be shared, so everything is somewhat referenced
to the machine id. This should be somewhat compatible with BLS (type
1).

* /boot/$machineid/
* ./grub/: config snippets, so we can do "no overwrite"

kernel-install and BLS support a bit more than the machine ID here, The relevant
keyword to look for in the kernel-install manpage and BLS is "entry-token".

This was added for golden master setups, because the machine ID will probably only generated on first boot, but the initrd would be generated beforehand.

best regards,
Jörg

[1] https://uapi-group.org/specifications/specs/boot_loader_specification/

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Jörg

On Mon, Nov 07, 2022 at 11:40:46AM +0100, Jörg Behrmann wrote:

On Tue, Nov 01, 2022 at 09:29:07PM +0100, Bastian Blank wrote:

## Prior works

[..]
- systemd install-kernel: only BLS as target, which nothing used by
default in Debian can read
[..]

To maybe clarify this a bit: While kernel-install does target BLS primarily, it
has support for differing layouts via the KERNEL_INSTALL_LAYOUT environment variable (set via layout= in {/usr/lib,/etc}/kernel/install.conf).

Okay. However no multi-selection.

kernel-install scripts are also just that, executables with two CLI entry points
(add and remove) and a set of fixed environment variables that they
receive. Currently all of this is written in POSIX shell (compatible with dash) - they can be made to do everything.

Add and remove are not enough for what I would like to have. And I'm
really not in the mood to try that in POSIX shell. Is it even able to
make sure stuff is actually written on the disk?

kernel-install and BLS support a bit more than the machine ID here, The relevant
keyword to look for in the kernel-install manpage and BLS is "entry-token". This was added for golden master setups, because the machine ID will probably only generated on first boot, but the initrd would be generated beforehand.

And I assume it got no way to migrate from an pre-defined entry-token to
the machine ID later on, which would be kind of useful to convert a
golden image into a real system where you can add unrelated systems
later.

Bastian

--
Those who hate and fight must stop themselves -- otherwise it is not stopped.
-- Spock, "Day of the Dove", stardate unknown

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Luca

On Sun, Nov 06, 2022 at 02:14:54PM +0000, Luca Boccassi wrote:

On Tue, 2022-11-01 at 21:29 +0100, Bastian Blank wrote:

## Goals

- Setup complete boot entries from packaged and generated files
- Support dumb file systems for /boot by default, so boot loaders can
  drop complex file system support.
- Re-create stuff in /boot from scratch
- Remove symlink handling from kernel package
- Single entry point for packages and admins, aka no tool specific
  "update-initramfs" anymore

Could you clarify what you mean by "single entry point" here? It's the
only point I can't quite decode. A trigger?

You have one tool to call, not right now multiple (the kernel maintainer scripts do some, "update-initramfs" scribles to it as well).

I would like to suggestion this as an additional, explicit goal, rather
than implicit:
End result should be fully compatible with the BLS (for the readers: https://uapi-group.org/specifications/specs/boot_loader_specification/
)

Yes. We will need extensions, some may be incompatible as well.

## Open questions

- How to select default entry if supported, just sort by version and
use
  newest?  This also works somewhat in BLS.

Yes, we should follow BLS on this, so that end result is predictable, well-defined and doesn't vary wildly from other distros.

Okay.

In fact, if Grub can do UKIs, do we even need Type 1 entries (separate textual config files) for anything at that point?

grub needs to be able to read and disect UKI for non-EFI without loader
code (aka for ppc, sparc, bios and xen). And we need a way to create
them without the really heavy binutils.

And for all the old bootloaders, we have no way to use UKI at all. So
if we want to ship UKI in packages, we also need to be able to disect
them in userspace.

Can you currently create UKI with multiple multi-boot capable initrd attachments? For Xen.

### Distribution file system (/usr)

* /usr/lib/boot/$package(_$modifier)/
  * ./data: raw data for item
  * ./metadata: info about item in undetermined format

What would 'metadata' be in this context?

A description what it is. We might have raw kernel blobs for some architectures. We might have Xen kernels, which require a nested Linux.

And all the information that need to go into the type 1 file and comes
from the package, like version, name, architecture.

Regards,
Bastian

--
She won' go Warp 7, Cap'n! The batteries are dead!

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Bastian!

On Wed, Nov 23, 2022 at 08:53:10AM +0100, Bastian Blank wrote:

On Mon, Nov 07, 2022 at 11:40:46AM +0100, Jörg Behrmann wrote:

On Tue, Nov 01, 2022 at 09:29:07PM +0100, Bastian Blank wrote:

## Prior works

[..]
- systemd install-kernel: only BLS as target, which nothing used by
default in Debian can read
[..]

To maybe clarify this a bit: While kernel-install does target BLS primarily, it
has support for differing layouts via the KERNEL_INSTALL_LAYOUT environment variable (set via layout= in {/usr/lib,/etc}/kernel/install.conf).

Okay. However no multi-selection.

I'm not sure I fully understand what you mean, but I think what you mean is, that you would like to be be able to generate stuff in /boot for multiple different values of KERNEL_INSTALL_LAYOUT, e.g. boot loader spec and something else.

While the variable itself is single-valued, kernel-install can of course be run multiple times for different values of KERNEL_INSTALL_LAYOUT or with different configs wholesale (kernel-install reads environment variables from /etc/kernel/install.conf, but this can be configured via the environment variable KERNEL_INSTALL_CONF_ROOT).

I'd be grateful if you could flesh out the scenario for multi-selection a bit, because the systems I think of will usually have a single way they are booted and while being able to switch between different setups is important for development work, I'm not sure whether this case common in production deployments.

kernel-install scripts are also just that, executables with two CLI entry points
(add and remove) and a set of fixed environment variables that they receive. Currently all of this is written in POSIX shell (compatible with dash) - they can be made to do everything.

Add and remove are not enough for what I would like to have.

Fair enough. I'd be grateful, though, for a bit more information on what your plan is, because, looking at the goals section of your initial mail, I think that these could be achieved within this framework.

And I'm
really not in the mood to try that in POSIX shell.

Fair enough, I'm not a lover of POSIX shell myself. :) While kernel-install itself is written in POSIX shell (for compatibility with dash, I think, because it initially was bash and used bashisms and was ported later), the install scripts can be whatever you want - even compiled binaries.

The API is described in the kernel-install manpage, although maybe a bit haphazardly, and consists of arguments the scripts/binaries in {/usr/lib,/etc}/install.d will be called with, environment variables they can expect and exit codes that re understood.

Is it even able to
make sure stuff is actually written on the disk?

This question has multiple answers. I'm not sure whether anybody does this yet, but you can add a script to run sync at the end and report an error if that was unsuccessful. Also, kernel-install does provide a temporary staging directory, so you can output everything there and have it copied to /boot at the end and keep the interactions with dumb filesystems to a minimum.

kernel-install and BLS support a bit more than the machine ID here, The relevant
keyword to look for in the kernel-install manpage and BLS is "entry-token". This was added for golden master setups, because the machine ID will probably
only generated on first boot, but the initrd would be generated beforehand.

And I assume it got no way to migrate from an pre-defined entry-token to
the machine ID later on, which would be kind of useful to convert a
golden image into a real system where you can add unrelated systems
later.

The entry-token is just the content of the /etc/kernel/entry-token file, so it can be reset later and if no entry token exists it falls back to machine id, IMAGE_ID and ID in the os-release file in that order.

best regards,
Jörg

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)