• Bug#1020533: dpkg should use /var/lib/dpkg/arch to determine native arc

    From Johannes Schauer Marin Rodrigues@1:229/2 to All on Fri Jun 16 16:40:01 2023
    XPost: linux.debian.bugs.dist
    From: josch@debian.org

    Hi Guillem,

    Quoting Guillem Jover (2022-10-10 12:23:58)
    On Thu, 2022-09-22 at 22:13:34 +0200, Johannes Schauer Marin Rodrigues wrote: As mentioned on IRC, the problem here (and on #825385) is indeed that dpkg considers its own arch the native one, and when operating on a cross-arch chroot, that goes wrong, both in dependency resolution and when outputting arch-qualified package names for example.

    Fixing this properly is tricky though, because there are multiple requirements in tension here:

    * The external dpkg should be able to tell what's the arch for the
    internal one w/o having to execute anything (that it might not be
    able to run anyway).
    * Setting a file on the database means either requiring a dpkg
    maintscript (for the bootstrap phase) which we are trying to get
    rid of, or offloading this to the bootstrappers (even worse), it
    also means it could be modified w/o dpkg noticing, which can break
    dependency resolution from an existing system.
    * Shipping a file with the arch would seem like the best option (as
    that is checksummed) and not in the db, but that means then, that
    pathname under /usr needs to be selectable too at run-time, as
    that encodes configure-time vendor-specific fsys layout.
    * Setting that directory from the config file is currently
    problematic as dpkg does not have a way to specify a different
    config directory.
    * Perhaps just adding a new --foodir option to dpkg could be enough
    for now, if the inner dpkg uses a different pathname, in the
    same way if it uses a different database pathname.
    * But then only specifying the db pathname would no longer be
    enough, and dependency resolution and arch-qualifying would still
    be wrong. :/
    * But then if the file does not exist (or cannot be found in the
    «right» place) it could still fallback to the currently running
    native arch, but that looks flaky (not worse than now, though,
    but not ideal). :/

    I guess I can prototype something with the --foodir idea though, but that's still rather meh.

    once you have a prototype (even if it's not release-ready) feel free to share it, because our CI setup is able to apply patches to source packages. So if you have something that we can test, just send it over and we'll build a patched dpkg to run this with our scripts.

    Thanks!

    cheers, josch
    --==============X56093630299215075=MIME-Version: 1.0
    Content-Transfer-Encoding: 7bit
    Content-Description: signature
    Content-Type: application/pgp-signature; name="signature.asc"; charset="us-ascii"

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEElFhU6KL81LF4wVq58sulx4+9g+EFAmSMcb0ACgkQ8sulx4+9 g+Ev0g//Z6NNagV982BMb3X4NR0k1NUjAUHERUOBsBNbGKiYTwnVK/SsoqOpT3im X0is6cx47usf8HBMFl7HshhS6v7p1F9l1EJljv7FbUgisl14AwFjTUj5w0GgQ/ui OFy0vXNHdzzImgFvqr2dx6aL12MChM/waDsH9p1ywCj6X5dZ0Tpk7FeVqcucS/gU KpAFAYupw7UtV+s3czGwe05NpFzTbtkwDiNJCPNEs9S9gDa3bG6ipm7zW560IBXO LEzU89iRNNIE/SBFZbpc/IBmLWkP/mNCblENaWJbA05w8MsO+OK0Au57lx/ZrsfK AbjUD2ryKRc35IkOU0va2zrt3N8/yD9rPpgANpaUPjh0oY4QW/RjMBT/yfDyuXDB HOY16I4xQiLRXWDaoAx7+q9HH0Pyfs3Y0hWSM5xAIb6CEtIfcedB6PPrN1T6ApCJ pc6oI6fAV+NRPdYIdS4FTsH98lVI5Vhf/mlnK4wtSqWtcbPAeN3QoYOwIOk0YZoC HR4IZdK8wyz0hPvw2ZS3VNM0NFWAn6bymZDNdLMrk010cT8PnAeEjDZd0RW2qZQT oseXGYtNrBZ7T+tfwItkhzh+ypqvQsUXsWKKipA1bhAm3TMLWvjnQXpOHcGwNSqR j4I7Tx9EAiJuJ1gEtalxnqaWFFK1ONMLUyxPB7OYKNFqQor7VWg=
    =fKDg
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: you cannot sedate... all the things you hate (1:229/2)
  • From Guillem Jover@1:229/2 to Johannes Schauer Marin Rodrigues on Sat Apr 20 19:40:01 2024
    XPost: linux.debian.bugs.dist
    From: guillem@debian.org

    Hi!

    On Fri, 2023-06-16 at 16:29:17 +0200, Johannes Schauer Marin Rodrigues wrote:
    Quoting Guillem Jover (2022-10-10 12:23:58)
    On Thu, 2022-09-22 at 22:13:34 +0200, Johannes Schauer Marin Rodrigues wrote:
    As mentioned on IRC, the problem here (and on #825385) is indeed that dpkg considers its own arch the native one, and when operating on a cross-arch chroot, that goes wrong, both in dependency resolution and when outputting arch-qualified package names for example.

    Fixing this properly is tricky though, because there are multiple requirements in tension here:

    * The external dpkg should be able to tell what's the arch for the
    internal one w/o having to execute anything (that it might not be
    able to run anyway).
    * Setting a file on the database means either requiring a dpkg
    maintscript (for the bootstrap phase) which we are trying to get
    rid of, or offloading this to the bootstrappers (even worse), it
    also means it could be modified w/o dpkg noticing, which can break
    dependency resolution from an existing system.
    * Shipping a file with the arch would seem like the best option (as
    that is checksummed) and not in the db, but that means then, that
    pathname under /usr needs to be selectable too at run-time, as
    that encodes configure-time vendor-specific fsys layout.
    * Setting that directory from the config file is currently
    problematic as dpkg does not have a way to specify a different
    config directory.
    * Perhaps just adding a new --foodir option to dpkg could be enough
    for now, if the inner dpkg uses a different pathname, in the
    same way if it uses a different database pathname.
    * But then only specifying the db pathname would no longer be
    enough, and dependency resolution and arch-qualifying would still
    be wrong. :/
    * But then if the file does not exist (or cannot be found in the
    «right» place) it could still fallback to the currently running
    native arch, but that looks flaky (not worse than now, though,
    but not ideal). :/

    I guess I can prototype something with the --foodir idea though, but that's still rather meh.

    once you have a prototype (even if it's not release-ready) feel free to share it, because our CI setup is able to apply patches to source packages. So if you
    have something that we can test, just send it over and we'll build a patched dpkg to run this with our scripts.

    I provided a prototype patch some time ago which got integrated in
    Helmut's dpkg-root-demo CI system. But I've not been entirely happy
    with it though. I've been rethinking about it from a clean slate and
    I went with a file in the db instead which seems way cleaner after all.
    My current reasoning is:

    * Shipping a file under /usr/share means:
    - The chroot has no defined native arch until dpkg is unpacked, which
    again imposes an installation ordering, which we'd rather avoid I
    think.
    - It also requires to install a dpkg in the chroot! (Which is generally
    desirable but might not be in cases such as for testing purposes.)
    - It requires hardcoding a filesystem pathname (worse) in addition to
    the database pathname.
    - It creates a public interface.
    * It creates a potential out-of-sync situation between the admindir
    and the root directory, as these can be specified separately.
    * It would require adding a new --foodir option for this new
    directory.
    * If we need to pre-create the file in the chroot, I think doing
    that inside the db is always preferable than creating one in the
    filesystem part, because the latter is a filesystem policy
    governed by the chroot distributor, and is part of the packaged
    data while the location of the latter is still a policy issue,
    it is something already handled by the tooling and more
    independent than the packaged bits.
    * In the future pre-creating the file could be done by extending
    dpkg to provide a new dpkg action to initialize a chroot db for
    example, that could hide any such detail, while doing the same for
    the chroot filesystem seems just wrong.

    The annoying part is that this information is already present in the Architecture field for the dpkg package, but that is also a downstream
    policy decision and I'd consider it a regression to try to infer the
    package name dpkg itself is being packaged as inside the chroot, not
    to mention that still does not cover some of the points above anyway.

    I've queued the changes and I'm planning to push them during the
    weekend.

    Thanks,
    Guillem

    --- SoupGate-Win32 v1.05
    * Origin: you cannot sedate... all the things you hate (1:229/2)