• Bug#1057199: debian-policy: express more clearly that Conflicts to not

    From Helmut Grohne@1:229/2 to All on Fri Dec 1 14:20:02 2023
    XPost: linux.debian.bugs.dist, linux.debian.policy
    From: helmut@subdivi.de

    Package: debian-policy
    Version: 4.6.2.0
    X-Debbugs-Cc: debian-dpkg@lists.debian.org, deity@lists.debian.org

    Hi,

    first of all huge thanks to David, Guillem and Julian for all of their explanations. In large parts, this bug report is yours and I'm just the
    one writing it down.

    7.4 currently starts with:

    When one binary package declares a conflict with another using a
    Conflicts field, dpkg will refuse to allow them to be unpacked on
    the system at the same time.

    I believe this is technically wrong. There are situations where dpkg
    will allow such unpacks to temporarily co-exist. 6.6 goes into further
    detail and is accurate.

    Suppose we have two arch:all packages a version 1 and b version 1 both
    of which are installed. Now we attempt to install a version 2, which
    happens to declare "Conflicts: b (<< 2)". We may therefore mark b for
    removal

    echo "b:all deinstall" | dpkg --set-selections

    and proceed to installing a:

    dpkg --auto-deconfigure --unpack a_2.deb

    When we do this, dpkg will unpack a version 2 before removing the files
    of b version 1. I argue this is very briefly allowing these packages to
    be unpacked at the same time as the next thing dpkg does is removing b's
    files.

    This situation can be forced if we add package b version 2, which
    declares "Breaks: a (<< 2)" and attempt to install both. apt figures
    that it has to temporarily remove b and hence issues the selection
    above. Then it proceeds to unpacking both packages.

    The difference actually is rather subtle. As dpkg is tracking ownership
    of files, one should not be observing a difference. What one can see is
    that a.preinst version 2 is run at a time where b version 1 is still
    unpacked (and that's fine as the statement only talks about unpack). The effects of concurrent unpack are theoretically not observable, due to
    dpkg tracking files. However when you add aliasing to the mix, dpkg can
    now delete files that are still needed via differences in aliasing. That
    way - and I am fully aware that this violates fundamental assumptions of
    dpkg - we can make the order of unpacks visible and demonstrate that
    indeed a version 2 is unpacked before b version 1 has its files removed.
    All of this is fully in line with the long description in 6.6. What I
    take issue with is the executive summary at the start of 7.4.

    In case you like some kind of test case to tinker with, I'm attaching a
    script that demonstrates the situation.

    Helmut

    #!/bin/shset -eset -uset -xif test -z "${MMDEBSTRAP_HOOK:-}"; then exec mmdebstrap \ sid \ /dev/null \ --variant=apt \ --include=dpkg-dev \ --hook-dir=/usr/share/mmdebstrap/hooks/merged-usr \ --customize-hook="upload '$0' /self" \ --
    chrooted-customize-hook="sh /self"fi: "${TESTCASE:=dpkgremoveb}"for p in a1 a2 b1 b2; do mkdir -p "/testcase/$p/DEBIAN"donecat > /testcase/a1/DEBIAN/control <<EOFPackage: aVersion: 1Architecture: allDescription: a version 1Maintainer: none <
    none@invalid.example>EOFcat > /testcase/b1/DEBIAN/control <<EOFPackage: bVersion: 1Architecture: allDescription: b version 1Maintainer: none <none@invalid.example>EOFmkdir -p /testcase/b1/libecho b1 > /testcase/b1/lib/flagfilecat > /testcase/
    a2/DEBIAN/control <<EOFPackage: aVersion: 2Architecture: allConflicts: b (<< 2)Description: a version 2Maintainer: none <none@invalid.example>EOFcat > /testcase/a2/DEBIAN/preinst <<EOF#!/bin/shif test "\$(cat /lib/flagfile)" = b1; then echo "
    b (= 1) is unpacked despite conflicts"fiEOFchmod +x /testcase/a2/DEBIAN/preinstmkdir -p /testcase/a2/usr/libecho a2 > /testcase/a2/usr/lib/flagfilecat > /testcase/b2/DEBIAN/control <<EOFPackage: bVersion: 2Architecture: allDescription: b
    version 2Maintainer: none <none@invalid.example>EOFif test "${TESTCASE#apt}" != "$TESTCASE"; then echo "Breaks: a (<< 2)" >>/testcase/b2/DEBIAN/controlfifor p in a1 a2 b1 b2; do dpkg-deb --build "/testcase/$p" "/testcase/$p.deb"doneapt-get -y
    install /testcase/a1.deb /testcase/b1.debcase "$TESTCASE" in dpkgremoveb) echo 'b:all deinstall' | /usr/bin/dpkg --set-selections dpkg --auto-deconfigure --unpack /testcase/a2.deb ;; dpkgbadorder) # This is how apt invokes dpkg during apt
    install a2 b2 echo 'b:all deinstall' | /usr/bin/dpkg --set-selections dpkg --auto-deconfigure --unpack /testcase/a2.deb /testcase/b2.deb ;; apt) apt-get install /testcase/a2.deb /testcase/b2.deb ;; aptdebug) # Shows the commands for
    dpkgbadorder, but doesn't actually do it apt-get -o Debug::pkgDPkgPM=true install /testcase/a2.deb /testcase/b2.deb ;; dpkgrightorder) dpkg --auto-deconfigure -i /testcase/b2.deb /testcase/a2.deb ;; dpkgrefuse) if dpkg --auto-deconfigure --
    unpack /testcase/a2.deb /testcase/b2.deb; then exit 1 else exit 0 fi ;;esacflag=$(cat /usr/lib/flagfile)if test "$flag" != a2; then echo "/flagfile has unexpected content >$flag<" exit 1fi

    --- SoupGate-Win32 v1.05
    * Origin: you cannot sedate... all the things you hate (1:229/2)
  • From Sean Whitton@1:229/2 to Helmut Grohne on Fri Dec 15 17:50:01 2023
    XPost: linux.debian.bugs.dist
    From: spwhitton@spwhitton.name

    Hello,

    On Fri 01 Dec 2023 at 02:11pm +01, Helmut Grohne wrote:

    §7.4 currently starts with:

    When one binary package declares a conflict with another using a
    Conflicts field, dpkg will refuse to allow them to be unpacked on
    the system at the same time.

    I believe this is technically wrong. There are situations where dpkg
    will allow such unpacks to temporarily co-exist. §6.6 goes into further detail and is accurate.

    Thank you for the detailed report.

    Do the dpkg and apt people think that the bug here is just in Policy, or
    are there any code changes under consideration in response to this work?

    --
    Sean Whitton

    --=-=-Content-Type: application/pgp-signature; name="signature.asc"

    -----BEGIN PGP SIGNATURE-----

    iQJNBAEBCgA3FiEEm5FwB64DDjbk/CSLaVt65L8GYkAFAmV8gWkZHHNwd2hpdHRv bkBzcHdoaXR0b24ubmFtZQAKCRBpW3rkvwZiQMP3D/47nubMgk3Jk8f+tO8FzTzg sQjYXjxUI04yJQZVHo8DhNK5F+QtNo1JKgS02d+EM4mS8f4u24OlpFDqDru8INrz aHsE+2KH7Y0odzf4FxMEXUTOcUT2ZuiA1EnSzHfhS66gSyPtf9fNU48jzCykZfyK MwdBvtcy0keSLCrE6N7w9GceupcYFioanLu7PdJI9wW9G4MpQcSq2yCyxE1n2W6B 0dFXp3GDWN8fmR9B+UkzhGWlGDToIvJ7Yofugs5XarekNCzoHNLObf6kLIA3ofsr 5K1MCntRUXvyWkxW68Z0bwiZ5rRidIn6bcQ+9YfiCFvUK+VNYVrihKhG1T3jG/tc V3EqUsXt6HC/++DyW0OLhaIQEF8ouSgztYZLF8bDO7NPHVp6DCJ6bXLCPJbKn6KG Oe53YGpa3N8pjPsLEUdutayO8Qy1AE3yBluMff8pbOkOESv8eqxJzLbCiygHTjos 7i2WK5N5es114CfFO6u1U5afpLGKHgOY/vUyZp3nCw70dS7WnoPFcw+dODflI0qB sC7XbHfGPmCn5RVgaBDpZWIsQ2WukDiFLFiMNjwsNtvN2tFV07jUDyLEWgaL7ZxL MJWZ5FEVOQRDYqot30mZDKkclX58sv1IRf8GdKeylgy9lIofcy+harX1f5qhu9vR XunIu8mq+1F303VdI1vS+A==qBqO
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: you canno
  • From Guillem Jover@1:229/2 to Sean Whitton on Wed Jan 3 20:30:01 2024
    XPost: linux.debian.bugs.dist, linux.debian.policy
    From: guillem@debian.org

    Hi!

    On Fri, 2023-12-15 at 16:40:09 +0000, Sean Whitton wrote:
    On Fri 01 Dec 2023 at 02:11pm +01, Helmut Grohne wrote:
    §7.4 currently starts with:

    When one binary package declares a conflict with another using a
    Conflicts field, dpkg will refuse to allow them to be unpacked on
    the system at the same time.

    I believe this is technically wrong. There are situations where dpkg
    will allow such unpacks to temporarily co-exist. §6.6 goes into further detail and is accurate.

    Thank you for the detailed report.

    Do the dpkg and apt people think that the bug here is just in Policy, or
    are there any code changes under consideration in response to this work?

    I think it is just a documentation issue in the Debian Policy, yes. At
    least the dpkg behavior seems entirely correct to me and required for safe upgrades (and definitely not something like an accidental regression as it
    has behaved that way since pretty much the beginning of its git history).

    In addition I think the paragraph in §7.4 that states:

    ,---
    A package will not cause a conflict merely because its configuration
    files are still installed; it must be at least “Half-Installed”.
    `---

    could also be clarified that what is stated here does not apply either
    to conflicting packages that are being removed, as those will be in half-installed state. Perhaps as part of this, also make this state
    change explicit in §6.6.2.3.

    Thanks,
    Guillem

    --- SoupGate-Win32 v1.05
    * Origin: you cannot sedate... all the things you hate (1:229/2)
  • From Sam Hartman@1:229/2 to All on Wed Jan 3 23:10:01 2024
    XPost: linux.debian.bugs.dist, linux.debian.policy
    From: hartmans@debian.org

    "Guillem" == Guillem Jover <guillem@debian.org> writes:

    Guillem> At least the dpkg behavior seems entirely
    Guillem> correct to me and required for safe upgrades (

    Can you help me understand the sentence above?
    Where is the case where this behavior is needed for safe upgrades?
    (I am asking out of curiosity; I'm guessing it's some corner case with essential packages, but I would like to understand.)

    --Sam

    --- SoupGate-Win32 v1.05
    * Origin: you cannot sedate... all the things you hate (1:229/2)
  • From Guillem Jover@1:229/2 to Sam Hartman on Fri Jan 19 01:00:01 2024
    XPost: linux.debian.bugs.dist, linux.debian.policy
    From: guillem@debian.org

    Hi!

    On Wed, 2024-01-03 at 15:04:01 -0700, Sam Hartman wrote:
    "Guillem" == Guillem Jover <guillem@debian.org> writes:
    Guillem> At least the dpkg behavior seems entirely
    Guillem> correct to me and required for safe upgrades (

    Can you help me understand the sentence above?
    Where is the case where this behavior is needed for safe upgrades?
    (I am asking out of curiosity; I'm guessing it's some corner case with essential packages, but I would like to understand.)

    I should probably have qualified that statement, where safe upgrades
    is restricted to the specific scenarios at hand. In any case, I don't
    think this is specific to essential packages, this seems more general.
    But of course the effects of not supporting such safe upgrades for
    essential packages would be potentially more severe (although an
    essential package can never be a conflictor as dpkg would refuse to
    remove it).

    In essence this involves a cyclic restriction where a set of packages
    are stating they cannot be unpacked at the same time, but later
    versions might be. There are several subcases for this depending on
    the strength of the dependencies from each side, whether these are
    versioned, and also whether the package manager front-end or the user
    decides whether to fully evict a conflictor or wants to upgrade to a
    latest version that can co-exist.

    What the current behavior permits is to safely intertwine an unpack,
    a conflictors files transfer and its removal in the same step, so that
    the front-end or the user does not need to apply --force-* options to
    forcibly remove the conflictor while breaking the dependency system
    for an undetermined amount of time, to be able to proceed with such
    upgrade.

    I don't see any other way around this kind of upgrade that does not
    break the dependency system besides the current behavior (if someone
    does, I'm happy to hear it). Of course there's always the option to
    prohibit those kinds of relationships, which means the behavior never
    is brought up, and there's no need either to be concerned about whether
    it needs to be supported or not. But that seems overly restrictive,
    because once or if you need to express that kind of relationship, then
    that path would be closed.


    All this being said, when Helmut brought this up, I noticed that dpkg
    is expecting to be hand held by giving to it the proper order for these operations (as front-ends do), or it will either fail to upgrade
    depending on the scenario, or unnecessarily use the conflictor eviction behavior, which are not ideal (but do not break the dependency system).
    I've got some test cases (Conflicts vs Breaks, Conflicts vs Conflicts),
    which I should expand to add addition restrictions in the form of
    Depends and Pre-Depends, unversioned relationships and similar, after
    which I'll look into improving the way dpkg schedules its processing
    queue in these cases so that it is a bit smarter about them and does
    not fail so easily or uses this behavior unnecessarily when there's a
    path forward at hand. But this seems besides the point of the conflictor eviction behavior.

    Thanks,
    Guillem

    --- SoupGate-Win32 v1.05
    * Origin: you cannot sedate... all the things you hate (1:229/2)