• Bug#1100677: Pending autoremoval of debian-reference* packages

    From Osamu Aoki@21:1/5 to textshell@uchuujin.de on Thu Apr 17 13:50:01 2025
    Hi,


    I now see this as a bug. I think this was caused by my post-bookworm change in
    debian-reference (2.109) on Mon, 18 Dec 2023.

    If I remember correctly, the intent of this change was to move all HTML/PDF/Plain_Text document to a path under /usr/share/doc/ for better policy compliance.

    This regression was partly addressed in debian-reference (2.114) by closing #1063590 on Sat, 10 Feb 2024. What is reported is the remaining issue.

    So the question is the best known method to mitigate potential upgrade problem.

    I now understand "adding Conflicts or upgrading the versioned dependency on debian-reference-common to a Pre-Depends" is one way.

    Before doing it, I would like to know the best established method.

    How about adding simpler versioned depends (no pre-depends) with pre-rm script? Isn't it simpler?

    Or is there any other simpler and cleaner established solution. (A pointer to a package using such trics will be good for me.)

    Any suggestion?

    Osamu


    On Sun, 2025-04-13 at 16:08 +0200, textshell@uchuujin.de wrote:
    I belive debian wants to provide its documentation as packages in stable releases.

    But currently debian-reference is scheduled for auto removal on 2025-04-15 (which this mail bumps a bit into the future).
    Maybe "nobody" is aware of this?

    Anyone up to taking a look at this?

    On Mon, 17 Mar 2025 08:14:02 +0100 Helmut Grohne <helmut@subdivi.de> wrote:
    Package: debian-reference-de,debian-reference-en,debian-reference-es,debian-
    reference-fr,debian-reference-id,debian-reference-it,debian-reference- ja,debian-reference-pt,debian-reference-pt-br,debian-reference-zh-cn,debian-
    reference-zh-tw
    Version: 2.125
    Severity: serious
    User: debian-qa@lists.debian.org
    Usertags: fileconflict
    Control: affects -1 + debian-reference-common

    The debian-reference packages have a tricky undeclared file conflict
    that may break bookworm to trixie upgrades. In bookworm, debian-reference-common contains a symlink /usr/share/doc/debian-reference-common/docs pointing to ../../debian-reference whereas the debian-references-* packages in
    trixie install the same location as a directory. On the face of it, this
    is an undeclared file conflict. Really though, a bad unpack order can
    cause the unpack of the trixie files to be redirected and then go
    missing as this is a symlink to directory conversion moving between packages. The debian-refefence-* packages really need to prevent
    concurrent unpack with bookworm's debian-reference-common. Breaks and Replaces is not sufficient here. I think the options basically are using Conflicts or upgrading the versioned dependency on
    debian-reference-common to a Pre-Depends (requires consultation with d-devel). The latter option is a larger hammer and prevents a weird
    corner case that is not covered by Conflicts.

    Helmut


    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Osamu Aoki@21:1/5 to Osamu Aoki on Thu Apr 17 14:10:01 2025
    Hi,

    Following up on my previous post.

    How about adding simpler versioned depends (no pre-depends) with pre-rm script?

    I am talking about tricks using the "dpkg-maintscript-helper symlink_to_dir ..."
    command. Any thought?


    Osamu


    On Thu, 2025-04-17 at 20:40 +0900, Osamu Aoki wrote:
    Hi,


    I now see this as a bug.  I think this was caused by my post-bookworm change in
    debian-reference (2.109) on Mon, 18 Dec 2023.

    If I remember correctly, the intent of this change was to move all HTML/PDF/Plain_Text document to a path under /usr/share/doc/ for better policy
    compliance.

    This regression was partly addressed in debian-reference (2.114) by closing #1063590 on Sat, 10 Feb 2024. What is reported is the remaining issue.

    So the question is the best known method to mitigate potential upgrade problem.

    I now understand "adding Conflicts or upgrading the versioned dependency on debian-reference-common to a Pre-Depends" is one way.

    Before doing it, I would like to know the best established method.

    How about adding simpler versioned depends (no pre-depends) with pre-rm script?
    Isn't it simpler?

    Or is there any other simpler and cleaner established solution. (A pointer to a
    package using such trics will be good for me.)

    Any suggestion?

    Osamu


    On Sun, 2025-04-13 at 16:08 +0200, textshell@uchuujin.de wrote:
    I belive debian wants to provide its documentation as packages in stable releases.

    But currently debian-reference is scheduled for auto removal on 2025-04-15 (which this mail bumps a bit into the future).
    Maybe "nobody" is aware of this?

    Anyone up to taking a look at this?

    On Mon, 17 Mar 2025 08:14:02 +0100 Helmut Grohne <helmut@subdivi.de> wrote:
    Package: debian-reference-de,debian-reference-en,debian-reference- es,debian- reference-fr,debian-reference-id,debian-reference-it,debian-reference- ja,debian-reference-pt,debian-reference-pt-br,debian-reference-zh- cn,debian-
    reference-zh-tw
    Version: 2.125
    Severity: serious
    User: debian-qa@lists.debian.org
    Usertags: fileconflict
    Control: affects -1 + debian-reference-common

    The debian-reference packages have a tricky undeclared file conflict
    that may break bookworm to trixie upgrades. In bookworm, debian-reference-common contains a symlink /usr/share/doc/debian-reference-common/docs pointing to ../../debian-reference whereas the debian-references-* packages in
    trixie install the same location as a directory. On the face of it, this is an undeclared file conflict. Really though, a bad unpack order can cause the unpack of the trixie files to be redirected and then go
    missing as this is a symlink to directory conversion moving between packages. The debian-refefence-* packages really need to prevent concurrent unpack with bookworm's debian-reference-common. Breaks and Replaces is not sufficient here. I think the options basically are using Conflicts or upgrading the versioned dependency on debian-reference-common to a Pre-Depends (requires consultation with d-devel). The latter option is a larger hammer and prevents a weird corner case that is not covered by Conflicts.

    Helmut



    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Osamu Aoki@21:1/5 to All on Thu Apr 17 15:50:01 2025
    Control: severity -1 important
    Control: tags -1 pending

    My latest salsa changes: https://salsa.debian.org/debian/debian-reference/-/commit/b6a1085647982f0e62e550338c3fbb6d3cc08d4c

    As I assessed, since this bug hits user using testing only, this is not serious bug per Policy. Excuse me for my pedantic reasoning.

    Rationale: There is no error seen by the official testing of Debian package transition checks.

    Thank you for reporting this bug and excuse me for my slow response.

    Best regards,

    Osamu

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Helmut Grohne@21:1/5 to Osamu Aoki on Sun Apr 27 19:40:01 2025
    Hi Osamu,

    On Sun, Apr 27, 2025 at 07:01:44PM +0900, Osamu Aoki wrote:
    Since you seem to be very knowlegeable, I have a question:

    Isn't the APT system smart enough to run all preinst scripts of all downloaded
    packages?

    apt is only half the story. In principle, you may perform upgrades with
    dselect or dpkg directly. It just so happens that everyone uses apt. Now
    apt instructs dpkg to do its thing, but subtle changes in dependencies
    may instruct apt to order its instructions differently. Ultimately, what
    needs to happen here is making valid dpkg interactions work.

    Generally, dpkg may choose to unpack debian-reference-$LANG before
    unpacking debian-reference-common even though it may only configure debian-reference-$LANG after having configured debian-reference-common.
    In particular, debian-reference-$LANG may be unpacked before running debian-reference-common.preinst. I see how that is not desirable. We
    would like to tell apt and dpkg to always unpack debian-reference-common
    before unpacking debian-reference-$LANG. The way to do that is declaring suitable Conflicts or better Pre-Depends.

    I argue that debian-reference-$LANG should really declare:

    Depends: debian-reference-common (= ${source:Version})
    Pre-Depends: debian-reference-common (>= 2.109~)

    Helmut

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Osamu Aoki@21:1/5 to All on Tue Apr 29 04:10:01 2025
    Control: severity -1 wishlist

    Hi,

    RC bug as an attention grubber is OK but let's not use it until it is proven to be so. Let's continue discussion first.

    Also, before making noise on d-devel@l.d.o to discuss this package and the use of Pre-Depends, let's distill issues. I still need more clarification on "should" and "own" mentioned below.

    Let's discuss.

    I still see APT warning in your updated test install log.

    W: Download is performed unsandboxed as root as file '//debian-reference-de_2.126_all.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)
    Selecting previously unselected package debian-reference-de.
    (Reading database ... 6748 files and directories currently installed.) Preparing to unpack debian-reference-de_2.126_all.deb ...

    Are you insisting that "Preparing to unpack" process run under warned condition is a valid test case to warrant declaring pre-depends against the policy guidance.

    "Preparing to unpack" runs preinst in advance. If this process is run by _apt,
    it may be unable to find preinst and skipped. This may be the case in your test
    case due to this permission problem.

    Unpacking debian-reference-de (2.126) ...

    It looks like it failed since preinst script was skipped when it really meant to
    be executed.

    osamu@goofy:~/salsa/debian-reference/test 00:11:28 ↵
     $ sudo dpkg --unpack --auto-deconfigure debian-reference-de_2.126_all.deb (Reading database ... 636494 files and directories currently installed.) Preparing to unpack debian-reference-de_2.126_all.deb ...
    Unpacking debian-reference-de (2.126) over (2.100) ...
    Processing triggers for doc-base (0.11.1) ...
    Processing 1 changed doc-base file... osamu@goofy:~/salsa/debian-reference/test 00:13:12 ↵

    This is closely following my reproducer. At this point, the damage
    should be there. Whilst, debian-reference-de.preinst attempts to perform
    the link to directory conversion, the link is owned by the "wrong"
    package and hence the helper does not act. At this point, dpkg --verify should tell you about missing files.


    You say "should". But really? I don't see such problem in my test case since debian-reference-de.preinst is accessible and executed. Here is the situation:

    osamu@goofy:~/salsa/debian-reference/test 10:19:23 ↵
    $ sudo dpkg --unpack --auto-deconfigure debian-reference-de_2.126_all.deb (Reading database ... 636574 files and directories currently installed.) Preparing to unpack debian-reference-de_2.126_all.deb ...
    Unpacking debian-reference-de (2.126) over (2.126) ...
    Processing triggers for doc-base (0.11.1) ...
    Processing 1 changed doc-base file... osamu@goofy:~/salsa/debian-reference/test 10:21:20 ↵
    $ ls -l /usr/share/doc/debian-reference-common/
    total 24
    -rw-r--r-- 1 root root 6616 Apr 24 11:18 changelog.gz
    -rw-r--r-- 1 root root 1430 Apr 24 11:18 copyright
    drwxr-xr-x 1 root root 548 Apr 29 10:20 docs
    -rw-r--r-- 1 root root 8893 Apr 24 11:18 README.md.gz osamu@goofy:~/salsa/debian-reference/test 10:21:28 ↵
    $ ls -l /usr/share/doc/debian-reference-common/docs
    total 4136
    -rw-r--r-- 1 root root 11784 Apr 24 11:18 apa.de.html
    -rw-r--r-- 1 root root 312453 Apr 24 11:18 ch01.de.html
    -rw-r--r-- 1 root root 338061 Apr 24 11:18 ch02.de.html
    -rw-r--r-- 1 root root 97281 Apr 24 11:18 ch03.de.html
    -rw-r--r-- 1 root root 91827 Apr 24 11:18 ch04.de.html
    -rw-r--r-- 1 root root 105029 Apr 24 11:18 ch05.de.html
    -rw-r--r-- 1 root root 153626 Apr 24 11:18 ch06.de.html
    -rw-r--r-- 1 root root 134147 Apr 24 11:18 ch07.de.html
    -rw-r--r-- 1 root root 52121 Apr 24 11:18 ch08.de.html
    -rw-r--r-- 1 root root 421040 Apr 24 11:18 ch09.de.html
    -rw-r--r-- 1 root root 226593 Apr 24 11:18 ch10.de.html
    -rw-r--r-- 1 root root 208745 Apr 24 11:18 ch11.de.html
    -rw-r--r-- 1 root root 189855 Apr 24 11:18 ch12.de.html
    -rw-r--r-- 1 root root 3396 Apr 24 11:18 debian-reference.css
    -rw-r--r-- 1 root root 1406403 Apr 24 11:18 debian-reference.de.pdf -rw-r--r-- 1 root root 271388 Apr 24 11:18 debian-reference.de.txt.gz drwxr-xr-x 1 root root 160 Apr 29 00:13 images
    -rw-r--r-- 1 root root 142757 Apr 24 11:18 index.de.html
    -rw-r--r-- 1 root root 986 Apr 29 10:20 index.html
    -rw-r--r-- 1 root root 36145 Apr 24 11:18 pr01.de.html

    Please note this package uses "dpkg-maintscript-helper symlink_to_dir ..." instead of "dpkg-maintscript-helper dir_to_symlink ...".

    I see "own" reference only in "Switching a directory to symlink" for "dpkg- maintscript-helper dir_to_symlink ...". Your argument doesn't make sense.

     $ sudo apt-get -y -f install
    ...
    Unpacking debian-reference-common (2.126) over (2.100) ...
    Setting up debian-reference-common (2.126) ...

    This is when the link really got converted into a directory.

    No. It was done by debian-reference-de.preinst as shown in the above and debian-reference-common.preinst here becomes NOP since there is no symlink.

    There is very tall policy requirement for Pre-depends:


    Pre-Depends should be used sparingly, preferably only by packages whose premature upgrade or installation would hamper the ability of the system to continue with any upgrade that might be in progress.

    I need to see it but so far I haven't.


    I am aware of this policy aspect. I argue that this is one of those exceptional cases where Pre-Depends really is the cure and the risk of downsides is manageable, because the packages that shall issue
    Pre-Depends do not have any reverse dependencies.

    "It works" is not good enough excuse for using Pre-depends.

    There is another relatively simple argument that might help with understanding the matter. Take a moment to think about file ownership in
    a packaging sense. Initially,
    /usr/share/doc/debian-reference-common/docs is a symbolic link owned by debian-reference-common. Now you unpack debian-reference-de, which
    installs the same location as a directory. Which package is now the
    proper owner of that file? After unpacking debian-reference-de, there is
    no obligation to configure it. 

    I agree it doesn't need to be "configured".

    I still think that it needs to be "unpacked" properly after running its preinst scripts properly.

    Are you sure your test case with warning qualify as a proper unpack example?

    And yeah, by all means, discuss the use of Pre-Depends with
    d-devel@l.d.o.

    Let me think about it after getting your updated thoughts on your "should" and "own" comments.

    Regards,

    Osamu

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)