• Re: Debian 12.5: pigz 2.6-1 fails with error message (Upstream issue 11

    From Chung Jonathan@21:1/5 to All on Sat Apr 6 09:50:18 2024
    RGVhciBGcmFuY28gTWFydGVsbGksIGRlYXIgVGhvbWFzIFNjaG1pdHQsDQoNClllcywgL2V0Yy9k ZWJpYW5fdmVyc2lvbiByZXBvcnRzIDEyLjUuIEhvd2V2ZXIsIGFzIEkgaGF2ZSB0aGUgZG9ja2Vy IHNvdXJjZXMgYXMgZXh0ZXJuYWwgcmVwb3NpdG9yeSB0aG9zZSBtaWdodCBoYXZlIGludHJvZHVj ZWQgYW4gdXBkYXRlIHRvIHpsaWIuIEkgd2lsbCB0cnkgdG8gbmFycm93IGl0IGRvd24gYW5kIHRo ZW4gb3BlbiBhIHN1YnNlcXVlbnQgYnVnIHJlcG9ydCAtIHBvc3NpYmx5IG9uIGEgY2xlYW4gVk0g dG9vLiANCg0KVGhlIGVycm9yIGZpcnN0IG9jY3VycmVkIGFzIEkgaW5zdGFsbGVkIGEgZG9ja2Vy IGltYWdlLiBIb3dldmVyLCBwaWd6IC0tdmVyc2lvbiBhbHNvIGdhdmUgdGhpcyBlcnJvci4gDQoN ClRoYW5rIHlvdSBmb3IgcG9pbnRpbmcgbWUgdG8gdGhlIGNvcnJlY3QgZGlyZWN0aW9ucywNCg0K Sm9uYXRoYW4gQ2h1bmcNCg0KPiBBbSAwMi4wNC4yMDI0IHVtIDIxOjA3IHNjaHJpZWIgRnJhbmNv IE1hcnRlbGxpIDxtYXJ0ZWxsaWY2N0BnbWFpbC5jb20+Og0KPiDvu79PbiAwMi8wNC8yNCBhdCAx MDoyNywgSm9uYXRoYW4gQ2h1bmcgd3JvdGU6DQo+PiBEZWFyIHNpciBvciBtYWRhbSwNCj4+IEkn bSBuZXcgd2l0aCBEZWJpYW4gYnVnIHJlcG9ydGluZyBhbmQgdGh1cyBuZWVkIHNvbWUgaGVscCB3 aXRoIHRoYXQuDQo+PiBwaWd6IDIuNi0xIG9uIERlYmlhbiAxMi41IGZhaWxzIHRvIGV4ZWN1dGUg ZHVlIHRvIGEgZml4ZWQgYnVnIG9uDQo+PiB1cHN0cmVhbSBodHRwczovL2dpdGh1Yi5jb20vbWFk bGVyL3BpZ3ovaXNzdWVzLzExMQ0KPj4gSW5zdGFsbGluZyB0aGUgdmVyc2lvbiBmcm9tIHNpZCBy ZXNvbHZlcyB0aGUgaXNzdWUgd2hpY2ggaXMgY2xlYXJseSBub3QNCj4+IG9wdGltYWwuIEkgdGhp bmsgdGhlIGZpeCBzaG91bGQgYmUgYmFja3BvcnRlZC4NCj4+IENhbiBzb21lb25lIGhlbHAgbWUg dG8gZmlsZSBhIGJ1ZyByZXBvcnQ/DQo+IA0KPiBUbyBmaWxlIGEgYnVnIHJlcG9ydCB1c2UgInJl cG9ydGJ1ZyBwaWd6IiBhbmQgYW5zd2VyIHRoZSBxdWVzdGlvbnPigKYgYnV0IEkgdXNlIHJlZ3Vs YXJseSAicGlneiIgaW4gbXkgYmFja3VwIHNjcmlwdHMgYW5kIEkgZGlkbid0IG5vdGljZSBub3Ro aW5nIHdyb25nID9fPw0KPiBNeSBpbnN0YWxsZWQgdmVyc2lvbiBpczoNCj4gDQo+IH4kIGRwa2cg LWwgfCBncmVwIHBpZ3oNCj4gaWkgIHBpZ3ogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgIDIuNi0xICAgICAgICAgICAgICAgICAgICAgICAgICAgICBh bWQ2NCAgICAgICAgUGFyYWxsZWwgSW1wbGVtZW50YXRpb24gb2YgR1ppcA0KPiANCj4gSSBqdXN0 IHRlc3RlZCBpdCBhbmQgSSd2ZSBub3RoaW5nIHRvIHJlcG9ydDoNCj4gDQo+IH4kIHRhciB0ZiBm cmFua2JjazIudGd6IC0tdXNlLWNvbXByZXNzLXByb2dyYW09cGlneiA+L2Rldi9udWxsDQo+IH4k IGVjaG8gJD8NCj4gMA0KPiANCj4gQXJlIHlvdSBzdXJlIHlvdSBhcmUgdXNpbmcgRGViaWFuIDEy LjUgPw0KPiANCj4gS2luZCByZWdhcmRzLg0KPiANCj4gLS0NCj4gRnJhbmNvIE1hcnRlbGxpDQo=

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Thomas Schmitt@21:1/5 to Jonathan on Sat Apr 6 09:50:21 2024
    Hi,

    Chung Jonathan wrote:
    Yes, I think the local fix is the way to go.

    I wrote:
    (You forgot to Cc: debian-user@lists.debian.org.
    Consider to send your mail to the list address, too. I too would then resend my following reply to the list.)

    Since my "following reply" is quoted in Jonathan Chung's reply to the list
    i don't have to resend it. (I gave my opinion that the problem is not a
    bug in the context of Debian 12 or 13 and pointed to https://wiki.debian.org/BuildingTutorial for a private fix of the problem.)


    Jeffrey Walton wrote:
    Your problem is one that plagues Linux. You compile and link against
    one version of a library, and then you runtime link against another
    version.

    This should not be a problem with a well maintained library which cares
    to stay ABI compatible with its older releases.
    In the present case it was a bug in the loading program pigz which
    prevented zlib from being usable.


    I consider it a
    security bug since essentially random libraries are being loaded at
    runtime.
    To fix the problem yourself, add an RPATH to your LDFLAGS when
    building your program:
    -Wl,-rpath=/path/to/expected/libz -Wl,--enable-new-dtags

    Well, this is nearly as unflexible as static compilation but does not
    seem to prevent the use of a replaced library at the given path.

    Using .so files has its advantages and disadvantages. For a distro the advantage (without the pigz bug) is that customers of different versions
    of a library can be consolidated to using the newest available version.
    An advantage for the user is that bugs in a library can be fixed without
    the need for re-building all its customers.


    Have a nice day :)

    Thomas

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Charles Curley@21:1/5 to Jonathan Chung on Sat Apr 6 09:50:22 2024
    On Tue, 2 Apr 2024 10:27:33 +0200
    Jonathan Chung <jchung@student.ethz.ch> wrote:

    I'm new with Debian bug reporting and thus need some help with that.
    pigz 2.6-1 on Debian 12.5 fails to execute due to a fixed bug on
    upstream https://github.com/madler/pigz/issues/111

    Are you sure this is a bug and not a problem with your system? I am
    also using pigz 2.6-1 on Debian 12.5, and I don't see it.

    charles@hawk:~$ pigz --version
    pigz 2.6
    charles@hawk:~$ pre zlib pigz
    pigz 2.6-1 amd64
    zlib1g 1:1.2.13.dfsg-1 amd64
    zlib1g-dev 1:1.2.13.dfsg-1 amd64
    charles@hawk:~$

    What version of zlib are you running?

    --
    Does anybody read signatures any more?

    https://charlescurley.com
    https://charlescurley.com/blog/

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Thomas Schmitt@21:1/5 to Jonathan Chung on Sat Apr 6 09:51:44 2024
    Hi,

    Jonathan Chung wrote:
    pigz 2.6-1 on Debian 12.5 fails to execute due to a fixed bug on
    upstream https://github.com/madler/pigz/issues/111
    Installing the version from sid resolves the issue which is clearly not optimal. I think the fix should be backported.
    Can someone help me to file a bug report?

    Michael Kjörling wrote:
    https://www.debian.org/Bugs/Reporting

    This covers the question "how", but not "what".
    So my two cents:

    "Backported" is probably the wrong wish.
    https://backports.debian.org/Contribute/
    says:
    "Backports are about additional features that are only offered in a
    new version, not a replacement for getting fixes into stable - use
    stable-updates for that."

    That would probably mean to patch the existing pigz package in Debian 12
    by the upstream remedy
    "Make pigz compatible with two-component zlib version numbers. "
    https://github.com/madler/pigz/commit/907ca0763be4547a9b0cce8c105721748814974 ------------------------------------------------------------------------ pigz.c:
    @@ -1333,7 +1333,7 @@ local long zlib_vernum(void) {
    }
    ver++;
    } while (left);
    - return left < 2 ? num << (left << 2) : -1;
    + return left < 3 ? num << (left << 2) : -1;
    }

    // -- check value combination routines for parallel calculation -- ------------------------------------------------------------------------

    The new state can already be seen in
    https://sources.debian.org/src/pigz/2.8-1/pigz.c/#L1339
    The old state is in
    https://sources.debian.org/src/pigz/2.6-1/pigz.c/#L1317
    But there the "// -- check ..." comment is missing in favor of a line
    #ifndef NOTHREAD
    So a patch proposal would need some minor hand work.


    There remains the question, though, why you run into zlib-1.3 on a
    Debian "stable" system where the zlib version is "1:1.2.13.dfsg-1":
    https://tracker.debian.org/pkg/zlib


    Have a nice day :)

    Thomas

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Greg Wooledge@21:1/5 to Jeffrey Walton on Sat Apr 6 09:51:48 2024
    On Tue, Apr 02, 2024 at 06:34:43PM -0400, Jeffrey Walton wrote:
    On Tue, Apr 2, 2024 at 6:24 PM Chung Jonathan <jchung@student.ethz.ch> wrote:

    Dear Franco Martelli, dear Thomas Schmitt,

    Sorry for the potential duplication. This mail should now also go to the list.

    I believe I found the problem which was on my side. I do have libz.so.1.3, since I manually compiled grpc on my machine and this also uses a newer version of zlib appearently. So this is not a Debian problem but rather specific to my setup. A clean
    install in a VM indeed works as expected.
    Do you still think a bug report is worth it?

    Your problem is one that plagues Linux. You compile and link against
    one version of a library, and then you runtime link against another
    version. This should have been fixed for users a long time ago, but
    the folks responsible leave users to suffer it. I consider it a
    security bug since essentially random libraries are being loaded at
    runtime.

    To fix the problem yourself, add an RPATH to your LDFLAGS when
    building your program:

    -Wl,-rpath=/path/to/expected/libz -Wl,--enable-new-dtags

    The loader will encounter the RPATH when loading your executable, and
    load the correct library for your program.

    I've run into a variant of this issue myself. I had compiled an
    upstream version of Tcl, which builds a shared library named libtcl8.6.so
    in the lib/ subdirectory of the --prefix used for ./configure, where
    the default value of --prefix is /usr/local.

    This places the shared library in /usr/local/lib which happens to be in
    the default set of paths used by ldconfig and ld.so. Which in turn
    means that the locally built library gets used when running Debian's
    packaged version of Tcl, taking priority over the version installed in /usr/lib/x86_64-linux-gnu/ or whatver arch-specific directory is used.

    In my case, the solution was to build my local version of Tcl with a
    different --prefix. Then there's nothing dropped into /usr/local/lib
    and there's no conflict with the system's packages.

    Of course, this is a particularly subtle problem that most users are not
    going to be aware of. It's specific to packages that build part of
    themselves as a shared library, which immediately makes most programs
    immune to it. But when it *does* affect you, it can be tricky to
    figure out what's happening.

    It sounds like the OP's pigz issue was of a similar nature. If this is
    the case, then filing a Debian bug report isn't going to be helpful,
    unless it's done as a sort of documentation for anyone else who runs
    into the same problem in the future.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jonathan Chung@21:1/5 to All on Sat Apr 6 09:52:18 2024
    Hi,

    I think that the core problem which led to this also was on GRPCs side.
    They should not tell users to compile their library but rather point
    them to install it from their distros' packet manager (https://grpc.io/docs/languages/cpp/quickstart/#install-grpc) if
    possible. I will try to do the switch tonight as I just use GRPC as
    library in a project and don't need to change anything in the GRPC
    source.

    Have a nice day,

    Jonathan Chung

    Am Mittwoch, dem 03.04.2024 um 08:35 +0200 schrieb Thomas Schmitt:
    Hi,

    Chung  Jonathan wrote:
    Yes, I think the local fix is the way to go.

    I wrote:
    (You forgot to Cc: debian-user@lists.debian.org.
    Consider to send your mail to the list address, too. I too would
    then
    resend my following reply to the list.)

    Since my "following reply" is quoted in Jonathan Chung's reply to the
    list
    i don't have to resend it. (I gave my opinion that the problem is not
    a
    bug in the context of Debian 12 or 13 and pointed to https://wiki.debian.org/BuildingTutorial for a private fix of the
    problem.)


    Jeffrey Walton wrote:
    Your problem is one that plagues Linux. You compile and link
    against
    one version of a library, and then you runtime link against another version.

    This should not be a problem with a well maintained library which
    cares
    to stay ABI compatible with its older releases.
    In the present case it was a bug in the loading program pigz which
    prevented zlib from being usable.


    I consider it a
    security bug since essentially random libraries are being loaded at runtime.
    To fix the problem yourself, add an RPATH to your LDFLAGS when
    building your program:
        -Wl,-rpath=/path/to/expected/libz -Wl,--enable-new-dtags

    Well, this is nearly as unflexible as static compilation but does not
    seem to prevent the use of a replaced library at the given path.

    Using .so files has its advantages and disadvantages. For a distro
    the
    advantage (without the pigz bug) is that customers of different
    versions
    of a library can be consolidated to using the newest available
    version.
    An advantage for the user is that bugs in a library can be fixed
    without
    the need for re-building all its customers.


    Have a nice day :)

    Thomas


    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)