• [gentoo-dev] Re: Current unavoidable use of xz utils in Gentoo

    From Duncan@21:1/5 to All on Sat Mar 30 11:50:01 2024
    Dale posted on Sat, 30 Mar 2024 02:06:26 -0500 as excerpted:

    Gentoo has some awesome devs.

    Agreed with the whole thing and the above is a bit of an aside from the
    thread, but it's worth repeating!

    Thanks devs! (And security contributors, infra providers, testers,
    tinder-box runners, bug reporters and wranglers, docs/wiki/lists/forums/
    chat contributors, and all of the above for our upstreams too... it takes
    a big community to make a full distro and we all have our part! =:^)

    --
    Duncan - List replies preferred. No HTML msgs.
    "Every nonfree program has a lord, a master --
    and if you use the program, he is your master." Richard Stallman

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Sam James@21:1/5 to Duncan on Sat Apr 6 09:50:31 2024
    Duncan <1i5t5.duncan@cox.net> writes:

    Eddie Chapman posted on Tue, 2 Apr 2024 20:32:41 +0100 as excerpted:

    Yes, I have no issue with the format at all, just with the xz utils
    project.

    FWIW, feel free to do that bug-fix or package-bump if you'd rather instead
    of reading this long thing! I won't complain! =:^)

    Something's wrong - we're agreeing ;)

    You're spot on. Tangible real changes and efforts are useful, not
    "please boil the ocean".

    [...]

    thanks,
    sam

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Duncan@21:1/5 to All on Sat Apr 6 09:50:39 2024
    Kévin GASPARD DE RENEFORT posted on Wed, 3 Apr 2024 14:22:18 +0200 as excerpted:

    Fork Gentoo, or any other distros, start a LFS…

    In fact, Gentoo has been forked in this way at least three times. The
    first time was over 20 years ago, before 2004 as I remember researching it before I switched to Gentoo myself. IDR what it was called but it had
    already by then pretty much fizzled out.

    #2 and #3 are I believe still around and there's still some healthy
    interaction between them and Gentoo. Funtoo is one, created by Gentoo's original founder. It still uses portage and some of portage's features
    are primarily used there. As a result, their contributions back to
    portage have continued to make it better for all users. The other IDR the
    name (maybe Herb...?, with paludis as the package-manager) but PMS, the package-management-specification, that defines a portage- and package--
    manager independent spec and the EAPI that packages use, is one of the
    major efforts to come of it as they split off. And while most gentooers
    still use portage, pms is the reason other package managers can really
    work at all, and the thing much of the automated CI testing uses now,
    making it a BIG benefit to Gentoo.

    So forking is a legitimate and respected if not necessarily pleasant while
    it's happening route to go, and often, some contributions from the process continue to be useful to and benefit both forks for many years after the
    fork. While forks do generally mean duplication of effort in some areas
    and are often unpleasant to go through, the results aren't necessarily all
    bad, particularly when viewed with an appreciation that people often
    aren't paid for their work and could simply quite contributing entirely,
    which means the duplication of effort isn't all negative if it still means
    more contributions to the community than would happen if they just quit.

    So if Gentoo's not doing it for you, in addition to the option of creating
    your own fork being a not necessarily entirely bad option, there's two
    other existing forks you might wish to look into, in case they're a better
    fit for you than Gentoo.

    --
    Duncan - List replies preferred. No HTML msgs.
    "Every nonfree program has a lord, a master --
    and if you use the program, he is your master." Richard Stallman

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?K=C3=A9vin_GASPARD_DE_REN@21:1/5 to All on Sat Apr 6 09:51:00 2024
    Sorry but I wanted to add something to what is written below:

    I'll insist as other did before: An other alternative would be to start
    your own overlay, push something to help Gentoo's dev, anything, because
    saying more or less "Do that because actually it's bad" is something
    rarely appreciated from peoples already giving their free time (as in
    free beer), for zero money at the end of the month for it, mostly. Here
    we only see someone that is 100% sure he's right, when responsible*S*
    peoples says he's not. At least, start something, share, propose by
    actions. Then they will listen you a bit further and taking you way more seriously, I'm pretty sure of that.

    Please, respect that.

    Sorry again for my English.


    Hello,

    After so much reading and seeing almost a dead-end to this talk and
    from this citation above I had an idea for OP.

    1/ OP is sure that Gentoo and others distro *should* avoid using
    xz-utils, at all cost.

    (IMHO that is a respectable choice, *IF* it's possible without adding tremendous of works while Gentoo's dev could works on something else…
    Like being sure xz-utils is now safe to use…)

    2/ Gentoo's dev stating that it's:

        a) Non-required, to not say useless.

        b) Would ask a lot of money to extend the infrastructure of Gentoo (two times the compressed file and the new non-xz would take like +30%
    in size…) and some works in addition for the systems administrators.
    As someone that had this job for some years, that is not always easy
    as it looks like and having more works is never fun while you already
    have some cooking… specially when you are not paid for this.

        c) Would ask a *LOT* of works for Gentoo's devs, ebuild mainteneurs…

        d) For, from Gentoos's dev opinion, something that only a very few users will actually use, without speaking about adding a layer of
    complexity in every process, from installing Gentoo or maintaining the packages. Looks like an awful jobs to be honest.

    If OP is really that sure that Gentoo's dev are having a cavalier
    attitude, non-thinking enough about security in this subject, while
    (sorry but that's true) not paying much respect to the works into the community (Gentoo and free software in general)… Well:

    Fork Gentoo, or any other distros, start a LFS…

    I mean, this is *free software* (as in freedom), what makes you not
    starting your own project with peoples sharing your point-of-view ?

    Some debian's user didn't liked the coming of SystemD, some made
    Devian (not even know if it's still around, but that is a simple
    example). Don't some *BSD distribution were borne for technical
    different point-of-view ? Yes, some did and are still here, since
    decades.

    I think, IMHO, you should try to see if peoples around are having the
    same philosophy as you, if you find a bunch of peoples having times
    and willing to do it.

    I suppose you have some knowledge, but I can only assume, maybe you
    don't have enough, could take years even if you have already these.
    Even more if you start from 0.

    If you are alone, you have two choices:

    1/ Do like Slackware, create as a lone-wolf your own distribution.

    2/ Accept the idea that your idea is maybe not true, or good.

    When a lot of peoples state that you are wrong, it doesn't means you
    are all the time. But at the same time, you were explained more than
    once that it's not a good idea, a really better way or they (Gentoo's
    dev) have other matter to take care of. Maybe Gentoo's dev are wrong.
    But in my case, I'll keep my side for the peoples that has proven
    theirs skills by their works. For more than 20 years, now.

    That is just my opinion. You don't like it ? Fork it, find an
    alternative OR accept your faith. Or change for a distribution sharing
    your opinion about that.

    PS : Sorry for my English.

    Regards,
    GASPARD DE RENEFORT Kévin



    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?K=C3=A9vin_GASPARD_DE_REN@21:1/5 to All on Sat Apr 6 09:52:23 2024
    Helping with any of these three would certainly be reasonable. But
    demanding a *LOT* of work to alternative-force an already attack-reverted package, when we actually KNOW about that one, it's reverted to pre-attack and there's likely to be no more mischief there /because/ everybody's
    looking at it now, when it could have been any of a number of packages,
    some of which might already be compromised and we just didn't happen to
    find it, IMO really doesn't make much sense.

    Hello,

    After so much reading and seeing almost a dead-end to this talk and from
    this citation above I had an idea for OP.

    1/ OP is sure that Gentoo and others distro *should* avoid using
    xz-utils, at all cost.

    (IMHO that is a respectable choice, *IF* it's possible without adding tremendous of works while Gentoo's dev could works on something else…
    Like being sure xz-utils is now safe to use…)

    2/ Gentoo's dev stating that it's:

        a) Non-required, to not say useless.

        b) Would ask a lot of money to extend the infrastructure of Gentoo
    (two times the compressed file and the new non-xz would take like +30%
    in size…) and some works in addition for the systems administrators. As someone that had this job for some years, that is not always easy as it
    looks like and having more works is never fun while you already have
    some cooking… specially when you are not paid for this.

        c) Would ask a *LOT* of works for Gentoo's devs, ebuild mainteneurs…

        d) For, from Gentoos's dev opinion, something that only a very few users will actually use, without speaking about adding a layer of
    complexity in every process, from installing Gentoo or maintaining the packages. Looks like an awful jobs to be honest.

    If OP is really that sure that Gentoo's dev are having a cavalier
    attitude, non-thinking enough about security in this subject, while
    (sorry but that's true) not paying much respect to the works into the
    community (Gentoo and free software in general)… Well:

    Fork Gentoo, or any other distros, start a LFS…

    I mean, this is *free software* (as in freedom), what makes you not
    starting your own project with peoples sharing your point-of-view ?

    Some debian's user didn't liked the coming of SystemD, some made Devian
    (not even know if it's still around, but that is a simple example).
    Don't some *BSD distribution were borne for technical different
    point-of-view ? Yes, some did and are still here, since decades.

    I think, IMHO, you should try to see if peoples around are having the
    same philosophy as you, if you find a bunch of peoples having times and
    willing to do it.

    I suppose you have some knowledge, but I can only assume, maybe you
    don't have enough, could take years even if you have already these. Even
    more if you start from 0.

    If you are alone, you have two choices:

    1/ Do like Slackware, create as a lone-wolf your own distribution.

    2/ Accept the idea that your idea is maybe not true, or good.

    When a lot of peoples state that you are wrong, it doesn't means you are
    all the time. But at the same time, you were explained more than once
    that it's not a good idea, a really better way or they (Gentoo's dev)
    have other matter to take care of. Maybe Gentoo's dev are wrong. But in
    my case, I'll keep my side for the peoples that has proven theirs skills
    by their works. For more than 20 years, now.

    That is just my opinion. You don't like it ? Fork it, find an
    alternative OR accept your faith. Or change for a distribution sharing
    your opinion about that.

    PS : Sorry for my English.

    Regards,
    GASPARD DE RENEFORT Kévin

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Duncan@21:1/5 to All on Sat Apr 6 09:52:39 2024
    Eddie Chapman posted on Tue, 2 Apr 2024 20:32:41 +0100 as excerpted:

    Yes, I have no issue with the format at all, just with the xz utils
    project.

    FWIW, feel free to do that bug-fix or package-bump if you'd rather instead
    of reading this long thing! I won't complain! =:^)

    IMO...

    The thing is, the actual problem isn't the xz-utils project. Roll the
    dice. The attack could have happened to any one (or more) of a number of projects... and a few years ago, before the Linux Foundation sponsored a project to coordinate helping out various one-person upstreams with
    security sponsorships and etc, it could have been quite a few more. xz-
    utils just happens to be the one one with the bad luck to be attacked and
    where we actually discovered the attack... that wasn't yet covered by the
    LF security support program as its "core Linux infra" connection via
    systemd is relatively new and only developed more or less in parallel with
    that program, so it had until now fallen through the cracks.

    Now that the attack on xz-utils has been exposed, we've already rolled
    back to before the attacker got involved. So we see the (symptomatic/
    surface) problem there and have already addressed it, tho as I said the
    real problem isn't xz-utils at all.

    Sure we could go to more extreme lengths to allow xz-utils to be taken out
    of the picture, but what would that do? We've already reverted to before
    the attacker was in the picture, and it's simply impossible to alternative-force /all/ of the currently at-risk packages, some of which
    might already be compromised only we don't know it yet, actually putting
    them in a WORSE position, or there'd likely not be enough left to make a working distro!

    In fact, the attacker (or one of the near-zero-history posters that urged
    his xz-utils releases be accepted, I don't remember the specifics as I
    read a lot of articles and a lot of followup discussion over the weekend)
    had apparently also made a commit to libarchive, tho it's quite possible
    it was of the "gain their trust with an innocent commit first" kind or
    that they abandoned that effort when the xz-utils effort appeared to be
    working better. Of course by the time I read about that over the weekend people were already scouring that and looking for others.

    Are we going to kill libarchive too, when it's just one commit and it's
    now known and scoured? How many other packages are going to have low- history/bad-history commits that look iffy now? How may of them can we
    really find alternatives for that don't have the same or worse problems?

    So we can't throw the baby out with the bath-water, which is where we'd
    end up if we took the same approach for all the packages in a similar
    situation if not worse because we can't fix what we haven't discovered
    yet!

    Rather, the (IMO) more reasonable approach is what people are already
    doing, addressing the systemic issues.

    One of these systemic issues is that for not until now examined historical reasons, it's considered reasonable for release tarballs to differ from
    the tarball created from the pure repo release-tag checkout. In some
    cases that's at least presently needed due to the way autotools and
    traditional release processes work, but that's being reexamined now, with
    some packages already able to switch to release-tag-corresponding
    tarballs, while others can't yet, but are high-priority examining changes
    in procedure and/or release tooling so they can. So that's likely to
    change for many packages within a release or two, and people will be
    pressuring the ones that don't and/or considering switching to
    alternatives.

    *That* is actually a (IMO) /reasonable/ alternative-consideration -- over
    the next months, examine packages that aren't already switching to tag- corresponding release tarballs and consider alternatives to /them/!

    Another of the systemic issues is the number of Linux-core-infra packages
    with a "bus-factor" of one or even two (consider that until this happened, xz-utils seemed to now have two, nobody realizing the one was actually an attacker). Now that xz-utils had this known attack AND it's now known to
    be core-critical (for distros with that systemd integration... which of
    course includes both two big names and two enterprise products with real
    money behind them), I'm sure the original author has all sorts of offers
    of help, both for simple maintenance, and scouring for any other security issues. We really shouldn't have to worry about it any longer. And I've already mentioned the LF security help program which helps for many. But there's likely a dozen (more?) other packages that are either relatively
    newly core-integrated or that have fallen through the cracks until now for other reasons. There's going to be real-money efforts to find these and
    add them to the security-help list now, because there's real-money
    products at stake!

    Yet another issue, once tarballs can be verified against release tags, is
    that a lot of distro maintainers don't actually verify the code changes.
    Some simply don't have the necesary skills. Others have the skills, but
    still don't verify, because the tooling isn't there to make it fast/simple enough for them and there's always more packages to bump then time to
    actually do it.

    Now, due to the xz-utils attack revealing the problem, there's already community efforts to improve the tooling to make it easier for distro maintainers to not only look at the commit logs, but go a bit deeper than
    that and better show the changed code. And many maintainers are
    redoubling their efforts to make routine at least minimal change-audits
    with the existing tooling in the mean time.


    Helping with any of these three would certainly be reasonable. But
    demanding a *LOT* of work to alternative-force an already attack-reverted package, when we actually KNOW about that one, it's reverted to pre-attack
    and there's likely to be no more mischief there /because/ everybody's
    looking at it now, when it could have been any of a number of packages,
    some of which might already be compromised and we just didn't happen to
    find it, IMO really doesn't make much sense.

    --
    Duncan - List replies preferred. No HTML msgs.
    "Every nonfree program has a lord, a master --
    and if you use the program, he is your master." Richard Stallman

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Duncan@21:1/5 to All on Fri Apr 12 09:20:01 2024
    Joonas Niilola posted on Thu, 11 Apr 2024 08:21:39 +0300 as excerpted:

    I just want to point out you
    may not have to edit ebuilds at all. If xz-utils is package.provided
    portage should ignore the dependency without you removing the dep from
    an ebuild.

    package.provided:

    YMMV, but here rather than doing package.provided I create a "null-ebuild"
    for the package in my overlay. Much like virtual/* packages from which I
    took the idea except of course that they're named as the category/package they're replacing instead of virtual/*, null-ebuilds install no files but
    allow detailed control of IUSE, slot, etc, in case some of the revdeps
    need that.

    For versioning, my convention is -999 or -n.999 to imply a virtual (tho I
    do have a real perl bigint package v 1.999.842 installed...), much like
    the -9999/-n.9999 variants imply a live-package, with similar effect in
    terms of preferring it to any reasonable real version number as well. So
    for xz-utils it'd be app-arch/xz-utils-999 as it's not slotted, or app- arch/xz-utils-5.999 if it were or if something wants 5.x specifically. Or
    use five-nines or six-nines or ten nines...

    A null-pkg I actually use here? sys-fs/udisks-2.999:2 (slot 2 dep actually required by some of its rev-deps). udisks itself is a script so doesn't provide headers necessary to build other things and should be a runtime-
    only dep. As a script the installation itself would be too trivial to
    bother with, were it not for its absolutely wicked pulled-in deps for functionality I'm not going to use and don't have turned on for my kernel
    in any case. Luckily kde/solid/kio/etc degrade functionality gracefully
    if their attempted udisks calls return command-not-found, making it an
    ideal candidate for null-pkging.

    --
    Duncan - List replies preferred. No HTML msgs.
    "Every nonfree program has a lord, a master --
    and if you use the program, he is your master." Richard Stallman

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)