XPost: linux.debian.maint.glibc, linux.debian.maint.perl

Hello fellow developers,

how about libc6-dev stops depending on libcrypt-dev?

I mean for real. We cannot do it right away. The proposal is forky
material of course. Also libc6-dev would still "Recommends:
libcrypt-dev", but libcrypt-dev would no longer be build-essential.

The obvious "why" question needs an answer. libxcrypt and glibc pose a
2-cycle during architecture cross bootstrap and dropping the dependency
solves that cycle. It also is not that many packages that actually use libcrypt-dev as we shall see.

Let's move into the "how" part. The easy part is preparing a glibc
package with the dependency dropped. You may find one at https://people.debian.org/~helmutg/glibc-no-crypt/build/. Santiago Vila
kindly performed an archive rebuild using this modified glibc (and with libcrypt-dev removed) and handed me some logs. I sorted those logs and
you may now find them at https://people.debian.org/~helmutg/glibc-no-crypt/logs/.

Now we need to talk about numbers. It's 803 FTBFS. Of those, 576 issue a dependency on libperl-dev and since perl #includes <crypt.h> somewhere, libperl-dev should be depending on libcrypt-dev. There are also 95
packages that build a perl extension module without depending on
perl-xs-dev. Such a practice causes them to FTCBFS, but with
libcrypt-dev that FTCBFS is turned into a FTBFS. Beyond that, 11
packages build a perl extension module for testing purposes and
therefore need "Build-Depends: perl-xs-dev <!nocheck>". One package
actually embeds a perl interpreter without depending on libperl-dev.

So far so good. That's 1 + 95 + 11 + 1 = 108 source packages from the
perl ecosystem in need of changes. What about others?

I classified 110 of the packages as needing a direct libcrypt-dev
dependency as they use it in their source directly. And then we have a
few packages that indirectly use libcrypt. The following packages ship a pkgconfig file containing -lcrypt and therefore need "Depends:
libcrypt-dev".
* guile-2.2-dev
* guile-3.0-dev
* heimdal-multidev
* libapr1-dev
* libidzebra-2.0-dev
* librep-dev
* libuser1-dev
* ruby3.1-dev
* ruby3.3-dev
In addition, gauche-dev has gauche-config that also emits -lcrypt. Now
matching this up with the build failures yields four that will be fixed
be one of these being modified.

There is one further category. 6 packages link -lcrypt without using it
in any way. I went right ahead and filed patches: https://udd.debian.org/cgi-bin/bts-usertags.cgi?user=helmutg@debian.org&tag=libcrypt-unused

So that's modifying another 110 + 9 + 1 + 6 = 126 source packages
outside the perl ecosystem. You'll find all of the mentioned categories
in the published logs as subdirectories. Please bear in mind that among
the packages that FTBFS in unstable, a small fraction would additionally
FTBFS without libcrypt and I've missed those. Expect a few more.

If we assume all of these changes, we've addressed all FTBFS, but we're
not done yet. What about packages that check for libcrypt and silently
opt out of it when it is unavailable? Typically those packages would
emit a runtime dependency on libcrypt1, right? So I also investigated
all apt-cache rdepends libcrypt1. That results in 151 source packages.
Of those, a large fraction will already be covered by investigating
build failures. Additionally, we had to add runtime dependencies to a
few packages above. And then there are 11 source packags that already
build depend on libcrypt-dev (mostly to support bootstrapping). So if
you disregard all of those duplicates, what remains is 28 packages
missed in the FTBFS-based analysis:
* alpine
* asterisk
* burp
* ccrypt
* courier-authlib
* cpu
* cvm
* freewnn
* frr
* guestfs-tools
* ircii
* lighttpd
* lsh-utils
* mariadb
* netatalk
* pike8.0
* ppp
* python-oslo.utils
* python3.12
* regina-rexx
* rserve
* scrollz
* steam-installer
* tinymux
* ukui-control-center
* virt-v2v
* x11vnc
* xemacs21

I also considered running autopkgtests of glibc's reverse dependencies,
but that proved to be harder than expected.

In the end, we also need to modify glibc to actually demote the
dependency. So now we've reached my final number. 108 + 126 + 28 + 1 =
263 source packages need modifications. Some issues may remain here, but
at this time I don't expect large numbers anymore. On the perl side, a practical side effect will be fixing couple of cross builds.

Question 1: Do you see important aspects missed in this analysis?

Question 2: Do you agree that this change is worth the effort?

Assuming "no" and "yes" as answers, I'd like to proceed with mass bug
filing. I propose severity normal for all bugs. The additional
dependencies can be piggy-backed onto uploads aimed for trixie as their
risk of causing breakage is really low. For the perl-related changes we
might skip filing the bugs and consider mass-committing the changes for
a later upload. The glibc change must not be done during the trixie
cycle, but can happen early in the forky one. At that point, the
remaining bugs would become RC.

Question 3: Can I move forward with the MBF?

I'd appreciate explicit replies from:
* libxcrypt maintainer Marco
* glibc maintainers (e.g. Aurelien)
* Debian Perl Group (e.g. Gregor) (high number of affected packages)

Thanks for considering

Helmut

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 10 Apr 2025 at 07:37:32 +0200, Helmut Grohne wrote:

how about libc6-dev stops depending on libcrypt-dev?

I think this is a good idea for early in the forky cycle.

I also investigated
all apt-cache rdepends libcrypt1. That results in 151 source packages.

...

* steam-installer

This particular one is a false positive, it hard-codes a dependency on libcrypt1 (or more precisely libcrypt1 | libc6 (<< 2.29-4)) as one of
the SONAMEs that Steam relies on the host system to provide (because it
was historically part of glibc). It shouldn't need a -dev build-dependency
for this purpose.

smcv

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

XPost: linux.debian.maint.glibc, linux.debian.maint.perl

On Apr 10, Helmut Grohne <helmut@subdivi.de> wrote:

how about libc6-dev stops depending on libcrypt-dev?

Sure.

material of course. Also libc6-dev would still "Recommends:
libcrypt-dev", but libcrypt-dev would no longer be build-essential.

What purpose would this Recommends solve?

Assuming "no" and "yes" as answers, I'd like to proceed with mass bug
filing. I propose severity normal for all bugs. The additional

Go for it, great job!
Removing the package from the build-essential set is a worthwhile goal.

But please mention prominently that there is no need to work on this
before the release.

--
ciao,
Marco

-----BEGIN PGP SIGNATURE-----

iHUEABYIAB0WIQQnKUXNg20437dCfobLPsM64d7XgQUCZ/eI5AAKCRDLPsM64d7X gZPcAQCPwNzAZKtjavej2ygcFfWDpPK5PyhJzYdeTsP3T6QadgD9G8fVFQzr6XSq 1zhQxjV2QrixlZhQ1XVISl2CTAVDQgE=
=eMq4
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

XPost: linux.debian.maint.glibc, linux.debian.maint.perl

Hi Marco,


On Thu, Apr 10, 2025 at 11:01:24AM +0200, Marco d'Itri wrote:

On Apr 10, Helmut Grohne <helmut@subdivi.de> wrote:

how about libc6-dev stops depending on libcrypt-dev?

Sure.

Thanks for the feedback.

material of course. Also libc6-dev would still "Recommends:
libcrypt-dev", but libcrypt-dev would no longer be build-essential.

What purpose would this Recommends solve?

My thinking here was to reduce the annoyance for users by degrading the dependency softly. I was told that some users expect to be able to build
perl extensions without installing libperl-dev (which is why libperlVER contains the headers that #include <crypt.h>). If we were right dropping
the dependency, libcrypt-dev could be automatically be removed from
those systems and we'd get angry bug reports. Going to Recommends is a
means to limit the annoyance while still meeting the original goal.

This is not to be read as an endorsement of libc6-dev permanently
recommending libcrypt-dev. For instance downgrading it to Suggests or
dropping it entirely could be an option for later. To me, the key
feature here is making it removable in the first place.

In any case, I'll leave this particular aspect to the glibc and
libxcrypt maintainers. If you feel, Recommends is not warranted, so be
it.

Helmut

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

[ Thanks for this. Awesome work! ]

El 10/4/25 a las 11:43, Helmut Grohne escribió:

My thinking here was to reduce the annoyance for users by degrading the dependency softly. I was told that some users expect to be able to build
perl extensions without installing libperl-dev (which is why libperlVER contains the headers that #include <crypt.h>). If we were right dropping
the dependency, libcrypt-dev could be automatically be removed from
those systems and we'd get angry bug reports. Going to Recommends is a
means to limit the annoyance while still meeting the original goal.

On the other hand:

- A recommends will only have some effect for people not having
libcrypt-dev installed yet, so people upgrading from trixie to forky
who have both libc6-dev and libcrypt-dev installed will not see the libcrypt-dev package mysteriously removed from their systems.

- In general, Debian users are already aware that all sorts of breakage in unstable is expected to happen shortly after a stable release, so I would
not really expect angry bugs about this from users.

So, I agree with Marco that Recommends is not necessary.

For the purposes of being nice to people, I would maybe extend
the period where the bugs are not RC as necessary so that we don't
introduce a lots of RC bugs at once (i.e. raise to serious not
in a date-based scheme but when the number of bugs is low enough).

Thanks.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

XPost: linux.debian.maint.perl

On Thu, 10 Apr 2025 07:37:32 +0200, Helmut Grohne wrote:

I sorted those logs and
you may now find them at >https://people.debian.org/~helmutg/glibc-no-crypt/logs/.

Thanks.

Beyond that, 11
packages build a perl extension module for testing purposes and
therefore need "Build-Depends: perl-xs-dev <!nocheck>".

I learned something new :)

For the perl-related changes we
might skip filing the bugs and consider mass-committing the changes for
a later upload.

At first glance that sounds reasonable (and save us from the work of correlating and closing bugs in packages). At second glance, i.e.
after looking at <https://people.debian.org/~helmutg/glibc-no-crypt/logs/needs-perl-xs-dev/>,
I note that many packages there don't match the 'lib.+-perl' pattern,
i.e. are not maintained by the Debian Perl Team, and from the
remaining ones, (without checking them) I also don't recognize quite
a few package names. ¹

I think the "mass" for "mass-commiting" would not be high enough to
rely on this mechanism, so I guess including them in the MBF makes
more sense.


Cheers,
gregor


¹ I'm not surprised that this problem mainly affects
non-team-maintained packages, as we've been using perl-xs-dev since
quite some time …

--
.''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org
: :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06
`. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
`-

-----BEGIN PGP SIGNATURE-----

iQKTBAEBCgB9FiEE0eExbpOnYKgQTYX6uzpoAYZJqgYFAmf3zqtfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEQx RTEzMTZFOTNBNzYwQTgxMDREODVGQUJCM0E2ODAxODY0OUFBMDYACgkQuzpoAYZJ qgYhfBAAwPn3q0uTcupjd4jc4PHDG1voYLPb3Xy+UsFJG6IDulKMnD9LyQ1LvMSB rJkxeEU0QiiJpvlVuZ/SU9+6RVrKZoeYyCpmio1HwNUWaT6ZLPQ8iqFgsuu8vnax GaMv/6l7j9FcXWQdz8hNRUXixGh1Fc7WtvNrf3eo0BAPM5vdIaaquSMQTy/DwhYJ QUjVMp8crQFl4sKDfePSmDyrVsfWR1/Q13knV/6YjyC2myXqlAA/J3BGir6xzGw2 yxNzIYYHC08DeEeAIwUT1Y79BSYXIg86f+E9jAqSgBOslD46IFdMNV7voqgXPd77 G4VaDp1nNWOLTqlOK/SRAKdXpMHkYJy+OckWou1ahrPJKJQ0V7unCQbRRNsrkYRm H179PvpUNM4e8lP1A+Dr2DS1iS31ebWQuG06MPbU/efNLKsbPOsQ1z9g0M3I7v1x saNfG5wVzUWDnLqcGHDo5rKR58HBf1k11+ToEqpzdvFHLTz7hOPHBkzoH9g1U7Oy
H7YS

XPost: linux.debian.maint.glibc, linux.debian.maint.perl

Hi,

On 2025-04-10 07:37, Helmut Grohne wrote:

Hello fellow developers,

[ Snip ]

Question 1: Do you see important aspects missed in this analysis?

No

Question 2: Do you agree that this change is worth the effort?

I don't know. I do not see a huge benefit from the glibc point of view,
but I understand that it could be a benefit for other parts of Debian.
As long as these other part leads the effort that sounds good.

Assuming "no" and "yes" as answers, I'd like to proceed with mass bug
filing. I propose severity normal for all bugs. The additional
dependencies can be piggy-backed onto uploads aimed for trixie as their
risk of causing breakage is really low. For the perl-related changes we
might skip filing the bugs and consider mass-committing the changes for
a later upload. The glibc change must not be done during the trixie
cycle, but can happen early in the forky one. At that point, the
remaining bugs would become RC.

It seems to me that this way of doing might block glibc development, ie
prevent glibc from migrating to testing for many weeks, as I guess it
will causes autopkgtest failures in addition to FTBFS.

I think we should proceed with the change on the glibc side once we know
that the bugs can be be solved *in testing* in a reasonable time frame.
This could mean massive NMUs to fix the remaining issues (but this could
be seen as aggressive). An other option is to upgrade the severity to
serious before the change on the glibc side, which should remove most
unfixed packages from testing within 1 month, and the remaining packages
can be fixed through NMU.

Question 3: Can I move forward with the MBF?

Sounds good to me. Whatever is decided, having an explicit dependency is
always better, than relying on de facto build-essential.

Regards
Aurelien

--
Aurelien Jarno GPG: 4096R/1DDD8C9B aurelien@aurel32.net http://aurel32.net

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

XPost: linux.debian.maint.glibc, linux.debian.maint.perl

Hello fellow developers,

On Thu, Apr 10, 2025 at 07:37:32AM +0200, Helmut Grohne wrote:

how about libc6-dev stops depending on libcrypt-dev?

with minor disagreement in details, I have received much positive
feedback and therefore moved forward.

So far so good. That's 1 + 95 + 11 + 1 = 108 source packages from the
perl ecosystem in need of changes. What about others?

All of the perl ones have been filed and gregor already fixed (thanks!)
a significant fraction including all 11 perl-xs-dev <!nocheck> ones. I
guess that on third of these is fixed in git or unstable.

few packages that indirectly use libcrypt. The following packages ship a pkgconfig file containing -lcrypt and therefore need "Depends:
libcrypt-dev".
* guile-2.2-dev
* guile-3.0-dev
* heimdal-multidev
* libapr1-dev
* libidzebra-2.0-dev
* librep-dev
* libuser1-dev
* ruby3.1-dev
* ruby3.3-dev
In addition, gauche-dev has gauche-config that also emits -lcrypt. Now matching this up with the build failures yields four that will be fixed
be one of these being modified.

Bugs with patches are filed for these. https://udd.debian.org/cgi-bin/bts-usertags.cgi?user=helmutg@debian.org&tag=libcrypt-pkgconf
Notably, libuser is missing as I did a QA upload instead.

So that's modifying another 110 + 9 + 1 + 6 = 126 source packages
outside the perl ecosystem. You'll find all of the mentioned categories
in the published logs as subdirectories. Please bear in mind that among
the packages that FTBFS in unstable, a small fraction would additionally FTBFS without libcrypt and I've missed those. Expect a few more.

...

build depend on libcrypt-dev (mostly to support bootstrapping). So if
you disregard all of those duplicates, what remains is 28 packages
missed in the FTBFS-based analysis:

I have not yet filed bugs for packages lacking "Build-Depends:
libcrypt-dev". That's a next step. I consider the perl-xs-dev
dependencies and the runtime dependencies more important as both of them
also affect other use cases (such as cross building).

I'd appreciate explicit replies from:

Thanks for the quick feedback!

Regarding the timing of the glibc upload, I also am in favor of not
upgrading lots of these bugs to rc severity. Given the usertags, we may
monitor how the situation evolves in forky. I suggest once the remaining unfixed bug count (across all categories) is 30 or less, we may proceed
and upgrade the remaining ones to rc.

Helmut

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

XPost: linux.debian.maint.glibc, linux.debian.maint.perl

Hello fellow developers,

On Sun, Apr 13, 2025 at 05:25:59PM +0200, Helmut Grohne wrote:

All of the perl ones have been filed and gregor already fixed (thanks!)
a significant fraction including all 11 perl-xs-dev <!nocheck> ones. I
guess that on third of these is fixed in git or unstable.

I guess half of the perl ones are fixed. That is despite them being
filed during the trixie freeze. Hope this didn't cause disruption.

So that's modifying another 110 + 9 + 1 + 6 = 126 source packages
outside the perl ecosystem. You'll find all of the mentioned categories
in the published logs as subdirectories. Please bear in mind that among
the packages that FTBFS in unstable, a small fraction would additionally FTBFS without libcrypt and I've missed those. Expect a few more.

...

build depend on libcrypt-dev (mostly to support bootstrapping). So if
you disregard all of those duplicates, what remains is 28 packages
missed in the FTBFS-based analysis:

I have not yet filed bugs for packages lacking "Build-Depends:
libcrypt-dev". That's a next step. I consider the perl-xs-dev
dependencies and the runtime dependencies more important as both of them
also affect other use cases (such as cross building).

I have now collected the remaining packages. I didn't fully account for packages with binary package dependencies earlier, so it's 155 bugs.
They're all filed now and this was the last category to be filed. I
explicitly noted that they should not be uploaded to trixie.

Regarding the timing of the glibc upload, I also am in favor of not
upgrading lots of these bugs to rc severity. Given the usertags, we may monitor how the situation evolves in forky. I suggest once the remaining unfixed bug count (across all categories) is 30 or less, we may proceed
and upgrade the remaining ones to rc.

All of the bugs are usertagged with a number of tags sharing a
"libcrypt-" prefix:

https://udd.debian.org/cgi-bin/bts-usertags.cgi?user=helmutg@debian.org

At the time of this writing, there are 233 remaining bugs.

Helmut

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)