On Sun, Apr 20, 2025 at 04:46:52PM -0500, G. Branden Robinson wrote:

At 2025-04-20T23:22:04+0500, Andrey Rakhmatullin wrote:

On Sun, Apr 20, 2025 at 06:25:53PM +0100, Josh Triplett wrote:

What I'm suggesting here is that if every individual package that
needs awk has a Depends on it (via a package that allows switching implementations), rather than relying on Essential, then it becomes possible to make incremental progress, and that incremental progress benefits people who are willing to carefully remove some of what
Debian normally always has installed packages.

Should we start declaring deps on all essential packages explicitly?

I think that's a good idea. "Explicit is better than implicit," as the
Zen of Python puts it.[1]

Factual statements about one's run-time dependencies should be as
decoupled from the details of the set of "Essential" packages as
possible. One reason is that the identities of the people making these decisions are disjoint. Often a package maintainer lacks this power;
except for dependencies they introduce through operation of their
maintainer scripts (or Debian add-ons), such run-time dependencies are
beyond their control.
...

While this might sound good in theory, in practice it would be horrible.

As an example, libc6 has a preinst script that calls dpkg, sed,
grep and rm.

Making these dependencies explicit would be something like
Pre-Depends: sh, dpkg, sed, grep, coreutils

I would expect that such Pre-Depends cycles between essential packages
and libc6 will result in broken systems during upgrades.


And then there's the normal time-waste like "the package ships
a bash-completion file that uses awk, grep, sed and sort - that's
dependencies on four essential packages".

Regards,
Branden
...

cu
Adrian

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

G. Branden Robinson wrote:

At 2025-04-20T23:22:04+0500, Andrey Rakhmatullin wrote:

On Sun, Apr 20, 2025 at 06:25:53PM +0100, Josh Triplett wrote:

What I'm suggesting here is that if every individual package that
needs awk has a Depends on it (via a package that allows switching implementations), rather than relying on Essential, then it becomes possible to make incremental progress, and that incremental progress benefits people who are willing to carefully remove some of what
Debian normally always has installed packages.

Should we start declaring deps on all essential packages explicitly?

I think that's a good idea. "Explicit is better than implicit," as the
Zen of Python puts it.[1]

Agreed, but...

Factual statements about one's run-time dependencies should be as
decoupled from the details of the set of "Essential" packages as
possible.

[...]

By contrast, the population of the Essential set is up to...well, I'm
not sure who. Some vaguely defined intersection of the dpkg
maintainer(s), the release managers, and installer team, I guess.

In principle, the all of the developers collectively (and interested discussants) are responsible for such decisions. Unfortunately,
decisions in Debian are sometimes not made by those whom we claim.

"You must not tag any packages essential before this has been discussed
on the debian-devel mailing list and a consensus about doing that has
been reached." -- Debian Policy Manual, �3.8[2]

That implies to me that a package can be taken _out_ of the essential
set unilaterally by the package maintainer(s) of a package that's in it,
but because of the status quo of being able to depend on an essential
package without declaring that fact, in practice that probably wouldn't
work well, and we should update the Policy Manual to require discussion
of the dropping of such a "tag" as well.

I think that's a bug in Policy as written, rather than a bug in
practice. Historical practice has definitely been to discuss such
removals (extensively).

We should have a well-defined process for this, that includes discussion transition plans (involving the introduction of Depends as needed
first), and similar.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sun, Apr 20, 2025 at 06:58:36PM -0500, G. Branden Robinson wrote:

Hi Adrian,

Hi Branden,

At 2025-04-21T01:34:56+0300, Adrian Bunk wrote:
...
My impression from watching countless upgrades over the years is that,
when any Essential package is available for upgrade, upgrades performed
by apt launch a full, discrete run of dpkg for each.[3] So the only way
any such cycles could exist is if the Pre-Depends were versioned and the versionings themselves produced the cycle, in which case there likely
_really is_ a breakage problem: one package or the other needs to delay
its aggressive employment of a newly available feature.

awk is a virtual essential package, it is theoretically possible
that dependency resolution during an upgrade deinstalls the previously installed awk implementation and installs a different one instead.

And then there's the normal time-waste like "the package ships
a bash-completion file that uses awk, grep, sed and sort - that's dependencies on four essential packages".

Hmm. In my opinion bash-completion scripts should produce "Recommends"-strength dependencies at most; not everybody's going to be
using bash as their interactive shell, and completion matters only for interactive shells.

You want to invent a huge amount of error-prone manual work for
contributors that also adds completely new problems like this?

You want all contributors to know that rm and sort are in coreutils, but
grep and sed have own packages, and then add dependencies based on that?

A program using system() needs /bin/sh, which implies it would need a dependency on an essential (virtual?) shell package.

The amount of dependencies in the archive is already a resource issue
for dependency resolution.

All this is just an incredibly stupid idea.

The essential set exists so that users and developers always have all
the basic tools of a Linux system available.

Debian is a binary distribution with nearly 40k source packages,
and there are use cases where the only sane answer is to use a
different distribution.

If anyone whould actually care about the size of awk on small images,
the 8 year old #861343 would be a low-hanging fruit to get a third of
that removed in small images without requiring any archive-wide changes.

That said, I haven't looked but would not be surprised if
bash-completion scripts make extremely aggressive assumptions about
what's available on the system. That is, they seldom if ever look
before the leap, and happily assume the presence of plenty of things
that _aren't_ in Essential, or even "Priority: required" packages.

But that's just a hunch. I could be wrong.

Missing dependencies (which are rare) are reported fast by users since
they break the bash completion.

Regards,
Branden
...

cu
Adrian

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Mon, 21 Apr 2025 01:34:56 +0300, Adrian Bunk <bunk@debian.org>
wrote:

On Sun, Apr 20, 2025 at 04:46:52PM -0500, G. Branden Robinson wrote:

Factual statements about one's run-time dependencies should be as
decoupled from the details of the set of "Essential" packages as
possible. One reason is that the identities of the people making these
decisions are disjoint. Often a package maintainer lacks this power;
except for dependencies they introduce through operation of their
maintainer scripts (or Debian add-ons), such run-time dependencies are
beyond their control.
...

While this might sound good in theory, in practice it would be horrible.

As an example, libc6 has a preinst script that calls dpkg, sed,
grep and rm.

Making these dependencies explicit would be something like
Pre-Depends: sh, dpkg, sed, grep, coreutils

I would expect that such Pre-Depends cycles between essential packages
and libc6 will result in broken systems during upgrades.

I have always been less than a fan of not being allowed to declare
dependency on bash or coreutils, but a (non-logical) tradeoff would be
allowing to declare such dependencies, and liberally removing (or
demoting) them if they cause a cycle. I don't expect that to happen
too often.

And then there's the normal time-waste like "the package ships
a bash-completion file that uses awk, grep, sed and sort - that's >dependencies on four essential packages".

I'd rather do that than having to spend hours updating year numbers in
machine readable debian/copyright.

Greetings
Marc
--
---------------------------------------------------------------------------- Marc Haber | " Questions are the | Mailadresse im Header Rhein-Neckar, DE | Beginning of Wisdom " |
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 6224 1600402

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sun Apr 20, 2025 at 11:34 PM BST, Adrian Bunk wrote:

While this might sound good in theory, in practice it would be
horrible.

As an example, libc6

It's a good example of something that could be a problem and would need careful attention, but an essential library, and in particular libc6,
doesn't likely reflect the experience that we'd have for the vast
majority of packages.


--
Please do not CC me for listmail.

👱🏻 Jonathan Dowland
✎ jmtd@debian.org
🔗 https://jmtd.net

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

From: g.branden.robinson@gmail.com

At 2025-04-20T23:22:04+0500, Andrey Rakhmatullin wrote:

On Sun, Apr 20, 2025 at 06:25:53PM +0100, Josh Triplett wrote:

What I'm suggesting here is that if every individual package that
needs awk has a Depends on it (via a package that allows switching implementations), rather than relying on Essential, then it becomes possible to make incremental progress, and that incremental progress benefits people who are willing to carefully remove some of what
Debian normally always has installed packages.

Should we start declaring deps on all essential packages explicitly?

I think that's a good idea. "Explicit is better than implicit," as the
Zen of Python puts it.[1]

Factual statements about one's run-time dependencies should be as
decoupled from the details of the set of "Essential" packages as
possible. One reason is that the identities of the people making these decisions are disjoint. Often a package maintainer lacks this power;
except for dependencies they introduce through operation of their
maintainer scripts (or Debian add-ons), such run-time dependencies are
beyond their control.

By contrast, the population of the Essential set is up to...well, I'm
not sure who. Some vaguely defined intersection of the dpkg
maintainer(s), the release managers, and installer team, I guess.

In principle, the all of the developers collectively (and interested discussants) are responsible for such decisions. Unfortunately,
decisions in Debian are sometimes not made by those whom we claim.

"You must not tag any packages essential before this has been discussed
on the debian-devel mailing list and a consensus about doing that has
been reached." -- Debian Policy Manual, �3.8[2]

That implies to me that a package can be taken _out_ of the essential
set unilaterally by the package maintainer(s) of a package that's in it,
but because of the status quo of being able to depend on an essential
package without declaring that fact, in practice that probably wouldn't
work well, and we should update the Policy Manual to require discussion
of the dropping of such a "tag" as well.

...for which achievement of the goal you propose is a prerequisite.

One of the current Policy Manual editors might opine.

Regards,
Branden

[1] https://peps.python.org/pep-0020/
[2] https://www.debian.org/doc/debian-policy/ch-binary.html#essential-packages

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEh3PWHWjjDgcrENwa0Z6cfXEmbc4FAmgFa0UACgkQ0Z6cfXEm bc6sVg/8D/naeYviLTklYDn5G2a310xvOTbrr8B1DBBcngEoYyj2TIz1La7gsvWG KWSQYnDoxJQSCAholu5pvMv9Gm4dN6/efI0SW1cPInn1tgbudRfnR+kuTZHGymgS +x6U/MtaGxVh/QJWUcOx3D4kWhHjgGRLcGhckk4/H96BBoCwzvTbAFnzNfI1zD+k lcVDu9jnaaK0xBb2NFhsCstB0vxSH9BOH4KKv4TmAId4a6JdwmvJjdrdOJBaai/k /lrYQ6VMM3PGf/jl0wTFV8XpK8ObCMePsvY9Z2z4NYBrob9hp3YL18hk8wVTWRMM i2kwALITcE3TA8syHTpa7xyECf+sj9E784IKAvEY2zJaaKRwG8CF08fvhNV4CQOa LmyMJ+k94ZdUYmt3h9SAt4805g7ssEQiS2lBNiuDrWaosI4Y5SEcMmiIEXizgMW7 bR1FmQXgHQ+NyObNG9Ixao8vtsLIKV2nGG/BYJl6Ae62sDiuvQLOa8ddENFGvqjv NpJo6G8iqzieQFZoCkFsb2hyDpfa15mguDWGnkM7qERcNo1G5doC49+3ZyGlRR3E CVm2eNd7YeHbTMnX6XWpo6URiDswDCKBX1eGmA09qpqEVi+cDGPMF0W15BKZZysa JEw7lCq6d6TpC6CTtG4pI6sfglr+4NgjtQuZp9VOLgPuUfF3/qo=
=GEkz
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: you cannot sedate... all the things you hate (1:229/2)

From: g.branden.robinson@gmail.com

Hi Adrian,

At 2025-04-21T01:34:56+0300, Adrian Bunk wrote:

On Sun, Apr 20, 2025 at 04:46:52PM -0500, G. Branden Robinson wrote:

Factual statements about one's run-time dependencies should be as
decoupled from the details of the set of "Essential" packages as
possible. One reason is that the identities of the people making
these decisions are disjoint. Often a package maintainer lacks this
power; except for dependencies they introduce through operation of
their maintainer scripts (or Debian add-ons), such run-time
dependencies are beyond their control.
...

While this might sound good in theory, in practice it would be
horrible.

I'm not convinced...yet.

As an example, libc6 has a preinst script that calls dpkg, sed,
grep and rm.

A maintainer script that calls `dpkg`, I think, violates the Principle
of Least Astonishment.[1]

Making these dependencies explicit would be something like
Pre-Depends: sh, dpkg, sed, grep, coreutils

I would expect that such Pre-Depends cycles between essential packages
and libc6 will result in broken systems during upgrades.

My impression from watching countless upgrades over the years is that,
when any Essential package is available for upgrade, upgrades performed
by apt launch a full, discrete run of dpkg for each.[3] So the only way
any such cycles could exist is if the Pre-Depends were versioned and the versionings themselves produced the cycle, in which case there likely
_really is_ a breakage problem: one package or the other needs to delay
its aggressive employment of a newly available feature.

And then there's the normal time-waste like "the package ships
a bash-completion file that uses awk, grep, sed and sort - that's dependencies on four essential packages".

Hmm. In my opinion bash-completion scripts should produce "Recommends"-strength dependencies at most; not everybody's going to be
using bash as their interactive shell, and completion matters only for interactive shells.

That said, I haven't looked but would not be surprised if
bash-completion scripts make extremely aggressive assumptions about
what's available on the system. That is, they seldom if ever look
before the leap, and happily assume the presence of plenty of things
that _aren't_ in Essential, or even "Priority: required" packages.

But that's just a hunch. I could be wrong.

Regards,
Branden

[1] ...but it happens more often than it could because a lot of
maintainer scripts need `dpkg --compare-versions`--functionality
that ideally would be put someplace that sees less vigorous
development, like the debianutils package. Debian's version
comparison algorithm is pretty stable and has served us well for 30
years, with only one change in that interval I'm aware of, that
being the logic to support `~` for "pre-release" semantics. This
algorithm has overcome significant external indifference and/or
hostility to Debian and been embraced elsewhere.[2]

In general, the package manager does not need to be, and should not
be, re-entrant.

[2] https://git.savannah.gnu.org/cgit/gnulib.git/tree/lib/filevercmp.c#n81

[3] ...by which I mean all stages of ��6.6 and 6.7 of the Debian Policy
Manual.

https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-unpack-phase-of-installation-or-upgrade
https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#details-of-configuration

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEh3PWHWjjDgcrENwa0Z6cfXEmbc4FAmgFiiUACgkQ0Z6cfXEm bc6sHA/9E4qtjL7kgQ1JX8X9chubvz+Ea+Mj1eay+F8+mX4TneKJgK62UHOOG13Z 6K7vCeIFhQQAQpzOVkvcDKXJdmEqMtoElTK/opw5/0cPfgc+KuOjki5+KiBzx1L6 euWZ9p1+rRH/9g2hE0wdROj5tGJC0S9+EBBzbVgjX3+52gSpZwDibkyFgBXPNakS kB/ULb9cNfDOi+4PYIk87NxTgkaSqPjZjoZrtvKPjWA0ZxzeosugITLDGSD1uRj4 TNtRp4iojRIEAbyS5QYxdGVg7kwLXUGqOSAM+fUktNPt1dvMG4SUnDwelOQShLKc EqnZPTRUSeWyVIEpnnx5mQj366YdKk8iLMCxPTxTXapfLyFFyNqAcxKFTbkdWxHC gBooWRnBkFas3UP5fqdQnWyFlE0+626FQ2oVIvrhvCWLsCbq7DpyT0s+XmlberoP l0NJiIWgWHeMsj2oqNkJRdly101HtUSL6gRtjolNOhYkFD5kApTioSsDa6ziLrhf yGHqyG0qb/9f/t+mF0fOL1qG9L07MGSbGCdvjY8Wio7jjRrCBB/be9qyx/LO+wUa ReMDhSrSbj664pu9MK7J77JhuNuyVwOmNUw8mmBFiF0QTrl1OzW1VM4jhLyzi0q1 A3T6BZJFFgpLyvzSbiTr8uudcbXo5tzcraDIbAsp98uzro093l0=
=7nqs
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: you cannot sedate... all the things you hate (1:229/2)

From: g.branden.robinson@gmail.com

Hi Josh (it's been a long time since XFree86 packaging days!),

At 2025-04-21T01:42:52+0100, Josh Triplett wrote:

G. Branden Robinson wrote:

"You must not tag any packages essential before this has been
discussed on the debian-devel mailing list and a consensus about
doing that has been reached." -- Debian Policy Manual, �3.8[2]

That implies to me that a package can be taken _out_ of the
essential set unilaterally by the package maintainer(s) of a package
that's in it, but because of the status quo of being able to depend
on an essential package without declaring that fact, in practice
that probably wouldn't work well, and we should update the Policy
Manual to require discussion of the dropping of such a "tag" as
well.

I think that's a bug in Policy as written, rather than a bug in
practice. Historical practice has definitely been to discuss such
removals (extensively).

We should have a well-defined process for this, that includes
discussion transition plans (involving the introduction of Depends as
needed first), and similar.

I agree that the Policy Manual should be fixed to document the
deliberative process we actually use in this case as well as the one it
already contemplates.

But I want to re�mphasize that explicit Dependency declarations would
make it easier to see and reason about such things.

Regards,
Branden

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEh3PWHWjjDgcrENwa0Z6cfXEmbc4FAmgFme4ACgkQ0Z6cfXEm bc657xAAmvXZkF2/ns2/pocQgM6PsxtGW7VgqEVtsgnqaTBm6Sw4cCjvGt7VT4n3 zp0mn1+6e05DynsC/jssliRp0mChpsOHfIwKcOdmGSSIPPA53pwpGvrq8jznaZpV kxRAnVWYOMKH+ijRnLisb/ISXK+fa4XMNalBBY9jeuV5+6/D9ennfgycfJrre36g rdvFnO3Eo4WNFAJkVm1FtAaglNXt1WeDFC3FVRmpuVPL6V0yvI20NXmbyjEqeejI MnuxMsZW2IhdAKVbVXD2I1elccYXK+OGOXwBRVL1cC92PeyV6JtlzXAkb3wuZFjq cezUi7QMNJSuuaknbZFqQjL1Wt1wdI4eVUJfoNZAeTIboBQE70RVwX0piG1q65K2 il4roe1Ms6JWW+auV2kdkPS0ukHjesIUNy/q/frHMvAfHFj1RI2Pm0HIhMIunQMa q19dxPeItBVIdzjHdXjjEk3AfLHAGHSzR8xDrfJpEyeOhTzFolWA+sesNoGkNM82 QtqxVDVTHV9Pn6LJgTW01E4nJ8tyIvWZI8CMJV8k0i3wVBRd9tnyGltB4NcDB6Z5 PdhC/HUNpjoZcdx4QKgfGd+445ccehiRVHO3aTfaSOr2CY/bATfjDIR8exyPgY6A NMM/IpoOM34yenXHBDL1M9tGHgWKR5hVrVq31fyA0S937MQmg58=
=Rg4Y
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: you cannot sedate... all the things you hate (1:229/2)