According to the Debian Policy Manual, section 7.2, the Recommends field in Debian packages "declares a strong, but not absolute, dependency. The Recommends field should list packages that would be found together with this one in all but unusual installations." While this is a very useful definition, the actual way in which Recommends are used seems to differ substantially from this.
If this was just a diffoscope problem, it would be easy to just file a bug asking that most of these packages be demoted to Suggests, but this is a much more pervasive issue, as evidenced by the fact that the live-build manual has special instructions for how to disable the installation of *all* recommended packages when building a live image[1]. I have built live images that ended up with all sorts of weird packages installed on them, which issue was resolved by disabling the installation of recommended packages.
Furthermore, the current (ab)use of Recommends in Debian packages illustrates something important - there is a real need for packagers to specify packages that should automatically be installed alongside another package, but that aren't necessarily strong dependencies. Using diffoscope again as an example, it's reasonable that the diffoscope maintainers want *all* of diffoscope's functionality to "just work" out of the box, even if that means installing over three and a half gigabytes of packages to do it.[2] This may not be policy-compliant, but demoting these packages to the "Suggests" field doesn't feel right. Should a user who just wants to compare things have to figure out the right combo of packages to make diffoscope work for their particular use case?[3] There's also the question of logistics - going through and "fixing" all of the packages with overkill Recommends could be a massive undertaking, and it's more than likely that there will be some packagers who won't be thrilled with the changes.
Weak-Depends would basically just be a stronger "Recommends".
Some of the advantages of this solution include:
* It requires comparatively few changes to initially implement.
All
existing packages in the Debian repository will be compliant with a
Debian Policy Manual update that adds Weak-Depends, without changes.
Packagers can start using Weak-Depends if they want to or if a bug
report requests that they do. Some of the packages that would need to
change to implement this would be dpkg, apt, possibly the libraries
they depend on, and live-build.
There's probably others here, but
nonetheless it wouldn't require a massive overhaul of the whole
archive to make it start working, the way a mass-"demote to Suggests"
operation would.
* Change the definition of the "Recommends" field to match the way
the field is oftentimes used in the Debian archive.
* Suggest that all active package maintainers in Debian review the
packages they maintain and see if they feel there are some packages
that should be promoted from "Recommends" to "Weak-Depends".
* (Potentially?) Scan the Debian archive and see if there are
dependencies that should be promoted from "Recommends" to
"Weak-Depends". This would probably be something that interested
Debian Developers and other volunteers could slowly chip away at, as
they had time and the desire to do so.
Have one package diffoscope and one package diffoscope-full and you could even
have a package diffoscope-minimal and there you have user-selectable granularity.
* Add a "Weak-Depends" field to the list of binary dependency control
fields in the Debian Policy Manual section 7.2, with a definition
very similar to the existing definition for "Recommends".
* Change the definition of the "Recommends" field to match the way
the field is oftentimes used in the Debian archive.
* Johannes Schauer Marin Rodrigues <josch@debian.org> [2024-11-06 02:28]: [...]
Have one package diffoscope and one package diffoscope-full and you could even
have a package diffoscope-minimal and there you have user-selectable granularity.
We already have two diffoscope packages for exactly this reason (I work on diffoscope and only have -minimal installed myself):
$ apt-cache show diffoscope/sid | grep -A1 full
This is a dependency package that recommends the full set of external tools,
to support as many type of files as possible.
$ apt-cache show diffoscope-minimal/sid | grep -A2 partial
This -minimal package only recommends a partial set of the supported 3rd party
tools needed to produce file-format-specific comparisons, excluding those that
are considered too large or niche for general use.
Afaict, the problem is that we have 3 options to pick from, and it's hard for people to decide which is the "right one".
I do not see how adding a 4th option will make this decision any easier (esp. since that new option is so similar to an already existing one).
[Added diffoscope@lists.reproducible-builds.org to Cc]
* Fay Stegerman <flx@obfusk.net> [2024-11-06 17:43]:
* Johannes Schauer Marin Rodrigues <josch@debian.org> [2024-11-06 02:28]:
[...]
Have one package diffoscope and one package diffoscope-full and you could even
have a package diffoscope-minimal and there you have user-selectable
granularity.
We already have two diffoscope packages for exactly this reason (I work on >> diffoscope and only have -minimal installed myself):
$ apt-cache show diffoscope/sid | grep -A1 full
This is a dependency package that recommends the full set of external tools,
to support as many type of files as possible.
$ apt-cache show diffoscope-minimal/sid | grep -A2 partial
This -minimal package only recommends a partial set of the supported 3rd party
tools needed to produce file-format-specific comparisons, excluding those that
are considered too large or niche for general use.
IMO in the case of diffoscope it could make sense to have multiple tools metapackages, like a diffoscope-tools-android etc., to both make it easier to avoid those dependencies if you know you don't need them, but also to easily install the dependencies for the file formats you do work with.
For instance, gwenview currently
recommends kamera. gwenview is an image viewer, kamera is a tool for
working with digital cameras. Now it is true that kamera enhances
gwenview's functionality by allowing it to see pictures on a digital
camera that is plugged into the system, but by no means is gwenview
useless or even substantially degraded from a functionality standpoint
when kamera is missing.
(Side note, I wonder if there's a way to implement Weak-Depends that *doesn't* require modifying all of the tons of packages Johannes
mentioned. Maybe some way of annotating packages in Recommends as
"important" would permit a distinction to be made here in such a way
that most tools wouldn't need changes? Those that were updated would be
able to understand the distinction, those that weren't would continue
to treat Recommends the same way they do today. No idea if that's
feasible, just something that went through my head.)
And this brings us back to the original idea of creating a Weak-Depends field. From my viewpoint, policy states that Recommends is for
declaring a strong (heavy emphasis on "strong" here), but not absolute, dependency.
At this point, we have two options. We can either explicitly remove all
of the extra packages that get installed, or we can skip installing recommends at all. Both of these come with their own severe
disadvantages. If we manually uninstall everything we don't need, that
means we have to maintain a list of packages that get incorrectly
installed, and keep it up-to-date as package dependencies in the
archive change. On top of that, if a package happens to not uninstall cleanly, or it otherwise causes problems when installed or uninstalled,
we'd have to figure out how to manually fix that, or avoid letting that particular package ever get installed in the first place via held
packages or a similar mechanism. This is a rather large amount of work,
and it's not easy to maintain.
The other option is to skip installing recommends. The reason that
doesn't work well is because if we do that, we can't use recommends in
our metapackages at all - anything we specify as a recommends in the metapackages won't end up installed. To work around that, we have to
use depends for everything, and to do *that*, we have to basically reimplement the whole darn recommended packages mechanism in
metapackages. This results in the metapackages setup that we have
today, which can be seen at [1]. There's some "dependencies" packages,
some "recommends" packages, a complex network between the packages to
make things work right, and a smattering of dummy dependencies to allow
users to override certain depends. Our current metapackages scheme has
some rather inconsistent naming and could use a lot of improvement, but
even under ideal conditions (like what I've proposed for Kicksecure in
[2]), this is a lot of ugly hacks that reimplement existing apt
features without using those features just to get around problematic recommends in Debian's packages. And this is likely *easier* to
maintain than manually uninstalling everything we don't want.
So, that's the issue we're having, and the solutions I'm pursuing
(using Recommends differently, adding a Weak-Depends field, or
implementing something like Python's extras like Colin was mentioning)
are things I think would fix the problem for us and others. I'm
definitely open to suggestions for other ways we could avoid these
problems though.
Again, this isn't a problem limited to a derivative distribution. I
respect that your opinion of how Recommends should work differs from
mine. That doesn't change the policy though, and it doesn't change that neglecting or changing the policy's rules in this area will cause and
is causing problems to some of Debian's users. Ultimately I don't care exactly how those problems are solved, I just want to solve them.
In some ways I think what we're missing is really a way to do the
equivalent of "extras" in Python packages (https://peps.python.org/pep-0508/#extras): effectively groups of
additional dependencies to enable some kind of feature that you can opt
into if you need that feature, rather than having to pick from an
just to get around problematic recommends in Debian's packages.
Then again,
given that policy is clear about how Recommends ought to be used and
it's pretty clear that there are packages that just don't use it right,
Some packages are clearly in Depends, Recommends, or Suggests. Others might be right
on the line between two of the categories. In these cases, a maintainer has to make a
judgement call. If a user thinks they have got it wrong, they are welcome to submit a bug
report explaining why they think it should be in the other category.
The distinction between Depends, Recommends or Suggests is not a
true/false thing; this is not a question of mathematics or science
but always a judgement call. Adding another category won't solve
anything IMO but only extend the sometimes blurry area.
Clarifying policy may or may not help, in the end there will always
be uncertainties, clarifications, bug reports, and the common effort
to find the best solution for most users.
On Wednesday, November 6, 2024 10:41:46 PM MST Aaron Rainbolt wrote:
Again, this isn't a problem limited to a derivative distribution. I
respect that your opinion of how Recommends should work differs from
mine. That doesn't change the policy though, and it doesn't change that neglecting or changing the policy's rules in this area will cause and
is causing problems to some of Debian's users. Ultimately I don't care exactly how those problems are solved, I just want to solve them.
Let me try to explain what I see as the core of the problem.
First, some background on the three existing categories.
1. Depends: These are the packages necessary to install and run the basic functionality of
the package.
2. Recommends: These are the packages required to enable all the features of the
package.
3. Suggests: These are packages that enhance the functionality of the package.
With all this in mind, I'd like to call some attention to a feature
request made by Patrick Schleizer some time ago, whom I've copied on
this email. The feature request suggests the addition of a new field to Debian's binary dependency relationship fields, "Weak-Depends".[4] In Patrick's own words, Weak-Depends "[d]eclares a weak dependency. Most
users of this package may benefit from installing packages listed in
this field, but can have reasonable functionality without them." The
exact way in which this would be implemented is that Weak-Depends
packages would get installed when using `apt install
--no-install-recommends package`, but any package listed there could be removed without removing the package which referenced the Weak-Depends.
[4]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=942303
Sysop: | Keyop |
---|---|
Location: | Huddersfield, West Yorkshire, UK |
Users: | 546 |
Nodes: | 16 (0 / 16) |
Uptime: | 161:46:32 |
Calls: | 10,385 |
Calls today: | 2 |
Files: | 14,057 |
Messages: | 6,416,500 |