"Serafeim (Serafi) Zanikolas" <sez@debian.org> writes:
hi lintian and piuparts folks! relatively new maintainer of adequate(1) here.
No-one replied so i thought i'd have a go, but i have no role in any of
this, just a user who has also tried to understand these tools
it seems to me that it'd be useful to write down some criteria to use as
guidance on how to decide where new checks should be implemented, to avoid >> duplication.
source-package checks clearly belong in lintian. binary-package checks are >> trickier:
- lintian is great to check requirements around mechanics (e.g. that a certain
helper is used appropriately, rather than using ad-hoc code)
i'd think:
lintian is static analysis, it doesnt install the deb, just looks at
its contents vs policy
piuparts is mostly about upgrades and removals -- and interactions
with other debs
this leaves adequate as "things lintian cant do as it would need the
deb to be installed" but which dont relate to upgrades/removals
(perhaps adequate and the "i" bit of piuparts could be merged, but
maybe the difference is that adequate only looks at once package? and
not sure anyone maintains piuparts any more?)
One of the strengths of lintian and piuparts is a lot of people run them
each time they build a package. Adequate needing the package to be installed makes it harder to integrate in that workflow.
Maybe if it was a feature in sbuild it would help?
Hello,
On Tue, 29 Oct 2024, Louis-Philippe Véronneau wrote:
One of the strengths of lintian and piuparts is a lot of people run them each time they build a package. Adequate needing the package to be installed
makes it harder to integrate in that workflow.
Maybe if it was a feature in sbuild it would help?
Or maybe it would help if adequate could be run in a debusine task. We already have tasks to run lintian and piuparts but we are lacking one for adequate.
On 2024-10-27 7 h 16 p.m., Richard Lewis wrote:
"Serafeim (Serafi) Zanikolas" <sez@debian.org> writes:
hi lintian and piuparts folks! relatively new maintainer of adequate(1) here.
it seems to me that it'd be useful to write down some criteria to use as >> guidance on how to decide where new checks should be implemented, to avoid >> duplication.
source-package checks clearly belong in lintian. binary-package checks are >> trickier:
- lintian is great to check requirements around mechanics (e.g. that a certain
helper is used appropriately, rather than using ad-hoc code)
i'd think:
lintian is static analysis, it doesnt install the deb, just looks at
its contents vs policy
piuparts is mostly about upgrades and removals -- and interactions
with other debs
this leaves adequate as "things lintian cant do as it would need the
deb to be installed" but which dont relate to upgrades/removals
(perhaps adequate and the "i" bit of piuparts could be merged, but
maybe the difference is that adequate only looks at once package? and
not sure anyone maintains piuparts any more?)
Hi,
Thanks for reaching out.
I can't really speak for piuparts, but I agree with what Serafeim wrote
for the lintian part.
One of the strengths of lintian and piuparts is a lot of people run them each time they build a package. Adequate needing the package to be
installed makes it harder to integrate in that workflow.
Maybe if it was a feature in sbuild it would help?
Or maybe it would help if adequate could be run in a debusine task. We already have tasks to run lintian and piuparts but we are lacking one for adequate.
thanks for the pointer to debusine. however it seems to be meant for expensive
tasks, and adequate is not expensive at all. in my mind, the right context is autopkgtest (details in another response of mine in the same thread)
Hello,
On Fri, 01 Nov 2024, Serafeim (Serafi) Zanikolas wrote:
Or maybe it would help if adequate could be run in a debusine task. We already have tasks to run lintian and piuparts but we are lacking one for adequate.
thanks for the pointer to debusine. however it seems to be meant for expensive
tasks, and adequate is not expensive at all. in my mind, the right context is
autopkgtest (details in another response of mine in the same thread)
It's not restricted to expensive tasks. And expensive depends a bit on
your point of view. Setting up a clean chroot where to install the package and then run adequate can be considered as expensive by some developers
that would not have the proper environment ready to use.
Which is certainly one of the reason why piuparts is also not widely used
by maintainers prior to upload. And it's precisely that kind of gap in the test coverage that debusine wants to fill.
I understand that autopkgtest's test infrastructure is ideal to run
adequate but I question the usefulness of adding manual tests to each
source package just to run adequate. If anything, it should hook with something like autodep8 (i.e. auto-generated tests).
On Fri, 01 Nov 2024, Serafeim (Serafi) Zanikolas wrote:
Or maybe it would help if adequate could be run in a debusine task. We already have tasks to run lintian and piuparts but we are lacking one for adequate.
thanks for the pointer to debusine. however it seems to be meant for expensive
tasks, and adequate is not expensive at all. in my mind, the right context is
autopkgtest (details in another response of mine in the same thread)
It's not restricted to expensive tasks. And expensive depends a bit on
your point of view. Setting up a clean chroot where to install the package and then run adequate can be considered as expensive by some developers
that would not have the proper environment ready to use.
Which is certainly one of the reason why piuparts is also not widely used
by maintainers prior to upload. And it's precisely that kind of gap in the test coverage that debusine wants to fill.
I understand that autopkgtest's test infrastructure is ideal to run
adequate but I question the usefulness of adding manual tests to each
source package just to run adequate. If anything, it should hook with something like autodep8 (i.e. auto-generated tests).
Why not create a Salsa CI job and add it to the default pipeline?
I think most people who use Salsa's CI use the default pipeline, so this seems like a very low friction way of getting people to run adequate
tests? Maintainers won't have to change anything on their side.
The default pipeline already has a job for autopkgtest (but it doesn't
seem to run adequate), lintian and piuparts, so it seems like an
excellent fit.
hi Diederik, thanks for your suggestions :)
On Tue Nov 5, 2024 at 2:50 PM CET, Diederik de Haas wrote:
[..]
Why not create a Salsa CI job and add it to the default pipeline?
I think most people who use Salsa's CI use the default pipeline, so this seems like a very low friction way of getting people to run adequate
tests? Maintainers won't have to change anything on their side.
The default pipeline already has a job for autopkgtest (but it doesn't
seem to run adequate), lintian and piuparts, so it seems like an
excellent fit.
that's a great idea! technically, piuparts can run adequate but it's disabled in
the piuparts job, so I'll give that a try first (we can always fall back to adding a dedicated adequate job if that causes any issues)
regardless, until DEP-18 really takes off (and I surely hope it will), I think
it'd still be useful to have an autodep8 test (not least because an autopkgtest
failure blocks migration to testing, while a broken CI does not)
tangential sidenote: a package with defined autopkgtests gets baked in unstable
for 2 days (rather than 5); it'd make sense to me that a package with configured
and passing CI status should also get that favorable treatment, and a failing CI
status should block migration to testing)
On Tue Nov 5, 2024 at 10:09 PM CET, Serafeim (Serafi) Zanikolas wrote:
On Tue Nov 5, 2024 at 2:50 PM CET, Diederik de Haas wrote:
[..]
that's a great idea! technically, piuparts can run adequate but it's disabled in
the piuparts job, so I'll give that a try first (we can always fall back to adding a dedicated adequate job if that causes any issues)
I just took a quick look at the piuparts Salsa CI job definition, but I didn't see anything related to adequate? https://salsa.debian.org/salsa-ci-team/pipeline/raw/master/salsa-ci.yml
But I'd prefer and recommend to make it a separate Salsa CI job and not
(try to) integrate it into another.
FWIW: not just wrt adequate, but with every (such) job
tangential sidenote: a package with defined autopkgtests gets baked in unstable
for 2 days (rather than 5); it'd make sense to me that a package with configured
and passing CI status should also get that favorable treatment, and a failing CI
status should block migration to testing)
I think CI in general is *very* beneficial, but I'm aware and understand
that not everyone is a fan of it or Salsa in general.
There's a lot more to it, but it's probably best to leave that out of
this discussion.
I'm fine with adding a dedicated adequate job, if you feel strongly about this.
On Tuesday, November 5, 2024 3:41:59 PM MST Serafeim (Serafi) Zanikolas wrote:
I'm fine with adding a dedicated adequate job, if you feel strongly about this.
In addition to adding an adequate Salsa CI job, you might also consider updating the .sbuildrc example to contain the necessary information to also run adequate (assuming it isn’t too complicated).
https://wiki.debian.org/sbuild#Setup
On Tue Nov 12, 2024 at 4:12 AM CET, Soren Stoutner wrote:also
In addition to adding an adequate Salsa CI job, you might also consider updating the .sbuildrc example to contain the necessary information to
cutrun adequate (assuming it isn’t too complicated).
https://wiki.debian.org/sbuild#Setup
that makes sense. here's the .sbuildrc snippet I came up with:
$external_commands = {
"chroot-cleanup-commands" => [
[ 'apt install -y --install-recommends adequate' ],
[ 'perl -e \'open(FH, ">", "/var/lib/adequate/pending") or die $!; while (<>) { my @c=split(" ", $_); print FH "$c[1]\n" if $c[0] eq "Package:" }; close(FH);\' %SBUILD_PKGBUILD_DIR/debian/control' ], [ 'adequate', '--fail', '--pending' ],
]
};
before I update the wiki, any suggestions on how to simplify? it'd be much simpler if:
- sbuild external commands would allow for subshells:
adequate --fail $(grep ^Binary: %SBUILD_PKGBUILD_DIR/debian/control' |
-d: -f2) - sbuild would populate an escape variable with the list of built binary packages:
adequate --fail %SBUILD_BINARY_PKG_NAMES
thanks,
serafi
The sbuild page has recently been edited to handle the new unshare backend. The current version of the adequate code on the page is as follows:
## run adequate(1) and fail upon policy errors. Depends on adequate >= 0.17.2.
$external_commands = {
"chroot-cleanup-commands" => [
[ 'apt install -y --install-recommends adequate' ],
[ ‘/usr/share/doc/adequate/examples/sbuild-hook’, ’%SBUILD_PKGBUILD_DIR’],
]
};
This code produces an error for me:
I: Finished running 'apt install -y --install-recommends adequate'.
/usr/share/doc/adequate/examples/sbuild-hook /<<PKGBUILDDIR>> ---------------------------------------------------------------------------------------------------
2024/11/25 22:37:58 "dpkg-query -Wf ${binary:Package} ${Package};${Status};$ {Provides}\n -- feather-wallet" failed: exit status 1
it seems to me that it'd be useful to write down some criteria to use as guidance on how to decide where new checks should be implemented, to avoid duplication.
- checks hermetic to the installed state of a package (e.g. #1071061 "lintian:
Should warn if a package ships /usr/lib/python*/dist-packages/__init__.py")
could live in either of lintian/adequate; I can't think of strong reasons either way; what do you think?
#658210 lintian: Check that update-alternatives --install/--remove are always used in pairs
overlaps with:
#909342 adequate: warn about broken alternatives
#455740 lintian: Please use desktop-file-validate
overlaps with:
#854208 adequate: check that Exec/Icon references in .desktop files resolve to existing files
#524357 lintian: Ensure that packages providing x-window-manager register the alternative
this check is already implemented by adequate (also for x-terminal-emulator), so
#524357 can just be closed?
[adequate] overlap with piuparts:
adequate has a broken-symlink tag, but piuparts detects broken symlinks on its
own so disables broken-symlink checks on adequate invocations. should adequate
remove broken-symlink, or piuparts drop its implementation of broken-symlink in
favor of adequate's? arguably, there's some value in adequate keeping the functionality so that one could catch issues early, by running adequate as an autopkgtest.
ditto for adequate obsolete-conffile -- piuparts disables it in favor of its own
logic. it'd seem to me that the logic should stay in piuparts, given that install/uninstall/upgrade is core to what piuparts tests. so there's a case for
adequate's obsolete-conffile to, if not to be dropped, to at least not run by default (since, e.g. it's useless in an autopkgtest context)
On Sun, 2024-10-27 at 00:43 +0200, Serafeim (Serafi) Zanikolas wrote:
it seems to me that it'd be useful to write down some criteria to use as guidance on how to decide where new checks should be implemented, to avoid duplication.
Checks that can only happen after install such as cross-package checks
should go to adequate and checks that can be done statically for a
single binary package or all binary packages from the same source
should go to lintian.
- checks hermetic to the installed state of a package (e.g. #1071061 "lintian:
Should warn if a package ships /usr/lib/python*/dist-packages/__init__.py")
could live in either of lintian/adequate; I can't think of strong reasons
either way; what do you think?
A check in lintian seems appropriate. The more general problem of
detecting whole-archive file conflicts cannot be done in piuparts
or adequate, you need something like dumat or dedup.d.n for that.
Even that isn't going to detect conflicts created after install.
#658210 lintian: Check that update-alternatives --install/--remove are always used in pairs
overlaps with:
#909342 adequate: warn about broken alternatives
I would suggest both lintian and adequate, since lintian does not have
a shell script parser for interpreting maintainer scripts and even if
it did probably couldn't detect all situations that result in broken alternatives, and can't detect breakage on a system caused by a package
from a third-party, the sysadmin or a rogue malicious program or user.
#455740 lintian: Please use desktop-file-validate
overlaps with:
#854208 adequate: check that Exec/Icon references in .desktop files resolve to existing files
Both should be implemented. Adequate since Exec/Icon references
are potentially cross-package, but aren't always.
#524357 lintian: Ensure that packages providing x-window-manager register the alternative
this check is already implemented by adequate (also for x-terminal-emulator), so
#524357 can just be closed?
IIRC piuparts.d.o doesn't expose adequate results to tracker.d.o, while lintian does, and lots of people run lintian locally but not piuparts
or adequate. So the audience for lintian is much higher than the
audience for adequate. So having this in lintian would be useful too.
[adequate] overlap with piuparts:
adequate can be used in more locations than just in piuparts,
so please keep that mind when deciding on where to add checks
and deciding to remove checks entirely.
adequate has a broken-symlink tag, but piuparts detects broken symlinks on its
own so disables broken-symlink checks on adequate invocations. should adequate
remove broken-symlink, or piuparts drop its implementation of broken-symlink in
favor of adequate's? arguably, there's some value in adequate keeping the functionality so that one could catch issues early, by running adequate as an
autopkgtest.
Probably adequate is the logical place for this test, but adequate
doesn't build/run on ports architectures since it moved to golang,
so piuparts should probably keep its tests on those arches until
adequate moves to a more portable language or golang gets ported.
Since piuparts already runs adequate under all the package scenarios
that autopkgtest runs tests for, there is no need for autopkgtest to
run adequate. If anything, piuparts should run autopkgtests after it
installs the packages and runs adequate. The current situation with autopkgtest seems like a bunch of layering violations to me.
ditto for adequate obsolete-conffile -- piuparts disables it in favor of its own
logic. it'd seem to me that the logic should stay in piuparts, given that install/uninstall/upgrade is core to what piuparts tests. so there's a case for
adequate's obsolete-conffile to, if not to be dropped, to at least not run by
default (since, e.g. it's useless in an autopkgtest context)
Several folks use this specific test on our systems (outside piuparts)
for filing bug reports about obsolete conffiles, so please retain it.
Probably adequate is the logical place for this test, but adequatethat's because unsupported ports architectures have not caught up to go 1.21, which was released ~1.5 year ago. I'd claim that that says more about the viability of those ports, than the suitability of go for Debian tooling. if you
doesn't build/run on ports architectures since it moved to golang,
so piuparts should probably keep its tests on those arches until
adequate moves to a more portable language or golang gets ported.
feel strongly otherwise, I'd be happy to continue this discussion with a wider
audience at <D4CWM2UJBXDS.339WS39VNMVYO@debian.org> rather than reply here
piuparts relies on humans to file bugs. autopkgtest on the other hand blocks package migration to testing (that is, when one does not run autopkgtst beforeproven to be viable long-term.
upload). it's nice that some humans (including yourself) file bugs, but that doesn't seem viable long-term.
90% of the piuparts bugs in the last 2 decades have been filed by 3 people, this might not seem viable long-term to you, but in practice it has been
On Tue, Dec 10, 2024 at 09:38:57PM +0100, Serafeim (Serafi) Zanikolas wrote:
Probably adequate is the logical place for this test, but adequate doesn't build/run on ports architectures since it moved to golang,that's because unsupported ports architectures have not caught up to go 1.21,
so piuparts should probably keep its tests on those arches until
adequate moves to a more portable language or golang gets ported.
which was released ~1.5 year ago. I'd claim that that says more about the viability of those ports, than the suitability of go for Debian tooling. if you
feel strongly otherwise, I'd be happy to continue this discussion with a wider
audience at <D4CWM2UJBXDS.339WS39VNMVYO@debian.org> rather than reply here
I dont feel strongly about this, but I'd like to point out that I
disagree. IMO it was wrong to rewrite adequate (as any central QA tool
for Debian) in a language which is not available *everywhere*.
piuparts relies on humans to file bugs. autopkgtest on the other hand blocks
package migration to testing (that is, when one does not run autopkgtst before
upload). it's nice that some humans (including yourself) file bugs, but that
doesn't seem viable long-term.
90% of the piuparts bugs in the last 2 decades have been filed by 3 people, this might not seem viable long-term to you, but in practice it has been proven to be viable long-term.
also: piuparts failures block migrations without anyone having to file bugs, just like autopkgtest. (oh, and for autopkgtests related bugs, there are also very few humans filing them.)
IIRC piuparts.d.o doesn't expose adequate results to tracker.d.o,is true (as in, migration is blocked without a hint as to why)
my bad. this fails because feather-wallet is not installed and presumably you've not enabled autopkgtest in sbuild (which would have the side effect of installing it). I didn't catch this because, I've only tested the scriptwith
adequate itself (doh!). I'll upload a new adequate release to make sbuild-hook install all binary packages of %SBUILD_PKGBUILD_DIR/debian/control
meanwhile, I've rolled back the edit for now.
Serafeim,
On Tuesday, November 26, 2024 3:08:26 PM MST Serafeim (Serafi) Zanikolas wrote:
my bad. this fails because feather-wallet is not installed and presumably you've not enabled autopkgtest in sbuild (which would have the side effect ofwith
installing it). I didn't catch this because, I've only tested the script
adequate itself (doh!). I'll upload a new adequate release to make sbuild-hook install all binary packages of %SBUILD_PKGBUILD_DIR/debian/control
meanwhile, I've rolled back the edit for now.
Are things in place to be able to add adequate back into the example sbuild config.pl?
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 491 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 90:25:31 |
| Calls: | 9,679 |
| Files: | 13,722 |
| Messages: | 6,173,898 |