XPost: linux.debian.maint.python, linux.debian.maint.dpkg

Picking up a thread that started on debian-python@lists.debian.org: https://lists.debian.org/msgid-search/14198883.O9o76ZdvQC@galatea

Upstreams that care about supply chain security have been building
mechanisms to authenticate their releases, beyond PGP signatures.
For example, Python started providing sigstore signatures a couple of
years ago, and is now talking about the idea of dropping PGP signatures. https://discuss.python.org/t/pre-pep-discussion-stop-providing-gpg-signatures-for-cpython-artifacts/65058

We currently support including PGP signatures in source packages, and
verifying them in uscan.

Should we expand this to include some of these new mechanisms?
Things brought up in the debian-python thread include:
1. sigstore https://docs.sigstore.dev/
2. ssh signatures
3. signify https://man.openbsd.org/signify.1

There is a general trend towards getting upstream sources from Git
rather than tarballs in Debian, but we're a long way from moving across completely, or even finding consensus to do so.
These signature mechanisms can generally be applied to git commits as
well as tarballs.

I see supporting them in Debian requiring:
1. Decisions on which schemes to support. I'd assume we support all that
are available in Debian and have some significant use.
Some, like sigstore, can be used in multiple modes, we'd have to make
some selections.
2. Support in uscan to verify at download/checkout time.
2.1: Syntax in watch files to locate signature files.
2.2: Path in source packages / watch files to declare trusted signers.
2.3: Syntax in watch files for signature verification in git mode.
3. Support in dpkg-source to include detached signatures in source
packages.
3.1: Declare expected formats and filename extensions.
4. Support in the archive? (Is anything necessary?)

Is this something people are interested in pursuing?

For sigstore, we probably need to package the Python / go client in
Debian. We have rekor-cli for low-level verification, but not tools for verifying bundles like the ones Python provides.

Stefano

--
Stefano Rivera
http://tumbleweed.org.za/
+1 415 683 3272

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

XPost: linux.debian.maint.python, linux.debian.maint.dpkg

* Stefano Rivera: " Alternative signature mechanisms for upstream source
verification" (Fri, 4 Oct 2024 18:21:01 +0000):

[...]

Should we expand this to include some of these new mechanisms?
Things brought up in the debian-python thread include:
1. sigstore https://docs.sigstore.dev/
2. ssh signatures
3. signify https://man.openbsd.org/signify.1

JFTR: Tryton moved from pgp signatures to signify, so of course I would appreciate if this format could be handled as well.



--

Mathias Behrle
PGP/GnuPG key availabable from any keyserver, ID: 0xD6D09BE48405BBF6
AC29 7E5C 46B9 D0B6 1C71 7681 D6D0 9BE4 8405 BBF6

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

XPost: linux.debian.maint.dpkg, linux.debian.maint.python

Hi!

On Fri, 2024-10-04 at 18:21:01 +0000, Stefano Rivera wrote:

Picking up a thread that started on debian-python@lists.debian.org: https://lists.debian.org/msgid-search/14198883.O9o76ZdvQC@galatea

Upstreams that care about supply chain security have been building
mechanisms to authenticate their releases, beyond PGP signatures.
For example, Python started providing sigstore signatures a couple of
years ago, and is now talking about the idea of dropping PGP signatures. https://discuss.python.org/t/pre-pep-discussion-stop-providing-gpg-signatures-for-cpython-artifacts/65058

Hmm, I find this usual conflation (in the upstream discussion) of GPG
(or GnuPG) as if it was OpenPGP itself rather problematic, because the
OpenPGP ecosystem is way richer than that, and as such way more active,
and there has been and there is lots of work going on to implement and
provide more usable, intuitive, secure and modern interfaces and code,
be those CLI or library-first ones (not just wrapping a CLI from a
library), including implementation neutral interfaces like the Stateless OpenPGP CLI (SOP). The OpenPGP work group also just got a new RFC9580
published that obsoletes RFC4880. Of course the schism between GnuPG and
the OpenPGP work group is rather problematic too, but I'd hope we can
manage to move into something like SOP backed by any of the many new implementations that provide it (see [S]), or barring that into any of
the native interfaces by implementations that provide easier and more
secure to use ones, all of which while eventually following the new RFC.

[S] https://gitlab.com/dkg/openpgp-stateless-cli/-/wikis/Stateless-OpenPGP-status

For an example of the activity that is going on in the OpenPGP ecosystem, here's a list of some of the non-GnuPG implementations already present
in Debian, by programming language:

* Rust:
- Sequoia-PGP
- rPGP
* Haskell:
- hOpenPGP
* Golang:
- GopenPGP
* Java:
- Bouncy Castle
- PGPainless
* Python:
- PGPy
* C:
- RNP

In addition, I'd strongly recommend checking SOP (https://datatracker.ietf.org/doc/draft-dkg-openpgp-stateless-cli/),
which should be nicer to integrate into test suites and tools, by way
of any of the currently available implementations in Debian, such as:

* sqop / sqopv (Rust)
* rsop / rsopv (Rust)
* pgpainless-cli (Java)
* gosop (Golang)
* sopv-gpgv (C + Python)

For the sopv subset we also now have a virtual package.

And finally, I'd also recommend taking a look at both the Sequoia-PGP
native interfaces (but note these are not yet stable, but should be
RSN AFAIUI!):

* sq / sqv / sq-wot / sq-keyring-linter

Where in addition to the usual commands to verify, sign, etc,
you have extremely useful stuff like:

$ sq inspect ...
$ sq cert ...
$ sq toolbox keyring ...
$ sq toolbox packet ...
$ sq network ...
$ sq ...

And its GnuPG compatibility drop-in replacements, for either:

* CLI:
- sequoia-chameleon-gnupg (gpg-sq / gpgv-sq)
- gpg-from-sq / gpgv-from-sq
* Thunderbird (if you use that):
- libsequoia-octopus-librnp

Oh, and you can already use either SOP or the Sequoia-PGP interfaces
with dpkg itself! See dpkg-buildpackage's --sign-backend.

We currently support including PGP signatures in source packages, and verifying them in uscan.

Should we expand this to include some of these new mechanisms?
Things brought up in the debian-python thread include:
1. sigstore https://docs.sigstore.dev/

Although I've heard of this before, I never really checked what is
the actual design behind it, and its implications. I'm not sure how
reliant on centralization this is, or on the apparent specific OIDC
providers currently in use, about offline operations and whether that
is a first class use, or if that implies limitations, etc. Even though
in the Python upstream thread it's mentioned that many upstreams are
moving into using it, it's not clear to me either what are the
long-term prospects of this either. I've not checked either what is
the format of the certificates and signatures for this, how detectable
they are, their size, etc.

From Python upstream and your comment below, I take the only
implementations are either Python or Golang, which seems problematic as something to have to pull into say a build-essential chroots by default.
(Not to mention that these are not even yet in Debian. :)

2. ssh signatures

AFAIK these are used via ssh-keygen. The signatures are pretty easy to
detect, as they are surrounded by «-----BEGIN SSH SIGNATURE-----» and «-----END SSH SIGNATURE-----» and are ASCII encoded. The certificates
and signatures can be few lines or many lines, but should be
relatively small, probably similar to at most an OpenPGP one. I think
depending on the format the certificates can also be easily detectable.

I'm not sure I'd be comfortable with having to depend (even weakly) on openssh-client to be able to work with these. At least the implementation
is portable C so it should be available everywhere, so in theory such
(weak?) dependency would be possible everywhere. I'm not sure how you'd automatically verify signatures if you need to specify the namespace,
the allowed signers and the signer identity, I guess you'd probably
need to store this alongside as well. And the ssh-keygen CLI seems
rather brittle. :/

Is this really being used widely, or much at all?

3. signify https://man.openbsd.org/signify.1

The certificates and signatures are extremely short, as in around
100 chars or so? But they are going to be hard to detect, as there is
no marker, besides a convention of a prefixed line with
«untrusted comment: <comment-here>». This looks short enough that
perhaps instead of shipping it in a file, it could be added somewhere
as a field value?

The signify-openbsd package is rather tiny, and its CLI is not too
bad. I assume projects with strong BSD origins might use this to sign
releases, but how common is this?

I see supporting them in Debian requiring:
1. Decisions on which schemes to support. I'd assume we support all that
are available in Debian and have some significant use.

I think that just like some compression formats that might be deemed
not worth the effort, to me this could look similar.

I also think even then, we could decide that we do not want full
support for all of these, but perhaps still partial support might be
good enough for now. Say shipping the signatures and certs somewhere
that requires no integration infra work, for example, or only
supporting them in say uscan.

Some, like sigstore, can be used in multiple modes, we'd have to make
some selections.
2. Support in uscan to verify at download/checkout time.
2.1: Syntax in watch files to locate signature files.
2.2: Path in source packages / watch files to declare trusted signers.

Right, see my comments about detectability, so we might need to place
some of these in different places to make them easily recognizable or distinguishable from other things.

2.3: Syntax in watch files for signature verification in git mode.
3. Support in dpkg-source to include detached signatures in source
packages.
3.1: Declare expected formats and filename extensions.

dpkg-source currently normalizes binary OpenPGP signatures into ASCII
armored ones, includes references to them into the .dsc when building,
and verifies the signatures (using the upstream signing certificates)
on build and extraction. It also warns when there's a signing
certificate but no signature present.

4. Support in the archive? (Is anything necessary?)

Yes, DAK needs explicit support for this. (At least AFAIR from when
adding the current upstream signature support.)

Is this something people are interested in pursuing?

In a way the fragmentation in the signing formats and WOT, seems
rather annoying. I understand some people (for whatever reason) are
not happy with OpenPGP, so that reaction seems to be expected, but
it's still a bit meh. :) I'm not sure all new contenders might be
ideal to integrate, and doing so also implies "legitimizing" them.

For the dpkg part I'm of course open to considerations, but I'm not
sure what level of integration might end up making sense there TBH.

Thanks,
Guillem

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

XPost: linux.debian.maint.dpkg, linux.debian.maint.python

SGkgR3VpbGxlbSAoMjAyNC4xMC4wNV8wMTozMjo0NV8rMDAwMCkKPiA+IDEuIHNpZ3N0b3JlIGh0 dHBzOi8vZG9jcy5zaWdzdG9yZS5kZXYvCj4gCj4gQWx0aG91Z2ggSSd2ZSBoZWFyZCBvZiB0aGlz IGJlZm9yZSwgSSBuZXZlciByZWFsbHkgY2hlY2tlZCB3aGF0IGlzCj4gdGhlIGFjdHVhbCBkZXNp Z24gYmVoaW5kIGl0LCBhbmQgaXRzIGltcGxpY2F0aW9ucy4KCkknbSBuZXcgdG8gYWxsIHRoaXMg dG9vLCBidXQgSSBjYW4gYW5zd2VyIHNvbWUgb2YgdGhvc2UgcXVlc3Rpb25zIGZyb20KbXkgb3du IHJlYWRpbmc6Cgo+IEknbSBub3Qgc3VyZSBob3cgcmVsaWFudCBvbiBjZW50cmFsaXphdGlvbiB0 aGlzIGlzLCBvciBvbiB0aGUgYXBwYXJlbnQKPiBzcGVjaWZpYyBPSURDIHByb3ZpZGVycyBjdXJy ZW50bHkgaW4gdXNlCgpUaGUgc2lnbmluZyBwcm9jZXNzIGlzIHJlbGlhbnQgb24gY2VudHJhbGl6 YXRpb24uIFRoZSBzaWduYXR1cmUgc2NoZW1lCnRydXN0cyB0aGUgY2VudHJhbCBzZXJ2ZXIgYW5k IE9JREMgcHJvdmlkZXIgdG8gaWRlbnRpZnkgdGhlIHNpZ25lci4gSXQKc2lnbnMgdGhhdCBhc3Nl cnRpb24gYW5kIHN0b3JlcyBpdCBpbiBhIENUIGxvZy4KCj4gYWJvdXQgb2ZmbGluZSBvcGVyYXRp b25zCgpWZXJpZmljYXRpb24gaXMgcG9zc2libGUgb2ZmbGluZSwgdGhlIGJ1bmRsZSBpbmNsdWRl cyBDVCBwcm9vZi4gVGhlCmRvd25zaWRlIG9mIHZlcmlmeWluZyBvZmZsaW5lIGlzIHRoYXQgeW91 IGNhbiBtaXNzIHJldm9jYXRpb24uCgo+IGFuZCB3aGV0aGVyIHRoYXQgaXMgYSBmaXJzdCBjbGFz cyB1c2UsIG9yIGlmIHRoYXQgaW1wbGllcyBsaW1pdGF0aW9ucywKPiBldGMuCgpUaGVyZSBhcmUg ZGVmaW5pdGVseSBsaW1pdGF0aW9ucy4gVGhlIHNldCBvZiB0cnVzdGVkIHBhcnRpZXMgaXMgbXVj aApsYXJnZXIgdGhhbiB3aGVuIHZlcmlmeWluZyBhbiBPcGVuUEdQIHNpZ25hdHVyZS4gSW5zdGVh ZCBvZiByZWx5aW5nIG9uCnRoZSBzaWduZXIgbWFpbnRhaW5pbmcgcGVyZmVjdCBzZWN1cml0eSBv ZiB0aGVpciBPcGVuUEdQIGtleXMsIHdlIGFyZQpyZWx5aW5nIG9uIHRoZSBzZWN1cml0eSBvZiB0 aGVpciBPSURDIHNlcnZlciAmIHNpZ3N0b3JlLiBTaWduYXR1cmVzIGFyZQpwdWJsaWNseSBhdWRp dGFibGUsIHNvIGZyYXVkdWxlbnQgc2lnbmF0dXJlcyBhcmUgdGhlb3JldGljYWxseQpkZXRlY3Rh YmxlIGJ5IHRoZSBpZGVudGl0eSBvd25lci4gVGhlcmUgYXJlIHJldm9jYXRpb24gbWVjaGFuaXNt cy4KCkEgc3lzdGVtIGxpa2UgdGhpcyBoYXMgYSBjaGFuY2Ugb2YgYmVpbmcgd2lkZWx5IGFkb3B0 ZWQgaW4gc29mdHdhcmUKZWNvc3lzdGVtcyBpbiBhIHdheSB0aGF0IEkgZG9uJ3QgdGhpbmsgT3Bl blBHUCBoYXMgYW55IG1vcmUuCgo+IEV2ZW4gdGhvdWdoIGluIHRoZSBQeXRob24gdXBzdHJlYW0g dGhyZWFkIGl0J3MgbWVudGlvbmVkIHRoYXQgbWFueQo+IHVwc3RyZWFtcyBhcmUgbW92aW5nIGlu dG8gdXNpbmcgaXQsIGl0J3Mgbm90IGNsZWFyIHRvIG1lIGVpdGhlciB3aGF0Cj4gYXJlIHRoZSBs b25nLXRlcm0gcHJvc3BlY3RzIG9mIHRoaXMgZWl0aGVyLgoKWWVwLiBIYXJkIHRvIHNheS4gV2Ug Y2FuIHNpdCBvbiB0aGUgZmVuY2UgdW50aWwgaXQgYmVjb21lcyBvYnZpb3VzLiBUaGlzCmlzIGhv dyB3ZSBvZnRlbiBkbyB0aGluZ3MgOikKCkluIHRoZSBtZWFudGltZSwgcGFja2FnZXMgbGlrZSBQ eXRob24gbWF5IGRyb3AgT3BlblBHUCBpbiBmYXZvciBvZiBpdC4KVGhhdCBtYXkgYmUgYSBzbWFs bCBsb3NzLiBXZSB3ZXJlbid0IGV2ZW4gdmVyaWZ5aW5nIFB5dGhvbidzIE9wZW5QR1AKc2lnbmF0 dXJlcyB1bnRpbCB5ZXN0ZXJkYXkuCgo+IEkndmUgbm90IGNoZWNrZWQgZWl0aGVyIHdoYXQgaXMg dGhlIGZvcm1hdCBvZiB0aGUgY2VydGlmaWNhdGVzIGFuZAo+IHNpZ25hdHVyZXMgZm9yIHRoaXMs IGhvdyBkZXRlY3RhYmxlIHRoZXkgYXJlLCB0aGVpciBzaXplLCBldGMuCgpUaGUgYnVuZGxlIGZv cm1hdCAoY2FwYWJsZSBvZiBvZmZsaW5lIHVzZSkgaXMgYSBKU09OIGRvY3VtZW50IHdpdGggYQpt ZWRpYVR5cGUga2V5LiBJdCdzIGRldGVjdGFibGUuCgo+IEZyb20gUHl0aG9uIHVwc3RyZWFtIGFu ZCB5b3VyIGNvbW1lbnQgYmVsb3csIEkgdGFrZSB0aGUgb25seQo+IGltcGxlbWVudGF0aW9ucyBh cmUgZWl0aGVyIFB5dGhvbiBvciBHb2xhbmcsIHdoaWNoIHNlZW1zIHByb2JsZW1hdGljIGFzCj4g c29tZXRoaW5nIHRvIGhhdmUgdG8gcHVsbCBpbnRvIHNheSBhIGJ1aWxkLWVzc2VudGlhbCBjaHJv b3RzIGJ5IGRlZmF1bHQuCj4gKE5vdCB0byBtZW50aW9uIHRoYXQgdGhlc2UgYXJlIG5vdCBldmVu IHlldCBpbiBEZWJpYW4uIDopCgpXZSBjYW4gdGFrZSBhIHNsb3dlciBhcHByb2FjaCBhbmQgbm90 IGluY2x1ZGUgc3VwcG9ydCBpbiBidWlsZC1lc3NlbnRpYWwKY2hyb290cyBmb3IgdGhlc2UgbmV3 ZXIgc2NoZW1lcy4gUmUtZXZhbHVhdGUgbGF0ZXIsIGlmIG9uZSBvZiB0aGVtCnJlYWxseSBibG93 cyB1cCBpbiBwb3B1bGFyaXR5LgoKQW5kIG9mIGNvdXJzZSwgd2Ugc2hvdWxkIHByb2JhYmx5IGp1 c3Qgc3RhcnQgd2l0aCB1c2NhbiwgYW5kIHRhbGsgYWJvdXQKYWRkaW5nIHN1cHBvcnQgdG8gZHBr ZyBsYXRlciwgd2hlbiB3ZSBjYW4gc2VlIHNpZ25pZmljYW50IHVzZS4KCj4gPiAyLiBzc2ggc2ln bmF0dXJlcwo+IAo+IEFGQUlLIHRoZXNlIGFyZSB1c2VkIHZpYSBzc2gta2V5Z2VuLiBUaGUgc2ln bmF0dXJlcyBhcmUgcHJldHR5IGVhc3kgdG8KPiBkZXRlY3QsIGFzIHRoZXkgYXJlIHN1cnJvdW5k ZWQgYnkgwqstLS0tLUJFR0lOIFNTSCBTSUdOQVRVUkUtLS0tLcK7IGFuZAo+IMKrLS0tLS1FTkQg U1NIIFNJR05BVFVSRS0tLS0twrsgYW5kIGFyZSBBU0NJSSBlbmNvZGVkLiBUaGUgY2VydGlmaWNh dGVzCj4gYW5kIHNpZ25hdHVyZXMgY2FuIGJlIGZldyBsaW5lcyBvciBtYW55IGxpbmVzLCBidXQg c2hvdWxkIGJlCj4gcmVsYXRpdmVseSBzbWFsbCwgcHJvYmFibHkgc2ltaWxhciB0byBhdCBtb3N0 IGFuIE9wZW5QR1Agb25lLiBJIHRoaW5rCj4gZGVwZW5kaW5nIG9uIHRoZSBmb3JtYXQgdGhlIGNl cnRpZmljYXRlcyBjYW4gYWxzbyBiZSBlYXNpbHkgZGV0ZWN0YWJsZS4KPiAKPiBJJ20gbm90IHN1 cmUgSSdkIGJlIGNvbWZvcnRhYmxlIHdpdGggaGF2aW5nIHRvIGRlcGVuZCAoZXZlbiB3ZWFrbHkp IG9uCj4gb3BlbnNzaC1jbGllbnQgdG8gYmUgYWJsZSB0byB3b3JrIHdpdGggdGhlc2UuIEF0IGxl YXN0IHRoZSBpbXBsZW1lbnRhdGlvbgo+IGlzIHBvcnRhYmxlIEMgc28gaXQgc2hvdWxkIGJlIGF2 YWlsYWJsZSBldmVyeXdoZXJlLCBzbyBpbiB0aGVvcnkgc3VjaAo+ICh3ZWFrPykgZGVwZW5kZW5j eSB3b3VsZCBiZSBwb3NzaWJsZSBldmVyeXdoZXJlLiBJJ20gbm90IHN1cmUgaG93IHlvdSdkCj4g YXV0b21hdGljYWxseSB2ZXJpZnkgc2lnbmF0dXJlcyBpZiB5b3UgbmVlZCB0byBzcGVjaWZ5IHRo ZSBuYW1lc3BhY2UsCj4gdGhlIGFsbG93ZWQgc2lnbmVycyBhbmQgdGhlIHNpZ25lciBpZGVudGl0 eSwgSSBndWVzcyB5b3UnZCBwcm9iYWJseQo+IG5lZWQgdG8gc3RvcmUgdGhpcyBhbG9uZ3NpZGUg YXMgd2VsbC4gQW5kIHRoZSBzc2gta2V5Z2VuIENMSSBzZWVtcwo+IHJhdGhlciBicml0dGxlLiA6 Lwo+IAo+IElzIHRoaXMgcmVhbGx5IGJlaW5nIHVzZWQgd2lkZWx5LCBvciBtdWNoIGF0IGFsbD8K Ckkga25vdyBpdCdzIHVzZWQgaW4gc29tZSBnaXQgdGFnIHNpZ25pbmcuIFNlZTogIzEwMjMxNDAK VGhvc2UgYXJlbid0IHJlbGV2YW50IHRvIGRwa2csIGF0IGFsbCAod2l0aG91dCBnaXQgc291cmNl IHBhY2thZ2VzKS4KCj4gPiBJIHNlZSBzdXBwb3J0aW5nIHRoZW0gaW4gRGViaWFuIHJlcXVpcmlu ZzoKPiA+IDEuIERlY2lzaW9ucyBvbiB3aGljaCBzY2hlbWVzIHRvIHN1cHBvcnQuIEknZCBhc3N1 bWUgd2Ugc3VwcG9ydCBhbGwgdGhhdAo+ID4gICAgYXJlIGF2YWlsYWJsZSBpbiBEZWJpYW4gYW5k IGhhdmUgc29tZSBzaWduaWZpY2FudCB1c2UuCj4gCj4gSSB0aGluayB0aGF0IGp1c3QgbGlrZSBz b21lIGNvbXByZXNzaW9uIGZvcm1hdHMgdGhhdCBtaWdodCBiZSBkZWVtZWQKPiBub3Qgd29ydGgg dGhlIGVmZm9ydCwgdG8gbWUgdGhpcyBjb3VsZCBsb29rIHNpbWlsYXIuCj4gCj4gSSBhbHNvIHRo aW5rIGV2ZW4gdGhlbiwgd2UgY291bGQgZGVjaWRlIHRoYXQgd2UgZG8gbm90IHdhbnQgZnVsbAo+ IHN1cHBvcnQgZm9yIGFsbCBvZiB0aGVzZSwgYnV0IHBlcmhhcHMgc3RpbGwgcGFydGlhbCBzdXBw b3J0IG1pZ2h0IGJlCj4gZ29vZCBlbm91Z2ggZm9yIG5vdy4gU2F5IHNoaXBwaW5nIHRoZSBzaWdu YXR1cmVzIGFuZCBjZXJ0cyBzb21ld2hlcmUKPiB0aGF0IHJlcXVpcmVzIG5vIGludGVncmF0aW9u IGluZnJhIHdvcmssIGZvciBleGFtcGxlLCBvciBvbmx5Cj4gc3VwcG9ydGluZyB0aGVtIGluIHNh eSB1c2Nhbi4KClllcywgSSB0aGluayBzdGFydGluZyB3aXRoIHVzY2FuIGlzIHRoZSBvYnZpb3Vz IHdheSwgYnV0IEkgd291bGQgd2FudCB0bwpkbyBzbyBpbiBhIHdheSB0aGF0IGxlYXZlcyB0aGUg d2F5IG9wZW4gdG8gc3VwcG9ydCBpbiBkcGtnIGxhdGVyLgpJZGVhbGx5IHdpdGhvdXQgYW55IHNv dXJjZSBwYWNrYWdlIEFQSSBjaGFuZ2VzLCBsYXRlci4KClN0ZWZhbm8K

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

XPost: linux.debian.maint.dpkg, linux.debian.maint.python

On 2024-10-05 03:32, Guillem Jover wrote:

For an example of the activity that is going on in the OpenPGP ecosystem, here's a list of some of the non-GnuPG implementations already present
in Debian, by programming language:

Thanks for the list! I was aware of some of them, but not all.

* Python:
- PGPy

I wonder, how that could escape my radar ;-) Thanks!

(Need to file an RFP for https://openpgpjs.org/ btw.)

Cheers

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

XPost: linux.debian.maint.python, linux.debian.maint.dpkg

Stefano Rivera <stefanor@debian.org> writes:

Should we expand this to include some of these new mechanisms?
Things brought up in the debian-python thread include:
1. sigstore https://docs.sigstore.dev/
2. ssh signatures
3. signify https://man.openbsd.org/signify.1

+1

I believe all signatures we trust should be encoded in a non-mutable transparency log like Sigstore/Sigsum etc. But the first step towards
that is to add support for verifying that property.

There is a general trend towards getting upstream sources from Git
rather than tarballs in Debian, but we're a long way from moving across completely, or even finding consensus to do so.
These signature mechanisms can generally be applied to git commits as
well as tarballs.

Signatures of git commits is the same as a signature on a SHA1 object
which is broken for authentication purposes. But it is possible to
discuss these issues separately, paving the way for git commit signing
to be trustworthy when GitHub/GitLab moves to SHA256.

/Simon

-----BEGIN PGP SIGNATURE-----

iIoEARYIADIWIQSjzJyHC50xCrrUzy9RcisI/kdFogUCZwEWnRQcc2ltb25Aam9z ZWZzc29uLm9yZwAKCRBRcisI/kdFoiJzAQCiA63P/cLfQZKPpjVnAYhNfBTNX52l hEWyk7krfYlWWQEArjo1j/4/yWdL48UlmgvWMLaotP8eStD9+AP3Bzu5agk=
=8MHr
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

No objections to have this kind of capability, but I still strongly
believe that importing tar archives is highly suboptimal and directly branching off the upstream git repository is an highly superior workflow
and should be used as much as possible.

This being said, I maintain some packages which are released by the
upstream maintainers only as tar archives (because OpenBSD).

--
ciao,
Marco

-----BEGIN PGP SIGNATURE-----

iHUEABYIAB0WIQQnKUXNg20437dCfobLPsM64d7XgQUCZwFIEwAKCRDLPsM64d7X gW2fAP92FAvNW60zI/3D96EXaewMMEK9caphx/SRBZQKofnhrgEArxLvThEUd5rJ RtIIBr8sR3FGPNHGQNxLiLR7SXksxwc=
=j5ei
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

No objections to have this kind of capability, but I still strongly
believe that importing tar archives is highly suboptimal and directly branching off the upstream git repository is an highly superior workflow
and should be used as much as possible.

This being said, I maintain some packages which are released by the
upstream maintainers only as tar archives (because OpenBSD).

Also some projects release tarballs with extra additions that are not
in the same git, or they strip away directories/files that are in git,
but are irrelevant for users. If upstreams do that, then their intent
is that downstreams are better off consuming the tarballs.

This is not a problem though, We can have the best of both, as
git-buildpackage supports dual import from both upstream git and
tarball to maximize supply chain auditability.

You can see this in action in e.g. https://salsa.debian.org/mariadb-team/mariadb-server/-/network/debian%2Flatest?extended_sha1=f134a53ffcaad16eabedb30809837d5ee8170bc8&filter_ref=1
The upstream branch 11.4 and tag mariadb-11.4.3 has the upstream git
release contents, while the branch upstream/latest and tag
upstream/11.4.3 shows the contents of the release tarball. The diff
between these two branches shows how the upstream tarball differs from
the upstream git commit at the time. The git side can be verified with
git tag signature, and the tarball side is verified by tarball
signature (thanks to also pristine-tar being used). This
upstream/latest was then merged on debian/latest, which has git tags
signed by Debian maintainer.

Although, as stated by Simon, git signatures are based on SHA-1 so not unbreakable anymore, but the point here is to illustrate the
git-buildpackage feature.

If you want to see the details, see gbp.conf in the package (https://salsa.debian.org/mariadb-team/mariadb-server/-/blob/debian/latest/debian/gbp.conf).

- Otto

PS. Massive thanks to Guido for maintaining git-buildpackage, it is a gem!

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Oct 06, Otto Kekäläinen <otto@debian.org> wrote:

Also some projects release tarballs with extra additions that are not
in the same git, or they strip away directories/files that are in git,

I am sure that there are counterexamples, but usually they exclude
things that we want to rebuild anyway, like configure.

Although, as stated by Simon, git signatures are based on SHA-1 so not unbreakable anymore, but the point here is to illustrate the
git-buildpackage feature.

This has been already discussed and it is not true.

--
ciao,
Marco

-----BEGIN PGP SIGNATURE-----

iHUEABYIAB0WIQQnKUXNg20437dCfobLPsM64d7XgQUCZwI0cQAKCRDLPsM64d7X gR2WAP0Ysmk+u+++SiHV5tQMBsshFlKYmWWCkd2P6bzwjFGLYQD+KqNQX08niufS q/EhnTtcFs6MsBZqkPHS0058fsjcfwU=
=LeYe
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Otto Kekäläinen <otto@debian.org> writes:

No objections to have this kind of capability, but I still strongly
believe that importing tar archives is highly suboptimal and directly
branching off the upstream git repository is an highly superior workflow
and should be used as much as possible.

This being said, I maintain some packages which are released by the
upstream maintainers only as tar archives (because OpenBSD).

Also some projects release tarballs with extra additions that are not
in the same git, or they strip away directories/files that are in git,
but are irrelevant for users. If upstreams do that, then their intent
is that downstreams are better off consuming the tarballs.

Indeed. Some projects also release both variants -- for example
'libntlm' and 'oath-toolkit' upstream releases both traditional autoconf tarballs and minimal source-only 'git archive' style tarballs.

This is not a problem though, We can have the best of both, as git-buildpackage supports dual import from both upstream git and
tarball to maximize supply chain auditability.

You can see this in action in e.g. https://salsa.debian.org/mariadb-team/mariadb-server/-/network/debian%2Flatest?extended_sha1=f134a53ffcaad16eabedb30809837d5ee8170bc8&filter_ref=1
The upstream branch 11.4 and tag mariadb-11.4.3 has the upstream git
release contents, while the branch upstream/latest and tag
upstream/11.4.3 shows the contents of the release tarball. The diff
between these two branches shows how the upstream tarball differs from
the upstream git commit at the time. The git side can be verified with
git tag signature, and the tarball side is verified by tarball
signature (thanks to also pristine-tar being used). This
upstream/latest was then merged on debian/latest, which has git tags
signed by Debian maintainer.

...

If you want to see the details, see gbp.conf in the package (https://salsa.debian.org/mariadb-team/mariadb-server/-/blob/debian/latest/debian/gbp.conf).

That setup sounds nice! What is your workflow to import a new upstream release?

I see the watch file still points to the tarball:

https://salsa.debian.org/mariadb-team/mariadb-server/-/blob/debian/latest/debian/watch

Would it be possible to extend the debian/watch syntax to be able to
express both a tarball and upstream git branch?

Then 'gbp import-orig' would import both the tarball and sync git, after performing PGP tarball verification and Git branch signature
verification. I suppose you are doing that manually somehow now?

/Simon

--=-=-Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iIoEARYIADIWIQSjzJyHC50xCrrUzy9RcisI/kdFogUCZwJEMBQcc2ltb25Aam9z ZWZzc29uLm9yZwAKCRBRcisI/kdFoqR1AP9utdPHeWTEe1YkZ2tb3bLIRKct0wud g4A58gzRyUeIYgD/b6dI7sjSzFks0P4xjiJYEzA9SzHyUjggHSgnWSVKbA4=meXt
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sun, 06 Oct 2024 at 10:02:56 +0200, Simon Josefsson wrote:

Otto Kekäläinen <otto@debian.org> writes:

git-buildpackage supports dual import from both upstream git and
tarball to maximize supply chain auditability.

That setup sounds nice! What is your workflow to import a new upstream release?

The GNOME team does this too. If the upstream git repo has "nice" tag names
and the tarball doesn't need repacking (e.g. src:glib2.0), the import
workflow is almost the same as if you were only using tarballs:

# once per checkout
# ("gbp clone --add-upstream-vcs" will do this for you automatically if
# your debian/upstream/metadata lists a Repository)
git remote add upstreamvcs https://...

# once per new upstream release
git fetch upstreamvcs
uscan --download
gbp import-orig .../glib2.0_*.tar.*

The necessary glue to set this up is that `debian/gbp.conf` has
"[import-orig] upstream-vcs-tag = %(version%~%.)s" or similar, to tell git-buildpackage how to map a tarball version number to a tag. See dbus and flatpak for examples of packages where the tag name has a prefix.

If the upstream tarball is repacked, for example to include inconveniently large generated files (like GTK's HTML documentation) or vendored
convenience copies whose copyright status we would prefer not to have to document, then you need to use something like

gbp import-orig --upstream-vcs-tag=4.16.3 .../gtk4_4.16.3+ds.orig.tar.xz

to compensate for that: I don't think git-buildpackage supports removing
an arbitrary suffix like +ds from the version number to get the tag name.

smcv

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi!

...

That setup sounds nice! What is your workflow to import a new upstream release?

I see the watch file still points to the tarball:

https://salsa.debian.org/mariadb-team/mariadb-server/-/blob/debian/latest/debian/watch

Would it be possible to extend the debian/watch syntax to be able to
express both a tarball and upstream git branch?

Then 'gbp import-orig' would import both the tarball and sync git, after performing PGP tarball verification and Git branch signature
verification. I suppose you are doing that manually somehow now?

Simon already replied and explained use of the git-buildpackage flag --add-upstream-vcs, but I encourage you to read up more details at https://manpages.debian.org/unstable/git-buildpackage/gbp-import-orig.1.en.html#upstream~3
and note that you can define it in your package debian/gbp.conf
permanently, as upstream rarely changes the syntax they use, and
whatever is permanently in debian/gbp.conf can be omitted from the
command-line and thus makes using gbp commands simpler and uniform for
all packages (as if there are per-upstream quirks, they are handled automatically per gbp.conf configs).

I try to maintain both gbp.conf and README.source(.md) in all my
packages to so that anyone can easily contribute, but in this package
the REDME was a bit outdated, so this was a good trigger to update it
- see now https://salsa.debian.org/mariadb-team/mariadb-server/-/blob/debian/latest/debian/README.source.md

A typical import starts with fetching upstream branch and tag with one command:

# git fetch --verbose upstreamvcs
POST git-upload-pack (371 bytes)

From https://github.com/MariaDB/server

= [up to date] 11.4 -> upstreamvcs/11.4
* [new tag] mariadb-11.4.3 -> mariadb-11.4.3

..followed by the tarball download and automatic import that ties both
git and tarball together:

# gbp import-orig --uscan --verbose
gbp:info: Launching uscan...
uscan warn: The directory name packages doesn't match the requirement of
--check-dirname-level=1 --check-dirname-regex=groonga\-normalizer\-mysql(-.+)? .
Set --check-dirname-level=0 to disable this sanity check feature., skipping Newest version of mariadb on remote site is 11.4.3, local version is 11.4.2

Newer package available from:

=> https://archive.mariadb.org/mariadb-11.4.3/source/mariadb-11.4.3.tar.gz
gpgv: Signature made Tue Aug 6 14:56:23 2024 UTC
gpgv: using RSA key 177F4010FE56CA3336300305F1656F24C74CD1D8 gpgv: Good signature from "MariaDB Signing Key <signing-key@mariadb.org>" Leaving ../mariadb_11.4.3.orig.tar.gz where it is.
gbp:info: Using uscan downloaded tarball ../mariadb_11.4.3.orig.tar.gz gbp:info: <DebianUpstreamSource path='../mariadb_11.4.3.orig.tar.gz' signaturefile='../mariadb_11.4.3.orig.tar.gz.asc'>
gbp:info: Importing '../mariadb_11.4.3.orig.tar.gz' to branch 'upstream/latest'...
gbp:info: Source package is mariadb
gbp:info: Upstream version is 11.4.3
gbp:info: Replacing upstream source on 'debian/latest'
gbp:info: Running Postimport hook
gbp:debug: dch -v 1:11.4.3-1 "New upstream release" [] []
gbp:debug: rm ['-rf', '/tmp/test/tmpfd4j_x0p'] []
gbp:info: Successfully imported version 11.4.3 of ../mariadb_11.4.3.orig.tar.gz

Try running the above with `--verbose` to see all the details on what
it does to git, signatures and pristine-tar.

If I forget the fetch upstreamvcs, gbp will automatically error:

# gbp import-orig --uscan --verbose
gbp:info: Launching uscan...
...
gbp:info: Importing '../mariadb_11.4.3.orig.tar.gz' to branch 'upstream/latest'...
...
gbp:info: Upstream version is 11.4.3
gbp:error: Import of ../mariadb_11.4.3.orig.tar.gz failed: Can't find
upstream vcs tag/revision at 'mariadb-11.4.3'

Git-buildpackage will automatically error and abort on many important
checks, just as one would expect:
- if tarball signature is missing, or it does not verify with upstream
signing key
- if upstream git tag signature is missing (unfortunately it will
accept any signature, there is currently no way to say which keyring
of upstream devs are valid release tag signatores)
- if my own gpg key is missing, and I can't sign the upstream import tag

I guess debian/watch could be extended to scan for both upstream git
tag and tarball availability in parallel, but as long as gbp.conf is
correctly configured, the overall process is pretty smooth. Also git-buildpackage itself could be extended to have even more
automation, but it is already on a level that I am delighted with it
and use this workflow with all my packages.


After the import I continue by manually writing noteworthy things
about the upstream release in changelog:

# gbp dch --distribution=UNRELEASED \
--commit --commit-msg="Update changelog and refresh patches
after %(version)s import" \
-- debian
gbp:info: Changelog last touched at 'b585ff1ab2762e9c81bc7b9cc8be379fefa880a5' gbp:info: Continuing from commit 'b585ff1ab2762e9c81bc7b9cc8be379fefa880a5' gbp:info: Only looking for changes on 'debian'
gbp:info: Changelog committed for version 1:11.4.3-1

Then I continue by refreshing debian/patches/* contents:

gbp pq rebase
gbp pq export

Plus whatever else I think is needed, one git commit per self-standing
change, and then finally when time for upload I run:

gbp dch --release --commit

..and then finally build and upload.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)