I have applied patches 0001, 0002 and 0003 to our working tree, and should include them by the next keyring push. But, of course, patch 0004 is the
most significative of them all.
I reviewed it, and find the renaming/editing thorough. However, due to the indirect uses, I would like a further review of the code. I see your patch
is taking care of providing the symlinks in the packages only, but not in
the generated output/ directory we push to keyring.debian.org; given the
way this system is used (keys are used via rsync), I think adding symlinks there would be enough (but I don't want to create that wide breakage
without another pair of eyes confirming so!).
Anyway, it would be even better (of course, it would be a future step) to
get a listing of what actually uses the keyrings, and push to update said programs as well.
I am writing to you to help us decide whether it is pertinent to do a[...]
move right now, or whether we should wait for you to implement
anything.
Bug #1101418 (for which you have been Cc:d already) requests renaming
the Debian keyrings from *.gpg to *.pgp
Guillem kindly prepared some patches to do this, and after some
discussion, added the creation of some symlinks. The result of the
patch in question will be that the keyrings published in
kaufmann.debian.org will be renamed from *.gpg to *.pgp, and that
symlinks will be created preserving the original names; in the
future, we might work towards removing the ymlinks.
We need input from DSA to make sure we can perform this change. some
specific points we discussed on IRC are:
- ftp-master processing
- Email address to db.debian.org
- vote.debian.org
- Validation of mails sent to signed-only lists
But, of course, other subsystems might also need it. We believe
everything will work transparently if rsync is properly set to mirror symlinks.
On Mon, 2025-04-21 at 13:39 -0600, Gunnar Wolf wrote:
I am writing to you to help us decide whether it is pertinent to do a
move right now, or whether we should wait for you to implement
anything.
Bug #1101418 (for which you have been Cc:d already) requests renaming[...]
the Debian keyrings from *.gpg to *.pgp
Guillem kindly prepared some patches to do this, and after some
discussion, added the creation of some symlinks. The result of the
patch in question will be that the keyrings published in kaufmann.debian.org will be renamed from *.gpg to *.pgp, and that
symlinks will be created preserving the original names; in the
future, we might work towards removing the ymlinks.
We need input from DSA to make sure we can perform this change. some specific points we discussed on IRC are:
- ftp-master processing
- Email address to db.debian.org
- vote.debian.org
- Validation of mails sent to signed-only lists
But, of course, other subsystems might also need it. We believe
everything will work transparently if rsync is properly set to mirror symlinks.
This is primarily my personal thoughts rather than an official DSA ack.
To start from the beginning, as it were, the method by which debian.org systems receive updated keyrings is userdir-ldap. The script which
pulls the current active keyrings from keyring.d.o to db.d.o (so that
they can be deployed by ud-ldap) is https://salsa.debian.org/dsa-team/mirror/dsa-misc/-/blob/master/scripts/sync-keyring?ref_type=heads
That script runs from cron every 15 minutes on the db.d.o host.
This appears to be the primary point at which rsync's behaviour with
respect to symlinks is relevant. The various copy steps each use
rsync's "-a" option, so should preserve the symlinks.
The final stage of the sync - to "keyring-final" - copies only the
files listed in sha512sums.txt, so you would need to ensure that both
the .gpg and .pgp files / symlinks are included there.
userdir-ldap maintains its own list of keyrings which are deployed to
those hosts requesting them (e.g. ftp-master). While the keyrings are
copied from db.d.o to each host via rsync, the preparation of the rsync source area uses Python's shutil.copy(), so AFAICS would follow the new symlinks and continue to deploy *.gpg to var/lib/misc/thishost/ on
relevant hosts as real files. That should mean that things would
continue to work, but does mean that the rename would *not* propagate
to client hosts.
Hi!
On Mon, 2025-04-21 at 22:00:09 +0100, Adam D. Barratt wrote:
userdir-ldap maintains its own list of keyrings which are deployed
to those hosts requesting them (e.g. ftp-master). While the
keyrings are copied from db.d.o to each host via rsync, the
preparation of the rsync source area uses Python's shutil.copy(),
so AFAICS would follow the new symlinks and continue to deploy
*.gpg to var/lib/misc/thishost/ on relevant hosts as real files.
That should mean that things would continue to work, but does mean
that the rename would *not* propagate to client hosts.
Ah, then this would seem to be safe to deploy now, and the file types
problem could be fixed later on. I have had several changes for
userdir-ldap pending submission, but not this one about
shutil.copy(), thanks. Will see how to improve that, and then send
patches for userdir-ldap to DSA (I think I already sent out patches
for userdir-ldap-cgi).
I think though, the other related patch I sent for dsa-puppet, might self-heal the symlinks?
Also, (I'm not sure whether I mentioned this before, besides Gunnar), something I noticed while trying to make sense how this all works was
that:
* At least on usper.debian.org, the
/srv/keyring.debian.org/keyrings/ directory contains a non-
symlink
debian-maintainer.gpg file (missing final «s»).
* On keyring.debian.org there's an extra-keys.pgp leftover(?) file,
perhaps as part of some old transition?
Ah, then this would seem to be safe to deploy now, and the file types
problem could be fixed later on. I have had several changes for
userdir-ldap pending submission, but not this one about
shutil.copy(), thanks. Will see how to improve that, and then send
patches for userdir-ldap to DSA (I think I already sent out patches
for userdir-ldap-cgi).
"Probably". If it doesn't work for some reason, however, the effects
could include things such as dak no longer accepting any uploads from
DDs because it can no longer find their public keys.
I'd therefore be tempted to disable both the "pull" and "push" sides on >db.d.o shortly before the keyring side is deployed, and test them by
hand afterwards.
I can't personally guarantee being around at any particular time this
week though I'm afraid.
Also, (I'm not sure whether I mentioned this before, besides Gunnar),
something I noticed while trying to make sense how this all works was
that:
* At least on usper.debian.org, the
/srv/keyring.debian.org/keyrings/ directory contains a non-
symlink
debian-maintainer.gpg file (missing final «s»).
I think that was me fat-fingering something when testing a while back; >removed.
* On keyring.debian.org there's an extra-keys.pgp leftover(?) file,
perhaps as part of some old transition?
That I'd have to defer to keyring-maint on.
Adam D. Barratt dijo [Wed, Apr 23, 2025 at 05:13:07PM +0100]:
Ah, then this would seem to be safe to deploy now, and the file types
problem could be fixed later on. I have had several changes for
userdir-ldap pending submission, but not this one about
shutil.copy(), thanks. Will see how to improve that, and then send
patches for userdir-ldap to DSA (I think I already sent out patches
for userdir-ldap-cgi).
"Probably". If it doesn't work for some reason, however, the effects
could include things such as dak no longer accepting any uploads from
DDs because it can no longer find their public keys.
I'd therefore be tempted to disable both the "pull" and "push" sides
on
db.d.o shortly before the keyring side is deployed, and test them by
hand afterwards.
I can't personally guarantee being around at any particular time this
week though I'm afraid.
..It makes sense to make sure we have a DSA person available to fix
things
in case it all bursts up in flames.
I was planning on doing this push this Friday, 2025.04.25, in the
morning
(say, anywhere between 09:00–14:00 GMT-6). Can a DSA member be
available in
case this messes up something?
Otherwise, I think it's better to listen to Adam's instinct and delay
the
move. It does not necessarily have to be aligned with a "full" keyring
push.
On 2025-04-23 19:17, Gunnar Wolf wrote:
Adam D. Barratt dijo [Wed, Apr 23, 2025 at 05:13:07PM +0100]:
Ah, then this would seem to be safe to deploy now, and the file types problem could be fixed later on. I have had several changes for userdir-ldap pending submission, but not this one about
shutil.copy(), thanks. Will see how to improve that, and then send patches for userdir-ldap to DSA (I think I already sent out patches
for userdir-ldap-cgi).
"Probably". If it doesn't work for some reason, however, the effects could include things such as dak no longer accepting any uploads from
DDs because it can no longer find their public keys.
I'd therefore be tempted to disable both the "pull" and
"push" sides on
db.d.o shortly before the keyring side is deployed, and test them by
hand afterwards.
I can't personally guarantee being around at any particular time this week though I'm afraid.
..It makes sense to make sure we have a DSA person available to
fix things
in case it all bursts up in flames.
I was planning on doing this push this Friday, 2025.04.25, in
the morning
(say, anywhere between 09:00–14:00 GMT-6). Can a DSA member be
available in
case this messes up something?
Otherwise, I think it's better to listen to Adam's instinct and
delay the
move. It does not necessarily have to be aligned with a "full" keyring push.
Can we push .gpg and .pgp files with identical content for a while?
That'd decouple necessary realtime interactions. I realize that this
does not fix the fact that we might break stuff once we replace .gpg
files with a symlink - but it also opens up the opportunity that we
can also replace the references instead.
On Wed, 2025-04-23 at 21:23:20 +0200, Philipp Kern wrote:
Can we push .gpg and .pgp files with identical content for a while?
That'd decouple necessary realtime interactions. I realize that this
does not fix the fact that we might break stuff once we replace .gpg
files with a symlink - but it also opens up the opportunity that we
can also replace the references instead.
I was thinking something along those lines (as in decouple the archive
switch from the infra switch), as my current main interest right now
would be for the changes to hit Debian trixie, via the .deb in the
archive. But it's true that just duplicating the files might be safer,
and as you mention then we can just simply flip the infra users, and
then later on whether it's a symlink or not will not matter as much I
guess.
I'll try to come up with an updated patch for debian-keyring trying to decouple the package generation from the infra syncing, hopefully by
tomorrow (Thu), or maybe later today.
Can we push .gpg and .pgp files with identical content for a while?
That'd decouple necessary realtime interactions. I realize that this
does not fix the fact that we might break stuff once we replace .gpg
files with a symlink - but it also opens up the opportunity that we
can also replace the references instead.
Philipp Kern dijo [Wed, Apr 23, 2025 at 09:23:20PM +0200]:
Can we push .gpg and .pgp files with identical content for a while?
That'd decouple necessary realtime interactions. I realize that
this does not fix the fact that we might break stuff once we
replace .gpg files with a symlink - but it also opens up the
opportunity that we can also replace the references instead.
Thanks for this simple but brilliant idea, Philipp! That's what we
did — and, at least for a couple of hours, no part of the project has
come down crashing and burning over us ☺
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 493 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 174:42:10 |
| Calls: | 9,705 |
| Calls today: | 5 |
| Files: | 13,736 |
| Messages: | 6,178,843 |