Hi all,

Below is the security review that I did of the tag2upload design.

I am not a neutral party, in the sense that I think tag2upload is a good
idea and should be deployed. However, I do these types of security
reviews professionally, and I tried to approach this review the same way
that I would approach a major work project that needed a security review
to ensure we weren't deploying something with security issues. I
encourage any Debian community member with security expertise to check my
work; with security reviews, the more eyes, the better.

I will also post this review on my web site, probably later tonight if I
have time.


# Security review of tag2upload architecture

Last updated 2024-06-02.

## Introduction

tag2upload is an architecture and protocol for uploading a new revision of
a Debian package by pushing a signed tag to a repository hosted on
Salsa. It has been proposed as an optional alternative to the current (and only) supported mechanism of uploading a signed source package to ftp.upload.debian.org or its SSH equivalent.

This is a security review of the proposed tag2upload architecture as of 2024-06-02. It is based on the following documents retrieved as of that
date:

- https://salsa.debian.org/dgit-team/dgit/-/blob/3704eb1397d27dbd25f19d4c7345ba6e2edf5aa1/TAG2UPLOAD-DESIGN.txt
- https://salsa.debian.org/dgit-team/dgit/-/blob/master/tag2upload.5.pod
- https://manpages.debian.org/git-debpush

## Summary of conclusions

Compared to the existing upload architecture, tag2upload provides
additional defenses against injection of malicious code into source
packages and better traceability of source package contents, at the cost
of some minor additional security risk and infrastructure complexity. I
believe tag2upload has somewhat stronger security properties than the
current upload mechanism but not a profound advantage. I do not believe
it introduces any significant security regressions.

The decision on whether to adopt tag2upload should be made primarily on non-security grounds.

I have several recommendations for follow-on work should tag2upload be
adopted.

## Terminology

In this document, the following terms have precise meanings:

- _prevent_: Stop an attack before it has been successful in a way that
ensures malicious code is never introduced into the archive.

- _detect_: Flag that an attacker may have introduced malicious code into
the archive, simultaneous with or after the attempt. The malicious code
is still introduced and may propagate.

- _trace_: After malicious code has been detected in either the upload
process or in the archive, provide facilities to trace that malicious
code to the keys or account used to introduce it, to specific Git
commits or history, or otherwise closer to its origin in time and
authentication.

Familiarity with two terms from cryptography will also be helpful:

- _collision resistance_: A property of a hash function that makes it
infeasible to construct two inputs, both under the control of the
attacker, that hash to the same hash digest.

- _second-preimage resistance_: A property of a hash function that makes
it infeasible for an attacker, given a hash digest constructed from
input that is not under their control, to construct malicious input that
hashes to the same hash digest. A second-preimage attack on a hash
function is considerably more difficult than a collision attack.

## Threat model

I evaluated both the existing source package upload architecture and the tag2upload architecture against the following threats:

- Someone not in the keyring uploads a malicious source package, possibly
via a sponsor.

- Someone in the keyring (either a Debian Developer or a Debian Maintainer
for a package) uploads a malicious source package but makes it appear
that the package was uploaded by someone else in the keyring.

- An attacker compromises the system a Debian uploader uses to build
source packages and uses that access to inject malicious code into a
source package.

- Someone with administrative access to the archive processing machinery
(DAK, the archive signing key, or similar infrastructure) uploads a
malicious source package.

- Someone with administrative access to the tag2upload server or its
signing key uploads a malicious source package.

- Someone with administrative access to Salsa uploads a malicious source
package.

In each case, I looked at prevention, detection, and tracing.

Neither the existing upload mechanism nor tag2upload attempt to prevent or detect (as opposed to trace) the upload of a malicious source package by someone in full possession of a key in the keyring, so this threat is not considered in this document, although tracing for this threat is
discussed briefly.

## Brief architecture summary

### Existing upload system

The existing source package upload mechanism requires that the uploader construct a Debian source package control file (`*.dsc`) for the upload
and sign it with an OpenPGP signature from a key on the keyring. This file contains SHA-256 hash digests of each tar or diff file contained in the
upload, which together constitute the contents of the source package. The uploader then constructs and signs an upload changes control file (`*.changes`), which contains the SHA-256 hash digests of all files
included in the upload, including the source package control file.

These signatures are verified and checked against the relevant keyring on
a secure system managed by project delegates. If that check passes, the
source package is introduced into the archive and included in archive
metadata (via a SHA-256 hash digest of the source package control file)
signed by the archive key. This triggers the buildds to download and build
the source package and upload a corresponding binary package, which is
signed by a buildd OpenPGP key and verified in the same manner as the
source package upload.

Only the OpenPGP signature of the archive metadata is verified by systems running Debian. Any package whose SHA-256 hash digest is included in that signed archive metadata is treated as trusted by a Debian system. The
original signature on the source package control file, made by the
uploader, is preserved and included in the archive, but it is not
separately checked by the buildds or tools such as `apt-get source`.

Each of these files contains multiple hash digests, but SHA-256 is the strongest of those hashes. Multiple hash digests may add some theoretical collision resistance, but this analysis assumes that SHA-256 is secure
against collision attacks and therefore does not consider the additional
hash digests. Weaknesses in SHA-256 would presumably be addressed by
adding a new secure hash.

### tag2upload

tag2upload replaces the first step of this upload process with the
following:

1. The uploader pushes a signed tag in a specific format to Salsa. For
non-native packages, this may reference an upstream tree in the same
Git repository by commit ID, which will be used to create the `orig`
tar file if needed.

2. Salsa notifies a web hook on a secure project-maintained system that a
new tag of interest has been pushed.

3. That system (with internal privilege separation) retrieves the Git tag
and corresponding commit, verifies the signature and tag metadata, and
verifies that the signer is in the relevant keyring.

4. Inside a VM or schroot, that system retrieves the Git tree and upstream
source tree if applicable, constructs or retrieves the `orig` tar file,
and constructs the Debian source package and source package control
file. This VM or schroot in essence operates as a source package
buildd.

5. The tag2upload server adds control header fields specifying the Git
object ID and the identity string and fingerprint of the uploader,
signs the resulting source package control file, constructs an upload
changes control file, signs it, and creates and signs another Git tag
reflecting any additional Git commits that were required to put the
repository into a canonical format (the "dgit view").

6. The tag2upload server pushes the original Git tag, its referenced tree,
and the additional "dgit view" tag to the publicly-accessible
dgit-repos Git server as a permanent archive.

7. The tag2upload server uploads the signed source package to the normal
archive incoming queue.

Subsequent processing of the upload happens identically to the existing
upload system.

## Analysis

### Source package construction

The existing upload architecture requires trusting the host used by the uploader to build the source package. If that host is compromised, an
attacker could inject malicious code into the source package, either by modifying the upstream tar file (if signed upstream tar files are not
used) or by injecting it into the Debian package build system, maintainer scripts, or patches.

This attack is not equivalent to compromise of the uploader's OpenPGP key, which neither upload architecture defends against. Many Debian uploaders
build source packages on less-trusted systems where they also build and
test binary packages, and then sign the source package from a more-trusted system or use a hardware key.

With tag2upload, the construction of the source package is done by a host
that is similar in construction to a buildd, based on the contents of a
Git repository on Salsa. This forces the attacker to have access to push commits to Salsa and to risk detection from anyone watching the Salsa repository. Even if they later removed those commits from Salsa, the
commits are archived permanently on the dgit-repos Git server.

tag2upload is therefore superior in prevention, detection, and tracing
against attacks on uploader systems.

### Sponsored package upload

Debian relies on the sponsor's review to prevent upload of sponsored
packages that contain malicious code. This remains true in both upload architectures. However, the tag2upload protocol requires that all code in
the uploaded package be committed to Git, which provides more tools and opportunities for the sponsor to detect introduction of malicious code
before it is uploaded to the archive. The prevention benefits of using Git
are already available to sponsors if they require sponsored packages to be
in Git, so tag2upload does not offer new prevention capabilities for
sponsors, but to the extent that sponsors use this upload mechanism it standardizes the availability of those capabilities.

Should a malicious package uploaded via tag2upload later be detected, the
Git history provides better tracing of the malicious code than sponsored uploads done without Git.

tag2upload therefore nudges sponsors towards practices that provide better prevention and tracing.

### tag2upload infrastructure

The tag2upload design introduces a new trusted server that can be
attacked. This is the flip side of the previous point: moving source
package construction off the hosts of each individual uploader requires introducing a new component. If the tag2upload server is compromised, that access could be used to sign and upload malicious source packages that
would be accepted by the archive.

#### Conceptual analysis

This is the classic security trade-off of replacing distributed risk with centralized risk. Neither approach is inherently superior. The security analysis depends on an analysis of the nature of the risk.

A rule of thumb in such cases is to prefer distributed risk when defenses
are mutually reinforcing (sometimes called the Swiss cheese model). If hardening work done on one system prevents or detects compromises of other systems, the distributed nature of the risk can be a source of resiliency.
If, however, each distributed component fails independently and an
attacker only has to compromise one of them to achieve their objectives,
prefer centralizing that risk so that defensive resources can be
concentrated on securing one trusted system. This is often referred to as
"put all your eggs in one basket and protect that basket."

Source package construction follows the second pattern. In the current
upload model, each uploader system has to be protected independently, and
work done to protect one system doesn't prevent or detect compromises of a different system until the malicious package is already in the archive. It
is difficult to achieve consistent security across all uploader systems
due to Debian's highly distributed nature and (intentional) lack of any
central management of those systems. Debian can publish best-practice guidelines and provide better tools, but achieving a universal standard of security for uploader source package construction would be difficult.

My security recommendation in this case is therefore to centralize the
risk as much as possible, moving it off of individual uploader systems
with unknown security profiles and onto a central system that can be
analyzed and iteratively improved.

#### tag2upload server

The new tag2upload server architecture introduces a new type of build sandboxing that is similar but not identical to buildds (source package construction requires sufficient network access to Salsa, for example,
while buildds can be cut off from the network completely) and new code
that has to parse untrusted input.

The sandboxing design of the tag2upload server does a good job of reducing
that risk. Signatures are checked early, so only attackers able to create
a valid OpenPGP signature with a key in the keyring can attack the most security-sensitive part of the system. The signing key is isolated from
both the component that processes incoming requests from Salsa and the component that constructs the source package, only interacting with them
via a restricted protocol.

The best way to detect whether the tag2upload server has been compromised
would be to independently verify its output via a reproducible source
package construction system that starts from the same inputs, namely a
signed Git tag on a Salsa repository. This could be as simple as an
independent tag2upload server, or could involve auditing or independent reimplementation of the steps the tag2upload server performs.

We don't have reproducible source package builds today, so this is not a regression. We currently blindly trust whatever the uploader uploads, and
the tag2upload proposal does not make that risk worse, merely shifts it to central infrastructure. I therefore don't consider reproducible source
builds to be a security prerequisite for adoption of the tag2upload
proposal. It is, however, obvious follow-on work that would improve
detection of some classes of attacks.

The tag2upload server adds additional source package control fields that identify the signed Git tag on which the source package is based. To the
extent that uploaders use tag2upload, this provides a substantial
improvement in source package tracing compared to the current heuristics
used to associate a source package in the archive with the Git repository
that produced it.

#### Source package construction sandbox

Compromise of the tag2upload VM or schroot used for source package
construction would be roughly equivalent in impact to the compromise of an amd64 buildd: the tag2upload source package construction sandbox can cause malicious packages to be produced for all architectures, but compromising
the amd64 architecture alone would be sufficient to cause considerable
damage.

The sandbox requires limited access to the network, which means it is
slightly weaker than the isolation used for a buildd. An attacker may be
able to leverage malicious code in a Salsa repository that is being
uploaded into interactive access to the sandbox during construction of the source package for that repository. However, the sandbox is reset after
each operation, so the attacker will still have significant difficulty
using that access to compromise other source package builds. Leveraging
that access into a general tag2upload server compromise would require
finding an additional security vulnerability in the dgit rpush protocol or
in the sandboxing.

At present, a compromise of the amd64 buildd is easier to detect due to
the reproducible builds project, which may detect injected malicious code
as a regression in binary package reproducibility. As discussed above, the tag2upload construction of a source package from a signed Git tag should
also be reproducible, so a reproducible source package system would
improve detection to the same level or better than Debian's detection
abilities for binary buildds.

### Archive processing

Someone with administrative access to the archive processing machinery, including the archive signing key, could inject a malicious source (or
binary) package into the archive. This is not prevented by either the
current upload architecture or by tag2upload.

This attack on source packages can be detected by verifying the signatures
on all of the source packages in the archive. This remains true after the introduction of tag2upload; some of those source packages will be signed
by the tag2upload key instead of a maintainer key, but neither of those
keys are available to the archive processing machinery and are therefore equivalent when detecting this specific attack. Compromise of the
tag2upload server or its keys is discussed separately above.

tag2upload therefore does not change any security properties of archive processing.

### Salsa compromise

The tag2upload architecture does not generally rely on the security of
Salsa, and therefore compromising Salsa mostly does not make it easier
to upload a malicious source package. There are two exceptions:

- Administrative access to Salsa would make SHA-1 collision attacks
easier, as discussed below. However, this still assumes the attacker is
able to create Git trees with colliding hash digests.

- Security vulnerabilities in the Git client used by the tag2upload source
package construction sandbox could be exploited by a malicious Salsa Git
server to compromise the VM and introduce malicious code into the source
package it constructs. Since a malicious Git server could similarly be
used to compromise the systems of the numerous Debian contributors who
use Salsa via Git clients regularly, I don't believe this introduces
substantial new risk, but it does create a new avenue of attack that is
possibly less likely to be detected.

### Git object collisions

The current Git repository format and wire protocols use SHA-1 hash
digests (and only SHA-1 hash digests) to identify objects in the Git repository. Git uses a SHA-1 hash function that has been
[hardened against the SHAttered attack on SHA-1](https://github.com/cr-marcstevens/sha1collisiondetection),
and therefore is probably not vulnerable to known collision attacks. Given
the widespread use of Git, there is also a reasonable likelihood that any future attacks similar to SHAttered will receive prompt attention from Git maintainers and the broader open source community. However, SHA-1 is
considered cryptographically weak and future collision attacks that will
not be defeated by that hardening are possible.

Work to add SHA-256 support to Git is
[in progress](https://lwn.net/Articles/898522/) but is
[not yet supported by the software underlying Salsa](https://gitlab.com/groups/gitlab-org/-/epics/794).
I therefore also analyzed the behavior of the tag2upload protocol assuming
that an attacker could generate two Git trees that hash to the same SHA-1
hash digest, one benign and one malicious. I made the additional
assumption that the manipulation required to make the hashes collide could
be hidden in out-of-the-way places (test files, history) that would not be noticed by a reviewer.

I do not believe either of these assumptions are true currently or likely
to become true in the near future. My understanding is that hash collision attacks on Git repositories of the type required to attack tag2upload are currently entirely theoretical and are likely to remain so for some time
to come. This section is therefore a somewhat pedantic exercise in thoroughness, rather than an analysis of an attack I consider likely.

This analysis assumes the attacker has full access to the Salsa
repository to which the tag2upload tags are pushed, since this will be a
common scenario for sponsored uploads.

This analysis is relevant only for SHA-1-based Git repositories. Once
Salsa supports SHA-256 Git repositories, tag2upload could decline to act
on any repository that uses SHA-1 hash digests, making this entire section moot.

#### Replaying the tag

The attack: Construct a benign and malicious Git repository pair. Present
the benign repository to a sponsor for review. Once the sponsor has pushed
a signed tag for the benign repository, push the same tag to the malicious
Git repository, triggering tag2upload processing of that repository.

Since the signed tag that triggers tag2upload processing must specify both
the source package and version, the malicious repository must be for the
same source package and version. The attacker must therefore win a race
against the sponsor so that the malicious package is uploaded first.

This attack is noisy and thus vulnerable to detection. The sponsor would receive two tag2upload notifications and an error from whichever upload
lost the race. The notification for the malicious repository would also
include its Salsa URL, which would not match the Salsa URL that the
sponsor was expecting.

#### Moving the tag

The attack: As above, construct a benign and malicious Git repository pair
and get the benign repository signed by a sponsor. Race the tag2upload
server by deleting the signed tag from the benign repository, and then
push the same tag to the malicious Git repository.

This attack removes the doubled notification of the previous attack.
However, the race window is narrow, particularly if the attacker wants to
avoid an error notification to the sponsor from the tag2upload server
after it starts processing and then is unable to pull the signed tag from
the benign repository. Such an error would be suspicious and might lead to detection.

As with the previous attack, the tag2upload notification to the sponsor
would include the Salsa URL of the malicious repository rather than the
one that the sponsor signed, which increases the likelihood of detection.

#### Replacing the upstream tree

The attack: Construct a benign and malicious Git tree pair containing only
the upstream source. Reference the benign tree in a source package and get
that source package signed by a sponsor to trigger tag2upload processing.
Race the tag2upload server by deleting the upstream tag and commit ID and
then pushing the malicious Git repository as a new commit with the same
commit ID.

The upstream tag name is present in the signed tag metadata, but since
that tag itself is not required to be signed, the attacker can move it at
will. The upstream tag therefore provides no protection against this
attack apart from a small detection risk. Authentication of the upstream
tree comes only from the inclusion of its commit ID in the tag metadata.

I suspect (but am not certain) that this attack would normally be
prevented by the Salsa Git service. The benign tree already existed in the
same repository with the referenced commit ID (presumed to be checked by
the sponsor during review), and even if references to that object are
deleted via branch deletion, I believe Git will reject the push of the malicious commit ID until the old objects have been garbage-collected.
This presumably will take long enough that the tag2upload process will
fail because the upstream commit is missing.

This attack could be done by someone with administrative access to Salsa,
and thus in a position to force an immediate garbage collection of the unreferenced objects so that the tree underlying the upstream commit ID
can be replaced. Administrative access to Salsa would also make it trivial
to win the race against the tag2upload server. This attack is less prone
to detection than moving the tag to a different Salsa repository.

There is a variation on this attack where the attacker deletes the Git tag
and tree that it references, pushes a colliding tree, and then repushes
the Git tag. I believe this has essentially the same properties as the
above attack.

### Second-preimage attacks

An attacker could attempt to take the signature from a tag2upload tag
pushed to an arbitrary repository on Salsa and apply that same signature
to a Git tag for a malicious package on Salsa with the same version
number, triggering the tag2upload process. However, this requires
constructing a malicious repository with the same SHA-1 hash digest as the repository containing the original tag. This is a second-preimage attack
on SHA-1, which is believed to be currently infeasible. (Second-preimage attacks are believed to be currently infeasible even against MD5, which is
a much weaker hash function.) tag2upload therefore prevents this attack.

This same attack is could be tried against the existing upload mechanism
by attempting to reuse the signature of an upload changes control file published in the
[debian-devel-changes list archive](https://lists.debian.org/debian-devel-changes/).
The second-preimage resistance of the hash function used by the OpenPGP signature similarly prevents this attack.

### Hash weakness in the OpenPGP signature

The OpenPGP signature over a source package control file, upload changes control file, or Git tag also relies on a hash function to create the
digest that the public key signs.

To protect against collision attacks on the OpenPGP signature, both the existing upload system and tag2upload should require that the signature
use a strong hash function, such as SHA-256. It is not clear to me what
hash functions the current upload architecture permits; this may have
already been done there. This restriction is part of the tag2upload
design.

## Conclusions

The tag2upload architecture has some security advantages over the existing upload architecture:

- Source package construction for those who use tag2upload is sandboxed,
run on trusted systems, and less prone to variations or compromise from
differences in the local build environment on every Debian uploader's
personal systems.

- Each source package is associated with a Git repository and tag, which
are permanently archived on the dgit-repos Git server. This provides
more granular tracing information than retaining only the uploaded
artifacts, which may or may not be traceable to a VCS repository or tag.

- Sponsors are nudged towards reviewing sponsored packages in Git if they
wish to use the tag2upload service, which gives them more tools to
notice introduction of malicious code and prevent it from being
uploaded.

It introduces some new security risks, all of which I consider minor:

- The tag2upload server introduces another trusted component and some
additional attack surface. The additional risk seems manageable, but is
not zero. This centralizes risk that previously was distributed across
the systems of individual uploaders.

- The tag2upload protocol relies on collision resistance of SHA-1 hashes
of Git repository objects. Some attacks, particularly against a package
sponsorship workflow, are possible if an attacker can construct a benign
and malicious Git repository pair with the same SHA-1 hash digest. This
attack is at present theoretical and seems likely to remain very
challenging for some time to come.

I believe widespread adoption of tag2upload would represent a security improvement for Debian. The availability of a more secure source package construction system outweighs, in my opinion, the small additional risks
it would introduce. I do not believe it introduces any significant
security regressions.

Were tag2upload adopted, I would recommend some follow-on work:

- Verify that there are securely-archived backups of the dgit-repos Git
server, since they contain useful information for tracing any discovered
malicious packages.

- Set up a reproducible source package construction system that can
independently verify the construction of a source package from a signed
Git tag, which would provide additional assurance that the tag2upload
server has not been compromised. This may be as simple as running an
independent tag2upload server on separate infrastructure that would not
be compromised by a compromise of Debian project systems.

- Adopt SHA-256 support for Git in Salsa as soon as it is available.
Restrict tag2upload support to repositories using SHA-256 once the
support is mature.

The purpose of the tag2upload design is not purely to improve the security
of source package uploads. It is intended to enable a new workflow that
some Debian contributors may prefer. In most cases in software
architecture, the security cost of new features can be reduced but not eliminated entirely. There is some irreducible security risk from
introducing a new feature and thus new attack surface. Whether the risk is worth the benefit is not a decision that can be made by a security review.

--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2024-06-11 18:39:04, Russ Allbery wrote:

Hi all,

Below is the security review that I did of the tag2upload design.

Hi Russ, and thank you so much for taking the time to do this excellent
work. It's really comforting to think that we have an actual
professional look at our stuff, and I think we should do this more
often and systematically. :)

I am not a neutral party, in the sense that I think tag2upload is a good
idea and should be deployed. However, I do these types of security
reviews professionally, and I tried to approach this review the same way
that I would approach a major work project that needed a security review
to ensure we weren't deploying something with security issues. I
encourage any Debian community member with security expertise to check my work; with security reviews, the more eyes, the better.

For context, I am *not* a security professional (anymore?)... I don't
have formal security training, but I did work for Debian LTS for a while
and got involved in security audits and uploads in the past.

So, I guess I offer you my poorly adjusted eye-balls...

I guess I'm not a neutral party either, because I have Opinions, but I
guess then no one is neutral here, we'd need someone completely
disinterested in Debian to provide us with an opinion here if we would
want that, and even *that* wouldn't be quite neutral. ;)

I will also post this review on my web site, probably later tonight if I
have time.

I didn't find that post, btw... No big deal of course!

Anyways, here are my comments!

[...]

## Threat model

I evaluated both the existing source package upload architecture and the tag2upload architecture against the following threats:

- Someone not in the keyring uploads a malicious source package, possibly
via a sponsor.

- Someone in the keyring (either a Debian Developer or a Debian Maintainer
for a package) uploads a malicious source package but makes it appear
that the package was uploaded by someone else in the keyring.

- An attacker compromises the system a Debian uploader uses to build
source packages and uses that access to inject malicious code into a
source package.

- Someone with administrative access to the archive processing machinery
(DAK, the archive signing key, or similar infrastructure) uploads a
malicious source package.

- Someone with administrative access to the tag2upload server or its
signing key uploads a malicious source package.

- Someone with administrative access to Salsa uploads a malicious source
package.

In each case, I looked at prevention, detection, and tracing.

Neither the existing upload mechanism nor tag2upload attempt to prevent or detect (as opposed to trace) the upload of a malicious source package by someone in full possession of a key in the keyring, so this threat is not considered in this document, although tracing for this threat is
discussed briefly.

I'm actually curious as to why that is treated as a separate
possibility, because if kind of overlaps with the second model ("someone uploads a malicious package appearing from someone else")...

For me, that case and the "xz-utils" case are actually quite pressing
matters, they don't quite keep me up at night, but they're the kind of
threat models I do worry about and that we should address head on. But
*maybe* this is not the right vector to address them, that said...

## Brief architecture summary

[...]

### tag2upload

tag2upload replaces the first step of this upload process with the
following:

1. The uploader pushes a signed tag in a specific format to Salsa. For
non-native packages, this may reference an upstream tree in the same
Git repository by commit ID, which will be used to create the `orig`
tar file if needed.

2. Salsa notifies a web hook on a secure project-maintained system that a
new tag of interest has been pushed.

3. That system (with internal privilege separation) retrieves the Git tag
and corresponding commit, verifies the signature and tag metadata, and
verifies that the signer is in the relevant keyring.

4. Inside a VM or schroot, that system retrieves the Git tree and upstream
source tree if applicable, constructs or retrieves the `orig` tar file,
and constructs the Debian source package and source package control
file. This VM or schroot in essence operates as a source package
buildd.

5. The tag2upload server adds control header fields specifying the Git
object ID and the identity string and fingerprint of the uploader,
signs the resulting source package control file, constructs an upload
changes control file, signs it, and creates and signs another Git tag
reflecting any additional Git commits that were required to put the
repository into a canonical format (the "dgit view").

6. The tag2upload server pushes the original Git tag, its referenced tree,
and the additional "dgit view" tag to the publicly-accessible
dgit-repos Git server as a permanent archive.

7. The tag2upload server uploads the signed source package to the normal
archive incoming queue.

Subsequent processing of the upload happens identically to the existing upload system.

Nice. I didn't realize Salsa was involved in tag2upload, it answers part
of my question in the other thread of "can i keep using salsa", so
that's actually quite nice!

Thanks for that neat overview!

## Analysis

[...]

#### tag2upload server

The new tag2upload server architecture introduces a new type of build sandboxing that is similar but not identical to buildds (source package construction requires sufficient network access to Salsa, for example,
while buildds can be cut off from the network completely) and new code
that has to parse untrusted input.

The sandboxing design of the tag2upload server does a good job of reducing that risk. Signatures are checked early, so only attackers able to create
a valid OpenPGP signature with a key in the keyring can attack the most security-sensitive part of the system. The signing key is isolated from
both the component that processes incoming requests from Salsa and the component that constructs the source package, only interacting with them
via a restricted protocol.

This is more a question to the dgit people, but what kind of hardening
do we have on the tag2upload server? I think dak has the cryptographic
keys in a HSM (Hardware Security Module) to prevent a threat actor from grabbing those keys for offline attacks...

Is there something similar (HSM or YubiKey) on the tag2upload server? If
not, why?

The best way to detect whether the tag2upload server has been compromised would be to independently verify its output via a reproducible source
package construction system that starts from the same inputs, namely a
signed Git tag on a Salsa repository. This could be as simple as an independent tag2upload server, or could involve auditing or independent reimplementation of the steps the tag2upload server performs.

We don't have reproducible source package builds today, so this is not a regression. We currently blindly trust whatever the uploader uploads, and
the tag2upload proposal does not make that risk worse, merely shifts it to central infrastructure. I therefore don't consider reproducible source
builds to be a security prerequisite for adoption of the tag2upload
proposal. It is, however, obvious follow-on work that would improve
detection of some classes of attacks.

Does tag2upload make reproducible source packages harder?

[...]

#### Replacing the upstream tree

The attack: Construct a benign and malicious Git tree pair containing only the upstream source. Reference the benign tree in a source package and get that source package signed by a sponsor to trigger tag2upload processing. Race the tag2upload server by deleting the upstream tag and commit ID and then pushing the malicious Git repository as a new commit with the same commit ID.

The upstream tag name is present in the signed tag metadata, but since
that tag itself is not required to be signed, the attacker can move it at will. The upstream tag therefore provides no protection against this
attack apart from a small detection risk. Authentication of the upstream
tree comes only from the inclusion of its commit ID in the tag metadata.

I just wanted to state that this is a really nice attack, good modeling
there, I didn't think of that one!

I suspect (but am not certain) that this attack would normally be
prevented by the Salsa Git service. The benign tree already existed in the same repository with the referenced commit ID (presumed to be checked by
the sponsor during review), and even if references to that object are
deleted via branch deletion, I believe Git will reject the push of the malicious commit ID until the old objects have been garbage-collected.
This presumably will take long enough that the tag2upload process will
fail because the upstream commit is missing.

Yeah, that's a reasonable assumption, but I believe those jobs are ran
on a schedule on GitLab servers, so an attacker could *time* their
attack just so, to make sure the old tree gets GC'd just in time. It's a
heck of a race to win though, especially, since you need to time it on
the other side as well...

This attack could be done by someone with administrative access to Salsa,
and thus in a position to force an immediate garbage collection of the unreferenced objects so that the tree underlying the upstream commit ID
can be replaced. Administrative access to Salsa would also make it trivial
to win the race against the tag2upload server. This attack is less prone
to detection than moving the tag to a different Salsa repository.

That too, of course...

There is a variation on this attack where the attacker deletes the Git tag and tree that it references, pushes a colliding tree, and then repushes
the Git tag. I believe this has essentially the same properties as the
above attack.

... and probably just easier.

[...]

I wonder if you've considered the "we need to revoke access to compromised/hostile developer" threat model. Right now, we have a
relatively centralized model here (modulo DAM, dak, debian-keyring), and
we're introducing a new component... How does tag2upload manage keys and
does it introduce additional response time or issues when revoking
access to retiring or revoked developers?

## Conclusions

[...]

I believe widespread adoption of tag2upload would represent a security improvement for Debian. The availability of a more secure source package construction system outweighs, in my opinion, the small additional risks
it would introduce. I do not believe it introduces any significant
security regressions.

I agree with this assessment.

Were tag2upload adopted, I would recommend some follow-on work:

- Verify that there are securely-archived backups of the dgit-repos Git
server, since they contain useful information for tracing any discovered
malicious packages.

Having uploads in Git brings a whole set of interesting properties and
tools we could leverage there as well, to ensure the integrity of the
dgit repository itself. When Tor transitioned from gitolite to GitLab,
one of the concerns was exactly that kind of problem space where we're
not sure we want to trust GitLab with our code. So I did a significant
amount of work researching Git integrity solutions, and my findings are documented here:

https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/gitlab#git-repository-integrity-solutions

The work the kernel.org folks have been doing about publishing a
transparency log for the kernel git repository might be particularly
relevant here.

A.
--
May your trails be crooked, winding, lonesome, dangerous, leading to
the most amazing view. May your mountains rise into and above the
clouds.
- Edward Abbey

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, 12 Jun 2024 at 13:03:04 -0400, Antoine Beaupré wrote:

On 2024-06-11 18:39:04, Russ Allbery wrote:

- Someone in the keyring (either a Debian Developer or a Debian Maintainer
for a package) uploads a malicious source package but makes it appear
that the package was uploaded by someone else in the keyring.

...

Neither the existing upload mechanism nor tag2upload attempt to prevent or detect (as opposed to trace) the upload of a malicious source package by someone in full possession of a key in the keyring, so this threat is not considered in this document, although tracing for this threat is
discussed briefly.

I'm actually curious as to why that is treated as a separate
possibility, because if kind of overlaps with the second model ("someone uploads a malicious package appearing from someone else")...

Using "victim" as a shorter name for the "someone else" that the attacker
wants to blame for the upload:

For the "makes it appear that..." scenario, the upload would have the
victim's name in all of its non-cryptographic metadata (debian/changelog,
git tag GIT_COMMITTER_NAME, etc.), but it would have to be signed by
the attacker's key - because the threat model in this scenario is that
the attacker's own private key is the only one in the keyring that they
have access to, so they can't sign it with the victim's key, or with
some third developer's key.

(Or I suppose the attacker could also generate a new keypair under their control and sign with that, but it seems fairly obvious that it would
be rejected, because tag2upload wouldn't find it in the keyring -
hopefully there is a test-case for that!)

For the "full possession of a key" scenario, the threat model is that the attacker *would* have access to the victim's private key, and therefore a rational attacker would validly sign the upload with that, making it indistinguishable from a legitimate upload by the victim (except for the
fact that, afterwards, the victim would hopefully say "wait, I didn't
upload that?" and start raising the alarm).

I think that's the intended difference here?

For me, that case and the "xz-utils" case are actually quite pressing
matters

The bottom line is that if the attacker has the victim's private key
material, then the attacker can do anything that the victim can do,
because in our key-based security model the only way we can authenticate
the victim remotely is that they have control of their own private key(s),
and the attacker (we assume) doesn't.

smcv

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Antoine Beaupré <anarcat@debian.org> writes:

On 2024-06-11 18:39:04, Russ Allbery wrote:

Below is the security review that I did of the tag2upload design.

Hi Russ, and thank you so much for taking the time to do this excellent
work. It's really comforting to think that we have an actual
professional look at our stuff, and I think we should do this more often
and systematically. :)

Thank you, that's very kind!

I will also post this review on my web site, probably later tonight if
I have time.

I didn't find that post, btw... No big deal of course!

Ah, yes, indeed, I once again overestimated the amount of time I'd have available in the evening and used it all responding to the thread instead. Maybe tonight?

[... lots of trimming throughout ...]

Neither the existing upload mechanism nor tag2upload attempt to prevent
or detect (as opposed to trace) the upload of a malicious source
package by someone in full possession of a key in the keyring, so this
threat is not considered in this document, although tracing for this
threat is discussed briefly.

I'm actually curious as to why that is treated as a separate
possibility, because if kind of overlaps with the second model ("someone uploads a malicious package appearing from someone else")...

I think it's important to note that, in both the old and new system, if
someone compromises a Debian uploader's key, they can upload malicious
packages signed with that key and we have no systematic prevention or
detection for that. It's a bound on the security problem that we're
trying to solve. We're still trusting the OpenPGP keys of people with
upload rights.

The second model is the impersonation case: while someone with a
compromised key is going to be able to upload packages signed with that
key, they shouldn't be able to hide by making the packages appear to be
signed with a different key. This is important for tracing the damage
once we realize the key has been compromised, and also for ensuring authorization checks are done correctly in the Debian Maintainer case.

For me, that case and the "xz-utils" case are actually quite pressing matters, they don't quite keep me up at night, but they're the kind of
threat models I do worry about and that we should address head on. But *maybe* this is not the right vector to address them, that said...

I think tag2upload is not quite the right vector to address that specific problem, although it provides some assistance. In that case, upstream
injected malicious code into the upstream release. To the extent that the Debian uploader trusts that code and does not detect that it's malicious,
we're going to continue to be vulnerable to that attack. It is extremely difficult to prevent entirely.

tag2upload offers some benefit here in that it forces all of the code into
Git. That does not make it any less malicious, obviously, but it *might*
make it easier to detect, because diffs in Git are somewhat easier to
analyze than tarballs.

That depends entirely on how much the maintainer audits the upstream code, though. If the maintainer blindly imports upstream code into the Git repository and then builds and uploads a source package (the current
system) or signs a Git tag of that code (the tag2upload system), we're
still going to get malicious code in the archive. And I have certainly
been that maintainer at some points in the past. There's a lot of work to
do in Debian and it is extremely hard to keep up while still reviewing and understanding every line of upstream's changes.

The sandboxing design of the tag2upload server does a good job of
reducing that risk. Signatures are checked early, so only attackers
able to create a valid OpenPGP signature with a key in the keyring can
attack the most security-sensitive part of the system. The signing key
is isolated from both the component that processes incoming requests
from Salsa and the component that constructs the source package, only
interacting with them via a restricted protocol.

This is more a question to the dgit people, but what kind of hardening
do we have on the tag2upload server? I think dak has the cryptographic
keys in a HSM (Hardware Security Module) to prevent a threat actor from grabbing those keys for offline attacks...

Is there something similar (HSM or YubiKey) on the tag2upload server? If
not, why?

I believe that using an external hardware key for the tag2upload signing
key is indeed the plan.

We don't have reproducible source package builds today, so this is not
a regression. We currently blindly trust whatever the uploader uploads,
and the tag2upload proposal does not make that risk worse, merely
shifts it to central infrastructure. I therefore don't consider
reproducible source builds to be a security prerequisite for adoption
of the tag2upload proposal. It is, however, obvious follow-on work that
would improve detection of some classes of attacks.

Does tag2upload make reproducible source packages harder?

I personally have not done any work on reproducible source packages and therefore do not know the exact shape of the problem or the unexpected pitfalls, but just intuitively I believe tag2upload would make
reproducible source packages easier for those packages that are uploaded
via tag2upload. tag2upload builds the source package in a well-known, controlled, reproducible environment with known tool versions and a
specific implementation of the process for turning a Git repository into a source package. One should therefore be able to get a long way towards reproducing those source packages by starting with a second tag2upload
server with the same code but running in a separate security domain.

My understanding is that variations in the output of git archive, tar, and compression programs mean that reproducible source packages are probably
best approached by trying to make the contents of the tarball reproducible rather than the compressed tarball itself. This does miss some amount of attack surface, though. This seems to be a general problem that's mostly orthogonal to tag2upload. If anything, I'd guess the tag2upload process
would make extending reproducibility to the compressed tarball itself
somewhat easier since it can record precise tool versions. But that may
not be enough if the same version of the tool can produce different output given the same Git tree.

I suspect (but am not certain) that this attack would normally be
prevented by the Salsa Git service. The benign tree already existed in
the same repository with the referenced commit ID (presumed to be
checked by the sponsor during review), and even if references to that
object are deleted via branch deletion, I believe Git will reject the
push of the malicious commit ID until the old objects have been
garbage-collected. This presumably will take long enough that the
tag2upload process will fail because the upstream commit is missing.

Yeah, that's a reasonable assumption, but I believe those jobs are ran
on a schedule on GitLab servers, so an attacker could *time* their
attack just so, to make sure the old tree gets GC'd just in time. It's a
heck of a race to win though, especially, since you need to time it on
the other side as well...

I was hoping that the Salsa scheduled cleanup tasks would use git gc with close-to-default options, which among other things means a prune threshold
of two weeks. I am not an expert in the Git storage format, but I believe
that means unreferenced objects would still be retained for two weeks and
thus would continue to block a push of new objects with the same SHA-1
hash for that period of time, which should be more than enough time to
prevent this race.

I wonder if you've considered the "we need to revoke access to compromised/hostile developer" threat model. Right now, we have a
relatively centralized model here (modulo DAM, dak, debian-keyring), and we're introducing a new component... How does tag2upload manage keys and
does it introduce additional response time or issues when revoking
access to retiring or revoked developers?

The intent of the design is that the authorization checks done in
tag2upload are for its own security boundary and are redundant with checks performed by dak, with the exception of verifying the Git tag signature
and recording the fingerprint of the signed key. (That necessarily has to
be done by the tag2upload server.)

Therefore, in the case you describe, even if tag2upload was slow in
receiving an updated keyring and thus accepted a package from someone
whose access was revoked, the only thing that person could do is induce tag2upload to construct the source package and pass it to dak. dak would
then retrieve the key fingerprint from the source package metadata,
perform its normal authorization checks, see that the uploader's access
has been revoked, and reject the package.

Rejected packages are still recorded in the dgit-repos archive server by
design (this is true of how it's being used today with dgit), so after a package reject the source version will have to be changed before it can be uploaded again.

Having uploads in Git brings a whole set of interesting properties and
tools we could leverage there as well, to ensure the integrity of the
dgit repository itself. When Tor transitioned from gitolite to GitLab,
one of the concerns was exactly that kind of problem space where we're
not sure we want to trust GitLab with our code. So I did a significant
amount of work researching Git integrity solutions, and my findings are documented here:

https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/gitlab#git-repository-integrity-solutions

The work the kernel.org folks have been doing about publishing a
transparency log for the kernel git repository might be particularly
relevant here.

This is a great analysis. Thank you!

git-evtag seems like the system that's the most relevant to the specific tag2upload problem, although it has the substantial drawback of requiring
a non-standard tool and tag format.

--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

I will write a more detailed response to Russ's analysis later.
I am behind on getting my packages into shape and I want to concentrate
on that for now.
I do agree with Russ's basic conclusion: we should decide whether to
adopt tag2upload for reasons other than security of the architecture.


Simon> For the "full possession of a key" scenario, the threat model
Simon> is that the attacker *would* have access to the victim's
Simon> private key, and therefore a rational attacker would validly
Simon> sign the upload with that, making it indistinguishable from a
Simon> legitimate upload by the victim
I think this is true for traditional uploads but not for tag2upload.

>> For me, that case and the "xz-utils" case are actually quite
>> pressing matters

Simon> The bottom line is that if the attacker has the victim's
Simon> private key material, then the attacker can do anything that
Simon> the victim can do, because in our key-based security model
Simon> the only way we can authenticate the victim remotely is that
Simon> they have control of their own private key(s),

We also have credentials that are used to log into salsa.
So, for tag2upload, we might find that we had additional mechanisms to
trace an attack because gitlab keeps track of which authenticated user
made a particular push.

I appreciate that in most cases the signatures are stronger, more
distributed credentials than gitlab's logs of who does a push.
But they are parallel mechanisms and a discrepancy between these can
help us detect and trace attacks.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi,

On 6/13/24 22:27, Simon Josefsson wrote:

Generally I reach the same conclusion, although I think there are real security problems with both the existing and the proposed tag2upload mechanism that we should all be aware of. It is acceptable to realize
that we cannot protect against all attacks with reasonable costs.

In that case it is kind of disingenuous to highlight the necessity of
this change by pointing at the xz-utils scenario.

Simon

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Russ Allbery <rra@debian.org> writes:

The decision on whether to adopt tag2upload should be made primarily on non-security grounds.

Generally I reach the same conclusion, although I think there are real
security problems with both the existing and the proposed tag2upload
mechanism that we should all be aware of. It is acceptable to realize
that we cannot protect against all attacks with reasonable costs.
That's why we need the ability to transparently audit all steps, to
detect them when they occur. Reversely: it would be unfortunate to say
no to new functionality because the new functionality don't solve all
possible problems. That just stalls progress.

## Threat model

I evaluated both the existing source package upload architecture and the tag2upload architecture against the following threats:

- Someone not in the keyring uploads a malicious source package, possibly
via a sponsor.

- Someone in the keyring (either a Debian Developer or a Debian Maintainer
for a package) uploads a malicious source package but makes it appear
that the package was uploaded by someone else in the keyring.

- An attacker compromises the system a Debian uploader uses to build
source packages and uses that access to inject malicious code into a
source package.

- Someone with administrative access to the archive processing machinery
(DAK, the archive signing key, or similar infrastructure) uploads a
malicious source package.

- Someone with administrative access to the tag2upload server or its
signing key uploads a malicious source package.

- Someone with administrative access to Salsa uploads a malicious source
package.

Having a threat model is great. I find the notion of "uploads a source package" is poorly defined here though.

What threat model of those (if any) cover the situation were someone in
the keyring uploads a (benign) source package and something on Debian's
side (e.g., design of tag2upload) enables an attacker to substitute some
part of the intended upload with something malicious?

With tag2upload, I don't think we can reasonable talk about "upload a
source package" any more. What would you define that to actually mean?
Maybe I'm missing some introductionary documentation here.

### Git object collisions

The current Git repository format and wire protocols use SHA-1 hash
digests (and only SHA-1 hash digests) to identify objects in the Git repository. Git uses a SHA-1 hash function that has been
[hardened against the SHAttered attack on SHA-1](https://github.com/cr-marcstevens/sha1collisiondetection),
and therefore is probably not vulnerable to known collision attacks.

Can this be substantiated? Using SHA1CD in Git does not necessarily
mean someone cannot manually create a Git repository with a colliding
git commit somewhere in the history that gets accepted by git, and
allows someone to replace actual file contents. That may be the case,
but I haven't seen any detailed analysis answering that.

This analysis is relevant only for SHA-1-based Git repositories. Once
Salsa supports SHA-256 Git repositories, tag2upload could decline to act
on any repository that uses SHA-1 hash digests, making this entire section moot.

I don't think it will be as simple as that: the git SHA256 transition
documents suggests to me that even signed tags may refer to both SHA256
and SHA1 commits:

https://git-scm.com/docs/hash-function-transition#_signed_tags

Thus tag2upload would need to require 1) SHA256 Git repository support,
AND 2) that git tags refer to a SHA256 commit id, AND 3) any git
submodules used also rely on SHA256 rather than SHA1.

#### Replacing the upstream tree

The attack: Construct a benign and malicious Git tree pair containing only the upstream source. Reference the benign tree in a source package and get that source package signed by a sponsor to trigger tag2upload processing. Race the tag2upload server by deleting the upstream tag and commit ID and then pushing the malicious Git repository as a new commit with the same commit ID.

I think this is an important and realistic attack vector that we
shouldn't be vulnerable to.

The upstream tag name is present in the signed tag metadata, but since
that tag itself is not required to be signed, the attacker can move it at will. The upstream tag therefore provides no protection against this
attack apart from a small detection risk. Authentication of the upstream
tree comes only from the inclusion of its commit ID in the tag metadata.

Which is SHA1 currently, and thus vulnerable to a collision attack,
which are known to be possible.

I suspect (but am not certain) that this attack would normally be
prevented by the Salsa Git service. The benign tree already existed in the same repository with the referenced commit ID (presumed to be checked by
the sponsor during review), and even if references to that object are
deleted via branch deletion, I believe Git will reject the push of the malicious commit ID until the old objects have been garbage-collected.
This presumably will take long enough that the tag2upload process will
fail because the upstream commit is missing.

This attack could be done by someone with administrative access to Salsa,
and thus in a position to force an immediate garbage collection of the unreferenced objects so that the tree underlying the upstream commit ID
can be replaced. Administrative access to Salsa would also make it trivial
to win the race against the tag2upload server. This attack is less prone
to detection than moving the tag to a different Salsa repository.

There is a variation on this attack where the attacker deletes the Git tag and tree that it references, pushes a colliding tree, and then repushes
the Git tag. I believe this has essentially the same properties as the
above attack.

Is Salsa admin access necessary? Isn't 'Maintainer' access sufficient?

To me, the protection against this attack seems weak, and I don't really
see any strong protection against it. Couldn't we require that the
signed tag2upload tag comment contains a hash of the upstream code,
somehow?

### Second-preimage attacks

An attacker could attempt to take the signature from a tag2upload tag
pushed to an arbitrary repository on Salsa and apply that same signature
to a Git tag for a malicious package on Salsa with the same version
number, triggering the tag2upload process. However, this requires constructing a malicious repository with the same SHA-1 hash digest as the repository containing the original tag. This is a second-preimage attack
on SHA-1, which is believed to be currently infeasible. (Second-preimage attacks are believed to be currently infeasible even against MD5, which is
a much weaker hash function.) tag2upload therefore prevents this attack.

This same attack is could be tried against the existing upload mechanism
by attempting to reuse the signature of an upload changes control file published in the
[debian-devel-changes list archive](https://lists.debian.org/debian-devel-changes/).
The second-preimage resistance of the hash function used by the OpenPGP signature similarly prevents this attack.

Is a second preimage really required to mount this attack? Consider if
someone creates a collision for a good and a bad version, gets the good
version signed by a sponsor, and then re-use that signature for the bad version.

/Simon

--=-=-Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iIoEARYIADIWIQSjzJyHC50xCrrUzy9RcisI/kdFogUCZmrzzhQcc2ltb25Aam9z ZWZzc29uLm9yZwAKCRBRcisI/kdFogDDAQCqEEykYEA+bqfD80xZRaausPJmOzp8 2ajb67egNG5c/wD8Dc/7pdaHglQ1eAhBc4t/mfgob1mybYzK8zL1PzhF9wU=PN5R
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Simon Richter <sjr@debian.org> writes:

Hi,

On 6/13/24 22:27, Simon Josefsson wrote:

Generally I reach the same conclusion, although I think there are real
security problems with both the existing and the proposed tag2upload
mechanism that we should all be aware of. It is acceptable to realize
that we cannot protect against all attacks with reasonable costs.

In that case it is kind of disingenuous to highlight the necessity of
this change by pointing at the xz-utils scenario.

Agreed. I don't think tag2upload solves anything important from a
security point of view. I believe tag2upload will enable new attacks,
some attacks that are realistic and will actually occur. Still I find
myself in mild support of tag2upload, since it enables a workflow that
some people seems to prefer. IMHO, that's the important aspect.

Excluding people's reasonable positions has demotivated Debian
contributors historically, and I don't think the project or resulting
release artifact is significantly better off as a result. Using Devuan
to avoid systemd, or Trisquel to avoid non-free software, is poor human resource utilization.

Git-based workflows seems popular. If there is some method to support
it (tag2upload), and there are people willing to baby-sit the
implementation (I dunno but assume so), and it doesn't break existing
workflows (I dunno), then my opinion is: why not.

But, please, don't hype this as a solution to xz-utils problems. The ftpmaster's conservative response is reasonable, and there are many
unanswered questions about tag2upload, and it is easy to shoot it down
on those grounds. It would make the case stronger to admit that there
are unanswered questions, and would invite collaborative work to improve
the design.

/Simon

-----BEGIN PGP SIGNATURE-----

iIoEARYIADIWIQSjzJyHC50xCrrUzy9RcisI/kdFogUCZmsJNBQcc2ltb25Aam9z ZWZzc29uLm9yZwAKCRBRcisI/kdFolO4AP92OsYnwSwQg3MjVKVLRA/qyd6eCyrs j8iCBz66mQ6GnAEAzjZgJ6YgpGueVeyjN92AcBDUwTlpC/W6uAaR9qBgLwE=
=Uol9
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

simon@josefsson.org wrote:

Can this be substantiated? Using SHA1CD in Git does not necessarily
mean someone cannot manually create a Git repository with a colliding
git commit somewhere in the history that gets accepted by git, and
allows someone to replace actual file contents. That may be the case,
but I haven't seen any detailed analysis answering that.

This is quite a strong assertion, and it is up to you to prove it.
The current consensus among cryptography experts is that SHA-1 is still resistant to preimage attacks.

--
ciao,
Marco

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Marco d'Itri <md@Linux.IT> writes:

simon@josefsson.org wrote:

Can this be substantiated? Using SHA1CD in Git does not necessarily
mean someone cannot manually create a Git repository with a colliding
git commit somewhere in the history that gets accepted by git, and
allows someone to replace actual file contents. That may be the case,
but I haven't seen any detailed analysis answering that.

This is quite a strong assertion, and it is up to you to prove it. The current consensus among cryptography experts is that SHA-1 is still
resistant to preimage attacks.

The attack that Simon is talking about doesn't require a preimage attack,
only a successful collision attack against Git trees using SHAttered plus
some assumptions about where Git may be lazy about revalidating hashes.
It's an interesting point that I didn't think of, although I'm not sure
that it would work against GitLab and thus against Salsa and I think it's fairly trivial to protect against regardless. I'm working on a longer response; I needed to do a bit of research first.

--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Thank you very much for this review! You were one of the people I most
wanted to hear from, since I know you have substantial expertise in the cryptography parts of this and a lot of security experience. I know you
also had concerns in the past about hash collisions specifically.

Simon Josefsson <simon@josefsson.org> writes:

Generally I reach the same conclusion, although I think there are real security problems with both the existing and the proposed tag2upload mechanism that we should all be aware of.

Yes. tag2upload doesn't fundamentally change the security model of
uploads; most flaws that we currently have we still have with tag2upload.

It is acceptable to realize that we cannot protect against all attacks
with reasonable costs. That's why we need the ability to transparently
audit all steps, to detect them when they occur. Reversely: it would be unfortunate to say no to new functionality because the new functionality don't solve all possible problems. That just stalls progress.

Wholeheartedly agreed.

## Threat model

I evaluated both the existing source package upload architecture and the
tag2upload architecture against the following threats:

- Someone not in the keyring uploads a malicious source package, possibly
via a sponsor.

- Someone in the keyring (either a Debian Developer or a Debian Maintainer >> for a package) uploads a malicious source package but makes it appear
that the package was uploaded by someone else in the keyring.

- An attacker compromises the system a Debian uploader uses to build
source packages and uses that access to inject malicious code into a
source package.

- Someone with administrative access to the archive processing machinery
(DAK, the archive signing key, or similar infrastructure) uploads a
malicious source package.

- Someone with administrative access to the tag2upload server or its
signing key uploads a malicious source package.

- Someone with administrative access to Salsa uploads a malicious source
package.

Having a threat model is great. I find the notion of "uploads a source package" is poorly defined here though.

Yes, I used this sloppily. In some places I mean uploads the package to
dak, in some places I mean pushed a signed Git tag, and in other places I
mean introduces the package into the archive. I'll try to find some time
to tighten up the wording to be a bit clearer about which specific action
I'm talking about in each case.

What threat model of those (if any) cover the situation were someone in
the keyring uploads a (benign) source package and something on Debian's
side (e.g., design of tag2upload) enables an attacker to substitute some
part of the intended upload with something malicious?

It's effectively equivalent to:

- Someone with administrative access to the tag2upload server or its
signing key uploads a malicious source package.

and is discussed at some length in the corresponding section, but I should
call that out as a separate threat.

### Git object collisions

The current Git repository format and wire protocols use SHA-1 hash
digests (and only SHA-1 hash digests) to identify objects in the Git
repository. Git uses a SHA-1 hash function that has been
[hardened against the SHAttered attack on SHA-1](https://github.com/cr-marcstevens/sha1collisiondetection),
and therefore is probably not vulnerable to known collision attacks.

Can this be substantiated? Using SHA1CD in Git does not necessarily
mean someone cannot manually create a Git repository with a colliding
git commit somewhere in the history that gets accepted by git, and
allows someone to replace actual file contents. That may be the case,
but I haven't seen any detailed analysis answering that.

This was a really interesting point that I didn't catch. Thank you! Let
me try to rephrase this in the form of an attack and see if this captures
what you were getting at.

The attack: Using a pre-SHA-1DC version of Git, construct a benign and malicious pair of Git trees that diverge at some point by abusing the hash
of an object vulnerable to SHAttered. Push the benign tree to Salsa,
relying on Salsa not reverifying the hashes of new objects with a hardened hash, or alternately have already planted the benign tree in a Git
repository imported into Salsa before SHA-1DC was in use. Get that tree
signed by a sponsor, again relying on the sponsor's git client not
revalidating object hashes, and then follow the same attack pattern in
either "Moving the tag" or "Replacing the upstream tree." Rely on the tag2upload server not reverifying the hashes of the Git tree when it pulls
it to construct the signed source package.

In essence, this attack exploits the fact that Git is lazy about
performing hashes and usually only does so when it has to. I'm not sure
this assumption is correct for Salsa in particular, but it's at least plausible. The trees used in this attack would fail git fsck, because the critical object would hash to a different value using SHA-1DC than it does
with SHA-1, but it's not clear that git fsck is called at any of the
points that would detect this attack.

I believe this attack would be prevented by setting transfer.fsckObjects
to true in the Git configuration of the tag2upload worker and failing the operation if it detects anything. (Or, equivalently, calling git fsck
after git clone and failing on any detected problems.) I believe this
forces recomputation of the hashes of all received objects. The object
used in this attack would fail that hash recomputation because the
tag2upload server would use a version of Git that uses SHA-1DC. The cost
is a performance penalty on git clone, which would be trivial for most repositories but which might be noticable for particularly large Git
trees.

My personal opinion is that always setting transfer.fsckObjects to true is
good practice anyway to catch more banal problems such as disk corruption
and memory bit flips, so while I'm not sure I would bother just for this attack, it might be a good idea on general principles.

This analysis is relevant only for SHA-1-based Git repositories. Once
Salsa supports SHA-256 Git repositories, tag2upload could decline to
act on any repository that uses SHA-1 hash digests, making this entire
section moot.

I don't think it will be as simple as that: the git SHA256 transition documents suggests to me that even signed tags may refer to both SHA256
and SHA1 commits:

https://git-scm.com/docs/hash-function-transition#_signed_tags

Thus tag2upload would need to require 1) SHA256 Git repository support,
AND 2) that git tags refer to a SHA256 commit id, AND 3) any git
submodules used also rely on SHA256 rather than SHA1.

This is what I meant by "decline to act on any repository that uses SHA-1
hash digests." I didn't get into the details because this isn't something
that we can adopt currently, so working out the specifics seemed
premature. Once this is an option, we should figure out a transition
plan.

#### Replacing the upstream tree

The attack: Construct a benign and malicious Git tree pair containing
only the upstream source. Reference the benign tree in a source package
and get that source package signed by a sponsor to trigger tag2upload
processing. Race the tag2upload server by deleting the upstream tag
and commit ID and then pushing the malicious Git repository as a new
commit with the same commit ID.

I think this is an important and realistic attack vector that we
shouldn't be vulnerable to.

For the record, I am still dubious that this is that important of an
attack vector. Even using SHAttered to create conflicting Git trees is
quite difficult; the mere presence of a hash collision is not enough to
attack Git in a way that results in a creation of a malicious binary
package. Git objects have structure and hash collisions require hiding attacker-generated data in that structure, which is not trivial. And a successful attack on Git does not automatically mean that tag2upload is vulnerable, since using a centralized Git server makes the attack more difficult.

I'm also dubious that Debian would see a zero-day attack with a novel
SHA-1 collision. It's more likely that, as with SHAttered, we would get substantial advance warning of a new SHA-1 collision before an attack on
Git specifically was feasible. We are far from the only entity that
relies on Git security, and there are significant resources behind the Git community, so I think the most likely outcome of a new SHA-1 collision
would be something similar to what happened with SHAttered.

I agree that this attack is theoretically possible, and I agree that the
longer we go on using SHA-1 as a basis for Git trees, the more likely the attack becomes. But there are substantial obstacles in the way, and I
think we're still some years away from the point where I would lose sleep worrying about this.

Or, put another way, if this were anywhere close to the easiest way to
attack Debian, we would be doing astonishingly well in our security
posture.

I'm still talking about it here because I do want to be thorough, and I'm
in favor of protecting against unlikely attacks if one can do so easily.
There were a few minor tweaks to the design that I already suggested and
that were accepted which make the detection story for hash collisions
better. But since you specifically said that the attack was important, I
did want to push back on that a bit.

The upstream tag name is present in the signed tag metadata, but since
that tag itself is not required to be signed, the attacker can move it
at will. The upstream tag therefore provides no protection against this
attack apart from a small detection risk. Authentication of the
upstream tree comes only from the inclusion of its commit ID in the tag
metadata.

Which is SHA1 currently, and thus vulnerable to a collision attack,
which are known to be possible.

Correct.

This attack could be done by someone with administrative access to
Salsa, and thus in a position to force an immediate garbage collection
of the unreferenced objects so that the tree underlying the upstream
commit ID can be replaced. Administrative access to Salsa would also
make it trivial to win the race against the tag2upload server. This
attack is less prone to detection than moving the tag to a different
Salsa repository.

There is a variation on this attack where the attacker deletes the Git
tag and tree that it references, pushes a colliding tree, and then
repushes the Git tag. I believe this has essentially the same
properties as the above attack.

Is Salsa admin access necessary? Isn't 'Maintainer' access sufficient?

I dug into this a bit, and I believe GitLab imposes a 30 minute minimum on unreferenced object retention even if it's manually triggered by a
maintainer. See:

https://salsa.debian.org/help/administration/housekeeping.md#prune-unreachable-objects

That is going to make it hard to win a race after the tag has been pushed.

It may be more feasible to win a race *before* the tag has been pushed.
In other words, ask the Debian uploader to review your package, wait for
them to clone the package that they're going to review, and then replace
the underlying upstream tree before they push the signed tag. The timing
is tricky but it may be possible.

A Salsa administrator can run git gc --prune=now directly in the
underlying file system, which makes winning a race much easier.

Sneaking malicious code into the upstream tree without any cryptographic tricker and just assuming the sponsor will miss it is a more likely
attack. I'm not sure that an attacker really gains much by doing lots of
novel cryptographic attacks and racing a maintainer just to hide the code
from sponsorship review. I fear that most sponsorship review is
insufficiently thorough to catch malicious code. This therefore feels to
me a bit like a highly elaborate attack on a window lock for a house whose front door is wide open, which is part of why I'm dubious that it's that important.

To me, the protection against this attack seems weak, and I don't really
see any strong protection against it.

The primary protection is that there are fairly good reasons to believe no
one currently knows how to generate colliding Git trees that would make
any of these attacks possible.

Couldn't we require that the signed tag2upload tag comment contains a
hash of the upstream code, somehow?

Technically, we could. This is what git-evtag does. The drawback is that
it requires a non-standard tag format and an external (and somewhat non-trivial) tool.

### Second-preimage attacks

An attacker could attempt to take the signature from a tag2upload tag
pushed to an arbitrary repository on Salsa and apply that same
signature to a Git tag for a malicious package on Salsa with the same
version number, triggering the tag2upload process. However, this
requires constructing a malicious repository with the same SHA-1 hash
digest as the repository containing the original tag. This is a
second-preimage attack on SHA-1, which is believed to be currently
infeasible. (Second-preimage attacks are believed to be currently
infeasible even against MD5, which is a much weaker hash function.)
tag2upload therefore prevents this attack.

Is a second preimage really required to mount this attack? Consider if someone creates a collision for a good and a bad version, gets the good version signed by a sponsor, and then re-use that signature for the bad version.

That's a different attack discussed earlier under "Moving the tag." This section is about performing the attack on an arbitrary tag2upload Git tag
for a repository that isn't under the attacker's control.

Arguably I should drop this section, since what it's saying is so obvious
to anyone with knowledge of the cryptography properties in play that
people may read too much into it and assume it's saying something it
isn't.

--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

"Russ" == Russ Allbery <rra@debian.org> writes:

Russ> The attack that Simon is talking about doesn't require a
Russ> preimage attack, only a successful collision attack against
Russ> Git trees using SHAttered plus some assumptions about where
Russ> Git may be lazy about revalidating hashes. It's an
Russ> interesting point that I didn't think of, although I'm not
Russ> sure that it would work against GitLab and thus against Salsa
Russ> and I think it's fairly trivial to protect against regardless.

Russ, I'm trying and failing to find time to write a long response to
your security review.

But I'm going to try to make this point because I think it is relevant
and because it came up a number of times when I'm thinking about your
analysis.

You talk a number of times about whether an attack is possible against
salsa. But especially when thinking about detection and tracing, I
think that things that are verified by signatures made with keys not
held by the system in question are harder to modify than things that can
be verified only so long as a system remains trusted.
One of the things that I found striking in the xz-utils attack was how
two different systems (the release tarballs) and git archives might have
been exploited to hide an attack. If people looked at git and not the
archive, they might conclude there was no attack, even if they had been
alerted that there might be a problem.

Which is to say, especially in the moment when considering an incident,
people are very bad about reasoning about whether views of a system are equivalent.

So, I consider the following to be useful to an attacker--to be threats
worth mitigating:

1) Attacker uploads malicious code to the archive.

2) Attacker possibly through a compromise of the dgit server and salsa
changes the git view to be something harmless.

Such an attack can be detected by regularly verifying the archive
contents against git versions.
I do not have confidence that verification will hapen.
At least in my experience if I have a choice between git and unpacking a
dsc, I'll take git almost all the time.
I realize there are DDs who prefer the dsc.

Still, my initial read of your analysis is that you discount attacks
like this more than makes sense to me.
I also believe that hash collisions may make attacks like the above more possible.

My strong suspicion is that even if the classes of attack I am thinking
about are given the consideration I think they need, tag2upload will
still be reasonable from a security architecture standpoint.

-----BEGIN PGP SIGNATURE-----

iHUEARYIAB0WIQSj2jRwbAdKzGY/4uAsbEw8qDeGdAUCZmtdJgAKCRAsbEw8qDeG dJ4qAP49gdVx/RjULCf8s94OfmZNE63mrLhgi+zUzHjoKR/4BAD/cG5bt8utyUT5 43gCJtR02lqX/k9UO5f55Cd+ITdlFQo=
=u2yL
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Russ Allbery <rra@debian.org> writes:

Can this be substantiated? Using SHA1CD in Git does not necessarily
mean someone cannot manually create a Git repository with a colliding
git commit somewhere in the history that gets accepted by git, and
allows someone to replace actual file contents. That may be the case,
but I haven't seen any detailed analysis answering that.

This was a really interesting point that I didn't catch. Thank you! Let
me try to rephrase this in the form of an attack and see if this captures what you were getting at.

The attack: Using a pre-SHA-1DC version of Git, construct a benign and malicious pair of Git trees that diverge at some point by abusing the hash
of an object vulnerable to SHAttered.

Right, and that this happened at some point in the past rather than on
the git tag commit id. This git repository could be the upstream or the
debian git repository.

Push the benign tree to Salsa, relying on Salsa not reverifying the
hashes of new objects with a hardened hash, or alternately have
already planted the benign tree in a Git repository imported into
Salsa before SHA-1DC was in use. Get that tree signed by a sponsor,
again relying on the sponsor's git client not revalidating object
hashes, and then follow the same attack pattern in either "Moving the
tag" or "Replacing the upstream tree." Rely on the tag2upload server
not reverifying the hashes of the Git tree when it pulls it to
construct the signed source package.

In essence, this attack exploits the fact that Git is lazy about
performing hashes and usually only does so when it has to. I'm not sure
this assumption is correct for Salsa in particular, but it's at least plausible. The trees used in this attack would fail git fsck, because the critical object would hash to a different value using SHA-1DC than it does with SHA-1, but it's not clear that git fsck is called at any of the
points that would detect this attack.

I believe this attack would be prevented by setting transfer.fsckObjects
to true in the Git configuration of the tag2upload worker and failing the operation if it detects anything. (Or, equivalently, calling git fsck
after git clone and failing on any detected problems.) I believe this
forces recomputation of the hashes of all received objects. The object
used in this attack would fail that hash recomputation because the
tag2upload server would use a version of Git that uses SHA-1DC. The cost
is a performance penalty on git clone, which would be trivial for most repositories but which might be noticable for particularly large Git
trees.

My personal opinion is that always setting transfer.fsckObjects to true is good practice anyway to catch more banal problems such as disk corruption
and memory bit flips, so while I'm not sure I would bother just for this attack, it might be a good idea on general principles.

I have had that settings in my .gitconfig for several years, and the
number of git repositories that fail to clone with it is not negligible.
I encourage people to enable it and experiment for themselves. Try https://git.savannah.gnu.org/git/coreutils.git for example. I hack
around it by adding a 'fclone' alias, and I still need to use it once in
a while.

[transfer]
fsckObjects = true
[alias]
fclone = clone -c "fetch.fsckObjects=false"

Maybe the number of repositories on Salsa with this problem is low, but
isn't the tag2upload design vulnerable to upstream git repositories
having this problem too? In the part of tag2upload that re-generate the *.orig.tar.gz file. Maybe I'm missing how that is supposed to work,
relying on debian/watch does not strongly/uniquely identify a release
artifact. Many distributions store SHA256 hashes of the expected
upstream release tarball, and this is a good practice that Debian
doesn't support to my knowledge.

/Simon

-----BEGIN PGP SIGNATURE-----

iIoEARYIADIWIQSjzJyHC50xCrrUzy9RcisI/kdFogUCZmtq0xQcc2ltb25Aam9z ZWZzc29uLm9yZwAKCRBRcisI/kdFolQgAP4wF/ONJ1V94XubUlcI3L3f2NbVUlIw itk6+QOFSovSBQEA9XCUoJi61S02yRy+Tca6FKbtpnWkclks22lD0jgtGAM=
=aqr3
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Simon Josefsson <simon@josefsson.org> writes:

I have had that settings in my .gitconfig for several years, and the
number of git repositories that fail to clone with it is not negligible.
I encourage people to enable it and experiment for themselves. Try https://git.savannah.gnu.org/git/coreutils.git for example. I hack
around it by adding a 'fclone' alias, and I still need to use it once in
a while.

[transfer]
fsckObjects = true
[alias]
fclone = clone -c "fetch.fsckObjects=false"

I've been using this setting for over four years (I think maybe about six years) and have never had a failure before this email message, just to add another data point. It may depend on whether you work with a lot of Git repositories from very early adopters.

I started using this setting when I discovered a Git repository at work
that had become silently corrupted due to disk or memory corruption and
went undetected for a disturbing length of time.

I agree that the coreutils repository does fail:

error: object a6727941433ee1c91a20ede6cb381af1d18c566d: missingSpaceBeforeDate: invalid author/committer line - missing space before date

(That also confirms my suspicion that if you set this Git configuration setting, git clone will fail if it detects any invalid objects.)

I don't know if there is a setting that disables the checks for more
pendatic issues such as malformed author/committer lines but retains the verification of object hashes.

Maybe the number of repositories on Salsa with this problem is low, but
isn't the tag2upload design vulnerable to upstream git repositories
having this problem too?

I think it would be fine for tag2upload to refuse to work on repositories
that cannot pass git fsck. I know there are some good reasons why people decline to fix old repositories with things that Git considers to be corruption, but tag2upload is not intended to be universal and is strictly optional and already doesn't support some things that people may wish to
do. This feels like a reasonable one to add to that list, to me.

The alternative would be to add some sort of support for fsck.skipList,
but that seems like annoying and arguably unnecessary complexity that potentially reintroduces the same security problem via a different route.

Or, of course, find a way to disable the author/committer checks, which I suspect are most of the failures, and keep the object hash checks.

--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Russ Allbery <rra@debian.org> writes:

Or, of course, find a way to disable the author/committer checks, which I suspect are most of the failures, and keep the object hash checks.

Apologies, I should have done a bit more research before sending my
message. Adding the following to my .gitconfig allows me to clone the coreutils repository:

[fetch "fsck"]
missingSpaceBeforeDate = ignore

So it would be possible for us to develop a list of problems like this to ignore for the purposes of tag2upload if we believed they were
unimportant.

I'm a little bit dubious about this. If I were the dgit-repos archive maintainer, I would want to enforce an invariant that all repositories
passed git fsck because it feels like an annoying slippery slope to open.
But it would be possible to go this direction.

The alternative would be to add some sort of support for fsck.skipList,
but that seems like annoying and arguably unnecessary complexity that potentially reintroduces the same security problem via a different
route.

I see that fsck.skipList explicitly says that corrupt objects cannot be skipped, so while the rest of this paragraph continues to apply, I am now
less concerned about this introducing security issues.

--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Tuesday, June 11, 2024 9:39:04 PM EDT Russ Allbery wrote:

Hi all,

Below is the security review that I did of the tag2upload design.

I am not a neutral party, in the sense that I think tag2upload is a good
idea and should be deployed. However, I do these types of security
reviews professionally, and I tried to approach this review the same way
that I would approach a major work project that needed a security review
to ensure we weren't deploying something with security issues. I
encourage any Debian community member with security expertise to check my work; with security reviews, the more eyes, the better.

I will also post this review on my web site, probably later tonight if I
have time.

I appreciate the thought and effort that went into this review.

If I'm following your description correctly, the tag2upload "package" flow is:

developer --> salsa --> tag2upload --> ftp.upload.debian.org
machine --> dgit-repos

Is that right?

While it may not matter from a post attack detection security trace perspective, I think there are more routine trace activities that this complicates. A couple of examples are the signed by listing in the
tracker.d.o news section for packages and who-uploads from devscripts.

While making package signing information less visible isn't directly a
security issue, it does seem like a complication that makes it harder to keep up with what's going on.

Would you consider these kind of indirect effects relevant from a security analysis perspective or are they just non-security concerns from your POV?

Scott K

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEE53Kb/76FQA/u7iOxeNfe+5rVmvEFAmZuViUACgkQeNfe+5rV mvFHMw//VtPmopIPxMKy+2VilbOjitTbpsOTNF2+iNwjZKEtKGuSR3YeilLrvgBt acj3JlB2dSc5X8vBNj26v9r8fYc/BMPd1EpTvtPvyIFLaoEyAnp+rPTJOec+2LPf jaRg9ww/RTWebT+HyggJkRnhUwdjja6wh6WhKWTXzPohXvQt0/FbED44WqKB322W kpd4w1ZxffCxI4m6/dqzsxfFBGJtEV0TzGUOCDLpJphi6Ez9H8bD2AvwVdSBEOzV zlYVPQ9o6kO21nsumxDqpB3jFgn8RF+ajWZ16LYiCDolsv9DKNu2UfTRq9yyD+kp y2EnC4l7oQ/oIAffqnZXQ/pOHzAC/MX1kIsQF8kgByiY8QobA2YyucUwVv+t4TAU IP+R2NPcysNrtmSS7yVIoCxiI1uMWncC0jhlWIqu8dopCvH0qMHj4Lj49dMk24Tv pkSjNyMKXm25tzSwZtr8w6wpIn09nJCzdIQULjpt+BASa/maSR0mAnk+9pVqOmol mqf7L0SP/W2VnFqrMhCxszhJgWubPSehZG6ohrF/0lS3+4p3GqnoqQmGfSQ0W9OR 8QudwWD+V7Z3QgIGeTvFAsLkH0FVnxofT0BEZYzxFaGGypBB4UfK9MklSGXokkec x2seB496x0FVCrGQ29PIFga9XMqK/sIdFHfbbDsFDoFcIgwLPLc=
=Be8C
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Scott Kitterman <debian@kitterman.com> writes:

I appreciate the thought and effort that went into this review.

If I'm following your description correctly, the tag2upload "package" flow is:

developer --> salsa --> tag2upload --> ftp.upload.debian.org
machine --> dgit-repos

Is that right?

Yes, I think so.

While it may not matter from a post attack detection security trace perspective, I think there are more routine trace activities that this complicates. A couple of examples are the signed by listing in the tracker.d.o news section for packages and who-uploads from devscripts.

While making package signing information less visible isn't directly a security issue, it does seem like a complication that makes it harder to
keep up with what's going on.

Would you consider these kind of indirect effects relevant from a
security analysis perspective or are they just non-security concerns
from your POV?

I made the assumption that, if tag2upload were deployed, those tools would
be modified to pick up the signer information from the *.changes fields
where tag2upload puts it. That metadata is put into both the *.dsc and
the *.changes files.

As with the other parts of this proposed design, that does require
trusting tag2upload to do the authentication check properly, so a
compromised tag2upload server could write erroneous trace information and therefore would not be detected by either of those tools.

A tag2upload server compromise is fairly serious. A compromise of any of tag2upload, dak, or the buildds have roughly equally serious potential
impact on the archive as far as I can tell, although the details differ.
In all three cases, you need reproducible builds to reliably detect the compromise, although in the tag2upload case you only need reproducible
source builds for the specific set of source transformations that
tag2upload is willing to perform, which I believe is a much easier problem
than the reproducible binary builds required to detect buildd or dak compromises. dak, uniquely, can meddle with either source *or* binary packages, but dak meddling with source packages will break the signatures
on those packages, so is somewhat easier to detect than dak meddling with binary packages.

(This is assuming I'm not missing some security control in dak, which is entirely possible because I've not done a comprehensive security review of
dak and am not certain of all the details of the architecture. If I'm
missing something, please do correct me!)

--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Russ Allbery dijo [Sat, Jun 15, 2024 at 11:44:35PM -0700]:

A tag2upload server compromise is fairly serious. A compromise of any of tag2upload, dak, or the buildds have roughly equally serious potential
impact on the archive as far as I can tell, although the details differ.
In all three cases, you need reproducible builds to reliably detect the compromise, although in the tag2upload case you only need reproducible
source builds for the specific set of source transformations that
tag2upload is willing to perform, which I believe is a much easier problem than the reproducible binary builds required to detect buildd or dak compromises. dak, uniquely, can meddle with either source *or* binary packages, but dak meddling with source packages will break the signatures
on those packages, so is somewhat easier to detect than dak meddling with binary packages.

This paragraph can even enable what could amount to an interesting
synergy.

More than one tag2upload instance can be set up. Due to its
architecture, they can be non-internet-accessible systems (naturally,
they will require to _access_ the network, or at least Salsa). This
can be paired with the requirement that all uploads made via
tag2upload are reproducible.

Because, if they are, dak can be made to take a tag2upload-provided
package as valid only if two identical packages are presented, built
by at least two independent tag2upload instances.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On June 16, 2024 6:44:35 AM UTC, Russ Allbery <rra@debian.org> wrote:

Scott Kitterman <debian@kitterman.com> writes:

I appreciate the thought and effort that went into this review.

If I'm following your description correctly, the tag2upload "package" flow is:

developer --> salsa --> tag2upload --> ftp.upload.debian.org
machine --> dgit-repos

Is that right?

Yes, I think so.

While it may not matter from a post attack detection security trace
perspective, I think there are more routine trace activities that this
complicates. A couple of examples are the signed by listing in the
tracker.d.o news section for packages and who-uploads from devscripts.

While making package signing information less visible isn't directly a
security issue, it does seem like a complication that makes it harder to
keep up with what's going on.

Would you consider these kind of indirect effects relevant from a
security analysis perspective or are they just non-security concerns
from your POV?

I made the assumption that, if tag2upload were deployed, those tools would
be modified to pick up the signer information from the *.changes fields
where tag2upload puts it. That metadata is put into both the *.dsc and
the *.changes files.

As with the other parts of this proposed design, that does require
trusting tag2upload to do the authentication check properly, so a
compromised tag2upload server could write erroneous trace information and >therefore would not be detected by either of those tools.

A tag2upload server compromise is fairly serious. A compromise of any of >tag2upload, dak, or the buildds have roughly equally serious potential
impact on the archive as far as I can tell, although the details differ.
In all three cases, you need reproducible builds to reliably detect the >compromise, although in the tag2upload case you only need reproducible
source builds for the specific set of source transformations that
tag2upload is willing to perform, which I believe is a much easier problem >than the reproducible binary builds required to detect buildd or dak >compromises. dak, uniquely, can meddle with either source *or* binary >packages, but dak meddling with source packages will break the signatures
on those packages, so is somewhat easier to detect than dak meddling with >binary packages.

(This is assuming I'm not missing some security control in dak, which is >entirely possible because I've not done a comprehensive security review of >dak and am not certain of all the details of the architecture. If I'm >missing something, please do correct me!)

Yes and no. The difference is that currently, I can download the source package and verify it myself. Not just who signed it and with what key, but that the signature verifies. I don't need to trust assurances from any service.

From the perspective of Debian, the project, that's presumably not significant and can be accounted for by updating our tools. From the perspective of some Debian users, I'm less certain of the significance.

Scott K

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Scott Kitterman <debian@kitterman.com> writes:

Yes and no. The difference is that currently, I can download the source package and verify it myself. Not just who signed it and with what key,
but that the signature verifies. I don't need to trust assurances from
any service.

No, that's not quite true. You're still trusting assurances from the uploader's system. The uploader did not, in general, directly check the artifact whose signature you're verifying; they, and you, are trusting
that the source package construction was done correctly from their working tree.

There's been a lot of discussion of the implications of the xz backdoor
for source package construction, but one of the takeaways that I took from
it is to be even less sure of the security of the uploader systems that
are generating our source packages. Imagine if xz had been backdoored to,
say, inject the installation of a malicious maintainer script into source packages constructed on that system. How long would it have been before
we noticed? The malicious code would have been signed by the uploader and
all the signatures would verify without difficulty.

Certainly we would have noticed eventually. Probably we would have
noticed before the next stable release. But I'm not at all sure we would
have noticed before a lot of Debian uploader systems were backdoored and potentially a lot of uploader keys were stolen depending on uploader key storage practices. And there are probably sneakier attacks that I haven't thought of.

From the perspective of Debian, the project, that's presumably not significant and can be accounted for by updating our tools. From the perspective of some Debian users, I'm less certain of the significance.

I think it would be hugely valuable to have something like a "dgit
verification mode" where you can ask dgit, which already has all the
source package construction logic, to take a tag2uplod-generated source package, start from the tag object and signature, and reproduce that
source package and verify it. Except for the retrieval of the signed Git
tag, in theory all of that could be done locally. I'm not sure how hard
that would be (this comes back to the question of how difficult it is to
ensure that the tag2upload source construction algorithm is easily reproducible), but I think something like that would go a long way towards providing some really interesting security properties.

--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Sam Hartman <hartmans@debian.org> writes:

You talk a number of times about whether an attack is possible against
salsa. But especially when thinking about detection and tracing, I
think that things that are verified by signatures made with keys not
held by the system in question are harder to modify than things that can
be verified only so long as a system remains trusted.

I agree with this.

Which is to say, especially in the moment when considering an incident, people are very bad about reasoning about whether views of a system are equivalent.

Yes, I agree with this too, and definitely include myself. These sorts of threats are very tricky to reason about and I have doubtless missed some.

So, I consider the following to be useful to an attacker--to be threats
worth mitigating:

1) Attacker uploads malicious code to the archive.

2) Attacker possibly through a compromise of the dgit server and salsa changes the git view to be something harmless.

I completely agree. An undetected compromise of Salsa in which an
attacker makes the web view not match the Git repository is nasty, and I
fear that attack could do a lot of damage, since we do a lot of our work
via the web view and but what we upload to the repository is based on the
Git repository. But I'm not sure that tag2upload makes this attack any
worse.

I think we already trust Salsa's view of Git archives quite a lot. This involves reasoning about sponsor behavior, which is always tricky since I
can't view it personally, we don't really have much data, and there's a
lot of variation between sponsors. But my guess is that the workflow of reviewing code on Salsa (possibly incrementally via MRs) and then doing a
git pull, building a source package, and uploading it without further
review is very common.

So I think you are correctly identifying a risk that I didn't talk about
in my security review, but I'm also not sure that it's relevant to a differential comparison of life before tag2upload and life after
tag2upload. Maybe tag2upload does make us more vulnerable to this by
changing behavior: if the existence of tag2upload shifts sponsors from
doing code reviews locally on exactly the thing that they're going to turn
into a source package to doing code reviews via the Salsa view and then
blindly signing a git pull, that does make us more vulnerable to Salsa compromises where an attacker can make those two views not match. But I'm
not sure I see a reason why that would be the case.

Such an attack can be detected by regularly verifying the archive
contents against git versions.

Hm, can it? If the attacker has injected code into the Salsa Git
repository but hidden it from the Salsa UI, wouldn't the verification step
also use the Salsa Git repository as its point of reference, and thus miss
the attack?

I think this is a special case of an attacker sneaking malicious code into
a package past the audit of a sponsor so that a sponsor unknowingly signs
that malicious code and uploads it into the archive. There are several
ways to perform that attack, including just making the code complicated
and annoying to review and relying on the sponsor not spotting the
problem. This is a particularly nasty variation since it can also affect packages that aren't sponsored, assuming that the maintainer doesn't
notice commits snuck into their package that they didn't make, but I think
the implication is roughly the same.

I don't think Debian has very good protections against this attack either
with or without tag2upload, and I don't think tag2upload changes the
properties of this attack very much. It provides some assistance around
the edges by forcing all of the code into Git, where we at least have a
better tracing story, but I would not rely on that to head off this
attack.

Still, my initial read of your analysis is that you discount attacks
like this more than makes sense to me.

That's not my intent. I did think about some attacks like that (not that
one in particular, but various similar sponsorship scenarios) and my
intent was to say that I didn't think tag2upload changed the problem one
way or the other. I think some of that got lost in editing.

Maybe I'm missing some reason why tag2upload is more vulnerable to attacks
of this type, though?

--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sunday, June 16, 2024 12:01:31 PM EDT Russ Allbery wrote:

Scott Kitterman <debian@kitterman.com> writes:

Yes and no. The difference is that currently, I can download the source package and verify it myself. Not just who signed it and with what key, but that the signature verifies. I don't need to trust assurances from
any service.

No, that's not quite true. You're still trusting assurances from the uploader's system. The uploader did not, in general, directly check the artifact whose signature you're verifying; they, and you, are trusting
that the source package construction was done correctly from their working tree.

There's been a lot of discussion of the implications of the xz backdoor
for source package construction, but one of the takeaways that I took from
it is to be even less sure of the security of the uploader systems that
are generating our source packages. Imagine if xz had been backdoored to, say, inject the installation of a malicious maintainer script into source packages constructed on that system. How long would it have been before
we noticed? The malicious code would have been signed by the uploader and all the signatures would verify without difficulty.

Certainly we would have noticed eventually. Probably we would have
noticed before the next stable release. But I'm not at all sure we would have noticed before a lot of Debian uploader systems were backdoored and potentially a lot of uploader keys were stolen depending on uploader key storage practices. And there are probably sneakier attacks that I haven't thought of.

I agree that there's a risk that what the uploader thought they were uploading and what they actually uploaded are different, but that's independent of tag2upload or not. I also agree there are tradeoffs on all this. In the particular case of source package construction, there's a tradeoff between doing it on a centralized, managed service with a known configuration that is internet exposed versus the variety of unknowns associated with individual developer machines.

There are different risks for the end user. Currently dget uses dscverify by default before unpacking a source package. I'm not an expert at all, so I don't have any appreciate for the perceived risks that led to that being the default (IIRC, it wasn't always). I am assuming that wasn't random. I'm not sure how that would work in this new paradigm.

From the perspective of Debian, the project, that's presumably not significant and can be accounted for by updating our tools. From the perspective of some Debian users, I'm less certain of the significance.

I think it would be hugely valuable to have something like a "dgit verification mode" where you can ask dgit, which already has all the
source package construction logic, to take a tag2uplod-generated source package, start from the tag object and signature, and reproduce that
source package and verify it. Except for the retrieval of the signed Git tag, in theory all of that could be done locally. I'm not sure how hard
that would be (this comes back to the question of how difficult it is to ensure that the tag2upload source construction algorithm is easily reproducible), but I think something like that would go a long way towards providing some really interesting security properties.

I think this is much more manageable if you assume the whole world uses git
all the time for everything and git is the interface, but that's not reality. Personally, I think the ability to interact with the archive to do the verification and not relying on things that are not the archive for the code to verify is an important property of the existing system and I don't think it's feasible to maintain it in a tag2upload world.

Maybe that's OK, but I don't think it's nothing and it should be acknowledged as a cost of moving in this direction.

Scott K
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEE53Kb/76FQA/u7iOxeNfe+5rVmvEFAmZvEFsACgkQeNfe+5rV mvGgAQ//W2j7VeudJC0Hd3Hir4W5JMxTXUiXQwwzNTG1uCGkh+cLwxcL4vKHJCH3 lhDmx/5OqE1prObq6h5h72LDx5EtnkZbx/jO99L7UDOcmx33Xtsd7hArhr2z1T5B ZMrhYLGn8rIopS61+IvgWIfLGM+V6Tg8M4vZmQrsZZlZqAANUQEyjdKsUQBkGT1G rCnS++hHjAjeDyRRH5vpl5oA91WJgNzmYeBYuGQLb/ji/JL03dmUZJfQgnXjkXLE z7J6HVBDJKB85qT8mYpH9Y2xfzz/2IshbS+IOXpmaTWX1xtQ9WoRBDKQVLzK4Frz teeXoTM5sfmP8KSN/z5KcfUSNUS5u2Cznlgr/Iqdb3+XjnURnN8NuK4DEhaWVHJM wPsCPUIwdhrop/Q2QDMy4SYQCH8zHbi2pwMps5N5FTfjHvFnq2p+nKBau6arhL1d 2/ybiHfooXitqgtNpe5BBcVXy6uplziGYUKv3Ux6Q4O95c6vIP7ffXfkWUHiWdy4 3Em3YtBEqkru5tEzKblRf21LqqQ5qJGLVdaz+1LGVA8BkPNQCNUJ5ybbi/o3SDMi F/UfFFynkZcqiwEtGZffnD5MFjrD4pY0VqBiuTpnPiVV6dQXmfx9sD5TvtTSD7PR Uq8h3DYPhaWZs9wjwWf4aYMw4txf7wLFLmq2ayH2hFKAjeanbhQ=
=nz31
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Scott Kitterman <debian@kitterman.com> writes:

I agree that there's a risk that what the uploader thought they were uploading and what they actually uploaded are different, but that's independent of tag2upload or not.

But it's not independent; tag2upload makes this story somewhat better. tag2upload is based on a signed Git tag and moves the source package construction off of the uploader's system onto more-secure project infrastructure. It therefore moves the uploader's signature closer to
their intent: it's a signature over the thing that they are more likely to
have directly reviewed, not over a build artifact derived from their Git
tree.

I also agree there are tradeoffs on all this. In the particular case of source package construction, there's a tradeoff between doing it on a centralized, managed service with a known configuration that is internet exposed versus the variety of unknowns associated with individual
developer machines.

Right, for some value of Internet-exposed that can be fairly restrictive.

There are different risks for the end user. Currently dget uses
dscverify by default before unpacking a source package. I'm not an
expert at all, so I don't have any appreciate for the perceived risks
that led to that being the default (IIRC, it wasn't always). I am
assuming that wasn't random. I'm not sure how that would work in this
new paradigm.

Well, it obviously still works (once it's aware of the tag2upload key) but
the signature is by the entity that constructed the source package, so dscverify will trace the signature back to the tag2upload server and not
to the uploader's system.

I think it would be hugely valuable to have something like a "dgit
verification mode" where you can ask dgit, which already has all the
source package construction logic, to take a tag2uplod-generated source
package, start from the tag object and signature, and reproduce that
source package and verify it. Except for the retrieval of the signed
Git tag, in theory all of that could be done locally. I'm not sure how
hard that would be (this comes back to the question of how difficult it
is to ensure that the tag2upload source construction algorithm is
easily reproducible), but I think something like that would go a long
way towards providing some really interesting security properties.

I think this is much more manageable if you assume the whole world uses
git all the time for everything and git is the interface, but that's not reality.

I'm extremely confused. Of course you can assume that for any package
signed with tag2upload. tag2upload will only act on Git repositories and therefore anything that it has worked on necessarily had to use Git as the interface.

Maybe you thought I was implying that this dgit verification mode would
work with general source packages and not just tag2upload packages? No,
it cannot, because in the general case we have absolutely no idea how to
map a source package in the archive back to a Git tree. That's exactly
the problem that tag2upload is trying to solve. For non-tag2upload
packages, we still have to rely on the source package as the farthest back
that we can trace the code without diverging into package-specific
analysis and diverging maintainer workflows.

Personally, I think the ability to interact with the archive to do the verification and not relying on things that are not the archive for the
code to verify is an important property of the existing system and I
don't think it's feasible to maintain it in a tag2upload world.

Here too, I don't understand exactly what you are saying. All of the
source packages uploaded to the archive via tag2upload will verify. They
have valid OpenPGP signatures. That OpenPGP signature traces the
provenance of the source package back to the entity that constructed the
source package just as the source package signatures in the archive do
today.

The entity that does the source package construction has changed from the uploader's system to the tag2upload server, so tracing the package to a specific maintainer (absent a dgit verifiation mode) requires relying on
that server, *but* you also now have the option of doing additional work
to trace the provenance back further to a signed Git tree, something
that's not possible in the general case with source packages today. This
has various security trade offs as discussed above. But I do not agree
that it breaks the property that you claim it breaks; in both cases, you
can trace the source package back to the entity responsible for its construction.

--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sunday, June 16, 2024 12:46:41 PM EDT Russ Allbery wrote:

Scott Kitterman <debian@kitterman.com> writes:

I agree that there's a risk that what the uploader thought they were uploading and what they actually uploaded are different, but that's independent of tag2upload or not.

But it's not independent; tag2upload makes this story somewhat better. tag2upload is based on a signed Git tag and moves the source package construction off of the uploader's system onto more-secure project infrastructure. It therefore moves the uploader's signature closer to
their intent: it's a signature over the thing that they are more likely to have directly reviewed, not over a build artifact derived from their Git tree.

I can see that, but that leads to what I view as a problem. The thing in the archive is signed by a machine, not the human who decided it should be uploaded.

I also agree there are tradeoffs on all this. In the particular case of source package construction, there's a tradeoff between doing it on a centralized, managed service with a known configuration that is internet exposed versus the variety of unknowns associated with individual
developer machines.

Right, for some value of Internet-exposed that can be fairly restrictive.

There are different risks for the end user. Currently dget uses
dscverify by default before unpacking a source package. I'm not an
expert at all, so I don't have any appreciate for the perceived risks
that led to that being the default (IIRC, it wasn't always). I am
assuming that wasn't random. I'm not sure how that would work in this
new paradigm.

Well, it obviously still works (once it's aware of the tag2upload key) but the signature is by the entity that constructed the source package, so dscverify will trace the signature back to the tag2upload server and not
to the uploader's system.

It's signed by tag2upload, not the human that decided to upload it.
Personally, I think that's significant.

I think it would be hugely valuable to have something like a "dgit
verification mode" where you can ask dgit, which already has all the
source package construction logic, to take a tag2uplod-generated source
package, start from the tag object and signature, and reproduce that
source package and verify it. Except for the retrieval of the signed
Git tag, in theory all of that could be done locally. I'm not sure how
hard that would be (this comes back to the question of how difficult it
is to ensure that the tag2upload source construction algorithm is
easily reproducible), but I think something like that would go a long
way towards providing some really interesting security properties.

I think this is much more manageable if you assume the whole world uses
git all the time for everything and git is the interface, but that's not reality.

I'm extremely confused. Of course you can assume that for any package
signed with tag2upload. tag2upload will only act on Git repositories and therefore anything that it has worked on necessarily had to use Git as the interface.

Maybe you thought I was implying that this dgit verification mode would
work with general source packages and not just tag2upload packages? No,
it cannot, because in the general case we have absolutely no idea how to
map a source package in the archive back to a Git tree. That's exactly
the problem that tag2upload is trying to solve. For non-tag2upload
packages, we still have to rely on the source package as the farthest back that we can trace the code without diverging into package-specific
analysis and diverging maintainer workflows.

No, I understood that. I guess I view this somewhat differently. In my mind the signature on a source package is a developer saying that they believe it should be in the Debian archive and the archive tools verify that the human making that assertion has the authority to do so. In my view, the tag2upload signature is something fundamentally different.

Personally, I think the ability to interact with the archive to do the verification and not relying on things that are not the archive for the code to verify is an important property of the existing system and I
don't think it's feasible to maintain it in a tag2upload world.

Here too, I don't understand exactly what you are saying. All of the
source packages uploaded to the archive via tag2upload will verify. They have valid OpenPGP signatures. That OpenPGP signature traces the
provenance of the source package back to the entity that constructed the source package just as the source package signatures in the archive do
today.

The entity that does the source package construction has changed from the uploader's system to the tag2upload server, so tracing the package to a specific maintainer (absent a dgit verifiation mode) requires relying on
that server, *but* you also now have the option of doing additional work
to trace the provenance back further to a signed Git tree, something
that's not possible in the general case with source packages today. This
has various security trade offs as discussed above. But I do not agree
that it breaks the property that you claim it breaks; in both cases, you
can trace the source package back to the entity responsible for its construction.

I think it's just that I view a signature by a mechanized service as something different that a signature made by an actual person. Technically you are correct, but I think it's fundamentally different. I don't think the computer is responsible for anything. I think it has to trace to a person if you want to talk about responsibility.

Scott K
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEE53Kb/76FQA/u7iOxeNfe+5rVmvEFAmZvGhIACgkQeNfe+5rV mvFmkg/8Dr/XIs2vFXstF2STnbYA8AxrFB/CiwpYjRSDulzTyiyMh6GDuootntbn DMOvQrjl/GZfc42CDxdaTQvbda4uwYyRI7ohQhSotf9Rb6YvlVMgruZC3hUm0keE JmWENiMnYYpPDZeZn8dshEVXWUFsGCwPp0w5wlxnDg9FD6tyHAaHsoz7nTBcMozE 2ViEm37mwjVIhisfUsECGdbJpUbeEacduk7lSXz95lpG9RB9XXputnblvm5+8cxR VDWJoeDU0UiMBY90HSJoKqDCQd1T4O+sOmDHpcqz90db0g+71X6G9WjKXYIYqm2Y +keGJVcgA+lI/DeGdap09h/mUwQjqr/TsuDOlXWmQLw/72AnVScFhWb6qP7yxE1m uw6YFDOI20EMCLZyE55yLi1kxzeSbHSicWeUIaHhsn7GoLRYIo3CKGSqhxgne3zy DMvNWn10xYGp8GmuDPuE6C6YEiZETB2CfUpQretnYKxdluwWgScLA1NoqQBq25zC oesITSqupdVCi+umHr9VnEhnMZSI2RRuygSVicnyjS9wWFbimzYbm7Ylg9XbHJhC rt9/yEJLwJZ8lfMaOWHQ3mtnjnsMDrpNcGBk9pciVdXo8qrZ0Qkpz4eX7PJRBZTf SKe0vxA6xa4BWefgbAAjqZb744mzYisw1kU0tYE9KATXmGNGwFQ=
=trHU
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Scott (2024.06.16_16:18:35_+0000)

There are different risks for the end user. Currently dget uses dscverify by default before unpacking a source package. I'm not an expert at all, so I don't have any appreciate for the perceived risks that led to that being the default (IIRC, it wasn't always). I am assuming that wasn't random. I'm not sure how that would work in this new paradigm.

This mechanism is also far from perfect. The signature was made when the
source package was built, and that could have been several releases
ago for a slow-moving package. This key may or may not be present in any
of the keyring packages on the user's machine, and it could be expired.

This check is done in dget, because it's all it can really do. There
are many different use-cases for dget. In some of them it's a useful
check, for others maybe not.

A tag2upload source package will have been signed with tag2upload's key.
So, assuming this is available in a local keyring, you can still verify
that this was a package that was signed by tag2upload for uploading to
Debian. That actually tells you more than being signed directly by a
developer, because you know it was uploaded to the Debian archive.

I imagine many tools like dscverify will have to learn to extract the
metadata added by tag2upload, to name the tag signer.

Stefano

--
Stefano Rivera
http://tumbleweed.org.za/
+1 415 683 3272

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Scott Kitterman <debian@kitterman.com> writes:

I think it's just that I view a signature by a mechanized service as something different that a signature made by an actual person.
Technically you are correct, but I think it's fundamentally different.
I don't think the computer is responsible for anything. I think it has
to trace to a person if you want to talk about responsibility.

Okay, thanks, I think this is the core of our disagreement. Let me sum up
my side, just to be very clear about what I think the disagreement is.

I don't believe that "a signature made by an actual person" is something
that exists in the real world. Humans do not sign things. We do not have
an OpenPGP implementation in our heads. Signatures are always made by software, running on a possibly compromised computer, directed by humans.
Any link between the human and the signature is a point of possible
attack.

For the existing source package signatures, a simplified sequence looks
like this:

human --> (1) dpkg-buildpackage --> (2) debsign --> (3) archive

For tag2upload, a simplified sequence looks like:

human --> (1) Git --> (2) tag2upload --> (3) debsign --> (4) archive

In our current system, the source package signature can be traced back to
(2). In the tag2upload case, the source package signature can be traced
back to (3) using the existing techniques and, with more work and new techniques, all the way back to (1).

In neither case can the source package signature be traced back to a
human, which is what I am arguing makes them similar. What we're arguing
about is which system has the better design (both security and otherwise)
for the pieces prior to (2) in the first case and (3)/(1) in the second
case.

--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On June 16, 2024 6:23:18 PM UTC, Russ Allbery <rra@debian.org> wrote:

Scott Kitterman <debian@kitterman.com> writes:

I think it's just that I view a signature by a mechanized service as
something different that a signature made by an actual person.
Technically you are correct, but I think it's fundamentally different.
I don't think the computer is responsible for anything. I think it has
to trace to a person if you want to talk about responsibility.

Okay, thanks, I think this is the core of our disagreement. Let me sum up
my side, just to be very clear about what I think the disagreement is.

I don't believe that "a signature made by an actual person" is something
that exists in the real world. Humans do not sign things. We do not have
an OpenPGP implementation in our heads. Signatures are always made by >software, running on a possibly compromised computer, directed by humans.
Any link between the human and the signature is a point of possible
attack.

For the existing source package signatures, a simplified sequence looks
like this:

human --> (1) dpkg-buildpackage --> (2) debsign --> (3) archive

For tag2upload, a simplified sequence looks like:

human --> (1) Git --> (2) tag2upload --> (3) debsign --> (4) archive

In our current system, the source package signature can be traced back to >(2). In the tag2upload case, the source package signature can be traced
back to (3) using the existing techniques and, with more work and new >techniques, all the way back to (1).

In neither case can the source package signature be traced back to a
human, which is what I am arguing makes them similar. What we're arguing >about is which system has the better design (both security and otherwise)
for the pieces prior to (2) in the first case and (3)/(1) in the second
case.

Yes. I think that's the core of the disagreement. In my view, when I type the passphrase for my key, I'm asserting responsibility for the contents of what I'm signing. It doesn't mean it is correct or uncompromised, but I am taking responsibility for
it.

Scott K

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Scott Kitterman <debian@kitterman.com> writes:

Yes. I think that's the core of the disagreement. In my view, when I
type the passphrase for my key, I'm asserting responsibility for the
contents of what I'm signing. It doesn't mean it is correct or uncompromised, but I am taking responsibility for it.

Right. And I come from a culture that emphasized blameless postmortems
and systems design and a way of thinking about security review from a
similar perspective, which is that assigning responsibility is not in and
of itself a useful thing to do. Just because someone is responsible
doesn't mean that we're more secure. It may mean that you have someone
you can punish afterwards, but it's very questionable how much that helps
with security, really.

Assigning responsibility is, in that model, only important to the degree
to which it will change people's actual behavior towards behavior that is
more secure, either before or after the fact. If one assigns
responsibility for something that isn't realistically under their control,
or in a way that doesn't cause their behavior to change, the argument is
that nothing is truly accomplished from a security standpoint. It's an illusion of security without actual security.

One of my goals in doing security design is to try to reduce the degree to which humans are performing repetitive validation tasks because humans are
not good at maintaining constant vigilance. We know this from a bunch of empircal studies on, for example, airport screening. If a human does a repetitive task with a very low rate of true positives, their attention
will fade and there will be a lot of false negatives. Asking humans to do
this is a recipe for failure, and making the humans responsible for doing
this correctly and threatening them with consequences for not doing it correctly only slightly decreases the risk of failure.

This is exactly why reproducible builds are so important: that involves
finding a way for computers to do the sorts of repetitive validation tasks
that computers are good at and that humans are very bad at.

--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------1TqV8A01T0AwFndM2WnVzMjp
Content-Type: multipart/alternative;
boundary="------------uzDnf35M0ZcCXLoBShomK0Zl"

--------------uzDnf35M0ZcCXLoBShomK0Zl
Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64

T24gMTYuMDYuMjQgMjE6MTEsIFNjb3R0IEtpdHRlcm1hbiB3cm90ZToNCj4gWWVzLiAgSSB0 aGluayB0aGF0J3MgdGhlIGNvcmUgb2YgdGhlIGRpc2FncmVlbWVudC4gIEluIG15IHZpZXcs IHdoZW4gSQ0KPiB0eXBlIHRoZSBwYXNzcGhyYXNlIGZvciBteSBrZXksIEknbSBhc3NlcnRp bmcgcmVzcG9uc2liaWxpdHkgZm9yIHRoZQ0KPiBjb250ZW50cyBvZiB3aGF0IEknbSBzaWdu aW5nLg0KDQpSaWdodC4gQnV0IGluIGJvdGggY2FzZXMgd2hhdCB5b3UncmUgc2lnbmluZyBp cyBhIHJhbmRvbSBoYXNoIGNvbnNpc3RpbmcgDQpvZiBiaXRzIHlvdSBkaWRuJ3QgcGVyc29u YWxseSB2ZXJpZnkuIElNSE8sIGFzc2VydGluZyB0aGF0IA0KcmVzcG9uc2liaWxpdHkgbWVh bnMgdGhhdCBJJ20gYWJsZSB0byB2ZXJpZnkgdGhhdCB5ZXMsIHdoYXQgSSBzYXkgSSdtIA0K c2lnbmluZyBpcyB3aGF0IEkgYWN0dWFsbHkgaW50ZW5kIHRvIHNpZ24g4oCTIGFuZCB0aGF0 IEknbSBhYmxlIHRvIHByb3ZlIA0KdGhpcyBhc3NlcnRpb24gbmV4dCBtb250aCB3aGVuIHNv bWVib2R5IGFjY3VzZXMgbWUgb2Ygc211Z2dsaW5nIGFuIA0KZXhwbG9pdCBpbnRvIERlYmlh bi4NCg0KVXNpbmcgdGFnMnVwbG9hZCBwcmVzdW1hYmx5IHlvdSBjYW4gZG8gYSAiZ2l0IGZz Y2siIGFuZCAiZ2l0IHN0YXR1cyIgYW5kIA0Kd2hhdG5vdCB0byB2ZXJpZnkgdGhhdCB3aGF0 IGdpdCBzYXlzIHlvdSBzaWduZWQgY29ycmVzcG9uZHMgdG8gd2hhdCBnaXQgDQpzYXlzIHRo ZSBjb250ZW50cyByZWZlcnJlZCB0byBieSB0aGUgY3VycmVudCBIRUFEIChhbmQgaGVuY2Ug dGhlIA0KZ2VuZXJhdGVkIHRhZyBhcmUpLiBBc3N1bWluZyBvZiBjb3Vyc2UgdGhhdCB5b3Vy IGdpdCBhbmQgdGhlIChmZXcpIA0KbGlicmFyaWVzIGl0IHVzZXMgYXJlbid0IGNvbXByb21p c2VkLg0KDQpNb3JlIHRvIHRoZSBwb2ludCwgdGhlIHN5c3RlbSB0aGF0IHJlY2VpdmVzIHlv dXIgdGFnIGRvZXNuJ3QgZXZlbiBuZWVkIA0KdG8gdHJ1c3QgeW91ciBnaXQuIEl0IGhhcyBp dHMgb3duLCB0aHVzIHlvdXIgY29tcHJvbWlzZWQgc291cmNlIG5lZWRzIHRvIA0KZ2V0IHB1 c2hlZCB0byBTYWxzYSwgd2hlcmUgdGhlIGJhZCBjb2RlIGlzIHJlYWRpbHkgdmlzaWJsZSAo YXNzdW1pbmcgDQpzb21lYm9keSB0YWtlcyBhIGxvb2spICh1bmxlc3Mgc29tZWJvZHkgbWFu YWdlZCB0byBhbHNvIGNvbXByb21pc2UgU2Fsc2EpLg0KDQpPbiB0aGUgb3RoZXIgaGFuZCwg d2hlbiB5b3UncmUgdXBsb2FkaW5nIHNpZ25lZCBzb3VyY2VzIHlvdSB0cnVzdCANCndoYXRl dmVyIHRvb2wgY2xlYW5lZCwgDQpwYXRjaGVkL2Fzc2VtYmxlZC9jb21iaW5lZC95b3VyLWdp dC13b3JrZmxvdy1oZXJlJ2QgYW5kIHRhcmJhbGxlZCB5b3VyIA0Kc291cmNlcy4gVGhpcyBp bmNsdWRlcyBhIHNpZ25pZmljYW50bHkgbGFyZ2VyIHRvb2xjaGFpbiBhcyB3ZWxsIGFzIHRo ZSANCmFic2VuY2Ugb2YgdW5yZWxhdGVkIGNvbXByb21pc2VkIGNvZGUgb24geW91ciBzeXN0 ZW0gdGhhdCBlLmcuIGRldGVjdHMgDQp0aGUgInRhciIgY29tbWFuZCBhbmQgc3dpZnRseSBh ZGRzK3VuZG9lcyBhIGJhY2tkb29yIHRvIHNvbWUgZmlsZSDigJMgb25lIA0KYXR0YWNrIHZl Y3RvciBvZiBtYW55LiBXb3JzZSwgaXQncyB1cCB0byByYW5kb20gY2hhbmNlIHdoZXRoZXIg YW55Ym9keSANCndpbGwgZXZlciBsb29rIGF0IHRoZSBzb3VyY2UgeW91IGp1c3QgZ2VuZXJh dGVkLCB5b3Vyc2VsZiBpbmNsdWRlZCwgDQpnaXZlbiB0aGF0IGV2ZXJ5dGhpbmcncyBzdXBw b3NlZCB0byBiZSBpbiBnaXQuIFRoZSB4eiBjb21wcm9taXNlIGFtcGx5IA0KZGVtb25zdHJh dGVkIHRoaXMgcHJvYmxlbS4NCg0KLS0gDQotLSBtaXQgZnJldW5kbGljaGVuIEdyw7zDn2Vu DQotLSANCi0tIE1hdHRoaWFzIFVybGljaHMNCg0K
--------------uzDnf35M0ZcCXLoBShomK0Zl
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">On 16.06.24 21:11, Scott Kitterman
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:663F8928-2CDC-4E53-9B56-282E8901F7B5@kitterman.com">
<pre>Yes. I think that's the core of the disagreement. In my view, when I
type the passphrase for my key, I'm asserting responsibility for the
contents of what I'm signing. </pre>
</blockquote>
<p>Right. But in both cases what you're signing is a random hash
consisting of bits you didn't personally verify. IMHO, asserting
that responsibility means that I'm able to verify that yes, what I
say I'm signing is what I actually intend to sign – and that I'm
able to prove this assertion next month when somebody accuses me
of smuggling an exploit into Debian.<br>
</p>
<p>Using tag2upload presumably you can do a "git fsck" and "git
status" and whatnot to verify that what git says you signed
corresponds to what git says the contents referred to by the
current HEAD (and hence the generated tag are). Assuming of course
that your git and the (few) libraries it uses aren't compromised.<br>
</p>
<p>More to the point, the system that receives your tag doesn't even
need to trust your git. It has its own, thus your compromised
source needs to get pushed to Salsa, where the bad code is readily
visible (assuming somebody takes a look) (unless somebody managed
to also compromise Salsa).<br>
</p>
<p>On the other hand, when you're uploading signed sources you trust
whatever tool cleaned,
patched/assembled/combined/your-git-workflow-here'd and tarballed
your sources. This includes a significantly larger toolchain as
well as the absence of unrelated compromised code on your system
that e.g. detects the "tar" command and swiftly adds+undoes a
backdoor to some file – one attack vector of many. Worse, it's up
to random chance whether anybody will ever look at the source you
just generated, yourself included, given that everything's
supposed to be in git. The xz compromise amply demonstrated this
problem.<br>
</p>
<pre class="moz-signature" cols="72">--
-- mit freundlichen Grüßen
--
-- Matthias Urlichs</pre>
</body>
</html>

--------------uzDnf35M0ZcCXLoBShomK0Zl--

--------------1TqV8A01T0AwFndM2WnVzMjp--

-----BEGIN PGP SIGNATURE-----

wsF5BAABCAAjFiEEr9eXgvO67AILKKGfcs+OXiW0wpMFAmZvQ1AFAwAAAAAACgkQcs+OXiW0wpMx AQ//fJv0GIwLCLF1MVMWKH+btpdzFNmn5q8bbLNJybAAWMO5XFTFMK5ax1gJ1zIRM/cJZkrnlG/T JsVmaX82QR+zTy5TWMF0gwSTqMKZQz/CSS4vUPlgahnI2ws/vifM4SWQc5AsLhgR5kTg8rmYXc9K twZe7AekIpby+G1YW/3a17etvu2wx3yCKsEsAOxp2GQqq7SPlUHpR511UCU2EPbX/pUv94Q1HRzm LIrxPN1oVpGMl5HiYlYZ3gRDS4WnDOXSGQ/fEj7Fx59n37bSgNvttU995qWsNirTQUcOesr+YV9Y 7uv9Pbephlv69S/Alb5A4SAU9K2U2aInj3iesCVMLgqrJj7VhASTBWtHzrbY6EIGhFLtg59PTUOx iY9kocgJFXT0behMECSdmANJdLooqMdQ5dDw48xqdh8CcJ5RTOb7c6U2omX7qemBBFnBYrFzCq9K Kwa9TvCv0K65QxfMYykSMSzBnsIcmNKJ6yNTGmOQ/9lktc9wXMlpoYWlwyyybWfvo6356RUaPXvd HV76mP73s3Eg8fWSxLvN4eCb5bFKiFG6FgVTYO+7YxoHpTGfC4yQopdYEbe3u+3X/NY8t3nEFIt+ c6RXZ6qyFHL/IACwprXMU/QE0uDYGcOtmbOHAgjpCdFlAVZw84k6CXru+OvftEpQudmXyYYpa3kW 7LU=
=pyPB
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05

"Russ" == Russ Allbery <rra@debian.org> writes:

>> 2) Attacker possibly through a compromise of the dgit server and
>> salsa changes the git view to be something harmless.


Sorry I was assuming that the web ui and the git repository were still consistent, but were inconsistent with what was uploaded to the
archive.

I.E.
t =0 attacker uploads something undesirable through tag2upload.

t=1 attacker convinces salsa and dgit to replace that git tree with a
tree that is not undesirable and that has the same hash (so signatures
still verify).

If you look at git, everything looks fine.
If you clone git or dgit, everything looks fine.
If you look at the archive you still see the undesirable code.

I had not considered having the web UI diverge from the git tree. I
think that is entirely unrelated to tag2upload, although I agree it is
scary.

I didn't want to specifically talk about hash collision because there
might be other ways to create similar situations.

However, I think your comment about tag2upload not making this worse may
still apply.
In particular, today it's obvious that I don't need to upload sources
that differ from what I push to salsa (if I do not use dgit).

The only way in which tag2upload really comes into play is that it
increases dgit use and increases the probability that people will assume looking at salsa|dgit is the same as looking at the archive.
And as we've discussed here, that may or may not be true.

Again, I'm really sorry I'm still not finding time to be thorough about
this.
I got krb5 into shape last week.
There's one free software task (thinking about the OSI AI definition discussion) that is ahead of actually spending time on tag2upload on my
queue.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sunday, June 16, 2024 3:59:40 PM EDT Russ Allbery wrote:

Scott Kitterman <debian@kitterman.com> writes:

Yes. I think that's the core of the disagreement. In my view, when I
type the passphrase for my key, I'm asserting responsibility for the contents of what I'm signing. It doesn't mean it is correct or uncompromised, but I am taking responsibility for it.

Right. And I come from a culture that emphasized blameless postmortems
and systems design and a way of thinking about security review from a
similar perspective, which is that assigning responsibility is not in and
of itself a useful thing to do. Just because someone is responsible
doesn't mean that we're more secure. It may mean that you have someone
you can punish afterwards, but it's very questionable how much that helps with security, really.

Assigning responsibility is, in that model, only important to the degree
to which it will change people's actual behavior towards behavior that is more secure, either before or after the fact. If one assigns
responsibility for something that isn't realistically under their control,
or in a way that doesn't cause their behavior to change, the argument is
that nothing is truly accomplished from a security standpoint. It's an illusion of security without actual security.

One of my goals in doing security design is to try to reduce the degree to which humans are performing repetitive validation tasks because humans are not good at maintaining constant vigilance. We know this from a bunch of empircal studies on, for example, airport screening. If a human does a repetitive task with a very low rate of true positives, their attention
will fade and there will be a lot of false negatives. Asking humans to do this is a recipe for failure, and making the humans responsible for doing this correctly and threatening them with consequences for not doing it correctly only slightly decreases the risk of failure.

This is exactly why reproducible builds are so important: that involves finding a way for computers to do the sorts of repetitive validation tasks that computers are good at and that humans are very bad at.

I don't equate responsibility and blame. If I'm responsible for something and it blows up, then that means I'm responsible to help clean up the mess, regardless of if the thing that went wrong is my fault or not.

Not security related, but a couple of times I have (as a member of the FTP Team) removed packages that shouldn't have been removed. Even though it
wasn't particularly my fault (in the cases I'm remembering, the rm bugs had been filled out unclearly or incorrectly, I was still responsible to straighten it out and I did. I don't think that we are that different in our views of what's important, but we do describe it a bit differently.

Scott K

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEE53Kb/76FQA/u7iOxeNfe+5rVmvEFAmZvvbMACgkQeNfe+5rV mvGccRAAggHLfCUkU7Qu3mLfCuBETBOJjN6Hsm7HcmT1dj/Se2a1j7SuPaCrUUU/ gl4DgQhAYB1SSOCCAUvsxVyEvQ0NrVS6JmbzGqZn7/LatzhtWJ4+0sW/KLzPzNZv JMsWhCa1bSDN8YMX8VaGZZ1c5thBcsaEfC/4BNO1exS2gi05Kep0lReSpUuyVupH jgf9Sq7q8aNCuEb+1tutQ2VNffcdstql1+gYxb1JVvHbEJkIZA6hvQDaAJPt7URt 501wDVYXJNtknE0RZEeJL786Q4OLLqmvmJHr3AL9K9cJMOIekHjj980F2iHeZmK2 b5CkA9/vvZWrVBSFM+Tr7EC60hYmjxOl4iOI6dmOka3nSb3k4nhkHRcsKuuzOGFh uxzPDOx2Ar+rPv/5xu8xAE1NTinKe/xZIW/HmEB3cUdumrtb+sxTpWxt6khRmdpC dyey15IjPT+8I2T9bca7FKDk9AphqxYWAWz8wYYj9uHvco1gFVHVdkDH3MAmmHN5 NSLKxx+AtqvsGKv2CypTU/nWTBM0IrNOZbmK5vKz++s2bVtxvsgf7D4jSYR+GtBo byNSIY/eCP3OL05Z4+emk9mvb2PFWmQYR0m8husNCvj0xbVKApnY+sWusPD+MaK2 go+HXWNsfS7nFkpQ24t901T/X+5CTSSmnuOOmX4FT+rxpFmufEQ=
=2ofg
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Scott Kitterman <debian@kitterman.com> writes:

I don't equate responsibility and blame. If I'm responsible for
something and it blows up, then that means I'm responsible to help clean
up the mess, regardless of if the thing that went wrong is my fault or
not.

How is that type of responsibility not correctly represented by
tag2upload? tag2upload is taking responsibility for construction of the
source package from a Git tree. If that blows up, it's the responsibility
of the tag2upload maintainers to help clean up the mess. The maintainer
is declaring responsibility for the Git tree that they signed. If that
blows up, it's their responsibility to help clean up the mess.

--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On June 17, 2024 5:29:02 AM UTC, Russ Allbery <rra@debian.org> wrote:

Scott Kitterman <debian@kitterman.com> writes:

I don't equate responsibility and blame. If I'm responsible for
something and it blows up, then that means I'm responsible to help clean
up the mess, regardless of if the thing that went wrong is my fault or
not.

How is that type of responsibility not correctly represented by
tag2upload? tag2upload is taking responsibility for construction of the >source package from a Git tree. If that blows up, it's the responsibility
of the tag2upload maintainers to help clean up the mess. The maintainer
is declaring responsibility for the Git tree that they signed. If that
blows up, it's their responsibility to help clean up the mess.

I meant it more as a general point about responsibility versus blame in response to your point about a culture of blameless post-incident autopsies.

Given the 5 year latency on this discussion, I'm not particularly convinced that's true, but I wasn't really trying to tie this into the broader discussion.

Scott K

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

T24gMjAyNC0wNi0xNiAyIGggMjMgcC5tLiwgUnVzcyBBbGxiZXJ5IHdyb3RlOg0KPiANCj4g Rm9yIHRoZSBleGlzdGluZyBzb3VyY2UgcGFja2FnZSBzaWduYXR1cmVzLCBhIHNpbXBsaWZp ZWQgc2VxdWVuY2UgbG9va3MNCj4gbGlrZSB0aGlzOg0KPiANCj4gICAgICBodW1hbiAtLT4g KDEpIGRwa2ctYnVpbGRwYWNrYWdlIC0tPiAoMikgZGVic2lnbiAtLT4gKDMpIGFyY2hpdmUN Cj4gDQo+IEZvciB0YWcydXBsb2FkLCBhIHNpbXBsaWZpZWQgc2VxdWVuY2UgbG9va3MgbGlr ZToNCj4gDQo+ICAgICAgaHVtYW4gLS0+ICgxKSBHaXQgLS0+ICgyKSB0YWcydXBsb2FkIC0t PiAoMykgZGVic2lnbiAtLT4gKDQpIGFyY2hpdmUNCg0KUGxlYXNlIGV4Y3VzZSBteSBuYWl2 ZXTDqSwgYnV0IGhvdyBkbyB5b3UgYWN0dWFsbHkga25vdyB0aGF0IHlvdXIgcGFja2FnZSAN CiJ3b3JrcyIgd2l0aCB0aGUgdGFnMnVwbG9hZCB3b3JrZmxvdyBpZiB5b3UncmUgbm90IGJ1 aWxkaW5nIGFueXRoaW5nIA0KbG9jYWxseSBiZWZvcmUgcHVzaGluZz8NCg0KQnkgIndvcmtz IiwgSSBtZWFuLCBob3cgaGF2ZSB5b3UgdGVzdGVkIGl0IHdpbGwgYnVpbGQgYW5kIHdpbGwg cGFzcyBhbGwgDQp0aGUgcHJvcGVyIHByZS11cGxvYWQgdGVzdHM/DQoNCk9uIG15IHNpZGUs IEkgdGVuZCB0byB3b3JrIG9uIGEgR2l0IHRyZWUgYW5kIHdoZW4gSSdtIGhhcHB5IHdpdGgg aXQgSSANCnVzZSBzYnVpbGQgdG86DQoNCjEuIGJ1aWxkIHRoZSBzb3VyY2UgYW5kIHRoZSBi aW5hcnkgcGFja2FnZXMgKGFuZCB0aHVzIHJ1biBidWlsZCB0ZXN0cykNCjIuIHJ1biBMaW50 aWFuDQozLiBydW4gYXV0b3BrZ3Rlc3RzDQoNCk9ubHkgaWYgYWxsIG9mIHRoZXNlIHN0ZXBz IHNlZW0gT0sgd2lsbCBJIGNvbnNpZGVyIHNpZ25pbmcgYW5kIHVwbG9hZGluZyANCnRoZSBy ZXN1bHRpbmcgc291cmNlIHBhY2thZ2UgKGFuZCB5ZXMsIGluIHJlYWxpdHkgd2hhdCBJIGFj dHVhbGx5IGludGVuZCANCnRvIHNpZ24gaXMgdGhlIEdpdCB0cmVlIEkgd29ya2VkIG9uKS4N Cg0KSW1wbGVtZW50YXRpb24gbm90d2l0aHN0YW5kaW5nLCBJJ2QgYmUgbW9yZSB0aGFuIGhh cHB5IHRvIGhhdmUgYSAiZ2l0IA0KJHNvbWV0aGluZyIgcmVwbGFjZSBteSB1c2Ugb2YgZGVi c2lnbiBhbmQgZHB1dCwgYnV0IEkgYW0gZ2VudWluZWx5IA0KY3VyaW91cyB0byBrbm93IHdo eSB3ZSB3b3VsZCBtYWtlIGl0IGVhc2llciB0byB1cGxvYWQgc29tZXRoaW5nIHRoYXQgDQpo YXNuJ3QgcGFzc2VkIHdoYXQgSSBiZWxpZXZlIGFyZSBpbXBvcnRhbnQgUUEgc3RlcHMgYmVm b3JlIHVwbG9hZGluZz8NCg0KQW5kcmVhcyBUaWxsZSBhbHJlYWR5IHJhaXNlZCB0aGF0IHBv aW50IGluIGFub3RoZXIgdGhyZWFkLCBidXQgdGhlIA0KYW5zd2VyIHNlZW1zIHRvIGhhdmUg YmVlbiB0aGF0IGl0J3MgYWxyZWFkeSBwb3NzaWJsZS4gSW5jZW50aXZpc2luZyBzdWNoIA0K YSBiZWhhdmlvciBkb2Vzbid0IHNvdW5kIHBvc2l0aXZlIHRvIG1lLg0KDQotLSANCiAgIOKi gOKjtOKgvuKgu+KituKjpuKggA0KICAg4qO+4qCB4qKg4qCS4qCA4qO/4qGBICBMb3Vpcy1Q aGlsaXBwZSBWw6lyb25uZWF1DQogICDior/ioYTioJjioLfioJrioIsgICBwb2xsb0BkZWJp YW4ub3JnIC8gdmVyb25uZWF1Lm9yZw0KICAg4qCI4qCz4qOEDQoNCg==

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Quoting Louis-Philippe Véronneau (2024-06-17 07:40:51)

On 2024-06-16 2 h 23 p.m., Russ Allbery wrote:

For the existing source package signatures, a simplified sequence looks like this:

human --> (1) dpkg-buildpackage --> (2) debsign --> (3) archive

For tag2upload, a simplified sequence looks like:

human --> (1) Git --> (2) tag2upload --> (3) debsign --> (4) archive

Please excuse my naiveté, but how do you actually know that your package "works" with the tag2upload workflow if you're not building anything
locally before pushing?

By "works", I mean, how have you tested it will build and will pass all
the proper pre-upload tests?

As I understand it, neither of above processes ensure that.

More accurately, (1) in the first sequence is `dpkg-buildpackage -S`
(which might be done as part of a larger process, not called as such, but
the *functionality* narrowly relevant for the sequence is only that the
human uses a tool to produce a _source_ package (not whatever other tasks
the human may or may additionally do).

On my side, I tend to work on a Git tree and when I'm happy with it I
use sbuild to:

1. build the source and the binary packages (and thus run build tests)
2. run Lintian
3. run autopkgtests

Only if all of these steps seem OK will I consider signing and uploading
the resulting source package (and yes, in reality what I actually intend
to sign is the Git tree I worked on).

Implementation notwithstanding, I'd be more than happy to have a "git $something" replace my use of debsign and dput, but I am genuinely
curious to know why we would make it easier to upload something that
hasn't passed what I believe are important QA steps before uploading?

Andreas Tille already raised that point in another thread, but the
answer seems to have been that it's already possible. Incentivising such
a behavior doesn't sound positive to me.

I rarely call `dpkg-buildpackage -S` directly. Instead I call `debuild`,
or some wrapper around that.

If tag2upload becomes part of Debian, I would expect debuild and/or some
of its wrappers to suppor hooking into tag2upload, for a single command
to do both test-building and signing off.

In summary, I don't see how this is any different from what we have
today.

- Jonas

--
* Jonas Smedegaard - idealist & Internet-arkitekt
* Tlf.: +45 40843136 Website: http://dr.jones.dk/
* Sponsorship: https://ko-fi.com/drjones

[x] quote me freely [ ] ask before reusing [ ] keep private --==============331390175793389959=MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Description: signature
Content-Type: application/pgp-signature; name="signature.asc"; charset="us-ascii"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEn+Ppw2aRpp/1PMaELHwxRsGgASEFAmZv04QACgkQLHwxRsGg ASFE4A/9EDOFHLfUFet9H2oVDc0gd8I36pRUsNIeVmig2sWKuPG5miwG5UQbUFci wWNjUGxvtxv0e0XHj0I4mB2ExSu15fbMyvNqRCvfhMDsBp6WPB6LZ1zaWbQUaa5l TXODPCzFbn/odRpJsPHkYo40N8QLXAc8p4A6A1Igqp9B+ey8xnIiUZgpZ0xP/aaz yf9FTrZiZoNfeqKV6vo5XLQh82zc0NutW6POHiaKaDxB0Lwhl4FesgvrkNsmWIoI kAQdmTL3AajLU9ZQhrSFSxOS5QVE9ejcVIW/Fr4WmJuuuhmMp/oMJRIpK9iAAxKl /HvV4iTRC4B8j85iGA6yewJCntha5hNhn4EH/ZEH6O4BZeNyk4z6hkQFJNkdM9ya GWg8VJUctVGjCZwnFzVDXB+7XWQQy0dWzFzcSN1aGc//Y2dWcWU+pSc8Io6hgdgI Xk87UUDcrPEWfS9q4beDVgn64uOGMLPiP5Ir/1TD

T24gMjAyNC0wNi0xNyAyIGggMTEgYS5tLiwgSm9uYXMgU21lZGVnYWFyZCB3cm90ZToNCj4g SSByYXJlbHkgY2FsbCBgZHBrZy1idWlsZHBhY2thZ2UgLVNgIGRpcmVjdGx5LiAgSW5zdGVh ZCBJIGNhbGwgYGRlYnVpbGRgLA0KPiBvciBzb21lIHdyYXBwZXIgYXJvdW5kIHRoYXQuDQo+ IA0KPiBJZiB0YWcydXBsb2FkIGJlY29tZXMgcGFydCBvZiBEZWJpYW4sIEkgd291bGQgZXhw ZWN0IGRlYnVpbGQgYW5kL29yIHNvbWUNCj4gb2YgaXRzIHdyYXBwZXJzIHRvIHN1cHBvciBo b29raW5nIGludG8gdGFnMnVwbG9hZCwgZm9yIGEgc2luZ2xlIGNvbW1hbmQNCj4gdG8gZG8g Ym90aCB0ZXN0LWJ1aWxkaW5nIGFuZCBzaWduaW5nIG9mZi4NCj4gDQo+IEluIHN1bW1hcnks IEkgZG9uJ3Qgc2VlIGhvdyB0aGlzIGlzIGFueSBkaWZmZXJlbnQgZnJvbSB3aGF0IHdlIGhh dmUNCj4gdG9kYXkuDQoNClRoYXQncyBoZWxwZnVsLCB0aGFua3MhDQoNCi0tIA0KICAg4qKA 4qO04qC+4qC74qK24qOm4qCADQogICDio77ioIHioqDioJLioIDio7/ioYEgIExvdWlzLVBo aWxpcHBlIFbDqXJvbm5lYXUNCiAgIOKiv+KhhOKgmOKgt+KgmuKgiyAgIHBvbGxvQGRlYmlh bi5vcmcgLyB2ZXJvbm5lYXUub3JnDQogICDioIjioLPio4QNCg0K

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------sjOqmkIiCXrzUF7LXlyvficJ
Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64

T24gMTcuMDYuMjQgMDc6NDAsIExvdWlzLVBoaWxpcHBlIFbDqXJvbm5lYXUgd3JvdGU6DQo+ IFBsZWFzZSBleGN1c2UgbXkgbmFpdmV0w6ksIGJ1dCBob3cgZG8geW91IGFjdHVhbGx5IGtu b3cgdGhhdCB5b3VyIA0KPiBwYWNrYWdlICJ3b3JrcyIgd2l0aCB0aGUgdGFnMnVwbG9hZCB3 b3JrZmxvdyBpZiB5b3UncmUgbm90IGJ1aWxkaW5nIA0KPiBhbnl0aGluZyBsb2NhbGx5IGJl Zm9yZSBwdXNoaW5nPyANCg0KV2hlbiBJIGNoYW5nZSB0aGUgVXBzdHJlYW0gc291cmNlcyB3 aXRob3V0IHRvdWNoaW5nIHRoZSBwYWNrYWdpbmcgDQooc2VjdXJpdHksIHJjMT5yYzIsIGV0 Yy4pIEkgbWlnaHQgcnVuICJtYWtlIiBsb2NhbGx5LCBqdXN0IHRvIGJlIHN1cmUgbXkgDQpw YXRjaCBpcyBwZWFjaHkuIEkgdGhlbiAiZ2l0IGNvbW1pdCIgYW5kIHB1c2ggdGhlIChtaW5p bWFsKSByZXN1bHQuIEknbSANCm5vdCBnb2luZyB0byBydW4gdGhlIHdob2xlICJhc3NlbWJs ZSBhbmQgY29tcHJlc3MgdGhlIHNvdXJjZSBwYWNrYWdlIA0KanVzdCB0byB0aHJvdyBpdCBh d2F5IiBrYWJvb2RsZSBpZiBJIGNhbiBwb3NzaWJseSBoZWxwIGl0Lg0KDQpBbHNvLCB3aGVu IEknbSBvbiBhIHdvbmt5IEludGVybmV0IGNvbm5lY3Rpb24gKGxpa2UgYSB0cmFpbiwgb3Ig cnVyYWwgDQpHZXJtYW55IChkb24ndCBhc2sg4oCmICpzaWdoKikpIHB1c2hpbmcgYW4gaW5j cmVtZW50YWwgZ2l0IHRhZyB0YWtlcyB1cCBhIA0KbG90IGxlc3MgYmFuZHdpZHRoLCBoZW5j ZSBoYXMgbW9yZSBjaGFuY2Ugb2Ygc3VjY2VlZGluZywgdGhhbiB1cGxvYWRpbmcgDQphIHRh cmJhbGwuDQoNCi0tIA0KLS0gcmVnYXJkcw0KLS0gDQotLSBNYXR0aGlhcyBVcmxpY2hzDQoN
Cg==

--------------sjOqmkIiCXrzUF7LXlyvficJ--

-----BEGIN PGP SIGNATURE-----

wsF5BAABCAAjFiEEr9eXgvO67AILKKGfcs+OXiW0wpMFAmZv4l8FAwAAAAAACgkQcs+OXiW0wpNz ZhAAu0FovPhaq4WHIeh4FbyjFodaWqhr+RZOuVDz7ZGWk+D/wQ6r40k6zeBuKzqMV03kIaUMK+pU PTTLN0+yhVFtzdo9neljiAn09mOporJQJth4C/Hq+qzjBZvMYuNIcW1DZuVt7YToMCYWfGiXjsfJ XYr7SFFVtS7UUh1ETk2X7yAIFqnpzX1X94Deg2lHO84Kc6JudTPYld0Rt+9cTFdHd8PjkBTpMtej 2mNM4WL0CcutkHU+69xt2O/T9BpN8cMN3YE9QLZfz6Jj7jsFvfbs5pWSkJAme+cee1c8YcfzafSs mymt1xk2/+iFiSIueYNWhEdjvwiY+KN7sCveOuq//SrJJx8sRtpfyQ0R9JI7Y6UTkTzDB6alOMAr 7i530FCoPqEfyJE9eyRXCd/4Km31ixbqmKWgOuw4ve+eoa0AUIVjkPcIizvEpbKQpKF1DaHoHOUF KjHBStF9FweKuA4duipildL/2ZRSiqrp/R95MCwCVpzC5ZsNMmcgFDWIbp8Yc8WEZevgazg1j67w GsD+Waa130rFNabdOJrE30iJz5HfoRIC0eSJtSIUORf1BawLSyELQvtHAE5RX35aJVDU/E7Vl62C uxkn4Y7qKYfhFkU75CvECvN97wsmUERpxvGr4q0FcENvZzWyF5PSc5L05pF4Y9kmOKCPs9jmbDKS Yc0=
=BX30
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Russ Allbery <rra@debian.org> writes:

Scott Kitterman <debian@kitterman.com> writes:

I agree that there's a risk that what the uploader thought they were
uploading and what they actually uploaded are different, but that's
independent of tag2upload or not.

But it's not independent; tag2upload makes this story somewhat better. tag2upload is based on a signed Git tag and moves the source package construction off of the uploader's system onto more-secure project infrastructure.

I've seen this notion repeated a couple of times now, and I don't think
it is a good argument nor that it is particulary relevant to tag2upload.

One problem is that the tag2upload design aggregates and amplify the consequences of one successful compromise. Thus the standards of
security of that single centralized machine has to be higher than the
standards for security of developer machines, since there is a more
limited number of things each individual developer machine can do.

A more reasonable security comparison here is to compare the security of
the tag2upload design with the security of compromising ALL of the
Debian developers. Both goals leads to similar abilities for the
attacker. You have to look at security consequences from the attacker
point of view, not the point of view of each (hopefully benign) user of
the system.

Successfully attacking ALL individual developers, with each own
individual security weaknesses, seems to me more costly than attacking a
single known publicly run instance like tag2upload or Salsa. So now
attackers are probably put off attacking large subsets of individual
developer machines, but may be more interested in attacking the
tag2upload or Salsa box if that leads to the same ability with lower
cost.

Debian has to my knowledge not published any information how private
keys on centralized machines are protected and managed. Some developers
(like myself) do publish information how they store and protect their
private keys. This suggests to me that the argument that centralized
Debian systems are unconditionally more secure than individual
developers systems is not valid.

My perspective is that for all projects that rely on non-free
software/hardware we have to assume that the output is compromised,
since it is impossible to do a complete audit of the components.

Still, I think Debian is already too deep in too many security and trust
issues already that adding another attack vector will not break this
camel's back. We are collectively good at handling technical/security incidents when they become public, so let's utilize that property to
allow people to do what they want. The bigger problem in Debian is IMHO
how some people use their technical powers to limit what other people
can do effectively, and I think this is harmful.

/Simon

-----BEGIN PGP SIGNATURE-----

iIoEARYIADIWIQSjzJyHC50xCrrUzy9RcisI/kdFogUCZm/5LBQcc2ltb25Aam9z ZWZzc29uLm9yZwAKCRBRcisI/kdFok8XAQD8t+jpeNNK3PwAtd7RcwNnf/Doc+nR 3JqGjNX1HP1pjQD9HEUMd96wOzKUMGwFEGu4q6Com7ASphelpF8aeqh7OAU=
=JHJx
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Simon Josefsson <simon@josefsson.org> writes:

Successfully attacking ALL individual developers, with each own
individual security weaknesses, seems to me more costly than attacking a single known publicly run instance like tag2upload or Salsa.

You only need to be able to sucessfully attack *one* developer in order
to cause significant damage.

The more popular that developers packages are, the more damage you can
do.

So the developer with the weakest security practises and most popular
packages is probably a prime candidate.
--
Brian May @ Debian

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sun, 16 Jun 2024 at 19:00, Scott Kitterman <debian@kitterman.com> wrote:

I can see that, but that leads to what I view as a problem. The thing in the archive is signed by a machine, not the human who decided it should be uploaded.

That is nothing new or particularly controversial for Debian Archive.

Nowadays, most uploads are source-only, so all binary packages are
built by buildd servers and thus are signed by a machine.
Even before that, the key index files like Release and Packages in the
archive have always been generated and signed by a machine.

Those are things that actual end-users use and that can directly
compromise millions of machines in a few hours. Pretty
important stuff IMHO.

Compared to that, a machine signing a Debian-specific intermediate
source package artifact (that normally is only
used by Debian buildd servers) sounds quite inconsequential.

The key is that a human initiated the chain of events and signatures
with an authenticated signature. Where that happens does not
really matter all that much and has changed in the past in Debian already.
--
Best regards,
Aigars Mahinovs mailto:aigarius@debian.org
#--------------------------------------------------------------#
| .''`. Debian GNU/Linux (http://www.debian.org) |
| : :' : Latvian Open Source Assoc. (http://www.laka.lv) |
| `. `' Linux Administration and Free Software Consulting |
| `- (http://www.aiteki.com) |
#--------------------------------------------------------------#

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Brian May <bam@debian.org> writes:

Simon Josefsson <simon@josefsson.org> writes:

Successfully attacking ALL individual developers, with each own
individual security weaknesses, seems to me more costly than attacking a
single known publicly run instance like tag2upload or Salsa.

You only need to be able to sucessfully attack *one* developer in order
to cause significant damage.

The more popular that developers packages are, the more damage you can
do.

So the developer with the weakest security practises and most popular packages is probably a prime candidate.

Agreed. That is already the case, and tag2upload won't change this.

I was talking about comparing the tag2upload design with the current
design (Russ's claim "tag2upload makes this story somewhat better).

I don't think it is a good claim to say that tag2upload will be a
security improvement because "the central server is more secure" than
the weakest developer machine. To compare apples with apples, from the attacker point of view, you would have to claim that "the central server
is more secure" than ALL developers. Otherwise you aren't comparing the
same attacker ability after successful attack. I don't think it is
reasonable to claim that the central tag2upload server will be more
secure than ALL developer machines.

I believe it would be more honest to admit that tag2upload will lower
the security story for package uploads, and argue that it is still the
right thing to do for other reasons.

Generally, I think the only way to improve security of a system is to
REMOVE functionality, never by adding more new complex functionality. I
don't see anyone suggesting we will remove any functionality here. More security audits of tag2upload are great, and will improve confidence in
it, but claiming that tag2upload will improve overall security of Debian
seems like overstating what it delivers.

/Simon

-----BEGIN PGP SIGNATURE-----

iIoEARYIADIWIQSjzJyHC50xCrrUzy9RcisI/kdFogUCZnAduhQcc2ltb25Aam9z ZWZzc29uLm9yZwAKCRBRcisI/kdFojWaAP9X4JACQSH7uCUjvXNQ2+F0ZPVaKr9p 0Xk8BfoR8zTiDwD9FPbEqGKGwbVp5Xgz0YFNpnNAPgdxgaqTKU91MwTcDQk=
=110b
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Russ Allbery writes ("Re: Security review of tag2upload"):

I think it would be hugely valuable to have something like a "dgit verification mode" where you can ask dgit, which already has all the
source package construction logic, to take a tag2uplod-generated source package, start from the tag object and signature, and reproduce that
source package and verify it. Except for the retrieval of the signed Git tag, in theory all of that could be done locally. I'm not sure how hard
that would be (this comes back to the question of how difficult it is to ensure that the tag2upload source construction algorithm is easily reproducible), but I think something like that would go a long way towards providing some really interesting security properties.

I think doing a fairly good job of this is a fairly simple shell
script.

1. Check out the maintainer view git tag
(DEP-14, signed by the maintainer).
Run dgit quilt-fixup with an appropriate --quilt= option
(this is the mode that does only the git canonicalisation).
If you want to do this automatically you need to read the quilt
mode out of the maintainer tag, by parsing the tag2upload metadata.

2. Run dgit import-dsc on the allegedly-corresponding .dsc
from the tag2upload service.

The results of (1) and (2) should be treesame. This ought to be very
stable, because the precise correspondence between the various git
views and .dsc contents is part of dgit's data model.

You can try this out right now with dgit-repos, since `dgit push` is
supposed to have basically the same properties.

That isn't bit-for-bit reproducibility of the whole .dsc. I think in
practice the output of `dgit build-source` may well already be
bit-for-bit reproducible (assuming identical versions of all the
programs involved ioncluding gbp, quilt, git, patch, dpkg-source,
etc. etc. etc.), but I haven't tested this.

Ian.

--
Ian Jackson <ijackson@chiark.greenend.org.uk> These opinions are my own.

Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------qt04Xp8pnuNEe8nuhChtryOI
Content-Type: multipart/alternative;
boundary="------------Tg0L6nELYLp1mxrqPbheNPd3"

--------------Tg0L6nELYLp1mxrqPbheNPd3
Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64

T24gMTcuMDYuMjQgMTA6NTEsIFNpbW9uIEpvc2Vmc3NvbiB3cm90ZToNCj4gU3VjY2Vzc2Z1 bGx5IGF0dGFja2luZyBBTEwgaW5kaXZpZHVhbCBkZXZlbG9wZXJzLCB3aXRoIGVhY2ggb3du DQo+IGluZGl2aWR1YWwgc2VjdXJpdHkgd2Vha25lc3Nlcywgc2VlbXMgdG8gbWUgbW9yZSBj b3N0bHkgdGhhbiBhdHRhY2tpbmcgYQ0KPiBzaW5nbGUga25vd24gcHVibGljbHkgcnVuIGlu c3RhbmNlIGxpa2UgdGFnMnVwbG9hZCBvciBTYWxzYS4NCg0KVGhlIHRoaW5nIGlzLCB5b3Ug ZG9uJ3QgbmVlZCB0byBoYWNrIEFMTCBvZiB0aGVtIHRvIHN1Y2NlZWQuIFlvdSBvbmx5IA0K bmVlZCBvbmUg4oCTIHRoZSBvbmUgd2l0aCB0aGUgd29yc3Qgc2VjdXJpdHktdG8tdXNlZnVs bmVzcyAodG8gdGhlIA0KYXR0YWNrZXIpIHJhdGlvLCBhcyBYWiBhbXBseSBkZW1vbnN0cmF0 ZWQuIEF0IHRoYXQgcG9pbnQgaXQncyBnYW1lIG92ZXIuDQoNCkFsc28sIHdlIGNhbiBhdWRp dCB0MnUgaWYgd2UgZGVjaWRlIHRvLiBXZSBjYW4gd3JpdGUgYSBzZWNvbmQgDQppbXBsZW1l bnRhdGlvbiwgaW4gYSBkaWZmZXJlbnQgbGFuZ3VhZ2UgYW5kIHVzaW5nIGEgZGlmZmVyZW50 IGNvbnRhaW5lciANCnJ1bnRpbWUgZm9yIHJ1bm5pbmcgZGdpdCwgYW5kIHZlcmlmeSB0aGF0 IHRoZSBvdXRwdXQgaXMgdGhlIHNhbWUuIE9yIHdlIA0KY2FuIGltcGxlbWVudCB0aGUgdGFz ayBpbiBkYWsgZGlyZWN0bHkuDQoNCldlIGNhbiBoYXJkZW4gdGhlIGNvbnRhaW5lci4gSGVj ayB3ZSBjYW4gZXZlbiB1c2UgYSBzZXBhcmF0ZSBjb21wdXRlciBpZiANCndlIGRlY2lkZSB0 aGF0IFZNcyBhcmVuJ3QgZ29vZCBlbm91Z2gsIGRvIGEgaGFyZCByZWJvb3QgYmV0d2VlbiAN CnBhY2thZ2VzLCBhbmQgdXNlIGRtLXZlcml0eSB0byBlbnN1cmUgdGhhdCBpdHMgZGlzayBp c24ndCB0YW1wZXJlZCB3aXRoLCANCmJ1dCB0aGF0J3Mgd2F5IGJleW9uZCB3aGF0IG91ciBi dWlsZGRzIGFyZSBkb2luZyBzbyBwcm9iYWJseSBvdmVya2lsbC4NCg0KV2UgY2Fubm90IGF1 ZGl0IChub3QgYXMgZWFzaWx5IGFueXdheSkgc291cmNlIHBhY2thZ2VzIHRoYXQgZ2V0IGJ1 aWx0IG9uIA0Kc29tZSBERCdzIGxhcHRvcC4NCg0KLS0gDQotLSByZWdhcmRzDQotLSANCi0t IE1hdHRoaWFzIFVybGljaHMNCg0K
--------------Tg0L6nELYLp1mxrqPbheNPd3
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">On 17.06.24 10:51, Simon Josefsson
wrote:<br>
</div>
<blockquote type="cite" cite="mid:87iky8uf2r.fsf@kaka.sjd.se">
<pre>Successfully attacking ALL individual developers, with each own individual security weaknesses, seems to me more costly than attacking a
single known publicly run instance like tag2upload or Salsa. </pre>
</blockquote>
<p>The thing is, you don't need to hack ALL of them to succeed. You
only need one – the one with the worst security-to-usefulness (to
the attacker) ratio, as XZ amply demonstrated. At that point it's
game over.</p>
<p>Also, we can audit t2u if we decide to. We can write a second
implementation, in a different language and using a different
container runtime for running dgit, and verify that the output is
the same. Or we can implement the task in dak directly.</p>
<p>We can harden the container. Heck we can even use a separate
computer if we decide that VMs aren't good enough, do a hard
reboot between packages, and use dm-verity to ensure that its disk
isn't tampered with, but that's way beyond what our buildds are
doing so probably overkill.<br>
</p>
<p>We cannot audit (not as easily anyway) source packages that get
built on some DD's laptop.<br>
</p>
<pre class="moz-signature" cols="72">--
-- regards
--
-- Matthias Urlichs</pre>
</body>
</html>

--------------Tg0L6nELYLp1mxrqPbheNPd3--

--------------qt04Xp8pnuNEe8nuhChtryOI--

-----BEGIN PGP SIGNATURE-----

wsF5BAABCAAjFiEEr9eXgvO67AILKKGfcs+OXiW0wpMFAmZwEN0FAwAAAAAACgkQcs+OXiW0wpP+ bQ//XAml7bLdkZgnz9WvZu1h+aDa3ItRIqeCaIoMg/xfZFOImHL43LsQQKWJV3LX2H5jdEhrrdZs sgQ6QZR1oxIgiZy3PmnSBbS9/dFeC1lWlZTGLyo941gW4bNspims9VRCBQdPcg8ZThYrKpNFvv3/ /9wDUccDlYHIjhf55Fvq1UnryjZqYon2ZLCV5PI2oSgC4W31Irp+f1mAE+4QG6tbZnGCFdifHM7T 9Rks7zyP0K5+N2DsTI03/heLJ+F98EX/GdYNGEWfWk1qTud+WSdu3LGxCFaYUbXulyPNEP/sYap5 TSoeOSmUo/IMTu8iN4TX6jMv25s0W81SF6T7kJv3+EbUtFlYWbymroRbvbL6ZhtEdzsqgce3xLcr moS6N1eBaVsOv4vYQqodtR6qtlkG/TSQMznRy58lBFIzucNv+71XiT7l6jybI+eDEF524LO4v9wB XdUtF/szv7vNEN7DblDSG5fP2jJTfjmr2oZa9gLFM0XlIaLIVxEJHCTJH4qbnRgwfobDryOdqKr5 gvmi/Xw5O71kbdS9mKd+8fhi/puxtZsUZmcRjUNFG6q8CxLr4CvDq7cVJFAxUMqcWzhZWyb15TsP fblrTP0EExlLCxBEfdMEgJNGkQhk1DqrmCsmUCjwv32GNoop+2v2g99UqtJfAvRblb2xrk0Qxhwq nIg=
=EsoQ
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05

Russ Allbery writes ("Re: Security review of tag2upload"):

[...] in the general case we have absolutely no idea how to
map a source package in the archive back to a Git tree. That's exactly
the problem that tag2upload is trying to solve. For non-tag2upload
packages, we still have to rely on the source package as the farthest back that we can trace the code without diverging into package-specific
analysis and diverging maintainer workflows.

In fact, you *can* trace this back further if the uploader used
`dgit push`. The .dsc contains a Dgit: field naming a commit,
and dgit-repos contains all the data you need to trace that back to
the maintainer's git branch, and has the maintainer's signed tag.

This formal, standardised, link between the .dsc and git is one of the
biggest reasons why using dgit push is a good thing. Of course it
mostly benefits downstrems and users, so it's less visible to the
maintainer.

With `dgit push` the correspondence between source package and git
branch is assured on the uploader's machine, rather than a central
service, so it could be breached by attacks or serious malfunctions.

The point of tag2upload is to move all this stuff from the
maintainer's machine to a central service, where it is more reliable,
more secure, and more convenient.

Ian.

--
Ian Jackson <ijackson@chiark.greenend.org.uk> These opinions are my own.

Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Louis-Philippe Véronneau <pollo@debian.org> writes:

On 2024-06-16 2 h 23 p.m., Russ Allbery wrote:

For the existing source package signatures, a simplified sequence looks
like this:
human --> (1) dpkg-buildpackage --> (2) debsign --> (3) archive
For tag2upload, a simplified sequence looks like:
human --> (1) Git --> (2) tag2upload --> (3) debsign --> (4)
archive

Please excuse my naiveté, but how do you actually know that your package "works" with the tag2upload workflow if you're not building anything
locally before pushing?

Jonas is correct that this is a highly simplified picture intended only to highlight the security properties, and I personally will do all sorts of
other things locally to verify my work before I decide to upload it.

But that's a boring answer; the really interesting answer for the purposes
of what tag2upload could accomplish is "Salsa CI." Salsa is already
capable of doing a lot of those tests, like running Lintian and
autopkgtests, so if you have a good suite of autopkgtests, it can do most
of the heavy lifting for you. You can put up a merge request, walk away
and have a cup of coffee or work on some other problem, and come back to
see if it found any issues.

This is a development model used a lot outside of Debian: development
consists mostly of reviewing and merging merge requests in a forge, the
forge CI takes care of testing every change with every useful test that
people have accumulated for the software package, and when it seems ready
for release, that release is done via Git tag.

Doing local testing then becomes an option that you can use if you want a faster iteration cycle or better logs or local information than the CI can
do, but routine patches can be tested only with CI without bothering to
set up a full local development environment.

--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Simon Josefsson <simon@josefsson.org> writes:

Russ Allbery <rra@debian.org> writes:

But it's not independent; tag2upload makes this story somewhat better.
tag2upload is based on a signed Git tag and moves the source package
construction off of the uploader's system onto more-secure project
infrastructure.

I've seen this notion repeated a couple of times now, and I don't think
it is a good argument nor that it is particulary relevant to tag2upload.

I hear what you're saying, I think your analysis of the security trade off
is partly correct, and I still disagree about the conclusion: I think tag2upload makes this better. I think the level of security that we can provide to a central host really is that much higher than the level of
security on individual uploader machines, even though this centralizes
risk.

One problem is that the tag2upload design aggregates and amplify the consequences of one successful compromise. Thus the standards of
security of that single centralized machine has to be higher than the standards for security of developer machines, since there is a more
limited number of things each individual developer machine can do.

Yes, agreed. This is standard for a "put all your eggs in one basket and protect that basket" design. As with dak and with buildds, we should try
to use as many hardening approaches as we can manage for the tag2upload
server.

A more reasonable security comparison here is to compare the security of
the tag2upload design with the security of compromising ALL of the
Debian developers. Both goals leads to similar abilities for the
attacker.

This, however, I will object to. I can see the argument *if* tag2upload completely broke the provenance chain and didn't publish any information
about how to reproduce its work. Then one could argue that it would be equivalent to the source package construction on each maintainer system,
but centralized. However, that's not the case.

Compromising the tag2upload server gives you the ability to upload source packages purporting to be from arbitrary Debian developers, but the
detection and tracing is much better than compromising every developer
system. The information required to figure out that the tag2upload server
is compromised is published, and I have a lot of hope that we can automate
that detection.

Compare that situation to a compromise of an uploader's system. Even if
they keep their signing key secure (on a hardware device, on a separate
system from where they do source package builds), they are likely to sign malicious code without noticing if it is injected at the right point. And
in that case there is then no trace in the archive that we could use to
detect that this has happened except for the malicious code itself.

You have to look at security consequences from the attacker point of
view, not the point of view of each (hopefully benign) user of the
system.

Yes, I completely agree.

Successfully attacking ALL individual developers, with each own
individual security weaknesses, seems to me more costly than attacking a single known publicly run instance like tag2upload or Salsa.

I also agree with this; I just don't think these cases are equivalent.

Debian has to my knowledge not published any information how private
keys on centralized machines are protected and managed.

My understanding is that dak stores its key in a hardware device so that
the private key cannot be exfiltrated by a dak compromise, and the plan is
to use the same design for the tag2upload server. That's been mentioned
deep inside this thread, but I forget if that is explicitly called out in
the design. If not, it would be a good thing to mention.

I'm all in favor of more documentation about Debian as a whole documents private keys.

My perspective is that for all projects that rely on non-free software/hardware we have to assume that the output is compromised,
since it is impossible to do a complete audit of the components.

I think that's an interesting ideological position but not really that defensible of a security analysis. That said, if there are good OpenPGP hardware keys that are also free software and hardware, I'm certainly in
favor of using them if feasible.

--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

This message from a while back was still in my queue to respond to, and
since I was updating my security review anyway, I wanted to think through
it a bit more. I think I understand what Sam is getting at.

Sam Hartman <hartmans@debian.org> writes:

Sorry I was assuming that the web ui and the git repository were still consistent, but were inconsistent with what was uploaded to the
archive.

I.E.
t =0 attacker uploads something undesirable through tag2upload.

t=1 attacker convinces salsa and dgit to replace that git tree with a
tree that is not undesirable and that has the same hash (so signatures
still verify).

If you look at git, everything looks fine.
If you clone git or dgit, everything looks fine.
If you look at the archive you still see the undesirable code.

I see, thank you!

A Git hash collision is sufficient to perform this attack on Salsa. I do
not believe a Git hash collision gets an attacker far enough to perform
this attack on the dgit-repos server unless the dgit-repos server is also compromised. This is where it's append-only nature helps considerably.
Even if there is a hash collision, the first instance of that hash pushed
to the server will win. Subsequent pushes of an object with the same hash
will be ignored because the server already has that hash. Unlike with
Salsa, there is no way to delete the references to that object and force
it to be pruned so that a different object can be pushed. I therefore
believe it's not possible to perform the object substitution attack that attacks via hash collisions rely on unless the server is compromised and
the attacker can delete or replace objects.

*If* one verifies the signature on the Git tag, I'm not sure how one can perform an attack like this without a Git hash collision, a compromise
of the OpenPGP signing protocol or signature algorithm, or some similar cryptographic attack.

A more interesting case is if people do this investigation without
verifying the Git tag signature. That step is vital in the case of a
Debian infrastructure compromise. If the personal credentials of someone
with general Debian infrastructure access, and therefore administrative
access to all of dak, tag2upload, Salsa, and dgit-repos, are compromised,
the attacker is then in a position to invent a whole new view of the
archive with whatever properties that they wish. At that point, our last
line of defense is uploader Git signatures, whether over the source
package (the current case) or Git tags (the tag2upload case).

One interesting fine point here is that the tool that retrieves the Git
tag and corresponding Git tree from dgit-repos to verify it should use transfer.fsckObjects to ensure the integrity of the resulting Git tree.
This protects against some types of compromise of the dgit-repos server.

I *think* this scenario is reasonably well-covered in the current draft of
my security review, which emphasizes that checking the signature on the
Git tag is necessary to detect several attacks that involve compromise of
the tag2upload server.

It is of course possible that there will be some new comprehensive attack
on Git tags or on OpenPGP signatures that we haven't anticipated.

However, I think your comment about tag2upload not making this worse may still apply.
In particular, today it's obvious that I don't need to upload sources
that differ from what I push to salsa (if I do not use dgit).

Right, the tracing we have today, for packages that don't use dgit, is
entirely ad hoc and may not be possible at all.

The only way in which tag2upload really comes into play is that it
increases dgit use and increases the probability that people will assume looking at salsa|dgit is the same as looking at the archive.
And as we've discussed here, that may or may not be true.

Yes, this is also a good point. If people rely on dgit-repos to be an
accurate reflection of the archive, a compromise of the dgit-repos server allows the attacker to deceive people about the archive contents.

Reproducible source builds detect a wide variety of these attacks,
particularly if they are ongoing (in other words, not just on initial
upload, but periodically pulling all the source packages in the archive
and reproducing them again).

--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

To: debian-vote@lists.debian.org

This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------c3IZmHHQKv0gcUyicOzTdm7w
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Russ Allbery:

Below is the security review that I did of the tag2upload design.
[...]
The existing upload architecture requires trusting the host used by the uploader to build the source package. If that host is compromised, an attacker could inject malicious code into the source package, either by modifying the upstream tar file (if signed upstream tar files are not
used) or by injecting it into the Debian package build system, maintainer scripts, or patches.

This attack is not equivalent to compromise of the uploader's OpenPGP key, which neither upload architecture defends against. Many Debian uploaders build source packages on less-trusted systems where they also build and
test binary packages, and then sign the source package from a more-trusted system or use a hardware key.

Is this really common practice that Debian uploaders sign (source)
packages they built on less-trusted systems?

And, if yes: Why wouldn't they do the equivalent with the sources in git
(work on the less trusted system, transfer commits (git push/pull) to
the system with signing access and sign there, without review)?

--------------c3IZmHHQKv0gcUyicOzTdm7w--

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEqieyzvOmi9FGaQcT5KzJJ4pkaBYFAmZ5+LIACgkQ5KzJJ4pk aBaDww//VPs0oSh4Wzc0aRn5z7T7B3sRAAkwk70Klsc3kZWVXZigzDI3ylNxraLQ 4uckXbStWKC91+P88/rjpRtL15at/qsp31hbp83pf5Xv+nCnZyLesyaFisER/DPS GvB9kjQczEWxaDyyZ+VtkFB+x4vBGuzW2mrOWoBBJAOHnLVOPfGTDWmWZs8BThTS jsweO9gQ57xRTaoMCSavqYTSV7u1sWK24UVbPBtkV08/Lj9MC4OyBotZCr5YmLmC /t+HED+xArof+4NJaAYmPDGz7QotwCaEQ1EAgvX/t9ick9oZqxhOhDl67gG49RlY lTMkfQ3ZvkfX9gzAfbKqi2HPwcsnzmgxggKpm1K1mSlp+4C5EJHypgFvzd9LuvHY 6jOBl5f40uk5NPaEO6BdKqkT+UCx4eBtOkoVoOdXIxdgZRITwHkYhO2LxQApfpOV vbHclklZRBgKqTsQeoUSAyceBPBxwJcU/b+EEXEh9Gl57wdE1E9LO1uKRLDxrGYg tR2q/JtZZuaG1JgqScWO/baEphla8j3zlDpzEeH/OoKY9zgW+iBYPOCwjF+drz/U sg102NLgzc6oh1peKk7TeKv9A/G0CuJMl4NmPerc2pUDEa6hZaQJFPsoBHmlqwsg H3wUM1cBJOplZAGFY0yL0PTFQ/0TV3TriQtFvvAF85/lYshpsEk=
=Fy85
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

HW42 <hw42@ipsumj.de> writes:

Russ Allbery:

This attack is not equivalent to compromise of the uploader's OpenPGP
key, which neither upload architecture defends against. Many Debian
uploaders build source packages on less-trusted systems where they also
build and test binary packages, and then sign the source package from a
more-trusted system or use a hardware key.

Is this really common practice that Debian uploaders sign (source)
packages they built on less-trusted systems?

I have no idea if it's common practice, which to me that implies a
majority or near majority. I have no data. I think "many" is correct
based on the number of Debian uploaders who have said in various
discussions over the years that they do this.

And, if yes: Why wouldn't they do the equivalent with the sources in git (work on the less trusted system, transfer commits (git push/pull) to
the system with signing access and sign there, without review)?

They will, I assume. But tag2upload requires that the malicious code that could be added during that process be pushed to Salsa or the signature
will not validate and tag2upload will fail. My contention is that this
makes detection of the attack easier.

--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Russ Allbery <rra@debian.org> writes:

HW42 <hw42@ipsumj.de> writes:

And, if yes: Why wouldn't they do the equivalent with the sources in
git (work on the less trusted system, transfer commits (git push/pull)
to the system with signing access and sign there, without review)?

They will, I assume. But tag2upload requires that the malicious code
that could be added during that process be pushed to Salsa or the
signature will not validate and tag2upload will fail. My contention is
that this makes detection of the attack easier.

Oh, I'm sorry, now I see your point. You're saying that my contention
that tag2upload is better at prevention of attacks via the developer's
local system is not correct because the same attack can be done during Git signing.

I think I didn't get this quite right in that part of the review, but my
point is that having to push commits to the Git repository makes detection
more likely *before* the upload, and detection before the upload is
prevention. I'm not sure how to get at this point without using a lot
more words, though, so maybe I should just relax the claim to say that I
think tag2upload is better at detection and tracing, which is obviously correct. But let me write this up anyway, since I'm thinking about.

It is true, in theory, that if a system is compromised, every command run
on that system is suspect, and attackers can hide any activity and modify
any tool output that they choose. In general, because I'm a hardening
person primarily, I try to analyze systems assuming that case. However,
*in practice*, most attackers, even sophisticated attackers, are nowhere
near that careful. Most attackers get caught because they do something
that makes noise. Staying entirely silent is quite challenging.

What that means in practice is that the more things a human looks at that
the attacker has to hide, the more likely it is that someone will detect
the attack, possibly early enough to prevent it. Pushing a Git branch
produces different output than pushing a Git tag: the uploader might
notice. Pushing a tag that isn't on a commit in the direct history of the working branch means the tag doesn't appear in git log: the uploader might notice. If the addition isn't done carefully, the uploader may see the attacker's commit in git log or on Salsa and notice.

Building a source package is a lot more opaque and gives the attacker a
lot more room to hide. Adding malicious code to tar to inject something
into source packages is a lot quieter: the contents of the tar file are no longer what the uploader expected, but there's no reason for the uploader
to look at the tar file after the build process is done, or use much care
in eyeballing it, since 99.9% of the time it will look exactly like what
they expect. Back in the day when we were all using the 1.0 source
format, I often did read at least a filterdiff version of the Debian diff;
now, with 3.0 (quilt) packages, I never bother. And if the malicious code
is hidden in debian/rules in the source package during the build process,
I don't think I'd ever notice unless Lintian did.

I think (my guess at) your critique is correct: I shouldn't say prevention
here because there isn't real protection against a compromised uploader
system in either approach. It's specifically a detection and tracing advantage. But I do think having to push the attack into the Git
repository raises the chances that the detection might be early enough
that someone can throw up a quick alarm and get the source package rolled
back before there's serious damage done. Stuff hiding in the build source package is a lot sneakier.

--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

HW42 writes ("Re: Security review of tag2upload"):

Is this really common practice that Debian uploaders sign (source)
packages they built on less-trusted systems?

Like Russ, I don't really know how common this is.

I do know this: people asked for, and I implemented, a special mode
with dgit, called `dgit rpush`, which allows you to have your signing
key on a different system to the one with the source code, source
package clean target, dpkg-source, etc. That enables that same
workflow. (I did this because I want people to be able to use dgit
without changing other aspects of their workflows.)

But it might be worth stepping back a bit and seeing *why* people
might want to work this way.

The reason is probably that the source package build is complicated
and inconvenient. You'll need the source package tarballs, which may
be large. In pre-dgit workflows you typically have to run the package
clean target, so you must have its build dependencies installed, so
perhaps it's a chroot. If you're doing a with-binaries upload, it can
be convenient to do all the building parts on the same machine. I
wouldn't be surprised if some people use porterboxes for this. It's
even possible that some maintainers' main systems aren't Debian
derivatives, so they don't conveniently have Debian source package
tools.

Some of these use cases could be replaced with tag2upload, especially
if we can widen acceptance of source-only uploads. So perhaps we
could hope that fewer maintainers will want to work this way.

(Perhaps, even if the problem was the source code size. git is much
better at cacheing and incremental updates, than source packages -
updating a git tree, and pushing a tag, are a lot cheaper than
transferring new tarballs in both directions.)

Ian.

--
Ian Jackson <ijackson@chiark.greenend.org.uk> These opinions are my own.

Pronouns: they/he. If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------4UI4D0peIKUHm0jusSCedbKx
Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64

T24gMjUuMDYuMjQgMDA6NTIsIEhXNDIgd3JvdGU6DQo+IElzIHRoaXMgcmVhbGx5IGNvbW1v biBwcmFjdGljZSB0aGF0IERlYmlhbiB1cGxvYWRlcnMgc2lnbiAoc291cmNlKQ0KPiBwYWNr YWdlcyB0aGV5IGJ1aWx0IG9uIGxlc3MtdHJ1c3RlZCBzeXN0ZW1zPw0KDQpEYXRhIHBvaW50 OiBteSBtYWluIGJ1aWxkIHN5c3RlbSBpcyBhIDI0LWNvcmUgVk0gaW4gb3VyIGRhdGEgY2Vu dGVyLg0KDQpJJ20gbm90IGdvaW5nIHRvIHVwbG9hZCBteSBwcml2YXRlIGtleSB0byBhIGRh dGEgY2VudGVyIOKApiBhbmQgcGVvcGxlIA0Kd2hvc2Uga2V5IGlzIG9uIHNvbWUgaGFyZHdh cmUgY2FyZCBvciBzdGljayBjYW4ndCB1cGxvYWQgdGhlaXIga2V5IGF0IGFsbC4NCg0KVGh1 cyBpdCdzIG5vdCBuZWNlc3NhcmlseSBhIHByb2JsZW0gb2YgImxlc3MgdHJ1c3QiLCAiSSdt IG5vdCBwaHlzaWNhbGx5IA0Kd2hlcmUgdGhlIGhhcmR3YXJlIGlzIiAoYW5kIGRvbid0IGFs bG93IHR1bm5lbGVkIGFjY2VzcyB0byBteSBoYXJkd2FyZSANCmtleSB0byBhbnl3aGVyZSwg YXMgYSBtYXR0ZXIgb2Ygc2VjdXJpdHkgcHJpbmNpcGxlKSBpcyBzdWZmaWNpZW50Lg0KDQot LSANCi0tIHJlZ2FyZHMNCi0tIA0KLS0gTWF0dGhpYXMgVXJsaWNocw0KDQo=

--------------4UI4D0peIKUHm0jusSCedbKx--

-----BEGIN PGP SIGNATURE-----

wsF5BAABCAAjFiEEr9eXgvO67AILKKGfcs+OXiW0wpMFAmZ6kaMFAwAAAAAACgkQcs+OXiW0wpNy 7w/+MRduruL1QSRw+nueIZ4EbWnZyzTKlu/pX2yGh8Spb876NOT0Bk1d5csbhdtZokt2Avt6cYTD Cu+Oh12qlrBryWTzfZHjutEqUa6F0j52DvnC1fgHLSPRBubuopslnGJknGEbuq66MP60l3SzPZBZ r9MTeWiS1LpJx+OQCihirm1WfEiq3y2prEuIqHLV9jAzxiuUw3aWq37SFjfjjtj5w0PfoSAkLiFe /Wl+Ik7DQxQH9IV0z1xkjFsYB76jcU8qx93cN8EQOpvnM7/OE0KrLH1tj6B9DBTMTsLx4FdsZsst 8XoK67RiAgAl/B2QXKbJCZRRAqIbRAJMz8vBJS21ozTBzo6O43nFMqEm5r6l0RVUmobpk8yJE35M lumxTYFqn/o5BeT+sle/UnQ4AOd9umWJyMlqdyJMy+vsFRWzmNDCwBx1EPcj/TgG0IJp22foPdy7 SrLjkwuqjdvnAvaT+4yv9z3s59vaIqz/RGTcHoz8lD3cAhCAVnPtU5bgzblMPTx4qOakWQmGx3ok FWqPrVfIx31y6Kaejh/+gWlDVqIc2Uyobo+WhCn0tspGyM6JFI8JjbvLWfnczs8FREeaIiJ2jCvG nJaINDACxcdY9MXydULqgi94vKfH4WqO6AywyihuSiMqsNa7Pfq4o0XN2gHehy//i0fr1Itcvg1r wOo=
=Xoo9
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Building a source package is a lot more opaque and gives the attacker a
lot more room to hide. Adding malicious code to tar to inject something
into source packages is a lot quieter

How many packages have a pubkey for the orig file?

Perhaps we should encourage upstreams to sign more?

I guess that means giving up pypi as a place to download from, since they have removed support for signatures.

But for example kde tarballs are all signed.

--
Salvo Tomaselli

"Io non mi sento obbligato a credere che lo stesso Dio che ci ha dotato di senso, ragione ed intelletto intendesse che noi ne facessimo a meno."
-- Galileo Galilei

https://ltworf.codeberg.page/

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Quoting Salvo Tomaselli (2024-06-26 09:25:37)

Building a source package is a lot more opaque and gives the attacker a
lot more room to hide. Adding malicious code to tar to inject something into source packages is a lot quieter

How many packages have a pubkey for the orig file?

Perhaps we should encourage upstreams to sign more?

What I have learned from all this, is that we should not encourage to
sign more, but encourage to cautiously sign more.

Both ourselves and our upstreams.

My point being that signatures have little value if automated or done
manually without related examination.

I am sure that's also what you meant, Salvo, I just find it quite
relevant to be explicit that it is the care that need a boost, not the
amount of signatures.

- Jonas

--
* Jonas Smedegaard - idealist & Internet-arkitekt
* Tlf.: +45 40843136 Website: http://dr.jones.dk/
* Sponsorship: https://ko-fi.com/drjones

[x] quote me freely [ ] ask before reusing [ ] keep private --==============u49898616573240206=MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Description: signature
Content-Type: application/pgp-signature; name="signature.asc"; charset="us-ascii"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEn+Ppw2aRpp/1PMaELHwxRsGgASEFAmZ7xeYACgkQLHwxRsGg ASFZuA//RQUsztJb6Alck36KCVlhqPP4D5WVJ5PfB10LvBF4queKJXcUIrDbXNux rCHGGWqNFV2ETDmazUr3ZtVsxixaCpU+6nOFUSgYMHopAxQ1XbyPuQzXaFZcjARJ attLS+JcX/nx8CeoJ2xDYv0EpJpJwPc/mQ6Fw6bc6/zn+6mruouWgo8CF3vFqqTc v7muyMmb2G5Hu8e0BsmQOdQoXF2PnrzJWOebWIDLK5xQOn9SxwZRM5cHhN5DGScW P04JX9KBx05CNYEoFcVNMUfyFvgrqH2gtgGxTVpmNNoDXtY1GLu22n3RIBIIkioT Dv9s6UVTh8sXpwHKhgMCihXyFRnkrk2MtCoH80Oeh130AEBdvQKTJiUOIPfJxYtr dSRHwzje0zhwp2kZUTvd1l3EVItt97wiHyENBQaGOIgmPY4LhDuPx3p9SXyEFjqN 4ATz+NswdCrzzIGIu8/Oa5F2pXs6wohhPthW7vSV

I am sure that's also what you meant, Salvo, I just find it quite
relevant to be explicit that it is the care that need a boost, not the
amount of signatures.

Well if you manually check very carefully every line and then don't sign… it's
harder to discover it got modified.

--
Salvo Tomaselli

"Io non mi sento obbligato a credere che lo stesso Dio che ci ha dotato di senso, ragione ed intelletto intendesse che noi ne facessimo a meno."
-- Galileo Galilei

https://ltworf.codeberg.page/

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Quoting Salvo Tomaselli (2024-06-26 10:29:53)

I am sure that's also what you meant, Salvo, I just find it quite
relevant to be explicit that it is the care that need a boost, not the amount of signatures.

Well if you manually check very carefully every line and then don't sign… it's
harder to discover it got modified.

Yes, I agree.

We can encourage upstream to...

a) sign
b) sign what is carefully checked
c) carefully check (and not sign)

Assuming good faith, I believe that in your earlier email you meant b)
although you only wrote a). Now in your followup email you say that c)
is not as good.

I agree with your earlier, explicit point that a) is good.

I agree with your earlier (assumed) implied point, that b) is good.

I agree with you new point, that c) is not as good.

My point is that *explicitly* encouraging b) is better than *implicitly* encouraging it by explicitly saying only a).


- Jonas

--
* Jonas Smedegaard - idealist & Internet-arkitekt
* Tlf.: +45 40843136 Website: http://dr.jones.dk/
* Sponsorship: https://ko-fi.com/drjones

[x] quote me freely [ ] ask before reusing [ ] keep private --==============D76495285522418290=MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Description: signature
Content-Type: application/pgp-signature; name="signature.asc"; charset="us-ascii"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEn+Ppw2aRpp/1PMaELHwxRsGgASEFAmZ77msACgkQLHwxRsGg ASF+ghAAqn6MkPX+Bnx9pOoNIo+NAfFdfFHZEPKVpuuZ3weC5b8r6cGtuKX5tmrf UFKgBj3CXWqui82bBkUEkJwY6vwx5pmFMa5mDqOb1MltvD1qC2vGhiXrU1z609hL 0v6YRZb59Hs7lkWt8IqmD4EqyYWxqlDnZ1ZtVGJwIQ6KnOte5Se8Z9f610FA1wdQ co7qsxoni8AHs7xG/ODyMklXU4xqJafvi7p864FRrz0L15JUOB4GFKMdbOTbApN8 MleHQJR8Y1vdJ2ftLOnqScmoac0KyVdndkFDCmYFNe2OWh5Kw6kcrvg5D4Zb7ObZ 2BpE3Ic2Ml+8Zd94FbltfOHKb9zFlJK6q7y80rxmWln042Ym2qGmxBrP9TC93eiq bcsi+sOB9n2Cl6ZjsQgdCDJkIscYHexbXcxoupV9jgICZ5uDoxO5RV66KiBwUN7T lXO74U1IuVykiAZBt6s4Ro5qfAjYAB5+wq0dIy6z