I endorse basically all of this. (I had really hoped to be able to put significant effort into modernizing debbugs over the past year or so,
modernized in a satisfying way while retaining most if not all of its
current interfaces. This would minimize breakage and inconvenience for >developers that are mostly fine with the way it currently works.
I'm considering getting my hands into that thing later this year, so
let me try to summarize the relevant parts of the previous threads
(with the intent of documenting this in a wiki page).
We would like debbugs to:
0. keep all the e-mail features it currently offers
1. process new requests and give feedback instantly
2. hide e-mail addresses from public (unauthenticated) web browsing
3. have a web UI that makes it possible to submit bugs, reply to bugs, >manipulate bugs
4. have some GitLab (Salsa) integration
5. have better, restructured, simplified documentation with full
examples
6. track merge requests.
Implementing 2. is likely to break habits and maybe some tools, as
currently there is no authentication at all on debbugs web UI and you
can use it to view full headers, download mboxes and generate a
working reply mailto: link. It also won't completely solve the privacy
issue, as e-mail addresses can also be found in git repositories and
mailing list archives. IMO it would be better to recommend using a
dedicated e-mail address.
I'm considering getting my hands into that thing later this year, so let
me try to summarize the relevant parts of the previous threads (with the intent of documenting this in a wiki page).
We would like debbugs to:
0. keep all the e-mail features it currently offers
1. process new requests and give feedback instantly
2. hide e-mail addresses from public (unauthenticated) web browsing
3. have a web UI that makes it possible to submit bugs, reply to bugs, manipulate bugs
4. have some GitLab (Salsa) integration
5. have better, restructured, simplified documentation with full examples
6. track merge requests.
BTW I don't think that Bugzilla, GitLab issues or even JIRA would be
suitable replacements for debbugs. FWIW some Apache projects including
the whole Maven project are now migrating away from JIRA to GitHub
issues, but they also have a fairly different usage pattern than Debian.
For Debian maybe something like Redmine could work (I'm not suggesting a change, but looking at alternatives may give some ideas).
6. track merge requests.
I would assume Debbugs might evolve without you having to personally
do all the improvements, if you allow improvements done by others flow
in.
As an example, I have had >https://salsa.debian.org/debbugs-team/debbugs/-/merge_requests/6 open
for 4 years now.
0. keep all the e-mail features it currently offers
IMHO, this is a security flaw, not a feature. I hear that everyone loves
it, so at least the emails should be authenticated somehow (@debian only maybe? one-time register-confirm emails?).
- add a new field that allows pointing to a merge request (similar to
the 'forwarded' field)
- have a bot that processes bugs with merge requests and adjust BTS
tags
(add 'patch' if the bug has a MR; add 'pending' if the MR has been
merged)
On 27/05/2025 17:46, Julien Plissonneau Duquène wrote:
0. keep all the e-mail features it currently offers
IMHO, this is a security flaw, not a feature. I hear that everyone
loves it, so at least the emails should be authenticated somehow
(@debian only maybe? one-time register-confirm emails?).
1. process new requests and give feedback instantly
My opinion now is that debbugs runs pretty frequently, but emails get rejected randomly, leading to a 30min wait for mail queue reruns. This
is probably a mail server config issue.
Current documentation is good actually, but you have to find the right
page.
May I also add:
- Proper email handling
they all have a Database data model (faster search/query)
some API endpoint (REST/SOAP/...)
My opinion now is that debbugs runs pretty frequently, but emails
get rejected randomly, leading to a 30min wait for mail queue
reruns. This is probably a mail server config issue.
That should be investigated, but it could also be caused by the remote
mail server.
they all have a Database data model (faster search/query)
Having some experience with databases, I'm not convinced that a RDBMS (SQL) is a necessity here. Better indexing and caching would definitely help though.
We would like debbugs to:
0. keep all the e-mail features it currently offers
1. process new requests and give feedback instantly
2. hide e-mail addresses from public (unauthenticated) web browsing
3. have a web UI that makes it possible to submit bugs, reply to bugs, manipulate bugs
4. have some GitLab (Salsa) integration
5. have better, restructured, simplified documentation with full examples
6. track merge requests.
On Tue May 27, 2025 at 7:30 PM BST, Ahmad Khalifa wrote:
0. keep all the e-mail features it currently offers
IMHO, this is a security flaw, not a feature. I hear that everyone
loves it, so at least the emails should be authenticated somehow
(@debian only maybe? one-time register-confirm emails?).
Can you expand on why you think manipulating bugs by mail is a security
flaw?
Le 2025-05-27 20:30, Ahmad Khalifa a écrit :
On 27/05/2025 17:46, Julien Plissonneau Duquène wrote:
0. keep all the e-mail features it currently offers
IMHO, this is a security flaw, not a feature. I hear that everyone
loves it, so at least the emails should be authenticated somehow
(@debian only maybe? one-time register-confirm emails?).
We have to be careful with changes here, as they could easily become
major annoyances for casual and even regular contributors.
Current documentation is good actually, but you have to find the right
page.
This "you have to find (or remember) the right page" part is one of the
areas that could greatly be improved ;)
they all have a Database data model (faster search/query)
Having some experience with databases, I'm not convinced that a RDBMS
(SQL) is a necessity here. Better indexing and caching would definitely
help though.
Anyone can manipulate any bug without restriction and in bulk.Security measures should be proportional to the specific threat, and
Untag it as RC, email 0-99999 with the -done suffix, spam it with
links, target packages that echo to a mailing list.
I'm considering getting my hands into that thing later this year, so
let me try to summarize the relevant parts of the previous threads
(with the intent of documenting this in a wiki page).
We would like debbugs to:
0. keep all the e-mail features it currently offers
1. process new requests and give feedback instantly
2. hide e-mail addresses from public (unauthenticated) web browsing
3. have a web UI that makes it possible to submit bugs, reply to bugs, manipulate bugs
4. have some GitLab (Salsa) integration
5. have better, restructured, simplified documentation with full
examples
6. track merge requests.
6. is not clear enough. What would we like debbugs to achieve, more precisely, here?
Can you expand on why you think manipulating bugs by mail is aAnyone can manipulate any bug without restriction and in bulk.
security flaw?
Maybe I'm exaggerating a bit,
they all have a Database data model (faster search/query)
Having some experience with databases, I'm not convinced that a RDBMS
(SQL) is a necessity here. Better indexing and caching would
definitely help though.
Most of these I also very much agree with, however I doubt that *we* want
to hide e-mail addresses from public (unauthenticated) web browsing. In my book
the open development model of Debian is tied to the fact that we the developers
are responsible *and* reachable. Our users can see who made the distribution >they are using. That is a feature.
I've also seen very very few complaints about the fact that the BTS shows >email addresses if submitters and contributors. And I'm definitly not aware >that we identified this as a problem!
Also, I don't really see how to keep all the e-mail features it currently offers,
while hiding email addresses. I quite often look up email addresses in bugs >and contact people directly, definitly more than once per months.
I've also seen very very few complaints about the fact that the BTS shows email addresses if submitters and contributors. And I'm definitly not aware that we identified this as a problem!owner@bugs gets complaints about this fairly often.
I've also seen very very few complaints about the fact that the BTS shows email addresses if submitters and contributors. And I'm definitly not aware that we identified this as a problem!It definitely attracts huge amounts of spam.
While it might be possible to carry on doing without, the data
fundamentally has many relational properties and an RDBMS would make
life a lot easier. I wish I'd known what I know now about PostgreSQL
when I was in my period of working on debbugs very actively; I would
100% have switched to it.
There is always a trade-off between usability and security.Security measures should be proportional to the specific threat, andWe know that they didn't happen so far. I would not be so sure about
we actually know that targeted malicious attacks to the BTS are not >>happening.
the future.
Anyway there are backups, and incoming legitimate messages could beRight.
replayed should such abuse occur, right?
(still, I think the answer to that should not be to hide email
addresses but something else, eg maybe asking new bts users if they
are aware that there email address will become public and block their submissions until they agree...)
I think that there is no need to rush a move. Working on it for a
while and experimenting with the real data in there will certainly
help with figuring out what could be done about the storage.
That data is somewhat transactional, moderately relational, also with >relations like bugs that are merged with bugs, block bugs, affect
packages which are dependencies of packages, are found in versions
which have versions as successors ... and SQL is not that great at >recursivity or working with graphs. Or handling large binary objects.
Or doing finely tunable full-text indexing and search.
On Thu May 29, 2025 at 9:58 AM BST, Holger Levsen wrote:
But their contribution is the thing that's valuable to us, more so than
their email address.
Unlike some people I believe that debbugs can be improved and modernized in
a satisfying way while retaining most if not all of its current interfaces. This would minimize breakage and inconvenience for developers that are mostly fine with the way it currently works.
I'm considering getting my hands into that thing later this year, so let me try to summarize the relevant parts of the previous threads (with the intent of documenting this in a wiki page).
We would like debbugs to:
0. keep all the e-mail features it currently offers
1. process new requests and give feedback instantly
2. hide e-mail addresses from public (unauthenticated) web browsing
3. have a web UI that makes it possible to submit bugs, reply to bugs, manipulate bugs
4. have some GitLab (Salsa) integration
5. have better, restructured, simplified documentation with full examples
6. track merge requests.
Implementing 2. is likely to break habits and maybe some tools, as
currently there is no authentication at all on debbugs web UI and you can use it to view full headers, download mboxes and generate a working reply mailto: link. It also won't completely solve the privacy issue, as e-mail addresses can also be found in git repositories and mailing list archives. IMO it would be better to recommend using a dedicated e-mail address.
6. is not clear enough. What would we like debbugs to achieve, more precisely, here?
BTW I don't think that Bugzilla, GitLab issues or even JIRA would be
suitable replacements for debbugs.
I would assume Debbugs might evolve without you having to personally
do all the improvements, if you allow improvements done by others
flow in. As an example, I have had https://salsa.debian.org/debbugs-team/debbugs/-/merge_requests/6
open for 4 years now.
On Wed, May 28, 2025 at 02:22:00PM +0000, Holger Levsen wrote:
Also, I don't really see how to keep all the e-mail features it currently offers,
while hiding email addresses. I quite often look up email addresses in bugs and contact people directly, definitly more than once per months.
One possibility would be for the BTS to offer a way to follow up privately, similar to the NNNNN-submitter@ addresses. (This idea would obviously need refinement.)
On Tue, 27 May 2025, Colin Watson wrote:n
In my view, an absolute prerequisite for all of this is moving from the current flat-file structure to a proper database; the current data structures just don't perform well enough to be usable as the foundation for
a modern web UI. Don has some branches on https://salsa.debian.org/debbugs-team/debbugs that make a start at switching
to PostgreSQL, and he can probably fill you in on more details.
Absolutely.
My personal goal is to replace the entire codebase with python+sqlalchemy+postgresql+fastapi, while keeping the flat file
system only for archiving the bug log (the .log files) for at least a significant period of time, but that bug log would be write only.
The code for this is about 25% there, but I've been working on it for
years now in my very limited Debian development time, so I don't have
a realistic timeline for completion.
On Sun, Jun 01, 2025 at 09:54:09AM -0700, Don Armstrong wrote:
On Tue, 27 May 2025, Otto Kekäläinen wrote:n
I would assume Debbugs might evolve without you having to personally
do all the improvements, if you allow improvements done by others
flow in. As an example, I have had https://salsa.debian.org/debbugs-team/debbugs/-/merge_requests/6
open for 4 years now.
That's not an MR that I'll apply, because the actual version that
debbugs runs is at https://bugs.debian.org/debbugs-source/debbugs/
FWIW the actual/working repository URL is https://bugs.debian.org/debbugs-source/debbugs.git
On Sun, 01 Jun 2025, Antonio Terceiro wrote:n
On Sun, Jun 01, 2025 at 10:04:53AM -0700, Don Armstrong wrote:
My personal goal is to replace the entire codebase with python+sqlalchemy+postgresql+fastapi, while keeping the flat file
system only for archiving the bug log (the .log files) for at
least a significant period of time, but that bug log would be
write only.
The code for this is about 25% there, but I've been working on it
for years now in my very limited Debian development time, so I
don't have a realistic timeline for completion.
Do you have some design documentation and/or a TODO list that others
could use to be able to contribute to this effort?
Not currently in a state that it would be useful for others to
contribute, but I'd be willing to get in front of that if there were
others who would be willing to commit to some amount of time (while
dealing with my limited availability.)
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 546 |
| Nodes: | 16 (0 / 16) |
| Uptime: | 161:57:19 |
| Calls: | 10,385 |
| Calls today: | 2 |
| Files: | 14,057 |
| Messages: | 6,416,500 |