Copy: debian-qa@lists.debian.org

On Friday, 27 January 2023 20:14:43 CET Jelmer Vernooij wrote:

Hi Diederik!

Hi Jelmer!

I agree, I think anything that reduces the complexity of the ecosystem
is great and the lowers the barrier for entry. As pretty anything uses
Git now, I think migrating more things from SVN to Git would be great.

Great and I agree :-)

On Fri, Jan 27, 2023 at 07:56:37PM +0100, Diederik de Haas wrote:

I wouldn't be surprised if most people would bail out at the first 404 message.

There must be a better way?
- I don't know if someone has already done this migration and could share their experience/tools/etc?
- If it needs to be done, isn't it way better to do a mass-migration of
all the repos which haven't been converted yet? There may be a high similarity between the various SVN repos, but all the projects within one SVN repo likely share many things? Like svn-user to git-user mapping?
- And then update d/control so that the PTS page links directly to the salsa repo?
(- I don't know if snapshot.d.o could/should play a role in this)

Would love to hear some thoughts on this.

I've been looking at how to do a mass conversion. There's about 375 packages still listed as being on alioth (~100 in SVN, ~267 in Git, the rest in something else). https://janitor.debian.net/cupboard/result-codes/hosted-on-alioth?campaign=u nchanged&include_transient=off&include_historical=off

I'm so happy with these things and the janitor work in general :-)
I have a lot to learn and most of what I do is manual labor. Doable when the volume is low, but for mass-* that would get problematic.

Here's what I have so far:

* A scheduler script that determines what things still claim to
be hosted on alioth and finds the matching archive

https://salsa.debian.org/janitor-team/janitor.debian.net/-/blob/master/sche dule/vcswatch-candidates.py (based on vcswatch data in UDD)

* A "import-from-alioth.py" script
(https://salsa.debian.org/janitor-team/janitor.debian.net/-/blob/master/imp ort-alioth-archive.py) that takes an alioth archive and pulls it into the local repository.

* lintian-brush has a mapping of team names to locations in salsa,
here: https://salsa.debian.org/jelmer/lintian-brush/-/blob/master/lintian_brush/s alsa.py

I have recently (re-)started my attempts to learn Python, which is something I actually want to learn ;-)
I'll look at/study them, but I probably won't be able to help much (soon).

But I won't mind helping out in other ways with f.e. manual labor.

What still needs to happen is:

* The mapping still needs to be tied together with the import script, to
generate correct URLs to push to and set the Vcs-* headers
appropriately

I'm not sure what to do with packages whether the owning user or team
is not on salsa. Add them to the "debian" group?

I think it would be easier (and possibly better) to create a separate subgroup under the debian group. Moving the converted repo is probably sth that should be done manually. F.e. the id3lib repo does exist on salsa, but is useless. Detecting the exact state of a repo (directly) under 'debian' may not be possible and/or be WAY too involved. And you probably also don't want to run the risk of accidentally overwriting an existing repo.
As I *think* the majority of the work is actually converting repos and getting them in salsa at all, doing that in a new 'namespace' seems easiest. And safest. And then you can still automatically set the Vcs-* headers.

If at a later point the choice is made to move a repo to a better place, then updating the Vcs-* headers should be rather trivial.

* The import script supports just git right now, not svn. There's ~8
repositories in a VCS other than SVN or Git, which we could just
migrate manually.

Agreed.

* Verification that this is all working well

* and then perhaps gradually rolling it out, starting with QA packages.
then perhaps getting buyin from the maintainers of the packages
involved and doing the rest.

I do have the 'collab-maint' archive locally, both in .xz format as extracted, so I could run test conversions 'endlessly'.
I believe that's the biggest one, but I could also retrieve a smaller one for initial testing and when that works properly, do it on 'the big one'.

Cheers,
Diederik
-----BEGIN PGP SIGNATURE-----

iHUEABYIAB0WIQT1sUPBYsyGmi4usy/XblvOeH7bbgUCY9RNqgAKCRDXblvOeH7b bk1oAP9JaTZME6mTrvVVgd0wuOz+Qt4gvFuqxd6qEpJgVvZlTgD/cbAF8ksBoH49 nvR2DB7cSVjjmyI7eqw9K6stMPvJkgE=
=pZu6
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Diederik!

I agree, I think anything that reduces the complexity of the ecosystem
is great and the lowers the barrier for entry. As pretty anything uses
Git now, I think migrating more things from SVN to Git would be great.

On Fri, Jan 27, 2023 at 07:56:37PM +0100, Diederik de Haas wrote:

I wouldn't be surprised if most people would bail out at the first 404 message.

There must be a better way?
- I don't know if someone has already done this migration and could share their experience/tools/etc?
- If it needs to be done, isn't it way better to do a mass-migration of all the repos which haven't been converted yet? There may be a high similarity between the various SVN repos, but all the projects within one SVN repo likely
share many things? Like svn-user to git-user mapping?
- And then update d/control so that the PTS page links directly to the salsa repo?
(- I don't know if snapshot.d.o could/should play a role in this)

Would love to hear some thoughts on this.

I've been looking at how to do a mass conversion. There's about 375 packages still
listed as being on alioth (~100 in SVN, ~267 in Git, the rest in
something else). https://janitor.debian.net/cupboard/result-codes/hosted-on-alioth?campaign=unchanged&include_transient=off&include_historical=off

Here's what I have so far:

* A scheduler script that determines what things still claim to
be hosted on alioth and finds the matching archive
https://salsa.debian.org/janitor-team/janitor.debian.net/-/blob/master/schedule/vcswatch-candidates.py
(based on vcswatch data in UDD)

* A "import-from-alioth.py" script
(https://salsa.debian.org/janitor-team/janitor.debian.net/-/blob/master/import-alioth-archive.py)
that takes an alioth archive and pulls it into the local repository.

* lintian-brush has a mapping of team names to locations in salsa,
here: https://salsa.debian.org/jelmer/lintian-brush/-/blob/master/lintian_brush/salsa.py

What still needs to happen is:

* The mapping still needs to be tied together with the import script, to
generate correct URLs to push to and set the Vcs-* headers
appropriately

I'm not sure what to do with packages whether the owning user or team
is not on salsa. Add them to the "debian" group?

* The import script supports just git right now, not svn. There's ~8
repositories in a VCS other than SVN or Git, which we could just
migrate manually.

* Verification that this is all working well

* and then perhaps gradually rolling it out, starting with QA packages.
then perhaps getting buyin from the maintainers of the packages
involved and doing the rest.

Cheers,

Jelmer

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hello,

On Fri, 27 Jan 2023, Jelmer Vernooij wrote:

I agree, I think anything that reduces the complexity of the ecosystem
is great and the lowers the barrier for entry. As pretty anything uses
Git now, I think migrating more things from SVN to Git would be great.

I will not forbid anyone to work on this but IMO the time to handle mass migration is long past. Anything that still has subversion listed
in its VCS is effectively unmaintained.

If I were to take over one of those packages I would just use "gbp
import-dscs" to recreate some partial history based on snapshot.debian.org
and I would not put any more effort than this.

Even if I wanted to take care of all those packages as QA work to
increase their chance of someone taking over, I would do the same
and move the new repositories in the debian namespace.

Cheers,
--
⢀⣴⠾⠻⢶⣦⠀ Raphaël Hertzog <hertzog@debian.org>
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋ The Debian Handbook: https://debian-handbook.info/get/
⠈⠳⣄⠀⠀⠀⠀ Debian Long Term Support: https://deb.li/LTS

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sat, Jan 28, 2023 at 04:30:39PM +0100, Raphael Hertzog wrote:

Hello,

On Fri, 27 Jan 2023, Jelmer Vernooij wrote:

I agree, I think anything that reduces the complexity of the ecosystem
is great and the lowers the barrier for entry. As pretty anything uses
Git now, I think migrating more things from SVN to Git would be great.

I will not forbid anyone to work on this but IMO the time to handle mass migration is long past. Anything that still has subversion listed
in its VCS is effectively unmaintained.

If I were to take over one of those packages I would just use "gbp import-dscs" to recreate some partial history based on snapshot.debian.org and I would not put any more effort than this.

Even if I wanted to take care of all those packages as QA work to
increase their chance of someone taking over, I would do the same
and move the new repositories in the debian namespace.

FWIW I have mostly been looking at the Git repositories (rather than the
SVN ones), where I think it would actually be less effort to just import
the alioth archives than to import from snapshot.debian.org.

I haven't checked how complicated importing the SVN repositories is.
It'd be nice to keep the history, but if it's too much effort I agree
just importing the uploaded versions might be good enough.

Cheers,

Jelmer

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)