• Building multiple flavors of a package with a HUGE upstream

    From Marc Haber@21:1/5 to All on Fri Jul 18 22:20:01 2025
    Hi,

    I would like to re-work my linux kernel building process and would like
    to do it right this time: Build a clean source package that dak and
    sbuild will accept. I do not plan to upload this to Debian, but I think
    that we all can learn from some discussion.

    I would like to build differnt "flavors", lets call them linux-server, linux-vm, linux-desktop and linux-notebook. But I'd like to have them
    from - at least - the same .orig.tar.gz, just for the reason that this
    file is about 150 MB large.

    So my idea would be:

    linux-6.4.15.orig.tar.gz
    linux-6.4.15-server_6.4.15-1.debian.tar.gz
    linux-6.5.14-server_6.4.15-1.dsc
    linux-6.4.15-vm_6.4.15-1.debian.tar.gz
    linux-6.5.14-vm_6.4.15-1.dsc
    linux-6.4.15-desktop_6.4.15-1.debian.tar.gz
    linux-6.5.14-desktop_6.4.15-1.dsc
    linux-6.4.15-notebook_6.4.15-1.debian.tar.gz
    linux-6.5.14-notebook_6.4.15-1.dsc

    with all those .dscs refering to the same, identical
    linux-6.4.15.orig.tar.gz.

    The source packages would then build, linux-image-6.5.14-server_6.4.15-1_amd64.deb linux-image-6.5.14-vm_6.4.15-1_amd64.deb linux-image-6.5.14-desktop_6.4.15-1_amd64.deb linux-image-6.5.14-notebook_6.4.15-1_amd64.deb

    and the respective other packages like linux-headers and linux-libc-dev.

    Is our tooling able to do this? How would the control file have to look
    like?

    For the real work, I'd like to make use of the make srcdeb-pkg and make bindeb-pkg that come with the upstream sources. Those mechanisms were
    written bei Wichert 20 years ago but have been (carefully) updated so
    that they still work. I would probably have to throw away the source
    package that make srcdeb-pkg builds and patch the debian/ directory made
    in the process before building "my" source package, but that sounde easy enough.

    Would dpkg's "new" feature of having source packages with more than one orig.tar.gz work here?

    Any ideas?

    Greetings from (still) Brest
    Marc

    -- ----------------------------------------------------------------------------- Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany | lose things." Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ahmad Khalifa@21:1/5 to Marc Haber on Fri Jul 18 22:30:01 2025
    On 18/07/2025 21:16, Marc Haber wrote:
    Hi,

    I would like to re-work my linux kernel building process and would like
    to do it right this time: Build a clean source package that dak and
    sbuild will accept. I do not plan to upload this to Debian, but I think
    that we all can learn from some discussion.

    I would like to build differnt "flavors", lets call them linux-server, linux-vm, linux-desktop and linux-notebook. But I'd like to have them
    from - at least - the same .orig.tar.gz, just for the reason that this
    file is about 150 MB large.

    So my idea would be:

    linux-6.4.15.orig.tar.gz
    linux-6.4.15-server_6.4.15-1.debian.tar.gz
    linux-6.5.14-server_6.4.15-1.dsc
    linux-6.4.15-vm_6.4.15-1.debian.tar.gz
    linux-6.5.14-vm_6.4.15-1.dsc
    linux-6.4.15-desktop_6.4.15-1.debian.tar.gz
    linux-6.5.14-desktop_6.4.15-1.dsc linux-6.4.15-notebook_6.4.15-1.debian.tar.gz linux-6.5.14-notebook_6.4.15-1.dsc

    with all those .dscs refering to the same, identical linux-6.4.15.orig.tar.gz.

    The source packages would then build, linux-image-6.5.14-server_6.4.15-1_amd64.deb linux-image-6.5.14-vm_6.4.15-1_amd64.deb linux-image-6.5.14-desktop_6.4.15-1_amd64.deb linux-image-6.5.14-notebook_6.4.15-1_amd64.deb

    and the respective other packages like linux-headers and linux-libc-dev.

    Is our tooling able to do this? How would the control file have to look
    like?

    Imagemagick does a really good job of taking a single source and
    producing multiple binary flavours if you take a look at its d/rules
    iterating over {q16,q16hdri}.

    I believe d/control has to be a static file, but that's ok in your case
    as you need the kernel's .config files to match in the first place.

    For the kernel, you probably want to place the relevant .config file
    first and then use `make -O build-flavourX TARGET' to
    configure/build/install multiple binary dirs (e.g. build-server,
    build-vm, ...).

    There are other packages that do that too, but I'm not familiar with how
    e.g. exim, nginx, ...

    For the real work, I'd like to make use of the make srcdeb-pkg and make bindeb-pkg that come with the upstream sources. Those mechanisms were
    written bei Wichert 20 years ago but have been (carefully) updated so
    that they still work. I would probably have to throw away the source
    package that make srcdeb-pkg builds and patch the debian/ directory made
    in the process before building "my" source package, but that sounde easy enough.

    Would dpkg's "new" feature of having source packages with more than one orig.tar.gz work here?

    Any ideas?

    Greetings from (still) Brest
    Marc


    --
    Regards,
    Ahmad

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ahmad Khalifa@21:1/5 to Ahmad Khalifa on Fri Jul 18 22:50:01 2025
    On 18/07/2025 21:28, Ahmad Khalifa wrote:
    On 18/07/2025 21:16, Marc Haber wrote:
    Hi,

    I would like to re-work my linux kernel building process and would
    like to do it right this time: Build a clean source package that dak
    and sbuild will accept. I do not plan to upload this to Debian, but I
    think that we all can learn from some discussion.

    I would like to build differnt "flavors", lets call them linux-server,
    linux-vm, linux-desktop and linux-notebook. But I'd like to have them
    from - at least - the same .orig.tar.gz, just for the reason that this
    file is about 150 MB large.

    So my idea would be:

    linux-6.4.15.orig.tar.gz
    linux-6.4.15-server_6.4.15-1.debian.tar.gz
    linux-6.5.14-server_6.4.15-1.dsc
    linux-6.4.15-vm_6.4.15-1.debian.tar.gz
    linux-6.5.14-vm_6.4.15-1.dsc
    linux-6.4.15-desktop_6.4.15-1.debian.tar.gz
    linux-6.5.14-desktop_6.4.15-1.dsc
    linux-6.4.15-notebook_6.4.15-1.debian.tar.gz
    linux-6.5.14-notebook_6.4.15-1.dsc

    with all those .dscs refering to the same, identical
    linux-6.4.15.orig.tar.gz.

    The source packages would then build,
    linux-image-6.5.14-server_6.4.15-1_amd64.deb
    linux-image-6.5.14-vm_6.4.15-1_amd64.deb
    linux-image-6.5.14-desktop_6.4.15-1_amd64.deb
    linux-image-6.5.14-notebook_6.4.15-1_amd64.deb

    and the respective other packages like linux-headers and linux-libc-dev.

    Is our tooling able to do this? How would the control file have to
    look like?

    Imagemagick does a really good job of taking a single source and
    producing multiple binary flavours if you take a look at its d/rules iterating over {q16,q16hdri}.

    I believe d/control has to be a static file, but that's ok in your case
    as you need the kernel's .config files to match in the first place.

    For the kernel, you probably want to place the relevant .config file
    first and then use `make -O build-flavourX TARGET' to configure/build/ install multiple binary dirs (e.g. build-server, build-vm, ...).

    There are other packages that do that too, but I'm not familiar with how
    e.g. exim, nginx, ...

    For the real work, I'd like to make use of the make srcdeb-pkg and
    make bindeb-pkg that come with the upstream sources. Those mechanisms
    were written bei Wichert 20 years ago but have been (carefully)
    updated so that they still work. I would probably have to throw away
    the source package that make srcdeb-pkg builds and patch the debian/
    directory made in the process before building "my" source package, but
    that sounde easy enough.

    On second thought, if it's not a package, a shell script with a few
    lines could do this for you.

    Still relying on kbuild's -O argument, but basically something like:
    for b in build-server build-vm ...; do
    mkdir $b
    make -O $b mrproper
    cp config-for-server-that-I-prepared-already $b/.config
    make -O $b olddefconfig
    make -O $b vmlinux modules # or whatever you're building
    make -O $b deb-pkg # I think
    done

    Then copy your deb files out.

    Would dpkg's "new" feature of having source packages with more than
    one orig.tar.gz work here?

    Any ideas?

    Greetings from (still) Brest
    Marc


    --
    Regards,
    Ahmad

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Josh Triplett@21:1/5 to Marc Haber on Fri Jul 18 23:30:02 2025
    Marc Haber wrote:
    I would like to build differnt "flavors", lets call them linux-server, linux-vm, linux-desktop and linux-notebook. But I'd like to have them
    from - at least - the same .orig.tar.gz, just for the reason that this
    file is about 150 MB large.

    Is there a reason to have multiple source packages, rather than having
    one source package that builds multiple binaries?

    You could store all the config files in debian/, iterate over them, and
    build and install the files for the corresponding linux-image binary
    package.

    Also, *in theory* you shouldn't need a separate linux-headers and linux-libc-dev package for each of these, just one for all of them.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Marc Haber@21:1/5 to Josh Triplett on Fri Jul 18 23:50:01 2025
    On Fri, Jul 18, 2025 at 02:17:50PM -0700, Josh Triplett wrote:
    Marc Haber wrote:
    I would like to build differnt "flavors", lets call them linux-server,
    linux-vm, linux-desktop and linux-notebook. But I'd like to have them
    from - at least - the same .orig.tar.gz, just for the reason that this
    file is about 150 MB large.

    Is there a reason to have multiple source packages, rather than having
    one source package that builds multiple binaries?

    I'd like to be able to build just one kernel flavor at a time, and for
    just one architecture.

    You could store all the config files in debian/, iterate over them, and
    build and install the files for the corresponding linux-image binary
    package.

    That's a bit too far away from upstream's method of doing things. I have
    spent quite some time to understand what all those scripts are doing,
    and I'd rather not reinvent too much of them.

    Greetings
    Marc

    -- ----------------------------------------------------------------------------- Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany | lose things." Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Bastien Roucaries@21:1/5 to All on Fri Jul 18 23:40:39 2025
    Copy: ahmad@khalifa.ws (Ahmad Khalifa)

    Le vendredi 18 juillet 2025, 22:28:28 heure d’été d’Europe centrale Ahmad
    Khalifa a écrit :
    On 18/07/2025 21:16, Marc Haber wrote:
    Hi,

    I would like to re-work my linux kernel building process and would like
    to do it right this time: Build a clean source package that dak and
    sbuild will accept. I do not plan to upload this to Debian, but I think that we all can learn from some discussion.

    I would like to build differnt "flavors", lets call them linux-server, linux-vm, linux-desktop and linux-notebook. But I'd like to have them
    from - at least - the same .orig.tar.gz, just for the reason that this
    file is about 150 MB large.

    So my idea would be:

    linux-6.4.15.orig.tar.gz
    linux-6.4.15-server_6.4.15-1.debian.tar.gz
    linux-6.5.14-server_6.4.15-1.dsc
    linux-6.4.15-vm_6.4.15-1.debian.tar.gz
    linux-6.5.14-vm_6.4.15-1.dsc
    linux-6.4.15-desktop_6.4.15-1.debian.tar.gz linux-6.5.14-desktop_6.4.15-1.dsc linux-6.4.15-notebook_6.4.15-1.debian.tar.gz linux-6.5.14-notebook_6.4.15-1.dsc

    with all those .dscs refering to the same, identical linux-6.4.15.orig.tar.gz.

    The source packages would then build, linux-image-6.5.14-server_6.4.15-1_amd64.deb linux-image-6.5.14-vm_6.4.15-1_amd64.deb linux-image-6.5.14-desktop_6.4.15-1_amd64.deb linux-image-6.5.14-notebook_6.4.15-1_amd64.deb

    and the respective other packages like linux-headers and linux-libc-dev.

    Is our tooling able to do this? How would the control file have to look like?

    Imagemagick does a really good job of taking a single source and
    producing multiple binary flavours if you take a look at its d/rules iterating over {q16,q16hdri}.

    I believe d/control has to be a static file, but that's ok in your case
    as you need the kernel's .config files to match in the first place.
    For imagemagick control file is generated from template by ugly template using sed

    it could be reimplemented in perl or python but it work

    rouca

    For the kernel, you probably want to place the relevant .config file
    first and then use `make -O build-flavourX TARGET' to
    configure/build/install multiple binary dirs (e.g. build-server,
    build-vm, ...).

    There are other packages that do that too, but I'm not familiar with how
    e.g. exim, nginx, ...

    For the real work, I'd like to make use of the make srcdeb-pkg and make bindeb-pkg that come with the upstream sources. Those mechanisms were written bei Wichert 20 years ago but have been (carefully) updated so
    that they still work. I would probably have to throw away the source package that make srcdeb-pkg builds and patch the debian/ directory made
    in the process before building "my" source package, but that sounde easy enough.

    Would dpkg's "new" feature of having source packages with more than one orig.tar.gz work here?

    Any ideas?

    Greetings from (still) Brest
    Marc


    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEXQGHuUCiRbrXsPVqADoaLapBCF8FAmh6v1cACgkQADoaLapB CF9SXA//fRk2XZlKPM0/uQ/bKUPYkkwZ27x8Ob+2GtDOe08sOG/Q8M/BlV+65G8B jhL+8C0ktUIrjUlJhPDz3CUIK1FG80Iej2VgDZf4DsmYrqACtpymF5UmGFL4Jwf4 EiwRtpzwJ6BIzG5XKP0P3lM3dtsTcLZmXFcU9kSEp7S9cDhWG1f7Npp159ULIYFy Qo8Sz3b9kMYiL6PbocaabO5iJKTVtjIrpDVrf3LD/OnxhSMwtR0uVg1h7U87WmI4 lTGXIGWz9WZLfACeQjdws+bSnXzngQf9XnH//R4P0DKQAW/yOdXyWpaVRWDG8qcg dU0ZLJq9wf8oECSz75Kqg5dH27h4U2nGFwyFGOiKDEKNcR7J0uI4EUYUeXIPgHlo 07z/9hGZnB2G2aDxulDus/YngSa3ALut4ry10drBdCtdGFsfnL2MX1V1Vl6Dj9JV A+7xCD6b0QoNMfK+K4CjgbOuxARGR/d4IoYI0AlFMPPr8UFZLM9cOlRj7zDSrTOY NTQLcZSCDOAdRRzgcBzH+PyOuoUa2hmM84CCs/QVIkGCpZpwIi/tYWTZMdn4xAiB rOewopthmPJVssM5Z1b2Bs+BPaNaowfMabgWBBuH+vXiIKYliDoyptYGnYA9g4iN OuPLT/gMVzMpQar2ay3+o+ETeo7lD6LfoIKxefavc4AySmlcwVY=
    =z/VF
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Johannes Schauer Marin Rodrigues@21:1/5 to All on Sat Jul 19 21:30:01 2025
    Hi,

    your subject suggests that you are looking for a solution for huge upstream packages in general but reading your mail I get the feeling that you are looking for a solution for src:linux specifically? I'm asking because the way that src:linux is packaged in Debian contains a *lot* of custom code (a bit more than 4000 lines of Python) which afaik is only used there and nowhere else in Debian. So any solution you get for src:linux will probably only be applicable there and at the same time because src:linux does contain so much custom tooling, any general purpose solution will probably not work for it..

    Quoting Marc Haber (2025-07-18 22:16:25)
    I would like to re-work my linux kernel building process and would like to do it right this time: Build a clean source package that dak and sbuild will accept. I do not plan to upload this to Debian, but I think that we all can learn from some discussion.

    The Debian packaging of src:linux contains a lot of Python code which re-generates the control files and Makefile code that you tell it to generate. So it is that Python code that you would have to learn how to interact with so that it can generate the source package you want for you.

    I'm maintaining a downstream "fork" or patch-stack against the Debian Linux packaging from Debian to build the custom kernel packages for the MNT Reform open hardware laptops. We publish these kernels on the official Debian repositories by MNT and I use the same tooling to build them against Debian Trixie (and Trixie-backports in the future) and publish them on reform.debian.net.

    I met with Ben Hutchings at MiniDebConf in Berlin last year and received a lot of tips on how to better use the Debian Linux kernel packaging in the MNT downstream derivative. One essential tip was that what I want should probably become its own "flavour". The Debian kernel packaging is able to build the kernel for multiple flavours, each with their own configuration and they end up in differently named binary packages. This is how we now produce kernel packages for the MNT laptops which are named linux-image-mnt-reform-arm64 or linux-headers-mnt-reform-arm64 where "mnt-reform-arm64" is the flavour name.

    To create our own flavour, we create a file in debian/config.local which will override files that the official packaging is putting into debian/config. Specifically, our debian/config.local/arm64/defines.toml looks like this:

    --%<-----------------------------------------------
    [[flavour]]
    name = 'mnt-reform-arm64'
    [flavour.defs]
    is_default = true
    [flavour.packages]
    installer = false
    docs = false
    [flavour.description]
    hardware = '64-bit ARMv8 machines (MNT Reform)'
    hardware_long = 'MNT Reform 2, MNT Pocket Reform and Reform Next'

    [[featureset]]
    name = 'none'

    [[featureset.flavour]]
    name = 'mnt-reform-arm64'

    [build]
    enable_signed = false
    %-----------------------------------------------

    As for the kernel config, we just dump all the custom kernel config settings which are not part of the Debian packaging into debian/config/arm64/config (that should probably also go into debian/config.local?)

    After you edit these things you have to run a bunch of scripts like kernel-team/utils/kconfigeditor2/process.py which will clean up your kernel config, which I think is optional?

    Non-optional is running debian/rules targets like "source" which ends up running
    "make -f debian/rules.gen source" to regenerate the dsc and this:

    make -f debian/rules debian/control-real

    This will regenerate the control files and Makefile snippets given the new information that you configured using toml configs like the one above.

    I would like to build differnt "flavors", lets call them linux-server, linux-vm, linux-desktop and linux-notebook. But I'd like to have them
    from - at least - the same .orig.tar.gz, just for the reason that this file is about 150 MB large.

    I think you used the right term when you called what you want to build "flavours". I think what you might want to do might be to hack up your own flavour in the respective toml file in debian/config.local, throw in your custom kernel config, run the debian/rules target to regenerate debian/control and the start sbuild. We throw in the following build profiles to vastly speed up our builds: nodoc, pkg.linux.nokerneldbg, pkg.linux.nokerneldbginfo.

    I also learned a lot reading the excellent handbook of the kernel team: https://kernel-team.pages.debian.net/kernel-handbook/

    I'd also like to mention that a lot of things have changed in this department recently. For example, the toml config format was only introduced in kernel version 6.7 and more and more things have recently become configurable via toml. For example we used to patch the kernel team's Python code to do things but two months ago, Ben submitted https://salsa.debian.org/kernel-team/linux/-/merge_requests/1481 and now instead of patching code, we can just edit the toml.

    The kernel team has been very helpful with supporting me as their downstream and has added more and more convenience features which make work for me as their downstream easier over the past months. I'm sure you'll have a pleasant time as well as their downstream. :)

    Thanks!

    cheers, josch
    --==============‚76959428096054564=MIME-Version: 1.0
    Content-Transfer-Encoding: 7bit
    Content-Description: signature
    Content-Type: application/pgp-signature; name="signature.asc"; charset="us-ascii"

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEElFhU6KL81LF4wVq58sulx4+9g+EFAmh78WQACgkQ8sulx4+9 g+GzYQ//QoQ+uwOT5AqHiShrggblZtHglYTMd8llJJdx9S68wl1O/SketHKFbyXY lIzS6TIDrOvzuk53J2VCFU2VTacmw2b4h2IE6MCxS0GIcRzQKJXk7MipceMprB4E LjrIrJYDkORIkAp6m0D/+gTNRrdbOjgpSnHjBRXEIrzdbYf5Fbnuk9wfy5/57ZvH I1S7SpxMdkw642pEAxUjqgdJrLE7pJCAjlDDmDiXn/KC+IOQhee16EsYXzqORwDo /zIquREnE+6n1qsSyZN6kIT3z7ofGuFTHGDvMFwHBy1rslA8iwE38PA5Bq9qv50r iV+usI/CNNo+Cfa2/pN13tlGTqQYQhcq/rB4b7g7FO/B/Bb5J8lHDayGhssi8VHi WFpGTL5GwBxdfOvQ+Qbh6rJ/BaUQFbGAzcA97BcENf/JlejxDOtRSm5qMaG6aIDi nLGKxOFFXCwfaEtKWO9VHxWEMfVeOVuPwJxHy5YPu+CkssirrwP3yDzk62rkn6lq n9Cquv34rSkuig9e9CCFm9bgczGV+XB03yA6Ymi9iJslImUjK+fjPYzlEotP34Sz cNN1PLlDE/5Y/nXYahixnlJmO5kYXD8Y7u7d/hqxs8hxRI/hE+Rh3dm84uS1srL4 h6ULRGGT+zBYa8AT1Y2Q5vx5gDyOiZzoyjMvP7U/30OuyLNgd3g=
    =+NQg
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ben Hutchings@21:1/5 to Marc Haber on Sun Jul 20 20:20:01 2025
    On Fri, 2025-07-18 at 22:16 +0200, Marc Haber wrote:
    Hi,

    I would like to re-work my linux kernel building process and would like
    to do it right this time: Build a clean source package that dak and
    sbuild will accept. I do not plan to upload this to Debian, but I think
    that we all can learn from some discussion.

    I would like to build differnt "flavors", lets call them linux-server, linux-vm, linux-desktop and linux-notebook. But I'd like to have them
    from - at least - the same .orig.tar.gz, just for the reason that this
    file is about 150 MB large.
    [...]

    You can build a linux-source-<version> binary package (or use the ones
    Debian already builds) and then build-depend on that in your per-
    configuration source packages.

    You can also do most of this as a derivative version of src:linux. We
    don't define a profile for building a single flavour (and I think that
    would get out of hand), but if you only need that option for development purposes the debian/bin/test-patches script shows how to do that.

    Ben.

    --
    Ben Hutchings
    It is impossible to make anything foolproof
    because fools are so ingenious.

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEErCspvTSmr92z9o8157/I7JWGEQkFAmh9McUACgkQ57/I7JWG EQnbchAAq1+j021nXWiPXF9KlUGA+gh+UoG/rUlROug+ScjLlWgxiDy25e5bI5Lf P3TLeALr8eh9W8bP4c+plwrBjkS1NWlY5HcvNWJmp0wKPiEMtxEKlFL4V733SgVL 2Nxiu6qIAIsrwqTBWx2II10rGRGRt89Vw3fb9RTYxKgSulfy5uek8o3ggNaUi2AH kyNLb51BrLniHwyFhckzOfGAWB03wkjDLEj8jeNlL95Sb4P2ixBLfTNYpIANO1lJ lLOK4U8tH88W6wUWf2iE4q44xbTtCQIVOG1EMPhJpbVRey7A5cuQNeh2yfT6ciyc laZCq2lv9nWJeVOyB7jMaxeJhb54aMvjt1hoUXrnLoSAlo98R5Kr1HQNsC/BGXlx lbuTpdxx2R0QhrShoNXyQjBs6ZdyukkzZ7ipOkYolZlrKJLCUXdmV+M61xD8rbj1 LMm9/FpHp7NJHqSj/sS/fgwW3L6R3schAlyL000JAKAWcZA3dzphUN6RYXPk82mH j90Cim+Q+P3D9eIlGUCp8HTOMwtqAFTj2LXKPw1+GnZK9NmOcivfkkpR3qY48LZK gMXj6of+7l5wjBL2LXI44D65mbKcBBTugDa6Bj/M8UFhK0zOx7tBQN7waAUlveX+ Skxax0fj6R0N4PUFYUQEq0ShyZ6uXzGxxZfH/gfPu8W4Pz1DOjc=
    =jTYI
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Marc Haber@21:1/5 to All on Fri Jul 25 20:20:01 2025
    Hi,

    I have solved the issue for me. I decided to stick to the most
    conservative approach of one fully fledged source package per kernel
    flavor and will solve the issue of wasted disk space on both the
    internal archive host and on the publicly visible download server with
    offline deduplication on the filesystem level. I think I will use btrfs
    on the internal archive host and XFS on the download server to get
    familiar with both methods, albeit similar. Zugschlus' leftover plate ("Resterampe") of packages that noone wants will probably grow by one in
    this process as I have started the salvaging process for duperemove
    (which can thankfully both handle XFS and btrfs).

    Of the suggestions I received the idea of having a
    linux-zugschlus-source binary package that the flavor packages could build-depend on the most enticing. I still decided to go a different way
    since I didn't have so much time to spend. I wrote a python program (>
    1000 lines *cough*) that builds the source packages using big parts of
    what upstream (and Wichert) have in scripts/packages, generating new
    stuff, including glue code to handle the kernel configuration files,
    updates, etc, and orchestrates the build of the four source packages and
    the respective binaries.

    I decided against using Debian's src:linux because the packaging is WAY
    more complicated (and harder to understand) that I'll ever need, and I
    am usually faster in deploying new stable kernel releases than Debian
    is.

    Build profiles I thankfully didn't need since the upstream packaging
    already fiddles around with the control file to control the creation of
    kernel headers, linux-libc6-dev and debug packages. It is an interesting concept but I'd rather take more time to understand it, and first use it
    the way it was intended to be used.

    My original idea of having one orig.tar.gz for more than one source
    package wouldn't work because that use case is not supposed by
    dpkg-source: the orig.tar.gz always must have the same base name as the
    source package (even multiple source tarballs have to share the same
    prefix), hence I would need multiple, different, but identically named
    source packages which you can't put in an archive. That was a stupid
    idea of mine and I shouldn't have bothered you with that.

    The most obvious but also time-consuming takeway was that you need to
    have ARCH and KBUILD_DEBARCH correctly set even for the
    {old,n,menu}config step or you will end up with a non-working kernel
    that will miss all parts specific to the target architecture when cross-building.

    What I still don't really know is how to handle the linux-libc-dev
    package which doesn't have its name versioned and will therefore be
    built non-identically, but identically named, by every flavor, doubling
    the package name in the archive which doesn't make dak happy. I decided
    to only build that package from one of my flavors.

    With the current results I can work for the time being. I might
    simplify the packaging at some future time (build profiles might
    actually help here, making the debian/rules way simpler than the one
    that comes with the kernel).

    Thank you all for trying to help.

    Greetings
    Marc

    -- ----------------------------------------------------------------------------- Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany | lose things." Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Marc Haber@21:1/5 to Johannes Schauer Marin Rodrigues on Fri Jul 25 20:20:01 2025
    On Sat, Jul 19, 2025 at 09:26:33PM +0200, Johannes Schauer Marin Rodrigues wrote:
    your subject suggests that you are looking for a solution for huge upstream >packages in general but reading your mail I get the feeling that you are >looking for a solution for src:linux specifically?

    The Linux kernel is probably the most prominent example of this use case
    and I can't come up with another package that has this kind of
    complexity, but I actually tried to take a few steps back from the
    kernel and to look at the challenge from a more general point of view.
    That didn't work though.

    I'd also like to mention that a lot of things have changed in this department >recently. For example, the toml config format was only introduced in kernel >version 6.7 and more and more things have recently become configurable via >toml.

    The last time I looked at the Debian kernel packaging was 20 years ago.
    Glad to hear that things have become simpler.

    Greetings
    Marc


    -- ----------------------------------------------------------------------------- Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany | lose things." Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)