1. Forum
  2. Usenet
  3. LINUX.DEBIAN.DEVEL.CD

  • Re: Nearly reproducible Bookworm 12.6 live images

    From Holger Levsen@21:1/5 to Roland Clobus on Wed Jul 3 19:00:01 2024
    Hi Roland,

    On Wed, Jul 03, 2024 at 06:44:37PM +0200, Roland Clobus wrote:
    I'm sooo close...
    [...]

    hehe, congrats & thanks for keeping us in the loop! In honor of your
    work I've manually written the signature of this email...! :)


    --
    cheers,
    Holger

    ⢀⣴⠾⠻⢶⣦⠀
    ⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org
    ⢿⡄⠘⠷⠚⠋⠀ OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
    ⠈⠳⣄

    The devel is in the details.

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEuL9UE3sJ01zwJv6dCRq4VgaaqhwFAmaFglkACgkQCRq4Vgaa qhwi1Q/8CQuqRCM6wh4Ve+ArZh2fDh8Sy2khW1tIQEDGG56KcXMqiwQFPcO6L8er xPeZrRwQr0Ky0YpU3u1oxD4gR51RAskC6vqKqAIB2g5G6/9INATcCjV+tN2ly3cP d2S0TWXkoN/iUbpovnSFVq5kakZKOmuF5nuEeSZC3ZOT6M+UNaKSs/YlsNCrBqb3 yBUcnAJEQqFiX9+jpdT96YgAgUO11vQbiHAbCayxE+LJXa9UJqhWm3VxBkVca7PF PQ5FohTg1aAUkos/Rx4gyzFHR1dyY+jezyZmpWJkpgDxYN6FSWJcqp+X7XmLxGiG ybYJNAfDXF4o+cV3z4Xrrpf7/bINRgu+dNTvzMqd3DsoI+sfYxsRuYeXkvJWVFWd w7AvU9Cr8gUkwtar2m6+akysE+Ie/L8rfzgx+e9MRMVr2CMN6jJDekFEXSsai80+ XjKeu7oUis2KZiI5KMnbFB5cD8l2EeaayAWkmd/YUUWVReesOBfitja5X74QsIGJ VMBRI72wgNB/rfmNvBJjcg4tbJFjBTP50lY2cSga6V1csDEUTtBTkFYvd2K6Cvkp lP+WM9iJuCDEXYUoSQ8XQ56kRYwQHKD7sCJvGdsHCGL80q1FpsKhnUG/FNZJ5Db3 SUOmX65RjJFwvQlpw6
  • From Philip Hands@21:1/5 to Roland Clobus on Wed Jul 3 21:40:01 2024
    Roland Clobus <rclobus@rclobus.nl> writes:

    Hello lists,

    I'm sooo close...

    Now that that 12.6 live images have been generated [1], there is only
    one embedded timestamp left (/boot/grub/live-theme/theme.txt and its
    parent folder)
    I've not seen this timestamp issue before, as I am using a slightly different way of building the image than live-setup uses.

    live-setup uses a local cache of the git repository of live-build, to
    avoid being blocklisted by salsa for heavy traffic, caused by parallel builds.
    This local cache brings that this file could have an old timestamp,
    whereas my local builds always used 'now' (caused by git clone), which
    would be properly truncated to SOURCE_DATE_EPOCH, and therefore be reproducible.

    AFAIK it's the repeated full clones that get you blocklisted.

    I was having the same issue with the openQA workers which I've fixed by
    adding a --reference-if-able option pointing at a locally accessible
    copy of the repo on the workers (in fact the "local" copy is an nfs
    mount, but it's enough to ensure that the worker is not trying to do a
    full clone from salsa, and so avoids the blocklist)

    I guess the thing to do is a fetch against the cache repo first just to
    make sure it stays updated, then a clone while pointing at that repo
    via the --reference-if-able option, then it'll still give you the same
    result as doing a clone, without overloading salsa.

    Cheers, Phil.
    --
    Philip Hands -- https://hands.com/~phil

    --=-=-Content-Type: application/pgp-signature; name="signature.asc"

    -----BEGIN PGP SIGNATURE-----

    iQIzBAEBCgAdFiEE3/FBWs4yJ/zyBwfW0EujoAEl1cAFAmaFpwIACgkQ0EujoAEl 1cDIbxAAmSv6K+Le4AJNC4uR6A42+Ju4sgYEbBl8Fq0oa9GK4Lz0Xzpx+kb+T3wE ya+VabVmkvkAqhqEseVorcct67RCpA6bmQYvS+tNOwzAe0sC2iqUkf6MHAi9Ch7e c0gIiIhOaB+bckrmQHmFlv2iTveCJEbfjZ1DHDZtN+mgyBbYKgNe5pvdPD9F+DGQ mb13H4ke8YPCzlbClRoOWl+n+l5EAiCWzJFKrVkJis6VlPiWDzivJbcauFHKpqud pPt96IDBsgtL90EPfyoy7CWVPoE/g55enxeySr9WKrynaQlLMtNdxrp/zXeg0bMe SKgU+IrCXh2BxeDaolxDtRFyPL23tjRdLIiqgGjn8F3DtD7A+16nMkaM6acH0nw8 CqeX4HB3xMJremaPdfDmUvfXsH1Npt2QruH91i28unRlV/d3CkW0gnPvRgCTKqIC OydNeKr72dS5XIyxU6jX4AMg78HNiGUgGSedGaK7Ul5qX9eThsX4UgWMXsZIM/Ip XIDJqtBoHQP0aZaJo6x4SvHF/irXsVKR+hu6KGbcuWsh5JiglyNAwJlz4uL7vzno akxW9mqkAjlZUbeZ/K2B2xak6UI6IokcIbFBXAfB/nHdzyxpWl5CFLJn0IWm2hJk aIGboVDD2+Yd3Wv8QO83eif4PFZkvWsFEcRK/c6jvGB2dd0VgaY=0qar
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gatewa
  • From Steve McIntyre@21:1/5 to Roland Clobus on Wed Jul 3 23:20:01 2024
    On Wed, Jul 03, 2024 at 06:44:37PM +0200, Roland Clobus wrote:
    Hello lists,

    I'm sooo close...

    Now that that 12.6 live images have been generated [1], there is only one >embedded timestamp left (/boot/grub/live-theme/theme.txt and its parent >folder)
    I've not seen this timestamp issue before, as I am using a slightly different >way of building the image than live-setup uses.

    live-setup uses a local cache of the git repository of live-build, to avoid >being blocklisted by salsa for heavy traffic, caused by parallel builds.
    This local cache brings that this file could have an old timestamp, whereas >my local builds always used 'now' (caused by git clone), which would be >properly truncated to SOURCE_DATE_EPOCH, and therefore be reproducible.

    I'll prepare a patch, and hopefully the next set of live images (12.6.1 or >12.7.0) will be fully reproducible.

    Awesome stuff! :-)

    --
    Steve McIntyre, Cambridge, UK. steve@einval.com "When C++ is your hammer, everything looks like a thumb." -- Steven M. Haflich

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From James Addison@21:1/5 to Roland Clobus on Tue Jul 16 00:40:01 2024
    Hi Roland!

    This sounds great - I have one specific question / concern:

    On Wed, 3 Jul 2024 at 17:44, Roland Clobus <rclobus@rclobus.nl> wrote:
    [ ... snip ... ]
    live-setup uses a local cache of the git repository of live-build, to
    avoid being blocklisted by salsa for heavy traffic, caused by parallel builds.
    This local cache brings that this file could have an old timestamp,
    whereas my local builds always used 'now' (caused by git clone), which
    would be properly truncated to SOURCE_DATE_EPOCH, and therefore be reproducible.

    Limiting the timestamp to a maximum of SOURCE_DATE_EPOCH makes sense so that the modification-time is consistent for equivalent rebuilds.

    However: if the contents of the txt file / other content in the git repository change, would that prevent an independent rebuilder from recreating the identical CD image as output?

    (note: given that the package archive is regularly updated, I think that there are other reasons why an identical point-in-time rebuild may require sharing
    of some build-time/archive metadata -- the reason I'm asking is that I'd like to check whether this git repo could require similar co-ordination during rebuilds. fixed commit ID / signed tag, or other mechanism for example)

    Regards,
    James

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)