In line with the email thread and email title, The purpose of this email is
to start a collaboration on how best we can optimize Salsa CI by migrating
from using *dpkg-buildpackage* to *sbuild* for building packages. The
reason is to reduce the build time it may take for testing packages. Other reasons are referenced in this MR <https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/404>'s description.

So far there has been a lot of progress as reflected in the diffs <https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/404/diffs>. However. One major block that we are having is integrating sbuild with
ccache. The pipeline generally has 3 stages;

1. Images - This creates reusable docker containers using these
Dockerfiles
<https://salsa.debian.org/kaxada/pipeline/-/tree/optimization-with-sbuild/images/dockerfiles>
that
are stored in GitLab to be reused anytime a job is called. these images
need to be built and availed so that the build time is reduced.
2. Build - This fetches the necessary upstream resources for a
particular package and uses builds the package depending on the *host,
architecture and build environment variables*. Currently, this process
is using dpkg-buildpackage and this is the section we want to integrate
with sbuild
3. Clean - This cleans up the used images for further use.

We have been thinking of integrating ccache with sbuild but we are having issues with which stage we can implement that so that it runs successfully.
I created a long comment that tries to explain some of my analysis of the
whole pipeline <https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/404#note_393173>
and why it would be failing. please use it as a reference. The latest build error that I'd also need some enlightenment on is on this line. <https://salsa.debian.org/kaxada/pipeline/-/jobs/4117029#L283>

I would have been verbose in my here but I have tried to provide the
necessary links and a basic explanation of what the scenario is.

I hope to collect perspectives and also answer some questions where I can.

Regards,
Enock


On Fri, Apr 7, 2023 at 6:46 AM Johannes Schauer Marin Rodrigues < josch@debian.org> wrote:

Hi Enoch,

Quoting Enoch Kaxada (2023-04-07 01:32:24)

I am writing this email to open up a collaboration so that we can work together.

We are working on integrating sbuild into salsa CI to reduce the build

time

mostly. Here is the MR <https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/404>. There are several threads of discussions under the MR to see that we set

up

sbuild in the best way possible but we think we need extra eyes.

Our main challenge is using ccache together with sbuild and the latest implementation is on this HEAD
<

https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/404/diffs

.

We are looking so solid explanations as to why the pipelines errors are

the

way they are.

I got recently diagnosed with multiple sclerosis and am still unable to
feel my
feet and fingers. The latter makes using my computer very hard and frustrating.
Please be more specific what problem you are having with sbuild so that I
can
minimize my time in front of the keyboard.

Also note, that while I probably know subild best, I have no idea about ccache
integration with it. So I might not be of much help.

Thanks!

cheers, josch

<div dir="ltr">In line with the email thread and email title, The purpose of this email is to start a collaboration on how best we can optimize Salsa CI by migrating from using <b>dpkg-buildpackage</b> to <b>sbuild</b> for building packages. The reason
is to reduce the build time it may take for testing packages. Other reasons are referenced in <a href="https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/404">this MR</a>&#39;s description. <div><br></div><div>So far there has been a lot 
of progress as reflected in the <a href="https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/404/diffs">diffs</a>. However. One major block that we are having is integrating sbuild with ccache. The pipeline generally has 3 stages;</div><

<ol><li>Images - This creates reusable docker containers using these <a href="https://salsa.debian.org/kaxada/pipeline/-/tree/optimization-with-sbuild/images/dockerfiles">Dockerfiles</a> that are stored in GitLab to be reused anytime a job is called.

these images need to be built and availed so that the build time is reduced. </li><li>Build - This fetches the necessary upstream resources for a particular package and uses builds the package depending on the <b>host, architecture and build
environment variables</b>. Currently, this process is using dpkg-buildpackage and this is the section we want to integrate with sbuild</li><li>Clean - This cleans up the used images for further use. </li></ol><div>We have been thinking of integrating
ccache with sbuild but we are having issues with which stage we can implement that so that it runs successfully. I created a <a href="https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/404#note_393173">long comment that tries to explain
some of my analysis of the whole pipeline</a> and why it would be failing. please use it as a reference. The latest build error that I&#39;d also need some enlightenment on is <a href="https://salsa.debian.org/kaxada/pipeline/-/jobs/4117029#L283">on this
line.</a> </div></div><div><br></div><div>I would have been verbose in my here but I have tried to provide the necessary links and a basic explanation of what the scenario is. </div><div><br></div><div>I hope to collect perspectives and also answer
some questions where I can. </div><div><br></div><div>Regards,</div><div>Enock</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Apr 7, 2023 at 6:46 AM Johannes Schauer Marin Rodrigues &lt;<a href="mailto:
josch@debian.org">josch@debian.org</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi Enoch,<br>

Quoting Enoch Kaxada (2023-04-07 01:32:24)<br>
&gt; I am writing this email to open up a collaboration so that we can work<br> &gt; together.<br>
&gt; <br>
&gt; We are working on integrating sbuild into salsa CI to reduce the build time<br>
&gt; mostly. Here is the MR<br>
&gt; &lt;<a href="https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/404" rel="noreferrer" target="_blank">https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/404</a>&gt;.<br>
&gt; There are several threads of discussions under the MR to see that we set up<br>
&gt; sbuild in the best way possible but we think we need extra eyes.<br>
&gt; <br>
&gt; Our main challenge is using ccache together with sbuild and the latest<br> &gt; implementation is on this HEAD<br>
&gt; &lt;<a href="https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/404/diffs" rel="noreferrer" target="_blank">https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/404/diffs</a>&gt;<br>
&gt; .<br>
&gt; <br>
&gt; We are looking so solid explanations as to why the pipelines errors are the<br>
&gt; way they are.<br>

I got recently diagnosed with multiple sclerosis and am still unable to feel my<br>
feet and fingers. The latter makes using my computer very hard and frustrating.<br>
Please be more specific what problem you are having with sbuild so that I can<br>
minimize my time in front of the keyboard.<br>

Also note, that while I probably know subild best, I have no idea about ccache<br>
integration with it. So I might not be of much help.<br>

Thanks!<br>

cheers, josch</blockquote></div>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)