1. Forum
  2. Usenet
  3. LINUX.DEBIAN.MAINT.DPKG

  • Re: Building with many cores without OOM (2/2)

    From Guillem Jover@1:229/2 to Helmut Grohne on Mon Dec 9 14:50:01 2024
    [continued from previous message]

    standardize environment variables would seem sufficient I think,
    but while all this seems kind of optional, this goes a bit into
    reliance on dpkg-buildpackage being the only supported build
    entry point. :)

    To me, this reads as an argument for using an inside-out approach.

    Given all of the other replies (on-list and off-list), my vision of how
    I'd like to see this approached has changed. I see more and more value
    in leaving this in close control of the package maintainer (i.e.
    inside-out) to the point where different parts of the build may use
    different limits.

    I think this would be fine too. The above point was more an option to
    implement an hybrid permeable outside-in, where dpkg-buildpackage
    could then try to gather say the amount of available memory, number of
    cores, and any other potentially relevant additional data, then perhaps
    try to use a declarative hint from the packaging to tune either the
    default parallel jobs value or provide a new variable, and then let
    the inner processes decide based on those variables, which would avoid
    having to duplicate much of the data gathering and potential
    portability issues.

    This could also imply alternatively or in addition, providing a tool
    or adding some querying logic in an existing tools (in the dpkg toolset)
    to gather that information which the packaging could use, or…

    How about instead we try to extend coreutils' nproc? How about adding
    more options to it?

    --assume-units=N
    --max-units=N
    --min-ram-per-unit=Z

    …as you mention, another build-essential package's tools, so that again
    we do not need to duplicate much of the logic. :)

    Then, we could continue to use buildopts.mk and other mechanism to
    extract the passed parallel value from DEB_BUILD_OPTIONS as before and
    run it through an nproc invocation for passing it down to a build system
    in the specific ways that the build system requires. More options could
    be added to nproc as-needed.

    So, yeah that would also work.

    Thanks,
    Guillem

    --- SoupGate-Win32 v1.05
    * Origin: you cannot sedate... all the things you hate (1:229/2)