• Re: [gentoo-dev] Interest in a yarn / NPM eclass

    From Andrew Ammerlaan@21:1/5 to Chris Pritchard on Mon Jun 6 13:20:01 2022
    Hi Chris,

    I think @Alessandro (CC) has already done some work on this over at [1].
    I'm not sure what the status of it is though. Maybe you two can discuss
    this together to avoid doing double work.

    Best regards,
    Andrew

    [1] https://github.com/Alessandro-Barbieri/node-overlay

    On 05/06/2022 07:47, Chris Pritchard wrote:
    Hello,

    Would there be any interest in a yarn / NPM eclass that supports offline installs. For a personal overlay I’ve got a working yarn.eclass (https://github.com/chriscpritchard/overseerr-overlay/blob/main/eclass/yarn.eclass
    <https://github.com/chriscpritchard/overseerr-overlay/blob/main/eclass/yarn.eclass>)
    and I’ve been able to make a version that supports npm from
    NPM-Shrinkwrap or npm-lock.json files (this is still being tested).

    If there is an interest, would anyone be willing to support me in having
    an eclass added to the tree?

    Thanks,

    Chris


    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ionen Wolkens@21:1/5 to Andrew Ammerlaan on Mon Jun 6 13:40:02 2022
    On Mon, Jun 06, 2022 at 01:16:54PM +0200, Andrew Ammerlaan wrote:
    Hi Chris,

    I think @Alessandro (CC) has already done some work on this over at [1].
    I'm not sure what the status of it is though. Maybe you two can discuss
    this together to avoid doing double work.

    There's been others too, e.g. https://git.sr.ht/~dblsaiko/ebuilds/tree/master/item/eclass/npm2x.eclass

    Haven't needed npm stuff myself but I was keeping looking at that
    one in mind if I ever needed to. Albeit at this point I'd likely
    not want the giant manifest+variables required for these without
    a vendor tarball.


    Best regards,
    Andrew

    [1] https://github.com/Alessandro-Barbieri/node-overlay

    On 05/06/2022 07:47, Chris Pritchard wrote:
    Hello,

    Would there be any interest in a yarn / NPM eclass that supports offline installs. For a personal overlay I’ve got a working yarn.eclass (https://github.com/chriscpritchard/overseerr-overlay/blob/main/eclass/yarn.eclass
    <https://github.com/chriscpritchard/overseerr-overlay/blob/main/eclass/yarn.eclass>)
    and I’ve been able to make a version that supports npm from NPM-Shrinkwrap or npm-lock.json files (this is still being tested).

    If there is an interest, would anyone be willing to support me in having an eclass added to the tree?

    Thanks,

    Chris




    --
    ionen

    -----BEGIN PGP SIGNATURE-----

    iQEzBAABCAAdFiEEx3SLh1HBoPy/yLVYskQGsLCsQzQFAmKd5x8ACgkQskQGsLCs QzR/5gf+KmrR4Zc1khVHO8VCTKbXdHjbwfn97EORx+yJu7JURbqpgKVq7bK+gW1P XKgI/4DBavix7W+bm4rMSDQ8pBdJ0wYlGONSuDlF8mgPpQFo7SxqP5m/hiQ4XS4l pz0mqEi26fJiagZtAxElvb2+tts68wD1evxoBS9hC1msqAhaPzDLgjOafBR/9DAn w8cjVIKBlwPOjDpIzk0pa4B+31kT+wiF60h7RwFQ23t7iWi5NlABzQNvK2OzJRwI XVyldefxxl8QZYC3M+7tW86kI3U1uZF2OxAVdIL3k73x7auHIIRhRFRwtU+US4+D j4pNKkapfY5tFhvp9zHA9MG+OcEfkg==
    =AdRX
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?iso-8859-1?Q?Rapha=EBl_Barrois?=@21:1/5 to Ionen Wolkens on Mon Jun 6 20:10:01 2022
    On Monday, 6 June 2022 13:38:07 CEST, Ionen Wolkens wrote:
    On Mon, Jun 06, 2022 at 01:16:54PM +0200, Andrew Ammerlaan wrote:
    Hi Chris,

    I think @Alessandro (CC) has already done some work on this over at [1].
    I'm not sure what the status of it is though. Maybe you two can discuss
    this together to avoid doing double work.

    There's been others too, e.g. https://git.sr.ht/~dblsaiko/ebuilds/tree/master/item/eclass/npm2x.eclass

    Haven't needed npm stuff myself but I was keeping looking at that
    one in mind if I ever needed to. Albeit at this point I'd likely
    not want the giant manifest+variables required for these without
    a vendor tarball.

    Best regards,
    Andrew

    [1] https://github.com/Alessandro-Barbieri/node-overlay

    On 05/06/2022 07:47, Chris Pritchard wrote: ...


    For what it's worth, I have made similar attempts in my personal overlay,
    based on a small helper that exposes a local folder as an NPM cache which
    will work as expected for the NPM command line with minimal updates.

    The helper is at https://github.com/rbarrois/cjs-ledger
    The eclass (could be better) at https://git.xelnor.net/?p=overlay.git;a=blob;f=eclass/xel-nodejs.eclass;hb=HEAD

    An example usage can be found here: https://git.xelnor.net/?p=overlay.git;a=blob;f=www-apps/xelpaste/xelpaste-3.2.1.ebuild;hb=HEAD

    If this can be of interest, feel free to ping me!

    --
    Xelnor

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Alessandro Barbieri@21:1/5 to All on Wed Jun 8 19:30:01 2022
    Il Lun 6 Giu 2022, 13:16 Andrew Ammerlaan <andrewammerlaan@gentoo.org> ha scritto:

    Hi Chris,

    I think @Alessandro (CC) has already done some work on this over at [1].
    I'm not sure what the status of it is though. Maybe you two can discuss
    this together to avoid doing double work.

    Best regards,
    Andrew

    [1] https://github.com/Alessandro-Barbieri/node-overlay

    On 05/06/2022 07:47, Chris Pritchard wrote:
    Hello,

    Would there be any interest in a yarn / NPM eclass that supports offline installs. For a personal overlay I’ve got a working yarn.eclass
    (
    https://github.com/chriscpritchard/overseerr-overlay/blob/main/eclass/yarn.eclass
    <
    https://github.com/chriscpritchard/overseerr-overlay/blob/main/eclass/yarn.eclass>)

    and I’ve been able to make a version that supports npm from NPM-Shrinkwrap or npm-lock.json files (this is still being tested).

    If there is an interest, would anyone be willing to support me in having
    an eclass added to the tree?

    Thanks,

    Chris



    I'm interested in an eclass that doesn't bundle everything together. Also
    I'm interested in anyone that can share the package maintainership (in guru first).

    I've already tried 3 approaches:
    1) one package per dependency. The npm eclass is working fine for now, you
    can unbundle packages that depend on system libs (like sqlite), the major
    issue I've found is in the circular dependencies of the rollup package.

    2) bundle everything. This approach doesn't always work. Some packages fail
    to build dependencies written in C and you can't unbundle them.

    3) package every runtime dependency and bundle build time dependencies.
    Since rollup is a build time dep, I've tried to bundle every build time dep
    but this require to create a custom stuff and host it somewhere.

    <div dir="auto"><div><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Il Lun 6 Giu 2022, 13:16 Andrew Ammerlaan &lt;<a href="mailto:andrewammerlaan@gentoo.org">andrewammerlaan@gentoo.org</a>&gt; ha scritto:<br></div><blockquote class="gmail_
    quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Chris,<br>

    I think @Alessandro (CC) has already done some work on this over at [1]. <br> I&#39;m not sure what the status of it is though. Maybe you two can discuss <br>
    this together to avoid doing double work.<br>

    Best regards,<br>
    Andrew<br>

    [1] <a href="https://github.com/Alessandro-Barbieri/node-overlay" rel="noreferrer noreferrer" target="_blank">https://github.com/Alessandro-Barbieri/node-overlay</a><br>

    On 05/06/2022 07:47, Chris Pritchard wrote:<br>
    &gt; Hello,<br>
    &gt; <br>
    &gt; Would there be any interest in a yarn / NPM eclass that supports offline <br>
    &gt; installs. For a personal overlay I’ve got a working yarn.eclass <br> &gt; (<a href="https://github.com/chriscpritchard/overseerr-overlay/blob/main/eclass/yarn.eclass" rel="noreferrer noreferrer" target="_blank">https://github.com/chriscpritchard/overseerr-overlay/blob/main/eclass/yarn.eclass</a> <br>
    &gt; &lt;<a href="https://github.com/chriscpritchard/overseerr-overlay/blob/main/eclass/yarn.eclass" rel="noreferrer noreferrer" target="_blank">https://github.com/chriscpritchard/overseerr-overlay/blob/main/eclass/yarn.eclass</a>&gt;) <br>
    &gt; and I’ve been able to make a version that supports npm from <br>
    &gt; NPM-Shrinkwrap or npm-lock.json files (this is still being tested).<br> &gt; <br>
    &gt; If there is an interest, would anyone be willing to support me in having <br>
    &gt; an eclass added to the tree?<br>
    &gt; <br>
    &gt; Thanks,<br>
    &gt; <br>
    &gt; Chris<br>
    &gt;</blockquote></div></div><div dir="auto"><br></div><div dir="auto">I&#39;m interested in an eclass that doesn&#39;t bundle everything together. Also I&#39;m interested in anyone that can share the package maintainership (in guru first).</div><div dir=
    "auto"><br></div><div dir="auto">I&#39;ve already tried 3 approaches:</div><div dir="auto">1) one package per dependency. The npm eclass is working fine for now, you can unbundle packages that depend on system libs (like sqlite), the major issue I&#39;ve
    found is in the circular dependencies of the rollup package.</div><div dir="auto"><br></div><div dir="auto">2) bundle everything. This approach doesn&#39;t always work. Some packages fail to build dependencies written in C and you can&#39;t unbundle them.
    </div><div dir="auto"><br></div><div dir="auto">3) package every runtime dependency and bundle build time dependencies. Since rollup is a build time dep, I&#39;ve tried to bundle every build time dep but this require to create a custom stuff and host it
    somewhere.</div><div dir="auto"></div></div>

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Robin H. Johnson@21:1/5 to Alessandro Barbieri on Wed Jun 8 22:50:01 2022
    On Wed, Jun 08, 2022 at 07:23:15PM +0200, Alessandro Barbieri wrote:
    I'm interested in an eclass that doesn't bundle everything together. Also
    I'm interested in anyone that can share the package maintainership (in guru first).

    I've already tried 3 approaches:
    ...

    Since you know this yarn/NPM ecosystem well, could you evaluate two
    other ideas?
    4) Solutions like EGO_SUM
    5) EGO_SUM successor of 2nd-level-Metadata-Manifest that I described in the recent EGO_SUM thread.

    --
    Robin Hugh Johnson
    Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
    E-Mail : robbat2@gentoo.org
    GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
    GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v2
    Comment: Robbat2 @ Orbis-Terrarum Networks - The text below is a digital signature. If it doesn't make any sense to you, ignore it.

    iQKTBAABCgB9FiEEveu2pS8Vb98xaNkRGTlfI8WIJsQFAmKhCkFfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEJE RUJCNkE1MkYxNTZGREYzMTY4RDkxMTE5Mzk1RjIzQzU4ODI2QzQACgkQGTlfI8WI JsSGEw//YxouAQjR7NJ7WWtSHaeMM2wei8Tt6+cS5UmsoDUH76E1VdTk0R6aUeki KLV1qLvpMiALs44c+Bd5eTwBEjeAjQP8sD05GxcPAXjMRiAzR0wT3hr0Es8YXaV4 2Dkngu6grosG8At68bcYZ/WPbf0reI2RHL0mwV0B/Ydp/uJ8d1wvW/4enODTL+TS RbRuDQ2ffvogmZWZ6gnB5LSp1GhLksgmicq9QfV73fCh0L59Gs0yyRfNPOsMRu4H k8vGKFZVXeGeOcuhbKWRncyefyGrIeuETQJHLEX27c/xmRPJxkfwXiq7PswryOYJ DSwcH9iJNT51t7M9d8IF/9NatOMiDCTZweWescc/Loxl+NAV7BpXrt2ALI38OdiV xX4q+Be34K8xRBSf5Ddv
  • From Ryan Qian@21:1/5 to robbat2@gentoo.org on Thu Jun 9 07:00:02 2022
    I have an npm.eclass ( https://github.com/bekcpear/ryans-repos/blob/main/eclass/npm.eclass) that
    works like EGO_SUM.

    However, the package-lock.json file should be patched to convert sha1 to
    sha512 due to the default mechanism (script: https://github.com/bekcpear/npm-lockfile-to-sha512.sh).

    I maintain a package www-apps/filebrowser by this eclass in my overlay.


    On Thu, Jun 9, 2022 at 4:44 AM Robin H. Johnson <robbat2@gentoo.org> wrote:

    On Wed, Jun 08, 2022 at 07:23:15PM +0200, Alessandro Barbieri wrote:
    I'm interested in an eclass that doesn't bundle everything together. Also I'm interested in anyone that can share the package maintainership (in
    guru
    first).

    I've already tried 3 approaches:
    ...

    Since you know this yarn/NPM ecosystem well, could you evaluate two
    other ideas?
    4) Solutions like EGO_SUM
    5) EGO_SUM successor of 2nd-level-Metadata-Manifest that I described in
    the recent EGO_SUM thread.

    --
    Robin Hugh Johnson
    Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
    E-Mail : robbat2@gentoo.org
    GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
    GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136


    <div dir="ltr">I have an npm.eclass (<a href="https://github.com/bekcpear/ryans-repos/blob/main/eclass/npm.eclass">https://github.com/bekcpear/ryans-repos/blob/main/eclass/npm.eclass</a>) that works like EGO_SUM.<div><br></div><div>However, the package-
    lock.json file should be patched to convert sha1 to sha512 due to the default mechanism (script: <a href="https://github.com/bekcpear/npm-lockfile-to-sha512.sh">https://github.com/bekcpear/npm-lockfile-to-sha512.sh</a>).</div><div><div><br></div><div>I
    maintain a package www-apps/filebrowser by this eclass in my overlay.</div></div></div><div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jun 9, 2022 at 4:44 AM Robin H. Johnson &lt;<a href="
    mailto:robbat2@gentoo.org" target="_blank">robbat2@gentoo.org</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Wed, Jun 08, 2022 at 07:23:15PM +0200,
    Alessandro Barbieri wrote:<br>
    &gt; I&#39;m interested in an eclass that doesn&#39;t bundle everything together. Also<br>
    &gt; I&#39;m interested in anyone that can share the package maintainership (in guru<br>
    &gt; first).<br>
    &gt; <br>
    &gt; I&#39;ve already tried 3 approaches:<br>
    ...<br>

    Since you know this yarn/NPM ecosystem well, could you evaluate two<br>
    other ideas?<br>
    4) Solutions like EGO_SUM<br>
    5) EGO_SUM successor of 2nd-level-Metadata-Manifest that I described in the recent EGO_SUM thread.<br>

    -- <br>
    Robin Hugh Johnson<br>
    Gentoo Linux: Dev, Infra Lead, Foundation Treasurer<br>
    E-Mail   : <a href="mailto:robbat2@gentoo.org" target="_blank">robbat2@gentoo.org</a><br>
    GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85<br>
    GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136<br> </blockquote></div></div>

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Alessandro Barbieri@21:1/5 to All on Sat Jun 11 14:50:01 2022
    Il giorno mer 8 giu 2022 alle ore 22:45 Robin H. Johnson <robbat2@gentoo.org> ha scritto:

    On Wed, Jun 08, 2022 at 07:23:15PM +0200, Alessandro Barbieri wrote:
    I'm interested in an eclass that doesn't bundle everything together. Also I'm interested in anyone that can share the package maintainership (in
    guru
    first).

    I've already tried 3 approaches:
    ...

    Since you know this yarn/NPM ecosystem well, could you evaluate two
    other ideas?
    4) Solutions like EGO_SUM


    My second approach was like that


    5) EGO_SUM successor of 2nd-level-Metadata-Manifest that I described in
    the recent EGO_SUM thread.

    I havent' well understood it. In practice would be the same as 2/4 with
    the same issues.
    Let me remind you that unlike go/rust, nodejs packages aren't compiled and
    if you bundle everything you'll end up with a massive amount of files installed.
    Please don't consider node any different from python/ruby/perl/php.

    <div dir="auto"><div dir="ltr"><div dir="ltr">Il giorno mer 8 giu 2022 alle ore 22:45 Robin H. Johnson &lt;<a href="mailto:robbat2@gentoo.org" target="_blank" rel="noreferrer">robbat2@gentoo.org</a>&gt; ha scritto:<br></div><div dir="ltr"><div class="
    gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Wed, Jun 08, 2022 at 07:23:15PM +0200, Alessandro Barbieri wrote:<br>
    &gt; I&#39;m interested in an eclass that doesn&#39;t bundle everything together. Also<br>
    &gt; I&#39;m interested in anyone that can share the package maintainership (in guru<br>
    &gt; first).<br>
    &gt; <br>
    &gt; I&#39;ve already tried 3 approaches:<br>
    ...<br>

    Since you know this yarn/NPM ecosystem well, could you evaluate two<br>
    other ideas?<br>
    4) Solutions like EGO_SUM<br></blockquote><div> </div><div>My second approach was like that<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
    5) EGO_SUM successor of 2nd-level-Metadata-Manifest that I described in the recent EGO_SUM thread.<br></blockquote><div>I havent&#39; well understood it. In practice would be the same as  2/4 with the same issues.</div><div>Let me remind you that unlike
    go/rust, nodejs packages aren&#39;t compiled and if you bundle everything you&#39;ll end up with a massive amount of files installed.<br></div><div dir="auto">Please don&#39;t consider node any different from python/ruby/perl/php.</div></div></div>
    </div></div>

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)