• Re: Finalizing 'inhibit-automatic-native-compilation'

    From Sean Whitton@21:1/5 to Eli Zaretskii on Thu Feb 2 06:50:01 2023
    Hello,

    [quoting whole message for the benefit of debian-emacsen]

    On Fri 27 Jan 2023 at 02:57PM +02, Eli Zaretskii wrote:

    The variable 'inhibit-automatic-native-compilation' was introduced in
    last October, as result of various discussions on this list regarding
    the need to disable async native-compilation in some situations.
    Since its introduction was met with some opposition, in particular
    from Andrea, the final decision about whether this variable should
    stay in Emacs was deferred, with the purpose of collecting more data
    points and user experience.

    With the pretest of Emacs 29 just around the corner, I think now is
    the time to make that final decision. With that in mind, I will first summarize the changes which this variable introduced into Emacs, and
    then ask for opinions regarding some of its aspects.

    This variable was introduced (under the name
    'inhibit-native-compilation') in commit 5fec9182db. In that commit:

    . comp-trampoline-compile was changed to avoid writing the
    trampolines to the eln-cache if this variable is non-nil (instead,
    it writes the trampolines to a temporary-file directory, and
    attempts to delete them after that, which on Posix platforms will
    cause their deletion when Emacs which produced them exits, and on
    Windows currently fails).
    . In normal-top-level, we set this variable if the environment
    variable EMACS_INHIBIT_AUTOMATIC_NATIVE_COMPILATION is set.
    . In several places this variable either replaces
    native-comp-deferred-compilation or has the same effect as the
    latter (modulo the opposite meaning of nil/t value), therefore
    disabling async compilation of *.el files that Emacs loads for
    which there are no corresponding *.eln files.

    Here are the questions I think we want to be answered to finalize the implementation and effects of 'inhibit-automatic-native-compilation':

    . Do people actually use 'inhibit-automatic-native-compilation' or
    the corresponding environment variable? If so, for what reasons,
    and why tweaking 'native-comp-eln-load-path' to direct the *.eln
    files to some other place, including the temporary-file directory,
    was not enough?

    . What do we want to do about compiling trampolines when
    native-compilation is disabled?

    Currently, 'inhibit-automatic-native-compilation' doesn't really
    disable compilation of trampolines, it just causes them to be
    written to a temporary location, and hopefully deleted when the
    session ends. This means that, for example, if the user has a
    broken installation of GCC and Binutils, loading Lisp code that
    uses advices will signal errors when Emacs compiles the
    trampolines (because the compilation fails).
    The alternative is to disable compilation of trampolines, but that
    has a downside that advices for primitives will not have effect.
    It is not clear to me which alternative is better, as they both
    have failure modes. Note that the build process was augmented so
    you can say, after building Emacs as usual

    make trampolines

    and have all the trampolines for the built-in functions
    (a.k.a. "primitives") compiled and written to the build tree, from
    where they will be installed by "make install", thus minimizing
    potential problems with the need to build trampolines when running
    the installed Emacs.

    If we leave the current build-trampolines-then-delete-them
    machinery intact, is it a problem to have those trampolines not
    deleted on MS-Windows? They will then be left in the temporary
    directory, and are supposed to be removed by system cleanup
    processes, or maybe remain there forever. Or do we have to add a
    mechanism for deleting them at exit?

    . Do we want to keep the EMACS_INHIBIT_AUTOMATIC_NATIVE_COMPILATION
    environment variable?

    I dislike having environment variables that alter Emacs behavior,
    because environment variables are inherited by sub-processes. So
    having this variable set runs the risk of affecting all the
    sub-processes, something that could be unexpected and is not easy
    to prevent. We had similar problems with EMACSLOADPATH, for
    example, which is especially painful when building another version
    of Emacs from a shell buffer inside Emacs. This causes some
    hard-to-debug problems.
    So if this environment variable is largely unused, maybe we should
    remove it, even if we keep 'inhibit-automatic-native-compilation'.

    Debian is now relying on this environment variable. It is going into
    our next stable release, which is already partially frozen, and it has
    already propagated to many (probably most) of our derivatives.
    We took the decision to start using the variable after Lars said it was
    going to stick around and, specifically, that it was therefore okay to
    backport it to Emacs 28.

    Over in our own channels, we had actually been planning to add our own
    version of this environment variable, before it got added upstream.
    In other words, we independently came to the conclusion that a mechanism
    of this nature was what we needed; as Stefan says, one reason is that we
    need to handle invocations of Emacs deep within third party Makefiles.

    In the shorter term, were it to be removed, we'd probably patch it back
    in with a DEBIAN_something name, rather than break all our packages
    again, or, rather than delaying uploading Emacs 29 to Debian.
    But in the longer term, I'm sure that we would be open to alternatives,
    if you really do want to remove the variable.

    (Just fyi, I am now co-maintaining Emacs in Debian alongside Rob.)

    --
    Sean Whitton

    --=-=-Content-Type: application/pgp-signature; name="signature.asc"

    -----BEGIN PGP SIGNATURE-----

    iQJNBAEBCgA3FiEEm5FwB64DDjbk/CSLaVt65L8GYkAFAmPbTOAZHHNwd2hpdHRv bkBzcHdoaXR0b24ubmFtZQAKCRBpW3rkvwZiQF8MD/9aUWxb2G9VSIsiSKM3QUgj 1as4K8qFW3LnLz47PihYtTy72KPA2M5gqMMAPkTJix7xxRKiUJppYRz2ZeYXzo0t 5cbWGxBRNzrO2dov4bCUejjbJSIQ0/JP4pbCEJ8/yPXCxKrlpkyreS+q/P41Yb+v BGstHRIX18y6oUrrk1FrMh8oKjRQ2yleCt6MbDHoL7vVA+V568KfHCehAiWlrCyd GteHcW4ifwrZtKoVabnp8rR9B1VaBsbcJ2MdzyPLcDFh97pTvZldZrWXTxKi70ng rh7vmhQKZZguJElzAepfpL7lxv2VA/w/f85RVnFPo0odMmuIc1fwVhTViVRIw1Ig +IPTzMKVjo73H7ctUghfsyaVvd5wWFwTshxd4YjRFxHTBdFyjOHLNaStxDDC2LMH QIeAoDVzFgj8hKa7rmHiQoSQMv55JyCV6ppgBlUIDYfyrWWL8OuaKdZEw9c/QFkV XitE1n5oJeSkFA37h808iMnIZiV7PUsOU0okBhWh9AUwJL4Xd2XnMhFS8yV3KPaP xcSmDr2Y8g0Vriyu3pclPaBw9OaFRnEWbDTDOD94gbHwnKmzfqPqmOaq6kKl8YOe lbEgfZzAW4IXUUTfwNVAphByzGfJW7K1ZBsBUvukoOKMq9iXPckV6L93F+geSS1D EWK01sk2kPMZOvOb3RFL/Q==rvv6
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Us
  • From Eli Zaretskii@21:1/5 to All on Thu Feb 2 09:20:02 2023
    From: Sean Whitton <spwhitton@spwhitton.name>
    Cc: emacs-devel@gnu.org, Andrea Corallo <akrl@sdf.org>, Lars Ingebrigtsen
    <larsi@gnus.org>, Stefan Monnier <monnier@iro.umontreal.ca>, Rob
    Browning <rlb@defaultvalue.org>, debian-emacsen@lists.debian.org
    Date: Wed, 01 Feb 2023 22:40:48 -0700

    Debian is now relying on this environment variable. It is going into
    our next stable release, which is already partially frozen, and it has already propagated to many (probably most) of our derivatives.
    We took the decision to start using the variable after Lars said it was
    going to stick around and, specifically, that it was therefore okay to backport it to Emacs 28.

    Over in our own channels, we had actually been planning to add our own version of this environment variable, before it got added upstream.
    In other words, we independently came to the conclusion that a mechanism
    of this nature was what we needed; as Stefan says, one reason is that we
    need to handle invocations of Emacs deep within third party Makefiles.

    In the shorter term, were it to be removed, we'd probably patch it back
    in with a DEBIAN_something name, rather than break all our packages
    again, or, rather than delaying uploading Emacs 29 to Debian.
    But in the longer term, I'm sure that we would be open to alternatives,
    if you really do want to remove the variable.

    It is fine by me if Debian decides to patch Emacs to solve whatever
    problems you have in your work on the distribution. That's the
    prerogative of downstream packages, and that's why we have Free
    Software to begin with.

    What I'm interested in is hearing the _reasons_ why you decided to go
    with the environment variable, because those reasons might be
    important in other uses. Was it just the convenience, or was it
    something else?

    (Just fyi, I am now co-maintaining Emacs in Debian alongside Rob.)

    Congrats.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Sean Whitton@21:1/5 to Eli Zaretskii on Thu Feb 2 17:30:01 2023
    Hello,

    On Thu 02 Feb 2023 at 10:02AM +02, Eli Zaretskii wrote:

    It is fine by me if Debian decides to patch Emacs to solve whatever
    problems you have in your work on the distribution. That's the
    prerogative of downstream packages, and that's why we have Free
    Software to begin with.

    Of course. But we don't want to be maintaining a fork of Emacs.
    We are all on the same side when it comes to wanting to preserve
    manpower for more important tasks, right?

    What I'm interested in is hearing the _reasons_ why you decided to go
    with the environment variable, because those reasons might be
    important in other uses. Was it just the convenience, or was it
    something else?

    Without excluding the possibility of an improved solution, we did not
    think that we needed an env var out of convenience, or in the manner of
    a temporary fix.

    I am not able to answer myself, right now, whether what has been
    proposed in this thread about redirecting trampoline writes would be
    sufficient for us. Hopefully someone else can chime in.

    One thing that I can say for sure is that we need Emacs not to crash
    when HOME is non-existent or otherwise not writeable.
    I am not completely sure whether that is the whole of the issue, but it
    may well be.

    (Just fyi, I am now co-maintaining Emacs in Debian alongside Rob.)

    Congrats.

    Dubious :P I just wanted to ask, please CC both me and Rob in the
    future, else I might have missed this thread.

    --
    Sean Whitton

    --=-=-Content-Type: application/pgp-signature; name="signature.asc"

    -----BEGIN PGP SIGNATURE-----

    iQJNBAEBCgA3FiEEm5FwB64DDjbk/CSLaVt65L8GYkAFAmPb5LcZHHNwd2hpdHRv bkBzcHdoaXR0b24ubmFtZQAKCRBpW3rkvwZiQBx9EACqUaOs3DYqcvhSaLxyp98o S7pqSpmbpYsWCPYpga9J153psQyJeJ8rE7rk/xHLDSx0K7Tb9l/zpJXyIkDzQ4je DsywgoHJRB5dVWtu5Dt9gdNmyHZEO0OvXN32EeSFlcX5QuxaH126boqu3cgsdLxB 1wk1twtiVls7B0vPVdvTeCFTYwstJl+2IsauBbOeu6RY80tf6MBvAG1t7JqnQpE0 E9/xutf0ZNSh05DS03+TooNdlsuAwcKmPkK4Pu7D7tepBIRUS2KUNrTFU1jZJLSf Djex+4QLArYyYz1GLqm3z0HsQuC2EFQytIZ8IId9BmSi7GPcEdVp3uMmCj8/uk22 epuqu179vWJ6rb6LfUmJEiIxeCBOiGfEYeSCapidLVwthHDHVikTUY6QUGMywsFO jInwCjpEpvumFQJ+DmBZat7lcvuk4HGdt6DyGyvABDSsfBxevIb+4TXp8HhJz6D9 /QdJdfo/rBkxhg57f5BzjtVn3Mz3P61umTcr6GYsyKerb+m6c9D44zcWASvvOkHP ifLolZ30qQ4MI8OcFxPhz/gmMKfr9doFmvV79jQSi40MkpVrdxmsE+YoGDlsPWUl 3dCrq2Yjxo+ZYFW0xay2tc9VA2pQsXOVp4J3uUDeG9Oye7UeZhqKsKVeqFcFDuc3 l3IEwoz5iiTfrAQ9PucMPQ==t023
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Us
  • From Eli Zaretskii@21:1/5 to All on Thu Feb 2 18:40:01 2023
    From: Sean Whitton <spwhitton@spwhitton.name>
    Cc: emacs-devel@gnu.org, akrl@sdf.org, larsi@gnus.org,
    monnier@iro.umontreal.ca, rlb@defaultvalue.org,
    debian-emacsen@lists.debian.org
    Date: Thu, 02 Feb 2023 09:28:39 -0700

    I am not able to answer myself, right now, whether what has been
    proposed in this thread about redirecting trampoline writes would be sufficient for us. Hopefully someone else can chime in.

    Let's hope. The changes I'm talking about were largely motivated by
    Debian's reports, so I'd very much prefer not to overturn some of
    those changes without understanding well enough what it does to Debian
    and in particular making sure that Debian can still do what they need
    even if some of those changes are reverted.

    One thing that I can say for sure is that we need Emacs not to crash
    when HOME is non-existent or otherwise not writeable.
    I am not completely sure whether that is the whole of the issue, but it
    may well be.

    I don't think it crashes in that case. It definitely shouldn't. So
    if you have a recipe where it does, please report it with the details.

    However, that isn't directly related to the issue at hand, since I
    don't expect anyone to point HOME to an invalid place as a means for
    disabling native-compilation: we have much more direct means for that.

    (Just fyi, I am now co-maintaining Emacs in Debian alongside Rob.)

    Congrats.

    Dubious :P I just wanted to ask, please CC both me and Rob in the
    future, else I might have missed this thread.

    Will do.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Sean Whitton@21:1/5 to Eli Zaretskii on Thu Feb 9 22:20:01 2023
    Hello,

    On Thu 02 Feb 2023 at 07:21PM +02, Eli Zaretskii wrote:

    From: Sean Whitton <spwhitton@spwhitton.name>
    Cc: emacs-devel@gnu.org, akrl@sdf.org, larsi@gnus.org,
    monnier@iro.umontreal.ca, rlb@defaultvalue.org,
    debian-emacsen@lists.debian.org
    Date: Thu, 02 Feb 2023 09:28:39 -0700

    One thing that I can say for sure is that we need Emacs not to crash
    when HOME is non-existent or otherwise not writeable.
    I am not completely sure whether that is the whole of the issue, but it
    may well be.

    I don't think it crashes in that case. It definitely shouldn't. So
    if you have a recipe where it does, please report it with the details.

    https://bugs.debian.org/1021842

    --
    Sean Whitton

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)