• Bug#1040062: dpkg-dev: Please drop pie-{compile,link}.spec

    From Guillem Jover@1:229/2 to Adrian Bunk on Tue Oct 31 11:00:01 2023
    XPost: linux.debian.bugs.dist, linux.debian.ports.alpha, linux.debian.ports.ia64
    From: guillem@debian.org

    Hi!

    On Tue, 2023-07-04 at 13:12:48 +0300, Adrian Bunk wrote:
    On Tue, Jul 04, 2023 at 09:23:43AM +0200, Guillem Jover wrote:
    On Sun, 2023-07-02 at 00:02:46 +0300, Adrian Bunk wrote:
    There are some problems with this:

    1. PIE should either be default or not be used

    I suspect x32 might be able to default to PIE without problems
    (there might just not be enough interest left to change the default).

    On alpha the toolchain has already become quite brittle
    with frequent issues like (reproducible) linker segfaults,
    any variations that affects the toolchain are bad.

    It is for the port maintainers to decide whether or not PIE
    is considered stable on a port, and accordingly either make
    it default (which also avoids the other issues below) or not.

    It is clear that a non-PIE architecture would no longer be
    considered suitable as release architecture.

    In general the way this is handled in dpkg, is that if the flags
    supposedly work on that arch they are allowed, but if they are not supported or are broken then they are masked.

    I recalled this report before the 1.22.1 release, but didn't feel it appropriate to push a last minute change for it, before giving some
    advance notice, and because I was also expecting some word from the
    relevant arch porters.

    I guess I could do it the other way, and given this is apparently
    causing issues as reported by Adrian, and as seen recently from
    the referenced bug report which might require patching a specific
    package to disable PIE there, I'm inclined to completely mask PIE
    in dpkg-buildflags for alpha and ia64 in around a couple of weeks,
    if I don't hear anything.

    Please drop pie-{compile,link}.spec, on the architectures
    where it has any effect it is doing more harm than good.

    For example hppa has pie masked for build flags. If the porters for
    alpha and/or ia64 consider that they should also get pie masked for
    those arches, I'm fine doing the changes. Although that means on those ports it will not be possible to enable pie at all, even if asked for explicitly, as in «hardening=+pie».

    Semi-PIE non-release architectures shouldn't exist, PIE on an·
    architecture should be a binary option decided by the porters
    for this architecture.

    If there are any porters left on an architecture and they consider PIE· stable, they can always ask the gcc maintainer to change the default.[1]

    ...
    The lowest effort fix would be to patch debian/rules of affected
    packages to disable hardening=+pie on affected architectures,
    but that would still be spending time on working around a problem
    that shouldn't exist.

    Yeah, that does not look like the right thing to be spending time on.
    ...

    As long as we have at least one semi-PIE architecture, this is the only· realistic option for porters. If the porters for alpha and/or ia64 want·
    to have pie masked, such changes will still be required for x32.

    If PIE (via specs files) appears to work on x32, and changing the
    defaults in gcc is too much to bother, I think leaving it as is looks
    fine by me. But if this is causing problems as well and the x32 porters
    (if there's any remaining :), want to mask it alongside the other ports,
    let me know and I can also flip the switch for that one.

    Thanks,
    Guillem

    --- SoupGate-Win32 v1.05
    * Origin: you cannot sedate... all the things you hate (1:229/2)
  • From Adrian Bunk@1:229/2 to Guillem Jover on Sun Nov 26 14:10:02 2023
    XPost: linux.debian.bugs.dist, linux.debian.ports.alpha, linux.debian.ports.ia64
    From: bunk@debian.org

    On Fri, Nov 24, 2023 at 01:53:04AM +0100, Guillem Jover wrote:
    Hi!

    Hi Guillem!

    Apologies for not replying to these emails earlier.

    On Tue, 2023-10-31 at 10:52:40 +0100, Guillem Jover wrote:
    ...
    If PIE (via specs files) appears to work on x32, and changing the
    defaults in gcc is too much to bother, I think leaving it as is looks
    fine by me. But if this is causing problems as well and the x32 porters
    (if there's any remaining :), want to mask it alongside the other ports, let me know and I can also flip the switch for that one.

    If the porters would also like to see x32 masked, let me know and I
    can also include it, otherwise I'll leave it as it is now.

    AFAIK for all ports architectures except Hurd and hppa the other Adrian
    is a porter, so should be able to ACK it.

    As I already wrote in [1] there is architecture other than alpha/ia64/x32
    where the pie specs did actually enable PIE.

    On m68k and sh4 the specs might still be passed and cause FTBFS in
    packages like gluegen2 that pass LDFLAGS to ld, but there is no
    PIE enabling effect.

    Thanks,
    Guillem

    cu
    Adrian

    [1] https://lists.debian.org/debian-alpha/2023/10/msg00002.html

    --- SoupGate-Win32 v1.05
    * Origin: you cannot sedate... all the things you hate (1:229/2)
  • From Adrian Bunk@1:229/2 to All on Sat Jul 1 23:10:01 2023
    XPost: linux.debian.bugs.dist, linux.debian.ports.alpha, linux.debian.ports.ia64
    From: bunk@debian.org

    Package: dpkg-dev
    Version: 1.21.22
    Severity: normal
    X-Debbugs-Cc: debian-alpha@lists.debian.org, debian-ia64@lists.debian.org

    [ Cc set to debian-alpha@ and debian-ia64@ since they are most affected ]

    Since stretch all release architectures are using PIE by default,
    and all future release architectures (including riscv64) will also
    use PIE by default.

    Many packages in Debian are building with hardening=+all, and the
    effect regarding PIE is "enable PIE for this package on some obscure
    ports architectures that don't have it enabled by default" which is
    unlikely to be what the maintainer intended.

    There are also some pre-stretch "hardening=+pie" left
    in some packages.

    There are some problems with this:


    1. PIE should either be default or not be used

    I suspect x32 might be able to default to PIE without problems
    (there might just not be enough interest left to change the default).

    On alpha the toolchain has already become quite brittle
    with frequent issues like (reproducible) linker segfaults,
    any variations that affects the toolchain are bad.

    It is for the port maintainers to decide whether or not PIE
    is considered stable on a port, and accordingly either make
    it default (which also avoids the other issues below) or not.

    It is clear that a non-PIE architecture would no longer be
    considered suitable as release architecture.


    2. It causes weird issues on undersupported architectures

    gluegen2 passes LDFLAGS to ld instead of gcc.

    Several packages have relocation errors only on affected
    architectures.

    ...

    Such issues could be debugged and fixed, but in practice
    trying to handle such issues that happen only with
    pie-{compile,link}.spec creates additional work that frustrates
    the few people keeping these non-release architectures alive.

    The lowest effort fix would be to patch debian/rules of affected
    packages to disable hardening=+pie on affected architectures,
    but that would still be spending time on working around a problem
    that shouldn't exist.


    3. It breaks some cases of static linking

    Linking a package with hardening=+all against a static library
    from a package not using hardening=+all cannot work on the
    affected architectures.

    Static linking is relatively rare, but I remember requesting binNMUs
    for static linking cases to fix FTBFS on release architectures when
    the default changed before stretch.


    Please drop pie-{compile,link}.spec, on the architectures
    where it has any effect it is doing more harm than good.

    Thanks

    --- SoupGate-Win32 v1.05
    * Origin: you cannot sedate... all the things you hate (1:229/2)
  • From Guillem Jover@1:229/2 to Adrian Bunk on Tue Jul 4 09:30:03 2023
    XPost: linux.debian.bugs.dist, linux.debian.ports.alpha, linux.debian.ports.ia64
    From: guillem@debian.org

    Hi!

    On Sun, 2023-07-02 at 00:02:46 +0300, Adrian Bunk wrote:
    Package: dpkg-dev
    Version: 1.21.22
    Severity: normal
    X-Debbugs-Cc: debian-alpha@lists.debian.org, debian-ia64@lists.debian.org

    Since stretch all release architectures are using PIE by default,
    and all future release architectures (including riscv64) will also
    use PIE by default.

    Many packages in Debian are building with hardening=+all, and the
    effect regarding PIE is "enable PIE for this package on some obscure
    ports architectures that don't have it enabled by default" which is
    unlikely to be what the maintainer intended.

    There are also some pre-stretch "hardening=+pie" left
    in some packages.

    Yeah, I've never been very satisfied with our pie handling. :/

    There are some problems with this:

    1. PIE should either be default or not be used

    I suspect x32 might be able to default to PIE without problems
    (there might just not be enough interest left to change the default).

    On alpha the toolchain has already become quite brittle
    with frequent issues like (reproducible) linker segfaults,
    any variations that affects the toolchain are bad.

    It is for the port maintainers to decide whether or not PIE
    is considered stable on a port, and accordingly either make
    it default (which also avoids the other issues below) or not.

    It is clear that a non-PIE architecture would no longer be
    considered suitable as release architecture.

    In general the way this is handled in dpkg, is that if the flags
    supposedly work on that arch they are allowed, but if they are not
    supported or are broken then they are masked.

    2. It causes weird issues on undersupported architectures

    gluegen2 passes LDFLAGS to ld instead of gcc.

    Several packages have relocation errors only on affected
    architectures.

    ...

    Such issues could be debugged and fixed, but in practice
    trying to handle such issues that happen only with
    pie-{compile,link}.spec creates additional work that frustrates
    the few people keeping these non-release architectures alive.

    Regardless of this report, I think this would still be worthwhile,
    as otherwise you cannot for example disable them globally on arches
    where it is a builtin in the compiler (as those will also need the
    spec files.

    The lowest effort fix would be to patch debian/rules of affected
    packages to disable hardening=+pie on affected architectures,
    but that would still be spending time on working around a problem
    that shouldn't exist.

    Yeah, that does not look like the right thing to be spending time on.

    3. It breaks some cases of static linking

    Linking a package with hardening=+all against a static library
    from a package not using hardening=+all cannot work on the
    affected architectures.

    Static linking is relatively rare, but I remember requesting binNMUs
    for static linking cases to fix FTBFS on release architectures when
    the default changed before stretch.

    Hmm, ah or even packages not respecting DEB_BUILD_OPTIONS, right.

    Please drop pie-{compile,link}.spec, on the architectures
    where it has any effect it is doing more harm than good.

    For example hppa has pie masked for build flags. If the porters for
    alpha and/or ia64 consider that they should also get pie masked for
    those arches, I'm fine doing the changes. Although that means on those
    ports it will not be possible to enable pie at all, even if asked for explicitly, as in «hardening=+pie».

    Thanks,
    Guillem

    --- SoupGate-Win32 v1.05
    * Origin: you cannot sedate... all the things you hate (1:229/2)
  • From Adrian Bunk@1:229/2 to Guillem Jover on Tue Jul 4 12:20:01 2023
    XPost: linux.debian.bugs.dist, linux.debian.ports.alpha, linux.debian.ports.ia64
    From: bunk@debian.org

    On Tue, Jul 04, 2023 at 09:23:43AM +0200, Guillem Jover wrote:
    Hi!

    Hi Guillem!

    On Sun, 2023-07-02 at 00:02:46 +0300, Adrian Bunk wrote:
    ...
    There are some problems with this:

    1. PIE should either be default or not be used

    I suspect x32 might be able to default to PIE without problems
    (there might just not be enough interest left to change the default).

    On alpha the toolchain has already become quite brittle
    with frequent issues like (reproducible) linker segfaults,
    any variations that affects the toolchain are bad.

    It is for the port maintainers to decide whether or not PIE
    is considered stable on a port, and accordingly either make
    it default (which also avoids the other issues below) or not.

    It is clear that a non-PIE architecture would no longer be
    considered suitable as release architecture.

    In general the way this is handled in dpkg, is that if the flags
    supposedly work on that arch they are allowed, but if they are not
    supported or are broken then they are masked.

    PIE is unusual in being enabled by default on all release architectures,
    but not being enabled by default on some non-release architectures.

    With some teams like Debian Med putting hardening=+all into every source package, 20% of the source packages in the archive have it already set.

    This makes both sides of the PIE/non-PIE archive split on the affected
    semi-PIE architectures huge.

    2. It causes weird issues on undersupported architectures

    gluegen2 passes LDFLAGS to ld instead of gcc.

    Several packages have relocation errors only on affected
    architectures.

    ...

    Such issues could be debugged and fixed, but in practice
    trying to handle such issues that happen only with
    pie-{compile,link}.spec creates additional work that frustrates
    the few people keeping these non-release architectures alive.

    Regardless of this report, I think this would still be worthwhile,
    as otherwise you cannot for example disable them globally on arches
    where it is a builtin in the compiler (as those will also need the
    spec files.

    Packages like python3.*-nopie could also set -fno-pie/-no-pie in
    CFLAGS/LDFLAGS manually instead of using hardening=-pie, but different
    from hardening=+all disabling PIE is not something that has unexpected
    effects only on some ports architectures.

    ...
    3. It breaks some cases of static linking

    Linking a package with hardening=+all against a static library
    from a package not using hardening=+all cannot work on the
    affected architectures.

    Static linking is relatively rare, but I remember requesting binNMUs
    for static linking cases to fix FTBFS on release architectures when
    the default changed before stretch.

    Hmm, ah or even packages not respecting DEB_BUILD_OPTIONS, right.

    Bugs would fall under point 2.

    Point 3 is about cases that cannot work despite no package doing
    anything wrong.

    Many packages needed fixing (or still have to be fixed) to enable
    PIE because they had set hardening=+all,-pie before the default
    changed in stretch. When all architectures defaulted to non-PIE,
    hardening=+all did also cause PIE-related FTBFS on release architectures.

    Semi-PIE architectures still have such problems between packages on
    different sides of the PIE/non-PIE archive split.

    Please drop pie-{compile,link}.spec, on the architectures
    where it has any effect it is doing more harm than good.

    For example hppa has pie masked for build flags. If the porters for
    alpha and/or ia64 consider that they should also get pie masked for
    those arches, I'm fine doing the changes. Although that means on those
    ports it will not be possible to enable pie at all, even if asked for explicitly, as in «hardening=+pie».

    Semi-PIE non-release architectures shouldn't exist, PIE on an
    architecture should be a binary option decided by the porters
    for this architecture.

    If there are any porters left on an architecture and they consider PIE
    stable, they can always ask the gcc maintainer to change the default.[1]

    ...
    The lowest effort fix would be to patch debian/rules of affected
    packages to disable hardening=+pie on affected architectures,
    but that would still be spending time on working around a problem
    that shouldn't exist.

    Yeah, that does not look like the right thing to be spending time on.
    ...

    As long as we have at least one semi-PIE architecture, this is the only realistic option for porters. If the porters for alpha and/or ia64 want
    to have pie masked, such changes will still be required for x32.

    Thanks,
    Guillem

    Thanks
    Adrian

    [1] no transition with rebuilds required, in the rare cases of FTBFS
    when linking with a non-PIE static library this static library
    package could be binNMU'ed when the problem appears

    --- SoupGate-Win32 v1.05
    * Origin: you cannot sedate... all the things you hate (1:229/2)