XPost: linux.debian.bugs.dist
From: guillem@debian.org

Hi!

On Mon, 2024-03-04 at 10:59:36 -0700, Cordell Bloor wrote:

Package: dpkg-dev
Version: 1.22.5
Severity: wishlist
X-Debbugs-Cc: cgmb@slerp.xyz, debian-ai@lists.debian.org

When packaging the AMD ROCm GPU libraries for Debian, we are currently
using CXX=hipcc or CXX=clang++ to build libraries written in HIP as if
they were written in C++.

I guess we should also add HIPCXX (defaulting to <host-triplet>-hipcc
and HIPCXX_FOR_BUILD (defaulting to <build-triplet>-hipcc when cross
compiling, otherwise to hipcc) like with the other toolchains? An
apt-file query seems to indicate thee hipcc package provides no triplet-qualified toolchains? Which means automatic cross-compiling
is going to be painful given our current infrastructure.

If support for this is really missing from the looks of it, we can
always postpone adding the compiler tool variables for now until this
is implemented (we can still add the HIPFLAGS variables though). I'm
CCing the debian-cross list for further insight.

This necessitates filtering out flags that are not supported when
building HIP language code. For example, the rocsparse d/rules include:

export CXX=hipcc
export DEB_BUILD_MAINT_OPTIONS = hardening=+all,-fortify optimize=-lto

# filter incompatible options from affecting device code
CXXFLAGS := $(subst -fstack-protector-strong,-Xarch_host -fstack-protector-strong,$(CXXFLAGS))
CXXFLAGS := $(subst -fcf-protection,-Xarch_host -fcf-protection,$(CXXFLAGS))

In the lines above, we are prepending `-Xarch_host` to prevent certain
flags from being applied to device code (i.e., GPU code) while still
ensuring that they are applied to host code (i.e., CPU code).

However, there is HIP language support in CMake. We should use it! dpkg-buildflags should set HIPFLAGS [1]. The CXXFLAGS make a good
starting place for the HIPFLAGS values, but `-Xarch_host` should be
added to `-fstack-protector-strong` and `-fcf-protection`, like in the example above.

[1]: https://cmake.org/cmake/help/v3.28/envvar/HIPFLAGS.html

It would be helpful if you could verify whether all flags currently
added to CXXFLAGS would work for HIPFLAGS. You can check for all such
instances by searching for either default_flags, @compiler_flags and
CXXFLAGS from:

$ perldoc -m Dpkg::Vendor::Debian

Once we have the complete list, I'm happy to add the handling for
these flags in the code.

Thanks,
Guillem

--- SoupGate-Win32 v1.05
* Origin: you cannot sedate... all the things you hate (1:229/2)

XPost: linux.debian.bugs.dist
From: cgmb@slerp.xyz

Package: dpkg-dev
Version: 1.22.5
Severity: wishlist
X-Debbugs-Cc: cgmb@slerp.xyz, debian-ai@lists.debian.org

Dear Maintainer,

When packaging the AMD ROCm GPU libraries for Debian, we are currently
using CXX=hipcc or CXX=clang++ to build libraries written in HIP as if
they were written in C++.

This necessitates filtering out flags that are not supported when
building HIP language code. For example, the rocsparse d/rules include:

export CXX=hipcc
export DEB_BUILD_MAINT_OPTIONS = hardening=+all,-fortify optimize=-lto

# filter incompatible options from affecting device code
CXXFLAGS := $(subst -fstack-protector-strong,-Xarch_host -fstack-protector-strong,$(CXXFLAGS))
CXXFLAGS := $(subst -fcf-protection,-Xarch_host -fcf-protection,$(CXXFLAGS))

In the lines above, we are prepending `-Xarch_host` to prevent certain
flags from being applied to device code (i.e., GPU code) while still
ensuring that they are applied to host code (i.e., CPU code).

However, there is HIP language support in CMake. We should use it! dpkg-buildflags should set HIPFLAGS [1]. The CXXFLAGS make a good
starting place for the HIPFLAGS values, but `-Xarch_host` should be
added to `-fstack-protector-strong` and `-fcf-protection`, like in the
example above.

Sincerely,
Cory Bloor

[1]: https://cmake.org/cmake/help/v3.28/envvar/HIPFLAGS.html

-- Package-specific info:

-- System Information:
Debian Release: trixie/sid
APT prefers unstable
APT policy: (500, 'unstable'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 6.6.15-amd64 (SMP w/32 CPU threads; PREEMPT)
Locale: LANG=C, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: unable to detect

Versions of packages dpkg-dev depends on:
ii binutils 2.42-3
ii bzip2 1.0.8-5+b2
ii libdpkg-perl 1.22.5
ii make 4.3-4.1
ii patch 2.7.6-7
ii perl 5.38.2-3
ii tar 1.35+dfsg-3
ii xz-utils 5.6.0-0.1

Versions of packages dpkg-dev recommends:
ii build-essential 12.10
ii clang-17 [c-compiler] 1:17.0.6-5
ii fakeroot 1.33-1
ii gcc [c-compiler] 4:13.2.0-7
ii gcc-13 [c-compiler] 13.2.0-16.1
ii gnupg 2.2.40-1.1
ii gpgv 2.2.40-1.1+b1
ii libalgorithm-merge-perl 0.08-5

Versions of packages dpkg-dev suggests:
ii debian-keyring 2023.12.24

-- no debconf information

--- SoupGate-Win32 v1.05
* Origin: you cannot sedate... all the things you hate (1:229/2)

XPost: linux.debian.bugs.dist
From: helmut@subdivi.de

On Thu, Mar 07, 2024 at 04:00:22AM +0100, Guillem Jover wrote:

When packaging the AMD ROCm GPU libraries for Debian, we are currently using CXX=hipcc or CXX=clang++ to build libraries written in HIP as if
they were written in C++.

I guess we should also add HIPCXX (defaulting to <host-triplet>-hipcc
and HIPCXX_FOR_BUILD (defaulting to <build-triplet>-hipcc when cross compiling, otherwise to hipcc) like with the other toolchains? An
apt-file query seems to indicate thee hipcc package provides no triplet-qualified toolchains? Which means automatic cross-compiling
is going to be painful given our current infrastructure.

I've tried to read a bit into their faq and my impression is that HIP
currently is x86_64-only and when it is not, the use of clang (which notoriously refuse cooperating with cross building efforts) makes it practically impossible to do any cross building. In essence, the HIP
ecosystem will opt out of cross building, but then the kind of software
that HIP targets requires beefy hardware where cross building isn't very relevant, right? Just make sure to not require HIP for building HIP
(i.e. do not cause bootstrapping problems).

If support for this is really missing from the looks of it, we can
always postpone adding the compiler tool variables for now until this
is implemented (we can still add the HIPFLAGS variables though). I'm
CCing the debian-cross list for further insight.

I think HIPFLAGS is the right way to go about this.

This necessitates filtering out flags that are not supported when
building HIP language code. For example, the rocsparse d/rules include:

export CXX=hipcc
export DEB_BUILD_MAINT_OPTIONS = hardening=+all,-fortify optimize=-lto

# filter incompatible options from affecting device code
CXXFLAGS := $(subst -fstack-protector-strong,-Xarch_host -fstack-protector-strong,$(CXXFLAGS))
CXXFLAGS := $(subst -fcf-protection,-Xarch_host -fcf-protection,$(CXXFLAGS))

In the lines above, we are prepending `-Xarch_host` to prevent certain flags from being applied to device code (i.e., GPU code) while still ensuring that they are applied to host code (i.e., CPU code).

If dpkg were to provide HIPFLAGS, you could just export
CXXFLAGS:=$(HIPFLAGS).

Generally, reusing CXX in this way is a really bad idea on the upstream
side, but hardly preventable. It is very plausible to eventually have to
build an source package containing both C++ code and HIP code and then
you have no correct way of setting CXX. So from a dpkg point of view,
treating HIP as a new language with new variables makes most sense, but
it also means that source packages using these variables will have to do
the variable renaming themselves forever (and thus retaining the ability
to correctly scope those renames).

Helmut

--- SoupGate-Win32 v1.05
* Origin: you cannot sedate... all the things you hate (1:229/2)