On Fri, May 23, 2025 at 08:56:11PM +0200, Christian Kastner wrote:
I maintain llama.cpp and its underlying tensor library ggml. ggml has
various CPU and GPU backends that are heavily optimized, and include
various forms of dynamic dispatching. My goal for configuring the build
is to have something that maximally exploits CPU features on recent
hardware, whilst supporting everything down to the baseline.
I started with amd64 and found a simple and effective solution: I have a
very recent CPU supporting up to AVX512, and using QEMU+kvm, I can efficiently emulate older CPUs in a VM, and run tests there.
Now I'd like to do the same with ARM, but I wonder how to best approach
this.
The analogous solution would be to either get access to, or purchase a
system with an ARMv9 CPU, and use that to emulate older CPUs. However I
don't know how to best go about that.
I see that Graviton4 is ARMv9, so that would be interesting, but I don't
know if they'll support nested kvm. I also looked into SBCs but the only ARMv9 I found was the Radxa Orion O6, which was announced only months
ago, so I'm not sure that it would be well supported by Debian yet.
Though they seem to want to support EFI which in my eyes is a huge plus.
I would appreciate any ideas or suggestions that might help me achieve
my goal. Most importantly, am I perhaps being too aggressive in looking
for ARMv9? Should I settle for something less recent?
Which older ARM chips are you hoping to emulate?
If you mean doing what qemu+kvm can do on x64 where you can hide CPU
feature flags to pretend to have an older CPU, I am not so sure you
can do that with ARM. It does appear when searching that most 64bit
arm chips do support 32 bit still, so perhaps it would work. I know at
least a few of the high end server chips dropped 32 bit support though,
but apparently it is not a common design choice yet.
That Orion O6 does look pretty neat. It seems they even claim it comes
with Debian although with a custom kernel installed and probably a few
other extra packages added.
--
Len Sorensen
--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)