1. Forum
  2. Usenet
  3. LINUX.DEBIAN.PORTS.ARM

  • kvm on arm64

    From Ross Vandegrift@21:1/5 to All on Fri Aug 11 08:10:01 2023
    (please CC me, as I'm not subscribed)

    Hi folks,

    New to running arm64 stuff on physical arm64 hardware, and I'm unable to start a kvm guest. I'm sure I'm missing something, hoping someone can point me in the right direction.

    I'm running arm64 bookworm (plus vendor junk for orangepi 5 plus). Without kvm, I can boot a trixie arm64 image [1] just fine. But if I enable kvm, qemu spins without booting. It produces no output, not even the UEFI firmware output.

    This works:

    dd if=/dev/zero of=flash1.img bs=1M count=64
    dd if=/dev/zero of=flash0.img bs=1M count=64
    dd if=/usr/share/qemu-efi-aarch64/QEMU_EFI.fd of=flash0.img conv=notrunc qemu-system-aarch64 \
    -nographic \
    -machine virt,gic-version=max \
    -m 512M \
    -cpu max \
    -netdev user,id=vnet \
    -device virtio-net-pci,netdev=vnet \
    -drive file=debian-13-nocloud-arm64-daily-20230809-1467.qcow2,if=none,id=drive0 \
    -device virtio-blk,drive=drive0,bootindex=0 \
    -drive file=flash0.img,format=raw,if=pflash \
    -drive file=flash1.img,format=raw,if=pflash \
    -snapshot


    But it breaks if I add -enable-kvm. Any hints?

    Thanks,
    Ross

    [1] - https://cloud.debian.org/images/cloud/trixie/daily/20230809-1467/debian-13-nocloud-arm64-daily-20230809-1467.qcow2

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Emanuele Rocca@21:1/5 to Ross Vandegrift on Fri Aug 11 11:40:01 2023
    Hi Ross,

    On Thu, Aug 10, 2023 at 10:45:54PM -0700, Ross Vandegrift wrote:
    New to running arm64 stuff on physical arm64 hardware, and I'm unable to start
    a kvm guest. I'm sure I'm missing something, hoping someone can point me in the right direction.
    [...]

    qemu-system-aarch64 \
    -nographic \
    -machine virt,gic-version=max \
    -m 512M \
    -cpu max \
    -netdev user,id=vnet \
    -device virtio-net-pci,netdev=vnet \
    -drive file=debian-13-nocloud-arm64-daily-20230809-1467.qcow2,if=none,id=drive0 \
    -device virtio-blk,drive=drive0,bootindex=0 \
    -drive file=flash0.img,format=raw,if=pflash \
    -drive file=flash1.img,format=raw,if=pflash \
    -snapshot


    But it breaks if I add -enable-kvm. Any hints?

    Nothing wrong with your command, I've tried the exact same thing (with -enable-kvm) and it works fine on my M1.

    - What happens if you try with: -m 1G -smp 2, any differences?
    - What does ls -l /dev/kvm say? Is your user in the kvm group?
    - Anything interesting in dmesg?
    - What if you add -serial file:serial.log ? Anything in serial.log?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ross Vandegrift@21:1/5 to Emanuele Rocca on Sat Aug 12 02:00:01 2023
    On Fri, Aug 11, 2023 at 11:31:09AM +0200, Emanuele Rocca wrote:
    But it breaks if I add -enable-kvm. Any hints?

    Nothing wrong with your command, I've tried the exact same thing (with -enable-kvm) and it works fine on my M1.

    Thanks a bunch for confirming on other hardware, that makes me feel
    better!

    - What happens if you try with: -m 1G -smp 2, any differences?

    No change.

    - What does ls -l /dev/kvm say? Is your user in the kvm group?

    All looks good:
    $ ls -la /dev/kvm
    crw-rw---- 1 root kvm 10, 232 Aug 10 22:30 /dev/kvm
    $ groups
    ross adm tty disk dialout sudo audio video plugdev games users input kvm netdev libvirt systemd-journal libvirt-qemu

    - Anything interesting in dmesg?

    Nothing at all. :(

    - What if you add -serial file:serial.log ? Anything in serial.log?

    No, even if I modify the image to enable early printk. Since I'm not
    seeing the UEFI firmware banner, I think it's failing before the kernel
    starts. Here's where my arm ignorance steps in:

    - Is it possible my hardware is lacking virt support?
    - Could my hardware not support UEFI even with qemu providing firmware?

    Ross

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ross Vandegrift@21:1/5 to Ross Vandegrift on Sat Aug 12 08:00:01 2023
    On Fri, Aug 11, 2023 at 04:34:02PM -0700, Ross Vandegrift wrote:
    No, even if I modify the image to enable early printk. Since I'm not
    seeing the UEFI firmware banner, I think it's failing before the kernel starts. Here's where my arm ignorance steps in:

    - Is it possible my hardware is lacking virt support?

    Not the issue - I extracted the kernel/initrd, and I can boot with kvm by loading linux directly.

    - Could my hardware not support UEFI even with qemu providing firmware?

    So something is going wrong with the UEFI firmware or qemu. I tried upgrading to qemu-efi-aarch64 from sid, but it doesn't help. I looked into backporting a newer qemu, but it'd take more time than I have tonight.

    Ross

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ross Vandegrift@21:1/5 to Ross Vandegrift on Wed Aug 23 21:10:02 2023
    On Fri, Aug 11, 2023 at 10:36:51PM -0700, Ross Vandegrift wrote:
    So something is going wrong with the UEFI firmware or qemu. I tried upgrading
    to qemu-efi-aarch64 from sid, but it doesn't help. I looked into backporting a
    newer qemu, but it'd take more time than I have tonight.

    Closing the loop on this: it's a bug in the crappy vendor kernel.
    Armbian has trixie-based images with mainline 6.5-rc5 kernel and kvm
    boots the UEFI firmware right up.

    Ross

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)