Issues with OpenStack Instance Creation and Nested Virtualization on ARM Host with QEMU

Q&A
,
1

I am trying to deploy OpenStack on an ARM-based Ampere server (host) and encountering issues related to nested virtualization and instance creation. Below are the details of my setup and the problems observed:
Setup Details:
Host Environment:
Architecture: ARM (Ampere server)
OS: Ubuntu 22.04
Nested Virtualization: Not supported (no virt feature in /proc/cpuinfo)
/dev/kvm exists and is functional on the host.

Guest Environment:
    QEMU-based VM running on the ARM host.
    /dev/kvm is not available in the guest due to lack of nested virtualization support.
    Flags used in QEMU command for the VM to enable virtualization:

qemu-system-aarch64 -enable-kvm -machine virt,gic-version=3,ras=on,kernel-irqchip=on,accel=kvm,virtualization=on ..........

This results in the error:
qemu-system-aarch64: mach-virt: KVM does not support providing Virtualization extensions to the guest CPU.

OpenStack Setup:

Installed OpenStack on the QEMU-based VM using DevStack.
DevStack defaulted to qemu as the hypervisor in the guest due to the absence of /dev/kvm.

Errors Observed:

During OpenStack instance creation:
    Error:

Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance

Logs from Nova Compute service:

Error:

libvirt.libvirtError: unsupported configuration: CPU mode 'host-passthrough' for aarch64 qemu domain on aarch64 host is not supported by hypervisor

QEMU error when virtualization=on is added:

Error:

    qemu-system-aarch64: mach-virt: KVM does not support providing Virtualization extensions to the guest CPU

Questions:

On Nested Virtualization:
    Is it possible to enable nested virtualization on ARM hosts?
    If not, are there any workarounds to expose /dev/kvm to the guest VM to improve OpenStack performance?

QEMU Command Configuration:
    Is the flag virtualization=on a valid approach to expose virtualization extensions to the guest?
    What are the recommended QEMU parameters for an ARM host that lacks nested virtualization support?

OpenStack and DevStack:
    How can I configure OpenStack to use KVM (instead of QEMU) in the guest VM when /dev/kvm is unavailable?
    Are there alternative configurations or optimizations for running OpenStack on ARM-based hosts without nested virtualization?
2

Just answering the first question. Altra and Altra Max processors are Arm ISA 8.2 and cannot support nested virtualisation. AmpereOne at Arm ISA 8.6 does support it.
My OpenStack knowledge is limited so I’ll leave it to others for the rest.

1 Like
3

Like @vikingforties wrote: Ampere Altra lacks nested virtualization support.

If you want to play with nested then they will be fully emulated (TCG mode).

FEAT_NV2 needed for nested virtualization is an optional feature in v8.4 ISA and for years no one bothered with implementing it. In meantime devops world moved from using VM instances to containers and a need for NV became even less needed.

Then cloud providers started offering Arm instances and it made a need for NV even smaller.

1 Like