qemu crash 100% CPU with Ubuntu10.04 guest (solved)

qemu crash 100% CPU with Ubuntu10.04 guest (solved)

[Ben Smith]

Hi All,

I'm cross-posting this from Reddit qemu_kvm, in case it helps in some
way. I know my setup is ancient and unique; let me know if you would
like more info.

Symptoms:
1. Ubuntu10.04 32-bit guest locks up randomly between 0 and 30 days.
2. The console shows a CPU trace dump, nothing else logged on the guest or host.
3. Host system (Ubuntu20.04) 100% CPU for qemu process.

Solution:
When using virt-install, always use the "--os-variant" parameter!
e.g. --os-variant ubuntu10.04

From the man page "--os-variant... Optimize the guest configuration
for a specific operating system".
In this case, "optimize" apparently means "stop the crashing".

I was deliberately avoiding the option because the VM was already
performing much better than expected and I didn't want to complicate
the configuration.

This was very, very painful to troubleshoot; Involving spinning up 60
VMs simultaneously, waiting for a failure, changing one parameter,
repeat. :(

Re: qemu crash 100% CPU with Ubuntu10.04 guest (solved)

[Kashyap Chamarthy]

On Thu, Feb 17, 2022 at 12:07:15PM +1100, Ben Smith wrote:
> Hi All,
Hi,

> I'm cross-posting this from Reddit qemu_kvm, in case it helps in some
> way. I know my setup is ancient and unique; let me know if you would
> like more info.
> 
> Symptoms:
> 1. Ubuntu10.04 32-bit guest locks up randomly between 0 and 30 days.
> 2. The console shows a CPU trace dump, nothing else logged on the guest or host.
> 3. Host system (Ubuntu20.04) 100% CPU for qemu process.
> 
> Solution:
> When using virt-install, always use the "--os-variant" parameter!
> e.g. --os-variant ubuntu10.04
> 
> From the man page "--os-variant... Optimize the guest configuration
> for a specific operating system".
> In this case, "optimize" apparently means "stop the crashing".

The "--os-variant" will use virtio devices where applicable, recommended
machine type, guest resources (e.g. CPU, memory, disk size) and other
things that'll improve performance.

> I was deliberately avoiding the option because the VM was already
> performing much better than expected and I didn't want to complicate
> the configuration.

Using it is always recommended when using `virt-install`.  The command
`osinfo-query os` will list all the OSes that you can use with
"--os-variant".  Note: even if you don't find the latest version of $OS
in `osinfo-query`, just using the most recent version still suffices.


> This was very, very painful to troubleshoot; Involving spinning up 60
> VMs simultaneously, waiting for a failure, changing one parameter,
> repeat. :(

Yikes!  Kudos for having the high threshold for frustration.

I think providing a clear reproducer can still be useful.  E.g. your
full guest QEMU command-line and your QEMU version.  (The
libvirt-generated QEMu log contains the version info.)


-- 
/kashyap

Re: qemu crash 100% CPU with Ubuntu10.04 guest (solved)

[Andrea Bolognani]

On Thu, Feb 17, 2022 at 12:07:15PM +1100, Ben Smith wrote:
> Hi All,
>
> I'm cross-posting this from Reddit qemu_kvm, in case it helps in some
> way. I know my setup is ancient and unique; let me know if you would
> like more info.
>
> Symptoms:
> 1. Ubuntu10.04 32-bit guest locks up randomly between 0 and 30 days.
> 2. The console shows a CPU trace dump, nothing else logged on the guest or host.
> 3. Host system (Ubuntu20.04) 100% CPU for qemu process.
>
> Solution:
> When using virt-install, always use the "--os-variant" parameter!
> e.g. --os-variant ubuntu10.04

FWIW, the --os-variant / --osinfo argument is going to be mandatory
starting with the upcoming virt-manager release.

https://listman.redhat.com/archives/virt-tools-list/2022-February/msg00021.html

-- 
Andrea Bolognani / Red Hat / Virtualization