[Qemu-devel] Segfault with coalesced mmio and boot CPU removal

[Qemu-devel] Segfault with coalesced mmio and boot CPU removal

[Bharata B Rao]

Hi,

Coalesced mmio buffer is part of vCPU 0's kvm_run mmap'ed area
and with the introduction of CPU hotplug, vCPU 0 can be removed on
PowerPC leading to the below seen segfault in QEMU.

0x00000000100a1d34 in kvm_flush_coalesced_mmio_buffer ()
    at qemu/kvm-all.c:1828
1828	        while (ring->first != ring->last) {

#0  0x00000000100a1d34 in kvm_flush_coalesced_mmio_buffer ()
    at qemu/kvm-all.c:1828
#1  0x00000000100160e0 in qemu_flush_coalesced_mmio_buffer ()
    at qemu/exec.c:1206
#2  0x00000000100a961c in memory_region_transaction_begin ()
    at qemu/memory.c:904
#3  0x00000000100add90 in memory_region_set_enabled (mr=0x3fff96420310, enabled=false)
    at qemu/memory.c:1974
#4  0x00000000104577b0 in pci_default_write_config (d=0x3fff96420010, addr=4, val_in=258, 
    l=2) at hw/pci/pci.c:1340
#5  0x0000000010465d0c in pci_host_config_write_common (pci_dev=0x3fff96420010, addr=4, 
    limit=4096, val=258, len=2) at hw/pci/pci_host.c:66
#6  0x0000000010170500 in finish_write_pci_config (spapr=0x10fc6290, 
    buid=576460752840294400, addr=4, size=2, val=258, rets=20547656)
    at qemu/hw/ppc/spapr_pci.c:199
#7  0x0000000010170620 in rtas_ibm_write_pci_config (cpu=0x3fffac590010, spapr=0x10fc6290, 
    token=8215, nargs=5, args=20547636, nret=1, rets=20547656)
    at qemu/hw/ppc/spapr_pci.c:223
#8  0x000000001016e540 in spapr_rtas_call (cpu=0x3fffac590010, spapr=0x10fc6290, token=8215, 
    nargs=5, args=20547636, nret=1, rets=20547656)
    at qemu/hw/ppc/spapr_rtas.c:675
#9  0x0000000010167dfc in h_rtas (cpu=0x3fffac590010, spapr=0x10fc6290, opcode=61440, 
    args=0x3fffac570030) at qemu/hw/ppc/spapr_hcall.c:665
#10 0x00000000101693ec in spapr_hypercall (cpu=0x3fffac590010, opcode=61440, 
    args=0x3fffac570030) at qemu/hw/ppc/spapr_hcall.c:1094
#11 0x000000001026c82c in kvm_arch_handle_exit (cs=0x3fffac590010, run=0x3fffac570000)
    at qemu/target-ppc/kvm.c:1731
#12 0x00000000100a246c in kvm_cpu_exec (cpu=0x3fffac590010)
    at qemu/kvm-all.c:2005
#13 0x000000001007d8d4 in qemu_kvm_cpu_thread_fn (arg=0x3fffac590010) 

This happens because during CPU removal, though we park the kvm_fd
corresponding to the removed vCPU thread, we unmap the kvm_run (and
hence coalesced mmio ring).

What would be the best way to fix this ? Is disassociating coalesced_mmio_ring
from vCPU 0's kvm_run the correct solution ?

Regards,
Bharata.

Re: [Qemu-devel] Segfault with coalesced mmio and boot CPU removal

[Bharata B Rao]

On Wed, Jul 27, 2016 at 08:22:51AM +0530, Bharata B Rao wrote:
> Hi,
> 
> Coalesced mmio buffer is part of vCPU 0's kvm_run mmap'ed area
> and with the introduction of CPU hotplug, vCPU 0 can be removed on
> PowerPC leading to the below seen segfault in QEMU.
> 
> 
> This happens because during CPU removal, though we park the kvm_fd
> corresponding to the removed vCPU thread, we unmap the kvm_run (and
> hence coalesced mmio ring).
> 
<snip>
> What would be the best way to fix this ? Is disassociating coalesced_mmio_ring
> from vCPU 0's kvm_run the correct solution ?

May be PowerPC too should do what x86 does like below for now ?

commit 73360e27850b213327011f7e22e03865b8c0dd5b
Author: Igor Mammedov <imammedo@redhat.com>
Date:   Mon Jul 18 10:31:22 2016 +0200

    pc: Forbid BSP removal
    
    Boot CPU is assumed to always present in QEMU code, so
    untile that assumptions are gone, deny removal request,
    In another words QEMU won't support BSP hot-unplug.

Re: [Qemu-devel] Segfault with coalesced mmio and boot CPU removal

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1527 bytes --]

On Wed, Jul 27, 2016 at 10:01:41AM +0530, Bharata B Rao wrote:
> On Wed, Jul 27, 2016 at 08:22:51AM +0530, Bharata B Rao wrote:
> > Hi,
> > 
> > Coalesced mmio buffer is part of vCPU 0's kvm_run mmap'ed area
> > and with the introduction of CPU hotplug, vCPU 0 can be removed on
> > PowerPC leading to the below seen segfault in QEMU.
> > 
> > 
> > This happens because during CPU removal, though we park the kvm_fd
> > corresponding to the removed vCPU thread, we unmap the kvm_run (and
> > hence coalesced mmio ring).
> > 
> <snip>
> > What would be the best way to fix this ? Is disassociating coalesced_mmio_ring
> > from vCPU 0's kvm_run the correct solution ?
> 
> May be PowerPC too should do what x86 does like below for now ?

I think that's what we need to do short term.  In the 2.8 timeframe,
separating the mmio ring out from the vcpu state sounds like a good
idea, but I don't really know how complicated that will be.

> 
> commit 73360e27850b213327011f7e22e03865b8c0dd5b
> Author: Igor Mammedov <imammedo@redhat.com>
> Date:   Mon Jul 18 10:31:22 2016 +0200
> 
>     pc: Forbid BSP removal
>     
>     Boot CPU is assumed to always present in QEMU code, so
>     untile that assumptions are gone, deny removal request,
>     In another words QEMU won't support BSP hot-unplug.
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]