Re: [Qemu-devel] [PATCH] ppc/xics: fix ICP reset path

[Greg Kurz <groug@kaod.org>]

On Mon, 23 Jul 2018 13:36:36 +0200
Cédric Le Goater <clg@kaod.org> wrote:

> On 07/12/2018 12:01 PM, Greg Kurz wrote:
> > Recent cleanup in commit a028dd423ee6 dropped the ICPStateClass::reset
> > handler. It is now up to child ICP classes to call the DeviceClass::reset
> > handler of the parent class, thanks to device_class_set_parent_reset().
> > This is a better object programming pattern,  
> 
> I think that the device_class_set_parent* routines just obfuscate a little
> more the object programming approach of QEMU. 
> 

Really ?

> > but unfortunately it causes QEMU to crash during CPU hotplug:  
> 
> I guess I did not try that :/ Would it be complex to add a spapr unit test 
> for CPU hotplug ? It would be useful.
> 

Actually, there's one in tests/cpu-plug-test.c but it runs with
accel=qtest, and thus it doesn't exercise the KVM ICP code.

I've patched it to use accel=kvm:tcg on pseries: it would have caught
the SEGV indeed. There's another problem though: QEMU sometimes stays 
stuck in kvm_vcpu_ioctl() in 'disk sleep' state, causing the test
to hang until someone kills QEMU with SIGKILL.

Not sure yet what's happening...

> > (qemu) device_add host-spapr-cpu-core,id=core1,core-id=1
> > Segmentation fault (core dumped)
> > 
> > When the hotplug path tries to reset the ICP device, we end up calling:
> > 
> > static void icp_kvm_reset(DeviceState *dev)
> > {
> >     ICPStateClass *icpc = ICP_GET_CLASS(dev);
> > 
> >     icpc->parent_reset(dev);
> > 
> > but icpc->parent_reset is NULL... This happens because icp_kvm_class_init()
> > calls:
> > 
> >     device_class_set_parent_reset(dc, icp_kvm_reset,
> >                                   &icpc->parent_reset);
> > 
> > but dc->reset, ie, DeviceClass::reset for the TYPE_ICP type, is
> > itself NULL.>
> > This patch hence sets DeviceClass::reset for the TYPE_ICP type to
> > point to icp_reset(). It then registers a reset handler that calls
> > DeviceClass::reset. If the ICP subtype has configured its own reset
> > handler with device_class_set_parent_reset(), this ensures it will
> > be called first and it can then call ICPStateClass::parent_reset
> > safely. This fixes the reset path for the TYPE_KVM_ICP type, which
> > is the only subtype that defines its own reset function.  
> 
> So, now, isn't ICP reset called twice on KVM ? 
> 

No. We have one path for cold plugged CPUS:

#0  icp_reset (dev=0x113db100) at /home/greg/Work/qemu/qemu-spapr/hw/intc/xics.c:296
#1  0x0000000010146d0c in icp_kvm_reset (dev=0x113db100) at /home/greg/Work/qemu/qemu-spapr/hw/intc/xics_kvm.c:121
#2  0x00000000101434b0 in icp_reset_handler (dev=0x113db100) at /home/greg/Work/qemu/qemu-spapr/hw/intc/xics.c:310
#3  0x00000000104c3744 in qemu_devices_reset () at /home/greg/Work/qemu/qemu-spapr/hw/core/reset.c:69
#4  0x00000000101d61c8 in spapr_machine_reset () at /home/greg/Work/qemu/qemu-spapr/hw/ppc/spapr.c:1639
#5  0x00000000103de654 in qemu_system_reset (reason=SHUTDOWN_CAUSE_NONE) at /home/greg/Work/qemu/qemu-spapr/vl.c:1645
...

and one path for hot plugged CPUs:

#0  icp_reset (dev=0x11268e40) at /home/greg/Work/qemu/qemu-spapr/hw/intc/xics.c:296
#1  0x0000000010146d0c in icp_kvm_reset (dev=0x11268e40) at /home/greg/Work/qemu/qemu-spapr/hw/intc/xics_kvm.c:121
#2  0x00000000104be300 in device_reset (dev=0x11268e40) at /home/greg/Work/qemu/qemu-spapr/hw/core/qdev.c:1082
#3  0x00000000104bd62c in device_set_realized (obj=0x11268e40, value=true, errp=0x7fffffffcfa8) at /home/greg/Work/qemu/qemu-spapr/hw/core/qdev.c:867
...

> Anyway, we can sort that out if necessary.
> 
> Thanks,
> 
> C.
>  
> > Reported-by: Satheesh Rajendran <sathnaga@linux.vnet.ibm.com>
> > Suggested-by: David Gibson <david@gibson.dropbear.id.au>
> > Fixes: a028dd423ee6dfd091a8c63028240832bf10f671
> > Signed-off-by: Greg Kurz <groug@kaod.org>
> > ---
> >  hw/intc/xics.c |   14 +++++++++++---
> >  1 file changed, 11 insertions(+), 3 deletions(-)
> > 
> > diff --git a/hw/intc/xics.c b/hw/intc/xics.c
> > index b9f1a3c97214..c90c893228dc 100644
> > --- a/hw/intc/xics.c
> > +++ b/hw/intc/xics.c
> > @@ -291,7 +291,7 @@ static const VMStateDescription vmstate_icp_server = {
> >      },
> >  };
> >  
> > -static void icp_reset(void *dev)
> > +static void icp_reset(DeviceState *dev)
> >  {
> >      ICPState *icp = ICP(dev);
> >  
> > @@ -303,6 +303,13 @@ static void icp_reset(void *dev)
> >      qemu_set_irq(icp->output, 0);
> >  }
> >  
> > +static void icp_reset_handler(void *dev)
> > +{
> > +    DeviceClass *dc = DEVICE_GET_CLASS(dev);
> > +
> > +    dc->reset(dev);
> > +}
> > +
> >  static void icp_realize(DeviceState *dev, Error **errp)
> >  {
> >      ICPState *icp = ICP(dev);
> > @@ -345,7 +352,7 @@ static void icp_realize(DeviceState *dev, Error **errp)
> >          return;
> >      }
> >  
> > -    qemu_register_reset(icp_reset, dev);
> > +    qemu_register_reset(icp_reset_handler, dev);
> >      vmstate_register(NULL, icp->cs->cpu_index, &vmstate_icp_server, icp);
> >  }
> >  
> > @@ -354,7 +361,7 @@ static void icp_unrealize(DeviceState *dev, Error **errp)
> >      ICPState *icp = ICP(dev);
> >  
> >      vmstate_unregister(NULL, &vmstate_icp_server, icp);
> > -    qemu_unregister_reset(icp_reset, dev);
> > +    qemu_unregister_reset(icp_reset_handler, dev);
> >  }
> >  
> >  static void icp_class_init(ObjectClass *klass, void *data)
> > @@ -363,6 +370,7 @@ static void icp_class_init(ObjectClass *klass, void *data)
> >  
> >      dc->realize = icp_realize;
> >      dc->unrealize = icp_unrealize;
> > +    dc->reset = icp_reset;
> >  }
> >  
> >  static const TypeInfo icp_info = {
> >   
>