From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:44791) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1buwZJ-0004W7-FB for qemu-devel@nongnu.org; Fri, 14 Oct 2016 03:06:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1buwZE-0005wz-8u for qemu-devel@nongnu.org; Fri, 14 Oct 2016 03:06:11 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:44941) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1buwZE-0005wh-10 for qemu-devel@nongnu.org; Fri, 14 Oct 2016 03:06:08 -0400 Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.17/8.16.0.17) with SMTP id u9E73kL6089601 for ; Fri, 14 Oct 2016 03:06:05 -0400 Received: from e28smtp04.in.ibm.com (e28smtp04.in.ibm.com [125.16.236.4]) by mx0a-001b2d01.pphosted.com with ESMTP id 262rva4vu5-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Fri, 14 Oct 2016 03:06:05 -0400 Received: from localhost by e28smtp04.in.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 14 Oct 2016 12:36:02 +0530 Date: Fri, 14 Oct 2016 12:35:39 +0530 From: Bharata B Rao Reply-To: bharata@linux.vnet.ibm.com References: <1476314039-9520-1-git-send-email-mdroth@linux.vnet.ibm.com> <1476314039-9520-12-git-send-email-mdroth@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1476314039-9520-12-git-send-email-mdroth@linux.vnet.ibm.com> Message-Id: <20161014070539.GA28693@in.ibm.com> Subject: Re: [Qemu-devel] [PATCH 11/11] spapr: Memory hot-unplug support List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Michael Roth Cc: qemu-devel@nongnu.org, qemu-ppc@nongnu.org, david@gibson.dropbear.id.au, nfont@linux.vnet.ibm.com, jallen@linux.vnet.ibm.com On Wed, Oct 12, 2016 at 06:13:59PM -0500, Michael Roth wrote: > From: Bharata B Rao > > Add support to hot remove pc-dimm memory devices. > > Signed-off-by: Bharata B Rao > * add hooks to CAS/cmdline enablement of hotplug ACR support > Signed-off-by: Michael Roth > --- > hw/ppc/spapr.c | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++++- > hw/ppc/spapr_drc.c | 17 +++++++++ > 2 files changed, 122 insertions(+), 1 deletion(-) > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c > index 9af4268..180fa3d 100644 > --- a/hw/ppc/spapr.c > +++ b/hw/ppc/spapr.c > @@ -2310,6 +2310,90 @@ out: > error_propagate(errp, local_err); > } > > +typedef struct sPAPRDIMMState { > + uint32_t nr_lmbs; > +} sPAPRDIMMState; > + > +static void spapr_lmb_release(DeviceState *dev, void *opaque) > +{ > + sPAPRDIMMState *ds = (sPAPRDIMMState *)opaque; > + HotplugHandler *hotplug_ctrl = NULL; > + > + if (--ds->nr_lmbs) { > + return; > + } > + > + g_free(ds); > + > + /* > + * Now that all the LMBs have been removed by the guest, call the > + * pc-dimm unplug handler to cleanup up the pc-dimm device. > + */ > + hotplug_ctrl = qdev_get_hotplug_handler(dev); > + hotplug_handler_unplug(hotplug_ctrl, dev, &error_abort); > +} > + > +static void spapr_del_lmbs(DeviceState *dev, uint64_t addr_start, uint64_t size, > + Error **errp) > +{ > + sPAPRDRConnector *drc; > + sPAPRDRConnectorClass *drck; > + uint32_t nr_lmbs = size / SPAPR_MEMORY_BLOCK_SIZE; > + int i; > + sPAPRDIMMState *ds = g_malloc0(sizeof(sPAPRDIMMState)); > + uint64_t addr = addr_start; > + > + ds->nr_lmbs = nr_lmbs; > + for (i = 0; i < nr_lmbs; i++) { > + drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_LMB, > + addr / SPAPR_MEMORY_BLOCK_SIZE); > + g_assert(drc); > + > + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc); > + drck->detach(drc, dev, spapr_lmb_release, ds, errp); > + addr += SPAPR_MEMORY_BLOCK_SIZE; > + } > + > + drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_LMB, > + addr_start / SPAPR_MEMORY_BLOCK_SIZE); > + drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc); > + spapr_hotplug_req_remove_by_count_indexed(SPAPR_DR_CONNECTOR_TYPE_LMB, > + nr_lmbs, > + drck->get_index(drc)); > +} > + > +static void spapr_memory_unplug(HotplugHandler *hotplug_dev, DeviceState *dev, > + Error **errp) > +{ > + sPAPRMachineState *ms = SPAPR_MACHINE(hotplug_dev); > + PCDIMMDevice *dimm = PC_DIMM(dev); > + PCDIMMDeviceClass *ddc = PC_DIMM_GET_CLASS(dimm); > + MemoryRegion *mr = ddc->get_memory_region(dimm); > + > + pc_dimm_memory_unplug(dev, &ms->hotplug_memory, mr); > + object_unparent(OBJECT(dev)); > +} > + > +static void spapr_memory_unplug_request(HotplugHandler *hotplug_dev, > + DeviceState *dev, Error **errp) > +{ > + Error *local_err = NULL; > + PCDIMMDevice *dimm = PC_DIMM(dev); > + PCDIMMDeviceClass *ddc = PC_DIMM_GET_CLASS(dimm); > + MemoryRegion *mr = ddc->get_memory_region(dimm); > + uint64_t size = memory_region_size(mr); > + uint64_t addr; > + > + addr = object_property_get_int(OBJECT(dimm), PC_DIMM_ADDR_PROP, &local_err); > + if (local_err) { > + goto out; > + } > + > + spapr_del_lmbs(dev, addr, size, &error_abort); > +out: > + error_propagate(errp, local_err); > +} > + > void *spapr_populate_hotplug_cpu_dt(CPUState *cs, int *fdt_offset, > sPAPRMachineState *spapr) > { > @@ -2383,10 +2467,21 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev, > static void spapr_machine_device_unplug(HotplugHandler *hotplug_dev, > DeviceState *dev, Error **errp) > { > + sPAPRMachineState *sms = SPAPR_MACHINE(qdev_get_machine()); > MachineClass *mc = MACHINE_GET_CLASS(qdev_get_machine()); > > if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) { > - error_setg(errp, "Memory hot unplug not supported by sPAPR"); > + if (spapr_ovec_test(sms->ov5_cas, OV5_HP_EVT)) { > + spapr_memory_unplug(hotplug_dev, dev, errp); > + } else { > + /* NOTE: this means there is a window after guest reset, prior to > + * CAS negotiation, where unplug requests will fail due to the > + * capability not being detected yet. This is a bit different than > + * the case with PCI unplug, where the events will be queued and > + * eventually handled by the guest after boot > + */ > + error_setg(errp, "Memory hot unplug not supported for this guest"); > + } > } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) { > if (!mc->query_hotpluggable_cpus) { > error_setg(errp, "CPU hot unplug not supported on this machine"); > @@ -2396,6 +2491,14 @@ static void spapr_machine_device_unplug(HotplugHandler *hotplug_dev, > } > } > > +static void spapr_machine_device_unplug_request(HotplugHandler *hotplug_dev, > + DeviceState *dev, Error **errp) > +{ > + if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) { > + spapr_memory_unplug_request(hotplug_dev, dev, errp); > + } > +} > + > static void spapr_machine_device_pre_plug(HotplugHandler *hotplug_dev, > DeviceState *dev, Error **errp) > { > @@ -2482,6 +2585,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) > hc->plug = spapr_machine_device_plug; > hc->unplug = spapr_machine_device_unplug; > mc->cpu_index_to_socket_id = spapr_cpu_index_to_socket_id; > + hc->unplug_request = spapr_machine_device_unplug_request; After this, spapr_core_unplug() call needs to be moved to ->unplug_request() as ->unplug() won't be called if ->unplug_request() is present. However I am not able to get CPU hotplug working with your patchset. Looks like RTAS event is getting missed or not getting generated. Will get back after further debugging. Regards, Bharata.