Re: [Qemu-devel] mem1 is in use, can not be deleted

[Igor Mammedov <imammedo@redhat.com>]

On Fri, 10 Jul 2015 17:35:08 +0200
Eduardo Otubo <eduardo.otubo@profitbricks.com> wrote:

> > > > > Yes, you're right. The reason is surely because dimm1 wasn't deleted
> > > > > -- and I think I didn't make my point very clear -- my question was
> > > > > more about: Is there any reason for dimm1 not being deleted? The
> > > > > reason why I tested with the guest OS fully running and on GRUB is
> > > > > because I guessed the guest OS was using this memory and couldn't be
> > > > > deallocated. If that's the case, and qemu did a best effort to remove
> > > > > and couldn't because guest was using it, then Ok, I just need to
> > > > > adapt my tests. Other than that perhaps I hit a bug.
> > > > Guest OS has to:
> > > >  1. support memory hot remove
> > > 
> > > How do I know if guest OS supports memory hot remove? I'm testing on
> > > Debian 8 with kernel 4.1. I start qemu with "-m 2G,slots=32,maxmem=8G".
> > kernel should be compiled with memory remove options
> 
> Double checked that and yes, my guest kernel has memory hotplug support.
> 
> > 
> > Also memory removal is allowed to fail if guest kernel is not able
> > to offline corresponding memory sections but it probably should notify
> > QEMU via ACPI about failure.
> 
> How can I check this notification?
build QEMU with tracing enabled and you can use it to see what's going on
in QEMU<->guest ACPI interface.

$grep mhp trace-events 
mhp_acpi_invalid_slot_selected(uint32_t slot) "0x%"PRIx32
mhp_acpi_ejecting_invalid_slot(uint32_t slot) "0x%"PRIx32
mhp_acpi_read_addr_lo(uint32_t slot, uint32_t addr) "slot[0x%"PRIx32"] addr lo: 0x%"PRIx32
mhp_acpi_read_addr_hi(uint32_t slot, uint32_t addr) "slot[0x%"PRIx32"] addr hi: 0x%"PRIx32
mhp_acpi_read_size_lo(uint32_t slot, uint32_t size) "slot[0x%"PRIx32"] size lo: 0x%"PRIx32
mhp_acpi_read_size_hi(uint32_t slot, uint32_t size) "slot[0x%"PRIx32"] size hi: 0x%"PRIx32
mhp_acpi_read_pxm(uint32_t slot, uint32_t pxm) "slot[0x%"PRIx32"] proximity: 0x%"PRIx32
mhp_acpi_read_flags(uint32_t slot, uint32_t flags) "slot[0x%"PRIx32"] flags: 0x%"PRIx32
mhp_acpi_write_slot(uint32_t slot) "set active slot: 0x%"PRIx32
mhp_acpi_write_ost_ev(uint32_t slot, uint32_t ev) "slot[0x%"PRIx32"] OST EVENT: 0x%"PRIx32
mhp_acpi_write_ost_status(uint32_t slot, uint32_t st) "slot[0x%"PRIx32"] OST STATUS: 0x%"PRIx32
mhp_acpi_clear_insert_evt(uint32_t slot) "slot[0x%"PRIx32"] clear insert event"
mhp_acpi_clear_remove_evt(uint32_t slot) "slot[0x%"PRIx32"] clear remove event"
mhp_acpi_pc_dimm_deleted(uint32_t slot) "slot[0x%"PRIx32"] pc-dimm deleted"
mhp_acpi_pc_dimm_delete_failed(uint32_t slot) "slot[0x%"PRIx32"] pc-dimm delete failed"
mhp_pc_dimm_assigned_slot(int slot) "0x%d"
mhp_pc_dimm_assigned_address(uint64_t addr) "0x%"PRIx64

events of interest for you may be *_ost_*
you can check hw/acpi/memory_hotplug.c for events meaning

You also might need to enable ACPI debugging in guest to see what's happening
on guest side if there is no *pc_dimm_deleted event in the trace.

> 
> > 
> > 
> > > 
> > > >  2. eject memory device using ACPI _EJ0 method, once it has handled
> > > >  removal request, provided it is able to free corresponding memory pages
> > > > See docs they should have flows described for success and failure case.
> > > 
> > > When I issue the command "device_del dimm1" I see no output on dmesg on
> > > the guest OS. I guess this is a sign that perhaps the guest does not
> > > support it?
> > > 
> > > From the (very nice) diagram I found at docs/specs/acpi_mem_hotplug.txt,
> > > Qemu QMP should output some sort of failure if Guest OS fails to
> > > process ejection right? The only information I see is:
> > you need to use query-acpi-ospm-status command to see slot status.
> 
> Yep, this command outputs that the dimm is still there, no news :/
> 
> > 
> > > 
> > >  (qemu) device_del dimm1
> > >  device_del dimm1
> > > 
> > >  (qemu) info memory-devices
> > >  info memory-devices
> > >  Memory device [dimm]: "dimm1"
> > >    addr: 0x100000000
> > >    slot: 0
> > >    node: 0
> > >    size: 1073741824
> > >    memdev: /objects/mem1
> > >    hotplugged: true
> > >    hotpluggable: true
> > > 
> > >  (qemu) info memdev
> > >  info memdev
> > >  memory backend: 0
> > >    size:  1073741824
> > >    merge: true
> > >    dump: true
> > >    prealloc: false
> > >    policy: default
> > >    host nodes: 
> > > 
> > > How was the environment when you tested this feature?
> > Most likely I've used RHEL7.1 as guest with latest systemd
> > which onlines hotplugged memory automatically on hotplug.
> 
> I tried with Ubuntu 15.04, latest kernel 4.2 and systemd, still not
> working. I'm downloading CentOS-7, I'll setup with systemd and proper
> kernel configuration. I'll let you know the results.
> 
> Thanks a lot for the help so far! :)
>