Re: [PATCH v1] pc: Support coldplugging of virtio-pmem-pci devices on all buses

["Daniel P. Berrangé" <berrange@redhat.com>]

On Tue, May 26, 2020 at 04:43:35PM +0200, David Hildenbrand wrote:
> On 26.05.20 16:22, Vivek Goyal wrote:
> > On Tue, May 26, 2020 at 03:44:10PM +0200, David Hildenbrand wrote:
> >> On 26.05.20 15:28, Vivek Goyal wrote:
> >>> On Mon, May 25, 2020 at 10:45:11AM +0200, David Hildenbrand wrote:
> >>>> E.g., with "pc-q35-4.2", trying to coldplug a virtio-pmem-pci devices
> >>>> results in
> >>>>     "virtio-pmem-pci not supported on this bus"
> >>>>
> >>>> Reasons is, that the bus does not support hotplug and, therefore, does
> >>>> not have a hotplug handler. Let's allow coldplugging virtio-pmem devices
> >>>> on such buses. The hotplug order is only relevant for virtio-pmem-pci
> >>>> when the guest is already alive and the device is visible before
> >>>> memory_device_plug() wired up the memory device bits.
> >>>>
> >>>> Hotplug attempts will still fail with:
> >>>>     "Error: Bus 'pcie.0' does not support hotplugging"
> >>>>
> >>>> Hotunplug attempts will still fail with:
> >>>>     "Error: Bus 'pcie.0' does not support hotplugging"
> >>>>
> >>>> Reported-by: Vivek Goyal <vgoyal@redhat.com>
> >>>> Cc: Pankaj Gupta <pankaj.gupta.linux@gmail.com>
> >>>> Cc: Igor Mammedov <imammedo@redhat.com>
> >>>> Cc: Paolo Bonzini <pbonzini@redhat.com>
> >>>> Cc: Richard Henderson <rth@twiddle.net>
> >>>> Cc: Eduardo Habkost <ehabkost@redhat.com>
> >>>> Cc: "Michael S. Tsirkin" <mst@redhat.com>
> >>>> Cc: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
> >>>> Signed-off-by: David Hildenbrand <david@redhat.com>
> >>>> ---
> >>>>  hw/i386/pc.c | 18 ++++++++++--------
> >>>>  1 file changed, 10 insertions(+), 8 deletions(-)
> >>>
> >>> Thanks for the patch David. I still seem to face a different error though.
> >>>
> >>> 2020-05-26T13:26:05.720617Z qemu-system-x86_64: -device virtio-pmem-pci,memdev=pmem1,id=nv1: memory devices (e.g. for memory hotplug) are not enabled, please specify the maxmem option
> >>>
> >>> Following is my domain xml file.
> >>>
> >>> Vivek
> >>
> >> Hi Vivek,
> >>
> >> you have to declare the maxMemory option. Memory devices like
> >> virtio-pmem-pci reside in RAM like a pc-dimm or a nvdimm. If your
> >> virtio-pmem device will be 4GB, you have to add that to maxMemory.
> >>
> >>   <memory unit='GiB'>64</memory>
> >>   <maxMemory unit='GiB'>68</maxMemory>
> >>   <currentMemory unit='GiB'>64</currentMemory>
> >>
> >> (you might have to add "slots='0'" or "slots='1'" to maxMemory to make
> >> libvirt happy)
> > 
> > Ok, tried that.
> > 
> > <maxMemory slots='1' unit='KiB'>134217728</maxMemory>
> > 
> > And now it complains about.
> > 
> > error: unsupported configuration: At least one numa node has to be configured when enabling memory hotplug
> > 
> > So ultimately it seems to be wanting me to somehow enable memory hotplug
> > to be able to use virtio-pmem?
> 
> That's a libvirt error message. Maybe I am confused how libvirt maps
> these parameters to QEMU ...
> 
> NVDIMMs under libvirt seem to be easy:
> 
> https://www.redhat.com/archives/libvir-list/2016-August/msg00055.html
> 
> Maybe the issue is that virtio-pmem has not been properly integrated
> into libvirt yet:
> 
> https://www.redhat.com/archives/libvir-list/2019-August/msg00007.html

While libvirt has generic pmem support, it doesn't have virtio-pmem:

https://libvirt.org/formatdomain.html#elementsMemory

eg

  <memory model='nvdimm' access='shared'>
    <uuid>
    <source>
      <path>/dev/dax0.0</path>
      <alignsize unit='KiB'>2048</alignsize>
      <pmem/>
    </source>
    <target>
      <size unit='KiB'>524288</size>
      <node>1</node>
      <label>
        <size unit='KiB'>128</size>
      </label>
    </target>
  </memory>

> Maybe maxMemory in libvirt does not directly map to the QEMU variant to
> define the maximum physical address space reserved also for any memory
> devices (DIMMs, NVDIMMs, virtio-pmem, ...). Any libvirt experts that can
> help?

<maxMemory> reflects the upper limit on what you can hot-plug at
runtime:

   https://libvirt.org/formatdomain.html#elementsMemoryAllocation


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|