Handling multiple inheritance [for CXL]

Handling multiple inheritance [for CXL]

[Ben Widawsky]

I'm working on CXL 2.0 type 3 memory devices [1]. In short, these are PCIe devices
that have persistent memory on them. As such, it would be nice to inherit from
both a PCI_DEVICE class as well as an NVDIMM device class.

Truth be told, using TYPE_MEMORY_DEVICE as the interface does provide most of
what I need. I'm wondering what the best way to handle this is. Currently, the
only thing NVDIMM class provides is write/read_label_data, this is driven by
_DSM. For CXL, the mechanism to read/write the equivalent area is not done via
_DSM, but done directly via a mailbox interface. However, the intent is the
same, and so utilizing similar code seems ideal.

If there's a desire to unify these code paths, I'd need something like multiple
inheritance. I'm looking for some feedback here on how to do it.

Thanks.
Ben

[1]: https://lore.kernel.org/qemu-devel/20210105165323.783725-1-ben.widawsky@intel.com/

Re: Handling multiple inheritance [for CXL]

[Daniel P. Berrangé]

On Tue, Jan 26, 2021 at 01:33:52PM -0800, Ben Widawsky wrote:
> I'm working on CXL 2.0 type 3 memory devices [1]. In short, these are PCIe devices
> that have persistent memory on them. As such, it would be nice to inherit from
> both a PCI_DEVICE class as well as an NVDIMM device class.
> 
> Truth be told, using TYPE_MEMORY_DEVICE as the interface does provide most of
> what I need. I'm wondering what the best way to handle this is. Currently, the
> only thing NVDIMM class provides is write/read_label_data, this is driven by
> _DSM. For CXL, the mechanism to read/write the equivalent area is not done via
> _DSM, but done directly via a mailbox interface. However, the intent is the
> same, and so utilizing similar code seems ideal.
> 
> If there's a desire to unify these code paths, I'd need something like multiple
> inheritance. I'm looking for some feedback here on how to do it.

We don't have a direct concept of multiple inheritance in QOM.

The closest you can get is to turn the NVDIMM class into an
interface. You can inherit from PCI_DEVICE and then implement
the NVDIMM interface.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: Handling multiple inheritance [for CXL]

[Ben Widawsky]

On 21-01-27 10:06:48, Daniel P. Berrangé wrote:
> On Tue, Jan 26, 2021 at 01:33:52PM -0800, Ben Widawsky wrote:
> > I'm working on CXL 2.0 type 3 memory devices [1]. In short, these are PCIe devices
> > that have persistent memory on them. As such, it would be nice to inherit from
> > both a PCI_DEVICE class as well as an NVDIMM device class.
> > 
> > Truth be told, using TYPE_MEMORY_DEVICE as the interface does provide most of
> > what I need. I'm wondering what the best way to handle this is. Currently, the
> > only thing NVDIMM class provides is write/read_label_data, this is driven by
> > _DSM. For CXL, the mechanism to read/write the equivalent area is not done via
> > _DSM, but done directly via a mailbox interface. However, the intent is the
> > same, and so utilizing similar code seems ideal.
> > 
> > If there's a desire to unify these code paths, I'd need something like multiple
> > inheritance. I'm looking for some feedback here on how to do it.
> 
> We don't have a direct concept of multiple inheritance in QOM.
> 
> The closest you can get is to turn the NVDIMM class into an
> interface. You can inherit from PCI_DEVICE and then implement
> the NVDIMM interface.
> 
> Regards,
> Daniel

Is there a concise summary of what the tradeoffs would be of moving NVDIMM to an
interface? AFAICT, there's a lot of things done through subclassing that can
just as easily be done as an interface, but I don't understand the reason for
that.

Re: Handling multiple inheritance [for CXL]

[Igor Mammedov]

On Tue, 26 Jan 2021 13:33:52 -0800
Ben Widawsky <ben@bwidawsk.net> wrote:

> I'm working on CXL 2.0 type 3 memory devices [1]. In short, these are PCIe devices
> that have persistent memory on them. As such, it would be nice to inherit from
> both a PCI_DEVICE class as well as an NVDIMM device class.
> 
> Truth be told, using TYPE_MEMORY_DEVICE as the interface does provide most of
> what I need.
could you be more specific on what you need from it?

>I'm wondering what the best way to handle this is. Currently, the
> only thing NVDIMM class provides is write/read_label_data, this is driven by
> _DSM. For CXL, the mechanism to read/write the equivalent area is not done via
> _DSM, but done directly via a mailbox interface. However, the intent is the
> same, and so utilizing similar code seems ideal.
> 
> If there's a desire to unify these code paths, I'd need something like multiple
> inheritance. I'm looking for some feedback here on how to do it.
> 
> Thanks.
> Ben
> 
> [1]: https://lore.kernel.org/qemu-devel/20210105165323.783725-1-ben.widawsky@intel.com/
> 

Re: Handling multiple inheritance [for CXL]

[Ben Widawsky]

On 21-01-27 21:18:24, Igor Mammedov wrote:
> On Tue, 26 Jan 2021 13:33:52 -0800
> Ben Widawsky <ben@bwidawsk.net> wrote:
> 
> > I'm working on CXL 2.0 type 3 memory devices [1]. In short, these are PCIe devices
> > that have persistent memory on them. As such, it would be nice to inherit from
> > both a PCI_DEVICE class as well as an NVDIMM device class.
> > 
> > Truth be told, using TYPE_MEMORY_DEVICE as the interface does provide most of
> > what I need.
> could you be more specific on what you need from it?
> 

I'm trying to register my persistent memory as normal system memory. I assume
it's required that I implement the memory interface to do that. If it's not,
that's fine too.

For reference:
https://gitlab.com/bwidawsk/qemu/-/blob/cxl-2.0v3/hw/mem/cxl_type3.c

> >I'm wondering what the best way to handle this is. Currently, the
> > only thing NVDIMM class provides is write/read_label_data, this is driven by
> > _DSM. For CXL, the mechanism to read/write the equivalent area is not done via
> > _DSM, but done directly via a mailbox interface. However, the intent is the
> > same, and so utilizing similar code seems ideal.
> > 
> > If there's a desire to unify these code paths, I'd need something like multiple
> > inheritance. I'm looking for some feedback here on how to do it.
> > 
> > Thanks.
> > Ben
> > 
> > [1]: https://lore.kernel.org/qemu-devel/20210105165323.783725-1-ben.widawsky@intel.com/
> > 
> 

Re: Handling multiple inheritance [for CXL]

[Igor Mammedov]

On Wed, 27 Jan 2021 12:25:44 -0800
Ben Widawsky <ben@bwidawsk.net> wrote:

> On 21-01-27 21:18:24, Igor Mammedov wrote:
> > On Tue, 26 Jan 2021 13:33:52 -0800
> > Ben Widawsky <ben@bwidawsk.net> wrote:
> >   
> > > I'm working on CXL 2.0 type 3 memory devices [1]. In short, these are PCIe devices
> > > that have persistent memory on them. As such, it would be nice to inherit from
> > > both a PCI_DEVICE class as well as an NVDIMM device class.
> > > 
> > > Truth be told, using TYPE_MEMORY_DEVICE as the interface does provide most of
> > > what I need.  
> > could you be more specific on what you need from it?
> >   
> 
> I'm trying to register my persistent memory as normal system memory. I assume
> it's required that I implement the memory interface to do that. If it's not,
> that's fine too.
> 
> For reference:
> https://gitlab.com/bwidawsk/qemu/-/blob/cxl-2.0v3/hw/mem/cxl_type3.c

if you use TYPE_MEMORY_DEVICE machinery, then address/(max)size a device takes in
hotplug ram window, is fixed at device creation time.
If you use PCI BAR to map memory, it should be possible to reprogram BAR
anywhere in PCI address space at runtime.

> > >I'm wondering what the best way to handle this is. Currently, the
> > > only thing NVDIMM class provides is write/read_label_data, this is driven by
> > > _DSM. For CXL, the mechanism to read/write the equivalent area is not done via
> > > _DSM, but done directly via a mailbox interface. However, the intent is the
> > > same, and so utilizing similar code seems ideal.
> > > 
> > > If there's a desire to unify these code paths, I'd need something like multiple
> > > inheritance. I'm looking for some feedback here on how to do it.
> > > 
> > > Thanks.
> > > Ben
> > > 
> > > [1]: https://lore.kernel.org/qemu-devel/20210105165323.783725-1-ben.widawsky@intel.com/
> > >   
> >   
> 

Re: Handling multiple inheritance [for CXL]

[Ben Widawsky]

On 21-01-27 22:33:37, Igor Mammedov wrote:
> On Wed, 27 Jan 2021 12:25:44 -0800
> Ben Widawsky <ben@bwidawsk.net> wrote:
> 
> > On 21-01-27 21:18:24, Igor Mammedov wrote:
> > > On Tue, 26 Jan 2021 13:33:52 -0800
> > > Ben Widawsky <ben@bwidawsk.net> wrote:
> > >   
> > > > I'm working on CXL 2.0 type 3 memory devices [1]. In short, these are PCIe devices
> > > > that have persistent memory on them. As such, it would be nice to inherit from
> > > > both a PCI_DEVICE class as well as an NVDIMM device class.
> > > > 
> > > > Truth be told, using TYPE_MEMORY_DEVICE as the interface does provide most of
> > > > what I need.  
> > > could you be more specific on what you need from it?
> > >   
> > 
> > I'm trying to register my persistent memory as normal system memory. I assume
> > it's required that I implement the memory interface to do that. If it's not,
> > that's fine too.
> > 
> > For reference:
> > https://gitlab.com/bwidawsk/qemu/-/blob/cxl-2.0v3/hw/mem/cxl_type3.c
> 
> if you use TYPE_MEMORY_DEVICE machinery, then address/(max)size a device takes in
> hotplug ram window, is fixed at device creation time.
> If you use PCI BAR to map memory, it should be possible to reprogram BAR
> anywhere in PCI address space at runtime.
> 

This is not part of the PCI address space. I believe there will be quite a bit
of work to support hotplug properly for CXL devices, but I believe making it a
PCI BAR is the wrong approach.

If you're not yet familiar with the spec, it might make some sense to take a
look as I think I'm not doing a good job conveying what this hardware is.

https://www.computeexpresslink.org/download-the-specification

> > > >I'm wondering what the best way to handle this is. Currently, the
> > > > only thing NVDIMM class provides is write/read_label_data, this is driven by
> > > > _DSM. For CXL, the mechanism to read/write the equivalent area is not done via
> > > > _DSM, but done directly via a mailbox interface. However, the intent is the
> > > > same, and so utilizing similar code seems ideal.
> > > > 
> > > > If there's a desire to unify these code paths, I'd need something like multiple
> > > > inheritance. I'm looking for some feedback here on how to do it.
> > > > 
> > > > Thanks.
> > > > Ben
> > > > 
> > > > [1]: https://lore.kernel.org/qemu-devel/20210105165323.783725-1-ben.widawsky@intel.com/
> > > >   
> > >   
> > 
>