Re: Support for CXL v3.0 spec with QEMU

[Jonathan Cameron <Jonathan.Cameron@Huawei.com>]

On Thu, 26 Oct 2023 07:22:16 +0530
Ravi Kanth <mvrravikanth@gmail.com> wrote:

> Thanks Jonathan. For switch devices via IOCTL, Do you mean that CXL
> driver will not have functionality to support or notify host
> applications about async events / interrupts via IOCTL or this has not
> yet been discussed?

No notifications yet. We'll figure it out, but just not gotten there
yet. For events, we might just implement the tracepoint stuff from
the type3 driver.  Should be fairly easy to do.. though I've not yet done enough
diving in the spec to be sure...

> 
> Is it possible to provide an interface to read "Device status
> registers" and specifically "Event status register" section 8.2.8.3
> and 8.2.8.3.1 via IOCTL interface from the switch CCI endpoint via
> IOCTL using CXL driver?

I'd rather avoid that if we can as it will share even less infrastructure
with the main driver than we currently do.

Jonathan

> 
> Ravi
> 
> On Mon, Oct 23, 2023 at 7:20 PM Jonathan Cameron
> <Jonathan.Cameron@huawei.com> wrote:
> >
> > On Sun, 22 Oct 2023 12:15:13 +0530
> > Ravi Kanth <mvrravikanth@gmail.com> wrote:
> >  
> > > Thanks Ira for providing the references.
> > >
> > > I am specifically looking for event support for the switch CCI
> > > endpoint(/dev/cxl/switch) via the IOCTL interface. I could see
> > > cxl_swmb_setup_mailbox() in the switchdev.c file. However I am not
> > > able to understand how user space applications could make use of it.  
> >
> > If you have all the moving parts (i.e. recent qemu + the kernel patches
> > I just linked to in another branch of this thread) then the example at:
> > https://gitlab.com/jic23/cxl-fmapi-tests
> >
> > will let you interact with the emulated switch mailbox CCI
> > via the raw ioctl command.  So far it does basic crawl out and enumerate
> > what what it finds.
> >  
> > >
> > > The ndctl project does not also have references on how we could make
> > > use of the switch CCI endpoint to get the interrupts nor on how we can
> > > read "Device status registers" and specifically "Event status
> > > register" section 8.2.8.3 and 8.2.8.3.1 via IOCTL interface from the
> > > switch CCI endpoint.  
> >
> > Whilst we haven't really discussed it yet, I'd not expect ndctl (which
> > is focused on host interaction) will support much in the way of specific
> > features for fabric management.
> >
> > The cxl-fmapi-tests are not intended to be used for production use cases
> > either. My expectation is that one of the projects more generally looking
> > at CXL fabric management will provide that functionality. I've not really
> > been keeping track of these but I gather there is work in various standards
> > orgs (outside of the CXL consortium) to define how it will be done at a
> > higher level.
> >
> > Jonathan
> >  
> > >
> > > - Ravi
> > >
> > >
> > >
> > > On Sat, Oct 21, 2023 at 2:08 AM Ira Weiny <ira.weiny@intel.com> wrote:  
> > > >
> > > > Ravi Kanth wrote:  
> > > > > Hi Jonathan / Gregory,
> > > > > Just wanted to touch base on below questions and if you have any
> > > > > inputs on the same. Thanks for your help.
> > > > >
> > > > > Thanks
> > > > > Ravi
> > > > >
> > > > > On Wed, Oct 18, 2023 at 4:00 PM Ravi Kanth <mvrravikanth@gmail.com> wrote:  
> > > > > >
> > > > > > Hi Jonathan / Gregory,
> > > > > >
> > > > > > 1. Do we have the support in CXL driver to read the "Device status
> > > > > > registers" and specifically "Event status register" section 8.2.8.3
> > > > > > and 8.2.8.3.1?  
> > > >
> > > > Yes the upstream driver reads this when processing the Event interrupt
> > > > from the device.  Then it uses the value to chose which logs to read.
> > > >
> > > > See cxl_event_thread() in the kernel source.
> > > >  
> > > > > > 2. If an interrupt is posted by the device firmware, how will user
> > > > > > space applications be notified ? Do we have an interface for the same
> > > > > > in CXL driver?  
> > > >
> > > > All events are reported through the trace infrastructure.
> > > >  
> > > > > >
> > > > > > If the above features are already supported in CXL driver, Can you
> > > > > > please point us to the sample code snippets to achieve the same?  
> > > >
> > > > ndctl has the ability to monitor these events and example C code in there.
> > > >
> > > > See .../cxl/event_trace.c in the ndctl project.[1]
> > > >
> > > > [1] https://github.com/pmem/ndctl
> > > >  
> > > > > >
> > > > > > Also does the switch cci function change part of the mainline?  
> > > >
> > > > I'm not sure I parse this question but I'm also not super familiar with
> > > > switch cci.  So I'll let Jonathan answer this.
> > > >
> > > > Ira
> > > >  
> > > > > >
> > > > > > Thanks for your help in advance.
> > > > > >
> > > > > > Thanks
> > > > > > Ravi
> > > > > >  
> > > >
> > > > [snip]  
> > >
> > >
> > >  
> >  
> 
>