Re: [RFC PATCH v2 09/20] fwctl/cxl: Add driver for CXL mailbox for handling CXL features commands

[Jason Gunthorpe <jgg@nvidia.com>]

On Thu, Dec 05, 2024 at 05:21:13PM -0800, Dan Williams wrote:
> Dave Jiang wrote:
> > Add a cxl fwctl driver to allow sending of CXL feature
> > commands from userspace through as ioctls. Create a driver skeleton for
> > initial setup.
> 
> I don't understand why fwctl support of the *existing* mailbox needs
> yet another sub-device and driver?

It is how to modprobe the fwctl driver independently from the main cxl
codebase.

You want the fwctl parts to be in their own loadable module even if
cxl core is built in, and the loadable module should auto load when
cxl is present in the system.

Spawning a new struct device is the best way to do this.

> In the mlx5 case, as I understand it, multiple families of device can
> register the common mlx5_fwctl auxiliary device to share a common driver
> for that piece. For CXL that shared object is the 'struct cxl_mailbox'.

There is only one mlx5_core driver. The use of auxiliary bus in mlx5
is purely to split the driver up into many modules in many
subsystems. Each subsystem probes its own driver that autoloads if the
underlying HW supports that subsystem.

Basically, it creates modularity within the kernel for complex
multi-subsystem devices.

> Now, I am not opposed to drivers/fwctl/cxl.c as a library that the CXL
> core can use to implement CXL fwctl support, if it helps to keep all
> fwctl users co-located, but I don't see a need to make it a new CXL
> sub-device.

I was expecting that subsystem drivers would reside in the subsystem
directory, so it shouldn't be a library, it should be the fwctl
particular parts here.

> @@ -5772,6 +5772,7 @@ L:        linux-cxl@vger.kernel.org
>  S:     Maintained
>  F:     Documentation/driver-api/cxl
>  F:     drivers/cxl/
> +F:     drivers/fwctl/cxl*
>  F:     include/cxl/
>  F:     include/uapi/linux/cxl_mem.h
> 
> ...and call it a day.

Makes sense

Jason