Re: [RFC PATCH v2 15/15] misc/arm-cla: Add vfio-mdev user interface

[Ryan Roberts <ryan.roberts@arm.com>]

On 20/10/2026 15:15, Jason Gunthorpe wrote:
> On Tue, Oct 20, 2026 at 11:41:57AM +0100, Ryan Roberts wrote:
>> On 16/10/2026 15:42, Jason Gunthorpe wrote:
>>> On Fri, Oct 16, 2026 at 03:22:00PM +0100, Jean-Philippe Brucker wrote:
>>>> On Thu, 15 Oct 2026 at 17:41, Jason Gunthorpe <jgg@nvidia.com> wrote:
>>>>> I think I said this last time, please don't use vfio mdev for
>>>>> anything. There is nothing mediated about this.
>>>>
>>>> We don't multiplex a device between guests, but the host driver must
>>>> control part of the device registers while the remainder is assigned to
>>>> a guest. The host configures accelerator access, stage-2 translation,
>>>> handles CPUHP and context save/restore. So we can't bind the device to
>>>> vfio-platform.
>>>
>>> That's fine, you can make a proper vfio-cla driver.
>>
>> Hi Jason,
>>
>> Thanks for taking a look at this! I've been chewing on this comment and trying
>> to educate myself deeper on the subject. I have a proposal...
>>
>> Currently we create a struct device per CLA device instance. But multiple CLA
>> device instances cooperate within a domain and the domain is the "unit of
>> isolation"; 
> 
> Well that's weird and a bit confusing.

Yes; this conversation is helping me to see that... :)

I was originally trying to avoid creating any globally visible entity that
describes a "domain", since I thought there was a possibility that each process
might want to define it's own view of a domain. But in hindsight I don't think
that's helpful...

> 
>> we can't have some CLAs from a domain assigned to a VM, while others
>> from the same domain are used by the host. Therefore it didn't feel like the
>> correct model to unbind one of these devices from the native driver and rebind
>> it to a cla-vfio driver. mdev seemed to solve this problem nicely for us.
>>
>> But looking into it in more detail, I guess something like this might be a bit
>> more standard:
>>
>>  - Split the existing CLA driver into three components:
>>     - cla-core: Hardware discovery, domain configuration, struct device
>>       registration and CPU hotplug
>>     - cla-native driver: Provides the user space accelerator interfaces
>>     - cla-vfio driver: Enables domains to be assigned to virtual machines
>>  - Each domain has a struct device (created by the core)
>>     - An individual CLA is not represented by a struct device
>>  - Either cla-native or cla-vfio can then bind to a domain device
>>     - cla-native exposes a char device per domain, per accelerator class
>>     - cla-vfio
>>
>> For VFIO, I guess we could use one VFIO region per CLA, which would allow the
>> VMM to construct a device tree. But it would need to additionally determine
>> which CPU each CLA is bound to; perhaps that's a sysfs interface that cla-core
>> could expose?
> 
> I don't know alot about your CLA, but this sounds reasonable. There
> may be uses with containers to have the split exposed like this too.

...This all feels much clearer and simpler. I'll work on an approach like this.

Thanks,
Ryan

> 
> Jason