Re: [RFC PATCH v1 1/8] misc/arm-cla: Add driver skeleton and documentation

["Arnd Bergmann" <arnd@arndb.de>]

On Fri, Jul 17, 2026, at 17:44, Ryan Roberts wrote:
> On 17/07/2026 14:49, Arnd Bergmann wrote:
>> On Fri, Jul 17, 2026, at 12:47, Ryan Roberts wrote:
>> This sounds a lot like st64bv or st64bv0, passing an 8-word payload and returning
>> a single word per accelerator operation with shared addressing.
>
> We're actually passing 9 words here; 8 DATA words plus the LAUNCH word. CLA
> supports only 64 bit aligned and sized accesses (other accesses are RAZ/WI) so
> they have to be written as 9x64bit stores. Then poll using 64bit loads.
>
>> 
>> Why are there now two interfaces to do the same thing?
>
> Good question. This is how the HW operates.
>
>> 
>> Can a user process use st64bv to do the four steps in a
>> single instruction?
>
> No, unfortunately not.
>

Ok

>> Can a CPU start multiple asynchronous operations concurrently?
>
> Yes; STATUS indicates READY while it can accept more asynchronous operations
> ("comamnds").

How does userspace know which operations have already completed
then? (not worried about this bit, just trying to understand)

>> Do these continue to run if the starting process is scheduled out
>> and another process also tries to use CLA?
>
> Yes; the driver manages assignment of a CLA to a process context completely
> separately from the thread scheduler's decisions about which threads run on
> which CPUs and when. If another process is scheduled onto the CPU and it
> attempts to access it's VA for the CLA, it will fault into the driver's handler
> and be put to sleep until the driver decides to reassign the CLA.

This part does sound dangerous, not in the sense that I think it's
fundamentally broken, but in the complexity it adds.

I wonder if it's feasible to simplify this by always canceling
any ongoing CLA operations during switch_mm():

Is there an upper bound on how long a single operation can take,
or a guarantee that an operation at least provides a partial result
in hardware?

From your earlier descriptions, it sounds like the CPU is usually
assumed to wait for completion with WFE anyway, so from the
scheduler's perspective, the thread is active while waiting for
the accelerator to complete a job (even if from hardware side
the CPU is powered down during WFE).

If this is how it generally operates, and the accelerator jobs
are usually fast, forcing the CLA TTBR0 to be the same as the
CPU TTBR0 would avoid the entire problem of unmapping the registers
on context switch, but instead let this hook into the same place
as the corresponding iommu_mm_data switch on x86, which seems
to handle this more nicely.

> I'm not sure what you mean by "Which instruction specifically causes the fault".
> A fault occurs within the accelerator if it tries to access a virtual address
> that is not mapped by the page table or if the permissions of the mapping are
> not sufficient, etc... The fact that the accelerator has faulted is reported to
> the SW that is polling the accelerator's STATUS register within user space. That
> SW is expected to trigger fault handling by the usual kernel mechanisms by
> accessing the VA. Then it issues a RESOLVE operation to tell the accelerator it
> can continue.

Got it now. I had assumed that the page fault is delivered asynchronously
to the kernel without user space getting involved. In this case, I think
the interface is actually cleaner, though it does add a little bit
of overhead for userspace having to decipher the status. 
 
>>> +User space software is expected to create a thread to drive each CLA it is
>>> +using, and for each thread to be pinned to the CLA's local CPU.
>> 
>> What happens if multiple processes have the same chardev open and
>> each mmap() that, e.g. after a fork()? Does each process see its
>> own virtual instance of the accelerator and interact with it through
>> the same physical MMIO register range but its own process address space,
>> or do you have to rely on the registers being mapped only into a
>> single mm_struct to prevent a process from messing with another process
>> data?
>
> The driver maintains a cla_ctx for each {file description, mm_struct} pair. So
> in this case, even though the file description is shared between the parent and
> child processes, they still have distinct mm_structs so still have separate
> contexts allowing the driver to virtualize access correctly.

Ok, got it.

     Arnd