Re: [RFC net-next 0/7] Provide an ism layer

Andrew Lunn
2025-01-20T16:01:44Z

> What is central to ISM is the DMB (Direct Memory Buffer). The concept
> that there is a DMB dedicated to one writer and one reader. It is owned
> by the reader and only this writer can write at any offset into the DMB
> (Fabric controlled). (Reader can technically read/write as well).
>
> So for the client API I think the core functions are
> - move_data(*data, target_dmb_token, offset) - called by the sending
> client, to move data at some offset into a DMB.

Missing a length, but otherwise this looks O.K.

> - receive_signal(dmb_token, some_signal_info) - called by the ism layer
> to signal the client, that this DMB needs handling. (currently called
> handle_irq)

So there is no indication where in the DMB there is new content?

And when you say "This DMB" does that imply there are multiple DMB
shared between two peers?

Maybe i have the wrong idea about a DMB. I was thinking of maybe 64K
to a few Mega bytes of memory, in a memory which could truly be shared
by CPUs. But maybe a DMB is just a 4K Page, and you have lots of them?
If you are 'faking' a shared memory with DMA, they can be anywhere in
the address space where the DMA engine can access them.

> I would not want to abstract that to a message based API, because then
> we need queues etc and are almost at a net_device. All that is not
> needed for ism, because DMBs are dedicated to a single writer (who has
> the responsibility).

But i assume there are "protocols" above this. You talked about
running a TTY over this. That should be standardized, so everybody
implements TTYs in exactly the same way.

> > One thing we cannot hide, however, is whether the operation is zero-copy
> > or copy. This distinction is important because we can reuse the data at
> > different times in copy mode and zero-copy mode.

This needs more explanation. Are you talking about putting data into
the DMB, or moving the DMB to the peer?

If you have a DMA engine
moving stuff around, the data can be anywhere the DMA engine can
access. But if you have a true shared memory, ideally you want to
avoid copying into it.

Then you have the API used by your protocol drivers above. For a TTY
running at 9600 baud, a copy into the DMB does not matter. But if you
are talking about a network protocol stack on top, your copy from user
space to kernel space probably wants to go direct into the DMB. So
maybe your API also needs to include allocating/freeing DMBs in an
abstract way so it can hide the difference between true shared memory,
and kernel memory which can be DMAed?

Andrew