Re: [RFC PATCH 00/13] Ultra Ethernet driver introduction

Ziemba, Ian
2025-04-04T16:53:13Z

On 4/1/2025 2:39 PM, Jason Gunthorpe wrote:
>> I don't know that I can talk about the UEC spec,
>
> Right, it is too early to talk about UEC and Linux until people can
> freely talk about what it actually needs.

While the UE specs are not yet out, concepts can be discussed. I
recognize this may be annoying without specs.

The following is my understanding of the UE job model.

A job ID identifies "who am I." There are two different operational
modes:

1. Relative Addressing

The endpoint address is only valid within the scope of the job ID.
Endpoints can only transmit and receive on the associated job ID.
Parallel applications will use this to restrict communication to
only processes within the job.

Processes must be granted access to the job ID. In addition,
multiple processes may share a job ID. Some mechanism is required to
restrict what job IDs an endpoint can be configured against. Having
a device-level RDMA job object and a path to associate the job
object with an endpoint seems reasonable.

2. Absolute Addressing

The target endpoint address is outside the scope of the job ID. This
behavior allows an endpoint to receive on all job IDs and transmit
on only authorized job IDs. This mode enables server endpoints to
support multiple clients with different job IDs.

Since this mode impacts the job IDs transmitted in packets,
processes must be granted access. A device-level RDMA job object
seems reasonable for this as well.

An optional mechanism to restrict a receive buffer and MR to a
specific job ID is needed. This enables a server endpoint to have
per client job ID resources. Job ID verification is unnecessary
since the job ID associated with a receive buffer or MR does not
impact the packet job ID.

UE is going to need some object to restrict registered user-space
memory. Having the PD as the object supporting local memory
registration isolation seems ok. The UE object relationship could look
like job <- 1 --- 0..* -> endpoint <- 0..* --- 1 -> PD.

Thanks,

Ian