Re: [RFC PATCH 00/13] Ultra Ethernet driver introduction

Yunsheng Lin
2025-03-28T12:20:20Z

On 2025/3/27 21:26, Jason Gunthorpe wrote:

...

>
>> which I think will work for NICs that
>> need a PD and ones that don't. It can support MR -> PD -> Job, but
>> I considered the PD -> job relationship as 1 to many.
>
> Yes, and the 1:1 is degenerate.
>
>> Sure, It's challenging in that a UET endpoint (QP) may communicate
>> with multiple jobs, and a MR may be accessible by a single job, all
>> jobs, or only a few.
>
> I would suggest that the PD is a superset of all jobs and the objects
> (endpoint, mr, etc) get to choose a subset of the PD's jobs during
> allocation?
>
> Or you keep job/pd as 1:1 and allow specifying multiple PDs during
> object allocation.
>
> But to be clear, this is largely verbs modeling stuff - however there
> is a certain practicality to trying to fit this multi-job ability into
> a PD because it allow reusing alot of existing uAPI kernel code.
>
> Especially if people are going to take existing RDMA HW and tweak it
> to some level of UET (ie support only single job) and still require a
> HW level PD under the covers.

Through reading this patchset, it seems the semantics of 'job' for UEC
is about how to identify a PDC(Packet Delivery Context) instance, which
is specified by src fep_address/pdc_id and dst fep_address/pdc_id as
there seems to be more than one PDC instance between two nodes, so the
'job' is really about grouping processes from the same 'job' to use the
same PDC instance and preventing processes from different 'job' from
using the same PDC instance?

And the interesting part about the PDC seems to be:
1. It is created dynamically when a request packet is sent on the target
side or received on the initiator side, and destroyed dynamically through
timeout mechanism.
2. It seems a PDC instance can be shared between different processes from the
same 'job'.

And SRD seems to be using AH(address handle) to specify a SRD connection between
two nodes, and there is only one SRD context instance between two nodes, so different
process may have their own AH pointing to the same SRD context instance between the
same two nodes?