Re: [RFC PATCH 00/13] Ultra Ethernet driver introduction

["Ziemba, Ian" <ian.ziemba@hpe.com>]

> There's also the MR relationship:
> 
> Job <- 1 --- 0..n -> MR <- 0..n --- 1 -> PD

Current UE memory registration is centered around the libfabric
FI_MR_ENDPOINT mode which states memory regions are associated with an
endpoint instead of libfabric domain. For remotely accessible UE MRs,
this translates to the following.

- Relative Addressing: {job ID, endpoint, RKEY} identifies the MR.

- Absolute Addressing: {endpoint, RKEY} identifies the MR with optional
  MR job-ID access control check.

In addition, UE memory registration supports user-defined RKEYs. This
enables programming model implementations the optimization to use
well-known per process endpoint RKEYs. For relative addressing, this
could result in the same RKEY value existing multiple times at the
{job ID} level, but only once at the {job ID, endpoint} level.

The UE remote MR relationship would look like:

Job <- 1 --- 0..n -> EP <- 0..n ---------------------- 1 -> PD
                      ^                                      ^
                      |--- 1 --- 0..n -> MR <- 0..n --- 1 ---|

> There's discussion on defining this relationship:
> 
> Job <- 0..n --- 1 -> PD
> 
> I can't think of a technical reason why that's needed.

From my UE perspective, I agree. UE needs to share job IDs across
processes while still having inter-process isolation for things like
local memory registrations.

Thanks,

Ian