Re: [RFC PATCH 00/13] Ultra Ethernet driver introduction

[Jason Gunthorpe <jgg@nvidia.com>]

On Wed, Apr 16, 2025 at 11:58:45PM +0000, Sean Hefty wrote:
> > > There's discussion on defining this relationship:
> > >
> > > Job <- 0..n --- 1 -> PD
> > >
> > > I can't think of a technical reason why that's needed.
> > 
> > From my UE perspective, I agree. UE needs to share job IDs across processes
> > while still having inter-process isolation for things like local memory
> > registrations.
> 
> We seem stuck on this.  Here's a specific proposal that I'm considering:

I still think it is hard to have this discussion without information
flowing from UET..

I think the "Relative Addressing" Ian described is just a PD pointing
to a single job and all MRs within the PD linked to a single job. Is
there more than that?

"Absolute Addressing" seems confusing from a OS perspective. You can
receive packets on any Job ID but the OS prevents you from sending on
unauthorized Job IDs. Implying authorization happens dynamically.  So
if you Rx a packet, how does an unpriv process go about getting OS
permission to use the Rx'd Job ID as a Tx? How does it NAK the Rx that
it isn't permitted? Why would you want to create an entire special
security mechanism just to partition MRs in this funny mode?

How does receive buffer job key partitioning work? UET will HW match
receive buffers to specific packets?

> 1. Define a device level 'security key'.  The skey encapsulates encryption attributes.
>     The skey may be shared between processes.
> 2. Define a device level 'job', or maybe more generic 'communication domain'*.
>     A job object is associated with a transport protocol and these optional attributes:
>     address, job id (required for UET), and security key.
>     The job object may be shared between processes.
> 3. Define a PD level 'job key'.  The job key references a single job object.
>     Multiple job keys may be created under a single PD, if each references a separate job.
> 4. Support creating MRs that reference job keys.

This seems reasonable as a starting framework to me. I have wondered
if the 'security key' is really addressing information though. Sharing
IP's/MAC's/Encryption/etc across all job users seems appealing for MPI
type workloads.

But is one job key under a MR sufficient or does UET expect this to be
a list of job keys?

Jason