Re: [EXT] Re: [RFC rdma 1/3] RDMA/core: Create a common mmap function

[Jason Gunthorpe <jgg@ziepe.ca>]

On Fri, Jul 05, 2019 at 03:29:03PM +0000, Michal Kalderon wrote:
> > From: Jason Gunthorpe <jgg@ziepe.ca>
> > Sent: Thursday, July 4, 2019 3:35 PM
> > 
> > External Email
> > 
> > On Wed, Jul 03, 2019 at 11:19:34AM +0300, Gal Pressman wrote:
> > > On 03/07/2019 1:31, Jason Gunthorpe wrote:
> > > >> Seems except Mellanox + hns the mmap flags aren't ABI.
> > > >> Also, current Mellanox code seems like it won't benefit from mmap
> > > >> cookie helper functions in any case as the mmap function is very
> > > >> specific and the flags used indicate the address and not just how to map
> > it.
> > > >
> > > > IMHO, mlx5 has a goofy implementaiton here as it codes all of the
> > > > object type, handle and cachability flags in one thing.
> > >
> > > Do we need object type flags as well in the generic mmap code?
> > 
> > At the end of the day the driver needs to know what page to map during the
> > mmap syscall.
> > 
> > mlx5 does this by encoding the page type in the address, and then many
> > types have seperate lookups based onthe offset for the actual page.
> > 
> > IMHO the single lookup and opaque offset is generally better..
> > 
> > Since the mlx5 scheme is ABI it can't be changed unfortunately.
> > 
> > If you want to do user controlled cachability flags, or not, is a fair question,
> > but they still become ABI..
> > 
> > I'm wondering if it really makes sense to do that during the mmap, or if the
> > cachability should be set as part of creating the cookie?
> > 
> > > Another issue is that these flags aren't exposed in an ABI file, so a
> > > userspace library can't really make use of it in current state.
> > 
> > Woops.
> > 
> > Ah, this is all ABI so you need to dig out of this hole ASAP :)
> >
> Jason, I didn't follow - what is all ABI? 
> currently EFA implementation encodes the cachability inside the key,
> It's not exposed in ABI file and is opaque to user-space. The kernel decides on the cachability
> And get's it back in the key when mmap is called. It seems good
> enough for the current cases.

Then the key 'offset' should not include cachability information at
all.

Jason