Re: xe vs amdgpu userptr handling

[Daniel Vetter <daniel@ffwll.ch>]

On Wed, Feb 07, 2024 at 08:08:42AM -0300, Maíra Canal wrote:
> Adding another point to this discussion, would it make sense to somehow
> create a generic structure that all drivers, including shmem drivers, could
> use it?


So the issue is a bit that at least the userptr for shmem drivers I've
seen all just use pin_user_pages(FOLL_LONGTERM), which nails the memory in
place and makes it unshrinkable. I think that would be fairly easy to
integrate into shmem helpers, and might already be a win (to catch stuff
like userspace trying to share these). This memory probably should be
accounted against mlock rlimit, but that's an entire different can of
worms.

Going full dynamic cross driver is a lot more infrastructure, because your
command submission path needs to substantially change. I think that only
makes when you have a lot more cross-driver code and not just the bare
minimum buffer helpers, and I think gpuvm might be a good place to fit.
Since that already has the concepts around "prepare this entire vm", and
extending that is a lot more reasonable than building an entire new thing.

I'm on board with Dave and agree that we really shouldn't have a diverse
bouqet of driver specific implementations of all this, but I think
fundamentally we will end up with the above two flavours for various
reasons.

So which userptr do you mean?

Cheers, Sima

> 
> Best Regards,
> - Maíra
> 
> On 2/7/24 03:56, Dave Airlie wrote:
> > I'm just looking over the userptr handling in both drivers, and of
> > course they've chosen different ways to represent things. Again this
> > is a divergence that is just going to get more annoying over time and
> > eventually I'd like to make hmm/userptr driver independent as much as
> > possible, so we get consistent semantics in userspace.
> > 
> > AFAICS the main difference is that amdgpu builds the userptr handling
> > inside a GEM object in the kernel, whereas xe doesn't bother creating
> > a holding object and just handles things directly in the VM binding
> > code.
> > 
> > Is this just different thinking at different times here?
> > like since we have VM BIND in xe, it made sense not to bother creating
> > a gem object for userptrs?
> > or is there some other advantages to going one way or the other?
> > 
> > Dave.

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch