[PATCH v1 0/7] Add virtio gpu userptr support

[PATCH v1 0/7] Add virtio gpu userptr support

[Honglei Huang]

From: Honglei Huang <Honglei1.Huang@amd.com>

Hello,

This series add virtio gpu userptr support and add libhsakmt capset.
The userptr feature is used for let host access guest user space memory,
this feature is used for GPU compute use case, to enable ROCm/OpenCL native
context. It should be pointed out that we are not to implement SVM here, 
this is just a buffer based userptr implementation.
The libhsakmt capset is used for ROCm context, libhsakmt is like the role 
of libdrm in Mesa.

Patches 1-2 add libhsakmt capset and userptr blob resource flag.
Patches 3-5 implement basic userptr feature, in some popular bench marks,
it has an efficiency of about 70% compared to bare metal in OpenCL API.
Patche 6 adds interval tree.
Patche 7 adds MMU notifier, let UMD do not need to manage userptr and
increase efficiency by 20% to 30%. With this patch, OpenCL in ROCm can
achieve 95%+ efficiency compared to bare metal in some popular bench marks.

Honglei Huang (7):
  virtio-gpu api: add blob userptr resource
  drm/virtgpu api: add blob userptr resource
  drm/virtio: implement userptr: probe for the feature
  drm/virtio: implement userptr: add userptr obj
  drm/virtio: advertise base userptr feature to userspace
  drm/virtio: implement userptr: add interval tree
  drm/virtio: implement userptr: add mmu notifier

 drivers/gpu/drm/virtio/Makefile          |   3 +-
 drivers/gpu/drm/virtio/virtgpu_debugfs.c |   1 +
 drivers/gpu/drm/virtio/virtgpu_drv.c     |   1 +
 drivers/gpu/drm/virtio/virtgpu_drv.h     |  91 +++
 drivers/gpu/drm/virtio/virtgpu_ioctl.c   |  22 +-
 drivers/gpu/drm/virtio/virtgpu_kms.c     |  10 +-
 drivers/gpu/drm/virtio/virtgpu_object.c  |   5 +
 drivers/gpu/drm/virtio/virtgpu_userptr.c | 766 +++++++++++++++++++++++
 include/uapi/drm/virtgpu_drm.h           |   5 +-
 include/uapi/linux/virtio_gpu.h          |   7 +
 10 files changed, 905 insertions(+), 6 deletions(-)
 create mode 100644 drivers/gpu/drm/virtio/virtgpu_userptr.c

-- 
2.34.1

[PATCH v1 1/7] virtio-gpu api: add blob userptr resource

[Honglei Huang]

From: Honglei Huang <Honglei1.Huang@amd.com>

Add a new resource for blob resource, called userptr, used for let
host access guest user space memory, to acquire buffer based userptr
feature in virtio GPU.

- The capset VIRTIO_GPU_CAPSET_HSAKMT used for context init,
in this series patches only HSAKMT context can use the userptr
feature. HSAKMT is a GPU compute library in HSA stack, like
the role libdrm in mesa stack.
- New flag VIRTIO_GPU_BLOB_FLAG_USE_USERPTR used in blob create
to indicate the blob create ioctl is used for create a userptr
blob resource.

Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
---
 include/uapi/linux/virtio_gpu.h | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/include/uapi/linux/virtio_gpu.h b/include/uapi/linux/virtio_gpu.h
index bf2c9cabd207..4da36a1e62c4 100644
--- a/include/uapi/linux/virtio_gpu.h
+++ b/include/uapi/linux/virtio_gpu.h
@@ -65,6 +65,11 @@
  */
 #define VIRTIO_GPU_F_CONTEXT_INIT        4
 
+/*
+ * VIRTGPU_BLOB_FLAG_USE_USERPTR
+ */
+#define VIRTIO_GPU_F_RESOURCE_USERPTR    5
+
 enum virtio_gpu_ctrl_type {
 	VIRTIO_GPU_UNDEFINED = 0,
 
@@ -312,6 +317,7 @@ struct virtio_gpu_cmd_submit {
 /* 3 is reserved for gfxstream */
 #define VIRTIO_GPU_CAPSET_VENUS 4
 #define VIRTIO_GPU_CAPSET_DRM 6
+#define VIRTIO_GPU_CAPSET_HSAKMT 8
 
 /* VIRTIO_GPU_CMD_GET_CAPSET_INFO */
 struct virtio_gpu_get_capset_info {
@@ -404,6 +410,7 @@ struct virtio_gpu_resource_create_blob {
 #define VIRTIO_GPU_BLOB_FLAG_USE_MAPPABLE     0x0001
 #define VIRTIO_GPU_BLOB_FLAG_USE_SHAREABLE    0x0002
 #define VIRTIO_GPU_BLOB_FLAG_USE_CROSS_DEVICE 0x0004
+#define VIRTIO_GPU_BLOB_FLAG_USE_USERPTR      0x0008
 	/* zero is invalid blob mem */
 	__le32 blob_mem;
 	__le32 blob_flags;
-- 
2.34.1

Re: [PATCH v1 1/7] virtio-gpu api: add blob userptr resource

[Demi Marie Obenour]

On 2/28/25 12:36 AM, Honglei Huang wrote:
> From: Honglei Huang <Honglei1.Huang@amd.com>
> 
> Add a new resource for blob resource, called userptr, used for let
> host access guest user space memory, to acquire buffer based userptr
> feature in virtio GPU.
> 
> - The capset VIRTIO_GPU_CAPSET_HSAKMT used for context init,
> in this series patches only HSAKMT context can use the userptr
> feature. HSAKMT is a GPU compute library in HSA stack, like
> the role libdrm in mesa stack.

Userptr should not be limited to HSMKMT contexts.  Userptr can
accelerate shm buffers by avoiding a copy from guest to host, and
it can be implemented using grant tables on Xen.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

Re: [PATCH v1 1/7] virtio-gpu api: add blob userptr resource

[Huang, Honglei1]

On 2025/3/1 5:21, Demi Marie Obenour wrote:
> On 2/28/25 12:36 AM, Honglei Huang wrote:
>> From: Honglei Huang <Honglei1.Huang@amd.com>
>>
>> Add a new resource for blob resource, called userptr, used for let
>> host access guest user space memory, to acquire buffer based userptr
>> feature in virtio GPU.
>>
>> - The capset VIRTIO_GPU_CAPSET_HSAKMT used for context init,
>> in this series patches only HSAKMT context can use the userptr
>> feature. HSAKMT is a GPU compute library in HSA stack, like
>> the role libdrm in mesa stack.
> 
> Userptr should not be limited to HSMKMT contexts.  Userptr can
> accelerate shm buffers by avoiding a copy from guest to host, and
> it can be implemented using grant tables on Xen.

Yes, I totally agree userptr can accelerate shm buffers, but I currently
don't know if there are any other projects working on similar features,
or if maintainers have any opinions or better ways to implement them, so
I temporarily limit this feature to HSAKMT context only.

I am waiting for everyone's opinions, please provide your thoughts.

Regards,
Honglei

Re: [PATCH v1 1/7] virtio-gpu api: add blob userptr resource

[Gurchetan Singh]

[-- Attachment #1: Type: text/plain, Size: 1774 bytes --]

On Thu, Mar 6, 2025 at 2:52 AM Huang, Honglei1 <Honglei1.Huang@amd.com>
wrote:

>
> On 2025/3/1 5:21, Demi Marie Obenour wrote:
> > On 2/28/25 12:36 AM, Honglei Huang wrote:
> >> From: Honglei Huang <Honglei1.Huang@amd.com>
> >>
> >> Add a new resource for blob resource, called userptr, used for let
> >> host access guest user space memory, to acquire buffer based userptr
> >> feature in virtio GPU.
> >>
> >> - The capset VIRTIO_GPU_CAPSET_HSAKMT used for context init,
> >> in this series patches only HSAKMT context can use the userptr
> >> feature. HSAKMT is a GPU compute library in HSA stack, like
> >> the role libdrm in mesa stack.
> >
> > Userptr should not be limited to HSMKMT contexts.  Userptr can
> > accelerate shm buffers by avoiding a copy from guest to host, and
> > it can be implemented using grant tables on Xen.
>
> Yes, I totally agree userptr can accelerate shm buffers, but I currently
> don't know if there are any other projects working on similar features,
> or if maintainers have any opinions or better ways to implement them, so
> I temporarily limit this feature to HSAKMT context only.
>
> I am waiting for everyone's opinions, please provide your thoughts.
>

I wonder if you can emulate userptr using udmabuf on the host-side?

Essentially for the guest, it'll be a malloc'ed memory, which means a guest
sg list.  We can convert the guest sg-list to udmabuf using well-known
mechanisms on the host side.  I hope amdkfd can operate on dma-bufs too?

I do such a feature that would have a more generic utility outside of
HSAKMT contexts and not rely on Xen-specific grant tables  ...
checkout VIRTIO_GPU_BLOB_FLAG_CREATE_GUEST_HANDLE in crosvm for an
example.




>
> Regards,
> Honglei
>

[-- Attachment #2: Type: text/html, Size: 2552 bytes --]

Re: [PATCH v1 1/7] virtio-gpu api: add blob userptr resource

[Huang, Honglei1]

On 2025/3/18 9:37, Gurchetan Singh wrote:
> 
> 
> On Thu, Mar 6, 2025 at 2:52 AM Huang, Honglei1 <Honglei1.Huang@amd.com 
> <mailto:Honglei1.Huang@amd.com>> wrote:
> 
> 
>     On 2025/3/1 5:21, Demi Marie Obenour wrote:
>      > On 2/28/25 12:36 AM, Honglei Huang wrote:
>      >> From: Honglei Huang <Honglei1.Huang@amd.com
>     <mailto:Honglei1.Huang@amd.com>>
>      >>
>      >> Add a new resource for blob resource, called userptr, used for let
>      >> host access guest user space memory, to acquire buffer based userptr
>      >> feature in virtio GPU.
>      >>
>      >> - The capset VIRTIO_GPU_CAPSET_HSAKMT used for context init,
>      >> in this series patches only HSAKMT context can use the userptr
>      >> feature. HSAKMT is a GPU compute library in HSA stack, like
>      >> the role libdrm in mesa stack.
>      >
>      > Userptr should not be limited to HSMKMT contexts.  Userptr can
>      > accelerate shm buffers by avoiding a copy from guest to host, and
>      > it can be implemented using grant tables on Xen.
> 
>     Yes, I totally agree userptr can accelerate shm buffers, but I currently
>     don't know if there are any other projects working on similar features,
>     or if maintainers have any opinions or better ways to implement them, so
>     I temporarily limit this feature to HSAKMT context only.
> 
>     I am waiting for everyone's opinions, please provide your thoughts.
> 
> 
> I wonder if you can emulate userptr using udmabuf on the host-side?
> 
> Essentially for the guest, it'll be a malloc'ed memory, which means a 
> guest sg list.  We can convert the guest sg-list to udmabuf using well- 
> known mechanisms on the host side.  I hope amdkfd can operate on dma- 
> bufs too?
> 
> I do such a feature that would have a more generic utility outside of 
> HSAKMT contexts and not rely on Xen-specific grant tables  ... 
> checkout VIRTIO_GPU_BLOB_FLAG_CREATE_GUEST_HANDLE in crosvm for an example.

I think userptr can be emulated by udmabuf if the sg list is from guest 
user sapce and not moveable, amdkfd can operate dma-buf, and the userptr 
feature can be achieved by partial functions of dma-buf as far as I can see.

But the udmabuf need managed by UMD, this will result in some 
performance loss, and we are planning to implement virtio GPU userptr 
using hmm_rang_fault, udmabuf solution may be completely different from 
this one.

I saw the feature VIRTIO_GPU_BLOB_FLAG_CREATE_GUEST_HANDLE is used by 
the crosvm in beta branch, but I can not find the kernel virtio gpu 
part, how to access it? Maybe I can try or help.

> 
> 
> 
>     Regards,
>     Honglei
> 

Re: [PATCH v1 1/7] virtio-gpu api: add blob userptr resource

[Huang, Honglei1]

On 2025/3/18 19:25, Huang, Honglei1 wrote:
> 
> 
> On 2025/3/18 9:37, Gurchetan Singh wrote:
>>
>>
>> On Thu, Mar 6, 2025 at 2:52 AM Huang, Honglei1 <Honglei1.Huang@amd.com 
>> <mailto:Honglei1.Huang@amd.com>> wrote:
>>
>>
>>     On 2025/3/1 5:21, Demi Marie Obenour wrote:
>>      > On 2/28/25 12:36 AM, Honglei Huang wrote:
>>      >> From: Honglei Huang <Honglei1.Huang@amd.com
>>     <mailto:Honglei1.Huang@amd.com>>
>>      >>
>>      >> Add a new resource for blob resource, called userptr, used for 
>> let
>>      >> host access guest user space memory, to acquire buffer based 
>> userptr
>>      >> feature in virtio GPU.
>>      >>
>>      >> - The capset VIRTIO_GPU_CAPSET_HSAKMT used for context init,
>>      >> in this series patches only HSAKMT context can use the userptr
>>      >> feature. HSAKMT is a GPU compute library in HSA stack, like
>>      >> the role libdrm in mesa stack.
>>      >
>>      > Userptr should not be limited to HSMKMT contexts.  Userptr can
>>      > accelerate shm buffers by avoiding a copy from guest to host, and
>>      > it can be implemented using grant tables on Xen.
>>
>>     Yes, I totally agree userptr can accelerate shm buffers, but I 
>> currently
>>     don't know if there are any other projects working on similar 
>> features,
>>     or if maintainers have any opinions or better ways to implement 
>> them, so
>>     I temporarily limit this feature to HSAKMT context only.
>>
>>     I am waiting for everyone's opinions, please provide your thoughts.
>>
>>
>> I wonder if you can emulate userptr using udmabuf on the host-side?
>>
>> Essentially for the guest, it'll be a malloc'ed memory, which means a 
>> guest sg list.  We can convert the guest sg-list to udmabuf using 
>> well- known mechanisms on the host side.  I hope amdkfd can operate on 
>> dma- bufs too?
>>
>> I do such a feature that would have a more generic utility outside of 
>> HSAKMT contexts and not rely on Xen-specific grant tables  ... 
>> checkout VIRTIO_GPU_BLOB_FLAG_CREATE_GUEST_HANDLE in crosvm for an 
>> example.
> 
> I think userptr can be emulated by udmabuf if the sg list is from guest 
> user sapce and not moveable, amdkfd can operate dma-buf, and the userptr 
> feature can be achieved by partial functions of dma-buf as far as I can 
> see.
> 
> But the udmabuf need managed by UMD, this will result in some 
> performance loss, and we are planning to implement virtio GPU userptr 
> using hmm_rang_fault, udmabuf solution may be completely different from 
> this one.
> 
> I saw the feature VIRTIO_GPU_BLOB_FLAG_CREATE_GUEST_HANDLE is used by 
> the crosvm in beta branch, but I can not find the kernel virtio gpu 
> part, how to access it? Maybe I can try or help.

Hi Gurchetan,

	I found VIRTIO_GPU_BLOB_FLAG_CREATE_GUEST_HANDLE has been added into 
crosvm 4 years ago. But I can not find the corresponding kernel 
implementation, also can not find in chromeos kernel:
https://chromium.googlesource.com/chromiumos/third_party/kernel/.
	And it seems like the VIRTIO_GPU_BLOB_FLAG_CREATE_GUEST_HANDLE only 
support the system memory according to the commit message, but I can see 
the virtio has plan to support guest 
heap:https://github.com/google/crosvm/blob/dee7838993bae90cad49d5595b958cfe4c28b58d/devices/src/virtio/gpu/virtio_gpu.rs#L1056. 

	Can I create new virtio gpu patches for adding heap supported virtio 
gpu udmabuf?

Regards,
Honglei

> 
>>
>>
>>
>>     Regards,
>>     Honglei
>>
> 

Re: [PATCH v1 1/7] virtio-gpu api: add blob userptr resource

[Dmitry Osipenko]

On 3/6/25 13:51, Huang, Honglei1 wrote:
> 
> On 2025/3/1 5:21, Demi Marie Obenour wrote:
>> On 2/28/25 12:36 AM, Honglei Huang wrote:
>>> From: Honglei Huang <Honglei1.Huang@amd.com>
>>>
>>> Add a new resource for blob resource, called userptr, used for let
>>> host access guest user space memory, to acquire buffer based userptr
>>> feature in virtio GPU.
>>>
>>> - The capset VIRTIO_GPU_CAPSET_HSAKMT used for context init,
>>> in this series patches only HSAKMT context can use the userptr
>>> feature. HSAKMT is a GPU compute library in HSA stack, like
>>> the role libdrm in mesa stack.
>>
>> Userptr should not be limited to HSMKMT contexts.  Userptr can
>> accelerate shm buffers by avoiding a copy from guest to host, and
>> it can be implemented using grant tables on Xen.
> 
> Yes, I totally agree userptr can accelerate shm buffers, but I currently
> don't know if there are any other projects working on similar features,
> or if maintainers have any opinions or better ways to implement them, so
> I temporarily limit this feature to HSAKMT context only.
> 
> I am waiting for everyone's opinions, please provide your thoughts.

USERPTR should be relevant for anything Vulkan-related, like Venus and
native contexts. I expect that this new feature will work universally
good for all context types.

In order to merge USERPTR support upstream, we at least will need to
prototype the guest USERPTR in one of native context driver to know that
it works. You'll need to post the whole set of host/guest USERPTR
patches including QEMU and etc, not just the kernel patches.

-- 
Best regards,
Dmitry

Re: [PATCH v1 1/7] virtio-gpu api: add blob userptr resource

[Huang, Honglei1]

On 2025/3/26 22:46, Dmitry Osipenko wrote:
> On 3/6/25 13:51, Huang, Honglei1 wrote:
>>
>> On 2025/3/1 5:21, Demi Marie Obenour wrote:
>>> On 2/28/25 12:36 AM, Honglei Huang wrote:
>>>> From: Honglei Huang <Honglei1.Huang@amd.com>
>>>>
>>>> Add a new resource for blob resource, called userptr, used for let
>>>> host access guest user space memory, to acquire buffer based userptr
>>>> feature in virtio GPU.
>>>>
>>>> - The capset VIRTIO_GPU_CAPSET_HSAKMT used for context init,
>>>> in this series patches only HSAKMT context can use the userptr
>>>> feature. HSAKMT is a GPU compute library in HSA stack, like
>>>> the role libdrm in mesa stack.
>>>
>>> Userptr should not be limited to HSMKMT contexts.  Userptr can
>>> accelerate shm buffers by avoiding a copy from guest to host, and
>>> it can be implemented using grant tables on Xen.
>>
>> Yes, I totally agree userptr can accelerate shm buffers, but I currently
>> don't know if there are any other projects working on similar features,
>> or if maintainers have any opinions or better ways to implement them, so
>> I temporarily limit this feature to HSAKMT context only.
>>
>> I am waiting for everyone's opinions, please provide your thoughts.
> 
> USERPTR should be relevant for anything Vulkan-related, like Venus and
> native contexts. I expect that this new feature will work universally
> good for all context types.
> 
> In order to merge USERPTR support upstream, we at least will need to
> prototype the guest USERPTR in one of native context driver to know that
> it works. You'll need to post the whole set of host/guest USERPTR
> patches including QEMU and etc, not just the kernel patches.

Ok totally understood, I will upload other components as soon as 
possiable, cause we only have limited resource for this matter, please 
understand the speed is relatively slow.
And for Vulkan use case, yes it is very important, I will make some 
prototype implementation for it, and will test it locally. The 
implementation for USERPTR must comply with Vulakan's standard.
Really thanks for your comments.

Reagrds,
Honglei

> 

Re: [PATCH v1 1/7] virtio-gpu api: add blob userptr resource

[Dmitry Osipenko]

On 3/27/25 06:00, Huang, Honglei1 wrote:
> 
> 
> On 2025/3/26 22:46, Dmitry Osipenko wrote:
>> On 3/6/25 13:51, Huang, Honglei1 wrote:
>>>
>>> On 2025/3/1 5:21, Demi Marie Obenour wrote:
>>>> On 2/28/25 12:36 AM, Honglei Huang wrote:
>>>>> From: Honglei Huang <Honglei1.Huang@amd.com>
>>>>>
>>>>> Add a new resource for blob resource, called userptr, used for let
>>>>> host access guest user space memory, to acquire buffer based userptr
>>>>> feature in virtio GPU.
>>>>>
>>>>> - The capset VIRTIO_GPU_CAPSET_HSAKMT used for context init,
>>>>> in this series patches only HSAKMT context can use the userptr
>>>>> feature. HSAKMT is a GPU compute library in HSA stack, like
>>>>> the role libdrm in mesa stack.
>>>>
>>>> Userptr should not be limited to HSMKMT contexts.  Userptr can
>>>> accelerate shm buffers by avoiding a copy from guest to host, and
>>>> it can be implemented using grant tables on Xen.
>>>
>>> Yes, I totally agree userptr can accelerate shm buffers, but I currently
>>> don't know if there are any other projects working on similar features,
>>> or if maintainers have any opinions or better ways to implement them, so
>>> I temporarily limit this feature to HSAKMT context only.
>>>
>>> I am waiting for everyone's opinions, please provide your thoughts.
>>
>> USERPTR should be relevant for anything Vulkan-related, like Venus and
>> native contexts. I expect that this new feature will work universally
>> good for all context types.
>>
>> In order to merge USERPTR support upstream, we at least will need to
>> prototype the guest USERPTR in one of native context driver to know that
>> it works. You'll need to post the whole set of host/guest USERPTR
>> patches including QEMU and etc, not just the kernel patches.
> 
> Ok totally understood, I will upload other components as soon as
> possiable, cause we only have limited resource for this matter, please
> understand the speed is relatively slow.
> And for Vulkan use case, yes it is very important, I will make some
> prototype implementation for it, and will test it locally. The
> implementation for USERPTR must comply with Vulakan's standard.
> Really thanks for your comments.

We aren't in rush, thanks for working on this feature.

-- 
Best regards,
Dmitry

Re: [PATCH v1 1/7] virtio-gpu api: add blob userptr resource

[Demi Marie Obenour]

On 3/26/25 10:46 AM, Dmitry Osipenko wrote:
> On 3/6/25 13:51, Huang, Honglei1 wrote:
>>
>> On 2025/3/1 5:21, Demi Marie Obenour wrote:
>>> On 2/28/25 12:36 AM, Honglei Huang wrote:
>>>> From: Honglei Huang <Honglei1.Huang@amd.com>
>>>>
>>>> Add a new resource for blob resource, called userptr, used for let
>>>> host access guest user space memory, to acquire buffer based userptr
>>>> feature in virtio GPU.
>>>>
>>>> - The capset VIRTIO_GPU_CAPSET_HSAKMT used for context init,
>>>> in this series patches only HSAKMT context can use the userptr
>>>> feature. HSAKMT is a GPU compute library in HSA stack, like
>>>> the role libdrm in mesa stack.
>>>
>>> Userptr should not be limited to HSMKMT contexts.  Userptr can
>>> accelerate shm buffers by avoiding a copy from guest to host, and
>>> it can be implemented using grant tables on Xen.
>>
>> Yes, I totally agree userptr can accelerate shm buffers, but I currently
>> don't know if there are any other projects working on similar features,
>> or if maintainers have any opinions or better ways to implement them, so
>> I temporarily limit this feature to HSAKMT context only.
>>
>> I am waiting for everyone's opinions, please provide your thoughts.
> 
> USERPTR should be relevant for anything Vulkan-related, like Venus and
> native contexts. I expect that this new feature will work universally
> good for all context types.
> 
> In order to merge USERPTR support upstream, we at least will need to
> prototype the guest USERPTR in one of native context driver to know that
> it works. You'll need to post the whole set of host/guest USERPTR
> patches including QEMU and etc, not just the kernel patches.

Does the user-mode VMM need to be QEMU or would patches to
another open-source VMM, such as crosvm, be sufficient?
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

Re: [PATCH v1 1/7] virtio-gpu api: add blob userptr resource

[Dmitry Osipenko]

On 3/29/25 22:50, Demi Marie Obenour wrote:
> On 3/26/25 10:46 AM, Dmitry Osipenko wrote:
>> On 3/6/25 13:51, Huang, Honglei1 wrote:
>>>
>>> On 2025/3/1 5:21, Demi Marie Obenour wrote:
>>>> On 2/28/25 12:36 AM, Honglei Huang wrote:
>>>>> From: Honglei Huang <Honglei1.Huang@amd.com>
>>>>>
>>>>> Add a new resource for blob resource, called userptr, used for let
>>>>> host access guest user space memory, to acquire buffer based userptr
>>>>> feature in virtio GPU.
>>>>>
>>>>> - The capset VIRTIO_GPU_CAPSET_HSAKMT used for context init,
>>>>> in this series patches only HSAKMT context can use the userptr
>>>>> feature. HSAKMT is a GPU compute library in HSA stack, like
>>>>> the role libdrm in mesa stack.
>>>>
>>>> Userptr should not be limited to HSMKMT contexts.  Userptr can
>>>> accelerate shm buffers by avoiding a copy from guest to host, and
>>>> it can be implemented using grant tables on Xen.
>>>
>>> Yes, I totally agree userptr can accelerate shm buffers, but I currently
>>> don't know if there are any other projects working on similar features,
>>> or if maintainers have any opinions or better ways to implement them, so
>>> I temporarily limit this feature to HSAKMT context only.
>>>
>>> I am waiting for everyone's opinions, please provide your thoughts.
>>
>> USERPTR should be relevant for anything Vulkan-related, like Venus and
>> native contexts. I expect that this new feature will work universally
>> good for all context types.
>>
>> In order to merge USERPTR support upstream, we at least will need to
>> prototype the guest USERPTR in one of native context driver to know that
>> it works. You'll need to post the whole set of host/guest USERPTR
>> patches including QEMU and etc, not just the kernel patches.
> 
> Does the user-mode VMM need to be QEMU or would patches to
> another open-source VMM, such as crosvm, be sufficient?

General-purpose features should be made supported by QEMU when possible.
Testing feedback for other VMMs will be very valuable too.

-- 
Best regards,
Dmitry

[PATCH v1 2/7] drm/virtgpu api: add blob userptr resource

[Honglei Huang]

From: Honglei Huang <Honglei1.Huang@amd.com>

This makes blob userptr resource available to guest userspace.

- Flag VIRTGPU_BLOB_FLAG_USE_USERPTR for guest userspace blob create,
enable this flag to indicate blob userptr resource create.
- Flag VIRTGPU_BLOB_FLAG_USERPTR_RDONLY used for read only userptr,
if not set then the userptr will be writeable.
- New parameter blob_userptr for bypass userspace memory address to
virtio GPU, like other userptr design, virtio GPU needs a userspace
memory for device access.

Used for userptr feature, in compute side, this feature is basic and
essential. Let device to access userspace memory directly instead of
copying.

Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
---
 include/uapi/drm/virtgpu_drm.h | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/include/uapi/drm/virtgpu_drm.h b/include/uapi/drm/virtgpu_drm.h
index c2ce71987e9b..071f31752721 100644
--- a/include/uapi/drm/virtgpu_drm.h
+++ b/include/uapi/drm/virtgpu_drm.h
@@ -179,13 +179,14 @@ struct drm_virtgpu_resource_create_blob {
 #define VIRTGPU_BLOB_FLAG_USE_MAPPABLE     0x0001
 #define VIRTGPU_BLOB_FLAG_USE_SHAREABLE    0x0002
 #define VIRTGPU_BLOB_FLAG_USE_CROSS_DEVICE 0x0004
+#define VIRTGPU_BLOB_FLAG_USE_USERPTR      0x0008
+#define VIRTGPU_BLOB_FLAG_USERPTR_RDONLY   0x0010
 	/* zero is invalid blob_mem */
 	__u32 blob_mem;
 	__u32 blob_flags;
 	__u32 bo_handle;
 	__u32 res_handle;
 	__u64 size;
-
 	/*
 	 * for 3D contexts with VIRTGPU_BLOB_MEM_HOST3D_GUEST and
 	 * VIRTGPU_BLOB_MEM_HOST3D otherwise, must be zero.
@@ -194,6 +195,7 @@ struct drm_virtgpu_resource_create_blob {
 	__u32 cmd_size;
 	__u64 cmd;
 	__u64 blob_id;
+	__u64 userptr;
 };
 
 #define VIRTGPU_CONTEXT_PARAM_CAPSET_ID       0x0001
-- 
2.34.1

[PATCH v1 3/7] drm/virtio: implement userptr: probe for the feature

[Honglei Huang]

From: Honglei Huang <Honglei1.Huang@amd.com>

Add probe code path for virtio gpu userptr.

Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
---
 drivers/gpu/drm/virtio/virtgpu_debugfs.c | 1 +
 drivers/gpu/drm/virtio/virtgpu_drv.c     | 1 +
 drivers/gpu/drm/virtio/virtgpu_drv.h     | 1 +
 drivers/gpu/drm/virtio/virtgpu_kms.c     | 8 ++++++--
 4 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_debugfs.c b/drivers/gpu/drm/virtio/virtgpu_debugfs.c
index 853dd9aa397e..da9fa034db0e 100644
--- a/drivers/gpu/drm/virtio/virtgpu_debugfs.c
+++ b/drivers/gpu/drm/virtio/virtgpu_debugfs.c
@@ -57,6 +57,7 @@ static int virtio_gpu_features(struct seq_file *m, void *data)
 	virtio_gpu_add_bool(m, "context init", vgdev->has_context_init);
 	virtio_gpu_add_int(m, "cap sets", vgdev->num_capsets);
 	virtio_gpu_add_int(m, "scanouts", vgdev->num_scanouts);
+	virtio_gpu_add_int(m, "blob userptr", vgdev->has_resource_userptr);
 	if (vgdev->host_visible_region.len) {
 		seq_printf(m, "%-16s : 0x%lx +0x%lx\n", "host visible region",
 			   (unsigned long)vgdev->host_visible_region.addr,
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.c b/drivers/gpu/drm/virtio/virtgpu_drv.c
index ffca6e2e1c9a..d79558139084 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.c
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.c
@@ -151,6 +151,7 @@ static unsigned int features[] = {
 	VIRTIO_GPU_F_RESOURCE_UUID,
 	VIRTIO_GPU_F_RESOURCE_BLOB,
 	VIRTIO_GPU_F_CONTEXT_INIT,
+	VIRTIO_GPU_F_RESOURCE_USERPTR,
 };
 static struct virtio_driver virtio_gpu_driver = {
 	.feature_table = features,
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
index 64c236169db8..7bdcbaa20ef1 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.h
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
@@ -249,6 +249,7 @@ struct virtio_gpu_device {
 	bool has_resource_blob;
 	bool has_host_visible;
 	bool has_context_init;
+	bool has_resource_userptr;
 	struct virtio_shm_region host_visible_region;
 	struct drm_mm host_visible_mm;
 
diff --git a/drivers/gpu/drm/virtio/virtgpu_kms.c b/drivers/gpu/drm/virtio/virtgpu_kms.c
index 7dfb2006c561..3d5158caef46 100644
--- a/drivers/gpu/drm/virtio/virtgpu_kms.c
+++ b/drivers/gpu/drm/virtio/virtgpu_kms.c
@@ -174,6 +174,9 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)
 	if (virtio_has_feature(vgdev->vdev, VIRTIO_GPU_F_RESOURCE_BLOB)) {
 		vgdev->has_resource_blob = true;
 	}
+	if (virtio_has_feature(vgdev->vdev, VIRTIO_GPU_F_RESOURCE_USERPTR)) {
+		vgdev->has_resource_userptr = true;
+	}
 	if (virtio_get_shm_region(vgdev->vdev, &vgdev->host_visible_region,
 				  VIRTIO_GPU_SHM_ID_HOST_VISIBLE)) {
 		if (!devm_request_mem_region(&vgdev->vdev->dev,
@@ -197,11 +200,12 @@ int virtio_gpu_init(struct virtio_device *vdev, struct drm_device *dev)
 		vgdev->has_context_init = true;
 	}
 
-	DRM_INFO("features: %cvirgl %cedid %cresource_blob %chost_visible",
+	DRM_INFO("features: %cvirgl %cedid %cresource_blob %chost_visible %cresource_userptr",
 		 vgdev->has_virgl_3d    ? '+' : '-',
 		 vgdev->has_edid        ? '+' : '-',
 		 vgdev->has_resource_blob ? '+' : '-',
-		 vgdev->has_host_visible ? '+' : '-');
+		 vgdev->has_host_visible ? '+' : '-',
+		 vgdev->has_resource_userptr ? '+' : '-');
 
 	DRM_INFO("features: %ccontext_init\n",
 		 vgdev->has_context_init ? '+' : '-');
-- 
2.34.1

[PATCH v1 4/7] drm/virtio: implement userptr: add userptr obj

[Honglei Huang]

From: Honglei Huang <Honglei1.Huang@amd.com>

Add implement for virtio gpu userptr. Current solution is pinning
all the user space memory. The UMD needs manage all the userptrs.

Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
---
 drivers/gpu/drm/virtio/Makefile          |   3 +-
 drivers/gpu/drm/virtio/virtgpu_drv.h     |  33 ++++
 drivers/gpu/drm/virtio/virtgpu_object.c  |   5 +
 drivers/gpu/drm/virtio/virtgpu_userptr.c | 230 +++++++++++++++++++++++
 4 files changed, 270 insertions(+), 1 deletion(-)
 create mode 100644 drivers/gpu/drm/virtio/virtgpu_userptr.c

diff --git a/drivers/gpu/drm/virtio/Makefile b/drivers/gpu/drm/virtio/Makefile
index d2e1788a8227..fe7332a621aa 100644
--- a/drivers/gpu/drm/virtio/Makefile
+++ b/drivers/gpu/drm/virtio/Makefile
@@ -6,6 +6,7 @@
 virtio-gpu-y := virtgpu_drv.o virtgpu_kms.o virtgpu_gem.o virtgpu_vram.o \
 	virtgpu_display.o virtgpu_vq.o \
 	virtgpu_fence.o virtgpu_object.o virtgpu_debugfs.o virtgpu_plane.o \
-	virtgpu_ioctl.o virtgpu_prime.o virtgpu_trace_points.o virtgpu_submit.o
+	virtgpu_ioctl.o virtgpu_prime.o virtgpu_trace_points.o virtgpu_submit.o \
+	virtgpu_userptr.o
 
 obj-$(CONFIG_DRM_VIRTIO_GPU) += virtio-gpu.o
diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
index 7bdcbaa20ef1..f3dcbd241f5a 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.h
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
@@ -85,6 +85,7 @@ struct virtio_gpu_object_params {
 	uint32_t blob_mem;
 	uint32_t blob_flags;
 	uint64_t blob_id;
+	uint64_t userptr;
 };
 
 struct virtio_gpu_object {
@@ -112,12 +113,38 @@ struct virtio_gpu_object_vram {
 	struct drm_mm_node vram_node;
 };
 
+struct virtio_gpu_object_userptr;
+
+struct virtio_gpu_object_userptr_ops {
+	int (*get_pages)(struct virtio_gpu_object_userptr *userptr);
+	void (*put_pages)(struct virtio_gpu_object_userptr *userptr);
+	void (*release)(struct virtio_gpu_object_userptr *userptr);
+};
+struct virtio_gpu_object_userptr {
+	struct virtio_gpu_object base;
+	const struct virtio_gpu_object_userptr_ops *ops;
+	struct mutex lock;
+
+	uint64_t start;
+	uint32_t npages;
+	uint32_t bo_handle;
+	uint32_t flags;
+
+	struct virtio_gpu_device *vgdev;
+	struct drm_file *file;
+	struct page **pages;
+	struct sg_table *sgt;
+};
+
 #define to_virtio_gpu_shmem(virtio_gpu_object) \
 	container_of((virtio_gpu_object), struct virtio_gpu_object_shmem, base)
 
 #define to_virtio_gpu_vram(virtio_gpu_object) \
 	container_of((virtio_gpu_object), struct virtio_gpu_object_vram, base)
 
+#define to_virtio_gpu_userptr(virtio_gpu_object) \
+	container_of((virtio_gpu_object), struct virtio_gpu_object_userptr, base)
+
 struct virtio_gpu_object_array {
 	struct ww_acquire_ctx ticket;
 	struct list_head next;
@@ -489,4 +516,10 @@ void virtio_gpu_vram_unmap_dma_buf(struct device *dev,
 int virtio_gpu_execbuffer_ioctl(struct drm_device *dev, void *data,
 				struct drm_file *file);
 
+/* virtgpu_userptr.c */
+int virtio_gpu_userptr_create(struct virtio_gpu_device *vgdev,
+			      struct drm_file *file,
+			      struct virtio_gpu_object_params *params,
+			      struct virtio_gpu_object **bo_ptr);
+bool virtio_gpu_is_userptr(struct virtio_gpu_object *bo);
 #endif
diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c b/drivers/gpu/drm/virtio/virtgpu_object.c
index c7e74cf13022..31659b0a028d 100644
--- a/drivers/gpu/drm/virtio/virtgpu_object.c
+++ b/drivers/gpu/drm/virtio/virtgpu_object.c
@@ -80,6 +80,11 @@ void virtio_gpu_cleanup_object(struct virtio_gpu_object *bo)
 		drm_gem_free_mmap_offset(&vram->base.base.base);
 		drm_gem_object_release(&vram->base.base.base);
 		kfree(vram);
+	} else if (virtio_gpu_is_userptr(bo)) {
+		struct virtio_gpu_object_userptr *userptr = to_virtio_gpu_userptr(bo);
+
+		drm_gem_object_release(&userptr->base.base.base);
+		kfree(userptr);
 	}
 }
 
diff --git a/drivers/gpu/drm/virtio/virtgpu_userptr.c b/drivers/gpu/drm/virtio/virtgpu_userptr.c
new file mode 100644
index 000000000000..b4a08811d345
--- /dev/null
+++ b/drivers/gpu/drm/virtio/virtgpu_userptr.c
@@ -0,0 +1,230 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/dma-mapping.h>
+#include <linux/mm.h>
+#include <linux/pid.h>
+#include <linux/vmalloc.h>
+
+#include "virtgpu_drv.h"
+#include "drm/drm_gem.h"
+
+static struct sg_table *
+virtio_gpu_userptr_get_sg_table(struct drm_gem_object *obj);
+
+static void virtio_gpu_userptr_free(struct drm_gem_object *obj)
+{
+	struct virtio_gpu_object *bo = gem_to_virtio_gpu_obj(obj);
+	struct virtio_gpu_device *vgdev = obj->dev->dev_private;
+	struct virtio_gpu_object_userptr *userptr = to_virtio_gpu_userptr(bo);
+
+	if (bo->created) {
+		userptr->ops->release(userptr);
+
+		virtio_gpu_cmd_unref_resource(vgdev, bo);
+		virtio_gpu_notify(vgdev);
+	}
+}
+
+static void virtio_gpu_userptr_object_close(struct drm_gem_object *obj,
+					    struct drm_file *file)
+{
+	virtio_gpu_gem_object_close(obj, file);
+}
+
+static const struct drm_gem_object_funcs virtio_gpu_userptr_funcs = {
+	.open = virtio_gpu_gem_object_open,
+	.close = virtio_gpu_userptr_object_close,
+	.free = virtio_gpu_userptr_free,
+	.get_sg_table = virtio_gpu_userptr_get_sg_table,
+};
+
+bool virtio_gpu_is_userptr(struct virtio_gpu_object *bo)
+{
+	return bo->base.base.funcs == &virtio_gpu_userptr_funcs;
+}
+
+static int
+virtio_gpu_userptr_get_pages(struct virtio_gpu_object_userptr *userptr)
+{
+	unsigned int flag = FOLL_LONGTERM;
+	unsigned int num_pages, pinned = 0;
+	int ret = 0;
+
+	if (userptr->pages)
+		return 0;
+
+	userptr->pages = kvmalloc_array(userptr->npages, sizeof(struct page *),
+					GFP_KERNEL);
+	if (!userptr->pages)
+		return -ENOMEM;
+
+	if (!(userptr->flags & VIRTGPU_BLOB_FLAG_USERPTR_RDONLY))
+		flag |= FOLL_WRITE;
+
+	do {
+		num_pages = userptr->npages - pinned;
+
+		ret = pin_user_pages_fast(userptr->start + pinned * PAGE_SIZE,
+					  num_pages, flag,
+					  userptr->pages + pinned);
+
+		if (ret < 0) {
+			if (pinned)
+				unpin_user_pages(userptr->pages, pinned);
+			kvfree(userptr->pages);
+			return ret;
+		}
+
+		pinned += ret;
+
+	} while (pinned < userptr->npages);
+
+	return 0;
+}
+
+static void
+virtio_gpu_userptr_put_pages(struct virtio_gpu_object_userptr *userptr)
+{
+	if (userptr->pages) {
+		unpin_user_pages(userptr->pages, userptr->npages);
+		kvfree(userptr->pages);
+		userptr->pages = NULL;
+	}
+
+	if (userptr->sgt) {
+		sg_free_table(userptr->sgt);
+		kfree(userptr->sgt);
+		userptr->sgt = NULL;
+	}
+}
+
+static void
+virtio_gpu_userptr_release(struct virtio_gpu_object_userptr *userptr)
+{
+	mutex_lock(&userptr->lock);
+	userptr->ops->put_pages(userptr);
+	mutex_unlock(&userptr->lock);
+}
+
+static struct sg_table *
+virtio_gpu_userptr_get_sg_table(struct drm_gem_object *obj)
+{
+	struct virtio_gpu_object *bo = gem_to_virtio_gpu_obj(obj);
+	struct virtio_gpu_object_userptr *userptr = to_virtio_gpu_userptr(bo);
+
+	mutex_lock(&userptr->lock);
+	if (!userptr->pages) {
+		if (userptr->ops->get_pages(userptr)) {
+			mutex_unlock(&userptr->lock);
+			return ERR_PTR(-ENOMEM);
+		}
+	}
+
+	if (!userptr->sgt)
+		userptr->sgt = drm_prime_pages_to_sg(NULL, userptr->pages,
+						     userptr->npages);
+	mutex_unlock(&userptr->lock);
+
+	return userptr->sgt;
+}
+
+static int
+virtio_gpu_userptr_init(struct drm_device *dev, struct drm_file *file,
+			struct virtio_gpu_object_userptr *userptr,
+			struct virtio_gpu_object_params *params,
+			const struct virtio_gpu_object_userptr_ops *ops)
+{
+	uint32_t page_offset;
+	uint64_t aligned_size;
+	uint64_t aligned_addr;
+	int ret;
+	struct drm_gem_object *obj;
+
+	page_offset = params->userptr & (PAGE_SIZE - 1UL);
+	aligned_addr = params->userptr - page_offset;
+	aligned_size = roundup(page_offset + params->size, PAGE_SIZE);
+
+	userptr->start = aligned_addr;
+	userptr->npages = aligned_size >> PAGE_SHIFT;
+	userptr->flags = params->blob_flags;
+
+	mutex_init(&userptr->lock);
+	userptr->vgdev = dev->dev_private;
+	userptr->file = file;
+	userptr->ops = ops;
+
+	obj = &userptr->base.base.base;
+	obj->funcs = &virtio_gpu_userptr_funcs;
+
+	drm_gem_private_object_init(dev, obj, aligned_size);
+
+	ret = virtio_gpu_resource_id_get(userptr->vgdev,
+					 &userptr->base.hw_res_handle);
+
+	return ret;
+}
+
+static const struct virtio_gpu_object_userptr_ops virtio_gpu_userptr_ops = {
+	.get_pages = virtio_gpu_userptr_get_pages,
+	.put_pages = virtio_gpu_userptr_put_pages,
+	.release = virtio_gpu_userptr_release,
+};
+
+int virtio_gpu_userptr_create(struct virtio_gpu_device *vgdev,
+			      struct drm_file *file,
+			      struct virtio_gpu_object_params *params,
+			      struct virtio_gpu_object **bo_ptr)
+{
+	struct virtio_gpu_object_userptr *userptr;
+	int ret, si;
+	struct sg_table *sgt;
+	struct scatterlist *sg;
+	struct virtio_gpu_mem_entry *ents;
+
+	if (!params->size)
+		return -EINVAL;
+
+	if (!access_ok((char __user *)(unsigned long)params->userptr,
+		       params->size))
+		return -EFAULT;
+
+	userptr = kzalloc(sizeof(*userptr), GFP_KERNEL);
+	if (!userptr)
+		return -ENOMEM;
+
+	ret = virtio_gpu_userptr_init(vgdev->ddev, file, userptr, params,
+				      &virtio_gpu_userptr_ops);
+
+	if (ret)
+		goto failed_free;
+
+	sgt = virtio_gpu_userptr_get_sg_table(&userptr->base.base.base);
+
+	if (IS_ERR(sgt)) {
+		ret = PTR_ERR(sgt);
+		goto failed_free;
+	}
+
+	ents = kvmalloc_array(sgt->nents, sizeof(struct virtio_gpu_mem_entry),
+			      GFP_KERNEL);
+
+	if (!ents) {
+		ret = -ENOMEM;
+		goto failed_free;
+	}
+
+	for_each_sgtable_sg(sgt, sg, si) {
+		(ents)[si].addr = cpu_to_le64(sg_phys(sg));
+		(ents)[si].length = cpu_to_le32(sg->length);
+		(ents)[si].padding = 0;
+	}
+
+	virtio_gpu_cmd_resource_create_blob(vgdev, &userptr->base, params, ents,
+					    sgt->nents);
+
+	*bo_ptr = &userptr->base;
+	return 0;
+
+failed_free:
+	kfree(userptr);
+	return ret;
+}
-- 
2.34.1

[PATCH v1 5/7] drm/virtio: advertise base userptr feature to userspace

[Honglei Huang]

From: Honglei Huang <Honglei1.Huang@amd.com>

Introduce the basic userptr feature to userspace.

Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
---
 drivers/gpu/drm/virtio/virtgpu_ioctl.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_ioctl.c b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
index e4f76f315550..8a89774d0737 100644
--- a/drivers/gpu/drm/virtio/virtgpu_ioctl.c
+++ b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
@@ -36,7 +36,9 @@
 
 #define VIRTGPU_BLOB_FLAG_USE_MASK (VIRTGPU_BLOB_FLAG_USE_MAPPABLE | \
 				    VIRTGPU_BLOB_FLAG_USE_SHAREABLE | \
-				    VIRTGPU_BLOB_FLAG_USE_CROSS_DEVICE)
+				    VIRTGPU_BLOB_FLAG_USE_CROSS_DEVICE | \
+				    VIRTGPU_BLOB_FLAG_USE_USERPTR | \
+				    VIRTGPU_BLOB_FLAG_USERPTR_RDONLY)
 
 /* Must be called with &virtio_gpu_fpriv.struct_mutex held. */
 static void virtio_gpu_create_context_locked(struct virtio_gpu_device *vgdev,
@@ -444,6 +446,8 @@ static int verify_blob(struct virtio_gpu_device *vgdev,
 {
 	if (!vgdev->has_resource_blob)
 		return -EINVAL;
+	if (!vgdev->has_resource_userptr && rc_blob->userptr)
+		return -EINVAL;
 
 	if (rc_blob->blob_flags & ~VIRTGPU_BLOB_FLAG_USE_MASK)
 		return -EINVAL;
@@ -489,6 +493,7 @@ static int verify_blob(struct virtio_gpu_device *vgdev,
 	params->size = rc_blob->size;
 	params->blob = true;
 	params->blob_flags = rc_blob->blob_flags;
+	params->userptr = rc_blob->userptr;
 	return 0;
 }
 
@@ -527,8 +532,10 @@ static int virtio_gpu_resource_create_blob_ioctl(struct drm_device *dev,
 				      vfpriv->ctx_id, NULL, NULL);
 	}
 
-	if (guest_blob)
+	if (guest_blob && !params.userptr)
 		ret = virtio_gpu_object_create(vgdev, &params, &bo, NULL);
+	else if (guest_blob && params.userptr)
+		ret = virtio_gpu_userptr_create(vgdev, file, &params, &bo);
 	else if (!guest_blob && host3d_blob)
 		ret = virtio_gpu_vram_create(vgdev, &params, &bo);
 	else
-- 
2.34.1

[PATCH v1 6/7] drm/virtio: implement userptr: add interval tree

[Honglei Huang]

From: Honglei Huang <Honglei1.Huang@amd.com>

Add interval tree to manage the userptrs to prevent repeat creation.
If the userptr exists, the ioctl will return the existing BO, and it's
offset with the create ioctl address.

Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
---
 drivers/gpu/drm/virtio/virtgpu_drv.h     |  16 ++-
 drivers/gpu/drm/virtio/virtgpu_ioctl.c   |  13 ++-
 drivers/gpu/drm/virtio/virtgpu_userptr.c | 129 ++++++++++++++++++++++-
 include/uapi/drm/virtgpu_drm.h           |   1 +
 4 files changed, 152 insertions(+), 7 deletions(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
index f3dcbd241f5a..fa5dd46e3732 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.h
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
@@ -54,6 +54,7 @@
 #define STATE_INITIALIZING 0
 #define STATE_OK 1
 #define STATE_ERR 2
+#define STATE_RES_EXISTS 3
 
 #define MAX_CAPSET_ID 63
 #define MAX_RINGS 64
@@ -114,18 +115,23 @@ struct virtio_gpu_object_vram {
 };
 
 struct virtio_gpu_object_userptr;
+struct virtio_gpu_fpriv;
 
 struct virtio_gpu_object_userptr_ops {
 	int (*get_pages)(struct virtio_gpu_object_userptr *userptr);
 	void (*put_pages)(struct virtio_gpu_object_userptr *userptr);
 	void (*release)(struct virtio_gpu_object_userptr *userptr);
+	int (*insert)(struct virtio_gpu_object_userptr *userptr, struct virtio_gpu_fpriv *fpriv);
+	int (*remove)(struct virtio_gpu_object_userptr *userptr, struct virtio_gpu_fpriv *fpriv);
 };
 struct virtio_gpu_object_userptr {
 	struct virtio_gpu_object base;
 	const struct virtio_gpu_object_userptr_ops *ops;
 	struct mutex lock;
 
+	uint64_t ptr;
 	uint64_t start;
+	uint64_t last;
 	uint32_t npages;
 	uint32_t bo_handle;
 	uint32_t flags;
@@ -134,6 +140,8 @@ struct virtio_gpu_object_userptr {
 	struct drm_file *file;
 	struct page **pages;
 	struct sg_table *sgt;
+
+	struct interval_tree_node it_node;
 };
 
 #define to_virtio_gpu_shmem(virtio_gpu_object) \
@@ -307,6 +315,8 @@ struct virtio_gpu_fpriv {
 	struct mutex context_lock;
 	char debug_name[DEBUG_NAME_MAX_LEN];
 	bool explicit_debug_name;
+	struct rb_root_cached userptrs_tree;
+	struct mutex userptrs_tree_lock;
 };
 
 /* virtgpu_ioctl.c */
@@ -520,6 +530,10 @@ int virtio_gpu_execbuffer_ioctl(struct drm_device *dev, void *data,
 int virtio_gpu_userptr_create(struct virtio_gpu_device *vgdev,
 			      struct drm_file *file,
 			      struct virtio_gpu_object_params *params,
-			      struct virtio_gpu_object **bo_ptr);
+			      struct virtio_gpu_object **bo_ptr,
+			      struct drm_virtgpu_resource_create_blob *rc_blob);
 bool virtio_gpu_is_userptr(struct virtio_gpu_object *bo);
+void virtio_gpu_userptr_interval_tree_init(struct virtio_gpu_fpriv *vfpriv);
+void virtio_gpu_userptr_set_handle(struct virtio_gpu_object *qobj,
+				   uint32_t handle);
 #endif
diff --git a/drivers/gpu/drm/virtio/virtgpu_ioctl.c b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
index 8a89774d0737..ad1ac8d0eadf 100644
--- a/drivers/gpu/drm/virtio/virtgpu_ioctl.c
+++ b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
@@ -534,8 +534,11 @@ static int virtio_gpu_resource_create_blob_ioctl(struct drm_device *dev,
 
 	if (guest_blob && !params.userptr)
 		ret = virtio_gpu_object_create(vgdev, &params, &bo, NULL);
-	else if (guest_blob && params.userptr)
-		ret = virtio_gpu_userptr_create(vgdev, file, &params, &bo);
+	else if (guest_blob && params.userptr) {
+		ret = virtio_gpu_userptr_create(vgdev, file, &params, &bo, rc_blob);
+		if (ret > 0)
+			return ret;
+	}
 	else if (!guest_blob && host3d_blob)
 		ret = virtio_gpu_vram_create(vgdev, &params, &bo);
 	else
@@ -567,6 +570,9 @@ static int virtio_gpu_resource_create_blob_ioctl(struct drm_device *dev,
 	rc_blob->res_handle = bo->hw_res_handle;
 	rc_blob->bo_handle = handle;
 
+	if (guest_blob && params.userptr)
+		virtio_gpu_userptr_set_handle(bo, handle);
+
 	/*
 	 * The handle owns the reference now.  But we must drop our
 	 * remaining reference *after* we no longer need to dereference
@@ -691,6 +697,9 @@ static int virtio_gpu_context_init_ioctl(struct drm_device *dev,
 		}
 	}
 
+	if (vfpriv->context_init & VIRTIO_GPU_CAPSET_HSAKMT)
+		virtio_gpu_userptr_interval_tree_init(vfpriv);
+
 	virtio_gpu_create_context_locked(vgdev, vfpriv);
 	virtio_gpu_notify(vgdev);
 
diff --git a/drivers/gpu/drm/virtio/virtgpu_userptr.c b/drivers/gpu/drm/virtio/virtgpu_userptr.c
index b4a08811d345..03398c3b9f30 100644
--- a/drivers/gpu/drm/virtio/virtgpu_userptr.c
+++ b/drivers/gpu/drm/virtio/virtgpu_userptr.c
@@ -10,6 +10,92 @@
 static struct sg_table *
 virtio_gpu_userptr_get_sg_table(struct drm_gem_object *obj);
 
+static int virtio_gpu_userptr_insert(struct virtio_gpu_object_userptr *userptr,
+				     struct virtio_gpu_fpriv *vfpriv)
+{
+	if (!userptr->ops->insert)
+		return -EINVAL;
+
+	return userptr->ops->insert(userptr, vfpriv);
+}
+
+static int virtio_gpu_userptr_remove(struct virtio_gpu_object_userptr *userptr,
+				     struct virtio_gpu_fpriv *vfpriv)
+{
+	if (!userptr->ops->remove)
+		return -EINVAL;
+
+	return userptr->ops->remove(userptr, vfpriv);
+}
+
+static uint64_t virtio_gpu_userptr_get_offset(struct virtio_gpu_object *qobj,
+					      uint64_t addr)
+{
+	struct virtio_gpu_object_userptr *userptr = to_virtio_gpu_userptr(qobj);
+
+	return PAGE_ALIGN_DOWN(addr) - PAGE_ALIGN_DOWN(userptr->ptr);
+}
+
+static struct virtio_gpu_object_userptr *
+virtio_gpu_userptr_from_addr_range(struct virtio_gpu_fpriv *vfpriv,
+				   u_int64_t start, u_int64_t last)
+{
+	struct interval_tree_node *node;
+	struct virtio_gpu_object_userptr *userptr = NULL;
+	struct virtio_gpu_object_userptr *ret = NULL;
+
+	node = interval_tree_iter_first(&vfpriv->userptrs_tree, start, last);
+
+	while (node) {
+		struct interval_tree_node *next;
+
+		userptr = container_of(node, struct virtio_gpu_object_userptr,
+				       it_node);
+
+		if (start >= userptr->start && last <= userptr->last) {
+			ret = userptr;
+			return ret;
+		}
+
+		next = interval_tree_iter_next(node, start, last);
+		node = next;
+	}
+
+	return ret;
+}
+
+static int virtio_gpu_userptr_insert_interval_tree(
+	struct virtio_gpu_object_userptr *userptr,
+	struct virtio_gpu_fpriv *vfpriv)
+{
+	if (userptr->it_node.start != 0 && userptr->it_node.last != 0) {
+		userptr->it_node.start = userptr->start;
+		userptr->it_node.last = userptr->last;
+		interval_tree_insert(&userptr->it_node, &vfpriv->userptrs_tree);
+		return 0;
+	} else
+		return -EINVAL;
+}
+
+static int virtio_gpu_userptr_remove_interval_tree(
+	struct virtio_gpu_object_userptr *userptr,
+	struct virtio_gpu_fpriv *vfpriv)
+{
+	if (userptr->it_node.start != 0 && userptr->it_node.last != 0) {
+		interval_tree_remove(&userptr->it_node, &vfpriv->userptrs_tree);
+		return 0;
+	} else
+		return -EINVAL;
+}
+
+void virtio_gpu_userptr_set_handle(struct virtio_gpu_object *qobj,
+				   uint32_t handle)
+{
+	struct virtio_gpu_object_userptr *userptr = to_virtio_gpu_userptr(qobj);
+
+	userptr->bo_handle = handle;
+}
+
 static void virtio_gpu_userptr_free(struct drm_gem_object *obj)
 {
 	struct virtio_gpu_object *bo = gem_to_virtio_gpu_obj(obj);
@@ -27,6 +113,11 @@ static void virtio_gpu_userptr_free(struct drm_gem_object *obj)
 static void virtio_gpu_userptr_object_close(struct drm_gem_object *obj,
 					    struct drm_file *file)
 {
+	struct virtio_gpu_object *bo = gem_to_virtio_gpu_obj(obj);
+	struct virtio_gpu_object_userptr *userptr = to_virtio_gpu_userptr(bo);
+
+	virtio_gpu_userptr_remove(userptr, file->driver_priv);
+
 	virtio_gpu_gem_object_close(obj, file);
 }
 
@@ -63,9 +154,9 @@ virtio_gpu_userptr_get_pages(struct virtio_gpu_object_userptr *userptr)
 	do {
 		num_pages = userptr->npages - pinned;
 
-		ret = pin_user_pages_fast(userptr->start + pinned * PAGE_SIZE,
-					  num_pages, flag,
-					  userptr->pages + pinned);
+		ret = pin_user_pages_fast(
+			PAGE_ALIGN_DOWN(userptr->start) + pinned * PAGE_SIZE,
+			num_pages, flag, userptr->pages + pinned);
 
 		if (ret < 0) {
 			if (pinned)
@@ -127,6 +218,12 @@ virtio_gpu_userptr_get_sg_table(struct drm_gem_object *obj)
 	return userptr->sgt;
 }
 
+void virtio_gpu_userptr_interval_tree_init(struct virtio_gpu_fpriv *vfpriv)
+{
+	vfpriv->userptrs_tree = RB_ROOT_CACHED;
+	mutex_init(&vfpriv->userptrs_tree_lock);
+}
+
 static int
 virtio_gpu_userptr_init(struct drm_device *dev, struct drm_file *file,
 			struct virtio_gpu_object_userptr *userptr,
@@ -144,6 +241,8 @@ virtio_gpu_userptr_init(struct drm_device *dev, struct drm_file *file,
 	aligned_size = roundup(page_offset + params->size, PAGE_SIZE);
 
 	userptr->start = aligned_addr;
+	userptr->last = aligned_addr + aligned_size - 1UL;
+	userptr->ptr = params->userptr;
 	userptr->npages = aligned_size >> PAGE_SHIFT;
 	userptr->flags = params->blob_flags;
 
@@ -167,13 +266,17 @@ static const struct virtio_gpu_object_userptr_ops virtio_gpu_userptr_ops = {
 	.get_pages = virtio_gpu_userptr_get_pages,
 	.put_pages = virtio_gpu_userptr_put_pages,
 	.release = virtio_gpu_userptr_release,
+	.insert = virtio_gpu_userptr_insert_interval_tree,
+	.remove = virtio_gpu_userptr_remove_interval_tree,
 };
 
 int virtio_gpu_userptr_create(struct virtio_gpu_device *vgdev,
 			      struct drm_file *file,
 			      struct virtio_gpu_object_params *params,
-			      struct virtio_gpu_object **bo_ptr)
+			      struct virtio_gpu_object **bo_ptr,
+			      struct drm_virtgpu_resource_create_blob *rc_blob)
 {
+	struct virtio_gpu_fpriv *vfpriv = file->driver_priv;
 	struct virtio_gpu_object_userptr *userptr;
 	int ret, si;
 	struct sg_table *sgt;
@@ -187,6 +290,20 @@ int virtio_gpu_userptr_create(struct virtio_gpu_device *vgdev,
 		       params->size))
 		return -EFAULT;
 
+	mutex_lock(&vfpriv->userptrs_tree_lock);
+
+	userptr = virtio_gpu_userptr_from_addr_range(
+		vfpriv, params->userptr, params->userptr + params->size - 1UL);
+	if (userptr) {
+		*bo_ptr = &userptr->base;
+		rc_blob->res_handle = userptr->base.hw_res_handle;
+		rc_blob->bo_handle = userptr->bo_handle;
+		rc_blob->offset = virtio_gpu_userptr_get_offset(
+			&userptr->base, rc_blob->userptr);
+		mutex_unlock(&vfpriv->userptrs_tree_lock);
+		return STATE_RES_EXISTS;
+	}
+
 	userptr = kzalloc(sizeof(*userptr), GFP_KERNEL);
 	if (!userptr)
 		return -ENOMEM;
@@ -218,6 +335,9 @@ int virtio_gpu_userptr_create(struct virtio_gpu_device *vgdev,
 		(ents)[si].padding = 0;
 	}
 
+	virtio_gpu_userptr_insert(userptr, vfpriv);
+	mutex_unlock(&vfpriv->userptrs_tree_lock);
+
 	virtio_gpu_cmd_resource_create_blob(vgdev, &userptr->base, params, ents,
 					    sgt->nents);
 
@@ -225,6 +345,7 @@ int virtio_gpu_userptr_create(struct virtio_gpu_device *vgdev,
 	return 0;
 
 failed_free:
+	mutex_unlock(&vfpriv->userptrs_tree_lock);
 	kfree(userptr);
 	return ret;
 }
diff --git a/include/uapi/drm/virtgpu_drm.h b/include/uapi/drm/virtgpu_drm.h
index 071f31752721..07c22cf1a9e0 100644
--- a/include/uapi/drm/virtgpu_drm.h
+++ b/include/uapi/drm/virtgpu_drm.h
@@ -196,6 +196,7 @@ struct drm_virtgpu_resource_create_blob {
 	__u64 cmd;
 	__u64 blob_id;
 	__u64 userptr;
+	__u64 offset;
 };
 
 #define VIRTGPU_CONTEXT_PARAM_CAPSET_ID       0x0001
-- 
2.34.1

[PATCH v1 7/7] drm/virtio: implement userptr: add mmu notifier

[Honglei Huang]

From: Honglei Huang <Honglei1.Huang@amd.com>

Add mmu notifier, there are some benefits:
- UMD do not need manage the userptrs, just alloc and free user space
memory, with the MMU notifier userpters can be managed by kernel.
- Can achieve a performance improvement of 20%~30%. With the MMU notifier
UMD like OpenCL can achieve 98% performance compared to bare metal in
some bench marks like Geekbench and CLpeak.

Signed-off-by: Honglei Huang <Honglei1.Huang@amd.com>
---
 drivers/gpu/drm/virtio/virtgpu_drv.h     |  47 ++-
 drivers/gpu/drm/virtio/virtgpu_ioctl.c   |   4 +-
 drivers/gpu/drm/virtio/virtgpu_kms.c     |   2 +
 drivers/gpu/drm/virtio/virtgpu_userptr.c | 423 ++++++++++++++++++++++-
 4 files changed, 469 insertions(+), 7 deletions(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
index fa5dd46e3732..6fa6dd9d1738 100644
--- a/drivers/gpu/drm/virtio/virtgpu_drv.h
+++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
@@ -42,6 +42,7 @@
 #include <drm/drm_ioctl.h>
 #include <drm/drm_probe_helper.h>
 #include <drm/virtgpu_drm.h>
+#include <linux/mmu_notifier.h>
 
 #define DRIVER_NAME "virtio_gpu"
 #define DRIVER_DESC "virtio GPU"
@@ -121,9 +122,33 @@ struct virtio_gpu_object_userptr_ops {
 	int (*get_pages)(struct virtio_gpu_object_userptr *userptr);
 	void (*put_pages)(struct virtio_gpu_object_userptr *userptr);
 	void (*release)(struct virtio_gpu_object_userptr *userptr);
-	int (*insert)(struct virtio_gpu_object_userptr *userptr, struct virtio_gpu_fpriv *fpriv);
-	int (*remove)(struct virtio_gpu_object_userptr *userptr, struct virtio_gpu_fpriv *fpriv);
+	int (*insert)(struct virtio_gpu_object_userptr *userptr,
+		      struct virtio_gpu_fpriv *fpriv);
+	int (*remove)(struct virtio_gpu_object_userptr *userptr,
+		      struct virtio_gpu_fpriv *fpriv);
+	bool (*valid)(struct virtio_gpu_object_userptr *userptr);
+	void (*notifier_init)(struct virtio_gpu_object_userptr *userptr,
+			      struct mm_struct *mm);
+	int (*notifier_add)(struct virtio_gpu_object_userptr *userptr,
+			    unsigned long start, unsigned long length);
+	void (*notifier_remove)(struct virtio_gpu_object_userptr *userptr);
+	int (*split)(struct virtio_gpu_object_userptr *userptr,
+		     unsigned long start, unsigned long last,
+		     struct virtio_gpu_object_userptr **pnew);
+	void (*evict)(struct virtio_gpu_object_userptr *userptr);
+	void (*update)(struct virtio_gpu_object_userptr *userptr);
+	struct virtio_gpu_object_userptr *(*split_new)(
+		struct virtio_gpu_object_userptr *userptr, unsigned long start,
+		unsigned long last);
 };
+
+enum userptr_work_list_ops {
+	USERPTR_OP_NULL,
+	USERPTR_OP_UNMAP,
+	USERPTR_OP_UPDATE,
+	USERPTR_OP_EVICT,
+};
+
 struct virtio_gpu_object_userptr {
 	struct virtio_gpu_object base;
 	const struct virtio_gpu_object_userptr_ops *ops;
@@ -142,6 +167,16 @@ struct virtio_gpu_object_userptr {
 	struct sg_table *sgt;
 
 	struct interval_tree_node it_node;
+
+#ifdef CONFIG_MMU_NOTIFIER
+	struct list_head work_list;
+	enum userptr_work_list_ops op;
+	atomic_t in_release;
+	struct mm_struct *mm;
+	uint64_t notifier_start;
+	uint64_t notifier_last;
+	struct mmu_interval_notifier notifier;
+#endif
 };
 
 #define to_virtio_gpu_shmem(virtio_gpu_object) \
@@ -317,6 +352,12 @@ struct virtio_gpu_fpriv {
 	bool explicit_debug_name;
 	struct rb_root_cached userptrs_tree;
 	struct mutex userptrs_tree_lock;
+
+#ifdef CONFIG_MMU_NOTIFIER
+	struct work_struct userptr_work;
+	struct list_head userptr_work_list;
+	spinlock_t userptr_work_list_lock;
+#endif
 };
 
 /* virtgpu_ioctl.c */
@@ -536,4 +577,6 @@ bool virtio_gpu_is_userptr(struct virtio_gpu_object *bo);
 void virtio_gpu_userptr_interval_tree_init(struct virtio_gpu_fpriv *vfpriv);
 void virtio_gpu_userptr_set_handle(struct virtio_gpu_object *qobj,
 				   uint32_t handle);
+uint32_t virtio_gpu_userptr_get_handle(struct virtio_gpu_object *qobj);
+void virtio_gpu_userptr_list_work_init(struct virtio_gpu_fpriv *vfpriv);
 #endif
diff --git a/drivers/gpu/drm/virtio/virtgpu_ioctl.c b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
index ad1ac8d0eadf..14326fd8fee9 100644
--- a/drivers/gpu/drm/virtio/virtgpu_ioctl.c
+++ b/drivers/gpu/drm/virtio/virtgpu_ioctl.c
@@ -697,8 +697,10 @@ static int virtio_gpu_context_init_ioctl(struct drm_device *dev,
 		}
 	}
 
-	if (vfpriv->context_init & VIRTIO_GPU_CAPSET_HSAKMT)
+	if (vfpriv->context_init & VIRTIO_GPU_CAPSET_HSAKMT) {
+		virtio_gpu_userptr_list_work_init(vfpriv);
 		virtio_gpu_userptr_interval_tree_init(vfpriv);
+	}
 
 	virtio_gpu_create_context_locked(vgdev, vfpriv);
 	virtio_gpu_notify(vgdev);
diff --git a/drivers/gpu/drm/virtio/virtgpu_kms.c b/drivers/gpu/drm/virtio/virtgpu_kms.c
index 3d5158caef46..3dc44eb16fb8 100644
--- a/drivers/gpu/drm/virtio/virtgpu_kms.c
+++ b/drivers/gpu/drm/virtio/virtgpu_kms.c
@@ -345,6 +345,8 @@ void virtio_gpu_driver_postclose(struct drm_device *dev, struct drm_file *file)
 		return;
 
 	if (vfpriv->context_created) {
+		if (vfpriv->context_init & VIRTIO_GPU_CAPSET_HSAKMT)
+			flush_work(&vfpriv->userptr_work);
 		virtio_gpu_cmd_context_destroy(vgdev, vfpriv->ctx_id);
 		virtio_gpu_notify(vgdev);
 	}
diff --git a/drivers/gpu/drm/virtio/virtgpu_userptr.c b/drivers/gpu/drm/virtio/virtgpu_userptr.c
index 03398c3b9f30..10264227f3e7 100644
--- a/drivers/gpu/drm/virtio/virtgpu_userptr.c
+++ b/drivers/gpu/drm/virtio/virtgpu_userptr.c
@@ -3,6 +3,7 @@
 #include <linux/mm.h>
 #include <linux/pid.h>
 #include <linux/vmalloc.h>
+#include <linux/mmu_notifier.h>
 
 #include "virtgpu_drv.h"
 #include "drm/drm_gem.h"
@@ -13,21 +14,422 @@ virtio_gpu_userptr_get_sg_table(struct drm_gem_object *obj);
 static int virtio_gpu_userptr_insert(struct virtio_gpu_object_userptr *userptr,
 				     struct virtio_gpu_fpriv *vfpriv)
 {
+	int ret;
+
 	if (!userptr->ops->insert)
 		return -EINVAL;
 
-	return userptr->ops->insert(userptr, vfpriv);
+	ret = userptr->ops->insert(userptr, vfpriv);
+	if (ret)
+		return ret;
+
+	if (userptr->ops->notifier_add)
+		ret = userptr->ops->notifier_add(userptr, userptr->start,
+						 userptr->last -
+							 userptr->start + 1UL);
+
+	return ret;
 }
 
 static int virtio_gpu_userptr_remove(struct virtio_gpu_object_userptr *userptr,
 				     struct virtio_gpu_fpriv *vfpriv)
 {
+	int ret;
+
 	if (!userptr->ops->remove)
 		return -EINVAL;
 
-	return userptr->ops->remove(userptr, vfpriv);
+	ret = userptr->ops->remove(userptr, vfpriv);
+	if (ret)
+		return ret;
+
+	if (userptr->ops->notifier_remove)
+		userptr->ops->notifier_remove(userptr);
+
+	return ret;
+}
+
+static bool virtio_gpu_userptr_valid(struct virtio_gpu_object_userptr *userptr)
+{
+	if (userptr->ops->valid)
+		return userptr->ops->valid(userptr);
+
+	return true;
+}
+
+#ifdef CONFIG_MMU_NOTIFIER
+
+static bool
+virtio_gpu_userptr_invalidate(struct mmu_interval_notifier *mn,
+			      const struct mmu_notifier_range *range,
+			      unsigned long cur_seq);
+
+static const struct mmu_interval_notifier_ops virtio_gpu_userptr_mn_ops = {
+	.invalidate = virtio_gpu_userptr_invalidate,
+};
+
+static int
+virtio_gpu_userptr_add_notifier(struct virtio_gpu_object_userptr *userptr,
+				unsigned long start, unsigned long length)
+{
+	if (!start || !length)
+		return -EINVAL;
+
+	return mmu_interval_notifier_insert(&userptr->notifier, userptr->mm,
+					    start, length,
+					    &virtio_gpu_userptr_mn_ops);
+}
+
+static void
+virtio_gpu_userptr_remove_notifier(struct virtio_gpu_object_userptr *userptr)
+{
+	mmu_interval_notifier_remove(&userptr->notifier);
+}
+
+static void virtio_gpu_userptr_unmap(struct virtio_gpu_object_userptr *userptr)
+{
+	drm_gem_handle_delete(userptr->file, userptr->bo_handle);
+}
+
+static void virtio_gpu_userptr_update_notifier_and_interval_tree(
+	struct virtio_gpu_object_userptr *userptr)
+{
+	unsigned long start = userptr->notifier.interval_tree.start;
+	unsigned long last = userptr->notifier.interval_tree.last;
+
+	if (userptr->start == start && userptr->last == last)
+		return;
+
+	if (start != 0 && last != 0)
+		virtio_gpu_userptr_remove(userptr, userptr->file->driver_priv);
+
+	virtio_gpu_userptr_insert(userptr, userptr->file->driver_priv);
+	userptr->op = 0;
 }
 
+static int virtio_gpu_userptr_split(struct virtio_gpu_object_userptr *userptr,
+				    unsigned long valid_start,
+				    unsigned long valid_last,
+				    struct virtio_gpu_object_userptr **new)
+{
+	uint64_t old_start = userptr->start;
+	uint64_t old_last = userptr->last;
+
+	if (old_start != valid_start && old_last != valid_last)
+		return -EINVAL;
+	if (valid_start < old_start || valid_last > old_last)
+		return -EINVAL;
+
+	if (userptr->ops->split_new)
+		*new = userptr->ops->split_new(userptr, valid_start,
+					       valid_last);
+
+	userptr->start = valid_start;
+	userptr->last = valid_last;
+
+	return 0;
+}
+
+static void
+virtio_gpu_userptr_update_split(struct virtio_gpu_object_userptr *userptr,
+				unsigned long mn_start, unsigned long mn_last)
+{
+	struct virtio_gpu_object_userptr *head;
+	struct virtio_gpu_object_userptr *tail;
+
+	if (!userptr->ops->split)
+		return;
+	if (userptr->op == USERPTR_OP_UNMAP)
+		return;
+
+	if (mn_start > userptr->last || mn_last < userptr->start)
+		return;
+
+	head = tail = userptr;
+	if (mn_start > userptr->start)
+		userptr->ops->split(userptr, userptr->start, mn_start - 1UL,
+				    &tail);
+	else if (mn_last < userptr->last)
+		userptr->ops->split(userptr, mn_last + 1UL, userptr->last,
+				    &head);
+}
+
+static void
+virtio_gpu_userptr_add_list_work(struct virtio_gpu_object_userptr *userptr,
+				 int op)
+{
+	struct virtio_gpu_fpriv *vfpriv = userptr->file->driver_priv;
+
+	spin_lock(&vfpriv->userptr_work_list_lock);
+
+	if (!list_empty(&userptr->work_list)) {
+		if (op != USERPTR_OP_NULL && userptr->op != USERPTR_OP_UNMAP)
+			userptr->op = op;
+	} else {
+		userptr->op = op;
+		list_add_tail(&userptr->work_list, &vfpriv->userptr_work_list);
+	}
+
+	spin_unlock(&vfpriv->userptr_work_list_lock);
+}
+
+static int virtio_gpu_follow_pfn(struct vm_area_struct *vma, uint64_t addr,
+				 unsigned long *pfn)
+{
+	struct follow_pfnmap_args args = { .vma = vma, .address = addr };
+
+	if (follow_pfnmap_start(&args))
+		return -EINVAL;
+
+	*pfn = args.pfn;
+	follow_pfnmap_end(&args);
+
+	return 0;
+}
+
+static int virtio_gpu_userptr_check(struct virtio_gpu_object_userptr *userptr,
+				    struct vm_area_struct *vma, uint64_t start,
+				    uint64_t end)
+{
+	uint64_t addr;
+	int ret;
+	unsigned long pfn;
+
+	for (addr = start; addr < end; addr += PAGE_SIZE) {
+		ret = virtio_gpu_follow_pfn(vma, addr, &pfn);
+		if (ret)
+			return -EINVAL;
+
+		if (page_to_pfn(userptr->pages[(addr - userptr->start) >>
+					       PAGE_SHIFT]) != pfn)
+			return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int
+virtio_gpu_userptr_check_range(struct virtio_gpu_object_userptr *userptr,
+			       uint64_t notifier_start, uint64_t notifier_last)
+{
+	uint64_t start, end, addr;
+	int r = 0;
+
+	start = notifier_start;
+	end = notifier_last + (1UL << PAGE_SHIFT);
+
+	for (addr = start; !r && addr < end;) {
+		struct vm_area_struct *vma;
+		uint64_t next = 0;
+
+		vma = vma_lookup(userptr->mm, addr);
+
+		if (vma) {
+			next = min(vma->vm_end, end);
+			r = virtio_gpu_userptr_check(userptr, vma, start, next);
+			if (r)
+				break;
+		} else {
+			r = -EFAULT;
+			break;
+		}
+
+		addr = next;
+	}
+
+	return r;
+}
+
+static void
+virtio_gpu_update_or_remove_userptr(struct virtio_gpu_object_userptr *userptr,
+				    unsigned long start, unsigned long last)
+{
+	if ((userptr->start) >= start && (userptr->last) <= last) {
+		if (atomic_xchg(&userptr->in_release, 1) == 0) {
+			virtio_gpu_userptr_add_list_work(userptr,
+							 USERPTR_OP_UNMAP);
+		}
+	} else {
+		virtio_gpu_userptr_update_split(userptr, start, last);
+		virtio_gpu_userptr_add_list_work(userptr, USERPTR_OP_UPDATE);
+	}
+}
+
+static void virtio_gpu_userptr_evict(struct virtio_gpu_object_userptr *userptr)
+{
+	if (!userptr->notifier_start || !userptr->notifier_last)
+		return;
+
+	if (userptr->notifier_start < userptr->start ||
+	    userptr->notifier_last > userptr->last)
+		return;
+
+	if (virtio_gpu_userptr_check_range(userptr, userptr->notifier_start,
+					   userptr->notifier_last)) {
+		virtio_gpu_update_or_remove_userptr(
+			userptr, userptr->notifier_start,
+			userptr->notifier_last + (1UL << PAGE_SHIFT) - 1UL);
+	}
+
+	userptr->notifier_start = 0;
+	userptr->notifier_last = 0;
+}
+
+static void
+virtio_gpu_userptr_handle_list_work(struct virtio_gpu_object_userptr *userptr)
+{
+	switch (userptr->op) {
+	case USERPTR_OP_NULL:
+		break;
+	case USERPTR_OP_UNMAP:
+		virtio_gpu_userptr_unmap(userptr);
+		break;
+	case USERPTR_OP_UPDATE:
+		if (userptr->ops->update)
+			userptr->ops->update(userptr);
+		break;
+	case USERPTR_OP_EVICT:
+		if (userptr->ops->evict)
+			userptr->ops->evict(userptr);
+		break;
+	default:
+		break;
+	}
+}
+
+static void virtio_gpu_userptr_invalidate_work(struct work_struct *work)
+{
+	struct virtio_gpu_fpriv *vfpriv;
+	struct virtio_gpu_object_userptr *userptr;
+
+	vfpriv = container_of(work, struct virtio_gpu_fpriv, userptr_work);
+
+	spin_lock(&vfpriv->userptr_work_list_lock);
+	while (!list_empty(&vfpriv->userptr_work_list)) {
+		userptr = list_first_entry(&vfpriv->userptr_work_list,
+					   struct virtio_gpu_object_userptr,
+					   work_list);
+		spin_unlock(&vfpriv->userptr_work_list_lock);
+
+		mmap_write_lock(userptr->mm);
+
+		spin_lock(&vfpriv->userptr_work_list_lock);
+		list_del_init(&userptr->work_list);
+		spin_unlock(&vfpriv->userptr_work_list_lock);
+
+		mutex_lock(&vfpriv->userptrs_tree_lock);
+
+		virtio_gpu_userptr_handle_list_work(userptr);
+
+		mutex_unlock(&vfpriv->userptrs_tree_lock);
+		mmap_write_unlock(userptr->mm);
+
+		spin_lock(&vfpriv->userptr_work_list_lock);
+	}
+	spin_unlock(&vfpriv->userptr_work_list_lock);
+}
+
+void virtio_gpu_userptr_list_work_init(struct virtio_gpu_fpriv *vfpriv)
+{
+	INIT_WORK(&vfpriv->userptr_work, virtio_gpu_userptr_invalidate_work);
+	INIT_LIST_HEAD(&vfpriv->userptr_work_list);
+	spin_lock_init(&vfpriv->userptr_work_list_lock);
+}
+
+static void
+virtio_gpu_userptr_schedule_list_work(struct virtio_gpu_fpriv *vfpriv)
+{
+	spin_lock(&vfpriv->userptr_work_list_lock);
+	if (!list_empty(&vfpriv->userptr_work_list))
+		schedule_work(&vfpriv->userptr_work);
+	spin_unlock(&vfpriv->userptr_work_list_lock);
+}
+
+static bool
+virtio_gpu_userptr_invalidate(struct mmu_interval_notifier *mn,
+			      const struct mmu_notifier_range *range,
+			      unsigned long cur_seq)
+{
+	struct virtio_gpu_object_userptr *userptr;
+	unsigned long start;
+	unsigned long last;
+
+	if (range->event == MMU_NOTIFY_RELEASE)
+		return true;
+	if (!mmget_not_zero(mn->mm))
+		return true;
+
+	start = mn->interval_tree.start;
+	last = mn->interval_tree.last;
+	start = (max(start, range->start) >> PAGE_SHIFT) << PAGE_SHIFT;
+	last = (min(last, range->end - 1UL) >> PAGE_SHIFT) << PAGE_SHIFT;
+
+	userptr = container_of(mn, struct virtio_gpu_object_userptr, notifier);
+	userptr->mm = mn->mm;
+
+	mutex_lock(&userptr->lock);
+	mmu_interval_set_seq(mn, cur_seq);
+
+	if (userptr->op != USERPTR_OP_UNMAP) {
+		switch (range->event) {
+		case MMU_NOTIFY_UNMAP:
+			virtio_gpu_update_or_remove_userptr(
+				userptr, start,
+				last + (1UL << PAGE_SHIFT) - 1UL);
+			break;
+		default:
+			userptr->notifier_start = start;
+			userptr->notifier_last = last;
+			virtio_gpu_userptr_add_list_work(userptr,
+							 USERPTR_OP_EVICT);
+			break;
+		}
+	}
+
+	virtio_gpu_userptr_schedule_list_work(userptr->file->driver_priv);
+
+	mutex_unlock(&userptr->lock);
+	mmput(mn->mm);
+	return true;
+}
+
+static void
+virtio_gpu_userptr_lock_and_flush_work(struct virtio_gpu_fpriv *vfpriv)
+{
+retry_flush_work:
+	flush_work(&vfpriv->userptr_work);
+
+	if (list_empty(&vfpriv->userptr_work_list))
+		return;
+
+	goto retry_flush_work;
+}
+
+static bool virtio_gpu_userptr_valid_with_notifier(
+	struct virtio_gpu_object_userptr *userptr)
+{
+	return (!atomic_read(&userptr->in_release)) && (!userptr->op);
+}
+
+static void
+virtio_gpu_userptr_notifier_init(struct virtio_gpu_object_userptr *userptr,
+				 struct mm_struct *mm)
+{
+	userptr->notifier_start = 0;
+	userptr->notifier_last = 0;
+	atomic_set(&userptr->in_release, 0);
+	INIT_LIST_HEAD(&userptr->work_list);
+	mutex_init(&userptr->lock);
+	userptr->mm = mm;
+}
+
+#else
+static void
+virtio_gpu_userptr_lock_and_flush_work(struct virtio_gpu_fpriv *vfpriv)
+{
+}
+#endif /* CONFIG_MMU_NOTIFIER */
+
 static uint64_t virtio_gpu_userptr_get_offset(struct virtio_gpu_object *qobj,
 					      uint64_t addr)
 {
@@ -52,7 +454,8 @@ virtio_gpu_userptr_from_addr_range(struct virtio_gpu_fpriv *vfpriv,
 		userptr = container_of(node, struct virtio_gpu_object_userptr,
 				       it_node);
 
-		if (start >= userptr->start && last <= userptr->last) {
+		if (start >= userptr->start && last <= userptr->last &&
+		    virtio_gpu_userptr_valid(userptr)) {
 			ret = userptr;
 			return ret;
 		}
@@ -92,7 +495,6 @@ void virtio_gpu_userptr_set_handle(struct virtio_gpu_object *qobj,
 				   uint32_t handle)
 {
 	struct virtio_gpu_object_userptr *userptr = to_virtio_gpu_userptr(qobj);
-
 	userptr->bo_handle = handle;
 }
 
@@ -254,6 +656,9 @@ virtio_gpu_userptr_init(struct drm_device *dev, struct drm_file *file,
 	obj = &userptr->base.base.base;
 	obj->funcs = &virtio_gpu_userptr_funcs;
 
+	if (userptr->ops->notifier_init)
+		userptr->ops->notifier_init(userptr, current->mm);
+
 	drm_gem_private_object_init(dev, obj, aligned_size);
 
 	ret = virtio_gpu_resource_id_get(userptr->vgdev,
@@ -268,6 +673,15 @@ static const struct virtio_gpu_object_userptr_ops virtio_gpu_userptr_ops = {
 	.release = virtio_gpu_userptr_release,
 	.insert = virtio_gpu_userptr_insert_interval_tree,
 	.remove = virtio_gpu_userptr_remove_interval_tree,
+#ifdef CONFIG_MMU_NOTIFIER
+	.valid = virtio_gpu_userptr_valid_with_notifier,
+	.notifier_init = virtio_gpu_userptr_notifier_init,
+	.notifier_add = virtio_gpu_userptr_add_notifier,
+	.notifier_remove = virtio_gpu_userptr_remove_notifier,
+	.split = virtio_gpu_userptr_split,
+	.update = virtio_gpu_userptr_update_notifier_and_interval_tree,
+	.evict = virtio_gpu_userptr_evict,
+#endif
 };
 
 int virtio_gpu_userptr_create(struct virtio_gpu_device *vgdev,
@@ -290,6 +704,7 @@ int virtio_gpu_userptr_create(struct virtio_gpu_device *vgdev,
 		       params->size))
 		return -EFAULT;
 
+	virtio_gpu_userptr_lock_and_flush_work(vfpriv);
 	mutex_lock(&vfpriv->userptrs_tree_lock);
 
 	userptr = virtio_gpu_userptr_from_addr_range(
-- 
2.34.1