* [PATCH] virtio-gpu: OUT_OF_MEMORY if failing to create udmabuf
@ 2023-06-21 21:19 Dongwon Kim
2023-06-22 13:24 ` Marc-André Lureau
0 siblings, 1 reply; 3+ messages in thread
From: Dongwon Kim @ 2023-06-21 21:19 UTC (permalink / raw)
To: qemu-devel
Cc: Dongwon Kim, Gerd Hoffmann, Marc-André Lureau,
Vivek Kasireddy
Respond with VIRTIO_GPU_RESP_ERR_OUT_OF_MEMORY if it fails to create
an udmabuf for the blob resource.
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Cc: Vivek Kasireddy <vivek.kasireddy@intel.com>
Signed-off-by: Dongwon Kim <dongwon.kim@intel.com>
---
hw/display/virtio-gpu.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
index 66cddd94d9..efe66ca7a3 100644
--- a/hw/display/virtio-gpu.c
+++ b/hw/display/virtio-gpu.c
@@ -635,9 +635,11 @@ static void virtio_gpu_do_set_scanout(VirtIOGPU *g,
if (!virtio_gpu_update_dmabuf(g, scanout_id, res, fb, r)) {
virtio_gpu_update_scanout(g, scanout_id, res, r);
return;
+ } else {
+ *error = VIRTIO_GPU_RESP_ERR_OUT_OF_MEMORY;
+ return;
}
}
-
data = res->blob;
} else {
data = (uint8_t *)pixman_image_get_data(res->image);
--
2.34.1
^ permalink raw reply related [flat|nested] 3+ messages in thread* Re: [PATCH] virtio-gpu: OUT_OF_MEMORY if failing to create udmabuf
2023-06-21 21:19 [PATCH] virtio-gpu: OUT_OF_MEMORY if failing to create udmabuf Dongwon Kim
@ 2023-06-22 13:24 ` Marc-André Lureau
2023-06-27 0:34 ` [PATCH v2] " Dongwon Kim
0 siblings, 1 reply; 3+ messages in thread
From: Marc-André Lureau @ 2023-06-22 13:24 UTC (permalink / raw)
To: Dongwon Kim; +Cc: qemu-devel, Gerd Hoffmann, Vivek Kasireddy
[-- Attachment #1: Type: text/plain, Size: 1336 bytes --]
Hi
On Wed, Jun 21, 2023 at 11:40 PM Dongwon Kim <dongwon.kim@intel.com> wrote:
> Respond with VIRTIO_GPU_RESP_ERR_OUT_OF_MEMORY if it fails to create
> an udmabuf for the blob resource.
>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
> Cc: Vivek Kasireddy <vivek.kasireddy@intel.com>
> Signed-off-by: Dongwon Kim <dongwon.kim@intel.com>
> ---
> hw/display/virtio-gpu.c | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
> index 66cddd94d9..efe66ca7a3 100644
> --- a/hw/display/virtio-gpu.c
> +++ b/hw/display/virtio-gpu.c
> @@ -635,9 +635,11 @@ static void virtio_gpu_do_set_scanout(VirtIOGPU *g,
> if (!virtio_gpu_update_dmabuf(g, scanout_id, res, fb, r)) {
> virtio_gpu_update_scanout(g, scanout_id, res, r);
> return;
> + } else {
> + *error = VIRTIO_GPU_RESP_ERR_OUT_OF_MEMORY;
> + return;
> }
> }
> -
>
unrelated style change
> data = res->blob;
> } else {
> data = (uint8_t *)pixman_image_get_data(res->image);
> --
> 2.34.1
>
>
>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
--
Marc-André Lureau
[-- Attachment #2: Type: text/html, Size: 2407 bytes --]
^ permalink raw reply [flat|nested] 3+ messages in thread* [PATCH v2] virtio-gpu: OUT_OF_MEMORY if failing to create udmabuf
2023-06-22 13:24 ` Marc-André Lureau
@ 2023-06-27 0:34 ` Dongwon Kim
0 siblings, 0 replies; 3+ messages in thread
From: Dongwon Kim @ 2023-06-27 0:34 UTC (permalink / raw)
To: qemu-devel
Cc: Dongwon Kim, Gerd Hoffmann, Marc-André Lureau,
Vivek Kasireddy
Respond with VIRTIO_GPU_RESP_ERR_OUT_OF_MEMORY if it fails to create
an udmabuf for the blob resource.
v2: consolidated return statments and removed an unnecessary style change
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
Cc: Vivek Kasireddy <vivek.kasireddy@intel.com>
Signed-off-by: Dongwon Kim <dongwon.kim@intel.com>
---
hw/display/virtio-gpu.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
index 66cddd94d9..ce019941b8 100644
--- a/hw/display/virtio-gpu.c
+++ b/hw/display/virtio-gpu.c
@@ -634,8 +634,10 @@ static void virtio_gpu_do_set_scanout(VirtIOGPU *g,
if (console_has_gl(scanout->con)) {
if (!virtio_gpu_update_dmabuf(g, scanout_id, res, fb, r)) {
virtio_gpu_update_scanout(g, scanout_id, res, r);
- return;
+ } else {
+ *error = VIRTIO_GPU_RESP_ERR_OUT_OF_MEMORY;
}
+ return;
}
data = res->blob;
--
2.34.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
end of thread, other threads:[~2023-06-27 0:55 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-06-21 21:19 [PATCH] virtio-gpu: OUT_OF_MEMORY if failing to create udmabuf Dongwon Kim
2023-06-22 13:24 ` Marc-André Lureau
2023-06-27 0:34 ` [PATCH v2] " Dongwon Kim
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).