[v5 0/2] virtio-gpu: fix error handling and improve consistency

[v5 0/2] virtio-gpu: fix error handling and improve consistency

[Honglei Huang]

This series addresses error handling issues in virtio-gpu and improves
code consistency across the virtio-gpu subsystem.

The first patch fixes a critical bug in virgl_cmd_resource_create_blob()
where an inverted error check causes the function to fail when it should
succeed. This is a standalone bug fix that should be backported.

The second patch improves code consistency by unifying the error checking
style for virtio_gpu_create_mapping_iov() across all virtio-gpu files,
following the preferred QEMU coding convention.

Changes since v4:
- Split the single patch into two separate patches for better clarity
- Separated the critical bug fix from the style consistency improvements
- The bug fix (patch 1) can now be easily identified and backported
- The consistency improvements (patch 2) are clearly marked as cleanup

Changes since v3:
- Extended consistency improvements to virtio-gpu-rutabaga.c
- Changed CHECK(!ret) to CHECK(ret >= 0) and CHECK(!result) to
  CHECK(result >= 0) in rutabaga functions for consistency
- Now covers all virtio-gpu files that use virtio_gpu_create_mapping_iov()

Changes since v2:
- Use 'if (ret < 0)' instead of 'if (ret != 0)' following maintainer's
  feedback on preferred QEMU coding style for error checking functions
  that return 0 on success and negative on error
- Updated all similar usages across virtio-gpu files for consistency
- Expanded scope from single function fix to codebase-wide style consistency

Honglei Huang (2):
  virtio-gpu: fix error handling in virgl_cmd_resource_create_blob
  virtio-gpu: use consistent error checking for
    virtio_gpu_create_mapping_iov

 hw/display/virtio-gpu-rutabaga.c | 4 ++--
 hw/display/virtio-gpu-virgl.c    | 4 ++--
 hw/display/virtio-gpu.c          | 4 ++--
 3 files changed, 6 insertions(+), 6 deletions(-)

-- 
2.34.1

[v5 1/2] virtio-gpu: fix error handling in virgl_cmd_resource_create_blob

[Honglei Huang]

Fix inverted error check in virgl_cmd_resource_create_blob() that causes
the function to return error when virtio_gpu_create_mapping_iov() succeeds.

virtio_gpu_create_mapping_iov() returns 0 on success and negative values
on error. The check 'if (!ret)' incorrectly treats success (ret=0) as an
error condition, causing the function to fail when it should succeed.

Change the condition to 'if (ret != 0)' to properly detect errors.

Fixes: 7c092f17ccee ("virtio-gpu: Handle resource blob commands")
Signed-off-by: Honglei Huang <honghuan@amd.com>
Reviewed-by: Akihiko Odaki <odaki@rsg.ci.i.u-tokyo.ac.jp>
---
 hw/display/virtio-gpu-virgl.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/display/virtio-gpu-virgl.c b/hw/display/virtio-gpu-virgl.c
index 94ddc01f91..e60e1059df 100644
--- a/hw/display/virtio-gpu-virgl.c
+++ b/hw/display/virtio-gpu-virgl.c
@@ -701,7 +701,7 @@ static void virgl_cmd_resource_create_blob(VirtIOGPU *g,
         ret = virtio_gpu_create_mapping_iov(g, cblob.nr_entries, sizeof(cblob),
                                             cmd, &res->base.addrs,
                                             &res->base.iov, &res->base.iov_cnt);
-        if (!ret) {
+        if (ret != 0) {
             cmd->error = VIRTIO_GPU_RESP_ERR_UNSPEC;
             return;
         }
-- 
2.34.1

[v5 2/2] virtio-gpu: use consistent error checking for virtio_gpu_create_mapping_iov

[Honglei Huang]

Unify error checking style for virtio_gpu_create_mapping_iov() across the
codebase to improve consistency and readability.

virtio_gpu_create_mapping_iov() returns 0 on success and negative values
on error. The original code used inconsistent patterns for checking errors:
- Some used 'if (ret != 0)'
- Some used 'CHECK(!ret, cmd)'
- Some used 'CHECK(!result, cmd)'

Change all checks to use the consistent 'if (ret < 0)' or 'CHECK(ret >= 0)'
patterns, following the preferred QEMU coding convention for functions that
return 0 on success and negative on error. This makes the return value
convention immediately clear to code readers without needing to look up
the function definition.

Updated locations:
- hw/display/virtio-gpu-virgl.c: virgl_resource_attach_backing()
- hw/display/virtio-gpu.c: virtio_gpu_resource_create_blob()
- hw/display/virtio-gpu.c: virtio_gpu_resource_attach_backing()
- hw/display/virtio-gpu-rutabaga.c: rutabaga_cmd_attach_backing()
- hw/display/virtio-gpu-rutabaga.c: rutabaga_cmd_resource_create_blob()

Signed-off-by: Honglei Huang <honghuan@amd.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
---
 hw/display/virtio-gpu-rutabaga.c | 4 ++--
 hw/display/virtio-gpu-virgl.c    | 4 ++--
 hw/display/virtio-gpu.c          | 4 ++--
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/hw/display/virtio-gpu-rutabaga.c b/hw/display/virtio-gpu-rutabaga.c
index ed5ae52acb..ea2928b706 100644
--- a/hw/display/virtio-gpu-rutabaga.c
+++ b/hw/display/virtio-gpu-rutabaga.c
@@ -466,7 +466,7 @@ rutabaga_cmd_attach_backing(VirtIOGPU *g, struct virtio_gpu_ctrl_command *cmd)
 
     ret = virtio_gpu_create_mapping_iov(g, att_rb.nr_entries, sizeof(att_rb),
                                         cmd, NULL, &res->iov, &res->iov_cnt);
-    CHECK(!ret, cmd);
+    CHECK(ret >= 0, cmd);
 
     vecs.iovecs = res->iov;
     vecs.num_iovecs = res->iov_cnt;
@@ -616,7 +616,7 @@ rutabaga_cmd_resource_create_blob(VirtIOGPU *g,
         result = virtio_gpu_create_mapping_iov(g, cblob.nr_entries,
                                                sizeof(cblob), cmd, &res->addrs,
                                                &res->iov, &res->iov_cnt);
-        CHECK(!result, cmd);
+        CHECK(result >= 0, cmd);
     }
 
     rc_blob.blob_id = cblob.blob_id;
diff --git a/hw/display/virtio-gpu-virgl.c b/hw/display/virtio-gpu-virgl.c
index e60e1059df..6ebd9293e5 100644
--- a/hw/display/virtio-gpu-virgl.c
+++ b/hw/display/virtio-gpu-virgl.c
@@ -557,7 +557,7 @@ static void virgl_resource_attach_backing(VirtIOGPU *g,
 
     ret = virtio_gpu_create_mapping_iov(g, att_rb.nr_entries, sizeof(att_rb),
                                         cmd, NULL, &res_iovs, &res_niov);
-    if (ret != 0) {
+    if (ret < 0) {
         cmd->error = VIRTIO_GPU_RESP_ERR_UNSPEC;
         return;
     }
@@ -701,7 +701,7 @@ static void virgl_cmd_resource_create_blob(VirtIOGPU *g,
         ret = virtio_gpu_create_mapping_iov(g, cblob.nr_entries, sizeof(cblob),
                                             cmd, &res->base.addrs,
                                             &res->base.iov, &res->base.iov_cnt);
-        if (ret != 0) {
+        if (ret < 0) {
             cmd->error = VIRTIO_GPU_RESP_ERR_UNSPEC;
             return;
         }
diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
index 0a1a625b0e..1038c6a49f 100644
--- a/hw/display/virtio-gpu.c
+++ b/hw/display/virtio-gpu.c
@@ -352,7 +352,7 @@ static void virtio_gpu_resource_create_blob(VirtIOGPU *g,
     ret = virtio_gpu_create_mapping_iov(g, cblob.nr_entries, sizeof(cblob),
                                         cmd, &res->addrs, &res->iov,
                                         &res->iov_cnt);
-    if (ret != 0) {
+    if (ret < 0) {
         cmd->error = VIRTIO_GPU_RESP_ERR_UNSPEC;
         g_free(res);
         return;
@@ -931,7 +931,7 @@ virtio_gpu_resource_attach_backing(VirtIOGPU *g,
 
     ret = virtio_gpu_create_mapping_iov(g, ab.nr_entries, sizeof(ab), cmd,
                                         &res->addrs, &res->iov, &res->iov_cnt);
-    if (ret != 0) {
+    if (ret < 0) {
         cmd->error = VIRTIO_GPU_RESP_ERR_UNSPEC;
         return;
     }
-- 
2.34.1

Re: [v5 1/2] virtio-gpu: fix error handling in virgl_cmd_resource_create_blob

[Dmitry Osipenko]

On 11/24/25 06:24, Honglei Huang wrote:
> Fix inverted error check in virgl_cmd_resource_create_blob() that causes
> the function to return error when virtio_gpu_create_mapping_iov() succeeds.
> 
> virtio_gpu_create_mapping_iov() returns 0 on success and negative values
> on error. The check 'if (!ret)' incorrectly treats success (ret=0) as an
> error condition, causing the function to fail when it should succeed.
> 
> Change the condition to 'if (ret != 0)' to properly detect errors.
> 
> Fixes: 7c092f17ccee ("virtio-gpu: Handle resource blob commands")
> Signed-off-by: Honglei Huang <honghuan@amd.com>
> Reviewed-by: Akihiko Odaki <odaki@rsg.ci.i.u-tokyo.ac.jp>
> ---
>  hw/display/virtio-gpu-virgl.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/hw/display/virtio-gpu-virgl.c b/hw/display/virtio-gpu-virgl.c
> index 94ddc01f91..e60e1059df 100644
> --- a/hw/display/virtio-gpu-virgl.c
> +++ b/hw/display/virtio-gpu-virgl.c
> @@ -701,7 +701,7 @@ static void virgl_cmd_resource_create_blob(VirtIOGPU *g,
>          ret = virtio_gpu_create_mapping_iov(g, cblob.nr_entries, sizeof(cblob),
>                                              cmd, &res->base.addrs,
>                                              &res->base.iov, &res->base.iov_cnt);
> -        if (!ret) {
> +        if (ret != 0) {
>              cmd->error = VIRTIO_GPU_RESP_ERR_UNSPEC;
>              return;
>          }

Reviewed-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>

-- 
Best regards,
Dmitry

Re: [v5 2/2] virtio-gpu: use consistent error checking for virtio_gpu_create_mapping_iov

[Dmitry Osipenko]

On 11/24/25 06:24, Honglei Huang wrote:
> diff --git a/hw/display/virtio-gpu-rutabaga.c b/hw/display/virtio-gpu-rutabaga.c
> index ed5ae52acb..ea2928b706 100644
> --- a/hw/display/virtio-gpu-rutabaga.c
> +++ b/hw/display/virtio-gpu-rutabaga.c
> @@ -466,7 +466,7 @@ rutabaga_cmd_attach_backing(VirtIOGPU *g, struct virtio_gpu_ctrl_command *cmd)
>  
>      ret = virtio_gpu_create_mapping_iov(g, att_rb.nr_entries, sizeof(att_rb),
>                                          cmd, NULL, &res->iov, &res->iov_cnt);
> -    CHECK(!ret, cmd);
> +    CHECK(ret >= 0, cmd);
>  
>      vecs.iovecs = res->iov;
>      vecs.num_iovecs = res->iov_cnt;
> @@ -616,7 +616,7 @@ rutabaga_cmd_resource_create_blob(VirtIOGPU *g,
>          result = virtio_gpu_create_mapping_iov(g, cblob.nr_entries,
>                                                 sizeof(cblob), cmd, &res->addrs,
>                                                 &res->iov, &res->iov_cnt);
> -        CHECK(!result, cmd);
> +        CHECK(result >= 0, cmd);
>      }

The ret < 0 part looks okay, but ret >= 0 feels dubious to me given that
this func doesn't return positive values.

-- 
Best regards,
Dmitry

Re: [v5 2/2] virtio-gpu: use consistent error checking for virtio_gpu_create_mapping_iov

[Honglei Huang]

On 2025/11/25 05:31, Dmitry Osipenko wrote:
> On 11/24/25 06:24, Honglei Huang wrote:
>> diff --git a/hw/display/virtio-gpu-rutabaga.c b/hw/display/virtio-gpu-rutabaga.c
>> index ed5ae52acb..ea2928b706 100644
>> --- a/hw/display/virtio-gpu-rutabaga.c
>> +++ b/hw/display/virtio-gpu-rutabaga.c
>> @@ -466,7 +466,7 @@ rutabaga_cmd_attach_backing(VirtIOGPU *g, struct virtio_gpu_ctrl_command *cmd)
>>   
>>       ret = virtio_gpu_create_mapping_iov(g, att_rb.nr_entries, sizeof(att_rb),
>>                                           cmd, NULL, &res->iov, &res->iov_cnt);
>> -    CHECK(!ret, cmd);
>> +    CHECK(ret >= 0, cmd);
>>   
>>       vecs.iovecs = res->iov;
>>       vecs.num_iovecs = res->iov_cnt;
>> @@ -616,7 +616,7 @@ rutabaga_cmd_resource_create_blob(VirtIOGPU *g,
>>           result = virtio_gpu_create_mapping_iov(g, cblob.nr_entries,
>>                                                  sizeof(cblob), cmd, &res->addrs,
>>                                                  &res->iov, &res->iov_cnt);
>> -        CHECK(!result, cmd);
>> +        CHECK(result >= 0, cmd);
>>       }
> 
> The ret < 0 part looks okay, but ret >= 0 feels dubious to me given that
> this func doesn't return positive values.
> 

Really thanks for the review.

How about split patch 1 and patch 2 into different threads, for the 
error checking style I will submit a another thread, maybe change the 
virtio_gpu_create_mapping_iov return value to true or false is a better 
choice?

Re: [v5 2/2] virtio-gpu: use consistent error checking for virtio_gpu_create_mapping_iov

[Honglei Huang]

On 2025/11/25 05:31, Dmitry Osipenko wrote:
> On 11/24/25 06:24, Honglei Huang wrote:
>> diff --git a/hw/display/virtio-gpu-rutabaga.c b/hw/display/virtio-gpu-rutabaga.c
>> index ed5ae52acb..ea2928b706 100644
>> --- a/hw/display/virtio-gpu-rutabaga.c
>> +++ b/hw/display/virtio-gpu-rutabaga.c
>> @@ -466,7 +466,7 @@ rutabaga_cmd_attach_backing(VirtIOGPU *g, struct virtio_gpu_ctrl_command *cmd)
>>   
>>       ret = virtio_gpu_create_mapping_iov(g, att_rb.nr_entries, sizeof(att_rb),
>>                                           cmd, NULL, &res->iov, &res->iov_cnt);
>> -    CHECK(!ret, cmd);
>> +    CHECK(ret >= 0, cmd);
>>   
>>       vecs.iovecs = res->iov;
>>       vecs.num_iovecs = res->iov_cnt;
>> @@ -616,7 +616,7 @@ rutabaga_cmd_resource_create_blob(VirtIOGPU *g,
>>           result = virtio_gpu_create_mapping_iov(g, cblob.nr_entries,
>>                                                  sizeof(cblob), cmd, &res->addrs,
>>                                                  &res->iov, &res->iov_cnt);
>> -        CHECK(!result, cmd);
>> +        CHECK(result >= 0, cmd);
>>       }
> 
> The ret < 0 part looks okay, but ret >= 0 feels dubious to me given that
> this func doesn't return positive values.
> 

Will remove the ret >= 0 check according to your requirements in V6.