[PATCH v2 0/5] hw/virtio: Minor housekeeping patches

[PATCH v2 0/5] hw/virtio: Minor housekeeping patches

[Philippe Mathieu-Daudé]

Hi,

This series contains few patches I gathered while tooking notes
trying to understand issues #300-#302.

Since v1:
- Added virtqueue_flush comment (Stefano)
- Call RCU_READ_LOCK_GUARD in virtqueue_packed_drop_all (Stefano)

Philippe Mathieu-Daudé (5):
  hw/virtio: Document virtio_queue_packed_empty_rcu is called within RCU
  hw/virtio: Comment virtqueue_flush() must be called with RCU read lock
  hw/virtio: Remove NULL check in virtio_free_region_cache()
  hw/virtio: Acquire RCU read lock in virtqueue_packed_drop_all()
  hw/virtio: Have virtqueue_get_avail_bytes() pass caches arg to callees

 include/hw/virtio/virtio.h |  7 +++++++
 hw/virtio/virtio.c         | 39 ++++++++++++++++++--------------------
 2 files changed, 25 insertions(+), 21 deletions(-)

-- 
2.31.1

[PATCH v2 1/5] hw/virtio: Document virtio_queue_packed_empty_rcu is called within RCU

[Philippe Mathieu-Daudé]

While virtio_queue_packed_empty_rcu() uses the '_rcu' suffix,
it is not obvious it is called within rcu_read_lock(). All other
functions from this file called with the RCU locked have a comment
describing it. Document this one similarly for consistency.

Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 hw/virtio/virtio.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 874377f37a7..a5214bca612 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -634,6 +634,7 @@ static int virtio_queue_split_empty(VirtQueue *vq)
     return empty;
 }
 
+/* Called within rcu_read_lock().  */
 static int virtio_queue_packed_empty_rcu(VirtQueue *vq)
 {
     struct VRingPackedDesc desc;
-- 
2.31.1

[PATCH v2 2/5] hw/virtio: Comment virtqueue_flush() must be called with RCU read lock

[Philippe Mathieu-Daudé]

Reported-by: Stefano Garzarella <sgarzare@redhat.com>
Suggested-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 include/hw/virtio/virtio.h | 7 +++++++
 hw/virtio/virtio.c         | 1 +
 2 files changed, 8 insertions(+)

diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index 8bab9cfb750..c1c5f6e53c8 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -186,6 +186,13 @@ void virtio_delete_queue(VirtQueue *vq);
 
 void virtqueue_push(VirtQueue *vq, const VirtQueueElement *elem,
                     unsigned int len);
+/**
+ * virtqueue_flush:
+ * @vq: The #VirtQueue
+ * @count: Number of elements to flush
+ *
+ * Must be called within RCU critical section.
+ */
 void virtqueue_flush(VirtQueue *vq, unsigned int count);
 void virtqueue_detach_element(VirtQueue *vq, const VirtQueueElement *elem,
                               unsigned int len);
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index a5214bca612..b37344bb5e1 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -898,6 +898,7 @@ static void virtqueue_packed_flush(VirtQueue *vq, unsigned int count)
     }
 }
 
+/* Called within rcu_read_lock().  */
 void virtqueue_flush(VirtQueue *vq, unsigned int count)
 {
     if (virtio_device_disabled(vq->vdev)) {
-- 
2.31.1

[PATCH v2 3/5] hw/virtio: Remove NULL check in virtio_free_region_cache()

[Philippe Mathieu-Daudé]

virtio_free_region_cache() is called within call_rcu(),
always with a non-NULL argument. Ensure new code keep it
that way by replacing the NULL check by an assertion.
Add a comment this function is called within call_rcu().

Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
Worth mentioning the left-over from c611c76417f?
("virtio: add MemoryListener to cache ring translations")
---
 hw/virtio/virtio.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index b37344bb5e1..97f60017466 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -133,12 +133,10 @@ struct VirtQueue
     QLIST_ENTRY(VirtQueue) node;
 };
 
+/* Called within call_rcu().  */
 static void virtio_free_region_cache(VRingMemoryRegionCaches *caches)
 {
-    if (!caches) {
-        return;
-    }
-
+    assert(caches != NULL);
     address_space_cache_destroy(&caches->desc);
     address_space_cache_destroy(&caches->avail);
     address_space_cache_destroy(&caches->used);
-- 
2.31.1

[PATCH v2 4/5] hw/virtio: Acquire RCU read lock in virtqueue_packed_drop_all()

[Philippe Mathieu-Daudé]

vring_get_region_caches() must be called with the RCU read lock
acquired. virtqueue_packed_drop_all() does not, and uses the
'caches' pointer. Fix that by using the RCU_READ_LOCK_GUARD()
macro.

Reported-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 hw/virtio/virtio.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 97f60017466..7d3bf9091ee 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -1704,6 +1704,8 @@ static unsigned int virtqueue_packed_drop_all(VirtQueue *vq)
     VirtIODevice *vdev = vq->vdev;
     VRingPackedDesc desc;
 
+    RCU_READ_LOCK_GUARD();
+
     caches = vring_get_region_caches(vq);
     if (!caches) {
         return 0;
-- 
2.31.1

[PATCH v2 5/5] hw/virtio: Have virtqueue_get_avail_bytes() pass caches arg to callees

[Philippe Mathieu-Daudé]

Both virtqueue_packed_get_avail_bytes() and
virtqueue_split_get_avail_bytes() access the region cache, but
their caller also does. Simplify by having virtqueue_get_avail_bytes
calling both with RCU lock held, and passing the caches as argument.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 hw/virtio/virtio.c | 29 ++++++++++++-----------------
 1 file changed, 12 insertions(+), 17 deletions(-)

diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 7d3bf9091ee..0dbfb53e51b 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -985,28 +985,23 @@ static int virtqueue_split_read_next_desc(VirtIODevice *vdev, VRingDesc *desc,
     return VIRTQUEUE_READ_DESC_MORE;
 }
 
+/* Called within rcu_read_lock().  */
 static void virtqueue_split_get_avail_bytes(VirtQueue *vq,
                             unsigned int *in_bytes, unsigned int *out_bytes,
-                            unsigned max_in_bytes, unsigned max_out_bytes)
+                            unsigned max_in_bytes, unsigned max_out_bytes,
+                            VRingMemoryRegionCaches *caches)
 {
     VirtIODevice *vdev = vq->vdev;
     unsigned int max, idx;
     unsigned int total_bufs, in_total, out_total;
-    VRingMemoryRegionCaches *caches;
     MemoryRegionCache indirect_desc_cache = MEMORY_REGION_CACHE_INVALID;
     int64_t len = 0;
     int rc;
 
-    RCU_READ_LOCK_GUARD();
-
     idx = vq->last_avail_idx;
     total_bufs = in_total = out_total = 0;
 
     max = vq->vring.num;
-    caches = vring_get_region_caches(vq);
-    if (!caches) {
-        goto err;
-    }
 
     while ((rc = virtqueue_num_heads(vq, idx)) > 0) {
         MemoryRegionCache *desc_cache = &caches->desc;
@@ -1125,32 +1120,28 @@ static int virtqueue_packed_read_next_desc(VirtQueue *vq,
     return VIRTQUEUE_READ_DESC_MORE;
 }
 
+/* Called within rcu_read_lock().  */
 static void virtqueue_packed_get_avail_bytes(VirtQueue *vq,
                                              unsigned int *in_bytes,
                                              unsigned int *out_bytes,
                                              unsigned max_in_bytes,
-                                             unsigned max_out_bytes)
+                                             unsigned max_out_bytes,
+                                             VRingMemoryRegionCaches *caches)
 {
     VirtIODevice *vdev = vq->vdev;
     unsigned int max, idx;
     unsigned int total_bufs, in_total, out_total;
     MemoryRegionCache *desc_cache;
-    VRingMemoryRegionCaches *caches;
     MemoryRegionCache indirect_desc_cache = MEMORY_REGION_CACHE_INVALID;
     int64_t len = 0;
     VRingPackedDesc desc;
     bool wrap_counter;
 
-    RCU_READ_LOCK_GUARD();
     idx = vq->last_avail_idx;
     wrap_counter = vq->last_avail_wrap_counter;
     total_bufs = in_total = out_total = 0;
 
     max = vq->vring.num;
-    caches = vring_get_region_caches(vq);
-    if (!caches) {
-        goto err;
-    }
 
     for (;;) {
         unsigned int num_bufs = total_bufs;
@@ -1251,6 +1242,8 @@ void virtqueue_get_avail_bytes(VirtQueue *vq, unsigned int *in_bytes,
     uint16_t desc_size;
     VRingMemoryRegionCaches *caches;
 
+    RCU_READ_LOCK_GUARD();
+
     if (unlikely(!vq->vring.desc)) {
         goto err;
     }
@@ -1269,10 +1262,12 @@ void virtqueue_get_avail_bytes(VirtQueue *vq, unsigned int *in_bytes,
 
     if (virtio_vdev_has_feature(vq->vdev, VIRTIO_F_RING_PACKED)) {
         virtqueue_packed_get_avail_bytes(vq, in_bytes, out_bytes,
-                                         max_in_bytes, max_out_bytes);
+                                         max_in_bytes, max_out_bytes,
+                                         caches);
     } else {
         virtqueue_split_get_avail_bytes(vq, in_bytes, out_bytes,
-                                        max_in_bytes, max_out_bytes);
+                                        max_in_bytes, max_out_bytes,
+                                        caches);
     }
 
     return;
-- 
2.31.1

Re: [PATCH v2 0/5] hw/virtio: Minor housekeeping patches

[Michael S. Tsirkin]

On Thu, Sep 02, 2021 at 06:50:34PM +0200, Philippe Mathieu-Daudé wrote:
> Hi,
> 
> This series contains few patches I gathered while tooking notes
> trying to understand issues #300-#302.

v1 was includes in my pull request already, pls send
incremental patches on top. Thanks!

> Since v1:
> - Added virtqueue_flush comment (Stefano)
> - Call RCU_READ_LOCK_GUARD in virtqueue_packed_drop_all (Stefano)
> 
> Philippe Mathieu-Daudé (5):
>   hw/virtio: Document virtio_queue_packed_empty_rcu is called within RCU
>   hw/virtio: Comment virtqueue_flush() must be called with RCU read lock
>   hw/virtio: Remove NULL check in virtio_free_region_cache()
>   hw/virtio: Acquire RCU read lock in virtqueue_packed_drop_all()
>   hw/virtio: Have virtqueue_get_avail_bytes() pass caches arg to callees
> 
>  include/hw/virtio/virtio.h |  7 +++++++
>  hw/virtio/virtio.c         | 39 ++++++++++++++++++--------------------
>  2 files changed, 25 insertions(+), 21 deletions(-)
> 
> -- 
> 2.31.1
>