[Qemu-devel] [PATCH v3] virtio-blk: Fix double completion for werror=stop

[Qemu-devel] [PATCH v3] virtio-blk: Fix double completion for werror=stop

[Fam Zheng]

When a request R is absorbed by request M, it is appended to the
"mr_next" queue led by M, and is completed together with the completion
of M, in virtio_blk_rw_complete.

During DMA restart in virtio_blk_dma_restart_bh, requests in s->rq are
parsed and submitted again, possibly with a stale req->mr_next. It could
be a problem if the request merging in virtio_blk_handle_request hasn't
refreshed every mr_next pointer, in which case, virtio_blk_rw_complete
could walk through unexpected requests following the stale pointers.

Fix this by unsetting the pointer in virtio_blk_rw_complete. It is safe
because this req is either completed and freed right away, or it will be
restarted and parsed from scratch out of the vq later.

Signed-off-by: Fam Zheng <famz@redhat.com>

---

v3: Fix as Stefan suggested.
---
 hw/block/virtio-blk.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index e70fccf..848f3fe 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -112,6 +112,10 @@ static void virtio_blk_rw_complete(void *opaque, int ret)
              * happen on the other side of the migration).
              */
             if (virtio_blk_handle_rw_error(req, -ret, is_read)) {
+                /* Break the link in case the next request is added to the
+                 * restart queue and is going to be parsed from the ring again.
+                 */
+                req->mr_next = NULL;
                 continue;
             }
         }
-- 
2.4.3

Re: [Qemu-devel] [PATCH v3] virtio-blk: Fix double completion for werror=stop

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1097 bytes --]

On Tue, Nov 17, 2015 at 06:20:11PM +0800, Fam Zheng wrote:
> When a request R is absorbed by request M, it is appended to the
> "mr_next" queue led by M, and is completed together with the completion
> of M, in virtio_blk_rw_complete.
> 
> During DMA restart in virtio_blk_dma_restart_bh, requests in s->rq are
> parsed and submitted again, possibly with a stale req->mr_next. It could
> be a problem if the request merging in virtio_blk_handle_request hasn't
> refreshed every mr_next pointer, in which case, virtio_blk_rw_complete
> could walk through unexpected requests following the stale pointers.
> 
> Fix this by unsetting the pointer in virtio_blk_rw_complete. It is safe
> because this req is either completed and freed right away, or it will be
> restarted and parsed from scratch out of the vq later.
> 
> Signed-off-by: Fam Zheng <famz@redhat.com>
> 
> ---
> 
> v3: Fix as Stefan suggested.
> ---
>  hw/block/virtio-blk.c | 4 ++++
>  1 file changed, 4 insertions(+)

Thanks, applied to my block tree:
https://github.com/stefanha/qemu/commits/block

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Qemu-devel] [PATCH v3] virtio-blk: Fix double completion for werror=stop

[Laurent Vivier]

On 17/11/2015 11:34, Stefan Hajnoczi wrote:
> On Tue, Nov 17, 2015 at 06:20:11PM +0800, Fam Zheng wrote:
>> When a request R is absorbed by request M, it is appended to the
>> "mr_next" queue led by M, and is completed together with the completion
>> of M, in virtio_blk_rw_complete.
>>
>> During DMA restart in virtio_blk_dma_restart_bh, requests in s->rq are
>> parsed and submitted again, possibly with a stale req->mr_next. It could
>> be a problem if the request merging in virtio_blk_handle_request hasn't
>> refreshed every mr_next pointer, in which case, virtio_blk_rw_complete
>> could walk through unexpected requests following the stale pointers.
>>
>> Fix this by unsetting the pointer in virtio_blk_rw_complete. It is safe
>> because this req is either completed and freed right away, or it will be
>> restarted and parsed from scratch out of the vq later.
>>
>> Signed-off-by: Fam Zheng <famz@redhat.com>
>>
>> ---
>>
>> v3: Fix as Stefan suggested.
>> ---
>>  hw/block/virtio-blk.c | 4 ++++
>>  1 file changed, 4 insertions(+)
> 
> Thanks, applied to my block tree:
> https://github.com/stefanha/qemu/commits/block

I have also tested this one and it works fine.

Tested-by: Laurent Vivier <lvivier@redhat.com>

Laurent

Re: [Qemu-devel] [PATCH v3] virtio-blk: Fix double completion for werror=stop

[Paolo Bonzini]

On 17/11/2015 11:20, Fam Zheng wrote:
> When a request R is absorbed by request M, it is appended to the
> "mr_next" queue led by M, and is completed together with the completion
> of M, in virtio_blk_rw_complete.
> 
> During DMA restart in virtio_blk_dma_restart_bh, requests in s->rq are
> parsed and submitted again, possibly with a stale req->mr_next. It could
> be a problem if the request merging in virtio_blk_handle_request hasn't
> refreshed every mr_next pointer, in which case, virtio_blk_rw_complete
> could walk through unexpected requests following the stale pointers.
> 
> Fix this by unsetting the pointer in virtio_blk_rw_complete. It is safe
> because this req is either completed and freed right away, or it will be
> restarted and parsed from scratch out of the vq later.
> 
> Signed-off-by: Fam Zheng <famz@redhat.com>
> 
> ---
> 
> v3: Fix as Stefan suggested.
> ---
>  hw/block/virtio-blk.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
> index e70fccf..848f3fe 100644
> --- a/hw/block/virtio-blk.c
> +++ b/hw/block/virtio-blk.c
> @@ -112,6 +112,10 @@ static void virtio_blk_rw_complete(void *opaque, int ret)
>               * happen on the other side of the migration).
>               */
>              if (virtio_blk_handle_rw_error(req, -ret, is_read)) {
> +                /* Break the link in case the next request is added to the
> +                 * restart queue and is going to be parsed from the ring again.
> +                 */
> +                req->mr_next = NULL;
>                  continue;
>              }
>          }
> 

This is now a write-after-free for rerror/werror=stop.  The right place
to set req->mr_next is inside virtio_blk_handle_rw_error, I think.

Paolo

Re: [Qemu-devel] [PATCH v3] virtio-blk: Fix double completion for werror=stop

[Fam Zheng]

On Sun, 11/22 20:41, Paolo Bonzini wrote:
> > diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
> > index e70fccf..848f3fe 100644
> > --- a/hw/block/virtio-blk.c
> > +++ b/hw/block/virtio-blk.c
> > @@ -112,6 +112,10 @@ static void virtio_blk_rw_complete(void *opaque, int ret)
> >               * happen on the other side of the migration).
> >               */
> >              if (virtio_blk_handle_rw_error(req, -ret, is_read)) {
> > +                /* Break the link in case the next request is added to the
> > +                 * restart queue and is going to be parsed from the ring again.
> > +                 */
> > +                req->mr_next = NULL;
> >                  continue;
> >              }
> >          }
> > 
> 
> This is now a write-after-free for rerror/werror=stop.  The right place
> to set req->mr_next is inside virtio_blk_handle_rw_error, I think.
> 

Oh yes :( Sending another patch.

Fam