[Qemu-devel] [PATCH for-4.1 0/2] block: bdrv_drained_end() changes fallout

[Qemu-devel] [PATCH for-4.1 0/2] block: bdrv_drained_end() changes fallout

[Max Reitz]

Hi,

I noted that test-bdrv-drain sometimgs hangs (very rarely, though), and
tried to write a test that triggers the issue.  I failed to do so (there
is a good reason for that, see patch 1), but on my way I noticed that
calling bdrv_set_aio_context_ignore() from any AioContext but the main
one is a bad idea.  Hence patch 2.

Anyway, I found the problem, which is fixed by patch 1 -- I think it’s
rather obvious.  There is no dedicated test because I don’t think it’s
possible to write one, as I explain there.


Max Reitz (2):
  block: Dec. drained_end_counter before bdrv_wakeup
  block: Only the main loop can change AioContexts

 include/block/block.h |  8 +++-----
 block.c               | 13 ++++++++-----
 block/io.c            |  5 ++---
 3 files changed, 13 insertions(+), 13 deletions(-)

-- 
2.21.0

[Qemu-devel] [PATCH for-4.1 1/2] block: Dec. drained_end_counter before bdrv_wakeup

[Max Reitz]

Decrementing drained_end_counter after bdrv_dec_in_flight() (which in
turn invokes bdrv_wakeup() and thus aio_wait_kick()) is not very clever.
We should decrement it beforehand, so that any waiting aio_poll() that
is woken by bdrv_dec_in_flight() sees the decremented
drained_end_counter.

Because the time window between decrementing drained_end_counter and
aio_wait_kick() is very small, I cannot supply a reliable regression
test.  However, running e.g. the /bdrv-drain/blockjob/iothread/drain_all
test in test-bdrv-drain has a small chance of hanging without this
patch (about 1/200 or so; it gets to nearly 100 % if you add e.g. an
fputc(' ', stderr); after the bdrv_dec_flight()).

Fixes: e037c09c78520cbdb6da7cfc6ad0256d5870b814
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/io.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/block/io.c b/block/io.c
index b89e155d21..06305c6ea6 100644
--- a/block/io.c
+++ b/block/io.c
@@ -217,13 +217,12 @@ static void coroutine_fn bdrv_drain_invoke_entry(void *opaque)
         bs->drv->bdrv_co_drain_end(bs);
     }
 
-    /* Set data->done before reading bs->wakeup.  */
+    /* Set data->done and decrement drained_end_counter before bdrv_wakeup() */
     atomic_mb_set(&data->done, true);
-    bdrv_dec_in_flight(bs);
-
     if (!data->begin) {
         atomic_dec(data->drained_end_counter);
     }
+    bdrv_dec_in_flight(bs);
 
     g_free(data);
 }
-- 
2.21.0

[Qemu-devel] [PATCH for-4.1 2/2] block: Only the main loop can change AioContexts

[Max Reitz]

bdrv_set_aio_context_ignore() can only work in the main loop:
bdrv_drained_begin() only works in the main loop and the node's (old)
AioContext; and bdrv_drained_end() really only works in the main loop
and the node's (new) AioContext (contrary to its current comment, which
is just wrong).

Consequentially, bdrv_set_aio_context_ignore() must be called from the
main loop.  Luckily, assuming that we can make block graph changes only
from the main loop as well, all its callers do that already.

Note that changing a node's context in a sense is an operation that
changes the block graph, so it actually makes sense to require this
function to be called from the main loop.

Also, fix bdrv_drained_end()'s description.  You can only use it from
the main loop or the node's AioContext, and in the latter case, the
whole subtree must be in the same context.

Fixes: e037c09c78520cbdb6da7cfc6ad0256d5870b814
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 include/block/block.h |  8 +++-----
 block.c               | 13 ++++++++-----
 2 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index 60f00479e0..50a07c1c33 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -667,11 +667,9 @@ void bdrv_subtree_drained_begin(BlockDriverState *bs);
  *
  * This polls @bs's AioContext until all scheduled sub-drained_ends
  * have settled.  On one hand, that may result in graph changes.  On
- * the other, this requires that all involved nodes (@bs and all of
- * its parents) are in the same AioContext, and that the caller has
- * acquired it.
- * If there are any nodes that are in different contexts from @bs,
- * these contexts must not be acquired.
+ * the other, this requires that the caller either runs in the main
+ * loop; or that all involved nodes (@bs and all of its parents) are
+ * in the caller's AioContext.
  */
 void bdrv_drained_end(BlockDriverState *bs);
 
diff --git a/block.c b/block.c
index 9c94f7f28a..cbd8da5f3b 100644
--- a/block.c
+++ b/block.c
@@ -5914,6 +5914,8 @@ static void bdrv_attach_aio_context(BlockDriverState *bs,
  * Changes the AioContext used for fd handlers, timers, and BHs by this
  * BlockDriverState and all its children and parents.
  *
+ * Must be called from the main AioContext.
+ *
  * The caller must own the AioContext lock for the old AioContext of bs, but it
  * must not own the AioContext lock for new_context (unless new_context is the
  * same as the current context of bs).
@@ -5925,9 +5927,10 @@ void bdrv_set_aio_context_ignore(BlockDriverState *bs,
                                  AioContext *new_context, GSList **ignore)
 {
     AioContext *old_context = bdrv_get_aio_context(bs);
-    AioContext *current_context = qemu_get_current_aio_context();
     BdrvChild *child;
 
+    g_assert(qemu_get_current_aio_context() == qemu_get_aio_context());
+
     if (old_context == new_context) {
         return;
     }
@@ -5953,7 +5956,7 @@ void bdrv_set_aio_context_ignore(BlockDriverState *bs,
     bdrv_detach_aio_context(bs);
 
     /* Acquire the new context, if necessary */
-    if (current_context != new_context) {
+    if (qemu_get_aio_context() != new_context) {
         aio_context_acquire(new_context);
     }
 
@@ -5965,16 +5968,16 @@ void bdrv_set_aio_context_ignore(BlockDriverState *bs,
      * subtree that have not yet been moved to the new AioContext.
      * Release the old one so bdrv_drained_end() can poll them.
      */
-    if (current_context != old_context) {
+    if (qemu_get_aio_context() != old_context) {
         aio_context_release(old_context);
     }
 
     bdrv_drained_end(bs);
 
-    if (current_context != old_context) {
+    if (qemu_get_aio_context() != old_context) {
         aio_context_acquire(old_context);
     }
-    if (current_context != new_context) {
+    if (qemu_get_aio_context() != new_context) {
         aio_context_release(new_context);
     }
 }
-- 
2.21.0

Re: [Qemu-devel] [PATCH for-4.1 2/2] block: Only the main loop can change AioContexts

[Kevin Wolf]

Am 22.07.2019 um 15:30 hat Max Reitz geschrieben:
> bdrv_set_aio_context_ignore() can only work in the main loop:
> bdrv_drained_begin() only works in the main loop and the node's (old)
> AioContext; and bdrv_drained_end() really only works in the main loop
> and the node's (new) AioContext (contrary to its current comment, which
> is just wrong).
> 
> Consequentially, bdrv_set_aio_context_ignore() must be called from the
> main loop.  Luckily, assuming that we can make block graph changes only
> from the main loop as well, all its callers do that already.
> 
> Note that changing a node's context in a sense is an operation that
> changes the block graph, so it actually makes sense to require this
> function to be called from the main loop.
> 
> Also, fix bdrv_drained_end()'s description.  You can only use it from
> the main loop or the node's AioContext, and in the latter case, the
> whole subtree must be in the same context.
> 
> Fixes: e037c09c78520cbdb6da7cfc6ad0256d5870b814
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
>  include/block/block.h |  8 +++-----
>  block.c               | 13 ++++++++-----
>  2 files changed, 11 insertions(+), 10 deletions(-)
> 
> diff --git a/include/block/block.h b/include/block/block.h
> index 60f00479e0..50a07c1c33 100644
> --- a/include/block/block.h
> +++ b/include/block/block.h
> @@ -667,11 +667,9 @@ void bdrv_subtree_drained_begin(BlockDriverState *bs);
>   *
>   * This polls @bs's AioContext until all scheduled sub-drained_ends
>   * have settled.  On one hand, that may result in graph changes.  On
> - * the other, this requires that all involved nodes (@bs and all of
> - * its parents) are in the same AioContext, and that the caller has
> - * acquired it.
> - * If there are any nodes that are in different contexts from @bs,
> - * these contexts must not be acquired.
> + * the other, this requires that the caller either runs in the main
> + * loop; or that all involved nodes (@bs and all of its parents) are
> + * in the caller's AioContext.
>   */
>  void bdrv_drained_end(BlockDriverState *bs);

I think you are right about the requirement that bdrv_drained_end() is
only called from the main or the BDS AioContext, which is a requirement
that directly comes from AIO_WAIT_WHILE().

However, I don't see why we have requirements on the AioContext of the
parent nodes (or any other nodes), except possibly not holding their
lock.  We don't poll their context, so it shouldn't matter in which
context they are?

Kevin

Re: [Qemu-devel] [PATCH for-4.1 2/2] block: Only the main loop can change AioContexts

[Max Reitz]

[-- Attachment #1.1: Type: text/plain, Size: 3098 bytes --]

On 23.07.19 10:52, Kevin Wolf wrote:
> Am 22.07.2019 um 15:30 hat Max Reitz geschrieben:
>> bdrv_set_aio_context_ignore() can only work in the main loop:
>> bdrv_drained_begin() only works in the main loop and the node's (old)
>> AioContext; and bdrv_drained_end() really only works in the main loop
>> and the node's (new) AioContext (contrary to its current comment, which
>> is just wrong).
>>
>> Consequentially, bdrv_set_aio_context_ignore() must be called from the
>> main loop.  Luckily, assuming that we can make block graph changes only
>> from the main loop as well, all its callers do that already.
>>
>> Note that changing a node's context in a sense is an operation that
>> changes the block graph, so it actually makes sense to require this
>> function to be called from the main loop.
>>
>> Also, fix bdrv_drained_end()'s description.  You can only use it from
>> the main loop or the node's AioContext, and in the latter case, the
>> whole subtree must be in the same context.
>>
>> Fixes: e037c09c78520cbdb6da7cfc6ad0256d5870b814
>> Signed-off-by: Max Reitz <mreitz@redhat.com>
>> ---
>>  include/block/block.h |  8 +++-----
>>  block.c               | 13 ++++++++-----
>>  2 files changed, 11 insertions(+), 10 deletions(-)
>>
>> diff --git a/include/block/block.h b/include/block/block.h
>> index 60f00479e0..50a07c1c33 100644
>> --- a/include/block/block.h
>> +++ b/include/block/block.h
>> @@ -667,11 +667,9 @@ void bdrv_subtree_drained_begin(BlockDriverState *bs);
>>   *
>>   * This polls @bs's AioContext until all scheduled sub-drained_ends
>>   * have settled.  On one hand, that may result in graph changes.  On
>> - * the other, this requires that all involved nodes (@bs and all of
>> - * its parents) are in the same AioContext, and that the caller has
>> - * acquired it.
>> - * If there are any nodes that are in different contexts from @bs,
>> - * these contexts must not be acquired.
>> + * the other, this requires that the caller either runs in the main
>> + * loop; or that all involved nodes (@bs and all of its parents) are
>> + * in the caller's AioContext.
>>   */
>>  void bdrv_drained_end(BlockDriverState *bs);
> 
> I think you are right about the requirement that bdrv_drained_end() is
> only called from the main or the BDS AioContext, which is a requirement
> that directly comes from AIO_WAIT_WHILE().
> 
> However, I don't see why we have requirements on the AioContext of the
> parent nodes (or any other nodes), except possibly not holding their
> lock.  We don't poll their context, so it shouldn't matter in which
> context they are?

Hm.  I don’t know how I got confused there, you’re right.

I don’t think the (falsely given) restriction hurts, though, because a
subtree should be within a single context anyway (unless you’re in
bdrv_set_aio_context_ignore(), which needs to be in the main context).

So, hm, yes, I messed up this comment a bit now.  But now it’s just more
restrictive than it needs to be and I don’t think callers are going to
care, so...

Max


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Qemu-devel] [PATCH for-4.1 2/2] block: Only the main loop can change AioContexts

[Kevin Wolf]

[-- Attachment #1: Type: text/plain, Size: 3638 bytes --]

Am 23.07.2019 um 11:41 hat Max Reitz geschrieben:
> On 23.07.19 10:52, Kevin Wolf wrote:
> > Am 22.07.2019 um 15:30 hat Max Reitz geschrieben:
> >> bdrv_set_aio_context_ignore() can only work in the main loop:
> >> bdrv_drained_begin() only works in the main loop and the node's (old)
> >> AioContext; and bdrv_drained_end() really only works in the main loop
> >> and the node's (new) AioContext (contrary to its current comment, which
> >> is just wrong).
> >>
> >> Consequentially, bdrv_set_aio_context_ignore() must be called from the
> >> main loop.  Luckily, assuming that we can make block graph changes only
> >> from the main loop as well, all its callers do that already.
> >>
> >> Note that changing a node's context in a sense is an operation that
> >> changes the block graph, so it actually makes sense to require this
> >> function to be called from the main loop.
> >>
> >> Also, fix bdrv_drained_end()'s description.  You can only use it from
> >> the main loop or the node's AioContext, and in the latter case, the
> >> whole subtree must be in the same context.
> >>
> >> Fixes: e037c09c78520cbdb6da7cfc6ad0256d5870b814
> >> Signed-off-by: Max Reitz <mreitz@redhat.com>
> >> ---
> >>  include/block/block.h |  8 +++-----
> >>  block.c               | 13 ++++++++-----
> >>  2 files changed, 11 insertions(+), 10 deletions(-)
> >>
> >> diff --git a/include/block/block.h b/include/block/block.h
> >> index 60f00479e0..50a07c1c33 100644
> >> --- a/include/block/block.h
> >> +++ b/include/block/block.h
> >> @@ -667,11 +667,9 @@ void bdrv_subtree_drained_begin(BlockDriverState *bs);
> >>   *
> >>   * This polls @bs's AioContext until all scheduled sub-drained_ends
> >>   * have settled.  On one hand, that may result in graph changes.  On
> >> - * the other, this requires that all involved nodes (@bs and all of
> >> - * its parents) are in the same AioContext, and that the caller has
> >> - * acquired it.
> >> - * If there are any nodes that are in different contexts from @bs,
> >> - * these contexts must not be acquired.
> >> + * the other, this requires that the caller either runs in the main
> >> + * loop; or that all involved nodes (@bs and all of its parents) are
> >> + * in the caller's AioContext.
> >>   */
> >>  void bdrv_drained_end(BlockDriverState *bs);
> > 
> > I think you are right about the requirement that bdrv_drained_end() is
> > only called from the main or the BDS AioContext, which is a requirement
> > that directly comes from AIO_WAIT_WHILE().
> > 
> > However, I don't see why we have requirements on the AioContext of the
> > parent nodes (or any other nodes), except possibly not holding their
> > lock.  We don't poll their context, so it shouldn't matter in which
> > context they are?
> 
> Hm.  I don’t know how I got confused there, you’re right.
> 
> I don’t think the (falsely given) restriction hurts, though, because a
> subtree should be within a single context anyway (unless you’re in
> bdrv_set_aio_context_ignore(), which needs to be in the main context).
> 
> So, hm, yes, I messed up this comment a bit now.  But now it’s just more
> restrictive than it needs to be and I don’t think callers are going to
> care, so...

Nothing that should hold up your pull request, but I'd prefer to fix the
comment in a follow-up.

One thing where I could imagine it becoming relevant in the future is
cross-context block jobs. At the moment, we automatically pull the
target node into the AioContext of the source and fail if this isn't
possible, but that's really overly restrictive.

Kevin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [Qemu-devel] [PATCH for-4.1 2/2] block: Only the main loop can change AioContexts

[Max Reitz]

[-- Attachment #1.1: Type: text/plain, Size: 4024 bytes --]

On 23.07.19 12:02, Kevin Wolf wrote:
> Am 23.07.2019 um 11:41 hat Max Reitz geschrieben:
>> On 23.07.19 10:52, Kevin Wolf wrote:
>>> Am 22.07.2019 um 15:30 hat Max Reitz geschrieben:
>>>> bdrv_set_aio_context_ignore() can only work in the main loop:
>>>> bdrv_drained_begin() only works in the main loop and the node's (old)
>>>> AioContext; and bdrv_drained_end() really only works in the main loop
>>>> and the node's (new) AioContext (contrary to its current comment, which
>>>> is just wrong).
>>>>
>>>> Consequentially, bdrv_set_aio_context_ignore() must be called from the
>>>> main loop.  Luckily, assuming that we can make block graph changes only
>>>> from the main loop as well, all its callers do that already.
>>>>
>>>> Note that changing a node's context in a sense is an operation that
>>>> changes the block graph, so it actually makes sense to require this
>>>> function to be called from the main loop.
>>>>
>>>> Also, fix bdrv_drained_end()'s description.  You can only use it from
>>>> the main loop or the node's AioContext, and in the latter case, the
>>>> whole subtree must be in the same context.
>>>>
>>>> Fixes: e037c09c78520cbdb6da7cfc6ad0256d5870b814
>>>> Signed-off-by: Max Reitz <mreitz@redhat.com>
>>>> ---
>>>>  include/block/block.h |  8 +++-----
>>>>  block.c               | 13 ++++++++-----
>>>>  2 files changed, 11 insertions(+), 10 deletions(-)
>>>>
>>>> diff --git a/include/block/block.h b/include/block/block.h
>>>> index 60f00479e0..50a07c1c33 100644
>>>> --- a/include/block/block.h
>>>> +++ b/include/block/block.h
>>>> @@ -667,11 +667,9 @@ void bdrv_subtree_drained_begin(BlockDriverState *bs);
>>>>   *
>>>>   * This polls @bs's AioContext until all scheduled sub-drained_ends
>>>>   * have settled.  On one hand, that may result in graph changes.  On
>>>> - * the other, this requires that all involved nodes (@bs and all of
>>>> - * its parents) are in the same AioContext, and that the caller has
>>>> - * acquired it.
>>>> - * If there are any nodes that are in different contexts from @bs,
>>>> - * these contexts must not be acquired.
>>>> + * the other, this requires that the caller either runs in the main
>>>> + * loop; or that all involved nodes (@bs and all of its parents) are
>>>> + * in the caller's AioContext.
>>>>   */
>>>>  void bdrv_drained_end(BlockDriverState *bs);
>>>
>>> I think you are right about the requirement that bdrv_drained_end() is
>>> only called from the main or the BDS AioContext, which is a requirement
>>> that directly comes from AIO_WAIT_WHILE().
>>>
>>> However, I don't see why we have requirements on the AioContext of the
>>> parent nodes (or any other nodes), except possibly not holding their
>>> lock.  We don't poll their context, so it shouldn't matter in which
>>> context they are?
>>
>> Hm.  I don’t know how I got confused there, you’re right.
>>
>> I don’t think the (falsely given) restriction hurts, though, because a
>> subtree should be within a single context anyway (unless you’re in
>> bdrv_set_aio_context_ignore(), which needs to be in the main context).
>>
>> So, hm, yes, I messed up this comment a bit now.  But now it’s just more
>> restrictive than it needs to be and I don’t think callers are going to
>> care, so...
> 
> Nothing that should hold up your pull request, but I'd prefer to fix the
> comment in a follow-up.

On second thought, does aio_wait_kick() wake up any context but the main
context?  I was under the impression that it doesn’t.  If it doesn’t, I
don’t know how bdrv_drained_end()’s AIO_WAIT_WHILE() will be woken up if
it doesn’t run in the main context and it has to wait for yet another
thread.

Max

> One thing where I could imagine it becoming relevant in the future is
> cross-context block jobs. At the moment, we automatically pull the
> target node into the AioContext of the source and fail if this isn't
> possible, but that's really overly restrictive.
> 
> Kevin


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [Qemu-devel] [PATCH for-4.1 2/2] block: Only the main loop can change AioContexts

[Kevin Wolf]

[-- Attachment #1: Type: text/plain, Size: 4146 bytes --]

Am 23.07.2019 um 12:21 hat Max Reitz geschrieben:
> On 23.07.19 12:02, Kevin Wolf wrote:
> > Am 23.07.2019 um 11:41 hat Max Reitz geschrieben:
> >> On 23.07.19 10:52, Kevin Wolf wrote:
> >>> Am 22.07.2019 um 15:30 hat Max Reitz geschrieben:
> >>>> bdrv_set_aio_context_ignore() can only work in the main loop:
> >>>> bdrv_drained_begin() only works in the main loop and the node's (old)
> >>>> AioContext; and bdrv_drained_end() really only works in the main loop
> >>>> and the node's (new) AioContext (contrary to its current comment, which
> >>>> is just wrong).
> >>>>
> >>>> Consequentially, bdrv_set_aio_context_ignore() must be called from the
> >>>> main loop.  Luckily, assuming that we can make block graph changes only
> >>>> from the main loop as well, all its callers do that already.
> >>>>
> >>>> Note that changing a node's context in a sense is an operation that
> >>>> changes the block graph, so it actually makes sense to require this
> >>>> function to be called from the main loop.
> >>>>
> >>>> Also, fix bdrv_drained_end()'s description.  You can only use it from
> >>>> the main loop or the node's AioContext, and in the latter case, the
> >>>> whole subtree must be in the same context.
> >>>>
> >>>> Fixes: e037c09c78520cbdb6da7cfc6ad0256d5870b814
> >>>> Signed-off-by: Max Reitz <mreitz@redhat.com>
> >>>> ---
> >>>>  include/block/block.h |  8 +++-----
> >>>>  block.c               | 13 ++++++++-----
> >>>>  2 files changed, 11 insertions(+), 10 deletions(-)
> >>>>
> >>>> diff --git a/include/block/block.h b/include/block/block.h
> >>>> index 60f00479e0..50a07c1c33 100644
> >>>> --- a/include/block/block.h
> >>>> +++ b/include/block/block.h
> >>>> @@ -667,11 +667,9 @@ void bdrv_subtree_drained_begin(BlockDriverState *bs);
> >>>>   *
> >>>>   * This polls @bs's AioContext until all scheduled sub-drained_ends
> >>>>   * have settled.  On one hand, that may result in graph changes.  On
> >>>> - * the other, this requires that all involved nodes (@bs and all of
> >>>> - * its parents) are in the same AioContext, and that the caller has
> >>>> - * acquired it.
> >>>> - * If there are any nodes that are in different contexts from @bs,
> >>>> - * these contexts must not be acquired.
> >>>> + * the other, this requires that the caller either runs in the main
> >>>> + * loop; or that all involved nodes (@bs and all of its parents) are
> >>>> + * in the caller's AioContext.
> >>>>   */
> >>>>  void bdrv_drained_end(BlockDriverState *bs);
> >>>
> >>> I think you are right about the requirement that bdrv_drained_end() is
> >>> only called from the main or the BDS AioContext, which is a requirement
> >>> that directly comes from AIO_WAIT_WHILE().
> >>>
> >>> However, I don't see why we have requirements on the AioContext of the
> >>> parent nodes (or any other nodes), except possibly not holding their
> >>> lock.  We don't poll their context, so it shouldn't matter in which
> >>> context they are?
> >>
> >> Hm.  I don’t know how I got confused there, you’re right.
> >>
> >> I don’t think the (falsely given) restriction hurts, though, because a
> >> subtree should be within a single context anyway (unless you’re in
> >> bdrv_set_aio_context_ignore(), which needs to be in the main context).
> >>
> >> So, hm, yes, I messed up this comment a bit now.  But now it’s just more
> >> restrictive than it needs to be and I don’t think callers are going to
> >> care, so...
> > 
> > Nothing that should hold up your pull request, but I'd prefer to fix the
> > comment in a follow-up.
> 
> On second thought, does aio_wait_kick() wake up any context but the main
> context?  I was under the impression that it doesn’t.  If it doesn’t, I
> don’t know how bdrv_drained_end()’s AIO_WAIT_WHILE() will be woken up if
> it doesn’t run in the main context and it has to wait for yet another
> thread.

Hm, I think you're right.

And your comment actually says main loop _or_ everything in the same
context, so it's correct (but I misread it at first and thought the
condition would always apply).

Kevin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [Qemu-devel] [PATCH for-4.1 0/2] block: bdrv_drained_end() changes fallout

[Max Reitz]

[-- Attachment #1.1.1: Type: text/plain, Size: 2830 bytes --]

On 22.07.19 15:30, Max Reitz wrote:
> Hi,
> 
> I noted that test-bdrv-drain sometimgs hangs (very rarely, though), and
> tried to write a test that triggers the issue.  I failed to do so (there
> is a good reason for that, see patch 1), but on my way I noticed that
> calling bdrv_set_aio_context_ignore() from any AioContext but the main
> one is a bad idea.  Hence patch 2.
> 
> Anyway, I found the problem, which is fixed by patch 1 -- I think it’s
> rather obvious.  There is no dedicated test because I don’t think it’s
> possible to write one, as I explain there.
> 
> 
> Max Reitz (2):
>   block: Dec. drained_end_counter before bdrv_wakeup
>   block: Only the main loop can change AioContexts
> 
>  include/block/block.h |  8 +++-----
>  block.c               | 13 ++++++++-----
>  block/io.c            |  5 ++---
>  3 files changed, 13 insertions(+), 13 deletions(-)

Sorry, applied to my block branch.


“Sorry” obviously because I didn’t really give much time to review this
series.  My justification is:

- rc2’s tomorrow.  I know the current code is broken, so I’d rather take
  the chance to have a fixed rc2 than to know it to be broken and force
  an rc4 by sending this for rc3.

- Patch 1 looks really obvious and simple to me.  It makes sense to
  decrement the drained_end_counter exactly where .done is set to true.

- Patch 2 is not so obvious.  But the only dangerous change it
  introduces is an assertion that bdrv_set_aio_context_ignore() is
  called from the main loop.  Right now, if it is called from anywhere
  but the main loop, we *will* run into assertions elsewhere:
  - bdrv_drained_begin() does a BDRV_POLL_WHILE(bs, ...).  This only
    works from the main context or bs's context.
  - bdrv_drained_end() does the same, but now bs's context has changed.
  Ergo, right now (on master), you can run bdrv_set_aio_context_ignore()
  only from the main loop anyway.  The new assertion only makes that
  fact more explicit.

Now this makes it look like before e037c09c78520, you could run
bdrv_set_aio_context_ignore() from the old context (like the comment
therein said).  But that’s wrong.  Even before e037c09c78520,
bdrv_drained_end() didn’t work for mixed-context trees unless you call
it from the main context: It schedules the drained_end callbacks in the
respective node's context, but then it polls them from the original
context.  So if you modify e.g. test_set_aio_context() in
test-bdrv-drain to run bdrv_try_set_aio_context() from a BH in the old
context, you will see a failed assertion in bdrv_drain_invoke()'s
BDRV_POLL_WHILE.  (I’ve attached a diff for use on e037c09c78520^.)

Therefore, I’m confident this series doesn’t make things worse, which is
why I’m taking it without reviews.

Max

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1.2: test-bdrv-drain.diff --]
[-- Type: text/x-patch; name="test-bdrv-drain.diff", Size: 2000 bytes --]

diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c
index 3503ce3b69..cf1194754e 100644
--- a/tests/test-bdrv-drain.c
+++ b/tests/test-bdrv-drain.c
@@ -1497,6 +1497,19 @@ static void test_append_to_drained(void)
     blk_unref(blk);
 }
 
+typedef struct BHParams {
+    BlockDriverState *bs;
+    AioContext *target;
+    bool done;
+} BHParams;
+
+static void bh_fun(void *opaque)
+{
+    BHParams *bhp = opaque;
+    bdrv_try_set_aio_context(bhp->bs, bhp->target, &error_abort);
+    bhp->done = true;
+}
+
 static void test_set_aio_context(void)
 {
     BlockDriverState *bs;
@@ -1504,22 +1517,38 @@ static void test_set_aio_context(void)
     IOThread *b = iothread_new();
     AioContext *ctx_a = iothread_get_aio_context(a);
     AioContext *ctx_b = iothread_get_aio_context(b);
+    BHParams bhp;
 
     bs = bdrv_new_open_driver(&bdrv_test, "test-node", BDRV_O_RDWR,
                               &error_abort);
 
     bdrv_drained_begin(bs);
-    bdrv_try_set_aio_context(bs, ctx_a, &error_abort);
+    bhp = (BHParams){ bs, ctx_a };
+    aio_bh_schedule_oneshot(qemu_get_aio_context(), bh_fun, &bhp);
+    while (!bhp.done) {
+        aio_poll(qemu_get_aio_context(), true);
+    }
 
     aio_context_acquire(ctx_a);
     bdrv_drained_end(bs);
 
     bdrv_drained_begin(bs);
-    bdrv_try_set_aio_context(bs, ctx_b, &error_abort);
+
+    bhp = (BHParams){ bs, ctx_b };
+    aio_bh_schedule_oneshot(ctx_a, bh_fun, &bhp);
     aio_context_release(ctx_a);
+    while (!bhp.done) {
+        aio_poll(qemu_get_aio_context(), true);
+    }
+
     aio_context_acquire(ctx_b);
-    bdrv_try_set_aio_context(bs, qemu_get_aio_context(), &error_abort);
+    bhp = (BHParams){ bs, qemu_get_aio_context() };
+    aio_bh_schedule_oneshot(ctx_b, bh_fun, &bhp);
     aio_context_release(ctx_b);
+    while (!bhp.done) {
+        aio_poll(qemu_get_aio_context(), true);
+    }
+
     bdrv_drained_end(bs);
 
     bdrv_unref(bs);

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]