[Qemu-devel] ahci-test failures with 'Failed to get "write" lock'

[Qemu-devel] ahci-test failures with 'Failed to get "write" lock'

[Dr. David Alan Gilbert]

Hi,
  Peter noticed that there were some hangs in the ahci-test;
I can reproduce them on my laptop as long as the laptop has
some other load on it (a few md5sum /dev/zero's help).

/x86_64/ahci/flush/migrate: qemu-system-x86_64: Failed to get "write" lock
Is another process using the image?
OK
/x86_64/ahci/migrate/sanity: qemu-system-x86_64: Failed to get "write" lock
Is another process using the image?

and that hangs in sanity.

That seems to be the common case, I've also had one:

/x86_64/ahci/migrate/ncq/halted: qemu-system-x86_64: Failed to get "write" lock
Is another process using the image?
qemu-system-x86_64: /home/dgilbert/git/qemu/block/io.c:1352: bdrv_aligned_pwritev: Assertion `child->perm & BLK_PERM_WRITE' failed.


Neither of these always fail.

Dave
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] ahci-test failures with 'Failed to get "write" lock'

[John Snow]

On 06/15/2017 11:28 AM, Dr. David Alan Gilbert wrote:
> Hi,
>   Peter noticed that there were some hangs in the ahci-test;
> I can reproduce them on my laptop as long as the laptop has
> some other load on it (a few md5sum /dev/zero's help).
> 
> /x86_64/ahci/flush/migrate: qemu-system-x86_64: Failed to get "write" lock
> Is another process using the image?
> OK
> /x86_64/ahci/migrate/sanity: qemu-system-x86_64: Failed to get "write" lock
> Is another process using the image?
> 
> and that hangs in sanity.
> 
> That seems to be the common case, I've also had one:
> 
> /x86_64/ahci/migrate/ncq/halted: qemu-system-x86_64: Failed to get "write" lock
> Is another process using the image?
> qemu-system-x86_64: /home/dgilbert/git/qemu/block/io.c:1352: bdrv_aligned_pwritev: Assertion `child->perm & BLK_PERM_WRITE' failed.
> 
> 
> Neither of these always fail.
> 
> Dave
> --
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> 

Haven't observed myself and I run the tests pretty frequently. I'll keep
an eye out. I see many similar problems in iotests for other areas, too.

AHCI tests *should* never overlap and *should* never be using images in
an unsafe way, and do a full set-up and tear-down for each individual
unit test.

Of course, maybe the migrate code isn't behaving quite so nicely with
loopback migrations...?

Re: [Qemu-devel] ahci-test failures with 'Failed to get "write" lock'

[Peter Maydell]

On 15 June 2017 at 17:16, John Snow <jsnow@redhat.com> wrote:
> On 06/15/2017 11:28 AM, Dr. David Alan Gilbert wrote:
>> Hi,
>>   Peter noticed that there were some hangs in the ahci-test;
>> I can reproduce them on my laptop as long as the laptop has
>> some other load on it (a few md5sum /dev/zero's help).
>>
>> /x86_64/ahci/flush/migrate: qemu-system-x86_64: Failed to get "write" lock
>> Is another process using the image?
>> OK
>> /x86_64/ahci/migrate/sanity: qemu-system-x86_64: Failed to get "write" lock
>> Is another process using the image?
>>
>> and that hangs in sanity.
>>
>> That seems to be the common case, I've also had one:
>>
>> /x86_64/ahci/migrate/ncq/halted: qemu-system-x86_64: Failed to get "write" lock
>> Is another process using the image?
>> qemu-system-x86_64: /home/dgilbert/git/qemu/block/io.c:1352: bdrv_aligned_pwritev: Assertion `child->perm & BLK_PERM_WRITE' failed.
>>
>>
>> Neither of these always fail.

> Haven't observed myself and I run the tests pretty frequently. I'll keep
> an eye out. I see many similar problems in iotests for other areas, too.

If we can figure out this quickly that would be good -- it's
causing my build tests to fail often enough that I'm a bit
reluctant to try to merge more pull requests until we've
squished this bug...

thanks
-- PMM

Re: [Qemu-devel] ahci-test failures with 'Failed to get "write" lock'

[Fam Zheng]

On Fri, 06/16 10:23, Peter Maydell wrote:
> On 15 June 2017 at 17:16, John Snow <jsnow@redhat.com> wrote:
> > On 06/15/2017 11:28 AM, Dr. David Alan Gilbert wrote:
> >> Hi,
> >>   Peter noticed that there were some hangs in the ahci-test;
> >> I can reproduce them on my laptop as long as the laptop has
> >> some other load on it (a few md5sum /dev/zero's help).
> >>
> >> /x86_64/ahci/flush/migrate: qemu-system-x86_64: Failed to get "write" lock
> >> Is another process using the image?
> >> OK
> >> /x86_64/ahci/migrate/sanity: qemu-system-x86_64: Failed to get "write" lock
> >> Is another process using the image?
> >>
> >> and that hangs in sanity.
> >>
> >> That seems to be the common case, I've also had one:
> >>
> >> /x86_64/ahci/migrate/ncq/halted: qemu-system-x86_64: Failed to get "write" lock
> >> Is another process using the image?
> >> qemu-system-x86_64: /home/dgilbert/git/qemu/block/io.c:1352: bdrv_aligned_pwritev: Assertion `child->perm & BLK_PERM_WRITE' failed.
> >>
> >>
> >> Neither of these always fail.
> 
> > Haven't observed myself and I run the tests pretty frequently. I'll keep
> > an eye out. I see many similar problems in iotests for other areas, too.
> 
> If we can figure out this quickly that would be good -- it's
> causing my build tests to fail often enough that I'm a bit
> reluctant to try to merge more pull requests until we've
> squished this bug...

I have posted a patch:

[PATCH] migration: Fix race of image locking between src and dst

(You are Cc'ed)

Fam