* iotest failure -- test possibly not using sufficiently unique temp filename? @ 2019-09-27 16:39 Peter Maydell 2019-09-27 16:44 ` Max Reitz 0 siblings, 1 reply; 6+ messages in thread From: Peter Maydell @ 2019-09-27 16:39 UTC (permalink / raw) To: QEMU Developers; +Cc: Kevin Wolf, Qemu-block, Max Reitz Hi; I just saw this iotest failure (on an s390x box, as it happens): TEST iotest-qcow2: 130 [fail] QEMU -- "/home/linux1/qemu/build/all/tests/qemu-iotests/../../s390x-softmmu/qemu-system-s390x" -nodefaults -display none -machine accel=qtest QEMU_IMG -- "/home/linux1/qemu/build/all/tests/qemu-iotests/../../qemu-img" QEMU_IO -- "/home/linux1/qemu/build/all/tests/qemu-iotests/../../qemu-io" --cache writeback -f qcow2 QEMU_NBD -- "/home/linux1/qemu/build/all/tests/qemu-iotests/../../qemu-nbd" IMGFMT -- qcow2 (compat=1.1) IMGPROTO -- file PLATFORM -- Linux/s390x lxub05 4.15.0-58-generic TEST_DIR -- /home/linux1/qemu/build/all/tests/qemu-iotests/scratch SOCKET_SCM_HELPER -- /home/linux1/qemu/build/all/tests/qemu-iotests/socket_scm_helper --- /home/linux1/qemu/tests/qemu-iotests/130.out 2019-05-10 12:27:16.948075733 -0400 +++ /home/linux1/qemu/build/all/tests/qemu-iotests/130.out.bad 2019-09-27 12:01:23.649722655 -0400 @@ -18,20 +18,22 @@ QEMU X.Y.Z monitor - type 'help' for more information (qemu) commit testdisk (qemu) -image: TEST_DIR/t.IMGFMT -file format: IMGFMT -virtual size: 64 MiB (67108864 bytes) -backing file: TEST_DIR/t.IMGFMT.orig -backing file format: raw +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Failed to get shared "write" lock +Is another process using the image [TEST_DIR/t.IMGFMT]? === Marking image dirty (lazy refcounts) === +qemu-img: TEST_DIR/t.IMGFMT: Failed to get "write" lock +Is another process using the image [TEST_DIR/t.IMGFMT]? Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 -wrote 4096/4096 bytes at offset 0 -4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +qemu-io: can't open device /home/linux1/qemu/build/all/tests/qemu-iotests/scratch/t.qcow2: Failed to get "write" lock +Is another process using the image [/home/linux1/qemu/build/all/tests/qemu-iotests/scratch/t.qcow2]? +no file open, try 'help open' image: TEST_DIR/t.IMGFMT file format: IMGFMT virtual size: 64 MiB (67108864 bytes) +backing file: TEST_DIR/t.IMGFMT.orig +backing file format: raw Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.orig backing_fmt=raw wrote 4096/4096 bytes at offset 0 4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) This looks suspiciously like the test isn't using a unique filename for its disk image: "qemu-iotests/scratch/t.qcow2" in the build directory, and so perhaps it has collided with another iotest ? If we run 'make check' with a -j<something> option do the iotests all get run serially anyway, or do they run in parallel against each other ? thanks -- PMM ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: iotest failure -- test possibly not using sufficiently unique temp filename? 2019-09-27 16:39 iotest failure -- test possibly not using sufficiently unique temp filename? Peter Maydell @ 2019-09-27 16:44 ` Max Reitz 2019-10-17 16:41 ` Peter Maydell 0 siblings, 1 reply; 6+ messages in thread From: Max Reitz @ 2019-09-27 16:44 UTC (permalink / raw) To: Peter Maydell, QEMU Developers; +Cc: Kevin Wolf, Thomas Huth, Qemu-block [-- Attachment #1.1: Type: text/plain, Size: 3365 bytes --] On 27.09.19 18:39, Peter Maydell wrote: > Hi; I just saw this iotest failure (on an s390x box, as it happens): > > TEST iotest-qcow2: 130 [fail] > QEMU -- > "/home/linux1/qemu/build/all/tests/qemu-iotests/../../s390x-softmmu/qemu-system-s390x" > -nodefaults -display none -machine accel=qtest > QEMU_IMG -- "/home/linux1/qemu/build/all/tests/qemu-iotests/../../qemu-img" > QEMU_IO -- > "/home/linux1/qemu/build/all/tests/qemu-iotests/../../qemu-io" > --cache writeback -f qcow2 > QEMU_NBD -- "/home/linux1/qemu/build/all/tests/qemu-iotests/../../qemu-nbd" > IMGFMT -- qcow2 (compat=1.1) > IMGPROTO -- file > PLATFORM -- Linux/s390x lxub05 4.15.0-58-generic > TEST_DIR -- /home/linux1/qemu/build/all/tests/qemu-iotests/scratch > SOCKET_SCM_HELPER -- > /home/linux1/qemu/build/all/tests/qemu-iotests/socket_scm_helper > > --- /home/linux1/qemu/tests/qemu-iotests/130.out 2019-05-10 > 12:27:16.948075733 -0400 > +++ /home/linux1/qemu/build/all/tests/qemu-iotests/130.out.bad > 2019-09-27 12:01:23.649722655 -0400 > @@ -18,20 +18,22 @@ > QEMU X.Y.Z monitor - type 'help' for more information > (qemu) commit testdisk > (qemu) > -image: TEST_DIR/t.IMGFMT > -file format: IMGFMT > -virtual size: 64 MiB (67108864 bytes) > -backing file: TEST_DIR/t.IMGFMT.orig > -backing file format: raw > +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Failed to get shared "write" lock > +Is another process using the image [TEST_DIR/t.IMGFMT]? > > === Marking image dirty (lazy refcounts) === > > +qemu-img: TEST_DIR/t.IMGFMT: Failed to get "write" lock > +Is another process using the image [TEST_DIR/t.IMGFMT]? > Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 > -wrote 4096/4096 bytes at offset 0 > -4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +qemu-io: can't open device > /home/linux1/qemu/build/all/tests/qemu-iotests/scratch/t.qcow2: Failed > to get "write" lock > +Is another process using the image > [/home/linux1/qemu/build/all/tests/qemu-iotests/scratch/t.qcow2]? > +no file open, try 'help open' > image: TEST_DIR/t.IMGFMT > file format: IMGFMT > virtual size: 64 MiB (67108864 bytes) > +backing file: TEST_DIR/t.IMGFMT.orig > +backing file format: raw > Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 > backing_file=TEST_DIR/t.IMGFMT.orig backing_fmt=raw > wrote 4096/4096 bytes at offset 0 > 4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > > > > This looks suspiciously like the test isn't using a unique > filename for its disk image: "qemu-iotests/scratch/t.qcow2" > in the build directory, and so perhaps it has collided with > another iotest ? > > If we run 'make check' with a -j<something> option do the > iotests all get run serially anyway, or do they run in > parallel against each other ? As far as I know, all iotests are executed serially. Anything else would not work with the same scratch directory. The only thing I suspect is that some tool has been accidentally left running by some previous test that still accesses its own image. But I don’t know. Max (I’m personally still very skeptical of running some iotests in make check, so I don’t have an opinion on the matter other than “If it fails, we should remove it from make check”.) [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: iotest failure -- test possibly not using sufficiently unique temp filename? 2019-09-27 16:44 ` Max Reitz @ 2019-10-17 16:41 ` Peter Maydell 2019-10-18 6:20 ` Thomas Huth 0 siblings, 1 reply; 6+ messages in thread From: Peter Maydell @ 2019-10-17 16:41 UTC (permalink / raw) To: Max Reitz; +Cc: Kevin Wolf, Thomas Huth, QEMU Developers, Qemu-block On Fri, 27 Sep 2019 at 17:44, Max Reitz <mreitz@redhat.com> wrote: > > On 27.09.19 18:39, Peter Maydell wrote: > > Hi; I just saw this iotest failure (on an s390x box, as it happens): > > > > TEST iotest-qcow2: 130 [fail] > > QEMU -- > > "/home/linux1/qemu/build/all/tests/qemu-iotests/../../s390x-softmmu/qemu-system-s390x" > > -nodefaults -display none -machine accel=qtest > > QEMU_IMG -- "/home/linux1/qemu/build/all/tests/qemu-iotests/../../qemu-img" > > QEMU_IO -- > > "/home/linux1/qemu/build/all/tests/qemu-iotests/../../qemu-io" > > --cache writeback -f qcow2 > > QEMU_NBD -- "/home/linux1/qemu/build/all/tests/qemu-iotests/../../qemu-nbd" > > IMGFMT -- qcow2 (compat=1.1) > > IMGPROTO -- file > > PLATFORM -- Linux/s390x lxub05 4.15.0-58-generic > > TEST_DIR -- /home/linux1/qemu/build/all/tests/qemu-iotests/scratch > > SOCKET_SCM_HELPER -- > > /home/linux1/qemu/build/all/tests/qemu-iotests/socket_scm_helper > > > > --- /home/linux1/qemu/tests/qemu-iotests/130.out 2019-05-10 > > 12:27:16.948075733 -0400 > > +++ /home/linux1/qemu/build/all/tests/qemu-iotests/130.out.bad > > 2019-09-27 12:01:23.649722655 -0400 > > @@ -18,20 +18,22 @@ > > QEMU X.Y.Z monitor - type 'help' for more information > > (qemu) commit testdisk > > (qemu) > > -image: TEST_DIR/t.IMGFMT > > -file format: IMGFMT > > -virtual size: 64 MiB (67108864 bytes) > > -backing file: TEST_DIR/t.IMGFMT.orig > > -backing file format: raw > > +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Failed to get shared "write" lock > > +Is another process using the image [TEST_DIR/t.IMGFMT]? > > > > === Marking image dirty (lazy refcounts) === > > > > +qemu-img: TEST_DIR/t.IMGFMT: Failed to get "write" lock > > +Is another process using the image [TEST_DIR/t.IMGFMT]? > > Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 > > -wrote 4096/4096 bytes at offset 0 > > -4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > > +qemu-io: can't open device > > /home/linux1/qemu/build/all/tests/qemu-iotests/scratch/t.qcow2: Failed > > to get "write" lock > > +Is another process using the image > > [/home/linux1/qemu/build/all/tests/qemu-iotests/scratch/t.qcow2]? > > +no file open, try 'help open' > > image: TEST_DIR/t.IMGFMT > > file format: IMGFMT > > virtual size: 64 MiB (67108864 bytes) > > +backing file: TEST_DIR/t.IMGFMT.orig > > +backing file format: raw > > Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 > > backing_file=TEST_DIR/t.IMGFMT.orig backing_fmt=raw > > wrote 4096/4096 bytes at offset 0 > > 4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > > > > > > > > This looks suspiciously like the test isn't using a unique > > filename for its disk image: "qemu-iotests/scratch/t.qcow2" > > in the build directory, and so perhaps it has collided with > > another iotest ? > > > > If we run 'make check' with a -j<something> option do the > > iotests all get run serially anyway, or do they run in > > parallel against each other ? > > As far as I know, all iotests are executed serially. Anything else > would not work with the same scratch directory. > > The only thing I suspect is that some tool has been accidentally left > running by some previous test that still accesses its own image. But I > don’t know. Just saw this one again with the same iotest 130 on the same s390 box; only difference is that the log this time around has the first part where qemu-img fails, but not the second part where qemu-io fails: --- /home/linux1/qemu/tests/qemu-iotests/130.out 2019-05-10 12:27:16.948075733 -0400 +++ /home/linux1/qemu/build/all/tests/qemu-iotests/130.out.bad 2019-10-17 11:56:43.450750873 -0400 @@ -18,11 +18,8 @@ QEMU X.Y.Z monitor - type 'help' for more information (qemu) commit testdisk (qemu) -image: TEST_DIR/t.IMGFMT -file format: IMGFMT -virtual size: 64 MiB (67108864 bytes) -backing file: TEST_DIR/t.IMGFMT.orig -backing file format: raw +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Failed to get shared "write" lock +Is another process using the image [TEST_DIR/t.IMGFMT]? === Marking image dirty (lazy refcounts) === On the host machine there don't seem to be any stray processes which might have held the file open, and indeed the file doesn't exist at all, so it got removed by some cleanup or other. thanks -- PMM ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: iotest failure -- test possibly not using sufficiently unique temp filename? 2019-10-17 16:41 ` Peter Maydell @ 2019-10-18 6:20 ` Thomas Huth 2019-10-18 8:42 ` Max Reitz 0 siblings, 1 reply; 6+ messages in thread From: Thomas Huth @ 2019-10-18 6:20 UTC (permalink / raw) To: Peter Maydell, Max Reitz; +Cc: Kevin Wolf, QEMU Developers, Qemu-block On 17/10/2019 18.41, Peter Maydell wrote: > On Fri, 27 Sep 2019 at 17:44, Max Reitz <mreitz@redhat.com> wrote: >> >> On 27.09.19 18:39, Peter Maydell wrote: >>> Hi; I just saw this iotest failure (on an s390x box, as it happens): >>> >>> TEST iotest-qcow2: 130 [fail] >>> QEMU -- >>> "/home/linux1/qemu/build/all/tests/qemu-iotests/../../s390x-softmmu/qemu-system-s390x" >>> -nodefaults -display none -machine accel=qtest >>> QEMU_IMG -- "/home/linux1/qemu/build/all/tests/qemu-iotests/../../qemu-img" >>> QEMU_IO -- >>> "/home/linux1/qemu/build/all/tests/qemu-iotests/../../qemu-io" >>> --cache writeback -f qcow2 >>> QEMU_NBD -- "/home/linux1/qemu/build/all/tests/qemu-iotests/../../qemu-nbd" >>> IMGFMT -- qcow2 (compat=1.1) >>> IMGPROTO -- file >>> PLATFORM -- Linux/s390x lxub05 4.15.0-58-generic >>> TEST_DIR -- /home/linux1/qemu/build/all/tests/qemu-iotests/scratch >>> SOCKET_SCM_HELPER -- >>> /home/linux1/qemu/build/all/tests/qemu-iotests/socket_scm_helper >>> >>> --- /home/linux1/qemu/tests/qemu-iotests/130.out 2019-05-10 >>> 12:27:16.948075733 -0400 >>> +++ /home/linux1/qemu/build/all/tests/qemu-iotests/130.out.bad >>> 2019-09-27 12:01:23.649722655 -0400 >>> @@ -18,20 +18,22 @@ >>> QEMU X.Y.Z monitor - type 'help' for more information >>> (qemu) commit testdisk >>> (qemu) >>> -image: TEST_DIR/t.IMGFMT >>> -file format: IMGFMT >>> -virtual size: 64 MiB (67108864 bytes) >>> -backing file: TEST_DIR/t.IMGFMT.orig >>> -backing file format: raw >>> +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Failed to get shared "write" lock >>> +Is another process using the image [TEST_DIR/t.IMGFMT]? >>> >>> === Marking image dirty (lazy refcounts) === >>> >>> +qemu-img: TEST_DIR/t.IMGFMT: Failed to get "write" lock >>> +Is another process using the image [TEST_DIR/t.IMGFMT]? >>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 >>> -wrote 4096/4096 bytes at offset 0 >>> -4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >>> +qemu-io: can't open device >>> /home/linux1/qemu/build/all/tests/qemu-iotests/scratch/t.qcow2: Failed >>> to get "write" lock >>> +Is another process using the image >>> [/home/linux1/qemu/build/all/tests/qemu-iotests/scratch/t.qcow2]? >>> +no file open, try 'help open' >>> image: TEST_DIR/t.IMGFMT >>> file format: IMGFMT >>> virtual size: 64 MiB (67108864 bytes) >>> +backing file: TEST_DIR/t.IMGFMT.orig >>> +backing file format: raw >>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 >>> backing_file=TEST_DIR/t.IMGFMT.orig backing_fmt=raw >>> wrote 4096/4096 bytes at offset 0 >>> 4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >>> >>> >>> >>> This looks suspiciously like the test isn't using a unique >>> filename for its disk image: "qemu-iotests/scratch/t.qcow2" >>> in the build directory, and so perhaps it has collided with >>> another iotest ? >>> >>> If we run 'make check' with a -j<something> option do the >>> iotests all get run serially anyway, or do they run in >>> parallel against each other ? >> >> As far as I know, all iotests are executed serially. Anything else >> would not work with the same scratch directory. >> >> The only thing I suspect is that some tool has been accidentally left >> running by some previous test that still accesses its own image. But I >> don’t know. > > Just saw this one again with the same iotest 130 on the same > s390 box; only difference is that the log this time around > has the first part where qemu-img fails, but not the second part > where qemu-io fails: > > --- /home/linux1/qemu/tests/qemu-iotests/130.out 2019-05-10 > 12:27:16.948075733 -0400 > +++ /home/linux1/qemu/build/all/tests/qemu-iotests/130.out.bad > 2019-10-17 11:56:43.450750873 -0400 > @@ -18,11 +18,8 @@ > QEMU X.Y.Z monitor - type 'help' for more information > (qemu) commit testdisk > (qemu) > -image: TEST_DIR/t.IMGFMT > -file format: IMGFMT > -virtual size: 64 MiB (67108864 bytes) > -backing file: TEST_DIR/t.IMGFMT.orig > -backing file format: raw > +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Failed to get shared "write" lock > +Is another process using the image [TEST_DIR/t.IMGFMT]? > > === Marking image dirty (lazy refcounts) === > > On the host machine there don't seem to be any stray > processes which might have held the file open, and > indeed the file doesn't exist at all, so it got removed > by some cleanup or other. Ok, so unless someone has a clue what might be going on here (is there a race in the test?), I'd suggest that we simply remove 130 from the auto group again. Shall I send a patch? Thomas ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: iotest failure -- test possibly not using sufficiently unique temp filename? 2019-10-18 6:20 ` Thomas Huth @ 2019-10-18 8:42 ` Max Reitz 2019-10-18 11:43 ` Thomas Huth 0 siblings, 1 reply; 6+ messages in thread From: Max Reitz @ 2019-10-18 8:42 UTC (permalink / raw) To: Thomas Huth, Peter Maydell; +Cc: Kevin Wolf, QEMU Developers, Qemu-block [-- Attachment #1.1: Type: text/plain, Size: 5456 bytes --] On 18.10.19 08:20, Thomas Huth wrote: > On 17/10/2019 18.41, Peter Maydell wrote: >> On Fri, 27 Sep 2019 at 17:44, Max Reitz <mreitz@redhat.com> wrote: >>> >>> On 27.09.19 18:39, Peter Maydell wrote: >>>> Hi; I just saw this iotest failure (on an s390x box, as it happens): >>>> >>>> TEST iotest-qcow2: 130 [fail] >>>> QEMU -- >>>> "/home/linux1/qemu/build/all/tests/qemu-iotests/../../s390x-softmmu/qemu-system-s390x" >>>> -nodefaults -display none -machine accel=qtest >>>> QEMU_IMG -- "/home/linux1/qemu/build/all/tests/qemu-iotests/../../qemu-img" >>>> QEMU_IO -- >>>> "/home/linux1/qemu/build/all/tests/qemu-iotests/../../qemu-io" >>>> --cache writeback -f qcow2 >>>> QEMU_NBD -- "/home/linux1/qemu/build/all/tests/qemu-iotests/../../qemu-nbd" >>>> IMGFMT -- qcow2 (compat=1.1) >>>> IMGPROTO -- file >>>> PLATFORM -- Linux/s390x lxub05 4.15.0-58-generic >>>> TEST_DIR -- /home/linux1/qemu/build/all/tests/qemu-iotests/scratch >>>> SOCKET_SCM_HELPER -- >>>> /home/linux1/qemu/build/all/tests/qemu-iotests/socket_scm_helper >>>> >>>> --- /home/linux1/qemu/tests/qemu-iotests/130.out 2019-05-10 >>>> 12:27:16.948075733 -0400 >>>> +++ /home/linux1/qemu/build/all/tests/qemu-iotests/130.out.bad >>>> 2019-09-27 12:01:23.649722655 -0400 >>>> @@ -18,20 +18,22 @@ >>>> QEMU X.Y.Z monitor - type 'help' for more information >>>> (qemu) commit testdisk >>>> (qemu) >>>> -image: TEST_DIR/t.IMGFMT >>>> -file format: IMGFMT >>>> -virtual size: 64 MiB (67108864 bytes) >>>> -backing file: TEST_DIR/t.IMGFMT.orig >>>> -backing file format: raw >>>> +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Failed to get shared "write" lock >>>> +Is another process using the image [TEST_DIR/t.IMGFMT]? >>>> >>>> === Marking image dirty (lazy refcounts) === >>>> >>>> +qemu-img: TEST_DIR/t.IMGFMT: Failed to get "write" lock >>>> +Is another process using the image [TEST_DIR/t.IMGFMT]? >>>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 >>>> -wrote 4096/4096 bytes at offset 0 >>>> -4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >>>> +qemu-io: can't open device >>>> /home/linux1/qemu/build/all/tests/qemu-iotests/scratch/t.qcow2: Failed >>>> to get "write" lock >>>> +Is another process using the image >>>> [/home/linux1/qemu/build/all/tests/qemu-iotests/scratch/t.qcow2]? >>>> +no file open, try 'help open' >>>> image: TEST_DIR/t.IMGFMT >>>> file format: IMGFMT >>>> virtual size: 64 MiB (67108864 bytes) >>>> +backing file: TEST_DIR/t.IMGFMT.orig >>>> +backing file format: raw >>>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 >>>> backing_file=TEST_DIR/t.IMGFMT.orig backing_fmt=raw >>>> wrote 4096/4096 bytes at offset 0 >>>> 4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >>>> >>>> >>>> >>>> This looks suspiciously like the test isn't using a unique >>>> filename for its disk image: "qemu-iotests/scratch/t.qcow2" >>>> in the build directory, and so perhaps it has collided with >>>> another iotest ? >>>> >>>> If we run 'make check' with a -j<something> option do the >>>> iotests all get run serially anyway, or do they run in >>>> parallel against each other ? >>> >>> As far as I know, all iotests are executed serially. Anything else >>> would not work with the same scratch directory. >>> >>> The only thing I suspect is that some tool has been accidentally left >>> running by some previous test that still accesses its own image. But I >>> don’t know. >> >> Just saw this one again with the same iotest 130 on the same >> s390 box; only difference is that the log this time around >> has the first part where qemu-img fails, but not the second part >> where qemu-io fails: >> >> --- /home/linux1/qemu/tests/qemu-iotests/130.out 2019-05-10 >> 12:27:16.948075733 -0400 >> +++ /home/linux1/qemu/build/all/tests/qemu-iotests/130.out.bad >> 2019-10-17 11:56:43.450750873 -0400 >> @@ -18,11 +18,8 @@ >> QEMU X.Y.Z monitor - type 'help' for more information >> (qemu) commit testdisk >> (qemu) >> -image: TEST_DIR/t.IMGFMT >> -file format: IMGFMT >> -virtual size: 64 MiB (67108864 bytes) >> -backing file: TEST_DIR/t.IMGFMT.orig >> -backing file format: raw >> +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Failed to get shared "write" lock >> +Is another process using the image [TEST_DIR/t.IMGFMT]? >> >> === Marking image dirty (lazy refcounts) === >> >> On the host machine there don't seem to be any stray >> processes which might have held the file open, and >> indeed the file doesn't exist at all, so it got removed >> by some cleanup or other. > > Ok, so unless someone has a clue what might be going on here (is there a > race in the test?), I'd suggest that we simply remove 130 from the auto > group again. Shall I send a patch? I don’t have much of an idea. It looks like maybe the qemu process (which dos the commit) is lingering, but that shouldn’t be because _cleanup_qemu always waits for it. (Also, I can’t reproduce the problem on my system.) The only hunch that I have is that 130 seems to be the only test that uses _cleanup_qemu to kill a qemu process (i.e. without wait=1) while it has taken locks on an image and then still uses the image afterwards. Maybe making it quit qemu through HMP would fix the problem. But knowing is difficult because I can’t reproduce it. Max [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: iotest failure -- test possibly not using sufficiently unique temp filename? 2019-10-18 8:42 ` Max Reitz @ 2019-10-18 11:43 ` Thomas Huth 0 siblings, 0 replies; 6+ messages in thread From: Thomas Huth @ 2019-10-18 11:43 UTC (permalink / raw) To: Max Reitz, Peter Maydell; +Cc: Kevin Wolf, QEMU Developers, Qemu-block [-- Attachment #1.1: Type: text/plain, Size: 5815 bytes --] On 18/10/2019 10.42, Max Reitz wrote: > On 18.10.19 08:20, Thomas Huth wrote: >> On 17/10/2019 18.41, Peter Maydell wrote: >>> On Fri, 27 Sep 2019 at 17:44, Max Reitz <mreitz@redhat.com> wrote: >>>> >>>> On 27.09.19 18:39, Peter Maydell wrote: >>>>> Hi; I just saw this iotest failure (on an s390x box, as it happens): >>>>> >>>>> TEST iotest-qcow2: 130 [fail] >>>>> QEMU -- >>>>> "/home/linux1/qemu/build/all/tests/qemu-iotests/../../s390x-softmmu/qemu-system-s390x" >>>>> -nodefaults -display none -machine accel=qtest >>>>> QEMU_IMG -- "/home/linux1/qemu/build/all/tests/qemu-iotests/../../qemu-img" >>>>> QEMU_IO -- >>>>> "/home/linux1/qemu/build/all/tests/qemu-iotests/../../qemu-io" >>>>> --cache writeback -f qcow2 >>>>> QEMU_NBD -- "/home/linux1/qemu/build/all/tests/qemu-iotests/../../qemu-nbd" >>>>> IMGFMT -- qcow2 (compat=1.1) >>>>> IMGPROTO -- file >>>>> PLATFORM -- Linux/s390x lxub05 4.15.0-58-generic >>>>> TEST_DIR -- /home/linux1/qemu/build/all/tests/qemu-iotests/scratch >>>>> SOCKET_SCM_HELPER -- >>>>> /home/linux1/qemu/build/all/tests/qemu-iotests/socket_scm_helper >>>>> >>>>> --- /home/linux1/qemu/tests/qemu-iotests/130.out 2019-05-10 >>>>> 12:27:16.948075733 -0400 >>>>> +++ /home/linux1/qemu/build/all/tests/qemu-iotests/130.out.bad >>>>> 2019-09-27 12:01:23.649722655 -0400 >>>>> @@ -18,20 +18,22 @@ >>>>> QEMU X.Y.Z monitor - type 'help' for more information >>>>> (qemu) commit testdisk >>>>> (qemu) >>>>> -image: TEST_DIR/t.IMGFMT >>>>> -file format: IMGFMT >>>>> -virtual size: 64 MiB (67108864 bytes) >>>>> -backing file: TEST_DIR/t.IMGFMT.orig >>>>> -backing file format: raw >>>>> +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Failed to get shared "write" lock >>>>> +Is another process using the image [TEST_DIR/t.IMGFMT]? >>>>> >>>>> === Marking image dirty (lazy refcounts) === >>>>> >>>>> +qemu-img: TEST_DIR/t.IMGFMT: Failed to get "write" lock >>>>> +Is another process using the image [TEST_DIR/t.IMGFMT]? >>>>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 >>>>> -wrote 4096/4096 bytes at offset 0 >>>>> -4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >>>>> +qemu-io: can't open device >>>>> /home/linux1/qemu/build/all/tests/qemu-iotests/scratch/t.qcow2: Failed >>>>> to get "write" lock >>>>> +Is another process using the image >>>>> [/home/linux1/qemu/build/all/tests/qemu-iotests/scratch/t.qcow2]? >>>>> +no file open, try 'help open' >>>>> image: TEST_DIR/t.IMGFMT >>>>> file format: IMGFMT >>>>> virtual size: 64 MiB (67108864 bytes) >>>>> +backing file: TEST_DIR/t.IMGFMT.orig >>>>> +backing file format: raw >>>>> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 >>>>> backing_file=TEST_DIR/t.IMGFMT.orig backing_fmt=raw >>>>> wrote 4096/4096 bytes at offset 0 >>>>> 4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >>>>> >>>>> >>>>> >>>>> This looks suspiciously like the test isn't using a unique >>>>> filename for its disk image: "qemu-iotests/scratch/t.qcow2" >>>>> in the build directory, and so perhaps it has collided with >>>>> another iotest ? >>>>> >>>>> If we run 'make check' with a -j<something> option do the >>>>> iotests all get run serially anyway, or do they run in >>>>> parallel against each other ? >>>> >>>> As far as I know, all iotests are executed serially. Anything else >>>> would not work with the same scratch directory. >>>> >>>> The only thing I suspect is that some tool has been accidentally left >>>> running by some previous test that still accesses its own image. But I >>>> don’t know. >>> >>> Just saw this one again with the same iotest 130 on the same >>> s390 box; only difference is that the log this time around >>> has the first part where qemu-img fails, but not the second part >>> where qemu-io fails: >>> >>> --- /home/linux1/qemu/tests/qemu-iotests/130.out 2019-05-10 >>> 12:27:16.948075733 -0400 >>> +++ /home/linux1/qemu/build/all/tests/qemu-iotests/130.out.bad >>> 2019-10-17 11:56:43.450750873 -0400 >>> @@ -18,11 +18,8 @@ >>> QEMU X.Y.Z monitor - type 'help' for more information >>> (qemu) commit testdisk >>> (qemu) >>> -image: TEST_DIR/t.IMGFMT >>> -file format: IMGFMT >>> -virtual size: 64 MiB (67108864 bytes) >>> -backing file: TEST_DIR/t.IMGFMT.orig >>> -backing file format: raw >>> +qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Failed to get shared "write" lock >>> +Is another process using the image [TEST_DIR/t.IMGFMT]? >>> >>> === Marking image dirty (lazy refcounts) === >>> >>> On the host machine there don't seem to be any stray >>> processes which might have held the file open, and >>> indeed the file doesn't exist at all, so it got removed >>> by some cleanup or other. >> >> Ok, so unless someone has a clue what might be going on here (is there a >> race in the test?), I'd suggest that we simply remove 130 from the auto >> group again. Shall I send a patch? > > I don’t have much of an idea. It looks like maybe the qemu process > (which dos the commit) is lingering, but that shouldn’t be because > _cleanup_qemu always waits for it. (Also, I can’t reproduce the problem > on my system.) > > The only hunch that I have is that 130 seems to be the only test that > uses _cleanup_qemu to kill a qemu process (i.e. without wait=1) while it > has taken locks on an image and then still uses the image afterwards. > Maybe making it quit qemu through HMP would fix the problem. But > knowing is difficult because I can’t reproduce it. I also can't reproduce the problem. I've now run 130 several hundred times in a loop on a s390x box and never saw the problem. I think we should disable it for the time being. Thomas [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-10-18 11:44 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2019-09-27 16:39 iotest failure -- test possibly not using sufficiently unique temp filename? Peter Maydell 2019-09-27 16:44 ` Max Reitz 2019-10-17 16:41 ` Peter Maydell 2019-10-18 6:20 ` Thomas Huth 2019-10-18 8:42 ` Max Reitz 2019-10-18 11:43 ` Thomas Huth
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).