Re: [Qemu-devel] [Qemu-block] [PATCH v2 3/4] qemu-iotests: Test that "stop" doesn't drain block jobs

[Max Reitz <mreitz@redhat.com>]

On 27.04.2015 07:14, Fam Zheng wrote:
> On Fri, 04/24 17:43, Max Reitz wrote:
>> On 03.04.2015 16:05, Fam Zheng wrote:
>>> Signed-off-by: Fam Zheng <famz@redhat.com>
>>> ---
>>>   tests/qemu-iotests/129     | 86 ++++++++++++++++++++++++++++++++++++++++++++++
>>>   tests/qemu-iotests/129.out |  5 +++
>>>   tests/qemu-iotests/group   |  1 +
>>>   3 files changed, 92 insertions(+)
>>>   create mode 100644 tests/qemu-iotests/129
>>>   create mode 100644 tests/qemu-iotests/129.out
>>>
>>> diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129
>>> new file mode 100644
>>> index 0000000..9e87e1c
>>> --- /dev/null
>>> +++ b/tests/qemu-iotests/129
>>> @@ -0,0 +1,86 @@
>>> +#!/usr/bin/env python
>>> +#
>>> +# Tests that "bdrv_drain_all" doesn't drain block jobs
>>> +#
>>> +# Copyright (C) 2015 Red Hat, Inc.
>>> +#
>>> +# This program is free software; you can redistribute it and/or modify
>>> +# it under the terms of the GNU General Public License as published by
>>> +# the Free Software Foundation; either version 2 of the License, or
>>> +# (at your option) any later version.
>>> +#
>>> +# This program is distributed in the hope that it will be useful,
>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>>> +# GNU General Public License for more details.
>>> +#
>>> +# You should have received a copy of the GNU General Public License
>>> +# along with this program.  If not, see <http://www.gnu.org/licenses/>.
>>> +#
>>> +
>>> +import os
>>> +import iotests
>>> +import time
>>> +
>>> +class TestStopWithBlockJob(iotests.QMPTestCase):
>>> +    test_img = os.path.join(iotests.test_dir, 'test.img')
>>> +    target_img = os.path.join(iotests.test_dir, 'target.img')
>>> +    base_img = os.path.join(iotests.test_dir, 'base.img')
>>> +
>>> +    def setUp(self):
>>> +        iotests.qemu_img('create', '-f', iotests.imgfmt, self.base_img, "1G")
>>> +        iotests.qemu_img('create', '-f', iotests.imgfmt, self.test_img, "-b", self.base_img)
>>> +        iotests.qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x5d 1M 128M', self.test_img)
>>> +        self.vm = iotests.VM().add_drive(self.test_img)
>>> +        self.vm.launch()
>>> +
>>> +    def tearDown(self):
>>> +        params = {"device": "drive0",
>>> +                  "bps": 0,
>>> +                  "bps_rd": 0,
>>> +                  "bps_wr": 0,
>>> +                  "iops": 0,
>>> +                  "iops_rd": 0,
>>> +                  "iops_wr": 0,
>>> +                 }
>>> +        result = self.vm.qmp("block_set_io_throttle", conv_keys=False,
>>> +                             **params)
>>> +        self.vm.shutdown()
>>> +
>>> +    def do_test_stop(self, cmd, **args):
>>> +        """Test 'stop' while block job is running on a throttled drive.
>>> +        The 'stop' command shouldn't drain the job"""
>>> +        params = {"device": "drive0",
>>> +                  "bps": 1024,
>>> +                  "bps_rd": 0,
>>> +                  "bps_wr": 0,
>>> +                  "iops": 0,
>>> +                  "iops_rd": 0,
>>> +                  "iops_wr": 0,
>>> +                 }
>>> +        result = self.vm.qmp("block_set_io_throttle", conv_keys=False,
>>> +                             **params)
>>> +        self.assert_qmp(result, 'return', {})
>>> +        result = self.vm.qmp(cmd, **args)
>>> +        self.assert_qmp(result, 'return', {})
>>> +        result = self.vm.qmp("stop")
>>> +        self.assert_qmp(result, 'return', {})
>>> +        result = self.vm.qmp("query-block-jobs")
>>> +        self.assert_qmp(result, 'return[0]/busy', True)
>> I don't know why, but this assertion fails in tmpfs (for me, at least).
> I also run tests in tmpfs, but I don't see the failure.  What is in "result"?
> Is the block job completed?
>
> (I assume you've applied and compiled patch 1-2. Worth mentioning because
> that's exactly what this assertion is testing against.)

On Stefan's block-next branch:

--- tests/qemu-iotests/129.out 2015-04-24 17:39:47.089438232 +0200
+++ 129.out.bad 2015-04-27 11:57:04.537258961 +0200
@@ -1,5 +1,17 @@
-...
+F..
+======================================================================
+FAIL: test_block_commit (__main__.TestStopWithBlockJob)
+----------------------------------------------------------------------
+Traceback (most recent call last):
+  File "129", line 83, in test_block_commit
+    self.do_test_stop("block-commit", device="drive0")
+  File "129", line 69, in do_test_stop
+    self.assert_qmp(result, 'return[0]/busy', True)
+  File "tests/qemu-iotests/iotests.py", line 287, in assert_qmp
+    self.assertEqual(result, value, 'values not equal "%s" and "%s"' % 
(str(result), str(value)))
+AssertionError: values not equal "False" and "True"
+
  ----------------------------------------------------------------------
  Ran 3 tests

-OK
+FAILED (failures=1)

On my SSD it works just fine.

Adding a "print cmd"; "print self.vm.get_qmp_events(wait=False)" and 
"print result" gives:

block-commit
[{u'timestamp': {u'seconds': 1430130787, u'microseconds': 779056}, 
u'event': u'STOP'}, {u'timestamp': {u'seconds': 1430130787, 
u'microseconds': 868729}, u'data': {u'device': u'drive0', u'type': 
u'commit', u'speed': 0, u'len': 134217728, 'offset': 134217728}, 
u'event': u'BLOCK_JOB_READY'}]
{u'return': [{u'busy': False, u'type': u'commit', u'len': 134217728, 
u'paused': False, u'ready': True, u'io-status': u'ok', u'offset': 
134217728, u'device': u'drive0', u'speed': 0}]}
drive-backup
[{u'timestamp': {u'seconds': 1430130788, u'microseconds': 11275}, 
u'event': u'STOP'}]
{u'return': [{u'busy': True, u'type': u'backup', u'len': 1073741824, 
u'paused': False, u'ready': False, u'io-status': u'ok', u'offset': 
65536, u'device': u'drive0', u'speed': 0}]}
drive-mirror
[{u'timestamp': {u'seconds': 1430130788, u'microseconds': 672984}, 
u'event': u'STOP'}]
{u'return': [{u'busy': True, u'type': u'mirror', u'len': 134217728, 
u'paused': False, u'ready': False, u'io-status': u'ok', u'offset': 0, 
u'device': u'drive0', u'speed': 0}]}

So apparently the block-commit job did complete. I guess this might be 
because while the device itself is throttled, the backing file is not, 
so the commit job is not throttled...?

Okay, half an hour of investigation later: It is because block/mirror 
takes a clever shortcut when delay_ns is 0 and simply skips sleeping, 
which makes pausing the mirror (or active commit) block job in 
bdrv_drain_all() moot. I sent a patch for that, because I might as well...

Max