[Qemu-devel] [PATCH 0/3] Fix "stop" draining block jobs

[Qemu-devel] [PATCH 0/3] Fix "stop" draining block jobs

[Fam Zheng]

Stopping the vm will drive the block job all the way to the end, because the
sleep duration is too short, which means the block_job_sleep_ns in the block
jobs are unhelpful. That is because the timer will fire too soon, even before
the aio_poll in bdrv_drain_all returns.

Lengthen the sleep and add a test case to catch this issue in the future.

It's not perfect, because the aio_poll returning point could still be far
enough that we wake up the job earlier, but this patch is already making it
better in common cases - setting up a timer with timeout=0 was definitely too
short anyway.

A complete solution would be adding a "sleep until next iteration" timer/BH
API, but I'm not sure that is worth the complexity.

Please review!

Fam

Fam Zheng (3):
  blockjob: Don't sleep meaninglessly short
  qemu-iotests: Test that "stop" doesn't drain block jobs
  blockjob: Update function name in comments

 block/backup.c             |  2 +-
 block/mirror.c             |  2 +-
 blockjob.c                 |  3 +-
 include/block/blockjob.h   |  6 +++-
 tests/qemu-iotests/129     | 86 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/129.out |  5 +++
 tests/qemu-iotests/group   |  1 +
 7 files changed, 101 insertions(+), 4 deletions(-)
 create mode 100644 tests/qemu-iotests/129
 create mode 100644 tests/qemu-iotests/129.out

-- 
2.1.0

[Qemu-devel] [PATCH 1/3] blockjob: Don't sleep meaninglessly short

[Fam Zheng]

A timer that will immediately wake the job up spoils the point to call
block_job_sleep_ns in the first place - bdrv_drain_all has to return.

Let's sleep longer to let aio_poll return, before the job resumes and
submitts more IO requests.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 blockjob.c               | 3 ++-
 include/block/blockjob.h | 6 +++++-
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/blockjob.c b/blockjob.c
index ba2255d..f04a16c 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -232,7 +232,8 @@ void block_job_sleep_ns(BlockJob *job, QEMUClockType type, int64_t ns)
     if (block_job_is_paused(job)) {
         qemu_coroutine_yield();
     } else {
-        co_aio_sleep_ns(bdrv_get_aio_context(job->bs), type, ns);
+        co_aio_sleep_ns(bdrv_get_aio_context(job->bs), type,
+                        MAX(ns, BLOCK_JOB_SLEEP_MIN));
     }
     job->busy = true;
 }
diff --git a/include/block/blockjob.h b/include/block/blockjob.h
index b6d4ebb..97835aa 100644
--- a/include/block/blockjob.h
+++ b/include/block/blockjob.h
@@ -140,11 +140,15 @@ void *block_job_create(const BlockJobDriver *driver, BlockDriverState *bs,
                        int64_t speed, BlockCompletionFunc *cb,
                        void *opaque, Error **errp);
 
+#define BLOCK_JOB_SLEEP_MIN 100000LL
 /**
  * block_job_sleep_ns:
  * @job: The job that calls the function.
  * @clock: The clock to sleep on.
- * @ns: How many nanoseconds to stop for.
+ * @ns: How many nanoseconds to stop for.  It would be meaningless if it is
+ * very short, because the timer to wake up the job could fire right very
+ * quickly even before anything else is done.  Hence, a minimum,
+ * BLOCK_JOB_SLEEP_MIN, is defined.
  *
  * Put the job to sleep (assuming that it wasn't canceled) for @ns
  * nanoseconds.  Canceling the job will interrupt the wait immediately.
-- 
2.1.0

[Qemu-devel] [PATCH 2/3] qemu-iotests: Test that "stop" doesn't drain block jobs

[Fam Zheng]

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 tests/qemu-iotests/129     | 86 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/129.out |  5 +++
 tests/qemu-iotests/group   |  1 +
 3 files changed, 92 insertions(+)
 create mode 100644 tests/qemu-iotests/129
 create mode 100644 tests/qemu-iotests/129.out

diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129
new file mode 100644
index 0000000..9e87e1c
--- /dev/null
+++ b/tests/qemu-iotests/129
@@ -0,0 +1,86 @@
+#!/usr/bin/env python
+#
+# Tests that "bdrv_drain_all" doesn't drain block jobs
+#
+# Copyright (C) 2015 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import iotests
+import time
+
+class TestStopWithBlockJob(iotests.QMPTestCase):
+    test_img = os.path.join(iotests.test_dir, 'test.img')
+    target_img = os.path.join(iotests.test_dir, 'target.img')
+    base_img = os.path.join(iotests.test_dir, 'base.img')
+
+    def setUp(self):
+        iotests.qemu_img('create', '-f', iotests.imgfmt, self.base_img, "1G")
+        iotests.qemu_img('create', '-f', iotests.imgfmt, self.test_img, "-b", self.base_img)
+        iotests.qemu_io('-f', iotests.imgfmt, '-c', 'write -P0x5d 1M 128M', self.test_img)
+        self.vm = iotests.VM().add_drive(self.test_img)
+        self.vm.launch()
+
+    def tearDown(self):
+        params = {"device": "drive0",
+                  "bps": 0,
+                  "bps_rd": 0,
+                  "bps_wr": 0,
+                  "iops": 0,
+                  "iops_rd": 0,
+                  "iops_wr": 0,
+                 }
+        result = self.vm.qmp("block_set_io_throttle", conv_keys=False,
+                             **params)
+        self.vm.shutdown()
+
+    def do_test_stop(self, cmd, **args):
+        """Test 'stop' while block job is running on a throttled drive.
+        The 'stop' command shouldn't drain the job"""
+        params = {"device": "drive0",
+                  "bps": 1024,
+                  "bps_rd": 0,
+                  "bps_wr": 0,
+                  "iops": 0,
+                  "iops_rd": 0,
+                  "iops_wr": 0,
+                 }
+        result = self.vm.qmp("block_set_io_throttle", conv_keys=False,
+                             **params)
+        self.assert_qmp(result, 'return', {})
+        result = self.vm.qmp(cmd, **args)
+        self.assert_qmp(result, 'return', {})
+        result = self.vm.qmp("stop")
+        self.assert_qmp(result, 'return', {})
+        result = self.vm.qmp("query-block-jobs")
+        self.assert_qmp(result, 'return[0]/busy', True)
+        self.assert_qmp(result, 'return[0]/ready', False)
+
+    def test_drive_mirror(self):
+        self.do_test_stop("drive-mirror", device="drive0",
+                          target=self.target_img,
+                          sync="full")
+
+    def test_drive_backup(self):
+        self.do_test_stop("drive-backup", device="drive0",
+                          target=self.target_img,
+                          sync="full")
+
+    def test_block_commit(self):
+        self.do_test_stop("block-commit", device="drive0")
+
+if __name__ == '__main__':
+    iotests.main(supported_fmts=["qcow2"])
diff --git a/tests/qemu-iotests/129.out b/tests/qemu-iotests/129.out
new file mode 100644
index 0000000..8d7e996
--- /dev/null
+++ b/tests/qemu-iotests/129.out
@@ -0,0 +1,5 @@
+...
+----------------------------------------------------------------------
+Ran 3 tests
+
+OK
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 6262127..69071ac 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -124,3 +124,4 @@
 121 rw auto
 123 rw auto quick
 128 rw auto quick
+129 rw auto quick
-- 
2.1.0

[Qemu-devel] [PATCH 3/3] blockjob: Update function name in comments

[Fam Zheng]

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 block/backup.c | 2 +-
 block/mirror.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/block/backup.c b/block/backup.c
index 1c535b1..3312476 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -287,7 +287,7 @@ static void coroutine_fn backup_run(void *opaque)
                 break;
             }
 
-            /* we need to yield so that qemu_aio_flush() returns.
+            /* we need to yield so that bdrv_drain_all() returns.
              * (without, VM does not reboot)
              */
             if (job->common.speed) {
diff --git a/block/mirror.c b/block/mirror.c
index 4056164..513f19c 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -475,7 +475,7 @@ static void coroutine_fn mirror_run(void *opaque)
                         (cnt + s->sectors_in_flight) * BDRV_SECTOR_SIZE;
 
         /* Note that even when no rate limit is applied we need to yield
-         * periodically with no pending I/O so that qemu_aio_flush() returns.
+         * periodically with no pending I/O so that bdrv_drain_all() returns.
          * We do so every SLICE_TIME nanoseconds, or when there is an error,
          * or when the source is clean, whichever comes first.
          */
-- 
2.1.0

Re: [Qemu-devel] [PATCH 0/3] Fix "stop" draining block jobs

[Paolo Bonzini]

On 03/04/2015 12:38, Fam Zheng wrote:
> Stopping the vm will drive the block job all the way to the end, because the
> sleep duration is too short, which means the block_job_sleep_ns in the block
> jobs are unhelpful. That is because the timer will fire too soon, even before
> the aio_poll in bdrv_drain_all returns.
> 
> Lengthen the sleep and add a test case to catch this issue in the future.
> 
> It's not perfect, because the aio_poll returning point could still be far
> enough that we wake up the job earlier, but this patch is already making it
> better in common cases - setting up a timer with timeout=0 was definitely too
> short anyway.
> 
> A complete solution would be adding a "sleep until next iteration" timer/BH
> API, but I'm not sure that is worth the complexity.

Would it work if vm_stop pauses block jobs before drain, and restarts
the paused ones afterwards?

Paolo

Re: [Qemu-devel] [PATCH 0/3] Fix "stop" draining block jobs

[Fam Zheng]

On Fri, 04/03 13:10, Paolo Bonzini wrote:
> 
> 
> On 03/04/2015 12:38, Fam Zheng wrote:
> > Stopping the vm will drive the block job all the way to the end, because the
> > sleep duration is too short, which means the block_job_sleep_ns in the block
> > jobs are unhelpful. That is because the timer will fire too soon, even before
> > the aio_poll in bdrv_drain_all returns.
> > 
> > Lengthen the sleep and add a test case to catch this issue in the future.
> > 
> > It's not perfect, because the aio_poll returning point could still be far
> > enough that we wake up the job earlier, but this patch is already making it
> > better in common cases - setting up a timer with timeout=0 was definitely too
> > short anyway.
> > 
> > A complete solution would be adding a "sleep until next iteration" timer/BH
> > API, but I'm not sure that is worth the complexity.
> 
> Would it work if vm_stop pauses block jobs before drain, and restarts
> the paused ones afterwards?

Yeah. It would. Will need a bit more code, but still be a neat solution.

Fam