From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
To: qemu-block@nongnu.org
Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, ehabkost@redhat.com,
wencongyang2@huawei.com, xiechanglong.d@gmail.com,
armbru@redhat.com, qemu-devel@nongnu.org, jsnow@redhat.com,
crosa@redhat.com, den@openvz.org, mreitz@redhat.com
Subject: [RFC 23/24] python: add qemu/bench_block_job.py
Date: Fri, 15 Nov 2019 17:14:43 +0300 [thread overview]
Message-ID: <20191115141444.24155-24-vsementsov@virtuozzo.com> (raw)
In-Reply-To: <20191115141444.24155-1-vsementsov@virtuozzo.com>
Add block-job benchmarking helper functions.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
python/qemu/bench_block_job.py | 114 +++++++++++++++++++++++++++++++++
1 file changed, 114 insertions(+)
create mode 100755 python/qemu/bench_block_job.py
diff --git a/python/qemu/bench_block_job.py b/python/qemu/bench_block_job.py
new file mode 100755
index 0000000000..6a72be9ba2
--- /dev/null
+++ b/python/qemu/bench_block_job.py
@@ -0,0 +1,114 @@
+#!/usr/bin/env python
+#
+# Benchmark block jobs
+#
+# Copyright (c) 2019 Virtuozzo International GmbH.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+
+import socket
+import json
+from .machine import QEMUMachine
+
+
+def bench_block_job(cmd, cmd_args, qemu_args):
+ """Benchmark block-job
+
+ cmd -- qmp command to run block-job (like blockdev-backup)
+ cmd_args -- dict of qmp command arguments
+ qemu_args -- list of Qemu command line arguments, including path to Qemu
+ binary
+
+ Returns {'seconds': int} on success and {'error': str} on failure, dict may
+ contain addional 'vm-log' field. Return value is compatible with
+ simplebench lib.
+ """
+
+ vm = QEMUMachine(qemu_args[0], args=qemu_args[1:])
+
+ try:
+ vm.launch()
+ except OSError as e:
+ return {'error': 'popen failed: ' + str(e)}
+ except socket.timeout:
+ return {'error': 'qemu failed: ' + str(vm.get_log())}
+
+ try:
+ res = vm.qmp(cmd, **cmd_args)
+ if res != {'return': {}}:
+ vm.shutdown()
+ return {'error': '"{}" command failed: {}'.format(cmd, str(res))}
+
+ e = vm.event_wait('JOB_STATUS_CHANGE')
+ assert e['data']['status'] == 'created'
+ start_ms = e['timestamp']['seconds'] * 1000000 + \
+ e['timestamp']['microseconds']
+
+ e = vm.events_wait((('BLOCK_JOB_READY', None),
+ ('BLOCK_JOB_COMPLETED', None),
+ ('BLOCK_JOB_FAILED', None)), timeout=True)
+ if e['event'] not in ('BLOCK_JOB_READY', 'BLOCK_JOB_COMPLETED'):
+ vm.shutdown()
+ return {'error': 'block-job failed: ' + str(e),
+ 'vm-log': vm.get_log()}
+ end_ms = e['timestamp']['seconds'] * 1000000 + \
+ e['timestamp']['microseconds']
+ finally:
+ vm.shutdown()
+
+ return {'seconds': (end_ms - start_ms) / 1000000.0}
+
+
+# Bench backup or mirror
+def bench_block_copy(qemu_binary, cmd, source, target):
+ """Helper to run bench_block_job() for mirror or backup"""
+ assert cmd in ('blockdev-backup', 'blockdev-mirror')
+
+ source['node-name'] = 'source'
+ target['node-name'] = 'target'
+
+ return bench_block_job(cmd,
+ {'job-id': 'job0', 'device': 'source',
+ 'target': 'target', 'sync': 'full'},
+ [qemu_binary,
+ '-blockdev', json.dumps(source),
+ '-blockdev', json.dumps(target)])
+
+
+def drv_file(filename):
+ return {'driver': 'file', 'filename': filename,
+ 'cache': {'direct': True}, 'aio': 'native'}
+
+
+def drv_nbd(host, port):
+ return {'driver': 'nbd',
+ 'server': {'type': 'inet', 'host': host, 'port': port}}
+
+
+if __name__ == '__main__':
+ import sys
+
+ if len(sys.argv) < 4:
+ print('USAGE: {} <qmp block-job command name> '
+ '<json string of arguments for the command> '
+ '<qemu binary path and arguments>'.format(sys.argv[0]))
+ exit(1)
+
+ res = bench_block_job(sys.argv[1], json.loads(sys.argv[2]), sys.argv[3:])
+ if 'seconds' in res:
+ print('{:.2f}'.format(res['seconds']))
+ else:
+ print(res)
--
2.21.0
next prev parent reply other threads:[~2019-11-15 14:33 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-11-15 14:14 [RFC 00/24] backup performance: block_status + async Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 01/24] block/block-copy: specialcase first copy_range request Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 02/24] block/block-copy: use block_status Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 03/24] block/block-copy: factor out block_copy_find_inflight_req Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 04/24] block/block-copy: refactor interfaces to use bytes instead of end Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 05/24] block/block-copy: rename start to offset in interfaces Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 06/24] block/block-copy: reduce intersecting request lock Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 07/24] block/block-copy: hide structure definitions Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 08/24] block/block-copy: rename in-flight requests to tasks Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 09/24] block/block-copy: alloc task on each iteration Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 10/24] block/block-copy: add state pointer to BlockCopyTask Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 11/24] block/block-copy: move task size initial calculation to _task_create Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 12/24] block/block-copy: move block_copy_task_create down Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 13/24] block/block-copy: use aio-task-pool API Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 14/24] block/block-copy: More explicit call_state Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 15/24] block/block-copy: implement block_copy_async Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 16/24] block/block-copy: add max_chunk and max_workers paramters Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 17/24] block/block-copy: add ratelimit to block-copy Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 18/24] block/block-copy: add block_copy_cancel Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 19/24] blockjob: add set_speed to BlockJobDriver Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 20/24] job: call job_enter from job_user_pause Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 21/24] backup: move to block-copy Vladimir Sementsov-Ogievskiy
2019-11-15 17:58 ` Eric Blake
2019-11-16 9:28 ` Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` [RFC 22/24] python: add simplebench.py Vladimir Sementsov-Ogievskiy
2019-11-15 14:14 ` Vladimir Sementsov-Ogievskiy [this message]
2019-11-15 14:14 ` [RFC 24/24] python: benchmark new backup architecture Vladimir Sementsov-Ogievskiy
2019-11-15 17:30 ` [RFC 00/24] backup performance: block_status + async no-reply
2019-11-15 17:33 ` Vladimir Sementsov-Ogievskiy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191115141444.24155-24-vsementsov@virtuozzo.com \
--to=vsementsov@virtuozzo.com \
--cc=armbru@redhat.com \
--cc=crosa@redhat.com \
--cc=den@openvz.org \
--cc=ehabkost@redhat.com \
--cc=jsnow@redhat.com \
--cc=kwolf@redhat.com \
--cc=mreitz@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=wencongyang2@huawei.com \
--cc=xiechanglong.d@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).