* [Qemu-devel] [PULL 1/4] qemu.py: make VM() a context manager
2017-09-05 13:29 [Qemu-devel] [PULL 0/4] Block patches Stefan Hajnoczi
@ 2017-09-05 13:29 ` Stefan Hajnoczi
2017-09-05 13:29 ` [Qemu-devel] [PULL 2/4] iotests.py: add FilePath " Stefan Hajnoczi
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Stefan Hajnoczi @ 2017-09-05 13:29 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Maydell, Stefan Hajnoczi, Eduardo Habkost
There are a number of ways to ensure that the QEMU process is shut down
when the test ends, including atexit.register(), try: finally:, or
unittest.teardown() methods. All of these require extra code and the
programmer must remember to add vm.shutdown().
A nice solution is context managers:
with VM(binary) as vm:
...
# vm is guaranteed to be shut down here
Cc: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Message-id: 20170824072202.26818-2-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
scripts/qemu.py | 16 +++++++++++++++-
1 file changed, 15 insertions(+), 1 deletion(-)
diff --git a/scripts/qemu.py b/scripts/qemu.py
index 880e3e8219..4d8ee10943 100644
--- a/scripts/qemu.py
+++ b/scripts/qemu.py
@@ -21,7 +21,14 @@ import qmp.qmp
class QEMUMachine(object):
- '''A QEMU VM'''
+ '''A QEMU VM
+
+ Use this object as a context manager to ensure the QEMU process terminates::
+
+ with VM(binary) as vm:
+ ...
+ # vm is guaranteed to be shut down here
+ '''
def __init__(self, binary, args=[], wrapper=[], name=None, test_dir="/var/tmp",
monitor_address=None, socket_scm_helper=None, debug=False):
@@ -40,6 +47,13 @@ class QEMUMachine(object):
self._socket_scm_helper = socket_scm_helper
self._debug = debug
+ def __enter__(self):
+ return self
+
+ def __exit__(self, exc_type, exc_val, exc_tb):
+ self.shutdown()
+ return False
+
# This can be used to add an unused monitor instance.
def add_monitor_telnet(self, ip, port):
args = 'tcp:%s:%d,server,nowait,telnet' % (ip, port)
--
2.13.5
^ permalink raw reply related [flat|nested] 6+ messages in thread* [Qemu-devel] [PULL 2/4] iotests.py: add FilePath context manager
2017-09-05 13:29 [Qemu-devel] [PULL 0/4] Block patches Stefan Hajnoczi
2017-09-05 13:29 ` [Qemu-devel] [PULL 1/4] qemu.py: make VM() a context manager Stefan Hajnoczi
@ 2017-09-05 13:29 ` Stefan Hajnoczi
2017-09-05 13:29 ` [Qemu-devel] [PULL 3/4] qemu-iotests: use context managers for resource cleanup in 194 Stefan Hajnoczi
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Stefan Hajnoczi @ 2017-09-05 13:29 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Maydell, Stefan Hajnoczi
The scratch/ (TEST_DIR) directory is not automatically cleaned up after
test execution. It is the responsibility of tests to remove any files
they create.
A nice way of doing this is to declare files at the beginning of the
test and automatically remove them with a context manager:
with iotests.FilePath('test.img') as img_path:
qemu_img(...)
qemu_io(...)
# img_path is guaranteed to be deleted here
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20170824072202.26818-3-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
tests/qemu-iotests/iotests.py | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 7233983f3c..07fa1626a0 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -160,6 +160,32 @@ class Timeout:
def timeout(self, signum, frame):
raise Exception(self.errmsg)
+
+class FilePath(object):
+ '''An auto-generated filename that cleans itself up.
+
+ Use this context manager to generate filenames and ensure that the file
+ gets deleted::
+
+ with TestFilePath('test.img') as img_path:
+ qemu_img('create', img_path, '1G')
+ # migration_sock_path is automatically deleted
+ '''
+ def __init__(self, name):
+ filename = '{0}-{1}'.format(os.getpid(), name)
+ self.path = os.path.join(test_dir, filename)
+
+ def __enter__(self):
+ return self.path
+
+ def __exit__(self, exc_type, exc_val, exc_tb):
+ try:
+ os.remove(self.path)
+ except OSError:
+ pass
+ return False
+
+
class VM(qtest.QEMUQtestMachine):
'''A QEMU VM'''
--
2.13.5
^ permalink raw reply related [flat|nested] 6+ messages in thread* [Qemu-devel] [PULL 3/4] qemu-iotests: use context managers for resource cleanup in 194
2017-09-05 13:29 [Qemu-devel] [PULL 0/4] Block patches Stefan Hajnoczi
2017-09-05 13:29 ` [Qemu-devel] [PULL 1/4] qemu.py: make VM() a context manager Stefan Hajnoczi
2017-09-05 13:29 ` [Qemu-devel] [PULL 2/4] iotests.py: add FilePath " Stefan Hajnoczi
@ 2017-09-05 13:29 ` Stefan Hajnoczi
2017-09-05 13:29 ` [Qemu-devel] [PULL 4/4] block: document semantics of bdrv_co_preadv|pwritev Stefan Hajnoczi
2017-09-05 15:36 ` [Qemu-devel] [PULL 0/4] Block patches Peter Maydell
4 siblings, 0 replies; 6+ messages in thread
From: Stefan Hajnoczi @ 2017-09-05 13:29 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Maydell, Stefan Hajnoczi
Switch from atexit.register() to a more elegant idiom of declaring
resources in a with statement:
with FilePath('monitor.sock') as monitor_path,
VM() as vm:
...
The files and VMs will be automatically cleaned up whether the test
passes or fails.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20170824072202.26818-4-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
tests/qemu-iotests/194 | 117 ++++++++++++++++++++++++-------------------------
1 file changed, 58 insertions(+), 59 deletions(-)
diff --git a/tests/qemu-iotests/194 b/tests/qemu-iotests/194
index a3e3bad664..6449b9b64a 100755
--- a/tests/qemu-iotests/194
+++ b/tests/qemu-iotests/194
@@ -19,66 +19,65 @@
#
# Non-shared storage migration test using NBD server and drive-mirror
-import os
-import atexit
import iotests
iotests.verify_platform(['linux'])
-img_size = '1G'
-source_img_path = os.path.join(iotests.test_dir, 'source.img')
-dest_img_path = os.path.join(iotests.test_dir, 'dest.img')
-iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, source_img_path, img_size)
-iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, dest_img_path, img_size)
-
-iotests.log('Launching VMs...')
-migration_sock_path = os.path.join(iotests.test_dir, 'migration.sock')
-nbd_sock_path = os.path.join(iotests.test_dir, 'nbd.sock')
-source_vm = iotests.VM('source').add_drive(source_img_path)
-dest_vm = (iotests.VM('dest').add_drive(dest_img_path)
- .add_incoming('unix:{0}'.format(migration_sock_path)))
-source_vm.launch()
-atexit.register(source_vm.shutdown)
-dest_vm.launch()
-atexit.register(dest_vm.shutdown)
-
-iotests.log('Launching NBD server on destination...')
-iotests.log(dest_vm.qmp('nbd-server-start', addr={'type': 'unix', 'data': {'path': nbd_sock_path}}))
-iotests.log(dest_vm.qmp('nbd-server-add', device='drive0', writable=True))
-
-iotests.log('Starting `drive-mirror` on source...')
-iotests.log(source_vm.qmp(
- 'drive-mirror',
- device='drive0',
- target='nbd+unix:///drive0?socket={0}'.format(nbd_sock_path),
- sync='full',
- format='raw', # always raw, the server handles the format
- mode='existing',
- job_id='mirror-job0'))
-
-iotests.log('Waiting for `drive-mirror` to complete...')
-iotests.log(source_vm.event_wait('BLOCK_JOB_READY'),
- filters=[iotests.filter_qmp_event])
-
-iotests.log('Starting migration...')
-source_vm.qmp('migrate-set-capabilities',
- capabilities=[{'capability': 'events', 'state': True}])
-dest_vm.qmp('migrate-set-capabilities',
- capabilities=[{'capability': 'events', 'state': True}])
-iotests.log(source_vm.qmp('migrate', uri='unix:{0}'.format(migration_sock_path)))
-
-while True:
- event1 = source_vm.event_wait('MIGRATION')
- iotests.log(event1, filters=[iotests.filter_qmp_event])
- if event1['data']['status'] in ('completed', 'failed'):
- iotests.log('Gracefully ending the `drive-mirror` job on source...')
- iotests.log(source_vm.qmp('block-job-cancel', device='mirror-job0'))
- break
-
-while True:
- event2 = source_vm.event_wait('BLOCK_JOB_COMPLETED')
- iotests.log(event2, filters=[iotests.filter_qmp_event])
- if event2['event'] == 'BLOCK_JOB_COMPLETED':
- iotests.log('Stopping the NBD server on destination...')
- iotests.log(dest_vm.qmp('nbd-server-stop'))
- break
+with iotests.FilePath('source.img') as source_img_path, \
+ iotests.FilePath('dest.img') as dest_img_path, \
+ iotests.FilePath('migration.sock') as migration_sock_path, \
+ iotests.FilePath('nbd.sock') as nbd_sock_path, \
+ iotests.VM('source') as source_vm, \
+ iotests.VM('dest') as dest_vm:
+
+ img_size = '1G'
+ iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, source_img_path, img_size)
+ iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, dest_img_path, img_size)
+
+ iotests.log('Launching VMs...')
+ (source_vm.add_drive(source_img_path)
+ .launch())
+ (dest_vm.add_drive(dest_img_path)
+ .add_incoming('unix:{0}'.format(migration_sock_path))
+ .launch())
+
+ iotests.log('Launching NBD server on destination...')
+ iotests.log(dest_vm.qmp('nbd-server-start', addr={'type': 'unix', 'data': {'path': nbd_sock_path}}))
+ iotests.log(dest_vm.qmp('nbd-server-add', device='drive0', writable=True))
+
+ iotests.log('Starting `drive-mirror` on source...')
+ iotests.log(source_vm.qmp(
+ 'drive-mirror',
+ device='drive0',
+ target='nbd+unix:///drive0?socket={0}'.format(nbd_sock_path),
+ sync='full',
+ format='raw', # always raw, the server handles the format
+ mode='existing',
+ job_id='mirror-job0'))
+
+ iotests.log('Waiting for `drive-mirror` to complete...')
+ iotests.log(source_vm.event_wait('BLOCK_JOB_READY'),
+ filters=[iotests.filter_qmp_event])
+
+ iotests.log('Starting migration...')
+ source_vm.qmp('migrate-set-capabilities',
+ capabilities=[{'capability': 'events', 'state': True}])
+ dest_vm.qmp('migrate-set-capabilities',
+ capabilities=[{'capability': 'events', 'state': True}])
+ iotests.log(source_vm.qmp('migrate', uri='unix:{0}'.format(migration_sock_path)))
+
+ while True:
+ event1 = source_vm.event_wait('MIGRATION')
+ iotests.log(event1, filters=[iotests.filter_qmp_event])
+ if event1['data']['status'] in ('completed', 'failed'):
+ iotests.log('Gracefully ending the `drive-mirror` job on source...')
+ iotests.log(source_vm.qmp('block-job-cancel', device='mirror-job0'))
+ break
+
+ while True:
+ event2 = source_vm.event_wait('BLOCK_JOB_COMPLETED')
+ iotests.log(event2, filters=[iotests.filter_qmp_event])
+ if event2['event'] == 'BLOCK_JOB_COMPLETED':
+ iotests.log('Stopping the NBD server on destination...')
+ iotests.log(dest_vm.qmp('nbd-server-stop'))
+ break
--
2.13.5
^ permalink raw reply related [flat|nested] 6+ messages in thread* [Qemu-devel] [PULL 4/4] block: document semantics of bdrv_co_preadv|pwritev
2017-09-05 13:29 [Qemu-devel] [PULL 0/4] Block patches Stefan Hajnoczi
` (2 preceding siblings ...)
2017-09-05 13:29 ` [Qemu-devel] [PULL 3/4] qemu-iotests: use context managers for resource cleanup in 194 Stefan Hajnoczi
@ 2017-09-05 13:29 ` Stefan Hajnoczi
2017-09-05 15:36 ` [Qemu-devel] [PULL 0/4] Block patches Peter Maydell
4 siblings, 0 replies; 6+ messages in thread
From: Stefan Hajnoczi @ 2017-09-05 13:29 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Maydell, Daniel P. Berrange, Stefan Hajnoczi
From: "Daniel P. Berrange" <berrange@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id: 20170831105456.9558-1-berrange@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
include/block/block_int.h | 31 +++++++++++++++++++++++++++++++
1 file changed, 31 insertions(+)
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 7571c0aaaf..7816b43a27 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -146,12 +146,43 @@ struct BlockDriver {
int coroutine_fn (*bdrv_co_readv)(BlockDriverState *bs,
int64_t sector_num, int nb_sectors, QEMUIOVector *qiov);
+
+ /**
+ * @offset: position in bytes to read at
+ * @bytes: number of bytes to read
+ * @qiov: the buffers to fill with read data
+ * @flags: currently unused, always 0
+ *
+ * @offset and @bytes will be a multiple of 'request_alignment',
+ * but the length of individual @qiov elements does not have to
+ * be a multiple.
+ *
+ * @bytes will always equal the total size of @qiov, and will be
+ * no larger than 'max_transfer'.
+ *
+ * The buffer in @qiov may point directly to guest memory.
+ */
int coroutine_fn (*bdrv_co_preadv)(BlockDriverState *bs,
uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags);
int coroutine_fn (*bdrv_co_writev)(BlockDriverState *bs,
int64_t sector_num, int nb_sectors, QEMUIOVector *qiov);
int coroutine_fn (*bdrv_co_writev_flags)(BlockDriverState *bs,
int64_t sector_num, int nb_sectors, QEMUIOVector *qiov, int flags);
+ /**
+ * @offset: position in bytes to write at
+ * @bytes: number of bytes to write
+ * @qiov: the buffers containing data to write
+ * @flags: zero or more bits allowed by 'supported_write_flags'
+ *
+ * @offset and @bytes will be a multiple of 'request_alignment',
+ * but the length of individual @qiov elements does not have to
+ * be a multiple.
+ *
+ * @bytes will always equal the total size of @qiov, and will be
+ * no larger than 'max_transfer'.
+ *
+ * The buffer in @qiov may point directly to guest memory.
+ */
int coroutine_fn (*bdrv_co_pwritev)(BlockDriverState *bs,
uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags);
--
2.13.5
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [Qemu-devel] [PULL 0/4] Block patches
2017-09-05 13:29 [Qemu-devel] [PULL 0/4] Block patches Stefan Hajnoczi
` (3 preceding siblings ...)
2017-09-05 13:29 ` [Qemu-devel] [PULL 4/4] block: document semantics of bdrv_co_preadv|pwritev Stefan Hajnoczi
@ 2017-09-05 15:36 ` Peter Maydell
4 siblings, 0 replies; 6+ messages in thread
From: Peter Maydell @ 2017-09-05 15:36 UTC (permalink / raw)
To: Stefan Hajnoczi; +Cc: QEMU Developers
On 5 September 2017 at 14:29, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> The following changes since commit 2b483739791b33c46e6084b51edcf62107058ae1:
>
> Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20170904-2' into staging (2017-09-04 17:21:24 +0100)
>
> are available in the git repository at:
>
> git://github.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to b461151ff31c7925f271c297e8abed20231ac7d3:
>
> block: document semantics of bdrv_co_preadv|pwritev (2017-09-05 11:07:02 +0100)
>
> ----------------------------------------------------------------
>
> ----------------------------------------------------------------
>
> Daniel P. Berrange (1):
> block: document semantics of bdrv_co_preadv|pwritev
>
> Stefan Hajnoczi (3):
> qemu.py: make VM() a context manager
> iotests.py: add FilePath context manager
> qemu-iotests: use context managers for resource cleanup in 194
>
> include/block/block_int.h | 31 +++++++++++
> scripts/qemu.py | 16 +++++-
> tests/qemu-iotests/194 | 117 +++++++++++++++++++++---------------------
> tests/qemu-iotests/iotests.py | 26 ++++++++++
> 4 files changed, 130 insertions(+), 60 deletions(-)
Applied, thanks.
-- PMM
^ permalink raw reply [flat|nested] 6+ messages in thread