* [Qemu-devel] [PATCH 1/6] gluster: Add image resize support
2013-07-29 15:36 [Qemu-devel] [PULL for-1.6 0/6] Block patches Stefan Hajnoczi
@ 2013-07-29 15:36 ` Stefan Hajnoczi
2013-07-29 15:36 ` [Qemu-devel] [PATCH 2/6] block: Repair the throttling code Stefan Hajnoczi
` (4 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Stefan Hajnoczi @ 2013-07-29 15:36 UTC (permalink / raw)
To: qemu-devel; +Cc: Paolo Bonzini, Anthony Liguori, Stefan Hajnoczi, Bharata B Rao
From: Paolo Bonzini <pbonzini@redhat.com>
Implement .bdrv_truncate in GlusterFS block driver so that GlusterFS backend
can support image resizing.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Tested-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
block/gluster.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/block/gluster.c b/block/gluster.c
index 6de418c..645b7f1 100644
--- a/block/gluster.c
+++ b/block/gluster.c
@@ -493,6 +493,19 @@ out:
return NULL;
}
+static int qemu_gluster_truncate(BlockDriverState *bs, int64_t offset)
+{
+ int ret;
+ BDRVGlusterState *s = bs->opaque;
+
+ ret = glfs_ftruncate(s->fd, offset);
+ if (ret < 0) {
+ return -errno;
+ }
+
+ return 0;
+}
+
static BlockDriverAIOCB *qemu_gluster_aio_readv(BlockDriverState *bs,
int64_t sector_num, QEMUIOVector *qiov, int nb_sectors,
BlockDriverCompletionFunc *cb, void *opaque)
@@ -631,6 +644,7 @@ static BlockDriver bdrv_gluster = {
.bdrv_create = qemu_gluster_create,
.bdrv_getlength = qemu_gluster_getlength,
.bdrv_get_allocated_file_size = qemu_gluster_allocated_file_size,
+ .bdrv_truncate = qemu_gluster_truncate,
.bdrv_aio_readv = qemu_gluster_aio_readv,
.bdrv_aio_writev = qemu_gluster_aio_writev,
.bdrv_aio_flush = qemu_gluster_aio_flush,
@@ -650,6 +664,7 @@ static BlockDriver bdrv_gluster_tcp = {
.bdrv_create = qemu_gluster_create,
.bdrv_getlength = qemu_gluster_getlength,
.bdrv_get_allocated_file_size = qemu_gluster_allocated_file_size,
+ .bdrv_truncate = qemu_gluster_truncate,
.bdrv_aio_readv = qemu_gluster_aio_readv,
.bdrv_aio_writev = qemu_gluster_aio_writev,
.bdrv_aio_flush = qemu_gluster_aio_flush,
@@ -669,6 +684,7 @@ static BlockDriver bdrv_gluster_unix = {
.bdrv_create = qemu_gluster_create,
.bdrv_getlength = qemu_gluster_getlength,
.bdrv_get_allocated_file_size = qemu_gluster_allocated_file_size,
+ .bdrv_truncate = qemu_gluster_truncate,
.bdrv_aio_readv = qemu_gluster_aio_readv,
.bdrv_aio_writev = qemu_gluster_aio_writev,
.bdrv_aio_flush = qemu_gluster_aio_flush,
@@ -688,6 +704,7 @@ static BlockDriver bdrv_gluster_rdma = {
.bdrv_create = qemu_gluster_create,
.bdrv_getlength = qemu_gluster_getlength,
.bdrv_get_allocated_file_size = qemu_gluster_allocated_file_size,
+ .bdrv_truncate = qemu_gluster_truncate,
.bdrv_aio_readv = qemu_gluster_aio_readv,
.bdrv_aio_writev = qemu_gluster_aio_writev,
.bdrv_aio_flush = qemu_gluster_aio_flush,
--
1.8.1.4
^ permalink raw reply related [flat|nested] 7+ messages in thread* [Qemu-devel] [PATCH 2/6] block: Repair the throttling code.
2013-07-29 15:36 [Qemu-devel] [PULL for-1.6 0/6] Block patches Stefan Hajnoczi
2013-07-29 15:36 ` [Qemu-devel] [PATCH 1/6] gluster: Add image resize support Stefan Hajnoczi
@ 2013-07-29 15:36 ` Stefan Hajnoczi
2013-07-29 15:36 ` [Qemu-devel] [PATCH 3/6] migration: notify migration state before starting thread Stefan Hajnoczi
` (3 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Stefan Hajnoczi @ 2013-07-29 15:36 UTC (permalink / raw)
To: qemu-devel; +Cc: Anthony Liguori, Benoît Canet, Stefan Hajnoczi
From: Benoît Canet <benoit@irqsave.net>
The throttling code was segfaulting since commit
02ffb504485f0920cfc75a0982a602f824a9a4f4 because some qemu_co_queue_next caller
does not run in a coroutine.
qemu_co_queue_do_restart assume that the caller is a coroutinne.
As suggested by Stefan fix this by entering the coroutine directly.
Also make sure like suggested that qemu_co_queue_next() and
qemu_co_queue_restart_all() can be called only in coroutines.
Signed-off-by: Benoit Canet <benoit@irqsave.net>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
block.c | 7 +++----
include/block/coroutine.h | 9 +++++++--
qemu-coroutine-lock.c | 20 ++++++++++++++++++--
3 files changed, 28 insertions(+), 8 deletions(-)
diff --git a/block.c b/block.c
index c77cfd1..01b66d8 100644
--- a/block.c
+++ b/block.c
@@ -127,7 +127,7 @@ void bdrv_io_limits_disable(BlockDriverState *bs)
{
bs->io_limits_enabled = false;
- while (qemu_co_queue_next(&bs->throttled_reqs));
+ do {} while (qemu_co_enter_next(&bs->throttled_reqs));
if (bs->block_timer) {
qemu_del_timer(bs->block_timer);
@@ -143,7 +143,7 @@ static void bdrv_block_timer(void *opaque)
{
BlockDriverState *bs = opaque;
- qemu_co_queue_next(&bs->throttled_reqs);
+ qemu_co_enter_next(&bs->throttled_reqs);
}
void bdrv_io_limits_enable(BlockDriverState *bs)
@@ -1452,8 +1452,7 @@ void bdrv_drain_all(void)
* a busy wait.
*/
QTAILQ_FOREACH(bs, &bdrv_states, list) {
- if (!qemu_co_queue_empty(&bs->throttled_reqs)) {
- qemu_co_queue_restart_all(&bs->throttled_reqs);
+ while (qemu_co_enter_next(&bs->throttled_reqs)) {
busy = true;
}
}
diff --git a/include/block/coroutine.h b/include/block/coroutine.h
index 377805a..1f2db3e 100644
--- a/include/block/coroutine.h
+++ b/include/block/coroutine.h
@@ -130,12 +130,17 @@ void coroutine_fn qemu_co_queue_wait_insert_head(CoQueue *queue);
*
* Returns true if a coroutine was restarted, false if the queue is empty.
*/
-bool qemu_co_queue_next(CoQueue *queue);
+bool coroutine_fn qemu_co_queue_next(CoQueue *queue);
/**
* Restarts all coroutines in the CoQueue and leaves the queue empty.
*/
-void qemu_co_queue_restart_all(CoQueue *queue);
+void coroutine_fn qemu_co_queue_restart_all(CoQueue *queue);
+
+/**
+ * Enter the next coroutine in the queue
+ */
+bool qemu_co_enter_next(CoQueue *queue);
/**
* Checks if the CoQueue is empty.
diff --git a/qemu-coroutine-lock.c b/qemu-coroutine-lock.c
index d9fea49..aeb33b9 100644
--- a/qemu-coroutine-lock.c
+++ b/qemu-coroutine-lock.c
@@ -88,16 +88,32 @@ static bool qemu_co_queue_do_restart(CoQueue *queue, bool single)
return true;
}
-bool qemu_co_queue_next(CoQueue *queue)
+bool coroutine_fn qemu_co_queue_next(CoQueue *queue)
{
+ assert(qemu_in_coroutine());
return qemu_co_queue_do_restart(queue, true);
}
-void qemu_co_queue_restart_all(CoQueue *queue)
+void coroutine_fn qemu_co_queue_restart_all(CoQueue *queue)
{
+ assert(qemu_in_coroutine());
qemu_co_queue_do_restart(queue, false);
}
+bool qemu_co_enter_next(CoQueue *queue)
+{
+ Coroutine *next;
+
+ next = QTAILQ_FIRST(&queue->entries);
+ if (!next) {
+ return false;
+ }
+
+ QTAILQ_REMOVE(&queue->entries, next, co_queue_next);
+ qemu_coroutine_enter(next, NULL);
+ return true;
+}
+
bool qemu_co_queue_empty(CoQueue *queue)
{
return (QTAILQ_FIRST(&queue->entries) == NULL);
--
1.8.1.4
^ permalink raw reply related [flat|nested] 7+ messages in thread* [Qemu-devel] [PATCH 3/6] migration: notify migration state before starting thread
2013-07-29 15:36 [Qemu-devel] [PULL for-1.6 0/6] Block patches Stefan Hajnoczi
2013-07-29 15:36 ` [Qemu-devel] [PATCH 1/6] gluster: Add image resize support Stefan Hajnoczi
2013-07-29 15:36 ` [Qemu-devel] [PATCH 2/6] block: Repair the throttling code Stefan Hajnoczi
@ 2013-07-29 15:36 ` Stefan Hajnoczi
2013-07-29 15:36 ` [Qemu-devel] [PATCH 4/6] migration: fix spice migration Stefan Hajnoczi
` (2 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Stefan Hajnoczi @ 2013-07-29 15:36 UTC (permalink / raw)
To: qemu-devel; +Cc: Anthony Liguori, Stefan Hajnoczi
The migration thread runs outside the QEMU global mutex when possible.
Therefore we must notify migration state change *before* starting the
migration thread.
This allows registered listeners to act before live migration iterations
begin. Therefore they can get into a state that allows for live
migration. When the migration thread starts everything will be ready.
Without this patch there is a race condition during migration setup,
depending on whether the migration thread has already transitioned from
SETUP to ACTIVE state.
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
migration.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/migration.c b/migration.c
index e12e784..a5ed26b 100644
--- a/migration.c
+++ b/migration.c
@@ -658,7 +658,9 @@ void migrate_fd_connect(MigrationState *s)
qemu_file_set_rate_limit(s->file,
s->bandwidth_limit / XFER_LIMIT_RATIO);
+ /* Notify before starting migration thread */
+ notifier_list_notify(&migration_state_notifiers, s);
+
qemu_thread_create(&s->thread, migration_thread, s,
QEMU_THREAD_JOINABLE);
- notifier_list_notify(&migration_state_notifiers, s);
}
--
1.8.1.4
^ permalink raw reply related [flat|nested] 7+ messages in thread* [Qemu-devel] [PATCH 4/6] migration: fix spice migration
2013-07-29 15:36 [Qemu-devel] [PULL for-1.6 0/6] Block patches Stefan Hajnoczi
` (2 preceding siblings ...)
2013-07-29 15:36 ` [Qemu-devel] [PATCH 3/6] migration: notify migration state before starting thread Stefan Hajnoczi
@ 2013-07-29 15:36 ` Stefan Hajnoczi
2013-07-29 15:36 ` [Qemu-devel] [PATCH 5/6] dataplane: enable virtio-blk x-data-plane=on live migration Stefan Hajnoczi
2013-07-29 15:36 ` [Qemu-devel] [PATCH 6/6] dataplane: refuse to start if device is already in use Stefan Hajnoczi
5 siblings, 0 replies; 7+ messages in thread
From: Stefan Hajnoczi @ 2013-07-29 15:36 UTC (permalink / raw)
To: qemu-devel; +Cc: Anthony Liguori, Michael R. Hines, Stefan Hajnoczi
Commit 29ae8a4133082e16970c9d4be09f4b6a15034617 ("rdma: introduce
MIG_STATE_NONE and change MIG_STATE_SETUP state transition") changed the
state transitions during migration setup.
Spice used to be notified with MIG_STATE_ACTIVE and it detected this
using migration_is_active(). Spice is now notified with
MIG_STATE_SETUP and migration_is_active() no longer works.
Replace migration_is_active() with migration_in_setup() to fix spice
migration.
Cc: Michael R. Hines <mrhines@us.ibm.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
include/migration/migration.h | 2 +-
migration.c | 4 ++--
ui/spice-core.c | 2 +-
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/include/migration/migration.h b/include/migration/migration.h
index 08c772d..140e6b4 100644
--- a/include/migration/migration.h
+++ b/include/migration/migration.h
@@ -90,7 +90,7 @@ int migrate_fd_close(MigrationState *s);
void add_migration_state_change_notifier(Notifier *notify);
void remove_migration_state_change_notifier(Notifier *notify);
-bool migration_is_active(MigrationState *);
+bool migration_in_setup(MigrationState *);
bool migration_has_finished(MigrationState *);
bool migration_has_failed(MigrationState *);
MigrationState *migrate_get_current(void);
diff --git a/migration.c b/migration.c
index a5ed26b..9fc7294 100644
--- a/migration.c
+++ b/migration.c
@@ -338,9 +338,9 @@ void remove_migration_state_change_notifier(Notifier *notify)
notifier_remove(notify);
}
-bool migration_is_active(MigrationState *s)
+bool migration_in_setup(MigrationState *s)
{
- return s->state == MIG_STATE_ACTIVE;
+ return s->state == MIG_STATE_SETUP;
}
bool migration_has_finished(MigrationState *s)
diff --git a/ui/spice-core.c b/ui/spice-core.c
index f308fd9..033fd89 100644
--- a/ui/spice-core.c
+++ b/ui/spice-core.c
@@ -563,7 +563,7 @@ static void migration_state_notifier(Notifier *notifier, void *data)
{
MigrationState *s = data;
- if (migration_is_active(s)) {
+ if (migration_in_setup(s)) {
spice_server_migrate_start(spice_server);
} else if (migration_has_finished(s)) {
spice_server_migrate_end(spice_server, true);
--
1.8.1.4
^ permalink raw reply related [flat|nested] 7+ messages in thread* [Qemu-devel] [PATCH 5/6] dataplane: enable virtio-blk x-data-plane=on live migration
2013-07-29 15:36 [Qemu-devel] [PULL for-1.6 0/6] Block patches Stefan Hajnoczi
` (3 preceding siblings ...)
2013-07-29 15:36 ` [Qemu-devel] [PATCH 4/6] migration: fix spice migration Stefan Hajnoczi
@ 2013-07-29 15:36 ` Stefan Hajnoczi
2013-07-29 15:36 ` [Qemu-devel] [PATCH 6/6] dataplane: refuse to start if device is already in use Stefan Hajnoczi
5 siblings, 0 replies; 7+ messages in thread
From: Stefan Hajnoczi @ 2013-07-29 15:36 UTC (permalink / raw)
To: qemu-devel; +Cc: Anthony Liguori, Stefan Hajnoczi
Although the dataplane thread does not cooperate with dirty memory
logging yet it's fairly easy to temporarily disable dataplane during
live migration. This way virtio-blk can live migrate when
x-data-plane=on.
The dataplane thread will restart after migration is cancelled or if the
guest resuming virtio-blk operation after migration completes.
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
hw/block/dataplane/virtio-blk.c | 9 ---------
hw/block/virtio-blk.c | 32 ++++++++++++++++++++++++++++++++
include/hw/virtio/virtio-blk.h | 1 +
3 files changed, 33 insertions(+), 9 deletions(-)
diff --git a/hw/block/dataplane/virtio-blk.c b/hw/block/dataplane/virtio-blk.c
index 2faed43..63c3ffa 100644
--- a/hw/block/dataplane/virtio-blk.c
+++ b/hw/block/dataplane/virtio-blk.c
@@ -18,7 +18,6 @@
#include "qemu/error-report.h"
#include "hw/virtio/dataplane/vring.h"
#include "ioq.h"
-#include "migration/migration.h"
#include "block/block.h"
#include "hw/virtio/virtio-blk.h"
#include "virtio-blk.h"
@@ -69,8 +68,6 @@ struct VirtIOBlockDataPlane {
queue */
unsigned int num_reqs;
-
- Error *migration_blocker;
};
/* Raise an interrupt to signal guest, if necessary */
@@ -433,10 +430,6 @@ bool virtio_blk_data_plane_create(VirtIODevice *vdev, VirtIOBlkConf *blk,
/* Prevent block operations that conflict with data plane thread */
bdrv_set_in_use(blk->conf.bs, 1);
- error_setg(&s->migration_blocker,
- "x-data-plane does not support migration");
- migrate_add_blocker(s->migration_blocker);
-
*dataplane = s;
return true;
}
@@ -448,8 +441,6 @@ void virtio_blk_data_plane_destroy(VirtIOBlockDataPlane *s)
}
virtio_blk_data_plane_stop(s);
- migrate_del_blocker(s->migration_blocker);
- error_free(s->migration_blocker);
bdrv_set_in_use(s->blk->conf.bs, 0);
g_free(s);
}
diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index cf12469..1237b6a 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -19,6 +19,7 @@
#include "hw/virtio/virtio-blk.h"
#ifdef CONFIG_VIRTIO_BLK_DATA_PLANE
# include "dataplane/virtio-blk.h"
+# include "migration/migration.h"
#endif
#include "block/scsi.h"
#ifdef __linux__
@@ -628,6 +629,34 @@ void virtio_blk_set_conf(DeviceState *dev, VirtIOBlkConf *blk)
memcpy(&(s->blk), blk, sizeof(struct VirtIOBlkConf));
}
+#ifdef CONFIG_VIRTIO_BLK_DATA_PLANE
+/* Disable dataplane thread during live migration since it does not
+ * update the dirty memory bitmap yet.
+ */
+static void virtio_blk_migration_state_changed(Notifier *notifier, void *data)
+{
+ VirtIOBlock *s = container_of(notifier, VirtIOBlock,
+ migration_state_notifier);
+ MigrationState *mig = data;
+
+ if (migration_in_setup(mig)) {
+ if (!s->dataplane) {
+ return;
+ }
+ virtio_blk_data_plane_destroy(s->dataplane);
+ s->dataplane = NULL;
+ } else if (migration_has_finished(mig) ||
+ migration_has_failed(mig)) {
+ if (s->dataplane) {
+ return;
+ }
+ bdrv_drain_all(); /* complete in-flight non-dataplane requests */
+ virtio_blk_data_plane_create(VIRTIO_DEVICE(s), &s->blk,
+ &s->dataplane);
+ }
+}
+#endif /* CONFIG_VIRTIO_BLK_DATA_PLANE */
+
static int virtio_blk_device_init(VirtIODevice *vdev)
{
DeviceState *qdev = DEVICE(vdev);
@@ -664,6 +693,8 @@ static int virtio_blk_device_init(VirtIODevice *vdev)
virtio_cleanup(vdev);
return -1;
}
+ s->migration_state_notifier.notify = virtio_blk_migration_state_changed;
+ add_migration_state_change_notifier(&s->migration_state_notifier);
#endif
s->change = qemu_add_vm_change_state_handler(virtio_blk_dma_restart_cb, s);
@@ -683,6 +714,7 @@ static int virtio_blk_device_exit(DeviceState *dev)
VirtIODevice *vdev = VIRTIO_DEVICE(dev);
VirtIOBlock *s = VIRTIO_BLK(dev);
#ifdef CONFIG_VIRTIO_BLK_DATA_PLANE
+ remove_migration_state_change_notifier(&s->migration_state_notifier);
virtio_blk_data_plane_destroy(s->dataplane);
s->dataplane = NULL;
#endif
diff --git a/include/hw/virtio/virtio-blk.h b/include/hw/virtio/virtio-blk.h
index fc71853..b87cf49 100644
--- a/include/hw/virtio/virtio-blk.h
+++ b/include/hw/virtio/virtio-blk.h
@@ -125,6 +125,7 @@ typedef struct VirtIOBlock {
unsigned short sector_mask;
VMChangeStateEntry *change;
#ifdef CONFIG_VIRTIO_BLK_DATA_PLANE
+ Notifier migration_state_notifier;
struct VirtIOBlockDataPlane *dataplane;
#endif
} VirtIOBlock;
--
1.8.1.4
^ permalink raw reply related [flat|nested] 7+ messages in thread* [Qemu-devel] [PATCH 6/6] dataplane: refuse to start if device is already in use
2013-07-29 15:36 [Qemu-devel] [PULL for-1.6 0/6] Block patches Stefan Hajnoczi
` (4 preceding siblings ...)
2013-07-29 15:36 ` [Qemu-devel] [PATCH 5/6] dataplane: enable virtio-blk x-data-plane=on live migration Stefan Hajnoczi
@ 2013-07-29 15:36 ` Stefan Hajnoczi
5 siblings, 0 replies; 7+ messages in thread
From: Stefan Hajnoczi @ 2013-07-29 15:36 UTC (permalink / raw)
To: qemu-devel; +Cc: Anthony Liguori, qemu-stable, Stefan Hajnoczi
Dataplane must check whether a block device is in use before launching
the dataplane thread. This is necessary since the thread does not
synchronize with the main loop and I/O requests could cause corruption.
One example is when a drive is added and a block job is started before
hotplugging the virtio-blk-pci adapter. In this case we must not use
dataplane mode.
Cc: qemu-stable@nongnu.org
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
hw/block/dataplane/virtio-blk.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/hw/block/dataplane/virtio-blk.c b/hw/block/dataplane/virtio-blk.c
index 63c3ffa..411becc 100644
--- a/hw/block/dataplane/virtio-blk.c
+++ b/hw/block/dataplane/virtio-blk.c
@@ -415,6 +415,14 @@ bool virtio_blk_data_plane_create(VirtIODevice *vdev, VirtIOBlkConf *blk,
return false;
}
+ /* If dataplane is (re-)enabled while the guest is running there could be
+ * block jobs that can conflict.
+ */
+ if (bdrv_in_use(blk->conf.bs)) {
+ error_report("cannot start dataplane thread while device is in use");
+ return false;
+ }
+
fd = raw_get_aio_fd(blk->conf.bs);
if (fd < 0) {
error_report("drive is incompatible with x-data-plane, "
--
1.8.1.4
^ permalink raw reply related [flat|nested] 7+ messages in thread