* [Qemu-devel] [PATCH v8 0/4] The intro of QEMU block I/O throttling
@ 2011-09-07 12:31 Zhi Yong Wu
2011-09-07 12:31 ` [Qemu-devel] [PATCH v8 1/4] block: add the block queue support Zhi Yong Wu
2011-09-07 12:31 ` [Qemu-devel] [PATCH v8 2/4] block: add the command line support Zhi Yong Wu
0 siblings, 2 replies; 13+ messages in thread
From: Zhi Yong Wu @ 2011-09-07 12:31 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, aliguori, stefanha, kvm, mtosatti, Zhi Yong Wu, zwu.kernel,
ryanh
The main goal of the patch is to effectively cap the disk I/O speed or counts of one single VM.It is only one draft, so it unavoidably has some drawbacks, if you catch them, please let me know.
The patch will mainly introduce one block I/O throttling algorithm, one timer and one block queue for each I/O limits enabled drive.
When a block request is coming in, the throttling algorithm will check if its I/O rate or counts exceed the limits; if yes, then it will enqueue to the block queue; The timer will handle the I/O requests in it.
Some available features follow as below:
(1) global bps limit.
-drive bps=xxx in bytes/s
(2) only read bps limit
-drive bps_rd=xxx in bytes/s
(3) only write bps limit
-drive bps_wr=xxx in bytes/s
(4) global iops limit
-drive iops=xxx in ios/s
(5) only read iops limit
-drive iops_rd=xxx in ios/s
(6) only write iops limit
-drive iops_wr=xxx in ios/s
(7) the combination of some limits.
-drive bps=xxx,iops=xxx
Known Limitations:
(1) #1 can not coexist with #2, #3
(2) #4 can not coexist with #5, #6
(3) When bps/iops limits are specified to a small value such as 511 bytes/s, this VM will hang up. We are considering how to handle this senario.
Changes since code V7:
fix the build per patch based on stefan's comments.
Zhi Yong Wu (4):
block: add the command line support
block: add the block queue support
block: add block timer and throttling algorithm
qmp/hmp: add block_set_io_throttle
v7: Mainly simply the block queue.
Adjust codes based on stefan's comments.
v6: Mainly fix the aio callback issue for block queue.
Adjust codes based on Ram Pai's comments.
v5: add qmp/hmp support.
Adjust the codes based on stefan's comments
qmp/hmp: add block_set_io_throttle
v4: fix memory leaking based on ryan's feedback.
v3: Added the code for extending slice time, and modified the method to compute wait time for the timer.
v2: The codes V2 for QEMU disk I/O limits.
Modified the codes mainly based on stefan's comments.
v1: Submit the codes for QEMU disk I/O limits.
Only a code draft.
Makefile.objs | 2 +-
block.c | 331 +++++++++++++++++++++++++++++++++++++++++++++++++++--
block.h | 6 +-
block/blk-queue.c | 184 +++++++++++++++++++++++++++++
block/blk-queue.h | 59 ++++++++++
block_int.h | 30 +++++
blockdev.c | 98 ++++++++++++++++
blockdev.h | 2 +
hmp-commands.hx | 15 +++
qemu-config.c | 24 ++++
qemu-options.hx | 1 +
qerror.c | 4 +
qerror.h | 3 +
qmp-commands.hx | 52 ++++++++-
14 files changed, 796 insertions(+), 15 deletions(-)
create mode 100644 block/blk-queue.c
create mode 100644 block/blk-queue.h
--
1.7.6
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Qemu-devel] [PATCH v8 1/4] block: add the block queue support
2011-09-07 12:31 [Qemu-devel] [PATCH v8 0/4] The intro of QEMU block I/O throttling Zhi Yong Wu
@ 2011-09-07 12:31 ` Zhi Yong Wu
2011-09-07 12:31 ` [Qemu-devel] [PATCH v8 2/4] block: add the command line support Zhi Yong Wu
1 sibling, 0 replies; 13+ messages in thread
From: Zhi Yong Wu @ 2011-09-07 12:31 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, aliguori, stefanha, kvm, mtosatti, Zhi Yong Wu, zwu.kernel,
ryanh
Signed-off-by: Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
---
Makefile.objs | 2 +-
block/blk-queue.c | 184 +++++++++++++++++++++++++++++++++++++++++++++++++++++
block/blk-queue.h | 59 +++++++++++++++++
block_int.h | 27 ++++++++
4 files changed, 271 insertions(+), 1 deletions(-)
create mode 100644 block/blk-queue.c
create mode 100644 block/blk-queue.h
diff --git a/Makefile.objs b/Makefile.objs
index 26b885b..5dcf456 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -33,7 +33,7 @@ block-nested-y += raw.o cow.o qcow.o vdi.o vmdk.o cloop.o dmg.o bochs.o vpc.o vv
block-nested-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o
block-nested-y += qed.o qed-gencb.o qed-l2-cache.o qed-table.o qed-cluster.o
block-nested-y += qed-check.o
-block-nested-y += parallels.o nbd.o blkdebug.o sheepdog.o blkverify.o
+block-nested-y += parallels.o nbd.o blkdebug.o sheepdog.o blkverify.o blk-queue.o
block-nested-$(CONFIG_WIN32) += raw-win32.o
block-nested-$(CONFIG_POSIX) += raw-posix.o
block-nested-$(CONFIG_CURL) += curl.o
diff --git a/block/blk-queue.c b/block/blk-queue.c
new file mode 100644
index 0000000..da01fcb
--- /dev/null
+++ b/block/blk-queue.c
@@ -0,0 +1,184 @@
+/*
+ * QEMU System Emulator queue definition for block layer
+ *
+ * Copyright (c) IBM, Corp. 2011
+ *
+ * Authors:
+ * Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
+ * Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "block_int.h"
+#include "block/blk-queue.h"
+#include "qemu-common.h"
+
+/* The APIs for block request queue on qemu block layer.
+ */
+
+struct BlockQueueAIOCB {
+ BlockDriverAIOCB common;
+ QTAILQ_ENTRY(BlockQueueAIOCB) entry;
+ BlockRequestHandler *handler;
+ BlockDriverAIOCB *real_acb;
+
+ int64_t sector_num;
+ QEMUIOVector *qiov;
+ int nb_sectors;
+};
+
+typedef struct BlockQueueAIOCB BlockQueueAIOCB;
+
+struct BlockQueue {
+ QTAILQ_HEAD(requests, BlockQueueAIOCB) requests;
+ bool flushing;
+};
+
+static void qemu_block_queue_dequeue(BlockQueue *queue,
+ BlockQueueAIOCB *request)
+{
+ BlockQueueAIOCB *req;
+
+ assert(queue);
+ while (!QTAILQ_EMPTY(&queue->requests)) {
+ req = QTAILQ_FIRST(&queue->requests);
+ if (req == request) {
+ QTAILQ_REMOVE(&queue->requests, req, entry);
+ break;
+ }
+ }
+}
+
+static void qemu_block_queue_cancel(BlockDriverAIOCB *acb)
+{
+ BlockQueueAIOCB *request = container_of(acb, BlockQueueAIOCB, common);
+ if (request->real_acb) {
+ bdrv_aio_cancel(request->real_acb);
+ } else {
+ assert(request->common.bs->block_queue);
+ qemu_block_queue_dequeue(request->common.bs->block_queue,
+ request);
+ }
+
+ qemu_aio_release(request);
+}
+
+static AIOPool block_queue_pool = {
+ .aiocb_size = sizeof(struct BlockQueueAIOCB),
+ .cancel = qemu_block_queue_cancel,
+};
+
+BlockQueue *qemu_new_block_queue(void)
+{
+ BlockQueue *queue;
+
+ queue = g_malloc0(sizeof(BlockQueue));
+
+ QTAILQ_INIT(&queue->requests);
+
+ queue->flushing = false;
+
+ return queue;
+}
+
+void qemu_del_block_queue(BlockQueue *queue)
+{
+ BlockQueueAIOCB *request, *next;
+
+ QTAILQ_FOREACH_SAFE(request, &queue->requests, entry, next) {
+ QTAILQ_REMOVE(&queue->requests, request, entry);
+ qemu_aio_release(request);
+ }
+
+ g_free(queue);
+}
+
+BlockDriverAIOCB *qemu_block_queue_enqueue(BlockQueue *queue,
+ BlockDriverState *bs,
+ BlockRequestHandler *handler,
+ int64_t sector_num,
+ QEMUIOVector *qiov,
+ int nb_sectors,
+ BlockDriverCompletionFunc *cb,
+ void *opaque)
+{
+ BlockDriverAIOCB *acb;
+ BlockQueueAIOCB *request;
+
+ if (queue->flushing) {
+ return NULL;
+ } else {
+ acb = qemu_aio_get(&block_queue_pool, bs,
+ cb, opaque);
+ request = container_of(acb, BlockQueueAIOCB, common);
+ request->handler = handler;
+ request->sector_num = sector_num;
+ request->qiov = qiov;
+ request->nb_sectors = nb_sectors;
+ request->real_acb = NULL;
+ QTAILQ_INSERT_TAIL(&queue->requests, request, entry);
+ }
+
+ return acb;
+}
+
+static int qemu_block_queue_handler(BlockQueueAIOCB *request)
+{
+ int ret;
+ BlockDriverAIOCB *res;
+
+ res = request->handler(request->common.bs, request->sector_num,
+ request->qiov, request->nb_sectors,
+ request->common.cb, request->common.opaque);
+ if (res) {
+ request->real_acb = res;
+ }
+
+ ret = (res == NULL) ? 0 : 1;
+
+ return ret;
+}
+
+void qemu_block_queue_flush(BlockQueue *queue)
+{
+ queue->flushing = true;
+ while (!QTAILQ_EMPTY(&queue->requests)) {
+ BlockQueueAIOCB *request = NULL;
+ int ret = 0;
+
+ request = QTAILQ_FIRST(&queue->requests);
+ QTAILQ_REMOVE(&queue->requests, request, entry);
+
+ ret = qemu_block_queue_handler(request);
+ if (ret == 0) {
+ QTAILQ_INSERT_HEAD(&queue->requests, request, entry);
+ break;
+ }
+
+ qemu_aio_release(request);
+ }
+
+ queue->flushing = false;
+}
+
+bool qemu_block_queue_has_pending(BlockQueue *queue)
+{
+ return !queue->flushing && !QTAILQ_EMPTY(&queue->requests);
+}
diff --git a/block/blk-queue.h b/block/blk-queue.h
new file mode 100644
index 0000000..c1529f7
--- /dev/null
+++ b/block/blk-queue.h
@@ -0,0 +1,59 @@
+/*
+ * QEMU System Emulator queue declaration for block layer
+ *
+ * Copyright (c) IBM, Corp. 2011
+ *
+ * Authors:
+ * Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
+ * Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#ifndef QEMU_BLOCK_QUEUE_H
+#define QEMU_BLOCK_QUEUE_H
+
+#include "block.h"
+#include "qemu-queue.h"
+
+typedef BlockDriverAIOCB* (BlockRequestHandler) (BlockDriverState *bs,
+ int64_t sector_num, QEMUIOVector *qiov,
+ int nb_sectors, BlockDriverCompletionFunc *cb,
+ void *opaque);
+
+typedef struct BlockQueue BlockQueue;
+
+BlockQueue *qemu_new_block_queue(void);
+
+void qemu_del_block_queue(BlockQueue *queue);
+
+BlockDriverAIOCB *qemu_block_queue_enqueue(BlockQueue *queue,
+ BlockDriverState *bs,
+ BlockRequestHandler *handler,
+ int64_t sector_num,
+ QEMUIOVector *qiov,
+ int nb_sectors,
+ BlockDriverCompletionFunc *cb,
+ void *opaque);
+
+void qemu_block_queue_flush(BlockQueue *queue);
+
+bool qemu_block_queue_has_pending(BlockQueue *queue);
+
+#endif /* QEMU_BLOCK_QUEUE_H */
diff --git a/block_int.h b/block_int.h
index 8a72b80..201e635 100644
--- a/block_int.h
+++ b/block_int.h
@@ -29,10 +29,18 @@
#include "qemu-queue.h"
#include "qemu-coroutine.h"
#include "qemu-timer.h"
+#include "block/blk-queue.h"
#define BLOCK_FLAG_ENCRYPT 1
#define BLOCK_FLAG_COMPAT6 4
+#define BLOCK_IO_LIMIT_READ 0
+#define BLOCK_IO_LIMIT_WRITE 1
+#define BLOCK_IO_LIMIT_TOTAL 2
+
+#define BLOCK_IO_SLICE_TIME 100000000
+#define NANOSECONDS_PER_SECOND 1000000000.0
+
#define BLOCK_OPT_SIZE "size"
#define BLOCK_OPT_ENCRYPT "encryption"
#define BLOCK_OPT_COMPAT6 "compat6"
@@ -49,6 +57,16 @@ typedef struct AIOPool {
BlockDriverAIOCB *free_aiocb;
} AIOPool;
+typedef struct BlockIOLimit {
+ uint64_t bps[3];
+ uint64_t iops[3];
+} BlockIOLimit;
+
+typedef struct BlockIODisp {
+ uint64_t bytes[2];
+ uint64_t ios[2];
+} BlockIODisp;
+
struct BlockDriver {
const char *format_name;
int instance_size;
@@ -184,6 +202,15 @@ struct BlockDriverState {
void *sync_aiocb;
+ /* the time for latest disk I/O */
+ int64_t slice_start;
+ int64_t slice_end;
+ BlockIOLimit io_limits;
+ BlockIODisp io_disps;
+ BlockQueue *block_queue;
+ QEMUTimer *block_timer;
+ bool io_limits_enabled;
+
/* I/O stats (display with "info blockstats"). */
uint64_t nr_bytes[BDRV_MAX_IOTYPE];
uint64_t nr_ops[BDRV_MAX_IOTYPE];
--
1.7.6
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [Qemu-devel] [PATCH v8 2/4] block: add the command line support
2011-09-07 12:31 [Qemu-devel] [PATCH v8 0/4] The intro of QEMU block I/O throttling Zhi Yong Wu
2011-09-07 12:31 ` [Qemu-devel] [PATCH v8 1/4] block: add the block queue support Zhi Yong Wu
@ 2011-09-07 12:31 ` Zhi Yong Wu
1 sibling, 0 replies; 13+ messages in thread
From: Zhi Yong Wu @ 2011-09-07 12:31 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, aliguori, stefanha, kvm, mtosatti, Zhi Yong Wu, zwu.kernel,
ryanh
Signed-off-by: Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
---
block.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
block.h | 5 ++++
block_int.h | 3 ++
blockdev.c | 29 +++++++++++++++++++++++++++
qemu-config.c | 24 ++++++++++++++++++++++
qemu-options.hx | 1 +
6 files changed, 121 insertions(+), 0 deletions(-)
diff --git a/block.c b/block.c
index 43742b7..cd75183 100644
--- a/block.c
+++ b/block.c
@@ -104,6 +104,57 @@ int is_windows_drive(const char *filename)
}
#endif
+/* throttling disk I/O limits */
+void bdrv_io_limits_disable(BlockDriverState *bs)
+{
+ bs->io_limits_enabled = false;
+
+ if (bs->block_queue) {
+ qemu_block_queue_flush(bs->block_queue);
+ qemu_del_block_queue(bs->block_queue);
+ bs->block_queue = NULL;
+ }
+
+ if (bs->block_timer) {
+ qemu_del_timer(bs->block_timer);
+ qemu_free_timer(bs->block_timer);
+ bs->block_timer = NULL;
+ }
+
+ bs->slice_start = 0;
+
+ bs->slice_end = 0;
+}
+
+static void bdrv_block_timer(void *opaque)
+{
+ BlockDriverState *bs = opaque;
+ BlockQueue *queue = bs->block_queue;
+
+ qemu_block_queue_flush(queue);
+}
+
+void bdrv_io_limits_enable(BlockDriverState *bs)
+{
+ bs->block_queue = qemu_new_block_queue();
+ bs->block_timer = qemu_new_timer_ns(vm_clock, bdrv_block_timer, bs);
+
+ bs->slice_start = qemu_get_clock_ns(vm_clock);
+
+ bs->slice_end = bs->slice_start + BLOCK_IO_SLICE_TIME;
+}
+
+bool bdrv_io_limits_enabled(BlockDriverState *bs)
+{
+ BlockIOLimit *io_limits = &bs->io_limits;
+ return io_limits->bps[BLOCK_IO_LIMIT_READ]
+ || io_limits->bps[BLOCK_IO_LIMIT_WRITE]
+ || io_limits->bps[BLOCK_IO_LIMIT_TOTAL]
+ || io_limits->iops[BLOCK_IO_LIMIT_READ]
+ || io_limits->iops[BLOCK_IO_LIMIT_WRITE]
+ || io_limits->iops[BLOCK_IO_LIMIT_TOTAL];
+}
+
/* check if the path starts with "<protocol>:" */
static int path_has_protocol(const char *path)
{
@@ -1453,6 +1504,14 @@ void bdrv_get_geometry_hint(BlockDriverState *bs,
*psecs = bs->secs;
}
+/* throttling disk io limits */
+void bdrv_set_io_limits(BlockDriverState *bs,
+ BlockIOLimit *io_limits)
+{
+ bs->io_limits = *io_limits;
+ bs->io_limits_enabled = bdrv_io_limits_enabled(bs);
+}
+
/* Recognize floppy formats */
typedef struct FDFormat {
FDriveType drive;
diff --git a/block.h b/block.h
index 3ac0b94..a3e69db 100644
--- a/block.h
+++ b/block.h
@@ -58,6 +58,11 @@ void bdrv_info(Monitor *mon, QObject **ret_data);
void bdrv_stats_print(Monitor *mon, const QObject *data);
void bdrv_info_stats(Monitor *mon, QObject **ret_data);
+/* disk I/O throttling */
+void bdrv_io_limits_enable(BlockDriverState *bs);
+void bdrv_io_limits_disable(BlockDriverState *bs);
+bool bdrv_io_limits_enabled(BlockDriverState *bs);
+
void bdrv_init(void);
void bdrv_init_with_whitelist(void);
BlockDriver *bdrv_find_protocol(const char *filename);
diff --git a/block_int.h b/block_int.h
index 201e635..368c776 100644
--- a/block_int.h
+++ b/block_int.h
@@ -257,6 +257,9 @@ void qemu_aio_release(void *p);
void *qemu_blockalign(BlockDriverState *bs, size_t size);
+void bdrv_set_io_limits(BlockDriverState *bs,
+ BlockIOLimit *io_limits);
+
#ifdef _WIN32
int is_windows_drive(const char *filename);
#endif
diff --git a/blockdev.c b/blockdev.c
index 2602591..619ae9f 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -236,6 +236,7 @@ DriveInfo *drive_init(QemuOpts *opts, int default_to_scsi)
int on_read_error, on_write_error;
const char *devaddr;
DriveInfo *dinfo;
+ BlockIOLimit io_limits;
int snapshot = 0;
int ret;
@@ -354,6 +355,31 @@ DriveInfo *drive_init(QemuOpts *opts, int default_to_scsi)
}
}
+ /* disk I/O throttling */
+ io_limits.bps[BLOCK_IO_LIMIT_TOTAL] =
+ qemu_opt_get_number(opts, "bps", 0);
+ io_limits.bps[BLOCK_IO_LIMIT_READ] =
+ qemu_opt_get_number(opts, "bps_rd", 0);
+ io_limits.bps[BLOCK_IO_LIMIT_WRITE] =
+ qemu_opt_get_number(opts, "bps_wr", 0);
+ io_limits.iops[BLOCK_IO_LIMIT_TOTAL] =
+ qemu_opt_get_number(opts, "iops", 0);
+ io_limits.iops[BLOCK_IO_LIMIT_READ] =
+ qemu_opt_get_number(opts, "iops_rd", 0);
+ io_limits.iops[BLOCK_IO_LIMIT_WRITE] =
+ qemu_opt_get_number(opts, "iops_wr", 0);
+
+ if (((io_limits.bps[BLOCK_IO_LIMIT_TOTAL] != 0)
+ && ((io_limits.bps[BLOCK_IO_LIMIT_READ] != 0)
+ || (io_limits.bps[BLOCK_IO_LIMIT_WRITE] != 0)))
+ || ((io_limits.iops[BLOCK_IO_LIMIT_TOTAL] != 0)
+ && ((io_limits.iops[BLOCK_IO_LIMIT_READ] != 0)
+ || (io_limits.iops[BLOCK_IO_LIMIT_WRITE] != 0)))) {
+ error_report("bps(iops) and bps_rd/bps_wr(iops_rd/iops_wr)"
+ "cannot be used at the same time");
+ return NULL;
+ }
+
on_write_error = BLOCK_ERR_STOP_ENOSPC;
if ((buf = qemu_opt_get(opts, "werror")) != NULL) {
if (type != IF_IDE && type != IF_SCSI && type != IF_VIRTIO && type != IF_NONE) {
@@ -461,6 +487,9 @@ DriveInfo *drive_init(QemuOpts *opts, int default_to_scsi)
bdrv_set_on_error(dinfo->bdrv, on_read_error, on_write_error);
+ /* disk I/O throttling */
+ bdrv_set_io_limits(dinfo->bdrv, &io_limits);
+
switch(type) {
case IF_IDE:
case IF_SCSI:
diff --git a/qemu-config.c b/qemu-config.c
index 7a7854f..405e587 100644
--- a/qemu-config.c
+++ b/qemu-config.c
@@ -85,6 +85,30 @@ static QemuOptsList qemu_drive_opts = {
.name = "readonly",
.type = QEMU_OPT_BOOL,
.help = "open drive file as read-only",
+ },{
+ .name = "iops",
+ .type = QEMU_OPT_NUMBER,
+ .help = "limit total I/O operations per second",
+ },{
+ .name = "iops_rd",
+ .type = QEMU_OPT_NUMBER,
+ .help = "limit read operations per second",
+ },{
+ .name = "iops_wr",
+ .type = QEMU_OPT_NUMBER,
+ .help = "limit write operations per second",
+ },{
+ .name = "bps",
+ .type = QEMU_OPT_NUMBER,
+ .help = "limit total bytes per second",
+ },{
+ .name = "bps_rd",
+ .type = QEMU_OPT_NUMBER,
+ .help = "limit read bytes per second",
+ },{
+ .name = "bps_wr",
+ .type = QEMU_OPT_NUMBER,
+ .help = "limit write bytes per second",
},
{ /* end of list */ }
},
diff --git a/qemu-options.hx b/qemu-options.hx
index 659ecb2..2e42c5c 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -136,6 +136,7 @@ DEF("drive", HAS_ARG, QEMU_OPTION_drive,
" [,cache=writethrough|writeback|none|directsync|unsafe][,format=f]\n"
" [,serial=s][,addr=A][,id=name][,aio=threads|native]\n"
" [,readonly=on|off]\n"
+ " [[,bps=b]|[[,bps_rd=r][,bps_wr=w]]][[,iops=i]|[[,iops_rd=r][,iops_wr=w]]\n"
" use 'file' as a drive image\n", QEMU_ARCH_ALL)
STEXI
@item -drive @var{option}[,@var{option}[,@var{option}[,...]]]
--
1.7.6
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [Qemu-devel] [PATCH v8 1/4] block: add the block queue support
2011-09-08 10:11 [Qemu-devel] [PATCH v8 0/4] The intro of QEMU block I/O throttling Zhi Yong Wu
@ 2011-09-08 10:11 ` Zhi Yong Wu
2011-09-23 15:32 ` Kevin Wolf
0 siblings, 1 reply; 13+ messages in thread
From: Zhi Yong Wu @ 2011-09-08 10:11 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, aliguori, stefanha, kvm, mtosatti, Zhi Yong Wu, pair,
zwu.kernel, ryanh
Signed-off-by: Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
---
Makefile.objs | 2 +-
block/blk-queue.c | 201 +++++++++++++++++++++++++++++++++++++++++++++++++++++
block/blk-queue.h | 59 ++++++++++++++++
block_int.h | 27 +++++++
4 files changed, 288 insertions(+), 1 deletions(-)
create mode 100644 block/blk-queue.c
create mode 100644 block/blk-queue.h
diff --git a/Makefile.objs b/Makefile.objs
index 26b885b..5dcf456 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -33,7 +33,7 @@ block-nested-y += raw.o cow.o qcow.o vdi.o vmdk.o cloop.o dmg.o bochs.o vpc.o vv
block-nested-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o
block-nested-y += qed.o qed-gencb.o qed-l2-cache.o qed-table.o qed-cluster.o
block-nested-y += qed-check.o
-block-nested-y += parallels.o nbd.o blkdebug.o sheepdog.o blkverify.o
+block-nested-y += parallels.o nbd.o blkdebug.o sheepdog.o blkverify.o blk-queue.o
block-nested-$(CONFIG_WIN32) += raw-win32.o
block-nested-$(CONFIG_POSIX) += raw-posix.o
block-nested-$(CONFIG_CURL) += curl.o
diff --git a/block/blk-queue.c b/block/blk-queue.c
new file mode 100644
index 0000000..adef497
--- /dev/null
+++ b/block/blk-queue.c
@@ -0,0 +1,201 @@
+/*
+ * QEMU System Emulator queue definition for block layer
+ *
+ * Copyright (c) IBM, Corp. 2011
+ *
+ * Authors:
+ * Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
+ * Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "block_int.h"
+#include "block/blk-queue.h"
+#include "qemu-common.h"
+
+/* The APIs for block request queue on qemu block layer.
+ */
+
+struct BlockQueueAIOCB {
+ BlockDriverAIOCB common;
+ QTAILQ_ENTRY(BlockQueueAIOCB) entry;
+ BlockRequestHandler *handler;
+ BlockDriverAIOCB *real_acb;
+
+ int64_t sector_num;
+ QEMUIOVector *qiov;
+ int nb_sectors;
+};
+
+typedef struct BlockQueueAIOCB BlockQueueAIOCB;
+
+struct BlockQueue {
+ QTAILQ_HEAD(requests, BlockQueueAIOCB) requests;
+ bool req_failed;
+ bool flushing;
+};
+
+static void qemu_block_queue_dequeue(BlockQueue *queue,
+ BlockQueueAIOCB *request)
+{
+ BlockQueueAIOCB *req;
+
+ assert(queue);
+ while (!QTAILQ_EMPTY(&queue->requests)) {
+ req = QTAILQ_FIRST(&queue->requests);
+ if (req == request) {
+ QTAILQ_REMOVE(&queue->requests, req, entry);
+ break;
+ }
+ }
+}
+
+static void qemu_block_queue_cancel(BlockDriverAIOCB *acb)
+{
+ BlockQueueAIOCB *request = container_of(acb, BlockQueueAIOCB, common);
+ if (request->real_acb) {
+ bdrv_aio_cancel(request->real_acb);
+ } else {
+ assert(request->common.bs->block_queue);
+ qemu_block_queue_dequeue(request->common.bs->block_queue,
+ request);
+ }
+
+ qemu_aio_release(request);
+}
+
+static AIOPool block_queue_pool = {
+ .aiocb_size = sizeof(struct BlockQueueAIOCB),
+ .cancel = qemu_block_queue_cancel,
+};
+
+static void qemu_block_queue_callback(void *opaque, int ret)
+{
+ BlockQueueAIOCB *acb = opaque;
+
+ if (acb->common.cb) {
+ acb->common.cb(acb->common.opaque, ret);
+ }
+
+ qemu_aio_release(acb);
+}
+
+BlockQueue *qemu_new_block_queue(void)
+{
+ BlockQueue *queue;
+
+ queue = g_malloc0(sizeof(BlockQueue));
+
+ QTAILQ_INIT(&queue->requests);
+
+ queue->req_failed = true;
+ queue->flushing = false;
+
+ return queue;
+}
+
+void qemu_del_block_queue(BlockQueue *queue)
+{
+ BlockQueueAIOCB *request, *next;
+
+ QTAILQ_FOREACH_SAFE(request, &queue->requests, entry, next) {
+ QTAILQ_REMOVE(&queue->requests, request, entry);
+ qemu_aio_release(request);
+ }
+
+ g_free(queue);
+}
+
+BlockDriverAIOCB *qemu_block_queue_enqueue(BlockQueue *queue,
+ BlockDriverState *bs,
+ BlockRequestHandler *handler,
+ int64_t sector_num,
+ QEMUIOVector *qiov,
+ int nb_sectors,
+ BlockDriverCompletionFunc *cb,
+ void *opaque)
+{
+ BlockDriverAIOCB *acb;
+ BlockQueueAIOCB *request;
+
+ if (queue->flushing) {
+ queue->req_failed = false;
+ return NULL;
+ } else {
+ acb = qemu_aio_get(&block_queue_pool, bs,
+ cb, opaque);
+ request = container_of(acb, BlockQueueAIOCB, common);
+ request->handler = handler;
+ request->sector_num = sector_num;
+ request->qiov = qiov;
+ request->nb_sectors = nb_sectors;
+ request->real_acb = NULL;
+ QTAILQ_INSERT_TAIL(&queue->requests, request, entry);
+ }
+
+ return acb;
+}
+
+static int qemu_block_queue_handler(BlockQueueAIOCB *request)
+{
+ int ret;
+ BlockDriverAIOCB *res;
+
+ res = request->handler(request->common.bs, request->sector_num,
+ request->qiov, request->nb_sectors,
+ qemu_block_queue_callback, request);
+ if (res) {
+ request->real_acb = res;
+ }
+
+ ret = (res == NULL) ? 0 : 1;
+
+ return ret;
+}
+
+void qemu_block_queue_flush(BlockQueue *queue)
+{
+ queue->flushing = true;
+ while (!QTAILQ_EMPTY(&queue->requests)) {
+ BlockQueueAIOCB *request = NULL;
+ int ret = 0;
+
+ request = QTAILQ_FIRST(&queue->requests);
+ QTAILQ_REMOVE(&queue->requests, request, entry);
+
+ queue->req_failed = true;
+ ret = qemu_block_queue_handler(request);
+ if (ret == 0) {
+ QTAILQ_INSERT_HEAD(&queue->requests, request, entry);
+ if (queue->req_failed) {
+ qemu_block_queue_callback(request, -EIO);
+ break;
+ }
+ }
+ }
+
+ queue->req_failed = true;
+ queue->flushing = false;
+}
+
+bool qemu_block_queue_has_pending(BlockQueue *queue)
+{
+ return !queue->flushing && !QTAILQ_EMPTY(&queue->requests);
+}
diff --git a/block/blk-queue.h b/block/blk-queue.h
new file mode 100644
index 0000000..c1529f7
--- /dev/null
+++ b/block/blk-queue.h
@@ -0,0 +1,59 @@
+/*
+ * QEMU System Emulator queue declaration for block layer
+ *
+ * Copyright (c) IBM, Corp. 2011
+ *
+ * Authors:
+ * Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
+ * Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#ifndef QEMU_BLOCK_QUEUE_H
+#define QEMU_BLOCK_QUEUE_H
+
+#include "block.h"
+#include "qemu-queue.h"
+
+typedef BlockDriverAIOCB* (BlockRequestHandler) (BlockDriverState *bs,
+ int64_t sector_num, QEMUIOVector *qiov,
+ int nb_sectors, BlockDriverCompletionFunc *cb,
+ void *opaque);
+
+typedef struct BlockQueue BlockQueue;
+
+BlockQueue *qemu_new_block_queue(void);
+
+void qemu_del_block_queue(BlockQueue *queue);
+
+BlockDriverAIOCB *qemu_block_queue_enqueue(BlockQueue *queue,
+ BlockDriverState *bs,
+ BlockRequestHandler *handler,
+ int64_t sector_num,
+ QEMUIOVector *qiov,
+ int nb_sectors,
+ BlockDriverCompletionFunc *cb,
+ void *opaque);
+
+void qemu_block_queue_flush(BlockQueue *queue);
+
+bool qemu_block_queue_has_pending(BlockQueue *queue);
+
+#endif /* QEMU_BLOCK_QUEUE_H */
diff --git a/block_int.h b/block_int.h
index 8a72b80..201e635 100644
--- a/block_int.h
+++ b/block_int.h
@@ -29,10 +29,18 @@
#include "qemu-queue.h"
#include "qemu-coroutine.h"
#include "qemu-timer.h"
+#include "block/blk-queue.h"
#define BLOCK_FLAG_ENCRYPT 1
#define BLOCK_FLAG_COMPAT6 4
+#define BLOCK_IO_LIMIT_READ 0
+#define BLOCK_IO_LIMIT_WRITE 1
+#define BLOCK_IO_LIMIT_TOTAL 2
+
+#define BLOCK_IO_SLICE_TIME 100000000
+#define NANOSECONDS_PER_SECOND 1000000000.0
+
#define BLOCK_OPT_SIZE "size"
#define BLOCK_OPT_ENCRYPT "encryption"
#define BLOCK_OPT_COMPAT6 "compat6"
@@ -49,6 +57,16 @@ typedef struct AIOPool {
BlockDriverAIOCB *free_aiocb;
} AIOPool;
+typedef struct BlockIOLimit {
+ uint64_t bps[3];
+ uint64_t iops[3];
+} BlockIOLimit;
+
+typedef struct BlockIODisp {
+ uint64_t bytes[2];
+ uint64_t ios[2];
+} BlockIODisp;
+
struct BlockDriver {
const char *format_name;
int instance_size;
@@ -184,6 +202,15 @@ struct BlockDriverState {
void *sync_aiocb;
+ /* the time for latest disk I/O */
+ int64_t slice_start;
+ int64_t slice_end;
+ BlockIOLimit io_limits;
+ BlockIODisp io_disps;
+ BlockQueue *block_queue;
+ QEMUTimer *block_timer;
+ bool io_limits_enabled;
+
/* I/O stats (display with "info blockstats"). */
uint64_t nr_bytes[BDRV_MAX_IOTYPE];
uint64_t nr_ops[BDRV_MAX_IOTYPE];
--
1.7.6
^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [Qemu-devel] [PATCH v8 1/4] block: add the block queue support
2011-09-08 10:11 ` [Qemu-devel] [PATCH v8 1/4] block: add the block queue support Zhi Yong Wu
@ 2011-09-23 15:32 ` Kevin Wolf
2011-09-26 8:01 ` Zhi Yong Wu
0 siblings, 1 reply; 13+ messages in thread
From: Kevin Wolf @ 2011-09-23 15:32 UTC (permalink / raw)
To: Zhi Yong Wu
Cc: aliguori, stefanha, kvm, mtosatti, qemu-devel, pair, zwu.kernel,
ryanh
Am 08.09.2011 12:11, schrieb Zhi Yong Wu:
> Signed-off-by: Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
> ---
> Makefile.objs | 2 +-
> block/blk-queue.c | 201 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> block/blk-queue.h | 59 ++++++++++++++++
> block_int.h | 27 +++++++
> 4 files changed, 288 insertions(+), 1 deletions(-)
> create mode 100644 block/blk-queue.c
> create mode 100644 block/blk-queue.h
>
> diff --git a/Makefile.objs b/Makefile.objs
> index 26b885b..5dcf456 100644
> --- a/Makefile.objs
> +++ b/Makefile.objs
> @@ -33,7 +33,7 @@ block-nested-y += raw.o cow.o qcow.o vdi.o vmdk.o cloop.o dmg.o bochs.o vpc.o vv
> block-nested-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o
> block-nested-y += qed.o qed-gencb.o qed-l2-cache.o qed-table.o qed-cluster.o
> block-nested-y += qed-check.o
> -block-nested-y += parallels.o nbd.o blkdebug.o sheepdog.o blkverify.o
> +block-nested-y += parallels.o nbd.o blkdebug.o sheepdog.o blkverify.o blk-queue.o
> block-nested-$(CONFIG_WIN32) += raw-win32.o
> block-nested-$(CONFIG_POSIX) += raw-posix.o
> block-nested-$(CONFIG_CURL) += curl.o
> diff --git a/block/blk-queue.c b/block/blk-queue.c
> new file mode 100644
> index 0000000..adef497
> --- /dev/null
> +++ b/block/blk-queue.c
> @@ -0,0 +1,201 @@
> +/*
> + * QEMU System Emulator queue definition for block layer
> + *
> + * Copyright (c) IBM, Corp. 2011
> + *
> + * Authors:
> + * Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
> + * Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a copy
> + * of this software and associated documentation files (the "Software"), to deal
> + * in the Software without restriction, including without limitation the rights
> + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
> + * copies of the Software, and to permit persons to whom the Software is
> + * furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included in
> + * all copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
> + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
> + * THE SOFTWARE.
> + */
> +
> +#include "block_int.h"
> +#include "block/blk-queue.h"
> +#include "qemu-common.h"
> +
> +/* The APIs for block request queue on qemu block layer.
> + */
> +
> +struct BlockQueueAIOCB {
> + BlockDriverAIOCB common;
> + QTAILQ_ENTRY(BlockQueueAIOCB) entry;
> + BlockRequestHandler *handler;
> + BlockDriverAIOCB *real_acb;
> +
> + int64_t sector_num;
> + QEMUIOVector *qiov;
> + int nb_sectors;
> +};
The idea is that each request is first queued on the QTAILQ, and at some
point it's removed from the queue and gets a real_acb. But it never has
both at the same time. Correct?
Can we have the basic principle of operation spelled out as a comment
somewhere near the top of the file?
> +
> +typedef struct BlockQueueAIOCB BlockQueueAIOCB;
> +
> +struct BlockQueue {
> + QTAILQ_HEAD(requests, BlockQueueAIOCB) requests;
> + bool req_failed;
> + bool flushing;
> +};
I find req_failed pretty confusing. Needs documentation at least, but
most probably also a better name.
> +
> +static void qemu_block_queue_dequeue(BlockQueue *queue,
> + BlockQueueAIOCB *request)
> +{
> + BlockQueueAIOCB *req;
> +
> + assert(queue);
> + while (!QTAILQ_EMPTY(&queue->requests)) {
> + req = QTAILQ_FIRST(&queue->requests);
> + if (req == request) {
> + QTAILQ_REMOVE(&queue->requests, req, entry);
> + break;
> + }
> + }
> +}
Is it just me or is this an endless loop if the request isn't the first
element in the list?
> +
> +static void qemu_block_queue_cancel(BlockDriverAIOCB *acb)
> +{
> + BlockQueueAIOCB *request = container_of(acb, BlockQueueAIOCB, common);
> + if (request->real_acb) {
> + bdrv_aio_cancel(request->real_acb);
> + } else {
> + assert(request->common.bs->block_queue);
> + qemu_block_queue_dequeue(request->common.bs->block_queue,
> + request);
> + }
> +
> + qemu_aio_release(request);
> +}
> +
> +static AIOPool block_queue_pool = {
> + .aiocb_size = sizeof(struct BlockQueueAIOCB),
> + .cancel = qemu_block_queue_cancel,
> +};
> +
> +static void qemu_block_queue_callback(void *opaque, int ret)
> +{
> + BlockQueueAIOCB *acb = opaque;
> +
> + if (acb->common.cb) {
> + acb->common.cb(acb->common.opaque, ret);
> + }
> +
> + qemu_aio_release(acb);
> +}
> +
> +BlockQueue *qemu_new_block_queue(void)
> +{
> + BlockQueue *queue;
> +
> + queue = g_malloc0(sizeof(BlockQueue));
> +
> + QTAILQ_INIT(&queue->requests);
> +
> + queue->req_failed = true;
> + queue->flushing = false;
> +
> + return queue;
> +}
> +
> +void qemu_del_block_queue(BlockQueue *queue)
> +{
> + BlockQueueAIOCB *request, *next;
> +
> + QTAILQ_FOREACH_SAFE(request, &queue->requests, entry, next) {
> + QTAILQ_REMOVE(&queue->requests, request, entry);
> + qemu_aio_release(request);
> + }
> +
> + g_free(queue);
> +}
Can we be sure that no AIO requests are in flight that still use the now
released AIOCB?
> +
> +BlockDriverAIOCB *qemu_block_queue_enqueue(BlockQueue *queue,
> + BlockDriverState *bs,
> + BlockRequestHandler *handler,
> + int64_t sector_num,
> + QEMUIOVector *qiov,
> + int nb_sectors,
> + BlockDriverCompletionFunc *cb,
> + void *opaque)
> +{
> + BlockDriverAIOCB *acb;
> + BlockQueueAIOCB *request;
> +
> + if (queue->flushing) {
> + queue->req_failed = false;
> + return NULL;
> + } else {
> + acb = qemu_aio_get(&block_queue_pool, bs,
> + cb, opaque);
> + request = container_of(acb, BlockQueueAIOCB, common);
> + request->handler = handler;
> + request->sector_num = sector_num;
> + request->qiov = qiov;
> + request->nb_sectors = nb_sectors;
> + request->real_acb = NULL;
> + QTAILQ_INSERT_TAIL(&queue->requests, request, entry);
> + }
> +
> + return acb;
> +}
> +
> +static int qemu_block_queue_handler(BlockQueueAIOCB *request)
> +{
> + int ret;
> + BlockDriverAIOCB *res;
> +
> + res = request->handler(request->common.bs, request->sector_num,
> + request->qiov, request->nb_sectors,
> + qemu_block_queue_callback, request);
> + if (res) {
> + request->real_acb = res;
> + }
> +
> + ret = (res == NULL) ? 0 : 1;
> +
> + return ret;
You mean return (res != NULL); and want to have bool as the return value
of this function.
> +}
> +
> +void qemu_block_queue_flush(BlockQueue *queue)
> +{
> + queue->flushing = true;
> + while (!QTAILQ_EMPTY(&queue->requests)) {
> + BlockQueueAIOCB *request = NULL;
> + int ret = 0;
> +
> + request = QTAILQ_FIRST(&queue->requests);
> + QTAILQ_REMOVE(&queue->requests, request, entry);
> +
> + queue->req_failed = true;
> + ret = qemu_block_queue_handler(request);
> + if (ret == 0) {
> + QTAILQ_INSERT_HEAD(&queue->requests, request, entry);
> + if (queue->req_failed) {
> + qemu_block_queue_callback(request, -EIO);
> + break;
> + }
> + }
> + }
> +
> + queue->req_failed = true;
> + queue->flushing = false;
> +}
> +
> +bool qemu_block_queue_has_pending(BlockQueue *queue)
> +{
> + return !queue->flushing && !QTAILQ_EMPTY(&queue->requests);
> +}
Why doesn't the queue have pending requests in the middle of a flush
operation? (That is, the flush hasn't completed yet)
> diff --git a/block/blk-queue.h b/block/blk-queue.h
> new file mode 100644
> index 0000000..c1529f7
> --- /dev/null
> +++ b/block/blk-queue.h
> @@ -0,0 +1,59 @@
> +/*
> + * QEMU System Emulator queue declaration for block layer
> + *
> + * Copyright (c) IBM, Corp. 2011
> + *
> + * Authors:
> + * Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
> + * Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a copy
> + * of this software and associated documentation files (the "Software"), to deal
> + * in the Software without restriction, including without limitation the rights
> + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
> + * copies of the Software, and to permit persons to whom the Software is
> + * furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included in
> + * all copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
> + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
> + * THE SOFTWARE.
> + */
> +
> +#ifndef QEMU_BLOCK_QUEUE_H
> +#define QEMU_BLOCK_QUEUE_H
> +
> +#include "block.h"
> +#include "qemu-queue.h"
> +
> +typedef BlockDriverAIOCB* (BlockRequestHandler) (BlockDriverState *bs,
> + int64_t sector_num, QEMUIOVector *qiov,
> + int nb_sectors, BlockDriverCompletionFunc *cb,
> + void *opaque);
> +
> +typedef struct BlockQueue BlockQueue;
> +
> +BlockQueue *qemu_new_block_queue(void);
> +
> +void qemu_del_block_queue(BlockQueue *queue);
> +
> +BlockDriverAIOCB *qemu_block_queue_enqueue(BlockQueue *queue,
> + BlockDriverState *bs,
> + BlockRequestHandler *handler,
> + int64_t sector_num,
> + QEMUIOVector *qiov,
> + int nb_sectors,
> + BlockDriverCompletionFunc *cb,
> + void *opaque);
> +
> +void qemu_block_queue_flush(BlockQueue *queue);
> +
> +bool qemu_block_queue_has_pending(BlockQueue *queue);
> +
> +#endif /* QEMU_BLOCK_QUEUE_H */
> diff --git a/block_int.h b/block_int.h
> index 8a72b80..201e635 100644
> --- a/block_int.h
> +++ b/block_int.h
> @@ -29,10 +29,18 @@
> #include "qemu-queue.h"
> #include "qemu-coroutine.h"
> #include "qemu-timer.h"
> +#include "block/blk-queue.h"
>
> #define BLOCK_FLAG_ENCRYPT 1
> #define BLOCK_FLAG_COMPAT6 4
>
> +#define BLOCK_IO_LIMIT_READ 0
> +#define BLOCK_IO_LIMIT_WRITE 1
> +#define BLOCK_IO_LIMIT_TOTAL 2
> +
> +#define BLOCK_IO_SLICE_TIME 100000000
> +#define NANOSECONDS_PER_SECOND 1000000000.0
> +
> #define BLOCK_OPT_SIZE "size"
> #define BLOCK_OPT_ENCRYPT "encryption"
> #define BLOCK_OPT_COMPAT6 "compat6"
> @@ -49,6 +57,16 @@ typedef struct AIOPool {
> BlockDriverAIOCB *free_aiocb;
> } AIOPool;
>
> +typedef struct BlockIOLimit {
> + uint64_t bps[3];
> + uint64_t iops[3];
> +} BlockIOLimit;
> +
> +typedef struct BlockIODisp {
> + uint64_t bytes[2];
> + uint64_t ios[2];
> +} BlockIODisp;
> +
> struct BlockDriver {
> const char *format_name;
> int instance_size;
> @@ -184,6 +202,15 @@ struct BlockDriverState {
>
> void *sync_aiocb;
>
> + /* the time for latest disk I/O */
> + int64_t slice_start;
> + int64_t slice_end;
> + BlockIOLimit io_limits;
> + BlockIODisp io_disps;
> + BlockQueue *block_queue;
> + QEMUTimer *block_timer;
> + bool io_limits_enabled;
> +
> /* I/O stats (display with "info blockstats"). */
> uint64_t nr_bytes[BDRV_MAX_IOTYPE];
> uint64_t nr_ops[BDRV_MAX_IOTYPE];
The changes to block_int.h look unrelated to this patch. Maybe they
should come later in the series.
Kevin
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [Qemu-devel] [PATCH v8 1/4] block: add the block queue support
2011-09-23 15:32 ` Kevin Wolf
@ 2011-09-26 8:01 ` Zhi Yong Wu
2011-10-17 10:17 ` Kevin Wolf
0 siblings, 1 reply; 13+ messages in thread
From: Zhi Yong Wu @ 2011-09-26 8:01 UTC (permalink / raw)
To: Kevin Wolf
Cc: aliguori, stefanha, kvm, mtosatti, qemu-devel, pair, ryanh,
Zhi Yong Wu
On Fri, Sep 23, 2011 at 11:32 PM, Kevin Wolf <kwolf@redhat.com> wrote:
> Am 08.09.2011 12:11, schrieb Zhi Yong Wu:
>> Signed-off-by: Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
>> ---
>> Makefile.objs | 2 +-
>> block/blk-queue.c | 201 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>> block/blk-queue.h | 59 ++++++++++++++++
>> block_int.h | 27 +++++++
>> 4 files changed, 288 insertions(+), 1 deletions(-)
>> create mode 100644 block/blk-queue.c
>> create mode 100644 block/blk-queue.h
>>
>> diff --git a/Makefile.objs b/Makefile.objs
>> index 26b885b..5dcf456 100644
>> --- a/Makefile.objs
>> +++ b/Makefile.objs
>> @@ -33,7 +33,7 @@ block-nested-y += raw.o cow.o qcow.o vdi.o vmdk.o cloop.o dmg.o bochs.o vpc.o vv
>> block-nested-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o
>> block-nested-y += qed.o qed-gencb.o qed-l2-cache.o qed-table.o qed-cluster.o
>> block-nested-y += qed-check.o
>> -block-nested-y += parallels.o nbd.o blkdebug.o sheepdog.o blkverify.o
>> +block-nested-y += parallels.o nbd.o blkdebug.o sheepdog.o blkverify.o blk-queue.o
>> block-nested-$(CONFIG_WIN32) += raw-win32.o
>> block-nested-$(CONFIG_POSIX) += raw-posix.o
>> block-nested-$(CONFIG_CURL) += curl.o
>> diff --git a/block/blk-queue.c b/block/blk-queue.c
>> new file mode 100644
>> index 0000000..adef497
>> --- /dev/null
>> +++ b/block/blk-queue.c
>> @@ -0,0 +1,201 @@
>> +/*
>> + * QEMU System Emulator queue definition for block layer
>> + *
>> + * Copyright (c) IBM, Corp. 2011
>> + *
>> + * Authors:
>> + * Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
>> + * Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
>> + *
>> + * Permission is hereby granted, free of charge, to any person obtaining a copy
>> + * of this software and associated documentation files (the "Software"), to deal
>> + * in the Software without restriction, including without limitation the rights
>> + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
>> + * copies of the Software, and to permit persons to whom the Software is
>> + * furnished to do so, subject to the following conditions:
>> + *
>> + * The above copyright notice and this permission notice shall be included in
>> + * all copies or substantial portions of the Software.
>> + *
>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
>> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
>> + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
>> + * THE SOFTWARE.
>> + */
>> +
>> +#include "block_int.h"
>> +#include "block/blk-queue.h"
>> +#include "qemu-common.h"
>> +
>> +/* The APIs for block request queue on qemu block layer.
>> + */
>> +
>> +struct BlockQueueAIOCB {
>> + BlockDriverAIOCB common;
>> + QTAILQ_ENTRY(BlockQueueAIOCB) entry;
>> + BlockRequestHandler *handler;
>> + BlockDriverAIOCB *real_acb;
>> +
>> + int64_t sector_num;
>> + QEMUIOVector *qiov;
>> + int nb_sectors;
>> +};
>
> The idea is that each request is first queued on the QTAILQ, and at some
> point it's removed from the queue and gets a real_acb. But it never has
> both at the same time. Correct?
NO. if block I/O throttling is enabled and I/O rate at runtime exceed
this limits, this request will be enqueued.
It represents the whole lifecycle of one enqueued request.
>
> Can we have the basic principle of operation spelled out as a comment
> somewhere near the top of the file?
OK, i will.
>
>> +
>> +typedef struct BlockQueueAIOCB BlockQueueAIOCB;
>> +
>> +struct BlockQueue {
>> + QTAILQ_HEAD(requests, BlockQueueAIOCB) requests;
>> + bool req_failed;
>> + bool flushing;
>> +};
>
> I find req_failed pretty confusing. Needs documentation at least, but
> most probably also a better name.
OK. request_has_failed?
>
>> +
>> +static void qemu_block_queue_dequeue(BlockQueue *queue,
>> + BlockQueueAIOCB *request)
>> +{
>> + BlockQueueAIOCB *req;
>> +
>> + assert(queue);
>> + while (!QTAILQ_EMPTY(&queue->requests)) {
>> + req = QTAILQ_FIRST(&queue->requests);
>> + if (req == request) {
>> + QTAILQ_REMOVE(&queue->requests, req, entry);
>> + break;
>> + }
>> + }
>> +}
>
> Is it just me or is this an endless loop if the request isn't the first
> element in the list?
queue->requests is only used to store requests which exceed the limits.
Why is the request not the first evlement?
>
>> +
>> +static void qemu_block_queue_cancel(BlockDriverAIOCB *acb)
>> +{
>> + BlockQueueAIOCB *request = container_of(acb, BlockQueueAIOCB, common);
>> + if (request->real_acb) {
>> + bdrv_aio_cancel(request->real_acb);
>> + } else {
>> + assert(request->common.bs->block_queue);
>> + qemu_block_queue_dequeue(request->common.bs->block_queue,
>> + request);
>> + }
>> +
>> + qemu_aio_release(request);
>> +}
>> +
>> +static AIOPool block_queue_pool = {
>> + .aiocb_size = sizeof(struct BlockQueueAIOCB),
>> + .cancel = qemu_block_queue_cancel,
>> +};
>> +
>> +static void qemu_block_queue_callback(void *opaque, int ret)
>> +{
>> + BlockQueueAIOCB *acb = opaque;
>> +
>> + if (acb->common.cb) {
>> + acb->common.cb(acb->common.opaque, ret);
>> + }
>> +
>> + qemu_aio_release(acb);
>> +}
>> +
>> +BlockQueue *qemu_new_block_queue(void)
>> +{
>> + BlockQueue *queue;
>> +
>> + queue = g_malloc0(sizeof(BlockQueue));
>> +
>> + QTAILQ_INIT(&queue->requests);
>> +
>> + queue->req_failed = true;
>> + queue->flushing = false;
>> +
>> + return queue;
>> +}
>> +
>> +void qemu_del_block_queue(BlockQueue *queue)
>> +{
>> + BlockQueueAIOCB *request, *next;
>> +
>> + QTAILQ_FOREACH_SAFE(request, &queue->requests, entry, next) {
>> + QTAILQ_REMOVE(&queue->requests, request, entry);
>> + qemu_aio_release(request);
>> + }
>> +
>> + g_free(queue);
>> +}
>
> Can we be sure that no AIO requests are in flight that still use the now
> released AIOCB?
Yeah, since qemu core code is serially performed, i think that when
qemu_del_block_queue is performed, no requests are in flight. Right?
>
>> +
>> +BlockDriverAIOCB *qemu_block_queue_enqueue(BlockQueue *queue,
>> + BlockDriverState *bs,
>> + BlockRequestHandler *handler,
>> + int64_t sector_num,
>> + QEMUIOVector *qiov,
>> + int nb_sectors,
>> + BlockDriverCompletionFunc *cb,
>> + void *opaque)
>> +{
>> + BlockDriverAIOCB *acb;
>> + BlockQueueAIOCB *request;
>> +
>> + if (queue->flushing) {
>> + queue->req_failed = false;
>> + return NULL;
>> + } else {
>> + acb = qemu_aio_get(&block_queue_pool, bs,
>> + cb, opaque);
>> + request = container_of(acb, BlockQueueAIOCB, common);
>> + request->handler = handler;
>> + request->sector_num = sector_num;
>> + request->qiov = qiov;
>> + request->nb_sectors = nb_sectors;
>> + request->real_acb = NULL;
>> + QTAILQ_INSERT_TAIL(&queue->requests, request, entry);
>> + }
>> +
>> + return acb;
>> +}
>> +
>> +static int qemu_block_queue_handler(BlockQueueAIOCB *request)
>> +{
>> + int ret;
>> + BlockDriverAIOCB *res;
>> +
>> + res = request->handler(request->common.bs, request->sector_num,
>> + request->qiov, request->nb_sectors,
>> + qemu_block_queue_callback, request);
>> + if (res) {
>> + request->real_acb = res;
>> + }
>> +
>> + ret = (res == NULL) ? 0 : 1;
>> +
>> + return ret;
>
> You mean return (res != NULL); and want to have bool as the return value
> of this function.
Yeah, thanks. i will modify as below:
ret = (res == NULL) ? false : true;
and
static bool qemu_block_queue_handler()
>
>> +}
>> +
>> +void qemu_block_queue_flush(BlockQueue *queue)
>> +{
>> + queue->flushing = true;
>> + while (!QTAILQ_EMPTY(&queue->requests)) {
>> + BlockQueueAIOCB *request = NULL;
>> + int ret = 0;
>> +
>> + request = QTAILQ_FIRST(&queue->requests);
>> + QTAILQ_REMOVE(&queue->requests, request, entry);
>> +
>> + queue->req_failed = true;
>> + ret = qemu_block_queue_handler(request);
>> + if (ret == 0) {
>> + QTAILQ_INSERT_HEAD(&queue->requests, request, entry);
>> + if (queue->req_failed) {
>> + qemu_block_queue_callback(request, -EIO);
>> + break;
>> + }
>> + }
>> + }
>> +
>> + queue->req_failed = true;
>> + queue->flushing = false;
>> +}
>> +
>> +bool qemu_block_queue_has_pending(BlockQueue *queue)
>> +{
>> + return !queue->flushing && !QTAILQ_EMPTY(&queue->requests);
>> +}
>
> Why doesn't the queue have pending requests in the middle of a flush
> operation? (That is, the flush hasn't completed yet)
It is possible for the queue to have pending requests. if yes, how about?
>
>> diff --git a/block/blk-queue.h b/block/blk-queue.h
>> new file mode 100644
>> index 0000000..c1529f7
>> --- /dev/null
>> +++ b/block/blk-queue.h
>> @@ -0,0 +1,59 @@
>> +/*
>> + * QEMU System Emulator queue declaration for block layer
>> + *
>> + * Copyright (c) IBM, Corp. 2011
>> + *
>> + * Authors:
>> + * Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
>> + * Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
>> + *
>> + * Permission is hereby granted, free of charge, to any person obtaining a copy
>> + * of this software and associated documentation files (the "Software"), to deal
>> + * in the Software without restriction, including without limitation the rights
>> + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
>> + * copies of the Software, and to permit persons to whom the Software is
>> + * furnished to do so, subject to the following conditions:
>> + *
>> + * The above copyright notice and this permission notice shall be included in
>> + * all copies or substantial portions of the Software.
>> + *
>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
>> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
>> + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
>> + * THE SOFTWARE.
>> + */
>> +
>> +#ifndef QEMU_BLOCK_QUEUE_H
>> +#define QEMU_BLOCK_QUEUE_H
>> +
>> +#include "block.h"
>> +#include "qemu-queue.h"
>> +
>> +typedef BlockDriverAIOCB* (BlockRequestHandler) (BlockDriverState *bs,
>> + int64_t sector_num, QEMUIOVector *qiov,
>> + int nb_sectors, BlockDriverCompletionFunc *cb,
>> + void *opaque);
>> +
>> +typedef struct BlockQueue BlockQueue;
>> +
>> +BlockQueue *qemu_new_block_queue(void);
>> +
>> +void qemu_del_block_queue(BlockQueue *queue);
>> +
>> +BlockDriverAIOCB *qemu_block_queue_enqueue(BlockQueue *queue,
>> + BlockDriverState *bs,
>> + BlockRequestHandler *handler,
>> + int64_t sector_num,
>> + QEMUIOVector *qiov,
>> + int nb_sectors,
>> + BlockDriverCompletionFunc *cb,
>> + void *opaque);
>> +
>> +void qemu_block_queue_flush(BlockQueue *queue);
>> +
>> +bool qemu_block_queue_has_pending(BlockQueue *queue);
>> +
>> +#endif /* QEMU_BLOCK_QUEUE_H */
>> diff --git a/block_int.h b/block_int.h
>> index 8a72b80..201e635 100644
>> --- a/block_int.h
>> +++ b/block_int.h
>> @@ -29,10 +29,18 @@
>> #include "qemu-queue.h"
>> #include "qemu-coroutine.h"
>> #include "qemu-timer.h"
>> +#include "block/blk-queue.h"
>>
>> #define BLOCK_FLAG_ENCRYPT 1
>> #define BLOCK_FLAG_COMPAT6 4
>>
>> +#define BLOCK_IO_LIMIT_READ 0
>> +#define BLOCK_IO_LIMIT_WRITE 1
>> +#define BLOCK_IO_LIMIT_TOTAL 2
>> +
>> +#define BLOCK_IO_SLICE_TIME 100000000
>> +#define NANOSECONDS_PER_SECOND 1000000000.0
>> +
>> #define BLOCK_OPT_SIZE "size"
>> #define BLOCK_OPT_ENCRYPT "encryption"
>> #define BLOCK_OPT_COMPAT6 "compat6"
>> @@ -49,6 +57,16 @@ typedef struct AIOPool {
>> BlockDriverAIOCB *free_aiocb;
>> } AIOPool;
>>
>> +typedef struct BlockIOLimit {
>> + uint64_t bps[3];
>> + uint64_t iops[3];
>> +} BlockIOLimit;
>> +
>> +typedef struct BlockIODisp {
>> + uint64_t bytes[2];
>> + uint64_t ios[2];
>> +} BlockIODisp;
>> +
>> struct BlockDriver {
>> const char *format_name;
>> int instance_size;
>> @@ -184,6 +202,15 @@ struct BlockDriverState {
>>
>> void *sync_aiocb;
>>
>> + /* the time for latest disk I/O */
>> + int64_t slice_start;
>> + int64_t slice_end;
>> + BlockIOLimit io_limits;
>> + BlockIODisp io_disps;
>> + BlockQueue *block_queue;
>> + QEMUTimer *block_timer;
>> + bool io_limits_enabled;
>> +
>> /* I/O stats (display with "info blockstats"). */
>> uint64_t nr_bytes[BDRV_MAX_IOTYPE];
>> uint64_t nr_ops[BDRV_MAX_IOTYPE];
>
> The changes to block_int.h look unrelated to this patch. Maybe they
> should come later in the series.
OK, i move them to related series. thanks.
>
> Kevin
>
>
--
Regards,
Zhi Yong Wu
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [Qemu-devel] [PATCH v8 1/4] block: add the block queue support
2011-09-26 8:01 ` Zhi Yong Wu
@ 2011-10-17 10:17 ` Kevin Wolf
[not found] ` <4E9C00D2.1040004@redhat.com>
2011-10-18 8:07 ` Zhi Yong Wu
0 siblings, 2 replies; 13+ messages in thread
From: Kevin Wolf @ 2011-10-17 10:17 UTC (permalink / raw)
To: Zhi Yong Wu
Cc: pair, stefanha, kvm, mtosatti, Zhi Yong Wu, aliguori, qemu-devel,
ryanh
Am 26.09.2011 10:01, schrieb Zhi Yong Wu:
> On Fri, Sep 23, 2011 at 11:32 PM, Kevin Wolf <kwolf@redhat.com> wrote:
>> Am 08.09.2011 12:11, schrieb Zhi Yong Wu:
>>> Signed-off-by: Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
>>> ---
>>> Makefile.objs | 2 +-
>>> block/blk-queue.c | 201 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>>> block/blk-queue.h | 59 ++++++++++++++++
>>> block_int.h | 27 +++++++
>>> 4 files changed, 288 insertions(+), 1 deletions(-)
>>> create mode 100644 block/blk-queue.c
>>> create mode 100644 block/blk-queue.h
>>>
>>> diff --git a/Makefile.objs b/Makefile.objs
>>> index 26b885b..5dcf456 100644
>>> --- a/Makefile.objs
>>> +++ b/Makefile.objs
>>> @@ -33,7 +33,7 @@ block-nested-y += raw.o cow.o qcow.o vdi.o vmdk.o cloop.o dmg.o bochs.o vpc.o vv
>>> block-nested-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o
>>> block-nested-y += qed.o qed-gencb.o qed-l2-cache.o qed-table.o qed-cluster.o
>>> block-nested-y += qed-check.o
>>> -block-nested-y += parallels.o nbd.o blkdebug.o sheepdog.o blkverify.o
>>> +block-nested-y += parallels.o nbd.o blkdebug.o sheepdog.o blkverify.o blk-queue.o
>>> block-nested-$(CONFIG_WIN32) += raw-win32.o
>>> block-nested-$(CONFIG_POSIX) += raw-posix.o
>>> block-nested-$(CONFIG_CURL) += curl.o
>>> diff --git a/block/blk-queue.c b/block/blk-queue.c
>>> new file mode 100644
>>> index 0000000..adef497
>>> --- /dev/null
>>> +++ b/block/blk-queue.c
>>> @@ -0,0 +1,201 @@
>>> +/*
>>> + * QEMU System Emulator queue definition for block layer
>>> + *
>>> + * Copyright (c) IBM, Corp. 2011
>>> + *
>>> + * Authors:
>>> + * Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
>>> + * Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
>>> + *
>>> + * Permission is hereby granted, free of charge, to any person obtaining a copy
>>> + * of this software and associated documentation files (the "Software"), to deal
>>> + * in the Software without restriction, including without limitation the rights
>>> + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
>>> + * copies of the Software, and to permit persons to whom the Software is
>>> + * furnished to do so, subject to the following conditions:
>>> + *
>>> + * The above copyright notice and this permission notice shall be included in
>>> + * all copies or substantial portions of the Software.
>>> + *
>>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
>>> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
>>> + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
>>> + * THE SOFTWARE.
>>> + */
>>> +
>>> +#include "block_int.h"
>>> +#include "block/blk-queue.h"
>>> +#include "qemu-common.h"
>>> +
>>> +/* The APIs for block request queue on qemu block layer.
>>> + */
>>> +
>>> +struct BlockQueueAIOCB {
>>> + BlockDriverAIOCB common;
>>> + QTAILQ_ENTRY(BlockQueueAIOCB) entry;
>>> + BlockRequestHandler *handler;
>>> + BlockDriverAIOCB *real_acb;
>>> +
>>> + int64_t sector_num;
>>> + QEMUIOVector *qiov;
>>> + int nb_sectors;
>>> +};
>>
>> The idea is that each request is first queued on the QTAILQ, and at some
>> point it's removed from the queue and gets a real_acb. But it never has
>> both at the same time. Correct?
> NO. if block I/O throttling is enabled and I/O rate at runtime exceed
> this limits, this request will be enqueued.
> It represents the whole lifecycle of one enqueued request.
What are the conditions under which the request will still be enqueued,
but has a real_acb at the same time?
>>> +
>>> +typedef struct BlockQueueAIOCB BlockQueueAIOCB;
>>> +
>>> +struct BlockQueue {
>>> + QTAILQ_HEAD(requests, BlockQueueAIOCB) requests;
>>> + bool req_failed;
>>> + bool flushing;
>>> +};
>>
>> I find req_failed pretty confusing. Needs documentation at least, but
>> most probably also a better name.
> OK. request_has_failed?
No, that doesn't describe what it's really doing.
You set req_failed = true by default and then on some obscure condition
clear it or not. It's tracking something, but I'm not sure what meaning
it has during the whole process.
>>> +
>>> +static void qemu_block_queue_dequeue(BlockQueue *queue,
>>> + BlockQueueAIOCB *request)
>>> +{
>>> + BlockQueueAIOCB *req;
>>> +
>>> + assert(queue);
>>> + while (!QTAILQ_EMPTY(&queue->requests)) {
>>> + req = QTAILQ_FIRST(&queue->requests);
>>> + if (req == request) {
>>> + QTAILQ_REMOVE(&queue->requests, req, entry);
>>> + break;
>>> + }
>>> + }
>>> +}
>>
>> Is it just me or is this an endless loop if the request isn't the first
>> element in the list?
> queue->requests is only used to store requests which exceed the limits.
> Why is the request not the first evlement?
Why do you have a loop if it's always the first element?
>>> +void qemu_del_block_queue(BlockQueue *queue)
>>> +{
>>> + BlockQueueAIOCB *request, *next;
>>> +
>>> + QTAILQ_FOREACH_SAFE(request, &queue->requests, entry, next) {
>>> + QTAILQ_REMOVE(&queue->requests, request, entry);
>>> + qemu_aio_release(request);
>>> + }
>>> +
>>> + g_free(queue);
>>> +}
>>
>> Can we be sure that no AIO requests are in flight that still use the now
>> released AIOCB?
> Yeah, since qemu core code is serially performed, i think that when
> qemu_del_block_queue is performed, no requests are in flight. Right?
Patch 2 has this code:
+void bdrv_io_limits_disable(BlockDriverState *bs)
+{
+ bs->io_limits_enabled = false;
+
+ if (bs->block_queue) {
+ qemu_block_queue_flush(bs->block_queue);
+ qemu_del_block_queue(bs->block_queue);
+ bs->block_queue = NULL;
+ }
Does this mean that you can't disable I/O limits while the VM is running?
>>> +
>>> +BlockDriverAIOCB *qemu_block_queue_enqueue(BlockQueue *queue,
>>> + BlockDriverState *bs,
>>> + BlockRequestHandler *handler,
>>> + int64_t sector_num,
>>> + QEMUIOVector *qiov,
>>> + int nb_sectors,
>>> + BlockDriverCompletionFunc *cb,
>>> + void *opaque)
>>> +{
>>> + BlockDriverAIOCB *acb;
>>> + BlockQueueAIOCB *request;
>>> +
>>> + if (queue->flushing) {
>>> + queue->req_failed = false;
>>> + return NULL;
>>> + } else {
>>> + acb = qemu_aio_get(&block_queue_pool, bs,
>>> + cb, opaque);
>>> + request = container_of(acb, BlockQueueAIOCB, common);
>>> + request->handler = handler;
>>> + request->sector_num = sector_num;
>>> + request->qiov = qiov;
>>> + request->nb_sectors = nb_sectors;
>>> + request->real_acb = NULL;
>>> + QTAILQ_INSERT_TAIL(&queue->requests, request, entry);
>>> + }
>>> +
>>> + return acb;
>>> +}
>>> +
>>> +static int qemu_block_queue_handler(BlockQueueAIOCB *request)
>>> +{
>>> + int ret;
>>> + BlockDriverAIOCB *res;
>>> +
>>> + res = request->handler(request->common.bs, request->sector_num,
>>> + request->qiov, request->nb_sectors,
>>> + qemu_block_queue_callback, request);
>>> + if (res) {
>>> + request->real_acb = res;
>>> + }
>>> +
>>> + ret = (res == NULL) ? 0 : 1;
>>> +
>>> + return ret;
>>
>> You mean return (res != NULL); and want to have bool as the return value
>> of this function.
> Yeah, thanks. i will modify as below:
> ret = (res == NULL) ? false : true;
ret = (res != NULL) is really more readable.
> and
> static bool qemu_block_queue_handler()
>
>>
>>> +}
>>> +
>>> +void qemu_block_queue_flush(BlockQueue *queue)
>>> +{
>>> + queue->flushing = true;
>>> + while (!QTAILQ_EMPTY(&queue->requests)) {
>>> + BlockQueueAIOCB *request = NULL;
>>> + int ret = 0;
>>> +
>>> + request = QTAILQ_FIRST(&queue->requests);
>>> + QTAILQ_REMOVE(&queue->requests, request, entry);
>>> +
>>> + queue->req_failed = true;
>>> + ret = qemu_block_queue_handler(request);
>>> + if (ret == 0) {
>>> + QTAILQ_INSERT_HEAD(&queue->requests, request, entry);
>>> + if (queue->req_failed) {
>>> + qemu_block_queue_callback(request, -EIO);
>>> + break;
>>> + }
>>> + }
>>> + }
>>> +
>>> + queue->req_failed = true;
>>> + queue->flushing = false;
>>> +}
>>> +
>>> +bool qemu_block_queue_has_pending(BlockQueue *queue)
>>> +{
>>> + return !queue->flushing && !QTAILQ_EMPTY(&queue->requests);
>>> +}
>>
>> Why doesn't the queue have pending requests in the middle of a flush
>> operation? (That is, the flush hasn't completed yet)
> It is possible for the queue to have pending requests. if yes, how about?
Sorry, can't parse this.
I don't understand why the !queue->flushing part is correct.
Kevin
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [Qemu-devel] [PATCH v8 1/4] block: add the block queue support
[not found] ` <4E9C00D2.1040004@redhat.com>
@ 2011-10-18 7:00 ` Zhi Yong Wu
0 siblings, 0 replies; 13+ messages in thread
From: Zhi Yong Wu @ 2011-10-18 7:00 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: qemu-devel, kvm
On Mon, Oct 17, 2011 at 6:17 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> On 10/17/2011 12:17 PM, Kevin Wolf wrote:
>>
>> > > > +
>> > > > +static int qemu_block_queue_handler(BlockQueueAIOCB *request)
>> > > > +{
>> > > > + int ret;
>> > > > + BlockDriverAIOCB *res;
>> > > > +
>> > > > + res = request->handler(request->common.bs,
>> > > > request->sector_num,
>> > > > + request->qiov, request->nb_sectors,
>> > > > + qemu_block_queue_callback, request);
>> > > > + if (res) {
>> > > > + request->real_acb = res;
>> > > > + }
>> > > > +
>> > > > + ret = (res == NULL) ? 0 : 1;
>> > > > +
>> > > > + return ret;
>> > >
>> > > You mean return (res != NULL); and want to have bool as the return
>> > > value
>> > > of this function.
>> >
>> > Yeah, thanks. i will modify as below:
>> > ret = (res == NULL) ? false : true;
>>
>> ret = (res != NULL) is really more readable.
>
> "return (res != NULL);" is even nicer! :)
Great, thanks
>
> Paolo
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
Regards,
Zhi Yong Wu
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [Qemu-devel] [PATCH v8 1/4] block: add the block queue support
2011-10-17 10:17 ` Kevin Wolf
[not found] ` <4E9C00D2.1040004@redhat.com>
@ 2011-10-18 8:07 ` Zhi Yong Wu
2011-10-18 8:36 ` Kevin Wolf
1 sibling, 1 reply; 13+ messages in thread
From: Zhi Yong Wu @ 2011-10-18 8:07 UTC (permalink / raw)
To: Kevin Wolf; +Cc: Zhi Yong Wu, stefanha, kvm, qemu-devel
On Mon, Oct 17, 2011 at 6:17 PM, Kevin Wolf <kwolf@redhat.com> wrote:
> Am 26.09.2011 10:01, schrieb Zhi Yong Wu:
>> On Fri, Sep 23, 2011 at 11:32 PM, Kevin Wolf <kwolf@redhat.com> wrote:
>>> Am 08.09.2011 12:11, schrieb Zhi Yong Wu:
>>>> Signed-off-by: Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
>>>> ---
>>>> Makefile.objs | 2 +-
>>>> block/blk-queue.c | 201 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>> block/blk-queue.h | 59 ++++++++++++++++
>>>> block_int.h | 27 +++++++
>>>> 4 files changed, 288 insertions(+), 1 deletions(-)
>>>> create mode 100644 block/blk-queue.c
>>>> create mode 100644 block/blk-queue.h
>>>>
>>>> diff --git a/Makefile.objs b/Makefile.objs
>>>> index 26b885b..5dcf456 100644
>>>> --- a/Makefile.objs
>>>> +++ b/Makefile.objs
>>>> @@ -33,7 +33,7 @@ block-nested-y += raw.o cow.o qcow.o vdi.o vmdk.o cloop.o dmg.o bochs.o vpc.o vv
>>>> block-nested-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o
>>>> block-nested-y += qed.o qed-gencb.o qed-l2-cache.o qed-table.o qed-cluster.o
>>>> block-nested-y += qed-check.o
>>>> -block-nested-y += parallels.o nbd.o blkdebug.o sheepdog.o blkverify.o
>>>> +block-nested-y += parallels.o nbd.o blkdebug.o sheepdog.o blkverify.o blk-queue.o
>>>> block-nested-$(CONFIG_WIN32) += raw-win32.o
>>>> block-nested-$(CONFIG_POSIX) += raw-posix.o
>>>> block-nested-$(CONFIG_CURL) += curl.o
>>>> diff --git a/block/blk-queue.c b/block/blk-queue.c
>>>> new file mode 100644
>>>> index 0000000..adef497
>>>> --- /dev/null
>>>> +++ b/block/blk-queue.c
>>>> @@ -0,0 +1,201 @@
>>>> +/*
>>>> + * QEMU System Emulator queue definition for block layer
>>>> + *
>>>> + * Copyright (c) IBM, Corp. 2011
>>>> + *
>>>> + * Authors:
>>>> + * Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
>>>> + * Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
>>>> + *
>>>> + * Permission is hereby granted, free of charge, to any person obtaining a copy
>>>> + * of this software and associated documentation files (the "Software"), to deal
>>>> + * in the Software without restriction, including without limitation the rights
>>>> + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
>>>> + * copies of the Software, and to permit persons to whom the Software is
>>>> + * furnished to do so, subject to the following conditions:
>>>> + *
>>>> + * The above copyright notice and this permission notice shall be included in
>>>> + * all copies or substantial portions of the Software.
>>>> + *
>>>> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
>>>> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
>>>> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
>>>> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
>>>> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
>>>> + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
>>>> + * THE SOFTWARE.
>>>> + */
>>>> +
>>>> +#include "block_int.h"
>>>> +#include "block/blk-queue.h"
>>>> +#include "qemu-common.h"
>>>> +
>>>> +/* The APIs for block request queue on qemu block layer.
>>>> + */
>>>> +
>>>> +struct BlockQueueAIOCB {
>>>> + BlockDriverAIOCB common;
>>>> + QTAILQ_ENTRY(BlockQueueAIOCB) entry;
>>>> + BlockRequestHandler *handler;
>>>> + BlockDriverAIOCB *real_acb;
>>>> +
>>>> + int64_t sector_num;
>>>> + QEMUIOVector *qiov;
>>>> + int nb_sectors;
>>>> +};
>>>
>>> The idea is that each request is first queued on the QTAILQ, and at some
>>> point it's removed from the queue and gets a real_acb. But it never has
>>> both at the same time. Correct?
>> NO. if block I/O throttling is enabled and I/O rate at runtime exceed
>> this limits, this request will be enqueued.
>> It represents the whole lifecycle of one enqueued request.
>
> What are the conditions under which the request will still be enqueued,
> but has a real_acb at the same time?
When the request is not serviced and still need to be enqueued, one
real_acb will be existable at the same time.
thanks for your good catch.:)
When one request will be mallocated at the first time, we should save
the got acb to real_acb
qemu_block_queue_enqueue():
request->real_acb = acb;
>
>>>> +
>>>> +typedef struct BlockQueueAIOCB BlockQueueAIOCB;
>>>> +
>>>> +struct BlockQueue {
>>>> + QTAILQ_HEAD(requests, BlockQueueAIOCB) requests;
>>>> + bool req_failed;
>>>> + bool flushing;
>>>> +};
>>>
>>> I find req_failed pretty confusing. Needs documentation at least, but
>>> most probably also a better name.
>> OK. request_has_failed?
>
> No, that doesn't describe what it's really doing.
>
> You set req_failed = true by default and then on some obscure condition
> clear it or not. It's tracking something, but I'm not sure what meaning
> it has during the whole process.
In qemu_block_queue_flush,
When bdrv_aio_readv/writev return NULL, if req_failed is changed to
false, it indicates that the request exceeds the limits again; if
req_failed is still true, it indicates that one I/O error takes place,
at the monent, qemu_block_queue_callback(request, -EIO) need to be
called to report this to upper layer.
>
>>>> +
>>>> +static void qemu_block_queue_dequeue(BlockQueue *queue,
>>>> + BlockQueueAIOCB *request)
>>>> +{
>>>> + BlockQueueAIOCB *req;
>>>> +
>>>> + assert(queue);
>>>> + while (!QTAILQ_EMPTY(&queue->requests)) {
>>>> + req = QTAILQ_FIRST(&queue->requests);
>>>> + if (req == request) {
>>>> + QTAILQ_REMOVE(&queue->requests, req, entry);
>>>> + break;
>>>> + }
>>>> + }
>>>> +}
>>>
>>> Is it just me or is this an endless loop if the request isn't the first
>>> element in the list?
>> queue->requests is only used to store requests which exceed the limits.
>> Why is the request not the first evlement?
>
> Why do you have a loop if it's always the first element?
Ah, it can cause dead loop, and QTAILQ_FOREACH_SAFE should be adopted
here. thanks.
QTAILQ_FOREACH_SAFE(req, &queue->requests, entry, next) {
if (*req == *request) {
QTAILQ_REMOVE(&queue->requests, req, entry);
break;
}
}
>
>>>> +void qemu_del_block_queue(BlockQueue *queue)
>>>> +{
>>>> + BlockQueueAIOCB *request, *next;
>>>> +
>>>> + QTAILQ_FOREACH_SAFE(request, &queue->requests, entry, next) {
>>>> + QTAILQ_REMOVE(&queue->requests, request, entry);
>>>> + qemu_aio_release(request);
>>>> + }
>>>> +
>>>> + g_free(queue);
>>>> +}
>>>
>>> Can we be sure that no AIO requests are in flight that still use the now
>>> released AIOCB?
>> Yeah, since qemu core code is serially performed, i think that when
>> qemu_del_block_queue is performed, no requests are in flight. Right?
>
> Patch 2 has this code:
>
> +void bdrv_io_limits_disable(BlockDriverState *bs)
> +{
> + bs->io_limits_enabled = false;
> +
> + if (bs->block_queue) {
> + qemu_block_queue_flush(bs->block_queue);
> + qemu_del_block_queue(bs->block_queue);
> + bs->block_queue = NULL;
> + }
>
> Does this mean that you can't disable I/O limits while the VM is running?
NO, you can even though VM is running.
>
>>>> +
>>>> +BlockDriverAIOCB *qemu_block_queue_enqueue(BlockQueue *queue,
>>>> + BlockDriverState *bs,
>>>> + BlockRequestHandler *handler,
>>>> + int64_t sector_num,
>>>> + QEMUIOVector *qiov,
>>>> + int nb_sectors,
>>>> + BlockDriverCompletionFunc *cb,
>>>> + void *opaque)
>>>> +{
>>>> + BlockDriverAIOCB *acb;
>>>> + BlockQueueAIOCB *request;
>>>> +
>>>> + if (queue->flushing) {
>>>> + queue->req_failed = false;
>>>> + return NULL;
>>>> + } else {
>>>> + acb = qemu_aio_get(&block_queue_pool, bs,
>>>> + cb, opaque);
>>>> + request = container_of(acb, BlockQueueAIOCB, common);
>>>> + request->handler = handler;
>>>> + request->sector_num = sector_num;
>>>> + request->qiov = qiov;
>>>> + request->nb_sectors = nb_sectors;
>>>> + request->real_acb = NULL;
>>>> + QTAILQ_INSERT_TAIL(&queue->requests, request, entry);
>>>> + }
>>>> +
>>>> + return acb;
>>>> +}
>>>> +
>>>> +static int qemu_block_queue_handler(BlockQueueAIOCB *request)
>>>> +{
>>>> + int ret;
>>>> + BlockDriverAIOCB *res;
>>>> +
>>>> + res = request->handler(request->common.bs, request->sector_num,
>>>> + request->qiov, request->nb_sectors,
>>>> + qemu_block_queue_callback, request);
>>>> + if (res) {
>>>> + request->real_acb = res;
>>>> + }
>>>> +
>>>> + ret = (res == NULL) ? 0 : 1;
>>>> +
>>>> + return ret;
>>>
>>> You mean return (res != NULL); and want to have bool as the return value
>>> of this function.
>> Yeah, thanks. i will modify as below:
>> ret = (res == NULL) ? false : true;
>
> ret = (res != NULL) is really more readable.
I have adopted Paolo's suggestion.
>
>> and
>> static bool qemu_block_queue_handler()
>>
>>>
>>>> +}
>>>> +
>>>> +void qemu_block_queue_flush(BlockQueue *queue)
>>>> +{
>>>> + queue->flushing = true;
>>>> + while (!QTAILQ_EMPTY(&queue->requests)) {
>>>> + BlockQueueAIOCB *request = NULL;
>>>> + int ret = 0;
>>>> +
>>>> + request = QTAILQ_FIRST(&queue->requests);
>>>> + QTAILQ_REMOVE(&queue->requests, request, entry);
>>>> +
>>>> + queue->req_failed = true;
>>>> + ret = qemu_block_queue_handler(request);
>>>> + if (ret == 0) {
>>>> + QTAILQ_INSERT_HEAD(&queue->requests, request, entry);
>>>> + if (queue->req_failed) {
>>>> + qemu_block_queue_callback(request, -EIO);
>>>> + break;
>>>> + }
>>>> + }
>>>> + }
>>>> +
>>>> + queue->req_failed = true;
>>>> + queue->flushing = false;
>>>> +}
>>>> +
>>>> +bool qemu_block_queue_has_pending(BlockQueue *queue)
>>>> +{
>>>> + return !queue->flushing && !QTAILQ_EMPTY(&queue->requests);
>>>> +}
>>>
>>> Why doesn't the queue have pending requests in the middle of a flush
>>> operation? (That is, the flush hasn't completed yet)
>> It is possible for the queue to have pending requests. if yes, how about?
>
> Sorry, can't parse this.
>
> I don't understand why the !queue->flushing part is correct.
>
> Kevin
>
--
Regards,
Zhi Yong Wu
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [Qemu-devel] [PATCH v8 1/4] block: add the block queue support
2011-10-18 8:07 ` Zhi Yong Wu
@ 2011-10-18 8:36 ` Kevin Wolf
2011-10-18 9:29 ` Zhi Yong Wu
0 siblings, 1 reply; 13+ messages in thread
From: Kevin Wolf @ 2011-10-18 8:36 UTC (permalink / raw)
To: Zhi Yong Wu; +Cc: Zhi Yong Wu, stefanha, kvm, qemu-devel
Am 18.10.2011 10:07, schrieb Zhi Yong Wu:
> On Mon, Oct 17, 2011 at 6:17 PM, Kevin Wolf <kwolf@redhat.com> wrote:
>>>>> +
>>>>> +typedef struct BlockQueueAIOCB BlockQueueAIOCB;
>>>>> +
>>>>> +struct BlockQueue {
>>>>> + QTAILQ_HEAD(requests, BlockQueueAIOCB) requests;
>>>>> + bool req_failed;
>>>>> + bool flushing;
>>>>> +};
>>>>
>>>> I find req_failed pretty confusing. Needs documentation at least, but
>>>> most probably also a better name.
>>> OK. request_has_failed?
>>
>> No, that doesn't describe what it's really doing.
>>
>> You set req_failed = true by default and then on some obscure condition
>> clear it or not. It's tracking something, but I'm not sure what meaning
>> it has during the whole process.
> In qemu_block_queue_flush,
> When bdrv_aio_readv/writev return NULL, if req_failed is changed to
> false, it indicates that the request exceeds the limits again; if
> req_failed is still true, it indicates that one I/O error takes place,
> at the monent, qemu_block_queue_callback(request, -EIO) need to be
> called to report this to upper layer.
Okay, this makes some more sense now.
How about reversing the logic and maintaining a limit_exceeded flag
instead of a req_failed? I think this would be clearer.
>>>>> +void qemu_del_block_queue(BlockQueue *queue)
>>>>> +{
>>>>> + BlockQueueAIOCB *request, *next;
>>>>> +
>>>>> + QTAILQ_FOREACH_SAFE(request, &queue->requests, entry, next) {
>>>>> + QTAILQ_REMOVE(&queue->requests, request, entry);
>>>>> + qemu_aio_release(request);
>>>>> + }
>>>>> +
>>>>> + g_free(queue);
>>>>> +}
>>>>
>>>> Can we be sure that no AIO requests are in flight that still use the now
>>>> released AIOCB?
>>> Yeah, since qemu core code is serially performed, i think that when
>>> qemu_del_block_queue is performed, no requests are in flight. Right?
>>
>> Patch 2 has this code:
>>
>> +void bdrv_io_limits_disable(BlockDriverState *bs)
>> +{
>> + bs->io_limits_enabled = false;
>> +
>> + if (bs->block_queue) {
>> + qemu_block_queue_flush(bs->block_queue);
>> + qemu_del_block_queue(bs->block_queue);
>> + bs->block_queue = NULL;
>> + }
>>
>> Does this mean that you can't disable I/O limits while the VM is running?
> NO, you can even though VM is running.
Okay, in general qemu_block_queue_flush() empties the queue so that
there are no requests left that qemu_del_block_queue() could drop from
the queue. So in the common case it doesn't even enter the FOREACH loop.
I think problems start when requests have failed or exceeded the limit
again, then you have requests queued even after
qemu_block_queue_flush(). You must be aware of this, otherwise the code
in qemu_del_block_queue() wouldn't exist.
But you can't release the ACBs without having called their callback,
otherwise the caller would still assume that its ACB pointer is valid.
Maybe calling the callback before releasing the ACB would be enough.
However, for failed requests see below.
>>
>>>>> +
>>>>> +BlockDriverAIOCB *qemu_block_queue_enqueue(BlockQueue *queue,
>>>>> + BlockDriverState *bs,
>>>>> + BlockRequestHandler *handler,
>>>>> + int64_t sector_num,
>>>>> + QEMUIOVector *qiov,
>>>>> + int nb_sectors,
>>>>> + BlockDriverCompletionFunc *cb,
>>>>> + void *opaque)
>>>>> +{
>>>>> + BlockDriverAIOCB *acb;
>>>>> + BlockQueueAIOCB *request;
>>>>> +
>>>>> + if (queue->flushing) {
>>>>> + queue->req_failed = false;
>>>>> + return NULL;
>>>>> + } else {
>>>>> + acb = qemu_aio_get(&block_queue_pool, bs,
>>>>> + cb, opaque);
>>>>> + request = container_of(acb, BlockQueueAIOCB, common);
>>>>> + request->handler = handler;
>>>>> + request->sector_num = sector_num;
>>>>> + request->qiov = qiov;
>>>>> + request->nb_sectors = nb_sectors;
>>>>> + request->real_acb = NULL;
>>>>> + QTAILQ_INSERT_TAIL(&queue->requests, request, entry);
>>>>> + }
>>>>> +
>>>>> + return acb;
>>>>> +}
>>>>> +
>>>>> +static int qemu_block_queue_handler(BlockQueueAIOCB *request)
>>>>> +{
>>>>> + int ret;
>>>>> + BlockDriverAIOCB *res;
>>>>> +
>>>>> + res = request->handler(request->common.bs, request->sector_num,
>>>>> + request->qiov, request->nb_sectors,
>>>>> + qemu_block_queue_callback, request);
>>>>> + if (res) {
>>>>> + request->real_acb = res;
>>>>> + }
>>>>> +
>>>>> + ret = (res == NULL) ? 0 : 1;
>>>>> +
>>>>> + return ret;
>>>>
>>>> You mean return (res != NULL); and want to have bool as the return value
>>>> of this function.
>>> Yeah, thanks. i will modify as below:
>>> ret = (res == NULL) ? false : true;
>>
>> ret = (res != NULL) is really more readable.
> I have adopted Paolo's suggestion.
>
>>
>>> and
>>> static bool qemu_block_queue_handler()
>>>
>>>>
>>>>> +}
>>>>> +
>>>>> +void qemu_block_queue_flush(BlockQueue *queue)
>>>>> +{
>>>>> + queue->flushing = true;
>>>>> + while (!QTAILQ_EMPTY(&queue->requests)) {
>>>>> + BlockQueueAIOCB *request = NULL;
>>>>> + int ret = 0;
>>>>> +
>>>>> + request = QTAILQ_FIRST(&queue->requests);
>>>>> + QTAILQ_REMOVE(&queue->requests, request, entry);
>>>>> +
>>>>> + queue->req_failed = true;
>>>>> + ret = qemu_block_queue_handler(request);
>>>>> + if (ret == 0) {
>>>>> + QTAILQ_INSERT_HEAD(&queue->requests, request, entry);
>>>>> + if (queue->req_failed) {
>>>>> + qemu_block_queue_callback(request, -EIO);
>>>>> + break;
When a request has failed, you call its callback, but still leave it
queued. I think this is wrong.
>>>>> + }
>>>>> + }
>>>>> + }
>>>>> +
>>>>> + queue->req_failed = true;
>>>>> + queue->flushing = false;
>>>>> +}
>>>>> +
>>>>> +bool qemu_block_queue_has_pending(BlockQueue *queue)
>>>>> +{
>>>>> + return !queue->flushing && !QTAILQ_EMPTY(&queue->requests);
>>>>> +}
>>>>
>>>> Why doesn't the queue have pending requests in the middle of a flush
>>>> operation? (That is, the flush hasn't completed yet)
>>> It is possible for the queue to have pending requests. if yes, how about?
>>
>> Sorry, can't parse this.
>>
>> I don't understand why the !queue->flushing part is correct.
What about this?
Kevin
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [Qemu-devel] [PATCH v8 1/4] block: add the block queue support
2011-10-18 8:36 ` Kevin Wolf
@ 2011-10-18 9:29 ` Zhi Yong Wu
2011-10-18 9:56 ` Kevin Wolf
0 siblings, 1 reply; 13+ messages in thread
From: Zhi Yong Wu @ 2011-10-18 9:29 UTC (permalink / raw)
To: Kevin Wolf; +Cc: Zhi Yong Wu, stefanha, kvm, qemu-devel
On Tue, Oct 18, 2011 at 4:36 PM, Kevin Wolf <kwolf@redhat.com> wrote:
> Am 18.10.2011 10:07, schrieb Zhi Yong Wu:
>> On Mon, Oct 17, 2011 at 6:17 PM, Kevin Wolf <kwolf@redhat.com> wrote:
>>>>>> +
>>>>>> +typedef struct BlockQueueAIOCB BlockQueueAIOCB;
>>>>>> +
>>>>>> +struct BlockQueue {
>>>>>> + QTAILQ_HEAD(requests, BlockQueueAIOCB) requests;
>>>>>> + bool req_failed;
>>>>>> + bool flushing;
>>>>>> +};
>>>>>
>>>>> I find req_failed pretty confusing. Needs documentation at least, but
>>>>> most probably also a better name.
>>>> OK. request_has_failed?
>>>
>>> No, that doesn't describe what it's really doing.
>>>
>>> You set req_failed = true by default and then on some obscure condition
>>> clear it or not. It's tracking something, but I'm not sure what meaning
>>> it has during the whole process.
>> In qemu_block_queue_flush,
>> When bdrv_aio_readv/writev return NULL, if req_failed is changed to
>> false, it indicates that the request exceeds the limits again; if
>> req_failed is still true, it indicates that one I/O error takes place,
>> at the monent, qemu_block_queue_callback(request, -EIO) need to be
>> called to report this to upper layer.
>
> Okay, this makes some more sense now.
>
> How about reversing the logic and maintaining a limit_exceeded flag
> instead of a req_failed? I think this would be clearer.
OK
>
>>>>>> +void qemu_del_block_queue(BlockQueue *queue)
>>>>>> +{
>>>>>> + BlockQueueAIOCB *request, *next;
>>>>>> +
>>>>>> + QTAILQ_FOREACH_SAFE(request, &queue->requests, entry, next) {
>>>>>> + QTAILQ_REMOVE(&queue->requests, request, entry);
>>>>>> + qemu_aio_release(request);
>>>>>> + }
>>>>>> +
>>>>>> + g_free(queue);
>>>>>> +}
>>>>>
>>>>> Can we be sure that no AIO requests are in flight that still use the now
>>>>> released AIOCB?
>>>> Yeah, since qemu core code is serially performed, i think that when
>>>> qemu_del_block_queue is performed, no requests are in flight. Right?
>>>
>>> Patch 2 has this code:
>>>
>>> +void bdrv_io_limits_disable(BlockDriverState *bs)
>>> +{
>>> + bs->io_limits_enabled = false;
>>> +
>>> + if (bs->block_queue) {
>>> + qemu_block_queue_flush(bs->block_queue);
>>> + qemu_del_block_queue(bs->block_queue);
>>> + bs->block_queue = NULL;
>>> + }
>>>
>>> Does this mean that you can't disable I/O limits while the VM is running?
>> NO, you can even though VM is running.
>
> Okay, in general qemu_block_queue_flush() empties the queue so that
> there are no requests left that qemu_del_block_queue() could drop from
> the queue. So in the common case it doesn't even enter the FOREACH loop.
I think that we should adopt !QTAILQ_EMPTY(&queue->requests), not
QTAILQ_FOREACH_SAFE in qemu_del_block_queue(),
right?
>
> I think problems start when requests have failed or exceeded the limit
> again, then you have requests queued even after
> qemu_block_queue_flush(). You must be aware of this, otherwise the code
> in qemu_del_block_queue() wouldn't exist.
>
> But you can't release the ACBs without having called their callback,
> otherwise the caller would still assume that its ACB pointer is valid.
> Maybe calling the callback before releasing the ACB would be enough.
Good, thanks.
>
> However, for failed requests see below.
>
>>>
>>>>>> +
>>>>>> +BlockDriverAIOCB *qemu_block_queue_enqueue(BlockQueue *queue,
>>>>>> + BlockDriverState *bs,
>>>>>> + BlockRequestHandler *handler,
>>>>>> + int64_t sector_num,
>>>>>> + QEMUIOVector *qiov,
>>>>>> + int nb_sectors,
>>>>>> + BlockDriverCompletionFunc *cb,
>>>>>> + void *opaque)
>>>>>> +{
>>>>>> + BlockDriverAIOCB *acb;
>>>>>> + BlockQueueAIOCB *request;
>>>>>> +
>>>>>> + if (queue->flushing) {
>>>>>> + queue->req_failed = false;
>>>>>> + return NULL;
>>>>>> + } else {
>>>>>> + acb = qemu_aio_get(&block_queue_pool, bs,
>>>>>> + cb, opaque);
>>>>>> + request = container_of(acb, BlockQueueAIOCB, common);
>>>>>> + request->handler = handler;
>>>>>> + request->sector_num = sector_num;
>>>>>> + request->qiov = qiov;
>>>>>> + request->nb_sectors = nb_sectors;
>>>>>> + request->real_acb = NULL;
>>>>>> + QTAILQ_INSERT_TAIL(&queue->requests, request, entry);
>>>>>> + }
>>>>>> +
>>>>>> + return acb;
>>>>>> +}
>>>>>> +
>>>>>> +static int qemu_block_queue_handler(BlockQueueAIOCB *request)
>>>>>> +{
>>>>>> + int ret;
>>>>>> + BlockDriverAIOCB *res;
>>>>>> +
>>>>>> + res = request->handler(request->common.bs, request->sector_num,
>>>>>> + request->qiov, request->nb_sectors,
>>>>>> + qemu_block_queue_callback, request);
>>>>>> + if (res) {
>>>>>> + request->real_acb = res;
>>>>>> + }
>>>>>> +
>>>>>> + ret = (res == NULL) ? 0 : 1;
>>>>>> +
>>>>>> + return ret;
>>>>>
>>>>> You mean return (res != NULL); and want to have bool as the return value
>>>>> of this function.
>>>> Yeah, thanks. i will modify as below:
>>>> ret = (res == NULL) ? false : true;
>>>
>>> ret = (res != NULL) is really more readable.
>> I have adopted Paolo's suggestion.
>>
>>>
>>>> and
>>>> static bool qemu_block_queue_handler()
>>>>
>>>>>
>>>>>> +}
>>>>>> +
>>>>>> +void qemu_block_queue_flush(BlockQueue *queue)
>>>>>> +{
>>>>>> + queue->flushing = true;
>>>>>> + while (!QTAILQ_EMPTY(&queue->requests)) {
>>>>>> + BlockQueueAIOCB *request = NULL;
>>>>>> + int ret = 0;
>>>>>> +
>>>>>> + request = QTAILQ_FIRST(&queue->requests);
>>>>>> + QTAILQ_REMOVE(&queue->requests, request, entry);
>>>>>> +
>>>>>> + queue->req_failed = true;
>>>>>> + ret = qemu_block_queue_handler(request);
>>>>>> + if (ret == 0) {
>>>>>> + QTAILQ_INSERT_HEAD(&queue->requests, request, entry);
>>>>>> + if (queue->req_failed) {
>>>>>> + qemu_block_queue_callback(request, -EIO);
>>>>>> + break;
>
> When a request has failed, you call its callback, but still leave it
> queued. I think this is wrong.
Right, we should not insert the failed request back to block queue.
>
>>>>>> + }
>>>>>> + }
>>>>>> + }
>>>>>> +
>>>>>> + queue->req_failed = true;
>>>>>> + queue->flushing = false;
>>>>>> +}
>>>>>> +
>>>>>> +bool qemu_block_queue_has_pending(BlockQueue *queue)
>>>>>> +{
>>>>>> + return !queue->flushing && !QTAILQ_EMPTY(&queue->requests);
>>>>>> +}
>>>>>
>>>>> Why doesn't the queue have pending requests in the middle of a flush
>>>>> operation? (That is, the flush hasn't completed yet)
>>>> It is possible for the queue to have pending requests. if yes, how about?
>>>
>>> Sorry, can't parse this.
>>>
>>> I don't understand why the !queue->flushing part is correct.
>
> What about this?
When bdrv_aio_readv/writev handle one request, it will determine if
block queue is not being flushed and isn't NULL; if yes, It assume
that this request is one new request from upper layer, so it won't
determine if the I/O rate at runtime has exceeded the limits, but
immediately insert it into block queue.
>
> Kevin
>
--
Regards,
Zhi Yong Wu
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [Qemu-devel] [PATCH v8 1/4] block: add the block queue support
2011-10-18 9:29 ` Zhi Yong Wu
@ 2011-10-18 9:56 ` Kevin Wolf
2011-10-18 13:29 ` Zhi Yong Wu
0 siblings, 1 reply; 13+ messages in thread
From: Kevin Wolf @ 2011-10-18 9:56 UTC (permalink / raw)
To: Zhi Yong Wu; +Cc: Zhi Yong Wu, stefanha, kvm, qemu-devel
Am 18.10.2011 11:29, schrieb Zhi Yong Wu:
>>>>>>> +void qemu_del_block_queue(BlockQueue *queue)
>>>>>>> +{
>>>>>>> + BlockQueueAIOCB *request, *next;
>>>>>>> +
>>>>>>> + QTAILQ_FOREACH_SAFE(request, &queue->requests, entry, next) {
>>>>>>> + QTAILQ_REMOVE(&queue->requests, request, entry);
>>>>>>> + qemu_aio_release(request);
>>>>>>> + }
>>>>>>> +
>>>>>>> + g_free(queue);
>>>>>>> +}
>>>>>>
>>>>>> Can we be sure that no AIO requests are in flight that still use the now
>>>>>> released AIOCB?
>>>>> Yeah, since qemu core code is serially performed, i think that when
>>>>> qemu_del_block_queue is performed, no requests are in flight. Right?
>>>>
>>>> Patch 2 has this code:
>>>>
>>>> +void bdrv_io_limits_disable(BlockDriverState *bs)
>>>> +{
>>>> + bs->io_limits_enabled = false;
>>>> +
>>>> + if (bs->block_queue) {
>>>> + qemu_block_queue_flush(bs->block_queue);
>>>> + qemu_del_block_queue(bs->block_queue);
>>>> + bs->block_queue = NULL;
>>>> + }
>>>>
>>>> Does this mean that you can't disable I/O limits while the VM is running?
>>> NO, you can even though VM is running.
>>
>> Okay, in general qemu_block_queue_flush() empties the queue so that
>> there are no requests left that qemu_del_block_queue() could drop from
>> the queue. So in the common case it doesn't even enter the FOREACH loop.
> I think that we should adopt !QTAILQ_EMPTY(&queue->requests), not
> QTAILQ_FOREACH_SAFE in qemu_del_block_queue(),
> right?
I think QTAILQ_FOREACH_SAFE is fine.
>>
>> I think problems start when requests have failed or exceeded the limit
>> again, then you have requests queued even after
>> qemu_block_queue_flush(). You must be aware of this, otherwise the code
>> in qemu_del_block_queue() wouldn't exist.
>>
>> But you can't release the ACBs without having called their callback,
>> otherwise the caller would still assume that its ACB pointer is valid.
>> Maybe calling the callback before releasing the ACB would be enough.
> Good, thanks.
>>>>
>>>>>>> + }
>>>>>>> + }
>>>>>>> + }
>>>>>>> +
>>>>>>> + queue->req_failed = true;
>>>>>>> + queue->flushing = false;
>>>>>>> +}
>>>>>>> +
>>>>>>> +bool qemu_block_queue_has_pending(BlockQueue *queue)
>>>>>>> +{
>>>>>>> + return !queue->flushing && !QTAILQ_EMPTY(&queue->requests);
>>>>>>> +}
>>>>>>
>>>>>> Why doesn't the queue have pending requests in the middle of a flush
>>>>>> operation? (That is, the flush hasn't completed yet)
>>>>> It is possible for the queue to have pending requests. if yes, how about?
>>>>
>>>> Sorry, can't parse this.
>>>>
>>>> I don't understand why the !queue->flushing part is correct.
>>
>> What about this?
> When bdrv_aio_readv/writev handle one request, it will determine if
> block queue is not being flushed and isn't NULL; if yes, It assume
> that this request is one new request from upper layer, so it won't
> determine if the I/O rate at runtime has exceeded the limits, but
> immediately insert it into block queue.
Hm, I think I understand what you're saying, but only after looking at
patch 3. This is not really implementing a has_pending(), but
has_pending_and_caller_wasnt_called_during_flush(). I think it would be
better to handle the queue->flushing condition in the caller where its
use is more obvious.
Kevin
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [Qemu-devel] [PATCH v8 1/4] block: add the block queue support
2011-10-18 9:56 ` Kevin Wolf
@ 2011-10-18 13:29 ` Zhi Yong Wu
0 siblings, 0 replies; 13+ messages in thread
From: Zhi Yong Wu @ 2011-10-18 13:29 UTC (permalink / raw)
To: Kevin Wolf; +Cc: Zhi Yong Wu, stefanha, kvm, qemu-devel
On Tue, Oct 18, 2011 at 5:56 PM, Kevin Wolf <kwolf@redhat.com> wrote:
> Am 18.10.2011 11:29, schrieb Zhi Yong Wu:
>>>>>>>> +void qemu_del_block_queue(BlockQueue *queue)
>>>>>>>> +{
>>>>>>>> + BlockQueueAIOCB *request, *next;
>>>>>>>> +
>>>>>>>> + QTAILQ_FOREACH_SAFE(request, &queue->requests, entry, next) {
>>>>>>>> + QTAILQ_REMOVE(&queue->requests, request, entry);
>>>>>>>> + qemu_aio_release(request);
>>>>>>>> + }
>>>>>>>> +
>>>>>>>> + g_free(queue);
>>>>>>>> +}
>>>>>>>
>>>>>>> Can we be sure that no AIO requests are in flight that still use the now
>>>>>>> released AIOCB?
>>>>>> Yeah, since qemu core code is serially performed, i think that when
>>>>>> qemu_del_block_queue is performed, no requests are in flight. Right?
>>>>>
>>>>> Patch 2 has this code:
>>>>>
>>>>> +void bdrv_io_limits_disable(BlockDriverState *bs)
>>>>> +{
>>>>> + bs->io_limits_enabled = false;
>>>>> +
>>>>> + if (bs->block_queue) {
>>>>> + qemu_block_queue_flush(bs->block_queue);
>>>>> + qemu_del_block_queue(bs->block_queue);
>>>>> + bs->block_queue = NULL;
>>>>> + }
>>>>>
>>>>> Does this mean that you can't disable I/O limits while the VM is running?
>>>> NO, you can even though VM is running.
>>>
>>> Okay, in general qemu_block_queue_flush() empties the queue so that
>>> there are no requests left that qemu_del_block_queue() could drop from
>>> the queue. So in the common case it doesn't even enter the FOREACH loop.
>> I think that we should adopt !QTAILQ_EMPTY(&queue->requests), not
>> QTAILQ_FOREACH_SAFE in qemu_del_block_queue(),
>> right?
>
> I think QTAILQ_FOREACH_SAFE is fine.
>
>>>
>>> I think problems start when requests have failed or exceeded the limit
>>> again, then you have requests queued even after
>>> qemu_block_queue_flush(). You must be aware of this, otherwise the code
>>> in qemu_del_block_queue() wouldn't exist.
>>>
>>> But you can't release the ACBs without having called their callback,
>>> otherwise the caller would still assume that its ACB pointer is valid.
>>> Maybe calling the callback before releasing the ACB would be enough.
>> Good, thanks.
>>>>>
>>>>>>>> + }
>>>>>>>> + }
>>>>>>>> + }
>>>>>>>> +
>>>>>>>> + queue->req_failed = true;
>>>>>>>> + queue->flushing = false;
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> +bool qemu_block_queue_has_pending(BlockQueue *queue)
>>>>>>>> +{
>>>>>>>> + return !queue->flushing && !QTAILQ_EMPTY(&queue->requests);
>>>>>>>> +}
>>>>>>>
>>>>>>> Why doesn't the queue have pending requests in the middle of a flush
>>>>>>> operation? (That is, the flush hasn't completed yet)
>>>>>> It is possible for the queue to have pending requests. if yes, how about?
>>>>>
>>>>> Sorry, can't parse this.
>>>>>
>>>>> I don't understand why the !queue->flushing part is correct.
>>>
>>> What about this?
>> When bdrv_aio_readv/writev handle one request, it will determine if
>> block queue is not being flushed and isn't NULL; if yes, It assume
>> that this request is one new request from upper layer, so it won't
>> determine if the I/O rate at runtime has exceeded the limits, but
>> immediately insert it into block queue.
>
> Hm, I think I understand what you're saying, but only after looking at
> patch 3. This is not really implementing a has_pending(), but
> has_pending_and_caller_wasnt_called_during_flush(). I think it would be
Correct.
> better to handle the queue->flushing condition in the caller where its
> use is more obvious.
OK. i will do as this.
>
> Kevin
>
--
Regards,
Zhi Yong Wu
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2011-10-18 13:29 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-09-07 12:31 [Qemu-devel] [PATCH v8 0/4] The intro of QEMU block I/O throttling Zhi Yong Wu
2011-09-07 12:31 ` [Qemu-devel] [PATCH v8 1/4] block: add the block queue support Zhi Yong Wu
2011-09-07 12:31 ` [Qemu-devel] [PATCH v8 2/4] block: add the command line support Zhi Yong Wu
-- strict thread matches above, loose matches on Subject: below --
2011-09-08 10:11 [Qemu-devel] [PATCH v8 0/4] The intro of QEMU block I/O throttling Zhi Yong Wu
2011-09-08 10:11 ` [Qemu-devel] [PATCH v8 1/4] block: add the block queue support Zhi Yong Wu
2011-09-23 15:32 ` Kevin Wolf
2011-09-26 8:01 ` Zhi Yong Wu
2011-10-17 10:17 ` Kevin Wolf
[not found] ` <4E9C00D2.1040004@redhat.com>
2011-10-18 7:00 ` Zhi Yong Wu
2011-10-18 8:07 ` Zhi Yong Wu
2011-10-18 8:36 ` Kevin Wolf
2011-10-18 9:29 ` Zhi Yong Wu
2011-10-18 9:56 ` Kevin Wolf
2011-10-18 13:29 ` Zhi Yong Wu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).