From: David Edmondson <david.edmondson@oracle.com>
To: qemu-devel@nongnu.org
Cc: Kevin Wolf <kwolf@redhat.com>, Fam Zheng <fam@euphon.net>,
qemu-block@nongnu.org, Markus Armbruster <armbru@redhat.com>,
Max Reitz <mreitz@redhat.com>,
David Edmondson <david.edmondson@oracle.com>,
Stefan Hajnoczi <stefanha@redhat.com>
Subject: [RFC PATCH 7/9] block/curl: Allow the user to control the number of cache blocks
Date: Tue, 18 Aug 2020 12:08:43 +0100 [thread overview]
Message-ID: <20200818110845.3825105-8-david.edmondson@oracle.com> (raw)
In-Reply-To: <20200818110845.3825105-1-david.edmondson@oracle.com>
Rather than using a fixed number, allow the user to specify the number
of cache blocks allocated. This cannot be less than the number of Curl
states and defaults to that value.
Signed-off-by: David Edmondson <david.edmondson@oracle.com>
---
block/curl.c | 20 +++++++++++++++++---
docs/system/device-url-syntax.rst.inc | 4 ++++
qapi/block-core.json | 4 ++++
3 files changed, 25 insertions(+), 3 deletions(-)
diff --git a/block/curl.c b/block/curl.c
index 0ea9eedebd..27fa77c351 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -75,14 +75,15 @@ static CURLMcode __curl_multi_socket_action(CURLM *multi_handle,
#define CURL_BLOCK_OPT_PROXY_PASSWORD_SECRET "proxy-password-secret"
#define CURL_BLOCK_OPT_OFFSET "offset"
#define CURL_BLOCK_OPT_BLOCKSIZE "blocksize"
+#define CURL_BLOCK_OPT_BLOCKCOUNT "blockcount"
#define CURL_BLOCK_OPT_SSLVERIFY_DEFAULT true
#define CURL_BLOCK_OPT_TIMEOUT_DEFAULT 5
/* Must be a non-zero power of 2. */
#define CURL_BLOCK_OPT_BLOCKSIZE_DEFAULT (256 * 1024)
+/* The defaultnumber of blocks to store in the cache. */
+#define CURL_BLOCK_OPT_BLOCKCOUNT_DEFAULT (CURL_NUM_STATES)
-/* The maximum number of blocks to store in the cache. */
-#define CURL_BLOCK_CACHE_MAX_BLOCKS 100
/* The number of heads in the hash table. */
#define CURL_BLOCK_CACHE_HASH 37
@@ -161,6 +162,7 @@ typedef struct BDRVCURLState {
char *proxypassword;
size_t offset;
size_t blocksize;
+ int cache_max;
int cache_allocated; /* The number of block_t currently allocated. */
QLIST_HEAD(, block) cache_free;
QTAILQ_HEAD(, block) cache_lru;
@@ -287,7 +289,7 @@ static block_t *curl_cache_get(BDRVCURLState *s)
}
/* If not at the limit, try get a new one. */
- if (s->cache_allocated < CURL_BLOCK_CACHE_MAX_BLOCKS) {
+ if (s->cache_allocated < s->cache_max) {
b = curl_cache_alloc(s);
if (b) {
b->use++;
@@ -929,6 +931,11 @@ static QemuOptsList runtime_opts = {
.type = QEMU_OPT_SIZE,
.help = "Block size for IO requests"
},
+ {
+ .name = CURL_BLOCK_OPT_BLOCKCOUNT,
+ .type = QEMU_OPT_SIZE,
+ .help = "Maximum number of cached blocks"
+ },
{ /* end of list */ }
},
};
@@ -1039,6 +1046,13 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
error_setg(errp, "blocksize must be a non-zero power of two");
goto out_noclean;
}
+ s->cache_max = qemu_opt_get_size(opts, CURL_BLOCK_OPT_BLOCKCOUNT,
+ CURL_BLOCK_OPT_BLOCKCOUNT_DEFAULT);
+ if (s->cache_max < CURL_NUM_STATES) {
+ error_setg(errp, "blockcount must be larger than %d",
+ CURL_NUM_STATES - 1);
+ goto out_noclean;
+ }
trace_curl_open(file);
qemu_co_queue_init(&s->free_state_waitq);
diff --git a/docs/system/device-url-syntax.rst.inc b/docs/system/device-url-syntax.rst.inc
index ee504ee41a..56843cb38f 100644
--- a/docs/system/device-url-syntax.rst.inc
+++ b/docs/system/device-url-syntax.rst.inc
@@ -201,6 +201,10 @@ These are specified using a special URL syntax.
bytes. The value must be a non-zero power of two. It defaults
to 256kB.
+ ``blockcount``
+ The number of ``blocksize`` blocks that the system may allocate
+ to store data read from the remote server.
+
Note that when passing options to qemu explicitly, ``driver`` is the
value of <protocol>.
diff --git a/qapi/block-core.json b/qapi/block-core.json
index cd16197e1e..91888166fa 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3767,11 +3767,15 @@
# @blocksize: Size of all IO requests sent to the remote server; must
# be a non-zero power of two (defaults to 1 256kB)
#
+# @blockcount: The number of IO blocks used to cache data from the
+# remote server.
+#
# Since: 2.9
##
{ 'struct': 'BlockdevOptionsCurlBase',
'data': { 'url': 'str',
'*blocksize': 'int',
+ '*blockcount': 'int',
'*timeout': 'int',
'*username': 'str',
'*password-secret': 'str',
--
2.27.0
next prev parent reply other threads:[~2020-08-18 11:12 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-18 11:08 [RFC PATCH 0/9] block/curl: Add caching of data downloaded from the remote server David Edmondson
2020-08-18 11:08 ` [RFC PATCH 1/9] block/curl: Add an 'offset' parameter, affecting all range requests David Edmondson
2020-08-18 11:08 ` [RFC PATCH 2/9] block/curl: Remove readahead support David Edmondson
2020-08-18 11:08 ` [RFC PATCH 3/9] block/curl: Tracing David Edmondson
2020-08-18 11:08 ` [RFC PATCH 4/9] block/curl: Perform IO in fixed size chunks David Edmondson
2020-08-18 11:08 ` [RFC PATCH 5/9] block/curl: Allow the blocksize to be specified by the user David Edmondson
2020-08-24 13:19 ` Markus Armbruster
2020-08-24 14:21 ` David Edmondson
2020-08-18 11:08 ` [RFC PATCH 6/9] block/curl: Cache downloaded blocks David Edmondson
2020-08-18 11:08 ` David Edmondson [this message]
2020-08-18 11:08 ` [RFC PATCH 8/9] block/curl: Allow 16 sockets/ACB David Edmondson
2020-08-18 11:08 ` [RFC PATCH 9/9] block/curl: Add readahead support David Edmondson
2020-08-19 14:11 ` [RFC PATCH 0/9] block/curl: Add caching of data downloaded from the remote server Stefan Hajnoczi
2020-08-19 14:19 ` David Edmondson
2020-08-25 10:03 ` Max Reitz
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200818110845.3825105-8-david.edmondson@oracle.com \
--to=david.edmondson@oracle.com \
--cc=armbru@redhat.com \
--cc=fam@euphon.net \
--cc=kwolf@redhat.com \
--cc=mreitz@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=stefanha@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).