qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: Fam Zheng <famz@redhat.com>
To: qemu-devel@nongnu.org
Cc: kwolf@redhat.com, jcody@redhat.com, Fam Zheng <famz@redhat.com>,
	rjones@redhat.com, stefanha@redhat.com
Subject: [Qemu-devel] [PATCH v6 09/12] curl: add cache quota.
Date: Fri, 24 May 2013 13:37:04 +0800	[thread overview]
Message-ID: <1369373827-9152-10-git-send-email-famz@redhat.com> (raw)
In-Reply-To: <1369373827-9152-1-git-send-email-famz@redhat.com>

Introduce a cache quota: BDRVCURLState.cache_quota.
When adding new CURLDataCache to BDRVCURLState, if number of existing
CURLDataCache is larger than CURL_CACHE_QUOTA, try to release some first
to limit the in memory cache size.

A least used entry is selected for releasing.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 block/curl.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/block/curl.c b/block/curl.c
index 9b18238..6e893d0 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -40,6 +40,7 @@
 
 #define SECTOR_SIZE     512
 #define READ_AHEAD_SIZE (256 * 1024)
+#define CURL_CACHE_QUOTA 10
 
 struct BDRVCURLState;
 
@@ -90,6 +91,8 @@ typedef struct BDRVCURLState {
     QEMUTimer *timer;
     /* List of data cache ordered by access, freed from tail */
     QLIST_HEAD(, CURLDataCache) cache;
+    /* Threshold to release unused cache when cache list is longer than it */
+    int cache_quota;
     /* Whether http server accept range in header */
     bool accept_range;
 } BDRVCURLState;
@@ -526,6 +529,8 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags)
     curl_multi_socket_action(s->multi, CURL_SOCKET_TIMEOUT, 0, &running);
 
     qemu_opts_del(opts);
+    s->cache_quota = CURL_CACHE_QUOTA;
+
     return 0;
 
 out:
@@ -595,6 +600,27 @@ static void curl_readv_bh_cb(void *p)
     cache->data_len = aio_bytes + s->readahead_size;
     cache->write_pos = 0;
     cache->data = g_malloc(cache->data_len);
+    /* Try to release some cache */
+    while (s->cache_quota <= 0) {
+        CURLDataCache *p;
+        CURLDataCache *q = NULL;
+        assert(!QLIST_EMPTY(&s->cache));
+        for (p = QLIST_FIRST(&s->cache);
+             p; p = QLIST_NEXT(p, next)) {
+            if (p->use_count == 0) {
+                q = p;
+            }
+        }
+        if (!q) {
+            break;
+        }
+        QLIST_REMOVE(q, next);
+        g_free(q->data);
+        q->data = NULL;
+        g_free(q);
+        q = NULL;
+        s->cache_quota++;
+    }
 
     QLIST_INSERT_HEAD(&s->acbs, acb, next);
     snprintf(state->range, sizeof(state->range) - 1, "%zd-%zd", cache->base_pos,
@@ -605,6 +631,7 @@ static void curl_readv_bh_cb(void *p)
     QLIST_INSERT_HEAD(&s->cache, cache, next);
     state->cache = cache;
     cache->use_count++;
+    s->cache_quota--;
     curl_multi_add_handle(s->multi, state->curl);
     /* kick off curl to start the action */
     curl_multi_socket_action(s->multi, 0, CURL_SOCKET_TIMEOUT, &running);
-- 
1.8.2.3

  parent reply	other threads:[~2013-05-24  5:38 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-05-24  5:36 [Qemu-devel] [PATCH v6 00/12] curl: fix curl read Fam Zheng
2013-05-24  5:36 ` [Qemu-devel] [PATCH v6 01/12] curl: introduce CURLSockInfo to BDRVCURLState Fam Zheng
2013-05-24  5:36 ` [Qemu-devel] [PATCH v6 02/12] curl: change magic number to sizeof Fam Zheng
2013-05-24  5:36 ` [Qemu-devel] [PATCH v6 03/12] curl: change curl_multi_do to curl_fd_handler Fam Zheng
2013-05-24  5:36 ` [Qemu-devel] [PATCH v6 04/12] curl: fix curl_open Fam Zheng
2013-05-24  5:37 ` [Qemu-devel] [PATCH v6 05/12] curl: add timer to BDRVCURLState Fam Zheng
2013-05-24  5:37 ` [Qemu-devel] [PATCH v6 06/12] curl: introduce CURLDataCache Fam Zheng
2013-05-24  5:37 ` [Qemu-devel] [PATCH v6 07/12] curl: make use of CURLDataCache Fam Zheng
2013-05-24  5:37 ` [Qemu-devel] [PATCH v6 08/12] curl: use list to store CURLState Fam Zheng
2013-05-24  5:37 ` Fam Zheng [this message]
2013-05-24  5:37 ` [Qemu-devel] [PATCH v6 10/12] curl: introduce ssl_no_cert runtime option Fam Zheng
2013-05-24  5:37 ` [Qemu-devel] [PATCH v6 11/12] block/curl.c: Refuse to open the handle for writes Fam Zheng
2013-05-24  5:37 ` [Qemu-devel] [PATCH v6 12/12] curl: set s->url to NULL after free Fam Zheng
2013-05-24  8:07 ` [Qemu-devel] [PATCH v6 00/12] curl: fix curl read Richard W.M. Jones
2013-05-27  2:25   ` Fam Zheng
2013-05-27 11:57     ` Richard W.M. Jones
2013-05-28  7:30       ` Fam Zheng
2013-05-28  7:47         ` Richard W.M. Jones
2013-05-28  8:46           ` Richard W.M. Jones
2013-05-28  8:52             ` Fam Zheng
2013-05-28  8:53             ` Richard W.M. Jones
2013-05-28 10:35 ` Richard W.M. Jones
2013-05-28 11:01   ` Richard W.M. Jones
2013-05-28 11:14     ` Fam Zheng
2013-05-28 11:25       ` Richard W.M. Jones
2013-05-28 11:32         ` Richard W.M. Jones
2013-05-29  1:07           ` Fam Zheng
2013-05-29  9:25             ` Richard W.M. Jones
2013-05-28 11:34     ` Stefan Hajnoczi

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1369373827-9152-10-git-send-email-famz@redhat.com \
    --to=famz@redhat.com \
    --cc=jcody@redhat.com \
    --cc=kwolf@redhat.com \
    --cc=qemu-devel@nongnu.org \
    --cc=rjones@redhat.com \
    --cc=stefanha@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).