[Qemu-devel] [PATCH v2 00/11] curl: fix curl read

[Qemu-devel] [PATCH v2 00/11] curl: fix curl read

[Fam Zheng]

CURL library API has changed, the current curl driver is not working.
This patch rewrites the use of API as well as the structure of internal
states. (It is hard to split this to multiple patches as basically all
these changes need to work together.)

BDRVCURLState holds the pointer to curl multi interface (man 3
libcurl-multi), and 4 lists for internal states:
 - CURLState holds state for libcurl connection (man 3 libcurl-easy)
 - CURLSockInfo holds information for libcurl socket interface (man 3
   curl_multi_socket_action).
 - CURLDataCache holds the user data read from libcurl, it is in a list
   ordered by access, the used cache is moved to list head on access, so
   the tail element is freed first. BDRVCURLState.cache_quota is the
   threshold to start freeing cache.
 - CURLAIOCB holds ongoing aio information.

Changes from v1:
   Split to multiple patches. The final status should be identical to v1.

Fam Zheng (11):
  curl: introduce CURLSockInfo to BDRVCURLState.
  curl: change magic number to macro
  curl: change curl_multi_do to curl_fd_handler
  curl: fix curl_open
  curl: add timer to BDRVCURLState
  curl: introduce CURLDataCache
  curl: make use of CURLDataCache.
  curl: use list to store CURLState
  curl: release everything on curl_close
  curl: add cache quota.
  curl: introduce ssl_no_cert runtime option.

 block/curl.c | 553 ++++++++++++++++++++++++++++++++++++-----------------------
 1 file changed, 339 insertions(+), 214 deletions(-)

-- 
1.8.1.4

[Qemu-devel] [PATCH v2 01/11] curl: introduce CURLSockInfo to BDRVCURLState.

[Fam Zheng]

We use socket provided by curl in the driver.  Libcurl multi interface
has option CURLMOPT_SOCKETFUNCTION for socket.

Per man 3 curl_multi_setopt:

    ...
    CURLMOPT_SOCKETFUNCTION

    Pass a pointer to a function matching the curl_socket_callback
    prototype. The curl_multi_socket_action(3) function informs the
    application about updates in the socket (file descriptor) status by
    doing none, one, or multiple calls to the curl_socket_callback given in
    the param argument. They update the status with changes since the
    previous time a curl_multi_socket(3) function was called. If the given
    callback pointer is NULL, no callback will be called. Set the callback's
    userp argument with CURLMOPT_SOCKETDATA. See curl_multi_socket(3) for
    more callback details.
    ...

The added structure store information for all the socket returned by
libcurl. The most important field is action, which is used to keep what
actions are needed when this socket's fd handler is called.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 block/curl.c | 21 +++++++++++++++++++--
 1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index b8935fd..14f4552 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -75,10 +75,18 @@ typedef struct CURLState
     char in_use;
 } CURLState;
 
+typedef struct CURLSockInfo {
+    curl_socket_t fd;
+    int action;
+    struct BDRVCURLState *s;
+    QLIST_ENTRY(CURLSockInfo) next;
+} CURLSockInfo;
+
 typedef struct BDRVCURLState {
     CURLM *multi;
     size_t len;
     CURLState states[CURL_NUM_STATES];
+    QLIST_HEAD(, CURLSockInfo) socks;
     char *url;
     size_t readahead_size;
 } BDRVCURLState;
@@ -90,7 +98,16 @@ static int curl_aio_flush(void *opaque);
 static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
                         void *s, void *sp)
 {
+    BDRVCURLState *bs = (BDRVCURLState *)s;
     DPRINTF("CURL (AIO): Sock action %d on fd %d\n", action, fd);
+    CURLSockInfo *sock = (CURLSockInfo *)sp;
+    if (!sp) {
+        sock = g_malloc0(sizeof(CURLSockInfo));
+        sock->fd = fd;
+        sock->s = bs;
+        QLIST_INSERT_HEAD(&bs->socks, sock, next);
+        curl_multi_assign(bs->multi, fd, sock);
+    }
     switch (action) {
         case CURL_POLL_IN:
             qemu_aio_set_fd_handler(fd, curl_multi_do, NULL, curl_aio_flush, s);
@@ -462,8 +479,8 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags)
     // initialize the multi interface!
 
     s->multi = curl_multi_init();
-    curl_multi_setopt( s->multi, CURLMOPT_SOCKETDATA, s); 
-    curl_multi_setopt( s->multi, CURLMOPT_SOCKETFUNCTION, curl_sock_cb ); 
+    curl_multi_setopt(s->multi, CURLMOPT_SOCKETDATA, s);
+    curl_multi_setopt(s->multi, CURLMOPT_SOCKETFUNCTION, curl_sock_cb);
     curl_multi_do(s);
 
     qemu_opts_del(opts);
-- 
1.8.1.4

Re: [Qemu-devel] [PATCH v2 01/11] curl: introduce CURLSockInfo to BDRVCURLState.

[Stefan Hajnoczi]

On Tue, May 14, 2013 at 10:26:20AM +0800, Fam Zheng wrote:
> We use socket provided by curl in the driver.  Libcurl multi interface
> has option CURLMOPT_SOCKETFUNCTION for socket.
> 
> Per man 3 curl_multi_setopt:
> 
>     ...
>     CURLMOPT_SOCKETFUNCTION
> 
>     Pass a pointer to a function matching the curl_socket_callback
>     prototype. The curl_multi_socket_action(3) function informs the
>     application about updates in the socket (file descriptor) status by
>     doing none, one, or multiple calls to the curl_socket_callback given in
>     the param argument. They update the status with changes since the
>     previous time a curl_multi_socket(3) function was called. If the given
>     callback pointer is NULL, no callback will be called. Set the callback's
>     userp argument with CURLMOPT_SOCKETDATA. See curl_multi_socket(3) for
>     more callback details.
>     ...
> 
> The added structure store information for all the socket returned by
> libcurl. The most important field is action, which is used to keep what
> actions are needed when this socket's fd handler is called.
> 
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
>  block/curl.c | 21 +++++++++++++++++++--
>  1 file changed, 19 insertions(+), 2 deletions(-)
> 
> diff --git a/block/curl.c b/block/curl.c
> index b8935fd..14f4552 100644
> --- a/block/curl.c
> +++ b/block/curl.c
> @@ -75,10 +75,18 @@ typedef struct CURLState
>      char in_use;
>  } CURLState;
>  
> +typedef struct CURLSockInfo {
> +    curl_socket_t fd;
> +    int action;
> +    struct BDRVCURLState *s;
> +    QLIST_ENTRY(CURLSockInfo) next;
> +} CURLSockInfo;

This patch doesn't make sense by itself.  CURLSockInfo is unused and
->action is never assigned to.  I guess this will eventually replace
BDRVCURLState->states[] but at this point in the patch series it is hard
to tell - it would be nice to explain this in the commit description.

>  typedef struct BDRVCURLState {
>      CURLM *multi;
>      size_t len;
>      CURLState states[CURL_NUM_STATES];
> +    QLIST_HEAD(, CURLSockInfo) socks;
>      char *url;
>      size_t readahead_size;
>  } BDRVCURLState;
> @@ -90,7 +98,16 @@ static int curl_aio_flush(void *opaque);
>  static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
>                          void *s, void *sp)
>  {
> +    BDRVCURLState *bs = (BDRVCURLState *)s;

void * is automatically coerced to any pointer type.  No need for a type
cast.

>      DPRINTF("CURL (AIO): Sock action %d on fd %d\n", action, fd);
> +    CURLSockInfo *sock = (CURLSockInfo *)sp;

Same here.

> +    if (!sp) {
> +        sock = g_malloc0(sizeof(CURLSockInfo));
> +        sock->fd = fd;
> +        sock->s = bs;
> +        QLIST_INSERT_HEAD(&bs->socks, sock, next);
> +        curl_multi_assign(bs->multi, fd, sock);
> +    }
>      switch (action) {
>          case CURL_POLL_IN:
>              qemu_aio_set_fd_handler(fd, curl_multi_do, NULL, curl_aio_flush, s);
> @@ -462,8 +479,8 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags)
>      // initialize the multi interface!
>  
>      s->multi = curl_multi_init();
> -    curl_multi_setopt( s->multi, CURLMOPT_SOCKETDATA, s); 
> -    curl_multi_setopt( s->multi, CURLMOPT_SOCKETFUNCTION, curl_sock_cb ); 
> +    curl_multi_setopt(s->multi, CURLMOPT_SOCKETDATA, s);
> +    curl_multi_setopt(s->multi, CURLMOPT_SOCKETFUNCTION, curl_sock_cb);
>      curl_multi_do(s);
>  
>      qemu_opts_del(opts);
> -- 
> 1.8.1.4
> 
> 

[Qemu-devel] [PATCH v2 02/11] curl: change magic number to macro

[Fam Zheng]

String field length is duplicated in two places. Make it a macro.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 block/curl.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index 14f4552..aa78086 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -70,7 +70,8 @@ typedef struct CURLState
     size_t buf_start;
     size_t buf_off;
     size_t buf_len;
-    char range[128];
+#define CURL_RANGE_SIZE 128
+    char range[CURL_RANGE_SIZE];
     char errmsg[CURL_ERROR_SIZE];
     char in_use;
 } CURLState;
@@ -567,7 +568,7 @@ static void curl_readv_bh_cb(void *p)
     state->orig_buf = g_malloc(state->buf_len);
     state->acb[0] = acb;
 
-    snprintf(state->range, 127, "%zd-%zd", start, end);
+    snprintf(state->range, CURL_RANGE_SIZE - 1, "%zd-%zd", start, end);
     DPRINTF("CURL (AIO): Reading %d at %zd (%s)\n",
             (acb->nb_sectors * SECTOR_SIZE), start, state->range);
     curl_easy_setopt(state->curl, CURLOPT_RANGE, state->range);
-- 
1.8.1.4

Re: [Qemu-devel] [PATCH v2 02/11] curl: change magic number to macro

[Stefan Hajnoczi]

On Tue, May 14, 2013 at 10:26:21AM +0800, Fam Zheng wrote:
> @@ -70,7 +70,8 @@ typedef struct CURLState
>      size_t buf_start;
>      size_t buf_off;
>      size_t buf_len;
> -    char range[128];
> +#define CURL_RANGE_SIZE 128
> +    char range[CURL_RANGE_SIZE];
>      char errmsg[CURL_ERROR_SIZE];
>      char in_use;
>  } CURLState;
> @@ -567,7 +568,7 @@ static void curl_readv_bh_cb(void *p)
>      state->orig_buf = g_malloc(state->buf_len);
>      state->acb[0] = acb;
>  
> -    snprintf(state->range, 127, "%zd-%zd", start, end);
> +    snprintf(state->range, CURL_RANGE_SIZE - 1, "%zd-%zd", start, end);

Simpler solution: sizeof(state->range) - 1

Then you don't need to introduce "CURL_RANGE_SIZE" which risks
conflicting with libcurl's namespace.

[Qemu-devel] [PATCH v2 03/11] curl: change curl_multi_do to curl_fd_handler

[Fam Zheng]

The driver calls curl_multi_do to take action at several points, while
it's also registered as socket fd handler. This patch removes internal
call of curl_multi_do because they are not necessary when handler can be
called by socket data update.

Since curl_multi_do becomes a pure fd handler, the function is renamed.
It takes a pointer to CURLSockInfo now, instead of pointer to
BDRVCURLState.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 block/curl.c | 28 +++++++++++++++++-----------
 1 file changed, 17 insertions(+), 11 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index aa78086..9a8019d 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -93,7 +93,7 @@ typedef struct BDRVCURLState {
 } BDRVCURLState;
 
 static void curl_clean_state(CURLState *s);
-static void curl_multi_do(void *arg);
+static void curl_fd_handler(void *arg);
 static int curl_aio_flush(void *opaque);
 
 static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
@@ -111,17 +111,23 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
     }
     switch (action) {
         case CURL_POLL_IN:
-            qemu_aio_set_fd_handler(fd, curl_multi_do, NULL, curl_aio_flush, s);
+            qemu_aio_set_fd_handler(fd, curl_fd_handler, NULL,
+                                    curl_aio_flush, sock);
+            sock->action |= CURL_CSELECT_IN;
             break;
         case CURL_POLL_OUT:
-            qemu_aio_set_fd_handler(fd, NULL, curl_multi_do, curl_aio_flush, s);
+            qemu_aio_set_fd_handler(fd, NULL, curl_fd_handler, curl_aio_flush,
+                                    sock);
+            sock->action |= CURL_CSELECT_OUT;
             break;
         case CURL_POLL_INOUT:
-            qemu_aio_set_fd_handler(fd, curl_multi_do, curl_multi_do,
-                                    curl_aio_flush, s);
+            qemu_aio_set_fd_handler(fd, curl_fd_handler, curl_fd_handler,
+                                    curl_aio_flush, sock);
+            sock->action |= CURL_CSELECT_IN | CURL_CSELECT_OUT;
             break;
         case CURL_POLL_REMOVE:
             qemu_aio_set_fd_handler(fd, NULL, NULL, NULL, NULL);
+            sock->action = 0;
             break;
     }
 
@@ -227,9 +233,10 @@ static int curl_find_buf(BDRVCURLState *s, size_t start, size_t len,
     return FIND_RET_NONE;
 }
 
-static void curl_multi_do(void *arg)
+static void curl_fd_handler(void *arg)
 {
-    BDRVCURLState *s = (BDRVCURLState *)arg;
+    CURLSockInfo *sock = (CURLSockInfo *)arg;
+    BDRVCURLState *s = sock->s;
     int running;
     int r;
     int msgs_in_queue;
@@ -238,7 +245,9 @@ static void curl_multi_do(void *arg)
         return;
 
     do {
-        r = curl_multi_socket_all(s->multi, &running);
+        r = curl_multi_socket_action(s->multi,
+                sock->fd, sock->action,
+                &running);
     } while(r == CURLM_CALL_MULTI_PERFORM);
 
     /* Try to find done transfers, so we can free the easy
@@ -303,7 +312,6 @@ static CURLState *curl_init_state(BDRVCURLState *s)
         }
         if (!state) {
             g_usleep(100);
-            curl_multi_do(s);
         }
     } while(!state);
 
@@ -482,7 +490,6 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags)
     s->multi = curl_multi_init();
     curl_multi_setopt(s->multi, CURLMOPT_SOCKETDATA, s);
     curl_multi_setopt(s->multi, CURLMOPT_SOCKETFUNCTION, curl_sock_cb);
-    curl_multi_do(s);
 
     qemu_opts_del(opts);
     return 0;
@@ -574,7 +581,6 @@ static void curl_readv_bh_cb(void *p)
     curl_easy_setopt(state->curl, CURLOPT_RANGE, state->range);
 
     curl_multi_add_handle(s->multi, state->curl);
-    curl_multi_do(s);
 
 }
 
-- 
1.8.1.4

[Qemu-devel] [PATCH v2 04/11] curl: fix curl_open

[Fam Zheng]

Change curl_size_cb to curl_header_cb, as what the function is really
doing. Fix the registering, CURLOPT_WRITEFUNCTION is apparently wrong,
should be CURLOPT_HEADERFUNCTION.

Parsing size from header is not necessary as we're using

  curl_easy_getinfo(state->curl, CURLINFO_CONTENT_LENGTH_DOWNLOAD, &d)

The validity of d is version dependant per man 3 curl_easy_getinfo:

    ...
    CURLINFO_CONTENT_LENGTH_DOWNLOAD

    Pass a pointer to a double to receive the content-length of the
    download. This is the value read from the Content-Length: field.
    Since 7.19.4, this returns -1 if the size isn't known.
    ...

And Accept-Ranges is essential for curl to fetch right data from
http/https servers, so we check for this in the header and fail the open
if it's not supported.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 block/curl.c | 42 ++++++++++++++++++++++++++++++------------
 1 file changed, 30 insertions(+), 12 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index 9a8019d..f47796b 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -90,6 +90,8 @@ typedef struct BDRVCURLState {
     QLIST_HEAD(, CURLSockInfo) socks;
     char *url;
     size_t readahead_size;
+    /* Whether http server accept range in header */
+    bool accept_range;
 } BDRVCURLState;
 
 static void curl_clean_state(CURLState *s);
@@ -134,14 +136,14 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
     return 0;
 }
 
-static size_t curl_size_cb(void *ptr, size_t size, size_t nmemb, void *opaque)
+static size_t curl_header_cb(void *ptr, size_t size, size_t nmemb, void *opaque)
 {
-    CURLState *s = ((CURLState*)opaque);
+    BDRVCURLState *s = (BDRVCURLState *)opaque;
     size_t realsize = size * nmemb;
-    size_t fsize;
+    const char *accept_line = "Accept-Ranges: bytes";
 
-    if(sscanf(ptr, "Content-Length: %zd", &fsize) == 1) {
-        s->s->len = fsize;
+    if (strncmp((char *)ptr, accept_line, strlen(accept_line)) == 0) {
+        s->accept_range = true;
     }
 
     return realsize;
@@ -429,6 +431,7 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags)
     Error *local_err = NULL;
     const char *file;
     double d;
+    int running;
 
     static int inited = 0;
 
@@ -468,16 +471,27 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags)
     // Get file size
 
     curl_easy_setopt(state->curl, CURLOPT_NOBODY, 1);
-    curl_easy_setopt(state->curl, CURLOPT_WRITEFUNCTION, (void *)curl_size_cb);
-    if (curl_easy_perform(state->curl))
+    curl_easy_setopt(state->curl, CURLOPT_HEADERFUNCTION,
+                     curl_header_cb);
+    curl_easy_setopt(state->curl, CURLOPT_HEADERDATA, s);
+    if (curl_easy_perform(state->curl)) {
         goto out;
+    }
     curl_easy_getinfo(state->curl, CURLINFO_CONTENT_LENGTH_DOWNLOAD, &d);
-    curl_easy_setopt(state->curl, CURLOPT_WRITEFUNCTION, (void *)curl_read_cb);
     curl_easy_setopt(state->curl, CURLOPT_NOBODY, 0);
-    if (d)
+#if LIBCURL_VERSION_NUM > 0x071304
+    if (d != -1) {
+#else
+    if (d) {
+#endif
         s->len = (size_t)d;
-    else if(!s->len)
+    } else if (!s->len) {
+        goto out;
+    }
+    if (!strncmp(s->url, "http://", strlen("http://")) && !s->accept_range) {
+        strncpy(state->errmsg, "Server not supporting range.", CURL_ERROR_SIZE);
         goto out;
+    }
     DPRINTF("CURL: Size = %zd\n", s->len);
 
     curl_clean_state(state);
@@ -486,10 +500,13 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags)
 
     // Now we know the file exists and its size, so let's
     // initialize the multi interface!
-
     s->multi = curl_multi_init();
+    if (!s->multi) {
+        goto out_noclean;
+    }
     curl_multi_setopt(s->multi, CURLMOPT_SOCKETDATA, s);
     curl_multi_setopt(s->multi, CURLMOPT_SOCKETFUNCTION, curl_sock_cb);
+    curl_multi_socket_action(s->multi, CURL_SOCKET_TIMEOUT, 0, &running);
 
     qemu_opts_del(opts);
     return 0;
@@ -499,8 +516,9 @@ out:
     curl_easy_cleanup(state->curl);
     state->curl = NULL;
 out_noclean:
-    g_free(s->url);
     qemu_opts_del(opts);
+    g_free(s->url);
+    s->url = NULL;
     return -EINVAL;
 }
 
-- 
1.8.1.4

Re: [Qemu-devel] [PATCH v2 04/11] curl: fix curl_open

[Stefan Hajnoczi]

On Tue, May 14, 2013 at 10:26:23AM +0800, Fam Zheng wrote:
> +    if (!strncmp(s->url, "http://", strlen("http://")) && !s->accept_range) {
> +        strncpy(state->errmsg, "Server not supporting range.", CURL_ERROR_SIZE);

Please use pstrcpy(), it always NUL-terminates.  strncpy(3) does not
when the buffer size is reached.

Also, what happens when the URL is given in uppercase?
HTTP://GOOGLE.COM/

[Qemu-devel] [PATCH v2 05/11] curl: add timer to BDRVCURLState

[Fam Zheng]

libcurl uses timer to manage ongoing sockets, it needs us to supply
timer. This patch introduce QEMUTimer to BDRVCURLState and handles
timeouts as libcurl expects (curl_multi_timer_cb sets given timeout
value on the timer and curl_timer_cb calls curl_multi_socket_action on
triggered).

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 block/curl.c | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/block/curl.c b/block/curl.c
index f47796b..61d0d6f 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -90,6 +90,7 @@ typedef struct BDRVCURLState {
     QLIST_HEAD(, CURLSockInfo) socks;
     char *url;
     size_t readahead_size;
+    QEMUTimer *timer;
     /* Whether http server accept range in header */
     bool accept_range;
 } BDRVCURLState;
@@ -149,6 +150,38 @@ static size_t curl_header_cb(void *ptr, size_t size, size_t nmemb, void *opaque)
     return realsize;
 }
 
+static void curl_timer_cb(void *opaque)
+{
+    int running;
+    BDRVCURLState *bs = (BDRVCURLState *)opaque;
+    DPRINTF("curl timeout!\n");
+    curl_multi_socket_action(bs->multi, CURL_SOCKET_TIMEOUT, 0, &running);
+}
+
+/* Call back for curl_multi interface */
+static int curl_multi_timer_cb(CURLM *multi, long timeout_ms, void *s)
+{
+    BDRVCURLState *bs = (BDRVCURLState *)s;
+    DPRINTF("curl multi timer cb, timeout: %ld (ms)\n", timeout_ms);
+    if (timeout_ms < 0) {
+        if (bs->timer) {
+            qemu_del_timer(bs->timer);
+            qemu_free_timer(bs->timer);
+            bs->timer = NULL;
+        }
+    } else if (timeout_ms == 0) {
+        curl_timer_cb(bs);
+    } else {
+        if (!bs->timer) {
+            bs->timer = qemu_new_timer_ms(host_clock, curl_timer_cb, s);
+            assert(bs->timer);
+        }
+        qemu_mod_timer(bs->timer, qemu_get_clock_ms(host_clock) + timeout_ms);
+    }
+
+    return 0;
+}
+
 static size_t curl_read_cb(void *ptr, size_t size, size_t nmemb, void *opaque)
 {
     CURLState *s = ((CURLState*)opaque);
@@ -506,6 +539,8 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags)
     }
     curl_multi_setopt(s->multi, CURLMOPT_SOCKETDATA, s);
     curl_multi_setopt(s->multi, CURLMOPT_SOCKETFUNCTION, curl_sock_cb);
+    curl_multi_setopt(s->multi, CURLMOPT_TIMERDATA, s);
+    curl_multi_setopt(s->multi, CURLMOPT_TIMERFUNCTION, curl_multi_timer_cb);
     curl_multi_socket_action(s->multi, CURL_SOCKET_TIMEOUT, 0, &running);
 
     qemu_opts_del(opts);
-- 
1.8.1.4

[Qemu-devel] [PATCH v2 06/11] curl: introduce CURLDataCache

[Fam Zheng]

Data buffer was contained by CURLState, they are allocated and freed
together. This patch try to isolate them, by introducing a dedicated
cache list to BDRVCURLState. The benifit is we can now release the
CURLState (and associated sockets) while keep the fetched data for later
use, and simplies the prefetch and buffer logic for some degree.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 block/curl.c | 123 ++++++++++++++++++++++++++---------------------------------
 1 file changed, 54 insertions(+), 69 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index 61d0d6f..0c4d865 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -43,10 +43,6 @@
 #define SECTOR_SIZE     512
 #define READ_AHEAD_SIZE (256 * 1024)
 
-#define FIND_RET_NONE   0
-#define FIND_RET_OK     1
-#define FIND_RET_WAIT   2
-
 struct BDRVCURLState;
 
 typedef struct CURLAIOCB {
@@ -61,6 +57,16 @@ typedef struct CURLAIOCB {
     size_t end;
 } CURLAIOCB;
 
+typedef struct CURLDataCache {
+    char *data;
+    size_t base_pos;
+    size_t data_len;
+    size_t write_pos;
+    /* Ref count for CURLState */
+    int use_count;
+    QLIST_ENTRY(CURLDataCache) next;
+} CURLDataCache;
+
 typedef struct CURLState
 {
     struct BDRVCURLState *s;
@@ -91,6 +97,8 @@ typedef struct BDRVCURLState {
     char *url;
     size_t readahead_size;
     QEMUTimer *timer;
+    /* List of data cache ordered by access, freed from tail */
+    QLIST_HEAD(, CURLDataCache) cache;
     /* Whether http server accept range in header */
     bool accept_range;
 } BDRVCURLState;
@@ -99,6 +107,19 @@ static void curl_clean_state(CURLState *s);
 static void curl_fd_handler(void *arg);
 static int curl_aio_flush(void *opaque);
 
+static CURLDataCache *curl_find_cache(BDRVCURLState *bs,
+                                      size_t start, size_t len)
+{
+    CURLDataCache *c;
+    QLIST_FOREACH(c, &bs->cache, next) {
+        if (start >= c->base_pos &&
+            start + len <= c->base_pos + c->write_pos) {
+            return c;
+        }
+    }
+    return NULL;
+}
+
 static int curl_sock_cb(CURL *curl, curl_socket_t fd, int action,
                         void *s, void *sp)
 {
@@ -182,6 +203,23 @@ static int curl_multi_timer_cb(CURLM *multi, long timeout_ms, void *s)
     return 0;
 }
 
+static void curl_complete_io(BDRVCURLState *bs, CURLAIOCB *acb,
+                             CURLDataCache *cache)
+{
+    size_t aio_base = acb->sector_num * SECTOR_SIZE;
+    size_t aio_bytes = acb->nb_sectors * SECTOR_SIZE;
+    size_t off = aio_base - cache->base_pos;
+
+    qemu_iovec_from_buf(acb->qiov, 0, cache->data + off, aio_bytes);
+    acb->common.cb(acb->common.opaque, 0);
+    DPRINTF("AIO Request OK: %10zd %10zd\n", aio_base, aio_bytes);
+    qemu_aio_release(acb);
+    acb = NULL;
+    /* Move cache next in the list */
+    QLIST_REMOVE(cache, next);
+    QLIST_INSERT_HEAD(&bs->cache, cache, next);
+}
+
 static size_t curl_read_cb(void *ptr, size_t size, size_t nmemb, void *opaque)
 {
     CURLState *s = ((CURLState*)opaque);
@@ -215,59 +253,6 @@ read_end:
     return realsize;
 }
 
-static int curl_find_buf(BDRVCURLState *s, size_t start, size_t len,
-                         CURLAIOCB *acb)
-{
-    int i;
-    size_t end = start + len;
-
-    for (i=0; i<CURL_NUM_STATES; i++) {
-        CURLState *state = &s->states[i];
-        size_t buf_end = (state->buf_start + state->buf_off);
-        size_t buf_fend = (state->buf_start + state->buf_len);
-
-        if (!state->orig_buf)
-            continue;
-        if (!state->buf_off)
-            continue;
-
-        // Does the existing buffer cover our section?
-        if ((start >= state->buf_start) &&
-            (start <= buf_end) &&
-            (end >= state->buf_start) &&
-            (end <= buf_end))
-        {
-            char *buf = state->orig_buf + (start - state->buf_start);
-
-            qemu_iovec_from_buf(acb->qiov, 0, buf, len);
-            acb->common.cb(acb->common.opaque, 0);
-
-            return FIND_RET_OK;
-        }
-
-        // Wait for unfinished chunks
-        if ((start >= state->buf_start) &&
-            (start <= buf_fend) &&
-            (end >= state->buf_start) &&
-            (end <= buf_fend))
-        {
-            int j;
-
-            acb->start = start - state->buf_start;
-            acb->end = acb->start + len;
-
-            for (j=0; j<CURL_NUM_ACB; j++) {
-                if (!state->acb[j]) {
-                    state->acb[j] = acb;
-                    return FIND_RET_WAIT;
-                }
-            }
-        }
-    }
-
-    return FIND_RET_NONE;
-}
-
 static void curl_fd_handler(void *arg)
 {
     CURLSockInfo *sock = (CURLSockInfo *)arg;
@@ -300,7 +285,9 @@ static void curl_fd_handler(void *arg)
             case CURLMSG_DONE:
             {
                 CURLState *state = NULL;
-                curl_easy_getinfo(msg->easy_handle, CURLINFO_PRIVATE, (char**)&state);
+                curl_easy_getinfo(msg->easy_handle,
+                                  CURLINFO_PRIVATE,
+                                  (char **)&state);
 
                 /* ACBs for successful messages get completed in curl_read_cb */
                 if (msg->data.result != CURLE_OK) {
@@ -586,26 +573,24 @@ static const AIOCBInfo curl_aiocb_info = {
 static void curl_readv_bh_cb(void *p)
 {
     CURLState *state;
-
+    CURLDataCache *cache = NULL;
     CURLAIOCB *acb = p;
     BDRVCURLState *s = acb->common.bs->opaque;
+    size_t aio_base, aio_bytes;
 
     qemu_bh_delete(acb->bh);
     acb->bh = NULL;
 
+    aio_base = acb->sector_num * SECTOR_SIZE;
+    aio_bytes = acb->nb_sectors * SECTOR_SIZE;
+
     size_t start = acb->sector_num * SECTOR_SIZE;
     size_t end;
 
-    // In case we have the requested data already (e.g. read-ahead),
-    // we can just call the callback and be done.
-    switch (curl_find_buf(s, start, acb->nb_sectors * SECTOR_SIZE, acb)) {
-        case FIND_RET_OK:
-            qemu_aio_release(acb);
-            // fall through
-        case FIND_RET_WAIT:
-            return;
-        default:
-            break;
+    cache = curl_find_cache(s, aio_base, aio_bytes);
+    if (cache) {
+        curl_complete_io(s, acb, cache);
+        return;
     }
 
     // No cache found, so let's start a new request
-- 
1.8.1.4

[Qemu-devel] [PATCH v2 07/11] curl: make use of CURLDataCache.

[Fam Zheng]

Make subsequecial changes to make use of introduced CURLDataCache. Moved
acb struct from CURLState to BDRVCURLState, and changed to list.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 block/curl.c | 156 +++++++++++++++++++++++++++++------------------------------
 1 file changed, 78 insertions(+), 78 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index 0c4d865..97642e3 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -39,7 +39,6 @@
                    CURLPROTO_TFTP)
 
 #define CURL_NUM_STATES 8
-#define CURL_NUM_ACB    8
 #define SECTOR_SIZE     512
 #define READ_AHEAD_SIZE (256 * 1024)
 
@@ -52,9 +51,7 @@ typedef struct CURLAIOCB {
 
     int64_t sector_num;
     int nb_sectors;
-
-    size_t start;
-    size_t end;
+    QLIST_ENTRY(CURLAIOCB) next;
 } CURLAIOCB;
 
 typedef struct CURLDataCache {
@@ -70,15 +67,11 @@ typedef struct CURLDataCache {
 typedef struct CURLState
 {
     struct BDRVCURLState *s;
-    CURLAIOCB *acb[CURL_NUM_ACB];
     CURL *curl;
-    char *orig_buf;
-    size_t buf_start;
-    size_t buf_off;
-    size_t buf_len;
 #define CURL_RANGE_SIZE 128
     char range[CURL_RANGE_SIZE];
     char errmsg[CURL_ERROR_SIZE];
+    CURLDataCache *cache;
     char in_use;
 } CURLState;
 
@@ -93,6 +86,7 @@ typedef struct BDRVCURLState {
     CURLM *multi;
     size_t len;
     CURLState states[CURL_NUM_STATES];
+    QLIST_HEAD(, CURLAIOCB) acbs;
     QLIST_HEAD(, CURLSockInfo) socks;
     char *url;
     size_t readahead_size;
@@ -222,30 +216,31 @@ static void curl_complete_io(BDRVCURLState *bs, CURLAIOCB *acb,
 
 static size_t curl_read_cb(void *ptr, size_t size, size_t nmemb, void *opaque)
 {
-    CURLState *s = ((CURLState*)opaque);
+    CURLState *s = (CURLState *)opaque;
+    CURLDataCache *c = s->cache;
     size_t realsize = size * nmemb;
-    int i;
-
-    DPRINTF("CURL: Just reading %zd bytes\n", realsize);
+    CURLAIOCB *acb;
 
-    if (!s || !s->orig_buf)
+    if (!c || !c->data) {
         goto read_end;
+    }
+    if (c->write_pos >= c->data_len) {
+        goto read_end;
+    }
+    memcpy(c->data + c->write_pos, ptr,
+           MIN(realsize, c->data_len - c->write_pos));
+    c->write_pos += realsize;
+    if (c->write_pos >= c->data_len) {
+        c->write_pos = c->data_len;
+    }
 
-    memcpy(s->orig_buf + s->buf_off, ptr, realsize);
-    s->buf_off += realsize;
-
-    for(i=0; i<CURL_NUM_ACB; i++) {
-        CURLAIOCB *acb = s->acb[i];
-
-        if (!acb)
-            continue;
-
-        if ((s->buf_off >= acb->end)) {
-            qemu_iovec_from_buf(acb->qiov, 0, s->orig_buf + acb->start,
-                                acb->end - acb->start);
-            acb->common.cb(acb->common.opaque, 0);
-            qemu_aio_release(acb);
-            s->acb[i] = NULL;
+    QLIST_FOREACH(acb, &s->s->acbs, next) {
+        size_t aio_base = acb->sector_num * SECTOR_SIZE;
+        size_t aio_len = acb->nb_sectors * SECTOR_SIZE;
+        if (aio_base >= c->base_pos &&
+            aio_base + aio_len <= c->base_pos + c->write_pos) {
+            QLIST_REMOVE(acb, next);
+            curl_complete_io(s->s, acb, c);
         }
     }
 
@@ -276,10 +271,12 @@ static void curl_fd_handler(void *arg)
         CURLMsg *msg;
         msg = curl_multi_info_read(s->multi, &msgs_in_queue);
 
-        if (!msg)
+        if (!msg) {
             break;
-        if (msg->msg == CURLMSG_NONE)
+        }
+        if (msg->msg == CURLMSG_NONE) {
             break;
+        }
 
         switch (msg->msg) {
             case CURLMSG_DONE:
@@ -289,19 +286,17 @@ static void curl_fd_handler(void *arg)
                                   CURLINFO_PRIVATE,
                                   (char **)&state);
 
-                /* ACBs for successful messages get completed in curl_read_cb */
+                /* ACBs for successful messages get completed in curl_read_cb,
+                 * fail existing acbs for now */
                 if (msg->data.result != CURLE_OK) {
-                    int i;
-                    for (i = 0; i < CURL_NUM_ACB; i++) {
-                        CURLAIOCB *acb = state->acb[i];
-
-                        if (acb == NULL) {
-                            continue;
-                        }
-
+                    CURLAIOCB *acb = QLIST_FIRST(&s->acbs);
+                    while (acb) {
+                        CURLAIOCB *next = QLIST_NEXT(acb, next);
+                        DPRINTF("EIO, %s\n", state->errmsg);
                         acb->common.cb(acb->common.opaque, -EIO);
+                        QLIST_REMOVE(acb, next);
                         qemu_aio_release(acb);
-                        state->acb[i] = NULL;
+                        acb = next;
                     }
                 }
 
@@ -318,13 +313,10 @@ static void curl_fd_handler(void *arg)
 static CURLState *curl_init_state(BDRVCURLState *s)
 {
     CURLState *state = NULL;
-    int i, j;
+    int i;
 
     do {
         for (i=0; i<CURL_NUM_STATES; i++) {
-            for (j=0; j<CURL_NUM_ACB; j++)
-                if (s->states[i].acb[j])
-                    continue;
             if (s->states[i].in_use)
                 continue;
 
@@ -381,6 +373,10 @@ static void curl_clean_state(CURLState *s)
     if (s->s->multi)
         curl_multi_remove_handle(s->s->multi, s->curl);
     s->in_use = 0;
+    if (s->cache) {
+        s->cache->use_count--;
+        assert(s->cache->use_count >= 0);
+    }
 }
 
 static void curl_parse_filename(const char *filename, QDict *options,
@@ -547,14 +543,8 @@ out_noclean:
 static int curl_aio_flush(void *opaque)
 {
     BDRVCURLState *s = opaque;
-    int i, j;
-
-    for (i=0; i < CURL_NUM_STATES; i++) {
-        for(j=0; j < CURL_NUM_ACB; j++) {
-            if (s->states[i].acb[j]) {
-                return 1;
-            }
-        }
+    if (!QLIST_EMPTY(&s->acbs)) {
+        return 1;
     }
     return 0;
 }
@@ -576,6 +566,7 @@ static void curl_readv_bh_cb(void *p)
     CURLDataCache *cache = NULL;
     CURLAIOCB *acb = p;
     BDRVCURLState *s = acb->common.bs->opaque;
+    int running;
     size_t aio_base, aio_bytes;
 
     qemu_bh_delete(acb->bh);
@@ -584,8 +575,9 @@ static void curl_readv_bh_cb(void *p)
     aio_base = acb->sector_num * SECTOR_SIZE;
     aio_bytes = acb->nb_sectors * SECTOR_SIZE;
 
-    size_t start = acb->sector_num * SECTOR_SIZE;
-    size_t end;
+    if (aio_base + aio_bytes > s->len) {
+        goto err_release;
+    }
 
     cache = curl_find_cache(s, aio_base, aio_bytes);
     if (cache) {
@@ -596,29 +588,41 @@ static void curl_readv_bh_cb(void *p)
     // No cache found, so let's start a new request
     state = curl_init_state(s);
     if (!state) {
-        acb->common.cb(acb->common.opaque, -EIO);
-        qemu_aio_release(acb);
-        return;
+        goto err_release;
     }
 
-    acb->start = 0;
-    acb->end = (acb->nb_sectors * SECTOR_SIZE);
-
-    state->buf_off = 0;
-    if (state->orig_buf)
-        g_free(state->orig_buf);
-    state->buf_start = start;
-    state->buf_len = acb->end + s->readahead_size;
-    end = MIN(start + state->buf_len, s->len) - 1;
-    state->orig_buf = g_malloc(state->buf_len);
-    state->acb[0] = acb;
-
-    snprintf(state->range, CURL_RANGE_SIZE - 1, "%zd-%zd", start, end);
-    DPRINTF("CURL (AIO): Reading %d at %zd (%s)\n",
-            (acb->nb_sectors * SECTOR_SIZE), start, state->range);
-    curl_easy_setopt(state->curl, CURLOPT_RANGE, state->range);
+    cache = g_malloc0(sizeof(CURLDataCache));
+    cache->base_pos = acb->sector_num * SECTOR_SIZE;
+    cache->data_len = aio_bytes + s->readahead_size;
+    cache->write_pos = 0;
+    cache->data = g_malloc(cache->data_len);
 
+    QLIST_INSERT_HEAD(&s->acbs, acb, next);
+    snprintf(state->range, CURL_RANGE_SIZE - 1, "%zd-%zd", cache->base_pos,
+             cache->base_pos + cache->data_len);
+    DPRINTF("Reading range: %s\n", state->range);
+    curl_easy_setopt(state->curl, CURLOPT_RANGE, state->range);
+    QLIST_INSERT_HEAD(&s->cache, cache, next);
+    state->cache = cache;
+    cache->use_count++;
     curl_multi_add_handle(s->multi, state->curl);
+    /* kick off curl to start the action */
+    curl_multi_socket_action(s->multi, 0, CURL_SOCKET_TIMEOUT, &running);
+    return;
+
+err_release:
+    if (cache) {
+        if (cache->data) {
+            g_free(cache->data);
+            cache->data = NULL;
+        }
+        g_free(cache);
+        cache = NULL;
+    }
+    acb->common.cb(acb->common.opaque, -EIO);
+    qemu_aio_release(acb);
+    return;
+
 
 }
 
@@ -658,10 +662,6 @@ static void curl_close(BlockDriverState *bs)
             curl_easy_cleanup(s->states[i].curl);
             s->states[i].curl = NULL;
         }
-        if (s->states[i].orig_buf) {
-            g_free(s->states[i].orig_buf);
-            s->states[i].orig_buf = NULL;
-        }
     }
     if (s->multi)
         curl_multi_cleanup(s->multi);
-- 
1.8.1.4

[Qemu-devel] [PATCH v2 08/11] curl: use list to store CURLState

[Fam Zheng]

Make it consistent to other structures to use QLIST to store CURLState.
It also simplifies initialization and releasing of data.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 block/curl.c | 79 +++++++++++++++++++++++++++---------------------------------
 1 file changed, 35 insertions(+), 44 deletions(-)

diff --git a/block/curl.c b/block/curl.c
index 97642e3..f20d81c 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -38,7 +38,6 @@
                    CURLPROTO_FTP | CURLPROTO_FTPS | \
                    CURLPROTO_TFTP)
 
-#define CURL_NUM_STATES 8
 #define SECTOR_SIZE     512
 #define READ_AHEAD_SIZE (256 * 1024)
 
@@ -64,15 +63,14 @@ typedef struct CURLDataCache {
     QLIST_ENTRY(CURLDataCache) next;
 } CURLDataCache;
 
-typedef struct CURLState
-{
+typedef struct CURLState {
     struct BDRVCURLState *s;
     CURL *curl;
 #define CURL_RANGE_SIZE 128
     char range[CURL_RANGE_SIZE];
     char errmsg[CURL_ERROR_SIZE];
     CURLDataCache *cache;
-    char in_use;
+    QLIST_ENTRY(CURLState) next;
 } CURLState;
 
 typedef struct CURLSockInfo {
@@ -85,7 +83,7 @@ typedef struct CURLSockInfo {
 typedef struct BDRVCURLState {
     CURLM *multi;
     size_t len;
-    CURLState states[CURL_NUM_STATES];
+    QLIST_HEAD(, CURLState) curl_states;
     QLIST_HEAD(, CURLAIOCB) acbs;
     QLIST_HEAD(, CURLSockInfo) socks;
     char *url;
@@ -301,6 +299,9 @@ static void curl_fd_handler(void *arg)
                 }
 
                 curl_clean_state(state);
+                QLIST_REMOVE(state, next);
+                g_free(state);
+                state = NULL;
                 break;
             }
             default:
@@ -312,29 +313,17 @@ static void curl_fd_handler(void *arg)
 
 static CURLState *curl_init_state(BDRVCURLState *s)
 {
-    CURLState *state = NULL;
-    int i;
-
-    do {
-        for (i=0; i<CURL_NUM_STATES; i++) {
-            if (s->states[i].in_use)
-                continue;
-
-            state = &s->states[i];
-            state->in_use = 1;
-            break;
-        }
-        if (!state) {
-            g_usleep(100);
-        }
-    } while(!state);
-
-    if (state->curl)
-        goto has_curl;
+    CURLState *state;
 
+    state = g_malloc0(sizeof(CURLState));
+    state->s = s;
     state->curl = curl_easy_init();
-    if (!state->curl)
-        return NULL;
+    if (!state->curl) {
+        DPRINTF("CURL: curl_easy_init failed\n");
+        g_free(state);
+        state = NULL;
+        goto out;
+    }
     curl_easy_setopt(state->curl, CURLOPT_URL, s->url);
     curl_easy_setopt(state->curl, CURLOPT_TIMEOUT, 5);
     curl_easy_setopt(state->curl, CURLOPT_WRITEFUNCTION, (void *)curl_read_cb);
@@ -360,19 +349,19 @@ static CURLState *curl_init_state(BDRVCURLState *s)
 #ifdef DEBUG_VERBOSE
     curl_easy_setopt(state->curl, CURLOPT_VERBOSE, 1);
 #endif
-
-has_curl:
-
-    state->s = s;
-
+out:
     return state;
 }
 
 static void curl_clean_state(CURLState *s)
 {
-    if (s->s->multi)
-        curl_multi_remove_handle(s->s->multi, s->curl);
-    s->in_use = 0;
+    if (s->curl) {
+        if (s->s->multi) {
+            curl_multi_remove_handle(s->s->multi, s->curl);
+        }
+        curl_easy_cleanup(s->curl);
+        s->curl = NULL;
+    }
     if (s->cache) {
         s->cache->use_count--;
         assert(s->cache->use_count >= 0);
@@ -512,7 +501,8 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags)
 
     curl_clean_state(state);
     curl_easy_cleanup(state->curl);
-    state->curl = NULL;
+    g_free(state);
+    state = NULL;
 
     // Now we know the file exists and its size, so let's
     // initialize the multi interface!
@@ -602,6 +592,7 @@ static void curl_readv_bh_cb(void *p)
              cache->base_pos + cache->data_len);
     DPRINTF("Reading range: %s\n", state->range);
     curl_easy_setopt(state->curl, CURLOPT_RANGE, state->range);
+    QLIST_INSERT_HEAD(&s->curl_states, state, next);
     QLIST_INSERT_HEAD(&s->cache, cache, next);
     state->cache = cache;
     cache->use_count++;
@@ -623,7 +614,6 @@ err_release:
     qemu_aio_release(acb);
     return;
 
-
 }
 
 static BlockDriverAIOCB *curl_aio_readv(BlockDriverState *bs,
@@ -652,17 +642,18 @@ static BlockDriverAIOCB *curl_aio_readv(BlockDriverState *bs,
 static void curl_close(BlockDriverState *bs)
 {
     BDRVCURLState *s = bs->opaque;
-    int i;
 
     DPRINTF("CURL: Close\n");
-    for (i=0; i<CURL_NUM_STATES; i++) {
-        if (s->states[i].in_use)
-            curl_clean_state(&s->states[i]);
-        if (s->states[i].curl) {
-            curl_easy_cleanup(s->states[i].curl);
-            s->states[i].curl = NULL;
-        }
+
+    while (!QLIST_EMPTY(&s->curl_states)) {
+        CURLState *state = QLIST_FIRST(&s->curl_states);
+        /* Remove and clean curl easy handles */
+        curl_clean_state(state);
+        QLIST_REMOVE(state, next);
+        g_free(state);
+        state = NULL;
     }
+
     if (s->multi)
         curl_multi_cleanup(s->multi);
     g_free(s->url);
-- 
1.8.1.4

[Qemu-devel] [PATCH v2 09/11] curl: release everything on curl_close

[Fam Zheng]

Release everything properly on curl_close.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 block/curl.c | 37 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 36 insertions(+), 1 deletion(-)

diff --git a/block/curl.c b/block/curl.c
index f20d81c..bde2a55 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -644,6 +644,11 @@ static void curl_close(BlockDriverState *bs)
     BDRVCURLState *s = bs->opaque;
 
     DPRINTF("CURL: Close\n");
+    if (s->timer) {
+        qemu_del_timer(s->timer);
+        qemu_free_timer(s->timer);
+        s->timer = NULL;
+    }
 
     while (!QLIST_EMPTY(&s->curl_states)) {
         CURLState *state = QLIST_FIRST(&s->curl_states);
@@ -654,9 +659,39 @@ static void curl_close(BlockDriverState *bs)
         state = NULL;
     }
 
-    if (s->multi)
+    if (s->multi) {
         curl_multi_cleanup(s->multi);
+    }
+
+    while (!QLIST_EMPTY(&s->acbs)) {
+        CURLAIOCB *acb = QLIST_FIRST(&s->acbs);
+        acb->common.cb(acb->common.opaque, -EIO);
+        QLIST_REMOVE(acb, next);
+        g_free(acb);
+        acb = NULL;
+    }
+
+    while (!QLIST_EMPTY(&s->cache)) {
+        CURLDataCache *cache = QLIST_FIRST(&s->cache);
+        assert(cache->use_count == 0);
+        if (cache->data) {
+            g_free(cache->data);
+            cache->data = NULL;
+        }
+        QLIST_REMOVE(cache, next);
+        g_free(cache);
+        cache = NULL;
+    }
+
+    while (!QLIST_EMPTY(&s->socks)) {
+        CURLSockInfo *sock = QLIST_FIRST(&s->socks);
+        QLIST_REMOVE(sock, next);
+        g_free(sock);
+        sock = NULL;
+    }
+
     g_free(s->url);
+    s->url = NULL;
 }
 
 static int64_t curl_getlength(BlockDriverState *bs)
-- 
1.8.1.4

Re: [Qemu-devel] [PATCH v2 09/11] curl: release everything on curl_close

[Stefan Hajnoczi]

On Tue, May 14, 2013 at 10:26:28AM +0800, Fam Zheng wrote:
> Release everything properly on curl_close.
> 
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
>  block/curl.c | 37 ++++++++++++++++++++++++++++++++++++-
>  1 file changed, 36 insertions(+), 1 deletion(-)

Cleanup should be included in the patch that introduces a resource, for
example, s->timer.  Please squash relevant pieces into those patches.

[Qemu-devel] [PATCH v2 10/11] curl: add cache quota.

[Fam Zheng]

Introduce a cache quota: BDRVCURLState.cache_quota.
When adding new CURLDataCache to BDRVCURLState, if number of existing
CURLDataCache is larger than CURL_CACHE_QUOTA, try to release some first
to limit the in memory cache size.

A least used entry is selected for releasing.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 block/curl.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/block/curl.c b/block/curl.c
index bde2a55..fa9960d 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -40,6 +40,7 @@
 
 #define SECTOR_SIZE     512
 #define READ_AHEAD_SIZE (256 * 1024)
+#define CURL_CACHE_QUOTA 10
 
 struct BDRVCURLState;
 
@@ -91,6 +92,8 @@ typedef struct BDRVCURLState {
     QEMUTimer *timer;
     /* List of data cache ordered by access, freed from tail */
     QLIST_HEAD(, CURLDataCache) cache;
+    /* Threshold to release unused cache when cache list is longer than it */
+    int cache_quota;
     /* Whether http server accept range in header */
     bool accept_range;
 } BDRVCURLState;
@@ -517,6 +520,8 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags)
     curl_multi_socket_action(s->multi, CURL_SOCKET_TIMEOUT, 0, &running);
 
     qemu_opts_del(opts);
+    s->cache_quota = CURL_CACHE_QUOTA;
+
     return 0;
 
 out:
@@ -586,6 +591,27 @@ static void curl_readv_bh_cb(void *p)
     cache->data_len = aio_bytes + s->readahead_size;
     cache->write_pos = 0;
     cache->data = g_malloc(cache->data_len);
+    /* Try to release some cache */
+    while (0 && s->cache_quota <= 0) {
+        CURLDataCache *p;
+        CURLDataCache *q = NULL;
+        assert(!QLIST_EMPTY(&s->cache));
+        for (p = QLIST_FIRST(&s->cache);
+             p; p = QLIST_NEXT(p, next)) {
+            if (p->use_count == 0) {
+                q = p;
+            }
+        }
+        if (!q) {
+            break;
+        }
+        QLIST_REMOVE(q, next);
+        g_free(q->data);
+        q->data = NULL;
+        g_free(q);
+        q = NULL;
+        s->cache_quota++;
+    }
 
     QLIST_INSERT_HEAD(&s->acbs, acb, next);
     snprintf(state->range, CURL_RANGE_SIZE - 1, "%zd-%zd", cache->base_pos,
@@ -596,6 +622,7 @@ static void curl_readv_bh_cb(void *p)
     QLIST_INSERT_HEAD(&s->cache, cache, next);
     state->cache = cache;
     cache->use_count++;
+    s->cache_quota--;
     curl_multi_add_handle(s->multi, state->curl);
     /* kick off curl to start the action */
     curl_multi_socket_action(s->multi, 0, CURL_SOCKET_TIMEOUT, &running);
-- 
1.8.1.4

[Qemu-devel] [PATCH v2 11/11] curl: introduce ssl_no_cert runtime option.

[Fam Zheng]

Added an option to let curl disable ssl certificate check.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 block/curl.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/block/curl.c b/block/curl.c
index fa9960d..21a357b 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -96,6 +96,8 @@ typedef struct BDRVCURLState {
     int cache_quota;
     /* Whether http server accept range in header */
     bool accept_range;
+    /* Whether certificated ssl only */
+    bool ssl_no_cert;
 } BDRVCURLState;
 
 static void curl_clean_state(CURLState *s);
@@ -337,6 +339,8 @@ static CURLState *curl_init_state(BDRVCURLState *s)
     curl_easy_setopt(state->curl, CURLOPT_NOSIGNAL, 1);
     curl_easy_setopt(state->curl, CURLOPT_ERRORBUFFER, state->errmsg);
     curl_easy_setopt(state->curl, CURLOPT_FAILONERROR, 1);
+    curl_easy_setopt(state->curl, CURLOPT_SSL_VERIFYPEER,
+                     s->ssl_no_cert ? 0 : 1);
 
     /* Restrict supported protocols to avoid security issues in the more
      * obscure protocols.  For example, do not allow POP3/SMTP/IMAP see
@@ -427,7 +431,12 @@ static QemuOptsList runtime_opts = {
             .type = QEMU_OPT_SIZE,
             .help = "Readahead size",
         },
-        { /* end of list */ }
+        {
+            .name = "ssl_no_cert",
+            .type = QEMU_OPT_BOOL,
+            .help = "SSL certificate check",
+        },
+        { /* End of list */ }
     },
 };
 
@@ -465,6 +474,7 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags)
         goto out_noclean;
     }
 
+    s->ssl_no_cert = qemu_opt_get_bool(opts, "ssl_no_cert", true);
     if (!inited) {
         curl_global_init(CURL_GLOBAL_ALL);
         inited = 1;
-- 
1.8.1.4

Re: [Qemu-devel] [PATCH v2 00/11] curl: fix curl read

[Stefan Hajnoczi]

On Tue, May 14, 2013 at 10:26:19AM +0800, Fam Zheng wrote:
>  - CURLDataCache holds the user data read from libcurl, it is in a list
>    ordered by access, the used cache is moved to list head on access, so
>    the tail element is freed first. BDRVCURLState.cache_quota is the
>    threshold to start freeing cache.

Can you explain the need for a cache?

The guest kernel already does readahead if it wants to.  It can be tuned
from inside the guest if performance doesn't meet expectations.  The
block/curl.c-specific cache cannot be tuned from the guest.

Stefan

Re: [Qemu-devel] [PATCH v2 00/11] curl: fix curl read

[Fam Zheng]

On Tue, 05/14 10:22, Stefan Hajnoczi wrote:
> On Tue, May 14, 2013 at 10:26:19AM +0800, Fam Zheng wrote:
> >  - CURLDataCache holds the user data read from libcurl, it is in a list
> >    ordered by access, the used cache is moved to list head on access, so
> >    the tail element is freed first. BDRVCURLState.cache_quota is the
> >    threshold to start freeing cache.
> 
> Can you explain the need for a cache?
> 
> The guest kernel already does readahead if it wants to.  It can be tuned
> from inside the guest if performance doesn't meet expectations.  The
> block/curl.c-specific cache cannot be tuned from the guest.
> 

It's prefetch. If we send request for every read request, there'll be
too much overhead for protocal transactions.

I just name it CURLDataCache. Guest may have multiple sequencial reads,
so I used a list of it.

If you think this is reasonable, I'll update the commit messages on this.

-- 
Fam

Re: [Qemu-devel] [PATCH v2 00/11] curl: fix curl read

[Fam Zheng]

On Tue, 05/14 10:22, Stefan Hajnoczi wrote:
> On Tue, May 14, 2013 at 10:26:19AM +0800, Fam Zheng wrote:
> >  - CURLDataCache holds the user data read from libcurl, it is in a list
> >    ordered by access, the used cache is moved to list head on access, so
> >    the tail element is freed first. BDRVCURLState.cache_quota is the
> >    threshold to start freeing cache.
> 
> Can you explain the need for a cache?
> 
> The guest kernel already does readahead if it wants to.  It can be tuned
> from inside the guest if performance doesn't meet expectations.  The
> block/curl.c-specific cache cannot be tuned from the guest.

The guest may not do well on this. E.g. GRUB sends sequencial 2KB aio
requests loading the kernel of several MBs. This may make the curl
performs horribly to boot a guest from http served iso.

-- 
Fam

Re: [Qemu-devel] [PATCH v2 00/11] curl: fix curl read

[Stefan Hajnoczi]

On Wed, May 15, 2013 at 11:46:36AM +0800, Fam Zheng wrote:
> On Tue, 05/14 10:22, Stefan Hajnoczi wrote:
> > On Tue, May 14, 2013 at 10:26:19AM +0800, Fam Zheng wrote:
> > >  - CURLDataCache holds the user data read from libcurl, it is in a list
> > >    ordered by access, the used cache is moved to list head on access, so
> > >    the tail element is freed first. BDRVCURLState.cache_quota is the
> > >    threshold to start freeing cache.
> > 
> > Can you explain the need for a cache?
> > 
> > The guest kernel already does readahead if it wants to.  It can be tuned
> > from inside the guest if performance doesn't meet expectations.  The
> > block/curl.c-specific cache cannot be tuned from the guest.
> 
> The guest may not do well on this. E.g. GRUB sends sequencial 2KB aio
> requests loading the kernel of several MBs. This may make the curl
> performs horribly to boot a guest from http served iso.

Good point, this is probably the reason.  Once the kernel is running the
requests should be bigger but during boot the BIOS and bootloader may
issue very small requests.

Stefan