[Qemu-devel] [PATCH v2] curl: Allow a cookie or cookies to be sent with http/https

[Qemu-devel] [PATCH v2] curl: Allow a cookie or cookies to be sent with http/https

[Richard W.M. Jones]

Since v1:

 - Remove the conditional around g_strdup (thanks Matt Booth).

Rich.

[Qemu-devel] [PATCH v2] curl: Allow a cookie or cookies to be sent with http/https requests.

[Richard W.M. Jones]

In order to access VMware ESX efficiently, we need to send a session
cookie.  This patch is very simple and just allows you to send that
session cookie.  It punts on the question of how you get the session
cookie in the first place, but in practice you can just run a `curl'
command against the server and extract the cookie that way.

To use it, add file.cookie to the curl URL.  For example:

$ qemu-img info 'json: {
    "file.driver":"https",
    "file.url":"https://vcenter/folder/Windows%202003/Windows%202003-flat.vmdk?dcPath=Datacenter&dsName=datastore1",
    "file.sslverify":"off",
    "file.cookie":"vmware_soap_session=\"52a01262-bf93-ccce-d379-8dabb3e55560\""}'
image: [...]
file format: raw
virtual size: 8.0G (8589934592 bytes)
disk size: unavailable

Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
---
 block/curl.c    | 16 ++++++++++++++++
 qemu-options.hx |  5 +++++
 2 files changed, 21 insertions(+)

diff --git a/block/curl.c b/block/curl.c
index f59615d..fa1a258 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -71,6 +71,7 @@ static CURLMcode __curl_multi_socket_action(CURLM *multi_handle,
 #define CURL_BLOCK_OPT_URL       "url"
 #define CURL_BLOCK_OPT_READAHEAD "readahead"
 #define CURL_BLOCK_OPT_SSLVERIFY "sslverify"
+#define CURL_BLOCK_OPT_COOKIE    "cookie"
 
 struct BDRVCURLState;
 
@@ -109,6 +110,7 @@ typedef struct BDRVCURLState {
     char *url;
     size_t readahead_size;
     bool sslverify;
+    char *cookie;
     bool accept_range;
     AioContext *aio_context;
 } BDRVCURLState;
@@ -382,6 +384,9 @@ static CURLState *curl_init_state(BlockDriverState *bs, BDRVCURLState *s)
         curl_easy_setopt(state->curl, CURLOPT_URL, s->url);
         curl_easy_setopt(state->curl, CURLOPT_SSL_VERIFYPEER,
                          (long) s->sslverify);
+        if (s->cookie) {
+            curl_easy_setopt(state->curl, CURLOPT_COOKIE, s->cookie);
+        }
         curl_easy_setopt(state->curl, CURLOPT_TIMEOUT, 5);
         curl_easy_setopt(state->curl, CURLOPT_WRITEFUNCTION,
                          (void *)curl_read_cb);
@@ -489,6 +494,11 @@ static QemuOptsList runtime_opts = {
             .type = QEMU_OPT_BOOL,
             .help = "Verify SSL certificate"
         },
+        {
+            .name = CURL_BLOCK_OPT_COOKIE,
+            .type = QEMU_OPT_STRING,
+            .help = "Pass the cookie or list of cookies with each request"
+        },
         { /* end of list */ }
     },
 };
@@ -501,6 +511,7 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
     QemuOpts *opts;
     Error *local_err = NULL;
     const char *file;
+    const char *cookie;
     double d;
 
     static int inited = 0;
@@ -527,6 +538,9 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
 
     s->sslverify = qemu_opt_get_bool(opts, CURL_BLOCK_OPT_SSLVERIFY, true);
 
+    cookie = qemu_opt_get(opts, CURL_BLOCK_OPT_COOKIE);
+    s->cookie = g_strdup(cookie);
+
     file = qemu_opt_get(opts, CURL_BLOCK_OPT_URL);
     if (file == NULL) {
         error_setg(errp, "curl block driver requires an 'url' option");
@@ -582,6 +596,7 @@ out:
     curl_easy_cleanup(state->curl);
     state->curl = NULL;
 out_noclean:
+    g_free(s->cookie);
     g_free(s->url);
     qemu_opts_del(opts);
     return -EINVAL;
@@ -684,6 +699,7 @@ static void curl_close(BlockDriverState *bs)
     DPRINTF("CURL: Close\n");
     curl_detach_aio_context(bs);
 
+    g_free(s->cookie);
     g_free(s->url);
 }
 
diff --git a/qemu-options.hx b/qemu-options.hx
index c573dd8..7b4a58a 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -2351,6 +2351,11 @@ multiple of 512 bytes. It defaults to 256k.
 @item sslverify
 Whether to verify the remote server's certificate when connecting over SSL. It
 can have the value 'on' or 'off'. It defaults to 'on'.
+
+@item cookie
+Send this cookie (it can also be a list of cookies separated by ';') with
+each outgoing request.  Only supported when using protocols such as HTTP
+which support cookies, otherwise ignored.
 @end table
 
 Note that when passing options to qemu explicitly, @option{driver} is the value
-- 
2.0.4

Re: [Qemu-devel] [PATCH v2] curl: Allow a cookie or cookies to be sent with http/https requests.

[Matthew Booth]

Looks good. Please add:

Reviewed-by: Matthew Booth <mbooth@redhat.com>

Matt

On 29/08/14 16:03, Richard W.M. Jones wrote:
> In order to access VMware ESX efficiently, we need to send a session
> cookie.  This patch is very simple and just allows you to send that
> session cookie.  It punts on the question of how you get the session
> cookie in the first place, but in practice you can just run a `curl'
> command against the server and extract the cookie that way.
> 
> To use it, add file.cookie to the curl URL.  For example:
> 
> $ qemu-img info 'json: {
>     "file.driver":"https",
>     "file.url":"https://vcenter/folder/Windows%202003/Windows%202003-flat.vmdk?dcPath=Datacenter&dsName=datastore1",
>     "file.sslverify":"off",
>     "file.cookie":"vmware_soap_session=\"52a01262-bf93-ccce-d379-8dabb3e55560\""}'
> image: [...]
> file format: raw
> virtual size: 8.0G (8589934592 bytes)
> disk size: unavailable
> 
> Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
> ---
>  block/curl.c    | 16 ++++++++++++++++
>  qemu-options.hx |  5 +++++
>  2 files changed, 21 insertions(+)
> 
> diff --git a/block/curl.c b/block/curl.c
> index f59615d..fa1a258 100644
> --- a/block/curl.c
> +++ b/block/curl.c
> @@ -71,6 +71,7 @@ static CURLMcode __curl_multi_socket_action(CURLM *multi_handle,
>  #define CURL_BLOCK_OPT_URL       "url"
>  #define CURL_BLOCK_OPT_READAHEAD "readahead"
>  #define CURL_BLOCK_OPT_SSLVERIFY "sslverify"
> +#define CURL_BLOCK_OPT_COOKIE    "cookie"
>  
>  struct BDRVCURLState;
>  
> @@ -109,6 +110,7 @@ typedef struct BDRVCURLState {
>      char *url;
>      size_t readahead_size;
>      bool sslverify;
> +    char *cookie;
>      bool accept_range;
>      AioContext *aio_context;
>  } BDRVCURLState;
> @@ -382,6 +384,9 @@ static CURLState *curl_init_state(BlockDriverState *bs, BDRVCURLState *s)
>          curl_easy_setopt(state->curl, CURLOPT_URL, s->url);
>          curl_easy_setopt(state->curl, CURLOPT_SSL_VERIFYPEER,
>                           (long) s->sslverify);
> +        if (s->cookie) {
> +            curl_easy_setopt(state->curl, CURLOPT_COOKIE, s->cookie);
> +        }
>          curl_easy_setopt(state->curl, CURLOPT_TIMEOUT, 5);
>          curl_easy_setopt(state->curl, CURLOPT_WRITEFUNCTION,
>                           (void *)curl_read_cb);
> @@ -489,6 +494,11 @@ static QemuOptsList runtime_opts = {
>              .type = QEMU_OPT_BOOL,
>              .help = "Verify SSL certificate"
>          },
> +        {
> +            .name = CURL_BLOCK_OPT_COOKIE,
> +            .type = QEMU_OPT_STRING,
> +            .help = "Pass the cookie or list of cookies with each request"
> +        },
>          { /* end of list */ }
>      },
>  };
> @@ -501,6 +511,7 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
>      QemuOpts *opts;
>      Error *local_err = NULL;
>      const char *file;
> +    const char *cookie;
>      double d;
>  
>      static int inited = 0;
> @@ -527,6 +538,9 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
>  
>      s->sslverify = qemu_opt_get_bool(opts, CURL_BLOCK_OPT_SSLVERIFY, true);
>  
> +    cookie = qemu_opt_get(opts, CURL_BLOCK_OPT_COOKIE);
> +    s->cookie = g_strdup(cookie);
> +
>      file = qemu_opt_get(opts, CURL_BLOCK_OPT_URL);
>      if (file == NULL) {
>          error_setg(errp, "curl block driver requires an 'url' option");
> @@ -582,6 +596,7 @@ out:
>      curl_easy_cleanup(state->curl);
>      state->curl = NULL;
>  out_noclean:
> +    g_free(s->cookie);
>      g_free(s->url);
>      qemu_opts_del(opts);
>      return -EINVAL;
> @@ -684,6 +699,7 @@ static void curl_close(BlockDriverState *bs)
>      DPRINTF("CURL: Close\n");
>      curl_detach_aio_context(bs);
>  
> +    g_free(s->cookie);
>      g_free(s->url);
>  }
>  
> diff --git a/qemu-options.hx b/qemu-options.hx
> index c573dd8..7b4a58a 100644
> --- a/qemu-options.hx
> +++ b/qemu-options.hx
> @@ -2351,6 +2351,11 @@ multiple of 512 bytes. It defaults to 256k.
>  @item sslverify
>  Whether to verify the remote server's certificate when connecting over SSL. It
>  can have the value 'on' or 'off'. It defaults to 'on'.
> +
> +@item cookie
> +Send this cookie (it can also be a list of cookies separated by ';') with
> +each outgoing request.  Only supported when using protocols such as HTTP
> +which support cookies, otherwise ignored.
>  @end table
>  
>  Note that when passing options to qemu explicitly, @option{driver} is the value
> 


-- 
Matthew Booth
Red Hat Engineering, Virtualisation Team

Phone: +442070094448 (UK)
GPG ID:  D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490

Re: [Qemu-devel] [PATCH v2] curl: Allow a cookie or cookies to be sent with http/https requests.

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1645 bytes --]

On 08/29/2014 09:03 AM, Richard W.M. Jones wrote:
> In order to access VMware ESX efficiently, we need to send a session
> cookie.  This patch is very simple and just allows you to send that
> session cookie.  It punts on the question of how you get the session
> cookie in the first place, but in practice you can just run a `curl'
> command against the server and extract the cookie that way.
> 

> +++ b/qemu-options.hx
> @@ -2351,6 +2351,11 @@ multiple of 512 bytes. It defaults to 256k.
>  @item sslverify
>  Whether to verify the remote server's certificate when connecting over SSL. It
>  can have the value 'on' or 'off'. It defaults to 'on'.
> +
> +@item cookie
> +Send this cookie (it can also be a list of cookies separated by ';') with
> +each outgoing request.  Only supported when using protocols such as HTTP
> +which support cookies, otherwise ignored.

';' has to be quoted to enter it in the shell command line (but then
again, the cookie probably contains literal " which also has to be quoted).

We still don't have a QMP mapping for curl device hotplug.  But when we
gain one, do we really want to have a single (long) string containing
multiple cookies, or would it be better to make this an array argument?
 On the command-line, which is nicer, taking the cookie option multiple
times ('file.cookie=xyz,file.cookie.abc'), taking it as an automatic
array ('file.cookie.0=xyz,file.cookie.1=abc') or forcing the user to
cram all cookies into a single option ('file.cookie="xyz;abc"')?

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 539 bytes --]

Re: [Qemu-devel] [PATCH v2] curl: Allow a cookie or cookies to be sent with http/https

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 264 bytes --]

On Fri, Aug 29, 2014 at 04:03:11PM +0100, Richard W.M. Jones wrote:
> Since v1:
> 
>  - Remove the conditional around g_strdup (thanks Matt Booth).
> 
> Rich.
> 

Thanks, applied to my block tree:
https://github.com/stefanha/qemu/commits/block

Stefan

[-- Attachment #2: Type: application/pgp-signature, Size: 473 bytes --]

Re: [Qemu-devel] [PATCH v2] curl: Allow a cookie or cookies to be sent with http/https requests.

[Richard W.M. Jones]

On Fri, Aug 29, 2014 at 09:10:10AM -0600, Eric Blake wrote:
> On 08/29/2014 09:03 AM, Richard W.M. Jones wrote:
> > In order to access VMware ESX efficiently, we need to send a session
> > cookie.  This patch is very simple and just allows you to send that
> > session cookie.  It punts on the question of how you get the session
> > cookie in the first place, but in practice you can just run a `curl'
> > command against the server and extract the cookie that way.
> > 
> 
> > +++ b/qemu-options.hx
> > @@ -2351,6 +2351,11 @@ multiple of 512 bytes. It defaults to 256k.
> >  @item sslverify
> >  Whether to verify the remote server's certificate when connecting over SSL. It
> >  can have the value 'on' or 'off'. It defaults to 'on'.
> > +
> > +@item cookie
> > +Send this cookie (it can also be a list of cookies separated by ';') with
> > +each outgoing request.  Only supported when using protocols such as HTTP
> > +which support cookies, otherwise ignored.
> 
> ';' has to be quoted to enter it in the shell command line (but then
> again, the cookie probably contains literal " which also has to be quoted).
> 
> We still don't have a QMP mapping for curl device hotplug.  But when we
> gain one, do we really want to have a single (long) string containing
> multiple cookies, or would it be better to make this an array argument?
>  On the command-line, which is nicer, taking the cookie option multiple
> times ('file.cookie=xyz,file.cookie.abc'), taking it as an automatic
> array ('file.cookie.0=xyz,file.cookie.1=abc') or forcing the user to
> cram all cookies into a single option ('file.cookie="xyz;abc"')?

For my immediate needs, I don't care at all about multiple cookies.
It's just a side-effect of the CURL API that they would work here.
I'm happy to drop all references to them from the documentation ...

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v

Re: [Qemu-devel] [PATCH v2] curl: Allow a cookie or cookies to be sent with http/https requests.

[Matthew Booth]

On 29/08/14 16:10, Eric Blake wrote:
> On 08/29/2014 09:03 AM, Richard W.M. Jones wrote:
>> In order to access VMware ESX efficiently, we need to send a session
>> cookie.  This patch is very simple and just allows you to send that
>> session cookie.  It punts on the question of how you get the session
>> cookie in the first place, but in practice you can just run a `curl'
>> command against the server and extract the cookie that way.
>>
> 
>> +++ b/qemu-options.hx
>> @@ -2351,6 +2351,11 @@ multiple of 512 bytes. It defaults to 256k.
>>  @item sslverify
>>  Whether to verify the remote server's certificate when connecting over SSL. It
>>  can have the value 'on' or 'off'. It defaults to 'on'.
>> +
>> +@item cookie
>> +Send this cookie (it can also be a list of cookies separated by ';') with
>> +each outgoing request.  Only supported when using protocols such as HTTP
>> +which support cookies, otherwise ignored.
> 
> ';' has to be quoted to enter it in the shell command line (but then
> again, the cookie probably contains literal " which also has to be quoted).
> 
> We still don't have a QMP mapping for curl device hotplug.  But when we
> gain one, do we really want to have a single (long) string containing
> multiple cookies, or would it be better to make this an array argument?
>  On the command-line, which is nicer, taking the cookie option multiple
> times ('file.cookie=xyz,file.cookie.abc'), taking it as an automatic
> array ('file.cookie.0=xyz,file.cookie.1=abc') or forcing the user to
> cram all cookies into a single option ('file.cookie="xyz;abc"')?

I thought about this, too. We're really only passing on curl's cookie
syntax:

http://curl.haxx.se/libcurl/c/CURLOPT_COOKIE.html

So even if we did it differently, the driver would still have to
reconstruct this string with ';' separation and handle escaping issues.

I doubt this will be a commonly used option, and even less frequently
used with multiple cookies, if ever[1]. Given that it is possible to use
multiple cookies without massive effort, I think the substantially
simpler code is a reasonable trade-off.

Matt

[1] Feel free to lart me with this at a later date ;)
-- 
Matthew Booth
Red Hat Engineering, Virtualisation Team

Phone: +442070094448 (UK)
GPG ID:  D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490

Re: [Qemu-devel] [PATCH v2] curl: Allow a cookie or cookies to be sent with http/https requests.

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1748 bytes --]

On 08/29/2014 09:28 AM, Richard W.M. Jones wrote:
> On Fri, Aug 29, 2014 at 09:10:10AM -0600, Eric Blake wrote:
>> On 08/29/2014 09:03 AM, Richard W.M. Jones wrote:
>>> In order to access VMware ESX efficiently, we need to send a session
>>> cookie.  This patch is very simple and just allows you to send that
>>> session cookie.  It punts on the question of how you get the session
>>> cookie in the first place, but in practice you can just run a `curl'
>>> command against the server and extract the cookie that way.
>>>

>>
>> We still don't have a QMP mapping for curl device hotplug.  But when we
>> gain one, do we really want to have a single (long) string containing
>> multiple cookies, or would it be better to make this an array argument?
>>  On the command-line, which is nicer, taking the cookie option multiple
>> times ('file.cookie=xyz,file.cookie.abc'), taking it as an automatic
>> array ('file.cookie.0=xyz,file.cookie.1=abc') or forcing the user to
>> cram all cookies into a single option ('file.cookie="xyz;abc"')?
> 
> For my immediate needs, I don't care at all about multiple cookies.
> It's just a side-effect of the CURL API that they would work here.
> I'm happy to drop all references to them from the documentation ...

Okay, given that we are treating the string as an opaque passthrough to
keep curl happy, and not something we are processing internally, I'm
happy with keeping it as a single (long) string, and just simplifying
the docs to mention that it is a string for curl to use, perhaps without
needing to go into details on whether it allows one vs. multiple cookies.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 539 bytes --]