[Qemu-devel] [PATCH] curl: Add override_accept_ranges flag to force sending range

[Qemu-devel] [PATCH] curl: Add override_accept_ranges flag to force sending range

[Richard W.M. Jones]

If you use this patch + Daniel Henrique Barboza's patch that adds a
timeout, then you can access guest files on VMware ESX servers.

Rich.

[Qemu-devel] [PATCH] curl: Add override_accept_ranges flag to force sending range requests.

[Richard W.M. Jones]

Some servers (notably VMware ESX) accept range requests, but don't
send back the Accept-Ranges: bytes header in their initial response.

For these servers you can set override_accept_ranges to 'on' which
forces this block driver to send range requests anyway.

Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
---
 block/curl.c    | 10 +++++++++-
 qemu-options.hx |  5 +++++
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/block/curl.c b/block/curl.c
index 095b5a4..3905c6e 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -71,6 +71,7 @@ static CURLMcode __curl_multi_socket_action(CURLM *multi_handle,
 #define CURL_BLOCK_OPT_URL       "url"
 #define CURL_BLOCK_OPT_READAHEAD "readahead"
 #define CURL_BLOCK_OPT_SSLVERIFY "sslverify"
+#define CURL_BLOCK_OPT_OVERRIDE_ACCEPT_RANGES "override_accept_ranges"
 
 struct BDRVCURLState;
 
@@ -489,6 +490,11 @@ static QemuOptsList runtime_opts = {
             .type = QEMU_OPT_BOOL,
             .help = "Verify SSL certificate"
         },
+        {
+            .name = CURL_BLOCK_OPT_OVERRIDE_ACCEPT_RANGES,
+            .type = QEMU_OPT_BOOL,
+            .help = "Server accepts range requests"
+        },
         { /* end of list */ }
     },
 };
@@ -547,7 +553,9 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
 
     // Get file size
 
-    s->accept_range = false;
+    s->accept_range =
+        qemu_opt_get_bool(opts, CURL_BLOCK_OPT_OVERRIDE_ACCEPT_RANGES,
+                          false);
     curl_easy_setopt(state->curl, CURLOPT_NOBODY, 1);
     curl_easy_setopt(state->curl, CURLOPT_HEADERFUNCTION,
                      curl_header_cb);
diff --git a/qemu-options.hx b/qemu-options.hx
index c573dd8..626ebb3 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -2351,6 +2351,11 @@ multiple of 512 bytes. It defaults to 256k.
 @item sslverify
 Whether to verify the remote server's certificate when connecting over SSL. It
 can have the value 'on' or 'off'. It defaults to 'on'.
+
+@item override_accept_ranges
+Some servers (notably VMware ESX) accept range requests, but don't
+declare this fact in the headers that they return. Setting this option
+to 'on' forces CURL to use range requests. The default is 'off'.
 @end table
 
 Note that when passing options to qemu explicitly, @option{driver} is the value
-- 
2.0.4

Re: [Qemu-devel] [PATCH] curl: Add override_accept_ranges flag to force sending range requests.

[Fam Zheng]

On Tue, 08/26 21:48, Richard W.M. Jones wrote:
> Some servers (notably VMware ESX) accept range requests, but don't
> send back the Accept-Ranges: bytes header in their initial response.
> 
> For these servers you can set override_accept_ranges to 'on' which
> forces this block driver to send range requests anyway.
> 
> Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
> ---
>  block/curl.c    | 10 +++++++++-
>  qemu-options.hx |  5 +++++
>  2 files changed, 14 insertions(+), 1 deletion(-)
> 
> diff --git a/block/curl.c b/block/curl.c
> index 095b5a4..3905c6e 100644
> --- a/block/curl.c
> +++ b/block/curl.c
> @@ -71,6 +71,7 @@ static CURLMcode __curl_multi_socket_action(CURLM *multi_handle,
>  #define CURL_BLOCK_OPT_URL       "url"
>  #define CURL_BLOCK_OPT_READAHEAD "readahead"
>  #define CURL_BLOCK_OPT_SSLVERIFY "sslverify"
> +#define CURL_BLOCK_OPT_OVERRIDE_ACCEPT_RANGES "override_accept_ranges"
>  
>  struct BDRVCURLState;
>  
> @@ -489,6 +490,11 @@ static QemuOptsList runtime_opts = {
>              .type = QEMU_OPT_BOOL,
>              .help = "Verify SSL certificate"
>          },
> +        {
> +            .name = CURL_BLOCK_OPT_OVERRIDE_ACCEPT_RANGES,
> +            .type = QEMU_OPT_BOOL,
> +            .help = "Server accepts range requests"
> +        },
>          { /* end of list */ }
>      },
>  };
> @@ -547,7 +553,9 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
>  
>      // Get file size
>  
> -    s->accept_range = false;
> +    s->accept_range =
> +        qemu_opt_get_bool(opts, CURL_BLOCK_OPT_OVERRIDE_ACCEPT_RANGES,
> +                          false);
>      curl_easy_setopt(state->curl, CURLOPT_NOBODY, 1);
>      curl_easy_setopt(state->curl, CURLOPT_HEADERFUNCTION,
>                       curl_header_cb);
> diff --git a/qemu-options.hx b/qemu-options.hx
> index c573dd8..626ebb3 100644
> --- a/qemu-options.hx
> +++ b/qemu-options.hx
> @@ -2351,6 +2351,11 @@ multiple of 512 bytes. It defaults to 256k.
>  @item sslverify
>  Whether to verify the remote server's certificate when connecting over SSL. It
>  can have the value 'on' or 'off'. It defaults to 'on'.
> +
> +@item override_accept_ranges
> +Some servers (notably VMware ESX) accept range requests, but don't
> +declare this fact in the headers that they return. Setting this option
> +to 'on' forces CURL to use range requests. The default is 'off'.
>  @end table
>  
>  Note that when passing options to qemu explicitly, @option{driver} is the value
> -- 
> 2.0.4
> 
> 

Reviewed-by: Fam Zheng <famz@redhat.com>

Re: [Qemu-devel] [PATCH] curl: Add override_accept_ranges flag to force sending range requests.

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1139 bytes --]

On 08/26/2014 08:38 PM, Fam Zheng wrote:
> On Tue, 08/26 21:48, Richard W.M. Jones wrote:
>> Some servers (notably VMware ESX) accept range requests, but don't
>> send back the Accept-Ranges: bytes header in their initial response.
>>
>> For these servers you can set override_accept_ranges to 'on' which
>> forces this block driver to send range requests anyway.

Is this a case where we should be naming with dashes instead of
underscores, as in override-accept-ranges?

>> @@ -489,6 +490,11 @@ static QemuOptsList runtime_opts = {
>>              .type = QEMU_OPT_BOOL,
>>              .help = "Verify SSL certificate"
>>          },
>> +        {
>> +            .name = CURL_BLOCK_OPT_OVERRIDE_ACCEPT_RANGES,
>> +            .type = QEMU_OPT_BOOL,
>> +            .help = "Server accepts range requests"
>> +        },

It would be nice for someone to finally port the QAPI counterpart
(BlockdevOptions) so that curl devices can be hotplugged via
blockdev-add.  But I guess that's a separate patch.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 539 bytes --]

Re: [Qemu-devel] [PATCH] curl: Add override_accept_ranges flag to force sending range requests.

[Markus Armbruster]

Eric Blake <eblake@redhat.com> writes:

> On 08/26/2014 08:38 PM, Fam Zheng wrote:
>> On Tue, 08/26 21:48, Richard W.M. Jones wrote:
>>> Some servers (notably VMware ESX) accept range requests, but don't
>>> send back the Accept-Ranges: bytes header in their initial response.
>>>
>>> For these servers you can set override_accept_ranges to 'on' which
>>> forces this block driver to send range requests anyway.
>
> Is this a case where we should be naming with dashes instead of
> underscores, as in override-accept-ranges?

Yes.

>>> @@ -489,6 +490,11 @@ static QemuOptsList runtime_opts = {
>>>              .type = QEMU_OPT_BOOL,
>>>              .help = "Verify SSL certificate"
>>>          },
>>> +        {
>>> +            .name = CURL_BLOCK_OPT_OVERRIDE_ACCEPT_RANGES,
>>> +            .type = QEMU_OPT_BOOL,
>>> +            .help = "Server accepts range requests"
>>> +        },
>
> It would be nice for someone to finally port the QAPI counterpart
> (BlockdevOptions) so that curl devices can be hotplugged via
> blockdev-add.  But I guess that's a separate patch.

Yes again.

Re: [Qemu-devel] [PATCH] curl: Add override_accept_ranges flag to force sending range requests.

[Richard W.M. Jones]

On Wed, Aug 27, 2014 at 08:37:22AM +0200, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
> 
> > On 08/26/2014 08:38 PM, Fam Zheng wrote:
> >> On Tue, 08/26 21:48, Richard W.M. Jones wrote:
> >>> Some servers (notably VMware ESX) accept range requests, but don't
> >>> send back the Accept-Ranges: bytes header in their initial response.
> >>>
> >>> For these servers you can set override_accept_ranges to 'on' which
> >>> forces this block driver to send range requests anyway.
> >
> > Is this a case where we should be naming with dashes instead of
> > underscores, as in override-accept-ranges?
> 
> Yes.

Thanks for reviewing.  I'm actually going to drop this for a couple of
reasons:

 - VMware vSphere does send the header, only ESX doesn't.

 - When you access ESX (using this patch) eventually the ESX web
   server crashes.  In a sense ESX was correct that it doesn't support
   ranges -- because it's buggy.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

Re: [Qemu-devel] [PATCH] curl: Add override_accept_ranges flag to force sending range requests.

[Richard W.M. Jones]

On Tue, Aug 26, 2014 at 08:57:30PM -0600, Eric Blake wrote:
> On 08/26/2014 08:38 PM, Fam Zheng wrote:
> > On Tue, 08/26 21:48, Richard W.M. Jones wrote:
> >> Some servers (notably VMware ESX) accept range requests, but don't
> >> send back the Accept-Ranges: bytes header in their initial response.
> >>
> >> For these servers you can set override_accept_ranges to 'on' which
> >> forces this block driver to send range requests anyway.
> 
> Is this a case where we should be naming with dashes instead of
> underscores, as in override-accept-ranges?

Yes.

I'm not particularly happy with the long name either, but couldn't
think of anything shorter.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/

Re: [Qemu-devel] [PATCH] curl: Add override_accept_ranges flag to force sending range requests.

[Daniel H Barboza]

On 08/26/2014 05:48 PM, Richard W.M. Jones wrote:
> Some servers (notably VMware ESX) accept range requests, but don't
> send back the Accept-Ranges: bytes header in their initial response.
>
> For these servers you can set override_accept_ranges to 'on' which
> forces this block driver to send range requests anyway.
>
> Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
> ---
>   block/curl.c    | 10 +++++++++-
>   qemu-options.hx |  5 +++++
>   2 files changed, 14 insertions(+), 1 deletion(-)
>
> diff --git a/block/curl.c b/block/curl.c
> index 095b5a4..3905c6e 100644
> --- a/block/curl.c
> +++ b/block/curl.c
> @@ -71,6 +71,7 @@ static CURLMcode __curl_multi_socket_action(CURLM *multi_handle,
>   #define CURL_BLOCK_OPT_URL       "url"
>   #define CURL_BLOCK_OPT_READAHEAD "readahead"
>   #define CURL_BLOCK_OPT_SSLVERIFY "sslverify"
> +#define CURL_BLOCK_OPT_OVERRIDE_ACCEPT_RANGES "override_accept_ranges"
>
>   struct BDRVCURLState;
>
> @@ -489,6 +490,11 @@ static QemuOptsList runtime_opts = {
>               .type = QEMU_OPT_BOOL,
>               .help = "Verify SSL certificate"
>           },
> +        {
> +            .name = CURL_BLOCK_OPT_OVERRIDE_ACCEPT_RANGES,
> +            .type = QEMU_OPT_BOOL,
> +            .help = "Server accepts range requests"
> +        },
>           { /* end of list */ }
>       },
>   };
> @@ -547,7 +553,9 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags,
>
>       // Get file size
>
> -    s->accept_range = false;
> +    s->accept_range =
> +        qemu_opt_get_bool(opts, CURL_BLOCK_OPT_OVERRIDE_ACCEPT_RANGES,
> +                          false);
>       curl_easy_setopt(state->curl, CURLOPT_NOBODY, 1);
>       curl_easy_setopt(state->curl, CURLOPT_HEADERFUNCTION,
>                        curl_header_cb);
> diff --git a/qemu-options.hx b/qemu-options.hx
> index c573dd8..626ebb3 100644
> --- a/qemu-options.hx
> +++ b/qemu-options.hx
> @@ -2351,6 +2351,11 @@ multiple of 512 bytes. It defaults to 256k.
>   @item sslverify
>   Whether to verify the remote server's certificate when connecting over SSL. It
>   can have the value 'on' or 'off'. It defaults to 'on'.
> +
> +@item override_accept_ranges
> +Some servers (notably VMware ESX) accept range requests, but don't
> +declare this fact in the headers that they return. Setting this option
> +to 'on' forces CURL to use range requests. The default is 'off'.
>   @end table
>
>   Note that when passing options to qemu explicitly, @option{driver} is the value
Reviewed-by: Daniel Barboza <danielhb@linux.vnet.ibm.com>
Tested-by: Daniel Barboza <danielhb@linux.vnet.ibm.com>