[Qemu-devel] [PATCH] block/nfs: cache allocated filesize for read-only files

[Qemu-devel] [PATCH] block/nfs: cache allocated filesize for read-only files

[Peter Lieven]

If the file is readonly its not expected to grow so
save the blocking call to nfs_fstat_async and use
the value saved at connection time. Also important
the monitor (and thus the main loop) will not hang
if block device info is queried and the NFS share
is unresponsive.

Signed-off-by: Peter Lieven <pl@kamp.de>
---
 block/nfs.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/block/nfs.c b/block/nfs.c
index 02eb4e4..dc9ed21 100644
--- a/block/nfs.c
+++ b/block/nfs.c
@@ -43,6 +43,7 @@ typedef struct NFSClient {
     int events;
     bool has_zero_init;
     AioContext *aio_context;
+    blkcnt_t st_blocks;
 } NFSClient;
 
 typedef struct NFSRPC {
@@ -374,6 +375,7 @@ static int64_t nfs_client_open(NFSClient *client, const char *filename,
     }
 
     ret = DIV_ROUND_UP(st.st_size, BDRV_SECTOR_SIZE);
+    client->st_blocks = st.st_blocks;
     client->has_zero_init = S_ISREG(st.st_mode);
     goto out;
 fail:
@@ -464,6 +466,10 @@ static int64_t nfs_get_allocated_file_size(BlockDriverState *bs)
     NFSRPC task = {0};
     struct stat st;
 
+    if (bdrv_is_read_only(bs)) {
+        return client->st_blocks * 512;
+    }
+
     task.st = &st;
     if (nfs_fstat_async(client->context, client->fh, nfs_co_generic_cb,
                         &task) != 0) {
-- 
1.9.1

Re: [Qemu-devel] [PATCH] block/nfs: cache allocated filesize for read-only files

[Max Reitz]

On 2015-08-21 at 00:49, Peter Lieven wrote:
> If the file is readonly its not expected to grow so
> save the blocking call to nfs_fstat_async and use
> the value saved at connection time. Also important
> the monitor (and thus the main loop) will not hang
> if block device info is queried and the NFS share
> is unresponsive.
>
> Signed-off-by: Peter Lieven <pl@kamp.de>
> ---
>   block/nfs.c | 6 ++++++
>   1 file changed, 6 insertions(+)

First, I don't like the idea of this patch very much, but since I've 
never used qemu's native NFS client, it's not up to me to decide whether 
it's worth it.

When it comes to breaking this, what comes to mind first is some 
external program opening the image read-write outside of qemu and 
writing to it. Maybe that's a case we generally don't want, but maybe 
that's something some people do on purpose, knowing what they're doing 
(with raw images), you never know.

Other than that, there's reopening. As far as I'm aware, qemu can reopen 
a R/W image read-only, and if that happens, st_blocks may be stale.

Max

> diff --git a/block/nfs.c b/block/nfs.c
> index 02eb4e4..dc9ed21 100644
> --- a/block/nfs.c
> +++ b/block/nfs.c
> @@ -43,6 +43,7 @@ typedef struct NFSClient {
>       int events;
>       bool has_zero_init;
>       AioContext *aio_context;
> +    blkcnt_t st_blocks;
>   } NFSClient;
>
>   typedef struct NFSRPC {
> @@ -374,6 +375,7 @@ static int64_t nfs_client_open(NFSClient *client, const char *filename,
>       }
>
>       ret = DIV_ROUND_UP(st.st_size, BDRV_SECTOR_SIZE);
> +    client->st_blocks = st.st_blocks;
>       client->has_zero_init = S_ISREG(st.st_mode);
>       goto out;
>   fail:
> @@ -464,6 +466,10 @@ static int64_t nfs_get_allocated_file_size(BlockDriverState *bs)
>       NFSRPC task = {0};
>       struct stat st;
>
> +    if (bdrv_is_read_only(bs)) {
> +        return client->st_blocks * 512;
> +    }
> +
>       task.st = &st;
>       if (nfs_fstat_async(client->context, client->fh, nfs_co_generic_cb,
>                           &task) != 0) {
>

Re: [Qemu-devel] [PATCH] block/nfs: cache allocated filesize for read-only files

[Peter Lieven]

Am 21.08.2015 um 18:46 schrieb Max Reitz:
> On 2015-08-21 at 00:49, Peter Lieven wrote:
>> If the file is readonly its not expected to grow so
>> save the blocking call to nfs_fstat_async and use
>> the value saved at connection time. Also important
>> the monitor (and thus the main loop) will not hang
>> if block device info is queried and the NFS share
>> is unresponsive.
>>
>> Signed-off-by: Peter Lieven <pl@kamp.de>
>> ---
>>   block/nfs.c | 6 ++++++
>>   1 file changed, 6 insertions(+)
>
> First, I don't like the idea of this patch very much, but since I've never used qemu's native NFS client, it's not up to me to decide whether it's worth it.

I am trying to solve that a stale NFS Server with a CDROM ISO on it can hang Qemus main loop. One of the things that happens is that
you query "info block" in hmp or "query-block" via QMP and indirectly call bdrv_get_allocated_file_size and bang, Qemu hangs. Also I don't
know if its worth to issue an RPC call for each executing of info block.


>
> When it comes to breaking this, what comes to mind first is some external program opening the image read-write outside of qemu and writing to it. Maybe that's a case we generally don't want, but maybe that's something some people do on purpose, knowing what they're doing (with raw images), you never know.

I would consider this bad behaviour. However, allocated file size shouldn't matter for raw images. If you resize the image from external you have to call bdrv_truncate anyway to make Qemu aware
of that change.

>
> Other than that, there's reopening. As far as I'm aware, qemu can reopen a R/W image read-only, and if that happens, st_blocks may be stale.

Thats a valid point. But it can be solved be implementing .bdrv_reopen_prepare and update st_blocks there.

Thanks for you thoughts,
Peter

Re: [Qemu-devel] [PATCH] block/nfs: cache allocated filesize for read-only files

[Max Reitz]

On 2015-08-21 at 11:11, Peter Lieven wrote:
> Am 21.08.2015 um 18:46 schrieb Max Reitz:
>> On 2015-08-21 at 00:49, Peter Lieven wrote:
>>> If the file is readonly its not expected to grow so
>>> save the blocking call to nfs_fstat_async and use
>>> the value saved at connection time. Also important
>>> the monitor (and thus the main loop) will not hang
>>> if block device info is queried and the NFS share
>>> is unresponsive.
>>>
>>> Signed-off-by: Peter Lieven <pl@kamp.de>
>>> ---
>>>    block/nfs.c | 6 ++++++
>>>    1 file changed, 6 insertions(+)
>>
>> First, I don't like the idea of this patch very much, but since I've never used qemu's native NFS client, it's not up to me to decide whether it's worth it.
>
> I am trying to solve that a stale NFS Server with a CDROM ISO on it can hang Qemus main loop. One of the things that happens is that
> you query "info block" in hmp or "query-block" via QMP and indirectly call bdrv_get_allocated_file_size and bang, Qemu hangs. Also I don't
> know if its worth to issue an RPC call for each executing of info block.

OK.

>>
>> When it comes to breaking this, what comes to mind first is some external program opening the image read-write outside of qemu and writing to it. Maybe that's a case we generally don't want, but maybe that's something some people do on purpose, knowing what they're doing (with raw images), you never know.
>
> I would consider this bad behaviour. However, allocated file size shouldn't matter for raw images.

I don't know about NFS, but for other file systems it does.

$ ./qemu-img create -f raw test.raw 1G
$ ./qemu-io -c 'write 1023M 1M' test.raw
$ ./qemu-img info test.raw
...
virtual size: 1.0G (1073741824 bytes)
disk size: 1.0M

I do consider it bad behavior, too, but with raw images it should 
actually be valid for some use cases.

> If you resize the image from external you have to call bdrv_truncate anyway to make Qemu aware
> of that change.
>
>>
>> Other than that, there's reopening. As far as I'm aware, qemu can reopen a R/W image read-only, and if that happens, st_blocks may be stale.
>
> Thats a valid point. But it can be solved be implementing .bdrv_reopen_prepare and update st_blocks there.

Right. Since the allocated size is not that important of an information, 
generally, I think I'd be fine with breaking the use case of writing to 
an image with an external tool while qemu is using that image read-only, 
as long as reopening works.

> Thanks for you thoughts,

Thanks for your patch. :-)

Max