[Qemu-devel] [PATCH v5.1 0/2] qga: report the usage of fs in guests

[Qemu-devel] [PATCH v5.1 0/2] qga: report the usage of fs in guests

[Chen Hanxiao]

This series report the usage of guest's fs
by "used_bytes" and "total_bytes" in guest-get-fsinfo.

Chen Hanxiao (2):
  qga: add mountpoint usage info to GuestFilesystemInfo
  qga-win: add driver path usage to GuestFilesystemInfo

 qga/commands-posix.c | 18 ++++++++++++++++++
 qga/commands-win32.c | 12 ++++++++++++
 qga/qapi-schema.json |  3 +++
 3 files changed, 33 insertions(+)

-- 
2.17.0

[Qemu-devel] [PATCH v5.1 1/2] qga: add mountpoint usage info to GuestFilesystemInfo

[Chen Hanxiao]

From: Chen Hanxiao <chenhanxiao@gmail.com>

This patch adds support for getting the usage of mounted
filesystem.
The usage of fs stored as used_bytes and total_bytes.
It's very useful when we try to monitor guest's filesystem.

Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
Cc: Eric Blake <eblake@redhat.com>
Cc: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Chen Hanxiao <chenhanxiao@gmail.com>
---
v2:
   add description in qapi-schema and version numbers
v3:
   use float for usage to get more precision.
v4:
   make usage a best-effort query and mark it as optional.
v5:
   report used-bytes and total-bytes in usage
v5.1:
   unpack usage as used-bytes and total-bytes

 qga/commands-posix.c | 18 ++++++++++++++++++
 qga/qapi-schema.json |  3 +++
 2 files changed, 21 insertions(+)

diff --git a/qga/commands-posix.c b/qga/commands-posix.c
index 0dc219dbcf..c60f10577e 100644
--- a/qga/commands-posix.c
+++ b/qga/commands-posix.c
@@ -46,6 +46,7 @@ extern char **environ;
 #include <arpa/inet.h>
 #include <sys/socket.h>
 #include <net/if.h>
+#include <sys/statvfs.h>
 
 #ifdef FIFREEZE
 #define CONFIG_FSFREEZE
@@ -1072,6 +1073,8 @@ static GuestFilesystemInfo *build_guest_fsinfo(struct FsMount *mount,
                                                Error **errp)
 {
     GuestFilesystemInfo *fs = g_malloc0(sizeof(*fs));
+    struct statvfs buf;
+    unsigned long used, nonroot_total, fr_size;
     char *devpath = g_strdup_printf("/sys/dev/block/%u:%u",
                                     mount->devmajor, mount->devminor);
 
@@ -1079,7 +1082,22 @@ static GuestFilesystemInfo *build_guest_fsinfo(struct FsMount *mount,
     fs->type = g_strdup(mount->devtype);
     build_guest_fsinfo_for_device(devpath, fs, errp);
 
+    if (statvfs(fs->mountpoint, &buf)) {
+        fs->has_total_bytes = false;
+        fs->has_used_bytes = false;
+    } else {
+        fr_size = buf.f_frsize;
+        used = buf.f_blocks - buf.f_bfree;
+        nonroot_total = used + buf.f_bavail;
+        fs->used_bytes = used * fr_size;
+        fs->total_bytes = nonroot_total * fr_size;
+
+        fs->has_total_bytes = true;
+        fs->has_used_bytes = true;
+    }
+
     g_free(devpath);
+
     return fs;
 }
 
diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json
index 17884c7c70..56ce2d08e2 100644
--- a/qga/qapi-schema.json
+++ b/qga/qapi-schema.json
@@ -846,6 +846,8 @@
 # @name: disk name
 # @mountpoint: mount point path
 # @type: file system type string
+# @used-bytes: file system used bytes (since 3.0)
+# @total-bytes: nonroot file system total bytes (since 3.0)
 # @disk: an array of disk hardware information that the volume lies on,
 #        which may be empty if the disk type is not supported
 #
@@ -853,6 +855,7 @@
 ##
 { 'struct': 'GuestFilesystemInfo',
   'data': {'name': 'str', 'mountpoint': 'str', 'type': 'str',
+           '*used-bytes': 'uint64', '*total-bytes': 'uint64',
            'disk': ['GuestDiskAddress']} }
 
 ##
-- 
2.17.0

Re: [Qemu-devel] [PATCH v5.1 1/2] qga: add mountpoint usage info to GuestFilesystemInfo

[Eric Blake]

On 06/01/2018 11:57 AM, Chen Hanxiao wrote:
> From: Chen Hanxiao <chenhanxiao@gmail.com>
> 
> This patch adds support for getting the usage of mounted
> filesystem.
> The usage of fs stored as used_bytes and total_bytes.
> It's very useful when we try to monitor guest's filesystem.
> 
> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
> Cc: Eric Blake <eblake@redhat.com>
> Cc: Daniel P. Berrangé <berrange@redhat.com>
> Signed-off-by: Chen Hanxiao <chenhanxiao@gmail.com>
> ---
> +++ b/qga/commands-posix.c
> @@ -46,6 +46,7 @@ extern char **environ;
>   #include <arpa/inet.h>
>   #include <sys/socket.h>
>   #include <net/if.h>
> +#include <sys/statvfs.h>
>   
>   #ifdef FIFREEZE
>   #define CONFIG_FSFREEZE
> @@ -1072,6 +1073,8 @@ static GuestFilesystemInfo *build_guest_fsinfo(struct FsMount *mount,
>                                                  Error **errp)
>   {
>       GuestFilesystemInfo *fs = g_malloc0(sizeof(*fs));

Because this is 0-initialized,

> +    struct statvfs buf;
> +    unsigned long used, nonroot_total, fr_size;
>       char *devpath = g_strdup_printf("/sys/dev/block/%u:%u",
>                                       mount->devmajor, mount->devminor);
>   
> @@ -1079,7 +1082,22 @@ static GuestFilesystemInfo *build_guest_fsinfo(struct FsMount *mount,
>       fs->type = g_strdup(mount->devtype);
>       build_guest_fsinfo_for_device(devpath, fs, errp);
>   
> +    if (statvfs(fs->mountpoint, &buf)) {
> +        fs->has_total_bytes = false;
> +        fs->has_used_bytes = false;

this branch is dead assignments.  You could simplify to just 'if 
(statvfs(...) == 0)' with no 'else' branch needed.

> +    } else {
> +        fr_size = buf.f_frsize;
> +        used = buf.f_blocks - buf.f_bfree;
> +        nonroot_total = used + buf.f_bavail;
> +        fs->used_bytes = used * fr_size;
> +        fs->total_bytes = nonroot_total * fr_size;
> +
> +        fs->has_total_bytes = true;
> +        fs->has_used_bytes = true;
> +    }
> +
>       g_free(devpath);
> +
>       return fs;
>   }
>   
> diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json
> index 17884c7c70..56ce2d08e2 100644
> --- a/qga/qapi-schema.json
> +++ b/qga/qapi-schema.json
> @@ -846,6 +846,8 @@
>   # @name: disk name
>   # @mountpoint: mount point path
>   # @type: file system type string
> +# @used-bytes: file system used bytes (since 3.0)
> +# @total-bytes: nonroot file system total bytes (since 3.0)

s/nonroot/non-root/

>   # @disk: an array of disk hardware information that the volume lies on,
>   #        which may be empty if the disk type is not supported
>   #
> @@ -853,6 +855,7 @@
>   ##
>   { 'struct': 'GuestFilesystemInfo',
>     'data': {'name': 'str', 'mountpoint': 'str', 'type': 'str',
> +           '*used-bytes': 'uint64', '*total-bytes': 'uint64',
>              'disk': ['GuestDiskAddress']} }
>   
>   ##
> 

Both those changes are minor, so at this point, I'm comfortable with:
Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

[Qemu-devel] [PATCH v5.1 2/2] qga-win: add driver path usage to GuestFilesystemInfo

[Chen Hanxiao]

From: Chen Hanxiao <chenhanxiao@gmail.com>

This patch adds support for getting the usage of
windows driver path.
The usage of fs stored as used_bytes and total_bytes.

Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Chen Hanxiao <chenhanxiao@gmail.com>
---
v2:
  unpack usage as used-bytes and total-bytes

 qga/commands-win32.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/qga/commands-win32.c b/qga/commands-win32.c
index 2d48394748..eff26c177e 100644
--- a/qga/commands-win32.c
+++ b/qga/commands-win32.c
@@ -670,6 +670,7 @@ static GuestFilesystemInfo *build_guest_fsinfo(char *guid, Error **errp)
     char fs_name[32];
     char vol_info[MAX_PATH+1];
     size_t len;
+    uint64_t i64FreeBytesToCaller, i64TotalBytes, i64FreeBytes;
     GuestFilesystemInfo *fs = NULL;
 
     GetVolumePathNamesForVolumeName(guid, (LPCH)&mnt, 0, &info_size);
@@ -699,10 +700,21 @@ static GuestFilesystemInfo *build_guest_fsinfo(char *guid, Error **errp)
     fs_name[sizeof(fs_name) - 1] = 0;
     fs = g_malloc(sizeof(*fs));
     fs->name = g_strdup(guid);
+    fs->has_total_bytes = false;
+    fs->has_used_bytes = false;
     if (len == 0) {
         fs->mountpoint = g_strdup("System Reserved");
     } else {
         fs->mountpoint = g_strndup(mnt_point, len);
+        if (GetDiskFreeSpaceEx(fs->mountpoint,
+                               (PULARGE_INTEGER) & i64FreeBytesToCaller,
+                               (PULARGE_INTEGER) & i64TotalBytes,
+                               (PULARGE_INTEGER) & i64FreeBytes)) {
+            fs->used_bytes = i64TotalBytes - i64FreeBytes;
+            fs->total_bytes = i64TotalBytes;
+            fs->has_total_bytes = true;
+            fs->has_used_bytes = true;
+        }
     }
     fs->type = g_strdup(fs_name);
     fs->disk = build_guest_disk_info(guid, errp);
-- 
2.17.0

Re: [Qemu-devel] [PATCH v5.1 2/2] qga-win: add driver path usage to GuestFilesystemInfo

[Eric Blake]

On 06/01/2018 11:57 AM, Chen Hanxiao wrote:
> From: Chen Hanxiao <chenhanxiao@gmail.com>
> 
> This patch adds support for getting the usage of
> windows driver path.
> The usage of fs stored as used_bytes and total_bytes.
> 
> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
> Signed-off-by: Chen Hanxiao <chenhanxiao@gmail.com>
> ---
> v2:
>    unpack usage as used-bytes and total-bytes
> 

> @@ -699,10 +700,21 @@ static GuestFilesystemInfo *build_guest_fsinfo(char *guid, Error **errp)
>       fs_name[sizeof(fs_name) - 1] = 0;
>       fs = g_malloc(sizeof(*fs));
>       fs->name = g_strdup(guid);
> +    fs->has_total_bytes = false;
> +    fs->has_used_bytes = false;

Dead assignments.

>       if (len == 0) {
>           fs->mountpoint = g_strdup("System Reserved");
>       } else {
>           fs->mountpoint = g_strndup(mnt_point, len);
> +        if (GetDiskFreeSpaceEx(fs->mountpoint,
> +                               (PULARGE_INTEGER) & i64FreeBytesToCaller,
> +                               (PULARGE_INTEGER) & i64TotalBytes,
> +                               (PULARGE_INTEGER) & i64FreeBytes)) {
> +            fs->used_bytes = i64TotalBytes - i64FreeBytes;
> +            fs->total_bytes = i64TotalBytes;
> +            fs->has_total_bytes = true;
> +            fs->has_used_bytes = true;
> +        }

I'm less familiar with Windows API, so I won't give this R-b. But now 
that I read this patch, I do have a question: since both statvfs() and 
GetDiskFreeSpaceEx() give you free bytes rather than used bytes, and you 
had to perform a subtraction to report 'used-bytes' over QGA, would it 
make any more sense to instead report 'free-bytes' over QGA for less 
subtraction work on your end and so that the QGA interface is more 
similar to existing interfaces?

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [Qemu-devel] [PATCH v5.1 2/2] qga-win: add driver path usage to GuestFilesystemInfo

[Chen Hanxiao]

At 2018-06-02 01:15:14, "Eric Blake" <eblake@redhat.com> wrote:
>On 06/01/2018 11:57 AM, Chen Hanxiao wrote:
>> From: Chen Hanxiao <chenhanxiao@gmail.com>
...
>>           fs->mountpoint = g_strdup("System Reserved");
>>       } else {
>>           fs->mountpoint = g_strndup(mnt_point, len);
>> +        if (GetDiskFreeSpaceEx(fs->mountpoint,
>> +                               (PULARGE_INTEGER) & i64FreeBytesToCaller,
>> +                               (PULARGE_INTEGER) & i64TotalBytes,
>> +                               (PULARGE_INTEGER) & i64FreeBytes)) {
>> +            fs->used_bytes = i64TotalBytes - i64FreeBytes;
>> +            fs->total_bytes = i64TotalBytes;
>> +            fs->has_total_bytes = true;
>> +            fs->has_used_bytes = true;
>> +        }
>
>I'm less familiar with Windows API, so I won't give this R-b. But now 
>that I read this patch, I do have a question: since both statvfs() and 
>GetDiskFreeSpaceEx() give you free bytes rather than used bytes, and you 
>had to perform a subtraction to report 'used-bytes' over QGA, would it 
>make any more sense to instead report 'free-bytes' over QGA for less 
>subtraction work on your end and so that the QGA interface is more 
>similar to existing interfaces?
>


All build-in tools report used-bytes for linux and windows.

For windows, GetDiskFreeSpaceEx would tell total and free bytes,
used bytes is  total - free,  which is exactly same
from the view of disk properties.

But for linux, df(1) do some calculation for showing used-bytes and free-bytes.

df(1) takes f_blocks - f_bfree for 'Used' ,
which take 'Used' + 'Available'(f_bavail)  for non-root total, 
which is different from 'Size'(f_blocks);
Then calculate  'Use%' by Used / non-root-total.

If we want get what df(3) outputs, we must report more members of struct statvfs  from STATVFS(3) ,
which windows does not need it.
And client had to do some calculation with if-else dealing with data from windows/linux.

Customs usually take 'Use%' to monitor their disks,
so I think we should report nums that can easily get the right percentage of usage from it,
for both linux and windows.

Regards,
- Chen