[PATCH 0/1] balloon: introduce 6 memory statistics

[PATCH 0/1] balloon: introduce 6 memory statistics

[zhenwei pi]

Hi,

There is a previous discussion about the new memory statistics:
https://lore.kernel.org/linux-kernel/20240423034109.1552866-1-pizhenwei@bytedance.com/T/

This may be helpful to describe the usage of these counters.

zhenwei pi (1):
  balloon: introduce 6 memory statistics

 device-types/balloon/description.tex | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

-- 
2.34.1

[PATCH 1/1] balloon: introduce 6 memory statistics

[zhenwei pi]

Expose more memory statistics of the virtual memory subsystem from
guest, it's helpful to analyze the memory performance and pressure
from host side.

Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
---
 device-types/balloon/description.tex | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/device-types/balloon/description.tex b/device-types/balloon/description.tex
index a1d9603..bc8e109 100644
--- a/device-types/balloon/description.tex
+++ b/device-types/balloon/description.tex
@@ -305,6 +305,12 @@ \subsubsection{Memory Statistics}\label{sec:Device Types / Memory Balloon Device
 #define VIRTIO_BALLOON_S_CACHES   7
 #define VIRTIO_BALLOON_S_HTLB_PGALLOC 8
 #define VIRTIO_BALLOON_S_HTLB_PGFAIL  9
+#define VIRTIO_BALLOON_S_OOM_KILL       10
+#define VIRTIO_BALLOON_S_ALLOC_STALL    11
+#define VIRTIO_BALLOON_S_ASYNC_SCAN     12
+#define VIRTIO_BALLOON_S_DIRECT_SCAN    13
+#define VIRTIO_BALLOON_S_ASYNC_RECLAIM  14
+#define VIRTIO_BALLOON_S_DIRECT_RECLAIM 15
         le16 tag;
         le64 val;
 } __attribute__((packed));
@@ -399,6 +405,24 @@ \subsubsection{Memory Statistics Tags}\label{sec:Device Types / Memory Balloon D
 
 \item[VIRTIO_BALLOON_S_HTLB_PGFAIL (9)] The number of failed hugetlb page
   allocations in the guest.
+
+\item[VIRTIO_BALLOON_S_OOM_KILL (10)] The number of Out-Of-Memory killer
+  invocations.
+
+\item[VIRTIO_BALLOON_S_ALLOC_STALL (11)] The number of stall count on memory
+  allocation.
+
+\item[VIRTIO_BALLOON_S_ASYNC_SCAN (12)] The amount of memory that has been
+  scanned asynchronously (in bytes).
+
+\item[VIRTIO_BALLOON_S_DIRECT_SCAN (13)] The amount of memory that has been
+  scanned directly (in bytes).
+
+\item[VIRTIO_BALLOON_S_ASYNC_RECLAIM (14)] The amount of memory that has been
+  reclaimed asynchronously (in bytes).
+
+\item[VIRTIO_BALLOON_S_DIRECT_RECLAIM (15)] The amount of memory that has been
+  reclaimed directly (in bytes).
 \end{description}
 
 \subsubsection{Free Page Hinting}\label{sec:Device Types / Memory Balloon Device / Device Operation / Free Page Hinting}
-- 
2.34.1

Re: [PATCH 1/1] balloon: introduce 6 memory statistics

[Michael S. Tsirkin]

On Mon, May 13, 2024 at 08:10:05PM +0800, zhenwei pi wrote:
> Expose more memory statistics of the virtual memory subsystem from
> guest, it's helpful to analyze the memory performance and pressure
> from host side.
> 
> Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>

zhenwei

when subsriving to the lists you should have received this text:


Not to do:
- copy both virtio-dev and virtio-comment mailing lists (instead, pick
  one)

did you not get this text? was this, somehow, unclear that you copied both
virtip-comment and virtio-dev on your patch?


> ---
>  device-types/balloon/description.tex | 24 ++++++++++++++++++++++++
>  1 file changed, 24 insertions(+)
> 
> diff --git a/device-types/balloon/description.tex b/device-types/balloon/description.tex
> index a1d9603..bc8e109 100644
> --- a/device-types/balloon/description.tex
> +++ b/device-types/balloon/description.tex
> @@ -305,6 +305,12 @@ \subsubsection{Memory Statistics}\label{sec:Device Types / Memory Balloon Device
>  #define VIRTIO_BALLOON_S_CACHES   7
>  #define VIRTIO_BALLOON_S_HTLB_PGALLOC 8
>  #define VIRTIO_BALLOON_S_HTLB_PGFAIL  9
> +#define VIRTIO_BALLOON_S_OOM_KILL       10
> +#define VIRTIO_BALLOON_S_ALLOC_STALL    11
> +#define VIRTIO_BALLOON_S_ASYNC_SCAN     12
> +#define VIRTIO_BALLOON_S_DIRECT_SCAN    13
> +#define VIRTIO_BALLOON_S_ASYNC_RECLAIM  14
> +#define VIRTIO_BALLOON_S_DIRECT_RECLAIM 15
>          le16 tag;
>          le64 val;
>  } __attribute__((packed));
> @@ -399,6 +405,24 @@ \subsubsection{Memory Statistics Tags}\label{sec:Device Types / Memory Balloon D
>  
>  \item[VIRTIO_BALLOON_S_HTLB_PGFAIL (9)] The number of failed hugetlb page
>    allocations in the guest.
> +
> +\item[VIRTIO_BALLOON_S_OOM_KILL (10)] The number of Out-Of-Memory killer
> +  invocations.
> +
> +\item[VIRTIO_BALLOON_S_ALLOC_STALL (11)] The number of stall count on memory
> +  allocation.
> +
> +\item[VIRTIO_BALLOON_S_ASYNC_SCAN (12)] The amount of memory that has been
> +  scanned asynchronously (in bytes).
> +
> +\item[VIRTIO_BALLOON_S_DIRECT_SCAN (13)] The amount of memory that has been
> +  scanned directly (in bytes).
> +
> +\item[VIRTIO_BALLOON_S_ASYNC_RECLAIM (14)] The amount of memory that has been
> +  reclaimed asynchronously (in bytes).
> +
> +\item[VIRTIO_BALLOON_S_DIRECT_RECLAIM (15)] The amount of memory that has been
> +  reclaimed directly (in bytes).
>  \end{description}
>  
>  \subsubsection{Free Page Hinting}\label{sec:Device Types / Memory Balloon Device / Device Operation / Free Page Hinting}
> -- 
> 2.34.1
> 

Re: Re: [PATCH 1/1] balloon: introduce 6 memory statistics

[zhenwei pi]

[-- Attachment #1: Type: text/plain, Size: 2680 bytes --]

Hi Michael,

Sorry, I received this text, but read that email from my mobile phone
carelessly.

On 5/21/24 11:50, Michael S. Tsirkin wrote:
> On Mon, May 13, 2024 at 08:10:05PM +0800, zhenwei pi wrote:
>> Expose more memory statistics of the virtual memory subsystem from
>> guest, it's helpful to analyze the memory performance and pressure
>> from host side.
>>
>> Signed-off-by: zhenwei pi
>
> zhenwei
>
> when subsriving to the lists you should have received this text:
>
>
> Not to do:
> - copy both virtio-dev and virtio-comment mailing lists (instead, pick
> one)
>
> did you not get this text? was this, somehow, unclear that you copied
both
> virtip-comment and virtio-dev on your patch?
>
>
>> ---
>> device-types/balloon/description.tex | 24 ++++++++++++++++++++++++
>> 1 file changed, 24 insertions(+)
>>
>> diff --git a/device-types/balloon/description.tex
b/device-types/balloon/description.tex
>> index a1d9603..bc8e109 100644
>> --- a/device-types/balloon/description.tex
>> +++ b/device-types/balloon/description.tex
>> @@ -305,6 +305,12 @@ \subsubsection{Memory Statistics}\label{sec:Device
Types / Memory Balloon Device
>> #define VIRTIO_BALLOON_S_CACHES 7
>> #define VIRTIO_BALLOON_S_HTLB_PGALLOC 8
>> #define VIRTIO_BALLOON_S_HTLB_PGFAIL 9
>> +#define VIRTIO_BALLOON_S_OOM_KILL 10
>> +#define VIRTIO_BALLOON_S_ALLOC_STALL 11
>> +#define VIRTIO_BALLOON_S_ASYNC_SCAN 12
>> +#define VIRTIO_BALLOON_S_DIRECT_SCAN 13
>> +#define VIRTIO_BALLOON_S_ASYNC_RECLAIM 14
>> +#define VIRTIO_BALLOON_S_DIRECT_RECLAIM 15
>> le16 tag;
>> le64 val;
>> } __attribute__((packed));
>> @@ -399,6 +405,24 @@ \subsubsection{Memory Statistics
Tags}\label{sec:Device Types / Memory Balloon D
>>
>> \item[VIRTIO_BALLOON_S_HTLB_PGFAIL (9)] The number of failed hugetlb
page
>> allocations in the guest.
>> +
>> +\item[VIRTIO_BALLOON_S_OOM_KILL (10)] The number of Out-Of-Memory
killer
>> + invocations.
>> +
>> +\item[VIRTIO_BALLOON_S_ALLOC_STALL (11)] The number of stall count on
memory
>> + allocation.
>> +
>> +\item[VIRTIO_BALLOON_S_ASYNC_SCAN (12)] The amount of memory that has
been
>> + scanned asynchronously (in bytes).
>> +
>> +\item[VIRTIO_BALLOON_S_DIRECT_SCAN (13)] The amount of memory that has
been
>> + scanned directly (in bytes).
>> +
>> +\item[VIRTIO_BALLOON_S_ASYNC_RECLAIM (14)] The amount of memory that
has been
>> + reclaimed asynchronously (in bytes).
>> +
>> +\item[VIRTIO_BALLOON_S_DIRECT_RECLAIM (15)] The amount of memory that
has been
>> + reclaimed directly (in bytes).
>> \end{description}
>>
>> \subsubsection{Free Page Hinting}\label{sec:Device Types / Memory
Balloon Device / Device Operation / Free Page Hinting}
>> --
>> 2.34.1
>>
>

-- 
zhenwei pi

[-- Attachment #2: Type: text/html, Size: 3447 bytes --]

PING: [PATCH 1/1] balloon: introduce 6 memory statistics

[zhenwei pi]

[-- Attachment #1: Type: text/plain, Size: 2842 bytes --]

On 5/21/24 12:56, zhenwei pi wrote:
> Hi Michael,
>
> Sorry, I received this text, but read that email from my mobile phone
> carelessly.
>
> On 5/21/24 11:50, Michael S. Tsirkin wrote:
>> On Mon, May 13, 2024 at 08:10:05PM +0800, zhenwei pi wrote:
>>> Expose more memory statistics of the virtual memory subsystem from
>>> guest, it's helpful to analyze the memory performance and pressure
>>> from host side.
>>>
>>> Signed-off-by: zhenwei pi
>>
>> zhenwei
>>
>> when subsriving to the lists you should have received this text:
>>
>>
>> Not to do:
>> - copy both virtio-dev and virtio-comment mailing lists (instead, pick
>> one)
>>
>> did you not get this text? was this, somehow, unclear that you copied
> both
>> virtip-comment and virtio-dev on your patch?
>>
>>
>>> ---
>>> device-types/balloon/description.tex | 24 ++++++++++++++++++++++++
>>> 1 file changed, 24 insertions(+)
>>>
>>> diff --git a/device-types/balloon/description.tex
> b/device-types/balloon/description.tex
>>> index a1d9603..bc8e109 100644
>>> --- a/device-types/balloon/description.tex
>>> +++ b/device-types/balloon/description.tex
>>> @@ -305,6 +305,12 @@ \subsubsection{Memory
> Statistics}\label{sec:Device Types / Memory Balloon Device
>>> #define VIRTIO_BALLOON_S_CACHES 7
>>> #define VIRTIO_BALLOON_S_HTLB_PGALLOC 8
>>> #define VIRTIO_BALLOON_S_HTLB_PGFAIL 9
>>> +#define VIRTIO_BALLOON_S_OOM_KILL 10
>>> +#define VIRTIO_BALLOON_S_ALLOC_STALL 11
>>> +#define VIRTIO_BALLOON_S_ASYNC_SCAN 12
>>> +#define VIRTIO_BALLOON_S_DIRECT_SCAN 13
>>> +#define VIRTIO_BALLOON_S_ASYNC_RECLAIM 14
>>> +#define VIRTIO_BALLOON_S_DIRECT_RECLAIM 15
>>> le16 tag;
>>> le64 val;
>>> } __attribute__((packed));
>>> @@ -399,6 +405,24 @@ \subsubsection{Memory Statistics
> Tags}\label{sec:Device Types / Memory Balloon D
>>>
>>> \item[VIRTIO_BALLOON_S_HTLB_PGFAIL (9)] The number of failed hugetlb
> page
>>> allocations in the guest.
>>> +
>>> +\item[VIRTIO_BALLOON_S_OOM_KILL (10)] The number of Out-Of-Memory
> killer
>>> + invocations.
>>> +
>>> +\item[VIRTIO_BALLOON_S_ALLOC_STALL (11)] The number of stall count
> on memory
>>> + allocation.
>>> +
>>> +\item[VIRTIO_BALLOON_S_ASYNC_SCAN (12)] The amount of memory that
> has been
>>> + scanned asynchronously (in bytes).
>>> +
>>> +\item[VIRTIO_BALLOON_S_DIRECT_SCAN (13)] The amount of memory that
> has been
>>> + scanned directly (in bytes).
>>> +
>>> +\item[VIRTIO_BALLOON_S_ASYNC_RECLAIM (14)] The amount of memory that
> has been
>>> + reclaimed asynchronously (in bytes).
>>> +
>>> +\item[VIRTIO_BALLOON_S_DIRECT_RECLAIM (15)] The amount of memory
> that has been
>>> + reclaimed directly (in bytes).
>>> \end{description}
>>>
>>> \subsubsection{Free Page Hinting}\label{sec:Device Types / Memory
> Balloon Device / Device Operation / Free Page Hinting}
>>> --
>>> 2.34.1
>>>
>>
>
> --
> zhenwei pi
>

-- 
zhenwei pi

[-- Attachment #2: Type: text/html, Size: 3886 bytes --]

Re: [PATCH 1/1] balloon: introduce 6 memory statistics

[Michael S. Tsirkin]

On Mon, May 13, 2024 at 08:10:05PM +0800, zhenwei pi wrote:
> Expose more memory statistics of the virtual memory subsystem from
> guest, it's helpful to analyze the memory performance and pressure
> from host side.
> 
> Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
> ---
>  device-types/balloon/description.tex | 24 ++++++++++++++++++++++++
>  1 file changed, 24 insertions(+)
> 
> diff --git a/device-types/balloon/description.tex b/device-types/balloon/description.tex
> index a1d9603..bc8e109 100644
> --- a/device-types/balloon/description.tex
> +++ b/device-types/balloon/description.tex
> @@ -305,6 +305,12 @@ \subsubsection{Memory Statistics}\label{sec:Device Types / Memory Balloon Device
>  #define VIRTIO_BALLOON_S_CACHES   7
>  #define VIRTIO_BALLOON_S_HTLB_PGALLOC 8
>  #define VIRTIO_BALLOON_S_HTLB_PGFAIL  9
> +#define VIRTIO_BALLOON_S_OOM_KILL       10
> +#define VIRTIO_BALLOON_S_ALLOC_STALL    11
> +#define VIRTIO_BALLOON_S_ASYNC_SCAN     12
> +#define VIRTIO_BALLOON_S_DIRECT_SCAN    13
> +#define VIRTIO_BALLOON_S_ASYNC_RECLAIM  14
> +#define VIRTIO_BALLOON_S_DIRECT_RECLAIM 15
>          le16 tag;
>          le64 val;
>  } __attribute__((packed));
> @@ -399,6 +405,24 @@ \subsubsection{Memory Statistics Tags}\label{sec:Device Types / Memory Balloon D
>  
>  \item[VIRTIO_BALLOON_S_HTLB_PGFAIL (9)] The number of failed hugetlb page
>    allocations in the guest.


Would it make sense to reference
https://docs.kernel.org/admin-guide/mm/concepts.html ?
you seem to be using terminology from there.
And then pls actually use terminology from there.


> +
> +\item[VIRTIO_BALLOON_S_OOM_KILL (10)] The number of Out-Of-Memory killer
> +  invocations.


What exactly are "invocations"? The number of times a task was killed
to free memory?

> +
> +\item[VIRTIO_BALLOON_S_ALLOC_STALL (11)] The number of stall count on memory
> +  allocation.
>

count and number is mostly the same.

Again what is this counting? pages? stalls? If stalls isn't this always
the same as direct reclaim?

> +\item[VIRTIO_BALLOON_S_ASYNC_SCAN (12)] The amount of memory that has been
> +  scanned asynchronously (in bytes).

scanned by what? which memory do you count as "scanned"?

> +
> +\item[VIRTIO_BALLOON_S_DIRECT_SCAN (13)] The amount of memory that has been
> +  scanned directly (in bytes).

what is "scanned directly"?


> +
> +\item[VIRTIO_BALLOON_S_ASYNC_RECLAIM (14)] The amount of memory that has been
> +  reclaimed asynchronously (in bytes).

again reclaimed by what? free? evict?

> +
> +\item[VIRTIO_BALLOON_S_DIRECT_RECLAIM (15)] The amount of memory that has been
> +  reclaimed directly (in bytes).
>  \end{description}
>  
>  \subsubsection{Free Page Hinting}\label{sec:Device Types / Memory Balloon Device / Device Operation / Free Page Hinting}
> -- 
> 2.34.1
> 

Re: Re: [PATCH 1/1] balloon: introduce 6 memory statistics

[zhenwei pi]

[-- Attachment #1: Type: text/plain, Size: 3833 bytes --]

On 5/27/24 14:37, Michael S. Tsirkin wrote:
> On Mon, May 13, 2024 at 08:10:05PM +0800, zhenwei pi wrote:
>> Expose more memory statistics of the virtual memory subsystem from
>> guest, it's helpful to analyze the memory performance and pressure
>> from host side.
>>
>> Signed-off-by: zhenwei pi
>> ---
>> device-types/balloon/description.tex | 24 ++++++++++++++++++++++++
>> 1 file changed, 24 insertions(+)
>>
>> diff --git a/device-types/balloon/description.tex
b/device-types/balloon/description.tex
>> index a1d9603..bc8e109 100644
>> --- a/device-types/balloon/description.tex
>> +++ b/device-types/balloon/description.tex
>> @@ -305,6 +305,12 @@ \subsubsection{Memory Statistics}\label{sec:Device
Types / Memory Balloon Device
>> #define VIRTIO_BALLOON_S_CACHES 7
>> #define VIRTIO_BALLOON_S_HTLB_PGALLOC 8
>> #define VIRTIO_BALLOON_S_HTLB_PGFAIL 9
>> +#define VIRTIO_BALLOON_S_OOM_KILL 10
>> +#define VIRTIO_BALLOON_S_ALLOC_STALL 11
>> +#define VIRTIO_BALLOON_S_ASYNC_SCAN 12
>> +#define VIRTIO_BALLOON_S_DIRECT_SCAN 13
>> +#define VIRTIO_BALLOON_S_ASYNC_RECLAIM 14
>> +#define VIRTIO_BALLOON_S_DIRECT_RECLAIM 15
>> le16 tag;
>> le64 val;
>> } __attribute__((packed));
>> @@ -399,6 +405,24 @@ \subsubsection{Memory Statistics
Tags}\label{sec:Device Types / Memory Balloon D
>>
>> \item[VIRTIO_BALLOON_S_HTLB_PGFAIL (9)] The number of failed hugetlb
page
>> allocations in the guest.
>
>
> Would it make sense to reference
> https://docs.kernel.org/admin-guide/mm/concepts.html ?
> you seem to be using terminology from there.
> And then pls actually use terminology from there.
>

OK.

>
>> +
>> +\item[VIRTIO_BALLOON_S_OOM_KILL (10)] The number of Out-Of-Memory
killer
>> + invocations.
>
>
> What exactly are "invocations"? The number of times a task was killed
> to free memory?
>

What about:
The count of OOM killer invocations increases when the kernel invokes
the OOM killer. The OOM killer selects a task to terminate in order to
free up memory.

>> +
>> +\item[VIRTIO_BALLOON_S_ALLOC_STALL (11)] The number of stall count on
memory
>> + allocation.
>>
>
> count and number is mostly the same.
>
> Again what is this counting? pages? stalls? If stalls isn't this always
> the same as direct reclaim?
>

Once the memory pressure gets critical, a task hits ALLOC STALL, and
reclaims pages in batch directly. So the ALLOC STALL is not the same as
direct reclaim.

What about:
The count of stalls on memory allocation.


>> +\item[VIRTIO_BALLOON_S_ASYNC_SCAN (12)] The amount of memory that has
been
>> + scanned asynchronously (in bytes).
>
> scanned by what? which memory do you count as "scanned"?
>

What about:
The amount of memory scanned asynchronously by the kernel background
task (in bytes).

>> +
>> +\item[VIRTIO_BALLOON_S_DIRECT_SCAN (13)] The amount of memory that has
been
>> + scanned directly (in bytes).
>
> what is "scanned directly"?
>

The amount of memory scanned directly by the running task (in bytes).

>
>> +
>> +\item[VIRTIO_BALLOON_S_ASYNC_RECLAIM (14)] The amount of memory that
has been
>> + reclaimed asynchronously (in bytes).
>
> again reclaimed by what? free? evict?
>

The clean pages could be just freed, but the dirty pages need to write
back to the the backing storage device.('evict' is used in
https://docs.kernel.org/admin-guide/mm/concepts.html). They are
considered as 'reclaim'.

What about:
The amount of memory reclaimed asynchronously by the kernel background
task (in bytes).

>> +
>> +\item[VIRTIO_BALLOON_S_DIRECT_RECLAIM (15)] The amount of memory that
has been
>> + reclaimed directly (in bytes).
>> \end{description}
>>

What about:
The amount of memory reclaimed directly by the running task (in bytes).

>> \subsubsection{Free Page Hinting}\label{sec:Device Types / Memory
Balloon Device / Device Operation / Free Page Hinting}
>> --
>> 2.34.1
>>
>

-- 
zhenwei pi

[-- Attachment #2: Type: text/html, Size: 5037 bytes --]

PING: [PATCH 0/1] balloon: introduce 6 memory statistics

[zhenwei pi]

[-- Attachment #1: Type: text/plain, Size: 504 bytes --]

Hi,

This patch seems to be ignored over 1 week...


On 5/13/24 20:10, zhenwei pi wrote:
> Hi,
>
> There is a previous discussion about the new memory statistics:
>
https://lore.kernel.org/linux-kernel/20240423034109.1552866-1-pizhenwei@bytedance.com/T/
>
> This may be helpful to describe the usage of these counters.
>
> zhenwei pi (1):
> balloon: introduce 6 memory statistics
>
> device-types/balloon/description.tex | 24 ++++++++++++++++++++++++
> 1 file changed, 24 insertions(+)
>

-- 
zhenwei pi

[-- Attachment #2: Type: text/html, Size: 746 bytes --]

Re: [PATCH 0/1] balloon: introduce 6 memory statistics

[Michael S. Tsirkin]

On Mon, May 13, 2024 at 08:10:04PM +0800, zhenwei pi wrote:
> Hi,
> 
> There is a previous discussion about the new memory statistics:
> https://lore.kernel.org/linux-kernel/20240423034109.1552866-1-pizhenwei@bytedance.com/T/

> This may be helpful to describe the usage of these counters.

It's ok to add a link but you really should include the
motivation with the patch, and preferably in the commit log
not the cover letter.


I think the qemu patches actually include a lot of good info
about the statistics collected. But they also raise a lot
of questions that would be of interest to the TC.



> 
> zhenwei pi (1):
>   balloon: introduce 6 memory statistics
> 
>  device-types/balloon/description.tex | 24 ++++++++++++++++++++++++
>  1 file changed, 24 insertions(+)
> 
> -- 
> 2.34.1
>