[PATCH v2] mm: show total hugetlb memory consumption in /proc/meminfo

[PATCH v2] mm: show total hugetlb memory consumption in /proc/meminfo

[Roman Gushchin]

Currently we display some hugepage statistics (total, free, etc)
in /proc/meminfo, but only for default hugepage size (e.g. 2Mb).

If hugepages of different sizes are used (like 2Mb and 1Gb on x86-64),
/proc/meminfo output can be confusing, as non-default sized hugepages
are not reflected at all, and there are no signs that they are
existing and consuming system memory.

To solve this problem, let's display the total amount of memory,
consumed by hugetlb pages of all sized (both free and used).
Let's call it "Hugetlb", and display size in kB to match generic
/proc/meminfo style.

For example, (1024 2Mb pages and 2 1Gb pages are pre-allocated):
  $ cat /proc/meminfo
  MemTotal:        8168984 kB
  MemFree:         3789276 kB
  <...>
  CmaFree:               0 kB
  HugePages_Total:    1024
  HugePages_Free:     1024
  HugePages_Rsvd:        0
  HugePages_Surp:        0
  Hugepagesize:       2048 kB
  Hugetlb:         4194304 kB
  DirectMap4k:       32632 kB
  DirectMap2M:     4161536 kB
  DirectMap1G:     6291456 kB

Also, this patch updates corresponding docs to reflect
Hugetlb entry meaning and difference between Hugetlb and
HugePages_Total * Hugepagesize.

Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Cc: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: David Rientjes <rientjes@google.com>
Cc: kernel-team@fb.com
Cc: linux-mm@kvack.org
Cc: linux-kernel@vger.kernel.org
---
 Documentation/vm/hugetlbpage.txt | 27 ++++++++++++++++++---------
 mm/hugetlb.c                     | 36 ++++++++++++++++++++++++------------
 2 files changed, 42 insertions(+), 21 deletions(-)

diff --git a/Documentation/vm/hugetlbpage.txt b/Documentation/vm/hugetlbpage.txt
index 59cbc803aad6..faf077d50d42 100644
--- a/Documentation/vm/hugetlbpage.txt
+++ b/Documentation/vm/hugetlbpage.txt
@@ -20,19 +20,20 @@ options.
 
 The /proc/meminfo file provides information about the total number of
 persistent hugetlb pages in the kernel's huge page pool.  It also displays
-information about the number of free, reserved and surplus huge pages and the
-default huge page size.  The huge page size is needed for generating the
-proper alignment and size of the arguments to system calls that map huge page
-regions.
+default huge page size and information about the number of free, reserved
+and surplus huge pages in the pool of huge pages of default size.
+The huge page size is needed for generating the proper alignment and
+size of the arguments to system calls that map huge page regions.
 
 The output of "cat /proc/meminfo" will include lines like:
 
 .....
-HugePages_Total: vvv
-HugePages_Free:  www
-HugePages_Rsvd:  xxx
-HugePages_Surp:  yyy
-Hugepagesize:    zzz kB
+HugePages_Total: uuu
+HugePages_Free:  vvv
+HugePages_Rsvd:  www
+HugePages_Surp:  xxx
+Hugepagesize:    yyy kB
+Hugetlb:         zzz kB
 
 where:
 HugePages_Total is the size of the pool of huge pages.
@@ -47,6 +48,14 @@ HugePages_Surp  is short for "surplus," and is the number of huge pages in
                 the pool above the value in /proc/sys/vm/nr_hugepages. The
                 maximum number of surplus huge pages is controlled by
                 /proc/sys/vm/nr_overcommit_hugepages.
+Hugepagesize    is the default hugepage size (in Kb).
+Hugetlb         is the total amount of memory (in kB), consumed by huge
+                pages of all sizes.
+                If huge pages of different sizes are in use, this number
+                will exceed HugePages_Total * Hugepagesize. To get more
+                detailed information, please, refer to
+                /sys/kernel/mm/hugepages (described below).
+
 
 /proc/filesystems should also show a filesystem of type "hugetlbfs" configured
 in the kernel.
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 4b3bbd2980bb..672377e6de9f 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -2973,20 +2973,32 @@ int hugetlb_overcommit_handler(struct ctl_table *table, int write,
 
 void hugetlb_report_meminfo(struct seq_file *m)
 {
-	struct hstate *h = &default_hstate;
+	struct hstate *h;
+	unsigned long total = 0;
+
 	if (!hugepages_supported())
 		return;
-	seq_printf(m,
-			"HugePages_Total:   %5lu\n"
-			"HugePages_Free:    %5lu\n"
-			"HugePages_Rsvd:    %5lu\n"
-			"HugePages_Surp:    %5lu\n"
-			"Hugepagesize:   %8lu kB\n",
-			h->nr_huge_pages,
-			h->free_huge_pages,
-			h->resv_huge_pages,
-			h->surplus_huge_pages,
-			1UL << (huge_page_order(h) + PAGE_SHIFT - 10));
+
+	for_each_hstate(h) {
+		unsigned long count = h->nr_huge_pages;
+
+		total += (PAGE_SIZE << huge_page_order(h)) * count;
+
+		if (h == &default_hstate)
+			seq_printf(m,
+				   "HugePages_Total:   %5lu\n"
+				   "HugePages_Free:    %5lu\n"
+				   "HugePages_Rsvd:    %5lu\n"
+				   "HugePages_Surp:    %5lu\n"
+				   "Hugepagesize:   %8lu kB\n",
+				   count,
+				   h->free_huge_pages,
+				   h->resv_huge_pages,
+				   h->surplus_huge_pages,
+				   (PAGE_SIZE << huge_page_order(h)) / 1024);
+	}
+
+	seq_printf(m, "Hugetlb:        %8lu kB\n", total / 1024);
 }
 
 int hugetlb_report_node_meminfo(int nid, char *buf)
-- 
2.13.6

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: show total hugetlb memory consumption in /proc/meminfo

[Michal Hocko]

On Wed 15-11-17 23:14:09, Roman Gushchin wrote:
> Currently we display some hugepage statistics (total, free, etc)
> in /proc/meminfo, but only for default hugepage size (e.g. 2Mb).
> 
> If hugepages of different sizes are used (like 2Mb and 1Gb on x86-64),
> /proc/meminfo output can be confusing, as non-default sized hugepages
> are not reflected at all, and there are no signs that they are
> existing and consuming system memory.
> 
> To solve this problem, let's display the total amount of memory,
> consumed by hugetlb pages of all sized (both free and used).
> Let's call it "Hugetlb", and display size in kB to match generic
> /proc/meminfo style.
> 
> For example, (1024 2Mb pages and 2 1Gb pages are pre-allocated):
>   $ cat /proc/meminfo
>   MemTotal:        8168984 kB
>   MemFree:         3789276 kB
>   <...>
>   CmaFree:               0 kB
>   HugePages_Total:    1024
>   HugePages_Free:     1024
>   HugePages_Rsvd:        0
>   HugePages_Surp:        0
>   Hugepagesize:       2048 kB
>   Hugetlb:         4194304 kB
>   DirectMap4k:       32632 kB
>   DirectMap2M:     4161536 kB
>   DirectMap1G:     6291456 kB
> 
> Also, this patch updates corresponding docs to reflect
> Hugetlb entry meaning and difference between Hugetlb and
> HugePages_Total * Hugepagesize.
> 
> Signed-off-by: Roman Gushchin <guro@fb.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Michal Hocko <mhocko@suse.com>
> Cc: Johannes Weiner <hannes@cmpxchg.org>
> Cc: Mike Kravetz <mike.kravetz@oracle.com>
> Cc: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
> Cc: Andrea Arcangeli <aarcange@redhat.com>
> Cc: Dave Hansen <dave.hansen@intel.com>
> Cc: David Rientjes <rientjes@google.com>
> Cc: kernel-team@fb.com
> Cc: linux-mm@kvack.org
> Cc: linux-kernel@vger.kernel.org

Acked-by: Michal Hocko <mhocko@suse.com>

Thanks!

> ---
>  Documentation/vm/hugetlbpage.txt | 27 ++++++++++++++++++---------
>  mm/hugetlb.c                     | 36 ++++++++++++++++++++++++------------
>  2 files changed, 42 insertions(+), 21 deletions(-)
> 
> diff --git a/Documentation/vm/hugetlbpage.txt b/Documentation/vm/hugetlbpage.txt
> index 59cbc803aad6..faf077d50d42 100644
> --- a/Documentation/vm/hugetlbpage.txt
> +++ b/Documentation/vm/hugetlbpage.txt
> @@ -20,19 +20,20 @@ options.
>  
>  The /proc/meminfo file provides information about the total number of
>  persistent hugetlb pages in the kernel's huge page pool.  It also displays
> -information about the number of free, reserved and surplus huge pages and the
> -default huge page size.  The huge page size is needed for generating the
> -proper alignment and size of the arguments to system calls that map huge page
> -regions.
> +default huge page size and information about the number of free, reserved
> +and surplus huge pages in the pool of huge pages of default size.
> +The huge page size is needed for generating the proper alignment and
> +size of the arguments to system calls that map huge page regions.
>  
>  The output of "cat /proc/meminfo" will include lines like:
>  
>  .....
> -HugePages_Total: vvv
> -HugePages_Free:  www
> -HugePages_Rsvd:  xxx
> -HugePages_Surp:  yyy
> -Hugepagesize:    zzz kB
> +HugePages_Total: uuu
> +HugePages_Free:  vvv
> +HugePages_Rsvd:  www
> +HugePages_Surp:  xxx
> +Hugepagesize:    yyy kB
> +Hugetlb:         zzz kB
>  
>  where:
>  HugePages_Total is the size of the pool of huge pages.
> @@ -47,6 +48,14 @@ HugePages_Surp  is short for "surplus," and is the number of huge pages in
>                  the pool above the value in /proc/sys/vm/nr_hugepages. The
>                  maximum number of surplus huge pages is controlled by
>                  /proc/sys/vm/nr_overcommit_hugepages.
> +Hugepagesize    is the default hugepage size (in Kb).
> +Hugetlb         is the total amount of memory (in kB), consumed by huge
> +                pages of all sizes.
> +                If huge pages of different sizes are in use, this number
> +                will exceed HugePages_Total * Hugepagesize. To get more
> +                detailed information, please, refer to
> +                /sys/kernel/mm/hugepages (described below).
> +
>  
>  /proc/filesystems should also show a filesystem of type "hugetlbfs" configured
>  in the kernel.
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index 4b3bbd2980bb..672377e6de9f 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -2973,20 +2973,32 @@ int hugetlb_overcommit_handler(struct ctl_table *table, int write,
>  
>  void hugetlb_report_meminfo(struct seq_file *m)
>  {
> -	struct hstate *h = &default_hstate;
> +	struct hstate *h;
> +	unsigned long total = 0;
> +
>  	if (!hugepages_supported())
>  		return;
> -	seq_printf(m,
> -			"HugePages_Total:   %5lu\n"
> -			"HugePages_Free:    %5lu\n"
> -			"HugePages_Rsvd:    %5lu\n"
> -			"HugePages_Surp:    %5lu\n"
> -			"Hugepagesize:   %8lu kB\n",
> -			h->nr_huge_pages,
> -			h->free_huge_pages,
> -			h->resv_huge_pages,
> -			h->surplus_huge_pages,
> -			1UL << (huge_page_order(h) + PAGE_SHIFT - 10));
> +
> +	for_each_hstate(h) {
> +		unsigned long count = h->nr_huge_pages;
> +
> +		total += (PAGE_SIZE << huge_page_order(h)) * count;
> +
> +		if (h == &default_hstate)
> +			seq_printf(m,
> +				   "HugePages_Total:   %5lu\n"
> +				   "HugePages_Free:    %5lu\n"
> +				   "HugePages_Rsvd:    %5lu\n"
> +				   "HugePages_Surp:    %5lu\n"
> +				   "Hugepagesize:   %8lu kB\n",
> +				   count,
> +				   h->free_huge_pages,
> +				   h->resv_huge_pages,
> +				   h->surplus_huge_pages,
> +				   (PAGE_SIZE << huge_page_order(h)) / 1024);
> +	}
> +
> +	seq_printf(m, "Hugetlb:        %8lu kB\n", total / 1024);
>  }
>  
>  int hugetlb_report_node_meminfo(int nid, char *buf)
> -- 
> 2.13.6
> 

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: show total hugetlb memory consumption in /proc/meminfo

[Johannes Weiner]

On Wed, Nov 15, 2017 at 11:14:09PM +0000, Roman Gushchin wrote:
> Currently we display some hugepage statistics (total, free, etc)
> in /proc/meminfo, but only for default hugepage size (e.g. 2Mb).
> 
> If hugepages of different sizes are used (like 2Mb and 1Gb on x86-64),
> /proc/meminfo output can be confusing, as non-default sized hugepages
> are not reflected at all, and there are no signs that they are
> existing and consuming system memory.
> 
> To solve this problem, let's display the total amount of memory,
> consumed by hugetlb pages of all sized (both free and used).
> Let's call it "Hugetlb", and display size in kB to match generic
> /proc/meminfo style.
> 
> For example, (1024 2Mb pages and 2 1Gb pages are pre-allocated):
>   $ cat /proc/meminfo
>   MemTotal:        8168984 kB
>   MemFree:         3789276 kB
>   <...>
>   CmaFree:               0 kB
>   HugePages_Total:    1024
>   HugePages_Free:     1024
>   HugePages_Rsvd:        0
>   HugePages_Surp:        0
>   Hugepagesize:       2048 kB
>   Hugetlb:         4194304 kB
>   DirectMap4k:       32632 kB
>   DirectMap2M:     4161536 kB
>   DirectMap1G:     6291456 kB
> 
> Also, this patch updates corresponding docs to reflect
> Hugetlb entry meaning and difference between Hugetlb and
> HugePages_Total * Hugepagesize.
> 
> Signed-off-by: Roman Gushchin <guro@fb.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Michal Hocko <mhocko@suse.com>
> Cc: Johannes Weiner <hannes@cmpxchg.org>
> Cc: Mike Kravetz <mike.kravetz@oracle.com>
> Cc: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
> Cc: Andrea Arcangeli <aarcange@redhat.com>
> Cc: Dave Hansen <dave.hansen@intel.com>
> Cc: David Rientjes <rientjes@google.com>
> Cc: kernel-team@fb.com
> Cc: linux-mm@kvack.org
> Cc: linux-kernel@vger.kernel.org

Acked-by: Johannes Weiner <hannes@cmpxchg.org>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: show total hugetlb memory consumption in /proc/meminfo

[David Rientjes]

On Wed, 15 Nov 2017, Roman Gushchin wrote:

> Currently we display some hugepage statistics (total, free, etc)
> in /proc/meminfo, but only for default hugepage size (e.g. 2Mb).
> 
> If hugepages of different sizes are used (like 2Mb and 1Gb on x86-64),
> /proc/meminfo output can be confusing, as non-default sized hugepages
> are not reflected at all, and there are no signs that they are
> existing and consuming system memory.
> 
> To solve this problem, let's display the total amount of memory,
> consumed by hugetlb pages of all sized (both free and used).
> Let's call it "Hugetlb", and display size in kB to match generic
> /proc/meminfo style.
> 
> For example, (1024 2Mb pages and 2 1Gb pages are pre-allocated):
>   $ cat /proc/meminfo
>   MemTotal:        8168984 kB
>   MemFree:         3789276 kB
>   <...>
>   CmaFree:               0 kB
>   HugePages_Total:    1024
>   HugePages_Free:     1024
>   HugePages_Rsvd:        0
>   HugePages_Surp:        0
>   Hugepagesize:       2048 kB
>   Hugetlb:         4194304 kB
>   DirectMap4k:       32632 kB
>   DirectMap2M:     4161536 kB
>   DirectMap1G:     6291456 kB
> 
> Also, this patch updates corresponding docs to reflect
> Hugetlb entry meaning and difference between Hugetlb and
> HugePages_Total * Hugepagesize.
> 
> Signed-off-by: Roman Gushchin <guro@fb.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Michal Hocko <mhocko@suse.com>
> Cc: Johannes Weiner <hannes@cmpxchg.org>
> Cc: Mike Kravetz <mike.kravetz@oracle.com>
> Cc: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
> Cc: Andrea Arcangeli <aarcange@redhat.com>
> Cc: Dave Hansen <dave.hansen@intel.com>
> Cc: David Rientjes <rientjes@google.com>
> Cc: kernel-team@fb.com
> Cc: linux-mm@kvack.org
> Cc: linux-kernel@vger.kernel.org

Acked-by: David Rientjes <rientjes@google.com>

Nice!

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: show total hugetlb memory consumption in /proc/meminfo

[Andrew Morton]

On Wed, 15 Nov 2017 23:14:09 +0000 Roman Gushchin <guro@fb.com> wrote:

> Currently we display some hugepage statistics (total, free, etc)
> in /proc/meminfo, but only for default hugepage size (e.g. 2Mb).
> 
> If hugepages of different sizes are used (like 2Mb and 1Gb on x86-64),
> /proc/meminfo output can be confusing, as non-default sized hugepages
> are not reflected at all, and there are no signs that they are
> existing and consuming system memory.
> 
> To solve this problem, let's display the total amount of memory,
> consumed by hugetlb pages of all sized (both free and used).
> Let's call it "Hugetlb", and display size in kB to match generic
> /proc/meminfo style.
> 
> For example, (1024 2Mb pages and 2 1Gb pages are pre-allocated):
>   $ cat /proc/meminfo
>   MemTotal:        8168984 kB
>   MemFree:         3789276 kB
>   <...>
>   CmaFree:               0 kB
>   HugePages_Total:    1024
>   HugePages_Free:     1024
>   HugePages_Rsvd:        0
>   HugePages_Surp:        0
>   Hugepagesize:       2048 kB
>   Hugetlb:         4194304 kB
>   DirectMap4k:       32632 kB
>   DirectMap2M:     4161536 kB
>   DirectMap1G:     6291456 kB
> 
> Also, this patch updates corresponding docs to reflect
> Hugetlb entry meaning and difference between Hugetlb and
> HugePages_Total * Hugepagesize.
> 
> ...
>
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -2973,20 +2973,32 @@ int hugetlb_overcommit_handler(struct ctl_table *table, int write,
>  
>  void hugetlb_report_meminfo(struct seq_file *m)
>  {
> -	struct hstate *h = &default_hstate;
> +	struct hstate *h;
> +	unsigned long total = 0;
> +
>  	if (!hugepages_supported())
>  		return;
> -	seq_printf(m,
> -			"HugePages_Total:   %5lu\n"
> -			"HugePages_Free:    %5lu\n"
> -			"HugePages_Rsvd:    %5lu\n"
> -			"HugePages_Surp:    %5lu\n"
> -			"Hugepagesize:   %8lu kB\n",
> -			h->nr_huge_pages,
> -			h->free_huge_pages,
> -			h->resv_huge_pages,
> -			h->surplus_huge_pages,
> -			1UL << (huge_page_order(h) + PAGE_SHIFT - 10));
> +
> +	for_each_hstate(h) {
> +		unsigned long count = h->nr_huge_pages;
> +
> +		total += (PAGE_SIZE << huge_page_order(h)) * count;
> +
> +		if (h == &default_hstate)

I'm not understanding this test.  Are we assuming that default_hstate
always refers to the highest-index hstate?  If so why, and is that
valid?

> +			seq_printf(m,
> +				   "HugePages_Total:   %5lu\n"
> +				   "HugePages_Free:    %5lu\n"
> +				   "HugePages_Rsvd:    %5lu\n"
> +				   "HugePages_Surp:    %5lu\n"
> +				   "Hugepagesize:   %8lu kB\n",
> +				   count,
> +				   h->free_huge_pages,
> +				   h->resv_huge_pages,
> +				   h->surplus_huge_pages,
> +				   (PAGE_SIZE << huge_page_order(h)) / 1024);
> +	}
> +
> +	seq_printf(m, "Hugetlb:        %8lu kB\n", total / 1024);
>  }

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: show total hugetlb memory consumption in /proc/meminfo

[Mike Kravetz]

On 11/20/2017 04:51 PM, Andrew Morton wrote:
> On Wed, 15 Nov 2017 23:14:09 +0000 Roman Gushchin <guro@fb.com> wrote:
> 
>> Currently we display some hugepage statistics (total, free, etc)
>> in /proc/meminfo, but only for default hugepage size (e.g. 2Mb).
>>
>> If hugepages of different sizes are used (like 2Mb and 1Gb on x86-64),
>> /proc/meminfo output can be confusing, as non-default sized hugepages
>> are not reflected at all, and there are no signs that they are
>> existing and consuming system memory.
>>
>> To solve this problem, let's display the total amount of memory,
>> consumed by hugetlb pages of all sized (both free and used).
>> Let's call it "Hugetlb", and display size in kB to match generic
>> /proc/meminfo style.
>>
>> For example, (1024 2Mb pages and 2 1Gb pages are pre-allocated):
>>   $ cat /proc/meminfo
>>   MemTotal:        8168984 kB
>>   MemFree:         3789276 kB
>>   <...>
>>   CmaFree:               0 kB
>>   HugePages_Total:    1024
>>   HugePages_Free:     1024
>>   HugePages_Rsvd:        0
>>   HugePages_Surp:        0
>>   Hugepagesize:       2048 kB
>>   Hugetlb:         4194304 kB
>>   DirectMap4k:       32632 kB
>>   DirectMap2M:     4161536 kB
>>   DirectMap1G:     6291456 kB
>>
>> Also, this patch updates corresponding docs to reflect
>> Hugetlb entry meaning and difference between Hugetlb and
>> HugePages_Total * Hugepagesize.
>>
>> ...
>>
>> --- a/mm/hugetlb.c
>> +++ b/mm/hugetlb.c
>> @@ -2973,20 +2973,32 @@ int hugetlb_overcommit_handler(struct ctl_table *table, int write,
>>  
>>  void hugetlb_report_meminfo(struct seq_file *m)
>>  {
>> -	struct hstate *h = &default_hstate;
>> +	struct hstate *h;
>> +	unsigned long total = 0;
>> +
>>  	if (!hugepages_supported())
>>  		return;
>> -	seq_printf(m,
>> -			"HugePages_Total:   %5lu\n"
>> -			"HugePages_Free:    %5lu\n"
>> -			"HugePages_Rsvd:    %5lu\n"
>> -			"HugePages_Surp:    %5lu\n"
>> -			"Hugepagesize:   %8lu kB\n",
>> -			h->nr_huge_pages,
>> -			h->free_huge_pages,
>> -			h->resv_huge_pages,
>> -			h->surplus_huge_pages,
>> -			1UL << (huge_page_order(h) + PAGE_SHIFT - 10));
>> +
>> +	for_each_hstate(h) {
>> +		unsigned long count = h->nr_huge_pages;
>> +
>> +		total += (PAGE_SIZE << huge_page_order(h)) * count;
>> +
>> +		if (h == &default_hstate)
> 
> I'm not understanding this test.  Are we assuming that default_hstate
> always refers to the highest-index hstate?  If so why, and is that
> valid?

Actually default_hstate is defined as:

#define default_hstate (hstates[default_hstate_idx])

default_hstate_idx is set during hugetlb_init based upon default_hstate_size
which defaults to HPAGE_SIZE.  However, it can be overridden by the kernel
command line argument "default_hugepagesz=<size>".

By definition and history /proc/meminfo lists information on the default
huge page size.  This code is looping through all hstates to get the total
memory consumed by hugetlb pages for the new "Hugetlb" field.  When it gets
to the default huge page size, it prints the historic fields.

Hope that helps,
-- 
Mike Kravetz

> 
>> +			seq_printf(m,
>> +				   "HugePages_Total:   %5lu\n"
>> +				   "HugePages_Free:    %5lu\n"
>> +				   "HugePages_Rsvd:    %5lu\n"
>> +				   "HugePages_Surp:    %5lu\n"
>> +				   "Hugepagesize:   %8lu kB\n",
>> +				   count,
>> +				   h->free_huge_pages,
>> +				   h->resv_huge_pages,
>> +				   h->surplus_huge_pages,
>> +				   (PAGE_SIZE << huge_page_order(h)) / 1024);
>> +	}
>> +
>> +	seq_printf(m, "Hugetlb:        %8lu kB\n", total / 1024);
>>  }
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: show total hugetlb memory consumption in /proc/meminfo

[Michal Hocko]

On Mon 20-11-17 16:51:10, Andrew Morton wrote:
> On Wed, 15 Nov 2017 23:14:09 +0000 Roman Gushchin <guro@fb.com> wrote:
> 
> > Currently we display some hugepage statistics (total, free, etc)
> > in /proc/meminfo, but only for default hugepage size (e.g. 2Mb).
> > 
> > If hugepages of different sizes are used (like 2Mb and 1Gb on x86-64),
> > /proc/meminfo output can be confusing, as non-default sized hugepages
> > are not reflected at all, and there are no signs that they are
> > existing and consuming system memory.
> > 
> > To solve this problem, let's display the total amount of memory,
> > consumed by hugetlb pages of all sized (both free and used).
> > Let's call it "Hugetlb", and display size in kB to match generic
> > /proc/meminfo style.
> > 
> > For example, (1024 2Mb pages and 2 1Gb pages are pre-allocated):
> >   $ cat /proc/meminfo
> >   MemTotal:        8168984 kB
> >   MemFree:         3789276 kB
> >   <...>
> >   CmaFree:               0 kB
> >   HugePages_Total:    1024
> >   HugePages_Free:     1024
> >   HugePages_Rsvd:        0
> >   HugePages_Surp:        0
> >   Hugepagesize:       2048 kB
> >   Hugetlb:         4194304 kB
> >   DirectMap4k:       32632 kB
> >   DirectMap2M:     4161536 kB
> >   DirectMap1G:     6291456 kB
> > 
> > Also, this patch updates corresponding docs to reflect
> > Hugetlb entry meaning and difference between Hugetlb and
> > HugePages_Total * Hugepagesize.
> > 
> > ...
> >
> > --- a/mm/hugetlb.c
> > +++ b/mm/hugetlb.c
> > @@ -2973,20 +2973,32 @@ int hugetlb_overcommit_handler(struct ctl_table *table, int write,
> >  
> >  void hugetlb_report_meminfo(struct seq_file *m)
> >  {
> > -	struct hstate *h = &default_hstate;
> > +	struct hstate *h;
> > +	unsigned long total = 0;
> > +
> >  	if (!hugepages_supported())
> >  		return;
> > -	seq_printf(m,
> > -			"HugePages_Total:   %5lu\n"
> > -			"HugePages_Free:    %5lu\n"
> > -			"HugePages_Rsvd:    %5lu\n"
> > -			"HugePages_Surp:    %5lu\n"
> > -			"Hugepagesize:   %8lu kB\n",
> > -			h->nr_huge_pages,
> > -			h->free_huge_pages,
> > -			h->resv_huge_pages,
> > -			h->surplus_huge_pages,
> > -			1UL << (huge_page_order(h) + PAGE_SHIFT - 10));
> > +
> > +	for_each_hstate(h) {
> > +		unsigned long count = h->nr_huge_pages;
> > +
> > +		total += (PAGE_SIZE << huge_page_order(h)) * count;
> > +
> > +		if (h == &default_hstate)
> 
> I'm not understanding this test.  Are we assuming that default_hstate
> always refers to the highest-index hstate?  If so why, and is that
> valid?

The whole point of this checks is to provide hugetlb detailed stats _only_
for the default hstate because that is what we have been doing
traditionally. The loop is there only to gather total amount and display
it separately.

> > +			seq_printf(m,
> > +				   "HugePages_Total:   %5lu\n"
> > +				   "HugePages_Free:    %5lu\n"
> > +				   "HugePages_Rsvd:    %5lu\n"
> > +				   "HugePages_Surp:    %5lu\n"
> > +				   "Hugepagesize:   %8lu kB\n",
> > +				   count,
> > +				   h->free_huge_pages,
> > +				   h->resv_huge_pages,
> > +				   h->surplus_huge_pages,
> > +				   (PAGE_SIZE << huge_page_order(h)) / 1024);
> > +	}
> > +
> > +	seq_printf(m, "Hugetlb:        %8lu kB\n", total / 1024);
> >  }
> 

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: show total hugetlb memory consumption in /proc/meminfo

[Roman Gushchin]

On Mon, Nov 20, 2017 at 04:51:10PM -0800, Andrew Morton wrote:
> On Wed, 15 Nov 2017 23:14:09 +0000 Roman Gushchin <guro@fb.com> wrote:
> 
> > Currently we display some hugepage statistics (total, free, etc)
> > in /proc/meminfo, but only for default hugepage size (e.g. 2Mb).
> > 
> > If hugepages of different sizes are used (like 2Mb and 1Gb on x86-64),
> > /proc/meminfo output can be confusing, as non-default sized hugepages
> > are not reflected at all, and there are no signs that they are
> > existing and consuming system memory.
> > 
> > To solve this problem, let's display the total amount of memory,
> > consumed by hugetlb pages of all sized (both free and used).
> > Let's call it "Hugetlb", and display size in kB to match generic
> > /proc/meminfo style.
> > 
> > For example, (1024 2Mb pages and 2 1Gb pages are pre-allocated):
> >   $ cat /proc/meminfo
> >   MemTotal:        8168984 kB
> >   MemFree:         3789276 kB
> >   <...>
> >   CmaFree:               0 kB
> >   HugePages_Total:    1024
> >   HugePages_Free:     1024
> >   HugePages_Rsvd:        0
> >   HugePages_Surp:        0
> >   Hugepagesize:       2048 kB
> >   Hugetlb:         4194304 kB
> >   DirectMap4k:       32632 kB
> >   DirectMap2M:     4161536 kB
> >   DirectMap1G:     6291456 kB
> > 
> > Also, this patch updates corresponding docs to reflect
> > Hugetlb entry meaning and difference between Hugetlb and
> > HugePages_Total * Hugepagesize.
> > 
> > ...
> >
> > --- a/mm/hugetlb.c
> > +++ b/mm/hugetlb.c
> > @@ -2973,20 +2973,32 @@ int hugetlb_overcommit_handler(struct ctl_table *table, int write,
> >  
> >  void hugetlb_report_meminfo(struct seq_file *m)
> >  {
> > -	struct hstate *h = &default_hstate;
> > +	struct hstate *h;
> > +	unsigned long total = 0;
> > +
> >  	if (!hugepages_supported())
> >  		return;
> > -	seq_printf(m,
> > -			"HugePages_Total:   %5lu\n"
> > -			"HugePages_Free:    %5lu\n"
> > -			"HugePages_Rsvd:    %5lu\n"
> > -			"HugePages_Surp:    %5lu\n"
> > -			"Hugepagesize:   %8lu kB\n",
> > -			h->nr_huge_pages,
> > -			h->free_huge_pages,
> > -			h->resv_huge_pages,
> > -			h->surplus_huge_pages,
> > -			1UL << (huge_page_order(h) + PAGE_SHIFT - 10));
> > +
> > +	for_each_hstate(h) {
> > +		unsigned long count = h->nr_huge_pages;
> > +
> > +		total += (PAGE_SIZE << huge_page_order(h)) * count;
> > +
> > +		if (h == &default_hstate)
> 
> I'm not understanding this test.  Are we assuming that default_hstate
> always refers to the highest-index hstate?  If so why, and is that
> valid?

As Mike and Michal pointed, default_hstate is defined as
  #define default_hstate (hstates[default_hstate_idx]),
where default_hstate_idx can be altered by a boot argument.

We're iterating over all states to calculate total and also
print some additional info for the default size. Having a single
loop guarantees consistency of these numbers.

Thanks!

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: show total hugetlb memory consumption in /proc/meminfo

[Andrew Morton]

On Tue, 21 Nov 2017 15:15:55 +0000 Roman Gushchin <guro@fb.com> wrote:

> > > +
> > > +	for_each_hstate(h) {
> > > +		unsigned long count = h->nr_huge_pages;
> > > +
> > > +		total += (PAGE_SIZE << huge_page_order(h)) * count;
> > > +
> > > +		if (h == &default_hstate)
> > 
> > I'm not understanding this test.  Are we assuming that default_hstate
> > always refers to the highest-index hstate?  If so why, and is that
> > valid?
> 
> As Mike and Michal pointed, default_hstate is defined as
>   #define default_hstate (hstates[default_hstate_idx]),
> where default_hstate_idx can be altered by a boot argument.
> 
> We're iterating over all states to calculate total and also
> print some additional info for the default size. Having a single
> loop guarantees consistency of these numbers.
> 

OK, I misread the handling of `count' -> HugePages_Total.

It seems unnecessarily obscure?

	for_each_hstate(h) {
		unsigned long count = h->nr_huge_pages;

		total += (PAGE_SIZE << huge_page_order(h)) * count;

		if (h == &default_hstate)
			seq_printf(m,
				   "HugePages_Total:   %5lu\n"
				   "HugePages_Free:    %5lu\n"
				   "HugePages_Rsvd:    %5lu\n"
				   "HugePages_Surp:    %5lu\n"
				   "Hugepagesize:   %8lu kB\n",
				   count,
				   h->free_huge_pages,
				   h->resv_huge_pages,
				   h->surplus_huge_pages,
				   (PAGE_SIZE << huge_page_order(h)) / 1024);
	}

	seq_printf(m, "Hugetlb:        %8lu kB\n", total / 1024);


Why not

	seq_printf(m,
			"HugePages_Total:   %5lu\n"
			"HugePages_Free:    %5lu\n"
			"HugePages_Rsvd:    %5lu\n"
			"HugePages_Surp:    %5lu\n"
			"Hugepagesize:   %8lu kB\n",
			h->nr_huge_pages,
			h->free_huge_pages,
			h->resv_huge_pages,
			h->surplus_huge_pages,
			1UL << (huge_page_order(h) + PAGE_SHIFT - 10));

	for_each_hstate(h)
		total += (PAGE_SIZE << huge_page_order(h)) * h->nr_huge_pages;
	seq_printf(m, "Hugetlb:        %8lu kB\n", total / 1024);
	
?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: show total hugetlb memory consumption in /proc/meminfo

[Roman Gushchin]

On Tue, Nov 21, 2017 at 11:19:07AM -0800, Andrew Morton wrote:
> On Tue, 21 Nov 2017 15:15:55 +0000 Roman Gushchin <guro@fb.com> wrote:
> 
> > > > +
> > > > +	for_each_hstate(h) {
> > > > +		unsigned long count = h->nr_huge_pages;
> > > > +
> > > > +		total += (PAGE_SIZE << huge_page_order(h)) * count;
> > > > +
> > > > +		if (h == &default_hstate)
> > > 
> > > I'm not understanding this test.  Are we assuming that default_hstate
> > > always refers to the highest-index hstate?  If so why, and is that
> > > valid?
> > 
> > As Mike and Michal pointed, default_hstate is defined as
> >   #define default_hstate (hstates[default_hstate_idx]),
> > where default_hstate_idx can be altered by a boot argument.
> > 
> > We're iterating over all states to calculate total and also
> > print some additional info for the default size. Having a single
> > loop guarantees consistency of these numbers.
> > 
> 
> OK, I misread the handling of `count' -> HugePages_Total.
> 
> It seems unnecessarily obscure?
> 
> 	for_each_hstate(h) {
> 		unsigned long count = h->nr_huge_pages;
> 
> 		total += (PAGE_SIZE << huge_page_order(h)) * count;
> 
> 		if (h == &default_hstate)
> 			seq_printf(m,
> 				   "HugePages_Total:   %5lu\n"
> 				   "HugePages_Free:    %5lu\n"
> 				   "HugePages_Rsvd:    %5lu\n"
> 				   "HugePages_Surp:    %5lu\n"
> 				   "Hugepagesize:   %8lu kB\n",
> 				   count,
> 				   h->free_huge_pages,
> 				   h->resv_huge_pages,
> 				   h->surplus_huge_pages,
> 				   (PAGE_SIZE << huge_page_order(h)) / 1024);
> 	}
> 
> 	seq_printf(m, "Hugetlb:        %8lu kB\n", total / 1024);
> 
> 
> Why not
> 
> 	seq_printf(m,
> 			"HugePages_Total:   %5lu\n"
> 			"HugePages_Free:    %5lu\n"
> 			"HugePages_Rsvd:    %5lu\n"
> 			"HugePages_Surp:    %5lu\n"
> 			"Hugepagesize:   %8lu kB\n",
> 			h->nr_huge_pages,
> 			h->free_huge_pages,
> 			h->resv_huge_pages,
> 			h->surplus_huge_pages,
> 			1UL << (huge_page_order(h) + PAGE_SHIFT - 10));
> 
> 	for_each_hstate(h)
> 		total += (PAGE_SIZE << huge_page_order(h)) * h->nr_huge_pages;
> 	seq_printf(m, "Hugetlb:        %8lu kB\n", total / 1024);
> 	
> ?

The idea was that the local variable guarantees the consistency
between Hugetlb and HugePages_Total numbers. Otherwise we have
to take hugetlb_lock.

What we can do, is to rename "count" into "nr_huge_pages", like:

	for_each_hstate(h) {
		unsigned long nr_huge_pages = h->nr_huge_pages;

		total += (PAGE_SIZE << huge_page_order(h)) * nr_huge_pages;

		if (h == &default_hstate)
			seq_printf(m,
				   "HugePages_Total:   %5lu\n"
				   "HugePages_Free:    %5lu\n"
				   "HugePages_Rsvd:    %5lu\n"
				   "HugePages_Surp:    %5lu\n"
				   "Hugepagesize:   %8lu kB\n",
				   nr_huge_pages,
				   h->free_huge_pages,
				   h->resv_huge_pages,
				   h->surplus_huge_pages,
				   (PAGE_SIZE << huge_page_order(h)) / 1024);
	}

	seq_printf(m, "Hugetlb:        %8lu kB\n", total / 1024);

But maybe taking a lock is not a bad idea, because it will also
guarantee consistency between other numbers (like HugePages_Free) as well,
which is not true right now.

Thanks!

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: show total hugetlb memory consumption in /proc/meminfo

[Mike Kravetz]

On 11/21/2017 11:59 AM, Roman Gushchin wrote:
> On Tue, Nov 21, 2017 at 11:19:07AM -0800, Andrew Morton wrote:
>>
>> Why not
>>
>> 	seq_printf(m,
>> 			"HugePages_Total:   %5lu\n"
>> 			"HugePages_Free:    %5lu\n"
>> 			"HugePages_Rsvd:    %5lu\n"
>> 			"HugePages_Surp:    %5lu\n"
>> 			"Hugepagesize:   %8lu kB\n",
>> 			h->nr_huge_pages,
>> 			h->free_huge_pages,
>> 			h->resv_huge_pages,
>> 			h->surplus_huge_pages,
>> 			1UL << (huge_page_order(h) + PAGE_SHIFT - 10));
>>
>> 	for_each_hstate(h)
>> 		total += (PAGE_SIZE << huge_page_order(h)) * h->nr_huge_pages;
>> 	seq_printf(m, "Hugetlb:        %8lu kB\n", total / 1024);
>> 	
>> ?
> 
> The idea was that the local variable guarantees the consistency
> between Hugetlb and HugePages_Total numbers. Otherwise we have
> to take hugetlb_lock.

Most important it prevents HugePages_Total from being larger than 
Hugetlb.

> What we can do, is to rename "count" into "nr_huge_pages", like:
> 
> 	for_each_hstate(h) {
> 		unsigned long nr_huge_pages = h->nr_huge_pages;
> 
> 		total += (PAGE_SIZE << huge_page_order(h)) * nr_huge_pages;
> 
> 		if (h == &default_hstate)
> 			seq_printf(m,
> 				   "HugePages_Total:   %5lu\n"
> 				   "HugePages_Free:    %5lu\n"
> 				   "HugePages_Rsvd:    %5lu\n"
> 				   "HugePages_Surp:    %5lu\n"
> 				   "Hugepagesize:   %8lu kB\n",
> 				   nr_huge_pages,
> 				   h->free_huge_pages,
> 				   h->resv_huge_pages,
> 				   h->surplus_huge_pages,
> 				   (PAGE_SIZE << huge_page_order(h)) / 1024);
> 	}
> 
> 	seq_printf(m, "Hugetlb:        %8lu kB\n", total / 1024);
> 
> But maybe taking a lock is not a bad idea, because it will also
> guarantee consistency between other numbers (like HugePages_Free) as well,
> which is not true right now.

You are correct in that there is no consistency guarantee for the numbers
with the default huge page size today.  However, I am not really a fan of
taking the lock for that guarantee.  IMO, the above code is fine.

This discussion reminds me that ideally there should be a per-hstate lock.
My guess is that the global lock is a carry over from the days when only
a single huge page size was supported.  In practice, I don't think this is
much of an issue as people typically only use a single huge page size.  But,
if anyone thinks is/may be an issue I am happy to make the changes.

-- 
Mike Kravetz

> 
> Thanks!
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] mm: show total hugetlb memory consumption in /proc/meminfo

[Michal Hocko]

On Tue 21-11-17 16:27:38, Mike Kravetz wrote:
> On 11/21/2017 11:59 AM, Roman Gushchin wrote:
[...]
> > What we can do, is to rename "count" into "nr_huge_pages", like:
> > 
> > 	for_each_hstate(h) {
> > 		unsigned long nr_huge_pages = h->nr_huge_pages;
> > 
> > 		total += (PAGE_SIZE << huge_page_order(h)) * nr_huge_pages;
> > 
> > 		if (h == &default_hstate)
> > 			seq_printf(m,
> > 				   "HugePages_Total:   %5lu\n"
> > 				   "HugePages_Free:    %5lu\n"
> > 				   "HugePages_Rsvd:    %5lu\n"
> > 				   "HugePages_Surp:    %5lu\n"
> > 				   "Hugepagesize:   %8lu kB\n",
> > 				   nr_huge_pages,
> > 				   h->free_huge_pages,
> > 				   h->resv_huge_pages,
> > 				   h->surplus_huge_pages,
> > 				   (PAGE_SIZE << huge_page_order(h)) / 1024);
> > 	}
> > 
> > 	seq_printf(m, "Hugetlb:        %8lu kB\n", total / 1024);
> > 
> > But maybe taking a lock is not a bad idea, because it will also
> > guarantee consistency between other numbers (like HugePages_Free) as well,
> > which is not true right now.
> 
> You are correct in that there is no consistency guarantee for the numbers
> with the default huge page size today.  However, I am not really a fan of
> taking the lock for that guarantee.  IMO, the above code is fine.

I agree

> This discussion reminds me that ideally there should be a per-hstate lock.
> My guess is that the global lock is a carry over from the days when only
> a single huge page size was supported.  In practice, I don't think this is
> much of an issue as people typically only use a single huge page size.  But,
> if anyone thinks is/may be an issue I am happy to make the changes.

Well, it kind of makes sense but I am not sure it is worth bothering.

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>