* Re: [PATCH] selftests/mm: run_vmtests.sh: Fix hugetlb mem size calculation
2024-03-21 21:50 [PATCH] selftests/mm: run_vmtests.sh: Fix hugetlb mem size calculation peterx
@ 2024-03-22 3:50 ` Muchun Song
2024-03-22 17:54 ` David Hildenbrand
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Muchun Song @ 2024-03-22 3:50 UTC (permalink / raw)
To: Peter Xu
Cc: Linux-MM, LKML, Andrew Morton, Muhammad Usama Anjum,
David Hildenbrand, Nico Pache
> On Mar 22, 2024, at 05:50, peterx@redhat.com wrote:
>
> From: Peter Xu <peterx@redhat.com>
>
> The script calculates a mininum required size of hugetlb memories, but
> it'll stop working with <1MB huge page sizes, reporting all zeros even if
> huge pages are available.
>
> In reality, the calculation doesn't really need to be as comlicated either.
^
complicated?
> Make it simpler and work for KB-level hugepages too.
>
> Cc: Muhammad Usama Anjum <usama.anjum@collabora.com>
> Cc: David Hildenbrand <david@redhat.com>
> Cc: Nico Pache <npache@redhat.com>
> Cc: Muchun Song <muchun.song@linux.dev>
> Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Muchun Song <muchun.song@linux.dev>
Thanks.
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH] selftests/mm: run_vmtests.sh: Fix hugetlb mem size calculation
2024-03-21 21:50 [PATCH] selftests/mm: run_vmtests.sh: Fix hugetlb mem size calculation peterx
2024-03-22 3:50 ` Muchun Song
@ 2024-03-22 17:54 ` David Hildenbrand
2024-03-24 17:56 ` Muhammad Usama Anjum
2024-04-03 11:04 ` Ryan Roberts
3 siblings, 0 replies; 6+ messages in thread
From: David Hildenbrand @ 2024-03-22 17:54 UTC (permalink / raw)
To: peterx, linux-mm, linux-kernel
Cc: Andrew Morton, Muhammad Usama Anjum, Nico Pache, Muchun Song
On 21.03.24 22:50, peterx@redhat.com wrote:
> From: Peter Xu <peterx@redhat.com>
>
> The script calculates a mininum required size of hugetlb memories, but
> it'll stop working with <1MB huge page sizes, reporting all zeros even if
> huge pages are available.
>
> In reality, the calculation doesn't really need to be as comlicated either.
> Make it simpler and work for KB-level hugepages too.
>
> Cc: Muhammad Usama Anjum <usama.anjum@collabora.com>
> Cc: David Hildenbrand <david@redhat.com>
> Cc: Nico Pache <npache@redhat.com>
> Cc: Muchun Song <muchun.song@linux.dev>
> Signed-off-by: Peter Xu <peterx@redhat.com>
> ---
> tools/testing/selftests/mm/run_vmtests.sh | 10 +++++++---
> 1 file changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> index c2c542fe7b17..b1b78e45d613 100755
> --- a/tools/testing/selftests/mm/run_vmtests.sh
> +++ b/tools/testing/selftests/mm/run_vmtests.sh
> @@ -152,9 +152,13 @@ done < /proc/meminfo
> # both of these requirements into account and attempt to increase
> # number of huge pages available.
> nr_cpus=$(nproc)
> -hpgsize_MB=$((hpgsize_KB / 1024))
> -half_ufd_size_MB=$((((nr_cpus * hpgsize_MB + 127) / 128) * 128))
> -needmem_KB=$((half_ufd_size_MB * 2 * 1024))
> +uffd_min_KB=$((hpgsize_KB * nr_cpus * 2))
> +hugetlb_min_KB=$((256 * 1024))
> +if [[ $uffd_min_KB -gt $hugetlb_min_KB ]]; then
> + needmem_KB=$uffd_min_KB
> +else
> + needmem_KB=$hugetlb_min_KB
> +fi
>
> # set proper nr_hugepages
> if [ -n "$freepgs" ] && [ -n "$hpgsize_KB" ]; then
Reviewed-by: David Hildenbrand <david@redhat.com>
--
Cheers,
David / dhildenb
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] selftests/mm: run_vmtests.sh: Fix hugetlb mem size calculation
2024-03-21 21:50 [PATCH] selftests/mm: run_vmtests.sh: Fix hugetlb mem size calculation peterx
2024-03-22 3:50 ` Muchun Song
2024-03-22 17:54 ` David Hildenbrand
@ 2024-03-24 17:56 ` Muhammad Usama Anjum
2024-04-03 11:04 ` Ryan Roberts
3 siblings, 0 replies; 6+ messages in thread
From: Muhammad Usama Anjum @ 2024-03-24 17:56 UTC (permalink / raw)
To: peterx, linux-mm, linux-kernel
Cc: Muhammad Usama Anjum, Andrew Morton, David Hildenbrand,
Nico Pache, Muchun Song
On 3/22/24 2:50 AM, peterx@redhat.com wrote:
> From: Peter Xu <peterx@redhat.com>
>
> The script calculates a mininum required size of hugetlb memories, but
> it'll stop working with <1MB huge page sizes, reporting all zeros even if
> huge pages are available.
>
> In reality, the calculation doesn't really need to be as comlicated either.
> Make it simpler and work for KB-level hugepages too.
>
> Cc: Muhammad Usama Anjum <usama.anjum@collabora.com>
> Cc: David Hildenbrand <david@redhat.com>
> Cc: Nico Pache <npache@redhat.com>
> Cc: Muchun Song <muchun.song@linux.dev>
> Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
> ---
> tools/testing/selftests/mm/run_vmtests.sh | 10 +++++++---
> 1 file changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> index c2c542fe7b17..b1b78e45d613 100755
> --- a/tools/testing/selftests/mm/run_vmtests.sh
> +++ b/tools/testing/selftests/mm/run_vmtests.sh
> @@ -152,9 +152,13 @@ done < /proc/meminfo
> # both of these requirements into account and attempt to increase
> # number of huge pages available.
> nr_cpus=$(nproc)
> -hpgsize_MB=$((hpgsize_KB / 1024))
> -half_ufd_size_MB=$((((nr_cpus * hpgsize_MB + 127) / 128) * 128))
> -needmem_KB=$((half_ufd_size_MB * 2 * 1024))
> +uffd_min_KB=$((hpgsize_KB * nr_cpus * 2))
> +hugetlb_min_KB=$((256 * 1024))
> +if [[ $uffd_min_KB -gt $hugetlb_min_KB ]]; then
> + needmem_KB=$uffd_min_KB
> +else
> + needmem_KB=$hugetlb_min_KB
> +fi
>
> # set proper nr_hugepages
> if [ -n "$freepgs" ] && [ -n "$hpgsize_KB" ]; then
--
BR,
Muhammad Usama Anjum
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] selftests/mm: run_vmtests.sh: Fix hugetlb mem size calculation
2024-03-21 21:50 [PATCH] selftests/mm: run_vmtests.sh: Fix hugetlb mem size calculation peterx
` (2 preceding siblings ...)
2024-03-24 17:56 ` Muhammad Usama Anjum
@ 2024-04-03 11:04 ` Ryan Roberts
2024-04-03 18:19 ` Peter Xu
3 siblings, 1 reply; 6+ messages in thread
From: Ryan Roberts @ 2024-04-03 11:04 UTC (permalink / raw)
To: peterx, linux-mm, linux-kernel
Cc: Andrew Morton, Muhammad Usama Anjum, David Hildenbrand,
Nico Pache, Muchun Song
Hi Peter,
On 21/03/2024 21:50, peterx@redhat.com wrote:
> From: Peter Xu <peterx@redhat.com>
>
> The script calculates a mininum required size of hugetlb memories, but
> it'll stop working with <1MB huge page sizes, reporting all zeros even if
> huge pages are available.
>
> In reality, the calculation doesn't really need to be as comlicated either.
> Make it simpler and work for KB-level hugepages too.
>
> Cc: Muhammad Usama Anjum <usama.anjum@collabora.com>
> Cc: David Hildenbrand <david@redhat.com>
> Cc: Nico Pache <npache@redhat.com>
> Cc: Muchun Song <muchun.song@linux.dev>
> Signed-off-by: Peter Xu <peterx@redhat.com>
> ---
> tools/testing/selftests/mm/run_vmtests.sh | 10 +++++++---
> 1 file changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> index c2c542fe7b17..b1b78e45d613 100755
> --- a/tools/testing/selftests/mm/run_vmtests.sh
> +++ b/tools/testing/selftests/mm/run_vmtests.sh
> @@ -152,9 +152,13 @@ done < /proc/meminfo
> # both of these requirements into account and attempt to increase
> # number of huge pages available.
> nr_cpus=$(nproc)
> -hpgsize_MB=$((hpgsize_KB / 1024))
> -half_ufd_size_MB=$((((nr_cpus * hpgsize_MB + 127) / 128) * 128))
Removing this has broken the uffd-stress "hugetlb" and "hugetlb-private" tests
(further down the file), which rely on $half_ufd_size_MB. Now that this is not
defined, they are called with too few params:
# # ---------------------------------
# # running ./uffd-stress hugetlb 32
# # ---------------------------------
# # ERROR: invalid MiB (errno=0, @uffd-stress.c:454)
# #
# # Usage: ./uffd-stress <test type> <MiB> <bounces>
# #
# # Supported <test type>: anon, hugetlb, hugetlb-private, shmem, shmem-private
# #
# # Examples:
# #
# # # Run anonymous memory test on 100MiB region with 99999 bounces:
# # ./uffd-stress anon 100 99999
# #
# # # Run share memory test on 1GiB region with 99 bounces:
# # ./uffd-stress shmem 1000 99
# #
# # # Run hugetlb memory test on 256MiB region with 50 bounces:
# # ./uffd-stress hugetlb 256 50
# #
# # # Run the same hugetlb test but using private file:
# # ./uffd-stress hugetlb-private 256 50
# #
# # # 10MiB-~6GiB 999 bounces anonymous test, continue forever unless an error
triggers
# # while ./uffd-stress anon $[RANDOM % 6000 + 10] 999; do true; done
# #
# # [FAIL]
# not ok 16 uffd-stress hugetlb 32 # exit=1
# # -----------------------------------------
# # running ./uffd-stress hugetlb-private 32
# # -----------------------------------------
# # ERROR: invalid MiB (errno=0, @uffd-stress.c:454)
# #
# # Usage: ./uffd-stress <test type> <MiB> <bounces>
# #
# # Supported <test type>: anon, hugetlb, hugetlb-private, shmem, shmem-private
# #
# # Examples:
# #
# # # Run anonymous memory test on 100MiB region with 99999 bounces:
# # ./uffd-stress anon 100 99999
# #
# # # Run share memory test on 1GiB region with 99 bounces:
# # ./uffd-stress shmem 1000 99
# #
# # # Run hugetlb memory test on 256MiB region with 50 bounces:
# # ./uffd-stress hugetlb 256 50
# #
# # # Run the same hugetlb test but using private file:
# # ./uffd-stress hugetlb-private 256 50
# #
# # # 10MiB-~6GiB 999 bounces anonymous test, continue forever unless an error
triggers
# # while ./uffd-stress anon $[RANDOM % 6000 + 10] 999; do true; done
# #
# # [FAIL]
# not ok 17 uffd-stress hugetlb-private 32 # exit=1
Thanks,
Ryan
> -needmem_KB=$((half_ufd_size_MB * 2 * 1024))
> +uffd_min_KB=$((hpgsize_KB * nr_cpus * 2))
> +hugetlb_min_KB=$((256 * 1024))
> +if [[ $uffd_min_KB -gt $hugetlb_min_KB ]]; then
> + needmem_KB=$uffd_min_KB
> +else
> + needmem_KB=$hugetlb_min_KB
> +fi
>
> # set proper nr_hugepages
> if [ -n "$freepgs" ] && [ -n "$hpgsize_KB" ]; then
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH] selftests/mm: run_vmtests.sh: Fix hugetlb mem size calculation
2024-04-03 11:04 ` Ryan Roberts
@ 2024-04-03 18:19 ` Peter Xu
0 siblings, 0 replies; 6+ messages in thread
From: Peter Xu @ 2024-04-03 18:19 UTC (permalink / raw)
To: Ryan Roberts
Cc: linux-mm, linux-kernel, Andrew Morton, Muhammad Usama Anjum,
David Hildenbrand, Nico Pache, Muchun Song
On Wed, Apr 03, 2024 at 12:04:00PM +0100, Ryan Roberts wrote:
> > diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> > index c2c542fe7b17..b1b78e45d613 100755
> > --- a/tools/testing/selftests/mm/run_vmtests.sh
> > +++ b/tools/testing/selftests/mm/run_vmtests.sh
> > @@ -152,9 +152,13 @@ done < /proc/meminfo
> > # both of these requirements into account and attempt to increase
> > # number of huge pages available.
> > nr_cpus=$(nproc)
> > -hpgsize_MB=$((hpgsize_KB / 1024))
> > -half_ufd_size_MB=$((((nr_cpus * hpgsize_MB + 127) / 128) * 128))
>
> Removing this has broken the uffd-stress "hugetlb" and "hugetlb-private" tests
> (further down the file), which rely on $half_ufd_size_MB. Now that this is not
> defined, they are called with too few params:
Those FAILs can be burried in some other libc mismatch issues for me so I
overlooked.. My apologies.
I'll send a fixup soon, thank you Ryan!
--
Peter Xu
^ permalink raw reply [flat|nested] 6+ messages in thread