Re: [PATCH v2 2/2] selftests/mm: Add a new test for madv and hugetlb

[Ryan Roberts <ryan.roberts@arm.com>]

Hi Breno,


On 05/10/2023 17:39, Breno Leitao wrote:
> Create a selftest that exercises the race between page faults and
> madvise(MADV_DONTNEED) in the same huge page. Do it by running two
> threads that touches the huge page and madvise(MADV_DONTNEED) at the same
> time.
> 
> In case of a SIGBUS coming at pagefault, the test should fail, since we
> hit the bug.
> 
> The test doesn't have a signal handler, and if it fails, it fails like
> the following
> 
>   ----------------------------------
>   running ./hugetlb_fault_after_madv
>   ----------------------------------
>   ./run_vmtests.sh: line 186: 595563 Bus error    (core dumped) "$@"
>   [FAIL]
> 
> This selftest goes together with the fix of the bug[1] itself.
> 
> [1] https://lore.kernel.org/all/20231001005659.2185316-1-riel@surriel.com/#r
> 
> Signed-off-by: Breno Leitao <leitao@debian.org>
> ---
>  tools/testing/selftests/mm/Makefile           |  1 +
>  .../selftests/mm/hugetlb_fault_after_madv.c   | 73 +++++++++++++++++++
>  tools/testing/selftests/mm/run_vmtests.sh     |  4 +
>  3 files changed, 78 insertions(+)
>  create mode 100644 tools/testing/selftests/mm/hugetlb_fault_after_madv.c
> 
> diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
> index 6a9fc5693145..e71ec9910c62 100644
> --- a/tools/testing/selftests/mm/Makefile
> +++ b/tools/testing/selftests/mm/Makefile
> @@ -68,6 +68,7 @@ TEST_GEN_FILES += split_huge_page_test
>  TEST_GEN_FILES += ksm_tests
>  TEST_GEN_FILES += ksm_functional_tests
>  TEST_GEN_FILES += mdwe_test
> +TEST_GEN_FILES += hugetlb_fault_after_madv
>  
>  ifneq ($(ARCH),arm64)
>  TEST_GEN_PROGS += soft-dirty
> diff --git a/tools/testing/selftests/mm/hugetlb_fault_after_madv.c b/tools/testing/selftests/mm/hugetlb_fault_after_madv.c
> new file mode 100644
> index 000000000000..73b81c632366
> --- /dev/null
> +++ b/tools/testing/selftests/mm/hugetlb_fault_after_madv.c
> @@ -0,0 +1,73 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <pthread.h>
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <sys/mman.h>
> +#include <sys/types.h>
> +#include <unistd.h>
> +
> +#include "vm_util.h"
> +#include "../kselftest.h"
> +
> +#define MMAP_SIZE (1 << 21)
> +#define INLOOP_ITER 100
> +
> +char *huge_ptr;
> +
> +/* Touch the memory while it is being madvised() */
> +void *touch(void *unused)
> +{
> +	char *ptr = (char *)huge_ptr;
> +
> +	for (int i = 0; i < INLOOP_ITER; i++)
> +		ptr[0] = '.';
> +
> +	return NULL;
> +}
> +
> +void *madv(void *unused)
> +{
> +	usleep(rand() % 10);
> +
> +	for (int i = 0; i < INLOOP_ITER; i++)
> +		madvise(huge_ptr, MMAP_SIZE, MADV_DONTNEED);
> +
> +	return NULL;
> +}
> +
> +int main(void)
> +{
> +	unsigned long free_hugepages;
> +	pthread_t thread1, thread2;
> +	/*
> +	 * On kernel 6.4, we are able to reproduce the problem with ~1000
> +	 * interactions
> +	 */
> +	int max = 10000;
> +
> +	srand(getpid());
> +
> +	free_hugepages = get_free_hugepages();
> +	if (free_hugepages != 1) {
> +		ksft_exit_skip("This test needs one and only one page to execute. Got %lu\n",
> +			       free_hugepages);
> +	}
> +
> +	while (max--) {
> +		huge_ptr = mmap(NULL, MMAP_SIZE, PROT_READ | PROT_WRITE,
> +				MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB,
> +				-1, 0);
> +
> +		if ((unsigned long)huge_ptr == -1)
> +			ksft_exit_skip("Failed to allocated huge page\n");
> +
> +		pthread_create(&thread1, NULL, madv, NULL);
> +		pthread_create(&thread2, NULL, touch, NULL);
> +
> +		pthread_join(thread1, NULL);
> +		pthread_join(thread2, NULL);
> +		munmap(huge_ptr, MMAP_SIZE);
> +	}
> +
> +	return KSFT_PASS;
> +}
> diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
> index 3e2bc818d566..9f53f7318a38 100755
> --- a/tools/testing/selftests/mm/run_vmtests.sh
> +++ b/tools/testing/selftests/mm/run_vmtests.sh
> @@ -221,6 +221,10 @@ CATEGORY="hugetlb" run_test ./hugepage-mremap
>  CATEGORY="hugetlb" run_test ./hugepage-vmemmap
>  CATEGORY="hugetlb" run_test ./hugetlb-madvise
>  
> +# For this test, we need one and just one huge page
> +echo 1 > /proc/sys/vm/nr_hugepages

I've noticed that this change breaks some of the uffd-stress tests further down
the file, because you have freed previously reserved hugepages that the test
requires to run. I notice that the patch is already in mm-stable, so perhaps its
possible to submit a patch that does a save and restore?

Although I'm not sure if that might be tricky because the previous reservation
is per-size and per-node (our CI does this on the kernel command line), and I
suspect if you want just 1 huge page in the entire system you won't be able to
get back to the previous state by just restoring this value?

These are the failing tests for reference:

# ------------------------------------
# running ./uffd-stress hugetlb 128 32
# ------------------------------------
# nr_pages: 64, nr_pages_per_cpu: 8
# ERROR: context init failed (errno=12, @uffd-stress.c:254)
# [FAIL]
# --------------------------------------------
# running ./uffd-stress hugetlb-private 128 32
# --------------------------------------------
# nr_pages: 64, nr_pages_per_cpu: 8
# ERROR: context init failed (errno=12, @uffd-stress.c:254)
# [FAIL]

Thanks,
Ryan



> +CATEGORY="hugetlb" run_test ./hugetlb_fault_after_madv
> +
>  if test_selected "hugetlb"; then
>  	echo "NOTE: These hugetlb tests provide minimal coverage.  Use"
>  	echo "      https://github.com/libhugetlbfs/libhugetlbfs.git for"