[PATCH v1 0/2] selftests/mm: thuge-gen defines

[PATCH v1 0/2] selftests/mm: thuge-gen defines

[Edward Liaw]

When compiling with Android bionic, the MAP_HUGE_* and SHM_HUGE_* macros
are redefined because they are included from the uapi by sys/mman.h and
sys/shm.h:

INFO: From Compiling common/tools/testing/selftests/mm/thuge-gen.c:
common/tools/testing/selftests/mm/thuge-gen.c:32:9: warning: 'MAP_HUGE_2MB' macro redefined [-Wmacro-redefined]
   32 | #define MAP_HUGE_2MB    (21 << MAP_HUGE_SHIFT)
      |         ^
external/_main~_repo_rules~prebuilt_ndk/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/linux/mman.h:38:9: note: previous definition is here
   38 | #define MAP_HUGE_2MB HUGETLB_FLAG_ENCODE_2MB
      |         ^
common/tools/testing/selftests/mm/thuge-gen.c:33:9: warning: 'MAP_HUGE_1GB' macro redefined [-Wmacro-redefined]
   33 | #define MAP_HUGE_1GB    (30 << MAP_HUGE_SHIFT)
      |         ^
external/_main~_repo_rules~prebuilt_ndk/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/linux/mman.h:44:9: note: previous definition is here

This test should probably use the uapi definitions instead of redefining
them.  However, glibc gets struct redefinitions when including sys/shm.h
and linux/shm.h together.  So, add guards for the SHM_HUGE_* macros
instead.

Edward Liaw (2):
  selftests/mm: Include linux/mman.h
  selftests/mm: Guard defines from shm

 tools/testing/selftests/mm/thuge-gen.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

--
2.45.1.467.gbab1589fc0-goog

[PATCH v1 1/2] selftests/mm: Include linux/mman.h

[Edward Liaw]

thuge-gen defines MAP_HUGE_* macros that are provided by linux/mman.h
since 4.15. Removes the macros and includes linux/mman.h instead.

Signed-off-by: Edward Liaw <edliaw@google.com>
---
 tools/testing/selftests/mm/thuge-gen.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/tools/testing/selftests/mm/thuge-gen.c b/tools/testing/selftests/mm/thuge-gen.c
index ea7fd8fe2876..034635317935 100644
--- a/tools/testing/selftests/mm/thuge-gen.c
+++ b/tools/testing/selftests/mm/thuge-gen.c
@@ -15,6 +15,7 @@
 
 #define _GNU_SOURCE 1
 #include <sys/mman.h>
+#include <linux/mman.h>
 #include <stdlib.h>
 #include <stdio.h>
 #include <sys/ipc.h>
@@ -28,10 +29,6 @@
 #include "vm_util.h"
 #include "../kselftest.h"
 
-#define MAP_HUGE_2MB    (21 << MAP_HUGE_SHIFT)
-#define MAP_HUGE_1GB    (30 << MAP_HUGE_SHIFT)
-#define MAP_HUGE_SHIFT  26
-#define MAP_HUGE_MASK   0x3f
 #if !defined(MAP_HUGETLB)
 #define MAP_HUGETLB	0x40000
 #endif
-- 
2.45.1.467.gbab1589fc0-goog

Re: [PATCH v1 1/2] selftests/mm: Include linux/mman.h

[Carlos Llamas]

On Wed, Jun 05, 2024 at 10:36:34PM +0000, Edward Liaw wrote:
> thuge-gen defines MAP_HUGE_* macros that are provided by linux/mman.h
> since 4.15. Removes the macros and includes linux/mman.h instead.
> 
> Signed-off-by: Edward Liaw <edliaw@google.com>
> ---
>  tools/testing/selftests/mm/thuge-gen.c | 5 +----
>  1 file changed, 1 insertion(+), 4 deletions(-)
> 
> diff --git a/tools/testing/selftests/mm/thuge-gen.c b/tools/testing/selftests/mm/thuge-gen.c
> index ea7fd8fe2876..034635317935 100644
> --- a/tools/testing/selftests/mm/thuge-gen.c
> +++ b/tools/testing/selftests/mm/thuge-gen.c
> @@ -15,6 +15,7 @@
>  
>  #define _GNU_SOURCE 1
>  #include <sys/mman.h>
> +#include <linux/mman.h>
>  #include <stdlib.h>
>  #include <stdio.h>
>  #include <sys/ipc.h>
> @@ -28,10 +29,6 @@
>  #include "vm_util.h"
>  #include "../kselftest.h"
>  
> -#define MAP_HUGE_2MB    (21 << MAP_HUGE_SHIFT)
> -#define MAP_HUGE_1GB    (30 << MAP_HUGE_SHIFT)
> -#define MAP_HUGE_SHIFT  26
> -#define MAP_HUGE_MASK   0x3f
>  #if !defined(MAP_HUGETLB)
>  #define MAP_HUGETLB	0x40000
>  #endif
> -- 
> 2.45.1.467.gbab1589fc0-goog
> 

This makes sense to me, particularly when other mm selftests are using
the same include pattern.

Reviewed-by: Carlos Llamas <cmllamas@google.com>

Re: [PATCH v1 1/2] selftests/mm: Include linux/mman.h

[Muhammad Usama Anjum]

On 6/6/24 3:36 AM, Edward Liaw wrote:
> thuge-gen defines MAP_HUGE_* macros that are provided by linux/mman.h
> since 4.15. Removes the macros and includes linux/mman.h instead.
> 
> Signed-off-by: Edward Liaw <edliaw@google.com>
> ---
>  tools/testing/selftests/mm/thuge-gen.c | 5 +----
>  1 file changed, 1 insertion(+), 4 deletions(-)
> 
> diff --git a/tools/testing/selftests/mm/thuge-gen.c b/tools/testing/selftests/mm/thuge-gen.c
> index ea7fd8fe2876..034635317935 100644
> --- a/tools/testing/selftests/mm/thuge-gen.c
> +++ b/tools/testing/selftests/mm/thuge-gen.c
> @@ -15,6 +15,7 @@
>  
>  #define _GNU_SOURCE 1
>  #include <sys/mman.h>
> +#include <linux/mman.h>
>  #include <stdlib.h>
>  #include <stdio.h>
>  #include <sys/ipc.h>
> @@ -28,10 +29,6 @@
>  #include "vm_util.h"
>  #include "../kselftest.h"
>  
> -#define MAP_HUGE_2MB    (21 << MAP_HUGE_SHIFT)
> -#define MAP_HUGE_1GB    (30 << MAP_HUGE_SHIFT)
> -#define MAP_HUGE_SHIFT  26
> -#define MAP_HUGE_MASK   0x3f
Totally makes sense.
Reviewed-by: Muhammad Usama Anjum <usama.anju@collabora.com>

>  #if !defined(MAP_HUGETLB)
>  #define MAP_HUGETLB	0x40000
>  #endif

-- 
BR,
Muhammad Usama Anjum

Re: [PATCH v1 1/2] selftests/mm: Include linux/mman.h

[Muhammad Usama Anjum]

On 6/7/24 2:44 PM, Muhammad Usama Anjum wrote:
> On 6/6/24 3:36 AM, Edward Liaw wrote:
>> thuge-gen defines MAP_HUGE_* macros that are provided by linux/mman.h
>> since 4.15. Removes the macros and includes linux/mman.h instead.
>>
>> Signed-off-by: Edward Liaw <edliaw@google.com>
>> ---
>>  tools/testing/selftests/mm/thuge-gen.c | 5 +----
>>  1 file changed, 1 insertion(+), 4 deletions(-)
>>
>> diff --git a/tools/testing/selftests/mm/thuge-gen.c b/tools/testing/selftests/mm/thuge-gen.c
>> index ea7fd8fe2876..034635317935 100644
>> --- a/tools/testing/selftests/mm/thuge-gen.c
>> +++ b/tools/testing/selftests/mm/thuge-gen.c
>> @@ -15,6 +15,7 @@
>>  
>>  #define _GNU_SOURCE 1
>>  #include <sys/mman.h>
>> +#include <linux/mman.h>
>>  #include <stdlib.h>
>>  #include <stdio.h>
>>  #include <sys/ipc.h>
>> @@ -28,10 +29,6 @@
>>  #include "vm_util.h"
>>  #include "../kselftest.h"
>>  
>> -#define MAP_HUGE_2MB    (21 << MAP_HUGE_SHIFT)
>> -#define MAP_HUGE_1GB    (30 << MAP_HUGE_SHIFT)
>> -#define MAP_HUGE_SHIFT  26
>> -#define MAP_HUGE_MASK   0x3f
> Totally makes sense.
> Reviewed-by: Muhammad Usama Anjum <usama.anju@collabora.com>
Fixing typo:
Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com>

> 
>>  #if !defined(MAP_HUGETLB)
>>  #define MAP_HUGETLB	0x40000
>>  #endif
> 

-- 
BR,
Muhammad Usama Anjum

[PATCH v1 2/2] selftests/mm: Guard defines from shm

[Edward Liaw]

thuge-gen.c defines SHM_HUGE_* macros that are provided by the uapi
since 4.14. These macros get redefined when compiling with Android's
bionic because its sys/shm.h will import the uapi definitions.

However if linux/shm.h is included, with glibc, sys/shm.h will clash on
some struct definitions:

  /usr/include/linux/shm.h:26:8: error: redefinition of ‘struct shmid_ds’
     26 | struct shmid_ds {
        |        ^~~~~~~~
  In file included from /usr/include/x86_64-linux-gnu/bits/shm.h:45,
                   from /usr/include/x86_64-linux-gnu/sys/shm.h:30:
  /usr/include/x86_64-linux-gnu/bits/types/struct_shmid_ds.h:24:8: note: originally defined here
     24 | struct shmid_ds
        |        ^~~~~~~~

For now, guard the SHM_HUGE_* defines with ifndef to prevent
redefinition warnings on Android bionic.

Signed-off-by: Edward Liaw <edliaw@google.com>
---
 tools/testing/selftests/mm/thuge-gen.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/tools/testing/selftests/mm/thuge-gen.c b/tools/testing/selftests/mm/thuge-gen.c
index 034635317935..d50dc71cac32 100644
--- a/tools/testing/selftests/mm/thuge-gen.c
+++ b/tools/testing/selftests/mm/thuge-gen.c
@@ -34,10 +34,18 @@
 #endif
 
 #define SHM_HUGETLB     04000   /* segment will use huge TLB pages */
+#ifndef SHM_HUGE_SHIFT
 #define SHM_HUGE_SHIFT  26
+#endif
+#ifndef SHM_HUGE_MASK
 #define SHM_HUGE_MASK   0x3f
+#endif
+#ifndef SHM_HUGE_2MB
 #define SHM_HUGE_2MB    (21 << SHM_HUGE_SHIFT)
+#endif
+#ifndef SHM_HUGE_1GB
 #define SHM_HUGE_1GB    (30 << SHM_HUGE_SHIFT)
+#endif
 
 #define NUM_PAGESIZES   5
 #define NUM_PAGES 4
-- 
2.45.1.467.gbab1589fc0-goog

Re: [PATCH v1 2/2] selftests/mm: Guard defines from shm

[Carlos Llamas]

On Wed, Jun 05, 2024 at 10:36:35PM +0000, Edward Liaw wrote:
> thuge-gen.c defines SHM_HUGE_* macros that are provided by the uapi
> since 4.14. These macros get redefined when compiling with Android's
> bionic because its sys/shm.h will import the uapi definitions.
> 
> However if linux/shm.h is included, with glibc, sys/shm.h will clash on
> some struct definitions:
> 
>   /usr/include/linux/shm.h:26:8: error: redefinition of ‘struct shmid_ds’
>      26 | struct shmid_ds {
>         |        ^~~~~~~~
>   In file included from /usr/include/x86_64-linux-gnu/bits/shm.h:45,
>                    from /usr/include/x86_64-linux-gnu/sys/shm.h:30:
>   /usr/include/x86_64-linux-gnu/bits/types/struct_shmid_ds.h:24:8: note: originally defined here
>      24 | struct shmid_ds
>         |        ^~~~~~~~

I also hit this issue while using bionic. I have a thread reporting this
issue here: https://lore.kernel.org/all/Zl4LC9lTNptB2xTJ@google.com/

According to Andi Kleen, glibc and others should add these defines in
their headers. So it make sense to me that we would need indefs to
prevent a redefine macro issue.

> 
> For now, guard the SHM_HUGE_* defines with ifndef to prevent
> redefinition warnings on Android bionic.
> 
> Signed-off-by: Edward Liaw <edliaw@google.com>
> ---
>  tools/testing/selftests/mm/thuge-gen.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/tools/testing/selftests/mm/thuge-gen.c b/tools/testing/selftests/mm/thuge-gen.c
> index 034635317935..d50dc71cac32 100644
> --- a/tools/testing/selftests/mm/thuge-gen.c
> +++ b/tools/testing/selftests/mm/thuge-gen.c
> @@ -34,10 +34,18 @@
>  #endif
>  
>  #define SHM_HUGETLB     04000   /* segment will use huge TLB pages */
> +#ifndef SHM_HUGE_SHIFT

nit: I see this file uses the form "#if !defined()" above for
MAP_HUGETLB. Perhaps it makes sense to use that and keep things
consistent? I'm fine either way.

Reviewed-by: Carlos Llamas <cmllamas@google.com>

>  #define SHM_HUGE_SHIFT  26
> +#endif
> +#ifndef SHM_HUGE_MASK
>  #define SHM_HUGE_MASK   0x3f
> +#endif
> +#ifndef SHM_HUGE_2MB
>  #define SHM_HUGE_2MB    (21 << SHM_HUGE_SHIFT)
> +#endif
> +#ifndef SHM_HUGE_1GB
>  #define SHM_HUGE_1GB    (30 << SHM_HUGE_SHIFT)
> +#endif
>  
>  #define NUM_PAGESIZES   5
>  #define NUM_PAGES 4
> -- 
> 2.45.1.467.gbab1589fc0-goog
> 

Re: [PATCH v1 2/2] selftests/mm: Guard defines from shm

[Muhammad Usama Anjum]

On 6/6/24 3:36 AM, Edward Liaw wrote:
> thuge-gen.c defines SHM_HUGE_* macros that are provided by the uapi
> since 4.14. These macros get redefined when compiling with Android's
> bionic because its sys/shm.h will import the uapi definitions.
> 
> However if linux/shm.h is included, with glibc, sys/shm.h will clash on
> some struct definitions:
> 
>   /usr/include/linux/shm.h:26:8: error: redefinition of ‘struct shmid_ds’
>      26 | struct shmid_ds {
>         |        ^~~~~~~~
>   In file included from /usr/include/x86_64-linux-gnu/bits/shm.h:45,
>                    from /usr/include/x86_64-linux-gnu/sys/shm.h:30:
>   /usr/include/x86_64-linux-gnu/bits/types/struct_shmid_ds.h:24:8: note: originally defined here
>      24 | struct shmid_ds
>         |        ^~~~~~~~
> 
> For now, guard the SHM_HUGE_* defines with ifndef to prevent
> redefinition warnings on Android bionic.
LGTM
Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com>

> 
> Signed-off-by: Edward Liaw <edliaw@google.com>
> ---
>  tools/testing/selftests/mm/thuge-gen.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/tools/testing/selftests/mm/thuge-gen.c b/tools/testing/selftests/mm/thuge-gen.c
> index 034635317935..d50dc71cac32 100644
> --- a/tools/testing/selftests/mm/thuge-gen.c
> +++ b/tools/testing/selftests/mm/thuge-gen.c
> @@ -34,10 +34,18 @@
>  #endif
>  
>  #define SHM_HUGETLB     04000   /* segment will use huge TLB pages */
> +#ifndef SHM_HUGE_SHIFT
>  #define SHM_HUGE_SHIFT  26
> +#endif
> +#ifndef SHM_HUGE_MASK
>  #define SHM_HUGE_MASK   0x3f
> +#endif
> +#ifndef SHM_HUGE_2MB
>  #define SHM_HUGE_2MB    (21 << SHM_HUGE_SHIFT)
> +#endif
> +#ifndef SHM_HUGE_1GB
>  #define SHM_HUGE_1GB    (30 << SHM_HUGE_SHIFT)
> +#endif
>  
>  #define NUM_PAGESIZES   5
>  #define NUM_PAGES 4

-- 
BR,
Muhammad Usama Anjum