Re: [PATCH 0/1] prctl: allow overriding system THP policy to always

[Usama Arif <usamaarif642@gmail.com>]

On 07/05/2025 15:57, Zi Yan wrote:
> +Yafang, who is also looking at changing THP config at cgroup/container level.
> 
> On 7 May 2025, at 10:00, Usama Arif wrote:
> 
>> Allowing override of global THP policy per process allows workloads
>> that have shown to benefit from hugepages to do so, without regressing
>> workloads that wouldn't benefit. This will allow such types of
>> workloads to be run/stacked on the same machine.
>>
>> It also helps in rolling out hugepages in hyperscaler configurations
>> for workloads that benefit from them, where a single THP policy is
>> likely to be used across the entire fleet, and prctl will help override it.
>>
>> An advantage of doing it via prctl vs creating a cgroup specific
>> option (like /sys/fs/cgroup/test/memory.transparent_hugepage.enabled) is
>> that this will work even when there are no cgroups present, and my
>> understanding is there is a strong preference of cgroups controls being
>> hierarchical which usually means them having a numerical value.
> 
> Hi Usama,
> 
> Do you mind giving an example on how to change THP policy for a set of
> processes running in a container (under a cgroup)?

Hi Zi,

In our case, we create the processes in the cgroup via systemd. The way we will enable THP=always
for processes in a cgroup is in the same way we enable KSM for the cgroup.
The change in systemd would be very similar to the line in [1], where we would set prctl PR_SET_THP_ALWAYS
in exec-invoke.
This is at the start of the process, but you would already know at the start of the process
whether you want THP=always for it or not.

[1] https://github.com/systemd/systemd/blob/2e72d3efafa88c1cb4d9b28dd4ade7c6ab7be29a/src/core/exec-invoke.c#L5045

Thanks,
Usama

> 
> Yafang mentioned that the prctl approach would require restarting all running
> services[1] and other inflexiblities, so he proposed to use BPF to change THP
> policy[2]. I wonder if Yafang's issues also apply to your case and if you
> have a solution to them.
> 
> Thanks.
> 
> [1] https://lore.kernel.org/linux-mm/CALOAHbCXMi2GaZdHJaNLXxGsJf-hkDTrztsQiceaBcJ8d8p3cA@mail.gmail.com/
> [2] https://lore.kernel.org/linux-mm/20250429024139.34365-1-laoar.shao@gmail.com/
>>
>>
>> The output and code of test program is below:
>>
>> [root@vm4 vmuser]# echo madvise > /sys/kernel/mm/transparent_hugepage/enabled
>> [root@vm4 vmuser]# echo inherit > /sys/kernel/mm/transparent_hugepage/hugepages-2048kB/enabled
>> [root@vm4 vmuser]# ./a.out
>> Default THP setting:
>> THP is not set to 'always'.
>> PR_SET_THP_ALWAYS = 1
>> THP is set to 'always'.
>> PR_SET_THP_ALWAYS = 0
>> THP is not set to 'always'.
>>
>>
>> #include <stdio.h>
>> #include <stdlib.h>
>> #include <string.h>
>> #include <unistd.h>
>> #include <sys/mman.h>
>> #include <sys/prctl.h>
>>
>> #define PR_SET_THP_ALWAYS 78
>> #define SIZE 12 * (2 * 1024 * 1024) // 24 MB
>>
>> void check_smaps(void) {
>>     FILE *file = fopen("/proc/self/smaps", "r");
>>     if (!file) {
>>         perror("fopen");
>>         return;
>>     }
>>
>>     char line[256];
>>     int is_hugepage = 0;
>>     while (fgets(line, sizeof(line), file)) {
>>         // if (strstr(line, "AnonHugePages:"))
>>         //     printf("%s\n", line);
>>         if (strstr(line, "AnonHugePages:") && strstr(line, "24576 kB"))
>> {
>>             // printf("%s\n", line);
>>             is_hugepage = 1;
>>             break;
>>         }
>>     }
>>     fclose(file);
>>     if (is_hugepage) {
>>         printf("THP is set to 'always'.\n");
>>     } else {
>>         printf("THP is not set to 'always'.\n");
>>     }
>> }
>>
>> void test_mmap_thp(void) {
>>     char *buffer = (char *)mmap(NULL, SIZE, PROT_READ | PROT_WRITE,
>>                                 MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
>>     if (buffer == MAP_FAILED) {
>>         perror("mmap");
>>         return;
>>     }
>>     // Touch the memory to ensure it's allocated
>>     memset(buffer, 0, SIZE);
>>     check_smaps();
>>     munmap(buffer, SIZE);
>> }
>>
>> int main() {
>>     printf("Default THP setting: \n");
>>     test_mmap_thp();
>>     printf("PR_SET_THP_ALWAYS = 1 \n");
>>     prctl(PR_SET_THP_ALWAYS, 1, NULL, NULL, NULL);
>>     test_mmap_thp();
>>     printf("PR_SET_THP_ALWAYS = 0 \n");
>>     prctl(PR_SET_THP_ALWAYS, 0, NULL, NULL, NULL);
>>     test_mmap_thp();
>>
>>     return 0;
>> }
>>
>>
>> Usama Arif (1):
>>   prctl: allow overriding system THP policy to always per process
>>
>>  include/linux/huge_mm.h                          |  3 ++-
>>  include/linux/mm_types.h                         |  7 ++-----
>>  include/uapi/linux/prctl.h                       |  3 +++
>>  kernel/sys.c                                     | 16 ++++++++++++++++
>>  tools/include/uapi/linux/prctl.h                 |  3 +++
>>  .../perf/trace/beauty/include/uapi/linux/prctl.h |  3 +++
>>  6 files changed, 29 insertions(+), 6 deletions(-)
>>
>> -- 
>> 2.47.1
> 
> 
> --
> Best Regards,
> Yan, Zi