[PATCH v2 1/2] kernel/fork.c: get totalram_pages from memblock to calculate max_threads

[PATCH v2 1/2] kernel/fork.c: get totalram_pages from memblock to calculate max_threads

[Wei Yang]

Since we plan to move the accounting into __free_pages_core(),
totalram_pages may not represent the total usable pages on system
at this point when defer_init is enabled.

Instead we can get the total usable pages from memblock directly.

Signed-off-by: Wei Yang <richard.weiyang@gmail.com>
CC: Mike Rapoport (IBM) <rppt@kernel.org>
CC: David Hildenbrand <david@redhat.com>
CC: Oleg Nesterov <oleg@redhat.com>
---
 kernel/fork.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/kernel/fork.c b/kernel/fork.c
index 99076dbe27d8..eb2b5f96aa61 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -44,6 +44,7 @@
 #include <linux/fs.h>
 #include <linux/mm.h>
 #include <linux/mm_inline.h>
+#include <linux/memblock.h>
 #include <linux/nsproxy.h>
 #include <linux/capability.h>
 #include <linux/cpu.h>
@@ -999,7 +1000,7 @@ void __init __weak arch_task_cache_init(void) { }
 static void set_max_threads(unsigned int max_threads_suggested)
 {
 	u64 threads;
-	unsigned long nr_pages = totalram_pages();
+	unsigned long nr_pages = PHYS_PFN(memblock_phys_mem_size() - memblock_reserved_size());
 
 	/*
 	 * The number of threads shall be limited such that the thread
-- 
2.34.1

[PATCH v2 2/2] kernel/fork.c: put set_max_threads()/task_struct_whitelist() in __init section

[Wei Yang]

Function set_max_threads()/task_struct_whitelist() is only used by
fork_init() during bootup.

Let's add __init tag to them.

Signed-off-by: Wei Yang <richard.weiyang@gmail.com>
Suggested-by: Oleg Nesterov <oleg@redhat.com>
---
 kernel/fork.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/fork.c b/kernel/fork.c
index eb2b5f96aa61..b42334b53bd6 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -997,7 +997,7 @@ void __init __weak arch_task_cache_init(void) { }
 /*
  * set_max_threads
  */
-static void set_max_threads(unsigned int max_threads_suggested)
+static void __init set_max_threads(unsigned int max_threads_suggested)
 {
 	u64 threads;
 	unsigned long nr_pages = PHYS_PFN(memblock_phys_mem_size() - memblock_reserved_size());
@@ -1023,7 +1023,7 @@ static void set_max_threads(unsigned int max_threads_suggested)
 int arch_task_struct_size __read_mostly;
 #endif
 
-static void task_struct_whitelist(unsigned long *offset, unsigned long *size)
+static void __init task_struct_whitelist(unsigned long *offset, unsigned long *size)
 {
 	/* Fetch thread_struct whitelist for the architecture. */
 	arch_thread_struct_whitelist(offset, size);
-- 
2.34.1

Re: [PATCH v2 1/2] kernel/fork.c: get totalram_pages from memblock to calculate max_threads

[Andrew Morton]

On Mon,  1 Jul 2024 01:34:09 +0000 Wei Yang <richard.weiyang@gmail.com> wrote:

> Since we plan to move the accounting into __free_pages_core(),
> totalram_pages may not represent the total usable pages on system
> at this point when defer_init is enabled.

Yes, things like totalram_pages() are very old, and were a good idea at the
time, but things moved on.

> Instead we can get the total usable pages from memblock directly.
> 
> --- a/kernel/fork.c
> +++ b/kernel/fork.c
> @@ -44,6 +44,7 @@
>  #include <linux/fs.h>
>  #include <linux/mm.h>
>  #include <linux/mm_inline.h>
> +#include <linux/memblock.h>
>  #include <linux/nsproxy.h>
>  #include <linux/capability.h>
>  #include <linux/cpu.h>
> @@ -999,7 +1000,7 @@ void __init __weak arch_task_cache_init(void) { }
>  static void set_max_threads(unsigned int max_threads_suggested)
>  {
>  	u64 threads;
> -	unsigned long nr_pages = totalram_pages();
> +	unsigned long nr_pages = PHYS_PFN(memblock_phys_mem_size() - memblock_reserved_size());

The result of this subtraction has meaning.  Even if it is only used
once, I suspect it should be in a standalone function which has
documentation which describes that meaning.  Having fork.c make an
open-coded poke into memblock internals seems wrong, no?

Re: [PATCH v2 1/2] kernel/fork.c: get totalram_pages from memblock to calculate max_threads

[David Hildenbrand]

On 02.07.24 06:26, Andrew Morton wrote:
> On Mon,  1 Jul 2024 01:34:09 +0000 Wei Yang <richard.weiyang@gmail.com> wrote:
> 
>> Since we plan to move the accounting into __free_pages_core(),
>> totalram_pages may not represent the total usable pages on system
>> at this point when defer_init is enabled.
> 
> Yes, things like totalram_pages() are very old, and were a good idea at the
> time, but things moved on.
> 
>> Instead we can get the total usable pages from memblock directly.
>>
>> --- a/kernel/fork.c
>> +++ b/kernel/fork.c
>> @@ -44,6 +44,7 @@
>>   #include <linux/fs.h>
>>   #include <linux/mm.h>
>>   #include <linux/mm_inline.h>
>> +#include <linux/memblock.h>
>>   #include <linux/nsproxy.h>
>>   #include <linux/capability.h>
>>   #include <linux/cpu.h>
>> @@ -999,7 +1000,7 @@ void __init __weak arch_task_cache_init(void) { }
>>   static void set_max_threads(unsigned int max_threads_suggested)
>>   {
>>   	u64 threads;
>> -	unsigned long nr_pages = totalram_pages();
>> +	unsigned long nr_pages = PHYS_PFN(memblock_phys_mem_size() - memblock_reserved_size());
> 
> The result of this subtraction has meaning.  Even if it is only used
> once, I suspect it should be in a standalone function which has
> documentation which describes that meaning.  Having fork.c make an
> open-coded poke into memblock internals seems wrong, no?

I was just about the comment the same thing:

I'm starting to wonder if we should have a helper like

memblock_estimated_nr_pages()

or sth like that that abstracts this?

-- 
Cheers,

David / dhildenb

Re: [PATCH v2 1/2] kernel/fork.c: get totalram_pages from memblock to calculate max_threads

[Wei Yang]

On Tue, Jul 02, 2024 at 09:02:31AM +0200, David Hildenbrand wrote:
>On 02.07.24 06:26, Andrew Morton wrote:
>> On Mon,  1 Jul 2024 01:34:09 +0000 Wei Yang <richard.weiyang@gmail.com> wrote:
>> 
>> > Since we plan to move the accounting into __free_pages_core(),
>> > totalram_pages may not represent the total usable pages on system
>> > at this point when defer_init is enabled.
>> 
>> Yes, things like totalram_pages() are very old, and were a good idea at the
>> time, but things moved on.
>> 
>> > Instead we can get the total usable pages from memblock directly.
>> > 
>> > --- a/kernel/fork.c
>> > +++ b/kernel/fork.c
>> > @@ -44,6 +44,7 @@
>> >   #include <linux/fs.h>
>> >   #include <linux/mm.h>
>> >   #include <linux/mm_inline.h>
>> > +#include <linux/memblock.h>
>> >   #include <linux/nsproxy.h>
>> >   #include <linux/capability.h>
>> >   #include <linux/cpu.h>
>> > @@ -999,7 +1000,7 @@ void __init __weak arch_task_cache_init(void) { }
>> >   static void set_max_threads(unsigned int max_threads_suggested)
>> >   {
>> >   	u64 threads;
>> > -	unsigned long nr_pages = totalram_pages();
>> > +	unsigned long nr_pages = PHYS_PFN(memblock_phys_mem_size() - memblock_reserved_size());
>> 
>> The result of this subtraction has meaning.  Even if it is only used
>> once, I suspect it should be in a standalone function which has
>> documentation which describes that meaning.  Having fork.c make an
>> open-coded poke into memblock internals seems wrong, no?

Agree, this is not a good practise.

>
>I was just about the comment the same thing:
>
>I'm starting to wonder if we should have a helper like
>
>memblock_estimated_nr_pages()
>
>or sth like that that abstracts this?

Hmm... not good at naming, I am ok with this. 

I would prepare v2 with this. If someone have a better idea, I would change
it.

>
>-- 
>Cheers,
>
>David / dhildenb

-- 
Wei Yang
Help you, Help me