[PATCH v2 1/2] init/Kconfig: Fix CPU number in LOG_CPU_MAX_BUF_SHIFT description

[PATCH v2 1/2] init/Kconfig: Fix CPU number in LOG_CPU_MAX_BUF_SHIFT description

[Paul Menzel]

Currently, LOG_BUF_SHIFT defaults to 17, which is 2 ^ 17 bytes = 128 KB,
and LOG_CPU_MAX_BUF_SHIFT defaults to 12, which is 2 ^ 12 bytes = 4 KB.

Half of 128 KB is 64 KB, so more than 16 CPUs are required for the value
to be used, as then the sum of contributions is greater than 64 KB for
the first time. My guess is, that the description was written with the
configuration values used in the SUSE in mind.

Fixes: 23b2899f7f ("printk: allow increasing the ring buffer depending on the number of CPUs")
Cc: Luis R. Rodriguez <mcgrof@suse.com>
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Petr Mladek <pmladek@suse.com>
Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
---
v2: Add Reviewed-by tag

 init/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/init/Kconfig b/init/Kconfig
index d6a0b31b13dc..9dc607e3806f 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -718,7 +718,7 @@ config LOG_CPU_MAX_BUF_SHIFT
 	  with more CPUs. Therefore this value is used only when the sum of
 	  contributions is greater than the half of the default kernel ring
 	  buffer as defined by LOG_BUF_SHIFT. The default values are set
-	  so that more than 64 CPUs are needed to trigger the allocation.
+	  so that more than 16 CPUs are needed to trigger the allocation.
 
 	  Also this option is ignored when "log_buf_len" kernel parameter is
 	  used as it forces an exact (power of two) size of the ring buffer.
-- 
2.28.0

[PATCH v2 2/2] init/Kconfig: Increase default log buffer size from 128 KB to 512 KB

[Paul Menzel]

Commit f17a32e97e (let LOG_BUF_SHIFT default to 17) from 2008 was the
last time, the the default log buffer size bump was increased.

Machines have evolved, and on current hardware, enough memory is
present, and some devices have over 200 PCI devices, like a two socket
Skylake-E server, resulting a lot of lines.

Therefore, increase the default from 128 KB to 512 KB. Anyone, with
limited memory, can still lower it.

Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
Cc: linux-kernel@vger.kernel.org
---
v2: New patch in series.

Is sending it to linux-kernel enough? If not, who to send it also to?

 init/Kconfig | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/init/Kconfig b/init/Kconfig
index 9dc607e3806f..13df63517cc2 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -681,9 +681,9 @@ config IKHEADERS
 	  kheaders.ko is built which can be loaded on-demand to get access to headers.
 
 config LOG_BUF_SHIFT
-	int "Kernel log buffer size (16 => 64KB, 17 => 128KB)"
+	int "Kernel log buffer size (17 => 128KB, 19 => 512KB)"
 	range 12 25
-	default 17
+	default 19
 	depends on PRINTK
 	help
 	  Select the minimal kernel log buffer size as a power of 2.
@@ -692,6 +692,8 @@ config LOG_BUF_SHIFT
 	  by "log_buf_len" boot parameter.
 
 	  Examples:
+		     19 => 512 KB
+		     18 => 256 KB
 		     17 => 128 KB
 		     16 => 64 KB
 		     15 => 32 KB
@@ -718,7 +720,7 @@ config LOG_CPU_MAX_BUF_SHIFT
 	  with more CPUs. Therefore this value is used only when the sum of
 	  contributions is greater than the half of the default kernel ring
 	  buffer as defined by LOG_BUF_SHIFT. The default values are set
-	  so that more than 16 CPUs are needed to trigger the allocation.
+	  so that more than 64 CPUs are needed to trigger the allocation.
 
 	  Also this option is ignored when "log_buf_len" kernel parameter is
 	  used as it forces an exact (power of two) size of the ring buffer.
-- 
2.28.0

Re: [PATCH v2 2/2] init/Kconfig: Increase default log buffer size from 128 KB to 512 KB

[Petr Mladek]

On Tue 2020-08-11 11:29:24, Paul Menzel wrote:
> Commit f17a32e97e (let LOG_BUF_SHIFT default to 17) from 2008 was the
> last time, the the default log buffer size bump was increased.
> 
> Machines have evolved, and on current hardware, enough memory is
> present, and some devices have over 200 PCI devices, like a two socket
> Skylake-E server, resulting a lot of lines.
> 
> Therefore, increase the default from 128 KB to 512 KB. Anyone, with
> limited memory, can still lower it.
> 
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -681,9 +681,9 @@ config IKHEADERS
>  	  kheaders.ko is built which can be loaded on-demand to get access to headers.
>  
>  config LOG_BUF_SHIFT
> -	int "Kernel log buffer size (16 => 64KB, 17 => 128KB)"
> +	int "Kernel log buffer size (17 => 128KB, 19 => 512KB)"
>  	range 12 25
> -	default 17
> +	default 19
>  	depends on PRINTK
>  	help
>  	  Select the minimal kernel log buffer size as a power of 2.

Honestly, I do not have experience with changing the defaults. People
hacking small devices might complain. Well, this can be solved
by increasing the default only when BASE_FULL is set.

I am personally fine with increasing the default when BASE_FULL
is set. The amount of messages is growing over time because of
increasing complexity of both the hardware and software.
Fortunately also the amount of available memory is growing.

Well, this should get discussed in wider audience. Adding some
people into CC.

JFYI, it started with report of lost messages, see
https://lore.kernel.org/lkml/264bfbae-122d-9c41-59ea-6413f91bd866@molgen.mpg.de/

Best Regards,
Petr

Re: [PATCH v2 2/2] init/Kconfig: Increase default log buffer size from 128 KB to 512 KB

[Paul Menzel]

Dear Petr,


Am 11.08.20 um 12:53 schrieb Petr Mladek:
> On Tue 2020-08-11 11:29:24, Paul Menzel wrote:
>> Commit f17a32e97e (let LOG_BUF_SHIFT default to 17) from 2008 was the
>> last time, the the default log buffer size bump was increased.
>>
>> Machines have evolved, and on current hardware, enough memory is
>> present, and some devices have over 200 PCI devices, like a two socket
>> Skylake-E server, resulting a lot of lines.
>>
>> Therefore, increase the default from 128 KB to 512 KB. Anyone, with
>> limited memory, can still lower it.
>>
>> --- a/init/Kconfig
>> +++ b/init/Kconfig
>> @@ -681,9 +681,9 @@ config IKHEADERS
>>   	  kheaders.ko is built which can be loaded on-demand to get access to headers.
>>   
>>   config LOG_BUF_SHIFT
>> -	int "Kernel log buffer size (16 => 64KB, 17 => 128KB)"
>> +	int "Kernel log buffer size (17 => 128KB, 19 => 512KB)"
>>   	range 12 25
>> -	default 17
>> +	default 19
>>   	depends on PRINTK
>>   	help
>>   	  Select the minimal kernel log buffer size as a power of 2.
> 
> Honestly, I do not have experience with changing the defaults. People
> hacking small devices might complain. Well, this can be solved
> by increasing the default only when BASE_FULL is set.
> 
> I am personally fine with increasing the default when BASE_FULL
> is set. The amount of messages is growing over time because of
> increasing complexity of both the hardware and software.
> Fortunately also the amount of available memory is growing.
> 
> Well, this should get discussed in wider audience. Adding some
> people into CC.
> 
> JFYI, it started with report of lost messages, see
> https://lore.kernel.org/lkml/264bfbae-122d-9c41-59ea-6413f91bd866@molgen.mpg.de/

As there was no objection, is it possible to apply the two patches, and 
maybe even get them into Linux 5.10?


Kind regards,

Paul

Re: [PATCH v2 2/2] init/Kconfig: Increase default log buffer size from 128 KB to 512 KB

[Petr Mladek]

On Thu 2020-10-29 23:16:01, Paul Menzel wrote:
> Dear Petr,
> 
> 
> Am 11.08.20 um 12:53 schrieb Petr Mladek:
> > On Tue 2020-08-11 11:29:24, Paul Menzel wrote:
> > > Commit f17a32e97e (let LOG_BUF_SHIFT default to 17) from 2008 was the
> > > last time, the the default log buffer size bump was increased.
> > > 
> > > Machines have evolved, and on current hardware, enough memory is
> > > present, and some devices have over 200 PCI devices, like a two socket
> > > Skylake-E server, resulting a lot of lines.
> > > 
> > > Therefore, increase the default from 128 KB to 512 KB. Anyone, with
> > > limited memory, can still lower it.
> > > 
> > > --- a/init/Kconfig
> > > +++ b/init/Kconfig
> > > @@ -681,9 +681,9 @@ config IKHEADERS
> > >   	  kheaders.ko is built which can be loaded on-demand to get access to headers.
> > >   config LOG_BUF_SHIFT
> > > -	int "Kernel log buffer size (16 => 64KB, 17 => 128KB)"
> > > +	int "Kernel log buffer size (17 => 128KB, 19 => 512KB)"
> > >   	range 12 25
> > > -	default 17
> > > +	default 19
> > >   	depends on PRINTK
> > >   	help
> > >   	  Select the minimal kernel log buffer size as a power of 2.
> > 
> > Honestly, I do not have experience with changing the defaults. People
> > hacking small devices might complain. Well, this can be solved
> > by increasing the default only when BASE_FULL is set.
> > 
> > I am personally fine with increasing the default when BASE_FULL
> > is set. The amount of messages is growing over time because of
> > increasing complexity of both the hardware and software.
> > Fortunately also the amount of available memory is growing.
> > 
> > Well, this should get discussed in wider audience. Adding some
> > people into CC.
> > 
> > JFYI, it started with report of lost messages, see
> > https://lore.kernel.org/lkml/264bfbae-122d-9c41-59ea-6413f91bd866@molgen.mpg.de/
> 
> As there was no objection, is it possible to apply the two patches, and
> maybe even get them into Linux 5.10?

Thanks for reminding me. I am sorry but it is too late for
5.10. Such a change should be added during the merge window.

Well, the size of the ring buffer has effectively increased in 5.10.
The lockless implementation stores strings and metadata separately.
It basically doubled the memory needs and people around embedded
devices were not happy, see
https://lore.kernel.org/r/CAMuHMdXHFFUrjRMEHnXXU8QQkgD9x_S6R3N0Q7Q4H2RSfy2GGw@mail.gmail.com

Please update the patch so that the default stays the same for
BASE_SMALL. Please, add Rasmus Villemoes
<linux@rasmusvillemoes.dk> and Geert Uytterhoeven
<geert@linux-m68k.org> into CC when you send it.

Best Regards,
Petr

Re: [PATCH v2 2/2] init/Kconfig: Increase default log buffer size from 128 KB to 512 KB

[Geert Uytterhoeven]

On Tue, Aug 11, 2020 at 12:55 PM Petr Mladek <pmladek@suse.com> wrote:
> On Tue 2020-08-11 11:29:24, Paul Menzel wrote:
> > Commit f17a32e97e (let LOG_BUF_SHIFT default to 17) from 2008 was the
> > last time, the the default log buffer size bump was increased.
> >
> > Machines have evolved, and on current hardware, enough memory is
> > present, and some devices have over 200 PCI devices, like a two socket
> > Skylake-E server, resulting a lot of lines.
> >
> > Therefore, increase the default from 128 KB to 512 KB. Anyone, with
> > limited memory, can still lower it.
> >
> > --- a/init/Kconfig
> > +++ b/init/Kconfig
> > @@ -681,9 +681,9 @@ config IKHEADERS
> >         kheaders.ko is built which can be loaded on-demand to get access to headers.
> >
> >  config LOG_BUF_SHIFT
> > -     int "Kernel log buffer size (16 => 64KB, 17 => 128KB)"
> > +     int "Kernel log buffer size (17 => 128KB, 19 => 512KB)"
> >       range 12 25
> > -     default 17
> > +     default 19
> >       depends on PRINTK
> >       help
> >         Select the minimal kernel log buffer size as a power of 2.
>
> Honestly, I do not have experience with changing the defaults. People
> hacking small devices might complain. Well, this can be solved
> by increasing the default only when BASE_FULL is set.
>
> I am personally fine with increasing the default when BASE_FULL
> is set. The amount of messages is growing over time because of
> increasing complexity of both the hardware and software.
> Fortunately also the amount of available memory is growing.

Note that making this change means that some of the embedded
defconfigs may need to gain a CONFIG_LOG_BUF_SHIFT=17
line...

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH v2 1/2] init/Kconfig: Fix CPU number in LOG_CPU_MAX_BUF_SHIFT description

[Paul Menzel]

Dear Petr,


Am 11.08.20 um 11:29 schrieb Paul Menzel:
> Currently, LOG_BUF_SHIFT defaults to 17, which is 2 ^ 17 bytes = 128 KB,
> and LOG_CPU_MAX_BUF_SHIFT defaults to 12, which is 2 ^ 12 bytes = 4 KB.
> 
> Half of 128 KB is 64 KB, so more than 16 CPUs are required for the value
> to be used, as then the sum of contributions is greater than 64 KB for
> the first time. My guess is, that the description was written with the
> configuration values used in the SUSE in mind.
> 
> Fixes: 23b2899f7f ("printk: allow increasing the ring buffer depending on the number of CPUs")
> Cc: Luis R. Rodriguez <mcgrof@suse.com>
> Cc: linux-kernel@vger.kernel.org
> Reviewed-by: Petr Mladek <pmladek@suse.com>
> Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
> ---
> v2: Add Reviewed-by tag
> 
>   init/Kconfig | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/init/Kconfig b/init/Kconfig
> index d6a0b31b13dc..9dc607e3806f 100644
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -718,7 +718,7 @@ config LOG_CPU_MAX_BUF_SHIFT
>   	  with more CPUs. Therefore this value is used only when the sum of
>   	  contributions is greater than the half of the default kernel ring
>   	  buffer as defined by LOG_BUF_SHIFT. The default values are set
> -	  so that more than 64 CPUs are needed to trigger the allocation.
> +	  so that more than 16 CPUs are needed to trigger the allocation.
>   
>   	  Also this option is ignored when "log_buf_len" kernel parameter is
>   	  used as it forces an exact (power of two) size of the ring buffer.

Could you please apply this trivial patch from the two patches already, 
so I do not have to resend it?


Kind regards,

Paul

Re: [PATCH v2 1/2] init/Kconfig: Fix CPU number in LOG_CPU_MAX_BUF_SHIFT description

[Petr Mladek]

On Fri 2020-10-30 17:00:18, Paul Menzel wrote:
> Dear Petr,
> 
> 
> Am 11.08.20 um 11:29 schrieb Paul Menzel:
> > Currently, LOG_BUF_SHIFT defaults to 17, which is 2 ^ 17 bytes = 128 KB,
> > and LOG_CPU_MAX_BUF_SHIFT defaults to 12, which is 2 ^ 12 bytes = 4 KB.
> > 
> > Half of 128 KB is 64 KB, so more than 16 CPUs are required for the value
> > to be used, as then the sum of contributions is greater than 64 KB for
> > the first time. My guess is, that the description was written with the
> > configuration values used in the SUSE in mind.
> > 
> > Fixes: 23b2899f7f ("printk: allow increasing the ring buffer depending on the number of CPUs")
> > Cc: Luis R. Rodriguez <mcgrof@suse.com>
> > Cc: linux-kernel@vger.kernel.org
> > Reviewed-by: Petr Mladek <pmladek@suse.com>
> > Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
> > ---
> > v2: Add Reviewed-by tag
> > 
> >   init/Kconfig | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/init/Kconfig b/init/Kconfig
> > index d6a0b31b13dc..9dc607e3806f 100644
> > --- a/init/Kconfig
> > +++ b/init/Kconfig
> > @@ -718,7 +718,7 @@ config LOG_CPU_MAX_BUF_SHIFT
> >   	  with more CPUs. Therefore this value is used only when the sum of
> >   	  contributions is greater than the half of the default kernel ring
> >   	  buffer as defined by LOG_BUF_SHIFT. The default values are set
> > -	  so that more than 64 CPUs are needed to trigger the allocation.
> > +	  so that more than 16 CPUs are needed to trigger the allocation.
> >   	  Also this option is ignored when "log_buf_len" kernel parameter is
> >   	  used as it forces an exact (power of two) size of the ring buffer.
> 
> Could you please apply this trivial patch from the two patches already, so I
> do not have to resend it?

The patch is committed in printk/linux.git, branch for-5.10-trivial.

I am not going to create pull request just for this trivial fix.
I will push it for-5.10 only together with eventual more urgent fix.
It is very likely that it will have to wait for 5.11.

Best Regards,
Petr