OpenBMC kernel and CONFIG_PREEMPT

OpenBMC kernel and CONFIG_PREEMPT

[Patrick Williams]

[-- Attachment #1: Type: text/plain, Size: 572 bytes --]

Joel,

Would you mind looking at this commit[1]?  Nuvoton seems to have a lot
of data that suggests that disabling CONFIG_PREEMPT leads to
significantly better performance.  The boot time is almost 50% faster.

I'm surprised that:

    1. CONFIG_PREEMPT really results in that worse of performance.
    2. The Nuvoton chip is special from any other ARM variant in this
       regard.

Should we disable this across the board on all our platforms?

1. https://github.com/openbmc/openbmc/commit/52ec37dd0b8776ce7f43e65e0be578c3606340b7
-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: OpenBMC kernel and CONFIG_PREEMPT

[William Kennington]

[-- Attachment #1: Type: text/plain, Size: 1127 bytes --]

It's interesting because I think it has a bigger impact on the 8xx chip
than the 7xx chip. AFAIK I was indirectly comparing this and it seems like
it didn't make a huge difference for us on our older BMCs. We have it
disabled on our 5.15 configs, but used it in 5.10. I don't think we noticed
a meaningful difference when porting forward. Probably you need a certain
amount of startup services trying to steal time from each other.

On Fri, Apr 7, 2023 at 12:10 PM Patrick Williams <patrick@stwcx.xyz> wrote:

> Joel,
>
> Would you mind looking at this commit[1]?  Nuvoton seems to have a lot
> of data that suggests that disabling CONFIG_PREEMPT leads to
> significantly better performance.  The boot time is almost 50% faster.
>
> I'm surprised that:
>
>     1. CONFIG_PREEMPT really results in that worse of performance.
>     2. The Nuvoton chip is special from any other ARM variant in this
>        regard.
>
> Should we disable this across the board on all our platforms?
>
> 1.
> https://github.com/openbmc/openbmc/commit/52ec37dd0b8776ce7f43e65e0be578c3606340b7
> --
> Patrick Williams
>

[-- Attachment #2: Type: text/html, Size: 1585 bytes --]

Re: OpenBMC kernel and CONFIG_PREEMPT

[William Kennington]

[-- Attachment #1: Type: text/plain, Size: 1428 bytes --]

I just looked again, AFAICT it was never enabled on aspeed or npcm7xx, just
the new 8xx part. Hopefully someone else with aspeed chips will confirm
from their actual `.config`

On Fri, Apr 7, 2023 at 1:05 PM William Kennington <wak@google.com> wrote:

> It's interesting because I think it has a bigger impact on the 8xx chip
> than the 7xx chip. AFAIK I was indirectly comparing this and it seems like
> it didn't make a huge difference for us on our older BMCs. We have it
> disabled on our 5.15 configs, but used it in 5.10. I don't think we noticed
> a meaningful difference when porting forward. Probably you need a certain
> amount of startup services trying to steal time from each other.
>
> On Fri, Apr 7, 2023 at 12:10 PM Patrick Williams <patrick@stwcx.xyz>
> wrote:
>
>> Joel,
>>
>> Would you mind looking at this commit[1]?  Nuvoton seems to have a lot
>> of data that suggests that disabling CONFIG_PREEMPT leads to
>> significantly better performance.  The boot time is almost 50% faster.
>>
>> I'm surprised that:
>>
>>     1. CONFIG_PREEMPT really results in that worse of performance.
>>     2. The Nuvoton chip is special from any other ARM variant in this
>>        regard.
>>
>> Should we disable this across the board on all our platforms?
>>
>> 1.
>> https://github.com/openbmc/openbmc/commit/52ec37dd0b8776ce7f43e65e0be578c3606340b7
>> --
>> Patrick Williams
>>
>

[-- Attachment #2: Type: text/html, Size: 2135 bytes --]

Re: OpenBMC kernel and CONFIG_PREEMPT

[Zev Weiss]

On Fri, Apr 07, 2023 at 01:12:21PM PDT, William Kennington wrote:
>I just looked again, AFAICT it was never enabled on aspeed or npcm7xx, just
>the new 8xx part. Hopefully someone else with aspeed chips will confirm
>from their actual `.config`
>

Yeah, I'm pretty sure that's correct -- my aspeed builds already show:

    $ zcat /proc/config.gz | grep PREEMPT
    CONFIG_PREEMPT_NONE_BUILD=y
    CONFIG_PREEMPT_NONE=y
    # CONFIG_PREEMPT_VOLUNTARY is not set
    # CONFIG_PREEMPT is not set

Incidentally, out of curiosity I tried booting kernels built with
CONFIG_PREEMPT=y and didn't see any major changes in boot performance on
either ast2500 or ast2400.


Zev


>On Fri, Apr 7, 2023 at 1:05 PM William Kennington <wak@google.com> wrote:
>
>> It's interesting because I think it has a bigger impact on the 8xx chip
>> than the 7xx chip. AFAIK I was indirectly comparing this and it seems like
>> it didn't make a huge difference for us on our older BMCs. We have it
>> disabled on our 5.15 configs, but used it in 5.10. I don't think we noticed
>> a meaningful difference when porting forward. Probably you need a certain
>> amount of startup services trying to steal time from each other.
>>
>> On Fri, Apr 7, 2023 at 12:10 PM Patrick Williams <patrick@stwcx.xyz>
>> wrote:
>>
>>> Joel,
>>>
>>> Would you mind looking at this commit[1]?  Nuvoton seems to have a lot
>>> of data that suggests that disabling CONFIG_PREEMPT leads to
>>> significantly better performance.  The boot time is almost 50% faster.
>>>
>>> I'm surprised that:
>>>
>>>     1. CONFIG_PREEMPT really results in that worse of performance.
>>>     2. The Nuvoton chip is special from any other ARM variant in this
>>>        regard.
>>>
>>> Should we disable this across the board on all our platforms?
>>>
>>> 1.
>>> https://github.com/openbmc/openbmc/commit/52ec37dd0b8776ce7f43e65e0be578c3606340b7
>>> --
>>> Patrick Williams
>>>
>>

Re: OpenBMC kernel and CONFIG_PREEMPT

[Joel Stanley]

On Fri, 7 Apr 2023 at 22:07, Zev Weiss <zweiss@equinix.com> wrote:
>
> On Fri, Apr 07, 2023 at 01:12:21PM PDT, William Kennington wrote:
> >I just looked again, AFAICT it was never enabled on aspeed or npcm7xx, just
> >the new 8xx part. Hopefully someone else with aspeed chips will confirm
> >from their actual `.config`
> >
>
> Yeah, I'm pretty sure that's correct -- my aspeed builds already show:
>
>     $ zcat /proc/config.gz | grep PREEMPT
>     CONFIG_PREEMPT_NONE_BUILD=y
>     CONFIG_PREEMPT_NONE=y
>     # CONFIG_PREEMPT_VOLUNTARY is not set
>     # CONFIG_PREEMPT is not set
>
> Incidentally, out of curiosity I tried booting kernels built with
> CONFIG_PREEMPT=y and didn't see any major changes in boot performance on
> either ast2500 or ast2400.

Thanks for collecting some data. If someone wants to do the same test
on an ast2600 that would be handy.

I came across the commit today too, thanks for posting it to the list Patrick.

We should keep this in mind for future BMCs as they move to arm64 cores.

Cheers,

Joel