SCHEDL_SMT forced on!

SCHEDL_SMT forced on!

[Barry Kauler]

Mr Gleixner,
I was upset when I compiled 4.14.87 found that SCHED_SMT had been
forced on. At the time, I just reported it to my blog, and posted a
question about it to a couple of forums, and stayed with an earlier
kernel.

However, when 4.14.88 came out, and still the same situation, alarm
bells went off, and I looked through the kernel changelog. Found it,
4.14.86:

"x86/Kconfig: Select SCHED_SMT if SMP enabled"

Then:

"CONFIG_SCHED_SMT is enabled by all distros, so there is not a real point to
    have it configurable. ..."

...that is a lie. It would be correct to state that is true of the
distros you use, and presumably also for all of you guys who signed
off on it.

Puppy Linux is an example of a distro that has mostly not had
SCHED_SMT enabled. Ditto for most of the forks of Puppy. Two distros
that I currently maintain, Quirky and EasyOS (easyos.org) have SMP
enabled but not SCHED_SMT.

The difference between them is important, they should remain
independently settable. I am so surprised that all of you guys went
along with forcing it on.

For the record, my blog post:

http://bkhome.org/news/201812/kernel-41487-compiled.html

Regards,
Barry Kauler

Re: SCHEDL_SMT forced on!

[Greg KH]

On Sun, Dec 16, 2018 at 02:21:55PM +0800, Barry Kauler wrote:
> Mr Gleixner,
> I was upset when I compiled 4.14.87 found that SCHED_SMT had been
> forced on. At the time, I just reported it to my blog, and posted a
> question about it to a couple of forums, and stayed with an earlier
> kernel.
> 
> However, when 4.14.88 came out, and still the same situation, alarm
> bells went off, and I looked through the kernel changelog. Found it,
> 4.14.86:
> 
> "x86/Kconfig: Select SCHED_SMT if SMP enabled"
> 
> Then:
> 
> "CONFIG_SCHED_SMT is enabled by all distros, so there is not a real point to
>     have it configurable. ..."
> 
> ...that is a lie. It would be correct to state that is true of the
> distros you use, and presumably also for all of you guys who signed
> off on it.
> 
> Puppy Linux is an example of a distro that has mostly not had
> SCHED_SMT enabled. Ditto for most of the forks of Puppy. Two distros
> that I currently maintain, Quirky and EasyOS (easyos.org) have SMP
> enabled but not SCHED_SMT.
> 
> The difference between them is important, they should remain
> independently settable. I am so surprised that all of you guys went
> along with forcing it on.

Is having this option enabled causing a problem for you?  Does it cause
slowdowns or runtime issues?

thanks,

greg k-h

Re: SCHEDL_SMT forced on!

[Andi Kleen]

On Sun, Dec 16, 2018 at 09:27:46AM +0100, Greg KH wrote:
> On Sun, Dec 16, 2018 at 02:21:55PM +0800, Barry Kauler wrote:
> > Mr Gleixner,
> > I was upset when I compiled 4.14.87 found that SCHED_SMT had been
> > forced on. At the time, I just reported it to my blog, and posted a
> > question about it to a couple of forums, and stayed with an earlier
> > kernel.
> > 
> > However, when 4.14.88 came out, and still the same situation, alarm
> > bells went off, and I looked through the kernel changelog. Found it,
> > 4.14.86:
> > 
> > "x86/Kconfig: Select SCHED_SMT if SMP enabled"
> > 
> > Then:
> > 
> > "CONFIG_SCHED_SMT is enabled by all distros, so there is not a real point to
> >     have it configurable. ..."
> > 
> > ...that is a lie. It would be correct to state that is true of the
> > distros you use, and presumably also for all of you guys who signed
> > off on it.
> > 
> > Puppy Linux is an example of a distro that has mostly not had
> > SCHED_SMT enabled. Ditto for most of the forks of Puppy. Two distros
> > that I currently maintain, Quirky and EasyOS (easyos.org) have SMP
> > enabled but not SCHED_SMT.
> > 
> > The difference between them is important, they should remain
> > independently settable. I am so surprised that all of you guys went
> > along with forcing it on.
> 
> Is having this option enabled causing a problem for you?  Does it cause
> slowdowns or runtime issues?

>From the blog:

>>>
Note, way back in the early days when I was first compiling the kernel for multi-core CPUs, SMT (hyperthreading), which simulates more than one core for each actual core, was causing file corruption. So, disabled it, and never trusted it after that.
<<

Since nearly everyone else has the option enabled we 
are quite confident there is no data corruption caused by it.
So there shouldn't be any reason to turn it off.

-Andi

Re: SCHEDL_SMT forced on!

[Barry Kauler]

On 12/17/18, Andi Kleen <ak@linux.intel.com> wrote:
> ...
> From the blog:
>
>>>>
> Note, way back in the early days when I was first compiling the kernel for
> multi-core CPUs, SMT (hyperthreading), which simulates more than one core
> for each actual core, was causing file corruption. So, disabled it, and
> never trusted it after that.
> <<
>
> Since nearly everyone else has the option enabled we
> are quite confident there is no data corruption caused by it.
> So there shouldn't be any reason to turn it off.
>

I am an outsider, looking in, have only followed the RETPOLINE etc.
stuff fairly superficially. Would this statement be correct or
incorrect?:

With CONFIG_SCHED_SMT disabled, CONFIG_RETPOLINE is not needed.
With CONFIG_SCHED_SMT enabled, then CONFIG_RETPOLINE is required for
security reasons.
Enabling CONFIG_SCHED_SMT speeds up some operations, enabling
CONFIG_RETPOLINE slows it down again.
End result, not much speed gain, far more complicated kernel.

Regards,
Barry Kauler

Re: SCHEDL_SMT forced on!

[Jiri Kosina]

On Mon, 17 Dec 2018, Barry Kauler wrote:

> With CONFIG_SCHED_SMT disabled, CONFIG_RETPOLINE is not needed.

This is not true. Retpoline protects also against a scenario where process 
A posions BTB for process B that gets scheduled afterwards on the same 
core, irrespective of SMT.

> With CONFIG_SCHED_SMT enabled, then CONFIG_RETPOLINE is required for
> security reasons.

This is answered by the above as well. Retpoline is needed irrespective of 
SMT.

> Enabling CONFIG_SCHED_SMT speeds up some operations, enabling
> CONFIG_RETPOLINE slows it down again.

There is no 'again', as those two things are not really dependent in any 
way.

> End result, not much speed gain, far more complicated kernel.

And therefore this is bogus :)

-- 
Jiri Kosina
SUSE Labs

Re: SCHEDL_SMT forced on!

[Barry Kauler]

On 12/17/18, Jiri Kosina <jikos@kernel.org> wrote:
> On Mon, 17 Dec 2018, Barry Kauler wrote:
>
>> With CONFIG_SCHED_SMT disabled, CONFIG_RETPOLINE is not needed.
>
> This is not true. Retpoline protects also against a scenario where process
> A posions BTB for process B that gets scheduled afterwards on the same
> core, irrespective of SMT.
>
>> With CONFIG_SCHED_SMT enabled, then CONFIG_RETPOLINE is required for
>> security reasons.
>
> This is answered by the above as well. Retpoline is needed irrespective of
> SMT.
>
>> Enabling CONFIG_SCHED_SMT speeds up some operations, enabling
>> CONFIG_RETPOLINE slows it down again.
>
> There is no 'again', as those two things are not really dependent in any
> way.
>
>> End result, not much speed gain, far more complicated kernel.
>
> And therefore this is bogus :)
>

Thanks for the response.
So I guess it comes down to basic philosophy:

"SMT is a distinct feature, separate from SMP, and I think that it is
important to keep that distinction."

You guys have chosen to remove that distinction, and to remove the
choice for many specialised, such as embedded, situations, where they
would maybe want that choice.

I guess this will have to be my last post on the topic, as I can't
offer any other justification for my stance, other than the above.

Regards,
Barry Kauler

Re: SCHEDL_SMT forced on!

[Jiri Kosina]

On Mon, 17 Dec 2018, Barry Kauler wrote:

> Thanks for the response.
> So I guess it comes down to basic philosophy:
> 
> "SMT is a distinct feature, separate from SMP, and I think that it is
> important to keep that distinction."
> 
> You guys have chosen to remove that distinction, and to remove the
> choice for many specialised, such as embedded, situations, where they
> would maybe want that choice.

They have that choice. SMT can be disabled on kernel cmdline, and in many 
cases also in BIOS.

-- 
Jiri Kosina
SUSE Labs

Re: SCHEDL_SMT forced on!

[Jiri Kosina]

On Sun, 16 Dec 2018, Jiri Kosina wrote:

> They have that choice. SMT can be disabled on kernel cmdline, and in 
> many cases also in BIOS.

And if your worry is about the code size (that the kernel scheduler always 
has the SMT-aware bits in it), I guess making it conditional should be 
backed by real usecase.

The current situation is backed mostly by the fact that there is no known 
usecase that'd want it disabled, everybody we are aware of out there is 
enabling it.

-- 
Jiri Kosina
SUSE Labs

Re: SCHEDL_SMT forced on!

[Barry Kauler]

On 12/17/18, Jiri Kosina <jikos@kernel.org> wrote:
> On Mon, 17 Dec 2018, Barry Kauler wrote:
>
>> Thanks for the response.
>> So I guess it comes down to basic philosophy:
>>
>> "SMT is a distinct feature, separate from SMP, and I think that it is
>> important to keep that distinction."
>>
>> You guys have chosen to remove that distinction, and to remove the
>> choice for many specialised, such as embedded, situations, where they
>> would maybe want that choice.
>
> They have that choice. SMT can be disabled on kernel cmdline, and in many
> cases also in BIOS.
>

Being forced to do it on the kernel commandline is not satisfactory.
If it can be done there, why not also retain it as a configuration
option?

Regards,
Barry Kauler

Re: SCHEDL_SMT forced on!

[Jiri Kosina]

On Mon, 17 Dec 2018, Barry Kauler wrote:

> Being forced to do it on the kernel commandline is not satisfactory. If 
> it can be done there, why not also retain it as a configuration option?

It's not realy the same thing.

SMT can be turned on and off dynamically either during boot, or during 
runtime (by offlining/onlining SMT siblings).

If scheduler is compiled as SMT unaware, then it stays so "forever". So 
you can't populate that kernel to SMT and non-SMT system configurations, 
you can't migrate the running system between SMT and non-SMT systems, etc.

-- 
Jiri Kosina
SUSE Labs

Re: SCHEDL_SMT forced on!

[Barry Kauler]

On 12/17/18, Jiri Kosina <jikos@kernel.org> wrote:
> On Sun, 16 Dec 2018, Jiri Kosina wrote:
>
>> They have that choice. SMT can be disabled on kernel cmdline, and in
>> many cases also in BIOS.
>
> And if your worry is about the code size (that the kernel scheduler always
> has the SMT-aware bits in it), I guess making it conditional should be
> backed by real usecase.
>
> The current situation is backed mostly by the fact that there is no known
> usecase that'd want it disabled, everybody we are aware of out there is
> enabling it.
>

I have already pointed out that some distros are not enabling it.

Your argument is like saying "All of the distros have enabled
CONFIG_SOUND, so let us always force it enabled".

Yet, there are situations where they don't want sound. And perhaps, in
an embedded application, they might want to par the kernel down to the
essentials, for maximum simplicity and security, keeping SMP if the
kernel has multiple cores, but not needing SCHED_SMT. I can see an
engineer wanting that choice.

Regards,
Barry Kauler

Re: SCHEDL_SMT forced on!

[Arjan van de Ven]

On 12/16/2018 2:01 PM, Jiri Kosina wrote:
> On Mon, 17 Dec 2018, Barry Kauler wrote:
> 
>> Thanks for the response.
>> So I guess it comes down to basic philosophy:
>>
>> "SMT is a distinct feature, separate from SMP, and I think that it is
>> important to keep that distinction."
>>
>> You guys have chosen to remove that distinction, and to remove the
>> choice for many specialised, such as embedded, situations, where they
>> would maybe want that choice.
> 
> They have that choice. SMT can be disabled on kernel cmdline, and in many
> cases also in BIOS.
> 

SCHED_SMT does not control hyperthreading... it just controls the OS awareness
of it in the scheduler (e.g. it knows about shared caches).
Even without SCHED_SMT will you see all the logical CPUs... you just get more shitty
scheduling behavior.

Re: SCHEDL_SMT forced on!

[Peter Zijlstra]

On Mon, Dec 17, 2018 at 05:58:11AM +0800, Barry Kauler wrote:
> 
> "SMT is a distinct feature, separate from SMP, and I think that it is
> important to keep that distinction."

Note that building a kernel without CONFIG_SCHED_SMT does _NOT_ disable
SMT. All it accomplishes is that the scheduler will not be SMT aware and
consider each SMT-sibling as a fully independent CPU.