Re: [2.6 patch] i386: always use 4k stacks

Re: [2.6 patch] i386: always use 4k stacks

[Stefan Rompf]

Andi Kleen wrote:

> Kernel code is getting more complex all the time and running with
> very tight stack is just risky.

Btw., has anyone yet *measured* maximum stack usage for some weeks on several 
machines, e.g. desktop system with one NIC, reiserfs; server with several 
NICs, stacked device-mapper targets, fiber channel, appletalk...; web server 
with SQL database running on it etc?

Right now I have the impression that the 4k stack flamewars base on make 
checkstack output, waiting for bugreports and other guesswork. Removing the 
safety net on such a basis is just *very bad engineering*.

Stefan

Re: [2.6 patch] i386: always use 4k/4k stacks

[Arjan van de Ven]

> Btw., has anyone yet *measured* maximum stack usage for some weeks on several 
> machines, e.g. desktop system with one NIC, reiserfs; server with several 
> NICs, stacked device-mapper targets, fiber channel, appletalk...; web server 
> with SQL database running on it etc?

partially, see below

> Right now I have the impression that the 4k stack flamewars base on make 
> checkstack output, waiting for bugreports and other guesswork. Removing the 
> safety net on such a basis is just *very bad engineering*.

your impression is wrong.

the kernel has a stack overflow detector, which checks at irq entry time
if the stack is "rather high" (7kb into the stack on a 8kb stack, 3.5kb
on a 4k stack). When this warning hits there's still runway left (like
12.5 percent), but lets say the end becomes in sight. If the stack usage
would be really tight, this "early warning" detector would be hitting a
lot of people, right? Well the good news is that it isn't being hit in
the distributions that use 4Kb stacks (at least the fedora releases and
RHEL, maybe others), with a few exceptions related to XFS use several
months ago (which got fixed since). 

While this isn't a measure of how deep things ACTUALLY go, it's a
measure that they don't go past the 3.5Kb limit, let alone go past 4Kb
limit. 

In addition someone did a chain analysis (which no doubt isn't 100%
complete but still a pretty good effort) and that didn't show major
problems either.

The guesswork in this thread is all from the people on the other side of
the argument, with lots fear and doubt but with no data ;)

(and the "safety net" is a bit of misnomer, since it's not really safe,
just "statistically different" if the shit hits the fan)

Re: [2.6 patch] i386: always use 4k/4k stacks

[Stefan Rompf]

Am Sonntag 18 Dezember 2005 12:21 schrieb Arjan van de Ven:

> the kernel has a stack overflow detector, which checks at irq entry time
> if the stack is "rather high" (7kb into the stack on a 8kb stack, 3.5kb
> on a 4k stack). When this warning hits there's still runway left (like
> 12.5 percent), but lets say the end becomes in sight. If the stack usage
> would be really tight, this "early warning" detector would be hitting a
> lot of people, right?

Wrong. The probability that an interrupt happens just during the codepath with 
highest stack usage is very small. Anyway CONFIG_DEBUG_STACKOVERFLOW is not 
enabled in 2.6.14.4 i386 defconfig. Don't know about vendor kernel kernels 
though.

I thought more about filling the stack with some arbitrary value on thread 
startup and checking how much has been overwritten on a regular basis. Part 
of it is alreay there, hidden unter CONFIG_DEBUG_STACK_USAGE. The 
verification should just happen timer-controlled, not only on sysrq-whatever.

> (and the "safety net" is a bit of misnomer, since it's not really safe,
> just "statistically different" if the shit hits the fan)

If you can't even guarantee that 8k (or 6k) is enough, how can you vote for 4k 
then ;-) Just a little provocation, I don't plan getting too involved into 
this dicussion, hell, this is just about a ridiculously small amount of self 
contained #ifdef'd code ;-)

Stefan

Re: [2.6 patch] i386: always use 4k/4k stacks

[Arjan van de Ven]

On Sun, 2005-12-18 at 13:04 +0100, Stefan Rompf wrote:
> Am Sonntag 18 Dezember 2005 12:21 schrieb Arjan van de Ven:
> 
> > the kernel has a stack overflow detector, which checks at irq entry time
> > if the stack is "rather high" (7kb into the stack on a 8kb stack, 3.5kb
> > on a 4k stack). When this warning hits there's still runway left (like
> > 12.5 percent), but lets say the end becomes in sight. If the stack usage
> > would be really tight, this "early warning" detector would be hitting a
> > lot of people, right?
> 
> Wrong. The probability that an interrupt happens just during the codepath with 
> highest stack usage is very small

so it samples over 1000 times per second, more when busy. Multiplied
over a very large number of users, and 2 years of time. "very small"...
I don't quite agree there.


> Anyway CONFIG_DEBUG_STACKOVERFLOW is not 
> enabled in 2.6.14.4 i386 defconfig. Don't know about vendor kernel kernels 
> though.

the RH/Fedora ones have this enabled

> > (and the "safety net" is a bit of misnomer, since it's not really safe,
> > just "statistically different" if the shit hits the fan)
> 
> If you can't even guarantee that 8k (or 6k) is enough, how can you vote for 4k 
> then ;-) 

it's not 4k it is 4k+4k btw. And my argument is that it's not less
safe.. nor unsafe

Re: [2.6 patch] i386: always use 4k/4k stacks

[Helge Hafting]

Stefan Rompf wrote:

>Wrong. The probability that an interrupt happens just during the codepath with 
>highest stack usage is very small. Anyway CONFIG_DEBUG_STACKOVERFLOW is not 
>enabled in 2.6.14.4 i386 defconfig. Don't know about vendor kernel kernels 
>though.
>  
>
Well, the interrupts have their own stack (if using 4k stacks) so
the interrupt timing shouldn't matter. 

Helge Hafting

Re: [2.6 patch] i386: always use 4k stacks

[Alan Cox]

On Sul, 2005-12-18 at 11:49 +0100, Stefan Rompf wrote:
> Btw., has anyone yet *measured* maximum stack usage for some weeks on several 
> machines, e.g. desktop system with one NIC, reiserfs; server with several 
> NICs, stacked device-mapper targets, fiber channel, appletalk...; web server 
> with SQL database running on it etc?

Some vendors have shipped distributions configured with 4K stacks for a
long time and monitored bug reports.