Re: 2.6.6-rc3-mm2 (4KSTACK)

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Ralf Hildebrandt]

* Dominik Karall <dominik.karall@gmx.net>:
> On Wednesday 05 May 2004 10:31, you wrote:
> > +make-4k-stacks-permanent.patch
> >
> >  Fill my inbox.
> 
> Hi Andrew!
> 
> Is there any reason why this patch was applied? Because NVidia users can't 
> work with the original drivers now without removing this patch every time.

Yes, this really sucks (you can choose which "this" I mean)

-- 
Ralf Hildebrandt (Im Auftrag des Referat V a)   Ralf.Hildebrandt@charite.de
Charite - Universitätsmedizin Berlin            Tel.  +49 (0)30-450 570-155
Gemeinsame Einrichtung von FU- und HU-Berlin    Fax.  +49 (0)30-450 570-916
IT-Zentrum Standort Campus Mitte                          AIM.  ralfpostfix

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Dominik Karall]

On Wednesday 05 May 2004 10:31, you wrote:
> +make-4k-stacks-permanent.patch
>
>  Fill my inbox.

Hi Andrew!

Is there any reason why this patch was applied? Because NVidia users can't 
work with the original drivers now without removing this patch every time.

greets Dominik

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 868 bytes --]

On Wed, 2004-05-05 13:12:30 +0200, Dominik Karall <dominik.karall@gmx.net>
wrote in message <200405051312.30626.dominik.karall@gmx.net>:
> On Wednesday 05 May 2004 10:31, you wrote:
> > +make-4k-stacks-permanent.patch
> >
> >  Fill my inbox.
> 
> Is there any reason why this patch was applied? Because NVidia users can't 
> work with the original drivers now without removing this patch every time.

Wrong place to ask:)  Go, ask NVidia for a solution. Their driver would
long be working with 4K stacks if they had open-sourced it...

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
   ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 367 bytes --]

On Wed, 2004-05-05 at 13:12, Dominik Karall wrote:
> On Wednesday 05 May 2004 10:31, you wrote:
> > +make-4k-stacks-permanent.patch
> >
> >  Fill my inbox.
> 
> Hi Andrew!
> 
> Is there any reason why this patch was applied? Because NVidia users 
 
nvidia users can use the nv driver instead? And/Or ask the nvidia binary
only people for a fixed driver ?

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Andrew Morton]

Dominik Karall <dominik.karall@gmx.net> wrote:
>
> On Wednesday 05 May 2004 10:31, you wrote:
> > +make-4k-stacks-permanent.patch
> >
> >  Fill my inbox.
> 
> Hi Andrew!
> 
> Is there any reason why this patch was applied? Because NVidia users can't 
> work with the original drivers now without removing this patch every time.
> 

We need to push this issue along quickly.  The single-page stack generally
gives us a better kernel and having the stack size configurable creates
pain.

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Rene Herman]

[-- Attachment #1: Type: text/plain, Size: 265 bytes --]

Andrew Morton wrote:

> We need to push this issue along quickly.  The single-page stack generally
> gives us a better kernel and having the stack size configurable creates
> pain.

Hi. No idea if you want this. Not seeing 4KSTACKS in there made me recheck.

Rene.

[-- Attachment #2: linux-2.6.6-rc3-mm2_vermagic-4kstacks.diff --]
[-- Type: text/plain, Size: 456 bytes --]

# restore 4KSTACKS to VERMAGIC

--- linux-2.6.6-rc3-mm2/include/asm-i386/module.h.orig	2004-05-05 13:51:29.000000000 +0200
+++ linux-2.6.6-rc3-mm2/include/asm-i386/module.h	2004-05-05 13:52:26.000000000 +0200
@@ -60,11 +60,7 @@
 #define MODULE_REGPARM ""
 #endif
 
-#ifdef CONFIG_4KSTACKS
 #define MODULE_STACKSIZE "4KSTACKS "
-#else
-#define MODULE_STACKSIZE ""
-#endif
 
 #define MODULE_ARCH_VERMAGIC MODULE_PROC_FAMILY MODULE_REGPARM MODULE_STACKSIZE
 

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Steve Lord]

Andrew Morton wrote:
> Dominik Karall <dominik.karall@gmx.net> wrote:
> 
>>On Wednesday 05 May 2004 10:31, you wrote:
>>
>>>+make-4k-stacks-permanent.patch
>>>
>>> Fill my inbox.
>>
>>Hi Andrew!
>>
>>Is there any reason why this patch was applied? Because NVidia users can't 
>>work with the original drivers now without removing this patch every time.
>>
> 
> 
> We need to push this issue along quickly.  The single-page stack generally
> gives us a better kernel and having the stack size configurable creates
> pain.

Is it less pain than making something like a memory allocation which comes
out of a deep stack? Say, nfs server -> filesystem -> lvm/raid -> fiber channel,
which itself does something like a writepage into an nfs filesystem and ends
up in the networking stack? OK, getting back into the filesystem on a
memory allocation from the block layer should not happen, but you could
certainly be down in the bowels of the first filesystem when this happens.

There are other combinations which worry me. I do wonder how close to the
edge some of these are living now and cutting them off at the knees, stack
wise, is going to bite later. How many folks run the mm kernel in production
server environments?

Maybe there should be a competition to see how convoluted a stack you can
generate out of the kernel ;-)

Steve

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 710 bytes --]

On Wed, 05 May 2004 13:12:30 +0200, Dominik Karall <dominik.karall@gmx.net>  said:

> Is there any reason why this patch was applied? Because NVidia users can't 
> work with the original drivers now without removing this patch every time.

The NVidia users will also have to back out the regparm patch, or insert
'asmlinkage' on all the appropriate definitions in the glue code.  Note that
the patches on www.minion.de already fix this for the 5341 drivers as of
03/24/2004.

In any case, even though I'm one of those NVidia users, I'm OK with the
patch being in there - anybody who's clued enough to build and run a -mm
kernel shouldn't have much trouble figuring out how to build it with 2 patches
reverted.

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Felipe Alfaro Solana]

On Wed, 2004-05-05 at 18:47, Steve Lord wrote:

> There are other combinations which worry me. I do wonder how close to the
> edge some of these are living now and cutting them off at the knees, stack
> wise, is going to bite later. How many folks run the mm kernel in production
> server environments?

Me... Although the servers aren't critical. However, I know I'm a
minority...

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 1104 bytes --]


> Is it less pain than making something like a memory allocation which comes
> out of a deep stack? Say, nfs server -> filesystem -> lvm/raid -> fiber channel,
> which itself does something like a writepage into an nfs filesystem and ends
> up in the networking stack? OK, getting back into the filesystem on a
> memory allocation from the block layer should not happen, but you could
> certainly be down in

it's not really much different than the 2.4 kernel already has.
In 2.4 you have a 8Kb stack of which

First 1600 bytes (+/-) are for the task struct
about 4Kb of user context stack
>= 2Kb stack which is needed for irq context (both soft and hard)

In 2.6 with this patch you have

32 bytes for thread info
4Kb minus 32 bytes for context stack
SEPARATE softirq stack of 4K
SEPARATE hardirq stack of 4K

so in a way you have MORE stack space than in 2.4.

Now I'll fully admit the 2Kb is somewhat of a stochast, you only hit it
if you have iptable rules and 2 nic irq's arriving on the same cpu at an
unfortionate moment, but that doesn't mean it's a safe situation.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Steve Lord]

Arjan van de Ven wrote:
>>Is it less pain than making something like a memory allocation which comes
>>out of a deep stack? Say, nfs server -> filesystem -> lvm/raid -> fiber channel,
>>which itself does something like a writepage into an nfs filesystem and ends
>>up in the networking stack? OK, getting back into the filesystem on a
>>memory allocation from the block layer should not happen, but you could
>>certainly be down in
> 
> 
> it's not really much different than the 2.4 kernel already has.
> In 2.4 you have a 8Kb stack of which
> 
> First 1600 bytes (+/-) are for the task struct
> about 4Kb of user context stack
> 
>>= 2Kb stack which is needed for irq context (both soft and hard)
> 
> 
> In 2.6 with this patch you have
> 
> 32 bytes for thread info
> 4Kb minus 32 bytes for context stack
> SEPARATE softirq stack of 4K
> SEPARATE hardirq stack of 4K
> 
> so in a way you have MORE stack space than in 2.4.
> 
> Now I'll fully admit the 2Kb is somewhat of a stochast, you only hit it
> if you have iptable rules and 2 nic irq's arriving on the same cpu at an
> unfortionate moment, but that doesn't mean it's a safe situation.
> 

I agree with that, but that combination had a heck of a lot of soak time
on it. This stack crunching exercise is happening fairly early on in a
stable kernel series, that makes me nervous. A lot of new code showed up
in the meantime during 2.5 development which changed many of these
code paths.

You can do static code analysis and say which functions are stack hogs and
rework them, but the dynamic analysis is really hard to do for all possibly
combinations. Doing the dynamic analysis on someones production fileserver may
not be the wisest move ever made.

Steve

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 252 bytes --]


> 
> Now I'll fully admit the 2Kb is somewhat of a stochast, you only hit it
> if you have iptable rules and 2 nic irq's arriving on the same cpu at an
> unfortionate moment, but that doesn't mean it's a safe situation.

s/safe/safer than 2.6/

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Bill Davidsen]

Andrew Morton wrote:
> Dominik Karall <dominik.karall@gmx.net> wrote:
> 
>>On Wednesday 05 May 2004 10:31, you wrote:
>>
>>>+make-4k-stacks-permanent.patch
>>>
>>> Fill my inbox.
>>
>>Hi Andrew!
>>
>>Is there any reason why this patch was applied? Because NVidia users can't 
>>work with the original drivers now without removing this patch every time.
>>
> 
> 
> We need to push this issue along quickly.  The single-page stack generally
> gives us a better kernel and having the stack size configurable creates
> pain.

Add my voice to those who don't think 4k stacks are a good idea as a 
default, they break some things and seem to leave other paths (as others 
have noted) on the edge. I'm not sure what you have in mind as a "better 
kernel" but I'd rather have a worse kernel and not have to check 4k 
stack as a possible problem before looking at other things if I get bad 
behaviour.

Reliability first, performance later. We've lived with the config for a 
while, pain there is better than pain at runtime.

-- 
    -bill davidsen (davidsen@tmr.com)
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Jörn Engel]

On Wed, 5 May 2004 14:22:02 -0400, Valdis.Kletnieks@vt.edu wrote:
> On Wed, 05 May 2004 13:12:30 +0200, Dominik Karall <dominik.karall@gmx.net>  said:
> 
> > Is there any reason why this patch was applied? Because NVidia users can't 
> > work with the original drivers now without removing this patch every time.
> 
> The NVidia users will also have to back out the regparm patch, or insert
> 'asmlinkage' on all the appropriate definitions in the glue code.  Note that
> the patches on www.minion.de already fix this for the 5341 drivers as of
> 03/24/2004.
> 
> In any case, even though I'm one of those NVidia users, I'm OK with the
> patch being in there - anybody who's clued enough to build and run a -mm
> kernel shouldn't have much trouble figuring out how to build it with 2 patches
> reverted.

I disagree.  -mm is the testing ground for -linus.  If this patch
would only break the nvidia module, I couldn't care less.  But
breaking it in such a way that random kernel memory gets corrupted,
which can soon backfire to the filesystem as well?  That's not what
most people understand when calling something "stable".

If this goes in, we should at least detect the nvidia module and
refuse to load it.  Maybe do the same for other modules as well or
refuse to load any module that doesn't somehow state to be ok with 4k
stacks.  Details don't matter, as long as the filesystems survive.

Jörn

-- 
Das Aufregende am Schreiben ist es, eine Ordnung zu schaffen, wo
vorher keine existiert hat.
-- Doris Lessing

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Bartlomiej Zolnierkiewicz]

On Wednesday 05 of May 2004 22:31, Bill Davidsen wrote:
> Andrew Morton wrote:
> > Dominik Karall <dominik.karall@gmx.net> wrote:
> >>On Wednesday 05 May 2004 10:31, you wrote:
> >>>+make-4k-stacks-permanent.patch
> >>>
> >>> Fill my inbox.
> >>
> >>Hi Andrew!
> >>
> >>Is there any reason why this patch was applied? Because NVidia users
> >> can't work with the original drivers now without removing this patch
> >> every time.
> >
> > We need to push this issue along quickly.  The single-page stack
> > generally gives us a better kernel and having the stack size configurable
> > creates pain.
>
> Add my voice to those who don't think 4k stacks are a good idea as a
> default, they break some things and seem to leave other paths (as others
> have noted) on the edge. I'm not sure what you have in mind as a "better
> kernel" but I'd rather have a worse kernel and not have to check 4k
> stack as a possible problem before looking at other things if I get bad
> behaviour.
>
> Reliability first, performance later. We've lived with the config for a
> while, pain there is better than pain at runtime.

Opposite opinion here.

If you want 100% reliability you shouldn't use -mm in the first place.

Making 4kb stacks default in -mm is very good idea so it will get necessary
testing and fixing before being integrated into mainline.

Please also note that users of binary only modules always have choice:
- new kernels without binary only modules
- old kernels with binary only modules

It is really that simple.

Regards,
Bartlomiej

Re: 2.6.6-rc3-mm2 (4KSTACK)

[h.verhagen]

No wonder the module doesn't work with 4K stacks.
It wonders that it actually  works with 8K stacks :)
The nividia module seems extremely stack hungry. 

[harm@node-d-8d2e c]$ objdump -d /lib/modules/2.4.26/kernel/drivers/video/nvidia.o | ./checkstack.pl
0xb6ff3 _nv002427rm: sub $0x908,%esp          (almost 4K !)
0x7ff93 _nv003775rm: sub $0x64c,%esp          ( ~1-2K)
0x21fd3 _nv000899rm: sub $0x648,%esp
f53f: 81 ec 94 05 00 00 sub $0x594,%esp
_nv003402rm: sub $0x594,%esp
0x10247 _nv003354rm: sub $0x520,%esp
0x42633 _nv003333rm: sub $0x4a8,%esp
0x100bb _nv003353rm: sub $0x490,%esp
0x842ff _nv004811rm: sub $0x41c,%esp          (1K from here)


Regards,
Harm

Re: 2.6.6-rc3-mm2 (4KSTACK)

[h.verhagen]

No wonder the nvidia binary module doesn't work with 4K stacks. I'm wondered that it even works with 8KB stacks.
The module seems extremely stack hungry.

[harm@node-d-8d2e c]$ objdump -d /lib/modules/2.4.26/kernel/drivers/video/nvidia.o | ./checkstack.pl
0xb6ff3 _nv002427rm: sub $0x908,%esp             (almost 4K !)
0x7ff93 _nv003775rm: sub $0x64c,%esp             ( ~1-2K)
0x21fd3 _nv000899rm: sub $0x648,%esp
f53f: 81 ec 94 05 00 00 sub $0x594,%esp
_nv003402rm: sub $0x594,%esp
0x10247 _nv003354rm: sub $0x520,%esp
0x42633 _nv003333rm: sub $0x4a8,%esp
0x100bb _nv003353rm: sub $0x490,%esp
0x842ff _nv004811rm: sub $0x41c,%esp          (1K from here)


Kind regards,
Harm

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Sid Boyce]

Not only does it not work, I've cratered two reiserfs installs using 
4KSTACKS and the nvidia driver, once when I didn't know about the 
problem and once when I forgot to reverse the patch. I now always check 
my .config before I start the build. Nvidia like most outfits don't 
react that swiftly to kernel changes.
Regards
Sid.

-- 
Sid Boyce .... Hamradio G3VBV and keen Flyer
Linux Only Shop.

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Helge Hafting]

Bill Davidsen wrote:

> Andrew Morton wrote:
>
>>
>> We need to push this issue along quickly.  The single-page stack 
>> generally
>> gives us a better kernel and having the stack size configurable creates
>> pain.
>
>
> Add my voice to those who don't think 4k stacks are a good idea as a 
> default, they break some things and seem to leave other paths (as 
> others have noted) on the edge. I'm not sure what you have in mind as 
> a "better kernel" but I'd rather have a worse kernel and not have to 
> check 4k stack as a possible problem before looking at other things if 
> I get bad behaviour. 

I think 4k stacks is perfectly ok for mm, as mm is an experimental
testing ground anyway.  Not everything in mm goes into the next 2.6.x.


Wether 4k goes into some 2.6 release or waits for 2.7 is another debate.

Helge Hafting

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Bill Davidsen]

On Thu, 6 May 2004, Helge Hafting wrote:

> Bill Davidsen wrote:
> 
> > Andrew Morton wrote:
> >
> >>
> >> We need to push this issue along quickly.  The single-page stack 
> >> generally
> >> gives us a better kernel and having the stack size configurable creates
> >> pain.
> >
> >
> > Add my voice to those who don't think 4k stacks are a good idea as a 
> > default, they break some things and seem to leave other paths (as 
> > others have noted) on the edge. I'm not sure what you have in mind as 
> > a "better kernel" but I'd rather have a worse kernel and not have to 
> > check 4k stack as a possible problem before looking at other things if 
> > I get bad behaviour. 
> 
> I think 4k stacks is perfectly ok for mm, as mm is an experimental
> testing ground anyway.  Not everything in mm goes into the next 2.6.x.
> 
> 
> Wether 4k goes into some 2.6 release or waits for 2.7 is another debate.

I think it's fine as an option, but taking it out of config and making it
an immutable part of the kernel is probably undesirable. It appears to be
a small gain (nothing I can easily measure), and a larger risk. I'd like
that to be an optional risk, like many other things in the kernel,
available to be used if the last drop of size or performance is desired.

Does someone has any numbers showing what this gains? I didn't see
anything obvious when I tried it, so it's either very small or only in
some case(s) I didn't try.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Norberto Bensa]

Bartlomiej Zolnierkiewicz wrote:
> Making 4kb stacks default in -mm is very good idea so it will get necessary
> testing and fixing before being integrated into mainline.

Then let us test with _AND_ without 4KSTACKS.

I love the -mm three, but I use a nvidia GFX card too. Making 4KSTACkS 
permanent excludes me from more testing.

Best regards,
Norberto

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Bartlomiej Zolnierkiewicz]

On Thursday 06 of May 2004 14:55, Norberto Bensa wrote:
> Bartlomiej Zolnierkiewicz wrote:
> > Making 4kb stacks default in -mm is very good idea so it will get
> > necessary testing and fixing before being integrated into mainline.
>
> Then let us test with _AND_ without 4KSTACKS.

You are free to remove this patch from your kernel. 8)

> I love the -mm three, but I use a nvidia GFX card too. Making 4KSTACkS
> permanent excludes me from more testing.

We currently need testing for 4KSTACKS.  Please also note that your testing
with binary modules loaded has limited value as you should always reproduce
problem w/o binary modules before reporting problem.

Regards,
Bartlomiej

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1101 bytes --]

On Wed, 05 May 2004 23:51:36 +0200, =?iso-8859-1?Q?J=F6rn?= Engel said:

> I disagree.  -mm is the testing ground for -linus.  If this patch
> would only break the nvidia module, I couldn't care less.

OK.. I need to clarify - I'm OK on the patch being *in -mm* precisely *because*
it's a testing ground.  Anybody who's running a -mm kernel should have the
technical savvy to deal with the issue by reverting the one patch in question,
and to recover if it eats their file system (Yes, I'm running 2.6.6-rc3-mm2 and
the NVidia driver as I type.  No, making it work wasn't a problem.  Yes, I spin
everything needed to rebuild out to CD/RW at least once a week, just because it
*is* a -mm kernel. ;)

It's a Good Idea to do this in -mm, to flush out all the binary modules that
are known to have issues with this (have we identified anybody other than NVidia
that actually *has* a problem)?

It's probably a Bad Idea to push this to Linus before the vendors that have
significant market-impact issues (again - anybody other than NVidia here?)
have gotten their stuff cleaned up...


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 336 bytes --]


> It's probably a Bad Idea to push this to Linus before the vendors that have
> significant market-impact issues (again - anybody other than NVidia here?)
> have gotten their stuff cleaned up...

Ok I don't want to start a flamewar but... Do we want to hold linux back
until all binary only module vendors have caught up ??



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Malte Schröder]

[-- Attachment #1: Type: text/plain, Size: 1529 bytes --]

On Thu, 06 May 2004 11:18:10 -0400
Valdis.Kletnieks@vt.edu wrote:

> On Wed, 05 May 2004 23:51:36 +0200, =?iso-8859-1?Q?J=F6rn?= Engel said:
> 
> > I disagree.  -mm is the testing ground for -linus.  If this patch
> > would only break the nvidia module, I couldn't care less.
> 
> OK.. I need to clarify - I'm OK on the patch being *in -mm* precisely *because*
> it's a testing ground.  Anybody who's running a -mm kernel should have the
> technical savvy to deal with the issue by reverting the one patch in question,
> and to recover if it eats their file system (Yes, I'm running 2.6.6-rc3-mm2 and
> the NVidia driver as I type.  No, making it work wasn't a problem.  Yes, I spin
> everything needed to rebuild out to CD/RW at least once a week, just because it
> *is* a -mm kernel. ;)
> 
> It's a Good Idea to do this in -mm, to flush out all the binary modules that
> are known to have issues with this (have we identified anybody other than NVidia
> that actually *has* a problem)?
I use 2.6.6-rc3  w/ 4k-stack enabled (-mm is a bit too experimental for my taste ;) ), ATIs binary-module is working w/o problems.
but IIRC I had to disable REGPARMS.

> 
> It's probably a Bad Idea to push this to Linus before the vendors that have
> significant market-impact issues (again - anybody other than NVidia here?)
> have gotten their stuff cleaned up...
> 
> 


-- 
---------------------------------------
Malte Schröder
MalteSch@gmx.de
ICQ# 68121508
---------------------------------------


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 598 bytes --]

On Thu, 06 May 2004 18:03:14 +0200, Malte =?ISO-8859-1?B?U2NocvZkZXI=?= said:

> I use 2.6.6-rc3  w/ 4k-stack enabled (-mm is a bit too experimental for my =
> taste ;) ), ATIs binary-module is working w/o problems.
> but IIRC I had to disable REGPARMS.

Alternatively, www.minion.de has a patch against the 5341 drivers that makes
it work with the regparms (basically, it sticks an asmlinkage in all the right
places)....

http://www.minion.de/files/NVIDIA_kernel-1.0-5341-2.6.diff

Of course, if you hit problems running a 3rd party patch against a mostly-binary
driver, you're on your own... ;)

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 631 bytes --]

On Thu, 06 May 2004 17:40:33 +0200, Arjan van de Ven said:
> Ok I don't want to start a flamewar but... Do we want to hold linux back
> until all binary only module vendors have caught up ??

No.. I merely suggested that coordinating with as few as possibly one vendor to
clean their module up might minimize the pain considerably.  NVidia is aware of
the issue, and rumor has it that the 6xxx series of Linux drivers are targeted
for the end of May and will have a fix for the 4K stack (which is an issue for
the Fedora Core 2 release already) since they need to push out a revision to
support the 6800 series cards anyhow....




[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Matt Mackall]

On Thu, May 06, 2004 at 11:18:10AM -0400, Valdis.Kletnieks@vt.edu wrote:
> It's a Good Idea to do this in -mm, to flush out all the binary
> modules that are known to have issues with this (have we identified
> anybody other than NVidia that actually *has* a problem)?

Anything that uses current() or the like will be unhappy, though it
may work half the time. So just about anything binary-only is liable
to hit it. We can catch most of this by adding "4KSTACKS" to the
global build flags, but I think stuff like the Nvidia wrapper layer
shoots itself in the foot here by silently ignoring this check.

-- 
Matt Mackall : http://www.selenic.com : Linux development and consulting

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Max Valdez]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I do

and  Ihave nvidia video card :-), last couple of week been running 2.6.5 
because of the problem with the module nvidia.o and teh 4STACK thing

But as soon as I get to compile the new mm kernel, and reboot, will continue 
to work on a mm like the last months.

Actually I've been using mm kernels most of the time since AC patches stoped 
when Alan left to his Ph.D., or what ever he's doing right now.

Max
- -- 
Linux garaged 2.6.5-rc2-mm3 #1 Fri Mar 26 11:07:16 CST 2004 i686 Intel(R) 
Pentium(R) 4 CPU 2.80GHz GenuineIntel GNU/Linux
- -----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GS/S d- s: a-29 C++(+++) ULAHI+++ P+ L++>+++ E--- W++ N* o-- K- w++++ O- M-- 
V-- PS+ PE Y-- PGP++ t- 5- X+ R tv++ b+ DI+++ D- G++ e++ h+ r+ z**
- ------END GEEK CODE BLOCK------
gpg-key: http://garaged.homeip.net/gpg-key.txt
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFAmnmNNNkpVEFxW78RAm+5AJ0c5/U5nmrzFFh9aS0p3iGBcwZVlgCdGMx9
1CwtWd5W01omEDfWO6iHpVk=
=KM7C
-----END PGP SIGNATURE-----

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Norberto Bensa]

Bartlomiej Zolnierkiewicz wrote:
> > Then let us test with _AND_ without 4KSTACKS.
>
> You are free to remove this patch from your kernel. 8)

I already do ;-)

That was not my point.

Thanks anyway,
Norberto

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Paul Jakma]

On Thu, 6 May 2004, Arjan van de Ven wrote:

> Ok I don't want to start a flamewar but... Do we want to hold linux
> back until all binary only module vendors have caught up ??

What about normal linux modules though? Eg, NFS (most likely):

	https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=121804

regards,
-- 
Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
	warning: do not ever send email to spam@dishone.st
Fortune:
Disraeli was pretty close: actually, there are Lies, Damn lies, Statistics,
Benchmarks, and Delivery dates.

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Andrew Morton]

Paul Jakma <paul@clubi.ie> wrote:
>
> On Thu, 6 May 2004, Arjan van de Ven wrote:
> 
>  > Ok I don't want to start a flamewar but... Do we want to hold linux
>  > back until all binary only module vendors have caught up ??
> 
>  What about normal linux modules though? Eg, NFS (most likely):
> 
>  	https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=121804

That's a misdiagnosis.  The problem here is that the kernel is taking a
pagefault within show_trace(), and the pagefault handler calls
show_trace().  It has gone infinitely recursive.

The bug is unrelated to the stack size.  It is in show_trace() or
thereabouts.  That code tries to protect itself from recursive faults, but
it's a vendor kernel and may be different from the public tree.

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Paul Jakma]

On Thu, 6 May 2004, Andrew Morton wrote:

> That's a misdiagnosis.  The problem here is that the kernel is
> taking a pagefault within show_trace(), and the pagefault handler
> calls show_trace().  It has gone infinitely recursive.

That happens after the initial stack overflow with a trace (from what
i could discern before it scrolled into oblivion) in NFS -> IP path
similar to the other non-recursive trace, see below.

> The bug is unrelated to the stack size.  It is in show_trace() or
> thereabouts.  That code tries to protect itself from recursive
> faults, but it's a vendor kernel and may be different from the
> public tree.

Fair enough but have a look at the other fault from that bug though:

	https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=99855&action=view 

That one did not recurse for some reason.

regards,
-- 
Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
	warning: do not ever send email to spam@dishone.st
Fortune:
The program isn't debugged until the last user is dead.

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Andrew Morton]

Paul Jakma <paul@clubi.ie> wrote:
>
> Fair enough but have a look at the other fault from that bug though:
> 
>  	https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=99855&action=view 
> 
>  That one did not recurse for some reason.

OK.

So we're 50 to 60 levels deep in function calls there and simply ran out
of 4k stack.

Based on this and upon the few other feedbackings I've had on this issue it
seems that the 4k stack experiment will come back saying "no".

Thanks.

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 452 bytes --]


On Fri, May 07, 2004 at 01:37:54AM +0100, Paul Jakma wrote:
> On Thu, 6 May 2004, Arjan van de Ven wrote:
> 
> > Ok I don't want to start a flamewar but... Do we want to hold linux
> > back until all binary only module vendors have caught up ??
> 
> What about normal linux modules though? Eg, NFS (most likely):
> 
> 	https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=121804

NFSv4 has a > 1Kb stack user; Dave Jones has a fix pending for that...

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 475 bytes --]


On Thu, May 06, 2004 at 08:58:38PM -0700, Andrew Morton wrote:
> Paul Jakma <paul@clubi.ie> wrote:
> >
> > Fair enough but have a look at the other fault from that bug though:
> > 
> >  	https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=99855&action=view 
> > 
> >  That one did not recurse for some reason.
> 
> OK.
> 
> So we're 50 to 60 levels deep in function calls there and simply ran out
> of 4k stack.


no that call trace is AFTER you overflow, eg wrong stack

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Helge Hafting]

Valdis.Kletnieks@vt.edu wrote:

>On Thu, 06 May 2004 17:40:33 +0200, Arjan van de Ven said:
>  
>
>>Ok I don't want to start a flamewar but... Do we want to hold linux back
>>until all binary only module vendors have caught up ??
>>    
>>
>
>No.. I merely suggested that coordinating with as few as possibly one vendor to
>clean their module up might minimize the pain considerably.  
>

I don't see much of a problem.  So what if Linus puts 4k stacks in 2.6.6 
tomorrow?
It won't kill linux for all those nvidia users.  They'll simply have to 
stop at 2.6.5
until nvidia catch up.  Not much of a problem, considering how the majority
still runs various versions of 2.4.x.

Helge Hafting

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Dave Jones]

On Fri, May 07, 2004 at 08:51:05AM +0200, Arjan van de Ven wrote:
 > 
 > On Fri, May 07, 2004 at 01:37:54AM +0100, Paul Jakma wrote:
 > > On Thu, 6 May 2004, Arjan van de Ven wrote:
 > > 
 > > > Ok I don't want to start a flamewar but... Do we want to hold linux
 > > > back until all binary only module vendors have caught up ??
 > > 
 > > What about normal linux modules though? Eg, NFS (most likely):
 > > 
 > > 	https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=121804
 > 
 > NFSv4 has a > 1Kb stack user; Dave Jones has a fix pending for that...

Hmm, this one maybe?
 
		Dave

--- linux-2.6.5/net/sunrpc/auth_gss/auth_gss.c~	2004-05-05 13:34:31.000000000 +0100
+++ linux-2.6.5/net/sunrpc/auth_gss/auth_gss.c	2004-05-05 13:33:05.000000000 +0100
@@ -429,10 +429,8 @@ gss_pipe_upcall(struct file *filp, struc
 static ssize_t
 gss_pipe_downcall(struct file *filp, const char *src, size_t mlen)
 {
-	char buf[1024];
 	struct xdr_netobj obj = {
 		.len	= mlen,
-		.data	= buf,
 	};
 	struct inode *inode = filp->f_dentry->d_inode;
 	struct rpc_inode *rpci = RPC_I(inode);
@@ -448,11 +446,19 @@ gss_pipe_downcall(struct file *filp, con
 	int err;
 	int gss_err;
 
-	if (mlen > sizeof(buf))
+	obj.data = kmalloc(1024, GFP_KERNEL);
+	if (!obj.data)
+		return -ENOMEM;
+
+	if (mlen > 1024) {
+		kfree (obj.data);
 		return -ENOSPC;
-	left = copy_from_user(buf, src, mlen);
-	if (left)
+	}
+	left = copy_from_user(obj.data, src, mlen);
+	if (left) {
+		kfree (obj.data);
 		return -EFAULT;
+	}
 	clnt = rpci->private;
 	atomic_inc(&clnt->cl_users);
 	auth = clnt->cl_auth;
@@ -477,12 +483,14 @@ gss_pipe_downcall(struct file *filp, con
 	} else
 		spin_unlock(&gss_auth->lock);
 	rpc_release_client(clnt);
+	kfree (obj.data);
 	return mlen;
 err:
 	if (ctx)
 		gss_destroy_ctx(ctx);
 	rpc_release_client(clnt);
 	dprintk("RPC: gss_pipe_downcall returning %d\n", err);
+	kfree (obj.data);
 	return err;
 }
 

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Martin J. Bligh]

--Andrew Morton <akpm@osdl.org> wrote (on Thursday, May 06, 2004 20:58:38 -0700):

> Paul Jakma <paul@clubi.ie> wrote:
>> 
>> Fair enough but have a look at the other fault from that bug though:
>> 
>>  	https://bugzilla.redhat.com/bugzilla/attachment.cgi?id=99855&action=view 
>> 
>>  That one did not recurse for some reason.
> 
> OK.
> 
> So we're 50 to 60 levels deep in function calls there and simply ran out
> of 4k stack.
> 
> Based on this and upon the few other feedbackings I've had on this issue it
> seems that the 4k stack experiment will come back saying "no"

There's two problems with that stack .... 

1. it seems to have the IRQ on it as well as normal traffic instead of 
using the separate irqstacks ... why isn't that working?

2 nfs_writepage_sync is a known stack-abuser ;-) 1632 bytes on PPC64 at least
(from Anton's data). Maybe it's that struct nfs_write_data ?

M.

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Steve Lord]

Dave Jones wrote:
> On Fri, May 07, 2004 at 08:51:05AM +0200, Arjan van de Ven wrote:
>  > 
>  > On Fri, May 07, 2004 at 01:37:54AM +0100, Paul Jakma wrote:
>  > > On Thu, 6 May 2004, Arjan van de Ven wrote:
>  > > 
>  > > > Ok I don't want to start a flamewar but... Do we want to hold linux
>  > > > back until all binary only module vendors have caught up ??
>  > > 
>  > > What about normal linux modules though? Eg, NFS (most likely):
>  > > 
>  > > 	https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=121804
>  > 
>  > NFSv4 has a > 1Kb stack user; Dave Jones has a fix pending for that...
> 
> Hmm, this one maybe?
>  
> 		Dave
> 

>  
> -	if (mlen > sizeof(buf))
> +	obj.data = kmalloc(1024, GFP_KERNEL);
> +	if (!obj.data)
> +		return -ENOMEM;
> +
> +	if (mlen > 1024) {

That's what I hate about all of this, just think how much stack that
kmalloc can take in low memory situations.... it might end up in
writepage on another nfs file.... Moving stuff off the stack and
into kmalloc just reduces the possibility of stack overflow, it
does not fix the problem. Having memory reclaim take place inside
the thread which is waiting for memory makes that a pretty hard
problem to fix.

Steve

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 413 bytes --]

On Fri, May 07, 2004 at 10:47:56AM -0500, Steve Lord wrote:
> >-	if (mlen > sizeof(buf))
> >+	obj.data = kmalloc(1024, GFP_KERNEL);
> >+	if (!obj.data)
> >+		return -ENOMEM;
> >+
> >+	if (mlen > 1024) {
> 
> That's what I hate about all of this, just think how much stack that
> kmalloc can take in low memory situations.... it might end up in
> writepage on another nfs file....

it clearly needs to be GFP_NOFS

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: 2.6.6-rc3-mm2 (4KSTACK)

[J. Bruce Fields]

On Fri, May 07, 2004 at 05:59:43PM +0200, Arjan van de Ven wrote:
> On Fri, May 07, 2004 at 10:47:56AM -0500, Steve Lord wrote:
> > >-	if (mlen > sizeof(buf))
> > >+	obj.data = kmalloc(1024, GFP_KERNEL);
> > >+	if (!obj.data)
> > >+		return -ENOMEM;
> > >+
> > >+	if (mlen > 1024) {
> > 
> > That's what I hate about all of this, just think how much stack that
> > kmalloc can take in low memory situations.... it might end up in
> > writepage on another nfs file....
> 
> it clearly needs to be GFP_NOFS

The function is question is essentially a write method for a virtual
filesystem (rpc_pipefs) that's used to communicate with some
NFSv4-related daemons.  It isn't called from any of the NFS fileystem
code.

--Bruce Fields

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Steve Lord]

Arjan van de Ven wrote:
> On Fri, May 07, 2004 at 10:47:56AM -0500, Steve Lord wrote:
> 
>>>-	if (mlen > sizeof(buf))
>>>+	obj.data = kmalloc(1024, GFP_KERNEL);
>>>+	if (!obj.data)
>>>+		return -ENOMEM;
>>>+
>>>+	if (mlen > 1024) {
>>
>>That's what I hate about all of this, just think how much stack that
>>kmalloc can take in low memory situations.... it might end up in
>>writepage on another nfs file....
> 
> 
> it clearly needs to be GFP_NOFS

That was not really my point, consider any memory allocation on the
stack which is being replaced with an allocate to save space. Then replace
the saved stack space with the potential stack space used to
free memory by writing it out via a filesystem. You cannot make all
the allocations in the kernel GFP_NOFS.

Now at least if the memory is allocated high enough up in the
call chain it fixes the problems of a function with a large
stack frame with a deep stack underneath it. It does not fix
anything if the function is already deep in the stack.

All this is doing is papering over the cracks.

Steve

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Jörn Engel]

On Fri, 7 May 2004 11:11:30 -0500, Steve Lord wrote:
> 
> That was not really my point, consider any memory allocation on the
> stack which is being replaced with an allocate to save space. Then replace
> the saved stack space with the potential stack space used to
> free memory by writing it out via a filesystem. You cannot make all
> the allocations in the kernel GFP_NOFS.
> 
> Now at least if the memory is allocated high enough up in the
> call chain it fixes the problems of a function with a large
> stack frame with a deep stack underneath it. It does not fix
> anything if the function is already deep in the stack.
> 
> All this is doing is papering over the cracks.

No, it turns two problems into one.  We have the problem you describe
anyway, there is no point avoiding it here.  It remains unsolved, but
that's just one problem left, not two.

Would it make sense to switch stacks if memory allocation has to free
other memory first?  Sounds a bit insane, but that problem needs a
solution as well.

Jörn

-- 
...one more straw can't possibly matter...
-- Kirby Bakken

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Andi Kleen]

Steve Lord <lord@xfs.org> writes:


> That was not really my point, consider any memory allocation on the
> stack which is being replaced with an allocate to save space. Then replace
> the saved stack space with the potential stack space used to
> free memory by writing it out via a filesystem. You cannot make all
> the allocations in the kernel GFP_NOFS.

I suspect making memory reclamation less multithreaded (doing 
it in specialized threads only) would be actually a good thing.
I remember there were lots of races in the past with too many
memory reclaimers in parallel that were just papered over with
various hacks. Maybe doing it with less threads would make
this all a bit more controlled.

The question is just how many threads it would need and if
any of the existing threads can do the job.

Overall it sounds more like 2.7 material.

-Andi

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Andrew Morton]

"Martin J. Bligh" <mbligh@aracnet.com> wrote:
>
> 2 nfs_writepage_sync is a known stack-abuser ;-) 1632 bytes on PPC64 at least
>  (from Anton's data). Maybe it's that struct nfs_write_data ?

 (from Anton's data). Maybe it's that struct nfs_write_data ?


diff -puN fs/nfs/write.c~nfs_writepage_sync-stack-reduction fs/nfs/write.c
--- 25/fs/nfs/write.c~nfs_writepage_sync-stack-reduction	2004-05-07 12:36:51.648098192 -0700
+++ 25-akpm/fs/nfs/write.c	2004-05-07 12:39:41.320304096 -0700
@@ -179,7 +179,13 @@ static int nfs_writepage_sync(struct fil
 {
 	unsigned int	wsize = NFS_SERVER(inode)->wsize;
 	int		result, written = 0;
-	struct nfs_write_data	wdata = {
+	struct nfs_write_data *wdata;
+
+	wdata = kmalloc(sizeof(*wdata), GFP_NOFS);
+	if (!wdata)
+		return -ENOMEM;
+
+	*wdata = (struct nfs_write_data) {
 		.flags		= how,
 		.cred		= NULL,
 		.inode		= inode,
@@ -192,8 +198,8 @@ static int nfs_writepage_sync(struct fil
 			.count		= wsize,
 		},
 		.res		= {
-			.fattr		= &wdata.fattr,
-			.verf		= &wdata.verf,
+			.fattr		= &wdata->fattr,
+			.verf		= &wdata->verf,
 		},
 	};
 
@@ -205,22 +211,22 @@ static int nfs_writepage_sync(struct fil
 	nfs_begin_data_update(inode);
 	do {
 		if (count < wsize)
-			wdata.args.count = count;
-		wdata.args.offset = page_offset(page) + wdata.args.pgbase;
+			wdata->args.count = count;
+		wdata->args.offset = page_offset(page) + wdata->args.pgbase;
 
-		result = NFS_PROTO(inode)->write(&wdata, file);
+		result = NFS_PROTO(inode)->write(wdata, file);
 
 		if (result < 0) {
 			/* Must mark the page invalid after I/O error */
 			ClearPageUptodate(page);
 			goto io_error;
 		}
-		if (result < wdata.args.count)
+		if (result < wdata->args.count)
 			printk(KERN_WARNING "NFS: short write, count=%u, result=%d\n",
-					wdata.args.count, result);
+					wdata->args.count, result);
 
-		wdata.args.offset += result;
-	        wdata.args.pgbase += result;
+		wdata->args.offset += result;
+	        wdata->args.pgbase += result;
 		written += result;
 		count -= result;
 	} while (count);
@@ -234,9 +240,10 @@ static int nfs_writepage_sync(struct fil
 
 io_error:
 	nfs_end_data_update_defer(inode);
-	if (wdata.cred)
-		put_rpccred(wdata.cred);
+	if (wdata->cred)
+		put_rpccred(wdata->cred);
 
+	kfree(wdata);
 	return written ? written : result;
 }
 

_

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Paul Jakma]

On Fri, 7 May 2004, Arjan van de Ven wrote:

> NFSv4 has a > 1Kb stack user; Dave Jones has a fix pending for
> that...

I'm using NFSv3 though.

regards,
-- 
Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
	warning: do not ever send email to spam@dishone.st
Fortune:
Never invest your money in anything that eats or needs repainting.
		-- Billy Rose

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Paul Jakma]

On Fri, 7 May 2004, Paul Jakma wrote:

> On Fri, 7 May 2004, Arjan van de Ven wrote:
> 
> > NFSv4 has a > 1Kb stack user; Dave Jones has a fix pending for
> > that...
> 
> I'm using NFSv3 though.

Well.. the mount itself is NFSv3 at least. The kernel does however
have NFSv4 client support and support for the rpc_pipefs fs.

regards,
-- 
Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
	warning: do not ever send email to spam@dishone.st
Fortune:
Forgive and forget.
		-- Cervantes

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Bill Davidsen]

Bartlomiej Zolnierkiewicz wrote:
> On Wednesday 05 of May 2004 22:31, Bill Davidsen wrote:
> 
>>Andrew Morton wrote:
>>
>>>Dominik Karall <dominik.karall@gmx.net> wrote:
>>>
>>>>On Wednesday 05 May 2004 10:31, you wrote:
>>>>
>>>>>+make-4k-stacks-permanent.patch
>>>>>
>>>>>Fill my inbox.
>>>>
>>>>Hi Andrew!
>>>>
>>>>Is there any reason why this patch was applied? Because NVidia users
>>>>can't work with the original drivers now without removing this patch
>>>>every time.
>>>
>>>We need to push this issue along quickly.  The single-page stack
>>>generally gives us a better kernel and having the stack size configurable
>>>creates pain.
>>
>>Add my voice to those who don't think 4k stacks are a good idea as a
>>default, they break some things and seem to leave other paths (as others
>>have noted) on the edge. I'm not sure what you have in mind as a "better
>>kernel" but I'd rather have a worse kernel and not have to check 4k
>>stack as a possible problem before looking at other things if I get bad
>>behaviour.
>>
>>Reliability first, performance later. We've lived with the config for a
>>while, pain there is better than pain at runtime.
> 
> 
> Opposite opinion here.
> 
> If you want 100% reliability you shouldn't use -mm in the first place.
> 
> Making 4kb stacks default in -mm is very good idea so it will get necessary
> testing and fixing before being integrated into mainline.
> 
> Please also note that users of binary only modules always have choice:
> - new kernels without binary only modules
> - old kernels with binary only modules
> 
> It is really that simple.

No it's not that simple, this has nothing to do with binary modules, and 
everything to do with not making 4k stack the only available 
configuration in 2.6. Options are fine, but in a stable kernel series I 
don't think think that the default should change part way into the 
series, and certainly the availability of the original functionality 
shouldn't go away, which is what I read AKPMs original post to state as 
the goal.

Making changes to the kernel which will break existing applications 
seems to be the opposite of "stable." People who want a new kernel for 
fixes don't usually want to have to upgrade and/or rewrite their 
applications. The "we change the system interface everything we fix a 
bug" approach comes from a well-known software company, but shouldn't be 
the way *good* software is done.

-- 
bill davidsen <davidsen@tmr.com>
   CTO TMR Associates, Inc
   Doing interesting things with small computers since 1979

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Bartlomiej Zolnierkiewicz]

On Sunday 09 of May 2004 19:00, Bill Davidsen wrote:
> Bartlomiej Zolnierkiewicz wrote:
> > On Wednesday 05 of May 2004 22:31, Bill Davidsen wrote:
> >>Andrew Morton wrote:
> >>>Dominik Karall <dominik.karall@gmx.net> wrote:
> >>>>On Wednesday 05 May 2004 10:31, you wrote:
> >>>>>+make-4k-stacks-permanent.patch
> >>>>>
> >>>>>Fill my inbox.
> >>>>
> >>>>Hi Andrew!
> >>>>
> >>>>Is there any reason why this patch was applied? Because NVidia users
> >>>>can't work with the original drivers now without removing this patch
> >>>>every time.
> >>>
> >>>We need to push this issue along quickly.  The single-page stack
> >>>generally gives us a better kernel and having the stack size
> >>> configurable creates pain.
> >>
> >>Add my voice to those who don't think 4k stacks are a good idea as a
> >>default, they break some things and seem to leave other paths (as others
> >>have noted) on the edge. I'm not sure what you have in mind as a "better
> >>kernel" but I'd rather have a worse kernel and not have to check 4k
> >>stack as a possible problem before looking at other things if I get bad
> >>behaviour.
> >>
> >>Reliability first, performance later. We've lived with the config for a
> >>while, pain there is better than pain at runtime.
> >
> > Opposite opinion here.
> >
> > If you want 100% reliability you shouldn't use -mm in the first place.
> >
> > Making 4kb stacks default in -mm is very good idea so it will get
> > necessary testing and fixing before being integrated into mainline.
> >
> > Please also note that users of binary only modules always have choice:
> > - new kernels without binary only modules
> > - old kernels with binary only modules
> >
> > It is really that simple.
>
> No it's not that simple, this has nothing to do with binary modules, and
> everything to do with not making 4k stack the only available
> configuration in 2.6. Options are fine, but in a stable kernel series I
> don't think think that the default should change part way into the
> series, and certainly the availability of the original functionality
> shouldn't go away, which is what I read AKPMs original post to state as
> the goal.

What functionality are you talking about?
We don't care about out of tree kernel code (be it GPL or Proprietary).

> Making changes to the kernel which will break existing applications
> seems to be the opposite of "stable." People who want a new kernel for
> fixes don't usually want to have to upgrade and/or rewrite their
> applications. The "we change the system interface everything we fix a

You don't understand what the patch is really about.

This is kernel stack not the user-space one so
this change can't brake any application.

> bug" approach comes from a well-known software company, but shouldn't be
> the way *good* software is done.

It doesn't change any kernel interface visible to user-space
and stack hungry kernel code needs fixing anyway.

Regards,
Bartlomiej

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Bill Davidsen]

Arjan van de Ven wrote:
>>It's probably a Bad Idea to push this to Linus before the vendors that have
>>significant market-impact issues (again - anybody other than NVidia here?)
>>have gotten their stuff cleaned up...
> 
> 
> Ok I don't want to start a flamewar but... Do we want to hold linux back
> until all binary only module vendors have caught up ??

My questions is, hold it back from what? Having the 4k option is fine, 
it's just eliminating the current default which I think is undesirable. 
I tried 4k stack, I couldn't measure any improvement in anything (as in 
no visible speedup or saving in memory). For an embedded system, where 
space is tight and the code paths well known, sure, but I haven't been 
able to find or generate any objective improvement, other than some 
posts saying smaller is always better. Nothing slows a system down like 
a crash, even if it isn't followed by a restore from backup.

Feel free to point me at some results showing major improvement from 4k 
stacks, I'm open to data.

-- 
    -bill davidsen (davidsen@tmr.com)
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Horst von Brand]

Bill Davidsen <davidsen@tmr.com> said:

[...]

> I tried 4k stack, I couldn't measure any improvement in anything (as in 
> no visible speedup or saving in memory).

4K stacks lets the kernel create new threads/processes as long as there is
free memory; with 8K stacks it needs two consecutive free page frames in
physical memory, when memory is fragmented (and large) they are hard to
come by...
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Andrew Morton]

Horst von Brand <vonbrand@inf.utfsm.cl> wrote:
>
> Bill Davidsen <davidsen@tmr.com> said:
> 
> [...]
> 
> > I tried 4k stack, I couldn't measure any improvement in anything (as in 
> > no visible speedup or saving in memory).
> 
> 4K stacks lets the kernel create new threads/processes as long as there is
> free memory; with 8K stacks it needs two consecutive free page frames in
> physical memory, when memory is fragmented (and large) they are hard to
> come by...

This is true to a surprising extent.  A couple of weeks ago I observed my
256MB box freeing over 20MB of pages before it could successfully acquire a
single 1-order page.

That was during an updatedb run.

And a 1-order GFP_NOFS allocation was actually livelocking, because
!__GFP_FS allocations aren't allowed to enter dentry reclaim.  Which is why
VFS caches are now forced to use 0-order allocations.

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Helge Hafting]

Bill Davidsen wrote:

> Arjan van de Ven wrote:
>
>>> It's probably a Bad Idea to push this to Linus before the vendors 
>>> that have
>>> significant market-impact issues (again - anybody other than NVidia 
>>> here?)
>>> have gotten their stuff cleaned up...
>>
>>
>>
>> Ok I don't want to start a flamewar but... Do we want to hold linux back
>> until all binary only module vendors have caught up ??
>
>
> My questions is, hold it back from what? Having the 4k option is fine, 
> it's just eliminating the current default which I think is 
> undesirable. I tried 4k stack, I couldn't measure any improvement in 
> anything (as in no visible speedup or saving in memory). 

The memory saving is usually modest: 4k per thread. It might make a 
difference for
those with many thousands of threads.   I believe this is unswappable 
memory,
which is much more valuable than ordinary process memory.

More interesting is that it removes one way for fork() to fail. With 8k 
stacks,
the new process needs to allocate two consecutive pages for those 8k.  That
might be impossible due to fragmentation, even if there are megabytes of 
free
memory. Such a problem usually only shows up after a long time.  Now we 
only need to
allocate a single page, which always works as long as there is any free 
memory at all.

> For an embedded system, where space is tight and the code paths well 
> known, sure, but I haven't been able to find or generate any objective 
> improvement, other than some posts saying smaller is always better. 
> Nothing slows a system down like a crash, even if it isn't followed by 
> a restore from backup.

Consider the case when your server (web/mail/other) fails to fork, and then
you can't login because that requires fork() too.  4k stacks remove this 
scenario,
and is a stability improvement.

Helge Hafting

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Bill Davidsen]

On Sun, 9 May 2004, Bartlomiej Zolnierkiewicz wrote:

> On Sunday 09 of May 2004 19:00, Bill Davidsen wrote:
> > No it's not that simple, this has nothing to do with binary modules, and
> > everything to do with not making 4k stack the only available
> > configuration in 2.6. Options are fine, but in a stable kernel series I
> > don't think think that the default should change part way into the
> > series, and certainly the availability of the original functionality
> > shouldn't go away, which is what I read AKPMs original post to state as
> > the goal.
> 
> What functionality are you talking about?
> We don't care about out of tree kernel code (be it GPL or Proprietary).

Let me say this one more time, since you keep changing the topic so you
can say that you don't care about something I never mentioned. I am
**NOT** talking about binary modules, I am **NOT** talking about out of
tree code, I am talking about applications which make calls that cause the
**IN TREE** code to use more than 4k.

> 
> > Making changes to the kernel which will break existing applications
> > seems to be the opposite of "stable." People who want a new kernel for
> > fixes don't usually want to have to upgrade and/or rewrite their
> > applications. The "we change the system interface everything we fix a
> 
> You don't understand what the patch is really about.
> 
> This is kernel stack not the user-space one so
> this change can't brake any application.

Right, the kernel code does not contain any places where the data passed
in a system call isn't reflected in stack usage.

> 
> > bug" approach comes from a well-known software company, but shouldn't be
> > the way *good* software is done.
> 
> It doesn't change any kernel interface visible to user-space
> and stack hungry kernel code needs fixing anyway.

And what better way to detect it than to release it in a stable kernel.
Don't bother to say "don't use -mm" AKPM has said it is intended for the
stable kernel, work or not.

===
Third request for info
===
I still haven't seen any objective data showing that there is any
measureable benefit from this, although I agree that smaller is good
practice, I don't think that throwing in a feature in a stable kernel,
which has been reported by others to corrupt data, is the best way to do
it.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Bartlomiej Zolnierkiewicz]

On Tuesday 11 of May 2004 18:24, Bill Davidsen wrote:
> On Sun, 9 May 2004, Bartlomiej Zolnierkiewicz wrote:
> > On Sunday 09 of May 2004 19:00, Bill Davidsen wrote:
> > > No it's not that simple, this has nothing to do with binary modules,
> > > and everything to do with not making 4k stack the only available
> > > configuration in 2.6. Options are fine, but in a stable kernel series I
> > > don't think think that the default should change part way into the
> > > series, and certainly the availability of the original functionality
> > > shouldn't go away, which is what I read AKPMs original post to state as
> > > the goal.
> >
> > What functionality are you talking about?
> > We don't care about out of tree kernel code (be it GPL or Proprietary).
>
> Let me say this one more time, since you keep changing the topic so you
> can say that you don't care about something I never mentioned. I am
> **NOT** talking about binary modules, I am **NOT** talking about out of
> tree code, I am talking about applications which make calls that cause the
> **IN TREE** code to use more than 4k.

No need to flame, I really didn't know what you were talking about.

I agree that this is a very good argument against pushing this
change to mainline quickly (proposed originally by AKPM).

> > > Making changes to the kernel which will break existing applications
> > > seems to be the opposite of "stable." People who want a new kernel for
> > > fixes don't usually want to have to upgrade and/or rewrite their
> > > applications. The "we change the system interface everything we fix a
> >
> > You don't understand what the patch is really about.
> >
> > This is kernel stack not the user-space one so
> > this change can't brake any application.
>
> Right, the kernel code does not contain any places where the data passed
> in a system call isn't reflected in stack usage.

It won't break applications it will break kernel first. ;-)

You need to fix kernel code not the user space.

> > > bug" approach comes from a well-known software company, but shouldn't
> > > be the way *good* software is done.
> >
> > It doesn't change any kernel interface visible to user-space
> > and stack hungry kernel code needs fixing anyway.
>
> And what better way to detect it than to release it in a stable kernel.
> Don't bother to say "don't use -mm" AKPM has said it is intended for the
> stable kernel, work or not.

I see no problem with this approach (this patch in -mm then in linus')
but issues mentioned by you need fixing first.  I'm not proposing to
push it to mainline NOW - it needs to be done CAREFULLY but CAN be
done in 2.6 (i.e. 2.6.15).

I guess this is what we can't agree on.

> ===
> Third request for info
> ===
> I still haven't seen any objective data showing that there is any
> measureable benefit from this, although I agree that smaller is good
> practice, I don't think that throwing in a feature in a stable kernel,
> which has been reported by others to corrupt data, is the best way to do
> it.

There was some evidence from AKPM (and Arjan AFAIR).
[ BTW wasn't the corruption only seen with nvidia module? ]
I think we can prevent it by adding something ala 4kstack flag
to the module.

Regards,
Bartlomiej

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Andrew Morton]

Bartlomiej Zolnierkiewicz <B.Zolnierkiewicz@elka.pw.edu.pl> wrote:
>
> There was some evidence from AKPM (and Arjan AFAIR).
> [ BTW wasn't the corruption only seen with nvidia module? ]
> I think we can prevent it by adding something ala 4kstack flag
> to the module.

"4KSTACKS" already is present in the module version string.

And RHL is shipping now with 4k stacks, so presumably any disasters
are relatively uncommon...

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 815 bytes --]

On Tue, 11 May 2004 16:50:13 PDT, Andrew Morton said:

> And RHL is shipping now with 4k stacks, so presumably any disasters
> are relatively uncommon...

I think currently, it's only the people running the -test or -devel Fedora
trees that can get bit - Fedora Core 1 didn't have it, as it was still a
2.4 kernel, and Fedora Core 2 just kicked over from FC 1.92 last night..

And then they'd have to have NVidia cards, and use the NVIdia drivers..

and I suspect that most of the people who are in that group visit either
one of the NVidia forums or www.minion.de, both of which have lots
of tags saying that patch needs reverting...

So right now it's probably NOT biting too many, as only the clueful are
getting into a situation where it's a problem.

Wait a week or two, when .ISO's of FC2 hit the streets. :)

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Bill Davidsen]

On Tue, 11 May 2004, Andrew Morton wrote:

> Bartlomiej Zolnierkiewicz <B.Zolnierkiewicz@elka.pw.edu.pl> wrote:
> >
> > There was some evidence from AKPM (and Arjan AFAIR).
> > [ BTW wasn't the corruption only seen with nvidia module? ]
> > I think we can prevent it by adding something ala 4kstack flag
> > to the module.
> 
> "4KSTACKS" already is present in the module version string.
> 
> And RHL is shipping now with 4k stacks, so presumably any disasters
> are relatively uncommon...

RHL and kernel.org have a lot of unshared bugs and features,
unfortunately. I take that information as an encouraging proof of concept,
not a waranty that the kernel.org code will behave in a similar way.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 601 bytes --]


> > "4KSTACKS" already is present in the module version string.
> > 
> > And Fedora is shipping now with 4k stacks, so presumably any disasters
> > are relatively uncommon...
> 
> Fedora and kernel.org have a lot of unshared bugs and features,
> unfortunately. I take that information as an encouraging proof of concept,
> not a waranty that the kernel.org code will behave in a similar way.

Hey! That's slander of title! :-)
Seriously the difference between the Fedora Core 2 kernel and the
matching kernel.org kernel aren't THAT big. The 4g/4g split patch being
the biggest delta.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: 2.6.6-rc3-mm2 (4KSTACK)

[Bill Davidsen]

Arjan van de Ven wrote:
>>>"4KSTACKS" already is present in the module version string.
>>>
>>>And Fedora is shipping now with 4k stacks, so presumably any disasters
>>>are relatively uncommon...
>>
>>Fedora and kernel.org have a lot of unshared bugs and features,
>>unfortunately. I take that information as an encouraging proof of concept,
>>not a waranty that the kernel.org code will behave in a similar way.
> 
> 
> Hey! That's slander of title! :-)
> Seriously the difference between the Fedora Core 2 kernel and the
> matching kernel.org kernel aren't THAT big. The 4g/4g split patch being
> the biggest delta.
> 
I was thinking about the one you charge for... I wouldn't use FCn for 
production on a bet. I was thinking of my RHEL AS3.0 vs. kernel.org, 
which are kernels stable enough for commercial use.

-- 
bill davidsen <davidsen@tmr.com>
   CTO TMR Associates, Inc
   Doing interesting things with small computers since 1979