Re: [PATCH] (3/3) stack overflow checking for x86

[Andreas Dilger <adilger@clusterfs.com>]

On Oct 18, 2002  15:31 -0700, Dave Hansen wrote:
> Greg KH just pointed out that someone else snuck in an overflow check. 
>  However, they take completely different approaches.  The Ben LaHaise 
> one that I posted uses GCC features to check the stack on entry to all 
> functions.  The one in the tree now is much, much simpler than Ben's, 
> but only works only for detecting problems at the time that an 
> interrupt actually occurs.

Yes, we had that patch in the RH 2.4 kernel, but it didn't detect the
stack overflow we spent days to track.  Our problem was just one of
a really deep stack: RPC-to-network-filesystem-on-ext3-with-htree.
This overflowed the 8kB stack even without an interrupt happening.

We fixed the htree and the network filesystem stack usage, but if we
added LVM/EVMS into the mix (which we would have in production) and
threw in a piggy interrupt handler we would possibly again overflow
an 8kB stack, so having a per-entry/exit stack checker would be great.

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/