2.4 SRMMU bug revisited

2.4 SRMMU bug revisited

[Tomas Szepe]

>	From: Tomas Szepe <szepe@pinerecords.com>
>	Date: Thu, 23 May 2002 09:02:44 +0200
> 
>	> > 2. What is the "proper" fix for the patch collision between the raid
>	> > patch and the ext3 patch in /include/linux/fs.h? 
>	> 
>	> Use 2.4.
>	
>	Impossible on sparc32 on account of the lurking SRMMU bug.
>	(See yesterday's post by Joris Braakman <jorisb@nl.euro.net>.)
>
> There have been several patches posted to deal with that
> problem, you can apply them yourself or grab Marcelo's
> current 2.4.x BK tree.

I finally got round to trying the patches out and --

unfortunately, things got even worse. While before the machine would
oops (still allowing one to ssh in and reboot) under heavy loads, now
it doesn't bother to log the slightest notice that something might
have broken, and freezes entirely (it can be pinged, though).

I can't say I like these fixes much.


T.

Re: 2.4 SRMMU bug revisited

[David S. Miller]

   From: Tomas Szepe <szepe@pinerecords.com>
   Date: Mon, 27 May 2002 11:24:08 +0200
   
   I can't say I like these fixes much.

How about contributing some time to figuring out what
the problem is instead of complaining all the time?

%99 of people with sparc32 problems are totally unwilling (or unable)
to help fix the bugs themselves, they merely whine.  It gets a bit
old after a while (which for me amount to roughly 7 years).

Re: 2.4 SRMMU bug revisited

[Tomas Szepe]

>    From: Tomas Szepe <szepe@pinerecords.com>
>    Date: Mon, 27 May 2002 11:24:08 +0200
>    
>    I can't say I like these fixes much.
> 
> How about contributing some time to figuring out what
> the problem is instead of complaining all the time?
> 
> %99 of people with sparc32 problems are totally unwilling (or unable)
> to help fix the bugs themselves, they merely whine.  It gets a bit
> old after a while (which for me amount to roughly 7 years).


Sure, I'm willing to try to write the fix if someone takes the time
to explain to me what's happening -- I'm quite fluent in C but know
nothing about the anatomy of the problem.

T.

Re: 2.4 SRMMU bug revisited

[David S. Miller]

   From: Tomas Szepe <szepe@pinerecords.com>
   Date: Mon, 27 May 2002 08:20:44 +0200

   > %99 of people with sparc32 problems are totally unwilling (or unable)
   > to help fix the bugs themselves, they merely whine.  It gets a bit
   > old after a while (which for me amount to roughly 7 years).
   
   Sure, I'm willing to try to write the fix if someone takes the time
   to explain to me what's happening -- I'm quite fluent in C but know
   nothing about the anatomy of the problem.

See the part where I mention "unable".

Re: [aurora-sparc-devel] Re: 2.4 SRMMU bug revisited

[Tomas Szepe]

>    > %99 of people with sparc32 problems are totally unwilling (or unable)
>    > to help fix the bugs themselves, they merely whine.  It gets a bit
>    > old after a while (which for me amount to roughly 7 years).
>    
>    Sure, I'm willing to try to write the fix if someone takes the time
>    to explain to me what's happening -- I'm quite fluent in C but know
>    nothing about the anatomy of the problem.
> 
> See the part where I mention "unable".


Whoa. Let me recapitulate:
	- first I provide feedback on patches.

	-> I'm told I'm not helping but merely whining.

	- then I offer to fix the problem myself given somebody explains
	what's going on in the code so that I at least know where to start
	and what to look for.

	-> I am told by DaveM to re-read a sentence by DaveM that seems
	to inform me that even if I'm willing to help, I'm likely not able
	to, anyway.

Hello!! How exactly is *this* helping then?


T.

Re: [aurora-sparc-devel] Re: 2.4 SRMMU bug revisited

[Denis Vlasenko]

On 27 May 2002 09:19, Tomas Szepe wrote:
> Whoa. Let me recapitulate:
> 	- first I provide feedback on patches.
>
> 	-> I'm told I'm not helping but merely whining.
>
> 	- then I offer to fix the problem myself given somebody explains
> 	what's going on in the code so that I at least know where to start
> 	and what to look for.
>
> 	-> I am told by DaveM to re-read a sentence by DaveM that seems
> 	to inform me that even if I'm willing to help, I'm likely not able
> 	to, anyway.
>
> Hello!! How exactly is *this* helping then?

Seems like DaveM says you have to dig deeper. Alone.
Hm, maybe he has a reason. He may tell you what's up there.
He may as well fix it and send you a patch (using roughly
same amount of time).

But he wants some of "%99 of people with sparc32 problems" to move
into "%1" category instead.
--
vda

Re: [aurora-sparc-devel] Re: 2.4 SRMMU bug revisited

[David S. Miller]

   From: Denis Vlasenko <vda@port.imtp.ilyichevsk.odessa.ua>
   Date: Tue, 28 May 2002 14:41:20 -0200
   
   But he wants some of "%99 of people with sparc32 problems" to move
   into "%1" category instead.

Thank you, that is exactly what I meant.

Look, if someone reports some bug once, that's ok.  You report it
twice, I'll give you the benefit of the doubt that you're merely
reminding us.  The third and further time, it's a whine.  Everyone
here knows there are sun4m problems, and Tom in particular knows it is
a well known problem because I know he reads this list.  Therefore Tom
was engaging in what I'd like to term a double-whine.

Re: 2.4 SRMMU bug revisited

[Colin Gibbs]

On Mon, 2002-05-27 at 04:24, Tomas Szepe wrote:

> I finally got round to trying the patches out and --
> 
> unfortunately, things got even worse. While before the machine would
> oops (still allowing one to ssh in and reboot) under heavy loads, now
> it doesn't bother to log the slightest notice that something might
> have broken, and freezes entirely (it can be pinged, though).
> 
> I can't say I like these fixes much.
> 
> 
> T.

What kinds of heavy loads? If you were triggering the out of nocache
memory BUG, then this patch may help. I fixes a bug where fork fails and
calls destroy_context on the parent's mm or more precisely a memcpy'd
duplicate of it. In that case when fork returns to the parent, it
continuously faults.

But if your load does not fork heavily, then this is probably not the
problem.


Colin

--- 2.4.19-pre4/kernel/fork.c	Thu Mar 28 19:49:36 2002
+++ tortoise-19-pre4/kernel/fork.c	Sun Apr 21 22:01:18 2002
@@ -336,6 +336,9 @@
 	if (!mm_init(mm))
 		goto fail_nomem;
 
+	if (init_new_context(tsk,mm))
+		goto free_pt;
+
 	down_write(&oldmm->mmap_sem);
 	retval = dup_mmap(mm);
 	up_write(&oldmm->mmap_sem);
@@ -347,9 +350,6 @@
 	 * child gets a private LDT (if there was an LDT in the parent)
 	 */
 	copy_segments(tsk, mm);
-
-	if (init_new_context(tsk,mm))
-		goto free_pt;
 
 good_mm:
 	tsk->mm = mm;

Re: 2.4 SRMMU bug revisited

[Colin Gibbs]

On Mon, 2002-05-27 at 13:46, Colin Gibbs wrote:

> What kinds of heavy loads? If you were triggering the out of nocache
> memory BUG, then this patch may help. I fixes a bug where fork fails and
> calls destroy_context on the parent's mm or more precisely a memcpy'd
> duplicate of it. In that case when fork returns to the parent, it
> continuously faults.
> 
> But if your load does not fork heavily, then this is probably not the
> problem.

It seems this is in the bitkeeper tree, so ignore that if you used the
bitkeeper tree. However I'd still like to know what sort of loads are
causing you problems.


Colin

Re: 2.4 SRMMU bug revisited

[Tomas Szepe]

> > What kinds of heavy loads? If you were triggering the out of nocache
> > memory BUG, then this patch may help. I fixes a bug where fork fails and
> > calls destroy_context on the parent's mm or more precisely a memcpy'd
> > duplicate of it. In that case when fork returns to the parent, it
> > continuously faults.
> > 
> > But if your load does not fork heavily, then this is probably not the
> > problem.
> 
> It seems this is in the bitkeeper tree, so ignore that if you used the
> bitkeeper tree. However I'd still like to know what sort of loads are
> causing you problems.

Jup, I used the bk tree.

n=100
i=0
while [ ! "$i" = "$n" ]; do
        echo test|mail -s test user@targetmachine
        i=$(expr $i + 1)
done

targetmachine (running sendmail) goes down in under 20 seconds.

T.

Re: 2.4 SRMMU bug revisited

[Tomas Szepe]

> --- 2.4.19-pre4/kernel/fork.c	Thu Mar 28 19:49:36 2002
> +++ tortoise-19-pre4/kernel/fork.c	Sun Apr 21 22:01:18 2002
> @@ -336,6 +336,9 @@
>  	if (!mm_init(mm))
>  		goto fail_nomem;
>  
> +	if (init_new_context(tsk,mm))
> +		goto free_pt;
> +
>  	down_write(&oldmm->mmap_sem);
>  	retval = dup_mmap(mm);
>  	up_write(&oldmm->mmap_sem);
> @@ -347,9 +350,6 @@
>  	 * child gets a private LDT (if there was an LDT in the parent)
>  	 */
>  	copy_segments(tsk, mm);
> -
> -	if (init_new_context(tsk,mm))
> -		goto free_pt;
>  
>  good_mm:
>  	tsk->mm = mm;


Hmmm, upon closer inspection I found out this patch had *not* been in
what I extracted from the linux-2.4 bitkeeper tree, so my report was
probably not of much value. I'll retest tomorrow with the above applied.
Sorry for the confusion <-- I still can't see the patch in the listings
at http://linux.bkbits.net:8080/linux-2.4/

The patch I used is
http://www.dragon.cz/~kala/patch-2.4.19-pre8-sparcfixes-upto020523-1.gz
(I assembled it by hand using the bk web interface, looking up all sparc
related changes since -pre8.)

T.

Re: 2.4 SRMMU bug revisited

[David S. Miller]

   From: Tomas Szepe <szepe@pinerecords.com>
   Date: Mon, 27 May 2002 23:30:16 +0200

   http://linux.bkbits.net:8080/linux-2.4/
   
The BK repository to use has the URL:

	bk://linux.bkbits.net/linux-2.4

The web stuff is updated still by hand and is as a result chronically
out of date.

Re: 2.4 SRMMU bug revisited

[Tomas Szepe]

>    From: Tomas Szepe <szepe@pinerecords.com>
>    Date: Mon, 27 May 2002 23:30:16 +0200
> 
>    http://linux.bkbits.net:8080/linux-2.4/
>    
> The BK repository to use has the URL:
> 
> 	bk://linux.bkbits.net/linux-2.4
> 
> The web stuff is updated still by hand and is as a result chronically
> out of date.


So much for not having to install BK.  ;D

Re: 2.4 SRMMU bug revisited

[David Woodhouse]

szepe@pinerecords.com said:
> > The web stuff is updated still by hand and is as a result chronically
> > out of date.

> So much for not having to install BK.  ;D

	http://ftp.kernel.org/pub/linux/kernel/people/dwmw2/bk-2.4/

That web stuff is updated by cron and is as a result never more than an hour 
out of date (w.r.t. bk//linux.bkbits.net/linux-2.4) unless something breaks.

--
dwmw2

Re: 2.4 SRMMU bug revisited

[Alan Cox]

On Tue, 2002-05-28 at 04:41, David S. Miller wrote:
>    From: Tomas Szepe <szepe@pinerecords.com>
>    Date: Mon, 27 May 2002 23:30:16 +0200
> 
>    http://linux.bkbits.net:8080/linux-2.4/
>    
> The BK repository to use has the URL:
> 
> 	bk://linux.bkbits.net/linux-2.4
> 
> The web stuff is updated still by hand and is as a result chronically
> out of date.

Which is a concern since both Linus and Larry made it clear bitkeeper
would *NEVER* be required of contributors. Is there nothing generating
nightly tarballs off cron right now ?

Re: 2.4 SRMMU bug revisited

[David S. Miller]

   From: Alan Cox <alan@lxorguk.ukuu.org.uk>
   Date: 28 May 2002 12:58:38 +0100
   
   Which is a concern since both Linus and Larry made it clear bitkeeper
   would *NEVER* be required of contributors. Is there nothing generating
   nightly tarballs off cron right now ?
   
As stated in a followup posting, that kind of thing is available.

I don't see why people fly off the handle on this.  It only matters if
you need the tree as of 5 minutes ago bleeding edge type stuff.  If
you can wait a few days, then just hang on for Marcelo's next
prepatch.

Re: 2.4 SRMMU bug revisited

[Ben Collins]

On Tue, May 28, 2002 at 04:05:24AM -0700, David S. Miller wrote:
>    From: Alan Cox <alan@lxorguk.ukuu.org.uk>
>    Date: 28 May 2002 12:58:38 +0100
>    
>    Which is a concern since both Linus and Larry made it clear bitkeeper
>    would *NEVER* be required of contributors. Is there nothing generating
>    nightly tarballs off cron right now ?
>    
> As stated in a followup posting, that kind of thing is available.
> 
> I don't see why people fly off the handle on this.  It only matters if
> you need the tree as of 5 minutes ago bleeding edge type stuff.  If
> you can wait a few days, then just hang on for Marcelo's next
> prepatch.

Not trying to add fuel to the fire, but Marcelo's last prepatch was
almost 4 weeks ago (26 days). It makes life hard on even me, because
somehow a little tweak ends up changing the ieee1394 subsystem, and I
find myself resyncing over several pre-releases to get my patches in.

It sometimes seems that the bitkeeper repo is being used to pacify the
bleeding edge junkies (and rightfully so), but at the same time is
allowing the tree maintainers (not point fingers) to be more lax in
getting out frequent pre/full releases to the non-bitkeeper public (like
me). This inadvertently stalls productivity for the rest of us.

My problem getting things into the 2.5 tree seems more related to the
amount of work getting done in that tree as opposed to the lack of
releases, which is entirely acceptable. I'd prefer chasing frequent
releases to chasing invisible ones :)

-- 
Debian     - http://www.debian.org/
Linux 1394 - http://linux1394.sourceforge.net/
Subversion - http://subversion.tigris.org/
Deqo       - http://www.deqo.com/

Re: 2.4 SRMMU bug revisited

[David S. Miller]

   From: Ben Collins <bcollins@debian.org>
   Date: Tue, 28 May 2002 08:10:44 -0400
   
   Not trying to add fuel to the fire, but Marcelo's last prepatch was
   almost 4 weeks ago (26 days).

For most of that time I wasn't even getting responses from him, only
recently (within the last week) did he start pulling in my changes.

If you want Marcelo to make prepatches more often, simply ask him to
do it :-)

A lot of the BK grievences amount to "Well, how would you approach
this situation if BK didn't exit?" --> "Oh, I'd ask the maintainer to
make prepatches more often."

Re: 2.4 SRMMU bug revisited

[David S. Miller]

I pushed this fix to Marcelo already and it is already in
his BK tree.

Re: 2.4 SRMMU bug revisited

[Tomas Szepe]

> --- 2.4.19-pre4/kernel/fork.c	Thu Mar 28 19:49:36 2002
> +++ tortoise-19-pre4/kernel/fork.c	Sun Apr 21 22:01:18 2002
> @@ -336,6 +336,9 @@
>  	if (!mm_init(mm))
>  		goto fail_nomem;
>  
> +	if (init_new_context(tsk,mm))
> +		goto free_pt;
> +
>  	down_write(&oldmm->mmap_sem);
>  	retval = dup_mmap(mm);
>  	up_write(&oldmm->mmap_sem);
> @@ -347,9 +350,6 @@
>  	 * child gets a private LDT (if there was an LDT in the parent)
>  	 */
>  	copy_segments(tsk, mm);
> -
> -	if (init_new_context(tsk,mm))
> -		goto free_pt;
>  
>  good_mm:
>  	tsk->mm = mm;


A big, big thankyou to Colin.

This patch indeed makes difference. I stressed -pre9 as much as I could
(simultaneous reading from raid devices - ext3/reiserfs/ext2, NFS traffic,
sendmail+apache fork storms... you name it, I ran it, all at the same time)
and the kernel lives. Processes still get killed by VM much earlier than
they should (100+ MB RAM free), but that's not critical.

A nice conclusion to this thread, isn't it? :)

T.

Re: 2.4 SRMMU bug revisited

[Pete Zaitcev]

>> The BK repository to use has the URL:
>> 
>> 	bk://linux.bkbits.net/linux-2.4
>> 
>> The web stuff is updated still by hand and is as a result chronically
>> out of date.
> 
> Which is a concern since both Linus and Larry made it clear bitkeeper
> would *NEVER* be required of contributors. Is there nothing generating
> nightly tarballs off cron right now ?

It is still not required, in theory. A developer may elect
to wait until the change recycles all the way to Linus,
then receive it with regular kernels. This is what I do --
tired of BK... Also, I am too stupid to understand the
concept of changeset.

-- Pete