[uml-devel] [PATCH] Fix stack corruption

[uml-devel] [PATCH] Fix stack corruption

[Jeff Dike]

This patch fixes a long-standing problem in skas mode process creation.  Chris
Aker has been seeing it at linode, and found a way of reproducing it.  Once
I spotted the bug, I found an easier way:

ping flood the UML from the host while running
	while true; do ls > /dev/null; done

In 10-15 seconds, UML will simply exit back to the shell with a segfault,
no panic, no output, no nothing.

When UML sets up the kernel stack for a new process, it sends itself a 
SA_ONSTACK signal with the signal stack being the new kernel stack.  It 
calls setjmp there to set up a context that it can longjmp to when the new
process is run for the first time.

The problem was that, while signals were blocked during this, they were 
re-enabled before SA_ONSTACK was disabled.  Thus, a signal arriving at the
wrong time, between signals being turned on and SA_ONSTACK being disabled,
would cause the signal to be handled on the stack, destroying the context
that had been set up there.

When the new process ran, it would longjmp to this trashed stack, and UML
would die.

The fix is obvious:

Index: 2.6.10/arch/um/kernel/skas/process.c
===================================================================
--- 2.6.10.orig/arch/um/kernel/skas/process.c	2005-01-12 11:17:22.000000000 -0500
+++ 2.6.10/arch/um/kernel/skas/process.c	2005-01-12 11:18:03.000000000 -0500
@@ -323,9 +323,10 @@
 	block_signals();
 	if(sigsetjmp(fork_buf, 1) == 0)
 		new_thread_proc(stack, handler);
-	set_signals(flags);
 
 	remove_sigstack();
+
+	set_signals(flags);
 }
 
 void thread_wait(void *sw, void *fb)

				Jeff



-------------------------------------------------------
The SF.Net email is sponsored by: Beat the post-holiday blues
Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] [PATCH] Fix stack corruption

[Blaisorblade]

On Wednesday 12 January 2005 19:56, Jeff Dike wrote:
> This patch fixes a long-standing problem in skas mode process creation. 
So, probably, unless confirmed otherwise, the problem with CONFIG_KMOD (when 
the kernel starts modprobe from a kernel thread) could be due to this bug... 
however, I'm not so sure, because IIRC that problem give a verbose error 
message (I think even a stack trace).
-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729
http://www.user-mode-linux.org/~blaisorblade


-------------------------------------------------------
The SF.Net email is sponsored by: Beat the post-holiday blues
Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] [PATCH] Fix stack corruption

[Christopher S. Aker]

> The fix is obvious:
>
> Index: 2.6.10/arch/um/kernel/skas/process.c
> ===================================================================
> --- 2.6.10.orig/arch/um/kernel/skas/process.c 2005-01-12 11:17:22.000000000 -0500
> +++ 2.6.10/arch/um/kernel/skas/process.c 2005-01-12 11:18:03.000000000 -0500
> @@ -323,9 +323,10 @@
>   block_signals();
>   if(sigsetjmp(fork_buf, 1) == 0)
>   new_thread_proc(stack, handler);
> - set_signals(flags);
>
>   remove_sigstack();
> +
> + set_signals(flags);
>  }
>
>  void thread_wait(void *sw, void *fb)

Just a follow up on this for the sake of a paper trail...  This doesn't fix the
test-case below:

http://www.theshore.net/~caker/uml/crashkit-no-console-output/

readme.txt:
----
All you need is an UML guest assigned an IP on your network, iptables, and
the two files below in the same directory.  Run the script, while ping-flooding
the UML's IP address from the host, or another machine on your network.

This script works best with a 2.6 based UML.  I can recreate the crash easily using
2.6.9-linode9 (based on -bb4), also available on this website.
----

I believe we've figured out that this is skas3 specific.  Once you start
ping-flooding the UML, it crashes within a second.

-Chris



-------------------------------------------------------
The SF.Net email is sponsored by: Beat the post-holiday blues
Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] [PATCH] Fix stack corruption

[Blaisorblade]

On Friday 14 January 2005 00:30, Christopher S. Aker wrote:
> > The fix is obvious:
> >
> > Index: 2.6.10/arch/um/kernel/skas/process.c
> > ===================================================================
> > --- 2.6.10.orig/arch/um/kernel/skas/process.c 2005-01-12
> > 11:17:22.000000000 -0500 +++ 2.6.10/arch/um/kernel/skas/process.c
> > 2005-01-12 11:18:03.000000000 -0500 @@ -323,9 +323,10 @@
> >   block_signals();
> >   if(sigsetjmp(fork_buf, 1) == 0)
> >   new_thread_proc(stack, handler);
> > - set_signals(flags);
> >
> >   remove_sigstack();
> > +
> > + set_signals(flags);
> >  }
> >
> >  void thread_wait(void *sw, void *fb)
>
> Just a follow up on this for the sake of a paper trail...  This doesn't fix
> the test-case below:
>
> http://www.theshore.net/~caker/uml/crashkit-no-console-output/
>
> readme.txt:
> ----
> All you need is an UML guest assigned an IP on your network, iptables, and
> the two files below in the same directory.  Run the script, while
> ping-flooding the UML's IP address from the host, or another machine on
> your network.
>
> This script works best with a 2.6 based UML.  I can recreate the crash
> easily using 2.6.9-linode9 (based on -bb4), also available on this website.
> ----
>
> I believe we've figured out that this is skas3 specific.  Once you start
> ping-flooding the UML, it crashes within a second.
I wasn't able to riproduce it... I ran two instances of the above loop + a 
ping flood from the host, but it didn't crash UML, even after letting it that 
way for some minutes.

-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729
http://www.user-mode-linux.org/~blaisorblade


-------------------------------------------------------
The SF.Net email is sponsored by: Beat the post-holiday blues
Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] [PATCH] Fix stack corruption

[Christopher S. Aker]

> > I believe we've figured out that this is skas3 specific.  Once you start
> > ping-flooding the UML, it crashes within a second.
> I wasn't able to riproduce it... I ran two instances of the above loop + a
> ping flood from the host, but it didn't crash UML, even after letting it that
> way for some minutes.

What UML version are you running?  Also, can you reproduce it without Jeff's patch?

Under skas3, I can't crash my UML's using Jeff's "ls > /dev/null" method, with or
without his patch.

I've been able to reproduce my crash with two different host kernels, an older 2.6.4,
and the newest 2.6.10+skas-v7.  Guest kernels that crash that I've tried include
2.6.9-bb4, 2.6.10, and 2.6.10-mm2 plus the incrementals from yesterday.  Also, I
believe Frank Sorenson is also able to reproduce this crash.

Thanks,
-Chris



-------------------------------------------------------
The SF.Net email is sponsored by: Beat the post-holiday blues
Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] [PATCH] Fix stack corruption

[Blaisorblade]

On Friday 14 January 2005 01:35, Christopher S. Aker wrote:
> > > I believe we've figured out that this is skas3 specific.  Once you
> > > start ping-flooding the UML, it crashes within a second.
> >
> > I wasn't able to riproduce it... I ran two instances of the above loop +
> > a ping flood from the host, but it didn't crash UML, even after letting
> > it that way for some minutes.
>
> What UML version are you running?  Also, can you reproduce it without
> Jeff's patch?
Sorry, forgot about that - I did the test only on 2.6.9-SKASv7 host and 
2.6.9-bb4 guest.
> Under skas3, I can't crash my UML's using Jeff's "ls > /dev/null" method,
> with or without his patch.

> I've been able to reproduce my crash
I've forgot to look at your recipe - I'll do it tomorrow.
> with two different host kernels, an 
> older 2.6.4, and the newest 2.6.10+skas-v7.  Guest kernels that crash that
> I've tried include 2.6.9-bb4, 2.6.10, and 2.6.10-mm2 plus the incrementals
> from yesterday.  Also, I believe Frank Sorenson is also able to reproduce
> this crash.

> Thanks,
> -Chris
Thanks to you!
-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729
http://www.user-mode-linux.org/~blaisorblade


-------------------------------------------------------
The SF.Net email is sponsored by: Beat the post-holiday blues
Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel