Problem with ucontext_t struct in signal handler

Problem with ucontext_t struct in signal handler

[XComp]

Hello everyone,
I want to switch between user contexts using a signal handler  
(something like a preemptive scheduler for userlevel threads). I've  
found several sources, which say that it's not a good idea to use  
setcontext or swapcontext in a signal handler. Nevertheless there also  
exists at least one sample code of such an preemptive scheduler, which  
seems to work well, at least on my machine (Ubuntu 8.04 with linux  
kernel 2.6.24-22): www.seas.upenn.edu/~cse381/context_demo.c

// [...]
static ucontext_t thread_context;
static ucontext_t scheduler_context;

int thread_finished;
int i;

static void simple_function(void) {
	// do nothing but counting
	for (i = 0; i < 1000000000; ++i) { }

	if (i == 1000000000) {
		printf("\n[Thread Function]\t1st counting worked fine...");
	} else {
		printf("\n[Thread Function]\tError: Counting didn't finished  
(%d)...", i);
	}

	thread_finished = 1;
}

static void other_function() {
	// thread_finished is a global variable, which is set to 1, if the  
thread function is finished
	while(thread_finished != 1) { swapcontext(&scheduler_context,  
&thread_context); }
}

static void signal_handler_function(int sig_nr, siginfo_t* info, void  
*old_context) {
	if (sig_nr == SIGPROF) {
		// saves the thread context
		thread_context = *((ucontext_t*) context);

		// swaps back to scheduler context
		setcontext(&scheduler_context);
	}
}
// [...]

I ran into the following problem which belongs to the code above. I  
interrupted simple_function several times by using a ITimer. But the  
for-loop doesn't finish successfully everytime. Often the if condition  
is false. But it does not cancel after the first signal is raised.  
I've found out that using the third parameter old_context for storing  
the old context is the reason. But I don't know why. So I thought  
there might be a problem in the kernel. Am I right? I was afraid to  
post the whole code, so I hope that this code snippet is enough.
I would appreciate if someone can give me a comment whether this  
strange behaviour is because of a wrong thinking of mine or because of  
a error in the kernel which needs to be fixed.

Thanks a lot!
Matthias

Re: Problem with ucontext_t struct in signal handler

[Glynn Clements]

XComp wrote:

> I want to switch between user contexts using a signal handler  
> (something like a preemptive scheduler for userlevel threads). I've  
> found several sources, which say that it's not a good idea to use  
> setcontext or swapcontext in a signal handler. Nevertheless there also  
> exists at least one sample code of such an preemptive scheduler, which  
> seems to work well, at least on my machine (Ubuntu 8.04 with linux  
> kernel 2.6.24-22): www.seas.upenn.edu/~cse381/context_demo.c
> 
> // [...]
> static ucontext_t thread_context;
> static ucontext_t scheduler_context;
> 
> int thread_finished;
> int i;
> 
> static void simple_function(void) {
> 	// do nothing but counting
> 	for (i = 0; i < 1000000000; ++i) { }
> 
> 	if (i == 1000000000) {
> 		printf("\n[Thread Function]\t1st counting worked fine...");
> 	} else {
> 		printf("\n[Thread Function]\tError: Counting didn't finished  (%d)...", i);
> 	}
> 
> 	thread_finished = 1;
> }
> 
> static void other_function() {
> 	// thread_finished is a global variable, which is set to 1, if the  thread function is finished
> 	while(thread_finished != 1) { swapcontext(&scheduler_context,  &thread_context); }
> }
> 
> static void signal_handler_function(int sig_nr, siginfo_t* info, void  *old_context) {
> 	if (sig_nr == SIGPROF) {
> 		// saves the thread context
> 		thread_context = *((ucontext_t*) context);
> 
> 		// swaps back to scheduler context
> 		setcontext(&scheduler_context);
> 	}
> }
> // [...]
> 
> I ran into the following problem which belongs to the code above. I  
> interrupted simple_function several times by using a ITimer. But the  
> for-loop doesn't finish successfully everytime. Often the if condition  
> is false.

It seems likely that either the registers or the stack (wherever "i"
is stored) is getting trashed. What is "i" in the cases where the test
fails?

> But it does not cancel after the first signal is raised.  
> I've found out that using the third parameter old_context for storing  
> the old context is the reason. But I don't know why.

Note that the old_context parameter to the signal handler won't be
pointing to any of your context "slots". When a signal occurs, the
current context will be saved in a ucontext_t on the current context's
stack, and the old_context argument will point to that.

It needs to be borne in mind that a ucontext_t isn't "the context"
itself. It's merely a structure for storing information about a
context, either for receiving information (e.g. getcontext) or
providing it (e.g. setcontext).

> So I thought there might be a problem in the kernel. Am I right?

I don't think so.

> I was afraid to post the whole code, so I hope that this code
> snippet is enough.

It would help if it was accurate (e.g. the signal handler refers to
"context" which isn't declared anywhere; is this supposed to be the
old_context parameter?) and more complete.

> I would appreciate if someone can give me a comment whether this  
> strange behaviour is because of a wrong thinking of mine or because of  
> a error in the kernel which needs to be fixed.

I suspect the former.

-- 
Glynn Clements <glynn@gclements.plus.com>

Re: Problem with ucontext_t struct in signal handler

[XComp]

On 18.12.2008, at 18:51, Glynn Clements wrote:

>
> XComp wrote:
>
>> I want to switch between user contexts using a signal handler
>> (something like a preemptive scheduler for userlevel threads). I've
>> found several sources, which say that it's not a good idea to use
>> setcontext or swapcontext in a signal handler. Nevertheless there  
>> also
>> exists at least one sample code of such an preemptive scheduler,  
>> which
>> seems to work well, at least on my machine (Ubuntu 8.04 with linux
>> kernel 2.6.24-22): www.seas.upenn.edu/~cse381/context_demo.c
>>
>> // [...]
>> static ucontext_t thread_context;
>> static ucontext_t scheduler_context;
>>
>> int thread_finished;
>> int i;
>>
>> static void simple_function(void) {
>> 	// do nothing but counting
>> 	for (i = 0; i < 1000000000; ++i) { }
>>
>> 	if (i == 1000000000) {
>> 		printf("\n[Thread Function]\t1st counting worked fine...");
>> 	} else {
>> 		printf("\n[Thread Function]\tError: Counting didn't finished   
>> (%d)...", i);
>> 	}
>>
>> 	thread_finished = 1;
>> }
>>
>> static void other_function() {
>> 	// thread_finished is a global variable, which is set to 1, if  
>> the  thread function is finished
>> 	while(thread_finished != 1) { swapcontext(&scheduler_context,   
>> &thread_context); }
>> }
>>
>> static void signal_handler_function(int sig_nr, siginfo_t* info,  
>> void  *old_context) {
>> 	if (sig_nr == SIGPROF) {
>> 		// saves the thread context
>> 		thread_context = *((ucontext_t*) context);
>>
>> 		// swaps back to scheduler context
>> 		setcontext(&scheduler_context);
>> 	}
>> }
>> // [...]
>>
>> I ran into the following problem which belongs to the code above. I
>> interrupted simple_function several times by using a ITimer. But the
>> for-loop doesn't finish successfully everytime. Often the if  
>> condition
>> is false.
>
> It seems likely that either the registers or the stack (wherever "i"
> is stored) is getting trashed. What is "i" in the cases where the test
> fails?

You find a sample output below. It seems to work in some cases. So you  
might be right with the stack. But why? I mean... there is nothing  
else but the for-loop. Or do I have the wrong idea of how the stack  
works?! Can a stack be overflown by a simple for-loop? Is an alternate  
signal stack needed? I thought it doesn't because the current stack  
should be big enough.

>
>
>> But it does not cancel after the first signal is raised.
>> I've found out that using the third parameter old_context for storing
>> the old context is the reason. But I don't know why.
>
> Note that the old_context parameter to the signal handler won't be
> pointing to any of your context "slots". When a signal occurs, the
> current context will be saved in a ucontext_t on the current context's
> stack, and the old_context argument will point to that.

I'm not sure whether I understood this correctly. What do you mean  
with slots?

>
>
> It needs to be borne in mind that a ucontext_t isn't "the context"
> itself. It's merely a structure for storing information about a
> context, either for receiving information (e.g. getcontext) or
> providing it (e.g. setcontext).
>
>> So I thought there might be a problem in the kernel. Am I right?
>
> I don't think so.

Because of the fact that ucontext_t is part of the C library? I  
thought that it might be a kernel issue because of the signal handling  
which is implemented in the kernel.
>
>
>> I was afraid to post the whole code, so I hope that this code
>> snippet is enough.
>
> It would help if it was accurate (e.g. the signal handler refers to
> "context" which isn't declared anywhere; is this supposed to be the
> old_context parameter?) and more complete.

I posted the complete code below. Sorry for the little mistake.

>
>
>> I would appreciate if someone can give me a comment whether this
>> strange behaviour is because of a wrong thinking of mine or because  
>> of
>> a error in the kernel which needs to be fixed.
>
> I suspect the former.
>
> -- 
> Glynn Clements <glynn@gclements.plus.com>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-c- 
> programming" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Sorry, the context variable in the signal handler should be called  
old_context. Here you have the complete code sample:

#include <stdlib.h>
#include <unistd.h>

// signal handling
#include <signal.h>
// context switch
#include <ucontext.h>
// timer
#include <sys/time.h>

// upper bound of the for loops inside thread_function
#define UPPER_BOUND 100000000
// number of seconds for setting the interval used by the timer
#define QUANTUM_SEC 0
// number of microseconds for setting the interval used by the timer  
(0 - 99999)
#define QUANTUM_MICRO_SEC 100000

#define STACKSIZE 4096

static ucontext_t thread_context;
static ucontext_t scheduler_context;

int thread_finished;
int i;

static void thread_function(void) {
        printf("\n[Thread Function]\tFunction starts...");

        for (i = 0; i < UPPER_BOUND; ++i) {
                // do nothing but counting
        }

        if (i == UPPER_BOUND) {
                printf("\n[Thread Function]\t1st counting worked  
fine...");
        } else {
                printf("\n[Thread Function]\tError: 1st counting  
didn't finished (%d)...", i);
        }

        thread_finished = 1;

        printf("\n[Thread Function]\tFunction finishes...");
}

static void scheduler_function() {

        printf("\n[Scheduler Function]\tScheduler starts...");

        // thread_finished is a global variable, which is set to 1, if  
the thread function is finished
        while (thread_finished != 1) {
                swapcontext(&scheduler_context, &thread_context);
        }

        printf("\n[Scheduler Function]\tScheduler finishes...");
}

static void signal_handler_function(int sig_nr, siginfo_t* info, void  
*old_context) {

        if (sig_nr == SIGPROF) {
                printf("\n[Signal Handler]\tSIGPROF was raised at  
%d...", i);

                // saves the thread context
                thread_context = *((ucontext_t*) old_context);

                // swaps back to scheduler context
                setcontext(&scheduler_context);
        } else {
                printf("\n[Signal Handler]\tA different signal was  
raised...");
                return;
        }

}

int main(int argc, char **argv) {

        printf("\n[Main Function]\t\tProgram starts...");

        thread_finished = 0;

        char thread_stack[STACKSIZE];
        char scheduler_stack[STACKSIZE];

        // initializing scheduler context
        if (getcontext(&scheduler_context) == 0) {
                scheduler_context.uc_stack.ss_sp = scheduler_stack;
                scheduler_context.uc_stack.ss_size =  
sizeof(scheduler_stack);
                scheduler_context.uc_stack.ss_flags = 0;
                scheduler_context.uc_link = NULL;
                printf("\n[Main Function]\t\tscheduler_context was  
initialized...");
        } else {
                printf("\n[Main Function]\t\tError while initializing  
scheduler_context...");
                exit(-1);
        }

        // initializing thread context
        if (getcontext(&thread_context) == 0) {
                thread_context.uc_stack.ss_sp = thread_stack;
                thread_context.uc_stack.ss_size = sizeof(thread_stack);
                thread_context.uc_stack.ss_flags = 0;
                thread_context.uc_link = &scheduler_context;
                printf("\n[Main Function]\t\tthread_context was  
initialized...");
        } else {
                printf("\n[Main Function]\t\tError while initializing  
thread_context...");
                exit(-1);
        }

        // sets the signal handler for swapping to the scheduler context
        struct sigaction scheduling_interuption_handler;
        scheduling_interuption_handler.sa_sigaction =  
signal_handler_function;
        scheduling_interuption_handler.sa_flags = SA_RESTART |  
SA_SIGINFO;
        sigemptyset(&scheduling_interuption_handler.sa_mask);
        if (sigaction(SIGPROF, &scheduling_interuption_handler, NULL)  
== -1) {
                printf("\n[Main Function]\t\tAn error occurred while  
initializing the signal handler for swapping to the scheduler  
context...");
                exit(-1);
        }

        // sets the timer which sends SIGPROF periodically
        struct itimerval timeslice;
        timeslice.it_value.tv_sec = QUANTUM_SEC;
        timeslice.it_value.tv_usec = QUANTUM_MICRO_SEC;
        timeslice.it_interval = timeslice.it_value;

        if (setitimer(ITIMER_PROF, &timeslice, NULL) == 0) {
                printf("\n[Main Function]\t\tThe timer was  
initialized...");
        } else {
                printf("\n[Main Function]\t\tAn error occurred while  
executing setitimer...");
                exit(-1);
        }

        // sets the thread function
        makecontext(&thread_context, thread_function, 0);

        // this function handles the swapping part
        scheduler_function();

        printf("\n[Main Function]\t\tProgram finishes...");
}

----

The possible output for these settings is:

[Main Function]                Program starts...
[Main Function]                scheduler_context was initialized...
[Main Function]                thread_context was initialized...
[Main Function]                The timer was initialized...
[Scheduler Function]        Scheduler starts...
[Thread Function]        Function starts...
[Signal Handler]        SIGPROF was raised at 30811962...
[Signal Handler]        SIGPROF was raised at 62309593...
[Signal Handler]        SIGPROF was raised at 93262549...
[Thread Function]        1st counting worked fine...
[Signal Handler]        SIGPROF was raised at 24910729...
[Signal Handler]        SIGPROF was raised at 55303135...
[Signal Handler]        SIGPROF was raised at 86507356...
[Thread Function]        Error: 2nd counting didn't finished  
(86507356)...
[Thread Function]        Function finishes...
[Scheduler Function]        Scheduler finishes...
[Main Function]                Program finishes...

So you see that the for-loops quits right after a signal was caught. I  
am really in a dead end now. Do you see any parts of my code, which  
might be wrong?

Re: Problem with ucontext_t struct in signal handler

[Glynn Clements]

XComp wrote:

> > It seems likely that either the registers or the stack (wherever "i"
> > is stored) is getting trashed. What is "i" in the cases where the test
> > fails?
> 
> You find a sample output below. It seems to work in some cases. So you  
> might be right with the stack. But why? I mean... there is nothing  
> else but the for-loop. Or do I have the wrong idea of how the stack  
> works?! Can a stack be overflown by a simple for-loop? Is an alternate  
> signal stack needed? I thought it doesn't because the current stack  
> should be big enough.

It isn't an issue with it overflowing. I was assuming that the loop
terminated because "i" was getting set to a value > 1000000000, which
implies it was being corrupted.

Your output indicates that isn't actually the case; it appears that
the branch back to the beginning of the loop is being skipped.

> >> But it does not cancel after the first signal is raised.
> >> I've found out that using the third parameter old_context for storing
> >> the old context is the reason. But I don't know why.
> >
> > Note that the old_context parameter to the signal handler won't be
> > pointing to any of your context "slots". When a signal occurs, the
> > current context will be saved in a ucontext_t on the current context's
> > stack, and the old_context argument will point to that.
> 
> I'm not sure whether I understood this correctly. What do you mean  
> with slots?

Well, I was assuming that your code vaguely resembles the URL you
gave:

	http://www.seas.upenn.edu/~cse381/context_demo.c

By "slots", I was refering to:

	ucontext_t contexts[NUMCONTEXTS];   /* store our context info */

OTOH, if the only contexts involved are these two:

>> static ucontext_t thread_context;
>> static ucontext_t scheduler_context;

then those are the "slots".

In any case, I don't think that you can rely upon the old_context
being usable once you leave the signal handler. Even if you copy the
contents, the uc_mcontext field may not be meaningful outside of the
handler.

> >> So I thought there might be a problem in the kernel. Am I right?
> >
> > I don't think so.
> 
> Because of the fact that ucontext_t is part of the C library? I  
> thought that it might be a kernel issue because of the signal handling  
> which is implemented in the kernel.

No, I'm just saying that a failure for context-switching to behave as
you expect with regard to signal handlers isn't a reason to assume a
bug. The standards are fairly vague about how signal handling and
contexts interact.

However, it's also true that the context handling is almost purely in
user-space. The only system call involved is sigprocmask() to get the
signal mask in getcontext() and set it in setcontext()/swapcontext().

> So you see that the for-loops quits right after a signal was caught. I  
> am really in a dead end now. Do you see any parts of my code, which  
> might be wrong?

Probably the line:

	thread_context = *((ucontext_t*) old_context);

I wouldn't assume that you can do anything with old_context other than
pass it to setcontext() or swapcontext() within the signal handler. In
the absence of documentation to the contrary, I wouldn't assume that
it can be stored and used outside of the handler.

It's quite possible that the context_t passed to a signal handler and
a context created by calling getcontext() within a signal handler have
some "magic" within the mcontext_t to unwind the signal handling

I note that the code at the above URL doesn't use the old_context
parameter at all.

-- 
Glynn Clements <glynn@gclements.plus.com>

Re: Problem with ucontext_t struct in signal handler

[XComp]

> XComp wrote:
>
>>> It seems likely that either the registers or the stack (wherever "i"
>>> is stored) is getting trashed. What is "i" in the cases where the  
>>> test
>>> fails?
>>
>> You find a sample output below. It seems to work in some cases. So  
>> you
>> might be right with the stack. But why? I mean... there is nothing
>> else but the for-loop. Or do I have the wrong idea of how the stack
>> works?! Can a stack be overflown by a simple for-loop? Is an  
>> alternate
>> signal stack needed? I thought it doesn't because the current stack
>> should be big enough.
>
> It isn't an issue with it overflowing. I was assuming that the loop
> terminated because "i" was getting set to a value > 1000000000, which
> implies it was being corrupted.
>
> Your output indicates that isn't actually the case; it appears that
> the branch back to the beginning of the loop is being skipped.
>
>>>> But it does not cancel after the first signal is raised.
>>>> I've found out that using the third parameter old_context for  
>>>> storing
>>>> the old context is the reason. But I don't know why.
>>>
>>> Note that the old_context parameter to the signal handler won't be
>>> pointing to any of your context "slots". When a signal occurs, the
>>> current context will be saved in a ucontext_t on the current  
>>> context's
>>> stack, and the old_context argument will point to that.
>>
>> I'm not sure whether I understood this correctly. What do you mean
>> with slots?
>
> Well, I was assuming that your code vaguely resembles the URL you
> gave:
>
> 	http://www.seas.upenn.edu/~cse381/context_demo.c
>
> By "slots", I was refering to:
>
> 	ucontext_t contexts[NUMCONTEXTS];   /* store our context info */
>
> OTOH, if the only contexts involved are these two:
>
>>> static ucontext_t thread_context;
>>> static ucontext_t scheduler_context;
>
> then those are the "slots".
>
> In any case, I don't think that you can rely upon the old_context
> being usable once you leave the signal handler. Even if you copy the
> contents, the uc_mcontext field may not be meaningful outside of the
> handler.

So you think that solving the problem in this way will lead me in a  
dead end?
But I am confused because of the fact that it works sometimes.  
Shouldn't it work the whole time or at no time only?

>
>>>> So I thought there might be a problem in the kernel. Am I right?
>>>
>>> I don't think so.
>>
>> Because of the fact that ucontext_t is part of the C library? I
>> thought that it might be a kernel issue because of the signal  
>> handling
>> which is implemented in the kernel.
>
> No, I'm just saying that a failure for context-switching to behave as
> you expect with regard to signal handlers isn't a reason to assume a
> bug. The standards are fairly vague about how signal handling and
> contexts interact.
>
> However, it's also true that the context handling is almost purely in
> user-space. The only system call involved is sigprocmask() to get the
> signal mask in getcontext() and set it in setcontext()/swapcontext().
>
>> So you see that the for-loops quits right after a signal was  
>> caught. I
>> am really in a dead end now. Do you see any parts of my code, which
>> might be wrong?
>
> Probably the line:
>
> 	thread_context = *((ucontext_t*) old_context);
>
> I wouldn't assume that you can do anything with old_context other than
> pass it to setcontext() or swapcontext() within the signal handler. In
> the absence of documentation to the contrary, I wouldn't assume that
> it can be stored and used outside of the handler.

I've tested it. If I use the old_context with setcontext() and go  
directly back to the thread in this case (the scheduler_function would  
be ignored), the behaviour is the same. Sometimes the loop finishes,  
sometimes not.

>
>
> It's quite possible that the context_t passed to a signal handler and
> a context created by calling getcontext() within a signal handler have
> some "magic" within the mcontext_t to unwind the signal handling
>
> I note that the code at the above URL doesn't use the old_context
> parameter at all.
>
> -- 
> Glynn Clements <glynn@gclements.plus.com>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-c- 
> programming" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Problem with ucontext_t struct in signal handler

[Glynn Clements]

XComp wrote:

> > In any case, I don't think that you can rely upon the old_context
> > being usable once you leave the signal handler. Even if you copy the
> > contents, the uc_mcontext field may not be meaningful outside of the
> > handler.
> 
> So you think that solving the problem in this way will lead me in a  
> dead end?
> But I am confused because of the fact that it works sometimes.  
> Shouldn't it work the whole time or at no time only?

Intermittent faults are common with multi-threaded programming.

If a register is getting trashed, that will matter if you actually
need the register's value, but not if you're about to store a new
value there.

Looking at the disassembly:

	movl	$0, i
.L2:
	cmpl	$99999999, i
	jg	.L3
	incl	i
	jmp	.L2
.L3:

with absolute addresses:

 8048606:	movl   $0, i
 804860d:	
 8048610:	cmpl   $99999999, i
 8048617:	
 804861a:	jg     0x8048624
 804861c:	incl   i
 8048622:	jmp    0x8048610
 8048624:	...

my guess is that the flags aren't being saved, so when the signal
occurs between the "cmpl" and "jg" instructions, the sign flag is
being cleared, triggering the jump prematurely.

Adding the following:

	printf("\nold_context.uc_mcontext.gregs[REG_EFL] = %08x\n",
		 ((ucontext_t*) old_context)->uc_mcontext.gregs[REG_EFL]);
	printf("\nold_context.uc_mcontext.gregs[REG_IPL] = %08x\n",
		 ((ucontext_t*) old_context)->uc_mcontext.gregs[REG_EIP]);
shows the following:

	[Signal Handler]	SIGPROF was raised at  21883912...
	old_context.uc_mcontext.gregs[REG_EFL] = 00000203
	old_context.uc_mcontext.gregs[REG_IPL] = 08048622
	[Signal Handler]	SIGPROF was raised at  43508833...
	old_context.uc_mcontext.gregs[REG_EFL] = 00000203
	old_context.uc_mcontext.gregs[REG_IPL] = 08048610
	[Signal Handler]	SIGPROF was raised at  65140032...
	old_context.uc_mcontext.gregs[REG_EFL] = 00000297
	old_context.uc_mcontext.gregs[REG_IPL] = 0804861a
	[Thread Function]	Error: 1st counting  didn't finished (65140032)...

More precisely, in the cases where it fails, EIP is always 0x0804861a
(the "jg" instruction), and EFL always has the sign flag (0x80) set.
In the cases where it succeeds:

	[Signal Handler]	SIGPROF was raised at  22107659...
	old_context.uc_mcontext.gregs[REG_EFL] = 00000297
	old_context.uc_mcontext.gregs[REG_IPL] = 0804861c
	[Signal Handler]	SIGPROF was raised at  43727791...
	old_context.uc_mcontext.gregs[REG_EFL] = 00000207
	old_context.uc_mcontext.gregs[REG_IPL] = 08048610
	[Signal Handler]	SIGPROF was raised at  65357434...
	old_context.uc_mcontext.gregs[REG_EFL] = 00000203
	old_context.uc_mcontext.gregs[REG_IPL] = 08048610
	[Signal Handler]	SIGPROF was raised at  86987791...
	old_context.uc_mcontext.gregs[REG_EFL] = 00000283
	old_context.uc_mcontext.gregs[REG_IPL] = 0804861c
	[Thread Function]	1st counting worked  fine...

the above scenario never occurs.

It appears that you can't even use old_context to jump out of the
handler. In fact, the setcontext(2) manual page says:

       If  the	context	 was  obtained by a call to a signal handler, then old
       standard text says that "program execution continues with  the  program
       instruction following the instruction interrupted by the signal".  How-
       ever, this sentence was removed in SUSv2, and the  present  verdict  is
       "the result is unspecified".

I suspect that by the time that the kernel passes control to
user-space, the flags which libc saves are no longer correct. If
that's the case, the only solution is to return from the signal
handler back into the kernel, and allow the kernel to restore the
registers.

Note that the original code doesn't have this problem. The
swapcontext() which jumps out of the signal handler into user-space
jumps to a context which has just been created by makecontext() to
invoke scheduler().

And scheduler() never resumes a saved user-space context. The context
which it passes to setcontext() is either the original context created
by makecontext(), or it is the signal-space context which was saved
from within the signal handler by the swapcontext(). In the latter
case, the task resumes within the signal handler, from which it
returns normally.

So the code never attempts to resume a saved user-space context from
within the signal handler. From the perspective of the thread1() or
thread2() function, it gets a signal, enters the signal handler, then
(after all of the other tasks have had a turn) leaves the signal
handler.

-- 
Glynn Clements <glynn@gclements.plus.com>