tasklet latency and system calls on mips

tasklet latency and system calls on mips

[Sirotkin, Alexander]

Hello dearest all.

I have a question regarding tasklets on MIPS. I suspect that there is a
bug in generic MIPS kernel, but I'm not sure yet.

Linux kernel has a couple of so called "checkpoints" when the system
should check if there are tasklets to
run and run them in the following way :

if (softirq_pending(cpu))
                    do_softirq();

One of these places is at the end of interrupt handler (do_IRQ()),
however this is not the only place. I was under
impression that this code should be called after system call too. The
caveat here is that on MIPS (contrary to
other architectures, such as x86) system call is not an interrupt (it's
a different exception) and has completely
different handler. So in x86 it is sufficient to call

if (softirq_pending(cpu))
                    do_softirq();

at the end of do_IRQ because do_IRQ handles system call too, but on MIPS
it is not. Therefore I believe
these lines should be added to the end of sys_syscall function on MIPS.

What do you think ?

P.S. The whole issue started when we noticed that user process making
many system calls has very
significant impact on device drivers running in tasklet mode and no
impact whatsoever on device
drivers running in interrupt mode.

Re: tasklet latency and system calls on mips

[Jun Sun]

On Wed, Aug 13, 2003 at 04:46:04PM +0300, Sirotkin, Alexander wrote:
> Hello dearest all.
> 
> I have a question regarding tasklets on MIPS. I suspect that there is a
> bug in generic MIPS kernel, but I'm not sure yet.
> 
> Linux kernel has a couple of so called "checkpoints" when the system
> should check if there are tasklets to
> run and run them in the following way :
> 
> if (softirq_pending(cpu))
>                     do_softirq();
> 
> One of these places is at the end of interrupt handler (do_IRQ()),
> however this is not the only place. I was under
> impression that this code should be called after system call too. The
> caveat here is that on MIPS (contrary to
> other architectures, such as x86) system call is not an interrupt (it's
> a different exception) and has completely
> different handler. So in x86 it is sufficient to call
> 
> if (softirq_pending(cpu))
>                     do_softirq();
> 
> at the end of do_IRQ because do_IRQ handles system call too, but on MIPS
> it is not. Therefore I believe
> these lines should be added to the end of sys_syscall function on MIPS.
> 
> What do you think ?
> 

softirq/tasklet/bottom_half/etc should only be raised from interrupt
context.  Checking at the end of do_IRQ should be good enough.

One possible mistake in MIPS porting is that if the board uses its private
time interrupt routine poeple may forget to put the above two lines
at the end.  Check against that.

> P.S. The whole issue started when we noticed that user process making
> many system calls has very
> significant impact on device drivers running in tasklet mode 

What kind of impact?  On i386? Or on MIPS?

Jun

Re: tasklet latency and system calls on mips

[Sirotkin, Alexander]

[-- Attachment #1: Type: text/plain, Size: 3035 bytes --]

Hi.

See my comments below.

Jun Sun wrote:


On Wed, Aug 13, 2003 at 04:46:04PM +0300, Sirotkin, Alexander wrote:

  

Hello dearest all.



I have a question regarding tasklets on MIPS. I suspect that there is a

bug in generic MIPS kernel, but I'm not sure yet.



Linux kernel has a couple of so called "checkpoints" when the system

should check if there are tasklets to

run and run them in the following way :



if (softirq_pending(cpu))

                    do_softirq();



One of these places is at the end of interrupt handler (do_IRQ()),

however this is not the only place. I was under

impression that this code should be called after system call too. The

caveat here is that on MIPS (contrary to

other architectures, such as x86) system call is not an interrupt (it's

a different exception) and has completely

different handler. So in x86 it is sufficient to call



if (softirq_pending(cpu))

                    do_softirq();



at the end of do_IRQ because do_IRQ handles system call too, but on MIPS

it is not. Therefore I believe

these lines should be added to the end of sys_syscall function on MIPS.



What do you think ?



    



softirq/tasklet/bottom_half/etc should only be raised from interrupt

context.  Checking at the end of do_IRQ should be good enough.

  

On mips interrupt is an exception and system call is a different
exception. Different exceptions has different exception handlers,
at least that's what I was able to figure from entry.S file. So the
system call does not go through do_IRQ and do_softirq
is not called.





One possible mistake in MIPS porting is that if the board uses its
private

time interrupt routine poeple may forget to put the above two lines

at the end.  Check against that.

  

In our kernel port we do have these lines in the timer interrupt, that
is not a problem. 



P.S. The whole issue started when we noticed that user process making

many system calls has very

significant impact on device drivers running in tasklet mode 

    



What kind of impact?  On i386? Or on MIPS?



  

Jun


The impact is that if our driver works in tasklet mode then some user
mode application making system calls
causes some (although quite small) packet loss. It does not happen if we
don't use tasklet and do everything in the
ISR.

I suspect that what happens is as follows :

system call arrives and while it's being processed and interrupt to one
of the drivers arrives. This interrupt 
schedules a tasklet which however is not executed after the system call
finishes, only after the next timer
interrupt which causes up to 10 ms latency (not all the time, only when
somebody makes a system call).

It only happens on MIPS. There is no easy way to check this on x86.
-- 

Alexander Sirotkin

SW Engineer



Texas Instruments

Broadband Communications Israel (BCIL)

Tel:  +972-9-9706587

________________________________________________________________________

"Those who do not understand Unix are condemned to reinvent it, poorly."

      -- Henry Spencer 

[-- Attachment #2: Type: text/html, Size: 3855 bytes --]

Re: tasklet latency and system calls on mips

[Jun Sun]

On Thu, Aug 14, 2003 at 12:17:52PM +0300, Sirotkin, Alexander wrote:
> 
> I suspect that what happens is as follows :
> 
> system call arrives and while it's being processed and interrupt to one
> of the drivers arrives. This interrupt 
> schedules a tasklet which however is not executed after the system call
> finishes, 

The tasklet should be executed at the return of interrupt handling.
If not, there is a bug.


> only after the next timer
> interrupt which causes up to 10 ms latency (not all the time, only when
> somebody makes a system call).
> 

BTW, make sure tasklet_schedule() is indeed called in an interrupt handler.
I am not sure why will happen otherwise.

If you suspect it is a bug, you can easily trace them.  You may my
little tracing tool useful,

	http://linux.junsun.net/patches/generic/experimental/030716.a-jstrace.patch

Jun

Re: tasklet latency and system calls on mips

[Sirotkin, Alexander]

[-- Attachment #1: Type: text/plain, Size: 1778 bytes --]



Jun Sun wrote:


On Thu, Aug 14, 2003 at 12:17:52PM +0300, Sirotkin, Alexander wrote:

  

I suspect that what happens is as follows :



system call arrives and while it's being processed and interrupt to one

of the drivers arrives. This interrupt 

schedules a tasklet which however is not executed after the system call

finishes, 

    



The tasklet should be executed at the return of interrupt handling.

If not, there is a bug.

  

I have a feeling that we are going in circles. Tasklets are executed at
the return of interrupt handler.
However, I suspect that this is not enough. On mips (contrary to x86),
system call is NOT an interrupt.
It's a different exception with different handler. Therefore I suspect
that tasklets are NOT called at 
the end of system call exception handler (which is a different handler,
not do_IRQ).






  

only after the next timer

interrupt which causes up to 10 ms latency (not all the time, only when

somebody makes a system call).



    



BTW, make sure tasklet_schedule() is indeed called in an interrupt
handler.

I am not sure why will happen otherwise.



If you suspect it is a bug, you can easily trace them.  You may my

little tracing tool useful,

I can try to trace it, I just wanted to ensure that what I was saying
makes sense. See above.






	
http://linux.junsun.net/patches/generic/experimental/030716.a-jstrace.pa
tch
<http://linux.junsun.net/patches/generic/experimental/030716.a-jstrace.p
atch> 



Jun

  


-- 

Alexander Sirotkin

SW Engineer



Texas Instruments

Broadband Communications Israel (BCIL)

Tel:  +972-9-9706587

________________________________________________________________________

"Those who do not understand Unix are condemned to reinvent it, poorly."

      -- Henry Spencer 

[-- Attachment #2: Type: text/html, Size: 2505 bytes --]

Re: tasklet latency and system calls on mips

[Jun Sun]

On Mon, Aug 18, 2003 at 06:29:52PM +0300, Sirotkin, Alexander wrote:
> 
> 
> 
> The tasklet should be executed at the return of interrupt handling.
> 
> If not, there is a bug.
> 
>   
> 
> I have a feeling that we are going in circles. Tasklets are executed at
> the return of interrupt handler.
> However, I suspect that this is not enough. 

If you follow this, plus "tasklet_schedule() is indeed called in an interrupt
handler", you will should see "executing tasklet at the return of interrupt
handler"  is _obviously_ enough.

> On mips (contrary to x86),
> system call is NOT an interrupt.
> It's a different exception with different handler. Therefore I suspect
> that tasklets are NOT called at 
> the end of system call exception handler (which is a different handler,
> not do_IRQ).
>

... which is fine, if you can follow the above logic.

Jun