Re: NOHZ: local_softirq_pending

[Iratxo Pichel Ortiz <ipichel@albentia.com>]

Thomas Gleixner wrote:
> On Tue, 16 Jun 2009, Iratxo Pichel Ortiz wrote:
>   
>> I have tried disabling the CONFIG_NOHZ kernel option. Of course the trace is
>> gone, but the weird behavior is still there. When I run my software without
>> load from the network, the main task of the system experiences runtimes of
>> about 700us. When I load the system, there are latencies of 50700us, so the
>> 50ms delay is again there, and again the time when the task finishes is always
>> X.296, 1 jiffy after the "NOHZ: pending..." was shown with CONFIG_NOHZ
>> enabled.
>>     
>
> What kind of system is this ? 
>   
It is an IXP425 custom board, mainly equivalent to the Intel's IXDP425 
development board, so it is ARM (xscale) architecture.

The problem is perfectly repeatable without the need to have other code 
loaded in the system than the standard kernel, but I first detected it 
with a custom development I am working on, that run a RT task (prio 98) 
every 2.5ms to schedule a radio link.

To detect the problem, with my attached .config, I just need to 
massively ping the ethernet interface and I start to see "NOHZ: 
pending..." every second or couple of seconds from jiffie XXX950 to 
XXX999 (HZ 1000). The SOFTIRQ pendings that I have seen pending are 
almost all of them: RCU, NETTX, NETRX, TIMER...

Having tested this I first thought that it could be related with 
ixp4xx_eth driver but I used a different interface and driver and the 
problem is also there. I have also, at least apparently, discarded the 
bridging code and vlan code, that I was using at first.
> Can you please check whether you can reproduce with NOHZ=y and function
> tracer enabled ?
>
>   
I have already tested with NOHZ, and the behavior is exactly the same, 
there is a delay of about 50ms that always finishes at trace  with 
timestamp XXX.296 seconds buf, of course, the "NOHZ: ..." trace is not 
shown.

I have not tried this with the function tracer enabled, I will build the 
kernel again and enable this tracer. Which function should I trace? Any 
hint about this? I have a couple of systems running so I can make any 
trace you would like. I will try to figure out something and give 
feedback later.

I have also tested the system with the latest 2.29.5-rt21 kernel and RT 
patch.

Thanks a lot,

    Iratxo.
> That would be great.
>
> Thanks,
>
> 	tglx
>
>   


-- 
Iratxo Pichel Ortiz
Software Development Manager

Albentia Systems S.A.
http://www.albentia.com

Tel: +34 914400567
Cel: +34 663808405
Fax: +34 914400569

C\Margarita Salas 22
Parque Tecnológico de Leganés
Leganés (28918)
Madrid
Spain

--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html