Re: 2.6.24-rc8-rt1: Strange latencies on mpc5200 powerpc - RCU issue?

[Wolfgang Grandegger <wg@grandegger.com>]

Hi Fun,

Luotao Fu wrote:
> Hi,
> 
> I found some time and played with my mpc5200B Board again on this issue
> (2.6.25.8-rt7, same test environment)
> 
> On Sun, Jul 06, 2008 at 11:41:28AM +0200, Wolfgang Grandegger wrote:
> ....
>> Yep, in 2.6.25.8-rt7 trace_mark() is not used any more, maybe for that  
>> reason. Nevertheless, this "bug" kept the latency on my MPC5200 box low.
>> I will try with the old tracer to understand what the real cause of the  
>> high latencies is (with CONFIG_RCU_TRACE disabled).
>>
> As you said, in 2.6.25.8-rt7 there're rcu_trace_boost_unboost_called() and
> rcu_trace_boost_unboosted() are used instead of trace_mark. I still haven't had
> time to take a closer look waht these routines exactly do. Seemed however to me,
> that they possibly do the same thing. I played a little bit around with
> the RCU_TRACE and RCU_BOOST Option. Also I tried to comment out the
> rcu_trace_boost_unboost_called() and rcu_trace_boost_unboosted() calls. My Tests
> last appr. 30 Minutes with non-rt Payload "while [ 1 ]; do ls /;done". None of
> my test showed extraordinary results. My worstcase values stay at about 220 us
> with all my test combinations. I will attache a config with RCU_BOOST and
> RCU_TRACE turned off to this mail, which provide a worst case value at 223 us
> after about 40 Minutes test run on my board.

I think I understood why CONFIG_RCU_TRACEin 2.6.24-rt helped to keep 
latencies low. See:

   ttp://marc.info/?l=linux-rt-users&m=121499677026236&w=4

But as Steven pointed out, the preempt_enable_noresched at that place 
might be illegal, anyway. In 2.6.25-rt, trace_mark() is no longer used 
and I do not see an improvement with CONFIG_RCU_TRAC any more. Latencies 
go up to 600us. I also tried with CONFIG_RCU_BOOST disabled, but it did 
not improve the latencies. Actually, I still measure significantly 
different latencies with 2.6.24.4-rt4, 2.6.24.7-rt14 and 2.6.25.8-rt7, 
which is quite frustrating. As you seem not be able to reproduce my high 
latencies, I wonder if there are some toolchain or glibc related issues.

Using ftrace to localized the latency spot would be the next step. But 
it's not yet available for PowerPC. Looking into that now.

> Are you using the FEC all a pci network card?

I use only the FEC. There is no PCI device.

Wolfgang.