Re: 2.6.24-rc8-rt1: Strange latencies on mpc5200 powerpc - RCU issue?

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Wed, Jan 30, 2008 at 09:18:49AM +0100, Wolfgang Grandegger wrote:
> Paul E. McKenney wrote:
> > On Tue, Jan 29, 2008 at 02:38:04PM +0100, Wolfgang Grandegger wrote:
> >> Luotao Fu wrote:
> >>> Hi,
> >>>
> >>> Wolfgang Grandegger wrote:
> >>> ..........
> >>>> Do you still get high latencies with:
> >>>>
> >>>>   CONFIG_PREEMPT_RCU_BOOST=y
> >>>>   CONFIG_RCU_TRACE=y
> >>>>   CONFIG_NO_HZ is not set
> >>>>
> >>>> With this setting I have not yet realized latencies > 150us. Could you
> >>>> please give it a try? If I change one of the parameters above, latency
> >>>> increases in short time.
> >>> I played through some combination of the RCU options and can back your
> >>> observation this time: With the rcu Tracer or the priority boost turned
> >>> off I also could measure reliably extraordinory high latencies. If they
> >>> are both turned on, no high latencies could be measured. Turning on the
> >>> dynamic ticker however doesn't seem to cause high latencies during my
> >>> test runs. Seemed like an rcu issue here.
> >> I'm just making a long test run on my TQM5200 module with my good
> >> settings. After more than 4.5 hours under load, cyclictest shows a
> >> maximum latency of 177 us. I'm going to re-check the effect of CONFIG_NO_HZ.
> >>
> >>> Further such results only appear if the target board is booted with
> >>> nfsrootfs. (As I already have mentioned several times before), which
> >>> leads my suspection to rcu usage in nfs implementation. In this case
> >>> this problem might even be platformindependent. I'd have to do some
> >>> tests on one of our arm boards later to test this. Since there're no
> >>> reports like this for other architecture as powerpc till now, I doubt
> >>> quite if this is verifiable.
> >> It's also my suspicion that the high latencies are related to the RCU
> >> usage in the network layer, where it's heavily used. What is really
> >> wired is that switching off CONFIG_RCU_TRACE has a negative impact on
> >> the latency. As I see it, it just adds some trace points, but I might
> >> have missed something.
> > 
> > I would expect that CONFIG_RCU_TRACE=n (as in "no" rather than "module")
> > would have low latencies rather than high latencies.  So I am quite
> > surprised by your result.  I will dig into this more.
> 
> Thanks a lot. To be clear. I need "CONFIG_PREEMPT_RCU_BOOST=y" *and*
> "CONFIG_RCU_TRACE=y" to achieve reasonable latencies below 180us. With
> CONFIG_PREEMPT_RCU_BOOST or CONFIG_RCU_TRACE not set or
> CONFIG_RCU_TRACE=m is rmeasure latencies up to 600us within a minute or so.

OK, thank you for the confirmation.

The large latencies were from cyclictest, correct?  Did other tests
also show these latencies?  In either case, could you please send the
exact command line you used to run the test?

							Thanx, Paul