Need help debugging RT-preempt patch

Need help debugging RT-preempt patch

[Tim Bird]

I've applied patch-2.6.21-rt1 to a 2.6.21 kernel I'm using.
With the patch applied, but no RT-Preempt options enabled,
I'm getting network failures (eth0 transmit timeouts) on
an OMAP board.

I'm not sure what the best method is to debug this.  Should
I turn on the latency tracer?  Should I post my configuration?
Should I try to break up the patch and apply changes in small doses?
I'm not familiar with the network code, or with the RT code (yet),
so for now I'm kind of stumbling in the dark.

Note that I have other patches applied, for platform support
for the OMAP.

I believe ARM (or OMAP) is not supported by the RT-preempt patch
at the moment.  But I'd like to carry the patches in my kernel even
if the options for RT-preempt are off, because I want to experiment
with RT-preempt on other platforms (x86 and MIPS).

Any suggestions for where to start to isolate this would
be appreciated.

Thanks,
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Electronics
=============================

Re: Need help debugging RT-preempt patch

[Daniel Walker]

On Mon, 2007-05-07 at 11:47 -0700, Tim Bird wrote:
> I've applied patch-2.6.21-rt1 to a 2.6.21 kernel I'm using.
> With the patch applied, but no RT-Preempt options enabled,
> I'm getting network failures (eth0 transmit timeouts) on
> an OMAP board.

Are you sure you don't have HardIRQS in Threads enabled? Also are you
sure the kernel boots without the rt patch applied.

> I'm not sure what the best method is to debug this.  Should
> I turn on the latency tracer?  Should I post my configuration?
> Should I try to break up the patch and apply changes in small doses?
> I'm not familiar with the network code, or with the RT code (yet),
> so for now I'm kind of stumbling in the dark.
> 
> Note that I have other patches applied, for platform support
> for the OMAP.
> 
> I believe ARM (or OMAP) is not supported by the RT-preempt patch
> at the moment.  But I'd like to carry the patches in my kernel even
> if the options for RT-preempt are off, because I want to experiment
> with RT-preempt on other platforms (x86 and MIPS).

ARM is supported, but specific boards are hit or miss.. 

I've had OMAP working _in the past_ with this patch applied,

http://lkml.org/lkml/2006/12/10/59

There was also a patch released on one of the OMAP lists a while back
from Dirk Behme to get OMAP working on a mor recent rt release..

Daniel

Re: Need help debugging RT-preempt patch

[Tim Bird]

Daniel Walker wrote:
> Are you sure you don't have HardIRQS in Threads enabled? Also are you
> sure the kernel boots without the rt patch applied.

Yes on both counts.  I double-checked the configs.  I'm using quilt
and the RT patch is at the end of my series, so it was easy to
pop the patch and try again (module .config fixups).

I may take a few more blind stabs at different configs before
I go diving into the networking code trying to trace something.
(or before starting to bisect the patch).  Bisecting the patch
will be a bit painful since it's currently an all-in-one.

> 
> ARM is supported, but specific boards are hit or miss.. 
> 
> I've had OMAP working _in the past_ with this patch applied,
> 
> http://lkml.org/lkml/2006/12/10/59
> 
> There was also a patch released on one of the OMAP lists a while back
> from Dirk Behme to get OMAP working on a mor recent rt release..

OK.  This is REALLY helpful info.  I'll take a look.

Thanks very much!!
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Electronics
=============================

Re: Need help debugging RT-preempt patch

[Daniel Walker]

On Mon, 2007-05-07 at 14:02 -0700, Tim Bird wrote:
> Daniel Walker wrote:
> > Are you sure you don't have HardIRQS in Threads enabled? Also are you
> > sure the kernel boots without the rt patch applied.
> 
> Yes on both counts.  I double-checked the configs.  I'm using quilt
> and the RT patch is at the end of my series, so it was easy to
> pop the patch and try again (module .config fixups).
> 
> I may take a few more blind stabs at different configs before
> I go diving into the networking code trying to trace something.
> (or before starting to bisect the patch).  Bisecting the patch
> will be a bit painful since it's currently an all-in-one.

I'm not sure I'd get into "bisecting" one big patch. Especially if you
new to the patches inner workings.. 

You can bisect by kernel version using the older patches but it's
clearly limited,

http://people.redhat.com/mingo/realtime-preempt/older/


Daniel

Re: Need help debugging RT-preempt patch

[john cooper]

Daniel Walker wrote:

> I'm not sure I'd get into "bisecting" one big patch. Especially if you
> new to the patches inner workings.. 
> 
> You can bisect by kernel version using the older patches but it's
> clearly limited,
> 
> http://people.redhat.com/mingo/realtime-preempt/older/

Actually working on this more closely over a year or so ago I
found a binary chop among patch versions to often be the
fastest means of locating the source of breakage.  Though that
may have been more applicable when several patch versions per
day were being generated.

-john

-- 
john.cooper@third-harmonic.com