Re: [tracing, hang] dumping events gets stuck in synchronise_sched

[Dave Chinner <david@fromorbit.com>]

On Tue, Aug 17, 2010 at 09:02:38AM -0400, Steven Rostedt wrote:
> On Tue, 2010-08-17 at 21:52 +1000, Dave Chinner wrote:
> > On Tue, Aug 17, 2010 at 04:37:44PM +0800, Lai Jiangshan wrote:
> > > On 08/17/2010 03:37 PM, Dave Chinner wrote:
> > > > Tracing folks,
> > > > 
> > > > I've got a machine stuck with a cpu spinning in a tight loop (the
> > > > new writeback/sync livelock avoidance code is, well, livelocking),
> > > > and I was trying to find out what triggered by using the writeback
> > > > trace events. Unfortunately, I can't dump the trace events because
> > > > it gets stuck here:
> > > 
> > > > 
> > > > Given that the trace events are there mainly for debugging, this
> > > > seems like a bit of an oversight - hanging a CPU in a tight loop is
> > > > not an uncommon event during code development....
> > > > 
> > > 
> > > You can try 'cat trace_pipe', if I did not miss you meaning.
> > 
> > I'll try it, but I'm really after the static event list which is why
> > I'm using the trace file rather than trace_pipe. I want the history,
> > not new events as they happen. 
> 
> When the systems locks up, I assume you want to see why? The trace_pipe
> should show that without locking the system.

Exactly.

> You could also try downloading trace-cmd and running the tracer with
> that. That will save all traces to a file while running the trace.

I don't have tens of GB available to store all the traces that an
xfstests test run generates. In general, I don't need the traces,
either, and when  I do the problem is usually in the current ring
buffer, which is why I typically dump the events after the fact.

If the trace file cannot be made to handle this type of use
robustly, then perhaps it should be removed...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com