Re: Still seeing hangs in xlog_grant_log_space

Juerg Haefliger
2012-05-19T07:28:58Z

Hi Ben,

> Hey Juerg,
>
> On Wed, May 09, 2012 at 09:54:08AM +0200, Juerg Haefliger wrote:
>> > On Sat, May 05, 2012 at 09:44:35AM +0200, Juerg Haefliger wrote:
>> >> Did anybody have a chance to look at the data?
>> >
>> > https://bugs.launchpad.net/ubuntu/+source/linux/+bug/979498
>> >
>> > Here you indicate that you have created a reproducer.  Can you post it to the list?
>>
>> Canonical attached them to the bug report that they filed yesterday:
>> http://oss.sgi.com/bugzilla/show_bug.cgi?id=922
>
> I'm interested in understanding to what extent the hang you see in production
> on 2.6.38 is similar to the hang of the reproducer.  Mark is seeing a situation
> where there is nothing on the AIL and is clogged up in the CIL, others are
> seeing items on the AIL that don't seem to be making progress.  Could you
> provide a dump or traces from a hang on a filesystem with a normal sized log?
> Can the reproducer hit the hang eventually without resorting to the tiny log?

I'm not certain that the reproducer hang is identical to the
production hang. One difference that I've noticed is that a reproducer
hang can be cleared with an emergency sync while a production hang
can't. I'm working on trying to get a trace from a production machine.
Any ideas how to do the tracing without filling up the filesystem with
trace data? I need to run it for at least a week to catch a hang. I
was thinking of tracing in 15 min batches and just keeping 30 mins
worth of trace data but that will leave some gaps when I stop/restart
the tracing. I'm not familiar with ftrace, maybe it provides the
functionality to do some sort of ring buffer dumping to only keep the
last x mins of data?

Thanks
...Juerg

> Regards,
>        Ben