From: Dave Chinner <david@fromorbit.com>
To: linux-kernel@vger.kernel.org
Cc: linux-fsdevel@vger.kernel.org
Subject: Re: [2.6.36-rc1] unmount livelock due to racing with bdi-flusher threads
Date: Mon, 13 Sep 2010 12:41:28 +1000 [thread overview]
Message-ID: <20100913024128.GC411@dastard> (raw)
In-Reply-To: <20100821084126.GE31488@dastard>
ping?
On Sat, Aug 21, 2010 at 06:41:26PM +1000, Dave Chinner wrote:
> Folks,
>
> I just had an umount take a very long time burning a CPU the entire
> time. It wasn't the unmount thread, either, it was the the bdi
> flusher thread for the the filesystem being unmounted. It was
> spinning with this perf top trace:
>
> 553144.00 76.9% writeback_inodes_wb [kernel.kallsyms]
> 106434.00 14.8% __ticket_spin_lock [kernel.kallsyms]
> 25646.00 3.6% __ticket_spin_unlock [kernel.kallsyms]
> 10512.00 1.5% _raw_spin_lock [kernel.kallsyms]
> 9606.00 1.3% put_super [kernel.kallsyms]
> 7920.00 1.1% __put_super [kernel.kallsyms]
> 5592.00 0.8% down_read_trylock [kernel.kallsyms]
> 46.00 0.0% kfree [kernel.kallsyms]
> 22.00 0.0% __do_softirq [kernel.kallsyms]
> 19.00 0.0% wb_writeback [kernel.kallsyms]
> 16.00 0.0% wb_do_writeback [kernel.kallsyms]
> 8.00 0.0% queue_io [kernel.kallsyms]
> 6.00 0.0% run_timer_softirq [kernel.kallsyms]
> 6.00 0.0% local_bh_enable_ip [kernel.kallsyms]
>
> This went on for ~7m25s (according to the pmchart trace I had on
> screen) before something broke the livelock by writing the inodes to
> disk (maybe the xfssyncd) and the unmount then completed a couple
> of seconds later.
>
> From the above profile, I'm assuming that writeback_inodes_wb() was
> seeing pin_sb_for_writeback(sb) failing and moving dirty inodes from
> the the b_io to the b_more_io list, then being called again,
> splicing the inodes on b_more_io back to b_io, and then failed again
> to pin_sb_for_writeback() for each inode, moving them back to the
> b_more_io list....
>
> This is on 2.6.36-rc1 + the radix tree fixes for writeback.
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
Dave Chinner
david@fromorbit.com
next prev parent reply other threads:[~2010-09-13 2:41 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-08-21 8:41 [2.6.36-rc1] unmount livelock due to racing with bdi-flusher threads Dave Chinner
2010-09-13 2:41 ` Dave Chinner [this message]
2010-09-30 21:02 ` Jan Kara
2010-10-01 2:59 ` Christoph Hellwig
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20100913024128.GC411@dastard \
--to=david@fromorbit.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).