From: Dave Chinner <david@fromorbit.com>
To: "Михаил Гаврилов" <mikhail.v.gavrilov@gmail.com>
Cc: Jan Kara <jack@suse.cz>, Christoph Hellwig <hch@infradead.org>,
linux-xfs@vger.kernel.org, linux-mm@kvack.org
Subject: Re: kernel BUG at fs/xfs/xfs_aops.c:853! in kernel 4.13 rc6
Date: Mon, 9 Oct 2017 11:05:29 +1100 [thread overview]
Message-ID: <20171009000529.GY3666@dastard> (raw)
In-Reply-To: <CABXGCsMorRzy-dJrjTO6sP80BSb0RAeMhF3QGwSkk50m7VYzOA@mail.gmail.com> <CABXGCsOeex62Y4qQJwvMJ+fJ+MnKyKGDj9eRbKemeMVWo5huKw@mail.gmail.com>
On Sat, Oct 07, 2017 at 01:10:58PM +0500, Михаил Гаврилов wrote:
> But seems now got another issue:
>
> [ 1966.953781] INFO: task tracker-store:8578 blocked for more than 120 seconds.
> [ 1966.953797] Not tainted 4.13.4-301.fc27.x86_64+debug #1
> [ 1966.953800] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> disables this message.
> [ 1966.953804] tracker-store D12840 8578 1655 0x00000000
> [ 1966.953811] Call Trace:
> [ 1966.953823] __schedule+0x2dc/0xbb0
> [ 1966.953830] ? wait_on_page_bit_common+0xfb/0x1a0
> [ 1966.953838] schedule+0x3d/0x90
> [ 1966.953843] io_schedule+0x16/0x40
> [ 1966.953847] wait_on_page_bit_common+0x10a/0x1a0
> [ 1966.953857] ? page_cache_tree_insert+0x170/0x170
> [ 1966.953865] __filemap_fdatawait_range+0x101/0x1a0
> [ 1966.953883] file_write_and_wait_range+0x63/0xc0
Ok, that's in wait_on_page_writeback(page)
......
> And yet another
>
> [41288.797026] INFO: task tracker-store:4535 blocked for more than 120 seconds.
> [41288.797034] Not tainted 4.13.4-301.fc27.x86_64+debug #1
> [41288.797037] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> disables this message.
> [41288.797041] tracker-store D10616 4535 1655 0x00000000
> [41288.797049] Call Trace:
> [41288.797061] __schedule+0x2dc/0xbb0
> [41288.797072] ? bit_wait+0x60/0x60
> [41288.797076] schedule+0x3d/0x90
> [41288.797082] io_schedule+0x16/0x40
> [41288.797086] bit_wait_io+0x11/0x60
> [41288.797091] __wait_on_bit+0x31/0x90
> [41288.797099] out_of_line_wait_on_bit+0x94/0xb0
> [41288.797106] ? bit_waitqueue+0x40/0x40
> [41288.797113] __block_write_begin_int+0x265/0x550
> [41288.797132] iomap_write_begin.constprop.14+0x7d/0x130
And that's in wait_on_buffer().
In both cases we are waiting on a bit lock for IO completion. In the
first case it is on page, the second it's on sub-page read IO
completion during a write.
Triggeringa hung task timeouts like this doesn't usually indicate a
filesystem problem. In general, it means that the
IO subsystem is overloaded such that iowait times are blowing out
to >120s, an IO has been lost and/or not completed correctly or you
are getting single bit memory errors or corruption occurring.
None of these things usually filesystem problems, and the trainsmash
of blocked tasks on filesystem locks is typical for these types of
"blocked indefinitely with locks held" type of situations. It does
tend to indicate taht there is quite a bit of load on the
filesystem, though...
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
next prev parent reply other threads:[~2017-10-09 0:06 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-09-03 4:22 kernel BUG at fs/xfs/xfs_aops.c:853! in kernel 4.13 rc6 Михаил Гаврилов
2017-09-03 7:43 ` Christoph Hellwig
2017-09-03 14:08 ` Михаил Гаврилов
2017-09-04 12:30 ` Jan Kara
2017-10-07 8:10 ` Михаил Гаврилов
2017-10-07 9:22 ` Михаил Гаврилов
2017-10-09 0:05 ` Dave Chinner [this message]
2017-10-09 18:31 ` Luis R. Rodriguez
2017-10-09 19:02 ` Eric W. Biederman
2017-10-15 8:53 ` Aleksa Sarai
2017-10-15 13:06 ` Theodore Ts'o
2017-10-15 22:14 ` Eric W. Biederman
2017-10-15 23:22 ` Dave Chinner
2017-10-16 17:44 ` Eric W. Biederman
2017-10-16 21:38 ` Dave Chinner
2017-10-16 1:13 ` Theodore Ts'o
2017-10-16 17:53 ` Eric W. Biederman
2017-10-16 18:50 ` Theodore Ts'o
2017-10-16 22:00 ` Dave Chinner
2017-10-17 1:34 ` Theodore Ts'o
2017-10-17 0:59 ` Aleksa Sarai
2017-10-17 9:20 ` Jan Kara
2017-10-17 14:12 ` Theodore Ts'o
2017-11-06 19:25 ` Luis R. Rodriguez
2017-11-07 15:26 ` Jan Kara
2017-10-09 22:28 ` Dave Chinner
2017-10-10 7:57 ` Jan Kara
2017-09-04 1:43 ` Dave Chinner
2017-09-04 2:20 ` Darrick J. Wong
2017-09-04 12:14 ` Jan Kara
2017-09-04 22:36 ` Dave Chinner
2017-09-05 16:17 ` Jan Kara
2017-09-05 23:42 ` Dave Chinner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20171009000529.GY3666@dastard \
--to=david@fromorbit.com \
--cc=hch@infradead.org \
--cc=jack@suse.cz \
--cc=linux-mm@kvack.org \
--cc=linux-xfs@vger.kernel.org \
--cc=mikhail.v.gavrilov@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).