From: Brian Foster <bfoster@redhat.com>
To: Shyam Kaushik <shyam@zadarastorage.com>
Cc: Alex Lyakas <alex@zadarastorage.com>, xfs@oss.sgi.com
Subject: Re: XFS hung task in xfs_ail_push_all_sync() when unmounting FS after disk failure/recovery
Date: Tue, 22 Mar 2016 08:19:23 -0400 [thread overview]
Message-ID: <20160322121922.GA53693@bfoster.bfoster> (raw)
In-Reply-To: <f049419a2ab10f8e3c4fef0e4f4ca1ba@mail.gmail.com>
On Tue, Mar 22, 2016 at 04:51:39PM +0530, Shyam Kaushik wrote:
> Hi XFS developers,
>
> We are seeing the following issue with XFS on kernel 3.18.19.
>
> We have XFS mounted over a raw disk. Disk was pulled out manually. There
> were async writes on files that were errored like this
>
...
>
> And XFS hit metadata & Log IO errors that it decides to shutdown:
>
> Mar 16 16:03:22 host0 kernel: [ 4637.351841] XFS (dm-29): metadata I/O
> error: block 0x3a27fbd0 ("xlog_iodone") error 5 numblks 64
> Mar 16 16:03:22 host0 kernel: [ 4637.352820] XFS(dm-29): SHUTDOWN!!!
> old_flags=0x0 new_flags=0x2
> Mar 16 16:03:22 host0 kernel: [ 4637.353187] XFS (dm-29): Log I/O Error
> Detected. Shutting down filesystem
...
> Later the drive was re-inserted back. After the drive was re-inserted, XFS
> was attempted to be unmounted
>
> Mar 16 16:16:53 host0 controld: [2557] [ ] umount[202]
> : umount(/sdisk/vol5b0, xfs)
>
> But nothing happens except for the 30-secs xfs_log_force errors that keeps
> repeating
>
...
>
> This problem doesn't happen consistently, but happens periodically with a
> drive failure/recovery followed by XFS unmount. I couldn't find this issue
> fixed in later kernels. Can you please suggest how I can debug this issue
> further?
>
Similar problems have been reproduced due to racy/incorrect EFI/EFD
object tracking, which are internal data structures associated with
freeing extents.
What happens if you enable tracepoints while the fs is in this hung
unmount state?
# trace-cmd start -e "xfs:*"
# cat /sys/kernel/debug/tracing/trace_pipe
Brian
> Thanks!
>
> --Shyam
>
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
next prev parent reply other threads:[~2016-03-22 12:19 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-03-22 11:21 XFS hung task in xfs_ail_push_all_sync() when unmounting FS after disk failure/recovery Shyam Kaushik
2016-03-22 12:19 ` Brian Foster [this message]
2016-03-22 13:01 ` Shyam Kaushik
2016-03-22 14:03 ` Brian Foster
2016-03-22 15:38 ` Carlos Maiolino
2016-03-22 15:56 ` Carlos Maiolino
2016-03-23 9:43 ` Shyam Kaushik
2016-03-23 12:30 ` Brian Foster
2016-03-23 15:32 ` Carlos Maiolino
2016-03-23 22:37 ` Dave Chinner
2016-03-24 11:08 ` Carlos Maiolino
2016-03-24 16:52 ` Carlos Maiolino
2016-03-24 21:56 ` Dave Chinner
2016-04-01 12:31 ` Carlos Maiolino
2016-03-23 9:52 ` Shyam Kaushik
2016-03-24 13:38 ` Shyam Kaushik
2016-04-08 10:51 ` Shyam Kaushik
2016-04-08 13:16 ` Brian Foster
2016-04-08 13:35 ` Shyam Kaushik
2016-04-08 14:31 ` Carlos Maiolino
2016-04-08 17:48 ` Shyam Kaushik
2016-04-08 19:00 ` Brian Foster
2016-04-08 17:51 ` Shyam Kaushik
2016-04-08 22:46 ` Dave Chinner
2016-04-10 18:40 ` Alex Lyakas
2016-04-11 1:21 ` Dave Chinner
2016-04-11 14:52 ` Shyam Kaushik
2016-04-11 22:47 ` Dave Chinner
2016-04-12 5:20 ` Shyam Kaushik
2016-04-12 6:59 ` Shyam Kaushik
2016-04-12 8:19 ` Dave Chinner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160322121922.GA53693@bfoster.bfoster \
--to=bfoster@redhat.com \
--cc=alex@zadarastorage.com \
--cc=shyam@zadarastorage.com \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox