From: "Darrick J. Wong" <djwong@kernel.org>
To: Haiwei Li <lihaiwei.kernel@gmail.com>
Cc: linux-fsdevel@vger.kernel.org
Subject: Re: Problem with xfs in an old version
Date: Thu, 17 Jun 2021 10:09:22 -0700 [thread overview]
Message-ID: <20210617170922.GA158165@locust> (raw)
In-Reply-To: <CAB5KdOat4A7ZP1MDKHuXra7YN8cZ1J_K5W4M+G_Ye44un79_BQ@mail.gmail.com>
On Fri, Jun 18, 2021 at 12:06:40AM +0800, Haiwei Li wrote:
> Hi,
>
> Sorry to bother. I get a xfs error on kernel 3.10.x. And i don't know how to
> debug. I got nothing useful from search engines. So I sent an e-mail
> here. If there
> are other more suitable ways to discuss the problem, please let me know, thanks!
>
> I have gotten a message on the console.
>
> '-bash: /data/.my_history: Input/output error'
>
> I tried:
>
> # ls -l / | grep data
> ls: cannot access /data: Input/output error
> d????????? ? ? ? ? ? data
>
> The mount point info is:
>
> '/dev/vdb on /data type xfs (rw,noatime,attr2,inode64,prjquota)'
>
> System log messages as below:
>
> ffff882b86a34000: 31 38 38 32 30 31 36 0a 00 00 00 00 00 00 00 00
> 1882016.........
> ffff882b86a34010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> ................
> ffff882b86a34020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> ................
> ffff882b86a34030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> ................
> XFS (vdb): Internal error xfs_inode_buf_verify at line 410 of file
> XXXX/fs/x[2021-06-17 18:28:51]fs/xfs_inode.c. Caller
> 0xffffffffa04d410e
>
> CPU: 0 PID: 7715 Comm: kworker/0:1H Tainted: G U O
> 3.10.107-1-tlinux2_kvm_guest-0051 #1
> Hardware name: Smdbmds KVM, BIOS seabios-1.9.1-qemu-project.org 04/01/2014
> Workqueue: xfslogd xfs_buf_iodone_work [xfs]
> ffff882ec52d9000 000000001a1ab7e6 ffff882efa97fd50 ffffffff819f1d23
> ffff882efa97fd68 ffffffffa047da9b ffffffffa04d410e ffff882efa97fda0
> ffffffffa047daf5 0000019a00000001 0000000000000001 ffff882b86a34000
> Call Trace:
> [<ffffffff819f1d23>] dump_stack+0x19/0x1b
> [<ffffffffa047da9b>] xfs_error_report+0x3b/0x40 [xfs]
> [<ffffffffa04d410e>] ? xfs_inode_buf_read_verify+0xe/0x10 [xfs]
> [<ffffffffa047daf5>] xfs_corruption_error+0x55/0x80 [xfs]
> [<ffffffffa04d40a4>] xfs_inode_buf_verify+0x94/0xe0 [xfs]
> [<ffffffffa04d410e>] ? xfs_inode_buf_read_verify+0xe/0x10 [xfs]
> [<ffffffffa04d410e>] xfs_inode_buf_read_verify+0xe/0x10 [xfs]
> [<ffffffffa047b305>] xfs_buf_iodone_work+0xa5/0xd0 [xfs]
> [<ffffffff8106c00c>] process_one_work+0x17c/0x450
> [<ffffffff8106cebb>] worker_thread+0x11b/0x3a0
> [<ffffffff8106cda0>] ? manage_workers.isra.26+0x2a0/0x2a0
> [<ffffffff810737cf>] kthread+0xcf/0xe0
> [<ffffffff81073700>] ? insert_kthread_work+0x40/0x40
> [<ffffffff81ad5908>] ret_from_fork+0x58/0x90
> [<ffffffff81073700>] ? insert_kthread_work+0x40/0x40
> XFS (vdb): Corruption detected. Unmount and run xfs_repair
> ffff882b86a34100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> ................
> ffff882b86a34110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> ................
> ffff882b86a34120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> ................
> ffff882b86a34130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> ................
>
> I reboot and remount. It works normally. No error again. I guess data from the
> wrong blocks was returned to XFS.
>
> I have no idea how to reproduce. Our workload sometimes triggers the problem.
> To data, the problem only occurs on 3.10.x in three versions 3.10.x, 4.14.x and
> 5.4.x.
>
> Environment: Containers with workload are running in a kvm vm. The problem
> occurs in the kvm vm.
>
> Any ideas on how to debug? Thanks!
Uh, does xfs_repair -n on the unmounted filesystem complain about this
corrupt inode?
--D
>
> --
> Haiwei
next prev parent reply other threads:[~2021-06-17 17:09 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-17 16:06 Problem with xfs in an old version Haiwei Li
2021-06-17 17:09 ` Darrick J. Wong [this message]
2021-06-18 2:22 ` Haiwei Li
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210617170922.GA158165@locust \
--to=djwong@kernel.org \
--cc=lihaiwei.kernel@gmail.com \
--cc=linux-fsdevel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.