XFS mounting fails on MIPS

XFS mounting fails on MIPS

[Ajeet Yadav]

[-- Attachment #1.1: Type: text/plain, Size: 1919 bytes --]

Linux XFS version : 2.6.34
Architecure: MIPS
I am getting the following error during mount.

XFS mounting filesystem sda2
Starting XFS recovery on filesystem: sda2 (logdev: internal)
XFS internal error XFS_WANT_CORRUPTED_GOTO at line 1518 of file
fs/xfs/xfs_alloc.c.  Caller 0x801174a0
Call Trace:
[<800050bc>] dump_stack+0x8/0x34
[<80115254>] xfs_free_ag_extent+0x128/0x7a8
[<801174a0>] xfs_free_extent+0xa0/0xcc
[<80155278>] xlog_recover_process_efi+0x15c/0x210
[<801553cc>] xlog_recover_process_efis+0xa0/0x12c
[<8015590c>] xlog_recover_finish+0x28/0xcc
[<8015d4fc>] xfs_mountfs+0x4f0/0x5d0
[<801758d8>] xfs_fs_fill_super+0x158/0x360
[<8008b67c>] get_sb_bdev+0x11c/0x1c4
[<801734e0>] xfs_fs_get_sb+0x20/0x2c
[<80089cd0>] vfs_kern_mount+0x68/0xd0
[<80089d9c>] do_kern_mount+0x54/0x118
[<800a4e98>] do_mount+0x7f0/0x86c
[<800a4fb0>] sys_mount+0x9c/0xf8
[<80002124>] stack_done+0x20/0x3c

Filesystem "sda2": XFS internal error xfs_trans_cancel at line 1161 of file
fs/xfs/xfs_trans.c.  Caller 0x801552f8

Call Trace:
[<800050bc>] dump_stack+0x8/0x34
[<801601f0>] xfs_trans_cancel+0x88/0x118
[<801552f8>] xlog_recover_process_efi+0x1dc/0x210
[<801553cc>] xlog_recover_process_efis+0xa0/0x12c
[<8015590c>] xlog_recover_finish+0x28/0xcc
[<8015d4fc>] xfs_mountfs+0x4f0/0x5d0
[<801758d8>] xfs_fs_fill_super+0x158/0x360
[<8008b67c>] get_sb_bdev+0x11c/0x1c4
[<801734e0>] xfs_fs_get_sb+0x20/0x2c
[<80089cd0>] vfs_kern_mount+0x68/0xd0
[<80089d9c>] do_kern_mount+0x54/0x118
[<800a4e98>] do_mount+0x7f0/0x86c
[<800a4fb0>] sys_mount+0x9c/0xf8
[<80002124>] stack_done+0x20/0x3c

xfs_force_shutdown(sda2,0x8) called from line 1162 of file
fs/xfs/xfs_trans.c.  Return address = 0x80160204
Filesystem "sda2": Corruption of in-memory data detected.  Shutting down
filesystem: sda2
Please umount the filesystem, and rectify the problem(s)
Failed to recover EFIs on filesystem: sda2
XFS: log mount finish failed

With Regards
Ajeet Yadav

[-- Attachment #1.2: Type: text/html, Size: 2384 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS mounting fails on MIPS

[Stan Hoeppner]

Ajeet Yadav put forth on 11/4/2010 12:57 AM:

> Filesystem "sda2": Corruption of in-memory data detected.

That would lead me to believe you have a hardware problem.  Could be a
bad DIMM, a bad board, defective PSU with too much voltage swing, etc,
etc.  I'd start by replacing the DIMM(s) and go from there with process
of elimination.

-- 
Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS mounting fails on MIPS

[Dave Chinner]

On Thu, Nov 04, 2010 at 11:27:24AM +0530, Ajeet Yadav wrote:
> Linux XFS version : 2.6.34
> Architecure: MIPS
> I am getting the following error during mount.
> 
> XFS mounting filesystem sda2
> Starting XFS recovery on filesystem: sda2 (logdev: internal)
> XFS internal error XFS_WANT_CORRUPTED_GOTO at line 1518 of file
> fs/xfs/xfs_alloc.c.  Caller 0x801174a0

Corrupted freespace btree. Run xfs_repair to find out if it is
on-disk corruption (and to fix it).

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS mounting fails on MIPS

[Christoph Hellwig]

On Thu, Nov 04, 2010 at 11:27:24AM +0530, Ajeet Yadav wrote:
> Linux XFS version : 2.6.34
> Architecure: MIPS
> I am getting the following error during mount.

Is this the same mips system that you have the vmap aliasing problems
during log recovery with?

As long as that is not fixed I would deeply distrust any callchain
from the recovery code like this one.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS mounting fails on MIPS

[Ajeet Yadav]

[-- Attachment #1.1: Type: text/plain, Size: 1518 bytes --]

True, its the same system and you were right it was cache VIPT cache problem
the cache hold the stale value even after xlog_bread() update the buffer.
I do not know whether its correct ways to resolve the problem, but the
problem no longer occur.


diff -Naur -X fs/xfs/linux-2.6/xfs_buf.c fs-dirty/xfs/linux-2.6/xfs_buf.c

--- fs/xfs/linux-2.6/xfs_buf.c 2010-07-18 12:27:11.000000000 +0530

+++ fs-dirty/xfs/linux-2.6/xfs_buf.c 2010-10-25 11:01:28.252762355 +0530

@@ -1166,6 +1166,13 @@

xfs_buf_ioerror(bp, -error);

+ if (!error &&

+ (bp->b_flags & XBF_MAPPED) &&

+ (bp->b_page_count > 1) &&

+ (bp->b_flags & XBF_READ)) {

+ dma_cache_inv((unsigned long)bp->b_addr,

+ (bp->b_page_count * PAGE_SIZE) - bp->b_offset);

+ }

do {

struct page *page = bvec->bv_page;

@@ -1275,6 +1282,10 @@

submit_io:

if (likely(bio->bi_size)) {

+ if ((bp->b_flags & XBF_MAPPED) && (bp->b_page_count > 1)) {

+ dma_cache_wback_inv((unsigned long)bp->b_addr,

+ (bp->b_page_count * PAGE_SIZE) - bp->b_offset);

+ }

submit_bio(rw, bio);

if (size)

goto next_chunk;


On Thu, Nov 4, 2010 at 6:20 PM, Christoph Hellwig <hch@infradead.org> wrote:

> On Thu, Nov 04, 2010 at 11:27:24AM +0530, Ajeet Yadav wrote:
> > Linux XFS version : 2.6.34
> > Architecure: MIPS
> > I am getting the following error during mount.
>
> Is this the same mips system that you have the vmap aliasing problems
> during log recovery with?
>
> As long as that is not fixed I would deeply distrust any callchain
> from the recovery code like this one.
>

[-- Attachment #1.2: Type: text/html, Size: 2216 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS mounting fails on MIPS

[Christoph Hellwig]

Hi Ajeet,

On Tue, Nov 09, 2010 at 04:43:04PM +0530, Ajeet Yadav wrote:
> True, its the same system and you were right it was cache VIPT cache problem
> the cache hold the stale value even after xlog_bread() update the buffer.
> I do not know whether its correct ways to resolve the problem, but the
> problem no longer occur.

It seems like you more less re-implemented the vmap coherency hooks
inside XFS, hardcoded to the mips implementation.

The actual helpers would looks something like:

static inline void flush_kernel_vmap_range(void *addr, int size)
{
	dma_cache_inv(addr, size);
}

static inline void invalidate_kernel_vmap_range(void *addr, int size)
{
	dma_cache_inv(addr, size);
}

For some reason the kernel also expects flush_dcache_page to be
implemented by an architecture if we want to implement these two
(it's keyed off ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE).

Can someone of the mips folks helps with this?

The testcase is easy, mounting an xfs filesystem after an unclean
shutdown on a machine with virtually indexed caches.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS mounting fails on MIPS

[Ajeet Yadav]

[-- Attachment #1.1: Type: text/plain, Size: 3173 bytes --]

Coming back to problem, I wish to know about this problem
Linux XFS version : 2.6.34
Architecure: MIPS
I am getting the following error during mount.

XFS mounting filesystem sda2
Starting XFS recovery on filesystem: sda2 (logdev: internal)
XFS internal error XFS_WANT_CORRUPTED_GOTO at line 1518 of file
fs/xfs/xfs_alloc.c.  Caller 0x801174a0
Call Trace:
[<800050bc>] dump_stack+0x8/0x34
[<80115254>] xfs_free_ag_extent+0x128/0x7a8
[<801174a0>] xfs_free_extent+0xa0/0xcc
[<80155278>] xlog_recover_process_efi+0x15c/0x210
[<801553cc>] xlog_recover_process_efis+0xa0/0x12c
[<8015590c>] xlog_recover_finish+0x28/0xcc
[<8015d4fc>] xfs_mountfs+0x4f0/0x5d0
[<801758d8>] xfs_fs_fill_super+0x158/0x360
[<8008b67c>] get_sb_bdev+0x11c/0x1c4
[<801734e0>] xfs_fs_get_sb+0x20/0x2c
[<80089cd0>] vfs_kern_mount+0x68/0xd0
[<80089d9c>] do_kern_mount+0x54/0x118
[<800a4e98>] do_mount+0x7f0/0x86c
[<800a4fb0>] sys_mount+0x9c/0xf8
[<80002124>] stack_done+0x20/0x3c

Filesystem "sda2": XFS internal error xfs_trans_cancel at line 1161 of file
fs/xfs/xfs_trans.c.  Caller 0x801552f8

Call Trace:
[<800050bc>] dump_stack+0x8/0x34
[<801601f0>] xfs_trans_cancel+0x88/0x118
[<801552f8>] xlog_recover_process_efi+0x1dc/0x210
[<801553cc>] xlog_recover_process_efis+0xa0/0x12c
[<8015590c>] xlog_recover_finish+0x28/0xcc
[<8015d4fc>] xfs_mountfs+0x4f0/0x5d0
[<801758d8>] xfs_fs_fill_super+0x158/0x360
[<8008b67c>] get_sb_bdev+0x11c/0x1c4
[<801734e0>] xfs_fs_get_sb+0x20/0x2c
[<80089cd0>] vfs_kern_mount+0x68/0xd0
[<80089d9c>] do_kern_mount+0x54/0x118
[<800a4e98>] do_mount+0x7f0/0x86c
[<800a4fb0>] sys_mount+0x9c/0xf8
[<80002124>] stack_done+0x20/0x3c

xfs_force_shutdown(sda2,0x8) called from line 1162 of file
fs/xfs/xfs_trans.c.  Return address = 0x80160204
Filesystem "sda2": Corruption of in-memory data detected.  Shutting down
filesystem: sda2
Please umount the filesystem, and rectify the problem(s)
Failed to recover EFIs on filesystem: sda2
XFS: log mount finish failed

With Regards
Ajeet Yadav

On Tue, Nov 9, 2010 at 7:35 PM, Christoph Hellwig <hch@infradead.org> wrote:

> Hi Ajeet,
>
> On Tue, Nov 09, 2010 at 04:43:04PM +0530, Ajeet Yadav wrote:
> > True, its the same system and you were right it was cache VIPT cache
> problem
> > the cache hold the stale value even after xlog_bread() update the buffer.
> > I do not know whether its correct ways to resolve the problem, but the
> > problem no longer occur.
>
> It seems like you more less re-implemented the vmap coherency hooks
> inside XFS, hardcoded to the mips implementation.
>
> The actual helpers would looks something like:
>
> static inline void flush_kernel_vmap_range(void *addr, int size)
> {
>        dma_cache_inv(addr, size);
> }
>
> static inline void invalidate_kernel_vmap_range(void *addr, int size)
> {
>        dma_cache_inv(addr, size);
> }
>
> For some reason the kernel also expects flush_dcache_page to be
> implemented by an architecture if we want to implement these two
> (it's keyed off ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE).
>
> Can someone of the mips folks helps with this?
>
> The testcase is easy, mounting an xfs filesystem after an unclean
> shutdown on a machine with virtually indexed caches.
>
>

[-- Attachment #1.2: Type: text/html, Size: 4100 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS mounting fails on MIPS

[Ajeet Yadav]

[-- Attachment #1.1: Type: text/plain, Size: 3722 bytes --]

I think mips folks agree with the change. I really wish to have there
comment.

I also wish to know do we really need fix in XFS for virtual indexed
architecture, I think its generic issue as many architecture now use VIVT or
VIPT caches. Do we want to say XFS is relatively unstable with virtual
indexed architecture ?


On Thu, Nov 11, 2010 at 10:27 AM, Ajeet Yadav <ajeet.yadav.77@gmail.com>wrote:

> Coming back to problem, I wish to know about this problem
> Linux XFS version : 2.6.34
> Architecure: MIPS
> I am getting the following error during mount.
>
> XFS mounting filesystem sda2
> Starting XFS recovery on filesystem: sda2 (logdev: internal)
> XFS internal error XFS_WANT_CORRUPTED_GOTO at line 1518 of file
> fs/xfs/xfs_alloc.c.  Caller 0x801174a0
> Call Trace:
> [<800050bc>] dump_stack+0x8/0x34
> [<80115254>] xfs_free_ag_extent+0x128/0x7a8
> [<801174a0>] xfs_free_extent+0xa0/0xcc
> [<80155278>] xlog_recover_process_efi+0x15c/0x210
> [<801553cc>] xlog_recover_process_efis+0xa0/0x12c
> [<8015590c>] xlog_recover_finish+0x28/0xcc
> [<8015d4fc>] xfs_mountfs+0x4f0/0x5d0
> [<801758d8>] xfs_fs_fill_super+0x158/0x360
> [<8008b67c>] get_sb_bdev+0x11c/0x1c4
> [<801734e0>] xfs_fs_get_sb+0x20/0x2c
> [<80089cd0>] vfs_kern_mount+0x68/0xd0
> [<80089d9c>] do_kern_mount+0x54/0x118
> [<800a4e98>] do_mount+0x7f0/0x86c
> [<800a4fb0>] sys_mount+0x9c/0xf8
> [<80002124>] stack_done+0x20/0x3c
>
> Filesystem "sda2": XFS internal error xfs_trans_cancel at line 1161 of file
> fs/xfs/xfs_trans.c.  Caller 0x801552f8
>
> Call Trace:
> [<800050bc>] dump_stack+0x8/0x34
> [<801601f0>] xfs_trans_cancel+0x88/0x118
> [<801552f8>] xlog_recover_process_efi+0x1dc/0x210
> [<801553cc>] xlog_recover_process_efis+0xa0/0x12c
> [<8015590c>] xlog_recover_finish+0x28/0xcc
> [<8015d4fc>] xfs_mountfs+0x4f0/0x5d0
> [<801758d8>] xfs_fs_fill_super+0x158/0x360
> [<8008b67c>] get_sb_bdev+0x11c/0x1c4
> [<801734e0>] xfs_fs_get_sb+0x20/0x2c
> [<80089cd0>] vfs_kern_mount+0x68/0xd0
> [<80089d9c>] do_kern_mount+0x54/0x118
> [<800a4e98>] do_mount+0x7f0/0x86c
> [<800a4fb0>] sys_mount+0x9c/0xf8
> [<80002124>] stack_done+0x20/0x3c
>
> xfs_force_shutdown(sda2,0x8) called from line 1162 of file
> fs/xfs/xfs_trans.c.  Return address = 0x80160204
> Filesystem "sda2": Corruption of in-memory data detected.  Shutting down
> filesystem: sda2
> Please umount the filesystem, and rectify the problem(s)
> Failed to recover EFIs on filesystem: sda2
> XFS: log mount finish failed
>
> With Regards
> Ajeet Yadav
>
> On Tue, Nov 9, 2010 at 7:35 PM, Christoph Hellwig <hch@infradead.org>wrote:
>
>> Hi Ajeet,
>>
>> On Tue, Nov 09, 2010 at 04:43:04PM +0530, Ajeet Yadav wrote:
>> > True, its the same system and you were right it was cache VIPT cache
>> problem
>> > the cache hold the stale value even after xlog_bread() update the
>> buffer.
>> > I do not know whether its correct ways to resolve the problem, but the
>> > problem no longer occur.
>>
>> It seems like you more less re-implemented the vmap coherency hooks
>> inside XFS, hardcoded to the mips implementation.
>>
>> The actual helpers would looks something like:
>>
>> static inline void flush_kernel_vmap_range(void *addr, int size)
>> {
>>        dma_cache_inv(addr, size);
>> }
>>
>> static inline void invalidate_kernel_vmap_range(void *addr, int size)
>> {
>>        dma_cache_inv(addr, size);
>> }
>>
>> For some reason the kernel also expects flush_dcache_page to be
>> implemented by an architecture if we want to implement these two
>> (it's keyed off ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE).
>>
>> Can someone of the mips folks helps with this?
>>
>> The testcase is easy, mounting an xfs filesystem after an unclean
>> shutdown on a machine with virtually indexed caches.
>>
>>
>

[-- Attachment #1.2: Type: text/html, Size: 4889 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS mounting fails on MIPS

[Christoph Hellwig]

On Thu, Nov 11, 2010 at 11:10:33AM +0530, Ajeet Yadav wrote:
> I think mips folks agree with the change. I really wish to have there
> comment.
> 
> I also wish to know do we really need fix in XFS for virtual indexed
> architecture, I think its generic issue as many architecture now use VIVT or
> VIPT caches. Do we want to say XFS is relatively unstable with virtual
> indexed architecture ?

The vmap flushing APIs I repeatedly pointed you at are the generic
solution.  They've been implement for arm, parisc and sh already, it's
just that no one has bothered to do it for mips yet.

Without the proper flushing anything that uses vmap, which includes
XFS is not "relatively unstable" but in fact not usable on systems
with virtually indexed caches.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS mounting fails on MIPS

[Ajeet Yadav]

[-- Attachment #1.1: Type: text/plain, Size: 906 bytes --]

Ah! I got your point
thank you very much
On Thu, Nov 11, 2010 at 6:19 PM, Christoph Hellwig <hch@infradead.org>wrote:

> On Thu, Nov 11, 2010 at 11:10:33AM +0530, Ajeet Yadav wrote:
> > I think mips folks agree with the change. I really wish to have there
> > comment.
> >
> > I also wish to know do we really need fix in XFS for virtual indexed
> > architecture, I think its generic issue as many architecture now use VIVT
> or
> > VIPT caches. Do we want to say XFS is relatively unstable with virtual
> > indexed architecture ?
>
> The vmap flushing APIs I repeatedly pointed you at are the generic
> solution.  They've been implement for arm, parisc and sh already, it's
> just that no one has bothered to do it for mips yet.
>
> Without the proper flushing anything that uses vmap, which includes
> XFS is not "relatively unstable" but in fact not usable on systems
> with virtually indexed caches.
>

[-- Attachment #1.2: Type: text/html, Size: 1254 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs