Re: xfs_repair segfault

Rui Gomes
2015-03-09T16:24:40Z

Hello Eric,

I would love to send you the xfs metadump but it segfaults as well.

[root@icess8a ~]# xfs_metadump -w /dev/sdb1 xfs_metadata.dump
Metadata corruption detected at block 0x4ffed6d08/0x1000
xfs_metadump: cannot init perag data (117). Continuing anyway.
xfs_metadump: invalid block number (103589472/7327271) in inode 31138 bmapbtd root
xfs_metadump: bad number of extents 101 in inode 438913
xfs_metadump: bad number of extents 8126465 in inode 438922
xfs_metadump: bad number of extents 106 in inode 438930
xfs_metadump: bad number of extents 95 in inode 438931
xfs_metadump: bad number of extents 124 in inode 438932
xfs_metadump: bad number of extents 33648898 in inode 438933
xfs_metadump: bad number of extents 1134 in inode 438933
xfs_metadump: bad number of extents 1678966914 in inode 438942
xfs_metadump: bad number of extents 107 in inode 438947
xfs_metadump: bad number of extents 83917828 in inode 438952
xfs_metadump: bad number of extents 106 in inode 438952
xfs_metadump: bad number of extents 793014134 in inode 438957
xfs_metadump: invalid magic in dir inode 754758 block 0
xfs_metadump: invalid magic in dir inode 754767 block 0
xfs_metadump: invalid magic in dir inode 191761973 block 2
xfs_metadump: invalid magic in dir inode 191761973 block 3
xfs_metadump: invalid magic in dir inode 191761973 block 4
xfs_metadump: bad number of extents 201326593 in inode 252685314
xfs_metadump: invalid size in dir inode 252685315
xfs_metadump: zero length entry in dir inode 252685315
xfs_metadump: invalid size in dir inode 252685316
xfs_metadump: invalid size in dir inode 252685317
xfs_metadump: bad number of extents 268497404 in inode 252685318
xfs_metadump: bad number of extents 256 in inode 252685322
xfs_metadump: bad number of extents 754974721 in inode 252685326
xfs_metadump: bad number of extents 1931505779 in inode 252685349
xfs_metadump: suspicious count 860276 in bmap extent 0 in symlink ino 252685351
xfs_metadump: bad number of extents -2097020927 in inode 252685358
xfs_metadump: bad number of extents 50332161 in inode 252685374
xfs_metadump: bad number of extents 301989889 in inode 252685380
xfs_metadump: bad number of extents 301992705 in inode 252685383
xfs_metadump: entry length in dir inode 255735873 overflows space
xfs_metadump: bad number of extents 10229761 in inode 255735925
xfs_metadump: invalid attr size in inode 255735926
xfs_metadump: attr entry length in inode 255735926 overflows space
xfs_metadump: bad number of extents 301989889 in inode 259383043
xfs_metadump: bad number of extents 1952407297 in inode 259383045
xfs_metadump: bad number of extents 117440513 in inode 259383051
xfs_metadump: invalid block number 3840/16213200 (515412288720) in bmap extent 0 in symlink ino 259383052
xfs_metadump: bad number of extents 59484 in inode 259383054
xfs_metadump: entry length in dir inode 259383076 overflows space
xfs_metadump: invalid size in dir inode 259383077
xfs_metadump: entry length in dir inode 259383077 overflows space
xfs_metadump: invalid magic in dir inode 260215042 block 0
xfs_metadump: entry length in dir inode 260256256 overflows space
/usr/sbin/xfs_metadump: line 32: 16695 Segmentation fault xfs_db$DBOPTS -i -p xfs_metadump -c "metadump$OPTS $2" $1

This is the output of xfs_repair truncated:

entry "epl-v10.html" in shortform directory 259383076 references invalid inode 472470915093
would have junked entry "epl-v10.html" in directory inode 259383076
entry "fea�{re.xml" in shortform directory 259383076 references invalid inode 3940649933342743
would have junked entry "fea�{re.xml" in directory inode 259383076
entry " licen�k.html" in shortform directory 259383076 references invalid inode 1275257465539072
size of last entry overflows space left in in shortform dir 259383076, would reset to 20
entry contains illegal character in shortform dir 259383076
would have junked entry "licen�k.html" in directory inode 259383076
would have corrected directory 259383076 size from 107 to 115
bogus .. inode number (44169981931271) in directory inode 259383076, would clear inode number
bad magic number 0x2741 on inode 259383077, would reset magic number
bad (negative) size -2445736072638889871 on inode 259383077
would have cleared inode 259383077
bad magic number 0x4755 on inode 259383078, would reset magic number
bad non-zero extent size 2149318656 for non-realtime/extsize inode 259383078, would reset to zero
would have cleared inode 259383078
bad magic number 0x673d on inode 259383079, would reset magic number
bad version number 0x66 on inode 259383079, would reset version number
bad inode format in inode 259383079
would have cleared inode 259383079
bad non-zero extent size 33554432 for non-realtime/extsize inode 259383080, would reset to zero
bad nblocks 792633534534647812 for inode 259383080, would reset to 4
bad anextents 36865 for inode 259383080, would reset to 0
bad magic number 0x794e on inode 259383081, would reset magic number
bad non-zero extent size 33554432 for non-realtime/extsize inode 259383081, would reset to zero
data fork in ino 259383081 claims free block 16227590
data fork in ino 259383081 claims free block 16227591
data fork in regular inode 259383081 claims used block 16228000
correcting nextents for inode 259383081
bad data fork in inode 259383081
would have cleared inode 259383081
bad magic number 0x174e on inode 259383082, would reset magic number
bad version number 0x6 on inode 259383082, would reset version number
bad non-zero extent size 100663296 for non-realtime/extsize inode 259383082, would reset to zero
bad nblocks 7493989779961282574 for inode 259383082, would reset to 14
bad anextents 16398 for inode 259383082, would reset to 0
would have cleared inode 259383082
bad magic number 0x3d4e on inode 259383083, would reset magic number
bad non-zero extent size 33554432 for non-realtime/extsize inode 259383083, would reset to zero
zero length extent (off = 562949953421312, fsbno = 0) in ino 259383083
correcting nextents for inode 259383083
bad data fork in inode 259383083
would have cleared inode 259383083
bad magic number 0xd34e on inode 259383084, would reset magic number
bad non-zero extent size 50331648 for non-realtime/extsize inode 259383084, would reset to zero
bad attr fork offset 32 in inode 259383084, max=10
would have cleared inode 259383084
bad magic number 0xf34e on inode 259383085, would reset magic number
bad version number 0x4 on inode 259383085, would reset version number
bad non-zero extent size 50331648 for non-realtime/extsize inode 259383085, would reset to zero
bad attr fork offset 32 in inode 259383085, max=19
would have cleared inode 259383085
bad magic number 0x9e4e on inode 259383086, would reset magic number
bad version number 0xa on inode 259383086, would reset version number
bad nblocks 15852670688360923137 for inode 259383086, would reset to 1
would have cleared inode 259383086
data fork in regular inode 259393564 claims used block 16230080
correcting nextents for inode 259393564
bad data fork in inode 259393564
would have cleared inode 259393564
data fork in regular inode 259393566 claims used block 16230128
correcting nextents for inode 259393566
bad data fork in inode 259393566
would have cleared inode 259393566
data fork in regular inode 259702849 claims used block 16231728
correcting nextents for inode 259702849
bad data fork in inode 259702849
would have cleared inode 259702849
Metadata corruption detected at block 0x7c14878/0x1000
bad directory block magic # 0x3d012146 in block 0 for directory inode 260215042
corrupt block 0 in directory inode 260215042
would junk block
no . entry for directory 260215042
no .. entry for directory 260215042
problem with directory contents in inode 260215042
would have cleared inode 260215042
bad nblocks 7 for inode 260256256, would reset to 0
bad nextents 1 for inode 260256256, would reset to 0
entry " kchnfig" in shortform directory 260256256 references invalid inode 28428972647780227
entry contains illegal character in shortform dir 260256256
would have junked entry "kchnfig" in directory inode 260256256
entry " " in shortform directory 260256256 references invalid inode 0
size of last entry overflows space left in in shortform dir 260256256, would reset to -1
*** buffer overflow detected ***: /usr/sbin/xfs_repair terminated
======= Backtrace: =========
/lib64/libc.so.6(__fortify_fail+0x37)[0x7ffff74ff9c7]
/lib64/libc.so.6(+0x10bb90)[0x7ffff74fdb90]
/usr/sbin/xfs_repair[0x414ea8]
/usr/sbin/xfs_repair[0x41189f]
/usr/sbin/xfs_repair[0x4124ce]
/usr/sbin/xfs_repair[0x40b96f]
/usr/sbin/xfs_repair[0x40cddd]
/usr/sbin/xfs_repair[0x41964e]
/usr/sbin/xfs_repair[0x4265da]
/usr/sbin/xfs_repair[0x42666c]
/usr/sbin/xfs_repair[0x41975d]
/usr/sbin/xfs_repair[0x40388e]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7ffff7413af5]
/usr/sbin/xfs_repair[0x403f89]
======= Memory map: ========
00400000-00481000 r-xp 00000000 08:05 1075058659 /usr/sbin/xfs_repair
00680000-00681000 r--p 00080000 08:05 1075058659 /usr/sbin/xfs_repair
00681000-00684000 rw-p 00081000 08:05 1075058659 /usr/sbin/xfs_repair
00684000-1e571000 rw-p 00000000 00:00 0 [heap]
7fff58000000-7fff5884a000 rw-p 00000000 00:00 0
7fff5884a000-7fff5c000000 ---p 00000000 00:00 0
7fff60000000-7fff607c4000 rw-p 00000000 00:00 0
7fff607c4000-7fff64000000 ---p 00000000 00:00 0
7fff64000000-7fff6471c000 rw-p 00000000 00:00 0
7fff6471c000-7fff68000000 ---p 00000000 00:00 0
7fff68000000-7fff68723000 rw-p 00000000 00:00 0
7fff68723000-7fff6c000000 ---p 00000000 00:00 0
7fff6c000000-7fff6c75b000 rw-p 00000000 00:00 0
7fff6c75b000-7fff70000000 ---p 00000000 00:00 0
7fff70000000-7fff7080e000 rw-p 00000000 00:00 0
7fff7080e000-7fff74000000 ---p 00000000 00:00 0
7fff74000000-7fff749e8000 rw-p 00000000 00:00 0
7fff749e8000-7fff78000000 ---p 00000000 00:00 0
7fff78000000-7fff7873d000 rw-p 00000000 00:00 0
7fff7873d000-7fff7c000000 ---p 00000000 00:00 0
7fff7c000000-7fff7c794000 rw-p 00000000 00:00 0
7fff7c794000-7fff80000000 ---p 00000000 00:00 0
7fff80000000-7fff804ed000 rw-p 00000000 00:00 0
7fff804ed000-7fff84000000 ---p 00000000 00:00 0
7fff84000000-7fff847bf000 rw-p 00000000 00:00 0
7fff847bf000-7fff88000000 ---p 00000000 00:00 0
7fff88000000-7fff8875d000 rw-p 00000000 00:00 0
7fff8875d000-7fff8c000000 ---p 00000000 00:00 0
7fff8c000000-7fff8c021000 rw-p 00000000 00:00 0
7fff8c021000-7fff90000000 ---p 00000000 00:00 0
7fff90000000-7fff90729000 rw-p 00000000 00:00 0
7fff90729000-7fff94000000 ---p 00000000 00:00 0
7fff94000000-7fff947bf000 rw-p 00000000 00:00 0
7fff947bf000-7fff98000000 ---p 00000000 00:00 0
7fff98000000-7fff9876c000 rw-p 00000000 00:00 0
7fff9876c000-7fff9c000000 ---p 00000000 00:00 0
7fff9c000000-7fff9c8b6000 rw-p 00000000 00:00 0
7fff9c8b6000-7fffa0000000 ---p 00000000 00:00 0
7fffa0000000-7fffa0759000 rw-p 00000000 00:00 0
7fffa0759000-7fffa4000000 ---p 00000000 00:00 0
7fffa4000000-7fffa451c000 rw-p 00000000 00:00 0
7fffa451c000-7fffa8000000 ---p 00000000 00:00 0
7fffa8000000-7fffa8714000 rw-p 00000000 00:00 0
7fffa8714000-7fffac000000 ---p 00000000 00:00 0
7fffac000000-7fffac71d000 rw-p 00000000 00:00 0
7fffac71d000-7fffb0000000 ---p 00000000 00:00 0
7fffb0000000-7fffb081d000 rw-p 00000000 00:00 0
7fffb081d000-7fffb4000000 ---p 00000000 00:00 0
7fffb4000000-7fffb469d000 rw-p 00000000 00:00 0
7fffb469d000-7fffb8000000 ---p 00000000 00:00 0
7fffb8000000-7fffb84b6000 rw-p 00000000 00:00 0
7fffb84b6000-7fffbc000000 ---p 00000000 00:00 0
7fffbc000000-7fffbc6a2000 rw-p 00000000 00:00 0
7fffbc6a2000-7fffc0000000 ---p 00000000 00:00 0
7fffc0000000-7fffc1f31000 rw-p 00000000 00:00 0
7fffc1f31000-7fffc4000000 ---p 00000000 00:00 0
7fffc4000000-7fffc47a0000 rw-p 00000000 00:00 0
7fffc47a0000-7fffc8000000 ---p 00000000 00:00 0
7fffc8000000-7fffc8782000 rw-p 00000000 00:00 0
7fffc8782000-7fffcc000000 ---p 00000000 00:00 0
7fffcc000000-7fffcc7a0000 rw-p 00000000 00:00 0
7fffcc7a0000-7fffd0000000 ---p 00000000 00:00 0
7fffd0000000-7fffd0719000 rw-p 00000000 00:00 0
7fffd0719000-7fffd4000000 ---p 00000000 00:00 0
7fffd4000000-7fffd4798000 rw-p 00000000 00:00 0
7fffd4798000-7fffd8000000 ---p 00000000 00:00 0
7fffd8000000-7fffd8635000 rw-p 00000000 00:00 0
7fffd8635000-7fffdc000000 ---p 00000000 00:00 0
7fffdf931000-7fffdf932000 ---p 00000000 00:00 0
7fffdf932000-7fffe0132000 rw-p 00000000 00:00 0
7fffe0132000-7fffe0133000 ---p 00000000 00:00 0
7fffe0133000-7fffe0933000 rw-p 00000000 00:00 0
7fffe0933000-7fffe0934000 ---p 00000000 00:00 0
7fffe0934000-7fffe1134000 rw-p 00000000 00:00 0
7fffe1134000-7fffe1135000 ---p 00000000 00:00 0
7fffe1135000-7fffe1935000 rw-p 00000000 00:00 0
7fffef73b000-7fffef750000 r-xp 00000000 08:05 1075245487 /usr/lib64/libgcc_s-4.8.2-20140120.so.1
7fffef750000-7fffef94f000 ---p 00015000 08:05 1075245487 /usr/lib64/libgcc_s-4.8.2-20140120.so.1
7fffef94f000-7fffef950000 r--p 00014000 08:05 1075245487 /usr/lib64/libgcc_s-4.8.2-20140120.so.1
7fffef950000-7fffef951000 rw-p 00015000 08:05 1075245487 /usr/lib64/libgcc_s-4.8.2-20140120.so.1
7fffef951000-7ffff0ecb000 rw-p 00000000 00:00 0
7ffff0ecb000-7ffff73f2000 r--p 00000000 08:05 1610627723 /usr/lib/locale/locale-archive
7ffff73f2000-7ffff75a8000 r-xp 00000000 08:05 1074635524 /usr/lib64/libc-2.17.so
7ffff75a8000-7ffff77a8000 ---p 001b6000 08:05 1074635524 /usr/lib64/libc-2.17.so
7ffff77a8000-7ffff77ac000 r--p 001b6000 08:05 1074635524 /usr/lib64/libc-2.17.so
7ffff77ac000-7ffff77ae000 rw-p 001ba000 08:05 1074635524 /usr/lib64/libc-2.17.so
7ffff77ae000-7ffff77b3000 rw-p 00000000 00:00 0
7ffff77b3000-7ffff77c9000 r-xp 00000000 08:05 1074635679 /usr/lib64/libpthread-2.17.so
7ffff77c9000-7ffff79c9000 ---p 00016000 08:05 1074635679 /usr/lib64/libpthread-2.17.so
7ffff79c9000-7ffff79ca000 r--p 00016000 08:05 1074635679 /usr/lib64/libpthread-2.17.so
7ffff79ca000-7ffff79cb000 rw-p 00017000 08:05 1074635679 /usr/lib64/libpthread-2.17.so
7ffff79cb000-7ffff79cf000 rw-p 00000000 00:00 0
7ffff79cf000-7ffff79d6000 r-xp 00000000 08:05 1074635683 /usr/lib64/librt-2.17.so
7ffff79d6000-7ffff7bd5000 ---p 00007000 08:05 1074635683 /usr/lib64/librt-2.17.so
7ffff7bd5000-7ffff7bd6000 r--p 00006000 08:05 1074635683 /usr/lib64/librt-2.17.so
7ffff7bd6000-7ffff7bd7000 rw-p 00007000 08:05 1074635683 /usr/lib64/librt-2.17.so
7ffff7bd7000-7ffff7bdb000 r-xp 00000000 08:05 1074635744 /usr/lib64/libuuid.so.1.3.0
7ffff7bdb000-7ffff7dda000 ---p 00004000 08:05 1074635744 /usr/lib64/libuuid.so.1.3.0
7ffff7dda000-7ffff7ddb000 r--p 00003000 08:05 1074635744 /usr/lib64/libuuid.so.1.3.0
7ffff7ddb000-7ffff7ddc000 rw-p 00004000 08:05 1074635744 /usr/lib64/libuuid.so.1.3.0
7ffff7ddc000-7ffff7dfd000 r-xp 00000000 08:05 1075245500 /usr/lib64/ld-2.17.so
7ffff7e91000-7ffff7ff2000 rw-p 00000000 00:00 0
7ffff7ff8000-7ffff7ffa000 rw-p 00000000 00:00 0
7ffff7ffa000-7ffff7ffc000 r-xp 00000000 00:00 0 [vdso]
7ffff7ffc000-7ffff7ffd000 r--p 00020000 08:05 1075245500 /usr/lib64/ld-2.17.so
7ffff7ffd000-7ffff7ffe000 rw-p 00021000 08:05 1075245500 /usr/lib64/ld-2.17.so
7ffff7ffe000-7ffff7fff000 rw-p 00000000 00:00 0
7ffffffde000-7ffffffff000 rw-p 00000000 00:00 0 [stack]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]

Program received signal SIGABRT, Aborted.
0x00007ffff74275c9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);

Regards

-------------------------------
Rui Gomes
CTO

RVX - Reykjavik Visual Effects
Seljavegur 2,
101 Reykjavik
Iceland

Tel: + 354 527 3330
Mob: + 354 663 3360

----- Original Message -----
From: "Eric Sandeen" <sandeen@sandeen.net>
To: "Rui Gomes" <rgomes@rvx.is>, "xfs" <xfs@oss.sgi.com>
Cc: "omar" <omar@rvx.is>
Sent: Monday, 9 March, 2015 16:14:52
Subject: Re: xfs_repair segfault

On 3/9/15 11:50 AM, Rui Gomes wrote:
> Program received signal SIGABRT, Aborted.
> 0x00007ffff74275c9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
> 56 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
> #0 0x00007ffff74275c9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
> #1 0x00007ffff7428cd8 in __GI_abort () at abort.c:90
> #2 0x00007ffff7467db7 in __libc_message (do_abort=do_abort@entry=2, fmt=fmt@entry=0x7ffff756f561 "*** %s ***: %s terminated\n") at ../sysdeps/unix/sysv/linux/libc_fatal.c:196
> #3 0x00007ffff74ff9c7 in __GI___fortify_fail (msg=msg@entry=0x7ffff756f507 "buffer overflow detected") at fortify_fail.c:31
> #4 0x00007ffff74fdb90 in __GI___chk_fail () at chk_fail.c:28
> #5 0x0000000000414ea8 in memmove (__len=18446744073709551615, __src=0x1e562094, __dest=0x7fffffffd8f0) at /usr/include/bits/string3.h:57
> #6 process_sf_dir2 (dirname=0x46b0e2 "", repair=<synthetic pointer>, parent=0x7fffffffdc20, dino_dirty=0x7fffffffdc18, ino_discovery=1, dip=0x1e562000, ino=260256256, mp=0x1e562091) at dir2.c:992

That's here:

if (junkit) {
memmove(name, sfep->name, namelen); <<<<
name[namelen] = '\0';

and the len passed to memmove, 18446744073709551615, is 0xFFFFFFFFFFFFFFFF
or -1 according to gdb.

What are the few lines of xfs_repair output prior to this, i.e. messages
containing "shortform dir"?

If you'd like to create & compress an xfs_metadump & provide it to me offline,
I'll see if that recreates the segfault & look into it further.

Thanks,
-Eric