* new fs, xfs_admin new label, metadata corruption detected
@ 2016-03-15 23:08 Chris Murphy
2016-03-16 0:34 ` Eric Sandeen
0 siblings, 1 reply; 13+ messages in thread
From: Chris Murphy @ 2016-03-15 23:08 UTC (permalink / raw)
To: xfs@oss.sgi.com
Is this expected? File system continues to work OK, sha256sum on the
single file on this fs matches the source, but the metadata corruption
messages are a bit scary so figured I'd ask about it.
Filesystem created with
xfsprogs-4.3.0-1.fc23
kernel-4.4.4-300.fc23.x86_64
mkfs.xfs defaults used on an LVM thinly provisioned volume on a single
spinning disk.
# xfs_info /dev/mapper/VG-testxfs
meta-data=/dev/mapper/VG-testxfs isize=512 agcount=23, agsize=399984 blks
= sectsz=4096 attr=2, projid32bit=1
= crc=1 finobt=1 spinodes=0
data = bsize=4096 blocks=8960000, imaxpct=25
= sunit=16 swidth=16 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal bsize=4096 blocks=3124, version=2
= sectsz=4096 sunit=1 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
Mounted the file system, copied one decently large ISO, deleted it,
unmounted the file system. Upgraded to kernel 4.5.0, and
xfsprogs-4.3.0-3.fc23, reboot, mounted the file system again, copied a
file, unmounted, and then tried to rename its label:
[root@f23s ~]# xfs_admin -L testxfs /dev/mapper/VG-testxfs
Metadata corruption detected at xfs_agf block 0x30d3808/0x1000
xfs_admin: cannot init perag data (-117). Continuing anyway.
writing all SBs
new label = "testxfs"
Kernel messages:
[snip]
[ 226.804218] nf_conntrack: automatic helper assignment is deprecated
and it will be removed soon. Use the iptables CT target to attach
helpers instead.
[ 370.527700] SGI XFS with ACLs, security attributes, no debug enabled
[ 370.546159] XFS (dm-5): Mounting V5 Filesystem
[ 370.621943] XFS (dm-5): Ending clean mount
[ 438.064901] EXT4-fs (dm-7): mounted filesystem with ordered data
mode. Opts: (null)
[ 837.965252] BTRFS: device label testbtr devid 1 transid 3
/dev/mapper/VG-testbtrfs
[ 851.690987] BTRFS info (device dm-8): disk space caching is enabled
[ 851.691064] BTRFS: has skinny extents
[ 851.691100] BTRFS: flagging fs with big metadata feature
[ 851.702530] BTRFS: creating UUID tree
[ 1057.799823] XFS (dm-5): Unmounting Filesystem
[root@f23s ~]# xfs_repair /dev/mapper/VG-testxfs
Phase 1 - find and verify superblock...
- reporting progress in intervals of 15 minutes
Phase 2 - using internal log
- zero log...
- scan filesystem freespace and inode maps...
Metadata corruption detected at xfs_agf block 0x4322d08/0x1000
Metadata corruption detected at xfs_agf block 0x30d3808/0x1000
fllast 1014 in agf 22 too large (max = 1014)
Metadata corruption detected at xfs_agf block 0x4015988/0x1000
Metadata corruption detected at xfs_agf block 0x3d08608/0x1000
Metadata corruption detected at xfs_agf block 0x33e0b88/0x1000
fllast 1014 in agf 16 too large (max = 1014)
fllast 1014 in agf 21 too large (max = 1014)
fllast 1014 in agf 20 too large (max = 1014)
Metadata corruption detected at xfs_agf block 0x39fb288/0x1000
fllast 1014 in agf 17 too large (max = 1014)
fllast 1014 in agf 19 too large (max = 1014)
Metadata corruption detected at xfs_agf block 0x36edf08/0x1000
fllast 1014 in agf 18 too large (max = 1014)
- 16:47:27: scanning filesystem freespace - 23 of 23
allocation groups done
- found root inode chunk
Phase 3 - for each AG...
- scan and clear agi unlinked lists...
- 16:47:27: scanning agi unlinked lists - 23 of 23 allocation
groups done
- process known inodes and perform inode discovery...
- agno = 0
- agno = 15
- agno = 16
- agno = 1
- agno = 2
- agno = 17
- agno = 3
- agno = 18
- agno = 4
- agno = 19
- agno = 5
- agno = 20
- agno = 6
- agno = 7
- agno = 21
- agno = 8
- agno = 22
- agno = 9
- agno = 10
- agno = 11
- agno = 12
- agno = 13
- agno = 14
- 16:47:27: process known inodes and inode discovery - 64 of
64 inodes done
- process newly discovered inodes...
- 16:47:27: process newly discovered inodes - 23 of 23
allocation groups done
Phase 4 - check for duplicate blocks...
- setting up duplicate extent list...
- 16:47:27: setting up duplicate extent list - 23 of 23
allocation groups done
- check for inodes claiming duplicate blocks...
- agno = 0
- agno = 1
- agno = 2
- agno = 3
- agno = 4
- agno = 5
- agno = 6
- agno = 7
- agno = 8
- agno = 9
- agno = 10
- agno = 11
- agno = 12
- agno = 13
- agno = 14
- agno = 15
- agno = 16
- agno = 17
- agno = 18
- agno = 19
- agno = 20
- agno = 21
- agno = 22
- 16:47:27: check for inodes claiming duplicate blocks - 64 of
64 inodes done
Phase 5 - rebuild AG headers and trees...
- 16:47:27: rebuild AG headers and trees - 23 of 23 allocation
groups done
- reset superblock...
Phase 6 - check inode connectivity...
- resetting contents of realtime bitmap and summary inodes
- traversing filesystem ...
- traversal finished ...
- moving disconnected inodes to lost+found ...
Phase 7 - verify and correct link counts...
done
I haven't tried reproducing this.
--
Chris Murphy
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: new fs, xfs_admin new label, metadata corruption detected
2016-03-15 23:08 new fs, xfs_admin new label, metadata corruption detected Chris Murphy
@ 2016-03-16 0:34 ` Eric Sandeen
2016-03-16 7:23 ` Chris Murphy
0 siblings, 1 reply; 13+ messages in thread
From: Eric Sandeen @ 2016-03-16 0:34 UTC (permalink / raw)
To: xfs
On 3/15/16 6:08 PM, Chris Murphy wrote:
> Is this expected? File system continues to work OK, sha256sum on the
> single file on this fs matches the source, but the metadata corruption
> messages are a bit scary so figured I'd ask about it.
>
> Filesystem created with
> xfsprogs-4.3.0-1.fc23
> kernel-4.4.4-300.fc23.x86_64
>
> mkfs.xfs defaults used on an LVM thinly provisioned volume on a single
> spinning disk.
>
> # xfs_info /dev/mapper/VG-testxfs
> meta-data=/dev/mapper/VG-testxfs isize=512 agcount=23, agsize=399984 blks
^^^^^^^^^^
It seems that you left a growfs step out of it, where does that come in
to the testcase?
-Eric
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: new fs, xfs_admin new label, metadata corruption detected
2016-03-16 0:34 ` Eric Sandeen
@ 2016-03-16 7:23 ` Chris Murphy
2016-03-17 2:39 ` Chris Murphy
0 siblings, 1 reply; 13+ messages in thread
From: Chris Murphy @ 2016-03-16 7:23 UTC (permalink / raw)
To: Eric Sandeen, xfs
[-- Attachment #1.1: Type: text/plain, Size: 1229 bytes --]
On Tue, Mar 15, 2016, 6:34 PM Eric Sandeen <sandeen@sandeen.net> wrote:
> On 3/15/16 6:08 PM, Chris Murphy wrote:
> > Is this expected? File system continues to work OK, sha256sum on the
> > single file on this fs matches the source, but the metadata corruption
> > messages are a bit scary so figured I'd ask about it.
> >
> > Filesystem created with
> > xfsprogs-4.3.0-1.fc23
> > kernel-4.4.4-300.fc23.x86_64
> >
> > mkfs.xfs defaults used on an LVM thinly provisioned volume on a single
> > spinning disk.
> >
> > # xfs_info /dev/mapper/VG-testxfs
> > meta-data=/dev/mapper/VG-testxfs isize=512 agcount=23, agsize=399984
> blks
> ^^^^^^^^^^
>
> It seems that you left a growfs step out of it, where does that come in
> to the testcase?
>
Oops I spaced it out entirely.
Much of this happened in Cockpit, which uses storaged. So no matter what
I'll have to backtrack to get exact reproduce steps. In the meantime:
Create thin pool
Create thin volume
Mkfs
Mount
Copy file
Grow (resize done in cockpit)
Delete file
Umount
Kernel and progs update and reboot
Mount
Copy file
Unmount
Relabel
<error>
Repair
Mount
Something amiss with growfs and lvresize steps?
Chris Murphy
[-- Attachment #1.2: Type: text/html, Size: 1711 bytes --]
[-- Attachment #2: Type: text/plain, Size: 121 bytes --]
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: new fs, xfs_admin new label, metadata corruption detected
2016-03-16 7:23 ` Chris Murphy
@ 2016-03-17 2:39 ` Chris Murphy
2016-03-17 2:40 ` Chris Murphy
` (3 more replies)
0 siblings, 4 replies; 13+ messages in thread
From: Chris Murphy @ 2016-03-17 2:39 UTC (permalink / raw)
To: Chris Murphy; +Cc: Eric Sandeen, xfs@oss.sgi.com
OK I can consistently reproduce this on the CLI. I end up with totally different
# lvcreate -L 40g VG -n testxfs
Logical volume "testxfs" created.
# lvs
LV VG Attr LSize Pool Origin Data% Meta%
Move Log Cpy%Sync Convert
testbtrfs VG Vwi-a-tz-- 39.06g thintastic 0.01
testext4 VG Vwi-a-tz-- 39.06g thintastic 3.96
testxfs VG -wi-a----- 40.00g
thintastic VG twi-aotz-- 39.06g 3.97 4.86
# mkfs.xfs /dev/VG/testxfs
meta-data=/dev/VG/testxfs isize=512 agcount=4, agsize=2621440 blks
= sectsz=4096 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=0
data = bsize=4096 blocks=10485760, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=5120, version=2
= sectsz=4096 sunit=1 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
# mount /dev/VG/testxfs /mnt/0
# cp file.iso /mnt/0
# lvresize -L 80g VG/testxfs
Size of logical volume VG/testxfs changed from 40.00 GiB (10240
extents) to 80.00 GiB (20480 extents).
Logical volume testxfs successfully resized.
# xfs_growfs -d /mnt/0
meta-data=/dev/mapper/VG-testxfs isize=512 agcount=4, agsize=2621440 blks
= sectsz=4096 attr=2, projid32bit=1
= crc=1 finobt=1 spinodes=0
data = bsize=4096 blocks=10485760, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal bsize=4096 blocks=5120, version=2
= sectsz=4096 sunit=1 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
data blocks changed from 10485760 to 20971520
# df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 1.9G 0 1.9G 0% /dev
tmpfs 1.9G 0 1.9G 0% /dev/shm
tmpfs 1.9G 904K 1.9G 1% /run
tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/sda1 180G 15G 165G 9% /
tmpfs 1.9G 4.0K 1.9G 1% /tmp
/dev/sda1 180G 15G 165G 9% /boot
/dev/sda6 525M 8.4M 517M 2% /boot/efi
/dev/mapper/brick1 562G 355G 205G 64% /brick1
tmpfs 388M 0 388M 0% /run/user/1000
/dev/mapper/VG-testxfs 80G 857M 80G 2% /mnt/0
[root@f23s ~]# lvs
LV VG Attr LSize Pool Origin Data% Meta%
Move Log Cpy%Sync Convert
testbtrfs VG Vwi-a-tz-- 39.06g thintastic 0.01
testext4 VG Vwi-a-tz-- 39.06g thintastic 3.96
testxfs VG -wi-ao---- 80.00g
thintastic VG twi-aotz-- 39.06g 3.97 4.86
# xfs_info /dev/VG/testxfs
meta-data=/dev/mapper/VG-testxfs isize=512 agcount=8, agsize=2621440 blks
= sectsz=4096 attr=2, projid32bit=1
= crc=1 finobt=1 spinodes=0
data = bsize=4096 blocks=20971520, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal bsize=4096 blocks=5120, version=2
= sectsz=4096 sunit=1 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
# xfs_repair -n /dev/VG/testxfs
Phase 1 - find and verify superblock...
Phase 2 - using internal log
- zero log...
- scan filesystem freespace and inode maps...
Metadata corruption detected at xfs_agf block 0x8c00008/0x1000
fllast 1014 in agf 7 too large (max = 1014)
Metadata corruption detected at xfs_agf block 0x6400008/0x1000
fllast 1014 in agf 5 too large (max = 1014)
Metadata corruption detected at xfs_agf block 0x5000008/0x1000
fllast 1014 in agf 4 too large (max = 1014)
Metadata corruption detected at xfs_agf block 0x7800008/0x1000
fllast 1014 in agf 6 too large (max = 1014)
- found root inode chunk
Phase 3 - for each AG...
- scan (but don't clear) agi unlinked lists...
- process known inodes and perform inode discovery...
- agno = 0
- agno = 1
- agno = 2
- agno = 3
- agno = 4
- agno = 5
- agno = 6
- agno = 7
- process newly discovered inodes...
Phase 4 - check for duplicate blocks...
- setting up duplicate extent list...
- check for inodes claiming duplicate blocks...
- agno = 0
- agno = 1
- agno = 4
- agno = 2
- agno = 7
- agno = 5
- agno = 6
- agno = 3
No modify flag set, skipping phase 5
Phase 6 - check inode connectivity...
- traversing filesystem ...
- traversal finished ...
- moving disconnected inodes to lost+found ...
Phase 7 - verify link counts...
No modify flag set, skipping filesystem flush and exiting.
# mount /dev/VG/testxfs /mnt/0
No mount errors.
[14189.495433] XFS (dm-5): Mounting V5 Filesystem
[14189.710141] XFS (dm-5): Ending clean mount
[14326.856144] XFS (dm-5): Unmounting Filesystem
[14413.691734] XFS (dm-5): Mounting V5 Filesystem
[14413.833598] XFS (dm-5): Ending clean mount
Chris Murphy
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: new fs, xfs_admin new label, metadata corruption detected
2016-03-17 2:39 ` Chris Murphy
@ 2016-03-17 2:40 ` Chris Murphy
2016-03-17 2:55 ` Chris Murphy
` (2 subsequent siblings)
3 siblings, 0 replies; 13+ messages in thread
From: Chris Murphy @ 2016-03-17 2:40 UTC (permalink / raw)
To: xfs@oss.sgi.com
On Wed, Mar 16, 2016 at 8:39 PM, Chris Murphy <lists@colorremedies.com> wrote:
> OK I can consistently reproduce this on the CLI. I end up with totally different
Ignore the incomplete thought.
--
Chris Murphy
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: new fs, xfs_admin new label, metadata corruption detected
2016-03-17 2:39 ` Chris Murphy
2016-03-17 2:40 ` Chris Murphy
@ 2016-03-17 2:55 ` Chris Murphy
2016-03-17 6:39 ` Chris Murphy
2016-03-17 7:32 ` Zorro Lang
2016-03-17 20:57 ` Dave Chinner
3 siblings, 1 reply; 13+ messages in thread
From: Chris Murphy @ 2016-03-17 2:55 UTC (permalink / raw)
To: xfs@oss.sgi.com
On Wed, Mar 16, 2016 at 8:39 PM, Chris Murphy <lists@colorremedies.com> wrote:
> # xfs_repair -n /dev/VG/testxfs
It's amateur hour... yes I did unmount before running xfs_repair or it
fails. And I also did the test on conventional LV, not thinp.
If I do the same test on a thinp LV, I consistently get more errors
(than conventional LV) when I run xfs_repair, probably because there
are 4x the AG's on thinp LV's for some reason.
# xfs_repair -n /dev/VG/testxfs
Phase 1 - find and verify superblock...
- reporting progress in intervals of 15 minutes
Phase 2 - using internal log
- zero log...
- scan filesystem freespace and inode maps...
Metadata corruption detected at xfs_agf block 0x72ff488/0x1000
Metadata corruption detected at xfs_agf block 0x90ff188/0x1000Metadata
corruption detected at xfs_agf block 0x8bff208/0x1000
Metadata corruption detected at xfs_agf block 0x86ff288/0x1000
Metadata corruption detected at xfs_agf block 0x81ff308/0x1000
Metadata corruption detected at xfs_agf block 0x77ff408/0x1000
Metadata corruption detected at xfs_agf block 0x7cff388/0x1000
fllast 1014 in agf 23 too large (max = 1014)
Metadata corruption detected at xfs_agf block 0x6dff508/0x1000
Metadata corruption detected at xfs_agf block 0x68ff588/0x1000Metadata
corruption detected at xfs_agf block 0x63ff608/0x1000
Metadata corruption detected at xfs_agf block 0x5eff688/0x1000
Metadata corruption detected at xfs_agf block 0x54ff788/0x1000
Metadata corruption detected at xfs_agf block 0x59ff708/0x1000
Metadata corruption detected at xfs_agf block 0x4fff808/0x1000
fllast 1014 in agf 28 too large (max = 1014)
fllast 1014 in agf 27 too large (max = 1014)
fllast 1014 in agf 26 too large (max = 1014)
fllast 1014 in agf 29 too large (max = 1014)
fllast 1014 in agf 25 too large (max = 1014)
fllast 1014 in agf 24 too large (max = 1014)
fllast 1014 in agf 22 too large (max = 1014)
fllast 1014 in agf 21 too large (max = 1014)
fllast 1014 in agf 19 too large (max = 1014)
fllast 1014 in agf 20 too large (max = 1014)
fllast 1014 in agf 17 too large (max = 1014)
fllast 1014 in agf 18 too large (max = 1014)
fllast 1014 in agf 16 too large (max = 1014)
Metadata corruption detected at xfs_agf block 0x9fff008/0x1000
Metadata corruption detected at xfs_agf block 0x95ff108/0x1000
fllast 1014 in agf 32 too large (max = 1014)
fllast 1014 in agf 30 too large (max = 1014)
Metadata corruption detected at xfs_agf block 0x9aff088/0x1000
fllast 1014 in agf 31 too large (max = 1014)
- 20:51:37: scanning filesystem freespace - 33 of 33
allocation groups done
- found root inode chunk
Phase 3 - for each AG...
- scan (but don't clear) agi unlinked lists...
- 20:51:37: scanning agi unlinked lists - 33 of 33 allocation
groups done
- process known inodes and perform inode discovery...
- agno = 0
- agno = 30
- agno = 15
- agno = 1
- agno = 31
- agno = 16
- agno = 2
- agno = 3
- agno = 17
- agno = 32
- agno = 4
- agno = 18
- agno = 19
- agno = 5
- agno = 20
- agno = 6
- agno = 21
- agno = 7
- agno = 22
- agno = 8
- agno = 23
- agno = 9
- agno = 24
- agno = 10
- agno = 25
- agno = 11
- agno = 26
- agno = 12
- agno = 27
- agno = 13
- agno = 28
- agno = 14
- agno = 29
- 20:51:37: process known inodes and inode discovery - 64 of
64 inodes done
- process newly discovered inodes...
- 20:51:37: process newly discovered inodes - 33 of 33
allocation groups done
Phase 4 - check for duplicate blocks...
- setting up duplicate extent list...
- 20:51:37: setting up duplicate extent list - 33 of 33
allocation groups done
- check for inodes claiming duplicate blocks...
- agno = 1
- agno = 2
- agno = 3
- agno = 6
- agno = 7
- agno = 8
- agno = 9
- agno = 10
- agno = 11
- agno = 12
- agno = 13
- agno = 14
- agno = 15
- agno = 16
- agno = 17
- agno = 18
- agno = 19
- agno = 20
- agno = 21
- agno = 22
- agno = 23
- agno = 24
- agno = 25
- agno = 26
- agno = 27
- agno = 28
- agno = 29
- agno = 30
- agno = 31
- agno = 32
- agno = 4
- agno = 0
- agno = 5
- 20:51:37: check for inodes claiming duplicate blocks - 64 of
64 inodes done
No modify flag set, skipping phase 5
Phase 6 - check inode connectivity...
- traversing filesystem ...
- traversal finished ...
- moving disconnected inodes to lost+found ...
Phase 7 - verify link counts...
No modify flag set, skipping filesystem flush and exiting.
--
Chris Murphy
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: new fs, xfs_admin new label, metadata corruption detected
2016-03-17 2:55 ` Chris Murphy
@ 2016-03-17 6:39 ` Chris Murphy
0 siblings, 0 replies; 13+ messages in thread
From: Chris Murphy @ 2016-03-17 6:39 UTC (permalink / raw)
To: xfs@oss.sgi.com
Using the same procedure but doing it with Btrfs and ext4, there are
no errors. 'btrfs check' and 'btrfs scrub' both come up clean, no
complaints. And e2fsck has no complaints for a resized ext4 volume.
Chris Murphy
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: new fs, xfs_admin new label, metadata corruption detected
2016-03-17 2:39 ` Chris Murphy
2016-03-17 2:40 ` Chris Murphy
2016-03-17 2:55 ` Chris Murphy
@ 2016-03-17 7:32 ` Zorro Lang
2016-03-17 20:57 ` Dave Chinner
3 siblings, 0 replies; 13+ messages in thread
From: Zorro Lang @ 2016-03-17 7:32 UTC (permalink / raw)
To: Chris Murphy; +Cc: Eric Sandeen, xfs@oss.sgi.com
On Wed, Mar 16, 2016 at 08:39:21PM -0600, Chris Murphy wrote:
> OK I can consistently reproduce this on the CLI. I end up with totally different
>
> # lvcreate -L 40g VG -n testxfs
> Logical volume "testxfs" created.
> # lvs
> LV VG Attr LSize Pool Origin Data% Meta%
> Move Log Cpy%Sync Convert
> testbtrfs VG Vwi-a-tz-- 39.06g thintastic 0.01
> testext4 VG Vwi-a-tz-- 39.06g thintastic 3.96
> testxfs VG -wi-a----- 40.00g
> thintastic VG twi-aotz-- 39.06g 3.97 4.86
> # mkfs.xfs /dev/VG/testxfs
> meta-data=/dev/VG/testxfs isize=512 agcount=4, agsize=2621440 blks
> = sectsz=4096 attr=2, projid32bit=1
> = crc=1 finobt=1, sparse=0
> data = bsize=4096 blocks=10485760, imaxpct=25
> = sunit=0 swidth=0 blks
> naming =version 2 bsize=4096 ascii-ci=0 ftype=1
> log =internal log bsize=4096 blocks=5120, version=2
> = sectsz=4096 sunit=1 blks, lazy-count=1
> realtime =none extsz=4096 blocks=0, rtextents=0
> # mount /dev/VG/testxfs /mnt/0
> # cp file.iso /mnt/0
> # lvresize -L 80g VG/testxfs
> Size of logical volume VG/testxfs changed from 40.00 GiB (10240
> extents) to 80.00 GiB (20480 extents).
> Logical volume testxfs successfully resized.
> # xfs_growfs -d /mnt/0
> meta-data=/dev/mapper/VG-testxfs isize=512 agcount=4, agsize=2621440 blks
> = sectsz=4096 attr=2, projid32bit=1
> = crc=1 finobt=1 spinodes=0
> data = bsize=4096 blocks=10485760, imaxpct=25
> = sunit=0 swidth=0 blks
> naming =version 2 bsize=4096 ascii-ci=0 ftype=1
> log =internal bsize=4096 blocks=5120, version=2
> = sectsz=4096 sunit=1 blks, lazy-count=1
> realtime =none extsz=4096 blocks=0, rtextents=0
> data blocks changed from 10485760 to 20971520
> # df -h
> Filesystem Size Used Avail Use% Mounted on
> devtmpfs 1.9G 0 1.9G 0% /dev
> tmpfs 1.9G 0 1.9G 0% /dev/shm
> tmpfs 1.9G 904K 1.9G 1% /run
> tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
> /dev/sda1 180G 15G 165G 9% /
> tmpfs 1.9G 4.0K 1.9G 1% /tmp
> /dev/sda1 180G 15G 165G 9% /boot
> /dev/sda6 525M 8.4M 517M 2% /boot/efi
> /dev/mapper/brick1 562G 355G 205G 64% /brick1
> tmpfs 388M 0 388M 0% /run/user/1000
> /dev/mapper/VG-testxfs 80G 857M 80G 2% /mnt/0
> [root@f23s ~]# lvs
> LV VG Attr LSize Pool Origin Data% Meta%
> Move Log Cpy%Sync Convert
> testbtrfs VG Vwi-a-tz-- 39.06g thintastic 0.01
> testext4 VG Vwi-a-tz-- 39.06g thintastic 3.96
> testxfs VG -wi-ao---- 80.00g
> thintastic VG twi-aotz-- 39.06g 3.97 4.86
> # xfs_info /dev/VG/testxfs
> meta-data=/dev/mapper/VG-testxfs isize=512 agcount=8, agsize=2621440 blks
> = sectsz=4096 attr=2, projid32bit=1
> = crc=1 finobt=1 spinodes=0
> data = bsize=4096 blocks=20971520, imaxpct=25
> = sunit=0 swidth=0 blks
> naming =version 2 bsize=4096 ascii-ci=0 ftype=1
> log =internal bsize=4096 blocks=5120, version=2
> = sectsz=4096 sunit=1 blks, lazy-count=1
> realtime =none extsz=4096 blocks=0, rtextents=0
> # xfs_repair -n /dev/VG/testxfs
> Phase 1 - find and verify superblock...
> Phase 2 - using internal log
> - zero log...
> - scan filesystem freespace and inode maps...
> Metadata corruption detected at xfs_agf block 0x8c00008/0x1000
> fllast 1014 in agf 7 too large (max = 1014)
> Metadata corruption detected at xfs_agf block 0x6400008/0x1000
> fllast 1014 in agf 5 too large (max = 1014)
> Metadata corruption detected at xfs_agf block 0x5000008/0x1000
> fllast 1014 in agf 4 too large (max = 1014)
> Metadata corruption detected at xfs_agf block 0x7800008/0x1000
> fllast 1014 in agf 6 too large (max = 1014)
Hi,
I can't reproduce this bug by run the same steps with you said on:
linux 4.5.0-rc2
xfsprogs v4.5.0-rc1
Could you reproduce you problem on the newest linux/xfsprogs?
As above results, agf 4, 5, 6 and 7 are the *new* AGs which created by
growfs. So the corruption only on the *new* AGs. Generally the
agf_fllast should < XFS_AGFL_SIZE(mp), but your fllast is equal to
XFS_AGFL_SIZE(mp).
BTW, if you can reproduce this bug on the newest linux/xfsprogs, could
you check when "crc=0, finobt=0" or "crc=1, finobt=0", it still can be
reproduced? If you can't reproduce on the newest, it maybe a bug of fedora.
Thanks,
Zorro
> - found root inode chunk
> Phase 3 - for each AG...
> - scan (but don't clear) agi unlinked lists...
> - process known inodes and perform inode discovery...
> - agno = 0
> - agno = 1
> - agno = 2
> - agno = 3
> - agno = 4
> - agno = 5
> - agno = 6
> - agno = 7
> - process newly discovered inodes...
> Phase 4 - check for duplicate blocks...
> - setting up duplicate extent list...
> - check for inodes claiming duplicate blocks...
> - agno = 0
> - agno = 1
> - agno = 4
> - agno = 2
> - agno = 7
> - agno = 5
> - agno = 6
> - agno = 3
> No modify flag set, skipping phase 5
> Phase 6 - check inode connectivity...
> - traversing filesystem ...
> - traversal finished ...
> - moving disconnected inodes to lost+found ...
> Phase 7 - verify link counts...
> No modify flag set, skipping filesystem flush and exiting.
> # mount /dev/VG/testxfs /mnt/0
>
> No mount errors.
> [14189.495433] XFS (dm-5): Mounting V5 Filesystem
> [14189.710141] XFS (dm-5): Ending clean mount
> [14326.856144] XFS (dm-5): Unmounting Filesystem
> [14413.691734] XFS (dm-5): Mounting V5 Filesystem
> [14413.833598] XFS (dm-5): Ending clean mount
>
>
>
> Chris Murphy
>
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: new fs, xfs_admin new label, metadata corruption detected
2016-03-17 2:39 ` Chris Murphy
` (2 preceding siblings ...)
2016-03-17 7:32 ` Zorro Lang
@ 2016-03-17 20:57 ` Dave Chinner
2016-03-17 21:56 ` Dave Chinner
2016-03-17 23:21 ` Chris Murphy
3 siblings, 2 replies; 13+ messages in thread
From: Dave Chinner @ 2016-03-17 20:57 UTC (permalink / raw)
To: Chris Murphy; +Cc: Eric Sandeen, xfs@oss.sgi.com
On Wed, Mar 16, 2016 at 08:39:21PM -0600, Chris Murphy wrote:
> OK I can consistently reproduce this on the CLI. I end up with totally different
>
> # lvcreate -L 40g VG -n testxfs
> Logical volume "testxfs" created.
....
[grow]
....
> # xfs_repair -n /dev/VG/testxfs
> Phase 1 - find and verify superblock...
> Phase 2 - using internal log
> - zero log...
> - scan filesystem freespace and inode maps...
> Metadata corruption detected at xfs_agf block 0x8c00008/0x1000
> fllast 1014 in agf 7 too large (max = 1014)
> Metadata corruption detected at xfs_agf block 0x6400008/0x1000
> fllast 1014 in agf 5 too large (max = 1014)
> Metadata corruption detected at xfs_agf block 0x5000008/0x1000
> fllast 1014 in agf 4 too large (max = 1014)
> Metadata corruption detected at xfs_agf block 0x7800008/0x1000
> fllast 1014 in agf 6 too large (max = 1014)
> - found root inode chunk
That's caused by commit 96f859d ("libxfs: pack the agfl header
structure so XFS_AGFL_SIZE is correct") and growfs setting the
last free list entry to something that a userspace without the
commit 9fccb9f ("libxfs: pack the agfl header structure so
XFS_AGFL_SIZE is correct") fails to validate correctly.
i.e. update userspace to 4.5.0, and the repair noise will go away.
I think we probably need to fix the growfs code to set it's initial
flfirst/fllast values to be 1/0, not 0/EOFL, and the problem will
then go away....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: new fs, xfs_admin new label, metadata corruption detected
2016-03-17 20:57 ` Dave Chinner
@ 2016-03-17 21:56 ` Dave Chinner
2016-03-17 23:21 ` Chris Murphy
1 sibling, 0 replies; 13+ messages in thread
From: Dave Chinner @ 2016-03-17 21:56 UTC (permalink / raw)
To: Chris Murphy; +Cc: Eric Sandeen, xfs@oss.sgi.com
On Fri, Mar 18, 2016 at 07:57:29AM +1100, Dave Chinner wrote:
> On Wed, Mar 16, 2016 at 08:39:21PM -0600, Chris Murphy wrote:
> > OK I can consistently reproduce this on the CLI. I end up with totally different
> >
> > # lvcreate -L 40g VG -n testxfs
> > Logical volume "testxfs" created.
> ....
> [grow]
> ....
> > # xfs_repair -n /dev/VG/testxfs
> > Phase 1 - find and verify superblock...
> > Phase 2 - using internal log
> > - zero log...
> > - scan filesystem freespace and inode maps...
> > Metadata corruption detected at xfs_agf block 0x8c00008/0x1000
> > fllast 1014 in agf 7 too large (max = 1014)
> > Metadata corruption detected at xfs_agf block 0x6400008/0x1000
> > fllast 1014 in agf 5 too large (max = 1014)
> > Metadata corruption detected at xfs_agf block 0x5000008/0x1000
> > fllast 1014 in agf 4 too large (max = 1014)
> > Metadata corruption detected at xfs_agf block 0x7800008/0x1000
> > fllast 1014 in agf 6 too large (max = 1014)
> > - found root inode chunk
>
> That's caused by commit 96f859d ("libxfs: pack the agfl header
> structure so XFS_AGFL_SIZE is correct") and growfs setting the
> last free list entry to something that a userspace without the
> commit 9fccb9f ("libxfs: pack the agfl header structure so
> XFS_AGFL_SIZE is correct") fails to validate correctly.
>
> i.e. update userspace to 4.5.0, and the repair noise will go away.
>
> I think we probably need to fix the growfs code to set it's initial
> flfirst/fllast values to be 1/0, not 0/EOFL, and the problem will
> then go away....
Patch to do this below if you want to test it.
-Dave.
--
Dave Chinner
david@fromorbit.com
xfs: Don't wrap growfs AGFL indexes
From: Dave Chinner <dchinner@redhat.com>
Commit 96f859d ("libxfs: pack the agfl header structure so
XFS_AGFL_SIZE is correct") allowed the freelist to use the empty
slot at the end of the freelist on 64 bit systems that was not
being used due to sizeof() rounding up the structure size.
This has caused versions of xfs_repair prior to 4.5.0 (which also
has the fix) to report this as a corruption once the filesystem has
been grown. Older kernels can also have problems (seen from a whacky
container/vm management environment) mounting filesystems grown on a
system with a newer kernel than the vm/container it is deployed on.
To avoid this problem, change the initial free list indexes not to
wrap across the end of the AGFL, hence avoiding the initialisation
of agf_fllast to the last index in the AGFL.
cc: <stable@vger.kernel.org> # 4.4-4.5
Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
fs/xfs/xfs_fsops.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/fs/xfs/xfs_fsops.c b/fs/xfs/xfs_fsops.c
index ee3aaa0a..ca0d3eb 100644
--- a/fs/xfs/xfs_fsops.c
+++ b/fs/xfs/xfs_fsops.c
@@ -243,8 +243,8 @@ xfs_growfs_data_private(
agf->agf_roots[XFS_BTNUM_CNTi] = cpu_to_be32(XFS_CNT_BLOCK(mp));
agf->agf_levels[XFS_BTNUM_BNOi] = cpu_to_be32(1);
agf->agf_levels[XFS_BTNUM_CNTi] = cpu_to_be32(1);
- agf->agf_flfirst = 0;
- agf->agf_fllast = cpu_to_be32(XFS_AGFL_SIZE(mp) - 1);
+ agf->agf_flfirst = cpu_to_be32(1);
+ agf->agf_fllast = 0;
agf->agf_flcount = 0;
tmpsize = agsize - XFS_PREALLOC_BLOCKS(mp);
agf->agf_freeblks = cpu_to_be32(tmpsize);
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: new fs, xfs_admin new label, metadata corruption detected
2016-03-17 20:57 ` Dave Chinner
2016-03-17 21:56 ` Dave Chinner
@ 2016-03-17 23:21 ` Chris Murphy
2016-03-18 2:22 ` Dave Chinner
2016-03-18 4:43 ` Zorro Lang
1 sibling, 2 replies; 13+ messages in thread
From: Chris Murphy @ 2016-03-17 23:21 UTC (permalink / raw)
To: Dave Chinner; +Cc: Eric Sandeen, Chris Murphy, xfs@oss.sgi.com
On Thu, Mar 17, 2016 at 2:57 PM, Dave Chinner <david@fromorbit.com> wrote:
> On Wed, Mar 16, 2016 at 08:39:21PM -0600, Chris Murphy wrote:
>> OK I can consistently reproduce this on the CLI. I end up with totally different
>>
>> # lvcreate -L 40g VG -n testxfs
>> Logical volume "testxfs" created.
> ....
> [grow]
> ....
>> # xfs_repair -n /dev/VG/testxfs
>> Phase 1 - find and verify superblock...
>> Phase 2 - using internal log
>> - zero log...
>> - scan filesystem freespace and inode maps...
>> Metadata corruption detected at xfs_agf block 0x8c00008/0x1000
>> fllast 1014 in agf 7 too large (max = 1014)
>> Metadata corruption detected at xfs_agf block 0x6400008/0x1000
>> fllast 1014 in agf 5 too large (max = 1014)
>> Metadata corruption detected at xfs_agf block 0x5000008/0x1000
>> fllast 1014 in agf 4 too large (max = 1014)
>> Metadata corruption detected at xfs_agf block 0x7800008/0x1000
>> fllast 1014 in agf 6 too large (max = 1014)
>> - found root inode chunk
>
> That's caused by commit 96f859d ("libxfs: pack the agfl header
> structure so XFS_AGFL_SIZE is correct") and growfs setting the
> last free list entry to something that a userspace without the
> commit 9fccb9f ("libxfs: pack the agfl header structure so
> XFS_AGFL_SIZE is correct") fails to validate correctly.
>
> i.e. update userspace to 4.5.0, and the repair noise will go away.
Yep. The problem is not reproducible with xfsprogs 4.5.0.
--
Chris Murphy
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: new fs, xfs_admin new label, metadata corruption detected
2016-03-17 23:21 ` Chris Murphy
@ 2016-03-18 2:22 ` Dave Chinner
2016-03-18 4:43 ` Zorro Lang
1 sibling, 0 replies; 13+ messages in thread
From: Dave Chinner @ 2016-03-18 2:22 UTC (permalink / raw)
To: Chris Murphy; +Cc: Eric Sandeen, xfs@oss.sgi.com
On Thu, Mar 17, 2016 at 05:21:51PM -0600, Chris Murphy wrote:
> On Thu, Mar 17, 2016 at 2:57 PM, Dave Chinner <david@fromorbit.com> wrote:
> > On Wed, Mar 16, 2016 at 08:39:21PM -0600, Chris Murphy wrote:
> >> OK I can consistently reproduce this on the CLI. I end up with totally different
> >>
> >> # lvcreate -L 40g VG -n testxfs
> >> Logical volume "testxfs" created.
> > ....
> > [grow]
> > ....
> >> # xfs_repair -n /dev/VG/testxfs
> >> Phase 1 - find and verify superblock...
> >> Phase 2 - using internal log
> >> - zero log...
> >> - scan filesystem freespace and inode maps...
> >> Metadata corruption detected at xfs_agf block 0x8c00008/0x1000
> >> fllast 1014 in agf 7 too large (max = 1014)
> >> Metadata corruption detected at xfs_agf block 0x6400008/0x1000
> >> fllast 1014 in agf 5 too large (max = 1014)
> >> Metadata corruption detected at xfs_agf block 0x5000008/0x1000
> >> fllast 1014 in agf 4 too large (max = 1014)
> >> Metadata corruption detected at xfs_agf block 0x7800008/0x1000
> >> fllast 1014 in agf 6 too large (max = 1014)
> >> - found root inode chunk
> >
> > That's caused by commit 96f859d ("libxfs: pack the agfl header
> > structure so XFS_AGFL_SIZE is correct") and growfs setting the
> > last free list entry to something that a userspace without the
> > commit 9fccb9f ("libxfs: pack the agfl header structure so
> > XFS_AGFL_SIZE is correct") fails to validate correctly.
> >
> > i.e. update userspace to 4.5.0, and the repair noise will go away.
>
> Yep. The problem is not reproducible with xfsprogs 4.5.0.
Thanks for confirming that, Chris.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: new fs, xfs_admin new label, metadata corruption detected
2016-03-17 23:21 ` Chris Murphy
2016-03-18 2:22 ` Dave Chinner
@ 2016-03-18 4:43 ` Zorro Lang
1 sibling, 0 replies; 13+ messages in thread
From: Zorro Lang @ 2016-03-18 4:43 UTC (permalink / raw)
To: Chris Murphy; +Cc: Eric Sandeen, xfs@oss.sgi.com
On Thu, Mar 17, 2016 at 05:21:51PM -0600, Chris Murphy wrote:
> On Thu, Mar 17, 2016 at 2:57 PM, Dave Chinner <david@fromorbit.com> wrote:
> > On Wed, Mar 16, 2016 at 08:39:21PM -0600, Chris Murphy wrote:
> >> OK I can consistently reproduce this on the CLI. I end up with totally different
> >>
> >> # lvcreate -L 40g VG -n testxfs
> >> Logical volume "testxfs" created.
> > ....
> > [grow]
> > ....
> >> # xfs_repair -n /dev/VG/testxfs
> >> Phase 1 - find and verify superblock...
> >> Phase 2 - using internal log
> >> - zero log...
> >> - scan filesystem freespace and inode maps...
> >> Metadata corruption detected at xfs_agf block 0x8c00008/0x1000
> >> fllast 1014 in agf 7 too large (max = 1014)
> >> Metadata corruption detected at xfs_agf block 0x6400008/0x1000
> >> fllast 1014 in agf 5 too large (max = 1014)
> >> Metadata corruption detected at xfs_agf block 0x5000008/0x1000
> >> fllast 1014 in agf 4 too large (max = 1014)
> >> Metadata corruption detected at xfs_agf block 0x7800008/0x1000
> >> fllast 1014 in agf 6 too large (max = 1014)
> >> - found root inode chunk
> >
> > That's caused by commit 96f859d ("libxfs: pack the agfl header
> > structure so XFS_AGFL_SIZE is correct") and growfs setting the
> > last free list entry to something that a userspace without the
> > commit 9fccb9f ("libxfs: pack the agfl header structure so
> > XFS_AGFL_SIZE is correct") fails to validate correctly.
> >
> > i.e. update userspace to 4.5.0, and the repair noise will go away.
>
> Yep. The problem is not reproducible with xfsprogs 4.5.0.
HaHa, Dave always knows more XFS change history:)
BTW, this bug don't need LVM resize, just need
---
mkfs.xfs -d size=$half_size $dev
mount $dev $mnt
xfs_growfs -d $mnt
umount $mnt
xfs_repair -n $dev
---
to reproduce it.
And I think xfstests xfs/015 can cover this test.
Thanks,
Zorro
>
>
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2016-03-18 4:43 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-03-15 23:08 new fs, xfs_admin new label, metadata corruption detected Chris Murphy
2016-03-16 0:34 ` Eric Sandeen
2016-03-16 7:23 ` Chris Murphy
2016-03-17 2:39 ` Chris Murphy
2016-03-17 2:40 ` Chris Murphy
2016-03-17 2:55 ` Chris Murphy
2016-03-17 6:39 ` Chris Murphy
2016-03-17 7:32 ` Zorro Lang
2016-03-17 20:57 ` Dave Chinner
2016-03-17 21:56 ` Dave Chinner
2016-03-17 23:21 ` Chris Murphy
2016-03-18 2:22 ` Dave Chinner
2016-03-18 4:43 ` Zorro Lang
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox