Re: xfs_repair after data corruption (not caused by xfs, but by failing nvme drive)

[Dave Chinner <david@fromorbit.com>]

On Mon, Jan 20, 2025 at 04:15:00PM +0100, Christian Brauner wrote:
> Hey,
> 
> so last week I got a nice surprise when my (relatively new) nvme drive
> decided to tell me to gf myself. I managed to recover by now and get
> pull requests out and am back in a working state.
....

> I honestly am just curious why xfs_repair fails to validate any
> superblocks.

Ditto. It should be doing the same checks as the runtime validation
code...

> This is the splat I get when mounting without norecovery,ro:
> 
> [88222.149672] XFS (dm-4): Mounting V5 Filesystem 80526d30-90c7-4347-9d9e-333db3f5353b
> [88222.632954] XFS (dm-4): Starting recovery (logdev: internal)
> [88224.056721] XFS (dm-4): Metadata CRC error detected at xfs_agfl_read_verify+0xa5/0x120 [xfs], xfs_agfl block 0xeb6f603
> [88224.057319] XFS (dm-4): Unmount and run xfs_repair
> [88224.057328] XFS (dm-4): First 128 bytes of corrupted metadata buffer:
> [88224.057338] 00000000: f1 80 cf 13 6c 73 aa 39 55 20 29 5c 2a ca ee 9a  ....ls.9U )\*...
> [88224.057346] 00000010: 5a 0f 56 de ff da 93 5a 95 f2 01 ff 9f e7 6f 86  Z.V....Z......o.
> [88224.057353] 00000020: dc 90 f4 ad 8b 7c 6d 47 87 1d b6 47 80 25 d0 d5  .....|mG...G.%..
> [88224.057359] 00000030: da 36 1c f4 ee 22 e0 f4 b4 19 9a 74 bf d2 7d 49  .6...".....t..}I
> [88224.057366] 00000040: 2e 1c 0d 62 a9 93 7b c0 53 b5 52 b7 eb 58 d3 52  ...b..{.S.R..X.R
> [88224.057371] 00000050: fc 4b 13 cc 42 c7 36 88 1d 52 28 ef c7 20 cb 39  .K..B.6..R(.. .9
> [88224.057377] 00000060: f7 db 9a 83 2c eb 23 52 b3 1a 85 bb d6 5e ff 4b  ....,.#R.....^.K
> [88224.057383] 00000070: c3 3d 88 a6 dd bf ab 2a 94 1d 2d 19 6c b5 d1 e5  .=.....*..-.l...

Yeah, that's garbage.

> With xfs_metadump I get:
> 
> > sudo xfs_metadump /dev/mapper/dm-sdd4 xfs_corrupt.metadump                                    
> 
> Superblock has bad magic number 0xa604f4c6. Not an XFS filesystem?
> Metadata CRC error detected at 0x55d6d5e1c553, xfs_agfl block 0xeb6f603/0x200

That might be complaining about a secondary superblock, not the
primary. That would explain why it mounts...

> I can generate a metadump image if that's helpful and there's interest
> in looking into this. But as I said, I've recovered so I don't want to
> waste your time.

I'd like to have a look at the metadump image of the broken fs if
you've still got it.

-Dave.
-- 
Dave Chinner
david@fromorbit.com