XFS corruption after power surge/outage

[Jorge Garcia <jgarcia@soe.ucsc.edu>]

Hello,

We have a server with a very large (300+ TB) XFS filesystem that we
use to provide downloads to the world. Last week's storms in
California caused damage to our machine room, causing unexpected power
surges and power outages, even in our UPS and generator backed data
center. One of the end results was some data corruption on our server
(running Centos 8). After looking around the internet for solutions to
our issues, the general consensus seemed to be to run xfs_repair on
the filesystem to get it to recover. We tried that (xfs_repair V 5.0)
and it seemed to report lots of issues before eventually failing
during "Phase 6" with an error like:

  Metadata corruption detected at 0x46d6c4, inode 0x8700657ff8 dinode

  fatal error -- couldn't map inode 579827236856, err = 117

After another set of internet searches, we found some postings that
suggested this could be a bug that may have been fixed in later
versions, so we built xfs_repair V 6.5 and tried the repair again. The
results were the same. We even tried "xfs_repair -L", and no joy. So
now we're desperate. Is the data all lost? We can't mount the
filesystem. We tried using xfs_metadump (another suggestion from our
searches) and it reports lots of metadata corruption ending with:

Metadata corruption detected at 0x4382f0, xfs_cntbt block 0x1300023518/0x1000
Metadata corruption detected at 0x4382f0, xfs_cntbt block 0x1300296bf8/0x1000
Metadata corruption detected at 0x4382f0, xfs_bnobt block 0x137fffb258/0x1000
Metadata corruption detected at 0x4382f0, xfs_bnobt block 0x138009ebd8/0x1000
Metadata corruption detected at 0x467858, xfs_inobt block 0x138067f550/0x1000
Metadata corruption detected at 0x467858, xfs_inobt block 0x13834b39e0/0x1000
xfs_metadump: bad starting inode offset 5

Not sure what to try next. Any help would be greatly appreciated. Thanks!

Jorge