Re: Need help to recover root filesystem after a power supply issue

Eric Sandeen
2019-07-10T14:23:44Z

On 7/10/19 8:58 AM, Andrey Zhunev wrote:
> Wednesday, July 10, 2019, 4:26:14 PM, you wrote:
>
>> On 7/10/19 4:56 AM, Andrey Zhunev wrote:
>>> Hello All,
>>>
>>> I am struggling to recover my system after a PSU failure, and I was
>>> suggested to ask here for support.
>>>
>>> One of the hard drives throws some read errors, and that happen to be
>>> my root drive...
>>> My system is CentOS 7, and the root partition is a part of LVM.
>>>
>>> [root@mgmt ~]# lvscan
>>> ACTIVE '/dev/centos/root' [<98.83 GiB] inherit
>>> ACTIVE '/dev/centos/home' [<638.31 GiB] inherit
>>> ACTIVE '/dev/centos/swap' [<7.52 GiB] inherit
>>> [root@mgmt ~]#
>>>
>>> [root@tftp ~]# file -s /dev/centos/root
>>> /dev/centos/root: symbolic link to `../dm-3'
>>> [root@tftp ~]# file -s /dev/centos/home
>>> /dev/centos/home: symbolic link to `../dm-4'
>>> [root@tftp ~]# file -s /dev/dm-3
>>> /dev/dm-3: SGI XFS filesystem data (blksz 4096, inosz 256, v2 dirs)
>>> [root@tftp ~]# file -s /dev/dm-4
>>> /dev/dm-4: SGI XFS filesystem data (blksz 4096, inosz 256, v2 dirs)
>>>
>>>
>>> [root@tftp ~]# xfs_repair /dev/centos/root
>>> Phase 1 - find and verify superblock...
>>> superblock read failed, offset 53057945600, size 131072, ag 2, rval -1
>>>
>>> fatal error -- Input/output error
>
>> look at dmesg, see what the kernel says about the read failure.
>
>> You might be able to use https://www.gnu.org/software/ddrescue/
>> to read as many sectors off the device into an image file as possible,
>> and that image might be enough to work with for recovery. That would be
>> my first approach:
>
>> 1) use dd-rescue to create an image file of the device
>> 2) make a copy of that image file
>> 3) run xfs_repair -n on the copy to see what it would do
>> 4) if that looks reasonable run xfs_repair on the copy
>> 5) mount the copy and see what you get
>
>> But if your drive simply cannot be read at all, this is not a filesystem
>> problem, it is a hardware problem. If this is critical data you may wish
>> to hire a data recovery service.
>
>> -Eric
>
>
> Hi Eric,
>
> Thanks for your message!
> I already started to copy the failing drive with ddrescue. This is a
> large drive, so it takes some time to complete...
>
> When I tried to run xfs_repair on the original (failing) drive, the
> xfs_repair was unable to read the superblock and then just quitted
> with an 'io error'.
> Do you think it can behave differently on a copied image ?

As I said, look at dmesg to see what failed on the original drive read
attempt.

ddrescue will fill unreadable sectors with 0, and then of course that
can be read from the image file.

-Eric

> I will definitely give it a try once the ddrescue finishes.
>
>
> P.S. The data on this drive is not THAT critical to hire a
> professional data recovery service. Still, there are some files I
> would really like to restore (mostly settings and configuration
> files - nothing large, but important)... This will save me weeks to
> reconfigure and get the system back to its original state...
> Backups, always make backups... yeah, I know... :(
>
>
> ---
> Best regards,
> Andrey
>