Volume Group and XFS Partition Recovery

Volume Group and XFS Partition Recovery

[Andrew Ragone]

[-- Attachment #1.1: Type: text/plain, Size: 3502 bytes --]

Hey everyone:

I hope everyone's summers have been exciting so far. The other day mine had
a little hiccup thrown in there that I was hoping some of the bright members
of this group could shed some light on. Rundown of the problem is as
follows:

Alright so I have an XFS partition inside of a volume group on my server.
It's been configured as such for awhile and is functional. Regardless, I
recently added some drives and expanded the raid on my 3ware card
successfully from 14 TB to 20TB. Of course the next step is to expand the
partition and then the file system. (The partition lives in a Volume Group
in a Logical Volume). I have done this entire process once before
successfully without data loss when expanding from 10TB to 14TB but for some
reason it all blew up in my face this time. Commands used were similar to
this:

umount -a (unmount all partitions)

      vgchange -an (disable all volume groups)
      partprobe (reread partition tables on all disks)
      parted /dev/sda resize 1 0.017 267466 (resize the partition)
      partprobe (reread partition tables on all disks)
      pvresize /dev/<disk_partition> (resize the physical volume << seems to
be about the point this went awry)
      vgchange -ay (enable all volume groups)
      mount -a (remount all disk partitions)

Basically I've got things in this strange state where the partition
/dev/sda1 (being based off the logical block device for the raid) looks like
it has successfully expanded from End Cylinder 1945212 (previous config) to
267466 (using all 20 TB) but can't be mounted. It actually seems to have
initially disappeared from the volume group listing (correct me if this
terminology is wrong): initially the block device was mapped to
/dev/oasis/puddle yet /dev/oasis is now empty. So to "recapture" the
partition I tried to all a new v
  olume group now called vol0x that should house the /dev/sda1 physical
volume. Of course in /dev/oasis I show only /dev/oasis/vol0x and not
/dev/oasis/puddle

On this, my question is* can I reconfigure volume groups and logical volumes
on the fly in order to "recapture" the actual partition without affecting
the data?* If I am correct, the data is only a simple mapping in the GPT
table on the first sector of the drive.

Further, the actual volume group that houses the XFS file system now exists
as "lvm2pv" which I'm fairly certain needs to be "xfs". Should this be the
case?

I've shut down the machine so no data is accessed, attempted to be accessed,
etc. until I can get some ideas on recovery approach.

   - Basically, can anyone provide some insight on what order LVs, PVs, and
   VGs need to be configured to work properly?
   - What are some approaches to attempt to grab this data back?
      - Is anyone familiar with what xfs_repair does and how I might be able
      to apply it here?
      - Worse case scenario: having software such as TestDisk remap the file
      info, though I've read XFS is a huge pain in filesystem world to
do this on.

Somehow the mapping of the file system to the physical volume blew up and at
this point I think I just need to play with the pointers in the Logical
Volumes, Volume Groups, and file system types in the partition. But I have
had limited experience with this advanced of a problem and having to adapt 3
objects with the same parameters to work in unison and give me my data back
gets complicated very quickly and just wanted to see if there are a few
minds to bounce ideas off of and help out.

Thanks,
Andrew

[-- Attachment #1.2: Type: text/html, Size: 7169 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Volume Group and XFS Partition Recovery

[Dave Chinner]

On Sat, Jul 03, 2010 at 01:26:18PM -0400, Andrew Ragone wrote:
> Hey everyone:
> 
> I hope everyone's summers have been exciting so far. The other day mine had
> a little hiccup thrown in there that I was hoping some of the bright members
> of this group could shed some light on. Rundown of the problem is as
> follows:
> 
> Alright so I have an XFS partition inside of a volume group on my server.
> It's been configured as such for awhile and is functional. Regardless, I
> recently added some drives and expanded the raid on my 3ware card
> successfully from 14 TB to 20TB.

First question is: what kernel version?

> Of course the next step is to expand the
> partition and then the file system. (The partition lives in a Volume Group
> in a Logical Volume). I have done this entire process once before
> successfully without data loss when expanding from 10TB to 14TB but for some
> reason it all blew up in my face this time.

OK, so growing within 16TB was fine, but growing past 16TB blew up.
There's several possibilities there.

> Commands used were similar to
> this:
> 
> umount -a (unmount all partitions)
> 
>       vgchange -an (disable all volume groups)
>       partprobe (reread partition tables on all disks)
>       parted /dev/sda resize 1 0.017 267466 (resize the partition)
>       partprobe (reread partition tables on all disks)
>       pvresize /dev/<disk_partition> (resize the physical volume << seems to
> be about the point this went awry)
>       vgchange -ay (enable all volume groups)
>       mount -a (remount all disk partitions)
> 
> Basically I've got things in this strange state where the partition
> /dev/sda1 (being based off the logical block device for the raid) looks like
> it has successfully expanded from End Cylinder 1945212 (previous config) to
> 267466 (using all 20 TB) but can't be mounted.

How does decreasing the end cylinder from 1945212 to 267466 increase
the partition size? What does /proc/partitions tell you about the
partition sizes?

> It actually seems to have
> initially disappeared from the volume group listing (correct me if this
> terminology is wrong): initially the block device was mapped to
> /dev/oasis/puddle yet /dev/oasis is now empty. So to "recapture" the
> partition I tried to all a new v
>   olume group now called vol0x that should house the /dev/sda1 physical
> volume. Of course in /dev/oasis I show only /dev/oasis/vol0x and not
> /dev/oasis/puddle

Ok, so you have some LVM problem you need to sort out before even
looking at XFS. I can't really help you much there.

> On this, my question is* can I reconfigure volume groups and logical volumes
> on the fly in order to "recapture" the actual partition without affecting
> the data?* If I am correct, the data is only a simple mapping in the GPT
> table on the first sector of the drive.

If changing the LVM configuration involves writing to the drives,
then I'd say the answer is no.

> Further, the actual volume group that houses the XFS file system now exists
> as "lvm2pv" which I'm fairly certain needs to be "xfs". Should this be the
> case?

Sounds like you need to recover the LV information before worrying
about the filesystem...

> I've shut down the machine so no data is accessed, attempted to be accessed,
> etc. until I can get some ideas on recovery approach.
> 
>    - Basically, can anyone provide some insight on what order LVs, PVs, and
>    VGs need to be configured to work properly?

You'd do better asking this on the LVM lists.

>    - What are some approaches to attempt to grab this data back?

Recover you LVM configuration first before worrying about the
filesystem. Once you've got the LV back without damaging anything
else, then worry about the filesystem.

>       - Is anyone familiar with what xfs_repair does and how I might be able
>       to apply it here?

Only run it in no-modify mode (xfs_repair -n) until you are certain
everything is reconstructed correctly. If the lvm information is
reconstructed correctly then repair should most likely run without errors.

>       - Worse case scenario: having software such as TestDisk remap the file
>       info, though I've read XFS is a huge pain in filesystem world to
> do this on.

I've never used such software - you're on your own with that one.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs