[linux-lvm] dealing with a failing drive (esp pvmove)

[linux-lvm] dealing with a failing drive (esp pvmove)

[Ross Boylan]

I have a volume group made from 3 partitions on 2 different physical
disks.  One of the disks is failing.  I have a new disk connected via
USB; it is large but a bit slow (5400RPM, though the USB might be
temporary).  It is large enough to hold the entire contents of the
existing VG.

I am looking for suggestions about how to  approach this.  The pvmove
command looks promising, but the docs warn that things are in a delicate
state during the migration (and advise backups).  Since it seems almost
certain there will be I/O errors on the old drive, I'm not sure if
pvmove is a good idea.  Is it?

Alternately, I could do a file system level rsync, but I would probably
need to boot into an alternate system for the last step and there are
quite a few LVs.

Would any kind of RAID/mirroring be an option?  My impression is I need
to set that up on initial creation of the VG.

BTW, any advice on partitioning the new drive?  I know I could just use
the whole physical device, but I'm concerned that if I want to boot
using the disk, or install another (non-Linux) OS that I'll need more
traditional partitions on the drive.  My first cut was to create a GPT
partitioning scheme with a 1G blank paritition and partition 2 holding
the rest of the 2TB for LVM.  Linux is my primary OS.

Thanks.
Ross Boylan

Re: [linux-lvm] dealing with a failing drive (esp pvmove)

[Richard Shaw]

On Wed, Dec 22, 2010 at 1:55 PM, Ross Boylan <ross@biostat.ucsf.edu> wrote:
> I have a volume group made from 3 partitions on 2 different physical
> disks. �One of the disks is failing. �I have a new disk connected via
> USB; it is large but a bit slow (5400RPM, though the USB might be
> temporary). �It is large enough to hold the entire contents of the
> existing VG.
>
> I am looking for suggestions about how to �approach this. �The pvmove
> command looks promising, but the docs warn that things are in a delicate
> state during the migration (and advise backups). �Since it seems almost
> certain there will be I/O errors on the old drive, I'm not sure if
> pvmove is a good idea. �Is it?

I can't find consistent information but I think you're on the right
path. It will try to copy all the data. I'm not sure how it deals with
errors. If you can get it to continue through errors you should get
all the files/extents possible.You'll definitely need to fsck it
afterwards.

Using this link[1] as a guide, I think the general process would be
something like this:
vgextend /dev/<new_disk>
pvmove /dev/<old_disk>
vgreduce <vol_group> /dev/<old_disk>


> Alternately, I could do a file system level rsync, but I would probably
> need to boot into an alternate system for the last step and there are
> quite a few LVs.

This would keep the existing data on the old drive so you could get
more than one go at it, but I think that's about it.

The only other option I can think of would be to do a clone of your
old drive/partition (i.e. Clonezilla or similar). I haven't used it
though so I don't know if it can resize a LV on restore or not.

I'm not sure how LVM will handle that though. The disk UUID will be
different but the partition (PV) UUID will be the same as the old PV.
I'm not sure if LVM uses both or just the partition UUID.


> BTW, any advice on partitioning the new drive? �I know I could just use
> the whole physical device, but I'm concerned that if I want to boot
> using the disk, or install another (non-Linux) OS that I'll need more
> traditional partitions on the drive. �My first cut was to create a GPT
> partitioning scheme with a 1G blank paritition and partition 2 holding
> the rest of the 2TB for LVM. �Linux is my primary OS.

That should be fine as long as everything you plan to use with it
supports GPT partitions. Also keep in mind that if you use GPT w/ dos
compatibility you're limited to 4 partitions total.

Richard

Re: [linux-lvm] dealing with a failing drive (esp pvmove)

[Stuart D. Gathman]

On Wed, 22 Dec 2010, Ross Boylan wrote:

> I have a volume group made from 3 partitions on 2 different physical
> disks.  One of the disks is failing.  I have a new disk connected via
> USB; it is large but a bit slow (5400RPM, though the USB might be
> temporary).  It is large enough to hold the entire contents of the
> existing VG.
> 
> I am looking for suggestions about how to  approach this.  The pvmove
> command looks promising, but the docs warn that things are in a delicate
> state during the migration (and advise backups).  Since it seems almost
> certain there will be I/O errors on the old drive, I'm not sure if
> pvmove is a good idea.  Is it?

If it is not necessary to keep the VG live, the following will handle
errors better and be faster (depending on how much of the failing 
PV is in use):

1) Make a new partition(s) at least as big as the failing partition, and use
ddrescue to copy the failing partition(s).  Don't do anything that might
cause a vgscan (lvm commands) during the copy (since the copy will have the
same UUID).  (Or else do the copy with the VG offline, booted from CD
if needed.)

2) Shutdown and remove the failing drive.  The new copy should be 
recognized by its UUID.

3) Record blocks skipped due to io errors, and map them to files
and filesystems to take appropriate action: 

  http://smartmontools.sourceforge.net/badblockhowto.html#id3073907

-- 
	      Stuart D. Gathman <stuart@bmsi.com>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.