[linux-lvm] Best way to image devices in an LVM environment

[linux-lvm] Best way to image devices in an LVM environment

[Andy Smith]

[-- Attachment #1: Type: text/plain, Size: 1488 bytes --]

Hello,

I have a server I'm currently finalising the setup of, which uses
two 80G SATA drives in a RAID-1 (md) with LVM on top.  When I've got it
how I want it I need to replace the 80G drives with a much larger
pair.

Now, I've kept notes about my setup of course so I can just put the
new disks in and do it all over again, but that strikes me as a big
waste of time.

I also of course have space elsewhere to take an image of the md
devices with dd and I could copy them back over the top once md is
set up again on the new disks.  The majority of space though is in
my single LVM volume group and only 18.5G of this 72.5G is used, so
taking an 80G image seems to also be rather a waste of time.

Here's the setup:

/dev/md0 (sda1, sdb1) is swap
/dev/md1 (sda2, sdb2) is /boot
/dev/md2 (sda3, sdb3) is /
/dev/md4 (sda5, sdb5) is an LVM PV and VG 'mainvg'

Everything else is LVs within mainvg and all filesystems are ext3.

So how would you accomplish the task of replacing sda and sdb with
larger disks in the least amount of effort?  Downtime is not a
problem, taking backups of the data is not a problem, booting off a
live cd is not a problem; the goal is to find the least
labour-intensive way to do this.

The best I can think of booting off a live cd, taking a dd of each
md device to another server, installing the new drives, setting up
the md devices again and then overwriting them with the images.
Will I run into problems with their UUIDs being overwritten?

Cheers,
Andy

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [linux-lvm] Best way to image devices in an LVM environment

[Jim Morgan]

At 27/08/2006 12:42, you wrote:
>So how would you accomplish the task of replacing sda and sdb with
>larger disks in the least amount of effort?  Downtime is not a
>problem, taking backups of the data is not a problem, booting off a
>live cd is not a problem; the goal is to find the least
>labour-intensive way to do this.

As I understand it -- and as I've proved before in this list, my 
understanding is fundamentally flawed :-) -- there is a way to do this.

Basically you have to force one disk to fail using the raid 
administration software. You then remove the disk, replace it with 
the bigger disk, and rebuild it from the first one. The partition 
sizes may need adjusting. Then you force the other disk to fail, and 
replace it with the larger one, and then rebuild it from the other large disk.

Hopefully someone can flesh this out for me, or you may now have a 
starting point for some googling.

Jim

Re: [linux-lvm] Best way to image devices in an LVM environment

[Andy Smith]

[-- Attachment #1: Type: text/plain, Size: 1267 bytes --]

On Sun, Aug 27, 2006 at 01:20:26PM +0800, Jim Morgan wrote:
> At 27/08/2006 12:42, you wrote:
> >So how would you accomplish the task of replacing sda and sdb with
> >larger disks in the least amount of effort?  Downtime is not a
> >problem, taking backups of the data is not a problem, booting off a
> >live cd is not a problem; the goal is to find the least
> >labour-intensive way to do this.
> 
> As I understand it -- and as I've proved before in this list, my 
> understanding is fundamentally flawed :-) -- there is a way to do this.
> 
> Basically you have to force one disk to fail using the raid 
> administration software. You then remove the disk, replace it with 
> the bigger disk, and rebuild it from the first one. The partition 
> sizes may need adjusting. Then you force the other disk to fail, and 
> replace it with the larger one, and then rebuild it from the other large 
> disk.

Well I would then be left with two large disks where sd[ab]5 (and
the sd[ab]4 primary partition that sd[ab]5 is the only logical
partition within) does not extend all the way to the end of the
disk.  I wasn't aware that you could resize RAID-1 md arrays though,
I thought it was restricted to RAID-5.  Is this not the case?

Cheers,
Andy

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [linux-lvm] Best way to image devices in an LVM environment

[Matthew Gillen]

Andy Smith wrote:
> On Sun, Aug 27, 2006 at 01:20:26PM +0800, Jim Morgan wrote:
>> At 27/08/2006 12:42, you wrote:
>>> So how would you accomplish the task of replacing sda and sdb with
>>> larger disks in the least amount of effort?  Downtime is not a
>>> problem, taking backups of the data is not a problem, booting off a
>>> live cd is not a problem; the goal is to find the least
>>> labour-intensive way to do this.
>> As I understand it -- and as I've proved before in this list, my 
>> understanding is fundamentally flawed :-) -- there is a way to do this.
>>
>> Basically you have to force one disk to fail using the raid 
>> administration software. You then remove the disk, replace it with 
>> the bigger disk, and rebuild it from the first one. The partition 
>> sizes may need adjusting. Then you force the other disk to fail, and 
>> replace it with the larger one, and then rebuild it from the other large 
>> disk.
> 
> Well I would then be left with two large disks where sd[ab]5 (and
> the sd[ab]4 primary partition that sd[ab]5 is the only logical
> partition within) does not extend all the way to the end of the
> disk.  I wasn't aware that you could resize RAID-1 md arrays though,
> I thought it was restricted to RAID-5.  Is this not the case?

To add another "I'm not an expert" opinion:
Supposing you can get all 4 disks in the server at the same time, you could
add your new disk's large partition to your existing logical volume, then do
a pvmove.  The advantage over the previous suggestion is that you don't need
to resize your RAID device when you're done (although you'd still want to
resize your filesystem to take advantage of all the new space).

Although now that I look at the instructions for doing this
(http://tldp.org/HOWTO/LVM-HOWTO/removeadisk.html; section 13.5.1), it seems
that is a dangerous operation.  If it worked though, it would probably be
the least labor intensive.

Matt

Re: [linux-lvm] Best way to image devices in an LVM environment

[Andy Smith]

[-- Attachment #1: Type: text/plain, Size: 2301 bytes --]

On Sun, Aug 27, 2006 at 07:51:09AM -0400, Matthew Gillen wrote:
> Andy Smith wrote:
> > On Sun, Aug 27, 2006 at 01:20:26PM +0800, Jim Morgan wrote:
> >> Basically you have to force one disk to fail using the raid 
> >> administration software. You then remove the disk, replace it with 
> >> the bigger disk, and rebuild it from the first one. The partition 
> >> sizes may need adjusting. Then you force the other disk to fail, and 
> >> replace it with the larger one, and then rebuild it from the other large 
> >> disk.
> > 
> > Well I would then be left with two large disks where sd[ab]5 (and
> > the sd[ab]4 primary partition that sd[ab]5 is the only logical
> > partition within) does not extend all the way to the end of the
> > disk.  I wasn't aware that you could resize RAID-1 md arrays though,
> > I thought it was restricted to RAID-5.  Is this not the case?
> 
> To add another "I'm not an expert" opinion:
> Supposing you can get all 4 disks in the server at the same time,

I can't; it's a rackmount server with only two disk bays and two
SATA ports on the motherboard.  I could have it open with another
2-port SATA controller on a PCI card, but this seems to be going a
bit far.

Since I only want to give the extra space to the single volume group
then it seems I should be able to follow this method:

http://www.debian-administration.org/articles/424

(basically involves doing what Jim said: failing out one disk,
replacing it with a bigger one, failing the other, replacing it,
then expanding partitions and filesystems.)

Seems that "mdadm --grow" works even with a RAID-1 which I wasn't
aware of.

The only problem then remaining would be how would I grow the single
pv/vg out to the full extent of the new larger block device it would
be on.  There doesn't seem to be a pvresize in Debain Sarge's lvm2
package..

  LVM version:     2.01.04 (2005-02-09)
  Library version: 1.01.00-ioctl (2005-01-17)
  Driver version:  4.5.0

I see from:

http://www.robot101.net/2005/05/20/resizing-lvm2-physical-volumes/

that lvm2 has had pvresize since 2.02.00 so if I upgrade userland
tools (my kernel is 2.6.16.19) then would I just need to do a
pvresize after expanding the block device that my single pv/vg is
on?

Cheers,
Andy

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [linux-lvm] Best way to image devices in an LVM environment

[Graham Wood]

On Mon, Aug 28, 2006 at 07:48:43PM +0000, Andy Smith wrote:
> that lvm2 has had pvresize since 2.02.00 so if I upgrade userland
> tools (my kernel is 2.6.16.19) then would I just need to do a
> pvresize after expanding the block device that my single pv/vg is
> on?
Would it not be easier to just replace with a larger disk - mirror to the same size, and 
have another partition afterwards.

You end up with 2pvs (both mirrored, I would expect), rather than the "nicer" answer of a 
single pv, but as long as the whole thing is then controlled through LVM is there really any 
loss as a result?

Graham

Re: [linux-lvm] Best way to image devices in an LVM environment

[Andy Smith]

[-- Attachment #1: Type: text/plain, Size: 1050 bytes --]

On Mon, Aug 28, 2006 at 08:51:46PM +0100, Graham Wood wrote:
> On Mon, Aug 28, 2006 at 07:48:43PM +0000, Andy Smith wrote:
> > that lvm2 has had pvresize since 2.02.00 so if I upgrade userland
> > tools (my kernel is 2.6.16.19) then would I just need to do a
> > pvresize after expanding the block device that my single pv/vg is
> > on?
> Would it not be easier to just replace with a larger disk - mirror to the same size, and 
> have another partition afterwards.
> 
> You end up with 2pvs (both mirrored, I would expect), rather than the "nicer" answer of a 
> single pv, but as long as the whole thing is then controlled through LVM is there really any 
> loss as a result?

Hmm yes as you say I think I could just make another partition in
the empty space, RAID-1 that, put a PV on the new MD device and then
vgextend onto it I suppose.  It just seems a litlle messy though to
needlessly have the extra partitions and if I can fix it before the
machine is put in production then I feel like I probably should.

Cheers,
Andy

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]