[linux-lvm] How to 'copy' a volume?

[linux-lvm] How to 'copy' a volume?

[Erich Weiler]

Greetings all-

I have a problem I'm trying to solve, was hoping someone would know how 
to get around it...

I have a 2TB volume group, and one 500GB volume in it.  There is a Xen 
VM in that volume group.  What I'd like to do is 'copy' the volume and 
name it something else so I can use it as a 'template' for other VMs.

I know that LVM has the 'snapshot' capability.  But this doesn't look 
like it's what I need, as I don't want my duplicate volume to have any 
affiliation with the original at all.  I actually want to duplicate VM 
to take up just as much space as the first and be completely independent 
of any changes on the first.  Is there a way of achieving this?  Could I 
maybe simply make sure the volume is unmounted and not in use, then copy 
the /dev/mapper/myvolume file to something else?  I bet it's more 
involved than that...  :)

Thanks in advance for any info!!

-erich

Re: [linux-lvm] How to 'copy' a volume?

[Stuart D. Gathman]

On Thu, 3 Jan 2008, Erich Weiler wrote:

> I know that LVM has the 'snapshot' capability.  But this doesn't look 
> like it's what I need, as I don't want my duplicate volume to have any 
> affiliation with the original at all.  I actually want to duplicate VM 
> to take up just as much space as the first and be completely independent 
> of any changes on the first.  Is there a way of achieving this?  Could I 
> maybe simply make sure the volume is unmounted and not in use, then copy 
> the /dev/mapper/myvolume file to something else?  I bet it's more 
> involved than that...  :)

That's all these is to it.  You can also take a snapshot first, then
copy the snapshot.  With a journalling filesystem (e.g. ext3), it will recover
the copy when mounted.  Caveat: if you are mounting by filesystem label
(LABEL=foo), then both the copy and the snapshot will have duplicate labels,
and the filesystem will refuse to mount.  Use e2label (for ext3) to change the
label of the copy.  If not mounting by label, the duplicates won't matter.

-- 
	      Stuart D. Gathman <stuart@bmsi.com>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

Re: [linux-lvm] How to 'copy' a volume?

[David Robinson]

Erich Weiler wrote:
> Greetings all-
> 
> I have a problem I'm trying to solve, was hoping someone would know how 
> to get around it...
> 
> I have a 2TB volume group, and one 500GB volume in it.  There is a Xen 
> VM in that volume group.  What I'd like to do is 'copy' the volume and 
> name it something else so I can use it as a 'template' for other VMs.
> 
> I know that LVM has the 'snapshot' capability.  But this doesn't look 
> like it's what I need, as I don't want my duplicate volume to have any 
> affiliation with the original at all.  I actually want to duplicate VM 
> to take up just as much space as the first and be completely independent 
> of any changes on the first.  Is there a way of achieving this?  Could I 
> maybe simply make sure the volume is unmounted and not in use, then copy 
> the /dev/mapper/myvolume file to something else?  I bet it's more 
> involved than that...  :)

I find 'dd' is the simplest way of doing this. Unmount the volume, 
create a new logical volume of equal or greater size, then 'dd' from one 
to the other:

lvcreate -L 500G -n clone volgroup00
dd if=/dev/volgroup00/original of=/dev/volgroup/clone

Rather than copying VMs I snapshot them. I create a template system, 
create a snapshot of it, then modify the VMs config so it uses the 
snapshot as its disk (so the original is unmodified). If I need another 
copy of the VM its only a matter of creating another snapshot of the 
template LV, then copying and modifying the config. I usually set the 
extent size low so that modifications within the VMs don't chew up lots 
of space too.

--Dave

Re: [linux-lvm] How to 'copy' a volume?

[Erich Weiler]

David Robinson wrote:
> I find 'dd' is the simplest way of doing this. Unmount the volume, 
> create a new logical volume of equal or greater size, then 'dd' from one 
> to the other:
> 
> lvcreate -L 500G -n clone volgroup00
> dd if=/dev/volgroup00/original of=/dev/volgroup/clone

Hmmm...  I think it didn't work as expected.  I brought down the VM and 
made sure that no LVM volumes were mounted, then:

# lvcreate -L 500G -n clone volgroup00
# dd if=/dev/volgroup00/original of=/dev/volgroup00/clone

The machine then had a kernel panic (I think).  I'm remote so I'm not 
sure, but I'll be going in soon to check on it.  Not a critical system 
so no big deal.  I'll let everyone one what I see at the console.

-erich

Re: [linux-lvm] How to 'copy' a volume?

[Stuart D. Gathman]

On Fri, 4 Jan 2008, Erich Weiler wrote:

> # lvcreate -L 500G -n clone volgroup00
> # dd if=/dev/volgroup00/original of=/dev/volgroup00/clone
> 
> The machine then had a kernel panic (I think).  I'm remote so I'm not 
> sure, but I'll be going in soon to check on it.  Not a critical system 
> so no big deal.  I'll let everyone one what I see at the console.

Could be very slow.  On one of my systems, a blocksize of 300k is optimal.
The default block size is usually 512 bytes.

# dd if=/dev/volgroup00/original of=/dev/volgroup00/clone bs=300k

-- 
	      Stuart D. Gathman <stuart@bmsi.com>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

Re: [linux-lvm] How to 'copy' a volume?

[Erich Weiler]

Stuart D. Gathman wrote:
>> The machine then had a kernel panic (I think).  I'm remote so I'm not 
>> sure, but I'll be going in soon to check on it.  Not a critical system 
>> so no big deal.  I'll let everyone one what I see at the console.
> 
> Could be very slow.  On one of my systems, a blocksize of 300k is optimal.
> The default block size is usually 512 bytes.
> 
> # dd if=/dev/volgroup00/original of=/dev/volgroup00/clone bs=300k

It was probably slow, as you say, but it turns out our machines were 
powered off by one of the other guys in our group because the UPS was 
draining fast.  Power was lost because of a major storm on the central 
California coast.

I have a feeling it was working but was interrupted.  I'll try again 
when power is restored...

Thanks so much for your input!

-erich

Re: [linux-lvm] How to 'copy' a volume?

[Stuart D. Gathman]

On Fri, 4 Jan 2008, Erich Weiler wrote:

> > # dd if=/dev/volgroup00/original of=/dev/volgroup00/clone bs=300k
> 
> It was probably slow, as you say, but it turns out our machines were 
> powered off by one of the other guys in our group because the UPS was 
> draining fast.  Power was lost because of a major storm on the central 
> California coast.

You can use iostat to see throughput while dd is in progress.  If
you're ambitious, you can try various blocksizes to find the optimum
for your system.

Oh.  And *don't* get source and target mixed up with dd.  I did that
once ....

-- 
	      Stuart D. Gathman <stuart@bmsi.com>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

Re: [linux-lvm] How to 'copy' a volume?

[Erich Weiler]

OK, cool, this worked great!  Was able to dd the contents of one LV to
another and it worked.

So the next thing I did was reduce the size of one of the filesystems on
the Xen VM from 500GB to 5GB.  Then I reduced the size of the Logical
Volume that the VM was sitting on from 524GB to 30GB.  Again, worked fine,
still enough space to house all the data in the VM.  The filesystem resize worked fine.  Then
I rebooted the VM.  Now I'm getting odd error during the VM's boot that say
something about the disk being not the correct size, or something along
those lines.

[root@xenvm ~]# fdisk -l

Disk /dev/xvda: 32.2 GB, 32212254720 bytes
255 heads, 63 sectors/track, 3916 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

    Device Boot      Start         End      Blocks   Id  System
/dev/xvda1   *           1          13      104391   83  Linux
/dev/xvda2              14        1288    10241437+  83  Linux
/dev/xvda3            1289        2308     8193150   82  Linux swap /
Solaris
/dev/xvda4            2309       67620   524618640    5  Extended
/dev/xvda5            2309       67620   524618608+  83  Linux
[root@xenvm ~]#

Notice it says that /dev/xvda is 32.2GB (which is what I want, and is
correct), but the
filesystem that I reduced from 500GB to 5GB, /dev/xvda5, is still listed
as being 524GB, even though it is no longer that big!  Even though the
filesystem on /dev/xvda5 is just 5GB:

[root@xenvm ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda2            9.5G  2.0G  7.1G  22% /
/dev/xvda5            5.0G  4.4G  365M  93% /data
/dev/xvda1             99M   19M   76M  20% /boot
tmpfs                 2.0G     0  2.0G   0% /dev/shm
[root@xenvm ~]#

This makes me think the disk label is not showing the correct block device
size.  The VM still boots and runs fine despite the errors, but I'd like
to fix it because I'll be resizing stuff often and want things to go
smoothly when the resizes happen.  Does anyone know of a way to correct
this?

Thanks so much for the help!

-erich

On Fri, 4 Jan 2008, Stuart D. Gathman wrote:

> On Fri, 4 Jan 2008, Erich Weiler wrote:
>
> > > # dd if=/dev/volgroup00/original of=/dev/volgroup00/clone bs=300k
> >
> > It was probably slow, as you say, but it turns out our machines were
> > powered off by one of the other guys in our group because the UPS was
> > draining fast.  Power was lost because of a major storm on the central
> > California coast.
>
> You can use iostat to see throughput while dd is in progress.  If
> you're ambitious, you can try various blocksizes to find the optimum
> for your system.
>
> Oh.  And *don't* get source and target mixed up with dd.  I did that
> once ....
>
> --
> 	      Stuart D. Gathman <stuart@bmsi.com>
>     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
> "Confutatis maledictis, flammis acribus addictis" - background song for
> a Microsoft sponsored "Where do you want to go from here?" commercial.
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>

Re: [linux-lvm] How to 'copy' a volume?

[Stuart D. Gathman]

On Mon, 7 Jan 2008, Erich Weiler wrote:

> OK, cool, this worked great!  Was able to dd the contents of one LV to
> another and it worked.
> 
> So the next thing I did was reduce the size of one of the filesystems on
> the Xen VM from 500GB to 5GB.  Then I reduced the size of the Logical
> Volume that the VM was sitting on from 524GB to 30GB.  Again, worked fine,
> still enough space to house all the data in the VM.  The filesystem resize
> worked fine.  Then I rebooted the VM.  Now I'm getting odd error during the
> VM's boot that say something about the disk being not the correct size, or
> something along those lines.
> 
> [root@xenvm ~]# fdisk -l
> 
> Disk /dev/xvda: 32.2 GB, 32212254720 bytes
> 255 heads, 63 sectors/track, 3916 cylinders
> Units = cylinders of 16065 * 512 = 8225280 bytes
> 
>     Device Boot      Start         End      Blocks   Id  System
> /dev/xvda1   *           1          13      104391   83  Linux
> /dev/xvda2              14        1288    10241437+  83  Linux
> /dev/xvda3            1289        2308     8193150   82  Linux swap /
> Solaris
> /dev/xvda4            2309       67620   524618640    5  Extended
> /dev/xvda5            2309       67620   524618608+  83  Linux
> [root@xenvm ~]#
> 
> Notice it says that /dev/xvda is 32.2GB (which is what I want, and is
> correct), but the
> filesystem that I reduced from 500GB to 5GB, /dev/xvda5, is still listed
> as being 524GB, even though it is no longer that big!  Even though the
> filesystem on /dev/xvda5 is just 5GB:

You didn't tell us that the LV you were reducing contained an whole disk
including partition table.  You will need to resize the partition table
also - and I am not aware of any tools to make that easy.  You should
be able to just delete and reallocate the partition from within the VM,
but I haven't tried it.  When resizing the entire disk, you'll need to 
leave plenty of extra room for the other partitions as well.   Sounds
like you did, leaving 25G extra for 18G worth of other partitions.

I generally export an LV for each slice, like this (for xen):

disk       = [ 'phy:mapper/rootvg-USEXP,sda1,w',
               'phy:mapper/rootvg-GENSWAP,sda2,w' ]

As far as I'm concerned, DOS partition tables are obsolete.  I just put
LVM directly onto physical devices and don't bother with partition tables.
The only exception is that most versions of grub in the field still require a 
DOS partition table for a boot partition.  But the latest and greatest
grub understands lvm.  Yay!

-- 
	      Stuart D. Gathman <stuart@bmsi.com>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.