[linux-lvm] Question on duplicating equally-sized HDs with dd and changing VG name and PV UUIDs

[linux-lvm] Question on duplicating equally-sized HDs with dd and changing VG name and PV UUIDs

[Holger Rauch]

[-- Attachment #1: Type: text/plain, Size: 1734 bytes --]

Hi,

I got a machine having two HDs of the same size (250 GB each), devices
/dev/sda and /dev/sdb. /dev/sda contains VG rootvg which in turn contains
several LVs. VG rootvg consists of one PV, /dev/sda5. All this was set up
using LVM2.

I want to have the same HD partition/VG/LV layout on /dev/sdb with the
following differences:

- A different VG name: backupvg instead of rootvg
- A different UUID for PV /dev/sdb5 (since that's supposed to be the underlying
  PV for VG backupvg)

Purpose of that setup is to be able to easily copy filesystem contents
between filesystems located on LVs of the same name but in different VGs.
(Apart from that, the filesystems are of the same type and size).

I looked at the LVM2 FAQ, the LVM HOWTO and googled but unfortunately
there seems to be no HOWTO for this task (which still seems somewhat
common to me since many PCs these days contain 2 equally-sized HDs).

I know that

dd if=/dev/sda of=/dev/sdb

is going to give me the same partition layout but what LVM commands can I
use for the following tasks (provided that the newly created VG is not
online yet):

- Generating a new UUID and assigning that to PV /dev/sdb5
- Renaming a VG while it's offline. Does vgrename support this?
- "Telling" the renamed VG about the fact that /dev/sdb5 has got a new UUID
  but still it's supposed to be used as the underlying PV for VG backupvg

I also know that LVM2 metadata resides in /etc/lvm but I don't feel
confident playing around there without having additional information
(provided this is the right place to look at for the above mentioned tasks).

Thanks in advance for any hints, links, etc.

In case there are any additional questions, please feel free to ask.

Kind regards,

	Holger

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [linux-lvm] Question on duplicating equally-sized HDs with dd and changing VG name and PV UUIDs

[Lars Ellenberg]

On Fri, Sep 07, 2007 at 02:13:07PM +0200, Holger Rauch wrote:
> Hi,
> 
> I got a machine having two HDs of the same size (250 GB each), devices
> /dev/sda and /dev/sdb. /dev/sda contains VG rootvg which in turn contains
> several LVs. VG rootvg consists of one PV, /dev/sda5. All this was set up
> using LVM2.
> 
> I want to have the same HD partition/VG/LV layout on /dev/sdb with the
> following differences:
> 
> - A different VG name: backupvg instead of rootvg
> - A different UUID for PV /dev/sdb5 (since that's supposed to be the underlying
>   PV for VG backupvg)
> 
> Purpose of that setup is to be able to easily copy filesystem contents
> between filesystems located on LVs of the same name but in different VGs.
> (Apart from that, the filesystems are of the same type and size).

now, why would you want to dd the full disk,
and then change thr uuids?
what is wrong with
 sfdisk -d /dev/sda | sfdisk /dev/sdb
 lvs -o attr,name,size --noheadings rootvg |
 while read a n s; do
 	[[ $a == -* ]] || continue;
	lvcreate -L $s -n $n backupvg ;
	# dd bs=1M if=/dev/rootvg/$n of=/dev/backupvg/$n
 done
?

-- 
: Lars Ellenberg                            Tel +43-1-8178292-55 :
: LINBIT Information Technologies GmbH      Fax +43-1-8178292-82 :
: Vivenotgasse 48, A-1120 Vienna/Europe    http://www.linbit.com :

Re: [linux-lvm] Question on duplicating equally-sized HDs with dd and changing VG name and PV UUIDs

[Holger Rauch]

[-- Attachment #1: Type: text/plain, Size: 766 bytes --]

Hi Lars (and all the other list members),

first of all, thanks for this nice little script (and I see nothing wrong
with that :-))) ), but it assumes that VG backupvg and the underlying VG
has already been created, right? I see no pvcreate and vgcreate invocations
in there. Is this on purpose, i.e. is that already done just by duplicating
the partition table using sfdisk?

Thanks in advance for clarifying things.

Kind regards,

	Holger

On Fri, 07 Sep 2007, Lars Ellenberg wrote:

> [...]
> what is wrong with
>  sfdisk -d /dev/sda | sfdisk /dev/sdb
>  lvs -o attr,name,size --noheadings rootvg |
>  while read a n s; do
>  	[[ $a == -* ]] || continue;
> 	lvcreate -L $s -n $n backupvg ;
> 	# dd bs=1M if=/dev/rootvg/$n of=/dev/backupvg/$n
>  done
> ?
> [...]

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [linux-lvm] Question on duplicating equally-sized HDs with dd and changing VG name and PV UUIDs

[malahal]

I don't think so. /dev/sdb will have the same partition table as
/dev/sda from the sfdisk command.  I bet you need to run pvcreate and
vgcreate before running the rest of the script.

Thanks, Malahal.

Holger Rauch [holger.rauch@heitec.de] wrote:
> Hi Lars (and all the other list members),
> 
> first of all, thanks for this nice little script (and I see nothing wrong
> with that :-))) ), but it assumes that VG backupvg and the underlying VG
> has already been created, right? I see no pvcreate and vgcreate invocations
> in there. Is this on purpose, i.e. is that already done just by duplicating
> the partition table using sfdisk?
> 
> Thanks in advance for clarifying things.
> 
> Kind regards,
> 
> 	Holger
> 
> On Fri, 07 Sep 2007, Lars Ellenberg wrote:
> 
> > [...]
> > what is wrong with
> >  sfdisk -d /dev/sda | sfdisk /dev/sdb
> >  lvs -o attr,name,size --noheadings rootvg |
> >  while read a n s; do
> >  	[[ $a == -* ]] || continue;
> > 	lvcreate -L $s -n $n backupvg ;
> > 	# dd bs=1M if=/dev/rootvg/$n of=/dev/backupvg/$n
> >  done
> > ?
> > [...]

Re: [linux-lvm] Question on duplicating equally-sized HDs with dd and changing VG name and PV UUIDs

[Holger Rauch]

[-- Attachment #1: Type: text/plain, Size: 538 bytes --]

Hi to everybody,

ok, if running pvcreate is necessary, what tool can I use for generating
my own UUID (for passing them to the --uuid cmd line switch of pvcreate)
just in case the PV previously belonged to a different VG?

TIA.

Greetings,

	Holger

On Mon, 10 Sep 2007, malahal@us.ibm.com wrote:

> I don't think so. /dev/sdb will have the same partition table as
> /dev/sda from the sfdisk command.  I bet you need to run pvcreate and
> vgcreate before running the rest of the script.
> 
> Thanks, Malahal.
> [...]

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [linux-lvm] Question on duplicating equally-sized HDs with dd and changing VG name and PV UUIDs

[Holger Rauch]

[-- Attachment #1: Type: text/plain, Size: 1282 bytes --]

Replying to myself this time :-) Just found out about uuidgen. Would that be
OK? I'm thinking about invoking pvcreate like this:

pvcreate --uuid $(uuidgen) /dev/sdb

(I don't want't to forcefully recreate a PV, I just want to be able to
assign a different UUID to it in order keep LVM from "thinking" that PV
still belongs to some other VG).

But, on the other hand, I also just noticed that the UUIDs of PVs, VGs, LVs
are formatted differently from those generated by uuidgen. Or can LVM also
use UUIDs that are formatted differently?

So, my question about a UUID generation tool for proper UUID usage within
LVM seems to be still valid...

On Tue, 11 Sep 2007, Holger Rauch wrote:

> Hi to everybody,
> 
> ok, if running pvcreate is necessary, what tool can I use for generating
> my own UUID (for passing them to the --uuid cmd line switch of pvcreate)
> just in case the PV previously belonged to a different VG?
> 
> TIA.
> 
> Greetings,
> 
> 	Holger
> 
> On Mon, 10 Sep 2007, malahal@us.ibm.com wrote:
> 
> > I don't think so. /dev/sdb will have the same partition table as
> > /dev/sda from the sfdisk command.  I bet you need to run pvcreate and
> > vgcreate before running the rest of the script.
> > 
> > Thanks, Malahal.
> > [...]

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

LVM UUID generation (was Re: [linux-lvm] Question on duplicating equally-sized HDs with dd and changing VG name and PV UUIDs)

[Daniel Tuijnman]

[-- Attachment #1: Type: text/plain, Size: 1137 bytes --]

> (I don't want't to forcefully recreate a PV, I just want to be able to
> assign a different UUID to it in order keep LVM from "thinking" that PV
> still belongs to some other VG).
> 
> But, on the other hand, I also just noticed that the UUIDs of PVs, VGs, LVs
> are formatted differently from those generated by uuidgen. Or can LVM also
> use UUIDs that are formatted differently?

In my experience: no. I ran into the same some time ago with a similar
problem: I wanted to be able to clone LV's which I use as disk for Xen-VM's.
Of course, within the LV I also use a VG to partition the disk.

Attached is my script for automating this. I didn't know at the time about
uuidgen, so I used some Perl to generate a UUID and then to massage it into
the LVM UUID format.

> So, my question about a UUID generation tool for proper UUID usage within
> LVM seems to be still valid...

I'd be very anxious to know about this too.

-- 
Dani�l Tuijnman                     phone:  +31-24-377.7329
De Voorstenkamp 19-59               mobile: +31-6-4611.4589
NL-6545 GR Nijmegen                 fax:    +31-24-373.3632

[-- Attachment #2: clone_lvm --]
[-- Type: text/plain, Size: 4414 bytes --]

#!/bin/bash
#
# Daniel Tuijnman
# Fri Mar  2 20:34:13 CET 2007
#
# This script clones a LVM based paravirtualized Xen-VM
# it makes a number of assumptions:

# 1. The single disk of the VM is in a Volume Group VG_domU
VMBASE="/dev/VG_domU"

# 2. The VM disk contains two partitions:
#    (1) /boot
#    (2) a single Physical Volume for the Volume Group in the VM disk
BOOTPART=1
PVPART=2

# 3. The root filesystem of the VM is in a LV called 'root'
ROOTLV="root"

# 4. The name of the Volume Group on the VM disk is the name of the Xen-VM
# with "VG_" prepended
function vm_to_vg {
    echo "VG_$1"
}

# 5. the VM boots the first kernel present in the grub.conf
# (no variables to change here, sorry)

# set paranoia mode
set -eu
# set debug mode
# set -x

function cleanup {
    echo "Cleaning up on exit"
    umount "$MNTBOOT" || true
    umount "$MNT" || true
    vgchange -a n "$SRC_VOLG" || true
    vgchange -a n "$DST_VOLG" || true
    kpartx -d "$DST_VM"
    vgscan
    return
    /bin/rm -rf "$CFGFILE" "$NEWCFGFILE" "$MNT" "$INITRDDIR"
}

# test input
if (( $# != 2 )); then
    echo "$0: wrong number of arguments" >&2
    echo "$0: usage: $0 <src-lvm> <dst-lvm>" >&2
    exit 1
fi

SRC="$1"
DST="$2"
SRC_VM="${VMBASE}/${SRC}"
DST_VM="${VMBASE}/${DST}"

if [[ ! -b "$SRC_VM" ]]; then
    echo "$0: source LV does not exist" >&2
    exit 2
fi

if [[ ! -b "$DST_VM" ]]; then
    echo "$0: destination LV does not exist, will be created" >&2
    SIZE=$(lvs --noheadings --units m -o lv_size "$SRC_VM")
    lvcreate -v -L $SIZE -n "$DST" "$VMBASE"
else
    echo "$0: warning: destination LV will be overwritten" >&2
    echo "$0: you have 5 seconds to bail out" >&2
    sleep 5
fi

# set various variables
SRC_VOLG=$(vm_to_vg "$SRC")
echo "Source VolGroup: $SRC_VOLG"
DST_VOLG=$(vm_to_vg "$DST")
echo "Destination VolGroup: $DST_VOLG"
CFGFILE="/tmp/vgcfg$$"
DST_BOOT="/dev/mapper/${DST}p${BOOTPART}"
DST_ROOT="$DST_VOLG/$ROOTLV"
DST_PV="/dev/mapper/${DST}p${PVPART}"
MNT="/tmp/mnt$$"
MNTBOOT="$MNT/boot"
INITRDDIR="/tmp/initrd$$"
mkdir "$MNT"
mkdir "$INITRDDIR"
trap cleanup EXIT ERR SIGQUIT SIGINT

# now start with the real work
echo "$0: copying LV contents" >&2
echo "$0: this may take some time" >&2
dd if="$SRC_VM" of="$DST_VM"

# make device nodes for partitions on destination VM
kpartx -a "$DST_VM"
# read VG's and backup the current config
# (VG is still SRC named!)
vgscan
vgchange -a y "$SRC_VOLG"
vgcfgbackup -f "$CFGFILE" "$SRC_VOLG"
vgchange -a n "$SRC_VOLG"

# create new UUIDs for the PV, the VG and the LV's
# for the PV, we use pvchange to also change it on disk
# and subsequently, pvs to read the new UUID
# LVM UUID's are not official UUID's, so we have to massage
# the output of the Perl modules
perl -pi -e '
    BEGIN {
        use Data::UUID;
        $uuid = new Data::UUID;
        $physvol = 0;
    }
    if ( /physical_volumes/ ) { $physvol = 1 }
    if ( /logical_volumes/  ) { $physvol = 0 }
    if ( /^\s*id\s*\=\s*\"([A-Za-z0-9-]+)\"\s*$/ ) {
        my $newuuid;
        if ( $physvol ) {
            system "pvchange", "-u", "'"$DST_PV"'";
            $newuuid = qx!pvs --noheadings --options pv_uuid '"$DST_PV"'!;
            $newuuid =~ s/\s//g;
        } else {
            $newuuid = $uuid->create_str();
            $newuuid =~ s/(\w\w)-/-$1/g;
            $newuuid =~ s/(\w{4})(\w{6})$/-$1-$2/;
        }
        s/$1/$newuuid/;
    }
' "$CFGFILE";

# restore the VG and rename it
echo "Restoring volume group" >&2
vgcfgrestore -f "$CFGFILE" "$SRC_VOLG"
echo "Renaming volume group" >&2
vgrename "$SRC_VOLG" "$DST_VOLG"

# now change the VG name in the three places it's mentioned:
# 1 /etc/fstab
# 2 /boot/grub/grub.conf
# 3 'init' on the initrd...

vgchange -a y "$DST_VOLG"
mount "/dev/$DST_VOLG/$ROOTLV" "$MNT"
mount "$DST_BOOT" "$MNTBOOT"

# /etc/fstab
ed -s "$MNT/etc/fstab" <<-EDEND
	1,\$s/$SRC_VOLG/$DST_VOLG/
	w
	q
EDEND
# /boot/grub/grub.conf
ed -s "$MNTBOOT/grub/grub.conf" <<-EDEND
	1,\$s/$SRC_VOLG/$DST_VOLG/
	w
	q
EDEND
# 'init' on initrd
# first, what's the right initrd to change?
INITRD=$(sed -n '/^[ 	]*initrd/{
	s/[ 	]*initrd[ 	]*//
	p
	q
}' "$MNTBOOT"/grub/grub.conf)

cd "$INITRDDIR"
zcat "$MNTBOOT/$INITRD" | cpio -i
ed -s init <<-EDEND
	1,\$s/$SRC_VOLG/$DST_VOLG/
	w
	q
EDEND
find . | cpio -o -H newc | gzip > "$MNTBOOT/$INITRD"

# finally, wrap up
umount "$MNTBOOT"
umount "$MNT"
vgchange -a n "$DST_VOLG"
kpartx -d "$DST_VM"
vgscan
exit 0

Re: [linux-lvm] Question on duplicating equally-sized HDs with dd and changing VG name and PV UUIDs

[Alasdair G Kergon]

On Tue, Sep 11, 2007 at 12:12:50PM +0200, Holger Rauch wrote:
> Replying to myself this time :-) Just found out about uuidgen. Would that be
> OK? I'm thinking about invoking pvcreate like this:
> pvcreate --uuid $(uuidgen) /dev/sdb

There's something strange going on here.
The tools should be quite capable of generating a new uuid for you.
Perhaps it's a documentation problem.

Try pvchange -u.

Alasdair
-- 
agk@redhat.com

Re: [linux-lvm] Question on duplicating equally-sized HDs with dd and changing VG name and PV UUIDs

[Holger Rauch]

[-- Attachment #1: Type: text/plain, Size: 979 bytes --]

Hi Alasdair (and all the other list members),

first of all, thanks for having mentioned pvchange -u but the man page of
pvchange really gives no clue on how to actually use the -u switch. The
output of pvchange --help suggests that the -u switch takes no additional
parameter. So, if I understand you right, pvchange -u should both *generate
and assign* a new UUID to the specified PV and thus do what I need?

TIA for clarification on this issue.

Greetings,

	Holger

On Mon, 17 Sep 2007, Alasdair G Kergon wrote:

> On Tue, Sep 11, 2007 at 12:12:50PM +0200, Holger Rauch wrote:
> > Replying to myself this time :-) Just found out about uuidgen. Would that be
> > OK? I'm thinking about invoking pvcreate like this:
> > pvcreate --uuid $(uuidgen) /dev/sdb
> 
> There's something strange going on here.
> The tools should be quite capable of generating a new uuid for you.
> Perhaps it's a documentation problem.
> 
> Try pvchange -u.
> 
> Alasdair

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]