[linux-lvm] Script to import hardware snapshoted VGs

[linux-lvm] Script to import hardware snapshoted VGs

[chris procter]

Hi,

After getting kicking from Alasdair at a conference a few weeks ago for asking him about Netapp snapshots I decided it would by useful to implement http://sources.redhat.com/lvm2/wiki/HardwareSnapshotRestore (step 5 onwards) as a script. I've tested it on RHEL 5  and it seems pretty solid but I don't quite know the best way to release it to the world so here it is, all comments and advice gratefully received :)

I will get around to writing a man page, but in the meantime
1) present in your clone luns
2) figure out what devices they appear as
3) run importclone /dev/sda /dev/sdb /dev/sdc  (replacing /dev/sda etc by your clone luns obviously)

If your VG is called myvg you should now also have a new vg called myvg.1 with different uuids. It will ignore incomplete volume groups, or exported volume groups unless the -i flag is used.

chris

P.S. I'm using ascii 001 and 002 characters as seperators which might get mangled by email sorry :(

################# importclone #############################

!/bin/sh

# Copyright (C) 2009 Chris Procter All rights reserved.
# This copyrighted material is made available to anyone wishing to use,
# modify, copy, or redistribute it subject to the terms and conditions
# of the GNU General Public License v.2.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software Foundation,
# Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA

exec 2> ./$0.log
set -x

function appenddisk {
### add a blockdevice path (/dev/sda etc) to a list if its valid
    LIST=$1
    DISK=$2

    if [ ! -b "${DISK}" ]
    then
        echo " ${DISK} not a valid block device - ignoring" >&2
        echo "${LIST}"
        return
    fi

    if [ -z "${LIST}" ]
    then
        LIST="${DISK}"
    else
        LIST="${LIST} ${DISK}"
    fi
    echo ${LIST} 
} 

function getvgname {
### get a unique vg name  
###        $1 = list of exists VGs
###        $2 = the name we want
    VGLIST=$1
    VG=$2
    NEWVG=$3

    BASENAME="${NEWVG:-${VG}}"
    NAME="${BASENAME}"
    I=0

    while [[ "${VGLIST}" =~ "${NAME}" ]]
    do
        I=$(($I+1))
        NAME="${BASENAME}.$I"    
    done
    echo "${NAME}"
} 


function checkvalue {
### check return value and error if non zero
        if [ $1 -ne 0 ]
        then
                echo "FAIL!: $2 value: $1"
                exit $1
        fi
}


function usage {
### display usage message

    SCRIPT=`basename $0`
    echo "${SCRIPT} - Restore LVM data from a hardware snapshot"
    echo -e "Usage: ${SCRIPT} [disk1...]"
    echo -e "\t\t-h\t\t- Display this usage message"
    echo -e "\t\t-i\t\t- Import any exported volume groups found"
    echo -e "\t\t-n\t\t- Name for the new volume group(s)"
    echo -e "\t\t-l [path]\t - location of lvm.conf (default ${LVMCONF})"
    echo -e "\t\t-t [path]\t - directory for temporary lvm directory (default ${TMPDIR})"
    exit 0
}

function cleanup {
    #set to use old lvm.conf
    LVM_SYSTEM_DIR=${ORIG_LVM_SYS_DIR}

}


SHOW=0
DISKS=""
LVMCONF="/etc/lvm/lvm.conf"
TMPDIR="/tmp/lvm"
NOVGFLAG=0
IMPORT=0

export ORIG_LVM_SYS_DIR=${LVM_SYSTEM_DIR}

trap  cleanup 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

#####################################################################
### Get and check arguments
#####################################################################
while [ $# -ne 0 ]
do
    case $1 in
        -h)    usage
            shift
            ;;
        -i)    IMPORT=1
            shift
            ;;
        -l)    LVMCONF="$2"
            shift
            shift
            ;;
        -t)    TMPDIR="$2"
            shift
            shift
            ;;
        -n)    NEWVG="$2"
            shift
            shift
            ;;
        *)
            DISKS=`appenddisk "${DISKS}" "$1"`;
            shift
            ;;
    esac

done

### check we have suitable values for important variables
if [ -z "${DISKS}" ]
then
    usage
fi

#####################################################################
### Get the existing state so we can use it later
#####################################################################
#pvs --noheadings --trustcache 2>/dev/null
OLDVGS=`pvs --noheadings --trustcache 2>/dev/null  | awk '(NF==6){printf("%s ",$2)}'`


#####################################################################
### Prepare the temporay lvm environment
#####################################################################
if [ ! -d "${TMPDIR}" ]
then
    mkdir "${TMPDIR}"
    checkvalue $? "Unable to create ${TMPDIR}"
fi


###create filter
for BLOCK in ${DISKS}
do
    FILTER="\"a|^${BLOCK}$|\",${FILTER}"
done

FILTER="filter=[ ${FILTER} \"r|.*|\" ]"

awk -v var="${FILTER}" '/^[[:space:]]*filter/{print var;next};{print $0}' < ${LVMCONF} > ${TMPDIR}/lvm.conf


### set to use new lvm.conf
export LVM_SYSTEM_DIR=${TMPDIR}


#####################################################################
### Change the uuids.
#####################################################################
PVSCAN=`pvscan`

VGS=`echo "${PVSCAN}" |awk '$1~/PV/{for(i=1;i<=NF;i++){if($i=="VG"){vg[$(i+1)]=vg[$(i+1)]""$2}if($i=="exported"){x[$(i+2)]="x"}}}END{for(k in vg){printf k""x[k]""vg[k]" "}}'`

echo "$VGS"

for VG in ${VGS}
do
    VGNAME=`echo -e "${VG}" |cut -d -f1`
    EXPORTED=`echo -e "${VG}" | cut -d -f2`
    PVLIST=`echo -e "${VG}" | cut -d -f3`

    if [ -n "${EXPORTED}" ]
    then
        if [ ${IMPORT} -eq 1 ]
        then
            vgimport ${VGNAME}
        else
            echo "Volume Group ${VGNAME} exported, skipping."
            continue
        fi

    fi

    ### change the pv uuids
    BLOCKDEVS=`echo ${PVLIST} | tr '' ' '`
    if [ "${PVLIST}" =~ "unknown" ]
    then
        echo "Volume Group ${VGNAME} incomplete, skipping."
        continue
    fi

    for BLOCKDEV in ${BLOCKDEVS}
    do
        pvchange --uuid ${BLOCKDEV} --config 'global{activation=0}' 
        checkvalue $? "Unable to change pvuuid for ${BLOCKDEV}"
    done

    NEWVGNAME=`getvgname "${OLDVGS}" "${VGNAME}" "${NEWVG}"`

    vgchange --uuid ${VGNAME} --config 'global{activation=0}' 
    checkvalue $? "Unable to change vguuid for ${VGNAME}"

    ## if the name isn't going to get changed dont even try.
    if [ "${VGNAME}" != "${NEWVGNAME}" ]
    then
        vgrename "${VGNAME}" "${NEWVGNAME}"
        checkvalue $? "Unable to rename ${VGNAME} to ${NEWVGNAME}"
    fi

done

#####################################################################
### Restore the old environment
#####################################################################
### set to use old lvm.conf
LVM_SYSTEM_DIR=${ORIG_LVM_SYS_DIR}

### make sure all the device nodes we need are straight
vgmknodes >/dev/null

### sort out caches.

pvscan

exit 0



      

[linux-lvm] Re: Script to import hardware snapshoted VGs

[Mike Snitzer]

Hi Chris,

I actually worked on such a script a little over a month ago but haven't
had a chance to get back to it.  I've reviewed your script and have
inlined some comments/suggestions below.


On Thu, May 07 2009 at  6:05pm -0400,
chris procter <chris-procter@talk21.com> wrote:

> If your VG is called myvg you should now also have a new vg called
> myvg.1 with different uuids. It will ignore incomplete volume groups,
> or exported volume groups unless the -i flag is used. 

I applaud your support of volume groups that are either imported or
exported.  Not strictly needed but thorough.


> function appenddisk {
> ### add a blockdevice path (/dev/sda etc) to a list if its valid
>     LIST=$1
>     DISK=$2
> 
>     if [ ! -b "${DISK}" ]
>     then
>         echo " ${DISK} not a valid block device - ignoring" >&2
>         echo "${LIST}"
>         return
>     fi
> 
>     if [ -z "${LIST}" ]
>     then
>         LIST="${DISK}"
>     else
>         LIST="${LIST} ${DISK}"
>     fi
>     echo ${LIST} 
> } 

One use-case that was off my radar until others pointed it out is when
obscure PV names exist in the system.  Using shell and not explicitly
checking for such awkward PV names can cause the script to fail.  Names
might exclude any of the following (taken from the LVM2 source's
test/t-mdata-strings.sh): "__\"!@#\$%^&*,()|@||'\\\""

The lvm tools can handle such names as long as the user takes care to
properly escape the special characters because once passed in as an arg
the C-code won't interpret the characters at all; unfortunately shell
will.

So we likely need to at least detect that the supplied PV name(s)
contain "unsupported" (by this script) characters and error out
accordingly.  This is the part that caused me to put my script stall.

...
> SHOW=0
> DISKS=""
> LVMCONF="/etc/lvm/lvm.conf"
> TMPDIR="/tmp/lvm"
...
> 
> #####################################################################
> ### Prepare the temporay lvm environment
> #####################################################################
> if [ ! -d "${TMPDIR}" ]
> then
>     mkdir "${TMPDIR}"
>     checkvalue $? "Unable to create ${TMPDIR}"
> fi
...
> awk -v var="${FILTER}" '/^[[:space:]]*filter/{print var;next};{print $0}' < ${LVMCONF} > ${TMPDIR}/lvm.conf
 

You have a security hole here because some user _could_ create /tmp/lvm
in advance and symlink /tmp/lvm/lvm.conf to some random file they want
overwritten by root.

Also, TMPDIR is a standard name used by other tools; likley best to
side-step it by using something like "TMP_LVM_SYSTEM_DIR".

Using the following would be best:
TMP_LVM_SYSTEM_DIR=$(mktemp -d --tmpdir snap.XXXXXXXX)

There really isn't any need to expose which TMPDIR to use with -t; just
allow the user to set TMPDIR in their environment and mktemp will "just
work" with it.

Quick side-bar: you should probably add something like the following
early on:

if test "$UID" != "0" && test "$EUID" != "0"; then
  echo "WARNING: Running as a non-root user. Functionality may be unavailable."
fi

...
> OLDVGS=`pvs --noheadings --trustcache 2>/dev/null  | awk '(NF==6){printf("%s ",$2)}'`

Existing LVM2 scripts (e.g. LVM2/script/lvm_dump.sh) allow the user to
override which lvm binary is used for all command by doing the
following:

# user may override lvm location by setting LVM_BINARY
LVM=${LVM_BINARY-lvm}

die() {
    code=$1; shift
    echo "$@" 1>&2
    exit $code
}

"$LVM" version >& /dev/null || die 2 "Could not run lvm binary '$LVM'"

> #####################################################################
> ### Change the uuids.
> #####################################################################
> PVSCAN=`pvscan`

The above would become:
PVSCAN=`"$LVM" pvscan`



Once these things are cleaned up we should get it committed to the LVM2
tree; in the scripts/ dir.

But these things aside, the script looks pretty good.

BTW, it should probably get renamed to have to a more standard lvm
prefix, e.g.: vgimportclone

Regards,
Mike

[linux-lvm] Re: Script to import hardware snapshoted VGs

[chris procter]

> Hi Chris,

> 
> I actually worked on such a script a little over a month ago but haven't
> had a chance to get back to it.  I've reviewed your script and have
> inlined some comments/suggestions below.

Thanks a lot :)

Most of your suggestions are pretty straightforward and I'll put them in, but the PV name problem is nasty, sorting out my script to deal with them is doable, but there is an issue with setting lvm.conf to filter devices called /dev/$*^  would really upset the regular expression parser. I'm tempted to say that people who call their devices things like that deserve everything they get, but maybe symlinking them to a sensible name (e.g $TMP_LVM_SYSTEM_DIR/dev/temp1) would be better. I'll have a think.  

Oh and it deals with exported volume groups because I hit that one during testing, maybe I have weird usage patterns!

chris


> On Thu, May 07 2009 at  6:05pm -0400,
> chris procter wrote:
> 
> > If your VG is called myvg you should now also have a new vg called
> > myvg.1 with different uuids. It will ignore incomplete volume groups,
> > or exported volume groups unless the -i flag is used. 
> 
> I applaud your support of volume groups that are either imported or
> exported.  Not strictly needed but thorough.
>
> > function appenddisk {
> > ### add a blockdevice path (/dev/sda etc) to a list if its valid
> >     LIST=$1
> >     DISK=$2
> > 
> >     if [ ! -b "${DISK}" ]
> >     then
> >         echo " ${DISK} not a valid block device - ignoring" >&2
> >         echo "${LIST}"
> >         return
> >     fi
> > 
> >     if [ -z "${LIST}" ]
> >     then
> >         LIST="${DISK}"
> >     else
> >         LIST="${LIST} ${DISK}"
> >     fi
> >     echo ${LIST} 
> > } 
>
> One use-case that was off my radar until others pointed it out is when
> obscure PV names exist in the system.  Using shell and not explicitly
> checking for such awkward PV names can cause the script to fail.  Names
> might exclude any of the following (taken from the LVM2 source's
> test/t-mdata-strings.sh): "__\"!@#\$%^&*,()|@||'\\\""
> 
> The lvm tools can handle such names as long as the user takes care to
> properly escape the special characters because once passed in as an arg
> the C-code won't interpret the characters at all; unfortunately shell
> will.
> 
> So we likely need to at least detect that the supplied PV name(s)
> contain "unsupported" (by this script) characters and error out
> accordingly.  This is the part that caused me to put my script stall.
>
> > SHOW=0
> > DISKS=""
> > LVMCONF="/etc/lvm/lvm.conf"
> > TMPDIR="/tmp/lvm"
> ...
> > 
> > #####################################################################
> > ### Prepare the temporay lvm environment
> > #####################################################################
> > if [ ! -d "${TMPDIR}" ]
> > then
> >     mkdir "${TMPDIR}"
> >     checkvalue $? "Unable to create ${TMPDIR}"
> > fi
> ...
> > awk -v var="${FILTER}" '/^[[:space:]]*filter/{print var;next};{print $0}' < 
> ${LVMCONF} > ${TMPDIR}/lvm.conf
> 
> 
> You have a security hole here because some user _could_ create /tmp/lvm
> in advance and symlink /tmp/lvm/lvm.conf to some random file they want
> overwritten by root.
> 
> Also, TMPDIR is a standard name used by other tools; likley best to
> side-step it by using something like "TMP_LVM_SYSTEM_DIR".
> 
> Using the following would be best:
> TMP_LVM_SYSTEM_DIR=$(mktemp -d --tmpdir snap.XXXXXXXX)
> 
> There really isn't any need to expose which TMPDIR to use with -t; just
> allow the user to set TMPDIR in their environment and mktemp will "just
> work" with it.
> 
> Quick side-bar: you should probably add something like the following
> early on:
> 
> if test "$UID" != "0" && test "$EUID" != "0"; then
>   echo "WARNING: Running as a non-root user. Functionality may be unavailable."
> fi
> 
> ...
> > OLDVGS=`pvs --noheadings --trustcache 2>/dev/null  | awk '(NF==6){printf("%s 
> ",$2)}'`
> 
> Existing LVM2 scripts (e.g. LVM2/script/lvm_dump.sh) allow the user to
> override which lvm binary is used for all command by doing the
> following:
> 
> # user may override lvm location by setting LVM_BINARY
> LVM=${LVM_BINARY-lvm}
> 
> die() {
>     code=$1; shift
>     echo "$@" 1>&2
>     exit $code
> }
> 
> "$LVM" version >& /dev/null || die 2 "Could not run lvm binary '$LVM'"
> 
> > #####################################################################
> > ### Change the uuids.
> > #####################################################################
> > PVSCAN=`pvscan`
> 
> The above would become:
> PVSCAN=`"$LVM" pvscan`
> 
> 
> 
> Once these things are cleaned up we should get it committed to the LVM2
> tree; in the scripts/ dir.
> 
> But these things aside, the script looks pretty good.
> 
> BTW, it should probably get renamed to have to a more standard lvm
> prefix, e.g.: vgimportclone
> 
> Regards,
> Mike



      

Re: [linux-lvm] Re: Script to import hardware snapshoted VGs

[Eric Brunson]

On 05/09/2009 01:34 PM, chris procter wrote:
>
>> I actually worked on such a script a little over a month ago but haven't
>> had a chance to get back to it.  I've reviewed your script and have
>> inlined some comments/suggestions below.
>>      
>
> Thanks a lot :)
>
> Most of your suggestions are pretty straightforward and I'll put them in, but the PV name problem is nasty, sorting out my script to deal with them is doable, but there is an issue with setting lvm.conf to filter devices called /dev/$*^  would really upset the regular expression parser. I'm tempted to say that people who call their devices things like that deserve everything they get,

http://xkcd.com/327/

:-)

>   but maybe symlinking them to a sensible name (e.g $TMP_LVM_SYSTEM_DIR/dev/temp1) would be better. I'll have a think.
>
> Oh and it deals with exported volume groups because I hit that one during testing, maybe I have weird usage patterns!
>
> chris
>
>
>    

[linux-lvm] Re: Script to import hardware snapshoted VGs

[chris procter]

Hi,

Here's version 2. I've got it to play nicely with device names like:-
/dev/"__\"!@#\$%^&*,()|@||\\\""
/dev/\*

/dev/''*   (with trailing space and single quotes)
/dev/$* ^"
which should cover pretty much all possibilities by symlinking them from the temp directory and using the links which all seems to work and gets round issues with the filter regular expressions in lvm.conf  Hopefully that covers all the weirdness anyone is likely to use. 

One day I will work out how to use CVS and do proper diffs :)

################# vgimportclone #############################
#!/bin/sh

# Copyright (C) 2009 Chris Procter All rights reserved.
# This copyrighted material is made available to anyone wishing to use,
# modify, copy, or redistribute it subject to the terms and conditions
# of the GNU General Public License v.2.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software Foundation,
# Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA


function getvgname {
### get a unique vg name
###        $1 = list of exists VGs
###        $2 = the name we want
    VGLIST=$1
    VG=$2
    NEWVG=$3

    BASENAME="${NEWVG:-${VG}}"
    NAME="${BASENAME}"
    I=0

    while [[ "${VGLIST}" =~ "${NAME}" ]]
    do
        I=$(($I+1))
        NAME="${BASENAME}.$I"
    done
    echo "${NAME}"
} 


function checkvalue {
### check return value and error if non zero
        if [ $1 -ne 0 ]
        then
                echo "FAIL!: $2 value: $1"
                exit $1
        fi
}


function usage {
### display usage message

    SCRIPT=`basename $0`
    echo "${SCRIPT} - Restore LVM data from a hardware snapshot"
    echo -e "Usage: ${SCRIPT} [options] [disk1...]"
    echo -e "\t\t-h\t\t- Display this usage message"
    echo -e "\t\t-i\t\t- Import any exported volume groups found"
    echo -e "\t\t-n\t\t- Name for the new volume group(s)"
    echo -e "\t\t-l [path]\t - location of lvm.conf (default ${LVMCONF})"
    exit 0
}

function cleanup {
    #set to use old lvm.conf
    LVM_SYSTEM_DIR=${ORIG_LVM_SYS_DIR}

    if [ ${DEBUG} -eq 0 ]
    then 
         rm -r ${TMP_LVM_SYSTEM_DIR}
    fi
}

SCRIPTNAME=`basename $0`


if [ "$UID" != "0" -a "$EUID" != "0" ]
then
    echo "${SCRIPTNAME} must be run as root, and you aren't."
    exit 0;
fi

SHOW=0
DISKS=""
LVMCONF="/etc/lvm/lvm.conf"
TMP_LVM_SYSTEM_DIR=`mktemp -d -t snap.XXXXXXXX`
NOVGFLAG=0
IMPORT=0
DEBUG=0
DEVNO=0

export ORIG_LVM_SYS_DIR="${LVM_SYSTEM_DIR}"

trap  cleanup 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

#####################################################################
### Get and check arguments
#####################################################################
while [ $# -ne 0 ]
do
    case $1 in
        -d)    DEBUG=1
            exec 2> ./${SCRIPTNAME}.log
               set -x
            echo "Using $TMP_LVM_SYSTEM_DIR/lvm.conf"
            shift
            ;;
        -h)    usage
            shift
            ;;
        -i)    IMPORT=1
            shift
            ;;
        -l)    LVMCONF="$2"
            shift
            shift
            ;;
        -n)    NEWVG="$2"
            shift
            shift
            ;;
        *)    if [ -b "$1" ]
            then
                ln -s "$1" ${TMP_LVM_SYSTEM_DIR}/vgimport${DEVNO}
                DISKS="${DISKS} ${TMP_LVM_SYSTEM_DIR}/vgimport${DEVNO}"
                DEVNO=$((${DEVNO}+1))
            fi
            shift
            ;;
    esac

done


### check we have suitable values for important variables
if [ -z "${DISKS}" ]
then
    usage
fi

LVM="${LVM_BINARY:-lvm}"
"${LVM}" version >&/dev/null
checkvalue $? "${LVM} doesn't look like an lvm binary."


#####################################################################
### Get the existing state so we can use it later
#####################################################################

OLDVGS=`"${LVM}" pvs --noheadings --trustcache --separator  2>/dev/null  | awk -F  '/lvm2/{printf("%s ",$2)}'`


#####################################################################
### Prepare the temporay lvm environment
#####################################################################

###create filter
for BLOCK in ${DISKS}
do
    FILTER="\"a|^${BLOCK}$|\",${FILTER}"
done

export FILTER="filter=[ ${FILTER} \"r|.*|\" ]"

awk -v DEV=${TMP_LVM_SYSTEM_DIR} '/^[[:space:]]*filter/{print ENVIRON["FILTER"];next}\
                    /^[[:space:]]*scan/{print "scan = [ \"" DEV  "\" ]";next} \
                    {print $0}' < ${LVMCONF} > ${TMP_LVM_SYSTEM_DIR}/lvm.conf

### set to use new lvm.conf
export LVM_SYSTEM_DIR=${TMP_LVM_SYSTEM_DIR}


#####################################################################
### Change the uuids.
#####################################################################
PVSCAN=`"${LVM}" pvscan`

### create a space seperated list of VGs where each VG looks like: nameexported?disk1disk2... 
VGS=`echo "${PVSCAN}" |awk '$1~/PV/{for(i=1;i<=NF;i++){if($i=="VG"){vg[$(i+1)]=vg[$(i+1)]""$2} \
                            if($i=="exported"){x[$(i+2)]="x"}}} \
                END{for(k in vg){printf k""x[k] vg[k]" "}}'`

for VG in ${VGS}
do
    VGNAME=`echo -e "${VG}" |cut -d -f1`
    EXPORTED=`echo -e "${VG}" | cut -d -f2`
    PVLIST=`echo -e "${VG}" | cut -d -f3-`

    if [ -n "${EXPORTED}" ]
    then
        if [ ${IMPORT} -eq 1 ]
        then
            "${LVM}" vgimport ${VGNAME}
        else
            echo "Volume Group ${VGNAME} exported, skipping."
            continue
        fi

    fi

    ### change the pv uuids
    BLOCKDEVS=`echo ${PVLIST} | tr '' ' '`
    if [[ "${BLOCKDEVS}" =~ "unknown" ]]
    then
        echo "Volume Group ${VGNAME} incomplete, skipping."
        continue
    fi

    for BLOCKDEV in ${BLOCKDEVS}
    do
        "${LVM}" pvchange --uuid ${BLOCKDEV} --config 'global{activation=0}' 
        checkvalue $? "Unable to change pvuuid for ${BLOCKDEV}"
    done

    NEWVGNAME=`getvgname "${OLDVGS}" "${VGNAME}" "${NEWVG}"`

    "${LVM}" vgchange --uuid ${VGNAME} --config 'global{activation=0}'
    checkvalue $? "Unable to change vguuid for ${VGNAME}"

    ## if the name isn't going to get changed dont even try.
    if [ "${VGNAME}" != "${NEWVGNAME}" ]
    then
        "${LVM}" vgrename "${VGNAME}" "${NEWVGNAME}"
        checkvalue $? "Unable to rename ${VGNAME} to ${NEWVGNAME}"
    fi

done

#####################################################################
### Restore the old environment
#####################################################################
### set to use old lvm.conf
LVM_SYSTEM_DIR=${ORIG_LVM_SYS_DIR}

### make sure all the device nodes we need are straight
"${LVM}" vgmknodes >/dev/null

### sort out caches.

"${LVM}" pvscan
"${LVM}" vgscan >/dev/null

exit 0



      

Re: [linux-lvm] Re: Script to import hardware snapshoted VGs

[Eric Brunson]

On 05/11/2009 12:39 PM, chris procter wrote:
> Hi,
>
> Here's version 2. I've got it to play nicely with device names like:-
> /dev/"__\"!@#\$%^&*,()|@||\\\""
> /dev/\*
>
> /dev/''*   (with trailing space and single quotes)
> /dev/$* ^"
> which should cover pretty much all possibilities by symlinking them from the temp directory and using the links which all seems to work and gets round issues with the filter regular expressions in lvm.conf  Hopefully that covers all the weirdness anyone is likely to use.
>
> One day I will work out how to use CVS and do proper diffs :)
>
>    

cvs add filename
cvs commit filename
cvs diff filename

:-)

Re: [linux-lvm] Re: Script to import hardware snapshoted VGs

[Stuart D. Gathman]

On Mon, 11 May 2009, Eric Brunson wrote:

> > One day I will work out how to use CVS and do proper diffs :)
> >
> >    
> 
> cvs add filename
> cvs commit filename
> cvs diff filename

He meant he hasn't configured a CVS repository yet.  That is
relatively easy too, but if you just want diffs on a single file,
RCS is very handy:

$ ci -l filename
$ vim filename
$ rcsdiff filename
$ ls filename*
filename filename,v

There is no repository, the versions are stored in ,v files.

-- 
	      Stuart D. Gathman <stuart@bmsi.com>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

Re: [linux-lvm] Re: Script to import hardware snapshoted VGs

[Eric Brunson]

On 05/11/2009 03:15 PM, Stuart D. Gathman wrote:
>
> He meant he hasn't configured a CVS repository yet.  That is
> relatively easy too, but if you just want diffs on a single file,
> RCS is very handy:
>
> $ ci -l filename
> $ vim filename
> $ rcsdiff filename
> $ ls filename*
> filename filename,v
>
> There is no repository, the versions are stored in ,v files.
>
>    

RCS is very handy for that.  I recently discovered Mercurial, which will 
turn any directory into a repository, it's awesome.

[linux-lvm] Re: Script to import hardware snapshoted VGs

[Mike Snitzer]

On Mon, May 11 2009 at  2:39pm -0400,
chris procter <chris-procter@talk21.com> wrote:

> 
> Hi,
> 
> Here's version 2. I've got it to play nicely with device names like:-
> /dev/"__\"!@#\$%^&*,()|@||\\\""
> /dev/\*
> 
> /dev/''*   (with trailing space and single quotes)
> /dev/$* ^"
> which should cover pretty much all possibilities by symlinking them from the temp directory and using the links which all seems to work and gets round issues with the filter regular expressions in lvm.conf  Hopefully that covers all the weirdness anyone is likely to use. 
> 
> One day I will work out how to use CVS and do proper diffs :)

Chris,

I reviewed and tested your latest script.  I had issues with your use of
whitespace delimiters (could be email ate it).  In any case, I reworked
it so that delimiters are all non-whitespace.

As you'll see in the below patch I made various changes:
. cleaned up whitespace
. added some extra negative checks for safety
. use different lvm commands that lend themselves to simpler parsing
. set cache_dir override in lvm.conf
. eliminated vgscan at the end

Let me know what you think, I'll commit this into the LVM2 CVS once I
get your feedback.

Thanks,
Mike


diff --git a/vgimportclone b/vgimportclone
index 7781d66..0bb77c6
--- a/vgimportclone
+++ b/vgimportclone
@@ -1,6 +1,10 @@
 #!/bin/sh
 
 # Copyright (C) 2009 Chris Procter All rights reserved.
+# Copyright (C) 2009 Red Hat, Inc. All rights reserved.
+#
+# This file is part of LVM2.
+#
 # This copyrighted material is made available to anyone wishing to use,
 # modify, copy, or redistribute it subject to the terms and conditions
 # of the GNU General Public License v.2.
@@ -8,6 +12,9 @@
 # You should have received a copy of the GNU General Public License
 # along with this program; if not, write to the Free Software Foundation,
 # Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+#
+# vgimportclone: This script is used to rename the VG and change the associated
+#                VG and PV UUIDs (primary application being HW snapshot restore)
 
 
 function getvgname {
@@ -28,7 +35,7 @@ function getvgname {
         NAME="${BASENAME}.$I"
     done
     echo "${NAME}"
-} 
+}
 
 
 function checkvalue {
@@ -50,7 +57,7 @@ function usage {
     echo -e "\t\t-h\t\t- Display this usage message"
     echo -e "\t\t-i\t\t- Import any exported volume groups found"
     echo -e "\t\t-n\t\t- Name for the new volume group(s)"
-    echo -e "\t\t-l [path]\t - location of lvm.conf (default ${LVMCONF})"
+    echo -e "\t\t-l [path]\t- location of lvm.conf (default ${LVMCONF})"
     exit 0
 }
 
@@ -59,7 +66,7 @@ function cleanup {
     LVM_SYSTEM_DIR=${ORIG_LVM_SYS_DIR}
 
     if [ ${DEBUG} -eq 0 ]
-    then 
+    then
          rm -r ${TMP_LVM_SYSTEM_DIR}
     fi
 }
@@ -76,13 +83,16 @@ fi
 SHOW=0
 DISKS=""
 LVMCONF="/etc/lvm/lvm.conf"
-TMP_LVM_SYSTEM_DIR=`mktemp -d -t snap.XXXXXXXX`
+TMP_LVM_SYSTEM_DIR=`mktemp -d --tmpdir snap.XXXXXXXX`
 NOVGFLAG=0
 IMPORT=0
 DEBUG=0
 DEVNO=0
 
-export ORIG_LVM_SYS_DIR="${LVM_SYSTEM_DIR}"
+if [ -n "${LVM_SYSTEM_DIR}" ]; then
+    export ORIG_LVM_SYS_DIR="${LVM_SYSTEM_DIR}"
+    LVMCONF="${LVM_SYSTEM_DIR}/lvm.conf"
+fi
 
 trap  cleanup 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
 
@@ -94,7 +104,7 @@ do
     case $1 in
         -d)    DEBUG=1
             exec 2> ./${SCRIPTNAME}.log
-               set -x
+            set -x
             echo "Using $TMP_LVM_SYSTEM_DIR/lvm.conf"
             shift
             ;;
@@ -135,13 +145,22 @@ LVM="${LVM_BINARY:-lvm}"
 "${LVM}" version >&/dev/null
 checkvalue $? "${LVM} doesn't look like an lvm binary."
 
+if [ -n "$NEWVG" ] ; then
+    "${LVM}" vgs $NEWVG >& /dev/null
+    if [ $? -eq 0 ] ; then
+        echo "Error: New VG ($NEWVG) already exists." >&2
+        exit 1
+    fi
+fi
+
+test -f $LVMCONF
+checkvalue $? "${LVMCONF} doesn't exist."
 
 #####################################################################
 ### Get the existing state so we can use it later
 #####################################################################
 
-OLDVGS=`"${LVM}" pvs --noheadings --trustcache --separator  2>/dev/null  | awk -F  '/lvm2/{printf("%s ",$2)}'`
-
+OLDVGS=`"${LVM}" vgs -o name --noheadings 2>/dev/null`
 
 #####################################################################
 ### Prepare the temporay lvm environment
@@ -150,15 +169,19 @@ OLDVGS=`"${LVM}" pvs --noheadings --trustcache --separator  2>/dev/null  | awk -
 ###create filter
 for BLOCK in ${DISKS}
 do
-    FILTER="\"a|^${BLOCK}$|\",${FILTER}"
+    FILTER="\"a|^${BLOCK}$|\", ${FILTER}"
 done
 
 export FILTER="filter=[ ${FILTER} \"r|.*|\" ]"
 
-awk -v DEV=${TMP_LVM_SYSTEM_DIR} '/^[[:space:]]*filter/{print ENVIRON["FILTER"];next}\
-                    /^[[:space:]]*scan/{print "scan = [ \"" DEV  "\" ]";next} \
+awk -v DEV=${TMP_LVM_SYSTEM_DIR} -v CACHE=${TMP_LVM_SYSTEM_DIR}/cache \
+                   '/^[[:space:]]*filter/{print ENVIRON["FILTER"];next} \
+                    /^[[:space:]]*scan/{print "scan = [ \"" DEV "\" ]";next} \
+                    /^[[:space:]]*cache_dir/{print "cache_dir = \"" CACHE "\"";next} \
                     {print $0}' < ${LVMCONF} > ${TMP_LVM_SYSTEM_DIR}/lvm.conf
 
+checkvalue $? "Failed to generate ${TMP_LVM_SYSTEM_DIR}/lvm.conf"
+
 ### set to use new lvm.conf
 export LVM_SYSTEM_DIR=${TMP_LVM_SYSTEM_DIR}
 
@@ -166,18 +189,30 @@ export LVM_SYSTEM_DIR=${TMP_LVM_SYSTEM_DIR}
 #####################################################################
 ### Change the uuids.
 #####################################################################
-PVSCAN=`"${LVM}" pvscan`
 
-### create a space seperated list of VGs where each VG looks like: nameexported?disk1disk2... 
-VGS=`echo "${PVSCAN}" |awk '$1~/PV/{for(i=1;i<=NF;i++){if($i=="VG"){vg[$(i+1)]=vg[$(i+1)]""$2} \
-                            if($i=="exported"){x[$(i+2)]="x"}}} \
-                END{for(k in vg){printf k""x[k] vg[k]" "}}'`
+PVINFO=`"${LVM}" pvs -o pv_name,vg_name,vg_attr --noheadings --separator : | sed -e "s/ //g"`
+
+# output VG info so each line looks like: name:exported?:disk1,disk2,...
+VGINFO=`echo "${PVINFO}" | \
+    awk -F : '{{vg[$2]=$1","vg[$2]} \
+    if($3 ~ /^..x/){x[$2]="x"}} \
+    END{for(k in vg){printf("%s:%s:%s\n", k, x[k], vg[k])}}'`
 
-for VG in ${VGS}
+for VG in ${VGINFO}
 do
-    VGNAME=`echo -e "${VG}" |cut -d -f1`
-    EXPORTED=`echo -e "${VG}" | cut -d -f2`
-    PVLIST=`echo -e "${VG}" | cut -d -f3-`
+    VGNAME=`echo "${VG}" | cut -d: -f1`
+    EXPORTED=`echo "${VG}" | cut -d: -f2`
+    PVLIST=`echo "${VG}" | cut -d: -f3- | tr , ' '`
+
+    if [ -z "${VGNAME}" ] ; then
+        FOLLOWLIST=""
+        for DEV in $PVLIST; do
+            FOLLOW=`readlink $DEV`
+            FOLLOWLIST="$FOLLOW $FOLLOWLIST"
+        done
+        echo "Error: Specified PV(s) ($FOLLOWLIST) don't belong to a VG." >&2
+        exit 1
+    fi
 
     if [ -n "${EXPORTED}" ]
     then
@@ -188,20 +223,18 @@ do
             echo "Volume Group ${VGNAME} exported, skipping."
             continue
         fi
-
     fi
 
     ### change the pv uuids
-    BLOCKDEVS=`echo ${PVLIST} | tr '' ' '`
-    if [[ "${BLOCKDEVS}" =~ "unknown" ]]
+    if [[ "${PVLIST}" =~ "unknown" ]]
     then
         echo "Volume Group ${VGNAME} incomplete, skipping."
         continue
     fi
 
-    for BLOCKDEV in ${BLOCKDEVS}
+    for BLOCKDEV in ${PVLIST}
     do
-        "${LVM}" pvchange --uuid ${BLOCKDEV} --config 'global{activation=0}' 
+        "${LVM}" pvchange --uuid ${BLOCKDEV} --config 'global{activation=0}'
         checkvalue $? "Unable to change pvuuid for ${BLOCKDEV}"
     done
 
@@ -225,12 +258,10 @@ done
 ### set to use old lvm.conf
 LVM_SYSTEM_DIR=${ORIG_LVM_SYS_DIR}
 
-### make sure all the device nodes we need are straight
-"${LVM}" vgmknodes >/dev/null
-
-### sort out caches.
+### update the device cache and make sure all
+### the device nodes we need are straight
 
 "${LVM}" pvscan
-"${LVM}" vgscan >/dev/null
+"${LVM}" vgmknodes
 
 exit 0

[linux-lvm] Re: Script to import hardware snapshoted VGs

[chris procter]

> Chris,

> 
> I reviewed and tested your latest script.  I had issues with your use of
> whitespace delimiters (could be email ate it).  In any case, I reworked
> it so that delimiters are all non-whitespace.
>
> As you'll see in the below patch I made various changes:
> . cleaned up whitespace
> . added some extra negative checks for safety
> . use different lvm commands that lend themselves to simpler parsing
> . set cache_dir override in lvm.conf
> . eliminated vgscan at the end
> 
> Let me know what you think, I'll commit this into the LVM2 CVS once I
> get your feedback.
> 
> Thanks,
> Mike

I was using whitespace as a seperator in a
couple of places but mostly I was using ascii \002 which do appear to
have been mangled somewhere on the line. This may also explain why I
cant get your changes to apply cleanly, but they mostly look pretty
sensible.   I've inlined my comments below.



> diff --git a/vgimportclone b/vgimportclone
> index 7781d66..0bb77c6
> --- a/vgimportclone
> +++ b/vgimportclone
> @@ -1,6 +1,10 @@
> #!/bin/sh
> 
> # Copyright (C) 2009 Chris Procter All rights reserved.
> +# Copyright (C) 2009 Red Hat, Inc. All rights reserved.
> +#
> +# This file is part of LVM2.
> +#
> # This copyrighted material is made available to anyone wishing to use,
> # modify, copy, or redistribute it subject to the terms and conditions
> # of the GNU General Public License v.2.
> @@ -8,6 +12,9 @@
> # You should have received a copy of the GNU General Public License
> # along with this program; if not, write to the Free Software Foundation,
> # Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
> +#
> +# vgimportclone: This script is used to rename the VG and change the associated
> +#                VG and PV UUIDs (primary application being HW snapshot 
> restore)
> 
> 
> function getvgname {
> @@ -28,7 +35,7 @@ function getvgname {
>          NAME="${BASENAME}.$I"
>      done
>      echo "${NAME}"
> -} 
> +}
> 
> 
> function checkvalue {
> @@ -50,7 +57,7 @@ function usage {
>      echo -e "\t\t-h\t\t- Display this usage message"
>      echo -e "\t\t-i\t\t- Import any exported volume groups found"
>      echo -e "\t\t-n\t\t- Name for the new volume group(s)"
> -    echo -e "\t\t-l [path]\t - location of lvm.conf (default ${LVMCONF})"
> +    echo -e "\t\t-l [path]\t- location of lvm.conf (default ${LVMCONF})"
>      exit 0
> }
> 
> @@ -59,7 +66,7 @@ function cleanup {
>      LVM_SYSTEM_DIR=${ORIG_LVM_SYS_DIR}
> 
>      if [ ${DEBUG} -eq 0 ]
> -    then 
> +    then
>           rm -r ${TMP_LVM_SYSTEM_DIR}
>      fi
> }
> @@ -76,13 +83,16 @@ fi
> SHOW=0
> DISKS=""
> LVMCONF="/etc/lvm/lvm.conf"
> -TMP_LVM_SYSTEM_DIR=`mktemp -d -t snap.XXXXXXXX`
> +TMP_LVM_SYSTEM_DIR=`mktemp -d --tmpdir snap.XXXXXXXX`


[root@boyle disktools]# mktemp -d --tmpdir snap.XXXXXXXX
mktemp: invalid option -- -
Usage: mktemp [-V] | [-dqtu] [-p prefix] [template]

So the long option doesn't work. The result of an old version of coreutils (coreutils-5.97-19.el5)  on CentOS5.3  not tested it on RHEL yet but if its different I'd be surprised!



> NOVGFLAG=0

The SHOW and  NOVGFLAG variables aren't actually used any more so could be removed (my fault!)


> IMPORT=0
> DEBUG=0
> DEVNO=0
> 
> -export ORIG_LVM_SYS_DIR="${LVM_SYSTEM_DIR}"
> +if [ -n "${LVM_SYSTEM_DIR}" ]; then
> +    export ORIG_LVM_SYS_DIR="${LVM_SYSTEM_DIR}"
> +    LVMCONF="${LVM_SYSTEM_DIR}/lvm.conf"
> +fi
> 
> trap  cleanup 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
> 
> @@ -94,7 +104,7 @@ do
>      case $1 in
>          -d)    DEBUG=1
>              exec 2> ./${SCRIPTNAME}.log
> -               set -x
> +            set -x
>              echo "Using $TMP_LVM_SYSTEM_DIR/lvm.conf"
>              shift
>              ;;
> @@ -135,13 +145,22 @@ LVM="${LVM_BINARY:-lvm}"
> "${LVM}" version >&/dev/null
> checkvalue $? "${LVM} doesn't look like an lvm binary."
> 
> +if [ -n "$NEWVG" ] ; then
> +    "${LVM}" vgs $NEWVG >& /dev/null
> +    if [ $? -eq 0 ] ; then
> +        echo "Error: New VG ($NEWVG) already exists." >&2
> +        exit 1
> +    fi
> +fi

You dont have to test if the name given with the -n option is unique
because it gets run through the getvgname function which uses the given
name as a starting point to build a unique name, but if you prefer to
enforce the given name rather then just use it as a hint then thats
fine.



> +
> +test -f $LVMCONF
> +checkvalue $? "${LVMCONF} doesn't exist."
> 
> #####################################################################
> ### Get the existing state so we can use it later
> #####################################################################
> 
> -OLDVGS=`"${LVM}" pvs --noheadings --trustcache --separator  2>/dev/null  | awk 
> -F  '/lvm2/{printf("%s ",$2)}'`
> -
> +OLDVGS=`"${LVM}" vgs -o name --noheadings 2>/dev/null`
> 
> #####################################################################
> ### Prepare the temporay lvm environment
> @@ -150,15 +169,19 @@ OLDVGS=`"${LVM}" pvs --noheadings --trustcache 
> --separator  2>/dev/null  | awk -
> ###create filter
> for BLOCK in ${DISKS}
> do
> -    FILTER="\"a|^${BLOCK}$|\",${FILTER}"
> +    FILTER="\"a|^${BLOCK}$|\", ${FILTER}"
> done
> 
> export FILTER="filter=[ ${FILTER} \"r|.*|\" ]"
> 
> -awk -v DEV=${TMP_LVM_SYSTEM_DIR} '/^[[:space:]]*filter/{print 
> ENVIRON["FILTER"];next}\
> -                    /^[[:space:]]*scan/{print "scan = [ \"" DEV  "\" ]";next} \
> +awk -v DEV=${TMP_LVM_SYSTEM_DIR} -v CACHE=${TMP_LVM_SYSTEM_DIR}/cache \
> +                   '/^[[:space:]]*filter/{print ENVIRON["FILTER"];next} \
> +                    /^[[:space:]]*scan/{print "scan = [ \"" DEV "\" ]";next} \
> +                    /^[[:space:]]*cache_dir/{print "cache_dir = \"" CACHE 
> "\"";next} \
>                      {print $0}' < ${LVMCONF} > ${TMP_LVM_SYSTEM_DIR}/lvm.conf


This doesn't work on my setup, there is no cache_dir directive as far as I can tell, I think you're trying to change "cache =" instead so that should be:

awk -v DEV=${TMP_LVM_SYSTEM_DIR} -v CACHE=${TMP_LVM_SYSTEM_DIR}/cache \
                   '/^[[:space:]]*filter/{print ENVIRON["FILTER"];next} \
                    /^[[:space:]]*scan/{print "scan = [ \"" DEV "\" ]";next} \
                    /^[[:space:]]*cache/{print "cache = \"" CACHE "\"";next} \
                     {print $0}' < ${LVMCONF} > ${TMP_LVM_SYSTEM_DIR}/lvm.conf

Otherwise it fails :(   Is this an "upgrade to the latest version" thing?


> +checkvalue $? "Failed to generate ${TMP_LVM_SYSTEM_DIR}/lvm.conf"
> +
> ### set to use new lvm.conf
> export LVM_SYSTEM_DIR=${TMP_LVM_SYSTEM_DIR}
> 
> @@ -166,18 +189,30 @@ export LVM_SYSTEM_DIR=${TMP_LVM_SYSTEM_DIR}
> #####################################################################
> ### Change the uuids.
> #####################################################################
> -PVSCAN=`"${LVM}" pvscan`

I was trying to keep the number of lvm commands to the minimum because they can be pretty slow on some of our systems with a lot of luns presented, so I was trading complicated awk for speed. Wether it really makes a difference is debatable though.


> -### create a space seperated list of VGs where each VG looks like: 
> nameexported?disk1disk2... 
> -VGS=`echo "${PVSCAN}" |awk 
> '$1~/PV/{for(i=1;i<=NF;i++){if($i=="VG"){vg[$(i+1)]=vg[$(i+1)]""$2} \
> -                            if($i=="exported"){x[$(i+2)]="x"}}} \
> -                END{for(k in vg){printf k""x[k] vg[k]" "}}'`


> +PVINFO=`"${LVM}" pvs -o pv_name,vg_name,vg_attr --noheadings --separator : | 
> sed -e "s/ //g"`
> +
> +# output VG info so each line looks like: name:exported?:disk1,disk2,...
> +VGINFO=`echo "${PVINFO}" | \
> +    awk -F : '{{vg[$2]=$1","vg[$2]} \
> +    if($3 ~ /^..x/){x[$2]="x"}} \
> +    END{for(k in vg){printf("%s:%s:%s\n", k, x[k], vg[k])}}'`

You could replace from PVINFO= to here with

VGINFO=`lvm pvs -o pv_name,vg_name,vg_attr --noheadings --separator : | \
                  awk -F : '{sub(/^[[:space:]]*/,"");vg[$2]=$1","vg[$2];if($3 ~ /^..x/){x[$2]="x"}} \ 
                                END{for(k in vg){printf("%s:%s:%s\n", k, x[k], vg[k])}}'`

But I'm a sed-a-phobe ;)


> -for VG in ${VGS}
> +for VG in ${VGINFO}
> do
> -    VGNAME=`echo -e "${VG}" |cut -d -f1`
> -    EXPORTED=`echo -e "${VG}" | cut -d -f2`
> -    PVLIST=`echo -e "${VG}" | cut -d -f3-`
> +    VGNAME=`echo "${VG}" | cut -d: -f1`
> +    EXPORTED=`echo "${VG}" | cut -d: -f2`
> +    PVLIST=`echo "${VG}" | cut -d: -f3- | tr , ' '`

I was avoiding using colons as the seperator because its a valid character in device names, which is why I was then using control characters as separators, but given that we're now linking them all to sensible names it probably doesn't matter.


> +    if [ -z "${VGNAME}" ] ; then
> +        FOLLOWLIST=""
> +        for DEV in $PVLIST; do
> +            FOLLOW=`readlink $DEV`
> +            FOLLOWLIST="$FOLLOW $FOLLOWLIST"
> +        done
> +        echo "Error: Specified PV(s) ($FOLLOWLIST) don't belong to a VG." >&2
> +        exit 1
> +    fi
> 
>      if [ -n "${EXPORTED}" ]
>      then
> @@ -188,20 +223,18 @@ do
>              echo "Volume Group ${VGNAME} exported, skipping."
>              continue
>          fi
> -
>      fi
> 
>      ### change the pv uuids
> -    BLOCKDEVS=`echo ${PVLIST} | tr '' ' '`
> -    if [[ "${BLOCKDEVS}" =~ "unknown" ]]
> +    if [[ "${PVLIST}" =~ "unknown" ]]
>      then
>          echo "Volume Group ${VGNAME} incomplete, skipping."
>          continue
>      fi
> 
> -    for BLOCKDEV in ${BLOCKDEVS}
> +    for BLOCKDEV in ${PVLIST}
>      do
> -        "${LVM}" pvchange --uuid ${BLOCKDEV} --config 'global{activation=0}' 
> +        "${LVM}" pvchange --uuid ${BLOCKDEV} --config 'global{activation=0}'
>          checkvalue $? "Unable to change pvuuid for ${BLOCKDEV}"
>      done
> 
> @@ -225,12 +258,10 @@ done
> ### set to use old lvm.conf
> LVM_SYSTEM_DIR=${ORIG_LVM_SYS_DIR}
> 
> -### make sure all the device nodes we need are straight
> -"${LVM}" vgmknodes >/dev/null
> -
> -### sort out caches.
> +### update the device cache and make sure all
> +### the device nodes we need are straight
> 
> "${LVM}" pvscan
> -"${LVM}" vgscan >/dev/null
>
> +"${LVM}" vgmknodes
> 
> exit 0



      

[linux-lvm] Re: Script to import hardware snapshoted VGs

[Mike Snitzer]

On Tue, May 12 2009 at  7:15pm -0400,
chris procter <chris-procter@talk21.com> wrote:

> > @@ -76,13 +83,16 @@ fi
> > SHOW=0
> > DISKS=""
> > LVMCONF="/etc/lvm/lvm.conf"
> > -TMP_LVM_SYSTEM_DIR=`mktemp -d -t snap.XXXXXXXX`
> > +TMP_LVM_SYSTEM_DIR=`mktemp -d --tmpdir snap.XXXXXXXX`
> 
> 
> [root@boyle disktools]# mktemp -d --tmpdir snap.XXXXXXXX
> mktemp: invalid option -- -
> Usage: mktemp [-V] | [-dqtu] [-p prefix] [template]
> 
> So the long option doesn't work. The result of an old version of coreutils (coreutils-5.97-19.el5)  on CentOS5.3  not tested it on RHEL yet but if its different I'd be surprised!

OK.

> > NOVGFLAG=0
> 
> The SHOW and  NOVGFLAG variables aren't actually used any more so could be removed (my fault!)

OK.

> > @@ -135,13 +145,22 @@ LVM="${LVM_BINARY:-lvm}"
> > "${LVM}" version >&/dev/null
> > checkvalue $? "${LVM} doesn't look like an lvm binary."
> > 
> > +if [ -n "$NEWVG" ] ; then
> > +    "${LVM}" vgs $NEWVG >& /dev/null
> > +    if [ $? -eq 0 ] ; then
> > +        echo "Error: New VG ($NEWVG) already exists." >&2
> > +        exit 1
> > +    fi
> > +fi
> 
> You dont have to test if the name given with the -n option is unique
> because it gets run through the getvgname function which uses the given
> name as a starting point to build a unique name, but if you prefer to
> enforce the given name rather then just use it as a hint then thats
> fine.

I understood as much but I'm on the fence as to whether that behavior is
desirable if a specific name is explicitly provided.

> > export FILTER="filter=[ ${FILTER} \"r|.*|\" ]"
> > 
> > -awk -v DEV=${TMP_LVM_SYSTEM_DIR} '/^[[:space:]]*filter/{print 
> > ENVIRON["FILTER"];next}\
> > -                    /^[[:space:]]*scan/{print "scan = [ \"" DEV  "\" ]";next} \
> > +awk -v DEV=${TMP_LVM_SYSTEM_DIR} -v CACHE=${TMP_LVM_SYSTEM_DIR}/cache \
> > +                   '/^[[:space:]]*filter/{print ENVIRON["FILTER"];next} \
> > +                    /^[[:space:]]*scan/{print "scan = [ \"" DEV "\" ]";next} \
> > +                    /^[[:space:]]*cache_dir/{print "cache_dir = \"" CACHE 
> > "\"";next} \
> >                      {print $0}' < ${LVMCONF} > ${TMP_LVM_SYSTEM_DIR}/lvm.conf
> 
> 
> This doesn't work on my setup, there is no cache_dir directive as far as I can tell, I think you're trying to change "cache =" instead so that should be:
> 
> awk -v DEV=${TMP_LVM_SYSTEM_DIR} -v CACHE=${TMP_LVM_SYSTEM_DIR}/cache \
>                    '/^[[:space:]]*filter/{print ENVIRON["FILTER"];next} \
>                     /^[[:space:]]*scan/{print "scan = [ \"" DEV "\" ]";next} \
>                     /^[[:space:]]*cache/{print "cache = \"" CACHE "\"";next} \
>                      {print $0}' < ${LVMCONF} > ${TMP_LVM_SYSTEM_DIR}/lvm.conf
> 
> Otherwise it fails :(   Is this an "upgrade to the latest version" thing?

Right, the latest version uses "cache_dir".  Could change both I suppose.

> > @@ -166,18 +189,30 @@ export LVM_SYSTEM_DIR=${TMP_LVM_SYSTEM_DIR}
> > #####################################################################
> > ### Change the uuids.
> > #####################################################################
> > -PVSCAN=`"${LVM}" pvscan`
> 
> I was trying to keep the number of lvm commands to the minimum because
> they can be pretty slow on some of our systems with a lot of luns
> presented, so I was trading complicated awk for speed. Wether it
> really makes a difference is debatable though.

AFAIK the latest version of lvm should not have those problems.

> > -### create a space seperated list of VGs where each VG looks like: 
> > nameexported?disk1disk2... 
> > -VGS=`echo "${PVSCAN}" |awk 
> > '$1~/PV/{for(i=1;i<=NF;i++){if($i=="VG"){vg[$(i+1)]=vg[$(i+1)]""$2} \
> > -                            if($i=="exported"){x[$(i+2)]="x"}}} \
> > -                END{for(k in vg){printf k""x[k] vg[k]" "}}'`
> 
> 
> > +PVINFO=`"${LVM}" pvs -o pv_name,vg_name,vg_attr --noheadings --separator : | 
> > sed -e "s/ //g"`
> > +
> > +# output VG info so each line looks like: name:exported?:disk1,disk2,...
> > +VGINFO=`echo "${PVINFO}" | \
> > +    awk -F : '{{vg[$2]=$1","vg[$2]} \
> > +    if($3 ~ /^..x/){x[$2]="x"}} \
> > +    END{for(k in vg){printf("%s:%s:%s\n", k, x[k], vg[k])}}'`
> 
> You could replace from PVINFO= to here with
> 
> VGINFO=`lvm pvs -o pv_name,vg_name,vg_attr --noheadings --separator : | \
>                   awk -F : '{sub(/^[[:space:]]*/,"");vg[$2]=$1","vg[$2];if($3 ~ /^..x/){x[$2]="x"}} \ 
>                                 END{for(k in vg){printf("%s:%s:%s\n", k, x[k], vg[k])}}'`
> 
> But I'm a sed-a-phobe ;)

Sure.

> > -for VG in ${VGS}
> > +for VG in ${VGINFO}
> > do
> > -    VGNAME=`echo -e "${VG}" |cut -d -f1`
> > -    EXPORTED=`echo -e "${VG}" | cut -d -f2`
> > -    PVLIST=`echo -e "${VG}" | cut -d -f3-`
> > +    VGNAME=`echo "${VG}" | cut -d: -f1`
> > +    EXPORTED=`echo "${VG}" | cut -d: -f2`
> > +    PVLIST=`echo "${VG}" | cut -d: -f3- | tr , ' '`
> 
> I was avoiding using colons as the seperator because its a valid
> character in device names, which is why I was then using control
> characters as separators, but given that we're now linking them all
> to sensible names it probably doesn't matter. 

It doesn't matter for PV names but it does for VG names.  I'll have to
think about it.

Mike