[linux-lvm] vgimport/vgexport commands when logically moving disks between systems

[linux-lvm] vgimport/vgexport commands when logically moving disks between systems

[Dave]

Hello,

I believe I might be misunderstanding whether vgimport and vgexport are needed in my
particular situation.  It would be great to get some feedback for clarification.

THE SETUP:

LVM1 (1.0.8-14) on two RedHat AS3 systems (kernel: 2.4.21-47.ELsmp).
I think the same concept applies for LVM2 as well though.
Machine A is primary and Machine B is backup in a two-node Linux heartbeat cluster.
Both machines are connected to the same SAN via fiber, and see the same disks, where
the volume groups reside.

THE STRATEGY:

The idea is for A to have the LVM file systems mounted, and when a failure is
detected, have the LVM file systems "moved" to (or seen by) system B.  The way this
is currently accomplished is for A to do the following upon detection of a failure:

+ unmount file systems
+ deactivate (vgchange -an $vg)
+ export (vgexport $vg)

then, on system B:

+ import & activate (vgimport $vg $disks)
+ mount file systems

THE ISSUE:

The export works as expected on A, but upon import on B, a return code of 4 is
returned meaning "volume group already exists".  The mounting works properly, but
all the disks are shown like this:

"inactive PV /dev/sdx  is in EXPORTED VG $vg"  

when inspected with pvscan.


Does a vgimport and/or vgexport mark the disks themselves, or simply update the
system on which the commands are run???  I suppose that is essentially the heart of
this issue.

I'm starting to believe that for our strategy the vgexport and vgimport commands are
not necessary, and are actually causing the problem.  (The HOWTO mentions these
commands are used to move disks between systems, but perhaps that is meant to refer
to disks that are only physically moved?)

Instead, the following strategy might be correct in case system A fails (Note: no
vgimport or vgexport commands):

+ unmount fs
+ vgchange -an $vg

then, on system B:

+ vgchange -ay $vg
+ mount fs


IS THIS CORRECT???


Thanks in advance for any helpful feedback.
Cheers,
Dave




 
____________________________________________________________________________________
Do you Yahoo!?
Everyone is raving about the all-new Yahoo! Mail beta.
http://new.mail.yahoo.com

Re: [linux-lvm] vgimport/vgexport commands when logically moving disks between systems

[Patrick Caulfield]

Dave wrote:
> Hello,
> 
> I believe I might be misunderstanding whether vgimport and vgexport are needed in my
> particular situation.  It would be great to get some feedback for clarification.
> 
> THE SETUP:
> 
> LVM1 (1.0.8-14) on two RedHat AS3 systems (kernel: 2.4.21-47.ELsmp).
> I think the same concept applies for LVM2 as well though.
> Machine A is primary and Machine B is backup in a two-node Linux heartbeat cluster.
> Both machines are connected to the same SAN via fiber, and see the same disks, where
> the volume groups reside.
> 
> THE STRATEGY:
> 
> The idea is for A to have the LVM file systems mounted, and when a failure is
> detected, have the LVM file systems "moved" to (or seen by) system B.  The way this
> is currently accomplished is for A to do the following upon detection of a failure:
> 
> + unmount file systems
> + deactivate (vgchange -an $vg)
> + export (vgexport $vg)
> 
> then, on system B:
> 
> + import & activate (vgimport $vg $disks)
> + mount file systems
> 
> THE ISSUE:
> 
> The export works as expected on A, but upon import on B, a return code of 4 is
> returned meaning "volume group already exists".  The mounting works properly, but
> all the disks are shown like this:
> 
> "inactive PV /dev/sdx  is in EXPORTED VG $vg"  
> 
> when inspected with pvscan.
> 
> 
> Does a vgimport and/or vgexport mark the disks themselves, or simply update the
> system on which the commands are run???  I suppose that is essentially the heart of
> this issue.


Yes, vgimport/export marks the disks in the volume group. It's really for moving disks between systems where the target system might
have a volume group with the same name as the one to be imported.

> I'm starting to believe that for our strategy the vgexport and vgimport commands are
> not necessary, and are actually causing the problem.  (The HOWTO mentions these
> commands are used to move disks between systems, but perhaps that is meant to refer
> to disks that are only physically moved?)
> 
> Instead, the following strategy might be correct in case system A fails (Note: no
> vgimport or vgexport commands):
> 
> + unmount fs
> + vgchange -an $vg
> 
> then, on system B:
> 
> + vgchange -ay $vg
> + mount fs
> 
> 
> IS THIS CORRECT???
> 

Yes. IF YOU'RE VERY CAREFUL!
vgimport/vgexport are not the tools you want for this job.


-- 

patrick

Re: [linux-lvm] vgimport/vgexport commands when logically moving disks between systems

[Dave]

Hi Patrick, et al.

Thanks for your comments!
I've been doing some testing on this topic, and encountering a bit of strange
behavior which seems to confirm the murkiness with switching VGs between hosts. 
Please bear with my lengthy description, as I'm trying to be as clear as possible. 
I really need to work this out.

THE TEST:

Hosts are P10 (primary node) and P11 (backup node)
VG is activated on P10 and fs is mounted.  To test a switch to P11, I deactivate the
VG on P10, but DO NOT run vgexport (following your suggestion).  I then, run
vgimport on P11 (but P11 reports it already knows about the VG - that's fine), and
then run an activate.  However, when I try to run an e2fsck on the fs, I get the
following error:

/sbin/e2fsck: No such device or address while trying to open /dev/tux-ao/app
Possibly non-existent or swap device?

However, the device does exist, and looks identical to the one on P10:
[P11]$ ls -l /dev/tux-ao/app
brw-rw----    1 root     disk      58,  14 Dec  6 14:46 /dev/tux-ao/app

I was able to fix the problem by putting vgexport back into the mix.  In this case,
I export the VG from P10 and then after an import on P11 I was able to run e2fsck
and mount successfully.  

Also, this (unknown VG) message is somewhat common in pvscan, if a vgexport is not
performed:

> pvscan -- inactive PV "/dev/sdk"  is associated to unknown VG "tux-ao" (run
vgscan)


Here is another clear example of some unexpected behavior (at least to me)...
1. status when $vg on P11 successful - notice the status of ACTIVE on P11 and
EXPORTED on P10

[P11]$ sudo pvscan
pvscan -- reading all physical volumes (this may take a while...)
 ...
pvscan -- ACTIVE   PV "/dev/sdk" of VG "tux-ao"   [27.09 GB / 9.09 GB free]

[P10]$ sudo pvscan
pvscan -- reading all physical volumes (this may take a while...)
 ...
pvscan -- inactive PV "/dev/sdk"  is in EXPORTED VG "tux-ao" [27.09 GB / 9.09 GB
free]

Then...
2. status after $vg is deactivated on P11 and activated on P10 (no vgexport run on
P11 before activation on P10) - Notice the "unknown VG" message on P10!!!  

[P11]$ sudo pvscan
pvscan -- reading all physical volumes (this may take a while...)
 ...
pvscan -- inactive PV "/dev/sdk" of VG "tux-ao"   [27.09 GB / 9.09 GB free]

[P10]$ sudo pvscan
pvscan -- reading all physical volumes (this may take a while...)
 ...
pvscan -- inactive PV "/dev/sdk"  is associated to unknown VG "tux-ao" (run vgscan)

Then...
3.  Then, when I tried an activation on P10 I get no joy at all (ie. cannot perform
operations on the logical volume), despite the fact that it exists:
[P10]$ ls -l /dev/tux-ao/app
brw-rw----    1 root     disk      58,  14 Dec  6 15:04 /dev/tux-ao/app

/sbin/e2fsck: No such device or address while trying to open /dev/tux-ao/app
Possibly non-existent or swap device?

Is this simply flakey behavior with LVM 1.0.8 ?

We have several VGs on the machine, and sometimes we need to move one or two at a
time.  LVM (v1) seems to have trouble here, at least without vgexport.  Anyone know
what might be happening behind the scenes to cause this behavior?  It's looking to
me like I really do need vgexport to make things work the way we want.

Thanks again for helping me to clarify this situation.
Dave





--- Patrick Caulfield <pcaulfie@redhat.com> wrote:

> Dave wrote:
> > Hello,
> > 
> > I believe I might be misunderstanding whether vgimport and vgexport are needed
> in my
> > particular situation.  It would be great to get some feedback for clarification.
> > 
> > THE SETUP:
> > 
> > LVM1 (1.0.8-14) on two RedHat AS3 systems (kernel: 2.4.21-47.ELsmp).
> > I think the same concept applies for LVM2 as well though.
> > Machine A is primary and Machine B is backup in a two-node Linux heartbeat
> cluster.
> > Both machines are connected to the same SAN via fiber, and see the same disks,
> where
> > the volume groups reside.
> > 
> > THE STRATEGY:
> > 
> > The idea is for A to have the LVM file systems mounted, and when a failure is
> > detected, have the LVM file systems "moved" to (or seen by) system B.  The way
> this
> > is currently accomplished is for A to do the following upon detection of a
> failure:
> > 
> > + unmount file systems
> > + deactivate (vgchange -an $vg)
> > + export (vgexport $vg)
> > 
> > then, on system B:
> > 
> > + import & activate (vgimport $vg $disks)
> > + mount file systems
> > 
> > THE ISSUE:
> > 
> > The export works as expected on A, but upon import on B, a return code of 4 is
> > returned meaning "volume group already exists".  The mounting works properly,
> but
> > all the disks are shown like this:
> > 
> > "inactive PV /dev/sdx  is in EXPORTED VG $vg"  
> > 
> > when inspected with pvscan.
> > 
> > 
> > Does a vgimport and/or vgexport mark the disks themselves, or simply update the
> > system on which the commands are run???  I suppose that is essentially the heart
> of
> > this issue.
> 
> 
> Yes, vgimport/export marks the disks in the volume group. It's really for moving
> disks between systems where the target system might
> have a volume group with the same name as the one to be imported.
> 
> > I'm starting to believe that for our strategy the vgexport and vgimport commands
> are
> > not necessary, and are actually causing the problem.  (The HOWTO mentions these
> > commands are used to move disks between systems, but perhaps that is meant to
> refer
> > to disks that are only physically moved?)
> > 
> > Instead, the following strategy might be correct in case system A fails (Note:
> no
> > vgimport or vgexport commands):
> > 
> > + unmount fs
> > + vgchange -an $vg
> > 
> > then, on system B:
> > 
> > + vgchange -ay $vg
> > + mount fs
> > 
> > 
> > IS THIS CORRECT???
> > 
> 
> Yes. IF YOU'RE VERY CAREFUL!
> vgimport/vgexport are not the tools you want for this job.
> 
> 
> -- 
> 
> patrick
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> 



 
____________________________________________________________________________________
Cheap talk?
Check out Yahoo! Messenger's low PC-to-Phone call rates.
http://voice.yahoo.com

Re: [linux-lvm] vgimport/vgexport commands when logically moving disks between systems

[Patrick Caulfield]

Dave wrote:
> Hi Patrick, et al.
> 
> Thanks for your comments!
> I've been doing some testing on this topic, and encountering a bit of strange
> behavior which seems to confirm the murkiness with switching VGs between hosts. 
> Please bear with my lengthy description, as I'm trying to be as clear as possible. 
> I really need to work this out.
> 
> THE TEST:
> 
> Hosts are P10 (primary node) and P11 (backup node)
> VG is activated on P10 and fs is mounted.  To test a switch to P11, I deactivate the
> VG on P10, but DO NOT run vgexport (following your suggestion).  I then, run
> vgimport on P11 (but P11 reports it already knows about the VG - that's fine), and
> then run an activate.  However, when I try to run an e2fsck on the fs, I get the
> following error:
> 
> /sbin/e2fsck: No such device or address while trying to open /dev/tux-ao/app
> Possibly non-existent or swap device?
> 
> However, the device does exist, and looks identical to the one on P10:
> [P11]$ ls -l /dev/tux-ao/app
> brw-rw----    1 root     disk      58,  14 Dec  6 14:46 /dev/tux-ao/app
> 
> I was able to fix the problem by putting vgexport back into the mix.  In this case,
> I export the VG from P10 and then after an import on P11 I was able to run e2fsck
> and mount successfully.  
> 
> Also, this (unknown VG) message is somewhat common in pvscan, if a vgexport is not
> performed:
> 
>> pvscan -- inactive PV "/dev/sdk"  is associated to unknown VG "tux-ao" (run
> vgscan)
> 
> 
> Here is another clear example of some unexpected behavior (at least to me)...
> 1. status when $vg on P11 successful - notice the status of ACTIVE on P11 and
> EXPORTED on P10
> 
> [P11]$ sudo pvscan
> pvscan -- reading all physical volumes (this may take a while...)
>  ...
> pvscan -- ACTIVE   PV "/dev/sdk" of VG "tux-ao"   [27.09 GB / 9.09 GB free]
> 
> [P10]$ sudo pvscan
> pvscan -- reading all physical volumes (this may take a while...)
>  ...
> pvscan -- inactive PV "/dev/sdk"  is in EXPORTED VG "tux-ao" [27.09 GB / 9.09 GB
> free]
> 
> Then...
> 2. status after $vg is deactivated on P11 and activated on P10 (no vgexport run on
> P11 before activation on P10) - Notice the "unknown VG" message on P10!!!  
> 
> [P11]$ sudo pvscan
> pvscan -- reading all physical volumes (this may take a while...)
>  ...
> pvscan -- inactive PV "/dev/sdk" of VG "tux-ao"   [27.09 GB / 9.09 GB free]
> 
> [P10]$ sudo pvscan
> pvscan -- reading all physical volumes (this may take a while...)
>  ...
> pvscan -- inactive PV "/dev/sdk"  is associated to unknown VG "tux-ao" (run vgscan)
> 
> Then...
> 3.  Then, when I tried an activation on P10 I get no joy at all (ie. cannot perform
> operations on the logical volume), despite the fact that it exists:
> [P10]$ ls -l /dev/tux-ao/app
> brw-rw----    1 root     disk      58,  14 Dec  6 15:04 /dev/tux-ao/app
> 
> /sbin/e2fsck: No such device or address while trying to open /dev/tux-ao/app
> Possibly non-existent or swap device?
> 
> Is this simply flakey behavior with LVM 1.0.8 ?
> 
> We have several VGs on the machine, and sometimes we need to move one or two at a
> time.  LVM (v1) seems to have trouble here, at least without vgexport.  Anyone know
> what might be happening behind the scenes to cause this behavior?  It's looking to
> me like I really do need vgexport to make things work the way we want.
> 
> Thanks again for helping me to clarify this situation.

It's rather worrying that the two nodes seem to be reading different data from the same disks.
I can't remember off-hand whether lvm1 does direct-io when it updates metadata, possibly not.
In which case you might have to upgrade to lvm2 (which does).

lvm1 is /not/ a clustering tool ;-)

-- 

patrick

Re: [linux-lvm] vgimport/vgexport commands when logically moving disks between systems

[Dave]

Hi Patrick,

Thanks for clarifying the status of LVM1 (ie. *not* a clustering tool).  That's
somewhat surprising since it does include an LVM script in the resource.d directory.
 In any case, we're not in a huge rush to upgrade to LVM2 unless it really is
necessary, as these are RH AS3 boxes running the 2.4.21-47.ELsmp kernel, and the
docs I've read suggest LVM2 really needs/wants to run under a 2.6.x kernel.

If direct-io to metadata could be set as an option under LVM1, assuming that isn't
the default, that would be helpful.  But, it sounds like no one really wants to
support LVM1 anymore?!

Assuming we want to go forward with an upgrade to LVM2 on the current systems, how
involved is that procedure?  Do we risk losing all our data?

Thanks,
Dave

--- Patrick Caulfield <pcaulfie@redhat.com> wrote:

> Dave wrote:
> > Hi Patrick, et al.
> > 
> > Thanks for your comments!
> > I've been doing some testing on this topic, and encountering a bit of strange
> > behavior which seems to confirm the murkiness with switching VGs between hosts. 
> > Please bear with my lengthy description, as I'm trying to be as clear as
> possible. 
> > I really need to work this out.
> > 
> > THE TEST:
> > 
> > Hosts are P10 (primary node) and P11 (backup node)
> > VG is activated on P10 and fs is mounted.  To test a switch to P11, I deactivate
> the
> > VG on P10, but DO NOT run vgexport (following your suggestion).  I then, run
> > vgimport on P11 (but P11 reports it already knows about the VG - that's fine),
> and
> > then run an activate.  However, when I try to run an e2fsck on the fs, I get the
> > following error:
> > 
> > /sbin/e2fsck: No such device or address while trying to open /dev/tux-ao/app
> > Possibly non-existent or swap device?
> > 
> > However, the device does exist, and looks identical to the one on P10:
> > [P11]$ ls -l /dev/tux-ao/app
> > brw-rw----    1 root     disk      58,  14 Dec  6 14:46 /dev/tux-ao/app
> > 
> > I was able to fix the problem by putting vgexport back into the mix.  In this
> case,
> > I export the VG from P10 and then after an import on P11 I was able to run
> e2fsck
> > and mount successfully.  
> > 
> > Also, this (unknown VG) message is somewhat common in pvscan, if a vgexport is
> not
> > performed:
> > 
> >> pvscan -- inactive PV "/dev/sdk"  is associated to unknown VG "tux-ao" (run
> > vgscan)
> > 
> > 
> > Here is another clear example of some unexpected behavior (at least to me)...
> > 1. status when $vg on P11 successful - notice the status of ACTIVE on P11 and
> > EXPORTED on P10
> > 
> > [P11]$ sudo pvscan
> > pvscan -- reading all physical volumes (this may take a while...)
> >  ...
> > pvscan -- ACTIVE   PV "/dev/sdk" of VG "tux-ao"   [27.09 GB / 9.09 GB free]
> > 
> > [P10]$ sudo pvscan
> > pvscan -- reading all physical volumes (this may take a while...)
> >  ...
> > pvscan -- inactive PV "/dev/sdk"  is in EXPORTED VG "tux-ao" [27.09 GB / 9.09 GB
> > free]
> > 
> > Then...
> > 2. status after $vg is deactivated on P11 and activated on P10 (no vgexport run
> on
> > P11 before activation on P10) - Notice the "unknown VG" message on P10!!!  
> > 
> > [P11]$ sudo pvscan
> > pvscan -- reading all physical volumes (this may take a while...)
> >  ...
> > pvscan -- inactive PV "/dev/sdk" of VG "tux-ao"   [27.09 GB / 9.09 GB free]
> > 
> > [P10]$ sudo pvscan
> > pvscan -- reading all physical volumes (this may take a while...)
> >  ...
> > pvscan -- inactive PV "/dev/sdk"  is associated to unknown VG "tux-ao" (run
> vgscan)
> > 
> > Then...
> > 3.  Then, when I tried an activation on P10 I get no joy at all (ie. cannot
> perform
> > operations on the logical volume), despite the fact that it exists:
> > [P10]$ ls -l /dev/tux-ao/app
> > brw-rw----    1 root     disk      58,  14 Dec  6 15:04 /dev/tux-ao/app
> > 
> > /sbin/e2fsck: No such device or address while trying to open /dev/tux-ao/app
> > Possibly non-existent or swap device?
> > 
> > Is this simply flakey behavior with LVM 1.0.8 ?
> > 
> > We have several VGs on the machine, and sometimes we need to move one or two at
> a
> > time.  LVM (v1) seems to have trouble here, at least without vgexport.  Anyone
> know
> > what might be happening behind the scenes to cause this behavior?  It's looking
> to
> > me like I really do need vgexport to make things work the way we want.
> > 
> > Thanks again for helping me to clarify this situation.
> 
> It's rather worrying that the two nodes seem to be reading different data from the
> same disks.
> I can't remember off-hand whether lvm1 does direct-io when it updates metadata,
> possibly not.
> In which case you might have to upgrade to lvm2 (which does).
> 
> lvm1 is /not/ a clustering tool ;-)
> 
> -- 
> 
> patrick
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> 



 
____________________________________________________________________________________
Yahoo! Music Unlimited
Access over 1 million songs.
http://music.yahoo.com/unlimited

Re: [linux-lvm] vgimport/vgexport commands when logically moving disks between systems

[Patrick Caulfield]

Dave wrote:
> Hi Patrick,
> 
> Thanks for clarifying the status of LVM1 (ie. *not* a clustering tool).  That's
> somewhat surprising since it does include an LVM script in the resource.d directory.
>  In any case, we're not in a huge rush to upgrade to LVM2 unless it really is
> necessary, as these are RH AS3 boxes running the 2.4.21-47.ELsmp kernel, and the
> docs I've read suggest LVM2 really needs/wants to run under a 2.6.x kernel.

Yes. there are device-mapper patches for 2.4 but it's really recommended to run 2.6.

> If direct-io to metadata could be set as an option under LVM1, assuming that isn't
> the default, that would be helpful.  But, it sounds like no one really wants to
> support LVM1 anymore?!

That's true. lvm1 is in maintenance mode for RHEL3.

> Assuming we want to go forward with an upgrade to LVM2 on the current systems, how
> involved is that procedure?  Do we risk losing all our data?

No. lvm2 can read and write lvm1 metadata. you can upgrade it to lvm2 format metadata with a single command (vgconvert)
-- 

patrick

Re: [linux-lvm] vgimport/vgexport commands when logically moving disks between systems

[Alasdair G Kergon]

On Thu, Dec 14, 2006 at 01:49:29PM +0000, Patrick Caulfield wrote:
> Yes. there are device-mapper patches for 2.4 but it's really recommended to run 2.6.

The 2.4 patches have not been updated for a long time so they include bugs
that got fixed in 2.6 and they are missing features.

Anyone fancy updating them?

Alasdair
-- 
agk@redhat.com