[linux-lvm] Exporting VG

[linux-lvm] Exporting VG

[Xavi Fustero]

I have a cluster with 4 nodes. 3 of them has several disks which I need
to see as an only one disk. After a really strong work I got LVM working
on all the 3 machines !!!! i created three Volume Groups, one for
machine. Now, I would like to export them to get only one virtual group.
I am trying to export them to a machine and merge the volume groups but
I do not success trying to export them.I have not find many
documentation about this ?!?
Thats my 3 machines: N1, N2 and N3.  From N2 I try to export my VG to
N1.
N2> vgchange -a n  'vg_name'
N2> vgexport 'vg_name'

From N1 I type,
N1>vgimport 'vg_name'  /dev/sda3   (this vg is made from /dev/sda3 )
but I only get error messages !!!!


pv_create_name_from_kdev_t -- LEAVING with dev_name: /dev/sda3
pv_read -- LEAVING with ret: 0
vgimport -- ERROR: physical volume "/dev/sda3" doesn't belong to an
exported volume group

Does anybody try this ?!!!!
Thanks in advance,
Xavi !



--
Xavi Fustero Benavent
Institut Municipal d'Investigacio Medica (IMIM)
Departament Genome Informatics
Tel.: +34-93-221 10 09    Ext: 2016

Re: [linux-lvm] Exporting VG

[Andreas Dilger]

Xavi Benavent writes:
> I have a cluster with 4 nodes. 3 of them has several disks which I need
> to see as an only one disk. After a really strong work I got LVM working
> on all the 3 machines !!!! i created three Volume Groups, one for
> machine. Now, I would like to export them to get only one virtual group.

When you do vgexport, this does not do anything like NFS export (does NOT
allow network access to the VG).  It allows you to physically move the
disks to another machine.

> Thats my 3 machines: N1, N2 and N3.  From N2 I try to export my VG to N1.
> N2> vgchange -a n  'vg_name'
> N2> vgexport 'vg_name'
> 
> From N1 I type,
> N1>vgimport 'vg_name'  /dev/sda3   (this vg is made from /dev/sda3 )

Are the disks connected via shared SCSI or Fibre Channel?  Is /dev/sda3
the disk on the local machine?

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: [linux-lvm] Exporting VG

[Jan Niehusmann]

> When you do vgexport, this does not do anything like NFS export (does NOT
> allow network access to the VG).  It allows you to physically move the
> disks to another machine.

Is vgexport/vgimport doing something important? I just moved a harddisk with
a vg to another computer without vgexport/vgimport and didnot notice any 
problems. Or is it only needed if I add a vg to an already running system?

Jan

Re: [linux-lvm] Exporting VG

[Andreas Dilger]

Jan writes:
> > When you do vgexport, this does not do anything like NFS export (does NOT
> > allow network access to the VG).  It allows you to physically move the
> > disks to another machine.
> 
> Is vgexport/vgimport doing something important? I just moved a harddisk with
> a vg to another computer without vgexport/vgimport and didnot notice any 
> problems. Or is it only needed if I add a vg to an already running system?

I think the only thing that vgexport does is mark on disk that the VG
is "exported" and not shown on vgscan or imported when a "vgchange -a"
is done.  On AIX, there is no difference on disk for imported and exported
VGs, the VG state is stored in a config file.

IMHO, we should not store this state in the VG, nor advocate running
"vgscan" on each boot.  From what I have read in the past on this mailing
list, running vgscan on boot is likely to delete important info from
/etc/lvmtab if there is a minor problem, that has to later be restored
(if possible).

It would be a lot safer to keep the state of LVM in /etc/lvmtab, and
only add or remove VGs with administrator actions (explicit vgimport,
vgexport, vgscan (for a new system or major problems).  This at least
lets you know at boot time that a VG has a problem, rather than vgscan
simply deleting it from lvmtab.

What will make this a lot more likely is the work that Heinz is doing
on PVIDs, so that you only store the VGID + vgname in /etc/lvmtab
and then only the PVIDs in the VGDA.  You don't have to worry about
device names changing if disks are added/moved/removed, because you
never depend on the device name.

PVIDs will also make vgimport much easier (like AIX) so you only specify
a single device and the VGDA knows all of the PVIDs that belong to that
volume group.  Using VG names is dangerous if you are moving VGs between
systems in case the two systems have identical VG names.

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: [linux-lvm] Exporting VG

[Heinz J. Mauelshagen]

On Tue, Oct 31, 2000 at 02:05:33PM -0700, Andreas Dilger wrote:
> Jan writes:
> > > When you do vgexport, this does not do anything like NFS export (does NOT
> > > allow network access to the VG).  It allows you to physically move the
> > > disks to another machine.
> > 
> > Is vgexport/vgimport doing something important? I just moved a harddisk with
> > a vg to another computer without vgexport/vgimport and didnot notice any 
> > problems. Or is it only needed if I add a vg to an already running system?
> 
> I think the only thing that vgexport does is mark on disk that the VG
> is "exported" and not shown on vgscan or imported when a "vgchange -a"
> is done.

Correct.

> On AIX, there is no difference on disk for imported and exported
> VGs, the VG state is stored in a config file.
> 
> IMHO, we should not store this state in the VG, nor advocate running
> "vgscan" on each boot.

You are right that there is no need to run vgscan on each boot.
Nevertheless vgscan takes care of any changes in the i/o configuration of the
system in order to make physical volumes accessable using different
device nodes.

> From what I have read in the past on this mailing
> list, running vgscan on boot is likely to delete important info from
> /etc/lvmtab if there is a minor problem, that has to later be restored
> (if possible).

Assumed that it is really that likely it s rather easy to get /etc/lvmtab.d/
files containing the metadata copies back, because they are format
identical to /etc/lvmconf/ files.
/etc/lvmtab contains the 0 delimited volume group names.

> 
> It would be a lot safer to keep the state of LVM in /etc/lvmtab, and
> only add or remove VGs with administrator actions (explicit vgimport,
> vgexport, vgscan (for a new system or major problems).  This at least
> lets you know at boot time that a VG has a problem, rather than vgscan
> simply deleting it from lvmtab.

If we had a database keeping track of changes in the i/o configuration
of the system, we could run vgscan conditionally if needed. Because there's
none I recommended to run it at every boot to avoid i/o path inconsistencies.

> 
> What will make this a lot more likely is the work that Heinz is doing
> on PVIDs, so that you only store the VGID + vgname in /etc/lvmtab
> and then only the PVIDs in the VGDA.

/etc/lvmtab only contains the 0 delimited volume group names.

/etc/lvmtab.d/VolumeGroupName contains a metadata copy of the current
state of a volume group. In case of i/o configuration changes it is up
to vgscan to update this file's contents.
In order to do i/o to the physical volumes we need to know the actual
device node to access the device anyway. This could well happen at
vgchange time wich wouldn't be a valuable different solution.
We could change the concept though to deal with pv_uuids to access a device
in the kernel as well but we needed a mapping function between uuids and
device addresses within the kernel in order to achieve this.

> You don't have to worry about
> device names changing if disks are added/moved/removed, because you
> never depend on the device name.
> 
> PVIDs will also make vgimport much easier (like AIX) so you only specify
> a single device and the VGDA knows all of the PVIDs that belong to that
> volume group.

All right.
Need to implement vgimport support for this though....

> Using VG names is dangerous if you are moving VGs between
> systems in case the two systems have identical VG names.

That's why you should use vgexport/vgimport which avoids problems with
regard to identical volume group names.

-- 

Regards,
Heinz      -- The LVM guy --

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Sistina Software Inc.
Senior Consultant/Developer                       Bartningstr. 12
                                                  64289 Darmstadt
                                                  Germany
Mauelshagen@Sistina.com                           +49 6151 7103 86
                                                       FAX 7103 96
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Re: [linux-lvm] Exporting VG

[Eric M. Hopper]

[-- Attachment #1: Type: text/plain, Size: 2266 bytes --]

On Wed, Nov 01, 2000 at 01:34:33AM +0000, Heinz J. Mauelshagen wrote:
> 
>> From what I have read in the past on this mailing list, running
>> vgscan on boot is likely to delete important info from /etc/lvmtab if
>> there is a minor problem, that has to later be restored (if
>> possible).
> 
> Assumed that it is really that likely it s rather easy to get /etc/lvmtab.d/
> files containing the metadata copies back, because they are format
> identical to /etc/lvmconf/ files.
> /etc/lvmtab contains the 0 delimited volume group names.

	I actually had this problem and it scared the willies out of me.
I was quite surprised to discover I had any left, given my antipathy
towards Microsft and William Gates III, but I digress.

	I didn't know this about /etc/lvmconf and /etc/lvmtab.d.  I had
to very carefully restore them using vcfgrestore.

> If we had a database keeping track of changes in the i/o configuration
> of the system, we could run vgscan conditionally if needed. Because
> there's none I recommended to run it at every boot to avoid i/o path
> inconsistencies.

	You kinda do.  It's called /etc/lvmtab.d.

> /etc/lvmtab.d/VolumeGroupName contains a metadata copy of the current
> state of a volume group. In case of i/o configuration changes it is up
> to vgscan to update this file's contents.

	Why not have vgscan proceed on the initial opinion that the
files contents are correct, and scan the devices mentioned in the file
to see if they still contain exactly the VG data the file says they
ought to?

	This would save a LOT of vgscan time because the most common
case is that everything is the way it was before the machine went down.
If it wasn't, vgscan could revert to its normal behavior.

	It wouldn't help the problem I had though, because apparently
one of the PVs somehow had corrupt VG information, though I have no clue
how it happened.  I should've saved the headers, but I was in too much
of a panic, and too relieved when I got everything working again.

Have fun (if at all possible),
-- 
The best we can hope for concerning the people at large is that they
be properly armed.  -- Alexander Hamilton
-- Eric Hopper (hopper@omnifarious.mn.org http://www.omnifarious.org/~hopper) --

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [linux-lvm] Exporting VG

[Andreas Dilger]

Heinz writes:
> You are right that there is no need to run vgscan on each boot.
> Nevertheless vgscan takes care of any changes in the i/o configuration of
> the system in order to make physical volumes accessable using different
> device nodes.

If we did something like a pvscan on each boot, which identified the
PVID->device mappings, we would not need to change the VG configuration
information each time.  In the future, devfs will automatically do
PVID->device mappings dynamically for us, from what Richard said.

> Assumed that it is really that likely it's rather easy to get /etc/lvmtab.d/
> files containing the metadata copies back, because they are format
> identical to /etc/lvmconf/ files.

But people have problems nonetheless.  It may not be because of vgscan,
or because it's hard to do restores, but that's an extra step that
people have to make when trying to fix a broken system.

> /etc/lvmtab contains the 0 delimited volume group names.

Yes, I was just referring to the whole LVM configuration information.

> /etc/lvmtab.d/VolumeGroupName contains a metadata copy of the current
> state of a volume group. In case of i/o configuration changes it is up
> to vgscan to update this file's contents.

Looking on my development system, /etc/lvmtab.d/vgtest is not quite the
same data as /etc/lvmconf/vgtest.conf.  Is it possible that the backups
in /etc/lvmconf are 1 version behind what's on disk?

For example, the /etc/lvmconf/vgtest.conf file says V_EXP, while
/etc/lvmtab.d/vgtest does not have this (I was testing vgexport and
vgimport last).  The issue is that this is fine for the case where you
made a mistake changing a VG, but if the disk is corrupted, and you
run vgscan it will overwrite the last good copy of the current config
(/etc/lvmtab.d/vgtest).  Any action you do to restore the VGDA will only
return the previous configuration, not the current one.

I think that once we have on-disk backups of the VGDA, and we only
update one at a time, we will be OK.  However, maybe for now we should
also backup the _current_ configuration to /etc/lvmconf/vgtest.conf,
and have the _previous_ configuration in /etc/lvmconf/vgtest.conf.1, etc.

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

[linux-lvm] Exporting VG

[Xavi Fustero]

I have 3 computers: N1, N2 and N3.
N1 has 3 disks  (vg1)
N2 has 1 disk   (vg2)
N3 has 1 disk   (vg3)
I built a Volume group on each machine, now what I would like to do is
to unify these volume groups under a new one volume group visible from
all the 3 machines  (new_vg=vg1 + vg2 + vg3). Is this possible ???
Let me know if anybody tried this.
Thanks in advance,
Xavi !



--
Xavi Fustero Benavent
Institut Municipal d'Investigacio Medica (IMIM)
Departament Genome Informatics
Tel.: +34-93-221 10 09    Ext: 2016

Re: [linux-lvm] Exporting VG

[Eric M. Hopper]

[-- Attachment #1: Type: text/plain, Size: 947 bytes --]

On Thu, Nov 02, 2000 at 11:34:25AM +0100, Xavi Fustero wrote:
> I have 3 computers: N1, N2 and N3.
> N1 has 3 disks  (vg1)
> N2 has 1 disk   (vg2)
> N3 has 1 disk   (vg3)
> I built a Volume group on each machine, now what I would like to do is
> to unify these volume groups under a new one volume group visible from
> all the 3 machines  (new_vg=vg1 + vg2 + vg3). Is this possible ???
> Let me know if anybody tried this.

	No, it isn't with LVM.

	You probably want (Global File System) or GFS.  Though I think
that one tends to assume that your storage is actually free-floating on
a Fibre Channel network, not attached to particular machines.

	LVM doesn't do stuff over a network by itself.

Have fun (if at all possible),
-- 
The best we can hope for concerning the people at large is that they
be properly armed.  -- Alexander Hamilton
-- Eric Hopper (hopper@omnifarious.mn.org http://www.omnifarious.org/~hopper) --

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]