RE: [linux-lvm] LVM1 - "VGDA in kernel and lvmtab are NOT consistent"error following lvm operations

["Roger Lucas" <roger@planbit.co.uk>]

Small world - I was chasing a similar problem this morning with LVM2.

I don't know if your problem is the same as mine, but...

In my system I am using LVM within Dom0.  I am then creating "disks" for the
DomUs from the LVM partitions.  E.g.

(Hydra = Dom0)

root@hydra:~# pvs
  PV         VG    Fmt  Attr PSize   PFree
  /dev/hda3  xenvg lvm2 a-   176.05G 162.43G
root@hydra:~# vgs
  VG    #PV #LV #SN Attr   VSize   VFree
  xenvg   1   7   0 wz--n- 176.05G 162.43G
root@hydra:~# lvs
  LV           VG    Attr   LSize   Origin Snap%  Move Log Copy%
  backupimage  xenvg -ri-ao 512.00M
  harpseal     xenvg -wi-ao   5.00G
  harpseal-lvm xenvg -wi-ao   1.00G
  octopus      xenvg -wi-ao   1.00G
  octopus-lvm  xenvg -wi-ao   1.00G
  tarantula    xenvg -wi-ao   5.00G
  userdisk     xenvg -wi-a- 128.00M
root@hydra:~# cat /etc/xen/octopus
kernel = "/boot/vmlinuz-2.6.16-xen"
ramdisk = "/boot/initrd.img-2.6.16-xen"
memory = 128
name = "octopus"
# Remember in Xen we are limited to three virtual network interfaces per
DomU...
vif = ['mac=aa:00:00:00:00:e7,bridge=xenbr0',
'mac=aa:00:00:00:01:01,bridge=xenbr1',
'mac=aa:00:00:00:02:01,bridge=xenbr2']
disk = ['phy:/dev/xenvg/octopus,hda1,w','phy:/dev/xenvg/octopus-lvm,hda2,w']
hostname = "octopus"
root = "/dev/hda1 ro"
extra = "4"
root@hydra:~#

Now, the DomU is also using LVM:

root@octopus:~# pvs
  PV         VG      Fmt  Attr PSize    PFree
  /dev/hda2  storage lvm2 a-   1020.00M 380.00M
root@octopus:~# vgs
  VG      #PV #LV #SN Attr   VSize    VFree
  storage   1   2   0 wz--n- 1020.00M 380.00M
root@octopus:~# lvs
  LV          VG      Attr   LSize   Origin Snap%  Move Log Copy%
  backupimage storage -ri-ao 512.00M
  userdisk    storage -wi-a- 128.00M
root@octopus:~#


Now we have the problem!  When the Dom0 scans for LVs, it will look on
/dev/hda3 and find the "xenvg" and all its LVs.  It will then look _inside_
these LVs and find the "/dev/storage" group that really belongs to the DomU.
At this point, you have two machines accessing the same LV, which is "bad".

The solution is to restrict the Dom0 LVM to only use the devices that we
know are for scanning.  This is not the default behaviour - the default
behaviour (at least under Debian/(K)Ubuntu) is to scan pretty much every
block device in /dev - and this why the problem occurs.

I changed by Dom0 LVM configuration to only scan /dev/hda as below.

root@hydra:~# cat /etc/lvm/lvm.conf
devices {
    dir = "/dev"
    scan = [ "/dev" ]
    filter =[ "a|/dev/hda|", "r|.*|" ]
    cache = "/etc/lvm/.cache"
    write_cache_state = 1
    sysfs_scan = 1
    md_component_detection = 1
}
/snip/
root@hydra:~#

I'm not an LVM expert and I cannot tell enough from your e-mail to know if
this is your problem, but hopefully this will help you (or maybe someone
else).

BR,

Roger


> -----Original Message-----
> From: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com]
> On Behalf Of Dave
> Sent: 28 July 2006 10:39
> To: linux-lvm@redhat.com
> Subject: [linux-lvm] LVM1 - "VGDA in kernel and lvmtab are NOT
> consistent"error following lvm operations
> 
> Hello,
> 
> I've been using LVM (1.08), which comes by default with RedHat AS3update5,
> for over a year now and am consistently running into a problem.  I hope
> there are still some LVM version 1 users out there who have some knowledge
> about this!!
> 
> After a reboot, LVM typically functions as expected, however, oftentimes,
> after some LVM operations, I get the following sequence:
> 
> [root@mucrrp10 vb]# vgdisplay
> vgdisplay -- ERROR: VGDA in kernel and lvmtab are NOT consistent; please
> run vgscan
> 
> [root@mucrrp10 vb]# vgscan
> vgscan -- reading all physical volumes (this may take a while...)
> vgscan -- found active volume group "prdxptux"
> vgscan -- found exported volume group "prdtuxPV_EXP"
> vgscan -- "/etc/lvmtab" and "/etc/lvmtab.d" successfully created
> vgscan -- WARNING: This program does not do a VGDA backup of your volume
> groups
> 
> [root@mucrrp10 vb]# vgdisplay
> vgdisplay -- ERROR: VGDA in kernel and lvmtab are NOT consistent; please
> run vgscan
> 
> This is troublesome because it makes it difficult to reliably use certain
> LVM commands and trust their results.  I need to minimize reboots as much
> as possible.
> 
> Any help in understanding the cause of this problem, and how to resolve
> it, or avoid it, are greatly appreciated!!
> 
> Here is some version info about the system and software (I can provide
> more information if needed):
> 
> [root@mucrrp10 vb]# cat /etc/redhat-release
>  Red Hat Enterprise Linux AS release 3 (Taroon Update 5)
>  [root@mucrrp10 vb]# uname -a
>  Linux mucrrp10 2.4.21-32.ELsmp #1 SMP Fri Apr 15 21:17:59 EDT 2005 i686
> i686 i386 GNU/Linux
>  [root@mucrrp10 vb]# rpm -qa|grep lvm
>  lvm-1.0.8-12.2
> 
> Thanks in advance for any assistance.
> Regards,
> David
> 
> 
> 
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/