[linux-lvm] Two questions about cluster support (2 problems, actually)

[linux-lvm] Two questions about cluster support (2 problems, actually)

[Digimer]

Hi all,

   Last year, when I last setup an LVM for cluster mode (running on a 
DRBD partition in Primary/Primary mode), I needed to recompile the RPM 
for CentOS 5.4 in order to get the liblvm2clusterlock.so to install. 
This was in reference to an initial error report I saw here: 
http://www.centos.org/modules/newbb/viewtopic.php?forum=37&post_id=28736&topic_id=8449 
in 2007.

   Is this still a problem? I don't see that file still (fully updated 
CentOS 5.4 x86_64). Without that file, I can't try 'locking_type = 2'.

   Further, when I try to use 'locking_type = 3', restart 'cman' and 
then try to run 'pvdisplay', I *only* see the DRBD LVM PV. The 
underlying PV used by Xen's dom0 no longer shows:

# df -hP
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/san02-lv01   19G  3.0G   15G  17% /
/dev/sda1             244M   35M  196M  16% /boot
tmpfs                 845M     0  845M   0% /dev/shm
none                  845M  104K  845M   1% /var/lib/xenstored

# pvdisplay
   connect() failed on local socket: Connection refused
   WARNING: Falling back to local file-based locking.
   Volume Groups with the clustered attribute will be inaccessible.
   "/dev/drbd0" is a new physical volume of "399.99 GB"
   --- NEW Physical volume ---
   PV Name               /dev/drbd0
   VG Name
   PV Size               399.99 GB
   Allocatable           NO
   PE Size (KByte)       0
   Total PE              0
   Free PE               0
   Allocated PE          0
   PV UUID               kklTST-D2ta-NRxL-Zaa2-Fa0T-ebkB-KhSq1b

   Resetting 'locking_type=1' and even after a reboot, I still can't see 
the original LVM PV.

-- 
Digimer
E-Mail:         linux@alteeve.com
AN!Whitepapers: http://alteeve.com
Node Assassin:  http://nodeassassin.org

Re: [linux-lvm] Two questions about cluster support (2 problems, actually)

[Milan Broz]

On 03/16/2010 08:23 PM, Digimer wrote:
>    Last year, when I last setup an LVM for cluster mode (running on a 
> DRBD partition in Primary/Primary mode), I needed to recompile the RPM 
> for CentOS 5.4 in order to get the liblvm2clusterlock.so to install. 

Don't do this. Use "locking_type = 3", or better, use
lvmconf --enable-cluster /--disable-cluster to set proper cluster locking type.

(locking_type = 2 was external locking which need that *.so,
3 is internal cluster locking - basically the same, just compiled in base package.
CentOS 5 should always use 3 for cluster locking.)

(but configuring DRBD is separate thing to clustered lvm)

> # pvdisplay
>    connect() failed on local socket: Connection refused
>    WARNING: Falling back to local file-based locking.
>    Volume Groups with the clustered attribute will be inaccessible.
>    "/dev/drbd0" is a new physical volume of "399.99 GB"

apparently clvmd is not running, you cannot see clustered PVs.

Anyway, the DRBD need to setup filters to work properly with lvm2,
(filter out underlying drbd device)
(or update to new lvm2 version, RHEL5/CentOS5 build will be available
in next minor update).
For more info see https://bugzilla.redhat.com/show_bug.cgi?id=530881
(not sure it is related, but for archive and other users...)

Milan
--
mbroz@redhat.com

Re: [linux-lvm] Two questions about cluster support (2 problems, actually)

[Digimer]

On 10-03-16 04:50 PM, Milan Broz wrote:
>> # pvdisplay
>>     connect() failed on local socket: Connection refused
>>     WARNING: Falling back to local file-based locking.
>>     Volume Groups with the clustered attribute will be inaccessible.
>>     "/dev/drbd0" is a new physical volume of "399.99 GB"
>
> apparently clvmd is not running, you cannot see clustered PVs.
>
> Anyway, the DRBD need to setup filters to work properly with lvm2,
> (filter out underlying drbd device)
> (or update to new lvm2 version, RHEL5/CentOS5 build will be available
> in next minor update).
> For more info see https://bugzilla.redhat.com/show_bug.cgi?id=530881
> (not sure it is related, but for archive and other users...)
>
> Milan

Thanks for the reply, Milan!

I had made the change to the 'filter' argument, but I was using 'filter 
= [ "a|drbd.*|", "r|.*|" ]'. I switched to 'filter = [ "a|^/dev/drbd|", 
"a/sd*/", "r/.*/" ]' and started 'clvmd'. This seems to have solved the 
problem, so thank you.

If I may ask another question though;

My DRBD device is '/dev/drbd0' which sits on either node's 
'/dev/san0x/lv02'. When I ran 'pvcreate /dev/drbd0' the PV was created 
successfully and 'pvdisplay' shows the new PV on both nodes. However, I 
see this:

# pvdisplay
   --- Physical volume ---
   PV Name               /dev/sda2
   VG Name               san02
   PV Size               465.51 GB / not usable 14.52 MB
   Allocatable           yes
   PE Size (KByte)       32768
   Total PE              14896
   Free PE               1407
   Allocated PE          13489
   PV UUID               ru7tmz-Qd4d-aatg-4ReY-TPPq-gMll-Z5ZOkL

   "/dev/san02/lv02" is a new physical volume of "399.99 GB"
   --- NEW Physical volume ---
   PV Name               /dev/san02/lv02
   VG Name
   PV Size               399.99 GB
   Allocatable           NO
   PE Size (KByte)       0
   Total PE              0
   Free PE               0
   Allocated PE          0
   PV UUID               MjwXhY-T3dw-IUkI-ZdSI-FKEB-KJdq-nl9GUo

Note that the 'PV Name' is '/dev/san02/lv02' but the device passed was 
'pvcreate /dev/drbd0'. Is this correct? I'd expect it to show the DRBD 
device, not the DRBD's underlying logical volume.

-- 
Digimer
E-Mail:         linux@alteeve.com
AN!Whitepapers: http://alteeve.com
Node Assassin:  http://nodeassassin.org

[linux-lvm] Fixed! (was: Re: Two questions about cluster support (2 problems, actually))

[Digimer]

On 10-03-16 10:57 PM, Digimer wrote:
> Thanks for the reply, Milan!
>
> I had made the change to the 'filter' argument, but I was using 'filter
> = [ "a|drbd.*|", "r|.*|" ]'. I switched to 'filter = [ "a|^/dev/drbd|",
> "a/sd*/", "r/.*/" ]' and started 'clvmd'. This seems to have solved the
> problem, so thank you.
>
> If I may ask another question though;
>
> My DRBD device is '/dev/drbd0' which sits on either node's
> '/dev/san0x/lv02'. When I ran 'pvcreate /dev/drbd0' the PV was created
> successfully and 'pvdisplay' shows the new PV on both nodes. However, I
> see this:
>
> # pvdisplay
> --- Physical volume ---
> PV Name /dev/sda2
> VG Name san02
> PV Size 465.51 GB / not usable 14.52 MB
> Allocatable yes
> PE Size (KByte) 32768
> Total PE 14896
> Free PE 1407
> Allocated PE 13489
> PV UUID ru7tmz-Qd4d-aatg-4ReY-TPPq-gMll-Z5ZOkL
>
> "/dev/san02/lv02" is a new physical volume of "399.99 GB"
> --- NEW Physical volume ---
> PV Name /dev/san02/lv02
> VG Name
> PV Size 399.99 GB
> Allocatable NO
> PE Size (KByte) 0
> Total PE 0
> Free PE 0
> Allocated PE 0
> PV UUID MjwXhY-T3dw-IUkI-ZdSI-FKEB-KJdq-nl9GUo
>
> Note that the 'PV Name' is '/dev/san02/lv02' but the device passed was
> 'pvcreate /dev/drbd0'. Is this correct? I'd expect it to show the DRBD
> device, not the DRBD's underlying logical volume.
>

I switched back to my old syntax and added an entry for 'sdX' devices, 
restarted clvmd on both nodes and every seems to be fine now! Here is my 
latest, working, filter line:

filter = [ "a|drbd.*|", "a|sd.*|", "r|.*|" ]

Thanks for the help! I'll be looking forward to testing out the fix in 
CentOS 5.5. :)

-- 
Digimer
E-Mail:         linux@alteeve.com
AN!Whitepapers: http://alteeve.com
Node Assassin:  http://nodeassassin.org