[linux-lvm] add_lockspace done -19

[linux-lvm] add_lockspace done -19

[Moshe Lazarov]

[-- Attachment #1.1: Type: text/plain, Size: 1325 bytes --]

Hi,

I am using the following script to create shared VGs using SANLOCK. However,
I keep getting an error when trying to add the lock space.

 

#!/bin/bash

 

killall sanlock

/etc/init.d/sanlock stop

killall lvmetad

lvmetad

sleep 2

vgremove -ff --config global/use_lvmlockd=0 vglock

 

echo 'y' | pvremove -ff --config global/use_lvmlockd=0 /dev/sdb1

echo 'y' | pvremove -ff --config global/use_lvmlockd=0 /dev/sdc1

echo 'y' | pvremove -ff --config global/use_lvmlockd=0 /dev/sdb2

 

#http://man7.org/linux/man-pages/man8/lvmlockd.8.html

#The first vgcreate command needs to be given the path to a device that has
not yet been initialized with pvcreate. /dev/sdb2 is used for the lock and
should not be

#initialized with pvcreate.

 

#Lock VG & LV

vgcreate --verbose --config global/use_lvmlockd=0 vglock /dev/sdb2

lvcreate --config global/use_lvmlockd=0 -n locklv -L 10MB vglock

 

sanlock direct init -s lsgl:1:/dev/vglock/locklv:0

 

lvmlockd

sleep 2

systemctl start sanlock

lvmlockd -g sanlock --host-id 1

sleep 2

systemctl status sanlock

 

sanlock client add_lockspace -s lsgl:1:/dev/vglock/locklv:0

 

pvcreate -ff --config global/use_lvmlockd=0 /dev/sdc1

vgcreate --verbose --shared vg1 /dev/sdc1 --lock-type sanlock

lvcreate -n slowlv -L 150G vg1 /dev/sdc1

 

Please advise.

 

Thanks.


[-- Attachment #1.2: Type: text/html, Size: 7992 bytes --]

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5524 bytes --]

Re: [linux-lvm] add_lockspace done -19

[David Teigland]

On Thu, Oct 12, 2017 at 06:30:24PM +0000, Moshe Lazarov wrote:
> I am using the following script to create shared VGs using SANLOCK. However,
> I keep getting an error when trying to add the lock space.

The lvmlockd man page you mention lists the steps you should follow
under "Initial set up", but the commands you have shown are nothing
at all like that.  Please follow these steps.  I'm copying from the
man page and adding some of the commands you have used:


1. choose a lock manager

You have chosen sanlock.

2. configure hosts to use lvmlockd

You must edit /etc/lvm/lvm.conf on all of your nodes, setting
global {
 locking_type = 1
 use_lvmlockd = 1
 use_lvmetad = 1 (this is optional)

Then edit /etc/lvm/lvmlocal.conf on all of your nodes, setting
local {
 host_id = 1  (on the first host)
 or
 host_id = 2  (on the second host)
 etc

3. start lvmlockd

# lvmlockd

4. start lock manager

# systemctl start sanlock

5. create VG on shared devices

I get the impression that you would like to create a special, small VG on
sdb2 to hold the sanlock global lock, and then a general purpose VG on
sdc1.  To do that:

# vgcreate --shared global /dev/sdb2

# vgcreate --shared vg1 /dev/sdc1

(You are not required to create the special small VG for the global lock.
If you want to use a single general purpose VG, you could just leave out
the first vgcreate.)

6. start VG on all hosts

# vgchange --lock-start

(run that on all your hosts; they should all start "global" and "vg1")

7. create and activate LVs

lvcreate -n slowlv -L 150G vg1 /dev/sdc1

Once that works, please let me know what part of the man page was not
clear to you.  Thanks


> #Lock VG & LV
> 
> vgcreate --verbose --config global/use_lvmlockd=0 vglock /dev/sdb2
> 
> lvcreate --config global/use_lvmlockd=0 -n locklv -L 10MB vglock
> 
> sanlock direct init -s lsgl:1:/dev/vglock/locklv:0
> 
> sanlock client add_lockspace -s lsgl:1:/dev/vglock/locklv:0

I'm curious what led you to believe that you should use commands like
these?

Re: [linux-lvm] add_lockspace done -19

[Moshe Lazarov]

[-- Attachment #1.1: Type: text/plain, Size: 3011 bytes --]

The command "vgcreate --shared global /dev/sdb2" creates a VG that
eventually is REMOVED with an error:

 

                VG global init failed: invalid parameters for sanlock

                    Removing global-lvmlock (252:0)

                  Failed to initialize lock args for lock type sanlock

                    Removing physical volume "/dev/sdb2" from volume group
"global"

                  Volume group "global" successfully removed

 

From _init_vg_sanlock()

                _lockd_result flags_str (null)

                lockd_result 0xffffff2c flags none lm none

 

Any ideas?

 

 

Thanks.

 

 

 

 

 

-----Original Message-----
From: David Teigland [mailto:teigland@redhat.com] 
Sent: Friday, October 13, 2017 7:41 AM
To: LVM general discussion and development <linux-lvm@redhat.com>
Cc: Moshe Lazarov <Moshe.Lazarov@axxana.com>
Subject: Re: [linux-lvm] add_lockspace done -19

 

On Thu, Oct 12, 2017 at 06:30:24PM +0000, Moshe Lazarov wrote:

> I am using the following script to create shared VGs using SANLOCK. 

> However, I keep getting an error when trying to add the lock space.

 

The lvmlockd man page you mention lists the steps you should follow under
"Initial set up", but the commands you have shown are nothing at all like
that.  Please follow these steps.  I'm copying from the man page and adding
some of the commands you have used:

 

 

1. choose a lock manager

 

You have chosen sanlock.

 

2. configure hosts to use lvmlockd

 

You must edit /etc/lvm/lvm.conf on all of your nodes, setting global {
locking_type = 1  use_lvmlockd = 1  use_lvmetad = 1 (this is optional)

 

Then edit /etc/lvm/lvmlocal.conf on all of your nodes, setting local {
host_id = 1  (on the first host)  or  host_id = 2  (on the second host)  etc

 

3. start lvmlockd

 

# lvmlockd

 

4. start lock manager

 

# systemctl start sanlock

 

5. create VG on shared devices

 

I get the impression that you would like to create a special, small VG on

sdb2 to hold the sanlock global lock, and then a general purpose VG on sdc1.
To do that:

 

# vgcreate --shared global /dev/sdb2

 

# vgcreate --shared vg1 /dev/sdc1

 

(You are not required to create the special small VG for the global lock.

If you want to use a single general purpose VG, you could just leave out the
first vgcreate.)

 

6. start VG on all hosts

 

# vgchange --lock-start

 

(run that on all your hosts; they should all start "global" and "vg1")

 

7. create and activate LVs

 

lvcreate -n slowlv -L 150G vg1 /dev/sdc1

 

Once that works, please let me know what part of the man page was not clear
to you.  Thanks

 

 

> #Lock VG & LV

> 

> vgcreate --verbose --config global/use_lvmlockd=0 vglock /dev/sdb2

> 

> lvcreate --config global/use_lvmlockd=0 -n locklv -L 10MB vglock

> 

> sanlock direct init -s lsgl:1:/dev/vglock/locklv:0

> 

> sanlock client add_lockspace -s lsgl:1:/dev/vglock/locklv:0

 

I'm curious what led you to believe that you should use commands like these?

 


[-- Attachment #1.2: Type: text/html, Size: 17636 bytes --]

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5524 bytes --]

Re: [linux-lvm] add_lockspace done -19

[David Teigland]

On Mon, Oct 16, 2017 at 06:40:17PM +0000, Moshe Lazarov wrote:
> The command "vgcreate --shared global /dev/sdb2" creates a VG that
> eventually is REMOVED with an error:

Please send me:

- the versions of lvm and sanlock you are using

- that vgcreate command with -vvvv, e.g.
  vgcreate -vvvv --shared global /dev/sdb2 2>&1 | tee vgs.log

- after the failure, the debug logs from lvmlockd and sanlock:
  lvmlockctl -d > lvmlockd.log
  sanlock log_dump > sanlock.log

Thanks

Re: [linux-lvm] add_lockspace done -19

[Moshe Lazarov]

[-- Attachment #1.1: Type: text/plain, Size: 850 bytes --]

LVM version - 2.02.173
Sanlock version - 3.5.0

The log files are attached.

Thanks a lot!


-----Original Message-----
From: David Teigland [mailto:teigland@redhat.com] 
Sent: Monday, October 16, 2017 11:54 AM
To: Moshe Lazarov <Moshe.Lazarov@axxana.com>
Cc: LVM general discussion and development <linux-lvm@redhat.com>
Subject: Re: [linux-lvm] add_lockspace done -19

On Mon, Oct 16, 2017 at 06:40:17PM +0000, Moshe Lazarov wrote:
> The command "vgcreate --shared global /dev/sdb2" creates a VG that 
> eventually is REMOVED with an error:

Please send me:

- the versions of lvm and sanlock you are using

- that vgcreate command with -vvvv, e.g.
  vgcreate -vvvv --shared global /dev/sdb2 2>&1 | tee vgs.log

- after the failure, the debug logs from lvmlockd and sanlock:
  lvmlockctl -d > lvmlockd.log
  sanlock log_dump > sanlock.log

Thanks


[-- Attachment #1.2: sanlock.log --]
[-- Type: application/octet-stream, Size: 305 bytes --]

2017-10-17 01:03:12 257069 [8426]: sanlock daemon started 3.5.0 host 04e4b8c2-8e63-4e37-a2f6-e0cf8c5fa6f6.ubuntu16-1
2017-10-17 01:03:16 257073 [8431]: cmd_align 2,9
2017-10-17 01:03:16 257073 [8431]: open error -13 /dev/mapper/global-lvmlock
2017-10-17 01:03:16 257073 [8431]: cmd_align 2,9 done -19

[-- Attachment #1.3: lvmlockd.log --]
[-- Type: application/octet-stream, Size: 1084 bytes --]

1508191390 lvmlockd started
1508191395 new cl 1 pi 2 fd 9
1508191395 recv client[8435] cl 1 running_lm . "" mode iv flags 0
1508191395 send client[8435] cl 1 running_lm . rv 3  
1508191395 set gl_use_sanlock
1508191395 recv vgcreate[8435] cl 1 lock gl "" mode ex flags 0
1508191395 lockspace "" not found for sanlock gl, searching...
1508191395 work search for gl
1508191395 worker found no gl_is_enabled
1508191395 lockspace "" not found after search
1508191395 send vgcreate[8435] cl 1 lock gl rv -210 ENOLS NO_LOCKSPACES,NO_GL_LS,
1508191396 recv vgcreate[8435] cl 1 init vg "global" mode iv flags 0
1508191396 work init_vg global
1508191396 S lvm_global init_vg_san path /dev/mapper/global-lvmlock
1508191396 sanlock daemon version 03050000 proto 00000001
1508191396 S lvm_global init_vg_san bad disk align size -19 /dev/mapper/global-lvmlock
1508191396 send vgcreate[8435] cl 1 init vg rv -212 vg_args lvmlock lv_args none
1508191397 close vgcreate[8435] cl 1 fd 9
1508191473 new cl 2 pi 2 fd 9
1508191473 recv client[8458] cl 2 dump_log . "" mode iv flags 0

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5524 bytes --]

Re: [linux-lvm] add_lockspace done -19

[David Teigland]

It looks like you have a permission problem.  The sanlock daemon needs to
be able to use /dev/mapper/global-lvmlockd, but in your case it is getting
an EACCES error when it tries to use that device.

Check what UID/GID the sanlock daemon is running as, and that this GID is
allowed to access disks.  The sanlock daemon by default will try to run
with UID/GID of sanlock/sanlock.  I have things set up like this to allow
that (the sanlock rpm will typically set this up for you):

$ cat /etc/passwd | grep sanlock
sanlock:x:179:179:sanlock:/var/run/sanlock:/sbin/nologin

$ cat /etc/group | grep sanlock
disk:x:6:sanlock
tape:x:33:sanlock
sanlock:x:179:

You could also configure sanlock to run as root and not bother with the
user/group setup.  Set "uname = root" and "gname = root" in
/etc/sanlock/sanlock.conf.

Dave