[linux-lvm] mirror and snapshot incompatible

[linux-lvm] mirror and snapshot incompatible

[Yoav]

According to lvcreate, a snapshot of a mirrored volume cannot be created 
("Snapshots and mirrors may not yet be mixed").  However, devmapper does 
not seem to have the same limitation when used directly (not through lvm).

Is there a current reason for lvcreate to enforce this rather harsh 
limitation?

What are the risks I'm taking by manually creating a dm-snapshot from a 
dm-mirror device?  (of course I'm also replacing the dm-mirror device 
with a snapshot-origin of the actual dm-mirror device).

If there is an actual reason for preventing this, what is the suggested 
way to consistently backup a mirror volume?  Is there a way to 
temporarily take one of the mirror "legs" (mimage devices)  offline in 
order to back it up, and resync it later?

    Yoav

Re: [linux-lvm] mirror and snapshot incompatible

[Jonathan Brassow]

It is possible to do all the things you are asking through device- 
mapper... but there is no easy way to do it.  (and LVM does not yet  
have the capabilities to manage this yet either.)

  brassow

On Oct 27, 2008, at 9:28 PM, Yoav wrote:

> According to lvcreate, a snapshot of a mirrored volume cannot be  
> created ("Snapshots and mirrors may not yet be mixed").  However,  
> devmapper does not seem to have the same limitation when used  
> directly (not through lvm).
>
> Is there a current reason for lvcreate to enforce this rather harsh  
> limitation?
>
> What are the risks I'm taking by manually creating a dm-snapshot  
> from a dm-mirror device?  (of course I'm also replacing the dm- 
> mirror device with a snapshot-origin of the actual dm-mirror device).
>
> If there is an actual reason for preventing this, what is the  
> suggested way to consistently backup a mirror volume?  Is there a  
> way to temporarily take one of the mirror "legs" (mimage devices)   
> offline in order to back it up, and resync it later?
>
>   Yoav
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] mirror and snapshot incompatible

[Yoav]

I know how to do it through device-mapper.  I'm just worried that maybe 
there's a reason why lvm prevents it.  Are you aware of such reason, or 
is it safe?

Jonathan Brassow wrote:
> It is possible to do all the things you are asking through 
> device-mapper... but there is no easy way to do it.  (and LVM does not 
> yet have the capabilities to manage this yet either.)
>
>  brassow
>
> On Oct 27, 2008, at 9:28 PM, Yoav wrote:
>
>> According to lvcreate, a snapshot of a mirrored volume cannot be 
>> created ("Snapshots and mirrors may not yet be mixed").  However, 
>> devmapper does not seem to have the same limitation when used 
>> directly (not through lvm).
>>
>> Is there a current reason for lvcreate to enforce this rather harsh 
>> limitation?
>>
>> What are the risks I'm taking by manually creating a dm-snapshot from 
>> a dm-mirror device?  (of course I'm also replacing the dm-mirror 
>> device with a snapshot-origin of the actual dm-mirror device).
>>
>> If there is an actual reason for preventing this, what is the 
>> suggested way to consistently backup a mirror volume?  Is there a way 
>> to temporarily take one of the mirror "legs" (mimage devices)  
>> offline in order to back it up, and resync it later?
>>
>>   Yoav
>>
>> _______________________________________________
>> linux-lvm mailing list
>> linux-lvm@redhat.com
>> https://www.redhat.com/mailman/listinfo/linux-lvm
>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

[linux-lvm] clvmd locking disabled

[Stepan Kadlec]

	hello,
	I can't make the clustered LVM running. it can't start the built-in 
clustered locking mechanism (using cman).

	lvm is compiled with following options:

./configure --with-clvmd=cman  --with-cluster=shared 
--libdir=/usr/lib64/ --enable-dmeventd --enable-cmdlib

	does anyone have some experiences that could help me make it working?
	thanks, steve

	when the clvmd starts it reports this:

CLVMD[1772a6d0]: Oct 29 13:16:57 CLVMD started
CLVMD[1772a6d0]: Oct 29 13:16:57 Connected to CMAN
CLVMD[1772a6d0]: Oct 29 13:16:57 CMAN initialisation complete
CLVMD[1772a6d0]: Oct 29 13:16:57 DLM initialisation complete
CLVMD[1772a6d0]: Oct 29 13:16:57 Cluster ready, doing some more 
initialisation
CLVMD[1772a6d0]: Oct 29 13:16:57 starting LVM thread
CLVMD[41d0c940]: Oct 29 13:16:57 LVM thread function started
CLVMD[1772a6d0]: Oct 29 13:16:57 clvmd ready for work
CLVMD[1772a6d0]: Oct 29 13:16:57 Using timeout of 60 seconds
File descriptor 3 (/dev/tty) leaked on lvm invocation. Parent PID 13867: 
clvmd
File descriptor 5 (/dev/pts/3) leaked on lvm invocation. Parent PID 
13867: clvmd
File descriptor 7 (pipe:[12892]) leaked on lvm invocation. Parent PID 
13867: clvmd
File descriptor 8 (/dev/zero) leaked on lvm invocation. Parent PID 
13867: clvmd
   WARNING: Locking disabled. Be careful! This could corrupt your metadata.
CLVMD[41d0c940]: Oct 29 13:16:57 LVM thread waiting for work

Re: [linux-lvm] clvmd locking disabled

[Stepan Kadlec]

one more report:

# vgscan
File descriptor 3 (/dev/tty) leaked on vgscan invocation. Parent PID 
3389: bash
File descriptor 5 (/dev/pts/1) leaked on vgscan invocation. Parent PID 
3389: bash
File descriptor 7 (pipe:[9066]) leaked on vgscan invocation. Parent PID 
3389: bash
   Unknown locking type requested.
   Locking type 3 initialisation failed.

steve

Stepan Kadlec wrote:
>     hello,
>     I can't make the clustered LVM running. it can't start the built-in 
> clustered locking mechanism (using cman).
> 
>     lvm is compiled with following options:
> 
> ./configure --with-clvmd=cman  --with-cluster=shared 
> --libdir=/usr/lib64/ --enable-dmeventd --enable-cmdlib
> 
>     does anyone have some experiences that could help me make it working?
>     thanks, steve
> 
>     when the clvmd starts it reports this:
> 
> CLVMD[1772a6d0]: Oct 29 13:16:57 CLVMD started
> CLVMD[1772a6d0]: Oct 29 13:16:57 Connected to CMAN
> CLVMD[1772a6d0]: Oct 29 13:16:57 CMAN initialisation complete
> CLVMD[1772a6d0]: Oct 29 13:16:57 DLM initialisation complete
> CLVMD[1772a6d0]: Oct 29 13:16:57 Cluster ready, doing some more 
> initialisation
> CLVMD[1772a6d0]: Oct 29 13:16:57 starting LVM thread
> CLVMD[41d0c940]: Oct 29 13:16:57 LVM thread function started
> CLVMD[1772a6d0]: Oct 29 13:16:57 clvmd ready for work
> CLVMD[1772a6d0]: Oct 29 13:16:57 Using timeout of 60 seconds
> File descriptor 3 (/dev/tty) leaked on lvm invocation. Parent PID 13867: 
> clvmd
> File descriptor 5 (/dev/pts/3) leaked on lvm invocation. Parent PID 
> 13867: clvmd
> File descriptor 7 (pipe:[12892]) leaked on lvm invocation. Parent PID 
> 13867: clvmd
> File descriptor 8 (/dev/zero) leaked on lvm invocation. Parent PID 
> 13867: clvmd
>   WARNING: Locking disabled. Be careful! This could corrupt your metadata.
> CLVMD[41d0c940]: Oct 29 13:16:57 LVM thread waiting for work
> 
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> 

-- 
Eurosoftware s.r.o.
skadlec@gk-software.com
+420 379 307 379
+420 724 554 104

Re: [linux-lvm] clvmd locking disabled

[Christine Caulfield]

Stepan Kadlec wrote:
> one more report:
> 
> # vgscan
> File descriptor 3 (/dev/tty) leaked on vgscan invocation. Parent PID
> 3389: bash
> File descriptor 5 (/dev/pts/1) leaked on vgscan invocation. Parent PID
> 3389: bash
> File descriptor 7 (pipe:[9066]) leaked on vgscan invocation. Parent PID
> 3389: bash
>   Unknown locking type requested.
>   Locking type 3 initialisation failed.
> 
> steve
> 
> Stepan Kadlec wrote:
>>     hello,
>>     I can't make the clustered LVM running. it can't start the
>> built-in clustered locking mechanism (using cman).
>>
>>     lvm is compiled with following options:
>>
>> ./configure --with-clvmd=cman  --with-cluster=shared
>> --libdir=/usr/lib64/ --enable-dmeventd --enable-cmdlib


If you've built using with-cluster=shared then the locking type in
lvm.conf should be 2 and not 3, provided you have remembered to install
the shared library that gets built.

I recommend you don't build it shared (the default), and use locking type 3

Chrissie

>>     does anyone have some experiences that could help me make it working?
>>     thanks, steve
>>
>>     when the clvmd starts it reports this:
>>
>> CLVMD[1772a6d0]: Oct 29 13:16:57 CLVMD started
>> CLVMD[1772a6d0]: Oct 29 13:16:57 Connected to CMAN
>> CLVMD[1772a6d0]: Oct 29 13:16:57 CMAN initialisation complete
>> CLVMD[1772a6d0]: Oct 29 13:16:57 DLM initialisation complete
>> CLVMD[1772a6d0]: Oct 29 13:16:57 Cluster ready, doing some more
>> initialisation
>> CLVMD[1772a6d0]: Oct 29 13:16:57 starting LVM thread
>> CLVMD[41d0c940]: Oct 29 13:16:57 LVM thread function started
>> CLVMD[1772a6d0]: Oct 29 13:16:57 clvmd ready for work
>> CLVMD[1772a6d0]: Oct 29 13:16:57 Using timeout of 60 seconds
>> File descriptor 3 (/dev/tty) leaked on lvm invocation. Parent PID
>> 13867: clvmd
>> File descriptor 5 (/dev/pts/3) leaked on lvm invocation. Parent PID
>> 13867: clvmd
>> File descriptor 7 (pipe:[12892]) leaked on lvm invocation. Parent PID
>> 13867: clvmd
>> File descriptor 8 (/dev/zero) leaked on lvm invocation. Parent PID
>> 13867: clvmd
>>   WARNING: Locking disabled. Be careful! This could corrupt your
>> metadata.
>> CLVMD[41d0c940]: Oct 29 13:16:57 LVM thread waiting for work
>>
>>

Re: [linux-lvm] clvmd locking disabled

[Stepan Kadlec]

Christine Caulfield wrote:
> Stepan Kadlec wrote:
>> one more report:
>>
>> # vgscan
>> File descriptor 3 (/dev/tty) leaked on vgscan invocation. Parent PID
>> 3389: bash
>> File descriptor 5 (/dev/pts/1) leaked on vgscan invocation. Parent PID
>> 3389: bash
>> File descriptor 7 (pipe:[9066]) leaked on vgscan invocation. Parent PID
>> 3389: bash
>>   Unknown locking type requested.
>>   Locking type 3 initialisation failed.
>>
>> steve
>>
>> Stepan Kadlec wrote:
>>>     hello,
>>>     I can't make the clustered LVM running. it can't start the
>>> built-in clustered locking mechanism (using cman).
>>>
>>>     lvm is compiled with following options:
>>>
>>> ./configure --with-clvmd=cman  --with-cluster=shared
>>> --libdir=/usr/lib64/ --enable-dmeventd --enable-cmdlib
> 
> 
> If you've built using with-cluster=shared then the locking type in
> lvm.conf should be 2 and not 3, provided you have remembered to install
> the shared library that gets built.
> 
> I recommend you don't build it shared (the default), and use locking type 3
> 

ok, I have recompiled it with --with-cluster=internal, but still seeing:

WARNING: Locking disabled. Be careful! This could corrupt your metadata.

in lvm.conf I have locking_type = 3, complete configure line was:

./configure --with-clvmd=cman  --with-cluster=internal 
--libdir=/usr/lib64/ --enable-dmeventd --enable-cmdlib

cluster was compiled with following options:

./configure --disable_kernel_check --enable_xen --libdir=/usr/lib64/ 
--without_gnbd --without_kernel_modules --without_gfs --without_gfs2

running on kernel 2.6.25

any suggestion?

thanks steve

> Chrissie
> 
>>>     does anyone have some experiences that could help me make it working?
>>>     thanks, steve
>>>
>>>     when the clvmd starts it reports this:
>>>
>>> CLVMD[1772a6d0]: Oct 29 13:16:57 CLVMD started
>>> CLVMD[1772a6d0]: Oct 29 13:16:57 Connected to CMAN
>>> CLVMD[1772a6d0]: Oct 29 13:16:57 CMAN initialisation complete
>>> CLVMD[1772a6d0]: Oct 29 13:16:57 DLM initialisation complete
>>> CLVMD[1772a6d0]: Oct 29 13:16:57 Cluster ready, doing some more
>>> initialisation
>>> CLVMD[1772a6d0]: Oct 29 13:16:57 starting LVM thread
>>> CLVMD[41d0c940]: Oct 29 13:16:57 LVM thread function started
>>> CLVMD[1772a6d0]: Oct 29 13:16:57 clvmd ready for work
>>> CLVMD[1772a6d0]: Oct 29 13:16:57 Using timeout of 60 seconds
>>> File descriptor 3 (/dev/tty) leaked on lvm invocation. Parent PID
>>> 13867: clvmd
>>> File descriptor 5 (/dev/pts/3) leaked on lvm invocation. Parent PID
>>> 13867: clvmd
>>> File descriptor 7 (pipe:[12892]) leaked on lvm invocation. Parent PID
>>> 13867: clvmd
>>> File descriptor 8 (/dev/zero) leaked on lvm invocation. Parent PID
>>> 13867: clvmd
>>>   WARNING: Locking disabled. Be careful! This could corrupt your
>>> metadata.
>>> CLVMD[41d0c940]: Oct 29 13:16:57 LVM thread waiting for work
>>>
>>>
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> 

-- 
Eurosoftware s.r.o.
skadlec@gk-software.com
+420 379 307 379
+420 724 554 104

Re: [linux-lvm] clvmd locking disabled

[Milan Broz]

Stepan Kadlec wrote:
> ok, I have recompiled it with --with-cluster=internal, but still seeing:
> 
> WARNING: Locking disabled. Be careful! This could corrupt your metadata.

Is it during clvmd initialization only?

clvmd tries to initialize locks for possible already activated volumes
(mostly safety/force restart check, clvmd must start before clustered LVs
are activated anyway)

It simply run lvs command (forcing nolocking for this command)
to check which volumes are activated.
The log message is misleading here...

After clvmd initialization, it should work as expected.

Milan

Re: [linux-lvm] clvmd locking disabled

[Stepan Kadlec]

Milan Broz wrote:
> Stepan Kadlec wrote:
>> ok, I have recompiled it with --with-cluster=internal, but still seeing:
>>
>> WARNING: Locking disabled. Be careful! This could corrupt your metadata.
> 
> Is it during clvmd initialization only?
> 

yes. where else could it occur?

> clvmd tries to initialize locks for possible already activated volumes
> (mostly safety/force restart check, clvmd must start before clustered LVs
> are activated anyway)
> 
> It simply run lvs command (forcing nolocking for this command)
> to check which volumes are activated.
> The log message is misleading here...
> 

the LVs are imho not activated but CLVMd still complains about disabled 
locking:

xen01:/etc/init.d # lvscan
File descriptor 3 (/dev/tty) leaked on lvscan invocation. Parent PID 
2637: bash
File descriptor 5 (/dev/pts/5) leaked on lvscan invocation. Parent PID 
2637: bash
File descriptor 7 (pipe:[721181]) leaked on lvscan invocation. Parent 
PID 2637: bash
   inactive          '/dev/xen/test' [3,00 GB] inherit
   inactive          '/dev/xen/test2' [3,00 GB] inherit
xen01:/etc/init.d # lvs
File descriptor 3 (/dev/tty) leaked on lvs invocation. Parent PID 2637: bash
File descriptor 5 (/dev/pts/5) leaked on lvs invocation. Parent PID 
2637: bash
File descriptor 7 (pipe:[721181]) leaked on lvs invocation. Parent PID 
2637: bash
   LV    VG   Attr   LSize Origin Snap%  Move Log Copy%  Convert
   test  xen  -wi--- 3,00G
   test2 xen  -wi--- 3,00G

this is the CLVMd log written during those command invocation:

xen01:/usr/local/src/LVM2.2.02.41 # clvmd -d 2
CLVMD[c7bbc6d0]: Nov  3 19:49:56 CLVMD started
CLVMD[c7bbc6d0]: Nov  3 19:49:56 Connected to CMAN
CLVMD[c7bbc6d0]: Nov  3 19:49:56 CMAN initialisation complete
CLVMD[c7bbc6d0]: Nov  3 19:49:57 DLM initialisation complete
CLVMD[c7bbc6d0]: Nov  3 19:49:57 Cluster ready, doing some more 
initialisation
CLVMD[c7bbc6d0]: Nov  3 19:49:57 starting LVM thread
CLVMD[426e3940]: Nov  3 19:49:57 LVM thread function started
CLVMD[c7bbc6d0]: Nov  3 19:49:57 clvmd ready for work
CLVMD[c7bbc6d0]: Nov  3 19:49:57 Using timeout of 60 seconds
File descriptor 3 (/dev/tty) leaked on lvm invocation. Parent PID 13833: 
clvmd
File descriptor 5 (/dev/pts/3) leaked on lvm invocation. Parent PID 
13833: clvmd
File descriptor 7 (pipe:[712876]) leaked on lvm invocation. Parent PID 
13833: clvmd
File descriptor 8 (/dev/zero) leaked on lvm invocation. Parent PID 
13833: clvmd
   WARNING: Locking disabled. Be careful! This could corrupt your metadata.
CLVMD[426e3940]: Nov  3 19:49:57 LVM thread waiting for work
CLVMD[c7bbc6d0]: Nov  3 19:50:09 Got new connection on fd 12
CLVMD[c7bbc6d0]: Nov  3 19:50:09 Read on local socket 12, len = 26
CLVMD[c7bbc6d0]: Nov  3 19:50:09 creating pipe, [13, 14]
CLVMD[c7bbc6d0]: Nov  3 19:50:09 Creating pre&post thread
CLVMD[40f3e940]: Nov  3 19:50:09 in sub thread: client = 0x592d60
CLVMD[40f3e940]: Nov  3 19:50:09 Sub thread ready for work.
CLVMD[40f3e940]: Nov  3 19:50:09 doing PRE command LOCK_VG 'V_xen' at 1 
(client=0x592d60)
CLVMD[40f3e940]: Nov  3 19:50:09 sync_lock: 'V_xen' mode:3 flags=0
CLVMD[40f3e940]: Nov  3 19:50:09 sync_lock: returning lkid 5d0001
CLVMD[40f3e940]: Nov  3 19:50:09 Writing status 0 down pipe 14
CLVMD[40f3e940]: Nov  3 19:50:09 Waiting to do post command - state = 0
CLVMD[c7bbc6d0]: Nov  3 19:50:09 Created pre&post thread, state = 0
CLVMD[c7bbc6d0]: Nov  3 19:50:09 read on PIPE 13: 4 bytes: status: 0
CLVMD[c7bbc6d0]: Nov  3 19:50:09 background routine status was 0, 
sock_client=0x592d60
CLVMD[c7bbc6d0]: Nov  3 19:50:09 distribute command: XID = 0
CLVMD[c7bbc6d0]: Nov  3 19:50:09 add_to_lvmqueue: cmd=0x593200. 
client=0x592d60, msg=0x592ee0, len=26, csid=(nil), xid=0
CLVMD[426e3940]: Nov  3 19:50:09 process_work_item: local
CLVMD[426e3940]: Nov  3 19:50:09 process_local_command: LOCK_VG (0x33) 
msg=0x593240, msglen =26, client=0x592d60
CLVMD[426e3940]: Nov  3 19:50:09 Dropping metadata for VG xen
CLVMD[426e3940]: Nov  3 19:50:09 Reply from node 
xen01.es.gk-software.com: 0 bytes
CLVMD[426e3940]: Nov  3 19:50:09 Got 1 replies, expecting: 1
CLVMD[426e3940]: Nov  3 19:50:09 LVM thread waiting for work
CLVMD[40f3e940]: Nov  3 19:50:09 Got post command condition...
CLVMD[40f3e940]: Nov  3 19:50:09 Waiting for next pre command
CLVMD[c7bbc6d0]: Nov  3 19:50:09 read on PIPE 13: 4 bytes: status: 0
CLVMD[c7bbc6d0]: Nov  3 19:50:09 background routine status was 0, 
sock_client=0x592d60
CLVMD[c7bbc6d0]: Nov  3 19:50:09 Send local reply
CLVMD[c7bbc6d0]: Nov  3 19:50:09 Read on local socket 12, len = 26
CLVMD[40f3e940]: Nov  3 19:50:09 Got pre command condition...
CLVMD[40f3e940]: Nov  3 19:50:09 doing PRE command LOCK_VG 'V_xen' at 6 
(client=0x592d60)
CLVMD[40f3e940]: Nov  3 19:50:09 sync_unlock: 'V_xen' lkid:5d0001
CLVMD[40f3e940]: Nov  3 19:50:09 Writing status 0 down pipe 14
CLVMD[40f3e940]: Nov  3 19:50:09 Waiting to do post command - state = 0
CLVMD[c7bbc6d0]: Nov  3 19:50:09 read on PIPE 13: 4 bytes: status: 0
CLVMD[c7bbc6d0]: Nov  3 19:50:09 background routine status was 0, 
sock_client=0x592d60
CLVMD[c7bbc6d0]: Nov  3 19:50:09 distribute command: XID = 1
CLVMD[c7bbc6d0]: Nov  3 19:50:09 add_to_lvmqueue: cmd=0x593200. 
client=0x592d60, msg=0x592ee0, len=26, csid=(nil), xid=1
CLVMD[426e3940]: Nov  3 19:50:09 process_work_item: local
CLVMD[426e3940]: Nov  3 19:50:09 process_local_command: LOCK_VG (0x33) 
msg=0x5931d0, msglen =26, client=0x592d60
CLVMD[426e3940]: Nov  3 19:50:09 Dropping metadata for VG xen
CLVMD[426e3940]: Nov  3 19:50:09 Reply from node 
xen01.es.gk-software.com: 0 bytes
CLVMD[426e3940]: Nov  3 19:50:09 Got 1 replies, expecting: 1
CLVMD[426e3940]: Nov  3 19:50:09 LVM thread waiting for work
CLVMD[40f3e940]: Nov  3 19:50:09 Got post command condition...
CLVMD[40f3e940]: Nov  3 19:50:09 Waiting for next pre command
CLVMD[c7bbc6d0]: Nov  3 19:50:09 read on PIPE 13: 4 bytes: status: 0
CLVMD[c7bbc6d0]: Nov  3 19:50:09 background routine status was 0, 
sock_client=0x592d60
CLVMD[c7bbc6d0]: Nov  3 19:50:09 Send local reply
CLVMD[c7bbc6d0]: Nov  3 19:50:09 Read on local socket 12, len = 0
CLVMD[c7bbc6d0]: Nov  3 19:50:09 EOF on local socket: inprogress=0
CLVMD[c7bbc6d0]: Nov  3 19:50:09 Waiting for child thread
CLVMD[40f3e940]: Nov  3 19:50:09 Got pre command condition...
CLVMD[40f3e940]: Nov  3 19:50:09 Subthread finished
CLVMD[c7bbc6d0]: Nov  3 19:50:09 Joined child thread
CLVMD[c7bbc6d0]: Nov  3 19:50:09 ret == 0, errno = 2. removing client
CLVMD[c7bbc6d0]: Nov  3 19:50:09 add_to_lvmqueue: cmd=0x592f10. 
client=0x592d60, msg=(nil), len=0, csid=(nil), xid=1
CLVMD[426e3940]: Nov  3 19:50:09 process_work_item: free fd 12
CLVMD[426e3940]: Nov  3 19:50:09 LVM thread waiting for work
CLVMD[c7bbc6d0]: Nov  3 19:50:12 Got new connection on fd 12
CLVMD[c7bbc6d0]: Nov  3 19:50:12 Read on local socket 12, len = 26
CLVMD[c7bbc6d0]: Nov  3 19:50:12 creating pipe, [13, 14]
CLVMD[c7bbc6d0]: Nov  3 19:50:12 Creating pre&post thread
CLVMD[40f3e940]: Nov  3 19:50:12 in sub thread: client = 0x592d60
CLVMD[40f3e940]: Nov  3 19:50:12 Sub thread ready for work.
CLVMD[40f3e940]: Nov  3 19:50:12 doing PRE command LOCK_VG 'V_xen' at 1 
(client=0x592d60)
CLVMD[40f3e940]: Nov  3 19:50:12 sync_lock: 'V_xen' mode:3 flags=0
CLVMD[40f3e940]: Nov  3 19:50:12 sync_lock: returning lkid 38f0001
CLVMD[40f3e940]: Nov  3 19:50:12 Writing status 0 down pipe 14
CLVMD[40f3e940]: Nov  3 19:50:12 Waiting to do post command - state = 0
CLVMD[c7bbc6d0]: Nov  3 19:50:12 Created pre&post thread, state = 0
CLVMD[c7bbc6d0]: Nov  3 19:50:12 read on PIPE 13: 4 bytes: status: 0
CLVMD[c7bbc6d0]: Nov  3 19:50:12 background routine status was 0, 
sock_client=0x592d60
CLVMD[c7bbc6d0]: Nov  3 19:50:12 distribute command: XID = 2
CLVMD[c7bbc6d0]: Nov  3 19:50:12 add_to_lvmqueue: cmd=0x592f10. 
client=0x592d60, msg=0x592ee0, len=26, csid=(nil), xid=2
CLVMD[426e3940]: Nov  3 19:50:12 process_work_item: local
CLVMD[426e3940]: Nov  3 19:50:12 process_local_command: LOCK_VG (0x33) 
msg=0x592f80, msglen =26, client=0x592d60
CLVMD[426e3940]: Nov  3 19:50:12 Dropping metadata for VG xen
CLVMD[426e3940]: Nov  3 19:50:12 Reply from node 
xen01.es.gk-software.com: 0 bytes
CLVMD[426e3940]: Nov  3 19:50:12 Got 1 replies, expecting: 1
CLVMD[426e3940]: Nov  3 19:50:12 LVM thread waiting for work
CLVMD[40f3e940]: Nov  3 19:50:12 Got post command condition...
CLVMD[40f3e940]: Nov  3 19:50:12 Waiting for next pre command
CLVMD[c7bbc6d0]: Nov  3 19:50:12 read on PIPE 13: 4 bytes: status: 0
CLVMD[c7bbc6d0]: Nov  3 19:50:12 background routine status was 0, 
sock_client=0x592d60
CLVMD[c7bbc6d0]: Nov  3 19:50:12 Send local reply
CLVMD[c7bbc6d0]: Nov  3 19:50:12 Read on local socket 12, len = 26
CLVMD[40f3e940]: Nov  3 19:50:12 Got pre command condition...
CLVMD[40f3e940]: Nov  3 19:50:12 doing PRE command LOCK_VG 'V_xen' at 6 
(client=0x592d60)
CLVMD[40f3e940]: Nov  3 19:50:12 sync_unlock: 'V_xen' lkid:38f0001
CLVMD[40f3e940]: Nov  3 19:50:12 Writing status 0 down pipe 14
CLVMD[40f3e940]: Nov  3 19:50:12 Waiting to do post command - state = 0
CLVMD[c7bbc6d0]: Nov  3 19:50:12 read on PIPE 13: 4 bytes: status: 0
CLVMD[c7bbc6d0]: Nov  3 19:50:12 background routine status was 0, 
sock_client=0x592d60
CLVMD[c7bbc6d0]: Nov  3 19:50:12 distribute command: XID = 3
CLVMD[c7bbc6d0]: Nov  3 19:50:12 add_to_lvmqueue: cmd=0x592f10. 
client=0x592d60, msg=0x592ee0, len=26, csid=(nil), xid=3
CLVMD[426e3940]: Nov  3 19:50:12 process_work_item: local
CLVMD[426e3940]: Nov  3 19:50:12 process_local_command: LOCK_VG (0x33) 
msg=0x592f50, msglen =26, client=0x592d60
CLVMD[426e3940]: Nov  3 19:50:12 Dropping metadata for VG xen
CLVMD[426e3940]: Nov  3 19:50:12 Reply from node 
xen01.es.gk-software.com: 0 bytes
CLVMD[426e3940]: Nov  3 19:50:12 Got 1 replies, expecting: 1
CLVMD[426e3940]: Nov  3 19:50:12 LVM thread waiting for work
CLVMD[40f3e940]: Nov  3 19:50:12 Got post command condition...
CLVMD[40f3e940]: Nov  3 19:50:12 Waiting for next pre command
CLVMD[c7bbc6d0]: Nov  3 19:50:12 read on PIPE 13: 4 bytes: status: 0
CLVMD[c7bbc6d0]: Nov  3 19:50:12 background routine status was 0, 
sock_client=0x592d60
CLVMD[c7bbc6d0]: Nov  3 19:50:12 Send local reply
CLVMD[c7bbc6d0]: Nov  3 19:50:12 Read on local socket 12, len = 0
CLVMD[c7bbc6d0]: Nov  3 19:50:12 EOF on local socket: inprogress=0
CLVMD[c7bbc6d0]: Nov  3 19:50:12 Waiting for child thread
CLVMD[40f3e940]: Nov  3 19:50:12 Got pre command condition...
CLVMD[40f3e940]: Nov  3 19:50:12 Subthread finished
CLVMD[c7bbc6d0]: Nov  3 19:50:12 Joined child thread
CLVMD[c7bbc6d0]: Nov  3 19:50:12 ret == 0, errno = 9. removing client
CLVMD[c7bbc6d0]: Nov  3 19:50:12 add_to_lvmqueue: cmd=0x592f10. 
client=0x592d60, msg=(nil), len=0, csid=(nil), xid=3
CLVMD[426e3940]: Nov  3 19:50:12 process_work_item: free fd 12
CLVMD[426e3940]: Nov  3 19:50:12 LVM thread waiting for work


bye stepan

> After clvmd initialization, it should work as expected.
> 
> Milan
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> 

-- 
Eurosoftware s.r.o.
skadlec@gk-software.com
+420 379 307 379
+420 724 554 104

Re: [linux-lvm] clvmd locking disabled

[Stepan Kadlec]

Milan Broz wrote:
> Stepan Kadlec wrote:
>> ok, I have recompiled it with --with-cluster=internal, but still seeing:
>>
>> WARNING: Locking disabled. Be careful! This could corrupt your metadata.
> 
> Is it during clvmd initialization only?
> 
> clvmd tries to initialize locks for possible already activated volumes
> (mostly safety/force restart check, clvmd must start before clustered LVs
> are activated anyway)
> 
> It simply run lvs command (forcing nolocking for this command)
> to check which volumes are activated.
> The log message is misleading here...
> 
> After clvmd initialization, it should work as expected.
> 

I have looked into the sources:

1) the clvmd starts the thread lvm_thread_fn() from clvmd.c
2) it calls the init_lvm() function from lvm-functions.c
3) it calls the get_initial_state() function from lvm-functions.c
4) in get_initial_state is following call:

FILE *lvs = popen
             ("lvm lvs  --config 'log{command_names=0 prefix=\"\"}' 
--nolocking --noheadings -o vg_uuid,lv_uuid,lv_attr,vg_attr",
              "r");

-> it calls lvm command with the --nolocking option - this is passed to 
all following steps:

5) the lvm commands calls lvm_run_command() function from lvmcmdline.c
6) it calls init_locking() function from locking.c
7) init_locking() than print the "locking disabled" warning, because the 
--nolocking option

SO:
a) is it alright, that the "lvm lvs" command is internally called with 
--nolocking option?
b) does it mean, that locking works alright for the rest of clvmd 
operations but the get_initial_state() call? in this case the debug 
message is very confusing.

bye stepan

> Milan
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> 

-- 
Eurosoftware s.r.o.
skadlec@gk-software.com
+420 379 307 379
+420 724 554 104

Re: [linux-lvm] clvmd locking disabled

[Christine Caulfield]

Stepan Kadlec wrote:
> Christine Caulfield wrote:
>> Stepan Kadlec wrote:
>>> one more report:
>>>
>>> # vgscan
>>> File descriptor 3 (/dev/tty) leaked on vgscan invocation. Parent PID
>>> 3389: bash
>>> File descriptor 5 (/dev/pts/1) leaked on vgscan invocation. Parent PID
>>> 3389: bash
>>> File descriptor 7 (pipe:[9066]) leaked on vgscan invocation. Parent PID
>>> 3389: bash
>>>   Unknown locking type requested.
>>>   Locking type 3 initialisation failed.
>>>
>>> steve
>>>
>>> Stepan Kadlec wrote:
>>>>     hello,
>>>>     I can't make the clustered LVM running. it can't start the
>>>> built-in clustered locking mechanism (using cman).
>>>>
>>>>     lvm is compiled with following options:
>>>>
>>>> ./configure --with-clvmd=cman  --with-cluster=shared
>>>> --libdir=/usr/lib64/ --enable-dmeventd --enable-cmdlib
>>
>>
>> If you've built using with-cluster=shared then the locking type in
>> lvm.conf should be 2 and not 3, provided you have remembered to install
>> the shared library that gets built.
>>
>> I recommend you don't build it shared (the default), and use locking
>> type 3
>>
> 
> ok, I have recompiled it with --with-cluster=internal, but still seeing:
> 
> WARNING: Locking disabled. Be careful! This could corrupt your metadata.
> 
> in lvm.conf I have locking_type = 3, complete configure line was:


Check there isn't another line in lvm.conf that puts the locking_type
back to 1. clvmd prints that line out at startup if it detects that
locking_type is not set to 3, or it is set to 2 and the library name is
wrong, or locking_type is other than 2 or 3.


> ./configure --with-clvmd=cman  --with-cluster=internal
> --libdir=/usr/lib64/ --enable-dmeventd --enable-cmdlib
> 
> cluster was compiled with following options:
> 
> ./configure --disable_kernel_check --enable_xen --libdir=/usr/lib64/
> --without_gnbd --without_kernel_modules --without_gfs --without_gfs2
> 
> running on kernel 2.6.25
> 
> any suggestion?
> 
> thanks steve
> 

Chrissie

Re: [linux-lvm] clvmd locking disabled

[Milan Broz]

Stepan Kadlec wrote:
> one more report:
> 
> # vgscan
> File descriptor 3 (/dev/tty) leaked on vgscan invocation. Parent PID 
> 3389: bash
> File descriptor 5 (/dev/pts/1) leaked on vgscan invocation. Parent PID 
> 3389: bash
> File descriptor 7 (pipe:[9066]) leaked on vgscan invocation. Parent PID 
> 3389: bash

So the calling application (bash) forgot to close some descriptors
(running lvm in mc shell usually produces this:-)

That's just warning, lvm close all descriptors, you can ignore these messages.
(and if the parent process is clvmd, it is bug in clvmd and we should fix it :)

>    Unknown locking type requested.
>    Locking type 3 initialisation failed.

Locking type 2 and 3 are cluster locking
Type 2 is for external locking library, 3 is internal (compiled-in)

For type 3 you should use  --with-cluster=internal in configure.
(you can still switch type 2 with this setting, but if you do not
use own locking library, it is probably not needed).

Milan
--
mbroz@redhat.com

Re: [linux-lvm] mirror and snapshot incompatible

[Jonathan Brassow]

No, there isn't a reason LVM prevents it, other than we haven't gotten  
to it yet.  That being said... because we haven't gotten to it yet  
also means there probably hasn't been a whole lot of testing of such a  
configuration.  Therefore, it's difficult to quantify the safety.

  brassow

On Oct 28, 2008, at 7:32 PM, Yoav wrote:

> I know how to do it through device-mapper.  I'm just worried that  
> maybe there's a reason why lvm prevents it.  Are you aware of such  
> reason, or is it safe?
>
> Jonathan Brassow wrote:
>> It is possible to do all the things you are asking through device- 
>> mapper... but there is no easy way to do it.  (and LVM does not yet  
>> have the capabilities to manage this yet either.)
>>
>> brassow
>>
>> On Oct 27, 2008, at 9:28 PM, Yoav wrote:
>>
>>> According to lvcreate, a snapshot of a mirrored volume cannot be  
>>> created ("Snapshots and mirrors may not yet be mixed").  However,  
>>> devmapper does not seem to have the same limitation when used  
>>> directly (not through lvm).
>>>
>>> Is there a current reason for lvcreate to enforce this rather  
>>> harsh limitation?
>>>
>>> What are the risks I'm taking by manually creating a dm-snapshot  
>>> from a dm-mirror device?  (of course I'm also replacing the dm- 
>>> mirror device with a snapshot-origin of the actual dm-mirror  
>>> device).
>>>
>>> If there is an actual reason for preventing this, what is the  
>>> suggested way to consistently backup a mirror volume?  Is there a  
>>> way to temporarily take one of the mirror "legs" (mimage devices)   
>>> offline in order to back it up, and resync it later?
>>>
>>>  Yoav
>>>
>>> _______________________________________________
>>> linux-lvm mailing list
>>> linux-lvm@redhat.com
>>> https://www.redhat.com/mailman/listinfo/linux-lvm
>>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>>
>> _______________________________________________
>> linux-lvm mailing list
>> linux-lvm@redhat.com
>> https://www.redhat.com/mailman/listinfo/linux-lvm
>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] mirror and snapshot incompatible

[Yoav]

For production environment I guess I'll switch to md+lvm for mirroring 
so that I'll be able to safely snapshot the mirrored fs.  I'll keep 
playing with devmapper mirror combined with snapshot (without lvm) and 
report if it misbehaves.  So far it hasn't, but I only did some limited 
experiments.  I hope some future version of lvm will add support for 
this combination.

    Yoav

Jonathan Brassow wrote:
> No, there isn't a reason LVM prevents it, other than we haven't gotten 
> to it yet.  That being said... because we haven't gotten to it yet 
> also means there probably hasn't been a whole lot of testing of such a 
> configuration.  Therefore, it's difficult to quantify the safety.
>
>  brassow
>
> On Oct 28, 2008, at 7:32 PM, Yoav wrote:
>
>> I know how to do it through device-mapper.  I'm just worried that 
>> maybe there's a reason why lvm prevents it.  Are you aware of such 
>> reason, or is it safe?
>>
>> Jonathan Brassow wrote:
>>> It is possible to do all the things you are asking through 
>>> device-mapper... but there is no easy way to do it.  (and LVM does 
>>> not yet have the capabilities to manage this yet either.)
>>>
>>> brassow
>>>
>>> On Oct 27, 2008, at 9:28 PM, Yoav wrote:
>>>
>>>> According to lvcreate, a snapshot of a mirrored volume cannot be 
>>>> created ("Snapshots and mirrors may not yet be mixed").  However, 
>>>> devmapper does not seem to have the same limitation when used 
>>>> directly (not through lvm).
>>>>
>>>> Is there a current reason for lvcreate to enforce this rather harsh 
>>>> limitation?
>>>>
>>>> What are the risks I'm taking by manually creating a dm-snapshot 
>>>> from a dm-mirror device?  (of course I'm also replacing the 
>>>> dm-mirror device with a snapshot-origin of the actual dm-mirror 
>>>> device).
>>>>
>>>> If there is an actual reason for preventing this, what is the 
>>>> suggested way to consistently backup a mirror volume?  Is there a 
>>>> way to temporarily take one of the mirror "legs" (mimage devices)  
>>>> offline in order to back it up, and resync it later?
>>>>
>>>>  Yoav
>>>>
>>>> _______________________________________________
>>>> linux-lvm mailing list
>>>> linux-lvm@redhat.com
>>>> https://www.redhat.com/mailman/listinfo/linux-lvm
>>>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>>>
>>> _______________________________________________
>>> linux-lvm mailing list
>>> linux-lvm@redhat.com
>>> https://www.redhat.com/mailman/listinfo/linux-lvm
>>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>>
>> _______________________________________________
>> linux-lvm mailing list
>> linux-lvm@redhat.com
>> https://www.redhat.com/mailman/listinfo/linux-lvm
>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

[linux-lvm] snapshot cloning

[marcin.kaluza]

There's another subject I'd like to raise here - snapshot cloning.
Since we use snapshots for copying our (large) oracle databases for
development and crash-test(especially crash test:) purposes, we needed to
come up with a way to clone snapshots.

By the way - it's a cool way to copy databases and it's being used in
solaris/zfs. For devel/(crash)testing it is great, because at little
storage expense you can very quickly copy very large databases (we
actually create snaps of live oracle databases(no archivelogs) and it
works;) I wonder why it's so rarely mentioned here - everybody seem to use
them as temporary backup snapshots only...

I haven't found a way to do it via lvm, but two ways without it. Both
assume than origin (vg00/test) and cloned snapshot(vg00/snap) are not
mounted or written to in any way.

I. the crude but simple way:

1) lvcreate -s -L orig_snap_size -n snapclone vg00/snap
Create a cloning target

2) dd if=/dev/mapper/vg00-snap-cow of=/dev/mapper/vg00-snapclone-cow bs=1M
count=percent_used
Copy the cow table from old to new snapshot. I assume, that since snapshot
cow tables are filled up incrementally,it'll be sufficient to copy just
the meaningful beginning instead of all useless data. This makes cloning a
30GB snapshot that is only 5% full very fast. Correct me if I'm wrong on
this.

3) lvchange -an vg00/test; lvchange -ay vg00/test
Make lvm reload new data for the cloned snapshot. I haven't found other
way to do it so far, so if you know any please let me know.

Done;)

The biggest drawback of this is than lvchange takes off line all other
snapshots as well (so all cloned DB's would have to be closed), and this
is evil;)

II. More elegant way. I've found a perl scipt posted on this list that was
used to restore the origin from a snapshot. It simply reads cow table
offsets and creates a set of dd commands to copy them from cow to the
origin. However replacing the origin with another just-made snapshot of it
creates a clone of the first one.

The drawback is that it uses dd's, so it launches enormous amount of
processes and creates a lot of scattered reads/writes for an actually
continuous area. And this'll probably be slow. But this can probably be
optimized.

However both origin and snapshot still have to be unmounted there's no
need to reload cow tables, so when the script is done, the clone is ready
to use.

III. Finally - the ultimate way ;) It's just an idea - how I imagine it
could work. Create clones on live snaps/lvs in a similar way pvmove works.

1) replace the cow-lv with a mirror target, that will create a copy of the
cow table and still allow it to be used during the operation. If possible,
mirror just enough from the beginnig, so that big snapshots with low %
usage clone quickly.
2) when it's done, suspend the origin and cloned snap to ensure data
cosistency
3) separate the mirrors
4) add a new snapshot entry with cow table on the just-mirrored lv.
5) resume normal operation

done:)

Am I wrong and it's not that simple? I don't know lvm/devmapper internals
yet... However, if it's the right way to go, I might try to write
something like this when I have some time to spare.

I think it's simpler than making snapshots of snapshots (from
implementation's point of view)

What do you think about it?
Martin

Re: [linux-lvm] snapshot cloning

[Les Mikesell]

marcin.kaluza@comarch.pl wrote:
> There's another subject I'd like to raise here - snapshot cloning.
> Since we use snapshots for copying our (large) oracle databases for
> development and crash-test(especially crash test:) purposes, we needed to
> come up with a way to clone snapshots.
> 
> By the way - it's a cool way to copy databases and it's being used in
> solaris/zfs.

Zfs has a send/receive with an incremental mode for remote cloning.  Is 
there anything like that for lvm?

-- 
   Les Mikesell
    lesmikesell@gmail.com