[linux-lvm] the lvm shared storage question of the week

[linux-lvm] the lvm shared storage question of the week

[Jean Spirat]

Hello there,

  Is there any hint or any links anyone could share for the use of lvm 
on an AOE storage (shared block device over the network). I wanted to 
know how to create lvm group on a device that is shared among several host.


so i use debian 4 servers that acces a AOE target drive. The servers all 
mount the etherdrive and i want to use lvm to partition this 4Terabyte  
block device in smaller chunck. what should be the best way to do that ?

 While googling about this i found  references to clvm but no 
documentation i could use on how to set it up. Is this the way to go ?


regards,
Jean.

Re: [linux-lvm] the lvm shared storage question of the week

[Jonathan Brassow]

Our testing group has machines that use AOE.  It works well with  
CLVM.  However, they use their machines as a cluster.  It seems to me  
that you want to carve up your storage, but you only want one machine  
to use each piece.

LVM manual briefly mentions CLVM (should apply equally well to debian):
http://www.redhat.com/docs/manuals/enterprise/RHEL-5-manual/Cluster_Logical_Volume_Manager/index.html

You will want to come up to speed on the cluster infrastructure (at  
least the setup) if you want to use CLVM....

  brassow

On Aug 5, 2008, at 8:36 AM, Jean Spirat wrote:

> Hello there,
>
> Is there any hint or any links anyone could share for the use of lvm  
> on an AOE storage (shared block device over the network). I wanted  
> to know how to create lvm group on a device that is shared among  
> several host.
>
>
> so i use debian 4 servers that acces a AOE target drive. The servers  
> all mount the etherdrive and i want to use lvm to partition this  
> 4Terabyte  block device in smaller chunck. what should be the best  
> way to do that ?
>
> While googling about this i found  references to clvm but no  
> documentation i could use on how to set it up. Is this the way to go ?
>
>
> regards,
> Jean.
>
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] the lvm shared storage question of the week

[Jean Spirat]

Jonathan Brassow a �crit :
> Our testing group has machines that use AOE.  It works well with 
> CLVM.  However, they use their machines as a cluster.  It seems to me 
> that you want to carve up your storage, but you only want one machine 
> to use each piece.
>
yes each server will mount a part of the filesystem and not the same 
part on each. This mean a single lvm logical volume will be mounted only 
on one host.

> LVM manual briefly mentions CLVM (should apply equally well to debian):
> http://www.redhat.com/docs/manuals/enterprise/RHEL-5-manual/Cluster_Logical_Volume_Manager/index.html 
>
yes it just speak about it but there is no doc at all, nothing tells us 
if it listen on a tcp port or just communicate by using data on the disk 
or anything about the setup. Also it speak about special configuration 
in lvm.conf but nowhere i see WHAT should be configured :)

i guess with:

locking_type What type of locking to use. 1 is the default, which use 
flocks on files in locking_dir (see below) to avoid conflicting LVM2 
commands running concurrently on a single machine. 0 disables locking 
and risks corrupting your metadata. If set to 2, the tools will load the 
external locking_library (see below). If the tools were configured 
--with-cluster=internal (the default) then 3 means to use built-in 
cluster-wide locking. All changes to logical volumes and their states 
are communicated using locks.

that it should use

locking_type=2


but is this enough ?
>
> You will want to come up to speed on the cluster infrastructure (at 
> least the setup) if you want to use CLVM....
>
sorry i am not native english speaker, what do you mean by that ?

regards,
Jean.

Re: [linux-lvm] the lvm shared storage question of the week

[John Leach]

On Wed, 2008-08-06 at 08:38 +0200, Jean Spirat wrote:

Hi,

> > LVM manual briefly mentions CLVM (should apply equally well to debian):
> > http://www.redhat.com/docs/manuals/enterprise/RHEL-5-manual/Cluster_Logical_Volume_Manager/index.html 
> >
> yes it just speak about it but there is no doc at all, nothing tells us 
> if it listen on a tcp port or just communicate by using data on the disk 
> or anything about the setup. Also it speak about special configuration 
> in lvm.conf but nowhere i see WHAT should be configured :)

CLVM needs a distributed lock manager to work (which communicates over
the network, not via the disk).  This is usually provided with the help
of the RedHat Cluster Suite.

>that it should use
>
>locking_type=2
>
>but is this enough ?

by default, locking type 2 or 3 requires clvmd, which as I say will
require a cluster setup of some kind.

You *can* work without a cluster manager but I'm not sure how advised it
is.  So you leave your locking type as 1 and use "lvchange
--refresh /dev/vg/*" on all nodes to reload metadata from disk whenever
you make a change.  Can be a bit risky.

John.
-- 
http://johnleach.co.uk

Re: [linux-lvm] the lvm shared storage question of the week

[Jean Spirat]

> CLVM needs a distributed lock manager to work (which communicates over
> the network, not via the disk).  This is usually provided with the help
> of the RedHat Cluster Suite.
>   

locking on the disk seems easier to setup than network port firewall 
rules and so on but if it was not done i assume there was a good reason 
for it that my poor brain cannot see :)  if each node is gived a unique 
key then you could use on disk  locking with a special lock metadata and 
a timestamps for reloading configuration for the other node. Of course 
it seems too easy to have not been thinked about so there must be a 
major flaw in my thinking :)



> by default, locking type 2 or 3 requires clvmd, which as I say will
> require a cluster setup of some kind.
>
> You *can* work without a cluster manager but I'm not sure how advised it
> is.  So you leave your locking type as 1 and use "lvchange
> --refresh /dev/vg/*" on all nodes to reload metadata from disk whenever
> you make a change.  Can be a bit risky.
>
> John.
>   
strange that this use case has not handled in an safer/easier way :)


if each lvm partition is mounted on only one box where does the risk 
lies in your opinion ?

regards,
Jean.