From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx1.redhat.com (ext-mx16.extmail.prod.ext.phx2.redhat.com [10.5.110.21]) by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id qAFDa0Or023698 for ; Thu, 15 Nov 2012 08:36:01 -0500 Received: from tropek.jajcus.net (tropek.jajcus.net [84.205.176.49]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id qAFDZw78008064 for ; Thu, 15 Nov 2012 08:35:58 -0500 Received: from localhost (jajo.ipv6.eggsoft.pl [IPv6:2001:6a0:117::1]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by tropek.jajcus.net (Postfix) with ESMTPSA id A8E1B5002 for ; Thu, 15 Nov 2012 14:30:18 +0100 (CET) Date: Thu, 15 Nov 2012 14:30:15 +0100 From: Jacek Konieczny Message-ID: <20121115133014.GB19632@jajo.eggsoft> References: <20121114151612.GD14396@jajo.eggsoft> <50A4B14F.5050700@gmail.com> <20121115100852.GA19632@jajo.eggsoft> <50A4CB76.3020403@gmail.com> MIME-Version: 1.0 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <50A4CB76.3020403@gmail.com> Subject: Re: [linux-lvm] Why do lvcreate with clvmd insist on VG being available on all nodes? Reply-To: LVM general discussion and development List-Id: LVM general discussion and development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Id: Content-Type: text/plain; charset="utf-8" To: LVM general discussion and development On Thu, Nov 15, 2012 at 12:01:10PM +0100, Zdenek Kabelac wrote: > Dne 15.11.2012 11:08, Jacek Konieczny napsal(a): > > On Thu, Nov 15, 2012 at 10:09:35AM +0100, Zdenek Kabelac wrote: > >>> work properly, as I would expect (make the volume available/unavailable > >>> on the node). But an attempt to create a new volume: > >>> > >>> lvcreate -n new_volume -L 1M shared_vg > >>> > >>> fails with: > >>> > >>> Error locking on node 1: Volume group for uuid not found: Hlk5NeaVF0qhDF20RBq61EZaIj5yyUJgGyMo5AQcLfZpJS0DZUcgj7QMd3QPWICL > >>> > >> Haven't really tried to understand what are you trying to achieve, > >> but if you want to have node being activated only on one cluster node, > >> you may easily use lvcreate -aey option. > >> > > My stupid mistake, indeed. > > > > 'lvcreate -an -Z n' and 'lvcreate -aey' do work in such case. > > Though, LVM have some problems with tracking the exclusive activations > > later… > > If you know about any such bug - just open rhbz with full description of such > erroneous case. It was just another mistake of mine. LVM properly tracks the exclusive locks – the volumes were being deactivated by something else. > > Clusters do not have to be symmetrical. Cluster when different nodes > > have a bit different set of resources available are still clusters. > > You want to support different scheme - thus you need to probably write your > own clvmd-like daemon to cover all new cases you bring in with non-symmetrical > cases. I think this will not be needed. > clvmd typical use case is 'vg' used on couple cluster nodes. I see. > While you are probably trying to use N:M mapping of vg and clustered nodes. Exactly. But it seems it should not be a problem in my case. After knowing my mistake I can see LVM already provides the On Thu, Nov 15, 2012 at 12:01:10PM +0100, Zdenek Kabelac wrote: > Dne 15.11.2012 11:08, Jacek Konieczny napsal(a): > > On Thu, Nov 15, 2012 at 10:09:35AM +0100, Zdenek Kabelac wrote: > >>> work properly, as I would expect (make the volume available/unavailable > >>> on the node). But an attempt to create a new volume: > >>> > >>> lvcreate -n new_volume -L 1M shared_vg > >>> > >>> fails with: > >>> > >>> Error locking on node 1: Volume group for uuid not found: Hlk5NeaVF0qhDF20RBq61EZaIj5yyUJgGyMo5AQcLfZpJS0DZUcgj7QMd3QPWICL > >>> > >> Haven't really tried to understand what are you trying to achieve, > >> but if you want to have node being activated only on one cluster node, > >> you may easily use lvcreate -aey option. > >> > > My stupid mistake, indeed. > > > > 'lvcreate -an -Z n' and 'lvcreate -aey' do work in such case. > > Though, LVM have some problems with tracking the exclusive activations > > later… > > If you know about any such bug - just open rhbz with full description of such > erroneous case. It was just another mistake of mine. LVM properly tracks the exclusive locks – the volumes were being deactivated by something else. > > Clusters do not have to be symmetrical. Cluster when different nodes > > have a bit different set of resources available are still clusters. > > You want to support different scheme - thus you need to probably write your > own clvmd-like daemon to cover all new cases you bring in with non-symmetrical > cases. I think this will not be needed. > clvmd typical use case is 'vg' used on couple cluster nodes. I see. > While you are probably trying to use N:M mapping of vg and clustered nodes. Exactly. But it seems it should not be a problem in my case. After knowing my mistake I can see LVM already provides the functionality I need. To summarize: - The default LV activation mode is '-ay', which means, for clustered volume groups, that the volume is to be active on every node in the cluster. – The activation is not always explicit. 'lvcreate' uses '-ay' when no other '-a' option is given. – The '-ay' activation won't work if any node in the cluster cannot access the volume group (e.g. when the DRBD device holding it is Secondary or not configured at that node). – However, my use case doesn't need more than one node using any of the volumes at any time. In fact, it is very important that only a single machine uses each LV at a time. – For this scenario '-aey' should be always used. – Explicitly using '-aey' with 'lvcreate' fixes the 'Volume group for uuid not found' for me. – Other tests made showed that the locking works as expected when a volume group becomes available and unavailable on various cluster nodes. A node having access to a VG can create and/or activate LVs there in exclusive node and all other nodes will comply with that lock whenever they gain access to this VG. So, it seems that clvmd is not that bound to the 'symmetrical cluster' scenario, provided no more than one node needs to access a volume at a time. Does this make sense? Greets, Jacek