RE: Question: array locking, possible?

RE: Question: array locking, possible?

[Stern, Rick (Serviceguard Linux)]

There is more interest, just not vocal.

May want to look at LVM2 and its ability to use tagging to control enablement of VGs. This way it is not HW dependent.

-----Original Message-----
From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Chris Osicki
Sent: Thursday, February 09, 2006 2:26 AM
To: linux-raid@vger.kernel.org
Subject: Re: Question: array locking, possible?



It looks like we are the only two md users interested in such a
feature.
Not enough to get Neil's attention ;-)

Regards,
Chris

On Wed, 8 Feb 2006 21:45:33 +0100
Jure Peèar <pegasus@nerv.eu.org> wrote:

> On Wed, 8 Feb 2006 11:55:49 +0100
> Chris Osicki <osk@admin.swisscom-mobile.ch> wrote:
> 
> > 
> > 
> > I was thinking about it, I have no idea how to do it on Linux if ever possible.
> > I connect over fibre channel SAN, using QLogic QLA2312 HBAS, if it matters.
> > 
> > Anyone any hints?
> 
> I too am running a jbod with md raid between two machines. So far md never
> caused any kind of problems, altough I did have situations where both
> machines were syncing mirrors at once.
> 
> If there's a little tool to reserve a disk via scsi, I'd like to know about
> it too. Even a piece of code would be enough.
> 
> 
> -- 
> 
> Jure Peèar
> http://jure.pecar.org/
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Question: array locking, possible?

[Luca Berra]

On Thu, Feb 09, 2006 at 10:28:58AM -0800, Stern, Rick (Serviceguard Linux) wrote:
>There is more interest, just not vocal.
>
>May want to look at LVM2 and its ability to use tagging to control enablement of VGs. This way it is not HW dependent.
>
I believe there is space in md1 superblock for a "cluster/exclusive"
flag, if not the name field could be used
what is missing is an interface between mdadm and cmcld so mdadm can ask
cmcld permission to activate an array with the "cluster/exclusive" flag
set.

L.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: Question: array locking, possible?

[Chris Osicki]

Luca

On Thu, 9 Feb 2006 21:48:48 +0100
Luca Berra <luca.berra@comedia.it> wrote:

> On Thu, Feb 09, 2006 at 10:28:58AM -0800, Stern, Rick (Serviceguard Linux) wrote:
> >There is more interest, just not vocal.
> >
> >May want to look at LVM2 and its ability to use tagging to control enablement of VGs. This way it is not HW dependent.
> >
> I believe there is space in md1 superblock for a "cluster/exclusive"
> flag, if not the name field could be used

Great if there is space for it there is a hope.
Unfortunately I don't think my programming skills are up to
such a task as making proof-of-concept patches.

> what is missing is an interface between mdadm and cmcld so mdadm can ask
> cmcld permission to activate an array with the "cluster/exclusive" flag
> set.

For the time being we could live without it. I'm convinced HP would
make use of it once it's there.

And I wouldn't say mdadm should get permission from cmcld (for those
who don't know Service Guard cluster software from HP: cmcld is
the Cluster daemon). IMHO cmcld should clear the flag on the array
when initiating a fail-over in case the host which used it crashed.

Once again, what I would like it for is for preventing two hosts writing
the array at the same time because I accidentally activated it.
Without cmcld's awareness of the "cluster/exclusive" flag I would
always run mdadm with the '--force' option to enable the array during
package startup, because if I trust the cluster software I know the
fail-over is happening because the other node crashed or it is a
manual (clean) fail-over. 

We can discuss details of SG integration after Neil implemented this
flag. I can hope, you already found space for it ... ;-)

Regards,
Chris


> 
> L.
> 
> -- 
> Luca Berra -- bluca@comedia.it
>         Communication Media & Services S.r.l.
>  /"\
>  \ /     ASCII RIBBON CAMPAIGN
>   X        AGAINST HTML MAIL
>  / \
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: Question: array locking, possible?

[Luca Berra]

[-- Attachment #1: Type: text/plain, Size: 2847 bytes --]

On Mon, Feb 13, 2006 at 06:52:47PM +0100, Chris Osicki wrote:
>
>Luca
>
>On Thu, 9 Feb 2006 21:48:48 +0100
>Luca Berra <luca.berra@comedia.it> wrote:
>
>> On Thu, Feb 09, 2006 at 10:28:58AM -0800, Stern, Rick (Serviceguard Linux) wrote:
>> >There is more interest, just not vocal.
>> >
>> >May want to look at LVM2 and its ability to use tagging to control enablement of VGs. This way it is not HW dependent.
>> >
>> I believe there is space in md1 superblock for a "cluster/exclusive"
>> flag, if not the name field could be used
>
>Great if there is space for it there is a hope.
>Unfortunately I don't think my programming skills are up to
>such a task as making proof-of-concept patches.

i was thinking of adding a bit in the feature_map flags to enable this
kind of behaviour, the downside of it is that kernel space code has to
be updated to account for this flags, as it is for anything in the
superblock except for name.

Neil, what would you think of reserving some more space in the superblock for
other data which can be used from user-space?

i believe playing with name is a kludge.

>> what is missing is an interface between mdadm and cmcld so mdadm can ask
>> cmcld permission to activate an array with the "cluster/exclusive" flag
>> set.
>
>For the time being we could live without it. I'm convinced HP would
>make use of it once it's there.

i was thinking something like a socket based interface between mdadm and
a generic cluster daemon, non necessarily cmcld.

>And I wouldn't say mdadm should get permission from cmcld (for those
>who don't know Service Guard cluster software from HP: cmcld is
>the Cluster daemon). IMHO cmcld should clear the flag on the array
>when initiating a fail-over in case the host which used it crashed.
no, i don't like the flag to be cleared, there is too much space for a
race. The flag should be permanent (unless it is forcibly removed with
mdadm --grow).

>Once again, what I would like it for is for preventing two hosts writing
>the array at the same time because I accidentally activated it.
>Without cmcld's awareness of the "cluster/exclusive" flag I would
>always run mdadm with the '--force' option to enable the array during
>package startup, because if I trust the cluster software I know the
>fail-over is happening because the other node crashed or it is a
>manual (clean) fail-over. 

if you only want this, it could be entirely implemented into mdadm, just
adding a exclusive flag to the ARRAY line in mdadm.conf
this is not foolproof, as it will only prevent "mdadm -As" from assembling
a device, providing the identification information on the command line
or running something like "mdadm -Asc partitions", to fool it.


-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

[-- Attachment #2: mdadm-2.3.1-exclusive.patch --]
[-- Type: text/plain, Size: 3766 bytes --]

diff -urN mdadm-2.3.1/Assemble.c mdadm-2.3.1.exclusive/Assemble.c
--- mdadm-2.3.1/Assemble.c	2006-01-25 08:01:10.000000000 +0100
+++ mdadm-2.3.1.exclusive/Assemble.c	2006-02-13 22:48:04.000000000 +0100
@@ -34,7 +34,7 @@
 	     mddev_dev_t devlist,
 	     int readonly, int runstop,
 	     char *update,
-	     int verbose, int force)
+	     int verbose, int force, int exclusive)
 {
 	/*
 	 * The task of Assemble is to find a collection of
@@ -255,6 +255,15 @@
 			continue;
 		}
 
+		if (ident->exclusive != UnSet &&
+		    !exclusive ) {
+			if ((inargv && verbose >= 0) || verbose > 0)
+				fprintf(stderr, Name ": %s can be activated in exclusive mode only.\n",
+					devname);
+			continue;
+		}
+
+
 		/* If we are this far, then we are commited to this device.
 		 * If the super_block doesn't exist, or doesn't match others,
 		 * then we cannot continue
diff -urN mdadm-2.3.1/ReadMe.c mdadm-2.3.1.exclusive/ReadMe.c
--- mdadm-2.3.1/ReadMe.c	2006-02-06 05:09:35.000000000 +0100
+++ mdadm-2.3.1.exclusive/ReadMe.c	2006-02-13 22:27:26.000000000 +0100
@@ -147,6 +147,7 @@
     {"scan",      0, 0, 's'},
     {"force",	  0, 0, 'f'},
     {"update",	  1, 0, 'U'},
+    {"exclusive", 0, 0, 'x'},
 
     /* Management */
     {"add",       0, 0, 'a'},
diff -urN mdadm-2.3.1/config.c mdadm-2.3.1.exclusive/config.c
--- mdadm-2.3.1/config.c	2005-12-09 06:00:47.000000000 +0100
+++ mdadm-2.3.1.exclusive/config.c	2006-02-13 22:23:02.000000000 +0100
@@ -286,6 +286,7 @@
 	mis.st = NULL;
 	mis.bitmap_fd = -1;
 	mis.name[0] = 0;
+	mis.exclusive = 0;
 
 	for (w=dl_next(line); w!=line; w=dl_next(w)) {
 		if (w[0] == '/') {
@@ -386,6 +387,8 @@
 					fprintf(stderr, Name ": auto type of \"%s\" ignored for %s\n",
 						w+5, mis.devname?mis.devname:"unlabeled-array");
 			}
+		} else if (strncasecmp(w, "exclusive", 9) == 0 ) {
+			mis.exclusive = 1;
 		} else {
 			fprintf(stderr, Name ": unrecognised word on ARRAY line: %s\n",
 				w);
diff -urN mdadm-2.3.1/mdadm.c mdadm-2.3.1.exclusive/mdadm.c
--- mdadm-2.3.1/mdadm.c	2006-02-06 04:58:01.000000000 +0100
+++ mdadm-2.3.1.exclusive/mdadm.c	2006-02-13 22:45:35.000000000 +0100
@@ -72,6 +72,7 @@
 	int quiet = 0;
 	int brief = 0;
 	int force = 0;
+	int exclusive = 0;
 	int test = 0;
 	int assume_clean = 0;
 	int autof = 0; /* -2 means create device based on name:
@@ -808,6 +809,11 @@
 				}
 			}
 			continue;
+
+		case O(ASSEMBLE,'x'):
+			exclusive = 1;
+			continue;
+
 		}
 		/* We have now processed all the valid options. Anything else is
 		 * an error
@@ -928,7 +934,7 @@
 				else {
 					rv |= Assemble(ss, devlist->devname, mdfd, array_ident, configfile,
 						       NULL,
-						       readonly, runstop, update, verbose-quiet, force);
+						       readonly, runstop, update, verbose-quiet, force, exclusive);
 					close(mdfd);
 				}
 			}
@@ -957,7 +963,7 @@
 				}
 				rv |= Assemble(ss, dv->devname, mdfd, array_ident, configfile,
 					       NULL,
-					       readonly, runstop, update, verbose-quiet, force);
+					       readonly, runstop, update, verbose-quiet, force, exclusive);
 				close(mdfd);
 			}
 		} else {
@@ -981,7 +987,7 @@
 						rv |= Assemble(ss, array_list->devname, mdfd,
 							       array_list, configfile,
 							       NULL,
-							       readonly, runstop, NULL, verbose-quiet, force);
+							       readonly, runstop, NULL, verbose-quiet, force, exclusive);
 					close(mdfd);
 				}
 		}
diff -urN mdadm-2.3.1/mdadm.h mdadm-2.3.1.exclusive/mdadm.h
--- mdadm-2.3.1/mdadm.h	2006-02-06 04:52:12.000000000 +0100
+++ mdadm-2.3.1.exclusive/mdadm.h	2006-02-13 22:21:21.000000000 +0100
@@ -142,6 +142,7 @@
 	int	autof;		/* 1 for normal, 2 for partitioned */
 	char	*spare_group;
 	int	bitmap_fd;
+	int	exclusive;
 
 	struct mddev_ident_s *next;
 } *mddev_ident_t;

Re: Question: array locking, possible?

[Luca Berra]

On Mon, Feb 13, 2006 at 10:53:43PM +0100, Luca Berra wrote:

>diff -urN mdadm-2.3.1/Assemble.c mdadm-2.3.1.exclusive/Assemble.c

please note that the patch was written while i was composing the email
as a proof-of-concept, it should not be considered working (or even
compiling code)

L.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: Question: array locking, possible?

[Chris Osicki]

Rick

On HP-UX disk mirroring is done in LVM. I'm using md driver for
mirroring and LVM on top of it.  Controlling access to my disks in LVM
is just too late. I would have to assemble the array before I can activate
VGs. If the array in question is being used on the other host nobody
can guarantee that bad thing wont happen. And what I would like to
prevent is: two hosts accessing (writing) an array.
Thanks anyway for the hint.

Regards,
Chris


On Thu, 9 Feb 2006 10:28:58 -0800
"Stern, Rick (Serviceguard Linux)" <rick.stern@hp.com> wrote:

> There is more interest, just not vocal.
> 
> May want to look at LVM2 and its ability to use tagging to control enablement of VGs. This way it is not HW dependent.
> 
> -----Original Message-----
> From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Chris Osicki
> Sent: Thursday, February 09, 2006 2:26 AM
> To: linux-raid@vger.kernel.org
> Subject: Re: Question: array locking, possible?
> 
> 
> 
> It looks like we are the only two md users interested in such a
> feature.
> Not enough to get Neil's attention ;-)
> 
> Regards,
> Chris
> 
> On Wed, 8 Feb 2006 21:45:33 +0100
> Jure Peèar <pegasus@nerv.eu.org> wrote:
> 
> > On Wed, 8 Feb 2006 11:55:49 +0100
> > Chris Osicki <osk@admin.swisscom-mobile.ch> wrote:
> > 
> > > 
> > > 
> > > I was thinking about it, I have no idea how to do it on Linux if ever possible.
> > > I connect over fibre channel SAN, using QLogic QLA2312 HBAS, if it matters.
> > > 
> > > Anyone any hints?
> > 
> > I too am running a jbod with md raid between two machines. So far md never
> > caused any kind of problems, altough I did have situations where both
> > machines were syncing mirrors at once.
> > 
> > If there's a little tool to reserve a disk via scsi, I'd like to know about
> > it too. Even a piece of code would be enough.
> > 
> > 
> > -- 
> > 
> > Jure Peèar
> > http://jure.pecar.org/
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Question: array locking, possible?

[Stern, Rick (Serviceguard Linux)]

I understand about HP-UX mirroring/LVM.

I was a little too obtuse.

LVM2 has a feature (not well advertised) that allows an VG to be tagged so it will not be activated by "system b" if it is already tagged as being in use by "system a".  I was suggesting that a similar feature could be added to MD.  This way a MD array could be marked as "owned" and, if so, mdadm would not activate it from another system.  This way all of the MD control is still within mdadm.

If Neil is interested, I'll try to dig up more info.

Regards,
Rick  

-----Original Message-----
From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Chris Osicki
Sent: Monday, February 13, 2006 9:13 AM
To: linux-raid@vger.kernel.org
Subject: Re: Question: array locking, possible?



Rick

On HP-UX disk mirroring is done in LVM. I'm using md driver for
mirroring and LVM on top of it.  Controlling access to my disks in LVM
is just too late. I would have to assemble the array before I can activate
VGs. If the array in question is being used on the other host nobody
can guarantee that bad thing wont happen. And what I would like to
prevent is: two hosts accessing (writing) an array.
Thanks anyway for the hint.

Regards,
Chris


On Thu, 9 Feb 2006 10:28:58 -0800
"Stern, Rick (Serviceguard Linux)" <rick.stern@hp.com> wrote:

> There is more interest, just not vocal.
> 
> May want to look at LVM2 and its ability to use tagging to control enablement of VGs. This way it is not HW dependent.
> 
> -----Original Message-----
> From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Chris Osicki
> Sent: Thursday, February 09, 2006 2:26 AM
> To: linux-raid@vger.kernel.org
> Subject: Re: Question: array locking, possible?
> 
> 
> 
> It looks like we are the only two md users interested in such a
> feature.
> Not enough to get Neil's attention ;-)
> 
> Regards,
> Chris
> 
> On Wed, 8 Feb 2006 21:45:33 +0100
> Jure Peèar <pegasus@nerv.eu.org> wrote:
> 
> > On Wed, 8 Feb 2006 11:55:49 +0100
> > Chris Osicki <osk@admin.swisscom-mobile.ch> wrote:
> > 
> > > 
> > > 
> > > I was thinking about it, I have no idea how to do it on Linux if ever possible.
> > > I connect over fibre channel SAN, using QLogic QLA2312 HBAS, if it matters.
> > > 
> > > Anyone any hints?
> > 
> > I too am running a jbod with md raid between two machines. So far md never
> > caused any kind of problems, altough I did have situations where both
> > machines were syncing mirrors at once.
> > 
> > If there's a little tool to reserve a disk via scsi, I'd like to know about
> > it too. Even a piece of code would be enough.
> > 
> > 
> > -- 
> > 
> > Jure Peèar
> > http://jure.pecar.org/
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Question: array locking, possible?

[Chris Osicki]

Rick

You must have missed my first posting, or maybe I was not clear enough.
We _are_ talking about the same thing.

Now we are already three or four thinking of it as a useful feature,
the pression on Neil is dramatically increasing ... ;-)

Regards,
Chris

On Mon, 13 Feb 2006 09:21:06 -0800
"Stern, Rick (Serviceguard Linux)" <rick.stern@hp.com> wrote:

> I understand about HP-UX mirroring/LVM.
> 
> I was a little too obtuse.
> 
> LVM2 has a feature (not well advertised) that allows an VG to be tagged so it will not be activated by "system b" if it is already tagged as being in use by "system a".  I was suggesting that a similar feature could be added to MD.  This way a MD array could be marked as "owned" and, if so, mdadm would not activate it from another system.  This way all of the MD control is still within mdadm.
> 
> If Neil is interested, I'll try to dig up more info.
> 
> Regards,
> Rick  
> 
> -----Original Message-----
> From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Chris Osicki
> Sent: Monday, February 13, 2006 9:13 AM
> To: linux-raid@vger.kernel.org
> Subject: Re: Question: array locking, possible?
> 
> 
> 
> Rick
> 
> On HP-UX disk mirroring is done in LVM. I'm using md driver for
> mirroring and LVM on top of it.  Controlling access to my disks in LVM
> is just too late. I would have to assemble the array before I can activate
> VGs. If the array in question is being used on the other host nobody
> can guarantee that bad thing wont happen. And what I would like to
> prevent is: two hosts accessing (writing) an array.
> Thanks anyway for the hint.
> 
> Regards,
> Chris
> 
> 
> On Thu, 9 Feb 2006 10:28:58 -0800
> "Stern, Rick (Serviceguard Linux)" <rick.stern@hp.com> wrote:
> 
> > There is more interest, just not vocal.
> > 
> > May want to look at LVM2 and its ability to use tagging to control enablement of VGs. This way it is not HW dependent.
> > 
> > -----Original Message-----
> > From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Chris Osicki
> > Sent: Thursday, February 09, 2006 2:26 AM
> > To: linux-raid@vger.kernel.org
> > Subject: Re: Question: array locking, possible?
> > 
> > 
> > 
> > It looks like we are the only two md users interested in such a
> > feature.
> > Not enough to get Neil's attention ;-)
> > 
> > Regards,
> > Chris
> > 
> > On Wed, 8 Feb 2006 21:45:33 +0100
> > Jure Peèar <pegasus@nerv.eu.org> wrote:
> > 
> > > On Wed, 8 Feb 2006 11:55:49 +0100
> > > Chris Osicki <osk@admin.swisscom-mobile.ch> wrote:
> > > 
> > > > 
> > > > 
> > > > I was thinking about it, I have no idea how to do it on Linux if ever possible.
> > > > I connect over fibre channel SAN, using QLogic QLA2312 HBAS, if it matters.
> > > > 
> > > > Anyone any hints?
> > > 
> > > I too am running a jbod with md raid between two machines. So far md never
> > > caused any kind of problems, altough I did have situations where both
> > > machines were syncing mirrors at once.
> > > 
> > > If there's a little tool to reserve a disk via scsi, I'd like to know about
> > > it too. Even a piece of code would be enough.
> > > 
> > > 
> > > -- 
> > > 
> > > Jure Peèar
> > > http://jure.pecar.org/
> > > -
> > > To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > > 
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Question: array locking, possible?

[Chris Osicki]

Hi

Is there any way to "lock" an active array?

To make clear what I'm after:

I have two machines forming a fail-over cluster connected over SAN to
two storage boxes which provide them with a "disk" each.
I created a RAID1 using those two "disks". 

Both cluster nodes see both disks but only one node, the active
one uses them by assembling the array.

During a fail-over I stop the array on the active node and assemble
it on the node becoming active. It works OK, or at least I haven't
seen any problems for quite a long time now.

The problem now is how to prevent somebody on the other host from
accidentally assembling the array. Because the result of doing so would
be something from strange to catastrophic ;-)

To rephrase my question, is there any way to make it visible to the
other host that the array is up an running on the this host?

Any comments, ideas?

Thanks for your time.

Regards,
Chris

Re: Question: array locking, possible?

[Mike Hardy]

Chris Osicki wrote:

> 
> To rephrase my question, is there any way to make it visible to the
> other host that the array is up an running on the this host?
> 
> Any comments, ideas?

Would that not imply an "unlock" command before you could run the array
on the other host?

Would that not then break the automatic fail-over you want, as no
machine that died or hung would issue the unlock command, meaning that
the fail-over node could not then use the disks

It's an interesting idea, I just can't think of a way to make it work
unattended

It might be possible wrap the 'mdadm' binary with a script that "checks"
(maybe via some deep check using ssh to execute remote commands, or just
a ping) the hosts status and just prints a little table of host status
that can only be avoided by passing a special --yes-i-know flag to the
wrapper


-Mike

Re: Question: array locking, possible?

[Chris Osicki]

On Tue, 07 Feb 2006 10:16:20 -0800
Mike Hardy <mhardy@h3c.com> wrote:

> 
> 
> Chris Osicki wrote:
> 
> > 
> > To rephrase my question, is there any way to make it visible to the
> > other host that the array is up an running on the this host?
> > 
> > Any comments, ideas?
> 
> Would that not imply an "unlock" command before you could run the array
> on the other host?

Yes, it would. I was thinking about an "advisory" lock, and a well
known -f option for those who know what they are doing ;-)

> 
> Would that not then break the automatic fail-over you want, as no
> machine that died or hung would issue the unlock command, meaning that
> the fail-over node could not then use the disks

If I trust my cluster software it's not a problem, I use the -f.
My concern is as I said accidentally array activation on the other node.

> 
> It's an interesting idea, I just can't think of a way to make it work
> unattended

> 
> It might be possible wrap the 'mdadm' binary with a script that "checks"
> (maybe via some deep check using ssh to execute remote commands, or just
> a ping) the hosts status and just prints a little table of host status
> that can only be avoided by passing a special --yes-i-know flag to the
> wrapper

It has been done, more or less what you are thinking about. The
cluster I'm currently working on is Service Guard on Linux. The 
original platform is HP-UX. They use LVM for mirroring and device
locking is on LVM level.  The active cluster node activates a volume
group in exclusive mode. This writes a kind of flag onto the
disk. Should the node die without a chance to clear the flag, the node
taking over the service knows what happened and forces the take-over
of the volume group.  This feature is missing on Linux.

I already have a Linux cluster which has been running for over one
year w/o problems.  I've just setup three more and to sleep better I'm
looking for a way to diminish chances of a disaster due to a operation
fault. 

Regards,
Chris

> 
> 
> -Mike
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: Question: array locking, possible?

[Paul Clements]

Chris Osicki wrote:
> The problem now is how to prevent somebody on the other host from
> accidentally assembling the array. Because the result of doing so would
> be something from strange to catastrophic ;-)
> 
> To rephrase my question, is there any way to make it visible to the
> other host that the array is up an running on the this host?

I don't know how the storage boxes are attached to the servers, but you 
might be able to use SCSI reservations, if the storage supports them.

--
Paul

Re: Question: array locking, possible?

[Chris Osicki]

I was thinking about it, I have no idea how to do it on Linux if ever possible.
I connect over fibre channel SAN, using QLogic QLA2312 HBAS, if it matters.

Anyone any hints?

Thanks and regards,
Chris

On Tue, 07 Feb 2006 14:26:13 -0500
Paul Clements <paul.clements@steeleye.com> wrote:

> Chris Osicki wrote:
> > The problem now is how to prevent somebody on the other host from
> > accidentally assembling the array. Because the result of doing so would
> > be something from strange to catastrophic ;-)
> > 
> > To rephrase my question, is there any way to make it visible to the
> > other host that the array is up an running on the this host?
> 
> I don't know how the storage boxes are attached to the servers, but you 
> might be able to use SCSI reservations, if the storage supports them.
> 
> --
> Paul
> 

Re: Question: array locking, possible?

[Jure Pečar]

On Wed, 8 Feb 2006 11:55:49 +0100
Chris Osicki <osk@admin.swisscom-mobile.ch> wrote:

> 
> 
> I was thinking about it, I have no idea how to do it on Linux if ever possible.
> I connect over fibre channel SAN, using QLogic QLA2312 HBAS, if it matters.
> 
> Anyone any hints?

I too am running a jbod with md raid between two machines. So far md never
caused any kind of problems, altough I did have situations where both
machines were syncing mirrors at once.

If there's a little tool to reserve a disk via scsi, I'd like to know about
it too. Even a piece of code would be enough.


-- 

Jure Pečar
http://jure.pecar.org/
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Question: array locking, possible?

[Chris Osicki]

It looks like we are the only two md users interested in such a
feature.
Not enough to get Neil's attention ;-)

Regards,
Chris

On Wed, 8 Feb 2006 21:45:33 +0100
Jure Peèar <pegasus@nerv.eu.org> wrote:

> On Wed, 8 Feb 2006 11:55:49 +0100
> Chris Osicki <osk@admin.swisscom-mobile.ch> wrote:
> 
> > 
> > 
> > I was thinking about it, I have no idea how to do it on Linux if ever possible.
> > I connect over fibre channel SAN, using QLogic QLA2312 HBAS, if it matters.
> > 
> > Anyone any hints?
> 
> I too am running a jbod with md raid between two machines. So far md never
> caused any kind of problems, altough I did have situations where both
> machines were syncing mirrors at once.
> 
> If there's a little tool to reserve a disk via scsi, I'd like to know about
> it too. Even a piece of code would be enough.
> 
> 
> -- 
> 
> Jure Peèar
> http://jure.pecar.org/
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Question: array locking, possible?

[Neil Brown]

On Thursday February 9, Krzysztof.Osicki@swisscom.com wrote:
> 
> 
> It looks like we are the only two md users interested in such a
> feature.
> Not enough to get Neil's attention ;-)

:-)

Just because I haven't said anything doesn't mean I'm not listening.

Cluster awareness is definitely on my radar.  I have no firm plans yet
but I'm certainly interested in  understanding the issues.  So I'm
listening..

NeilBrown

Re: Question: array locking, possible?

[Paul Clements]

Jure Pečar wrote:

> I too am running a jbod with md raid between two machines. So far md never
> caused any kind of problems, altough I did have situations where both
> machines were syncing mirrors at once.
> 
> If there's a little tool to reserve a disk via scsi, I'd like to know about
> it too. Even a piece of code would be enough.

The sg3_utils can do this, if your hardware is compatible.

--
Paul
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html