mirroring: [patch 1 of 8] device failure tolerance

mirroring: [patch 1 of 8] device failure tolerance

[Jonathan E Brassow]

This patch defines a couple more states that logs can return, and 
checks for those states in the mirror code.  The states are useful for 
logs that have cluster support.

  brassow

diff -urN linux-2.6.12-orig/drivers/md/dm-log.c 
linux-2.6.12-001/drivers/md/dm-log.c
--- linux-2.6.12-orig/drivers/md/dm-log.c	2005-06-17 14:48:29.000000000 
-0500
+++ linux-2.6.12-001/drivers/md/dm-log.c	2005-06-28 14:38:40.349390239 
-0500
@@ -517,13 +517,13 @@
  static int core_is_clean(struct dirty_log *log, region_t region)
  {
  	struct log_c *lc = (struct log_c *) log->context;
-	return log_test_bit(lc->clean_bits, region);
+	return (log_test_bit(lc->clean_bits, region)) ? LOG_CLEAN : LOG_DIRTY;
  }

  static int core_in_sync(struct dirty_log *log, region_t region, int 
block)
  {
  	struct log_c *lc = (struct log_c *) log->context;
-	return log_test_bit(lc->sync_bits, region);
+	return (log_test_bit(lc->sync_bits, region)) ? LOG_CLEAN : LOG_NOSYNC;
  }

  static int core_flush(struct dirty_log *log)
diff -urN linux-2.6.12-orig/drivers/md/dm-log.h 
linux-2.6.12-001/drivers/md/dm-log.h
--- linux-2.6.12-orig/drivers/md/dm-log.h	2005-06-17 14:48:29.000000000 
-0500
+++ linux-2.6.12-001/drivers/md/dm-log.h	2005-06-28 14:23:54.655103852 
-0500
@@ -9,6 +9,12 @@

  #include "dm.h"

+#define LOG_DIRTY              0
+#define LOG_CLEAN              1
+#define LOG_NOSYNC             2
+#define LOG_RECOVERING         3
+#define LOG_REMOTE_RECOVERING  4
+
  typedef sector_t region_t;

  struct dirty_log_type;
diff -urN linux-2.6.12-orig/drivers/md/dm-raid1.c 
linux-2.6.12-001/drivers/md/dm-raid1.c
--- linux-2.6.12-orig/drivers/md/dm-raid1.c	2005-06-17 
14:48:29.000000000 -0500
+++ linux-2.6.12-001/drivers/md/dm-raid1.c	2005-06-28 
14:49:38.985786409 -0500
@@ -91,7 +91,8 @@
  	RH_CLEAN,
  	RH_DIRTY,
  	RH_NOSYNC,
-	RH_RECOVERING
+	RH_RECOVERING,
+	RH_REMOTE_RECOVERING
  };

  struct region {
@@ -234,7 +235,7 @@

  	read_unlock(&rh->hash_lock);
  	nreg = mempool_alloc(rh->region_pool, GFP_NOIO);
-	nreg->state = rh->log->type->in_sync(rh->log, region, 1) ?
+	nreg->state = (rh->log->type->in_sync(rh->log, region, 1) == 
LOG_CLEAN) ?
  		RH_CLEAN : RH_NOSYNC;
  	nreg->rh = rh;
  	nreg->key = region;
@@ -292,13 +293,26 @@
  	 * The region wasn't in the hash, so we fall back to the
  	 * dirty log.
  	 */
-	r = rh->log->type->in_sync(rh->log, region, may_block);
-
+	switch (rh->log->type->in_sync(rh->log, region, may_block)) {
+	case LOG_CLEAN:
+		r = RH_CLEAN;
+		break;
+	case LOG_DIRTY:
+		r = RH_DIRTY;
+		break;
+	case LOG_REMOTE_RECOVERING:
+		r = RH_REMOTE_RECOVERING;
+		break;
+	default:
+		r = RH_NOSYNC;
+		break;
+	}
+	
  	/*
  	 * Any error from the dirty log (eg. -EWOULDBLOCK) gets
  	 * taken as a RH_NOSYNC
  	 */
-	return r == 1 ? RH_CLEAN : RH_NOSYNC;
+	return r;
  }

  static inline int rh_in_sync(struct region_hash *rh,
@@ -702,7 +716,7 @@
  		/*
  		 * We can only read balance if the region is in sync.
  		 */
-		if (rh_in_sync(&ms->rh, region, 0))
+		if (rh_in_sync(&ms->rh, region, 0) == RH_CLEAN)
  			m = choose_mirror(ms, bio->bi_sector);
  		else
  			m = ms->mirror + DEFAULT_MIRROR;
@@ -1117,7 +1131,7 @@
  	if (r < 0 && r != -EWOULDBLOCK)
  		return r;

-	if (r == -EWOULDBLOCK)	/* FIXME: ugly */
+	if (r != RH_CLEAN)
  		r = 0;

  	/*

Re: mirroring: [patch 1 of 8] device failure tolerance

[Lars Marowsky-Bree]

On 2005-06-29T13:07:49, Jonathan E Brassow <jbrassow@redhat.com> wrote:

> This patch defines a couple more states that logs can return, and 
> checks for those states in the mirror code.  The states are useful for 
> logs that have cluster support.

OK. Let me just ask one question, which I'll preface with the following:

I totally love the idea of md / DM convergence. I'm not yet quite clear
how this is to be achieved, but having two separate frameworks which at
least considerable overlap doesn't make sense - this isn't like two
filesystems, this is like two VFS layers and obviously silly. So, I
appreciate movement into that direction.

However, what's going on now is that features are being added to the DM
raid code, instead of figuring out a path of convergence.

RAID code takes, that's the experience we've got with Linux so far,
quite some time to stabilize. Duplicating it definetely doesn't seem
smart, if I may be so blunt. The RAID code in md is, nowadays, very
stable, and even already has patches for faster resync etc (merged in
-mm).

md is, in certain ways, more mature, which is exemplified by md's
ability to actually provide sane logging. DM has new cool features (like
the online remapping and stuff).

Why is the approach of reinventing a wheel with a slightly different
number of edges taken?

Is it believed that DM will supersede all aspects of md one day? Ain't
that slightly silly? ;-)


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
High Availability & Clustering
SUSE Labs, Research and Development
SUSE LINUX Products GmbH - A Novell Business	 -- Charles Darwin
"Ignorance more frequently begets confidence than does knowledge"

Re: [dm-devel] mirroring: [patch 1 of 8] device failure tolerance

[Lars Marowsky-Bree]

On 2005-06-29T13:07:49, Jonathan E Brassow <jbrassow@redhat.com> wrote:

> This patch defines a couple more states that logs can return, and 
> checks for those states in the mirror code.  The states are useful for 
> logs that have cluster support.

OK. Let me just ask one question, which I'll preface with the following:

I totally love the idea of md / DM convergence. I'm not yet quite clear
how this is to be achieved, but having two separate frameworks which at
least considerable overlap doesn't make sense - this isn't like two
filesystems, this is like two VFS layers and obviously silly. So, I
appreciate movement into that direction.

However, what's going on now is that features are being added to the DM
raid code, instead of figuring out a path of convergence.

RAID code takes, that's the experience we've got with Linux so far,
quite some time to stabilize. Duplicating it definetely doesn't seem
smart, if I may be so blunt. The RAID code in md is, nowadays, very
stable, and even already has patches for faster resync etc (merged in
-mm).

md is, in certain ways, more mature, which is exemplified by md's
ability to actually provide sane logging. DM has new cool features (like
the online remapping and stuff).

Why is the approach of reinventing a wheel with a slightly different
number of edges taken?

Is it believed that DM will supersede all aspects of md one day? Ain't
that slightly silly? ;-)


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
High Availability & Clustering
SUSE Labs, Research and Development
SUSE LINUX Products GmbH - A Novell Business	 -- Charles Darwin
"Ignorance more frequently begets confidence than does knowledge"

Re: mirroring: [patch 1 of 8] device failure tolerance

[Greg Freemyer]

On 6/29/05, Jonathan E Brassow <jbrassow@redhat.com> wrote:
> This patch defines a couple more states that logs can return, and
> checks for those states in the mirror code.  The states are useful for
> logs that have cluster support.
> 
>   brassow

Have I been asleep?  This is the first I heard of DM having cluster
mirror support.  I assume the goal is something along the lines of
what DRBD provides.

Can someone give a very high-level overview of current status, plans,
etc. related to DM Cluster Mirror support.  And if this related to the
DRBD project?

Thanks
Greg
-- 
Greg Freemyer
The Norcross Group
Forensics for the 21st Century

Re: mirroring: [patch 1 of 8] device failure tolerance

[Jonathan E Brassow]

On Jun 30, 2005, at 10:13 AM, Greg Freemyer wrote:

> On 6/29/05, Jonathan E Brassow <jbrassow@redhat.com> wrote:
>> This patch defines a couple more states that logs can return, and
>> checks for those states in the mirror code.  The states are useful for
>> logs that have cluster support.
>>
>>   brassow
>
> Have I been asleep?  This is the first I heard of DM having cluster
> mirror support.  I assume the goal is something along the lines of
> what DRBD provides.
>
> Can someone give a very high-level overview of current status, plans,
> etc. related to DM Cluster Mirror support.  And if this related to the
> DRBD project?

Please note that the "mirroring [patch x of 8]..." set for device fault 
tolerance has been replaced with the "mirroring [patch x of 6]..." set.

The cluster mirroring that I'm talking about is active/active and 
capable of handling more than 2 nodes.  It would be used in conjunction 
with a clustered file system - like GFS.  The application sitting on 
the cluster mirror should be cluster-aware.  You wouldn't be able to 
just put ext3 on there and expect it to work in a cluster capacity.

Although there are some minor things that need to happen in the mirror 
proper code (as can be seen from the smallness of the cluster patch), 
the heavy lifting is done by a cluster-aware log.  That is were you are 
keeping track of clean/dirty/recovering state.

An out-dated (but still useful for background) dock can be found at 
http://www.brassow.com/mirroring/index.html

  brassow

Re: mirroring: [patch 1 of 8] device failure tolerance

[Greg Freemyer]

On 6/30/05, Jonathan E Brassow <jbrassow@redhat.com> wrote:
> 
> On Jun 30, 2005, at 10:13 AM, Greg Freemyer wrote:
> 
> > On 6/29/05, Jonathan E Brassow <jbrassow@redhat.com> wrote:
> >> This patch defines a couple more states that logs can return, and
> >> checks for those states in the mirror code.  The states are useful for
> >> logs that have cluster support.
> >>
> >>   brassow
> >
> > Have I been asleep?  This is the first I heard of DM having cluster
> > mirror support.  I assume the goal is something along the lines of
> > what DRBD provides.
> >
> > Can someone give a very high-level overview of current status, plans,
> > etc. related to DM Cluster Mirror support.  And if this related to the
> > DRBD project?
> 
> Please note that the "mirroring [patch x of 8]..." set for device fault
> tolerance has been replaced with the "mirroring [patch x of 6]..." set.
> 
> The cluster mirroring that I'm talking about is active/active and
> capable of handling more than 2 nodes.  It would be used in conjunction
> with a clustered file system - like GFS.  The application sitting on
> the cluster mirror should be cluster-aware.  You wouldn't be able to
> just put ext3 on there and expect it to work in a cluster capacity.
> 
> Although there are some minor things that need to happen in the mirror
> proper code (as can be seen from the smallness of the cluster patch),
> the heavy lifting is done by a cluster-aware log.  That is were you are
> keeping track of clean/dirty/recovering state.
> 
> An out-dated (but still useful for background) dock can be found at
> http://www.brassow.com/mirroring/index.html
> 
>   brassow
> 

Sounds like you have more ambitious plans than DRBD.   (They have
active-passive now and IIRC they plan the next major release to have
active-active, but I think only on 2 nodes.)

Is the GFS team planning on supporting this infrastructure.  I hope so.

Greg
-- 
Greg Freemyer
The Norcross Group
Forensics for the 21st Century

Re: mirroring: [patch 1 of 8] device failure tolerance

[Jonathan E Brassow]

On Jun 30, 2005, at 11:38 AM, Greg Freemyer wrote:

> On 6/30/05, Jonathan E Brassow <jbrassow@redhat.com> wrote:
>>
>> On Jun 30, 2005, at 10:13 AM, Greg Freemyer wrote:
>>
>>> On 6/29/05, Jonathan E Brassow <jbrassow@redhat.com> wrote:
>>>> This patch defines a couple more states that logs can return, and
>>>> checks for those states in the mirror code.  The states are useful 
>>>> for
>>>> logs that have cluster support.
>>>>
>>>>   brassow
>>>
>>> Have I been asleep?  This is the first I heard of DM having cluster
>>> mirror support.  I assume the goal is something along the lines of
>>> what DRBD provides.
>>>
>>> Can someone give a very high-level overview of current status, plans,
>>> etc. related to DM Cluster Mirror support.  And if this related to 
>>> the
>>> DRBD project?
>>
>> Please note that the "mirroring [patch x of 8]..." set for device 
>> fault
>> tolerance has been replaced with the "mirroring [patch x of 6]..." 
>> set.
>>
>> The cluster mirroring that I'm talking about is active/active and
>> capable of handling more than 2 nodes.  It would be used in 
>> conjunction
>> with a clustered file system - like GFS.  The application sitting on
>> the cluster mirror should be cluster-aware.  You wouldn't be able to
>> just put ext3 on there and expect it to work in a cluster capacity.
>>
>> Although there are some minor things that need to happen in the mirror
>> proper code (as can be seen from the smallness of the cluster patch),
>> the heavy lifting is done by a cluster-aware log.  That is were you 
>> are
>> keeping track of clean/dirty/recovering state.
>>
>> An out-dated (but still useful for background) dock can be found at
>> http://www.brassow.com/mirroring/index.html
>>
>>   brassow
>>
>
> Sounds like you have more ambitious plans than DRBD.   (They have
> active-passive now and IIRC they plan the next major release to have
> active-active, but I think only on 2 nodes.)
>
> Is the GFS team planning on supporting this infrastructure.  I hope so.
>

Yes.  In fact, the cluster log being developed leverages the Red hat 
cluster manager (cman) - the same one that GFS uses.

CMAN is moving to user space (I think in the FC5 timeframe?), so the 
cluster log will have to adapt to that.  I won't be submitting the 
cluster log code until that takes place.  It is available in the 
cluster repository though (sources.redhat.com/cluster), but i will have 
to clean it up again to take advantage of the hooks I added to the 
logging code.

  brassow