[Drbd-dev] [PATCH] Supporting on-disk cache in DRBD, part 2

[Drbd-dev] [PATCH] Supporting on-disk cache in DRBD, part 2

[Graham, Simon]

[-- Attachment #1.1: Type: text/plain, Size: 1125 bytes --]

Attached is a proposed patch (against 8.0.6) for the second part of the
work to support on-disk caches in DRBD - I took Phil's excellent advice
and used the existing transaction log - the change was simply to
maintain this for Protocol C as well as A and B, everything else was
already in place so that whenever we lose contact with the secondary,
the existing TL log is added to the bitmap (covering everything sent
since the last time a barrier was executed on the secondary).

 

I was a tad concerned about the perf impact of this, so I ran some
simple non-scientific benchmarks to see - what I actually did was run 9
copies of bonnie (doing synchronous writes) against 9 different DRBD
devices that were spread between two physical disks per system and
averaged out the results for all the bonnie runs to get an idea of what
one copy can expect to achieve (possibly not the best calculation, but
it was the easiest to do). The results were actually encouraging -
98-100% of the throughput with no change in CPU usage (although I'm not
entirely convinced bonnie is really a good test).

 

Simon


[-- Attachment #1.2: Type: text/html, Size: 2914 bytes --]

[-- Attachment #2: drbd-tl.patch --]
[-- Type: application/octet-stream, Size: 1798 bytes --]

Index: drbd-8.0.6/drbd/drbd_main.c
===================================================================
--- drbd-8.0.6/drbd/drbd_main.c	(revision 21335)
+++ drbd-8.0.6/drbd/drbd_main.c	(working copy)
@@ -228,10 +228,7 @@
 	b = mdev->oldest_barrier;
 	mdev->oldest_barrier = b->next;
 
-	/* in protocol C this list should be empty,
-	 * unless there is local io pending.
-	 * in protocol A and B, this should not be empty, even though the
-	 * master_bio's could already been completed.  */
+	/* Clean up list of requests processed during current epoch */
 	list_for_each_safe(le, tle, &b->requests) {
 		r = list_entry(le, struct drbd_request,tl_requests);
 		_req_mod(r, barrier_acked, 0);
Index: drbd-8.0.6/drbd/drbd_req.c
===================================================================
--- drbd-8.0.6/drbd/drbd_req.c	(revision 21335)
+++ drbd-8.0.6/drbd/drbd_req.c	(working copy)
@@ -661,10 +661,14 @@
 		req->rq_state |= RQ_NET_SIS;
 	case conflict_discarded_by_peer:
 		/* interesstingly, this is the same thing! */
+		req->rq_state |= RQ_NET_DONE;
+		/* fall through */
 	case write_acked_by_peer:
 		/* assert something? */
 		/* protocol C; successfully written on peer */
-		req->rq_state |= RQ_NET_DONE;
+		/* Nothing to do here - used to set NET_DONE but no more */ 
+		/* as we want to keep the tl in place for all protocols */
+
 		/* rest is the same as for: */
 	case recv_acked_by_peer:
 		/* protocol B; pretends to be sucessfully written on peer.
@@ -688,9 +692,6 @@
 		break;
 
 	case barrier_acked:
-		/* can even happen for protocol C,
-		 * when local io is still pending.
-		 * in which case it does nothing. */
 		if (req->rq_state & RQ_NET_PENDING) {
 			/* barrier came in before all requests have been acked.
 			 * this is bad, because if the connection is lost now,

Re: [Drbd-dev] [PATCH] Supporting on-disk cache in DRBD, part 2

[Lars Ellenberg]

On Wed, Nov 28, 2007 at 10:14:21PM -0500, Graham, Simon wrote:
> Attached is a proposed patch (against 8.0.6) for the second part of the work to
> support on-disk caches in DRBD – I took Phil’s excellent advice and used the
> existing transaction log – the change was simply to maintain this for Protocol
> C as well as A and B, everything else was already in place so that whenever we
> lose contact with the secondary, the existing TL log is added to the bitmap
> (covering everything sent since the last time a barrier was executed on the
> secondary).
> 
>  
> 
> I was a tad concerned about the perf impact of this, so I ran some simple
> non-scientific benchmarks to see – what I actually did was run 9 copies of
> bonnie (doing synchronous writes) against 9 different DRBD devices that were
> spread between two physical disks per system and averaged out the results for
> all the bonnie runs to get an idea of what one copy can expect to achieve
> (possibly not the best calculation, but it was the easiest to do). The results
> were actually encouraging – 98-100% of the throughput with no change in CPU
> usage (although I’m not entirely convinced bonnie is really a good test).
> 

thanks, looks basically like I'd have done it.
this is also the reason we I consolidated all those request state
handling into the one central function.

and this part is probably harmless and potentially beneficial to do
anyways, regardless of whether barriers are supported or not.

-- 
: Lars Ellenberg                            Tel +43-1-8178292-55 :
: LINBIT Information Technologies GmbH      Fax +43-1-8178292-82 :
: Vivenotgasse 48, A-1120 Vienna/Europe    http://www.linbit.com :