linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jens Axboe <axboe@kernel.dk>
To: Philipp Reisner <philipp.reisner@linbit.com>
Cc: linux-kernel@vger.kernel.org, drbd-dev@lists.linbit.com,
	lars.ellenberg@linbit.com
Subject: Re: [PATCH 00/18] RFC: Non blocking submit for activity log misses
Date: Fri, 22 Mar 2013 11:17:06 -0600	[thread overview]
Message-ID: <20130322171706.GF1628@kernel.dk> (raw)
In-Reply-To: <1363713419-17803-1-git-send-email-philipp.reisner@linbit.com>

On Tue, Mar 19 2013, Philipp Reisner wrote:
> The Issues
> 
> Since the beginning DRBD was written with the assumption that the write
> pattern has spacial locality. (This assumption was driven from the fact,
> that rotating media performs better if you do not send the head too far too
> often)
> 
> Backed by this assumption a caller that submits a request that is outside of
> the current active set, was blocked until the active set was changed.
> (Changing the active set is a synchronous write operation to the meta-data
> area on the backing storage = "an AL-update" in DRBD-speak)
> 
> A second effect was that DRBD's meta-data was located in a very narrow
> area. When DRBD is used on top of a RAID0 stripe set, this causes all
> AL-updates to got to the same disk.
> 
> 
> The Proposed Solution
> 
> This patch series improves DRBD's behavior. A submitter is no longer blocked
> in the case of a AL-miss. For this a dedicated submitter worker is introduced
> (patch 13).
> 
> In order to better distribute the AL-updates to more disks in a stripe set
> this patch series also introduces an optional striped layout of the part
> of the meta-data that holds the AL-updates (patch 4).
> 
> 
> The Results
> 
> This of course drastically improves DRBD's performance if the write pattern
> does not have any spacial locality. E.g. random writes spread out over the
> whole device.
> 
> In the test systems we have SSDs with are able to do up to 50000 writes per
> second. The test does random distributed writes over a work set size of
> 128GiB with IO depths from 1 to 1024.
> 
> At an IO depth of 64:
> without this patch we observed ~100 IOPs.
> With this patches we observed about 20000 IOPs.
> 
> Please find charts of the results here:
> http://blogs.linbit.com/p/469/843-random-writes-faster/

Patchset doesn't apply to current tree. 10/18 gets into problems, last
hunk of drbd_main.c. What is this based on?

-- 
Jens Axboe


  parent reply	other threads:[~2013-03-22 17:17 UTC|newest]

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-03-19 17:16 [PATCH 00/18] RFC: Non blocking submit for activity log misses Philipp Reisner
2013-03-19 17:16 ` [PATCH 01/18] drbd: cleanup bogus assert message Philipp Reisner
2013-03-19 17:16 ` [PATCH 02/18] drbd: cleanup ondisk meta data layout calculations and defines Philipp Reisner
2013-03-19 17:16 ` [PATCH 03/18] drbd: prepare for new striped layout of activity log Philipp Reisner
2013-03-19 17:16 ` [PATCH 04/18] drbd: use the cached meta_dev_idx Philipp Reisner
2013-03-19 17:16 ` [PATCH 05/18] drbd: mechanically rename la_size to la_size_sect Philipp Reisner
2013-03-19 17:16 ` [PATCH 06/18] drbd: read meta data early, base on-disk offsets on super block Philipp Reisner
2013-03-19 17:16 ` [PATCH 07/18] drbd: Clarify when activity log I/O is delegated to the worker thread Philipp Reisner
2013-03-19 17:16 ` [PATCH 08/18] drbd: drbd_al_being_io: short circuit to reduce latency Philipp Reisner
2013-03-19 17:16 ` [PATCH 09/18] drbd: split __drbd_make_request in before and after drbd_al_begin_io Philipp Reisner
2013-03-19 17:16 ` [PATCH 10/18] drbd: prepare to queue write requests on a submit worker Philipp Reisner
2013-03-22 20:22   ` [Drbd-dev] [PATCH 10/18, update] " lars.ellenberg
2013-03-23  0:15     ` Jens Axboe
2013-03-19 17:16 ` [PATCH 11/18] drbd: split drbd_al_begin_io into fastpath, prepare, and commit Philipp Reisner
2013-03-19 17:16 ` [PATCH 12/18] drbd: split out some helper functions to drbd_al_begin_io Philipp Reisner
2013-03-19 17:16 ` [PATCH 13/18] drbd: queue writes on submitter thread, unless they pass the activity log fastpath Philipp Reisner
2013-03-19 17:16 ` [PATCH 14/18] lru_cache: introduce lc_get_cumulative() Philipp Reisner
2013-03-19 17:16 ` [PATCH 15/18] drbd: consolidate as many updates as possible into one AL transaction Philipp Reisner
2013-03-19 17:16 ` [PATCH 16/18] drbd: move start io accounting before activity log transaction Philipp Reisner
2013-03-19 17:16 ` [PATCH 17/18] drbd: try hard to max out the updates per AL transaction Philipp Reisner
2013-03-19 17:16 ` [PATCH 18/18] drbd: adjust upper limit for activity log extents Philipp Reisner
2013-03-22 17:17 ` Jens Axboe [this message]
2013-03-22 20:03   ` [Drbd-dev] [PATCH 00/18] RFC: Non blocking submit for activity log misses lars.ellenberg

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20130322171706.GF1628@kernel.dk \
    --to=axboe@kernel.dk \
    --cc=drbd-dev@lists.linbit.com \
    --cc=lars.ellenberg@linbit.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=philipp.reisner@linbit.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).