From: Jens Axboe <axboe@kernel.dk>
To: Philipp Reisner <philipp.reisner@linbit.com>
Cc: lars.ellenberg@linbit.com, linux-kernel@vger.kernel.org,
drbd-dev@lists.linbit.com
Subject: Re: [Drbd-dev] [PATCH 00/18] RFC: Non blocking submit for activity log misses
Date: Fri, 22 Mar 2013 11:17:06 -0600 [thread overview]
Message-ID: <20130322171706.GF1628@kernel.dk> (raw)
In-Reply-To: <1363713419-17803-1-git-send-email-philipp.reisner@linbit.com>
On Tue, Mar 19 2013, Philipp Reisner wrote:
> The Issues
>
> Since the beginning DRBD was written with the assumption that the write
> pattern has spacial locality. (This assumption was driven from the fact,
> that rotating media performs better if you do not send the head too far too
> often)
>
> Backed by this assumption a caller that submits a request that is outside of
> the current active set, was blocked until the active set was changed.
> (Changing the active set is a synchronous write operation to the meta-data
> area on the backing storage = "an AL-update" in DRBD-speak)
>
> A second effect was that DRBD's meta-data was located in a very narrow
> area. When DRBD is used on top of a RAID0 stripe set, this causes all
> AL-updates to got to the same disk.
>
>
> The Proposed Solution
>
> This patch series improves DRBD's behavior. A submitter is no longer blocked
> in the case of a AL-miss. For this a dedicated submitter worker is introduced
> (patch 13).
>
> In order to better distribute the AL-updates to more disks in a stripe set
> this patch series also introduces an optional striped layout of the part
> of the meta-data that holds the AL-updates (patch 4).
>
>
> The Results
>
> This of course drastically improves DRBD's performance if the write pattern
> does not have any spacial locality. E.g. random writes spread out over the
> whole device.
>
> In the test systems we have SSDs with are able to do up to 50000 writes per
> second. The test does random distributed writes over a work set size of
> 128GiB with IO depths from 1 to 1024.
>
> At an IO depth of 64:
> without this patch we observed ~100 IOPs.
> With this patches we observed about 20000 IOPs.
>
> Please find charts of the results here:
> http://blogs.linbit.com/p/469/843-random-writes-faster/
Patchset doesn't apply to current tree. 10/18 gets into problems, last
hunk of drbd_main.c. What is this based on?
--
Jens Axboe
WARNING: multiple messages have this Message-ID (diff)
From: Jens Axboe <axboe@kernel.dk>
To: Philipp Reisner <philipp.reisner@linbit.com>
Cc: linux-kernel@vger.kernel.org, drbd-dev@lists.linbit.com,
lars.ellenberg@linbit.com
Subject: Re: [PATCH 00/18] RFC: Non blocking submit for activity log misses
Date: Fri, 22 Mar 2013 11:17:06 -0600 [thread overview]
Message-ID: <20130322171706.GF1628@kernel.dk> (raw)
In-Reply-To: <1363713419-17803-1-git-send-email-philipp.reisner@linbit.com>
On Tue, Mar 19 2013, Philipp Reisner wrote:
> The Issues
>
> Since the beginning DRBD was written with the assumption that the write
> pattern has spacial locality. (This assumption was driven from the fact,
> that rotating media performs better if you do not send the head too far too
> often)
>
> Backed by this assumption a caller that submits a request that is outside of
> the current active set, was blocked until the active set was changed.
> (Changing the active set is a synchronous write operation to the meta-data
> area on the backing storage = "an AL-update" in DRBD-speak)
>
> A second effect was that DRBD's meta-data was located in a very narrow
> area. When DRBD is used on top of a RAID0 stripe set, this causes all
> AL-updates to got to the same disk.
>
>
> The Proposed Solution
>
> This patch series improves DRBD's behavior. A submitter is no longer blocked
> in the case of a AL-miss. For this a dedicated submitter worker is introduced
> (patch 13).
>
> In order to better distribute the AL-updates to more disks in a stripe set
> this patch series also introduces an optional striped layout of the part
> of the meta-data that holds the AL-updates (patch 4).
>
>
> The Results
>
> This of course drastically improves DRBD's performance if the write pattern
> does not have any spacial locality. E.g. random writes spread out over the
> whole device.
>
> In the test systems we have SSDs with are able to do up to 50000 writes per
> second. The test does random distributed writes over a work set size of
> 128GiB with IO depths from 1 to 1024.
>
> At an IO depth of 64:
> without this patch we observed ~100 IOPs.
> With this patches we observed about 20000 IOPs.
>
> Please find charts of the results here:
> http://blogs.linbit.com/p/469/843-random-writes-faster/
Patchset doesn't apply to current tree. 10/18 gets into problems, last
hunk of drbd_main.c. What is this based on?
--
Jens Axboe
next prev parent reply other threads:[~2013-03-22 17:33 UTC|newest]
Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-03-19 17:16 [Drbd-dev] [PATCH 00/18] RFC: Non blocking submit for activity log misses Philipp Reisner
2013-03-19 17:16 ` Philipp Reisner
2013-03-19 17:16 ` [Drbd-dev] [PATCH 01/18] drbd: cleanup bogus assert message Philipp Reisner
2013-03-19 17:16 ` Philipp Reisner
2013-03-19 17:16 ` [Drbd-dev] [PATCH 02/18] drbd: cleanup ondisk meta data layout calculations and defines Philipp Reisner
2013-03-19 17:16 ` Philipp Reisner
2013-03-19 17:16 ` [Drbd-dev] [PATCH 03/18] drbd: prepare for new striped layout of activity log Philipp Reisner
2013-03-19 17:16 ` Philipp Reisner
2013-03-19 17:16 ` [Drbd-dev] [PATCH 04/18] drbd: use the cached meta_dev_idx Philipp Reisner
2013-03-19 17:16 ` Philipp Reisner
2013-03-19 17:16 ` [Drbd-dev] [PATCH 05/18] drbd: mechanically rename la_size to la_size_sect Philipp Reisner
2013-03-19 17:16 ` Philipp Reisner
2013-03-19 17:16 ` [Drbd-dev] [PATCH 06/18] drbd: read meta data early, base on-disk offsets on super block Philipp Reisner
2013-03-19 17:16 ` Philipp Reisner
2013-03-19 17:16 ` [Drbd-dev] [PATCH 07/18] drbd: Clarify when activity log I/O is delegated to the worker thread Philipp Reisner
2013-03-19 17:16 ` Philipp Reisner
2013-03-19 17:16 ` [Drbd-dev] [PATCH 08/18] drbd: drbd_al_being_io: short circuit to reduce latency Philipp Reisner
2013-03-19 17:16 ` Philipp Reisner
2013-03-19 17:16 ` [Drbd-dev] [PATCH 09/18] drbd: split __drbd_make_request in before and after drbd_al_begin_io Philipp Reisner
2013-03-19 17:16 ` Philipp Reisner
2013-03-19 17:16 ` [Drbd-dev] [PATCH 10/18] drbd: prepare to queue write requests on a submit worker Philipp Reisner
2013-03-19 17:16 ` Philipp Reisner
2013-03-22 20:22 ` [Drbd-dev] [PATCH 10/18, update] " lars.ellenberg
2013-03-22 20:22 ` lars.ellenberg
2013-03-23 0:15 ` Jens Axboe
2013-03-23 0:15 ` Jens Axboe
2013-03-19 17:16 ` [Drbd-dev] [PATCH 11/18] drbd: split drbd_al_begin_io into fastpath, prepare, and commit Philipp Reisner
2013-03-19 17:16 ` Philipp Reisner
2013-03-19 17:16 ` [Drbd-dev] [PATCH 12/18] drbd: split out some helper functions to drbd_al_begin_io Philipp Reisner
2013-03-19 17:16 ` Philipp Reisner
2013-03-19 17:16 ` [Drbd-dev] [PATCH 13/18] drbd: queue writes on submitter thread, unless they pass the activity log fastpath Philipp Reisner
2013-03-19 17:16 ` Philipp Reisner
2013-03-19 17:16 ` [Drbd-dev] [PATCH 14/18] lru_cache: introduce lc_get_cumulative() Philipp Reisner
2013-03-19 17:16 ` Philipp Reisner
2013-03-19 17:16 ` [Drbd-dev] [PATCH 15/18] drbd: consolidate as many updates as possible into one AL transaction Philipp Reisner
2013-03-19 17:16 ` Philipp Reisner
2013-03-19 17:16 ` [Drbd-dev] [PATCH 16/18] drbd: move start io accounting before activity log transaction Philipp Reisner
2013-03-19 17:16 ` Philipp Reisner
2013-03-19 17:16 ` [Drbd-dev] [PATCH 17/18] drbd: try hard to max out the updates per AL transaction Philipp Reisner
2013-03-19 17:16 ` Philipp Reisner
2013-03-19 17:16 ` [Drbd-dev] [PATCH 18/18] drbd: adjust upper limit for activity log extents Philipp Reisner
2013-03-19 17:16 ` Philipp Reisner
2013-03-22 17:17 ` Jens Axboe [this message]
2013-03-22 17:17 ` [PATCH 00/18] RFC: Non blocking submit for activity log misses Jens Axboe
2013-03-22 20:03 ` [Drbd-dev] " lars.ellenberg
2013-03-22 20:03 ` lars.ellenberg
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130322171706.GF1628@kernel.dk \
--to=axboe@kernel.dk \
--cc=drbd-dev@lists.linbit.com \
--cc=lars.ellenberg@linbit.com \
--cc=linux-kernel@vger.kernel.org \
--cc=philipp.reisner@linbit.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.