From: Mike Snitzer <snitzer@redhat.com>
To: Derick Swanepoel <dswanepoel@gmail.com>
Cc: dm-devel@redhat.com
Subject: Re: 40% performance loss with multipath since kernel 3.12
Date: Thu, 25 Jun 2015 12:15:48 -0400 [thread overview]
Message-ID: <20150625161548.GA8924@redhat.com> (raw)
In-Reply-To: <81E477AB-B572-4EEA-97E8-79E370B6D23A@gmail.com>
On Thu, Jun 25 2015 at 12:07pm -0400,
Derick Swanepoel <dswanepoel@gmail.com> wrote:
> Hi
>
> We have an iSCSI-based SAN that uses dm-multipath to aggregate two 10 Gb links in multibus mode. With kernel 3.11.6 we got very good performance, achieving up to 1550 MB/s read on the multipath device.
>
> After upgrading to kernel 4.0.5, performance has dropped by 40%. Read speed is now limited to 960 MB/s, which is slower than the 975 MB/s that the individual paths can achieve.
>
> We did a lot of testing on 3.11.6 to find an optimal rr_min_io_rq value and settled on 9 using the round-robin path selector. With 4.0.5, we’ve tried all the path selectors with rr_min_io_rq from 1 to 100, but can’t seem to beat the performance of a single path any more. For interest’s sake, the default rr_min_io_rq of 1 was actually one of the worst performing values.
>
> We also tried kernel 3.12 and performance was also degraded but not quite as bad as 4.0.5. I am wondering if there was some significant change since 3.11 (dm-multipath 1.5.1) and 3.12 (dm-multipath 1.6.0).
Nothing stands out to me:
# git log --oneline --tags --source drivers/md/dm.c drivers/md/dm-mpath.c | grep 3.12- | tac
670368a dm-3.12-changes dm: stop using WQ_NON_REENTRANT
7e782af v3.12-rc1 [SCSI] Return ENODATA on medium error
00c4fc3 dm-3.12-changes dm ioctl: increase granularity of type_lock when loading table
fd2ed4d dm-3.12-changes dm: add statistics support
7426d62 v3.12-rc1 Merge tag 'dm-3.12-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm
cc9d3c3 dm-3.12-fixes dm mpath: do not fail path on -ENOSPC
f84cb8a dm-3.12-fixes dm mpath: disable WRITE SAME if it fails
6cfa585 dm-3.12-fixes dm: lower bio-based mempool reservation
f479082 dm-3.12-fixes dm: add reserved_rq_based_ios module parameter
e860313 dm-3.12-fixes dm: add reserved_bio_based_ios module parameter
--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
prev parent reply other threads:[~2015-06-25 16:15 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-06-25 16:07 40% performance loss with multipath since kernel 3.12 Derick Swanepoel
2015-06-25 16:15 ` Mike Snitzer [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150625161548.GA8924@redhat.com \
--to=snitzer@redhat.com \
--cc=dm-devel@redhat.com \
--cc=dswanepoel@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.