From: Josh Durgin <josh.durgin@inktank.com>
To: Somnath Roy <Somnath.Roy@sandisk.com>,
"ceph-devel@vger.kernel.org" <ceph-devel@vger.kernel.org>
Subject: Re: severe librbd performance degradation in Giant
Date: Wed, 17 Sep 2014 14:20:27 -0700 [thread overview]
Message-ID: <5419FB1B.4020202@inktank.com> (raw)
In-Reply-To: <755F6B91B3BE364F9BCA11EA3F9E0C6F27845984@SACMBXIP02.sdcorp.global.sandisk.com>
On 09/17/2014 01:55 PM, Somnath Roy wrote:
> Hi Sage,
> We are experiencing severe librbd performance degradation in Giant over firefly release. Here is the experiment we did to isolate it as a librbd problem.
>
> 1. Single OSD is running latest Giant and client is running fio rbd on top of firefly based librbd/librados. For one client it is giving ~11-12K iops (4K RR).
> 2. Single OSD is running Giant and client is running fio rbd on top of Giant based librbd/librados. For one client it is giving ~1.9K iops (4K RR).
> 3. Single OSD is running latest Giant and client is running Giant based ceph_smaiobench on top of giant librados. For one client it is giving ~11-12K iops (4K RR).
> 4. Giant RGW on top of Giant OSD is also scaling.
>
>
> So, it is obvious from the above that recent librbd has issues. I will raise a tracker to track this.
For giant the default cache settings changed to:
rbd cache = true
rbd cache writethrough until flush = true
If fio isn't sending flushes as the test is running, the cache will
stay in writethrough mode. Does the difference remain if you set rbd
cache writethrough until flush = false ?
Josh
next prev parent reply other threads:[~2014-09-17 21:28 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-09-17 20:55 severe librbd performance degradation in Giant Somnath Roy
2014-09-17 20:59 ` Mark Nelson
2014-09-17 21:01 ` Somnath Roy
[not found] ` <BA7B69AA-4906-4836-A2F6-5A6EE756A548@profihost.ag>
2014-09-17 21:08 ` Somnath Roy
2014-09-17 21:20 ` Josh Durgin [this message]
2014-09-17 21:29 ` Somnath Roy
2014-09-17 21:34 ` Mark Nelson
2014-09-17 21:37 ` Somnath Roy
2014-09-17 21:40 ` Josh Durgin
2014-09-17 21:35 ` Sage Weil
2014-09-17 21:38 ` Somnath Roy
2014-09-17 21:44 ` Somnath Roy
2014-09-17 23:44 ` Somnath Roy
2014-09-18 2:27 ` Haomai Wang
2014-09-18 3:03 ` Somnath Roy
2014-09-18 3:52 ` Sage Weil
2014-09-18 6:24 ` Somnath Roy
2014-09-18 8:45 ` Chen, Xiaoxi
2014-09-18 14:11 ` Sage Weil
2014-09-18 9:49 ` Alexandre DERUMIER
2014-09-18 12:38 ` Mark Nelson
2014-09-18 18:02 ` Somnath Roy
2014-09-19 1:08 ` Shu, Xinxin
2014-09-19 1:10 ` Shu, Xinxin
2014-09-19 6:53 ` Stefan Priebe
2014-09-19 13:02 ` Shu, Xinxin
2014-09-19 13:31 ` Stefan Priebe - Profihost AG
2014-09-19 13:49 ` David Moreau Simard
2014-09-19 13:56 ` Alexandre DERUMIER
2014-09-19 15:28 ` Sage Weil
2014-09-19 10:09 ` Alexandre DERUMIER
2014-09-19 11:30 ` Alexandre DERUMIER
2014-09-19 12:51 ` Alexandre DERUMIER
2014-09-19 15:15 ` Sage Weil
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5419FB1B.4020202@inktank.com \
--to=josh.durgin@inktank.com \
--cc=Somnath.Roy@sandisk.com \
--cc=ceph-devel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.