From: Mark Nelson <mark.nelson@inktank.com>
To: "Atchley, Scott" <atchleyes@ornl.gov>
Cc: Gandalf Corvotempesta <gandalf.corvotempesta@gmail.com>,
Sage Weil <sage@inktank.com>,
"ceph-devel@vger.kernel.org" <ceph-devel@vger.kernel.org>
Subject: Re: SSD journal suggestion
Date: Wed, 07 Nov 2012 10:41:51 -0600 [thread overview]
Message-ID: <509A8F4F.7090404@inktank.com> (raw)
In-Reply-To: <8F1EF04A-96C0-402F-9DE2-3891514BD04F@ornl.gov>
On 11/07/2012 10:35 AM, Atchley, Scott wrote:
> On Nov 7, 2012, at 11:20 AM, Mark Nelson <mark.nelson@inktank.com> wrote:
>
>>>> Right now I'm doing 3 journals per SSD, but topping out at about
>>>> 1.2-1.4GB/s from the client perspective for the node with 15+ drives and
>>>> 5 SSDs. It's possible newer versions of the code and tuning may
>>>> increase that.
>>>
>>> What interconnect is this? 10G Ethernet is 1.25 GB/s line rate and I would expect your Sockets and Ceph overhead to eat into that. Or is it dual 10G Ethernet?
>>
>> This is 8 concurrent instances of rados bench running on localhost.
>> Ceph is configured with 1x replication. 1.2-1.4GB/s is the aggregate
>> throughput of all of the rados bench instances.
>
> Ok, all local with no communication. Given this level of local performance, what does that translate into when talking over the network?
>
> Scott
>
Well, local, but still over tcp. Right now I'm focusing on pushing the
osds/filestores as far as I can, and after that I'm going to setup a
bonded 10GbE network to see what kind of messenger bottlenecks I run
into. Sadly the testing is going slower than I would like.
Mark
next prev parent reply other threads:[~2012-11-07 16:41 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-11-07 12:13 SSD journal suggestion Gandalf Corvotempesta
2012-11-07 12:17 ` Sage Weil
2012-11-07 12:28 ` Gandalf Corvotempesta
2012-11-07 15:01 ` Mark Nelson
2012-11-07 16:12 ` Atchley, Scott
2012-11-07 16:20 ` Mark Nelson
2012-11-07 16:35 ` Atchley, Scott
2012-11-07 16:41 ` Mark Nelson [this message]
2012-11-07 21:11 ` Martin Mailand
2012-11-07 21:14 ` Gandalf Corvotempesta
2012-11-07 21:35 ` Martin Mailand
2012-11-07 21:44 ` Stefan Priebe
2012-11-07 21:55 ` Martin Mailand
2012-11-07 21:59 ` Stefan Priebe
2012-11-07 22:13 ` Martin Mailand
2012-11-07 22:28 ` Gandalf Corvotempesta
2012-11-07 22:39 ` Martin Mailand
2012-11-07 22:51 ` Gandalf Corvotempesta
2012-11-07 23:12 ` Mark Nelson
2012-11-08 8:22 ` Gandalf Corvotempesta
2012-11-08 13:55 ` Atchley, Scott
2012-11-08 14:39 ` Mark Nelson
2012-11-08 15:00 ` Atchley, Scott
2012-11-08 15:02 ` Atchley, Scott
2012-11-08 16:19 ` Andrey Korolyov
2012-11-08 18:03 ` Atchley, Scott
2012-11-08 20:12 ` Joseph Glanville
2012-11-08 21:21 ` SSD journal suggestion / rsockets Dieter Kasper
2012-11-08 22:00 ` Joseph Glanville
2012-11-09 14:43 ` Atchley, Scott
2012-11-09 23:41 ` Joseph Glanville
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=509A8F4F.7090404@inktank.com \
--to=mark.nelson@inktank.com \
--cc=atchleyes@ornl.gov \
--cc=ceph-devel@vger.kernel.org \
--cc=gandalf.corvotempesta@gmail.com \
--cc=sage@inktank.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.