From: Hannes Reinecke <hare@suse.de>
To: Mark Nelson <mark.nelson@inktank.com>
Cc: Stefan Priebe <s.priebe@profihost.ag>,
"ceph-devel@vger.kernel.org" <ceph-devel@vger.kernel.org>
Subject: Re: Infiniband 40GB
Date: Mon, 04 Jun 2012 08:22:41 +0200 [thread overview]
Message-ID: <4FCC5431.6070002@suse.de> (raw)
In-Reply-To: <4FCB5EED.30909@inktank.com>
On 06/03/2012 02:56 PM, Mark Nelson wrote:
> On 6/3/12 3:10 AM, Stefan Priebe wrote:
>> Hi List,
>>
>> has anybody already tried CEPH over Infiniband 40GB?
>>
>> Stefan
>> --
>> To unsubscribe from this list: send the line "unsubscribe
>> ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> Hi Stefan,
>
> A couple of folks have done DDR IB. For now you are limited to
> ipoib though. If you have the hardware available I'd be really
> curious what kind of throughput/latencies you see.
>
Hehe.
Good luck with that.
We've tried on 10GigE with _disastrous_ results.
Up to the point where 1GigE was actually _faster_.
So far we've uncovered two issues:
- intel_idle was/is seriously broken (we've tried on 3.0-stable,
so might've been fixed by now)
- osd-server is calling 'fsync' on each and every write request.
Does wonders for performance ...
Cheers,
Hannes
--
Dr. Hannes Reinecke zSeries & Storage
hare@suse.de +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg)
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2012-06-04 6:22 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-06-03 8:10 Infiniband 40GB Stefan Priebe
2012-06-03 12:56 ` Mark Nelson
2012-06-04 6:22 ` Hannes Reinecke [this message]
2012-06-04 7:26 ` Stefan Priebe - Profihost AG
2012-06-04 7:39 ` Hannes Reinecke
2012-06-04 7:53 ` Stefan Priebe - Profihost AG
2012-06-04 8:02 ` Hannes Reinecke
2012-06-04 8:23 ` Stefan Majer
2012-06-04 9:21 ` Yann Dupont
2012-06-04 9:35 ` Alexandre DERUMIER
2012-06-04 9:53 ` Yann Dupont
2012-06-04 9:47 ` Amon Ott
2012-06-04 9:58 ` Yann Dupont
2012-06-04 11:40 ` Alexandre DERUMIER
2012-06-04 12:59 ` Mark Nelson
2012-06-04 13:07 ` Alexandre DERUMIER
2012-06-04 13:28 ` Mark Nelson
2012-06-04 15:11 ` Gregory Farnum
2012-06-04 15:34 ` Mark Nelson
2012-06-06 16:05 ` Alexandre DERUMIER
2012-06-06 16:43 ` Mark Nelson
2012-06-04 15:42 ` Stefan Priebe
2012-06-05 7:08 ` Amon Ott
2012-06-05 7:46 ` Stefan Priebe - Profihost AG
2012-06-06 10:48 ` Stefan Priebe - Profihost AG
2012-06-06 10:57 ` Amon Ott
2012-06-06 11:02 ` Stefan Priebe - Profihost AG
2012-06-07 11:33 ` Amon Ott
2012-06-07 12:44 ` Stefan Priebe - Profihost AG
2012-06-05 8:54 ` Stefan Priebe - Profihost AG
2012-06-04 12:28 ` Mark Nelson
2012-06-04 12:34 ` Tomasz Paszkowski
2012-06-04 12:40 ` Mark Nelson
[not found] <a81f3855-1c7d-447b-9bbf-6a891e372909@mailpro>
2012-06-07 3:31 ` Alexandre DERUMIER
2012-06-07 11:25 ` Alexandre DERUMIER
2012-06-07 17:15 ` Mark Nelson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4FCC5431.6070002@suse.de \
--to=hare@suse.de \
--cc=ceph-devel@vger.kernel.org \
--cc=mark.nelson@inktank.com \
--cc=s.priebe@profihost.ag \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox