From: Reeted <reeted-9AbUPqfR1/2XDw4h08c5KA@public.gmane.org>
To: "Atchley, Scott" <atchleyes-1Heg1YXhbW8@public.gmane.org>
Cc: linux-rdma <linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Subject: Re: IPoIB performance
Date: Wed, 05 Sep 2012 21:04:19 +0200 [thread overview]
Message-ID: <5047A233.2040602@shiftmail.org> (raw)
In-Reply-To: <64A9A0CD-00E0-4455-A641-324FA9BB8BC2-1Heg1YXhbW8@public.gmane.org>
On 09/05/12 19:59, Atchley, Scott wrote:
> On Sep 5, 2012, at 1:50 PM, Reeted wrote:
>
>>
>> I have read that with newer cards the datagram (unconnected) mode is
>> faster at IPoIB than connected mode. Do you want to check?
> I have read that the latency is lower (better) but the bandwidth is lower.
>
> Using datagram mode limits the MTU to 2044 and the throughput to ~3 Gb/s on these machines/cards. Connected mode at the same MTU performs roughly the same. The win in connected mode comes with larger MTUs. With a 9000 MTU, I see ~6 Gb/s. Pushing the MTU to 655120 (the maximum for ipoib), I can get ~13 Gb/s.
>
Have a look at an old thread in this ML by Sebastien Dugue "IPoIB to
Ethernet routing performance"
He had numbers much higher than yours on similar hardware, and was
suggested to use datagram to achieve offloading and even higher speeds.
Keep me informed if you can fix this, I am interested but can't test
infiniband myself right now.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2012-09-05 19:04 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-08-29 19:35 IPoIB performance Atchley, Scott
[not found] ` <FFD82983-4A73-4A52-B6BE-C63DA16A507C-1Heg1YXhbW8@public.gmane.org>
2012-09-05 15:51 ` Christoph Lameter
[not found] ` <0000013997217ec3-f6e2593f-2ff8-408d-814e-0345582b31ca-000000-p/GC64/jrecnJqMo6gzdpkEOCMrvLtNR@public.gmane.org>
2012-09-05 17:09 ` Atchley, Scott
[not found] ` <3F476926-8618-4233-A150-C5D487B55C68-1Heg1YXhbW8@public.gmane.org>
2012-09-05 18:20 ` Christoph Lameter
[not found] ` <0000013997a928ab-36ad5a02-3c82-4daf-8e8a-a86c65e92376-000000-p/GC64/jrecnJqMo6gzdpkEOCMrvLtNR@public.gmane.org>
2012-09-05 18:30 ` Atchley, Scott
[not found] ` <78D8717C-0505-408B-8625-A9124AB33C9E-1Heg1YXhbW8@public.gmane.org>
2012-09-05 19:06 ` Christoph Lameter
[not found] ` <0000013997d35848-57319f33-839b-4480-a075-53b36f67bfe2-000000-p/GC64/jrecnJqMo6gzdpkEOCMrvLtNR@public.gmane.org>
2012-09-05 19:48 ` Atchley, Scott
[not found] ` <E88DEA9F-416B-4663-A292-5780DFF9B641-1Heg1YXhbW8@public.gmane.org>
2012-09-05 19:53 ` Christoph Lameter
2012-09-05 20:12 ` Ezra Kissel
[not found] ` <5047B23C.6080604-GZvvpLG7cYSVc3sceRu5cw@public.gmane.org>
2012-09-05 20:32 ` Atchley, Scott
2012-09-05 19:13 ` Christoph Lameter
[not found] ` <0000013997da4014-44dda0e8-01f3-48b8-b0cd-fe41164d590c-000000-p/GC64/jrecnJqMo6gzdpkEOCMrvLtNR@public.gmane.org>
2012-09-05 19:52 ` Atchley, Scott
[not found] ` <F90894CF-B55C-4AF7-845C-279FDF44351E-1Heg1YXhbW8@public.gmane.org>
2012-09-05 20:26 ` Christoph Lameter
2012-09-05 17:52 ` Reeted
2012-09-05 17:50 ` Reeted
[not found] ` <504790D8.7010307-9AbUPqfR1/2XDw4h08c5KA@public.gmane.org>
2012-09-05 17:59 ` Atchley, Scott
[not found] ` <64A9A0CD-00E0-4455-A641-324FA9BB8BC2-1Heg1YXhbW8@public.gmane.org>
2012-09-05 19:04 ` Reeted [this message]
[not found] ` <5047A233.2040602-9AbUPqfR1/2XDw4h08c5KA@public.gmane.org>
2012-09-05 19:46 ` Atchley, Scott
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5047A233.2040602@shiftmail.org \
--to=reeted-9abupqfr1/2xdw4h08c5ka@public.gmane.org \
--cc=atchleyes-1Heg1YXhbW8@public.gmane.org \
--cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox