From: Shirley Ma <shirley.ma-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
To: Mark Lehrer <lehrer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Subject: Re: nfs-rdma performance
Date: Fri, 13 Jun 2014 00:25:40 -0700 [thread overview]
Message-ID: <539AA774.2020209@oracle.com> (raw)
In-Reply-To: <CADvaNzUbqwvXFiDPNr4f-DC8s7kf7t=N0UXkwUXrKDPn6WgMAQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
On 06/12/2014 04:06 PM, Mark Lehrer wrote:
> I am using ConnectX-3 HCA's and Dell R720 servers.
>
> On Thu, Jun 12, 2014 at 2:00 PM, Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org> wrote:
>> On 6/12/2014 2:54 PM, Mark Lehrer wrote:
>>>
>>> Awesome work on nfs-rdma in the later kernels! I had been having
>>> panic problems for awhile and now things appear to be quite reliable.
>>>
>>> Now that things are more reliable, I would like to help work on speed
>>> issues. On this same hardware with SMB Direct and the standard
>>> storage review 8k 70/30 test, I get combined read & write performance
>>> of around 2.5GB/sec. With nfs-rdma it is pushing about 850MB/sec.
>>> This is simply an unacceptable difference.
I was able to get close to 2.5GB/s with ConnectX-2 for direct I/O. What's your test case and wsize/rsize? Did you collect /proc/interrupts, cpu usage and profiling data?
>>>
>>> I'm using the standard settings -- connected mode, 65520 byte MTU,
>>> nfs-server-side "async", lots of nfsd's, and nfsver=3 with large
>>> buffers. Does anyone have any tuning suggestions and/or places to
>>> start looking for bottlenecks?
>>
>>
>> What RDMA device?
>>
>> Steve.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2014-06-13 7:25 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-06-12 19:54 nfs-rdma performance Mark Lehrer
[not found] ` <CADvaNzXHOafuzav9RwB2SHXma7EUCxMQX6sNEL6r5-a_e37pLQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-06-12 20:00 ` Steve Wise
[not found] ` <539A06C3.1030909-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
2014-06-12 23:06 ` Mark Lehrer
[not found] ` <CADvaNzUbqwvXFiDPNr4f-DC8s7kf7t=N0UXkwUXrKDPn6WgMAQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-06-13 7:25 ` Shirley Ma [this message]
2014-06-12 20:14 ` Wendy Cheng
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=539AA774.2020209@oracle.com \
--to=shirley.ma-qhclzuegtsvqt0dzr+alfa@public.gmane.org \
--cc=lehrer-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
--cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox