From: Or Gerlitz <ogerlitz-smomgflXvOZWk0Htik3J/w@public.gmane.org>
To: David Dillow <dillowda-1Heg1YXhbW8@public.gmane.org>
Cc: roland-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Subject: Re: [RFC 0/8] Reliably generate large request from SRP
Date: Mon, 24 Jan 2011 17:32:04 +0200 [thread overview]
Message-ID: <4D3D9B74.8090607@voltaire.com> (raw)
In-Reply-To: <1295528044.22825.64.camel-1q1vX8mYZiGLUyTwlgNVppKKF0rrzTr+@public.gmane.org>
David Dillow wrote:
>> if we look on the 50% for SAS/1M IOs that you're presenting, can you tell
>> what made the difference, srp went from sg_tablesize of 255 to 256 so the
>> upper layers where able to provide 1M as one IO
> This win is from sg_tablesize going from 255 to 256 in this case; the HW
> really likes that better than getting two requests -- one for 1020 KB
> and one for 4 KB.
Its always nice to find the simplest explanation to the greatest improvement... going to the 2nd largest gains
> SAS 2M 520 MB/s 861 MB/s
> SAS 4M 529 MB/s 921 MB/s
> SAS 8M 600 MB/s 951 MB/s
I wonder what made the difference here? it can't be only the 255 --> 256 sg_tablesize change, for the 2M case the change to use 512 pages FMRs could let you use one rkey/fmr for the whole IO but not for 4M/8M
Or.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2011-01-24 15:32 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-01-19 4:27 [RFC 0/8] Reliably generate large request from SRP David Dillow
[not found] ` <1295411242-26148-1-git-send-email-dillowda-1Heg1YXhbW8@public.gmane.org>
2011-01-19 4:27 ` [RFC 1/8] IB/srp: always avoid non-zero offsets into an FMR David Dillow
2011-01-19 4:27 ` [RFC 2/8] IB/srp: move IB CM setup completion into its own function David Dillow
2011-01-19 4:27 ` [RFC 3/8] IB/srp: allow sg_tablesize to be set for each target David Dillow
2011-01-19 4:27 ` [RFC 4/8] IB/srp: rework mapping engine to use multiple FMR entries David Dillow
[not found] ` <1295411242-26148-5-git-send-email-dillowda-1Heg1YXhbW8@public.gmane.org>
2011-01-20 10:04 ` Or Gerlitz
[not found] ` <AANLkTim6H063ta0w2A+zo9QH0jY5qL5uu1OxN4iqMFEm-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2011-01-20 12:36 ` David Dillow
2011-01-19 4:27 ` [RFC 5/8] IB/srp: add safety valve for large SG tables without HW support David Dillow
2011-01-19 4:27 ` [RFC 6/8] IB/srp: add support for indirect tables that don't fit in SRP_CMD David Dillow
2011-01-19 4:27 ` [RFC 7/8] IB/srp: try to use larger FMR sizes to cover our mappings David Dillow
[not found] ` <1295411242-26148-8-git-send-email-dillowda-1Heg1YXhbW8@public.gmane.org>
2011-01-20 10:24 ` Or Gerlitz
[not found] ` <4D380D4B.6060404-smomgflXvOZWk0Htik3J/w@public.gmane.org>
2011-01-20 12:40 ` David Dillow
2011-01-19 4:27 ` [RFC 8/8] IB/srp and direct IO: patches for testing large indirect tables David Dillow
[not found] ` <1295411242-26148-9-git-send-email-dillowda-1Heg1YXhbW8@public.gmane.org>
2011-01-20 10:07 ` Or Gerlitz
[not found] ` <4D38094B.9090101-smomgflXvOZWk0Htik3J/w@public.gmane.org>
2011-01-20 12:33 ` David Dillow
2011-01-19 5:31 ` [RFC 0/8] Reliably generate large request from SRP Roland Dreier
[not found] ` <aday66hxxwe.fsf-FYB4Gu1CFyUAvxtiuMwx3w@public.gmane.org>
2011-01-19 12:01 ` David Dillow
2011-01-20 9:52 ` Or Gerlitz
[not found] ` <4D3805C4.6010203-smomgflXvOZWk0Htik3J/w@public.gmane.org>
2011-01-20 12:54 ` David Dillow
[not found] ` <1295528044.22825.64.camel-1q1vX8mYZiGLUyTwlgNVppKKF0rrzTr+@public.gmane.org>
2011-01-24 15:32 ` Or Gerlitz [this message]
[not found] ` <4D3D9B74.8090607-smomgflXvOZWk0Htik3J/w@public.gmane.org>
2011-01-24 16:14 ` Bart Van Assche
[not found] ` <AANLkTikapxELx5B6knAm6CQaeLsKHWd9EMQeexmFdF1Q-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2011-01-24 18:00 ` David Dillow
2011-01-24 17:53 ` David Dillow
2011-01-20 17:50 ` Roland Dreier
2011-02-19 0:06 ` David Dillow
2011-02-19 0:07 ` David Dillow
[not found] ` <1298074037.15679.17.camel-FqX9LgGZnHWDB2HL1qBt2PIbXMQ5te18@public.gmane.org>
2011-02-22 6:36 ` Or Gerlitz
[not found] ` <4D635974.10807-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2011-02-22 14:49 ` David Dillow
[not found] ` <1298386190.18945.1.camel-1q1vX8mYZiGLUyTwlgNVppKKF0rrzTr+@public.gmane.org>
2011-02-22 23:38 ` Roland Dreier
[not found] ` <AANLkTikxDu5b=p4fXHMm8W+tF3Lru4vB7xRZEF+HDpyu-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2011-02-23 1:23 ` David Dillow
[not found] ` <1300148888.2772.15.camel@lap75545.ornl.gov>
[not found] ` <AANLkTinC9QcE8E_O3M0+dapVGEAZq_tw-3cb3GN4qf-q@mail.gmail.com>
[not found] ` <AANLkTinC9QcE8E_O3M0+dapVGEAZq_tw-3cb3GN4qf-q-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2011-03-15 23:51 ` David Dillow
[not found] ` <1300148888.2772.15.camel-FqX9LgGZnHWDB2HL1qBt2PIbXMQ5te18@public.gmane.org>
2011-03-16 8:27 ` Or Gerlitz
[not found] ` <4D807472.2060000-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2011-03-16 15:07 ` David Dillow
2011-03-16 16:50 ` Roland Dreier
[not found] ` <AANLkTimY74Wmsfc3F35SBuR2YyDW=ao78B=9uGh4LZNJ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2011-03-16 21:41 ` David Dillow
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4D3D9B74.8090607@voltaire.com \
--to=ogerlitz-smomgflxvozwk0htik3j/w@public.gmane.org \
--cc=dillowda-1Heg1YXhbW8@public.gmane.org \
--cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=roland-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox