From: "Mkrtchyan, Tigran" <tigran.mkrtchyan@desy.de>
To: Benjamin Coddington <bcodding@redhat.com>
Cc: Trond Myklebust <trondmy@gmail.com>,
linux-nfs <linux-nfs@vger.kernel.org>
Subject: Re: [PATCH v3 00/17] Readdir enhancements
Date: Sun, 8 Nov 2020 19:15:19 +0100 (CET) [thread overview]
Message-ID: <1298679928.7009739.1604859319295.JavaMail.zimbra@desy.de> (raw)
In-Reply-To: <0A45C334-A375-47DC-BA04-F25341F263FA@redhat.com>
----- Original Message -----
> From: "Benjamin Coddington" <bcodding@redhat.com>
> To: "Trond Myklebust" <trondmy@gmail.com>
> Cc: "linux-nfs" <linux-nfs@vger.kernel.org>
> Sent: Saturday, 7 November, 2020 13:49:31
> Subject: Re: [PATCH v3 00/17] Readdir enhancements
> On 4 Nov 2020, at 11:16, trondmy@gmail.com wrote:
>
>> From: Trond Myklebust <trond.myklebust@hammerspace.com>
>>
>> The following patch series performs a number of cleanups on the readdir
>> code.
>> It also adds support for 1MB readdir RPC calls on-the-wire, and modifies
>> the caching code to ensure that we cache the entire contents of that
>> 1MB call (instead of discarding the data that doesn't fit into a single
>> page).
>>
>> v2: Fix the handling of the NFSv3/v4 directory verifier
>> v3: Optimise searching when the readdir cookies are seen to be ordered
>
> Hi Trond, thanks for these.
>
> I did a bit of testing with these on 4-core/4G client listing 1.5M files
> with READDIR. I compared v5.10-rc2 without/with this set.
>
> +------+ v5.10.rc-2 +--+ this v3 patch set +
>| run | time | rpc calls | | time | rpc calls |
>
> nfsv3 with dtsize 262144:
> +------+---------+-----------+--+--------+-----------+
>| 1 | 81.583 | 14710 | | 53.568 | 215 |
>| 2 | 81.147 | 14710 | | 50.781 | 215 |
>| 3 | 81.61 | 14710 | | 50.514 | 215 |
>| 4 | 82.405 | 14710 | | 50.746 | 215 |
>| 5 | 82.066 | 14710 | | 50.397 | 215 |
>| 6 | 82.395 | 14710 | | 50.892 | 215 |
>| 7 | 81.657 | 14710 | | 50.882 | 215 |
>| 8 | 81.555 | 14710 | | 50.981 | 215 |
>| 9 | 81.421 | 14710 | | 50.558 | 215 |
>| 10 | 81.472 | 14710 | | 50.588 | 215 |
>
> nfsv3 with dtsize 1048576:
> +------+---------+-----------+--+--------+-----------+
>| 1 | 81.563 | 14710 | | 52.692 | 61 |
>| 2 | 82.123 | 14710 | | 49.934 | 61 |
>| 3 | 81.714 | 14710 | | 50.158 | 61 |
>| 4 | 81.707 | 14710 | | 50.083 | 61 |
>| 5 | 81.44 | 14710 | | 50.045 | 61 |
>| 6 | 81.685 | 14710 | | 50.021 | 61 |
>| 7 | 81.17 | 14710 | | 50.131 | 61 |
>| 8 | 81.366 | 14710 | | 49.928 | 61 |
>| 9 | 81.067 | 14710 | | 50.081 | 61 |
>| 10 | 81.524 | 14710 | | 50.442 | 61 |
>
> nfsv4 with dtsize 32768:
> +------+---------+-----------+--+--------+-----------+
>| 1 | 99.534 | 14712 | | 79.461 | 331 |
>| 2 | 98.998 | 14712 | | 79.338 | 331 |
>| 3 | 99.462 | 14712 | | 81.101 | 331 |
>| 4 | 99.891 | 14712 | | 78.888 | 331 |
>| 5 | 99.516 | 14712 | | 81.147 | 331 |
>| 6 | 98.649 | 14712 | | 83.084 | 331 |
>| 7 | 101.159 | 14712 | | 80.461 | 331 |
>| 8 | 100.402 | 14712 | | 79.003 | 331 |
>| 9 | 98.548 | 14712 | | 80.619 | 331 |
>| 10 | 97.456 | 14712 | | 81.317 | 331 |
>
> nfsv4 with 1048576:
> +------+---------+-----------+--+--------+-----------+
>| 1 | 100.357 | 14712 | | 78.976 | 91 |
>| 2 | 99.61 | 14712 | | 79.328 | 91 |
>| 3 | 101.095 | 14712 | | 80.649 | 91 |
>| 4 | 107.904 | 14712 | | 78.285 | 91 |
>| 5 | 103.665 | 14712 | | 79.258 | 91 |
>| 6 | 98.877 | 14712 | | 78.817 | 91 |
>| 7 | 99.567 | 14712 | | 81.11 | 91 |
>| 8 | 99.096 | 14712 | | 80.296 | 91 |
>| 9 | 100.124 | 14712 | | 78.865 | 91 |
>| 10 | 100.603 | 14712 | | 79.143 | 91 |
Hi Ben, hi Trond,
though number of RPC call between dtsize 1048576 and 32768
is x3 less, the time it takes almost the same. According to
your results, at some point (<= 32K) a bigger dtsize makes
no difference. As the original dtsize is 32K
(#define NFS_MAX_READDIR_PAGES 8), it looks like that the
performance enhancements mostly contributed by a change
not related to the buffer size.
On another, the number of RPC calls with v3-patch-set drops
by x40. What ever Trond have changed there has a big impact!
Thanks a lot for your efforts,
Tigran.
>
> These look great. Feel free to add either/both of my:
> Reviewed-by: Benjamin Coddington <bcodding@redhat.com>
> Tested-by: Benjamin Coddington <bcodding@redhat.com>
>
> Ben
prev parent reply other threads:[~2020-11-08 18:15 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-04 16:16 [PATCH v3 00/17] Readdir enhancements trondmy
2020-11-04 16:16 ` [PATCH v3 01/17] NFS: Ensure contents of struct nfs_open_dir_context are consistent trondmy
2020-11-04 16:16 ` [PATCH v3 02/17] NFS: Clean up readdir struct nfs_cache_array trondmy
2020-11-04 16:16 ` [PATCH v3 03/17] NFS: Clean up nfs_readdir_page_filler() trondmy
2020-11-04 16:16 ` [PATCH v3 04/17] NFS: Clean up directory array handling trondmy
2020-11-04 16:16 ` [PATCH v3 05/17] NFS: Don't discard readdir results trondmy
2020-11-04 16:16 ` [PATCH v3 06/17] NFS: Remove unnecessary kmap in nfs_readdir_xdr_to_array() trondmy
2020-11-04 16:16 ` [PATCH v3 07/17] NFS: Replace kmap() with kmap_atomic() in nfs_readdir_search_array() trondmy
2020-11-04 16:16 ` [PATCH v3 08/17] NFS: Simplify struct nfs_cache_array_entry trondmy
2020-11-04 16:16 ` [PATCH v3 09/17] NFS: Support larger readdir buffers trondmy
2020-11-04 16:16 ` [PATCH v3 10/17] NFS: More readdir cleanups trondmy
2020-11-04 16:16 ` [PATCH v3 11/17] NFS: nfs_do_filldir() does not return a value trondmy
2020-11-04 16:16 ` [PATCH v3 12/17] NFS: Reduce readdir stack usage trondmy
2020-11-04 16:16 ` [PATCH v3 13/17] NFS: Cleanup to remove nfs_readdir_descriptor_t typedef trondmy
2020-11-04 16:16 ` [PATCH v3 14/17] NFS: Allow the NFS generic code to pass in a verifier to readdir trondmy
2020-11-04 16:16 ` [PATCH v3 15/17] NFS: Handle NFS4ERR_NOT_SAME and NFSERR_BADCOOKIE from readdir calls trondmy
2020-11-04 16:16 ` [PATCH v3 16/17] NFS: Improve handling of directory verifiers trondmy
2020-11-04 16:16 ` [PATCH v3 17/17] NFS: Optimisations for monotonically increasing readdir cookies trondmy
2020-11-04 21:01 ` [PATCH v3 16/17] NFS: Improve handling of directory verifiers David Wysochanski
2020-11-04 21:31 ` Trond Myklebust
2020-11-04 21:40 ` David Wysochanski
2020-11-06 13:30 ` [PATCH v3 05/17] NFS: Don't discard readdir results David Wysochanski
2020-11-06 15:05 ` Trond Myklebust
2020-11-06 18:00 ` David Wysochanski
2020-11-07 12:49 ` [PATCH v3 00/17] Readdir enhancements Benjamin Coddington
2020-11-07 14:23 ` Trond Myklebust
2020-11-08 11:05 ` Benjamin Coddington
2020-11-08 18:15 ` Mkrtchyan, Tigran [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1298679928.7009739.1604859319295.JavaMail.zimbra@desy.de \
--to=tigran.mkrtchyan@desy.de \
--cc=bcodding@redhat.com \
--cc=linux-nfs@vger.kernel.org \
--cc=trondmy@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox