Re: Tuning Linux NFSv4 for high latency connections?

[Malahal Naineni <malahal@us.ibm.com>]

Cedric Blancher [cedric.blancher@gmail.com] wrote:
> On 23 April 2014 23:15, Malahal Naineni <malahal@us.ibm.com> wrote:
> > Cedric Blancher [cedric.blancher@gmail.com] wrote:
> >> On 23 April 2014 22:44, Malahal Naineni <malahal@us.ibm.com> wrote:
> >> > Cedric Blancher [cedric.blancher@gmail.com] wrote:
> >> >> On 23 April 2014 22:24, Malahal Naineni <malahal@us.ibm.com> wrote:
> >> >> > Cedric Blancher [cedric.blancher@gmail.com] wrote:
> >> >> >> Are there any options to improve the Linux NFSv4 performance over a
> >> >> >> high latency connection?
> >> >> >>
> >> >> >> We currently use Solaris/Illumos as NFSv4 server and client over a
> >> >> >> cross continental Internet connection. Latency is terrible (~220ms)
> >> >> >> but the counter this by running work in parallel so the latency is
> >> >> >> mostly mitigated.
> >> >> >>
> >> >> >> We now wish to migrate (short: Away from Oracle because support is
> >> >> >> basically unbearable) to Linux (tested SuSE 13.1 and current Fedora)
> >> >> >> and build times are 17 times (!!!) SLOWER than on Solaris/Illumos.
> >> >> >>
> >> >> >> Are there any tunables besides actimeo=300?
> >> >> >
> >> >> > rsize and wsize may help! You need to figure out if the read is the
> >> >> > issue or the write before you dig further.
> >> >>
> >> >> I already tried to tune rsize/wsize, making them both smaller or the
> >> >> maximum of 1048576 bytes, with no effect.
> >> >>
> >> >> One possible theory is that maybe something in Linux doesn't allow
> >> >> multiple requests to be issued in parallel and waits for each request
> >> >> to be completed before issuing the next one?
> >> >
> >> > Linux NFS client can issue I/Os in parallel. Should be limited by number
> >> > of RPC slots though.
> >>
> >> What controls the number of RPC slots? is there a tunable? Is there
> >> something to monitor the usage?
> >
> > sysctl sunrpc.tcp_slot_table_entries (if you are using tcp)
> 
> Its 16
> NFSv4 is tcp only
> 
> I tried to bump the value to 128 - without effect - but the change is
> not persistent across reboots. Is there something like Solaris
> /etc/system which the kernel reads to set these values?

Probably depends on your distro. Look at /etc/sysctl.conf if you have
that file.

> > Also, mountstats <mount-point> would be very helpful.
> 
> I don't have that command. likely my test machine is too old

Hmm, my RHEL6.4 has it. What nfs-utils package you have.

Regards, Malahal.