Re: file xfer over NFSv4 with 'sync' ~300X slower than with 'async' ?

[bfields@fieldses.org (J. Bruce Fields)]

On Tue, Mar 31, 2015 at 07:21:16AM -0700, lyndat3@your-mail.com wrote:
> Narrowing doen the issue, NFSv4 file xfer with 'sync' appears, here, to be ~ 300X slower than with 'async'.
> 
> (1) for NFSv4 mount with 'sync'
> 
> 	grep NAS1 /etc/auto.nfs4 
> 		NAS1   -fstype=nfs4,_netdev,rw,proto=tcp,sync,...  xen01.loc:/
> 
> a 100MB file xfer takes ~8 minutes
> 
> 	rm -f /mnt/NFS4/NAS1/file.out && \
> 	 time dd if=/dev/zero of=/mnt/NFS4/NAS1/file.out  bs=32K count=3K
> 		3072+0 records in
> 		3072+0 records out
> 		100663296 bytes (101 MB) copied, 485.721 s, 207 kB/s
> 
> 		real    8m5.861s
> 		user    0m0.012s
> 		sys     0m0.250s
> 
> (2) Change mount 'sync' -> 'async',
> 
> 	vi /etc/auto.nfs4 
> 	-	NAS1   -fstype=nfs4,_netdev,rw,proto=tcp,sync,...  xen01.loc:/
> 	+	NAS1   -fstype=nfs4,_netdev,rw,proto=tcp,async,...  xen01.loc:/
> 	systemctl restart autofs
> 
> the same 100MB file xfer takes ~ 2 seconds
> 
> 	rm -f /mnt/NFS4/NAS1/file.out && \
> 	 time dd if=/dev/zero of=/mnt/NFS4/NAS1/file.out  bs=32K count=3K
> 		3072+0 records in
> 		3072+0 records out
> 		100663296 bytes (101 MB) copied, 1.65577 s, 60.8 MB/s
> 
> 		real    0m1.658s
> 		user    0m0.000s
> 		sys     0m0.089s
> 
> 
> I'd expect 'sync' to be slower than 'async', but 300X ?

There's no maximum sync/async ratio.  You could make that ratio lower or
higher by varying dd's block size, for example.

The way I'd look at it, your dd of a 100MB file above is doing 3072
writes, and taking about 8*60/3072 =~ .16 seconds per write.

That does sound high.  Things to look at to understand why might include
the round-trip ping time to the server, and the time for the server's
disk to do a synchronous write.

--b.

> 
> Is there additional config that cures, or at least drastically improves, this slow down?
> 
> Some very old (10+ years) posts suggested kernel bugs, but those were fixed ages ago.  Maybe reemerged?
> 
> LT
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html