strange activity on otherwise idle linux client

strange activity on otherwise idle linux client

[Dorian Taylor (Lists)]

[-- Attachment #1: Type: text/plain, Size: 1144 bytes --]

Hello,

I noticed yesterday that my Linux (Ubuntu 21.10 x86_64, kernel 5.13.0) NFS4 mount was causing load on the (also Ubuntu, 20.04 LTS) server and when I went to investigate, it seemed like the client was polling the server at about 200 operations per second (100 ops read and the other 100 `nfsiostat` didn’t specify). When I looked at `nfsstat` on the server, I saw a profusion of `putfh` (32%), `sequence` (30%), `layoutget` (15%), and `layoutreturn` (15%) (`getattr` and `lookup` took another 3% between them, and the rest fell below the 1%  threshold, even though the total number of operations recorded was on the order of 100 million for a day or so of uptime with two clients connected).

The chatty client was had a couple files open in Emacs but I closed them and determined that they weren’t being accessed with `lost`, but the polling continued. The strangely round number of the polling rate is suspicious. Is this normal behaviour?

It looks like the NFS version on the client is 1.3.4-6ubuntu1 and server is 1.3.4-2.5ubuntu3.4.

Regards,

--
Dorian Taylor
Make things. Make sense.
https://doriantaylor.com


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: strange activity on otherwise idle linux client

[J. Bruce Fields]

On Sun, Jan 16, 2022 at 08:35:38AM -0500, Dorian Taylor (Lists) wrote:
> I noticed yesterday that my Linux (Ubuntu 21.10 x86_64, kernel 5.13.0)
> NFS4 mount was causing load on the (also Ubuntu, 20.04 LTS) server and
> when I went to investigate, it seemed like the client was polling the
> server at about 200 operations per second (100 ops read and the other
> 100 `nfsiostat` didn’t specify). When I looked at `nfsstat` on the
> server, I saw a profusion of `putfh` (32%), `sequence` (30%),
> `layoutget` (15%), and `layoutreturn` (15%)

The client is trying to use pnfs, which our server doesn't really have
much support for.  So probably either the server's returning
inconsistent results (making it look like it supports pnfs and then
failing when the client actually tries to use it), and/or the client is
failing to give up on pnfs when it should.

If you have the "pnfs" option set on any export, turn it off.

If it's not that, I'm not sure what's happening.  Looking at the traffic
in wireshark might be interesting.

--b.

> (`getattr` and `lookup`
> took another 3% between them, and the rest fell below the 1%
> threshold, even though the total number of operations recorded was on
> the order of 100 million for a day or so of uptime with two clients
> connected).
> 
> The chatty client was had a couple files open in Emacs but I closed
> them and determined that they weren’t being accessed with `lost`, but
> the polling continued. The strangely round number of the polling rate
> is suspicious. Is this normal behaviour?
> 
> It looks like the NFS version on the client is 1.3.4-6ubuntu1 and
> server is 1.3.4-2.5ubuntu3.4.
> 
> Regards,
> 
> -- Dorian Taylor Make things. Make sense.  https://doriantaylor.com
>