From: Sergio Gelato <sergio.gelato@astro.su.se>
To: linux-nfs@vger.kernel.org
Subject: rpc.idmapd runs out of file descriptors
Date: Wed, 5 Jun 2024 17:19:27 +0200 [thread overview]
Message-ID: <ZmCB_zqdu2cynJ1M@astro.su.se> (raw)
[-- Attachment #1: Type: text/plain, Size: 1430 bytes --]
Observed on Debian 12 (nfs-utils 2.6.2):
May 28 09:40:25 HOSTNAME rpc.idmapd[3602614]: dirscancb: scandir(/run/rpc_pipefs/nfs): Too many open files
[repeated multiple times]
Investigation with lsof on one of the affected systems shows that file desciptors are not being closed:
[...]
rpc.idmap 675 root 126r DIR 0,40 0 10813 /run/rpc_pipefs/nfs/clnt11e6 (deleted)
rpc.idmap 675 root 127u FIFO 0,40 0t0 10817 /run/rpc_pipefs/nfs/clnt11e6/idmap (deleted)
rpc.idmap 675 root 128r DIR 0,40 0 10834 /run/rpc_pipefs/nfs/clnt11ef (deleted)
rpc.idmap 675 root 129u FIFO 0,40 0t0 10838 /run/rpc_pipefs/nfs/clnt11ef/idmap (deleted)
rpc.idmap 675 root 130r DIR 0,40 0 10855 /run/rpc_pipefs/nfs/clnt11f8 (deleted)
rpc.idmap 675 root 131u FIFO 0,40 0t0 10859 /run/rpc_pipefs/nfs/clnt11f8/idmap (deleted)
Raising the verbosity level to 3 results in no "Stale client:" lines.
strace shows no close() calls other than for the /run/rpc_pipefs/nfs directory.
I believe this is because in dirscancb() the loop is exited prematurely
the first time nfsopen() returns -1, preventing later entries in the queue
from being reaped. I've tested the patch below, which seems indeed to cure
the problem. The bug appears to be still unfixed in the current master branch.
[-- Attachment #2: 0013-rpc.idmapd-nfsopen-failures-should-not-be-fatal.patch --]
[-- Type: text/x-diff, Size: 808 bytes --]
From: Sergio Gelato <Sergio.Gelato@astro.su.se>
Date: Tue, 4 Jun 2024 16:02:59 +0200
Subject: rpc.idmapd: nfsopen() failures should not be fatal
dirscancb() loops over all clnt* subdirectories of /run/rpc_pipefs/nfs/.
Some of these directories contain /idmap files, others don't. nfsopen()
returns -1 for the latter; we then want to skip the directory, not abort
the entire scan.
---
utils/idmapd/idmapd.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/utils/idmapd/idmapd.c b/utils/idmapd/idmapd.c
index e79c124..f3c540d 100644
--- a/utils/idmapd/idmapd.c
+++ b/utils/idmapd/idmapd.c
@@ -556,7 +556,7 @@ dirscancb(int fd, short UNUSED(which), void *data)
if (nfsopen(ic) == -1) {
close(ic->ic_dirfd);
free(ic);
- goto out;
+ continue;
}
if (verbose > 2)
next reply other threads:[~2024-06-05 15:28 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-06-05 15:19 Sergio Gelato [this message]
2024-09-14 7:29 ` rpc.idmapd runs out of file descriptors Salvatore Bonaccorso
2024-09-16 0:04 ` NeilBrown
2024-09-16 11:54 ` Steve Dickson
2024-09-16 12:28 ` Sergio.Gelato
2024-09-16 15:42 ` Salvatore Bonaccorso
2024-09-16 17:19 ` Salvatore Bonaccorso
2024-09-16 18:18 ` Salvatore Bonaccorso
2024-09-17 7:06 ` Sergio.Gelato
2024-09-21 18:42 ` Steve Dickson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZmCB_zqdu2cynJ1M@astro.su.se \
--to=sergio.gelato@astro.su.se \
--cc=linux-nfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox