linux-nfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Stefan Hajnoczi <stefanha@redhat.com>
To: linux-nfs@vger.kernel.org
Cc: "J. Bruce Fields" <bfields@fieldses.org>,
	Anna Schumaker <anna.schumaker@netapp.com>,
	Trond Myklebust <trond.myklebust@primarydata.com>
Subject: nfsd leaks 3 sunrpc module references
Date: Fri, 11 Sep 2015 11:45:17 +0100	[thread overview]
Message-ID: <20150911104517.GA4947@stefanha-thinkpad.home> (raw)

I'm having a little trouble debugging this sunrpc module refcnt leak in
nfsd.  Perhaps a fresh pair of eyes can help.

nfsd leaks 3 references to the sunrpc module here:

  # echo -n "asdf 1234" >/proc/fs/nfsd/portlist
  bash: echo: write error: Protocol not supported

Now stop nfsd and try unloading the kernel modules:

  # systemctl stop nfs-server
  # systemctl stop nfs
  # systemctl stop proc-fs-nfsd.mount
  # systemctl stop var-lib-nfs-rpc_pipefs.mount
  # rmmod nfsd
  # rmmod nfs_acl
  # rmmod lockd
  # rmmod auth_rpcgss
  # rmmod sunrpc
  rmmod: ERROR: Module sunrpc is in use
  # lsmod | grep rpc
  sunrpc                315392  3

I have reproduced this from source with a tree based on Linux 4.2-rc3
and also with Linux 4.1.6-200.fc22.x86_64.

The following unmatched module_get() calls are recorded by perf record
-e module:module_get -e module:module_put.  Perhaps the backtrace is a
little mangled because I don't see xdr_reserve_space() calls in
svc_create_pooled() (even indirectly):

nfsd  4291 [002]    87.914752: module:module_get: sunrpc call_site=0xffffffffa031c86as refcnt=7
                  318d93 try_module_get (vmlinux)
                    98da xs_tcp_setup_socket ([sunrpc])
                    5ca5 xprt_create_transport ([sunrpc])
                    34ad rpc_create ([sunrpc])
                   1892a rpcb_getport_async ([sunrpc])
                   196ea xdr_reserve_space ([sunrpc])
                   128dc svc_create_pooled ([sunrpc])
                     686 nfsd_create_serv ([nfsd])
                    1be9 write_ports ([nfsd])
                    182c nfsctl_transaction_write ([nfsd])
                  410d87 __vfs_write (vmlinux)
                  411706 vfs_write (vmlinux)
                  4123f5 sys_write (vmlinux)
                  95ed95 tracesys_phase2 (vmlinux)
                   f2ff0 __GI___libc_write (/usr/lib64/libc-2.21.so)

nfsd  4291 [002]    87.914775: module:module_get: sunrpc call_site=0xffffffffa03228a5s refcnt=8
                  318d93 try_module_get (vmlinux)
                    f915 rpcauth_cache_do_shrink ([sunrpc])
                    2bbd rpc_new_client ([sunrpc])
                    337a rpc_create_xprt ([sunrpc])
                    34d6 rpc_create ([sunrpc])
                   1892a rpcb_getport_async ([sunrpc])
                   196ea xdr_reserve_space ([sunrpc])
                   128dc svc_create_pooled ([sunrpc])
                     686 nfsd_create_serv ([nfsd])
                    1be9 write_ports ([nfsd])
                    182c nfsctl_transaction_write ([nfsd])
                  410d87 __vfs_write (vmlinux)
                  411706 vfs_write (vmlinux)
                  4123f5 sys_write (vmlinux)
                  95ed95 tracesys_phase2 (vmlinux)
                   f2ff0 __GI___libc_write (/usr/lib64/libc-2.21.so)

nfsd  4291 [002]    87.914936: module:module_get: sunrpc call_site=0xffffffffa03228a5s refcnt=9
                  318d93 try_module_get (vmlinux)
                    f915 rpcauth_cache_do_shrink ([sunrpc])
                    2bbd rpc_new_client ([sunrpc])
                    2ead __rpc_clone_client ([sunrpc])
                    35e7 rpc_bind_new_program ([sunrpc])
                   18949 rpcb_getport_async ([sunrpc])
                   196ea xdr_reserve_space ([sunrpc])
                   128dc svc_create_pooled ([sunrpc])
                     686 nfsd_create_serv ([nfsd])
                    1be9 write_ports ([nfsd])
                    182c nfsctl_transaction_write ([nfsd])
                  410d87 __vfs_write (vmlinux)
                  411706 vfs_write (vmlinux)
                  4123f5 sys_write (vmlinux)
                  95ed95 tracesys_phase2 (vmlinux)
                   f2ff0 __GI___libc_write (/usr/lib64/libc-2.21.so)

In this trace I modified nfs-utils nfsd to write to portlist but you get
the same result with echo from a shell (as shown above).

Any ideas?

Stefan

             reply	other threads:[~2015-09-11 10:45 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-09-11 10:45 Stefan Hajnoczi [this message]
2016-01-04  3:15 ` [PATCH] nfsd: Fix nfsd leaks sunrpc module references Kinglong Mee
2016-01-04  9:03   ` Stefan Hajnoczi
2016-01-04 22:39   ` J. Bruce Fields
2016-01-05  0:49     ` Kinglong Mee
2016-01-05 22:44       ` J. Bruce Fields

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20150911104517.GA4947@stefanha-thinkpad.home \
    --to=stefanha@redhat.com \
    --cc=anna.schumaker@netapp.com \
    --cc=bfields@fieldses.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=trond.myklebust@primarydata.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).