* [PATCH v4 1/3] sunrpc: add a struct rpc_stats arg to rpc_create_args
2024-01-26 21:48 [PATCH v4 0/3] Make nfs stats visible in network NS Josef Bacik
@ 2024-01-26 21:49 ` Josef Bacik
2024-01-26 21:49 ` [PATCH v4 2/3] nfs: expose /proc/net/sunrpc/nfs in net namespaces Josef Bacik
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Josef Bacik @ 2024-01-26 21:49 UTC (permalink / raw)
To: trond.myklebust, anna, linux-nfs, kernel-team
We want to be able to have our rpc stats handled in a per network
namespace manner, so add an option to rpc_create_args to specify a
different rpc_stats struct instead of using the one on the rpc_program.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
include/linux/sunrpc/clnt.h | 1 +
net/sunrpc/clnt.c | 2 +-
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/clnt.h
index 5e9d1469c6fa..5321585c778f 100644
--- a/include/linux/sunrpc/clnt.h
+++ b/include/linux/sunrpc/clnt.h
@@ -139,6 +139,7 @@ struct rpc_create_args {
const char *servername;
const char *nodename;
const struct rpc_program *program;
+ struct rpc_stat *stats;
u32 prognumber; /* overrides program->number */
u32 version;
rpc_authflavor_t authflavor;
diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c
index cda0935a68c9..bc8c209fc0c7 100644
--- a/net/sunrpc/clnt.c
+++ b/net/sunrpc/clnt.c
@@ -405,7 +405,7 @@ static struct rpc_clnt * rpc_new_client(const struct rpc_create_args *args,
clnt->cl_maxproc = version->nrprocs;
clnt->cl_prog = args->prognumber ? : program->number;
clnt->cl_vers = version->number;
- clnt->cl_stats = program->stats;
+ clnt->cl_stats = args->stats ? : program->stats;
clnt->cl_metrics = rpc_alloc_iostats(clnt);
rpc_init_pipe_dir_head(&clnt->cl_pipedir_objects);
err = -ENOMEM;
--
2.43.0
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PATCH v4 2/3] nfs: expose /proc/net/sunrpc/nfs in net namespaces
2024-01-26 21:48 [PATCH v4 0/3] Make nfs stats visible in network NS Josef Bacik
2024-01-26 21:49 ` [PATCH v4 1/3] sunrpc: add a struct rpc_stats arg to rpc_create_args Josef Bacik
@ 2024-01-26 21:49 ` Josef Bacik
2024-01-26 21:49 ` [PATCH v4 3/3] nfs: make the rpc_stat per net namespace Josef Bacik
2024-01-29 11:17 ` [PATCH v4 0/3] Make nfs stats visible in network NS Jeff Layton
3 siblings, 0 replies; 5+ messages in thread
From: Josef Bacik @ 2024-01-26 21:49 UTC (permalink / raw)
To: trond.myklebust, anna, linux-nfs, kernel-team
We're using nfs mounts inside of containers in production and noticed
that the nfs stats are not exposed in /proc. This is a problem for us
as we use these stats for monitoring, and have to do this awkward bind
mount from the main host into the container in order to get to these
states.
Add the rpc_proc_register call to the pernet operations entry and exit
points so these stats can be exposed inside of network namespaces.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
fs/nfs/inode.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
index ebb8d60e1152..e11e9c34aa56 100644
--- a/fs/nfs/inode.c
+++ b/fs/nfs/inode.c
@@ -2427,11 +2427,13 @@ EXPORT_SYMBOL_GPL(nfs_net_id);
static int nfs_net_init(struct net *net)
{
nfs_clients_init(net);
+ rpc_proc_register(net, &nfs_rpcstat);
return nfs_fs_proc_net_init(net);
}
static void nfs_net_exit(struct net *net)
{
+ rpc_proc_unregister(net, "nfs");
nfs_fs_proc_net_exit(net);
nfs_clients_exit(net);
}
@@ -2486,15 +2488,12 @@ static int __init init_nfs_fs(void)
if (err)
goto out1;
- rpc_proc_register(&init_net, &nfs_rpcstat);
-
err = register_nfs_fs();
if (err)
goto out0;
return 0;
out0:
- rpc_proc_unregister(&init_net, "nfs");
nfs_destroy_directcache();
out1:
nfs_destroy_writepagecache();
@@ -2524,7 +2523,6 @@ static void __exit exit_nfs_fs(void)
nfs_destroy_inodecache();
nfs_destroy_nfspagecache();
unregister_pernet_subsys(&nfs_net_ops);
- rpc_proc_unregister(&init_net, "nfs");
unregister_nfs_fs();
nfs_fs_proc_exit();
nfsiod_stop();
--
2.43.0
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PATCH v4 3/3] nfs: make the rpc_stat per net namespace
2024-01-26 21:48 [PATCH v4 0/3] Make nfs stats visible in network NS Josef Bacik
2024-01-26 21:49 ` [PATCH v4 1/3] sunrpc: add a struct rpc_stats arg to rpc_create_args Josef Bacik
2024-01-26 21:49 ` [PATCH v4 2/3] nfs: expose /proc/net/sunrpc/nfs in net namespaces Josef Bacik
@ 2024-01-26 21:49 ` Josef Bacik
2024-01-29 11:17 ` [PATCH v4 0/3] Make nfs stats visible in network NS Jeff Layton
3 siblings, 0 replies; 5+ messages in thread
From: Josef Bacik @ 2024-01-26 21:49 UTC (permalink / raw)
To: trond.myklebust, anna, linux-nfs, kernel-team
Now that we're exposing the rpc stats on a per-network namespace basis,
move this struct into struct nfs_net and use that to make sure only the
per-network namespace stats are exposed.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
---
fs/nfs/client.c | 5 ++++-
fs/nfs/inode.c | 4 +++-
fs/nfs/internal.h | 2 --
fs/nfs/netns.h | 2 ++
4 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index 44eca51b2808..4d9249c99989 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -73,7 +73,6 @@ const struct rpc_program nfs_program = {
.number = NFS_PROGRAM,
.nrvers = ARRAY_SIZE(nfs_version),
.version = nfs_version,
- .stats = &nfs_rpcstat,
.pipe_dir_name = NFS_PIPE_DIRNAME,
};
@@ -502,6 +501,7 @@ int nfs_create_rpc_client(struct nfs_client *clp,
const struct nfs_client_initdata *cl_init,
rpc_authflavor_t flavor)
{
+ struct nfs_net *nn = net_generic(clp->cl_net, nfs_net_id);
struct rpc_clnt *clnt = NULL;
struct rpc_create_args args = {
.net = clp->cl_net,
@@ -513,6 +513,7 @@ int nfs_create_rpc_client(struct nfs_client *clp,
.servername = clp->cl_hostname,
.nodename = cl_init->nodename,
.program = &nfs_program,
+ .stats = &nn->rpcstats,
.version = clp->rpc_ops->version,
.authflavor = flavor,
.cred = cl_init->cred,
@@ -1175,6 +1176,8 @@ void nfs_clients_init(struct net *net)
#endif
spin_lock_init(&nn->nfs_client_lock);
nn->boot_time = ktime_get_real();
+ memset(&nn->rpcstats, 0, sizeof(nn->rpcstats));
+ nn->rpcstats.program = &nfs_program;
nfs_netns_sysfs_setup(nn, net);
}
diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
index e11e9c34aa56..91b4d811958a 100644
--- a/fs/nfs/inode.c
+++ b/fs/nfs/inode.c
@@ -2426,8 +2426,10 @@ EXPORT_SYMBOL_GPL(nfs_net_id);
static int nfs_net_init(struct net *net)
{
+ struct nfs_net *nn = net_generic(net, nfs_net_id);
+
nfs_clients_init(net);
- rpc_proc_register(net, &nfs_rpcstat);
+ rpc_proc_register(net, &nn->rpcstats);
return nfs_fs_proc_net_init(net);
}
diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index e3722ce6722e..06253695fe53 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -449,8 +449,6 @@ int nfs_try_get_tree(struct fs_context *);
int nfs_get_tree_common(struct fs_context *);
void nfs_kill_super(struct super_block *);
-extern struct rpc_stat nfs_rpcstat;
-
extern int __init register_nfs_fs(void);
extern void __exit unregister_nfs_fs(void);
extern bool nfs_sb_active(struct super_block *sb);
diff --git a/fs/nfs/netns.h b/fs/nfs/netns.h
index c8374f74dce1..a68b21603ea9 100644
--- a/fs/nfs/netns.h
+++ b/fs/nfs/netns.h
@@ -9,6 +9,7 @@
#include <linux/nfs4.h>
#include <net/net_namespace.h>
#include <net/netns/generic.h>
+#include <linux/sunrpc/stats.h>
struct bl_dev_msg {
int32_t status;
@@ -34,6 +35,7 @@ struct nfs_net {
struct nfs_netns_client *nfs_client;
spinlock_t nfs_client_lock;
ktime_t boot_time;
+ struct rpc_stat rpcstats;
#ifdef CONFIG_PROC_FS
struct proc_dir_entry *proc_nfsfs;
#endif
--
2.43.0
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PATCH v4 0/3] Make nfs stats visible in network NS
2024-01-26 21:48 [PATCH v4 0/3] Make nfs stats visible in network NS Josef Bacik
` (2 preceding siblings ...)
2024-01-26 21:49 ` [PATCH v4 3/3] nfs: make the rpc_stat per net namespace Josef Bacik
@ 2024-01-29 11:17 ` Jeff Layton
3 siblings, 0 replies; 5+ messages in thread
From: Jeff Layton @ 2024-01-29 11:17 UTC (permalink / raw)
To: Josef Bacik, trond.myklebust, anna, linux-nfs, kernel-team
On Fri, 2024-01-26 at 16:48 -0500, Josef Bacik wrote:
> v1: https://lore.kernel.org/linux-nfs/cover.1706124811.git.josef@toxicpanda.com/
> v2: https://lore.kernel.org/linux-nfs/cover.1706212207.git.josef@toxicpanda.com/
> v3: https://lore.kernel.org/linux-nfs/cover.1706283674.git.josef@toxicpanda.com/
>
> v3->v4:
> - Fix a weird formatting thing that snuck into 1/3.
>
> v2->v3:
> - Split out the nfs and nfsd related changes into their own patches.
> - Dropped the change adding sv_stats throuch svc_create()
> - Changed the th_cnt to be global, re-arranged it's location.
>
> v1->v2:
> - rework the sunprc service creation to take a pointer to the sv_stats.
> - dropped ->pg_stats from the svc_program.
> - converted all of the nfsd global stats to per-network namespace.
> - added the ability to point at a specific rpc_stat for rpc program creation.
> - converted the rpc stats for nfs to per-network namespace.
>
> -- Original email --
> Hello,
>
> We're currently deploying NFS internally and have run into some oddities with
> our usage of containers. All of the services that mount and export NFS volumes
> run inside of containers, specifically all the namespaces including network
> namespaces. Our monitoring is done on a per-container basis, so we need access
> to the nfs and nfsd stats that are under /proc/net/sunrpc. However these are
> only tied to the init_net, which makes them invisible to containers in a
> different network namespace.
>
> Fix this so that these files are tied to the network namespace. This allows us
> to avoid the hack of bind mounting the hosts /proc into the container in order
> to do proper monitoring. Thanks,
>
> Josef
>
> Josef Bacik (3):
> sunrpc: add a struct rpc_stats arg to rpc_create_args
> nfs: expose /proc/net/sunrpc/nfs in net namespaces
> nfs: make the rpc_stat per net namespace
>
> fs/nfs/client.c | 5 ++++-
> fs/nfs/inode.c | 8 ++++----
> fs/nfs/internal.h | 2 --
> fs/nfs/netns.h | 2 ++
> include/linux/sunrpc/clnt.h | 1 +
> net/sunrpc/clnt.c | 2 +-
> 6 files changed, 12 insertions(+), 8 deletions(-)
>
This all looks good to me. You can add:
Reviewed-by: Jeff Layton <jlayton@kernel.org>
^ permalink raw reply [flat|nested] 5+ messages in thread