[PATCH] lockd: ratelimit "lockd: cannot monitor" messages

[PATCH] lockd: ratelimit "lockd: cannot monitor" messages

[Jeff Layton]

When lockd can't talk to a remote statd, it'll spew a warning message
to the ring buffer. If the application is really hammering on locks
however, it's possible for that message to spam the logs. Ratelimit it
to minimize the potential for harm.

Reported-by: Ian Collier <imc@cs.ox.ac.uk>
Signed-off-by: Jeff Layton <jlayton@primarydata.com>
---
 fs/lockd/mon.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/lockd/mon.c b/fs/lockd/mon.c
index 9106f42c472c..1cc6ec51e6b1 100644
--- a/fs/lockd/mon.c
+++ b/fs/lockd/mon.c
@@ -214,7 +214,7 @@ int nsm_monitor(const struct nlm_host *host)
 	if (unlikely(res.status != 0))
 		status = -EIO;
 	if (unlikely(status < 0)) {
-		printk(KERN_NOTICE "lockd: cannot monitor %s\n", nsm->sm_name);
+		pr_notice_ratelimited("lockd: cannot monitor %s\n", nsm->sm_name);
 		return status;
 	}
 
-- 
1.9.3

Re: [PATCH] lockd: ratelimit "lockd: cannot monitor" messages

[Trond Myklebust]

On Fri, Oct 31, 2014 at 2:28 PM, Jeff Layton <jlayton@primarydata.com> wrote:
> When lockd can't talk to a remote statd, it'll spew a warning message
> to the ring buffer. If the application is really hammering on locks
> however, it's possible for that message to spam the logs. Ratelimit it
> to minimize the potential for harm.
>
> Reported-by: Ian Collier <imc@cs.ox.ac.uk>
> Signed-off-by: Jeff Layton <jlayton@primarydata.com>
> ---
>  fs/lockd/mon.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/fs/lockd/mon.c b/fs/lockd/mon.c
> index 9106f42c472c..1cc6ec51e6b1 100644
> --- a/fs/lockd/mon.c
> +++ b/fs/lockd/mon.c
> @@ -214,7 +214,7 @@ int nsm_monitor(const struct nlm_host *host)
>         if (unlikely(res.status != 0))
>                 status = -EIO;
>         if (unlikely(status < 0)) {
> -               printk(KERN_NOTICE "lockd: cannot monitor %s\n", nsm->sm_name);
> +               pr_notice_ratelimited("lockd: cannot monitor %s\n", nsm->sm_name);
>                 return status;
>         }
>

How is this being triggered repeatedly? Normally, the 'cannot monitor'
message should be happening at client mount time or not at all.

-- 
Trond Myklebust

Linux NFS client maintainer, PrimaryData

trond.myklebust@primarydata.com

Re: [PATCH] lockd: ratelimit "lockd: cannot monitor" messages

[Jeff Layton]

On Fri, 31 Oct 2014 14:36:16 +0200
Trond Myklebust <trond.myklebust@primarydata.com> wrote:

> On Fri, Oct 31, 2014 at 2:28 PM, Jeff Layton <jlayton@primarydata.com> wrote:
> > When lockd can't talk to a remote statd, it'll spew a warning message
> > to the ring buffer. If the application is really hammering on locks
> > however, it's possible for that message to spam the logs. Ratelimit it
> > to minimize the potential for harm.
> >
> > Reported-by: Ian Collier <imc@cs.ox.ac.uk>
> > Signed-off-by: Jeff Layton <jlayton@primarydata.com>
> > ---
> >  fs/lockd/mon.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/fs/lockd/mon.c b/fs/lockd/mon.c
> > index 9106f42c472c..1cc6ec51e6b1 100644
> > --- a/fs/lockd/mon.c
> > +++ b/fs/lockd/mon.c
> > @@ -214,7 +214,7 @@ int nsm_monitor(const struct nlm_host *host)
> >         if (unlikely(res.status != 0))
> >                 status = -EIO;
> >         if (unlikely(status < 0)) {
> > -               printk(KERN_NOTICE "lockd: cannot monitor %s\n", nsm->sm_name);
> > +               pr_notice_ratelimited("lockd: cannot monitor %s\n", nsm->sm_name);
> >                 return status;
> >         }
> >
> 
> How is this being triggered repeatedly? Normally, the 'cannot monitor'
> message should be happening at client mount time or not at all.
> 

No, it's triggered on NLM lock activity. I don't think we really do
much with NLM at mount time, do we? In any case, the bug was reported
here:

    https://bugzilla.redhat.com/show_bug.cgi?id=1155232

In this case, it was an application that was apparently responding to
ENOLCK errors by retrying the lock.

-- 
Jeff Layton <jlayton@primarydata.com>

Re: [PATCH] lockd: ratelimit "lockd: cannot monitor" messages

[J. Bruce Fields]

On Fri, Oct 31, 2014 at 08:28:29AM -0400, Jeff Layton wrote:
> When lockd can't talk to a remote statd, it'll spew a warning message
> to the ring buffer. If the application is really hammering on locks
> however, it's possible for that message to spam the logs. Ratelimit it
> to minimize the potential for harm.

Thanks, applying for 3.19.--b.

> 
> Reported-by: Ian Collier <imc@cs.ox.ac.uk>
> Signed-off-by: Jeff Layton <jlayton@primarydata.com>
> ---
>  fs/lockd/mon.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/fs/lockd/mon.c b/fs/lockd/mon.c
> index 9106f42c472c..1cc6ec51e6b1 100644
> --- a/fs/lockd/mon.c
> +++ b/fs/lockd/mon.c
> @@ -214,7 +214,7 @@ int nsm_monitor(const struct nlm_host *host)
>  	if (unlikely(res.status != 0))
>  		status = -EIO;
>  	if (unlikely(status < 0)) {
> -		printk(KERN_NOTICE "lockd: cannot monitor %s\n", nsm->sm_name);
> +		pr_notice_ratelimited("lockd: cannot monitor %s\n", nsm->sm_name);
>  		return status;
>  	}
>  
> -- 
> 1.9.3
>