Re: VM issue causing high CPU loads

Andrew Morton
2009-09-03T00:07:07Z

On Mon, 31 Aug 2009 22:39:20 +0200
Yohan <ytordjman@corp.free.fr> wrote:

> Yohan wrote:
> > Andrew Morton wrote:
> >> On Mon, 24 Aug 2009 16:23:22 +0200
> >> Yohan <kernel@yohan.staff.proxad.net> wrote:
> >>> Hi,
> >>>
> >>> Is someone have an idea for that :
> >>>
> >>> http://bugzilla.kernel.org/show_bug.cgi?id=14024
> >>>
> >> Please generate a kernel profile to work out where all the CPU tie is
> >> being spent. Documentation/basic_profiling.txt is a starting point.
> >>
> > I post some new reports, it seems that the problem is in
> > rpcauth_lookup_credcache ...

Thanks, that helps a lot.

> > for information, this is an imap mail server that mounts ~10 netapp
> > over ~300 mountpoints..
> I saw that : http://patchwork.kernel.org/patch/24747/

I wonder what happened with Miquel's patch?

> I did only:
>
> --- linux-2.6.27.21/include/linux/sunrpc/auth.h 2009-03-23 23:04:09.000000000 +0100
> +++ linux-2.6.27.21/include/linux/sunrpc/auth.h 2009-05-19 16:02:35.000000000 +0200
> @@ -62,8 +62,12 @@
> */
> - #define RPC_CREDCACHE_HASHBITS 4
> + #define RPC_CREDCACHE_HASHBITS 12
>
>
> And i test it in prod since sunday: i only have 36% of one core used by
> system
> versus more than 3 cores used by system in another server that did a
> drop_caches at morning...
>

OK, but it's still pretty bad. Let's tell the NFS guys.

In http://bugzilla.kernel.org/show_bug.cgi?id=14024 we appear to have a
major meltdown caused by the linear search in
rpcauth_lookup_credcache() with Yohan's workload.