Re: rpc.mountd high cpu usage

[Sander Klein <roedie@roedie.nl>]

On , J. Bruce Fields wrote:
>> >I think what happens is that exportfs flushes the kernel's export cache
>> >at which point every use of an uncached export triggers an upcall to
>> >mountd.  That upcall is probably visible in the strace as a read of a
>> >file descriptor associated with /proc/net/sunrpc/nfsd.fh/content.
>> >
>> >That upcall is handled by nfs-utils/utils/mountd/cache.c:nfsd_fh(),
>> >which is given a filehandle fragment identifying the filesystem in
>> >question and has to match it to an export.
>> >
>> >That's done by match_fsid().  Which does do a stat of the export path,
>> >but not of all the devices....  That's probably happening in one of the
>> >libblkid calls in uuid_by_path()?  I wonder if there's something wrong
>> >with libblkid configuration or with the way we're using it?
>> 
>> Is there any way I can help getting this fixed? My coding skills are
>> limited but I am very willing to help in any way I can.
> 
> I wonder if ltrace could help determine if libblkid is where most of
> those stat's are coming from (and if so, which calls)?
> 
> May also be worth reading up on libblkid (man libblkid, etc.) and
> checking your configuration to make sure there's nothing obvious broken
> there.  (If so, maybe the libblkid commandline tools would exhibit the
> same problem?)

I didn't have any config in /etc/blkid.conf and the manpage says it will 
use EVALUATE=udev,scan which in fact scans through all the 
/dev/disk/by-* dirs and parses the /proc/partitions. I put 
'EVALUATE=scan' in /etc/blkid.conf but that doesn't help. I restarted 
anything NFS related...

I also attached ltrace to the rpc.mountd process. The output of the 
trace is at http://pastebin.com/ika1Vetq . I'm not sure what I'm looking 
for. The ltrace is done on a server with only 10 harddisks but the 
strace showed the same behavour of newfstatat-ing every disk in a lot of 
ways.

Greets,

Sander