From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932956AbbI3XSD (ORCPT ); Wed, 30 Sep 2015 19:18:03 -0400 Received: from ipmail05.adl6.internode.on.net ([150.101.137.143]:51219 "EHLO ipmail05.adl6.internode.on.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932779AbbI3XSB (ORCPT ); Wed, 30 Sep 2015 19:18:01 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2BACwANbQxWPK2+LHlegySBQYJdg32iVQaLDZErBAICgTdNAQEBAQEBBwEBAQFAAT+EJQEBBCcTHCMQCAMYCSUPBSUDBxoTiC3MCgEBAQcCAR8ZhhOFRIUNB4MYgRQBBJV4jQ6PDIxJgnMegWYsM4l6AQEB Date: Thu, 1 Oct 2015 09:17:42 +1000 From: Dave Chinner To: Jeff Layton Cc: "Huang, Ying" , kernel test robot , lkp@01.org, LKML , bfields@fieldses.org Subject: Re: [lkp] [nfsd] 4aac1bf05b: -2.9% fsmark.files_per_sec Message-ID: <20150930231742.GD3902@dastard> References: <87h9mfhwcj.fsf@yhuang-dev.intel.com> <20150929074151.0fe32fc9@tlielax.poochiereds.net> <87lhbokdqd.fsf@yhuang-dev.intel.com> <20150929200613.66c3ab4f@synchrony.poochiereds.net> <871tdg718x.fsf@yhuang-dev.intel.com> <20150930060359.070dba46@tlielax.poochiereds.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150930060359.070dba46@tlielax.poochiereds.net> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Sep 30, 2015 at 06:03:59AM -0400, Jeff Layton wrote: > Thanks for testing it and catching the problem in the first place! > > FWIW, the problem seems to have been bad hash distribution generated by > hash_ptr on struct inode pointers. When the cache had ~10000 entries in > it total, one of the hash chains had almost 2000 entries. When I > switched to hashing on inode->i_ino, the distribution was much better. > > I'm not sure if it was just rotten luck or there is something about > inode pointers that makes hash_ptr generate a lot of duplicates. That > really could use more investigation... Inode pointers have no entropy in the lower 9-10 bits because of their size, and being allocated from a slab they are all going to have the same set of values in the next 3-4 bits (i.e. offset into the slab page which is defined by sizeof(inode)). Pointers also have very similar upper bits, too, because they are all in kernel memory. hash_64 trys to fold all the entropy from the lower bits into into the upper bits and then takes the result from the upper bits. Hence if there is no entropy in either the lower or upper bits to start with, then the hash may not end up with much entropy in it at all... FWIW, see fs/inode.c::hash() to see how the fs code hashes inode numbers (called from insert_inode_hash()). It's very different because because inode numbers have the majority of their entropy in the lower bits and (usually) none in the upper bits... Cheers, Dave. -- Dave Chinner david@fromorbit.com