Re: [PATCH v2 03/28] dcache: convert dentry_stat.nr_unused to per-cpu counters

[Glauber Costa <glommer@parallels.com>]

On 04/05/2013 05:15 AM, Dave Chinner wrote:
> On Thu, Apr 04, 2013 at 06:09:31PM -0700, Greg Thelen wrote:
>> On Fri, Mar 29 2013, Glauber Costa wrote:
>>
>>> From: Dave Chinner <dchinner@redhat.com>
>>>
>>> Before we split up the dcache_lru_lock, the unused dentry counter
>>> needs to be made independent of the global dcache_lru_lock. Convert
>>> it to per-cpu counters to do this.
>>>
>>> Signed-off-by: Dave Chinner <dchinner@redhat.com>
>>> Reviewed-by: Christoph Hellwig <hch@lst.de>
>>> ---
>>>  fs/dcache.c | 17 ++++++++++++++---
>>>  1 file changed, 14 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/fs/dcache.c b/fs/dcache.c
>>> index fbfae008..f1196f2 100644
>>> --- a/fs/dcache.c
>>> +++ b/fs/dcache.c
>>> @@ -118,6 +118,7 @@ struct dentry_stat_t dentry_stat = {
>>>  };
>>>  
>>>  static DEFINE_PER_CPU(unsigned int, nr_dentry);
>>> +static DEFINE_PER_CPU(unsigned int, nr_dentry_unused);
>>>  
>>>  #if defined(CONFIG_SYSCTL) && defined(CONFIG_PROC_FS)
>>>  static int get_nr_dentry(void)
>>> @@ -129,10 +130,20 @@ static int get_nr_dentry(void)
>>>  	return sum < 0 ? 0 : sum;
>>>  }
>>>  
>>> +static int get_nr_dentry_unused(void)
>>> +{
>>> +	int i;
>>> +	int sum = 0;
>>> +	for_each_possible_cpu(i)
>>> +		sum += per_cpu(nr_dentry_unused, i);
>>> +	return sum < 0 ? 0 : sum;
>>> +}
>>
>> Just checking...  If cpu x is removed, then its per cpu nr_dentry_unused
>> count survives so we don't leak nr_dentry_unused.  Right?  I see code in
>> percpu_counter_sum_positive() to explicitly handle this case and I want
>> to make sure we don't need it here.
> 
> DEFINE_PER_CPU() gives a variable per possible CPU, and we sum for
> all possible CPUs. Therefore online/offline CPUs just don't matter.
> 
> The percpu_counter code uses for_each_online_cpu(), and so it has to
> be aware of hotplug operations so taht it doesn't leak counts.
> 

It is an unsigned quantity, however. Can't we go negative if it becomes
unused in one cpu, but used in another?

Ex:

nr_unused/0: 0
nr_unused/1: 0

dentry goes to the LRU at cpu 1:
nr_unused/0: 0
nr_unused/1: 1

CPU 1 goes down:
nr_unused/0: 0

dentry goes out of the LRU at cpu 0:
nr_unused/0: 1 << 32.

That would easily be fixed by using a normal signed long, and is in fact
what the percpu code does in its internal operations.

Any reason not to do it? Something I am not seeing?