Re: [PATCH] only irq-safe atomic ops

[Stephen Lord <lord@sgi.com>]

Andrew Morton wrote:

>Stephen Lord wrote:
>
>>You do want to avoid a leak in one direction or the other, the os would
>>start to think it
>>had lots of dirty pages, but not be able to find them, or think there is
>>no shortage
>>when in fact there was.
>>
>
>Oh absolutely.  Even a one-page-per-hour leak would be catastrophic.
>
>But there is a problem.  If CPUA is always setting pages dirty,
>and CPUB is always setting them clean, CPUA's counter will eventually
>overflow, and CPUB's will underflow.  Drat. 
>
>One fix for this would be special-case over- and under-flow handling:
>
>	if (TestSetPageDirty(page)) {
>		preempt_disable();
>		nr_dirty_pages[smp_processor_id()]++;
>		if (nr_dirty_pages[smp_processor_id()] > 2,000,000) {
>			fit_it_up();
>		}
>		preempt_enable();
>	}
>
>	void fix_it_up()
>	{
>		spin_lock(&global_counter_lock);
>		global_counter += 1,000,000;
>		nr_dirty_pages[smp_processor_id()] -= 1,000,000;
>		spin_unlock(&global_counter_lock);
>	}
>
>	int approx_total_dirty_pages()
>	{
>		int ret;
>
>		spin_lock(&global_counter_lock);
>		ret = global_counter;
>		for (all cpus) {
>			ret += nr_dirty_pages[cpu];
>		}
>		spin_unlock(&global_counter_lock);
>		return ret;
>	}
>
>Or something like that.
>
>Then again, maybe the underflows and overflows don't matter, because the
>sum of all counters is always correct.   Unless there's a counter roll-over
>during the summation.  No.  Even then it's OK.
>
>hmmm.
>
>-
>

As I was plumbing in a sink ;-) the thought also occurred that you could 
have allocate
and free counters per cpu. The fix up code could do leveling between 
them. Are you
sure you only want to look at the actual value infrequently though? 
Every time you
put a page into delalloc state you need to do something similar to 
balance_dirty(),
if you only poll the value periodically then it could get way out of 
balance before
you noticed. It is very easy to dirty memory this way, cat /dev/zero > 
xxxx runs
very quickly.

I would almost say you need to get a 'reservation' of a delalloc page 
before you
grab the memory. There are extra memory costs associated with getting the
page out of this state, and if you do not hold threads back from 
dirtying pages
you can end up in a situation where you cannot clean up again.

Steve