Re: [GIT PULL] Re: REGRESSION: Performance regressions from switching anon_vma->lock to mutex

[Frederic Weisbecker <fweisbec@gmail.com>]

On Fri, Jun 17, 2011 at 01:02:47AM +0200, Ingo Molnar wrote:
> 
> * Frederic Weisbecker <fweisbec@gmail.com> wrote:
> 
> > On Thu, Jun 16, 2011 at 10:25:50PM +0200, Ingo Molnar wrote:
> > > 
> > > * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> > > 
> > > > > The funny thing about this workload is that context-switches are 
> > > > > really a fastpath here and we are using anonymous IRQ-triggered 
> > > > > softirqs embedded in random task contexts as a workaround for 
> > > > > that.
> > > > 
> > > > The other thing that the IRQ-triggered softirqs do is to get the 
> > > > callbacks invoked in cases where a CPU-bound user thread is never 
> > > > context switching.
> > > 
> > > Yeah - but this workload didnt have that.
> > > 
> > > > Of course, one alternative might be to set_need_resched() to force 
> > > > entry into the scheduler as needed.
> > > 
> > > No need for that: we can just do the callback not in softirq but in 
> > > regular syscall context in that case, in the return-to-userspace 
> > > notifier. (see TIF_USER_RETURN_NOTIFY and the USER_RETURN_NOTIFIER 
> > > facility)
> > > 
> > > Abusing a facility like setting need_resched artificially will 
> > > generally cause trouble.
> > 
> > If the task enqueued callbacks in the kernel, thus started a new 
> > grace period, it might return to userspace before every CPUs have 
> > completed that grace period, and you need that full completion to 
> > happen before invoking the callbacks.
> > 
> > I think you need to keep the tick in such case because you can't 
> > count on the other CPUs to handle that completion as they may be 
> > all idle.
> > 
> > So when you resume to userspace and you started a GP, either you 
> > find another CPU to handle the GP completion and callbacks 
> > executions, or you keep the tick until you are done.
> 
> We'll have a scheduler tick in any case, which will act as a 
> worst-case RCU tick.
> 
> My main point is that we need to check whether this solution improves 
> performance over the current softirq code. I think there's a real 
> chance that it improves things like VFS workloads, because it 
> provides (much!) lower grace period latencies hence provides 
> fundamentally better cache locality.
> 
> If a workload pays the cost of frequent scheduling then it might as 
> well use a beneficial side-effect of that scheduling: high-freq grace 
> periods ...
> 
> If it improves performance we can figure out all the loose ends. If 
> it doesnt then the loose ends are not worth worrying about.

Yeah I see your point, seems worth trying.