Re: [RFC] Thread Migration Preemption

[Nick Piggin <nickpiggin@yahoo.com.au>]

Mathieu Desnoyers wrote:
> * Matt Mackall (mpm@selenic.com) wrote:
> 
>>On Wed, Jul 11, 2007 at 10:02:23AM +1000, Nick Piggin wrote:
>>
>>>>I like this patch a lot. Even if we don't add the underlying mechanism
>>>>right now, adding migration_disable as an alias for preempt_disable
>>>>will much better document quite a number of the users.
>>>
>>>I'd have no problem with that, and it might make it easier in future to
>>>justify a more complex scheme.
>>
>>What do you think, Mathieu?
>>
> 
> 
> That's an excellent idea. It should probably come with some
> documentation explaining what difference should be expected between
> preempt_disable and migrate_disable in the future so that people can
> choose the right alternative for their code.
> 
> 
>>Also, small nit: it ought to be migrate_disable to match the form of
>>preempt_disable.
> 
> 
> Yes, that's fine with me. :)
> 
> I though a little more about this whole concept of migrate_disable, and
> I think it could be brought further. One of the main problems with per
> cpu variables is that this concept cannot be ported to this grey place
> called "user-space" because preemption cannot be disabled. The
> equivalent (kind of) is currently per thread variables, but it can
> consume a lot of space if many threads are running.
> 
> However, it could be possible, if we implement a vsyscall based
> migration preemption counter accessible with read/write access from
> user-space, to tie a thread to a CPU during a user-space critical path.
> If we combine this with local atomic operations done in user-space, we
> could have highly scalable access to per cpu data structures reentrant
> with respect to signal handlers.

That's all well and good, but for most non-trivial stuff, you
have to disable preemption as well which you cannot do in
userspace. Which I suspect is why there is not a great deal
that can use it in kernelspace either.

So it will remain to be seen what kind of per-cpu data structures
you can access in a highly scalable way, and how big the niche is
between real per-thread data structures and real locking. I'm
skeptical :)

-- 
SUSE Labs, Novell Inc.