Re: [PATCH][resubmit] x86: enable preemption in delay

["Gregory Haskins" <ghaskins@novell.com>]

>>> On Wed, Jun 18, 2008 at  8:16 AM, in message
<1213791416.16944.222.camel@twins>, Peter Zijlstra <peterz@infradead.org>
wrote: 
> On Wed, 2008-06-18 at 06:08 -0600, Gregory Haskins wrote:
>> >>> On Wed, Jun 18, 2008 at  3:55 AM, in message <20080618075518.GD4135@elte.hu>,
>> Ingo Molnar <mingo@elte.hu> wrote: 
>> 
>> > * Marin Mitov <mitov@issp.bas.bg> wrote:
>> > 
>> >> Why not something like that (do keep in mind I am not an expert :-):
>> >> 
>> >> static void delay_tsc(unsigned long loops)
>> >> {
>> >> 	get and store the mask of allowed cpus;
>> >> 	/*     prevent the migration   */
>> >> 	set the mask of allowed cpus to the current cpu only;
>> >> 	/*     is it possible? could it be guaranteed?    */
>> >> 	loop for the delay;
>> >> 	restore the old mask of allowed cpus;
>> >> }
>> >> 
>> >> You have got the idea. Could it be realized? Is it more expensive than 
>> >> the current realization? So, comments, please.
>> > 
>> > hm, changing/saving/restorig cpus_allowed is really considered a 'heavy' 
>> > operation compared to preempt_disable(). On a 4096 CPUs box cpus_allowed 
>> > is 4096 bits which is half a kilobyte ...
>> > 
>> > preempt_disable()/enable() on the other hand only touches a single 
>> > variable, (thread_info->preempt_count which is an u32)
>> > 
>> > 	Ingo
>> 
>> FWIW:  I had submitted some "migration disable" patches a while back
>> that would solve this without the cpus_allowed manipulations described
>> here.  Its more expensive than a preempt-disable (but its
>> preemptible), yet its way cheaper (and more correct / less racy) than
>> chaning cpus_allowed.  I could resubmit if there was any interest,
>> though I think Ingo said he didnt like the concept on the first pass.
>> Anyway, FYI.
> 
> (please teach your mailer to wrap text)

Sorry...I know its really annoying, but I have no control over it in groupwise :(

Its a server side / MTA setting.  Go figure.  I will try to wrap manually.

> 
> Yeah - migrate_disable() has been proposed several times. The reason I
> don't like it is that is creates scheduling artefacts like latencies by
> not being able to load-balance (and thereby complicates all that, and
> you know we don't need more complication there).

True, and good point.  But this concept would certainly be useful to avoid 
the heavyweight (w.r.t. latency) preempt-disable() in quite a few different
areas, so if we can make it work with reasonable visibility, it might be nice
to have.

> 
> Things like preempt latency and irq off latency are rather easy to
> notice and debug in general. migrate_disable() would be fully
> preemptable/schedulable which makes it much much harder to instrument or
> even detect we have an issue. Which in turn makes it much harder to find
> abuse.

I wonder if we can come up with any creative instrumentation to get coverage
in this case.  I will think about it and add it to the migration-disable queue I
have to be submitted together (unless Ingo et. al. feel strongly that it will
never be accepted even with good instrumentation...then I will not waste
any effort on it).

Regards,
-Greg

> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html