Re: [PATCH][resubmit] x86: enable preemption in delay

Gregory Haskins
2008-06-18T12:25:55Z

>>> On Wed, Jun 18, 2008 at 8:16 AM, in message
<1213791416.16944.222.camel@twins>, Peter Zijlstra <peterz@infradead.org>
wrote:
> On Wed, 2008-06-18 at 06:08 -0600, Gregory Haskins wrote:
>> >>> On Wed, Jun 18, 2008 at 3:55 AM, in message <20080618075518.GD4135@elte.hu>,
>> Ingo Molnar <mingo@elte.hu> wrote:
>>
>> > * Marin Mitov <mitov@issp.bas.bg> wrote:
>> >
>> >> Why not something like that (do keep in mind I am not an expert :-):
>> >>
>> >> static void delay_tsc(unsigned long loops)
>> >> {
>> >> get and store the mask of allowed cpus;
>> >> /* prevent the migration */
>> >> set the mask of allowed cpus to the current cpu only;
>> >> /* is it possible? could it be guaranteed? */
>> >> loop for the delay;
>> >> restore the old mask of allowed cpus;
>> >> }
>> >>
>> >> You have got the idea. Could it be realized? Is it more expensive than
>> >> the current realization? So, comments, please.
>> >
>> > hm, changing/saving/restorig cpus_allowed is really considered a 'heavy'
>> > operation compared to preempt_disable(). On a 4096 CPUs box cpus_allowed
>> > is 4096 bits which is half a kilobyte ...
>> >
>> > preempt_disable()/enable() on the other hand only touches a single
>> > variable, (thread_info->preempt_count which is an u32)
>> >
>> > Ingo
>>
>> FWIW: I had submitted some "migration disable" patches a while back
>> that would solve this without the cpus_allowed manipulations described
>> here. Its more expensive than a preempt-disable (but its
>> preemptible), yet its way cheaper (and more correct / less racy) than
>> chaning cpus_allowed. I could resubmit if there was any interest,
>> though I think Ingo said he didnt like the concept on the first pass.
>> Anyway, FYI.
>
> (please teach your mailer to wrap text)

Sorry...I know its really annoying, but I have no control over it in groupwise :(

Its a server side / MTA setting. Go figure. I will try to wrap manually.

>
> Yeah - migrate_disable() has been proposed several times. The reason I
> don't like it is that is creates scheduling artefacts like latencies by
> not being able to load-balance (and thereby complicates all that, and
> you know we don't need more complication there).

True, and good point. But this concept would certainly be useful to avoid
the heavyweight (w.r.t. latency) preempt-disable() in quite a few different
areas, so if we can make it work with reasonable visibility, it might be nice
to have.

>
> Things like preempt latency and irq off latency are rather easy to
> notice and debug in general. migrate_disable() would be fully
> preemptable/schedulable which makes it much much harder to instrument or
> even detect we have an issue. Which in turn makes it much harder to find
> abuse.

I wonder if we can come up with any creative instrumentation to get coverage
in this case. I will think about it and add it to the migration-disable queue I
have to be submitted together (unless Ingo et. al. feel strongly that it will
never be accepted even with good instrumentation...then I will not waste
any effort on it).

Regards,
-Greg

>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html