Re: [PATCH v7 07/11] arch/x86: enable task isolation functionality

[Chris Metcalf <cmetcalf@ezchip.com>]

On 10/01/2015 04:12 AM, Thomas Gleixner wrote:
> On Wed, 30 Sep 2015, Andy Lutomirski wrote:
>> On Wed, Sep 30, 2015 at 3:02 PM, Thomas Gleixner <tglx@linutronix.de> wrote:
>>> On Wed, 30 Sep 2015, Chris Metcalf wrote:
>>>> So for now, if a task-isolation thread sets up a timer,
>>>> they're screwed: so, don't do that.  And it's really not part of
>>>> the typical programming model for these kinds of userspace
>>>> drivers anyway, so it's pretty reasonable to forbid it.
>>> There is a difference between forbidding it and looping for 10 minutes
>>> in the kernel.
>> I don't even like forbidding it.  Setting timers seems like an
>> entirely reasonable thing for even highly RT or isolated programs to
>> do, although admittedly they can do it on a non-RT thread and then
>> kick the RT thread when they're ready.
>>
>> Heck, even without the TSC deadline timer, the kernel could, in
>> principle, support that use case by having whatever core is doing
>> housekeeping keep kicking the can forward until it's time to IPI the
>> isolated core because it needs to wake up.
> That's simple. Just arm the timer on the other core. It's not rocket
> science to do that.

This is a plausible direction to go for alarms requested when
task isolation is enabled.  But as Christoph said, it's almost
certainly a bad idea anyway.  Our customers are advised to
do this kind of stuff (what we call "control-plane" activity) in
a separate process on a housekeeping core, which communicates
with the nohz_full cores via shared memory.  On the nohz_full
side the threads just use polling and simple atomics for
locking.  (That's fun too, because you can't actually allow
those locks to get into contended state since that obliges the
unlocker to invoke futex_wake in the kernel, so we can't just
use pthread mutexes or other common implementations.)

-- 
Chris Metcalf, EZChip Semiconductor
http://www.ezchip.com