Re: [PATCH v7 05/11] task_isolation: add debug boot flag

[Chris Metcalf <cmetcalf@ezchip.com>]

On 10/5/2015 1:07 PM, Luiz Capitulino wrote:
> On Mon, 28 Sep 2015 11:17:20 -0400
> Chris Metcalf <cmetcalf@ezchip.com> wrote:
>
>> The new "task_isolation_debug" flag simplifies debugging
>> of TASK_ISOLATION kernels when processes are running in
>> PR_TASK_ISOLATION_ENABLE mode.  Such processes should get no
>> interrupts from the kernel, and if they do, when this boot flag is
>> specified a kernel stack dump on the console is generated.
>>
>> It's possible to use ftrace to simply detect whether a task_isolation
>> core has unexpectedly entered the kernel.  But what this boot flag
>> does is allow the kernel to provide better diagnostics, e.g. by
>> reporting in the IPI-generating code what remote core and context
>> is preparing to deliver an interrupt to a task_isolation core.
>>
>> It may be worth considering other ways to generate useful debugging
>> output rather than console spew, but for now that is simple and direct.
> Honest question: does any of the task_isolation_debug() calls added
> by this patch take care of the case where vmstat_shepherd() may
> schedule vmstat_update() to run because a TASK_ISOLATION process is
> changing memory stats?

The task_isolation_debug() calls don't "take care of" any cases - they are
really just there to generate console dumps when the kernel unexpectedly
interrupts a task_isolated task.

The idea with vmstat is that before a task_isolated task returns to
userspace, it quiesces the vmstat thread (does a final sweep to collect
the stats and turns off the scheduled work item).  As a result, the vmstat
shepherd won't run while the task is in userspace.  When and if it returns
to the kernel, it will again sweep up the stats before returning to userspace.

The usual shepherd mechanism on a housekeeping core might notice
that the task had entered the kernel and started changing stats, and
might then asynchronously restart the scheduled work, but it should be
quiesced again regardless on the way back out to userspace.

> If that's not taken care of yet, should we? I just don't know if we
> should call task_isolation_exception() or task_isolation_debug().

task_isolation_exception() is called when an exception (page fault or
similar) is generated synchronously by the running task and we want
to make sure to notify the task with a signal if it has set up STRICT mode
to indicate that it is not planning to enter the kernel.

> In the case of the latter, wouldn't it be interesting to add it to
> __queue_work() then?

Well, queuing remote work involves sending an IPI, and we already tag
both the SMP send side AND the client side IRQ side with a task_isolation_debug(),
so I expect in practice it would be detected.

-- 
Chris Metcalf, EZChip Semiconductor
http://www.ezchip.com