Re: [PATCH RFC] hwspinlock: Don't take software spinlock before hwspinlock

[Lina Iyer <lina.iyer@linaro.org>]

On Sat, May 23 2015 at 01:36 -0600, Ohad Ben-Cohen wrote:
>Hi Lina,
>
>On Mon, May 18, 2015 at 6:03 PM, Lina Iyer <lina.iyer@linaro.org> wrote:
>> The lock in question is used differently than traditional locks across
>> processors. This lock helps synchronizes context transition from
>> non-secure to secure on the same processor.
>>
>> The usecase, goes like this. In cpuidle, any core can be the last core
>> to power down. The last man also holds the responsibility of shutting
>> down shared resources like caches etc. The way the power down of a core
>> works is, there are some high level decisions made in Linux and these
>> decisions (like to flush and invalidate caches) etc gets transferred
>> over to the the secure layer. The secure layer executes the ARM WFI that
>> powers down the cpu, but uses these decisions passed into to determine
>> if the cache needs to be invalidated upon wakeup etc.
>>
>> There is a possible race condition between what Linux thinks is the last
>> core, vs what secure layer thinks is the last core. Lets say, two cores
>> c0, c1 are going down. c1 is the second last core to go down from Linux
>> as such, will not carry information about shared resources when making
>> the SCM call. c1  made the SCM call, but is stuck handling some FIQs. In
>> the meanwhile c0, goes idle and since its the last core in Linux,
>> figures out the state of the shared resources. c0 calls into SCM, and
>> ends up powering down earlier than c1. Per secure layer, the last core
>> to go down is c1 and the votes of the shared resources are considered
>> from that core. Things like cache invalidation without flush may happen
>> as a result of this inconsistency of last man view point.
>>
>> The way we have solved it, Linux acquires a hw spinlock for each core,
>> when calling into SCM and the secure monitor releases the spinlock. At
>> any given time, only one core can switch the context from Linux to
>> secure for power down operations. This guarantees the last man is
>> synchronized between both Linux and secure. Another core may be spinning
>> waiting for hw mutex, but they all happen serialized. This mutex is held
>> in an irq disable context in cpuidle.
>>
>> There may be another processor spining to wait on hw mutex, but there
>> isnt much to do otherwise, because the only operation at this time while
>> holding the lock is to call into SCM and that would unlock the mutex.
>
>Just to make sure I understand, is this how your scenario is solved?
>
>- c1 goes down
>- c0 goes down, carries information about shared resources
>- c1 takes HWLOCK and calls into SCM, stuck handling FIQs
>- c0 wants to call into SCM but is waiting spinning on HWLOCK
>- c1 completes handling FIQs, goes idle, HWLOCK is released by secure monitor
>- c0 takes HWLOCK, calls into SCM, shared resources handled correctly,
>
>HWLOCK in this example is a single shared hwspinlock accessible by c0,
>c1 and secure monitor.
>
That is correct.

-- Lina