[Xenomai] I-pipe's determinism in handling hardware interrupts when GIC implements "Security Extensions"

[Xenomai] I-pipe's determinism in handling hardware interrupts when GIC implements "Security Extensions"

[Hongfei Cheng]

Hi All,

I am wondering (aloud) if anyone has experience running I-pipe on
ARMv7 platform which supports ARM's Security Extensions (TrustZone).

I believe, starting in ARMv7-A/R architecture, ARM added the
(optional) Security Extensions (TrustZone) to provide hardware
security features. Some of the ARM SoCs on the supported list by
Xenomai, such as Freescale QorIQ LS1 and Xilinx Zynq, appear to have
implemented such extensions.

On an ARM SoC platform implementing the Security Extensions -
1). How does the I-pipe/Adeos deal with both the interrupt sources in
"Normal world" (non-secure interrupt) and "Secure world" (secure
interrupt)?
2). What would happen if a Xenomai real-time thread is taking on a
non-secure interrupt while a secure interrupt with higher priviledge
arrives which must be routed to a Linux thread?
3). Since a secure interrupt can take an unbounded amount of time to
complete its task, such as downloading firmware, will it break
I-pipe's determinism in serving non-secure interrupt on behalf of
Xenomai real-time threads?

Thanks,
Hongfei

Re: [Xenomai] I-pipe's determinism in handling hardware interrupts when GIC implements "Security Extensions"

[Gilles Chanteperdrix]

On Thu, Aug 13, 2015 at 11:51:59AM -0700, Hongfei Cheng wrote:
> Hi All,
> 
> I am wondering (aloud) if anyone has experience running I-pipe on
> ARMv7 platform which supports ARM's Security Extensions (TrustZone).

I think we have that on some omap3/omap4 boards.

> 
> I believe, starting in ARMv7-A/R architecture, ARM added the
> (optional) Security Extensions (TrustZone) to provide hardware
> security features. Some of the ARM SoCs on the supported list by
> Xenomai, such as Freescale QorIQ LS1 and Xilinx Zynq, appear to have
> implemented such extensions.
> 
> On an ARM SoC platform implementing the Security Extensions -
> 1). How does the I-pipe/Adeos deal with both the interrupt sources in
> "Normal world" (non-secure interrupt) and "Secure world" (secure
> interrupt)?

Correct me if I am wrong, I do not know which is which, but I
believe the Linux code only lives in one of the two worlds. The
other world is the one of a monitor, or whatever you call it, and
inaccessible to Linux (that is the aim of "securing the processor").
So, by definition I-pipe being a modification of Linux and not of
the monitor can not access the privileged operations.

> 2). What would happen if a Xenomai real-time thread is taking on a
> non-secure interrupt while a secure interrupt with higher priviledge
> arrives which must be routed to a Linux thread?
> 3). Since a secure interrupt can take an unbounded amount of time to
> complete its task, such as downloading firmware, will it break
> I-pipe's determinism in serving non-secure interrupt on behalf of
> Xenomai real-time threads?

I am not sure this case can exist. Are not these privileged
interrupt only handled by the monitor? Anyway, if when this
interrupt is masked at the interrupt controller level, non
privileged interrupts can be handled, then there is no problem. If
such an interrupt has to be handled before any more interrupt can be
taken, even if masked, then you have a problem. If, like I believe,
such a "privileged interrupt" is handled by the monitor behind
Linux/I-pipe's back, then yes, it will break determinism.

-- 
					    Gilles.
https://click-hack.org

Re: [Xenomai] I-pipe's determinism in handling hardware interrupts when GIC implements "Security Extensions"

[Hongfei Cheng]

On Thu, Aug 13, 2015 at 12:11 PM, Gilles Chanteperdrix
<gilles.chanteperdrix@xenomai.org> wrote:
>> On an ARM SoC platform implementing the Security Extensions -
>> 1). How does the I-pipe/Adeos deal with both the interrupt sources in
>> "Normal world" (non-secure interrupt) and "Secure world" (secure
>> interrupt)?
>
> Correct me if I am wrong, I do not know which is which, but I
> believe the Linux code only lives in one of the two worlds. The
> other world is the one of a monitor, or whatever you call it, and
> inaccessible to Linux (that is the aim of "securing the processor").
> So, by definition I-pipe being a modification of Linux and not of
> the monitor can not access the privileged operations.

Based on my limited understanding of the TrustZone, both the hardware
and software resources are partitioned into two worlds: the "Secure
world" for the security subsystem, and the "Normal world" for
everything else.

Yes, the "secure monitor" lives in the secure world. However, for
systems without dedicated secure world OS, the "secure apps" run in
the normal world using the API provided by the secure world. So, in a
sense, Linux system interacts with the secure world directly.
Furthermore, software running on Linux can trigger the secure monitor
by executing the Secure Monitor Call (SMC) instruction. I've seen the
effect of such call on an ARMv8 platform when when running the Xenomai
latency test. I can share the captured ipipe trace if you or anyone
else is interested.

>
>> 2). What would happen if a Xenomai real-time thread is taking on a
>> non-secure interrupt while a secure interrupt with higher priviledge
>> arrives which must be routed to a Linux thread?
>> 3). Since a secure interrupt can take an unbounded amount of time to
>> complete its task, such as downloading firmware, will it break
>> I-pipe's determinism in serving non-secure interrupt on behalf of
>> Xenomai real-time threads?
>
> I am not sure this case can exist. Are not these privileged
> interrupt only handled by the monitor?

For a specific processor core where the secure world executes on, I
believe all interrupts will go through the secure monitor which will
then route secure interrupts and non-secure interrupts to their
respective worlds.

> Anyway, if when this
> interrupt is masked at the interrupt controller level, non
> privileged interrupts can be handled, then there is no problem. If
> such an interrupt has to be handled before any more interrupt can be
> taken, even if masked, then you have a problem.

When TrustZone-aware GIC masks interrupts targeting the secure world,
non-privileged interrupts must wait until the secure/privileged
interrupts are cleared. Yes, looks like we do have a problem.


> If, like I believe,
> such a "privileged interrupt" is handled by the monitor behind
> Linux/I-pipe's back, then yes, it will break determinism.

Do you have any suggestion as to how I-pipe can be improved to work
with ARM's multiple exception levels in order to ensure determinism on
ARMv7 (and ARMv8)?

Here is a link to a white paper published by ARM:
http://infocenter.arm.com/help/topic/com.arm.doc.prdc013242/Critical_Interrupt_Prioritization_PRD04_PRDC_013242_1_0.pdf

Thanks!
Hongfei

Re: [Xenomai] I-pipe's determinism in handling hardware interrupts when GIC implements "Security Extensions"

[Jan Kiszka]

On 2015-08-14 22:38, Hongfei Cheng wrote:
>> If, like I believe,
>> such a "privileged interrupt" is handled by the monitor behind
>> Linux/I-pipe's back, then yes, it will break determinism.
> 
> Do you have any suggestion as to how I-pipe can be improved to work
> with ARM's multiple exception levels in order to ensure determinism on
> ARMv7 (and ARMv8)?

Conceptually, this is fairly similar to the SMM on x86. But, in contrast
to that arch, we tend to have the firmware sources on ARM.

You can check if you actually need the secure monitor on our platform
(it's most often used for PSCI) and disable it (may be trickier on ARMv8
where PSCI is mandatory IIRC) or patch it or the kernel to avoid the
problematic service calls (specifically CPU power control via PSCI).

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

Re: [Xenomai] I-pipe's determinism in handling hardware interrupts when GIC implements "Security Extensions"

[Hongfei Cheng]

On Fri, Aug 14, 2015 at 10:46 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
> On 2015-08-14 22:38, Hongfei Cheng wrote:
>>> If, like I believe,
>>> such a "privileged interrupt" is handled by the monitor behind
>>> Linux/I-pipe's back, then yes, it will break determinism.
>>
>> Do you have any suggestion as to how I-pipe can be improved to work
>> with ARM's multiple exception levels in order to ensure determinism on
>> ARMv7 (and ARMv8)?
>
> Conceptually, this is fairly similar to the SMM on x86. But, in contrast
> to that arch, we tend to have the firmware sources on ARM.
>
> You can check if you actually need the secure monitor on our platform
> (it's most often used for PSCI) and disable it (may be trickier on ARMv8
> where PSCI is mandatory IIRC) or patch it or the kernel to avoid the
> problematic service calls (specifically CPU power control via PSCI).
>
> Jan

Thanks for the suggestion, Jan.

You're right -- PSCI is an integral part of ARMv8 Linux kernel.
Besides, secure apps other than PSCI-based CPU power management rely
on the secure monitor, such as firmware verification and user
authentication.

I don't know the details of x86's SMM. Based on my quick reading, it
does appear to have similar effect as ARM in terms of privileged
interrupts. On ARMv7/8, such privileged interrupts can be routed to
secure applications in either secure OS or non-secure OS.

Since SMM has been in the x86 arch for a very long time, how do you
manage to maintain I-pipe's determinism on SMM-enable x86 platform? Is
this one of the reasons that I-pipe requires disabling power
management features (cpu-freq scaling, cpu-idle, hot-plug)?

Hongfei