Secure resources in device trees

Secure resources in device trees

[Greg Bellows]

The addition of ARM security extension (TrustZone) support to QEMU has
exposed the issue of how secure resources are communicated to secure
software responsible for booting the HLOS.  The natural choice for
communicating these details is the device tree.

In the case of real hardware, the device tree supplied to the HLOS
only needs to describe non-secure resources as secure software can
rely on static knowledge about the hardware.  This also holds true for
QEMU machines modeled after actual fixed hardware configurations.
However, this is not the case with QEMU's virtual machine models, such
as machvirt, where the hardware configuration can change over time.
In this case, secure software is dependent on QEMU's dynamically
constructed device tree to describe the hardware, making it impossible
for secure software to know ahead of time what is secure, non-secure,
or both.

Two possible approaches for handling this particular case are:

1) Create two device tree blobs; one describing the non-secure
configuration and the other the full configuration.  This would allow
secure software to see the full hardware picture including secure
resources while the non-secure world would only see the non-secure
device tree configuration.  The QEMU virt machine would be responsible
for producing the device tree blobs.

The drawbacks to this approach are:
* There are 2 device trees to manage
* The two DTBs will typically be almost identical.
* Not possible to identify whether a device is shared or not between
the secure and non-secure worlds.
* Identifying device available only to the secure world require
cumbersome comparison of the two device trees.
* A mechanism would be needed to pass an additional device tree.

2) Modify the standard device tree blob to include annotations or
modifications to describe which resources are secure or not.  In this
case, secure software would use the single device tree to identify the
secure resources.  The added information could be used by secure
software to trim the device tree before passing it.  Alternatively,
the information could be passed on to non-secure software with the
expectation that it would honor the device security.   It would be
crucial that any data added to the device tree adhere to existing
conventions or expectations.

The drawbacks to this approach are:
* Additional secure state details. potentially unrecognized by current
consumers,  would need to be added to the device tree.
* Unless a mechanism already exists, the new secure property runs the
risk of breaking backwards compatibility.
* Secure firmware is responsible for understanding and possibly
filtering the secure device tree data.

Feedback welcomed.

Greg
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Secure resources in device trees

[Rob Herring]

On Tue, Jan 20, 2015 at 1:15 PM, Greg Bellows <greg.bellows-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> wrote:
> The addition of ARM security extension (TrustZone) support to QEMU has
> exposed the issue of how secure resources are communicated to secure
> software responsible for booting the HLOS.  The natural choice for
> communicating these details is the device tree.

You could also have a secure OS use DT and the HLOS use something else
(ACPI). Or the secure OS and non-secure firmware (UEFI) use DT and the
OS uses ACPI.

> In the case of real hardware, the device tree supplied to the HLOS
> only needs to describe non-secure resources as secure software can
> rely on static knowledge about the hardware.  This also holds true for
> QEMU machines modeled after actual fixed hardware configurations.
> However, this is not the case with QEMU's virtual machine models, such
> as machvirt, where the hardware configuration can change over time.
> In this case, secure software is dependent on QEMU's dynamically
> constructed device tree to describe the hardware, making it impossible
> for secure software to know ahead of time what is secure, non-secure,
> or both.

For purposes of this discussion, what works better for QEMU is irrelevant IMO.

> Two possible approaches for handling this particular case are:
>
> 1) Create two device tree blobs; one describing the non-secure
> configuration and the other the full configuration.  This would allow
> secure software to see the full hardware picture including secure
> resources while the non-secure world would only see the non-secure
> device tree configuration.  The QEMU virt machine would be responsible
> for producing the device tree blobs.

I've thought about this some in the past and leaned toward this
direction mainly because I'd expect you are partitioning most nodes to
one side. You could do some crazy partitioning with secure and
non-secure world. It was suggested on highbank to use the secure bit
as a 33rd address bit to get 8GB of address space for example. You
could have entirely different view of the system.

> The drawbacks to this approach are:
> * There are 2 device trees to manage

You already have 2 bootloaders and 2 OS's to manage.

> * The two DTBs will typically be almost identical.

I don't really agree. If the secure dtb has all non-secure peripherals
too, then yes. But if you only include peripherals allocated to secure
world and secure view of peripherals, then I don't think there would
be much overlap. You pretty much have to statically allocate each
peripheral to one side or the other.

> * Not possible to identify whether a device is shared or not between
> the secure and non-secure worlds.

Typically, sharing requires a peripheral to be designed to be shared
like PL330 or MMU-400. I have seen some h/w with locking registers so
2 different cores/OSs can share an i2c bus. You could do something
like that for Trustzone as well I suppose. That's not really secure,
but allows shared access. I think it is generally a limited number of
peripherals which are shared.

> * Identifying device available only to the secure world require
> cumbersome comparison of the two device trees.

But that is pretty static. It doesn't seem like a big deal to me.

> * A mechanism would be needed to pass an additional device tree.
>
> 2) Modify the standard device tree blob to include annotations or
> modifications to describe which resources are secure or not.  In this
> case, secure software would use the single device tree to identify the
> secure resources.  The added information could be used by secure
> software to trim the device tree before passing it.  Alternatively,
> the information could be passed on to non-secure software with the
> expectation that it would honor the device security.   It would be
> crucial that any data added to the device tree adhere to existing
> conventions or expectations.

You could also do this w/o dts changes. You could start with a full
description and the knowledge of what to filter out resides in the
secure OS or bootloader. It depends where you want to put the
partitioning decisions.

I think there's 2 main cases to consider. Full nodes that are assigned
to one side and Trustzone aware nodes. The first could potentially
re-use status property adding just a "secure" setting or we just add a
new "secure" property. This should probably be inherited by child
nodes. The secure firmware would then need to remove or set status to
disabled for those nodes. For the latter case, I don't think we can
avoid having the firmware having specific knowledge of the bindings.
We'll have to filter out the secure only details as I think so far we
have only been creating bindings which describe the non-secure view. I
don't think this is a big deal as the peripheral set would be pretty
limited AFAIK.

Rob

> The drawbacks to this approach are:
> * Additional secure state details. potentially unrecognized by current
> consumers,  would need to be added to the device tree.
> * Unless a mechanism already exists, the new secure property runs the
> risk of breaking backwards compatibility.
> * Secure firmware is responsible for understanding and possibly
> filtering the secure device tree data.
>
> Feedback welcomed.
>
> Greg
> --
> To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Secure resources in device trees

[Grant Likely]

On Wed, Jan 21, 2015 at 4:21 PM, Rob Herring <robherring2-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> On Tue, Jan 20, 2015 at 1:15 PM, Greg Bellows <greg.bellows-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> wrote:
>> 2) Modify the standard device tree blob to include annotations or
>> modifications to describe which resources are secure or not.  In this
>> case, secure software would use the single device tree to identify the
>> secure resources.  The added information could be used by secure
>> software to trim the device tree before passing it.  Alternatively,
>> the information could be passed on to non-secure software with the
>> expectation that it would honor the device security.   It would be
>> crucial that any data added to the device tree adhere to existing
>> conventions or expectations.
>
> You could also do this w/o dts changes. You could start with a full
> description and the knowledge of what to filter out resides in the
> secure OS or bootloader. It depends where you want to put the
> partitioning decisions.
>
> I think there's 2 main cases to consider. Full nodes that are assigned
> to one side and Trustzone aware nodes. The first could potentially
> re-use status property adding just a "secure" setting or we just add a
> new "secure" property. This should probably be inherited by child
> nodes. The secure firmware would then need to remove or set status to
> disabled for those nodes. For the latter case, I don't think we can
> avoid having the firmware having specific knowledge of the bindings.
> We'll have to filter out the secure only details as I think so far we
> have only been creating bindings which describe the non-secure view. I
> don't think this is a big deal as the peripheral set would be pretty
> limited AFAIK.

This is close to what Greg and I discussed in a private thread. It
would be just fine to use something like status="secure-okay" for the
nodes that are secure world only. Secure world code would need to
understand the new status value, but the existing kernel will do the
correct thing. Only nodes with status="okay" or ="ok" will be used.
There is no need to modify the tree at all in this scenario.

g.
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Secure resources in device trees

[Mark Rutland]

On Tue, Jan 20, 2015 at 07:15:51PM +0000, Greg Bellows wrote:
> The addition of ARM security extension (TrustZone) support to QEMU has
> exposed the issue of how secure resources are communicated to secure
> software responsible for booting the HLOS.  The natural choice for
> communicating these details is the device tree.
> 
> In the case of real hardware, the device tree supplied to the HLOS
> only needs to describe non-secure resources as secure software can
> rely on static knowledge about the hardware.  This also holds true for
> QEMU machines modeled after actual fixed hardware configurations.
> However, this is not the case with QEMU's virtual machine models, such
> as machvirt, where the hardware configuration can change over time.

When you say this changes over time, I assume you mean for the lifetime
of the project, as opposed to the lifetime of a running instance? i.e.
non-probeable devices are not dynamically injected to a running system.

> In this case, secure software is dependent on QEMU's dynamically
> constructed device tree to describe the hardware, making it impossible
> for secure software to know ahead of time what is secure, non-secure,
> or both.
> 
> Two possible approaches for handling this particular case are:
> 
> 1) Create two device tree blobs; one describing the non-secure
> configuration and the other the full configuration.  This would allow
> secure software to see the full hardware picture including secure
> resources while the non-secure world would only see the non-secure
> device tree configuration.  The QEMU virt machine would be responsible
> for producing the device tree blobs.
> 
> The drawbacks to this approach are:
> * There are 2 device trees to manage
> * The two DTBs will typically be almost identical.
> * Not possible to identify whether a device is shared or not between
> the secure and non-secure worlds.
> * Identifying device available only to the secure world require
> cumbersome comparison of the two device trees.

I'm not sure I follow why such identification is necessary?

> * A mechanism would be needed to pass an additional device tree.
> 
> 2) Modify the standard device tree blob to include annotations or
> modifications to describe which resources are secure or not.  In this
> case, secure software would use the single device tree to identify the
> secure resources.  The added information could be used by secure
> software to trim the device tree before passing it.  Alternatively,
> the information could be passed on to non-secure software with the
> expectation that it would honor the device security.   It would be
> crucial that any data added to the device tree adhere to existing
> conventions or expectations.

This approach assumes assumes that the secure and non-secure physical
address spaces are identical bar some portions being masked out on the
non-secure side. This is not necessarily the case.

Architecturally the secure and non-secure physical address spaces are
entiorely separate, and do not necessarily mirror each other.

It's entirely valid for some devices/RAM to only exist in one of the
address spaces (we typically see secure-only devices, but
non-secure-only devices are also possible). It's also entirely valid for
the same device to be mapped at different addresses in each address
space (e.g. the same UART could be mapped at both S:0xffff0000 and also
at NS:0xcccc0000 and nowhere else in either address space).

So approach (2) does not fit the architecture generally. From what I
recall of previous discussions, we eventually figured out that you
either need separate trees or a higher level container to address the
secure vs nonsecure split.

Thanks,
Mark.
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Secure resources in device trees

[Greg Bellows]

Thanks Rob, comments inline.

On Wed, Jan 21, 2015 at 10:21 AM, Rob Herring <robherring2-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> On Tue, Jan 20, 2015 at 1:15 PM, Greg Bellows <greg.bellows-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> wrote:
>> The addition of ARM security extension (TrustZone) support to QEMU has
>> exposed the issue of how secure resources are communicated to secure
>> software responsible for booting the HLOS.  The natural choice for
>> communicating these details is the device tree.
>
> You could also have a secure OS use DT and the HLOS use something else
> (ACPI). Or the secure OS and non-secure firmware (UEFI) use DT and the
> OS uses ACPI.
>
>> In the case of real hardware, the device tree supplied to the HLOS
>> only needs to describe non-secure resources as secure software can
>> rely on static knowledge about the hardware.  This also holds true for
>> QEMU machines modeled after actual fixed hardware configurations.
>> However, this is not the case with QEMU's virtual machine models, such
>> as machvirt, where the hardware configuration can change over time.
>> In this case, secure software is dependent on QEMU's dynamically
>> constructed device tree to describe the hardware, making it impossible
>> for secure software to know ahead of time what is secure, non-secure,
>> or both.
>
> For purposes of this discussion, what works better for QEMU is irrelevant IMO.
>

We just want to make sure that QEMU embraces the expected mechanism
for handling such a scenario.

>> Two possible approaches for handling this particular case are:
>>
>> 1) Create two device tree blobs; one describing the non-secure
>> configuration and the other the full configuration.  This would allow
>> secure software to see the full hardware picture including secure
>> resources while the non-secure world would only see the non-secure
>> device tree configuration.  The QEMU virt machine would be responsible
>> for producing the device tree blobs.
>
> I've thought about this some in the past and leaned toward this
> direction mainly because I'd expect you are partitioning most nodes to
> one side. You could do some crazy partitioning with secure and
> non-secure world. It was suggested on highbank to use the secure bit
> as a 33rd address bit to get 8GB of address space for example. You
> could have entirely different view of the system.
>
>> The drawbacks to this approach are:
>> * There are 2 device trees to manage
>
> You already have 2 bootloaders and 2 OS's to manage.
>
>> * The two DTBs will typically be almost identical.
>
> I don't really agree. If the secure dtb has all non-secure peripherals
> too, then yes. But if you only include peripherals allocated to secure
> world and secure view of peripherals, then I don't think there would
> be much overlap. You pretty much have to statically allocate each
> peripheral to one side or the other.
>

The above option was assuming two fully populated DTs with one also
including secure resources.  If we strip the secure DT down to only
secure resources then they would indeed be unique.

>> * Not possible to identify whether a device is shared or not between
>> the secure and non-secure worlds.
>
> Typically, sharing requires a peripheral to be designed to be shared
> like PL330 or MMU-400. I have seen some h/w with locking registers so
> 2 different cores/OSs can share an i2c bus. You could do something
> like that for Trustzone as well I suppose. That's not really secure,
> but allows shared access. I think it is generally a limited number of
> peripherals which are shared.
>

This was more of an issue when the DTs overlapped.  It assumes there
was the possibility of two distinct resources of the same type
residing at the same address. One secure the other not.

>> * Identifying device available only to the secure world require
>> cumbersome comparison of the two device trees.
>
> But that is pretty static. It doesn't seem like a big deal to me.
>
>> * A mechanism would be needed to pass an additional device tree.
>>
>> 2) Modify the standard device tree blob to include annotations or
>> modifications to describe which resources are secure or not.  In this
>> case, secure software would use the single device tree to identify the
>> secure resources.  The added information could be used by secure
>> software to trim the device tree before passing it.  Alternatively,
>> the information could be passed on to non-secure software with the
>> expectation that it would honor the device security.   It would be
>> crucial that any data added to the device tree adhere to existing
>> conventions or expectations.
>
> You could also do this w/o dts changes. You could start with a full
> description and the knowledge of what to filter out resides in the
> secure OS or bootloader. It depends where you want to put the
> partitioning decisions.
>

This is likely what happens today.  Unfortunately, this does not help
in the QEMU case described above because the DT is created dynamically
for a virtual machine model whose configuration is not fixed.  The
approach does not truly embrace the advantage of the DT as it requires
internal HW knowledge which I thought the DT was aimed at alleviating.

> I think there's 2 main cases to consider. Full nodes that are assigned
> to one side and Trustzone aware nodes. The first could potentially
> re-use status property adding just a "secure" setting or we just add a
> new "secure" property. This should probably be inherited by child
> nodes. The secure firmware would then need to remove or set status to
> disabled for those nodes. For the latter case, I don't think we can
> avoid having the firmware having specific knowledge of the bindings.
> We'll have to filter out the secure only details as I think so far we
> have only been creating bindings which describe the non-secure view. I
> don't think this is a big deal as the peripheral set would be pretty
> limited AFAIK.
>
> Rob
>
>> The drawbacks to this approach are:
>> * Additional secure state details. potentially unrecognized by current
>> consumers,  would need to be added to the device tree.
>> * Unless a mechanism already exists, the new secure property runs the
>> risk of breaking backwards compatibility.
>> * Secure firmware is responsible for understanding and possibly
>> filtering the secure device tree data.
>>
>> Feedback welcomed.
>>
>> Greg
>> --
>> To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Secure resources in device trees

[Rob Herring]

On Wed, Jan 21, 2015 at 10:29 AM, Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org> wrote:
> On Tue, Jan 20, 2015 at 07:15:51PM +0000, Greg Bellows wrote:
>> The addition of ARM security extension (TrustZone) support to QEMU has
>> exposed the issue of how secure resources are communicated to secure
>> software responsible for booting the HLOS.  The natural choice for
>> communicating these details is the device tree.
>>
>> In the case of real hardware, the device tree supplied to the HLOS
>> only needs to describe non-secure resources as secure software can
>> rely on static knowledge about the hardware.  This also holds true for
>> QEMU machines modeled after actual fixed hardware configurations.
>> However, this is not the case with QEMU's virtual machine models, such
>> as machvirt, where the hardware configuration can change over time.
>
> When you say this changes over time, I assume you mean for the lifetime
> of the project, as opposed to the lifetime of a running instance? i.e.
> non-probeable devices are not dynamically injected to a running system.

The latter of course is exactly what PowerPC PSeries does with DT.

Which reminds me that this issue is also very similar to hypervisor
cases and Xen in particular. Xen uses DT, filters out what it uses and
passes on the modified DT to Dom0 kernel.

>> In this case, secure software is dependent on QEMU's dynamically
>> constructed device tree to describe the hardware, making it impossible
>> for secure software to know ahead of time what is secure, non-secure,
>> or both.
>>
>> Two possible approaches for handling this particular case are:
>>
>> 1) Create two device tree blobs; one describing the non-secure
>> configuration and the other the full configuration.  This would allow
>> secure software to see the full hardware picture including secure
>> resources while the non-secure world would only see the non-secure
>> device tree configuration.  The QEMU virt machine would be responsible
>> for producing the device tree blobs.
>>
>> The drawbacks to this approach are:
>> * There are 2 device trees to manage
>> * The two DTBs will typically be almost identical.
>> * Not possible to identify whether a device is shared or not between
>> the secure and non-secure worlds.
>> * Identifying device available only to the secure world require
>> cumbersome comparison of the two device trees.
>
> I'm not sure I follow why such identification is necessary?
>
>> * A mechanism would be needed to pass an additional device tree.
>>
>> 2) Modify the standard device tree blob to include annotations or
>> modifications to describe which resources are secure or not.  In this
>> case, secure software would use the single device tree to identify the
>> secure resources.  The added information could be used by secure
>> software to trim the device tree before passing it.  Alternatively,
>> the information could be passed on to non-secure software with the
>> expectation that it would honor the device security.   It would be
>> crucial that any data added to the device tree adhere to existing
>> conventions or expectations.
>
> This approach assumes assumes that the secure and non-secure physical
> address spaces are identical bar some portions being masked out on the
> non-secure side. This is not necessarily the case.

True, but I would guess this is the common case.

> Architecturally the secure and non-secure physical address spaces are
> entiorely separate, and do not necessarily mirror each other.
>
> It's entirely valid for some devices/RAM to only exist in one of the
> address spaces (we typically see secure-only devices, but
> non-secure-only devices are also possible). It's also entirely valid for
> the same device to be mapped at different addresses in each address
> space (e.g. the same UART could be mapped at both S:0xffff0000 and also
> at NS:0xcccc0000 and nowhere else in either address space).
>
> So approach (2) does not fit the architecture generally. From what I
> recall of previous discussions, we eventually figured out that you
> either need separate trees or a higher level container to address the
> secure vs nonsecure split.

Though there's no reason both approaches can't be supported. If the 2
views are radically different, then use 2 DTs. If they are similar and
just a matter of partitioning, then you can fix up the DT before
passing to non-secure world (or even do this with a script offline
(i.e. 1 dts and 2 dtb's)).

Rob
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Secure resources in device trees

[Greg Bellows]

Thanks Mark, comments inline.

On Wed, Jan 21, 2015 at 10:29 AM, Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org> wrote:
> On Tue, Jan 20, 2015 at 07:15:51PM +0000, Greg Bellows wrote:
>> The addition of ARM security extension (TrustZone) support to QEMU has
>> exposed the issue of how secure resources are communicated to secure
>> software responsible for booting the HLOS.  The natural choice for
>> communicating these details is the device tree.
>>
>> In the case of real hardware, the device tree supplied to the HLOS
>> only needs to describe non-secure resources as secure software can
>> rely on static knowledge about the hardware.  This also holds true for
>> QEMU machines modeled after actual fixed hardware configurations.
>> However, this is not the case with QEMU's virtual machine models, such
>> as machvirt, where the hardware configuration can change over time.
>
> When you say this changes over time, I assume you mean for the lifetime
> of the project, as opposed to the lifetime of a running instance? i.e.
> non-probeable devices are not dynamically injected to a running system.
>

Correct, the project.  The QEMU machvirt device can be adapted over
time to include additional resources.

>> In this case, secure software is dependent on QEMU's dynamically
>> constructed device tree to describe the hardware, making it impossible
>> for secure software to know ahead of time what is secure, non-secure,
>> or both.
>>
>> Two possible approaches for handling this particular case are:
>>
>> 1) Create two device tree blobs; one describing the non-secure
>> configuration and the other the full configuration.  This would allow
>> secure software to see the full hardware picture including secure
>> resources while the non-secure world would only see the non-secure
>> device tree configuration.  The QEMU virt machine would be responsible
>> for producing the device tree blobs.
>>
>> The drawbacks to this approach are:
>> * There are 2 device trees to manage
>> * The two DTBs will typically be almost identical.
>> * Not possible to identify whether a device is shared or not between
>> the secure and non-secure worlds.
>> * Identifying device available only to the secure world require
>> cumbersome comparison of the two device trees.
>
> I'm not sure I follow why such identification is necessary?
>

Identifying secure resources would be necessary in cases where secure
resources are removed prior to passing to the next level.  In some
cases, it may also be an indicator that the resource needs
initialization by secure SW.

>> * A mechanism would be needed to pass an additional device tree.
>>
>> 2) Modify the standard device tree blob to include annotations or
>> modifications to describe which resources are secure or not.  In this
>> case, secure software would use the single device tree to identify the
>> secure resources.  The added information could be used by secure
>> software to trim the device tree before passing it.  Alternatively,
>> the information could be passed on to non-secure software with the
>> expectation that it would honor the device security.   It would be
>> crucial that any data added to the device tree adhere to existing
>> conventions or expectations.
>
> This approach assumes assumes that the secure and non-secure physical
> address spaces are identical bar some portions being masked out on the
> non-secure side. This is not necessarily the case.
>
> Architecturally the secure and non-secure physical address spaces are
> entiorely separate, and do not necessarily mirror each other.
>

Considering that the secure world can access both the secure and
non-secure address spaces, it seems imperative that the resource
include information on whether it is secure or non-secure so the
correct address can be used.

> It's entirely valid for some devices/RAM to only exist in one of the
> address spaces (we typically see secure-only devices, but
> non-secure-only devices are also possible). It's also entirely valid for
> the same device to be mapped at different addresses in each address
> space (e.g. the same UART could be mapped at both S:0xffff0000 and also
> at NS:0xcccc0000 and nowhere else in either address space).
>

This certainly throws a wrench into things.  Short of adding a
parallel "non-secure" indicator, approach #2 would fall short.

> So approach (2) does not fit the architecture generally. From what I
> recall of previous discussions, we eventually figured out that you
> either need separate trees or a higher level container to address the
> secure vs nonsecure split.

I was unaware of past discussions but it sounds like the two DT
approach was decidedly more preferable.  It also sounds like the 2 DT
approach must include 2 full DTs rather than a secure-only and
non-secure only.  The latter approach would not allow non-secure only
resources to be distinguishable from shared non-secure resources.

>
> Thanks,
> Mark.
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Secure resources in device trees

[Christoffer Dall]

On Wed, Jan 21, 2015 at 7:01 PM, Greg Bellows <greg.bellows-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> wrote:
> Thanks Mark, comments inline.
>
> On Wed, Jan 21, 2015 at 10:29 AM, Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org> wrote:
>> On Tue, Jan 20, 2015 at 07:15:51PM +0000, Greg Bellows wrote:
>>> The addition of ARM security extension (TrustZone) support to QEMU has
>>> exposed the issue of how secure resources are communicated to secure
>>> software responsible for booting the HLOS.  The natural choice for
>>> communicating these details is the device tree.
>>>
>>> In the case of real hardware, the device tree supplied to the HLOS
>>> only needs to describe non-secure resources as secure software can
>>> rely on static knowledge about the hardware.  This also holds true for
>>> QEMU machines modeled after actual fixed hardware configurations.
>>> However, this is not the case with QEMU's virtual machine models, such
>>> as machvirt, where the hardware configuration can change over time.
>>
>> When you say this changes over time, I assume you mean for the lifetime
>> of the project, as opposed to the lifetime of a running instance? i.e.
>> non-probeable devices are not dynamically injected to a running system.
>>
>
> Correct, the project.  The QEMU machvirt device can be adapted over
> time to include additional resources.
>
>>> In this case, secure software is dependent on QEMU's dynamically
>>> constructed device tree to describe the hardware, making it impossible
>>> for secure software to know ahead of time what is secure, non-secure,
>>> or both.
>>>
>>> Two possible approaches for handling this particular case are:
>>>
>>> 1) Create two device tree blobs; one describing the non-secure
>>> configuration and the other the full configuration.  This would allow
>>> secure software to see the full hardware picture including secure
>>> resources while the non-secure world would only see the non-secure
>>> device tree configuration.  The QEMU virt machine would be responsible
>>> for producing the device tree blobs.
>>>
>>> The drawbacks to this approach are:
>>> * There are 2 device trees to manage
>>> * The two DTBs will typically be almost identical.
>>> * Not possible to identify whether a device is shared or not between
>>> the secure and non-secure worlds.
>>> * Identifying device available only to the secure world require
>>> cumbersome comparison of the two device trees.
>>
>> I'm not sure I follow why such identification is necessary?
>>
>
> Identifying secure resources would be necessary in cases where secure
> resources are removed prior to passing to the next level.  In some
> cases, it may also be an indicator that the resource needs
> initialization by secure SW.
>
>>> * A mechanism would be needed to pass an additional device tree.
>>>
>>> 2) Modify the standard device tree blob to include annotations or
>>> modifications to describe which resources are secure or not.  In this
>>> case, secure software would use the single device tree to identify the
>>> secure resources.  The added information could be used by secure
>>> software to trim the device tree before passing it.  Alternatively,
>>> the information could be passed on to non-secure software with the
>>> expectation that it would honor the device security.   It would be
>>> crucial that any data added to the device tree adhere to existing
>>> conventions or expectations.
>>
>> This approach assumes assumes that the secure and non-secure physical
>> address spaces are identical bar some portions being masked out on the
>> non-secure side. This is not necessarily the case.
>>
>> Architecturally the secure and non-secure physical address spaces are
>> entiorely separate, and do not necessarily mirror each other.
>>
>
> Considering that the secure world can access both the secure and
> non-secure address spaces, it seems imperative that the resource
> include information on whether it is secure or non-secure so the
> correct address can be used.
>
>> It's entirely valid for some devices/RAM to only exist in one of the
>> address spaces (we typically see secure-only devices, but
>> non-secure-only devices are also possible). It's also entirely valid for
>> the same device to be mapped at different addresses in each address
>> space (e.g. the same UART could be mapped at both S:0xffff0000 and also
>> at NS:0xcccc0000 and nowhere else in either address space).
>>
>
> This certainly throws a wrench into things.  Short of adding a
> parallel "non-secure" indicator, approach #2 would fall short.
>
>> So approach (2) does not fit the architecture generally. From what I
>> recall of previous discussions, we eventually figured out that you
>> either need separate trees or a higher level container to address the
>> secure vs nonsecure split.
>
> I was unaware of past discussions but it sounds like the two DT
> approach was decidedly more preferable.  It also sounds like the 2 DT
> approach must include 2 full DTs rather than a secure-only and
> non-secure only.  The latter approach would not allow non-secure only
> resources to be distinguishable from shared non-secure resources.
>
Using separate DTs, would there be any way for secure firmware to
identify a device mapped at S:0xFOO and NS:0xBAR as being actually the
same device?

-Christoffer
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Secure resources in device trees

[Greg Bellows]

On Wed, Jan 21, 2015 at 12:05 PM, Christoffer Dall
<christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> wrote:
> On Wed, Jan 21, 2015 at 7:01 PM, Greg Bellows <greg.bellows-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> wrote:
>> Thanks Mark, comments inline.
>>
>> On Wed, Jan 21, 2015 at 10:29 AM, Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org> wrote:
>>> On Tue, Jan 20, 2015 at 07:15:51PM +0000, Greg Bellows wrote:
>>>> The addition of ARM security extension (TrustZone) support to QEMU has
>>>> exposed the issue of how secure resources are communicated to secure
>>>> software responsible for booting the HLOS.  The natural choice for
>>>> communicating these details is the device tree.
>>>>
>>>> In the case of real hardware, the device tree supplied to the HLOS
>>>> only needs to describe non-secure resources as secure software can
>>>> rely on static knowledge about the hardware.  This also holds true for
>>>> QEMU machines modeled after actual fixed hardware configurations.
>>>> However, this is not the case with QEMU's virtual machine models, such
>>>> as machvirt, where the hardware configuration can change over time.
>>>
>>> When you say this changes over time, I assume you mean for the lifetime
>>> of the project, as opposed to the lifetime of a running instance? i.e.
>>> non-probeable devices are not dynamically injected to a running system.
>>>
>>
>> Correct, the project.  The QEMU machvirt device can be adapted over
>> time to include additional resources.
>>
>>>> In this case, secure software is dependent on QEMU's dynamically
>>>> constructed device tree to describe the hardware, making it impossible
>>>> for secure software to know ahead of time what is secure, non-secure,
>>>> or both.
>>>>
>>>> Two possible approaches for handling this particular case are:
>>>>
>>>> 1) Create two device tree blobs; one describing the non-secure
>>>> configuration and the other the full configuration.  This would allow
>>>> secure software to see the full hardware picture including secure
>>>> resources while the non-secure world would only see the non-secure
>>>> device tree configuration.  The QEMU virt machine would be responsible
>>>> for producing the device tree blobs.
>>>>
>>>> The drawbacks to this approach are:
>>>> * There are 2 device trees to manage
>>>> * The two DTBs will typically be almost identical.
>>>> * Not possible to identify whether a device is shared or not between
>>>> the secure and non-secure worlds.
>>>> * Identifying device available only to the secure world require
>>>> cumbersome comparison of the two device trees.
>>>
>>> I'm not sure I follow why such identification is necessary?
>>>
>>
>> Identifying secure resources would be necessary in cases where secure
>> resources are removed prior to passing to the next level.  In some
>> cases, it may also be an indicator that the resource needs
>> initialization by secure SW.
>>
>>>> * A mechanism would be needed to pass an additional device tree.
>>>>
>>>> 2) Modify the standard device tree blob to include annotations or
>>>> modifications to describe which resources are secure or not.  In this
>>>> case, secure software would use the single device tree to identify the
>>>> secure resources.  The added information could be used by secure
>>>> software to trim the device tree before passing it.  Alternatively,
>>>> the information could be passed on to non-secure software with the
>>>> expectation that it would honor the device security.   It would be
>>>> crucial that any data added to the device tree adhere to existing
>>>> conventions or expectations.
>>>
>>> This approach assumes assumes that the secure and non-secure physical
>>> address spaces are identical bar some portions being masked out on the
>>> non-secure side. This is not necessarily the case.
>>>
>>> Architecturally the secure and non-secure physical address spaces are
>>> entiorely separate, and do not necessarily mirror each other.
>>>
>>
>> Considering that the secure world can access both the secure and
>> non-secure address spaces, it seems imperative that the resource
>> include information on whether it is secure or non-secure so the
>> correct address can be used.
>>
>>> It's entirely valid for some devices/RAM to only exist in one of the
>>> address spaces (we typically see secure-only devices, but
>>> non-secure-only devices are also possible). It's also entirely valid for
>>> the same device to be mapped at different addresses in each address
>>> space (e.g. the same UART could be mapped at both S:0xffff0000 and also
>>> at NS:0xcccc0000 and nowhere else in either address space).
>>>
>>
>> This certainly throws a wrench into things.  Short of adding a
>> parallel "non-secure" indicator, approach #2 would fall short.
>>
>>> So approach (2) does not fit the architecture generally. From what I
>>> recall of previous discussions, we eventually figured out that you
>>> either need separate trees or a higher level container to address the
>>> secure vs nonsecure split.
>>
>> I was unaware of past discussions but it sounds like the two DT
>> approach was decidedly more preferable.  It also sounds like the 2 DT
>> approach must include 2 full DTs rather than a secure-only and
>> non-secure only.  The latter approach would not allow non-secure only
>> resources to be distinguishable from shared non-secure resources.
>>
> Using separate DTs, would there be any way for secure firmware to
> identify a device mapped at S:0xFOO and NS:0xBAR as being actually the
> same device?
>

Maybe alias nodes can be used that are common between the two?

> -Christoffer
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Secure resources in device trees

[Pawel Moll]

On Wed, 2015-01-21 at 16:21 +0000, Rob Herring wrote:
> > * Not possible to identify whether a device is shared or not between
> > the secure and non-secure worlds.
> 
> Typically, sharing requires a peripheral to be designed to be shared
> like PL330 or MMU-400. I have seen some h/w with locking registers so
> 2 different cores/OSs can share an i2c bus. You could do something
> like that for Trustzone as well I suppose. That's not really secure,
> but allows shared access. I think it is generally a limited number of
> peripherals which are shared.

Interestingly enough, the most common case for a shared peripheral is a
keypad/keyboard for secure PIN entry. A secure-programmable-only filter
like TZC-400 can be used to change the "designation" of a AXI master
connected to it.

Pawel

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Secure resources in device trees

[Peter Maydell]

On 21 January 2015 at 16:21, Rob Herring <robherring2-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> On Tue, Jan 20, 2015 at 1:15 PM, Greg Bellows <greg.bellows-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> wrote:
>> The addition of ARM security extension (TrustZone) support to QEMU has
>> exposed the issue of how secure resources are communicated to secure
>> software responsible for booting the HLOS.  The natural choice for
>> communicating these details is the device tree.
>
> You could also have a secure OS use DT and the HLOS use something else
> (ACPI). Or the secure OS and non-secure firmware (UEFI) use DT and the
> OS uses ACPI.
>
>> In the case of real hardware, the device tree supplied to the HLOS
>> only needs to describe non-secure resources as secure software can
>> rely on static knowledge about the hardware.  This also holds true for
>> QEMU machines modeled after actual fixed hardware configurations.
>> However, this is not the case with QEMU's virtual machine models, such
>> as machvirt, where the hardware configuration can change over time.
>> In this case, secure software is dependent on QEMU's dynamically
>> constructed device tree to describe the hardware, making it impossible
>> for secure software to know ahead of time what is secure, non-secure,
>> or both.
>
> For purposes of this discussion, what works better for QEMU is irrelevant IMO.

Given that this discussion is "how can QEMU do this thing in a
way that works with device tree", what works better for QEMU
is worth considering (though not the deciding factor).

>> Two possible approaches for handling this particular case are:
>>
>> 1) Create two device tree blobs; one describing the non-secure
>> configuration and the other the full configuration.  This would allow
>> secure software to see the full hardware picture including secure
>> resources while the non-secure world would only see the non-secure
>> device tree configuration.  The QEMU virt machine would be responsible
>> for producing the device tree blobs.
>
> I've thought about this some in the past and leaned toward this
> direction mainly because I'd expect you are partitioning most nodes to
> one side. You could do some crazy partitioning with secure and
> non-secure world. It was suggested on highbank to use the secure bit
> as a 33rd address bit to get 8GB of address space for example. You
> could have entirely different view of the system.
>
>> The drawbacks to this approach are:
>> * There are 2 device trees to manage
>
> You already have 2 bootloaders and 2 OS's to manage.

No, QEMU doesn't have multiple numbers of any of those to
manage. It wants to tell the (single) thing it's booting what
the config of the (virtual) hardware is, that's all.

>> * The two DTBs will typically be almost identical.
>
> I don't really agree. If the secure dtb has all non-secure peripherals
> too, then yes. But if you only include peripherals allocated to secure
> world and secure view of peripherals, then I don't think there would
> be much overlap. You pretty much have to statically allocate each
> peripheral to one side or the other.

As Pawel notes, the trustzone address space controller lets
the secure side dynamically choose to allocate some subnodes
to S or NS as it pleases. (That's not a situation I care about
at the moment, as it happens, though.)

>> * Not possible to identify whether a device is shared or not between
>> the secure and non-secure worlds.
>
> Typically, sharing requires a peripheral to be designed to be shared
> like PL330 or MMU-400.

Only if you care about actually dynamically letting both sides
use it at once. What Greg means by "shared" here is "appears in
both the S and NS address spaces", as opposed to "appears in only
one and is physically inaccessible from the other".

>> * Identifying device available only to the secure world require
>> cumbersome comparison of the two device trees.
>
> But that is pretty static. It doesn't seem like a big deal to me.

Static in what sense? The firmware being handed the DTB(s)
by QEMU by definition doesn't know what is secure and what
is not without looking at and comparing the DTBs (since the
DTBs are what tells the firmware what hardware exists and
whether it is S or NS).

> You could also do this w/o dts changes. You could start with a full
> description and the knowledge of what to filter out resides in the
> secure OS or bootloader. It depends where you want to put the
> partitioning decisions.

The firmware (secure OS, bootloader) can make partitioning
decisions, but it can't change what the hardware physically
is. It has to be told whether devices are visible in the S, NS or
both address spaces (via DTB).

thanks
-- PMM
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Secure resources in device trees

[Peter Maydell]

On 21 January 2015 at 16:29, Mark Rutland <mark.rutland-5wv7dgnIgG8@public.gmane.org> wrote:
> On Tue, Jan 20, 2015 at 07:15:51PM +0000, Greg Bellows wrote:
>> In the case of real hardware, the device tree supplied to the HLOS
>> only needs to describe non-secure resources as secure software can
>> rely on static knowledge about the hardware.  This also holds true for
>> QEMU machines modeled after actual fixed hardware configurations.
>> However, this is not the case with QEMU's virtual machine models, such
>> as machvirt, where the hardware configuration can change over time.
>
> When you say this changes over time, I assume you mean for the lifetime
> of the project, as opposed to the lifetime of a running instance? i.e.
> non-probeable devices are not dynamically injected to a running system.

We mean in the sense that any particular version of QEMU (or a
particular run of QEMU with different user supplied options) might
expose a different set of hardware to the firmware and OS. The
only thing the firmware can assume is the base address of RAM;
everything else it must determine via DTB.

For hotplug we're going to end up using PCIe I think, so not
a device tree issue at all.

>> The drawbacks to this approach are:
>> * There are 2 device trees to manage
>> * The two DTBs will typically be almost identical.
>> * Not possible to identify whether a device is shared or not between
>> the secure and non-secure worlds.
>> * Identifying device available only to the secure world require
>> cumbersome comparison of the two device trees.
>
> I'm not sure I follow why such identification is necessary?

If I'm the secure world firmware and I want to use a UART to
send some debug output to the world, I can look through the S
device tree blob and find a PL011. But I can't tell if it's the
PL011 that I should use, or the one that's shared with the NS
address space and which the nonsecure guest OS should use, unless
I go through both device tree blobs cross-checking for which
devices appear in both blobs (or unless the devices are marked
up for which address space they appear in, in which case why
have two blobs in the first place?).

>> * A mechanism would be needed to pass an additional device tree.
>>
>> 2) Modify the standard device tree blob to include annotations or
>> modifications to describe which resources are secure or not.  In this
>> case, secure software would use the single device tree to identify the
>> secure resources.  The added information could be used by secure
>> software to trim the device tree before passing it.  Alternatively,
>> the information could be passed on to non-secure software with the
>> expectation that it would honor the device security.   It would be
>> crucial that any data added to the device tree adhere to existing
>> conventions or expectations.
>
> This approach assumes assumes that the secure and non-secure physical
> address spaces are identical bar some portions being masked out on the
> non-secure side. This is not necessarily the case.
>
> Architecturally the secure and non-secure physical address spaces are
> entiorely separate, and do not necessarily mirror each other.
>
> It's entirely valid for some devices/RAM to only exist in one of the
> address spaces (we typically see secure-only devices, but
> non-secure-only devices are also possible). It's also entirely valid for
> the same device to be mapped at different addresses in each address
> space (e.g. the same UART could be mapped at both S:0xffff0000 and also
> at NS:0xcccc0000 and nowhere else in either address space).

This is all true, but the common case is "most things are shared
and a few devices are S-only". What I envisaged with this approach
was an easy way to describe the common case, with presumably syntax
that would permit the weird case to be described too.

> So approach (2) does not fit the architecture generally. From what I
> recall of previous discussions, we eventually figured out that you
> either need separate trees or a higher level container to address the
> secure vs nonsecure split.

I would definitely prefer some sort of single container or blob.
I only want to describe the hardware once and pass the DTB
consumer a single thing describing the hardware.

-- PMM
--
To unsubscribe from this list: send the line "unsubscribe devicetree-spec" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Secure resources in device trees

[Mark Rutland]

[...]

> >> * A mechanism would be needed to pass an additional device tree.
> >>
> >> 2) Modify the standard device tree blob to include annotations or
> >> modifications to describe which resources are secure or not.  In this
> >> case, secure software would use the single device tree to identify the
> >> secure resources.  The added information could be used by secure
> >> software to trim the device tree before passing it.  Alternatively,
> >> the information could be passed on to non-secure software with the
> >> expectation that it would honor the device security.   It would be
> >> crucial that any data added to the device tree adhere to existing
> >> conventions or expectations.
> >
> > This approach assumes assumes that the secure and non-secure physical
> > address spaces are identical bar some portions being masked out on the
> > non-secure side. This is not necessarily the case.
> 
> True, but I would guess this is the common case.

In practically every system I can think of, the address spaces are
essentially the same bar masking.

However, if we're trying to model the architectural envelope, then the
fact that a single DTB can't encode that needs some consideration in
this matter.

> > Architecturally the secure and non-secure physical address spaces are
> > entiorely separate, and do not necessarily mirror each other.
> >
> > It's entirely valid for some devices/RAM to only exist in one of the
> > address spaces (we typically see secure-only devices, but
> > non-secure-only devices are also possible). It's also entirely valid for
> > the same device to be mapped at different addresses in each address
> > space (e.g. the same UART could be mapped at both S:0xffff0000 and also
> > at NS:0xcccc0000 and nowhere else in either address space).
> >
> > So approach (2) does not fit the architecture generally. From what I
> > recall of previous discussions, we eventually figured out that you
> > either need separate trees or a higher level container to address the
> > secure vs nonsecure split.
> 
> Though there's no reason both approaches can't be supported. If the 2
> views are radically different, then use 2 DTs. If they are similar and
> just a matter of partitioning, then you can fix up the DT before
> passing to non-secure world (or even do this with a script offline
> (i.e. 1 dts and 2 dtb's)).

That sounds possible, yes.

Mark.
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html