* [ARM] SMC (and HVC) handling in hypervisor
@ 2017-02-11 0:14 Volodymyr Babchuk
2017-02-13 16:20 ` Tamas K Lengyel
` (2 more replies)
0 siblings, 3 replies; 25+ messages in thread
From: Volodymyr Babchuk @ 2017-02-11 0:14 UTC (permalink / raw)
To: Xen-devel, Tamas K Lengyel, Edgar E. Iglesias, Julien Grall,
Stefano Stabellini, Artem Mygaiev, Andrii Anisov,
Oleksandr Tyshchenko, Andrushchenko, Oleksandr
Hello,
This e-mail is sort of follow-up to the two threads: [1] (my thread
about TEE interaction) and [2] (Edgar's thread regarding handling SMC
calls in platform_hvc). I want to discuss more broad topic there.
Obviously, there are growing number of SMC users and current state of
SMC handling in Xen satisfies nobody. My team wants to handle SMCs in
secure way, Xilinx wants to forward some calls directly to Secure
Monitor, while allowing to handle other in userspace, etc.
My proposition is to gather all requirements to SMC (and HVC) handling
in one place (e.g. in this mail thread). After we' will have clear
picture of what we want, we will be able to develop some solution,
that will satisfy us all. At least, I hope so :)
Also I want to remind, that there are ARM document called "SMC Calling
Convention" [3]. According to it, any aarch64 hypervisor "must
implement the Standard Secure and Hypervisor Service calls". At this
moment XEN does not conform to this.
So, lets get started with the requirements:
0. There are no much difference between SMC and HVC handling (at least
according to SMCCC).
1. Hypervisor should at least provide own UUID and version while
called by SMC/HVC
2. Hypervisor should forward some calls from dom0 directly to Secure
Monitor (Xilinx use case)
3. Hypervisor should virtualize PSCI calls, CPU service calls, ARM
architecture service calls, etc.
4. Hypervisor should handle TEE calls in a secure way (e.g. no
untrusted handlers in Dom0 userspace).
5. Hypervisor should support multiple TEEs (at least at compilation time).
6. Hypervisor should do this as fast as possible (DRM playback use case).
7. All domains (including dom0) should be handled in the same way.
8. Not all domains will have right to issue certain SMCs.
9. Hypervisor will issue own SMCs in some cases.
This is high-level requirements. Feel free to expand this list.
Current SMC handling code does not even handle PSCI calls. Only HVC
trap handler have branch to handle PSCI calls. SMCs are forwarded to
VM monitor subsystem. There are even no advance_pc() call, so monitor
needs to advance PC by itself. Also, dom0 can't have monitor, so there
are no way to handle SMCs that originate from dom0. So, basically,
current code does not meet any requirements from above list. This
means that we can start from scratch and develop any solution.
But at this moment I only want to gather requirements. So feel free to
point at what I have missed.
[1] https://lists.xenproject.org/archives/html/xen-devel/2016-11/msg02220.html
[2] https://lists.xenproject.org/archives/html/xen-devel/2017-02/msg00635.html
[3] http://infocenter.arm.com/help/topic/com.arm.doc.den0028b/ARM_DEN0028B_SMC_Calling_Convention.pdf
--
WBR Volodymyr Babchuk aka lorc [+380976646013]
mailto: vlad.babchuk@gmail.com
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-02-11 0:14 [ARM] SMC (and HVC) handling in hypervisor Volodymyr Babchuk @ 2017-02-13 16:20 ` Tamas K Lengyel 2017-02-13 16:29 ` Volodymyr Babchuk 2017-02-13 16:37 ` Julien Grall 2017-02-28 13:51 ` Julien Grall 2017-03-02 17:54 ` Volodymyr Babchuk 2 siblings, 2 replies; 25+ messages in thread From: Tamas K Lengyel @ 2017-02-13 16:20 UTC (permalink / raw) To: Volodymyr Babchuk Cc: Edgar E. Iglesias, Stefano Stabellini, Andrushchenko, Oleksandr, Andrii Anisov, Oleksandr Tyshchenko, Julien Grall, Xen-devel, Artem Mygaiev On Fri, Feb 10, 2017 at 5:14 PM, Volodymyr Babchuk <vlad.babchuk@gmail.com> wrote: > Hello, > > This e-mail is sort of follow-up to the two threads: [1] (my thread > about TEE interaction) and [2] (Edgar's thread regarding handling SMC > calls in platform_hvc). I want to discuss more broad topic there. > > Obviously, there are growing number of SMC users and current state of > SMC handling in Xen satisfies nobody. My team wants to handle SMCs in > secure way, Xilinx wants to forward some calls directly to Secure > Monitor, while allowing to handle other in userspace, etc. > > My proposition is to gather all requirements to SMC (and HVC) handling > in one place (e.g. in this mail thread). After we' will have clear > picture of what we want, we will be able to develop some solution, > that will satisfy us all. At least, I hope so :) > > Also I want to remind, that there are ARM document called "SMC Calling > Convention" [3]. According to it, any aarch64 hypervisor "must > implement the Standard Secure and Hypervisor Service calls". At this > moment XEN does not conform to this. > > So, lets get started with the requirements: > 0. There are no much difference between SMC and HVC handling (at least > according to SMCCC). > 1. Hypervisor should at least provide own UUID and version while > called by SMC/HVC > 2. Hypervisor should forward some calls from dom0 directly to Secure > Monitor (Xilinx use case) > 3. Hypervisor should virtualize PSCI calls, CPU service calls, ARM > architecture service calls, etc. > 4. Hypervisor should handle TEE calls in a secure way (e.g. no > untrusted handlers in Dom0 userspace). > 5. Hypervisor should support multiple TEEs (at least at compilation time). > 6. Hypervisor should do this as fast as possible (DRM playback use case). > 7. All domains (including dom0) should be handled in the same way. > 8. Not all domains will have right to issue certain SMCs. > 9. Hypervisor will issue own SMCs in some cases. 10. Domains on which the monitor privileged call feature is enabled (which is by default disabled for all domains) should not be able to issue SMCs such that it reaches the firmware directly. Xen should not bounce such calls to the firmware on behalf of the domain. Xen should not alter the state of the domain automatically (ie. incrementing PC). These calls should be exclusively transfered to the monitor subscriber for further processing. HVC calls need not be included in the monitor forwarding as long as the HVC call can be governed by XSM. Tamas _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-02-13 16:20 ` Tamas K Lengyel @ 2017-02-13 16:29 ` Volodymyr Babchuk 2017-02-13 16:37 ` Tamas K Lengyel 2017-02-13 16:37 ` Julien Grall 1 sibling, 1 reply; 25+ messages in thread From: Volodymyr Babchuk @ 2017-02-13 16:29 UTC (permalink / raw) To: Tamas K Lengyel Cc: Edgar E. Iglesias, Stefano Stabellini, Andrushchenko, Oleksandr, Andrii Anisov, Oleksandr Tyshchenko, Julien Grall, Xen-devel, Artem Mygaiev Tamas, On 13 February 2017 at 18:20, Tamas K Lengyel <tamas.k.lengyel@gmail.com> wrote: > On Fri, Feb 10, 2017 at 5:14 PM, Volodymyr Babchuk > <vlad.babchuk@gmail.com> wrote: >> Hello, >> >> This e-mail is sort of follow-up to the two threads: [1] (my thread >> about TEE interaction) and [2] (Edgar's thread regarding handling SMC >> calls in platform_hvc). I want to discuss more broad topic there. >> >> Obviously, there are growing number of SMC users and current state of >> SMC handling in Xen satisfies nobody. My team wants to handle SMCs in >> secure way, Xilinx wants to forward some calls directly to Secure >> Monitor, while allowing to handle other in userspace, etc. >> >> My proposition is to gather all requirements to SMC (and HVC) handling >> in one place (e.g. in this mail thread). After we' will have clear >> picture of what we want, we will be able to develop some solution, >> that will satisfy us all. At least, I hope so :) >> >> Also I want to remind, that there are ARM document called "SMC Calling >> Convention" [3]. According to it, any aarch64 hypervisor "must >> implement the Standard Secure and Hypervisor Service calls". At this >> moment XEN does not conform to this. >> >> So, lets get started with the requirements: >> 0. There are no much difference between SMC and HVC handling (at least >> according to SMCCC). >> 1. Hypervisor should at least provide own UUID and version while >> called by SMC/HVC >> 2. Hypervisor should forward some calls from dom0 directly to Secure >> Monitor (Xilinx use case) >> 3. Hypervisor should virtualize PSCI calls, CPU service calls, ARM >> architecture service calls, etc. >> 4. Hypervisor should handle TEE calls in a secure way (e.g. no >> untrusted handlers in Dom0 userspace). >> 5. Hypervisor should support multiple TEEs (at least at compilation time). >> 6. Hypervisor should do this as fast as possible (DRM playback use case). >> 7. All domains (including dom0) should be handled in the same way. >> 8. Not all domains will have right to issue certain SMCs. >> 9. Hypervisor will issue own SMCs in some cases. > > 10. Domains on which the monitor privileged call feature is enabled > (which is by default disabled for all domains) should not be able to > issue SMCs such that it reaches the firmware directly. Xen should not > bounce such calls to the firmware on behalf of the domain. Xen should > not alter the state of the domain automatically (ie. incrementing PC). > These calls should be exclusively transfered to the monitor subscriber > for further processing. HVC calls need not be included in the monitor > forwarding as long as the HVC call can be governed by XSM. Looks like you are describing how SMC handling implemented at this moment. I agree that one can use VM monitor feature to handle SMCs. But are there any use case for this? Probably, you can implement userspace-based TEE in privileged domain. But for me this ruins the whole idea of TEE. -- WBR Volodymyr Babchuk aka lorc [+380976646013] mailto: vlad.babchuk@gmail.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-02-13 16:29 ` Volodymyr Babchuk @ 2017-02-13 16:37 ` Tamas K Lengyel 0 siblings, 0 replies; 25+ messages in thread From: Tamas K Lengyel @ 2017-02-13 16:37 UTC (permalink / raw) To: Volodymyr Babchuk Cc: Edgar E. Iglesias, Stefano Stabellini, Andrushchenko, Oleksandr, Andrii Anisov, Oleksandr Tyshchenko, Julien Grall, Xen-devel, Artem Mygaiev On Mon, Feb 13, 2017 at 9:29 AM, Volodymyr Babchuk <vlad.babchuk@gmail.com> wrote: > Tamas, > > On 13 February 2017 at 18:20, Tamas K Lengyel <tamas.k.lengyel@gmail.com> wrote: >> On Fri, Feb 10, 2017 at 5:14 PM, Volodymyr Babchuk >> <vlad.babchuk@gmail.com> wrote: >>> Hello, >>> >>> This e-mail is sort of follow-up to the two threads: [1] (my thread >>> about TEE interaction) and [2] (Edgar's thread regarding handling SMC >>> calls in platform_hvc). I want to discuss more broad topic there. >>> >>> Obviously, there are growing number of SMC users and current state of >>> SMC handling in Xen satisfies nobody. My team wants to handle SMCs in >>> secure way, Xilinx wants to forward some calls directly to Secure >>> Monitor, while allowing to handle other in userspace, etc. >>> >>> My proposition is to gather all requirements to SMC (and HVC) handling >>> in one place (e.g. in this mail thread). After we' will have clear >>> picture of what we want, we will be able to develop some solution, >>> that will satisfy us all. At least, I hope so :) >>> >>> Also I want to remind, that there are ARM document called "SMC Calling >>> Convention" [3]. According to it, any aarch64 hypervisor "must >>> implement the Standard Secure and Hypervisor Service calls". At this >>> moment XEN does not conform to this. >>> >>> So, lets get started with the requirements: >>> 0. There are no much difference between SMC and HVC handling (at least >>> according to SMCCC). >>> 1. Hypervisor should at least provide own UUID and version while >>> called by SMC/HVC >>> 2. Hypervisor should forward some calls from dom0 directly to Secure >>> Monitor (Xilinx use case) >>> 3. Hypervisor should virtualize PSCI calls, CPU service calls, ARM >>> architecture service calls, etc. >>> 4. Hypervisor should handle TEE calls in a secure way (e.g. no >>> untrusted handlers in Dom0 userspace). >>> 5. Hypervisor should support multiple TEEs (at least at compilation time). >>> 6. Hypervisor should do this as fast as possible (DRM playback use case). >>> 7. All domains (including dom0) should be handled in the same way. >>> 8. Not all domains will have right to issue certain SMCs. >>> 9. Hypervisor will issue own SMCs in some cases. >> >> 10. Domains on which the monitor privileged call feature is enabled >> (which is by default disabled for all domains) should not be able to >> issue SMCs such that it reaches the firmware directly. Xen should not >> bounce such calls to the firmware on behalf of the domain. Xen should >> not alter the state of the domain automatically (ie. incrementing PC). >> These calls should be exclusively transfered to the monitor subscriber >> for further processing. HVC calls need not be included in the monitor >> forwarding as long as the HVC call can be governed by XSM. > > Looks like you are describing how SMC handling implemented at this > moment. I agree that one can use VM monitor feature to handle SMCs. > But are there any use case for this? Probably, you can implement > userspace-based TEE in privileged domain. But for me this ruins the > whole idea of TEE. Yes, I have two separate usecases for this exact setup. The first is an experimental security setup for ARM (described in https://www.sec.in.tum.de/publications/publication/322); the second is stealthy malware analysis where untrusted code in a guest domain should be only able to interact with Xen and not the firmware. Also, not sure why having this option in Xen would ruin any other system needing SMCs like TEE in your case. The two use-cases may not be compatible with each other, ie. they could not be used simultaneously. But having the option for the user to decide which one it wants to use should have no detrimental effect. Tamas _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-02-13 16:20 ` Tamas K Lengyel 2017-02-13 16:29 ` Volodymyr Babchuk @ 2017-02-13 16:37 ` Julien Grall 2017-02-13 16:59 ` Tamas K Lengyel 1 sibling, 1 reply; 25+ messages in thread From: Julien Grall @ 2017-02-13 16:37 UTC (permalink / raw) To: Tamas K Lengyel, Volodymyr Babchuk Cc: Edgar E. Iglesias, Stefano Stabellini, Andrushchenko, Oleksandr, Andrii Anisov, Oleksandr Tyshchenko, Xen-devel, Artem Mygaiev Hi Tamas, On 13/02/17 16:20, Tamas K Lengyel wrote: > On Fri, Feb 10, 2017 at 5:14 PM, Volodymyr Babchuk > <vlad.babchuk@gmail.com> wrote: >> Hello, >> >> This e-mail is sort of follow-up to the two threads: [1] (my thread >> about TEE interaction) and [2] (Edgar's thread regarding handling SMC >> calls in platform_hvc). I want to discuss more broad topic there. >> >> Obviously, there are growing number of SMC users and current state of >> SMC handling in Xen satisfies nobody. My team wants to handle SMCs in >> secure way, Xilinx wants to forward some calls directly to Secure >> Monitor, while allowing to handle other in userspace, etc. >> >> My proposition is to gather all requirements to SMC (and HVC) handling >> in one place (e.g. in this mail thread). After we' will have clear >> picture of what we want, we will be able to develop some solution, >> that will satisfy us all. At least, I hope so :) >> >> Also I want to remind, that there are ARM document called "SMC Calling >> Convention" [3]. According to it, any aarch64 hypervisor "must >> implement the Standard Secure and Hypervisor Service calls". At this >> moment XEN does not conform to this. >> >> So, lets get started with the requirements: >> 0. There are no much difference between SMC and HVC handling (at least >> according to SMCCC). >> 1. Hypervisor should at least provide own UUID and version while >> called by SMC/HVC >> 2. Hypervisor should forward some calls from dom0 directly to Secure >> Monitor (Xilinx use case) >> 3. Hypervisor should virtualize PSCI calls, CPU service calls, ARM >> architecture service calls, etc. >> 4. Hypervisor should handle TEE calls in a secure way (e.g. no >> untrusted handlers in Dom0 userspace). >> 5. Hypervisor should support multiple TEEs (at least at compilation time). >> 6. Hypervisor should do this as fast as possible (DRM playback use case). >> 7. All domains (including dom0) should be handled in the same way. >> 8. Not all domains will have right to issue certain SMCs. >> 9. Hypervisor will issue own SMCs in some cases. > > 10. Domains on which the monitor privileged call feature is enabled > (which is by default disabled for all domains) should not be able to > issue SMCs such that it reaches the firmware directly. Xen should not > bounce such calls to the firmware on behalf of the domain. Xen should > not alter the state of the domain automatically (ie. incrementing PC). > These calls should be exclusively transfered to the monitor subscriber > for further processing. HVC calls need not be included in the monitor > forwarding as long as the HVC call can be governed by XSM. This should not be a strong requirement. Whilst in your use case you want to forward all the SMCs to the monitor app, there are use case where Xen would need to emulate SMCs on the behalf of the guest. For instance see PSCI (arch/arm/vpsci.c). Another valid use case is Xen handling power management for device assigned to the guest and having the monitor app acting as a "Trusted App". Regarding the HVC call governed by XSM. I think this is the wrong way to g. As it was mentioned a couple of time HVC is a valid conduit for Secure monitor call. You should not deny them on the basis that your monitor app is not able to handle it properly. A better way would be to have a list of Secure Monitor Call (HVC/SMC) that should be forwarded to the monitor app. Cheers, -- Julien Grall _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-02-13 16:37 ` Julien Grall @ 2017-02-13 16:59 ` Tamas K Lengyel 2017-02-13 18:06 ` Julien Grall 0 siblings, 1 reply; 25+ messages in thread From: Tamas K Lengyel @ 2017-02-13 16:59 UTC (permalink / raw) To: Julien Grall Cc: Edgar E. Iglesias, Stefano Stabellini, Volodymyr Babchuk, Andrii Anisov, Oleksandr Tyshchenko, Andrushchenko, Oleksandr, Xen-devel, Artem Mygaiev On Mon, Feb 13, 2017 at 9:37 AM, Julien Grall <julien.grall@arm.com> wrote: > Hi Tamas, > > > On 13/02/17 16:20, Tamas K Lengyel wrote: >> >> On Fri, Feb 10, 2017 at 5:14 PM, Volodymyr Babchuk >> <vlad.babchuk@gmail.com> wrote: >>> >>> Hello, >>> >>> This e-mail is sort of follow-up to the two threads: [1] (my thread >>> about TEE interaction) and [2] (Edgar's thread regarding handling SMC >>> calls in platform_hvc). I want to discuss more broad topic there. >>> >>> Obviously, there are growing number of SMC users and current state of >>> SMC handling in Xen satisfies nobody. My team wants to handle SMCs in >>> secure way, Xilinx wants to forward some calls directly to Secure >>> Monitor, while allowing to handle other in userspace, etc. >>> >>> My proposition is to gather all requirements to SMC (and HVC) handling >>> in one place (e.g. in this mail thread). After we' will have clear >>> picture of what we want, we will be able to develop some solution, >>> that will satisfy us all. At least, I hope so :) >>> >>> Also I want to remind, that there are ARM document called "SMC Calling >>> Convention" [3]. According to it, any aarch64 hypervisor "must >>> implement the Standard Secure and Hypervisor Service calls". At this >>> moment XEN does not conform to this. >>> >>> So, lets get started with the requirements: >>> 0. There are no much difference between SMC and HVC handling (at least >>> according to SMCCC). >>> 1. Hypervisor should at least provide own UUID and version while >>> called by SMC/HVC >>> 2. Hypervisor should forward some calls from dom0 directly to Secure >>> Monitor (Xilinx use case) >>> 3. Hypervisor should virtualize PSCI calls, CPU service calls, ARM >>> architecture service calls, etc. >>> 4. Hypervisor should handle TEE calls in a secure way (e.g. no >>> untrusted handlers in Dom0 userspace). >>> 5. Hypervisor should support multiple TEEs (at least at compilation >>> time). >>> 6. Hypervisor should do this as fast as possible (DRM playback use case). >>> 7. All domains (including dom0) should be handled in the same way. >>> 8. Not all domains will have right to issue certain SMCs. >>> 9. Hypervisor will issue own SMCs in some cases. >> >> >> 10. Domains on which the monitor privileged call feature is enabled >> (which is by default disabled for all domains) should not be able to >> issue SMCs such that it reaches the firmware directly. Xen should not >> bounce such calls to the firmware on behalf of the domain. Xen should >> not alter the state of the domain automatically (ie. incrementing PC). >> These calls should be exclusively transfered to the monitor subscriber >> for further processing. HVC calls need not be included in the monitor >> forwarding as long as the HVC call can be governed by XSM. > > > This should not be a strong requirement. Whilst in your use case you want to > forward all the SMCs to the monitor app, there are use case where Xen would > need to emulate SMCs on the behalf of the guest. For instance see PSCI > (arch/arm/vpsci.c). In my usecases it is a strong requirement. What happens when the monitor system is disabled is beyond my concerns - Xen can emulate or forward the call as it wishes. But when the monitor application is in use - in my usecase - it needs to be in exclusive control. If that breaks an in-guest application, that is acceptable in my usecase. As soon as there is another usecase that would need to support such an application while the monitor system is enabled, the monitor system can be fine-tuned for those needs to allow Xen to emulate. I've said it many times, I have nothing against doing that, but as I don't need it I won't be able to spend time implementing it. > > Another valid use case is Xen handling power management for device assigned > to the guest and having the monitor app acting as a "Trusted App". > > Regarding the HVC call governed by XSM. I think this is the wrong way to g. > As it was mentioned a couple of time HVC is a valid conduit for Secure > monitor call. You should not deny them on the basis that your monitor app is > not able to handle it properly. A better way would be to have a list of > Secure Monitor Call (HVC/SMC) that should be forwarded to the monitor app. I disagree. XSM needs to be in complete control of all hypercalls. Whether denying some of them will break an application or not is not Xen's concern. That is up to me as a user of Xen and XSM. If Xen overrides a XSM policy because we hard-coded HVCs that pass-through, that is a huge security policy violation. So even if we make a list of HVCs that should also fall under the monitor privileged call umbrella, it should still not override XSM. So since I would not be looking to emulate anything that gets forwarded as a result of an HVC call, XSM works for me just fine as the only thing I would do anyway is deny them. So why would that list help when I might as well just make my list more efficiently using XSM? Tamas _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-02-13 16:59 ` Tamas K Lengyel @ 2017-02-13 18:06 ` Julien Grall 2017-02-13 19:54 ` Tamas K Lengyel 0 siblings, 1 reply; 25+ messages in thread From: Julien Grall @ 2017-02-13 18:06 UTC (permalink / raw) To: Tamas K Lengyel Cc: Edgar E. Iglesias, Stefano Stabellini, Volodymyr Babchuk, Andrii Anisov, Oleksandr Tyshchenko, Andrushchenko, Oleksandr, Xen-devel, Artem Mygaiev On 13/02/17 16:59, Tamas K Lengyel wrote: > On Mon, Feb 13, 2017 at 9:37 AM, Julien Grall <julien.grall@arm.com> wrote: >> Hi Tamas, >> >> >> On 13/02/17 16:20, Tamas K Lengyel wrote: >>> >>> On Fri, Feb 10, 2017 at 5:14 PM, Volodymyr Babchuk >>> <vlad.babchuk@gmail.com> wrote: >>>> >>>> Hello, >>>> >>>> This e-mail is sort of follow-up to the two threads: [1] (my thread >>>> about TEE interaction) and [2] (Edgar's thread regarding handling SMC >>>> calls in platform_hvc). I want to discuss more broad topic there. >>>> >>>> Obviously, there are growing number of SMC users and current state of >>>> SMC handling in Xen satisfies nobody. My team wants to handle SMCs in >>>> secure way, Xilinx wants to forward some calls directly to Secure >>>> Monitor, while allowing to handle other in userspace, etc. >>>> >>>> My proposition is to gather all requirements to SMC (and HVC) handling >>>> in one place (e.g. in this mail thread). After we' will have clear >>>> picture of what we want, we will be able to develop some solution, >>>> that will satisfy us all. At least, I hope so :) >>>> >>>> Also I want to remind, that there are ARM document called "SMC Calling >>>> Convention" [3]. According to it, any aarch64 hypervisor "must >>>> implement the Standard Secure and Hypervisor Service calls". At this >>>> moment XEN does not conform to this. >>>> >>>> So, lets get started with the requirements: >>>> 0. There are no much difference between SMC and HVC handling (at least >>>> according to SMCCC). >>>> 1. Hypervisor should at least provide own UUID and version while >>>> called by SMC/HVC >>>> 2. Hypervisor should forward some calls from dom0 directly to Secure >>>> Monitor (Xilinx use case) >>>> 3. Hypervisor should virtualize PSCI calls, CPU service calls, ARM >>>> architecture service calls, etc. >>>> 4. Hypervisor should handle TEE calls in a secure way (e.g. no >>>> untrusted handlers in Dom0 userspace). >>>> 5. Hypervisor should support multiple TEEs (at least at compilation >>>> time). >>>> 6. Hypervisor should do this as fast as possible (DRM playback use case). >>>> 7. All domains (including dom0) should be handled in the same way. >>>> 8. Not all domains will have right to issue certain SMCs. >>>> 9. Hypervisor will issue own SMCs in some cases. >>> >>> >>> 10. Domains on which the monitor privileged call feature is enabled >>> (which is by default disabled for all domains) should not be able to >>> issue SMCs such that it reaches the firmware directly. Xen should not >>> bounce such calls to the firmware on behalf of the domain. Xen should >>> not alter the state of the domain automatically (ie. incrementing PC). >>> These calls should be exclusively transfered to the monitor subscriber >>> for further processing. HVC calls need not be included in the monitor >>> forwarding as long as the HVC call can be governed by XSM. >> >> >> This should not be a strong requirement. Whilst in your use case you want to >> forward all the SMCs to the monitor app, there are use case where Xen would >> need to emulate SMCs on the behalf of the guest. For instance see PSCI >> (arch/arm/vpsci.c). > > In my usecases it is a strong requirement. What happens when the > monitor system is disabled is beyond my concerns - Xen can emulate or > forward the call as it wishes. But when the monitor application is in > use - in my usecase - it needs to be in exclusive control. If that > breaks an in-guest application, that is acceptable in my usecase. As > soon as there is another usecase that would need to support such an > application while the monitor system is enabled, the monitor system > can be fine-tuned for those needs to allow Xen to emulate. I've said > it many times, I have nothing against doing that, but as I don't need > it I won't be able to spend time implementing it. Let me remind you that this discussion is not about what you implemented but what is a sensible design to fit everyone. I also never ask you to implement anything. > >> >> Another valid use case is Xen handling power management for device assigned >> to the guest and having the monitor app acting as a "Trusted App". >> >> Regarding the HVC call governed by XSM. I think this is the wrong way to g. >> As it was mentioned a couple of time HVC is a valid conduit for Secure >> monitor call. You should not deny them on the basis that your monitor app is >> not able to handle it properly. A better way would be to have a list of >> Secure Monitor Call (HVC/SMC) that should be forwarded to the monitor app. > > I disagree. XSM needs to be in complete control of all hypercalls. > Whether denying some of them will break an application or not is not > Xen's concern. That is up to me as a user of Xen and XSM. If Xen > overrides a XSM policy because we hard-coded HVCs that pass-through, > that is a huge security policy violation. So even if we make a list of > HVCs that should also fall under the monitor privileged call umbrella, > it should still not override XSM. So since I would not be looking to > emulate anything that gets forwarded as a result of an HVC call, XSM > works for me just fine as the only thing I would do anyway is deny > them. So why would that list help when I might as well just make my > list more efficiently using XSM? Again, why do you want to handle SMC and HVC differently given they are both a conduit for Secure Call? Cheers, -- Julien Grall _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-02-13 18:06 ` Julien Grall @ 2017-02-13 19:54 ` Tamas K Lengyel 2017-02-13 21:13 ` Stefano Stabellini 0 siblings, 1 reply; 25+ messages in thread From: Tamas K Lengyel @ 2017-02-13 19:54 UTC (permalink / raw) To: Julien Grall Cc: Edgar E. Iglesias, Stefano Stabellini, Volodymyr Babchuk, Andrii Anisov, Oleksandr Tyshchenko, Andrushchenko, Oleksandr, Xen-devel, Artem Mygaiev On Mon, Feb 13, 2017 at 11:06 AM, Julien Grall <julien.grall@arm.com> wrote: > > > On 13/02/17 16:59, Tamas K Lengyel wrote: >> >> On Mon, Feb 13, 2017 at 9:37 AM, Julien Grall <julien.grall@arm.com> >> wrote: >>> >>> Hi Tamas, >>> >>> >>> On 13/02/17 16:20, Tamas K Lengyel wrote: >>>> >>>> >>>> On Fri, Feb 10, 2017 at 5:14 PM, Volodymyr Babchuk >>>> <vlad.babchuk@gmail.com> wrote: >>>>> >>>>> >>>>> Hello, >>>>> >>>>> This e-mail is sort of follow-up to the two threads: [1] (my thread >>>>> about TEE interaction) and [2] (Edgar's thread regarding handling SMC >>>>> calls in platform_hvc). I want to discuss more broad topic there. >>>>> >>>>> Obviously, there are growing number of SMC users and current state of >>>>> SMC handling in Xen satisfies nobody. My team wants to handle SMCs in >>>>> secure way, Xilinx wants to forward some calls directly to Secure >>>>> Monitor, while allowing to handle other in userspace, etc. >>>>> >>>>> My proposition is to gather all requirements to SMC (and HVC) handling >>>>> in one place (e.g. in this mail thread). After we' will have clear >>>>> picture of what we want, we will be able to develop some solution, >>>>> that will satisfy us all. At least, I hope so :) >>>>> >>>>> Also I want to remind, that there are ARM document called "SMC Calling >>>>> Convention" [3]. According to it, any aarch64 hypervisor "must >>>>> implement the Standard Secure and Hypervisor Service calls". At this >>>>> moment XEN does not conform to this. >>>>> >>>>> So, lets get started with the requirements: >>>>> 0. There are no much difference between SMC and HVC handling (at least >>>>> according to SMCCC). >>>>> 1. Hypervisor should at least provide own UUID and version while >>>>> called by SMC/HVC >>>>> 2. Hypervisor should forward some calls from dom0 directly to Secure >>>>> Monitor (Xilinx use case) >>>>> 3. Hypervisor should virtualize PSCI calls, CPU service calls, ARM >>>>> architecture service calls, etc. >>>>> 4. Hypervisor should handle TEE calls in a secure way (e.g. no >>>>> untrusted handlers in Dom0 userspace). >>>>> 5. Hypervisor should support multiple TEEs (at least at compilation >>>>> time). >>>>> 6. Hypervisor should do this as fast as possible (DRM playback use >>>>> case). >>>>> 7. All domains (including dom0) should be handled in the same way. >>>>> 8. Not all domains will have right to issue certain SMCs. >>>>> 9. Hypervisor will issue own SMCs in some cases. >>>> >>>> >>>> >>>> 10. Domains on which the monitor privileged call feature is enabled >>>> (which is by default disabled for all domains) should not be able to >>>> issue SMCs such that it reaches the firmware directly. Xen should not >>>> bounce such calls to the firmware on behalf of the domain. Xen should >>>> not alter the state of the domain automatically (ie. incrementing PC). >>>> These calls should be exclusively transfered to the monitor subscriber >>>> for further processing. HVC calls need not be included in the monitor >>>> forwarding as long as the HVC call can be governed by XSM. >>> >>> >>> >>> This should not be a strong requirement. Whilst in your use case you want >>> to >>> forward all the SMCs to the monitor app, there are use case where Xen >>> would >>> need to emulate SMCs on the behalf of the guest. For instance see PSCI >>> (arch/arm/vpsci.c). >> >> >> In my usecases it is a strong requirement. What happens when the >> monitor system is disabled is beyond my concerns - Xen can emulate or >> forward the call as it wishes. But when the monitor application is in >> use - in my usecase - it needs to be in exclusive control. If that >> breaks an in-guest application, that is acceptable in my usecase. As >> soon as there is another usecase that would need to support such an >> application while the monitor system is enabled, the monitor system >> can be fine-tuned for those needs to allow Xen to emulate. I've said >> it many times, I have nothing against doing that, but as I don't need >> it I won't be able to spend time implementing it. > > > Let me remind you that this discussion is not about what you implemented but > what is a sensible design to fit everyone. I also never ask you to implement > anything. >> >>> >>> Another valid use case is Xen handling power management for device >>> assigned >>> to the guest and having the monitor app acting as a "Trusted App". >>> >>> Regarding the HVC call governed by XSM. I think this is the wrong way to >>> g. >>> As it was mentioned a couple of time HVC is a valid conduit for Secure >>> monitor call. You should not deny them on the basis that your monitor app >>> is >>> not able to handle it properly. A better way would be to have a list of >>> Secure Monitor Call (HVC/SMC) that should be forwarded to the monitor >>> app. >> >> >> I disagree. XSM needs to be in complete control of all hypercalls. >> Whether denying some of them will break an application or not is not >> Xen's concern. That is up to me as a user of Xen and XSM. If Xen >> overrides a XSM policy because we hard-coded HVCs that pass-through, >> that is a huge security policy violation. So even if we make a list of >> HVCs that should also fall under the monitor privileged call umbrella, >> it should still not override XSM. So since I would not be looking to >> emulate anything that gets forwarded as a result of an HVC call, XSM >> works for me just fine as the only thing I would do anyway is deny >> them. So why would that list help when I might as well just make my >> list more efficiently using XSM? > > > Again, why do you want to handle SMC and HVC differently given they are both > a conduit for Secure Call? AFAIU HVCs are used for hypercalls. If some HVCs will be used to go and issue a firmware call from Xen, it still doesn't change the fact that it was a hypercall in the first place. So it should be governed by XSM. Am I missing something? Tamas _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-02-13 19:54 ` Tamas K Lengyel @ 2017-02-13 21:13 ` Stefano Stabellini 2017-02-13 21:32 ` Tamas K Lengyel 0 siblings, 1 reply; 25+ messages in thread From: Stefano Stabellini @ 2017-02-13 21:13 UTC (permalink / raw) To: Tamas K Lengyel Cc: Edgar E. Iglesias, Stefano Stabellini, Volodymyr Babchuk, Xen-devel, Oleksandr Tyshchenko, Andrushchenko, Oleksandr, Julien Grall, Andrii Anisov, Artem Mygaiev On Mon, 13 Feb 2017, Tamas K Lengyel wrote: > On Mon, Feb 13, 2017 at 11:06 AM, Julien Grall <julien.grall@arm.com> wrote: > > > > > > On 13/02/17 16:59, Tamas K Lengyel wrote: > >> > >> On Mon, Feb 13, 2017 at 9:37 AM, Julien Grall <julien.grall@arm.com> > >> wrote: > >>> > >>> Hi Tamas, > >>> > >>> > >>> On 13/02/17 16:20, Tamas K Lengyel wrote: > >>>> > >>>> > >>>> On Fri, Feb 10, 2017 at 5:14 PM, Volodymyr Babchuk > >>>> <vlad.babchuk@gmail.com> wrote: > >>>>> > >>>>> > >>>>> Hello, > >>>>> > >>>>> This e-mail is sort of follow-up to the two threads: [1] (my thread > >>>>> about TEE interaction) and [2] (Edgar's thread regarding handling SMC > >>>>> calls in platform_hvc). I want to discuss more broad topic there. > >>>>> > >>>>> Obviously, there are growing number of SMC users and current state of > >>>>> SMC handling in Xen satisfies nobody. My team wants to handle SMCs in > >>>>> secure way, Xilinx wants to forward some calls directly to Secure > >>>>> Monitor, while allowing to handle other in userspace, etc. > >>>>> > >>>>> My proposition is to gather all requirements to SMC (and HVC) handling > >>>>> in one place (e.g. in this mail thread). After we' will have clear > >>>>> picture of what we want, we will be able to develop some solution, > >>>>> that will satisfy us all. At least, I hope so :) > >>>>> > >>>>> Also I want to remind, that there are ARM document called "SMC Calling > >>>>> Convention" [3]. According to it, any aarch64 hypervisor "must > >>>>> implement the Standard Secure and Hypervisor Service calls". At this > >>>>> moment XEN does not conform to this. > >>>>> > >>>>> So, lets get started with the requirements: > >>>>> 0. There are no much difference between SMC and HVC handling (at least > >>>>> according to SMCCC). > >>>>> 1. Hypervisor should at least provide own UUID and version while > >>>>> called by SMC/HVC > >>>>> 2. Hypervisor should forward some calls from dom0 directly to Secure > >>>>> Monitor (Xilinx use case) > >>>>> 3. Hypervisor should virtualize PSCI calls, CPU service calls, ARM > >>>>> architecture service calls, etc. > >>>>> 4. Hypervisor should handle TEE calls in a secure way (e.g. no > >>>>> untrusted handlers in Dom0 userspace). > >>>>> 5. Hypervisor should support multiple TEEs (at least at compilation > >>>>> time). > >>>>> 6. Hypervisor should do this as fast as possible (DRM playback use > >>>>> case). > >>>>> 7. All domains (including dom0) should be handled in the same way. > >>>>> 8. Not all domains will have right to issue certain SMCs. > >>>>> 9. Hypervisor will issue own SMCs in some cases. > >>>> > >>>> > >>>> > >>>> 10. Domains on which the monitor privileged call feature is enabled > >>>> (which is by default disabled for all domains) should not be able to > >>>> issue SMCs such that it reaches the firmware directly. Xen should not > >>>> bounce such calls to the firmware on behalf of the domain. Xen should > >>>> not alter the state of the domain automatically (ie. incrementing PC). > >>>> These calls should be exclusively transfered to the monitor subscriber > >>>> for further processing. HVC calls need not be included in the monitor > >>>> forwarding as long as the HVC call can be governed by XSM. > >>> > >>> > >>> > >>> This should not be a strong requirement. Whilst in your use case you want > >>> to > >>> forward all the SMCs to the monitor app, there are use case where Xen > >>> would > >>> need to emulate SMCs on the behalf of the guest. For instance see PSCI > >>> (arch/arm/vpsci.c). > >> > >> > >> In my usecases it is a strong requirement. What happens when the > >> monitor system is disabled is beyond my concerns - Xen can emulate or > >> forward the call as it wishes. But when the monitor application is in > >> use - in my usecase - it needs to be in exclusive control. If that > >> breaks an in-guest application, that is acceptable in my usecase. As > >> soon as there is another usecase that would need to support such an > >> application while the monitor system is enabled, the monitor system > >> can be fine-tuned for those needs to allow Xen to emulate. I've said > >> it many times, I have nothing against doing that, but as I don't need > >> it I won't be able to spend time implementing it. Right, as I wrote in the other thread, I think we should be able to find a way to satisfy Tamas' requirement as well as all the others. Of course, once you enable the "forward all SMCs to monitor" mode, some of the other requirements might not be met anymore, but that's acceptable because they are for different use-cases. > > Let me remind you that this discussion is not about what you implemented but > > what is a sensible design to fit everyone. I also never ask you to implement > > anything. > >> > >>> > >>> Another valid use case is Xen handling power management for device > >>> assigned > >>> to the guest and having the monitor app acting as a "Trusted App". > >>> > >>> Regarding the HVC call governed by XSM. I think this is the wrong way to > >>> g. > >>> As it was mentioned a couple of time HVC is a valid conduit for Secure > >>> monitor call. You should not deny them on the basis that your monitor app > >>> is > >>> not able to handle it properly. A better way would be to have a list of > >>> Secure Monitor Call (HVC/SMC) that should be forwarded to the monitor > >>> app. > >> > >> > >> I disagree. XSM needs to be in complete control of all hypercalls. > >> Whether denying some of them will break an application or not is not > >> Xen's concern. That is up to me as a user of Xen and XSM. If Xen > >> overrides a XSM policy because we hard-coded HVCs that pass-through, > >> that is a huge security policy violation. So even if we make a list of > >> HVCs that should also fall under the monitor privileged call umbrella, > >> it should still not override XSM. I agree with you on this. > >> So since I would not be looking to > >> emulate anything that gets forwarded as a result of an HVC call, XSM > >> works for me just fine as the only thing I would do anyway is deny > >> them. So why would that list help when I might as well just make my > >> list more efficiently using XSM? > > > > > > Again, why do you want to handle SMC and HVC differently given they are both > > a conduit for Secure Call? > > AFAIU HVCs are used for hypercalls. If some HVCs will be used to go > and issue a firmware call from Xen, it still doesn't change the fact > that it was a hypercall in the first place. So it should be governed > by XSM. Am I missing something? The problem is the following: on some platforms an operating system might issue a firmware call via HVC instead of SMC. It's outside of our control. The monitor won't be able to receive that event, because the infrastructure doesn't allow to forward HVC calls to the monitor. It might break your use-case. To properly support firmware calls monitoring, we should make no distinctions between SMC and HVC, as specified by the SMC calling convention. To do this right, we need to be able to specify which SMCs/HVCs get forwarded to the monitor. Forwarding all SMCs, but no HVCs, is not a complete solution, although it should work in the vast majority of cases. I would put down as a requirement that we want to be able to monitor any HVC/SMC calls, while accepting the compromise that as an initial implementation only SMCs can actually be monitored. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-02-13 21:13 ` Stefano Stabellini @ 2017-02-13 21:32 ` Tamas K Lengyel 2017-02-13 21:54 ` Stefano Stabellini 0 siblings, 1 reply; 25+ messages in thread From: Tamas K Lengyel @ 2017-02-13 21:32 UTC (permalink / raw) To: Stefano Stabellini Cc: Edgar E. Iglesias, Volodymyr Babchuk, Andrii Anisov, Oleksandr Tyshchenko, Andrushchenko, Oleksandr, Julien Grall, Xen-devel, Artem Mygaiev On Mon, Feb 13, 2017 at 2:13 PM, Stefano Stabellini <sstabellini@kernel.org> wrote: > On Mon, 13 Feb 2017, Tamas K Lengyel wrote: >> On Mon, Feb 13, 2017 at 11:06 AM, Julien Grall <julien.grall@arm.com> wrote: >> > >> > >> > On 13/02/17 16:59, Tamas K Lengyel wrote: >> >> >> >> On Mon, Feb 13, 2017 at 9:37 AM, Julien Grall <julien.grall@arm.com> >> >> wrote: >> >>> >> >>> Hi Tamas, >> >>> >> >>> >> >>> On 13/02/17 16:20, Tamas K Lengyel wrote: >> >>>> >> >>>> >> >>>> On Fri, Feb 10, 2017 at 5:14 PM, Volodymyr Babchuk >> >>>> <vlad.babchuk@gmail.com> wrote: >> >>>>> >> >>>>> >> >>>>> Hello, >> >>>>> >> >>>>> This e-mail is sort of follow-up to the two threads: [1] (my thread >> >>>>> about TEE interaction) and [2] (Edgar's thread regarding handling SMC >> >>>>> calls in platform_hvc). I want to discuss more broad topic there. >> >>>>> >> >>>>> Obviously, there are growing number of SMC users and current state of >> >>>>> SMC handling in Xen satisfies nobody. My team wants to handle SMCs in >> >>>>> secure way, Xilinx wants to forward some calls directly to Secure >> >>>>> Monitor, while allowing to handle other in userspace, etc. >> >>>>> >> >>>>> My proposition is to gather all requirements to SMC (and HVC) handling >> >>>>> in one place (e.g. in this mail thread). After we' will have clear >> >>>>> picture of what we want, we will be able to develop some solution, >> >>>>> that will satisfy us all. At least, I hope so :) >> >>>>> >> >>>>> Also I want to remind, that there are ARM document called "SMC Calling >> >>>>> Convention" [3]. According to it, any aarch64 hypervisor "must >> >>>>> implement the Standard Secure and Hypervisor Service calls". At this >> >>>>> moment XEN does not conform to this. >> >>>>> >> >>>>> So, lets get started with the requirements: >> >>>>> 0. There are no much difference between SMC and HVC handling (at least >> >>>>> according to SMCCC). >> >>>>> 1. Hypervisor should at least provide own UUID and version while >> >>>>> called by SMC/HVC >> >>>>> 2. Hypervisor should forward some calls from dom0 directly to Secure >> >>>>> Monitor (Xilinx use case) >> >>>>> 3. Hypervisor should virtualize PSCI calls, CPU service calls, ARM >> >>>>> architecture service calls, etc. >> >>>>> 4. Hypervisor should handle TEE calls in a secure way (e.g. no >> >>>>> untrusted handlers in Dom0 userspace). >> >>>>> 5. Hypervisor should support multiple TEEs (at least at compilation >> >>>>> time). >> >>>>> 6. Hypervisor should do this as fast as possible (DRM playback use >> >>>>> case). >> >>>>> 7. All domains (including dom0) should be handled in the same way. >> >>>>> 8. Not all domains will have right to issue certain SMCs. >> >>>>> 9. Hypervisor will issue own SMCs in some cases. >> >>>> >> >>>> >> >>>> >> >>>> 10. Domains on which the monitor privileged call feature is enabled >> >>>> (which is by default disabled for all domains) should not be able to >> >>>> issue SMCs such that it reaches the firmware directly. Xen should not >> >>>> bounce such calls to the firmware on behalf of the domain. Xen should >> >>>> not alter the state of the domain automatically (ie. incrementing PC). >> >>>> These calls should be exclusively transfered to the monitor subscriber >> >>>> for further processing. HVC calls need not be included in the monitor >> >>>> forwarding as long as the HVC call can be governed by XSM. >> >>> >> >>> >> >>> >> >>> This should not be a strong requirement. Whilst in your use case you want >> >>> to >> >>> forward all the SMCs to the monitor app, there are use case where Xen >> >>> would >> >>> need to emulate SMCs on the behalf of the guest. For instance see PSCI >> >>> (arch/arm/vpsci.c). >> >> >> >> >> >> In my usecases it is a strong requirement. What happens when the >> >> monitor system is disabled is beyond my concerns - Xen can emulate or >> >> forward the call as it wishes. But when the monitor application is in >> >> use - in my usecase - it needs to be in exclusive control. If that >> >> breaks an in-guest application, that is acceptable in my usecase. As >> >> soon as there is another usecase that would need to support such an >> >> application while the monitor system is enabled, the monitor system >> >> can be fine-tuned for those needs to allow Xen to emulate. I've said >> >> it many times, I have nothing against doing that, but as I don't need >> >> it I won't be able to spend time implementing it. > > Right, as I wrote in the other thread, I think we should be able to find > a way to satisfy Tamas' requirement as well as all the others. Of > course, once you enable the "forward all SMCs to monitor" mode, some of > the other requirements might not be met anymore, but that's acceptable > because they are for different use-cases. > > >> > Let me remind you that this discussion is not about what you implemented but >> > what is a sensible design to fit everyone. I also never ask you to implement >> > anything. >> >> >> >>> >> >>> Another valid use case is Xen handling power management for device >> >>> assigned >> >>> to the guest and having the monitor app acting as a "Trusted App". >> >>> >> >>> Regarding the HVC call governed by XSM. I think this is the wrong way to >> >>> g. >> >>> As it was mentioned a couple of time HVC is a valid conduit for Secure >> >>> monitor call. You should not deny them on the basis that your monitor app >> >>> is >> >>> not able to handle it properly. A better way would be to have a list of >> >>> Secure Monitor Call (HVC/SMC) that should be forwarded to the monitor >> >>> app. >> >> >> >> >> >> I disagree. XSM needs to be in complete control of all hypercalls. >> >> Whether denying some of them will break an application or not is not >> >> Xen's concern. That is up to me as a user of Xen and XSM. If Xen >> >> overrides a XSM policy because we hard-coded HVCs that pass-through, >> >> that is a huge security policy violation. So even if we make a list of >> >> HVCs that should also fall under the monitor privileged call umbrella, >> >> it should still not override XSM. > > I agree with you on this. > > >> >> So since I would not be looking to >> >> emulate anything that gets forwarded as a result of an HVC call, XSM >> >> works for me just fine as the only thing I would do anyway is deny >> >> them. So why would that list help when I might as well just make my >> >> list more efficiently using XSM? >> > >> > >> > Again, why do you want to handle SMC and HVC differently given they are both >> > a conduit for Secure Call? >> >> AFAIU HVCs are used for hypercalls. If some HVCs will be used to go >> and issue a firmware call from Xen, it still doesn't change the fact >> that it was a hypercall in the first place. So it should be governed >> by XSM. Am I missing something? > > The problem is the following: on some platforms an operating system > might issue a firmware call via HVC instead of SMC. It's outside of our > control. The monitor won't be able to receive that event, because the > infrastructure doesn't allow to forward HVC calls to the monitor. It > might break your use-case. That's right. But HVCs should still fall under XSM control, while SMCs aren't (and shouldn't). I mean, the operating system needs to have some understanding of the hypervisor it is running on in order to use HVCs for do firmware calls through the VMM. So it's not like the OS can just use arbitrary HVC calls to get bounced to the firmware by Xen, correct? > > To properly support firmware calls monitoring, we should make no > distinctions between SMC and HVC, as specified by the SMC calling > convention. Well, if in the future if we extend the monitoring system to also record these HVCs that would be something to record in the monitor event. That should be fairly straight forward though. > > To do this right, we need to be able to specify which SMCs/HVCs get > forwarded to the monitor. Forwarding all SMCs, but no HVCs, is not a > complete solution, although it should work in the vast majority of > cases. > > I would put down as a requirement that we want to be able to monitor any > HVC/SMC calls, while accepting the compromise that as an initial > implementation only SMCs can actually be monitored. I agree. While I understand that in those cases HVCs should get treated equivalent to SMCs, ie. get forwarded via the monitor interface, but if I can choose to just deny those with XSM while monitoring regular SMCs, that would be an OK solution for me for now. But if we really need to make a list, my list is fairly straight forward: all HVCs that would be used to trigger a firmware call need to be routed through the monitor system to ensure there is no back channel anywhere from an untrusted guest to the firmware bypassing Xen. Tamas _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-02-13 21:32 ` Tamas K Lengyel @ 2017-02-13 21:54 ` Stefano Stabellini 2017-02-13 22:11 ` Tamas K Lengyel 0 siblings, 1 reply; 25+ messages in thread From: Stefano Stabellini @ 2017-02-13 21:54 UTC (permalink / raw) To: Tamas K Lengyel Cc: Edgar E. Iglesias, Stefano Stabellini, Volodymyr Babchuk, Xen-devel, Oleksandr Tyshchenko, Andrushchenko, Oleksandr, Julien Grall, Andrii Anisov, Artem Mygaiev On Mon, 13 Feb 2017, Tamas K Lengyel wrote: > On Mon, Feb 13, 2017 at 2:13 PM, Stefano Stabellini > <sstabellini@kernel.org> wrote: > > On Mon, 13 Feb 2017, Tamas K Lengyel wrote: > >> On Mon, Feb 13, 2017 at 11:06 AM, Julien Grall <julien.grall@arm.com> wrote: > >> > > >> > > >> > On 13/02/17 16:59, Tamas K Lengyel wrote: > >> >> > >> >> On Mon, Feb 13, 2017 at 9:37 AM, Julien Grall <julien.grall@arm.com> > >> >> wrote: > >> >>> > >> >>> Hi Tamas, > >> >>> > >> >>> > >> >>> On 13/02/17 16:20, Tamas K Lengyel wrote: > >> >>>> > >> >>>> > >> >>>> On Fri, Feb 10, 2017 at 5:14 PM, Volodymyr Babchuk > >> >>>> <vlad.babchuk@gmail.com> wrote: > >> >>>>> > >> >>>>> > >> >>>>> Hello, > >> >>>>> > >> >>>>> This e-mail is sort of follow-up to the two threads: [1] (my thread > >> >>>>> about TEE interaction) and [2] (Edgar's thread regarding handling SMC > >> >>>>> calls in platform_hvc). I want to discuss more broad topic there. > >> >>>>> > >> >>>>> Obviously, there are growing number of SMC users and current state of > >> >>>>> SMC handling in Xen satisfies nobody. My team wants to handle SMCs in > >> >>>>> secure way, Xilinx wants to forward some calls directly to Secure > >> >>>>> Monitor, while allowing to handle other in userspace, etc. > >> >>>>> > >> >>>>> My proposition is to gather all requirements to SMC (and HVC) handling > >> >>>>> in one place (e.g. in this mail thread). After we' will have clear > >> >>>>> picture of what we want, we will be able to develop some solution, > >> >>>>> that will satisfy us all. At least, I hope so :) > >> >>>>> > >> >>>>> Also I want to remind, that there are ARM document called "SMC Calling > >> >>>>> Convention" [3]. According to it, any aarch64 hypervisor "must > >> >>>>> implement the Standard Secure and Hypervisor Service calls". At this > >> >>>>> moment XEN does not conform to this. > >> >>>>> > >> >>>>> So, lets get started with the requirements: > >> >>>>> 0. There are no much difference between SMC and HVC handling (at least > >> >>>>> according to SMCCC). > >> >>>>> 1. Hypervisor should at least provide own UUID and version while > >> >>>>> called by SMC/HVC > >> >>>>> 2. Hypervisor should forward some calls from dom0 directly to Secure > >> >>>>> Monitor (Xilinx use case) > >> >>>>> 3. Hypervisor should virtualize PSCI calls, CPU service calls, ARM > >> >>>>> architecture service calls, etc. > >> >>>>> 4. Hypervisor should handle TEE calls in a secure way (e.g. no > >> >>>>> untrusted handlers in Dom0 userspace). > >> >>>>> 5. Hypervisor should support multiple TEEs (at least at compilation > >> >>>>> time). > >> >>>>> 6. Hypervisor should do this as fast as possible (DRM playback use > >> >>>>> case). > >> >>>>> 7. All domains (including dom0) should be handled in the same way. > >> >>>>> 8. Not all domains will have right to issue certain SMCs. > >> >>>>> 9. Hypervisor will issue own SMCs in some cases. > >> >>>> > >> >>>> > >> >>>> > >> >>>> 10. Domains on which the monitor privileged call feature is enabled > >> >>>> (which is by default disabled for all domains) should not be able to > >> >>>> issue SMCs such that it reaches the firmware directly. Xen should not > >> >>>> bounce such calls to the firmware on behalf of the domain. Xen should > >> >>>> not alter the state of the domain automatically (ie. incrementing PC). > >> >>>> These calls should be exclusively transfered to the monitor subscriber > >> >>>> for further processing. HVC calls need not be included in the monitor > >> >>>> forwarding as long as the HVC call can be governed by XSM. > >> >>> > >> >>> > >> >>> > >> >>> This should not be a strong requirement. Whilst in your use case you want > >> >>> to > >> >>> forward all the SMCs to the monitor app, there are use case where Xen > >> >>> would > >> >>> need to emulate SMCs on the behalf of the guest. For instance see PSCI > >> >>> (arch/arm/vpsci.c). > >> >> > >> >> > >> >> In my usecases it is a strong requirement. What happens when the > >> >> monitor system is disabled is beyond my concerns - Xen can emulate or > >> >> forward the call as it wishes. But when the monitor application is in > >> >> use - in my usecase - it needs to be in exclusive control. If that > >> >> breaks an in-guest application, that is acceptable in my usecase. As > >> >> soon as there is another usecase that would need to support such an > >> >> application while the monitor system is enabled, the monitor system > >> >> can be fine-tuned for those needs to allow Xen to emulate. I've said > >> >> it many times, I have nothing against doing that, but as I don't need > >> >> it I won't be able to spend time implementing it. > > > > Right, as I wrote in the other thread, I think we should be able to find > > a way to satisfy Tamas' requirement as well as all the others. Of > > course, once you enable the "forward all SMCs to monitor" mode, some of > > the other requirements might not be met anymore, but that's acceptable > > because they are for different use-cases. > > > > > >> > Let me remind you that this discussion is not about what you implemented but > >> > what is a sensible design to fit everyone. I also never ask you to implement > >> > anything. > >> >> > >> >>> > >> >>> Another valid use case is Xen handling power management for device > >> >>> assigned > >> >>> to the guest and having the monitor app acting as a "Trusted App". > >> >>> > >> >>> Regarding the HVC call governed by XSM. I think this is the wrong way to > >> >>> g. > >> >>> As it was mentioned a couple of time HVC is a valid conduit for Secure > >> >>> monitor call. You should not deny them on the basis that your monitor app > >> >>> is > >> >>> not able to handle it properly. A better way would be to have a list of > >> >>> Secure Monitor Call (HVC/SMC) that should be forwarded to the monitor > >> >>> app. > >> >> > >> >> > >> >> I disagree. XSM needs to be in complete control of all hypercalls. > >> >> Whether denying some of them will break an application or not is not > >> >> Xen's concern. That is up to me as a user of Xen and XSM. If Xen > >> >> overrides a XSM policy because we hard-coded HVCs that pass-through, > >> >> that is a huge security policy violation. So even if we make a list of > >> >> HVCs that should also fall under the monitor privileged call umbrella, > >> >> it should still not override XSM. > > > > I agree with you on this. > > > > > >> >> So since I would not be looking to > >> >> emulate anything that gets forwarded as a result of an HVC call, XSM > >> >> works for me just fine as the only thing I would do anyway is deny > >> >> them. So why would that list help when I might as well just make my > >> >> list more efficiently using XSM? > >> > > >> > > >> > Again, why do you want to handle SMC and HVC differently given they are both > >> > a conduit for Secure Call? > >> > >> AFAIU HVCs are used for hypercalls. If some HVCs will be used to go > >> and issue a firmware call from Xen, it still doesn't change the fact > >> that it was a hypercall in the first place. So it should be governed > >> by XSM. Am I missing something? > > > > The problem is the following: on some platforms an operating system > > might issue a firmware call via HVC instead of SMC. It's outside of our > > control. The monitor won't be able to receive that event, because the > > infrastructure doesn't allow to forward HVC calls to the monitor. It > > might break your use-case. > > That's right. But HVCs should still fall under XSM control, while SMCs > aren't (and shouldn't). I mean, the operating system needs to have > some understanding of the hypervisor it is running on in order to use > HVCs for do firmware calls through the VMM. So it's not like the OS > can just use arbitrary HVC calls to get bounced to the firmware by > Xen, correct? As strange as it sounds, I don't think we can make that assumption, unfortunately. > > To properly support firmware calls monitoring, we should make no > > distinctions between SMC and HVC, as specified by the SMC calling > > convention. > > Well, if in the future if we extend the monitoring system to also > record these HVCs that would be something to record in the monitor > event. That should be fairly straight forward though. > > > > > To do this right, we need to be able to specify which SMCs/HVCs get > > forwarded to the monitor. Forwarding all SMCs, but no HVCs, is not a > > complete solution, although it should work in the vast majority of > > cases. > > > > I would put down as a requirement that we want to be able to monitor any > > HVC/SMC calls, while accepting the compromise that as an initial > > implementation only SMCs can actually be monitored. > > I agree. While I understand that in those cases HVCs should get > treated equivalent to SMCs, ie. get forwarded via the monitor > interface, but if I can choose to just deny those with XSM while > monitoring regular SMCs, that would be an OK solution for me for now. > But if we really need to make a list, my list is fairly straight > forward: all HVCs that would be used to trigger a firmware call need > to be routed through the monitor system to ensure there is no back > channel anywhere from an untrusted guest to the firmware bypassing > Xen. Right, makes sense, so the requirement should be: 10. Domains on which the monitor privileged call feature is enabled (which is by default disabled for all domains) should not be able to issue firmware calls via SMCs/HVCs so that such calls reach the firmware directly. Xen should not bounce such calls to the firmware on behalf of the domain. Xen should not alter the state of the domain automatically (ie. incrementing PC). These calls should be exclusively transfered to the monitor subscriber for further processing. Hypercalls, virtual PSCI calls, virtual CPU services calls and virtual ARM architecture service calls remain unaffected. Does that work for you? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-02-13 21:54 ` Stefano Stabellini @ 2017-02-13 22:11 ` Tamas K Lengyel 2017-02-13 22:14 ` Stefano Stabellini 0 siblings, 1 reply; 25+ messages in thread From: Tamas K Lengyel @ 2017-02-13 22:11 UTC (permalink / raw) To: Stefano Stabellini Cc: Edgar E. Iglesias, Volodymyr Babchuk, Andrii Anisov, Oleksandr Tyshchenko, Andrushchenko, Oleksandr, Julien Grall, Xen-devel, Artem Mygaiev On Mon, Feb 13, 2017 at 2:54 PM, Stefano Stabellini <sstabellini@kernel.org> wrote: > On Mon, 13 Feb 2017, Tamas K Lengyel wrote: >> On Mon, Feb 13, 2017 at 2:13 PM, Stefano Stabellini >> <sstabellini@kernel.org> wrote: >> > On Mon, 13 Feb 2017, Tamas K Lengyel wrote: >> >> On Mon, Feb 13, 2017 at 11:06 AM, Julien Grall <julien.grall@arm.com> wrote: >> >> > >> >> > >> >> > On 13/02/17 16:59, Tamas K Lengyel wrote: >> >> >> >> >> >> On Mon, Feb 13, 2017 at 9:37 AM, Julien Grall <julien.grall@arm.com> >> >> >> wrote: >> >> >>> >> >> >>> Hi Tamas, >> >> >>> >> >> >>> >> >> >>> On 13/02/17 16:20, Tamas K Lengyel wrote: >> >> >>>> >> >> >>>> >> >> >>>> On Fri, Feb 10, 2017 at 5:14 PM, Volodymyr Babchuk >> >> >>>> <vlad.babchuk@gmail.com> wrote: >> >> >>>>> >> >> >>>>> >> >> >>>>> Hello, >> >> >>>>> >> >> >>>>> This e-mail is sort of follow-up to the two threads: [1] (my thread >> >> >>>>> about TEE interaction) and [2] (Edgar's thread regarding handling SMC >> >> >>>>> calls in platform_hvc). I want to discuss more broad topic there. >> >> >>>>> >> >> >>>>> Obviously, there are growing number of SMC users and current state of >> >> >>>>> SMC handling in Xen satisfies nobody. My team wants to handle SMCs in >> >> >>>>> secure way, Xilinx wants to forward some calls directly to Secure >> >> >>>>> Monitor, while allowing to handle other in userspace, etc. >> >> >>>>> >> >> >>>>> My proposition is to gather all requirements to SMC (and HVC) handling >> >> >>>>> in one place (e.g. in this mail thread). After we' will have clear >> >> >>>>> picture of what we want, we will be able to develop some solution, >> >> >>>>> that will satisfy us all. At least, I hope so :) >> >> >>>>> >> >> >>>>> Also I want to remind, that there are ARM document called "SMC Calling >> >> >>>>> Convention" [3]. According to it, any aarch64 hypervisor "must >> >> >>>>> implement the Standard Secure and Hypervisor Service calls". At this >> >> >>>>> moment XEN does not conform to this. >> >> >>>>> >> >> >>>>> So, lets get started with the requirements: >> >> >>>>> 0. There are no much difference between SMC and HVC handling (at least >> >> >>>>> according to SMCCC). >> >> >>>>> 1. Hypervisor should at least provide own UUID and version while >> >> >>>>> called by SMC/HVC >> >> >>>>> 2. Hypervisor should forward some calls from dom0 directly to Secure >> >> >>>>> Monitor (Xilinx use case) >> >> >>>>> 3. Hypervisor should virtualize PSCI calls, CPU service calls, ARM >> >> >>>>> architecture service calls, etc. >> >> >>>>> 4. Hypervisor should handle TEE calls in a secure way (e.g. no >> >> >>>>> untrusted handlers in Dom0 userspace). >> >> >>>>> 5. Hypervisor should support multiple TEEs (at least at compilation >> >> >>>>> time). >> >> >>>>> 6. Hypervisor should do this as fast as possible (DRM playback use >> >> >>>>> case). >> >> >>>>> 7. All domains (including dom0) should be handled in the same way. >> >> >>>>> 8. Not all domains will have right to issue certain SMCs. >> >> >>>>> 9. Hypervisor will issue own SMCs in some cases. >> >> >>>> >> >> >>>> >> >> >>>> >> >> >>>> 10. Domains on which the monitor privileged call feature is enabled >> >> >>>> (which is by default disabled for all domains) should not be able to >> >> >>>> issue SMCs such that it reaches the firmware directly. Xen should not >> >> >>>> bounce such calls to the firmware on behalf of the domain. Xen should >> >> >>>> not alter the state of the domain automatically (ie. incrementing PC). >> >> >>>> These calls should be exclusively transfered to the monitor subscriber >> >> >>>> for further processing. HVC calls need not be included in the monitor >> >> >>>> forwarding as long as the HVC call can be governed by XSM. >> >> >>> >> >> >>> >> >> >>> >> >> >>> This should not be a strong requirement. Whilst in your use case you want >> >> >>> to >> >> >>> forward all the SMCs to the monitor app, there are use case where Xen >> >> >>> would >> >> >>> need to emulate SMCs on the behalf of the guest. For instance see PSCI >> >> >>> (arch/arm/vpsci.c). >> >> >> >> >> >> >> >> >> In my usecases it is a strong requirement. What happens when the >> >> >> monitor system is disabled is beyond my concerns - Xen can emulate or >> >> >> forward the call as it wishes. But when the monitor application is in >> >> >> use - in my usecase - it needs to be in exclusive control. If that >> >> >> breaks an in-guest application, that is acceptable in my usecase. As >> >> >> soon as there is another usecase that would need to support such an >> >> >> application while the monitor system is enabled, the monitor system >> >> >> can be fine-tuned for those needs to allow Xen to emulate. I've said >> >> >> it many times, I have nothing against doing that, but as I don't need >> >> >> it I won't be able to spend time implementing it. >> > >> > Right, as I wrote in the other thread, I think we should be able to find >> > a way to satisfy Tamas' requirement as well as all the others. Of >> > course, once you enable the "forward all SMCs to monitor" mode, some of >> > the other requirements might not be met anymore, but that's acceptable >> > because they are for different use-cases. >> > >> > >> >> > Let me remind you that this discussion is not about what you implemented but >> >> > what is a sensible design to fit everyone. I also never ask you to implement >> >> > anything. >> >> >> >> >> >>> >> >> >>> Another valid use case is Xen handling power management for device >> >> >>> assigned >> >> >>> to the guest and having the monitor app acting as a "Trusted App". >> >> >>> >> >> >>> Regarding the HVC call governed by XSM. I think this is the wrong way to >> >> >>> g. >> >> >>> As it was mentioned a couple of time HVC is a valid conduit for Secure >> >> >>> monitor call. You should not deny them on the basis that your monitor app >> >> >>> is >> >> >>> not able to handle it properly. A better way would be to have a list of >> >> >>> Secure Monitor Call (HVC/SMC) that should be forwarded to the monitor >> >> >>> app. >> >> >> >> >> >> >> >> >> I disagree. XSM needs to be in complete control of all hypercalls. >> >> >> Whether denying some of them will break an application or not is not >> >> >> Xen's concern. That is up to me as a user of Xen and XSM. If Xen >> >> >> overrides a XSM policy because we hard-coded HVCs that pass-through, >> >> >> that is a huge security policy violation. So even if we make a list of >> >> >> HVCs that should also fall under the monitor privileged call umbrella, >> >> >> it should still not override XSM. >> > >> > I agree with you on this. >> > >> > >> >> >> So since I would not be looking to >> >> >> emulate anything that gets forwarded as a result of an HVC call, XSM >> >> >> works for me just fine as the only thing I would do anyway is deny >> >> >> them. So why would that list help when I might as well just make my >> >> >> list more efficiently using XSM? >> >> > >> >> > >> >> > Again, why do you want to handle SMC and HVC differently given they are both >> >> > a conduit for Secure Call? >> >> >> >> AFAIU HVCs are used for hypercalls. If some HVCs will be used to go >> >> and issue a firmware call from Xen, it still doesn't change the fact >> >> that it was a hypercall in the first place. So it should be governed >> >> by XSM. Am I missing something? >> > >> > The problem is the following: on some platforms an operating system >> > might issue a firmware call via HVC instead of SMC. It's outside of our >> > control. The monitor won't be able to receive that event, because the >> > infrastructure doesn't allow to forward HVC calls to the monitor. It >> > might break your use-case. >> >> That's right. But HVCs should still fall under XSM control, while SMCs >> aren't (and shouldn't). I mean, the operating system needs to have >> some understanding of the hypervisor it is running on in order to use >> HVCs for do firmware calls through the VMM. So it's not like the OS >> can just use arbitrary HVC calls to get bounced to the firmware by >> Xen, correct? > > As strange as it sounds, I don't think we can make that assumption, > unfortunately. That certainly does sound very strange to me. > >> > To properly support firmware calls monitoring, we should make no >> > distinctions between SMC and HVC, as specified by the SMC calling >> > convention. >> >> Well, if in the future if we extend the monitoring system to also >> record these HVCs that would be something to record in the monitor >> event. That should be fairly straight forward though. >> >> > >> > To do this right, we need to be able to specify which SMCs/HVCs get >> > forwarded to the monitor. Forwarding all SMCs, but no HVCs, is not a >> > complete solution, although it should work in the vast majority of >> > cases. >> > >> > I would put down as a requirement that we want to be able to monitor any >> > HVC/SMC calls, while accepting the compromise that as an initial >> > implementation only SMCs can actually be monitored. >> >> I agree. While I understand that in those cases HVCs should get >> treated equivalent to SMCs, ie. get forwarded via the monitor >> interface, but if I can choose to just deny those with XSM while >> monitoring regular SMCs, that would be an OK solution for me for now. >> But if we really need to make a list, my list is fairly straight >> forward: all HVCs that would be used to trigger a firmware call need >> to be routed through the monitor system to ensure there is no back >> channel anywhere from an untrusted guest to the firmware bypassing >> Xen. > > Right, makes sense, so the requirement should be: > > 10. Domains on which the monitor privileged call feature is enabled > (which is by default disabled for all domains) should not be able to > issue firmware calls via SMCs/HVCs so that such calls reach the > firmware directly. Xen should not bounce such calls to the firmware on > behalf of the domain. Xen should not alter the state of the domain > automatically (ie. incrementing PC). These calls should be exclusively > transfered to the monitor subscriber for further processing. > Hypercalls, virtual PSCI calls, virtual CPU services calls and virtual > ARM architecture service calls remain unaffected. > > Does that work for you? It works iff hypercalls, virtual psci calls and virtual CPU services can be denied with XSM. If that's not the case, then no, in that case all those HVC calls that would be bounced to the firmware need to be hooked into the monitor system as well. Ideally as soon as they are being introduced to the Xen codebase. Tamas _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-02-13 22:11 ` Tamas K Lengyel @ 2017-02-13 22:14 ` Stefano Stabellini 2017-02-14 18:06 ` Julien Grall 0 siblings, 1 reply; 25+ messages in thread From: Stefano Stabellini @ 2017-02-13 22:14 UTC (permalink / raw) To: Tamas K Lengyel Cc: Edgar E. Iglesias, Stefano Stabellini, Volodymyr Babchuk, Xen-devel, Oleksandr Tyshchenko, Andrushchenko, Oleksandr, Julien Grall, Andrii Anisov, Artem Mygaiev On Mon, 13 Feb 2017, Tamas K Lengyel wrote: > On Mon, Feb 13, 2017 at 2:54 PM, Stefano Stabellini > <sstabellini@kernel.org> wrote: > > On Mon, 13 Feb 2017, Tamas K Lengyel wrote: > >> On Mon, Feb 13, 2017 at 2:13 PM, Stefano Stabellini > >> <sstabellini@kernel.org> wrote: > >> > On Mon, 13 Feb 2017, Tamas K Lengyel wrote: > >> >> On Mon, Feb 13, 2017 at 11:06 AM, Julien Grall <julien.grall@arm.com> wrote: > >> >> > > >> >> > > >> >> > On 13/02/17 16:59, Tamas K Lengyel wrote: > >> >> >> > >> >> >> On Mon, Feb 13, 2017 at 9:37 AM, Julien Grall <julien.grall@arm.com> > >> >> >> wrote: > >> >> >>> > >> >> >>> Hi Tamas, > >> >> >>> > >> >> >>> > >> >> >>> On 13/02/17 16:20, Tamas K Lengyel wrote: > >> >> >>>> > >> >> >>>> > >> >> >>>> On Fri, Feb 10, 2017 at 5:14 PM, Volodymyr Babchuk > >> >> >>>> <vlad.babchuk@gmail.com> wrote: > >> >> >>>>> > >> >> >>>>> > >> >> >>>>> Hello, > >> >> >>>>> > >> >> >>>>> This e-mail is sort of follow-up to the two threads: [1] (my thread > >> >> >>>>> about TEE interaction) and [2] (Edgar's thread regarding handling SMC > >> >> >>>>> calls in platform_hvc). I want to discuss more broad topic there. > >> >> >>>>> > >> >> >>>>> Obviously, there are growing number of SMC users and current state of > >> >> >>>>> SMC handling in Xen satisfies nobody. My team wants to handle SMCs in > >> >> >>>>> secure way, Xilinx wants to forward some calls directly to Secure > >> >> >>>>> Monitor, while allowing to handle other in userspace, etc. > >> >> >>>>> > >> >> >>>>> My proposition is to gather all requirements to SMC (and HVC) handling > >> >> >>>>> in one place (e.g. in this mail thread). After we' will have clear > >> >> >>>>> picture of what we want, we will be able to develop some solution, > >> >> >>>>> that will satisfy us all. At least, I hope so :) > >> >> >>>>> > >> >> >>>>> Also I want to remind, that there are ARM document called "SMC Calling > >> >> >>>>> Convention" [3]. According to it, any aarch64 hypervisor "must > >> >> >>>>> implement the Standard Secure and Hypervisor Service calls". At this > >> >> >>>>> moment XEN does not conform to this. > >> >> >>>>> > >> >> >>>>> So, lets get started with the requirements: > >> >> >>>>> 0. There are no much difference between SMC and HVC handling (at least > >> >> >>>>> according to SMCCC). > >> >> >>>>> 1. Hypervisor should at least provide own UUID and version while > >> >> >>>>> called by SMC/HVC > >> >> >>>>> 2. Hypervisor should forward some calls from dom0 directly to Secure > >> >> >>>>> Monitor (Xilinx use case) > >> >> >>>>> 3. Hypervisor should virtualize PSCI calls, CPU service calls, ARM > >> >> >>>>> architecture service calls, etc. > >> >> >>>>> 4. Hypervisor should handle TEE calls in a secure way (e.g. no > >> >> >>>>> untrusted handlers in Dom0 userspace). > >> >> >>>>> 5. Hypervisor should support multiple TEEs (at least at compilation > >> >> >>>>> time). > >> >> >>>>> 6. Hypervisor should do this as fast as possible (DRM playback use > >> >> >>>>> case). > >> >> >>>>> 7. All domains (including dom0) should be handled in the same way. > >> >> >>>>> 8. Not all domains will have right to issue certain SMCs. > >> >> >>>>> 9. Hypervisor will issue own SMCs in some cases. > >> >> >>>> > >> >> >>>> > >> >> >>>> > >> >> >>>> 10. Domains on which the monitor privileged call feature is enabled > >> >> >>>> (which is by default disabled for all domains) should not be able to > >> >> >>>> issue SMCs such that it reaches the firmware directly. Xen should not > >> >> >>>> bounce such calls to the firmware on behalf of the domain. Xen should > >> >> >>>> not alter the state of the domain automatically (ie. incrementing PC). > >> >> >>>> These calls should be exclusively transfered to the monitor subscriber > >> >> >>>> for further processing. HVC calls need not be included in the monitor > >> >> >>>> forwarding as long as the HVC call can be governed by XSM. > >> >> >>> > >> >> >>> > >> >> >>> > >> >> >>> This should not be a strong requirement. Whilst in your use case you want > >> >> >>> to > >> >> >>> forward all the SMCs to the monitor app, there are use case where Xen > >> >> >>> would > >> >> >>> need to emulate SMCs on the behalf of the guest. For instance see PSCI > >> >> >>> (arch/arm/vpsci.c). > >> >> >> > >> >> >> > >> >> >> In my usecases it is a strong requirement. What happens when the > >> >> >> monitor system is disabled is beyond my concerns - Xen can emulate or > >> >> >> forward the call as it wishes. But when the monitor application is in > >> >> >> use - in my usecase - it needs to be in exclusive control. If that > >> >> >> breaks an in-guest application, that is acceptable in my usecase. As > >> >> >> soon as there is another usecase that would need to support such an > >> >> >> application while the monitor system is enabled, the monitor system > >> >> >> can be fine-tuned for those needs to allow Xen to emulate. I've said > >> >> >> it many times, I have nothing against doing that, but as I don't need > >> >> >> it I won't be able to spend time implementing it. > >> > > >> > Right, as I wrote in the other thread, I think we should be able to find > >> > a way to satisfy Tamas' requirement as well as all the others. Of > >> > course, once you enable the "forward all SMCs to monitor" mode, some of > >> > the other requirements might not be met anymore, but that's acceptable > >> > because they are for different use-cases. > >> > > >> > > >> >> > Let me remind you that this discussion is not about what you implemented but > >> >> > what is a sensible design to fit everyone. I also never ask you to implement > >> >> > anything. > >> >> >> > >> >> >>> > >> >> >>> Another valid use case is Xen handling power management for device > >> >> >>> assigned > >> >> >>> to the guest and having the monitor app acting as a "Trusted App". > >> >> >>> > >> >> >>> Regarding the HVC call governed by XSM. I think this is the wrong way to > >> >> >>> g. > >> >> >>> As it was mentioned a couple of time HVC is a valid conduit for Secure > >> >> >>> monitor call. You should not deny them on the basis that your monitor app > >> >> >>> is > >> >> >>> not able to handle it properly. A better way would be to have a list of > >> >> >>> Secure Monitor Call (HVC/SMC) that should be forwarded to the monitor > >> >> >>> app. > >> >> >> > >> >> >> > >> >> >> I disagree. XSM needs to be in complete control of all hypercalls. > >> >> >> Whether denying some of them will break an application or not is not > >> >> >> Xen's concern. That is up to me as a user of Xen and XSM. If Xen > >> >> >> overrides a XSM policy because we hard-coded HVCs that pass-through, > >> >> >> that is a huge security policy violation. So even if we make a list of > >> >> >> HVCs that should also fall under the monitor privileged call umbrella, > >> >> >> it should still not override XSM. > >> > > >> > I agree with you on this. > >> > > >> > > >> >> >> So since I would not be looking to > >> >> >> emulate anything that gets forwarded as a result of an HVC call, XSM > >> >> >> works for me just fine as the only thing I would do anyway is deny > >> >> >> them. So why would that list help when I might as well just make my > >> >> >> list more efficiently using XSM? > >> >> > > >> >> > > >> >> > Again, why do you want to handle SMC and HVC differently given they are both > >> >> > a conduit for Secure Call? > >> >> > >> >> AFAIU HVCs are used for hypercalls. If some HVCs will be used to go > >> >> and issue a firmware call from Xen, it still doesn't change the fact > >> >> that it was a hypercall in the first place. So it should be governed > >> >> by XSM. Am I missing something? > >> > > >> > The problem is the following: on some platforms an operating system > >> > might issue a firmware call via HVC instead of SMC. It's outside of our > >> > control. The monitor won't be able to receive that event, because the > >> > infrastructure doesn't allow to forward HVC calls to the monitor. It > >> > might break your use-case. > >> > >> That's right. But HVCs should still fall under XSM control, while SMCs > >> aren't (and shouldn't). I mean, the operating system needs to have > >> some understanding of the hypervisor it is running on in order to use > >> HVCs for do firmware calls through the VMM. So it's not like the OS > >> can just use arbitrary HVC calls to get bounced to the firmware by > >> Xen, correct? > > > > As strange as it sounds, I don't think we can make that assumption, > > unfortunately. > > That certainly does sound very strange to me. > > > > >> > To properly support firmware calls monitoring, we should make no > >> > distinctions between SMC and HVC, as specified by the SMC calling > >> > convention. > >> > >> Well, if in the future if we extend the monitoring system to also > >> record these HVCs that would be something to record in the monitor > >> event. That should be fairly straight forward though. > >> > >> > > >> > To do this right, we need to be able to specify which SMCs/HVCs get > >> > forwarded to the monitor. Forwarding all SMCs, but no HVCs, is not a > >> > complete solution, although it should work in the vast majority of > >> > cases. > >> > > >> > I would put down as a requirement that we want to be able to monitor any > >> > HVC/SMC calls, while accepting the compromise that as an initial > >> > implementation only SMCs can actually be monitored. > >> > >> I agree. While I understand that in those cases HVCs should get > >> treated equivalent to SMCs, ie. get forwarded via the monitor > >> interface, but if I can choose to just deny those with XSM while > >> monitoring regular SMCs, that would be an OK solution for me for now. > >> But if we really need to make a list, my list is fairly straight > >> forward: all HVCs that would be used to trigger a firmware call need > >> to be routed through the monitor system to ensure there is no back > >> channel anywhere from an untrusted guest to the firmware bypassing > >> Xen. > > > > Right, makes sense, so the requirement should be: > > > > 10. Domains on which the monitor privileged call feature is enabled > > (which is by default disabled for all domains) should not be able to > > issue firmware calls via SMCs/HVCs so that such calls reach the > > firmware directly. Xen should not bounce such calls to the firmware on > > behalf of the domain. Xen should not alter the state of the domain > > automatically (ie. incrementing PC). These calls should be exclusively > > transfered to the monitor subscriber for further processing. > > Hypercalls, virtual PSCI calls, virtual CPU services calls and virtual > > ARM architecture service calls remain unaffected. > > > > Does that work for you? > > It works iff hypercalls, virtual psci calls and virtual CPU services > can be denied with XSM. If that's not the case, then no, in that case > all those HVC calls that would be bounced to the firmware need to be > hooked into the monitor system as well. Ideally as soon as they are > being introduced to the Xen codebase. I don't think we have XSM hooks for all of those today, but I think everybody would agree that it is be a good idea to have them. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-02-13 22:14 ` Stefano Stabellini @ 2017-02-14 18:06 ` Julien Grall 2017-02-14 18:52 ` Tamas K Lengyel 2017-02-14 19:11 ` Stefano Stabellini 0 siblings, 2 replies; 25+ messages in thread From: Julien Grall @ 2017-02-14 18:06 UTC (permalink / raw) To: Stefano Stabellini, Tamas K Lengyel Cc: Edgar E. Iglesias, nd, Volodymyr Babchuk, Andrii Anisov, Oleksandr Tyshchenko, Andrushchenko, Oleksandr, Xen-devel, Artem Mygaiev Hi, On 02/13/2017 10:14 PM, Stefano Stabellini wrote: > On Mon, 13 Feb 2017, Tamas K Lengyel wrote: >> On Mon, Feb 13, 2017 at 2:54 PM, Stefano Stabellini >> <sstabellini@kernel.org> wrote: >>> On Mon, 13 Feb 2017, Tamas K Lengyel wrote: >>>> On Mon, Feb 13, 2017 at 2:13 PM, Stefano Stabellini >>>> <sstabellini@kernel.org> wrote: >>>>> On Mon, 13 Feb 2017, Tamas K Lengyel wrote: >>>>>> On Mon, Feb 13, 2017 at 11:06 AM, Julien Grall <julien.grall@arm.com> wrote: >>>>>>> >>>>>>> >>>>>>> On 13/02/17 16:59, Tamas K Lengyel wrote: >>>>>>>> >>>>>>>> On Mon, Feb 13, 2017 at 9:37 AM, Julien Grall <julien.grall@arm.com> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Hi Tamas, >>>>>>>>> >>>>>>>>> >>>>>>>>> On 13/02/17 16:20, Tamas K Lengyel wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, Feb 10, 2017 at 5:14 PM, Volodymyr Babchuk >>>>>>>>>> <vlad.babchuk@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hello, >>>>>>>>>>> >>>>>>>>>>> This e-mail is sort of follow-up to the two threads: [1] (my thread >>>>>>>>>>> about TEE interaction) and [2] (Edgar's thread regarding handling SMC >>>>>>>>>>> calls in platform_hvc). I want to discuss more broad topic there. >>>>>>>>>>> >>>>>>>>>>> Obviously, there are growing number of SMC users and current state of >>>>>>>>>>> SMC handling in Xen satisfies nobody. My team wants to handle SMCs in >>>>>>>>>>> secure way, Xilinx wants to forward some calls directly to Secure >>>>>>>>>>> Monitor, while allowing to handle other in userspace, etc. >>>>>>>>>>> >>>>>>>>>>> My proposition is to gather all requirements to SMC (and HVC) handling >>>>>>>>>>> in one place (e.g. in this mail thread). After we' will have clear >>>>>>>>>>> picture of what we want, we will be able to develop some solution, >>>>>>>>>>> that will satisfy us all. At least, I hope so :) >>>>>>>>>>> >>>>>>>>>>> Also I want to remind, that there are ARM document called "SMC Calling >>>>>>>>>>> Convention" [3]. According to it, any aarch64 hypervisor "must >>>>>>>>>>> implement the Standard Secure and Hypervisor Service calls". At this >>>>>>>>>>> moment XEN does not conform to this. >>>>>>>>>>> >>>>>>>>>>> So, lets get started with the requirements: >>>>>>>>>>> 0. There are no much difference between SMC and HVC handling (at least >>>>>>>>>>> according to SMCCC). >>>>>>>>>>> 1. Hypervisor should at least provide own UUID and version while >>>>>>>>>>> called by SMC/HVC >>>>>>>>>>> 2. Hypervisor should forward some calls from dom0 directly to Secure >>>>>>>>>>> Monitor (Xilinx use case) >>>>>>>>>>> 3. Hypervisor should virtualize PSCI calls, CPU service calls, ARM >>>>>>>>>>> architecture service calls, etc. >>>>>>>>>>> 4. Hypervisor should handle TEE calls in a secure way (e.g. no >>>>>>>>>>> untrusted handlers in Dom0 userspace). >>>>>>>>>>> 5. Hypervisor should support multiple TEEs (at least at compilation >>>>>>>>>>> time). >>>>>>>>>>> 6. Hypervisor should do this as fast as possible (DRM playback use >>>>>>>>>>> case). >>>>>>>>>>> 7. All domains (including dom0) should be handled in the same way. >>>>>>>>>>> 8. Not all domains will have right to issue certain SMCs. >>>>>>>>>>> 9. Hypervisor will issue own SMCs in some cases. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 10. Domains on which the monitor privileged call feature is enabled >>>>>>>>>> (which is by default disabled for all domains) should not be able to >>>>>>>>>> issue SMCs such that it reaches the firmware directly. Xen should not >>>>>>>>>> bounce such calls to the firmware on behalf of the domain. Xen should >>>>>>>>>> not alter the state of the domain automatically (ie. incrementing PC). >>>>>>>>>> These calls should be exclusively transfered to the monitor subscriber >>>>>>>>>> for further processing. HVC calls need not be included in the monitor >>>>>>>>>> forwarding as long as the HVC call can be governed by XSM. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> This should not be a strong requirement. Whilst in your use case you want >>>>>>>>> to >>>>>>>>> forward all the SMCs to the monitor app, there are use case where Xen >>>>>>>>> would >>>>>>>>> need to emulate SMCs on the behalf of the guest. For instance see PSCI >>>>>>>>> (arch/arm/vpsci.c). >>>>>>>> >>>>>>>> >>>>>>>> In my usecases it is a strong requirement. What happens when the >>>>>>>> monitor system is disabled is beyond my concerns - Xen can emulate or >>>>>>>> forward the call as it wishes. But when the monitor application is in >>>>>>>> use - in my usecase - it needs to be in exclusive control. If that >>>>>>>> breaks an in-guest application, that is acceptable in my usecase. As >>>>>>>> soon as there is another usecase that would need to support such an >>>>>>>> application while the monitor system is enabled, the monitor system >>>>>>>> can be fine-tuned for those needs to allow Xen to emulate. I've said >>>>>>>> it many times, I have nothing against doing that, but as I don't need >>>>>>>> it I won't be able to spend time implementing it. >>>>> >>>>> Right, as I wrote in the other thread, I think we should be able to find >>>>> a way to satisfy Tamas' requirement as well as all the others. Of >>>>> course, once you enable the "forward all SMCs to monitor" mode, some of >>>>> the other requirements might not be met anymore, but that's acceptable >>>>> because they are for different use-cases. >>>>> >>>>> >>>>>>> Let me remind you that this discussion is not about what you implemented but >>>>>>> what is a sensible design to fit everyone. I also never ask you to implement >>>>>>> anything. >>>>>>>> >>>>>>>>> >>>>>>>>> Another valid use case is Xen handling power management for device >>>>>>>>> assigned >>>>>>>>> to the guest and having the monitor app acting as a "Trusted App". >>>>>>>>> >>>>>>>>> Regarding the HVC call governed by XSM. I think this is the wrong way to >>>>>>>>> g. >>>>>>>>> As it was mentioned a couple of time HVC is a valid conduit for Secure >>>>>>>>> monitor call. You should not deny them on the basis that your monitor app >>>>>>>>> is >>>>>>>>> not able to handle it properly. A better way would be to have a list of >>>>>>>>> Secure Monitor Call (HVC/SMC) that should be forwarded to the monitor >>>>>>>>> app. >>>>>>>> >>>>>>>> >>>>>>>> I disagree. XSM needs to be in complete control of all hypercalls. >>>>>>>> Whether denying some of them will break an application or not is not >>>>>>>> Xen's concern. That is up to me as a user of Xen and XSM. If Xen >>>>>>>> overrides a XSM policy because we hard-coded HVCs that pass-through, >>>>>>>> that is a huge security policy violation. So even if we make a list of >>>>>>>> HVCs that should also fall under the monitor privileged call umbrella, >>>>>>>> it should still not override XSM. >>>>> >>>>> I agree with you on this. >>>>> >>>>> >>>>>>>> So since I would not be looking to >>>>>>>> emulate anything that gets forwarded as a result of an HVC call, XSM >>>>>>>> works for me just fine as the only thing I would do anyway is deny >>>>>>>> them. So why would that list help when I might as well just make my >>>>>>>> list more efficiently using XSM? >>>>>>> >>>>>>> >>>>>>> Again, why do you want to handle SMC and HVC differently given they are both >>>>>>> a conduit for Secure Call? >>>>>> >>>>>> AFAIU HVCs are used for hypercalls. If some HVCs will be used to go >>>>>> and issue a firmware call from Xen, it still doesn't change the fact >>>>>> that it was a hypercall in the first place. So it should be governed >>>>>> by XSM. Am I missing something? >>>>> >>>>> The problem is the following: on some platforms an operating system >>>>> might issue a firmware call via HVC instead of SMC. It's outside of our >>>>> control. The monitor won't be able to receive that event, because the >>>>> infrastructure doesn't allow to forward HVC calls to the monitor. It >>>>> might break your use-case. >>>> >>>> That's right. But HVCs should still fall under XSM control, while SMCs >>>> aren't (and shouldn't). I mean, the operating system needs to have >>>> some understanding of the hypervisor it is running on in order to use >>>> HVCs for do firmware calls through the VMM. So it's not like the OS >>>> can just use arbitrary HVC calls to get bounced to the firmware by >>>> Xen, correct? >>> >>> As strange as it sounds, I don't think we can make that assumption, >>> unfortunately. >> >> That certainly does sound very strange to me. It is not strange, a guest knowing that it will run on a hypervisor could potentially only use HVC. Furthermore there are some platform with no EL3 support, so SMC instruction will become undefined. A guest might want to use HVC and expect the hypervisor to emulate it. >> >>> >>>>> To properly support firmware calls monitoring, we should make no >>>>> distinctions between SMC and HVC, as specified by the SMC calling >>>>> convention. >>>> >>>> Well, if in the future if we extend the monitoring system to also >>>> record these HVCs that would be something to record in the monitor >>>> event. That should be fairly straight forward though. >>>> >>>>> >>>>> To do this right, we need to be able to specify which SMCs/HVCs get >>>>> forwarded to the monitor. Forwarding all SMCs, but no HVCs, is not a >>>>> complete solution, although it should work in the vast majority of >>>>> cases. >>>>> >>>>> I would put down as a requirement that we want to be able to monitor any >>>>> HVC/SMC calls, while accepting the compromise that as an initial >>>>> implementation only SMCs can actually be monitored. >>>> >>>> I agree. While I understand that in those cases HVCs should get >>>> treated equivalent to SMCs, ie. get forwarded via the monitor >>>> interface, but if I can choose to just deny those with XSM while >>>> monitoring regular SMCs, that would be an OK solution for me for now. >>>> But if we really need to make a list, my list is fairly straight >>>> forward: all HVCs that would be used to trigger a firmware call need >>>> to be routed through the monitor system to ensure there is no back >>>> channel anywhere from an untrusted guest to the firmware bypassing >>>> Xen. >>> >>> Right, makes sense, so the requirement should be: >>> >>> 10. Domains on which the monitor privileged call feature is enabled >>> (which is by default disabled for all domains) should not be able to >>> issue firmware calls via SMCs/HVCs so that such calls reach the >>> firmware directly. Xen should not bounce such calls to the firmware on >>> behalf of the domain. Xen should not alter the state of the domain >>> automatically (ie. incrementing PC). These calls should be exclusively >>> transfered to the monitor subscriber for further processing. >>> Hypercalls, virtual PSCI calls, virtual CPU services calls and virtual >>> ARM architecture service calls remain unaffected. >>> >>> Does that work for you? >> >> It works iff hypercalls, virtual psci calls and virtual CPU services >> can be denied with XSM. If that's not the case, then no, in that case >> all those HVC calls that would be bounced to the firmware need to be >> hooked into the monitor system as well. Ideally as soon as they are >> being introduced to the Xen codebase. > > I don't think we have XSM hooks for all of those today, but I think > everybody would agree that it is be a good idea to have them. I disagree here. If you add XSM hook in vPSCI, it means you will allow the user to deny CPU bring up. In this case, what is the point to have a guest with multiple CPUs? Regarding forwarding to the monitor app, this is very similar to MMIO region emulated by either Xen or QEMU (in x86 case) they cannot be forwarded. Are you going to add XSM in the MMIO emulation too? Are you going to emulate the vITS/vGIC/vUART/pl011 in the monitor app? Cheers, -- Julien Grall _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-02-14 18:06 ` Julien Grall @ 2017-02-14 18:52 ` Tamas K Lengyel 2017-02-14 19:11 ` Stefano Stabellini 1 sibling, 0 replies; 25+ messages in thread From: Tamas K Lengyel @ 2017-02-14 18:52 UTC (permalink / raw) To: Julien Grall Cc: Edgar E. Iglesias, nd, Stefano Stabellini, Volodymyr Babchuk, Andrii Anisov, Oleksandr Tyshchenko, Andrushchenko, Oleksandr, Xen-devel, Artem Mygaiev On Tue, Feb 14, 2017 at 11:06 AM, Julien Grall <julien.grall@arm.com> wrote: > Hi, > > > On 02/13/2017 10:14 PM, Stefano Stabellini wrote: >> >> On Mon, 13 Feb 2017, Tamas K Lengyel wrote: >>> >>> On Mon, Feb 13, 2017 at 2:54 PM, Stefano Stabellini >>> <sstabellini@kernel.org> wrote: >>>> >>>> On Mon, 13 Feb 2017, Tamas K Lengyel wrote: >>>>> >>>>> On Mon, Feb 13, 2017 at 2:13 PM, Stefano Stabellini >>>>> <sstabellini@kernel.org> wrote: >>>>>> >>>>>> On Mon, 13 Feb 2017, Tamas K Lengyel wrote: >>>>>>> >>>>>>> On Mon, Feb 13, 2017 at 11:06 AM, Julien Grall <julien.grall@arm.com> >>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 13/02/17 16:59, Tamas K Lengyel wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Feb 13, 2017 at 9:37 AM, Julien Grall >>>>>>>>> <julien.grall@arm.com> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi Tamas, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 13/02/17 16:20, Tamas K Lengyel wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Fri, Feb 10, 2017 at 5:14 PM, Volodymyr Babchuk >>>>>>>>>>> <vlad.babchuk@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hello, >>>>>>>>>>>> >>>>>>>>>>>> This e-mail is sort of follow-up to the two threads: [1] (my >>>>>>>>>>>> thread >>>>>>>>>>>> about TEE interaction) and [2] (Edgar's thread regarding >>>>>>>>>>>> handling SMC >>>>>>>>>>>> calls in platform_hvc). I want to discuss more broad topic >>>>>>>>>>>> there. >>>>>>>>>>>> >>>>>>>>>>>> Obviously, there are growing number of SMC users and current >>>>>>>>>>>> state of >>>>>>>>>>>> SMC handling in Xen satisfies nobody. My team wants to handle >>>>>>>>>>>> SMCs in >>>>>>>>>>>> secure way, Xilinx wants to forward some calls directly to >>>>>>>>>>>> Secure >>>>>>>>>>>> Monitor, while allowing to handle other in userspace, etc. >>>>>>>>>>>> >>>>>>>>>>>> My proposition is to gather all requirements to SMC (and HVC) >>>>>>>>>>>> handling >>>>>>>>>>>> in one place (e.g. in this mail thread). After we' will have >>>>>>>>>>>> clear >>>>>>>>>>>> picture of what we want, we will be able to develop some >>>>>>>>>>>> solution, >>>>>>>>>>>> that will satisfy us all. At least, I hope so :) >>>>>>>>>>>> >>>>>>>>>>>> Also I want to remind, that there are ARM document called "SMC >>>>>>>>>>>> Calling >>>>>>>>>>>> Convention" [3]. According to it, any aarch64 hypervisor "must >>>>>>>>>>>> implement the Standard Secure and Hypervisor Service calls". At >>>>>>>>>>>> this >>>>>>>>>>>> moment XEN does not conform to this. >>>>>>>>>>>> >>>>>>>>>>>> So, lets get started with the requirements: >>>>>>>>>>>> 0. There are no much difference between SMC and HVC handling (at >>>>>>>>>>>> least >>>>>>>>>>>> according to SMCCC). >>>>>>>>>>>> 1. Hypervisor should at least provide own UUID and version while >>>>>>>>>>>> called by SMC/HVC >>>>>>>>>>>> 2. Hypervisor should forward some calls from dom0 directly to >>>>>>>>>>>> Secure >>>>>>>>>>>> Monitor (Xilinx use case) >>>>>>>>>>>> 3. Hypervisor should virtualize PSCI calls, CPU service calls, >>>>>>>>>>>> ARM >>>>>>>>>>>> architecture service calls, etc. >>>>>>>>>>>> 4. Hypervisor should handle TEE calls in a secure way (e.g. no >>>>>>>>>>>> untrusted handlers in Dom0 userspace). >>>>>>>>>>>> 5. Hypervisor should support multiple TEEs (at least at >>>>>>>>>>>> compilation >>>>>>>>>>>> time). >>>>>>>>>>>> 6. Hypervisor should do this as fast as possible (DRM playback >>>>>>>>>>>> use >>>>>>>>>>>> case). >>>>>>>>>>>> 7. All domains (including dom0) should be handled in the same >>>>>>>>>>>> way. >>>>>>>>>>>> 8. Not all domains will have right to issue certain SMCs. >>>>>>>>>>>> 9. Hypervisor will issue own SMCs in some cases. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> 10. Domains on which the monitor privileged call feature is >>>>>>>>>>> enabled >>>>>>>>>>> (which is by default disabled for all domains) should not be able >>>>>>>>>>> to >>>>>>>>>>> issue SMCs such that it reaches the firmware directly. Xen should >>>>>>>>>>> not >>>>>>>>>>> bounce such calls to the firmware on behalf of the domain. Xen >>>>>>>>>>> should >>>>>>>>>>> not alter the state of the domain automatically (ie. incrementing >>>>>>>>>>> PC). >>>>>>>>>>> These calls should be exclusively transfered to the monitor >>>>>>>>>>> subscriber >>>>>>>>>>> for further processing. HVC calls need not be included in the >>>>>>>>>>> monitor >>>>>>>>>>> forwarding as long as the HVC call can be governed by XSM. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This should not be a strong requirement. Whilst in your use case >>>>>>>>>> you want >>>>>>>>>> to >>>>>>>>>> forward all the SMCs to the monitor app, there are use case where >>>>>>>>>> Xen >>>>>>>>>> would >>>>>>>>>> need to emulate SMCs on the behalf of the guest. For instance see >>>>>>>>>> PSCI >>>>>>>>>> (arch/arm/vpsci.c). >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> In my usecases it is a strong requirement. What happens when the >>>>>>>>> monitor system is disabled is beyond my concerns - Xen can emulate >>>>>>>>> or >>>>>>>>> forward the call as it wishes. But when the monitor application is >>>>>>>>> in >>>>>>>>> use - in my usecase - it needs to be in exclusive control. If that >>>>>>>>> breaks an in-guest application, that is acceptable in my usecase. >>>>>>>>> As >>>>>>>>> soon as there is another usecase that would need to support such an >>>>>>>>> application while the monitor system is enabled, the monitor system >>>>>>>>> can be fine-tuned for those needs to allow Xen to emulate. I've >>>>>>>>> said >>>>>>>>> it many times, I have nothing against doing that, but as I don't >>>>>>>>> need >>>>>>>>> it I won't be able to spend time implementing it. >>>>>> >>>>>> >>>>>> Right, as I wrote in the other thread, I think we should be able to >>>>>> find >>>>>> a way to satisfy Tamas' requirement as well as all the others. Of >>>>>> course, once you enable the "forward all SMCs to monitor" mode, some >>>>>> of >>>>>> the other requirements might not be met anymore, but that's acceptable >>>>>> because they are for different use-cases. >>>>>> >>>>>> >>>>>>>> Let me remind you that this discussion is not about what you >>>>>>>> implemented but >>>>>>>> what is a sensible design to fit everyone. I also never ask you to >>>>>>>> implement >>>>>>>> anything. >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Another valid use case is Xen handling power management for device >>>>>>>>>> assigned >>>>>>>>>> to the guest and having the monitor app acting as a "Trusted App". >>>>>>>>>> >>>>>>>>>> Regarding the HVC call governed by XSM. I think this is the wrong >>>>>>>>>> way to >>>>>>>>>> g. >>>>>>>>>> As it was mentioned a couple of time HVC is a valid conduit for >>>>>>>>>> Secure >>>>>>>>>> monitor call. You should not deny them on the basis that your >>>>>>>>>> monitor app >>>>>>>>>> is >>>>>>>>>> not able to handle it properly. A better way would be to have a >>>>>>>>>> list of >>>>>>>>>> Secure Monitor Call (HVC/SMC) that should be forwarded to the >>>>>>>>>> monitor >>>>>>>>>> app. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> I disagree. XSM needs to be in complete control of all hypercalls. >>>>>>>>> Whether denying some of them will break an application or not is >>>>>>>>> not >>>>>>>>> Xen's concern. That is up to me as a user of Xen and XSM. If Xen >>>>>>>>> overrides a XSM policy because we hard-coded HVCs that >>>>>>>>> pass-through, >>>>>>>>> that is a huge security policy violation. So even if we make a list >>>>>>>>> of >>>>>>>>> HVCs that should also fall under the monitor privileged call >>>>>>>>> umbrella, >>>>>>>>> it should still not override XSM. >>>>>> >>>>>> >>>>>> I agree with you on this. >>>>>> >>>>>> >>>>>>>>> So since I would not be looking to >>>>>>>>> emulate anything that gets forwarded as a result of an HVC call, >>>>>>>>> XSM >>>>>>>>> works for me just fine as the only thing I would do anyway is deny >>>>>>>>> them. So why would that list help when I might as well just make my >>>>>>>>> list more efficiently using XSM? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Again, why do you want to handle SMC and HVC differently given they >>>>>>>> are both >>>>>>>> a conduit for Secure Call? >>>>>>> >>>>>>> >>>>>>> AFAIU HVCs are used for hypercalls. If some HVCs will be used to go >>>>>>> and issue a firmware call from Xen, it still doesn't change the fact >>>>>>> that it was a hypercall in the first place. So it should be governed >>>>>>> by XSM. Am I missing something? >>>>>> >>>>>> >>>>>> The problem is the following: on some platforms an operating system >>>>>> might issue a firmware call via HVC instead of SMC. It's outside of >>>>>> our >>>>>> control. The monitor won't be able to receive that event, because the >>>>>> infrastructure doesn't allow to forward HVC calls to the monitor. It >>>>>> might break your use-case. >>>>> >>>>> >>>>> That's right. But HVCs should still fall under XSM control, while SMCs >>>>> aren't (and shouldn't). I mean, the operating system needs to have >>>>> some understanding of the hypervisor it is running on in order to use >>>>> HVCs for do firmware calls through the VMM. So it's not like the OS >>>>> can just use arbitrary HVC calls to get bounced to the firmware by >>>>> Xen, correct? >>>> >>>> >>>> As strange as it sounds, I don't think we can make that assumption, >>>> unfortunately. >>> >>> >>> That certainly does sound very strange to me. > > > It is not strange, a guest knowing that it will run on a hypervisor could > potentially only use HVC. Furthermore there are some platform with no EL3 > support, so SMC instruction will become undefined. A guest might want to use > HVC and expect the hypervisor to emulate it. > >>> >>>> >>>>>> To properly support firmware calls monitoring, we should make no >>>>>> distinctions between SMC and HVC, as specified by the SMC calling >>>>>> convention. >>>>> >>>>> >>>>> Well, if in the future if we extend the monitoring system to also >>>>> record these HVCs that would be something to record in the monitor >>>>> event. That should be fairly straight forward though. >>>>> >>>>>> >>>>>> To do this right, we need to be able to specify which SMCs/HVCs get >>>>>> forwarded to the monitor. Forwarding all SMCs, but no HVCs, is not a >>>>>> complete solution, although it should work in the vast majority of >>>>>> cases. >>>>>> >>>>>> I would put down as a requirement that we want to be able to monitor >>>>>> any >>>>>> HVC/SMC calls, while accepting the compromise that as an initial >>>>>> implementation only SMCs can actually be monitored. >>>>> >>>>> >>>>> I agree. While I understand that in those cases HVCs should get >>>>> treated equivalent to SMCs, ie. get forwarded via the monitor >>>>> interface, but if I can choose to just deny those with XSM while >>>>> monitoring regular SMCs, that would be an OK solution for me for now. >>>>> But if we really need to make a list, my list is fairly straight >>>>> forward: all HVCs that would be used to trigger a firmware call need >>>>> to be routed through the monitor system to ensure there is no back >>>>> channel anywhere from an untrusted guest to the firmware bypassing >>>>> Xen. >>>> >>>> >>>> Right, makes sense, so the requirement should be: >>>> >>>> 10. Domains on which the monitor privileged call feature is enabled >>>> (which is by default disabled for all domains) should not be able to >>>> issue firmware calls via SMCs/HVCs so that such calls reach the >>>> firmware directly. Xen should not bounce such calls to the firmware on >>>> behalf of the domain. Xen should not alter the state of the domain >>>> automatically (ie. incrementing PC). These calls should be exclusively >>>> transfered to the monitor subscriber for further processing. >>>> Hypercalls, virtual PSCI calls, virtual CPU services calls and virtual >>>> ARM architecture service calls remain unaffected. >>>> >>>> Does that work for you? >>> >>> >>> It works iff hypercalls, virtual psci calls and virtual CPU services >>> can be denied with XSM. If that's not the case, then no, in that case >>> all those HVC calls that would be bounced to the firmware need to be >>> hooked into the monitor system as well. Ideally as soon as they are >>> being introduced to the Xen codebase. >> >> >> I don't think we have XSM hooks for all of those today, but I think >> everybody would agree that it is be a good idea to have them. > > > I disagree here. If you add XSM hook in vPSCI, it means you will allow the > user to deny CPU bring up. In this case, what is the point to have a guest > with multiple CPUs? > > Regarding forwarding to the monitor app, this is very similar to MMIO region > emulated by either Xen or QEMU (in x86 case) they cannot be forwarded. Are > you going to add XSM in the MMIO emulation too? Are you going to emulate the > vITS/vGIC/vUART/pl011 in the monitor app? > I have no plans to emulate anything. If it breaks an application, it breaks an application. My setup is not targeted towards arbitrary OS/app combinations. It is targeting a very specific setup. As long as the option is available to deny arbitrary applications talking directly to the firmware, I'm OK with it, be that with XSM or by forwarding via the monitor interface where I can deny such calls. At this point I don't think I can add any more information to drive this conversation forward.. Tamas _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-02-14 18:06 ` Julien Grall 2017-02-14 18:52 ` Tamas K Lengyel @ 2017-02-14 19:11 ` Stefano Stabellini 2017-02-14 20:14 ` Tamas K Lengyel 2017-02-23 18:16 ` Julien Grall 1 sibling, 2 replies; 25+ messages in thread From: Stefano Stabellini @ 2017-02-14 19:11 UTC (permalink / raw) To: Julien Grall Cc: Edgar E. Iglesias, Stefano Stabellini, Xen-devel, Volodymyr Babchuk, Tamas K Lengyel, Andrii Anisov, Oleksandr Tyshchenko, Andrushchenko, Oleksandr, nd, Artem Mygaiev On Tue, 14 Feb 2017, Julien Grall wrote: > > > > 10. Domains on which the monitor privileged call feature is enabled > > > > (which is by default disabled for all domains) should not be able to > > > > issue firmware calls via SMCs/HVCs so that such calls reach the > > > > firmware directly. Xen should not bounce such calls to the firmware on > > > > behalf of the domain. Xen should not alter the state of the domain > > > > automatically (ie. incrementing PC). These calls should be exclusively > > > > transfered to the monitor subscriber for further processing. > > > > Hypercalls, virtual PSCI calls, virtual CPU services calls and virtual > > > > ARM architecture service calls remain unaffected. > > > > > > > > Does that work for you? > > > > > > It works iff hypercalls, virtual psci calls and virtual CPU services > > > can be denied with XSM. If that's not the case, then no, in that case > > > all those HVC calls that would be bounced to the firmware need to be > > > hooked into the monitor system as well. Ideally as soon as they are > > > being introduced to the Xen codebase. > > > > I don't think we have XSM hooks for all of those today, but I think > > everybody would agree that it is be a good idea to have them. > > I disagree here. If you add XSM hook in vPSCI, it means you will allow the > user to deny CPU bring up. In this case, what is the point to have a guest > with multiple CPUs? > > Regarding forwarding to the monitor app, this is very similar to MMIO region > emulated by either Xen or QEMU (in x86 case) they cannot be forwarded. Are you > going to add XSM in the MMIO emulation too? Are you going to emulate the > vITS/vGIC/vUART/pl011 in the monitor app? Let's talk about your two concerns separately, because they are separate points. I won't go into the merits of the comparison between firmware calls and x86 MMIO region emulation. However, I think that forwarding firmware calls events to the monitor is fine. It allows new potential use-cases for Xen, without affecting any of the current use-cases, given that firmware calls forwarding would need to be enabled separately. In regards to XSM hooks for vPSCI, after looking at the code, it is true that there aren't any XSM hooks even for the corresponding PV vcpu_op hypercalls (used for PV guests on x86). Introducing them might not be as obvious as I thought. But I don't think we actually need XSM for this. Firstly, we need to distinguish between SMC/HVC calls that result in platform and TEE calls (the main topic of this thread), and SMC/HVC calls that result in virtual PSCI CPU operations and Xen hypercalls (do_trap_psci and do_trap_hypercall). The first set of calls is bounced to the firmware, while the second set of calls is emulated/virtualized. Clearly, the first set of calls would need to be forwarded to the monitor. I don't think that the second set of calls need to be forwarded or filtered, because they actually do not result in any firmware calls. Once the user enables "firmware calls forwarding to the monitor", we could just leave the virtual PSCI cpu ops and hypercalls implemented as they are today. We just need to write down in a document what SMC/HVC calls get forwarded to the monitor once the user enabled the appropriate option. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-02-14 19:11 ` Stefano Stabellini @ 2017-02-14 20:14 ` Tamas K Lengyel 2017-02-23 18:16 ` Julien Grall 1 sibling, 0 replies; 25+ messages in thread From: Tamas K Lengyel @ 2017-02-14 20:14 UTC (permalink / raw) To: Stefano Stabellini Cc: Edgar E. Iglesias, Xen-devel, Volodymyr Babchuk, Andrii Anisov, Oleksandr Tyshchenko, Andrushchenko, Oleksandr, Julien Grall, nd, Artem Mygaiev On Tue, Feb 14, 2017 at 12:11 PM, Stefano Stabellini <sstabellini@kernel.org> wrote: > On Tue, 14 Feb 2017, Julien Grall wrote: >> > > > 10. Domains on which the monitor privileged call feature is enabled >> > > > (which is by default disabled for all domains) should not be able to >> > > > issue firmware calls via SMCs/HVCs so that such calls reach the >> > > > firmware directly. Xen should not bounce such calls to the firmware on >> > > > behalf of the domain. Xen should not alter the state of the domain >> > > > automatically (ie. incrementing PC). These calls should be exclusively >> > > > transfered to the monitor subscriber for further processing. >> > > > Hypercalls, virtual PSCI calls, virtual CPU services calls and virtual >> > > > ARM architecture service calls remain unaffected. >> > > > >> > > > Does that work for you? >> > > >> > > It works iff hypercalls, virtual psci calls and virtual CPU services >> > > can be denied with XSM. If that's not the case, then no, in that case >> > > all those HVC calls that would be bounced to the firmware need to be >> > > hooked into the monitor system as well. Ideally as soon as they are >> > > being introduced to the Xen codebase. >> > >> > I don't think we have XSM hooks for all of those today, but I think >> > everybody would agree that it is be a good idea to have them. >> >> I disagree here. If you add XSM hook in vPSCI, it means you will allow the >> user to deny CPU bring up. In this case, what is the point to have a guest >> with multiple CPUs? >> >> Regarding forwarding to the monitor app, this is very similar to MMIO region >> emulated by either Xen or QEMU (in x86 case) they cannot be forwarded. Are you >> going to add XSM in the MMIO emulation too? Are you going to emulate the >> vITS/vGIC/vUART/pl011 in the monitor app? > > Let's talk about your two concerns separately, because they are > separate points. > > > I won't go into the merits of the comparison between firmware calls and > x86 MMIO region emulation. However, I think that forwarding firmware > calls events to the monitor is fine. It allows new potential use-cases > for Xen, without affecting any of the current use-cases, given that > firmware calls forwarding would need to be enabled separately. > > In regards to XSM hooks for vPSCI, after looking at the code, it is true > that there aren't any XSM hooks even for the corresponding PV vcpu_op > hypercalls (used for PV guests on x86). Introducing them might not be as > obvious as I thought. But I don't think we actually need XSM for this. > > > Firstly, we need to distinguish between SMC/HVC calls that result in > platform and TEE calls (the main topic of this thread), and SMC/HVC > calls that result in virtual PSCI CPU operations and Xen hypercalls > (do_trap_psci and do_trap_hypercall). The first set of calls is bounced > to the firmware, while the second set of calls is emulated/virtualized. > > Clearly, the first set of calls would need to be forwarded to the > monitor. I don't think that the second set of calls need to be forwarded > or filtered, because they actually do not result in any firmware calls. > Once the user enables "firmware calls forwarding to the monitor", we > could just leave the virtual PSCI cpu ops and hypercalls implemented as > they are today. We just need to write down in a document what SMC/HVC > calls get forwarded to the monitor once the user enabled the appropriate > option. Correct, calls that are emulated by Xen itself don't need to be forwarded to the monitor app. Tamas _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-02-14 19:11 ` Stefano Stabellini 2017-02-14 20:14 ` Tamas K Lengyel @ 2017-02-23 18:16 ` Julien Grall 1 sibling, 0 replies; 25+ messages in thread From: Julien Grall @ 2017-02-23 18:16 UTC (permalink / raw) To: Stefano Stabellini Cc: Edgar E. Iglesias, Xen-devel, Volodymyr Babchuk, Tamas K Lengyel, Andrii Anisov, Oleksandr Tyshchenko, Andrushchenko, Oleksandr, nd, Artem Mygaiev Hi Stefano, On 14/02/17 19:11, Stefano Stabellini wrote: > On Tue, 14 Feb 2017, Julien Grall wrote: >>>>> 10. Domains on which the monitor privileged call feature is enabled >>>>> (which is by default disabled for all domains) should not be able to >>>>> issue firmware calls via SMCs/HVCs so that such calls reach the >>>>> firmware directly. Xen should not bounce such calls to the firmware on >>>>> behalf of the domain. Xen should not alter the state of the domain >>>>> automatically (ie. incrementing PC). These calls should be exclusively >>>>> transfered to the monitor subscriber for further processing. >>>>> Hypercalls, virtual PSCI calls, virtual CPU services calls and virtual >>>>> ARM architecture service calls remain unaffected. >>>>> >>>>> Does that work for you? >>>> >>>> It works iff hypercalls, virtual psci calls and virtual CPU services >>>> can be denied with XSM. If that's not the case, then no, in that case >>>> all those HVC calls that would be bounced to the firmware need to be >>>> hooked into the monitor system as well. Ideally as soon as they are >>>> being introduced to the Xen codebase. >>> >>> I don't think we have XSM hooks for all of those today, but I think >>> everybody would agree that it is be a good idea to have them. >> >> I disagree here. If you add XSM hook in vPSCI, it means you will allow the >> user to deny CPU bring up. In this case, what is the point to have a guest >> with multiple CPUs? >> >> Regarding forwarding to the monitor app, this is very similar to MMIO region >> emulated by either Xen or QEMU (in x86 case) they cannot be forwarded. Are you >> going to add XSM in the MMIO emulation too? Are you going to emulate the >> vITS/vGIC/vUART/pl011 in the monitor app? > > Let's talk about your two concerns separately, because they are > separate points. > > > I won't go into the merits of the comparison between firmware calls and > x86 MMIO region emulation. However, I think that forwarding firmware > calls events to the monitor is fine. It allows new potential use-cases > for Xen, without affecting any of the current use-cases, given that > firmware calls forwarding would need to be enabled separately. > > In regards to XSM hooks for vPSCI, after looking at the code, it is true > that there aren't any XSM hooks even for the corresponding PV vcpu_op > hypercalls (used for PV guests on x86). Introducing them might not be as > obvious as I thought. But I don't think we actually need XSM for this. > > > Firstly, we need to distinguish between SMC/HVC calls that result in > platform and TEE calls (the main topic of this thread), and SMC/HVC > calls that result in virtual PSCI CPU operations and Xen hypercalls > (do_trap_psci and do_trap_hypercall). The first set of calls is bounced > to the firmware, while the second set of calls is emulated/virtualized. Agree here. Although, I think we would need a finer grain in the future to allow services (see Table 6-1 in the SMC convention) to be either forwarded to the monitor, emulated, or forwarded to the host secure firmware. > > Clearly, the first set of calls would need to be forwarded to the > monitor. I don't think that the second set of calls need to be forwarded > or filtered, because they actually do not result in any firmware calls. > Once the user enables "firmware calls forwarding to the monitor", we > could just leave the virtual PSCI cpu ops and hypercalls implemented as > they are today. We just need to write down in a document what SMC/HVC > calls get forwarded to the monitor once the user enabled the appropriate > option. > Sounds good to me. Cheers, -- Julien Grall _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-02-11 0:14 [ARM] SMC (and HVC) handling in hypervisor Volodymyr Babchuk 2017-02-13 16:20 ` Tamas K Lengyel @ 2017-02-28 13:51 ` Julien Grall 2017-03-01 14:13 ` Volodymyr Babchuk 2017-03-02 17:54 ` Volodymyr Babchuk 2 siblings, 1 reply; 25+ messages in thread From: Julien Grall @ 2017-02-28 13:51 UTC (permalink / raw) To: Volodymyr Babchuk, Xen-devel, Tamas K Lengyel, Edgar E. Iglesias, Stefano Stabellini, Artem Mygaiev, Andrii Anisov, Oleksandr Tyshchenko, Andrushchenko, Oleksandr Cc: nd On 11/02/17 00:14, Volodymyr Babchuk wrote: > Hello, Hi Volodymyr, > This e-mail is sort of follow-up to the two threads: [1] (my thread > about TEE interaction) and [2] (Edgar's thread regarding handling SMC > calls in platform_hvc). I want to discuss more broad topic there. > > Obviously, there are growing number of SMC users and current state of > SMC handling in Xen satisfies nobody. My team wants to handle SMCs in > secure way, Xilinx wants to forward some calls directly to Secure > Monitor, while allowing to handle other in userspace, etc. > > My proposition is to gather all requirements to SMC (and HVC) handling > in one place (e.g. in this mail thread). After we' will have clear > picture of what we want, we will be able to develop some solution, > that will satisfy us all. At least, I hope so :) > > Also I want to remind, that there are ARM document called "SMC Calling > Convention" [3]. According to it, any aarch64 hypervisor "must > implement the Standard Secure and Hypervisor Service calls". At this > moment XEN does not conform to this. > > So, lets get started with the requirements: > 0. There are no much difference between SMC and HVC handling (at least > according to SMCCC). > 1. Hypervisor should at least provide own UUID and version while > called by SMC/HVC Do we need to reserve the UUID for Xen? > 2. Hypervisor should forward some calls from dom0 directly to Secure > Monitor (Xilinx use case) > 3. Hypervisor should virtualize PSCI calls, CPU service calls, ARM > architecture service calls, etc. > 4. Hypervisor should handle TEE calls in a secure way (e.g. no > untrusted handlers in Dom0 userspace). > 5. Hypervisor should support multiple TEEs (at least at compilation time). > 6. Hypervisor should do this as fast as possible (DRM playback use case). > 7. All domains (including dom0) should be handled in the same way. +1 here. Same path for all domains means less code and you it will get tested more Cheers, -- Julien Grall _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-02-28 13:51 ` Julien Grall @ 2017-03-01 14:13 ` Volodymyr Babchuk 2017-03-01 16:09 ` Julien Grall 0 siblings, 1 reply; 25+ messages in thread From: Volodymyr Babchuk @ 2017-03-01 14:13 UTC (permalink / raw) To: Julien Grall Cc: Edgar E. Iglesias, nd, Stefano Stabellini, Andrushchenko, Oleksandr, Tamas K Lengyel, Andrii Anisov, Oleksandr Tyshchenko, Xen-devel, Artem Mygaiev Hi, Julien On 28 February 2017 at 15:51, Julien Grall <julien.grall@arm.com> wrote: >> This e-mail is sort of follow-up to the two threads: [1] (my thread >> about TEE interaction) and [2] (Edgar's thread regarding handling SMC >> calls in platform_hvc). I want to discuss more broad topic there. >> >> Obviously, there are growing number of SMC users and current state of >> SMC handling in Xen satisfies nobody. My team wants to handle SMCs in >> secure way, Xilinx wants to forward some calls directly to Secure >> Monitor, while allowing to handle other in userspace, etc. >> >> My proposition is to gather all requirements to SMC (and HVC) handling >> in one place (e.g. in this mail thread). After we' will have clear >> picture of what we want, we will be able to develop some solution, >> that will satisfy us all. At least, I hope so :) >> >> Also I want to remind, that there are ARM document called "SMC Calling >> Convention" [3]. According to it, any aarch64 hypervisor "must >> implement the Standard Secure and Hypervisor Service calls". At this >> moment XEN does not conform to this. >> >> So, lets get started with the requirements: >> 0. There are no much difference between SMC and HVC handling (at least >> according to SMCCC). >> 1. Hypervisor should at least provide own UUID and version while >> called by SMC/HVC > > > Do we need to reserve the UUID for Xen? Yes, I think so. But I don't know the procedure. Should be that UUID registered somewhere? > >> 2. Hypervisor should forward some calls from dom0 directly to Secure >> Monitor (Xilinx use case) >> 3. Hypervisor should virtualize PSCI calls, CPU service calls, ARM >> architecture service calls, etc. >> 4. Hypervisor should handle TEE calls in a secure way (e.g. no >> untrusted handlers in Dom0 userspace). >> 5. Hypervisor should support multiple TEEs (at least at compilation time). >> 6. Hypervisor should do this as fast as possible (DRM playback use case). >> 7. All domains (including dom0) should be handled in the same way. > > > +1 here. Same path for all domains means less code and you it will get > tested more Yep. But this is somewhat incompatible with current implementation, as currently VM monitor mechanism is being used. -- WBR Volodymyr Babchuk aka lorc [+380976646013] mailto: vlad.babchuk@gmail.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-03-01 14:13 ` Volodymyr Babchuk @ 2017-03-01 16:09 ` Julien Grall 2017-03-01 16:18 ` Julien Grall 2017-03-01 16:47 ` Volodymyr Babchuk 0 siblings, 2 replies; 25+ messages in thread From: Julien Grall @ 2017-03-01 16:09 UTC (permalink / raw) To: Volodymyr Babchuk Cc: Edgar E. Iglesias, Andrii Anisov, Stefano Stabellini, Andrushchenko, Oleksandr, Tamas K Lengyel, Xen-devel, Oleksandr Tyshchenko, nd, Artem Mygaiev On 01/03/17 14:13, Volodymyr Babchuk wrote: > Hi, Julien Hi Volodymyr, > On 28 February 2017 at 15:51, Julien Grall <julien.grall@arm.com> wrote: > >>> This e-mail is sort of follow-up to the two threads: [1] (my thread >>> about TEE interaction) and [2] (Edgar's thread regarding handling SMC >>> calls in platform_hvc). I want to discuss more broad topic there. >>> >>> Obviously, there are growing number of SMC users and current state of >>> SMC handling in Xen satisfies nobody. My team wants to handle SMCs in >>> secure way, Xilinx wants to forward some calls directly to Secure >>> Monitor, while allowing to handle other in userspace, etc. >>> >>> My proposition is to gather all requirements to SMC (and HVC) handling >>> in one place (e.g. in this mail thread). After we' will have clear >>> picture of what we want, we will be able to develop some solution, >>> that will satisfy us all. At least, I hope so :) >>> >>> Also I want to remind, that there are ARM document called "SMC Calling >>> Convention" [3]. According to it, any aarch64 hypervisor "must >>> implement the Standard Secure and Hypervisor Service calls". At this >>> moment XEN does not conform to this. >>> >>> So, lets get started with the requirements: >>> 0. There are no much difference between SMC and HVC handling (at least >>> according to SMCCC). >>> 1. Hypervisor should at least provide own UUID and version while >>> called by SMC/HVC >> >> >> Do we need to reserve the UUID for Xen? > Yes, I think so. But I don't know the procedure. Should be that UUID > registered somewhere? I will chat internally about it and come back to you. >> >>> 2. Hypervisor should forward some calls from dom0 directly to Secure >>> Monitor (Xilinx use case) >>> 3. Hypervisor should virtualize PSCI calls, CPU service calls, ARM >>> architecture service calls, etc. >>> 4. Hypervisor should handle TEE calls in a secure way (e.g. no >>> untrusted handlers in Dom0 userspace). >>> 5. Hypervisor should support multiple TEEs (at least at compilation time). >>> 6. Hypervisor should do this as fast as possible (DRM playback use case). >>> 7. All domains (including dom0) should be handled in the same way. >> >> >> +1 here. Same path for all domains means less code and you it will get >> tested more > Yep. But this is somewhat incompatible with current implementation, as > currently VM monitor mechanism is being used. I am not sure to understand here. Can you detail it? Cheers, -- Julien Grall _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-03-01 16:09 ` Julien Grall @ 2017-03-01 16:18 ` Julien Grall 2017-03-01 16:47 ` Volodymyr Babchuk 1 sibling, 0 replies; 25+ messages in thread From: Julien Grall @ 2017-03-01 16:18 UTC (permalink / raw) To: Volodymyr Babchuk Cc: Edgar E. Iglesias, Andrii Anisov, Stefano Stabellini, Andrushchenko, Oleksandr, Tamas K Lengyel, Xen-devel, Oleksandr Tyshchenko, nd, Artem Mygaiev Hi Volodymyr, On 01/03/17 16:09, Julien Grall wrote: > On 01/03/17 14:13, Volodymyr Babchuk wrote: >> Hi, Julien > > Hi Volodymyr, > >> On 28 February 2017 at 15:51, Julien Grall <julien.grall@arm.com> wrote: >> >>>> This e-mail is sort of follow-up to the two threads: [1] (my thread >>>> about TEE interaction) and [2] (Edgar's thread regarding handling SMC >>>> calls in platform_hvc). I want to discuss more broad topic there. >>>> >>>> Obviously, there are growing number of SMC users and current state of >>>> SMC handling in Xen satisfies nobody. My team wants to handle SMCs in >>>> secure way, Xilinx wants to forward some calls directly to Secure >>>> Monitor, while allowing to handle other in userspace, etc. >>>> >>>> My proposition is to gather all requirements to SMC (and HVC) handling >>>> in one place (e.g. in this mail thread). After we' will have clear >>>> picture of what we want, we will be able to develop some solution, >>>> that will satisfy us all. At least, I hope so :) >>>> >>>> Also I want to remind, that there are ARM document called "SMC Calling >>>> Convention" [3]. According to it, any aarch64 hypervisor "must >>>> implement the Standard Secure and Hypervisor Service calls". At this >>>> moment XEN does not conform to this. >>>> >>>> So, lets get started with the requirements: >>>> 0. There are no much difference between SMC and HVC handling (at least >>>> according to SMCCC). >>>> 1. Hypervisor should at least provide own UUID and version while >>>> called by SMC/HVC >>> >>> >>> Do we need to reserve the UUID for Xen? >> Yes, I think so. But I don't know the procedure. Should be that UUID >> registered somewhere? > > I will chat internally about it and come back to you. Answering to myself. I should have read more carefully the spec. You don't need to reserve it, UUID should be uniq as long as you respect the algorithm to generate it (see [1]). So someone would need to make this up when posting the patch. Cheers, [1] https://tools.ietf.org/html/rfc4122 -- Julien Grall _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-03-01 16:09 ` Julien Grall 2017-03-01 16:18 ` Julien Grall @ 2017-03-01 16:47 ` Volodymyr Babchuk 1 sibling, 0 replies; 25+ messages in thread From: Volodymyr Babchuk @ 2017-03-01 16:47 UTC (permalink / raw) To: Julien Grall Cc: Edgar E. Iglesias, Andrii Anisov, Stefano Stabellini, Andrushchenko, Oleksandr, Tamas K Lengyel, Xen-devel, Oleksandr Tyshchenko, nd, Artem Mygaiev Hello Julien, On 1 March 2017 at 18:09, Julien Grall <julien.grall@arm.com> wrote: > On 01/03/17 14:13, Volodymyr Babchuk wrote: >>>> This e-mail is sort of follow-up to the two threads: [1] (my thread >>>> about TEE interaction) and [2] (Edgar's thread regarding handling SMC >>>> calls in platform_hvc). I want to discuss more broad topic there. >>>> >>>> Obviously, there are growing number of SMC users and current state of >>>> SMC handling in Xen satisfies nobody. My team wants to handle SMCs in >>>> secure way, Xilinx wants to forward some calls directly to Secure >>>> Monitor, while allowing to handle other in userspace, etc. >>>> >>>> My proposition is to gather all requirements to SMC (and HVC) handling >>>> in one place (e.g. in this mail thread). After we' will have clear >>>> picture of what we want, we will be able to develop some solution, >>>> that will satisfy us all. At least, I hope so :) >>>> >>>> Also I want to remind, that there are ARM document called "SMC Calling >>>> Convention" [3]. According to it, any aarch64 hypervisor "must >>>> implement the Standard Secure and Hypervisor Service calls". At this >>>> moment XEN does not conform to this. >>>> >>>> So, lets get started with the requirements: >>>> 0. There are no much difference between SMC and HVC handling (at least >>>> according to SMCCC). >>>> 1. Hypervisor should at least provide own UUID and version while >>>> called by SMC/HVC >>> >>> >>> >>> Do we need to reserve the UUID for Xen? >> >> Yes, I think so. But I don't know the procedure. Should be that UUID >> registered somewhere? > > > I will chat internally about it and come back to you. > >>> >>>> 2. Hypervisor should forward some calls from dom0 directly to Secure >>>> Monitor (Xilinx use case) >>>> 3. Hypervisor should virtualize PSCI calls, CPU service calls, ARM >>>> architecture service calls, etc. >>>> 4. Hypervisor should handle TEE calls in a secure way (e.g. no >>>> untrusted handlers in Dom0 userspace). >>>> 5. Hypervisor should support multiple TEEs (at least at compilation >>>> time). >>>> 6. Hypervisor should do this as fast as possible (DRM playback use >>>> case). >>>> 7. All domains (including dom0) should be handled in the same way. >>> >>> >>> >>> +1 here. Same path for all domains means less code and you it will get >>> tested more >> >> Yep. But this is somewhat incompatible with current implementation, as >> currently VM monitor mechanism is being used. > > > I am not sure to understand here. Can you detail it? Dom0 can't have VM monitor, because it is constructed by hypervisor. And hypervisor does not create VM monitor control structures for it. Actually, this is right thing, because probably don't want to allow another domain to control dom0. Current SMC handler uses to VM monitor subsystem to handle calt. Thus, we can't handle SMCs from Dom0 at this moment. -- WBR Volodymyr Babchuk aka lorc [+380976646013] mailto: vlad.babchuk@gmail.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-02-11 0:14 [ARM] SMC (and HVC) handling in hypervisor Volodymyr Babchuk 2017-02-13 16:20 ` Tamas K Lengyel 2017-02-28 13:51 ` Julien Grall @ 2017-03-02 17:54 ` Volodymyr Babchuk 2017-03-02 20:24 ` Tamas K Lengyel 2 siblings, 1 reply; 25+ messages in thread From: Volodymyr Babchuk @ 2017-03-02 17:54 UTC (permalink / raw) To: Xen-devel, Tamas K Lengyel, Edgar E. Iglesias, Julien Grall, Stefano Stabellini, Artem Mygaiev, Andrii Anisov, Oleksandr Tyshchenko, Andrushchenko, Oleksandr Hello, Thank you all for the discussion. I want to summarize it a bit. Looks like there are no objections about my initial list of requirements. There was born another requirement in the discussion. Thanks to Stefano for formulation it: 10. Domains on which the monitor privileged call feature is enabled (which is by default disabled for all domains) should not be able to issue firmware calls via SMCs/HVCs so that such calls reach the firmware directly. Xen should not bounce such calls to the firmware on behalf of the domain. Xen should not alter the state of the domain automatically (ie. incrementing PC). These calls should be exclusively transfered to the monitor subscriber for further processing. Hypercalls, virtual PSCI calls, virtual CPU services calls and virtual ARM architecture service calls remain unaffected. Tamas, you have agreed with this, right? So, looks like the HVC/SMC calls should be handled in the next way: 1. Virtualized calls (like vPSCI) will be handled by hypervisor 2a. If a VM Monitor is installed for a domain, remaining calls will be forwarded to the VM Monitor 2b. In another case, call will be forwarded to some other handler. In absence of handler, SMCs from dom0 will be forwarded to firmware. All other calls (HVCs or SMCs from domU) will cause Undefined Instruction (Xilinx usecase). Case 2b is to be discussed further. I like idea of EL0 handler, but as I said on community call, I'm still working on the prototype. Have I missed something? On 11 February 2017 at 02:14, Volodymyr Babchuk <vlad.babchuk@gmail.com> wrote: > Hello, > > This e-mail is sort of follow-up to the two threads: [1] (my thread > about TEE interaction) and [2] (Edgar's thread regarding handling SMC > calls in platform_hvc). I want to discuss more broad topic there. > > Obviously, there are growing number of SMC users and current state of > SMC handling in Xen satisfies nobody. My team wants to handle SMCs in > secure way, Xilinx wants to forward some calls directly to Secure > Monitor, while allowing to handle other in userspace, etc. > > My proposition is to gather all requirements to SMC (and HVC) handling > in one place (e.g. in this mail thread). After we' will have clear > picture of what we want, we will be able to develop some solution, > that will satisfy us all. At least, I hope so :) > > Also I want to remind, that there are ARM document called "SMC Calling > Convention" [3]. According to it, any aarch64 hypervisor "must > implement the Standard Secure and Hypervisor Service calls". At this > moment XEN does not conform to this. > > So, lets get started with the requirements: > 0. There are no much difference between SMC and HVC handling (at least > according to SMCCC). > 1. Hypervisor should at least provide own UUID and version while > called by SMC/HVC > 2. Hypervisor should forward some calls from dom0 directly to Secure > Monitor (Xilinx use case) > 3. Hypervisor should virtualize PSCI calls, CPU service calls, ARM > architecture service calls, etc. > 4. Hypervisor should handle TEE calls in a secure way (e.g. no > untrusted handlers in Dom0 userspace). > 5. Hypervisor should support multiple TEEs (at least at compilation time). > 6. Hypervisor should do this as fast as possible (DRM playback use case). > 7. All domains (including dom0) should be handled in the same way. > 8. Not all domains will have right to issue certain SMCs. > 9. Hypervisor will issue own SMCs in some cases. > > This is high-level requirements. Feel free to expand this list. > > Current SMC handling code does not even handle PSCI calls. Only HVC > trap handler have branch to handle PSCI calls. SMCs are forwarded to > VM monitor subsystem. There are even no advance_pc() call, so monitor > needs to advance PC by itself. Also, dom0 can't have monitor, so there > are no way to handle SMCs that originate from dom0. So, basically, > current code does not meet any requirements from above list. This > means that we can start from scratch and develop any solution. > > But at this moment I only want to gather requirements. So feel free to > point at what I have missed. > > [1] https://lists.xenproject.org/archives/html/xen-devel/2016-11/msg02220.html > [2] https://lists.xenproject.org/archives/html/xen-devel/2017-02/msg00635.html > [3] http://infocenter.arm.com/help/topic/com.arm.doc.den0028b/ARM_DEN0028B_SMC_Calling_Convention.pdf > -- > WBR Volodymyr Babchuk aka lorc [+380976646013] > mailto: vlad.babchuk@gmail.com -- WBR Volodymyr Babchuk aka lorc [+380976646013] mailto: vlad.babchuk@gmail.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [ARM] SMC (and HVC) handling in hypervisor 2017-03-02 17:54 ` Volodymyr Babchuk @ 2017-03-02 20:24 ` Tamas K Lengyel 0 siblings, 0 replies; 25+ messages in thread From: Tamas K Lengyel @ 2017-03-02 20:24 UTC (permalink / raw) To: Volodymyr Babchuk Cc: Edgar E. Iglesias, Stefano Stabellini, Andrushchenko, Oleksandr, Andrii Anisov, Oleksandr Tyshchenko, Julien Grall, Xen-devel, Artem Mygaiev On Thu, Mar 2, 2017 at 10:54 AM, Volodymyr Babchuk <vlad.babchuk@gmail.com> wrote: > Hello, > > Thank you all for the discussion. I want to summarize it a bit. > > Looks like there are no objections about my initial list of > requirements. There was born another requirement in the discussion. > Thanks to Stefano for formulation it: > > 10. Domains on which the monitor privileged call feature is enabled > (which is by default disabled for all domains) should not be able to > issue firmware calls via SMCs/HVCs so that such calls reach the > firmware directly. Xen should not bounce such calls to the firmware on > behalf of the domain. Xen should not alter the state of the domain > automatically (ie. incrementing PC). These calls should be exclusively > transfered to the monitor subscriber for further processing. > Hypercalls, virtual PSCI calls, virtual CPU services calls and virtual > ARM architecture service calls remain unaffected. > > Tamas, you have agreed with this, right? Yes, that would work for me. Tamas _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2017-03-02 20:24 UTC | newest] Thread overview: 25+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2017-02-11 0:14 [ARM] SMC (and HVC) handling in hypervisor Volodymyr Babchuk 2017-02-13 16:20 ` Tamas K Lengyel 2017-02-13 16:29 ` Volodymyr Babchuk 2017-02-13 16:37 ` Tamas K Lengyel 2017-02-13 16:37 ` Julien Grall 2017-02-13 16:59 ` Tamas K Lengyel 2017-02-13 18:06 ` Julien Grall 2017-02-13 19:54 ` Tamas K Lengyel 2017-02-13 21:13 ` Stefano Stabellini 2017-02-13 21:32 ` Tamas K Lengyel 2017-02-13 21:54 ` Stefano Stabellini 2017-02-13 22:11 ` Tamas K Lengyel 2017-02-13 22:14 ` Stefano Stabellini 2017-02-14 18:06 ` Julien Grall 2017-02-14 18:52 ` Tamas K Lengyel 2017-02-14 19:11 ` Stefano Stabellini 2017-02-14 20:14 ` Tamas K Lengyel 2017-02-23 18:16 ` Julien Grall 2017-02-28 13:51 ` Julien Grall 2017-03-01 14:13 ` Volodymyr Babchuk 2017-03-01 16:09 ` Julien Grall 2017-03-01 16:18 ` Julien Grall 2017-03-01 16:47 ` Volodymyr Babchuk 2017-03-02 17:54 ` Volodymyr Babchuk 2017-03-02 20:24 ` Tamas K Lengyel
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).