From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Huang, Kai" Subject: Re: [intel-sgx-kernel-dev] [PATCH 08/10] kvm: vmx: add guest's IA32_SGXLEPUBKEYHASHn runtime switch support Date: Tue, 23 May 2017 17:55:10 +1200 Message-ID: <932af102-e93a-6348-a7d2-0dbd2c5e6670@linux.intel.com> References: <20170508052434.3627-1-kai.huang@linux.intel.com> <20170508052434.3627-9-kai.huang@linux.intel.com> <58dcdb2d-6894-b0a3-8d6f-2ab752fd6d22@linux.intel.com> <20170515124622.piupyk57vjdoppl5@intel.com> <478d9303-00b7-4f29-6124-0c1433851952@linux.intel.com> <1495030889.23465.13.camel@intel.com> <24a2198f-fd0f-6db3-274d-3392a9037265@linux.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Sean Christopherson , Jarkko Sakkinen , Paolo Bonzini , haim.cohen@intel.com, "intel-sgx-kernel-dev@lists.01.org" , kvm list , Radim Krcmar To: Andy Lutomirski Return-path: Received: from mga06.intel.com ([134.134.136.31]:33522 "EHLO mga06.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752384AbdEWFzS (ORCPT ); Tue, 23 May 2017 01:55:18 -0400 In-Reply-To: <24a2198f-fd0f-6db3-274d-3392a9037265@linux.intel.com> Sender: kvm-owner@vger.kernel.org List-ID: On 5/23/2017 5:43 PM, Huang, Kai wrote: > > > On 5/21/2017 9:55 AM, Andy Lutomirski wrote: >> On Thu, May 18, 2017 at 1:14 AM, Huang, Kai >> wrote: >>> You are making assumption that KVM will run ENCLS on behalf of guest. :) >>> >>> If we don't need to look into guest's SIGSTRUCT, EINITTOKEN, etc, then I >>> actually prefer to using MTF, as with MTF we don't have to do all the >>> remapping guest's virtual address to KVM's virtual address thing, if we >>> don't need to look into guest's ENCLS parameter. But if we need to >>> look into >>> guest's ENCLS parameters, for example, to locate physical SECS page, >>> or to >>> update physical EPC page's info (that KVM needs to maintain), maybe >>> we can >>> choose running ENCLS on behalf of guest. >> >> After thinking about this a bit, I don't see how MTF helps. >> Currently, KVM works kind of like this: >> >> local_irq_disable(); >> set up stuff; >> VMRESUME; >> restore some host state; >> local_irq_enable(); >> >> If the guest is going to run with the EINIT-exiting bit clear, the >> only way I see this working is to modify KVM along the lines of: >> >> local_irq_disable(); >> set up stuff; >> if (condition here) { >> WRMSR to SGXLEPUBKEYHASH; >> update percpu shadow copyl >> clear EINIT-exiting bit; >> } else { >> set EINIT-exiting bit; >> } >> VMRESUME; >> restore some host state; >> local_irq_enable(); >> >> where "condition here" might be something like "the last VMRESUME >> exited due to EINIT". >> >> I don't see how MTF helps much. And if I were the KVM maintainer, I >> would probably prefer to trap EINIT instead of adding a special case >> to the main vm entry code. >> > > Hi Andy, > > Thanks for your comments. However I didn't intend to use MTF in your > way. The idea of using MTF (along with ENCLS VMEXIT) is, by turning on > MTF VMEXIT upon ENCLS VMEXIT, we are able to mark a single step VMEXIT > after ENCLS so that ENCLS can run in guest as single step. > > Let me explain how the two approaches work below in general, so that we > can decide which is better. Only trapping EINIT in order to update > IA32_SGXLEPUBKEYHASHn is relatively simpler but I'd compare the two in > more general way, assuming we may want to trap more ENCLS in order to, > ex, track EPC/Enclave status/info, in the future to support, ex, EPC > oversubscription between KVM guests. > > Below diagram shows the basic idea of the two approaches. > > > -------------------------------------------------------------- > | ENCLS | > -------------------------------------------------------------- > | /|\ > ENCLS VMEXIT | | VMENTRY > | | > \|/ | Looks the diagrams are broken. Sorry. I need to verify before sending next time. But looks they are still understandable? Thanks, Kai > > 1) identify which ENCLS leaf (RAX) > 2) reconstruct/remap guest's ENCLS parameters, ex: > - remap any guest VA (virtual address) to KVM VA > - reconstruct PAGEINFO > 3) do whatever needed before ENCLS, ex: > - updating MSRs before EINIT > 4) run ENCLS on behalf of guest, and skip ENCLS > 5) emulate ENCLS result (succeeded or not) > - update guest's RAX-RDX. > - and/or inject #GP (or #UD). > 6) do whatever needed after ENCLS, ex: > - updating EPC/Enclave status/info > > 1) Run ENCLS on behalf of guest > > > -------------------------------------------------------------- > | ENCLS | > -------------------------------------------------------------- > |/|\ |/|\ > ENCLS VMEXIT | | VMENTRY MTF VMEXIT | | VMENTRY > | | | | > \|/| \|/| > 1) Turn off EMCLS VMEXIT 1) Turn off MTF VMEXIT > 2) turn on MTF VMEXIT 2) Turn on ENCLS VMEXIT > 3) cache ENCLS parameters 3) check whether ENCLS has run > (ENCLS changes RAX) 4) check whether ENCLS succeeded > 4) do whatever needed before or not. > ENCLS 5) do whatever needed after ENCLS > > 2) Using MTF > > The concern of running ENCLS on behalf of guest is emulating ENCLS > error. KVM needs to *correctly* emulate ENCLS error to guest so that the > error we inject to guest can reflect the right behavior as if ENCLS run > in guest. Running ENCLS in root-mode may be potentially different > running ENCLS in non-root mode, therefore we have to go through all > possible error codes to make sure we can emulate. And for some error > code, ex, SGX_LOCKFAIL, we can handle it in KVM and don't have to inject > error to guest. So the point is we have to go through all error code to > make sure KVM can emulate ENCLS error code correctly for guest. > Another argument is Intel may add new error codes in the future when > more SGX functionalities are introduced, so emulating error code may be > a burden. > > Using MTF is also a little bit tricky, as when we turn on MTF VMEXIT > upon ENCLS VMEXIT, the MTF won't be absolutely pending at end of that > ENCLS. For example, MTF may be pending at end of interrupt (cannot > recall exactly) if event is pending during VMENTRY from ENCLS VMEXIT. > Therefore we have to do additional thing to check whether this MTF > VMEXIT really happens after ENCLS run (step 3 above). And depending on > what we need to do, we may need to check whether ENCLS succeeded or not > in guest, which is also tricky, as ENCLS can fail in either setting > error code in RAX, or generating #GP or #UD (step 4 above). We may still > need to do gva->gpa->hpa, ex, in order to locate EPC/SECS page and > update status, depending on the purpose of trapping ENCLS. > > But by using MTF, we don't have to worry about ENCLS error emulation, as > ENCLS runs in guest, thus we don't need to worry about this root-mode > and non-root mode difference. I think this is the major reason that we > want to use MTF. >