From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=MAILING_LIST_MULTI,SPF_PASS, USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 548A3C43387 for ; Fri, 21 Dec 2018 18:28:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 293E721920 for ; Fri, 21 Dec 2018 18:28:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391468AbeLUS2M (ORCPT ); Fri, 21 Dec 2018 13:28:12 -0500 Received: from mga04.intel.com ([192.55.52.120]:19051 "EHLO mga04.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2389359AbeLUS2L (ORCPT ); Fri, 21 Dec 2018 13:28:11 -0500 X-Amp-Result: UNKNOWN X-Amp-Original-Verdict: FILE UNKNOWN X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Dec 2018 10:28:10 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,382,1539673200"; d="scan'208";a="120289196" Received: from sjchrist-coffee.jf.intel.com (HELO linux.intel.com) ([10.54.74.154]) by FMSMGA003.fm.intel.com with ESMTP; 21 Dec 2018 10:28:09 -0800 Date: Fri, 21 Dec 2018 10:28:09 -0800 From: Sean Christopherson To: Jarkko Sakkinen Cc: Andy Lutomirski , Dave Hansen , X86 ML , Platform Driver , linux-sgx@vger.kernel.org, nhorman@redhat.com, npmccallum@redhat.com, "Ayoun, Serge" , shay.katz-zamir@intel.com, Haitao Huang , Andy Shevchenko , Thomas Gleixner , "Svahn, Kai" , mark.shanahan@intel.com, Suresh Siddha , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Darren Hart , Andy Shevchenko , "open list:X86 ARCHITECTURE (32-BIT AND 64-BIT)" Subject: Re: [PATCH v17 18/23] platform/x86: Intel SGX driver Message-ID: <20181221182809.GB27371@linux.intel.com> References: <20181217180102.GA12560@linux.intel.com> <20181217183613.GD12491@linux.intel.com> <20181217184333.GA26920@linux.intel.com> <20181217222047.GG12491@linux.intel.com> <20181218154417.GC28326@linux.intel.com> <20181218185349.GC30082@linux.intel.com> <20181219050047.GC14295@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20181219050047.GC14295@linux.intel.com> User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Dec 19, 2018 at 07:00:47AM +0200, Jarkko Sakkinen wrote: > On Tue, Dec 18, 2018 at 10:53:49AM -0800, Sean Christopherson wrote: > > What if we re-organize the ioctls in such a way that we leave open the > > possibility of allocating raw EPC for KVM via /dev/sgx? I'm not 100% > > positive this approach will work[1], but conceptually it fits well with > > KVM's memory model, e.g. KVM is aware of the GPA<->HVA association but > > generally speaking doesn't know what's physically backing each memory > > region. > > Why would you want to pass EPC through user space to KVM rather than > KVM allocating it through kernel interfaces? Delegating EPC management to userspace fits better with KVM's existing memory ABI. KVM provides a single ioctl(), KVM_SET_USER_MEMORY_REGION[1], that allows userspace to create, move, modify and delete memory regions. Skipping over a lot of details, there are essentially three options for exposing EPC to a KVM guest: 1) Provide a dedicated KVM ioctl() to manage EPC without routing it through KVM_SET_USER_MEMORY_REGION. 2) Add a flag to 'struct kvm_userspace_memory_region' that denotes an EPC memory region and mmap() / allocate EPC in KVM. 3) Provide an ABI to allocate raw EPC and let userspace manage it like any other memory region. Option (1) requires duplicating all of KVM_SET_USER_MEMORY_REGION's functionality unless the ioctl() is severly restricted. Option (2) is an ugly abuse of KVM_SET_USER_MEMORY_REGION since the EPC flag would have completely different semantics than all other usage of KVM_SET_USER_MEMORY_REGION. Thus, option (3). Probably a better question to answer is why provide the ABI through /dev/sgx and not /dev/kvm. IMO /dev/sgx is a more logical way to advertise support to userspace, e.g. userspace can simply check if /dev/sgx (or /dev/sgx/epc) exists vs. probing a KVM capability. Without EPC oversubscription in KVM, /dev/sgx is easily the best fit since the EPC management would reside completely in x86/sgx, i.e. KVM would essentially have zero code related to EPC management. EPC oversubscription complicates things because the architecture forces aspects of VMM oversubscription into the KVM domain, e.g. requires a post-VMXON instruction (ENCLV) and a VM-Exit handler. I still think /dev/sgx is a better fit, my only concern is that the oversubscription code would be even more heinous due to splitting responsibilities. But, Andy's idea of having /dev/sgx/enclave vs. /dev/sgx/epc might help avoid that entirely.