[PATCH v11 00/13] Intel SGX1 support

[PATCH v11 00/13] Intel SGX1 support

[Jarkko Sakkinen]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=y, Size: 13837 bytes --]

Intel(R) SGX is a set of CPU instructions that can be used by applications
to set aside private regions of code and data. The code outside the enclave
is disallowed to access the memory inside the enclave by the CPU access
control.  In a way you can think that SGX provides inverted sandbox. It
protects the application from a malicious host.

There is a new hardware unit in the processor called Memory Encryption
Engine (MEE) starting from the Skylake microacrhitecture. BIOS can define
one or many MEE regions that can hold enclave data by configuring them with
PRMRR registers.

The MEE automatically encrypts the data leaving the processor package to
the MEE regions. The data is encrypted using a random key whose life-time
is exactly one power cycle.

You can tell if your CPU supports SGX by looking into /proc/cpuinfo:

	cat /proc/cpuinfo  | grep sgx

v11:
* Polished ENCLS wrappers with refined exception handling.
* ksgxswapd was not stopped (regression in v5) in
  sgx_page_cache_teardown(), which causes a leaked kthread after driver
  deinitialization.
* Shutdown sgx_le_proxy when going to suspend because its EPC pages will be
  invalidated when resuming, which will cause it not function properly
  anymore.
* Set EINITTOKEN.VALID to zero for a token that is passed when
  SGXLEPUBKEYHASH matches MRSIGNER as alloc_page() does not give a zero
  page.
* Fixed the check in sgx_edbgrd() for a TCS page. Allowed to read offsets
  around the flags field, which causes a #GP. Only flags read is readable.
* On read access memcpy() call inside sgx_vma_access() had src and dest
  parameters in wrong order.
* The build issue with CONFIG_KASAN is now fixed. Added undefined symbols
  to LE even if “KASAN_SANITIZE := false” was set in the makefile. 
* Fixed a regression in the #PF handler. If a page has
  SGX_ENCL_PAGE_RESERVED flag the #PF handler should unconditionally fail.
  It did not, which caused weird races when trying to change other parts of
  swapping code.
* EPC management has been refactored to a flat LRU cache and moved to
  arch/x86. The swapper thread reads a cluster of EPC pages and swaps all
  of them. It can now swap from multiple enclaves in the same round.
* For the sake of consistency with SGX_IOC_ENCLAVE_ADD_PAGE, return -EINVAL
  when an enclave is already initialized or dead instead of zero.

v10:
* Cleaned up anon inode based IPC between the ring-0 and ring-3 parts
  of the driver.
* Unset the reserved flag from an enclave page if EDBGRD/WR fails
  (regression in v6).
* Close the anon inode when LE is stopped (regression in v9).
* Update the documentation with a more detailed description of SGX.

v9:
* Replaced kernel-LE IPC based on pipes with an anonymous inode.
  The driver does not require anymore new exports.

v8:
* Check that public key MSRs match the LE public key hash in the
  driver initialization when the MSRs are read-only.
* Fix the race in VA slot allocation by checking the fullness
  immediately after succeesful allocation.
* Fix the race in hash mrsigner calculation between the launch
  enclave and user enclaves by having a separate lock for hash
  calculation.

v7:
* Fixed offset calculation in sgx_edbgr/wr(). Address was masked with PAGE_MASK
  when it should have been masked with ~PAGE_MASK.
* Fixed a memory leak in sgx_ioc_enclave_create().
* Simplified swapping code by using a pointer array for a cluster
  instead of a linked list.
* Squeezed struct sgx_encl_page to 32 bytes.
* Fixed deferencing of an RSA key on OpenSSL 1.1.0.
* Modified TC's CMAC to use kernel AES-NI. Restructured the code
  a bit in order to better align with kernel conventions.

v6:
* Fixed semaphore underrun when accessing /dev/sgx from the launch enclave.
* In sgx_encl_create() s/IS_ERR(secs)/IS_ERR(encl)/.
* Removed virtualization chapter from the documentation.
* Changed the default filename for the signing key as signing_key.pem.
* Reworked EPC management in a way that instead of a linked list of
  struct sgx_epc_page instances there is an array of integers that
  encodes address and bank of an EPC page (the same data as 'pa' field
  earlier). The locking has been moved to the EPC bank level instead
  of a global lock.
* Relaxed locking requirements for EPC management. EPC pages can be
  released back to the EPC bank concurrently.
* Cleaned up ptrace() code.
* Refined commit messages for new architectural constants.
* Sorted includes in every source file.
* Sorted local variable declarations according to the line length in
  every function.
* Style fixes based on Darren's comments to sgx_le.c.

v5:
* Described IPC between the Launch Enclave and kernel in the commit messages.
* Fixed all relevant checkpatch.pl issues that I have forgot fix in earlier
  versions except those that exist in the imported TinyCrypt code.
* Fixed spelling mistakes in the documentation.
* Forgot to check the return value of sgx_drv_subsys_init().
* Encapsulated properly page cache init and teardown.
* Collect epc pages to a temp list in sgx_add_epc_bank
* Removed SGX_ENCLAVE_INIT_ARCH constant.

v4:
* Tied life-cycle of the sgx_le_proxy process to /dev/sgx.
* Removed __exit annotation from sgx_drv_subsys_exit().
* Fixed a leak of a backing page in sgx_process_add_page_req() in the
  case when vm_insert_pfn() fails.
* Removed unused symbol exports for sgx_page_cache.c.
* Updated sgx_alloc_page() to require encl parameter and documented the
  behavior (Sean Christopherson).
* Refactored a more lean API for sgx_encl_find() and documented the behavior.
* Moved #PF handler to sgx_fault.c.
* Replaced subsys_system_register() with plain bus_register().
* Retry EINIT 2nd time only if MSRs are not locked.

v3:
* Check that FEATURE_CONTROL_LOCKED and FEATURE_CONTROL_SGX_ENABLE are set.
* Return -ERESTARTSYS in __sgx_encl_add_page() when sgx_alloc_page() fails.
* Use unused bits in epc_page->pa to store the bank number.
* Removed #ifdef for WQ_NONREENTRANT.
* If mmu_notifier_register() fails with -EINTR, return -ERESTARTSYS.
* Added --remove-section=.got.plt to objcopy flags in order to prevent a
  dummy .got.plt, which will cause an inconsistent size for the LE.
* Documented sgx_encl_* functions.
* Added remark about AES implementation used inside the LE.
* Removed redundant sgx_sys_exit() from le/main.c.
* Fixed struct sgx_secinfo alignment from 128 to 64 bytes.
* Validate miscselect in sgx_encl_create().
* Fixed SSA frame size calculation to take the misc region into account.
* Implemented consistent exception handling to __encls() and __encls_ret().
* Implemented a proper device model in order to allow sysfs attributes
  and in-kernel API.
* Cleaned up various "find enclave" implementations to the unified
  sgx_encl_find().
* Validate that vm_pgoff is zero.
* Discard backing pages with shmem_truncate_range() after EADD.
* Added missing EEXTEND operations to LE signing and launch.
* Fixed SSA size for GPRS region from 168 to 184 bytes.
* Fixed the checks for TCS flags. Now DBGOPTIN is allowed.
* Check that TCS addresses are in ELRANGE and not just page aligned.
* Require kernel to be compiled with X64_64 and CPU_SUP_INTEL.
* Fixed an incorrect value for SGX_ATTR_DEBUG from 0x01 to 0x02.

v2:
* get_rand_uint32() changed the value of the pointer instead of value
  where it is pointing at.
* Launch enclave incorrectly used sigstruct attributes-field instead of
  enclave attributes-field.
* Removed unused struct sgx_add_page_req from sgx_ioctl.c
* Removed unused sgx_has_sgx2.
* Updated arch/x86/include/asm/sgx.h so that it provides stub
  implementations when sgx in not enabled.
* Removed cruft rdmsr-calls from sgx_set_pubkeyhash_msrs().
* return -ENOMEM in sgx_alloc_page() when VA pages consume too much space
* removed unused global sgx_nr_pids
* moved sgx_encl_release to sgx_encl.c
* return -ERESTARTSYS instead of -EINTR in sgx_encl_init()


Jarkko Sakkinen (7):
  x86, sgx: updated MAINTAINERS
  x86, sgx: added ENCLS wrappers
  x86, sgx: basic routines for enclave page cache
  intel_sgx: driver for Intel Software Guard Extensions
  intel_sgx: ptrace() support
  intel_sgx: driver documentation
  intel_sgx: in-kernel launch enclave

Kai Huang (1):
  x86, sgx: add SGX definitions to cpufeature

Sean Christopherson (5):
  compiler.h, kasan: add __SANITIZE_ADDRESS__ check for
    __no_kasan_or_inline
  x86, sgx: add SGX definitions to msr-index.h
  x86, cpufeatures: add Intel-defined SGX leaf CPUID_12_EAX
  crypto: aesni: add minimal build option for SGX LE
  x86, sgx: detect Intel SGX

 Documentation/index.rst                       |   1 +
 Documentation/x86/intel_sgx.rst               | 195 ++++
 MAINTAINERS                                   |   7 +
 arch/x86/Kconfig                              |  19 +
 arch/x86/crypto/aesni-intel_asm.S             |  11 +
 arch/x86/include/asm/cpufeature.h             |   7 +-
 arch/x86/include/asm/cpufeatures.h            |  10 +-
 arch/x86/include/asm/disabled-features.h      |   3 +-
 arch/x86/include/asm/msr-index.h              |   8 +
 arch/x86/include/asm/required-features.h      |   3 +-
 arch/x86/include/asm/sgx.h                    | 289 +++++
 arch/x86/include/asm/sgx_arch.h               | 224 ++++
 arch/x86/include/asm/sgx_le.h                 |  17 +
 arch/x86/include/asm/sgx_pr.h                 |  36 +
 arch/x86/include/uapi/asm/sgx.h               | 138 +++
 arch/x86/kernel/cpu/Makefile                  |   1 +
 arch/x86/kernel/cpu/common.c                  |   7 +
 arch/x86/kernel/cpu/intel_sgx.c               | 492 +++++++++
 arch/x86/kvm/cpuid.h                          |   1 +
 drivers/platform/x86/Kconfig                  |   2 +
 drivers/platform/x86/Makefile                 |   1 +
 drivers/platform/x86/intel_sgx/Kconfig        |  34 +
 drivers/platform/x86/intel_sgx/Makefile       |  32 +
 drivers/platform/x86/intel_sgx/le/Makefile    |  34 +
 .../x86/intel_sgx/le/enclave/Makefile         |  54 +
 .../intel_sgx/le/enclave/aesni-intel_asm.S    |   1 +
 .../x86/intel_sgx/le/enclave/cmac_mode.c      | 209 ++++
 .../x86/intel_sgx/le/enclave/cmac_mode.h      |  54 +
 .../x86/intel_sgx/le/enclave/encl_bootstrap.S | 116 ++
 .../platform/x86/intel_sgx/le/enclave/main.c  | 146 +++
 .../platform/x86/intel_sgx/le/enclave/main.h  |  19 +
 .../x86/intel_sgx/le/enclave/sgx_le.lds       |  33 +
 .../x86/intel_sgx/le/enclave/sgxsign.c        | 551 ++++++++++
 .../x86/intel_sgx/le/enclave/string.c         |   1 +
 drivers/platform/x86/intel_sgx/le/entry.S     |  70 ++
 .../x86/intel_sgx/le/include/sgx_asm.h        |  15 +
 drivers/platform/x86/intel_sgx/le/main.c      | 140 +++
 drivers/platform/x86/intel_sgx/le/main.h      |  30 +
 .../platform/x86/intel_sgx/le/sgx_le_piggy.S  |  22 +
 drivers/platform/x86/intel_sgx/le/string.c    |  39 +
 drivers/platform/x86/intel_sgx/sgx.h          | 181 ++++
 drivers/platform/x86/intel_sgx/sgx_encl.c     | 988 ++++++++++++++++++
 .../platform/x86/intel_sgx/sgx_encl_page.c    | 294 ++++++
 drivers/platform/x86/intel_sgx/sgx_fault.c    | 159 +++
 drivers/platform/x86/intel_sgx/sgx_ioctl.c    | 235 +++++
 drivers/platform/x86/intel_sgx/sgx_le.c       | 303 ++++++
 .../x86/intel_sgx/sgx_le_proxy_piggy.S        |  22 +
 drivers/platform/x86/intel_sgx/sgx_main.c     | 373 +++++++
 drivers/platform/x86/intel_sgx/sgx_vma.c      | 182 ++++
 include/linux/compiler.h                      |   2 +-
 50 files changed, 5805 insertions(+), 6 deletions(-)
 create mode 100644 Documentation/x86/intel_sgx.rst
 create mode 100644 arch/x86/include/asm/sgx.h
 create mode 100644 arch/x86/include/asm/sgx_arch.h
 create mode 100644 arch/x86/include/asm/sgx_le.h
 create mode 100644 arch/x86/include/asm/sgx_pr.h
 create mode 100644 arch/x86/include/uapi/asm/sgx.h
 create mode 100644 arch/x86/kernel/cpu/intel_sgx.c
 create mode 100644 drivers/platform/x86/intel_sgx/Kconfig
 create mode 100644 drivers/platform/x86/intel_sgx/Makefile
 create mode 100644 drivers/platform/x86/intel_sgx/le/Makefile
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/Makefile
 create mode 120000 drivers/platform/x86/intel_sgx/le/enclave/aesni-intel_asm.S
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/cmac_mode.c
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/cmac_mode.h
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/encl_bootstrap.S
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/main.c
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/main.h
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/sgx_le.lds
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/sgxsign.c
 create mode 120000 drivers/platform/x86/intel_sgx/le/enclave/string.c
 create mode 100644 drivers/platform/x86/intel_sgx/le/entry.S
 create mode 100644 drivers/platform/x86/intel_sgx/le/include/sgx_asm.h
 create mode 100644 drivers/platform/x86/intel_sgx/le/main.c
 create mode 100644 drivers/platform/x86/intel_sgx/le/main.h
 create mode 100644 drivers/platform/x86/intel_sgx/le/sgx_le_piggy.S
 create mode 100644 drivers/platform/x86/intel_sgx/le/string.c
 create mode 100644 drivers/platform/x86/intel_sgx/sgx.h
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_encl.c
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_encl_page.c
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_fault.c
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_ioctl.c
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_le.c
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_le_proxy_piggy.S
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_main.c
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_vma.c

-- 
2.17.0

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v11 12/13] intel_sgx: driver documentation

[Jarkko Sakkinen]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=y, Size: 9854 bytes --]

Documentation of the features of the  Software Guard eXtensions usable
for the Linux kernel and how the driver internals uses these features.
In addition, contains documentation for the ioctl API.

Signed-off-by: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>
---
 Documentation/index.rst         |   1 +
 Documentation/x86/intel_sgx.rst | 195 ++++++++++++++++++++++++++++++++
 2 files changed, 196 insertions(+)
 create mode 100644 Documentation/x86/intel_sgx.rst

diff --git a/Documentation/index.rst b/Documentation/index.rst
index 3b99ab931d41..b9fb92928e8c 100644
--- a/Documentation/index.rst
+++ b/Documentation/index.rst
@@ -100,6 +100,7 @@ implementation.
    :maxdepth: 2
 
    sh/index
+   x86/index
 
 Korean translations
 -------------------
diff --git a/Documentation/x86/intel_sgx.rst b/Documentation/x86/intel_sgx.rst
new file mode 100644
index 000000000000..ecbe544eb2cb
--- /dev/null
+++ b/Documentation/x86/intel_sgx.rst
@@ -0,0 +1,195 @@
+===================
+Intel(R) SGX driver
+===================
+
+Introduction
+============
+
+Intel(R) SGX is a set of CPU instructions that can be used by applications to
+set aside private regions of code and data. The code outside the enclave is
+disallowed to access the memory inside the enclave by the CPU access control.
+In a way you can think that SGX provides inverted sandbox. It protects the
+application from a malicious host.
+
+You can tell if your CPU supports SGX by looking into ``/proc/cpuinfo``:
+
+	``cat /proc/cpuinfo  | grep sgx``
+
+Overview of SGX
+===============
+
+SGX has a set of data structures to maintain information about the enclaves and
+their security properties. BIOS reserves a fixed size region of physical memory
+for these structures by setting Processor Reserved Memory Range Registers
+(PRMRR).
+
+This memory range is protected from outside access by the CPU and all the data
+coming in and out of the CPU package is encrypted by a key that is generated for
+each boot cycle.
+
+Enclaves execute in ring-3 in a special enclave submode using pages from the
+reserved memory range. A fixed logical address range for the enclave is reserved
+by ENCLS(ECREATE), a leaf instruction used to create enclaves. It is referred in
+the documentation commonly as the ELRANGE.
+
+Every memory access to the ELRANGE is asserted by the CPU. If the CPU is not
+executing in the enclave mode inside the enclave, #GP is raised. On the other
+hand enclave code can make memory accesses both inside and outside of the
+ELRANGE.
+
+Enclave can only execute code inside the ELRANGE. Instructions that may cause
+VMEXIT, IO instructions and instructions that require a privilege change are
+prohibited inside the enclave. Interrupts and exceptions always cause enclave
+to exit and jump to an address outside the enclave given when the enclave is
+entered by using the leaf instruction ENCLS(EENTER).
+
+Data types
+----------
+
+The protected memory range contains the following data:
+
+* **Enclave Page Cache (EPC):** protected pages
+* **Enclave Page Cache Map (EPCM):** a database that describes the state of the
+  pages and link them to an enclave.
+
+EPC has a number of different types of pages:
+
+* **SGX Enclave Control Structure (SECS)**: describes the global
+  properties of an enclave.
+* **Regular (REG):** code and data pages in the ELRANGE.
+* **Thread Control Structure (TCS):** pages that define entry points inside an
+  enclave. The enclave can only be entered through these entry points and each
+  can host a single hardware thread at a time.
+* **Version Array (VA)**: 64-bit version numbers for pages that have been
+  swapped outside the enclave. Each page contains 512 version numbers.
+
+Launch control
+--------------
+
+To launch an enclave, two structures must be provided for ENCLS(EINIT):
+
+1. **SIGSTRUCT:** signed measurement of the enclave binary.
+2. **EINITTOKEN:** a cryptographic token CMAC-signed with a AES256-key called
+   *launch key*, which is re-generated for each boot cycle.
+
+The CPU holds a SHA256 hash of a 3072-bit RSA public key inside
+IA32_SGXLEPUBKEYHASHn MSRs. Enclaves with a SIGSTRUCT that is signed with this
+key do not require a valid EINITTOKEN and can be authorized with special
+privileges. One of those privileges is ability to acquire the launch key with
+ENCLS(EGETKEY).
+
+**IA32_FEATURE_CONTROL[17]** is used by to BIOS configure whether
+IA32_SGXLEPUBKEYHASH MSRs are read-only or read-write before locking the
+feature control register and handing over control to the operating system.
+
+Enclave construction
+--------------------
+
+The construction is started by filling out the SECS that contains enclave
+address range, privileged attributes and measurement of TCS and REG pages (pages
+that will be mapped to the address range) among the other things. This structure
+is passed out to the ENCLS(ECREATE) together with a physical address of a page
+in EPC that will hold the SECS.
+
+Then pages are added with ENCLS(EADD) and measured with ENCLS(EEXTEND).  Finally
+enclave is initialized with ENCLS(EINIT). ENCLS(INIT) checks that the SIGSTRUCT
+is signed with the contained public key and that the supplied EINITTOKEN is
+valid (CMAC'd with the launch key). If these hold, the enclave is successfully
+initialized.
+
+Swapping pages
+--------------
+
+Enclave pages can be swapped out with ENCLS(EWB) to the unprotected memory. In
+addition to the EPC page, ENCLS(EWB) takes in a VA page and address for PCMD
+structure (Page Crypto MetaData) as input. The VA page will seal a version
+number for the page. PCMD is 128 byte structure that contains tracking
+information for the page, most importantly its MAC. With these structures the
+enclave is sealed and rollback protected while it resides in the unprotected
+memory.
+
+Before the page can be swapped out it must not have any active TLB references.
+By using ENCLS(EBLOCK) instructions no new TLB entries can be created to it.
+After this the a counter called *epoch* associated hardware threads inside the
+enclave is increased with ENCLS(ETRACK). After all the threads from the previous
+epoch have exited the page can be safely swapped out.
+
+An enclave memory access to a swapped out pages will cause #PF. #PF handler can
+fault the page back by using ENCLS(ELDU).
+
+Kernel internals
+================
+
+Requirements
+------------
+
+Because SGX has an ever evolving and expanding feature set, it's possible for
+a BIOS or VMM to configure a system in such a way that not all cpus are equal,
+e.g. where Launch Control is only enabled on a subset of cpus.  Linux does
+*not* support such a heterogenous system configuration, nor does it even
+attempt to play nice in the face of a misconfigured system.  With the exception
+of Launch Control's hash MSRs, which can vary per cpu, Linux assumes that all
+cpus have a configuration that is identical to the boot cpu.
+
+
+Roles and responsibilities
+--------------------------
+
+SGX introduces system resources, e.g. EPC memory, that must be accessible to
+multiple entities, e.g. the native kernel driver (to expose SGX to userspace)
+and KVM (to expose SGX to VMs), ideally without introducing any dependencies
+between each SGX entity.  To that end, the kernel owns and manages the shared
+system resources, i.e. the EPC and Launch Control MSRs, and defines functions
+that provide appropriate access to the shared resources.  SGX support for
+userpace and VMs is left to the SGX platform driver and KVM respectively.
+
+Launching enclaves
+------------------
+
+For privileged enclaves the launch is performed simply by submitting the
+SIGSTRUCT for that enclave to ENCLS(EINIT). For unprivileged enclaves the
+driver hosts a process in ring-3 that hosts a launch enclave signed with a key
+supplied for kbuild.
+
+The current implementation of the launch enclave generates a token for any
+enclave. In the future it could be potentially extended to have ways to
+configure policy what can be lauched.
+
+The driver will fail to initialize if it cannot start its own launch enclave.
+A user space application can submit a SIGSTRUCT instance through the ioctl API.
+The kernel will take care of the rest.
+
+This design assures that the Linux kernel has always full control, which
+enclaves get to launch and which do not, even if the public key MSRs are
+read-only. Having launch intrinsics inside the kernel also enables easy
+development of enclaves without necessarily needing any heavy weight SDK.
+Having a low-barrier to implement enclaves could make sense for example for
+system daemons where amount of dependecies ought to be minimized.
+
+EPC management
+--------------
+
+Due to the unique requirements for swapping EPC pages, and because EPC pages
+(currently) do not have associated page structures, management of the EPC is
+not handled by the standard Linux swapper.  SGX directly handles swapping
+of EPC pages, including a kthread to initiate reclaim and a rudimentary LRU
+mechanism.  Consumsers of EPC pages, e.g. the SGX driver, are required to
+implement function callbacks that can be invoked by the kernel to age,
+swap, and/or forcefully reclaim a target EPC page.  In effect, the kernel
+controls what happens and when, while the consumers (driver, KVM, etc..) do
+the actual work.
+
+SGX uapi
+========
+
+.. kernel-doc:: drivers/platform/x86/intel_sgx/sgx_ioctl.c
+   :functions: sgx_ioc_enclave_create
+               sgx_ioc_enclave_add_page
+               sgx_ioc_enclave_init
+
+.. kernel-doc:: arch/x86/include/uapi/asm/sgx.h
+
+References
+==========
+
+* System Programming Manual: 39.1.4 Intel® SGX Launch Control Configuration
-- 
2.17.0

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v11 12/13] intel_sgx: driver documentation

[Jethro Beekman]

[-- Attachment #1: Type: text/plain, Size: 1511 bytes --]

On 2018-06-08 10:09, Jarkko Sakkinen wrote:
> +Launching enclaves
> +------------------
> +
> +For privileged enclaves the launch is performed simply by submitting the
> +SIGSTRUCT for that enclave to ENCLS(EINIT). For unprivileged enclaves the
> +driver hosts a process in ring-3 that hosts a launch enclave signed with a key
> +supplied for kbuild.
> +
> +The current implementation of the launch enclave generates a token for any
> +enclave. In the future it could be potentially extended to have ways to
> +configure policy what can be lauched.
> +
> +The driver will fail to initialize if it cannot start its own launch enclave.
> +A user space application can submit a SIGSTRUCT instance through the ioctl API.
> +The kernel will take care of the rest.
> +
> +This design assures that the Linux kernel has always full control, which
> +enclaves get to launch and which do not, even if the public key MSRs are

As discussed previously at length, since the kernel needs to execute 
ENCLS[EINIT], it has full control to deny the launching of enclaves 
regardless of any launch enclave implementation. Please change this 
misleading statement.

> +read-only. Having launch intrinsics inside the kernel also enables easy
> +development of enclaves without necessarily needing any heavy weight SDK.
> +Having a low-barrier to implement enclaves could make sense for example for
> +system daemons where amount of dependecies ought to be minimized.

--
Jethro Beekman | Fortanix


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3994 bytes --]

Re: [PATCH v11 12/13] intel_sgx: driver documentation

[Randy Dunlap]

On 06/08/2018 10:09 AM, Jarkko Sakkinen wrote:
> Documentation of the features of the  Software Guard eXtensions usable
> for the Linux kernel and how the driver internals uses these features.
> In addition, contains documentation for the ioctl API.
> 
> Signed-off-by: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>

Hi,

I have a few corrections below...


> ---
>  Documentation/index.rst         |   1 +
>  Documentation/x86/intel_sgx.rst | 195 ++++++++++++++++++++++++++++++++
>  2 files changed, 196 insertions(+)
>  create mode 100644 Documentation/x86/intel_sgx.rst
> 
> diff --git a/Documentation/index.rst b/Documentation/index.rst
> index 3b99ab931d41..b9fb92928e8c 100644
> --- a/Documentation/index.rst
> +++ b/Documentation/index.rst
> @@ -100,6 +100,7 @@ implementation.
>     :maxdepth: 2
>  
>     sh/index
> +   x86/index
>  
>  Korean translations
>  -------------------
> diff --git a/Documentation/x86/intel_sgx.rst b/Documentation/x86/intel_sgx.rst
> new file mode 100644
> index 000000000000..ecbe544eb2cb
> --- /dev/null
> +++ b/Documentation/x86/intel_sgx.rst
> @@ -0,0 +1,195 @@
> +===================
> +Intel(R) SGX driver
> +===================
> +
> +Introduction
> +============
> +
> +Intel(R) SGX is a set of CPU instructions that can be used by applications to
> +set aside private regions of code and data. The code outside the enclave is
> +disallowed to access the memory inside the enclave by the CPU access control.
> +In a way you can think that SGX provides inverted sandbox. It protects the
> +application from a malicious host.
> +
> +You can tell if your CPU supports SGX by looking into ``/proc/cpuinfo``:
> +
> +	``cat /proc/cpuinfo  | grep sgx``
> +
> +Overview of SGX
> +===============
> +
> +SGX has a set of data structures to maintain information about the enclaves and
> +their security properties. BIOS reserves a fixed size region of physical memory
> +for these structures by setting Processor Reserved Memory Range Registers
> +(PRMRR).
> +
> +This memory range is protected from outside access by the CPU and all the data
> +coming in and out of the CPU package is encrypted by a key that is generated for
> +each boot cycle.
> +
> +Enclaves execute in ring-3 in a special enclave submode using pages from the
> +reserved memory range. A fixed logical address range for the enclave is reserved
> +by ENCLS(ECREATE), a leaf instruction used to create enclaves. It is referred in
> +the documentation commonly as the ELRANGE.
> +
> +Every memory access to the ELRANGE is asserted by the CPU. If the CPU is not
> +executing in the enclave mode inside the enclave, #GP is raised. On the other
> +hand enclave code can make memory accesses both inside and outside of the
> +ELRANGE.
> +
> +Enclave can only execute code inside the ELRANGE. Instructions that may cause
> +VMEXIT, IO instructions and instructions that require a privilege change are
> +prohibited inside the enclave. Interrupts and exceptions always cause enclave
> +to exit and jump to an address outside the enclave given when the enclave is
> +entered by using the leaf instruction ENCLS(EENTER).
> +
> +Data types
> +----------
> +
> +The protected memory range contains the following data:
> +
> +* **Enclave Page Cache (EPC):** protected pages
> +* **Enclave Page Cache Map (EPCM):** a database that describes the state of the
> +  pages and link them to an enclave.
> +
> +EPC has a number of different types of pages:
> +
> +* **SGX Enclave Control Structure (SECS)**: describes the global
> +  properties of an enclave.
> +* **Regular (REG):** code and data pages in the ELRANGE.
> +* **Thread Control Structure (TCS):** pages that define entry points inside an
> +  enclave. The enclave can only be entered through these entry points and each
> +  can host a single hardware thread at a time.
> +* **Version Array (VA)**: 64-bit version numbers for pages that have been
> +  swapped outside the enclave. Each page contains 512 version numbers.
> +
> +Launch control
> +--------------
> +
> +To launch an enclave, two structures must be provided for ENCLS(EINIT):
> +
> +1. **SIGSTRUCT:** signed measurement of the enclave binary.
> +2. **EINITTOKEN:** a cryptographic token CMAC-signed with a AES256-key called
> +   *launch key*, which is re-generated for each boot cycle.
> +
> +The CPU holds a SHA256 hash of a 3072-bit RSA public key inside
> +IA32_SGXLEPUBKEYHASHn MSRs. Enclaves with a SIGSTRUCT that is signed with this
> +key do not require a valid EINITTOKEN and can be authorized with special
> +privileges. One of those privileges is ability to acquire the launch key with
> +ENCLS(EGETKEY).
> +
> +**IA32_FEATURE_CONTROL[17]** is used by to BIOS configure whether

                                        by the BIOS to configure whether

> +IA32_SGXLEPUBKEYHASH MSRs are read-only or read-write before locking the
> +feature control register and handing over control to the operating system.
> +
> +Enclave construction
> +--------------------
> +
> +The construction is started by filling out the SECS that contains enclave
> +address range, privileged attributes and measurement of TCS and REG pages (pages
> +that will be mapped to the address range) among the other things. This structure
> +is passed out to the ENCLS(ECREATE) together with a physical address of a page
> +in EPC that will hold the SECS.
> +
> +Then pages are added with ENCLS(EADD) and measured with ENCLS(EEXTEND).  Finally

"measured"?  what does that mean?

> +enclave is initialized with ENCLS(EINIT). ENCLS(INIT) checks that the SIGSTRUCT
> +is signed with the contained public key and that the supplied EINITTOKEN is
> +valid (CMAC'd with the launch key). If these hold, the enclave is successfully
> +initialized.
> +
> +Swapping pages
> +--------------
> +
> +Enclave pages can be swapped out with ENCLS(EWB) to the unprotected memory. In
> +addition to the EPC page, ENCLS(EWB) takes in a VA page and address for PCMD
> +structure (Page Crypto MetaData) as input. The VA page will seal a version
> +number for the page. PCMD is 128 byte structure that contains tracking
> +information for the page, most importantly its MAC. With these structures the
> +enclave is sealed and rollback protected while it resides in the unprotected
> +memory.
> +
> +Before the page can be swapped out it must not have any active TLB references.
> +By using ENCLS(EBLOCK) instructions no new TLB entries can be created to it.
> +After this the a counter called *epoch* associated hardware threads inside the

huh?

> +enclave is increased with ENCLS(ETRACK). After all the threads from the previous
> +epoch have exited the page can be safely swapped out.
> +
> +An enclave memory access to a swapped out pages will cause #PF. #PF handler can
> +fault the page back by using ENCLS(ELDU).
> +
> +Kernel internals
> +================
> +
> +Requirements
> +------------
> +
> +Because SGX has an ever evolving and expanding feature set, it's possible for
> +a BIOS or VMM to configure a system in such a way that not all cpus are equal,

                                                                  CPUs

> +e.g. where Launch Control is only enabled on a subset of cpus.  Linux does

                                                            CPUs.

> +*not* support such a heterogenous system configuration, nor does it even

                        heterogeneous

> +attempt to play nice in the face of a misconfigured system.  With the exception
> +of Launch Control's hash MSRs, which can vary per cpu, Linux assumes that all

                                                     CPU,

> +cpus have a configuration that is identical to the boot cpu.

   CPUs                                                    CPU.

> +
> +
> +Roles and responsibilities
> +--------------------------
> +
> +SGX introduces system resources, e.g. EPC memory, that must be accessible to
> +multiple entities, e.g. the native kernel driver (to expose SGX to userspace)
> +and KVM (to expose SGX to VMs), ideally without introducing any dependencies
> +between each SGX entity.  To that end, the kernel owns and manages the shared
> +system resources, i.e. the EPC and Launch Control MSRs, and defines functions
> +that provide appropriate access to the shared resources.  SGX support for
> +userpace and VMs is left to the SGX platform driver and KVM respectively.

   userspace

> +
> +Launching enclaves
> +------------------
> +
> +For privileged enclaves the launch is performed simply by submitting the
> +SIGSTRUCT for that enclave to ENCLS(EINIT). For unprivileged enclaves the
> +driver hosts a process in ring-3 that hosts a launch enclave signed with a key
> +supplied for kbuild.
> +
> +The current implementation of the launch enclave generates a token for any
> +enclave. In the future it could be potentially extended to have ways to
> +configure policy what can be lauched.

                                launched.

> +
> +The driver will fail to initialize if it cannot start its own launch enclave.
> +A user space application can submit a SIGSTRUCT instance through the ioctl API.
> +The kernel will take care of the rest.
> +
> +This design assures that the Linux kernel has always full control, which
> +enclaves get to launch and which do not, even if the public key MSRs are
> +read-only. Having launch intrinsics inside the kernel also enables easy
> +development of enclaves without necessarily needing any heavy weight SDK.
> +Having a low-barrier to implement enclaves could make sense for example for

            low barrier

> +system daemons where amount of dependecies ought to be minimized.

                                  dependencies

> +
> +EPC management
> +--------------
> +
> +Due to the unique requirements for swapping EPC pages, and because EPC pages
> +(currently) do not have associated page structures, management of the EPC is
> +not handled by the standard Linux swapper.  SGX directly handles swapping
> +of EPC pages, including a kthread to initiate reclaim and a rudimentary LRU
> +mechanism.  Consumsers of EPC pages, e.g. the SGX driver, are required to

               Consumers

> +implement function callbacks that can be invoked by the kernel to age,
> +swap, and/or forcefully reclaim a target EPC page.  In effect, the kernel
> +controls what happens and when, while the consumers (driver, KVM, etc..) do
> +the actual work.
> +
> +SGX uapi
> +========
> +
> +.. kernel-doc:: drivers/platform/x86/intel_sgx/sgx_ioctl.c
> +   :functions: sgx_ioc_enclave_create
> +               sgx_ioc_enclave_add_page
> +               sgx_ioc_enclave_init
> +
> +.. kernel-doc:: arch/x86/include/uapi/asm/sgx.h
> +
> +References
> +==========
> +
> +* System Programming Manual: 39.1.4 Intel® SGX Launch Control Configuration
> 


-- 
~Randy
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v11 00/13] Intel SGX1 support

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 769 bytes --]

On Fri 2018-06-08 19:09:35, Jarkko Sakkinen wrote:
> Intel(R) SGX is a set of CPU instructions that can be used by applications
> to set aside private regions of code and data. The code outside the enclave
> is disallowed to access the memory inside the enclave by the CPU access
> control.  In a way you can think that SGX provides inverted sandbox. It
> protects the application from a malicious host.

Do you intend to allow non-root applications to use SGX?

What are non-evil uses for SGX?

...because it is quite useful for some kinds of evil:

https://taesoo.kim/pubs/2017/jang:sgx-bomb.pdf

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [PATCH v11 12/13] intel_sgx: driver documentation

[Jarkko Sakkinen]

On Fri, Jun 08, 2018 at 11:32:17AM -0700, Jethro Beekman wrote:
> On 2018-06-08 10:09, Jarkko Sakkinen wrote:
> > +Launching enclaves
> > +------------------
> > +
> > +For privileged enclaves the launch is performed simply by submitting the
> > +SIGSTRUCT for that enclave to ENCLS(EINIT). For unprivileged enclaves the
> > +driver hosts a process in ring-3 that hosts a launch enclave signed with a key
> > +supplied for kbuild.
> > +
> > +The current implementation of the launch enclave generates a token for any
> > +enclave. In the future it could be potentially extended to have ways to
> > +configure policy what can be lauched.
> > +
> > +The driver will fail to initialize if it cannot start its own launch enclave.
> > +A user space application can submit a SIGSTRUCT instance through the ioctl API.
> > +The kernel will take care of the rest.
> > +
> > +This design assures that the Linux kernel has always full control, which
> > +enclaves get to launch and which do not, even if the public key MSRs are
> 
> As discussed previously at length, since the kernel needs to execute
> ENCLS[EINIT], it has full control to deny the launching of enclaves
> regardless of any launch enclave implementation. Please change this
> misleading statement.

Remember the discussion, forgot to change it. I'll fix this for the next
version.

/Jarkko
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v11 12/13] intel_sgx: driver documentation

[Jarkko Sakkinen]

On Fri, Jun 08, 2018 at 02:41:07PM -0700, Randy Dunlap wrote:
> On 06/08/2018 10:09 AM, Jarkko Sakkinen wrote:
> > Documentation of the features of the  Software Guard eXtensions usable
> > for the Linux kernel and how the driver internals uses these features.
> > In addition, contains documentation for the ioctl API.
> > 
> > Signed-off-by: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>
> 
> Hi,
> 
> I have a few corrections below...
> 
> 
> > ---
> >  Documentation/index.rst         |   1 +
> >  Documentation/x86/intel_sgx.rst | 195 ++++++++++++++++++++++++++++++++
> >  2 files changed, 196 insertions(+)
> >  create mode 100644 Documentation/x86/intel_sgx.rst
> > 
> > diff --git a/Documentation/index.rst b/Documentation/index.rst
> > index 3b99ab931d41..b9fb92928e8c 100644
> > --- a/Documentation/index.rst
> > +++ b/Documentation/index.rst
> > @@ -100,6 +100,7 @@ implementation.
> >     :maxdepth: 2
> >  
> >     sh/index
> > +   x86/index
> >  
> >  Korean translations
> >  -------------------
> > diff --git a/Documentation/x86/intel_sgx.rst b/Documentation/x86/intel_sgx.rst
> > new file mode 100644
> > index 000000000000..ecbe544eb2cb
> > --- /dev/null
> > +++ b/Documentation/x86/intel_sgx.rst
> > @@ -0,0 +1,195 @@
> > +===================
> > +Intel(R) SGX driver
> > +===================
> > +
> > +Introduction
> > +============
> > +
> > +Intel(R) SGX is a set of CPU instructions that can be used by applications to
> > +set aside private regions of code and data. The code outside the enclave is
> > +disallowed to access the memory inside the enclave by the CPU access control.
> > +In a way you can think that SGX provides inverted sandbox. It protects the
> > +application from a malicious host.
> > +
> > +You can tell if your CPU supports SGX by looking into ``/proc/cpuinfo``:
> > +
> > +	``cat /proc/cpuinfo  | grep sgx``
> > +
> > +Overview of SGX
> > +===============
> > +
> > +SGX has a set of data structures to maintain information about the enclaves and
> > +their security properties. BIOS reserves a fixed size region of physical memory
> > +for these structures by setting Processor Reserved Memory Range Registers
> > +(PRMRR).
> > +
> > +This memory range is protected from outside access by the CPU and all the data
> > +coming in and out of the CPU package is encrypted by a key that is generated for
> > +each boot cycle.
> > +
> > +Enclaves execute in ring-3 in a special enclave submode using pages from the
> > +reserved memory range. A fixed logical address range for the enclave is reserved
> > +by ENCLS(ECREATE), a leaf instruction used to create enclaves. It is referred in
> > +the documentation commonly as the ELRANGE.
> > +
> > +Every memory access to the ELRANGE is asserted by the CPU. If the CPU is not
> > +executing in the enclave mode inside the enclave, #GP is raised. On the other
> > +hand enclave code can make memory accesses both inside and outside of the
> > +ELRANGE.
> > +
> > +Enclave can only execute code inside the ELRANGE. Instructions that may cause
> > +VMEXIT, IO instructions and instructions that require a privilege change are
> > +prohibited inside the enclave. Interrupts and exceptions always cause enclave
> > +to exit and jump to an address outside the enclave given when the enclave is
> > +entered by using the leaf instruction ENCLS(EENTER).
> > +
> > +Data types
> > +----------
> > +
> > +The protected memory range contains the following data:
> > +
> > +* **Enclave Page Cache (EPC):** protected pages
> > +* **Enclave Page Cache Map (EPCM):** a database that describes the state of the
> > +  pages and link them to an enclave.
> > +
> > +EPC has a number of different types of pages:
> > +
> > +* **SGX Enclave Control Structure (SECS)**: describes the global
> > +  properties of an enclave.
> > +* **Regular (REG):** code and data pages in the ELRANGE.
> > +* **Thread Control Structure (TCS):** pages that define entry points inside an
> > +  enclave. The enclave can only be entered through these entry points and each
> > +  can host a single hardware thread at a time.
> > +* **Version Array (VA)**: 64-bit version numbers for pages that have been
> > +  swapped outside the enclave. Each page contains 512 version numbers.
> > +
> > +Launch control
> > +--------------
> > +
> > +To launch an enclave, two structures must be provided for ENCLS(EINIT):
> > +
> > +1. **SIGSTRUCT:** signed measurement of the enclave binary.
> > +2. **EINITTOKEN:** a cryptographic token CMAC-signed with a AES256-key called
> > +   *launch key*, which is re-generated for each boot cycle.
> > +
> > +The CPU holds a SHA256 hash of a 3072-bit RSA public key inside
> > +IA32_SGXLEPUBKEYHASHn MSRs. Enclaves with a SIGSTRUCT that is signed with this
> > +key do not require a valid EINITTOKEN and can be authorized with special
> > +privileges. One of those privileges is ability to acquire the launch key with
> > +ENCLS(EGETKEY).
> > +
> > +**IA32_FEATURE_CONTROL[17]** is used by to BIOS configure whether
> 
>                                         by the BIOS to configure whether
> 
> > +IA32_SGXLEPUBKEYHASH MSRs are read-only or read-write before locking the
> > +feature control register and handing over control to the operating system.
> > +
> > +Enclave construction
> > +--------------------
> > +
> > +The construction is started by filling out the SECS that contains enclave
> > +address range, privileged attributes and measurement of TCS and REG pages (pages
> > +that will be mapped to the address range) among the other things. This structure
> > +is passed out to the ENCLS(ECREATE) together with a physical address of a page
> > +in EPC that will hold the SECS.
> > +
> > +Then pages are added with ENCLS(EADD) and measured with ENCLS(EEXTEND).  Finally
> 
> "measured"?  what does that mean?
> 
> > +enclave is initialized with ENCLS(EINIT). ENCLS(INIT) checks that the SIGSTRUCT
> > +is signed with the contained public key and that the supplied EINITTOKEN is
> > +valid (CMAC'd with the launch key). If these hold, the enclave is successfully
> > +initialized.
> > +
> > +Swapping pages
> > +--------------
> > +
> > +Enclave pages can be swapped out with ENCLS(EWB) to the unprotected memory. In
> > +addition to the EPC page, ENCLS(EWB) takes in a VA page and address for PCMD
> > +structure (Page Crypto MetaData) as input. The VA page will seal a version
> > +number for the page. PCMD is 128 byte structure that contains tracking
> > +information for the page, most importantly its MAC. With these structures the
> > +enclave is sealed and rollback protected while it resides in the unprotected
> > +memory.
> > +
> > +Before the page can be swapped out it must not have any active TLB references.
> > +By using ENCLS(EBLOCK) instructions no new TLB entries can be created to it.
> > +After this the a counter called *epoch* associated hardware threads inside the
> 
> huh?
> 
> > +enclave is increased with ENCLS(ETRACK). After all the threads from the previous
> > +epoch have exited the page can be safely swapped out.
> > +
> > +An enclave memory access to a swapped out pages will cause #PF. #PF handler can
> > +fault the page back by using ENCLS(ELDU).
> > +
> > +Kernel internals
> > +================
> > +
> > +Requirements
> > +------------
> > +
> > +Because SGX has an ever evolving and expanding feature set, it's possible for
> > +a BIOS or VMM to configure a system in such a way that not all cpus are equal,
> 
>                                                                   CPUs
> 
> > +e.g. where Launch Control is only enabled on a subset of cpus.  Linux does
> 
>                                                             CPUs.
> 
> > +*not* support such a heterogenous system configuration, nor does it even
> 
>                         heterogeneous
> 
> > +attempt to play nice in the face of a misconfigured system.  With the exception
> > +of Launch Control's hash MSRs, which can vary per cpu, Linux assumes that all
> 
>                                                      CPU,
> 
> > +cpus have a configuration that is identical to the boot cpu.
> 
>    CPUs                                                    CPU.
> 
> > +
> > +
> > +Roles and responsibilities
> > +--------------------------
> > +
> > +SGX introduces system resources, e.g. EPC memory, that must be accessible to
> > +multiple entities, e.g. the native kernel driver (to expose SGX to userspace)
> > +and KVM (to expose SGX to VMs), ideally without introducing any dependencies
> > +between each SGX entity.  To that end, the kernel owns and manages the shared
> > +system resources, i.e. the EPC and Launch Control MSRs, and defines functions
> > +that provide appropriate access to the shared resources.  SGX support for
> > +userpace and VMs is left to the SGX platform driver and KVM respectively.
> 
>    userspace
> 
> > +
> > +Launching enclaves
> > +------------------
> > +
> > +For privileged enclaves the launch is performed simply by submitting the
> > +SIGSTRUCT for that enclave to ENCLS(EINIT). For unprivileged enclaves the
> > +driver hosts a process in ring-3 that hosts a launch enclave signed with a key
> > +supplied for kbuild.
> > +
> > +The current implementation of the launch enclave generates a token for any
> > +enclave. In the future it could be potentially extended to have ways to
> > +configure policy what can be lauched.
> 
>                                 launched.
> 
> > +
> > +The driver will fail to initialize if it cannot start its own launch enclave.
> > +A user space application can submit a SIGSTRUCT instance through the ioctl API.
> > +The kernel will take care of the rest.
> > +
> > +This design assures that the Linux kernel has always full control, which
> > +enclaves get to launch and which do not, even if the public key MSRs are
> > +read-only. Having launch intrinsics inside the kernel also enables easy
> > +development of enclaves without necessarily needing any heavy weight SDK.
> > +Having a low-barrier to implement enclaves could make sense for example for
> 
>             low barrier
> 
> > +system daemons where amount of dependecies ought to be minimized.
> 
>                                   dependencies
> 
> > +
> > +EPC management
> > +--------------
> > +
> > +Due to the unique requirements for swapping EPC pages, and because EPC pages
> > +(currently) do not have associated page structures, management of the EPC is
> > +not handled by the standard Linux swapper.  SGX directly handles swapping
> > +of EPC pages, including a kthread to initiate reclaim and a rudimentary LRU
> > +mechanism.  Consumsers of EPC pages, e.g. the SGX driver, are required to
> 
>                Consumers
> 
> > +implement function callbacks that can be invoked by the kernel to age,
> > +swap, and/or forcefully reclaim a target EPC page.  In effect, the kernel
> > +controls what happens and when, while the consumers (driver, KVM, etc..) do
> > +the actual work.
> > +
> > +SGX uapi
> > +========
> > +
> > +.. kernel-doc:: drivers/platform/x86/intel_sgx/sgx_ioctl.c
> > +   :functions: sgx_ioc_enclave_create
> > +               sgx_ioc_enclave_add_page
> > +               sgx_ioc_enclave_init
> > +
> > +.. kernel-doc:: arch/x86/include/uapi/asm/sgx.h
> > +
> > +References
> > +==========
> > +
> > +* System Programming Manual: 39.1.4 Intel® SGX Launch Control Configuration
> > 
> 
> 
> -- 
> ~Randy

Thank you, I'll refine the parts that you pointed out for the next
version.

/Jarkko
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v11 00/13] Intel SGX1 support

[Jarkko Sakkinen]

On Tue, Jun 12, 2018 at 12:50:12PM +0200, Pavel Machek wrote:
> On Fri 2018-06-08 19:09:35, Jarkko Sakkinen wrote:
> > Intel(R) SGX is a set of CPU instructions that can be used by applications
> > to set aside private regions of code and data. The code outside the enclave
> > is disallowed to access the memory inside the enclave by the CPU access
> > control.  In a way you can think that SGX provides inverted sandbox. It
> > protects the application from a malicious host.
> 
> Do you intend to allow non-root applications to use SGX?
> 
> What are non-evil uses for SGX?
> 
> ...because it is quite useful for some kinds of evil:

The default permissions for the device are 600.

/Jarkko
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v11 00/13] Intel SGX1 support

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1050 bytes --]

On Tue 2018-06-19 17:59:43, Jarkko Sakkinen wrote:
> On Tue, Jun 12, 2018 at 12:50:12PM +0200, Pavel Machek wrote:
> > On Fri 2018-06-08 19:09:35, Jarkko Sakkinen wrote:
> > > Intel(R) SGX is a set of CPU instructions that can be used by applications
> > > to set aside private regions of code and data. The code outside the enclave
> > > is disallowed to access the memory inside the enclave by the CPU access
> > > control.  In a way you can think that SGX provides inverted sandbox. It
> > > protects the application from a malicious host.
> > 
> > Do you intend to allow non-root applications to use SGX?
> > 
> > What are non-evil uses for SGX?
> > 
> > ...because it is quite useful for some kinds of evil:
> 
> The default permissions for the device are 600.

Good. This does not belong to non-root. But question still remains:

What are some non-evil uses for SGX?

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [PATCH v11 00/13] Intel SGX1 support

[Peter Zijlstra]

On Tue, Jun 19, 2018 at 10:04:15PM +0200, Pavel Machek wrote:
> What are some non-evil uses for SGX?

Soft-TPM is one that was proposed at some time.
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v11 00/13] Intel SGX1 support

[Peter Zijlstra]

*sigh*, could you not cross-post with closed/moderated lists, that's rude.
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v11 00/13] Intel SGX1 support

[Josh Triplett]

On Tue, Jun 19, 2018 at 10:04:15PM +0200, Pavel Machek wrote:
> On Tue 2018-06-19 17:59:43, Jarkko Sakkinen wrote:
> > On Tue, Jun 12, 2018 at 12:50:12PM +0200, Pavel Machek wrote:
> > > On Fri 2018-06-08 19:09:35, Jarkko Sakkinen wrote:
> > > > Intel(R) SGX is a set of CPU instructions that can be used by applications
> > > > to set aside private regions of code and data. The code outside the enclave
> > > > is disallowed to access the memory inside the enclave by the CPU access
> > > > control.  In a way you can think that SGX provides inverted sandbox. It
> > > > protects the application from a malicious host.
> > > 
> > > Do you intend to allow non-root applications to use SGX?
> > > 
> > > What are non-evil uses for SGX?
> > > 
> > > ...because it is quite useful for some kinds of evil:
> > 
> > The default permissions for the device are 600.
> 
> Good. This does not belong to non-root.

There are entirely legitimate use cases for using this as an
unprivileged user. However, that'll be up to system and distribution
policy, which can evolve over time, and it makes sense for the *initial*
kernel permission to start out root-only and then adjust permissions via
udev.

> What are some non-evil uses for SGX?

Building a software certificate store. Hardening key-agent software like
ssh-agent or gpg-agent. Building a challenge-response authentication
system. Providing more assurance that your server infrastructure is
uncompromised. Offloading computation to a system without having to
fully trust that system.

As one of many possibilities, imagine a distcc that didn't have to trust
the compile nodes. The compile nodes could fail to return results at
all, but they couldn't alter the results.
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v11 00/13] Intel SGX1 support

[Ingo Molnar]

* Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com> wrote:

>  create mode 100644 drivers/platform/x86/intel_sgx/Kconfig
>  create mode 100644 drivers/platform/x86/intel_sgx/Makefile
>  create mode 100644 drivers/platform/x86/intel_sgx/le/Makefile
>  create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/Makefile
>  create mode 120000 drivers/platform/x86/intel_sgx/le/enclave/aesni-intel_asm.S
>  create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/cmac_mode.c
>  create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/cmac_mode.h
>  create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/encl_bootstrap.S
>  create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/main.c
>  create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/main.h
>  create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/sgx_le.lds
>  create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/sgxsign.c
>  create mode 120000 drivers/platform/x86/intel_sgx/le/enclave/string.c
>  create mode 100644 drivers/platform/x86/intel_sgx/le/entry.S
>  create mode 100644 drivers/platform/x86/intel_sgx/le/include/sgx_asm.h
>  create mode 100644 drivers/platform/x86/intel_sgx/le/main.c
>  create mode 100644 drivers/platform/x86/intel_sgx/le/main.h
>  create mode 100644 drivers/platform/x86/intel_sgx/le/sgx_le_piggy.S
>  create mode 100644 drivers/platform/x86/intel_sgx/le/string.c
>  create mode 100644 drivers/platform/x86/intel_sgx/sgx.h
>  create mode 100644 drivers/platform/x86/intel_sgx/sgx_encl.c
>  create mode 100644 drivers/platform/x86/intel_sgx/sgx_encl_page.c
>  create mode 100644 drivers/platform/x86/intel_sgx/sgx_fault.c
>  create mode 100644 drivers/platform/x86/intel_sgx/sgx_ioctl.c
>  create mode 100644 drivers/platform/x86/intel_sgx/sgx_le.c
>  create mode 100644 drivers/platform/x86/intel_sgx/sgx_le_proxy_piggy.S
>  create mode 100644 drivers/platform/x86/intel_sgx/sgx_main.c
>  create mode 100644 drivers/platform/x86/intel_sgx/sgx_vma.c

Just some quick feedback: these are core kernel functionality and should be in 
arch/x86/, not in drivers/platform/.

Thanks,

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v11 00/13] Intel SGX1 support

[Jarkko Sakkinen]

On Thu, 2018-06-21 at 14:55 +0200, Ingo Molnar wrote:
> Just some quick feedback: these are core kernel functionality and should be
> in 
> arch/x86/, not in drivers/platform/.

We have in core what is used by both KVM and the driver right now (EPC
management, SGX detection and some other bits). The driver uses this
functionality to launch enclaves.

> Thanks,
> 
> 	Ingo

/Jarkko
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v11 00/13] Intel SGX1 support

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 2362 bytes --]

Hi!

(sorry to bring up old thread).

> > > > > Intel(R) SGX is a set of CPU instructions that can be used by applications
> > > > > to set aside private regions of code and data. The code outside the enclave
> > > > > is disallowed to access the memory inside the enclave by the CPU access
> > > > > control.  In a way you can think that SGX provides inverted sandbox. It
> > > > > protects the application from a malicious host.
> > > > 
> > > > Do you intend to allow non-root applications to use SGX?
> > > > 
> > > > What are non-evil uses for SGX?
> > > > 
> > > > ...because it is quite useful for some kinds of evil:
> > > 
> > > The default permissions for the device are 600.
> > 
> > Good. This does not belong to non-root.
> 
> There are entirely legitimate use cases for using this as an
> unprivileged user. However, that'll be up to system and distribution
> policy, which can evolve over time, and it makes sense for the *initial*
> kernel permission to start out root-only and then adjust permissions via
> udev.

Agreed.

> > What are some non-evil uses for SGX?
> 
> Building a software certificate store. Hardening key-agent software like
> ssh-agent or gpg-agent. Building a challenge-response authentication
> system. Providing more assurance that your server infrastructure is
> uncompromised. Offloading computation to a system without having to
> fully trust that system.

I think you can do the crypto stuff... as crypto already verifies the
results. But I don't think you can do the computation offload.

> As one of many possibilities, imagine a distcc that didn't have to trust
> the compile nodes. The compile nodes could fail to return results at
> all, but they couldn't alter the results.

distcc on untrusted nodes ... oh yes, that would be great.

Except that you can't do it, right? :-).

First, AFAICT it would be quite hard to get gcc to run under SGX. But
maybe you have spare month or three and can do it.

But ... you really can't guarantee getting right results. Evil owner
of the machine might intentionaly overheat the CPU, glitch the power,
induce single-bit errors using gamma source, ... You can't do it.

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [PATCH v11 00/13] Intel SGX1 support

[Josh Triplett]

On Sun, Dec 09, 2018 at 09:06:00PM +0100, Pavel Machek wrote:
...
> > > > The default permissions for the device are 600.
> > > 
> > > Good. This does not belong to non-root.
> > 
> > There are entirely legitimate use cases for using this as an
> > unprivileged user. However, that'll be up to system and distribution
> > policy, which can evolve over time, and it makes sense for the *initial*
> > kernel permission to start out root-only and then adjust permissions via
> > udev.
> 
> Agreed.
> 
> > Building a software certificate store. Hardening key-agent software like
> > ssh-agent or gpg-agent. Building a challenge-response authentication
> > system. Providing more assurance that your server infrastructure is
> > uncompromised. Offloading computation to a system without having to
> > fully trust that system.
> 
> I think you can do the crypto stuff... as crypto already verifies the
> results. But I don't think you can do the computation offload.

You can, as long as you can do attestation.

> > As one of many possibilities, imagine a distcc that didn't have to trust
> > the compile nodes. The compile nodes could fail to return results at
> > all, but they couldn't alter the results.
> 
> distcc on untrusted nodes ... oh yes, that would be great.
> 
> Except that you can't do it, right? :-).
> 
> First, AFAICT it would be quite hard to get gcc to run under SGX. But
> maybe you have spare month or three and can do it.

Assuming you don't need to #include files, gcc seems quite simple to run
in an enclave: data in, computation inside, data out.

Re: [PATCH v11 00/13] Intel SGX1 support

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 2220 bytes --]

On Sun 2018-12-09 23:47:17, Josh Triplett wrote:
> On Sun, Dec 09, 2018 at 09:06:00PM +0100, Pavel Machek wrote:
> ...
> > > > > The default permissions for the device are 600.
> > > > 
> > > > Good. This does not belong to non-root.
> > > 
> > > There are entirely legitimate use cases for using this as an
> > > unprivileged user. However, that'll be up to system and distribution
> > > policy, which can evolve over time, and it makes sense for the *initial*
> > > kernel permission to start out root-only and then adjust permissions via
> > > udev.
> > 
> > Agreed.
> > 
> > > Building a software certificate store. Hardening key-agent software like
> > > ssh-agent or gpg-agent. Building a challenge-response authentication
> > > system. Providing more assurance that your server infrastructure is
> > > uncompromised. Offloading computation to a system without having to
> > > fully trust that system.
> > 
> > I think you can do the crypto stuff... as crypto already verifies the
> > results. But I don't think you can do the computation offload.
> 
> You can, as long as you can do attestation.

You can not, because random errors are very easy to trigger for person
with physical access, as I explained in the part of email you
stripped.

> > > As one of many possibilities, imagine a distcc that didn't have to trust
> > > the compile nodes. The compile nodes could fail to return results at
> > > all, but they couldn't alter the results.
> > 
> > distcc on untrusted nodes ... oh yes, that would be great.
> > 
> > Except that you can't do it, right? :-).
> > 
> > First, AFAICT it would be quite hard to get gcc to run under SGX. But
> > maybe you have spare month or three and can do it.
> 
> Assuming you don't need to #include files, gcc seems quite simple to run
> in an enclave: data in, computation inside, data out.

So is there a plan to run dynamically linked binaries inside enclave?
Or maybe even python/shell scripts? It looked to me like virtual
memory will be "interesting" for enclaves.


								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [PATCH v11 00/13] Intel SGX1 support

[Josh Triplett]

On Mon, Dec 10, 2018 at 09:27:04AM +0100, Pavel Machek wrote:
> On Sun 2018-12-09 23:47:17, Josh Triplett wrote:
> > On Sun, Dec 09, 2018 at 09:06:00PM +0100, Pavel Machek wrote:
> > ...
> > > > > > The default permissions for the device are 600.
> > > > > 
> > > > > Good. This does not belong to non-root.
> > > > 
> > > > There are entirely legitimate use cases for using this as an
> > > > unprivileged user. However, that'll be up to system and distribution
> > > > policy, which can evolve over time, and it makes sense for the *initial*
> > > > kernel permission to start out root-only and then adjust permissions via
> > > > udev.
> > > 
> > > Agreed.
> > > 
> > > > Building a software certificate store. Hardening key-agent software like
> > > > ssh-agent or gpg-agent. Building a challenge-response authentication
> > > > system. Providing more assurance that your server infrastructure is
> > > > uncompromised. Offloading computation to a system without having to
> > > > fully trust that system.
> > > 
> > > I think you can do the crypto stuff... as crypto already verifies the
> > > results. But I don't think you can do the computation offload.
> > 
> > You can, as long as you can do attestation.
> 
> You can not, because random errors are very easy to trigger for person
> with physical access,

Random errors can also just happen, so if you're concerned about that
you might want to build each object on two different machines and
compare. Good luck generating the *same* random errors on two machines.

(And, of course, someone can also DoS you in any number of other ways,
such as accepting data and then never sending back a result. So you'll
need timeouts and failovers.)

> > > > As one of many possibilities, imagine a distcc that didn't have to trust
> > > > the compile nodes. The compile nodes could fail to return results at
> > > > all, but they couldn't alter the results.
> > > 
> > > distcc on untrusted nodes ... oh yes, that would be great.
> > > 
> > > Except that you can't do it, right? :-).
> > > 
> > > First, AFAICT it would be quite hard to get gcc to run under SGX. But
> > > maybe you have spare month or three and can do it.
> > 
> > Assuming you don't need to #include files, gcc seems quite simple to run
> > in an enclave: data in, computation inside, data out.
> 
> So is there a plan to run dynamically linked binaries inside enclave?

I've seen some approaches for that, but you could also just statically
link your compiler. (Since you'd need attestation for all the individual
libraries, you'd need to know the versions of all those libraries, so
you might as well just statically link.)

> Or maybe even python/shell scripts? It looked to me like virtual
> memory will be "interesting" for enclaves.

Memory management doesn't seem that hard to deal with.

Re: [PATCH v11 00/13] Intel SGX1 support

[Dave Hansen]

On 12/10/18 3:12 PM, Josh Triplett wrote:
>> Or maybe even python/shell scripts? It looked to me like virtual
>> memory will be "interesting" for enclaves.
> Memory management doesn't seem that hard to deal with.

The problems are:

1. SGX enclave memory (EPC) is statically allocated at boot and can't
   grow or shrink
2. EPC is much smaller than regular RAM
3. The core VM has no comprehension of EPC use, thus can not help
   with its algorithms, like the LRU
4. The SGX driver implements its own VM which is substantially simpler
   than the core VM, but less feature-rich, fast, or scalable

Re: [PATCH v11 00/13] Intel SGX1 support

[Sean Christopherson]

On Tue, Dec 11, 2018 at 10:10:38AM -0800, Dave Hansen wrote:
> On 12/10/18 3:12 PM, Josh Triplett wrote:
> >> Or maybe even python/shell scripts? It looked to me like virtual
> >> memory will be "interesting" for enclaves.
> > Memory management doesn't seem that hard to deal with.
> 
> The problems are:
> 
> 1. SGX enclave memory (EPC) is statically allocated at boot and can't
>    grow or shrink
> 2. EPC is much smaller than regular RAM
> 3. The core VM has no comprehension of EPC use, thus can not help
>    with its algorithms, like the LRU
> 4. The SGX driver implements its own VM which is substantially simpler
>    than the core VM, but less feature-rich, fast, or scalable

I'd also add:

  5. Swapping EPC pages can only be done through SGX specific ISA that
     has strict concurrency requirements and enforces TLB flushing.
  6. There are specialized types of EPC pages that have different
     swapping requirements than regular EPC pages.
  7. EPC pages that are exposed to a KVM guest have yet another set of
     swapping requirements.

In other words, extending the core VM to SGX EPC is painfully difficult.