Re: [PATCH for_v23 4/5] x86/vdso: sgx: Reorder params to callback to improve readability

[Sean Christopherson <sean.j.christopherson@intel.com>]

On Wed, Oct 16, 2019 at 03:24:54PM -0700, Xing, Cedric wrote:
> On 10/10/2019 5:40 PM, Sean Christopherson wrote:
> >Swap @ret and @ursp in the callback prototype so that the output from
> >the vDSO itself, @ret and @e, are grouped together.  Having the first
> >N parameters all share a type also makes the prototype easier to parse
> >by (some) humans.  And, passing @ursp via register saves one whole
> >MOV instruction!
> >
> >Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
> >---
> >  arch/x86/entry/vdso/vsgx_enter_enclave.S | 16 +++++++---------
> >  arch/x86/include/uapi/asm/sgx.h          | 10 +++++-----
> >  2 files changed, 12 insertions(+), 14 deletions(-)
> >
> >diff --git a/arch/x86/entry/vdso/vsgx_enter_enclave.S b/arch/x86/entry/vdso/vsgx_enter_enclave.S
> >index 3dd22780b7ef..94f613b53b13 100644
> >--- a/arch/x86/entry/vdso/vsgx_enter_enclave.S
> >+++ b/arch/x86/entry/vdso/vsgx_enter_enclave.S
> >@@ -134,23 +134,21 @@ ENTRY(__vdso_sgx_enter_enclave)
> >  	jmp	.Lhandle_exit
> >  .Linvoke_userspace_handler:
> >+	/* Pass the untrusted RSP (at exit) to the callback via %rcx. */
> >+	mov	%rsp, %rcx
> >+
> >  	/*
> >-	 * Align stack per x86_64 ABI. Save the original %rsp in %rbx to be
> >-	 * restored after the callback returns.  Note, %rsp needs to be 16-byte
> >-	 * aligned _after_ pushing the three parameters on the stack.
> >+	 * Align stack per x86_64 ABI. Note, %rsp needs to be 16-byte aligned
> >+	 * _after_ pushing the three parameters on the stack.
> >  	 */
> >-	mov	%rsp, %rbx
> 
> Per x86_64 ABI, %rcx is _not_ preserved across function call. How are you
> going to restore the stack after callback returns?

Magic?

I completely spaced on restoring RSP, as evidenced by the fact that I
didn't even change the restoration code to use RCX.

> 
> >  	and	$-0x10, %rsp
> >  	sub	$0x8, %rsp
> 
> Usually compilers would just use a 1-byte "push" instead of a 4-byte "sub"
> instruction here.

Ah!  Took me a minute to understand 'N-byte' was referring to the opcode...

I'll update to use a push.

> 
> >-	/* Push @e, u_rsp and @tcs as parameters to the callback. */
> >+	/* Push @e, the "return" value and @tcs as params to the callback. */
> >  	push	0x18(%rbp)
> >-	push	%rbx
> >+	push	%rax
> >  	push	0x10(%rbp)
> >-	/* Pass the "return" value to the callback via %rcx. */
> >-	mov	%eax, %ecx
> >-
> >  	/* Clear RFLAGS.DF per x86_64 ABI */
> >  	cld
> >diff --git a/arch/x86/include/uapi/asm/sgx.h b/arch/x86/include/uapi/asm/sgx.h
> >index 255392f5054f..95f69f938b58 100644
> >--- a/arch/x86/include/uapi/asm/sgx.h
> >+++ b/arch/x86/include/uapi/asm/sgx.h
> >@@ -88,16 +88,16 @@ struct sgx_enclave_exception {
> >   * @rdi:	RDI at the time of enclave exit
> >   * @rsi:	RSI at the time of enclave exit
> >   * @rdx:	RDX at the time of enclave exit
> >- * @ret:	0 on success (EEXIT), -EFAULT on an exception
> >+ * @ursp:	RSP at the time of enclave exit (untrusted stack)
> >   * @r8:		R8 at the time of enclave exit
> >   * @r9:		R9 at the time of enclave exit
> >   * @tcs:	Thread Control Structure used to enter enclave
> >- * @ursp:	RSP at the time of enclave exit
> >+ * @ret:	0 on success (EEXIT), -EFAULT on an exception
> >   * @e:		Pointer to struct sgx_enclave_exception (as provided by caller)
> >   */
> >-typedef int (*sgx_enclave_exit_handler_t)(long rdi, long rsi, long rdx, int ret,
> >-					  long r8, long r9, void *tcs,
> >-					  long ursp,
> >+typedef int (*sgx_enclave_exit_handler_t)(long rdi, long rsi, long rdx,
> >+					  long ursp, long r8, long r9,
> >+					  void *tcs, int ret,
> >  					  struct sgx_enclave_exception *e);
> >  #endif /* _UAPI_ASM_X86_SGX_H */
> >