Re: [PATCH kernel v2] x86/compressed/64: reduce #VC nesting for intercepted CPUID for SEV-SNP guest

Tom Lendacky
2023-09-26T18:29:03Z

On 9/25/23 23:05, Alexey Kardashevskiy wrote:
> For certain intercepts an SNP guest uses the GHCB protocol to talk to
> the hypervisor from the #VC handler. The protocol requires a shared page so
> there is one per vCPU. In case NMI arrives in a middle of #VC or the NMI
> handler triggers a #VC, there is another "backup" GHCB page which stores
> the content of the first one while SVM_VMGEXIT_NMI_COMPLETE is sent.
> The vc_raw_handle_exception() handler manages main and backup GHCB pages
> via __sev_get_ghcb/__sev_put_ghcb.
>
> This works fine for #VC and occasional NMIs. This does not work so fine if
> the #VC handler causes intercept + another #VC, if NMI arrives during
> the second #VC, there are no more pages for SVM_VMGEXIT_NMI_COMPLETE.
> The problem place is the #VC CPUID handler. Running perf in the SNP guest
> crashes with:
>
> Kernel panic - not syncing: Unable to handle #VC exception! GHCB and Backup GHCB are already in use
>
> vc_raw_handle_exception #1: exit_code 72 (CPUID) eax d ecx 1
> We lock the main GHCB and while it is locked we get to
> snp_cpuid_postprocess() which executes "rdmsr" of MSR_IA32_XSS==0xda0 which
> triggers:
>
> vc_raw_handle_exception #2: exit_code 7c (MSR) ecx da0
> Here we lock the backup ghcb.
>
> And then PMC NMI comes which cannot complete as there is no GHCB page left
> to use:
>
> CPU: 5 PID: 566 Comm: touch Not tainted 6.5.0-rc2-aik-ad9c-g7413e71d3dcf-dirty #27
> Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS unknown unknown
> Call Trace:
> <NMI>
> dump_stack_lvl+0x44/0x60
> panic+0x222/0x310
> ____sev_get_ghcb+0x21e/0x220
> __sev_es_nmi_complete+0x28/0xf0
> exc_nmi+0x1ac/0x1c0
> end_repeat_nmi+0x16/0x67
> ...
> </NMI>
> <TASK>
> vc_raw_handle_exception+0x9e/0x2c0
> kernel_exc_vmm_communication+0x4d/0xa0
> asm_exc_vmm_communication+0x31/0x60
> RIP: 0010:snp_cpuid+0x2ad/0x420
>
> Add a helper similar to rdmsr_safe() for making a direct hypercall in the SEV-ES
> environment. Use the new helper instead of the raw "rdmsr" to avoid the extra
> #VC event.
>
> Fixes: ee0bfa08a345 ("x86/compressed/64: Add support for SEV-SNP CPUID table in #VC handlers")
> Signed-off-by: Alexey Kardashevskiy <aik@amd.com>

Acked-by: Tom Lendacky <thomas.lendacky@amd.com>

> ---
>
> This is made on top of (which has the "efi/unaccepted: Make sure unaccepted table is mapped"
> fix for booting SNP):
> b996cbe1203c (tip/master) 15 hours ago Ingo Molnar Merge branch into tip/master: 'x86/tdx'
>
> plus:
> https://lore.kernel.org/lkml/a5856fa1ebe3879de91a8f6298b6bbd901c61881.1690578565.git.thomas.lendacky@amd.com/
>
>
> rdmsr_safe_GHCB() is in arch/x86/include/asm/svm.h as this is where
> the ghcb struct is defined.
>
>
> ---
> Changes:
> v2:
> * de-uglify by defining rdmsr_safe_GHCB()
> ---
> arch/x86/include/asm/svm.h | 14 ++++++++++++++
> arch/x86/kernel/sev-shared.c | 5 +++--
> 2 files changed, 17 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/include/asm/svm.h b/arch/x86/include/asm/svm.h
> index fb8366af59da..866ef7d6b7a9 100644
> --- a/arch/x86/include/asm/svm.h
> +++ b/arch/x86/include/asm/svm.h
> @@ -673,4 +673,18 @@ DEFINE_GHCB_ACCESSORS(sw_exit_info_2)
> DEFINE_GHCB_ACCESSORS(sw_scratch)
> DEFINE_GHCB_ACCESSORS(xcr0)
>
> +/* Paravirt SEV-ES rdmsr which avoids extra #VC event */
> +#define rdmsr_safe_GHCB(msr, low, high, ghcb, ctxt) ({ \
> + int __ret; \
> + \
> + ghcb_set_rcx((ghcb), (msr)); \
> + __ret = sev_es_ghcb_hv_call((ghcb), (ctxt), SVM_EXIT_MSR, 0, 0); \
> + if (__ret == ES_OK) { \
> + low = (ghcb)->save.rax; \
> + high = (ghcb)->save.rdx; \
> + /* Invalidate qwords for likely another following GHCB call */ \
> + vc_ghcb_invalidate(ghcb); \
> + } \
> + __ret; })
> +
> #endif
> diff --git a/arch/x86/kernel/sev-shared.c b/arch/x86/kernel/sev-shared.c
> index e73c90c9cc5b..b3fb9d0a07c6 100644
> --- a/arch/x86/kernel/sev-shared.c
> +++ b/arch/x86/kernel/sev-shared.c
> @@ -479,8 +479,9 @@ static int snp_cpuid_postprocess(struct ghcb *ghcb, struct es_em_ctxt *ctxt,
> if (leaf->eax & BIT(3)) {
> unsigned long lo, hi;
>
> - asm volatile("rdmsr" : "=a" (lo), "=d" (hi)
> - : "c" (MSR_IA32_XSS));
> + if (rdmsr_safe_GHCB(MSR_IA32_XSS, lo, hi, ghcb, ctxt) != ES_OK)
> + return -EINVAL;
> +
> xss = (hi << 32) | lo;
> }
>