[PATCH v2 0/4] Only export SME/SEV/SEV_ES features when activated

[PATCH v2 0/4] Only export SME/SEV/SEV_ES features when activated

[Mario Limonciello]

Currently SME/SEV/SEV_ES feature flags will export if the CPU supports
them but even if the kernel has not enabled the features for any reason.

To let userspace react to these feature flags, only set them when the
kernel has activated them.

changes from v1->v2:
 * Squash old patches 1/2 together
 * Add new patches to KVM/CCP code to use `cc_platform_has` instead
 * Don't use `cc_platform_has` in the early code, just check `sme_me_mask`
   and `sev_status` directly.
 * Split out SME and SEV/SEV_ES clearing as separate patches in case
   SEV/SEV_ES need other more involved work that what is in this series so
   far.

Mario Limonciello (4):
  crypto: ccp: Use cc_platform_has to determine SEV presence
  KVM: SVM: Use cc_platform_has instead of CPU feature flags
  x86/cpu: clear SME features when not in use
  x86/cpu: clear SEV/SEV_ES features when not in use

 arch/x86/kernel/cpu/amd.c    | 10 ++++++++++
 arch/x86/kernel/process.c    |  5 ++++-
 arch/x86/kvm/svm/sev.c       |  4 ++--
 drivers/crypto/ccp/sev-dev.c |  2 +-
 4 files changed, 17 insertions(+), 4 deletions(-)

-- 
2.34.1

[PATCH v2 1/4] crypto: ccp: Use cc_platform_has to determine SEV presence

[Mario Limonciello]

The CPU feature flag shouldn't be used to determine if SEV is active,
that's what `cc_platform_has` is for.

Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
---
 drivers/crypto/ccp/sev-dev.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/crypto/ccp/sev-dev.c b/drivers/crypto/ccp/sev-dev.c
index 8fd774a10edc..e061eb6a039c 100644
--- a/drivers/crypto/ccp/sev-dev.c
+++ b/drivers/crypto/ccp/sev-dev.c
@@ -1148,7 +1148,7 @@ int sev_dev_init(struct psp_device *psp)
 	struct sev_device *sev;
 	int ret = -ENOMEM;
 
-	if (!boot_cpu_has(X86_FEATURE_SEV)) {
+	if (!cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) {
 		dev_info_once(dev, "SEV: memory encryption not enabled by BIOS\n");
 		return 0;
 	}
-- 
2.34.1

[PATCH v2 2/4] KVM: SVM: Use cc_platform_has instead of CPU feature flags

[Mario Limonciello]

The CPU feature flags may have been cleared earlier by the kernel
if the BIOS didn't activate features.  Instead use `cc_platform_has`
to determine that SEV was enabled.

Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
---
 arch/x86/kvm/svm/sev.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 17b53457d866..d3d1ec78d927 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -2105,7 +2105,7 @@ void __init sev_hardware_setup(void)
 	 * CPU supports decode assists, which is mandatory for SEV guests to
 	 * support instruction emulation.
 	 */
-	if (!boot_cpu_has(X86_FEATURE_SEV) ||
+	if (!cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT) ||
 	    WARN_ON_ONCE(!boot_cpu_has(X86_FEATURE_DECODEASSISTS)))
 		goto out;
 
@@ -2153,7 +2153,7 @@ void __init sev_hardware_setup(void)
 		goto out;
 
 	/* Does the CPU support SEV-ES? */
-	if (!boot_cpu_has(X86_FEATURE_SEV_ES))
+	if (!cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT))
 		goto out;
 
 	/* Has the system been allocated ASIDs for SEV-ES? */
-- 
2.34.1

[PATCH v2 3/4] x86/cpu: clear SME features when not in use

[Mario Limonciello]

Currently the SME CPU feature flag is reflective of whether the CPU
supports the features but not whether they have been activated by the
kernel.

Change this around to clear the features if the kernel is not using
them so userspace can determine if they are available and in use
from `/proc/cpuinfo`.

As the feature flag is now cleared on systems that SME isn't active
use CPUID 0x8000001f to confirm SME availability before calling
`native_wbinvd`.

Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
---
A concern was raised whether the assembly generated from stop_this_cpu
is still safe.  Here is the assembly from compiling with this patch:

00000000000011d0 <stop_this_cpu>:
    11d0:       e8 00 00 00 00          call   11d5 <stop_this_cpu+0x5>
    11d5:       55                      push   %rbp
    11d6:       48 89 e5                mov    %rsp,%rbp
    11d9:       41 54                   push   %r12
    11db:       53                      push   %rbx
    11dc:       48 83 ec 18             sub    $0x18,%rsp
    11e0:       65 48 8b 04 25 28 00    mov    %gs:0x28,%rax
    11e7:       00 00
    11e9:       48 89 45 e8             mov    %rax,-0x18(%rbp)
    11ed:       31 c0                   xor    %eax,%eax
    11ef:       ff 14 25 00 00 00 00    call   *0x0
    11f6:       e8 00 00 00 00          call   11fb <stop_this_cpu+0x2b>
    11fb:       31 f6                   xor    %esi,%esi
    11fd:       48 c7 c3 00 00 00 00    mov    $0x0,%rbx
    1204:       89 c7                   mov    %eax,%edi
    1206:       e8 00 00 00 00          call   120b <stop_this_cpu+0x3b>
    120b:       e8 00 00 00 00          call   1210 <stop_this_cpu+0x40>
    1210:       e8 00 00 00 00          call   1215 <stop_this_cpu+0x45>
    1215:       41 89 c4                mov    %eax,%r12d
    1218:       3d ff 1f 00 00          cmp    $0x1fff,%eax
    121d:       77 49                   ja     1268 <stop_this_cpu+0x98>
    121f:       4a 03 1c e5 00 00 00    add    0x0(,%r12,8),%rbx
    1226:       00
    1227:       48 89 df                mov    %rbx,%rdi
    122a:       e8 00 00 00 00          call   122f <stop_this_cpu+0x5f>
    122f:       c7 45 d8 1f 00 00 80    movl   $0x8000001f,-0x28(%rbp)
    1236:       48 8d 7d d8             lea    -0x28(%rbp),%rdi
    123a:       48 8d 75 dc             lea    -0x24(%rbp),%rsi
    123e:       c7 45 e0 00 00 00 00    movl   $0x0,-0x20(%rbp)
    1245:       48 8d 55 e0             lea    -0x20(%rbp),%rdx
    1249:       48 8d 4d e4             lea    -0x1c(%rbp),%rcx
    124d:       ff 14 25 00 00 00 00    call   *0x0
    1254:       f6 45 d8 01             testb  $0x1,-0x28(%rbp)
    1258:       74 02                   je     125c <stop_this_cpu+0x8c>
    125a:       0f 09                   wbinvd
    125c:       eb 07                   jmp    1265 <stop_this_cpu+0x95>
    125e:       0f 00 2d 00 00 00 00    verw   0x0(%rip)        # 1265 <stop_this_cpu+0x95>
    1265:       f4                      hlt
    1266:       eb f4                   jmp    125c <stop_this_cpu+0x8c>
    1268:       4c 89 e6                mov    %r12,%rsi
    126b:       48 c7 c7 00 00 00 00    mov    $0x0,%rdi
    1272:       e8 00 00 00 00          call   1277 <stop_this_cpu+0xa7>
    1277:       eb a6                   jmp    121f <stop_this_cpu+0x4f>
    1279:       0f 1f 80 00 00 00 00    nopl   0x0(%rax) 

 arch/x86/kernel/cpu/amd.c | 3 +++
 arch/x86/kernel/process.c | 5 ++++-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index 4edb6f0f628c..9cea774344cb 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -607,6 +607,9 @@ static void early_detect_mem_encrypt(struct cpuinfo_x86 *c)
 		if (IS_ENABLED(CONFIG_X86_32))
 			goto clear_all;
 
+		if (!sme_me_mask)
+			goto clear_all;
+
 		rdmsrl(MSR_K7_HWCR, msr);
 		if (!(msr & MSR_K7_HWCR_SMMLOCK))
 			goto clear_sev;
diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index 81d8ef036637..e131d71b3cae 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -765,8 +765,11 @@ void stop_this_cpu(void *dummy)
 	 * without the encryption bit, they don't race each other when flushed
 	 * and potentially end up with the wrong entry being committed to
 	 * memory.
+	 *
+	 * Test the CPUID bit directly because the machine might've cleared
+	 * X86_FEATURE_SME due to cmdline options.
 	 */
-	if (boot_cpu_has(X86_FEATURE_SME))
+	if (cpuid_eax(0x8000001f) & BIT(0))
 		native_wbinvd();
 	for (;;) {
 		/*
-- 
2.34.1

Re: [PATCH v2 3/4] x86/cpu: clear SME features when not in use

[Tom Lendacky]

On 2/11/22 15:02, Mario Limonciello wrote:
> Currently the SME CPU feature flag is reflective of whether the CPU
> supports the features but not whether they have been activated by the
> kernel.
> 
> Change this around to clear the features if the kernel is not using
> them so userspace can determine if they are available and in use
> from `/proc/cpuinfo`.
> 
> As the feature flag is now cleared on systems that SME isn't active
> use CPUID 0x8000001f to confirm SME availability before calling
> `native_wbinvd`.
> 
> Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
> ---

>   
> +		if (!sme_me_mask)
> +			goto clear_all;
> +

This is incorrect. Just because host memory encryption is not active 
doesn't mean that SEV guests can't be run. This should only clear the SME 
feature bit.

Thanks,
Tom

[PATCH v2 4/4] x86/cpu: clear SEV/SEV_ES features when not in use

[Mario Limonciello]

As there are no consumers for these feature flags in the kernel,
don't export these flags to userspace unless the feature has been
enabled by the guest.

Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
---
 arch/x86/kernel/cpu/amd.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index 9cea774344cb..6623a4995df1 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -614,12 +614,19 @@ static void early_detect_mem_encrypt(struct cpuinfo_x86 *c)
 		if (!(msr & MSR_K7_HWCR_SMMLOCK))
 			goto clear_sev;
 
+		if (!(sev_status & MSR_AMD64_SEV_ENABLED))
+			goto clear_sev;
+
+		if (!(sev_status & MSR_AMD64_SEV_ES_ENABLED))
+			goto clear_sev_es;
+
 		return;
 
 clear_all:
 		setup_clear_cpu_cap(X86_FEATURE_SME);
 clear_sev:
 		setup_clear_cpu_cap(X86_FEATURE_SEV);
+clear_sev_es:
 		setup_clear_cpu_cap(X86_FEATURE_SEV_ES);
 	}
 }
-- 
2.34.1

Re: [PATCH v2 0/4] Only export SME/SEV/SEV_ES features when activated

[Tom Lendacky]

On 2/11/22 15:02, Mario Limonciello wrote:
> Currently SME/SEV/SEV_ES feature flags will export if the CPU supports
> them but even if the kernel has not enabled the features for any reason.
> 
> To let userspace react to these feature flags, only set them when the
> kernel has activated them.
> 

You really need to wait for more feedback before issuing a new version so 
quickly. Especially since you asked a question about doing this just for 
SME to which I agreed.

Also, you've used cc_platform_has() incorrectly in patches 1 and 2, so SEV 
support is broke in the hypervisor with these changes.

Please just fix the SME concern that you have.

Thanks,
Tom

> changes from v1->v2:
>   * Squash old patches 1/2 together
>   * Add new patches to KVM/CCP code to use `cc_platform_has` instead
>   * Don't use `cc_platform_has` in the early code, just check `sme_me_mask`
>     and `sev_status` directly.
>   * Split out SME and SEV/SEV_ES clearing as separate patches in case
>     SEV/SEV_ES need other more involved work that what is in this series so
>     far.
> 
> Mario Limonciello (4):
>    crypto: ccp: Use cc_platform_has to determine SEV presence
>    KVM: SVM: Use cc_platform_has instead of CPU feature flags
>    x86/cpu: clear SME features when not in use
>    x86/cpu: clear SEV/SEV_ES features when not in use
> 
>   arch/x86/kernel/cpu/amd.c    | 10 ++++++++++
>   arch/x86/kernel/process.c    |  5 ++++-
>   arch/x86/kvm/svm/sev.c       |  4 ++--
>   drivers/crypto/ccp/sev-dev.c |  2 +-
>   4 files changed, 17 insertions(+), 4 deletions(-)
>