[PATCH] KVM: x86: AMD's IBPB is not equivalent to Intel's IBPB

[PATCH] KVM: x86: AMD's IBPB is not equivalent to Intel's IBPB

[Jim Mattson]

From Intel's documention [1], "CPUID.(EAX=07H,ECX=0):EDX[26]
enumerates support for indirect branch restricted speculation (IBRS)
and the indirect branch predictor barrier (IBPB)." Further, from [2],
"Software that executed before the IBPB command cannot control the
predicted targets of indirect branches (4) executed after the command
on the same logical processor," where footnote 4 reads, "Note that
indirect branches include near call indirect, near jump indirect and
near return instructions. Because it includes near returns, it follows
that **RSB entries created before an IBPB command cannot control the
predicted targets of returns executed after the command on the same
logical processor.**" [emphasis mine]

On the other hand, AMD's "IBPB may not prevent return branch
predictions from being specified by pre-IBPB branch targets" [3].

Since Linux sets the synthetic feature bit, X86_FEATURE_IBPB, on AMD
CPUs that implement the weaker version of IBPB, it is incorrect to
infer from this and X86_FEATURE_IBRS that the CPU supports the
stronger version of IBPB indicated by CPUID.(EAX=07H,ECX=0):EDX[26].

Stop making this inference.

[1] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/cpuid-enumeration-and-architectural-msrs.html
[2] https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/speculative-execution-side-channel-mitigations.html#Footnotes
[3] https://www.amd.com/en/resources/product-security/bulletin/amd-sb-1040.html

Fixes: 0c54914d0c52 ("KVM: x86: use Intel speculation bugs and features as derived in generic x86 code")
Signed-off-by: Jim Mattson <jmattson@google.com>
---
 arch/x86/kvm/cpuid.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index bfc0bfcb2bc6..66f2761b2836 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -677,8 +677,6 @@ void kvm_set_cpu_caps(void)
 	kvm_cpu_cap_set(X86_FEATURE_TSC_ADJUST);
 	kvm_cpu_cap_set(X86_FEATURE_ARCH_CAPABILITIES);
 
-	if (boot_cpu_has(X86_FEATURE_IBPB) && boot_cpu_has(X86_FEATURE_IBRS))
-		kvm_cpu_cap_set(X86_FEATURE_SPEC_CTRL);
 	if (boot_cpu_has(X86_FEATURE_STIBP))
 		kvm_cpu_cap_set(X86_FEATURE_INTEL_STIBP);
 	if (boot_cpu_has(X86_FEATURE_AMD_SSBD))
-- 
2.44.0.683.g7961c838ac-goog

Re: [PATCH] KVM: x86: AMD's IBPB is not equivalent to Intel's IBPB

[Venkatesh Srinivas]

On Thu, Apr 11, 2024 at 1:59 PM Jim Mattson <jmattson@google.com> wrote:
>
> From Intel's documention [1], "CPUID.(EAX=07H,ECX=0):EDX[26]
> enumerates support for indirect branch restricted speculation (IBRS)
> and the indirect branch predictor barrier (IBPB)." Further, from [2],
> "Software that executed before the IBPB command cannot control the
> predicted targets of indirect branches (4) executed after the command
> on the same logical processor," where footnote 4 reads, "Note that
> indirect branches include near call indirect, near jump indirect and
> near return instructions. Because it includes near returns, it follows
> that **RSB entries created before an IBPB command cannot control the
> predicted targets of returns executed after the command on the same
> logical processor.**" [emphasis mine]
>
> On the other hand, AMD's "IBPB may not prevent return branch
> predictions from being specified by pre-IBPB branch targets" [3].
>
> Since Linux sets the synthetic feature bit, X86_FEATURE_IBPB, on AMD
> CPUs that implement the weaker version of IBPB, it is incorrect to
> infer from this and X86_FEATURE_IBRS that the CPU supports the
> stronger version of IBPB indicated by CPUID.(EAX=07H,ECX=0):EDX[26].

AMD's IBPB does apply to RET predictions if Fn8000_0008_EBX[IBPB_RET] = 1.
Spot checking, Zen4 sets that bit; and the bulletin doesn't apply there.

(Also checking - IA32_SPEC_CTRL and IA32_PRED_CMD are both still
available; is there anything in KVM that keys off just X86_FEATURE_SPEC_CTRL?
I'm not seeing it...)

-- vs;

Re: [PATCH] KVM: x86: AMD's IBPB is not equivalent to Intel's IBPB

[Jim Mattson]

On Thu, Apr 11, 2024 at 6:32 PM Venkatesh Srinivas
<venkateshs@chromium.org> wrote:
>
> On Thu, Apr 11, 2024 at 1:59 PM Jim Mattson <jmattson@google.com> wrote:
> >
> > From Intel's documention [1], "CPUID.(EAX=07H,ECX=0):EDX[26]
> > enumerates support for indirect branch restricted speculation (IBRS)
> > and the indirect branch predictor barrier (IBPB)." Further, from [2],
> > "Software that executed before the IBPB command cannot control the
> > predicted targets of indirect branches (4) executed after the command
> > on the same logical processor," where footnote 4 reads, "Note that
> > indirect branches include near call indirect, near jump indirect and
> > near return instructions. Because it includes near returns, it follows
> > that **RSB entries created before an IBPB command cannot control the
> > predicted targets of returns executed after the command on the same
> > logical processor.**" [emphasis mine]
> >
> > On the other hand, AMD's "IBPB may not prevent return branch
> > predictions from being specified by pre-IBPB branch targets" [3].
> >
> > Since Linux sets the synthetic feature bit, X86_FEATURE_IBPB, on AMD
> > CPUs that implement the weaker version of IBPB, it is incorrect to
> > infer from this and X86_FEATURE_IBRS that the CPU supports the
> > stronger version of IBPB indicated by CPUID.(EAX=07H,ECX=0):EDX[26].
>
> AMD's IBPB does apply to RET predictions if Fn8000_0008_EBX[IBPB_RET] = 1.
> Spot checking, Zen4 sets that bit; and the bulletin doesn't apply there.

So, with a definition of X86_FEATURE_AMD_IBPB_RET, this could be:

       if (boot_cpu_has(X86_FEATURE_AMD_IBPB_RET) &&
boot_cpu_has(X86_FEATURE_IBRS))
               kvm_cpu_cap_set(X86_FEATURE_SPEC_CTRL);

And, in the other direction,

    if (boot_cpu_has(X86_FEATURE_SPEC_CTRL))
        kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB_RET);

But, perhaps all of this cross-vendor equivalence logic belongs in user space.

> (Also checking - IA32_SPEC_CTRL and IA32_PRED_CMD are both still
> available; is there anything in KVM that keys off just X86_FEATURE_SPEC_CTRL?
> I'm not seeing it...)

I hope not. It looks like all of the guest_cpuid checks for SPEC_CTRL
also check for the AMD bits (e.g. guest_has_spec_ctrl_msr()).

Re: [PATCH] KVM: x86: AMD's IBPB is not equivalent to Intel's IBPB

[Jim Mattson]

On Thu, Apr 11, 2024 at 7:57 PM Jim Mattson <jmattson@google.com> wrote:
>
> On Thu, Apr 11, 2024 at 6:32 PM Venkatesh Srinivas
> <venkateshs@chromium.org> wrote:
> >
> > On Thu, Apr 11, 2024 at 1:59 PM Jim Mattson <jmattson@google.com> wrote:
> > >
> > > From Intel's documention [1], "CPUID.(EAX=07H,ECX=0):EDX[26]
> > > enumerates support for indirect branch restricted speculation (IBRS)
> > > and the indirect branch predictor barrier (IBPB)." Further, from [2],
> > > "Software that executed before the IBPB command cannot control the
> > > predicted targets of indirect branches (4) executed after the command
> > > on the same logical processor," where footnote 4 reads, "Note that
> > > indirect branches include near call indirect, near jump indirect and
> > > near return instructions. Because it includes near returns, it follows
> > > that **RSB entries created before an IBPB command cannot control the
> > > predicted targets of returns executed after the command on the same
> > > logical processor.**" [emphasis mine]
> > >
> > > On the other hand, AMD's "IBPB may not prevent return branch
> > > predictions from being specified by pre-IBPB branch targets" [3].
> > >
> > > Since Linux sets the synthetic feature bit, X86_FEATURE_IBPB, on AMD
> > > CPUs that implement the weaker version of IBPB, it is incorrect to
> > > infer from this and X86_FEATURE_IBRS that the CPU supports the
> > > stronger version of IBPB indicated by CPUID.(EAX=07H,ECX=0):EDX[26].
> >
> > AMD's IBPB does apply to RET predictions if Fn8000_0008_EBX[IBPB_RET] = 1.
> > Spot checking, Zen4 sets that bit; and the bulletin doesn't apply there.
>
> So, with a definition of X86_FEATURE_AMD_IBPB_RET, this could be:
>
>        if (boot_cpu_has(X86_FEATURE_AMD_IBPB_RET) &&
> boot_cpu_has(X86_FEATURE_IBRS))
>                kvm_cpu_cap_set(X86_FEATURE_SPEC_CTRL);
>
> And, in the other direction,
>
>     if (boot_cpu_has(X86_FEATURE_SPEC_CTRL))
>         kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB_RET);
>
> But, perhaps all of this cross-vendor equivalence logic belongs in user space.

In case it wasn't clear, I contend that any cross-vendor equivalence
logic *does* belong in userspace.

Thoughts?

> > (Also checking - IA32_SPEC_CTRL and IA32_PRED_CMD are both still
> > available; is there anything in KVM that keys off just X86_FEATURE_SPEC_CTRL?
> > I'm not seeing it...)
>
> I hope not. It looks like all of the guest_cpuid checks for SPEC_CTRL
> also check for the AMD bits (e.g. guest_has_spec_ctrl_msr()).

Re: [PATCH] KVM: x86: AMD's IBPB is not equivalent to Intel's IBPB

[Sean Christopherson]

On Tue, May 07, 2024, Jim Mattson wrote:
> On Thu, Apr 11, 2024 at 7:57 PM Jim Mattson <jmattson@google.com> wrote:
> >
> > On Thu, Apr 11, 2024 at 6:32 PM Venkatesh Srinivas
> > <venkateshs@chromium.org> wrote:
> > >
> > > On Thu, Apr 11, 2024 at 1:59 PM Jim Mattson <jmattson@google.com> wrote:
> > > >
> > > > From Intel's documention [1], "CPUID.(EAX=07H,ECX=0):EDX[26]
> > > > enumerates support for indirect branch restricted speculation (IBRS)
> > > > and the indirect branch predictor barrier (IBPB)." Further, from [2],
> > > > "Software that executed before the IBPB command cannot control the
> > > > predicted targets of indirect branches (4) executed after the command
> > > > on the same logical processor," where footnote 4 reads, "Note that
> > > > indirect branches include near call indirect, near jump indirect and
> > > > near return instructions. Because it includes near returns, it follows
> > > > that **RSB entries created before an IBPB command cannot control the
> > > > predicted targets of returns executed after the command on the same
> > > > logical processor.**" [emphasis mine]
> > > >
> > > > On the other hand, AMD's "IBPB may not prevent return branch
> > > > predictions from being specified by pre-IBPB branch targets" [3].
> > > >
> > > > Since Linux sets the synthetic feature bit, X86_FEATURE_IBPB, on AMD
> > > > CPUs that implement the weaker version of IBPB, it is incorrect to
> > > > infer from this and X86_FEATURE_IBRS that the CPU supports the
> > > > stronger version of IBPB indicated by CPUID.(EAX=07H,ECX=0):EDX[26].
> > >
> > > AMD's IBPB does apply to RET predictions if Fn8000_0008_EBX[IBPB_RET] = 1.
> > > Spot checking, Zen4 sets that bit; and the bulletin doesn't apply there.
> >
> > So, with a definition of X86_FEATURE_AMD_IBPB_RET, this could be:
> >
> >        if (boot_cpu_has(X86_FEATURE_AMD_IBPB_RET) &&
> > boot_cpu_has(X86_FEATURE_IBRS))
> >                kvm_cpu_cap_set(X86_FEATURE_SPEC_CTRL);
> >
> > And, in the other direction,
> >
> >     if (boot_cpu_has(X86_FEATURE_SPEC_CTRL))
> >         kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB_RET);
> >
> > But, perhaps all of this cross-vendor equivalence logic belongs in user space.
> 
> In case it wasn't clear, I contend that any cross-vendor equivalence
> logic *does* belong in userspace.
> 
> Thoughts?

Maybe?  I generally like punting these sorts of things to userspace, but as
evidenced by this patch, all of these mitigation "features" are such a godawful
mess that I don't have a problem with KVM doing the heavy lifting.

E.g. I suspect that having KVM enumerate both vendor's bits makes it much easier
for QEMU to support pre-defined uarch models while still retaining sanity checks
that the features being enumerated to the guest are indeed supported by the host.

Re: [PATCH] KVM: x86: AMD's IBPB is not equivalent to Intel's IBPB

[Sean Christopherson]

On Thu, Apr 11, 2024, Jim Mattson wrote:
> On Thu, Apr 11, 2024 at 6:32 PM Venkatesh Srinivas
> <venkateshs@chromium.org> wrote:
> >
> > On Thu, Apr 11, 2024 at 1:59 PM Jim Mattson <jmattson@google.com> wrote:
> > >
> > > From Intel's documention [1], "CPUID.(EAX=07H,ECX=0):EDX[26]
> > > enumerates support for indirect branch restricted speculation (IBRS)
> > > and the indirect branch predictor barrier (IBPB)." Further, from [2],
> > > "Software that executed before the IBPB command cannot control the
> > > predicted targets of indirect branches (4) executed after the command
> > > on the same logical processor," where footnote 4 reads, "Note that
> > > indirect branches include near call indirect, near jump indirect and
> > > near return instructions. Because it includes near returns, it follows
> > > that **RSB entries created before an IBPB command cannot control the
> > > predicted targets of returns executed after the command on the same
> > > logical processor.**" [emphasis mine]
> > >
> > > On the other hand, AMD's "IBPB may not prevent return branch
> > > predictions from being specified by pre-IBPB branch targets" [3].
> > >
> > > Since Linux sets the synthetic feature bit, X86_FEATURE_IBPB, on AMD
> > > CPUs that implement the weaker version of IBPB, it is incorrect to
> > > infer from this and X86_FEATURE_IBRS that the CPU supports the
> > > stronger version of IBPB indicated by CPUID.(EAX=07H,ECX=0):EDX[26].
> >
> > AMD's IBPB does apply to RET predictions if Fn8000_0008_EBX[IBPB_RET] = 1.
> > Spot checking, Zen4 sets that bit; and the bulletin doesn't apply there.
> 
> So, with a definition of X86_FEATURE_AMD_IBPB_RET, this could be:
> 
>        if (boot_cpu_has(X86_FEATURE_AMD_IBPB_RET) &&
> boot_cpu_has(X86_FEATURE_IBRS))
>                kvm_cpu_cap_set(X86_FEATURE_SPEC_CTRL);
> 
> And, in the other direction,
> 
>     if (boot_cpu_has(X86_FEATURE_SPEC_CTRL))
>         kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB_RET);

Jim, are you planning on sending a v2 with Venkatesh's suggested solution?

Re: [PATCH] KVM: x86: AMD's IBPB is not equivalent to Intel's IBPB

[Jim Mattson]

On Fri, Aug 16, 2024 at 7:25 AM Sean Christopherson <seanjc@google.com> wrote:
>
> On Thu, Apr 11, 2024, Jim Mattson wrote:
> > On Thu, Apr 11, 2024 at 6:32 PM Venkatesh Srinivas
> > <venkateshs@chromium.org> wrote:
> > >
> > > On Thu, Apr 11, 2024 at 1:59 PM Jim Mattson <jmattson@google.com> wrote:
> > > >
> > > > From Intel's documention [1], "CPUID.(EAX=07H,ECX=0):EDX[26]
> > > > enumerates support for indirect branch restricted speculation (IBRS)
> > > > and the indirect branch predictor barrier (IBPB)." Further, from [2],
> > > > "Software that executed before the IBPB command cannot control the
> > > > predicted targets of indirect branches (4) executed after the command
> > > > on the same logical processor," where footnote 4 reads, "Note that
> > > > indirect branches include near call indirect, near jump indirect and
> > > > near return instructions. Because it includes near returns, it follows
> > > > that **RSB entries created before an IBPB command cannot control the
> > > > predicted targets of returns executed after the command on the same
> > > > logical processor.**" [emphasis mine]
> > > >
> > > > On the other hand, AMD's "IBPB may not prevent return branch
> > > > predictions from being specified by pre-IBPB branch targets" [3].
> > > >
> > > > Since Linux sets the synthetic feature bit, X86_FEATURE_IBPB, on AMD
> > > > CPUs that implement the weaker version of IBPB, it is incorrect to
> > > > infer from this and X86_FEATURE_IBRS that the CPU supports the
> > > > stronger version of IBPB indicated by CPUID.(EAX=07H,ECX=0):EDX[26].
> > >
> > > AMD's IBPB does apply to RET predictions if Fn8000_0008_EBX[IBPB_RET] = 1.
> > > Spot checking, Zen4 sets that bit; and the bulletin doesn't apply there.
> >
> > So, with a definition of X86_FEATURE_AMD_IBPB_RET, this could be:
> >
> >        if (boot_cpu_has(X86_FEATURE_AMD_IBPB_RET) &&
> > boot_cpu_has(X86_FEATURE_IBRS))
> >                kvm_cpu_cap_set(X86_FEATURE_SPEC_CTRL);
> >
> > And, in the other direction,
> >
> >     if (boot_cpu_has(X86_FEATURE_SPEC_CTRL))
> >         kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB_RET);
>
> Jim, are you planning on sending a v2 with Venkatesh's suggested solution?

I really like the idea of moving all of the cross-vendor capability
settings into userspace, but if we want to keep this in the kernel,
then I'll send V2.

Re: [PATCH] KVM: x86: AMD's IBPB is not equivalent to Intel's IBPB

[Sean Christopherson]

On Fri, Aug 16, 2024, Jim Mattson wrote:
> On Fri, Aug 16, 2024 at 7:25 AM Sean Christopherson <seanjc@google.com> wrote:
> >
> > On Thu, Apr 11, 2024, Jim Mattson wrote:
> > > On Thu, Apr 11, 2024 at 6:32 PM Venkatesh Srinivas
> > > <venkateshs@chromium.org> wrote:
> > > >
> > > > On Thu, Apr 11, 2024 at 1:59 PM Jim Mattson <jmattson@google.com> wrote:
> > > > >
> > > > > From Intel's documention [1], "CPUID.(EAX=07H,ECX=0):EDX[26]
> > > > > enumerates support for indirect branch restricted speculation (IBRS)
> > > > > and the indirect branch predictor barrier (IBPB)." Further, from [2],
> > > > > "Software that executed before the IBPB command cannot control the
> > > > > predicted targets of indirect branches (4) executed after the command
> > > > > on the same logical processor," where footnote 4 reads, "Note that
> > > > > indirect branches include near call indirect, near jump indirect and
> > > > > near return instructions. Because it includes near returns, it follows
> > > > > that **RSB entries created before an IBPB command cannot control the
> > > > > predicted targets of returns executed after the command on the same
> > > > > logical processor.**" [emphasis mine]
> > > > >
> > > > > On the other hand, AMD's "IBPB may not prevent return branch
> > > > > predictions from being specified by pre-IBPB branch targets" [3].
> > > > >
> > > > > Since Linux sets the synthetic feature bit, X86_FEATURE_IBPB, on AMD
> > > > > CPUs that implement the weaker version of IBPB, it is incorrect to
> > > > > infer from this and X86_FEATURE_IBRS that the CPU supports the
> > > > > stronger version of IBPB indicated by CPUID.(EAX=07H,ECX=0):EDX[26].
> > > >
> > > > AMD's IBPB does apply to RET predictions if Fn8000_0008_EBX[IBPB_RET] = 1.
> > > > Spot checking, Zen4 sets that bit; and the bulletin doesn't apply there.
> > >
> > > So, with a definition of X86_FEATURE_AMD_IBPB_RET, this could be:
> > >
> > >        if (boot_cpu_has(X86_FEATURE_AMD_IBPB_RET) &&
> > > boot_cpu_has(X86_FEATURE_IBRS))
> > >                kvm_cpu_cap_set(X86_FEATURE_SPEC_CTRL);
> > >
> > > And, in the other direction,
> > >
> > >     if (boot_cpu_has(X86_FEATURE_SPEC_CTRL))
> > >         kvm_cpu_cap_set(X86_FEATURE_AMD_IBPB_RET);
> >
> > Jim, are you planning on sending a v2 with Venkatesh's suggested solution?
> 
> I really like the idea of moving all of the cross-vendor capability
> settings into userspace, but if we want to keep this in the kernel,
> then I'll send V2.

Unfortunately, that ship has likely sailed for the existing code, because I
suspect userspace has come to rely on KVM's behavior.  I do think it makes sense
to punt all future cross-vendor stuff to userspace, with exceptions for cases like
this where we're fixing existing KVM support.