Kernel KVM virtualization development
 help / color / mirror / Atom feed
From: Manali Shukla <manali.shukla@amd.com>
To: Jim Mattson <jmattson@google.com>
Cc: <seanjc@google.com>, <pbonzini@redhat.com>, <mingo@redhat.com>,
	<bp@alien8.de>, <kvm@vger.kernel.org>, <x86@kernel.org>,
	<santosh.shukla@amd.com>, <nikunj.dadhania@amd.com>,
	<Naveen.Rao@amd.com>, <dapeng1.mi@linux.intel.com>,
	<ravi.bangoria@amd.com>, <peterz@infradead.org>,
	<Sandipan.Das@amd.com>, Yosry Ahmed <yosry@kernel.org>,
	<manali.shukla@amd.com>
Subject: Re: [PATCH v3 6/9] KVM: SVM: Add support for IBS Virtualization
Date: Tue, 10 Nov 2026 17:00:21 +0530	[thread overview]
Message-ID: <a156f924-a430-47ff-9fb9-22723acccdfc@amd.com> (raw)
In-Reply-To: <20261008191950.816113-1-jmattson@google.com>

Hi Jim,

Thank you for reviewing my patches.

On 10/9/2026 12:49 AM, Jim Mattson wrote:
> On Tue, Mar 10, 2026 at 06:00:18AM +0000, Manali Shukla wrote:
>> From: Santosh Shukla <santosh.shukla@amd.com>
>>
>> IBS virtualization (VIBS) allows a guest to collect Instruction-Based
>> Sampling (IBS) data using hardware-assisted virtualization. With VIBS
>> enabled, the hardware automatically saves and restores guest IBS state
>> during VM-Entry and VM-Exit via the VMCB State Save Area.
>>
>> IBS-generated interrupts are delivered directly to the guest without
>> causing a VMEXIT.
>>
>> VIBS depends on mediated PMU mode and requires either AVIC or NMI
>> virtualization for interrupt delivery. However, since AVIC can be
>> dynamically inhibited, VIBS requires VNMI to be enabled to ensure
>> reliable interrupt delivery. If AVIC is inhibited and VNMI is
>> disabled, the guest can encounter a VMEXIT_INVALID when IBS
>> virtualization is enabled for the guest.
> 
> APM vol. 2, section 15.38, disagrees. Without virtualized interrupt
> delivery, "an IBS interrupt occurring in the guest will not be delivered to
> either the guest or the hypervisor."  The only VMEXIT_INVALID that section
> 15.38 describes is for VMRUN of an SEV-ES or SEV-SNP guest with IbsFetchEn
> or IbsOpEn set.

This is what I have observed: when I tested IBS virtualization with
kvm_amd.avic=1 and kvm_amd.vnmi=0, VMRUN fails with VMEXIT_INVALID
whenever AVIC is inhibited. The APM might have missed mentioning it.

> 
>> Because IBS state is classified as swap type C, the hypervisor must
>> save its own IBS state before VMRUN and restore it after VMEXIT. It
>> must also disable IBS before VMRUN and re-enable it afterward. This
>> will be handled using mediated PMU support in subsequent patches by
>> enabling mediated PMU capability for IBS PMUs.
> 
> On hardware with IBS_CAPS_DIS, perf_ibs_stop() does not clear IbsFetchEn or
> IbsOpEn. perf_ibs_disable_event() only sets CTL2[Dis], and
> IBS_{FETCH|OP}_CTL[En] stays 1. Section 15.38 says that these bits must be
> 0 at VMRUN of an SEV-ES or SEV-SNP guest with IBS virtualization enabled
> (otherwise VMRUN fails with VMEXIT_INVALID), and that they should be 0 for
> other guests, "to prevent host IBS interrupts from leaking across a world
> switch."  Section 15.38 does not say that CTL2[Dis]=1 is sufficient. Is it?
> If not, the host must also clear En before VMRUN on hardware with
> IBS_CAPS_DIS.
> 

Yes, Hardware checks the effective IBS op and fetch enables in VMRUN to
a SEV-SE/SEV-SNP guest. And it is already updated in APM.

https://docs.amd.com/v/u/en-US/40332_4.11_APM_Vol1-5

>> More details about IBS virtualization can be found at [1].
>>
>> [1]: https://bugzilla.kernel.org/attachment.cgi?id=306250
>>      AMD64 Architecture Programmer’s Manual, Vol 2, Section 15.38
>>      Instruction-Based Sampling Virtualization.
>>
>> Signed-off-by: Santosh Shukla <santosh.shukla@amd.com>
>> Co-developed-by: Manali Shukla <manali.shukla@amd.com>
>> Signed-off-by: Manali Shukla <manali.shukla@amd.com>
>> ---
>>  arch/x86/include/asm/svm.h |  2 ++
>>  arch/x86/kvm/svm/svm.c     | 73 +++++++++++++++++++++++++++++++++++++-
>>  2 files changed, 74 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/x86/include/asm/svm.h b/arch/x86/include/asm/svm.h
>> index 4296efc1dafe..17aa6bf76bce 100644
>> --- a/arch/x86/include/asm/svm.h
>> +++ b/arch/x86/include/asm/svm.h
>> @@ -226,6 +226,8 @@ struct __attribute__ ((__packed__)) vmcb_control_area {
>>
>>  #define SVM_INT_VECTOR_MASK GENMASK(7, 0)
>>
>> +#define SVM_MISC_ENABLE_V_IBS BIT_ULL(2)
>> +
> 
> This bit is in misc_ctl2, not misc_ctl. Please name it SVM_MISC2_ENABLE_V_IBS
> and place it after SVM_MISC2_ENABLE_V_VMLOAD_VMSAVE.
> 

Will update it in V4.

> [...]
> 
>> @@ -779,6 +782,26 @@ static void svm_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu)
>>  				  MSR_TYPE_RW, intercept);
>>  }
>>
>> +static void svm_recalc_ibs_msr_intercepts(struct kvm_vcpu *vcpu)
>> +{
>> +	bool intercept = !(guest_cpu_cap_has(vcpu, X86_FEATURE_IBS) &&
>> +			   kvm_vcpu_has_mediated_pmu(vcpu));
>> +
>> +	if (!enable_mediated_pmu || !vibs)
>> +		return;
>> +
>> +	svm_set_intercept_for_msr(vcpu, MSR_AMD64_IBSFETCHCTL, MSR_TYPE_RW, intercept);
> 
> For SEV-ES and SEV-SNP guests, section 15.38 says that IBS virtualization
> is enabled by bit 12 of SEV_FEATURES in the VMSA, not by bit 2 at offset
> B8h in the VMCB. This patch sets only the VMCB bit, but it disables these
> intercepts for every vCPU that has X86_FEATURE_IBS and a mediated PMU,
> *including* SEV-ES guests.  IBS virtualization is not enabled for SEV-ES
> guests, so they will be able to read and write the host's IBS MSRs!
> 

You are right. I will fix it in V4. IBS MSRs for SEV-ES/SEV-SNP guests
will be passed through only when bit 12 of SEV_FEATURES in VMSA is enabled.

> Since nested_vmcb02_prepare_control() does not set SVM_MISC_ENABLE_V_IBS in
> vmcb02->control.misc_ctl2, nested SVM has a similar problem. If L1 does not
> set INTERCEPT_MSR_PROT in vmcb12, vmcb02 uses vmcb01's msrpm, which
> disables these MSR intercepts. Because VIBS is disabled in vmcb02, L2 can
> read and write the host's physical IBS MSRs.
> 
> OTOH, if L1 sets INTERCEPT_MSR_PROT in vmcb12 and clears the IBS MSR
> intercept bits in msrpm12, nested_svm_init_msrpm_merge_offsets() does not
> include the IBS MSRs in merge_msrs[], so msrpm02 still intercepts them, and
> L0 injects #GP into L2 on accesses to IBS_FETCH_CTL, IBS_OP_CTL, etc.
> 

You are right. Both the scenarios are not handled currently. I will fix
them in V4.

> Furthermore, the guest's IBS state is in the VMCB save area, but userspace
> has no way to save or restore it. The IBS MSRs are not in msrs_to_save[],
> and svm_get_msr() and svm_set_msr() reject them, so KVM_{GET,SET}_MSRS will
> fail. Hence, the IBS state is lost on suspend/resume.
> 

Right. I will add the IBS MSRs to msrs_to_save[] and handle them in
svm_get_msr() and svm_set_msr() in V4.

> [...]
> 
>> @@ -2880,6 +2904,27 @@ static int svm_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
>>  	case MSR_AMD64_DE_CFG:
>>  		msr_info->data = svm->msr_decfg;
>>  		break;
>> +
>> +	case MSR_AMD64_IBSCTL:
>> +		if (guest_cpu_cap_has(vcpu, X86_FEATURE_IBS))
>> +			msr_info->data = IBSCTL_LVT_OFFSET_VALID;
>> +		else
>> +			msr_info->data = 0;
>> +		break;
> 
> Before this patch, KVM synthesized a #GP when a guest without
> X86_FEATURE_IBS read MSR_AMD64_IBSCTL. Now it returns 0.
> 
> The condition should also test for kvm_vcpu_has_mediated_pmu(vcpu).

Agreed. I will take care of this in V4.

> 
>> +
>> +
> 
> Nit: there is an extra blank line above.
> 

Ack.

>> +	/*
>> +	 * When IBS virtualization is enabled, guest reads from
>> +	 * MSR_AMD64_IBSFETCHPHYSAD and MSR_AMD64_IBSDCPHYSAD must return 0.
>> +	 * This is done for security reasons, as guests should not be allowed to
>> +	 * access or infer any information about the system's physical
>> +	 * addresses.
>> +	 */
>> +	case MSR_AMD64_IBSDCPHYSAD:
>> +	case MSR_AMD64_IBSFETCHPHYSAD:
>> +		msr_info->data = 0;
>> +		break;
>> +
> 
> These new cases should be conditional. Before this patch, KVM synthesized
> a #GP for a guest without X86_FEATURE_IBS. Now it returns 0.
> 
> But, is this even necessary? APM vol. 2, section 15.38, says that when IBS
> virtualization is enabled in the VMCB, the processor returns 0 on guest
> reads from IbsFetchPhysAd and IbsDcPhysAd and ignores guest writes to
> them. Can't we just disable interception of these MSRs when VIBS is enabled
> for the vCPU, and let the hardware handle them?
> 

I agree. This is an oversight from my side. I missed to disable the
interception of these MSRs when VIBS is enabled. I will disable the
interception in V4.

>>  	default:
>>  		return kvm_get_msr_common(vcpu, msr_info);
>>  	}
>> @@ -3171,6 +3216,16 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)
>>  		svm->msr_decfg = data;
>>  		break;
>>  	}
>> +	/*
>> +	 * When IBS virtualization is enabled, guest writes to
>> +	 * MSR_AMD64_IBSFETCHPHYSAD and MSR_AMD64_IBSDCPHYSAD must be ignored.
>> +	 * This is done for security reasons, as guests should not be allowed to
>> +	 * access or infer any information about the system's physical
>> +	 * addresses.
>> +	 */
>> +	case MSR_AMD64_IBSDCPHYSAD:
>> +	case MSR_AMD64_IBSFETCHPHYSAD:
>> +		return 1;
> 
> Returning 1 does not "ignore" the write; it induces a #GP. But, see the
> point I made above.

Ack.

> 
> [...]
> 
>> @@ -4678,6 +4733,11 @@ static void svm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
>>  	if (guest_cpuid_is_intel_compatible(vcpu))
>>  		guest_cpu_cap_clear(vcpu, X86_FEATURE_V_VMSAVE_VMLOAD);
>>
>> +	if (guest_cpu_cap_has(vcpu, X86_FEATURE_IBS))
>> +		svm->vmcb->control.misc_ctl2 |= SVM_MISC_ENABLE_V_IBS;
>> +	else
>> +		svm->vmcb->control.misc_ctl2 &= ~SVM_MISC_ENABLE_V_IBS;
>> +
> 
> The condition should also check kvm_vcpu_has_mediated_pmu(vcpu), as in
> svm_recalc_ibs_msr_intercepts().

Ack. I will add it in V4.


-Manali


  parent reply	other threads:[~2026-11-10 11:30 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-03-10  6:00 [PATCH v3 0/9] Implement support for IBS virtualization Manali Shukla
2026-03-10  6:00 ` [PATCH v3 1/9] perf/amd/ibs: Fix race condition in IBS Manali Shukla
2026-10-08 17:19   ` Jim Mattson
2026-03-10  6:00 ` [PATCH v3 2/9] x86/cpufeatures: Add CPUID feature bit for VIBS in SVM/SEV guests Manali Shukla
2026-10-08 17:28   ` Jim Mattson
2026-03-10  6:00 ` [PATCH v3 3/9] KVM: x86/cpuid: Add a KVM-only leaf for IBS capabilities Manali Shukla
2026-10-08 17:44   ` Jim Mattson
2026-10-22 10:16     ` Manali Shukla
2026-03-10  6:00 ` [PATCH v3 4/9] KVM: x86: Extend CPUID range to include new leaf Manali Shukla
2026-10-08 17:59   ` Jim Mattson
2026-10-22 12:00     ` Manali Shukla
2026-10-22 17:15       ` Jim Mattson
2026-03-10  6:00 ` [PATCH v3 5/9] KVM: SVM: Extend VMCB area for virtualized IBS registers Manali Shukla
2026-10-08 18:01   ` Jim Mattson
2026-03-10  6:00 ` [PATCH v3 6/9] KVM: SVM: Add support for IBS Virtualization Manali Shukla
2026-10-08 19:19   ` Jim Mattson
2026-10-08 21:28     ` Jim Mattson
2026-10-16 16:01       ` Sean Christopherson
2026-11-10 12:03         ` Manali Shukla
2026-11-10 11:30     ` Manali Shukla [this message]
2026-03-10  6:00 ` [PATCH v3 7/9] perf/x86/amd: Enable VPMU passthrough capability for IBS PMU Manali Shukla
2026-10-08 19:32   ` Jim Mattson
2026-11-18 13:39     ` Manali Shukla
2026-03-10  6:00 ` [PATCH v3 8/9] perf/x86/amd: Remove exclude_guest check from perf_ibs_init() Manali Shukla
2026-10-08 19:38   ` Jim Mattson
2026-11-18 13:38     ` Manali Shukla
2026-03-10  6:00 ` [PATCH v3 9/9] KVM: SVM: Add newly added IBS capabilities and MSRs Manali Shukla
2026-10-08 21:03   ` Jim Mattson
2026-11-11 12:11     ` Manali Shukla

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=a156f924-a430-47ff-9fb9-22723acccdfc@amd.com \
    --to=manali.shukla@amd.com \
    --cc=Naveen.Rao@amd.com \
    --cc=Sandipan.Das@amd.com \
    --cc=bp@alien8.de \
    --cc=dapeng1.mi@linux.intel.com \
    --cc=jmattson@google.com \
    --cc=kvm@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=nikunj.dadhania@amd.com \
    --cc=pbonzini@redhat.com \
    --cc=peterz@infradead.org \
    --cc=ravi.bangoria@amd.com \
    --cc=santosh.shukla@amd.com \
    --cc=seanjc@google.com \
    --cc=x86@kernel.org \
    --cc=yosry@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox