* [PATCH v2 0/2] KVM: Only use posted interrupts for Fixed/LowPrio MSIs @ 2019-09-04 13:35 Alexander Graf 2019-09-04 13:35 ` [PATCH v2 1/2] KVM: VMX: Disable posted interrupts for odd IRQs Alexander Graf 2019-09-04 13:35 ` [PATCH v2 2/2] KVM: SVM: " Alexander Graf 0 siblings, 2 replies; 8+ messages in thread From: Alexander Graf @ 2019-09-04 13:35 UTC (permalink / raw) To: kvm Cc: linux-kernel, x86, H. Peter Anvin, Borislav Petkov, Ingo Molnar, Thomas Gleixner, Joerg Roedel, Jim Mattson, Wanpeng Li, Vitaly Kuznetsov, Sean Christopherson, Radim Krčmář, Paolo Bonzini, Liran Alon The MSI-X descriptor has a "delivery mode" field which can be set to various different targets, such as "Fixed" (default), SMI, NMI or INIT. Usually when we pass devices into guests, we only ever see this MSI-X descriptor configured as Fixed, so nobody realized that the other modes were broken when using posted interrupts. With posted interrupts, we end up configuring these special modes just the same as a Fixed interrupt. That means instead of generating an SMI, we inject a normal GSI into the guest. Of course, that if completely broken. These two patches attempt to fix the situation for x86 systems. If anyone has a great idea how to generalize the filtering though, I'm all ears. Alex --- v1 -> v2: - Make error message more unique - Update commit message to point to __apic_accept_irq() Alexander Graf (2): KVM: VMX: Disable posted interrupts for odd IRQs KVM: SVM: Disable posted interrupts for odd IRQs arch/x86/kvm/svm.c | 16 ++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 22 ++++++++++++++++++++++ 2 files changed, 38 insertions(+) -- 2.17.1 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 1/2] KVM: VMX: Disable posted interrupts for odd IRQs 2019-09-04 13:35 [PATCH v2 0/2] KVM: Only use posted interrupts for Fixed/LowPrio MSIs Alexander Graf @ 2019-09-04 13:35 ` Alexander Graf 2019-09-04 14:40 ` Sean Christopherson 2019-09-04 13:35 ` [PATCH v2 2/2] KVM: SVM: " Alexander Graf 1 sibling, 1 reply; 8+ messages in thread From: Alexander Graf @ 2019-09-04 13:35 UTC (permalink / raw) To: kvm Cc: linux-kernel, x86, H. Peter Anvin, Borislav Petkov, Ingo Molnar, Thomas Gleixner, Joerg Roedel, Jim Mattson, Wanpeng Li, Vitaly Kuznetsov, Sean Christopherson, Radim Krčmář, Paolo Bonzini, Liran Alon We can easily route hardware interrupts directly into VM context when they target the "Fixed" or "LowPriority" delivery modes. However, on modes such as "SMI" or "Init", we need to go via KVM code to actually put the vCPU into a different mode of operation, so we can not post the interrupt Add code in the VMX PI logic to explicitly refuse to establish posted mappings for advanced IRQ deliver modes. This reflects the logic in __apic_accept_irq() which also only ever passes Fixed and LowPriority interrupts as posted interrupts into the guest. This fixes a bug I have with code which configures real hardware to inject virtual SMIs into my guest. Signed-off-by: Alexander Graf <graf@amazon.com> Reviewed-by: Liran Alon <liran.alon@oracle.com> --- v1 -> v2: - Make error message more unique - Update commit message to point to __apic_accept_irq() --- arch/x86/kvm/vmx/vmx.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 570a233e272b..8029fe658c30 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7401,6 +7401,28 @@ static int vmx_update_pi_irte(struct kvm *kvm, unsigned int host_irq, continue; } + switch (irq.delivery_mode) { + case dest_Fixed: + case dest_LowestPrio: + break; + default: + /* + * For non-trivial interrupt events, we need to go + * through the full KVM IRQ code, so refuse to take + * any direct PI assignments here. + */ + + ret = irq_set_vcpu_affinity(host_irq, NULL); + if (ret < 0) { + printk(KERN_INFO + "non-std IRQ failed to recover, irq: %u\n", + host_irq); + goto out; + } + + continue; + } + vcpu_info.pi_desc_addr = __pa(vcpu_to_pi_desc(vcpu)); vcpu_info.vector = irq.vector; -- 2.17.1 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH v2 1/2] KVM: VMX: Disable posted interrupts for odd IRQs 2019-09-04 13:35 ` [PATCH v2 1/2] KVM: VMX: Disable posted interrupts for odd IRQs Alexander Graf @ 2019-09-04 14:40 ` Sean Christopherson 2019-09-04 15:36 ` Alexander Graf 0 siblings, 1 reply; 8+ messages in thread From: Sean Christopherson @ 2019-09-04 14:40 UTC (permalink / raw) To: Alexander Graf Cc: kvm, linux-kernel, x86, H. Peter Anvin, Borislav Petkov, Ingo Molnar, Thomas Gleixner, Joerg Roedel, Jim Mattson, Wanpeng Li, Vitaly Kuznetsov, Radim Krčmář, Paolo Bonzini, Liran Alon On Wed, Sep 04, 2019 at 03:35:10PM +0200, Alexander Graf wrote: > We can easily route hardware interrupts directly into VM context when > they target the "Fixed" or "LowPriority" delivery modes. > > However, on modes such as "SMI" or "Init", we need to go via KVM code > to actually put the vCPU into a different mode of operation, so we can > not post the interrupt > > Add code in the VMX PI logic to explicitly refuse to establish posted > mappings for advanced IRQ deliver modes. This reflects the logic in > __apic_accept_irq() which also only ever passes Fixed and LowPriority > interrupts as posted interrupts into the guest. > > This fixes a bug I have with code which configures real hardware to > inject virtual SMIs into my guest. > > Signed-off-by: Alexander Graf <graf@amazon.com> > Reviewed-by: Liran Alon <liran.alon@oracle.com> > > --- > > v1 -> v2: > > - Make error message more unique > - Update commit message to point to __apic_accept_irq() > --- > arch/x86/kvm/vmx/vmx.c | 22 ++++++++++++++++++++++ > 1 file changed, 22 insertions(+) > > diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c > index 570a233e272b..8029fe658c30 100644 > --- a/arch/x86/kvm/vmx/vmx.c > +++ b/arch/x86/kvm/vmx/vmx.c > @@ -7401,6 +7401,28 @@ static int vmx_update_pi_irte(struct kvm *kvm, unsigned int host_irq, > continue; > } > > + switch (irq.delivery_mode) { > + case dest_Fixed: > + case dest_LowestPrio: > + break; > + default: > + /* > + * For non-trivial interrupt events, we need to go > + * through the full KVM IRQ code, so refuse to take > + * any direct PI assignments here. > + */ IMO, a beefy comment is unnecessary, anyone that is digging through this code has hopefully read the PI spec or at least understands the basic concepts. I.e. it should be obvious that PI can't be used for SMI, etc... > + ret = irq_set_vcpu_affinity(host_irq, NULL); > + if (ret < 0) { > + printk(KERN_INFO > + "non-std IRQ failed to recover, irq: %u\n", > + host_irq); > + goto out; > + } > + > + continue; Using a switch to filter out two types is a bit of overkill. It also probably makes sense to perform the deliver_mode checks before calling kvm_intr_is_single_vcpu(). Why not simply something like this? The existing comment and error message are even generic enough to keep as is. diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index c030c96fc81a..e0111a271a5e 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7372,7 +7372,9 @@ static int vmx_update_pi_irte(struct kvm *kvm, unsigned int host_irq, */ kvm_set_msi_irq(kvm, e, &irq); - if (!kvm_intr_is_single_vcpu(kvm, &irq, &vcpu)) { + if ((irq.delivery_mode != dest_Fixed && + irq.delivery_mode != dest_LowestPrio) || + !kvm_intr_is_single_vcpu(kvm, &irq, &vcpu)) { /* * Make sure the IRTE is in remapped mode if * we don't handle it in posted mode. > + } > + > vcpu_info.pi_desc_addr = __pa(vcpu_to_pi_desc(vcpu)); > vcpu_info.vector = irq.vector; > > -- > 2.17.1 > > > > > Amazon Development Center Germany GmbH > Krausenstr. 38 > 10117 Berlin > Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich > Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B > Sitz: Berlin > Ust-ID: DE 289 237 879 > > > ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH v2 1/2] KVM: VMX: Disable posted interrupts for odd IRQs 2019-09-04 14:40 ` Sean Christopherson @ 2019-09-04 15:36 ` Alexander Graf 2019-09-04 15:51 ` Sean Christopherson 0 siblings, 1 reply; 8+ messages in thread From: Alexander Graf @ 2019-09-04 15:36 UTC (permalink / raw) To: Sean Christopherson Cc: kvm, linux-kernel, x86, H. Peter Anvin, Borislav Petkov, Ingo Molnar, Thomas Gleixner, Joerg Roedel, Jim Mattson, Wanpeng Li, Vitaly Kuznetsov, Radim Krčmář, Paolo Bonzini, Liran Alon On 04.09.19 16:40, Sean Christopherson wrote: > On Wed, Sep 04, 2019 at 03:35:10PM +0200, Alexander Graf wrote: >> We can easily route hardware interrupts directly into VM context when >> they target the "Fixed" or "LowPriority" delivery modes. >> >> However, on modes such as "SMI" or "Init", we need to go via KVM code >> to actually put the vCPU into a different mode of operation, so we can >> not post the interrupt >> >> Add code in the VMX PI logic to explicitly refuse to establish posted >> mappings for advanced IRQ deliver modes. This reflects the logic in >> __apic_accept_irq() which also only ever passes Fixed and LowPriority >> interrupts as posted interrupts into the guest. >> >> This fixes a bug I have with code which configures real hardware to >> inject virtual SMIs into my guest. >> >> Signed-off-by: Alexander Graf <graf@amazon.com> >> Reviewed-by: Liran Alon <liran.alon@oracle.com> >> >> --- >> >> v1 -> v2: >> >> - Make error message more unique >> - Update commit message to point to __apic_accept_irq() >> --- >> arch/x86/kvm/vmx/vmx.c | 22 ++++++++++++++++++++++ >> 1 file changed, 22 insertions(+) >> >> diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c >> index 570a233e272b..8029fe658c30 100644 >> --- a/arch/x86/kvm/vmx/vmx.c >> +++ b/arch/x86/kvm/vmx/vmx.c >> @@ -7401,6 +7401,28 @@ static int vmx_update_pi_irte(struct kvm *kvm, unsigned int host_irq, >> continue; >> } >> >> + switch (irq.delivery_mode) { >> + case dest_Fixed: >> + case dest_LowestPrio: >> + break; >> + default: >> + /* >> + * For non-trivial interrupt events, we need to go >> + * through the full KVM IRQ code, so refuse to take >> + * any direct PI assignments here. >> + */ > > IMO, a beefy comment is unnecessary, anyone that is digging through this > code has hopefully read the PI spec or at least understands the basic > concepts. I.e. it should be obvious that PI can't be used for SMI, etc... > >> + ret = irq_set_vcpu_affinity(host_irq, NULL); >> + if (ret < 0) { >> + printk(KERN_INFO >> + "non-std IRQ failed to recover, irq: %u\n", >> + host_irq); >> + goto out; >> + } >> + >> + continue; > > Using a switch to filter out two types is a bit of overkill. It also The switch should compile into the same as the if() below, it's just a matter of being more verbose in code. > probably makes sense to perform the deliver_mode checks before calling > kvm_intr_is_single_vcpu(). Why not simply something like this? The > existing comment and error message are even generic enough to keep as is. Ok, so how about this, even though it goes against Liran's comment on the combined debug print? If you think it's reasonable despite the broken formatting, I'll be happy to fold the patches and submit as v3. Alex diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 44a5ce57a905..55f68fb0d791 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1581,6 +1581,12 @@ bool kvm_intr_is_single_vcpu(struct kvm *kvm, struct kvm_lapic_irq *irq, void kvm_set_msi_irq(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e, struct kvm_lapic_irq *irq); +static inline bool kvm_irq_is_generic(struct kvm_lapic_irq *irq) +{ + return (irq->delivery_mode == dest_Fixed || + irq->delivery_mode == dest_LowestPrio); +} + static inline void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu) { if (kvm_x86_ops->vcpu_blocking) diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index 1f220a85514f..34cc59518cbb 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -5260,7 +5260,8 @@ get_pi_vcpu_info(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e, kvm_set_msi_irq(kvm, e, &irq); - if (!kvm_intr_is_single_vcpu(kvm, &irq, &vcpu)) { + if (!kvm_intr_is_single_vcpu(kvm, &irq, &vcpu) || + !kvm_irq_is_generic(&irq)) { pr_debug("SVM: %s: use legacy intr remap mode for irq %u\n", __func__, irq.vector); return -1; @@ -5314,6 +5315,7 @@ static int svm_update_pi_irte(struct kvm *kvm, unsigned int host_irq, * 1. When cannot target interrupt to a specific vcpu. * 2. Unsetting posted interrupt. * 3. APIC virtialization is disabled for the vcpu. + * 4. IRQ has extended delivery mode (SMI, INIT, etc) */ if (!get_pi_vcpu_info(kvm, e, &vcpu_info, &svm) && set && kvm_vcpu_apicv_active(&svm->vcpu)) { diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 570a233e272b..69f53809c7bb 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7382,10 +7382,14 @@ static int vmx_update_pi_irte(struct kvm *kvm, unsigned int host_irq, * irqbalance to make the interrupts single-CPU. * * We will support full lowest-priority interrupt later. + * + * In addition, we can only inject generic interrupts using + * the PI mechanism, refuse to route others through it. */ kvm_set_msi_irq(kvm, e, &irq); - if (!kvm_intr_is_single_vcpu(kvm, &irq, &vcpu)) { + if (!kvm_intr_is_single_vcpu(kvm, &irq, &vcpu) || + !kvm_irq_is_generic(&irq)) { /* * Make sure the IRTE is in remapped mode if * we don't handle it in posted mode. Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH v2 1/2] KVM: VMX: Disable posted interrupts for odd IRQs 2019-09-04 15:36 ` Alexander Graf @ 2019-09-04 15:51 ` Sean Christopherson 2019-09-04 15:58 ` Alexander Graf 0 siblings, 1 reply; 8+ messages in thread From: Sean Christopherson @ 2019-09-04 15:51 UTC (permalink / raw) To: Alexander Graf Cc: kvm, linux-kernel, x86, H. Peter Anvin, Borislav Petkov, Ingo Molnar, Thomas Gleixner, Joerg Roedel, Jim Mattson, Wanpeng Li, Vitaly Kuznetsov, Radim Krčmář, Paolo Bonzini, Liran Alon On Wed, Sep 04, 2019 at 05:36:39PM +0200, Alexander Graf wrote: > > > On 04.09.19 16:40, Sean Christopherson wrote: > >On Wed, Sep 04, 2019 at 03:35:10PM +0200, Alexander Graf wrote: > >>We can easily route hardware interrupts directly into VM context when > >>they target the "Fixed" or "LowPriority" delivery modes. > >> > >>However, on modes such as "SMI" or "Init", we need to go via KVM code > >>to actually put the vCPU into a different mode of operation, so we can > >>not post the interrupt > >> > >>Add code in the VMX PI logic to explicitly refuse to establish posted > >>mappings for advanced IRQ deliver modes. This reflects the logic in > >>__apic_accept_irq() which also only ever passes Fixed and LowPriority > >>interrupts as posted interrupts into the guest. > >> > >>This fixes a bug I have with code which configures real hardware to > >>inject virtual SMIs into my guest. > >> > >>Signed-off-by: Alexander Graf <graf@amazon.com> > >>Reviewed-by: Liran Alon <liran.alon@oracle.com> > >> > >>--- > >> > >>v1 -> v2: > >> > >> - Make error message more unique > >> - Update commit message to point to __apic_accept_irq() > >>--- > >> arch/x86/kvm/vmx/vmx.c | 22 ++++++++++++++++++++++ > >> 1 file changed, 22 insertions(+) > >> > >>diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c > >>index 570a233e272b..8029fe658c30 100644 > >>--- a/arch/x86/kvm/vmx/vmx.c > >>+++ b/arch/x86/kvm/vmx/vmx.c > >>@@ -7401,6 +7401,28 @@ static int vmx_update_pi_irte(struct kvm *kvm, unsigned int host_irq, > >> continue; > >> } > >>+ switch (irq.delivery_mode) { > >>+ case dest_Fixed: > >>+ case dest_LowestPrio: > >>+ break; > >>+ default: > >>+ /* > >>+ * For non-trivial interrupt events, we need to go > >>+ * through the full KVM IRQ code, so refuse to take > >>+ * any direct PI assignments here. > >>+ */ > > > >IMO, a beefy comment is unnecessary, anyone that is digging through this > >code has hopefully read the PI spec or at least understands the basic > >concepts. I.e. it should be obvious that PI can't be used for SMI, etc... > > > >>+ ret = irq_set_vcpu_affinity(host_irq, NULL); > >>+ if (ret < 0) { > >>+ printk(KERN_INFO > >>+ "non-std IRQ failed to recover, irq: %u\n", > >>+ host_irq); > >>+ goto out; > >>+ } > >>+ > >>+ continue; > > > >Using a switch to filter out two types is a bit of overkill. It also > > The switch should compile into the same as the if() below, it's just a > matter of being more verbose in code. > > >probably makes sense to perform the deliver_mode checks before calling > >kvm_intr_is_single_vcpu(). Why not simply something like this? The > >existing comment and error message are even generic enough to keep as is. > > Ok, so how about this, even though it goes against Liran's comment on the > combined debug print? I missed that comment. How often do we expect irq_set_vcpu_affinity() to fail? If it's frequent enough that the debug message matters, maybe it should be a tracepoint. > If you think it's reasonable despite the broken formatting, I'll be happy to > fold the patches and submit as v3. > > > Alex > > > diff --git a/arch/x86/include/asm/kvm_host.h > b/arch/x86/include/asm/kvm_host.h > index 44a5ce57a905..55f68fb0d791 100644 > --- a/arch/x86/include/asm/kvm_host.h > +++ b/arch/x86/include/asm/kvm_host.h > @@ -1581,6 +1581,12 @@ bool kvm_intr_is_single_vcpu(struct kvm *kvm, struct > kvm_lapic_irq *irq, > void kvm_set_msi_irq(struct kvm *kvm, struct kvm_kernel_irq_routing_entry > *e, > struct kvm_lapic_irq *irq); > > +static inline bool kvm_irq_is_generic(struct kvm_lapic_irq *irq) > +{ > + return (irq->delivery_mode == dest_Fixed || > + irq->delivery_mode == dest_LowestPrio); > +} > + > static inline void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu) > { > if (kvm_x86_ops->vcpu_blocking) > diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c > index 1f220a85514f..34cc59518cbb 100644 > --- a/arch/x86/kvm/svm.c > +++ b/arch/x86/kvm/svm.c > @@ -5260,7 +5260,8 @@ get_pi_vcpu_info(struct kvm *kvm, struct > kvm_kernel_irq_routing_entry *e, > > kvm_set_msi_irq(kvm, e, &irq); > > - if (!kvm_intr_is_single_vcpu(kvm, &irq, &vcpu)) { > + if (!kvm_intr_is_single_vcpu(kvm, &irq, &vcpu) || > + !kvm_irq_is_generic(&irq)) { I've never heard/seen the term generic used to describe x86 interrupts. Maybe kvm_irq_is_intr() or kvm_irq_is_vectored_intr()? > pr_debug("SVM: %s: use legacy intr remap mode for irq %u\n", > __func__, irq.vector); > return -1; > @@ -5314,6 +5315,7 @@ static int svm_update_pi_irte(struct kvm *kvm, > unsigned int host_irq, > * 1. When cannot target interrupt to a specific vcpu. > * 2. Unsetting posted interrupt. > * 3. APIC virtialization is disabled for the vcpu. > + * 4. IRQ has extended delivery mode (SMI, INIT, etc) Similarly, 'extended delivery mode' isn't really a thing, it's simply the delivery mode. 4. IRQ is not a vectored interrupt. > */ > if (!get_pi_vcpu_info(kvm, e, &vcpu_info, &svm) && set && > kvm_vcpu_apicv_active(&svm->vcpu)) { > diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c > index 570a233e272b..69f53809c7bb 100644 > --- a/arch/x86/kvm/vmx/vmx.c > +++ b/arch/x86/kvm/vmx/vmx.c > @@ -7382,10 +7382,14 @@ static int vmx_update_pi_irte(struct kvm *kvm, > unsigned int host_irq, > * irqbalance to make the interrupts single-CPU. > * > * We will support full lowest-priority interrupt later. > + * > + * In addition, we can only inject generic interrupts using > + * the PI mechanism, refuse to route others through it. > */ > > kvm_set_msi_irq(kvm, e, &irq); > - if (!kvm_intr_is_single_vcpu(kvm, &irq, &vcpu)) { > + if (!kvm_intr_is_single_vcpu(kvm, &irq, &vcpu) || > + !kvm_irq_is_generic(&irq)) { > /* > * Make sure the IRTE is in remapped mode if > * we don't handle it in posted mode. > > > > > Amazon Development Center Germany GmbH > Krausenstr. 38 > 10117 Berlin > Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich > Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B > Sitz: Berlin > Ust-ID: DE 289 237 879 > > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v2 1/2] KVM: VMX: Disable posted interrupts for odd IRQs 2019-09-04 15:51 ` Sean Christopherson @ 2019-09-04 15:58 ` Alexander Graf 2019-09-04 16:44 ` Sean Christopherson 0 siblings, 1 reply; 8+ messages in thread From: Alexander Graf @ 2019-09-04 15:58 UTC (permalink / raw) To: Sean Christopherson Cc: kvm, linux-kernel, x86, H. Peter Anvin, Borislav Petkov, Ingo Molnar, Thomas Gleixner, Joerg Roedel, Jim Mattson, Wanpeng Li, Vitaly Kuznetsov, Radim Krčmář, Paolo Bonzini, Liran Alon On 04.09.19 17:51, Sean Christopherson wrote: > On Wed, Sep 04, 2019 at 05:36:39PM +0200, Alexander Graf wrote: >> >> >> On 04.09.19 16:40, Sean Christopherson wrote: >>> On Wed, Sep 04, 2019 at 03:35:10PM +0200, Alexander Graf wrote: >>>> We can easily route hardware interrupts directly into VM context when >>>> they target the "Fixed" or "LowPriority" delivery modes. >>>> >>>> However, on modes such as "SMI" or "Init", we need to go via KVM code >>>> to actually put the vCPU into a different mode of operation, so we can >>>> not post the interrupt >>>> >>>> Add code in the VMX PI logic to explicitly refuse to establish posted >>>> mappings for advanced IRQ deliver modes. This reflects the logic in >>>> __apic_accept_irq() which also only ever passes Fixed and LowPriority >>>> interrupts as posted interrupts into the guest. >>>> >>>> This fixes a bug I have with code which configures real hardware to >>>> inject virtual SMIs into my guest. >>>> >>>> Signed-off-by: Alexander Graf <graf@amazon.com> >>>> Reviewed-by: Liran Alon <liran.alon@oracle.com> >>>> >>>> --- >>>> >>>> v1 -> v2: >>>> >>>> - Make error message more unique >>>> - Update commit message to point to __apic_accept_irq() >>>> --- >>>> arch/x86/kvm/vmx/vmx.c | 22 ++++++++++++++++++++++ >>>> 1 file changed, 22 insertions(+) >>>> >>>> diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c >>>> index 570a233e272b..8029fe658c30 100644 >>>> --- a/arch/x86/kvm/vmx/vmx.c >>>> +++ b/arch/x86/kvm/vmx/vmx.c >>>> @@ -7401,6 +7401,28 @@ static int vmx_update_pi_irte(struct kvm *kvm, unsigned int host_irq, >>>> continue; >>>> } >>>> + switch (irq.delivery_mode) { >>>> + case dest_Fixed: >>>> + case dest_LowestPrio: >>>> + break; >>>> + default: >>>> + /* >>>> + * For non-trivial interrupt events, we need to go >>>> + * through the full KVM IRQ code, so refuse to take >>>> + * any direct PI assignments here. >>>> + */ >>> >>> IMO, a beefy comment is unnecessary, anyone that is digging through this >>> code has hopefully read the PI spec or at least understands the basic >>> concepts. I.e. it should be obvious that PI can't be used for SMI, etc... >>> >>>> + ret = irq_set_vcpu_affinity(host_irq, NULL); >>>> + if (ret < 0) { >>>> + printk(KERN_INFO >>>> + "non-std IRQ failed to recover, irq: %u\n", >>>> + host_irq); >>>> + goto out; >>>> + } >>>> + >>>> + continue; >>> >>> Using a switch to filter out two types is a bit of overkill. It also >> >> The switch should compile into the same as the if() below, it's just a >> matter of being more verbose in code. >> >>> probably makes sense to perform the deliver_mode checks before calling >>> kvm_intr_is_single_vcpu(). Why not simply something like this? The >>> existing comment and error message are even generic enough to keep as is. >> >> Ok, so how about this, even though it goes against Liran's comment on the >> combined debug print? > > I missed that comment. > > How often do we expect irq_set_vcpu_affinity() to fail? If it's frequent > enough that the debug message matters, maybe it should be a tracepoint. I don't expect to ever hit that debug print, so I don't think it matters really. > >> If you think it's reasonable despite the broken formatting, I'll be happy to >> fold the patches and submit as v3. >> >> >> Alex >> >> >> diff --git a/arch/x86/include/asm/kvm_host.h >> b/arch/x86/include/asm/kvm_host.h >> index 44a5ce57a905..55f68fb0d791 100644 >> --- a/arch/x86/include/asm/kvm_host.h >> +++ b/arch/x86/include/asm/kvm_host.h >> @@ -1581,6 +1581,12 @@ bool kvm_intr_is_single_vcpu(struct kvm *kvm, struct >> kvm_lapic_irq *irq, >> void kvm_set_msi_irq(struct kvm *kvm, struct kvm_kernel_irq_routing_entry >> *e, >> struct kvm_lapic_irq *irq); >> >> +static inline bool kvm_irq_is_generic(struct kvm_lapic_irq *irq) >> +{ >> + return (irq->delivery_mode == dest_Fixed || >> + irq->delivery_mode == dest_LowestPrio); >> +} >> + >> static inline void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu) >> { >> if (kvm_x86_ops->vcpu_blocking) >> diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c >> index 1f220a85514f..34cc59518cbb 100644 >> --- a/arch/x86/kvm/svm.c >> +++ b/arch/x86/kvm/svm.c >> @@ -5260,7 +5260,8 @@ get_pi_vcpu_info(struct kvm *kvm, struct >> kvm_kernel_irq_routing_entry *e, >> >> kvm_set_msi_irq(kvm, e, &irq); >> >> - if (!kvm_intr_is_single_vcpu(kvm, &irq, &vcpu)) { >> + if (!kvm_intr_is_single_vcpu(kvm, &irq, &vcpu) || >> + !kvm_irq_is_generic(&irq)) { > > I've never heard/seen the term generic used to describe x86 interrupts. > Maybe kvm_irq_is_intr() or kvm_irq_is_vectored_intr()? I was trying to come up with any name that describes "interrupt that we can post". If "intr" is that, I'll be happy to take it. Vectored_intr sounds even worse IMHO :). > >> pr_debug("SVM: %s: use legacy intr remap mode for irq %u\n", >> __func__, irq.vector); >> return -1; >> @@ -5314,6 +5315,7 @@ static int svm_update_pi_irte(struct kvm *kvm, >> unsigned int host_irq, >> * 1. When cannot target interrupt to a specific vcpu. >> * 2. Unsetting posted interrupt. >> * 3. APIC virtialization is disabled for the vcpu. >> + * 4. IRQ has extended delivery mode (SMI, INIT, etc) > > Similarly, 'extended delivery mode' isn't really a thing, it's simply the > delivery mode. s/extended/incompatible/ maybe? Alex Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879 ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v2 1/2] KVM: VMX: Disable posted interrupts for odd IRQs 2019-09-04 15:58 ` Alexander Graf @ 2019-09-04 16:44 ` Sean Christopherson 0 siblings, 0 replies; 8+ messages in thread From: Sean Christopherson @ 2019-09-04 16:44 UTC (permalink / raw) To: Alexander Graf Cc: kvm, linux-kernel, x86, H. Peter Anvin, Borislav Petkov, Ingo Molnar, Thomas Gleixner, Joerg Roedel, Jim Mattson, Wanpeng Li, Vitaly Kuznetsov, Radim Krčmář, Paolo Bonzini, Liran Alon On Wed, Sep 04, 2019 at 05:58:08PM +0200, Alexander Graf wrote: > > On 04.09.19 17:51, Sean Christopherson wrote: > >On Wed, Sep 04, 2019 at 05:36:39PM +0200, Alexander Graf wrote: > >> > >>- if (!kvm_intr_is_single_vcpu(kvm, &irq, &vcpu)) { > >>+ if (!kvm_intr_is_single_vcpu(kvm, &irq, &vcpu) || > >>+ !kvm_irq_is_generic(&irq)) { > > > >I've never heard/seen the term generic used to describe x86 interrupts. > >Maybe kvm_irq_is_intr() or kvm_irq_is_vectored_intr()? > > I was trying to come up with any name that describes "interrupt that we can > post". If "intr" is that, I'll be happy to take it. Vectored_intr sounds > even worse IMHO :). kvm_irq_is_intr() is fine by me if it's clear to everyone else. Alternatively, we could be more literal, e.g. kvm_irq_is_postable(). > > > > >> pr_debug("SVM: %s: use legacy intr remap mode for irq %u\n", > >> __func__, irq.vector); > >> return -1; > >>@@ -5314,6 +5315,7 @@ static int svm_update_pi_irte(struct kvm *kvm, > >>unsigned int host_irq, > >> * 1. When cannot target interrupt to a specific vcpu. > >> * 2. Unsetting posted interrupt. > >> * 3. APIC virtialization is disabled for the vcpu. > >>+ * 4. IRQ has extended delivery mode (SMI, INIT, etc) > > > >Similarly, 'extended delivery mode' isn't really a thing, it's simply the > >delivery mode. > > s/extended/incompatible/ maybe? Ya, much better. ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v2 2/2] KVM: SVM: Disable posted interrupts for odd IRQs 2019-09-04 13:35 [PATCH v2 0/2] KVM: Only use posted interrupts for Fixed/LowPrio MSIs Alexander Graf 2019-09-04 13:35 ` [PATCH v2 1/2] KVM: VMX: Disable posted interrupts for odd IRQs Alexander Graf @ 2019-09-04 13:35 ` Alexander Graf 1 sibling, 0 replies; 8+ messages in thread From: Alexander Graf @ 2019-09-04 13:35 UTC (permalink / raw) To: kvm Cc: linux-kernel, x86, H. Peter Anvin, Borislav Petkov, Ingo Molnar, Thomas Gleixner, Joerg Roedel, Jim Mattson, Wanpeng Li, Vitaly Kuznetsov, Sean Christopherson, Radim Krčmář, Paolo Bonzini, Liran Alon We can easily route hardware interrupts directly into VM context when they target the "Fixed" or "LowPriority" delivery modes. However, on modes such as "SMI" or "Init", we need to go via KVM code to actually put the vCPU into a different mode of operation, so we can not post the interrupt Add code in the SVM PI logic to explicitly refuse to establish posted mappings for advanced IRQ deliver modes. This reflects the logic in __apic_accept_irq() which also only ever passes Fixed and LowPriority interrupts as posted interrupts into the guest. This fixes a bug I have with code which configures real hardware to inject virtual SMIs into my guest. Signed-off-by: Alexander Graf <graf@amazon.com> Reviewed-by: Liran Alon <liran.alon@oracle.com> --- v1 -> v2: - Make error message more unique - Update commit message to point to __apic_accept_irq() --- arch/x86/kvm/svm.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index 1f220a85514f..b86b45b85da8 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -5266,6 +5266,21 @@ get_pi_vcpu_info(struct kvm *kvm, struct kvm_kernel_irq_routing_entry *e, return -1; } + switch (irq.delivery_mode) { + case dest_Fixed: + case dest_LowestPrio: + break; + default: + /* + * For non-trivial interrupt events, we need to go + * through the full KVM IRQ code, so refuse to take + * any direct PI assignments here. + */ + pr_debug("SVM: %s: use legacy intr mode for non-std irq %u\n", + __func__, irq.vector); + return -1; + } + pr_debug("SVM: %s: use GA mode for irq %u\n", __func__, irq.vector); *svm = to_svm(vcpu); @@ -5314,6 +5329,7 @@ static int svm_update_pi_irte(struct kvm *kvm, unsigned int host_irq, * 1. When cannot target interrupt to a specific vcpu. * 2. Unsetting posted interrupt. * 3. APIC virtialization is disabled for the vcpu. + * 4. IRQ has extended delivery mode (SMI, INIT, etc) */ if (!get_pi_vcpu_info(kvm, e, &vcpu_info, &svm) && set && kvm_vcpu_apicv_active(&svm->vcpu)) { -- 2.17.1 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879 ^ permalink raw reply related [flat|nested] 8+ messages in thread
end of thread, other threads:[~2019-09-04 16:44 UTC | newest] Thread overview: 8+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2019-09-04 13:35 [PATCH v2 0/2] KVM: Only use posted interrupts for Fixed/LowPrio MSIs Alexander Graf 2019-09-04 13:35 ` [PATCH v2 1/2] KVM: VMX: Disable posted interrupts for odd IRQs Alexander Graf 2019-09-04 14:40 ` Sean Christopherson 2019-09-04 15:36 ` Alexander Graf 2019-09-04 15:51 ` Sean Christopherson 2019-09-04 15:58 ` Alexander Graf 2019-09-04 16:44 ` Sean Christopherson 2019-09-04 13:35 ` [PATCH v2 2/2] KVM: SVM: " Alexander Graf
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).