From mboxrd@z Thu Jan 1 00:00:00 1970 From: DuanZhenzhong Subject: passthroughed msix device Date: Tue, 26 Feb 2013 16:56:50 +0800 Message-ID: <512C78D2.8030903@oracle.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: xen-devel Cc: Konrad Rzeszutek Wilk , Joe Jin , stefano.stabellini@eu.citrix.com List-Id: xen-devel@lists.xenproject.org Hi maintainer, Reprodue an bug on xen-unstable, it's an irq affinity issue for passthroughed msix device to uek1 pvhvm(2.6.32 stable). I passthrough two mptsas devices(0000:0d:0.0, 0000:1f:0.0) to a pvhvm, irq affinity can't be changed. Step to reproduce is as below: 1. xl -f pci-assignable-add 0000:0d:0.0; xl -f pci-assignable-add 0000:1f:0.0 2.xm cr -c vm.cfg [root@rac10box2 ~]# cat /proc/interrupts |grep mpt2sas0 48: 340449 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge mpt2sas0-msix0 [root@rac10box2 ~]# cat /proc/irq/48/smp_affinity 0fff [root@rac10box2 ~]# echo 2>/proc/irq/48/smp_affinity [root@rac10box2 ~]# cat /proc/irq/48/smp_affinity 0002 [root@rac10box2 ~]# cat /proc/interrupts |grep mpt 48: 342051 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge mpt2sas0-msix0 After change affinity to vcpu1, interrupt still happen on vcpu0. And sometimes we got "/No irq handler for vector (irq -1)/" and panic. qemu-dm.log shows error: pt_msixctrl_reg_write: guest enabling MSI-X, disable MSI-INTx translation pci_intx: intx=1 pt_msi_disable: Unmap msi with pirq 4f pt_msix_update_one: Update msix entry 0 with pirq 4f gvec 59 pt_msixctrl_reg_write: guest enabling MSI-X, disable MSI-INTx translation pci_intx: intx=1 pt_msi_disable: Unmap msi with pirq 4e pt_msix_update_one: Update msix entry 0 with pirq 4e gvec 69 pci_msix_writel: Can't update entry 0 since MSI-X is already enabled (fee00000 -> fee02000) pci_msix_writel: Can't update entry 0 since MSI-X is already enabled (00004059 -> 00004071) Can't reproduce with uek2(3.1 stable), but if I disable hvm_pirqs support in uek2, reproduce the same. Test patch below: diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c index a22ad4b..2d795d1 100644 --- a/arch/x86/pci/xen.c +++ b/arch/x86/pci/xen.c @-400,7 +400,6 @int __init pci_xen_init(void) . int __init pci_xen_hvm_init(void) { - if (!xen_have_vector_callback || !xen_feature(XENFEAT_hvm_pirqs)) return 0; . #ifdef CONFIG_ACPI diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c index bf28d69..64f6c6c 100644 --- a/arch/x86/xen/enlighten.c +++ b/arch/x86/xen/enlighten.c @-1536,8 +1536,6 @bool xen_hvm_need_lapic(void) return false; if (!xen_hvm_domain()) return false; - if (xen_feature(XENFEAT_hvm_pirqs) && xen_have_vector_callback) - return false; return true; } EXPORT_SYMBOL_GPL(xen_hvm_need_lapic); btw: can't reproduce too if disable msi support. I'll upload what you need, just tell me. thanks zduan