From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59094) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZC5cO-0008Ta-Nd for qemu-devel@nongnu.org; Mon, 06 Jul 2015 08:35:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZC5cN-0006D2-Iz for qemu-devel@nongnu.org; Mon, 06 Jul 2015 08:35:28 -0400 From: Paolo Bonzini Date: Mon, 6 Jul 2015 14:34:57 +0200 Message-Id: <1436186108-29747-9-git-send-email-pbonzini@redhat.com> In-Reply-To: <1436186108-29747-1-git-send-email-pbonzini@redhat.com> References: <1436186108-29747-1-git-send-email-pbonzini@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: [Qemu-devel] [PULL 08/19] Fix irq route entries exceeding KVM_MAX_IRQ_ROUTES List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: =?UTF-8?q?=E9=A9=AC=E6=96=87=E9=9C=9C?= , qemu-stable@nongnu.org From: =E9=A9=AC=E6=96=87=E9=9C=9C Last month, we experienced several guests crash(6cores-8cores), qemu logs display the following messages: qemu-system-x86_64: /build/qemu-2.1.2/kvm-all.c:976: kvm_irqchip_commit_routes: Assertion `ret =3D=3D 0' failed. After analysis and verification, we can confirm it's irq-balance daemon(in guest) leads to the assertion failure. Start a 8 core guest wit= h two disks, execute the following scripts will reproduce the BUG quickly: irq_affinity.sh =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D vda_irq_num=3D25 vdb_irq_num=3D27 while [ 1 ] do for irq in {1,2,4,8,10,20,40,80} do echo $irq > /proc/irq/$vda_irq_num/smp_affinity echo $irq > /proc/irq/$vdb_irq_num/smp_affinity dd if=3D/dev/vda of=3D/dev/zero bs=3D4K count=3D100 iflag=3Dd= irect dd if=3D/dev/vdb of=3D/dev/zero bs=3D4K count=3D100 iflag=3Dd= irect done done =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D QEMU setup static irq route entries in kvm_pc_setup_irq_routing(), PIC an= d IOAPIC share the first 15 GSI numbers, take up 23 GSI numbers, but take u= p 38 irq route entries. When change irq smp_affinity in guest, a dynamic ro= ute entry may be setup, the current logic is: if allocate GSI number succeeds= , a new route entry can be added. The available dynamic GSI numbers is 1021(KVM_MAX_IRQ_ROUTES-23), but available irq route entries is only 986(KVM_MAX_IRQ_ROUTES-38), GSI numbers greater than route entries. irq-balance's behavior will eventually leads to total irq route entries exceed KVM_MAX_IRQ_ROUTES, ioctl(KVM_SET_GSI_ROUTING) fail and kvm_irqchip_commit_routes() trigger assertion failure. This patch fix the BUG. Signed-off-by: Wenshuang Ma Cc: qemu-stable@nongnu.org Signed-off-by: Paolo Bonzini --- kvm-all.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/kvm-all.c b/kvm-all.c index 53e01d4..e98b08d 100644 --- a/kvm-all.c +++ b/kvm-all.c @@ -1099,9 +1099,17 @@ static int kvm_irqchip_get_virq(KVMState *s) uint32_t *word =3D s->used_gsi_bitmap; int max_words =3D ALIGN(s->gsi_count, 32) / 32; int i, zeroes; - bool retry =3D true; =20 -again: + /* + * PIC and IOAPIC share the first 16 GSI numbers, thus the available + * GSI numbers are more than the number of IRQ route. Allocating a G= SI + * number can succeed even though a new route entry cannot be added. + * When this happens, flush dynamic MSI entries to free IRQ route en= tries. + */ + if (!s->direct_msi && s->irq_routes->nr =3D=3D s->gsi_count) { + kvm_flush_dynamic_msi_routes(s); + } + /* Return the lowest unused GSI in the bitmap */ for (i =3D 0; i < max_words; i++) { zeroes =3D ctz32(~word[i]); @@ -1111,11 +1119,6 @@ again: =20 return zeroes + i * 32; } - if (!s->direct_msi && retry) { - retry =3D false; - kvm_flush_dynamic_msi_routes(s); - goto again; - } return -ENOSPC; =20 } --=20 2.4.3