From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1FBEECF6499 for ; Sun, 29 Sep 2024 10:09:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:MIME-Version:References:In-Reply-To:Subject:Cc:To:From: Message-ID:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=mqR6rzXrNqxGxluSCQWRDgzo+r+m03VXSetTOBuTfcU=; b=pjWEO8UAgipliKAwRejImFP6Lk yuhJfzF1vXguCh3fyUUKKg1NBQk5WoXCoo28Wi4+cOHiFznGo08+7/5piR888ZJPPXDFVqgks+5zi D0/joNYbANm1UmtHlvsiRMG/MFRRjj/zGMVMb4h5i5HYeNvMVKp2d6WF3e+V7Fwbs/kTpAFRQFcsV EBVA6gsCYXupZ+1D93ReRQQvXaZHmij4pJwT19GiKK+aF5pxCHeJPuM0I5eu/ruoOyfcD20J7zguu LcHIo94jIZrZ0F/khJFLSi82nSOkoJrkd+2KTckocNJsLQor5f3k5HJRxTfw2nk5sF9ruJGlIAu9W 1xA5nStg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1suqrJ-0000000Ea7k-0qPT; Sun, 29 Sep 2024 10:08:57 +0000 Received: from nyc.source.kernel.org ([147.75.193.91]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1suqq6-0000000EZyC-3hES for linux-arm-kernel@lists.infradead.org; Sun, 29 Sep 2024 10:07:44 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 9FE31A4042E; Sun, 29 Sep 2024 10:07:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 90C3FC4CEC5; Sun, 29 Sep 2024 10:07:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727604461; bh=+1tsaAkdafs2R/KYuS4+eiY1pbGwsl8xbGU+N6LiyW0=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=ipW+2SN70UgWObZAA4QScmZaRbtWf4ugPuJwRbN3pwSUqIBRQAQVwBcMOjMKWZGIt r0NJYtSilA8ts7jbwxdghvms1MD5EvpDWeTS2bXrmKWoopZbB8ELj1nWqVYlhCWqQz JWjejq3l/qfLBoDOxiCG3kCNwNR5hKJvz2hLLA9IoFbCQOvnwYvT06pOTJt4VeMjEK ZbbsZhAQgX/xdkskVH6esEwgGKxC+vrxE8YSuZkB+yA9Ib8yqDy27dbHKDuGGyUTvz xXpdeGvBkMS0y4B71Faan1AFbrZ8uKQhqON1wZQAN3JkZjqfPtN5XVjXxYvei/jpeC Kn9on9HvhbYfA== Received: from [185.143.37.16] (helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1suqq3-00GCGs-G3; Sun, 29 Sep 2024 11:07:39 +0100 Date: Sun, 29 Sep 2024 11:07:38 +0100 Message-ID: <87ttdyvklx.wl-maz@kernel.org> From: Marc Zyngier To: Kunkun Jiang Cc: Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , , , , , Subject: Re: [bug report] KVM: arm64: vgic-v4: Occasionally issue VMOVP to an unmapped VPE on GICv4.1 In-Reply-To: References: User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.4 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.143.37.16 X-SA-Exim-Rcpt-To: jiangkunkun@huawei.com, oliver.upton@linux.dev, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, wanghaibin.wang@huawei.com, tangnianyao@huawei.com, wangzhou1@hisilicon.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240929_030743_080459_23B86A71 X-CRM114-Status: GOOD ( 33.46 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Sun, 29 Sep 2024 08:18:41 +0100, Kunkun Jiang wrote: >=20 > Hi all, >=20 > I found a problem with occasionally issuing VMOVP to an unmapped VPE > on GICv4.1. In my test environment, operating an unmapped VPE will > generate RAS, so I found this problem. The detailed analysis is as > follows. >=20 > The vgic_v4_teardown() will be executed when VM is destroyed to free > the GICv4 data structures. The code is as follows: > > /** > > * vgic_v4_teardown - Free the GICv4 data structures > > * @kvm: Pointer to the VM being destroyed > > */ > > void vgic_v4_teardown(struct kvm *kvm) > > { > > struct its_vm *its_vm =3D &kvm->arch.vgic.its_vm; > > int i; > >=20 > > lockdep_assert_held(&kvm->arch.config_lock); > >=20 > > if (!its_vm->vpes) > > return; > >=20 > > for (i =3D 0; i < its_vm->nr_vpes; i++) { > > struct kvm_vcpu *vcpu =3D kvm_get_vcpu(kvm, i); > > int irq =3D its_vm->vpes[i]->irq; > >=20 > > irq_clear_status_flags(irq, DB_IRQ_FLAGS); > > free_irq(irq, vcpu); > > } > >=20 > > its_free_vcpu_irqs(its_vm); > > kfree(its_vm->vpes); > > its_vm->nr_vpes =3D 0; > > its_vm->vpes =3D NULL; > > } >=20 > [1] In irq_clear_status_flags(irq, DB_IRQ_FLAGS), the status flags of > a doorbell are cleared. DB_IRQ_FLAGS contains IRQ_NO_BALANCING. So > after this,the irqbalance.service can schedule the doorbell. > [2] In free_irq(), the VPE is unmaped. > [3] In its_free_vcpu_irqs(its_vm), unregister_irq_proc() is called to > delete the contents in /proc/irq/xx/ of the doorbell. >=20 > For VPEs in large-scale VM, there is a centain time window between [2] > and [3]. The irqbalance.service got a chance to schedule the > doorbell. Therefore, the VMOVP is issued to an unmapped VPE. >=20 > I tried not clearing IRQ_NO_BALANCING and the problem was solved. But > it's not clear if there's any other problem with doing so. I don't think that's a good idea, because whoever request the same interrupt number again for a different purpose will have the flag set and will experience odd behaviours. I'd rather fix it for good, given that we have all the necessary tracking in place already. Something like the patch below, as usual untested. Thanks, M. =46rom c0fe216c50651458fdf1ebb6b3a1e4ffe2bcc6c2 Mon Sep 17 00:00:00 2001 From: Marc Zyngier Date: Sun, 29 Sep 2024 10:58:19 +0100 Subject: [PATCH] irqchip/gic-v4: Don't allow a VMOVP on a dying VPE Kunkun Jiang reports that there is a small window of opportunity for userspace to force a change of affinity for a VPE while the VPE has already been unmapped, but the corresponding doorbell interrupt still visible in /proc/irq/. Plug the race by checking the value of vmapp_count, which tracks whether the VPE is mapped ot not, and returning an error in this case. Reported-by: Kunkun Jiang Signed-off-by: Marc Zyngier Link: https://lore.kernel.org/r/c182ece6-2ba0-ce4f-3404-dba7a3ab6c52@huawei= .com --- drivers/irqchip/irq-gic-v3-its.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-= its.c index fdec478ba5e7..ba9734d1a41f 100644 --- a/drivers/irqchip/irq-gic-v3-its.c +++ b/drivers/irqchip/irq-gic-v3-its.c @@ -3806,6 +3806,13 @@ static int its_vpe_set_affinity(struct irq_data *d, struct cpumask *table_mask; unsigned long flags; =20 + /* + * Check if we're racing against a VPE being destroyed, for + * which we don't want to allow a VMOVP. + */ + if (!atomic_read(&vpe->vmapp_count)) + return -EINVAL; + /* * Changing affinity is mega expensive, so let's be as lazy as * we can and only do it if we really have to. Also, if mapped --=20 2.43.0 --=20 Without deviation from the norm, progress is not possible.