From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8FFDB12B6A for ; Fri, 26 Jan 2024 10:53:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706266380; cv=none; b=TtKsbUKtiVI7fhYn715D72MEiDAMHcsNo3D7W+fEcq4w8we19yCrsA8ORjif8biEtlk5lPmVGPBRuC3mZznubzo/1gULYXAs1CSSBblv/sY63TIPx04uK8F0wdyl5SobTDjBf5N18hnS0z/JWmGRwQBuXjeX1tw6kOJTfjb5+js= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706266380; c=relaxed/simple; bh=sEgq4/YsbJNz7vu+BBSK9CDr8Ju97ZXzAHrlHF3qI5k=; h=Date:Message-ID:From:To:Cc:Subject:In-Reply-To:References: MIME-Version:Content-Type; b=hlIkI+y2vDR7jZiGBzzS4T9WEzecJSVYtFF8ScxTCZqtYl3nzxCt1y6zqYS090j9pTGA459D1ymWXFKYks7LusnzSq19qpcKxQwWFI2XMJDzxmEPNGxAkpMWmpljs7s2k75DSHcEqit5W+tlraS31q9HTTykXT6sWi1sIrdBVLU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=g4BiQYwD; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="g4BiQYwD" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 148BEC433B1; Fri, 26 Jan 2024 10:53:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706266380; bh=sEgq4/YsbJNz7vu+BBSK9CDr8Ju97ZXzAHrlHF3qI5k=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=g4BiQYwDNHe7n+2QTpulRT6DRltj441EHVKxGZFmfgxWkNGIhP7l7oKvaFabD+ZK7 cjlPBSyIigftTx/aJtsBWcIGWKB/E3YLRpQXHr/Jb5q/Sgjikel+TMl/CscwQdhuK9 t8LPWcgPhFI04W3QyCPg93JRCz4A8Zy3h6wc0GOHgm1gIL+YBtaYbXCIPd39qSTKq3 WBmesJBqKTwVWstblC+efr0/I2bKSL53iIwA3AwnplFu/kRiqlhOmQhDl5IGNeJGqa wwRSYFV8kGB/o6zFJCsXMGZg+0fFgy0TAmHkHs97KSlK+HSfQmiiLP+3CJNEdKWzGs g/SmRP7GJ2vuQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1rTJpR-00F1hi-QT; Fri, 26 Jan 2024 10:52:57 +0000 Date: Fri, 26 Jan 2024 10:52:57 +0000 Message-ID: <86sf2k74dy.wl-maz@kernel.org> From: Marc Zyngier To: Kunkun Jiang Cc: Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , Thomas Gleixner , , , Subject: Re: [PATCH v2 2/2] irqchip/gic-v4.1: Fix skipping VMOVP in cpu offline scenario In-Reply-To: <20240126103012.1020-3-jiangkunkun@huawei.com> References: <20240126103012.1020-1-jiangkunkun@huawei.com> <20240126103012.1020-3-jiangkunkun@huawei.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: jiangkunkun@huawei.com, oliver.upton@linux.dev, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, tglx@linutronix.de, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, wanghaibin.wang@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false On Fri, 26 Jan 2024 10:30:12 +0000, Kunkun Jiang wrote: > > commit dd3f050a216e ("irqchip/gic-v4.1: Implement the v4.1 flavour > of VMOVP") make an optimization, VMOVP can be skipped if moving > VPE to a cpu whose RD is sharing its VPE table with the current one. > > In the cpu offline scenario, the mask_val is the entire cpu range, > except for the offline cpu. Therefore, the first cpu is CPU0. In > corner case, this may result in lost interrupts: > 0. Each cpu die shares a VPE table and contains 32 CPUs > die0(CPU0-31) die1(CPU32-63)... > 1. VPE resides on CPU32, doorbell affinity to CPU32. > 2. Move VPE to CPU16, doorbell affinity to CPU16. > 3. Manually offline CPU16 on the host side: > 'echo 0 > /sys/devices/system/cpu/cpu16/online' > 4. VMOVP will be skipped. > 5. Subsequent doorbell interrupts will be lost. > > So VMOVP cannot be skipped when the affinity CPU is not in mask_val. > > Fixes: dd3f050a216e ("irqchip/gic-v4.1: Implement the v4.1 flavour of VMOVP") > Signed-off-by: Kunkun Jiang > --- > drivers/irqchip/irq-gic-v3-its.c | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c > index 4b1dbb697959..bfb922f16bc6 100644 > --- a/drivers/irqchip/irq-gic-v3-its.c > +++ b/drivers/irqchip/irq-gic-v3-its.c > @@ -3817,7 +3817,7 @@ static int its_vpe_set_affinity(struct irq_data *d, > bool force) > { > struct its_vpe *vpe = irq_data_get_irq_chip_data(d); > - int from, cpu = cpumask_first(mask_val); > + int cur, from, cpu = cpumask_first(mask_val); > unsigned long flags; > > /* > @@ -3839,11 +3839,13 @@ static int its_vpe_set_affinity(struct irq_data *d, > > vpe->col_idx = cpu; > > + cur = cpumask_first(irq_data_get_effective_affinity_mask(d)); When is this not equal to 'from'? > /* > * GICv4.1 allows us to skip VMOVP if moving to a cpu whose RD > * is sharing its VPE table with the current one. > */ > if (gic_data_rdist_cpu(cpu)->vpe_table_mask && > + cpumask_test_cpu(cur, mask_val) && > cpumask_test_cpu(from, gic_data_rdist_cpu(cpu)->vpe_table_mask)) > goto out; This looks utterly pointless to me. M. -- Without deviation from the norm, progress is not possible. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 22B3DC48260 for ; Fri, 26 Jan 2024 10:53:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Cy7fcSFzw2dR2v5QXuG1oaa0Rt2Dg7/fQDGfOvDVjtA=; b=ETkJ2/Tv5FSoa8 SKZuA6FBDkozcIB+f0Ehc6xv5RlDEYkLllCmnZmkEjABQceH6PvK4cDx9ua/ozKTut7/XroQEJrHh tzHnQ3fCBjV3x/w6G+DwSqKBz2R/fvV0LfOcvyLvAZowwkghb9EtSbpEybIROvkC5j8tAiCVXTmVj +O2WRpdCLJX8qrXmpwO5X+Vw/x5NLqT9t6djNmrs778UamsTqe+226PNj3baUSUVmkCmoLdo2XGw/ +3LNeIIzMgwvjt50QmR+hL2kxjeIZT9H5TUaVF03Gm0Oye2lg3IXVkAEfZ1GjmdmRgaquT1ztvip/ LKEaFHsjIjSvs7Wmv6qw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rTJpb-00000003u6D-3Hgi; Fri, 26 Jan 2024 10:53:08 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rTJpV-00000003u4b-06Y5 for linux-arm-kernel@lists.infradead.org; Fri, 26 Jan 2024 10:53:02 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 793A762481; Fri, 26 Jan 2024 10:53:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 148BEC433B1; Fri, 26 Jan 2024 10:53:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706266380; bh=sEgq4/YsbJNz7vu+BBSK9CDr8Ju97ZXzAHrlHF3qI5k=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=g4BiQYwDNHe7n+2QTpulRT6DRltj441EHVKxGZFmfgxWkNGIhP7l7oKvaFabD+ZK7 cjlPBSyIigftTx/aJtsBWcIGWKB/E3YLRpQXHr/Jb5q/Sgjikel+TMl/CscwQdhuK9 t8LPWcgPhFI04W3QyCPg93JRCz4A8Zy3h6wc0GOHgm1gIL+YBtaYbXCIPd39qSTKq3 WBmesJBqKTwVWstblC+efr0/I2bKSL53iIwA3AwnplFu/kRiqlhOmQhDl5IGNeJGqa wwRSYFV8kGB/o6zFJCsXMGZg+0fFgy0TAmHkHs97KSlK+HSfQmiiLP+3CJNEdKWzGs g/SmRP7GJ2vuQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1rTJpR-00F1hi-QT; Fri, 26 Jan 2024 10:52:57 +0000 Date: Fri, 26 Jan 2024 10:52:57 +0000 Message-ID: <86sf2k74dy.wl-maz@kernel.org> From: Marc Zyngier To: Kunkun Jiang Cc: Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , Thomas Gleixner , , , Subject: Re: [PATCH v2 2/2] irqchip/gic-v4.1: Fix skipping VMOVP in cpu offline scenario In-Reply-To: <20240126103012.1020-3-jiangkunkun@huawei.com> References: <20240126103012.1020-1-jiangkunkun@huawei.com> <20240126103012.1020-3-jiangkunkun@huawei.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: jiangkunkun@huawei.com, oliver.upton@linux.dev, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, tglx@linutronix.de, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, wanghaibin.wang@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240126_025301_251049_E4E8F9CE X-CRM114-Status: GOOD ( 25.41 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, 26 Jan 2024 10:30:12 +0000, Kunkun Jiang wrote: > > commit dd3f050a216e ("irqchip/gic-v4.1: Implement the v4.1 flavour > of VMOVP") make an optimization, VMOVP can be skipped if moving > VPE to a cpu whose RD is sharing its VPE table with the current one. > > In the cpu offline scenario, the mask_val is the entire cpu range, > except for the offline cpu. Therefore, the first cpu is CPU0. In > corner case, this may result in lost interrupts: > 0. Each cpu die shares a VPE table and contains 32 CPUs > die0(CPU0-31) die1(CPU32-63)... > 1. VPE resides on CPU32, doorbell affinity to CPU32. > 2. Move VPE to CPU16, doorbell affinity to CPU16. > 3. Manually offline CPU16 on the host side: > 'echo 0 > /sys/devices/system/cpu/cpu16/online' > 4. VMOVP will be skipped. > 5. Subsequent doorbell interrupts will be lost. > > So VMOVP cannot be skipped when the affinity CPU is not in mask_val. > > Fixes: dd3f050a216e ("irqchip/gic-v4.1: Implement the v4.1 flavour of VMOVP") > Signed-off-by: Kunkun Jiang > --- > drivers/irqchip/irq-gic-v3-its.c | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c > index 4b1dbb697959..bfb922f16bc6 100644 > --- a/drivers/irqchip/irq-gic-v3-its.c > +++ b/drivers/irqchip/irq-gic-v3-its.c > @@ -3817,7 +3817,7 @@ static int its_vpe_set_affinity(struct irq_data *d, > bool force) > { > struct its_vpe *vpe = irq_data_get_irq_chip_data(d); > - int from, cpu = cpumask_first(mask_val); > + int cur, from, cpu = cpumask_first(mask_val); > unsigned long flags; > > /* > @@ -3839,11 +3839,13 @@ static int its_vpe_set_affinity(struct irq_data *d, > > vpe->col_idx = cpu; > > + cur = cpumask_first(irq_data_get_effective_affinity_mask(d)); When is this not equal to 'from'? > /* > * GICv4.1 allows us to skip VMOVP if moving to a cpu whose RD > * is sharing its VPE table with the current one. > */ > if (gic_data_rdist_cpu(cpu)->vpe_table_mask && > + cpumask_test_cpu(cur, mask_val) && > cpumask_test_cpu(from, gic_data_rdist_cpu(cpu)->vpe_table_mask)) > goto out; This looks utterly pointless to me. M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel