From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 35978606A31A for ; Sat, 24 Oct 2026 11:50:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key, unprotected) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=bombadil.20210309 header.b=fImQH8fX; dkim=pass (2048-bit key, unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=WEjZ1CRI DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:MIME-Version: References:In-Reply-To:Subject:Cc:To:From:Message-ID:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Unsubscribe-Post:List-Owner; bh=FuqXf7oFIzUBEcQbKPY0llvBnP0Pu2NRXKoSWtEKfn0=; b=fImQH8fXY3Adc8l80DtxYxw2L7 8lCErDBRRm2tZqBw0Ei99RfTZj6sAgkaAYCnOTLAoIIFbJFu2Ir96EJ0MP93b0y/25T/KifNZtkaW mW/XzspXD9yR3aidA+FtLZBkUUcjrwbO7iilHOvxNhVpY7AlAWp0FxJUbvLVdp1k/o3bnLp8Z628f DcgGQH1Smg5gPBuxIx4qZ8xHqYNVveG2ZivZGERdyXUk5JhnSo92uV2SRbUV2RWWzDj/lc8sxot70 n5bbBlNnABdRYjLgTWLynu7untetZbnqlvEIqkoK4kgLIkIT7Q0xPZ0Rhc3Y6JZE0R5tdK1Gd1u07 MvVxSX5g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.100.1 #2 (Red Hat Linux)) id 1xKaFg-000000008fy-2iKR; Sat, 24 Oct 2026 11:49:32 +0000 Received: from sea.source.kernel.org ([2600:3c0a:e001:78e:0:1991:8:25]) by bombadil.infradead.org with esmtps (Exim 4.100.1 #2 (Red Hat Linux)) id 1xKaFe-000000008fj-2CEi for linux-arm-kernel@lists.infradead.org; Sat, 24 Oct 2026 11:49:30 +0000 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) by sea.source.kernel.org (Postfix) with ESMTP id CF22D41069; Sat, 24 Oct 2026 11:49:29 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AE9461F01825; Sat, 24 Oct 2026 11:49:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1792842569; bh=FuqXf7oFIzUBEcQbKPY0llvBnP0Pu2NRXKoSWtEKfn0=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=WEjZ1CRIcbCJAp/JIdnJ/lHTbaGXMf7MCXDF9mrTgwfRVdgWDLQeZZ3+EelpxSQX5 SKoLSEERZcVQTB9OD1Vh5eDSMwhPUsrhQPtlWsGhBJ8KDUc9XvShjbHdFA8AC5AZ8N V5SWdX6Gi4m49WsO7Lpey8V4+pLGfNAK7mQNzV/8MCnNUm8YN0W7ze63PhY4AcUCKi 7IrRuKbkGXgShhOliQe5X2QbCNrQ3SCjhjKtAAPNQSnwRdTLx5PpfjgxeBoJ69aXyB UJ/ZCC2KhCQVSKL/Tu/v9MmzMGbv3wnsDmUaLm8qHRfarR72rZxxao2RglWbwtoyJ9 AHufBBq1UK93g== Received: from sofa.misterjones.org ([185.219.108.64] helo=lobster-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xKaFb-00000006kGD-2If9; Sat, 24 Oct 2026 11:49:27 +0000 Date: Sat, 24 Oct 2026 12:52:44 +0100 Message-ID: <87tsmb8gdv.wl-maz@kernel.org> From: Marc Zyngier To: Amir Noohi Cc: Thomas Gleixner , Oliver Upton , Radu Rendec , Igor Velkov , Fuad Tabba , Joey Gouly , Steffen Eiden , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon , Mark Rutland , kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 0/2] genirq, KVM: arm64: Avoid the irq descriptor lock on vCPU load In-Reply-To: <20261024022450.629980-1-amirmnoohi@gmail.com> References: <20261024022450.629980-1-amirmnoohi@gmail.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/30.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) X-Archived-By: MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: amirmnoohi@gmail.com, tglx@kernel.org, oupton@kernel.org, radu@rendec.net, iav@iav.lv, fuad.tabba@linux.dev, joey.gouly@arm.com, seiden@linux.ibm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, catalin.marinas@arm.com, will@kernel.org, mark.rutland@arm.com, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" X-Spam-Status: No, score=-6.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=4.0.1 X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on aws-us-west-2-korg-listserv-alma10-1 On Sat, 24 Oct 2026 03:24:48 +0100, Amir Noohi wrote: > > KVM/arm64 vCPUs slow each other down even when every vCPU has a physical > CPU to itself. On a 32 CPU Ampere Altra (GICv3, VHE) running mainline, > with an in-kernel GICv3 and each vCPU thread pinned to its own otherwise > idle CPU, a vCPU which exits to userspace in a loop manages 320k exits/s > on its own, 120k when 16 vCPUs do the same and 59k with 31. The aggregate > rate stops growing at 8 vCPUs. No two vCPUs share a CPU and nothing is > over-subscribed. > > The cause is the interrupt descriptor lock of the host timer interrupts. > Every vCPU load sets their active state with irq_set_irqchip_state() > (twice on VHE: virtual and physical timer). The timer interrupts are per > CPU interrupts, so all CPUs share one descriptor and its lock, while the > state being written is per CPU (a redistributor register on GICv3, a > system register on GICv5). The lock serializes nothing but CPUs updating > their own state. At 8 vCPUs, 99.9% of all lock contention events during > the test have __irq_get_desc_lock() or irq_set_irqchip_state() on the > stack. Looking at the test case, this really isn't a representative or realistic workload. This is a contrived setup that has trapping to userspace as its only purpose. I'm not saying that this is useless observation. But it is not something that represents something exacerbated by "real" VMs. I'll look at the patches shortly. Thanks, M. -- Jazz isn't dead. It just smells funny.