From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 644B2D0BB6C for ; Thu, 24 Oct 2024 08:05:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:MIME-Version: References:In-Reply-To:Subject:Cc:To:From:Message-ID:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=CUCmxoBVZP2n1Hxz4HUcNrsLjzV7/dr3KOMsAQ6gxTg=; b=05grOe0HPm0a3f0tXsuuCKutWx X4Q2GJWYw+f/1Q0JZsABp8XvISlgoqR6gUSG54MpMkbNxnvmfQAqVQaFXvu35QgPkCbrtIxc6j3uh 8mc6u4Zb3/0B/27ye4LYXlPMH/9x30gB9u+kB9GiD+N/93Rp8weqrvrRJbpeysUt8dgZTw8K04HHe p30dFFftIh5WWGXfcwLIkgINDGipAwVoJk76+MaldsH/QlIuepUcOJUmwB8Xisy3DKSp8t7wdgi7O cZxygH76N9VkSilDq1tmPrqYw1NeGYPr/L9kZLt4bVHVczFySsfv4ZkGm5fvwfRYYsa2oeaP9aCrO kCIPJFgQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t3spx-0000000HCzq-20uI; Thu, 24 Oct 2024 08:04:53 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1t3smC-0000000HCOm-11rU for linux-arm-kernel@lists.infradead.org; Thu, 24 Oct 2024 08:01:01 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 0AA8A5C2C39; Thu, 24 Oct 2024 08:00:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1C4B3C4CEC7; Thu, 24 Oct 2024 08:00:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729756859; bh=HAP0L2CUBvGLHAnYq2VFTU0BIBBgFUpMKfUzSDe20kk=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=LWy2MlXXsV37H/QfYuBbQz/a7SCirBcAtsJN6O8PuAT09J+Vdr/5/ZfG9PtXWh3G5 sq8QXT7tUwn8T55YNnzY6ENlELnDbPrPt4/3Z+7LtRHkCgFSanUkk2Lm3V/fNdaT3C dJ4PHfc9toYuF1LCZMdDRA3LgQw+LU+IGxC8M3v38tXwAlNwncLDFIGAkWc9LkIzja Y9fYBD1WDxlkZQ8Fp9o1qIviNw8b+kgNQAxFpkLCDAC358SjWTZvPWb8IYJuKDRudt YhXrBG/x6fu+SZp5jfBXK7OcVtnyKlPQmdgRUWbpXFf662HzUIMfEr+XbyGPqrnu84 6wuH0ubFjncSg== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1t3sm8-006MXf-O8; Thu, 24 Oct 2024 09:00:57 +0100 Date: Thu, 24 Oct 2024 09:00:56 +0100 Message-ID: <86y12e2amf.wl-maz@kernel.org> From: Marc Zyngier To: Zhiqiang Ni Cc: Oliver Upton , , , , , , , , , Subject: Re: [bug report] KVM: arm64: vgic-its: Performance degradation on GICv3 LPI injection In-Reply-To: References: User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.4 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: nizhiqiang1@huawei.com, oliver.upton@linux.dev, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, wanghaibin.wang@huawei.com, tangnianyao@huawei.com, wangzhou1@hisilicon.com, yuzenghui@huawei.com, wangyanan55@huawei.com, jiangkunkun@huawei.com, jiaqingtong@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241024_010100_392779_713597C9 X-CRM114-Status: GOOD ( 19.88 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, 24 Oct 2024 06:06:58 +0100, Zhiqiang Ni wrote: > > Hi all, > > I found a performance degradation on GICv3 LPI injection after this > commit ad362fe07fecf0aba839ff2cc59a3617bd42c33f(KVM: arm64: vgic-its: > Avoid potential UAF in LPI translation cache). > > In my testcase, the vm's configuration is 60 VCPU 120G RAM with a > 32-queue NIC and the kernel version is 5.10. The number of new TCP > connections per second changed from 150,000 to 50,000 after this > patch, with the %sys of cpu changed from 15% to 85%. What is the VM running? How is the traffic generated? Without a reproducer, I struggle to see how we are going to analyse this issue. We can't go back to the previous situation anyway, as it has been shown that what we had before was simply unsafe (the commit message explains why). > From the ftrace, I found that the duration of vgic_put_irq() is > 13.320 us, which may be the reason for the performance degradation. > > The call stack looks like below: > kvm_arch_set_irq_inatomic() > vgic_has_its(); > vgic_its_inject_cached_translation() > vgic_its_check_cache() > vgic_queue_irq_unlock() > vgic_put_irq() Are you suggesting that it is the combination of vgic_get_irq_kref() + vgic_irq_put() that leads to excessive latency? Both are essentially atomic operations, which should be pretty cheap on a modern CPU (anything with FEAT_LSE). The patch below indicates that you are looking at a rather old kernel (6.8). What is the result on a more recent kernel (from 6.10)? Thanks, M. -- Without deviation from the norm, progress is not possible.