From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88EA5EE49AC for ; Tue, 22 Aug 2023 08:28:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233822AbjHVI2s (ORCPT ); Tue, 22 Aug 2023 04:28:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41180 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231754AbjHVI2q (ORCPT ); Tue, 22 Aug 2023 04:28:46 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49E801AD; Tue, 22 Aug 2023 01:28:45 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CD54561EBD; Tue, 22 Aug 2023 08:28:44 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 07E40C433C9; Tue, 22 Aug 2023 08:28:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1692692924; bh=7nk22NxlDXVnokJCemNbUMUy0LuKuJ9IPJQjaV1pArQ=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=cBiqOPScRD8EW5z/a14Usyd4J3GEN7eCXw9q4hjcOTml7W/UQpS5ZIGF/tAGhodp+ s23nv7RIpfmICx8M15KE1oRpHPW3NAQcc57JvkhguMCsvGVJipWhazPGA+5YpqI1CA UhBl2vLvTVAt6PZV4y7Fnx9GMEXDfGdOB1Q4FgSNGjGnKVfcVUCklNy2V4GtzLvJFT aA+qyJ5N0UGk91eMlMZI7mhv4lDnA+FctbvrEyNLkxOj1YQg04DxV7SoHSy4cbNr9f SkbyG4soYbiR8keXS/6zMrsJiPOKSzDGSOKCzJfTDiNihuAAmIvHKeTm+5gHq0kYdN KwZeoyzK0GPeg== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1qYMki-006xkD-Dd; Tue, 22 Aug 2023 09:28:41 +0100 Date: Tue, 22 Aug 2023 09:28:40 +0100 Message-ID: <86jztnfpl3.wl-maz@kernel.org> From: Marc Zyngier To: zhaoxu Cc: pbonzini@redhat.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, zhouyibo@bytedance.com, zhouliang.001@bytedance.com, Oliver Upton , kvmarm@lists.linux.dev, Mark Rutland Subject: Re: [RFC] KVM: arm/arm64: optimize vSGI injection performance In-Reply-To: References: <20230818104704.7651-1-zhaoxu.35@bytedance.com> <86msykg0ox.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: zhaoxu.35@bytedance.com, pbonzini@redhat.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, zhouyibo@bytedance.com, zhouliang.001@bytedance.com, oliver.upton@linux.dev, kvmarm@lists.linux.dev, mark.rutland@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On Tue, 22 Aug 2023 04:51:30 +0100, zhaoxu wrote: > > hi marc, thanks for reviewing. > > On 2023/8/21 18:16, Marc Zyngier wrote: > >>> This work is based on v5.4, and here is test data: > > > > This is a 4 year old kernel. I'm afraid you'll have to provide > > something that is relevant to a current (e.i. v6.5) kernel. > > > In fact, the core vCPU search algorithm remains the same in the latest > kernel: iterate all vCPUs, if mpidr matches, inject. next version will > based on latest kernel. My point is that performance numbers on such an ancient kernel hardly make any sense, as a large portion of the code will be different. We aim to live in the future, not in the past. > > >>> Based on the test results, the performance of vm with less than 16 cores remains almost the same, > >>> while significant improvement can be observed with more than 16 > >>> cores. > > > > This triggers multiple questions: > > > > - what is the test being used? on what hardware? how can I reproduce > > this data? > > > 1. I utilized the ipi_benchmark > (https://patchwork.kernel.org/project/linux-arm-kernel/patch/20171211141600.24401-1-ynorov@caviumnetworks.com/) > with a modification to the Normal IPI target in the following manner: > smp_call_function_single(31, handle_ipi, &time, 1). > 2. On kunpeng 920 platform. > 3. Using ipi_benchmark but change the target cpu in Normal IPI case, > and use bcc or bpftrace to measuret the execution time of > vgic_v3_dispatch_sgi. So this is not a self contained benchmark, that on top of it requires some vague additional changes. Great. > > - which current guest OS *currently* make use of broadcast or 1:N > > SGIs? Linux doesn't and overall SGI multicasting is pretty useless > > to an OS. > > > > [...] > Yes, arm64 linux almost never send broadcast ipi. I will use another > test data to prove performence improvement Exactly. I also contend that *no* operating system uses broadcast (or even multicast) signalling, because this is a very pointless operation. So what are you optimising for? > > > >>> /* > >>> - * Compare a given affinity (level 1-3 and a level 0 mask, from the SGI > >>> - * generation register ICC_SGI1R_EL1) with a given VCPU. > >>> - * If the VCPU's MPIDR matches, return the level0 affinity, otherwise > >>> - * return -1. > >>> + * Get affinity routing index from ICC_SGI_* register > >>> + * format: > >>> + * aff3 aff2 aff1 aff0 > >>> + * |- 8 bits -|- 8 bits -|- 8 bits -|- 4 bits or 8bits -| > > > > OK, so you are implementing RSS support: > > > > - Why isn't that mentioned anywhere in the commit log? > > > > - Given that KVM actively limits the MPIDR to 4 bits at Aff0, how does > > it even work the first place? > > > > - How is that advertised to the guest? > > > > - How can the guest enable RSS support? > > > thanks to mention that, I also checked the relevant code, guest can't > enable RSS, it was my oversight. This part has removed in next > version. Then what's the point of your patch? You don't explain anything, which makes it very hard to guess what you're aiming for. M. -- Without deviation from the norm, progress is not possible.