From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B9AB1134DD for ; Thu, 7 Sep 2023 18:15:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5449BC433C8; Thu, 7 Sep 2023 18:15:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1694110533; bh=5szpdA83l9R1ikFYVyG1shRGZ7+F4onPaR5XCxJ8/Tc=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=VCSZbDLz4YecrxSQkSQG7ueSL/fV36qFTMzY/NjxaHFGH+HHJIYEZK11CHQnAPzfX a+57O4h8ogot7mSjABe98hHAzcEjNcsGHO/3L6r8CEfIlKHTlPQ7KIm6eIF+hWW/1v 9cTFSB0Pu+8YM7fePGv5ni8QeZsuidRDawWn75nHpiVgMFXA8/nP62T7NI+l1gK4DH YBXT+/5Fh7xaS72TiwCcYxl2/k0ZPdg7j1iC5cGWoqVa7q+RbYbrfuzW5Tq49NaId0 ShX+dhcs5e+cYvKzoi7NdYJN/4cE3+m6Euty1v6dnuvdlvboFuJjaJFrrX47PyFjin cWgSfzwf0Q0UQ== Received: from [82.132.186.150] (helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1qeJXO-00BAmG-Lk; Thu, 07 Sep 2023 19:15:30 +0100 Date: Thu, 07 Sep 2023 19:15:38 +0100 Message-ID: <8734zpq27p.wl-maz@kernel.org> From: Marc Zyngier To: Joey Gouly Cc: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Xu Zhao Subject: Re: [PATCH 2/5] KVM: arm64: Build MPIDR to vcpu index cache at runtime In-Reply-To: <20230907152918.GB69899@e124191.cambridge.arm.com> References: <20230907100931.1186690-1-maz@kernel.org> <20230907100931.1186690-3-maz@kernel.org> <20230907152918.GB69899@e124191.cambridge.arm.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 82.132.186.150 X-SA-Exim-Rcpt-To: joey.gouly@arm.com, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, zhaoxu.35@bytedance.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false On Thu, 07 Sep 2023 16:29:18 +0100, Joey Gouly wrote: > > On Thu, Sep 07, 2023 at 11:09:28AM +0100, Marc Zyngier wrote: [...] > > @@ -578,6 +579,57 @@ static int kvm_vcpu_initialized(struct kvm_vcpu *vcpu) > > return vcpu_get_flag(vcpu, VCPU_INITIALIZED); > > } > > > > +static void kvm_init_mpidr_data(struct kvm *kvm) > > +{ > > + struct kvm_mpidr_data *data = NULL; > > + unsigned long c, mask, nr_entries; > > + u64 aff_set = 0, aff_clr = ~0UL; > > + struct kvm_vcpu *vcpu; > > + > > + mutex_lock(&kvm->arch.config_lock); > > + > > + if (kvm->arch.mpidr_data || atomic_read(&kvm->online_vcpus) == 1) > > + goto out; > > + > > + kvm_for_each_vcpu(c, vcpu, kvm) { > > + u64 aff = kvm_vcpu_get_mpidr_aff(vcpu); > > + aff_set |= aff; > > + aff_clr &= aff; > > + } > > + > > + /* > > + * A significant bit can be either 0 or 1, and will only appear in > > + * aff_set. Use aff_clr to weed out the useless stuff. > > + */ > > + mask = aff_set ^ aff_clr; > > + nr_entries = BIT_ULL(hweight_long(mask)); > > + > > + /* > > + * Don't let userspace fool us. If we need more than a single page > > + * to describe the compressed MPIDR array, just fall back to the > > + * iterative method. Single vcpu VMs do not need this either. > > + */ > > + if (struct_size(data, cmpidr_to_idx, nr_entries) <= PAGE_SIZE) > > + data = kzalloc(struct_size(data, cmpidr_to_idx, nr_entries), > > + GFP_KERNEL_ACCOUNT); > > + > > + if (!data) > > + goto out; > > Probably not a big deal, but if the data doesn't fit, every vCPU will run this > function up until this point (if the data fits or there's only 1 vCPU we bail > out earlier) Yeah, I thought about that when writing this code, and applied the following reasoning: - this code is only run once per vcpu - being able to remember that we cannot allocate the hash table requires at least an extra flag or a special value for the pointer - this sequence is pretty quick (one read/or/and * nr_vcpu^2), and even if you have 512 vcpus, it isn't *that* much stuff given that it is spread across vcpus Now, if someone can actually measure a significant boot-time speed-up, I'll happily add that flag. [...] > Reviewed-by: Joey Gouly Thanks! M. -- Without deviation from the norm, progress is not possible. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0867CEC874B for ; Thu, 7 Sep 2023 18:16:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wmon9jXuqmdmWCqhnLOBDo4MJdSp6iMTnbahGTXRonI=; b=0VmKMcPAmNg2Qe 9bkr8EhQrgl9aeW0jzlK7DR2TiXgPdCg2MXFiVj7ifOdvK2XlIRjXchIaI/MCBofU3+zLVhqJvuGj 0yDQEpF70fS/8vFisGsU6QZ0JEOO94GArv/BmioCT7KyxxoxIJN3ltXOVYxuRSzx7MllPvUjHu3TT D2uG4GIMODBhJfRWMHDUmEg+Llky+7tcDJIkkXeTrTuKhfWX0jlCZs7SDghrxYaBxnswkZOz2aWMa qZamBDwD3yQ1xgBR+yh6tK+HgAVffuSplKR5r3hlN9eXkok+fUHpMDuBtCoBAbQZ/4jiyMswwNwaR 9bBUfbBHdr8TcazTPM5Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qeJXZ-00CXMR-0G; Thu, 07 Sep 2023 18:15:41 +0000 Received: from sin.source.kernel.org ([2604:1380:40e1:4800::1]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qeJXV-00CXKs-0f for linux-arm-kernel@lists.infradead.org; Thu, 07 Sep 2023 18:15:39 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by sin.source.kernel.org (Postfix) with ESMTPS id 1A7AACE1997; Thu, 7 Sep 2023 18:15:35 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5449BC433C8; Thu, 7 Sep 2023 18:15:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1694110533; bh=5szpdA83l9R1ikFYVyG1shRGZ7+F4onPaR5XCxJ8/Tc=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=VCSZbDLz4YecrxSQkSQG7ueSL/fV36qFTMzY/NjxaHFGH+HHJIYEZK11CHQnAPzfX a+57O4h8ogot7mSjABe98hHAzcEjNcsGHO/3L6r8CEfIlKHTlPQ7KIm6eIF+hWW/1v 9cTFSB0Pu+8YM7fePGv5ni8QeZsuidRDawWn75nHpiVgMFXA8/nP62T7NI+l1gK4DH YBXT+/5Fh7xaS72TiwCcYxl2/k0ZPdg7j1iC5cGWoqVa7q+RbYbrfuzW5Tq49NaId0 ShX+dhcs5e+cYvKzoi7NdYJN/4cE3+m6Euty1v6dnuvdlvboFuJjaJFrrX47PyFjin cWgSfzwf0Q0UQ== Received: from [82.132.186.150] (helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1qeJXO-00BAmG-Lk; Thu, 07 Sep 2023 19:15:30 +0100 Date: Thu, 07 Sep 2023 19:15:38 +0100 Message-ID: <8734zpq27p.wl-maz@kernel.org> From: Marc Zyngier To: Joey Gouly Cc: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Xu Zhao Subject: Re: [PATCH 2/5] KVM: arm64: Build MPIDR to vcpu index cache at runtime In-Reply-To: <20230907152918.GB69899@e124191.cambridge.arm.com> References: <20230907100931.1186690-1-maz@kernel.org> <20230907100931.1186690-3-maz@kernel.org> <20230907152918.GB69899@e124191.cambridge.arm.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 82.132.186.150 X-SA-Exim-Rcpt-To: joey.gouly@arm.com, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, zhaoxu.35@bytedance.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230907_111537_611083_B730C704 X-CRM114-Status: GOOD ( 27.07 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, 07 Sep 2023 16:29:18 +0100, Joey Gouly wrote: > > On Thu, Sep 07, 2023 at 11:09:28AM +0100, Marc Zyngier wrote: [...] > > @@ -578,6 +579,57 @@ static int kvm_vcpu_initialized(struct kvm_vcpu *vcpu) > > return vcpu_get_flag(vcpu, VCPU_INITIALIZED); > > } > > > > +static void kvm_init_mpidr_data(struct kvm *kvm) > > +{ > > + struct kvm_mpidr_data *data = NULL; > > + unsigned long c, mask, nr_entries; > > + u64 aff_set = 0, aff_clr = ~0UL; > > + struct kvm_vcpu *vcpu; > > + > > + mutex_lock(&kvm->arch.config_lock); > > + > > + if (kvm->arch.mpidr_data || atomic_read(&kvm->online_vcpus) == 1) > > + goto out; > > + > > + kvm_for_each_vcpu(c, vcpu, kvm) { > > + u64 aff = kvm_vcpu_get_mpidr_aff(vcpu); > > + aff_set |= aff; > > + aff_clr &= aff; > > + } > > + > > + /* > > + * A significant bit can be either 0 or 1, and will only appear in > > + * aff_set. Use aff_clr to weed out the useless stuff. > > + */ > > + mask = aff_set ^ aff_clr; > > + nr_entries = BIT_ULL(hweight_long(mask)); > > + > > + /* > > + * Don't let userspace fool us. If we need more than a single page > > + * to describe the compressed MPIDR array, just fall back to the > > + * iterative method. Single vcpu VMs do not need this either. > > + */ > > + if (struct_size(data, cmpidr_to_idx, nr_entries) <= PAGE_SIZE) > > + data = kzalloc(struct_size(data, cmpidr_to_idx, nr_entries), > > + GFP_KERNEL_ACCOUNT); > > + > > + if (!data) > > + goto out; > > Probably not a big deal, but if the data doesn't fit, every vCPU will run this > function up until this point (if the data fits or there's only 1 vCPU we bail > out earlier) Yeah, I thought about that when writing this code, and applied the following reasoning: - this code is only run once per vcpu - being able to remember that we cannot allocate the hash table requires at least an extra flag or a special value for the pointer - this sequence is pretty quick (one read/or/and * nr_vcpu^2), and even if you have 512 vcpus, it isn't *that* much stuff given that it is spread across vcpus Now, if someone can actually measure a significant boot-time speed-up, I'll happily add that flag. [...] > Reviewed-by: Joey Gouly Thanks! M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel