From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43372C433EF for ; Sat, 6 Nov 2021 11:49:16 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 70D7160E05 for ; Sat, 6 Nov 2021 11:49:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 70D7160E05 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.ozlabs.org Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4HmbJ96Zsfz3c7Z for ; Sat, 6 Nov 2021 22:49:13 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=kernel.org (client-ip=198.145.29.99; helo=mail.kernel.org; envelope-from=maz@kernel.org; receiver=) Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4HmbHj4T99z2xt5 for ; Sat, 6 Nov 2021 22:48:49 +1100 (AEDT) Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 480FD60E05; Sat, 6 Nov 2021 11:48:47 +0000 (UTC) Received: from sofa.misterjones.org ([185.219.108.64] helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1mjKBg-003oWA-Ui; Sat, 06 Nov 2021 11:48:45 +0000 Date: Sat, 06 Nov 2021 11:48:39 +0000 Message-ID: <87mtmhec88.wl-maz@kernel.org> From: Marc Zyngier To: Sean Christopherson Subject: Re: [PATCH 5/5] KVM: Convert the kvm->vcpus array to a xarray In-Reply-To: References: <20211105192101.3862492-1-maz@kernel.org> <20211105192101.3862492-6-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: seanjc@google.com, kvm@vger.kernel.org, linux-mips@vger.kernel.org, kvmarm@lists.cs.columbia.edu, linuxppc-dev@lists.ozlabs.org, chenhuacai@kernel.org, aleksandar.qemu.devel@gmail.com, anup.patel@wdc.com, atish.patra@wdc.com, borntraeger@de.ibm.com, frankja@linux.ibm.com, david@redhat.com, imbrenda@linux.ibm.com, pbonzini@redhat.com, jgross@suse.com, npiggin@gmail.com, paulus@samba.org, mpe@ellerman.id.au, james.morse@arm.com, suzuki.poulose@arm.com, alexandru.elisei@arm.com, kernel-team@android.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Alexandru Elisei , Anup Patel , Janosch Frank , kvm@vger.kernel.org, Christian Borntraeger , Huacai Chen , David Hildenbrand , linux-mips@vger.kernel.org, Nicholas Piggin , Atish Patra , Aleksandar Markovic , Paul Mackerras , James Morse , Paolo Bonzini , kernel-team@android.com, Claudio Imbrenda , linuxppc-dev@lists.ozlabs.org, kvmarm@lists.cs.columbia.edu, Suzuki K Poulose Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Fri, 05 Nov 2021 20:21:36 +0000, Sean Christopherson wrote: > > On Fri, Nov 05, 2021, Marc Zyngier wrote: > > At least on arm64 and x86, the vcpus array is pretty huge (512 entries), > > and is mostly empty in most cases (running 512 vcpu VMs is not that > > common). This mean that we end-up with a 4kB block of unused memory > > in the middle of the kvm structure. > > Heh, x86 is now up to 1024 entries. Humph. I don't want to know whether people are actually using that in practice. The only time I create VMs with 512 vcpus is to check whether it still works... > > > Instead of wasting away this memory, let's use an xarray instead, > > which gives us almost the same flexibility as a normal array, but > > with a reduced memory usage with smaller VMs. > > > > Signed-off-by: Marc Zyngier > > --- > > @@ -693,7 +694,7 @@ static inline struct kvm_vcpu *kvm_get_vcpu(struct kvm *kvm, int i) > > > > /* Pairs with smp_wmb() in kvm_vm_ioctl_create_vcpu. */ > > smp_rmb(); > > - return kvm->vcpus[i]; > > + return xa_load(&kvm->vcpu_array, i); > > } > > It'd be nice for this series to convert kvm_for_each_vcpu() to use > xa_for_each() as well. Maybe as a patch on top so that potential > explosions from that are isolated from the initiali conversion? > > Or maybe even use xa_for_each_range() to cap at online_vcpus? > That's technically a functional change, but IMO it's easier to > reason about iterating over a snapshot of vCPUs as opposed to being > able to iterate over vCPUs as their being added. In practice I > doubt it matters. > > #define kvm_for_each_vcpu(idx, vcpup, kvm) \ > xa_for_each_range(&kvm->vcpu_array, idx, vcpup, 0, atomic_read(&kvm->online_vcpus)) > I think that's already the behaviour of this iterator (we stop at the first empty slot capped to online_vcpus. The only change in behaviour is that vcpup currently holds a pointer to the last vcpu in no empty slot has been encountered. xa_for_each{,_range}() would set the pointer to NULL at all times. I doubt anyone relies on that, but it is probably worth eyeballing some of the use cases... Thanks, M. -- Without deviation from the norm, progress is not possible.