From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A421CC3ABA3 for ; Thu, 1 May 2025 13:55:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=1kZwyCpGWpE7QSbaHB0+NeGXum0zzsshEtEYs3LkF0M=; b=SSFfqMxzTOQQAh vDSJ+E8UmqkreyothU/CZOAeOAvrUQGrnXPVm2uU1CePuyUKsRxf7jQNUQOR/h7uwRGk85eDyvsJv lk4wqyxdDk/S4MEA4Y3kZELSLXgSntSh8O6zBNUSg6jeDoVnuRnrA4Q8PM9IBJ73arEZktSibKDQd KfeT+Zm9MA6yghW2eYXoIOD73HdxNwC6DD0NMIl/JY+1xC1TQOD51f5KO4KMyZCydELMGXnhth7jj iu/KDskkEvDkuvM/rjVt9D1It7Fyat1RZpF8ZpYtl/9Me7GHBI+ac+bPH65Fu525qcLEKNzBXEkDF Wi3/+hrR1JY2E6614BvQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1uAUNf-0000000FtFP-1ZG7; Thu, 01 May 2025 13:55:15 +0000 Received: from tor.source.kernel.org ([172.105.4.254]) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1uAUNd-0000000FtEa-3hkD; Thu, 01 May 2025 13:55:13 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by tor.source.kernel.org (Postfix) with ESMTP id 11D4E68463; Thu, 1 May 2025 13:54:45 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7E48EC4CEE3; Thu, 1 May 2025 13:55:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1746107712; bh=HP9qPQcXPFBaNtd0LbBqA0JNCPyQw698ggOHjYeim0Y=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=Pfk1ZFoGFQBcJbn4eAazqmyuBJ6TkiGf2ekSQDgoa5Fjia9S2lMt5efsrXyzq1Vvm ULeuC83dmMDOduBWW3xv2ptavXQm1vdoGdMaOq6foW7rtSQf8ouqLVTIFegJWn1fzS EorN2jN0LePdF11Vpm6UW5UyEsjAfLLhsNMAJTWfxtw7S29pRommXhIjicwQXKd09K dja4faMd/aLvbeN4QKFNqUIDiiz0PK5zIlRXOlq9ONM7WY09BvtawseMmekfxpdUph 231yeafldT6FquJnpqz72EltQ8Tb6TsqSFX9obyV9riRozvuNYg8YfhJH7B6vS3/oQ Lfve/zxbzhOzw== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1uAUNa-00AakJ-3R; Thu, 01 May 2025 14:55:10 +0100 Date: Thu, 01 May 2025 14:55:08 +0100 Message-ID: <86v7qkh1vn.wl-maz@kernel.org> From: Marc Zyngier To: Peter Zijlstra Cc: Maxim Levitsky , kvm@vger.kernel.org, linux-riscv@lists.infradead.org, Kunkun Jiang , Waiman Long , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Catalin Marinas , Bjorn Helgaas , Boqun Feng , Borislav Petkov , Albert Ou , Anup Patel , Paul Walmsley , Suzuki K Poulose , Palmer Dabbelt , Alexandre Ghiti , Alexander Potapenko , Oliver Upton , Andre Przywara , x86@kernel.org, Joey Gouly , Thomas Gleixner , kvm-riscv@lists.infradead.org, Atish Patra , Ingo Molnar , Jing Zhang , "H. Peter Anvin" , Dave Hansen , kvmarm@lists.linux.dev, Will Deacon , Keisuke Nishimura , Sebastian Ott , Shusen Li , Paolo Bonzini , Randy Dunlap , Sean Christopherson , Zenghui Yu Subject: Re: [PATCH v4 2/5] arm64: KVM: use mutex_trylock_nest_lock when locking all vCPUs In-Reply-To: <20250501134126.GT4439@noisy.programming.kicks-ass.net> References: <20250430203013.366479-1-mlevitsk@redhat.com> <20250430203013.366479-3-mlevitsk@redhat.com> <864iy4ivro.wl-maz@kernel.org> <20250501111552.GO4198@noisy.programming.kicks-ass.net> <861pt8ijpv.wl-maz@kernel.org> <20250501134126.GT4439@noisy.programming.kicks-ass.net> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/30.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: peterz@infradead.org, mlevitsk@redhat.com, kvm@vger.kernel.org, linux-riscv@lists.infradead.org, jiangkunkun@huawei.com, longman@redhat.com, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, catalin.marinas@arm.com, bhelgaas@google.com, boqun.feng@gmail.com, bp@alien8.de, aou@eecs.berkeley.edu, anup@brainfault.org, paul.walmsley@sifive.com, suzuki.poulose@arm.com, palmer@dabbelt.com, alex@ghiti.fr, glider@google.com, oliver.upton@linux.dev, andre.przywara@arm.com, x86@kernel.org, joey.gouly@arm.com, tglx@linutronix.de, kvm-riscv@lists.infradead.org, atishp@atishpatra.org, mingo@redhat.com, jingzhangos@google.com, hpa@zytor.com, dave.hansen@linux.intel.com, kvmarm@lists.linux.dev, will@kernel.org, keisuke.nishimura@inria.fr, sebott@redhat.com, lishusen2@huawei.com, pbonzini@redhat.com, rdunlap@infradead.org, seanjc@google.com, yuzenghui@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-BeenThere: kvm-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "kvm-riscv" Errors-To: kvm-riscv-bounces+kvm-riscv=archiver.kernel.org@lists.infradead.org On Thu, 01 May 2025 14:41:26 +0100, Peter Zijlstra wrote: > > On Thu, May 01, 2025 at 01:44:28PM +0100, Marc Zyngier wrote: > > On Thu, 01 May 2025 12:15:52 +0100, > > Peter Zijlstra wrote: > > > > > > > > + */ > > > > > +int kvm_trylock_all_vcpus(struct kvm *kvm) > > > > > +{ > > > > > + struct kvm_vcpu *vcpu; > > > > > + unsigned long i, j; > > > > > + > > > > > + kvm_for_each_vcpu(i, vcpu, kvm) > > > > > + if (!mutex_trylock_nest_lock(&vcpu->mutex, &kvm->lock)) > > > > > > This one includes an assertion that kvm->lock is actually held. > > > > Ah, cunning. Thanks. > > > > > That said, I'm not at all sure what the purpose of all this trylock > > > stuff is here. > > > > > > Can someone explain? Last time I asked someone said something about > > > multiple VMs, but I don't know enough about kvm to know what that means. > > > > Multiple VMs? That'd be real fun. Not. > > > > > Are those vcpu->mutex another class for other VMs? Or what gives? > > > > Nah. This is firmly single VM. > > > > The purpose of this contraption is that there are some rare cases > > where we need to make sure that if we update some global state, all > > the vcpus of a VM need to see, or none of them. > > > > For these cases, the guarantee comes from luserspace, and it gives the > > pinky promise that none of the vcpus are running at that point. But > > being of a suspicious nature, we assert that this is true by trying to > > take all the vcpu mutexes in one go. This will fail if a vcpu is > > running, as KVM itself takes the vcpu mutex before doing anything. > > > > Similar requirement exists if we need to synthesise some state for > > userspace from all the individual vcpu states. > > Ah, okay. Because x86 is simply doing mutex_lock() instead of > mutex_trylock() -- which would end up waiting for this activity to > subside I suppose. > > Hence the use of the killable variant I suppose, for when they get tired > of waiting. Yeah, I remember some debate around that when this refactoring was first posted. I quickly paged it out. > If all the architectures are basically doing the same thing, it might > make sense to unify this particular behaviour. But what do I know. I don't know either. The trylock behaviour has been there since day-1 on the arm side, and changing it would have userspace visible effects. So I'm pretty keen on preserving it, warts and all. The last thing I need is a VMM person hitting my inbox on the grounds that their toy is broken. On the other hand, we're talking about virtualisation, so everything is more or less broken by design... M. -- Without deviation from the norm, progress is not possible. -- kvm-riscv mailing list kvm-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kvm-riscv