From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F4F1241114; Mon, 10 Feb 2025 15:57:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739203058; cv=none; b=F8ypmJK+bQQaKfvyxmSW6TJPt063YygA97XGb95TJ3fz1qWu8xCsU/9D/nPO7L/Cpd691qF+HCZ4MmP4X1NccV2nd/c9Y0crUS/RUzik4BWhQTkBv9+ELhMigoTUzf8Wpp7dEuqWrZgqXwHpRQ3+R/YPER8Clx7HL4omg7JNz4I= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739203058; c=relaxed/simple; bh=IUVaMNro/n/NUpJq8hNviZGcAVuVV7Lxbz8xW3JZfE8=; h=Date:Message-ID:From:To:Cc:Subject:In-Reply-To:References: MIME-Version:Content-Type; b=hy++7A3u+LorXPeSEonhcNn5PJiN/a1ggYx4kj/HR5Sgs9ouJEHw4xrWdmaPxgK+wXebjhLnWTanWHeKeArHYqOX9IvqyH/K1aXmCibA5UDbavcC5BQxb5dNuukrS0xlulqgo54nLB8RQC5lR1ixqC0UcP33YzbPhlmh/bWkWFE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Qvf2KvAN; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Qvf2KvAN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 92A22C4CED1; Mon, 10 Feb 2025 15:57:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1739203057; bh=IUVaMNro/n/NUpJq8hNviZGcAVuVV7Lxbz8xW3JZfE8=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=Qvf2KvANahCKQixPfrU2EYKluYYQbSmFYrMVshayoWeC0i3RWiHnakhF/F1VlDVQB zFsXk7cFnBXI3qaW9YFYhD1JAHfeRhmEyZBAPQrDabl3/IeS/V3KlCE6G9PpzE8M+n Tfw+bWZ2zr7xb0Y0a5zXTZp/zeF4V5ZXC4BkEtrA82nPKQnwvzI2vwtcq7WKmLxN06 ER9mWKm7K6NLKTYwxJraqcvwhL2PWbVxLjtc0iQ7qtQHoAt+T0PmkVEupDJXv1+ITz Hkb9wF8nJD+WKv83faLjTS2Q2GKjEcKP52utBdHqNLiXYimaFOhbRl3E9PEydlebMd 0nY1vqKJC2WAA== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1thWAB-002cIl-9U; Mon, 10 Feb 2025 15:57:35 +0000 Date: Mon, 10 Feb 2025 15:57:34 +0000 Message-ID: <864j11u70x.wl-maz@kernel.org> From: Marc Zyngier To: Maxim Levitsky Cc: kvmarm@lists.linux.dev, kvm@vger.kernel.org Subject: Re: Question about lock_all_vcpus In-Reply-To: References: User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.4 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: mlevitsk@redhat.com, kvmarm@lists.linux.dev, kvm@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false On Thu, 06 Feb 2025 20:08:10 +0000, Maxim Levitsky wrote: > > Hi! > > KVM on ARM has this function, and it seems to be only used in a couple of places, mostly for > initialization. > > We recently noticed a CI failure roughly like that: Did you only recently noticed because you only recently started testing with lockdep? As far as I remember this has been there forever. > > [ 328.171264] BUG: MAX_LOCK_DEPTH too low! > [ 328.175227] turning off the locking correctness validator. > [ 328.180726] Please attach the output of /proc/lock_stat to the bug report > [ 328.187531] depth: 48 max: 48! > [ 328.190678] 48 locks held by qemu-kvm/11664: > [ 328.194957] #0: ffff800086de5ba0 (&kvm->lock){+.+.}-{3:3}, at: kvm_ioctl_create_device+0x174/0x5b0 > [ 328.204048] #1: ffff0800e78800b8 (&vcpu->mutex){+.+.}-{3:3}, at: lock_all_vcpus+0x16c/0x2a0 > [ 328.212521] #2: ffff07ffeee51e98 (&vcpu->mutex){+.+.}-{3:3}, at: lock_all_vcpus+0x16c/0x2a0 > [ 328.220991] #3: ffff0800dc7d80b8 (&vcpu->mutex){+.+.}-{3:3}, at: lock_all_vcpus+0x16c/0x2a0 > [ 328.229463] #4: ffff07ffe0c980b8 (&vcpu->mutex){+.+.}-{3:3}, at: lock_all_vcpus+0x16c/0x2a0 > [ 328.237934] #5: ffff0800a3883c78 (&vcpu->mutex){+.+.}-{3:3}, at: lock_all_vcpus+0x16c/0x2a0 > [ 328.246405] #6: ffff07fffbe480b8 (&vcpu->mutex){+.+.}-{3:3}, at: lock_all_vcpus+0x16c/0x2a0 > > > .. > .. > .. > .. > > > As far as I see currently MAX_LOCK_DEPTH is 48 and the number of > vCPUs can easily be hundreds. 512 exactly. Both of which are pretty arbitrary limits. > > Do you think that it's possible? or know if there were any efforts > to get rid of lock_all_vcpus to avoid this problem? If not possible, > maybe we can exclude the lock_all_vcpus from the lockdep validator? I'd be very wary of excluding any form of locking from being checked by lockdep, and I'd rather we bump MAX_LOCK_DEPTH up if KVM is enabled on arm64. it's not like anyone is going to run that in production anyway. task_struct may not be happy about that though. The alternative is a full stop_machine(), and I don't think that will fly either. > AFAIK, on x86 most of the similar cases where lock_all_vcpus could > be used are handled by assuming and enforcing that userspace will > call these functions prior to first vCPU is created an/or run, thus > the need for such locking doesn't exist. This assertion doesn't hold on arm64, as this ordering requirement doesn't exist. We already have a bunch of established VMMs doing things in random orders (QEMU being the #1 offender), and the sad reality of the Linux ABI means this needs to be supported forever. Thanks, M. -- Without deviation from the norm, progress is not possible.