From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64DF2C53210 for ; Tue, 3 Jan 2023 17:42:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238495AbjACRmm (ORCPT ); Tue, 3 Jan 2023 12:42:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42688 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238325AbjACRmh (ORCPT ); Tue, 3 Jan 2023 12:42:37 -0500 Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 42B5A2A5 for ; Tue, 3 Jan 2023 09:42:36 -0800 (PST) Received: by mail-pl1-x631.google.com with SMTP id c6so148400pls.4 for ; Tue, 03 Jan 2023 09:42:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=OkcZf3Nc7+BU1+DO0enKeeF9MyS0vVC/gIcg/2SqswM=; b=gS6a/Etgsc3Y7wqJ1duGRSBfgWWVzFB4u1JMrxXTtmJzxj3QvXhb39dbInsEk3YqZX FoWN1FUPGmngIT6N6nqeyXb+BJIHU0LO87D2ByM2QmyUtps6LxE7DHpdveXReFP4vzAw 99HYN3J4tHMdgezOwSWNy7CyYU1D67c15zs7FaECgSU0CCqzDam22fzrxEV15mO7cqcp f5TbCO1iUh3Vl8usPlVbkfN9c7sCvDFIbXzwd8CH6R3PPJRGKK2woFXWIBZVn9bTIIDE yRji+6s9YSRhvi1peu9DzDqEnOxKqAE8qlTvvYpAl5WHesv3r4y0RARgwkoPR6qq3GDH 0zew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=OkcZf3Nc7+BU1+DO0enKeeF9MyS0vVC/gIcg/2SqswM=; b=Z2ydgBNo/4hGVVxRDUDiIdoo2ahHgeA6zZKmY2HLMfFGSssK2z4Fi22cxyYltlPYLq uUvTt6H10LcFPmFtNgIy4LRcuqtiU+hYKCWnEtVRvOai98WtQfOmoEQufbnyqnD3+htq QigsIChl/cHugfOnKhKmPPx3/pUGKAs2djuo/eOjWARJ+TEGfye1hOj8Nk2VkMTL8sNy hhq7A3MP+STxzJeKlfS5Lu25S18OE01RkDqHBQWIF1ILUEAYA0DMRarkygkjX9XAYefu yIOnO/ynPqmutpFKnFoyKLHLAUlBF7cPH1aLrEigTLW9Ai0D5OniVncJMKTL4sWNrFtR 1kXg== X-Gm-Message-State: AFqh2kq0vXMMe3BpHF3cGht3iuK+adcworK8P3NrLrkdZPjSeIrFbqSR vFNzqRXslbQxC39oE+/X+AyVLQ== X-Google-Smtp-Source: AMrXdXtPqHrkrzSlIRciJoetvHMCPwMdBuKzCM1WyhWc9yv71Rd92zOcdIG8AXUPiO47gGatcXvf6g== X-Received: by 2002:a17:902:9049:b0:189:6d32:afeb with SMTP id w9-20020a170902904900b001896d32afebmr4220394plz.1.1672767755628; Tue, 03 Jan 2023 09:42:35 -0800 (PST) Received: from google.com (7.104.168.34.bc.googleusercontent.com. [34.168.104.7]) by smtp.gmail.com with ESMTPSA id u18-20020a170903125200b00189adf6770fsm22749792plh.233.2023.01.03.09.42.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Jan 2023 09:42:35 -0800 (PST) Date: Tue, 3 Jan 2023 17:42:31 +0000 From: Sean Christopherson To: Paolo Bonzini Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Subject: Re: [PATCH] Documentation: kvm: clarify SRCU locking order Message-ID: References: <20221228110410.1682852-2-pbonzini@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20221228110410.1682852-2-pbonzini@redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Dec 28, 2022, Paolo Bonzini wrote: > Currently only the locking order of SRCU vs kvm->slots_arch_lock > and kvm->slots_lock is documented. Extend this to kvm->lock > since Xen emulation got it terribly wrong. > > Signed-off-by: Paolo Bonzini > --- > Documentation/virt/kvm/locking.rst | 19 ++++++++++++++----- > 1 file changed, 14 insertions(+), 5 deletions(-) > > diff --git a/Documentation/virt/kvm/locking.rst b/Documentation/virt/kvm/locking.rst > index 845a561629f1..a3ca76f9be75 100644 > --- a/Documentation/virt/kvm/locking.rst > +++ b/Documentation/virt/kvm/locking.rst > @@ -16,17 +16,26 @@ The acquisition orders for mutexes are as follows: > - kvm->slots_lock is taken outside kvm->irq_lock, though acquiring > them together is quite rare. > > -- Unlike kvm->slots_lock, kvm->slots_arch_lock is released before > - synchronize_srcu(&kvm->srcu). Therefore kvm->slots_arch_lock > - can be taken inside a kvm->srcu read-side critical section, > - while kvm->slots_lock cannot. > - > - kvm->mn_active_invalidate_count ensures that pairs of > invalidate_range_start() and invalidate_range_end() callbacks > use the same memslots array. kvm->slots_lock and kvm->slots_arch_lock > are taken on the waiting side in install_new_memslots, so MMU notifiers > must not take either kvm->slots_lock or kvm->slots_arch_lock. > > +For SRCU: > + > +- ``synchronize_srcu(&kvm->srcu)`` is called _inside_ > + the kvm->slots_lock critical section, therefore kvm->slots_lock > + cannot be taken inside a kvm->srcu read-side critical section. > + Instead, kvm->slots_arch_lock is released before the call > + to ``synchronize_srcu()`` and _can_ be taken inside a > + kvm->srcu read-side critical section. > + > +- kvm->lock is taken inside kvm->srcu, therefore Prior to the recent Xen change, is this actually true? There are many instances where kvm->srcu is taken inside kvm->lock, but I can't find any existing cases where the reverse is true. Logically, it makes sense to take kvm->lock first since kvm->srcu can be taken deep in helpers, e.g. for accessing guest memory. It's also more consistent to take kvm->lock first since kvm->srcu is taken inside vcpu->mutex, and vcpu->mutex is taken inside kvm->lock. Disallowing synchronize_srcu(kvm->srcu) inside kvm->lock isn't probelmatic per se, but it's going to result in a weird set of rules because synchronize_scru() can, and is, called while holding a variety of other locks. In other words, IMO taking kvm->srcu outside of kvm->lock in the Xen code is the real bug. > + ``synchronize_srcu(&kvm->srcu)`` cannot be called inside > + a kvm->lock critical section. If you cannot delay the > + call until after kvm->lock is released, use ``call_srcu``. > + > On x86: > > - vcpu->mutex is taken outside kvm->arch.hyperv.hv_lock > -- > 2.31.1 >