From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 86B851B6CF2 for ; Thu, 17 Oct 2024 07:58:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729151913; cv=none; b=vGHBPaXF6yB5fkpxaufMSyZs0XXMpdmt0mbWDClbZbcqwfk9+aCB8jj0RfjaULYlbqY4IkV0JG3KarNsC8Lqqu/QTs2bzRjhboKgjJkm4bGHFsEekLheEICgT0hQK9f55eYXvlKcL0vXhfWAkmy++1IE156grrmGxTjfo25NC+4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729151913; c=relaxed/simple; bh=zdF+B+r4Pu8Z2Tj6FzkV+Yd/5SsrHQvwfNXuYPWCjrc=; h=Date:Message-ID:From:To:Cc:Subject:In-Reply-To:References: MIME-Version:Content-Type; b=PdrldevUmOgBVWC5cG0KDHtDYKkZXd3iRKb3X/IZMTlAd3cufeZGOefqQ1IhshGXX+lFpxU/XzJG5jO9zxksbsDFh3+R4xtO246W4e5GNDoUlKp8Lgo8EGHs9zQ9qnft1pvmdtS23Kopp+puWPRgzRok/hfOCSQCSzy7o1tasPg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZcYIfyGW; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZcYIfyGW" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0ABE4C4CEC3; Thu, 17 Oct 2024 07:58:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729151913; bh=zdF+B+r4Pu8Z2Tj6FzkV+Yd/5SsrHQvwfNXuYPWCjrc=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=ZcYIfyGWbBAgpW1FRbfPtCGCN+apRgxxAr3/7NudeqFy/FY4qV/I1fytVguzfGK0j f2mrDVHew7pJfgNw9iRxfysrP/VB16QG7Jgw7f1O7OwPMDgZTj5XHprM3ai3fY4pli usuMPPhowD1LyWVX8uwQDddp+QXwg6ed1bapR4XzjRVeuCXt+seBFKv4usJdGrfUyX 8NEQPZzxomroMkgQoFcZrlMDqKYu9TR+iRkgt36NC4KFIzSgpxalpzIOlDN8G3V9em ElLHV0N968ICPocwAo40sj7GMNIoR+1AT5Z/RSHWVnZj0aVAVnQfe/AMFxMJPn/lyY gi/KO9wOTvVew== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1t1LOw-004KlD-Rw; Thu, 17 Oct 2024 08:58:30 +0100 Date: Thu, 17 Oct 2024 08:58:30 +0100 Message-ID: <86ttdb40ux.wl-maz@kernel.org> From: Marc Zyngier To: Oliver Upton Cc: kvmarm@lists.linux.dev, Joey Gouly , Suzuki K Poulose , Zenghui Yu Subject: Re: [PATCH 2/2] KVM: arm64: Ensure vgic_ready() is ordered against MMIO registration In-Reply-To: <20241017001947.2707312-3-oliver.upton@linux.dev> References: <20241017001947.2707312-1-oliver.upton@linux.dev> <20241017001947.2707312-3-oliver.upton@linux.dev> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.4 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: oliver.upton@linux.dev, kvmarm@lists.linux.dev, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false On Thu, 17 Oct 2024 01:19:47 +0100, Oliver Upton wrote: > > kvm_vgic_map_resources() prematurely marks the distributor as 'ready', > potentially allowing vCPUs to enter the guest before the distributor's > MMIO registration has been made visible. > > Plug the race by marking the distributor as ready only after MMIO > registration is completed. Rely on the implied ordering of > synchronize_srcu() to ensure the MMIO registration is visible before > vgic_dist::ready. This also means that writers to vgic_dist::ready are > now serialized by the slots_lock, which was effectively the case already > as all writers held the slots_lock in addition to the config_lock. > > Fixes: 59112e9c390b ("KVM: arm64: vgic: Fix a circular locking issue") > Signed-off-by: Oliver Upton > --- > arch/arm64/kvm/vgic/vgic-init.c | 13 +++++++++++-- > 1 file changed, 11 insertions(+), 2 deletions(-) > > diff --git a/arch/arm64/kvm/vgic/vgic-init.c b/arch/arm64/kvm/vgic/vgic-init.c > index d88fdeaf6144..48c952563e85 100644 > --- a/arch/arm64/kvm/vgic/vgic-init.c > +++ b/arch/arm64/kvm/vgic/vgic-init.c > @@ -544,14 +544,23 @@ int kvm_vgic_map_resources(struct kvm *kvm) > if (ret) > goto out; > > - dist->ready = true; > dist_base = dist->vgic_dist_base; > mutex_unlock(&kvm->arch.config_lock); > > ret = vgic_register_dist_iodev(kvm, dist_base, type); > - if (ret) > + if (ret) { > kvm_err("Unable to register VGIC dist MMIO regions\n"); > + goto out_slots; > + } > > + /* > + * kvm_io_bus_register_dev() guarantees all readers see the new MMIO > + * registration before returning through synchronize_srcu(), which also > + * implies a full memory barrier. As such, marking the distributor as > + * 'ready' here is guaranteed to be ordered after all vCPUs having seen > + * a completely configured distributor. > + */ > + dist->ready = true; > goto out_slots; > out: > mutex_unlock(&kvm->arch.config_lock); I thought we would benefit from the removal of the early-out at the beginning of the function, specially given that this is only called on the first run of any vcpu -- and we can probably afford to take the locks on such a rare event. My impression is that we'd gain in robustness and maintainability. This can come as a later patch if we agree that this is valuable. Thanks, M. -- Without deviation from the norm, progress is not possible.