From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F1B2D77620; Tue, 20 Feb 2024 17:53:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708451639; cv=none; b=r5sy+ZdWlO75HHxC+eUFRF1M9CvfSRMNJ6Oe/i38FMJMyQrJbXq1rShzyt91uY7rEN0sEoNIZf02v3A+T5B2qBHLr0lWMRvOkSWy7NthRZL1TzFs5H5Q1wnkqhM3m3PBSxvHIdoLPm0F7q5r/jJhSIHujrTIfrvqP0d+LoLWHsk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708451639; c=relaxed/simple; bh=9umemLLwhLGOtim3bsE/QpwCqY3bDrJnlA5yekeDHs8=; h=Date:Message-ID:From:To:Cc:Subject:In-Reply-To:References: MIME-Version:Content-Type; b=P1EvA0z3hlHMGaY0et7jPYdr/X4S9tbQcHRdZG9tNl991/t0cLjG+ziQFi8jojsQNM5doQhGNc+aui+LCEAlBAgpLERfsyJp3DBZ48yHmzrBTceSlriPZt076l/XUj8vWPLyHuSjO0sf/wC74uO3EE8Fle7aJDWcmSF+JOSXZqA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=rt52ISWN; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="rt52ISWN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 56519C433F1; Tue, 20 Feb 2024 17:53:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708451638; bh=9umemLLwhLGOtim3bsE/QpwCqY3bDrJnlA5yekeDHs8=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=rt52ISWNQk5smU4KzNB0UsguAtYdYSLkvpZ4RXjEHPfCtphigvdAMxTzxwetbPgQN +aqUsQfDZ7tt0KxwgnYFrs2zknrOVzccoshw3jQcyl4kZ1xhLEGvIsMr6sAw1NwUQf g0b8eeq45hsJO6rlUzwk+5hYVZk6lOKRKvAI63VjHOHvJ4CKYeJ5kKSa9iOJb6pSr7 el8YtSc6S4du0A7KWqwMKoJa8B42gmkKdJT8t7zixFdAdvvzmOAU2eCLnaDCQnTIK0 BNoYAyS7LTlg4a9G7eR3iXBPEjA58ONsaQ+QIE0xYp0IjoUZMhn8cobmTuQ3sO6crN 415DZmWL+5yVg== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1rcUJY-0050Ci-5s; Tue, 20 Feb 2024 17:53:56 +0000 Date: Tue, 20 Feb 2024 17:53:55 +0000 Message-ID: <86v86j2f9o.wl-maz@kernel.org> From: Marc Zyngier To: Oliver Upton Cc: Zenghui Yu , kvmarm@lists.linux.dev, kvm@vger.kernel.org, James Morse , Suzuki K Poulose , Zenghui Yu , linux-kernel@vger.kernel.org Subject: Re: [PATCH v3 01/10] KVM: arm64: vgic: Store LPIs in an xarray In-Reply-To: References: <20240216184153.2714504-1-oliver.upton@linux.dev> <20240216184153.2714504-2-oliver.upton@linux.dev> <86wmqz2gm5.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: oliver.upton@linux.dev, zenghui.yu@linux.dev, kvmarm@lists.linux.dev, kvm@vger.kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, linux-kernel@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false On Tue, 20 Feb 2024 17:43:03 +0000, Oliver Upton wrote: > > On Tue, Feb 20, 2024 at 05:24:50PM +0000, Marc Zyngier wrote: > > On Tue, 20 Feb 2024 16:30:24 +0000, > > Zenghui Yu wrote: > > > > > > On 2024/2/17 02:41, Oliver Upton wrote: > > > > Using a linked-list for LPIs is less than ideal as it of course requires > > > > iterative searches to find a particular entry. An xarray is a better > > > > data structure for this use case, as it provides faster searches and can > > > > still handle a potentially sparse range of INTID allocations. > > > > > > > > Start by storing LPIs in an xarray, punting usage of the xarray to a > > > > subsequent change. > > > > > > > > Signed-off-by: Oliver Upton > > > > > > [..] > > > > > > > diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c > > > > index db2a95762b1b..c126014f8395 100644 > > > > --- a/arch/arm64/kvm/vgic/vgic.c > > > > +++ b/arch/arm64/kvm/vgic/vgic.c > > > > @@ -131,6 +131,7 @@ void __vgic_put_lpi_locked(struct kvm *kvm, struct vgic_irq *irq) > > > > return; > > > > list_del(&irq->lpi_list); > > > > + xa_erase(&dist->lpi_xa, irq->intid); > > > > > > We can get here *after* grabbing the vgic_cpu->ap_list_lock (e.g., > > > vgic_flush_pending_lpis()/vgic_put_irq()). And as according to vGIC's > > > "Locking order", we should disable interrupts before taking the xa_lock > > > in xa_erase() and we would otherwise see bad things like deadlock.. > > > > > > It's not a problem before patch #10, where we drop the lpi_list_lock and > > > start taking the xa_lock with interrupts enabled. Consider switching to > > > use xa_erase_irq() instead? > > > > But does it actually work? xa_erase_irq() uses spin_lock_irq(), > > followed by spin_unlock_irq(). So if we were already in interrupt > > context, we would end-up reenabling interrupts. At least, this should > > be the irqsave version. > > This is what I was planning to do, although I may kick it out to patch > 10 to avoid churn. > > > The question is whether we manipulate LPIs (in the get/put sense) on > > the back of an interrupt handler (like we do for the timer). It isn't > > obvious to me that it is the case, but I haven't spent much time > > staring at this code recently. > > I think we can get into here both from contexts w/ interrupts disabled > or enabled. irqfd_wakeup() expects to be called w/ interrupts disabled. > > All the more reason to use irqsave() / irqrestore() flavors of all of > this, and a reminder to go check all callsites that implicitly take the > xa_lock. Sounds good. Maybe you can also update the locking order "documentation" to include the xa_lock? I expect that it will ultimately replace lpi_list_lock. Thanks, M. -- Without deviation from the norm, progress is not possible.