From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qv1-f54.google.com (mail-qv1-f54.google.com [209.85.219.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE21C374E6C for ; Tue, 21 Jul 2026 17:55:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.54 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784656535; cv=none; b=UTHr5j3H5yvKUYaIGluxV1ezsqoDG0D1SkS8fBepjBop9yruzivlOZov0HeBZuwWQKKggFadaq9Lnl7L0ldo23Q9vzMK7Iir2AOZatckB1nzD/fg+f5UAFnFkfY3Y5Nur4oR1Cb8dOMduHwTQiE6toW0i5TN/LBua8+Hct0v7E8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784656535; c=relaxed/simple; bh=fzgF3V5HF54yEqsnjagon7PnrwUxx4c7CuxO/Uie8BY=; h=MIME-Version:Content-Type:Subject:From:To:Cc:In-Reply-To: References:Date:Message-Id; b=m8W2n48GOB6RumEVvC41mfqeATt9BKm6XalU7YU8K0b1sntkCsQG3sDG2ktgXEpl0cxdGSIZIiN1XLuAO7rHZYEL+qtrG0alTAjLwcyA0ZI959FIbYLn12+RnBVmfXDGQg9qfQemtTgT+sBTn4hbzeEuqlOJWxQvE4vy70jaV3Y= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=soleen.com; spf=pass smtp.mailfrom=soleen.com; dkim=pass (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b=Fv/GQVrQ; arc=none smtp.client-ip=209.85.219.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=soleen.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=soleen.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="Fv/GQVrQ" Received: by mail-qv1-f54.google.com with SMTP id 6a1803df08f44-8eeadbc5e21so54488766d6.3 for ; Tue, 21 Jul 2026 10:55:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; t=1784656533; x=1785261333; darn=vger.kernel.org; h=message-id:date:references:in-reply-to:cc:to:from:subject :content-transfer-encoding:content-type:mime-version:from:to:cc :subject:date:message-id:reply-to:content-type; bh=ea/F6w9Zt+Er+ldXlrQTi7dDqHaImBI68nyXrbPqNKM=; b=Fv/GQVrQ7RL3cqdVi7mEB+YpDgdLpQDSSGffsD5OMcz/fXT1L5NpVPdZ0ccTm+Mm/0 2KGNP5Ux7sttdWB02OjSvltxROI95icearWQaw4aSEaRT7sTXR6da7L2sFqbuIrLVdCg RLB2Bs3ICcN8EM/ek+L/9NljM5RhWQ2dLnFe5sqhgufZryNqzQBBuhv5WzNfzTjsi2Vk 5nADXoYHEKmMo960DaU9sI3IxCU6i0+CYHZmaKOhC4sP3shwDUcfOCUKsssZ2ZIX6lM6 /tOF/1P1FMa6TfaEp2ohrVz0KY8AntDHsodGs9y7BTtiKbQPaOJ8YFLZHmYJDE6ovJt9 xL0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784656533; x=1785261333; h=message-id:date:references:in-reply-to:cc:to:from:subject :content-transfer-encoding:content-type:mime-version:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=ea/F6w9Zt+Er+ldXlrQTi7dDqHaImBI68nyXrbPqNKM=; b=Ta6kA65ojHMcrpkETqmG04pt+HaPX6Fr8lNw91qRA0H3Lm49bZdXF8vcO+xc0XNOxA WJwJkqVZIok50SOgt5muOTLjee7RalMBJ/Jruri9uP/rmAEa7QlRIBwDbmq8DuM593nF sRZMfnEJ5WFJbtbvOWT+NcOBFOEW2UCWbDJRJ2RfTuhqntcduuy+ddTaozA2W3Ory11t PxgD0ngLWwQNP6ewbB+qWu3vzNuRi/+w4df+E4FBzc3OhBoOnafieDoOudbo3huj+X1Q uHXhlbNSa34+kL5lF6JuPUvzHhY01ZX3sbO3wJhHuLpXpUInPDIg383dT2BN6ktjEc12 yxfQ== X-Forwarded-Encrypted: i=1; AHgh+RoCd+IK1Yu+ucCuQuN1XgNhu4ecQVMa3tziJr+9mhVYhZ14psFC4KslmsyxcgMo1fy+p15aAoTFSbo=@vger.kernel.org X-Gm-Message-State: AOJu0YwyxkC4ys+4j41b90Sew39paFz+D1k7WSIf6G0y+aKALpcZ4zpn rv53W0/bLzdVsy/HGjH1dHa8w89EVozfdM6D4rVQwaZ54Sk9Hn1gI6PljNVbAdPzQaw= X-Gm-Gg: AR+sD10mJQ+s2niPilH8xL/rWzEkIn3G+eKRN/9/2AVAeBra23Q6Vb+Bz3BVDbWlCnv j5bzPkfw8mMc/pIQb48+amJhubvqR2YHa36S2FCQtFNH3uUTyo/UL3i9mhbi2//mZltDlSOxyKZ CTDryiUi18wXgVSQRVlxdcQb1KB34uUoyEwJcsXC8w5KZGBnFJ6gfe7YTG5/sfs44SCHOeBgvCU KvZxHTksAwTc/qz0XqWlT7wY/5fx27pjadYLpVOKYNG+SJUpQ0kzxXJd9KOLuIt5DrI16dBk9xY zeXNGN9e3rKUMm1Eb9huLQzpPFghS97W34vZN/g8ntJlSMeW2bx3bcnzLOGAswYDMb+PtdUsVZq 3o3E63ZT7TAq7G4ar8RNt5ESooGFuPIXckF5+E2UoaRlLx97D1No+6q+4E8QE+WG5HApzXtSEIg h2umZOILsLvVhSHT25yisi5LYvUjgSnwYNzHfuhM0= X-Received: by 2002:a05:6214:2269:b0:8f1:8937:5dc7 with SMTP id 6a1803df08f44-907784ddf91mr239817166d6.56.1784656532475; Tue, 21 Jul 2026 10:55:32 -0700 (PDT) Received: from [127.0.1.1] ([71.181.43.54]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-907ba8df5d0sm1663826d6.18.2026.07.21.10.55.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Jul 2026 10:55:31 -0700 (PDT) Precedence: bulk X-Mailing-List: linux-doc@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit Subject: Re: [PATCH v7 03/12] PCI: liveupdate: Track incoming preserved PCI devices From: Pasha Tatashin To: David Matlack Cc: Pasha Tatashin , kexec@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Adithya Jayachandran , Alexander Graf , Alex Williamson , Bjorn Helgaas , Chris Li , David Rientjes , Jacob Pan , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Leon Romanovsky , Lukas Wunner , Mike Rapoport , Parav Pandit , Pranjal Shrivastava , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , William Tu , Yi Liu In-Reply-To: References: <20260710212616.1351130-1-dmatlack@google.com> <20260710212616.1351130-4-dmatlack@google.com> <178432481253.189683.3297727348836286619.b4-review@b4> <178458744511.332171.13770778781241262180.b4-reply@b4> Date: Tue, 21 Jul 2026 17:55:29 +0000 Message-Id: <178465652971.412167.17838319728990505256.b4-reply@b4> X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3765; i=pasha.tatashin@soleen.com; h=from:subject:message-id; bh=fzgF3V5HF54yEqsnjagon7PnrwUxx4c7CuxO/Uie8BY=; b=owEBbQKS/ZANAwAKAbt3KEzbc3reAcsmYgBqX7KSG2dHPEQ6Mkut+xegrodYi1A9KSpnUO3pn xL+SSaK1qOJAjMEAAEKAB0WIQRBMaqT7LRvGvB/NmK7dyhM23N63gUCal+ykgAKCRC7dyhM23N6 3sbLD/9VJw+QusBVAbX9+6asMemVQiDftMc/hr+I22YGh5yCipWq3pZjfM7mP8bljemHy6oIbj+ eg+gEXv7pWzzQ+8GV3PpuOBYxkjLRkI0gemShPjl4DIrRF1i5QKtbirG+j8HJShkbA0MUFQYF4w etoSvEkZ5OowXog4ZxpFFJrtp949gwoLonQntZS2YEdepjeuQ6O5+29aDjXTwrqt8dr3iq97QY7 PEegM9HW0HVvWmt7PoJ0dx0EEaKZJ0GqMpr9O8W2GdxDoZXl+Sk9BxKiwyOX4J5Jz7rdQAn/o7I Ns4QUk/VtBQL/1irI528G9H+z0gGxw4nCxP8fANN7VQW2Y68d+e7tNZOgJEXmtgQe6lh2ihna3W AJSXMdazYky84+h1aAgVPom/cLGQhgzwHgZSps6nDG9qMSTLZGGAP5VPvHpjCAt1MsLtajskJ9m Pzp1qdozYbecunkANckh7aN32FTvRvmPFrVjU4a/bjB1bsG9O6ocEd2lvY5VLJoI/7EpNjFQENT gzqd9ygy3xIcDV9ifvMhU0XWWUteFUjC96g3UGsi04IcPbR5Oe4Fe0S0UiiqqUxgncTyddsTlmF IyJ5BW5Xc76J0oYVAhS14CIxQNY+9LXWUbBAgmqx4XEDE427j+I0Ic4eKf2seStQEq8rQZ1xQbi o4SPKFvgFZN6Jyg== X-Developer-Key: i=pasha.tatashin@soleen.com; a=openpgp; fpr=CAAAB722DD22A081F0D49F35633A6A993D43B569 On 2026-07-20 16:07:47-07:00, David Matlack wrote: > On Mon, Jul 20, 2026 at 3:44 PM Pasha Tatashin > wrote: > > > On 2026-07-20 14:54:51-07:00, David Matlack wrote: > > > > Thanks for the explanation. As I understand, the most straightforward > > way to avoid holding the permanent reference is indeed to delete > > dev->liveupdate.incoming entirely and perform an xarray lookup on every > > access, like this: > > > > bool pci_liveupdate_is_incoming(struct pci_dev *dev) > > { > > ... > > incoming = pci_liveupdate_flb_get_incoming(); > > ... > > dev_ser = xa_load(&incoming->xa, key); > > ... > > pci_liveupdate_flb_put_incoming(); > > return dev_ser && dev_ser->refcount > 0; > > } > > > > However, as you note, this is inefficient because it affects every > > single device and adds lookup overhead to every access (not sure about > > the actual cost though, xarray access is pretty fast!). > > > > We can, however, still avoid tinkering with the lifecycle of the FLB, > > and instead treat dev->liveupdate.incoming as a "hint" that we validate > > on access with a fast, liveness check: > > Can you tell me more about your concern about FLB lifetime? > > The lifetime of the FLB will not be affected by this reference unless > there is a bug in the driver where it fails to call > pci_liveupdate_finish() during it's file handler finish callback. >From a design perspective, liveupdate_flb_get/put_incoming() is a logical read-lock/unlock pair on the FLB data. We use a refcount for optimization and sharing, but holding a get over a long asynchronous gap (from boot-time device setup to driver probe) is essentially holding an unbound lock. Unbound locks make it difficult to trace refcount leaks or debug lifecycle issues. > > > 1. At Setup: In pci_liveupdate_setup_device(), we do the xarray lookup > > once, cache the pointer in dev->liveupdate.incoming, and immediately > > call pci_liveupdate_flb_put_incoming(). We do not hold a permanent > > reference. > > > > 2. On Access: When an accessor runs, instead of doing a full xarray > > lookup, it just validates the cached pointer's liveness by temporarily > > securing the FLB: > > > > static struct pci_flb_incoming *pci_liveupdate_get_incoming(struct pci_dev *dev) > > { > > struct pci_flb_incoming *incoming; > > > > incoming = pci_liveupdate_flb_get_incoming(); > > if (!incoming) > > return NULL; > > > > if (dev->liveupdate.incoming) > > return incoming; > > > > pci_liveupdate_flb_put_incoming(); > > return NULL; > > } > > > > * If get_incoming() returns NULL (the FLB has already finished/freed), > > the hint is invalid and the device is no longer incoming. > > This avoids the xarray lookup but still requires taking the incoming > FLB mutex twice (once for get and once for put) on every access. And > if there's no incoming PCI FLB, the LUO will iterate over all incoming > FLBs under the mutex to find it. Can we do a fast-path check first? static struct pci_flb_incoming *pci_liveupdate_get_incoming(struct pci_dev *dev) { struct pci_flb_incoming *incoming; /* Fast-path to avoid unnecessary FLB querying */ if (!dev->liveupdate.incoming) return NULL; incoming = pci_liveupdate_flb_get_incoming(); if (!incoming) return NULL; /* Check again, now that FLB is acquired */ if (dev->liveupdate.incoming) return incoming; pci_liveupdate_flb_put_incoming(); return NULL; } This seems to gives us the best of both worlds: robust refcount hygiene and a sane fast path. What do you think?