From: Srirangan Madhavan <smadhavan@nvidia.com>
To: Alison Schofield <alison.schofield@intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>,
Dave Jiang <dave.jiang@intel.com>,
Davidlohr Bueso <dave@stgolabs.net>,
Ira Weiny <iweiny@kernel.org>,
Jonathan Cameron <jic23@kernel.org>,
Vishal Verma <vishal.l.verma@intel.com>,
linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org,
linux-kernel@vger.kernel.org, Li Ming <ming.li@zohomail.com>,
Joshua Machauer <machaujo@amazon.de>,
Alex Williamson <alex.williamson@redhat.com>,
vsethi@nvidia.com, alwilliamson@nvidia.com,
Sai Yashwanth Reddy Kancherla <skancherla@nvidia.com>,
Vishal Aslot <vaslot@nvidia.com>,
Manish Honap <mhonap@nvidia.com>, Jiandi An <jan@nvidia.com>,
Richard Cheng <icheng@nvidia.com>,
linux-tegra@vger.kernel.org
Subject: Re: [PATCH v17 13/19] cxl: Validate and synchronize HDM ranges around reset
Date: Fri, 20 Nov 2026 00:20:34 -0800 [thread overview]
Message-ID: <e5eaa99b-552e-4f6d-9c3f-98f7b309f46d@nvidia.com> (raw)
In-Reply-To: <avyPsvwV1_lc9va5@aschofie-mobl2.lan>
On 11/17/26 9:32 AM, Alison Schofield wrote:
>> Refuse reset unless enabled system-physical HDM ranges can be reserved
>> exclusively and CPU-cache invalidation is available. Invalidate before
>> reset and again before ending IOMMU exclusion, holding range reservations
>> until the second invalidation completes. A later patch places state
>> restoration before the second invalidation.
>>
>> Userspace is expected to offline CXL.mem users before reset. The
>> reservation enforces this for claimed ranges: the reset fails while a
>> conflicting resource is claimed, and no one can claim the range during
>> the reset. It cannot see accesses by code that uses the range without
>> claiming it, which the offline expectation also covers.
>>
>> Reject normalized-addressing decoders because their cached ranges are not
>> system physical addresses. Record Normalized Addressing in the cache when
>> region discovery sets it, since the cache is filled earlier. A reset that
>> takes its snapshot between endpoint probe and that update can still
>> miss it.
> What prevents reset during this window? If the snapshot misses the
> normalized-addressing flag can a reset proceed and use a range that is
> not a system physical address?
>
> Does this mean we need to prevent reset until discovery has resolved
> the address interpretation, or is there another check that makes
> reset safe before then?
>
On this first part: nothing prevents it in v17. For v18 I'd reject a range
that isn't inside a platform CXL window, on top of the existing flag
check.
>> Ignore zero-size decoders because they map no address range. Refuse reset
>> when the device decodes memory through its DVSEC ranges instead of HDM
>> decoders, since the cache holds no range to reserve for that memory.
>>
>> Hold the global cxl_rwsem.region write lock across snapshot, reset and
>> restoration to keep the decoder state and reserved ranges stable. This also
>> blocks region and decoder sysfs, region probe, poison operations, sanitize
>> and region-aware event tracing across CXL devices. The Memory Clear wait
>> lasts up to the timeout the device advertises, at most 100 seconds; cache
>> flushes and other reset work extend the hold.
> Similar to questions on previous versions and in cover letter -
>
> Can a reset acquire this lock while other CXL devices are still being
> discovered and their regions assembled, and block that work for the
> full reset duration?
>
> What limits the disruption to unrelated devices, including repeated
> resets? Could the locking be scoped to the affected device or regions
> rather than holding the global region lock throughout the wait?
On this part, yes. As posted the lock is held across the whole reset.
For v18 I'm proposing to hold it only briefly to snapshot and restore, and
refuse commits of that device's decoders while it resets.
Details are in my reply to the cover letter:
https://lore.kernel.org/linux-cxl/20261105185917.2667656-1-smadhavan@nvidia.com/T/#m82b76ee0215d67d91f67ec6fa72036615f1adb4f
--
Regards,
Srirangan
next prev parent reply other threads:[~2026-11-20 8:20 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-05 18:58 [PATCH v17 00/19] PCI/CXL: Add CXL reset support for Type 2 devices Srirangan Madhavan
2026-11-05 18:58 ` [PATCH v17 01/19] cxl: Clarify decoder interleave limit Srirangan Madhavan
2026-11-05 18:59 ` [PATCH v17 02/19] cxl: Share CXL port upstream PCI device lookup Srirangan Madhavan
2026-11-10 13:36 ` Li Ming
2026-11-13 15:50 ` Dave Jiang
2026-11-15 3:25 ` Li Ming
2026-11-05 18:59 ` [PATCH v17 03/19] cxl: Move decoder declarations to shared header Srirangan Madhavan
2026-11-05 18:59 ` [PATCH v17 04/19] cxl: Embed decoder configuration in a standalone structure Srirangan Madhavan
2026-11-05 18:59 ` [PATCH v17 05/19] cxl: Introduce endpoint HDM decoder settings Srirangan Madhavan
2026-11-05 18:59 ` [PATCH v17 06/19] cxl: Move HDM decoder helpers to built-in code Srirangan Madhavan
2026-11-05 18:59 ` [PATCH v17 07/19] cxl: Share HDM decoder register unpacking Srirangan Madhavan
2026-11-12 21:35 ` Dave Jiang
2026-11-05 18:59 ` [PATCH v17 08/19] cxl: Move cxl_rwsem into built-in code Srirangan Madhavan
2026-11-12 23:49 ` Dave Jiang
2026-11-05 18:59 ` [PATCH v17 09/19] cxl: Refresh cached PCI HDM decoder settings Srirangan Madhavan
2026-11-13 0:38 ` Dave Jiang
2026-11-05 18:59 ` [PATCH v17 10/19] cxl: Build register block helpers into the kernel Srirangan Madhavan
2026-11-13 18:02 ` Dave Jiang
2026-11-05 18:59 ` [PATCH v17 11/19] PCI/CXL: Cache endpoint HDM state during PCI enumeration Srirangan Madhavan
2026-11-10 14:12 ` Li Ming
2026-11-11 3:54 ` Li Ming
2026-11-17 17:29 ` Alison Schofield
2026-11-20 8:17 ` Srirangan Madhavan
2026-11-05 18:59 ` [PATCH v17 12/19] PCI/CXL: Add CXL Device Reset sequencing Srirangan Madhavan
2026-11-10 14:29 ` Li Ming
2026-11-11 10:12 ` Lucero Palau, Alejandro
2026-11-05 18:59 ` [PATCH v17 13/19] cxl: Validate and synchronize HDM ranges around reset Srirangan Madhavan
2026-11-17 17:32 ` Alison Schofield
2026-11-20 8:20 ` Srirangan Madhavan [this message]
2026-11-05 18:59 ` [PATCH v17 14/19] PCI/CXL: Reject reset with unsafe function scope Srirangan Madhavan
2026-11-05 18:59 ` [PATCH v17 15/19] PCI/CXL: Restore CXL state after PCI reset Srirangan Madhavan
2026-11-05 18:59 ` [PATCH v17 16/19] PCI/CXL: Expose CXL Reset as a PCI reset method Srirangan Madhavan
2026-11-17 17:37 ` Alison Schofield
2026-11-20 8:21 ` Srirangan Madhavan
2026-11-05 18:59 ` [PATCH v17 17/19] PCI/CXL: Restore CXL state after CXL bus reset Srirangan Madhavan
2026-11-13 17:37 ` Dave Jiang
2026-11-05 18:59 ` [PATCH v17 18/19] PCI/CXL: Flush host and device caches for " Srirangan Madhavan
2026-11-05 18:59 ` [PATCH v17 19/19] cxl: Add KUnit checks for HDM decoder unpacking Srirangan Madhavan
2026-11-05 19:31 ` [PATCH v17 00/19] PCI/CXL: Add CXL reset support for Type 2 devices Alison Schofield
2026-11-16 19:11 ` Dave Jiang
2026-11-20 7:32 ` Srirangan Madhavan
2026-11-17 17:28 ` Alison Schofield
2026-11-20 8:16 ` Srirangan Madhavan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e5eaa99b-552e-4f6d-9c3f-98f7b309f46d@nvidia.com \
--to=smadhavan@nvidia.com \
--cc=alex.williamson@redhat.com \
--cc=alison.schofield@intel.com \
--cc=alwilliamson@nvidia.com \
--cc=bhelgaas@google.com \
--cc=dave.jiang@intel.com \
--cc=dave@stgolabs.net \
--cc=icheng@nvidia.com \
--cc=iweiny@kernel.org \
--cc=jan@nvidia.com \
--cc=jic23@kernel.org \
--cc=linux-cxl@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=linux-tegra@vger.kernel.org \
--cc=machaujo@amazon.de \
--cc=mhonap@nvidia.com \
--cc=ming.li@zohomail.com \
--cc=skancherla@nvidia.com \
--cc=vaslot@nvidia.com \
--cc=vishal.l.verma@intel.com \
--cc=vsethi@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox