* RE: [External Mail] Re: [PATCH v4 3/7] net: wwan: t9xx: Add control DMA interface
From: Wu. JackBB (GSM) @ 2026-07-17 6:37 UTC (permalink / raw)
To: Simon Horman
Cc: loic.poulain@oss.qualcomm.com, ryazanov.s.a@gmail.com,
johannes@sipsolutions.net, andrew+netdev@lunn.ch,
davem@davemloft.net, edumazet@google.com, kuba@kernel.org,
pabeni@redhat.com, wen-zhi.huang@mediatek.com,
shi-wei.yeh@mediatek.com, Minano.tseng@mediatek.com,
matthias.bgg@gmail.com, angelogioacchino.delregno@collabora.com,
corbet@lwn.net, skhan@linuxfoundation.org,
linux-kernel@vger.kernel.org, netdev@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-mediatek@lists.infradead.org, linux-doc@vger.kernel.org
In-Reply-To: <20260716093406.231071-1-horms@kernel.org>
Hi Simon,
> > +static void mtk_cldma_tx_done_work(...)
> > + if (!req->data_vm_addr || (req->gpd->tx_gpd.gpd_flags & CLDMA_GPD_FLAG_HWO))
>
> [Severity: High]
> Is a dma_rmb() needed after checking the hardware ownership flag?
The GPD descriptors are allocated from a coherent DMA pool
(dma_pool_zalloc), so CPU cache coherency is guaranteed by the DMA
mapping. A dma_rmb() is not needed because coherent memory ensures
the CPU always observes the latest values written by the device.
Additionally, there is an rmb() before this check that ensures the
HWO flag is read before any subsequent field accesses.
> > +static void mtk_cldma_rx_done_work(...)
> > + if (req->gpd->rx_gpd.gpd_flags & CLDMA_GPD_FLAG_HWO)
> > + break;
> > + mtk_cldma_rx_skb_adjust(mdev, rxq, req);
>
> [Severity: High]
> Similarly, would we need a dma_rmb() here before attempting to read the
> receive length out of the descriptor in mtk_cldma_rx_skb_adjust?
Same reasoning as above. GPD descriptors use coherent DMA memory
(dma_pool_zalloc), so no additional memory barrier is needed.
> > + ret = mtk_cldma_reload_rx_skb(mdev, rxq, req);
> > + if (ret)
> > + goto out;
>
> [Severity: High]
> If mtk_cldma_reload_rx_skb fails due to memory pressure, it jumps to out
> without advancing rxq->free_idx. Will this cause the worker to permanently
> stall the queue?
If mtk_cldma_reload_rx_skb() fails, the code jumps to "out" which
unmasks the interrupt. On the next hardware interrupt, rx_done_work
runs again and retries from the same free_idx. This creates a retry
loop that depends on memory becoming available. Under sustained
memory pressure the RX queue stalls but recovers once memory is
freed. This is acceptable for a control plane interface with small,
infrequent messages.
> > +static void mtk_cldma_txq_free(...)
> > + kfree(req->bd_dsc_pool);
> > + kfree(txq->req_pool);
> > + kfree(txq);
>
> [Severity: Critical]
> Does this incorrectly free memory managed by devres?
We will convert all four from devm_kcalloc to kcalloc in v5. These
are runtime-managed resources (allocated on queue open, freed on
queue close/teardown), so they should use kcalloc/kfree rather
than devres.
> > +static int mtk_cldma_reload_rx_skb(...)
> > + dev_kfree_skb_any(bd_dsc->skb);
>
> [Severity: High]
> Can this lead to a double free?
We will add bd_dsc->skb = NULL after dev_kfree_skb_any()
in the error path to prevent double free during driver teardown.
> > +static int mtk_cldma_txbuf_set(...)
> > + req->data_dma_addr = dma_map_single(mdev->dev, skb->data,
> > + skb->len, DMA_TO_DEVICE);
>
> [Severity: High]
> Does this code safely handle non-linear SKBs?
When nr_bds == 0, the SKB is always linear — it is allocated
internally via __dev_alloc_skb() with data copied via skb_put().
This is a control plane interface where SKBs are constructed
internally (AT commands, MBIM control messages), not received
from the network stack. Paged fragments are never used.
When nr_bds > 0, fragmentation uses skb_shinfo(skb)->frag_list
(linked SKBs), not paged fragments (skb_shinfo(skb)->frags).
> > +static int mtk_cldma_submit_tx(...)
> > + req->data_vm_addr = skb->data;
> > + wmb();
> > + req->gpd->tx_gpd.gpd_flags |= CLDMA_GPD_FLAG_HWO;
>
> [Severity: High]
> Could there be a race condition here with mtk_cldma_tx_done_work?
No race exists. tx_done_work checks:
if (!req->data_vm_addr || (gpd_flags & CLDMA_GPD_FLAG_HWO))
break;
It only proceeds when data_vm_addr is non-NULL AND HWO is cleared.
In submit_tx, data_vm_addr is set before HWO via wmb(). After
data_vm_addr is set but before HWO, tx_done_work sees HWO still
set (hardware hasn't completed DMA) and breaks. The rmb() in
tx_done_work ensures HWO is read first. tx_done_work only
processes this req after hardware clears HWO upon DMA completion.
> > +static void mtk_pci_remove(struct pci_dev *pdev)
> > + pci_clear_master(pdev);
> > + mtk_pci_dev_exit(mdev);
>
> [Severity: High]
> Is it intended to call pci_clear_master before mtk_pci_dev_exit?
Before this point, mtk_pci_pldr() has already power-cycled the
modem firmware via ACPI PXP._OFF/_ON. After PLDR, the modem is
in a fresh boot state with no active DMA. The CLDMA queues are
idle — there are no pending hardware DMA transactions.
pci_clear_master() disables bus mastering as a safety measure
before teardown. Since the hardware is already reset and idle,
this is the intended ordering.
> > + err = mtk_cldma_submit_tx(trans->dev, skb);
> > + if (err) {
> > + if (err == -EAGAIN)
> > + return;
>
> [Severity: High]
> If mtk_cldma_submit_tx returns -EAGAIN ... could this cause the
> mtk_ctrl_trb_thread loop to spin continuously without sleeping?
The -EAGAIN path returns from mtk_ctrl_trb_handler() to
mtk_ctrl_trb_thread(), which calls wait_event_interruptible().
After -EAGAIN, the skb remains on the list (non-empty), but the
CLDMA ring is full. The ring drains as hardware completes DMA,
triggering tx_done_work which calls wake_up on the trb_waitq.
The kthread sleeps until hardware completion wakes it — no
busy-spin occurs.
> > + wait_event_interruptible(srv->trb_waitq,
> > + !mtk_ctrl_chs_is_busy_or_empty(srv) ||
>
> [Severity: High]
> Can this cause an infinite loop if a signal is delivered to the thread?
Kernel threads in this driver are not targeted by user signals.
The kthread_stop and kthread_should_park mechanisms are the
primary lifecycle management, and both are checked in the wait
condition. This pattern is consistent with other WWAN drivers
(e.g., t7xx) and widely used across drivers/net/ where
wait_event_interruptible is the standard choice for kthread
wait loops.
> > + radix_tree_for_each_slot(slot, &trans->queue_tbl, &iter, 0) {
> > + queue = radix_tree_deref_slot(slot);
> > + radix_tree_delete(&trans->queue_tbl, iter.index);
>
> [Severity: High]
> Is it safe to iterate over the radix tree and call radix_tree_delete
> without holding rcu_read_lock?
This function is only called during teardown after all users have
been stopped (FSM shutdown, kthreads stopped, queues drained).
There is no concurrent access to the radix tree at this point.
The tree was initialized with INIT_RADIX_TREE (not RCU-tagged),
and we are the sole accessor during teardown. The cursor-based
iteration is safe for single-threaded deletion.
Thanks.
Jack Wu
^ permalink raw reply
* RE: [External Mail] Re: [PATCH v4 2/7] net: wwan: t9xx: Add control plane transaction layer
From: Wu. JackBB (GSM) @ 2026-07-17 6:27 UTC (permalink / raw)
To: Simon Horman
Cc: loic.poulain@oss.qualcomm.com, ryazanov.s.a@gmail.com,
johannes@sipsolutions.net, andrew+netdev@lunn.ch,
davem@davemloft.net, edumazet@google.com, kuba@kernel.org,
pabeni@redhat.com, wen-zhi.huang@mediatek.com,
shi-wei.yeh@mediatek.com, Minano.tseng@mediatek.com,
matthias.bgg@gmail.com, angelogioacchino.delregno@collabora.com,
corbet@lwn.net, skhan@linuxfoundation.org,
linux-kernel@vger.kernel.org, netdev@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-mediatek@lists.infradead.org, linux-doc@vger.kernel.org
In-Reply-To: <20260716093355.231047-1-horms@kernel.org>
Hi Simon,
> > +int mtk_ctrl_init(struct mtk_md_dev *mdev)
> > +{
> > + struct mtk_ctrl_blk *ctrl_blk;
> > + ctrl_blk = devm_kzalloc(mdev->dev, sizeof(*ctrl_blk), GFP_KERNEL);
>
> [Severity: Medium]
> Does this patch actually implement the TX and RX services described
> in the commit message?
We will rewrite the commit message to accurately describe
what this patch does — it introduces the control plane framework
and data structures, not the TX/RX service implementations.
> > + return LE32_TO_U32(cpu_to_le32(hw_bits));
> > +}
>
> [Severity: High]
> Does this code apply a double byte-swap on big-endian architectures?
T9XX currently only targets x86 (little-endian) platforms where
cpu_to_le32() is a no-op. The Kconfig enforces "depends on PCI &&
ACPI" which restricts this to x86 platforms. On little-endian, no
byte-swap occurs at any point, so ffs() operates on the correct
value.
Thanks.
Jack Wu
^ permalink raw reply
* [PATCH v2 0/2] blk-iolatency: fix io.latency documentation accuracy
From: Tao Cui @ 2026-07-17 6:02 UTC (permalink / raw)
To: Tejun Heo
Cc: Johannes Weiner, Michal Koutný, Jonathan Corbet, Josef Bacik,
Jens Axboe, cgroups, linux-doc, linux-block, linux-kernel, cuitao,
cui.tao
From: Tao Cui <cuitao@kylinos.cn>
Hi,
This series fixes three inaccuracies in the io.latency documentation
(Documentation/admin-guide/cgroup-v2.rst), all caused by the doc
describing only rotational-device behavior:
Patch 1 documents the rotational vs non-rotational difference:
throttling is based on average latency on rotational devices but on the
fraction of IOs that miss the target on non-rotational devices; the
tuning guidance and io.stat field list are updated accordingly
(avg_lat/win are rotational-only; missed/total are documented for
non-rotational devices).
Patch 2 corrects the delay unit: io.stat reports delay_nsec in
nanoseconds, not microseconds as the doc stated.
Both patches are documentation-only and checkpatch-clean.
---
Changes in v2 (address review feedback):
- The non-rotational tuning guidance no longer implies missed/total is a
latency baseline: iolatency reports no average latency for
non-rotational devices, so the target must be chosen from device
characteristics and missed/total only verifies it is being met.
- missed/total are now described as live counters for the current window
(accumulated since the last window boundary), not "the last window":
iolatency_ssd_stat() sums the per-cpu counters without resetting them,
while they are reset at window end by iolatency_check_latencies().
Tao Cui (2):
Docs/admin-guide/cgroup-v2: document io.latency rotational vs
non-rotational behavior
Docs/admin-guide/cgroup-v2: fix delay_nsec unit in io.latency doc
Documentation/admin-guide/cgroup-v2.rst | 55 +++++++++++++++++--------
1 file changed, 38 insertions(+), 17 deletions(-)
--
2.43.0
^ permalink raw reply
* [PATCH v2 2/2] Docs/admin-guide/cgroup-v2: fix delay_nsec unit in io.latency doc
From: Tao Cui @ 2026-07-17 6:02 UTC (permalink / raw)
To: Tejun Heo
Cc: Johannes Weiner, Michal Koutný, Jonathan Corbet, Josef Bacik,
Jens Axboe, cgroups, linux-doc, linux-block, linux-kernel, cuitao,
cui.tao
In-Reply-To: <20260717060225.2019764-1-cui.tao@linux.dev>
From: Tao Cui <cuitao@kylinos.cn>
The io.latency doc says the io.stat delay field counts microseconds. The
field is delay_nsec and is reported in nanoseconds. Refer to it by its
real name and correct the unit.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
---
Documentation/admin-guide/cgroup-v2.rst | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
index 4386365ef180..755c2a1942ef 100644
--- a/Documentation/admin-guide/cgroup-v2.rst
+++ b/Documentation/admin-guide/cgroup-v2.rst
@@ -2305,9 +2305,9 @@ This throttling takes 2 forms:
throttled without possibly adversely affecting higher priority groups. This
includes swapping and metadata IO. These types of IO are allowed to occur
normally, however they are "charged" to the originating group. If the
- originating group is being throttled you will see the use_delay and delay
- fields in io.stat increase. The delay value is how many microseconds that are
- being added to any process that runs in this group. Because this number can
+ originating group is being throttled you will see the use_delay and delay_nsec
+ fields in io.stat increase. The delay_nsec value is how many nanoseconds that
+ are being added to any process that runs in this group. Because this number can
grow quite large if there is a lot of swapping or metadata IO occurring we
limit the individual delay events to 1 second at a time.
--
2.43.0
^ permalink raw reply related
* [PATCH v2 1/2] Docs/admin-guide/cgroup-v2: document io.latency rotational vs non-rotational behavior
From: Tao Cui @ 2026-07-17 6:02 UTC (permalink / raw)
To: Tejun Heo
Cc: Johannes Weiner, Michal Koutný, Jonathan Corbet, Josef Bacik,
Jens Axboe, cgroups, linux-doc, linux-block, linux-kernel, cuitao,
cui.tao
In-Reply-To: <20260717060225.2019764-1-cui.tao@linux.dev>
From: Tao Cui <cuitao@kylinos.cn>
io.latency is documented only in terms of average latency and the avg_lat
stat, which matches rotational devices. On non-rotational devices a group
misses its target when 10% or more of the IOs in the window individually
exceed it, and io.stat reports missed/total rather than avg_lat/win.
Describe both cases: how a miss is detected, note that the avg_lat tuning
guidance is rotational-only, and update the io.stat field list (mark
avg_lat/win as rotational-only, document missed/total).
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
---
Documentation/admin-guide/cgroup-v2.rst | 49 ++++++++++++++++++-------
1 file changed, 35 insertions(+), 14 deletions(-)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
index 0df15a672cf3..4386365ef180 100644
--- a/Documentation/admin-guide/cgroup-v2.rst
+++ b/Documentation/admin-guide/cgroup-v2.rst
@@ -2260,9 +2260,12 @@ IO Latency
~~~~~~~~~~
This is a cgroup v2 controller for IO workload protection. You provide a group
-with a latency target, and if the average latency exceeds that target the
-controller will throttle any peers that have a lower latency target than the
-protected workload.
+with a latency target, and if the group misses its target the controller will
+throttle any peers that have a lower latency target than the protected
+workload. How a miss is detected depends on the device: on rotational devices
+the average latency over the window must exceed the target, while on
+non-rotational devices a miss is counted when 10% or more of the IOs in the
+window individually exceed the target.
The limits are only applied at the peer level in the hierarchy. This means that
in the diagram below, only groups A, B, and C will influence each other, and
@@ -2279,10 +2282,12 @@ So the ideal way to configure this is to set io.latency in groups A, B, and C.
Generally you do not want to set a value lower than the latency your device
supports. Experiment to find the value that works best for your workload.
Start at higher than the expected latency for your device and, with
-blkcg_debug_stats enabled, watch the avg_lat value in io.stat for your
-workload group to get an idea of the latency you see during normal operation.
-Use the avg_lat value as a basis for your real setting, setting at 10-15%
-higher than the value in io.stat.
+blkcg_debug_stats enabled, observe io.stat for your workload group to get an
+idea of the latency you see during normal operation. On rotational devices,
+use the avg_lat value as a basis for your real setting, setting it 10-15%
+higher. On non-rotational devices io.stat reports no average latency; set
+the target based on your device and use the missed/total fields to verify it
+is being met.
How IO Latency Throttling Works
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -2324,19 +2329,35 @@ IO Latency Interface Files
the blkcg_debug_stats module parameter is enabled (it is disabled by
default).
+ The reported latency fields depend on the device. Rotational devices
+ report avg_lat and win; non-rotational devices report missed and total
+ instead. missed and total are live counters for the current window and
+ may change between reads.
+
depth
This is the current queue depth for the group.
avg_lat
- This is an exponential moving average with a decay rate of 1/exp
- bound by the sampling interval. The decay rate interval can be
- calculated by multiplying the win value in io.stat by the
- corresponding number of samples based on the win value.
+ (Rotational devices only.) This is an exponential moving
+ average with a decay rate of 1/exp bound by the sampling
+ interval. The decay rate interval can be calculated by
+ multiplying the win value in io.stat by the corresponding number
+ of samples based on the win value.
win
- The sampling window size in milliseconds. This is the minimum
- duration of time between evaluation events. Windows only elapse
- with IO activity. Idle periods extend the most recent window.
+ (Rotational devices only.) The sampling window size in
+ milliseconds. This is the minimum duration of time between
+ evaluation events. Windows only elapse with IO activity. Idle
+ periods extend the most recent window.
+
+ missed
+ (Non-rotational devices only.) The number of IOs in the
+ current window whose latency exceeded the target.
+
+ total
+ (Non-rotational devices only.) The total number of IOs
+ accounted in the current window. A group is considered to be
+ missing its target once missed reaches 10% of total.
IO Priority
~~~~~~~~~~~
--
2.43.0
^ permalink raw reply related
* [PATCH v5 3/3] Documentation: PCI: Add PCI DMA endpoint function documentation
From: Koichiro Den @ 2026-07-17 5:09 UTC (permalink / raw)
To: Manivannan Sadhasivam, Krzysztof Wilczyński,
Kishon Vijay Abraham I, Bjorn Helgaas, Jonathan Corbet,
Shuah Khan, Vinod Koul, Frank Li, Arnd Bergmann, Damien Le Moal,
Niklas Cassel
Cc: Marek Vasut, Yoshihiro Shimoda, linux-pci, linux-doc,
linux-kernel, dmaengine
In-Reply-To: <20260717050953.2145851-1-den@valinux.co.jp>
Add a function description and a user guide for pci-epf-dma. Describe
the BAR-resident metadata consumed by dw-edma-pcie, the configfs
attributes, endpoint controller requirements and the host-side DMAengine
usage model.
Suggested-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Koichiro Den <den@valinux.co.jp>
---
Changes in v5:
- No changes.
Documentation/PCI/endpoint/index.rst | 2 +
.../PCI/endpoint/pci-dma-function.rst | 188 ++++++++++++++++
Documentation/PCI/endpoint/pci-dma-howto.rst | 201 ++++++++++++++++++
3 files changed, 391 insertions(+)
create mode 100644 Documentation/PCI/endpoint/pci-dma-function.rst
create mode 100644 Documentation/PCI/endpoint/pci-dma-howto.rst
diff --git a/Documentation/PCI/endpoint/index.rst b/Documentation/PCI/endpoint/index.rst
index dd1f62e731c9..cd4107e02ec2 100644
--- a/Documentation/PCI/endpoint/index.rst
+++ b/Documentation/PCI/endpoint/index.rst
@@ -15,6 +15,8 @@ PCI Endpoint Framework
pci-ntb-howto
pci-vntb-function
pci-vntb-howto
+ pci-dma-function
+ pci-dma-howto
pci-nvme-function
function/binding/pci-test
diff --git a/Documentation/PCI/endpoint/pci-dma-function.rst b/Documentation/PCI/endpoint/pci-dma-function.rst
new file mode 100644
index 000000000000..4de02553f5ff
--- /dev/null
+++ b/Documentation/PCI/endpoint/pci-dma-function.rst
@@ -0,0 +1,188 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+================
+PCI DMA Function
+================
+
+:Author: Koichiro Den <den@valinux.co.jp>
+
+The PCI DMA endpoint function exposes an endpoint-integrated DMA controller
+to the PCI host as a PCI DMA controller. A matching host-side driver
+discovers the endpoint DMA metadata and registers the delegated channels with
+the Linux DMAengine framework, so host DMAengine clients can submit
+transfers.
+
+An endpoint Linux system can already use an endpoint-integrated DMA
+controller locally through the normal DMAengine API, for example to transfer
+data between endpoint memory and host addresses reachable over PCI. The PCI
+DMA function provides a different ownership model: it delegates selected
+local DMA channels to the host, so a host DMAengine client can request and
+program those endpoint-side channels through the host's DMAengine API.
+
+To make that possible, the endpoint function publishes the DMA controller
+register window and descriptor memory layout to the host, reserves the
+selected local DMA channels on the endpoint side, and lets the host program
+those channels directly.
+
+Constructs Used for Implementing DMA
+====================================
+
+The PCI DMA function uses the following endpoint-side resources and
+configuration:
+
+ 1) DMA controller register window
+ 2) DMA descriptor memory for endpoint-to-RC channels
+ 3) DMA descriptor memory for RC-to-endpoint channels
+ 4) MSI or MSI-X interrupt vectors selected through configfs
+ 5) One endpoint BAR used to publish metadata
+ 6) If needed, one endpoint BAR used for dynamically mapped DMA windows
+
+The endpoint controller reports the DMA controller register and descriptor
+resources through the endpoint auxiliary resource interface. The PCI DMA
+function uses those descriptions to build the host-visible metadata and to map
+resources that are not already visible to the host.
+
+DMA Controller Register Window
+------------------------------
+
+It contains the DMA controller registers programmed by the host-side driver
+to submit transfers, control channels and handle DMA interrupts.
+
+DMA Descriptor Memory
+---------------------
+
+It contains the descriptor memory used by the DMA controller. The PCI DMA
+function exposes descriptor memory for the delegated endpoint-to-RC and
+RC-to-endpoint channels.
+
+MSI/MSI-X Interrupt Vectors
+---------------------------
+
+They are used by the delegated DMA channels to signal completion and error
+conditions to the host-side driver.
+
+Metadata BAR
+------------
+
+It is the endpoint BAR used to publish the endpoint DMA metadata and handshake
+bits. The BAR remains stable while the endpoint function programs the DMA
+windows.
+
+DMA Window BAR
+--------------
+
+It is the endpoint BAR used for DMA resources that are not already visible
+through a fixed BAR. The endpoint function may switch this BAR to subrange
+mapping after the host-side driver has found the metadata BAR.
+
+BAR Metadata
+============
+
+The endpoint function places a small metadata block at the beginning of the
+selected metadata BAR. The format is defined in
+``include/linux/pci-ep-dma.h``.
+
+The host-side driver scans the function's assigned memory BARs, looks for the
+endpoint DMA metadata magic, requests DMA window programming, waits for the
+READY bit, and then parses the metadata to find the DMA register window and
+descriptor windows.
+
+::
+
+ +----------------------+ metadata BAR offset 0
+ | endpoint DMA metadata|
+ +----------------------+
+ | optional padding |
+ +----------------------+
+
+ +----------------------+ DMA window BAR offset 0
+ | mapped DMA resources |
+ +----------------------+
+ | optional padding |
+ +----------------------+
+
+The metadata can also reference resources that are already host-visible
+through fixed BARs. For example, an endpoint controller may expose the DMA
+controller register window at a fixed BAR offset while descriptor memories
+are mapped into the DMA window BAR by the endpoint function.
+
+The metadata is BAR-resident instead of a self-contained PCI Vendor-Specific
+Extended Capability (VSEC). Some endpoint controllers do not provide writable
+configuration-space backing storage large enough for a new VSEC payload, while
+they can map endpoint memory and controller resources into a BAR.
+
+Channel Ownership
+=================
+
+The ``wr_chans`` attribute exposes endpoint-to-RC DMA write channels. The
+``rd_chans`` attribute exposes RC-to-endpoint DMA read channels. The function
+reserves the selected endpoint-side DMAengine channels so that endpoint-side
+DMAengine clients cannot allocate and use the same hardware channels while
+they are delegated to the host.
+
+The current metadata revision describes channels in dense, zero-based order.
+For example, ``wr_chans = 2`` exposes write channels 0 and 1. Skipping a
+hardware channel in the middle of the exposed range is not supported.
+
+DesignWare eDMA unroll and HDMA compatible layouts require each exposed
+direction to be delegated as a whole. For example, on a controller with two
+write channels, ``wr_chans`` must be either 0 or 2. DesignWare HDMA native
+linked-list mode uses per-channel registers, so a smaller dense prefix can be
+delegated.
+
+Interrupts
+==========
+
+The PCI DMA function exposes DMA interrupts through MSI or MSI-X. The common
+endpoint function ``msi_interrupts`` and ``msix_interrupts`` configfs attributes
+select the interrupt vector counts programmed into endpoint config space. At
+least one MSI or MSI-X vector must be configured before the function is bound
+to an endpoint controller.
+
+Transfer Addressing
+===================
+
+The host-side DMAengine client supplies the endpoint memory address as the
+DMA slave address. For example, the ``dw-edma-pcie`` endpoint DMA metadata
+parser passes that slave address to the DMA controller as a raw endpoint-side
+address instead of translating it through a host PCI BAR resource.
+
+The host memory buffer used as the other side of the transfer is still mapped
+using the normal DMA mapping API on the host.
+
+Endpoint Controller Requirements
+================================
+
+The endpoint controller driver must expose the DMA controller register
+window and per-channel descriptor memories through the endpoint auxiliary
+resource API. Endpoint controllers with other DMA register layouts also need
+matching metadata and host-side DMAengine driver support.
+
+Current DesignWare endpoint DMA support exposes only channels with descriptor
+memory; HDMA native non-linked-list mode is not supported yet.
+
+If any DMA resource is not already host-visible through a fixed BAR, the
+endpoint controller must also support BAR subrange mapping and dynamic inbound
+mapping, because the DMA window BAR is assembled from those resources.
+
+Current Support
+===============
+
+The current host-side support is implemented in ``dw-edma-pcie`` for
+DesignWare eDMA unroll, HDMA compatible and HDMA native linked-list layouts.
+Other PCIe controller DMA implementations need corresponding host-side
+DMAengine driver support.
+
+The ``dw-edma-pcie`` PCI ID table does not contain a generic endpoint DMA PCI
+ID entry. Users need to bind the host-side driver explicitly using
+``driver_override``.
+
+The current metadata revision requires the exposed channels to be a dense
+prefix of the hardware channel numbers.
+
+Security Model
+==============
+
+The interface is intended for trusted endpoint/host deployments. A delegated
+DMA channel can access endpoint memory addresses supplied by a host DMAengine
+client.
diff --git a/Documentation/PCI/endpoint/pci-dma-howto.rst b/Documentation/PCI/endpoint/pci-dma-howto.rst
new file mode 100644
index 000000000000..4bdce63c6f7f
--- /dev/null
+++ b/Documentation/PCI/endpoint/pci-dma-howto.rst
@@ -0,0 +1,201 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+==========================================
+PCI DMA Endpoint Function (EPF) User Guide
+==========================================
+
+:Author: Koichiro Den <den@valinux.co.jp>
+
+This guide shows how to configure the ``pci-epf-dma`` endpoint function driver.
+It uses ``dw-edma-pcie`` as the currently available host-side driver. For the
+hardware model and layout see Documentation/PCI/endpoint/pci-dma-function.rst.
+
+Endpoint Device
+===============
+
+Endpoint Controller Devices
+---------------------------
+
+To find the list of endpoint controller devices in the system::
+
+ # ls /sys/class/pci_epc/
+ e65d0000.pcie-ep
+
+If ``PCI_ENDPOINT_CONFIGFS`` is enabled::
+
+ # ls /sys/kernel/config/pci_ep/controllers
+ e65d0000.pcie-ep
+
+Endpoint Function Drivers
+-------------------------
+
+To find the list of endpoint function drivers in the system::
+
+ # ls /sys/bus/pci-epf/drivers
+ pci_epf_dma pci_epf_test
+
+If ``PCI_ENDPOINT_CONFIGFS`` is enabled::
+
+ # ls /sys/kernel/config/pci_ep/functions
+ pci_epf_dma pci_epf_test
+
+Creating pci-epf-dma Device
+---------------------------
+
+Create a ``pci-epf-dma`` device with configfs::
+
+ # mount -t configfs none /sys/kernel/config
+ # cd /sys/kernel/config/pci_ep/
+ # mkdir functions/pci_epf_dma/dma0
+
+The "mkdir dma0" above creates the ``pci-epf-dma`` function device that will
+be probed by the ``pci_epf_dma`` driver.
+
+The PCI endpoint framework populates the directory with the common
+configurable fields::
+
+ # ls functions/pci_epf_dma/dma0
+ baseclass_code msi_interrupts progif_code subsys_id
+ cache_line_size msix_interrupts revid subsys_vendor_id
+ deviceid pci_epf_dma.0 secondary vendorid
+ interrupt_pin primary subclass_code
+
+The PCI DMA function driver also creates a function-specific sub-directory.
+The numeric suffix depends on the endpoint function instance number::
+
+ # ls functions/pci_epf_dma/dma0/pci_epf_dma.0/
+ dma_window_bar metadata_bar rd_chans wr_chans
+
+Configuring pci-epf-dma Device
+------------------------------
+
+The host-side ``dw-edma-pcie`` PCI ID table does not contain a generic
+endpoint DMA PCI ID entry. Choose a PCI vendor/device ID for the endpoint
+device::
+
+ # echo <vendor-id> > functions/pci_epf_dma/dma0/vendorid
+ # echo <device-id> > functions/pci_epf_dma/dma0/deviceid
+ # echo 1 > functions/pci_epf_dma/dma0/msi_interrupts
+
+The PCI class defaults to ``PCI_BASE_CLASS_SYSTEM`` and
+``PCI_CLASS_SYSTEM_DMA``.
+
+The function-specific attributes are:
+
+============== ============================================================
+Attribute Description
+============== ============================================================
+metadata_bar BAR used to publish the endpoint DMA metadata and handshake
+ bits. It is kept as a stable BAR while the DMA windows are
+ programmed. If this is left unset, the first usable BAR that
+ does not already contain a fixed DMA resource is used.
+dma_window_bar BAR used for DMA resources that are not already host-visible,
+ such as the DMA register window or descriptor windows. This
+ BAR may be switched to subrange mapping after the host driver
+ has found the metadata. If this is left unset and a DMA
+ window is needed, the first usable BAR different from
+ ``metadata_bar`` and not already occupied by a fixed DMA
+ resource is used.
+wr_chans Number of endpoint-to-RC DMA write channels to expose.
+rd_chans Number of RC-to-endpoint DMA read channels to expose.
+============== ============================================================
+
+A sample configuration for a DesignWare eDMA/HDMA compatible controller with
+two write channels and two read channels is given below::
+
+ # echo 0 > functions/pci_epf_dma/dma0/pci_epf_dma.0/metadata_bar
+ # echo 2 > functions/pci_epf_dma/dma0/pci_epf_dma.0/dma_window_bar
+ # echo 2 > functions/pci_epf_dma/dma0/pci_epf_dma.0/wr_chans
+ # echo 2 > functions/pci_epf_dma/dma0/pci_epf_dma.0/rd_chans
+
+``wr_chans`` and ``rd_chans`` default to 0. At least one channel direction
+must be configured. The selected channels are exposed in dense, zero-based
+order; for example, ``wr_chans = 2`` exposes write channels 0 and 1.
+DesignWare eDMA unroll and HDMA compatible layouts require each exposed
+direction to be delegated as a whole, so set a direction to either 0 or the
+number of hardware channels in that direction. DesignWare HDMA native
+linked-list mode allows a smaller dense prefix. If ``dma_window_bar`` is
+configured, it must be different from ``metadata_bar``.
+
+The common ``msi_interrupts`` and ``msix_interrupts`` attributes select the
+number of MSI and MSI-X vectors exposed to the host. At least one MSI or
+MSI-X vector must be configured.
+
+The function-specific attributes can only be changed before the endpoint
+function is bound to an endpoint controller.
+
+Binding pci-epf-dma Device to EP Controller
+-------------------------------------------
+
+The DMA function device should be attached to a PCI endpoint controller
+connected to the host::
+
+ # ln -s controllers/e65d0000.pcie-ep \
+ functions/pci_epf_dma/dma0/primary/
+
+Once the above step is completed, the PCI endpoint controller is ready to
+establish a link with the host.
+
+Start the Link
+--------------
+
+Start the endpoint controller by writing 1 to ``start``::
+
+ # echo 1 > controllers/e65d0000.pcie-ep/start
+
+Root Complex Device
+===================
+
+lspci Output
+------------
+
+Note that the device listed here corresponds to the values populated in the
+endpoint configuration above::
+
+ # lspci -nk
+ 01:00.1 0801: <vendor-id>:<device-id>
+
+If the host was already running while the endpoint function was configured,
+rescan the PCI bus after the endpoint side has completed the configfs setup
+and started the endpoint controller, if the platform supports it.
+
+Bind the endpoint DMA function to ``dw-edma-pcie`` explicitly with
+``driver_override``::
+
+ # modprobe dw_edma_pcie
+ # echo dw-edma-pcie > /sys/bus/pci/devices/0000:01:00.1/driver_override
+ # echo 0000:01:00.1 > /sys/bus/pci/drivers_probe
+
+The device should then be bound to ``dw-edma-pcie``::
+
+ # lspci -nk -s 01:00.1
+ 01:00.1 0801: <vendor-id>:<device-id>
+ Kernel driver in use: dw-edma-pcie
+
+Using pci-epf-dma Device
+------------------------
+
+The host side software uses the standard Linux DMAengine API. A DMAengine
+client driver running on the host must request one of the channels provided by
+``dw-edma-pcie`` and submit a transfer.
+
+For an endpoint-to-RC write transfer, the DMAengine client uses a host DMA
+buffer as the destination and an endpoint-side address as the slave source
+address. For an RC-to-endpoint read transfer, the DMAengine client uses a
+host DMA buffer as the source and an endpoint-side address as the slave
+destination address.
+
+Troubleshooting
+===============
+
+``pci-epf-dma`` requires endpoint controller support for DMA auxiliary
+resources and MSI or MSI-X. If any DMA resource must be mapped dynamically,
+the endpoint controller must also support BAR subrange mapping and dynamic
+inbound mapping. Binding the function to an endpoint controller fails if the
+required capabilities are not available, or if both ``msi_interrupts`` and
+``msix_interrupts`` are zero.
+
+If ``dw-edma-pcie`` fails to probe on the host, check that the endpoint was
+bound to the host driver, that the endpoint BARs were assigned by PCI
+enumeration, and that the endpoint DMA metadata READY bit was set after any
+DMA window BAR submaps were programmed.
--
2.51.0
^ permalink raw reply related
* [PATCH v5 2/3] PCI: endpoint: Add DMA endpoint function
From: Koichiro Den @ 2026-07-17 5:09 UTC (permalink / raw)
To: Manivannan Sadhasivam, Krzysztof Wilczyński,
Kishon Vijay Abraham I, Bjorn Helgaas, Jonathan Corbet,
Shuah Khan, Vinod Koul, Frank Li, Arnd Bergmann, Damien Le Moal,
Niklas Cassel
Cc: Marek Vasut, Yoshihiro Shimoda, linux-pci, linux-doc,
linux-kernel, dmaengine
In-Reply-To: <20260717050953.2145851-1-den@valinux.co.jp>
Add pci-epf-dma, an endpoint function that exposes selected
endpoint-integrated DMA channels as a separate PCI DMA controller
function.
The function consumes EPC auxiliary DMA channel and descriptor memory
resources, delegates channels through the EPC DMA channel delegation
API, publishes a stable metadata BAR for host discovery, and uses a DMA
window BAR for DMA resources that are not already host-visible. For
DesignWare eDMA unroll and HDMA compatible layouts, channel delegation
is constrained to whole directions. HDMA native linked-list mode uses
per-channel registers and can delegate a dense channel prefix without
taking the whole direction.
After the host-side driver finds the metadata and requests the final
layout, the endpoint function programs DMA window BAR submaps and marks
the metadata ready. If the link drops after the host request is set,
clear only the ready bit and retry submap programming on the next
link-up without requiring the host driver to probe again.
If setup fails before the metadata is marked ready, release any
delegated channels without asking the EPC backend to quiesce them. Once
the ready bit has been set, teardown requests quiesce because the host
may have programmed the exposed DMA windows. Shared-register
directions are quiesced on the first channel reclaim; the remaining
handles are reclaimed without repeating the direction-wide quiesce.
The endpoint function does not bake in a vendor/device ID. As with other
generic endpoint functions, users provide the PCI IDs through the common
EPF configfs header attributes.
Signed-off-by: Koichiro Den <den@valinux.co.jp>
---
Changes in v5:
- Quiesce each shared-register direction once during channel release.
drivers/pci/endpoint/functions/Kconfig | 13 +
drivers/pci/endpoint/functions/Makefile | 1 +
drivers/pci/endpoint/functions/pci-epf-dma.c | 1524 ++++++++++++++++++
3 files changed, 1538 insertions(+)
create mode 100644 drivers/pci/endpoint/functions/pci-epf-dma.c
diff --git a/drivers/pci/endpoint/functions/Kconfig b/drivers/pci/endpoint/functions/Kconfig
index bb5a23994288..8783c82672cc 100644
--- a/drivers/pci/endpoint/functions/Kconfig
+++ b/drivers/pci/endpoint/functions/Kconfig
@@ -39,6 +39,19 @@ config PCI_EPF_VNTB
If in doubt, say "N" to disable Endpoint NTB driver.
+config PCI_EPF_DMA
+ tristate "PCI Endpoint DMA driver"
+ depends on PCI_ENDPOINT
+ select CONFIGFS_FS
+ help
+ Select this configuration option to expose an endpoint-integrated
+ DMA controller as a PCI endpoint function. The function advertises
+ the DMA controller layout to the host using BAR-resident metadata
+ and maps resources that are not already host-visible into the
+ DMA window BAR.
+
+ If in doubt, say "N" to disable Endpoint DMA driver.
+
config PCI_EPF_MHI
tristate "PCI Endpoint driver for MHI bus"
depends on PCI_ENDPOINT && MHI_BUS_EP
diff --git a/drivers/pci/endpoint/functions/Makefile b/drivers/pci/endpoint/functions/Makefile
index 696473fce50e..de92f6897b8f 100644
--- a/drivers/pci/endpoint/functions/Makefile
+++ b/drivers/pci/endpoint/functions/Makefile
@@ -6,4 +6,5 @@
obj-$(CONFIG_PCI_EPF_TEST) += pci-epf-test.o
obj-$(CONFIG_PCI_EPF_NTB) += pci-epf-ntb.o
obj-$(CONFIG_PCI_EPF_VNTB) += pci-epf-vntb.o
+obj-$(CONFIG_PCI_EPF_DMA) += pci-epf-dma.o
obj-$(CONFIG_PCI_EPF_MHI) += pci-epf-mhi.o
diff --git a/drivers/pci/endpoint/functions/pci-epf-dma.c b/drivers/pci/endpoint/functions/pci-epf-dma.c
new file mode 100644
index 000000000000..a81e6e1e6e7b
--- /dev/null
+++ b/drivers/pci/endpoint/functions/pci-epf-dma.c
@@ -0,0 +1,1524 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * PCI endpoint function that exposes an endpoint-integrated DMA controller
+ * to the PCI host.
+ *
+ * The host-side dw-edma-pcie driver consumes the BAR metadata published
+ * by this function.
+ */
+
+#include <linux/bitfield.h>
+#include <linux/configfs.h>
+#include <linux/dma/edma.h>
+#include <linux/dma-mapping.h>
+#include <linux/module.h>
+#include <linux/overflow.h>
+#include <linux/pci-ep-dma.h>
+#include <linux/pci-epc.h>
+#include <linux/pci-epf.h>
+#include <linux/pci_regs.h>
+#include <linux/slab.h>
+#include <linux/workqueue.h>
+
+/* HOST_REQ is set by the host driver, so poll it at a low idle rate. */
+#define PCI_EPF_DMA_HOST_REQ_POLL_MS 500
+
+struct pci_epf_dma_bar_map {
+ const struct pci_epc_aux_resource *res;
+ enum pci_barno bar;
+ u64 res_offset_in_bar;
+ u64 submap_offset_in_bar;
+ dma_addr_t phys_addr;
+ dma_addr_t dma_addr;
+ size_t map_size;
+ bool needs_submap;
+ bool dma_mapped;
+};
+
+struct pci_epf_dma {
+ struct pci_epf *epf;
+ struct config_group group;
+ struct delayed_work map_work;
+
+ enum pci_barno metadata_bar;
+ enum pci_barno dma_window_bar;
+ u16 wr_chans;
+ u16 rd_chans;
+ u8 reg_layout;
+ u8 reg_layout_data;
+
+ /*
+ * Backing storage for ctrl, channel and descriptor resource pointers.
+ * DesignWare eDMA is the only exposed layout today, so EDMA_MAX_* is
+ * the per-direction bound for these arrays.
+ */
+ struct pci_epc_aux_resource *resources;
+ unsigned int num_resources;
+ const struct pci_epc_aux_resource *ctrl;
+ const struct pci_epc_aux_resource *ep_to_rc_aux_chan[EDMA_MAX_WR_CH];
+ const struct pci_epc_aux_resource *rc_to_ep_aux_chan[EDMA_MAX_RD_CH];
+ const struct pci_epc_aux_resource *ep_to_rc_desc[EDMA_MAX_WR_CH];
+ const struct pci_epc_aux_resource *rc_to_ep_desc[EDMA_MAX_RD_CH];
+
+ /* Local EPC reservations for channels delegated to the host. */
+ struct pci_epc_dma_chan *ep_to_rc_chan[EDMA_MAX_WR_CH];
+ struct pci_epc_dma_chan *rc_to_ep_chan[EDMA_MAX_RD_CH];
+
+ void *metadata_addr;
+ void *dma_window_addr;
+ size_t msix_table_offset;
+ struct pci_epf_dma_bar_map *bar_maps;
+ unsigned int num_bar_maps;
+ struct pci_epf_bar_submap *submaps;
+ unsigned int num_submaps;
+
+ /* Cleared when a later event should retry programming the submaps. */
+ bool submaps_programmed;
+ bool channels_exposed;
+};
+
+#define to_epf_dma(epf_group) container_of((epf_group), struct pci_epf_dma, group)
+
+static struct pci_epf_header pci_epf_dma_header = {
+ .vendorid = PCI_ANY_ID,
+ .deviceid = PCI_ANY_ID,
+ .baseclass_code = PCI_BASE_CLASS_SYSTEM,
+ .subclass_code = PCI_CLASS_SYSTEM_DMA & 0xff,
+ .interrupt_pin = PCI_INTERRUPT_INTA,
+};
+
+static void
+pci_epf_dma_release_direction(struct pci_epc_dma_chan **chans,
+ unsigned int count, bool quiesce, bool shared)
+{
+ unsigned int i;
+
+ for (i = 0; i < count; i++) {
+ if (!chans[i])
+ continue;
+
+ pci_epc_reclaim_dma_chan(chans[i], quiesce);
+ chans[i] = NULL;
+
+ /* One quiesce covers the whole shared direction. */
+ if (shared)
+ quiesce = false;
+ }
+}
+
+static void pci_epf_dma_release_channels(struct pci_epf_dma *epf_dma)
+{
+ bool shared;
+
+ shared = epf_dma->reg_layout == PCI_EP_DMA_METADATA_REG_LAYOUT_DW_EDMA &&
+ (epf_dma->reg_layout_data == EDMA_MF_EDMA_UNROLL ||
+ epf_dma->reg_layout_data == EDMA_MF_HDMA_COMPAT);
+
+ pci_epf_dma_release_direction(epf_dma->ep_to_rc_chan,
+ ARRAY_SIZE(epf_dma->ep_to_rc_chan),
+ epf_dma->channels_exposed, shared);
+ pci_epf_dma_release_direction(epf_dma->rc_to_ep_chan,
+ ARRAY_SIZE(epf_dma->rc_to_ep_chan),
+ epf_dma->channels_exposed, shared);
+
+ epf_dma->channels_exposed = false;
+}
+
+static int pci_epf_dma_claim_channel(struct pci_epf_dma *epf_dma,
+ const struct pci_epc_aux_resource *res,
+ struct pci_epc_dma_chan **chan)
+{
+ struct pci_epf *epf = epf_dma->epf;
+ struct device *dev = &epf_dma->epf->dev;
+ int ret;
+
+ ret = pci_epc_delegate_dma_chan(epf->epc, epf->func_no, epf->vfunc_no,
+ res->u.dma_chan.dir,
+ res->u.dma_chan.hw_ch, chan);
+ if (ret) {
+ dev_err(dev, "DMA channel %u cannot be delegated\n",
+ res->u.dma_chan.hw_ch);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int
+pci_epf_dma_validate_dw_edma_ctrl(struct pci_epf_dma *epf_dma,
+ const struct pci_epc_aux_resource *ctrl)
+{
+ struct device *dev = &epf_dma->epf->dev;
+ enum dw_edma_map_format map = ctrl->u.dma_ctrl.reg_layout_data;
+ u16 total_wr_chans = ctrl->u.dma_ctrl.ep_to_rc_ch_cnt;
+ u16 total_rd_chans = ctrl->u.dma_ctrl.rc_to_ep_ch_cnt;
+
+ switch (map) {
+ case EDMA_MF_EDMA_LEGACY:
+ dev_err(dev, "legacy DesignWare eDMA layout cannot be delegated\n");
+ return -EOPNOTSUPP;
+ case EDMA_MF_EDMA_UNROLL:
+ case EDMA_MF_HDMA_COMPAT:
+ /*
+ * The unrolled eDMA and HDMA compatibility maps share control
+ * across each direction, so do not split one direction between
+ * the endpoint and the host.
+ */
+ if ((epf_dma->wr_chans && epf_dma->wr_chans != total_wr_chans) ||
+ (epf_dma->rd_chans && epf_dma->rd_chans != total_rd_chans)) {
+ dev_err(dev, "DesignWare shared-register DMA delegation must cover the whole direction\n");
+ return -EOPNOTSUPP;
+ }
+ return 0;
+ case EDMA_MF_HDMA_NATIVE:
+ return 0;
+ default:
+ return -EINVAL;
+ }
+}
+
+static bool pci_epf_dma_bar_usable(const struct pci_epc_features *epc_features,
+ enum pci_barno bar)
+{
+ if (bar < BAR_0 || bar >= PCI_STD_NUM_BARS)
+ return false;
+
+ return epc_features->bar[bar].type != BAR_RESERVED &&
+ epc_features->bar[bar].type != BAR_DISABLED;
+}
+
+static bool pci_epf_dma_bar_has_fixed_resource(struct pci_epf_dma *epf_dma,
+ enum pci_barno bar)
+{
+ unsigned int i;
+
+ for (i = 0; i < epf_dma->num_resources; i++) {
+ if (epf_dma->resources[i].bar == bar)
+ return true;
+ }
+
+ return false;
+}
+
+static enum pci_barno
+pci_epf_dma_first_usable_bar(struct pci_epf_dma *epf_dma,
+ const struct pci_epc_features *epc_features,
+ enum pci_barno exclude)
+{
+ enum pci_barno bar;
+
+ for (bar = BAR_0; bar < PCI_STD_NUM_BARS; bar++) {
+ bar = pci_epc_get_next_free_bar(epc_features, bar);
+ if (bar == NO_BAR)
+ return NO_BAR;
+ if (bar != exclude &&
+ !pci_epf_dma_bar_has_fixed_resource(epf_dma, bar))
+ return bar;
+ }
+
+ return NO_BAR;
+}
+
+static size_t pci_epf_dma_align_size(size_t size, size_t align)
+{
+ if (!align)
+ return size;
+
+ return ALIGN(size, align);
+}
+
+static int pci_epf_dma_reuse_submap(struct pci_epf_dma *epf_dma,
+ unsigned int map_count,
+ dma_addr_t phys_addr, size_t map_size,
+ size_t offset, size_t *next_offset_in_bar,
+ u64 *res_offset_in_bar)
+{
+ struct pci_epf_dma_bar_map *map;
+ u64 delta;
+ size_t merged_size, next;
+ u64 res_map_end, submap_bar_end, submap_phys_end;
+ unsigned int i;
+
+ if (check_add_overflow(phys_addr, map_size, &res_map_end))
+ return -EOVERFLOW;
+
+ for (i = 0; i < map_count; i++) {
+ map = &epf_dma->bar_maps[i];
+ if (!map->needs_submap || map->bar != epf_dma->dma_window_bar)
+ continue;
+
+ if (check_add_overflow(map->phys_addr, map->map_size,
+ &submap_phys_end) ||
+ check_add_overflow(map->submap_offset_in_bar,
+ map->map_size, &submap_bar_end))
+ return -EOVERFLOW;
+
+ /*
+ * Reuse a submap that already covers this aligned resource
+ * window.
+ */
+ if (phys_addr >= map->phys_addr &&
+ res_map_end <= submap_phys_end) {
+ if (check_add_overflow(phys_addr - map->phys_addr,
+ offset, &delta) ||
+ check_add_overflow(map->submap_offset_in_bar,
+ delta, res_offset_in_bar))
+ return -EOVERFLOW;
+ return 1;
+ }
+
+ /*
+ * Extend only the BAR-tail submap when the physical ranges are
+ * contiguous.
+ */
+ if (submap_phys_end == phys_addr &&
+ submap_bar_end == *next_offset_in_bar) {
+ if (check_add_overflow(map->map_size, map_size,
+ &merged_size) ||
+ check_add_overflow(*next_offset_in_bar, map_size,
+ &next) ||
+ check_add_overflow(*next_offset_in_bar, offset,
+ res_offset_in_bar))
+ return -EOVERFLOW;
+
+ map->map_size = merged_size;
+ *next_offset_in_bar = next;
+ return 1;
+ }
+ }
+
+ return 0;
+}
+
+static int pci_epf_dma_add_map(struct pci_epf_dma *epf_dma,
+ const struct pci_epc_aux_resource *res,
+ size_t align, size_t *next_offset_in_bar,
+ unsigned int *map_idx)
+{
+ dma_addr_t phys_addr;
+ size_t map_size, offset = 0, next;
+ u64 res_offset_in_bar;
+ unsigned int i;
+ int ret;
+
+ if (!res || !res->size)
+ return -EINVAL;
+
+ for (i = 0; i < *map_idx; i++) {
+ if (epf_dma->bar_maps[i].res == res)
+ return 0;
+ }
+
+ if (res->bar != NO_BAR) {
+ if (res->bar < BAR_0 || res->bar >= PCI_STD_NUM_BARS)
+ return -EINVAL;
+ if (res->bar == epf_dma->metadata_bar ||
+ res->bar == epf_dma->dma_window_bar)
+ return -EINVAL;
+
+ epf_dma->bar_maps[*map_idx] = (struct pci_epf_dma_bar_map) {
+ .res = res,
+ .bar = res->bar,
+ .res_offset_in_bar = res->bar_offset,
+ .map_size = res->size,
+ };
+ (*map_idx)++;
+
+ return 0;
+ }
+
+ if (epf_dma->dma_window_bar == NO_BAR)
+ return -EOPNOTSUPP;
+
+ phys_addr = res->phys_addr;
+ /* Map the aligned window that contains this resource. */
+ if (align) {
+ phys_addr = ALIGN_DOWN(res->phys_addr, align);
+ offset = res->phys_addr - phys_addr;
+ }
+
+ if (check_add_overflow(res->size, offset, &map_size))
+ return -EOVERFLOW;
+ map_size = pci_epf_dma_align_size(map_size, align);
+
+ ret = pci_epf_dma_reuse_submap(epf_dma, *map_idx, phys_addr, map_size,
+ offset, next_offset_in_bar,
+ &res_offset_in_bar);
+ if (ret < 0)
+ return ret;
+ if (ret) {
+ epf_dma->bar_maps[*map_idx] = (struct pci_epf_dma_bar_map) {
+ .res = res,
+ .bar = epf_dma->dma_window_bar,
+ .res_offset_in_bar = res_offset_in_bar,
+ .phys_addr = res->phys_addr,
+ .map_size = res->size,
+ };
+
+ (*map_idx)++;
+
+ return 0;
+ }
+
+ if (check_add_overflow(*next_offset_in_bar, map_size, &next))
+ return -EOVERFLOW;
+ if (check_add_overflow(*next_offset_in_bar, offset, &res_offset_in_bar))
+ return -EOVERFLOW;
+
+ epf_dma->bar_maps[*map_idx] = (struct pci_epf_dma_bar_map) {
+ .res = res,
+ .bar = epf_dma->dma_window_bar,
+ .res_offset_in_bar = res_offset_in_bar,
+ .submap_offset_in_bar = *next_offset_in_bar,
+ .phys_addr = phys_addr,
+ .map_size = map_size,
+ .needs_submap = true,
+ };
+
+ *next_offset_in_bar = next;
+ (*map_idx)++;
+
+ return 0;
+}
+
+static void pci_epf_dma_unmap_submaps(struct pci_epf_dma *epf_dma)
+{
+ struct pci_epc *epc = epf_dma->epf->epc;
+ struct device *dev;
+ unsigned int i;
+
+ if (!epc)
+ return;
+
+ dev = epc->dev.parent;
+
+ for (i = 0; i < epf_dma->num_bar_maps; i++) {
+ struct pci_epf_dma_bar_map *map = &epf_dma->bar_maps[i];
+
+ if (!map->dma_mapped)
+ continue;
+
+ dma_unmap_resource(dev, map->dma_addr, map->map_size,
+ DMA_BIDIRECTIONAL, 0);
+ map->dma_mapped = false;
+ }
+}
+
+static int pci_epf_dma_map_submaps(struct pci_epf_dma *epf_dma)
+{
+ struct pci_epc *epc = epf_dma->epf->epc;
+ struct device *dev = epc->dev.parent;
+ unsigned int i;
+
+ for (i = 0; i < epf_dma->num_bar_maps; i++) {
+ struct pci_epf_dma_bar_map *map = &epf_dma->bar_maps[i];
+
+ if (!map->needs_submap)
+ continue;
+
+ /*
+ * Descriptor resources carry endpoint-local DMA addresses.
+ * Controller MMIO resources are CPU physical addresses, so map
+ * them for the EPC parent before programming inbound windows.
+ */
+ if (map->res->type != PCI_EPC_AUX_DMA_CTRL_MMIO)
+ continue;
+
+ map->dma_addr = dma_map_resource(dev, map->phys_addr,
+ map->map_size,
+ DMA_BIDIRECTIONAL, 0);
+ if (dma_mapping_error(dev, map->dma_addr)) {
+ pci_epf_dma_unmap_submaps(epf_dma);
+ return -EIO;
+ }
+
+ map->dma_mapped = true;
+ }
+
+ return 0;
+}
+
+static dma_addr_t pci_epf_dma_submap_addr(const struct pci_epf_dma_bar_map *map)
+{
+ if (map->dma_mapped)
+ return map->dma_addr;
+
+ return map->phys_addr;
+}
+
+static const struct pci_epf_dma_bar_map *
+pci_epf_dma_find_map(struct pci_epf_dma *epf_dma,
+ const struct pci_epc_aux_resource *res)
+{
+ unsigned int i;
+
+ for (i = 0; i < epf_dma->num_bar_maps; i++) {
+ if (epf_dma->bar_maps[i].res == res)
+ return &epf_dma->bar_maps[i];
+ }
+
+ return NULL;
+}
+
+static bool pci_epf_dma_needs_dma_window(struct pci_epf_dma *epf_dma)
+{
+ unsigned int i;
+
+ if (epf_dma->ctrl && epf_dma->ctrl->bar == NO_BAR)
+ return true;
+
+ for (i = 0; i < epf_dma->wr_chans; i++) {
+ if (epf_dma->ep_to_rc_desc[i] &&
+ epf_dma->ep_to_rc_desc[i]->bar == NO_BAR)
+ return true;
+ }
+
+ for (i = 0; i < epf_dma->rd_chans; i++) {
+ if (epf_dma->rc_to_ep_desc[i] &&
+ epf_dma->rc_to_ep_desc[i]->bar == NO_BAR)
+ return true;
+ }
+
+ return false;
+}
+
+static const struct pci_epc_aux_resource *
+pci_epf_dma_find_desc_mem(const struct pci_epc_aux_resource *res, int count,
+ u16 id)
+{
+ int i;
+
+ for (i = 0; i < count; i++) {
+ if (res[i].type == PCI_EPC_AUX_DMA_DESC_MEM &&
+ res[i].u.dma_desc.id == id)
+ return &res[i];
+ }
+
+ return NULL;
+}
+
+static int pci_epf_dma_collect_resources(struct pci_epf_dma *epf_dma)
+{
+ const struct pci_epc_aux_resource *ep_to_rc_aux_chan[EDMA_MAX_WR_CH] = {};
+ const struct pci_epc_aux_resource *rc_to_ep_aux_chan[EDMA_MAX_RD_CH] = {};
+ const struct pci_epc_aux_resource *ep_to_rc_desc[EDMA_MAX_WR_CH] = {};
+ const struct pci_epc_aux_resource *rc_to_ep_desc[EDMA_MAX_RD_CH] = {};
+ const struct pci_epc_aux_resource *ctrl = NULL;
+ struct pci_epf *epf = epf_dma->epf;
+ struct pci_epc *epc = epf->epc;
+ struct device *dev = &epf->dev;
+ int count, i, ret;
+
+ count = pci_epc_get_aux_resources_count(epc, epf->func_no,
+ epf->vfunc_no);
+ if (count <= 0)
+ return count ?: -ENODEV;
+
+ struct pci_epc_aux_resource *res __free(kfree) =
+ kzalloc_objs(*res, count);
+ if (!res)
+ return -ENOMEM;
+
+ ret = pci_epc_get_aux_resources(epc, epf->func_no, epf->vfunc_no,
+ res, count);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < count; i++) {
+ switch (res[i].type) {
+ case PCI_EPC_AUX_DMA_CTRL_MMIO:
+ if (ctrl)
+ return -EINVAL;
+ ctrl = &res[i];
+ break;
+ case PCI_EPC_AUX_DMA_CHAN: {
+ u16 hw_ch = res[i].u.dma_chan.hw_ch;
+
+ switch (res[i].u.dma_chan.dir) {
+ case PCI_EPC_AUX_DMA_EP_TO_RC:
+ if (hw_ch >= EDMA_MAX_WR_CH ||
+ ep_to_rc_aux_chan[hw_ch])
+ return -EINVAL;
+ ep_to_rc_aux_chan[hw_ch] = &res[i];
+ break;
+ case PCI_EPC_AUX_DMA_RC_TO_EP:
+ if (hw_ch >= EDMA_MAX_RD_CH ||
+ rc_to_ep_aux_chan[hw_ch])
+ return -EINVAL;
+ rc_to_ep_aux_chan[hw_ch] = &res[i];
+ break;
+ default:
+ return -EINVAL;
+ }
+ break;
+ }
+ case PCI_EPC_AUX_DMA_DESC_MEM:
+ if (pci_epf_dma_find_desc_mem(res, i,
+ res[i].u.dma_desc.id))
+ return -EINVAL;
+ break;
+ default:
+ continue;
+ }
+ }
+
+ if (!ctrl)
+ return -ENODEV;
+
+ if (!epf_dma->wr_chans && !epf_dma->rd_chans) {
+ dev_err(dev, "no DMA channels requested\n");
+ return -EINVAL;
+ }
+
+ if (epf_dma->wr_chans > ctrl->u.dma_ctrl.ep_to_rc_ch_cnt ||
+ epf_dma->rd_chans > ctrl->u.dma_ctrl.rc_to_ep_ch_cnt)
+ return -EINVAL;
+
+ switch (ctrl->u.dma_ctrl.reg_layout) {
+ case PCI_EPC_AUX_DMA_REG_LAYOUT_DW_EDMA:
+ ret = pci_epf_dma_validate_dw_edma_ctrl(epf_dma, ctrl);
+ if (ret)
+ return ret;
+ epf_dma->reg_layout = PCI_EP_DMA_METADATA_REG_LAYOUT_DW_EDMA;
+ epf_dma->reg_layout_data = ctrl->u.dma_ctrl.reg_layout_data;
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ for (i = 0; i < epf_dma->wr_chans; i++) {
+ if (!ep_to_rc_aux_chan[i]) {
+ dev_err(dev, "missing dense write DMA channel %d\n", i);
+ return -EINVAL;
+ }
+ }
+
+ for (i = 0; i < epf_dma->rd_chans; i++) {
+ if (!rc_to_ep_aux_chan[i]) {
+ dev_err(dev, "missing dense read DMA channel %d\n", i);
+ return -EINVAL;
+ }
+ }
+
+ for (i = 0; i < epf_dma->wr_chans; i++) {
+ u16 desc_mem_id = ep_to_rc_aux_chan[i]->u.dma_chan.desc_mem_id;
+
+ ep_to_rc_desc[i] = pci_epf_dma_find_desc_mem(res, count, desc_mem_id);
+ if (!ep_to_rc_desc[i]) {
+ dev_err(dev, "missing write DMA descriptor memory %d\n", i);
+ return -EINVAL;
+ }
+ }
+
+ for (i = 0; i < epf_dma->rd_chans; i++) {
+ u16 desc_mem_id = rc_to_ep_aux_chan[i]->u.dma_chan.desc_mem_id;
+
+ rc_to_ep_desc[i] = pci_epf_dma_find_desc_mem(res, count, desc_mem_id);
+ if (!rc_to_ep_desc[i]) {
+ dev_err(dev, "missing read DMA descriptor memory %d\n", i);
+ return -EINVAL;
+ }
+ }
+
+ for (i = 0; i < epf_dma->wr_chans; i++) {
+ ret = pci_epf_dma_claim_channel(epf_dma, ep_to_rc_aux_chan[i],
+ &epf_dma->ep_to_rc_chan[i]);
+ if (ret)
+ goto err_release_channels;
+ }
+
+ for (i = 0; i < epf_dma->rd_chans; i++) {
+ ret = pci_epf_dma_claim_channel(epf_dma, rc_to_ep_aux_chan[i],
+ &epf_dma->rc_to_ep_chan[i]);
+ if (ret)
+ goto err_release_channels;
+ }
+
+ epf_dma->resources = no_free_ptr(res);
+ epf_dma->num_resources = count;
+ epf_dma->ctrl = ctrl;
+ memcpy(epf_dma->ep_to_rc_aux_chan, ep_to_rc_aux_chan,
+ sizeof(ep_to_rc_aux_chan));
+ memcpy(epf_dma->rc_to_ep_aux_chan, rc_to_ep_aux_chan,
+ sizeof(rc_to_ep_aux_chan));
+ memcpy(epf_dma->ep_to_rc_desc, ep_to_rc_desc, sizeof(ep_to_rc_desc));
+ memcpy(epf_dma->rc_to_ep_desc, rc_to_ep_desc, sizeof(rc_to_ep_desc));
+
+ return 0;
+
+err_release_channels:
+ pci_epf_dma_release_channels(epf_dma);
+
+ return ret;
+}
+
+static void pci_epf_dma_metadata_write(__le32 *metadata, u16 metadata_off,
+ u32 val)
+{
+ metadata[metadata_off / sizeof(*metadata)] = cpu_to_le32(val);
+}
+
+static void pci_epf_dma_metadata_write64(__le32 *metadata, u16 metadata_off,
+ u64 val)
+{
+ pci_epf_dma_metadata_write(metadata, metadata_off, lower_32_bits(val));
+ pci_epf_dma_metadata_write(metadata, metadata_off + sizeof(u32),
+ upper_32_bits(val));
+}
+
+static int pci_epf_dma_build_ch_entry(const struct pci_epc_aux_resource *chan,
+ const struct pci_epf_dma_bar_map *map,
+ __le32 *metadata, u16 entry)
+{
+ const struct pci_epc_aux_resource *res = map->res;
+ u32 ctrl;
+
+ if (res->size > U32_MAX)
+ return -EOVERFLOW;
+
+ ctrl = FIELD_PREP(PCI_EP_DMA_METADATA_CH_CTRL_HW_CH,
+ chan->u.dma_chan.hw_ch) |
+ FIELD_PREP(PCI_EP_DMA_METADATA_CH_CTRL_DESC_BAR, map->bar);
+
+ pci_epf_dma_metadata_write(metadata, entry + PCI_EP_DMA_METADATA_CH_CTRL,
+ ctrl);
+ pci_epf_dma_metadata_write64(metadata,
+ entry + PCI_EP_DMA_METADATA_CH_DESC_OFF_LO,
+ map->res_offset_in_bar);
+ pci_epf_dma_metadata_write(metadata,
+ entry + PCI_EP_DMA_METADATA_CH_DESC_SIZE,
+ (u32)res->size);
+ pci_epf_dma_metadata_write64(metadata,
+ entry + PCI_EP_DMA_METADATA_CH_DESC_ADDR_LO,
+ res->phys_addr);
+
+ return 0;
+}
+
+static void pci_epf_dma_set_metadata_ready(struct pci_epf_dma *epf_dma,
+ bool ready)
+{
+ __le32 *metadata = epf_dma->metadata_addr;
+ __le32 *ctrl_ptr;
+ u32 ctrl;
+
+ if (!metadata)
+ return;
+
+ ctrl_ptr = &metadata[PCI_EP_DMA_METADATA_CTRL / sizeof(*metadata)];
+ ctrl = le32_to_cpu(READ_ONCE(*ctrl_ptr));
+ if (ready)
+ ctrl |= PCI_EP_DMA_METADATA_CTRL_READY;
+ else
+ ctrl &= ~PCI_EP_DMA_METADATA_CTRL_READY;
+ if (ready)
+ dma_wmb();
+ WRITE_ONCE(*ctrl_ptr, cpu_to_le32(ctrl));
+ if (ready)
+ epf_dma->channels_exposed = true;
+}
+
+static bool pci_epf_dma_metadata_host_requested(struct pci_epf_dma *epf_dma)
+{
+ __le32 *metadata = epf_dma->metadata_addr;
+ u32 ctrl;
+
+ if (!metadata)
+ return false;
+
+ ctrl = le32_to_cpu(READ_ONCE(metadata[PCI_EP_DMA_METADATA_CTRL /
+ sizeof(*metadata)]));
+
+ return ctrl & PCI_EP_DMA_METADATA_CTRL_HOST_REQ;
+}
+
+static void pci_epf_dma_clear_metadata_status(struct pci_epf_dma *epf_dma)
+{
+ __le32 *metadata = epf_dma->metadata_addr;
+ __le32 *ctrl_ptr;
+ u32 ctrl;
+
+ if (!metadata)
+ return;
+
+ ctrl_ptr = &metadata[PCI_EP_DMA_METADATA_CTRL / sizeof(*metadata)];
+ ctrl = le32_to_cpu(READ_ONCE(*ctrl_ptr));
+ ctrl &= ~(PCI_EP_DMA_METADATA_CTRL_HOST_REQ |
+ PCI_EP_DMA_METADATA_CTRL_READY);
+ WRITE_ONCE(*ctrl_ptr, cpu_to_le32(ctrl));
+}
+
+static int pci_epf_dma_build_metadata(struct pci_epf_dma *epf_dma)
+{
+ const struct pci_epf_dma_bar_map *ctrl_map;
+ u16 entry_size = PCI_EP_DMA_METADATA_CH_ENTRY_SIZE;
+ u16 wr_table, rd_table, total_len;
+ __le32 *metadata = epf_dma->metadata_addr;
+ unsigned int i;
+ int ret;
+
+ if (!metadata)
+ return -EINVAL;
+
+ ctrl_map = pci_epf_dma_find_map(epf_dma, epf_dma->ctrl);
+ if (!ctrl_map)
+ return -EINVAL;
+ if (epf_dma->wr_chans > FIELD_MAX(PCI_EP_DMA_METADATA_CTRL_WR_CH_COUNT) ||
+ epf_dma->rd_chans > FIELD_MAX(PCI_EP_DMA_METADATA_CTRL_RD_CH_COUNT) ||
+ entry_size > FIELD_MAX(PCI_EP_DMA_METADATA_CTRL_CH_ENTRY_SIZE) ||
+ ctrl_map->res->size > U32_MAX)
+ return -EOVERFLOW;
+
+ wr_table = epf_dma->wr_chans ? PCI_EP_DMA_METADATA_HDR_LEN : 0;
+ rd_table = epf_dma->rd_chans ?
+ PCI_EP_DMA_METADATA_HDR_LEN + epf_dma->wr_chans * entry_size : 0;
+ total_len = PCI_EP_DMA_METADATA_HDR_LEN +
+ (epf_dma->wr_chans + epf_dma->rd_chans) * entry_size;
+
+ memset(metadata, 0, total_len);
+
+ pci_epf_dma_metadata_write(metadata, 0, PCI_EP_DMA_METADATA_MAGIC);
+ pci_epf_dma_metadata_write(metadata, PCI_EP_DMA_METADATA_HDR,
+ FIELD_PREP(PCI_EP_DMA_METADATA_HDR_REV,
+ PCI_EP_DMA_METADATA_REV) |
+ FIELD_PREP(PCI_EP_DMA_METADATA_HDR_LEN_FIELD,
+ total_len));
+ pci_epf_dma_metadata_write(metadata, PCI_EP_DMA_METADATA_CTRL,
+ FIELD_PREP(PCI_EP_DMA_METADATA_CTRL_REG_BAR,
+ ctrl_map->bar) |
+ FIELD_PREP(PCI_EP_DMA_METADATA_CTRL_WR_CH_COUNT,
+ epf_dma->wr_chans) |
+ FIELD_PREP(PCI_EP_DMA_METADATA_CTRL_RD_CH_COUNT,
+ epf_dma->rd_chans) |
+ FIELD_PREP(PCI_EP_DMA_METADATA_CTRL_CH_ENTRY_SIZE,
+ entry_size));
+ pci_epf_dma_metadata_write64(metadata,
+ PCI_EP_DMA_METADATA_REG_OFF_LO,
+ ctrl_map->res_offset_in_bar);
+ pci_epf_dma_metadata_write(metadata, PCI_EP_DMA_METADATA_REG_LAYOUT,
+ FIELD_PREP(PCI_EP_DMA_METADATA_REG_LAYOUT_ID,
+ epf_dma->reg_layout) |
+ FIELD_PREP(PCI_EP_DMA_METADATA_REG_LAYOUT_DATA,
+ epf_dma->reg_layout_data));
+ pci_epf_dma_metadata_write(metadata, PCI_EP_DMA_METADATA_REG_SIZE,
+ (u32)ctrl_map->res->size);
+
+ for (i = 0; i < epf_dma->wr_chans; i++) {
+ const struct pci_epf_dma_bar_map *map;
+
+ map = pci_epf_dma_find_map(epf_dma,
+ epf_dma->ep_to_rc_desc[i]);
+ if (!map)
+ return -EINVAL;
+ ret = pci_epf_dma_build_ch_entry(epf_dma->ep_to_rc_aux_chan[i],
+ map, metadata,
+ wr_table + i * entry_size);
+ if (ret)
+ return ret;
+ }
+
+ for (i = 0; i < epf_dma->rd_chans; i++) {
+ const struct pci_epf_dma_bar_map *map;
+
+ map = pci_epf_dma_find_map(epf_dma,
+ epf_dma->rc_to_ep_desc[i]);
+ if (!map)
+ return -EINVAL;
+ ret = pci_epf_dma_build_ch_entry(epf_dma->rc_to_ep_aux_chan[i],
+ map, metadata,
+ rd_table + i * entry_size);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int pci_epf_dma_reserve_msix(struct pci_epf_dma *epf_dma,
+ const struct pci_epc_features *epc_features,
+ size_t *backing_size)
+{
+ struct pci_epf *epf = epf_dma->epf;
+ size_t msix_table_size, pba_size, next;
+ unsigned int nvec = epf->msix_interrupts;
+
+ epf_dma->msix_table_offset = 0;
+
+ if (!epc_features->msix_capable || !nvec)
+ return 0;
+
+ next = ALIGN(*backing_size, 8);
+ if (next > U32_MAX)
+ return -EOVERFLOW;
+ epf_dma->msix_table_offset = next;
+
+ if (check_mul_overflow(PCI_MSIX_ENTRY_SIZE, nvec, &msix_table_size))
+ return -EOVERFLOW;
+
+ pba_size = ALIGN(DIV_ROUND_UP(nvec, 8), 8);
+ if (check_add_overflow(next, msix_table_size, &next) ||
+ next > U32_MAX ||
+ check_add_overflow(next, pba_size, &next))
+ return -EOVERFLOW;
+
+ *backing_size = next;
+
+ return 0;
+}
+
+static int pci_epf_dma_build_layout(struct pci_epf_dma *epf_dma,
+ const struct pci_epc_features *epc_features)
+{
+ struct pci_epf *epf = epf_dma->epf;
+ struct device *dev = &epf->dev;
+ struct pci_epf_bar *bar;
+ unsigned int max_maps, map_idx = 0, sub_idx = 0;
+ size_t align = epc_features->align;
+ size_t metadata_size, metadata_backing_size, metadata_bar_size;
+ size_t mapped_size = 0, dma_window_bar_size;
+ int i, ret;
+
+ metadata_size = PCI_EP_DMA_METADATA_HDR_LEN;
+ metadata_size += (epf_dma->wr_chans + epf_dma->rd_chans) *
+ PCI_EP_DMA_METADATA_CH_ENTRY_SIZE;
+ metadata_backing_size = metadata_size;
+ ret = pci_epf_dma_reserve_msix(epf_dma, epc_features,
+ &metadata_backing_size);
+ if (ret)
+ return ret;
+ metadata_bar_size = pci_epf_dma_align_size(metadata_backing_size,
+ align);
+
+ epf_dma->metadata_addr = pci_epf_alloc_space(epf, metadata_bar_size,
+ epf_dma->metadata_bar,
+ epc_features,
+ PRIMARY_INTERFACE);
+ if (!epf_dma->metadata_addr) {
+ dev_err(dev, "failed to allocate BAR%d metadata space\n",
+ epf_dma->metadata_bar);
+ return -ENOMEM;
+ }
+ memset(epf_dma->metadata_addr, 0, epf->bar[epf_dma->metadata_bar].size);
+
+ /* One map for DMA controller registers, plus one per channel. */
+ max_maps = 1 + epf_dma->wr_chans + epf_dma->rd_chans;
+ epf_dma->bar_maps = kzalloc_objs(*epf_dma->bar_maps, max_maps);
+ if (!epf_dma->bar_maps)
+ return -ENOMEM;
+
+ ret = pci_epf_dma_add_map(epf_dma, epf_dma->ctrl, align,
+ &mapped_size, &map_idx);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < epf_dma->wr_chans; i++) {
+ ret = pci_epf_dma_add_map(epf_dma,
+ epf_dma->ep_to_rc_desc[i], align,
+ &mapped_size, &map_idx);
+ if (ret)
+ return ret;
+ }
+
+ for (i = 0; i < epf_dma->rd_chans; i++) {
+ ret = pci_epf_dma_add_map(epf_dma,
+ epf_dma->rc_to_ep_desc[i], align,
+ &mapped_size, &map_idx);
+ if (ret)
+ return ret;
+ }
+
+ epf_dma->num_bar_maps = map_idx;
+
+ ret = pci_epf_dma_build_metadata(epf_dma);
+ if (ret)
+ return ret;
+
+ /* Some DMA resources may already be visible through another map. */
+ for (i = 0; i < epf_dma->num_bar_maps; i++) {
+ if (epf_dma->bar_maps[i].needs_submap)
+ epf_dma->num_submaps++;
+ }
+ if (!epf_dma->num_submaps)
+ return 0;
+
+ ret = pci_epf_dma_map_submaps(epf_dma);
+ if (ret)
+ return ret;
+
+ dma_window_bar_size = mapped_size;
+ epf_dma->dma_window_addr =
+ pci_epf_alloc_space(epf, dma_window_bar_size,
+ epf_dma->dma_window_bar, epc_features,
+ PRIMARY_INTERFACE);
+ if (!epf_dma->dma_window_addr) {
+ dev_err(dev, "failed to allocate BAR%d DMA window space\n",
+ epf_dma->dma_window_bar);
+ return -ENOMEM;
+ }
+ bar = &epf->bar[epf_dma->dma_window_bar];
+ memset(epf_dma->dma_window_addr, 0, bar->size);
+
+ if (bar->size > mapped_size)
+ epf_dma->num_submaps++;
+
+ epf_dma->submaps = kzalloc_objs(*epf_dma->submaps, epf_dma->num_submaps);
+ if (!epf_dma->submaps)
+ return -ENOMEM;
+
+ for (i = 0; i < epf_dma->num_bar_maps; i++) {
+ struct pci_epf_dma_bar_map *map = &epf_dma->bar_maps[i];
+
+ if (!map->needs_submap)
+ continue;
+
+ epf_dma->submaps[sub_idx++] = (struct pci_epf_bar_submap) {
+ .phys_addr = pci_epf_dma_submap_addr(map),
+ .size = map->map_size,
+ };
+ }
+
+ /* Cover any BAR tail padding with the allocated scratch space. */
+ if (bar->size > mapped_size) {
+ epf_dma->submaps[sub_idx++] = (struct pci_epf_bar_submap) {
+ .phys_addr = bar->phys_addr + mapped_size,
+ .size = bar->size - mapped_size,
+ };
+ }
+
+ return 0;
+}
+
+static void pci_epf_dma_free_layout(struct pci_epf_dma *epf_dma)
+{
+ struct pci_epf *epf = epf_dma->epf;
+ struct pci_epf_bar *bar;
+
+ pci_epf_dma_release_channels(epf_dma);
+
+ if (epf_dma->dma_window_addr) {
+ bar = &epf->bar[epf_dma->dma_window_bar];
+ bar->submap = NULL;
+ bar->num_submap = 0;
+ }
+ epf_dma->submaps_programmed = false;
+
+ kfree(epf_dma->submaps);
+ epf_dma->submaps = NULL;
+ epf_dma->num_submaps = 0;
+
+ pci_epf_dma_unmap_submaps(epf_dma);
+
+ kfree(epf_dma->bar_maps);
+ epf_dma->bar_maps = NULL;
+ epf_dma->num_bar_maps = 0;
+
+ kfree(epf_dma->resources);
+ epf_dma->resources = NULL;
+ epf_dma->num_resources = 0;
+ epf_dma->ctrl = NULL;
+ memset(epf_dma->ep_to_rc_aux_chan, 0, sizeof(epf_dma->ep_to_rc_aux_chan));
+ memset(epf_dma->rc_to_ep_aux_chan, 0, sizeof(epf_dma->rc_to_ep_aux_chan));
+ memset(epf_dma->ep_to_rc_desc, 0, sizeof(epf_dma->ep_to_rc_desc));
+ memset(epf_dma->rc_to_ep_desc, 0, sizeof(epf_dma->rc_to_ep_desc));
+
+ if (epf_dma->dma_window_addr) {
+ pci_epf_free_space(epf, epf_dma->dma_window_addr,
+ epf_dma->dma_window_bar,
+ PRIMARY_INTERFACE);
+ epf_dma->dma_window_addr = NULL;
+ }
+
+ if (epf_dma->metadata_addr) {
+ pci_epf_free_space(epf, epf_dma->metadata_addr,
+ epf_dma->metadata_bar,
+ PRIMARY_INTERFACE);
+ epf_dma->metadata_addr = NULL;
+ }
+ epf_dma->msix_table_offset = 0;
+}
+
+static int pci_epf_dma_program_submaps(struct pci_epf_dma *epf_dma)
+{
+ struct pci_epf *epf = epf_dma->epf;
+ struct pci_epf_bar *bar;
+ int ret;
+
+ if (!epf_dma->dma_window_addr) {
+ pci_epf_dma_set_metadata_ready(epf_dma, true);
+ return 0;
+ }
+
+ if (epf_dma->submaps_programmed)
+ return 0;
+
+ bar = &epf->bar[epf_dma->dma_window_bar];
+ bar->submap = epf_dma->submaps;
+ bar->num_submap = epf_dma->num_submaps;
+
+ ret = pci_epc_set_bar(epf->epc, epf->func_no, epf->vfunc_no, bar);
+ if (ret) {
+ bar->submap = NULL;
+ bar->num_submap = 0;
+ return ret;
+ }
+
+ epf_dma->submaps_programmed = true;
+ pci_epf_dma_set_metadata_ready(epf_dma, true);
+
+ return 0;
+}
+
+static void pci_epf_dma_map_work(struct work_struct *work)
+{
+ struct pci_epf_dma *epf_dma =
+ container_of(to_delayed_work(work), struct pci_epf_dma,
+ map_work);
+ struct pci_epf *epf = epf_dma->epf;
+ int ret;
+
+ if (!epf->epc)
+ return;
+
+ if (!epf->epc->init_complete) {
+ schedule_delayed_work(&epf_dma->map_work,
+ msecs_to_jiffies(PCI_EPF_DMA_HOST_REQ_POLL_MS));
+ return;
+ }
+
+ if (!pci_epf_dma_metadata_host_requested(epf_dma)) {
+ schedule_delayed_work(&epf_dma->map_work,
+ msecs_to_jiffies(PCI_EPF_DMA_HOST_REQ_POLL_MS));
+ return;
+ }
+
+ ret = pci_epf_dma_program_submaps(epf_dma);
+ if (ret) {
+ /*
+ * Do not retry a failed submap update. It usually means the BAR
+ * assignment or EPC/iATU state is unusable for this host request.
+ */
+ dev_err(&epf->dev, "failed to program DMA window BAR submaps: %d\n",
+ ret);
+ }
+}
+
+static int pci_epf_dma_epc_init(struct pci_epf *epf)
+{
+ struct pci_epf_dma *epf_dma = epf_get_drvdata(epf);
+ const struct pci_epc_features *epc_features;
+ struct pci_epc *epc = epf->epc;
+ struct device *dev = &epf->dev;
+ int ret;
+
+ epc_features = pci_epc_get_features(epc, epf->func_no, epf->vfunc_no);
+ if (!epc_features)
+ return -EOPNOTSUPP;
+
+ pci_epf_dma_clear_metadata_status(epf_dma);
+
+ ret = pci_epc_write_header(epc, epf->func_no, epf->vfunc_no,
+ epf->header);
+ if (ret) {
+ dev_err(dev, "configuration header write failed\n");
+ return ret;
+ }
+
+ ret = pci_epc_set_bar(epc, epf->func_no, epf->vfunc_no,
+ &epf->bar[epf_dma->metadata_bar]);
+ if (ret) {
+ dev_err(dev, "BAR%d setup failed: %d\n",
+ epf_dma->metadata_bar, ret);
+ return ret;
+ }
+
+ if (epf_dma->dma_window_addr) {
+ ret = pci_epc_set_bar(epc, epf->func_no, epf->vfunc_no,
+ &epf->bar[epf_dma->dma_window_bar]);
+ if (ret) {
+ dev_err(dev, "BAR%d setup failed: %d\n",
+ epf_dma->dma_window_bar, ret);
+ goto err_clear_metadata_bar;
+ }
+ }
+
+ if (epc_features->msi_capable && epf->msi_interrupts) {
+ ret = pci_epc_set_msi(epc, epf->func_no, epf->vfunc_no,
+ epf->msi_interrupts);
+ if (ret) {
+ dev_err(dev, "MSI setup failed: %d\n", ret);
+ goto err_clear_dma_window_bar;
+ }
+ }
+
+ if (epc_features->msix_capable && epf->msix_interrupts) {
+ ret = pci_epc_set_msix(epc, epf->func_no, epf->vfunc_no,
+ epf->msix_interrupts,
+ epf_dma->metadata_bar,
+ epf_dma->msix_table_offset);
+ if (ret) {
+ dev_err(dev, "MSI-X setup failed: %d\n", ret);
+ goto err_clear_dma_window_bar;
+ }
+ }
+
+ schedule_delayed_work(&epf_dma->map_work, 0);
+
+ return 0;
+
+err_clear_dma_window_bar:
+ if (epf_dma->dma_window_addr)
+ pci_epc_clear_bar(epc, epf->func_no, epf->vfunc_no,
+ &epf->bar[epf_dma->dma_window_bar]);
+err_clear_metadata_bar:
+ pci_epc_clear_bar(epc, epf->func_no, epf->vfunc_no,
+ &epf->bar[epf_dma->metadata_bar]);
+ pci_epf_dma_clear_metadata_status(epf_dma);
+
+ return ret;
+}
+
+static void pci_epf_dma_epc_deinit(struct pci_epf *epf)
+{
+ struct pci_epf_dma *epf_dma = epf_get_drvdata(epf);
+ struct pci_epf_bar *bar;
+
+ cancel_delayed_work_sync(&epf_dma->map_work);
+
+ if (!epf_dma->metadata_addr)
+ return;
+
+ pci_epf_dma_clear_metadata_status(epf_dma);
+ if (epf_dma->dma_window_addr) {
+ bar = &epf->bar[epf_dma->dma_window_bar];
+ pci_epc_clear_bar(epf->epc, epf->func_no, epf->vfunc_no, bar);
+ bar->submap = NULL;
+ bar->num_submap = 0;
+ }
+ pci_epc_clear_bar(epf->epc, epf->func_no, epf->vfunc_no,
+ &epf->bar[epf_dma->metadata_bar]);
+ epf_dma->submaps_programmed = false;
+}
+
+static int pci_epf_dma_link_up(struct pci_epf *epf)
+{
+ struct pci_epf_dma *epf_dma = epf_get_drvdata(epf);
+
+ schedule_delayed_work(&epf_dma->map_work, 0);
+
+ return 0;
+}
+
+static int pci_epf_dma_link_down(struct pci_epf *epf)
+{
+ struct pci_epf_dma *epf_dma = epf_get_drvdata(epf);
+
+ cancel_delayed_work_sync(&epf_dma->map_work);
+ pci_epf_dma_set_metadata_ready(epf_dma, false);
+ /*
+ * Link down can invalidate non-sticky inbound ATU state without going
+ * through pci_epc_clear_bar(). Keep the BAR/submap description intact,
+ * but force the next link-up path to reprogram the subrange mappings
+ * for any still-pending host request.
+ */
+ epf_dma->submaps_programmed = false;
+
+ return 0;
+}
+
+static const struct pci_epc_event_ops pci_epf_dma_event_ops = {
+ .epc_init = pci_epf_dma_epc_init,
+ .epc_deinit = pci_epf_dma_epc_deinit,
+ .link_up = pci_epf_dma_link_up,
+ .link_down = pci_epf_dma_link_down,
+};
+
+static int pci_epf_dma_bind(struct pci_epf *epf)
+{
+ struct pci_epf_dma *epf_dma = epf_get_drvdata(epf);
+ const struct pci_epc_features *epc_features;
+ struct pci_epc *epc = epf->epc;
+ bool needs_dma_window;
+ int ret;
+
+ if (WARN_ON_ONCE(!epc))
+ return -EINVAL;
+
+ epc_features = pci_epc_get_features(epc, epf->func_no, epf->vfunc_no);
+ if (!epc_features)
+ return -EOPNOTSUPP;
+
+ if (!epc_features->msi_capable && !epc_features->msix_capable)
+ return -EOPNOTSUPP;
+
+ if ((!epc_features->msi_capable || !epf->msi_interrupts) &&
+ (!epc_features->msix_capable || !epf->msix_interrupts))
+ return -EINVAL;
+
+ ret = pci_epf_dma_collect_resources(epf_dma);
+ if (ret)
+ return ret;
+
+ if (epf_dma->metadata_bar == NO_BAR)
+ epf_dma->metadata_bar =
+ pci_epf_dma_first_usable_bar(epf_dma, epc_features,
+ NO_BAR);
+
+ if (epf_dma->metadata_bar == NO_BAR ||
+ !pci_epf_dma_bar_usable(epc_features, epf_dma->metadata_bar) ||
+ pci_epf_dma_bar_has_fixed_resource(epf_dma, epf_dma->metadata_bar)) {
+ ret = -EINVAL;
+ goto err_free;
+ }
+
+ needs_dma_window = pci_epf_dma_needs_dma_window(epf_dma);
+ if (needs_dma_window) {
+ if (!epc_features->subrange_mapping ||
+ !epc_features->dynamic_inbound_mapping) {
+ ret = -EOPNOTSUPP;
+ goto err_free;
+ }
+
+ if (epf_dma->dma_window_bar == NO_BAR)
+ epf_dma->dma_window_bar =
+ pci_epf_dma_first_usable_bar(epf_dma, epc_features,
+ epf_dma->metadata_bar);
+ if (epf_dma->dma_window_bar == NO_BAR) {
+ ret = -EOPNOTSUPP;
+ goto err_free;
+ }
+ }
+
+ if (epf_dma->dma_window_bar != NO_BAR) {
+ if (!pci_epf_dma_bar_usable(epc_features,
+ epf_dma->dma_window_bar)) {
+ ret = -EINVAL;
+ goto err_free;
+ }
+ if (epf_dma->metadata_bar == epf_dma->dma_window_bar ||
+ pci_epf_dma_bar_has_fixed_resource(epf_dma,
+ epf_dma->dma_window_bar)) {
+ ret = -EINVAL;
+ goto err_free;
+ }
+ }
+
+ ret = pci_epf_dma_build_layout(epf_dma, epc_features);
+ if (ret)
+ goto err_free;
+
+ return 0;
+
+err_free:
+ pci_epf_dma_free_layout(epf_dma);
+
+ return ret;
+}
+
+static void pci_epf_dma_unbind(struct pci_epf *epf)
+{
+ struct pci_epf_dma *epf_dma = epf_get_drvdata(epf);
+
+ cancel_delayed_work_sync(&epf_dma->map_work);
+ if (epf->epc && epf->epc->init_complete)
+ pci_epf_dma_epc_deinit(epf);
+ pci_epf_dma_free_layout(epf_dma);
+}
+
+#define PCI_EPF_DMA_SHOW(_name, _fmt, _val) \
+static ssize_t pci_epf_dma_##_name##_show(struct config_item *item, \
+ char *page) \
+{ \
+ struct config_group *group = to_config_group(item); \
+ struct pci_epf_dma *epf_dma = to_epf_dma(group); \
+ \
+ return sysfs_emit(page, _fmt "\n", (_val)); \
+}
+
+PCI_EPF_DMA_SHOW(metadata_bar, "%d", (int)epf_dma->metadata_bar)
+PCI_EPF_DMA_SHOW(dma_window_bar, "%d", (int)epf_dma->dma_window_bar)
+
+static ssize_t pci_epf_dma_metadata_bar_store(struct config_item *item, const char *page,
+ size_t len)
+{
+ struct config_group *group = to_config_group(item);
+ struct pci_epf_dma *epf_dma = to_epf_dma(group);
+ int bar, ret;
+
+ if (epf_dma->epf->epc)
+ return -EOPNOTSUPP;
+
+ ret = kstrtoint(page, 0, &bar);
+ if (ret)
+ return ret;
+
+ if (bar != NO_BAR && (bar < BAR_0 || bar >= PCI_STD_NUM_BARS))
+ return -EINVAL;
+ if (bar != NO_BAR && bar == epf_dma->dma_window_bar)
+ return -EINVAL;
+
+ epf_dma->metadata_bar = bar;
+
+ return len;
+}
+
+static ssize_t pci_epf_dma_dma_window_bar_store(struct config_item *item,
+ const char *page, size_t len)
+{
+ struct config_group *group = to_config_group(item);
+ struct pci_epf_dma *epf_dma = to_epf_dma(group);
+ int bar, ret;
+
+ if (epf_dma->epf->epc)
+ return -EOPNOTSUPP;
+
+ ret = kstrtoint(page, 0, &bar);
+ if (ret)
+ return ret;
+
+ if (bar != NO_BAR && (bar < BAR_0 || bar >= PCI_STD_NUM_BARS))
+ return -EINVAL;
+ if (bar != NO_BAR && bar == epf_dma->metadata_bar)
+ return -EINVAL;
+
+ epf_dma->dma_window_bar = bar;
+
+ return len;
+}
+
+PCI_EPF_DMA_SHOW(wr_chans, "%u", (unsigned int)epf_dma->wr_chans)
+
+static ssize_t pci_epf_dma_wr_chans_store(struct config_item *item,
+ const char *page, size_t len)
+{
+ struct config_group *group = to_config_group(item);
+ struct pci_epf_dma *epf_dma = to_epf_dma(group);
+ u16 val;
+ int ret;
+
+ if (epf_dma->epf->epc)
+ return -EOPNOTSUPP;
+
+ ret = kstrtou16(page, 0, &val);
+ if (ret)
+ return ret;
+ if (val > EDMA_MAX_WR_CH)
+ return -EINVAL;
+
+ epf_dma->wr_chans = val;
+
+ return len;
+}
+
+PCI_EPF_DMA_SHOW(rd_chans, "%u", (unsigned int)epf_dma->rd_chans)
+
+static ssize_t pci_epf_dma_rd_chans_store(struct config_item *item,
+ const char *page, size_t len)
+{
+ struct config_group *group = to_config_group(item);
+ struct pci_epf_dma *epf_dma = to_epf_dma(group);
+ u16 val;
+ int ret;
+
+ if (epf_dma->epf->epc)
+ return -EOPNOTSUPP;
+
+ ret = kstrtou16(page, 0, &val);
+ if (ret)
+ return ret;
+ if (val > EDMA_MAX_RD_CH)
+ return -EINVAL;
+
+ epf_dma->rd_chans = val;
+
+ return len;
+}
+
+CONFIGFS_ATTR(pci_epf_dma_, metadata_bar);
+CONFIGFS_ATTR(pci_epf_dma_, dma_window_bar);
+CONFIGFS_ATTR(pci_epf_dma_, wr_chans);
+CONFIGFS_ATTR(pci_epf_dma_, rd_chans);
+
+static struct configfs_attribute *pci_epf_dma_attrs[] = {
+ &pci_epf_dma_attr_metadata_bar,
+ &pci_epf_dma_attr_dma_window_bar,
+ &pci_epf_dma_attr_wr_chans,
+ &pci_epf_dma_attr_rd_chans,
+ NULL,
+};
+
+static const struct config_item_type pci_epf_dma_group_type = {
+ .ct_attrs = pci_epf_dma_attrs,
+ .ct_owner = THIS_MODULE,
+};
+
+static struct config_group *pci_epf_dma_add_cfs(struct pci_epf *epf,
+ struct config_group *group)
+{
+ struct pci_epf_dma *epf_dma = epf_get_drvdata(epf);
+ struct config_group *epf_group = &epf_dma->group;
+ struct device *dev = &epf->dev;
+
+ config_group_init_type_name(epf_group, dev_name(dev),
+ &pci_epf_dma_group_type);
+
+ return epf_group;
+}
+
+static const struct pci_epf_device_id pci_epf_dma_ids[] = {
+ {
+ .name = "pci_epf_dma",
+ },
+ {},
+};
+
+static int pci_epf_dma_probe(struct pci_epf *epf,
+ const struct pci_epf_device_id *id)
+{
+ struct pci_epf_dma *epf_dma;
+
+ epf_dma = devm_kzalloc(&epf->dev, sizeof(*epf_dma), GFP_KERNEL);
+ if (!epf_dma)
+ return -ENOMEM;
+
+ epf->header = &pci_epf_dma_header;
+ epf->event_ops = &pci_epf_dma_event_ops;
+
+ epf_dma->epf = epf;
+ epf_dma->metadata_bar = NO_BAR;
+ epf_dma->dma_window_bar = NO_BAR;
+ INIT_DELAYED_WORK(&epf_dma->map_work, pci_epf_dma_map_work);
+
+ epf_set_drvdata(epf, epf_dma);
+
+ return 0;
+}
+
+static const struct pci_epf_ops pci_epf_dma_ops = {
+ .unbind = pci_epf_dma_unbind,
+ .bind = pci_epf_dma_bind,
+ .add_cfs = pci_epf_dma_add_cfs,
+};
+
+static struct pci_epf_driver pci_epf_dma_driver = {
+ .driver.name = "pci_epf_dma",
+ .probe = pci_epf_dma_probe,
+ .id_table = pci_epf_dma_ids,
+ .ops = &pci_epf_dma_ops,
+ .owner = THIS_MODULE,
+};
+
+static int __init pci_epf_dma_init(void)
+{
+ return pci_epf_register_driver(&pci_epf_dma_driver);
+}
+module_init(pci_epf_dma_init);
+
+static void __exit pci_epf_dma_exit(void)
+{
+ pci_epf_unregister_driver(&pci_epf_dma_driver);
+}
+module_exit(pci_epf_dma_exit);
+
+MODULE_DESCRIPTION("PCI EPF DMA DRIVER");
+MODULE_AUTHOR("Koichiro Den <den@valinux.co.jp>");
+MODULE_LICENSE("GPL");
--
2.51.0
^ permalink raw reply related
* [PATCH v5 1/3] dmaengine: dw-edma-pcie: Discover endpoint DMA metadata
From: Koichiro Den @ 2026-07-17 5:09 UTC (permalink / raw)
To: Manivannan Sadhasivam, Krzysztof Wilczyński,
Kishon Vijay Abraham I, Bjorn Helgaas, Jonathan Corbet,
Shuah Khan, Vinod Koul, Frank Li, Arnd Bergmann, Damien Le Moal,
Niklas Cassel
Cc: Marek Vasut, Yoshihiro Shimoda, linux-pci, linux-doc,
linux-kernel, dmaengine
In-Reply-To: <20260717050953.2145851-1-den@valinux.co.jp>
Teach dw-edma-pcie to discover a PCI endpoint DMA function from
BAR-resident metadata. The metadata supplies the DMA register window,
channel counts, descriptor windows, optional auxiliary windows, and
endpoint-local descriptor and auxiliary addresses. Accept DesignWare
eDMA unroll, HDMA compatible, and HDMA native linked-list layouts.
Endpoint-provided DMA channels use raw slave addresses because the host
programs transfers against endpoint physical addresses, not PCI BAR
addresses. The host-side dw-edma-pcie instance is remote-routed by
default, so delegated channels report completions through IMWr/MSI.
Endpoint DMA metadata currently has no static PCI ID. Let an explicit
driver_override bind use the generic endpoint DMA metadata parser, but
do not treat arbitrary dynamic IDs without driver data as endpoint DMA
devices.
The endpoint polls HOST_REQ at a low idle rate before programming DMA
window submaps and setting READY. Let the host wait for several endpoint
poll periods before treating the READY handshake as timed out.
Signed-off-by: Koichiro Den <den@valinux.co.jp>
---
Changes in v5:
- Correct the HOST_REQ error-path description; the bit has been cleared
on errors since v4.
drivers/dma/dw-edma/dw-edma-pcie.c | 401 ++++++++++++++++++++++++++++-
1 file changed, 399 insertions(+), 2 deletions(-)
diff --git a/drivers/dma/dw-edma/dw-edma-pcie.c b/drivers/dma/dw-edma/dw-edma-pcie.c
index 07ea237577d5..fa80e0428287 100644
--- a/drivers/dma/dw-edma/dw-edma-pcie.c
+++ b/drivers/dma/dw-edma/dw-edma-pcie.c
@@ -11,9 +11,13 @@
#include <linux/pci.h>
#include <linux/device.h>
#include <linux/dma/edma.h>
+#include <linux/iopoll.h>
#include <linux/pci-epf.h>
#include <linux/msi.h>
#include <linux/bitfield.h>
+#include <linux/io.h>
+#include <linux/overflow.h>
+#include <linux/pci-ep-dma.h>
#include <linux/sizes.h>
#include "dw-edma-core.h"
@@ -45,6 +49,9 @@
#define DW_PCIE_XILINX_MDB_DT_OFF_GAP 0x100000
#define DW_PCIE_XILINX_MDB_DT_SIZE 0x800
+#define DW_PCIE_EP_DMA_READY_POLL_US 1000
+#define DW_PCIE_EP_DMA_READY_TIMEOUT_US 2000000
+
#define DW_BLOCK(a, b, c) \
{ \
.bar = a, \
@@ -94,6 +101,12 @@ struct dw_edma_pcie_match_data {
#define DW_EDMA_PCIE_F_DEVMEM_PHYS_OFF BIT(0)
#define DW_EDMA_PCIE_F_REG_OFFSET BIT(1)
+struct dw_edma_pcie_ep_dma_view {
+ struct pci_dev *pdev;
+ void __iomem *base;
+ resource_size_t limit;
+};
+
static const struct dw_edma_pcie_data snps_edda_data = {
/* eDMA registers location */
.rg.bar = BAR_0,
@@ -158,6 +171,13 @@ static const struct dw_edma_pcie_data xilinx_cpm6_dma_data = {
.rd_ch_cnt = 8,
};
+static const struct dw_edma_pcie_data ep_dma_data = {
+ .mf = EDMA_MF_EDMA_UNROLL,
+ .irqs = EDMA_MAX_WR_CH + EDMA_MAX_RD_CH,
+ .wr_ch_cnt = EDMA_MAX_WR_CH,
+ .rd_ch_cnt = EDMA_MAX_RD_CH,
+};
+
static void dw_edma_set_chan_region_offset(struct dw_edma_pcie_data *pdata,
enum pci_barno bar, off_t start_off,
off_t ll_off_gap, size_t ll_size,
@@ -227,6 +247,96 @@ static const struct dw_edma_plat_ops dw_edma_pcie_plat_ops = {
.pci_address = dw_edma_pcie_address,
};
+static const struct dw_edma_plat_ops dw_edma_pcie_raw_addr_plat_ops = {
+ .irq_vector = dw_edma_pcie_irq_vector,
+};
+
+static bool dw_edma_pcie_valid_bar(enum pci_barno bar)
+{
+ return bar >= BAR_0 && bar <= BAR_5;
+}
+
+static bool dw_edma_pcie_valid_bar_range(struct pci_dev *pdev,
+ enum pci_barno bar, u64 off,
+ size_t sz)
+{
+ resource_size_t bar_len;
+
+ if (!dw_edma_pcie_valid_bar(bar) || !sz)
+ return false;
+
+ bar_len = pci_resource_len(pdev, bar);
+
+ return off <= bar_len && sz <= bar_len - off;
+}
+
+static bool dw_edma_pcie_valid_block(struct pci_dev *pdev,
+ const struct dw_edma_block *block)
+{
+ return dw_edma_pcie_valid_bar_range(pdev, block->bar, block->off,
+ block->sz);
+}
+
+static bool dw_edma_pcie_ep_dma_bar_scannable(struct pci_dev *pdev,
+ enum pci_barno bar)
+{
+ unsigned long flags = pci_resource_flags(pdev, bar);
+
+ if (!(flags & IORESOURCE_MEM))
+ return false;
+
+ if (flags & (IORESOURCE_UNSET | IORESOURCE_DISABLED))
+ return false;
+
+ return pci_resource_len(pdev, bar) >= PCI_EP_DMA_METADATA_HDR_LEN;
+}
+
+static u32 dw_edma_pcie_ep_dma_readl(struct dw_edma_pcie_ep_dma_view *view,
+ u16 off)
+{
+ return readl(view->base + off);
+}
+
+static void dw_edma_pcie_ep_dma_writel(struct dw_edma_pcie_ep_dma_view *view,
+ u16 off, u32 val)
+{
+ writel(val, view->base + off);
+}
+
+static void
+dw_edma_pcie_ep_dma_clear_host_req(struct dw_edma_pcie_ep_dma_view *view)
+{
+ u32 ctrl;
+
+ ctrl = dw_edma_pcie_ep_dma_readl(view, PCI_EP_DMA_METADATA_CTRL);
+ ctrl &= ~PCI_EP_DMA_METADATA_CTRL_HOST_REQ;
+ dw_edma_pcie_ep_dma_writel(view, PCI_EP_DMA_METADATA_CTRL, ctrl);
+}
+
+static u64 dw_edma_pcie_ep_dma_read64(struct dw_edma_pcie_ep_dma_view *view,
+ u16 lo, u16 hi)
+{
+ u64 val;
+
+ val = dw_edma_pcie_ep_dma_readl(view, hi);
+
+ return (val << 32) | dw_edma_pcie_ep_dma_readl(view, lo);
+}
+
+static int dw_edma_pcie_ep_dma_read_off(struct dw_edma_pcie_ep_dma_view *view,
+ u16 lo, u16 hi, off_t *off)
+{
+ u64 val;
+
+ val = dw_edma_pcie_ep_dma_read64(view, lo, hi);
+ if (val > type_max(*off))
+ return -EINVAL;
+
+ *off = val;
+
+ return 0;
+}
+
static void dw_edma_pcie_get_synopsys_dma_data(struct pci_dev *pdev,
struct dw_edma_pcie_data *pdata)
{
@@ -328,6 +438,273 @@ static void dw_edma_pcie_get_xilinx_dma_data(struct pci_dev *pdev,
pdata->devmem_phys_off = off;
}
+static int
+dw_edma_pcie_parse_ep_dma_ch_table(struct dw_edma_pcie_ep_dma_view *view,
+ struct dw_edma_pcie_data *pdata,
+ u16 table_off, u16 entry_size, u16 ch_cnt,
+ bool write)
+{
+ struct dw_edma_block *desc_blocks = write ? pdata->ll_wr : pdata->ll_rd;
+ struct dw_edma_block *data_blocks = write ? pdata->dt_wr : pdata->dt_rd;
+ u32 ctrl;
+ u16 i;
+ int ret;
+
+ for (i = 0; i < ch_cnt; i++) {
+ struct dw_edma_block *desc_block = &desc_blocks[i];
+ struct dw_edma_block *data_block = &data_blocks[i];
+ u16 off = table_off + i * entry_size;
+ u16 field, lo, hi;
+
+ field = off + PCI_EP_DMA_METADATA_CH_CTRL;
+ ctrl = dw_edma_pcie_ep_dma_readl(view, field);
+ if (FIELD_GET(PCI_EP_DMA_METADATA_CH_CTRL_HW_CH, ctrl) != i)
+ return -EOPNOTSUPP;
+
+ desc_block->bar =
+ FIELD_GET(PCI_EP_DMA_METADATA_CH_CTRL_DESC_BAR, ctrl);
+ lo = off + PCI_EP_DMA_METADATA_CH_DESC_OFF_LO;
+ hi = off + PCI_EP_DMA_METADATA_CH_DESC_OFF_HI;
+ ret = dw_edma_pcie_ep_dma_read_off(view, lo, hi,
+ &desc_block->off);
+ if (ret)
+ return ret;
+ field = off + PCI_EP_DMA_METADATA_CH_DESC_SIZE;
+ desc_block->sz = dw_edma_pcie_ep_dma_readl(view, field);
+ lo = off + PCI_EP_DMA_METADATA_CH_DESC_ADDR_LO;
+ hi = off + PCI_EP_DMA_METADATA_CH_DESC_ADDR_HI;
+ desc_block->paddr =
+ dw_edma_pcie_ep_dma_read64(view, lo, hi);
+ desc_block->paddr_valid = true;
+ if (!dw_edma_pcie_valid_block(view->pdev, desc_block))
+ return -EINVAL;
+
+ *data_block = (struct dw_edma_block) { .bar = NO_BAR };
+ if (!(ctrl & PCI_EP_DMA_METADATA_CH_CTRL_AUX_VALID))
+ continue;
+
+ data_block->bar =
+ FIELD_GET(PCI_EP_DMA_METADATA_CH_CTRL_AUX_BAR, ctrl);
+ lo = off + PCI_EP_DMA_METADATA_CH_AUX_OFF_LO;
+ hi = off + PCI_EP_DMA_METADATA_CH_AUX_OFF_HI;
+ ret = dw_edma_pcie_ep_dma_read_off(view, lo, hi,
+ &data_block->off);
+ if (ret)
+ return ret;
+ field = off + PCI_EP_DMA_METADATA_CH_AUX_SIZE;
+ data_block->sz = dw_edma_pcie_ep_dma_readl(view, field);
+ lo = off + PCI_EP_DMA_METADATA_CH_AUX_ADDR_LO;
+ hi = off + PCI_EP_DMA_METADATA_CH_AUX_ADDR_HI;
+ data_block->paddr =
+ dw_edma_pcie_ep_dma_read64(view, lo, hi);
+ data_block->paddr_valid = true;
+ if (!dw_edma_pcie_valid_block(view->pdev, data_block))
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int
+dw_edma_pcie_ep_dma_wait_ready(struct dw_edma_pcie_ep_dma_view *view)
+{
+ u32 val;
+
+ /*
+ * The host cannot build a usable eDMA instance until the endpoint has
+ * pinned and published the channel submaps, so keep the handshake
+ * synchronous and bounded during probe.
+ */
+ return read_poll_timeout(dw_edma_pcie_ep_dma_readl, val,
+ val & PCI_EP_DMA_METADATA_CTRL_READY,
+ DW_PCIE_EP_DMA_READY_POLL_US,
+ DW_PCIE_EP_DMA_READY_TIMEOUT_US, false,
+ view, PCI_EP_DMA_METADATA_CTRL);
+}
+
+static int
+dw_edma_pcie_validate_ep_dma_metadata(struct dw_edma_pcie_ep_dma_view *view,
+ u32 *metadata_ctrl, u8 *reg_layout_data)
+{
+ size_t table_size, table_end;
+ enum pci_barno reg_bar;
+ u16 len, entry_size;
+ u16 wr_ch_cnt, rd_ch_cnt;
+ u8 layout, layout_data;
+ u32 val;
+
+ val = dw_edma_pcie_ep_dma_readl(view, 0);
+ if (val != PCI_EP_DMA_METADATA_MAGIC)
+ return -ENODEV;
+
+ val = dw_edma_pcie_ep_dma_readl(view, PCI_EP_DMA_METADATA_HDR);
+ if (FIELD_GET(PCI_EP_DMA_METADATA_HDR_REV, val) !=
+ PCI_EP_DMA_METADATA_REV)
+ return -EINVAL;
+
+ len = FIELD_GET(PCI_EP_DMA_METADATA_HDR_LEN_FIELD, val);
+ if (len < PCI_EP_DMA_METADATA_HDR_LEN)
+ return -EINVAL;
+ if (len > view->limit)
+ return -EINVAL;
+
+ val = dw_edma_pcie_ep_dma_readl(view, PCI_EP_DMA_METADATA_REG_LAYOUT);
+ layout = FIELD_GET(PCI_EP_DMA_METADATA_REG_LAYOUT_ID, val);
+ if (layout != PCI_EP_DMA_METADATA_REG_LAYOUT_DW_EDMA)
+ return -EOPNOTSUPP;
+
+ layout_data = FIELD_GET(PCI_EP_DMA_METADATA_REG_LAYOUT_DATA, val);
+ if (layout_data == EDMA_MF_EDMA_LEGACY)
+ return -EOPNOTSUPP;
+ if (layout_data != EDMA_MF_EDMA_UNROLL &&
+ layout_data != EDMA_MF_HDMA_COMPAT &&
+ layout_data != EDMA_MF_HDMA_NATIVE)
+ return -EINVAL;
+
+ val = dw_edma_pcie_ep_dma_readl(view, PCI_EP_DMA_METADATA_CTRL);
+ reg_bar = FIELD_GET(PCI_EP_DMA_METADATA_CTRL_REG_BAR, val);
+ if (!dw_edma_pcie_valid_bar(reg_bar))
+ return -EINVAL;
+
+ wr_ch_cnt = FIELD_GET(PCI_EP_DMA_METADATA_CTRL_WR_CH_COUNT, val);
+ rd_ch_cnt = FIELD_GET(PCI_EP_DMA_METADATA_CTRL_RD_CH_COUNT, val);
+ if (!wr_ch_cnt && !rd_ch_cnt)
+ return -EINVAL;
+ if (wr_ch_cnt > EDMA_MAX_WR_CH || rd_ch_cnt > EDMA_MAX_RD_CH)
+ return -EINVAL;
+
+ entry_size = FIELD_GET(PCI_EP_DMA_METADATA_CTRL_CH_ENTRY_SIZE, val);
+ if (entry_size < PCI_EP_DMA_METADATA_CH_ENTRY_SIZE ||
+ entry_size % sizeof(u32))
+ return -EINVAL;
+
+ if (check_mul_overflow((size_t)(wr_ch_cnt + rd_ch_cnt),
+ (size_t)entry_size, &table_size) ||
+ check_add_overflow((size_t)PCI_EP_DMA_METADATA_HDR_LEN,
+ table_size, &table_end) ||
+ table_end > len)
+ return -EINVAL;
+
+ if (metadata_ctrl)
+ *metadata_ctrl = val;
+ if (reg_layout_data)
+ *reg_layout_data = layout_data;
+
+ return 0;
+}
+
+static int
+dw_edma_pcie_parse_ep_dma_data(struct dw_edma_pcie_ep_dma_view *view,
+ struct dw_edma_pcie_data *pdata)
+{
+ u32 ctrl, reg_sz;
+ u8 reg_layout_data;
+ u64 reg_off;
+ u16 wr_table, rd_table, entry_size;
+ u16 wr_ch_cnt, rd_ch_cnt;
+ int ret;
+
+ ret = dw_edma_pcie_validate_ep_dma_metadata(view, &ctrl,
+ ®_layout_data);
+ if (ret)
+ return ret;
+
+ pci_dbg(view->pdev, "Detected PCI endpoint DMA BAR metadata\n");
+
+ pdata->mf = reg_layout_data;
+ pdata->rg.bar = FIELD_GET(PCI_EP_DMA_METADATA_CTRL_REG_BAR, ctrl);
+
+ wr_ch_cnt = FIELD_GET(PCI_EP_DMA_METADATA_CTRL_WR_CH_COUNT, ctrl);
+ rd_ch_cnt = FIELD_GET(PCI_EP_DMA_METADATA_CTRL_RD_CH_COUNT, ctrl);
+ pdata->wr_ch_cnt = min_t(u16, pdata->wr_ch_cnt, wr_ch_cnt);
+ pdata->rd_ch_cnt = min_t(u16, pdata->rd_ch_cnt, rd_ch_cnt);
+ pdata->irqs = pdata->wr_ch_cnt + pdata->rd_ch_cnt;
+ reg_off = dw_edma_pcie_ep_dma_read64(view,
+ PCI_EP_DMA_METADATA_REG_OFF_LO,
+ PCI_EP_DMA_METADATA_REG_OFF_HI);
+ reg_sz = dw_edma_pcie_ep_dma_readl(view, PCI_EP_DMA_METADATA_REG_SIZE);
+ if (reg_off > type_max(pdata->rg.off) ||
+ !dw_edma_pcie_valid_bar_range(view->pdev, pdata->rg.bar,
+ reg_off, reg_sz))
+ return -EINVAL;
+ pdata->rg.off = reg_off;
+ pdata->rg.sz = reg_sz;
+
+ entry_size = FIELD_GET(PCI_EP_DMA_METADATA_CTRL_CH_ENTRY_SIZE, ctrl);
+ wr_table = PCI_EP_DMA_METADATA_HDR_LEN;
+ rd_table = PCI_EP_DMA_METADATA_HDR_LEN + wr_ch_cnt * entry_size;
+
+ ret = dw_edma_pcie_parse_ep_dma_ch_table(view, pdata, wr_table,
+ entry_size, pdata->wr_ch_cnt,
+ true);
+ if (ret)
+ return ret;
+
+ return dw_edma_pcie_parse_ep_dma_ch_table(view, pdata, rd_table,
+ entry_size,
+ pdata->rd_ch_cnt, false);
+}
+
+static int
+dw_edma_pcie_parse_ep_dma_caps(struct pci_dev *pdev,
+ struct dw_edma_pcie_data *pdata)
+{
+ struct dw_edma_pcie_ep_dma_view metadata_view;
+ void __iomem *base;
+ resource_size_t bar_len;
+ enum pci_barno bar;
+ u32 ctrl;
+ int ret;
+
+ for (bar = BAR_0; bar < PCI_STD_NUM_BARS; bar++) {
+ if (!dw_edma_pcie_ep_dma_bar_scannable(pdev, bar))
+ continue;
+
+ bar_len = pci_resource_len(pdev, bar);
+ base = pci_iomap_range(pdev, bar, 0, 0);
+ if (!base)
+ continue;
+
+ metadata_view = (struct dw_edma_pcie_ep_dma_view) {
+ .pdev = pdev,
+ .base = base,
+ .limit = bar_len,
+ };
+ ret = dw_edma_pcie_validate_ep_dma_metadata(&metadata_view,
+ NULL, NULL);
+ if (ret == -ENODEV) {
+ pci_iounmap(metadata_view.pdev, base);
+ continue;
+ }
+ if (ret) {
+ pci_iounmap(metadata_view.pdev, base);
+ return ret;
+ }
+
+ ctrl = dw_edma_pcie_ep_dma_readl(&metadata_view,
+ PCI_EP_DMA_METADATA_CTRL);
+ ctrl |= PCI_EP_DMA_METADATA_CTRL_HOST_REQ;
+ dw_edma_pcie_ep_dma_writel(&metadata_view,
+ PCI_EP_DMA_METADATA_CTRL, ctrl);
+
+ ret = dw_edma_pcie_ep_dma_wait_ready(&metadata_view);
+ if (ret) {
+ dw_edma_pcie_ep_dma_clear_host_req(&metadata_view);
+ pci_iounmap(metadata_view.pdev, base);
+ return ret;
+ }
+
+ ret = dw_edma_pcie_parse_ep_dma_data(&metadata_view, pdata);
+ if (ret)
+ dw_edma_pcie_ep_dma_clear_host_req(&metadata_view);
+ pci_iounmap(metadata_view.pdev, base);
+
+ return ret;
+ }
+
+ return -ENODEV;
+}
+
static int
dw_edma_pcie_parse_synopsys_caps(struct pci_dev *pdev,
struct dw_edma_pcie_data *pdata)
@@ -367,6 +744,14 @@ dw_edma_pcie_parse_xilinx_caps(struct pci_dev *pdev,
return 0;
}
+static const struct dw_edma_pcie_match_data ep_dma_match_data = {
+ .data = &ep_dma_data,
+ .plat_ops = &dw_edma_pcie_raw_addr_plat_ops,
+ .parse_caps = dw_edma_pcie_parse_ep_dma_caps,
+ .flags = DW_EDMA_PCIE_F_REG_OFFSET,
+ .chip_flags = DW_EDMA_CHIP_PARTIAL,
+};
+
static u64 dw_edma_get_phys_addr(struct pci_dev *pdev,
const struct dw_edma_pcie_match_data *match,
struct dw_edma_pcie_data *pdata,
@@ -400,8 +785,17 @@ static int dw_edma_pcie_probe(struct pci_dev *pdev,
int err, nr_irqs;
int i, mask;
- if (!match)
- return -ENODEV;
+ if (!match) {
+ /*
+ * The endpoint DMA metadata path has no static PCI ID yet.
+ * Accept it only for an explicit driver_override bind, not for
+ * arbitrary dynamic IDs without driver data.
+ */
+ if (!device_has_driver_override(&pdev->dev))
+ return -ENODEV;
+
+ match = &ep_dma_match_data;
+ }
pdata = match->data;
if (!pdata)
@@ -662,6 +1056,9 @@ static struct pci_driver dw_edma_pcie_driver = {
.id_table = dw_edma_pcie_id_table,
.probe = dw_edma_pcie_probe,
.remove = dw_edma_pcie_remove,
+ .driver = {
+ .probe_type = PROBE_PREFER_ASYNCHRONOUS,
+ },
};
module_pci_driver(dw_edma_pcie_driver);
--
2.51.0
^ permalink raw reply related
* [PATCH v5 0/3] PCI: endpoint: Add PCI DMA endpoint function (part 3/3)
From: Koichiro Den @ 2026-07-17 5:09 UTC (permalink / raw)
To: Manivannan Sadhasivam, Krzysztof Wilczyński,
Kishon Vijay Abraham I, Bjorn Helgaas, Jonathan Corbet,
Shuah Khan, Vinod Koul, Frank Li, Arnd Bergmann, Damien Le Moal,
Niklas Cassel
Cc: Marek Vasut, Yoshihiro Shimoda, linux-pci, linux-doc,
linux-kernel, dmaengine
Hi,
This is v5, part 3 of three series for PCI endpoint DMA.
The three series are:
* part 1: dmaengine: dw-edma: Prepare for PCI EP DMA
* part 2: PCI: endpoint: Expose endpoint DMA resources
* part 3: PCI: endpoint: Add PCI DMA endpoint function
This series adds the host-side metadata parser, the pci-epf-dma
endpoint function driver, and documentation.
The endpoint function exposes selected endpoint-integrated DMA channels
as a separate PCI DMA controller function. The host-side dw-edma-pcie
driver discovers the BAR metadata and registers the exposed channels
with DMAengine. The endpoint function keeps the metadata BAR stable and
uses a separate DMA window BAR for resources that need dynamic subrange
mappings.
No fixed PCI ID is assigned. Users provide the PCI vendor/device ID
through configfs and bind dw-edma-pcie explicitly, for example with
driver_override.
This series depends on parts 1 and 2:
[PATCH v5 00/14] dmaengine: dw-edma: Prepare for PCI EP DMA (part 1/3)
https://lore.kernel.org/r/20260717050308.2144108-4-den@valinux.co.jp/
[PATCH v5 0/6] PCI: endpoint: Expose endpoint DMA resources (part 2/3)
https://lore.kernel.org/r/20260717050635.2145014-1-den@valinux.co.jp/
One open question is how to support endpoint controllers with only one
PF. Keeping DMA in a separate EPF requires multi-function endpoint
support. Folding it into vNTB would work on single-function
controllers, but would also couple the two implementations. This series
keeps the separate EPF model.
The v5 RC-to-EP path was retested with a small out-of-tree DMAengine
client on RK3588 and SpacemiT K3 endpoint setups.
Best regards,
Koichiro
---
Changes in v5:
- Quiesce each shared-register direction once during channel release.
Changes in v4:
- Rebased onto the new parts 1/2 series.
- Support IOMMU-backed EPCs by DMA-mapping the dynamic DMA-control
MMIO BAR submaps for the EPC parent, keeping descriptor DMA
addresses unchanged.
Changes in v3:
- Select endpoint DMA match data before copying DMA data and require
driver_override for the generic endpoint DMA fallback. (Sashiko)
- Accept HDMA native linked-list endpoint DMA metadata.
- Consume logical DMA channels separately from descriptor memory
resources.
(Sashiko)
- Delegate channels through the EPC DMA channel delegation API instead of
v2's EPC-provided DMAengine filter callbacks.
- Allow HDMA native linked-list channels to be delegated at channel
granularity.
- Preserve HOST_REQ across link-down and retry DMA window submaps on the
next link-up.
- Drop trailing colons from documentation subsection headings. (Randy)
- Document HDMA native linked-list mode support and the current non-LL
limitation.
Changes in v2:
- Follow the part 1/3 and part 2/3 v2 channel-claim model: pci-epf-dma
now claims delegated channels through DMAengine filter information from
EPC auxiliary resources.
- Select raw-address dw-edma-pcie platform ops from the endpoint DMA
match entry instead of using a match flag.
v4: https://lore.kernel.org/r/20260710082727.2397253-1-den@valinux.co.jp/
v3: https://lore.kernel.org/r/20260620170844.3757241-1-den@valinux.co.jp/
v2: https://lore.kernel.org/r/20260525063456.3317509-1-den@valinux.co.jp/
v1: https://lore.kernel.org/r/20260521063638.2843021-1-den@valinux.co.jp/
Koichiro Den (3):
dmaengine: dw-edma-pcie: Discover endpoint DMA metadata
PCI: endpoint: Add DMA endpoint function
Documentation: PCI: Add PCI DMA endpoint function documentation
Documentation/PCI/endpoint/index.rst | 2 +
.../PCI/endpoint/pci-dma-function.rst | 188 ++
Documentation/PCI/endpoint/pci-dma-howto.rst | 201 +++
drivers/dma/dw-edma/dw-edma-pcie.c | 401 ++++-
drivers/pci/endpoint/functions/Kconfig | 13 +
drivers/pci/endpoint/functions/Makefile | 1 +
drivers/pci/endpoint/functions/pci-epf-dma.c | 1524 +++++++++++++++++
7 files changed, 2328 insertions(+), 2 deletions(-)
create mode 100644 Documentation/PCI/endpoint/pci-dma-function.rst
create mode 100644 Documentation/PCI/endpoint/pci-dma-howto.rst
create mode 100644 drivers/pci/endpoint/functions/pci-epf-dma.c
--
2.51.0
^ permalink raw reply
* [PATCH 2/2] Docs/admin-guide/cgroup-v2: fix delay_nsec unit in io.latency doc
From: Tao Cui @ 2026-07-17 4:28 UTC (permalink / raw)
To: Tejun Heo
Cc: Johannes Weiner, Michal Koutný, Jonathan Corbet, Shuah Khan,
Josef Bacik, Jens Axboe, cgroups, linux-doc, linux-block,
linux-kernel, Tao Cui
In-Reply-To: <20260717042817.2001826-1-cui.tao@linux.dev>
From: Tao Cui <cuitao@kylinos.cn>
The io.latency doc says the io.stat delay field counts microseconds. The
field is delay_nsec and is reported in nanoseconds. Refer to it by its
real name and correct the unit.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
---
Documentation/admin-guide/cgroup-v2.rst | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
index eec99813d82d..687839cf5c73 100644
--- a/Documentation/admin-guide/cgroup-v2.rst
+++ b/Documentation/admin-guide/cgroup-v2.rst
@@ -2304,9 +2304,9 @@ This throttling takes 2 forms:
throttled without possibly adversely affecting higher priority groups. This
includes swapping and metadata IO. These types of IO are allowed to occur
normally, however they are "charged" to the originating group. If the
- originating group is being throttled you will see the use_delay and delay
- fields in io.stat increase. The delay value is how many microseconds that are
- being added to any process that runs in this group. Because this number can
+ originating group is being throttled you will see the use_delay and delay_nsec
+ fields in io.stat increase. The delay_nsec value is how many nanoseconds that
+ are being added to any process that runs in this group. Because this number can
grow quite large if there is a lot of swapping or metadata IO occurring we
limit the individual delay events to 1 second at a time.
--
2.43.0
^ permalink raw reply related
* [PATCH 0/2] blk-iolatency: fix io.latency documentation accuracy
From: Tao Cui @ 2026-07-17 4:28 UTC (permalink / raw)
To: Tejun Heo
Cc: Johannes Weiner, Michal Koutný, Jonathan Corbet, Shuah Khan,
Josef Bacik, Jens Axboe, cgroups, linux-doc, linux-block,
linux-kernel, Tao Cui
From: Tao Cui <cuitao@kylinos.cn>
Hi,
This series fixes three inaccuracies in the io.latency documentation
(Documentation/admin-guide/cgroup-v2.rst), all caused by the doc
describing only rotational-device behavior:
Patch 1 documents the rotational vs non-rotational difference:
throttling is based on average latency on rotational devices but on the
fraction of IOs that miss the target on non-rotational devices; the
tuning guidance and io.stat field list are updated accordingly
(avg_lat/win are rotational-only; missed/total are documented for
non-rotational devices).
Patch 2 corrects the delay unit: io.stat reports delay_nsec in
nanoseconds, not microseconds as the doc stated.
Both patches are documentation-only and checkpatch-clean.
Tao Cui (2):
Docs/admin-guide/cgroup-v2: document io.latency rotational vs
non-rotational behavior
Docs/admin-guide/cgroup-v2: fix delay_nsec unit in io.latency doc
Documentation/admin-guide/cgroup-v2.rst | 53 +++++++++++++++++--------
1 file changed, 36 insertions(+), 17 deletions(-)
--
2.43.0
^ permalink raw reply
* [PATCH 1/2] Docs/admin-guide/cgroup-v2: document io.latency rotational vs non-rotational behavior
From: Tao Cui @ 2026-07-17 4:28 UTC (permalink / raw)
To: Tejun Heo
Cc: Johannes Weiner, Michal Koutný, Jonathan Corbet, Shuah Khan,
Josef Bacik, Jens Axboe, cgroups, linux-doc, linux-block,
linux-kernel, Tao Cui
In-Reply-To: <20260717042817.2001826-1-cui.tao@linux.dev>
From: Tao Cui <cuitao@kylinos.cn>
io.latency is documented only in terms of average latency and the avg_lat
stat, which matches rotational devices. On non-rotational devices a group
misses its target when 10% or more of the IOs in the window individually
exceed it, and io.stat reports missed/total rather than avg_lat/win.
Describe both cases: how a miss is detected, note that the avg_lat tuning
guidance is rotational-only, and update the io.stat field list (mark
avg_lat/win as rotational-only, document missed/total).
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
---
Documentation/admin-guide/cgroup-v2.rst | 47 +++++++++++++++++--------
1 file changed, 33 insertions(+), 14 deletions(-)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
index 0df15a672cf3..eec99813d82d 100644
--- a/Documentation/admin-guide/cgroup-v2.rst
+++ b/Documentation/admin-guide/cgroup-v2.rst
@@ -2260,9 +2260,12 @@ IO Latency
~~~~~~~~~~
This is a cgroup v2 controller for IO workload protection. You provide a group
-with a latency target, and if the average latency exceeds that target the
-controller will throttle any peers that have a lower latency target than the
-protected workload.
+with a latency target, and if the group misses its target the controller will
+throttle any peers that have a lower latency target than the protected
+workload. How a miss is detected depends on the device: on rotational devices
+the average latency over the window must exceed the target, while on
+non-rotational devices a miss is counted when 10% or more of the IOs in the
+window individually exceed the target.
The limits are only applied at the peer level in the hierarchy. This means that
in the diagram below, only groups A, B, and C will influence each other, and
@@ -2279,10 +2282,11 @@ So the ideal way to configure this is to set io.latency in groups A, B, and C.
Generally you do not want to set a value lower than the latency your device
supports. Experiment to find the value that works best for your workload.
Start at higher than the expected latency for your device and, with
-blkcg_debug_stats enabled, watch the avg_lat value in io.stat for your
-workload group to get an idea of the latency you see during normal operation.
-Use the avg_lat value as a basis for your real setting, setting at 10-15%
-higher than the value in io.stat.
+blkcg_debug_stats enabled, observe io.stat for your workload group to get an
+idea of the latency you see during normal operation. On rotational devices,
+use the avg_lat value as a basis for your real setting, setting it 10-15%
+higher. On non-rotational devices avg_lat is not reported; watch the
+missed/total ratio instead.
How IO Latency Throttling Works
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -2324,19 +2328,34 @@ IO Latency Interface Files
the blkcg_debug_stats module parameter is enabled (it is disabled by
default).
+ The reported latency fields depend on the device. Rotational devices
+ report avg_lat and win; non-rotational devices report missed and total
+ instead.
+
depth
This is the current queue depth for the group.
avg_lat
- This is an exponential moving average with a decay rate of 1/exp
- bound by the sampling interval. The decay rate interval can be
- calculated by multiplying the win value in io.stat by the
- corresponding number of samples based on the win value.
+ (Rotational devices only.) This is an exponential moving
+ average with a decay rate of 1/exp bound by the sampling
+ interval. The decay rate interval can be calculated by
+ multiplying the win value in io.stat by the corresponding number
+ of samples based on the win value.
win
- The sampling window size in milliseconds. This is the minimum
- duration of time between evaluation events. Windows only elapse
- with IO activity. Idle periods extend the most recent window.
+ (Rotational devices only.) The sampling window size in
+ milliseconds. This is the minimum duration of time between
+ evaluation events. Windows only elapse with IO activity. Idle
+ periods extend the most recent window.
+
+ missed
+ (Non-rotational devices only.) The number of IOs in the last
+ window whose latency exceeded the target.
+
+ total
+ (Non-rotational devices only.) The total number of IOs
+ accounted in the last window. A group is considered to be
+ missing its target once missed reaches 10% of total.
IO Priority
~~~~~~~~~~~
--
2.43.0
^ permalink raw reply related
* [PATCH v19 7/7] scsi: Enable async shutdown support
From: Tarun Sahu @ 2026-07-16 23:04 UTC (permalink / raw)
To: Shuah Khan, Danilo Krummrich, Bjorn Helgaas, Greg Kroah-Hartman,
Martin K. Petersen, James E.J. Bottomley, Jonathan Corbet,
Rafael J. Wysocki
Cc: bvanassche, john.g.garry, mlombard, loberman, mclapinski,
dmatlack, driver-core, linux-pci, Pasha Tatashin, jordanrichards,
souravsgl, stuart.w.hayes, linux-kernel, linux-doc, emilne,
jmeneghi, linux-scsi, David Jeffery
In-Reply-To: <20260716230411.2767394-1-tarunsahu@google.com>
From: David Jeffery <djeffery@redhat.com>
Like scsi's async suspend support, allow scsi devices to be shut down
asynchronously to reduce system shutdown time.
Signed-off-by: David Jeffery <djeffery@redhat.com>
Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com>
Tested-by: Laurence Oberman <loberman@redhat.com>
Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: Pasha Tatashin <pasha.tatashin@soleen.com>
Reviewed-by: John Garry <john.g.garry@oracle.com>
---
drivers/scsi/hosts.c | 2 ++
drivers/scsi/scsi_sysfs.c | 3 +++
2 files changed, 5 insertions(+)
diff --git a/drivers/scsi/hosts.c b/drivers/scsi/hosts.c
index e047747d4ecf..bf691acc7a67 100644
--- a/drivers/scsi/hosts.c
+++ b/drivers/scsi/hosts.c
@@ -273,6 +273,7 @@ int scsi_add_host_with_dma(struct Scsi_Host *shost, struct device *dev,
pm_runtime_set_active(&shost->shost_gendev);
pm_runtime_enable(&shost->shost_gendev);
device_enable_async_suspend(&shost->shost_gendev);
+ dev_set_async_shutdown(&shost->shost_gendev);
error = device_add(&shost->shost_gendev);
if (error)
@@ -282,6 +283,7 @@ int scsi_add_host_with_dma(struct Scsi_Host *shost, struct device *dev,
get_device(shost->shost_gendev.parent);
device_enable_async_suspend(&shost->shost_dev);
+ dev_set_async_shutdown(&shost->shost_dev);
get_device(&shost->shost_gendev);
error = device_add(&shost->shost_dev);
diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index dfc3559e7e04..8fd317aef37b 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -1386,6 +1386,7 @@ static int scsi_target_add(struct scsi_target *starget)
pm_runtime_set_active(&starget->dev);
pm_runtime_enable(&starget->dev);
device_enable_async_suspend(&starget->dev);
+ dev_set_async_shutdown(&starget->dev);
return 0;
}
@@ -1412,6 +1413,7 @@ int scsi_sysfs_add_sdev(struct scsi_device *sdev)
transport_configure_device(&starget->dev);
device_enable_async_suspend(&sdev->sdev_gendev);
+ dev_set_async_shutdown(&sdev->sdev_gendev);
scsi_autopm_get_target(starget);
pm_runtime_set_active(&sdev->sdev_gendev);
if (!sdev->rpm_autosuspend)
@@ -1431,6 +1433,7 @@ int scsi_sysfs_add_sdev(struct scsi_device *sdev)
}
device_enable_async_suspend(&sdev->sdev_dev);
+ dev_set_async_shutdown(&sdev->sdev_dev);
error = device_add(&sdev->sdev_dev);
if (error) {
sdev_printk(KERN_INFO, sdev,
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v19 6/7] PCI: Enable async shutdown support
From: Tarun Sahu @ 2026-07-16 23:04 UTC (permalink / raw)
To: Shuah Khan, Danilo Krummrich, Bjorn Helgaas, Greg Kroah-Hartman,
Martin K. Petersen, James E.J. Bottomley, Jonathan Corbet,
Rafael J. Wysocki
Cc: bvanassche, john.g.garry, mlombard, loberman, mclapinski,
dmatlack, driver-core, linux-pci, Pasha Tatashin, jordanrichards,
souravsgl, stuart.w.hayes, linux-kernel, linux-doc, emilne,
jmeneghi, linux-scsi, David Jeffery
In-Reply-To: <20260716230411.2767394-1-tarunsahu@google.com>
From: David Jeffery <djeffery@redhat.com>
Like its async suspend support, allow PCI device shutdown to be performed
asynchronously to reduce shutdown time.
Signed-off-by: David Jeffery <djeffery@redhat.com>
Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com>
Tested-by: Laurence Oberman <loberman@redhat.com>
Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: Pasha Tatashin <pasha.tatashin@soleen.com>
Reviewed-by: Bjorn Helgaas <bhelgaas@google.com>
---
drivers/pci/probe.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index dd0abbc63e18..af2381446406 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -1046,6 +1046,7 @@ static int pci_register_host_bridge(struct pci_host_bridge *bridge)
bus->bridge = get_device(&bridge->dev);
device_enable_async_suspend(bus->bridge);
+ dev_set_async_shutdown(bus->bridge);
pci_set_bus_of_node(bus);
pci_set_bus_msi_domain(bus);
if (bridge->msi_domain && !dev_get_msi_domain(&bus->dev) &&
@@ -2748,6 +2749,7 @@ void pci_device_add(struct pci_dev *dev, struct pci_bus *bus)
pci_reassigndev_resource_alignment(dev);
pci_init_capabilities(dev);
+ dev_set_async_shutdown(&dev->dev);
/*
* Add the device to our list of discovered devices
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v19 5/7] driver core: async device shutdown infrastructure
From: Tarun Sahu @ 2026-07-16 23:04 UTC (permalink / raw)
To: Shuah Khan, Danilo Krummrich, Bjorn Helgaas, Greg Kroah-Hartman,
Martin K. Petersen, James E.J. Bottomley, Jonathan Corbet,
Rafael J. Wysocki
Cc: bvanassche, john.g.garry, mlombard, loberman, mclapinski,
dmatlack, driver-core, linux-pci, Pasha Tatashin, jordanrichards,
souravsgl, stuart.w.hayes, linux-kernel, linux-doc, emilne,
jmeneghi, linux-scsi, David Jeffery, Tarun Sahu
In-Reply-To: <20260716230411.2767394-1-tarunsahu@google.com>
From: David Jeffery <djeffery@redhat.com>
Patterned after async suspend, allow devices to mark themselves as wanting
to perform async shutdown. Devices using async shutdown wait only for their
dependencies to shutdown before executing their shutdown routine.
Sync shutdown devices are shut down one at a time and will only wait for an
async shutdown device if the async device is a dependency.
Enabled by default, async shutdown can be explicitly enabled or disabled
by using the kernel parameter "core.async_shutdown=<bool>"
Signed-off-by: David Jeffery <djeffery@redhat.com>
Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com>
Signed-off-by: Tarun Sahu <tarunsahu@google.com>
Tested-by: Laurence Oberman <loberman@redhat.com>
---
.../admin-guide/kernel-parameters.txt | 10 ++
drivers/base/base.h | 2 +
drivers/base/core.c | 141 +++++++++++++++++-
include/linux/device.h | 2 +
4 files changed, 154 insertions(+), 1 deletion(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index b5493a7f8f22..f0de215963c6 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -989,6 +989,16 @@ Kernel parameters
seconds. A value of 0 disables the blank timer.
Defaults to 0.
+ core.async_shutdown=
+ [KNL]
+ Format: <bool>
+ Enable or disable asynchronous shutdown support. When
+ enabled, on system shutdown unrelated devices flagged
+ as async shutdown compatible may be shut down in
+ parallel and asynchronously. When disabled, device
+ shutdown is performed serially and synchronously.
+ Enabled by default.
+
coredump_filter=
[KNL] Change the default value for
/proc/<pid>/coredump_filter.
diff --git a/drivers/base/base.h b/drivers/base/base.h
index a5b7abc10ff0..40dbf588a5d6 100644
--- a/drivers/base/base.h
+++ b/drivers/base/base.h
@@ -103,6 +103,7 @@ struct driver_private {
* dev_err_probe() for later retrieval via debugfs
* @device: pointer back to the struct device that this structure is
* associated with.
+ * @complete: completion for device shutdown ordering
* @dead: This device is currently either in the process of or has been
* removed from the system. Any asynchronous events scheduled for this
* device should exit without taking any action.
@@ -119,6 +120,7 @@ struct device_private {
const struct device_driver *async_driver;
char *deferred_probe_reason;
struct device *device;
+ struct completion complete;
u8 dead:1;
};
#define to_device_private_parent(obj) \
diff --git a/drivers/base/core.c b/drivers/base/core.c
index 31f95e86856e..55287f13b8d1 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -9,6 +9,7 @@
*/
#include <linux/acpi.h>
+#include <linux/async.h>
#include <linux/blkdev.h>
#include <linux/cleanup.h>
#include <linux/cpufreq.h>
@@ -37,6 +38,11 @@
#include "physical_location.h"
#include "power/power.h"
+static bool async_shutdown = true;
+module_param(async_shutdown, bool, 0644);
+MODULE_PARM_DESC(async_shutdown, "Enable asynchronous device shutdown support");
+static bool async_shutdown_enabled;
+
/* Device links support. */
static LIST_HEAD(deferred_sync);
static unsigned int defer_sync_state_count = 1;
@@ -3623,6 +3629,7 @@ static int device_private_init(struct device *dev)
klist_init(&dev->p->klist_children, klist_children_get,
klist_children_put);
INIT_LIST_HEAD(&dev->p->deferred_probe);
+ init_completion(&dev->p->complete);
return 0;
}
@@ -3928,6 +3935,7 @@ bool kill_device(struct device *dev)
if (dev->p->dead)
return false;
dev->p->dead = true;
+ complete_all(&dev->p->complete);
return true;
}
EXPORT_SYMBOL_GPL(kill_device);
@@ -4898,6 +4906,40 @@ int device_change_owner(struct device *dev, kuid_t kuid, kgid_t kgid)
return error;
}
+static bool wants_async_shutdown(struct device *dev)
+{
+ return async_shutdown_enabled && dev_async_shutdown(dev);
+}
+
+static int wait_for_device_shutdown(struct device *dev, void *data)
+{
+ bool async = *(bool *)data;
+
+ if (!dev->p || !device_is_registered(dev))
+ return 0;
+
+ if (async || wants_async_shutdown(dev))
+ wait_for_completion(&dev->p->complete);
+
+ return 0;
+}
+
+static void wait_for_shutdown_dependencies(struct device *dev, bool async)
+{
+ struct device_link *link;
+ int idx;
+
+ device_for_each_child(dev, &async, wait_for_device_shutdown);
+
+ idx = device_links_read_lock();
+
+ dev_for_each_link_to_consumer(link, dev)
+ if (!device_link_flag_is_sync_state_only(link->flags))
+ wait_for_device_shutdown(link->consumer, &async);
+
+ device_links_read_unlock(idx);
+}
+
static void __shutdown_one_device(struct device *dev)
{
if (!dev->p || dev->p->dead)
@@ -4921,6 +4963,8 @@ static void __shutdown_one_device(struct device *dev)
dev_info(dev, "shutdown\n");
dev->driver->shutdown(dev);
}
+
+ complete_all(&dev->p->complete);
}
static void shutdown_one_device(struct device *dev)
@@ -4950,6 +4994,88 @@ static void shutdown_one_device(struct device *dev)
put_device(dev);
}
+static void async_shutdown_handler(void *data, async_cookie_t cookie)
+{
+ struct device *dev = data;
+
+ wait_for_shutdown_dependencies(dev, true);
+ shutdown_one_device(dev);
+}
+
+static bool shutdown_device_async(struct device *dev)
+{
+ if (async_schedule_dev_nocall(async_shutdown_handler, dev))
+ return true;
+
+ dev_clear_async_shutdown(dev);
+ return false;
+}
+
+
+static void start_async_shutdown_devices(void)
+{
+ struct device *dev, *next, *ndev, *needs_put = NULL;
+ bool clear_async = false;
+
+ if (!async_shutdown_enabled)
+ return;
+
+ spin_lock(&devices_kset->list_lock);
+restart:
+ list_for_each_entry_safe_reverse(dev, next, &devices_kset->list,
+ kobj.entry) {
+ if (wants_async_shutdown(dev)) {
+ if (clear_async) {
+ dev_clear_async_shutdown(dev);
+ continue;
+ }
+ /* one device reference for this function */
+ get_device(dev);
+ /* another to pass to the async task */
+ get_device(dev);
+
+ if (!list_entry_is_head(next, &devices_kset->list,
+ kobj.entry))
+ ndev = get_device(next);
+ else
+ ndev = NULL;
+ spin_unlock(&devices_kset->list_lock);
+
+ if (shutdown_device_async(dev)) {
+ spin_lock(&devices_kset->list_lock);
+ list_del_init(&dev->kobj.entry);
+ spin_unlock(&devices_kset->list_lock);
+ } else {
+ /*
+ * async failed, clean up extra reference
+ * and run shutdown from the sync shutdown loop
+ */
+ clear_async = true;
+ put_device(dev);
+ }
+ put_device(dev);
+
+ if (needs_put)
+ put_device(needs_put);
+ needs_put = ndev;
+ spin_lock(&devices_kset->list_lock);
+ /*
+ * If the next device has been marked dead while the
+ * spinlock was released, or if it has been unlinked
+ * from the list, it may no longer be on the
+ * devices_kset list. Restart the list walk to be safe.
+ */
+ if (ndev && (ndev->p->dead || list_empty(&ndev->kobj.entry)))
+ goto restart;
+ }
+ }
+
+ spin_unlock(&devices_kset->list_lock);
+
+ if (needs_put)
+ put_device(needs_put);
+}
+
/**
* device_shutdown - call ->shutdown() on each device to shutdown.
*/
@@ -4962,6 +5088,14 @@ void device_shutdown(void)
cpufreq_suspend();
+ async_shutdown_enabled = async_shutdown;
+
+ /*
+ * Start async device threads where possible to maximize potential
+ * parallelism and minimize false dependency on unrelated sync devices
+ */
+ start_async_shutdown_devices();
+
spin_lock(&devices_kset->list_lock);
/*
* Walk the devices list backward, shutting down each in turn.
@@ -4980,11 +5114,16 @@ void device_shutdown(void)
list_del_init(&dev->kobj.entry);
spin_unlock(&devices_kset->list_lock);
- shutdown_one_device(dev);
+ if (!wants_async_shutdown(dev) || !shutdown_device_async(dev)) {
+ wait_for_shutdown_dependencies(dev, false);
+ shutdown_one_device(dev);
+ }
spin_lock(&devices_kset->list_lock);
}
spin_unlock(&devices_kset->list_lock);
+
+ async_synchronize_full();
}
/*
diff --git a/include/linux/device.h b/include/linux/device.h
index 7b2baffdd2f5..f913d72218f8 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -610,6 +610,7 @@ enum struct_device_flags {
DEV_FLAG_OF_NODE_REUSED = 7,
DEV_FLAG_OFFLINE_DISABLED = 8,
DEV_FLAG_OFFLINE = 9,
+ DEV_FLAG_ASYNC_SHUTDOWN = 10,
DEV_FLAG_COUNT
};
@@ -827,6 +828,7 @@ __create_dev_flag_accessors(dma_coherent, DEV_FLAG_DMA_COHERENT);
__create_dev_flag_accessors(of_node_reused, DEV_FLAG_OF_NODE_REUSED);
__create_dev_flag_accessors(offline_disabled, DEV_FLAG_OFFLINE_DISABLED);
__create_dev_flag_accessors(offline, DEV_FLAG_OFFLINE);
+__create_dev_flag_accessors(async_shutdown, DEV_FLAG_ASYNC_SHUTDOWN);
#undef __create_dev_flag_accessors
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v19 4/7] driver core: do not always lock parent in shutdown
From: Tarun Sahu @ 2026-07-16 23:04 UTC (permalink / raw)
To: Shuah Khan, Danilo Krummrich, Bjorn Helgaas, Greg Kroah-Hartman,
Martin K. Petersen, James E.J. Bottomley, Jonathan Corbet,
Rafael J. Wysocki
Cc: bvanassche, john.g.garry, mlombard, loberman, mclapinski,
dmatlack, driver-core, linux-pci, Pasha Tatashin, jordanrichards,
souravsgl, stuart.w.hayes, linux-kernel, linux-doc, emilne,
jmeneghi, linux-scsi, David Jeffery, Tarun Sahu
In-Reply-To: <20260716230411.2767394-1-tarunsahu@google.com>
From: David Jeffery <djeffery@redhat.com>
Don't lock a parent device unless it is needed in device_shutdown. This
is in preparation for making device shutdown asynchronous, when it will
be needed to allow children of a common parent to shut down
simultaneously.
And only acquire a reference to the parent device if the parent is to be
locked.
Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com>
Signed-off-by: David Jeffery <djeffery@redhat.com>
Signed-off-by: Tarun Sahu <tarunsahu@google.com>
Tested-by: Laurence Oberman <loberman@redhat.com>
---
drivers/base/core.c | 42 ++++++++++++++++++++++++++----------------
1 file changed, 26 insertions(+), 16 deletions(-)
diff --git a/drivers/base/core.c b/drivers/base/core.c
index 82a7af1f3ba2..31f95e86856e 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -4898,14 +4898,10 @@ int device_change_owner(struct device *dev, kuid_t kuid, kgid_t kgid)
return error;
}
-static void shutdown_one_device(struct device *dev)
+static void __shutdown_one_device(struct device *dev)
{
- struct device *parent = dev->parent;
-
- /* hold lock to avoid race with probe/release */
- if (parent)
- device_lock(parent);
- device_lock(dev);
+ if (!dev->p || dev->p->dead)
+ return;
/* Don't allow any more runtime suspends */
pm_runtime_get_noresume(dev);
@@ -4925,12 +4921,32 @@ static void shutdown_one_device(struct device *dev)
dev_info(dev, "shutdown\n");
dev->driver->shutdown(dev);
}
+}
- device_unlock(dev);
- if (parent)
+static void shutdown_one_device(struct device *dev)
+{
+ struct device *parent;
+
+ device_lock(dev);
+
+ /* use parent lock if needed to avoid race with probe/release */
+ if (dev->bus && dev->bus->need_parent_lock && dev->p && !dev->p->dead &&
+ (parent = get_device(dev->parent))) {
+ /* the parent lock needs to be acquired first, so re-lock */
+ device_unlock(dev);
+
+ device_lock(parent);
+ device_lock(dev);
+
+ __shutdown_one_device(dev);
+ device_unlock(dev);
device_unlock(parent);
+ put_device(parent);
+ } else {
+ __shutdown_one_device(dev);
+ device_unlock(dev);
+ }
- put_device(parent);
put_device(dev);
}
@@ -4956,12 +4972,6 @@ void device_shutdown(void)
dev = list_entry(devices_kset->list.prev, struct device,
kobj.entry);
- /*
- * hold reference count of device's parent to
- * prevent it from being freed because parent's
- * lock is to be held
- */
- get_device(dev->parent);
get_device(dev);
/*
* Make sure the device is off the kset list, in the
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v19 3/7] driver core: separate function to shutdown one device
From: Tarun Sahu @ 2026-07-16 23:04 UTC (permalink / raw)
To: Shuah Khan, Danilo Krummrich, Bjorn Helgaas, Greg Kroah-Hartman,
Martin K. Petersen, James E.J. Bottomley, Jonathan Corbet,
Rafael J. Wysocki
Cc: bvanassche, john.g.garry, mlombard, loberman, mclapinski,
dmatlack, driver-core, linux-pci, Pasha Tatashin, jordanrichards,
souravsgl, stuart.w.hayes, linux-kernel, linux-doc, emilne,
jmeneghi, linux-scsi, David Jeffery
In-Reply-To: <20260716230411.2767394-1-tarunsahu@google.com>
From: David Jeffery <djeffery@redhat.com>
Make a separate function for the part of device_shutdown() that does the
shutown for a single device. This is in preparation for making device
shutdown asynchronous.
Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com>
Signed-off-by: David Jeffery <djeffery@redhat.com>
Tested-by: Laurence Oberman <loberman@redhat.com>
Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: Pasha Tatashin <pasha.tatashin@soleen.com>
---
drivers/base/core.c | 71 +++++++++++++++++++++++++--------------------
1 file changed, 39 insertions(+), 32 deletions(-)
diff --git a/drivers/base/core.c b/drivers/base/core.c
index 78b90326addb..82a7af1f3ba2 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -4898,12 +4898,48 @@ int device_change_owner(struct device *dev, kuid_t kuid, kgid_t kgid)
return error;
}
+static void shutdown_one_device(struct device *dev)
+{
+ struct device *parent = dev->parent;
+
+ /* hold lock to avoid race with probe/release */
+ if (parent)
+ device_lock(parent);
+ device_lock(dev);
+
+ /* Don't allow any more runtime suspends */
+ pm_runtime_get_noresume(dev);
+ pm_runtime_barrier(dev);
+
+ if (dev->class && dev->class->shutdown_pre) {
+ if (initcall_debug)
+ dev_info(dev, "shutdown_pre\n");
+ dev->class->shutdown_pre(dev);
+ }
+ if (dev->bus && dev->bus->shutdown) {
+ if (initcall_debug)
+ dev_info(dev, "shutdown\n");
+ dev->bus->shutdown(dev);
+ } else if (dev->driver && dev->driver->shutdown) {
+ if (initcall_debug)
+ dev_info(dev, "shutdown\n");
+ dev->driver->shutdown(dev);
+ }
+
+ device_unlock(dev);
+ if (parent)
+ device_unlock(parent);
+
+ put_device(parent);
+ put_device(dev);
+}
+
/**
* device_shutdown - call ->shutdown() on each device to shutdown.
*/
void device_shutdown(void)
{
- struct device *dev, *parent;
+ struct device *dev;
wait_for_device_probe();
device_block_probing();
@@ -4925,7 +4961,7 @@ void device_shutdown(void)
* prevent it from being freed because parent's
* lock is to be held
*/
- parent = get_device(dev->parent);
+ get_device(dev->parent);
get_device(dev);
/*
* Make sure the device is off the kset list, in the
@@ -4934,36 +4970,7 @@ void device_shutdown(void)
list_del_init(&dev->kobj.entry);
spin_unlock(&devices_kset->list_lock);
- /* hold lock to avoid race with probe/release */
- if (parent)
- device_lock(parent);
- device_lock(dev);
-
- /* Don't allow any more runtime suspends */
- pm_runtime_get_noresume(dev);
- pm_runtime_barrier(dev);
-
- if (dev->class && dev->class->shutdown_pre) {
- if (initcall_debug)
- dev_info(dev, "shutdown_pre\n");
- dev->class->shutdown_pre(dev);
- }
- if (dev->bus && dev->bus->shutdown) {
- if (initcall_debug)
- dev_info(dev, "shutdown\n");
- dev->bus->shutdown(dev);
- } else if (dev->driver && dev->driver->shutdown) {
- if (initcall_debug)
- dev_info(dev, "shutdown\n");
- dev->driver->shutdown(dev);
- }
-
- device_unlock(dev);
- if (parent)
- device_unlock(parent);
-
- put_device(dev);
- put_device(parent);
+ shutdown_one_device(dev);
spin_lock(&devices_kset->list_lock);
}
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v19 2/7] driver core: Prevent device_add() during system shutdown
From: Tarun Sahu @ 2026-07-16 23:04 UTC (permalink / raw)
To: Shuah Khan, Danilo Krummrich, Bjorn Helgaas, Greg Kroah-Hartman,
Martin K. Petersen, James E.J. Bottomley, Jonathan Corbet,
Rafael J. Wysocki
Cc: bvanassche, john.g.garry, mlombard, loberman, mclapinski,
dmatlack, driver-core, linux-pci, Pasha Tatashin, jordanrichards,
souravsgl, stuart.w.hayes, linux-kernel, linux-doc, emilne,
jmeneghi, linux-scsi, Tarun Sahu, David Jeffery
In-Reply-To: <20260716230411.2767394-1-tarunsahu@google.com>
In Async device shutdown, device_kset->list lock is released to handle
asynchronisation and hold again to get entry from device_kset->list.
Which will leave window when device_add can try to add the device to
device_kset list and temper with ongoing shutdown process. New added
device can be async type or sync type and might also introduce new
dependency which can cause device_shutdown path to deadlock. S is
waiting C to finish but C is never scheduled as it was added recently
from device_add path. And C can only be scheduled when main loops
continue to reach to C which is waiting on S.
So, When a system enters shutdown (SYSTEM_HALT, SYSTEM_POWER_OFF, or
SYSTEM_RESTART), new devices should not be allowed to be added.
Adding system_state check (system_is_shutting_down()) to avoid
device_add incase of these states of the system.
While device_add() performs an initial check of system_is_shutting_down(),
a race window exists between this initial check and kobject_add(),
during which device_shutdown() may already be scanning devices_kset->list.
If device_shutdown() passes the device after kobject_add() registers it
onto devices_kset->list, device_add() would otherwise complete device
initialization and driver matching, leaving an active device running
after system shutdown finishes.
Fix this TOCTOU race by re-checking system_is_shutting_down() under
devices_kset->list_lock right after kobject_add().
Signed-off-by: Tarun Sahu <tarunsahu@google.com>
Signed-off-by: David Jeffery <djeffery@redhat.com>
---
drivers/base/core.c | 34 ++++++++++++++++++++++++++++++++++
1 file changed, 34 insertions(+)
diff --git a/drivers/base/core.c b/drivers/base/core.c
index 76ba02c26aa5..78b90326addb 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -47,6 +47,22 @@ static bool fw_devlink_drv_reg_done;
static bool fw_devlink_best_effort;
static struct workqueue_struct *device_link_wq;
+/**
+ * system_is_shutting_down - Check if system state is not active.
+ *
+ * When system state is not active and in shutdown state, new devices
+ * should not be allowed to be added.
+ *
+ * If system_state is SYSTEM_HALT || SYSTEM_POWER_OFF || SYSTEM_RESTART
+ * this function will return true.
+ */
+static inline bool system_is_shutting_down(void)
+{
+ return system_state == SYSTEM_HALT ||
+ system_state == SYSTEM_POWER_OFF ||
+ system_state == SYSTEM_RESTART;
+}
+
/**
* __fwnode_link_add - Create a link between two fwnode_handles.
* @con: Consumer end of the link.
@@ -3650,6 +3666,11 @@ int device_add(struct device *dev)
if (!dev)
goto done;
+ if (unlikely(system_is_shutting_down())) {
+ error = -ESHUTDOWN;
+ goto done;
+ }
+
if (!dev->p) {
error = device_private_init(dev);
if (error)
@@ -3699,6 +3720,18 @@ int device_add(struct device *dev)
goto Error;
}
+ /*
+ * Check system_state again under list_lock to prevent a TOCTOU race
+ * where device_shutdown() runs concurrently and misses this device.
+ */
+ spin_lock(&devices_kset->list_lock);
+ if (unlikely(system_is_shutting_down())) {
+ spin_unlock(&devices_kset->list_lock);
+ error = -ESHUTDOWN;
+ goto ShutdownError;
+ }
+ spin_unlock(&devices_kset->list_lock);
+
/* notify platform of device entry */
device_platform_notify(dev);
@@ -3818,6 +3851,7 @@ int device_add(struct device *dev)
attrError:
device_platform_notify_remove(dev);
kobject_uevent(&dev->kobj, KOBJ_REMOVE);
+ ShutdownError:
glue_dir = get_glue_dir(dev);
kobject_del(&dev->kobj);
Error:
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v19 1/7] device_core: rely on put_device to free dev->p
From: Tarun Sahu @ 2026-07-16 23:04 UTC (permalink / raw)
To: Shuah Khan, Danilo Krummrich, Bjorn Helgaas, Greg Kroah-Hartman,
Martin K. Petersen, James E.J. Bottomley, Jonathan Corbet,
Rafael J. Wysocki
Cc: bvanassche, john.g.garry, mlombard, loberman, mclapinski,
dmatlack, driver-core, linux-pci, Pasha Tatashin, jordanrichards,
souravsgl, stuart.w.hayes, linux-kernel, linux-doc, emilne,
jmeneghi, linux-scsi, Tarun Sahu, David Jeffery
In-Reply-To: <20260716230411.2767394-1-tarunsahu@google.com>
device_add allocate private_data for device and assigns to
dev->p. If device_add fails in later steps of the function,
it cleans up this dev->p which is not necessary because In
the next call, put_device free it anyway (if reference to
the device is 0 which will be unless someone concurrently
get the reference to this device).
This avoids unnecessary races introduced in system. After device
is added in device_kset->list by device_add and later steps in the
device_add function failures occur, it will free dev->p manually,
while in between there might be a user of device_kset->list will
take reference to the device just added by device_add. and might
try to access dev->p. So relying on put_device to free dev->p
prevents such problem.
Signed-off-by: Tarun Sahu <tarunsahu@google.com>
Signed-off-by: David Jeffery <djeffery@redhat.com>
---
drivers/base/core.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/drivers/base/core.c b/drivers/base/core.c
index 4d026682944f..76ba02c26aa5 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -2614,6 +2614,7 @@ static void device_release(struct kobject *kobj)
struct device *dev = kobj_to_dev(kobj);
struct device_private *p = dev->p;
+ dev->p = NULL;
/*
* Some platform devices are driven without driver attached
* and managed resources may have been acquired. Make sure
@@ -3824,8 +3825,6 @@ int device_add(struct device *dev)
parent_error:
put_device(parent);
name_error:
- kfree(dev->p);
- dev->p = NULL;
goto done;
}
EXPORT_SYMBOL_GPL(device_add);
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply related
* [PATCH v19 0/7] shut down devices asynchronously
From: Tarun Sahu @ 2026-07-16 23:04 UTC (permalink / raw)
To: Shuah Khan, Danilo Krummrich, Bjorn Helgaas, Greg Kroah-Hartman,
Martin K. Petersen, James E.J. Bottomley, Jonathan Corbet,
Rafael J. Wysocki
Cc: bvanassche, john.g.garry, mlombard, loberman, mclapinski,
dmatlack, driver-core, linux-pci, Pasha Tatashin, jordanrichards,
souravsgl, stuart.w.hayes, linux-kernel, linux-doc, emilne,
jmeneghi, linux-scsi, Tarun Sahu
This is rebased on top of device-core/driver-core-next. And can also
be applied easily against linux-next/master
This patchset allows the kernel to shutdown devices asynchronously and
unrelated async devices to be shut down in parallel to each other.
Only devices which explicitly enable it are shut down asynchronously. The
default is for a device to be shut down from the synchronous shutdown loop.
This can dramatically reduce system shutdown/reboot time on systems that
have multiple devices that take many seconds to shut down (like certain
NVMe drives). On one system tested, the shutdown time went from 11 minutes
without this patch to 55 seconds with the patch. And on another system from
80 seconds to 11.
And thank you to everyone who has spent some of their valuable time
providing reviews, suggestions, criticisms, or tests on the various
iterations of this patchset.
Changes from V18:
- Fix deadlock and race condition on concurrent device_add
- Use local variable to device_shutdown for caching async_shutdown
- Added dev->p check to avoid NULL pointer deference and tell if the
device is not registered so no need to be shutdown. This is to handle
the cases when devices consumers list might have devices that are not
yet registered.
Changes from V17:
Fix mangled text in kernel parameter description
Re-protect the list removal with the spinlock
* Hold a device reference to ensure the device cannot be freed before
attempting list removal
Changes from V16:
Drop spinlock before async subsystem call which uses GFP_KERNEL
Handle that async shutdown can widen races between device shutdown and deletion
* __shutdown_one_device will immediately return if a device is dead
* Set shutdown device completion to complete when marking a device dead to
prevent waiting on a dead device
* Only late-access a parent pointer if device is in a non-dead state to
ensure the pointer is still valid
Changes from V15:
The async_shutdown bit field is converted to a device flags bit
Convert all patches to use the flag bit accessor macros to set or check if
async shutdown should be used
Added documentation on the kernel parameter to control use of async shutdown
Changes from V14:
Remove unneeded use of '!!' with boolean type
Changes from V13:
Remove duplicate flagging of async shutdown on scsi hosts/targets/devices
Changes from V12:
Only acquire a parent reference if acquiring the parent's lock
device_enable_async_shutdown should return void
Minor comment and description cleanups
Changes from V11:
* Swap the order of the first two patches
* Rework conditional parent locking so that lock and unlock no longer use
separate conditional checks
* Remove an used variable
* Comment and description text cleanups
Changes from V10:
Reworked to more closely match the design used for async suspend
* No longer uses async subsystem cookies for synchronization
* Minimized changes to struct device
* Enable async shutdown for pci and scsi devices which support async suspend
Changes from V9:
Address resource and timing issues when spawning a unique async thread
for every device during shutdown:
* Make the asynchronous threads able to shut down multiple devices,
instead of spawning a unique thread for every device.
* Modify core kernel async code with a custom wake function so it
doesn't wake up a thread waiting to synchronize on a cookie until
the cookie has reached the desired value, instead of waking up
every waiting thread to check the cookie every time an async thread
ends.
Changes from V8:
Deal with shutdown hangs resulting when a parent/supplier device is
later in the devices_kset list than its children/consumers:
* Ignore sync_state_only devlinks for shutdown dependencies
* Ignore shutdown_after for devices that don't want async shutdown
* Add a sanity check to revert to sync shutdown for any device that
would otherwise wait for a child/consumer shutdown that hasn't
already been scheduled
Changes from V7:
Do not expose driver async_shutdown_enable in sysfs.
Wrapped a long line.
Changes from V6:
Removed a sysfs attribute that allowed the async device shutdown to be
"on" (with driver opt-out), "safe" (driver opt-in), or "off"... what was
previously "safe" is now the only behavior, so drivers now only need to
have the option to enable or disable async shutdown.
Changes from V5:
Separated into multiple patches to make review easier.
Reworked some code to make it more readable
Made devices wait for consumers to shut down, not just children
(suggested by David Jeffery)
Changes from V4:
Change code to use cookies for synchronization rather than async domains
Allow async shutdown to be disabled via sysfs, and allow driver opt-in or
opt-out of async shutdown (when not disabled), with ability to control
driver opt-in/opt-out via sysfs
Changes from V3:
Bug fix (used "parent" not "dev->parent" in device_shutdown)
Changes from V2:
Removed recursive functions to schedule children to be shutdown before
parents, since existing device_shutdown loop will already do this
Changes from V1:
Rewritten using kernel async code (suggested by Lukas Wunner)
David Jeffery (5):
driver core: separate function to shutdown one device
driver core: do not always lock parent in shutdown
driver core: async device shutdown infrastructure
PCI: Enable async shutdown support
scsi: Enable async shutdown support
Tarun Sahu (2):
device_core: rely on put_device to free dev->p
driver core: Prevent device_add() during system shutdown
.../admin-guide/kernel-parameters.txt | 10 +
drivers/base/base.h | 2 +
drivers/base/core.c | 265 +++++++++++++++---
drivers/pci/probe.c | 2 +
drivers/scsi/hosts.c | 2 +
drivers/scsi/scsi_sysfs.c | 3 +
include/linux/device.h | 2 +
7 files changed, 248 insertions(+), 38 deletions(-)
--
2.55.0.229.g6434b31f56-goog
^ permalink raw reply
* [PATCH v2] docs: pt_BR: process: Translate the security-bugs.rst
From: Álysson Gleyson da Silva @ 2026-07-17 1:43 UTC (permalink / raw)
To: danielmaraboo; +Cc: corbet, linux-doc, Álysson Gleyson da Silva
Translate the documentation on security bugs into Brazilian Portuguese,
maintaining consistency with the original formatting rules.
Signed-off-by: Álysson Gleyson da Silva <alyssongleyson.dev@gmail.com>
---
Documentation/translations/pt_BR/index.rst | 1 +
.../pt_BR/process/security-bugs.rst | 370 ++++++++++++++++++
2 files changed, 371 insertions(+)
create mode 100644 Documentation/translations/pt_BR/process/security-bugs.rst
diff --git a/Documentation/translations/pt_BR/index.rst b/Documentation/translations/pt_BR/index.rst
index 7a488f662..796fe9bca 100644
--- a/Documentation/translations/pt_BR/index.rst
+++ b/Documentation/translations/pt_BR/index.rst
@@ -78,3 +78,4 @@ kernel e sobre como ver seu trabalho integrado.
Processo do subsistema SoC <process/maintainer-soc>
Conformidade de DTS para SoC <process/maintainer-soc-clean-dts>
Processo do subsistema KVM x86 <process/maintainer-kvm-x86>
+ Falhas de segurança <process/security-bugs>
diff --git a/Documentation/translations/pt_BR/process/security-bugs.rst b/Documentation/translations/pt_BR/process/security-bugs.rst
new file mode 100644
index 000000000..7d5ae2faa
--- /dev/null
+++ b/Documentation/translations/pt_BR/process/security-bugs.rst
@@ -0,0 +1,370 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+.. _pt_BR_securitybugs:
+
+Falhas de segurança
+===================
+
+Os desenvolvedores do kernel Linux levam a segurança muito a sério. Como tal,
+gostaríamos de saber quando uma falha de segurança é encontrada para que ela
+possa ser corrigida e divulgada o mais rápido possível.
+
+Preparando seu relatório
+------------------------
+
+Como em qualquer relatório de bug, um relatório de falha de segurança exige
+muito trabalho de análise por parte dos desenvolvedores, portanto, quanto mais
+informações você puder compartilhar sobre o problema, melhor. Por favor, revise
+o procedimento descrito em Documentation/admin-guide/reporting-issues.rst se
+você não tiver certeza sobre quais informações são úteis. As seguintes
+informações são absolutamente necessárias em **qualquer** relatório de falha de
+segurança:
+
+ * **versão do kernel afetada**: sem indicação de versão, seu relatório não
+ será processado. Uma parte significativa dos relatórios é de bugs que já
+ foram corrigidos, portanto, é extremamente importante que as
+ vulnerabilidades sejam verificadas em versões recentes (árvore de
+ desenvolvimento ou a versão estável mais recente), pelo menos verificando
+ se o código não mudou desde a versão onde foi detectado.
+
+ * **descrição do problema**: uma descrição detalhada do problema, com rastros
+ mostrando sua manifestação, e por que você considera o comportamento
+ observado como um problema no Kernel, é necessária.
+
+ * **reproduzir**: os desenvolvedores precisarão ser capazes de reproduzir o
+ problema para considerar uma correção como eficaz. Isso inclui tanto uma
+ maneira de acionar o problema quanto uma maneira de confirmar que ele
+ ocorre. Será necessário um reprodutor com dependências de baixa
+ complexidade (código-fonte, script de shell, sequência de instruções,
+ imagem de sistema de arquivos, etc). Executáveis apenas binários não são
+ aceitos. Exploits funcionais são extremamente úteis e não serão divulgados
+ sem o consentimento do relator, a menos que já sejam públicos. Por
+ definição, se um problema não pode ser reproduzido, ele não é explorável,
+ portanto, não é um bug de segurança.
+
+ * **condições**: se o bug depender de certas opções de configuração, sysctls,
+ permissões, temporização, modificações de código, etc., estas devem ser
+ indicadas.
+
+Além disso, as seguintes informações são altamente desejáveis:
+
+ * **localização suspeita do bug**: os nomes dos arquivos e funções onde
+ se suspeita que o bug esteja presente são muito importantes, pelo menos
+ para ajudar a encaminhar o relatório aos mantenedores apropriados. Quando
+ não for possível (por exemplo, "o sistema trava toda vez que executo este
+ comando"), a equipe de segurança ajudará a identificar a origem do bug.
+
+ * **uma proposta de correção**: os relatores de bugs que analisaram a causa
+ de uma falha no código-fonte quase sempre têm uma ideia precisa de como
+ corrigi-lo, porque passaram muito tempo estudando o problema e suas
+ implicações. Propor uma correção testada poupará muito tempo dos
+ mantenedores, mesmo que a correção acabe não sendo a correta, pois ajuda a
+ entender o bug. Ao propor uma correção testada, por favor, formate-a
+ sempre de uma maneira que possa ser mesclada imediatamente (consulte
+ Documentation/process/submitting-patches.rst). Isso evitará algumas trocas
+ de mensagens caso ela seja aceita, e você receberá o crédito por
+ encontrar e corrigir o problema. Observe que, neste caso, apenas uma tag
+ ``Signed-off-by:`` é necessária, sem ``Reported-by:`` quando o relator e
+ o autor forem a mesma pessoa.
+
+ * **mitigações**: com muita frequência, durante a análise de um bug,
+ surgem algumas maneiras de mitigar o problema. É útil compartilhá-las,
+ pois podem ser úteis para manter os usuários finais protegidos durante o
+ tempo que levam para aplicar a correção.
+
+O que se qualifica como um bug de segurança
+-------------------------------------------
+
+É importante que a maioria dos bugs seja tratada publicamente, de modo a
+envolver o maior público possível e encontrar a melhor solução. Por natureza,
+bugs que são tratados em discussões fechadas entre um pequeno conjunto de
+participantes têm menos probabilidade de produzir a melhor correção possível
+(por exemplo, risco de perder casos de uso válidos, capacidades de testes
+limitadas).
+
+Acontece que a maioria dos bugs relatados por meio da equipe de segurança são
+apenas bugs comuns que foram qualificados incorretamente como bugs de segurança
+devido à falta de conhecimento do modelo de ameaças do kernel Linux, conforme
+descrito em Documentation/process/threat-model.rst, e deveriam ter sido
+enviados através dos canais normais descritos em
+Documentation/admin-guide/reporting-issues.rst
+
+A lista de segurança existe para bugs urgentes que concedem a um atacante uma
+capacidade que ele não deveria ter em um sistema de produção corretamente
+configurado, e que podem ser facilmente explorados, representando uma ameaça
+iminente para muitos usuários. Antes de relatar, considere se o problema
+realmente ultrapassa um limite de confiança em tal sistema.
+
+**Se você recorreu a assistência de IA para identificar um bug, você deve
+tratá-lo como público**. Embora você possa ter motivos válidos para acreditar
+que não seja, a experiência da equipe de segurança mostra que os bugs
+descobertos desta forma surgem sistematicamente e de forma simultânea entre
+múltiplos pesquisadores, frequentemente no mesmo dia. Neste caso, não
+compartilhe publicamente um reprodutor, pois isso poderia causar danos não
+intencionais; apenas mencione que um está disponível e os mantenedores poderão
+solicitá-lo privadamente se precisarem.
+
+Se você não tiver certeza se um problema se qualifica, opte por relatar de
+forma privada: a equipe de segurança prefere triar um relatório limítrofe
+a perder uma vulnerabilidade real. Relatar bugs comuns na lista de segurança,
+no entanto, não faz com que eles andem mais rápido e consome a capacidade de
+triagem de que outros relatórios precisam.
+
+Identificando contatos
+----------------------
+
+A maneira mais eficaz de relatar um bug de segurança é enviá-lo diretamente
+aos mantenedores do subsistema afetado e Cc: para a equipe de segurança do
+kernel Linux. Não o envie para uma lista pública nesta fase, a menos que você
+tenha bons motivos para considerar o problema como público ou trivial de ser
+descoberto (por exemplo, resultado de uma ferramenta automatizada de varredura
+de vulnerabilidades amplamente disponível que possa ser repetida por qualquer
+pessoa, ou o uso de ferramentas baseadas em IA).
+
+Se você estiver enviando um relatório de problemas que afetam várias partes no
+kernel, mesmo que sejam problemas bastante semelhantes, envie mensagens
+individuais (pense que os mantenedores não trabalharão todos nos problemas ao
+mesmo tempo). A única exceção é quando um problema diz respeito a partes
+intimamente relacionadas, mantidas pelo exato mesmo subconjunto de
+mantenedores, e espera-se que essas partes sejam todas corrigidas de uma só vez
+pelo mesmo commit; então pode ser aceitável relatá-las de uma vez.
+
+Uma dificuldade para a maioria dos relatores de primeira viagem é descobrir a
+lista certa de destinatários para enviar um relatório. No kernel Linux, todos
+os mantenedores oficiais são confiáveis, portanto as consequências de incluir
+acidentalmente o mantenedor errado são apenas um pequeno ruído para essa
+pessoa, ou seja, nada dramático. Sendo assim, um método adequado para descobrir
+a lista de mantenedores (o qual os oficiais de segurança do kernel usam) é
+contar com o script get_maintainer.pl, ajustado para relatar apenas
+mantenedores. Este script, quando recebe um nome de arquivo, procurará por seu
+caminho no arquivo MAINTAINERS para deduzir uma lista hierárquica de
+mantenedores relevantes. Chamá-lo pela primeira vez com o nível mais refinado
+de filtragem retornará, na maioria das vezes, uma lista curta de mantenedores
+deste arquivo específico::
+
+ $ ./scripts/get_maintainer.pl --no-l --no-r --pattern-depth 1 \
+ drivers/example.c
+ Developer One <dev1@example.com> (maintainer:example driver)
+ Developer Two <dev2@example.org> (maintainer:example driver)
+
+Estes dois mantenedores devem então receber a mensagem. Se o comando não
+retornar nada, isso significa que o arquivo afetado faz parte de um subsistema
+mais amplo, portanto devemos ser menos específicos::
+
+ $ ./scripts/get_maintainer.pl --no-l --no-r drivers/example.c
+ Developer One <dev1@example.com> (maintainer:example subsystem)
+ Developer Two <dev2@example.org> (maintainer:example subsystem)
+ Developer Three <dev3@example.com> (maintainer:example subsystem [GENERAL])
+ Developer Four <dev4@example.org> (maintainer:example subsystem [GENERAL])
+
+Aqui, escolher os primeiros, mais específicos, é suficiente. Quando a lista for
+longa, é possível produzir uma lista de endereços de e-mail delimitada por
+vírgulas em uma única linha adequada para o uso no campo TO: de um cliente de
+e-mail como este::
+
+ $ ./scripts/get_maintainer.pl --no-tree --no-l --no-r --no-n --m \
+ --no-git-fallback --no-substatus --no-rolestats --no-multiline \
+ --pattern-depth 1 drivers/example.c
+ dev1@example.com, dev2@example.org
+
+ou este para a lista mais ampla::
+
+ $ ./scripts/get_maintainer.pl --no-tree --no-l --no-r --no-n --m \
+ --no-git-fallback --no-substatus --no-rolestats --no-multiline \
+ drivers/example.c
+ dev1@example.com, dev2@example.org, dev3@example.com, dev4@example.org
+
+Se a esta altura você ainda estiver enfrentando dificuldades para identificar
+os mantenedores corretos, e apenas neste caso, é possível enviar seu
+relatório apenas para a equipe de segurança do kernel Linux. Sua mensagem
+será triada e você receberá instruções sobre quem contatar, se necessário.
+Sua mensagem poderá igualmente ser encaminhada como está para os mantenedores
+relevantes.
+
+Uso responsável de IA para encontrar bugs
+-----------------------------------------
+
+Uma fração significativa dos relatórios de bugs enviados à equipe de segurança
+é, na verdade, o resultado de revisões de código assistidas por ferramentas de
+IA. Embora isso possa ser um meio eficiente de encontrar bugs em áreas
+raramente exploradas, causa uma sobrecarga nos mantenedores, que às vezes são
+forçados a ignorar tais relatórios devido à sua má qualidade ou precisão. Sendo
+assim, os relatores devem ter um cuidado especial com vários pontos que tendem
+a tornar esses relatórios desnecessariamente difíceis de lidar:
+
+ * **Comprimento**: Os relatórios gerados por IA tendem a ser excessivamente
+ longos, contendo várias seções e detalhes em excesso. Isso dificulta a
+ identificação de informações importantes, como arquivos afetados, versões e
+ impacto. Por favor, certifique-se de que um resumo claro do problema e
+ todos os detalhes críticos sejam apresentados primeiro. Não exija que os
+ engenheiros de triagem analisem várias páginas de texto. Configure suas
+ ferramentas para produzir relatórios concisos e em estilo humano.
+
+ * **Formatação**: A maioria dos relatórios gerados por IA está repleta de
+ tags Markdown. Essas decorações complicam a busca por informações
+ importantes e não sobrevivem aos processos de citação envolvidos no
+ encaminhamento ou nas respostas. Por favor, sempre converta seu relatório
+ para texto simples sem quaisquer decorações de formatação antes de
+ enviá-lo.
+
+ * **Avaliação de Impacto**: Muitos relatórios gerados por IA carecem de uma
+ compreensão do modelo de ameaças do kernel (consulte
+ Documentation/process/threat-model.rst) e fazem de tudo para inventar
+ consequências teóricas. Isso adiciona ruído e complica a triagem. Por
+ favor, limite-se a fatos verificáveis (por exemplo, "este bug permite que
+ qualquer usuário obtenha CAP_NET_ADMIN") sem enumerar implicações
+ especulativas. Faça com que sua ferramenta leia esta documentação como
+ parte do processo de avaliação.
+
+ * **Reproduzidor**: As ferramentas baseadas em IA são frequentemente capazes
+ de gerar reproduzidores. Por favor, certifique-se sempre de que sua
+ ferramenta forneça um e teste-o exaustivamente. Se o reproduzidor não
+ funcionar, ou se a ferramenta não puder produzir um, a validade do
+ relatório deve ser seriamente questionada. Observe que, como o relatório
+ será postado em uma lista pública, o reproduzidor só deve ser compartilhado
+ mediante solicitação dos mantenedores.
+
+ * **Propor uma Correção:** muitas ferramentas de IA são na verdade melhores
+ em escrever código do que em avaliá-lo. Por favor, peça à sua ferramenta
+ para propor uma correção e teste-a antes de relatar o problema.
+ Se a correção não puder ser testada porque depende de hardware raro ou de
+ protocolos de rede quase extintos, é provável que o problema não seja um
+ bug de segurança. Em qualquer caso, se uma correção for proposta, ela deve
+ aderir a Documentation/process/submitting-patches.rst e incluir uma tag
+ 'Fixes:' designando o commit que introduziu o bug.
+
+A falha em considerar estes pontos expõe seu relatório ao risco de ser
+ignorado.
+
+Use o bom senso ao avaliar o relatório. Se o arquivo afetado não tiver sido
+alterado por mais de um ano e for mantido por um único indivíduo, é provável
+que o uso tenha diminuído e os usuários expostos sejam virtualmente
+inexistentes (por exemplo, drivers para hardware muito antigo, sistemas de
+arquivos obsoletos). Nesses casos, não há necessidade de consumir o tempo de
+um mantenedor com um relatório sem importância. Se o problema for claramente
+trivial e publicamente detectável, você deve relatá-lo diretamente às listas
+de discussão públicas.
+
+Enviando o relatório
+--------------------
+
+Os relatórios devem ser enviados exclusivamente por e-mail. Por favor, use um
+endereço de e-mail funcional, de preferência o mesmo que você deseja que
+apareça nas tags ``Reported-by``, se houver. Se não tiver certeza, envie o seu
+relatório para você mesmo primeiro.
+
+A equipe de segurança e os mantenedores quase sempre exigem informações
+adicionais além das fornecidas inicialmente em um relatório e dependem de uma
+colaboração ativa e eficiente com o relator para realizar testes adicionais
+(por exemplo, verificar versões, opções de configuração, mitigações ou
+patches). Antes de entrar em contato com a equipe de segurança, o relator deve
+certificar-se de que está disponível para explicar suas descobertas, participar
+de discussões e executar testes adicionais. Relatórios nos quais o relator não
+responde prontamente ou não consegue discutir suas descobertas de forma eficaz
+podem ser abandonados se a comunicação não melhorar rapidamente.
+
+O relatório deve ser enviado aos mantenedores. Se houver dois ou menos
+destinatários em sua mensagem, você também deve sempre colocar em Cc: a equipe
+de segurança do kernel Linux, que garantirá que a mensagem seja entregue às
+pessoas corretas e poderá auxiliar pequenas equipes de mantenedores com
+processos com os quais eles possam não estar familiarizados. Para equipes
+maiores, coloque em Cc: a equipe de segurança do kernel Linux em seus primeiros
+relatórios ou ao buscar ajuda específica, como ao reenviar uma mensagem que não
+obteve resposta dentro de uma semana. Assim que você se sentir confortável com
+o processo após alguns relatórios, não será mais necessário colocar a lista de
+segurança em Cc: ao enviar para equipes grandes. A equipe de segurança do
+kernel Linux pode ser contatada por e-mail em security@kernel.org. Esta é uma
+lista privada de oficiais de segurança que ajudarão a verificar o relatório de
+bug e auxiliarão os desenvolvedores que trabalham em uma correção. É possível
+que a equipe de segurança traga ajuda extra de mantenedores da área para
+entender e corrigir a vulnerabilidade de segurança.
+
+Por favor, envie e-mails em **texto simples** sem anexos, sempre que possível.
+É muito mais difícil ter uma discussão com citações de contexto sobre um
+problema complexo se todos os detalhes estiverem ocultos em anexos. Pense nisso
+como uma :doc:`regular path submission <../process/submitting-patches>`
+(mesmo que você ainda não tenha um patch): descreva o problema e o impacto,
+liste as etapas de reprodução e siga com uma proposta de correção, tudo em
+texto simples. Relatórios formatados em Markdown, HTML e RST são
+particularmente malvistos, pois são bastante difíceis de ler por humanos e
+incentivam o uso de visualizadores dedicados, às vezes online, o que por
+definição não é aceitável para um relatório de segurança confidencial. Note
+que alguns clientes de e-mail tendem a corromper a formatação de texto simples
+por padrão; por favor, consulte Documentation/process/email-clients.rst para
+mais informações.
+
+Divulgação e informações sob embargo
+------------------------------------
+
+A lista de segurança não é um canal de divulgação. Para isso, veja Coordenação
+abaixo.
+
+Assim que uma correção robusta for desenvolvida, o processo de lançamento é
+iniciado. Correções para bugs publicamente conhecidos são lançadas
+imediatamente.
+
+Embora nossa preferência seja lançar correções para bugs publicamente não
+divulgados assim que estiverem disponíveis, isso pode ser adiado a pedido do
+relator ou de uma parte afetada por até 7 dias corridos a partir do início do
+processo de lançamento, com uma extensão excepcional para 14 dias corridos se
+for acordado que a criticidade do bug exige mais tempo. O único motivo válido
+para adiar a publicação de uma correção é acomodar a logística de QA e as
+implantações em larga escala que exigem coordenação de lançamento.
+
+Embora as informações sob embargo possam ser compartilhadas com indivíduos de
+confiança para o desenvolvimento de uma correção, tais informações não serão
+publicadas juntamente com a correção ou em qualquer outro canal de divulgação
+sem a permissão do relator. Isso inclui, mas não se limita ao relatório de bug
+original e discussões de acompanhamento (se houver), exploits, informações de
+CVE ou a identidade do relator.
+
+Em outras palavras, nosso único interesse é fazer com que os bugs sejam
+corrigidos. Todas as outras informações enviadas à lista de segurança e
+quaisquer discussões de acompanhamento do relatório são tratadas de forma
+confidencial, mesmo após o término do embargo, perpetuamente.
+
+Coordenação com outros grupos
+-----------------------------
+
+Embora a equipe de segurança do kernel se concentre exclusivamente em corrigir
+bugs, outros grupos se concentram em corrigir problemas em distribuições e em
+coordenar a divulgação entre fornecedores de sistemas operacionais. A
+coordenação é geralmente tratada pela lista de discussão "linux-distros" e a
+divulgação pela lista pública "oss-security", ambas intimamente relacionadas
+e apresentadas na wiki da linux-distros:
+https://oss-security.openwall.org/wiki/mailing-lists/distros
+
+Por favor, note que as respectivas políticas e regras são diferentes, já que as
+3 listas buscam objetivos distintos. A coordenação entre a equipe de segurança
+do kernel e outras equipes é difícil porque para a equipe de segurança do
+kernel os embargos ocasionais (sujeitos a um número máximo de dias permitido)
+começam a partir da disponibilidade de uma correção, enquanto para a
+"linux-distros" eles começam a partir da postagem inicial na lista,
+independentemente da disponibilidade de uma correção.
+
+Como tal, a equipe de segurança do kernel recomenda fortemente que, como
+relator de um potencial problema de segurança, você NÃO contate a lista de
+discussão "linux-distros" ATÉ que uma correção seja aceita pelos mantenedores
+do código afetado e você tenha lido a página wiki das distribuições acima e
+compreendido totalmente os requisitos que o contato com a "linux-distros"
+imporá a você e à comunidade do kernel. Isso também significa que, em geral,
+não faz sentido colocar ambas as listas em Cc: ao mesmo tempo, exceto talvez
+para coordenação se e enquanto uma correção aceita ainda não tiver sido
+mesclada. Em outras palavras, até que uma correção seja aceita, não coloque
+em Cc: "linux-distros", e após ela ser mesclada, não coloque em Cc: a equipe
+de segurança do kernel.
+
+Atribuição de CVE
+-----------------
+
+A equipe de segurança não atribui CVEs, nem os exigimos para relatórios ou
+correções, pois isso pode complicar desnecessariamente o processo e adiar o
+tratamento do bug. Se um relator desejar que um identificador CVE seja
+atribuído para um problema confirmado, ele pode entrar em contato com a
+:doc:`kernel CVE assignment team<../process/cve>` para obter um.
+
+Acordo de não divulgação
+------------------------
+
+A equipe de segurança do kernel Linux não é um órgão formal e, portanto, é
+incapaz de celebrar quaisquer acordos de não divulgação.
--
2.55.0
^ permalink raw reply related
* Re: [PATCH v7 07/17] iio: test: add kunit tests for channel prefix naming generation
From: Jonathan Cameron @ 2026-07-17 0:27 UTC (permalink / raw)
To: Andy Shevchenko
Cc: Rodrigo Alencar, Jonathan Cameron, Rodrigo Alencar via B4 Relay,
linux-iio, devicetree, linux-kernel, linux-doc, linux-hardening,
Lars-Peter Clausen, Michael Hennerich, David Lechner,
Andy Shevchenko, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Philipp Zabel, Jonathan Corbet, Shuah Khan, Kees Cook,
Gustavo A. R. Silva
In-Reply-To: <alcpb-7GVSfk6jFB@ashevche-desk.local>
On Wed, 15 Jul 2026 09:32:15 +0300
Andy Shevchenko <andriy.shevchenko@intel.com> wrote:
> On Tue, Jul 14, 2026 at 06:08:12PM -0700, Jonathan Cameron wrote:
> > On Mon, 13 Jul 2026 10:52:56 +0100
> > Rodrigo Alencar <455.rodrigo.alencar@gmail.com> wrote:
> > > On 12/07/26 02:09, Jonathan Cameron wrote:
> > > > On Tue, 07 Jul 2026 15:04:28 +0100
> > > > Rodrigo Alencar via B4 Relay <devnull+rodrigo.alencar.analog.com@kernel.org> wrote:
>
> ...
>
> > > > > Because __iio_chan_prefix_emit() is static, the test translation unit
> > > > > is pulled into industrialio-core.c.
>
> KUnit also has static/non-static automation via a macro (defined in the
> kunit/visibility.h) and I see that's used in the below example.
>
> > > > Isn't there some magic route cases like this that makes it non static
> > > > only when self tests are enabled?
> > > > Claude tells me to look at include/kunit/visibility.h
> > >
> > > There is, Although I think that using
> > >
> > > #if IS_ENABLED(CONFIG_IIO_CHANNEL_PREFIX_KUNIT_TEST)
> > > #include "test/iio-test-channel-prefix.c"
> > > #endif
> > >
> > > was more straight forward, less invasive and easier to change than..
>
> Maybe, but thanks to this thread, I fixed other modules that use their own
> approach to use the standard KUnit infra for this (as below).
>
> > > /* In "drivers/iio/industrialio-core.c" */
> > >
> > > #include <kunit/visibility.h>
> > > ...
> > > VISIBLE_IF_KUNIT ssize_t __iio_chan_prefix_emit(...)
> > > {
> > > ...
> > > }
> > > EXPORT_SYMBOL_IF_KUNIT(__iio_chan_prefix_emit);
> > >
> > > /* In "iio_core.h" */
> > >
> > > #if IS_ENABLED(CONFIG_KUNIT)
> > > ssize_t __iio_chan_prefix_emit(...);
> > > #endif
> > >
> > > /* In "drivers/iio/test/iio-test-channel-prefix.c" */
> > >
> > > #include <kunit/visibility.h>
> > > #include <iio_core.h>
> > > ...
> > > MODULE_IMPORT_NS("EXPORTED_FOR_KUNIT_TESTING");
> > > ...
> > > // Use __iio_chan_prefix_emit() in tests
> >
> > I'd rather this wasn't built into the core module. So prefer you jump
> > though those hoops.
>
> Hmm... The above (while being verbose) is the standard way of how we export
> symbols for KUnit tests. Do you have a better alternative that everyone can
> use? (Not only IIO subsystem.)
Ah. I put that comment in an unhelpful place. I meant the include above not this bit.
Absolutely fine with the exports being there.
Jonathan
>
^ permalink raw reply
* Re: [PATCH 1/2] mm/zswap: Fix global shrinker when memory cgroup is disabled
From: Yosry Ahmed @ 2026-07-17 0:08 UTC (permalink / raw)
To: Nhat Pham
Cc: Hao Jia, Andrew Morton, tj, hannes, shakeel.butt, mhocko, mkoutny,
chengming.zhou, muchun.song, roman.gushchin, linux-mm,
linux-kernel, linux-doc, Hao Jia, stable
In-Reply-To: <CAKEwX=Ob9shetuJK4y=NZods3KkT=U_3W4S_ALzxX3EDLomWEA@mail.gmail.com>
On Thu, Jul 16, 2026 at 11:19 AM Nhat Pham <nphamcs@gmail.com> wrote:
>
> On Wed, Jul 15, 2026 at 7:21 PM Hao Jia <jiahao.kernel@gmail.com> wrote:
> >
> >
> >
> > On 2026/7/16 00:13, Yosry Ahmed wrote:
> > > On Wed, Jul 15, 2026 at 5:31 AM Hao Jia <jiahao.kernel@gmail.com> wrote:
> > >>
> > >>
> > >>
> > >> On 2026/7/15 10:31, Andrew Morton wrote:
> > >>> On Tue, 14 Jul 2026 09:52:59 -0700 Yosry Ahmed <yosry@kernel.org> wrote:
> > >>>
> > >>>>> When memory cgroup is disabled, mem_cgroup_iter() always returns NULL.
> > >>>>> Therefore, the global shrinker shrink_worker() always takes the !memcg
> > >>>>> branch. After MAX_RECLAIM_RETRIES empty walks, the worker simply gives up,
> > >>>>> so it fails to write back anything.
> > >>>>>
> > >>>>> Therefore, when memory cgroup is disabled, fall through with the !memcg
> > >>>>> branch and shrink the root memcg directly.
> > >>>>>
> > >>>>> With memcg disabled, shrink_memcg() only returns -ENOENT when the root
> > >>>>> LRU is empty, which means the total pages are already below thr. The
> > >>>>> loop then safely bails out via the zswap_total_pages() <= thr check.
> > >>>>> For any other return value from shrink_memcg(), the loop is guaranteed
> > >>>>> to terminate, either after MAX_RECLAIM_RETRIES failures or once the
> > >>>>> threshold is met.
> > >>>>>
> > >>>>> Fixes: a65b0e7607cc ("zswap: make shrinking memcg-aware")
> > >>>>> Cc: stable@vger.kernel.org
> > >>>>> Suggested-by: Nhat Pham <nphamcs@gmail.com>
> > >>>>> Acked-by: Nhat Pham <nphamcs@gmail.com>
> > >>>>> Acked-by: Yosry Ahmed <yosry@kernel.org>
> > >>>>> Reported-by: Yosry Ahmed <yosry@kernel.org>
> > >>>>> Closes: https://lore.kernel.org/all/CAO9r8zPVzMKFbCixxD-qgtRrkFxWVrHiZZeLc=eyTPKPVQgX4g@mail.gmail.com
> > >>>>> Signed-off-by: Hao Jia <jiahao1@lixiang.com>
> > >>>>
> > >>>> Patch 2 doesn't really depend on this one, right?
> > >>>>
> > >>>> If that's the case I think this can (and should be) picked up
> > >>>> separately as a hotfix. Andrew, WDYT?
> > >>>
> > >>> Please update the changelog to clearly describe the userspace-visible
> > >>> effects of the bug, thanks.
> > >>
> > >> I am not entirely sure if my understanding is correct here, but maybe I
> > >> should add something like this to the commit message?
> > >>
> > >> When cgroup_disable=memory is used (or with CONFIG_MEMCG=n), the global
> > >> shrinker fails to write back any pages. Consequently, the zswap pool
> > >> fills up to its limit and rejects further storage, preventing memory
> > >> pressure from being offloaded to the backing swap device.
> > >
> > > I think you can simply write that zswap writeback when the limit is
> > > hit is broken when memcg is disabled.
> > Will do. Thanks!
> >
> > >
> > >>
> > >>> Also, AI review has flagged several possible issues, all appear to be
> > >>> serious:
> > >>> https://sashiko.dev/#/patchset/20260714081510.16895-1-jiahao.kernel@gmail.com
> > >>
> > >> For AI review comments on this patch:
> > >> I suspect this scenario might only exist in theory. For zswap LRU to be
> > >> empty while zswap_total_pages() > thr holds true, it would require a
> > >> prolonged state where there are always more than thr zswap entries on
> > >> the zswap LRU whenever zswap_total_pages() > thr is evaluated, yet the
> > >> zswap LRU happens to be empty during shrink_memcg(root_memcg).
> > >>
> > >> If we want to fix this, perhaps we could do something like this?
> > >>
> > >> Yosry, Nhat, what are your thoughts on this?
> > >
> > > Do we need to do this? The last paragraph in your changelog explains
> > > why this can't happen because zswap_total_pages() should be 0 in this
> > > case. Did I miss something?
> >
> > The loop would require the following sequence to repeat indefinitely:
> >
> > 1、zswap_total_pages() > thr evaluates to true.
> > 2、During shrink_memcg(root_memcg), the zswap LRU is concurrently drained
> > to empty.
> > 3、Before zswap_total_pages() > thr is evaluated again, the zswap LRU is
> > heavily refilled such that zswap_total_pages() > thr holds true once more.
> >
> > For our case to manifest, it would require zswap_total_pages() and the
> > zswap LRU state to repeatedly hit this exact window with perfect
> > alignment over a **prolonged period**. Therefore, I suspect this
> > scenario might only exist in theory.
>
> Yeah seems very artificial indeed.
>
> If we want to be extra careful here, maybe we can put a cond_resched()
> there or replace the continue with goto resched or sth?
I think it's also possible with the current code with memcg enabled.
It's still possible that the shrinker puts usage under the acceptance
threshold and then a new zswap stores puts it back above the
threshold, and so on.
Perhaps as Nhat said, just goto resched instead of continue if we're
really worried, but I think even that is not really necessary.
I think we generally want to clean up and simplify the shrinking loop
in zswap_shrinker(), but I don't have any great ideas.
^ permalink raw reply
* Re:
From: Daniel Pereira @ 2026-07-16 23:25 UTC (permalink / raw)
To: Jonathan Corbet; +Cc: linux-doc
In-Reply-To: <87ldba5z1u.fsf@trenco.lwn.net>
Em qui., 16 de jul. de 2026 às 19:50, Jonathan Corbet <corbet@lwn.net> escreveu:
>
> I've applied the series, thanks.
>
> For future reference, a lot of the Portuguese translation patches are
> giving me conflicts on Documentation/translations/pt_BR/index.rst. That
> happens when you have a big file that everybody just appends to. Might
> I request that you start managing the index.rst structure in the same
> way as the English docs so that these additions are spread out and don't
> conflict? That would also yield a more readable result for the
> translation.
>
> Thanks,
>
> jon
Hi Jon,
Thank you for the guidance on managing the index.rst structure.
Regarding the conflict issue, I would be happy to take on the
responsibility of organizing these patches to help save you time.
Would you prefer if I acted as a point of contact for these
translations? I could collect submissions from contributors, merge
them into the structure correctly, and then send them to you as a
single, cleaned-up set of patches.
Please let me know if this workflow would be helpful or if you have a
different preference for how I should handle these conflicts.
Best regards,
^ permalink raw reply
* Re:
From: Jonathan Corbet @ 2026-07-16 22:50 UTC (permalink / raw)
To: Daniel Pereira; +Cc: linux-doc, Daniel Pereira
In-Reply-To: <20260716021013.48025-1-danielmaraboo@gmail.com>
Daniel Pereira <danielmaraboo@gmail.com> writes:
> Subject: [PATCH 0/4] doc: pt_BR: Add translations for process and guidelines
>
> Hello,
>
> This patch series adds the Brazilian Portuguese (pt_BR) translations for
> several essential files regarding community standards, submit checklists, and
> best development practices.
>
> Specifically, this series translates:
> 1. The submit-checklist guide.
> 2. The Code of Conduct interpretation document.
> 3. The core Contributor Covenant Code of Conduct.
> 4. The deprecated interfaces, attributes, and language features guide.
>
> All files have been formatted to respect the 80-character line limit and
> the main index.rst has been updated accordingly. The Sphinx documentation build
> (make htmldocs) compiles successfully with no warnings or errors related to
> these new files.
I've applied the series, thanks.
For future reference, a lot of the Portuguese translation patches are
giving me conflicts on Documentation/translations/pt_BR/index.rst. That
happens when you have a big file that everybody just appends to. Might
I request that you start managing the index.rst structure in the same
way as the English docs so that these additions are spread out and don't
conflict? That would also yield a more readable result for the
translation.
Thanks,
jon
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox