* [PATCH v8 0/9] dmaengine: dw-axi-dmac: fix termination and pause acknowledgment
@ 2026-11-19 16:00 James Hilliard
2026-11-19 16:00 ` [PATCH v8 1/9] dmaengine: dw-axi-dmac: keep cyclic mode in the descriptor James Hilliard
` (8 more replies)
0 siblings, 9 replies; 18+ messages in thread
From: James Hilliard @ 2026-11-19 16:00 UTC (permalink / raw)
To: Eugeniy Paltsev, Vinod Koul, Frank Li, Sergey Khimich,
Andy Shevchenko, Sia Jee Heng, Baruch Siach
Cc: Mitch Barnett, dmaengine, linux-kernel, James Hilliard, stable,
Frank Li, Wang YuWei
Keep cyclic mode in each descriptor so failed preparation and queued
work cannot change the completion handling of another transfer.
Use the layout-aware channel enable check during termination and retain
transfer resources when the channel does not stop. Preserve suspension
status for the pause and resume paths, including acknowledgments that
arrive after a polling timeout or during another channel's interrupt.
Keep a pending pause across cyclic segment completion and resume the
segment only after the client releases the pause. Track the active
descriptor until completion so queued work cannot reissue it while its
interrupt is pending.
Retain terminated descriptors until their callbacks have drained, including
cyclic descriptors that error handling has moved to the completed list.
Serialize interrupt status collection with channel reuse so an old event
cannot complete a replacement transfer.
Require at least one IRQ before registering the controller. Keep shared
IRQ handlers out of uninitialized or suspended hardware, and drain them
before disabling clocks or removing channel devices.
This series uses the shared virt-dma descriptor-retirement helper [0].
[0] https://lore.kernel.org/r/20261115-submit-sun4i-dma-callback-lifetime-v1-v2-1-fb7d258702f8@gmail.com/
Assisted-by: Codex:gpt-6-astra
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
---
Changes in v8:
- Use read_poll_timeout_atomic() for stop and suspend polling and a
scoped IRQ-save guard during termination, following Frank's review.
- Propagate deferred IRQ discovery and reject controllers without IRQs
before registering their DMA channels.
- Keep controller interrupts masked until DMA registration completes.
Serialize the hardware mask with handler admission and rearming across
power transitions, following Sashiko's review of v7 patch 8.
- Leave a suspended controller powered down on removal.
- Link to v7: https://patch.msgid.link/20261117-submit-dw-axi-dma-stop-pause-v1-v7-0-d5f2309afd72@gmail.com
Changes in v7:
- Guard shared interrupts until hardware and channel registration are ready,
following Sashiko's review, and hold an active runtime PM reference while
handling them.
- Balance probe/removal power references and drain handlers before clock
shutdown, including forced system suspend.
- Link to v6: https://patch.msgid.link/20261115-submit-dw-axi-dma-stop-pause-v1-v6-0-bcb23d1e7029@gmail.com
Changes in v6:
- Retain descriptors through callback synchronization using the posted
virt-dma helper, covering pending, executing and error-completed cyclic
callbacks raised in Mitch's follow-up.
- Serialize interrupt status collection with channel reuse and clear old
completion and error status after successful termination.
- Link to v5: https://patch.msgid.link/20261113-submit-dw-axi-dma-stop-pause-v1-v5-0-8523a27b5788@gmail.com
Changes in v5:
- Use a positive enabled-channel condition in the suspend-status helper,
following Andy's review.
- Initialize the resume result immediately before its first use.
- Separate the lock-protected channel bitfields from the preceding fields.
- Link to v4: https://patch.msgid.link/20261112-submit-dw-axi-dma-stop-pause-v1-v4-0-3ebc24b243db@gmail.com
Changes in v4:
- Add a preparatory fix for cyclic mode leaking between transfers,
following Sashiko's review. Store the mode in each descriptor so
failed preparation, error recovery and queued work cannot change
another transfer's completion handling.
- Link to v3: https://patch.msgid.link/20261110-submit-dw-axi-dma-stop-pause-v1-v3-0-35fdfba8b61d@gmail.com
Changes in v3:
- Wrap the atomic polling calls by argument group and correct their
continuation indentation, following Andy's review.
- Move the termination return-value check simplification into the
channel-enable polling patch.
- Separate the pause-loop conversion to atomic polling from the
pending-request and cyclic-restart changes.
- Annotate the suspend-wait helper with its channel-lock requirement.
- Pack the lock-protected flags into existing padding while keeping
cyclic separately writable, following Andy's review.
- Link to v2: https://patch.msgid.link/20261109-submit-dw-axi-dma-stop-pause-v1-v2-0-64adc48df793@gmail.com
Changes in v2:
- Use readx_poll_timeout_atomic() for the single-argument status helpers
and express the termination polling interval and timeout in milliseconds,
following Andy's review.
- Track pending suspend requests separately from confirmed suspension so
resume can handle acknowledgments received after a pause timeout.
- Use the atomic polling helper, including its final status check, and
report -EAGAIN if resume cannot yet cancel a pending suspend request.
- Defer cyclic restart while paused and track the active descriptor until
completion, preventing resume or issue_pending from reissuing it.
- Reset pause state on successful termination and channel allocation;
clear stale status when starting a new descriptor.
- Link to v1: https://patch.msgid.link/20261108-submit-dw-axi-dma-stop-pause-v1-v1-0-c48ec19e9ee4@gmail.com
To: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
To: Vinod Koul <vkoul@kernel.org>
To: Frank Li <Frank.Li@kernel.org>
To: Sergey Khimich <serghox@gmail.com>
To: Sia Jee Heng <jee.heng.sia@intel.com>
To: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Cc: dmaengine@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
---
James Hilliard (9):
dmaengine: dw-axi-dmac: keep cyclic mode in the descriptor
dmaengine: dw-axi-dmac: poll the correct enable bit during termination
dmaengine: dw-axi-dmac: retain descriptors when termination times out
dmaengine: dw-axi-dmac: use atomic polling for suspend acknowledgment
dmaengine: dw-axi-dmac: preserve pending pause requests
dmaengine: dw-axi-dmac: retain descriptors until callbacks drain
dmaengine: dw-axi-dmac: serialize interrupt status with channel reuse
dmaengine: dw-axi-dmac: require an IRQ before registering the controller
dmaengine: dw-axi-dmac: guard shared IRQs across the controller lifetime
drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c | 311 +++++++++++++++++--------
drivers/dma/dw-axi-dmac/dw-axi-dmac.h | 13 +-
2 files changed, 218 insertions(+), 106 deletions(-)
---
base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
base-tree: git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine.git fixes
change-id: 20261108-submit-dw-axi-dma-stop-pause-v1-c93f1e767169
prerequisite-patch-id: fc343480330da5ff95e5271997fa71107977c2b8
Best regards,
--
James Hilliard <james.hilliard1@gmail.com>
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH v8 1/9] dmaengine: dw-axi-dmac: keep cyclic mode in the descriptor
2026-11-19 16:00 [PATCH v8 0/9] dmaengine: dw-axi-dmac: fix termination and pause acknowledgment James Hilliard
@ 2026-11-19 16:00 ` James Hilliard
2026-11-19 16:00 ` [PATCH v8 2/9] dmaengine: dw-axi-dmac: poll the correct enable bit during termination James Hilliard
` (7 subsequent siblings)
8 siblings, 0 replies; 18+ messages in thread
From: James Hilliard @ 2026-11-19 16:00 UTC (permalink / raw)
To: Eugeniy Paltsev, Vinod Koul, Frank Li, Sergey Khimich,
Andy Shevchenko, Sia Jee Heng, Baruch Siach
Cc: Mitch Barnett, dmaengine, linux-kernel, James Hilliard, stable,
Frank Li
dw_axi_dma_chan_prep_cyclic() sets chan->cyclic before allocating all the
hardware descriptors and leaves it set when preparation fails. The flag
also survives cyclic transfer errors and channel resource release. A
subsequent noncyclic transfer then takes the cyclic completion path,
leaving its cookie incomplete and its descriptor on the issued list.
Its zero period_len can also cause a division by zero.
Resetting the flag on failure or channel allocation is not sufficient:
preparing cyclic work while an ordinary transfer is active changes how
that transfer's completion interrupt is handled.
Store the mode in each descriptor and use the completed descriptor's
mode in the interrupt handler. Cyclic preparation sets only its own
descriptor's flag, while the zero-initialized noncyclic descriptors keep
it clear. Error recovery and channel reuse then use the next descriptor's
mode without relying on channel-wide cleanup.
Fixes: 1deb96c0fa58 ("dmaegine: dw-axi-dmac: Support device_prep_dma_cyclic()")
Closes: https://lore.kernel.org/r/sashiko-outbox-208213@kernel.org/
Cc: stable@vger.kernel.org
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
---
drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c | 35 ++++++++++++--------------
drivers/dma/dw-axi-dmac/dw-axi-dmac.h | 2 +-
2 files changed, 17 insertions(+), 20 deletions(-)
diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
index 7520c78f55cc..6dae63024721 100644
--- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
+++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
@@ -801,7 +801,7 @@ dw_axi_dma_chan_prep_cyclic(struct dma_chan *dchan, dma_addr_t dma_addr,
chan->direction = direction;
desc->chan = chan;
- chan->cyclic = true;
+ desc->cyclic = true;
desc->length = 0;
desc->period_len = period_len;
@@ -1117,25 +1117,23 @@ static void axi_chan_block_xfer_complete(struct axi_dma_chan *chan)
goto out;
}
- if (chan->cyclic) {
- desc = vd_to_axi_desc(vd);
- if (desc) {
- llp = lo_hi_readq(chan->chan_regs + CH_LLP);
- for (i = 0; i < count; i++) {
- hw_desc = &desc->hw_desc[i];
- if (hw_desc->llp == llp) {
- axi_chan_irq_clear(chan, hw_desc->lli->status_lo);
- hw_desc->lli->ctl_hi |= cpu_to_le32(CH_CTL_H_LLI_VALID);
- desc->completed_blocks = i;
-
- if (((hw_desc->len * (i + 1)) % desc->period_len) == 0)
- vchan_cyclic_callback(vd);
- break;
- }
+ desc = vd_to_axi_desc(vd);
+ if (desc->cyclic) {
+ llp = lo_hi_readq(chan->chan_regs + CH_LLP);
+ for (i = 0; i < count; i++) {
+ hw_desc = &desc->hw_desc[i];
+ if (hw_desc->llp == llp) {
+ axi_chan_irq_clear(chan, hw_desc->lli->status_lo);
+ hw_desc->lli->ctl_hi |= cpu_to_le32(CH_CTL_H_LLI_VALID);
+ desc->completed_blocks = i;
+
+ if (((hw_desc->len * (i + 1)) % desc->period_len) == 0)
+ vchan_cyclic_callback(vd);
+ break;
}
-
- axi_chan_enable(chan);
}
+
+ axi_chan_enable(chan);
} else {
/* Remove the completed descriptor from issued list before completing */
list_del(&vd->node);
@@ -1204,7 +1202,6 @@ static int dma_chan_terminate_all(struct dma_chan *dchan)
vchan_get_all_descriptors(&chan->vc, &head);
- chan->cyclic = false;
spin_unlock_irqrestore(&chan->vc.lock, flags);
vchan_dma_desc_free_list(&chan->vc, &head);
diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac.h b/drivers/dma/dw-axi-dmac/dw-axi-dmac.h
index d8dd07efe60f..3c86e5d468ea 100644
--- a/drivers/dma/dw-axi-dmac/dw-axi-dmac.h
+++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac.h
@@ -50,7 +50,6 @@ struct axi_dma_chan {
struct axi_dma_desc *desc;
struct dma_slave_config config;
enum dma_transfer_direction direction;
- bool cyclic;
/* these other elements are all protected by vc.lock */
bool is_paused;
};
@@ -105,6 +104,7 @@ struct axi_dma_desc {
u32 length;
u32 period_len;
u32 nr_hw_descs;
+ bool cyclic;
};
struct axi_dma_chan_config {
--
2.53.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v8 2/9] dmaengine: dw-axi-dmac: poll the correct enable bit during termination
2026-11-19 16:00 [PATCH v8 0/9] dmaengine: dw-axi-dmac: fix termination and pause acknowledgment James Hilliard
2026-11-19 16:00 ` [PATCH v8 1/9] dmaengine: dw-axi-dmac: keep cyclic mode in the descriptor James Hilliard
@ 2026-11-19 16:00 ` James Hilliard
2026-11-19 16:17 ` Frank Li
2026-11-19 16:00 ` [PATCH v8 3/9] dmaengine: dw-axi-dmac: retain descriptors when termination times out James Hilliard
` (6 subsequent siblings)
8 siblings, 1 reply; 18+ messages in thread
From: James Hilliard @ 2026-11-19 16:00 UTC (permalink / raw)
To: Eugeniy Paltsev, Vinod Koul, Frank Li, Sergey Khimich,
Andy Shevchenko, Sia Jee Heng, Baruch Siach
Cc: Mitch Barnett, dmaengine, linux-kernel, James Hilliard, stable
For channels 16 through 31, the channel enable bits occupy bits 32 through
47 of the extended DMAC_CHEN register. dma_chan_terminate_all() only reads
the low word and tests BIT(chan->id), so it can conclude that an upper-bank
channel has stopped while its actual enable bit remains set. Descriptors
can then be freed while hardware still owns them.
Poll axi_chan_is_hw_enable(), which already handles both register banks
and the smaller channel layouts, instead of constructing another mask.
Fixes: 495e18b16e3d ("dmaengine: dw-axi-dmac: Add support DMAX_NUM_CHANNELS > 16")
Cc: stable@vger.kernel.org
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
---
drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
index 6dae63024721..f17a71183050 100644
--- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
+++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
@@ -28,6 +28,7 @@
#include <linux/property.h>
#include <linux/reset.h>
#include <linux/slab.h>
+#include <linux/time64.h>
#include <linux/types.h>
#include "dw-axi-dmac.h"
@@ -1179,17 +1180,17 @@ static irqreturn_t dw_axi_dma_interrupt(int irq, void *dev_id)
static int dma_chan_terminate_all(struct dma_chan *dchan)
{
struct axi_dma_chan *chan = dchan_to_axi_dma_chan(dchan);
- u32 chan_active = BIT(chan->id) << DMAC_CHAN_EN_SHIFT;
unsigned long flags;
- u32 val;
+ bool enabled;
int ret;
LIST_HEAD(head);
axi_chan_disable(chan);
- ret = readl_poll_timeout_atomic(chan->chip->regs + DMAC_CHEN, val,
- !(val & chan_active), 1000, 50000);
- if (ret == -ETIMEDOUT)
+ ret = read_poll_timeout_atomic(axi_chan_is_hw_enable, enabled, !enabled,
+ 1 * USEC_PER_MSEC, 50 * USEC_PER_MSEC,
+ false, chan);
+ if (ret)
dev_warn(dmaengine_chan_dev(dchan),
"%s failed to stop\n", axi_chan_name(chan));
--
2.53.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v8 3/9] dmaengine: dw-axi-dmac: retain descriptors when termination times out
2026-11-19 16:00 [PATCH v8 0/9] dmaengine: dw-axi-dmac: fix termination and pause acknowledgment James Hilliard
2026-11-19 16:00 ` [PATCH v8 1/9] dmaengine: dw-axi-dmac: keep cyclic mode in the descriptor James Hilliard
2026-11-19 16:00 ` [PATCH v8 2/9] dmaengine: dw-axi-dmac: poll the correct enable bit during termination James Hilliard
@ 2026-11-19 16:00 ` James Hilliard
2026-11-19 16:18 ` Frank Li
2026-11-19 16:00 ` [PATCH v8 4/9] dmaengine: dw-axi-dmac: use atomic polling for suspend acknowledgment James Hilliard
` (5 subsequent siblings)
8 siblings, 1 reply; 18+ messages in thread
From: James Hilliard @ 2026-11-19 16:00 UTC (permalink / raw)
To: Eugeniy Paltsev, Vinod Koul, Frank Li, Sergey Khimich,
Andy Shevchenko, Sia Jee Heng, Baruch Siach
Cc: Mitch Barnett, dmaengine, linux-kernel, James Hilliard, stable
dma_chan_terminate_all() warns when the channel enable bit does not clear,
but then frees the descriptors and returns success. Its synchronize
callback only drains virtual-channel callbacks, so terminate_sync() can
also report success while hardware still accesses the transfer.
Return the polling error before detaching descriptors or changing the
peripheral routing. This keeps the transfer resources available while
the channel remains enabled and lets the caller retry termination without
treating the timeout as a successful stop.
Hold the channel lock across disabling, polling and descriptor detachment
so an interrupt handler cannot restart the channel between the stop check
and cleanup.
Fixes: 1deb96c0fa58 ("dmaegine: dw-axi-dmac: Support device_prep_dma_cyclic()")
Cc: stable@vger.kernel.org
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
---
drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c | 37 ++++++++++++++------------
1 file changed, 20 insertions(+), 17 deletions(-)
diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
index f17a71183050..def591fc75ce 100644
--- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
+++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
@@ -9,6 +9,7 @@
#include <linux/bitfield.h>
#include <linux/bitops.h>
+#include <linux/cleanup.h>
#include <linux/delay.h>
#include <linux/device.h>
#include <linux/dmaengine.h>
@@ -1180,30 +1181,32 @@ static irqreturn_t dw_axi_dma_interrupt(int irq, void *dev_id)
static int dma_chan_terminate_all(struct dma_chan *dchan)
{
struct axi_dma_chan *chan = dchan_to_axi_dma_chan(dchan);
- unsigned long flags;
- bool enabled;
int ret;
LIST_HEAD(head);
- axi_chan_disable(chan);
-
- ret = read_poll_timeout_atomic(axi_chan_is_hw_enable, enabled, !enabled,
- 1 * USEC_PER_MSEC, 50 * USEC_PER_MSEC,
- false, chan);
- if (ret)
- dev_warn(dmaengine_chan_dev(dchan),
- "%s failed to stop\n", axi_chan_name(chan));
+ scoped_guard(spinlock_irqsave, &chan->vc.lock) {
+ bool enabled;
- if (chan->direction != DMA_MEM_TO_MEM)
- dw_axi_dma_set_hw_channel(chan, false);
- if (chan->direction == DMA_MEM_TO_DEV)
- dw_axi_dma_set_byte_halfword(chan, false);
+ axi_chan_disable(chan);
+ ret = read_poll_timeout_atomic(axi_chan_is_hw_enable, enabled, !enabled,
+ 1 * USEC_PER_MSEC, 50 * USEC_PER_MSEC,
+ false, chan);
+ if (ret)
+ break;
- spin_lock_irqsave(&chan->vc.lock, flags);
+ if (chan->direction != DMA_MEM_TO_MEM)
+ dw_axi_dma_set_hw_channel(chan, false);
+ if (chan->direction == DMA_MEM_TO_DEV)
+ dw_axi_dma_set_byte_halfword(chan, false);
- vchan_get_all_descriptors(&chan->vc, &head);
+ vchan_get_all_descriptors(&chan->vc, &head);
+ }
- spin_unlock_irqrestore(&chan->vc.lock, flags);
+ if (ret) {
+ dev_warn(dmaengine_chan_dev(dchan),
+ "%s failed to stop\n", axi_chan_name(chan));
+ return ret;
+ }
vchan_dma_desc_free_list(&chan->vc, &head);
--
2.53.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v8 4/9] dmaengine: dw-axi-dmac: use atomic polling for suspend acknowledgment
2026-11-19 16:00 [PATCH v8 0/9] dmaengine: dw-axi-dmac: fix termination and pause acknowledgment James Hilliard
` (2 preceding siblings ...)
2026-11-19 16:00 ` [PATCH v8 3/9] dmaengine: dw-axi-dmac: retain descriptors when termination times out James Hilliard
@ 2026-11-19 16:00 ` James Hilliard
2026-11-19 16:19 ` Frank Li
2026-11-19 16:01 ` [PATCH v8 5/9] dmaengine: dw-axi-dmac: preserve pending pause requests James Hilliard
` (4 subsequent siblings)
8 siblings, 1 reply; 18+ messages in thread
From: James Hilliard @ 2026-11-19 16:00 UTC (permalink / raw)
To: Eugeniy Paltsev, Vinod Koul, Frank Li, Sergey Khimich,
Andy Shevchenko, Sia Jee Heng, Baruch Siach
Cc: Mitch Barnett, dmaengine, linux-kernel, James Hilliard, stable
The pause loop delays after its final status read, so an acknowledgment
that arrives during that delay is cleared without being observed and
the caller receives -EAGAIN.
Use read_poll_timeout_atomic() to check the status after the final delay.
Keep the two-microsecond polling interval, 40-microsecond timeout and
-EAGAIN return value when suspension is not acknowledged.
Fixes: 1fe20f1b8454 ("dmaengine: Introduce DW AXI DMAC driver")
Cc: stable@vger.kernel.org
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
---
drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c | 14 ++++++--------
1 file changed, 6 insertions(+), 8 deletions(-)
diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
index def591fc75ce..8c115d20728b 100644
--- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
+++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
@@ -1219,7 +1219,8 @@ static int dma_chan_pause(struct dma_chan *dchan)
{
struct axi_dma_chan *chan = dchan_to_axi_dma_chan(dchan);
unsigned long flags;
- unsigned int timeout = 20; /* timeout iterations */
+ u32 status;
+ int ret;
u64 val;
spin_lock_irqsave(&chan->vc.lock, flags);
@@ -1250,12 +1251,9 @@ static int dma_chan_pause(struct dma_chan *dchan)
}
}
- do {
- if (axi_chan_irq_read(chan) & DWAXIDMAC_IRQ_SUSPENDED)
- break;
-
- udelay(2);
- } while (--timeout);
+ ret = read_poll_timeout_atomic(axi_chan_irq_read, status,
+ status & DWAXIDMAC_IRQ_SUSPENDED,
+ 2, 40, false, chan);
axi_chan_irq_clear(chan, DWAXIDMAC_IRQ_SUSPENDED);
@@ -1263,7 +1261,7 @@ static int dma_chan_pause(struct dma_chan *dchan)
spin_unlock_irqrestore(&chan->vc.lock, flags);
- return timeout ? 0 : -EAGAIN;
+ return ret ? -EAGAIN : 0;
}
/* Called in chan locked context */
--
2.53.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v8 5/9] dmaengine: dw-axi-dmac: preserve pending pause requests
2026-11-19 16:00 [PATCH v8 0/9] dmaengine: dw-axi-dmac: fix termination and pause acknowledgment James Hilliard
` (3 preceding siblings ...)
2026-11-19 16:00 ` [PATCH v8 4/9] dmaengine: dw-axi-dmac: use atomic polling for suspend acknowledgment James Hilliard
@ 2026-11-19 16:01 ` James Hilliard
2026-11-19 16:27 ` Frank Li
2026-11-19 16:01 ` [PATCH v8 6/9] dmaengine: dw-axi-dmac: retain descriptors until callbacks drain James Hilliard
` (3 subsequent siblings)
8 siblings, 1 reply; 18+ messages in thread
From: James Hilliard @ 2026-11-19 16:01 UTC (permalink / raw)
To: Eugeniy Paltsev, Vinod Koul, Frank Li, Sergey Khimich,
Andy Shevchenko, Sia Jee Heng, Baruch Siach
Cc: Mitch Barnett, dmaengine, linux-kernel, James Hilliard, stable
dma_chan_pause() clears SUSPENDED even when polling times out. Hardware
can set that status after the last poll, and the unconditional clear then
discards an acknowledgment the caller never observed. Another channel's
interrupt can also consume the status because the shared IRQ handler
clears every channel's status before taking its lock.
Leave SUSPENDED to the pause and resume paths. Track the suspend request
separately from its acknowledgment so a timeout can be followed by either
a pause retry or a resume. Wait for acknowledgment before clearing the
hardware request, since hardware ignores an earlier clear. If that wait
also times out, return -EAGAIN with the request still pending.
A completed transfer is also quiescent, but its completion handler may
still need to advance a cyclic descriptor. Defer that restart while a
pause is pending, and resume it without reprogramming the first LLI.
Track the active descriptor until its completion handler runs so that
resume and issue_pending cannot restart a completed cyclic segment or
reissue a completed noncyclic descriptor while its IRQ is pending.
Start queued work after the active descriptor is retired, respecting the
pause request.
Clear stale suspension status at the start of a new descriptor rather
than at each cyclic restart. Reset the software state after successful
termination and when allocating a channel for a new client.
Store the lock-protected pause flags as bitfields in existing padding.
Fixes: 1fe20f1b8454 ("dmaengine: Introduce DW AXI DMAC driver")
Cc: stable@vger.kernel.org
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
---
drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c | 99 ++++++++++++++++++++++----
drivers/dma/dw-axi-dmac/dw-axi-dmac.h | 8 ++-
2 files changed, 91 insertions(+), 16 deletions(-)
diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
index 8c115d20728b..99b648d4ca76 100644
--- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
+++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
@@ -482,6 +482,8 @@ static void axi_chan_block_xfer_start(struct axi_dma_chan *chan,
irq_mask |= DWAXIDMAC_IRQ_SUSPENDED;
axi_chan_irq_set(chan, irq_mask);
+ axi_chan_irq_clear(chan, DWAXIDMAC_IRQ_SUSPENDED);
+ chan->desc = first;
axi_chan_enable(chan);
}
@@ -490,6 +492,9 @@ static void axi_chan_start_first_queued(struct axi_dma_chan *chan)
struct axi_dma_desc *desc;
struct virt_dma_desc *vd;
+ if (chan->desc || chan->pause_requested)
+ return;
+
vd = vchan_next_desc(&chan->vc);
if (!vd)
return;
@@ -544,6 +549,10 @@ static int dma_chan_alloc_chan_resources(struct dma_chan *dchan)
pm_runtime_put(chan->chip->dev);
return -ENOMEM;
}
+ chan->desc = NULL;
+ chan->pause_requested = false;
+ chan->is_paused = false;
+ chan->cyclic_restart = false;
dev_vdbg(dmaengine_chan_dev(dchan), "%s: allocating\n", axi_chan_name(chan));
return 0;
@@ -1088,6 +1097,8 @@ static noinline void axi_chan_handle_err(struct axi_dma_chan *chan, u32 status)
vchan_cookie_complete(vd);
/* Try to restart the controller */
+ chan->desc = NULL;
+ chan->cyclic_restart = false;
axi_chan_start_first_queued(chan);
out:
@@ -1135,11 +1146,18 @@ static void axi_chan_block_xfer_complete(struct axi_dma_chan *chan)
}
}
- axi_chan_enable(chan);
+ if (chan->pause_requested) {
+ chan->is_paused = true;
+ chan->cyclic_restart = true;
+ } else {
+ axi_chan_enable(chan);
+ }
} else {
/* Remove the completed descriptor from issued list before completing */
list_del(&vd->node);
vchan_cookie_complete(vd);
+ chan->desc = NULL;
+ axi_chan_start_first_queued(chan);
}
out:
@@ -1161,6 +1179,8 @@ static irqreturn_t dw_axi_dma_interrupt(int irq, void *dev_id)
for (i = 0; i < dw->hdata->nr_channels; i++) {
chan = &dw->chan[i];
status = axi_chan_irq_read(chan);
+ /* Leave suspension status for the pause and resume paths. */
+ status &= ~DWAXIDMAC_IRQ_SUSPENDED;
axi_chan_irq_clear(chan, status);
dev_vdbg(chip->dev, "%s %u IRQ status: 0x%08x\n",
@@ -1200,6 +1220,12 @@ static int dma_chan_terminate_all(struct dma_chan *dchan)
dw_axi_dma_set_byte_halfword(chan, false);
vchan_get_all_descriptors(&chan->vc, &head);
+
+ axi_chan_irq_clear(chan, DWAXIDMAC_IRQ_SUSPENDED);
+ chan->desc = NULL;
+ chan->pause_requested = false;
+ chan->is_paused = false;
+ chan->cyclic_restart = false;
}
if (ret) {
@@ -1215,16 +1241,49 @@ static int dma_chan_terminate_all(struct dma_chan *dchan)
return 0;
}
+static bool axi_chan_is_suspended(struct axi_dma_chan *chan)
+{
+ if (axi_chan_is_hw_enable(chan))
+ return axi_chan_irq_read(chan) & DWAXIDMAC_IRQ_SUSPENDED;
+
+ /* A completed transfer cannot acknowledge a new suspend request. */
+ axi_chan_disable(chan);
+ return true;
+}
+
+/* A timed-out request can still be acknowledged. */
+static int axi_chan_wait_for_suspend(struct axi_dma_chan *chan)
+ __must_hold(&chan->vc.lock)
+{
+ bool suspended;
+ int ret;
+
+ if (chan->is_paused)
+ return 0;
+
+ ret = read_poll_timeout_atomic(axi_chan_is_suspended, suspended, suspended,
+ 2, 40, false, chan);
+ if (ret)
+ return -EAGAIN;
+
+ axi_chan_irq_clear(chan, DWAXIDMAC_IRQ_SUSPENDED);
+ chan->is_paused = true;
+ return 0;
+}
+
static int dma_chan_pause(struct dma_chan *dchan)
{
struct axi_dma_chan *chan = dchan_to_axi_dma_chan(dchan);
unsigned long flags;
- u32 status;
int ret;
u64 val;
spin_lock_irqsave(&chan->vc.lock, flags);
+ chan->pause_requested = true;
+ if (chan->is_paused || !axi_chan_is_hw_enable(chan))
+ goto wait;
+
if (chan->chip->dw->hdata->nr_channels >= DMAC_CHAN_16) {
val = axi_dma_ioread64(chan->chip, DMAC_CHSUSPREG);
if (chan->id >= DMAC_CHAN_16) {
@@ -1251,17 +1310,11 @@ static int dma_chan_pause(struct dma_chan *dchan)
}
}
- ret = read_poll_timeout_atomic(axi_chan_irq_read, status,
- status & DWAXIDMAC_IRQ_SUSPENDED,
- 2, 40, false, chan);
-
- axi_chan_irq_clear(chan, DWAXIDMAC_IRQ_SUSPENDED);
-
- chan->is_paused = true;
-
+wait:
+ ret = axi_chan_wait_for_suspend(chan);
spin_unlock_irqrestore(&chan->vc.lock, flags);
- return ret ? -EAGAIN : 0;
+ return ret;
}
/* Called in chan locked context */
@@ -1295,6 +1348,8 @@ static inline void axi_chan_resume(struct axi_dma_chan *chan)
}
}
+ axi_chan_irq_clear(chan, DWAXIDMAC_IRQ_SUSPENDED);
+ chan->pause_requested = false;
chan->is_paused = false;
}
@@ -1302,15 +1357,31 @@ static int dma_chan_resume(struct dma_chan *dchan)
{
struct axi_dma_chan *chan = dchan_to_axi_dma_chan(dchan);
unsigned long flags;
+ int ret;
spin_lock_irqsave(&chan->vc.lock, flags);
- if (chan->is_paused)
- axi_chan_resume(chan);
+ ret = 0;
+ if (!chan->pause_requested)
+ goto out;
+
+ /* Hardware ignores resume until the suspend request is acknowledged. */
+ ret = axi_chan_wait_for_suspend(chan);
+ if (ret)
+ goto out;
+
+ axi_chan_resume(chan);
+ if (chan->cyclic_restart) {
+ chan->cyclic_restart = false;
+ axi_chan_enable(chan);
+ } else {
+ axi_chan_start_first_queued(chan);
+ }
+out:
spin_unlock_irqrestore(&chan->vc.lock, flags);
- return 0;
+ return ret;
}
static int axi_dma_suspend(struct axi_dma_chip *chip)
diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac.h b/drivers/dma/dw-axi-dmac/dw-axi-dmac.h
index 3c86e5d468ea..b18c9880711a 100644
--- a/drivers/dma/dw-axi-dmac/dw-axi-dmac.h
+++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac.h
@@ -42,6 +42,12 @@ struct axi_dma_chan {
void __iomem *chan_regs;
u8 id;
u8 hw_handshake_num;
+
+ /* Protected by vc.lock. */
+ bool pause_requested:1;
+ bool is_paused:1;
+ bool cyclic_restart:1;
+
atomic_t descs_allocated;
struct dma_pool *desc_pool;
@@ -50,8 +56,6 @@ struct axi_dma_chan {
struct axi_dma_desc *desc;
struct dma_slave_config config;
enum dma_transfer_direction direction;
- /* these other elements are all protected by vc.lock */
- bool is_paused;
};
struct dw_axi_dma {
--
2.53.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v8 6/9] dmaengine: dw-axi-dmac: retain descriptors until callbacks drain
2026-11-19 16:00 [PATCH v8 0/9] dmaengine: dw-axi-dmac: fix termination and pause acknowledgment James Hilliard
` (4 preceding siblings ...)
2026-11-19 16:01 ` [PATCH v8 5/9] dmaengine: dw-axi-dmac: preserve pending pause requests James Hilliard
@ 2026-11-19 16:01 ` James Hilliard
2026-11-19 16:41 ` Frank Li
2026-11-19 16:01 ` [PATCH v8 7/9] dmaengine: dw-axi-dmac: serialize interrupt status with channel reuse James Hilliard
` (2 subsequent siblings)
8 siblings, 1 reply; 18+ messages in thread
From: James Hilliard @ 2026-11-19 16:01 UTC (permalink / raw)
To: Eugeniy Paltsev, Vinod Koul, Frank Li, Sergey Khimich,
Andy Shevchenko, Sia Jee Heng, Baruch Siach
Cc: Mitch Barnett, dmaengine, linux-kernel, James Hilliard,
Wang YuWei, stable
Termination frees every descriptor while a pending period callback can
still refer to one through vc->cyclic. An executing callback has already
cleared that pointer, but still receives a pointer to the descriptor's
tx_result and can use it after termination frees the descriptor.
An error interrupt can also move the cyclic descriptor to desc_completed
before termination. Looking only at the head of desc_issued therefore
does not cover every pending cyclic callback.
Use vchan_terminate_all() after stopping the channel to clear the pending
alias and move the collected descriptors to desc_terminated. The existing
synchronize callback drains the tasklet before reclaiming that list.
Descriptors already detached onto the tasklet's private completed list
remain owned by the tasklet. Repeated termination preserves the retired
list, including when called from a completion callback.
Fixes: 1deb96c0fa58 ("dmaegine: dw-axi-dmac: Support device_prep_dma_cyclic()")
Reported-by: Wang YuWei <1973615295@qq.com>
Link: https://lore.kernel.org/r/tencent_C00D995A9F70206DFD1330DCB1222EEBCB07@qq.com/
Reported-by: Mitch Barnett <mitch@accuratesound.ca>
Cc: Wang YuWei <1973615295@qq.com>
Cc: stable@vger.kernel.org
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
---
drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)
diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
index 99b648d4ca76..2547f9596ae6 100644
--- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
+++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
@@ -1202,7 +1202,6 @@ static int dma_chan_terminate_all(struct dma_chan *dchan)
{
struct axi_dma_chan *chan = dchan_to_axi_dma_chan(dchan);
int ret;
- LIST_HEAD(head);
scoped_guard(spinlock_irqsave, &chan->vc.lock) {
bool enabled;
@@ -1219,7 +1218,7 @@ static int dma_chan_terminate_all(struct dma_chan *dchan)
if (chan->direction == DMA_MEM_TO_DEV)
dw_axi_dma_set_byte_halfword(chan, false);
- vchan_get_all_descriptors(&chan->vc, &head);
+ vchan_terminate_all(&chan->vc);
axi_chan_irq_clear(chan, DWAXIDMAC_IRQ_SUSPENDED);
chan->desc = NULL;
@@ -1234,8 +1233,6 @@ static int dma_chan_terminate_all(struct dma_chan *dchan)
return ret;
}
- vchan_dma_desc_free_list(&chan->vc, &head);
-
dev_vdbg(dmaengine_chan_dev(dchan), "terminated: %s\n", axi_chan_name(chan));
return 0;
--
2.53.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v8 7/9] dmaengine: dw-axi-dmac: serialize interrupt status with channel reuse
2026-11-19 16:00 [PATCH v8 0/9] dmaengine: dw-axi-dmac: fix termination and pause acknowledgment James Hilliard
` (5 preceding siblings ...)
2026-11-19 16:01 ` [PATCH v8 6/9] dmaengine: dw-axi-dmac: retain descriptors until callbacks drain James Hilliard
@ 2026-11-19 16:01 ` James Hilliard
2026-11-19 16:44 ` Frank Li
2026-11-19 16:01 ` [PATCH v8 8/9] dmaengine: dw-axi-dmac: require an IRQ before registering the controller James Hilliard
2026-11-19 16:01 ` [PATCH v8 9/9] dmaengine: dw-axi-dmac: guard shared IRQs across the controller lifetime James Hilliard
8 siblings, 1 reply; 18+ messages in thread
From: James Hilliard @ 2026-11-19 16:01 UTC (permalink / raw)
To: Eugeniy Paltsev, Vinod Koul, Frank Li, Sergey Khimich,
Andy Shevchenko, Sia Jee Heng, Baruch Siach
Cc: Mitch Barnett, dmaengine, linux-kernel, James Hilliard, stable
The interrupt handler reads and acknowledges channel status before taking
the channel lock. Termination can finish and a new transfer can start
before the handler takes that lock. The saved completion or error status
then applies to the new descriptor, stopping it and completing its cookie
before its DMA transfer has finished.
Termination also leaves completion and error status pending. A subsequent
interrupt can consume that status after the channel has been reused, even
if no handler was running when the old transfer was stopped.
Read and acknowledge status under the channel lock and keep it held while
processing the event. Move locking out of the completion and error helpers
into their sole caller, and clear all channel status after a successful
stop. Status collection, descriptor ownership and channel reuse then share
the same serialization.
Initialize the virtual channels before requesting the shared IRQs so every
lock is ready before the interrupt handler can run.
Fixes: 1fe20f1b8454 ("dmaengine: Introduce DW AXI DMAC driver")
Cc: stable@vger.kernel.org
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
---
drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c | 31 ++++++++++++--------------
1 file changed, 14 insertions(+), 17 deletions(-)
diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
index 2547f9596ae6..2eeb902eaad3 100644
--- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
+++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
@@ -21,6 +21,7 @@
#include <linux/iopoll.h>
#include <linux/io-64-nonatomic-lo-hi.h>
#include <linux/kernel.h>
+#include <linux/lockdep.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_dma.h>
@@ -1072,9 +1073,8 @@ static void axi_chan_list_dump_lli(struct axi_dma_chan *chan,
static noinline void axi_chan_handle_err(struct axi_dma_chan *chan, u32 status)
{
struct virt_dma_desc *vd;
- unsigned long flags;
- spin_lock_irqsave(&chan->vc.lock, flags);
+ lockdep_assert_held(&chan->vc.lock);
axi_chan_disable(chan);
@@ -1083,7 +1083,7 @@ static noinline void axi_chan_handle_err(struct axi_dma_chan *chan, u32 status)
if (!vd) {
dev_err(vchan_chan_dev(&chan->vc), "BUG: %s, IRQ with no descriptors\n",
axi_chan_name(chan));
- goto out;
+ return;
}
/* Remove the completed descriptor from issued list */
list_del(&vd->node);
@@ -1100,9 +1100,6 @@ static noinline void axi_chan_handle_err(struct axi_dma_chan *chan, u32 status)
chan->desc = NULL;
chan->cyclic_restart = false;
axi_chan_start_first_queued(chan);
-
-out:
- spin_unlock_irqrestore(&chan->vc.lock, flags);
}
static void axi_chan_block_xfer_complete(struct axi_dma_chan *chan)
@@ -1111,11 +1108,11 @@ static void axi_chan_block_xfer_complete(struct axi_dma_chan *chan)
struct axi_dma_hw_desc *hw_desc;
struct axi_dma_desc *desc;
struct virt_dma_desc *vd;
- unsigned long flags;
u64 llp;
int i;
- spin_lock_irqsave(&chan->vc.lock, flags);
+ lockdep_assert_held(&chan->vc.lock);
+
if (unlikely(axi_chan_is_hw_enable(chan))) {
dev_err(vchan_chan_dev(&chan->vc), "BUG: %s caught DWAXIDMAC_IRQ_DMA_TRF, but channel not idle!\n",
axi_chan_name(chan));
@@ -1127,7 +1124,7 @@ static void axi_chan_block_xfer_complete(struct axi_dma_chan *chan)
if (!vd) {
dev_err(vchan_chan_dev(&chan->vc), "BUG: %s, IRQ with no descriptors\n",
axi_chan_name(chan));
- goto out;
+ return;
}
desc = vd_to_axi_desc(vd);
@@ -1159,9 +1156,6 @@ static void axi_chan_block_xfer_complete(struct axi_dma_chan *chan)
chan->desc = NULL;
axi_chan_start_first_queued(chan);
}
-
-out:
- spin_unlock_irqrestore(&chan->vc.lock, flags);
}
static irqreturn_t dw_axi_dma_interrupt(int irq, void *dev_id)
@@ -1169,6 +1163,7 @@ static irqreturn_t dw_axi_dma_interrupt(int irq, void *dev_id)
struct axi_dma_chip *chip = dev_id;
struct dw_axi_dma *dw = chip->dw;
struct axi_dma_chan *chan;
+ unsigned long flags;
u32 status, i;
@@ -1178,6 +1173,7 @@ static irqreturn_t dw_axi_dma_interrupt(int irq, void *dev_id)
/* Poll, clear and process every channel interrupt status */
for (i = 0; i < dw->hdata->nr_channels; i++) {
chan = &dw->chan[i];
+ spin_lock_irqsave(&chan->vc.lock, flags);
status = axi_chan_irq_read(chan);
/* Leave suspension status for the pause and resume paths. */
status &= ~DWAXIDMAC_IRQ_SUSPENDED;
@@ -1190,6 +1186,7 @@ static irqreturn_t dw_axi_dma_interrupt(int irq, void *dev_id)
axi_chan_handle_err(chan, status);
else if (status & DWAXIDMAC_IRQ_DMA_TRF)
axi_chan_block_xfer_complete(chan);
+ spin_unlock_irqrestore(&chan->vc.lock, flags);
}
/* Re-enable interrupts */
@@ -1220,7 +1217,7 @@ static int dma_chan_terminate_all(struct dma_chan *dchan)
vchan_terminate_all(&chan->vc);
- axi_chan_irq_clear(chan, DWAXIDMAC_IRQ_SUSPENDED);
+ axi_chan_irq_clear(chan, DWAXIDMAC_IRQ_ALL);
chan->desc = NULL;
chan->pause_requested = false;
chan->is_paused = false;
@@ -1600,10 +1597,6 @@ static int dw_probe(struct platform_device *pdev)
if (!dw->chan)
return -ENOMEM;
- ret = axi_req_irqs(pdev, chip);
- if (ret)
- return ret;
-
INIT_LIST_HEAD(&dw->dma.channels);
for (i = 0; i < hdata->nr_channels; i++) {
struct axi_dma_chan *chan = &dw->chan[i];
@@ -1617,6 +1610,10 @@ static int dw_probe(struct platform_device *pdev)
vchan_init(&chan->vc, &dw->dma);
}
+ ret = axi_req_irqs(pdev, chip);
+ if (ret)
+ return ret;
+
/* Set capabilities */
dma_cap_set(DMA_MEMCPY, dw->dma.cap_mask);
dma_cap_set(DMA_SLAVE, dw->dma.cap_mask);
--
2.53.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v8 8/9] dmaengine: dw-axi-dmac: require an IRQ before registering the controller
2026-11-19 16:00 [PATCH v8 0/9] dmaengine: dw-axi-dmac: fix termination and pause acknowledgment James Hilliard
` (6 preceding siblings ...)
2026-11-19 16:01 ` [PATCH v8 7/9] dmaengine: dw-axi-dmac: serialize interrupt status with channel reuse James Hilliard
@ 2026-11-19 16:01 ` James Hilliard
2026-11-19 16:47 ` Frank Li
2026-11-19 16:01 ` [PATCH v8 9/9] dmaengine: dw-axi-dmac: guard shared IRQs across the controller lifetime James Hilliard
8 siblings, 1 reply; 18+ messages in thread
From: James Hilliard @ 2026-11-19 16:01 UTC (permalink / raw)
To: Eugeniy Paltsev, Vinod Koul, Frank Li, Sergey Khimich,
Andy Shevchenko, Sia Jee Heng, Baruch Siach
Cc: Mitch Barnett, dmaengine, linux-kernel, James Hilliard, stable
If the interrupt controller is not ready, the DMA driver can bind
without an IRQ handler. Clients can then submit transfers but never
receive completion callbacks. platform_irq_count() returns -EPROBE_DEFER
in this case, which skips the IRQ request loop just like a zero count.
Propagate the error so the driver core retries probe when the IRQ
provider is ready. Return -ENXIO if there are no IRQs.
Fixes: e32634f466a9 ("dma: dw-axi-dmac: support per channel interrupt")
Cc: stable@vger.kernel.org
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
---
drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
index 2eeb902eaad3..531705f8f89b 100644
--- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
+++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
@@ -1518,6 +1518,10 @@ static int axi_req_irqs(struct platform_device *pdev, struct axi_dma_chip *chip)
int irq_count = platform_irq_count(pdev);
int ret;
+ if (irq_count <= 0)
+ return dev_err_probe(chip->dev, irq_count ?: -ENXIO,
+ "failed to get controller IRQs\n");
+
for (int i = 0; i < irq_count; i++) {
chip->irq[i] = platform_get_irq(pdev, i);
if (chip->irq[i] < 0)
--
2.53.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v8 9/9] dmaengine: dw-axi-dmac: guard shared IRQs across the controller lifetime
2026-11-19 16:00 [PATCH v8 0/9] dmaengine: dw-axi-dmac: fix termination and pause acknowledgment James Hilliard
` (7 preceding siblings ...)
2026-11-19 16:01 ` [PATCH v8 8/9] dmaengine: dw-axi-dmac: require an IRQ before registering the controller James Hilliard
@ 2026-11-19 16:01 ` James Hilliard
2026-11-19 16:50 ` Frank Li
8 siblings, 1 reply; 18+ messages in thread
From: James Hilliard @ 2026-11-19 16:01 UTC (permalink / raw)
To: Eugeniy Paltsev, Vinod Koul, Frank Li, Sergey Khimich,
Andy Shevchenko, Sia Jee Heng, Baruch Siach
Cc: Mitch Barnett, dmaengine, linux-kernel, James Hilliard
An interrupt from another device on the shared line can run the handler
before the DMA channel devices exist or while the register clocks are
off. Checking the runtime PM status in the handler is not sufficient:
the controller can already signal an interrupt in the resume callback,
before the PM core marks the device active.
Mask the controller and clear stale channel status before requesting
the IRQs. Keep the global mask in place until DMA registration returns.
A transfer that completes during registration leaves its channel status
pending and is serviced when the controller interrupts are enabled.
Serialize the hardware mask and handler readiness under a chip lock.
Mask the source before withdrawing the handler and prevent a handler
already processing channels from re-enabling it on exit. Drain each IRQ
before disabling the clocks. This also covers forced system suspend and
lets removal leave an already suspended controller powered down.
Fixes: 1fe20f1b8454 ("dmaengine: Introduce DW AXI DMAC driver")
Closes: https://lore.kernel.org/r/sashiko-outbox-216269@kernel.org/
Link: https://lore.kernel.org/r/sashiko-outbox-219846@kernel.org/
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
---
drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c | 119 ++++++++++++++++---------
drivers/dma/dw-axi-dmac/dw-axi-dmac.h | 3 +
2 files changed, 81 insertions(+), 41 deletions(-)
diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
index 531705f8f89b..eab3c31f77df 100644
--- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
+++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
@@ -277,6 +277,7 @@ static void axi_dma_hw_init(struct axi_dma_chip *chip)
for (i = 0; i < chip->dw->hdata->nr_channels; i++) {
axi_chan_irq_disable(&chip->dw->chan[i], DWAXIDMAC_IRQ_ALL);
axi_chan_disable(&chip->dw->chan[i]);
+ axi_chan_irq_clear(&chip->dw->chan[i], DWAXIDMAC_IRQ_ALL);
}
ret = dma_set_mask_and_coherent(chip->dev, DMA_BIT_MASK(64));
if (ret)
@@ -1167,8 +1168,11 @@ static irqreturn_t dw_axi_dma_interrupt(int irq, void *dev_id)
u32 status, i;
- /* Disable DMAC interrupts. We'll enable them after processing channels */
- axi_dma_irq_disable(chip);
+ scoped_guard(spinlock_irqsave, &chip->irq_lock) {
+ if (!chip->irq_ready)
+ return IRQ_NONE;
+ axi_dma_irq_disable(chip);
+ }
/* Poll, clear and process every channel interrupt status */
for (i = 0; i < dw->hdata->nr_channels; i++) {
@@ -1189,8 +1193,10 @@ static irqreturn_t dw_axi_dma_interrupt(int irq, void *dev_id)
spin_unlock_irqrestore(&chan->vc.lock, flags);
}
- /* Re-enable interrupts */
- axi_dma_irq_enable(chip);
+ scoped_guard(spinlock_irqsave, &chip->irq_lock) {
+ if (chip->irq_ready)
+ axi_dma_irq_enable(chip);
+ }
return IRQ_HANDLED;
}
@@ -1378,17 +1384,39 @@ static int dma_chan_resume(struct dma_chan *dchan)
return ret;
}
-static int axi_dma_suspend(struct axi_dma_chip *chip)
+static void axi_dma_start_irqs(struct axi_dma_chip *chip)
{
- axi_dma_irq_disable(chip);
- axi_dma_disable(chip);
+ guard(spinlock_irqsave)(&chip->irq_lock);
- clk_bulk_disable_unprepare(ARRAY_SIZE(chip->clks), chip->clks);
+ chip->irq_ready = true;
+ axi_dma_irq_enable(chip);
+}
- return 0;
+static bool axi_dma_quiesce_irqs(struct axi_dma_chip *chip)
+{
+ scoped_guard(spinlock_irqsave, &chip->irq_lock) {
+ if (!chip->irq_ready)
+ return false;
+ axi_dma_irq_disable(chip);
+ /* Flush the mask before withdrawing the handler. */
+ axi_dma_ioread32(chip, DMAC_CFG);
+ chip->irq_ready = false;
+ }
+
+ for (unsigned int i = 0; i < DMAC_MAX_CHANNELS; i++)
+ if (chip->irq[i] > 0)
+ synchronize_irq(chip->irq[i]);
+
+ return true;
+}
+
+static void axi_dma_power_off(struct axi_dma_chip *chip)
+{
+ axi_dma_disable(chip);
+ clk_bulk_disable_unprepare(ARRAY_SIZE(chip->clks), chip->clks);
}
-static int axi_dma_resume(struct axi_dma_chip *chip)
+static int axi_dma_power_on(struct axi_dma_chip *chip)
{
int ret;
@@ -1396,8 +1424,8 @@ static int axi_dma_resume(struct axi_dma_chip *chip)
if (ret < 0)
return ret;
+ axi_dma_irq_disable(chip);
axi_dma_enable(chip);
- axi_dma_irq_enable(chip);
return 0;
}
@@ -1406,14 +1434,23 @@ static int __maybe_unused axi_dma_runtime_suspend(struct device *dev)
{
struct axi_dma_chip *chip = dev_get_drvdata(dev);
- return axi_dma_suspend(chip);
+ if (axi_dma_quiesce_irqs(chip))
+ axi_dma_power_off(chip);
+
+ return 0;
}
static int __maybe_unused axi_dma_runtime_resume(struct device *dev)
{
struct axi_dma_chip *chip = dev_get_drvdata(dev);
+ int ret;
- return axi_dma_resume(chip);
+ ret = axi_dma_power_on(chip);
+ if (ret)
+ return ret;
+
+ axi_dma_start_irqs(chip);
+ return 0;
}
static struct dma_chan *dw_axi_dma_of_xlate(struct of_phandle_args *dma_spec,
@@ -1560,6 +1597,7 @@ static int dw_probe(struct platform_device *pdev)
chip->dw = dw;
chip->dev = &pdev->dev;
chip->dw->hdata = hdata;
+ spin_lock_init(&chip->irq_lock);
chip->regs = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(chip->regs))
@@ -1614,10 +1652,6 @@ static int dw_probe(struct platform_device *pdev)
vchan_init(&chan->vc, &dw->dma);
}
- ret = axi_req_irqs(pdev, chip);
- if (ret)
- return ret;
-
/* Set capabilities */
dma_cap_set(DMA_MEMCPY, dw->dma.cap_mask);
dma_cap_set(DMA_SLAVE, dw->dma.cap_mask);
@@ -1656,26 +1690,30 @@ static int dw_probe(struct platform_device *pdev)
dma_set_max_seg_size(&pdev->dev, MAX_BLOCK_SIZE);
platform_set_drvdata(pdev, chip);
- pm_runtime_enable(chip->dev);
-
- /*
- * We can't just call pm_runtime_get here instead of
- * pm_runtime_get_noresume + axi_dma_resume because we need
- * driver to work also without Runtime PM.
- */
+ /* Power up explicitly so probe also works without runtime PM. */
pm_runtime_get_noresume(chip->dev);
- ret = axi_dma_resume(chip);
+ ret = axi_dma_power_on(chip);
if (ret < 0)
- goto err_pm_disable;
+ goto err_pm_put;
axi_dma_hw_init(chip);
- pm_runtime_put(chip->dev);
+ ret = axi_req_irqs(pdev, chip);
+ if (ret)
+ goto err_power_off;
+
+ ret = pm_runtime_set_active(chip->dev);
+ if (ret)
+ goto err_power_off;
+ pm_runtime_enable(chip->dev);
ret = dmaenginem_async_device_register(&dw->dma);
if (ret)
goto err_pm_disable;
+ axi_dma_start_irqs(chip);
+ pm_runtime_put(chip->dev);
+
/* Register with OF helpers for DMA lookups */
ret = of_dma_controller_register(pdev->dev.of_node,
dw_axi_dma_of_xlate, dw);
@@ -1690,6 +1728,11 @@ static int dw_probe(struct platform_device *pdev)
err_pm_disable:
pm_runtime_disable(chip->dev);
+err_power_off:
+ axi_dma_power_off(chip);
+ pm_runtime_set_suspended(chip->dev);
+err_pm_put:
+ pm_runtime_put_noidle(chip->dev);
return ret;
}
@@ -1700,22 +1743,16 @@ static void dw_remove(struct platform_device *pdev)
struct dw_axi_dma *dw = chip->dw;
struct axi_dma_chan *chan, *_chan;
u32 i;
- int ret;
-
- /* Enable clk before accessing to registers */
- ret = clk_bulk_prepare_enable(ARRAY_SIZE(chip->clks), chip->clks);
- if (ret)
- dev_warn(chip->dev, "failed to enable clocks before remove: %d\n",
- ret);
- axi_dma_irq_disable(chip);
- for (i = 0; i < dw->hdata->nr_channels; i++) {
- axi_chan_disable(&chip->dw->chan[i]);
- axi_chan_irq_disable(&chip->dw->chan[i], DWAXIDMAC_IRQ_ALL);
- }
- axi_dma_disable(chip);
pm_runtime_disable(chip->dev);
- axi_dma_suspend(chip);
+ if (axi_dma_quiesce_irqs(chip)) {
+ for (i = 0; i < dw->hdata->nr_channels; i++) {
+ axi_chan_disable(&dw->chan[i]);
+ axi_chan_irq_disable(&dw->chan[i], DWAXIDMAC_IRQ_ALL);
+ }
+ axi_dma_power_off(chip);
+ }
+ pm_runtime_set_suspended(chip->dev);
for (i = 0; i < DMAC_MAX_CHANNELS; i++)
if (chip->irq[i] > 0)
diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac.h b/drivers/dma/dw-axi-dmac/dw-axi-dmac.h
index b18c9880711a..80e4632c0d55 100644
--- a/drivers/dma/dw-axi-dmac/dw-axi-dmac.h
+++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac.h
@@ -14,6 +14,7 @@
#include <linux/clk.h>
#include <linux/device.h>
#include <linux/dmaengine.h>
+#include <linux/spinlock.h>
#include <linux/types.h>
#include "../virt-dma.h"
@@ -70,6 +71,8 @@ struct dw_axi_dma {
struct axi_dma_chip {
struct device *dev;
int irq[DMAC_MAX_CHANNELS];
+ spinlock_t irq_lock; /* Protects irq_ready and the global IRQ mask. */
+ bool irq_ready;
void __iomem *regs;
void __iomem *apb_regs;
struct clk_bulk_data clks[2];
--
2.53.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH v8 2/9] dmaengine: dw-axi-dmac: poll the correct enable bit during termination
2026-11-19 16:00 ` [PATCH v8 2/9] dmaengine: dw-axi-dmac: poll the correct enable bit during termination James Hilliard
@ 2026-11-19 16:17 ` Frank Li
0 siblings, 0 replies; 18+ messages in thread
From: Frank Li @ 2026-11-19 16:17 UTC (permalink / raw)
To: James Hilliard
Cc: Eugeniy Paltsev, Vinod Koul, Frank Li, Sergey Khimich,
Andy Shevchenko, Sia Jee Heng, Baruch Siach, Mitch Barnett,
dmaengine, linux-kernel, stable
On Thu, Nov 19, 2026 at 09:00:57AM -0700, James Hilliard wrote:
> For channels 16 through 31, the channel enable bits occupy bits 32 through
> 47 of the extended DMAC_CHEN register. dma_chan_terminate_all() only reads
> the low word and tests BIT(chan->id), so it can conclude that an upper-bank
> channel has stopped while its actual enable bit remains set. Descriptors
> can then be freed while hardware still owns them.
>
> Poll axi_chan_is_hw_enable(), which already handles both register banks
> and the smaller channel layouts, instead of constructing another mask.
>
> Fixes: 495e18b16e3d ("dmaengine: dw-axi-dmac: Add support DMAX_NUM_CHANNELS > 16")
> Cc: stable@vger.kernel.org
> Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
> ---
Reviewed-by: Frank Li <Frank.Li@nxp.com>
> drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c | 11 ++++++-----
> 1 file changed, 6 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> index 6dae63024721..f17a71183050 100644
> --- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> +++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> @@ -28,6 +28,7 @@
> #include <linux/property.h>
> #include <linux/reset.h>
> #include <linux/slab.h>
> +#include <linux/time64.h>
> #include <linux/types.h>
>
> #include "dw-axi-dmac.h"
> @@ -1179,17 +1180,17 @@ static irqreturn_t dw_axi_dma_interrupt(int irq, void *dev_id)
> static int dma_chan_terminate_all(struct dma_chan *dchan)
> {
> struct axi_dma_chan *chan = dchan_to_axi_dma_chan(dchan);
> - u32 chan_active = BIT(chan->id) << DMAC_CHAN_EN_SHIFT;
> unsigned long flags;
> - u32 val;
> + bool enabled;
> int ret;
> LIST_HEAD(head);
>
> axi_chan_disable(chan);
>
> - ret = readl_poll_timeout_atomic(chan->chip->regs + DMAC_CHEN, val,
> - !(val & chan_active), 1000, 50000);
> - if (ret == -ETIMEDOUT)
> + ret = read_poll_timeout_atomic(axi_chan_is_hw_enable, enabled, !enabled,
> + 1 * USEC_PER_MSEC, 50 * USEC_PER_MSEC,
> + false, chan);
> + if (ret)
> dev_warn(dmaengine_chan_dev(dchan),
> "%s failed to stop\n", axi_chan_name(chan));
>
>
> --
> 2.53.0
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v8 3/9] dmaengine: dw-axi-dmac: retain descriptors when termination times out
2026-11-19 16:00 ` [PATCH v8 3/9] dmaengine: dw-axi-dmac: retain descriptors when termination times out James Hilliard
@ 2026-11-19 16:18 ` Frank Li
0 siblings, 0 replies; 18+ messages in thread
From: Frank Li @ 2026-11-19 16:18 UTC (permalink / raw)
To: James Hilliard
Cc: Eugeniy Paltsev, Vinod Koul, Frank Li, Sergey Khimich,
Andy Shevchenko, Sia Jee Heng, Baruch Siach, Mitch Barnett,
dmaengine, linux-kernel, stable
On Thu, Nov 19, 2026 at 09:00:58AM -0700, James Hilliard wrote:
> dma_chan_terminate_all() warns when the channel enable bit does not clear,
> but then frees the descriptors and returns success. Its synchronize
> callback only drains virtual-channel callbacks, so terminate_sync() can
> also report success while hardware still accesses the transfer.
>
> Return the polling error before detaching descriptors or changing the
> peripheral routing. This keeps the transfer resources available while
> the channel remains enabled and lets the caller retry termination without
> treating the timeout as a successful stop.
>
> Hold the channel lock across disabling, polling and descriptor detachment
> so an interrupt handler cannot restart the channel between the stop check
> and cleanup.
>
> Fixes: 1deb96c0fa58 ("dmaegine: dw-axi-dmac: Support device_prep_dma_cyclic()")
> Cc: stable@vger.kernel.org
> Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
> ---
Reviewed-by: Frank Li <Frank.Li@nxp.com>
> drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c | 37 ++++++++++++++------------
> 1 file changed, 20 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> index f17a71183050..def591fc75ce 100644
> --- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> +++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> @@ -9,6 +9,7 @@
>
> #include <linux/bitfield.h>
> #include <linux/bitops.h>
> +#include <linux/cleanup.h>
> #include <linux/delay.h>
> #include <linux/device.h>
> #include <linux/dmaengine.h>
> @@ -1180,30 +1181,32 @@ static irqreturn_t dw_axi_dma_interrupt(int irq, void *dev_id)
> static int dma_chan_terminate_all(struct dma_chan *dchan)
> {
> struct axi_dma_chan *chan = dchan_to_axi_dma_chan(dchan);
> - unsigned long flags;
> - bool enabled;
> int ret;
> LIST_HEAD(head);
>
> - axi_chan_disable(chan);
> -
> - ret = read_poll_timeout_atomic(axi_chan_is_hw_enable, enabled, !enabled,
> - 1 * USEC_PER_MSEC, 50 * USEC_PER_MSEC,
> - false, chan);
> - if (ret)
> - dev_warn(dmaengine_chan_dev(dchan),
> - "%s failed to stop\n", axi_chan_name(chan));
> + scoped_guard(spinlock_irqsave, &chan->vc.lock) {
> + bool enabled;
>
> - if (chan->direction != DMA_MEM_TO_MEM)
> - dw_axi_dma_set_hw_channel(chan, false);
> - if (chan->direction == DMA_MEM_TO_DEV)
> - dw_axi_dma_set_byte_halfword(chan, false);
> + axi_chan_disable(chan);
> + ret = read_poll_timeout_atomic(axi_chan_is_hw_enable, enabled, !enabled,
> + 1 * USEC_PER_MSEC, 50 * USEC_PER_MSEC,
> + false, chan);
> + if (ret)
> + break;
>
> - spin_lock_irqsave(&chan->vc.lock, flags);
> + if (chan->direction != DMA_MEM_TO_MEM)
> + dw_axi_dma_set_hw_channel(chan, false);
> + if (chan->direction == DMA_MEM_TO_DEV)
> + dw_axi_dma_set_byte_halfword(chan, false);
>
> - vchan_get_all_descriptors(&chan->vc, &head);
> + vchan_get_all_descriptors(&chan->vc, &head);
> + }
>
> - spin_unlock_irqrestore(&chan->vc.lock, flags);
> + if (ret) {
> + dev_warn(dmaengine_chan_dev(dchan),
> + "%s failed to stop\n", axi_chan_name(chan));
> + return ret;
> + }
>
> vchan_dma_desc_free_list(&chan->vc, &head);
>
>
> --
> 2.53.0
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v8 4/9] dmaengine: dw-axi-dmac: use atomic polling for suspend acknowledgment
2026-11-19 16:00 ` [PATCH v8 4/9] dmaengine: dw-axi-dmac: use atomic polling for suspend acknowledgment James Hilliard
@ 2026-11-19 16:19 ` Frank Li
0 siblings, 0 replies; 18+ messages in thread
From: Frank Li @ 2026-11-19 16:19 UTC (permalink / raw)
To: James Hilliard
Cc: Eugeniy Paltsev, Vinod Koul, Frank Li, Sergey Khimich,
Andy Shevchenko, Sia Jee Heng, Baruch Siach, Mitch Barnett,
dmaengine, linux-kernel, stable
On Thu, Nov 19, 2026 at 09:00:59AM -0700, James Hilliard wrote:
> The pause loop delays after its final status read, so an acknowledgment
> that arrives during that delay is cleared without being observed and
> the caller receives -EAGAIN.
>
> Use read_poll_timeout_atomic() to check the status after the final delay.
> Keep the two-microsecond polling interval, 40-microsecond timeout and
> -EAGAIN return value when suspension is not acknowledged.
>
> Fixes: 1fe20f1b8454 ("dmaengine: Introduce DW AXI DMAC driver")
> Cc: stable@vger.kernel.org
> Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
> ---
Reviewed-by: Frank Li <Frank.Li@nxp.com>
> drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c | 14 ++++++--------
> 1 file changed, 6 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> index def591fc75ce..8c115d20728b 100644
> --- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> +++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> @@ -1219,7 +1219,8 @@ static int dma_chan_pause(struct dma_chan *dchan)
> {
> struct axi_dma_chan *chan = dchan_to_axi_dma_chan(dchan);
> unsigned long flags;
> - unsigned int timeout = 20; /* timeout iterations */
> + u32 status;
> + int ret;
> u64 val;
>
> spin_lock_irqsave(&chan->vc.lock, flags);
> @@ -1250,12 +1251,9 @@ static int dma_chan_pause(struct dma_chan *dchan)
> }
> }
>
> - do {
> - if (axi_chan_irq_read(chan) & DWAXIDMAC_IRQ_SUSPENDED)
> - break;
> -
> - udelay(2);
> - } while (--timeout);
> + ret = read_poll_timeout_atomic(axi_chan_irq_read, status,
> + status & DWAXIDMAC_IRQ_SUSPENDED,
> + 2, 40, false, chan);
>
> axi_chan_irq_clear(chan, DWAXIDMAC_IRQ_SUSPENDED);
>
> @@ -1263,7 +1261,7 @@ static int dma_chan_pause(struct dma_chan *dchan)
>
> spin_unlock_irqrestore(&chan->vc.lock, flags);
>
> - return timeout ? 0 : -EAGAIN;
> + return ret ? -EAGAIN : 0;
> }
>
> /* Called in chan locked context */
>
> --
> 2.53.0
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v8 5/9] dmaengine: dw-axi-dmac: preserve pending pause requests
2026-11-19 16:01 ` [PATCH v8 5/9] dmaengine: dw-axi-dmac: preserve pending pause requests James Hilliard
@ 2026-11-19 16:27 ` Frank Li
0 siblings, 0 replies; 18+ messages in thread
From: Frank Li @ 2026-11-19 16:27 UTC (permalink / raw)
To: James Hilliard
Cc: Eugeniy Paltsev, Vinod Koul, Frank Li, Sergey Khimich,
Andy Shevchenko, Sia Jee Heng, Baruch Siach, Mitch Barnett,
dmaengine, linux-kernel, stable
On Thu, Nov 19, 2026 at 09:01:00AM -0700, James Hilliard wrote:
> dma_chan_pause() clears SUSPENDED even when polling times out. Hardware
> can set that status after the last poll, and the unconditional clear then
> discards an acknowledgment the caller never observed. Another channel's
> interrupt can also consume the status because the shared IRQ handler
> clears every channel's status before taking its lock.
>
> Leave SUSPENDED to the pause and resume paths. Track the suspend request
> separately from its acknowledgment so a timeout can be followed by either
> a pause retry or a resume. Wait for acknowledgment before clearing the
> hardware request, since hardware ignores an earlier clear. If that wait
> also times out, return -EAGAIN with the request still pending.
>
> A completed transfer is also quiescent, but its completion handler may
> still need to advance a cyclic descriptor. Defer that restart while a
> pause is pending, and resume it without reprogramming the first LLI.
> Track the active descriptor until its completion handler runs so that
> resume and issue_pending cannot restart a completed cyclic segment or
> reissue a completed noncyclic descriptor while its IRQ is pending.
> Start queued work after the active descriptor is retired, respecting the
> pause request.
>
> Clear stale suspension status at the start of a new descriptor rather
> than at each cyclic restart. Reset the software state after successful
> termination and when allocating a channel for a new client.
>
> Store the lock-protected pause flags as bitfields in existing padding.
>
> Fixes: 1fe20f1b8454 ("dmaengine: Introduce DW AXI DMAC driver")
> Cc: stable@vger.kernel.org
> Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
> ---
> drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c | 99 ++++++++++++++++++++++----
> drivers/dma/dw-axi-dmac/dw-axi-dmac.h | 8 ++-
> 2 files changed, 91 insertions(+), 16 deletions(-)
>
> diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> index 8c115d20728b..99b648d4ca76 100644
> --- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> +++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> @@ -482,6 +482,8 @@ static void axi_chan_block_xfer_start(struct axi_dma_chan *chan,
> irq_mask |= DWAXIDMAC_IRQ_SUSPENDED;
> axi_chan_irq_set(chan, irq_mask);
>
> + axi_chan_irq_clear(chan, DWAXIDMAC_IRQ_SUSPENDED);
> + chan->desc = first;
> axi_chan_enable(chan);
> }
>
> @@ -490,6 +492,9 @@ static void axi_chan_start_first_queued(struct axi_dma_chan *chan)
> struct axi_dma_desc *desc;
> struct virt_dma_desc *vd;
>
> + if (chan->desc || chan->pause_requested)
> + return;
> +
> vd = vchan_next_desc(&chan->vc);
> if (!vd)
> return;
> @@ -544,6 +549,10 @@ static int dma_chan_alloc_chan_resources(struct dma_chan *dchan)
> pm_runtime_put(chan->chip->dev);
> return -ENOMEM;
> }
> + chan->desc = NULL;
> + chan->pause_requested = false;
> + chan->is_paused = false;
> + chan->cyclic_restart = false;
> dev_vdbg(dmaengine_chan_dev(dchan), "%s: allocating\n", axi_chan_name(chan));
>
> return 0;
> @@ -1088,6 +1097,8 @@ static noinline void axi_chan_handle_err(struct axi_dma_chan *chan, u32 status)
> vchan_cookie_complete(vd);
>
> /* Try to restart the controller */
> + chan->desc = NULL;
> + chan->cyclic_restart = false;
> axi_chan_start_first_queued(chan);
>
> out:
> @@ -1135,11 +1146,18 @@ static void axi_chan_block_xfer_complete(struct axi_dma_chan *chan)
> }
> }
>
> - axi_chan_enable(chan);
> + if (chan->pause_requested) {
> + chan->is_paused = true;
> + chan->cyclic_restart = true;
> + } else {
> + axi_chan_enable(chan);
> + }
> } else {
> /* Remove the completed descriptor from issued list before completing */
> list_del(&vd->node);
> vchan_cookie_complete(vd);
> + chan->desc = NULL;
> + axi_chan_start_first_queued(chan);
> }
>
> out:
> @@ -1161,6 +1179,8 @@ static irqreturn_t dw_axi_dma_interrupt(int irq, void *dev_id)
> for (i = 0; i < dw->hdata->nr_channels; i++) {
> chan = &dw->chan[i];
> status = axi_chan_irq_read(chan);
> + /* Leave suspension status for the pause and resume paths. */
> + status &= ~DWAXIDMAC_IRQ_SUSPENDED;
If you don't clean DWAXIDMAC_IRQ_SUSPENDED, does it cause irq flood?
Frank
> axi_chan_irq_clear(chan, status);
>
> dev_vdbg(chip->dev, "%s %u IRQ status: 0x%08x\n",
> @@ -1200,6 +1220,12 @@ static int dma_chan_terminate_all(struct dma_chan *dchan)
> dw_axi_dma_set_byte_halfword(chan, false);
>
> vchan_get_all_descriptors(&chan->vc, &head);
> +
> + axi_chan_irq_clear(chan, DWAXIDMAC_IRQ_SUSPENDED);
> + chan->desc = NULL;
> + chan->pause_requested = false;
> + chan->is_paused = false;
> + chan->cyclic_restart = false;
> }
>
> if (ret) {
> @@ -1215,16 +1241,49 @@ static int dma_chan_terminate_all(struct dma_chan *dchan)
> return 0;
> }
>
> +static bool axi_chan_is_suspended(struct axi_dma_chan *chan)
> +{
> + if (axi_chan_is_hw_enable(chan))
> + return axi_chan_irq_read(chan) & DWAXIDMAC_IRQ_SUSPENDED;
> +
> + /* A completed transfer cannot acknowledge a new suspend request. */
> + axi_chan_disable(chan);
> + return true;
> +}
> +
> +/* A timed-out request can still be acknowledged. */
> +static int axi_chan_wait_for_suspend(struct axi_dma_chan *chan)
> + __must_hold(&chan->vc.lock)
> +{
> + bool suspended;
> + int ret;
> +
> + if (chan->is_paused)
> + return 0;
> +
> + ret = read_poll_timeout_atomic(axi_chan_is_suspended, suspended, suspended,
> + 2, 40, false, chan);
> + if (ret)
> + return -EAGAIN;
> +
> + axi_chan_irq_clear(chan, DWAXIDMAC_IRQ_SUSPENDED);
> + chan->is_paused = true;
> + return 0;
> +}
> +
> static int dma_chan_pause(struct dma_chan *dchan)
> {
> struct axi_dma_chan *chan = dchan_to_axi_dma_chan(dchan);
> unsigned long flags;
> - u32 status;
> int ret;
> u64 val;
>
> spin_lock_irqsave(&chan->vc.lock, flags);
>
> + chan->pause_requested = true;
> + if (chan->is_paused || !axi_chan_is_hw_enable(chan))
> + goto wait;
> +
> if (chan->chip->dw->hdata->nr_channels >= DMAC_CHAN_16) {
> val = axi_dma_ioread64(chan->chip, DMAC_CHSUSPREG);
> if (chan->id >= DMAC_CHAN_16) {
> @@ -1251,17 +1310,11 @@ static int dma_chan_pause(struct dma_chan *dchan)
> }
> }
>
> - ret = read_poll_timeout_atomic(axi_chan_irq_read, status,
> - status & DWAXIDMAC_IRQ_SUSPENDED,
> - 2, 40, false, chan);
> -
> - axi_chan_irq_clear(chan, DWAXIDMAC_IRQ_SUSPENDED);
> -
> - chan->is_paused = true;
> -
> +wait:
> + ret = axi_chan_wait_for_suspend(chan);
> spin_unlock_irqrestore(&chan->vc.lock, flags);
>
> - return ret ? -EAGAIN : 0;
> + return ret;
> }
>
> /* Called in chan locked context */
> @@ -1295,6 +1348,8 @@ static inline void axi_chan_resume(struct axi_dma_chan *chan)
> }
> }
>
> + axi_chan_irq_clear(chan, DWAXIDMAC_IRQ_SUSPENDED);
> + chan->pause_requested = false;
> chan->is_paused = false;
> }
>
> @@ -1302,15 +1357,31 @@ static int dma_chan_resume(struct dma_chan *dchan)
> {
> struct axi_dma_chan *chan = dchan_to_axi_dma_chan(dchan);
> unsigned long flags;
> + int ret;
>
> spin_lock_irqsave(&chan->vc.lock, flags);
>
> - if (chan->is_paused)
> - axi_chan_resume(chan);
> + ret = 0;
> + if (!chan->pause_requested)
> + goto out;
> +
> + /* Hardware ignores resume until the suspend request is acknowledged. */
> + ret = axi_chan_wait_for_suspend(chan);
> + if (ret)
> + goto out;
> +
> + axi_chan_resume(chan);
> + if (chan->cyclic_restart) {
> + chan->cyclic_restart = false;
> + axi_chan_enable(chan);
> + } else {
> + axi_chan_start_first_queued(chan);
> + }
>
> +out:
> spin_unlock_irqrestore(&chan->vc.lock, flags);
>
> - return 0;
> + return ret;
> }
>
> static int axi_dma_suspend(struct axi_dma_chip *chip)
> diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac.h b/drivers/dma/dw-axi-dmac/dw-axi-dmac.h
> index 3c86e5d468ea..b18c9880711a 100644
> --- a/drivers/dma/dw-axi-dmac/dw-axi-dmac.h
> +++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac.h
> @@ -42,6 +42,12 @@ struct axi_dma_chan {
> void __iomem *chan_regs;
> u8 id;
> u8 hw_handshake_num;
> +
> + /* Protected by vc.lock. */
> + bool pause_requested:1;
> + bool is_paused:1;
> + bool cyclic_restart:1;
> +
> atomic_t descs_allocated;
>
> struct dma_pool *desc_pool;
> @@ -50,8 +56,6 @@ struct axi_dma_chan {
> struct axi_dma_desc *desc;
> struct dma_slave_config config;
> enum dma_transfer_direction direction;
> - /* these other elements are all protected by vc.lock */
> - bool is_paused;
> };
>
> struct dw_axi_dma {
>
> --
> 2.53.0
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v8 6/9] dmaengine: dw-axi-dmac: retain descriptors until callbacks drain
2026-11-19 16:01 ` [PATCH v8 6/9] dmaengine: dw-axi-dmac: retain descriptors until callbacks drain James Hilliard
@ 2026-11-19 16:41 ` Frank Li
0 siblings, 0 replies; 18+ messages in thread
From: Frank Li @ 2026-11-19 16:41 UTC (permalink / raw)
To: James Hilliard
Cc: Eugeniy Paltsev, Vinod Koul, Frank Li, Sergey Khimich,
Andy Shevchenko, Sia Jee Heng, Baruch Siach, Mitch Barnett,
dmaengine, linux-kernel, Wang YuWei, stable
On Thu, Nov 19, 2026 at 09:01:01AM -0700, James Hilliard wrote:
> Termination frees every descriptor while a pending period callback can
> still refer to one through vc->cyclic. An executing callback has already
> cleared that pointer, but still receives a pointer to the descriptor's
> tx_result and can use it after termination frees the descriptor.
>
> An error interrupt can also move the cyclic descriptor to desc_completed
> before termination. Looking only at the head of desc_issued therefore
> does not cover every pending cyclic callback.
>
> Use vchan_terminate_all() after stopping the channel to clear the pending
> alias and move the collected descriptors to desc_terminated. The existing
> synchronize callback drains the tasklet before reclaiming that list.
> Descriptors already detached onto the tasklet's private completed list
> remain owned by the tasklet. Repeated termination preserves the retired
> list, including when called from a completion callback.
>
> Fixes: 1deb96c0fa58 ("dmaegine: dw-axi-dmac: Support device_prep_dma_cyclic()")
> Reported-by: Wang YuWei <1973615295@qq.com>
> Link: https://lore.kernel.org/r/tencent_C00D995A9F70206DFD1330DCB1222EEBCB07@qq.com/
> Reported-by: Mitch Barnett <mitch@accuratesound.ca>
> Cc: Wang YuWei <1973615295@qq.com>
> Cc: stable@vger.kernel.org
> Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
> ---
> drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c | 5 +----
> 1 file changed, 1 insertion(+), 4 deletions(-)
>
> diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> index 99b648d4ca76..2547f9596ae6 100644
> --- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> +++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> @@ -1202,7 +1202,6 @@ static int dma_chan_terminate_all(struct dma_chan *dchan)
> {
> struct axi_dma_chan *chan = dchan_to_axi_dma_chan(dchan);
> int ret;
> - LIST_HEAD(head);
>
> scoped_guard(spinlock_irqsave, &chan->vc.lock) {
> bool enabled;
> @@ -1219,7 +1218,7 @@ static int dma_chan_terminate_all(struct dma_chan *dchan)
> if (chan->direction == DMA_MEM_TO_DEV)
> dw_axi_dma_set_byte_halfword(chan, false);
>
> - vchan_get_all_descriptors(&chan->vc, &head);
> + vchan_terminate_all(&chan->vc);
I have not found this function, Any dependence I missed?
And keep allocate_queue when terminate,
https://lore.kernel.org/dmaengine/20261104-dma40-fixes-v1-1-48c5180f5210@kernel.org/
Look like we new APIs to distingiush terminate_all() and free channel.
when free channel, need clean alloc queue, but termiate_all() should be
keep alloc queue.
terminate should means terminate all DMA engine own's queue, before
submit, desc should be own by consumer.
Frank
>
> axi_chan_irq_clear(chan, DWAXIDMAC_IRQ_SUSPENDED);
> chan->desc = NULL;
> @@ -1234,8 +1233,6 @@ static int dma_chan_terminate_all(struct dma_chan *dchan)
> return ret;
> }
>
> - vchan_dma_desc_free_list(&chan->vc, &head);
> -
> dev_vdbg(dmaengine_chan_dev(dchan), "terminated: %s\n", axi_chan_name(chan));
>
> return 0;
>
> --
> 2.53.0
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v8 7/9] dmaengine: dw-axi-dmac: serialize interrupt status with channel reuse
2026-11-19 16:01 ` [PATCH v8 7/9] dmaengine: dw-axi-dmac: serialize interrupt status with channel reuse James Hilliard
@ 2026-11-19 16:44 ` Frank Li
0 siblings, 0 replies; 18+ messages in thread
From: Frank Li @ 2026-11-19 16:44 UTC (permalink / raw)
To: James Hilliard
Cc: Eugeniy Paltsev, Vinod Koul, Frank Li, Sergey Khimich,
Andy Shevchenko, Sia Jee Heng, Baruch Siach, Mitch Barnett,
dmaengine, linux-kernel, stable
On Thu, Nov 19, 2026 at 09:01:02AM -0700, James Hilliard wrote:
> The interrupt handler reads and acknowledges channel status before taking
> the channel lock. Termination can finish and a new transfer can start
> before the handler takes that lock. The saved completion or error status
> then applies to the new descriptor, stopping it and completing its cookie
> before its DMA transfer has finished.
>
> Termination also leaves completion and error status pending. A subsequent
> interrupt can consume that status after the channel has been reused, even
> if no handler was running when the old transfer was stopped.
>
> Read and acknowledge status under the channel lock and keep it held while
> processing the event. Move locking out of the completion and error helpers
> into their sole caller, and clear all channel status after a successful
> stop. Status collection, descriptor ownership and channel reuse then share
> the same serialization.
>
> Initialize the virtual channels before requesting the shared IRQs so every
> lock is ready before the interrupt handler can run.
>
> Fixes: 1fe20f1b8454 ("dmaengine: Introduce DW AXI DMAC driver")
> Cc: stable@vger.kernel.org
> Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
> ---
> drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c | 31 ++++++++++++--------------
> 1 file changed, 14 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> index 2547f9596ae6..2eeb902eaad3 100644
> --- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> +++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> @@ -21,6 +21,7 @@
> #include <linux/iopoll.h>
> #include <linux/io-64-nonatomic-lo-hi.h>
> #include <linux/kernel.h>
> +#include <linux/lockdep.h>
> #include <linux/module.h>
> #include <linux/of.h>
> #include <linux/of_dma.h>
> @@ -1072,9 +1073,8 @@ static void axi_chan_list_dump_lli(struct axi_dma_chan *chan,
> static noinline void axi_chan_handle_err(struct axi_dma_chan *chan, u32 status)
> {
> struct virt_dma_desc *vd;
> - unsigned long flags;
>
> - spin_lock_irqsave(&chan->vc.lock, flags);
> + lockdep_assert_held(&chan->vc.lock);
>
> axi_chan_disable(chan);
>
> @@ -1083,7 +1083,7 @@ static noinline void axi_chan_handle_err(struct axi_dma_chan *chan, u32 status)
> if (!vd) {
> dev_err(vchan_chan_dev(&chan->vc), "BUG: %s, IRQ with no descriptors\n",
> axi_chan_name(chan));
> - goto out;
> + return;
> }
> /* Remove the completed descriptor from issued list */
> list_del(&vd->node);
> @@ -1100,9 +1100,6 @@ static noinline void axi_chan_handle_err(struct axi_dma_chan *chan, u32 status)
> chan->desc = NULL;
> chan->cyclic_restart = false;
> axi_chan_start_first_queued(chan);
> -
> -out:
> - spin_unlock_irqrestore(&chan->vc.lock, flags);
> }
>
> static void axi_chan_block_xfer_complete(struct axi_dma_chan *chan)
> @@ -1111,11 +1108,11 @@ static void axi_chan_block_xfer_complete(struct axi_dma_chan *chan)
> struct axi_dma_hw_desc *hw_desc;
> struct axi_dma_desc *desc;
> struct virt_dma_desc *vd;
> - unsigned long flags;
> u64 llp;
> int i;
>
> - spin_lock_irqsave(&chan->vc.lock, flags);
> + lockdep_assert_held(&chan->vc.lock);
> +
> if (unlikely(axi_chan_is_hw_enable(chan))) {
> dev_err(vchan_chan_dev(&chan->vc), "BUG: %s caught DWAXIDMAC_IRQ_DMA_TRF, but channel not idle!\n",
> axi_chan_name(chan));
> @@ -1127,7 +1124,7 @@ static void axi_chan_block_xfer_complete(struct axi_dma_chan *chan)
> if (!vd) {
> dev_err(vchan_chan_dev(&chan->vc), "BUG: %s, IRQ with no descriptors\n",
> axi_chan_name(chan));
> - goto out;
> + return;
> }
>
> desc = vd_to_axi_desc(vd);
> @@ -1159,9 +1156,6 @@ static void axi_chan_block_xfer_complete(struct axi_dma_chan *chan)
> chan->desc = NULL;
> axi_chan_start_first_queued(chan);
> }
> -
> -out:
> - spin_unlock_irqrestore(&chan->vc.lock, flags);
> }
>
> static irqreturn_t dw_axi_dma_interrupt(int irq, void *dev_id)
> @@ -1169,6 +1163,7 @@ static irqreturn_t dw_axi_dma_interrupt(int irq, void *dev_id)
> struct axi_dma_chip *chip = dev_id;
> struct dw_axi_dma *dw = chip->dw;
> struct axi_dma_chan *chan;
> + unsigned long flags;
>
> u32 status, i;
>
> @@ -1178,6 +1173,7 @@ static irqreturn_t dw_axi_dma_interrupt(int irq, void *dev_id)
> /* Poll, clear and process every channel interrupt status */
> for (i = 0; i < dw->hdata->nr_channels; i++) {
> chan = &dw->chan[i];
> + spin_lock_irqsave(&chan->vc.lock, flags);
> status = axi_chan_irq_read(chan);
> /* Leave suspension status for the pause and resume paths. */
> status &= ~DWAXIDMAC_IRQ_SUSPENDED;
> @@ -1190,6 +1186,7 @@ static irqreturn_t dw_axi_dma_interrupt(int irq, void *dev_id)
> axi_chan_handle_err(chan, status);
> else if (status & DWAXIDMAC_IRQ_DMA_TRF)
> axi_chan_block_xfer_complete(chan);
> + spin_unlock_irqrestore(&chan->vc.lock, flags);
> }
>
> /* Re-enable interrupts */
> @@ -1220,7 +1217,7 @@ static int dma_chan_terminate_all(struct dma_chan *dchan)
>
> vchan_terminate_all(&chan->vc);
>
> - axi_chan_irq_clear(chan, DWAXIDMAC_IRQ_SUSPENDED);
> + axi_chan_irq_clear(chan, DWAXIDMAC_IRQ_ALL);
> chan->desc = NULL;
> chan->pause_requested = false;
> chan->is_paused = false;
> @@ -1600,10 +1597,6 @@ static int dw_probe(struct platform_device *pdev)
> if (!dw->chan)
> return -ENOMEM;
>
> - ret = axi_req_irqs(pdev, chip);
> - if (ret)
> - return ret;
> -
> INIT_LIST_HEAD(&dw->dma.channels);
> for (i = 0; i < hdata->nr_channels; i++) {
> struct axi_dma_chan *chan = &dw->chan[i];
> @@ -1617,6 +1610,10 @@ static int dw_probe(struct platform_device *pdev)
> vchan_init(&chan->vc, &dw->dma);
> }
>
> + ret = axi_req_irqs(pdev, chip);
> + if (ret)
> + return ret;
> +
Most likely, irq request should after dma engine register. irq handle may
use some DMAengine APIs.
Frank
> /* Set capabilities */
> dma_cap_set(DMA_MEMCPY, dw->dma.cap_mask);
> dma_cap_set(DMA_SLAVE, dw->dma.cap_mask);
>
> --
> 2.53.0
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v8 8/9] dmaengine: dw-axi-dmac: require an IRQ before registering the controller
2026-11-19 16:01 ` [PATCH v8 8/9] dmaengine: dw-axi-dmac: require an IRQ before registering the controller James Hilliard
@ 2026-11-19 16:47 ` Frank Li
0 siblings, 0 replies; 18+ messages in thread
From: Frank Li @ 2026-11-19 16:47 UTC (permalink / raw)
To: James Hilliard
Cc: Eugeniy Paltsev, Vinod Koul, Frank Li, Sergey Khimich,
Andy Shevchenko, Sia Jee Heng, Baruch Siach, Mitch Barnett,
dmaengine, linux-kernel, stable
On Thu, Nov 19, 2026 at 09:01:03AM -0700, James Hilliard wrote:
> If the interrupt controller is not ready, the DMA driver can bind
> without an IRQ handler. Clients can then submit transfers but never
> receive completion callbacks. platform_irq_count() returns -EPROBE_DEFER
> in this case, which skips the IRQ request loop just like a zero count.
>
> Propagate the error so the driver core retries probe when the IRQ
> provider is ready. Return -ENXIO if there are no IRQs.
>
> Fixes: e32634f466a9 ("dma: dw-axi-dmac: support per channel interrupt")
> Cc: stable@vger.kernel.org
> Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
> ---
Reviewed-by: Frank Li <Frank.Li@nxp.com>
> drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> index 2eeb902eaad3..531705f8f89b 100644
> --- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> +++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> @@ -1518,6 +1518,10 @@ static int axi_req_irqs(struct platform_device *pdev, struct axi_dma_chip *chip)
> int irq_count = platform_irq_count(pdev);
> int ret;
>
> + if (irq_count <= 0)
> + return dev_err_probe(chip->dev, irq_count ?: -ENXIO,
> + "failed to get controller IRQs\n");
> +
> for (int i = 0; i < irq_count; i++) {
> chip->irq[i] = platform_get_irq(pdev, i);
> if (chip->irq[i] < 0)
>
> --
> 2.53.0
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v8 9/9] dmaengine: dw-axi-dmac: guard shared IRQs across the controller lifetime
2026-11-19 16:01 ` [PATCH v8 9/9] dmaengine: dw-axi-dmac: guard shared IRQs across the controller lifetime James Hilliard
@ 2026-11-19 16:50 ` Frank Li
0 siblings, 0 replies; 18+ messages in thread
From: Frank Li @ 2026-11-19 16:50 UTC (permalink / raw)
To: James Hilliard
Cc: Eugeniy Paltsev, Vinod Koul, Frank Li, Sergey Khimich,
Andy Shevchenko, Sia Jee Heng, Baruch Siach, Mitch Barnett,
dmaengine, linux-kernel
On Thu, Nov 19, 2026 at 09:01:04AM -0700, James Hilliard wrote:
> An interrupt from another device on the shared line can run the handler
> before the DMA channel devices exist or while the register clocks are
> off. Checking the runtime PM status in the handler is not sufficient:
> the controller can already signal an interrupt in the resume callback,
> before the PM core marks the device active.
>
> Mask the controller and clear stale channel status before requesting
> the IRQs. Keep the global mask in place until DMA registration returns.
> A transfer that completes during registration leaves its channel status
> pending and is serviced when the controller interrupts are enabled.
>
> Serialize the hardware mask and handler readiness under a chip lock.
> Mask the source before withdrawing the handler and prevent a handler
> already processing channels from re-enabling it on exit. Drain each IRQ
> before disabling the clocks. This also covers forced system suspend and
> lets removal leave an already suspended controller powered down.
>
> Fixes: 1fe20f1b8454 ("dmaengine: Introduce DW AXI DMAC driver")
> Closes: https://lore.kernel.org/r/sashiko-outbox-216269@kernel.org/
> Link: https://lore.kernel.org/r/sashiko-outbox-219846@kernel.org/
> Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
> ---
> drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c | 119 ++++++++++++++++---------
> drivers/dma/dw-axi-dmac/dw-axi-dmac.h | 3 +
> 2 files changed, 81 insertions(+), 41 deletions(-)
>
> diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> index 531705f8f89b..eab3c31f77df 100644
> --- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> +++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> @@ -277,6 +277,7 @@ static void axi_dma_hw_init(struct axi_dma_chip *chip)
> for (i = 0; i < chip->dw->hdata->nr_channels; i++) {
> axi_chan_irq_disable(&chip->dw->chan[i], DWAXIDMAC_IRQ_ALL);
> axi_chan_disable(&chip->dw->chan[i]);
> + axi_chan_irq_clear(&chip->dw->chan[i], DWAXIDMAC_IRQ_ALL);
> }
> ret = dma_set_mask_and_coherent(chip->dev, DMA_BIT_MASK(64));
> if (ret)
> @@ -1167,8 +1168,11 @@ static irqreturn_t dw_axi_dma_interrupt(int irq, void *dev_id)
>
> u32 status, i;
>
> - /* Disable DMAC interrupts. We'll enable them after processing channels */
> - axi_dma_irq_disable(chip);
> + scoped_guard(spinlock_irqsave, &chip->irq_lock) {
> + if (!chip->irq_ready)
Does pm_runtime_get_if_active() can resolve your problem?
Frank
> + return IRQ_NONE;
> + axi_dma_irq_disable(chip);
> + }
>
> /* Poll, clear and process every channel interrupt status */
> for (i = 0; i < dw->hdata->nr_channels; i++) {
> @@ -1189,8 +1193,10 @@ static irqreturn_t dw_axi_dma_interrupt(int irq, void *dev_id)
> spin_unlock_irqrestore(&chan->vc.lock, flags);
> }
>
> - /* Re-enable interrupts */
> - axi_dma_irq_enable(chip);
> + scoped_guard(spinlock_irqsave, &chip->irq_lock) {
> + if (chip->irq_ready)
> + axi_dma_irq_enable(chip);
> + }
>
> return IRQ_HANDLED;
> }
> @@ -1378,17 +1384,39 @@ static int dma_chan_resume(struct dma_chan *dchan)
> return ret;
> }
>
> -static int axi_dma_suspend(struct axi_dma_chip *chip)
> +static void axi_dma_start_irqs(struct axi_dma_chip *chip)
> {
> - axi_dma_irq_disable(chip);
> - axi_dma_disable(chip);
> + guard(spinlock_irqsave)(&chip->irq_lock);
>
> - clk_bulk_disable_unprepare(ARRAY_SIZE(chip->clks), chip->clks);
> + chip->irq_ready = true;
> + axi_dma_irq_enable(chip);
> +}
>
> - return 0;
> +static bool axi_dma_quiesce_irqs(struct axi_dma_chip *chip)
> +{
> + scoped_guard(spinlock_irqsave, &chip->irq_lock) {
> + if (!chip->irq_ready)
> + return false;
> + axi_dma_irq_disable(chip);
> + /* Flush the mask before withdrawing the handler. */
> + axi_dma_ioread32(chip, DMAC_CFG);
> + chip->irq_ready = false;
> + }
> +
> + for (unsigned int i = 0; i < DMAC_MAX_CHANNELS; i++)
> + if (chip->irq[i] > 0)
> + synchronize_irq(chip->irq[i]);
> +
> + return true;
> +}
> +
> +static void axi_dma_power_off(struct axi_dma_chip *chip)
> +{
> + axi_dma_disable(chip);
> + clk_bulk_disable_unprepare(ARRAY_SIZE(chip->clks), chip->clks);
> }
>
> -static int axi_dma_resume(struct axi_dma_chip *chip)
> +static int axi_dma_power_on(struct axi_dma_chip *chip)
> {
> int ret;
>
> @@ -1396,8 +1424,8 @@ static int axi_dma_resume(struct axi_dma_chip *chip)
> if (ret < 0)
> return ret;
>
> + axi_dma_irq_disable(chip);
> axi_dma_enable(chip);
> - axi_dma_irq_enable(chip);
>
> return 0;
> }
> @@ -1406,14 +1434,23 @@ static int __maybe_unused axi_dma_runtime_suspend(struct device *dev)
> {
> struct axi_dma_chip *chip = dev_get_drvdata(dev);
>
> - return axi_dma_suspend(chip);
> + if (axi_dma_quiesce_irqs(chip))
> + axi_dma_power_off(chip);
> +
> + return 0;
> }
>
> static int __maybe_unused axi_dma_runtime_resume(struct device *dev)
> {
> struct axi_dma_chip *chip = dev_get_drvdata(dev);
> + int ret;
>
> - return axi_dma_resume(chip);
> + ret = axi_dma_power_on(chip);
> + if (ret)
> + return ret;
> +
> + axi_dma_start_irqs(chip);
> + return 0;
> }
>
> static struct dma_chan *dw_axi_dma_of_xlate(struct of_phandle_args *dma_spec,
> @@ -1560,6 +1597,7 @@ static int dw_probe(struct platform_device *pdev)
> chip->dw = dw;
> chip->dev = &pdev->dev;
> chip->dw->hdata = hdata;
> + spin_lock_init(&chip->irq_lock);
>
> chip->regs = devm_platform_ioremap_resource(pdev, 0);
> if (IS_ERR(chip->regs))
> @@ -1614,10 +1652,6 @@ static int dw_probe(struct platform_device *pdev)
> vchan_init(&chan->vc, &dw->dma);
> }
>
> - ret = axi_req_irqs(pdev, chip);
> - if (ret)
> - return ret;
> -
> /* Set capabilities */
> dma_cap_set(DMA_MEMCPY, dw->dma.cap_mask);
> dma_cap_set(DMA_SLAVE, dw->dma.cap_mask);
> @@ -1656,26 +1690,30 @@ static int dw_probe(struct platform_device *pdev)
> dma_set_max_seg_size(&pdev->dev, MAX_BLOCK_SIZE);
> platform_set_drvdata(pdev, chip);
>
> - pm_runtime_enable(chip->dev);
> -
> - /*
> - * We can't just call pm_runtime_get here instead of
> - * pm_runtime_get_noresume + axi_dma_resume because we need
> - * driver to work also without Runtime PM.
> - */
> + /* Power up explicitly so probe also works without runtime PM. */
> pm_runtime_get_noresume(chip->dev);
> - ret = axi_dma_resume(chip);
> + ret = axi_dma_power_on(chip);
> if (ret < 0)
> - goto err_pm_disable;
> + goto err_pm_put;
>
> axi_dma_hw_init(chip);
>
> - pm_runtime_put(chip->dev);
> + ret = axi_req_irqs(pdev, chip);
> + if (ret)
> + goto err_power_off;
> +
> + ret = pm_runtime_set_active(chip->dev);
> + if (ret)
> + goto err_power_off;
> + pm_runtime_enable(chip->dev);
>
> ret = dmaenginem_async_device_register(&dw->dma);
> if (ret)
> goto err_pm_disable;
>
> + axi_dma_start_irqs(chip);
> + pm_runtime_put(chip->dev);
> +
> /* Register with OF helpers for DMA lookups */
> ret = of_dma_controller_register(pdev->dev.of_node,
> dw_axi_dma_of_xlate, dw);
> @@ -1690,6 +1728,11 @@ static int dw_probe(struct platform_device *pdev)
>
> err_pm_disable:
> pm_runtime_disable(chip->dev);
> +err_power_off:
> + axi_dma_power_off(chip);
> + pm_runtime_set_suspended(chip->dev);
> +err_pm_put:
> + pm_runtime_put_noidle(chip->dev);
>
> return ret;
> }
> @@ -1700,22 +1743,16 @@ static void dw_remove(struct platform_device *pdev)
> struct dw_axi_dma *dw = chip->dw;
> struct axi_dma_chan *chan, *_chan;
> u32 i;
> - int ret;
> -
> - /* Enable clk before accessing to registers */
> - ret = clk_bulk_prepare_enable(ARRAY_SIZE(chip->clks), chip->clks);
> - if (ret)
> - dev_warn(chip->dev, "failed to enable clocks before remove: %d\n",
> - ret);
> - axi_dma_irq_disable(chip);
> - for (i = 0; i < dw->hdata->nr_channels; i++) {
> - axi_chan_disable(&chip->dw->chan[i]);
> - axi_chan_irq_disable(&chip->dw->chan[i], DWAXIDMAC_IRQ_ALL);
> - }
> - axi_dma_disable(chip);
>
> pm_runtime_disable(chip->dev);
> - axi_dma_suspend(chip);
> + if (axi_dma_quiesce_irqs(chip)) {
> + for (i = 0; i < dw->hdata->nr_channels; i++) {
> + axi_chan_disable(&dw->chan[i]);
> + axi_chan_irq_disable(&dw->chan[i], DWAXIDMAC_IRQ_ALL);
> + }
> + axi_dma_power_off(chip);
> + }
> + pm_runtime_set_suspended(chip->dev);
>
> for (i = 0; i < DMAC_MAX_CHANNELS; i++)
> if (chip->irq[i] > 0)
> diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac.h b/drivers/dma/dw-axi-dmac/dw-axi-dmac.h
> index b18c9880711a..80e4632c0d55 100644
> --- a/drivers/dma/dw-axi-dmac/dw-axi-dmac.h
> +++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac.h
> @@ -14,6 +14,7 @@
> #include <linux/clk.h>
> #include <linux/device.h>
> #include <linux/dmaengine.h>
> +#include <linux/spinlock.h>
> #include <linux/types.h>
>
> #include "../virt-dma.h"
> @@ -70,6 +71,8 @@ struct dw_axi_dma {
> struct axi_dma_chip {
> struct device *dev;
> int irq[DMAC_MAX_CHANNELS];
> + spinlock_t irq_lock; /* Protects irq_ready and the global IRQ mask. */
> + bool irq_ready;
> void __iomem *regs;
> void __iomem *apb_regs;
> struct clk_bulk_data clks[2];
>
> --
> 2.53.0
>
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2026-11-19 16:51 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-19 16:00 [PATCH v8 0/9] dmaengine: dw-axi-dmac: fix termination and pause acknowledgment James Hilliard
2026-11-19 16:00 ` [PATCH v8 1/9] dmaengine: dw-axi-dmac: keep cyclic mode in the descriptor James Hilliard
2026-11-19 16:00 ` [PATCH v8 2/9] dmaengine: dw-axi-dmac: poll the correct enable bit during termination James Hilliard
2026-11-19 16:17 ` Frank Li
2026-11-19 16:00 ` [PATCH v8 3/9] dmaengine: dw-axi-dmac: retain descriptors when termination times out James Hilliard
2026-11-19 16:18 ` Frank Li
2026-11-19 16:00 ` [PATCH v8 4/9] dmaengine: dw-axi-dmac: use atomic polling for suspend acknowledgment James Hilliard
2026-11-19 16:19 ` Frank Li
2026-11-19 16:01 ` [PATCH v8 5/9] dmaengine: dw-axi-dmac: preserve pending pause requests James Hilliard
2026-11-19 16:27 ` Frank Li
2026-11-19 16:01 ` [PATCH v8 6/9] dmaengine: dw-axi-dmac: retain descriptors until callbacks drain James Hilliard
2026-11-19 16:41 ` Frank Li
2026-11-19 16:01 ` [PATCH v8 7/9] dmaengine: dw-axi-dmac: serialize interrupt status with channel reuse James Hilliard
2026-11-19 16:44 ` Frank Li
2026-11-19 16:01 ` [PATCH v8 8/9] dmaengine: dw-axi-dmac: require an IRQ before registering the controller James Hilliard
2026-11-19 16:47 ` Frank Li
2026-11-19 16:01 ` [PATCH v8 9/9] dmaengine: dw-axi-dmac: guard shared IRQs across the controller lifetime James Hilliard
2026-11-19 16:50 ` Frank Li
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox