[RFC PATCH 0/1] dmaengine: introduce dmaengine_bh_wq and bh helpers

[RFC PATCH 0/1] dmaengine: introduce dmaengine_bh_wq and bh helpers

[Allen Pais]

Hi Vinod,

 This series contains a single patch that introduces a dedicated dmaengine
bottom-half workqueue along with per-channel BH helpers. The change grew
out of the earlier discussion about moving DMA tasklet usage over to a
BH workqueue and I took some time to get the semantics right
before sending this upstream:

Earlier discussion:
https://lore.kernel.org/all/2e9257af-c123-406b-a189-eaebeecc1d71@app.fastmail.com/

 I’m intentionally sending only the foundational helper patch for review
to avoid spamming the list. The full set of driver conversions is already
available here: https://github.com/allenpais/dmaengine-bh-work

 Once the base helper patch is reviewed, I will follow up with the
remaining driver conversions.

This series is based on v6.19-rc4 (f10c325a345fef0a688a2bcdfab1540d1c924148).

Thanks,
Allen

Allen Pais (1):
  dmaengine: introduce dmaengine_bh_wq and bh helpers

 drivers/dma/amd/qdma/qdma.c                   |   1 +
 drivers/dma/bcm2835-dma.c                     |   2 +-
 drivers/dma/dmaengine.c                       | 109 +++++++++++++++++-
 .../dma/dw-axi-dmac/dw-axi-dmac-platform.c    |   2 +-
 drivers/dma/dw-edma/dw-edma-core.c            |   2 +-
 drivers/dma/hisi_dma.c                        |   2 +-
 drivers/dma/img-mdc-dma.c                     |   2 +-
 drivers/dma/imx-sdma.c                        |   2 +-
 drivers/dma/k3dma.c                           |   2 +-
 drivers/dma/loongson1-apb-dma.c               |   2 +-
 drivers/dma/mediatek/mtk-cqdma.c              |   2 +-
 drivers/dma/mediatek/mtk-hsdma.c              |   2 +-
 drivers/dma/mediatek/mtk-uart-apdma.c         |   4 +-
 drivers/dma/owl-dma.c                         |   2 +-
 drivers/dma/pxa_dma.c                         |   2 +-
 drivers/dma/qcom/bam_dma.c                    |   4 +-
 drivers/dma/qcom/qcom_adm.c                   |   2 +-
 drivers/dma/sa11x0-dma.c                      |   2 +-
 drivers/dma/sprd-dma.c                        |   2 +-
 drivers/dma/sun6i-dma.c                       |   2 +-
 drivers/dma/tegra186-gpc-dma.c                |   2 +-
 drivers/dma/tegra210-adma.c                   |   2 +-
 drivers/dma/ti/k3-udma.c                      |   8 +-
 drivers/dma/ti/omap-dma.c                     |   2 +-
 drivers/dma/virt-dma.c                        |   6 +-
 drivers/dma/virt-dma.h                        |   8 +-
 include/linux/dmaengine.h                     |  50 ++++++++
 27 files changed, 189 insertions(+), 39 deletions(-)

-- 
2.43.0

[RFC PATCH 1/1] dmaengine: introduce dmaengine_bh_wq and bh helpers

[Allen Pais]

Create a dedicated dmaengine bottom-half workqueue (WQ_BH | WQ_PERCPU)
and provide helper APIs for queue/flush/cancel of BH work items. Add
per-channel BH helpers in dma_chan so drivers can schedule a BH callback
without maintaining their own tasklets.

Convert virt-dma to use the new per-channel BH helpers and remove the
per-channel tasklet. Update existing drivers that only need tasklet
teardown to use dma_chan_kill_bh().

This provides a common BH execution path for dmaengine and establishes
the base for converting remaining DMA tasklets to workqueue-based BHs.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/amd/qdma/qdma.c                   |   1 +
 drivers/dma/bcm2835-dma.c                     |   2 +-
 drivers/dma/dmaengine.c                       | 109 +++++++++++++++++-
 .../dma/dw-axi-dmac/dw-axi-dmac-platform.c    |   2 +-
 drivers/dma/dw-edma/dw-edma-core.c            |   2 +-
 drivers/dma/hisi_dma.c                        |   2 +-
 drivers/dma/img-mdc-dma.c                     |   2 +-
 drivers/dma/imx-sdma.c                        |   2 +-
 drivers/dma/k3dma.c                           |   2 +-
 drivers/dma/loongson1-apb-dma.c               |   2 +-
 drivers/dma/mediatek/mtk-cqdma.c              |   2 +-
 drivers/dma/mediatek/mtk-hsdma.c              |   2 +-
 drivers/dma/mediatek/mtk-uart-apdma.c         |   4 +-
 drivers/dma/owl-dma.c                         |   2 +-
 drivers/dma/pxa_dma.c                         |   2 +-
 drivers/dma/qcom/bam_dma.c                    |   4 +-
 drivers/dma/qcom/qcom_adm.c                   |   2 +-
 drivers/dma/sa11x0-dma.c                      |   2 +-
 drivers/dma/sprd-dma.c                        |   2 +-
 drivers/dma/sun6i-dma.c                       |   2 +-
 drivers/dma/tegra186-gpc-dma.c                |   2 +-
 drivers/dma/tegra210-adma.c                   |   2 +-
 drivers/dma/ti/k3-udma.c                      |   8 +-
 drivers/dma/ti/omap-dma.c                     |   2 +-
 drivers/dma/virt-dma.c                        |   6 +-
 drivers/dma/virt-dma.h                        |   8 +-
 include/linux/dmaengine.h                     |  50 ++++++++
 27 files changed, 189 insertions(+), 39 deletions(-)

diff --git a/drivers/dma/amd/qdma/qdma.c b/drivers/dma/amd/qdma/qdma.c
index 8fb2d5e1df20..b57f8ebf2446 100644
--- a/drivers/dma/amd/qdma/qdma.c
+++ b/drivers/dma/amd/qdma/qdma.c
@@ -13,6 +13,7 @@
 #include <linux/platform_device.h>
 #include <linux/platform_data/amd_qdma.h>
 #include <linux/regmap.h>
+#include <linux/interrupt.h>
 
 #include "qdma.h"
 
diff --git a/drivers/dma/bcm2835-dma.c b/drivers/dma/bcm2835-dma.c
index 321748e2983e..08a206cfbb76 100644
--- a/drivers/dma/bcm2835-dma.c
+++ b/drivers/dma/bcm2835-dma.c
@@ -846,7 +846,7 @@ static void bcm2835_dma_free(struct bcm2835_dmadev *od)
 	list_for_each_entry_safe(c, next, &od->ddev.channels,
 				 vc.chan.device_node) {
 		list_del(&c->vc.chan.device_node);
-		tasklet_kill(&c->vc.task);
+		dma_chan_kill_bh(&c->vc.chan);
 	}
 
 	dma_unmap_page_attrs(od->ddev.dev, od->zero_page, PAGE_SIZE,
diff --git a/drivers/dma/dmaengine.c b/drivers/dma/dmaengine.c
index ca13cd39330b..d3df4e943d37 100644
--- a/drivers/dma/dmaengine.c
+++ b/drivers/dma/dmaengine.c
@@ -54,6 +54,7 @@
 #include <linux/of_dma.h>
 #include <linux/mempool.h>
 #include <linux/numa.h>
+#include <linux/workqueue.h>
 
 #include "dmaengine.h"
 
@@ -61,6 +62,7 @@ static DEFINE_MUTEX(dma_list_mutex);
 static DEFINE_IDA(dma_ida);
 static LIST_HEAD(dma_device_list);
 static long dmaengine_ref_count;
+static struct workqueue_struct *dmaengine_bh_wq;
 
 /* --- debugfs implementation --- */
 #ifdef CONFIG_DEBUG_FS
@@ -1425,6 +1427,92 @@ static void dmaengine_destroy_unmap_pool(void)
 	}
 }
 
+static void dmaengine_destroy_bh_wq(void)
+{
+	if (!dmaengine_bh_wq)
+		return;
+
+	destroy_workqueue(dmaengine_bh_wq);
+	dmaengine_bh_wq = NULL;
+}
+
+bool dmaengine_queue_bh_work(struct work_struct *work)
+{
+	if (WARN_ON(!dmaengine_bh_wq))
+		return false;
+
+	return queue_work(dmaengine_bh_wq, work);
+}
+EXPORT_SYMBOL_GPL(dmaengine_queue_bh_work);
+
+void dmaengine_flush_bh_work(struct work_struct *work)
+{
+	if (!work)
+		return;
+
+	flush_work(work);
+}
+EXPORT_SYMBOL_GPL(dmaengine_flush_bh_work);
+
+void dmaengine_cancel_bh_work_sync(struct work_struct *work)
+{
+	if (!work)
+		return;
+
+	cancel_work_sync(work);
+}
+EXPORT_SYMBOL_GPL(dmaengine_cancel_bh_work_sync);
+
+static void dma_chan_bh_entry(struct work_struct *work)
+{
+	struct dma_chan *chan = container_of(work, struct dma_chan, bh_work);
+	dma_chan_bh_work_fn fn = READ_ONCE(chan->bh_work_fn);
+
+	if (fn)
+		fn(chan);
+}
+
+void dma_chan_init_bh(struct dma_chan *chan, dma_chan_bh_work_fn fn)
+{
+	if (WARN_ON(!fn))
+		return;
+
+	if (WARN_ON(chan->bh_work_initialized))
+		return;
+
+	chan->bh_work_fn = fn;
+	INIT_WORK(&chan->bh_work, dma_chan_bh_entry);
+	chan->bh_work_initialized = true;
+}
+EXPORT_SYMBOL_GPL(dma_chan_init_bh);
+
+bool dma_chan_schedule_bh(struct dma_chan *chan)
+{
+	if (WARN_ON(!chan->bh_work_initialized))
+		return false;
+
+	return dmaengine_queue_bh_work(&chan->bh_work);
+}
+EXPORT_SYMBOL_GPL(dma_chan_schedule_bh);
+
+void dma_chan_flush_bh(struct dma_chan *chan)
+{
+	if (!chan->bh_work_initialized)
+		return;
+
+	dmaengine_flush_bh_work(&chan->bh_work);
+}
+EXPORT_SYMBOL_GPL(dma_chan_flush_bh);
+
+void dma_chan_kill_bh(struct dma_chan *chan)
+{
+	if (!chan->bh_work_initialized)
+		return;
+
+	dmaengine_cancel_bh_work_sync(&chan->bh_work);
+}
+EXPORT_SYMBOL_GPL(dma_chan_kill_bh);
+
 static int __init dmaengine_init_unmap_pool(void)
 {
 	int i;
@@ -1621,15 +1709,28 @@ EXPORT_SYMBOL_GPL(dma_run_dependencies);
 
 static int __init dma_bus_init(void)
 {
-	int err = dmaengine_init_unmap_pool();
+	int err;
 
+	dmaengine_bh_wq = alloc_workqueue("dmaengine_bh",
+					  WQ_BH | WQ_PERCPU, 0);
+	if (!dmaengine_bh_wq)
+		return -ENOMEM;
+
+	err = dmaengine_init_unmap_pool();
 	if (err)
-		return err;
+		goto err_destroy_wq;
 
 	err = class_register(&dma_devclass);
-	if (!err)
-		dmaengine_debugfs_init();
+	if (err)
+		goto err_destroy_pool;
 
+	dmaengine_debugfs_init();
+	return 0;
+
+err_destroy_pool:
+	dmaengine_destroy_unmap_pool();
+err_destroy_wq:
+	dmaengine_destroy_bh_wq();
 	return err;
 }
 arch_initcall(dma_bus_init);
diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
index b23536645ff7..42018b21d2ab 100644
--- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
+++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
@@ -1649,7 +1649,7 @@ static void dw_remove(struct platform_device *pdev)
 	list_for_each_entry_safe(chan, _chan, &dw->dma.channels,
 			vc.chan.device_node) {
 		list_del(&chan->vc.chan.device_node);
-		tasklet_kill(&chan->vc.task);
+		dma_chan_kill_bh(&chan->vc.chan);
 	}
 }
 
diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-edma-core.c
index 8e5f7defa6b6..378650bcc430 100644
--- a/drivers/dma/dw-edma/dw-edma-core.c
+++ b/drivers/dma/dw-edma/dw-edma-core.c
@@ -1015,7 +1015,7 @@ int dw_edma_remove(struct dw_edma_chip *chip)
 	dma_async_device_unregister(&dw->dma);
 	list_for_each_entry_safe(chan, _chan, &dw->dma.channels,
 				 vc.chan.device_node) {
-		tasklet_kill(&chan->vc.task);
+		dma_chan_kill_bh(&chan->vc.chan);
 		list_del(&chan->vc.chan.device_node);
 	}
 
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 25a4134be36b..a15491945329 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -720,7 +720,7 @@ static void hisi_dma_disable_qps(struct hisi_dma_dev *hdma_dev)
 
 	for (i = 0; i < hdma_dev->chan_num; i++) {
 		hisi_dma_disable_qp(hdma_dev, i);
-		tasklet_kill(&hdma_dev->chan[i].vc.task);
+		dma_chan_kill_bh(&hdma_dev->chan[i].vc.chan);
 	}
 }
 
diff --git a/drivers/dma/img-mdc-dma.c b/drivers/dma/img-mdc-dma.c
index fd55bcd060ab..d4f6d839acca 100644
--- a/drivers/dma/img-mdc-dma.c
+++ b/drivers/dma/img-mdc-dma.c
@@ -1031,7 +1031,7 @@ static void mdc_dma_remove(struct platform_device *pdev)
 
 		devm_free_irq(&pdev->dev, mchan->irq, mchan);
 
-		tasklet_kill(&mchan->vc.task);
+		dma_chan_kill_bh(&mchan->vc.chan);
 	}
 
 	pm_runtime_disable(&pdev->dev);
diff --git a/drivers/dma/imx-sdma.c b/drivers/dma/imx-sdma.c
index ed9e56de5a9b..9cbc95fbb4ee 100644
--- a/drivers/dma/imx-sdma.c
+++ b/drivers/dma/imx-sdma.c
@@ -2427,7 +2427,7 @@ static void sdma_remove(struct platform_device *pdev)
 	for (i = 0; i < MAX_DMA_CHANNELS; i++) {
 		struct sdma_channel *sdmac = &sdma->channel[i];
 
-		tasklet_kill(&sdmac->vc.task);
+		dma_chan_kill_bh(&sdmac->vc.chan);
 		sdma_free_chan_resources(&sdmac->vc.chan);
 	}
 
diff --git a/drivers/dma/k3dma.c b/drivers/dma/k3dma.c
index 0f9cd7815f88..36d5df545b57 100644
--- a/drivers/dma/k3dma.c
+++ b/drivers/dma/k3dma.c
@@ -981,7 +981,7 @@ static void k3_dma_remove(struct platform_device *op)
 
 	list_for_each_entry_safe(c, cn, &d->slave.channels, vc.chan.device_node) {
 		list_del(&c->vc.chan.device_node);
-		tasklet_kill(&c->vc.task);
+		dma_chan_kill_bh(&c->vc.chan);
 	}
 	tasklet_kill(&d->task);
 	clk_disable_unprepare(d->clk);
diff --git a/drivers/dma/loongson1-apb-dma.c b/drivers/dma/loongson1-apb-dma.c
index 255fe7eca212..49a78ea1514f 100644
--- a/drivers/dma/loongson1-apb-dma.c
+++ b/drivers/dma/loongson1-apb-dma.c
@@ -552,7 +552,7 @@ static void ls1x_dma_chan_remove(struct ls1x_dma *dma)
 
 		if (chan->vc.chan.device == &dma->ddev) {
 			list_del(&chan->vc.chan.device_node);
-			tasklet_kill(&chan->vc.task);
+			dma_chan_kill_bh(&chan->vc.chan);
 		}
 	}
 }
diff --git a/drivers/dma/mediatek/mtk-cqdma.c b/drivers/dma/mediatek/mtk-cqdma.c
index 9f0c41ca7770..3ba54df12bae 100644
--- a/drivers/dma/mediatek/mtk-cqdma.c
+++ b/drivers/dma/mediatek/mtk-cqdma.c
@@ -895,7 +895,7 @@ static void mtk_cqdma_remove(struct platform_device *pdev)
 		vc = &cqdma->vc[i];
 
 		list_del(&vc->vc.chan.device_node);
-		tasklet_kill(&vc->vc.task);
+		dma_chan_kill_bh(&vc->vc.chan);
 	}
 
 	/* disable interrupt */
diff --git a/drivers/dma/mediatek/mtk-hsdma.c b/drivers/dma/mediatek/mtk-hsdma.c
index fa77bb24a430..0bbf865de24b 100644
--- a/drivers/dma/mediatek/mtk-hsdma.c
+++ b/drivers/dma/mediatek/mtk-hsdma.c
@@ -1020,7 +1020,7 @@ static void mtk_hsdma_remove(struct platform_device *pdev)
 		vc = &hsdma->vc[i];
 
 		list_del(&vc->vc.chan.device_node);
-		tasklet_kill(&vc->vc.task);
+		dma_chan_kill_bh(&vc->vc.chan);
 	}
 
 	/* Disable DMA interrupt */
diff --git a/drivers/dma/mediatek/mtk-uart-apdma.c b/drivers/dma/mediatek/mtk-uart-apdma.c
index 08e15177427b..257b9b77cc57 100644
--- a/drivers/dma/mediatek/mtk-uart-apdma.c
+++ b/drivers/dma/mediatek/mtk-uart-apdma.c
@@ -312,7 +312,7 @@ static void mtk_uart_apdma_free_chan_resources(struct dma_chan *chan)
 
 	free_irq(c->irq, chan);
 
-	tasklet_kill(&c->vc.task);
+	dma_chan_kill_bh(&c->vc.chan);
 
 	vchan_free_chan_resources(&c->vc);
 
@@ -463,7 +463,7 @@ static void mtk_uart_apdma_free(struct mtk_uart_apdmadev *mtkd)
 			struct mtk_chan, vc.chan.device_node);
 
 		list_del(&c->vc.chan.device_node);
-		tasklet_kill(&c->vc.task);
+		dma_chan_kill_bh(&c->vc.chan);
 	}
 }
 
diff --git a/drivers/dma/owl-dma.c b/drivers/dma/owl-dma.c
index 57cec757d8f5..4e6ebf990688 100644
--- a/drivers/dma/owl-dma.c
+++ b/drivers/dma/owl-dma.c
@@ -1055,7 +1055,7 @@ static inline void owl_dma_free(struct owl_dma *od)
 	list_for_each_entry_safe(vchan,
 				 next, &od->dma.channels, vc.chan.device_node) {
 		list_del(&vchan->vc.chan.device_node);
-		tasklet_kill(&vchan->vc.task);
+		dma_chan_kill_bh(&vchan->vc.chan);
 	}
 }
 
diff --git a/drivers/dma/pxa_dma.c b/drivers/dma/pxa_dma.c
index 249296389771..634b723e4891 100644
--- a/drivers/dma/pxa_dma.c
+++ b/drivers/dma/pxa_dma.c
@@ -1218,7 +1218,7 @@ static void pxad_free_channels(struct dma_device *dmadev)
 	list_for_each_entry_safe(c, cn, &dmadev->channels,
 				 vc.chan.device_node) {
 		list_del(&c->vc.chan.device_node);
-		tasklet_kill(&c->vc.task);
+		dma_chan_kill_bh(&c->vc.chan);
 	}
 }
 
diff --git a/drivers/dma/qcom/bam_dma.c b/drivers/dma/qcom/bam_dma.c
index 2cf060174795..6ec82adb89ce 100644
--- a/drivers/dma/qcom/bam_dma.c
+++ b/drivers/dma/qcom/bam_dma.c
@@ -1377,7 +1377,7 @@ static int bam_dma_probe(struct platform_device *pdev)
 	dma_async_device_unregister(&bdev->common);
 err_bam_channel_exit:
 	for (i = 0; i < bdev->num_channels; i++)
-		tasklet_kill(&bdev->channels[i].vc.task);
+		dma_chan_kill_bh(&bdev->channels[i].vc.chan);
 err_tasklet_kill:
 	tasklet_kill(&bdev->task);
 err_disable_clk:
@@ -1403,7 +1403,7 @@ static void bam_dma_remove(struct platform_device *pdev)
 
 	for (i = 0; i < bdev->num_channels; i++) {
 		bam_dma_terminate_all(&bdev->channels[i].vc.chan);
-		tasklet_kill(&bdev->channels[i].vc.task);
+		dma_chan_kill_bh(&bdev->channels[i].vc.chan);
 
 		if (!bdev->channels[i].fifo_virt)
 			continue;
diff --git a/drivers/dma/qcom/qcom_adm.c b/drivers/dma/qcom/qcom_adm.c
index 6be54fddcee1..0f9e906bd463 100644
--- a/drivers/dma/qcom/qcom_adm.c
+++ b/drivers/dma/qcom/qcom_adm.c
@@ -919,7 +919,7 @@ static void adm_dma_remove(struct platform_device *pdev)
 		/* mask IRQs for this channel/EE pair */
 		writel(0, adev->regs + ADM_CH_RSLT_CONF(achan->id, adev->ee));
 
-		tasklet_kill(&adev->channels[i].vc.task);
+		dma_chan_kill_bh(&adev->channels[i].vc.chan);
 		adm_terminate_all(&adev->channels[i].vc.chan);
 	}
 
diff --git a/drivers/dma/sa11x0-dma.c b/drivers/dma/sa11x0-dma.c
index dc1a9a05252e..42940079efbf 100644
--- a/drivers/dma/sa11x0-dma.c
+++ b/drivers/dma/sa11x0-dma.c
@@ -893,7 +893,7 @@ static void sa11x0_dma_free_channels(struct dma_device *dmadev)
 
 	list_for_each_entry_safe(c, cn, &dmadev->channels, vc.chan.device_node) {
 		list_del(&c->vc.chan.device_node);
-		tasklet_kill(&c->vc.task);
+		dma_chan_kill_bh(&c->vc.chan);
 		kfree(c);
 	}
 }
diff --git a/drivers/dma/sprd-dma.c b/drivers/dma/sprd-dma.c
index 6207e0b185e1..5124fe0a93bf 100644
--- a/drivers/dma/sprd-dma.c
+++ b/drivers/dma/sprd-dma.c
@@ -1253,7 +1253,7 @@ static void sprd_dma_remove(struct platform_device *pdev)
 	list_for_each_entry_safe(c, cn, &sdev->dma_dev.channels,
 				 vc.chan.device_node) {
 		list_del(&c->vc.chan.device_node);
-		tasklet_kill(&c->vc.task);
+		dma_chan_kill_bh(&c->vc.chan);
 	}
 
 	of_dma_controller_free(pdev->dev.of_node);
diff --git a/drivers/dma/sun6i-dma.c b/drivers/dma/sun6i-dma.c
index 2215ff877bf7..6b3ab99272a5 100644
--- a/drivers/dma/sun6i-dma.c
+++ b/drivers/dma/sun6i-dma.c
@@ -1073,7 +1073,7 @@ static inline void sun6i_dma_free(struct sun6i_dma_dev *sdev)
 		struct sun6i_vchan *vchan = &sdev->vchans[i];
 
 		list_del(&vchan->vc.chan.device_node);
-		tasklet_kill(&vchan->vc.task);
+		dma_chan_kill_bh(&vchan->vc.chan);
 	}
 }
 
diff --git a/drivers/dma/tegra186-gpc-dma.c b/drivers/dma/tegra186-gpc-dma.c
index 4d6fe0efa76e..3b9dc64eb635 100644
--- a/drivers/dma/tegra186-gpc-dma.c
+++ b/drivers/dma/tegra186-gpc-dma.c
@@ -1279,7 +1279,7 @@ static void tegra_dma_free_chan_resources(struct dma_chan *dc)
 	tegra_dma_terminate_all(dc);
 	synchronize_irq(tdc->irq);
 
-	tasklet_kill(&tdc->vc.task);
+	dma_chan_kill_bh(&tdc->vc.chan);
 	tdc->config_init = false;
 	tdc->slave_id = -1;
 	tdc->sid_dir = DMA_TRANS_NONE;
diff --git a/drivers/dma/tegra210-adma.c b/drivers/dma/tegra210-adma.c
index d0e8bb27a03b..67ccb1d48361 100644
--- a/drivers/dma/tegra210-adma.c
+++ b/drivers/dma/tegra210-adma.c
@@ -793,7 +793,7 @@ static void tegra_adma_free_chan_resources(struct dma_chan *dc)
 
 	tegra_adma_terminate_all(dc);
 	vchan_free_chan_resources(&tdc->vc);
-	tasklet_kill(&tdc->vc.task);
+	dma_chan_kill_bh(&tdc->vc.chan);
 	free_irq(tdc->irq, tdc);
 	pm_runtime_put(tdc2dev(tdc));
 
diff --git a/drivers/dma/ti/k3-udma.c b/drivers/dma/ti/k3-udma.c
index aa2dc762140f..89dd7926705d 100644
--- a/drivers/dma/ti/k3-udma.c
+++ b/drivers/dma/ti/k3-udma.c
@@ -4045,9 +4045,9 @@ static void udma_desc_pre_callback(struct virt_dma_chan *vc,
  * This tasklet handles the completion of a DMA descriptor by
  * calling its callback and freeing it.
  */
-static void udma_vchan_complete(struct tasklet_struct *t)
+static void udma_vchan_complete(struct dma_chan *chan)
 {
-	struct virt_dma_chan *vc = from_tasklet(vc, t, task);
+	struct virt_dma_chan *vc = to_virt_chan(chan);
 	struct virt_dma_desc *vd, *_vd;
 	struct dmaengine_desc_callback cb;
 	LIST_HEAD(head);
@@ -4112,7 +4112,7 @@ static void udma_free_chan_resources(struct dma_chan *chan)
 	}
 
 	vchan_free_chan_resources(&uc->vc);
-	tasklet_kill(&uc->vc.task);
+	dma_chan_kill_bh(&uc->vc.chan);
 
 	bcdma_free_bchan_resources(uc);
 	udma_free_tx_resources(uc);
@@ -5628,7 +5628,7 @@ static int udma_probe(struct platform_device *pdev)
 			return -ENOMEM;
 		vchan_init(&uc->vc, &ud->ddev);
 		/* Use custom vchan completion handling */
-		tasklet_setup(&uc->vc.task, udma_vchan_complete);
+		dma_chan_init_bh(&uc->vc.chan, udma_vchan_complete);
 		init_completion(&uc->teardown_completed);
 		INIT_DELAYED_WORK(&uc->tx_drain.work, udma_check_tx_completion);
 	}
diff --git a/drivers/dma/ti/omap-dma.c b/drivers/dma/ti/omap-dma.c
index 8c023c6e623a..f6aee92071e2 100644
--- a/drivers/dma/ti/omap-dma.c
+++ b/drivers/dma/ti/omap-dma.c
@@ -1521,7 +1521,7 @@ static void omap_dma_free(struct omap_dmadev *od)
 			struct omap_chan, vc.chan.device_node);
 
 		list_del(&c->vc.chan.device_node);
-		tasklet_kill(&c->vc.task);
+		dma_chan_kill_bh(&c->vc.chan);
 		kfree(c);
 	}
 }
diff --git a/drivers/dma/virt-dma.c b/drivers/dma/virt-dma.c
index 7961172a780d..c311397f3bd7 100644
--- a/drivers/dma/virt-dma.c
+++ b/drivers/dma/virt-dma.c
@@ -80,9 +80,9 @@ EXPORT_SYMBOL_GPL(vchan_find_desc);
  * This tasklet handles the completion of a DMA descriptor by
  * calling its callback and freeing it.
  */
-static void vchan_complete(struct tasklet_struct *t)
+static void vchan_complete(struct dma_chan *chan)
 {
-	struct virt_dma_chan *vc = from_tasklet(vc, t, task);
+	struct virt_dma_chan *vc = to_virt_chan(chan);
 	struct virt_dma_desc *vd, *_vd;
 	struct dmaengine_desc_callback cb;
 	LIST_HEAD(head);
@@ -131,7 +131,7 @@ void vchan_init(struct virt_dma_chan *vc, struct dma_device *dmadev)
 	INIT_LIST_HEAD(&vc->desc_completed);
 	INIT_LIST_HEAD(&vc->desc_terminated);
 
-	tasklet_setup(&vc->task, vchan_complete);
+	dma_chan_init_bh(&vc->chan, vchan_complete);
 
 	vc->chan.device = dmadev;
 	list_add_tail(&vc->chan.device_node, &dmadev->channels);
diff --git a/drivers/dma/virt-dma.h b/drivers/dma/virt-dma.h
index 59d9eabc8b67..5299fb7367ca 100644
--- a/drivers/dma/virt-dma.h
+++ b/drivers/dma/virt-dma.h
@@ -8,7 +8,6 @@
 #define VIRT_DMA_H
 
 #include <linux/dmaengine.h>
-#include <linux/interrupt.h>
 
 #include "dmaengine.h"
 
@@ -21,7 +20,6 @@ struct virt_dma_desc {
 
 struct virt_dma_chan {
 	struct dma_chan	chan;
-	struct tasklet_struct task;
 	void (*desc_free)(struct virt_dma_desc *);
 
 	spinlock_t lock;
@@ -106,7 +104,7 @@ static inline void vchan_cookie_complete(struct virt_dma_desc *vd)
 		 vd, cookie);
 	list_add_tail(&vd->node, &vc->desc_completed);
 
-	tasklet_schedule(&vc->task);
+	dma_chan_schedule_bh(&vc->chan);
 }
 
 /**
@@ -137,7 +135,7 @@ static inline void vchan_cyclic_callback(struct virt_dma_desc *vd)
 	struct virt_dma_chan *vc = to_virt_chan(vd->tx.chan);
 
 	vc->cyclic = vd;
-	tasklet_schedule(&vc->task);
+	dma_chan_schedule_bh(&vc->chan);
 }
 
 /**
@@ -223,7 +221,7 @@ static inline void vchan_synchronize(struct virt_dma_chan *vc)
 	LIST_HEAD(head);
 	unsigned long flags;
 
-	tasklet_kill(&vc->task);
+	dma_chan_kill_bh(&vc->chan);
 
 	spin_lock_irqsave(&vc->lock, flags);
 
diff --git a/include/linux/dmaengine.h b/include/linux/dmaengine.h
index 99efe2b9b4ea..00ad92a73c3b 100644
--- a/include/linux/dmaengine.h
+++ b/include/linux/dmaengine.h
@@ -12,6 +12,7 @@
 #include <linux/scatterlist.h>
 #include <linux/bitmap.h>
 #include <linux/types.h>
+#include <linux/workqueue.h>
 #include <asm/page.h>
 
 /**
@@ -295,6 +296,10 @@ enum dma_desc_metadata_mode {
 	DESC_METADATA_ENGINE = BIT(1),
 };
 
+struct dma_chan;
+
+typedef void (*dma_chan_bh_work_fn)(struct dma_chan *chan);
+
 /**
  * struct dma_chan_percpu - the per-CPU part of struct dma_chan
  * @memcpy_count: transaction counter
@@ -334,6 +339,9 @@ struct dma_router {
  * @router: pointer to the DMA router structure
  * @route_data: channel specific data for the router
  * @private: private data for certain client-channel associations
+ * @bh_work: bottom-half work item stored per-channel
+ * @bh_work_fn: callback executed when @bh_work runs
+ * @bh_work_initialized: indicates whether @bh_work has been initialized
  */
 struct dma_chan {
 	struct dma_device *device;
@@ -359,6 +367,9 @@ struct dma_chan {
 	void *route_data;
 
 	void *private;
+	struct work_struct bh_work;
+	dma_chan_bh_work_fn bh_work_fn;
+	bool bh_work_initialized;
 };
 
 /**
@@ -1528,6 +1539,14 @@ struct dma_chan *devm_dma_request_chan(struct device *dev, const char *name);
 
 void dma_release_channel(struct dma_chan *chan);
 int dma_get_slave_caps(struct dma_chan *chan, struct dma_slave_caps *caps);
+bool dmaengine_queue_bh_work(struct work_struct *work);
+void dmaengine_flush_bh_work(struct work_struct *work);
+void dmaengine_cancel_bh_work_sync(struct work_struct *work);
+
+void dma_chan_init_bh(struct dma_chan *chan, dma_chan_bh_work_fn fn);
+bool dma_chan_schedule_bh(struct dma_chan *chan);
+void dma_chan_flush_bh(struct dma_chan *chan);
+void dma_chan_kill_bh(struct dma_chan *chan);
 #else
 static inline struct dma_chan *dma_find_channel(enum dma_transaction_type tx_type)
 {
@@ -1575,6 +1594,37 @@ static inline int dma_get_slave_caps(struct dma_chan *chan,
 {
 	return -ENXIO;
 }
+
+static inline bool dmaengine_queue_bh_work(struct work_struct *work)
+{
+	return false;
+}
+
+static inline void dmaengine_flush_bh_work(struct work_struct *work)
+{
+}
+
+static inline void dmaengine_cancel_bh_work_sync(struct work_struct *work)
+{
+}
+
+static inline void dma_chan_init_bh(struct dma_chan *chan,
+				    dma_chan_bh_work_fn fn)
+{
+}
+
+static inline bool dma_chan_schedule_bh(struct dma_chan *chan)
+{
+	return false;
+}
+
+static inline void dma_chan_flush_bh(struct dma_chan *chan)
+{
+}
+
+static inline void dma_chan_kill_bh(struct dma_chan *chan)
+{
+}
 #endif
 
 static inline int dmaengine_desc_set_reuse(struct dma_async_tx_descriptor *tx)
-- 
2.43.0

Re: [RFC PATCH 1/1] dmaengine: introduce dmaengine_bh_wq and bh helpers

[Arnd Bergmann]

On Thu, Jan 8, 2026, at 09:03, Allen Pais wrote:
> Create a dedicated dmaengine bottom-half workqueue (WQ_BH | WQ_PERCPU)
> and provide helper APIs for queue/flush/cancel of BH work items. Add
> per-channel BH helpers in dma_chan so drivers can schedule a BH callback
> without maintaining their own tasklets.
>
> Convert virt-dma to use the new per-channel BH helpers and remove the
> per-channel tasklet. Update existing drivers that only need tasklet
> teardown to use dma_chan_kill_bh().
>
> This provides a common BH execution path for dmaengine and establishes
> the base for converting remaining DMA tasklets to workqueue-based BHs.
>
> Signed-off-by: Allen Pais <allen.lkml@gmail.com>

Hi Allen,

I agree that the dmaengine code should stop using tasklets here,
but I think the last time we discussed this, we ended up not
going with work queues as a replacement because of the inherent
scheduling overhead. 

The use of this tasklet is to invoke the dmaengine_desc_callback(),
which at the moment clearly expects to be called from tasklet
context, but in most cases probably should just be called from
hardirq context, e.g. when all it does is to call complete()
or wake_up(). In particular, I assume this breaks any console
driver that tries to use DMA to send output to a UART.

It may make sense to take the portions of your patch that
abstract the dmaengine drivers away from tasklet and have them
interact with shared functions, but I don't think we should
introduce a workqueue at all here, at least not until we
have identified dmaengine users that want workqueue behavior.

If your goal is to reduce the number of tasklet uses in the
kernel, I would suggest taking this on at one level higher up
the stack: assume that dma_async_tx_descriptor->callback()
is always called at tasklet context, and introduce an
alternative mechanism that is called from hardirq context,
then change over each user of dma_async_tx_descriptor to
use the hardirq method instead of the tasklet method, if
at all possible.

      Arnd

Re: [RFC PATCH 1/1] dmaengine: introduce dmaengine_bh_wq and bh helpers

[Allen]

> > Create a dedicated dmaengine bottom-half workqueue (WQ_BH | WQ_PERCPU)
> > and provide helper APIs for queue/flush/cancel of BH work items. Add
> > per-channel BH helpers in dma_chan so drivers can schedule a BH callback
> > without maintaining their own tasklets.
> >
> > Convert virt-dma to use the new per-channel BH helpers and remove the
> > per-channel tasklet. Update existing drivers that only need tasklet
> > teardown to use dma_chan_kill_bh().
> >
> > This provides a common BH execution path for dmaengine and establishes
> > the base for converting remaining DMA tasklets to workqueue-based BHs.
> >
> > Signed-off-by: Allen Pais <allen.lkml@gmail.com>
>
> Hi Allen,
>
> I agree that the dmaengine code should stop using tasklets here,
> but I think the last time we discussed this, we ended up not
> going with work queues as a replacement because of the inherent
> scheduling overhead.
>
> The use of this tasklet is to invoke the dmaengine_desc_callback(),
> which at the moment clearly expects to be called from tasklet
> context, but in most cases probably should just be called from
> hardirq context, e.g. when all it does is to call complete()
> or wake_up(). In particular, I assume this breaks any console
> driver that tries to use DMA to send output to a UART.
>
> It may make sense to take the portions of your patch that
> abstract the dmaengine drivers away from tasklet and have them
> interact with shared functions, but I don't think we should
> introduce a workqueue at all here, at least not until we
> have identified dmaengine users that want workqueue behavior.
>
> If your goal is to reduce the number of tasklet uses in the
> kernel, I would suggest taking this on at one level higher up
> the stack: assume that dma_async_tx_descriptor->callback()
> is always called at tasklet context, and introduce an
> alternative mechanism that is called from hardirq context,
> then change over each user of dma_async_tx_descriptor to
> use the hardirq method instead of the tasklet method, if
> at all possible.

Arnd,

   Thanks for the feedback. My intent with WQ_BH was to keep callbacks in
  softirq/BH context, but I agree the scheduling overhead and existing tasklet
  assumptions are valid concerns.

  I can re-spin the RFC to drop the workqueue entirely and keep
tasklet semantics,
  while still abstracting tasklet handling into dmaengine helpers so drivers no
  longer directly manipulate tasklets. That keeps
dmaengine_desc_callback_invoke()
  in tasklet context and avoids breaking DMA users that rely on that behavior.

  A hardirq callback path feels like a larger API change, so I’d
prefer to handle
  that as a separate follow‑up (e.g. explicit hardirq callback/flag + user
  migration where safe). Thoughts?

  Here’s a diff on top of v6.19-rc4 that does exactly this.

diff --git a/drivers/dma/dmaengine.c b/drivers/dma/dmaengine.c
  index ca13cd39330b..611047763b35 100644
  --- a/drivers/dma/dmaengine.c
  +++ b/drivers/dma/dmaengine.c
  @@ -1425,6 +1425,48 @@ static void dmaengine_destroy_unmap_pool(void)
       }
   }

  +static void dma_chan_bh_entry(struct tasklet_struct *t)
  +{
  +    struct dma_chan *chan = from_tasklet(chan, t, bh_tasklet);
  +    dma_chan_bh_work_fn fn = READ_ONCE(chan->bh_work_fn);
  +
  +    if (fn)
  +        fn(chan);
  +}
  +
  +void dma_chan_init_bh(struct dma_chan *chan, dma_chan_bh_work_fn fn)
  +{
  +    if (WARN_ON(!fn))
  +        return;
  +
  +    if (WARN_ON(chan->bh_work_initialized))
  +        return;
  +
  +    chan->bh_work_fn = fn;
  +    tasklet_setup(&chan->bh_tasklet, dma_chan_bh_entry);
  +    chan->bh_work_initialized = true;
  +}
  +EXPORT_SYMBOL_GPL(dma_chan_init_bh);
  +
  +bool dma_chan_schedule_bh(struct dma_chan *chan)
  +{
  +    if (WARN_ON(!chan->bh_work_initialized))
  +        return false;
  +
  +    tasklet_schedule(&chan->bh_tasklet);
  +    return true;
  +}
  +EXPORT_SYMBOL_GPL(dma_chan_schedule_bh);
  +
  +void dma_chan_kill_bh(struct dma_chan *chan)
  +{
  +    if (!chan->bh_work_initialized)
  +        return;
  +
  +    tasklet_kill(&chan->bh_tasklet);
  +}
  +EXPORT_SYMBOL_GPL(dma_chan_kill_bh);
  +
   static int __init dmaengine_init_unmap_pool(void)
   {
       int i;
  diff --git a/drivers/dma/virt-dma.c b/drivers/dma/virt-dma.c
  index 7961172a780d..3d3fadb81f8d 100644
  --- a/drivers/dma/virt-dma.c
  +++ b/drivers/dma/virt-dma.c
  @@ -77,12 +77,12 @@ struct virt_dma_desc *vchan_find_desc(struct
virt_dma_chan *vc,
   EXPORT_SYMBOL_GPL(vchan_find_desc);

   /*
  - * This tasklet handles the completion of a DMA descriptor by
  - * calling its callback and freeing it.
  + * This bottom-half handler completes a DMA descriptor by invoking its
  + * callback and freeing it.
    */
  -static void vchan_complete(struct tasklet_struct *t)
  +static void vchan_complete(struct dma_chan *chan)
   {
  -    struct virt_dma_chan *vc = from_tasklet(vc, t, task);
  +    struct virt_dma_chan *vc = to_virt_chan(chan);
       struct virt_dma_desc *vd, *_vd;
       struct dmaengine_desc_callback cb;
       LIST_HEAD(head);
  @@ -131,7 +131,7 @@ void vchan_init(struct virt_dma_chan *vc, struct
dma_device *dmadev)
       INIT_LIST_HEAD(&vc->desc_completed);
       INIT_LIST_HEAD(&vc->desc_terminated);

  -    tasklet_setup(&vc->task, vchan_complete);
  +    dma_chan_init_bh(&vc->chan, vchan_complete);

       vc->chan.device = dmadev;
       list_add_tail(&vc->chan.device_node, &dmadev->channels);
  diff --git a/drivers/dma/virt-dma.h b/drivers/dma/virt-dma.h
  index 59d9eabc8b67..75c6fee2f70e 100644
  --- a/drivers/dma/virt-dma.h
  +++ b/drivers/dma/virt-dma.h
  @@ -8,8 +8,6 @@
   #define VIRT_DMA_H

   #include <linux/dmaengine.h>
  -#include <linux/interrupt.h>
  -
   #include "dmaengine.h"

   struct virt_dma_desc {
  @@ -21,7 +19,6 @@ struct virt_dma_desc {

   struct virt_dma_chan {
       struct dma_chan    chan;
  -    struct tasklet_struct task;
       void (*desc_free)(struct virt_dma_desc *);

       spinlock_t lock;
  @@ -106,7 +103,7 @@ static inline void vchan_cookie_complete(struct
virt_dma_desc *vd)
            vd, cookie);
       list_add_tail(&vd->node, &vc->desc_completed);

  -    tasklet_schedule(&vc->task);
  +    dma_chan_schedule_bh(&vc->chan);
   }
  @@ -137,7 +134,7 @@ static inline void vchan_cyclic_callback(struct
virt_dma_desc *vd)
       struct virt_dma_chan *vc = to_virt_chan(vd->tx.chan);

       vc->cyclic = vd;
  -    tasklet_schedule(&vc->task);
  +    dma_chan_schedule_bh(&vc->chan);
   }
  @@ -223,7 +220,7 @@ static inline void vchan_synchronize(struct
virt_dma_chan *vc)
       LIST_HEAD(head);
       unsigned long flags;

  -    tasklet_kill(&vc->task);
  +    dma_chan_kill_bh(&vc->chan);

       spin_lock_irqsave(&vc->lock, flags);
  diff --git a/include/linux/dmaengine.h b/include/linux/dmaengine.h
  index 99efe2b9b4ea..2d2c8ab3764d 100644
  --- a/include/linux/dmaengine.h
  +++ b/include/linux/dmaengine.h
  @@ -12,6 +12,7 @@
   #include <linux/scatterlist.h>
   #include <linux/bitmap.h>
   #include <linux/types.h>
  +#include <linux/interrupt.h>
   #include <asm/page.h>
  @@ -295,6 +296,10 @@ enum dma_desc_metadata_mode {
       DESC_METADATA_ENGINE = BIT(1),
   };

  +struct dma_chan;
  +
  +typedef void (*dma_chan_bh_work_fn)(struct dma_chan *chan);
  +
  @@ -334,6 +339,9 @@ struct dma_router {
    * @router: pointer to the DMA router structure
    * @route_data: channel specific data for the router
    * @private: private data for certain client-channel associations
  + * @bh_tasklet: bottom-half tasklet stored per-channel
  + * @bh_work_fn: callback executed when @bh_tasklet runs
  + * @bh_work_initialized: indicates whether @bh_tasklet has been initialized
    */
   struct dma_chan {
  @@ -359,6 +367,9 @@ struct dma_chan {
       void *route_data;

       void *private;
  +    struct tasklet_struct bh_tasklet;
  +    dma_chan_bh_work_fn bh_work_fn;
  +    bool bh_work_initialized;
   };
  @@ -1528,6 +1539,9 @@ struct dma_chan *devm_dma_request_chan(struct
device *dev, const char *name);

   void dma_release_channel(struct dma_chan *chan);
   int dma_get_slave_caps(struct dma_chan *chan, struct dma_slave_caps *caps);
  +void dma_chan_init_bh(struct dma_chan *chan, dma_chan_bh_work_fn fn);
  +bool dma_chan_schedule_bh(struct dma_chan *chan);
  +void dma_chan_kill_bh(struct dma_chan *chan);
   #else
  @@ -1575,6 +1589,20 @@ static inline int dma_get_slave_caps(struct
dma_chan *chan,
   {
       return -ENXIO;
   }
  +
  +static inline void dma_chan_init_bh(struct dma_chan *chan,
  +                    dma_chan_bh_work_fn fn)
  +{
  +}
  +
  +static inline bool dma_chan_schedule_bh(struct dma_chan *chan)
  +{
  +    return false;
  +}
  +
  +static inline void dma_chan_kill_bh(struct dma_chan *chan)
  +{
  +}
   #endif


>
>       Arnd



-- 
       - Allen

Re: [RFC PATCH 1/1] dmaengine: introduce dmaengine_bh_wq and bh helpers

[Arnd Bergmann]

On Thu, Jan 8, 2026, at 20:22, Allen wrote:
>
>    Thanks for the feedback. My intent with WQ_BH was to keep callbacks in
>   softirq/BH context, but I agree the scheduling overhead and existing tasklet
>   assumptions are valid concerns.

Hi Allen,

Sorry about missing the bit about WQ_BH, I forgot about the details
of or previous discussion and this is pretty much what I had
suggested last time, 

>   I can re-spin the RFC to drop the workqueue entirely and keep
> tasklet semantics,
>   while still abstracting tasklet handling into dmaengine helpers so drivers no
>   longer directly manipulate tasklets. That keeps
> dmaengine_desc_callback_invoke()
>   in tasklet context and avoids breaking DMA users that rely on that behavior.

It's probably fine to do both, but a series of two patches (first introduce
the per-channel API, then move it over to WQ_BH) may be slightly
clearer here.

I'm not sure why the dmaengine_*_bh_wq() functions are exported
interfaces, as far as I can tell, you use them only internally
in the dma_chan_*_bh() functions, so making them static would
let the compiler inline them where possible.

There are of course dmaengine drivers that use tasklets for other
purposes than the channel callback. Was your idea here to use
the same workqueue for these? I would perhaps hold off on that for
the moment and see if there is a better alternative for those,
possibly hardirq context, threaded irq or a regular workqueue
depending on the driver.

>   A hardirq callback path feels like a larger API change, so I’d
> prefer to handle that as a separate follow‑up (e.g. explicit hardirq
>  callback/flag + user migration where safe). Thoughts?

Yes, definitely keep that separate. I still think that this is
what we want eventually for a bigger improvement, but your
patch seems valuable on its own as well.

Some more thoughts on where that later change could take us:

>    /*
>   - * This tasklet handles the completion of a DMA descriptor by
>   - * calling its callback and freeing it.
>   + * This bottom-half handler completes a DMA descriptor by invoking its
>   + * callback and freeing it.
>     */
>   -static void vchan_complete(struct tasklet_struct *t)
>   +static void vchan_complete(struct dma_chan *chan)
>    {
>   -    struct virt_dma_chan *vc = from_tasklet(vc, t, task);
>   +    struct virt_dma_chan *vc = to_virt_chan(chan);
>        struct virt_dma_desc *vd, *_vd;
>        struct dmaengine_desc_callback cb;
>        LIST_HEAD(head);
>   @@ -131,7 +131,7 @@ void vchan_init(struct virt_dma_chan *vc, struct
> dma_device *dmadev)
>        INIT_LIST_HEAD(&vc->desc_completed);
>        INIT_LIST_HEAD(&vc->desc_terminated);
>
>   -    tasklet_setup(&vc->task, vchan_complete);
>   +    dma_chan_init_bh(&vc->chan, vchan_complete);

This is where I think it makes sense to start, again for
the vchan imlmenentation. What the dmaengine drivers have
is a per-driver tasklet (or WQ_BH) with a single complete()
function that directly calls into the client drivers for
each completion that has happened.

Since the context we want depends on the type of client
driver, I think a good approach would be to start
by modifying vchan_cookie_complete() to allow it to
call the callback function directly from hardirq context
when the client asks for that, and avoid the round trip
through the tasklet where possible.

A new bit in the dma_ctrl_flags word could be used
to ask for hardirq vs softirq context, and the existing
drivers just fall back to using a tasklet for softirq
context.

      Arnd

Re: [RFC PATCH 1/1] dmaengine: introduce dmaengine_bh_wq and bh helpers

[Allen]

> >    Thanks for the feedback. My intent with WQ_BH was to keep callbacks in
> >   softirq/BH context, but I agree the scheduling overhead and existing tasklet
> >   assumptions are valid concerns.
>
> Hi Allen,
>
> Sorry about missing the bit about WQ_BH, I forgot about the details
> of or previous discussion and this is pretty much what I had
> suggested last time,
>
Hi Arnd,

  Sorry I missed the WQ_BH point in my reply, I’d forgotten the details of our
  earlier discussion, and I should have followed up on this sooner.

> >   I can re-spin the RFC to drop the workqueue entirely and keep
> > tasklet semantics,
> >   while still abstracting tasklet handling into dmaengine helpers so drivers no
> >   longer directly manipulate tasklets. That keeps
> > dmaengine_desc_callback_invoke()
> >   in tasklet context and avoids breaking DMA users that rely on that behavior.
>
> It's probably fine to do both, but a series of two patches (first introduce
> the per-channel API, then move it over to WQ_BH) may be slightly
> clearer here.
>
> I'm not sure why the dmaengine_*_bh_wq() functions are exported
> interfaces, as far as I can tell, you use them only internally
> in the dma_chan_*_bh() functions, so making them static would
> let the compiler inline them where possible.
>
> There are of course dmaengine drivers that use tasklets for other
> purposes than the channel callback. Was your idea here to use
> the same workqueue for these? I would perhaps hold off on that for
> the moment and see if there is a better alternative for those,
> possibly hardirq context, threaded irq or a regular workqueue
> depending on the driver.
>
> >   A hardirq callback path feels like a larger API change, so I’d
> > prefer to handle that as a separate follow‑up (e.g. explicit hardirq
> >  callback/flag + user migration where safe). Thoughts?
>
> Yes, definitely keep that separate. I still think that this is
> what we want eventually for a bigger improvement, but your
> patch seems valuable on its own as well.
>

Thanks for the detailed feedback. I’ll respin along these lines:

  - Split the series into two patches: (1) introduce the per‑channel BH API,
    (2) switch the vchan implementation over to the WQ_BH backend if we decide
    to keep that step. This should make the progression clearer.

  - The dmaengine_*_bh_wq() helpers will be made static; only dma_chan_*_bh()
    stays exported.

  - I won’t try to move the other per‑driver tasklets onto the shared queue in
    this series. That feels like a separate discussion, and the right context
    (hardirq/threaded/workqueue) may vary by driver.

  - I’ll keep the hardirq callback path separate. For that follow‑up, I can plan
    to add an explicit “hardirq safe” request bit (e.g. a new dma_ctrl_flags)
    and update vchan_cookie_complete() to invoke the callback directly when
    requested; otherwise it stays on the tasklet path.

  If you’d prefer I drop the WQ_BH conversion entirely for now and keep only the
  tasklet‑based per‑channel API, I can do that too.

Thanks,
Allen


> Some more thoughts on where that later change could take us:
>
> >    /*
> >   - * This tasklet handles the completion of a DMA descriptor by
> >   - * calling its callback and freeing it.
> >   + * This bottom-half handler completes a DMA descriptor by invoking its
> >   + * callback and freeing it.
> >     */
> >   -static void vchan_complete(struct tasklet_struct *t)
> >   +static void vchan_complete(struct dma_chan *chan)
> >    {
> >   -    struct virt_dma_chan *vc = from_tasklet(vc, t, task);
> >   +    struct virt_dma_chan *vc = to_virt_chan(chan);
> >        struct virt_dma_desc *vd, *_vd;
> >        struct dmaengine_desc_callback cb;
> >        LIST_HEAD(head);
> >   @@ -131,7 +131,7 @@ void vchan_init(struct virt_dma_chan *vc, struct
> > dma_device *dmadev)
> >        INIT_LIST_HEAD(&vc->desc_completed);
> >        INIT_LIST_HEAD(&vc->desc_terminated);
> >
> >   -    tasklet_setup(&vc->task, vchan_complete);
> >   +    dma_chan_init_bh(&vc->chan, vchan_complete);
>
> This is where I think it makes sense to start, again for
> the vchan imlmenentation. What the dmaengine drivers have
> is a per-driver tasklet (or WQ_BH) with a single complete()
> function that directly calls into the client drivers for
> each completion that has happened.
>
> Since the context we want depends on the type of client
> driver, I think a good approach would be to start
> by modifying vchan_cookie_complete() to allow it to
> call the callback function directly from hardirq context
> when the client asks for that, and avoid the round trip
> through the tasklet where possible.
>
> A new bit in the dma_ctrl_flags word could be used
> to ask for hardirq vs softirq context, and the existing
> drivers just fall back to using a tasklet for softirq
> context.
>
>       Arnd



-- 
       - Allen

Re: [RFC PATCH 1/1] dmaengine: introduce dmaengine_bh_wq and bh helpers

[Arnd Bergmann]

On Mon, Jan 12, 2026, at 23:20, Allen wrote:
>> >   A hardirq callback path feels like a larger API change, so I’d
>> > prefer to handle that as a separate follow‑up (e.g. explicit hardirq
>> >  callback/flag + user migration where safe). Thoughts?
>>
>> Yes, definitely keep that separate. I still think that this is
>> what we want eventually for a bigger improvement, but your
>> patch seems valuable on its own as well.
>>
>
> Thanks for the detailed feedback. I’ll respin along these lines:
>
>   - Split the series into two patches: (1) introduce the per‑channel BH API,
>     (2) switch the vchan implementation over to the WQ_BH backend if we decide
>     to keep that step. This should make the progression clearer.
>
>   - The dmaengine_*_bh_wq() helpers will be made static; only dma_chan_*_bh()
>     stays exported.

Sounds good, yes.

>   - I won’t try to move the other per‑driver tasklets onto the shared queue in
>     this series. That feels like a separate discussion, and the right context
>     (hardirq/threaded/workqueue) may vary by driver.

I'm not sure we are talking about the same thing here. I do think we should
try to have all the callbacks in each dmaengine driver go through the same
per-channel deferral mechanism. What I meant to say is that all the tasklets
that are not used for the client callbacks should be separate from those,
e.g. the pl330_dotask() handler is used for unexpected events unrelated
to a channel.

>   - I’ll keep the hardirq callback path separate. For that follow‑up, I can plan
>     to add an explicit “hardirq safe” request bit (e.g. a new dma_ctrl_flags)
>     and update vchan_cookie_complete() to invoke the callback directly when
>     requested; otherwise it stays on the tasklet path.
>
>   If you’d prefer I drop the WQ_BH conversion entirely for now and keep only the
>   tasklet‑based per‑channel API, I can do that too.

No, I think that part is fine.

     Arnd

Re: [RFC PATCH 1/1] dmaengine: introduce dmaengine_bh_wq and bh helpers

[Allen]

> >> >   A hardirq callback path feels like a larger API change, so I’d
> >> > prefer to handle that as a separate follow‑up (e.g. explicit hardirq
> >> >  callback/flag + user migration where safe). Thoughts?
> >>
> >> Yes, definitely keep that separate. I still think that this is
> >> what we want eventually for a bigger improvement, but your
> >> patch seems valuable on its own as well.
> >>
> >
> > Thanks for the detailed feedback. I’ll respin along these lines:
> >
> >   - Split the series into two patches: (1) introduce the per‑channel BH API,
> >     (2) switch the vchan implementation over to the WQ_BH backend if we decide
> >     to keep that step. This should make the progression clearer.
> >
> >   - The dmaengine_*_bh_wq() helpers will be made static; only dma_chan_*_bh()
> >     stays exported.
>
> Sounds good, yes.
>
> >   - I won’t try to move the other per‑driver tasklets onto the shared queue in
> >     this series. That feels like a separate discussion, and the right context
> >     (hardirq/threaded/workqueue) may vary by driver.
>
> I'm not sure we are talking about the same thing here. I do think we should
> try to have all the callbacks in each dmaengine driver go through the same
> per-channel deferral mechanism. What I meant to say is that all the tasklets
> that are not used for the client callbacks should be separate from those,
> e.g. the pl330_dotask() handler is used for unexpected events unrelated
> to a channel.

Hi Arnd,

Thanks for clarifying, I understand now. Yes, the intent will be to have
all per‑channel client callbacks go through the shared per‑channel deferral
mechanism (dma_chan_*_bh), while keeping any non‑callback tasklets/work
separate (e.g. pl330_dotask() for unexpected events).

I’ll keep WQ_BH and the per‑channel API as the common callback path, and in
follow‑on patches I’ll start converting driver callback paths to use it.

Thanks,
Allen


>
> >   - I’ll keep the hardirq callback path separate. For that follow‑up, I can plan
> >     to add an explicit “hardirq safe” request bit (e.g. a new dma_ctrl_flags)
> >     and update vchan_cookie_complete() to invoke the callback directly when
> >     requested; otherwise it stays on the tasklet path.
> >
> >   If you’d prefer I drop the WQ_BH conversion entirely for now and keep only the
> >   tasklet‑based per‑channel API, I can do that too.
>
> No, I think that part is fine.
>
>      Arnd



-- 
       - Allen

[PATCH v2 00/64] dmaengine: migrate channel tasklets to WQ_BH

[Allen Pais]

Hi Arnd, Vinod and Frank,

This series moves DMAengine client completion bottom halves from private
tasklets to a common per-channel API backed by WQ_BH.

The first patch introduces the dma_chan_*_bh() API with a tasklet backend
and converts virt-dma to it. This separates the driver-facing API from its
implementation without changing callback context.

The second patch switches that backend to a dedicated WQ_BH | WQ_PERCPU
workqueue. WQ_BH keeps callbacks in softirq context, while the common API
centralizes initialization, scheduling, and synchronization. The
workqueue helpers remain internal to DMAengine, and dma_chan_kill_bh()
drains scheduled callbacks to preserve tasklet_kill() semantics.

The remaining patches convert driver-owned channel completion tasklets
and update virt-dma teardown paths. Tasklets used for controller-level
processing, recovery, or other non-client-callback work are deliberately
left alone because the appropriate replacement may differ by driver.

As agreed in the RFC discussion, selecting hardirq callback delivery is a
separate API change and is not part of this series.

Earlier discussion:
https://lore.kernel.org/all/20260108080332.2341725-1-allen.lkml@gmail.com/

Changes since RFC v1:
- split the tasklet-backed channel API from the WQ_BH backend change
- keep the low-level workqueue helpers private to DMAengine
- preserve tasklet teardown semantics by draining scheduled BH work
- convert the remaining driver client-completion tasklets
- retain unrelated controller and recovery tasklets
- route SF-PDMA terminal errors through virt-dma completion handling
- add missing callback synchronization and teardown handling
- fix the build failures reported by the kernel test robot
- rebase onto the current Linus tree

Testing:
- x86_64 allmodconfig
- make -j$(nproc) W=1 drivers/dma/
- scripts/checkpatch.pl --strict --git origin/master..HEAD
- git diff --check origin/master..HEAD

Allen Pais (64):
  dmaengine: add tasklet-backed channel BH helpers
  dmaengine: back channel BH helpers with WQ_BH
  dmaengine: apple-admac: use dma_chan BH callback
  dmaengine: at_xdmac: move irq bottom half to dma_chan BH
  dmaengine: ep93xx: hook callbacks via dma_chan BH
  dmaengine: fsldma: migrate tasklet to dma_chan BH
  dmaengine: fsl_raid: run completions via dma_chan BH
  dmaengine: imx-dma: flip per-chan tasklet to dma_chan BH
  dmaengine: ioat: convert cleanup tasklet to dma_chan BH
  dmaengine: mmp_pdma: replace per-chan tasklet with dma_chan BH
  dmaengine: mmp_tdma: hook completions to dma_chan BH
  dmaengine: mv_xor: convert irq tasklet to dma_chan BH
  dmaengine: mxs-dma: use dma_chan BH scheduling
  dmaengine: nbpfaxi: switch callbacks to dma_chan BH
  dmaengine: pch_dma: convert tasklet to dma_chan BH
  dmaengine: ppc4xx: replace irq tasklet with dma_chan BH
  dmaengine: ste_dma40: convert per-channel tasklet to dma_chan BH
  dmaengine: xgene-dma: wire descriptor cleanup to dma_chan BH
  dmaengine: xilinx-dma: use dma_chan BH instead of tasklets
  dmaengine: xilinx-dpdma: kill vchan BH on remove
  dmaengine: zynqmp-dma: switch completion tasklet to dma_chan BH
  dmaengine: tegra20-apb: use channel BH helpers
  dmaengine: timb_dma: route callbacks via channel BH
  dmaengine: txx9dmac: route callbacks via channel BH
  dmaengine: mv_xor_v2: use channel BH helpers
  dmaengine: mpc512x: route callbacks via channel BH
  dmaengine: k3dma: kill vchan BH on remove
  dmaengine: plx_dma: use channel BH helpers
  dmaengine: sf-pdma: route error callbacks through channel BH
  dmaengine: sa11x0-dma: kill vchan BH on remove
  dmaengine: pl330: route callbacks via channel BH
  dmaengine: k3-udma: use channel BH for vchan completions
  dmaengine: sun6i: kill vchan BH on teardown
  dmaengine: mtk-cqdma: kill vchan BH on teardown
  dmaengine: altera-msgdma: use channel BH helpers
  dmaengine: sprd-dma: kill vchan BH on teardown
  dmaengine: idma64: kill vchan BH on teardown
  dmaengine: img-mdc-dma: kill vchan BH on teardown
  dmaengine: fsl-edma-common: kill vchan BH on teardown
  dmaengine: dw-axi-dmac: kill vchan BH on teardown
  dmaengine: hsu: kill vchan BH on teardown
  dmaengine: jz4780: kill vchan BH on teardown
  dmaengine: pxa_dma: kill vchan BH on teardown
  dmaengine: mtk-hsdma: kill vchan BH on teardown
  dmaengine: mtk-uart-apdma: kill vchan BH on teardown
  dmaengine: imx-sdma: kill vchan BH on teardown
  dmaengine: loongson1-apb: kill vchan BH on teardown
  dmaengine: owl-dma: kill vchan BH on teardown
  dmaengine: hisi: kill vchan BH on teardown
  dmaengine: dw-edma: kill vchan BH on teardown
  dmaengine: bcm2835: kill vchan BH on teardown
  dmaengine: tegra210-adma: kill vchan BH on teardown
  dmaengine: fsl-qdma: use dma_chan_kill_bh
  dmaengine: st_fdma: use dma_chan_kill_bh
  dmaengine: dma-axi-dmac: use dma_chan_kill_bh
  dmaengine: omap-dma: use dma_chan_kill_bh
  dmaengine: edma: use dma_chan_kill_bh
  dmaengine: qcom-adm: use dma_chan_kill_bh
  dmaengine: tegra186-gpc: use dma_chan_kill_bh
  dmaengine: bam-dma: use dma_chan_kill_bh
  dmaengine: dw: defer callbacks via channel BH
  dmaengine: hidma: defer callbacks via channel BH
  dmaengine: qcom-gpi: defer callbacks via vchan
  dmaengine: switchtec: use channel BH helpers

 drivers/dma/altera-msgdma.c                   | 14 +--
 drivers/dma/apple-admac.c                     | 16 ++--
 drivers/dma/at_xdmac.c                        | 13 ++-
 drivers/dma/bcm2835-dma.c                     |  2 +-
 drivers/dma/dma-axi-dmac.c                    | 10 ++-
 drivers/dma/dma-jz4780.c                      |  2 +-
 drivers/dma/dmaengine.c                       | 89 ++++++++++++++++++-
 .../dma/dw-axi-dmac/dw-axi-dmac-platform.c    |  2 +-
 drivers/dma/dw-edma/dw-edma-core.c            |  2 +-
 drivers/dma/dw/core.c                         | 42 +++++++--
 drivers/dma/dw/regs.h                         |  1 +
 drivers/dma/ep93xx_dma.c                      | 12 ++-
 drivers/dma/fsl-edma-common.c                 |  2 +-
 drivers/dma/fsl-qdma.c                        |  2 +-
 drivers/dma/fsl_raid.c                        | 11 +--
 drivers/dma/fsl_raid.h                        |  1 -
 drivers/dma/fsldma.c                          | 10 +--
 drivers/dma/fsldma.h                          |  1 -
 drivers/dma/hisi_dma.c                        |  2 +-
 drivers/dma/hsu/hsu.c                         |  2 +-
 drivers/dma/idma64.c                          |  4 +-
 drivers/dma/img-mdc-dma.c                     |  2 +-
 drivers/dma/imx-dma.c                         | 26 +++---
 drivers/dma/imx-sdma.c                        |  4 +-
 drivers/dma/ioat/dma.c                        | 14 +--
 drivers/dma/ioat/dma.h                        |  3 +-
 drivers/dma/ioat/init.c                       |  2 +-
 drivers/dma/k3dma.c                           |  2 +-
 drivers/dma/loongson/loongson1-apb-dma.c      |  2 +-
 drivers/dma/mediatek/mtk-cqdma.c              |  2 +-
 drivers/dma/mediatek/mtk-hsdma.c              |  2 +-
 drivers/dma/mediatek/mtk-uart-apdma.c         |  4 +-
 drivers/dma/mmp_pdma.c                        | 13 ++-
 drivers/dma/mmp_tdma.c                        |  9 +-
 drivers/dma/mpc512x_dma.c                     | 74 +++++++++------
 drivers/dma/mv_xor.c                          | 12 +--
 drivers/dma/mv_xor.h                          |  2 -
 drivers/dma/mv_xor_v2.c                       | 20 ++---
 drivers/dma/mxs-dma.c                         | 11 ++-
 drivers/dma/nbpfaxi.c                         | 13 ++-
 drivers/dma/owl-dma.c                         |  2 +-
 drivers/dma/pch_dma.c                         | 14 +--
 drivers/dma/pl330.c                           | 18 ++--
 drivers/dma/plx_dma.c                         | 12 +--
 drivers/dma/ppc4xx/adma.c                     | 14 +--
 drivers/dma/ppc4xx/adma.h                     |  2 -
 drivers/dma/pxa_dma.c                         |  2 +-
 drivers/dma/qcom/bam_dma.c                    |  4 +-
 drivers/dma/qcom/gpi.c                        | 17 +++-
 drivers/dma/qcom/hidma.c                      | 18 +++-
 drivers/dma/qcom/qcom_adm.c                   |  4 +-
 drivers/dma/sa11x0-dma.c                      |  2 +-
 drivers/dma/sf-pdma/sf-pdma.c                 | 14 ++-
 drivers/dma/sprd-dma.c                        |  2 +-
 drivers/dma/st_fdma.c                         |  2 +-
 drivers/dma/ste_dma40.c                       | 13 ++-
 drivers/dma/sun6i-dma.c                       |  2 +-
 drivers/dma/switchtec_dma.c                   | 16 ++--
 drivers/dma/tegra186-gpc-dma.c                |  2 +-
 drivers/dma/tegra20-apb-dma.c                 | 18 ++--
 drivers/dma/tegra210-adma.c                   |  2 +-
 drivers/dma/ti/edma.c                         |  2 +-
 drivers/dma/ti/k3-udma.c                      | 12 +--
 drivers/dma/ti/omap-dma.c                     |  2 +-
 drivers/dma/timb_dma.c                        | 21 +++--
 drivers/dma/txx9dmac.c                        | 12 +--
 drivers/dma/txx9dmac.h                        |  1 -
 drivers/dma/virt-dma.c                        | 12 +--
 drivers/dma/virt-dma.h                        |  7 +-
 drivers/dma/xgene-dma.c                       | 19 ++--
 drivers/dma/xilinx/xilinx_dma.c               | 20 ++---
 drivers/dma/xilinx/xilinx_dpdma.c             |  1 +
 drivers/dma/xilinx/zynqmp_dma.c               | 19 ++--
 include/linux/dmaengine.h                     | 28 ++++++
 74 files changed, 482 insertions(+), 311 deletions(-)


base-commit: f5098b6bae761e346ebcd9da7f95622c04733cff
-- 
2.43.0

[PATCH v2 01/64] dmaengine: add tasklet-backed channel BH helpers

[Allen Pais]

DMAengine drivers commonly use a per-channel tasklet to invoke client
callbacks. Add helpers that initialize, schedule, and kill a channel
bottom half, with an initial tasklet-backed implementation that preserves
the existing execution context.

Convert virt-dma to the new API and remove its private tasklet. While
touching the completion handler, avoid forming a result pointer from a
NULL cyclic descriptor.

This establishes a backend-independent API before changing how channel
bottom halves are dispatched.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/dmaengine.c   | 42 +++++++++++++++++++++++++++++++++++++++
 drivers/dma/virt-dma.c    | 12 +++++------
 drivers/dma/virt-dma.h    |  7 +++----
 include/linux/dmaengine.h | 28 ++++++++++++++++++++++++++
 4 files changed, 79 insertions(+), 10 deletions(-)

diff --git a/drivers/dma/dmaengine.c b/drivers/dma/dmaengine.c
index 9049171df857..f43aeecb47f9 100644
--- a/drivers/dma/dmaengine.c
+++ b/drivers/dma/dmaengine.c
@@ -1428,6 +1428,48 @@ static void dmaengine_destroy_unmap_pool(void)
 	}
 }
 
+static void dma_chan_bh_entry(struct tasklet_struct *tasklet)
+{
+	struct dma_chan *chan = from_tasklet(chan, tasklet, bh_tasklet);
+	dma_chan_bh_work_fn fn = READ_ONCE(chan->bh_work_fn);
+
+	if (fn)
+		fn(chan);
+}
+
+void dma_chan_init_bh(struct dma_chan *chan, dma_chan_bh_work_fn fn)
+{
+	if (WARN_ON(!fn))
+		return;
+
+	if (WARN_ON(chan->bh_work_initialized))
+		return;
+
+	chan->bh_work_fn = fn;
+	tasklet_setup(&chan->bh_tasklet, dma_chan_bh_entry);
+	chan->bh_work_initialized = true;
+}
+EXPORT_SYMBOL_GPL(dma_chan_init_bh);
+
+bool dma_chan_schedule_bh(struct dma_chan *chan)
+{
+	if (WARN_ON(!chan->bh_work_initialized))
+		return false;
+
+	tasklet_schedule(&chan->bh_tasklet);
+	return true;
+}
+EXPORT_SYMBOL_GPL(dma_chan_schedule_bh);
+
+void dma_chan_kill_bh(struct dma_chan *chan)
+{
+	if (!chan->bh_work_initialized)
+		return;
+
+	tasklet_kill(&chan->bh_tasklet);
+}
+EXPORT_SYMBOL_GPL(dma_chan_kill_bh);
+
 static int __init dmaengine_init_unmap_pool(void)
 {
 	int i;
diff --git a/drivers/dma/virt-dma.c b/drivers/dma/virt-dma.c
index 7961172a780d..89fcd51ddcab 100644
--- a/drivers/dma/virt-dma.c
+++ b/drivers/dma/virt-dma.c
@@ -77,12 +77,12 @@ struct virt_dma_desc *vchan_find_desc(struct virt_dma_chan *vc,
 EXPORT_SYMBOL_GPL(vchan_find_desc);
 
 /*
- * This tasklet handles the completion of a DMA descriptor by
- * calling its callback and freeing it.
+ * This bottom-half handler completes a DMA descriptor by invoking its
+ * callback and freeing it.
  */
-static void vchan_complete(struct tasklet_struct *t)
+static void vchan_complete(struct dma_chan *chan)
 {
-	struct virt_dma_chan *vc = from_tasklet(vc, t, task);
+	struct virt_dma_chan *vc = to_virt_chan(chan);
 	struct virt_dma_desc *vd, *_vd;
 	struct dmaengine_desc_callback cb;
 	LIST_HEAD(head);
@@ -98,7 +98,7 @@ static void vchan_complete(struct tasklet_struct *t)
 	}
 	spin_unlock_irq(&vc->lock);
 
-	dmaengine_desc_callback_invoke(&cb, &vd->tx_result);
+	dmaengine_desc_callback_invoke(&cb, vd ? &vd->tx_result : NULL);
 
 	list_for_each_entry_safe(vd, _vd, &head, node) {
 		dmaengine_desc_get_callback(&vd->tx, &cb);
@@ -131,7 +131,7 @@ void vchan_init(struct virt_dma_chan *vc, struct dma_device *dmadev)
 	INIT_LIST_HEAD(&vc->desc_completed);
 	INIT_LIST_HEAD(&vc->desc_terminated);
 
-	tasklet_setup(&vc->task, vchan_complete);
+	dma_chan_init_bh(&vc->chan, vchan_complete);
 
 	vc->chan.device = dmadev;
 	list_add_tail(&vc->chan.device_node, &dmadev->channels);
diff --git a/drivers/dma/virt-dma.h b/drivers/dma/virt-dma.h
index 59d9eabc8b67..abe2a4475744 100644
--- a/drivers/dma/virt-dma.h
+++ b/drivers/dma/virt-dma.h
@@ -21,7 +21,6 @@ struct virt_dma_desc {
 
 struct virt_dma_chan {
 	struct dma_chan	chan;
-	struct tasklet_struct task;
 	void (*desc_free)(struct virt_dma_desc *);
 
 	spinlock_t lock;
@@ -106,7 +105,7 @@ static inline void vchan_cookie_complete(struct virt_dma_desc *vd)
 		 vd, cookie);
 	list_add_tail(&vd->node, &vc->desc_completed);
 
-	tasklet_schedule(&vc->task);
+	dma_chan_schedule_bh(&vc->chan);
 }
 
 /**
@@ -137,7 +136,7 @@ static inline void vchan_cyclic_callback(struct virt_dma_desc *vd)
 	struct virt_dma_chan *vc = to_virt_chan(vd->tx.chan);
 
 	vc->cyclic = vd;
-	tasklet_schedule(&vc->task);
+	dma_chan_schedule_bh(&vc->chan);
 }
 
 /**
@@ -223,7 +222,7 @@ static inline void vchan_synchronize(struct virt_dma_chan *vc)
 	LIST_HEAD(head);
 	unsigned long flags;
 
-	tasklet_kill(&vc->task);
+	dma_chan_kill_bh(&vc->chan);
 
 	spin_lock_irqsave(&vc->lock, flags);
 
diff --git a/include/linux/dmaengine.h b/include/linux/dmaengine.h
index b3d251c9734e..c670781b3150 100644
--- a/include/linux/dmaengine.h
+++ b/include/linux/dmaengine.h
@@ -12,6 +12,7 @@
 #include <linux/scatterlist.h>
 #include <linux/bitmap.h>
 #include <linux/types.h>
+#include <linux/interrupt.h>
 #include <asm/page.h>
 
 /**
@@ -295,6 +296,10 @@ enum dma_desc_metadata_mode {
 	DESC_METADATA_ENGINE = BIT(1),
 };
 
+struct dma_chan;
+
+typedef void (*dma_chan_bh_work_fn)(struct dma_chan *chan);
+
 /**
  * struct dma_chan_percpu - the per-CPU part of struct dma_chan
  * @memcpy_count: transaction counter
@@ -334,6 +339,9 @@ struct dma_router {
  * @router: pointer to the DMA router structure
  * @route_data: channel specific data for the router
  * @private: private data for certain client-channel associations
+ * @bh_tasklet: bottom-half tasklet stored per-channel
+ * @bh_work_fn: callback executed when @bh_tasklet runs
+ * @bh_work_initialized: indicates whether @bh_tasklet has been initialized
  */
 struct dma_chan {
 	struct dma_device *device;
@@ -359,6 +367,9 @@ struct dma_chan {
 	void *route_data;
 
 	void *private;
+	struct tasklet_struct bh_tasklet;
+	dma_chan_bh_work_fn bh_work_fn;
+	bool bh_work_initialized;
 };
 
 /**
@@ -1529,6 +1540,9 @@ struct dma_chan *devm_dma_request_chan(struct device *dev, const char *name);
 
 void dma_release_channel(struct dma_chan *chan);
 int dma_get_slave_caps(struct dma_chan *chan, struct dma_slave_caps *caps);
+void dma_chan_init_bh(struct dma_chan *chan, dma_chan_bh_work_fn fn);
+bool dma_chan_schedule_bh(struct dma_chan *chan);
+void dma_chan_kill_bh(struct dma_chan *chan);
 #else
 static inline struct dma_chan *dma_find_channel(enum dma_transaction_type tx_type)
 {
@@ -1576,6 +1590,20 @@ static inline int dma_get_slave_caps(struct dma_chan *chan,
 {
 	return -ENXIO;
 }
+
+static inline void dma_chan_init_bh(struct dma_chan *chan,
+				    dma_chan_bh_work_fn fn)
+{
+}
+
+static inline bool dma_chan_schedule_bh(struct dma_chan *chan)
+{
+	return false;
+}
+
+static inline void dma_chan_kill_bh(struct dma_chan *chan)
+{
+}
 #endif
 
 static inline int dmaengine_desc_set_reuse(struct dma_async_tx_descriptor *tx)
-- 
2.43.0

[PATCH v2 02/64] dmaengine: back channel BH helpers with WQ_BH

[Allen Pais]

Replace the tasklet implementation of the channel BH helpers with a
dedicated WQ_BH | WQ_PERCPU workqueue. The public dma_chan_*_bh() API and
its softirq execution context remain unchanged.

Keep the workqueue operations internal to dmaengine. Drain scheduled work
in dma_chan_kill_bh() to preserve the completion semantics of
tasklet_kill().

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/dmaengine.c   | 59 ++++++++++++++++++++++++++++++++-------
 include/linux/dmaengine.h | 10 +++----
 2 files changed, 54 insertions(+), 15 deletions(-)

diff --git a/drivers/dma/dmaengine.c b/drivers/dma/dmaengine.c
index f43aeecb47f9..8b1b17794956 100644
--- a/drivers/dma/dmaengine.c
+++ b/drivers/dma/dmaengine.c
@@ -54,6 +54,7 @@
 #include <linux/rcupdate.h>
 #include <linux/slab.h>
 #include <linux/spinlock.h>
+#include <linux/workqueue.h>
 
 #include "dmaengine.h"
 
@@ -61,6 +62,7 @@ static DEFINE_MUTEX(dma_list_mutex);
 static DEFINE_IDA(dma_ida);
 static LIST_HEAD(dma_device_list);
 static long dmaengine_ref_count;
+static struct workqueue_struct *dmaengine_bh_wq;
 
 /* --- debugfs implementation --- */
 #ifdef CONFIG_DEBUG_FS
@@ -1428,9 +1430,34 @@ static void dmaengine_destroy_unmap_pool(void)
 	}
 }
 
-static void dma_chan_bh_entry(struct tasklet_struct *tasklet)
+static void dmaengine_destroy_bh_wq(void)
 {
-	struct dma_chan *chan = from_tasklet(chan, tasklet, bh_tasklet);
+	if (!dmaengine_bh_wq)
+		return;
+
+	destroy_workqueue(dmaengine_bh_wq);
+	dmaengine_bh_wq = NULL;
+}
+
+static bool dmaengine_queue_bh_work(struct work_struct *work)
+{
+	if (WARN_ON(!dmaengine_bh_wq))
+		return false;
+
+	return queue_work(dmaengine_bh_wq, work);
+}
+
+static void dmaengine_flush_bh_work(struct work_struct *work)
+{
+	if (!work)
+		return;
+
+	flush_work(work);
+}
+
+static void dma_chan_bh_entry(struct work_struct *work)
+{
+	struct dma_chan *chan = container_of(work, struct dma_chan, bh_work);
 	dma_chan_bh_work_fn fn = READ_ONCE(chan->bh_work_fn);
 
 	if (fn)
@@ -1446,7 +1473,7 @@ void dma_chan_init_bh(struct dma_chan *chan, dma_chan_bh_work_fn fn)
 		return;
 
 	chan->bh_work_fn = fn;
-	tasklet_setup(&chan->bh_tasklet, dma_chan_bh_entry);
+	INIT_WORK(&chan->bh_work, dma_chan_bh_entry);
 	chan->bh_work_initialized = true;
 }
 EXPORT_SYMBOL_GPL(dma_chan_init_bh);
@@ -1456,8 +1483,7 @@ bool dma_chan_schedule_bh(struct dma_chan *chan)
 	if (WARN_ON(!chan->bh_work_initialized))
 		return false;
 
-	tasklet_schedule(&chan->bh_tasklet);
-	return true;
+	return dmaengine_queue_bh_work(&chan->bh_work);
 }
 EXPORT_SYMBOL_GPL(dma_chan_schedule_bh);
 
@@ -1466,7 +1492,7 @@ void dma_chan_kill_bh(struct dma_chan *chan)
 	if (!chan->bh_work_initialized)
 		return;
 
-	tasklet_kill(&chan->bh_tasklet);
+	dmaengine_flush_bh_work(&chan->bh_work);
 }
 EXPORT_SYMBOL_GPL(dma_chan_kill_bh);
 
@@ -1666,15 +1692,28 @@ EXPORT_SYMBOL_GPL(dma_run_dependencies);
 
 static int __init dma_bus_init(void)
 {
-	int err = dmaengine_init_unmap_pool();
+	int err;
 
+	dmaengine_bh_wq = alloc_workqueue("dmaengine_bh",
+					  WQ_BH | WQ_PERCPU, 0);
+	if (!dmaengine_bh_wq)
+		return -ENOMEM;
+
+	err = dmaengine_init_unmap_pool();
 	if (err)
-		return err;
+		goto err_destroy_wq;
 
 	err = class_register(&dma_devclass);
-	if (!err)
-		dmaengine_debugfs_init();
+	if (err)
+		goto err_destroy_pool;
 
+	dmaengine_debugfs_init();
+	return 0;
+
+err_destroy_pool:
+	dmaengine_destroy_unmap_pool();
+err_destroy_wq:
+	dmaengine_destroy_bh_wq();
 	return err;
 }
 arch_initcall(dma_bus_init);
diff --git a/include/linux/dmaengine.h b/include/linux/dmaengine.h
index c670781b3150..c87974d5048b 100644
--- a/include/linux/dmaengine.h
+++ b/include/linux/dmaengine.h
@@ -12,7 +12,7 @@
 #include <linux/scatterlist.h>
 #include <linux/bitmap.h>
 #include <linux/types.h>
-#include <linux/interrupt.h>
+#include <linux/workqueue.h>
 #include <asm/page.h>
 
 /**
@@ -339,9 +339,9 @@ struct dma_router {
  * @router: pointer to the DMA router structure
  * @route_data: channel specific data for the router
  * @private: private data for certain client-channel associations
- * @bh_tasklet: bottom-half tasklet stored per-channel
- * @bh_work_fn: callback executed when @bh_tasklet runs
- * @bh_work_initialized: indicates whether @bh_tasklet has been initialized
+ * @bh_work: bottom-half work item stored per-channel
+ * @bh_work_fn: callback executed when @bh_work runs
+ * @bh_work_initialized: indicates whether @bh_work has been initialized
  */
 struct dma_chan {
 	struct dma_device *device;
@@ -367,7 +367,7 @@ struct dma_chan {
 	void *route_data;
 
 	void *private;
-	struct tasklet_struct bh_tasklet;
+	struct work_struct bh_work;
 	dma_chan_bh_work_fn bh_work_fn;
 	bool bh_work_initialized;
 };
-- 
2.43.0

[PATCH v2 03/64] dmaengine: apple-admac: use dma_chan BH callback

[Allen Pais]

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/apple-admac.c | 16 ++++++----------
 1 file changed, 6 insertions(+), 10 deletions(-)

diff --git a/drivers/dma/apple-admac.c b/drivers/dma/apple-admac.c
index 14a5ee14a481..84483247bde6 100644
--- a/drivers/dma/apple-admac.c
+++ b/drivers/dma/apple-admac.c
@@ -89,7 +89,6 @@ struct admac_chan {
 	unsigned int no;
 	struct admac_data *host;
 	struct dma_chan chan;
-	struct tasklet_struct tasklet;
 
 	u32 carveout;
 
@@ -521,10 +520,7 @@ static int admac_terminate_all(struct dma_chan *chan)
 		list_add_tail(&adchan->current_tx->node, &adchan->to_free);
 		adchan->current_tx = NULL;
 	}
-	/*
-	 * Descriptors can only be freed after the tasklet
-	 * has been killed (in admac_synchronize).
-	 */
+	/* Descriptors are released once the BH is flushed in admac_synchronize */
 	list_splice_tail_init(&adchan->submitted, &adchan->to_free);
 	list_splice_tail_init(&adchan->issued, &adchan->to_free);
 	spin_unlock_irqrestore(&adchan->lock, flags);
@@ -543,7 +539,7 @@ static void admac_synchronize(struct dma_chan *chan)
 	list_splice_tail_init(&adchan->to_free, &head);
 	spin_unlock_irqrestore(&adchan->lock, flags);
 
-	tasklet_kill(&adchan->tasklet);
+	dma_chan_kill_bh(&adchan->chan);
 
 	list_for_each_entry_safe(adtx, _adtx, &head, node) {
 		list_del(&adtx->node);
@@ -662,7 +658,7 @@ static void admac_handle_status_desc_done(struct admac_data *ad, int channo)
 		tx->reclaimed_pos %= 2 * tx->buf_len;
 
 		admac_cyclic_write_desc(ad, channo, tx);
-		tasklet_schedule(&adchan->tasklet);
+		dma_chan_schedule_bh(&adchan->chan);
 	}
 	spin_unlock_irqrestore(&adchan->lock, flags);
 }
@@ -712,9 +708,9 @@ static irqreturn_t admac_interrupt(int irq, void *devid)
 	return IRQ_HANDLED;
 }
 
-static void admac_chan_tasklet(struct tasklet_struct *t)
+static void admac_chan_bh(struct dma_chan *chan)
 {
-	struct admac_chan *adchan = from_tasklet(adchan, t, tasklet);
+	struct admac_chan *adchan = to_admac_chan(chan);
 	struct admac_tx *adtx;
 	struct dmaengine_desc_callback cb;
 	struct dmaengine_result tx_result;
@@ -886,7 +882,7 @@ static int admac_probe(struct platform_device *pdev)
 		INIT_LIST_HEAD(&adchan->issued);
 		INIT_LIST_HEAD(&adchan->to_free);
 		list_add_tail(&adchan->chan.device_node, &dma->channels);
-		tasklet_setup(&adchan->tasklet, admac_chan_tasklet);
+		dma_chan_init_bh(&adchan->chan, admac_chan_bh);
 	}
 
 	err = reset_control_reset(ad->rstc);
-- 
2.43.0

[PATCH v2 04/64] dmaengine: at_xdmac: move irq bottom half to dma_chan BH

[Allen Pais]

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/at_xdmac.c | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/drivers/dma/at_xdmac.c b/drivers/dma/at_xdmac.c
index 901971e8bae6..aa918043e37e 100644
--- a/drivers/dma/at_xdmac.c
+++ b/drivers/dma/at_xdmac.c
@@ -228,7 +228,6 @@ struct at_xdmac_chan {
 	u32				save_cndc;
 	u32				irq_status;
 	unsigned long			status;
-	struct tasklet_struct		tasklet;
 	struct dma_slave_config		sconfig;
 
 	spinlock_t			lock;
@@ -1759,9 +1758,9 @@ static void at_xdmac_handle_error(struct at_xdmac_chan *atchan)
 	/* Then continue with usual descriptor management */
 }
 
-static void at_xdmac_tasklet(struct tasklet_struct *t)
+static void at_xdmac_tasklet(struct dma_chan *chan)
 {
-	struct at_xdmac_chan	*atchan = from_tasklet(atchan, t, tasklet);
+	struct at_xdmac_chan	*atchan = to_at_xdmac_chan(chan);
 	struct at_xdmac		*atxdmac = to_at_xdmac(atchan->chan.device);
 	struct at_xdmac_desc	*desc;
 	struct dma_async_tx_descriptor *txd;
@@ -1865,7 +1864,7 @@ static irqreturn_t at_xdmac_interrupt(int irq, void *dev_id)
 			if (atchan->irq_status & (AT_XDMAC_CIS_RBEIS | AT_XDMAC_CIS_WBEIS))
 				at_xdmac_write(atxdmac, AT_XDMAC_GD, atchan->mask);
 
-			tasklet_schedule(&atchan->tasklet);
+			dma_chan_schedule_bh(&atchan->chan);
 			ret = IRQ_HANDLED;
 		}
 
@@ -2317,7 +2316,7 @@ static int at_xdmac_probe(struct platform_device *pdev)
 		return PTR_ERR(atxdmac->clk);
 	}
 
-	/* Do not use dev res to prevent races with tasklet */
+	/* Do not use devm resources to prevent races with the BH worker */
 	ret = request_irq(atxdmac->irq, at_xdmac_interrupt, 0, "at_xdmac", atxdmac);
 	if (ret) {
 		dev_err(&pdev->dev, "can't request irq\n");
@@ -2397,7 +2396,7 @@ static int at_xdmac_probe(struct platform_device *pdev)
 		spin_lock_init(&atchan->lock);
 		INIT_LIST_HEAD(&atchan->xfers_list);
 		INIT_LIST_HEAD(&atchan->free_descs_list);
-		tasklet_setup(&atchan->tasklet, at_xdmac_tasklet);
+		dma_chan_init_bh(&atchan->chan, at_xdmac_tasklet);
 
 		/* Clear pending interrupts. */
 		while (at_xdmac_chan_read(atchan, AT_XDMAC_CIS))
@@ -2458,7 +2457,7 @@ static void at_xdmac_remove(struct platform_device *pdev)
 	for (i = 0; i < atxdmac->dma.chancnt; i++) {
 		struct at_xdmac_chan *atchan = &atxdmac->chan[i];
 
-		tasklet_kill(&atchan->tasklet);
+		dma_chan_kill_bh(&atchan->chan);
 		at_xdmac_free_chan_resources(&atchan->chan);
 	}
 }
-- 
2.43.0

[PATCH v2 05/64] dmaengine: ep93xx: hook callbacks via dma_chan BH

[Allen Pais]

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/ep93xx_dma.c | 12 +++++-------
 1 file changed, 5 insertions(+), 7 deletions(-)

diff --git a/drivers/dma/ep93xx_dma.c b/drivers/dma/ep93xx_dma.c
index 311e55a97ba9..8f5200750fe9 100644
--- a/drivers/dma/ep93xx_dma.c
+++ b/drivers/dma/ep93xx_dma.c
@@ -167,7 +167,6 @@ struct ep93xx_dma_chan_cfg {
  * @dma_cfg: channel number, direction
  * @irq: interrupt number of the channel
  * @clk: clock used by this channel
- * @tasklet: channel specific tasklet used for callbacks
  * @lock: lock protecting the fields following
  * @flags: flags for the channel
  * @buffer: which buffer to use next (0/1)
@@ -196,7 +195,6 @@ struct ep93xx_dma_chan {
 	struct ep93xx_dma_chan_cfg	dma_cfg;
 	int				irq;
 	struct clk			*clk;
-	struct tasklet_struct		tasklet;
 	/* protects the fields following */
 	spinlock_t			lock;
 	unsigned long			flags;
@@ -801,9 +799,9 @@ static void ep93xx_dma_advance_work(struct ep93xx_dma_chan *edmac)
 	spin_unlock_irqrestore(&edmac->lock, flags);
 }
 
-static void ep93xx_dma_tasklet(struct tasklet_struct *t)
+static void ep93xx_dma_tasklet(struct dma_chan *chan)
 {
-	struct ep93xx_dma_chan *edmac = from_tasklet(edmac, t, tasklet);
+	struct ep93xx_dma_chan *edmac = to_ep93xx_dma_chan(chan);
 	struct ep93xx_dma_desc *desc, *d;
 	struct dmaengine_desc_callback cb;
 	LIST_HEAD(list);
@@ -858,12 +856,12 @@ static irqreturn_t ep93xx_dma_interrupt(int irq, void *dev_id)
 	switch (edmac->edma->hw_interrupt(edmac)) {
 	case INTERRUPT_DONE:
 		desc->complete = true;
-		tasklet_schedule(&edmac->tasklet);
+		dma_chan_schedule_bh(&edmac->chan);
 		break;
 
 	case INTERRUPT_NEXT_BUFFER:
 		if (test_bit(EP93XX_DMA_IS_CYCLIC, &edmac->flags))
-			tasklet_schedule(&edmac->tasklet);
+			dma_chan_schedule_bh(&edmac->chan);
 		break;
 
 	default:
@@ -1420,7 +1418,7 @@ static struct ep93xx_dma_engine *ep93xx_dma_of_probe(struct platform_device *pde
 		INIT_LIST_HEAD(&edmac->active);
 		INIT_LIST_HEAD(&edmac->queue);
 		INIT_LIST_HEAD(&edmac->free_list);
-		tasklet_setup(&edmac->tasklet, ep93xx_dma_tasklet);
+		dma_chan_init_bh(&edmac->chan, ep93xx_dma_tasklet);
 
 		list_add_tail(&edmac->chan.device_node,
 			      &dma_dev->channels);
-- 
2.43.0

[PATCH v2 06/64] dmaengine: fsldma: migrate tasklet to dma_chan BH

[Allen Pais]

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/fsldma.c | 10 +++++-----
 drivers/dma/fsldma.h |  1 -
 2 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/drivers/dma/fsldma.c b/drivers/dma/fsldma.c
index 22d62d958abd..1cb6ce327078 100644
--- a/drivers/dma/fsldma.c
+++ b/drivers/dma/fsldma.c
@@ -968,18 +968,18 @@ static irqreturn_t fsldma_chan_irq(int irq, void *data)
 		chan_err(chan, "irq: unhandled sr 0x%08x\n", stat);
 
 	/*
-	 * Schedule the tasklet to handle all cleanup of the current
+	 * Queue the BH worker to handle all cleanup of the current
 	 * transaction. It will start a new transaction if there is
 	 * one pending.
 	 */
-	tasklet_schedule(&chan->tasklet);
+	dma_chan_schedule_bh(&chan->common);
 	chan_dbg(chan, "irq: Exit\n");
 	return IRQ_HANDLED;
 }
 
-static void dma_do_tasklet(struct tasklet_struct *t)
+static void dma_do_tasklet(struct dma_chan *c)
 {
-	struct fsldma_chan *chan = from_tasklet(chan, t, tasklet);
+	struct fsldma_chan *chan = to_fsl_chan(c);
 
 	chan_dbg(chan, "tasklet entry\n");
 
@@ -1152,7 +1152,7 @@ static int fsl_dma_chan_probe(struct fsldma_device *fdev,
 	}
 
 	fdev->chan[chan->id] = chan;
-	tasklet_setup(&chan->tasklet, dma_do_tasklet);
+	dma_chan_init_bh(&chan->common, dma_do_tasklet);
 	snprintf(chan->name, sizeof(chan->name), "chan%d", chan->id);
 
 	/* Initialize the channel */
diff --git a/drivers/dma/fsldma.h b/drivers/dma/fsldma.h
index d7b7a3138b85..cab33d010d51 100644
--- a/drivers/dma/fsldma.h
+++ b/drivers/dma/fsldma.h
@@ -173,7 +173,6 @@ struct fsldma_chan {
 	struct device *dev;		/* Channel device */
 	int irq;			/* Channel IRQ */
 	int id;				/* Raw id of this channel */
-	struct tasklet_struct tasklet;
 	u32 feature;
 	bool idle;			/* DMA controller is idle */
 #ifdef CONFIG_PM
-- 
2.43.0

[PATCH v2 07/64] dmaengine: fsl_raid: run completions via dma_chan BH

[Allen Pais]

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/fsl_raid.c | 11 ++++++-----
 drivers/dma/fsl_raid.h |  1 -
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/dma/fsl_raid.c b/drivers/dma/fsl_raid.c
index 99945845d8b5..e91c18d3c327 100644
--- a/drivers/dma/fsl_raid.c
+++ b/drivers/dma/fsl_raid.c
@@ -155,9 +155,10 @@ static void fsl_re_cleanup_descs(struct fsl_re_chan *re_chan)
 	fsl_re_issue_pending(&re_chan->chan);
 }
 
-static void fsl_re_dequeue(struct tasklet_struct *t)
+static void fsl_re_dequeue(struct dma_chan *chan)
 {
-	struct fsl_re_chan *re_chan = from_tasklet(re_chan, t, irqtask);
+	struct fsl_re_chan *re_chan = container_of(chan, struct fsl_re_chan,
+						   chan);
 	struct fsl_re_desc *desc, *_desc;
 	struct fsl_re_hw_desc *hwdesc;
 	unsigned long flags;
@@ -224,7 +225,7 @@ static irqreturn_t fsl_re_isr(int irq, void *data)
 	/* Clear interrupt */
 	out_be32(&re_chan->jrregs->jr_interrupt_status, FSL_RE_CLR_INTR);
 
-	tasklet_schedule(&re_chan->irqtask);
+	dma_chan_schedule_bh(&re_chan->chan);
 
 	return IRQ_HANDLED;
 }
@@ -670,7 +671,7 @@ static int fsl_re_chan_probe(struct platform_device *ofdev,
 	snprintf(chan->name, sizeof(chan->name), "re_jr%02d", q);
 
 	chandev = &chan_ofdev->dev;
-	tasklet_setup(&chan->irqtask, fsl_re_dequeue);
+	dma_chan_init_bh(&chan->chan, fsl_re_dequeue);
 
 	ret = request_irq(chan->irq, fsl_re_isr, 0, chan->name, chandev);
 	if (ret) {
@@ -846,7 +847,7 @@ static int fsl_re_probe(struct platform_device *ofdev)
 
 static void fsl_re_remove_chan(struct fsl_re_chan *chan)
 {
-	tasklet_kill(&chan->irqtask);
+	dma_chan_kill_bh(&chan->chan);
 
 	dma_pool_free(chan->re_dev->hw_desc_pool, chan->inb_ring_virt_addr,
 		      chan->inb_phys_addr);
diff --git a/drivers/dma/fsl_raid.h b/drivers/dma/fsl_raid.h
index 69d743c04973..a40001a4db58 100644
--- a/drivers/dma/fsl_raid.h
+++ b/drivers/dma/fsl_raid.h
@@ -275,7 +275,6 @@ struct fsl_re_chan {
 	struct dma_chan chan;
 	struct fsl_re_chan_cfg *jrregs;
 	int irq;
-	struct tasklet_struct irqtask;
 	u32 alloc_count;
 
 	/* hw descriptor ring for inbound queue*/
-- 
2.43.0

[PATCH v2 08/64] dmaengine: imx-dma: flip per-chan tasklet to dma_chan BH

[Allen Pais]

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/imx-dma.c | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)

diff --git a/drivers/dma/imx-dma.c b/drivers/dma/imx-dma.c
index 81c6276436f8..3ff0bfbfa5ff 100644
--- a/drivers/dma/imx-dma.c
+++ b/drivers/dma/imx-dma.c
@@ -145,7 +145,6 @@ struct imxdma_channel {
 	struct imxdma_engine		*imxdma;
 	unsigned int			channel;
 
-	struct tasklet_struct		dma_tasklet;
 	struct list_head		ld_free;
 	struct list_head		ld_queue;
 	struct list_head		ld_active;
@@ -344,8 +343,8 @@ static void imxdma_watchdog(struct timer_list *t)
 
 	imx_dmav1_writel(imxdma, 0, DMA_CCR(channel));
 
-	/* Tasklet watchdog error handler */
-	tasklet_schedule(&imxdmac->dma_tasklet);
+	/* BH watchdog error handler */
+	dma_chan_schedule_bh(&imxdmac->chan);
 	dev_dbg(imxdma->dev, "channel %d: watchdog timeout!\n",
 		imxdmac->channel);
 }
@@ -390,8 +389,8 @@ static irqreturn_t imxdma_err_handler(int irq, void *dev_id)
 			imx_dmav1_writel(imxdma, 1 << i, DMA_DBOSR);
 			errcode |= IMX_DMA_ERR_BUFFER;
 		}
-		/* Tasklet error handler */
-		tasklet_schedule(&imxdma->channel[i].dma_tasklet);
+		/* BH error handler */
+		dma_chan_schedule_bh(&imxdma->channel[i].chan);
 
 		dev_warn(imxdma->dev,
 			 "DMA timeout on channel %d -%s%s%s%s\n", i,
@@ -448,8 +447,8 @@ static void dma_irq_handle_channel(struct imxdma_channel *imxdmac)
 			imx_dmav1_writel(imxdma, tmp, DMA_CCR(chno));
 
 			if (imxdma_chan_is_doing_cyclic(imxdmac))
-				/* Tasklet progression */
-				tasklet_schedule(&imxdmac->dma_tasklet);
+				/* BH progression */
+				dma_chan_schedule_bh(&imxdmac->chan);
 
 			return;
 		}
@@ -462,8 +461,8 @@ static void dma_irq_handle_channel(struct imxdma_channel *imxdmac)
 
 out:
 	imx_dmav1_writel(imxdma, 0, DMA_CCR(chno));
-	/* Tasklet irq */
-	tasklet_schedule(&imxdmac->dma_tasklet);
+	/* Schedule the IRQ BH */
+	dma_chan_schedule_bh(&imxdmac->chan);
 }
 
 static irqreturn_t dma_irq_handler(int irq, void *dev_id)
@@ -592,9 +591,10 @@ static int imxdma_xfer_desc(struct imxdma_desc *d)
 	return 0;
 }
 
-static void imxdma_tasklet(struct tasklet_struct *t)
+static void imxdma_tasklet(struct dma_chan *chan)
 {
-	struct imxdma_channel *imxdmac = from_tasklet(imxdmac, t, dma_tasklet);
+	struct imxdma_channel *imxdmac = container_of(chan, struct imxdma_channel,
+						      chan);
 	struct imxdma_engine *imxdma = imxdmac->imxdma;
 	struct imxdma_desc *desc, *next_desc;
 	unsigned long flags;
@@ -1142,7 +1142,7 @@ static int __init imxdma_probe(struct platform_device *pdev)
 		INIT_LIST_HEAD(&imxdmac->ld_free);
 		INIT_LIST_HEAD(&imxdmac->ld_active);
 
-		tasklet_setup(&imxdmac->dma_tasklet, imxdma_tasklet);
+		dma_chan_init_bh(&imxdmac->chan, imxdma_tasklet);
 		imxdmac->chan.device = &imxdma->dma_device;
 		dma_cookie_init(&imxdmac->chan);
 		imxdmac->channel = i;
@@ -1211,7 +1211,7 @@ static void imxdma_free_irq(struct platform_device *pdev, struct imxdma_engine *
 		if (!is_imx1_dma(imxdma))
 			disable_irq(imxdmac->irq);
 
-		tasklet_kill(&imxdmac->dma_tasklet);
+		dma_chan_kill_bh(&imxdmac->chan);
 	}
 }
 
-- 
2.43.0

[PATCH v2 09/64] dmaengine: ioat: convert cleanup tasklet to dma_chan BH

[Allen Pais]

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/ioat/dma.c  | 14 +++++++-------
 drivers/dma/ioat/dma.h  |  3 +--
 drivers/dma/ioat/init.c |  2 +-
 3 files changed, 9 insertions(+), 10 deletions(-)

diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c
index ee93b029f9e3..827b46c6354a 100644
--- a/drivers/dma/ioat/dma.c
+++ b/drivers/dma/ioat/dma.c
@@ -110,7 +110,7 @@ irqreturn_t ioat_dma_do_interrupt(int irq, void *data)
 	for_each_set_bit(bit, &attnstatus, BITS_PER_LONG) {
 		ioat_chan = ioat_chan_by_index(instance, bit);
 		if (test_bit(IOAT_RUN, &ioat_chan->state))
-			tasklet_schedule(&ioat_chan->cleanup_task);
+			dma_chan_schedule_bh(&ioat_chan->dma_chan);
 	}
 
 	writeb(intrctrl, instance->reg_base + IOAT_INTRCTRL_OFFSET);
@@ -127,7 +127,7 @@ irqreturn_t ioat_dma_do_interrupt_msix(int irq, void *data)
 	struct ioatdma_chan *ioat_chan = data;
 
 	if (test_bit(IOAT_RUN, &ioat_chan->state))
-		tasklet_schedule(&ioat_chan->cleanup_task);
+		dma_chan_schedule_bh(&ioat_chan->dma_chan);
 
 	return IRQ_HANDLED;
 }
@@ -161,11 +161,11 @@ void ioat_stop(struct ioatdma_chan *ioat_chan)
 	/* flush inflight timers */
 	timer_delete_sync(&ioat_chan->timer);
 
-	/* flush inflight tasklet runs */
-	tasklet_kill(&ioat_chan->cleanup_task);
+	/* flush inflight BH runs */
+	dma_chan_kill_bh(&ioat_chan->dma_chan);
 
 	/* final cleanup now that everything is quiesced and can't re-arm */
-	ioat_cleanup_event(&ioat_chan->cleanup_task);
+	ioat_cleanup_event(&ioat_chan->dma_chan);
 }
 
 static void __ioat_issue_pending(struct ioatdma_chan *ioat_chan)
@@ -690,9 +690,9 @@ static void ioat_cleanup(struct ioatdma_chan *ioat_chan)
 	spin_unlock_bh(&ioat_chan->cleanup_lock);
 }
 
-void ioat_cleanup_event(struct tasklet_struct *t)
+void ioat_cleanup_event(struct dma_chan *c)
 {
-	struct ioatdma_chan *ioat_chan = from_tasklet(ioat_chan, t, cleanup_task);
+	struct ioatdma_chan *ioat_chan = to_ioat_chan(c);
 
 	ioat_cleanup(ioat_chan);
 	if (!test_bit(IOAT_RUN, &ioat_chan->state))
diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h
index e8a880f338c6..4bdb1ae937e4 100644
--- a/drivers/dma/ioat/dma.h
+++ b/drivers/dma/ioat/dma.h
@@ -111,7 +111,6 @@ struct ioatdma_chan {
 	struct ioatdma_device *ioat_dma;
 	dma_addr_t completion_dma;
 	u64 *completion;
-	struct tasklet_struct cleanup_task;
 	struct kobject kobj;
 
 /* ioat v2 / v3 channel attributes
@@ -385,7 +384,7 @@ int ioat_reset_hw(struct ioatdma_chan *ioat_chan);
 enum dma_status
 ioat_tx_status(struct dma_chan *c, dma_cookie_t cookie,
 		struct dma_tx_state *txstate);
-void ioat_cleanup_event(struct tasklet_struct *t);
+void ioat_cleanup_event(struct dma_chan *c);
 void ioat_timer_event(struct timer_list *t);
 int ioat_check_space_lock(struct ioatdma_chan *ioat_chan, int num_descs);
 void ioat_issue_pending(struct dma_chan *chan);
diff --git a/drivers/dma/ioat/init.c b/drivers/dma/ioat/init.c
index 737496391109..314676e1cbc4 100644
--- a/drivers/dma/ioat/init.c
+++ b/drivers/dma/ioat/init.c
@@ -765,7 +765,7 @@ ioat_init_channel(struct ioatdma_device *ioat_dma,
 	list_add_tail(&ioat_chan->dma_chan.device_node, &dma->channels);
 	ioat_dma->idx[idx] = ioat_chan;
 	timer_setup(&ioat_chan->timer, ioat_timer_event, 0);
-	tasklet_setup(&ioat_chan->cleanup_task, ioat_cleanup_event);
+	dma_chan_init_bh(&ioat_chan->dma_chan, ioat_cleanup_event);
 }
 
 #define IOAT_NUM_SRC_TEST 6 /* must be <= 8 */
-- 
2.43.0

[PATCH v2 10/64] dmaengine: mmp_pdma: replace per-chan tasklet with dma_chan BH

[Allen Pais]

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/mmp_pdma.c | 13 +++++--------
 1 file changed, 5 insertions(+), 8 deletions(-)

diff --git a/drivers/dma/mmp_pdma.c b/drivers/dma/mmp_pdma.c
index 386e85cd4882..3fd9ef59a142 100644
--- a/drivers/dma/mmp_pdma.c
+++ b/drivers/dma/mmp_pdma.c
@@ -116,7 +116,6 @@ struct mmp_pdma_chan {
 						 * is in cyclic mode */
 
 	/* channel's basic info */
-	struct tasklet_struct tasklet;
 	u32 dcmd;
 	u32 drcmr;
 	u32 dev_addr;
@@ -374,7 +373,7 @@ static irqreturn_t mmp_pdma_chan_handler(int irq, void *dev_id)
 	if (clear_chan_irq(phy) != 0)
 		return IRQ_NONE;
 
-	tasklet_schedule(&phy->vchan->tasklet);
+	dma_chan_schedule_bh(&phy->vchan->chan);
 	return IRQ_HANDLED;
 }
 
@@ -1047,13 +1046,11 @@ static void mmp_pdma_issue_pending(struct dma_chan *dchan)
 }
 
 /*
- * dma_do_tasklet
- * Do call back
- * Start pending list
+ * dma_do_tasklet - BH worker to process completions and kick pending work
  */
-static void dma_do_tasklet(struct tasklet_struct *t)
+static void dma_do_tasklet(struct dma_chan *c)
 {
-	struct mmp_pdma_chan *chan = from_tasklet(chan, t, tasklet);
+	struct mmp_pdma_chan *chan = container_of(c, struct mmp_pdma_chan, chan);
 	struct mmp_pdma_desc_sw *desc, *_desc;
 	LIST_HEAD(chain_cleanup);
 	unsigned long flags;
@@ -1170,7 +1167,7 @@ static int mmp_pdma_chan_init(struct mmp_pdma_device *pdev, int idx, int irq)
 	spin_lock_init(&chan->desc_lock);
 	chan->dev = pdev->dev;
 	chan->chan.device = &pdev->device;
-	tasklet_setup(&chan->tasklet, dma_do_tasklet);
+	dma_chan_init_bh(&chan->chan, dma_do_tasklet);
 	INIT_LIST_HEAD(&chan->chain_pending);
 	INIT_LIST_HEAD(&chan->chain_running);
 
-- 
2.43.0

[PATCH v2 11/64] dmaengine: mmp_tdma: hook completions to dma_chan BH

[Allen Pais]

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/mmp_tdma.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/drivers/dma/mmp_tdma.c b/drivers/dma/mmp_tdma.c
index ba03321eeff7..d1ed2886f0e7 100644
--- a/drivers/dma/mmp_tdma.c
+++ b/drivers/dma/mmp_tdma.c
@@ -102,7 +102,6 @@ struct mmp_tdma_chan {
 	struct device			*dev;
 	struct dma_chan			chan;
 	struct dma_async_tx_descriptor	desc;
-	struct tasklet_struct		tasklet;
 
 	struct mmp_tdma_desc		*desc_arr;
 	dma_addr_t			desc_arr_phys;
@@ -320,7 +319,7 @@ static irqreturn_t mmp_tdma_chan_handler(int irq, void *dev_id)
 	struct mmp_tdma_chan *tdmac = dev_id;
 
 	if (mmp_tdma_clear_chan_irq(tdmac) == 0) {
-		tasklet_schedule(&tdmac->tasklet);
+		dma_chan_schedule_bh(&tdmac->chan);
 		return IRQ_HANDLED;
 	} else
 		return IRQ_NONE;
@@ -346,9 +345,9 @@ static irqreturn_t mmp_tdma_int_handler(int irq, void *dev_id)
 		return IRQ_NONE;
 }
 
-static void dma_do_tasklet(struct tasklet_struct *t)
+static void dma_do_tasklet(struct dma_chan *c)
 {
-	struct mmp_tdma_chan *tdmac = from_tasklet(tdmac, t, tasklet);
+	struct mmp_tdma_chan *tdmac = container_of(c, struct mmp_tdma_chan, chan);
 
 	dmaengine_desc_get_callback_invoke(&tdmac->desc, NULL);
 }
@@ -583,7 +582,7 @@ static int mmp_tdma_chan_init(struct mmp_tdma_device *tdev,
 	tdmac->pool	   = pool;
 	tdmac->status = DMA_COMPLETE;
 	tdev->tdmac[tdmac->idx] = tdmac;
-	tasklet_setup(&tdmac->tasklet, dma_do_tasklet);
+	dma_chan_init_bh(&tdmac->chan, dma_do_tasklet);
 
 	/* add the channel to tdma_chan list */
 	list_add_tail(&tdmac->chan.device_node,
-- 
2.43.0

[PATCH v2 12/64] dmaengine: mv_xor: convert irq tasklet to dma_chan BH

[Allen Pais]

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/mv_xor.c | 12 ++++++------
 drivers/dma/mv_xor.h |  2 --
 2 files changed, 6 insertions(+), 8 deletions(-)

diff --git a/drivers/dma/mv_xor.c b/drivers/dma/mv_xor.c
index 25ed61f1b089..3a70d565e1ef 100644
--- a/drivers/dma/mv_xor.c
+++ b/drivers/dma/mv_xor.c
@@ -327,7 +327,7 @@ static void mv_chan_slot_cleanup(struct mv_xor_chan *mv_chan)
 				 * some descriptors are still waiting
 				 * to be cleaned
 				 */
-				tasklet_schedule(&mv_chan->irq_tasklet);
+				dma_chan_schedule_bh(&mv_chan->dmachan);
 			}
 		}
 	}
@@ -336,9 +336,9 @@ static void mv_chan_slot_cleanup(struct mv_xor_chan *mv_chan)
 		mv_chan->dmachan.completed_cookie = cookie;
 }
 
-static void mv_xor_tasklet(struct tasklet_struct *t)
+static void mv_xor_tasklet(struct dma_chan *c)
 {
-	struct mv_xor_chan *chan = from_tasklet(chan, t, irq_tasklet);
+	struct mv_xor_chan *chan = to_mv_xor_chan(c);
 
 	spin_lock(&chan->lock);
 	mv_chan_slot_cleanup(chan);
@@ -372,7 +372,7 @@ mv_chan_alloc_slot(struct mv_xor_chan *mv_chan)
 	spin_unlock_bh(&mv_chan->lock);
 
 	/* try to free some slots if the allocation fails */
-	tasklet_schedule(&mv_chan->irq_tasklet);
+	dma_chan_schedule_bh(&mv_chan->dmachan);
 
 	return NULL;
 }
@@ -737,7 +737,7 @@ static irqreturn_t mv_xor_interrupt_handler(int irq, void *data)
 	if (intr_cause & XOR_INTR_ERRORS)
 		mv_chan_err_interrupt_handler(chan, intr_cause);
 
-	tasklet_schedule(&chan->irq_tasklet);
+	dma_chan_schedule_bh(&chan->dmachan);
 
 	mv_chan_clear_eoc_cause(chan);
 
@@ -1107,7 +1107,7 @@ mv_xor_channel_add(struct mv_xor_device *xordev,
 
 	mv_chan->mmr_base = xordev->xor_base;
 	mv_chan->mmr_high_base = xordev->xor_high_base;
-	tasklet_setup(&mv_chan->irq_tasklet, mv_xor_tasklet);
+	dma_chan_init_bh(&mv_chan->dmachan, mv_xor_tasklet);
 
 	/* clear errors before enabling interrupts */
 	mv_chan_clear_err_status(mv_chan);
diff --git a/drivers/dma/mv_xor.h b/drivers/dma/mv_xor.h
index c87cefd38a07..27774abe34a6 100644
--- a/drivers/dma/mv_xor.h
+++ b/drivers/dma/mv_xor.h
@@ -98,7 +98,6 @@ struct mv_xor_device {
  * @device: parent device
  * @common: common dmaengine channel object members
  * @slots_allocated: records the actual size of the descriptor slot pool
- * @irq_tasklet: bottom half where mv_xor_slot_cleanup runs
  * @op_in_desc: new mode of driver, each op is written to descriptor.
  */
 struct mv_xor_chan {
@@ -118,7 +117,6 @@ struct mv_xor_chan {
 	struct dma_device	dmadev;
 	struct dma_chan		dmachan;
 	int			slots_allocated;
-	struct tasklet_struct	irq_tasklet;
 	int                     op_in_desc;
 	char			dummy_src[MV_XOR_MIN_BYTE_COUNT];
 	char			dummy_dst[MV_XOR_MIN_BYTE_COUNT];
-- 
2.43.0

[PATCH v2 13/64] dmaengine: mxs-dma: use dma_chan BH scheduling

[Allen Pais]

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/mxs-dma.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/drivers/dma/mxs-dma.c b/drivers/dma/mxs-dma.c
index 7acb3d29dad3..8cecc0add82d 100644
--- a/drivers/dma/mxs-dma.c
+++ b/drivers/dma/mxs-dma.c
@@ -109,7 +109,6 @@ struct mxs_dma_chan {
 	struct mxs_dma_engine		*mxs_dma;
 	struct dma_chan			chan;
 	struct dma_async_tx_descriptor	desc;
-	struct tasklet_struct		tasklet;
 	unsigned int			chan_irq;
 	struct mxs_dma_ccw		*ccw;
 	dma_addr_t			ccw_phys;
@@ -300,9 +299,9 @@ static dma_cookie_t mxs_dma_tx_submit(struct dma_async_tx_descriptor *tx)
 	return dma_cookie_assign(tx);
 }
 
-static void mxs_dma_tasklet(struct tasklet_struct *t)
+static void mxs_dma_tasklet(struct dma_chan *chan)
 {
-	struct mxs_dma_chan *mxs_chan = from_tasklet(mxs_chan, t, tasklet);
+	struct mxs_dma_chan *mxs_chan = to_mxs_dma_chan(chan);
 
 	dmaengine_desc_get_callback_invoke(&mxs_chan->desc, NULL);
 }
@@ -386,8 +385,8 @@ static irqreturn_t mxs_dma_int_handler(int irq, void *dev_id)
 		dma_cookie_complete(&mxs_chan->desc);
 	}
 
-	/* schedule tasklet on this channel */
-	tasklet_schedule(&mxs_chan->tasklet);
+	/* schedule BH on this channel */
+	dma_chan_schedule_bh(&mxs_chan->chan);
 
 	return IRQ_HANDLED;
 }
@@ -781,7 +780,7 @@ static int mxs_dma_probe(struct platform_device *pdev)
 		mxs_chan->chan.device = &mxs_dma->dma_device;
 		dma_cookie_init(&mxs_chan->chan);
 
-		tasklet_setup(&mxs_chan->tasklet, mxs_dma_tasklet);
+		dma_chan_init_bh(&mxs_chan->chan, mxs_dma_tasklet);
 
 
 		/* Add the channel to mxs_chan list */
-- 
2.43.0

[PATCH v2 14/64] dmaengine: nbpfaxi: switch callbacks to dma_chan BH

[Allen Pais]

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/nbpfaxi.c | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/drivers/dma/nbpfaxi.c b/drivers/dma/nbpfaxi.c
index 05d7321629cc..aea94451258a 100644
--- a/drivers/dma/nbpfaxi.c
+++ b/drivers/dma/nbpfaxi.c
@@ -174,7 +174,6 @@ struct nbpf_desc_page {
 /**
  * struct nbpf_channel - one DMAC channel
  * @dma_chan:	standard dmaengine channel object
- * @tasklet:	channel specific tasklet used for callbacks
  * @base:	register address base
  * @nbpf:	DMAC
  * @name:	IRQ name
@@ -200,7 +199,6 @@ struct nbpf_desc_page {
  */
 struct nbpf_channel {
 	struct dma_chan dma_chan;
-	struct tasklet_struct tasklet;
 	void __iomem *base;
 	struct nbpf_device *nbpf;
 	char name[16];
@@ -1125,9 +1123,10 @@ static struct dma_chan *nbpf_of_xlate(struct of_phandle_args *dma_spec,
 	return dchan;
 }
 
-static void nbpf_chan_tasklet(struct tasklet_struct *t)
+static void nbpf_chan_tasklet(struct dma_chan *c)
 {
-	struct nbpf_channel *chan = from_tasklet(chan, t, tasklet);
+	struct nbpf_channel *chan = container_of(c, struct nbpf_channel,
+						 dma_chan);
 	struct nbpf_desc *desc, *tmp;
 	struct dmaengine_desc_callback cb;
 
@@ -1228,7 +1227,7 @@ static irqreturn_t nbpf_chan_irq(int irq, void *dev)
 	spin_unlock(&chan->lock);
 
 	if (bh)
-		tasklet_schedule(&chan->tasklet);
+		dma_chan_schedule_bh(&chan->dma_chan);
 
 	return ret;
 }
@@ -1272,7 +1271,7 @@ static int nbpf_chan_probe(struct nbpf_device *nbpf, int n)
 
 	snprintf(chan->name, sizeof(chan->name), "nbpf %d", n);
 
-	tasklet_setup(&chan->tasklet, nbpf_chan_tasklet);
+	dma_chan_init_bh(&chan->dma_chan, nbpf_chan_tasklet);
 	ret = devm_request_irq(dma_dev->dev, chan->irq,
 			nbpf_chan_irq, IRQF_SHARED,
 			chan->name, chan);
@@ -1478,7 +1477,7 @@ static void nbpf_remove(struct platform_device *pdev)
 
 		devm_free_irq(&pdev->dev, chan->irq, chan);
 
-		tasklet_kill(&chan->tasklet);
+		dma_chan_kill_bh(&chan->dma_chan);
 	}
 
 	of_dma_controller_free(pdev->dev.of_node);
-- 
2.43.0

[PATCH v2 15/64] dmaengine: pch_dma: convert tasklet to dma_chan BH

[Allen Pais]

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/pch_dma.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/drivers/dma/pch_dma.c b/drivers/dma/pch_dma.c
index bf805f1024f6..b0044bea8fe4 100644
--- a/drivers/dma/pch_dma.c
+++ b/drivers/dma/pch_dma.c
@@ -91,7 +91,6 @@ struct pch_dma_chan {
 	struct dma_chan		chan;
 	void __iomem *membase;
 	enum dma_transfer_direction dir;
-	struct tasklet_struct	tasklet;
 	unsigned long		err_status;
 
 	spinlock_t		lock;
@@ -665,9 +664,10 @@ static int pd_device_terminate_all(struct dma_chan *chan)
 	return 0;
 }
 
-static void pdc_tasklet(struct tasklet_struct *t)
+static void pdc_tasklet(struct dma_chan *c)
 {
-	struct pch_dma_chan *pd_chan = from_tasklet(pd_chan, t, tasklet);
+	struct pch_dma_chan *pd_chan = container_of(c, struct pch_dma_chan,
+						    chan);
 	unsigned long flags;
 
 	if (!pdc_is_idle(pd_chan)) {
@@ -707,7 +707,7 @@ static irqreturn_t pd_irq(int irq, void *devid)
 				if (sts0 & DMA_STATUS0_ERR(i))
 					set_bit(0, &pd_chan->err_status);
 
-				tasklet_schedule(&pd_chan->tasklet);
+				dma_chan_schedule_bh(&pd_chan->chan);
 				ret0 = IRQ_HANDLED;
 			}
 		} else {
@@ -715,7 +715,7 @@ static irqreturn_t pd_irq(int irq, void *devid)
 				if (sts2 & DMA_STATUS2_ERR(i))
 					set_bit(0, &pd_chan->err_status);
 
-				tasklet_schedule(&pd_chan->tasklet);
+				dma_chan_schedule_bh(&pd_chan->chan);
 				ret2 = IRQ_HANDLED;
 			}
 		}
@@ -877,7 +877,7 @@ static int pch_dma_probe(struct pci_dev *pdev,
 		INIT_LIST_HEAD(&pd_chan->queue);
 		INIT_LIST_HEAD(&pd_chan->free_list);
 
-		tasklet_setup(&pd_chan->tasklet, pdc_tasklet);
+		dma_chan_init_bh(&pd_chan->chan, pdc_tasklet);
 		list_add_tail(&pd_chan->chan.device_node, &pd->dma.channels);
 	}
 
@@ -930,7 +930,7 @@ static void pch_dma_remove(struct pci_dev *pdev)
 					 device_node) {
 			pd_chan = to_pd_chan(chan);
 
-			tasklet_kill(&pd_chan->tasklet);
+			dma_chan_kill_bh(&pd_chan->chan);
 		}
 
 		dma_pool_destroy(pd->pool);
-- 
2.43.0

[PATCH v2 16/64] dmaengine: ppc4xx: replace irq tasklet with dma_chan BH

[Allen Pais]

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/ppc4xx/adma.c | 14 +++++++-------
 drivers/dma/ppc4xx/adma.h |  2 --
 2 files changed, 7 insertions(+), 9 deletions(-)

diff --git a/drivers/dma/ppc4xx/adma.c b/drivers/dma/ppc4xx/adma.c
index 279a431ccae3..98cfd012ca5b 100644
--- a/drivers/dma/ppc4xx/adma.c
+++ b/drivers/dma/ppc4xx/adma.c
@@ -1658,9 +1658,9 @@ static void __ppc440spe_adma_slot_cleanup(struct ppc440spe_adma_chan *chan)
 /**
  * ppc440spe_adma_tasklet - clean up watch-dog initiator
  */
-static void ppc440spe_adma_tasklet(struct tasklet_struct *t)
+static void ppc440spe_adma_tasklet(struct dma_chan *c)
 {
-	struct ppc440spe_adma_chan *chan = from_tasklet(chan, t, irq_tasklet);
+	struct ppc440spe_adma_chan *chan = to_ppc440spe_adma_chan(c);
 
 	spin_lock_nested(&chan->lock, SINGLE_DEPTH_NESTING);
 	__ppc440spe_adma_slot_cleanup(chan);
@@ -1754,7 +1754,7 @@ static struct ppc440spe_adma_desc_slot *ppc440spe_adma_alloc_slots(
 		goto retry;
 
 	/* try to free some slots if the allocation fails */
-	tasklet_schedule(&chan->irq_tasklet);
+	dma_chan_schedule_bh(&chan->common);
 	return NULL;
 }
 
@@ -3595,7 +3595,7 @@ static irqreturn_t ppc440spe_adma_eot_handler(int irq, void *data)
 	dev_dbg(chan->device->common.dev,
 		"ppc440spe adma%d: %s\n", chan->device->id, __func__);
 
-	tasklet_schedule(&chan->irq_tasklet);
+	dma_chan_schedule_bh(&chan->common);
 	ppc440spe_adma_device_clear_eot_status(chan);
 
 	return IRQ_HANDLED;
@@ -3612,7 +3612,7 @@ static irqreturn_t ppc440spe_adma_err_handler(int irq, void *data)
 	dev_dbg(chan->device->common.dev,
 		"ppc440spe adma%d: %s\n", chan->device->id, __func__);
 
-	tasklet_schedule(&chan->irq_tasklet);
+	dma_chan_schedule_bh(&chan->common);
 	ppc440spe_adma_device_clear_eot_status(chan);
 
 	return IRQ_HANDLED;
@@ -4137,7 +4137,7 @@ static int ppc440spe_adma_probe(struct platform_device *ofdev)
 	chan->common.device = &adev->common;
 	dma_cookie_init(&chan->common);
 	list_add_tail(&chan->common.device_node, &adev->common.channels);
-	tasklet_setup(&chan->irq_tasklet, ppc440spe_adma_tasklet);
+	dma_chan_init_bh(&chan->common, ppc440spe_adma_tasklet);
 
 	/* allocate and map helper pages for async validation or
 	 * async_mult/async_sum_product operations on DMA0/1.
@@ -4247,7 +4247,7 @@ static void ppc440spe_adma_remove(struct platform_device *ofdev)
 				 device_node) {
 		ppc440spe_chan = to_ppc440spe_adma_chan(chan);
 		ppc440spe_adma_release_irqs(adev, ppc440spe_chan);
-		tasklet_kill(&ppc440spe_chan->irq_tasklet);
+		dma_chan_kill_bh(&ppc440spe_chan->common);
 		if (adev->id != PPC440SPE_XOR_ID) {
 			dma_unmap_page(&ofdev->dev, ppc440spe_chan->pdest,
 					PAGE_SIZE, DMA_BIDIRECTIONAL);
diff --git a/drivers/dma/ppc4xx/adma.h b/drivers/dma/ppc4xx/adma.h
index f8a5d7c1fb40..9af1feaf0b9a 100644
--- a/drivers/dma/ppc4xx/adma.h
+++ b/drivers/dma/ppc4xx/adma.h
@@ -80,7 +80,6 @@ struct ppc440spe_adma_device {
  * @pending: allows batching of hardware operations
  * @slots_allocated: records the actual size of the descriptor slot pool
  * @hw_chain_inited: h/w descriptor chain initialization flag
- * @irq_tasklet: bottom half where ppc440spe_adma_slot_cleanup runs
  * @needs_unmap: if buffers should not be unmapped upon final processing
  * @pdest_page: P destination page for async validate operation
  * @qdest_page: Q destination page for async validate operation
@@ -97,7 +96,6 @@ struct ppc440spe_adma_chan {
 	int pending;
 	int slots_allocated;
 	int hw_chain_inited;
-	struct tasklet_struct irq_tasklet;
 	u8 needs_unmap;
 	struct page *pdest_page;
 	struct page *qdest_page;
-- 
2.43.0

[PATCH v2 17/64] dmaengine: ste_dma40: convert per-channel tasklet to dma_chan BH

[Allen Pais]

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/ste_dma40.c | 13 +++++--------
 1 file changed, 5 insertions(+), 8 deletions(-)

diff --git a/drivers/dma/ste_dma40.c b/drivers/dma/ste_dma40.c
index 0d9ffa3e2663..f279a093a81f 100644
--- a/drivers/dma/ste_dma40.c
+++ b/drivers/dma/ste_dma40.c
@@ -461,8 +461,6 @@ struct d40_base;
  * @phy_chan: Pointer to physical channel which this instance runs on. If this
  * point is NULL, then the channel is not allocated.
  * @chan: DMA engine handle.
- * @tasklet: Tasklet that gets scheduled from interrupt context to complete a
- * transfer and call client callback.
  * @client: Cliented owned descriptor list.
  * @pending_queue: Submitted jobs, to be issued by issue_pending()
  * @active: Active descriptor.
@@ -489,7 +487,6 @@ struct d40_chan {
 	bool				 busy;
 	struct d40_phy_res		*phy_chan;
 	struct dma_chan			 chan;
-	struct tasklet_struct		 tasklet;
 	struct list_head		 client;
 	struct list_head		 pending_queue;
 	struct list_head		 active;
@@ -1587,13 +1584,13 @@ static void dma_tc_handle(struct d40_chan *d40c)
 	}
 
 	d40c->pending_tx++;
-	tasklet_schedule(&d40c->tasklet);
+	dma_chan_schedule_bh(&d40c->chan);
 
 }
 
-static void dma_tasklet(struct tasklet_struct *t)
+static void dma_tasklet(struct dma_chan *chan)
 {
-	struct d40_chan *d40c = from_tasklet(d40c, t, tasklet);
+	struct d40_chan *d40c = container_of(chan, struct d40_chan, chan);
 	struct d40_desc *d40d;
 	unsigned long flags;
 	bool callback_active;
@@ -1641,7 +1638,7 @@ static void dma_tasklet(struct tasklet_struct *t)
 	d40c->pending_tx--;
 
 	if (d40c->pending_tx)
-		tasklet_schedule(&d40c->tasklet);
+		dma_chan_schedule_bh(&d40c->chan);
 
 	spin_unlock_irqrestore(&d40c->lock, flags);
 
@@ -2815,7 +2812,7 @@ static void __init d40_chan_init(struct d40_base *base, struct dma_device *dma,
 		INIT_LIST_HEAD(&d40c->client);
 		INIT_LIST_HEAD(&d40c->prepare_queue);
 
-		tasklet_setup(&d40c->tasklet, dma_tasklet);
+		dma_chan_init_bh(&d40c->chan, dma_tasklet);
 
 		list_add_tail(&d40c->chan.device_node,
 			      &dma->channels);
-- 
2.43.0

[PATCH v2 18/64] dmaengine: xgene-dma: wire descriptor cleanup to dma_chan BH

[Allen Pais]

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/xgene-dma.c | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/drivers/dma/xgene-dma.c b/drivers/dma/xgene-dma.c
index fa1173e49900..8d0428c2944e 100644
--- a/drivers/dma/xgene-dma.c
+++ b/drivers/dma/xgene-dma.c
@@ -260,7 +260,6 @@ struct xgene_dma_desc_sw {
  *	These descriptors have already had their cleanup actions run. They
  *	are waiting for the ACK bit to be set by the async tx API.
  * @desc_pool: descriptor pool for DMA operations
- * @tasklet: bottom half where all completed descriptors cleans
  * @tx_ring: transmit ring descriptor that we use to prepare actual
  *	descriptors for further executions
  * @rx_ring: receive ring descriptor that we use to get completed DMA
@@ -280,7 +279,6 @@ struct xgene_dma_chan {
 	struct list_head ld_running;
 	struct list_head ld_completed;
 	struct dma_pool *desc_pool;
-	struct tasklet_struct tasklet;
 	struct xgene_dma_ring tx_ring;
 	struct xgene_dma_ring rx_ring;
 };
@@ -975,9 +973,10 @@ static enum dma_status xgene_dma_tx_status(struct dma_chan *dchan,
 	return dma_cookie_status(dchan, cookie, txstate);
 }
 
-static void xgene_dma_tasklet_cb(struct tasklet_struct *t)
+static void xgene_dma_tasklet_cb(struct dma_chan *c)
 {
-	struct xgene_dma_chan *chan = from_tasklet(chan, t, tasklet);
+	struct xgene_dma_chan *chan = container_of(c, struct xgene_dma_chan,
+						   dma_chan);
 
 	/* Run all cleanup for descriptors which have been completed */
 	xgene_dma_cleanup_descriptors(chan);
@@ -999,11 +998,11 @@ static irqreturn_t xgene_dma_chan_ring_isr(int irq, void *id)
 	disable_irq_nosync(chan->rx_irq);
 
 	/*
-	 * Schedule the tasklet to handle all cleanup of the current
+	 * Schedule the bottom half to handle all cleanup of the current
 	 * transaction. It will start a new transaction if there is
 	 * one pending.
 	 */
-	tasklet_schedule(&chan->tasklet);
+	dma_chan_schedule_bh(&chan->dma_chan);
 
 	return IRQ_HANDLED;
 }
@@ -1539,7 +1538,7 @@ static int xgene_dma_async_register(struct xgene_dma *pdma, int id)
 	INIT_LIST_HEAD(&chan->ld_pending);
 	INIT_LIST_HEAD(&chan->ld_running);
 	INIT_LIST_HEAD(&chan->ld_completed);
-	tasklet_setup(&chan->tasklet, xgene_dma_tasklet_cb);
+	dma_chan_init_bh(&chan->dma_chan, xgene_dma_tasklet_cb);
 
 	chan->pending = 0;
 	chan->desc_pool = NULL;
@@ -1556,7 +1555,7 @@ static int xgene_dma_async_register(struct xgene_dma *pdma, int id)
 	ret = dma_async_device_register(dma_dev);
 	if (ret) {
 		chan_err(chan, "Failed to register async device %d", ret);
-		tasklet_kill(&chan->tasklet);
+		dma_chan_kill_bh(&chan->dma_chan);
 
 		return ret;
 	}
@@ -1579,7 +1578,7 @@ static int xgene_dma_init_async(struct xgene_dma *pdma)
 		if (ret) {
 			for (j = 0; j < i; j++) {
 				dma_async_device_unregister(&pdma->dma_dev[j]);
-				tasklet_kill(&pdma->chan[j].tasklet);
+				dma_chan_kill_bh(&pdma->chan[j].dma_chan);
 			}
 
 			return ret;
@@ -1790,7 +1789,7 @@ static void xgene_dma_remove(struct platform_device *pdev)
 
 	for (i = 0; i < XGENE_DMA_MAX_CHANNEL; i++) {
 		chan = &pdma->chan[i];
-		tasklet_kill(&chan->tasklet);
+		dma_chan_kill_bh(&chan->dma_chan);
 		xgene_dma_delete_chan_rings(chan);
 	}
 
-- 
2.43.0

[PATCH v2 19/64] dmaengine: xilinx-dma: use dma_chan BH instead of tasklets

[Allen Pais]

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/xilinx/xilinx_dma.c | 20 +++++++++-----------
 1 file changed, 9 insertions(+), 11 deletions(-)

diff --git a/drivers/dma/xilinx/xilinx_dma.c b/drivers/dma/xilinx/xilinx_dma.c
index 404235c17353..bf9f22ed1265 100644
--- a/drivers/dma/xilinx/xilinx_dma.c
+++ b/drivers/dma/xilinx/xilinx_dma.c
@@ -402,7 +402,6 @@ struct xilinx_dma_tx_descriptor {
  * @err: Channel has errors
  * @idle: Check for channel idle
  * @terminating: Check for channel being synchronized by user
- * @tasklet: Cleanup work after irq
  * @config: Device configuration info
  * @flush_on_fsync: Flush on Frame sync
  * @desc_pendingcount: Descriptor pending count
@@ -441,7 +440,6 @@ struct xilinx_dma_chan {
 	bool err;
 	bool idle;
 	bool terminating;
-	struct tasklet_struct tasklet;
 	struct xilinx_vdma_config config;
 	bool flush_on_fsync;
 	u32 desc_pendingcount;
@@ -1115,11 +1113,12 @@ static void xilinx_dma_chan_desc_cleanup(struct xilinx_dma_chan *chan)
 
 /**
  * xilinx_dma_do_tasklet - Schedule completion tasklet
- * @t: Pointer to the Xilinx DMA channel structure
+ * @c: Pointer to the Xilinx DMA channel structure
  */
-static void xilinx_dma_do_tasklet(struct tasklet_struct *t)
+static void xilinx_dma_do_tasklet(struct dma_chan *c)
 {
-	struct xilinx_dma_chan *chan = from_tasklet(chan, t, tasklet);
+	struct xilinx_dma_chan *chan = container_of(c,
+			struct xilinx_dma_chan, common);
 
 	xilinx_dma_chan_desc_cleanup(chan);
 }
@@ -1898,7 +1897,7 @@ static irqreturn_t xilinx_mcdma_irq_handler(int irq, void *data)
 		spin_unlock(&chan->lock);
 	}
 
-	tasklet_hi_schedule(&chan->tasklet);
+	dma_chan_schedule_bh(&chan->common);
 	return IRQ_HANDLED;
 }
 
@@ -1955,7 +1954,7 @@ static irqreturn_t xilinx_dma_irq_handler(int irq, void *data)
 		spin_unlock(&chan->lock);
 	}
 
-	tasklet_schedule(&chan->tasklet);
+	dma_chan_schedule_bh(&chan->common);
 	return IRQ_HANDLED;
 }
 
@@ -2654,7 +2653,7 @@ static void xilinx_dma_synchronize(struct dma_chan *dchan)
 {
 	struct xilinx_dma_chan *chan = to_xilinx_chan(dchan);
 
-	tasklet_kill(&chan->tasklet);
+	dma_chan_kill_bh(&chan->common);
 }
 
 /**
@@ -2745,7 +2744,7 @@ static void xilinx_dma_chan_remove(struct xilinx_dma_chan *chan)
 	if (chan->irq > 0)
 		free_irq(chan->irq, chan);
 
-	tasklet_kill(&chan->tasklet);
+	dma_chan_kill_bh(&chan->common);
 
 	list_del(&chan->common.device_node);
 }
@@ -3075,8 +3074,7 @@ static int xilinx_dma_chan_probe(struct xilinx_dma_device *xdev,
 			str_enabled_disabled(chan->has_sg));
 	}
 
-	/* Initialize the tasklet */
-	tasklet_setup(&chan->tasklet, xilinx_dma_do_tasklet);
+	dma_chan_init_bh(&chan->common, xilinx_dma_do_tasklet);
 
 	/*
 	 * Initialize the DMA channel and add it to the DMA engine channels
-- 
2.43.0

[PATCH v2 20/64] dmaengine: xilinx-dpdma: kill vchan BH on remove

[Allen Pais]

virt-dma now dispatches completion callbacks through per-channel BH
work. Cancel that work when removing a channel, while retaining the
driver's separate error-handling tasklet.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/xilinx/xilinx_dpdma.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/dma/xilinx/xilinx_dpdma.c b/drivers/dma/xilinx/xilinx_dpdma.c
index d9a3542c4531..2ec82064fd2e 100644
--- a/drivers/dma/xilinx/xilinx_dpdma.c
+++ b/drivers/dma/xilinx/xilinx_dpdma.c
@@ -1685,6 +1685,7 @@ static void xilinx_dpdma_chan_remove(struct xilinx_dpdma_chan *chan)
 		return;
 
 	tasklet_kill(&chan->err_task);
+	dma_chan_kill_bh(&chan->vchan.chan);
 	list_del(&chan->vchan.chan.device_node);
 }
 
-- 
2.43.0

[PATCH v2 21/64] dmaengine: zynqmp-dma: switch completion tasklet to dma_chan BH

[Allen Pais]

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/xilinx/zynqmp_dma.c | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/drivers/dma/xilinx/zynqmp_dma.c b/drivers/dma/xilinx/zynqmp_dma.c
index f6a812e49ddc..6291126cdc9f 100644
--- a/drivers/dma/xilinx/zynqmp_dma.c
+++ b/drivers/dma/xilinx/zynqmp_dma.c
@@ -207,7 +207,6 @@ struct zynqmp_dma_desc_sw {
  * @dev: The dma device
  * @irq: Channel IRQ
  * @is_dmacoherent: Tells whether dma operations are coherent or not
- * @tasklet: Cleanup work after irq
  * @idle : Channel status;
  * @desc_size: Size of the low level descriptor
  * @err: Channel has errors
@@ -232,7 +231,6 @@ struct zynqmp_dma_chan {
 	struct device *dev;
 	int irq;
 	bool is_dmacoherent;
-	struct tasklet_struct tasklet;
 	bool idle;
 	size_t desc_size;
 	bool err;
@@ -735,7 +733,7 @@ static irqreturn_t zynqmp_dma_irq_handler(int irq, void *data)
 
 	writel(isr, chan->regs + ZYNQMP_DMA_ISR);
 	if (status & ZYNQMP_DMA_INT_DONE) {
-		tasklet_schedule(&chan->tasklet);
+		dma_chan_schedule_bh(&chan->common);
 		ret = IRQ_HANDLED;
 	}
 
@@ -744,7 +742,7 @@ static irqreturn_t zynqmp_dma_irq_handler(int irq, void *data)
 
 	if (status & ZYNQMP_DMA_INT_ERR) {
 		chan->err = true;
-		tasklet_schedule(&chan->tasklet);
+		dma_chan_schedule_bh(&chan->common);
 		dev_err(chan->dev, "Channel %p has errors\n", chan);
 		ret = IRQ_HANDLED;
 	}
@@ -760,11 +758,12 @@ static irqreturn_t zynqmp_dma_irq_handler(int irq, void *data)
 
 /**
  * zynqmp_dma_do_tasklet - Schedule completion tasklet
- * @t: Pointer to the ZynqMP DMA channel structure
+ * @c: Pointer to the ZynqMP DMA channel structure
  */
-static void zynqmp_dma_do_tasklet(struct tasklet_struct *t)
+static void zynqmp_dma_do_tasklet(struct dma_chan *c)
 {
-	struct zynqmp_dma_chan *chan = from_tasklet(chan, t, tasklet);
+	struct zynqmp_dma_chan *chan = container_of(c,
+			struct zynqmp_dma_chan, common);
 	u32 count;
 	unsigned long irqflags;
 
@@ -815,7 +814,7 @@ static void zynqmp_dma_synchronize(struct dma_chan *dchan)
 {
 	struct zynqmp_dma_chan *chan = to_chan(dchan);
 
-	tasklet_kill(&chan->tasklet);
+	dma_chan_kill_bh(&chan->common);
 }
 
 /**
@@ -887,7 +886,7 @@ static void zynqmp_dma_chan_remove(struct zynqmp_dma_chan *chan)
 
 	if (chan->irq)
 		devm_free_irq(chan->zdev->dev, chan->irq, chan);
-	tasklet_kill(&chan->tasklet);
+	dma_chan_kill_bh(&chan->common);
 	list_del(&chan->common.device_node);
 }
 
@@ -937,7 +936,7 @@ static int zynqmp_dma_chan_probe(struct zynqmp_dma_device *zdev,
 
 	chan->is_dmacoherent =  of_property_read_bool(node, "dma-coherent");
 	zdev->chan = chan;
-	tasklet_setup(&chan->tasklet, zynqmp_dma_do_tasklet);
+	dma_chan_init_bh(&chan->common, zynqmp_dma_do_tasklet);
 	spin_lock_init(&chan->lock);
 	INIT_LIST_HEAD(&chan->active_list);
 	INIT_LIST_HEAD(&chan->pending_list);
-- 
2.43.0

[PATCH v2 22/64] dmaengine: tegra20-apb: use channel BH helpers

[Allen Pais]

Switch the per-channel tasklet to dma_chan_*_bh so completion handling
runs through the shared per-channel deferral mechanism.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/tegra20-apb-dma.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/drivers/dma/tegra20-apb-dma.c b/drivers/dma/tegra20-apb-dma.c
index 640b8a218c9a..de15b8d3e8cb 100644
--- a/drivers/dma/tegra20-apb-dma.c
+++ b/drivers/dma/tegra20-apb-dma.c
@@ -194,9 +194,8 @@ struct tegra_dma_channel {
 	struct list_head	free_dma_desc;
 	struct list_head	cb_desc;
 
-	/* ISR handler and tasklet for bottom half of isr handling */
+	/* ISR handler for bottom-half processing */
 	dma_isr_handler		isr_handler;
-	struct tasklet_struct	tasklet;
 
 	/* Channel-slave specific configuration */
 	unsigned int slave_id;
@@ -632,9 +631,10 @@ static void handle_cont_sngl_cycle_dma_done(struct tegra_dma_channel *tdc,
 	}
 }
 
-static void tegra_dma_tasklet(struct tasklet_struct *t)
+static void tegra_dma_tasklet(struct dma_chan *chan)
 {
-	struct tegra_dma_channel *tdc = from_tasklet(tdc, t, tasklet);
+	struct tegra_dma_channel *tdc = container_of(chan, struct tegra_dma_channel,
+						     dma_chan);
 	struct dmaengine_desc_callback cb;
 	struct tegra_dma_desc *dma_desc;
 	unsigned int cb_count;
@@ -670,7 +670,7 @@ static irqreturn_t tegra_dma_isr(int irq, void *dev_id)
 	if (status & TEGRA_APBDMA_STATUS_ISE_EOC) {
 		tdc_write(tdc, TEGRA_APBDMA_CHAN_STATUS, status);
 		tdc->isr_handler(tdc, false);
-		tasklet_schedule(&tdc->tasklet);
+		dma_chan_schedule_bh(&tdc->dma_chan);
 		wake_up_all(&tdc->wq);
 		spin_unlock(&tdc->lock);
 		return IRQ_HANDLED;
@@ -819,7 +819,7 @@ static void tegra_dma_synchronize(struct dma_chan *dc)
 	 */
 	wait_event(tdc->wq, tegra_dma_eoc_interrupt_deasserted(tdc));
 
-	tasklet_kill(&tdc->tasklet);
+	dma_chan_kill_bh(&tdc->dma_chan);
 
 	pm_runtime_put(tdc->tdma->dev);
 }
@@ -1317,7 +1317,7 @@ static void tegra_dma_free_chan_resources(struct dma_chan *dc)
 	dev_dbg(tdc2dev(tdc), "Freeing channel %d\n", tdc->id);
 
 	tegra_dma_terminate_all(dc);
-	tasklet_kill(&tdc->tasklet);
+	dma_chan_kill_bh(&tdc->dma_chan);
 
 	list_splice_init(&tdc->pending_sg_req, &sg_req_list);
 	list_splice_init(&tdc->free_sg_req, &sg_req_list);
@@ -1511,7 +1511,7 @@ static int tegra_dma_probe(struct platform_device *pdev)
 		tdc->id = i;
 		tdc->slave_id = TEGRA_APBDMA_SLAVE_ID_INVALID;
 
-		tasklet_setup(&tdc->tasklet, tegra_dma_tasklet);
+		dma_chan_init_bh(&tdc->dma_chan, tegra_dma_tasklet);
 		spin_lock_init(&tdc->lock);
 		init_waitqueue_head(&tdc->wq);
 
@@ -1617,7 +1617,7 @@ static int __maybe_unused tegra_dma_dev_suspend(struct device *dev)
 	for (i = 0; i < tdma->chip_data->nr_channels; i++) {
 		struct tegra_dma_channel *tdc = &tdma->channels[i];
 
-		tasklet_kill(&tdc->tasklet);
+		dma_chan_kill_bh(&tdc->dma_chan);
 
 		spin_lock_irqsave(&tdc->lock, flags);
 		busy = tdc->busy;
-- 
2.43.0

[PATCH v2 23/64] dmaengine: timb_dma: route callbacks via channel BH

[Allen Pais]

Schedule per-channel BH work for completions so callbacks run through
the dma_chan deferral mechanism while keeping the controller tasklet for
IRQ handling.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/timb_dma.c | 21 ++++++++++++++++-----
 1 file changed, 16 insertions(+), 5 deletions(-)

diff --git a/drivers/dma/timb_dma.c b/drivers/dma/timb_dma.c
index 4fc5119854ec..7d43e5712ffe 100644
--- a/drivers/dma/timb_dma.c
+++ b/drivers/dma/timb_dma.c
@@ -438,6 +438,8 @@ static void td_free_chan_resources(struct dma_chan *chan)
 
 	dev_dbg(chan2dev(chan), "%s: Entry\n", __func__);
 
+	dma_chan_kill_bh(&td_chan->chan);
+
 	/* check that all descriptors are free */
 	BUG_ON(!list_empty(&td_chan->active_list));
 	BUG_ON(!list_empty(&td_chan->queue));
@@ -580,17 +582,25 @@ static void td_tasklet(struct tasklet_struct *t)
 	for (i = 0; i < td->dma.chancnt; i++)
 		if (ipr & (1 << i)) {
 			struct timb_dma_chan *td_chan = td->channels + i;
-			spin_lock(&td_chan->lock);
-			__td_finish(td_chan);
-			if (!list_empty(&td_chan->queue))
-				__td_start_next(td_chan);
-			spin_unlock(&td_chan->lock);
+			dma_chan_schedule_bh(&td_chan->chan);
 		}
 
 	ier = __td_ier_mask(td);
 	iowrite32(ier, td->membase + TIMBDMA_IER);
 }
 
+static void timb_dma_chan_bh(struct dma_chan *chan)
+{
+	struct timb_dma_chan *td_chan = container_of(chan, struct timb_dma_chan,
+						     chan);
+
+	spin_lock(&td_chan->lock);
+	__td_finish(td_chan);
+	if (!list_empty(&td_chan->queue))
+		__td_start_next(td_chan);
+	spin_unlock(&td_chan->lock);
+}
+
 
 static irqreturn_t td_irq(int irq, void *devid)
 {
@@ -697,6 +707,7 @@ static int td_probe(struct platform_device *pdev)
 		INIT_LIST_HEAD(&td_chan->active_list);
 		INIT_LIST_HEAD(&td_chan->queue);
 		INIT_LIST_HEAD(&td_chan->free_list);
+		dma_chan_init_bh(&td_chan->chan, timb_dma_chan_bh);
 
 		td_chan->descs = pchan->descriptors;
 		td_chan->desc_elems = pchan->descriptor_elements;
-- 
2.43.0

[PATCH v2 24/64] dmaengine: txx9dmac: route callbacks via channel BH

[Allen Pais]

Use the per-channel BH helpers for descriptor completion handling and
schedule per-channel work from both channel and controller IRQ paths.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/txx9dmac.c | 12 ++++++------
 drivers/dma/txx9dmac.h |  1 -
 2 files changed, 6 insertions(+), 7 deletions(-)

diff --git a/drivers/dma/txx9dmac.c b/drivers/dma/txx9dmac.c
index 05622b68a936..6269baa28c51 100644
--- a/drivers/dma/txx9dmac.c
+++ b/drivers/dma/txx9dmac.c
@@ -601,13 +601,13 @@ static void txx9dmac_scan_descriptors(struct txx9dmac_chan *dc)
 	}
 }
 
-static void txx9dmac_chan_tasklet(struct tasklet_struct *t)
+static void txx9dmac_chan_tasklet(struct dma_chan *chan)
 {
 	int irq;
 	u32 csr;
 	struct txx9dmac_chan *dc;
 
-	dc = from_tasklet(dc, t, tasklet);
+	dc = container_of(chan, struct txx9dmac_chan, chan);
 	csr = channel_readl(dc, CSR);
 	dev_vdbg(chan2dev(&dc->chan), "tasklet: status=%x\n", csr);
 
@@ -628,7 +628,7 @@ static irqreturn_t txx9dmac_chan_interrupt(int irq, void *dev_id)
 	dev_vdbg(chan2dev(&dc->chan), "interrupt: status=%#x\n",
 			channel_readl(dc, CSR));
 
-	tasklet_schedule(&dc->tasklet);
+	dma_chan_schedule_bh(&dc->chan);
 	/*
 	 * Just disable the interrupts. We'll turn them back on in the
 	 * softirq handler.
@@ -659,7 +659,7 @@ static void txx9dmac_tasklet(struct tasklet_struct *t)
 			spin_lock(&dc->lock);
 			if (csr & (TXX9_DMA_CSR_ABCHC | TXX9_DMA_CSR_NCHNC |
 				   TXX9_DMA_CSR_NTRNFC))
-				txx9dmac_scan_descriptors(dc);
+				dma_chan_schedule_bh(&dc->chan);
 			spin_unlock(&dc->lock);
 		}
 	}
@@ -1113,7 +1113,7 @@ static int __init txx9dmac_chan_probe(struct platform_device *pdev)
 		irq = platform_get_irq(pdev, 0);
 		if (irq < 0)
 			return irq;
-		tasklet_setup(&dc->tasklet, txx9dmac_chan_tasklet);
+		dma_chan_init_bh(&dc->chan, txx9dmac_chan_tasklet);
 		dc->irq = irq;
 		err = devm_request_irq(&pdev->dev, dc->irq,
 			txx9dmac_chan_interrupt, 0, dev_name(&pdev->dev), dc);
@@ -1159,7 +1159,7 @@ static void txx9dmac_chan_remove(struct platform_device *pdev)
 	dma_async_device_unregister(&dc->dma);
 	if (dc->irq >= 0) {
 		devm_free_irq(&pdev->dev, dc->irq, dc);
-		tasklet_kill(&dc->tasklet);
+		dma_chan_kill_bh(&dc->chan);
 	}
 	dc->ddev->chan[pdev->id % TXX9_DMA_MAX_NR_CHANNELS] = NULL;
 }
diff --git a/drivers/dma/txx9dmac.h b/drivers/dma/txx9dmac.h
index aa53eafb1519..f2abedb73d49 100644
--- a/drivers/dma/txx9dmac.h
+++ b/drivers/dma/txx9dmac.h
@@ -162,7 +162,6 @@ struct txx9dmac_chan {
 	struct dma_device	dma;
 	struct txx9dmac_dev	*ddev;
 	void __iomem		*ch_regs;
-	struct tasklet_struct	tasklet;
 	int			irq;
 	u32			ccr;
 
-- 
2.43.0

[PATCH v2 25/64] dmaengine: mv_xor_v2: use channel BH helpers

[Allen Pais]

Replace the device tasklet with the per-channel BH helper so descriptor
callbacks run through dma_chan_*_bh.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/mv_xor_v2.c | 20 +++++++++-----------
 1 file changed, 9 insertions(+), 11 deletions(-)

diff --git a/drivers/dma/mv_xor_v2.c b/drivers/dma/mv_xor_v2.c
index cad4d4fb51ac..e5fa7a221d5b 100644
--- a/drivers/dma/mv_xor_v2.c
+++ b/drivers/dma/mv_xor_v2.c
@@ -139,7 +139,6 @@ struct mv_xor_v2_descriptor {
  * @reg_clk: reference to the 'reg' clock
  * @dma_base: memory mapped DMA register base
  * @glob_base: memory mapped global register base
- * @irq_tasklet: tasklet used for IRQ handling call-backs
  * @free_sw_desc: linked list of free SW descriptors
  * @dmadev: dma device
  * @dmachan: dma channel
@@ -158,7 +157,6 @@ struct mv_xor_v2_device {
 	void __iomem *glob_base;
 	struct clk *clk;
 	struct clk *reg_clk;
-	struct tasklet_struct irq_tasklet;
 	struct list_head free_sw_desc;
 	struct dma_device dmadev;
 	struct dma_chan	dmachan;
@@ -290,8 +288,8 @@ static irqreturn_t mv_xor_v2_interrupt_handler(int irq, void *data)
 	if (!ndescs)
 		return IRQ_NONE;
 
-	/* schedule a tasklet to handle descriptors callbacks */
-	tasklet_schedule(&xor_dev->irq_tasklet);
+	/* schedule a bottom half to handle descriptors callbacks */
+	dma_chan_schedule_bh(&xor_dev->dmachan);
 
 	return IRQ_HANDLED;
 }
@@ -346,8 +344,8 @@ mv_xor_v2_prep_sw_desc(struct mv_xor_v2_device *xor_dev)
 
 	if (list_empty(&xor_dev->free_sw_desc)) {
 		spin_unlock_bh(&xor_dev->lock);
-		/* schedule tasklet to free some descriptors */
-		tasklet_schedule(&xor_dev->irq_tasklet);
+		/* schedule bottom half to free some descriptors */
+		dma_chan_schedule_bh(&xor_dev->dmachan);
 		return NULL;
 	}
 
@@ -553,10 +551,10 @@ int mv_xor_v2_get_pending_params(struct mv_xor_v2_device *xor_dev,
 /*
  * handle the descriptors after HW process
  */
-static void mv_xor_v2_tasklet(struct tasklet_struct *t)
+static void mv_xor_v2_tasklet(struct dma_chan *chan)
 {
-	struct mv_xor_v2_device *xor_dev = from_tasklet(xor_dev, t,
-							irq_tasklet);
+	struct mv_xor_v2_device *xor_dev =
+		container_of(chan, struct mv_xor_v2_device, dmachan);
 	int pending_ptr, num_of_pending, i;
 	struct mv_xor_v2_sw_desc *next_pending_sw_desc = NULL;
 
@@ -760,7 +758,7 @@ static int mv_xor_v2_probe(struct platform_device *pdev)
 	if (ret)
 		goto free_msi_irqs;
 
-	tasklet_setup(&xor_dev->irq_tasklet, mv_xor_v2_tasklet);
+	dma_chan_init_bh(&xor_dev->dmachan, mv_xor_v2_tasklet);
 
 	xor_dev->desc_size = mv_xor_v2_set_desc_size(xor_dev);
 
@@ -869,7 +867,7 @@ static void mv_xor_v2_remove(struct platform_device *pdev)
 
 	platform_device_msi_free_irqs_all(&pdev->dev);
 
-	tasklet_kill(&xor_dev->irq_tasklet);
+	dma_chan_kill_bh(&xor_dev->dmachan);
 }
 
 #ifdef CONFIG_OF
-- 
2.43.0

[PATCH v2 26/64] dmaengine: mpc512x: route callbacks via channel BH

[Allen Pais]

Schedule per-channel BH work for completions and process callbacks there,
leaving the controller tasklet for IRQ/error handling.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/mpc512x_dma.c | 74 +++++++++++++++++++++++++--------------
 1 file changed, 47 insertions(+), 27 deletions(-)

diff --git a/drivers/dma/mpc512x_dma.c b/drivers/dma/mpc512x_dma.c
index 0adc8e01057e..552ea72a9108 100644
--- a/drivers/dma/mpc512x_dma.c
+++ b/drivers/dma/mpc512x_dma.c
@@ -372,45 +372,47 @@ static irqreturn_t mpc_dma_irq(int irq, void *data)
 	return IRQ_HANDLED;
 }
 
-/* process completed descriptors */
-static void mpc_dma_process_completed(struct mpc_dma *mdma)
+static void mpc_dma_process_chan_completed(struct mpc_dma_chan *mchan)
 {
 	dma_cookie_t last_cookie = 0;
-	struct mpc_dma_chan *mchan;
 	struct mpc_dma_desc *mdesc;
 	struct dma_async_tx_descriptor *desc;
 	unsigned long flags;
 	LIST_HEAD(list);
-	int i;
 
-	for (i = 0; i < mdma->dma.chancnt; i++) {
-		mchan = &mdma->channels[i];
+	/* Get all completed descriptors */
+	spin_lock_irqsave(&mchan->lock, flags);
+	if (!list_empty(&mchan->completed))
+		list_splice_tail_init(&mchan->completed, &list);
+	spin_unlock_irqrestore(&mchan->lock, flags);
 
-		/* Get all completed descriptors */
-		spin_lock_irqsave(&mchan->lock, flags);
-		if (!list_empty(&mchan->completed))
-			list_splice_tail_init(&mchan->completed, &list);
-		spin_unlock_irqrestore(&mchan->lock, flags);
+	if (list_empty(&list))
+		return;
 
-		if (list_empty(&list))
-			continue;
+	/* Execute callbacks and run dependencies */
+	list_for_each_entry(mdesc, &list, node) {
+		desc = &mdesc->desc;
 
-		/* Execute callbacks and run dependencies */
-		list_for_each_entry(mdesc, &list, node) {
-			desc = &mdesc->desc;
+		dmaengine_desc_get_callback_invoke(desc, NULL);
 
-			dmaengine_desc_get_callback_invoke(desc, NULL);
+		last_cookie = desc->cookie;
+		dma_run_dependencies(desc);
+	}
 
-			last_cookie = desc->cookie;
-			dma_run_dependencies(desc);
-		}
+	/* Free descriptors */
+	spin_lock_irqsave(&mchan->lock, flags);
+	list_splice_tail_init(&list, &mchan->free);
+	mchan->chan.completed_cookie = last_cookie;
+	spin_unlock_irqrestore(&mchan->lock, flags);
+}
 
-		/* Free descriptors */
-		spin_lock_irqsave(&mchan->lock, flags);
-		list_splice_tail_init(&list, &mchan->free);
-		mchan->chan.completed_cookie = last_cookie;
-		spin_unlock_irqrestore(&mchan->lock, flags);
-	}
+/* process completed descriptors */
+static void mpc_dma_process_completed(struct mpc_dma *mdma)
+{
+	int i;
+
+	for (i = 0; i < mdma->dma.chancnt; i++)
+		mpc_dma_process_chan_completed(&mdma->channels[i]);
 }
 
 /* DMA Tasklet */
@@ -419,6 +421,7 @@ static void mpc_dma_tasklet(struct tasklet_struct *t)
 	struct mpc_dma *mdma = from_tasklet(mdma, t, tasklet);
 	unsigned long flags;
 	uint es;
+	int i;
 
 	spin_lock_irqsave(&mdma->error_status_lock, flags);
 	es = mdma->error_status;
@@ -453,7 +456,21 @@ static void mpc_dma_tasklet(struct tasklet_struct *t)
 			dev_err(mdma->dma.dev, "- Destination Bus Error\n");
 	}
 
-	mpc_dma_process_completed(mdma);
+	for (i = 0; i < mdma->dma.chancnt; i++) {
+		struct mpc_dma_chan *mchan = &mdma->channels[i];
+
+		spin_lock_irqsave(&mchan->lock, flags);
+		if (!list_empty(&mchan->completed))
+			dma_chan_schedule_bh(&mchan->chan);
+		spin_unlock_irqrestore(&mchan->lock, flags);
+	}
+}
+
+static void mpc_dma_chan_bh(struct dma_chan *chan)
+{
+	struct mpc_dma_chan *mchan = dma_chan_to_mpc_dma_chan(chan);
+
+	mpc_dma_process_chan_completed(mchan);
 }
 
 /* Submit descriptor to hardware */
@@ -551,6 +568,8 @@ static void mpc_dma_free_chan_resources(struct dma_chan *chan)
 	unsigned long flags;
 	LIST_HEAD(descs);
 
+	dma_chan_kill_bh(&mchan->chan);
+
 	spin_lock_irqsave(&mchan->lock, flags);
 
 	/* Channel must be idle */
@@ -1007,6 +1026,7 @@ static int mpc_dma_probe(struct platform_device *op)
 		INIT_LIST_HEAD(&mchan->completed);
 
 		spin_lock_init(&mchan->lock);
+		dma_chan_init_bh(&mchan->chan, mpc_dma_chan_bh);
 		list_add_tail(&mchan->chan.device_node, &dma->channels);
 	}
 
-- 
2.43.0

[PATCH v2 27/64] dmaengine: k3dma: kill vchan BH on remove

[Allen Pais]

Use dma_chan_kill_bh() for virt-dma cleanup now that vchan uses the
per-channel BH helper.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/k3dma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/k3dma.c b/drivers/dma/k3dma.c
index e84f197fea76..99a70b90a02b 100644
--- a/drivers/dma/k3dma.c
+++ b/drivers/dma/k3dma.c
@@ -976,7 +976,7 @@ static void k3_dma_remove(struct platform_device *op)
 
 	list_for_each_entry_safe(c, cn, &d->slave.channels, vc.chan.device_node) {
 		list_del(&c->vc.chan.device_node);
-		tasklet_kill(&c->vc.task);
+		dma_chan_kill_bh(&c->vc.chan);
 	}
 	tasklet_kill(&d->task);
 	clk_disable_unprepare(d->clk);
-- 
2.43.0

[PATCH v2 28/64] dmaengine: plx_dma: use channel BH helpers

[Allen Pais]

Replace the device tasklet with the per-channel BH helper so completion
callbacks run via dma_chan_*_bh.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/plx_dma.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/dma/plx_dma.c b/drivers/dma/plx_dma.c
index 84941a918b01..9d6bbdfa5a69 100644
--- a/drivers/dma/plx_dma.c
+++ b/drivers/dma/plx_dma.c
@@ -105,7 +105,6 @@ struct plx_dma_dev {
 	struct dma_chan dma_chan;
 	struct pci_dev __rcu *pdev;
 	void __iomem *bar;
-	struct tasklet_struct desc_task;
 
 	spinlock_t ring_lock;
 	bool ring_active;
@@ -241,9 +240,10 @@ static void plx_dma_stop(struct plx_dma_dev *plxdev)
 	rcu_read_unlock();
 }
 
-static void plx_dma_desc_task(struct tasklet_struct *t)
+static void plx_dma_desc_task(struct dma_chan *chan)
 {
-	struct plx_dma_dev *plxdev = from_tasklet(plxdev, t, desc_task);
+	struct plx_dma_dev *plxdev = container_of(chan, struct plx_dma_dev,
+						  dma_chan);
 
 	plx_dma_process_desc(plxdev);
 }
@@ -366,7 +366,7 @@ static irqreturn_t plx_dma_isr(int irq, void *devid)
 		return IRQ_NONE;
 
 	if (status & PLX_REG_INTR_STATUS_DESC_DONE && plxdev->ring_active)
-		tasklet_schedule(&plxdev->desc_task);
+		dma_chan_schedule_bh(&plxdev->dma_chan);
 
 	writew(status, plxdev->bar + PLX_REG_INTR_STATUS);
 
@@ -471,7 +471,7 @@ static void plx_dma_free_chan_resources(struct dma_chan *chan)
 	if (irq > 0)
 		synchronize_irq(irq);
 
-	tasklet_kill(&plxdev->desc_task);
+	dma_chan_kill_bh(&plxdev->dma_chan);
 
 	plx_dma_abort_desc(plxdev);
 
@@ -510,7 +510,7 @@ static int plx_dma_create(struct pci_dev *pdev)
 		goto free_plx;
 
 	spin_lock_init(&plxdev->ring_lock);
-	tasklet_setup(&plxdev->desc_task, plx_dma_desc_task);
+	dma_chan_init_bh(&plxdev->dma_chan, plx_dma_desc_task);
 
 	RCU_INIT_POINTER(plxdev->pdev, pdev);
 	plxdev->bar = pcim_iomap_table(pdev)[0];
-- 
2.43.0

[PATCH v2 29/64] dmaengine: sf-pdma: route error callbacks through channel BH

[Allen Pais]

virt-dma now dispatches completion callbacks through per-channel BH work,
but SF-PDMA invokes the client callback directly from its error tasklet
after exhausting its retries.

Complete failed descriptors through virt-dma with an aborted result and
the remaining transfer size, then start the next pending transfer. During
removal, kill the producer tasklets before the channel BH so they cannot
enqueue callback work after it has been canceled.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/sf-pdma/sf-pdma.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/drivers/dma/sf-pdma/sf-pdma.c b/drivers/dma/sf-pdma/sf-pdma.c
index 6f79cc28703e..a6c2f53f005e 100644
--- a/drivers/dma/sf-pdma/sf-pdma.c
+++ b/drivers/dma/sf-pdma/sf-pdma.c
@@ -328,7 +328,17 @@ static void sf_pdma_errbh_tasklet(struct tasklet_struct *t)
 	if (chan->retries <= 0) {
 		/* fail to recover */
 		spin_unlock_irqrestore(&chan->lock, flags);
-		dmaengine_desc_get_callback_invoke(desc->async_tx, NULL);
+
+		spin_lock_irqsave(&chan->vchan.lock, flags);
+		list_del(&desc->vdesc.node);
+		desc->vdesc.tx_result.result = DMA_TRANS_ABORTED;
+		desc->vdesc.tx_result.residue = desc->xfer_size;
+		vchan_cookie_complete(&desc->vdesc);
+
+		chan->desc = sf_pdma_get_first_pending_desc(chan);
+		if (chan->desc)
+			sf_pdma_xfer_desc(chan);
+		spin_unlock_irqrestore(&chan->vchan.lock, flags);
 	} else {
 		/* retry */
 		chan->retries--;
@@ -602,9 +612,9 @@ static void sf_pdma_remove(struct platform_device *pdev)
 		devm_free_irq(&pdev->dev, ch->txirq, ch);
 		devm_free_irq(&pdev->dev, ch->errirq, ch);
 		list_del(&ch->vchan.chan.device_node);
-		tasklet_kill(&ch->vchan.task);
 		tasklet_kill(&ch->done_tasklet);
 		tasklet_kill(&ch->err_tasklet);
+		dma_chan_kill_bh(&ch->vchan.chan);
 	}
 
 	if (pdev->dev.of_node)
-- 
2.43.0

[PATCH v2 30/64] dmaengine: sa11x0-dma: kill vchan BH on remove

[Allen Pais]

Use dma_chan_kill_bh() when tearing down virt-dma channels.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/sa11x0-dma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/sa11x0-dma.c b/drivers/dma/sa11x0-dma.c
index a6fa431530e3..ea4357da5e39 100644
--- a/drivers/dma/sa11x0-dma.c
+++ b/drivers/dma/sa11x0-dma.c
@@ -891,7 +891,7 @@ static void sa11x0_dma_free_channels(struct dma_device *dmadev)
 
 	list_for_each_entry_safe(c, cn, &dmadev->channels, vc.chan.device_node) {
 		list_del(&c->vc.chan.device_node);
-		tasklet_kill(&c->vc.task);
+		dma_chan_kill_bh(&c->vc.chan);
 		kfree(c);
 	}
 }
-- 
2.43.0

[PATCH v2 31/64] dmaengine: pl330: route callbacks via channel BH

[Allen Pais]

Convert the per-channel tasklet to dma_chan_*_bh while keeping the
controller tasklet for non-callback events.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/pl330.c | 18 ++++++++----------
 1 file changed, 8 insertions(+), 10 deletions(-)

diff --git a/drivers/dma/pl330.c b/drivers/dma/pl330.c
index 25ba84b18704..f4940bd1ab84 100644
--- a/drivers/dma/pl330.c
+++ b/drivers/dma/pl330.c
@@ -417,9 +417,6 @@ enum desc_status {
 };
 
 struct dma_pl330_chan {
-	/* Schedule desc completion */
-	struct tasklet_struct task;
-
 	/* DMA-Engine Channel */
 	struct dma_chan chan;
 
@@ -1574,7 +1571,7 @@ static void dma_pl330_rqcb(struct dma_pl330_desc *desc, enum pl330_op_err err)
 
 	spin_unlock_irqrestore(&pch->lock, flags);
 
-	tasklet_schedule(&pch->task);
+	dma_chan_schedule_bh(&pch->chan);
 }
 
 static void pl330_dotask(struct tasklet_struct *t)
@@ -2060,14 +2057,15 @@ static inline void fill_queue(struct dma_pl330_chan *pch)
 			desc->status = DONE;
 			dev_err(pch->dmac->ddma.dev, "%s:%d Bad Desc(%d)\n",
 					__func__, __LINE__, desc->txd.cookie);
-			tasklet_schedule(&pch->task);
+			dma_chan_schedule_bh(&pch->chan);
 		}
 	}
 }
 
-static void pl330_tasklet(struct tasklet_struct *t)
+static void pl330_tasklet(struct dma_chan *chan)
 {
-	struct dma_pl330_chan *pch = from_tasklet(pch, t, task);
+	struct dma_pl330_chan *pch = container_of(chan, struct dma_pl330_chan,
+						  chan);
 	struct dma_pl330_desc *desc, *_dt;
 	unsigned long flags;
 	bool power_down = false;
@@ -2173,7 +2171,7 @@ static int pl330_alloc_chan_resources(struct dma_chan *chan)
 		return -ENOMEM;
 	}
 
-	tasklet_setup(&pch->task, pl330_tasklet);
+	dma_chan_init_bh(&pch->chan, pl330_tasklet);
 
 	spin_unlock_irqrestore(&pl330->lock, flags);
 
@@ -2354,7 +2352,7 @@ static void pl330_free_chan_resources(struct dma_chan *chan)
 	struct pl330_dmac *pl330 = pch->dmac;
 	unsigned long flags;
 
-	tasklet_kill(&pch->task);
+	dma_chan_kill_bh(&pch->chan);
 
 	pm_runtime_get_sync(pch->dmac->ddma.dev);
 	spin_lock_irqsave(&pl330->lock, flags);
@@ -2490,7 +2488,7 @@ static void pl330_issue_pending(struct dma_chan *chan)
 	list_splice_tail_init(&pch->submitted_list, &pch->work_list);
 	spin_unlock_irqrestore(&pch->lock, flags);
 
-	pl330_tasklet(&pch->task);
+	pl330_tasklet(&pch->chan);
 }
 
 /*
-- 
2.43.0

[PATCH v2 31/64] dmaengine: pl330: route callbacks via channel BH

[Allen Pais]

Convert the per-channel tasklet to dma_chan_*_bh while keeping the
controller tasklet for non-callback events.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/pl330.c | 18 ++++++++----------
 1 file changed, 8 insertions(+), 10 deletions(-)

diff --git a/drivers/dma/pl330.c b/drivers/dma/pl330.c
index 25ba84b18704..f4940bd1ab84 100644
--- a/drivers/dma/pl330.c
+++ b/drivers/dma/pl330.c
@@ -417,9 +417,6 @@ enum desc_status {
 };
 
 struct dma_pl330_chan {
-	/* Schedule desc completion */
-	struct tasklet_struct task;
-
 	/* DMA-Engine Channel */
 	struct dma_chan chan;
 
@@ -1574,7 +1571,7 @@ static void dma_pl330_rqcb(struct dma_pl330_desc *desc, enum pl330_op_err err)
 
 	spin_unlock_irqrestore(&pch->lock, flags);
 
-	tasklet_schedule(&pch->task);
+	dma_chan_schedule_bh(&pch->chan);
 }
 
 static void pl330_dotask(struct tasklet_struct *t)
@@ -2060,14 +2057,15 @@ static inline void fill_queue(struct dma_pl330_chan *pch)
 			desc->status = DONE;
 			dev_err(pch->dmac->ddma.dev, "%s:%d Bad Desc(%d)\n",
 					__func__, __LINE__, desc->txd.cookie);
-			tasklet_schedule(&pch->task);
+			dma_chan_schedule_bh(&pch->chan);
 		}
 	}
 }
 
-static void pl330_tasklet(struct tasklet_struct *t)
+static void pl330_tasklet(struct dma_chan *chan)
 {
-	struct dma_pl330_chan *pch = from_tasklet(pch, t, task);
+	struct dma_pl330_chan *pch = container_of(chan, struct dma_pl330_chan,
+						  chan);
 	struct dma_pl330_desc *desc, *_dt;
 	unsigned long flags;
 	bool power_down = false;
@@ -2173,7 +2171,7 @@ static int pl330_alloc_chan_resources(struct dma_chan *chan)
 		return -ENOMEM;
 	}
 
-	tasklet_setup(&pch->task, pl330_tasklet);
+	dma_chan_init_bh(&pch->chan, pl330_tasklet);
 
 	spin_unlock_irqrestore(&pl330->lock, flags);
 
@@ -2354,7 +2352,7 @@ static void pl330_free_chan_resources(struct dma_chan *chan)
 	struct pl330_dmac *pl330 = pch->dmac;
 	unsigned long flags;
 
-	tasklet_kill(&pch->task);
+	dma_chan_kill_bh(&pch->chan);
 
 	pm_runtime_get_sync(pch->dmac->ddma.dev);
 	spin_lock_irqsave(&pl330->lock, flags);
@@ -2490,7 +2488,7 @@ static void pl330_issue_pending(struct dma_chan *chan)
 	list_splice_tail_init(&pch->submitted_list, &pch->work_list);
 	spin_unlock_irqrestore(&pch->lock, flags);
 
-	pl330_tasklet(&pch->task);
+	pl330_tasklet(&pch->chan);
 }
 
 /*
-- 
2.43.0

[PATCH v2 32/64] dmaengine: k3-udma: use channel BH for vchan completions

[Allen Pais]

Use the per-channel BH mechanism for custom vchan completion handling
and clean up via dma_chan_kill_bh().

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/ti/k3-udma.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/dma/ti/k3-udma.c b/drivers/dma/ti/k3-udma.c
index 1cf158eb7bdb..12b8a9caf4d2 100644
--- a/drivers/dma/ti/k3-udma.c
+++ b/drivers/dma/ti/k3-udma.c
@@ -4042,12 +4042,12 @@ static void udma_desc_pre_callback(struct virt_dma_chan *vc,
 }
 
 /*
- * This tasklet handles the completion of a DMA descriptor by
+ * This BH handles the completion of a DMA descriptor by
  * calling its callback and freeing it.
  */
-static void udma_vchan_complete(struct tasklet_struct *t)
+static void udma_vchan_complete(struct dma_chan *chan)
 {
-	struct virt_dma_chan *vc = from_tasklet(vc, t, task);
+	struct virt_dma_chan *vc = to_virt_chan(chan);
 	struct virt_dma_desc *vd, *_vd;
 	struct dmaengine_desc_callback cb;
 	LIST_HEAD(head);
@@ -4112,7 +4112,7 @@ static void udma_free_chan_resources(struct dma_chan *chan)
 	}
 
 	vchan_free_chan_resources(&uc->vc);
-	tasklet_kill(&uc->vc.task);
+	dma_chan_kill_bh(&uc->vc.chan);
 
 	bcdma_free_bchan_resources(uc);
 	udma_free_tx_resources(uc);
@@ -5627,8 +5627,8 @@ static int udma_probe(struct platform_device *pdev)
 		if (!uc->name)
 			return -ENOMEM;
 		vchan_init(&uc->vc, &ud->ddev);
-		/* Use custom vchan completion handling */
-		tasklet_setup(&uc->vc.task, udma_vchan_complete);
+		/* Override the default vchan completion handler */
+		uc->vc.chan.bh_work_fn = udma_vchan_complete;
 		init_completion(&uc->teardown_completed);
 		INIT_DELAYED_WORK(&uc->tx_drain.work, udma_check_tx_completion);
 	}
-- 
2.43.0

[PATCH v2 33/64] dmaengine: sun6i: kill vchan BH on teardown

[Allen Pais]

Use dma_chan_kill_bh() for virt-dma channel cleanup.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/sun6i-dma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/sun6i-dma.c b/drivers/dma/sun6i-dma.c
index a9a254dbf8cb..9289c24dfcfa 100644
--- a/drivers/dma/sun6i-dma.c
+++ b/drivers/dma/sun6i-dma.c
@@ -1097,7 +1097,7 @@ static inline void sun6i_dma_free(struct sun6i_dma_dev *sdev)
 		struct sun6i_vchan *vchan = &sdev->vchans[i];
 
 		list_del(&vchan->vc.chan.device_node);
-		tasklet_kill(&vchan->vc.task);
+		dma_chan_kill_bh(&vchan->vc.chan);
 	}
 }
 
-- 
2.43.0

[PATCH v2 34/64] dmaengine: mtk-cqdma: kill vchan BH on teardown

[Allen Pais]

Use dma_chan_kill_bh() for virt-dma channel cleanup.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/mediatek/mtk-cqdma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/mediatek/mtk-cqdma.c b/drivers/dma/mediatek/mtk-cqdma.c
index 80791e30aec2..3b0f1d8fa205 100644
--- a/drivers/dma/mediatek/mtk-cqdma.c
+++ b/drivers/dma/mediatek/mtk-cqdma.c
@@ -895,7 +895,7 @@ static void mtk_cqdma_remove(struct platform_device *pdev)
 		vc = &cqdma->vc[i];
 
 		list_del(&vc->vc.chan.device_node);
-		tasklet_kill(&vc->vc.task);
+		dma_chan_kill_bh(&vc->vc.chan);
 	}
 
 	/* disable interrupt */
-- 
2.43.0

Re: [PATCH v2 28/64] dmaengine: plx_dma: use channel BH helpers

[Logan Gunthorpe]

On 2026-07-27 14:28, Allen Pais wrote:
> Replace the device tasklet with the per-channel BH helper so completion
> callbacks run via dma_chan_*_bh.
> 
> Signed-off-by: Allen Pais <allen.lkml@gmail.com>

Looks good to me, thanks.

Reviewed-by: Logan Gunthorpe <logang@deltatee.com>

Re: [PATCH v2 09/64] dmaengine: ioat: convert cleanup tasklet to dma_chan BH

[Dave Jiang]

On 7/27/26 1:28 PM, Allen Pais wrote:
> Replace the per-channel tasklet with the shared dma_chan BH helper.
> The handler continues to run in softirq context while dmaengine owns
> the common scheduling and teardown mechanism.
> 
> Signed-off-by: Allen Pais <allen.lkml@gmail.com>

Acked-by: Dave Jiang <dave.jiang@intel.com>


> ---
>  drivers/dma/ioat/dma.c  | 14 +++++++-------
>  drivers/dma/ioat/dma.h  |  3 +--
>  drivers/dma/ioat/init.c |  2 +-
>  3 files changed, 9 insertions(+), 10 deletions(-)
> 
> diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c
> index ee93b029f9e3..827b46c6354a 100644
> --- a/drivers/dma/ioat/dma.c
> +++ b/drivers/dma/ioat/dma.c
> @@ -110,7 +110,7 @@ irqreturn_t ioat_dma_do_interrupt(int irq, void *data)
>  	for_each_set_bit(bit, &attnstatus, BITS_PER_LONG) {
>  		ioat_chan = ioat_chan_by_index(instance, bit);
>  		if (test_bit(IOAT_RUN, &ioat_chan->state))
> -			tasklet_schedule(&ioat_chan->cleanup_task);
> +			dma_chan_schedule_bh(&ioat_chan->dma_chan);
>  	}
>  
>  	writeb(intrctrl, instance->reg_base + IOAT_INTRCTRL_OFFSET);
> @@ -127,7 +127,7 @@ irqreturn_t ioat_dma_do_interrupt_msix(int irq, void *data)
>  	struct ioatdma_chan *ioat_chan = data;
>  
>  	if (test_bit(IOAT_RUN, &ioat_chan->state))
> -		tasklet_schedule(&ioat_chan->cleanup_task);
> +		dma_chan_schedule_bh(&ioat_chan->dma_chan);
>  
>  	return IRQ_HANDLED;
>  }
> @@ -161,11 +161,11 @@ void ioat_stop(struct ioatdma_chan *ioat_chan)
>  	/* flush inflight timers */
>  	timer_delete_sync(&ioat_chan->timer);
>  
> -	/* flush inflight tasklet runs */
> -	tasklet_kill(&ioat_chan->cleanup_task);
> +	/* flush inflight BH runs */
> +	dma_chan_kill_bh(&ioat_chan->dma_chan);
>  
>  	/* final cleanup now that everything is quiesced and can't re-arm */
> -	ioat_cleanup_event(&ioat_chan->cleanup_task);
> +	ioat_cleanup_event(&ioat_chan->dma_chan);
>  }
>  
>  static void __ioat_issue_pending(struct ioatdma_chan *ioat_chan)
> @@ -690,9 +690,9 @@ static void ioat_cleanup(struct ioatdma_chan *ioat_chan)
>  	spin_unlock_bh(&ioat_chan->cleanup_lock);
>  }
>  
> -void ioat_cleanup_event(struct tasklet_struct *t)
> +void ioat_cleanup_event(struct dma_chan *c)
>  {
> -	struct ioatdma_chan *ioat_chan = from_tasklet(ioat_chan, t, cleanup_task);
> +	struct ioatdma_chan *ioat_chan = to_ioat_chan(c);
>  
>  	ioat_cleanup(ioat_chan);
>  	if (!test_bit(IOAT_RUN, &ioat_chan->state))
> diff --git a/drivers/dma/ioat/dma.h b/drivers/dma/ioat/dma.h
> index e8a880f338c6..4bdb1ae937e4 100644
> --- a/drivers/dma/ioat/dma.h
> +++ b/drivers/dma/ioat/dma.h
> @@ -111,7 +111,6 @@ struct ioatdma_chan {
>  	struct ioatdma_device *ioat_dma;
>  	dma_addr_t completion_dma;
>  	u64 *completion;
> -	struct tasklet_struct cleanup_task;
>  	struct kobject kobj;
>  
>  /* ioat v2 / v3 channel attributes
> @@ -385,7 +384,7 @@ int ioat_reset_hw(struct ioatdma_chan *ioat_chan);
>  enum dma_status
>  ioat_tx_status(struct dma_chan *c, dma_cookie_t cookie,
>  		struct dma_tx_state *txstate);
> -void ioat_cleanup_event(struct tasklet_struct *t);
> +void ioat_cleanup_event(struct dma_chan *c);
>  void ioat_timer_event(struct timer_list *t);
>  int ioat_check_space_lock(struct ioatdma_chan *ioat_chan, int num_descs);
>  void ioat_issue_pending(struct dma_chan *chan);
> diff --git a/drivers/dma/ioat/init.c b/drivers/dma/ioat/init.c
> index 737496391109..314676e1cbc4 100644
> --- a/drivers/dma/ioat/init.c
> +++ b/drivers/dma/ioat/init.c
> @@ -765,7 +765,7 @@ ioat_init_channel(struct ioatdma_device *ioat_dma,
>  	list_add_tail(&ioat_chan->dma_chan.device_node, &dma->channels);
>  	ioat_dma->idx[idx] = ioat_chan;
>  	timer_setup(&ioat_chan->timer, ioat_timer_event, 0);
> -	tasklet_setup(&ioat_chan->cleanup_task, ioat_cleanup_event);
> +	dma_chan_init_bh(&ioat_chan->dma_chan, ioat_cleanup_event);
>  }
>  
>  #define IOAT_NUM_SRC_TEST 6 /* must be <= 8 */

[PATCH v2 35/64] dmaengine: altera-msgdma: use channel BH helpers

[Allen Pais]

Replace the device tasklet with the per-channel BH helper so completion
handling runs via dma_chan_*_bh.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/altera-msgdma.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/drivers/dma/altera-msgdma.c b/drivers/dma/altera-msgdma.c
index e23e5b441a24..5969bbc84ab9 100644
--- a/drivers/dma/altera-msgdma.c
+++ b/drivers/dma/altera-msgdma.c
@@ -170,7 +170,6 @@ struct msgdma_sw_desc {
 struct msgdma_device {
 	spinlock_t lock;
 	struct device *dev;
-	struct tasklet_struct irq_tasklet;
 	struct list_head pending_list;
 	struct list_head free_list;
 	struct list_head active_list;
@@ -684,11 +683,12 @@ static int msgdma_alloc_chan_resources(struct dma_chan *dchan)
 
 /**
  * msgdma_tasklet - Schedule completion tasklet
- * @t: Pointer to the Altera sSGDMA channel structure
+ * @chan: Pointer to the Altera sSGDMA channel structure
  */
-static void msgdma_tasklet(struct tasklet_struct *t)
+static void msgdma_tasklet(struct dma_chan *chan)
 {
-	struct msgdma_device *mdev = from_tasklet(mdev, t, irq_tasklet);
+	struct msgdma_device *mdev = container_of(chan, struct msgdma_device,
+						  dmachan);
 	u32 count;
 	u32 __maybe_unused size;
 	u32 __maybe_unused status;
@@ -748,7 +748,7 @@ static irqreturn_t msgdma_irq_handler(int irq, void *data)
 		spin_unlock(&mdev->lock);
 	}
 
-	tasklet_schedule(&mdev->irq_tasklet);
+	dma_chan_schedule_bh(&mdev->dmachan);
 
 	/* Clear interrupt in mSGDMA controller */
 	iowrite32(MSGDMA_CSR_STAT_IRQ, mdev->csr + MSGDMA_CSR_STATUS);
@@ -766,7 +766,7 @@ static void msgdma_dev_remove(struct msgdma_device *mdev)
 		return;
 
 	devm_free_irq(mdev->dev, mdev->irq, mdev);
-	tasklet_kill(&mdev->irq_tasklet);
+	dma_chan_kill_bh(&mdev->dmachan);
 	list_del(&mdev->dmachan.device_node);
 }
 
@@ -852,7 +852,7 @@ static int msgdma_probe(struct platform_device *pdev)
 	if (ret)
 		return ret;
 
-	tasklet_setup(&mdev->irq_tasklet, msgdma_tasklet);
+	dma_chan_init_bh(&mdev->dmachan, msgdma_tasklet);
 
 	dma_cookie_init(&mdev->dmachan);
 
-- 
2.43.0

[PATCH v2 36/64] dmaengine: sprd-dma: kill vchan BH on teardown

[Allen Pais]

Use dma_chan_kill_bh() for virt-dma cleanup.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/sprd-dma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/sprd-dma.c b/drivers/dma/sprd-dma.c
index 087fea3af2e4..42a0c1c34337 100644
--- a/drivers/dma/sprd-dma.c
+++ b/drivers/dma/sprd-dma.c
@@ -1253,7 +1253,7 @@ static void sprd_dma_remove(struct platform_device *pdev)
 	list_for_each_entry_safe(c, cn, &sdev->dma_dev.channels,
 				 vc.chan.device_node) {
 		list_del(&c->vc.chan.device_node);
-		tasklet_kill(&c->vc.task);
+		dma_chan_kill_bh(&c->vc.chan);
 	}
 
 	of_dma_controller_free(pdev->dev.of_node);
-- 
2.43.0

[PATCH v2 37/64] dmaengine: idma64: kill vchan BH on teardown

[Allen Pais]

Use dma_chan_kill_bh() for virt-dma cleanup.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/idma64.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/dma/idma64.c b/drivers/dma/idma64.c
index 5fcd1befc92d..8e25d828e553 100644
--- a/drivers/dma/idma64.c
+++ b/drivers/dma/idma64.c
@@ -617,14 +617,14 @@ static void idma64_remove(struct idma64_chip *chip)
 
 	/*
 	 * Explicitly call devm_request_irq() to avoid the side effects with
-	 * the scheduled tasklets.
+	 * scheduled BH work.
 	 */
 	devm_free_irq(chip->dev, chip->irq, idma64);
 
 	for (i = 0; i < idma64->dma.chancnt; i++) {
 		struct idma64_chan *idma64c = &idma64->chan[i];
 
-		tasklet_kill(&idma64c->vchan.task);
+		dma_chan_kill_bh(&idma64c->vchan.chan);
 	}
 }
 
-- 
2.43.0

[PATCH v2 38/64] dmaengine: img-mdc-dma: kill vchan BH on teardown

[Allen Pais]

Use dma_chan_kill_bh() for virt-dma cleanup.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/img-mdc-dma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/img-mdc-dma.c b/drivers/dma/img-mdc-dma.c
index b3765ba15803..374034c737c0 100644
--- a/drivers/dma/img-mdc-dma.c
+++ b/drivers/dma/img-mdc-dma.c
@@ -1031,7 +1031,7 @@ static void mdc_dma_remove(struct platform_device *pdev)
 
 		devm_free_irq(&pdev->dev, mchan->irq, mchan);
 
-		tasklet_kill(&mchan->vc.task);
+		dma_chan_kill_bh(&mchan->vc.chan);
 	}
 
 	pm_runtime_disable(&pdev->dev);
-- 
2.43.0

[PATCH v2 39/64] dmaengine: fsl-edma-common: kill vchan BH on teardown

[Allen Pais]

Use dma_chan_kill_bh() for virt-dma cleanup.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/fsl-edma-common.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/fsl-edma-common.c b/drivers/dma/fsl-edma-common.c
index bb7531c456df..70b40e60c39a 100644
--- a/drivers/dma/fsl-edma-common.c
+++ b/drivers/dma/fsl-edma-common.c
@@ -915,7 +915,7 @@ void fsl_edma_cleanup_vchan(struct dma_device *dmadev)
 	list_for_each_entry_safe(chan, _chan,
 				&dmadev->channels, vchan.chan.device_node) {
 		list_del(&chan->vchan.chan.device_node);
-		tasklet_kill(&chan->vchan.task);
+		dma_chan_kill_bh(&chan->vchan.chan);
 	}
 }
 
-- 
2.43.0

[PATCH v2 40/64] dmaengine: dw-axi-dmac: kill vchan BH on teardown

[Allen Pais]

Use dma_chan_kill_bh() for virt-dma cleanup.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
index bcefaff03b5c..5a5d569b188d 100644
--- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
+++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
@@ -1663,7 +1663,7 @@ static void dw_remove(struct platform_device *pdev)
 	list_for_each_entry_safe(chan, _chan, &dw->dma.channels,
 				 vc.chan.device_node) {
 		list_del(&chan->vc.chan.device_node);
-		tasklet_kill(&chan->vc.task);
+		dma_chan_kill_bh(&chan->vc.chan);
 	}
 }
 
-- 
2.43.0

[PATCH v2 41/64] dmaengine: hsu: kill vchan BH on teardown

[Allen Pais]

Use dma_chan_kill_bh() for virt-dma cleanup.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/hsu/hsu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/hsu/hsu.c b/drivers/dma/hsu/hsu.c
index f62d60d7bc6b..1a7f9df735a4 100644
--- a/drivers/dma/hsu/hsu.c
+++ b/drivers/dma/hsu/hsu.c
@@ -500,7 +500,7 @@ int hsu_dma_remove(struct hsu_dma_chip *chip)
 	for (i = 0; i < hsu->nr_channels; i++) {
 		struct hsu_dma_chan *hsuc = &hsu->chan[i];
 
-		tasklet_kill(&hsuc->vchan.task);
+		dma_chan_kill_bh(&hsuc->vchan.chan);
 	}
 
 	return 0;
-- 
2.43.0

[PATCH v2 42/64] dmaengine: jz4780: kill vchan BH on teardown

[Allen Pais]

Use dma_chan_kill_bh() for virt-dma cleanup.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/dma-jz4780.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/dma-jz4780.c b/drivers/dma/dma-jz4780.c
index 6070dfdb7114..45c1096563e3 100644
--- a/drivers/dma/dma-jz4780.c
+++ b/drivers/dma/dma-jz4780.c
@@ -1019,7 +1019,7 @@ static void jz4780_dma_remove(struct platform_device *pdev)
 	free_irq(jzdma->irq, jzdma);
 
 	for (i = 0; i < jzdma->soc_data->nb_channels; i++)
-		tasklet_kill(&jzdma->chan[i].vchan.task);
+		dma_chan_kill_bh(&jzdma->chan[i].vchan.chan);
 }
 
 static const struct jz4780_dma_soc_data jz4740_dma_soc_data = {
-- 
2.43.0

[PATCH v2 43/64] dmaengine: pxa_dma: kill vchan BH on teardown

[Allen Pais]

Use dma_chan_kill_bh() for virt-dma cleanup.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/pxa_dma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/pxa_dma.c b/drivers/dma/pxa_dma.c
index fa2ee0b3e09f..d57e5e0c422c 100644
--- a/drivers/dma/pxa_dma.c
+++ b/drivers/dma/pxa_dma.c
@@ -1215,7 +1215,7 @@ static void pxad_free_channels(struct dma_device *dmadev)
 	list_for_each_entry_safe(c, cn, &dmadev->channels,
 				 vc.chan.device_node) {
 		list_del(&c->vc.chan.device_node);
-		tasklet_kill(&c->vc.task);
+		dma_chan_kill_bh(&c->vc.chan);
 	}
 }
 
-- 
2.43.0

[PATCH v2 44/64] dmaengine: mtk-hsdma: kill vchan BH on teardown

[Allen Pais]

Use dma_chan_kill_bh() for virt-dma cleanup.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/mediatek/mtk-hsdma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/mediatek/mtk-hsdma.c b/drivers/dma/mediatek/mtk-hsdma.c
index a43412ff5edd..1c98fcedf3ac 100644
--- a/drivers/dma/mediatek/mtk-hsdma.c
+++ b/drivers/dma/mediatek/mtk-hsdma.c
@@ -1020,7 +1020,7 @@ static void mtk_hsdma_remove(struct platform_device *pdev)
 		vc = &hsdma->vc[i];
 
 		list_del(&vc->vc.chan.device_node);
-		tasklet_kill(&vc->vc.task);
+		dma_chan_kill_bh(&vc->vc.chan);
 	}
 
 	/* Disable DMA interrupt */
-- 
2.43.0

[PATCH v2 45/64] dmaengine: mtk-uart-apdma: kill vchan BH on teardown

[Allen Pais]

Use dma_chan_kill_bh() for virt-dma cleanup.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/mediatek/mtk-uart-apdma.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/dma/mediatek/mtk-uart-apdma.c b/drivers/dma/mediatek/mtk-uart-apdma.c
index c269d84d7bd2..4d89927922f5 100644
--- a/drivers/dma/mediatek/mtk-uart-apdma.c
+++ b/drivers/dma/mediatek/mtk-uart-apdma.c
@@ -312,7 +312,7 @@ static void mtk_uart_apdma_free_chan_resources(struct dma_chan *chan)
 
 	free_irq(c->irq, chan);
 
-	tasklet_kill(&c->vc.task);
+	dma_chan_kill_bh(&c->vc.chan);
 
 	vchan_free_chan_resources(&c->vc);
 
@@ -463,7 +463,7 @@ static void mtk_uart_apdma_free(struct mtk_uart_apdmadev *mtkd)
 			struct mtk_chan, vc.chan.device_node);
 
 		list_del(&c->vc.chan.device_node);
-		tasklet_kill(&c->vc.task);
+		dma_chan_kill_bh(&c->vc.chan);
 	}
 }
 
-- 
2.43.0

[PATCH v2 46/64] dmaengine: imx-sdma: kill vchan BH on teardown

[Allen Pais]

Use dma_chan_kill_bh() for virt-dma cleanup.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/imx-sdma.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/dma/imx-sdma.c b/drivers/dma/imx-sdma.c
index 36368835a845..46f66e5f9f30 100644
--- a/drivers/dma/imx-sdma.c
+++ b/drivers/dma/imx-sdma.c
@@ -2399,11 +2399,11 @@ static void sdma_remove(struct platform_device *pdev)
 	int i;
 
 	devm_free_irq(&pdev->dev, sdma->irq, sdma);
-	/* Kill the tasklet */
+	/* Kill the channel BH */
 	for (i = 0; i < MAX_DMA_CHANNELS; i++) {
 		struct sdma_channel *sdmac = &sdma->channel[i];
 
-		tasklet_kill(&sdmac->vc.task);
+		dma_chan_kill_bh(&sdmac->vc.chan);
 		sdma_free_chan_resources(&sdmac->vc.chan);
 	}
 
-- 
2.43.0

[PATCH v2 47/64] dmaengine: loongson1-apb: kill vchan BH on teardown

[Allen Pais]

Use dma_chan_kill_bh() for virt-dma cleanup.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/loongson/loongson1-apb-dma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/loongson/loongson1-apb-dma.c b/drivers/dma/loongson/loongson1-apb-dma.c
index 89786cbd20ab..7fc77d8e6db3 100644
--- a/drivers/dma/loongson/loongson1-apb-dma.c
+++ b/drivers/dma/loongson/loongson1-apb-dma.c
@@ -552,7 +552,7 @@ static void ls1x_dma_chan_remove(struct ls1x_dma *dma)
 
 		if (chan->vc.chan.device == &dma->ddev) {
 			list_del(&chan->vc.chan.device_node);
-			tasklet_kill(&chan->vc.task);
+			dma_chan_kill_bh(&chan->vc.chan);
 		}
 	}
 }
-- 
2.43.0

[PATCH v2 48/64] dmaengine: owl-dma: kill vchan BH on teardown

[Allen Pais]

Use dma_chan_kill_bh() for virt-dma cleanup.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/owl-dma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/owl-dma.c b/drivers/dma/owl-dma.c
index 7c80572fc71d..5f2140dc01ce 100644
--- a/drivers/dma/owl-dma.c
+++ b/drivers/dma/owl-dma.c
@@ -1055,7 +1055,7 @@ static inline void owl_dma_free(struct owl_dma *od)
 	list_for_each_entry_safe(vchan,
 				 next, &od->dma.channels, vc.chan.device_node) {
 		list_del(&vchan->vc.chan.device_node);
-		tasklet_kill(&vchan->vc.task);
+		dma_chan_kill_bh(&vchan->vc.chan);
 	}
 }
 
-- 
2.43.0

[PATCH v2 49/64] dmaengine: hisi: kill vchan BH on teardown

[Allen Pais]

Use dma_chan_kill_bh() for virt-dma cleanup.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/hisi_dma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 28bf818f9aa6..39bc88c46513 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -720,7 +720,7 @@ static void hisi_dma_disable_qps(struct hisi_dma_dev *hdma_dev)
 
 	for (i = 0; i < hdma_dev->chan_num; i++) {
 		hisi_dma_disable_qp(hdma_dev, i);
-		tasklet_kill(&hdma_dev->chan[i].vc.task);
+		dma_chan_kill_bh(&hdma_dev->chan[i].vc.chan);
 	}
 }
 
-- 
2.43.0

[PATCH v2 50/64] dmaengine: dw-edma: kill vchan BH on teardown

[Allen Pais]

Use dma_chan_kill_bh() for virt-dma cleanup.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/dw-edma/dw-edma-core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-edma-core.c
index 89a4c498a17b..a47a4674df83 100644
--- a/drivers/dma/dw-edma/dw-edma-core.c
+++ b/drivers/dma/dw-edma/dw-edma-core.c
@@ -1170,7 +1170,7 @@ int dw_edma_remove(struct dw_edma_chip *chip)
 	dma_async_device_unregister(&dw->dma);
 	list_for_each_entry_safe(chan, _chan, &dw->dma.channels,
 				 vc.chan.device_node) {
-		tasklet_kill(&chan->vc.task);
+		dma_chan_kill_bh(&chan->vc.chan);
 		list_del(&chan->vc.chan.device_node);
 	}
 
-- 
2.43.0

[PATCH v2 51/64] dmaengine: bcm2835: kill vchan BH on teardown

[Allen Pais]

Use dma_chan_kill_bh() for virt-dma cleanup.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/bcm2835-dma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/bcm2835-dma.c b/drivers/dma/bcm2835-dma.c
index 06d830d36882..68e86c8d3d43 100644
--- a/drivers/dma/bcm2835-dma.c
+++ b/drivers/dma/bcm2835-dma.c
@@ -829,7 +829,7 @@ static void bcm2835_dma_free(struct bcm2835_dmadev *od)
 	list_for_each_entry_safe(c, next, &od->ddev.channels,
 				 vc.chan.device_node) {
 		list_del(&c->vc.chan.device_node);
-		tasklet_kill(&c->vc.task);
+		dma_chan_kill_bh(&c->vc.chan);
 	}
 
 	dma_unmap_page_attrs(od->ddev.dev, od->zero_page, PAGE_SIZE,
-- 
2.43.0

[PATCH v2 52/64] dmaengine: tegra210-adma: kill vchan BH on teardown

[Allen Pais]

Use dma_chan_kill_bh() for virt-dma cleanup.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/tegra210-adma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/tegra210-adma.c b/drivers/dma/tegra210-adma.c
index ceaee1e33e68..598abcc4edaa 100644
--- a/drivers/dma/tegra210-adma.c
+++ b/drivers/dma/tegra210-adma.c
@@ -811,7 +811,7 @@ static void tegra_adma_free_chan_resources(struct dma_chan *dc)
 
 	tegra_adma_terminate_all(dc);
 	vchan_free_chan_resources(&tdc->vc);
-	tasklet_kill(&tdc->vc.task);
+	dma_chan_kill_bh(&tdc->vc.chan);
 	free_irq(tdc->irq, tdc);
 	pm_runtime_put(tdc2dev(tdc));
 
-- 
2.43.0

[PATCH v2 53/64] dmaengine: fsl-qdma: use dma_chan_kill_bh

[Allen Pais]

virt-dma now dispatches completion callbacks through per-channel BH
work instead of its tasklet. Cancel that work during teardown before
channel storage is released.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/fsl-qdma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/fsl-qdma.c b/drivers/dma/fsl-qdma.c
index df843fad0ece..e0fb39fe821c 100644
--- a/drivers/dma/fsl-qdma.c
+++ b/drivers/dma/fsl-qdma.c
@@ -1255,7 +1255,7 @@ static void fsl_qdma_cleanup_vchan(struct dma_device *dmadev)
 	list_for_each_entry_safe(chan, _chan,
 				 &dmadev->channels, vchan.chan.device_node) {
 		list_del(&chan->vchan.chan.device_node);
-		tasklet_kill(&chan->vchan.task);
+		dma_chan_kill_bh(&chan->vchan.chan);
 	}
 }
 
-- 
2.43.0

[PATCH v2 54/64] dmaengine: st_fdma: use dma_chan_kill_bh

[Allen Pais]

virt-dma now dispatches completion callbacks through per-channel BH
work instead of its tasklet. Cancel that work during teardown before
channel storage is released.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/st_fdma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/st_fdma.c b/drivers/dma/st_fdma.c
index d9547017f3bd..d4cff851e1d5 100644
--- a/drivers/dma/st_fdma.c
+++ b/drivers/dma/st_fdma.c
@@ -733,7 +733,7 @@ static void st_fdma_free(struct st_fdma_dev *fdev)
 	for (i = 0; i < fdev->nr_channels; i++) {
 		fchan = &fdev->chans[i];
 		list_del(&fchan->vchan.chan.device_node);
-		tasklet_kill(&fchan->vchan.task);
+		dma_chan_kill_bh(&fchan->vchan.chan);
 	}
 }
 
-- 
2.43.0

[PATCH v2 55/64] dmaengine: dma-axi-dmac: use dma_chan_kill_bh

[Allen Pais]

virt-dma now dispatches completion callbacks through per-channel BH
work instead of its tasklet. Cancel that work during teardown before
channel storage is released.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/dma-axi-dmac.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/dma/dma-axi-dmac.c b/drivers/dma/dma-axi-dmac.c
index d47ff27e1408..d26bfa852806 100644
--- a/drivers/dma/dma-axi-dmac.c
+++ b/drivers/dma/dma-axi-dmac.c
@@ -1195,9 +1195,11 @@ static int axi_dmac_detect_caps(struct axi_dmac *dmac, unsigned int version)
 	return 0;
 }
 
-static void axi_dmac_tasklet_kill(void *task)
+static void axi_dmac_kill_bh(void *data)
 {
-	tasklet_kill(task);
+	struct dma_chan *chan = data;
+
+	dma_chan_kill_bh(chan);
 }
 
 static void axi_dmac_free_dma_controller(void *of_node)
@@ -1302,8 +1304,8 @@ static int axi_dmac_probe(struct platform_device *pdev)
 	 * Put the action in here so it get's done before unregistering the DMA
 	 * device.
 	 */
-	ret = devm_add_action_or_reset(&pdev->dev, axi_dmac_tasklet_kill,
-				       &dmac->chan.vchan.task);
+	ret = devm_add_action_or_reset(&pdev->dev, axi_dmac_kill_bh,
+				       &dmac->chan.vchan.chan);
 	if (ret)
 		return ret;
 
-- 
2.43.0

[PATCH v2 56/64] dmaengine: omap-dma: use dma_chan_kill_bh

[Allen Pais]

virt-dma now dispatches completion callbacks through per-channel BH
work instead of its tasklet. Cancel that work during teardown before
channel storage is released.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/ti/omap-dma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/ti/omap-dma.c b/drivers/dma/ti/omap-dma.c
index 55ece7fd0d99..10cdbcd12fe6 100644
--- a/drivers/dma/ti/omap-dma.c
+++ b/drivers/dma/ti/omap-dma.c
@@ -1521,7 +1521,7 @@ static void omap_dma_free(struct omap_dmadev *od)
 			struct omap_chan, vc.chan.device_node);
 
 		list_del(&c->vc.chan.device_node);
-		tasklet_kill(&c->vc.task);
+		dma_chan_kill_bh(&c->vc.chan);
 		kfree(c);
 	}
 }
-- 
2.43.0

[PATCH v2 57/64] dmaengine: edma: use dma_chan_kill_bh

[Allen Pais]

virt-dma now dispatches completion callbacks through per-channel BH
work instead of its tasklet. Cancel that work during teardown before
channel storage is released.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/ti/edma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/ti/edma.c b/drivers/dma/ti/edma.c
index d97db5af3555..c0b16cd24752 100644
--- a/drivers/dma/ti/edma.c
+++ b/drivers/dma/ti/edma.c
@@ -2560,7 +2560,7 @@ static void edma_cleanupp_vchan(struct dma_device *dmadev)
 	list_for_each_entry_safe(echan, _echan,
 			&dmadev->channels, vchan.chan.device_node) {
 		list_del(&echan->vchan.chan.device_node);
-		tasklet_kill(&echan->vchan.task);
+		dma_chan_kill_bh(&echan->vchan.chan);
 	}
 }
 
-- 
2.43.0

[PATCH v2 58/64] dmaengine: qcom-adm: use dma_chan_kill_bh

[Allen Pais]

virt-dma now dispatches completion callbacks through per-channel BH
work instead of its tasklet. Cancel that work during teardown before
channel storage is released.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/qcom/qcom_adm.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/dma/qcom/qcom_adm.c b/drivers/dma/qcom/qcom_adm.c
index 07fbe32d31fa..ea24e2de5154 100644
--- a/drivers/dma/qcom/qcom_adm.c
+++ b/drivers/dma/qcom/qcom_adm.c
@@ -918,8 +918,8 @@ static void adm_dma_remove(struct platform_device *pdev)
 		/* mask IRQs for this channel/EE pair */
 		writel(0, adev->regs + ADM_CH_RSLT_CONF(achan->id, adev->ee));
 
-		tasklet_kill(&adev->channels[i].vc.task);
-		adm_terminate_all(&adev->channels[i].vc.chan);
+		dma_chan_kill_bh(&achan->vc.chan);
+		adm_terminate_all(&achan->vc.chan);
 	}
 
 	devm_free_irq(adev->dev, adev->irq, adev);
-- 
2.43.0

[PATCH v2 59/64] dmaengine: tegra186-gpc: use dma_chan_kill_bh

[Allen Pais]

virt-dma now dispatches completion callbacks through per-channel BH
work instead of its tasklet. Cancel that work during teardown before
channel storage is released.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/tegra186-gpc-dma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/dma/tegra186-gpc-dma.c b/drivers/dma/tegra186-gpc-dma.c
index 64cedef1050a..1ba8897ec614 100644
--- a/drivers/dma/tegra186-gpc-dma.c
+++ b/drivers/dma/tegra186-gpc-dma.c
@@ -1284,7 +1284,7 @@ static void tegra_dma_free_chan_resources(struct dma_chan *dc)
 	tegra_dma_terminate_all(dc);
 	synchronize_irq(tdc->irq);
 
-	tasklet_kill(&tdc->vc.task);
+	dma_chan_kill_bh(&tdc->vc.chan);
 	tdc->config_init = false;
 	tdc->slave_id = -1;
 	tdc->sid_dir = DMA_TRANS_NONE;
-- 
2.43.0

[PATCH v2 60/64] dmaengine: bam-dma: use dma_chan_kill_bh

[Allen Pais]

virt-dma now dispatches completion callbacks through per-channel BH
work instead of its tasklet. Cancel that work during teardown before
channel storage is released.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/qcom/bam_dma.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/dma/qcom/bam_dma.c b/drivers/dma/qcom/bam_dma.c
index 1bb26af0405f..74bc2fe445b3 100644
--- a/drivers/dma/qcom/bam_dma.c
+++ b/drivers/dma/qcom/bam_dma.c
@@ -1387,7 +1387,7 @@ static int bam_dma_probe(struct platform_device *pdev)
 	dma_async_device_unregister(&bdev->common);
 err_bam_channel_exit:
 	for (i = 0; i < bdev->num_channels; i++)
-		tasklet_kill(&bdev->channels[i].vc.task);
+		dma_chan_kill_bh(&bdev->channels[i].vc.chan);
 err_tasklet_kill:
 	tasklet_kill(&bdev->task);
 err_disable_clk:
@@ -1413,7 +1413,7 @@ static void bam_dma_remove(struct platform_device *pdev)
 
 	for (i = 0; i < bdev->num_channels; i++) {
 		bam_dma_terminate_all(&bdev->channels[i].vc.chan);
-		tasklet_kill(&bdev->channels[i].vc.task);
+		dma_chan_kill_bh(&bdev->channels[i].vc.chan);
 
 		if (!bdev->channels[i].fifo_virt)
 			continue;
-- 
2.43.0

[PATCH v2 61/64] dmaengine: dw: defer callbacks via channel BH

[Allen Pais]

Queue completed descriptors on the channel and invoke their callbacks
from per-channel BH work instead of the controller tasklet. Add a
synchronize hook so termination can wait for deferred callbacks before
their descriptor storage is released.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/dw/core.c | 42 ++++++++++++++++++++++++++++++++++--------
 drivers/dma/dw/regs.h |  1 +
 2 files changed, 35 insertions(+), 8 deletions(-)

diff --git a/drivers/dma/dw/core.c b/drivers/dma/dw/core.c
index dd75f97a33b3..e780116a944b 100644
--- a/drivers/dma/dw/core.c
+++ b/drivers/dma/dw/core.c
@@ -236,25 +236,24 @@ dwc_descriptor_complete(struct dw_dma_chan *dwc, struct dw_desc *desc,
 	struct dma_async_tx_descriptor	*txd = &desc->txd;
 	struct dw_desc			*child;
 	unsigned long			flags;
-	struct dmaengine_desc_callback	cb;
 
 	dev_vdbg(chan2dev(&dwc->chan), "descriptor %u complete\n", txd->cookie);
 
 	spin_lock_irqsave(&dwc->lock, flags);
 	dma_cookie_complete(txd);
-	if (callback_required)
-		dmaengine_desc_get_callback(txd, &cb);
-	else
-		memset(&cb, 0, sizeof(cb));
+	list_del_init(&desc->desc_node);
 
 	/* async_tx_ack */
 	list_for_each_entry(child, &desc->tx_list, desc_node)
 		async_tx_ack(&child->txd);
 	async_tx_ack(&desc->txd);
-	dwc_desc_put(dwc, desc);
+	if (callback_required) {
+		list_add_tail(&desc->desc_node, &dwc->completed_list);
+		dma_chan_schedule_bh(&dwc->chan);
+	} else {
+		dwc_desc_put(dwc, desc);
+	}
 	spin_unlock_irqrestore(&dwc->lock, flags);
-
-	dmaengine_desc_callback_invoke(&cb, NULL);
 }
 
 static void dwc_complete_all(struct dw_dma *dw, struct dw_dma_chan *dwc)
@@ -285,6 +284,24 @@ static void dwc_complete_all(struct dw_dma *dw, struct dw_dma_chan *dwc)
 		dwc_descriptor_complete(dwc, desc, true);
 }
 
+static void dwc_chan_bh(struct dma_chan *chan)
+{
+	struct dw_dma_chan *dwc = to_dw_dma_chan(chan);
+	struct dw_desc *desc, *_desc;
+	struct dmaengine_desc_callback cb;
+	LIST_HEAD(list);
+
+	spin_lock_irq(&dwc->lock);
+	list_splice_init(&dwc->completed_list, &list);
+	spin_unlock_irq(&dwc->lock);
+
+	list_for_each_entry_safe(desc, _desc, &list, desc_node) {
+		dmaengine_desc_get_callback(&desc->txd, &cb);
+		dmaengine_desc_callback_invoke(&cb, NULL);
+		dwc_desc_put(dwc, desc);
+	}
+}
+
 /* Returns how many bytes were already received from source */
 static inline u32 dwc_get_sent(struct dw_dma_chan *dwc)
 {
@@ -965,6 +982,11 @@ static int dwc_terminate_all(struct dma_chan *chan)
 	return 0;
 }
 
+static void dwc_synchronize(struct dma_chan *chan)
+{
+	dma_chan_kill_bh(chan);
+}
+
 static struct dw_desc *dwc_find_desc(struct dw_dma_chan *dwc, dma_cookie_t c)
 {
 	struct dw_desc *desc;
@@ -1256,6 +1278,8 @@ int do_dma_probe(struct dw_dma_chip *chip)
 
 		INIT_LIST_HEAD(&dwc->active_list);
 		INIT_LIST_HEAD(&dwc->queue);
+		INIT_LIST_HEAD(&dwc->completed_list);
+		dma_chan_init_bh(&dwc->chan, dwc_chan_bh);
 
 		channel_clear_bit(dw, CH_EN, dwc->mask);
 
@@ -1321,6 +1345,7 @@ int do_dma_probe(struct dw_dma_chip *chip)
 	dw->dma.device_pause = dwc_pause;
 	dw->dma.device_resume = dwc_resume;
 	dw->dma.device_terminate_all = dwc_terminate_all;
+	dw->dma.device_synchronize = dwc_synchronize;
 
 	dw->dma.device_tx_status = dwc_tx_status;
 	dw->dma.device_issue_pending = dwc_issue_pending;
@@ -1374,6 +1399,7 @@ int do_dma_remove(struct dw_dma_chip *chip)
 
 	list_for_each_entry_safe(dwc, _dwc, &dw->dma.channels,
 			chan.device_node) {
+		dma_chan_kill_bh(&dwc->chan);
 		list_del(&dwc->chan.device_node);
 		channel_clear_bit(dw, CH_EN, dwc->mask);
 	}
diff --git a/drivers/dma/dw/regs.h b/drivers/dma/dw/regs.h
index 5969d9cc8d7a..c9e32bd6110a 100644
--- a/drivers/dma/dw/regs.h
+++ b/drivers/dma/dw/regs.h
@@ -279,6 +279,7 @@ struct dw_dma_chan {
 	unsigned long		flags;
 	struct list_head	active_list;
 	struct list_head	queue;
+	struct list_head	completed_list;
 
 	unsigned int		descs_allocated;
 
-- 
2.43.0

[PATCH v2 62/64] dmaengine: hidma: defer callbacks via channel BH

[Allen Pais]

Move descriptor callback processing out of the low-level completion
path and schedule it through per-channel BH work. Drain that work while
freeing channels so callbacks cannot outlive channel storage.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/qcom/hidma.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/drivers/dma/qcom/hidma.c b/drivers/dma/qcom/hidma.c
index c939635be21d..1f875a499ab3 100644
--- a/drivers/dma/qcom/hidma.c
+++ b/drivers/dma/qcom/hidma.c
@@ -90,7 +90,13 @@ static inline struct hidma_chan *to_hidma_chan(struct dma_chan *dmach)
 
 static void hidma_free(struct hidma_dev *dmadev)
 {
-	INIT_LIST_HEAD(&dmadev->ddev.channels);
+	struct hidma_chan *mchan, *next;
+
+	list_for_each_entry_safe(mchan, next, &dmadev->ddev.channels,
+				 chan.device_node) {
+		dma_chan_kill_bh(&mchan->chan);
+		list_del(&mchan->chan.device_node);
+	}
 }
 
 static unsigned int nr_desc_prm;
@@ -155,6 +161,13 @@ static void hidma_process_completed(struct hidma_chan *mchan)
 	}
 }
 
+static void hidma_chan_bh(struct dma_chan *chan)
+{
+	struct hidma_chan *mchan = to_hidma_chan(chan);
+
+	hidma_process_completed(mchan);
+}
+
 /*
  * Called once for each submitted descriptor.
  * PM is locked once for each descriptor that is currently
@@ -181,7 +194,7 @@ static void hidma_callback(void *data)
 	}
 	spin_unlock_irqrestore(&mchan->lock, irqflags);
 
-	hidma_process_completed(mchan);
+	dma_chan_schedule_bh(&mchan->chan);
 
 	if (queued) {
 		pm_runtime_mark_last_busy(dmadev->ddev.dev);
@@ -203,6 +216,7 @@ static int hidma_chan_init(struct hidma_dev *dmadev, u32 dma_sig)
 	mchan->dmadev = dmadev;
 	mchan->chan.device = ddev;
 	dma_cookie_init(&mchan->chan);
+	dma_chan_init_bh(&mchan->chan, hidma_chan_bh);
 
 	INIT_LIST_HEAD(&mchan->free);
 	INIT_LIST_HEAD(&mchan->prepared);
-- 
2.43.0

[PATCH v2 63/64] dmaengine: qcom-gpi: defer callbacks via vchan

[Allen Pais]

Complete descriptors through virt-dma instead of invoking callbacks
directly from event processing. This preserves the transfer result for
the shared vchan BH, which invokes the callback and frees the descriptor;
drain that BH before releasing channel resources.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/qcom/gpi.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/drivers/dma/qcom/gpi.c b/drivers/dma/qcom/gpi.c
index a5055a6273af..4cf68cf8eb2e 100644
--- a/drivers/dma/qcom/gpi.c
+++ b/drivers/dma/qcom/gpi.c
@@ -981,8 +981,12 @@ static void gpi_process_imed_data_event(struct gchan *gchan,
 		result.result = DMA_TRANS_NOERROR;
 	result.residue = gpi_desc->len - imed_event->length;
 
-	dma_cookie_complete(&vd->tx);
-	dmaengine_desc_get_callback_invoke(&vd->tx, &result);
+	spin_lock_irqsave(&gchan->vc.lock, flags);
+	list_del(&vd->node);
+	vd->tx_result = result;
+	vchan_cookie_complete(vd);
+	spin_unlock_irqrestore(&gchan->vc.lock, flags);
+	return;
 
 gpi_free_desc:
 	spin_lock_irqsave(&gchan->vc.lock, flags);
@@ -1060,8 +1064,12 @@ static void gpi_process_xfer_compl_event(struct gchan *gchan,
 	result.residue = gpi_desc->len - compl_event->length;
 	dev_dbg(gpii->gpi_dev->dev, "Residue %d\n", result.residue);
 
-	dma_cookie_complete(&vd->tx);
-	dmaengine_desc_get_callback_invoke(&vd->tx, &result);
+	spin_lock_irqsave(&gchan->vc.lock, flags);
+	list_del(&vd->node);
+	vd->tx_result = result;
+	vchan_cookie_complete(vd);
+	spin_unlock_irqrestore(&gchan->vc.lock, flags);
+	return;
 
 gpi_free_desc:
 	spin_lock_irqsave(&gchan->vc.lock, flags);
@@ -2009,6 +2017,7 @@ static void gpi_free_chan_resources(struct dma_chan *chan)
 
 	/* free all allocated memory */
 	gpi_free_ring(&gchan->ch_ring, gpii);
+	dma_chan_kill_bh(&gchan->vc.chan);
 	vchan_free_chan_resources(&gchan->vc);
 	kfree(gchan->config);
 
-- 
2.43.0

[PATCH v2 64/64] dmaengine: switchtec: use channel BH helpers

[Allen Pais]

Replace the per-channel descriptor completion tasklet with the shared
dma_chan BH helpers. Client callbacks remain in softirq context while
using the common scheduling and teardown mechanism.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>
---
 drivers/dma/switchtec_dma.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/dma/switchtec_dma.c b/drivers/dma/switchtec_dma.c
index 3ef928640615..f3459799e05d 100644
--- a/drivers/dma/switchtec_dma.c
+++ b/drivers/dma/switchtec_dma.c
@@ -133,8 +133,6 @@ struct switchtec_dma_chan {
 	/* Serialize hardware control register access */
 	spinlock_t hw_ctrl_lock;
 
-	struct tasklet_struct desc_task;
-
 	/* Serialize descriptor preparation */
 	spinlock_t submit_lock;
 	bool ring_active;
@@ -584,6 +582,7 @@ static void switchtec_dma_synchronize(struct dma_chan *chan)
 
 	int rc;
 
+	dma_chan_kill_bh(chan);
 	switchtec_dma_abort_desc(swdma_chan, 1);
 
 	rc = enable_channel(swdma_chan);
@@ -820,9 +819,10 @@ static int switchtec_dma_resume(struct dma_chan *chan)
 	return ret;
 }
 
-static void switchtec_dma_desc_task(unsigned long data)
+static void switchtec_dma_desc_task(struct dma_chan *chan)
 {
-	struct switchtec_dma_chan *swdma_chan = (void *)data;
+	struct switchtec_dma_chan *swdma_chan =
+		container_of(chan, struct switchtec_dma_chan, dma_chan);
 
 	switchtec_dma_cleanup_completed(swdma_chan);
 }
@@ -832,7 +832,7 @@ static irqreturn_t switchtec_dma_isr(int irq, void *chan)
 	struct switchtec_dma_chan *swdma_chan = chan;
 
 	if (swdma_chan->comp_ring_active)
-		tasklet_schedule(&swdma_chan->desc_task);
+		dma_chan_schedule_bh(&swdma_chan->dma_chan);
 
 	return IRQ_HANDLED;
 }
@@ -1041,6 +1041,7 @@ static void switchtec_dma_free_chan_resources(struct dma_chan *chan)
 	spin_unlock_bh(&swdma_chan->complete_lock);
 
 	switchtec_dma_chan_stop(swdma_chan);
+	dma_chan_kill_bh(chan);
 	switchtec_dma_abort_desc(swdma_chan, 0);
 	switchtec_dma_free_desc(swdma_chan);
 
@@ -1067,8 +1068,6 @@ static int switchtec_dma_chan_init(struct switchtec_dma_dev *swdma_dev,
 	spin_lock_init(&swdma_chan->hw_ctrl_lock);
 	spin_lock_init(&swdma_chan->submit_lock);
 	spin_lock_init(&swdma_chan->complete_lock);
-	tasklet_init(&swdma_chan->desc_task, switchtec_dma_desc_task,
-		     (unsigned long)swdma_chan);
 
 	swdma_chan->mmio_chan_fw =
 		swdma_dev->bar + SWITCHTEC_DMAC_CHAN_CFG_STS_OFFSET +
@@ -1116,6 +1115,7 @@ static int switchtec_dma_chan_init(struct switchtec_dma_dev *swdma_dev,
 	chan = &swdma_chan->dma_chan;
 	chan->device = dma;
 	dma_cookie_init(chan);
+	dma_chan_init_bh(chan, switchtec_dma_desc_task);
 
 	list_add_tail(&chan->device_node, &dma->channels);
 
@@ -1138,7 +1138,7 @@ static int switchtec_dma_chan_free(struct pci_dev *pdev,
 	spin_unlock_bh(&swdma_chan->complete_lock);
 
 	pci_free_irq(pdev, swdma_chan->irq, swdma_chan);
-	tasklet_kill(&swdma_chan->desc_task);
+	dma_chan_kill_bh(&swdma_chan->dma_chan);
 
 	switchtec_dma_chan_stop(swdma_chan);
 
-- 
2.43.0

Re: [PATCH v2 21/64] dmaengine: zynqmp-dma: switch completion tasklet to dma_chan BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

Pre-existing issues:
- [High] On channel error, the driver incorrectly marks the currently active descriptor as successful and silently drops all other pending descriptors, causing clients to hang indefinitely.
- [High] The DMA completion handler `zynqmp_dma_chan_desc_cleanup` drops `chan->lock` during callback execution while using `list_for_each_entry_safe`, allowing list corruption if a callback calls `dmaengine_terminate_async`.
- [High] `zynqmp_dma_device_terminate_all` fails to actually halt the hardware DMA engine, leading to hardware silently overwriting memory after the client assumes the transfer is stopped.
--

--- Patch [21]: [PATCH v2 21/64] dmaengine: zynqmp-dma: switch completion tasklet to dma_chan BH ---
commit c7b5b53ae6dbca9c51452b38046b4252492aa34c
Author: Allen Pais <allen.lkml@gmail.com>
Subject: dmaengine: zynqmp-dma: switch completion tasklet to dma_chan BH

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

> diff --git a/drivers/dma/xilinx/zynqmp_dma.c b/drivers/dma/xilinx/zynqmp_dma.c
> index f6a812e49ddca..6291126cdc9f8 100644
> --- a/drivers/dma/xilinx/zynqmp_dma.c
> +++ b/drivers/dma/xilinx/zynqmp_dma.c
[ ... ]
> @@ -760,11 +758,12 @@ static irqreturn_t zynqmp_dma_irq_handler(int irq, void *data)
>  
>  /**
>   * zynqmp_dma_do_tasklet - Schedule completion tasklet
> - * @t: Pointer to the ZynqMP DMA channel structure
> + * @c: Pointer to the ZynqMP DMA channel structure
>   */
> -static void zynqmp_dma_do_tasklet(struct tasklet_struct *t)
> +static void zynqmp_dma_do_tasklet(struct dma_chan *c)
>  {
> -	struct zynqmp_dma_chan *chan = from_tasklet(chan, t, tasklet);
> +	struct zynqmp_dma_chan *chan = container_of(c,
> +			struct zynqmp_dma_chan, common);
>  	u32 count;
>  	unsigned long irqflags;
>  

[Severity: High]
This isn't a bug introduced by this patch, but I noticed a potential problem in
how channel errors are handled when this bottom half runs.

When chan->err is set, we call zynqmp_dma_reset(). Inside zynqmp_dma_reset():

drivers/dma/xilinx/zynqmp_dma.c:zynqmp_dma_reset() {
	...
	zynqmp_dma_complete_descriptor(chan);
	...
	zynqmp_dma_free_descriptors(chan);
	...
}

Does this incorrectly move the active descriptor to done_list and mark it
successful?

Also, it appears zynqmp_dma_free_descriptors() silently moves all remaining
descriptors from active_list and pending_list to the free_list without
invoking their callbacks.

Could this cause clients waiting on these DMA transfers to hang indefinitely?


[Severity: High]
This is a pre-existing issue, but could there be a list corruption risk in
zynqmp_dma_chan_desc_cleanup() which is called shortly after in this bottom
half?

drivers/dma/xilinx/zynqmp_dma.c:zynqmp_dma_chan_desc_cleanup() {
	...
	list_for_each_entry_safe(desc, next, &chan->done_list, node) {
		...
		if (dmaengine_desc_callback_valid(&cb)) {
			spin_unlock_irqrestore(&chan->lock, irqflags);
			dmaengine_desc_callback_invoke(&cb, NULL);
			spin_lock_irqsave(&chan->lock, irqflags);
		}
	...
}

The lock is dropped during callback execution. If the client calls
dmaengine_terminate_async() in the callback, all descriptors (including those
on done_list) are moved to free_list.

When the lock is reacquired, the loop continues using the next pointer which
is now on the free_list.

Could this lead to double-frees and traversing invalid entries?


[Severity: High]
This is a pre-existing issue, but does zynqmp_dma_device_terminate_all()
actually halt the hardware DMA engine?

drivers/dma/xilinx/zynqmp_dma.c:zynqmp_dma_device_terminate_all() {
	...
	writel(ZYNQMP_DMA_IDS_DEFAULT_MASK, chan->regs + ZYNQMP_DMA_IDS);
	zynqmp_dma_free_descriptors(chan);
	...
}

It masks interrupts and frees software descriptors, but doesn't seem to clear
the ZYNQMP_DMA_ENABLE bit or issue a hardware reset.

If the hardware controller remains active, could it silently overwrite memory
after the client assumes the transfer is stopped and unmaps the DMA buffers?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=21

Re: [PATCH v2 16/64] dmaengine: ppc4xx: replace irq tasklet with dma_chan BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing dma_chan_kill_bh() on probe error paths leads to use-after-free.
- [High] Lockless traversal of chan->chain in hardirq context races with list modification, causing panics.
- [High] Missing channel lock in ppc440spe_adma_issue_pending causes concurrent list/hardware corruption.
- [High] Direct freeing of DMA structures in remove() without waiting for active clients causes Use-After-Free.
--

--- Patch [16]: [PATCH v2 16/64] dmaengine: ppc4xx: replace irq tasklet with dma_chan BH ---
commit f5f6478c931c241a7fdae2271fa2dee782cc4570
Author: Allen Pais <allen.lkml@gmail.com>
Subject: dmaengine: ppc4xx: replace irq tasklet with dma_chan BH

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

> diff --git a/drivers/dma/ppc4xx/adma.c b/drivers/dma/ppc4xx/adma.c
> index 279a431ccae38..98cfd012ca5bf 100644
> --- a/drivers/dma/ppc4xx/adma.c
> +++ b/drivers/dma/ppc4xx/adma.c
> @@ -1658,9 +1658,9 @@ static void __ppc440spe_adma_slot_cleanup(struct ppc440spe_adma_chan *chan)
>  /**
>   * ppc440spe_adma_tasklet - clean up watch-dog initiator
>   */
> -static void ppc440spe_adma_tasklet(struct tasklet_struct *t)
> +static void ppc440spe_adma_tasklet(struct dma_chan *c)
>  {
> -	struct ppc440spe_adma_chan *chan = from_tasklet(chan, t, irq_tasklet);
> +	struct ppc440spe_adma_chan *chan = to_ppc440spe_adma_chan(c);
>  
>  	spin_lock_nested(&chan->lock, SINGLE_DEPTH_NESTING);
>  	__ppc440spe_adma_slot_cleanup(chan);

[Severity: High]
This is a pre-existing issue, but does this list modification race with
hardirq context?

The hardirq handlers (ppc440spe_adma_eot_handler and
ppc440spe_adma_err_handler) call ppc440spe_adma_device_clear_eot_status(),
which traverses chan->chain locklessly:

drivers/dma/ppc4xx/adma.c:ppc440spe_adma_device_clear_eot_status() {
    ...
    list_for_each_entry(iter, &chan->chain, chain_node) {
        if (iter->phys == phys)
            break;
    }
    ...
}

Meanwhile, this softirq/BH code modifies the list without disabling local
interrupts (using spin_lock_nested or spin_lock_bh). If a hardware interrupt
fires on the same CPU while the list is being modified, couldn't the hardirq
handler traverse a corrupted list and cause memory corruption or a panic?

[ ... ]

> @@ -4137,7 +4137,7 @@ static int ppc440spe_adma_probe(struct platform_device *ofdev)
>  	chan->common.device = &adev->common;
>  	dma_cookie_init(&chan->common);
>  	list_add_tail(&chan->common.device_node, &adev->common.channels);
> -	tasklet_setup(&chan->irq_tasklet, ppc440spe_adma_tasklet);
> +	dma_chan_init_bh(&chan->common, ppc440spe_adma_tasklet);
>  
>  	/* allocate and map helper pages for async validation or

[Severity: High]
This isn't a bug introduced by this patch, but does the probe error path
need to call dma_chan_kill_bh()?

In ppc440spe_adma_probe(), dma_chan_init_bh() initializes the bottom-half
work. Later, hardware IRQs are requested. If an interrupt fires and schedules
the BH via dma_chan_schedule_bh(), but a subsequent initialization step fails,
the error path uses free_irq() and then frees the channel structure without
canceling the pending work:

err_page_alloc:
    kfree(chan);

When the background work queue executes later, won't it access the freed
chan memory and cause a use-after-free?

[ ... ]

> @@ -4247,7 +4247,7 @@ static void ppc440spe_adma_remove(struct platform_device *ofdev)
>  				 device_node) {
>  		ppc440spe_chan = to_ppc440spe_adma_chan(chan);
>  		ppc440spe_adma_release_irqs(adev, ppc440spe_chan);
> -		tasklet_kill(&ppc440spe_chan->irq_tasklet);
> +		dma_chan_kill_bh(&ppc440spe_chan->common);
>  		if (adev->id != PPC440SPE_XOR_ID) {
>  			dma_unmap_page(&ofdev->dev, ppc440spe_chan->pdest,
>  					PAGE_SIZE, DMA_BIDIRECTIONAL);

[Severity: High]
This is a pre-existing issue, but does this function safely wait for all
clients to drop their references?

In ppc440spe_adma_remove(), the driver calls dma_async_device_unregister()
which unregisters the device but does not synchronously block for active
client references to be dropped.

Immediately after, it frees the channel and device structures:

    list_del(&chan->device_node);
    kfree(ppc440spe_chan);
    ...
    kfree(adev);

If a client (like the network stack or filesystem) still holds a reference
and attempts to use the channel or release it later, will it dereference
these freed structures and cause a use-after-free?

[ ... ]

[Severity: High]
This is another pre-existing issue in the driver. Is channel locking missing 
in ppc440spe_adma_issue_pending()?

The DMA engine callback executes without acquiring chan->lock when checking
chan->pending and calling ppc440spe_chan_append():

drivers/dma/ppc4xx/adma.c:ppc440spe_adma_issue_pending() {
    ...
    if (ppc440spe_chan->pending) {
        ppc440spe_chan->pending = 0;
        ppc440spe_chan_append(ppc440spe_chan);
    }
    ...
}

Other parts of the driver (like ppc440spe_adma_tx_submit) correctly lock
around these calls using spin_lock_bh. Because issue_pending is an
asynchronous dmaengine API that can be called concurrently across multiple
CPUs, couldn't this race and corrupt hardware MMIO registers and descriptor
lists?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=16

Re: [PATCH v2 10/64] dmaengine: mmp_pdma: replace per-chan tasklet with dma_chan BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

Pre-existing issues:
- [High] Unconditional dereference of phy->vchan in the interrupt handler races with channel teardown.
- [High] Missing cancellation of scheduled bottom-half work during channel teardown and device removal causes UAF and NULL pointer dereferences.
- [High] Terminating cyclic transfers frees descriptors but leaves a dangling pointer in `chan->cyclic_first`, causing UAF in the bottom-half handler.
--

--- Patch [10]: [PATCH v2 10/64] dmaengine: mmp_pdma: replace per-chan tasklet with dma_chan BH ---
commit 7ee6ebc1ebbe207f60ed8c3f6afa81414e0802c3
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: mmp_pdma: replace per-chan tasklet with dma_chan BH

This commit replaces the per-channel tasklet with the shared dma_chan BH
helper. The handler continues to run in softirq context while delegating
scheduling to the dmaengine's mechanism.

> diff --git a/drivers/dma/mmp_pdma.c b/drivers/dma/mmp_pdma.c
> index 386e85cd4882a..3fd9ef59a1420 100644
> --- a/drivers/dma/mmp_pdma.c
> +++ b/drivers/dma/mmp_pdma.c
[ ... ]
> @@ -374,7 +373,7 @@ static irqreturn_t mmp_pdma_chan_handler(int irq, void *dev_id)
>  	if (clear_chan_irq(phy) != 0)
>  		return IRQ_NONE;
>  
> -	tasklet_schedule(&phy->vchan->tasklet);
> +	dma_chan_schedule_bh(&phy->vchan->chan);

[Severity: High]
This is a pre-existing issue, but does this race with channel teardown?

If mmp_pdma_free_phy() sets phy->vchan to NULL while the shared physical
interrupt remains active:

drivers/dma/mmp_pdma.c:mmp_pdma_free_phy() {
    ...
    pchan->phy->vchan = NULL;
    ...
}

Could a pending interrupt be serviced immediately afterwards?

If clear_chan_irq() successfully clears the hardware interrupt and returns
zero, it appears this code will unconditionally dereference the NULL pointer
in phy->vchan when scheduling the bottom half.

>  	return IRQ_HANDLED;
>  }
[ ... ]
> @@ -1047,13 +1046,11 @@ static void mmp_pdma_issue_pending(struct dma_chan *dchan)
>  }
>  
>  /*
> - * dma_do_tasklet
> - * Do call back
> - * Start pending list
> + * dma_do_tasklet - BH worker to process completions and kick pending work
>   */
> -static void dma_do_tasklet(struct tasklet_struct *t)
> +static void dma_do_tasklet(struct dma_chan *c)
>  {
> -	struct mmp_pdma_chan *chan = from_tasklet(chan, t, tasklet);
> +	struct mmp_pdma_chan *chan = container_of(c, struct mmp_pdma_chan, chan);

[Severity: High]
This is a pre-existing issue, but is the bottom half properly synchronized
against device removal and channel teardown?

Looking at mmp_pdma_free_chan_resources(), the descriptor pool is destroyed
and chan->desc_pool is set to NULL:

drivers/dma/mmp_pdma.c:mmp_pdma_free_chan_resources() {
    ...
    dma_pool_destroy(chan->desc_pool);
    chan->desc_pool = NULL;
    ...
}

Since there isn't a device_synchronize() or explicit cancellation of the
pending bottom-half work during teardown or mmp_pdma_remove(), could this
background work execute asynchronously?

If it does, it looks like it would invoke dma_pool_free() on a NULL or
destroyed pool further down in this function, or access a chan structure
that was already freed by devres.

[Severity: High]
This isn't a bug introduced by this patch, but can this function access a
freed descriptor during cyclic transfers?

In mmp_pdma_terminate_all(), the driver correctly frees running descriptors
back to the DMA pool by calling mmp_pdma_free_desc_list(), but it doesn't
appear to clear chan->cyclic_first:

drivers/dma/mmp_pdma.c:mmp_pdma_terminate_all() {
    ...
    mmp_pdma_free_desc_list(chan, &chan->chain_running);
    ...
}

If a delayed or spurious bottom-half execution triggers this function,
could it evaluate chan->cyclic_first as non-null and subsequently
use-after-free the memory to invoke the callback?

>  	struct mmp_pdma_desc_sw *desc, *_desc;
>  	LIST_HEAD(chain_cleanup);
>  	unsigned long flags;

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=10

Re: [PATCH v2 07/64] dmaengine: fsl_raid: run completions via dma_chan BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:

Pre-existing issues:
- [High] The driver requests hardware interrupts via request_irq() without devres, but completely fails to call free_irq() in both the probe error paths and device removal path.
- [High] The interrupt is requested before the channel's data structures and driver data are fully initialized, leading to a potential NULL pointer dereference upon early interrupts.
- [High] The driver ignores the return value of fsl_re_chan_probe() during initialization, leading to NULL pointer dereferences in the cleanup path when iterating over failed channels.
- [High] Channel memory is freed before the device is unregistered from the DMA engine, allowing concurrent clients to use freed memory.
--

--- Patch [7]: [PATCH v2 07/64] dmaengine: fsl_raid: run completions via dma_chan BH ---
commit ab56ab807ba67c4bf376a52b60efd1139b5215d9
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: fsl_raid: run completions via dma_chan BH

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

> diff --git a/drivers/dma/fsl_raid.c b/drivers/dma/fsl_raid.c
> --- a/drivers/dma/fsl_raid.c
> +++ b/drivers/dma/fsl_raid.c

[ ... ]

> @@ -670,7 +671,7 @@ static int fsl_re_chan_probe(struct platform_device *ofdev,
>  	snprintf(chan->name, sizeof(chan->name), "re_jr%02d", q);
>  
>  	chandev = &chan_ofdev->dev;
> -	tasklet_setup(&chan->irqtask, fsl_re_dequeue);
> +	dma_chan_init_bh(&chan->chan, fsl_re_dequeue);
>  
>  	ret = request_irq(chan->irq, fsl_re_isr, 0, chan->name, chandev);

[Severity: High]
This is a pre-existing issue, but is it safe to request the IRQ before the
channel's driver data and locks are initialized?

If an interrupt fires immediately after request_irq() succeeds, fsl_re_isr()
will be executed:

fsl_re_isr()
    re_chan = dev_get_drvdata((struct device *)data);
    irqstate = in_be32(&re_chan->jrregs->jr_interrupt_status);

Since dev_set_drvdata() hasn't been called yet, dev_get_drvdata() returns
NULL, which is immediately dereferenced.

[Severity: High]
This is a pre-existing issue, but does this function leak the requested IRQ
on failure?

The driver requests the hardware interrupt without devres. If any subsequent
initialization fails, the function jumps to the err_free labels without
calling free_irq().

>  	if (ret) {

[ ... ]

> @@ -846,7 +847,7 @@ static int fsl_re_probe(struct platform_device *ofdev)
>  
>  static void fsl_re_remove_chan(struct fsl_re_chan *chan)
>  {
> -	tasklet_kill(&chan->irqtask);
> +	dma_chan_kill_bh(&chan->chan);

[Severity: High]
This is a pre-existing issue, but can this lead to a NULL pointer
dereference if the probe previously failed?

In fsl_re_probe(), the driver ignores the return value of
fsl_re_chan_probe():

fsl_re_probe()
    rc = of_device_is_compatible(child, "fsl,raideng-v1.0-job-ring");
    if (rc) {
        fsl_re_chan_probe(ofdev, child, ridx++, off);
        re_priv->total_chans++;
    }

If a channel fails to probe, its pointer in the re_jrs array remains NULL
but total_chans is still incremented. Later, when cleaning up, could passing
that NULL pointer into fsl_re_remove_chan() and calling dma_chan_kill_bh()
cause a kernel crash?

[Severity: High]
This is a pre-existing issue, but shouldn't there be a free_irq() call here?

Since the IRQ was requested without devres during probe, omitting free_irq()
in the teardown path leaves the IRQ active after the driver memory structures
are freed. If an interrupt occurs later, could it lead to a use-after-free?

[Severity: High]
This is a pre-existing issue, but does the driver free channel memory while
the DMA device is still registered?

Looking at fsl_re_remove(), it iterates over all channels and calls
fsl_re_remove_chan() before unregistering the DMA device:

fsl_re_remove()
    for (i = 0; i < re_priv->total_chans; i++)
        fsl_re_remove_chan(re_priv->re_jrs[i]);

    dma_async_device_unregister(&re_priv->dma_dev);

Could concurrent clients still submit requests while this is happening,
leading fsl_re_issue_pending() to copy data into a freed ring buffer?

>  
>  	dma_pool_free(chan->re_dev->hw_desc_pool, chan->inb_ring_virt_addr,
>  		      chan->inb_phys_addr);

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=7

Re: [PATCH v2 11/64] dmaengine: mmp_tdma: hook completions to dma_chan BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing teardown of bottom-half work during channel release, leading to potential use-after-free.
- [High] Incorrect devres initialization ordering for shared interrupts causes NULL pointer dereferences on probe and use-after-free on unbind.
- [High] SRAM descriptors allocated from the `gen_pool` are leaked when `mmp_tdma_prep_dma_cyclic` fails or is called multiple times.
--

--- Patch [11]: [PATCH v2 11/64] dmaengine: mmp_tdma: hook completions to dma_chan BH ---
commit c186dbcb9d7ae1ce6b3473be898cbd442e74d154
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: mmp_tdma: hook completions to dma_chan BH

This commit replaces the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine manages the
common scheduling and teardown mechanism.

> diff --git a/drivers/dma/mmp_tdma.c b/drivers/dma/mmp_tdma.c
> index ba03321eeff79..d1ed2886f0e70 100644
> --- a/drivers/dma/mmp_tdma.c
> +++ b/drivers/dma/mmp_tdma.c

[ ... ]

> @@ -320,7 +319,7 @@ static irqreturn_t mmp_tdma_chan_handler(int irq, void *dev_id)
>  	struct mmp_tdma_chan *tdmac = dev_id;
>  
>  	if (mmp_tdma_clear_chan_irq(tdmac) == 0) {
> -		tasklet_schedule(&tdmac->tasklet);
> +		dma_chan_schedule_bh(&tdmac->chan);
>  		return IRQ_HANDLED;
>  	} else
>  		return IRQ_NONE;
> @@ -346,9 +345,9 @@ static irqreturn_t mmp_tdma_int_handler(int irq, void *dev_id)
>  		return IRQ_NONE;
>  }

[Severity: High]
This is a pre-existing issue, but is it possible for this interrupt handler to
trigger before the channel structures are fully initialized?

Looking at mmp_tdma_probe(), devm_request_irq() is called with IRQF_SHARED
before the channel structures are allocated and assigned:

drivers/dma/mmp_tdma.c:mmp_tdma_probe() {
    ...
    ret = devm_request_irq(&pdev->dev, irq,
                           mmp_tdma_int_handler, IRQF_SHARED, "tdma", tdev);
    ...
}

If the shared IRQ line fires immediately, mmp_tdma_int_handler() iterates over
tdev->tdmac[] and unconditionally passes a potentially uninitialized NULL pointer
to mmp_tdma_chan_handler(). Could this lead to a NULL pointer dereference on
probe, or a use-after-free on unbind since devres reverses the initialization
order?

> -static void dma_do_tasklet(struct tasklet_struct *t)
> +static void dma_do_tasklet(struct dma_chan *c)
>  {
> -	struct mmp_tdma_chan *tdmac = from_tasklet(tdmac, t, tasklet);
> +	struct mmp_tdma_chan *tdmac = container_of(c, struct mmp_tdma_chan, chan);
>  
>  	dmaengine_desc_get_callback_invoke(&tdmac->desc, NULL);
>  }
> @@ -583,7 +582,7 @@ static int mmp_tdma_chan_init(struct mmp_tdma_device *tdev,
>  	tdmac->pool	   = pool;
>  	tdmac->status = DMA_COMPLETE;
>  	tdev->tdmac[tdmac->idx] = tdmac;
> -	tasklet_setup(&tdmac->tasklet, dma_do_tasklet);
> +	dma_chan_init_bh(&tdmac->chan, dma_do_tasklet);

[Severity: High]
This isn't a bug introduced by this patch, but should the bottom-half work be
explicitly torn down during channel release?

Currently, mmp_tdma_free_chan_resources() frees channel resources without
calling dma_chan_kill_bh():

drivers/dma/mmp_tdma.c:mmp_tdma_free_chan_resources() {
    ...
}

Similarly, when the device unbinds, devres frees the channel structure (tdmac)
but mmp_tdma_remove() does not explicitly kill the bottom-half work.

Could this leave pending bottom-half tasks to execute after memory is freed by
devres, leading to a use-after-free?

>  	/* add the channel to tdma_chan list */
>  	list_add_tail(&tdmac->chan.device_node,

[Severity: High]
This is a pre-existing issue, but can mmp_tdma_prep_dma_cyclic() leak SRAM
descriptors allocated from the gen_pool?

If mmp_tdma_config_write() fails, the function branches to err_out without
freeing the newly allocated descriptor:

drivers/dma/mmp_tdma.c:mmp_tdma_prep_dma_cyclic() {
    ...
    desc = mmp_tdma_alloc_descriptor(tdmac);
    if (!desc)
        goto err_out;

    if (mmp_tdma_config_write(chan, direction, &tdmac->slave_config))
        goto err_out;
    ...
}

Additionally, mmp_tdma_alloc_descriptor() unconditionally assigns the newly
allocated memory to tdmac->desc_arr:

drivers/dma/mmp_tdma.c:mmp_tdma_alloc_descriptor() {
    ...
    tdmac->desc_arr = gen_pool_dma_alloc(gpool, size, &tdmac->desc_arr_phys);
    ...
}

If mmp_tdma_prep_dma_cyclic() is called multiple times without an intervening
mmp_tdma_free_chan_resources(), the previous descriptor is blindly overwritten
and permanently leaked. Could this exhaust the limited device SRAM pool and
prevent further DMA operations?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=11

Re: [PATCH v2 01/64] dmaengine: add tasklet-backed channel BH helpers

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [Low] API breakage leading to widespread build failures in downstream DMA drivers due to the removal of the `task` field from `struct virt_dma_chan` without updating its consumers.

Pre-existing issues:
- [Critical] Race condition causing a Use-After-Free of a cyclic descriptor's `tx_result` pointer when a channel is concurrently terminated.
--

--- Patch [1]: [PATCH v2 01/64] dmaengine: add tasklet-backed channel BH helpers ---
commit 9141d715bfba5e26881326cc343ca063eaf8458a
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: add tasklet-backed channel BH helpers

This patch introduces helpers to initialize, schedule, and kill a channel
bottom half. It converts virt-dma to use the new API, replacing its private
tasklet while preserving the execution context.

> diff --git a/drivers/dma/virt-dma.c b/drivers/dma/virt-dma.c
> index 7961172a780dd..89fcd51ddcabe 100644
> --- a/drivers/dma/virt-dma.c
> +++ b/drivers/dma/virt-dma.c
[ ... ]
> @@ -98,7 +98,7 @@ static void vchan_complete(struct tasklet_struct *t)
>  	}
>  	spin_unlock_irq(&vc->lock);
>  
> -	dmaengine_desc_callback_invoke(&cb, &vd->tx_result);
> +	dmaengine_desc_callback_invoke(&cb, vd ? &vd->tx_result : NULL);

[Severity: Critical]
This is a pre-existing issue, but does dropping vc->lock just before
referencing vd->tx_result open a race window for a use-after-free?

If a concurrent dmaengine_terminate_sync() calls the driver's terminate_all
callback (like bcm2835_dma_terminate_all()), the driver could acquire the
lock and free vd via vchan_dma_desc_free_list():

CPU1
vchan_complete()
    spin_unlock_irq(&vc->lock);

CPU2
dmaengine_terminate_sync()
    bcm2835_dma_terminate_all()
        spin_lock_irqsave(&vc->lock, flags);
        vchan_dma_desc_free_list() // frees vd
        spin_unlock_irqrestore(&vc->lock, flags);

CPU1
    dmaengine_desc_callback_invoke(&cb, vd ? &vd->tx_result : NULL);

Could evaluating vd->tx_result here pass a dangling pointer to the client
callback if the descriptor was freed by the concurrent termination?

>  	list_for_each_entry_safe(vd, _vd, &head, node) {
>  		dmaengine_desc_get_callback(&vd->tx, &cb);
[ ... ]
> diff --git a/drivers/dma/virt-dma.h b/drivers/dma/virt-dma.h
> index 59d9eabc8b674..abe2a44757442 100644
> --- a/drivers/dma/virt-dma.h
> +++ b/drivers/dma/virt-dma.h
> @@ -21,7 +21,6 @@ struct virt_dma_desc {
>  
>  struct virt_dma_chan {
>  	struct dma_chan	chan;
> -	struct tasklet_struct task;

[Severity: Low]
Does removing the task field here cause build failures in downstream drivers?

Several DMA drivers like bcm2835-dma, hisi_dma, idma64, and tegra210-adma
directly access chan->vc.task during driver removal or error paths to kill
the tasklet. Without updating those consumers, they will fail to compile.

>  	void (*desc_free)(struct virt_dma_desc *);
>  
>  	spinlock_t lock;

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=1

Re: [PATCH v2 06/64] dmaengine: fsldma: migrate tasklet to dma_chan BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing dma_chan_kill_bh() during channel removal and resource freeing leaves pending bottom-half work queued, leading to use-after-free and NULL pointer dereference crashes.
- [High] Lock inversion deadlock when invoking client callbacks with the channel descriptor lock held.
--

--- Patch [6]: [PATCH v2 06/64] dmaengine: fsldma: migrate tasklet to dma_chan BH ---
commit eba223ea472b760cffa1e53fb0bb48d7fa6057d5
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: fsldma: migrate tasklet to dma_chan BH

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

> diff --git a/drivers/dma/fsldma.c b/drivers/dma/fsldma.c
> index 22d62d958abd9..1cb6ce3270782 100644
> --- a/drivers/dma/fsldma.c
> +++ b/drivers/dma/fsldma.c
> @@ -968,18 +968,18 @@ static irqreturn_t fsldma_chan_irq(int irq, void *data)

[ ... ]

> -static void dma_do_tasklet(struct tasklet_struct *t)
> +static void dma_do_tasklet(struct dma_chan *c)
>  {
> -	struct fsldma_chan *chan = from_tasklet(chan, t, tasklet);
> +	struct fsldma_chan *chan = to_fsl_chan(c);
>  
>  	chan_dbg(chan, "tasklet entry\n");

[Severity: High]
This is a pre-existing issue, but does this cleanup path have a lock inversion
deadlock?

dma_do_tasklet() acquires chan->desc_lock before calling
fsldma_cleanup_descriptors(). While holding this lock, it invokes
fsldma_run_tx_complete_actions(), which executes the client callback via
dmaengine_desc_get_callback_invoke() and processes dependencies via
dma_run_dependencies().

If the client callback submits a new DMA transaction, or if a dependent
transaction is submitted for the same channel, it calls fsl_dma_tx_submit():

drivers/dma/fsldma.c:fsl_dma_tx_submit() {
    ...
    spin_lock_bh(&chan->desc_lock);
    ...
}

Since the non-recursive lock is already held on the same CPU, won't this
lead to a deadlock in softirq context?

> @@ -1152,7 +1152,7 @@ static int fsl_dma_chan_probe(struct fsldma_device *fdev,
>  	}
>  
>  	fdev->chan[chan->id] = chan;
> -	tasklet_setup(&chan->tasklet, dma_do_tasklet);
> +	dma_chan_init_bh(&chan->common, dma_do_tasklet);

[Severity: High]
This is a pre-existing issue, but is it possible for pending bottom-half work
to run after the channel is removed?

If an IRQ fires and queues the BH right before unbinding,
fsl_dma_chan_remove() frees the channel structure and
fsl_dma_free_chan_resources() destroys the descriptor pool without calling
dma_chan_kill_bh() (or tasklet_kill() previously).

When the pending BH work later executes dma_do_tasklet(), could it dereference
the freed channel or pass a NULL desc_pool to dma_pool_free()?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=6

Re: [PATCH v2 04/64] dmaengine: at_xdmac: move irq bottom half to dma_chan BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:

Pre-existing issues:
- [High] This problem wasn't introduced by this patch, but an initialization race condition exists: `request_irq()` is called before hardware clocks are enabled and DMA channels are initialized, which can cause a kernel panic or infinite loop.
- [High] This problem wasn't introduced by this patch, but there is a resource leak and Use-After-Free due to a bypassed cleanup loop in `at_xdmac_remove()`.
- [High] This problem wasn't introduced by this patch, but the teardown order is flawed: Hardware clocks and PM runtime are disabled before freeing the IRQ and stopping BH workers.
- [High] This problem wasn't introduced by this patch, but out-of-bounds memory accesses and list corruption occur due to an unchecked `list_first_entry()` in the BH worker.
- [High] This problem wasn't introduced by this patch, but there is a sleeping in atomic bug: The error handler calls `pm_runtime_resume_and_get()` while holding an IRQ-disabled spinlock.
--

--- Patch [4]: [PATCH v2 04/64] dmaengine: at_xdmac: move irq bottom half to dma_chan BH ---
commit 91d37fd9a6c41425b793e9ed1792794d52b35014
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: at_xdmac: move irq bottom half to dma_chan BH

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

> diff --git a/drivers/dma/at_xdmac.c b/drivers/dma/at_xdmac.c
> index 901971e8bae69..aa918043e37ec 100644
> --- a/drivers/dma/at_xdmac.c
> +++ b/drivers/dma/at_xdmac.c

[ ... ]

> @@ -1759,9 +1758,9 @@ static void at_xdmac_handle_error(struct at_xdmac_chan *atchan)
>  	/* Then continue with usual descriptor management */
>  }
>  
> -static void at_xdmac_tasklet(struct tasklet_struct *t)
> +static void at_xdmac_tasklet(struct dma_chan *chan)
>  {
> -	struct at_xdmac_chan	*atchan = from_tasklet(atchan, t, tasklet);
> +	struct at_xdmac_chan	*atchan = to_at_xdmac_chan(chan);
>  	struct at_xdmac		*atxdmac = to_at_xdmac(atchan->chan.device);
>  	struct at_xdmac_desc	*desc;

[Severity: High]
This is a pre-existing issue, but does this function lack a list empty check?
Later in at_xdmac_tasklet(), the code does:

desc = list_first_entry(&atchan->xfers_list, struct at_xdmac_desc,
                        xfer_node);
if (!desc->active_xfer) { ... }

If at_xdmac_device_terminate_all() runs concurrently and empties
xfers_list, resuming the BH worker might resolve an invalid pointer for
the empty list, read out-of-bounds for desc->active_xfer, and corrupt
the list.

[Severity: High]
This is also a pre-existing issue, but can this path sleep while atomic?
In at_xdmac_tasklet(), an IRQ-disabled spinlock is acquired:

spin_lock_irq(&atchan->lock);
...
if (atchan->irq_status & error_mask)
        at_xdmac_handle_error(atchan);

Inside at_xdmac_handle_error(), it calls pm_runtime_resume_and_get(),
which can sleep since the device isn't marked as pm_runtime_irq_safe().
Could this lead to a scheduling while atomic bug if the device's
runtime status is not RPM_ACTIVE?

[ ... ]

> @@ -2317,7 +2316,7 @@ static int at_xdmac_probe(struct platform_device *pdev)
>  		return PTR_ERR(atxdmac->clk);
>  	}
>  
> -	/* Do not use dev res to prevent races with tasklet */
> +	/* Do not use devm resources to prevent races with the BH worker */
>  	ret = request_irq(atxdmac->irq, at_xdmac_interrupt, 0, "at_xdmac", atxdmac);

[Severity: High]
This isn't a bug introduced by this patch, but an initialization race
condition appears to exist here. Is it safe to call request_irq() before
the hardware clocks are enabled via clk_prepare_enable() and before
dma_async_device_register() initializes the channel count?

If an early or pending interrupt fires immediately after request_irq(),
the handler at_xdmac_interrupt() will execute while the controller's
clock is gated, which could trigger a bus fault.

Even if clocks were enabled, atxdmac->dma.chancnt is still 0 at this point.
The handler would skip the channel check loop, fail to clear the interrupt,
and spin infinitely in its do-while(pending) loop.

[ ... ]

> @@ -2458,7 +2457,7 @@ static void at_xdmac_remove(struct platform_device *pdev)
>  	for (i = 0; i < atxdmac->dma.chancnt; i++) {
>  		struct at_xdmac_chan *atchan = &atxdmac->chan[i];
>  
> -		tasklet_kill(&atchan->tasklet);
> +		dma_chan_kill_bh(&atchan->chan);
>  		at_xdmac_free_chan_resources(&atchan->chan);
>  	}
>  }

[Severity: High]
This is a pre-existing issue, but does the teardown order in
at_xdmac_remove() and the probe error path risk a bus fault?

In at_xdmac_remove(), clk_disable_unprepare() and pm_runtime_disable() are
called before free_irq(). If a pending interrupt fires before free_irq()
completes, the interrupt handler will attempt to read from the clock-gated
device registers, which could trigger a synchronous external abort.

[Severity: High]
This is another pre-existing issue, but will this cleanup loop ever execute?

Earlier in at_xdmac_remove(), dma_async_device_unregister() is called,
which iterates over all registered channels and decrements device->chancnt
to 0.

When this loop attempts to use atxdmac->dma.chancnt as its limit, it is
completely bypassed, meaning dma_chan_kill_bh() and
at_xdmac_free_chan_resources() are never called, potentially causing a
resource leak and use-after-free for pending BH workers.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=4

Re: [PATCH v2 15/64] dmaengine: pch_dma: convert tasklet to dma_chan BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing cleanup of scheduled BH work on the probe error path can lead to a use-after-free.
- [High] Early registration of a shared IRQ before channel count setup leads to an unhandled IRQ storm.
- [High] Client completion callbacks are invoked with the channel spinlock held, leading to a potential self-deadlock.
- [High] Process-context functions acquire the channel spinlock without disabling softirqs, risking a deadlock if preempted by the BH work.
- [High] Checking hardware idle state outside the lock causes a TOCTOU race leading to premature completion and use-after-free.
--

--- Patch [15]: [PATCH v2 15/64] dmaengine: pch_dma: convert tasklet to dma_chan BH ---
commit a2ed59e859fb44b7074fd4693b53c278ebc94f3d
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: pch_dma: convert tasklet to dma_chan BH
    
Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

> diff --git a/drivers/dma/pch_dma.c b/drivers/dma/pch_dma.c
> index bf805f1024f64..b0044bea8fe4c 100644
> --- a/drivers/dma/pch_dma.c
> +++ b/drivers/dma/pch_dma.c

[ ... ]

> @@ -665,9 +664,10 @@ static int pd_device_terminate_all(struct dma_chan *chan)
>  	return 0;
>  }
>  
> -static void pdc_tasklet(struct tasklet_struct *t)
> +static void pdc_tasklet(struct dma_chan *c)
>  {
> -	struct pch_dma_chan *pd_chan = from_tasklet(pd_chan, t, tasklet);
> +	struct pch_dma_chan *pd_chan = container_of(c, struct pch_dma_chan,
> +						    chan);
>  	unsigned long flags;
>  
>  	if (!pdc_is_idle(pd_chan)) {

[Severity: High]
This isn't a bug introduced by this patch, but might checking the hardware
idle state outside the lock cause a race condition?

If a concurrent pd_issue_pending() acquires the lock first and starts the
next request, could the preempted thread later acquire the lock, see a stale
idle state, and improperly complete the newly started descriptor?

This same check appears unprotected in pd_issue_pending():

drivers/dma/pch_dma.c:pd_issue_pending() {
    ...
    if (pdc_is_idle(pd_chan)) {
    ...
}

[Severity: High]
This is a pre-existing issue, but would acquiring the channel lock with a
plain spin_lock() in process-context functions risk a deadlock?

If a hardware interrupt preempts process context here in pd_tx_submit() or
pd_issue_pending():

drivers/dma/pch_dma.c:pd_tx_submit() {
    ...
    spin_lock(&pd_chan->lock);
    ...
}

and schedules this pdc_tasklet() work which attempts to acquire the exact
same lock using spin_lock_irqsave(), might this deadlock the CPU?

[Severity: High]
This is a pre-existing issue, but is there a risk of a self-deadlock if the
client callback submits a new DMA transaction?

If pdc_tasklet() acquires the lock, and then pdc_chain_complete() invokes
the callback:

drivers/dma/pch_dma.c:pdc_chain_complete() {
    ...
    if (callback)
        callback(param);
    ...
}

and that callback attempts to submit a new transaction via pd_tx_submit(),
it will attempt to acquire the same pd_chan->lock.

[ ... ]

> @@ -877,7 +877,7 @@ static int pch_dma_probe(struct pci_dev *pdev,
>  		INIT_LIST_HEAD(&pd_chan->queue);
>  		INIT_LIST_HEAD(&pd_chan->free_list);
>  
> -		tasklet_setup(&pd_chan->tasklet, pdc_tasklet);
> +		dma_chan_init_bh(&pd_chan->chan, pdc_tasklet);
>  		list_add_tail(&pd_chan->chan.device_node, &pd->dma.channels);
>  	}

[Severity: High]
This isn't a bug introduced by this patch, but can requesting the shared
IRQ before dma_async_device_register() initializes the channel count lead
to an unhandled IRQ storm?

In pch_dma_probe(), request_irq() is called with IRQF_SHARED before the
channel count is initialized.

The interrupt handler relies entirely on pd->dma.chancnt:

drivers/dma/pch_dma.c:pd_irq() {
    ...
    for (i = 0; i < pd->dma.chancnt; i++) {
    ...
}

If pd->dma.chancnt is initially 0, could it loop 0 times and return
IRQ_NONE without clearing the hardware interrupt status?

[Severity: High]
This is a pre-existing issue, but does the probe error path fail to kill
the scheduled bottom-half work before freeing the device structure?

If dma_async_device_register() fails, the error path frees the memory but
appears to skip cancelling pending BH works:

drivers/dma/pch_dma.c:pch_dma_probe() {
    ...
err_free_pool:
    dma_pool_destroy(pd->pool);
err_free_irq:
    free_irq(pdev->irq, pd);
    ...
}

If an interrupt fires and schedules the BH work during this window, could
it execute on freed memory?

[ ... ]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=15

Re: [PATCH v2 12/64] dmaengine: mv_xor: convert irq tasklet to dma_chan BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

Pre-existing issues:
- [High] The driver fails to explicitly cancel pending bottom-half work during module removal and initialization error paths, leading to a Use-After-Free.
- [High] Interrupts are requested and unmasked before the channel spinlock is initialized, creating a race condition that can lead to locking an uninitialized spinlock.
- [High] Recursive deadlock when submitting dependent operations from the completion handler.
--

--- Patch [12]: [PATCH v2 12/64] dmaengine: mv_xor: convert irq tasklet to dma_chan BH ---
commit a6fdd06d107411d5248196d2bc3b3e3a0282f10d
Author: Allen Pais <allen.lkml@gmail.com>
Subject: dmaengine: mv_xor: convert irq tasklet to dma_chan BH

This patch replaces the per-channel tasklet with the shared dma_chan BH
helper. The handler continues to run in softirq context while dmaengine
owns the common scheduling and teardown mechanism.

> diff --git a/drivers/dma/mv_xor.c b/drivers/dma/mv_xor.c
> index 25ed61f1b0899..3a70d565e1ef0 100644
> --- a/drivers/dma/mv_xor.c
> +++ b/drivers/dma/mv_xor.c

[ ... ]

> @@ -336,9 +336,9 @@ static void mv_chan_slot_cleanup(struct mv_xor_chan *mv_chan)
>  		mv_chan->dmachan.completed_cookie = cookie;
>  }
>  
> -static void mv_xor_tasklet(struct tasklet_struct *t)
> +static void mv_xor_tasklet(struct dma_chan *c)
>  {
> -	struct mv_xor_chan *chan = from_tasklet(chan, t, irq_tasklet);
> +	struct mv_xor_chan *chan = to_mv_xor_chan(c);
>  
>  	spin_lock(&chan->lock);
>  	mv_chan_slot_cleanup(chan);
>  	spin_unlock(&chan->lock);
>  }

[Severity: High]
This is a pre-existing issue, but can this sequence lead to a recursive
deadlock? Looking at mv_xor_tasklet(), it holds the lock while running
cleanup:

mv_xor_tasklet()
    spin_lock(&chan->lock);
    mv_chan_slot_cleanup(chan);

If mv_chan_slot_cleanup() processes completed slots and invokes
dma_run_dependencies(), the dependent operations might be submitted to the
same channel:

mv_chan_slot_cleanup()
    mv_desc_run_tx_complete_actions()
        dma_run_dependencies()
            mv_xor_tx_submit()
                spin_lock_bh(&mv_chan->lock);

Would this recursive lock attempt cause a system hang, or is there something
preventing dependencies from being queued on the same channel?

[ ... ]

> @@ -1107,7 +1107,7 @@ mv_xor_channel_add(struct mv_xor_device *xordev,
>  
>  	mv_chan->mmr_base = xordev->xor_base;
>  	mv_chan->mmr_high_base = xordev->xor_high_base;
> -	tasklet_setup(&mv_chan->irq_tasklet, mv_xor_tasklet);
> +	dma_chan_init_bh(&mv_chan->dmachan, mv_xor_tasklet);
>  
>  	/* clear errors before enabling interrupts */
>  	mv_chan_clear_err_status(mv_chan);

[Severity: High]
This is a pre-existing issue, but does the initialization error path properly
cancel the bottom half work before the channel structure is freed?

If a failure occurs later in mv_xor_channel_add() (like self-tests failing),
the code jumps to err_free_irq:

err_free_irq:
    free_irq(mv_chan->irq, mv_chan);
err_free_dma:
    ...

Should dma_chan_kill_bh() be called here and in mv_xor_channel_remove() to
ensure scheduled work doesn't execute after mv_chan is freed?

[Severity: High]
This is also a pre-existing issue, but is it possible for an interrupt to
fire before the channel spinlock is initialized?

In mv_xor_channel_add(), request_irq() is called and interrupts are unmasked
before the lock is initialized:

mv_xor_channel_add()
    request_irq(..., mv_xor_interrupt_handler, ...);
    mv_chan_unmask_interrupts(mv_chan);
    ...
    spin_lock_init(&mv_chan->lock);

If a spurious interrupt fires immediately and schedules the BH work, could
mv_xor_tasklet() attempt to acquire an uninitialized lock?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=12

Re: [PATCH v2 08/64] dmaengine: imx-dma: flip per-chan tasklet to dma_chan BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:

Pre-existing issues:
- [High] The per-channel watchdog timer is not synchronously canceled during driver teardown, leading to a Use-After-Free.
- [High] Pending BH works are not canceled on the probe error path, causing a Use-After-Free.
- [High] Early interrupts cause a NULL pointer dereference because IRQs are requested before channel lists are initialized.
- [High] Descriptor state is accessed after being placed on the free list and unlocking the spinlock, leading to a TOCTOU race with prep functions.
- [High] Channel state and lists are read and modified without proper lock protection in descriptor preparation functions.
- [High] The driver frees DMA descriptors in `imxdma_free_chan_resources` without synchronizing with the completion BH, causing a Use-After-Free.
--

--- Patch [8]: [PATCH v2 08/64] dmaengine: imx-dma: flip per-chan tasklet to dma_chan BH ---
commit 6c1578dd7a98a9ceebdc129c266b46d0c2bdf6e3
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: imx-dma: flip per-chan tasklet to dma_chan BH

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

> diff --git a/drivers/dma/imx-dma.c b/drivers/dma/imx-dma.c
> index 81c6276436f81..3ff0bfbfa5ffa 100644
> --- a/drivers/dma/imx-dma.c
> +++ b/drivers/dma/imx-dma.c

[ ... ]

> @@ -592,9 +591,10 @@ static int imxdma_xfer_desc(struct imxdma_desc *d)
>  	return 0;
>  }
>  
> -static void imxdma_tasklet(struct tasklet_struct *t)
> +static void imxdma_tasklet(struct dma_chan *chan)
>  {
> -	struct imxdma_channel *imxdmac = from_tasklet(imxdmac, t, dma_tasklet);
> +	struct imxdma_channel *imxdmac = container_of(chan, struct imxdma_channel,
> +						      chan);
>  	struct imxdma_engine *imxdma = imxdmac->imxdma;
>  	struct imxdma_desc *desc, *next_desc;
>  	unsigned long flags;

[Severity: High]
This is a pre-existing issue, but is it safe to invoke the callback after
releasing the spinlock and moving the descriptor to the free list?

At the end of imxdma_tasklet(), the lock is released and the callback is
subsequently invoked directly from the descriptor:

out:
        spin_unlock_irqrestore(&imxdma->lock, flags);

        dmaengine_desc_get_callback_invoke(&desc->desc, NULL);
}

Since the descriptor is already on the free list, couldn't a concurrent
call to imxdma_prep_dma_memcpy() pick this exact descriptor from ld_free
and clear desc->desc.callback = NULL, leading to execution of a NULL or
corrupted callback?

[Severity: High]
This isn't a bug introduced by this patch, but how are the channel lists
synchronized against concurrent preparation?

The completion BH actively splices and modifies ld_free and ld_active while
holding imxdma->lock. However, prep functions like imxdma_prep_dma_memcpy()
read ld_free and call list_first_entry() without acquiring the channel lock:

drivers/dma/imx-dma.c:imxdma_prep_dma_memcpy() {
        ...
        if (list_empty(&imxdmac->ld_free) ||
            imxdma_chan_is_doing_cyclic(imxdmac))
                return NULL;

        desc = list_first_entry(&imxdmac->ld_free, struct imxdma_desc, node);
        ...
}

Could this lead to linked list corruption or NULL pointer dereferences if
clients actively submit transfers while completions fire?

[ ... ]

> @@ -1142,7 +1142,7 @@ static int __init imxdma_probe(struct platform_device *pdev)
>  		INIT_LIST_HEAD(&imxdmac->ld_free);
>  		INIT_LIST_HEAD(&imxdmac->ld_active);
>  
> -		tasklet_setup(&imxdmac->dma_tasklet, imxdma_tasklet);
> +		dma_chan_init_bh(&imxdmac->chan, imxdma_tasklet);
>  		imxdmac->chan.device = &imxdma->dma_device;
>  		dma_cookie_init(&imxdmac->chan);
>  		imxdmac->channel = i;

[Severity: High]
This is a pre-existing issue, but what happens if the probe fails after
this point?

Looking at the error path later in imxdma_probe():

err_of_dma_controller:
        dma_async_device_unregister(&imxdma->dma_device);
disable_dma_ahb_clk:
        clk_disable_unprepare(imxdma->dma_ahb);
disable_dma_ipg_clk:
        clk_disable_unprepare(imxdma->dma_ipg);
        return ret;

It exits without calling dma_chan_kill_bh() or imxdma_free_irq(). If a
spurious or lingering hardware interrupt triggers before
dma_async_device_register() completes, dma_irq_handler() schedules the BH.
If a subsequent initialization step fails, devres frees the device memory
but the scheduled BH is never canceled, which might lead to a use-after-free.

[Severity: High]
This isn't a bug introduced by this patch, but does the early interrupt
setup risk a NULL pointer dereference?

The IRQ is requested earlier in imxdma_probe():

        ret = devm_request_irq(&pdev->dev, irq,
                               dma_irq_handler, 0, "DMA", imxdma);

This happens well before the list heads like ld_active are initialized in
the loop shown in the diff above. If an interrupt fires early,
dma_irq_handle_channel() checks the list:

        if (list_empty(&imxdmac->ld_active)) {
                spin_unlock_irqrestore(&imxdma->lock, flags);
                goto out;
        }

Since memory is 0-initialized, ld_active.next is NULL. This makes
list_empty() evaluate to false, leading directly to a crash via
list_first_entry() right below it.

[ ... ]

> @@ -1211,7 +1211,7 @@ static void imxdma_free_irq(struct platform_device *pdev, struct imxdma_engine *
>  		if (!is_imx1_dma(imxdma))
>  			disable_irq(imxdmac->irq);
>  
> -		tasklet_kill(&imxdmac->dma_tasklet);
> +		dma_chan_kill_bh(&imxdmac->chan);
>  	}
>  }

[Severity: High]
This is a pre-existing issue, but is it safe to kill the BH here without
also stopping the watchdog timer?

This function stops the BH but omits timer_shutdown_sync(&imxdmac->watchdog),
leaving it active. If the driver is unbound via sysfs while a transfer is
active, imxdma_remove() invokes imxdma_free_irq() and then unregisters
and frees the engine. Couldn't the still-armed timer fire on freed memory
and schedule the destroyed BH workqueue?

[Severity: High]
This isn't a bug introduced by this patch, but does
imxdma_free_chan_resources() safely synchronize with this BH?

When a channel is released, descriptors are indiscriminately freed without
stopping or waiting for the BH to finish:

drivers/dma/imx-dma.c:imxdma_free_chan_resources() {
        ...
        list_for_each_entry_safe(desc, _desc, &imxdmac->ld_free, node) {
                kfree(desc);
                imxdmac->descs_allocated--;
        }
        ...
}

Since the driver lacks a device_synchronize callback and does not call
dma_chan_kill_bh() during channel release, could the completion BH be
actively running on another CPU trying to invoke the callback for a
descriptor that is being freed?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=8

Re: [PATCH v2 19/64] dmaengine: xilinx-dma: use dma_chan BH instead of tasklets

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

Pre-existing issues:
- [High] request_irq() is called before channel initialization is complete, creating a race condition if an early interrupt fires.
- [High] Uninitialized channel access in xilinx_mcdma_irq_handler during probe.
- [High] Recursive spinlock deadlock when running dependencies on the same DMA channel.
--

--- Patch [19]: [PATCH v2 19/64] dmaengine: xilinx-dma: use dma_chan BH instead of tasklets ---
commit 2b1e6b23fed3e56575f2e8169196ee83cbca26e8
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: xilinx-dma: use dma_chan BH instead of tasklets

This patch replaces the per-channel tasklet with the shared dma_chan BH
helper. The handler continues to run in softirq context while dmaengine
owns the common scheduling and teardown mechanism.

> diff --git a/drivers/dma/xilinx/xilinx_dma.c b/drivers/dma/xilinx/xilinx_dma.c
> index 404235c173538..bf9f22ed12659 100644
> --- a/drivers/dma/xilinx/xilinx_dma.c
> +++ b/drivers/dma/xilinx/xilinx_dma.c
[ ... ]
> @@ -1115,11 +1113,12 @@ static void xilinx_dma_chan_desc_cleanup(struct xilinx_dma_chan *chan)
>  
>  /**
>   * xilinx_dma_do_tasklet - Schedule completion tasklet
> - * @t: Pointer to the Xilinx DMA channel structure
> + * @c: Pointer to the Xilinx DMA channel structure
>   */
> -static void xilinx_dma_do_tasklet(struct tasklet_struct *t)
> +static void xilinx_dma_do_tasklet(struct dma_chan *c)
>  {
> -	struct xilinx_dma_chan *chan = from_tasklet(chan, t, tasklet);
> +	struct xilinx_dma_chan *chan = container_of(c,
> +			struct xilinx_dma_chan, common);
>  
>  	xilinx_dma_chan_desc_cleanup(chan);
>  }

[Severity: High]
This is a pre-existing issue, but can this lead to a recursive spinlock
deadlock? 

Looking at xilinx_dma_chan_desc_cleanup(), the channel lock is acquired
before running dependencies:

xilinx_dma_chan_desc_cleanup() {
    ...
    spin_lock_irqsave(&chan->lock, flags);
    ...
    dma_run_dependencies(&desc->async_tx);
    ...
}

If a dependent transaction belongs to the same channel, dma_run_dependencies()
will invoke xilinx_dma_tx_submit(), which unconditionally attempts to acquire
the exact same chan->lock. Since standard spinlocks are not recursive, would
this cause the CPU to deadlock?

[ ... ]
> @@ -1898,7 +1897,7 @@ static irqreturn_t xilinx_mcdma_irq_handler(int irq, void *data)
>  		spin_unlock(&chan->lock);
>  	}
>  
> -	tasklet_hi_schedule(&chan->tasklet);
> +	dma_chan_schedule_bh(&chan->common);
>  	return IRQ_HANDLED;
>  }

[Severity: High]
This is a pre-existing issue, but could this handler dereference a NULL
pointer during device probe?

If a shared interrupt fires while the driver is still probing channels,
xilinx_mcdma_irq_handler() might compute a chan_offset for a channel that
has not been probed yet:

xilinx_mcdma_irq_handler() {
    ...
    chan = chan->xdev->chan[chan_offset];
    status = dma_ctrl_read(chan, XILINX_MCDMA_CHAN_SR_OFFSET(chan->tdest));
    ...
}

If the channel array entry is still NULL, would dma_ctrl_read() trigger a
crash?

[ ... ]
> @@ -3075,8 +3074,7 @@ static int xilinx_dma_chan_probe(struct xilinx_dma_device *xdev,
>  			str_enabled_disabled(chan->has_sg));
>  	}
>  
> -	/* Initialize the tasklet */
> -	tasklet_setup(&chan->tasklet, xilinx_dma_do_tasklet);
> +	dma_chan_init_bh(&chan->common, xilinx_dma_do_tasklet);

[Severity: High]
This is a pre-existing issue, but does calling request_irq() earlier in
xilinx_dma_chan_probe() create a race condition here?

In xilinx_dma_chan_probe(), request_irq() is called with IRQF_SHARED
before chan->start_transfer and the bottom-half work are initialized:

xilinx_dma_chan_probe() {
    ...
    err = request_irq(chan->irq, xdev->dma_config->irq_handler,
                      IRQF_SHARED, "xilinx-dma-controller", chan);
    ...
    chan->start_transfer = xilinx_dma_start_transfer;
    ...
    dma_chan_init_bh(&chan->common, xilinx_dma_do_tasklet);
    ...
}

If an interrupt fires on the shared line immediately after request_irq()
succeeds, the handler could try to invoke chan->start_transfer() or
dma_chan_schedule_bh() while they are still uninitialized. Should the IRQ
registration be moved after the channel is fully initialized?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=19

Re: [PATCH v2 13/64] dmaengine: mxs-dma: use dma_chan BH scheduling

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing synchronization/flushing of bottom-half work during channel teardown leads to a use-after-free of client resources.
- [High] Requesting IRQ before clock enable and descriptor initialization causes a kernel panic on pending interrupt.
- [High] The driver allocates the DMA engine state with devm_kzalloc but lacks a device_release callback, risking UAF on unbind.
--

--- Patch [13]: [PATCH v2 13/64] dmaengine: mxs-dma: use dma_chan BH scheduling ---
commit cc33b3c421743dc24481c221e9955791a7689087
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: mxs-dma: use dma_chan BH scheduling

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

> The handler continues to run in softirq context while dmaengine owns
> the common scheduling and teardown mechanism.

[Severity: High]
This is a pre-existing issue, but since the commit message notes that the
teardown mechanism is being updated, does mxs_dma_free_chan_resources()
need to call dma_chan_kill_bh() to prevent use-after-free conditions?

If a client releases the channel concurrently with an interrupt,
mxs_dma_free_chan_resources() currently returns without flushing pending
work:

drivers/dma/mxs-dma.c:mxs_dma_free_chan_resources() {
	...
	mxs_dma_disable_chan(chan);

	free_irq(mxs_chan->chan_irq, mxs_dma);
}

Could this allow the asynchronous work to outlive the channel allocation
and dereference a freed client callback when mxs_dma_tasklet() runs?

> diff --git a/drivers/dma/mxs-dma.c b/drivers/dma/mxs-dma.c
> index 7acb3d29dad30..8cecc0add82dc 100644
> --- a/drivers/dma/mxs-dma.c
> +++ b/drivers/dma/mxs-dma.c

[ ... ]

> @@ -385,8 +385,8 @@ static irqreturn_t mxs_dma_int_handler(int irq, void *dev_id)
>  		dma_cookie_complete(&mxs_chan->desc);
>  	}
>  
> -	/* schedule tasklet on this channel */
> -	tasklet_schedule(&mxs_chan->tasklet);
> +	/* schedule BH on this channel */
> +	dma_chan_schedule_bh(&mxs_chan->chan);
>  
>  	return IRQ_HANDLED;
>  }

[Severity: High]
This is a pre-existing issue, but can this handler be triggered before the
hardware clock is enabled and the descriptor is initialized?

Looking at mxs_dma_alloc_chan_resources(), the IRQ is requested before the
clock is enabled:

drivers/dma/mxs-dma.c:mxs_dma_alloc_chan_resources() {
	...
	ret = request_irq(mxs_chan->chan_irq, mxs_dma_int_handler,
			  0, "mxs-dma", mxs_dma);
	if (ret)
		goto err_irq;

	ret = clk_prepare_enable(mxs_dma->clk);
	...
}

If a spurious or bootloader-pending interrupt fires immediately, could
mxs_dma_int_handler() process a DMA_COMPLETE status and hit a BUG_ON when
it calls dma_cookie_complete(&mxs_chan->desc) because the cookie is
still 0?

[ ... ]

> @@ -780,7 +780,7 @@ static int mxs_dma_probe(struct platform_device *pdev)
>  		mxs_chan->chan.device = &mxs_dma->dma_device;
>  		dma_cookie_init(&mxs_chan->chan);
>  
> -		tasklet_setup(&mxs_chan->tasklet, mxs_dma_tasklet);
> +		dma_chan_init_bh(&mxs_chan->chan, mxs_dma_tasklet);
>  
>  
>  		/* Add the channel to mxs_chan list */

[Severity: High]
This is a pre-existing issue, but is the driver state vulnerable to a
use-after-free if the device is unbound?

The driver allocates mxs_dma using devm_kzalloc() but appears to register
the async device without providing a device_release callback:

drivers/dma/mxs-dma.c:mxs_dma_probe() {
	...
	mxs_dma = devm_kzalloc(dev, sizeof(*mxs_dma), GFP_KERNEL);
	if (!mxs_dma)
		return -ENOMEM;
	...
	ret = dmaenginem_async_device_register(&mxs_dma->dma_device);
	...
}

If the driver is unbound via sysfs while a client still holds a reference
to a DMA channel, will devres immediately free mxs_dma, causing a panic
when the client later accesses the channel?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=13

Re: [PATCH v2 05/64] dmaengine: ep93xx: hook callbacks via dma_chan BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing `dma_chan_kill_bh()` in `ep93xx_dma_synchronize()` and `ep93xx_dma_free_chan_resources()` violates the dmaengine API and leads to premature callback execution for new clients.
- [High] The ep93xx_dma driver is vulnerable to a Use-After-Free and kernel panic if unbound via sysfs, due to the lack of a .remove callback and missing .suppress_bind_attrs = true.
--

--- Patch [5]: [PATCH v2 05/64] dmaengine: ep93xx: hook callbacks via dma_chan BH ---
commit e03616bfd332cb745207d3436713d688064c9cb5
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: ep93xx: hook callbacks via dma_chan BH

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

> diff --git a/drivers/dma/ep93xx_dma.c b/drivers/dma/ep93xx_dma.c
> index 311e55a97ba9f..8f5200750fe98 100644
> --- a/drivers/dma/ep93xx_dma.c
> +++ b/drivers/dma/ep93xx_dma.c

[ ... ]

> @@ -1420,7 +1418,7 @@ static struct ep93xx_dma_engine *ep93xx_dma_of_probe(struct platform_device *pde
>  		INIT_LIST_HEAD(&edmac->active);
>  		INIT_LIST_HEAD(&edmac->queue);
>  		INIT_LIST_HEAD(&edmac->free_list);
> -		tasklet_setup(&edmac->tasklet, ep93xx_dma_tasklet);
> +		dma_chan_init_bh(&edmac->chan, ep93xx_dma_tasklet);
>  
>  		list_add_tail(&edmac->chan.device_node,
>  			      &dma_dev->channels);

[Severity: High]
This is a pre-existing issue, but I noticed ep93xx_dma_synchronize() and
ep93xx_dma_free_chan_resources() do not cancel the pending BH work.

If a client calls dma_terminate_all(), the active list is emptied, but a
scheduled BH is not cancelled because the driver does not call
dma_chan_kill_bh() (previously missing tasklet_kill()).

When the channel is reallocated, the pending BH can run, see the new client's
descriptor, and unconditionally invoke its callback before completion:

drivers/dma/ep93xx_dma.c:ep93xx_dma_tasklet()
    ...
	desc = ep93xx_dma_get_active(edmac);
	if (desc) {
		if (desc->complete) {
			/* mark descriptor complete for non cyclic case only */
			if (!test_bit(EP93XX_DMA_IS_CYCLIC, &edmac->flags))
				dma_cookie_complete(&desc->txd);
			list_splice_init(&edmac->active, &list);
		}
		dmaengine_desc_get_callback(&desc->txd, &cb);
	}
    ...

Does this need to call dma_chan_kill_bh() during synchronization or channel
release to prevent state corruption for new DMA clients?

[Severity: High]
This isn't a bug introduced by this patch, but the ep93xx_dma driver appears
vulnerable to a Use-After-Free and kernel panic if unbound via sysfs.

Because the driver lacks a .remove callback and doesn't set
.suppress_bind_attrs = true in ep93xx_dma_driver:

drivers/dma/ep93xx_dma.c:
static struct platform_driver ep93xx_dma_driver = {
	.driver		= {
		.name	= "ep93xx-dma",
		.of_match_table = ep93xx_dma_of_ids,
	},
	.probe		= ep93xx_dma_probe,
};

If a privileged user unbinds the driver via sysfs, it never calls
dma_async_device_unregister() or frees IRQs. The device core's devres
mechanism will free the edma and edmac structures, but any subsequent IRQ
firing or scheduled BH execution will dereference the freed memory.

Should this driver include a .remove callback or set
.suppress_bind_attrs = true to prevent this?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=5

Re: [PATCH v2 18/64] dmaengine: xgene-dma: wire descriptor cleanup to dma_chan BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:

Pre-existing issues:
- [High] During device teardown, the driver frees the IRQ before flushing the bottom half workqueue. This results in the bottom half handler calling `enable_irq()` on a freed IRQ descriptor, which violates the API and can cause warnings or oopses in the IRQ subsystem.
- [High] Potential spinlock recursion deadlock in `xgene_dma_cleanup_descriptors()` when called from process context via `xgene_dma_free_chan_resources()`.
- [High] Hardware interrupts are unmasked before the bottom halves are initialized in the probe sequence, allowing an early interrupt to permanently hang the channel.
- [High] In the probe error path, bottom halves are flushed before hardware interrupts are masked, leading to a race condition that can result in a Use-After-Free of driver structures when the workqueue later executes.
- [High] The driver returns ring slots to the hardware before clearing the descriptor completion signature, risking permanently lost completions.
--

--- Patch [18]: [PATCH v2 18/64] dmaengine: xgene-dma: wire descriptor cleanup to dma_chan BH ---
commit 3558adb2da1479ded074f1f3135c3903cdea9f57
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: xgene-dma: wire descriptor cleanup to dma_chan BH

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

> diff --git a/drivers/dma/xgene-dma.c b/drivers/dma/xgene-dma.c
> index fa1173e499009..8d0428c2944ef 100644
> --- a/drivers/dma/xgene-dma.c
> +++ b/drivers/dma/xgene-dma.c
> @@ -975,9 +973,10 @@ static enum dma_status xgene_dma_tx_status(struct dma_chan *dchan,
>  	return dma_cookie_status(dchan, cookie, txstate);
>  }
>  
> -static void xgene_dma_tasklet_cb(struct tasklet_struct *t)
> +static void xgene_dma_tasklet_cb(struct dma_chan *c)
>  {
> -	struct xgene_dma_chan *chan = from_tasklet(chan, t, tasklet);
> +	struct xgene_dma_chan *chan = container_of(c, struct xgene_dma_chan,
> +						   dma_chan);
>  
>  	/* Run all cleanup for descriptors which have been completed */
>  	xgene_dma_cleanup_descriptors(chan);

[Severity: High]
This is a pre-existing issue, but can this cause a spinlock recursion
deadlock?

Looking at xgene_dma_free_chan_resources(), it runs in process context and
calls xgene_dma_cleanup_descriptors(), which acquires chan->lock using
spin_lock() without disabling softirqs.

If a hardware interrupt occurs on the same CPU, it schedules the bottom half.
On interrupt exit, the softirq executes xgene_dma_tasklet_cb(), which also
calls xgene_dma_cleanup_descriptors() and attempts to acquire the
already-held chan->lock.

[Severity: High]
This is a pre-existing issue, but does this out-of-order execution risk
permanently losing DMA completions?

In xgene_dma_cleanup_descriptors(), the driver writes to ring->cmd to notify
the hardware that a descriptor slot is free, and then clears the descriptor's
completion signature in memory:

xgene_dma_cleanup_descriptors() {
    ...
    iowrite32(-1, ring->cmd);
    desc_hw->m0 = cpu_to_le64(XGENE_DMA_DESC_EMPTY_SIGNATURE);
    ...
}

Since the hardware engine is active, could it immediately write a new
completion signature into that slot before the CPU executes the assignment,
causing the CPU to unconditionally overwrite the valid completion signature
with the empty signature?

[ ... ]

> @@ -1539,7 +1538,7 @@ static int xgene_dma_async_register(struct xgene_dma *pdma, int id)
>  	INIT_LIST_HEAD(&chan->ld_pending);
>  	INIT_LIST_HEAD(&chan->ld_running);
>  	INIT_LIST_HEAD(&chan->ld_completed);
> -	tasklet_setup(&chan->tasklet, xgene_dma_tasklet_cb);
> +	dma_chan_init_bh(&chan->dma_chan, xgene_dma_tasklet_cb);

[Severity: High]
This is a pre-existing issue, but can an early interrupt permanently hang
the channel during probe?

In xgene_dma_probe(), xgene_dma_init_hw() enables the DMA engine and unmasks
interrupts before xgene_dma_init_async() initializes the bottom halves.

If a pending or spurious interrupt fires, the ISR attempts to schedule the
uninitialized BH, which fails. Because the BH never runs, enable_irq() is
never called, leaving the channel's IRQ permanently disabled.

[ ... ]

> @@ -1579,7 +1578,7 @@ static int xgene_dma_init_async(struct xgene_dma *pdma)
>  		if (ret) {
>  			for (j = 0; j < i; j++) {
>  				dma_async_device_unregister(&pdma->dma_dev[j]);
> -				tasklet_kill(&pdma->chan[j].tasklet);
> +				dma_chan_kill_bh(&pdma->chan[j].dma_chan);
>  			}

[Severity: High]
This is a pre-existing issue, but could this error path lead to a
use-after-free of the driver structures?

If xgene_dma_init_async() fails, it calls dma_chan_kill_bh() to clean up
initialized channels, but does not mask hardware interrupts.

An interrupt could fire immediately after, re-queuing the BH. The probe
function then calls xgene_dma_free_irqs(), returns an error, and devm
frees the pdma memory.

Could the global workqueue subsequently execute the re-scheduled BH,
dereferencing the freed chan structure?

[ ... ]

> @@ -1790,7 +1789,7 @@ static void xgene_dma_remove(struct platform_device *pdev)
>  
>  	for (i = 0; i < XGENE_DMA_MAX_CHANNEL; i++) {
>  		chan = &pdma->chan[i];
> -		tasklet_kill(&chan->tasklet);
> +		dma_chan_kill_bh(&chan->dma_chan);
>  		xgene_dma_delete_chan_rings(chan);
>  	}

[Severity: High]
This is a pre-existing issue, but is there a risk of accessing a freed IRQ
descriptor during this teardown sequence?

xgene_dma_remove() calls xgene_dma_free_irqs() which frees chan->rx_irq, and
then calls dma_chan_kill_bh().

If the bottom half was already queued, it executes after the IRQ is freed
and unconditionally calls enable_irq(chan->rx_irq), which violates the API
and could cause an oops in the IRQ subsystem.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=18

Re: [PATCH v2 20/64] dmaengine: xilinx-dpdma: kill vchan BH on remove

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

Pre-existing issues:
- [High] Premature clock disable and delayed IRQ free expose the driver to synchronous external aborts during teardown and probe error paths.
- [High] Hardware registers are accessed during probe before the AXI clock is prepared and enabled.
- [High] Concurrent modification of virt-dma `desc_issued` list due to missing `chan->vchan.lock`.
--

--- Patch [20]: [PATCH v2 20/64] dmaengine: xilinx-dpdma: kill vchan BH on remove ---
commit b6ee3342e2955cfec74b9591a7918040bbe546f8
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: xilinx-dpdma: kill vchan BH on remove

virt-dma now dispatches completion callbacks through per-channel BH
work. Cancel that work when removing a channel, while retaining the
driver's separate error-handling tasklet.

> diff --git a/drivers/dma/xilinx/xilinx_dpdma.c b/drivers/dma/xilinx/xilinx_dpdma.c
> index d9a3542c4531c..2ec82064fd2ed 100644
> --- a/drivers/dma/xilinx/xilinx_dpdma.c
> +++ b/drivers/dma/xilinx/xilinx_dpdma.c
> @@ -1685,6 +1685,7 @@ static void xilinx_dpdma_chan_remove(struct xilinx_dpdma_chan *chan)
>  		return;
>  
>  	tasklet_kill(&chan->err_task);
> +	dma_chan_kill_bh(&chan->vchan.chan);
>  	list_del(&chan->vchan.chan.device_node);
>  }

[Severity: High]
This is a pre-existing issue, but does the driver disable the clock too
early during teardown and probe error paths?

Looking at xilinx_dpdma_remove(), the AXI clock is disabled before calling
xilinx_dpdma_chan_remove() to kill the tasklets and BHs:

drivers/dma/xilinx/xilinx_dpdma.c:xilinx_dpdma_remove() {
...
	clk_disable_unprepare(xdev->axi_clk);

	for (i = 0; i < ARRAY_SIZE(xdev->chan); i++)
		xilinx_dpdma_chan_remove(xdev->chan[i]);
...
}

Similarly, in the xilinx_dpdma_probe() error path, the clock is disabled
before tasklets are killed and before the shared IRQ is freed:

drivers/dma/xilinx/xilinx_dpdma.c:xilinx_dpdma_probe() {
...
error_dma_async:
	clk_disable_unprepare(xdev->axi_clk);
error:
	for (i = 0; i < ARRAY_SIZE(xdev->chan); i++)
		xilinx_dpdma_chan_remove(xdev->chan[i]);

	free_irq(xdev->irq, xdev);
...
}

Could this cause synchronous external aborts if an active shared IRQ handler,
tasklet, or BH attempts to access hardware registers while the clock is
disabled?

[Severity: High]
This is a pre-existing issue, but are hardware registers accessed during
probe before the AXI clock is prepared and enabled?

In xilinx_dpdma_probe(), dpdma_hw_init() is called early to disable
interrupts and clear statuses, which accesses hardware registers:

drivers/dma/xilinx/xilinx_dpdma.c:xilinx_dpdma_probe() {
...
	dpdma_hw_init(xdev);

	xdev->irq = platform_get_irq(pdev, 0);
...
}

However, the clock is enabled much later in the probe sequence:

drivers/dma/xilinx/xilinx_dpdma.c:xilinx_dpdma_probe() {
...
	ret = clk_prepare_enable(xdev->axi_clk);
	if (ret) {
...
}

Will this cause a synchronous external abort on device probe if the
bootloader hasn't already enabled the clock?

[Severity: High]
This is a pre-existing issue, but does xilinx_dpdma_chan_handle_err()
modify the virt-dma desc_issued list without the correct lock?

It appears to modify chan->vchan.desc_issued without holding
chan->vchan.lock:

drivers/dma/xilinx/xilinx_dpdma.c:xilinx_dpdma_chan_handle_err() {
...
	if (!chan->desc.pending &&
	    list_empty(&chan->vchan.desc_issued)) {
		active->error = true;
		list_add_tail(&active->vdesc.node,
			      &chan->vchan.desc_issued);
...
}

The virt-dma layer expects chan->vchan.lock to protect the desc_issued list,
as seen in xilinx_dpdma_terminate_all() where chan->vchan.lock is taken:

drivers/dma/xilinx/xilinx_dpdma.c:xilinx_dpdma_terminate_all() {
...
	spin_lock_irqsave(&chan->vchan.lock, flags);
	vchan_get_all_descriptors(&chan->vchan, &descriptors);
	spin_unlock_irqrestore(&chan->vchan.lock, flags);
...
}

Could this concurrent modification lead to linked list corruption if a
hardware channel error occurs while a client calls
dmaengine_terminate_async()?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=20

Re: [PATCH v2 24/64] dmaengine: txx9dmac: route callbacks via channel BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] Uninitialized BH work and missing cleanup in shared IRQ mode
- [High] Interrupt storm and livelock due to premature IRQ unmasking in shared IRQ mode
- [High] Invalid `enable_irq()` parameter in shared IRQ mode
--

--- Patch [24]: [PATCH v2 24/64] dmaengine: txx9dmac: route callbacks via channel BH ---
commit ba661d7c1d6f85fdbcef7fb4031ee54ec6ee5c5a
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: txx9dmac: route callbacks via channel BH

Use the per-channel BH helpers for descriptor completion handling and
schedule per-channel work from both channel and controller IRQ paths.

> diff --git a/drivers/dma/txx9dmac.c b/drivers/dma/txx9dmac.c
> index 05622b68a9363..6269baa28c513 100644
> --- a/drivers/dma/txx9dmac.c
> +++ b/drivers/dma/txx9dmac.c
> @@ -601,13 +601,13 @@ static void txx9dmac_scan_descriptors(struct txx9dmac_chan *dc)
>  	}
>  }
>  
> -static void txx9dmac_chan_tasklet(struct tasklet_struct *t)
> +static void txx9dmac_chan_tasklet(struct dma_chan *chan)
>  {
>  	int irq;
>  	u32 csr;
>  	struct txx9dmac_chan *dc;
>  
> -	dc = from_tasklet(dc, t, tasklet);
> +	dc = container_of(chan, struct txx9dmac_chan, chan);
>  	csr = channel_readl(dc, CSR);
>  	dev_vdbg(chan2dev(&dc->chan), "tasklet: status=%x\n", csr);
>  

[Severity: High]
Does this code handle shared IRQs correctly?

When running in shared IRQ mode, dc->irq is set to -1. Now that the shared
controller path schedules this channel BH, txx9dmac_chan_tasklet() will run
and unconditionally call enable_irq(dc->irq) at the end of the function:

    irq = dc->irq;
    enable_irq(irq);

Could this end up calling enable_irq(-1) and causing an issue in the core IRQ
code?

[ ... ]

> @@ -659,7 +659,7 @@ static void txx9dmac_tasklet(struct tasklet_struct *t)
>  			spin_lock(&dc->lock);
>  			if (csr & (TXX9_DMA_CSR_ABCHC | TXX9_DMA_CSR_NCHNC |
>  				   TXX9_DMA_CSR_NTRNFC))
> -				txx9dmac_scan_descriptors(dc);
> +				dma_chan_schedule_bh(&dc->chan);
>  			spin_unlock(&dc->lock);
>  		}
>  	}

[Severity: High]
Could this result in an interrupt storm?

Previously, txx9dmac_scan_descriptors() would synchronously clear the hardware
interrupt condition before the shared tasklet unmasked the IRQ. 

Now, the work is deferred to the async BH handler, but enable_irq(irq) is
still called synchronously at the end of txx9dmac_tasklet():

    irq = ddev->irq;
    enable_irq(irq);

Will the unmasked IRQ immediately fire again before the BH has a chance to run
and clear the hardware interrupt condition?

[ ... ]

> @@ -1113,7 +1113,7 @@ static int __init txx9dmac_chan_probe(struct platform_device *pdev)
>  		irq = platform_get_irq(pdev, 0);
>  		if (irq < 0)
>  			return irq;
> -		tasklet_setup(&dc->tasklet, txx9dmac_chan_tasklet);
> +		dma_chan_init_bh(&dc->chan, txx9dmac_chan_tasklet);
>  		dc->irq = irq;
>  		err = devm_request_irq(&pdev->dev, dc->irq,
>  			txx9dmac_chan_interrupt, 0, dev_name(&pdev->dev), dc);

[Severity: High]
Is the BH work properly initialized when operating in shared IRQ mode?

dma_chan_init_bh() is only called if dc->ddev->irq < 0:

    if (dc->ddev->irq < 0) {
        ...
        dma_chan_init_bh(&dc->chan, txx9dmac_chan_tasklet);
        ...
    } else {
        dc->irq = -1;
    }

If the controller uses a shared IRQ, the BH work remains uninitialized.
When the shared controller tasklet attempts to schedule it, will it trigger a
WARN_ON(!chan->bh_work_initialized) and drop the completion request entirely?

[ ... ]

> @@ -1159,7 +1159,7 @@ static void txx9dmac_chan_remove(struct platform_device *pdev)
>  	dma_async_device_unregister(&dc->dma);
>  	if (dc->irq >= 0) {
>  		devm_free_irq(&pdev->dev, dc->irq, dc);
> -		tasklet_kill(&dc->tasklet);
> +		dma_chan_kill_bh(&dc->chan);
>  	}
>  	dc->ddev->chan[pdev->id % TXX9_DMA_MAX_NR_CHANNELS] = NULL;
>  }

[Severity: High]
Could we leak resources here in shared IRQ mode?

Similar to the probe function, the cleanup relies on dc->irq >= 0. If the
initialization issue above is fixed so the BH work is correctly used in shared
IRQ mode, dma_chan_kill_bh() would still be skipped here since dc->irq is -1.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=24

Re: [PATCH v2 17/64] dmaengine: ste_dma40: convert per-channel tasklet to dma_chan BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing `free_irq()` on `d40_probe` error paths leads to a leaked IRQ handler and a potential Use-After-Free of `base`.
- [High] Missing `device_synchronize` callback causes a race condition during channel release, potentially invoking callbacks in unloaded client modules.
- [High] Probe failure after exposing channels causes a Use-After-Free because devres forcefully frees memory while clients or scheduled BH work may be active.
--

--- Patch [17]: [PATCH v2 17/64] dmaengine: ste_dma40: convert per-channel tasklet to dma_chan BH ---
commit 7d6ec649e7f14a8affbdbde4ad9aa8498383284e
Author: Allen Pais <allen.lkml@gmail.com>
Subject: dmaengine: ste_dma40: convert per-channel tasklet to dma_chan BH

This commit replaces the per-channel tasklet with the shared dma_chan BH
helper. The handler continues to run in softirq context while dmaengine
manages the scheduling and teardown mechanism.

> diff --git a/drivers/dma/ste_dma40.c b/drivers/dma/ste_dma40.c
> index 0d9ffa3e26639..f279a093a81f6 100644
> --- a/drivers/dma/ste_dma40.c
> +++ b/drivers/dma/ste_dma40.c
[ ... ]
> @@ -1641,7 +1638,7 @@ static void dma_tasklet(struct tasklet_struct *t)
>  	d40c->pending_tx--;
>  
>  	if (d40c->pending_tx)
> -		tasklet_schedule(&d40c->tasklet);
> +		dma_chan_schedule_bh(&d40c->chan);
>  
>  	spin_unlock_irqrestore(&d40c->lock, flags);
>  

[Severity: High]
This is a pre-existing issue, but does this create a race condition during
channel release?

The driver doesn't seem to implement the device_synchronize callback, making
dmaengine_synchronize() a no-op. When a DMA client module unloads and
releases its channel, it relies on this synchronization to flush any pending
async work.

Could dma_release_channel() return and the client module unload while the
bh_work is still executing here? If the work is preempted right after
dropping the channel lock and before invoking the client callback:

	spin_unlock_irqrestore(&d40c->lock, flags);

	if (callback_active)
		dmaengine_desc_callback_invoke(&cb, NULL);

wouldn't it resume and attempt to execute the callback residing in the
now-freed module's .text segment?

Additionally, I noticed a couple of pre-existing issues in d40_probe():

[Severity: High]
This isn't a bug introduced by this patch, but does d40_probe() leak the
IRQ handler on error paths?

drivers/dma/ste_dma40.c:d40_probe() {
    ...
	ret = request_irq(base->irq, d40_handle_interrupt, 0, D40_NAME, base);
	if (ret) {
		d40_err(dev, "No IRQ defined\n");
		goto destroy_cache;
	}
    ...
	if (base->plat_data->use_esram_lcla) {

		base->lcpa_regulator = regulator_get(base->dev, "lcla_esram");
		if (IS_ERR(base->lcpa_regulator)) {
			d40_err(dev, "Failed to get lcpa_regulator\n");
			ret = PTR_ERR(base->lcpa_regulator);
			base->lcpa_regulator = NULL;
			goto destroy_cache;
		}
    ...
 destroy_cache:
	if (base->lcla_pool.dma_addr)
		dma_unmap_single(...)
    ...
 report_failure:
	d40_err(dev, "probe failed\n");
	return ret;
}

If d40_probe() fails after request_irq() (for example, in regulator_enable()
or of_dma_controller_register()), it branches to the destroy_cache label and
returns an error without freeing the registered IRQ.

Because base is automatically freed by the devres framework, if a spurious
or lingering interrupt fires afterwards, won't the still-registered
d40_handle_interrupt() execute and dereference the freed base pointer?


[Severity: High]
This isn't a bug introduced by this patch, but does a probe failure after
exposing channels cause a Use-After-Free?

drivers/dma/ste_dma40.c:d40_probe() {
    ...
	ret = d40_dmaengine_init(base, num_reserved_chans);
	if (ret)
		goto destroy_cache;
    ...
	ret = of_dma_controller_register(np, d40_xlate, NULL);
	if (ret) {
		dev_err(dev,
			"could not register of_dma_controller\n");
		goto destroy_cache;
	}
    ...
}

In d40_probe(), the driver exposes channels globally by calling
dmaenginem_async_device_register() midway through the function via
d40_dmaengine_init().

If a subsequent initialization step fails, like of_dma_controller_register(),
the function returns an error. The devres framework then automatically
unregisters the DMA device and frees the driver's memory.

In the window between registration and failure, could a concurrent client
driver request a channel and start a transfer, or an interrupt fire and
schedule the channel's BH work?

Since dma_async_device_unregister() doesn't wait for active clients to drop
references when called from devres, will the client or BH work access the
forcefully freed structures?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=17

Re: [PATCH v2 09/64] dmaengine: ioat: convert cleanup tasklet to dma_chan BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] The teardown sequence in `ioat_stop()` cancels the timer using `timer_delete_sync()`, but it can be subsequently re-armed by the final cleanup sequence, leading to a use-after-free when channel resources are freed.
- [High] `ioat_shutdown()` does not flush inflight BH runs, which can lead to a use-after-free or crash on module unload.
--

--- Patch [9]: [PATCH v2 09/64] dmaengine: ioat: convert cleanup tasklet to dma_chan BH ---
commit 1d64b3893af8e57fe59c7d2f05e40d26e9aec652
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: ioat: convert cleanup tasklet to dma_chan BH

Replace the per-channel tasklet with the shared dma_chan BH helper.
The handler continues to run in softirq context while dmaengine owns
the common scheduling and teardown mechanism.

> diff --git a/drivers/dma/ioat/dma.c b/drivers/dma/ioat/dma.c
> index ee93b029f9e3d..827b46c6354ad 100644
> --- a/drivers/dma/ioat/dma.c
> +++ b/drivers/dma/ioat/dma.c

[ ... ]

> @@ -161,11 +161,11 @@ void ioat_stop(struct ioatdma_chan *ioat_chan)
>  	/* flush inflight timers */
>  	timer_delete_sync(&ioat_chan->timer);
>  
> -	/* flush inflight tasklet runs */
> -	tasklet_kill(&ioat_chan->cleanup_task);
> +	/* flush inflight BH runs */
> +	dma_chan_kill_bh(&ioat_chan->dma_chan);
>  
>  	/* final cleanup now that everything is quiesced and can't re-arm */
> -	ioat_cleanup_event(&ioat_chan->cleanup_task);
> +	ioat_cleanup_event(&ioat_chan->dma_chan);

[Severity: High]
This is a pre-existing issue, but does this sequence inadvertently re-arm
the timer during teardown?

After timer_delete_sync() is called, ioat_cleanup_event() triggers a call
chain that can re-arm the timer unconditionally:

ioat_stop()
   timer_delete_sync(&ioat_chan->timer);
   ioat_cleanup_event()
      ioat_cleanup()
         ioat_cleanup_preamble()
            mod_timer(&ioat_chan->timer, jiffies + COMPLETION_TIMEOUT);

If the timer is re-armed here, could the delayed timer callback execute
after ioat_free_chan_resources() frees the completion pool memory?

Would upgrading timer_delete_sync() to timer_shutdown_sync() properly
prevent the timer from being re-armed?

[Severity: High]
This is also a pre-existing issue, but does ioat_shutdown() in 
drivers/dma/ioat/init.c have a similar teardown problem with the BH?

In ioat_shutdown(), the driver calls timer_delete_sync() and disables 
interrupts, but it does not appear to flush inflight BH runs like it does 
here in ioat_stop(). 

Since the IOAT_RUN bit is not cleared in ioat_shutdown():

drivers/dma/ioat/init.c:ioat_shutdown() {
    ...
    timer_delete_sync(&ioat_chan->timer);
    ioat_reset_hw(ioat_chan);
    ...
    ioat_disable_interrupts(ioat_dma);
}

Could an interrupt firing just before ioat_disable_interrupts() successfully
schedule the BH via dma_chan_schedule_bh()?

If the module is then unloaded, could the scheduled BH execute after 
release_ioatdma() has freed the channel structures, resulting in a 
use-after-free?

>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=9

Re: [PATCH v2 25/64] dmaengine: mv_xor_v2: use channel BH helpers

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

Pre-existing issues:
- [High] Incorrect teardown order in remove() leading to Use-After-Free.
- [High] Missing dma_chan_kill_bh() in probe error path leads to Use-After-Free.
- [High] Early interrupt request in probe allows uninitialized state access.
--

--- Patch [25]: [PATCH v2 25/64] dmaengine: mv_xor_v2: use channel BH helpers ---
commit 1d8bd96829601184f429fc1fa7947ce7a05990e9
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: mv_xor_v2: use channel BH helpers

This patch replaces the device tasklet with the per-channel BH helper.
Descriptor callbacks are updated to run through dma_chan_*_bh instead
of the tasklet API.

> diff --git a/drivers/dma/mv_xor_v2.c b/drivers/dma/mv_xor_v2.c
> index cad4d4fb51ac3..e5fa7a221d5b0 100644
> --- a/drivers/dma/mv_xor_v2.c
> +++ b/drivers/dma/mv_xor_v2.c
[ ... ]
> @@ -760,7 +758,7 @@ static int mv_xor_v2_probe(struct platform_device *pdev)
>  	if (ret)
>  		goto free_msi_irqs;
>  
> -	tasklet_setup(&xor_dev->irq_tasklet, mv_xor_v2_tasklet);
> +	dma_chan_init_bh(&xor_dev->dmachan, mv_xor_v2_tasklet);

[Severity: High]
This is a pre-existing issue, but does requesting the interrupt before
initializing the bottom-half allow an early interrupt to crash the system?

In mv_xor_v2_probe(), devm_request_irq() requests and enables the interrupt
before dma_chan_init_bh() and dma_alloc_coherent() are called. If the
hardware is left in an active state with pending events, an interrupt will
fire immediately.

The handler mv_xor_v2_interrupt_handler() calls dma_chan_schedule_bh().
Depending on the timing, this could crash trying to schedule an uninitialized
work struct, or the scheduled work could run and dereference a NULL
hw_desq_virt pointer.

[Severity: High]
This is a pre-existing issue, but is a call to dma_chan_kill_bh() missing
in the probe error paths?

If a setup step fails after initialization, such as dma_alloc_coherent()
failing, the probe function jumps to the error labels and returns without
calling dma_chan_kill_bh().

If an early interrupt had already fired and scheduled the bottom-half, the
work item remains active in the workqueue. When devres automatically frees
the xor_dev structure containing the work item, could the executing workqueue
corrupt memory?

>  
>  	xor_dev->desc_size = mv_xor_v2_set_desc_size(xor_dev);
[ ... ]
> @@ -869,7 +867,7 @@ static void mv_xor_v2_remove(struct platform_device *pdev)
>  
>  	platform_device_msi_free_irqs_all(&pdev->dev);
>  
> -	tasklet_kill(&xor_dev->irq_tasklet);
> +	dma_chan_kill_bh(&xor_dev->dmachan);
>  }

[Severity: High]
This is a pre-existing issue, but is the teardown order in mv_xor_v2_remove()
safe from use-after-free races?

In mv_xor_v2_remove(), dma_free_coherent() is called to free the hardware
descriptor ring (hw_desq_virt) before the device interrupt is disabled via
devm_free_irq() and before the bottom-half is killed via dma_chan_kill_bh().

If a pending hardware interrupt fires during this window, or if a previously
scheduled bottom-half is running concurrently, the executing bottom-half
could dereference the freed hw_desq_virt memory:

mv_xor_v2_tasklet() {
    ...
		struct mv_xor_v2_descriptor *next_pending_hw_desc =
			xor_dev->hw_desq_virt + pending_ptr;
    ...
}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=25

Re: [PATCH v2 27/64] dmaengine: k3dma: kill vchan BH on remove

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Concurrent execution of `k3_dma_tasklet` and `k3_dma_remove` leads to list corruption and a `LIST_POISON1` kernel panic.
--

--- Patch [27]: [PATCH v2 27/64] dmaengine: k3dma: kill vchan BH on remove ---
commit 0a41ab53724cb23d95bb02f7dd926541fc428dd1
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: k3dma: kill vchan BH on remove

This commit updates virt-dma cleanup in the k3dma driver. It switches from
using tasklet_kill() to dma_chan_kill_bh() during driver removal, reflecting
that vchan now uses the per-channel BH helper.

> diff --git a/drivers/dma/k3dma.c b/drivers/dma/k3dma.c
> index e84f197fea76f..99a70b90a02b1 100644
> --- a/drivers/dma/k3dma.c
> +++ b/drivers/dma/k3dma.c
> @@ -976,7 +976,7 @@ static void k3_dma_remove(struct platform_device *op)
>  
>  	list_for_each_entry_safe(c, cn, &d->slave.channels, vc.chan.device_node) {
>  		list_del(&c->vc.chan.device_node);

[Severity: High]
This is a pre-existing issue, but can this sequence cause a list corruption and
kernel panic if a hardware tasklet runs concurrently?

If an interrupt fires and schedules d->task just before devm_free_irq() runs,
k3_dma_tasklet() could run concurrently and iterate over d->slave.channels
locklessly:

drivers/dma/k3dma.c:k3_dma_tasklet() {
    ...
    list_for_each_entry_safe(c, cn, &d->slave.channels, vc.chan.device_node) {
        spin_lock_irq(&c->vc.lock);
        p = c->phy;
    ...
}

Concurrently, k3_dma_remove() calls list_del() on the channel nodes before
calling tasklet_kill(&d->task) below. If k3_dma_tasklet() fetches a deleted
node, will it see LIST_POISON1 in the next pointer and panic on the next
iteration?

Could tasklet_kill(&d->task) be moved before the channel list modification to
ensure the tasklet is stopped first?

> -		tasklet_kill(&c->vc.task);
> +		dma_chan_kill_bh(&c->vc.chan);
>  	}
>  	tasklet_kill(&d->task);

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=27

Re: [PATCH v2 29/64] dmaengine: sf-pdma: route error callbacks through channel BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 7 potential issue(s) to consider:

New issues:
- [Critical] Concurrent execution of errbh_tasklet and donebh_tasklet causes the driver to corrupt the active DMA hardware state.
- [High] Race condition between errbh_tasklet and donebh_tasklet causes a kernel panic via BUG_ON() during descriptor completion.

Pre-existing issues:
- [High] Double free of the active descriptor during channel termination or resource cleanup.
- [High] Memory and resource leak in `sf_pdma_remove` due to premature channel list removal.
- [High] Lockless read of `chan->desc` in `sf_pdma_errbh_tasklet` leads to use-after-free and virt-dma list corruption.
- [High] Missing NULL pointer check on `chan->desc` causes a kernel panic on concurrent termination or stray interrupts.
- [Medium] Residue reporting always returns 0 for actively running transfers.
--

--- Patch [29]: [PATCH v2 29/64] dmaengine: sf-pdma: route error callbacks through channel BH ---
commit 4199e17e8ac9bf5b639472caa463955dc33436b1
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: sf-pdma: route error callbacks through channel BH

This patch modifies the error handling path for SF-PDMA to route completions
through the virt-dma per-channel bottom half instead of invoking callbacks
directly from the error tasklet. It updates the error tasklet to complete
the aborted descriptor using virt-dma primitives and start the next transfer.
It also attempts to ensure tasklets are killed before the channel BH during
removal.

> diff --git a/drivers/dma/sf-pdma/sf-pdma.c b/drivers/dma/sf-pdma/sf-pdma.c
> index 6f79cc28703e6..a6c2f53f005ed 100644
> --- a/drivers/dma/sf-pdma/sf-pdma.c
> +++ b/drivers/dma/sf-pdma/sf-pdma.c
> @@ -328,7 +328,17 @@ static void sf_pdma_errbh_tasklet(struct tasklet_struct *t)

[Severity: High]
This is a pre-existing issue, but is there a risk of a use-after-free
here? At the beginning of sf_pdma_errbh_tasklet(), we read the descriptor
pointer outside of any lock:

sf_pdma_errbh_tasklet() {
    ...
    struct sf_pdma_desc *desc = chan->desc;
    unsigned long flags;

    spin_lock_irqsave(&chan->lock, flags);
}

If a client concurrently calls sf_pdma_terminate_all(), it frees chan->desc.
Since the tasklet later unconditionally dereferences this cached pointer,
could this corrupt memory?

>  	if (chan->retries <= 0) {
>  		/* fail to recover */
>  		spin_unlock_irqrestore(&chan->lock, flags);
> -		dmaengine_desc_get_callback_invoke(desc->async_tx, NULL);
> +
> +		spin_lock_irqsave(&chan->vchan.lock, flags);
> +		list_del(&desc->vdesc.node);
> +		desc->vdesc.tx_result.result = DMA_TRANS_ABORTED;
> +		desc->vdesc.tx_result.residue = desc->xfer_size;
> +		vchan_cookie_complete(&desc->vdesc);

[Severity: High]
If both sf_pdma_donebh_tasklet() and sf_pdma_errbh_tasklet() run concurrently
due to rapid sequential interrupts, can they race on this completion?

If the done tasklet acquires the lock first, it completes the descriptor
and clears its cookie. When this error tasklet acquires the lock next, it
will attempt to complete the exact same descriptor again. Will this double
completion trigger a BUG_ON() in the core dmaengine code?

> +
> +		chan->desc = sf_pdma_get_first_pending_desc(chan);
> +		if (chan->desc)
> +			sf_pdma_xfer_desc(chan);

[Severity: Critical]
Could this hardware re-triggering logic corrupt the DMA state if the done
tasklet is executing simultaneously?

If this error tasklet completes the current descriptor and starts the next
one in hardware, the done tasklet might still assume the original descriptor
is active. Will the done tasklet then incorrectly complete the actively
running descriptor and start a third one, corrupting the DMA registers
during an active transfer?

> +		spin_unlock_irqrestore(&chan->vchan.lock, flags);
>  	} else {
>  		/* retry */

[ ... ]

> @@ -602,9 +612,9 @@ static void sf_pdma_remove(struct platform_device *pdev)
>  		devm_free_irq(&pdev->dev, ch->txirq, ch);
>  		devm_free_irq(&pdev->dev, ch->errirq, ch);
>  		list_del(&ch->vchan.chan.device_node);

[Severity: High]
This is a pre-existing issue, but does removing this device node early
break the dmaengine core cleanup?

When sf_pdma_remove() calls list_del() here, it empties the channels list.
Later in the removal process, dma_async_device_unregister() iterates over
the channels list to free resources like per-cpu memory and IDAs. Since
the list is already empty, will those resources be permanently leaked?

> -		tasklet_kill(&ch->vchan.task);
>  		tasklet_kill(&ch->done_tasklet);
>  		tasklet_kill(&ch->err_tasklet);
> +		dma_chan_kill_bh(&ch->vchan.chan);
>  	}


[Severity: High]
This is a pre-existing issue, but can sf_pdma_terminate_all() trigger
a double free of the active descriptor?

sf_pdma_terminate_all() explicitly frees the descriptor:

sf_pdma_terminate_all() {
    ...
    kfree(chan->desc);
    chan->desc = NULL;
    ...
    vchan_get_all_descriptors(&chan->vchan, &head);
    ...
    vchan_dma_desc_free_list(&chan->vchan, &head);
}

Because the active descriptor isn't removed from the virt-dma desc_issued
list before being freed, won't vchan_get_all_descriptors() collect it
again, causing vchan_dma_desc_free_list() to free the same pointer a
second time?


[Severity: High]
This is a pre-existing issue, but is there a missing NULL pointer check
when handling partial completions?

If a concurrent client terminates the channel, chan->desc is set to NULL.
However, the done ISR unconditionally dereferences it:

sf_pdma_done_isr() {
    ...
    struct sf_pdma_desc *desc = chan->desc;

    desc->src_addr += desc->xfer_size - residue;
    ...
}

Could this cause a kernel panic if the ISR fires right after the channel
is terminated?


[Severity: Medium]
This is a pre-existing issue, but does sf_pdma_desc_residue() fail to
report accurate remaining bytes for actively running transfers?

sf_pdma_desc_residue() only searches the submitted list:

sf_pdma_desc_residue() {
    ...
    list_for_each_entry(vd, &chan->vchan.desc_submitted, node)
        if (vd->tx.cookie == cookie)
            tx = &vd->tx;

    if (!tx)
        goto out;
    ...
}

Since actively running descriptors are moved to the desc_issued list,
won't this always fail to find the active transfer and incorrectly return
0 remaining bytes?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=29

Re: [PATCH v2 35/64] dmaengine: altera-msgdma: use channel BH helpers

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] The device interrupt is requested before essential locks, lists, and bottom-half structures are initialized, risking a crash if an interrupt fires early.
- [High] Missing dma_async_device_unregister() in the probe error path causes Use-After-Free.
--

--- Patch [35]: [PATCH v2 35/64] dmaengine: altera-msgdma: use channel BH helpers ---
Note: The format of this review has been altered to use a free-form summary 
rather than inline diff quoting due to strict recitation restrictions.

commit bf40fd1cbe95a4c51ca6b86e4123ac5282a06f01
Author: Allen Pais <allen.lkml@gmail.com>
Subject: dmaengine: altera-msgdma: use channel BH helpers

Replace the device tasklet with the per-channel BH helper so completion 
handling runs via dma_chan_*_bh.

[Severity: High]
This isn't a bug introduced by this patch, but looking at msgdma_probe() in 
drivers/dma/altera-msgdma.c, the interrupt is requested via 
devm_request_irq() before mdev->lock, the pending_list, and the bottom-half 
structures are initialized.

If a pending hardware interrupt fires immediately, could msgdma_irq_handler() 
execute and attempt to acquire the uninitialized mdev->lock? Furthermore, if 
it then calls msgdma_start_transfer(), it appears list_first_entry_or_null() 
on the zero-initialized pending_list would return an invalid pointer. Would 
this cause a page fault during list_splice_tail_init() when it subsequently 
evaluates list_empty() inside the splice operations? Is it possible to move 
the devm_request_irq() call after all these structures and lists are fully 
initialized?

[Severity: High]
This is also a pre-existing issue, but I have a question about the error 
handling further down in msgdma_probe().

If of_dma_controller_register() fails with an error other than -EINVAL or 
-ENODEV, the code jumps to the fail label and calls msgdma_dev_remove(). 
However, msgdma_dev_remove() only cleans up the IRQ and bottom half, without 
calling dma_async_device_unregister().

Since dma_async_device_register() was already called earlier in the probe 
function, won't devres subsequently free the mdev structure while the DMA 
engine core still holds pointers to its embedded dma_dev and channels? Could 
this lead to a use-after-free if the DMA engine core later traverses its 
channel lists? Does dma_async_device_unregister() need to be added to the 
cleanup path?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=35

Re: [PATCH v2 26/64] dmaengine: mpc512x: route callbacks via channel BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 7 potential issue(s) to consider:

New issues:
- [High] Missing dma_chan_kill_bh() in mpc_dma_remove() allows per-channel bottom-half workqueue items to outlive the device structures.

Pre-existing issues:
- [High] The error handling path in mpc_dma_probe() fails to cancel the tasklet and BH work after the IRQ is registered.
- [High] Concurrent execution of mpc_dma_process_chan_completed() can cause a channel's completed_cookie to regress.
- [High] request_irq() is called before channel locks, lists, and tasklets are initialized.
- [High] Releasing a channel with pending completions triggers a BUG_ON() panic due to missing device_synchronize().
- [High] mpc_dma_free_chan_resources() can race with the global fallback mpc_dma_process_completed(), leading to a Use-After-Free of DMA descriptors.
- [High] The driver state structure allocated with devm_kzalloc() is freed prematurely if the driver is unbound while clients hold channel references.
--

--- Patch [26]: [PATCH v2 26/64] dmaengine: mpc512x: route callbacks via channel BH ---
commit 57dd5b81d14732d0a29ee229ec6a57e6e3b79c4c
Author: Allen Pais <allen.lkml@gmail.com>
Subject: dmaengine: mpc512x: route callbacks via channel BH

This commit transitions the mpc512x DMA driver from processing completions
entirely in the controller tasklet to scheduling per-channel bottom-half (BH)
work. This leaves the tasklet strictly for IRQ and error handling.

> diff --git a/drivers/dma/mpc512x_dma.c b/drivers/dma/mpc512x_dma.c
> index 0adc8e01057e1..552ea72a9108e 100644
> --- a/drivers/dma/mpc512x_dma.c
> +++ b/drivers/dma/mpc512x_dma.c
[ ... ]
> +static void mpc_dma_process_chan_completed(struct mpc_dma_chan *mchan)
> +{
[ ... ]
> +	/* Free descriptors */
> +	spin_lock_irqsave(&mchan->lock, flags);
> +	list_splice_tail_init(&list, &mchan->free);
> +	mchan->chan.completed_cookie = last_cookie;
> +	spin_unlock_irqrestore(&mchan->lock, flags);
> +}

[Severity: High]
This is a pre-existing issue, but does this unconditional overwrite of the
completed cookie leave us vulnerable to race conditions?

Since mpc_dma_process_completed() can be called synchronously from
mpc_dma_prep_memcpy(), it can race with the asynchronous processing of the
same channel's completions.

If CPU 1 grabs an older batch of completed descriptors and CPU 2 grabs a
newer batch, CPU 2 might finish executing callbacks first. When CPU 1
finishes, it overwrites mchan->chan.completed_cookie with its older
last_cookie, causing the cookie to regress.

[Severity: High]
This is a pre-existing issue, but could this list splicing race with channel
freeing and cause a use-after-free?

When a client runs out of descriptors, mpc_dma_process_completed() iterates
globally and extracts completed descriptors for another channel.

While the callbacks are executing outside the lock, another client could
concurrently free that channel via mpc_dma_free_chan_resources(), freeing
the coherent TCD memory.

After callbacks finish, splicing the local list back into mchan->free puts
leaked descriptors pointing to freed memory back into circulation.

[ ... ]
> @@ -551,6 +568,8 @@ static void mpc_dma_free_chan_resources(struct dma_chan *chan)
>  	unsigned long flags;
>  	LIST_HEAD(descs);
>  
> +	dma_chan_kill_bh(&mchan->chan);
> +
>  	spin_lock_irqsave(&mchan->lock, flags);
>  
>  	/* Channel must be idle */
>  	BUG_ON(!list_empty(&mchan->prepared));
>  	BUG_ON(!list_empty(&mchan->queued));
>  	BUG_ON(!list_empty(&mchan->active));
>  	BUG_ON(!list_empty(&mchan->completed));

[Severity: High]
This is a pre-existing issue, but can this trigger a kernel panic due to
missing synchronization?

When a client terminates a transfer, device_terminate_all() is called,
but it ignores mchan->completed. The core expects device_synchronize()
to wait for pending async tasks, but this driver doesn't implement it.

If mpc_dma_free_chan_resources() is called with pending completions,
the subsequent assertions like BUG_ON(!list_empty(&mchan->completed))
will trigger a panic.

[ ... ]
> @@ -1007,6 +1026,7 @@ static int mpc_dma_probe(struct platform_device *op)
>  		INIT_LIST_HEAD(&mchan->completed);
>  
>  		spin_lock_init(&mchan->lock);
> +		dma_chan_init_bh(&mchan->chan, mpc_dma_chan_bh);
>  		list_add_tail(&mchan->chan.device_node, &dma->channels);
>  	}

[Severity: High]
Is there a missing cleanup step for these bottom-half work items when the
driver is removed?

The teardown function mpc_dma_remove() correctly calls tasklet_kill(),
but does not cancel or flush these newly added BH work items.

If the driver is unbound while active:

mpc_dma_irq()
  tasklet_schedule(&mdma->tasklet);

mpc_dma_remove()
  tasklet_kill(&mdma->tasklet);

The tasklet's new purpose is to schedule dma_chan_schedule_bh(). These
work items will remain queued and execute asynchronously after the function
returns. Since mdma is allocated via devm_kzalloc(), devres frees it
immediately upon remove returning, causing the orphaned BH work to access
freed memory.

[Severity: High]
This is a pre-existing issue, but does tying the controller structure
lifecycle to the device binding risk a use-after-free?

In mpc_dma_probe(), the struct mpc_dma is allocated using devm_kzalloc(). If
the driver is unbound (e.g., via sysfs), mpc_dma_remove() completes and
devres automatically frees the allocation.

If a DMA client driver is still holding a reference to a channel embedded
in this structure, any subsequent API calls or asynchronous callbacks on
that channel will dereference the freed memory.

[Severity: High]
This is a pre-existing issue, but does the error handling path in
mpc_dma_probe() need to cancel the scheduled tasklet and BH?

If a late initialization step (like dma_async_device_register) fails,
the code jumps to the error labels and frees the IRQ:

err_free2:
    if (mdma->is_mpc8308)
        free_irq(mdma->irq2, mdma);
err_free1:
    free_irq(mdma->irq, mdma);

However, it fails to call tasklet_kill() or dma_chan_kill_bh(). If an
interrupt fired and scheduled the tasklet or BH, they will continue to run
after probe returns and devres frees the mdma structure, leading to a
use-after-free.

[Severity: High]
This is a pre-existing issue, but is it safe to request IRQs before the
device structures are fully initialized?

In mpc_dma_probe(), request_irq() is invoked before the channel lists,
spinlocks, and tasklets are set up.

If the hardware asserts an interrupt immediately, mpc_dma_irq() will run
and attempt to lock mdma->error_status_lock (uninitialized), and schedule
mdma->tasklet (uninitialized), crashing the kernel.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=26

Re: [PATCH v2 30/64] dmaengine: sa11x0-dma: kill vchan BH on remove

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] The driver's teardown sequence in `sa11x0_dma_remove()` and the probe error path is critically flawed, freeing DMA channels before disabling hardware interrupts and stopping the device tasklet.
--

--- Patch [30]: [PATCH v2 30/64] dmaengine: sa11x0-dma: kill vchan BH on remove ---
commit 34b16ea740696f927777e47c145e53481090efcc
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: sa11x0-dma: kill vchan BH on remove

Use dma_chan_kill_bh() when tearing down virt-dma channels.

> diff --git a/drivers/dma/sa11x0-dma.c b/drivers/dma/sa11x0-dma.c
> index a6fa431530e3d..ea4357da5e39d 100644
> --- a/drivers/dma/sa11x0-dma.c
> +++ b/drivers/dma/sa11x0-dma.c
> @@ -891,7 +891,7 @@ static void sa11x0_dma_free_channels(struct dma_device *dmadev)
>  
>  	list_for_each_entry_safe(c, cn, &dmadev->channels, vc.chan.device_node) {
>  		list_del(&c->vc.chan.device_node);
> -		tasklet_kill(&c->vc.task);
> +		dma_chan_kill_bh(&c->vc.chan);
>  		kfree(c);

[Severity: High]
This isn't a bug introduced by this patch, but is a pre-existing structural
flaw in the driver's teardown sequence.

In both sa11x0_dma_remove() and the error path of sa11x0_dma_probe(),
sa11x0_dma_free_channels() is called before the device interrupts are freed
and the main device tasklet is killed:

sa11x0_dma_remove() {
	...
	sa11x0_dma_free_channels(&d->slave);
	for (pch = 0; pch < NR_PHY_CHAN; pch++)
		sa11x0_dma_free_irq(pdev, pch, &d->phy[pch]);
	tasklet_kill(&d->task);
	...
}

If an unbind happens while a DMA transfer is active, could an interrupt fire
during this window? If so, would sa11x0_dma_irq() dereference a freed channel
(p->vchan) and cause a use-after-free when attempting to acquire c->vc.lock?

Additionally, could the lockless traversal of the d->slave.channels list in
sa11x0_dma_tasklet() encounter list poisoning or a use-after-free if channels
are freed concurrently by the list_del() and kfree() operations here?

>  	}
>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=30

Re: [PATCH v2 36/64] dmaengine: sprd-dma: kill vchan BH on teardown

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] The driver prematurely removes channels from `sdev->dma_dev.channels` before calling `dma_async_device_unregister()`, causing the core DMA framework to silently skip unregistering them, which leaks per-CPU memory, IDAs, and sysfs nodes, leading to a Use-After-Free.
- [High] IRQ is requested before the MMIO base is mapped and clocks are enabled, leading to a guaranteed NULL pointer dereference or bus hang if an interrupt is pending.
--

--- Patch [36]: [PATCH v2 36/64] dmaengine: sprd-dma: kill vchan BH on teardown ---
commit 90cb440f02a76b9f95f830c9b7cd74c3b47c9d5f
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: sprd-dma: kill vchan BH on teardown
    
Use dma_chan_kill_bh() for virt-dma cleanup.

> diff --git a/drivers/dma/sprd-dma.c b/drivers/dma/sprd-dma.c
> index 087fea3af2e41..42a0c1c34337b 100644
> --- a/drivers/dma/sprd-dma.c
> +++ b/drivers/dma/sprd-dma.c
> @@ -1253,7 +1253,7 @@ static void sprd_dma_remove(struct platform_device *pdev)
>  	list_for_each_entry_safe(c, cn, &sdev->dma_dev.channels,
>  				 vc.chan.device_node) {
>  		list_del(&c->vc.chan.device_node);

[Severity: High]
This isn't a bug introduced by this patch, but removing the channels from
sdev->dma_dev.channels here causes the core dma framework to skip
unregistering them when dma_async_device_unregister() is called later in
this function:

drivers/dma/dmaengine.c:dma_async_device_unregister() {
    ...
    list_for_each_entry_safe(chan, n, &device->channels, device_node)
        __dma_async_device_channel_unregister(device, chan);
    ...
}

By emptying the channel list early, the sysfs nodes for the channels are
never unregistered.

Since the driver data is devres-managed and freed after unbind, does this
leave orphaned sysfs nodes pointing to freed memory, leading to a
use-after-free if a user reads the sysfs files?

> -		tasklet_kill(&c->vc.task);
> +		dma_chan_kill_bh(&c->vc.chan);
>  	}
>  
>  	of_dma_controller_free(pdev->dev.of_node);

[Severity: High]
This is another pre-existing issue, but there appears to be a null pointer
dereference risk during probe in this driver.

The interrupt is requested before the mmio base is mapped:

drivers/dma/sprd-dma.c:sprd_dma_probe() {
    ...
    ret = devm_request_irq(&pdev->dev, sdev->irq, dma_irq_handle,
                           0, "sprd_dma", (void *)sdev);
    ...
    sdev->glb_base = devm_platform_ioremap_resource(pdev, 0);
    ...
}

If an interrupt is already pending when devm_request_irq() unmasks it,
the interrupt handler will fire immediately and dereference the uninitialized
glb_base pointer:

drivers/dma/sprd-dma.c:dma_irq_handle() {
    ...
    u32 irq_status = readl(sdev->glb_base + SPRD_DMA_GLB_INT_MSK_STS);
    ...
}

Should the memory mapping and clock enablement be done before requesting
the interrupt to avoid this panic during boot or module load?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=36

Re: [PATCH v2 37/64] dmaengine: idma64: kill vchan BH on teardown

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

Pre-existing issues:
- [High] The channel cleanup loop in idma64_remove() is dead code, leading to a Use-After-Free of bottom-half work items.
- [High] A race condition during device removal can cause an unhandled interrupt storm.
- [High] Use-After-Free due to missing device_release callback with devres-managed core structure.
--

--- Patch [37]: [PATCH v2 37/64] dmaengine: idma64: kill vchan BH on teardown ---
commit e368659298f3842bf78bcfc4c53afeb968ac845c
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: idma64: kill vchan BH on teardown

Use dma_chan_kill_bh() for virt-dma cleanup.

> diff --git a/drivers/dma/idma64.c b/drivers/dma/idma64.c
> index 5fcd1befc92da..8e25d828e553d 100644
> --- a/drivers/dma/idma64.c
> +++ b/drivers/dma/idma64.c
> @@ -617,14 +617,14 @@ static void idma64_remove(struct idma64_chip *chip)
>  
>  	/*
>  	 * Explicitly call devm_request_irq() to avoid the side effects with
> -	 * the scheduled tasklets.
> +	 * scheduled BH work.
>  	 */
>  	devm_free_irq(chip->dev, chip->irq, idma64);

[Severity: High]
This is a pre-existing issue, but does calling devm_free_irq() after
dma_async_device_unregister() create a race condition that can cause an
interrupt storm?

The earlier call to dma_async_device_unregister() iterates over the channels
and decrements idma64->dma.chancnt to 0.

If a hardware interrupt triggers in the narrow window before devm_free_irq()
disables it, idma64_irq() executes this loop:

idma64_irq() {
    ...
    for (i = 0; i < idma64->dma.chancnt; i++)
        idma64_chan_irq(idma64, i, status_err, status_xfer);
    ...
}

Because chancnt is 0, it skips clearing the active hardware interrupt but
still returns IRQ_HANDLED. Will the hardware continually re-assert the
uncleared interrupt and hang the CPU?

>  
>  	for (i = 0; i < idma64->dma.chancnt; i++) {
>  		struct idma64_chan *idma64c = &idma64->chan[i];
>  
> -		tasklet_kill(&idma64c->vchan.task);
> +		dma_chan_kill_bh(&idma64c->vchan.chan);
>  	}
>  }

[Severity: High]
This is a pre-existing issue, but will this cleanup loop ever execute?

Since the preceding call to dma_async_device_unregister() decrements
idma64->dma.chancnt to 0 before we reach this loop, the loop is completely
skipped and dma_chan_kill_bh() is never called.

When devres subsequently frees the idma64 structure, won't any pending
bottom-half work items execute on freed memory?

[Severity: High]
This is also a pre-existing issue, but is there a use-after-free risk because
struct idma64 is allocated with devm_kzalloc() but lacks a
dma_device->device_release callback?

When the driver unbinds and idma64_remove() completes, devres will
automatically free the idma64 structure:

idma64_probe() {
    ...
    idma64 = devm_kzalloc(chip->dev, sizeof(*idma64), GFP_KERNEL);
    ...
}

If a DMA client still holds a reference to a channel, won't the client
dereference the freed idma64 memory when accessing or releasing the channel
later, since the DMA core manages reference counts via device->ref?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=37

Re: [PATCH v2 31/64] dmaengine: pl330: route callbacks via channel BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:

New issues:
- [High] Calling dma_chan_init_bh() in the dynamic channel allocation path triggers a kernel WARN_ON when a channel is freed and subsequently re-allocated.

Pre-existing issues:
- [High] Concurrent channel error processing can schedule the bottom-half work after the channel thread is set to NULL during channel release, leading to a NULL pointer dereference.
- [High] The `desc_pool` list in the DMA controller structure is modified without holding `pool_lock` in `pl330_tasklet`, leading to linked list corruption.
- [High] Calling `pl330_tasklet` directly from `pl330_issue_pending` causes completed DMA callbacks to run synchronously in the caller's context, leading to deadlocks.
--

--- Patch [31]: [PATCH v2 31/64] dmaengine: pl330: route callbacks via channel BH ---
commit 03f651c688fce11fe6da583209843570b5a94019
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: pl330: route callbacks via channel BH

Convert the per-channel tasklet to dma_chan_*_bh while keeping the
controller tasklet for non-callback events.

> diff --git a/drivers/dma/pl330.c b/drivers/dma/pl330.c
> index 25ba84b187041..f4940bd1ab849 100644
> --- a/drivers/dma/pl330.c
> +++ b/drivers/dma/pl330.c
[ ... ]
> @@ -2060,14 +2057,15 @@ static inline void fill_queue(struct dma_pl330_chan *pch)
>  			desc->status = DONE;
>  			dev_err(pch->dmac->ddma.dev, "%s:%d Bad Desc(%d)\n",
>  					__func__, __LINE__, desc->txd.cookie);
> -			tasklet_schedule(&pch->task);
> +			dma_chan_schedule_bh(&pch->chan);
>  		}
>  	}
>  }
>  
> -static void pl330_tasklet(struct tasklet_struct *t)
> +static void pl330_tasklet(struct dma_chan *chan)
>  {
> -	struct dma_pl330_chan *pch = from_tasklet(pch, t, task);
> +	struct dma_pl330_chan *pch = container_of(chan, struct dma_pl330_chan,
> +						  chan);
>  	struct dma_pl330_desc *desc, *_dt;

[Severity: High]
This is a pre-existing issue, but can this function crash due to a NULL
pointer dereference if it races with a concurrent channel release?

When pl330_free_chan_resources() kills the bottom-half, it sets pch->thread
to NULL:

pl330_free_chan_resources()
    dma_chan_kill_bh(&pch->chan);
    ...
    pl330_release_channel(pch->thread);
    pch->thread = NULL;

If a concurrent DMAC tasklet calls dma_pl330_rqcb(), it unconditionally
queues the bottom-half again:

dma_pl330_rqcb()
    dma_chan_schedule_bh(&pch->chan);

Since dma_chan_schedule_bh() is not blocked after kill_bh, the workqueue
could execute pl330_tasklet() on the freed channel. Later in this function,
we acquire a lock via the thread pointer:

pl330_tasklet()
    spin_lock(&pch->thread->dmac->lock);

Would this dereference the NULL thread pointer?

[Severity: High]
This is also a pre-existing issue, but does moving descriptors to the global
desc_pool here corrupt the linked list?

Later in pl330_tasklet(), completed non-cyclic descriptors are moved back to
the global pool:

pl330_tasklet()
    list_move_tail(&desc->node, &pch->dmac->desc_pool);

The pch->dmac->desc_pool list is shared across the controller and normally
protected by pool_lock, but pl330_tasklet() only holds the channel-specific
pch->lock here. Could concurrent modifications to desc_pool from multiple
channels or allocations cause list corruption?

[ ... ]
> @@ -2173,7 +2171,7 @@ static int pl330_alloc_chan_resources(struct dma_chan *chan)
>  		return -ENOMEM;
>  	}
>  
> -	tasklet_setup(&pch->task, pl330_tasklet);
> +	dma_chan_init_bh(&pch->chan, pl330_tasklet);
>  
>  	spin_unlock_irqrestore(&pl330->lock, flags);

[Severity: High]
Does initializing the bottom-half work structure here trigger a warning on
subsequent channel allocations?

pl330_alloc_chan_resources() is called each time a DMA client dynamically
requests a channel. When the client releases the channel, dma_chan_kill_bh()
flushes the workqueue but does not clear the bh_work_initialized flag.

If a client requests this same channel again, calling dma_chan_init_bh()
will hit the WARN_ON(chan->bh_work_initialized) guard in the DMA engine core.
Should this initialization be moved to driver probe time instead?

[ ... ]
> @@ -2490,7 +2488,7 @@ static void pl330_issue_pending(struct dma_chan *chan)
>  	list_splice_tail_init(&pch->submitted_list, &pch->work_list);
>  	spin_unlock_irqrestore(&pch->lock, flags);
>  
> -	pl330_tasklet(&pch->task);
> +	pl330_tasklet(&pch->chan);
>  }

[Severity: High]
This is a pre-existing issue, but does calling the bottom-half worker
directly from the issue_pending path risk deadlocks?

Clients of the DMA engine API typically call dma_async_issue_pending()
while holding their own spinlocks. By invoking pl330_tasklet() synchronously
here:

pl330_issue_pending()
    pl330_tasklet(&pch->chan);

If any descriptors were marked as DONE by a concurrent IRQ before the
scheduled bottom-half could run, pl330_tasklet() will immediately process
them and invoke their registered client DMA callbacks inline.

If the client callback attempts to acquire the same spinlock the client
already holds, will this cause an AB-BA or AA deadlock?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=31

Re: [PATCH v2 64/64] dmaengine: switchtec: use channel BH helpers

[Logan Gunthorpe]

On 2026-07-27 14:39, Allen Pais wrote:
> Replace the per-channel descriptor completion tasklet with the shared
> dma_chan BH helpers. Client callbacks remain in softirq context while
> using the common scheduling and teardown mechanism.
> 
> Signed-off-by: Allen Pais <allen.lkml@gmail.com>

This one also looks good to me, thanks.

Reviewed-by: Logan Gunthorpe <logang@deltatee.com>

Re: [PATCH v2 33/64] dmaengine: sun6i: kill vchan BH on teardown

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Client callbacks flushed during cleanup can reschedule the driver tasklet after it was killed and clocks were disabled, leading to a hardware fault and list corruption.
--

--- Patch [33]: [PATCH v2 33/64] dmaengine: sun6i: kill vchan BH on teardown ---
commit 0c78bc25a98ea85ad52645998923d5aa57383201
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: sun6i: kill vchan BH on teardown
    
Use dma_chan_kill_bh() for virt-dma channel cleanup.

> diff --git a/drivers/dma/sun6i-dma.c b/drivers/dma/sun6i-dma.c
> index a9a254dbf8cb9..9289c24dfcfaf 100644
> --- a/drivers/dma/sun6i-dma.c
> +++ b/drivers/dma/sun6i-dma.c
> @@ -1097,7 +1097,7 @@ static inline void sun6i_dma_free(struct sun6i_dma_dev *sdev)
>  		struct sun6i_vchan *vchan = &sdev->vchans[i];
>  
>  		list_del(&vchan->vc.chan.device_node);
> -		tasklet_kill(&vchan->vc.task);
> +		dma_chan_kill_bh(&vchan->vc.chan);

[Severity: High]
This is a pre-existing issue, but could this cleanup sequence trigger a
hardware fault or list corruption if a client callback submits a new
transfer?

During device removal or sysfs unbind, sun6i_dma_remove() kills the
hardware tasklet and disables controller clocks before calling
sun6i_dma_free():

drivers/dma/sun6i-dma.c:sun6i_dma_remove() {
    ...
    sun6i_kill_tasklet(sdc);

    clk_disable_unprepare(sdc->clk_mbus);
    clk_disable_unprepare(sdc->clk);
    reset_control_assert(sdc->rstc);

    sun6i_dma_free(sdc);
}

sun6i_dma_free() then calls dma_chan_kill_bh() which flushes pending work.
If a client callback runs during this flush and submits a new transfer,
sun6i_dma_issue_pending() is invoked:

drivers/dma/sun6i-dma.c:sun6i_dma_issue_pending() {
    ...
    list_add_tail(&vchan->node, &sdev->pending);
    tasklet_schedule(&sdev->task);
}

Because tasklet_schedule() is called unconditionally without checking if
the device is tearing down, it resurrects the killed tasklet.

When the scheduled tasklet runs, it accesses hardware registers without
clocks enabled, leading to an external abort. Additionally, it concurrently
accesses the channel lists while sun6i_dma_free() mutates them via
list_del() without locks.

Should the teardown order in sun6i_dma_remove() be adjusted to flush
callbacks before disabling clocks, or should sun6i_dma_issue_pending()
check a shutdown flag before scheduling the tasklet?

>  	}
>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=33

Re: [PATCH v2 44/64] dmaengine: mtk-hsdma: kill vchan BH on teardown

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] Killing channel bottom-halves before disabling hardware interrupts allows pending work to be maliciously re-queued and executed after channel memory is freed.
- [High] Manual removal of virtual channels from the device list bypasses the DMA core's cleanup, leaking per-CPU memory and sysfs resources.
--

--- Patch [44]: [PATCH v2 44/64] dmaengine: mtk-hsdma: kill vchan BH on teardown ---
commit 2d206408161f8be2e557e668a6b7dd498f1ec43d
Author: Allen Pais <allen.lkml@gmail.com>
Subject: dmaengine: mtk-hsdma: kill vchan BH on teardown

This commit replaces the tasklet cleanup with dma_chan_kill_bh() for virt-dma
cleanup in the mtk-hsdma driver's teardown path.

> diff --git a/drivers/dma/mediatek/mtk-hsdma.c b/drivers/dma/mediatek/mtk-hsdma.c
> index a43412ff5eddc..1c98fcedf3acd 100644
> --- a/drivers/dma/mediatek/mtk-hsdma.c
> +++ b/drivers/dma/mediatek/mtk-hsdma.c
> @@ -1020,7 +1020,7 @@ static void mtk_hsdma_remove(struct platform_device *pdev)
>  		vc = &hsdma->vc[i];
>  
>  		list_del(&vc->vc.chan.device_node);

[Severity: High]
This is a pre-existing issue, but does removing the channel from the device
list here bypass the DMA core's cleanup?

The core function dma_async_device_unregister() relies on iterating over the
device->channels list to execute __dma_async_device_channel_unregister(),
which frees the per-CPU memory and unregisters the sysfs devices. Because the
list is emptied preemptively before calling dma_async_device_unregister(),
the core cleanup loop might be bypassed, which could lead to leaking per-cpu
variables and orphaned sysfs nodes.

> -		tasklet_kill(&vc->vc.task);
> +		dma_chan_kill_bh(&vc->vc.chan);
>  	}
>  
>  	/* Disable DMA interrupt */

[Severity: High]
This is also a pre-existing issue, but does killing the channel bottom-halves
before disabling hardware interrupts allow pending work to be maliciously
re-queued?

mtk_hsdma_remove() terminates the asynchronous workers here, but the interrupt
is disabled and synchronized later in the function:

	/* Disable DMA interrupt */
	mtk_dma_write(hsdma, MTK_HSDMA_INT_ENABLE, 0);

	/* Waits for any pending IRQ handlers to complete */
	synchronize_irq(hsdma->irq);

If an interrupt triggers or is handled concurrently during this window, the
ISR (mtk_hsdma_irq) could complete descriptors and call
vchan_cookie_complete(), which re-schedules the channel's bottom-half onto
the workqueue. Since the driver continues unbinding and eventually frees the
channel structures, the re-scheduled work might execute later and access
freed memory.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=44

Re: [PATCH v2 34/64] dmaengine: mtk-cqdma: kill vchan BH on teardown

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

Pre-existing issues:
- [High] Incorrect teardown order in mtk_cqdma_remove() allows physical channel tasklets to reschedule virtual channel bottom halves after they have been killed, leading to a use-after-free.
- [High] Manual removal of channels from the DMA device channel list via list_del() before unregistration bypasses dmaengine cleanup, leaking memory and IDA indexes.
- [High] The DMA device is registered before physical channel tasklets and hardware are initialized, allowing early interrupts to schedule uninitialized tasklets.
--

--- Patch [34]: [PATCH v2 34/64] dmaengine: mtk-cqdma: kill vchan BH on teardown ---
commit 94dd78bb3e2fc0bc1ffe5fdde7322901f8e3ea2c
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: mtk-cqdma: kill vchan BH on teardown
    
This patch updates the virt-dma channel cleanup in the MediaTek CQDMA driver
to use the standard dma_chan_kill_bh() helper rather than directly calling
tasklet_kill().

> diff --git a/drivers/dma/mediatek/mtk-cqdma.c b/drivers/dma/mediatek/mtk-cqdma.c
> index 80791e30aec24..3b0f1d8fa2055 100644
> --- a/drivers/dma/mediatek/mtk-cqdma.c
> +++ b/drivers/dma/mediatek/mtk-cqdma.c
> @@ -895,7 +895,7 @@ static void mtk_cqdma_remove(struct platform_device *pdev)
>  		vc = &cqdma->vc[i];
>  
>  		list_del(&vc->vc.chan.device_node);

[Severity: High]
This isn't a bug introduced by this patch, but does calling list_del() here
cause a memory and IDA index leak?

By manually removing the channel from the dmaengine device list before calling
dma_async_device_unregister(), the core subsystem's unregistration loop will
see an empty list.

This causes __dma_async_device_channel_unregister() to be skipped, which
means free_percpu(), sysfs cleanup, and ida_free() are never executed.

> -		tasklet_kill(&vc->vc.task);
> +		dma_chan_kill_bh(&vc->vc.chan);

[Severity: High]
This is a pre-existing issue, but does this teardown order leave a window for
a use-after-free?

If a physical channel interrupt fires after dma_chan_kill_bh() is called but
before the hardware interrupts are disabled and tasklets are killed below,
the interrupt handler will schedule the physical channel tasklet.

This tasklet processes completions and calls vchan_cookie_complete(), which
re-queues the virtual channel bottom-half work. After mtk_cqdma_remove()
returns, devres frees the cqdma device, leaving the re-queued work item
referencing freed memory.

Would it be safer to disable physical interrupts and kill tasklets before
killing the virtual channel bottom halves?

>  	}
>  
>  	/* disable interrupt */

[ ... ]

While reviewing this file, another pre-existing issue was noticed in
mtk_cqdma_probe():

[Severity: High]
This is a pre-existing issue, but is the DMA device registered before the
physical channel tasklets and hardware are fully initialized?

drivers/dma/mediatek/mtk-cqdma.c:mtk_cqdma_probe() {
    ...
    err = dma_async_device_register(dd);
    ...
    err = mtk_cqdma_hw_init(cqdma);
    ...
    for (i = 0; i < cqdma->dma_channels; ++i)
        tasklet_setup(&cqdma->pc[i]->tasklet, mtk_cqdma_tasklet_cb);
    ...
}

Once dma_async_device_register() is called, the controller is globally
visible. If a client probes asynchronously and immediately requests a channel,
it could trigger an early interrupt before tasklet_setup() completes,
potentially dereferencing a NULL pointer for the uninitialized tasklet
callback.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=34

Re: [PATCH v2 23/64] dmaengine: timb_dma: route callbacks via channel BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [High] Data race and out-of-bounds read in __td_ier_mask due to lockless traversal of active_list racing with asynchronous workqueue.

Pre-existing issues:
- [High] Premature completion of active transfers due to a race between td_issue_pending and timb_dma_chan_bh.
--

--- Patch [23]: [PATCH v2 23/64] dmaengine: timb_dma: route callbacks via channel BH ---
commit 2a07cb9b339578f9caa75d42037da515bda38de2
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: timb_dma: route callbacks via channel BH

This patch schedules per-channel BH work for completions so callbacks run
through the dma_chan deferral mechanism while keeping the controller
tasklet for IRQ handling.

> diff --git a/drivers/dma/timb_dma.c b/drivers/dma/timb_dma.c
> --- a/drivers/dma/timb_dma.c
> +++ b/drivers/dma/timb_dma.c

[ ... ]

> @@ -580,17 +582,25 @@ static void td_tasklet(struct tasklet_struct *t)
>  	for (i = 0; i < td->dma.chancnt; i++)
>  		if (ipr & (1 << i)) {
>  			struct timb_dma_chan *td_chan = td->channels + i;
> -			spin_lock(&td_chan->lock);
> -			__td_finish(td_chan);
> -			if (!list_empty(&td_chan->queue))
> -				__td_start_next(td_chan);
> -			spin_unlock(&td_chan->lock);
> +			dma_chan_schedule_bh(&td_chan->chan);
>  		}
>  
>  	ier = __td_ier_mask(td);

[Severity: High]
Can this lead to an out-of-bounds memory read?

With completion processing moved to an asynchronous workqueue via
dma_chan_schedule_bh(), td_tasklet() and timb_dma_chan_bh() can now run
concurrently. The tasklet calls __td_ier_mask() right here, which
locklessly evaluates td_chan->ongoing and traverses td_chan->active_list.

If __td_ier_mask() reads ongoing as true right before the workqueue removes
the final descriptor from active_list in __td_finish(), wouldn't list_entry()
calculate a bogus pointer relative to the empty list head? This could cause a
read of unmapped memory or garbage when accessing td_desc->interrupt.

>  	iowrite32(ier, td->membase + TIMBDMA_IER);
>  }
>  
> +static void timb_dma_chan_bh(struct dma_chan *chan)
> +{
> +	struct timb_dma_chan *td_chan = container_of(chan, struct timb_dma_chan,
> +						     chan);
> +
> +	spin_lock(&td_chan->lock);
> +	__td_finish(td_chan);

[Severity: High]
This is a pre-existing issue, but does moving to an asynchronous workqueue
widen a race condition here?

Currently, if td_issue_pending() races with a hardware interrupt, it can
acknowledge the interrupt and complete the transfer before the completion
path runs.

If td_issue_pending() also started the next transfer, this newly scheduled
timb_dma_chan_bh() workqueue will blindly call __td_finish() and complete
whatever is currently in the active_list. By changing the completion from a
tasklet (which had a very short race window) to a scheduled workqueue, the
delay is much longer. Will this result in erroneously aborting and completing
the newly started transfer?

> +	if (!list_empty(&td_chan->queue))
> +		__td_start_next(td_chan);
> +	spin_unlock(&td_chan->lock);
> +}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=23

Re: [PATCH v2 42/64] dmaengine: jz4780: kill vchan BH on teardown

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

Pre-existing issues:
- [Critical] Use-after-free and arbitrary physical memory corruption during descriptor setup failure
- [High] Hardware teardown race in jz4780_dma_remove due to incorrect devres ordering
- [High] NULL Pointer Dereference in jz4780_dma_tx_status
--

--- Patch [42]: [PATCH v2 42/64] dmaengine: jz4780: kill vchan BH on teardown ---
commit 80e35ad454b351671c271c48790a44cd6f63d096
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: jz4780: kill vchan BH on teardown

This commit updates virt-dma cleanup to use dma_chan_kill_bh() instead of
tasklet_kill().

> diff --git a/drivers/dma/dma-jz4780.c b/drivers/dma/dma-jz4780.c
> index 6070dfdb7114c..45c1096563e32 100644
> --- a/drivers/dma/dma-jz4780.c
> +++ b/drivers/dma/dma-jz4780.c
> @@ -1019,7 +1019,7 @@ static void jz4780_dma_remove(struct platform_device *pdev)
>  	free_irq(jzdma->irq, jzdma);
>  
>  	for (i = 0; i < jzdma->soc_data->nb_channels; i++)
> -		tasklet_kill(&jzdma->chan[i].vchan.task);
> +		dma_chan_kill_bh(&jzdma->chan[i].vchan.chan);

[Severity: High]
This is a pre-existing issue, but does disabling the hardware clock here
before devres gets a chance to unregister the DMA device create a race
condition?

If an interrupt fires or a client accesses the DMA engine (for example,
calling dmaengine_terminate_all()) during or immediately after
jz4780_dma_remove(), it appears this would trigger a fatal MMIO fault
because the device clock is already disabled.

>  }
>  
>  static const struct jz4780_dma_soc_data jz4740_dma_soc_data = {

[Severity: High]
This is a pre-existing issue, but is there a risk of a NULL pointer
dereference in jz4780_dma_tx_status() when a transfer isn't active?

drivers/dma/dma-jz4780.c:jz4780_dma_tx_status() {
    ...
	} else if (cookie == jzchan->desc->vdesc.tx.cookie) {
    ...
}

If a client prepares and submits a descriptor, moving it to the desc_submitted
list, and then polls its status before it is issued, vchan_find_desc()
would return NULL. If no transfer is active, jzchan->desc is NULL, which
seems to lead to an unguarded dereference in this branch.


[Severity: Critical]
This is a pre-existing issue, but could the error paths during descriptor
setup lead to a use-after-free and potential physical memory corruption?

drivers/dma/dma-jz4780.c:jz4780_dma_prep_slave_sg() {
    ...
			jz4780_dma_desc_free(&jzchan->desc->vdesc);
			return NULL;
    ...
}

If jz4780_dma_setup_hwdesc() fails, the error path frees jzchan->desc.
However, jzchan->desc appears to hold the currently active, running
descriptor rather than the newly allocated one. Does this free the running
descriptor while the DMA controller is still executing it? This pattern also
appears to exist in jz4780_dma_prep_dma_cyclic().

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=42

Re: [PATCH v2 28/64] dmaengine: plx_dma: use channel BH helpers

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:

Pre-existing issues:
- [High] Deadlock risk due to invoking DMA client callbacks with a spinlock held.
- [High] Deadlock risk due to inadequate lock context (plain spin_lock instead of spin_lock_bh) exposing the lock to softirq preemption.
- [High] Unbounded spinning (1000ms busy-wait) inside an RCU read-side critical section.
- [High] plx_dma_prep_memcpy() acquires a spinlock and returns with it held, violating locking and API rules.
- [High] The device IRQ is requested before the MMIO base pointer (`plxdev->bar`) is initialized, causing a potential NULL pointer dereference.
--

--- Patch [28]: [PATCH v2 28/64] dmaengine: plx_dma: use channel BH helpers ---
commit 978a06c861c9b53f0d972a684863de23c7d55eb8
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: plx_dma: use channel BH helpers

Replace the device tasklet with the per-channel BH helper so completion
callbacks run via dma_chan_*_bh.

> diff --git a/drivers/dma/plx_dma.c b/drivers/dma/plx_dma.c
> index 84941a918b01f..9d6bbdfa5a698 100644
> --- a/drivers/dma/plx_dma.c
> +++ b/drivers/dma/plx_dma.c
[ ... ]
> @@ -241,9 +240,10 @@ static void plx_dma_stop(struct plx_dma_dev *plxdev)
>  	rcu_read_unlock();
>  }

[Severity: High]
This isn't a bug introduced by this patch, but can this busy-wait loop
cause issues within the RCU read-side critical section?

plx_dma_stop() initiates an RCU read-side critical section and then calls
__plx_dma_stop():

plx_dma_stop()
{
    rcu_read_lock();
    ...
    __plx_dma_stop(plxdev);
}

__plx_dma_stop() then executes a busy-wait loop for up to 1000ms:

__plx_dma_stop()
{
    while (!time_after(jiffies, timeout)) {
        if (val & PLX_REG_CTRL_GRACEFUL_PAUSE_DONE)
            break;
        cpu_relax();
    }
}

Executing a 1-second busy loop inside an RCU read-side critical section could
lead to severe scheduling latency or RCU stalls.

> -static void plx_dma_desc_task(struct tasklet_struct *t)
> +static void plx_dma_desc_task(struct dma_chan *chan)
>  {
> -	struct plx_dma_dev *plxdev = from_tasklet(plxdev, t, desc_task);
> +	struct plx_dma_dev *plxdev = container_of(chan, struct plx_dma_dev,
> +						  dma_chan);
>  
>  	plx_dma_process_desc(plxdev);
>  }

[Severity: High]
This isn't a bug introduced by this patch, but could plx_dma_process_desc()
lead to a deadlock?

A client DMA driver submitting a new DMA request from within its completion
callback might cause a self-deadlock. The callback is invoked in
plx_dma_process_desc() while holding the ring_lock:

plx_dma_process_desc()
{
    spin_lock(&plxdev->ring_lock);
    ...
    dmaengine_desc_get_callback_invoke(&desc->txd, &res);
}

If the client's callback calls dmaengine_prep_dma_memcpy(), it will invoke
plx_dma_prep_memcpy() which attempts to acquire the same ring_lock.

[Severity: High]
This is also a pre-existing issue, but does ring_lock need to be acquired
with bottom halves disabled?

If a process context function like plx_dma_tx_status() calls
plx_dma_process_desc() and acquires ring_lock without disabling bottom halves:

plx_dma_process_desc()
{
    spin_lock(&plxdev->ring_lock);
}

A hardware interrupt could fire, and the softirq handler would then execute
plx_dma_desc_task() -> plx_dma_process_desc() on the same CPU, attempting
to acquire the already-held ring_lock, which could result in a hard deadlock.

[ ... ]
> @@ -510,7 +510,7 @@ static int plx_dma_create(struct pci_dev *pdev)
>  		goto free_plx;
>  
>  	spin_lock_init(&plxdev->ring_lock);
> -	tasklet_setup(&plxdev->desc_task, plx_dma_desc_task);
> +	dma_chan_init_bh(&plxdev->dma_chan, plx_dma_desc_task);
>  
>  	RCU_INIT_POINTER(plxdev->pdev, pdev);
>  	plxdev->bar = pcim_iomap_table(pdev)[0];

[Severity: High]
This isn't a bug introduced by this patch, but is there a race condition
during initialization?

The interrupt handler is registered before the MMIO region pointer is
initialized:

plx_dma_create()
{
    rc = request_irq(pci_irq_vector(pdev, 0), plx_dma_isr, ...);
    ...
    plxdev->bar = pcim_iomap_table(pdev)[0];
}

If an interrupt fires immediately after request_irq(), plx_dma_isr() could
read from plxdev->bar, which would still be an uninitialized pointer:

plx_dma_isr()
{
    status = readw(plxdev->bar + PLX_REG_INTR_STATUS);
}

[Severity: High]
This is another pre-existing issue outside the diff context, but does
plx_dma_prep_memcpy() intentionally return with the ring_lock held?

plx_dma_prep_memcpy() acquires the lock but returns without releasing it:

plx_dma_prep_memcpy()
{
    spin_lock_bh(&plxdev->ring_lock);
    ...
    return &plxdesc->txd;
}

The lock appears to only be released later in plx_dma_tx_submit(). If a client
prepares multiple descriptors before submitting, it might self-deadlock
trying to acquire the lock a second time. If it sleeps between prep and
submit, it would be sleeping in an atomic context.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=28

Re: [PATCH v2 38/64] dmaengine: img-mdc-dma: kill vchan BH on teardown

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing pm_runtime_set_suspended() in the driver removal path causes PM state mismatch on subsequent rebinds, leading to hardware faults.
- [High] The PM runtime usage counter is leaked in mdc_alloc_chan_resources() if pm_runtime_get_sync() fails.
- [High] Hardware registers are accessed in mdc_dma_probe() before the device clock is enabled.
- [High] The IRQ handler is registered before its associated virtual channel spinlock is initialized, allowing a spurious interrupt to lock an uninitialized spinlock.
--

--- Patch [38]: [PATCH v2 38/64] dmaengine: img-mdc-dma: kill vchan BH on teardown ---
commit b4d81ea29e678b96ef67f182eaf0b8fc7065fe66
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: img-mdc-dma: kill vchan BH on teardown

Use dma_chan_kill_bh() for virt-dma cleanup.

> diff --git a/drivers/dma/img-mdc-dma.c b/drivers/dma/img-mdc-dma.c
> index b3765ba158030..374034c737c0e 100644
> --- a/drivers/dma/img-mdc-dma.c
> +++ b/drivers/dma/img-mdc-dma.c
[ ... ]
> @@ -1031,7 +1031,7 @@ static void mdc_dma_remove(struct platform_device *pdev)
>  
>  		devm_free_irq(&pdev->dev, mchan->irq, mchan);
>  
> -		tasklet_kill(&mchan->vc.task);
> +		dma_chan_kill_bh(&mchan->vc.chan);
>  	}
>  
>  	pm_runtime_disable(&pdev->dev);
>  	if (!pm_runtime_status_suspended(&pdev->dev))
>  		img_mdc_runtime_suspend(&pdev->dev);

[Severity: High]
This is a pre-existing issue, but does mdc_dma_remove() need to call
pm_runtime_set_suspended() here?

When the driver is unbound and rebound via sysfs, the hardware is manually
suspended but the PM runtime state is left as RPM_ACTIVE. On rebind,
mdc_dma_probe() enables runtime PM but sees the device as already active.

When a channel is requested, won't pm_runtime_get_sync() return without
enabling the clock, leading to an imprecise external abort (kernel panic)
during subsequent register writes?


While reviewing this file, I noticed a few other pre-existing issues:

drivers/dma/img-mdc-dma.c:mdc_alloc_chan_resources()
> static int mdc_alloc_chan_resources(struct dma_chan *chan)
> {
> 	struct mdc_chan *mchan = to_mdc_chan(chan);
> 	struct device *dev = mdma2dev(mchan->mdma);
> 
> 	return pm_runtime_get_sync(dev);
> }

[Severity: High]
This isn't a bug introduced by this patch, but does mdc_alloc_chan_resources()
leak the PM runtime usage counter if pm_runtime_get_sync() fails?

If pm_runtime_get_sync() returns an error (e.g., -EACCES), the usage counter
is incremented but the error is directly propagated. Since the allocation
aborts, the free routine is never called to decrement the counter.

Doesn't this permanently prevent the device (and potentially the parent power
domain) from entering low-power suspend states? Should this check the return
value and call pm_runtime_put_noidle() on failure?


drivers/dma/img-mdc-dma.c:mdc_dma_probe()
> 	mdma->clk = devm_clk_get(&pdev->dev, "sys");
> 	if (IS_ERR(mdma->clk))
> 		return PTR_ERR(mdma->clk);
[ ... ]
> 	val = mdc_readl(mdma, MDC_GLOBAL_CONFIG_A);

[Severity: High]
This is a pre-existing issue, but are hardware registers accessed in
mdc_dma_probe() before the device clock is enabled?

mdc_readl() reads MDC_GLOBAL_CONFIG_A shortly after getting the "sys" clock,
but the clock appears to only be enabled later in img_mdc_runtime_resume().

On systems where the bootloader doesn't leave the clock enabled, wouldn't
reading from unclocked peripheral registers cause an imprecise external abort
(kernel panic) during device initialization?


drivers/dma/img-mdc-dma.c:mdc_dma_probe()
> 		ret = devm_request_irq(&pdev->dev, mchan->irq, mdc_chan_irq,
> 				       IRQ_TYPE_LEVEL_HIGH,
> 				       dev_name(&pdev->dev), mchan);
> 		if (ret < 0)
> 			return ret;
> 
> 		mchan->vc.desc_free = mdc_desc_free;
> 		vchan_init(&mchan->vc, &mdma->dma_dev);

[Severity: High]
This isn't a bug introduced by this patch, but is the IRQ handler registered
before its associated virtual channel spinlock is initialized?

devm_request_irq() is called before vchan_init() initializes mchan->vc.lock.
If the IRQ line is asserted immediately after registration (e.g. from a warm
boot or shared interrupt), mdc_chan_irq() will fire and attempt to acquire a
zero-initialized lock.

Could this trigger a kernel panic with lock debugging enabled?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=38

Re: [PATCH v2 32/64] dmaengine: k3-udma: use channel BH for vchan completions

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Dangling `vc->cyclic` pointer causes a Use-After-Free in `udma_vchan_complete` when a channel running a cyclic transfer is terminated.
--

--- Patch [32]: [PATCH v2 32/64] dmaengine: k3-udma: use channel BH for vchan completions ---
commit 8a0a241d872268ead0d3f5e593bb261aae3ac2ff
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: k3-udma: use channel BH for vchan completions
    
This patch converts the custom vchan completion handling in the k3-udma
driver from a tasklet to a per-channel bottom-half mechanism. It updates
the initialization and teardown paths to use the new dma_chan_kill_bh() API.

> diff --git a/drivers/dma/ti/k3-udma.c b/drivers/dma/ti/k3-udma.c
> index 1cf158eb7bdb5..12b8a9caf4d2b 100644
> --- a/drivers/dma/ti/k3-udma.c
> +++ b/drivers/dma/ti/k3-udma.c

[ ... ]

> @@ -4042,12 +4042,12 @@ static void udma_desc_pre_callback(struct virt_dma_chan *vc,
>  }
>  
>  /*
> - * This tasklet handles the completion of a DMA descriptor by
> + * This BH handles the completion of a DMA descriptor by
>   * calling its callback and freeing it.
>   */
> -static void udma_vchan_complete(struct tasklet_struct *t)
> +static void udma_vchan_complete(struct dma_chan *chan)
>  {
> -	struct virt_dma_chan *vc = from_tasklet(vc, t, task);
> +	struct virt_dma_chan *vc = to_virt_chan(chan);

[Severity: High]
This is a pre-existing issue, but could this bottom half handler trigger a
use-after-free of the cyclic descriptor?

Looking at the teardown path, if a channel is actively running a cyclic
transfer, udma_terminate_all() moves uc->desc to uc->terminated_desc but
crucially leaves vc->cyclic pointing to the active descriptor.

When the hardware sends a teardown completion message, the IRQ handler
udma_ring_irq_handler() frees the active descriptor via
udma_desc_free(&uc->terminated_desc->vd). At this point, vc->cyclic becomes
a dangling pointer.

The driver then calls udma_synchronize(), which calls vchan_synchronize().
This flushes the bottom half by calling dma_chan_kill_bh(), which happens
before vc->cyclic is checked and cleared:

drivers/dma/virt-dma.c:vchan_synchronize() {
    dma_chan_kill_bh(&vc->chan);

    spin_lock_irqsave(&vc->lock, flags);
    if (vc->cyclic) {
        vchan_terminate_vdesc(vc->cyclic);
        vc->cyclic = NULL;
    }
}

When the bottom half executes, it reads the dangling vc->cyclic pointer and
dereferences it:

    vd = vc->cyclic;
    if (vd) {
        vc->cyclic = NULL;
        dmaengine_desc_get_callback(&vd->tx, &cb);

Does this sequence cause a use-after-free when dmaengine_desc_get_callback()
accesses the freed descriptor?

>  	struct virt_dma_desc *vd, *_vd;
>  	struct dmaengine_desc_callback cb;
>  	LIST_HEAD(head);

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=32

Re: [PATCH v2 43/64] dmaengine: pxa_dma: kill vchan BH on teardown

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:

Pre-existing issues:
- [High] Prematurely deleting channels from `dmadev->channels` during driver removal bypasses the dmaengine core's unregistration logic, permanently breaking module reload.
- [High] Critical Use-After-Free vulnerability on driver unbind due to improper devres cleanup ordering and failure to quiesce the hardware.
- [High] Missing `of_dma_controller_free()` on driver removal leaves a dangling pointer in the OF DMA list.
- [High] Potential Use-After-Free on the probe error path due to missing synchronization of bottom-half work.
--

--- Patch [43]: [PATCH v2 43/64] dmaengine: pxa_dma: kill vchan BH on teardown ---
commit 542f54f4e01925d8e48b9a9c649593d23bb50a91
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: pxa_dma: kill vchan BH on teardown

Use dma_chan_kill_bh() for virt-dma cleanup.

> diff --git a/drivers/dma/pxa_dma.c b/drivers/dma/pxa_dma.c
> index fa2ee0b3e09f8..d57e5e0c422cb 100644
> --- a/drivers/dma/pxa_dma.c
> +++ b/drivers/dma/pxa_dma.c
> @@ -1215,7 +1215,7 @@ static void pxad_free_channels(struct dma_device *dmadev)
>  	list_for_each_entry_safe(c, cn, &dmadev->channels,
>  				 vc.chan.device_node) {
>  		list_del(&c->vc.chan.device_node);

[Severity: High]
This is a pre-existing issue, but does this premature list deletion bypass the
dmaengine core's unregistration logic?

In pxad_remove(), the driver calls pxad_free_channels(), which uses list_del()
to manually remove each channel from dmadev->channels. The core framework's
dma_async_device_unregister() is executed subsequently as a devres action and
relies on traversing dmadev->channels to unregister each channel.

Because the list has been artificially emptied,
__dma_async_device_channel_unregister() is never invoked, skipping
device_unregister(), ida_free(), and free_percpu(). Will this leak sysfs
nodes (e.g., "dma0chan0") and cause permanent sysfs name collisions on
subsequent module loads?

> -		tasklet_kill(&c->vc.task);
> +		dma_chan_kill_bh(&c->vc.chan);
>  	}
>  }

[Severity: High]
This is a pre-existing issue, but is there a use-after-free vulnerability on
driver unbind due to improper devres cleanup ordering?

During initialization, pxad_init_phys() requests IRQs via devm_request_irq(),
and subsequently pxad_init_dmadev() allocates channel memory via
devm_kzalloc(). Because devres releases resources in reverse order, the channel
memory is freed before the IRQ is released.

Since pxad_remove() fails to mask interrupts or explicitly stop the DMA
hardware, if the hardware asserts an interrupt during this window:

drivers/dma/pxa_dma.c:pxad_chan_handler() {
    ...
    struct pxad_chan *chan = phy->vchan;
    ...
    spin_lock(&chan->vc.lock);
    ...
}

Will this dereference the already freed phy->vchan pointer and attempt to
acquire a freed spinlock if an interrupt fires during teardown?

[Severity: High]
This is also a pre-existing issue, but does driver removal leave a dangling
pointer in the OF DMA list?

In pxad_probe(), the driver registers its DMA controller using
of_dma_controller_register(), adding an of_dma structure to the global
of_dma_list. However, neither pxad_remove() nor any error path calls
of_dma_controller_free().

When the driver is unbound, will the of_dma structure remain in the global
list, containing pointers to the freed pdev memory and the unloaded
pxad_dma_xlate code, leading to a use-after-free if subsequent DMA requests
from other devices traverse the DT?

[Severity: High]
This is a pre-existing issue, but is there a potential use-after-free on the
probe error path due to missing synchronization of bottom-half work?

In pxad_probe(), if of_dma_controller_register() fails, the function returns
an error without flushing bottom-half work:

drivers/dma/pxa_dma.c:pxad_probe() {
    ...
    if (ret < 0) {
        dev_err(pdev->slave.dev,
                "of_dma_controller_register failed\n");
        return ret;
    }
    ...
}

Because dmaenginem_async_device_register() executes prior to this failure, the
DMA controller is briefly globally visible. If an external client requests a
channel and queues bh_work during this window, will the devres cleanup free
the channel memory before the asynchronous work executes?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=43

Re: [PATCH v2 41/64] dmaengine: hsu: kill vchan BH on teardown

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The patch series structure breaks `git bisect` due to a build failure in the `hsu` driver.
--

--- Patch [41]: [PATCH v2 41/64] dmaengine: hsu: kill vchan BH on teardown ---
commit 53bdccad0464307508ef64c4477dc5f1414579fd
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: hsu: kill vchan BH on teardown

Use dma_chan_kill_bh() for virt-dma cleanup.

> diff --git a/drivers/dma/hsu/hsu.c b/drivers/dma/hsu/hsu.c
> index f62d60d7bc6b7..1a7f9df735a43 100644
> --- a/drivers/dma/hsu/hsu.c
> +++ b/drivers/dma/hsu/hsu.c
> @@ -500,7 +500,7 @@ int hsu_dma_remove(struct hsu_dma_chip *chip)
>  	for (i = 0; i < hsu->nr_channels; i++) {
>  		struct hsu_dma_chan *hsuc = &hsu->chan[i];
>  
> -		tasklet_kill(&hsuc->vchan.task);

[Severity: Medium]
Will this series structure break git bisect?

If the earlier commit "dmaengine: add tasklet-backed channel BH helpers"
removes the task member from struct virt_dma_chan, any kernel checkout
between that commit and this one will fail to compile if the hsu driver is
enabled because it still references hsuc->vchan.task in hsu_dma_remove().

Should the series be reordered so that affected drivers are converted to use the
new API before the task member is removed?

> +		dma_chan_kill_bh(&hsuc->vchan.chan);
>  	}
>  
>  	return 0;

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=41

Re: [PATCH v2 40/64] dmaengine: dw-axi-dmac: kill vchan BH on teardown

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] Manual deletion of DMA channels from the device list in dw_remove() sabotages the DMA engine core's unregistration process, leading to memory, IDA, and sysfs leaks.
- [High] Flushing the completion bottom half (dma_chan_kill_bh) after disabling hardware clocks can cause a system hang if a client callback issues a new transfer.
--

--- Patch [40]: [PATCH v2 40/64] dmaengine: dw-axi-dmac: kill vchan BH on teardown ---
commit d93e02ecaf0e9550e4241cd3c6ed8d45227cb4a9
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: dw-axi-dmac: kill vchan BH on teardown

This commit updates virt-dma cleanup during teardown to use
dma_chan_kill_bh() instead of tasklet_kill().

> diff --git a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> index bcefaff03b5c6..5a5d569b188d3 100644
> --- a/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> +++ b/drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c
> @@ -1663,7 +1663,7 @@ static void dw_remove(struct platform_device *pdev)
>  	list_for_each_entry_safe(chan, _chan, &dw->dma.channels,
>  				 vc.chan.device_node) {
>  		list_del(&chan->vc.chan.device_node);

[Severity: High]
This isn't a bug introduced by this patch, but does calling list_del()
on the channel here sabotage the DMA engine core's unregistration process?

The DMA core's dma_async_device_unregister() uses the device->channels list
to tear down each channel. By manually removing the channels from this list
in dw_remove(), the core teardown logic sees an empty list and skips channel
unregistration.

Does this leak the channel's IDA, its sysfs device node (dmaXchanY), and
its dynamically allocated per-CPU memory (chan->local)?

> -		tasklet_kill(&chan->vc.task);
> +		dma_chan_kill_bh(&chan->vc.chan);

[Severity: High]
This is a pre-existing issue, but does flushing the bottom half here risk
a system hang?

Earlier in dw_remove(), the hardware clocks are disabled via
axi_dma_suspend(). If a client callback (invoked by the bottom half flush)
submits a new DMA transfer, the driver will attempt to read or write DMA
controller registers to start the transfer.

Can accessing these registers without clocks enabled cause a synchronous
external abort?

>  	}
>  }
>

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=40

Re: [PATCH v2 45/64] dmaengine: mtk-uart-apdma: kill vchan BH on teardown

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

Pre-existing issues:
- [High] Resource leak on driver unbind due to prematurely emptying the DMA channels list.
- [High] Use-After-Free of channel context due to orphaned IRQ handler on unbind.
- [High] Double decrement of PM runtime usage counter on failure in channel allocation.
--

--- Patch [45]: [PATCH v2 45/64] dmaengine: mtk-uart-apdma: kill vchan BH on teardown ---
commit 58283f2545c03ebd5b55eff5e9a5d8fdde273b4d
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: mtk-uart-apdma: kill vchan BH on teardown

Use dma_chan_kill_bh() for virt-dma cleanup.

> diff --git a/drivers/dma/mediatek/mtk-uart-apdma.c b/drivers/dma/mediatek/mtk-uart-apdma.c
> index c269d84d7bd2b..4d89927922f52 100644
> --- a/drivers/dma/mediatek/mtk-uart-apdma.c
> +++ b/drivers/dma/mediatek/mtk-uart-apdma.c
> @@ -312,7 +312,7 @@ static void mtk_uart_apdma_free_chan_resources(struct dma_chan *chan)
>  
>  	free_irq(c->irq, chan);

[Severity: High]
This is a pre-existing issue, but is it possible for a use-after-free
to occur on unbind?

The IRQ is requested without devres tracking in
mtk_uart_apdma_alloc_chan_resources(). It is only freed here in
mtk_uart_apdma_free_chan_resources().

If the driver is unbound while a client still actively holds a channel,
mtk_uart_apdma_remove() completes without forcing the channel to be
freed and without disabling the active IRQ. Since the mtk_chan context
is managed by devres, it is automatically freed upon remove() completion.
If a hardware interrupt fires after this, could the orphaned handler
dereference the already-freed mtk_chan structure?

> -	tasklet_kill(&c->vc.task);
> +	dma_chan_kill_bh(&c->vc.chan);
>  
>  	vchan_free_chan_resources(&c->vc);
>  
> @@ -463,7 +463,7 @@ static void mtk_uart_apdma_free(struct mtk_uart_apdmadev *mtkd)
>  			struct mtk_chan, vc.chan.device_node);
>  
>  		list_del(&c->vc.chan.device_node);

[Severity: High]
This isn't a bug introduced by this patch, but does this list removal
cause a resource leak during driver unbind?

In mtk_uart_apdma_remove(), the driver calls mtk_uart_apdma_free()
before dma_async_device_unregister(). This loop explicitly removes
each channel via list_del(), leaving the mtkd->ddev.channels list empty.

Immediately after, dma_async_device_unregister() iterates over the
now-empty device->channels list to tear down the channels. Because the
list is empty, does this cause the teardown to be entirely skipped,
permanently leaking the per-CPU chan->local memory, IDR slots, and
leaving orphaned sysfs nodes?

> -		tasklet_kill(&c->vc.task);
> +		dma_chan_kill_bh(&c->vc.chan);
>  	}
>  }

[Severity: High]
This is a pre-existing issue, but does mtk_uart_apdma_alloc_chan_resources()
double decrement the PM runtime usage counter on failure?

drivers/dma/mediatek/mtk-uart-apdma.c:mtk_uart_apdma_alloc_chan_resources() {
    ...
    ret = pm_runtime_resume_and_get(mtkd->ddev.dev);
    if (ret < 0) {
        pm_runtime_put_noidle(chan->device->dev);
        return ret;
    }
    ...
}

Since pm_runtime_resume_and_get() intrinsically drops the PM runtime
usage counter if it fails (returns < 0), does calling pm_runtime_put_noidle()
again in the error path result in a double decrement and unbalanced
runtime PM state?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=45

Re: [PATCH v2 39/64] dmaengine: fsl-edma-common: kill vchan BH on teardown

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [Critical] Bypassed DMA channel teardown leading to resource leaks and Use-After-Free via dangling sysfs nodes.
--

--- Patch [39]: [PATCH v2 39/64] dmaengine: fsl-edma-common: kill vchan BH on teardown ---
commit 72f422b1de100bcd764808c0c9305bd937347a17
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: fsl-edma-common: kill vchan BH on teardown

This commit updates virt-dma cleanup to use dma_chan_kill_bh() instead of
tasklet_kill() when tearing down channels in fsl_edma_cleanup_vchan().

> diff --git a/drivers/dma/fsl-edma-common.c b/drivers/dma/fsl-edma-common.c
> index bb7531c456dfa..70b40e60c39a0 100644
> --- a/drivers/dma/fsl-edma-common.c
> +++ b/drivers/dma/fsl-edma-common.c
> @@ -915,7 +915,7 @@ void fsl_edma_cleanup_vchan(struct dma_device *dmadev)
>  	list_for_each_entry_safe(chan, _chan,
>  				&dmadev->channels, vchan.chan.device_node) {
>  		list_del(&chan->vchan.chan.device_node);

[Severity: Critical]
This is a pre-existing issue, but does unlinking the channel here bypass
the core DMA engine channel teardown?

During driver removal (for example, in fsl_edma_remove() or
mcf_edma_remove()), calling list_del() before dma_async_device_unregister()
means the core unregister function will see an empty dmadev->channels list.

If the channels are unlinked prematurely, dma_async_device_unregister()
skips __dma_async_device_channel_unregister() for all channels. Can this
leave dangling sysfs nodes active and leak resources like IDAs or percpu
memory, potentially leading to a use-after-free if devres subsequently
frees the underlying channel structures while the sysfs files remain
accessible?

(Note: This same unlinking pattern before unregister also appears to exist
in mtk_cqdma_remove() and sprd_dma_remove()).

> -		tasklet_kill(&chan->vchan.task);
> +		dma_chan_kill_bh(&chan->vchan.chan);
>  	}
>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=39

Re: [PATCH v2 46/64] dmaengine: imx-sdma: kill vchan BH on teardown

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:

Pre-existing issues:
- [High] Kernel OOPS on module removal due to uninitialized list head in channel 0.
- [High] Acquiring an uninitialized spinlock for channel 0 during teardown causes a kernel BUG.
- [High] Accessing hardware registers while clocks are disabled causes a synchronous external abort.
- [High] Unbalanced clock disables in `sdma_remove` trigger refcount underflow warnings.
- [High] Use-After-Free race condition in `sdma_update_channel_loop` during concurrent termination.
--

--- Patch [46]: [PATCH v2 46/64] dmaengine: imx-sdma: kill vchan BH on teardown ---
commit 3996df4e4fdc79870fd89eac795a9295e1141104
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: imx-sdma: kill vchan BH on teardown

This commit modifies the virt-dma cleanup path to use dma_chan_kill_bh()
during driver teardown.

> --- a/drivers/dma/imx-sdma.c
> +++ b/drivers/dma/imx-sdma.c
> @@ -2399,11 +2399,11 @@ static void sdma_remove(struct platform_device *pdev)
>  	int i;
>  
>  	devm_free_irq(&pdev->dev, sdma->irq, sdma);
> -	/* Kill the tasklet */
> +	/* Kill the channel BH */
>  	for (i = 0; i < MAX_DMA_CHANNELS; i++) {
>  		struct sdma_channel *sdmac = &sdma->channel[i];
>  
> -		tasklet_kill(&sdmac->vc.task);
> +		dma_chan_kill_bh(&sdmac->vc.chan);
>  		sdma_free_chan_resources(&sdmac->vc.chan);

[Severity: High]
This is a pre-existing issue, but does iterating over all channels here lead
to a null pointer dereference for channel 0?

Channel 0 intentionally skips vchan_init() during probe, leaving its list
heads 0-initialized. When sdma_free_chan_resources() calls
vchan_synchronize() for channel 0, it executes:

drivers/dma/virt-dma.h:vchan_synchronize() {
    ...
    list_splice_tail_init(&vc->desc_terminated, &head);

Could this evaluate an empty list on the null next pointer and cause a panic?


[Severity: High]
This isn't a bug introduced by this patch, but does calling
sdma_free_chan_resources() on channel 0 also attempt to acquire an
uninitialized spinlock?

During teardown, sdma_terminate_all() executes:

drivers/dma/imx-sdma.c:sdma_terminate_all() {
    ...
    spin_lock_irqsave(&sdmac->vc.lock, flags);

Since vchan_init() is skipped for channel 0, the lock remains uninitialized
from devm_kzalloc. Can this result in a bad magic system crash?


[Severity: High]
This is a pre-existing issue, but does writing to the hardware registers
during this loop cause a synchronous external abort?

When iterating over all channels, unallocated channels (or when all channels
are inactive) will have the ipg and ahb clocks disabled.

sdma_free_chan_resources() calls sdma_terminate_all(), which invokes
sdma_disable_channel() doing:

drivers/dma/imx-sdma.c:sdma_disable_channel() {
    ...
    writel_relaxed(BIT(channel), sdma->regs + SDMA_H_STATSTOP);

Writing to an i.MX SDMA register while the IP block clocks are gated
typically causes a hardware bus fault. Is there a way to verify the clocks
are enabled before this write?


[Severity: High]
This isn't a bug introduced by this patch, but does this cleanup loop
cause clock refcount underflow warnings?

sdma_free_chan_resources() unconditionally calls:

drivers/dma/imx-sdma.c:sdma_free_chan_resources() {
    ...
    clk_disable(sdma->clk_ipg);
    clk_disable(sdma->clk_ahb);

Since clk_enable() is only invoked dynamically when a client requests a
channel, calling disable on unallocated channels seems unbalanced. Could this
break clock tracking?


[Severity: High]
This is a pre-existing issue, but is there a use-after-free race condition
in sdma_update_channel_loop() during concurrent termination?

sdma_update_channel_loop() drops the lock to invoke the callback:

drivers/dma/imx-sdma.c:sdma_update_channel_loop() {
    ...
    spin_unlock(&sdmac->vc.lock);
    dmaengine_desc_get_callback_invoke(&desc->vd.tx, NULL);
    spin_lock(&sdmac->vc.lock);

    /* Assign buffer ownership to SDMA */
    bd->mode.status |= BD_DONE;

If a concurrent thread calls dmaengine_terminate_sync(), it can take the
lock, move the active descriptor to the terminated list, and schedule the
terminate_worker which frees the descriptor memory.

When the IRQ thread reacquires the lock, it writes to bd->mode.status. Since
bd points inside the freed descriptor, does this overwrite freed memory?

>  	}
>

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=46

Re: [PATCH v2 53/64] dmaengine: fsl-qdma: use dma_chan_kill_bh

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

Pre-existing issues:
- [High] Prematurely removing channels from the `dmadev->channels` list in `fsl_qdma_cleanup_vchan()` bypasses core DMA engine unregistration, leading to resource leaks and Use-After-Free vulnerabilities.
- [High] The driver fails to halt the DMA engine during device removal and tears down IRQs before unregistering the DMA controller.
- [High] Unconditionally destroying shared DMA pools in `fsl_qdma_free_chan_resources()` causes a Use-After-Free for other active channels.
--

--- Patch [53]: [PATCH v2 53/64] dmaengine: fsl-qdma: use dma_chan_kill_bh ---
commit 705da09605a777fa7dd32343007b9e054bcb5942
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: fsl-qdma: use dma_chan_kill_bh

virt-dma now dispatches completion callbacks through per-channel BH
work instead of its tasklet. Cancel that work during teardown before
channel storage is released.

Signed-off-by: Allen Pais <allen.lkml@gmail.com>

> diff --git a/drivers/dma/fsl-qdma.c b/drivers/dma/fsl-qdma.c
> index df843fad0ece5..e0fb39fe821c9 100644
> --- a/drivers/dma/fsl-qdma.c
> +++ b/drivers/dma/fsl-qdma.c
> @@ -1255,7 +1255,7 @@ static void fsl_qdma_cleanup_vchan(struct dma_device *dmadev)
>  	list_for_each_entry_safe(chan, _chan,
>  				 &dmadev->channels, vchan.chan.device_node) {
>  		list_del(&chan->vchan.chan.device_node);

[Severity: High]
This is a pre-existing issue, but does removing this channel from the
dmadev->channels list here bypass the core DMA engine unregistration?

If the driver empties the channel list before the core DMA framework
unregisters it in fsl_qdma_remove():

drivers/dma/fsl-qdma.c:fsl_qdma_remove() {
        ...
        fsl_qdma_cleanup_vchan(&fsl_qdma->dma_dev);
        ...
        dma_async_device_unregister(&fsl_qdma->dma_dev);
}

The framework's cleanup loop will be silently skipped:

drivers/dma/dmaengine.c:dma_async_device_unregister() {
        ...
        list_for_each_entry_safe(chan, n, &device->channels, device_node)
                __dma_async_device_channel_unregister(device, chan);
        ...
}

Could this leave sysfs entries exposed and cause a use-after-free when
the underlying memory is freed?

> -		tasklet_kill(&chan->vchan.task);
> +		dma_chan_kill_bh(&chan->vchan.chan);
>  	}
>  }

[Severity: High]
This is also a pre-existing issue, but does the driver fail to halt the
DMA engine during device removal?

Looking at fsl_qdma_remove(), IRQs are destroyed while the controller
might still be active:

drivers/dma/fsl-qdma.c:fsl_qdma_remove() {
        ...
        fsl_qdma_irq_exit(pdev, fsl_qdma);
        fsl_qdma_cleanup_vchan(&fsl_qdma->dma_dev);
        ...
}

Since fsl_qdma_halt() is not called here, could the hardware continue
performing reads or writes against freed memory?

[Severity: High]
This is another pre-existing issue, but does destroying the shared DMA
pools in fsl_qdma_free_chan_resources() cause a use-after-free for
other active channels?

During initialization, multiple channels can share the same physical
queue structure and memory pools:

drivers/dma/fsl-qdma.c:fsl_qdma_probe() {
        ...
        fsl_chan->queue = fsl_qdma->queue + i % (fsl_qdma->n_queues *
                                        fsl_qdma->block_number);
        ...
}

However, when a channel is freed, the shared pools are unconditionally
destroyed:

drivers/dma/fsl-qdma.c:fsl_qdma_free_chan_resources() {
        ...
        dma_pool_destroy(fsl_queue->comp_pool);
        dma_pool_destroy(fsl_queue->desc_pool);
        ...
}

If another active channel shares this same queue, will it crash or
corrupt memory upon subsequent operations?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=53

Re: [PATCH v2 47/64] dmaengine: loongson1-apb: kill vchan BH on teardown

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

New issues:
- [High] The patch series breaks bisectability and leaves the tree unbuildable by prematurely removing the `task` member from `struct virt_dma_chan` before updating all consuming drivers.

Pre-existing issues:
- [High] Premature removal of DMA channels from the dma_device channels list bypasses dmaengine core unregistration, causing massive resource leaks (sysfs nodes, IDA entries, per-CPU memory) and leading to a Use-After-Free vulnerability.
- [High] Flushing the bottom-half work without stopping the hardware and masking interrupts leaves a race window where an active IRQ can reschedule the bottom-half on freed memory, leading to a Use-After-Free.
--

--- Patch [47]: [PATCH v2 47/64] dmaengine: loongson1-apb: kill vchan BH on teardown ---
commit 3e0d38c61679a38b8f14c3f667abc7411f9f4c83
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: loongson1-apb: kill vchan BH on teardown
    
Use dma_chan_kill_bh() for virt-dma cleanup.

[Severity: High]
Does this patch series introduce a build regression that breaks
bisectability?

If the task member was removed from struct virt_dma_chan earlier in this
series, unmodified drivers that still rely on it, such as bcm2835-dma.c in
bcm2835_dma_free_chan_resources(), will fail to build.

Should all consuming drivers be updated to the new API before the task
member is removed from the core virt_dma_chan structure?

> --- a/drivers/dma/loongson/loongson1-apb-dma.c
> +++ b/drivers/dma/loongson/loongson1-apb-dma.c
> @@ -552,7 +552,7 @@ static void ls1x_dma_chan_remove(struct ls1x_dma *dma)
>  
>  		if (chan->vc.chan.device == &dma->ddev) {
>  			list_del(&chan->vc.chan.device_node);

[Severity: High]
This is a pre-existing issue, but does removing the channel from the
device_node list here bypass the dmaengine core unregistration?

When the devres-managed dma_async_device_unregister() later runs, the
channels list will be empty, causing it to silently skip unregistering the
channel devices. This leaks the channel device structures, their IDA
entries, and per-CPU memory.

Furthermore, because the channel device is never unregistered, the IRQ
handler registered via devm_request_irq() in ls1x_dma_alloc_chan_resources()
is never freed and remains active indefinitely. Since the platform device's
devres will eventually free the channel memory, can an active IRQ trigger a
use-after-free on that freed memory?

> -			tasklet_kill(&chan->vc.task);
> +			dma_chan_kill_bh(&chan->vc.chan);

[Severity: High]
This isn't a bug introduced by this patch, but is there a race condition here
since the hardware isn't stopped first?

In ls1x_dma_remove(), the hardware DMA controller is not stopped and its
interrupts are not masked before teardown. Because the IRQ is still active,
a hardware interrupt can fire immediately after dma_chan_kill_bh() returns.

If the interrupt handler calls vchan_cookie_complete(), which then calls
dma_chan_schedule_bh(), it will requeue the work. When the ls1x_dma
structure containing the work item is later freed by devres, could the
workqueue process freed memory and cause a use-after-free or corrupt the
workqueue linked list?

>  		}
>  	}
>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=47

Re: [PATCH v2 58/64] dmaengine: qcom-adm: use dma_chan_kill_bh

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] Teardown race condition in adm_dma_remove() leading to use-after-free
- [High] Missing device_synchronize callback exposes DMA clients to use-after-free
--

--- Patch [58]: [PATCH v2 58/64] dmaengine: qcom-adm: use dma_chan_kill_bh ---
commit be25e9ce5563c30843636df639280bce697e48d5
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: qcom-adm: use dma_chan_kill_bh

virt-dma now dispatches completion callbacks through per-channel BH
work instead of its tasklet. Cancel that work during teardown before
channel storage is released.

> diff --git a/drivers/dma/qcom/qcom_adm.c b/drivers/dma/qcom/qcom_adm.c
> index 07fbe32d31fa4..ea24e2de51544 100644
> --- a/drivers/dma/qcom/qcom_adm.c
> +++ b/drivers/dma/qcom/qcom_adm.c
> @@ -918,8 +918,8 @@ static void adm_dma_remove(struct platform_device *pdev)
>  		/* mask IRQs for this channel/EE pair */
>  		writel(0, adev->regs + ADM_CH_RSLT_CONF(achan->id, adev->ee));
>  
> -		tasklet_kill(&adev->channels[i].vc.task);
> -		adm_terminate_all(&adev->channels[i].vc.chan);
> +		dma_chan_kill_bh(&achan->vc.chan);

[Severity: High]
This is a pre-existing issue, but does this sequence leave a window for a
use-after-free?

If adm_dma_remove() calls dma_chan_kill_bh() before devm_free_irq()
synchronizes with the interrupt handler, a concurrent interrupt can still
fire:

cpu1
adm_dma_remove()
    writel(0, ...)
    dma_chan_kill_bh()

cpu2
adm_dma_irq()
    vchan_cookie_complete()
        dma_chan_schedule_bh() 

Because cpu1 proceeds to devm_free_irq() which waits for the interrupt handler
to finish but does not flush the newly queued bottom-half work, the device
context is subsequently freed. The workqueue could then execute the scheduled
bottom-half work on the freed memory. 

Could devm_free_irq() be explicitly called before tearing down the channels?

[Severity: High]
This isn't a bug introduced by this patch, but does this driver also lack a
device_synchronize callback during its initialization in adm_dma_probe()?

When clients call dmaengine_terminate_sync() to abort transactions, it relies
on device_synchronize (usually mapped to vchan_synchronize) to wait for
callbacks to complete.

Without it, dmaengine_terminate_sync() returns immediately, and clients may
free their buffers while the virt-dma bottom-half work is still executing.

Can this lead to a use-after-free in the client's completion callback?

> +		adm_terminate_all(&achan->vc.chan);
>  	}
>  
>  	devm_free_irq(adev->dev, adev->irq, adev);

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=58

Re: [PATCH v2 48/64] dmaengine: owl-dma: kill vchan BH on teardown

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:

Pre-existing issues:
- [High] The interrupt handler incorrectly processes cyclic DMA transactions as if they were one-shot transfers, breaking cyclic DMA (e.g., audio playback) by stopping the channel after the first period.
- [High] Virtual channels stall indefinitely if their transactions are queued when all physical DMA channels are busy.
- [High] Missing `device_synchronize` callback exposes client drivers to Use-After-Free vulnerabilities during channel teardown.
- [High] The driver fails to halt active hardware DMA channels during `remove()`, causing rogue DMA transfers and potential bus lockups.
--

--- Patch [48]: [PATCH v2 48/64] dmaengine: owl-dma: kill vchan BH on teardown ---
commit 2420d60507a6a038239d14db32bd418a2fe41702
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: owl-dma: kill vchan BH on teardown

This commit updates the driver to use dma_chan_kill_bh() for virt-dma cleanup
during teardown, replacing the older tasklet_kill() interface.

> diff --git a/drivers/dma/owl-dma.c b/drivers/dma/owl-dma.c
> index 7c80572fc71d1..5f2140dc01cec 100644
> --- a/drivers/dma/owl-dma.c
> +++ b/drivers/dma/owl-dma.c
> @@ -1055,7 +1055,7 @@ static inline void owl_dma_free(struct owl_dma *od)
>  	list_for_each_entry_safe(vchan,
>  				 next, &od->dma.channels, vc.chan.device_node) {
>  		list_del(&vchan->vc.chan.device_node);
> -		tasklet_kill(&vchan->vc.task);
> +		dma_chan_kill_bh(&vchan->vc.chan);
>  	}
>  }

[Severity: High]
This isn't a bug introduced by this patch, but does the interrupt handler
incorrectly process cyclic DMA transactions as if they were one-shot transfers?

In drivers/dma/owl-dma.c:owl_dma_interrupt():

    txd = vchan->txd;
    if (txd) {
        vchan->txd = NULL;
        vchan_cookie_complete(&txd->vd);

        if (vchan_next_desc(&vchan->vc))
            owl_dma_start_next_txd(vchan);
        else
            owl_dma_phy_free(od, vchan);
    }

It appears this unconditionally completes and removes the descriptor without
checking txd->cyclic or calling vchan_cyclic_callback(). Can this break
cyclic DMA transfers by stopping the channel after the first period?


[Severity: High]
This is a pre-existing issue, but could virtual channels stall indefinitely if
their transactions are queued when all physical DMA channels are busy?

When allocating a physical channel in owl_dma_phy_alloc_and_start():

    pchan = owl_dma_get_pchan(od, vchan);
    if (!pchan)
        return;

If no physical channel is available, the request silently remains in the
virtual queue. However, when a physical channel is later freed in
owl_dma_phy_free():

    owl_dma_terminate_pchan(od, vchan->pchan);
    vchan->pchan = NULL;

The physical channel is released, but it doesn't appear to be reassigned to
other waiting virtual channels. Does this cause queued transfers on those
unassigned virtual channels to stall indefinitely?


[Severity: High]
While this isn't a bug introduced by your change, is the device_synchronize
callback missing from the DMA engine operations?

In drivers/dma/owl-dma.c:owl_dma_probe():

    od->dma.device_pause = owl_dma_pause;
    od->dma.device_resume = owl_dma_resume;
    od->dma.device_terminate_all = owl_dma_terminate_all;
    od->dma.src_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_4_BYTES);

Without mapping device_synchronize to vchan_synchronize,
dmaengine_terminate_sync() acts as a no-op for synchronization. Could this
expose client drivers to use-after-free vulnerabilities if the bottom-half
continues to run concurrently after the client assumes the channel is safely
halted?


[Severity: High]
This is a pre-existing issue, but does the driver fail to cleanly halt active
hardware DMA channels during removal?

In drivers/dma/owl-dma.c:owl_dma_remove():

    of_dma_controller_free(pdev->dev.of_node);
    dma_async_device_unregister(&od->dma);

    /* Mask all interrupts for this execution environment */
    dma_writel(od, OWL_DMA_IRQ_EN0, 0x0);

    /* Make sure we won't have any further interrupts */
    devm_free_irq(od->dma.dev, od->irq, od);

    owl_dma_free(od);

    clk_disable_unprepare(od->clk);

The removal function destroys driver resources and disables clocks, but
it doesn't seem to loop over od->pchans to smoothly halt running physical
channels via owl_dma_terminate_pchan(). Can disabling the clock while the
DMA controller is actively mastering the bus lock up the system, or cause
rogue DMA writes to freed memory?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=48

Re: [PATCH v2 52/64] dmaengine: tegra210-adma: kill vchan BH on teardown

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] Incorrect ordering of bottom-half termination and IRQ freeing leaves a window for spurious execution and use-after-free.
- [High] Requesting the IRQ before powering up the device via Runtime PM causes a kernel panic if a pending or spurious interrupt fires.
--

--- Patch [52]: [PATCH v2 52/64] dmaengine: tegra210-adma: kill vchan BH on teardown ---
commit acf1e1ab4d7c08f6577663273485ed69a1d42e71
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: tegra210-adma: kill vchan BH on teardown

Use dma_chan_kill_bh() for virt-dma cleanup.

> diff --git a/drivers/dma/tegra210-adma.c b/drivers/dma/tegra210-adma.c
> index ceaee1e33e68a..598abcc4edaa1 100644
> --- a/drivers/dma/tegra210-adma.c
> +++ b/drivers/dma/tegra210-adma.c
> @@ -811,7 +811,7 @@ static void tegra_adma_free_chan_resources(struct dma_chan *dc)
>  
>  	tegra_adma_terminate_all(dc);
>  	vchan_free_chan_resources(&tdc->vc);
> -	tasklet_kill(&tdc->vc.task);
> +	dma_chan_kill_bh(&tdc->vc.chan);
>  	free_irq(tdc->irq, tdc);

[Severity: High]
This is a pre-existing issue, but does this ordering leave a window for a
use-after-free?

If a delayed or spurious interrupt fires after dma_chan_kill_bh() flushes the
bottom-half, but before free_irq() unregisters the interrupt, couldn't the
interrupt handler execute and reschedule the bottom half via
vchan_cyclic_callback()?

Because free_irq() only waits for hardirq handlers and does not flush pending
bottom halves, this rescheduled bottom half could execute asynchronously after
the channel teardown completes and the client driver has freed its buffers.

Would it be safer to call free_irq() before dma_chan_kill_bh() to ensure
hardware events are disabled before flushing the work?

>  	pm_runtime_put(tdc2dev(tdc));
>  

[Severity: High]
This is also a pre-existing issue, but looking at the allocation side in
tegra_adma_alloc_chan_resources(), is it safe to request the IRQ before
powering up the device?

drivers/dma/tegra210-adma.c:tegra_adma_alloc_chan_resources() {
    ...
    ret = request_irq(tdc->irq, tegra_adma_isr, 0, dma_chan_name(dc), tdc);
    if (ret) {
        ...
    }

    ret = pm_runtime_resume_and_get(tdc2dev(tdc));
    ...
}

If the interrupt line is already active from a previous session or a spurious
hardware event, the CPU could immediately execute tegra_adma_isr() upon IRQ
registration.

Since tegra_adma_isr() unconditionally reads the ADMA_CH_INT_STATUS register,
and the device clock is disabled before pm_runtime_resume_and_get(), can
this access to unclocked hardware cause a synchronous external abort?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=52

Re: [PATCH v2 55/64] dmaengine: dma-axi-dmac: use dma_chan_kill_bh

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

New issues:
- [Low] Several other DMA drivers fail to build because they still explicitly reference the removed `task` member in `struct virt_dma_chan`.

Pre-existing issues:
- [High] axi_dmac_terminate_all() frees active descriptors without clearing the vc->cyclic pointer, leading to a Use-After-Free in the completion bottom half.
- [High] The driver lacks a devm action or a remove callback to explicitly disable the DMA hardware upon device unbind, causing system hangs if the clock is disabled while a transfer is active.
--

--- Patch [55]: [PATCH v2 55/64] dmaengine: dma-axi-dmac: use dma_chan_kill_bh ---
commit d769925844316e4d839799af39da3b91a09472d0
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: dma-axi-dmac: use dma_chan_kill_bh
    
virt-dma now dispatches completion callbacks through per-channel BH
work instead of its tasklet. Cancel that work during teardown before
channel storage is released.

> diff --git a/drivers/dma/dma-axi-dmac.c b/drivers/dma/dma-axi-dmac.c
> index d47ff27e1408f..d26bfa852806f 100644
> --- a/drivers/dma/dma-axi-dmac.c
> +++ b/drivers/dma/dma-axi-dmac.c
> @@ -1195,9 +1195,11 @@ static int axi_dmac_detect_caps(struct axi_dmac *dmac, unsigned int version)
>  	return 0;
>  }
>  
> -static void axi_dmac_tasklet_kill(void *task)
> +static void axi_dmac_kill_bh(void *data)
>  {
> -	tasklet_kill(task);
> +	struct dma_chan *chan = data;
> +
> +	dma_chan_kill_bh(chan);
>  }

[Severity: Low]
Does this patch series miss updates to other drivers?

Several other DMA drivers fail to build because they still explicitly
reference the removed task member in struct virt_dma_chan. For example:

drivers/dma/qcom/bam_dma.c:bam_dma_remove() {
    ...
    tasklet_kill(&bdev->channels[i].vc.task);
    ...
}

Should these drivers be updated to use dma_chan_kill_bh() as well?

[ ... ]

> @@ -1302,8 +1304,8 @@ static int axi_dmac_probe(struct platform_device *pdev)
>  	 * Put the action in here so it get's done before unregistering the DMA
>  	 * device.
>  	 */
> -	ret = devm_add_action_or_reset(&pdev->dev, axi_dmac_tasklet_kill,
> -				       &dmac->chan.vchan.task);
> +	ret = devm_add_action_or_reset(&pdev->dev, axi_dmac_kill_bh,
> +				       &dmac->chan.vchan.chan);

[Severity: High]
This is a pre-existing issue, but can the system hang if the driver is
unbound while a transfer is active?

The driver relies entirely on devres for teardown and lacks an explicit
remove callback or devm action to halt the DMA hardware. 

In axi_dmac_probe(), the clock is enabled via devm_clk_get_enabled(). If the
driver is forcefully unbound via sysfs, devres unwinding will disable the
clock. 

Disabling the clock while the AXI DMA controller is actively performing bus
transactions could stall the AXI bus.

Should an action be added here to gracefully halt the DMA engine prior to
the clock being disabled?

[Severity: High]
This isn't a bug introduced by this patch, but does
axi_dmac_terminate_all() safely handle cyclic descriptor termination?

When a cyclic transfer period completes, it sets vc->cyclic to the active
descriptor and schedules the bottom half. 

If a client concurrently calls dmaengine_terminate_async(),
axi_dmac_terminate_all() frees the active descriptors without clearing the
vc->cyclic pointer:

drivers/dma/dma-axi-dmac.c:axi_dmac_terminate_all() {
    ...
    list_splice_tail_init(&chan->active_descs, &head);
    spin_unlock_irqrestore(&chan->vchan.lock, flags);

    vchan_dma_desc_free_list(&chan->vchan, &head);
    ...
}

When the scheduled BH runs, could it dereference the dangling vc->cyclic
pointer to fetch the callback, resulting in a use-after-free?

>  	if (ret)
>  		return ret;
>

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=55

Re: [PATCH v2 50/64] dmaengine: dw-edma: kill vchan BH on teardown

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [Critical] Virtual DMA descriptors are leaked during channel teardown and release due to missing vchan_free_chan_resources(), leading to memory leaks and critical use-after-free memory corruption on channel reuse.
--

--- Patch [50]: [PATCH v2 50/64] dmaengine: dw-edma: kill vchan BH on teardown ---
commit 5cb424de4644af7e57a2169fe1e94e001c0d3927
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: dw-edma: kill vchan BH on teardown

This commit modifies the dw-edma driver to use dma_chan_kill_bh() for
cleaning up virt-dma channels during teardown, replacing the older
tasklet_kill() usage.

> diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-edma-core.c
> index 89a4c498a17b8..a47a4674df838 100644
> --- a/drivers/dma/dw-edma/dw-edma-core.c
> +++ b/drivers/dma/dw-edma/dw-edma-core.c
> @@ -1170,7 +1170,7 @@ int dw_edma_remove(struct dw_edma_chip *chip)
>  	dma_async_device_unregister(&dw->dma);
>  	list_for_each_entry_safe(chan, _chan, &dw->dma.channels,
>  				 vc.chan.device_node) {
> -		tasklet_kill(&chan->vc.task);
> +		dma_chan_kill_bh(&chan->vc.chan);

[Severity: Critical]
This is a pre-existing issue, but does this teardown sequence leak virtual
DMA descriptors because it misses a call to vchan_free_chan_resources()?

If a client queues a DMA transfer, aborts it via
dw_edma_device_terminate_all(), and releases the channel, the aborted
descriptors are not removed from the virt-dma queues because
dw_edma_free_chan_resources() also lacks a call to
vchan_free_chan_resources() to clear the pending lists.

When the channel is requested again and started, could
dw_edma_start_transfer() mistakenly pull a stale descriptor from
vchan_next_desc(), leading to memory corruption and a use-after-free?

>  		list_del(&chan->vc.chan.device_node);
>  	}
>

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=50

Re: [PATCH v2 54/64] dmaengine: st_fdma: use dma_chan_kill_bh

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:

Pre-existing issues:
- [High] Calling `list_del()` in `st_fdma_free()` prematurely empties the channel list, breaking devres-based DMA engine unregistration and silently leaking resources on probe failure.
- [High] The per-channel completion BH work is never cancelled during driver removal, leading to a Use-After-Free if scheduled before devres frees channel memory.
- [High] The driver lacks a `device_synchronize` callback and destroys the DMA pool before pending BH work is cancelled, leading to a UAF or NULL pointer dereference on channel release.
- [High] Virtual DMA descriptors are permanently leaked because `st_fdma_free_chan_res()` omits a call to `vchan_free_chan_resources()`.
- [High] Inverted teardown sequence in `st_fdma_remove` vs devres allows DMA transactions on partially destroyed hardware.
--

--- Patch [54]: [PATCH v2 54/64] dmaengine: st_fdma: use dma_chan_kill_bh ---
commit 4cab1e1fac46ef4cc457cee5eab4226a73880829
Author: Allen Pais <allen.lkml@gmail.com>
Subject: dmaengine: st_fdma: use dma_chan_kill_bh

virt-dma now dispatches completion callbacks through per-channel BH
work instead of its tasklet. Cancel that work during teardown before
channel storage is released.

> diff --git a/drivers/dma/st_fdma.c b/drivers/dma/st_fdma.c
> index d9547017f3bd6..d4cff851e1d52 100644
> --- a/drivers/dma/st_fdma.c
> +++ b/drivers/dma/st_fdma.c
> @@ -733,7 +733,7 @@ static void st_fdma_free(struct st_fdma_dev *fdev)
>  	for (i = 0; i < fdev->nr_channels; i++) {
>  		fchan = &fdev->chans[i];
>  		list_del(&fchan->vchan.chan.device_node);

[Severity: High]
This is a pre-existing issue, but does calling list_del() here prematurely
empty the channel list and break devres-based DMA engine unregistration?

If probe fails at of_dma_controller_register(), st_fdma_free() is called and
unlinks all channels from fdev->dma_device.channels.

When devres subsequently invokes dma_async_device_unregister(), it iterates
over the now-empty list and skips calling
__dma_async_device_channel_unregister(). This could permanently leak DMA
engine channel structures, IDA allocations, and sysfs nodes.

> -		tasklet_kill(&fchan->vchan.task);
> +		dma_chan_kill_bh(&fchan->vchan.chan);
>  	}
>  }

[Severity: High]
This is a pre-existing issue, but since this patch updates teardown paths to
use dma_chan_kill_bh(), should st_fdma_remove() be updated as well?

drivers/dma/st_fdma.c:st_fdma_remove() {
    ...
	devm_free_irq(&pdev->dev, fdev->irq, fdev);
	st_slim_rproc_put(fdev->slim_rproc);
	of_dma_controller_free(pdev->dev.of_node);
}

The per-channel completion BH work is never cancelled during driver removal.
If the driver is unbound and an interrupt schedules the BH work just before
devm_free_irq() disables it, devres will free fdev->chans. 

The delayed BH (vchan_complete) would then execute on the freed memory,
causing a use-after-free.

[Severity: High]
This is a pre-existing issue, but could destroying the DMA pool without
synchronizing against pending BH work lead to a use-after-free or null
pointer dereference?

drivers/dma/st_fdma.c:st_fdma_free_chan_res() {
    ...
	dma_pool_destroy(fchan->node_pool);
	fchan->node_pool = NULL;
    ...
}

The driver lacks a device_synchronize callback (e.g., pointing to
vchan_synchronize). Because of this, dmaengine_synchronize() returns
immediately without waiting for pending virt-dma BH work.

If vchan_complete() executes concurrently or shortly after, it invokes
st_fdma_free_desc():

drivers/dma/st_fdma.c:st_fdma_free_desc() {
    ...
		dma_pool_free(fdesc->fchan->node_pool, fdesc->node[i].desc,
    ...
}

Passing the null node_pool to dma_pool_free() could crash the kernel.

[Severity: High]
This is a pre-existing issue, but are virtual DMA descriptors permanently
leaked here because st_fdma_free_chan_res() omits a call to
vchan_free_chan_resources()?

drivers/dma/st_fdma.c:st_fdma_free_chan_res() {
    ...
    // missing call to vchan_free_chan_resources(&fchan->vchan)
    ...
}

If a client prepares a DMA transaction (allocating virtual descriptors onto
vchan->desc_allocated) but releases the channel without submitting them,
st_fdma_free_chan_res() destroys the hardware node pool.

Because vchan_free_chan_resources() isn't called, the virtual descriptors are
orphaned on internal lists, leaking their memory.

[Severity: High]
This is a pre-existing issue, but does the teardown sequence in
st_fdma_remove() allow DMA transactions on partially destroyed hardware?

drivers/dma/st_fdma.c:st_fdma_remove() {
    ...
	st_slim_rproc_put(fdev->slim_rproc);
    ...
}

Manual hardware power-down occurs here before the devres-managed
dma_async_device_unregister() runs. 

During this window, clients can still issue DMA requests via
dmaengine_submit(), which would write to unclocked or powered-down hardware
registers.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=54

Re: [PATCH v2 60/64] dmaengine: bam-dma: use dma_chan_kill_bh

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:

Pre-existing issues:
- [High] In the probe error path, the BAM interrupts are not masked and the IRQ is not freed before disabling clocks and cleaning up state.
- [High] Hardware registers are accessed while the peripheral clock is disabled in `bam_dma_remove`, leading to bus faults (Data Abort).
- [High] Failing to drop the PM runtime usage counter on error paths prevents the device from ever entering runtime suspend.
- [High] The driver fails to implement the `device_synchronize` DMA engine callback, rendering `dmaengine_synchronize()` a no-op.
- [Medium] Calling `clk_disable_unprepare` in `bam_dma_remove` underflows the clock enable count.
--

--- Patch [60]: [PATCH v2 60/64] dmaengine: bam-dma: use dma_chan_kill_bh ---
commit 66f1d34a1a1de31180ac528ea6a897680d6ee0d9
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: bam-dma: use dma_chan_kill_bh

virt-dma now dispatches completion callbacks through per-channel BH
work instead of its tasklet. Cancel that work during teardown before
channel storage is released.

> diff --git a/drivers/dma/qcom/bam_dma.c b/drivers/dma/qcom/bam_dma.c
> index 1bb26af0405f3..74bc2fe445b32 100644
> --- a/drivers/dma/qcom/bam_dma.c
> +++ b/drivers/dma/qcom/bam_dma.c
> @@ -1387,7 +1387,7 @@ static int bam_dma_probe(struct platform_device *pdev)
>  	dma_async_device_unregister(&bdev->common);
>  err_bam_channel_exit:
>  	for (i = 0; i < bdev->num_channels; i++)
> -		tasklet_kill(&bdev->channels[i].vc.task);
> +		dma_chan_kill_bh(&bdev->channels[i].vc.chan);

[Severity: High]
This isn't a bug introduced by this patch, but the driver fails to implement
the device_synchronize DMA engine callback here in bam_dma_probe().

Because bdev->common.device_synchronize is never assigned,
dmaengine_synchronize() returns immediately. This could allow DMA clients
to free memory while the completion bottom-half work is still executing
concurrently. 

Should the driver provide a wrapper that calls vchan_synchronize()?

>  err_tasklet_kill:
>  	tasklet_kill(&bdev->task);
>  err_disable_clk:
>  	clk_disable_unprepare(bdev->bamclk);

[Severity: High]
This is a pre-existing issue, but does this error path leave the BAM
interrupts active before disabling the clock?

Because devm_request_irq() is used, the interrupt is not freed until after
probe() returns. If an interrupt fires during this window, bam_dma_irq()
will access BAM hardware registers while the clock is disabled via
clk_disable_unprepare(), which can trigger a synchronous external abort on
QCOM platforms.

Could we mask interrupts and free the IRQ explicitly before disabling clocks?

> @@ -1413,7 +1413,7 @@ static void bam_dma_remove(struct platform_device *pdev)
>  
>  	for (i = 0; i < bdev->num_channels; i++) {
>  		bam_dma_terminate_all(&bdev->channels[i].vc.chan);
> -		tasklet_kill(&bdev->channels[i].vc.task);
> +		dma_chan_kill_bh(&bdev->channels[i].vc.chan);

[Severity: High]
This isn't a bug introduced by this patch, but are hardware registers
accessed after the clock is disabled?

Looking at bam_dma_remove(), pm_runtime_force_suspend() is called early,
which disables the clock. Immediately after, writel_relaxed() is called to
mask interrupts, and bam_dma_terminate_all() accesses hardware. Accessing
unclocked peripheral registers on QCOM platforms triggers a fatal
synchronous external abort.

Should the hardware shutdown logic occur before forcing PM suspend?

[Severity: Medium]
This is a pre-existing issue, but is there a double-disable of the clock in
bam_dma_remove()?

Since pm_runtime_force_suspend() has already suspended the device and
disabled the clock, the later call to clk_disable_unprepare() at the end
of this function will disable it again, causing the clock enable count to
drop below zero.

[Severity: High]
This isn't a bug introduced by this patch, but does the
pm_runtime_get_sync() failure path in bam_dma_irq() leak the PM usage
counter?

If pm_runtime_get_sync() returns a negative error:

drivers/dma/qcom/bam_dma.c:bam_dma_irq() {
	...
	ret = pm_runtime_get_sync(bdev->dev);
	if (ret < 0)
		return IRQ_NONE;
	...
}

Returning IRQ_NONE immediately without calling pm_runtime_put_noidle()
permanently leaks the PM reference count.

Additionally, returning IRQ_NONE without clearing the hardware interrupt
status leaves the IRQ asserted, potentially causing an IRQ storm that leads
to a CPU soft-lockup.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=60

Re: [PATCH v2 51/64] dmaengine: bcm2835: kill vchan BH on teardown

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing of_dma_controller_free() in the probe error path and remove function leaves a dangling pointer in the OF DMA list.
- [High] Force unbinding the driver while a client holds a DMA channel leaks the channel's IRQ and DMA pool, leading to a Use-After-Free when the IRQ fires.
--

--- Patch [51]: [PATCH v2 51/64] dmaengine: bcm2835: kill vchan BH on teardown ---
commit 922bf2cdbf2470f43f83cc2698c5817aa6b5f545
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: bcm2835: kill vchan BH on teardown

Use dma_chan_kill_bh() for virt-dma cleanup instead of tasklet_kill().

> diff --git a/drivers/dma/bcm2835-dma.c b/drivers/dma/bcm2835-dma.c
> index 06d830d368822..68e86c8d3d439 100644
> --- a/drivers/dma/bcm2835-dma.c
> +++ b/drivers/dma/bcm2835-dma.c
> @@ -829,7 +829,7 @@ static void bcm2835_dma_free(struct bcm2835_dmadev *od)
>  	list_for_each_entry_safe(c, next, &od->ddev.channels,
>  				 vc.chan.device_node) {
>  		list_del(&c->vc.chan.device_node);
> -		tasklet_kill(&c->vc.task);
> +		dma_chan_kill_bh(&c->vc.chan);
>  	}
>  
>  	dma_unmap_page_attrs(od->ddev.dev, od->zero_page, PAGE_SIZE,

[Severity: High]
This is a pre-existing issue, but I noticed a potential leak and use-after-free
if dma_async_device_register() fails in bcm2835_dma_probe() or if the driver
is forcefully unbound via sysfs.

In bcm2835_dma_probe(), if dma_async_device_register() fails, the error path
jumps to err_no_dma:

drivers/dma/bcm2835-dma.c:bcm2835_dma_probe() {
    ...
	rc = dma_async_device_register(&od->ddev);
	if (rc) {
		dev_err(&pdev->dev,
			"Failed to register slave DMA engine device: %d\n", rc);
		goto err_no_dma;
	}
    ...
err_no_dma:
	bcm2835_dma_free(od);
	return rc;
}

Since of_dma_controller_register() was called earlier in the probe function,
does this error path leave a dangling pointer in the OF DMA list?

If the devm-managed bcm2835_dmadev struct is freed automatically by devres,
any subsequent DMA channel request from another driver matching this device
tree node might dereference the freed pointer during the bcm2835_dma_xlate
callback.

Similarly, in bcm2835_dma_remove(), the OF DMA controller is not freed:

drivers/dma/bcm2835-dma.c:bcm2835_dma_remove() {
	struct bcm2835_dmadev *od = platform_get_drvdata(pdev);

	dma_async_device_unregister(&od->ddev);
	bcm2835_dma_free(od);
}

Should of_dma_controller_free() be called in both of these paths?

[Severity: High]
This is also a pre-existing issue, but I noticed another potential
use-after-free and memory leak if the driver is forcefully unbound via sysfs
while a client still holds a DMA channel.

When a client requests a channel, an IRQ and a DMA pool are dynamically
allocated:

drivers/dma/bcm2835-dma.c:bcm2835_dma_alloc_chan_resources() {
    ...
	return request_irq(c->irq_number, bcm2835_dma_callback,
			   c->irq_flags, "DMA IRQ", c);
}

If the driver is unbound via sysfs, bcm2835_dma_remove() executes and returns
without tearing down active channel resources, as the DMA engine core skips
forcefully releasing channel resources.

Since the IRQ was requested without devm_, it remains registered after the
driver memory (struct bcm2835_chan) is automatically freed by devres.

Could a subsequent hardware interrupt execute bcm2835_dma_callback() and
dereference the freed channel pointer?

drivers/dma/bcm2835-dma.c:bcm2835_dma_callback() {
    ...
	spin_lock_irqsave(&c->vc.lock, flags);
    ...
}

Should we ensure that free_irq() and dma_pool_destroy() are called when
cleaning up these resources during driver removal?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=51

Re: [PATCH v2 49/64] dmaengine: hisi: kill vchan BH on teardown

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] Race condition during device teardown can lead to a use-after-free because the bottom-half work is killed before IRQs are synchronized, allowing a concurrent IRQ handler to reschedule the bottom-half work on freed memory.
- [High] The driver requests shared IRQs before fully initializing virtual channel structures, risking an uninitialized spinlock access and null pointer dereference if a shared or spurious interrupt occurs early.
--

--- Patch [49]: [PATCH v2 49/64] dmaengine: hisi: kill vchan BH on teardown ---
commit f8a5265e01e8ff281f13c0af8c64babbb3148800
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: hisi: kill vchan BH on teardown
    
This commit replaces tasklet_kill() with dma_chan_kill_bh() for virt-dma 
cleanup during device teardown.

> diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
> index 28bf818f9aa63..39bc88c465138 100644
> --- a/drivers/dma/hisi_dma.c
> +++ b/drivers/dma/hisi_dma.c
> @@ -720,7 +720,7 @@ static void hisi_dma_disable_qps(struct hisi_dma_dev *hdma_dev)
>  
>  	for (i = 0; i < hdma_dev->chan_num; i++) {
>  		hisi_dma_disable_qp(hdma_dev, i);
> -		tasklet_kill(&hdma_dev->chan[i].vc.task);
> +		dma_chan_kill_bh(&hdma_dev->chan[i].vc.chan);

[Severity: High]
This is a pre-existing issue, but does this teardown sequence leave us open to
a use-after-free regression?

During device unbind, devres executes hisi_dma_disable_hw_channels() which
calls hisi_dma_disable_qps(). However, devm_free_irq() hasn't been called yet. 

If hisi_dma_irq() is concurrently executing on another CPU, it can call
vchan_cookie_complete():

drivers/dma/hisi_dma.c:hisi_dma_irq() {
	...
	vchan_cookie_complete(&desc->vd);
	...
}

This can schedule the bottom half after dma_chan_kill_bh() has already
flushed it. When devres subsequently frees the device memory, could the
scheduled bottom half execute on the freed channel structure?

>  	}
>  }

[Severity: High]
This isn't a bug introduced by this patch, but is it possible for a shared
interrupt to fire before the channel structures are fully initialized?

In hisi_dma_enable_hw_channels(), the driver requests shared IRQs before
calling hisi_dma_enable_qps(), which initializes the channels:

drivers/dma/hisi_dma.c:hisi_dma_enable_hw_channels() {
	...
	ret = hisi_dma_request_qps_irq(hdma_dev);
	...
}

If a shared or spurious interrupt fires immediately, hisi_dma_irq() would
execute and dereference an uninitialized pointer and lock:

drivers/dma/hisi_dma.c:hisi_dma_irq() {
	...
	struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
	...
	spin_lock(&chan->vc.lock);
	...
}

Could this lead to a NULL pointer dereference or uninitialized lock access?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=49

Re: [PATCH v2 57/64] dmaengine: edma: use dma_chan_kill_bh

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] Channel unregistration bypass leading to memory leak and sysfs Use-After-Free.
- [High] Missing BH cancellation for `memcpy` channels causing Use-After-Free during teardown.
--

--- Patch [57]: [PATCH v2 57/64] dmaengine: edma: use dma_chan_kill_bh ---
commit 03c05180f9212195192cc327d33610888eef85fe
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: edma: use dma_chan_kill_bh

virt-dma now dispatches completion callbacks through per-channel BH work
instead of its tasklet. Cancel that work during teardown before channel
storage is released.

> diff --git a/drivers/dma/ti/edma.c b/drivers/dma/ti/edma.c
> index d97db5af35555..c0b16cd247525 100644
> --- a/drivers/dma/ti/edma.c
> +++ b/drivers/dma/ti/edma.c
> @@ -2560,7 +2560,7 @@ static void edma_cleanupp_vchan(struct dma_device *dmadev)
>  	list_for_each_entry_safe(echan, _echan,
>  			&dmadev->channels, vchan.chan.device_node) {
>  		list_del(&echan->vchan.chan.device_node);

[Severity: High]
This is a pre-existing issue, but does removing the channel from the
device list here cause problems during device unregistration?

When a user unbinds the driver, edma_remove() calls this function which
unconditionally removes all slave channels from dmadev->channels. When
dma_async_device_unregister() subsequently runs, it iterates over the
now-empty list and entirely skips unregistering the channels.

Can this cause sysfs device nodes, IDAs, and per-cpu variables to leak?
If the orphaned sysfs attributes (such as in_use or bytes_transferred)
are read after the driver is unbound, could it trigger a use-after-free
since the underlying chan structures are freed by devres?

> -		tasklet_kill(&echan->vchan.task);
> +		dma_chan_kill_bh(&echan->vchan.chan);
>  	}
>  }

[Severity: High]
This is a pre-existing issue, but are we missing a cleanup step for the
memcpy channels?

Looking at edma_remove(), edma_cleanupp_vchan() is explicitly called for
ecc->dma_slave, but it appears to be completely omitted for ecc->dma_memcpy:

drivers/dma/ti/edma.c:edma_remove() {
    ...
    edma_cleanupp_vchan(&ecc->dma_slave);
    ...
    dma_async_device_unregister(&ecc->dma_slave);
    if (ecc->dma_memcpy)
        dma_async_device_unregister(ecc->dma_memcpy);
    ...
}

Because the cleanup is skipped, the BH cancellation is never performed
for dma_memcpy channels. If a background transfer completes and schedules
the BH just before the driver is unbound, could the pending BH execute
after devres has freed the channel structures and cause a use-after-free?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=57

Re: [PATCH v2 64/64] dmaengine: switchtec: use channel BH helpers

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

Pre-existing issues:
- [High] dmaengine_synchronize() incorrectly aborts pending descriptors instead of waiting, violating the DMA API contract.
- [High] Lost completion results for out-of-order descriptors leads to incorrect callback data.
- [High] Channel reset in `switchtec_dma_synchronize` fails to zero host memory, causing permanent desynchronization.
--

--- Patch [64]: [PATCH v2 64/64] dmaengine: switchtec: use channel BH helpers ---
commit ae138df18f2b5dc2645b9652b5d982dbf7b2fde2
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: switchtec: use channel BH helpers

Replace the per-channel descriptor completion tasklet with the shared
dma_chan BH helpers. Client callbacks remain in softirq context while
using the common scheduling and teardown mechanism.

> diff --git a/drivers/dma/switchtec_dma.c b/drivers/dma/switchtec_dma.c
> index 3ef9286406159..f3459799e05d9 100644
> --- a/drivers/dma/switchtec_dma.c
> +++ b/drivers/dma/switchtec_dma.c

[ ... ]

> @@ -584,6 +582,7 @@ static void switchtec_dma_synchronize(struct dma_chan *chan)
>  
>  	int rc;
>  
> +	dma_chan_kill_bh(chan);
>  	switchtec_dma_abort_desc(swdma_chan, 1);

[Severity: High]
This is a pre-existing issue, but does switchtec_dma_synchronize() violate
the DMA API contract here by actively destroying the channel's state?

The DMA API requires dmaengine_synchronize() to merely wait for the
completion of previously terminated or submitted transfers. Instead, this
forcefully aborts all pending descriptors and skips their callbacks, which
would drop pending transfers and leave clients with missing completions.

[Severity: High]
This is also a pre-existing issue, but further down in
switchtec_dma_synchronize(), the software ring state is reset:

	spin_lock_bh(&swdma_chan->complete_lock);
	swdma_chan->comp_ring_active = true;
	swdma_chan->phase_tag = 0;
	swdma_chan->tail = 0;
	swdma_chan->cq_tail = 0;

Does this channel reset fail to zero the backing hardware memory (hw_cq)?

Since the memory retains previous phase tags, the software might incorrectly
interpret stale data at index 0 as a new hardware completion, causing it to
read garbage and permanently desynchronize from the hardware state machine.

>  
>  	rc = enable_channel(swdma_chan);

[ ... ]

> @@ -820,9 +819,10 @@ static int switchtec_dma_resume(struct dma_chan *chan)
>  	return ret;
>  }
>  
> -static void switchtec_dma_desc_task(unsigned long data)
> +static void switchtec_dma_desc_task(struct dma_chan *chan)
>  {
> -	struct switchtec_dma_chan *swdma_chan = (void *)data;
> +	struct switchtec_dma_chan *swdma_chan =
> +		container_of(chan, struct switchtec_dma_chan, dma_chan);
>  
>  	switchtec_dma_cleanup_completed(swdma_chan);

[Severity: High]
This isn't a bug introduced by this patch, but does
switchtec_dma_cleanup_completed() lose the completion results for
out-of-order descriptors?

When the hardware completes a descriptor out-of-order, res is computed but
immediately discarded by a continue:

		/*  Out of order CE */
		if (se_idx != tail) {
			spin_unlock_bh(&swdma_chan->complete_lock);
			continue;
		}

When se_idx == tail is eventually met, the catch-up loop reuses the exact
same local res variable for all queued completions:

		do {
			dma_cookie_complete(&desc->txd);
			dma_descriptor_unmap(&desc->txd);
			dmaengine_desc_get_callback_invoke(&desc->txd, &res);
			...

Could out-of-order DMA completions lose their actual result status and
residue, causing them to incorrectly inherit the status of the last
sequentially completed descriptor?

>  }
>

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=64

Re: [PATCH v2 59/64] dmaengine: tegra186-gpc: use dma_chan_kill_bh

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:

Pre-existing issues:
- [Critical] Zero-length DMA requests cause an integer underflow when calculating the hardware word count, resulting in an unbounded 4GB DMA transfer.
- [High] TOCTOU race condition in the interrupt handler `tegra_dma_isr` leading to a Use-After-Free of `dma_desc`.
- [High] Incorrect teardown order in `tegra_dma_free_chan_resources()` allows late interrupts to reschedule the channel bottom-half, leading to a Use-After-Free.
- [High] Ignored error return from `tegra_dma_terminate_all()` leads to descriptor memory leak and active hardware DMA during channel teardown.
- [High] An unsupported DMA slave bus width causes `get_transfer_param()` to silently propagate an error code (-EINVAL) into the DMA hardware's MMIO sequence register.
--

--- Patch [59]: [PATCH v2 59/64] dmaengine: tegra186-gpc: use dma_chan_kill_bh ---
commit a9e2657d46f0ef7a89cb50bf6bca1e73ae39ee11
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: tegra186-gpc: use dma_chan_kill_bh
    
virt-dma now dispatches completion callbacks through per-channel BH
work instead of its tasklet. Cancel that work during teardown before
channel storage is released.

> diff --git a/drivers/dma/tegra186-gpc-dma.c b/drivers/dma/tegra186-gpc-dma.c
> index 64cedef1050ae..1ba8897ec614e 100644
> --- a/drivers/dma/tegra186-gpc-dma.c
> +++ b/drivers/dma/tegra186-gpc-dma.c
> @@ -1284,7 +1284,7 @@ static void tegra_dma_free_chan_resources(struct dma_chan *dc)
>  	tegra_dma_terminate_all(dc);

[Severity: High]
This isn't a bug introduced by this patch, but should we check the return
value of tegra_dma_terminate_all() here?

If tegra_dma_terminate_all() fails due to a hardware timeout, it returns early
without moving the active descriptor to the terminated list and without
disabling the hardware.

Since the caller ignores the return value, vchan_free_chan_resources() might
fail to locate and free the active descriptor. Could this leave the hardware
executing a transfer into client memory that may be subsequently freed?

>  	synchronize_irq(tdc->irq);
>  
> -	tasklet_kill(&tdc->vc.task);
> +	dma_chan_kill_bh(&tdc->vc.chan);

[Severity: High]
This is a pre-existing issue, but does killing the bottom-half before freeing
the IRQ expose a race window?

If the DMA hardware remains active after a timeout, and dma_chan_kill_bh() is
called before free_irq(), could a late interrupt fire and invoke
vchan_cookie_complete() or vchan_cyclic_callback()?

This would re-queue the bottom-half work. After vchan_free_chan_resources()
subsequently frees all descriptors, the queued bottom-half might execute and
dereference the freed memory.

>  	tdc->config_init = false;
>  	tdc->slave_id = -1;
>  	tdc->sid_dir = DMA_TRANS_NONE;


While reviewing the file, I noticed a few other pre-existing issues:

[Severity: High]
This isn't a bug introduced by this patch, but does tegra_dma_isr() have a
race condition leading to a use-after-free of dma_desc?

drivers/dma/tegra186-gpc-dma.c:tegra_dma_isr() {
    ...
    struct tegra_dma_desc *dma_desc = tdc->dma_desc;
    ...
    spin_lock_irqsave(&tdc->vc.lock, flags);
    ...
    if (!dma_desc)
        goto irq_done;
    ...
}

The shared descriptor pointer is cached into a local variable before acquiring
the spinlock.

If the ISR is delayed while another CPU calls tegra_dma_terminate_all(), the
descriptor is freed. Could a subsequent DMA transfer quickly trigger another
interrupt, causing the delayed ISR to acquire the lock, pass the check, and
dereference the stale local pointer?


[Severity: High]
This is a pre-existing issue, but do zero-length DMA requests cause an
integer underflow when calculating the hardware word count?

In tegra_dma_prep_dma_memset():

drivers/dma/tegra186-gpc-dma.c:tegra_dma_prep_dma_memset() {
    ...
    sg_req[0].wcount = ((len - 4) >> 2);
    ...
}

The same calculation exists in tegra_dma_prep_dma_memcpy() and
tegra_dma_prep_slave_sg().

If a client requests a 0-length DMA transfer, the length checks appear to pass.
The computation then underflows to 0x3FFFFFFF. Because the hardware
interprets this as N+1 words, could this trigger an unbounded 4GB DMA
transfer?


[Severity: High]
This isn't a bug introduced by this patch, but does get_transfer_param()
silently propagate an error code to the MMIO sequence register?

drivers/dma/tegra186-gpc-dma.c:get_transfer_param() {
    ...
    *mmio_seq = get_bus_width(tdc, tdc->dma_sconfig.dst_addr_width);
    *burst_size = tdc->dma_sconfig.dst_maxburst;
    *slave_bw = tdc->dma_sconfig.dst_addr_width;
    *csr = TEGRA_GPCDMA_CSR_DMA_MEM2IO_FC;
    return 0;
}

If a client configures an unsupported dst_addr_width, get_bus_width() returns
-EINVAL. This negative error code is assigned directly to *mmio_seq, and the
function still returns 0 for success.

Could this signed error be bitwise-ORed with other flags and written directly
to the hardware register, corrupting the configuration bitfields?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=59

Re: [PATCH v2 63/64] dmaengine: qcom-gpi: defer callbacks via vchan

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Incorrect teardown sequence in gpi_free_chan_resources(): dma_chan_kill_bh() is called after gpi_free_ring(), exposing a freed ring buffer to concurrent client callbacks, leading to a divide-by-zero crash.
- [High] The driver defers callbacks to a virt-dma BH but fails to implement the device_synchronize callback, breaking dmaengine_terminate_sync().
--

--- Patch [63]: [PATCH v2 63/64] dmaengine: qcom-gpi: defer callbacks via vchan ---
commit e55f43a67f69dbf222b5c7eba067eedde92ac98b
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: qcom-gpi: defer callbacks via vchan

Complete descriptors through virt-dma instead of invoking callbacks
directly from event processing. This preserves the transfer result for
the shared vchan BH, which invokes the callback and frees the descriptor;
drain that BH before releasing channel resources.

> diff --git a/drivers/dma/qcom/gpi.c b/drivers/dma/qcom/gpi.c
> index a5055a6273af6..4cf68cf8eb2e5 100644
> --- a/drivers/dma/qcom/gpi.c
> +++ b/drivers/dma/qcom/gpi.c
[ ... ]
> @@ -1060,8 +1064,12 @@ static void gpi_process_xfer_compl_event(struct gchan *gchan,
>  	result.residue = gpi_desc->len - compl_event->length;
>  	dev_dbg(gpii->gpi_dev->dev, "Residue %d\n", result.residue);
>  
> -	dma_cookie_complete(&vd->tx);
> -	dmaengine_desc_get_callback_invoke(&vd->tx, &result);
> +	spin_lock_irqsave(&gchan->vc.lock, flags);
> +	list_del(&vd->node);
> +	vd->tx_result = result;
> +	vchan_cookie_complete(vd);

[Severity: High]
Since this patch defers callbacks to the virt-dma bottom-half, does the driver
now need to implement the device_synchronize callback?

When a client driver calls dmaengine_terminate_sync() to ensure no callbacks
are running before freeing its DMA buffers, the absence of device_synchronize
causes it to fall back to dmaengine_terminate_all(), which merely stops the
hardware.

Could the newly introduced deferred BH execute a completion callback after
dmaengine_terminate_sync() has already returned, potentially leading to a
use-after-free in the client driver?

Should gpi_probe() be updated to set gpi_dev->dma_device.device_synchronize
to a function that wraps vchan_synchronize()?

> +	spin_unlock_irqrestore(&gchan->vc.lock, flags);
> +	return;
>  
>  gpi_free_desc:
>  	spin_lock_irqsave(&gchan->vc.lock, flags);
[ ... ]
> @@ -2009,6 +2017,7 @@ static void gpi_free_chan_resources(struct dma_chan *chan)
>  
>  	/* free all allocated memory */
>  	gpi_free_ring(&gchan->ch_ring, gpii);
> +	dma_chan_kill_bh(&gchan->vc.chan);

[Severity: High]
Does this cleanup sequence introduce a race condition that could result in a
divide-by-zero crash?

Because dma_chan_kill_bh() is called after gpi_free_ring(), the bottom-half
is still permitted to run after the channel's ring has been freed and
zero-initialized.

If a deferred callback executes in the BH and calls dmaengine_prep_slave_sg()
to queue the next transfer, it will eventually call gpi_prep_slave_sg(). That
function calls gpi_ring_num_elements_avail(), which blindly accesses the now
zeroed ch_ring and divides by ring->el_size.

Could dma_chan_kill_bh() be moved before gpi_free_ring() to ensure the BH is
drained before tearing down the hardware ring resources?

>  	vchan_free_chan_resources(&gchan->vc);
>  	kfree(gchan->config);
>

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=63

Re: [PATCH v2 56/64] dmaengine: omap-dma: use dma_chan_kill_bh

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] Freeing channel memory in `omap_dma_free` without clearing `od->lch_map` leaves a dangling pointer that can be dereferenced by the interrupt handler in the probe error path.
- [High] In `omap_dma_remove`, the shared IRQ is freed before hardware interrupts are disabled, and the wrong interrupt enable register is cleared.
--

--- Patch [56]: [PATCH v2 56/64] dmaengine: omap-dma: use dma_chan_kill_bh ---
commit fe56c19ed9d7f6c471a97210acc16fdccedd520f
Author: Allen Pais <allen.lkml@gmail.com>

dmaengine: omap-dma: use dma_chan_kill_bh

virt-dma now dispatches completion callbacks through per-channel BH
work instead of its tasklet. Cancel that work during teardown before
channel storage is released.

> diff --git a/drivers/dma/ti/omap-dma.c b/drivers/dma/ti/omap-dma.c
> --- a/drivers/dma/ti/omap-dma.c
> +++ b/drivers/dma/ti/omap-dma.c

[ ... ]

> @@ -1521,7 +1521,7 @@ static void omap_dma_free(struct omap_dmadev *od)
>  			struct omap_chan, vc.chan.device_node);
>  
>  		list_del(&c->vc.chan.device_node);
> -		tasklet_kill(&c->vc.task);
> +		dma_chan_kill_bh(&c->vc.chan);
>  		kfree(c);

[Severity: High]
This is a pre-existing issue, but does freeing the channel memory here leave
a dangling pointer in od->lch_map?

If of_dma_controller_register() fails during probe, omap_dma_free() is called
to loop through all channels and free them. If a client has already requested a
channel after dma_async_device_register() succeeded, od->irq_enable_mask will
be non-zero and od->lch_map will still contain pointers to these now-freed
channels.

Since devm_request_irq() leaves the interrupt handler active until probe
finishes and devres cleans it up, an incoming interrupt could cause
omap_dma_irq() to read the mask, index into od->lch_map, and dereference
the freed memory. 

Should od->lch_map be cleared when freeing the channel?

[ ... ]

[Severity: High]
This is also a pre-existing issue, but does omap_dma_remove() clear the wrong 
IRQ enable register and free the shared IRQ too early?

Looking at omap_dma_remove() in drivers/dma/ti/omap-dma.c:

omap_dma_remove() {
    ...
    devm_free_irq(&pdev->dev, irq, od);
    ...
    if (!omap_dma_legacy(od)) {
        /* Disable all interrupts */
        omap_dma_glbl_write(od, IRQENABLE_L0, 0);
    }
    ...
}

The shared IRQ is freed before hardware interrupts are disabled. It also
writes to IRQENABLE_L0 instead of IRQENABLE_L1, which was used during
setup to enable them. 

If the hardware asserts an interrupt after the handler is removed, could it
assert the shared line indefinitely and cause an unhandled interrupt storm
since the source is never cleared?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=56

Re: [PATCH v2 62/64] dmaengine: hidma: defer callbacks via channel BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:

New issues:
- [Critical] The per-channel bottom-half (BH) is cancelled too late during driver unbind, allowing it to execute after the hardware context is destroyed and dereference a NULL pointer.
- [High] Missing synchronization and cancellation of BH during channel termination leads to use-after-free and memory leaks.

Pre-existing issues:
- [Critical] List manipulation without holding the channel lock in `hidma_terminate_channel` causes list corruption and data races.
- [High] Sleeping function called in atomic context and unsafe PM hardware access.
- [High] Unbalanced pm_runtime usage counter causing premature device suspension.
- [High] Invalid pointer dereference due to missing list emptiness check in `hidma_callback`.
--

--- Patch [62]: [PATCH v2 62/64] dmaengine: hidma: defer callbacks via channel BH ---
commit 3c9c3e0127e064313b46165b6ad21a91d6575ed1
Author: Allen Pais <allen.lkml@gmail.com>
Subject: dmaengine: hidma: defer callbacks via channel BH

Move descriptor callback processing out of the low-level completion
path and schedule it through per-channel BH work. Drain that work while
freeing channels so callbacks cannot outlive channel storage.

> diff --git a/drivers/dma/qcom/hidma.c b/drivers/dma/qcom/hidma.c
> index c939635be21d2..1f875a499ab3f 100644
> --- a/drivers/dma/qcom/hidma.c
> +++ b/drivers/dma/qcom/hidma.c
> @@ -90,7 +90,13 @@ static inline struct hidma_chan *to_hidma_chan(struct dma_chan *dmach)
>  
>  static void hidma_free(struct hidma_dev *dmadev)
>  {
> -	INIT_LIST_HEAD(&dmadev->ddev.channels);
> +	struct hidma_chan *mchan, *next;
> +
> +	list_for_each_entry_safe(mchan, next, &dmadev->ddev.channels,
> +				 chan.device_node) {
> +		dma_chan_kill_bh(&mchan->chan);

[Severity: Critical]
Does flushing the bottom half here happen too late in the teardown sequence?

During driver unbind, hidma_remove() calls hidma_ll_uninit() before
hidma_free().

hidma_ll_uninit() explicitly sets lldev->trepool = NULL. When hidma_free()
then calls dma_chan_kill_bh(), it synchronously flushes the pending bottom
half:

hidma_chan_bh()
  hidma_process_completed()
    hidma_ll_status()
      tre = &lldev->trepool[tre_ch];

Since trepool is already NULL, won't this cause a deterministic NULL pointer
dereference?

> +		list_del(&mchan->chan.device_node);
> +	}
>  }
>  
>  static unsigned int nr_desc_prm;

[ ... ]

> @@ -155,6 +161,13 @@ static void hidma_process_completed(struct hidma_chan *mchan)
>  	}
>  }
>  
> +static void hidma_chan_bh(struct dma_chan *chan)
> +{
> +	struct hidma_chan *mchan = to_hidma_chan(chan);
> +
> +	hidma_process_completed(mchan);
> +}
> +

[Severity: High]
Does the bottom half work need to be cancelled when a client terminates the
channel?

When a client calls dma_release_channel() or dmaengine_terminate_sync(), the
framework eventually calls hidma_free_chan_resources(), which frees the
descriptors.

If the bottom half executes concurrently or after this teardown:

hidma_chan_bh()
  hidma_process_completed()
    list_for_each_entry_safe(mdesc, next, &list, node) {
      dmaengine_desc_callback_invoke(&cb, &result);
    }

Won't it dereference descriptors that were just freed, causing a use-after-free,
and subsequently leak orphaned descriptors onto mchan->free?

>  /*
>   * Called once for each submitted descriptor.
>   * PM is locked once for each descriptor that is currently
> @@ -181,7 +194,7 @@ static void hidma_callback(void *data)
>  	}
>  	spin_unlock_irqrestore(&mchan->lock, irqflags);

[Severity: Critical]
This is a pre-existing issue, but does hidma_terminate_channel() safely
handle concurrent callbacks?

In hidma_terminate_channel(), the lock is dropped before the hardware is
paused:

hidma_terminate_channel()
  spin_unlock_irqrestore(&mchan->lock, irqflags);
  rc = hidma_ll_disable(dmadev->lldev);

During this window, a completed transaction can trigger hidma_callback()
which modifies the list without holding a lock on the local list:

hidma_callback()
  if (mdesc->node.next) {
    list_move_tail(&mdesc->node, &mchan->completed);
  }

Since hidma_terminate_channel() subsequently modifies the same list
locklessly, can this race with hidma_prep_dma_memcpy() and cause severe
list corruption?

[Severity: High]
This is a pre-existing issue, but does hidma_callback() correctly handle an
empty active list?

When updating the running descriptor, the code uses list_first_entry():

hidma_callback()
  mchan->running = list_first_entry(&mchan->active,
                                    struct hidma_desc, node);

If mchan->active is empty, list_first_entry() does not return NULL but
rather an invalid pointer computed from the list head itself.

If hidma_tx_status() is subsequently called, won't it dereference this
invalid pointer via mchan->running->desc.cookie and cause an out-of-bounds
memory read?

> -	hidma_process_completed(mchan);
> +	dma_chan_schedule_bh(&mchan->chan);
>  
>  	if (queued) {
>  		pm_runtime_mark_last_busy(dmadev->ddev.dev);
>  		pm_runtime_put_autosuspend(dmadev->ddev.dev);

[Severity: High]
This isn't a bug introduced by this patch, but is the pm_runtime usage
counter decrement balanced with the increment?

In hidma_issue_pending(), pm_runtime_get() is called only once per
submitted batch of descriptors.

However, the hardware triggers hidma_callback() for each individual
descriptor, and pm_runtime_put_autosuspend() is unconditionally called here
for every single descriptor.

If a client submits a batch of multiple descriptors, won't this underflow
the PM usage count and trigger device auto-suspension while hardware
operations are still in flight?

[Severity: High]
This is a pre-existing issue, but does the tasklet safely call sleeping PM
functions?

The driver implements hidma_issue_task() as a tasklet, which runs in
softirq (atomic) context, but it calls pm_runtime_get_sync():

hidma_issue_task()
  pm_runtime_get_sync(dmadev->ddev.dev);

Since pm_runtime_get_sync() will sleep if the device is suspended, doesn't
this trigger a sleeping in atomic bug?

Furthermore, in hidma_issue_pending(), if the asynchronous pm_runtime_get()
returns 0, the code falls through and immediately calls hidma_ll_start().
Doesn't this access hardware registers while the device is still unclocked
or powered off?

>  	}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=62

Re: [PATCH v2 61/64] dmaengine: dw: defer callbacks via channel BH

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Concurrent list modification between dwc_scan_descriptors() and dwc_terminate_all() leads to an infinite loop and list corruption.
--

--- Patch [61]: [PATCH v2 61/64] dmaengine: dw: defer callbacks via channel BH ---
commit d2107b5998f3b825a629b786e29b948ab851a743
Author: Allen Pais <allen.lkml@gmail.com>
Subject: dmaengine: dw: defer callbacks via channel BH

Queue completed descriptors on the channel and invoke their callbacks
from per-channel BH work instead of the controller tasklet. Add a
synchronize hook so termination can wait for deferred callbacks before
their descriptor storage is released.

> diff --git a/drivers/dma/dw/core.c b/drivers/dma/dw/core.c
> index dd75f97a33b3d..e780116a944bc 100644
> --- a/drivers/dma/dw/core.c
> +++ b/drivers/dma/dw/core.c
> @@ -236,25 +236,24 @@ dwc_descriptor_complete(struct dw_dma_chan *dwc, struct dw_desc *desc,
>  	struct dma_async_tx_descriptor	*txd = &desc->txd;
>  	struct dw_desc			*child;
>  	unsigned long			flags;
>  
>  	dev_vdbg(chan2dev(&dwc->chan), "descriptor %u complete\n", txd->cookie);
>  
>  	spin_lock_irqsave(&dwc->lock, flags);
>  	dma_cookie_complete(txd);
> -	if (callback_required)
> -		dmaengine_desc_get_callback(txd, &cb);
> -	else
> -		memset(&cb, 0, sizeof(cb));
> +	list_del_init(&desc->desc_node);
>  
>  	/* async_tx_ack */
>  	list_for_each_entry(child, &desc->tx_list, desc_node)
>  		async_tx_ack(&child->txd);
>  	async_tx_ack(&desc->txd);
> -	dwc_desc_put(dwc, desc);
> +	if (callback_required) {
> +		list_add_tail(&desc->desc_node, &dwc->completed_list);

[Severity: High]
Can this concurrent list modification lead to an infinite loop during channel
termination?

If dwc_tx_status() races with dwc_terminate_all(), dwc_scan_descriptors()
temporarily drops dwc->lock while iterating over dwc->active_list.

If dwc_terminate_all() runs concurrently, it acquires the lock, splices
active_list to a local list, drops the lock, and begins locklessly iterating
the local list:

drivers/dma/dw/core.c:dwc_terminate_all() {
    ...
    list_splice_init(&dwc->queue, &list);
    list_splice_init(&dwc->active_list, &list);

    spin_unlock_irqrestore(&dwc->lock, flags);

    /* Flush all pending and queued descriptors */
    list_for_each_entry_safe(desc, _desc, &list, desc_node)
        dwc_descriptor_complete(dwc, desc, false);
}

When dwc_scan_descriptors() resumes, it calls dwc_descriptor_complete()
which now modifies desc->desc_node by moving it to dwc->completed_list.

Because dwc_terminate_all()'s lockless list_for_each_entry_safe() traversal
may have already prefetched a descriptor that is subsequently moved, its
iteration could jump into dwc->completed_list.

This would prevent the loop termination condition from ever matching the
stack-allocated list head in dwc_terminate_all(), causing an infinite loop.
Could this sequence happen in practice?

> +		dma_chan_schedule_bh(&dwc->chan);
> +	} else {
> +		dwc_desc_put(dwc, desc);
> +	}
>  	spin_unlock_irqrestore(&dwc->lock, flags);
> -
> -	dmaengine_desc_callback_invoke(&cb, NULL);
>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1785183549.git.allen.lkml@gmail.com?part=61