[PATCH v2 0/5] dmaengine: Add batched scatter-gather DMA support

[PATCH v2 0/5] dmaengine: Add batched scatter-gather DMA support

[Sumit Kumar]

Synopsys DesignWare eDMA IP supports a linked-list (LL) mode where
each LL item carries independent source and destination addresses.
This allows multiple independent memory transfers to be described in
a single linked list and submitted to the hardware as one DMA
transaction, without any CPU intervention between items. The IP
processes LL items strictly in order, guaranteeing that
scatter-gather entries are never reordered.

The `DMA_SG` transaction type and `device_prep_dma_sg()` API were
previously removed from the dmaengine core in commit c678fa66341c
("dmaengine: remove DMA_SG as it is dead code in kernel") because
there were no in-tree users at the time.

This series reintroduces `DMA_SG` and `device_prep_dma_sg()`.

The reintroduced API takes separate source and destination
scatter-gather lists, where entry `i` of the source list is
transferred to entry `i` of the destination list. This enables the
eDMA hardware to process all pairs as a single transaction without
CPU intervention between transfers.

The primary use case is MHI endpoint ring caching. When an MHI ring
wraps around, data spans two non-contiguous memory regions (tail and
head portions). Previously this required two separate DMA
transactions with two interrupts. With this series, both regions are
submitted as a single batched transaction, reducing submission
overhead and interrupt count.

The series includes:
1. Core DMA engine API and DW eDMA driver implementation
2. PCI EPF MHI driver support for batched transfers
3. MHI endpoint ring caching optimization using batched reads

Expected benefits:
- Reduced DMA submission overhead for multiple transfers
- Better hardware utilization through batched operations
- Lower latency for ring wraparound scenarios

Signed-off-by: Sumit Kumar <sumit.kumar@oss.qualcomm.com>
---
Changes in v2:
- Reintroduce `DMA_SG` and `device_prep_dma_sg()` instead of a new
  API. (Vinod)
- Use two parallel scatterlist arrays (src_sg/dst_sg) instead of
  extending struct scatterlist with a second DMA address. (Robin
  Murphy)
- Add explicit eDMA LL hardware description and ordering guarantee to
  cover letter. (Mani)
- Link to v1: https://lore.kernel.org/r/20260313-dma_multi_sg-v1-0-8fabb0d1a759@oss.qualcomm.com

---
Sumit Kumar (5):
      dmaengine: Add DMA_SG support for multi-buffer scatter-gather transfers
      dmaengine: dw-edma: Add DMA_SG support
      PCI: epf-mhi: Use a define for the DMA transfer timeout
      PCI: epf-mhi: Add batched DMA read support
      bus: mhi: ep: Use batched read for ring caching

 Documentation/driver-api/dmaengine/provider.rst |  21 +++
 drivers/bus/mhi/ep/main.c                       |   3 +-
 drivers/bus/mhi/ep/ring.c                       |  36 +++---
 drivers/dma/dmaengine.c                         |   1 +
 drivers/dma/dw-edma/dw-edma-core.c              |  87 ++++++++++++-
 drivers/dma/dw-edma/dw-edma-core.h              |  10 +-
 drivers/pci/endpoint/functions/pci-epf-mhi.c    | 165 +++++++++++++++++++++++-
 include/linux/dmaengine.h                       |  35 +++++
 include/linux/mhi_ep.h                          |   7 +
 9 files changed, 339 insertions(+), 26 deletions(-)
---
base-commit: b4515cf4156356e8f4fe6e0fdc17f59adab9772f
change-id: 20260108-dma_multi_sg-c217650373c2

Best regards,
-- 
Sumit Kumar <sumit.kumar@oss.qualcomm.com>

[PATCH v2 1/5] dmaengine: Add DMA_SG support for multi-buffer scatter-gather transfers

[Sumit Kumar]

A client that needs to copy several independent, non-contiguous memory
regions in one operation currently has to submit one DMA_MEMCPY
transaction per region, each with its own completion interrupt, even
when every region is known up front and the copies could be described
as a single hardware transaction.

Partially reintroduce the DMA_SG transaction type and device_prep_dma_sg()
API that was removed in commit c678fa66341c ("dmaengine: remove DMA_SG as
it is dead code in kernel"). Only the core API is restored here.

The API accepts separate source and destination scatter-gather lists,
where entry i of the source list is transferred to entry i of the
destination list. This allows multiple independent (src[i] -> dst[i])
transfers to be batched into a single DMA transaction instead of N
separate submissions, reducing submission and interrupt overhead.

DMA_SG is a memcpy-class operation: both endpoints are memory buffers,
and neither DMA address is a FIFO-style peripheral register. The source
and destination scatter-gather lists must contain the same number of
entries; providers reject requests where the entry counts differ.

Restore the DMA_SG entry in
Documentation/driver-api/dmaengine/provider.rst and add
CHECK_CAP(dma_sg, DMA_SG) to dma_async_device_register() to validate
that drivers setting the capability provide the corresponding function
pointer.

Signed-off-by: Sumit Kumar <sumit.kumar@oss.qualcomm.com>
---
 Documentation/driver-api/dmaengine/provider.rst | 21 +++++++++++++++
 drivers/dma/dmaengine.c                         |  1 +
 include/linux/dmaengine.h                       | 35 +++++++++++++++++++++++++
 3 files changed, 57 insertions(+)

diff --git a/Documentation/driver-api/dmaengine/provider.rst b/Documentation/driver-api/dmaengine/provider.rst
index f4ed98f701c918ff81bc674845880f8d01efbf1d..638e4b83e9a2f90c056111dbdd7572a4ed0f536d 100644
--- a/Documentation/driver-api/dmaengine/provider.rst
+++ b/Documentation/driver-api/dmaengine/provider.rst
@@ -210,6 +210,27 @@ Currently, the types available are:
   - Used by the client drivers to register a callback that will be
     called on a regular basis through the DMA controller interrupt
 
+- DMA_SG
+
+  - The device supports memory to memory scatter-gather transfers
+    using paired source and destination scatter-gather lists, where
+    entry ``i`` of the source list is transferred to entry ``i`` of
+    the destination list in a single DMA transaction.
+
+  - The source and destination scatter-gather lists must contain the
+    same number of entries; providers reject (return NULL for) requests
+    where the entry counts differ. Providers that walk the two lists in
+    lockstep pair them entry-by-entry as passed in, so clients that
+    DMA-map the lists must ensure the mapped segmentation stays aligned
+    between the two lists (for example by not relying on the DMA layer
+    to merge entries of one list but not the other).
+
+  - Unlike DMA_MEMCPY, neither the source nor destination is a
+    FIFO-style peripheral register; both are memory buffers. Multiple
+    independent (src[i] -> dst[i]) copies are submitted as a single
+    DMA transaction, reducing submission and interrupt overhead
+    compared to N separate DMA_MEMCPY operations.
+
 - DMA_PRIVATE
 
   - The devices only supports slave transfers, and as such isn't
diff --git a/drivers/dma/dmaengine.c b/drivers/dma/dmaengine.c
index 6ffd8bd82154af2af2807d1c8b7ae7475eab56d3..9e790b9f165063d696438ace6370df98dffb8a32 100644
--- a/drivers/dma/dmaengine.c
+++ b/drivers/dma/dmaengine.c
@@ -1211,6 +1211,7 @@ int dma_async_device_register(struct dma_device *device)
 	CHECK_CAP(dma_pq_val,      DMA_PQ_VAL);
 	CHECK_CAP(dma_memset,      DMA_MEMSET);
 	CHECK_CAP(dma_interrupt,   DMA_INTERRUPT);
+	CHECK_CAP(dma_sg,          DMA_SG);
 	CHECK_CAP(dma_cyclic,      DMA_CYCLIC);
 	CHECK_CAP(interleaved_dma, DMA_INTERLEAVE);
 
diff --git a/include/linux/dmaengine.h b/include/linux/dmaengine.h
index fe33a20abc6146d539670e0e6fe6c9d27d96aa2a..61aa72149f5d5cf1828b89d13d0b35f1a12bb000 100644
--- a/include/linux/dmaengine.h
+++ b/include/linux/dmaengine.h
@@ -65,6 +65,7 @@ enum dma_transaction_type {
 	DMA_COMPLETION_NO_ORDER,
 	DMA_REPEAT,
 	DMA_LOAD_EOT,
+	DMA_SG,
 /* last transaction type for creation of the capabilities mask */
 	DMA_TX_TYPE_END,
 };
@@ -848,6 +849,7 @@ struct dma_filter {
  *	The function takes a buffer of size buf_len. The callback function will
  *	be called after period_len bytes have been transferred.
  * @device_prep_interleaved_dma: Transfer expression in a generic way.
+ * @device_prep_dma_sg: prepares a memory to memory scatter-gather operation
  * @device_caps: May be used to override the generic DMA slave capabilities
  *	with per-channel specific ones
  * @device_config: Pushes a new configuration to a channel, return 0 or an error
@@ -954,6 +956,11 @@ struct dma_device {
 	struct dma_async_tx_descriptor *(*device_prep_interleaved_dma)(
 		struct dma_chan *chan, struct dma_interleaved_template *xt,
 		unsigned long flags);
+	struct dma_async_tx_descriptor *(*device_prep_dma_sg)
+		(struct dma_chan *chan,
+		 struct scatterlist *dst_sg, unsigned int dst_nents,
+		 struct scatterlist *src_sg, unsigned int src_nents,
+		 unsigned long flags);
 
 	void (*device_caps)(struct dma_chan *chan, struct dma_slave_caps *caps);
 	int (*device_config)(struct dma_chan *chan, struct dma_slave_config *config);
@@ -1194,6 +1201,34 @@ static inline struct dma_async_tx_descriptor *dmaengine_prep_interleaved_dma(
 	return chan->device->device_prep_interleaved_dma(chan, xt, flags);
 }
 
+/**
+ * dmaengine_prep_dma_sg() - Prepare a memory-to-memory scatter-gather DMA descriptor.
+ * @chan: The channel to be used for this descriptor
+ * @dst_sg: Destination scatter list
+ * @dst_nents: Number of entries in destination scatter list
+ * @src_sg: Source scatter list
+ * @src_nents: Number of entries in source scatter list
+ * @flags: DMA engine flags
+ *
+ * Prepares a DMA transaction that copies data from multiple source memory
+ * regions to multiple destination memory regions in a single DMA transaction.
+ * Entry i of the source list is paired with entry i of the destination list,
+ * so both lists must contain the same number of entries; the call returns
+ * NULL otherwise.
+ */
+static inline struct dma_async_tx_descriptor *dmaengine_prep_dma_sg
+		(struct dma_chan *chan,
+		 struct scatterlist *dst_sg, unsigned int dst_nents,
+		 struct scatterlist *src_sg, unsigned int src_nents,
+		 unsigned long flags)
+{
+	if (!chan || !chan->device || !chan->device->device_prep_dma_sg)
+		return NULL;
+
+	return chan->device->device_prep_dma_sg(chan, dst_sg, dst_nents,
+			src_sg, src_nents, flags);
+}
+
 /**
  * dmaengine_prep_dma_memset() - Prepare a DMA memset descriptor.
  * @chan: The channel to be used for this descriptor

-- 
2.34.1

[PATCH v2 2/5] dmaengine: dw-edma: Add DMA_SG support

[Sumit Kumar]

Synopsys DesignWare eDMA supports a linked-list mode where each list item
carries independent source and destination addresses, letting multiple
independent memory transfers be described in one linked list and submitted
to the hardware as a single DMA transaction. The IP processes list items
strictly in order, so paired scatter-gather entries are never reordered.

Implement the DMA_SG capability by adding a new EDMA_XFER_DUAL_SG transfer
type and a corresponding struct dw_edma_dual_sg carrying the paired source
and destination SG lists. dw_edma_device_transfer() walks both lists in
lockstep, building a single hardware linked-list; a per-entry length
mismatch or premature list end fails the whole request.

The transfer direction is inferred from the channel hardware polarity
(EDMA_DIR_READ/WRITE), not from dma_slave_config.direction: for local eDMA
(DW_EDMA_CHIP_LOCAL) read channels handle DEV_TO_MEM and write channels
handle MEM_TO_DEV; for remote eDMA the mapping is inverted. PCIe bus
addresses are translated via dw_edma_get_pci_address() for the remote side
of each transfer. dmaengine_slave_config() must still be called before
dmaengine_prep_dma_sg() because dw_edma_device_transfer() gates transfers
on chan->configured, even though the direction field itself is unused by
the DMA_SG path.

Signed-off-by: Sumit Kumar <sumit.kumar@oss.qualcomm.com>
---
 drivers/dma/dw-edma/dw-edma-core.c | 87 +++++++++++++++++++++++++++++++++++---
 drivers/dma/dw-edma/dw-edma-core.h | 10 ++++-
 2 files changed, 90 insertions(+), 7 deletions(-)

diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-edma-core.c
index 1f893dc54c7938a9c45bafd975a4f99fdc0acb38..9fbfa5ad65b5421b164d2ab9883233225e0269e4 100644
--- a/drivers/dma/dw-edma/dw-edma-core.c
+++ b/drivers/dma/dw-edma/dw-edma-core.c
@@ -372,6 +372,7 @@ dw_edma_device_transfer(struct dw_edma_transfer *xfer,
 	struct dw_edma_chan *chan = dchan2dw_edma_chan(xfer->dchan);
 	enum dma_transfer_direction dir = xfer->direction;
 	struct scatterlist *sg = NULL;
+	struct scatterlist *dst_sg = NULL;
 	struct dw_edma_burst *burst;
 	struct dw_edma_desc *desc;
 	u64 src_addr, dst_addr;
@@ -429,6 +430,9 @@ dw_edma_device_transfer(struct dw_edma_transfer *xfer,
 			return NULL;
 		if (!xfer->xfer.il->src_inc || !xfer->xfer.il->dst_inc)
 			return NULL;
+	} else if (xfer->type == EDMA_XFER_DUAL_SG) {
+		if (xfer->xfer.dual_sg.len < 1)
+			return NULL;
 	} else {
 		return NULL;
 	}
@@ -441,16 +445,27 @@ dw_edma_device_transfer(struct dw_edma_transfer *xfer,
 		dst_addr = config->dst_addr;
 	}
 
-	if (dir == DMA_DEV_TO_MEM)
-		src_addr = dw_edma_get_pci_address(chan, (phys_addr_t)src_addr);
-	else
-		dst_addr = dw_edma_get_pci_address(chan, (phys_addr_t)dst_addr);
+	/*
+	 * DUAL_SG translates each src/dst sg entry individually below
+	 * (see the burst->sar/burst->dar assignment), so it is exempt
+	 * from the single up-front translation used by other types.
+	 */
+	if (xfer->type != EDMA_XFER_DUAL_SG) {
+		if (dir == DMA_DEV_TO_MEM)
+			src_addr = dw_edma_get_pci_address(chan, (phys_addr_t)src_addr);
+		else
+			dst_addr = dw_edma_get_pci_address(chan, (phys_addr_t)dst_addr);
+	}
 
 	if (xfer->type == EDMA_XFER_CYCLIC) {
 		cnt = xfer->xfer.cyclic.cnt;
 	} else if (xfer->type == EDMA_XFER_SCATTER_GATHER) {
 		cnt = xfer->xfer.sg.len;
 		sg = xfer->xfer.sg.sgl;
+	} else if (xfer->type == EDMA_XFER_DUAL_SG) {
+		cnt = xfer->xfer.dual_sg.len;
+		sg = xfer->xfer.dual_sg.src_sgl;
+		dst_sg = xfer->xfer.dual_sg.dst_sgl;
 	} else if (xfer->type == EDMA_XFER_INTERLEAVED) {
 		cnt = xfer->xfer.il->numf * xfer->xfer.il->frame_size;
 		fsz = xfer->xfer.il->frame_size;
@@ -463,12 +478,23 @@ dw_edma_device_transfer(struct dw_edma_transfer *xfer,
 	for (i = 0; i < cnt; i++) {
 		if (xfer->type == EDMA_XFER_SCATTER_GATHER && !sg)
 			break;
+		/*
+		 * DUAL_SG walks the source and destination lists in lockstep;
+		 * a premature end or a per-entry length mismatch would leave
+		 * the descriptor partially built, so fail the whole request.
+		 */
+		if (xfer->type == EDMA_XFER_DUAL_SG &&
+		    (!sg || !dst_sg || sg_dma_len(sg) != sg_dma_len(dst_sg))) {
+			kfree(desc);
+			return NULL;
+		}
 
 		burst = desc->burst + i;
 
 		if (xfer->type == EDMA_XFER_CYCLIC)
 			burst->sz = xfer->xfer.cyclic.len;
-		else if (xfer->type == EDMA_XFER_SCATTER_GATHER)
+		else if (xfer->type == EDMA_XFER_SCATTER_GATHER ||
+			 xfer->type == EDMA_XFER_DUAL_SG)
 			burst->sz = sg_dma_len(sg);
 		else if (xfer->type == EDMA_XFER_INTERLEAVED)
 			burst->sz = xfer->xfer.il->sgl[i % fsz].size;
@@ -492,6 +518,9 @@ dw_edma_device_transfer(struct dw_edma_transfer *xfer,
 				 */
 			} else if (xfer->type == EDMA_XFER_INTERLEAVED) {
 				burst->dar = dst_addr;
+			} else if (xfer->type == EDMA_XFER_DUAL_SG) {
+				burst->sar = dw_edma_get_pci_address(chan, sg_dma_address(sg));
+				burst->dar = sg_dma_address(dst_sg);
 			}
 		} else {
 			burst->dar = dst_addr;
@@ -507,13 +536,19 @@ dw_edma_device_transfer(struct dw_edma_transfer *xfer,
 				 * and destination addresses are increased
 				 * by the same portion (data length)
 				 */
-			}  else if (xfer->type == EDMA_XFER_INTERLEAVED) {
+			} else if (xfer->type == EDMA_XFER_INTERLEAVED) {
 				burst->sar = src_addr;
+			} else if (xfer->type == EDMA_XFER_DUAL_SG) {
+				burst->sar = sg_dma_address(sg);
+				burst->dar = dw_edma_get_pci_address(chan, sg_dma_address(dst_sg));
 			}
 		}
 
 		if (xfer->type == EDMA_XFER_SCATTER_GATHER) {
 			sg = sg_next(sg);
+		} else if (xfer->type == EDMA_XFER_DUAL_SG) {
+			sg = sg_next(sg);
+			dst_sg = sg_next(dst_sg);
 		} else if (xfer->type == EDMA_XFER_INTERLEAVED) {
 			struct dma_interleaved_template *il = xfer->xfer.il;
 			struct data_chunk *dc = &il->sgl[i % fsz];
@@ -613,6 +648,44 @@ static void dw_hdma_set_callback_result(struct virt_dma_desc *vd,
 	res->residue = residue;
 }
 
+static struct dma_async_tx_descriptor *
+dw_edma_device_prep_dma_sg(struct dma_chan *dchan,
+			   struct scatterlist *dst_sg, unsigned int dst_nents,
+			   struct scatterlist *src_sg, unsigned int src_nents,
+			   unsigned long flags)
+{
+	struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan);
+	struct dw_edma_transfer xfer;
+	enum dma_transfer_direction dir;
+
+	if (src_nents != dst_nents || !src_nents)
+		return NULL;
+
+	if (!src_sg || !dst_sg)
+		return NULL;
+
+	/* Determine direction from channel configuration */
+	if (chan->dw->chip->flags & DW_EDMA_CHIP_LOCAL)
+		dir = (chan->dir == EDMA_DIR_READ) ? DMA_DEV_TO_MEM : DMA_MEM_TO_DEV;
+	else
+		dir = (chan->dir == EDMA_DIR_WRITE) ? DMA_DEV_TO_MEM : DMA_MEM_TO_DEV;
+
+	xfer.dchan = dchan;
+	xfer.direction = dir;
+	xfer.xfer.dual_sg.src_sgl = src_sg;
+	xfer.xfer.dual_sg.dst_sgl = dst_sg;
+	xfer.xfer.dual_sg.len = src_nents;
+	xfer.flags = flags;
+	xfer.type = EDMA_XFER_DUAL_SG;
+
+	/*
+	 * dw_edma_device_transfer() rejects unconfigured channels, so
+	 * dmaengine_slave_config() must have been called on this channel
+	 * beforehand even though the direction field is unused here.
+	 */
+	return dw_edma_device_transfer(&xfer, dw_edma_device_get_config(dchan, NULL));
+}
+
 static void dw_edma_done_interrupt(struct dw_edma_chan *chan)
 {
 	struct dw_edma_desc *desc;
@@ -997,6 +1070,7 @@ static int dw_edma_channel_setup(struct dw_edma *dw, u32 wr_alloc, u32 rd_alloc)
 	dma_cap_set(DMA_CYCLIC, dma->cap_mask);
 	dma_cap_set(DMA_PRIVATE, dma->cap_mask);
 	dma_cap_set(DMA_INTERLEAVE, dma->cap_mask);
+	dma_cap_set(DMA_SG, dma->cap_mask);
 	dma->directions = BIT(DMA_DEV_TO_MEM) | BIT(DMA_MEM_TO_DEV);
 	dma->src_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_4_BYTES);
 	dma->dst_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_4_BYTES);
@@ -1017,6 +1091,7 @@ static int dw_edma_channel_setup(struct dw_edma *dw, u32 wr_alloc, u32 rd_alloc)
 	dma->device_prep_config_sg = dw_edma_device_prep_config_sg;
 	dma->device_prep_dma_cyclic = dw_edma_device_prep_dma_cyclic;
 	dma->device_prep_interleaved_dma = dw_edma_device_prep_interleaved_dma;
+	dma->device_prep_dma_sg = dw_edma_device_prep_dma_sg;
 
 	dma_set_max_seg_size(dma->dev, U32_MAX);
 
diff --git a/drivers/dma/dw-edma/dw-edma-core.h b/drivers/dma/dw-edma/dw-edma-core.h
index f6a5ad31756723e6fd72f7d9d903a1204e25c0b0..e3219ae96d84f528fb7d77dbe3a0b3d7c1264f05 100644
--- a/drivers/dma/dw-edma/dw-edma-core.h
+++ b/drivers/dma/dw-edma/dw-edma-core.h
@@ -38,7 +38,8 @@ enum dw_edma_status {
 enum dw_edma_xfer_type {
 	EDMA_XFER_SCATTER_GATHER = 0,
 	EDMA_XFER_CYCLIC,
-	EDMA_XFER_INTERLEAVED
+	EDMA_XFER_INTERLEAVED,
+	EDMA_XFER_DUAL_SG,
 };
 
 struct dw_edma_chan;
@@ -151,6 +152,12 @@ struct dw_edma_sg {
 	unsigned int			len;
 };
 
+struct dw_edma_dual_sg {
+	struct scatterlist		*src_sgl;
+	struct scatterlist		*dst_sgl;
+	unsigned int			len;
+};
+
 struct dw_edma_cyclic {
 	dma_addr_t			paddr;
 	size_t				len;
@@ -163,6 +170,7 @@ struct dw_edma_transfer {
 		struct dw_edma_sg		sg;
 		struct dw_edma_cyclic		cyclic;
 		struct dma_interleaved_template *il;
+		struct dw_edma_dual_sg		dual_sg;
 	} xfer;
 	enum dma_transfer_direction	direction;
 	unsigned long			flags;

-- 
2.34.1

Re: [PATCH v2 2/5] dmaengine: dw-edma: Add DMA_SG support

[Frank Li]

On Mon, Aug 03, 2026 at 04:01:44PM +0530, Sumit Kumar wrote:
> [You don't often get email from sumit.kumar@oss.qualcomm.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
>
> Synopsys DesignWare eDMA supports a linked-list mode where each list item
> carries independent source and destination addresses, letting multiple
> independent memory transfers be described in one linked list and submitted
> to the hardware as a single DMA transaction. The IP processes list items
> strictly in order, so paired scatter-gather entries are never reordered.
>
> Implement the DMA_SG capability by adding a new EDMA_XFER_DUAL_SG transfer
> type and a corresponding struct dw_edma_dual_sg carrying the paired source
> and destination SG lists. dw_edma_device_transfer() walks both lists in
> lockstep, building a single hardware linked-list; a per-entry length
> mismatch or premature list end fails the whole request.

It is not as simple as it.  for example, if you want to transfer 9k data
from src to dest

src virtual addr 0x1004,        dest 0xA0001008

when map_sg, src's sg maybe

0x1000  .. 4k   offset 4,
0x9000  .. 4k   offset 0
0xA000  .. 4k   offset 0

or
0x1000 .. 8k  offset 4
0xA000 .. 4k  offset 0

which totally depend on physical address allocation although most likely
first case happen

dest sg
0xA001000 .. 4k offset 8
0xA008000 .. 4k offset 0
0xA00E000 .. 4k offset 0

descriptors
1 	transfer 4k-8
2       transfer tail 4 byte
3 	transfer 4k-8
4       transfer tail 4 byte
...

the start address of src and dest is highly possible differences. So it
is very hard to match your requirement, both sg's structure is the same.

Frank

>
> The transfer direction is inferred from the channel hardware polarity
> (EDMA_DIR_READ/WRITE), not from dma_slave_config.direction: for local eDMA
> (DW_EDMA_CHIP_LOCAL) read channels handle DEV_TO_MEM and write channels
> handle MEM_TO_DEV; for remote eDMA the mapping is inverted. PCIe bus
> addresses are translated via dw_edma_get_pci_address() for the remote side
> of each transfer. dmaengine_slave_config() must still be called before
> dmaengine_prep_dma_sg() because dw_edma_device_transfer() gates transfers
> on chan->configured, even though the direction field itself is unused by
> the DMA_SG path.
>
> Signed-off-by: Sumit Kumar <sumit.kumar@oss.qualcomm.com>
> ---
>  drivers/dma/dw-edma/dw-edma-core.c | 87 +++++++++++++++++++++++++++++++++++---
>  drivers/dma/dw-edma/dw-edma-core.h | 10 ++++-
>  2 files changed, 90 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-edma-core.c
> index 1f893dc54c7938a9c45bafd975a4f99fdc0acb38..9fbfa5ad65b5421b164d2ab9883233225e0269e4 100644
> --- a/drivers/dma/dw-edma/dw-edma-core.c
> +++ b/drivers/dma/dw-edma/dw-edma-core.c
> @@ -372,6 +372,7 @@ dw_edma_device_transfer(struct dw_edma_transfer *xfer,
>         struct dw_edma_chan *chan = dchan2dw_edma_chan(xfer->dchan);
>         enum dma_transfer_direction dir = xfer->direction;
>         struct scatterlist *sg = NULL;
> +       struct scatterlist *dst_sg = NULL;
>         struct dw_edma_burst *burst;
>         struct dw_edma_desc *desc;
>         u64 src_addr, dst_addr;
> @@ -429,6 +430,9 @@ dw_edma_device_transfer(struct dw_edma_transfer *xfer,
>                         return NULL;
>                 if (!xfer->xfer.il->src_inc || !xfer->xfer.il->dst_inc)
>                         return NULL;
> +       } else if (xfer->type == EDMA_XFER_DUAL_SG) {
> +               if (xfer->xfer.dual_sg.len < 1)
> +                       return NULL;
>         } else {
>                 return NULL;
>         }
> @@ -441,16 +445,27 @@ dw_edma_device_transfer(struct dw_edma_transfer *xfer,
>                 dst_addr = config->dst_addr;
>         }
>
> -       if (dir == DMA_DEV_TO_MEM)
> -               src_addr = dw_edma_get_pci_address(chan, (phys_addr_t)src_addr);
> -       else
> -               dst_addr = dw_edma_get_pci_address(chan, (phys_addr_t)dst_addr);
> +       /*
> +        * DUAL_SG translates each src/dst sg entry individually below
> +        * (see the burst->sar/burst->dar assignment), so it is exempt
> +        * from the single up-front translation used by other types.
> +        */
> +       if (xfer->type != EDMA_XFER_DUAL_SG) {
> +               if (dir == DMA_DEV_TO_MEM)
> +                       src_addr = dw_edma_get_pci_address(chan, (phys_addr_t)src_addr);
> +               else
> +                       dst_addr = dw_edma_get_pci_address(chan, (phys_addr_t)dst_addr);
> +       }
>
>         if (xfer->type == EDMA_XFER_CYCLIC) {
>                 cnt = xfer->xfer.cyclic.cnt;
>         } else if (xfer->type == EDMA_XFER_SCATTER_GATHER) {
>                 cnt = xfer->xfer.sg.len;
>                 sg = xfer->xfer.sg.sgl;
> +       } else if (xfer->type == EDMA_XFER_DUAL_SG) {
> +               cnt = xfer->xfer.dual_sg.len;
> +               sg = xfer->xfer.dual_sg.src_sgl;
> +               dst_sg = xfer->xfer.dual_sg.dst_sgl;
>         } else if (xfer->type == EDMA_XFER_INTERLEAVED) {
>                 cnt = xfer->xfer.il->numf * xfer->xfer.il->frame_size;
>                 fsz = xfer->xfer.il->frame_size;
> @@ -463,12 +478,23 @@ dw_edma_device_transfer(struct dw_edma_transfer *xfer,
>         for (i = 0; i < cnt; i++) {
>                 if (xfer->type == EDMA_XFER_SCATTER_GATHER && !sg)
>                         break;
> +               /*
> +                * DUAL_SG walks the source and destination lists in lockstep;
> +                * a premature end or a per-entry length mismatch would leave
> +                * the descriptor partially built, so fail the whole request.
> +                */
> +               if (xfer->type == EDMA_XFER_DUAL_SG &&
> +                   (!sg || !dst_sg || sg_dma_len(sg) != sg_dma_len(dst_sg))) {
> +                       kfree(desc);
> +                       return NULL;
> +               }
>
>                 burst = desc->burst + i;
>
>                 if (xfer->type == EDMA_XFER_CYCLIC)
>                         burst->sz = xfer->xfer.cyclic.len;
> -               else if (xfer->type == EDMA_XFER_SCATTER_GATHER)
> +               else if (xfer->type == EDMA_XFER_SCATTER_GATHER ||
> +                        xfer->type == EDMA_XFER_DUAL_SG)
>                         burst->sz = sg_dma_len(sg);
>                 else if (xfer->type == EDMA_XFER_INTERLEAVED)
>                         burst->sz = xfer->xfer.il->sgl[i % fsz].size;
> @@ -492,6 +518,9 @@ dw_edma_device_transfer(struct dw_edma_transfer *xfer,
>                                  */
>                         } else if (xfer->type == EDMA_XFER_INTERLEAVED) {
>                                 burst->dar = dst_addr;
> +                       } else if (xfer->type == EDMA_XFER_DUAL_SG) {
> +                               burst->sar = dw_edma_get_pci_address(chan, sg_dma_address(sg));
> +                               burst->dar = sg_dma_address(dst_sg);
>                         }
>                 } else {
>                         burst->dar = dst_addr;
> @@ -507,13 +536,19 @@ dw_edma_device_transfer(struct dw_edma_transfer *xfer,
>                                  * and destination addresses are increased
>                                  * by the same portion (data length)
>                                  */
> -                       }  else if (xfer->type == EDMA_XFER_INTERLEAVED) {
> +                       } else if (xfer->type == EDMA_XFER_INTERLEAVED) {
>                                 burst->sar = src_addr;
> +                       } else if (xfer->type == EDMA_XFER_DUAL_SG) {
> +                               burst->sar = sg_dma_address(sg);
> +                               burst->dar = dw_edma_get_pci_address(chan, sg_dma_address(dst_sg));
>                         }
>                 }
>
>                 if (xfer->type == EDMA_XFER_SCATTER_GATHER) {
>                         sg = sg_next(sg);
> +               } else if (xfer->type == EDMA_XFER_DUAL_SG) {
> +                       sg = sg_next(sg);
> +                       dst_sg = sg_next(dst_sg);
>                 } else if (xfer->type == EDMA_XFER_INTERLEAVED) {
>                         struct dma_interleaved_template *il = xfer->xfer.il;
>                         struct data_chunk *dc = &il->sgl[i % fsz];
> @@ -613,6 +648,44 @@ static void dw_hdma_set_callback_result(struct virt_dma_desc *vd,
>         res->residue = residue;
>  }
>
> +static struct dma_async_tx_descriptor *
> +dw_edma_device_prep_dma_sg(struct dma_chan *dchan,
> +                          struct scatterlist *dst_sg, unsigned int dst_nents,
> +                          struct scatterlist *src_sg, unsigned int src_nents,
> +                          unsigned long flags)
> +{
> +       struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan);
> +       struct dw_edma_transfer xfer;
> +       enum dma_transfer_direction dir;
> +
> +       if (src_nents != dst_nents || !src_nents)
> +               return NULL;
> +
> +       if (!src_sg || !dst_sg)
> +               return NULL;
> +
> +       /* Determine direction from channel configuration */
> +       if (chan->dw->chip->flags & DW_EDMA_CHIP_LOCAL)
> +               dir = (chan->dir == EDMA_DIR_READ) ? DMA_DEV_TO_MEM : DMA_MEM_TO_DEV;
> +       else
> +               dir = (chan->dir == EDMA_DIR_WRITE) ? DMA_DEV_TO_MEM : DMA_MEM_TO_DEV;
> +
> +       xfer.dchan = dchan;
> +       xfer.direction = dir;
> +       xfer.xfer.dual_sg.src_sgl = src_sg;
> +       xfer.xfer.dual_sg.dst_sgl = dst_sg;
> +       xfer.xfer.dual_sg.len = src_nents;
> +       xfer.flags = flags;
> +       xfer.type = EDMA_XFER_DUAL_SG;
> +
> +       /*
> +        * dw_edma_device_transfer() rejects unconfigured channels, so
> +        * dmaengine_slave_config() must have been called on this channel
> +        * beforehand even though the direction field is unused here.
> +        */
> +       return dw_edma_device_transfer(&xfer, dw_edma_device_get_config(dchan, NULL));
> +}
> +
>  static void dw_edma_done_interrupt(struct dw_edma_chan *chan)
>  {
>         struct dw_edma_desc *desc;
> @@ -997,6 +1070,7 @@ static int dw_edma_channel_setup(struct dw_edma *dw, u32 wr_alloc, u32 rd_alloc)
>         dma_cap_set(DMA_CYCLIC, dma->cap_mask);
>         dma_cap_set(DMA_PRIVATE, dma->cap_mask);
>         dma_cap_set(DMA_INTERLEAVE, dma->cap_mask);
> +       dma_cap_set(DMA_SG, dma->cap_mask);
>         dma->directions = BIT(DMA_DEV_TO_MEM) | BIT(DMA_MEM_TO_DEV);
>         dma->src_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_4_BYTES);
>         dma->dst_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_4_BYTES);
> @@ -1017,6 +1091,7 @@ static int dw_edma_channel_setup(struct dw_edma *dw, u32 wr_alloc, u32 rd_alloc)
>         dma->device_prep_config_sg = dw_edma_device_prep_config_sg;
>         dma->device_prep_dma_cyclic = dw_edma_device_prep_dma_cyclic;
>         dma->device_prep_interleaved_dma = dw_edma_device_prep_interleaved_dma;
> +       dma->device_prep_dma_sg = dw_edma_device_prep_dma_sg;
>
>         dma_set_max_seg_size(dma->dev, U32_MAX);
>
> diff --git a/drivers/dma/dw-edma/dw-edma-core.h b/drivers/dma/dw-edma/dw-edma-core.h
> index f6a5ad31756723e6fd72f7d9d903a1204e25c0b0..e3219ae96d84f528fb7d77dbe3a0b3d7c1264f05 100644
> --- a/drivers/dma/dw-edma/dw-edma-core.h
> +++ b/drivers/dma/dw-edma/dw-edma-core.h
> @@ -38,7 +38,8 @@ enum dw_edma_status {
>  enum dw_edma_xfer_type {
>         EDMA_XFER_SCATTER_GATHER = 0,
>         EDMA_XFER_CYCLIC,
> -       EDMA_XFER_INTERLEAVED
> +       EDMA_XFER_INTERLEAVED,
> +       EDMA_XFER_DUAL_SG,
>  };
>
>  struct dw_edma_chan;
> @@ -151,6 +152,12 @@ struct dw_edma_sg {
>         unsigned int                    len;
>  };
>
> +struct dw_edma_dual_sg {
> +       struct scatterlist              *src_sgl;
> +       struct scatterlist              *dst_sgl;
> +       unsigned int                    len;
> +};
> +
>  struct dw_edma_cyclic {
>         dma_addr_t                      paddr;
>         size_t                          len;
> @@ -163,6 +170,7 @@ struct dw_edma_transfer {
>                 struct dw_edma_sg               sg;
>                 struct dw_edma_cyclic           cyclic;
>                 struct dma_interleaved_template *il;
> +               struct dw_edma_dual_sg          dual_sg;
>         } xfer;
>         enum dma_transfer_direction     direction;
>         unsigned long                   flags;
>
> --
> 2.34.1
>

[PATCH v2 3/5] PCI: epf-mhi: Use a define for the DMA transfer timeout

[Sumit Kumar]

The eDMA read and write paths open-code a 1000 ms completion timeout as a
msecs_to_jiffies(1000) literal. Replace both with a
PCI_EPF_MHI_DMA_TIMEOUT_MS define so the value is named and shared. No
functional change.

Signed-off-by: Sumit Kumar <sumit.kumar@oss.qualcomm.com>
---
 drivers/pci/endpoint/functions/pci-epf-mhi.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/endpoint/functions/pci-epf-mhi.c b/drivers/pci/endpoint/functions/pci-epf-mhi.c
index c3e3b58fb86cd75e175b69ca45530610c500b99e..6bac69fc84b472c5f1abbeede2b441b5db73c784 100644
--- a/drivers/pci/endpoint/functions/pci-epf-mhi.c
+++ b/drivers/pci/endpoint/functions/pci-epf-mhi.c
@@ -21,6 +21,8 @@
 /* Platform specific flags */
 #define MHI_EPF_USE_DMA BIT(0)
 
+#define PCI_EPF_MHI_DMA_TIMEOUT_MS 1000
+
 struct pci_epf_mhi_dma_transfer {
 	struct pci_epf_mhi *epf_mhi;
 	struct mhi_ep_buf_info buf_info;
@@ -357,7 +359,7 @@ static int pci_epf_mhi_edma_read(struct mhi_ep_cntrl *mhi_cntrl,
 	}
 
 	dma_async_issue_pending(chan);
-	ret = wait_for_completion_timeout(&complete, msecs_to_jiffies(1000));
+	ret = wait_for_completion_timeout(&complete, msecs_to_jiffies(PCI_EPF_MHI_DMA_TIMEOUT_MS));
 	if (!ret) {
 		dev_err(dev, "DMA transfer timeout\n");
 		dmaengine_terminate_sync(chan);
@@ -425,7 +427,7 @@ static int pci_epf_mhi_edma_write(struct mhi_ep_cntrl *mhi_cntrl,
 	}
 
 	dma_async_issue_pending(chan);
-	ret = wait_for_completion_timeout(&complete, msecs_to_jiffies(1000));
+	ret = wait_for_completion_timeout(&complete, msecs_to_jiffies(PCI_EPF_MHI_DMA_TIMEOUT_MS));
 	if (!ret) {
 		dev_err(dev, "DMA transfer timeout\n");
 		dmaengine_terminate_sync(chan);

-- 
2.34.1

[PATCH v2 4/5] PCI: epf-mhi: Add batched DMA read support

[Sumit Kumar]

Add support for batched DMA transfers in the PCI EPF MHI driver to
improve performance when reading multiple buffers from the host.

Implement two variants of the read_batch() callback:
pci_epf_mhi_edma_read_batch() is a DMA-optimized implementation that uses
dmaengine_prep_dma_sg() to transfer multiple buffers in a single DMA
transaction, while pci_epf_mhi_iatu_read_batch() serves as a CPU-copy
fallback for platforms without DMA support by sequentially processing each
buffer via IATU mapping. Wire up read_batch() to the eDMA variant only
when the RX DMA channel advertises the DMA_SG capability, falling back to
the IATU variant otherwise.

On a successful batch, notify completion for every buffer via its
caller-supplied buf_info->cb, mirroring the read_sync/read_async
completion semantics so the MHI stack can free each buffer and raise its
transfer completion event. Make read_batch() a mandatory callback in
mhi_ep_register_controller(), alongside read_sync/write_sync/read_async/
write_async.

This enables the MHI endpoint stack to cache ring data efficiently,
particularly for wraparound scenarios where ring data spans two
non-contiguous memory regions.

Signed-off-by: Sumit Kumar <sumit.kumar@oss.qualcomm.com>
---
 drivers/bus/mhi/ep/main.c                    |   3 +-
 drivers/pci/endpoint/functions/pci-epf-mhi.c | 159 +++++++++++++++++++++++++++
 include/linux/mhi_ep.h                       |   7 ++
 3 files changed, 168 insertions(+), 1 deletion(-)

diff --git a/drivers/bus/mhi/ep/main.c b/drivers/bus/mhi/ep/main.c
index 21bc2c50170ff7f0181c0d0d4eefd51db30f34c8..c45240fc481dc3260a61b546c223b6be8e105587 100644
--- a/drivers/bus/mhi/ep/main.c
+++ b/drivers/bus/mhi/ep/main.c
@@ -1459,7 +1459,8 @@ int mhi_ep_register_controller(struct mhi_ep_cntrl *mhi_cntrl,
 		return -EINVAL;
 
 	if (!mhi_cntrl->read_sync || !mhi_cntrl->write_sync ||
-	    !mhi_cntrl->read_async || !mhi_cntrl->write_async)
+	    !mhi_cntrl->read_async || !mhi_cntrl->write_async ||
+	    !mhi_cntrl->read_batch)
 		return -EINVAL;
 
 	ret = mhi_ep_chan_init(mhi_cntrl, config);
diff --git a/drivers/pci/endpoint/functions/pci-epf-mhi.c b/drivers/pci/endpoint/functions/pci-epf-mhi.c
index 6bac69fc84b472c5f1abbeede2b441b5db73c784..ad245148de1462cffc950ba9a9e232405514dfb7 100644
--- a/drivers/pci/endpoint/functions/pci-epf-mhi.c
+++ b/drivers/pci/endpoint/functions/pci-epf-mhi.c
@@ -444,6 +444,162 @@ static int pci_epf_mhi_edma_write(struct mhi_ep_cntrl *mhi_cntrl,
 	return ret;
 }
 
+static int pci_epf_mhi_iatu_read_batch(struct mhi_ep_cntrl *mhi_cntrl,
+				       struct mhi_ep_buf_info *buf_info_array,
+				       u32 num_buffers)
+{
+	struct pci_epf_mhi *epf_mhi = to_epf_mhi(mhi_cntrl);
+	struct device *dev = &epf_mhi->epf->dev;
+	u32 i;
+	int ret;
+
+	if (num_buffers == 0)
+		return -EINVAL;
+
+	for (i = 0; i < num_buffers; i++) {
+		ret = pci_epf_mhi_iatu_read(mhi_cntrl, &buf_info_array[i]);
+		if (ret < 0) {
+			dev_err(dev, "Failed to read buffer %u of %u in batch: %d\n",
+				i, num_buffers, ret);
+			return ret;
+		}
+	}
+
+	return 0;
+}
+
+static int pci_epf_mhi_edma_read_batch(struct mhi_ep_cntrl *mhi_cntrl,
+				       struct mhi_ep_buf_info *buf_info_array,
+				       u32 num_buffers)
+{
+	struct pci_epf_mhi *epf_mhi = to_epf_mhi(mhi_cntrl);
+	struct device *dma_dev = epf_mhi->epf->epc->dev.parent;
+	struct device *dev = &epf_mhi->epf->dev;
+	struct dma_async_tx_descriptor *desc;
+	struct dma_slave_config config = {};
+	DECLARE_COMPLETION_ONSTACK(complete);
+	struct scatterlist *src_sg;
+	struct scatterlist *dst_sg;
+	unsigned long time_left;
+	struct dma_chan *chan;
+	dma_cookie_t cookie;
+	unsigned int i;
+	int mapped;
+	void *buf;
+	int ret;
+
+	if (num_buffers == 0)
+		return -EINVAL;
+
+	/*
+	 * Single allocation carved into two arrays: src_sg[], dst_sg[].
+	 * Reduces allocator round-trips on the ring-cache hot path. Done
+	 * before taking the lock so direct reclaim cannot stall other
+	 * transfers waiting on epf_mhi->lock.
+	 */
+	buf = kcalloc(num_buffers, 2 * sizeof(*src_sg), GFP_KERNEL);
+	if (!buf)
+		return -ENOMEM;
+	src_sg = buf;
+	dst_sg = src_sg + num_buffers;
+
+	mutex_lock(&epf_mhi->lock);
+
+	chan = epf_mhi->dma_chan_rx;
+	if (!chan) {
+		ret = -ENODEV;
+		goto err_unlock;
+	}
+
+	sg_init_table(src_sg, num_buffers);
+	sg_init_table(dst_sg, num_buffers);
+
+	for (i = 0; i < num_buffers; i++) {
+		/*
+		 * src addresses are PCIe host bus addresses already visible to
+		 * the eDMA engine; no dma_map_sg() is needed for the source list.
+		 */
+		sg_dma_address(&src_sg[i]) = buf_info_array[i].host_addr;
+		sg_dma_len(&src_sg[i]) = buf_info_array[i].size;
+
+		sg_set_buf(&dst_sg[i], buf_info_array[i].dev_addr, buf_info_array[i].size);
+	}
+
+	mapped = dma_map_sg(dma_dev, dst_sg, num_buffers, DMA_FROM_DEVICE);
+	if (!mapped) {
+		dev_err(dev, "Failed to map destination buffers for %u-buffer batch read\n",
+			num_buffers);
+		ret = -EIO;
+		goto err_unlock;
+	}
+
+	config.direction = DMA_DEV_TO_MEM;
+	ret = dmaengine_slave_config(chan, &config);
+	if (ret) {
+		dev_err(dev, "Failed to configure DMA channel for %u-buffer batch read: %d\n",
+			num_buffers, ret);
+		goto err_unmap;
+	}
+
+	desc = dmaengine_prep_dma_sg(chan, dst_sg, num_buffers,
+				     src_sg, num_buffers,
+				     DMA_CTRL_ACK | DMA_PREP_INTERRUPT);
+	if (!desc) {
+		dev_err(dev, "Failed to prepare batch DMA\n");
+		ret = -EIO;
+		goto err_unmap;
+	}
+
+	desc->callback = pci_epf_mhi_dma_callback;
+	desc->callback_param = &complete;
+
+	cookie = dmaengine_submit(desc);
+	ret = dma_submit_error(cookie);
+	if (ret) {
+		dev_err(dev, "Failed to submit DMA\n");
+		if (dmaengine_terminate_sync(chan))
+			dev_err(dev, "Failed to terminate DMA channel after submit failure\n");
+		goto err_unmap;
+	}
+
+	dma_async_issue_pending(chan);
+
+	time_left = wait_for_completion_timeout(&complete,
+						msecs_to_jiffies(PCI_EPF_MHI_DMA_TIMEOUT_MS));
+	if (!time_left) {
+		dev_err(dev, "DMA transfer timeout\n");
+		if (dmaengine_terminate_sync(chan))
+			dev_err(dev, "Failed to terminate DMA channel after timeout\n");
+		ret = -ETIMEDOUT;
+		goto err_unmap;
+	}
+
+	ret = 0;
+
+err_unmap:
+	/* dma_unmap_sg() must use the nents passed to dma_map_sg(), not its return value */
+	dma_unmap_sg(dma_dev, dst_sg, num_buffers, DMA_FROM_DEVICE);
+err_unlock:
+	mutex_unlock(&epf_mhi->lock);
+
+	kfree(buf);
+
+	/*
+	 * On a successful batch, notify completion for every buffer via its
+	 * caller-supplied callback so MHI can free the backing buffer and raise
+	 * the transfer completion event per entry, matching the
+	 * read_sync/read_async semantics. Done after dropping the lock as a
+	 * callback may re-enter the driver and epf_mhi->lock is not reentrant.
+	 */
+	if (!ret) {
+		for (i = 0; i < num_buffers; i++)
+			if (buf_info_array[i].cb)
+				buf_info_array[i].cb(&buf_info_array[i]);
+	}
+
+	return ret;
+}
+
 static void pci_epf_mhi_dma_worker(struct work_struct *work)
 {
 	struct pci_epf_mhi *epf_mhi = container_of(work, struct pci_epf_mhi, dma_work);
@@ -789,11 +945,14 @@ static int pci_epf_mhi_link_up(struct pci_epf *epf)
 	mhi_cntrl->unmap_free = pci_epf_mhi_unmap_free;
 	mhi_cntrl->read_sync = mhi_cntrl->read_async = pci_epf_mhi_iatu_read;
 	mhi_cntrl->write_sync = mhi_cntrl->write_async = pci_epf_mhi_iatu_write;
+	mhi_cntrl->read_batch = pci_epf_mhi_iatu_read_batch;
 	if (info->flags & MHI_EPF_USE_DMA) {
 		mhi_cntrl->read_sync = pci_epf_mhi_edma_read;
 		mhi_cntrl->write_sync = pci_epf_mhi_edma_write;
 		mhi_cntrl->read_async = pci_epf_mhi_edma_read_async;
 		mhi_cntrl->write_async = pci_epf_mhi_edma_write_async;
+		if (dma_has_cap(DMA_SG, epf_mhi->dma_chan_rx->device->cap_mask))
+			mhi_cntrl->read_batch = pci_epf_mhi_edma_read_batch;
 	}
 
 	/* Register the MHI EP controller */
diff --git a/include/linux/mhi_ep.h b/include/linux/mhi_ep.h
index 7b40fc8cbe77ab8419d167e89264b69a817b9fb1..51f66cae937a53a93e9ae76cdcfaac7b1e08283f 100644
--- a/include/linux/mhi_ep.h
+++ b/include/linux/mhi_ep.h
@@ -107,6 +107,11 @@ struct mhi_ep_buf_info {
  * @write_sync: CB function for writing to host memory synchronously
  * @read_async: CB function for reading from host memory asynchronously
  * @write_async: CB function for writing to host memory asynchronously
+ * @read_batch: CB function for reading from host memory in batches synchronously.
+ *              Callers must pass at least one buffer (num_buffers > 0); each
+ *              buf_info entry must have host_addr, dev_addr and size set.
+ *              Each buffer's buf_info->cb, if set, is invoked once the batch
+ *              completes successfully, mirroring read_sync/read_async.
  * @mhi_state: MHI Endpoint state
  * @max_chan: Maximum channels supported by the endpoint controller
  * @mru: MRU (Maximum Receive Unit) value of the endpoint controller
@@ -164,6 +169,8 @@ struct mhi_ep_cntrl {
 	int (*write_sync)(struct mhi_ep_cntrl *mhi_cntrl, struct mhi_ep_buf_info *buf_info);
 	int (*read_async)(struct mhi_ep_cntrl *mhi_cntrl, struct mhi_ep_buf_info *buf_info);
 	int (*write_async)(struct mhi_ep_cntrl *mhi_cntrl, struct mhi_ep_buf_info *buf_info);
+	int (*read_batch)(struct mhi_ep_cntrl *mhi_cntrl, struct mhi_ep_buf_info *buf_info_array,
+			  u32 num_buffers);
 
 	enum mhi_state mhi_state;
 

-- 
2.34.1

Re: [PATCH v2 4/5] PCI: epf-mhi: Add batched DMA read support

[Frank Li]

On Mon, Aug 03, 2026 at 04:01:46PM +0530, Sumit Kumar wrote:
> [You don't often get email from sumit.kumar@oss.qualcomm.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
>
> Add support for batched DMA transfers in the PCI EPF MHI driver to
> improve performance when reading multiple buffers from the host.
>
> Implement two variants of the read_batch() callback:
> pci_epf_mhi_edma_read_batch() is a DMA-optimized implementation that uses
> dmaengine_prep_dma_sg() to transfer multiple buffers in a single DMA
> transaction, while pci_epf_mhi_iatu_read_batch() serves as a CPU-copy
> fallback for platforms without DMA support by sequentially processing each
> buffer via IATU mapping. Wire up read_batch() to the eDMA variant only
> when the RX DMA channel advertises the DMA_SG capability, falling back to
> the IATU variant otherwise.
>
> On a successful batch, notify completion for every buffer via its
> caller-supplied buf_info->cb, mirroring the read_sync/read_async
> completion semantics so the MHI stack can free each buffer and raise its
> transfer completion event. Make read_batch() a mandatory callback in
> mhi_ep_register_controller(), alongside read_sync/write_sync/read_async/
> write_async.
>
> This enables the MHI endpoint stack to cache ring data efficiently,
> particularly for wraparound scenarios where ring data spans two
> non-contiguous memory regions.
>
> Signed-off-by: Sumit Kumar <sumit.kumar@oss.qualcomm.com>
> ---
>  drivers/bus/mhi/ep/main.c                    |   3 +-
>  drivers/pci/endpoint/functions/pci-epf-mhi.c | 159 +++++++++++++++++++++++++++
>  include/linux/mhi_ep.h                       |   7 ++
>  3 files changed, 168 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/bus/mhi/ep/main.c b/drivers/bus/mhi/ep/main.c
> index 21bc2c50170ff7f0181c0d0d4eefd51db30f34c8..c45240fc481dc3260a61b546c223b6be8e105587 100644
> --- a/drivers/bus/mhi/ep/main.c
> +++ b/drivers/bus/mhi/ep/main.c
> @@ -1459,7 +1459,8 @@ int mhi_ep_register_controller(struct mhi_ep_cntrl *mhi_cntrl,
>                 return -EINVAL;
>
>         if (!mhi_cntrl->read_sync || !mhi_cntrl->write_sync ||
> -           !mhi_cntrl->read_async || !mhi_cntrl->write_async)
> +           !mhi_cntrl->read_async || !mhi_cntrl->write_async ||
> +           !mhi_cntrl->read_batch)
>                 return -EINVAL;
>
>         ret = mhi_ep_chan_init(mhi_cntrl, config);
> diff --git a/drivers/pci/endpoint/functions/pci-epf-mhi.c b/drivers/pci/endpoint/functions/pci-epf-mhi.c
> index 6bac69fc84b472c5f1abbeede2b441b5db73c784..ad245148de1462cffc950ba9a9e232405514dfb7 100644
> --- a/drivers/pci/endpoint/functions/pci-epf-mhi.c
> +++ b/drivers/pci/endpoint/functions/pci-epf-mhi.c
> @@ -444,6 +444,162 @@ static int pci_epf_mhi_edma_write(struct mhi_ep_cntrl *mhi_cntrl,
>         return ret;
>  }
>
> +static int pci_epf_mhi_iatu_read_batch(struct mhi_ep_cntrl *mhi_cntrl,
> +                                      struct mhi_ep_buf_info *buf_info_array,
> +                                      u32 num_buffers)
> +{
> +       struct pci_epf_mhi *epf_mhi = to_epf_mhi(mhi_cntrl);
> +       struct device *dev = &epf_mhi->epf->dev;
> +       u32 i;
> +       int ret;
> +
> +       if (num_buffers == 0)
> +               return -EINVAL;
> +
> +       for (i = 0; i < num_buffers; i++) {
> +               ret = pci_epf_mhi_iatu_read(mhi_cntrl, &buf_info_array[i]);
> +               if (ret < 0) {
> +                       dev_err(dev, "Failed to read buffer %u of %u in batch: %d\n",
> +                               i, num_buffers, ret);
> +                       return ret;
> +               }
> +       }
> +
> +       return 0;
> +}
> +
> +static int pci_epf_mhi_edma_read_batch(struct mhi_ep_cntrl *mhi_cntrl,
> +                                      struct mhi_ep_buf_info *buf_info_array,
> +                                      u32 num_buffers)
> +{
> +       struct pci_epf_mhi *epf_mhi = to_epf_mhi(mhi_cntrl);
> +       struct device *dma_dev = epf_mhi->epf->epc->dev.parent;
> +       struct device *dev = &epf_mhi->epf->dev;
> +       struct dma_async_tx_descriptor *desc;
> +       struct dma_slave_config config = {};
> +       DECLARE_COMPLETION_ONSTACK(complete);
> +       struct scatterlist *src_sg;
> +       struct scatterlist *dst_sg;
> +       unsigned long time_left;
> +       struct dma_chan *chan;
> +       dma_cookie_t cookie;
> +       unsigned int i;
> +       int mapped;
> +       void *buf;
> +       int ret;
> +
> +       if (num_buffers == 0)
> +               return -EINVAL;
> +
> +       /*
> +        * Single allocation carved into two arrays: src_sg[], dst_sg[].
> +        * Reduces allocator round-trips on the ring-cache hot path. Done
> +        * before taking the lock so direct reclaim cannot stall other
> +        * transfers waiting on epf_mhi->lock.
> +        */
> +       buf = kcalloc(num_buffers, 2 * sizeof(*src_sg), GFP_KERNEL);
> +       if (!buf)
> +               return -ENOMEM;
> +       src_sg = buf;
> +       dst_sg = src_sg + num_buffers;
> +
> +       mutex_lock(&epf_mhi->lock);
> +
> +       chan = epf_mhi->dma_chan_rx;
> +       if (!chan) {
> +               ret = -ENODEV;
> +               goto err_unlock;
> +       }
> +
> +       sg_init_table(src_sg, num_buffers);
> +       sg_init_table(dst_sg, num_buffers);
> +
> +       for (i = 0; i < num_buffers; i++) {
> +               /*
> +                * src addresses are PCIe host bus addresses already visible to
> +                * the eDMA engine; no dma_map_sg() is needed for the source list.
> +                */
> +               sg_dma_address(&src_sg[i]) = buf_info_array[i].host_addr;
> +               sg_dma_len(&src_sg[i]) = buf_info_array[i].size;
> +
> +               sg_set_buf(&dst_sg[i], buf_info_array[i].dev_addr, buf_info_array[i].size);
> +       }
> +
> +       mapped = dma_map_sg(dma_dev, dst_sg, num_buffers, DMA_FROM_DEVICE);
> +       if (!mapped) {
> +               dev_err(dev, "Failed to map destination buffers for %u-buffer batch read\n",
> +                       num_buffers);
> +               ret = -EIO;
> +               goto err_unlock;
> +       }
> +
> +       config.direction = DMA_DEV_TO_MEM;
> +       ret = dmaengine_slave_config(chan, &config);
> +       if (ret) {
> +               dev_err(dev, "Failed to configure DMA channel for %u-buffer batch read: %d\n",
> +                       num_buffers, ret);
> +               goto err_unmap;
> +       }
> +
> +       desc = dmaengine_prep_dma_sg(chan, dst_sg, num_buffers,
> +                                    src_sg, num_buffers,
> +                                    DMA_CTRL_ACK | DMA_PREP_INTERRUPT);

Although both side memory address, but as previous discussion, which is not
memcpy, still memory to IO space transfer.

Maybe enhence device_prep_peripheral_dma_vec(). Actually eDMA still split
it to small trunk and linked together.

you can submit more than one require before issue_pending().

Koichiro Den is working on dynmatic append request.

https://lore.kernel.org/dmaengine/20260729143036.3087722-1-den@valinux.co.jp/

Frank

> +       if (!desc) {
> +               dev_err(dev, "Failed to prepare batch DMA\n");
> +               ret = -EIO;
> +               goto err_unmap;
> +       }
> +
> +       desc->callback = pci_epf_mhi_dma_callback;
> +       desc->callback_param = &complete;
> +
> +       cookie = dmaengine_submit(desc);
> +       ret = dma_submit_error(cookie);
> +       if (ret) {
> +               dev_err(dev, "Failed to submit DMA\n");
> +               if (dmaengine_terminate_sync(chan))
> +                       dev_err(dev, "Failed to terminate DMA channel after submit failure\n");
> +               goto err_unmap;
> +       }
> +
> +       dma_async_issue_pending(chan);
> +
> +       time_left = wait_for_completion_timeout(&complete,
> +                                               msecs_to_jiffies(PCI_EPF_MHI_DMA_TIMEOUT_MS));
> +       if (!time_left) {
> +               dev_err(dev, "DMA transfer timeout\n");
> +               if (dmaengine_terminate_sync(chan))
> +                       dev_err(dev, "Failed to terminate DMA channel after timeout\n");
> +               ret = -ETIMEDOUT;
> +               goto err_unmap;
> +       }
> +
> +       ret = 0;
> +
> +err_unmap:
> +       /* dma_unmap_sg() must use the nents passed to dma_map_sg(), not its return value */
> +       dma_unmap_sg(dma_dev, dst_sg, num_buffers, DMA_FROM_DEVICE);
> +err_unlock:
> +       mutex_unlock(&epf_mhi->lock);
> +
> +       kfree(buf);
> +
> +       /*
> +        * On a successful batch, notify completion for every buffer via its
> +        * caller-supplied callback so MHI can free the backing buffer and raise
> +        * the transfer completion event per entry, matching the
> +        * read_sync/read_async semantics. Done after dropping the lock as a
> +        * callback may re-enter the driver and epf_mhi->lock is not reentrant.
> +        */
> +       if (!ret) {
> +               for (i = 0; i < num_buffers; i++)
> +                       if (buf_info_array[i].cb)
> +                               buf_info_array[i].cb(&buf_info_array[i]);
> +       }
> +
> +       return ret;
> +}
> +
>  static void pci_epf_mhi_dma_worker(struct work_struct *work)
>  {
>         struct pci_epf_mhi *epf_mhi = container_of(work, struct pci_epf_mhi, dma_work);
> @@ -789,11 +945,14 @@ static int pci_epf_mhi_link_up(struct pci_epf *epf)
>         mhi_cntrl->unmap_free = pci_epf_mhi_unmap_free;
>         mhi_cntrl->read_sync = mhi_cntrl->read_async = pci_epf_mhi_iatu_read;
>         mhi_cntrl->write_sync = mhi_cntrl->write_async = pci_epf_mhi_iatu_write;
> +       mhi_cntrl->read_batch = pci_epf_mhi_iatu_read_batch;
>         if (info->flags & MHI_EPF_USE_DMA) {
>                 mhi_cntrl->read_sync = pci_epf_mhi_edma_read;
>                 mhi_cntrl->write_sync = pci_epf_mhi_edma_write;
>                 mhi_cntrl->read_async = pci_epf_mhi_edma_read_async;
>                 mhi_cntrl->write_async = pci_epf_mhi_edma_write_async;
> +               if (dma_has_cap(DMA_SG, epf_mhi->dma_chan_rx->device->cap_mask))
> +                       mhi_cntrl->read_batch = pci_epf_mhi_edma_read_batch;
>         }
>
>         /* Register the MHI EP controller */
> diff --git a/include/linux/mhi_ep.h b/include/linux/mhi_ep.h
> index 7b40fc8cbe77ab8419d167e89264b69a817b9fb1..51f66cae937a53a93e9ae76cdcfaac7b1e08283f 100644
> --- a/include/linux/mhi_ep.h
> +++ b/include/linux/mhi_ep.h
> @@ -107,6 +107,11 @@ struct mhi_ep_buf_info {
>   * @write_sync: CB function for writing to host memory synchronously
>   * @read_async: CB function for reading from host memory asynchronously
>   * @write_async: CB function for writing to host memory asynchronously
> + * @read_batch: CB function for reading from host memory in batches synchronously.
> + *              Callers must pass at least one buffer (num_buffers > 0); each
> + *              buf_info entry must have host_addr, dev_addr and size set.
> + *              Each buffer's buf_info->cb, if set, is invoked once the batch
> + *              completes successfully, mirroring read_sync/read_async.
>   * @mhi_state: MHI Endpoint state
>   * @max_chan: Maximum channels supported by the endpoint controller
>   * @mru: MRU (Maximum Receive Unit) value of the endpoint controller
> @@ -164,6 +169,8 @@ struct mhi_ep_cntrl {
>         int (*write_sync)(struct mhi_ep_cntrl *mhi_cntrl, struct mhi_ep_buf_info *buf_info);
>         int (*read_async)(struct mhi_ep_cntrl *mhi_cntrl, struct mhi_ep_buf_info *buf_info);
>         int (*write_async)(struct mhi_ep_cntrl *mhi_cntrl, struct mhi_ep_buf_info *buf_info);
> +       int (*read_batch)(struct mhi_ep_cntrl *mhi_cntrl, struct mhi_ep_buf_info *buf_info_array,
> +                         u32 num_buffers);
>
>         enum mhi_state mhi_state;
>
>
> --
> 2.34.1
>

[PATCH v2 5/5] bus: mhi: ep: Use batched read for ring caching

[Sumit Kumar]

Optimise ring caching in __mhi_ep_cache_ring().
For the non-wraparound case, use read_sync(); for the wraparound case
where ring data spans two non-contiguous host memory regions, both the
tail portion (start -> ring_size) and head portion (0 -> end) are
submitted using the new read_batch() API in a single operation.
On DMA-capable platforms this transfers both segments in one DMA
transaction.

Signed-off-by: Sumit Kumar <sumit.kumar@oss.qualcomm.com>
---
 drivers/bus/mhi/ep/ring.c | 36 ++++++++++++++++++++----------------
 1 file changed, 20 insertions(+), 16 deletions(-)

diff --git a/drivers/bus/mhi/ep/ring.c b/drivers/bus/mhi/ep/ring.c
index 405ce16c02a89ea2268ea1033ff5c1c26b1c81bd..a369c1eb0c0b849d7ced01b99898ca461cf680b9 100644
--- a/drivers/bus/mhi/ep/ring.c
+++ b/drivers/bus/mhi/ep/ring.c
@@ -30,7 +30,6 @@ static int __mhi_ep_cache_ring(struct mhi_ep_ring *ring, size_t end)
 {
 	struct mhi_ep_cntrl *mhi_cntrl = ring->mhi_cntrl;
 	struct device *dev = &mhi_cntrl->mhi_dev->dev;
-	struct mhi_ep_buf_info buf_info = {};
 	size_t start;
 	int ret;
 
@@ -44,6 +43,8 @@ static int __mhi_ep_cache_ring(struct mhi_ep_ring *ring, size_t end)
 
 	start = ring->wr_offset;
 	if (start < end) {
+		struct mhi_ep_buf_info buf_info = {};
+
 		buf_info.size = (end - start) * sizeof(struct mhi_ring_element);
 		buf_info.host_addr = ring->rbase + (start * sizeof(struct mhi_ring_element));
 		buf_info.dev_addr = &ring->ring_cache[start];
@@ -51,27 +52,30 @@ static int __mhi_ep_cache_ring(struct mhi_ep_ring *ring, size_t end)
 		ret = mhi_cntrl->read_sync(mhi_cntrl, &buf_info);
 		if (ret)
 			return ret;
+
+		dev_dbg(dev, "Cached ring: start %zu end %zu size %zu\n", start, end,
+			buf_info.size);
 	} else {
-		buf_info.size = (ring->ring_size - start) * sizeof(struct mhi_ring_element);
-		buf_info.host_addr = ring->rbase + (start * sizeof(struct mhi_ring_element));
-		buf_info.dev_addr = &ring->ring_cache[start];
+		struct mhi_ep_buf_info buf_info[2] = {};
+		u32 count = 1;
 
-		ret = mhi_cntrl->read_sync(mhi_cntrl, &buf_info);
-		if (ret)
-			return ret;
+		buf_info[0].size = (ring->ring_size - start) * sizeof(struct mhi_ring_element);
+		buf_info[0].host_addr = ring->rbase + (start * sizeof(struct mhi_ring_element));
+		buf_info[0].dev_addr = &ring->ring_cache[start];
 
 		if (end) {
-			buf_info.host_addr = ring->rbase;
-			buf_info.dev_addr = &ring->ring_cache[0];
-			buf_info.size = end * sizeof(struct mhi_ring_element);
-
-			ret = mhi_cntrl->read_sync(mhi_cntrl, &buf_info);
-			if (ret)
-				return ret;
+			buf_info[1].size = end * sizeof(struct mhi_ring_element);
+			buf_info[1].host_addr = ring->rbase;
+			buf_info[1].dev_addr = &ring->ring_cache[0];
+			count = 2;
 		}
-	}
+		ret = mhi_cntrl->read_batch(mhi_cntrl, buf_info, count);
+		if (ret)
+			return ret;
 
-	dev_dbg(dev, "Cached ring: start %zu end %zu size %zu\n", start, end, buf_info.size);
+		dev_dbg(dev, "Cached ring (batched): start %zu end %zu tail_size %zu head_size %zu count %u\n",
+			start, end, buf_info[0].size, end ? buf_info[1].size : 0, count);
+	}
 
 	return 0;
 }

-- 
2.34.1