From: Frank Li <Frank.li@oss.nxp.com>
To: Koichiro Den <den@valinux.co.jp>
Cc: "Manivannan Sadhasivam" <mani@kernel.org>,
"Vinod Koul" <vkoul@kernel.org>,
"Gustavo Pimentel" <Gustavo.Pimentel@synopsys.com>,
"Kees Cook" <kees@kernel.org>,
"Gustavo A. R. Silva" <gustavoars@kernel.org>,
"Krzysztof Wilczyński" <kwilczynski@kernel.org>,
"Kishon Vijay Abraham I" <kishon@kernel.org>,
"Bjorn Helgaas" <bhelgaas@google.com>,
"Christoph Hellwig" <hch@lst.de>,
"Niklas Cassel" <cassel@kernel.org>,
dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-hardening@vger.kernel.org, linux-pci@vger.kernel.org,
linux-nvme@lists.infradead.org, imx@lists.linux.dev, "Verma,
Devendra" <devverma@amd.com>, "Frank Li" <Frank.Li@nxp.com>
Subject: Re: [PATCH v3 00/10] dmaengine: dw-edma: flatten desc structions and simple code
Date: Wed, 8 Jul 2026 15:55:21 -0500 [thread overview]
Message-ID: <ak65OTg6t7mPnizf@SMW015318> (raw)
In-Reply-To: <2z2ba5kwgtyjzipkhxqf2jxjscerbx35ep7jndedv5zk6l6xwk@esy2ayjyhcoo>
On Wed, Jul 08, 2026 at 11:59:21PM +0900, Koichiro Den wrote:
> On Mon, Jul 06, 2026 at 10:20:38PM +0900, Koichiro Den wrote:
> > On Thu, Jul 02, 2026 at 05:21:20PM -0400, Frank.Li@oss.nxp.com wrote:
> > > Koichiro Den:
> > > My hardware temperately is unavaible recently. Can you help test
> > > it.
> >
> > Sure, I can test it on my side. I'll report back once I have the results.
>
> Here are the results. For the series:
>
> Tested-by: Koichiro Den <den@valinux.co.jp>
Thank you for testing it. I just post v4, but still have issues reported
by sashiko.
Frank
>
> * I don't see a significant difference between Before and After, but I don't
> think that is an issue at all. Most of the differences look like normal
> run-to-run variation.
> * Each full fio test set was run three times in alternating order (B-A-B-A-B-A),
> with runtime=30s and ramp_time=5s.
>
> eDMA:
> - Testbed:
> * Endpoint: RK3588 (Rock 5B)
> controller IP version: v5.60a
> ll_max: 170
>
> - Summary by group (BW delta %)
> all n=26 mean= -2.3 median= +0.2 min= -28.0 max= +11.7
> read n=14 mean= -3.2 median= +0.2 min= -17.3 max= +3.1
> write n=11 mean= -1.5 median= -0.2 min= -28.0 max= +11.7
> qd32 n=16 mean= +0.5 median= +0.3 min= -3.8 max= +3.1
> q1 n= 9 mean= -7.6 median= -6.7 min= -28.0 max= +11.7
> small 4K n= 6 mean= -4.0 median= +1.1 min= -28.0 max= +3.1
> large >=128K n=20 mean= -1.8 median= +0.1 min= -17.3 max= +11.7
>
> - Before mean -> After mean (MiB/s)
>
> Case Before After Delta
> --------------------------- ----------------- ----------------- ------
> Rnd read 4KB q1 1j 33.4 (sd 10.1) 32.0 (sd 10.9) -4.0%
> Rnd read 4KB q32 1j 196.0 (sd 28.6) 202.0 (sd 29.5) +3.1%
> Rnd read 4KB q32 4j 196.7 (sd 29.2) 202.0 (sd 25.1) +2.7%
> Rnd read 128KB q1 1j 497.7 (sd 12.2) 420.7 (sd 181.3) -15.5%
> Rnd read 128KB q32 1j 2248.0 (sd 6.6) 2277.3 (sd 34.2) +1.3%
> Rnd read 128KB q32 4j 2381.3 (sd 2.5) 2386.3 (sd 17.9) +0.2%
> Rnd read 512KB q1 1j 627.3 (sd 15.2) 585.3 (sd 78.2) -6.7%
> Rnd read 512KB q32 1j 2376.0 (sd 5.2) 2381.3 (sd 21.4) +0.2%
> Rnd read 512KB q32 4j 2379.7 (sd 6.7) 2386.7 (sd 17.6) +0.3%
> Rnd write 4KB q1 1j 28.1 (sd 4.1) 20.2 (sd 10.1) -28.0%
> Rnd write 4KB q32 1j 120.3 (sd 6.1) 122.0 (sd 6.2) +1.4%
> Rnd write 4KB q32 4j 124.7 (sd 3.8) 125.7 (sd 4.6) +0.8%
> Rnd write 128KB q1 1j 318.7 (sd 44.7) 327.0 (sd 4.4) +2.6%
> Rnd write 128KB q32 1j 1080.0 (sd 20.2) 1077.3 (sd 37.6) -0.2%
> Rnd write 128KB q32 4j 1069.7 (sd 20.3) 1056.0 (sd 46.2) -1.3%
> Seq read 128KB q1 1j 486.3 (sd 138.3) 402.3 (sd 38.0) -17.3%
> Seq read 128KB q32 1j 2245.3 (sd 3.5) 2258.7 (sd 26.3) +0.6%
> Seq read 512KB q1 1j 662.0 (sd 29.2) 594.3 (sd 13.7) -10.2%
> Seq read 512KB q32 1j 2375.7 (sd 7.4) 2382.0 (sd 22.9) +0.3%
> Seq read 1MB q32 1j 2380.7 (sd 4.7) 2385.3 (sd 19.3) +0.2%
> Seq write 128KB q1 1j 342.0 (sd 58.9) 382.0 (sd 101.1) +11.7%
> Seq write 128KB q32 1j 1080.3 (sd 48.8) 1070.7 (sd 37.0) -0.9%
> Seq write 512KB q1 1j 509.7 (sd 35.4) 502.7 (sd 39.2) -1.4%
> Seq write 512KB q32 1j 1043.3 (sd 56.7) 1074.0 (sd 47.8) +2.9%
> Seq write 1MB q32 1j 989.3 (sd 23.0) 952.0 (sd 57.4) -3.8%
> Rnd rdwr 4K..1MB q8 4j 841.3 (sd 15.1) 841.7 (sd 12.9) +0.0%
>
> HDMA:
> - Testbed:
> * Endpoint: SpacemiT K3
> controller IP version: v6.30a
> ll_max: 170
>
> - Summary by group (BW delta %)
>
> all n=26 mean= +1.1 median= -0.6 min= -4.4 max= +9.5
> read n=14 mean= +2.1 median= +0.7 min= -2.5 max= +9.5
> write n=11 mean= -0.1 median= -0.8 min= -4.4 max= +4.7
> qd32 n=16 mean= +0.6 median= -0.8 min= -2.5 max= +9.5
> q1 n= 9 mean= +2.0 median= +4.4 min= -4.4 max= +7.9
> small 4K n= 6 mean= +4.8 median= +4.5 min= +0.2 max= +9.5
> large >=128K n=20 mean= -0.0 median= -0.9 min= -4.4 max= +7.9
>
> - Before mean -> After mean (MiB/s)
>
> Case Before After Delta
> --------------------------- ----------------- ----------------- ------
> Rnd read 4KB q1 1j 66.3 (sd 5.8) 69.4 (sd 7.0) +4.6%
> Rnd read 4KB q32 1j 300.3 (sd 45.5) 329.0 (sd 21.7) +9.5%
> Rnd read 4KB q32 4j 312.0 (sd 51.1) 341.7 (sd 3.1) +9.5%
> Rnd read 128KB q1 1j 705.7 (sd 34.8) 736.7 (sd 51.6) +4.4%
> Rnd read 128KB q32 1j 1507.7 (sd 25.6) 1486.3 (sd 5.9) -1.4%
> Rnd read 128KB q32 4j 1549.7 (sd 7.0) 1534.3 (sd 16.9) -1.0%
> Rnd read 512KB q1 1j 848.7 (sd 9.5) 858.0 (sd 15.5) +1.1%
> Rnd read 512KB q32 1j 1530.0 (sd 27.0) 1536.0 (sd 14.8) +0.4%
> Rnd read 512KB q32 4j 1519.0 (sd 66.7) 1544.3 (sd 15.0) +1.7%
> Rnd write 4KB q1 1j 64.0 (sd 6.2) 66.9 (sd 2.2) +4.5%
> Rnd write 4KB q32 1j 199.3 (sd 7.1) 199.7 (sd 2.9) +0.2%
> Rnd write 4KB q32 4j 199.7 (sd 7.6) 200.3 (sd 3.2) +0.3%
> Rnd write 128KB q1 1j 558.3 (sd 18.3) 533.7 (sd 5.0) -4.4%
> Rnd write 128KB q32 1j 1248.0 (sd 21.3) 1237.3 (sd 7.5) -0.9%
> Rnd write 128KB q32 4j 1248.7 (sd 23.0) 1238.0 (sd 6.1) -0.9%
> Seq read 128KB q1 1j 640.7 (sd 60.1) 691.3 (sd 9.0) +7.9%
> Seq read 128KB q32 1j 1507.7 (sd 24.2) 1488.3 (sd 5.1) -1.3%
> Seq read 512KB q1 1j 866.7 (sd 45.8) 847.0 (sd 32.2) -2.3%
> Seq read 512KB q32 1j 1532.3 (sd 31.7) 1516.7 (sd 40.3) -1.0%
> Seq read 1MB q32 1j 1550.7 (sd 7.2) 1512.0 (sd 32.9) -2.5%
> Seq write 128KB q1 1j 514.0 (sd 34.7) 538.0 (sd 15.1) +4.7%
> Seq write 128KB q32 1j 1248.0 (sd 22.1) 1237.0 (sd 7.8) -0.9%
> Seq write 512KB q1 1j 755.7 (sd 30.0) 739.3 (sd 2.1) -2.2%
> Seq write 512KB q32 1j 1248.7 (sd 22.6) 1238.3 (sd 6.7) -0.8%
> Seq write 1MB q32 1j 1248.0 (sd 22.5) 1238.3 (sd 7.6) -0.8%
> Rnd rdwr 4K..1MB q8 4j 869.0 (sd 8.9) 865.3 (sd 1.5) -0.4%
>
> Best regards,
> Koichiro
>
> >
> > Best regards,
> > Koichiro
> >
> > >
> > > Rebase and compile test only now.
> > >
> > > Verma, Devendra:
> > > Can you help check if block non-ll mode?
> > >
> > > Frank
> > >
> > > Basic change
> > >
> > > struct dw_edma_desc *desc
> > > └─ chunk list
> > > └─ burst list
> > >
> > > To
> > >
> > > struct dw_edma_desc *desc
> > > └─ burst[n]
> > >
> > > And reduce at least 2 times kzalloc() for each dma descriptor create.
> > >
> > > I only test eDMA part, not hardware test hdma part.
> > >
> > > The finial goal is dymatic add DMA request when DMA running. So needn't
> > > wait for irq for fetch next round DMA request.
> > >
> > > This work is neccesary to for dymatic DMA request appending.
> > >
> > > The post this part first to review and test firstly during working dymatic
> > > DMA part.
> > >
> > > performance is little bit better. Use NVME as EP function
> > >
> > > Before
> > >
> > > Rnd read, 4KB, QD=1, 1 job : IOPS=6660, BW=26.0MiB/s (27.3MB/s)
> > > Rnd read, 4KB, QD=32, 1 job : IOPS=28.6k, BW=112MiB/s (117MB/s)
> > > Rnd read, 4KB, QD=32, 4 jobs: IOPS=33.4k, BW=130MiB/s (137MB/s)
> > > Rnd read, 128KB, QD=1, 1 job : IOPS=914, BW=114MiB/s (120MB/s)
> > > Rnd read, 128KB, QD=32, 1 job : IOPS=1204, BW=151MiB/s (158MB/s)
> > > Rnd read, 128KB, QD=32, 4 jobs: IOPS=1255, BW=157MiB/s (165MB/s)
> > > Rnd read, 512KB, QD=1, 1 job : IOPS=248, BW=124MiB/s (131MB/s)
> > > Rnd read, 512KB, QD=32, 1 job : IOPS=353, BW=177MiB/s (185MB/s)
> > > Rnd read, 512KB, QD=32, 4 jobs: IOPS=388, BW=194MiB/s (204MB/s)
> > > Rnd write, 4KB, QD=1, 1 job : IOPS=6241, BW=24.4MiB/s (25.6MB/s)
> > > Rnd write, 4KB, QD=32, 1 job : IOPS=24.7k, BW=96.5MiB/s (101MB/s)
> > > Rnd write, 4KB, QD=32, 4 jobs: IOPS=26.9k, BW=105MiB/s (110MB/s)
> > > Rnd write, 128KB, QD=1, 1 job : IOPS=780, BW=97.5MiB/s (102MB/s)
> > > Rnd write, 128KB, QD=32, 1 job : IOPS=987, BW=123MiB/s (129MB/s)
> > > Rnd write, 128KB, QD=32, 4 jobs: IOPS=1021, BW=128MiB/s (134MB/s)
> > > Seq read, 128KB, QD=1, 1 job : IOPS=1190, BW=149MiB/s (156MB/s)
> > > Seq read, 128KB, QD=32, 1 job : IOPS=1400, BW=175MiB/s (184MB/s)
> > > Seq read, 512KB, QD=1, 1 job : IOPS=243, BW=122MiB/s (128MB/s)
> > > Seq read, 512KB, QD=32, 1 job : IOPS=355, BW=178MiB/s (186MB/s)
> > > Seq read, 1MB, QD=32, 1 job : IOPS=191, BW=192MiB/s (201MB/s)
> > > Seq write, 128KB, QD=1, 1 job : IOPS=784, BW=98.1MiB/s (103MB/s)
> > > Seq write, 128KB, QD=32, 1 job : IOPS=1030, BW=129MiB/s (135MB/s)
> > > Seq write, 512KB, QD=1, 1 job : IOPS=216, BW=108MiB/s (114MB/s)
> > > Seq write, 512KB, QD=32, 1 job : IOPS=295, BW=148MiB/s (155MB/s)
> > > Seq write, 1MB, QD=32, 1 job : IOPS=164, BW=165MiB/s (173MB/s)
> > > Rnd rdwr, 4K..1MB, QD=8, 4 jobs: IOPS=250, BW=126MiB/s (132MB/s)
> > > IOPS=261, BW=132MiB/s (138MB/s
> > >
> > > After
> > > Rnd read, 4KB, QD=1, 1 job : IOPS=6780, BW=26.5MiB/s (27.8MB/s)
> > > Rnd read, 4KB, QD=32, 1 job : IOPS=28.6k, BW=112MiB/s (117MB/s)
> > > Rnd read, 4KB, QD=32, 4 jobs: IOPS=33.4k, BW=130MiB/s (137MB/s)
> > > Rnd read, 128KB, QD=1, 1 job : IOPS=1188, BW=149MiB/s (156MB/s)
> > > Rnd read, 128KB, QD=32, 1 job : IOPS=1440, BW=180MiB/s (189MB/s)
> > > Rnd read, 128KB, QD=32, 4 jobs: IOPS=1282, BW=160MiB/s (168MB/s)
> > > Rnd read, 512KB, QD=1, 1 job : IOPS=254, BW=127MiB/s (134MB/s)
> > > Rnd read, 512KB, QD=32, 1 job : IOPS=354, BW=177MiB/s (186MB/s)
> > > Rnd read, 512KB, QD=32, 4 jobs: IOPS=388, BW=194MiB/s (204MB/s)
> > > Rnd write, 4KB, QD=1, 1 job : IOPS=6282, BW=24.5MiB/s (25.7MB/s)
> > > Rnd write, 4KB, QD=32, 1 job : IOPS=24.9k, BW=97.5MiB/s (102MB/s)
> > > Rnd write, 4KB, QD=32, 4 jobs: IOPS=27.4k, BW=107MiB/s (112MB/s)
> > > Rnd write, 128KB, QD=1, 1 job : IOPS=1098, BW=137MiB/s (144MB/s)
> > > Rnd write, 128KB, QD=32, 1 job : IOPS=1195, BW=149MiB/s (157MB/s)
> > > Rnd write, 128KB, QD=32, 4 jobs: IOPS=1120, BW=140MiB/s (147MB/s)
> > > Seq read, 128KB, QD=1, 1 job : IOPS=936, BW=117MiB/s (123MB/s)
> > > Seq read, 128KB, QD=32, 1 job : IOPS=1218, BW=152MiB/s (160MB/s)
> > > Seq read, 512KB, QD=1, 1 job : IOPS=301, BW=151MiB/s (158MB/s)
> > > Seq read, 512KB, QD=32, 1 job : IOPS=360, BW=180MiB/s (189MB/s)
> > > Seq read, 1MB, QD=32, 1 job : IOPS=193, BW=194MiB/s (203MB/s)
> > > Seq write, 128KB, QD=1, 1 job : IOPS=796, BW=99.5MiB/s (104MB/s)
> > > Seq write, 128KB, QD=32, 1 job : IOPS=1019, BW=127MiB/s (134MB/s)
> > > Seq write, 512KB, QD=1, 1 job : IOPS=213, BW=107MiB/s (112MB/s)
> > > Seq write, 512KB, QD=32, 1 job : IOPS=273, BW=137MiB/s (143MB/s)
> > > Seq write, 1MB, QD=32, 1 job : IOPS=168, BW=168MiB/s (177MB/s)
> > > Rnd rdwr, 4K..1MB, QD=8, 4 jobs: IOPS=255, BW=128MiB/s (134MB/s)
> > > IOPS=266, BW=135MiB/s (141MB/s)
> > >
> > > Signed-off-by: Frank Li <Frank.Li@nxp.com>
> > > ---
> > > Changes in v3:
> > > - remove patch dmaengine: dw-edma: Remove ll_max = -1 in dw_edma_channel_setup()
> > > - rebase to vnod's dmaengine topic/config_prep_api
> > > - Add non-ll-start() callback to handle non-ll mode transfer
> > > - Link to v2: https://lore.kernel.org/r/20260109-edma_ll-v2-0-5c0b27b2c664@nxp.com
> > >
> > > Changes in v2:
> > > - use 'eDMA' and 'HDMA' at commit message
> > > - remove debug code.
> > > - keep 'inline' to avoid build warning
> > > - Link to v1: https://lore.kernel.org/r/20251212-edma_ll-v1-0-fc863d9f5ca3@nxp.com
> > >
> > > ---
> > > Frank Li (10):
> > > dmaengine: dw-edma: Move control field update of DMA link to the last step
> > > dmaengine: dw-edma: Add xfer_sz field to struct dw_edma_chunk
> > > dmaengine: dw-edma: Move ll_region from struct dw_edma_chunk to struct dw_edma_chan
> > > dmaengine: dw-edma: Pass down dw_edma_chan to reduce one level of indirection
> > > dmaengine: dw-edma: Add helper dw_(edma|hdma)_v0_core_ch_enable()
> > > dmaengine: dw-edma: Add callbacks to fill link list entries
> > > dmaengine: dw-edma: Add non_ll_start() callback
> > > dmaengine: dw-edma: Use common dw_edma_core_start() for both eDMA and HDMA
> > > dmaengine: dw-edma: Use burst array instead of linked list
> > > dmaengine: dw-edma: Remove struct dw_edma_chunk
> > >
> > > drivers/dma/dw-edma/dw-edma-core.c | 216 ++++++++----------------------
> > > drivers/dma/dw-edma/dw-edma-core.h | 65 ++++++---
> > > drivers/dma/dw-edma/dw-edma-v0-core.c | 240 +++++++++++++++++-----------------
> > > drivers/dma/dw-edma/dw-hdma-v0-core.c | 169 ++++++++++++------------
> > > 4 files changed, 302 insertions(+), 388 deletions(-)
> > > ---
> > > base-commit: c9e9927c6d8346cdf6555a8f97da093980172e4b
> > > change-id: 20251211-edma_ll-0904ba089f01
> > >
> > > Best regards,
> > > --
> > > Frank Li <Frank.Li@nxp.com>
> > >
> >
prev parent reply other threads:[~2026-07-08 20:55 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-07-02 21:21 [PATCH v3 00/10] dmaengine: dw-edma: flatten desc structions and simple code Frank.Li
2026-07-02 21:21 ` [PATCH v3 01/10] dmaengine: dw-edma: Move control field update of DMA link to the last step Frank.Li
2026-07-02 21:31 ` sashiko-bot
2026-07-02 21:21 ` [PATCH v3 02/10] dmaengine: dw-edma: Add xfer_sz field to struct dw_edma_chunk Frank.Li
2026-07-02 21:39 ` sashiko-bot
2026-07-02 21:21 ` [PATCH v3 03/10] dmaengine: dw-edma: Move ll_region from struct dw_edma_chunk to struct dw_edma_chan Frank.Li
2026-07-02 21:31 ` sashiko-bot
2026-07-02 21:21 ` [PATCH v3 04/10] dmaengine: dw-edma: Pass down dw_edma_chan to reduce one level of indirection Frank.Li
2026-07-02 21:28 ` sashiko-bot
2026-07-02 21:21 ` [PATCH v3 05/10] dmaengine: dw-edma: Add helper dw_(edma|hdma)_v0_core_ch_enable() Frank.Li
2026-07-02 21:29 ` sashiko-bot
2026-07-02 21:21 ` [PATCH v3 06/10] dmaengine: dw-edma: Add callbacks to fill link list entries Frank.Li
2026-07-02 21:31 ` sashiko-bot
2026-07-02 21:21 ` [PATCH v3 07/10] dmaengine: dw-edma: Add non_ll_start() callback Frank.Li
2026-07-02 21:36 ` sashiko-bot
2026-07-02 21:21 ` [PATCH v3 08/10] dmaengine: dw-edma: Use common dw_edma_core_start() for both eDMA and HDMA Frank.Li
2026-07-02 21:38 ` sashiko-bot
2026-07-02 21:21 ` [PATCH v3 09/10] dmaengine: dw-edma: Use burst array instead of linked list Frank.Li
2026-07-02 21:40 ` sashiko-bot
2026-07-02 21:21 ` [PATCH v3 10/10] dmaengine: dw-edma: Remove struct dw_edma_chunk Frank.Li
2026-07-02 21:38 ` sashiko-bot
2026-07-06 13:20 ` [PATCH v3 00/10] dmaengine: dw-edma: flatten desc structions and simple code Koichiro Den
2026-07-08 14:59 ` Koichiro Den
2026-07-08 20:55 ` Frank Li [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ak65OTg6t7mPnizf@SMW015318 \
--to=frank.li@oss.nxp.com \
--cc=Frank.Li@nxp.com \
--cc=Gustavo.Pimentel@synopsys.com \
--cc=bhelgaas@google.com \
--cc=cassel@kernel.org \
--cc=den@valinux.co.jp \
--cc=devverma@amd.com \
--cc=dmaengine@vger.kernel.org \
--cc=gustavoars@kernel.org \
--cc=hch@lst.de \
--cc=imx@lists.linux.dev \
--cc=kees@kernel.org \
--cc=kishon@kernel.org \
--cc=kwilczynski@kernel.org \
--cc=linux-hardening@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=linux-pci@vger.kernel.org \
--cc=mani@kernel.org \
--cc=vkoul@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.