From: Frank Li <Frank.li@oss.nxp.com>
To: Koichiro Den <den@valinux.co.jp>
Cc: "Manivannan Sadhasivam" <mani@kernel.org>,
"Vinod Koul" <vkoul@kernel.org>,
"Gustavo Pimentel" <Gustavo.Pimentel@synopsys.com>,
"Kees Cook" <kees@kernel.org>,
"Gustavo A. R. Silva" <gustavoars@kernel.org>,
"Krzysztof Wilczyński" <kwilczynski@kernel.org>,
"Kishon Vijay Abraham I" <kishon@kernel.org>,
"Bjorn Helgaas" <bhelgaas@google.com>,
"Christoph Hellwig" <hch@lst.de>,
"Niklas Cassel" <cassel@kernel.org>,
dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-hardening@vger.kernel.org, linux-pci@vger.kernel.org,
linux-nvme@lists.infradead.org, imx@lists.linux.dev, "Verma,
Devendra" <devverma@amd.com>, "Frank Li" <Frank.Li@nxp.com>
Subject: Re: [PATCH v3 00/10] dmaengine: dw-edma: flatten desc structions and simple code
Date: Wed, 8 Jul 2026 15:55:21 -0500 [thread overview]
Message-ID: <ak65OTg6t7mPnizf@SMW015318> (raw)
In-Reply-To: <2z2ba5kwgtyjzipkhxqf2jxjscerbx35ep7jndedv5zk6l6xwk@esy2ayjyhcoo>
On Wed, Jul 08, 2026 at 11:59:21PM +0900, Koichiro Den wrote:
> On Mon, Jul 06, 2026 at 10:20:38PM +0900, Koichiro Den wrote:
> > On Thu, Jul 02, 2026 at 05:21:20PM -0400, Frank.Li@oss.nxp.com wrote:
> > > Koichiro Den:
> > > My hardware temperately is unavaible recently. Can you help test
> > > it.
> >
> > Sure, I can test it on my side. I'll report back once I have the results.
>
> Here are the results. For the series:
>
> Tested-by: Koichiro Den <den@valinux.co.jp>
Thank you for testing it. I just post v4, but still have issues reported
by sashiko.
Frank
>
> * I don't see a significant difference between Before and After, but I don't
> think that is an issue at all. Most of the differences look like normal
> run-to-run variation.
> * Each full fio test set was run three times in alternating order (B-A-B-A-B-A),
> with runtime=30s and ramp_time=5s.
>
> eDMA:
> - Testbed:
> * Endpoint: RK3588 (Rock 5B)
> controller IP version: v5.60a
> ll_max: 170
>
> - Summary by group (BW delta %)
> all n=26 mean= -2.3 median= +0.2 min= -28.0 max= +11.7
> read n=14 mean= -3.2 median= +0.2 min= -17.3 max= +3.1
> write n=11 mean= -1.5 median= -0.2 min= -28.0 max= +11.7
> qd32 n=16 mean= +0.5 median= +0.3 min= -3.8 max= +3.1
> q1 n= 9 mean= -7.6 median= -6.7 min= -28.0 max= +11.7
> small 4K n= 6 mean= -4.0 median= +1.1 min= -28.0 max= +3.1
> large >=128K n=20 mean= -1.8 median= +0.1 min= -17.3 max= +11.7
>
> - Before mean -> After mean (MiB/s)
>
> Case Before After Delta
> --------------------------- ----------------- ----------------- ------
> Rnd read 4KB q1 1j 33.4 (sd 10.1) 32.0 (sd 10.9) -4.0%
> Rnd read 4KB q32 1j 196.0 (sd 28.6) 202.0 (sd 29.5) +3.1%
> Rnd read 4KB q32 4j 196.7 (sd 29.2) 202.0 (sd 25.1) +2.7%
> Rnd read 128KB q1 1j 497.7 (sd 12.2) 420.7 (sd 181.3) -15.5%
> Rnd read 128KB q32 1j 2248.0 (sd 6.6) 2277.3 (sd 34.2) +1.3%
> Rnd read 128KB q32 4j 2381.3 (sd 2.5) 2386.3 (sd 17.9) +0.2%
> Rnd read 512KB q1 1j 627.3 (sd 15.2) 585.3 (sd 78.2) -6.7%
> Rnd read 512KB q32 1j 2376.0 (sd 5.2) 2381.3 (sd 21.4) +0.2%
> Rnd read 512KB q32 4j 2379.7 (sd 6.7) 2386.7 (sd 17.6) +0.3%
> Rnd write 4KB q1 1j 28.1 (sd 4.1) 20.2 (sd 10.1) -28.0%
> Rnd write 4KB q32 1j 120.3 (sd 6.1) 122.0 (sd 6.2) +1.4%
> Rnd write 4KB q32 4j 124.7 (sd 3.8) 125.7 (sd 4.6) +0.8%
> Rnd write 128KB q1 1j 318.7 (sd 44.7) 327.0 (sd 4.4) +2.6%
> Rnd write 128KB q32 1j 1080.0 (sd 20.2) 1077.3 (sd 37.6) -0.2%
> Rnd write 128KB q32 4j 1069.7 (sd 20.3) 1056.0 (sd 46.2) -1.3%
> Seq read 128KB q1 1j 486.3 (sd 138.3) 402.3 (sd 38.0) -17.3%
> Seq read 128KB q32 1j 2245.3 (sd 3.5) 2258.7 (sd 26.3) +0.6%
> Seq read 512KB q1 1j 662.0 (sd 29.2) 594.3 (sd 13.7) -10.2%
> Seq read 512KB q32 1j 2375.7 (sd 7.4) 2382.0 (sd 22.9) +0.3%
> Seq read 1MB q32 1j 2380.7 (sd 4.7) 2385.3 (sd 19.3) +0.2%
> Seq write 128KB q1 1j 342.0 (sd 58.9) 382.0 (sd 101.1) +11.7%
> Seq write 128KB q32 1j 1080.3 (sd 48.8) 1070.7 (sd 37.0) -0.9%
> Seq write 512KB q1 1j 509.7 (sd 35.4) 502.7 (sd 39.2) -1.4%
> Seq write 512KB q32 1j 1043.3 (sd 56.7) 1074.0 (sd 47.8) +2.9%
> Seq write 1MB q32 1j 989.3 (sd 23.0) 952.0 (sd 57.4) -3.8%
> Rnd rdwr 4K..1MB q8 4j 841.3 (sd 15.1) 841.7 (sd 12.9) +0.0%
>
> HDMA:
> - Testbed:
> * Endpoint: SpacemiT K3
> controller IP version: v6.30a
> ll_max: 170
>
> - Summary by group (BW delta %)
>
> all n=26 mean= +1.1 median= -0.6 min= -4.4 max= +9.5
> read n=14 mean= +2.1 median= +0.7 min= -2.5 max= +9.5
> write n=11 mean= -0.1 median= -0.8 min= -4.4 max= +4.7
> qd32 n=16 mean= +0.6 median= -0.8 min= -2.5 max= +9.5
> q1 n= 9 mean= +2.0 median= +4.4 min= -4.4 max= +7.9
> small 4K n= 6 mean= +4.8 median= +4.5 min= +0.2 max= +9.5
> large >=128K n=20 mean= -0.0 median= -0.9 min= -4.4 max= +7.9
>
> - Before mean -> After mean (MiB/s)
>
> Case Before After Delta
> --------------------------- ----------------- ----------------- ------
> Rnd read 4KB q1 1j 66.3 (sd 5.8) 69.4 (sd 7.0) +4.6%
> Rnd read 4KB q32 1j 300.3 (sd 45.5) 329.0 (sd 21.7) +9.5%
> Rnd read 4KB q32 4j 312.0 (sd 51.1) 341.7 (sd 3.1) +9.5%
> Rnd read 128KB q1 1j 705.7 (sd 34.8) 736.7 (sd 51.6) +4.4%
> Rnd read 128KB q32 1j 1507.7 (sd 25.6) 1486.3 (sd 5.9) -1.4%
> Rnd read 128KB q32 4j 1549.7 (sd 7.0) 1534.3 (sd 16.9) -1.0%
> Rnd read 512KB q1 1j 848.7 (sd 9.5) 858.0 (sd 15.5) +1.1%
> Rnd read 512KB q32 1j 1530.0 (sd 27.0) 1536.0 (sd 14.8) +0.4%
> Rnd read 512KB q32 4j 1519.0 (sd 66.7) 1544.3 (sd 15.0) +1.7%
> Rnd write 4KB q1 1j 64.0 (sd 6.2) 66.9 (sd 2.2) +4.5%
> Rnd write 4KB q32 1j 199.3 (sd 7.1) 199.7 (sd 2.9) +0.2%
> Rnd write 4KB q32 4j 199.7 (sd 7.6) 200.3 (sd 3.2) +0.3%
> Rnd write 128KB q1 1j 558.3 (sd 18.3) 533.7 (sd 5.0) -4.4%
> Rnd write 128KB q32 1j 1248.0 (sd 21.3) 1237.3 (sd 7.5) -0.9%
> Rnd write 128KB q32 4j 1248.7 (sd 23.0) 1238.0 (sd 6.1) -0.9%
> Seq read 128KB q1 1j 640.7 (sd 60.1) 691.3 (sd 9.0) +7.9%
> Seq read 128KB q32 1j 1507.7 (sd 24.2) 1488.3 (sd 5.1) -1.3%
> Seq read 512KB q1 1j 866.7 (sd 45.8) 847.0 (sd 32.2) -2.3%
> Seq read 512KB q32 1j 1532.3 (sd 31.7) 1516.7 (sd 40.3) -1.0%
> Seq read 1MB q32 1j 1550.7 (sd 7.2) 1512.0 (sd 32.9) -2.5%
> Seq write 128KB q1 1j 514.0 (sd 34.7) 538.0 (sd 15.1) +4.7%
> Seq write 128KB q32 1j 1248.0 (sd 22.1) 1237.0 (sd 7.8) -0.9%
> Seq write 512KB q1 1j 755.7 (sd 30.0) 739.3 (sd 2.1) -2.2%
> Seq write 512KB q32 1j 1248.7 (sd 22.6) 1238.3 (sd 6.7) -0.8%
> Seq write 1MB q32 1j 1248.0 (sd 22.5) 1238.3 (sd 7.6) -0.8%
> Rnd rdwr 4K..1MB q8 4j 869.0 (sd 8.9) 865.3 (sd 1.5) -0.4%
>
> Best regards,
> Koichiro
>
> >
> > Best regards,
> > Koichiro
> >
> > >
> > > Rebase and compile test only now.
> > >
> > > Verma, Devendra:
> > > Can you help check if block non-ll mode?
> > >
> > > Frank
> > >
> > > Basic change
> > >
> > > struct dw_edma_desc *desc
> > > └─ chunk list
> > > └─ burst list
> > >
> > > To
> > >
> > > struct dw_edma_desc *desc
> > > └─ burst[n]
> > >
> > > And reduce at least 2 times kzalloc() for each dma descriptor create.
> > >
> > > I only test eDMA part, not hardware test hdma part.
> > >
> > > The finial goal is dymatic add DMA request when DMA running. So needn't
> > > wait for irq for fetch next round DMA request.
> > >
> > > This work is neccesary to for dymatic DMA request appending.
> > >
> > > The post this part first to review and test firstly during working dymatic
> > > DMA part.
> > >
> > > performance is little bit better. Use NVME as EP function
> > >
> > > Before
> > >
> > > Rnd read, 4KB, QD=1, 1 job : IOPS=6660, BW=26.0MiB/s (27.3MB/s)
> > > Rnd read, 4KB, QD=32, 1 job : IOPS=28.6k, BW=112MiB/s (117MB/s)
> > > Rnd read, 4KB, QD=32, 4 jobs: IOPS=33.4k, BW=130MiB/s (137MB/s)
> > > Rnd read, 128KB, QD=1, 1 job : IOPS=914, BW=114MiB/s (120MB/s)
> > > Rnd read, 128KB, QD=32, 1 job : IOPS=1204, BW=151MiB/s (158MB/s)
> > > Rnd read, 128KB, QD=32, 4 jobs: IOPS=1255, BW=157MiB/s (165MB/s)
> > > Rnd read, 512KB, QD=1, 1 job : IOPS=248, BW=124MiB/s (131MB/s)
> > > Rnd read, 512KB, QD=32, 1 job : IOPS=353, BW=177MiB/s (185MB/s)
> > > Rnd read, 512KB, QD=32, 4 jobs: IOPS=388, BW=194MiB/s (204MB/s)
> > > Rnd write, 4KB, QD=1, 1 job : IOPS=6241, BW=24.4MiB/s (25.6MB/s)
> > > Rnd write, 4KB, QD=32, 1 job : IOPS=24.7k, BW=96.5MiB/s (101MB/s)
> > > Rnd write, 4KB, QD=32, 4 jobs: IOPS=26.9k, BW=105MiB/s (110MB/s)
> > > Rnd write, 128KB, QD=1, 1 job : IOPS=780, BW=97.5MiB/s (102MB/s)
> > > Rnd write, 128KB, QD=32, 1 job : IOPS=987, BW=123MiB/s (129MB/s)
> > > Rnd write, 128KB, QD=32, 4 jobs: IOPS=1021, BW=128MiB/s (134MB/s)
> > > Seq read, 128KB, QD=1, 1 job : IOPS=1190, BW=149MiB/s (156MB/s)
> > > Seq read, 128KB, QD=32, 1 job : IOPS=1400, BW=175MiB/s (184MB/s)
> > > Seq read, 512KB, QD=1, 1 job : IOPS=243, BW=122MiB/s (128MB/s)
> > > Seq read, 512KB, QD=32, 1 job : IOPS=355, BW=178MiB/s (186MB/s)
> > > Seq read, 1MB, QD=32, 1 job : IOPS=191, BW=192MiB/s (201MB/s)
> > > Seq write, 128KB, QD=1, 1 job : IOPS=784, BW=98.1MiB/s (103MB/s)
> > > Seq write, 128KB, QD=32, 1 job : IOPS=1030, BW=129MiB/s (135MB/s)
> > > Seq write, 512KB, QD=1, 1 job : IOPS=216, BW=108MiB/s (114MB/s)
> > > Seq write, 512KB, QD=32, 1 job : IOPS=295, BW=148MiB/s (155MB/s)
> > > Seq write, 1MB, QD=32, 1 job : IOPS=164, BW=165MiB/s (173MB/s)
> > > Rnd rdwr, 4K..1MB, QD=8, 4 jobs: IOPS=250, BW=126MiB/s (132MB/s)
> > > IOPS=261, BW=132MiB/s (138MB/s
> > >
> > > After
> > > Rnd read, 4KB, QD=1, 1 job : IOPS=6780, BW=26.5MiB/s (27.8MB/s)
> > > Rnd read, 4KB, QD=32, 1 job : IOPS=28.6k, BW=112MiB/s (117MB/s)
> > > Rnd read, 4KB, QD=32, 4 jobs: IOPS=33.4k, BW=130MiB/s (137MB/s)
> > > Rnd read, 128KB, QD=1, 1 job : IOPS=1188, BW=149MiB/s (156MB/s)
> > > Rnd read, 128KB, QD=32, 1 job : IOPS=1440, BW=180MiB/s (189MB/s)
> > > Rnd read, 128KB, QD=32, 4 jobs: IOPS=1282, BW=160MiB/s (168MB/s)
> > > Rnd read, 512KB, QD=1, 1 job : IOPS=254, BW=127MiB/s (134MB/s)
> > > Rnd read, 512KB, QD=32, 1 job : IOPS=354, BW=177MiB/s (186MB/s)
> > > Rnd read, 512KB, QD=32, 4 jobs: IOPS=388, BW=194MiB/s (204MB/s)
> > > Rnd write, 4KB, QD=1, 1 job : IOPS=6282, BW=24.5MiB/s (25.7MB/s)
> > > Rnd write, 4KB, QD=32, 1 job : IOPS=24.9k, BW=97.5MiB/s (102MB/s)
> > > Rnd write, 4KB, QD=32, 4 jobs: IOPS=27.4k, BW=107MiB/s (112MB/s)
> > > Rnd write, 128KB, QD=1, 1 job : IOPS=1098, BW=137MiB/s (144MB/s)
> > > Rnd write, 128KB, QD=32, 1 job : IOPS=1195, BW=149MiB/s (157MB/s)
> > > Rnd write, 128KB, QD=32, 4 jobs: IOPS=1120, BW=140MiB/s (147MB/s)
> > > Seq read, 128KB, QD=1, 1 job : IOPS=936, BW=117MiB/s (123MB/s)
> > > Seq read, 128KB, QD=32, 1 job : IOPS=1218, BW=152MiB/s (160MB/s)
> > > Seq read, 512KB, QD=1, 1 job : IOPS=301, BW=151MiB/s (158MB/s)
> > > Seq read, 512KB, QD=32, 1 job : IOPS=360, BW=180MiB/s (189MB/s)
> > > Seq read, 1MB, QD=32, 1 job : IOPS=193, BW=194MiB/s (203MB/s)
> > > Seq write, 128KB, QD=1, 1 job : IOPS=796, BW=99.5MiB/s (104MB/s)
> > > Seq write, 128KB, QD=32, 1 job : IOPS=1019, BW=127MiB/s (134MB/s)
> > > Seq write, 512KB, QD=1, 1 job : IOPS=213, BW=107MiB/s (112MB/s)
> > > Seq write, 512KB, QD=32, 1 job : IOPS=273, BW=137MiB/s (143MB/s)
> > > Seq write, 1MB, QD=32, 1 job : IOPS=168, BW=168MiB/s (177MB/s)
> > > Rnd rdwr, 4K..1MB, QD=8, 4 jobs: IOPS=255, BW=128MiB/s (134MB/s)
> > > IOPS=266, BW=135MiB/s (141MB/s)
> > >
> > > Signed-off-by: Frank Li <Frank.Li@nxp.com>
> > > ---
> > > Changes in v3:
> > > - remove patch dmaengine: dw-edma: Remove ll_max = -1 in dw_edma_channel_setup()
> > > - rebase to vnod's dmaengine topic/config_prep_api
> > > - Add non-ll-start() callback to handle non-ll mode transfer
> > > - Link to v2: https://lore.kernel.org/r/20260109-edma_ll-v2-0-5c0b27b2c664@nxp.com
> > >
> > > Changes in v2:
> > > - use 'eDMA' and 'HDMA' at commit message
> > > - remove debug code.
> > > - keep 'inline' to avoid build warning
> > > - Link to v1: https://lore.kernel.org/r/20251212-edma_ll-v1-0-fc863d9f5ca3@nxp.com
> > >
> > > ---
> > > Frank Li (10):
> > > dmaengine: dw-edma: Move control field update of DMA link to the last step
> > > dmaengine: dw-edma: Add xfer_sz field to struct dw_edma_chunk
> > > dmaengine: dw-edma: Move ll_region from struct dw_edma_chunk to struct dw_edma_chan
> > > dmaengine: dw-edma: Pass down dw_edma_chan to reduce one level of indirection
> > > dmaengine: dw-edma: Add helper dw_(edma|hdma)_v0_core_ch_enable()
> > > dmaengine: dw-edma: Add callbacks to fill link list entries
> > > dmaengine: dw-edma: Add non_ll_start() callback
> > > dmaengine: dw-edma: Use common dw_edma_core_start() for both eDMA and HDMA
> > > dmaengine: dw-edma: Use burst array instead of linked list
> > > dmaengine: dw-edma: Remove struct dw_edma_chunk
> > >
> > > drivers/dma/dw-edma/dw-edma-core.c | 216 ++++++++----------------------
> > > drivers/dma/dw-edma/dw-edma-core.h | 65 ++++++---
> > > drivers/dma/dw-edma/dw-edma-v0-core.c | 240 +++++++++++++++++-----------------
> > > drivers/dma/dw-edma/dw-hdma-v0-core.c | 169 ++++++++++++------------
> > > 4 files changed, 302 insertions(+), 388 deletions(-)
> > > ---
> > > base-commit: c9e9927c6d8346cdf6555a8f97da093980172e4b
> > > change-id: 20251211-edma_ll-0904ba089f01
> > >
> > > Best regards,
> > > --
> > > Frank Li <Frank.Li@nxp.com>
> > >
> >
prev parent reply other threads:[~2026-07-08 20:55 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-07-02 21:21 [PATCH v3 00/10] dmaengine: dw-edma: flatten desc structions and simple code Frank.Li
2026-07-02 21:21 ` [PATCH v3 01/10] dmaengine: dw-edma: Move control field update of DMA link to the last step Frank.Li
2026-07-02 21:31 ` sashiko-bot
2026-07-02 21:21 ` [PATCH v3 02/10] dmaengine: dw-edma: Add xfer_sz field to struct dw_edma_chunk Frank.Li
2026-07-02 21:39 ` sashiko-bot
2026-07-02 21:21 ` [PATCH v3 03/10] dmaengine: dw-edma: Move ll_region from struct dw_edma_chunk to struct dw_edma_chan Frank.Li
2026-07-02 21:31 ` sashiko-bot
2026-07-02 21:21 ` [PATCH v3 04/10] dmaengine: dw-edma: Pass down dw_edma_chan to reduce one level of indirection Frank.Li
2026-07-02 21:28 ` sashiko-bot
2026-07-02 21:21 ` [PATCH v3 05/10] dmaengine: dw-edma: Add helper dw_(edma|hdma)_v0_core_ch_enable() Frank.Li
2026-07-02 21:29 ` sashiko-bot
2026-07-02 21:21 ` [PATCH v3 06/10] dmaengine: dw-edma: Add callbacks to fill link list entries Frank.Li
2026-07-02 21:31 ` sashiko-bot
2026-07-02 21:21 ` [PATCH v3 07/10] dmaengine: dw-edma: Add non_ll_start() callback Frank.Li
2026-07-02 21:36 ` sashiko-bot
2026-07-02 21:21 ` [PATCH v3 08/10] dmaengine: dw-edma: Use common dw_edma_core_start() for both eDMA and HDMA Frank.Li
2026-07-02 21:38 ` sashiko-bot
2026-07-02 21:21 ` [PATCH v3 09/10] dmaengine: dw-edma: Use burst array instead of linked list Frank.Li
2026-07-02 21:40 ` sashiko-bot
2026-07-02 21:21 ` [PATCH v3 10/10] dmaengine: dw-edma: Remove struct dw_edma_chunk Frank.Li
2026-07-02 21:38 ` sashiko-bot
2026-07-06 13:20 ` [PATCH v3 00/10] dmaengine: dw-edma: flatten desc structions and simple code Koichiro Den
2026-07-08 14:59 ` Koichiro Den
2026-07-08 20:55 ` Frank Li [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ak65OTg6t7mPnizf@SMW015318 \
--to=frank.li@oss.nxp.com \
--cc=Frank.Li@nxp.com \
--cc=Gustavo.Pimentel@synopsys.com \
--cc=bhelgaas@google.com \
--cc=cassel@kernel.org \
--cc=den@valinux.co.jp \
--cc=devverma@amd.com \
--cc=dmaengine@vger.kernel.org \
--cc=gustavoars@kernel.org \
--cc=hch@lst.de \
--cc=imx@lists.linux.dev \
--cc=kees@kernel.org \
--cc=kishon@kernel.org \
--cc=kwilczynski@kernel.org \
--cc=linux-hardening@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=linux-pci@vger.kernel.org \
--cc=mani@kernel.org \
--cc=vkoul@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox