Low network throughput on i.MX28

[joerg.krause@embedded.rocks (Jörg Krause)]

On Sat, 2016-11-05 at 15:45 +0000, Koul, Vinod wrote:
> On Sat, 2016-11-05 at 14:14 +0100, J?rg Krause wrote:
> > On Sat, 2016-11-05 at 12:39 +0000, Koul, Vinod wrote:
> > > 
> > > On Sat, 2016-11-05 at 13:06 +0100, J?rg Krause wrote:
> > > > 
> > > > @ Vinod
> > > > In short, I noticed poor performance in the SSP2 (MMC/SD/SDIO)
> > > > interface on a custom i.MX28 board with a wifi chip attached.
> > > > Comparing
> > > > the bandwith with iperf I get >20Mbits/sec on the vendor kernel
> > > > and
> > > > <5Mbits/sec on the mainline kernel. I am trying to investigate
> > > > what
> > > > the
> > > > bottleneck is.
> > > 
> > > is this imx-dma or imx-sdma..
> > > 
> > > > 
> > > > 
> > > > @ Stefan, all
> > > > My understanding is that the tasklet in this case is
> > > > responsible
> > > > for
> > > > reading the response registers of the DMA controller and return
> > > > the
> > > > response to the MMC host driver.
> > > > 
> > > > The vendor kernel does this in the interrupt routine of mxs-mmc 
> > > > by
> > > > issueing a complete whereas the mainline kernel does this in
> > > > the
> > > > interrupt routine in mxs-dma by scheduling the tasklet.
> > > 
> > > Is vendor kernel using dmaengine APIs or not?
> > 
> > It's this engine [1].
> > 
> > [1] http://git.freescale.com/git/cgit.cgi/imx/linux-2.6-imx.git/tre
> > e/a
> > r
> > ch/arm/plat-mxs/dmaengine.c?h=imx_2.6.35_1.1.0
> 
> Thanks for info, this looks okay.
> 
> First can you confirm that register configuration for DMA transaction
> is
> same in both cases.

They are almost identical. The difference is that the mainline MMC
driver has SDIO IRQ enabled and the APB bus has burst mode enable. Both
don't have any influence.

> Second, looking at the driver I see that interrupt handler is not
> pushing next descriptor. Also the tasklet is doing callback action
> and
> not pushing any descriptors, did I miss anything in this?

Right. However, after observing the registers I noticed that the vendor
MMC kernel driver only issues one DMA command, whereas the mainline
driver issues two chained DMA commands. The relevant function in both
drivers is mxs_mmc_adtc().

The mainline function issues a DMA transaction with setting the PIO
words only and appends the data from the MMC host.

The vendor function copies the MMC host data from the scatterlist into
an owned DMA buffer, sets the buffer address as the next command
address and issues the descriptor to the DMA engine.

> For good dma throughput, you should have multiple dma transactions
> queued up and submitted as fast as possible. Can you check if this is
> being done.?
> 
> We need to minimize/eliminate the delay between two transactions.
> This
> can be done in SW or HW based on support from HW. If HW supports
> chaining of descriptors then next transaction which is given to
> dmaengine driver should be appended at the end. If not submit the
> descriptor to hw immediately on interrupt.?

I see! In this particular case, the vendor driver reduces the chaining
of descriptors, whereas the mainline driver chains two DMA commands.
Note, that the i.MX28 hardware does support chaining. So, might this be
an issue for poor performance?

J?rg