* [v3,2/2] dmaengine: sh: rcar-dmac: Fix glitch in dmaengine_tx_status
From: Dirk Behme @ 2019-04-12 5:29 UTC (permalink / raw)
To: linux-renesas-soc
Cc: dmaengine, vkoul, geert+renesas, niklas.soderlund+renesas,
laurent.pinchart+renesas, Achim.Dahlhoff, dirk.behme,
yoshihiro.shimoda.uh, ylhuajnu, hiroyuki.yokoyama.vx,
kuninori.morimoto.gx, stable
From: Achim Dahlhoff <Achim.Dahlhoff@de.bosch.com>
The tx_status poll in the rcar_dmac driver reads the status register
which indicates which chunk is busy (DMACHCRB). Afterwards the point
inside the chunk is read from DMATCRB. It is possible that the chunk
has changed between the two reads. The result is a non-monotonous
increase of the residue. Fix this by introducing a 'safe read' logic.
Fixes: 73a47bd0da66 ("dmaengine: rcar-dmac: use TCRB instead of TCR for residue")
Signed-off-by: Achim Dahlhoff <Achim.Dahlhoff@de.bosch.com>
Signed-off-by: Dirk Behme <dirk.behme@de.bosch.com>
Cc: <stable@vger.kernel.org> # v4.16+
---
Note: Patch done against mainline v5.0
Changes in v2: Switch goto/retry to for loop
Changes in v3: None
drivers/dma/sh/rcar-dmac.c | 26 +++++++++++++++++++++++---
1 file changed, 23 insertions(+), 3 deletions(-)
diff --git a/drivers/dma/sh/rcar-dmac.c b/drivers/dma/sh/rcar-dmac.c
index 54810ffd95e2..e2a5398f89b5 100644
--- a/drivers/dma/sh/rcar-dmac.c
+++ b/drivers/dma/sh/rcar-dmac.c
@@ -1282,6 +1282,9 @@ static unsigned int rcar_dmac_chan_get_residue(struct rcar_dmac_chan *chan,
enum dma_status status;
unsigned int residue = 0;
unsigned int dptr = 0;
+ unsigned int chcrb;
+ unsigned int tcrb;
+ unsigned int i;
if (!desc)
return 0;
@@ -1329,6 +1332,24 @@ static unsigned int rcar_dmac_chan_get_residue(struct rcar_dmac_chan *chan,
return 0;
}
+ /*
+ * We need to read two registers.
+ * Make sure the control register does not skip to next chunk
+ * while reading the counter.
+ * Trying it 3 times should be enough: Initial read, retry, retry
+ * for the paranoid.
+ */
+ for (i = 0; i < 3; i++) {
+ chcrb = rcar_dmac_chan_read(chan, RCAR_DMACHCRB) &
+ RCAR_DMACHCRB_DPTR_MASK;
+ tcrb = rcar_dmac_chan_read(chan, RCAR_DMATCRB);
+ /* Still the same? */
+ if (chcrb == (rcar_dmac_chan_read(chan, RCAR_DMACHCRB) &
+ RCAR_DMACHCRB_DPTR_MASK))
+ break;
+ }
+ WARN_ONCE(i >= 3, "residue might be not continuous!");
+
/*
* In descriptor mode the descriptor running pointer is not maintained
* by the interrupt handler, find the running descriptor from the
@@ -1336,8 +1357,7 @@ static unsigned int rcar_dmac_chan_get_residue(struct rcar_dmac_chan *chan,
* mode just use the running descriptor pointer.
*/
if (desc->hwdescs.use) {
- dptr = (rcar_dmac_chan_read(chan, RCAR_DMACHCRB) &
- RCAR_DMACHCRB_DPTR_MASK) >> RCAR_DMACHCRB_DPTR_SHIFT;
+ dptr = chcrb >> RCAR_DMACHCRB_DPTR_SHIFT;
if (dptr == 0)
dptr = desc->nchunks;
dptr--;
@@ -1355,7 +1375,7 @@ static unsigned int rcar_dmac_chan_get_residue(struct rcar_dmac_chan *chan,
}
/* Add the residue for the current chunk. */
- residue += rcar_dmac_chan_read(chan, RCAR_DMATCRB) << desc->xfer_shift;
+ residue += tcrb << desc->xfer_shift;
return residue;
}
^ permalink raw reply related
* [v3,1/2] dmaengine: sh: rcar-dmac: With cyclic DMA residue 0 is valid
From: Dirk Behme @ 2019-04-12 5:29 UTC (permalink / raw)
To: linux-renesas-soc
Cc: dmaengine, vkoul, geert+renesas, niklas.soderlund+renesas,
laurent.pinchart+renesas, Achim.Dahlhoff, dirk.behme,
yoshihiro.shimoda.uh, ylhuajnu, hiroyuki.yokoyama.vx,
kuninori.morimoto.gx, stable
Having a cyclic DMA, a residue 0 is not an indication of a completed
DMA. In case of cyclic DMA make sure that dma_set_residue() is called
and with this a residue of 0 is forwarded correctly to the caller.
Fixes: 3544d2878817 ("dmaengine: rcar-dmac: use result of updated get_residue in tx_status")
Signed-off-by: Dirk Behme <dirk.behme@de.bosch.com>
Signed-off-by: Achim Dahlhoff <Achim.Dahlhoff@de.bosch.com>
Signed-off-by: Hiroyuki Yokoyama <hiroyuki.yokoyama.vx@renesas.com>
Signed-off-by: Yao Lihua <ylhuajnu@outlook.com>
Cc: <stable@vger.kernel.org> # v4.8+
---
Note: Patch done against mainline v5.0
Changes in v2: None
Changes in v3: Move reading rchan into the spin lock protection.
drivers/dma/sh/rcar-dmac.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/dma/sh/rcar-dmac.c b/drivers/dma/sh/rcar-dmac.c
index 2b4f25698169..54810ffd95e2 100644
--- a/drivers/dma/sh/rcar-dmac.c
+++ b/drivers/dma/sh/rcar-dmac.c
@@ -1368,6 +1368,7 @@ static enum dma_status rcar_dmac_tx_status(struct dma_chan *chan,
enum dma_status status;
unsigned long flags;
unsigned int residue;
+ bool cyclic;
status = dma_cookie_status(chan, cookie, txstate);
if (status == DMA_COMPLETE || !txstate)
@@ -1375,10 +1376,11 @@ static enum dma_status rcar_dmac_tx_status(struct dma_chan *chan,
spin_lock_irqsave(&rchan->lock, flags);
residue = rcar_dmac_chan_get_residue(rchan, cookie);
+ cyclic = rchan->desc.running ? rchan->desc.running->cyclic : false;
spin_unlock_irqrestore(&rchan->lock, flags);
/* if there's no residue, the cookie is complete */
- if (!residue)
+ if (!residue && !cyclic)
return DMA_COMPLETE;
dma_set_residue(txstate, residue);
^ permalink raw reply related
* Re: [PATCH] dmaengine: rcar-dmac: Update copyright information
From: Niklas Söderlund @ 2019-04-11 15:17 UTC (permalink / raw)
To: Simon Horman; +Cc: dmaengine, linux-renesas-soc, Hiroyuki Yokoyama
In-Reply-To: <20190411084937.y5m6vzcwtkqqun7s@verge.net.au>
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="UTF-8", Size: 861 bytes --]
On 2019-04-11 10:49:37 +0200, Simon Horman wrote:
> On Wed, Apr 10, 2019 at 08:26:57PM +0200, Niklas Söderlund wrote:
> > From: Hiroyuki Yokoyama <hiroyuki.yokoyama.vx@renesas.com>
> >
> > Update copyright and string for Gen3.
> >
> > Signed-off-by: Hiroyuki Yokoyama <hiroyuki.yokoyama.vx@renesas.com>
> > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
>
> Reviewed-by: Simon Horman <horms+renesas@verge.net.au>
Thanks.
>
> > ---
> > drivers/dma/sh/rcar-dmac.c | 4 ++--
>
> Not strictly related, but is it appropriate to:
>
> 1. Move this driver and drivers/dma/sh/usb-dmac.c to drivers/dma/renesas/
> 2. Remove drivers/dma/sh/sudmac.c which appears unused
I let someone with a better grasp of history answer this one. From my
side removing drivers which are unused seems like a good idea :-)
--
Regards,
Niklas Söderlund
^ permalink raw reply
* dmaengine: rcar-dmac: Update copyright information
From: Niklas Söderlund @ 2019-04-11 15:17 UTC (permalink / raw)
To: Simon Horman; +Cc: dmaengine, linux-renesas-soc, Hiroyuki Yokoyama
On 2019-04-11 10:49:37 +0200, Simon Horman wrote:
> On Wed, Apr 10, 2019 at 08:26:57PM +0200, Niklas Söderlund wrote:
> > From: Hiroyuki Yokoyama <hiroyuki.yokoyama.vx@renesas.com>
> >
> > Update copyright and string for Gen3.
> >
> > Signed-off-by: Hiroyuki Yokoyama <hiroyuki.yokoyama.vx@renesas.com>
> > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
>
> Reviewed-by: Simon Horman <horms+renesas@verge.net.au>
Thanks.
>
> > ---
> > drivers/dma/sh/rcar-dmac.c | 4 ++--
>
> Not strictly related, but is it appropriate to:
>
> 1. Move this driver and drivers/dma/sh/usb-dmac.c to drivers/dma/renesas/
> 2. Remove drivers/dma/sh/sudmac.c which appears unused
I let someone with a better grasp of history answer this one. From my
side removing drivers which are unused seems like a good idea :-)
^ permalink raw reply
* Re: [PATCH] dmaengine: rcar-dmac: Update copyright information
From: Simon Horman @ 2019-04-11 8:49 UTC (permalink / raw)
To: Niklas Söderlund; +Cc: dmaengine, linux-renesas-soc, Hiroyuki Yokoyama
In-Reply-To: <20190410182657.23034-1-niklas.soderlund+renesas@ragnatech.se>
On Wed, Apr 10, 2019 at 08:26:57PM +0200, Niklas Söderlund wrote:
> From: Hiroyuki Yokoyama <hiroyuki.yokoyama.vx@renesas.com>
>
> Update copyright and string for Gen3.
>
> Signed-off-by: Hiroyuki Yokoyama <hiroyuki.yokoyama.vx@renesas.com>
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Simon Horman <horms+renesas@verge.net.au>
> ---
> drivers/dma/sh/rcar-dmac.c | 4 ++--
Not strictly related, but is it appropriate to:
1. Move this driver and drivers/dma/sh/usb-dmac.c to drivers/dma/renesas/
2. Remove drivers/dma/sh/sudmac.c which appears unused
^ permalink raw reply
* dmaengine: rcar-dmac: Update copyright information
From: Simon Horman @ 2019-04-11 8:49 UTC (permalink / raw)
To: Niklas Söderlund; +Cc: dmaengine, linux-renesas-soc, Hiroyuki Yokoyama
On Wed, Apr 10, 2019 at 08:26:57PM +0200, Niklas Söderlund wrote:
> From: Hiroyuki Yokoyama <hiroyuki.yokoyama.vx@renesas.com>
>
> Update copyright and string for Gen3.
>
> Signed-off-by: Hiroyuki Yokoyama <hiroyuki.yokoyama.vx@renesas.com>
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Reviewed-by: Simon Horman <horms+renesas@verge.net.au>
> ---
> drivers/dma/sh/rcar-dmac.c | 4 ++--
Not strictly related, but is it appropriate to:
1. Move this driver and drivers/dma/sh/usb-dmac.c to drivers/dma/renesas/
2. Remove drivers/dma/sh/sudmac.c which appears unused
^ permalink raw reply
* Re: [PATCH v11 1/4] dmaengine: 8250_mtk_dma: add MediaTek uart DMA support
From: Long Cheng @ 2019-04-11 8:20 UTC (permalink / raw)
To: Sean Wang
Cc: Vinod Koul, Randy Dunlap, Rob Herring, Mark Rutland, Ryder Lee,
Nicolas Boichat, Matthias Brugger, Dan Williams,
Greg Kroah-Hartman, Jiri Slaby, Sean Wang, dmaengine, devicetree,
linux-arm-kernel, linux-mediatek, linux-kernel, linux-serial,
srv_heupstream, Yingjoe Chen, YT Shen, Zhenbao Liu
In-Reply-To: <CAGp9Lzos-6mLxDAgVJ783Z=SZTnxnKH+UDOLzEDvK4tng8P8dw@mail.gmail.com>
On Sun, 2019-03-10 at 17:31 -0700, Sean Wang wrote:
> Hi, Long
>
> List some comments as the below and this week I will find a board to
> test and then improve the driver.
>
> Sean
>
> On Wed, Mar 6, 2019 at 5:45 PM Long Cheng <long.cheng@mediatek.com> wrote:
> >
> > In DMA engine framework, add 8250 uart dma to support MediaTek uart.
> > If MediaTek uart enabled(SERIAL_8250_MT6577), and want to improve
> > the performance, can enable the function.
> >
> > Signed-off-by: Long Cheng <long.cheng@mediatek.com>
> > ---
> > drivers/dma/mediatek/Kconfig | 11 +
> > drivers/dma/mediatek/Makefile | 1 +
> > drivers/dma/mediatek/mtk-uart-apdma.c | 660 +++++++++++++++++++++++++++++++++
> > 3 files changed, 672 insertions(+)
> > create mode 100644 drivers/dma/mediatek/mtk-uart-apdma.c
> >
> > diff --git a/drivers/dma/mediatek/Kconfig b/drivers/dma/mediatek/Kconfig
> > index 680fc05..ac49eb6 100644
> > --- a/drivers/dma/mediatek/Kconfig
> > +++ b/drivers/dma/mediatek/Kconfig
> > @@ -24,3 +24,14 @@ config MTK_CQDMA
> >
> > This controller provides the channels which is dedicated to
> > memory-to-memory transfer to offload from CPU.
> > +
> > +config MTK_UART_APDMA
> > + tristate "MediaTek SoCs APDMA support for UART"
> > + depends on OF && SERIAL_8250_MT6577
> > + select DMA_ENGINE
> > + select DMA_VIRTUAL_CHANNELS
> > + help
> > + Support for the UART DMA engine found on MediaTek MTK SoCs.
> > + When SERIAL_8250_MT6577 is enabled, and if you want to use DMA,
> > + you can enable the config. The DMA engine can only be used
> > + with MediaTek SoCs.
> > diff --git a/drivers/dma/mediatek/Makefile b/drivers/dma/mediatek/Makefile
> > index 41bb381..61a6d29 100644
> > --- a/drivers/dma/mediatek/Makefile
> > +++ b/drivers/dma/mediatek/Makefile
> > @@ -1,2 +1,3 @@
> > +obj-$(CONFIG_MTK_UART_APDMA) += mtk-uart-apdma.o
> > obj-$(CONFIG_MTK_HSDMA) += mtk-hsdma.o
> > obj-$(CONFIG_MTK_CQDMA) += mtk-cqdma.o
> > diff --git a/drivers/dma/mediatek/mtk-uart-apdma.c b/drivers/dma/mediatek/mtk-uart-apdma.c
> > new file mode 100644
> > index 0000000..9ed7a49
> > --- /dev/null
> > +++ b/drivers/dma/mediatek/mtk-uart-apdma.c
> > @@ -0,0 +1,660 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * MediaTek Uart APDMA driver.
> > + *
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Long Cheng <long.cheng@mediatek.com>
> > + */
> > +
> > +#include <linux/clk.h>
> > +#include <linux/dmaengine.h>
> > +#include <linux/dma-mapping.h>
> > +#include <linux/err.h>
> > +#include <linux/init.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/iopoll.h>
> > +#include <linux/kernel.h>
> > +#include <linux/list.h>
> > +#include <linux/module.h>
> > +#include <linux/of_device.h>
> > +#include <linux/of_dma.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/slab.h>
> > +#include <linux/spinlock.h>
> > +
> > +#include "../virt-dma.h"
> > +
> > +/* The default number of virtual channel */
> > +#define MTK_UART_APDMA_NR_VCHANS 8
> > +
> > +#define VFF_EN_B BIT(0)
> > +#define VFF_STOP_B BIT(0)
> > +#define VFF_FLUSH_B BIT(0)
> > +#define VFF_4G_SUPPORT_B BIT(0)
> > +#define VFF_RX_INT_EN0_B BIT(0) /* rx valid size >= vff thre */
> > +#define VFF_RX_INT_EN1_B BIT(1)
> > +#define VFF_TX_INT_EN_B BIT(0) /* tx left size >= vff thre */
> > +#define VFF_WARM_RST_B BIT(0)
> > +#define VFF_RX_INT_CLR_B (BIT(0) | BIT(1))
> > +#define VFF_TX_INT_CLR_B 0
> > +#define VFF_STOP_CLR_B 0
> > +#define VFF_INT_EN_CLR_B 0
> > +#define VFF_4G_SUPPORT_CLR_B 0
> > +
> > +/* interrupt trigger level for tx */
> > +#define VFF_TX_THRE(n) ((n) * 7 / 8)
> > +/* interrupt trigger level for rx */
> > +#define VFF_RX_THRE(n) ((n) * 3 / 4)
> > +
> > +#define VFF_RING_SIZE 0xffffU
> > +/* invert this bit when wrap ring head again */
> > +#define VFF_RING_WRAP 0x10000U
> > +
> > +#define VFF_INT_FLAG 0x00
> > +#define VFF_INT_EN 0x04
> > +#define VFF_EN 0x08
> > +#define VFF_RST 0x0c
> > +#define VFF_STOP 0x10
> > +#define VFF_FLUSH 0x14
> > +#define VFF_ADDR 0x1c
> > +#define VFF_LEN 0x24
> > +#define VFF_THRE 0x28
> > +#define VFF_WPT 0x2c
> > +#define VFF_RPT 0x30
> > +/* TX: the buffer size HW can read. RX: the buffer size SW can read. */
> > +#define VFF_VALID_SIZE 0x3c
> > +/* TX: the buffer size SW can write. RX: the buffer size HW can write. */
> > +#define VFF_LEFT_SIZE 0x40
> > +#define VFF_DEBUG_STATUS 0x50
> > +#define VFF_4G_SUPPORT 0x54
> > +
> > +struct mtk_uart_apdmadev {
> > + struct dma_device ddev;
> > + struct clk *clk;
> > + bool support_33bits;
> > + unsigned int dma_requests;
> > + unsigned int *dma_irq;
> > +};
> > +
> > +struct mtk_uart_apdma_desc {
> > + struct virt_dma_desc vd;
> > +
> > + unsigned int avail_len;
> > +};
> > +
> > +struct mtk_chan {
> > + struct virt_dma_chan vc;
> > + struct dma_slave_config cfg;
> > + void __iomem *base;
> > + struct mtk_uart_apdma_desc *desc;
> > +
> > + enum dma_transfer_direction dir;
> > +
> > + bool requested;
> > +
> > + unsigned int rx_status;
> > +};
> > +
> > +static inline struct mtk_uart_apdmadev *
> > +to_mtk_uart_apdma_dev(struct dma_device *d)
> > +{
> > + return container_of(d, struct mtk_uart_apdmadev, ddev);
> > +}
> > +
> > +static inline struct mtk_chan *to_mtk_uart_apdma_chan(struct dma_chan *c)
> > +{
> > + return container_of(c, struct mtk_chan, vc.chan);
> > +}
> > +
> > +static inline struct mtk_uart_apdma_desc *to_mtk_uart_apdma_desc
> > + (struct dma_async_tx_descriptor *t)
> > +{
> > + return container_of(t, struct mtk_uart_apdma_desc, vd.tx);
> > +}
> > +
> > +static void mtk_uart_apdma_write(struct mtk_chan *c,
> > + unsigned int reg, unsigned int val)
> > +{
> > + writel(val, c->base + reg);
> > +}
> > +
> > +static unsigned int mtk_uart_apdma_read(struct mtk_chan *c, unsigned int reg)
> > +{
> > + return readl(c->base + reg);
> > +}
> > +
> > +static void mtk_uart_apdma_desc_free(struct virt_dma_desc *vd)
> > +{
> > + struct dma_chan *chan = vd->tx.chan;
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > +
> > + kfree(c->desc);
> > +}
> > +
> > +static void mtk_uart_apdma_start_tx(struct mtk_chan *c)
> > +{
> > + unsigned int len, send, left, wpt, d_wpt, tmp;
> > + int ret;
> > +
> > + left = mtk_uart_apdma_read(c, VFF_LEFT_SIZE);
> > + if (!left) {
> > + mtk_uart_apdma_write(c, VFF_INT_EN, VFF_TX_INT_EN_B);
> > + return;
> > + }
> > +
> > + /* Wait 1sec for flush, can't sleep */
> > + ret = readx_poll_timeout(readl, c->base + VFF_FLUSH, tmp,
> > + tmp != VFF_FLUSH_B, 0, 1000000);
>
> It is really not a good idea that polling up to 1 second in an
> interrupt context.
>
I will modify it in next version.
> > + if (ret)
> > + dev_warn(c->vc.chan.device->dev, "tx: fail, debug=0x%x\n",
> > + mtk_uart_apdma_read(c, VFF_DEBUG_STATUS));
> > +
> > + send = min_t(unsigned int, left, c->desc->avail_len);
> > + wpt = mtk_uart_apdma_read(c, VFF_WPT);
> > + len = c->cfg.dst_port_window_size;
> > +
> > + d_wpt = wpt + send;
> > + if ((d_wpt & VFF_RING_SIZE) >= len) {
>
> I am confused what size of VFF is. Either VFF_RING_SIZE or
> c->cfg.dst_port_window_size?
>
VFF_RRING_SIZE is max length that HW can support.The
c->cfg.dst_port_window_size is actual length.
> > + d_wpt = d_wpt - len;
> > + d_wpt = d_wpt ^ VFF_RING_WRAP;
> > + }
> > + mtk_uart_apdma_write(c, VFF_WPT, d_wpt);
> > +
> > + c->desc->avail_len -= send;
> > +
> > + mtk_uart_apdma_write(c, VFF_INT_EN, VFF_TX_INT_EN_B);
>
> Why should we need to program interrupt enabled bit again?
HW request. the step is must.
>
> > + if (mtk_uart_apdma_read(c, VFF_FLUSH) == 0U)
> > + mtk_uart_apdma_write(c, VFF_FLUSH, VFF_FLUSH_B);
> > +}
> > +
> > +static void mtk_uart_apdma_start_rx(struct mtk_chan *c)
> > +{
> > + struct mtk_uart_apdma_desc *d = c->desc;
> > + unsigned int len, wg, rg;
> > + int cnt;
> > +
> > + if ((mtk_uart_apdma_read(c, VFF_VALID_SIZE) == 0U) ||
> > + !d || !vchan_next_desc(&c->vc))
> > + return;
>
> If the current descriptor is not available, the hardware should be
> idle or stopped. so I think the condition can be removed or there is
> somewhere your handle descriptors incorrectly.
I will modify it in next version.
>
> > +
> > + len = c->cfg.src_port_window_size;
> > + rg = mtk_uart_apdma_read(c, VFF_RPT);
> > + wg = mtk_uart_apdma_read(c, VFF_WPT);
> > + cnt = (wg & VFF_RING_SIZE) - (rg & VFF_RING_SIZE);
>
> Is it possible that rg and wg would be greater than VFF_RING_SIZE?
>
No.
> > + /*
> > + * The buffer is ring buffer. If wrap bit different,
> > + * represents the start of the next cycle for WPT
> > + */
> > + if ((rg ^ wg) & VFF_RING_WRAP)
> > + cnt += len;
>
> Again, I am confused what size of VFF is. Either VFF_RING_SIZE or
> c->cfg.dst_port_window_size?
>
> > +
> > + c->rx_status = d->avail_len - cnt;
> > + mtk_uart_apdma_write(c, VFF_RPT, wg);
> > +
> > + list_del(&d->vd.node);
> > + vchan_cookie_complete(&d->vd);
> > +}
> > +
> > +static irqreturn_t mtk_uart_apdma_irq_handler(int irq, void *dev_id)
> > +{
> > + struct dma_chan *chan = (struct dma_chan *)dev_id;
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + struct mtk_uart_apdma_desc *d;
> > + unsigned long flags;
> > +
> > + spin_lock_irqsave(&c->vc.lock, flags);
> > + if (c->dir == DMA_DEV_TO_MEM) {
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_RX_INT_CLR_B);
> > + mtk_uart_apdma_start_rx(c);
> > + } else if (c->dir == DMA_MEM_TO_DEV) {
> > + d = c->desc;
> > +
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_TX_INT_CLR_B);
> > +
> > + if (d->avail_len != 0U) {
> > + mtk_uart_apdma_start_tx(c);
> > + } else {
> > + list_del(&d->vd.node);
> > + vchan_cookie_complete(&d->vd);
> > + }
> > + }
> > + spin_unlock_irqrestore(&c->vc.lock, flags);
> > +
> > + return IRQ_HANDLED;
> > +}
> > +
> > +static int mtk_uart_apdma_alloc_chan_resources(struct dma_chan *chan)
> > +{
> > + struct mtk_uart_apdmadev *mtkd = to_mtk_uart_apdma_dev(chan->device);
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + unsigned int tmp;
> > + int ret;
> > +
> > + pm_runtime_get_sync(mtkd->ddev.dev);
>
> Add an error handling, something like
>
> err = pm_runtime_get_sync(mtkd->ddev.dev);
> if (err < 0) {
> pm_runtime_put_noidle(dev);
> ...
> }
>
I will modify it in next version.
> > +
> > + mtk_uart_apdma_write(c, VFF_ADDR, 0);
> > + mtk_uart_apdma_write(c, VFF_THRE, 0);
> > + mtk_uart_apdma_write(c, VFF_LEN, 0);
> > + mtk_uart_apdma_write(c, VFF_RST, VFF_WARM_RST_B);
> > +
> > + ret = readx_poll_timeout(readl, c->base + VFF_EN, tmp, !tmp, 10, 100);
> > + if (ret) {
> > + dev_err(chan->device->dev, "dma reset: fail, timeout\n");
> > + return ret;
> > + }
> > +
> > + if (!c->requested) {
> > + c->requested = true;
>
> The variable c->requested can be saved since the same channel
> shouldn't be requested more one time
>
I will modify it in next version.
> > + ret = request_irq(mtkd->dma_irq[chan->chan_id],
> > + mtk_uart_apdma_irq_handler, IRQF_TRIGGER_NONE,
> > + KBUILD_MODNAME, chan);
> > + if (ret < 0) {
> > + dev_err(chan->device->dev, "Can't request dma IRQ\n");
> > + return -EINVAL;
> > + }
> > + }
> > +
> > + if (mtkd->support_33bits)
> > + mtk_uart_apdma_write(c, VFF_4G_SUPPORT, VFF_4G_SUPPORT_CLR_B);
> > +
> > + return ret;
> > +}
> > +
> > +static void mtk_uart_apdma_free_chan_resources(struct dma_chan *chan)
> > +{
> > + struct mtk_uart_apdmadev *mtkd = to_mtk_uart_apdma_dev(chan->device);
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > +
> > + if (c->requested) {
>
> ditto as the above
I will modify it in next version.
>
> > + c->requested = false;
> > + free_irq(mtkd->dma_irq[chan->chan_id], chan);
> > + }
> > +
> > + tasklet_kill(&c->vc.task);
> > +
> > + vchan_free_chan_resources(&c->vc);
> > +
> > + pm_runtime_put_sync(mtkd->ddev.dev);
> > +}
> > +
> > +static enum dma_status mtk_uart_apdma_tx_status(struct dma_chan *chan,
> > + dma_cookie_t cookie,
> > + struct dma_tx_state *txstate)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + enum dma_status ret;
> > +
> > + ret = dma_cookie_status(chan, cookie, txstate);
> > +
> > + dma_set_residue(txstate, c->rx_status);
> > +
>
> The handling is not enough. You should get the descriptor
> corresponding to the cookie and then calculate and return the
> ->tx_status by the descriptor
>
Because UART can't get any interrupt except DMA interrupt. So in APDMA,
need notify UART to get data. And then descriptor will be released. So
Need keep the solution.
> > + return ret;
> > +}
> > +
> > +static void mtk_uart_apdma_config_write(struct dma_chan *chan,
> > + struct dma_slave_config *cfg,
> > + enum dma_transfer_direction dir)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + struct mtk_uart_apdmadev *mtkd =
> > + to_mtk_uart_apdma_dev(c->vc.chan.device);
> > + unsigned int tmp;
> > +
> > + if (mtk_uart_apdma_read(c, VFF_EN) == VFF_EN_B)
> > + return;
> > +
> > + c->dir = dir;
>
> The direction is fixed by the device, I don't think it is required to
> keep it in a software state.
Need save it. Because RX and TX isn't all same.
>
> > +
> > + if (dir == DMA_DEV_TO_MEM) {
> > + tmp = cfg->src_port_window_size;
> > +
> > + mtk_uart_apdma_write(c, VFF_ADDR, cfg->src_addr);
>
> That is wrong. ->src_addr is the physical address where DMA slave data
> should be read (RX), not the memory address.
>
> You should program the register VFF_ADDR and VFF_LEN by sg address and
> length from device_prep_slave_sg.
I will modify it in next version.
>
> > + mtk_uart_apdma_write(c, VFF_LEN, tmp);
> > + mtk_uart_apdma_write(c, VFF_THRE, VFF_RX_THRE(tmp));
> > + mtk_uart_apdma_write(c, VFF_INT_EN,
> > + VFF_RX_INT_EN0_B | VFF_RX_INT_EN1_B);
> > + mtk_uart_apdma_write(c, VFF_RPT, 0);
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_RX_INT_CLR_B);
> > + } else if (dir == DMA_MEM_TO_DEV) {
> > + tmp = cfg->dst_port_window_size;
> > +
> > + mtk_uart_apdma_write(c, VFF_ADDR, cfg->dst_addr);
>
> That is also wrong. st_addr: this is the physical address where DMA
> slave data should be written (TX), not the memory address similar to
> the above explanation.
I will modify it in next version.
>
> > + mtk_uart_apdma_write(c, VFF_LEN, tmp);
> > + mtk_uart_apdma_write(c, VFF_THRE, VFF_TX_THRE(tmp));
> > + mtk_uart_apdma_write(c, VFF_WPT, 0);
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_TX_INT_CLR_B);
> > + }
> > +
> > + mtk_uart_apdma_write(c, VFF_EN, VFF_EN_B);
> > +
> > + if (mtkd->support_33bits)
> > + mtk_uart_apdma_write(c, VFF_4G_SUPPORT, VFF_4G_SUPPORT_B);
> > +
> > + if (mtk_uart_apdma_read(c, VFF_EN) != VFF_EN_B)
> > + dev_err(chan->device->dev, "dir[%d] fail\n", dir);
> > +}
> > +
> > +/*
> > + * dmaengine_prep_slave_single will call the function. and sglen is 1.
> > + * 8250 uart using one ring buffer, and deal with one sg.
> > + */
> > +static struct dma_async_tx_descriptor *mtk_uart_apdma_prep_slave_sg
> > + (struct dma_chan *chan, struct scatterlist *sgl,
> > + unsigned int sglen, enum dma_transfer_direction dir,
> > + unsigned long tx_flags, void *context)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + struct mtk_uart_apdma_desc *d;
> > +
> > + if (!is_slave_direction(dir))
> > + return NULL;
> > +
> > + mtk_uart_apdma_config_write(chan, &c->cfg, dir);
> > +
> > + /* Now allocate and setup the descriptor */
> > + d = kzalloc(sizeof(*d), GFP_ATOMIC);
> > + if (!d)
> > + return NULL;
> > +
> > + /* sglen is 1 */
> > + d->avail_len = sg_dma_len(sgl);
> > + c->rx_status = d->avail_len;
> > +
> > + return vchan_tx_prep(&c->vc, &d->vd, tx_flags);
> > +}
> > +
> > +static void mtk_uart_apdma_issue_pending(struct dma_chan *chan)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + struct virt_dma_desc *vd;
> > + unsigned long flags;
> > +
> > + spin_lock_irqsave(&c->vc.lock, flags);
> > + if (vchan_issue_pending(&c->vc)) {
> > + vd = vchan_next_desc(&c->vc);
> > + c->desc = to_mtk_uart_apdma_desc(&vd->tx);
> > + }
> > +
> > + if (c->dir == DMA_DEV_TO_MEM)
> > + mtk_uart_apdma_start_rx(c);
> > + else if (c->dir == DMA_MEM_TO_DEV)
> > + mtk_uart_apdma_start_tx(c);
> > +
> > + spin_unlock_irqrestore(&c->vc.lock, flags);
> > +}
> > +
> > +static int mtk_uart_apdma_slave_config(struct dma_chan *chan,
> > + struct dma_slave_config *config)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > +
> > + memcpy(&c->cfg, config, sizeof(*config));
> > +
> > + return 0;
> > +}
> > +
> > +static int mtk_uart_apdma_terminate_all(struct dma_chan *chan)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + unsigned long flags;
> > + unsigned int tmp;
> > + int ret;
> > +
> > + spin_lock_irqsave(&c->vc.lock, flags);
> > +
> > + mtk_uart_apdma_write(c, VFF_FLUSH, VFF_FLUSH_B);
> > + /* Wait 1sec for flush, can't sleep */
> > + ret = readx_poll_timeout(readl, c->base + VFF_FLUSH, tmp,
> > + tmp != VFF_FLUSH_B, 0, 1000000);
>
> It is extremely bad pending so long is in the spin_lock_irqsave
I will modify it in next version.
>
> > + if (ret)
> > + dev_err(c->vc.chan.device->dev, "flush: fail, debug=0x%x\n",
> > + mtk_uart_apdma_read(c, VFF_DEBUG_STATUS));
> > +
> > + /* set stop as 1 -> wait until en is 0 -> set stop as 0 */
> > + mtk_uart_apdma_write(c, VFF_STOP, VFF_STOP_B);
> > + ret = readx_poll_timeout(readl, c->base + VFF_EN, tmp, !tmp, 10, 100);
> > + if (ret)
> > + dev_err(c->vc.chan.device->dev, "stop: fail, debug=0x%x\n",
> > + mtk_uart_apdma_read(c, VFF_DEBUG_STATUS));
> > +
> > + mtk_uart_apdma_write(c, VFF_STOP, VFF_STOP_CLR_B);
> > + mtk_uart_apdma_write(c, VFF_INT_EN, VFF_INT_EN_CLR_B);
> > +
> > + if (c->dir == DMA_DEV_TO_MEM)
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_RX_INT_CLR_B);
> > + else if (c->dir == DMA_MEM_TO_DEV)
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_TX_INT_CLR_B);
> > +
> > + spin_unlock_irqrestore(&c->vc.lock, flags);
> > +
> > + return 0;
> > +}
> > +
> > +static int mtk_uart_apdma_device_pause(struct dma_chan *chan)
> > +{
> > + /* just for check caps pass */
> > + dev_err(chan->device->dev, "Pause can't support\n");
> > +
>
> If the device can't support hardware pause, we can do it as a software
> pause in an implementation based on vdesc.
>
ok, i will try it.
> > + return 0;
> > +}
> > +
> > +static void mtk_uart_apdma_free(struct mtk_uart_apdmadev *mtkd)
> > +{
> > + while (!list_empty(&mtkd->ddev.channels)) {
> > + struct mtk_chan *c = list_first_entry(&mtkd->ddev.channels,
> > + struct mtk_chan, vc.chan.device_node);
> > +
> > + list_del(&c->vc.chan.device_node);
> > + tasklet_kill(&c->vc.task);
> > + }
> > +}
> > +
> > +static const struct of_device_id mtk_uart_apdma_match[] = {
> > + { .compatible = "mediatek,mt6577-uart-dma", },
> > + { /* sentinel */ },
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_uart_apdma_match);
> > +
> > +static int mtk_uart_apdma_probe(struct platform_device *pdev)
> > +{
> > + struct device_node *np = pdev->dev.of_node;
> > + struct mtk_uart_apdmadev *mtkd;
> > + struct resource *res;
> > + struct mtk_chan *c;
> > + int bit_mask = 32, rc;
> > + unsigned int i;
> > +
> > + mtkd = devm_kzalloc(&pdev->dev, sizeof(*mtkd), GFP_KERNEL);
> > + if (!mtkd)
> > + return -ENOMEM;
> > +
> > + mtkd->clk = devm_clk_get(&pdev->dev, NULL);
> > + if (IS_ERR(mtkd->clk)) {
> > + dev_err(&pdev->dev, "No clock specified\n");
> > + rc = PTR_ERR(mtkd->clk);
> > + return rc;
> > + }
> > +
> > + if (of_property_read_bool(np, "mediatek,dma-33bits"))
> > + mtkd->support_33bits = true;
> > +
> > + if (mtkd->support_33bits)
> > + bit_mask = 33;
> > +
> > + rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(bit_mask));
> > + if (rc)
> > + return rc;
> > +
> > + dma_cap_set(DMA_SLAVE, mtkd->ddev.cap_mask);
> > + mtkd->ddev.device_alloc_chan_resources =
> > + mtk_uart_apdma_alloc_chan_resources;
> > + mtkd->ddev.device_free_chan_resources =
> > + mtk_uart_apdma_free_chan_resources;
> > + mtkd->ddev.device_tx_status = mtk_uart_apdma_tx_status;
> > + mtkd->ddev.device_issue_pending = mtk_uart_apdma_issue_pending;
> > + mtkd->ddev.device_prep_slave_sg = mtk_uart_apdma_prep_slave_sg;
> > + mtkd->ddev.device_config = mtk_uart_apdma_slave_config;
> > + mtkd->ddev.device_pause = mtk_uart_apdma_device_pause;
> > + mtkd->ddev.device_terminate_all = mtk_uart_apdma_terminate_all;
> > + mtkd->ddev.src_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE);
> > + mtkd->ddev.dst_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE);
> > + mtkd->ddev.directions = BIT(DMA_DEV_TO_MEM) | BIT(DMA_MEM_TO_DEV);
> > + mtkd->ddev.residue_granularity = DMA_RESIDUE_GRANULARITY_SEGMENT;
> > + mtkd->ddev.dev = &pdev->dev;
> > + INIT_LIST_HEAD(&mtkd->ddev.channels);
> > +
> > + mtkd->dma_requests = MTK_UART_APDMA_NR_VCHANS;
> > + if (of_property_read_u32(np, "dma-requests", &mtkd->dma_requests)) {
> > + dev_info(&pdev->dev,
> > + "Using %u as missing dma-requests property\n",
> > + MTK_UART_APDMA_NR_VCHANS);
> > + }
> > +
> > + mtkd->dma_irq = devm_kcalloc(&pdev->dev, mtkd->dma_requests,
> > + sizeof(*mtkd->dma_irq), GFP_KERNEL);
> > + if (!mtkd->dma_irq)
> > + return -ENOMEM;
> > +
> > + for (i = 0; i < mtkd->dma_requests; i++) {
> > + c = devm_kzalloc(mtkd->ddev.dev, sizeof(*c), GFP_KERNEL);
> > + if (!c) {
> > + rc = -ENODEV;
> > + goto err_no_dma;
> > + }
> > +
> > + res = platform_get_resource(pdev, IORESOURCE_MEM, i);
> > + if (!res) {
> > + rc = -ENODEV;
> > + goto err_no_dma;
> > + }
> > +
> > + c->base = devm_ioremap_resource(&pdev->dev, res);
> > + if (IS_ERR(c->base)) {
> > + rc = PTR_ERR(c->base);
> > + goto err_no_dma;
> > + }
> > + c->requested = false;
> > + c->vc.desc_free = mtk_uart_apdma_desc_free;
> > + vchan_init(&c->vc, &mtkd->ddev);
> > +
> > + mtkd->dma_irq[i] = platform_get_irq(pdev, i);
> > + if ((int)mtkd->dma_irq[i] < 0) {
> > + dev_err(&pdev->dev, "failed to get IRQ[%d]\n", i);
> > + rc = -EINVAL;
> > + goto err_no_dma;
> > + }
> > + }
> > +
> > + pm_runtime_enable(&pdev->dev);
> > + pm_runtime_set_active(&pdev->dev);
> > +
> > + rc = dma_async_device_register(&mtkd->ddev);
> > + if (rc)
> > + goto rpm_disable;
> > +
> > + platform_set_drvdata(pdev, mtkd);
> > +
> > + /* Device-tree DMA controller registration */
> > + rc = of_dma_controller_register(np, of_dma_xlate_by_chan_id, mtkd);
> > + if (rc)
> > + goto dma_remove;
> > +
> > + return rc;
> > +
> > +dma_remove:
> > + dma_async_device_unregister(&mtkd->ddev);
> > +rpm_disable:
> > + pm_runtime_disable(&pdev->dev);
> > +err_no_dma:
> > + mtk_uart_apdma_free(mtkd);
> > + return rc;
> > +}
> > +
> > +static int mtk_uart_apdma_remove(struct platform_device *pdev)
> > +{
> > + struct mtk_uart_apdmadev *mtkd = platform_get_drvdata(pdev);
> > +
> > + if (pdev->dev.of_node)
> > + of_dma_controller_free(pdev->dev.of_node);
> > +
> > + pm_runtime_disable(&pdev->dev);
> > + pm_runtime_put_noidle(&pdev->dev);
>
> That pm_runtime_put_noidle should be removed or it causes an
> inconsistency with the probe handler.
>
Ok, I will remove it.
> > +
> > + dma_async_device_unregister(&mtkd->ddev);
> > + mtk_uart_apdma_free(mtkd);
> > +
> > + return 0;
> > +}
> > +
> > +#ifdef CONFIG_PM_SLEEP
> > +static int mtk_uart_apdma_suspend(struct device *dev)
> > +{
> > + struct mtk_uart_apdmadev *mtkd = dev_get_drvdata(dev);
> > +
> > + if (!pm_runtime_suspended(dev))
> > + clk_disable_unprepare(mtkd->clk);
> > +
> > + return 0;
> > +}
> > +
> > +static int mtk_uart_apdma_resume(struct device *dev)
> > +{
> > + int ret;
> > + struct mtk_uart_apdmadev *mtkd = dev_get_drvdata(dev);
> > +
> > + if (!pm_runtime_suspended(dev)) {
> > + ret = clk_prepare_enable(mtkd->clk);
> > + if (ret)
> > + return ret;
> > + }
> > +
> > + return 0;
> > +}
> > +#endif /* CONFIG_PM_SLEEP */
> > +
> > +#ifdef CONFIG_PM
> > +static int mtk_uart_apdma_runtime_suspend(struct device *dev)
> > +{
> > + struct mtk_uart_apdmadev *mtkd = dev_get_drvdata(dev);
> > +
> > + clk_disable_unprepare(mtkd->clk);
> > +
> > + return 0;
> > +}
> > +
> > +static int mtk_uart_apdma_runtime_resume(struct device *dev)
> > +{
> > + int ret;
> > + struct mtk_uart_apdmadev *mtkd = dev_get_drvdata(dev);
> > +
> > + ret = clk_prepare_enable(mtkd->clk);
> > + if (ret)
> > + return ret;
> > +
> > + return 0;
> > +}
> > +#endif /* CONFIG_PM */
> > +
> > +static const struct dev_pm_ops mtk_uart_apdma_pm_ops = {
> > + SET_SYSTEM_SLEEP_PM_OPS(mtk_uart_apdma_suspend, mtk_uart_apdma_resume)
> > + SET_RUNTIME_PM_OPS(mtk_uart_apdma_runtime_suspend,
> > + mtk_uart_apdma_runtime_resume, NULL)
> > +};
>
> It probably causes a build error when CONFIG_PM is not enabled.
> and use a UNIVERSAL_DEV_PM_OPS because the runtime suspend/resume and
> system suspend/resume for the dma are
> almost the same.
>
I remember that these had test. It's build pass.
> > +
> > +static struct platform_driver mtk_uart_apdma_driver = {
> > + .probe = mtk_uart_apdma_probe,
> > + .remove = mtk_uart_apdma_remove,
> > + .driver = {
> > + .name = KBUILD_MODNAME,
> > + .pm = &mtk_uart_apdma_pm_ops,
> > + .of_match_table = of_match_ptr(mtk_uart_apdma_match),
> > + },
> > +};
> > +
> > +module_platform_driver(mtk_uart_apdma_driver);
> > +
> > +MODULE_DESCRIPTION("MediaTek UART APDMA Controller Driver");
> > +MODULE_AUTHOR("Long Cheng <long.cheng@mediatek.com>");
> > +MODULE_LICENSE("GPL v2");
> > +
> > --
> > 1.7.9.5
> >
^ permalink raw reply
* [v11,1/4] dmaengine: 8250_mtk_dma: add MediaTek uart DMA support
From: Long Cheng @ 2019-04-11 8:20 UTC (permalink / raw)
To: Sean Wang
Cc: Vinod Koul, Randy Dunlap, Rob Herring, Mark Rutland, Ryder Lee,
Nicolas Boichat, Matthias Brugger, Dan Williams,
Greg Kroah-Hartman, Jiri Slaby, Sean Wang, dmaengine, devicetree,
linux-arm-kernel, linux-mediatek, linux-kernel, linux-serial,
srv_heupstream, Yingjoe Chen, YT Shen, Zhenbao Liu
On Sun, 2019-03-10 at 17:31 -0700, Sean Wang wrote:
> Hi, Long
>
> List some comments as the below and this week I will find a board to
> test and then improve the driver.
>
> Sean
>
> On Wed, Mar 6, 2019 at 5:45 PM Long Cheng <long.cheng@mediatek.com> wrote:
> >
> > In DMA engine framework, add 8250 uart dma to support MediaTek uart.
> > If MediaTek uart enabled(SERIAL_8250_MT6577), and want to improve
> > the performance, can enable the function.
> >
> > Signed-off-by: Long Cheng <long.cheng@mediatek.com>
> > ---
> > drivers/dma/mediatek/Kconfig | 11 +
> > drivers/dma/mediatek/Makefile | 1 +
> > drivers/dma/mediatek/mtk-uart-apdma.c | 660 +++++++++++++++++++++++++++++++++
> > 3 files changed, 672 insertions(+)
> > create mode 100644 drivers/dma/mediatek/mtk-uart-apdma.c
> >
> > diff --git a/drivers/dma/mediatek/Kconfig b/drivers/dma/mediatek/Kconfig
> > index 680fc05..ac49eb6 100644
> > --- a/drivers/dma/mediatek/Kconfig
> > +++ b/drivers/dma/mediatek/Kconfig
> > @@ -24,3 +24,14 @@ config MTK_CQDMA
> >
> > This controller provides the channels which is dedicated to
> > memory-to-memory transfer to offload from CPU.
> > +
> > +config MTK_UART_APDMA
> > + tristate "MediaTek SoCs APDMA support for UART"
> > + depends on OF && SERIAL_8250_MT6577
> > + select DMA_ENGINE
> > + select DMA_VIRTUAL_CHANNELS
> > + help
> > + Support for the UART DMA engine found on MediaTek MTK SoCs.
> > + When SERIAL_8250_MT6577 is enabled, and if you want to use DMA,
> > + you can enable the config. The DMA engine can only be used
> > + with MediaTek SoCs.
> > diff --git a/drivers/dma/mediatek/Makefile b/drivers/dma/mediatek/Makefile
> > index 41bb381..61a6d29 100644
> > --- a/drivers/dma/mediatek/Makefile
> > +++ b/drivers/dma/mediatek/Makefile
> > @@ -1,2 +1,3 @@
> > +obj-$(CONFIG_MTK_UART_APDMA) += mtk-uart-apdma.o
> > obj-$(CONFIG_MTK_HSDMA) += mtk-hsdma.o
> > obj-$(CONFIG_MTK_CQDMA) += mtk-cqdma.o
> > diff --git a/drivers/dma/mediatek/mtk-uart-apdma.c b/drivers/dma/mediatek/mtk-uart-apdma.c
> > new file mode 100644
> > index 0000000..9ed7a49
> > --- /dev/null
> > +++ b/drivers/dma/mediatek/mtk-uart-apdma.c
> > @@ -0,0 +1,660 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * MediaTek Uart APDMA driver.
> > + *
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Long Cheng <long.cheng@mediatek.com>
> > + */
> > +
> > +#include <linux/clk.h>
> > +#include <linux/dmaengine.h>
> > +#include <linux/dma-mapping.h>
> > +#include <linux/err.h>
> > +#include <linux/init.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/iopoll.h>
> > +#include <linux/kernel.h>
> > +#include <linux/list.h>
> > +#include <linux/module.h>
> > +#include <linux/of_device.h>
> > +#include <linux/of_dma.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/slab.h>
> > +#include <linux/spinlock.h>
> > +
> > +#include "../virt-dma.h"
> > +
> > +/* The default number of virtual channel */
> > +#define MTK_UART_APDMA_NR_VCHANS 8
> > +
> > +#define VFF_EN_B BIT(0)
> > +#define VFF_STOP_B BIT(0)
> > +#define VFF_FLUSH_B BIT(0)
> > +#define VFF_4G_SUPPORT_B BIT(0)
> > +#define VFF_RX_INT_EN0_B BIT(0) /* rx valid size >= vff thre */
> > +#define VFF_RX_INT_EN1_B BIT(1)
> > +#define VFF_TX_INT_EN_B BIT(0) /* tx left size >= vff thre */
> > +#define VFF_WARM_RST_B BIT(0)
> > +#define VFF_RX_INT_CLR_B (BIT(0) | BIT(1))
> > +#define VFF_TX_INT_CLR_B 0
> > +#define VFF_STOP_CLR_B 0
> > +#define VFF_INT_EN_CLR_B 0
> > +#define VFF_4G_SUPPORT_CLR_B 0
> > +
> > +/* interrupt trigger level for tx */
> > +#define VFF_TX_THRE(n) ((n) * 7 / 8)
> > +/* interrupt trigger level for rx */
> > +#define VFF_RX_THRE(n) ((n) * 3 / 4)
> > +
> > +#define VFF_RING_SIZE 0xffffU
> > +/* invert this bit when wrap ring head again */
> > +#define VFF_RING_WRAP 0x10000U
> > +
> > +#define VFF_INT_FLAG 0x00
> > +#define VFF_INT_EN 0x04
> > +#define VFF_EN 0x08
> > +#define VFF_RST 0x0c
> > +#define VFF_STOP 0x10
> > +#define VFF_FLUSH 0x14
> > +#define VFF_ADDR 0x1c
> > +#define VFF_LEN 0x24
> > +#define VFF_THRE 0x28
> > +#define VFF_WPT 0x2c
> > +#define VFF_RPT 0x30
> > +/* TX: the buffer size HW can read. RX: the buffer size SW can read. */
> > +#define VFF_VALID_SIZE 0x3c
> > +/* TX: the buffer size SW can write. RX: the buffer size HW can write. */
> > +#define VFF_LEFT_SIZE 0x40
> > +#define VFF_DEBUG_STATUS 0x50
> > +#define VFF_4G_SUPPORT 0x54
> > +
> > +struct mtk_uart_apdmadev {
> > + struct dma_device ddev;
> > + struct clk *clk;
> > + bool support_33bits;
> > + unsigned int dma_requests;
> > + unsigned int *dma_irq;
> > +};
> > +
> > +struct mtk_uart_apdma_desc {
> > + struct virt_dma_desc vd;
> > +
> > + unsigned int avail_len;
> > +};
> > +
> > +struct mtk_chan {
> > + struct virt_dma_chan vc;
> > + struct dma_slave_config cfg;
> > + void __iomem *base;
> > + struct mtk_uart_apdma_desc *desc;
> > +
> > + enum dma_transfer_direction dir;
> > +
> > + bool requested;
> > +
> > + unsigned int rx_status;
> > +};
> > +
> > +static inline struct mtk_uart_apdmadev *
> > +to_mtk_uart_apdma_dev(struct dma_device *d)
> > +{
> > + return container_of(d, struct mtk_uart_apdmadev, ddev);
> > +}
> > +
> > +static inline struct mtk_chan *to_mtk_uart_apdma_chan(struct dma_chan *c)
> > +{
> > + return container_of(c, struct mtk_chan, vc.chan);
> > +}
> > +
> > +static inline struct mtk_uart_apdma_desc *to_mtk_uart_apdma_desc
> > + (struct dma_async_tx_descriptor *t)
> > +{
> > + return container_of(t, struct mtk_uart_apdma_desc, vd.tx);
> > +}
> > +
> > +static void mtk_uart_apdma_write(struct mtk_chan *c,
> > + unsigned int reg, unsigned int val)
> > +{
> > + writel(val, c->base + reg);
> > +}
> > +
> > +static unsigned int mtk_uart_apdma_read(struct mtk_chan *c, unsigned int reg)
> > +{
> > + return readl(c->base + reg);
> > +}
> > +
> > +static void mtk_uart_apdma_desc_free(struct virt_dma_desc *vd)
> > +{
> > + struct dma_chan *chan = vd->tx.chan;
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > +
> > + kfree(c->desc);
> > +}
> > +
> > +static void mtk_uart_apdma_start_tx(struct mtk_chan *c)
> > +{
> > + unsigned int len, send, left, wpt, d_wpt, tmp;
> > + int ret;
> > +
> > + left = mtk_uart_apdma_read(c, VFF_LEFT_SIZE);
> > + if (!left) {
> > + mtk_uart_apdma_write(c, VFF_INT_EN, VFF_TX_INT_EN_B);
> > + return;
> > + }
> > +
> > + /* Wait 1sec for flush, can't sleep */
> > + ret = readx_poll_timeout(readl, c->base + VFF_FLUSH, tmp,
> > + tmp != VFF_FLUSH_B, 0, 1000000);
>
> It is really not a good idea that polling up to 1 second in an
> interrupt context.
>
I will modify it in next version.
> > + if (ret)
> > + dev_warn(c->vc.chan.device->dev, "tx: fail, debug=0x%x\n",
> > + mtk_uart_apdma_read(c, VFF_DEBUG_STATUS));
> > +
> > + send = min_t(unsigned int, left, c->desc->avail_len);
> > + wpt = mtk_uart_apdma_read(c, VFF_WPT);
> > + len = c->cfg.dst_port_window_size;
> > +
> > + d_wpt = wpt + send;
> > + if ((d_wpt & VFF_RING_SIZE) >= len) {
>
> I am confused what size of VFF is. Either VFF_RING_SIZE or
> c->cfg.dst_port_window_size?
>
VFF_RRING_SIZE is max length that HW can support.The
c->cfg.dst_port_window_size is actual length.
> > + d_wpt = d_wpt - len;
> > + d_wpt = d_wpt ^ VFF_RING_WRAP;
> > + }
> > + mtk_uart_apdma_write(c, VFF_WPT, d_wpt);
> > +
> > + c->desc->avail_len -= send;
> > +
> > + mtk_uart_apdma_write(c, VFF_INT_EN, VFF_TX_INT_EN_B);
>
> Why should we need to program interrupt enabled bit again?
HW request. the step is must.
>
> > + if (mtk_uart_apdma_read(c, VFF_FLUSH) == 0U)
> > + mtk_uart_apdma_write(c, VFF_FLUSH, VFF_FLUSH_B);
> > +}
> > +
> > +static void mtk_uart_apdma_start_rx(struct mtk_chan *c)
> > +{
> > + struct mtk_uart_apdma_desc *d = c->desc;
> > + unsigned int len, wg, rg;
> > + int cnt;
> > +
> > + if ((mtk_uart_apdma_read(c, VFF_VALID_SIZE) == 0U) ||
> > + !d || !vchan_next_desc(&c->vc))
> > + return;
>
> If the current descriptor is not available, the hardware should be
> idle or stopped. so I think the condition can be removed or there is
> somewhere your handle descriptors incorrectly.
I will modify it in next version.
>
> > +
> > + len = c->cfg.src_port_window_size;
> > + rg = mtk_uart_apdma_read(c, VFF_RPT);
> > + wg = mtk_uart_apdma_read(c, VFF_WPT);
> > + cnt = (wg & VFF_RING_SIZE) - (rg & VFF_RING_SIZE);
>
> Is it possible that rg and wg would be greater than VFF_RING_SIZE?
>
No.
> > + /*
> > + * The buffer is ring buffer. If wrap bit different,
> > + * represents the start of the next cycle for WPT
> > + */
> > + if ((rg ^ wg) & VFF_RING_WRAP)
> > + cnt += len;
>
> Again, I am confused what size of VFF is. Either VFF_RING_SIZE or
> c->cfg.dst_port_window_size?
>
> > +
> > + c->rx_status = d->avail_len - cnt;
> > + mtk_uart_apdma_write(c, VFF_RPT, wg);
> > +
> > + list_del(&d->vd.node);
> > + vchan_cookie_complete(&d->vd);
> > +}
> > +
> > +static irqreturn_t mtk_uart_apdma_irq_handler(int irq, void *dev_id)
> > +{
> > + struct dma_chan *chan = (struct dma_chan *)dev_id;
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + struct mtk_uart_apdma_desc *d;
> > + unsigned long flags;
> > +
> > + spin_lock_irqsave(&c->vc.lock, flags);
> > + if (c->dir == DMA_DEV_TO_MEM) {
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_RX_INT_CLR_B);
> > + mtk_uart_apdma_start_rx(c);
> > + } else if (c->dir == DMA_MEM_TO_DEV) {
> > + d = c->desc;
> > +
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_TX_INT_CLR_B);
> > +
> > + if (d->avail_len != 0U) {
> > + mtk_uart_apdma_start_tx(c);
> > + } else {
> > + list_del(&d->vd.node);
> > + vchan_cookie_complete(&d->vd);
> > + }
> > + }
> > + spin_unlock_irqrestore(&c->vc.lock, flags);
> > +
> > + return IRQ_HANDLED;
> > +}
> > +
> > +static int mtk_uart_apdma_alloc_chan_resources(struct dma_chan *chan)
> > +{
> > + struct mtk_uart_apdmadev *mtkd = to_mtk_uart_apdma_dev(chan->device);
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + unsigned int tmp;
> > + int ret;
> > +
> > + pm_runtime_get_sync(mtkd->ddev.dev);
>
> Add an error handling, something like
>
> err = pm_runtime_get_sync(mtkd->ddev.dev);
> if (err < 0) {
> pm_runtime_put_noidle(dev);
> ...
> }
>
I will modify it in next version.
> > +
> > + mtk_uart_apdma_write(c, VFF_ADDR, 0);
> > + mtk_uart_apdma_write(c, VFF_THRE, 0);
> > + mtk_uart_apdma_write(c, VFF_LEN, 0);
> > + mtk_uart_apdma_write(c, VFF_RST, VFF_WARM_RST_B);
> > +
> > + ret = readx_poll_timeout(readl, c->base + VFF_EN, tmp, !tmp, 10, 100);
> > + if (ret) {
> > + dev_err(chan->device->dev, "dma reset: fail, timeout\n");
> > + return ret;
> > + }
> > +
> > + if (!c->requested) {
> > + c->requested = true;
>
> The variable c->requested can be saved since the same channel
> shouldn't be requested more one time
>
I will modify it in next version.
> > + ret = request_irq(mtkd->dma_irq[chan->chan_id],
> > + mtk_uart_apdma_irq_handler, IRQF_TRIGGER_NONE,
> > + KBUILD_MODNAME, chan);
> > + if (ret < 0) {
> > + dev_err(chan->device->dev, "Can't request dma IRQ\n");
> > + return -EINVAL;
> > + }
> > + }
> > +
> > + if (mtkd->support_33bits)
> > + mtk_uart_apdma_write(c, VFF_4G_SUPPORT, VFF_4G_SUPPORT_CLR_B);
> > +
> > + return ret;
> > +}
> > +
> > +static void mtk_uart_apdma_free_chan_resources(struct dma_chan *chan)
> > +{
> > + struct mtk_uart_apdmadev *mtkd = to_mtk_uart_apdma_dev(chan->device);
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > +
> > + if (c->requested) {
>
> ditto as the above
I will modify it in next version.
>
> > + c->requested = false;
> > + free_irq(mtkd->dma_irq[chan->chan_id], chan);
> > + }
> > +
> > + tasklet_kill(&c->vc.task);
> > +
> > + vchan_free_chan_resources(&c->vc);
> > +
> > + pm_runtime_put_sync(mtkd->ddev.dev);
> > +}
> > +
> > +static enum dma_status mtk_uart_apdma_tx_status(struct dma_chan *chan,
> > + dma_cookie_t cookie,
> > + struct dma_tx_state *txstate)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + enum dma_status ret;
> > +
> > + ret = dma_cookie_status(chan, cookie, txstate);
> > +
> > + dma_set_residue(txstate, c->rx_status);
> > +
>
> The handling is not enough. You should get the descriptor
> corresponding to the cookie and then calculate and return the
> ->tx_status by the descriptor
>
Because UART can't get any interrupt except DMA interrupt. So in APDMA,
need notify UART to get data. And then descriptor will be released. So
Need keep the solution.
> > + return ret;
> > +}
> > +
> > +static void mtk_uart_apdma_config_write(struct dma_chan *chan,
> > + struct dma_slave_config *cfg,
> > + enum dma_transfer_direction dir)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + struct mtk_uart_apdmadev *mtkd =
> > + to_mtk_uart_apdma_dev(c->vc.chan.device);
> > + unsigned int tmp;
> > +
> > + if (mtk_uart_apdma_read(c, VFF_EN) == VFF_EN_B)
> > + return;
> > +
> > + c->dir = dir;
>
> The direction is fixed by the device, I don't think it is required to
> keep it in a software state.
Need save it. Because RX and TX isn't all same.
>
> > +
> > + if (dir == DMA_DEV_TO_MEM) {
> > + tmp = cfg->src_port_window_size;
> > +
> > + mtk_uart_apdma_write(c, VFF_ADDR, cfg->src_addr);
>
> That is wrong. ->src_addr is the physical address where DMA slave data
> should be read (RX), not the memory address.
>
> You should program the register VFF_ADDR and VFF_LEN by sg address and
> length from device_prep_slave_sg.
I will modify it in next version.
>
> > + mtk_uart_apdma_write(c, VFF_LEN, tmp);
> > + mtk_uart_apdma_write(c, VFF_THRE, VFF_RX_THRE(tmp));
> > + mtk_uart_apdma_write(c, VFF_INT_EN,
> > + VFF_RX_INT_EN0_B | VFF_RX_INT_EN1_B);
> > + mtk_uart_apdma_write(c, VFF_RPT, 0);
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_RX_INT_CLR_B);
> > + } else if (dir == DMA_MEM_TO_DEV) {
> > + tmp = cfg->dst_port_window_size;
> > +
> > + mtk_uart_apdma_write(c, VFF_ADDR, cfg->dst_addr);
>
> That is also wrong. st_addr: this is the physical address where DMA
> slave data should be written (TX), not the memory address similar to
> the above explanation.
I will modify it in next version.
>
> > + mtk_uart_apdma_write(c, VFF_LEN, tmp);
> > + mtk_uart_apdma_write(c, VFF_THRE, VFF_TX_THRE(tmp));
> > + mtk_uart_apdma_write(c, VFF_WPT, 0);
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_TX_INT_CLR_B);
> > + }
> > +
> > + mtk_uart_apdma_write(c, VFF_EN, VFF_EN_B);
> > +
> > + if (mtkd->support_33bits)
> > + mtk_uart_apdma_write(c, VFF_4G_SUPPORT, VFF_4G_SUPPORT_B);
> > +
> > + if (mtk_uart_apdma_read(c, VFF_EN) != VFF_EN_B)
> > + dev_err(chan->device->dev, "dir[%d] fail\n", dir);
> > +}
> > +
> > +/*
> > + * dmaengine_prep_slave_single will call the function. and sglen is 1.
> > + * 8250 uart using one ring buffer, and deal with one sg.
> > + */
> > +static struct dma_async_tx_descriptor *mtk_uart_apdma_prep_slave_sg
> > + (struct dma_chan *chan, struct scatterlist *sgl,
> > + unsigned int sglen, enum dma_transfer_direction dir,
> > + unsigned long tx_flags, void *context)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + struct mtk_uart_apdma_desc *d;
> > +
> > + if (!is_slave_direction(dir))
> > + return NULL;
> > +
> > + mtk_uart_apdma_config_write(chan, &c->cfg, dir);
> > +
> > + /* Now allocate and setup the descriptor */
> > + d = kzalloc(sizeof(*d), GFP_ATOMIC);
> > + if (!d)
> > + return NULL;
> > +
> > + /* sglen is 1 */
> > + d->avail_len = sg_dma_len(sgl);
> > + c->rx_status = d->avail_len;
> > +
> > + return vchan_tx_prep(&c->vc, &d->vd, tx_flags);
> > +}
> > +
> > +static void mtk_uart_apdma_issue_pending(struct dma_chan *chan)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + struct virt_dma_desc *vd;
> > + unsigned long flags;
> > +
> > + spin_lock_irqsave(&c->vc.lock, flags);
> > + if (vchan_issue_pending(&c->vc)) {
> > + vd = vchan_next_desc(&c->vc);
> > + c->desc = to_mtk_uart_apdma_desc(&vd->tx);
> > + }
> > +
> > + if (c->dir == DMA_DEV_TO_MEM)
> > + mtk_uart_apdma_start_rx(c);
> > + else if (c->dir == DMA_MEM_TO_DEV)
> > + mtk_uart_apdma_start_tx(c);
> > +
> > + spin_unlock_irqrestore(&c->vc.lock, flags);
> > +}
> > +
> > +static int mtk_uart_apdma_slave_config(struct dma_chan *chan,
> > + struct dma_slave_config *config)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > +
> > + memcpy(&c->cfg, config, sizeof(*config));
> > +
> > + return 0;
> > +}
> > +
> > +static int mtk_uart_apdma_terminate_all(struct dma_chan *chan)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + unsigned long flags;
> > + unsigned int tmp;
> > + int ret;
> > +
> > + spin_lock_irqsave(&c->vc.lock, flags);
> > +
> > + mtk_uart_apdma_write(c, VFF_FLUSH, VFF_FLUSH_B);
> > + /* Wait 1sec for flush, can't sleep */
> > + ret = readx_poll_timeout(readl, c->base + VFF_FLUSH, tmp,
> > + tmp != VFF_FLUSH_B, 0, 1000000);
>
> It is extremely bad pending so long is in the spin_lock_irqsave
I will modify it in next version.
>
> > + if (ret)
> > + dev_err(c->vc.chan.device->dev, "flush: fail, debug=0x%x\n",
> > + mtk_uart_apdma_read(c, VFF_DEBUG_STATUS));
> > +
> > + /* set stop as 1 -> wait until en is 0 -> set stop as 0 */
> > + mtk_uart_apdma_write(c, VFF_STOP, VFF_STOP_B);
> > + ret = readx_poll_timeout(readl, c->base + VFF_EN, tmp, !tmp, 10, 100);
> > + if (ret)
> > + dev_err(c->vc.chan.device->dev, "stop: fail, debug=0x%x\n",
> > + mtk_uart_apdma_read(c, VFF_DEBUG_STATUS));
> > +
> > + mtk_uart_apdma_write(c, VFF_STOP, VFF_STOP_CLR_B);
> > + mtk_uart_apdma_write(c, VFF_INT_EN, VFF_INT_EN_CLR_B);
> > +
> > + if (c->dir == DMA_DEV_TO_MEM)
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_RX_INT_CLR_B);
> > + else if (c->dir == DMA_MEM_TO_DEV)
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_TX_INT_CLR_B);
> > +
> > + spin_unlock_irqrestore(&c->vc.lock, flags);
> > +
> > + return 0;
> > +}
> > +
> > +static int mtk_uart_apdma_device_pause(struct dma_chan *chan)
> > +{
> > + /* just for check caps pass */
> > + dev_err(chan->device->dev, "Pause can't support\n");
> > +
>
> If the device can't support hardware pause, we can do it as a software
> pause in an implementation based on vdesc.
>
ok, i will try it.
> > + return 0;
> > +}
> > +
> > +static void mtk_uart_apdma_free(struct mtk_uart_apdmadev *mtkd)
> > +{
> > + while (!list_empty(&mtkd->ddev.channels)) {
> > + struct mtk_chan *c = list_first_entry(&mtkd->ddev.channels,
> > + struct mtk_chan, vc.chan.device_node);
> > +
> > + list_del(&c->vc.chan.device_node);
> > + tasklet_kill(&c->vc.task);
> > + }
> > +}
> > +
> > +static const struct of_device_id mtk_uart_apdma_match[] = {
> > + { .compatible = "mediatek,mt6577-uart-dma", },
> > + { /* sentinel */ },
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_uart_apdma_match);
> > +
> > +static int mtk_uart_apdma_probe(struct platform_device *pdev)
> > +{
> > + struct device_node *np = pdev->dev.of_node;
> > + struct mtk_uart_apdmadev *mtkd;
> > + struct resource *res;
> > + struct mtk_chan *c;
> > + int bit_mask = 32, rc;
> > + unsigned int i;
> > +
> > + mtkd = devm_kzalloc(&pdev->dev, sizeof(*mtkd), GFP_KERNEL);
> > + if (!mtkd)
> > + return -ENOMEM;
> > +
> > + mtkd->clk = devm_clk_get(&pdev->dev, NULL);
> > + if (IS_ERR(mtkd->clk)) {
> > + dev_err(&pdev->dev, "No clock specified\n");
> > + rc = PTR_ERR(mtkd->clk);
> > + return rc;
> > + }
> > +
> > + if (of_property_read_bool(np, "mediatek,dma-33bits"))
> > + mtkd->support_33bits = true;
> > +
> > + if (mtkd->support_33bits)
> > + bit_mask = 33;
> > +
> > + rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(bit_mask));
> > + if (rc)
> > + return rc;
> > +
> > + dma_cap_set(DMA_SLAVE, mtkd->ddev.cap_mask);
> > + mtkd->ddev.device_alloc_chan_resources =
> > + mtk_uart_apdma_alloc_chan_resources;
> > + mtkd->ddev.device_free_chan_resources =
> > + mtk_uart_apdma_free_chan_resources;
> > + mtkd->ddev.device_tx_status = mtk_uart_apdma_tx_status;
> > + mtkd->ddev.device_issue_pending = mtk_uart_apdma_issue_pending;
> > + mtkd->ddev.device_prep_slave_sg = mtk_uart_apdma_prep_slave_sg;
> > + mtkd->ddev.device_config = mtk_uart_apdma_slave_config;
> > + mtkd->ddev.device_pause = mtk_uart_apdma_device_pause;
> > + mtkd->ddev.device_terminate_all = mtk_uart_apdma_terminate_all;
> > + mtkd->ddev.src_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE);
> > + mtkd->ddev.dst_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE);
> > + mtkd->ddev.directions = BIT(DMA_DEV_TO_MEM) | BIT(DMA_MEM_TO_DEV);
> > + mtkd->ddev.residue_granularity = DMA_RESIDUE_GRANULARITY_SEGMENT;
> > + mtkd->ddev.dev = &pdev->dev;
> > + INIT_LIST_HEAD(&mtkd->ddev.channels);
> > +
> > + mtkd->dma_requests = MTK_UART_APDMA_NR_VCHANS;
> > + if (of_property_read_u32(np, "dma-requests", &mtkd->dma_requests)) {
> > + dev_info(&pdev->dev,
> > + "Using %u as missing dma-requests property\n",
> > + MTK_UART_APDMA_NR_VCHANS);
> > + }
> > +
> > + mtkd->dma_irq = devm_kcalloc(&pdev->dev, mtkd->dma_requests,
> > + sizeof(*mtkd->dma_irq), GFP_KERNEL);
> > + if (!mtkd->dma_irq)
> > + return -ENOMEM;
> > +
> > + for (i = 0; i < mtkd->dma_requests; i++) {
> > + c = devm_kzalloc(mtkd->ddev.dev, sizeof(*c), GFP_KERNEL);
> > + if (!c) {
> > + rc = -ENODEV;
> > + goto err_no_dma;
> > + }
> > +
> > + res = platform_get_resource(pdev, IORESOURCE_MEM, i);
> > + if (!res) {
> > + rc = -ENODEV;
> > + goto err_no_dma;
> > + }
> > +
> > + c->base = devm_ioremap_resource(&pdev->dev, res);
> > + if (IS_ERR(c->base)) {
> > + rc = PTR_ERR(c->base);
> > + goto err_no_dma;
> > + }
> > + c->requested = false;
> > + c->vc.desc_free = mtk_uart_apdma_desc_free;
> > + vchan_init(&c->vc, &mtkd->ddev);
> > +
> > + mtkd->dma_irq[i] = platform_get_irq(pdev, i);
> > + if ((int)mtkd->dma_irq[i] < 0) {
> > + dev_err(&pdev->dev, "failed to get IRQ[%d]\n", i);
> > + rc = -EINVAL;
> > + goto err_no_dma;
> > + }
> > + }
> > +
> > + pm_runtime_enable(&pdev->dev);
> > + pm_runtime_set_active(&pdev->dev);
> > +
> > + rc = dma_async_device_register(&mtkd->ddev);
> > + if (rc)
> > + goto rpm_disable;
> > +
> > + platform_set_drvdata(pdev, mtkd);
> > +
> > + /* Device-tree DMA controller registration */
> > + rc = of_dma_controller_register(np, of_dma_xlate_by_chan_id, mtkd);
> > + if (rc)
> > + goto dma_remove;
> > +
> > + return rc;
> > +
> > +dma_remove:
> > + dma_async_device_unregister(&mtkd->ddev);
> > +rpm_disable:
> > + pm_runtime_disable(&pdev->dev);
> > +err_no_dma:
> > + mtk_uart_apdma_free(mtkd);
> > + return rc;
> > +}
> > +
> > +static int mtk_uart_apdma_remove(struct platform_device *pdev)
> > +{
> > + struct mtk_uart_apdmadev *mtkd = platform_get_drvdata(pdev);
> > +
> > + if (pdev->dev.of_node)
> > + of_dma_controller_free(pdev->dev.of_node);
> > +
> > + pm_runtime_disable(&pdev->dev);
> > + pm_runtime_put_noidle(&pdev->dev);
>
> That pm_runtime_put_noidle should be removed or it causes an
> inconsistency with the probe handler.
>
Ok, I will remove it.
> > +
> > + dma_async_device_unregister(&mtkd->ddev);
> > + mtk_uart_apdma_free(mtkd);
> > +
> > + return 0;
> > +}
> > +
> > +#ifdef CONFIG_PM_SLEEP
> > +static int mtk_uart_apdma_suspend(struct device *dev)
> > +{
> > + struct mtk_uart_apdmadev *mtkd = dev_get_drvdata(dev);
> > +
> > + if (!pm_runtime_suspended(dev))
> > + clk_disable_unprepare(mtkd->clk);
> > +
> > + return 0;
> > +}
> > +
> > +static int mtk_uart_apdma_resume(struct device *dev)
> > +{
> > + int ret;
> > + struct mtk_uart_apdmadev *mtkd = dev_get_drvdata(dev);
> > +
> > + if (!pm_runtime_suspended(dev)) {
> > + ret = clk_prepare_enable(mtkd->clk);
> > + if (ret)
> > + return ret;
> > + }
> > +
> > + return 0;
> > +}
> > +#endif /* CONFIG_PM_SLEEP */
> > +
> > +#ifdef CONFIG_PM
> > +static int mtk_uart_apdma_runtime_suspend(struct device *dev)
> > +{
> > + struct mtk_uart_apdmadev *mtkd = dev_get_drvdata(dev);
> > +
> > + clk_disable_unprepare(mtkd->clk);
> > +
> > + return 0;
> > +}
> > +
> > +static int mtk_uart_apdma_runtime_resume(struct device *dev)
> > +{
> > + int ret;
> > + struct mtk_uart_apdmadev *mtkd = dev_get_drvdata(dev);
> > +
> > + ret = clk_prepare_enable(mtkd->clk);
> > + if (ret)
> > + return ret;
> > +
> > + return 0;
> > +}
> > +#endif /* CONFIG_PM */
> > +
> > +static const struct dev_pm_ops mtk_uart_apdma_pm_ops = {
> > + SET_SYSTEM_SLEEP_PM_OPS(mtk_uart_apdma_suspend, mtk_uart_apdma_resume)
> > + SET_RUNTIME_PM_OPS(mtk_uart_apdma_runtime_suspend,
> > + mtk_uart_apdma_runtime_resume, NULL)
> > +};
>
> It probably causes a build error when CONFIG_PM is not enabled.
> and use a UNIVERSAL_DEV_PM_OPS because the runtime suspend/resume and
> system suspend/resume for the dma are
> almost the same.
>
I remember that these had test. It's build pass.
> > +
> > +static struct platform_driver mtk_uart_apdma_driver = {
> > + .probe = mtk_uart_apdma_probe,
> > + .remove = mtk_uart_apdma_remove,
> > + .driver = {
> > + .name = KBUILD_MODNAME,
> > + .pm = &mtk_uart_apdma_pm_ops,
> > + .of_match_table = of_match_ptr(mtk_uart_apdma_match),
> > + },
> > +};
> > +
> > +module_platform_driver(mtk_uart_apdma_driver);
> > +
> > +MODULE_DESCRIPTION("MediaTek UART APDMA Controller Driver");
> > +MODULE_AUTHOR("Long Cheng <long.cheng@mediatek.com>");
> > +MODULE_LICENSE("GPL v2");
> > +
> > --
> > 1.7.9.5
> >
^ permalink raw reply
* [PATCH] dmaengine: rcar-dmac: Update copyright information
From: Niklas Söderlund @ 2019-04-10 18:26 UTC (permalink / raw)
To: dmaengine; +Cc: linux-renesas-soc, Hiroyuki Yokoyama, Niklas Söderlund
From: Hiroyuki Yokoyama <hiroyuki.yokoyama.vx@renesas.com>
Update copyright and string for Gen3.
Signed-off-by: Hiroyuki Yokoyama <hiroyuki.yokoyama.vx@renesas.com>
Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
drivers/dma/sh/rcar-dmac.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/dma/sh/rcar-dmac.c b/drivers/dma/sh/rcar-dmac.c
index 2b4f256981695662..580ca1454fe8cf27 100644
--- a/drivers/dma/sh/rcar-dmac.c
+++ b/drivers/dma/sh/rcar-dmac.c
@@ -1,8 +1,8 @@
// SPDX-License-Identifier: GPL-2.0
/*
- * Renesas R-Car Gen2 DMA Controller Driver
+ * Renesas R-Car Gen2/Gen3 DMA Controller Driver
*
- * Copyright (C) 2014 Renesas Electronics Inc.
+ * Copyright (C) 2014-2019 Renesas Electronics Inc.
*
* Author: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
*/
--
2.21.0
^ permalink raw reply related
* dmaengine: rcar-dmac: Update copyright information
From: Niklas Söderlund @ 2019-04-10 18:26 UTC (permalink / raw)
To: dmaengine; +Cc: linux-renesas-soc, Hiroyuki Yokoyama, Niklas Söderlund
From: Hiroyuki Yokoyama <hiroyuki.yokoyama.vx@renesas.com>
Update copyright and string for Gen3.
Signed-off-by: Hiroyuki Yokoyama <hiroyuki.yokoyama.vx@renesas.com>
Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
drivers/dma/sh/rcar-dmac.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/dma/sh/rcar-dmac.c b/drivers/dma/sh/rcar-dmac.c
index 2b4f256981695662..580ca1454fe8cf27 100644
--- a/drivers/dma/sh/rcar-dmac.c
+++ b/drivers/dma/sh/rcar-dmac.c
@@ -1,8 +1,8 @@
// SPDX-License-Identifier: GPL-2.0
/*
- * Renesas R-Car Gen2 DMA Controller Driver
+ * Renesas R-Car Gen2/Gen3 DMA Controller Driver
*
- * Copyright (C) 2014 Renesas Electronics Inc.
+ * Copyright (C) 2014-2019 Renesas Electronics Inc.
*
* Author: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
*/
^ permalink raw reply related
* Re: [PATCH v1] dmaengine: idma64: Move driver name to the header
From: Andy Shevchenko @ 2019-04-09 18:01 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Vinod Koul, dmaengine, Lee Jones, linux-kernel
In-Reply-To: <20190409163728.GA26872@infradead.org>
On Tue, Apr 09, 2019 at 09:37:28AM -0700, Christoph Hellwig wrote:
> On Tue, Apr 09, 2019 at 06:02:19PM +0300, Andy Shevchenko wrote:
> > There are two drivers that are relying on the iDMA 64-bit driver name
> > to match. Instead of duplicating string in both of them, dedicate
> > a header file and share it between users.
>
> Hmm. Why do we have two separate drivers for the same name?
>
> Something is rather fishy there.
We would like to be sure that the DMA controller driver is loaded before its
consumers.
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply
* [v1] dmaengine: idma64: Move driver name to the header
From: Andy Shevchenko @ 2019-04-09 18:01 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Vinod Koul, dmaengine, Lee Jones, linux-kernel
On Tue, Apr 09, 2019 at 09:37:28AM -0700, Christoph Hellwig wrote:
> On Tue, Apr 09, 2019 at 06:02:19PM +0300, Andy Shevchenko wrote:
> > There are two drivers that are relying on the iDMA 64-bit driver name
> > to match. Instead of duplicating string in both of them, dedicate
> > a header file and share it between users.
>
> Hmm. Why do we have two separate drivers for the same name?
>
> Something is rather fishy there.
We would like to be sure that the DMA controller driver is loaded before its
consumers.
^ permalink raw reply
* Re: [PATCH v1] dmaengine: idma64: Move driver name to the header
From: Christoph Hellwig @ 2019-04-09 16:37 UTC (permalink / raw)
To: Andy Shevchenko; +Cc: Vinod Koul, dmaengine, Lee Jones, linux-kernel
In-Reply-To: <20190409150219.15094-1-andriy.shevchenko@linux.intel.com>
On Tue, Apr 09, 2019 at 06:02:19PM +0300, Andy Shevchenko wrote:
> There are two drivers that are relying on the iDMA 64-bit driver name
> to match. Instead of duplicating string in both of them, dedicate
> a header file and share it between users.
Hmm. Why do we have two separate drivers for the same name?
Something is rather fishy there.
^ permalink raw reply
* [v1] dmaengine: idma64: Move driver name to the header
From: Christoph Hellwig @ 2019-04-09 16:37 UTC (permalink / raw)
To: Andy Shevchenko; +Cc: Vinod Koul, dmaengine, Lee Jones, linux-kernel
On Tue, Apr 09, 2019 at 06:02:19PM +0300, Andy Shevchenko wrote:
> There are two drivers that are relying on the iDMA 64-bit driver name
> to match. Instead of duplicating string in both of them, dedicate
> a header file and share it between users.
Hmm. Why do we have two separate drivers for the same name?
Something is rather fishy there.
^ permalink raw reply
* Re: [PATCH v2] mfd: intel-lpss: Set the device in reset state when init
From: Andy Shevchenko @ 2019-04-09 15:36 UTC (permalink / raw)
To: Binbin Wu
Cc: rjw, linux-pm, gregkh, lee.jones, mika.westerberg, linux-kernel,
dmaengine
In-Reply-To: <1554710950-21212-1-git-send-email-binbin.wu@intel.com>
On Mon, Apr 08, 2019 at 04:09:10PM +0800, Binbin Wu wrote:
> In virtualized setup, when system reboots due to warm
> reset interrupt storm is seen.
>
> Call Trace:
> <IRQ>
> dump_stack+0x70/0xa5
> __report_bad_irq+0x2e/0xc0
> note_interrupt+0x248/0x290
> ? add_interrupt_randomness+0x30/0x220
> handle_irq_event_percpu+0x54/0x80
> handle_irq_event+0x39/0x60
> handle_fasteoi_irq+0x91/0x150
> handle_irq+0x108/0x180
> do_IRQ+0x52/0xf0
> common_interrupt+0xf/0xf
> </IRQ>
> RIP: 0033:0x76fc2cfabc1d
> Code: 24 28 bf 03 00 00 00 31 c0 48 8d 35 63 77 0e 00 48 8d 15 2e
> 94 0e 00 4c 89 f9 49 89 d9 4c 89 d3 e8 b8 e2 01 00 48 8b 54 24 18
> <48> 89 ef 48 89 de 4c 89 e1 e8 d5 97 01 00 84 c0 74 2d 48 8b 04
> 24
> RSP: 002b:00007ffd247c1fc0 EFLAGS: 00000293 ORIG_RAX: ffffffffffffffda
> RAX: 0000000000000000 RBX: 00007ffd247c1ff0 RCX: 000000000003d3ce
> RDX: 0000000000000000 RSI: 00007ffd247c1ff0 RDI: 000076fc2cbb6010
> RBP: 000076fc2cded010 R08: 00007ffd247c2210 R09: 00007ffd247c22a0
> R10: 000076fc29465470 R11: 0000000000000000 R12: 00007ffd247c1fc0
> R13: 000076fc2ce8e470 R14: 000076fc27ec9960 R15: 0000000000000414
> handlers:
> [<000000000d3fa913>] idma64_irq
> Disabling IRQ #27
>
> To avoid interrupt storm, set the device in reset state
> before bringing out the device from reset state.
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
(It shouldn't prevent usage of no_console_suspend hack AFAICS)
>
> Changelog v2:
> - correct the subject line by adding "mfd: "
>
> Signed-off-by: Binbin Wu <binbin.wu@intel.com>
> Acked-by: Mika Westerberg <mika.westerberg@linux.intel.com>
> ---
> drivers/mfd/intel-lpss.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/drivers/mfd/intel-lpss.c b/drivers/mfd/intel-lpss.c
> index 50bffc3..ff3fba1 100644
> --- a/drivers/mfd/intel-lpss.c
> +++ b/drivers/mfd/intel-lpss.c
> @@ -273,6 +273,9 @@ static void intel_lpss_init_dev(const struct intel_lpss *lpss)
> {
> u32 value = LPSS_PRIV_SSP_REG_DIS_DMA_FIN;
>
> + /* Set the device in reset state */
> + writel(0, lpss->priv + LPSS_PRIV_RESETS);
> +
> intel_lpss_deassert_reset(lpss);
>
> intel_lpss_set_remap_addr(lpss);
> --
> 2.7.4
>
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply
* [v2] mfd: intel-lpss: Set the device in reset state when init
From: Andy Shevchenko @ 2019-04-09 15:36 UTC (permalink / raw)
To: Binbin Wu
Cc: rjw, linux-pm, gregkh, lee.jones, mika.westerberg, linux-kernel,
dmaengine
On Mon, Apr 08, 2019 at 04:09:10PM +0800, Binbin Wu wrote:
> In virtualized setup, when system reboots due to warm
> reset interrupt storm is seen.
>
> Call Trace:
> <IRQ>
> dump_stack+0x70/0xa5
> __report_bad_irq+0x2e/0xc0
> note_interrupt+0x248/0x290
> ? add_interrupt_randomness+0x30/0x220
> handle_irq_event_percpu+0x54/0x80
> handle_irq_event+0x39/0x60
> handle_fasteoi_irq+0x91/0x150
> handle_irq+0x108/0x180
> do_IRQ+0x52/0xf0
> common_interrupt+0xf/0xf
> </IRQ>
> RIP: 0033:0x76fc2cfabc1d
> Code: 24 28 bf 03 00 00 00 31 c0 48 8d 35 63 77 0e 00 48 8d 15 2e
> 94 0e 00 4c 89 f9 49 89 d9 4c 89 d3 e8 b8 e2 01 00 48 8b 54 24 18
> <48> 89 ef 48 89 de 4c 89 e1 e8 d5 97 01 00 84 c0 74 2d 48 8b 04
> 24
> RSP: 002b:00007ffd247c1fc0 EFLAGS: 00000293 ORIG_RAX: ffffffffffffffda
> RAX: 0000000000000000 RBX: 00007ffd247c1ff0 RCX: 000000000003d3ce
> RDX: 0000000000000000 RSI: 00007ffd247c1ff0 RDI: 000076fc2cbb6010
> RBP: 000076fc2cded010 R08: 00007ffd247c2210 R09: 00007ffd247c22a0
> R10: 000076fc29465470 R11: 0000000000000000 R12: 00007ffd247c1fc0
> R13: 000076fc2ce8e470 R14: 000076fc27ec9960 R15: 0000000000000414
> handlers:
> [<000000000d3fa913>] idma64_irq
> Disabling IRQ #27
>
> To avoid interrupt storm, set the device in reset state
> before bringing out the device from reset state.
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
(It shouldn't prevent usage of no_console_suspend hack AFAICS)
>
> Changelog v2:
> - correct the subject line by adding "mfd: "
>
> Signed-off-by: Binbin Wu <binbin.wu@intel.com>
> Acked-by: Mika Westerberg <mika.westerberg@linux.intel.com>
> ---
> drivers/mfd/intel-lpss.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/drivers/mfd/intel-lpss.c b/drivers/mfd/intel-lpss.c
> index 50bffc3..ff3fba1 100644
> --- a/drivers/mfd/intel-lpss.c
> +++ b/drivers/mfd/intel-lpss.c
> @@ -273,6 +273,9 @@ static void intel_lpss_init_dev(const struct intel_lpss *lpss)
> {
> u32 value = LPSS_PRIV_SSP_REG_DIS_DMA_FIN;
>
> + /* Set the device in reset state */
> + writel(0, lpss->priv + LPSS_PRIV_RESETS);
> +
> intel_lpss_deassert_reset(lpss);
>
> intel_lpss_set_remap_addr(lpss);
> --
> 2.7.4
>
^ permalink raw reply
* [PATCH v1] dmaengine: idma64: Move driver name to the header
From: Andy Shevchenko @ 2019-04-09 15:02 UTC (permalink / raw)
To: Vinod Koul, dmaengine, Lee Jones, linux-kernel; +Cc: Andy Shevchenko
There are two drivers that are relying on the iDMA 64-bit driver name
to match. Instead of duplicating string in both of them, dedicate
a header file and share it between users.
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
---
drivers/dma/idma64.c | 9 ++++-----
drivers/mfd/intel-lpss.c | 4 ++--
include/linux/dma/idma64.h | 14 ++++++++++++++
3 files changed, 20 insertions(+), 7 deletions(-)
create mode 100644 include/linux/dma/idma64.h
diff --git a/drivers/dma/idma64.c b/drivers/dma/idma64.c
index 0baf9797cc09..4180cf197823 100644
--- a/drivers/dma/idma64.c
+++ b/drivers/dma/idma64.c
@@ -19,10 +19,9 @@
#include <linux/platform_device.h>
#include <linux/slab.h>
-#include "idma64.h"
+#include <linux/dma/idma64.h>
-/* Platform driver name */
-#define DRV_NAME "idma64"
+#include "idma64.h"
/* For now we support only two channels */
#define IDMA64_NR_CHAN 2
@@ -697,7 +696,7 @@ static struct platform_driver idma64_platform_driver = {
.probe = idma64_platform_probe,
.remove = idma64_platform_remove,
.driver = {
- .name = DRV_NAME,
+ .name = LPSS_IDMA64_DRIVER_NAME,
.pm = &idma64_dev_pm_ops,
},
};
@@ -707,4 +706,4 @@ module_platform_driver(idma64_platform_driver);
MODULE_LICENSE("GPL v2");
MODULE_DESCRIPTION("iDMA64 core driver");
MODULE_AUTHOR("Andy Shevchenko <andriy.shevchenko@linux.intel.com>");
-MODULE_ALIAS("platform:" DRV_NAME);
+MODULE_ALIAS("platform:" LPSS_IDMA64_DRIVER_NAME);
diff --git a/drivers/mfd/intel-lpss.c b/drivers/mfd/intel-lpss.c
index 50bffc3382d7..45221e092ecf 100644
--- a/drivers/mfd/intel-lpss.c
+++ b/drivers/mfd/intel-lpss.c
@@ -28,6 +28,8 @@
#include <linux/seq_file.h>
#include <linux/io-64-nonatomic-lo-hi.h>
+#include <linux/dma/idma64.h>
+
#include "intel-lpss.h"
#define LPSS_DEV_OFFSET 0x000
@@ -96,8 +98,6 @@ static const struct resource intel_lpss_idma64_resources[] = {
DEFINE_RES_IRQ(0),
};
-#define LPSS_IDMA64_DRIVER_NAME "idma64"
-
/*
* Cells needs to be ordered so that the iDMA is created first. This is
* because we need to be sure the DMA is available when the host controller
diff --git a/include/linux/dma/idma64.h b/include/linux/dma/idma64.h
new file mode 100644
index 000000000000..621cfae60554
--- /dev/null
+++ b/include/linux/dma/idma64.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Definitions for the Intel integrated DMA 64-bit
+ *
+ * Copyright (C) 2019 Intel Corporation
+ */
+
+#ifndef __LINUX_DMA_IDMA64_H__
+#define __LINUX_DMA_IDMA64_H__
+
+/* Platform driver name */
+#define LPSS_IDMA64_DRIVER_NAME "idma64"
+
+#endif /* __LINUX_DMA_IDMA64_H__ */
--
2.20.1
^ permalink raw reply related
* [v1] dmaengine: idma64: Move driver name to the header
From: Andy Shevchenko @ 2019-04-09 15:02 UTC (permalink / raw)
To: Vinod Koul, dmaengine, Lee Jones, linux-kernel; +Cc: Andy Shevchenko
There are two drivers that are relying on the iDMA 64-bit driver name
to match. Instead of duplicating string in both of them, dedicate
a header file and share it between users.
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
---
drivers/dma/idma64.c | 9 ++++-----
drivers/mfd/intel-lpss.c | 4 ++--
include/linux/dma/idma64.h | 14 ++++++++++++++
3 files changed, 20 insertions(+), 7 deletions(-)
create mode 100644 include/linux/dma/idma64.h
diff --git a/drivers/dma/idma64.c b/drivers/dma/idma64.c
index 0baf9797cc09..4180cf197823 100644
--- a/drivers/dma/idma64.c
+++ b/drivers/dma/idma64.c
@@ -19,10 +19,9 @@
#include <linux/platform_device.h>
#include <linux/slab.h>
-#include "idma64.h"
+#include <linux/dma/idma64.h>
-/* Platform driver name */
-#define DRV_NAME "idma64"
+#include "idma64.h"
/* For now we support only two channels */
#define IDMA64_NR_CHAN 2
@@ -697,7 +696,7 @@ static struct platform_driver idma64_platform_driver = {
.probe = idma64_platform_probe,
.remove = idma64_platform_remove,
.driver = {
- .name = DRV_NAME,
+ .name = LPSS_IDMA64_DRIVER_NAME,
.pm = &idma64_dev_pm_ops,
},
};
@@ -707,4 +706,4 @@ module_platform_driver(idma64_platform_driver);
MODULE_LICENSE("GPL v2");
MODULE_DESCRIPTION("iDMA64 core driver");
MODULE_AUTHOR("Andy Shevchenko <andriy.shevchenko@linux.intel.com>");
-MODULE_ALIAS("platform:" DRV_NAME);
+MODULE_ALIAS("platform:" LPSS_IDMA64_DRIVER_NAME);
diff --git a/drivers/mfd/intel-lpss.c b/drivers/mfd/intel-lpss.c
index 50bffc3382d7..45221e092ecf 100644
--- a/drivers/mfd/intel-lpss.c
+++ b/drivers/mfd/intel-lpss.c
@@ -28,6 +28,8 @@
#include <linux/seq_file.h>
#include <linux/io-64-nonatomic-lo-hi.h>
+#include <linux/dma/idma64.h>
+
#include "intel-lpss.h"
#define LPSS_DEV_OFFSET 0x000
@@ -96,8 +98,6 @@ static const struct resource intel_lpss_idma64_resources[] = {
DEFINE_RES_IRQ(0),
};
-#define LPSS_IDMA64_DRIVER_NAME "idma64"
-
/*
* Cells needs to be ordered so that the iDMA is created first. This is
* because we need to be sure the DMA is available when the host controller
diff --git a/include/linux/dma/idma64.h b/include/linux/dma/idma64.h
new file mode 100644
index 000000000000..621cfae60554
--- /dev/null
+++ b/include/linux/dma/idma64.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Definitions for the Intel integrated DMA 64-bit
+ *
+ * Copyright (C) 2019 Intel Corporation
+ */
+
+#ifndef __LINUX_DMA_IDMA64_H__
+#define __LINUX_DMA_IDMA64_H__
+
+/* Platform driver name */
+#define LPSS_IDMA64_DRIVER_NAME "idma64"
+
+#endif /* __LINUX_DMA_IDMA64_H__ */
^ permalink raw reply related
* Re: [PATCH v11 1/4] dmaengine: 8250_mtk_dma: add MediaTek uart DMA support
From: Long Cheng @ 2019-04-09 9:42 UTC (permalink / raw)
To: Nicolas Boichat
Cc: Vinod Koul, Randy Dunlap, Rob Herring, Mark Rutland, Ryder Lee,
Sean Wang, Matthias Brugger, Dan Williams, Greg Kroah-Hartman,
Jiri Slaby, Sean Wang, dmaengine, devicetree,
linux-arm Mailing List, moderated list:ARM/Mediatek SoC support,
lkml, linux-serial, srv_heupstream, Yingjoe Chen, YT Shen,
Zhenbao Liu
In-Reply-To: <CANMq1KBvr9cTBdjX9UD83QuHYb7c9Le6299w8yTBw+-b9pz6ew@mail.gmail.com>
On Sun, 2019-03-10 at 19:15 +0800, Nicolas Boichat wrote:
> On Thu, Mar 7, 2019 at 9:45 AM Long Cheng <long.cheng@mediatek.com> wrote:
> >
> > In DMA engine framework, add 8250 uart dma to support MediaTek uart.
> > If MediaTek uart enabled(SERIAL_8250_MT6577), and want to improve
> > the performance, can enable the function.
> >
> > Signed-off-by: Long Cheng <long.cheng@mediatek.com>
> > ---
> > drivers/dma/mediatek/Kconfig | 11 +
> > drivers/dma/mediatek/Makefile | 1 +
> > drivers/dma/mediatek/mtk-uart-apdma.c | 660 +++++++++++++++++++++++++++++++++
> > 3 files changed, 672 insertions(+)
> > create mode 100644 drivers/dma/mediatek/mtk-uart-apdma.c
> >
> > diff --git a/drivers/dma/mediatek/Kconfig b/drivers/dma/mediatek/Kconfig
> > index 680fc05..ac49eb6 100644
> > --- a/drivers/dma/mediatek/Kconfig
> > +++ b/drivers/dma/mediatek/Kconfig
> > @@ -24,3 +24,14 @@ config MTK_CQDMA
> >
> > This controller provides the channels which is dedicated to
> > memory-to-memory transfer to offload from CPU.
> > +
> > +config MTK_UART_APDMA
> > + tristate "MediaTek SoCs APDMA support for UART"
> > + depends on OF && SERIAL_8250_MT6577
> > + select DMA_ENGINE
> > + select DMA_VIRTUAL_CHANNELS
> > + help
> > + Support for the UART DMA engine found on MediaTek MTK SoCs.
> > + When SERIAL_8250_MT6577 is enabled, and if you want to use DMA,
> > + you can enable the config. The DMA engine can only be used
> > + with MediaTek SoCs.
> > diff --git a/drivers/dma/mediatek/Makefile b/drivers/dma/mediatek/Makefile
> > index 41bb381..61a6d29 100644
> > --- a/drivers/dma/mediatek/Makefile
> > +++ b/drivers/dma/mediatek/Makefile
> > @@ -1,2 +1,3 @@
> > +obj-$(CONFIG_MTK_UART_APDMA) += mtk-uart-apdma.o
> > obj-$(CONFIG_MTK_HSDMA) += mtk-hsdma.o
> > obj-$(CONFIG_MTK_CQDMA) += mtk-cqdma.o
> > diff --git a/drivers/dma/mediatek/mtk-uart-apdma.c b/drivers/dma/mediatek/mtk-uart-apdma.c
> > new file mode 100644
> > index 0000000..9ed7a49
> > --- /dev/null
> > +++ b/drivers/dma/mediatek/mtk-uart-apdma.c
> > @@ -0,0 +1,660 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * MediaTek Uart APDMA driver.
> > + *
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Long Cheng <long.cheng@mediatek.com>
> > + */
> > +
> > +#include <linux/clk.h>
> > +#include <linux/dmaengine.h>
> > +#include <linux/dma-mapping.h>
> > +#include <linux/err.h>
> > +#include <linux/init.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/iopoll.h>
> > +#include <linux/kernel.h>
> > +#include <linux/list.h>
> > +#include <linux/module.h>
> > +#include <linux/of_device.h>
> > +#include <linux/of_dma.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/slab.h>
> > +#include <linux/spinlock.h>
> > +
> > +#include "../virt-dma.h"
> > +
> > +/* The default number of virtual channel */
> > +#define MTK_UART_APDMA_NR_VCHANS 8
> > +
> > +#define VFF_EN_B BIT(0)
> > +#define VFF_STOP_B BIT(0)
> > +#define VFF_FLUSH_B BIT(0)
> > +#define VFF_4G_SUPPORT_B BIT(0)
> > +#define VFF_RX_INT_EN0_B BIT(0) /* rx valid size >= vff thre */
> > +#define VFF_RX_INT_EN1_B BIT(1)
> > +#define VFF_TX_INT_EN_B BIT(0) /* tx left size >= vff thre */
> > +#define VFF_WARM_RST_B BIT(0)
> > +#define VFF_RX_INT_CLR_B (BIT(0) | BIT(1))
> > +#define VFF_TX_INT_CLR_B 0
> > +#define VFF_STOP_CLR_B 0
> > +#define VFF_INT_EN_CLR_B 0
> > +#define VFF_4G_SUPPORT_CLR_B 0
> > +
> > +/* interrupt trigger level for tx */
> > +#define VFF_TX_THRE(n) ((n) * 7 / 8)
> > +/* interrupt trigger level for rx */
> > +#define VFF_RX_THRE(n) ((n) * 3 / 4)
> > +
> > +#define VFF_RING_SIZE 0xffffU
>
> Drop the U, it's not very useful (there are a a few more below, grep
> for [0-9a-f]U).
>
OK, i will fix these.
> > +/* invert this bit when wrap ring head again */
> > +#define VFF_RING_WRAP 0x10000U
> > +
> > +#define VFF_INT_FLAG 0x00
> > +#define VFF_INT_EN 0x04
> > +#define VFF_EN 0x08
> > +#define VFF_RST 0x0c
> > +#define VFF_STOP 0x10
> > +#define VFF_FLUSH 0x14
> > +#define VFF_ADDR 0x1c
> > +#define VFF_LEN 0x24
> > +#define VFF_THRE 0x28
> > +#define VFF_WPT 0x2c
> > +#define VFF_RPT 0x30
> > +/* TX: the buffer size HW can read. RX: the buffer size SW can read. */
> > +#define VFF_VALID_SIZE 0x3c
> > +/* TX: the buffer size SW can write. RX: the buffer size HW can write. */
> > +#define VFF_LEFT_SIZE 0x40
> > +#define VFF_DEBUG_STATUS 0x50
> > +#define VFF_4G_SUPPORT 0x54
> > +
> > +struct mtk_uart_apdmadev {
> > + struct dma_device ddev;
> > + struct clk *clk;
> > + bool support_33bits;
> > + unsigned int dma_requests;
> > + unsigned int *dma_irq;
> > +};
> > +
> > +struct mtk_uart_apdma_desc {
> > + struct virt_dma_desc vd;
> > +
> > + unsigned int avail_len;
> > +};
> > +
> > +struct mtk_chan {
> > + struct virt_dma_chan vc;
> > + struct dma_slave_config cfg;
> > + void __iomem *base;
> > + struct mtk_uart_apdma_desc *desc;
> > +
> > + enum dma_transfer_direction dir;
> > +
> > + bool requested;
> > +
> > + unsigned int rx_status;
> > +};
> > +
> > +static inline struct mtk_uart_apdmadev *
> > +to_mtk_uart_apdma_dev(struct dma_device *d)
> > +{
> > + return container_of(d, struct mtk_uart_apdmadev, ddev);
> > +}
> > +
> > +static inline struct mtk_chan *to_mtk_uart_apdma_chan(struct dma_chan *c)
> > +{
> > + return container_of(c, struct mtk_chan, vc.chan);
> > +}
> > +
> > +static inline struct mtk_uart_apdma_desc *to_mtk_uart_apdma_desc
> > + (struct dma_async_tx_descriptor *t)
> > +{
> > + return container_of(t, struct mtk_uart_apdma_desc, vd.tx);
> > +}
> > +
> > +static void mtk_uart_apdma_write(struct mtk_chan *c,
> > + unsigned int reg, unsigned int val)
> > +{
> > + writel(val, c->base + reg);
> > +}
> > +
> > +static unsigned int mtk_uart_apdma_read(struct mtk_chan *c, unsigned int reg)
> > +{
> > + return readl(c->base + reg);
> > +}
> > +
> > +static void mtk_uart_apdma_desc_free(struct virt_dma_desc *vd)
> > +{
> > + struct dma_chan *chan = vd->tx.chan;
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > +
> > + kfree(c->desc);
> > +}
> > +
> > +static void mtk_uart_apdma_start_tx(struct mtk_chan *c)
> > +{
> > + unsigned int len, send, left, wpt, d_wpt, tmp;
> > + int ret;
> > +
> > + left = mtk_uart_apdma_read(c, VFF_LEFT_SIZE);
> > + if (!left) {
> > + mtk_uart_apdma_write(c, VFF_INT_EN, VFF_TX_INT_EN_B);
> > + return;
> > + }
> > +
> > + /* Wait 1sec for flush, can't sleep */
> > + ret = readx_poll_timeout(readl, c->base + VFF_FLUSH, tmp,
> > + tmp != VFF_FLUSH_B, 0, 1000000);
> > + if (ret)
> > + dev_warn(c->vc.chan.device->dev, "tx: fail, debug=0x%x\n",
> > + mtk_uart_apdma_read(c, VFF_DEBUG_STATUS));
> > +
> > + send = min_t(unsigned int, left, c->desc->avail_len);
> > + wpt = mtk_uart_apdma_read(c, VFF_WPT);
> > + len = c->cfg.dst_port_window_size;
> > +
> > + d_wpt = wpt + send;
> > + if ((d_wpt & VFF_RING_SIZE) >= len) {
> > + d_wpt = d_wpt - len;
> > + d_wpt = d_wpt ^ VFF_RING_WRAP;
> > + }
> > + mtk_uart_apdma_write(c, VFF_WPT, d_wpt);
> > +
> > + c->desc->avail_len -= send;
> > +
> > + mtk_uart_apdma_write(c, VFF_INT_EN, VFF_TX_INT_EN_B);
> > + if (mtk_uart_apdma_read(c, VFF_FLUSH) == 0U)
> > + mtk_uart_apdma_write(c, VFF_FLUSH, VFF_FLUSH_B);
> > +}
> > +
> > +static void mtk_uart_apdma_start_rx(struct mtk_chan *c)
> > +{
> > + struct mtk_uart_apdma_desc *d = c->desc;
> > + unsigned int len, wg, rg;
> > + int cnt;
> > +
> > + if ((mtk_uart_apdma_read(c, VFF_VALID_SIZE) == 0U) ||
> > + !d || !vchan_next_desc(&c->vc))
> > + return;
> > +
> > + len = c->cfg.src_port_window_size;
> > + rg = mtk_uart_apdma_read(c, VFF_RPT);
> > + wg = mtk_uart_apdma_read(c, VFF_WPT);
> > + cnt = (wg & VFF_RING_SIZE) - (rg & VFF_RING_SIZE);
> > + /*
> > + * The buffer is ring buffer. If wrap bit different,
> > + * represents the start of the next cycle for WPT
> > + */
> > + if ((rg ^ wg) & VFF_RING_WRAP)
> > + cnt += len;
> > +
> > + c->rx_status = d->avail_len - cnt;
> > + mtk_uart_apdma_write(c, VFF_RPT, wg);
> > +
> > + list_del(&d->vd.node);
> > + vchan_cookie_complete(&d->vd);
> > +}
> > +
> > +static irqreturn_t mtk_uart_apdma_irq_handler(int irq, void *dev_id)
> > +{
> > + struct dma_chan *chan = (struct dma_chan *)dev_id;
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + struct mtk_uart_apdma_desc *d;
> > + unsigned long flags;
> > +
> > + spin_lock_irqsave(&c->vc.lock, flags);
> > + if (c->dir == DMA_DEV_TO_MEM) {
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_RX_INT_CLR_B);
> > + mtk_uart_apdma_start_rx(c);
> > + } else if (c->dir == DMA_MEM_TO_DEV) {
> > + d = c->desc;
> > +
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_TX_INT_CLR_B);
> > +
> > + if (d->avail_len != 0U) {
> > + mtk_uart_apdma_start_tx(c);
> > + } else {
> > + list_del(&d->vd.node);
> > + vchan_cookie_complete(&d->vd);
> > + }
> > + }
> > + spin_unlock_irqrestore(&c->vc.lock, flags);
> > +
> > + return IRQ_HANDLED;
> > +}
> > +
> > +static int mtk_uart_apdma_alloc_chan_resources(struct dma_chan *chan)
> > +{
> > + struct mtk_uart_apdmadev *mtkd = to_mtk_uart_apdma_dev(chan->device);
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + unsigned int tmp;
> > + int ret;
> > +
> > + pm_runtime_get_sync(mtkd->ddev.dev);
> > +
> > + mtk_uart_apdma_write(c, VFF_ADDR, 0);
> > + mtk_uart_apdma_write(c, VFF_THRE, 0);
> > + mtk_uart_apdma_write(c, VFF_LEN, 0);
> > + mtk_uart_apdma_write(c, VFF_RST, VFF_WARM_RST_B);
> > +
> > + ret = readx_poll_timeout(readl, c->base + VFF_EN, tmp, !tmp, 10, 100);
> > + if (ret) {
> > + dev_err(chan->device->dev, "dma reset: fail, timeout\n");
> > + return ret;
> > + }
> > +
> > + if (!c->requested) {
> > + c->requested = true;
> > + ret = request_irq(mtkd->dma_irq[chan->chan_id],
> > + mtk_uart_apdma_irq_handler, IRQF_TRIGGER_NONE,
> > + KBUILD_MODNAME, chan);
> > + if (ret < 0) {
> > + dev_err(chan->device->dev, "Can't request dma IRQ\n");
> > + return -EINVAL;
> > + }
> > + }
> > +
> > + if (mtkd->support_33bits)
> > + mtk_uart_apdma_write(c, VFF_4G_SUPPORT, VFF_4G_SUPPORT_CLR_B);
> > +
> > + return ret;
> > +}
> > +
> > +static void mtk_uart_apdma_free_chan_resources(struct dma_chan *chan)
> > +{
> > + struct mtk_uart_apdmadev *mtkd = to_mtk_uart_apdma_dev(chan->device);
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > +
> > + if (c->requested) {
> > + c->requested = false;
> > + free_irq(mtkd->dma_irq[chan->chan_id], chan);
> > + }
> > +
> > + tasklet_kill(&c->vc.task);
> > +
> > + vchan_free_chan_resources(&c->vc);
> > +
> > + pm_runtime_put_sync(mtkd->ddev.dev);
> > +}
> > +
> > +static enum dma_status mtk_uart_apdma_tx_status(struct dma_chan *chan,
> > + dma_cookie_t cookie,
> > + struct dma_tx_state *txstate)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + enum dma_status ret;
> > +
> > + ret = dma_cookie_status(chan, cookie, txstate);
> > +
> > + dma_set_residue(txstate, c->rx_status);
> > +
> > + return ret;
> > +}
> > +
> > +static void mtk_uart_apdma_config_write(struct dma_chan *chan,
> > + struct dma_slave_config *cfg,
> > + enum dma_transfer_direction dir)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + struct mtk_uart_apdmadev *mtkd =
> > + to_mtk_uart_apdma_dev(c->vc.chan.device);
> > + unsigned int tmp;
> > +
> > + if (mtk_uart_apdma_read(c, VFF_EN) == VFF_EN_B)
> > + return;
> > +
> > + c->dir = dir;
> > +
> > + if (dir == DMA_DEV_TO_MEM) {
> > + tmp = cfg->src_port_window_size;
> > +
> > + mtk_uart_apdma_write(c, VFF_ADDR, cfg->src_addr);
> > + mtk_uart_apdma_write(c, VFF_LEN, tmp);
> > + mtk_uart_apdma_write(c, VFF_THRE, VFF_RX_THRE(tmp));
> > + mtk_uart_apdma_write(c, VFF_INT_EN,
> > + VFF_RX_INT_EN0_B | VFF_RX_INT_EN1_B);
> > + mtk_uart_apdma_write(c, VFF_RPT, 0);
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_RX_INT_CLR_B);
> > + } else if (dir == DMA_MEM_TO_DEV) {
> > + tmp = cfg->dst_port_window_size;
> > +
> > + mtk_uart_apdma_write(c, VFF_ADDR, cfg->dst_addr);
> > + mtk_uart_apdma_write(c, VFF_LEN, tmp);
> > + mtk_uart_apdma_write(c, VFF_THRE, VFF_TX_THRE(tmp));
> > + mtk_uart_apdma_write(c, VFF_WPT, 0);
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_TX_INT_CLR_B);
> > + }
> > +
> > + mtk_uart_apdma_write(c, VFF_EN, VFF_EN_B);
> > +
> > + if (mtkd->support_33bits)
> > + mtk_uart_apdma_write(c, VFF_4G_SUPPORT, VFF_4G_SUPPORT_B);
> > +
> > + if (mtk_uart_apdma_read(c, VFF_EN) != VFF_EN_B)
> > + dev_err(chan->device->dev, "dir[%d] fail\n", dir);
> > +}
> > +
> > +/*
> > + * dmaengine_prep_slave_single will call the function. and sglen is 1.
> > + * 8250 uart using one ring buffer, and deal with one sg.
> > + */
> > +static struct dma_async_tx_descriptor *mtk_uart_apdma_prep_slave_sg
> > + (struct dma_chan *chan, struct scatterlist *sgl,
> > + unsigned int sglen, enum dma_transfer_direction dir,
> > + unsigned long tx_flags, void *context)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + struct mtk_uart_apdma_desc *d;
> > +
> > + if (!is_slave_direction(dir))
> > + return NULL;
> > +
> > + mtk_uart_apdma_config_write(chan, &c->cfg, dir);
> > +
> > + /* Now allocate and setup the descriptor */
> > + d = kzalloc(sizeof(*d), GFP_ATOMIC);
> > + if (!d)
> > + return NULL;
> > +
> > + /* sglen is 1 */
> > + d->avail_len = sg_dma_len(sgl);
> > + c->rx_status = d->avail_len;
> > +
> > + return vchan_tx_prep(&c->vc, &d->vd, tx_flags);
> > +}
> > +
> > +static void mtk_uart_apdma_issue_pending(struct dma_chan *chan)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + struct virt_dma_desc *vd;
> > + unsigned long flags;
> > +
> > + spin_lock_irqsave(&c->vc.lock, flags);
> > + if (vchan_issue_pending(&c->vc)) {
> > + vd = vchan_next_desc(&c->vc);
> > + c->desc = to_mtk_uart_apdma_desc(&vd->tx);
> > + }
> > +
> > + if (c->dir == DMA_DEV_TO_MEM)
> > + mtk_uart_apdma_start_rx(c);
> > + else if (c->dir == DMA_MEM_TO_DEV)
> > + mtk_uart_apdma_start_tx(c);
> > +
> > + spin_unlock_irqrestore(&c->vc.lock, flags);
> > +}
> > +
> > +static int mtk_uart_apdma_slave_config(struct dma_chan *chan,
> > + struct dma_slave_config *config)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > +
> > + memcpy(&c->cfg, config, sizeof(*config));
> > +
> > + return 0;
> > +}
> > +
> > +static int mtk_uart_apdma_terminate_all(struct dma_chan *chan)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + unsigned long flags;
> > + unsigned int tmp;
> > + int ret;
> > +
> > + spin_lock_irqsave(&c->vc.lock, flags);
> > +
> > + mtk_uart_apdma_write(c, VFF_FLUSH, VFF_FLUSH_B);
> > + /* Wait 1sec for flush, can't sleep */
> > + ret = readx_poll_timeout(readl, c->base + VFF_FLUSH, tmp,
> > + tmp != VFF_FLUSH_B, 0, 1000000);
> > + if (ret)
> > + dev_err(c->vc.chan.device->dev, "flush: fail, debug=0x%x\n",
> > + mtk_uart_apdma_read(c, VFF_DEBUG_STATUS));
> > +
> > + /* set stop as 1 -> wait until en is 0 -> set stop as 0 */
> > + mtk_uart_apdma_write(c, VFF_STOP, VFF_STOP_B);
> > + ret = readx_poll_timeout(readl, c->base + VFF_EN, tmp, !tmp, 10, 100);
> > + if (ret)
> > + dev_err(c->vc.chan.device->dev, "stop: fail, debug=0x%x\n",
> > + mtk_uart_apdma_read(c, VFF_DEBUG_STATUS));
> > +
> > + mtk_uart_apdma_write(c, VFF_STOP, VFF_STOP_CLR_B);
> > + mtk_uart_apdma_write(c, VFF_INT_EN, VFF_INT_EN_CLR_B);
> > +
> > + if (c->dir == DMA_DEV_TO_MEM)
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_RX_INT_CLR_B);
> > + else if (c->dir == DMA_MEM_TO_DEV)
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_TX_INT_CLR_B);
> > +
> > + spin_unlock_irqrestore(&c->vc.lock, flags);
> > +
> > + return 0;
> > +}
> > +
> > +static int mtk_uart_apdma_device_pause(struct dma_chan *chan)
> > +{
> > + /* just for check caps pass */
> > + dev_err(chan->device->dev, "Pause can't support\n");
> > +
> > + return 0;
> > +}
>
> I think we've said repeatedly that leaving this stub function is incorrect.
>
I will try to implementation it. Thanks
> > +
> > +static void mtk_uart_apdma_free(struct mtk_uart_apdmadev *mtkd)
> > +{
> > + while (!list_empty(&mtkd->ddev.channels)) {
> > + struct mtk_chan *c = list_first_entry(&mtkd->ddev.channels,
> > + struct mtk_chan, vc.chan.device_node);
> > +
> > + list_del(&c->vc.chan.device_node);
> > + tasklet_kill(&c->vc.task);
> > + }
> > +}
> > +
> > +static const struct of_device_id mtk_uart_apdma_match[] = {
> > + { .compatible = "mediatek,mt6577-uart-dma", },
> > + { /* sentinel */ },
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_uart_apdma_match);
> > +
> > +static int mtk_uart_apdma_probe(struct platform_device *pdev)
> > +{
> > + struct device_node *np = pdev->dev.of_node;
> > + struct mtk_uart_apdmadev *mtkd;
> > + struct resource *res;
> > + struct mtk_chan *c;
> > + int bit_mask = 32, rc;
> > + unsigned int i;
> > +
> > + mtkd = devm_kzalloc(&pdev->dev, sizeof(*mtkd), GFP_KERNEL);
> > + if (!mtkd)
> > + return -ENOMEM;
> > +
> > + mtkd->clk = devm_clk_get(&pdev->dev, NULL);
> > + if (IS_ERR(mtkd->clk)) {
> > + dev_err(&pdev->dev, "No clock specified\n");
> > + rc = PTR_ERR(mtkd->clk);
> > + return rc;
> > + }
> > +
> > + if (of_property_read_bool(np, "mediatek,dma-33bits"))
> > + mtkd->support_33bits = true;
> > +
> > + if (mtkd->support_33bits)
> > + bit_mask = 33;
> > +
> > + rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(bit_mask));
> > + if (rc)
> > + return rc;
> > +
> > + dma_cap_set(DMA_SLAVE, mtkd->ddev.cap_mask);
> > + mtkd->ddev.device_alloc_chan_resources =
> > + mtk_uart_apdma_alloc_chan_resources;
> > + mtkd->ddev.device_free_chan_resources =
> > + mtk_uart_apdma_free_chan_resources;
> > + mtkd->ddev.device_tx_status = mtk_uart_apdma_tx_status;
> > + mtkd->ddev.device_issue_pending = mtk_uart_apdma_issue_pending;
> > + mtkd->ddev.device_prep_slave_sg = mtk_uart_apdma_prep_slave_sg;
> > + mtkd->ddev.device_config = mtk_uart_apdma_slave_config;
> > + mtkd->ddev.device_pause = mtk_uart_apdma_device_pause;
> > + mtkd->ddev.device_terminate_all = mtk_uart_apdma_terminate_all;
> > + mtkd->ddev.src_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE);
> > + mtkd->ddev.dst_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE);
> > + mtkd->ddev.directions = BIT(DMA_DEV_TO_MEM) | BIT(DMA_MEM_TO_DEV);
> > + mtkd->ddev.residue_granularity = DMA_RESIDUE_GRANULARITY_SEGMENT;
> > + mtkd->ddev.dev = &pdev->dev;
> > + INIT_LIST_HEAD(&mtkd->ddev.channels);
> > +
> > + mtkd->dma_requests = MTK_UART_APDMA_NR_VCHANS;
> > + if (of_property_read_u32(np, "dma-requests", &mtkd->dma_requests)) {
> > + dev_info(&pdev->dev,
> > + "Using %u as missing dma-requests property\n",
> > + MTK_UART_APDMA_NR_VCHANS);
> > + }
> > +
> > + mtkd->dma_irq = devm_kcalloc(&pdev->dev, mtkd->dma_requests,
> > + sizeof(*mtkd->dma_irq), GFP_KERNEL);
> > + if (!mtkd->dma_irq)
> > + return -ENOMEM;
> > +
> > + for (i = 0; i < mtkd->dma_requests; i++) {
> > + c = devm_kzalloc(mtkd->ddev.dev, sizeof(*c), GFP_KERNEL);
> > + if (!c) {
> > + rc = -ENODEV;
> > + goto err_no_dma;
> > + }
> > +
> > + res = platform_get_resource(pdev, IORESOURCE_MEM, i);
> > + if (!res) {
> > + rc = -ENODEV;
> > + goto err_no_dma;
> > + }
> > +
> > + c->base = devm_ioremap_resource(&pdev->dev, res);
> > + if (IS_ERR(c->base)) {
> > + rc = PTR_ERR(c->base);
> > + goto err_no_dma;
> > + }
> > + c->requested = false;
> > + c->vc.desc_free = mtk_uart_apdma_desc_free;
> > + vchan_init(&c->vc, &mtkd->ddev);
> > +
> > + mtkd->dma_irq[i] = platform_get_irq(pdev, i);
> > + if ((int)mtkd->dma_irq[i] < 0) {
> > + dev_err(&pdev->dev, "failed to get IRQ[%d]\n", i);
> > + rc = -EINVAL;
> > + goto err_no_dma;
> > + }
> > + }
> > +
> > + pm_runtime_enable(&pdev->dev);
> > + pm_runtime_set_active(&pdev->dev);
> > +
> > + rc = dma_async_device_register(&mtkd->ddev);
> > + if (rc)
> > + goto rpm_disable;
> > +
> > + platform_set_drvdata(pdev, mtkd);
> > +
> > + /* Device-tree DMA controller registration */
> > + rc = of_dma_controller_register(np, of_dma_xlate_by_chan_id, mtkd);
> > + if (rc)
> > + goto dma_remove;
> > +
> > + return rc;
> > +
> > +dma_remove:
> > + dma_async_device_unregister(&mtkd->ddev);
> > +rpm_disable:
> > + pm_runtime_disable(&pdev->dev);
> > +err_no_dma:
> > + mtk_uart_apdma_free(mtkd);
> > + return rc;
> > +}
> > +
> > +static int mtk_uart_apdma_remove(struct platform_device *pdev)
> > +{
> > + struct mtk_uart_apdmadev *mtkd = platform_get_drvdata(pdev);
> > +
> > + if (pdev->dev.of_node)
> > + of_dma_controller_free(pdev->dev.of_node);
> > +
> > + pm_runtime_disable(&pdev->dev);
> > + pm_runtime_put_noidle(&pdev->dev);
> > +
> > + dma_async_device_unregister(&mtkd->ddev);
> > + mtk_uart_apdma_free(mtkd);
> > +
> > + return 0;
> > +}
> > +
> > +#ifdef CONFIG_PM_SLEEP
> > +static int mtk_uart_apdma_suspend(struct device *dev)
> > +{
> > + struct mtk_uart_apdmadev *mtkd = dev_get_drvdata(dev);
> > +
> > + if (!pm_runtime_suspended(dev))
> > + clk_disable_unprepare(mtkd->clk);
> > +
> > + return 0;
> > +}
> > +
> > +static int mtk_uart_apdma_resume(struct device *dev)
> > +{
> > + int ret;
> > + struct mtk_uart_apdmadev *mtkd = dev_get_drvdata(dev);
> > +
> > + if (!pm_runtime_suspended(dev)) {
> > + ret = clk_prepare_enable(mtkd->clk);
> > + if (ret)
> > + return ret;
> > + }
> > +
> > + return 0;
> > +}
> > +#endif /* CONFIG_PM_SLEEP */
> > +
> > +#ifdef CONFIG_PM
> > +static int mtk_uart_apdma_runtime_suspend(struct device *dev)
> > +{
> > + struct mtk_uart_apdmadev *mtkd = dev_get_drvdata(dev);
> > +
> > + clk_disable_unprepare(mtkd->clk);
> > +
> > + return 0;
> > +}
> > +
> > +static int mtk_uart_apdma_runtime_resume(struct device *dev)
> > +{
> > + int ret;
> > + struct mtk_uart_apdmadev *mtkd = dev_get_drvdata(dev);
> > +
> > + ret = clk_prepare_enable(mtkd->clk);
> > + if (ret)
> > + return ret;
> > +
> > + return 0;
> > +}
> > +#endif /* CONFIG_PM */
> > +
> > +static const struct dev_pm_ops mtk_uart_apdma_pm_ops = {
> > + SET_SYSTEM_SLEEP_PM_OPS(mtk_uart_apdma_suspend, mtk_uart_apdma_resume)
> > + SET_RUNTIME_PM_OPS(mtk_uart_apdma_runtime_suspend,
> > + mtk_uart_apdma_runtime_resume, NULL)
> > +};
> > +
> > +static struct platform_driver mtk_uart_apdma_driver = {
> > + .probe = mtk_uart_apdma_probe,
> > + .remove = mtk_uart_apdma_remove,
> > + .driver = {
> > + .name = KBUILD_MODNAME,
> > + .pm = &mtk_uart_apdma_pm_ops,
> > + .of_match_table = of_match_ptr(mtk_uart_apdma_match),
> > + },
> > +};
> > +
> > +module_platform_driver(mtk_uart_apdma_driver);
> > +
> > +MODULE_DESCRIPTION("MediaTek UART APDMA Controller Driver");
> > +MODULE_AUTHOR("Long Cheng <long.cheng@mediatek.com>");
> > +MODULE_LICENSE("GPL v2");
> > +
> > --
> > 1.7.9.5
> >
^ permalink raw reply
* [v11,1/4] dmaengine: 8250_mtk_dma: add MediaTek uart DMA support
From: Long Cheng @ 2019-04-09 9:42 UTC (permalink / raw)
To: Nicolas Boichat
Cc: Vinod Koul, Randy Dunlap, Rob Herring, Mark Rutland, Ryder Lee,
Sean Wang, Matthias Brugger, Dan Williams, Greg Kroah-Hartman,
Jiri Slaby, Sean Wang, dmaengine, devicetree,
linux-arm Mailing List, moderated list:ARM/Mediatek SoC support,
lkml, linux-serial, srv_heupstream, Yingjoe Chen, YT Shen,
Zhenbao Liu
On Sun, 2019-03-10 at 19:15 +0800, Nicolas Boichat wrote:
> On Thu, Mar 7, 2019 at 9:45 AM Long Cheng <long.cheng@mediatek.com> wrote:
> >
> > In DMA engine framework, add 8250 uart dma to support MediaTek uart.
> > If MediaTek uart enabled(SERIAL_8250_MT6577), and want to improve
> > the performance, can enable the function.
> >
> > Signed-off-by: Long Cheng <long.cheng@mediatek.com>
> > ---
> > drivers/dma/mediatek/Kconfig | 11 +
> > drivers/dma/mediatek/Makefile | 1 +
> > drivers/dma/mediatek/mtk-uart-apdma.c | 660 +++++++++++++++++++++++++++++++++
> > 3 files changed, 672 insertions(+)
> > create mode 100644 drivers/dma/mediatek/mtk-uart-apdma.c
> >
> > diff --git a/drivers/dma/mediatek/Kconfig b/drivers/dma/mediatek/Kconfig
> > index 680fc05..ac49eb6 100644
> > --- a/drivers/dma/mediatek/Kconfig
> > +++ b/drivers/dma/mediatek/Kconfig
> > @@ -24,3 +24,14 @@ config MTK_CQDMA
> >
> > This controller provides the channels which is dedicated to
> > memory-to-memory transfer to offload from CPU.
> > +
> > +config MTK_UART_APDMA
> > + tristate "MediaTek SoCs APDMA support for UART"
> > + depends on OF && SERIAL_8250_MT6577
> > + select DMA_ENGINE
> > + select DMA_VIRTUAL_CHANNELS
> > + help
> > + Support for the UART DMA engine found on MediaTek MTK SoCs.
> > + When SERIAL_8250_MT6577 is enabled, and if you want to use DMA,
> > + you can enable the config. The DMA engine can only be used
> > + with MediaTek SoCs.
> > diff --git a/drivers/dma/mediatek/Makefile b/drivers/dma/mediatek/Makefile
> > index 41bb381..61a6d29 100644
> > --- a/drivers/dma/mediatek/Makefile
> > +++ b/drivers/dma/mediatek/Makefile
> > @@ -1,2 +1,3 @@
> > +obj-$(CONFIG_MTK_UART_APDMA) += mtk-uart-apdma.o
> > obj-$(CONFIG_MTK_HSDMA) += mtk-hsdma.o
> > obj-$(CONFIG_MTK_CQDMA) += mtk-cqdma.o
> > diff --git a/drivers/dma/mediatek/mtk-uart-apdma.c b/drivers/dma/mediatek/mtk-uart-apdma.c
> > new file mode 100644
> > index 0000000..9ed7a49
> > --- /dev/null
> > +++ b/drivers/dma/mediatek/mtk-uart-apdma.c
> > @@ -0,0 +1,660 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * MediaTek Uart APDMA driver.
> > + *
> > + * Copyright (c) 2018 MediaTek Inc.
> > + * Author: Long Cheng <long.cheng@mediatek.com>
> > + */
> > +
> > +#include <linux/clk.h>
> > +#include <linux/dmaengine.h>
> > +#include <linux/dma-mapping.h>
> > +#include <linux/err.h>
> > +#include <linux/init.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/iopoll.h>
> > +#include <linux/kernel.h>
> > +#include <linux/list.h>
> > +#include <linux/module.h>
> > +#include <linux/of_device.h>
> > +#include <linux/of_dma.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/slab.h>
> > +#include <linux/spinlock.h>
> > +
> > +#include "../virt-dma.h"
> > +
> > +/* The default number of virtual channel */
> > +#define MTK_UART_APDMA_NR_VCHANS 8
> > +
> > +#define VFF_EN_B BIT(0)
> > +#define VFF_STOP_B BIT(0)
> > +#define VFF_FLUSH_B BIT(0)
> > +#define VFF_4G_SUPPORT_B BIT(0)
> > +#define VFF_RX_INT_EN0_B BIT(0) /* rx valid size >= vff thre */
> > +#define VFF_RX_INT_EN1_B BIT(1)
> > +#define VFF_TX_INT_EN_B BIT(0) /* tx left size >= vff thre */
> > +#define VFF_WARM_RST_B BIT(0)
> > +#define VFF_RX_INT_CLR_B (BIT(0) | BIT(1))
> > +#define VFF_TX_INT_CLR_B 0
> > +#define VFF_STOP_CLR_B 0
> > +#define VFF_INT_EN_CLR_B 0
> > +#define VFF_4G_SUPPORT_CLR_B 0
> > +
> > +/* interrupt trigger level for tx */
> > +#define VFF_TX_THRE(n) ((n) * 7 / 8)
> > +/* interrupt trigger level for rx */
> > +#define VFF_RX_THRE(n) ((n) * 3 / 4)
> > +
> > +#define VFF_RING_SIZE 0xffffU
>
> Drop the U, it's not very useful (there are a a few more below, grep
> for [0-9a-f]U).
>
OK, i will fix these.
> > +/* invert this bit when wrap ring head again */
> > +#define VFF_RING_WRAP 0x10000U
> > +
> > +#define VFF_INT_FLAG 0x00
> > +#define VFF_INT_EN 0x04
> > +#define VFF_EN 0x08
> > +#define VFF_RST 0x0c
> > +#define VFF_STOP 0x10
> > +#define VFF_FLUSH 0x14
> > +#define VFF_ADDR 0x1c
> > +#define VFF_LEN 0x24
> > +#define VFF_THRE 0x28
> > +#define VFF_WPT 0x2c
> > +#define VFF_RPT 0x30
> > +/* TX: the buffer size HW can read. RX: the buffer size SW can read. */
> > +#define VFF_VALID_SIZE 0x3c
> > +/* TX: the buffer size SW can write. RX: the buffer size HW can write. */
> > +#define VFF_LEFT_SIZE 0x40
> > +#define VFF_DEBUG_STATUS 0x50
> > +#define VFF_4G_SUPPORT 0x54
> > +
> > +struct mtk_uart_apdmadev {
> > + struct dma_device ddev;
> > + struct clk *clk;
> > + bool support_33bits;
> > + unsigned int dma_requests;
> > + unsigned int *dma_irq;
> > +};
> > +
> > +struct mtk_uart_apdma_desc {
> > + struct virt_dma_desc vd;
> > +
> > + unsigned int avail_len;
> > +};
> > +
> > +struct mtk_chan {
> > + struct virt_dma_chan vc;
> > + struct dma_slave_config cfg;
> > + void __iomem *base;
> > + struct mtk_uart_apdma_desc *desc;
> > +
> > + enum dma_transfer_direction dir;
> > +
> > + bool requested;
> > +
> > + unsigned int rx_status;
> > +};
> > +
> > +static inline struct mtk_uart_apdmadev *
> > +to_mtk_uart_apdma_dev(struct dma_device *d)
> > +{
> > + return container_of(d, struct mtk_uart_apdmadev, ddev);
> > +}
> > +
> > +static inline struct mtk_chan *to_mtk_uart_apdma_chan(struct dma_chan *c)
> > +{
> > + return container_of(c, struct mtk_chan, vc.chan);
> > +}
> > +
> > +static inline struct mtk_uart_apdma_desc *to_mtk_uart_apdma_desc
> > + (struct dma_async_tx_descriptor *t)
> > +{
> > + return container_of(t, struct mtk_uart_apdma_desc, vd.tx);
> > +}
> > +
> > +static void mtk_uart_apdma_write(struct mtk_chan *c,
> > + unsigned int reg, unsigned int val)
> > +{
> > + writel(val, c->base + reg);
> > +}
> > +
> > +static unsigned int mtk_uart_apdma_read(struct mtk_chan *c, unsigned int reg)
> > +{
> > + return readl(c->base + reg);
> > +}
> > +
> > +static void mtk_uart_apdma_desc_free(struct virt_dma_desc *vd)
> > +{
> > + struct dma_chan *chan = vd->tx.chan;
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > +
> > + kfree(c->desc);
> > +}
> > +
> > +static void mtk_uart_apdma_start_tx(struct mtk_chan *c)
> > +{
> > + unsigned int len, send, left, wpt, d_wpt, tmp;
> > + int ret;
> > +
> > + left = mtk_uart_apdma_read(c, VFF_LEFT_SIZE);
> > + if (!left) {
> > + mtk_uart_apdma_write(c, VFF_INT_EN, VFF_TX_INT_EN_B);
> > + return;
> > + }
> > +
> > + /* Wait 1sec for flush, can't sleep */
> > + ret = readx_poll_timeout(readl, c->base + VFF_FLUSH, tmp,
> > + tmp != VFF_FLUSH_B, 0, 1000000);
> > + if (ret)
> > + dev_warn(c->vc.chan.device->dev, "tx: fail, debug=0x%x\n",
> > + mtk_uart_apdma_read(c, VFF_DEBUG_STATUS));
> > +
> > + send = min_t(unsigned int, left, c->desc->avail_len);
> > + wpt = mtk_uart_apdma_read(c, VFF_WPT);
> > + len = c->cfg.dst_port_window_size;
> > +
> > + d_wpt = wpt + send;
> > + if ((d_wpt & VFF_RING_SIZE) >= len) {
> > + d_wpt = d_wpt - len;
> > + d_wpt = d_wpt ^ VFF_RING_WRAP;
> > + }
> > + mtk_uart_apdma_write(c, VFF_WPT, d_wpt);
> > +
> > + c->desc->avail_len -= send;
> > +
> > + mtk_uart_apdma_write(c, VFF_INT_EN, VFF_TX_INT_EN_B);
> > + if (mtk_uart_apdma_read(c, VFF_FLUSH) == 0U)
> > + mtk_uart_apdma_write(c, VFF_FLUSH, VFF_FLUSH_B);
> > +}
> > +
> > +static void mtk_uart_apdma_start_rx(struct mtk_chan *c)
> > +{
> > + struct mtk_uart_apdma_desc *d = c->desc;
> > + unsigned int len, wg, rg;
> > + int cnt;
> > +
> > + if ((mtk_uart_apdma_read(c, VFF_VALID_SIZE) == 0U) ||
> > + !d || !vchan_next_desc(&c->vc))
> > + return;
> > +
> > + len = c->cfg.src_port_window_size;
> > + rg = mtk_uart_apdma_read(c, VFF_RPT);
> > + wg = mtk_uart_apdma_read(c, VFF_WPT);
> > + cnt = (wg & VFF_RING_SIZE) - (rg & VFF_RING_SIZE);
> > + /*
> > + * The buffer is ring buffer. If wrap bit different,
> > + * represents the start of the next cycle for WPT
> > + */
> > + if ((rg ^ wg) & VFF_RING_WRAP)
> > + cnt += len;
> > +
> > + c->rx_status = d->avail_len - cnt;
> > + mtk_uart_apdma_write(c, VFF_RPT, wg);
> > +
> > + list_del(&d->vd.node);
> > + vchan_cookie_complete(&d->vd);
> > +}
> > +
> > +static irqreturn_t mtk_uart_apdma_irq_handler(int irq, void *dev_id)
> > +{
> > + struct dma_chan *chan = (struct dma_chan *)dev_id;
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + struct mtk_uart_apdma_desc *d;
> > + unsigned long flags;
> > +
> > + spin_lock_irqsave(&c->vc.lock, flags);
> > + if (c->dir == DMA_DEV_TO_MEM) {
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_RX_INT_CLR_B);
> > + mtk_uart_apdma_start_rx(c);
> > + } else if (c->dir == DMA_MEM_TO_DEV) {
> > + d = c->desc;
> > +
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_TX_INT_CLR_B);
> > +
> > + if (d->avail_len != 0U) {
> > + mtk_uart_apdma_start_tx(c);
> > + } else {
> > + list_del(&d->vd.node);
> > + vchan_cookie_complete(&d->vd);
> > + }
> > + }
> > + spin_unlock_irqrestore(&c->vc.lock, flags);
> > +
> > + return IRQ_HANDLED;
> > +}
> > +
> > +static int mtk_uart_apdma_alloc_chan_resources(struct dma_chan *chan)
> > +{
> > + struct mtk_uart_apdmadev *mtkd = to_mtk_uart_apdma_dev(chan->device);
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + unsigned int tmp;
> > + int ret;
> > +
> > + pm_runtime_get_sync(mtkd->ddev.dev);
> > +
> > + mtk_uart_apdma_write(c, VFF_ADDR, 0);
> > + mtk_uart_apdma_write(c, VFF_THRE, 0);
> > + mtk_uart_apdma_write(c, VFF_LEN, 0);
> > + mtk_uart_apdma_write(c, VFF_RST, VFF_WARM_RST_B);
> > +
> > + ret = readx_poll_timeout(readl, c->base + VFF_EN, tmp, !tmp, 10, 100);
> > + if (ret) {
> > + dev_err(chan->device->dev, "dma reset: fail, timeout\n");
> > + return ret;
> > + }
> > +
> > + if (!c->requested) {
> > + c->requested = true;
> > + ret = request_irq(mtkd->dma_irq[chan->chan_id],
> > + mtk_uart_apdma_irq_handler, IRQF_TRIGGER_NONE,
> > + KBUILD_MODNAME, chan);
> > + if (ret < 0) {
> > + dev_err(chan->device->dev, "Can't request dma IRQ\n");
> > + return -EINVAL;
> > + }
> > + }
> > +
> > + if (mtkd->support_33bits)
> > + mtk_uart_apdma_write(c, VFF_4G_SUPPORT, VFF_4G_SUPPORT_CLR_B);
> > +
> > + return ret;
> > +}
> > +
> > +static void mtk_uart_apdma_free_chan_resources(struct dma_chan *chan)
> > +{
> > + struct mtk_uart_apdmadev *mtkd = to_mtk_uart_apdma_dev(chan->device);
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > +
> > + if (c->requested) {
> > + c->requested = false;
> > + free_irq(mtkd->dma_irq[chan->chan_id], chan);
> > + }
> > +
> > + tasklet_kill(&c->vc.task);
> > +
> > + vchan_free_chan_resources(&c->vc);
> > +
> > + pm_runtime_put_sync(mtkd->ddev.dev);
> > +}
> > +
> > +static enum dma_status mtk_uart_apdma_tx_status(struct dma_chan *chan,
> > + dma_cookie_t cookie,
> > + struct dma_tx_state *txstate)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + enum dma_status ret;
> > +
> > + ret = dma_cookie_status(chan, cookie, txstate);
> > +
> > + dma_set_residue(txstate, c->rx_status);
> > +
> > + return ret;
> > +}
> > +
> > +static void mtk_uart_apdma_config_write(struct dma_chan *chan,
> > + struct dma_slave_config *cfg,
> > + enum dma_transfer_direction dir)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + struct mtk_uart_apdmadev *mtkd =
> > + to_mtk_uart_apdma_dev(c->vc.chan.device);
> > + unsigned int tmp;
> > +
> > + if (mtk_uart_apdma_read(c, VFF_EN) == VFF_EN_B)
> > + return;
> > +
> > + c->dir = dir;
> > +
> > + if (dir == DMA_DEV_TO_MEM) {
> > + tmp = cfg->src_port_window_size;
> > +
> > + mtk_uart_apdma_write(c, VFF_ADDR, cfg->src_addr);
> > + mtk_uart_apdma_write(c, VFF_LEN, tmp);
> > + mtk_uart_apdma_write(c, VFF_THRE, VFF_RX_THRE(tmp));
> > + mtk_uart_apdma_write(c, VFF_INT_EN,
> > + VFF_RX_INT_EN0_B | VFF_RX_INT_EN1_B);
> > + mtk_uart_apdma_write(c, VFF_RPT, 0);
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_RX_INT_CLR_B);
> > + } else if (dir == DMA_MEM_TO_DEV) {
> > + tmp = cfg->dst_port_window_size;
> > +
> > + mtk_uart_apdma_write(c, VFF_ADDR, cfg->dst_addr);
> > + mtk_uart_apdma_write(c, VFF_LEN, tmp);
> > + mtk_uart_apdma_write(c, VFF_THRE, VFF_TX_THRE(tmp));
> > + mtk_uart_apdma_write(c, VFF_WPT, 0);
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_TX_INT_CLR_B);
> > + }
> > +
> > + mtk_uart_apdma_write(c, VFF_EN, VFF_EN_B);
> > +
> > + if (mtkd->support_33bits)
> > + mtk_uart_apdma_write(c, VFF_4G_SUPPORT, VFF_4G_SUPPORT_B);
> > +
> > + if (mtk_uart_apdma_read(c, VFF_EN) != VFF_EN_B)
> > + dev_err(chan->device->dev, "dir[%d] fail\n", dir);
> > +}
> > +
> > +/*
> > + * dmaengine_prep_slave_single will call the function. and sglen is 1.
> > + * 8250 uart using one ring buffer, and deal with one sg.
> > + */
> > +static struct dma_async_tx_descriptor *mtk_uart_apdma_prep_slave_sg
> > + (struct dma_chan *chan, struct scatterlist *sgl,
> > + unsigned int sglen, enum dma_transfer_direction dir,
> > + unsigned long tx_flags, void *context)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + struct mtk_uart_apdma_desc *d;
> > +
> > + if (!is_slave_direction(dir))
> > + return NULL;
> > +
> > + mtk_uart_apdma_config_write(chan, &c->cfg, dir);
> > +
> > + /* Now allocate and setup the descriptor */
> > + d = kzalloc(sizeof(*d), GFP_ATOMIC);
> > + if (!d)
> > + return NULL;
> > +
> > + /* sglen is 1 */
> > + d->avail_len = sg_dma_len(sgl);
> > + c->rx_status = d->avail_len;
> > +
> > + return vchan_tx_prep(&c->vc, &d->vd, tx_flags);
> > +}
> > +
> > +static void mtk_uart_apdma_issue_pending(struct dma_chan *chan)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + struct virt_dma_desc *vd;
> > + unsigned long flags;
> > +
> > + spin_lock_irqsave(&c->vc.lock, flags);
> > + if (vchan_issue_pending(&c->vc)) {
> > + vd = vchan_next_desc(&c->vc);
> > + c->desc = to_mtk_uart_apdma_desc(&vd->tx);
> > + }
> > +
> > + if (c->dir == DMA_DEV_TO_MEM)
> > + mtk_uart_apdma_start_rx(c);
> > + else if (c->dir == DMA_MEM_TO_DEV)
> > + mtk_uart_apdma_start_tx(c);
> > +
> > + spin_unlock_irqrestore(&c->vc.lock, flags);
> > +}
> > +
> > +static int mtk_uart_apdma_slave_config(struct dma_chan *chan,
> > + struct dma_slave_config *config)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > +
> > + memcpy(&c->cfg, config, sizeof(*config));
> > +
> > + return 0;
> > +}
> > +
> > +static int mtk_uart_apdma_terminate_all(struct dma_chan *chan)
> > +{
> > + struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
> > + unsigned long flags;
> > + unsigned int tmp;
> > + int ret;
> > +
> > + spin_lock_irqsave(&c->vc.lock, flags);
> > +
> > + mtk_uart_apdma_write(c, VFF_FLUSH, VFF_FLUSH_B);
> > + /* Wait 1sec for flush, can't sleep */
> > + ret = readx_poll_timeout(readl, c->base + VFF_FLUSH, tmp,
> > + tmp != VFF_FLUSH_B, 0, 1000000);
> > + if (ret)
> > + dev_err(c->vc.chan.device->dev, "flush: fail, debug=0x%x\n",
> > + mtk_uart_apdma_read(c, VFF_DEBUG_STATUS));
> > +
> > + /* set stop as 1 -> wait until en is 0 -> set stop as 0 */
> > + mtk_uart_apdma_write(c, VFF_STOP, VFF_STOP_B);
> > + ret = readx_poll_timeout(readl, c->base + VFF_EN, tmp, !tmp, 10, 100);
> > + if (ret)
> > + dev_err(c->vc.chan.device->dev, "stop: fail, debug=0x%x\n",
> > + mtk_uart_apdma_read(c, VFF_DEBUG_STATUS));
> > +
> > + mtk_uart_apdma_write(c, VFF_STOP, VFF_STOP_CLR_B);
> > + mtk_uart_apdma_write(c, VFF_INT_EN, VFF_INT_EN_CLR_B);
> > +
> > + if (c->dir == DMA_DEV_TO_MEM)
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_RX_INT_CLR_B);
> > + else if (c->dir == DMA_MEM_TO_DEV)
> > + mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_TX_INT_CLR_B);
> > +
> > + spin_unlock_irqrestore(&c->vc.lock, flags);
> > +
> > + return 0;
> > +}
> > +
> > +static int mtk_uart_apdma_device_pause(struct dma_chan *chan)
> > +{
> > + /* just for check caps pass */
> > + dev_err(chan->device->dev, "Pause can't support\n");
> > +
> > + return 0;
> > +}
>
> I think we've said repeatedly that leaving this stub function is incorrect.
>
I will try to implementation it. Thanks
> > +
> > +static void mtk_uart_apdma_free(struct mtk_uart_apdmadev *mtkd)
> > +{
> > + while (!list_empty(&mtkd->ddev.channels)) {
> > + struct mtk_chan *c = list_first_entry(&mtkd->ddev.channels,
> > + struct mtk_chan, vc.chan.device_node);
> > +
> > + list_del(&c->vc.chan.device_node);
> > + tasklet_kill(&c->vc.task);
> > + }
> > +}
> > +
> > +static const struct of_device_id mtk_uart_apdma_match[] = {
> > + { .compatible = "mediatek,mt6577-uart-dma", },
> > + { /* sentinel */ },
> > +};
> > +MODULE_DEVICE_TABLE(of, mtk_uart_apdma_match);
> > +
> > +static int mtk_uart_apdma_probe(struct platform_device *pdev)
> > +{
> > + struct device_node *np = pdev->dev.of_node;
> > + struct mtk_uart_apdmadev *mtkd;
> > + struct resource *res;
> > + struct mtk_chan *c;
> > + int bit_mask = 32, rc;
> > + unsigned int i;
> > +
> > + mtkd = devm_kzalloc(&pdev->dev, sizeof(*mtkd), GFP_KERNEL);
> > + if (!mtkd)
> > + return -ENOMEM;
> > +
> > + mtkd->clk = devm_clk_get(&pdev->dev, NULL);
> > + if (IS_ERR(mtkd->clk)) {
> > + dev_err(&pdev->dev, "No clock specified\n");
> > + rc = PTR_ERR(mtkd->clk);
> > + return rc;
> > + }
> > +
> > + if (of_property_read_bool(np, "mediatek,dma-33bits"))
> > + mtkd->support_33bits = true;
> > +
> > + if (mtkd->support_33bits)
> > + bit_mask = 33;
> > +
> > + rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(bit_mask));
> > + if (rc)
> > + return rc;
> > +
> > + dma_cap_set(DMA_SLAVE, mtkd->ddev.cap_mask);
> > + mtkd->ddev.device_alloc_chan_resources =
> > + mtk_uart_apdma_alloc_chan_resources;
> > + mtkd->ddev.device_free_chan_resources =
> > + mtk_uart_apdma_free_chan_resources;
> > + mtkd->ddev.device_tx_status = mtk_uart_apdma_tx_status;
> > + mtkd->ddev.device_issue_pending = mtk_uart_apdma_issue_pending;
> > + mtkd->ddev.device_prep_slave_sg = mtk_uart_apdma_prep_slave_sg;
> > + mtkd->ddev.device_config = mtk_uart_apdma_slave_config;
> > + mtkd->ddev.device_pause = mtk_uart_apdma_device_pause;
> > + mtkd->ddev.device_terminate_all = mtk_uart_apdma_terminate_all;
> > + mtkd->ddev.src_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE);
> > + mtkd->ddev.dst_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE);
> > + mtkd->ddev.directions = BIT(DMA_DEV_TO_MEM) | BIT(DMA_MEM_TO_DEV);
> > + mtkd->ddev.residue_granularity = DMA_RESIDUE_GRANULARITY_SEGMENT;
> > + mtkd->ddev.dev = &pdev->dev;
> > + INIT_LIST_HEAD(&mtkd->ddev.channels);
> > +
> > + mtkd->dma_requests = MTK_UART_APDMA_NR_VCHANS;
> > + if (of_property_read_u32(np, "dma-requests", &mtkd->dma_requests)) {
> > + dev_info(&pdev->dev,
> > + "Using %u as missing dma-requests property\n",
> > + MTK_UART_APDMA_NR_VCHANS);
> > + }
> > +
> > + mtkd->dma_irq = devm_kcalloc(&pdev->dev, mtkd->dma_requests,
> > + sizeof(*mtkd->dma_irq), GFP_KERNEL);
> > + if (!mtkd->dma_irq)
> > + return -ENOMEM;
> > +
> > + for (i = 0; i < mtkd->dma_requests; i++) {
> > + c = devm_kzalloc(mtkd->ddev.dev, sizeof(*c), GFP_KERNEL);
> > + if (!c) {
> > + rc = -ENODEV;
> > + goto err_no_dma;
> > + }
> > +
> > + res = platform_get_resource(pdev, IORESOURCE_MEM, i);
> > + if (!res) {
> > + rc = -ENODEV;
> > + goto err_no_dma;
> > + }
> > +
> > + c->base = devm_ioremap_resource(&pdev->dev, res);
> > + if (IS_ERR(c->base)) {
> > + rc = PTR_ERR(c->base);
> > + goto err_no_dma;
> > + }
> > + c->requested = false;
> > + c->vc.desc_free = mtk_uart_apdma_desc_free;
> > + vchan_init(&c->vc, &mtkd->ddev);
> > +
> > + mtkd->dma_irq[i] = platform_get_irq(pdev, i);
> > + if ((int)mtkd->dma_irq[i] < 0) {
> > + dev_err(&pdev->dev, "failed to get IRQ[%d]\n", i);
> > + rc = -EINVAL;
> > + goto err_no_dma;
> > + }
> > + }
> > +
> > + pm_runtime_enable(&pdev->dev);
> > + pm_runtime_set_active(&pdev->dev);
> > +
> > + rc = dma_async_device_register(&mtkd->ddev);
> > + if (rc)
> > + goto rpm_disable;
> > +
> > + platform_set_drvdata(pdev, mtkd);
> > +
> > + /* Device-tree DMA controller registration */
> > + rc = of_dma_controller_register(np, of_dma_xlate_by_chan_id, mtkd);
> > + if (rc)
> > + goto dma_remove;
> > +
> > + return rc;
> > +
> > +dma_remove:
> > + dma_async_device_unregister(&mtkd->ddev);
> > +rpm_disable:
> > + pm_runtime_disable(&pdev->dev);
> > +err_no_dma:
> > + mtk_uart_apdma_free(mtkd);
> > + return rc;
> > +}
> > +
> > +static int mtk_uart_apdma_remove(struct platform_device *pdev)
> > +{
> > + struct mtk_uart_apdmadev *mtkd = platform_get_drvdata(pdev);
> > +
> > + if (pdev->dev.of_node)
> > + of_dma_controller_free(pdev->dev.of_node);
> > +
> > + pm_runtime_disable(&pdev->dev);
> > + pm_runtime_put_noidle(&pdev->dev);
> > +
> > + dma_async_device_unregister(&mtkd->ddev);
> > + mtk_uart_apdma_free(mtkd);
> > +
> > + return 0;
> > +}
> > +
> > +#ifdef CONFIG_PM_SLEEP
> > +static int mtk_uart_apdma_suspend(struct device *dev)
> > +{
> > + struct mtk_uart_apdmadev *mtkd = dev_get_drvdata(dev);
> > +
> > + if (!pm_runtime_suspended(dev))
> > + clk_disable_unprepare(mtkd->clk);
> > +
> > + return 0;
> > +}
> > +
> > +static int mtk_uart_apdma_resume(struct device *dev)
> > +{
> > + int ret;
> > + struct mtk_uart_apdmadev *mtkd = dev_get_drvdata(dev);
> > +
> > + if (!pm_runtime_suspended(dev)) {
> > + ret = clk_prepare_enable(mtkd->clk);
> > + if (ret)
> > + return ret;
> > + }
> > +
> > + return 0;
> > +}
> > +#endif /* CONFIG_PM_SLEEP */
> > +
> > +#ifdef CONFIG_PM
> > +static int mtk_uart_apdma_runtime_suspend(struct device *dev)
> > +{
> > + struct mtk_uart_apdmadev *mtkd = dev_get_drvdata(dev);
> > +
> > + clk_disable_unprepare(mtkd->clk);
> > +
> > + return 0;
> > +}
> > +
> > +static int mtk_uart_apdma_runtime_resume(struct device *dev)
> > +{
> > + int ret;
> > + struct mtk_uart_apdmadev *mtkd = dev_get_drvdata(dev);
> > +
> > + ret = clk_prepare_enable(mtkd->clk);
> > + if (ret)
> > + return ret;
> > +
> > + return 0;
> > +}
> > +#endif /* CONFIG_PM */
> > +
> > +static const struct dev_pm_ops mtk_uart_apdma_pm_ops = {
> > + SET_SYSTEM_SLEEP_PM_OPS(mtk_uart_apdma_suspend, mtk_uart_apdma_resume)
> > + SET_RUNTIME_PM_OPS(mtk_uart_apdma_runtime_suspend,
> > + mtk_uart_apdma_runtime_resume, NULL)
> > +};
> > +
> > +static struct platform_driver mtk_uart_apdma_driver = {
> > + .probe = mtk_uart_apdma_probe,
> > + .remove = mtk_uart_apdma_remove,
> > + .driver = {
> > + .name = KBUILD_MODNAME,
> > + .pm = &mtk_uart_apdma_pm_ops,
> > + .of_match_table = of_match_ptr(mtk_uart_apdma_match),
> > + },
> > +};
> > +
> > +module_platform_driver(mtk_uart_apdma_driver);
> > +
> > +MODULE_DESCRIPTION("MediaTek UART APDMA Controller Driver");
> > +MODULE_AUTHOR("Long Cheng <long.cheng@mediatek.com>");
> > +MODULE_LICENSE("GPL v2");
> > +
> > --
> > 1.7.9.5
> >
^ permalink raw reply
* [V3 1/2] dmaengine: fsl-dpaa2-qdma: Add the DPDMAI(Data Path DMA Interface) support
From: Peng Ma @ 2019-04-09 7:22 UTC (permalink / raw)
To: vkoul, dan.j.williams, leoyang.li; +Cc: linux-kernel, dmaengine, Peng Ma
The MC exports the DPDMAI object as an interface to operate the DPAA2 QDMA
Engine. The DPDMAI enables sending frame-based requests to QDMA and receiving
back confirmation response on transaction completion, utilizing the DPAA2 QBMan
infrastructure. DPDMAI object provides up to two priorities for processing QDMA
requests.
The following list summarizes the DPDMAI main features and capabilities:
1. Supports up to two scheduling priorities for processing service
requests.
- Each DPDMAI transmit queue is mapped to one of two service priorities,
allowing further prioritization in hardware between requests from
different DPDMAI objects.
2. Supports up to two receive queues for incoming transaction completion
confirmations.
- Each DPDMAI receive queue is mapped to one of two receive priorities,
allowing further prioritization between other interfaces when associating
the DPDMAI receive queues to DPIO or DPCON objects.
3. Supports different scheduling options for processing received packets:
- Queues can be configured either in 'parked' mode (default), oattached
to a DPIO object, or attached to DPCON object.
4. Allows interaction with one or more DPIO objects for dequeueing/enqueueing
frame descriptors(FD) and for acquiring/releasing buffers.
5. Supports enable, disable, and reset operations.
Add dpdmai to support some platforms with dpaa2 qdma engine.
Signed-off-by: Peng Ma <peng.ma@nxp.com>
---
changed for v3:
- no changed
drivers/dma/fsl-dpaa2-qdma/dpdmai.c | 483 ++++++++++++++++++++++++++++
drivers/dma/fsl-dpaa2-qdma/dpdmai.h | 524 +++++++++++++++++++++++++++++++
drivers/dma/fsl-dpaa2-qdma/dpdmai_cmd.h | 197 ++++++++++++
3 files changed, 1204 insertions(+), 0 deletions(-)
create mode 100644 drivers/dma/fsl-dpaa2-qdma/dpdmai.c
create mode 100644 drivers/dma/fsl-dpaa2-qdma/dpdmai.h
create mode 100644 drivers/dma/fsl-dpaa2-qdma/dpdmai_cmd.h
diff --git a/drivers/dma/fsl-dpaa2-qdma/dpdmai.c b/drivers/dma/fsl-dpaa2-qdma/dpdmai.c
new file mode 100644
index 0000000..685eabe
--- /dev/null
+++ b/drivers/dma/fsl-dpaa2-qdma/dpdmai.c
@@ -0,0 +1,483 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright 2014-2018 NXP
+
+#include <linux/types.h>
+#include <linux/io.h>
+#include "dpdmai.h"
+#include "dpdmai_cmd.h"
+#include <linux/fsl/mc.h>
+
+struct dpdmai_cmd_open {
+ __le32 dpdmai_id;
+};
+
+struct dpdmai_rsp_get_attributes {
+ __le32 id;
+ u8 num_of_priorities;
+ u8 pad0[3];
+ __le16 major;
+ __le16 minor;
+};
+
+struct dpdmai_cmd_queue {
+ __le32 dest_id;
+ u8 priority;
+ u8 queue;
+ u8 dest_type;
+ u8 pad;
+ __le64 user_ctx;
+ union {
+ __le32 options;
+ __le32 fqid;
+ };
+};
+
+struct dpdmai_rsp_get_tx_queue {
+ __le64 pad;
+ __le32 fqid;
+};
+
+int dpdmai_open(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ int dpdmai_id,
+ u16 *token)
+{
+ struct fsl_mc_command cmd = { 0 };
+ struct dpdmai_cmd_open *cmd_params;
+ int err;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_OPEN,
+ cmd_flags,
+ 0);
+
+ cmd_params = (struct dpdmai_cmd_open *)cmd.params;
+ cmd_params->dpdmai_id = cpu_to_le32(dpdmai_id);
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+ *token = mc_cmd_hdr_read_token(&cmd);
+ return 0;
+}
+
+int dpdmai_close(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token)
+{
+ struct fsl_mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_CLOSE,
+ cmd_flags, token);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpdmai_create(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ const struct dpdmai_cfg *cfg,
+ u16 *token)
+{
+ struct fsl_mc_command cmd = { 0 };
+ int err;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_CREATE,
+ cmd_flags,
+ 0);
+ DPDMAI_CMD_CREATE(cmd, cfg);
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+ *token = MC_CMD_HDR_READ_TOKEN(cmd.header);
+
+ return 0;
+}
+
+int dpdmai_destroy(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token)
+{
+ struct fsl_mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_DESTROY,
+ cmd_flags,
+ token);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpdmai_enable(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token)
+{
+ struct fsl_mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_ENABLE,
+ cmd_flags,
+ token);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpdmai_disable(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token)
+{
+ struct fsl_mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_DISABLE,
+ cmd_flags,
+ token);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpdmai_is_enabled(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ int *en)
+{
+ struct fsl_mc_command cmd = { 0 };
+ int err;
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_IS_ENABLED,
+ cmd_flags,
+ token);
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+ DPDMAI_RSP_IS_ENABLED(cmd, *en);
+
+ return 0;
+}
+
+int dpdmai_reset(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token)
+{
+ struct fsl_mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_RESET,
+ cmd_flags,
+ token);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpdmai_get_irq(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ int *type,
+ struct dpdmai_irq_cfg *irq_cfg)
+{
+ struct fsl_mc_command cmd = { 0 };
+ int err;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_GET_IRQ,
+ cmd_flags,
+ token);
+ DPDMAI_CMD_GET_IRQ(cmd, irq_index);
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+ DPDMAI_RSP_GET_IRQ(cmd, *type, irq_cfg);
+
+ return 0;
+}
+
+int dpdmai_set_irq(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ struct dpdmai_irq_cfg *irq_cfg)
+{
+ struct fsl_mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_SET_IRQ,
+ cmd_flags,
+ token);
+ DPDMAI_CMD_SET_IRQ(cmd, irq_index, irq_cfg);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpdmai_get_irq_enable(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u8 *en)
+{
+ struct fsl_mc_command cmd = { 0 };
+ int err;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_GET_IRQ_ENABLE,
+ cmd_flags,
+ token);
+ DPDMAI_CMD_GET_IRQ_ENABLE(cmd, irq_index);
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+ DPDMAI_RSP_GET_IRQ_ENABLE(cmd, *en);
+
+ return 0;
+}
+
+int dpdmai_set_irq_enable(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u8 en)
+{
+ struct fsl_mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_SET_IRQ_ENABLE,
+ cmd_flags,
+ token);
+ DPDMAI_CMD_SET_IRQ_ENABLE(cmd, irq_index, en);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpdmai_get_irq_mask(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u32 *mask)
+{
+ struct fsl_mc_command cmd = { 0 };
+ int err;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_GET_IRQ_MASK,
+ cmd_flags,
+ token);
+ DPDMAI_CMD_GET_IRQ_MASK(cmd, irq_index);
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+ DPDMAI_RSP_GET_IRQ_MASK(cmd, *mask);
+
+ return 0;
+}
+
+int dpdmai_set_irq_mask(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u32 mask)
+{
+ struct fsl_mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_SET_IRQ_MASK,
+ cmd_flags,
+ token);
+ DPDMAI_CMD_SET_IRQ_MASK(cmd, irq_index, mask);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpdmai_get_irq_status(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u32 *status)
+{
+ struct fsl_mc_command cmd = { 0 };
+ int err;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_GET_IRQ_STATUS,
+ cmd_flags,
+ token);
+ DPDMAI_CMD_GET_IRQ_STATUS(cmd, irq_index, *status);
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+ DPDMAI_RSP_GET_IRQ_STATUS(cmd, *status);
+
+ return 0;
+}
+
+int dpdmai_clear_irq_status(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u32 status)
+{
+ struct fsl_mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_CLEAR_IRQ_STATUS,
+ cmd_flags,
+ token);
+ DPDMAI_CMD_CLEAR_IRQ_STATUS(cmd, irq_index, status);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpdmai_get_attributes(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ struct dpdmai_attr *attr)
+{
+ struct fsl_mc_command cmd = { 0 };
+ int err;
+ struct dpdmai_rsp_get_attributes *rsp_params;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_GET_ATTR,
+ cmd_flags,
+ token);
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+ rsp_params = (struct dpdmai_rsp_get_attributes *)cmd.params;
+ attr->id = le32_to_cpu(rsp_params->id);
+ attr->version.major = le16_to_cpu(rsp_params->major);
+ attr->version.minor = le16_to_cpu(rsp_params->minor);
+ attr->num_of_priorities = rsp_params->num_of_priorities;
+
+ return 0;
+}
+
+int dpdmai_set_rx_queue(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 priority,
+ const struct dpdmai_rx_queue_cfg *cfg)
+{
+ struct fsl_mc_command cmd = { 0 };
+ struct dpdmai_cmd_queue *cmd_params;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_SET_RX_QUEUE,
+ cmd_flags,
+ token);
+
+ cmd_params = (struct dpdmai_cmd_queue *)cmd.params;
+ cmd_params->dest_id = cpu_to_le32(cfg->dest_cfg.dest_id);
+ cmd_params->priority = cfg->dest_cfg.priority;
+ cmd_params->queue = priority;
+ cmd_params->dest_type = cfg->dest_cfg.dest_type;
+ cmd_params->user_ctx = cpu_to_le64(cfg->user_ctx);
+ cmd_params->options = cpu_to_le32(cfg->options);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpdmai_get_rx_queue(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 priority, struct dpdmai_rx_queue_attr *attr)
+{
+ struct fsl_mc_command cmd = { 0 };
+ struct dpdmai_cmd_queue *cmd_params;
+ int err;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_GET_RX_QUEUE,
+ cmd_flags,
+ token);
+
+ cmd_params = (struct dpdmai_cmd_queue *)cmd.params;
+ cmd_params->queue = priority;
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+ attr->dest_cfg.dest_id = le32_to_cpu(cmd_params->dest_id);
+ attr->dest_cfg.priority = cmd_params->priority;
+ attr->dest_cfg.dest_type = cmd_params->dest_type;
+ attr->user_ctx = le64_to_cpu(cmd_params->user_ctx);
+ attr->fqid = le32_to_cpu(cmd_params->fqid);
+
+ return 0;
+}
+
+int dpdmai_get_tx_queue(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 priority,
+ struct dpdmai_tx_queue_attr *attr)
+{
+ struct fsl_mc_command cmd = { 0 };
+ struct dpdmai_cmd_queue *cmd_params;
+ struct dpdmai_rsp_get_tx_queue *rsp_params;
+ int err;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_GET_TX_QUEUE,
+ cmd_flags,
+ token);
+
+ cmd_params = (struct dpdmai_cmd_queue *)cmd.params;
+ cmd_params->queue = priority;
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+
+ rsp_params = (struct dpdmai_rsp_get_tx_queue *)cmd.params;
+ attr->fqid = le32_to_cpu(rsp_params->fqid);
+
+ return 0;
+}
diff --git a/drivers/dma/fsl-dpaa2-qdma/dpdmai.h b/drivers/dma/fsl-dpaa2-qdma/dpdmai.h
new file mode 100644
index 0000000..c8a7b7f
--- /dev/null
+++ b/drivers/dma/fsl-dpaa2-qdma/dpdmai.h
@@ -0,0 +1,524 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright 2014-2018 NXP */
+
+/*
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of the above-listed copyright holders nor the
+ * names of any contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ *
+ * ALTERNATIVELY, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") as published by the Free Software
+ * Foundation, either version 2 of that License or (at your option) any
+ * later version.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+#ifndef __FSL_DPDMAI_H
+#define __FSL_DPDMAI_H
+
+struct fsl_mc_io;
+
+/* Data Path DMA Interface API
+ * Contains initialization APIs and runtime control APIs for DPDMAI
+ */
+
+/* General DPDMAI macros */
+
+/**
+ * Maximum number of Tx/Rx priorities per DPDMAI object
+ */
+#define DPDMAI_PRIO_NUM 2
+
+/**
+ * All queues considered; see dpdmai_set_rx_queue()
+ */
+#define DPDMAI_ALL_QUEUES (u8)(-1)
+
+/**
+ * dpdmai_open() - Open a control session for the specified object
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @dpdmai_id: DPDMAI unique ID
+ * @token: Returned token; use in subsequent API calls
+ *
+ * This function can be used to open a control session for an
+ * already created object; an object may have been declared in
+ * the DPL or by calling the dpdmai_create() function.
+ * This function returns a unique authentication token,
+ * associated with the specific object ID and the specific MC
+ * portal; this token must be used in all subsequent commands for
+ * this specific object.
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_open(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ int dpdmai_id,
+ u16 *token);
+
+/**
+ * dpdmai_close() - Close the control session of the object
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ *
+ * After this function is called, no further operations are
+ * allowed on the object without opening a new control session.
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_close(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token);
+
+/**
+ * struct dpdmai_cfg - Structure representing DPDMAI configuration
+ * @priorities: Priorities for the DMA hardware processing; valid priorities are
+ * configured with values 1-8; the entry following last valid entry
+ * should be configured with 0
+ */
+struct dpdmai_cfg {
+ u8 priorities[DPDMAI_PRIO_NUM];
+};
+
+/**
+ * dpdmai_create() - Create the DPDMAI object
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @cfg: Configuration structure
+ * @token: Returned token; use in subsequent API calls
+ *
+ * Create the DPDMAI object, allocate required resources and
+ * perform required initialization.
+ *
+ * The object can be created either by declaring it in the
+ * DPL file, or by calling this function.
+ *
+ * This function returns a unique authentication token,
+ * associated with the specific object ID and the specific MC
+ * portal; this token must be used in all subsequent calls to
+ * this specific object. For objects that are created using the
+ * DPL file, call dpdmai_open() function to get an authentication
+ * token first.
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_create(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ const struct dpdmai_cfg *cfg,
+ u16 *token);
+
+/**
+ * dpdmai_destroy() - Destroy the DPDMAI object and release all its resources.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ *
+ * Return: '0' on Success; error code otherwise.
+ */
+int dpdmai_destroy(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token);
+
+/**
+ * dpdmai_enable() - Enable the DPDMAI, allow sending and receiving frames.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_enable(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token);
+
+/**
+ * dpdmai_disable() - Disable the DPDMAI, stop sending and receiving frames.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_disable(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token);
+
+/**
+ * dpdmai_is_enabled() - Check if the DPDMAI is enabled.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @en: Returns '1' if object is enabled; '0' otherwise
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_is_enabled(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ int *en);
+
+/**
+ * dpdmai_reset() - Reset the DPDMAI, returns the object to initial state.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_reset(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token);
+
+/**
+ * struct dpdmai_irq_cfg - IRQ configuration
+ * @addr: Address that must be written to signal a message-based interrupt
+ * @val: Value to write into irq_addr address
+ * @irq_num: A user defined number associated with this IRQ
+ */
+struct dpdmai_irq_cfg {
+ u64 addr;
+ u32 val;
+ int irq_num;
+};
+
+/**
+ * dpdmai_set_irq() - Set IRQ information for the DPDMAI to trigger
+ * an interrupt.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @irq_index: Identifies the interrupt index to configure
+ * @irq_cfg: IRQ configuration
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_set_irq(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ struct dpdmai_irq_cfg *irq_cfg);
+
+/**
+ * dpdmai_get_irq() - Get IRQ information from the DPDMAI
+ *
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @irq_index: The interrupt index to configure
+ * @type: Interrupt type: 0 represents message interrupt
+ * type (both irq_addr and irq_val are valid)
+ * @irq_cfg: IRQ attributes
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_get_irq(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ int *type,
+ struct dpdmai_irq_cfg *irq_cfg);
+
+/**
+ * dpdmai_set_irq_enable() - Set overall interrupt state.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @irq_index: The interrupt index to configure
+ * @en: Interrupt state - enable = 1, disable = 0
+ *
+ * Allows GPP software to control when interrupts are generated.
+ * Each interrupt can have up to 32 causes. The enable/disable control's the
+ * overall interrupt state. if the interrupt is disabled no causes will cause
+ * an interrupt
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_set_irq_enable(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u8 en);
+
+/**
+ * dpdmai_get_irq_enable() - Get overall interrupt state
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @irq_index: The interrupt index to configure
+ * @en: Returned Interrupt state - enable = 1, disable = 0
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_get_irq_enable(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u8 *en);
+
+/**
+ * dpdmai_set_irq_mask() - Set interrupt mask.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @irq_index: The interrupt index to configure
+ * @mask: event mask to trigger interrupt;
+ * each bit:
+ * 0 = ignore event
+ * 1 = consider event for asserting IRQ
+ *
+ * Every interrupt can have up to 32 causes and the interrupt model supports
+ * masking/unmasking each cause independently
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_set_irq_mask(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u32 mask);
+
+/**
+ * dpdmai_get_irq_mask() - Get interrupt mask.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @irq_index: The interrupt index to configure
+ * @mask: Returned event mask to trigger interrupt
+ *
+ * Every interrupt can have up to 32 causes and the interrupt model supports
+ * masking/unmasking each cause independently
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_get_irq_mask(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u32 *mask);
+
+/**
+ * dpdmai_get_irq_status() - Get the current status of any pending interrupts
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @irq_index: The interrupt index to configure
+ * @status: Returned interrupts status - one bit per cause:
+ * 0 = no interrupt pending
+ * 1 = interrupt pending
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_get_irq_status(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u32 *status);
+
+/**
+ * dpdmai_clear_irq_status() - Clear a pending interrupt's status
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @irq_index: The interrupt index to configure
+ * @status: bits to clear (W1C) - one bit per cause:
+ * 0 = don't change
+ * 1 = clear status bit
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_clear_irq_status(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u32 status);
+
+/**
+ * struct dpdmai_attr - Structure representing DPDMAI attributes
+ * @id: DPDMAI object ID
+ * @version: DPDMAI version
+ * @num_of_priorities: number of priorities
+ */
+struct dpdmai_attr {
+ int id;
+ /**
+ * struct version - DPDMAI version
+ * @major: DPDMAI major version
+ * @minor: DPDMAI minor version
+ */
+ struct {
+ u16 major;
+ u16 minor;
+ } version;
+ u8 num_of_priorities;
+};
+
+/**
+ * dpdmai_get_attributes() - Retrieve DPDMAI attributes.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @attr: Returned object's attributes
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_get_attributes(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ struct dpdmai_attr *attr);
+
+/**
+ * enum dpdmai_dest - DPDMAI destination types
+ * @DPDMAI_DEST_NONE: Unassigned destination; The queue is set in parked mode
+ * and does not generate FQDAN notifications; user is expected to dequeue
+ * from the queue based on polling or other user-defined method
+ * @DPDMAI_DEST_DPIO: The queue is set in schedule mode and generates FQDAN
+ * notifications to the specified DPIO; user is expected to dequeue
+ * from the queue only after notification is received
+ * @DPDMAI_DEST_DPCON: The queue is set in schedule mode and does not generate
+ * FQDAN notifications, but is connected to the specified DPCON object;
+ * user is expected to dequeue from the DPCON channel
+ */
+enum dpdmai_dest {
+ DPDMAI_DEST_NONE = 0,
+ DPDMAI_DEST_DPIO = 1,
+ DPDMAI_DEST_DPCON = 2
+};
+
+/**
+ * struct dpdmai_dest_cfg - Structure representing DPDMAI destination parameters
+ * @dest_type: Destination type
+ * @dest_id: Either DPIO ID or DPCON ID, depending on the destination type
+ * @priority: Priority selection within the DPIO or DPCON channel; valid values
+ * are 0-1 or 0-7, depending on the number of priorities in that
+ * channel; not relevant for 'DPDMAI_DEST_NONE' option
+ */
+struct dpdmai_dest_cfg {
+ enum dpdmai_dest dest_type;
+ int dest_id;
+ u8 priority;
+};
+
+/* DPDMAI queue modification options */
+
+/**
+ * Select to modify the user's context associated with the queue
+ */
+#define DPDMAI_QUEUE_OPT_USER_CTX 0x00000001
+
+/**
+ * Select to modify the queue's destination
+ */
+#define DPDMAI_QUEUE_OPT_DEST 0x00000002
+
+/**
+ * struct dpdmai_rx_queue_cfg - DPDMAI RX queue configuration
+ * @options: Flags representing the suggested modifications to the queue;
+ * Use any combination of 'DPDMAI_QUEUE_OPT_<X>' flags
+ * @user_ctx: User context value provided in the frame descriptor of each
+ * dequeued frame;
+ * valid only if 'DPDMAI_QUEUE_OPT_USER_CTX' is contained in 'options'
+ * @dest_cfg: Queue destination parameters;
+ * valid only if 'DPDMAI_QUEUE_OPT_DEST' is contained in 'options'
+ */
+struct dpdmai_rx_queue_cfg {
+ u32 options;
+ u64 user_ctx;
+ struct dpdmai_dest_cfg dest_cfg;
+
+};
+
+/**
+ * dpdmai_set_rx_queue() - Set Rx queue configuration
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @priority: Select the queue relative to number of
+ * priorities configured at DPDMAI creation; use
+ * DPDMAI_ALL_QUEUES to configure all Rx queues
+ * identically.
+ * @cfg: Rx queue configuration
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_set_rx_queue(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 priority,
+ const struct dpdmai_rx_queue_cfg *cfg);
+
+/**
+ * struct dpdmai_rx_queue_attr - Structure representing attributes of Rx queues
+ * @user_ctx: User context value provided in the frame descriptor of each
+ * dequeued frame
+ * @dest_cfg: Queue destination configuration
+ * @fqid: Virtual FQID value to be used for dequeue operations
+ */
+struct dpdmai_rx_queue_attr {
+ u64 user_ctx;
+ struct dpdmai_dest_cfg dest_cfg;
+ u32 fqid;
+};
+
+/**
+ * dpdmai_get_rx_queue() - Retrieve Rx queue attributes.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @priority: Select the queue relative to number of
+ * priorities configured at DPDMAI creation
+ * @attr: Returned Rx queue attributes
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_get_rx_queue(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 priority,
+ struct dpdmai_rx_queue_attr *attr);
+
+/**
+ * struct dpdmai_tx_queue_attr - Structure representing attributes of Tx queues
+ * @fqid: Virtual FQID to be used for sending frames to DMA hardware
+ */
+
+struct dpdmai_tx_queue_attr {
+ u32 fqid;
+};
+
+/**
+ * dpdmai_get_tx_queue() - Retrieve Tx queue attributes.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @priority: Select the queue relative to number of
+ * priorities configured at DPDMAI creation
+ * @attr: Returned Tx queue attributes
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_get_tx_queue(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 priority,
+ struct dpdmai_tx_queue_attr *attr);
+
+#endif /* __FSL_DPDMAI_H */
diff --git a/drivers/dma/fsl-dpaa2-qdma/dpdmai_cmd.h b/drivers/dma/fsl-dpaa2-qdma/dpdmai_cmd.h
new file mode 100644
index 0000000..071a1a7
--- /dev/null
+++ b/drivers/dma/fsl-dpaa2-qdma/dpdmai_cmd.h
@@ -0,0 +1,197 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright 2014-2018 NXP */
+
+#ifndef _FSL_DPDMAI_CMD_H
+#define _FSL_DPDMAI_CMD_H
+
+/* DPDMAI Version */
+#define DPDMAI_VER_MAJOR 2
+#define DPDMAI_VER_MINOR 2
+
+#define DPDMAI_CMD_BASE_VERSION 0
+#define DPDMAI_CMD_ID_OFFSET 4
+
+#define DPDMAI_CMDID_FORMAT(x) (((x) << DPDMAI_CMD_ID_OFFSET) | \
+ DPDMAI_CMD_BASE_VERSION)
+
+/* Command IDs */
+#define DPDMAI_CMDID_CLOSE DPDMAI_CMDID_FORMAT(0x800)
+#define DPDMAI_CMDID_OPEN DPDMAI_CMDID_FORMAT(0x80E)
+#define DPDMAI_CMDID_CREATE DPDMAI_CMDID_FORMAT(0x90E)
+#define DPDMAI_CMDID_DESTROY DPDMAI_CMDID_FORMAT(0x900)
+
+#define DPDMAI_CMDID_ENABLE DPDMAI_CMDID_FORMAT(0x002)
+#define DPDMAI_CMDID_DISABLE DPDMAI_CMDID_FORMAT(0x003)
+#define DPDMAI_CMDID_GET_ATTR DPDMAI_CMDID_FORMAT(0x004)
+#define DPDMAI_CMDID_RESET DPDMAI_CMDID_FORMAT(0x005)
+#define DPDMAI_CMDID_IS_ENABLED DPDMAI_CMDID_FORMAT(0x006)
+
+#define DPDMAI_CMDID_SET_IRQ DPDMAI_CMDID_FORMAT(0x010)
+#define DPDMAI_CMDID_GET_IRQ DPDMAI_CMDID_FORMAT(0x011)
+#define DPDMAI_CMDID_SET_IRQ_ENABLE DPDMAI_CMDID_FORMAT(0x012)
+#define DPDMAI_CMDID_GET_IRQ_ENABLE DPDMAI_CMDID_FORMAT(0x013)
+#define DPDMAI_CMDID_SET_IRQ_MASK DPDMAI_CMDID_FORMAT(0x014)
+#define DPDMAI_CMDID_GET_IRQ_MASK DPDMAI_CMDID_FORMAT(0x015)
+#define DPDMAI_CMDID_GET_IRQ_STATUS DPDMAI_CMDID_FORMAT(0x016)
+#define DPDMAI_CMDID_CLEAR_IRQ_STATUS DPDMAI_CMDID_FORMAT(0x017)
+
+#define DPDMAI_CMDID_SET_RX_QUEUE DPDMAI_CMDID_FORMAT(0x1A0)
+#define DPDMAI_CMDID_GET_RX_QUEUE DPDMAI_CMDID_FORMAT(0x1A1)
+#define DPDMAI_CMDID_GET_TX_QUEUE DPDMAI_CMDID_FORMAT(0x1A2)
+
+#define MC_CMD_HDR_TOKEN_O 32 /* Token field offset */
+#define MC_CMD_HDR_TOKEN_S 16 /* Token field size */
+
+#define MAKE_UMASK64(_width) \
+ ((u64)((_width) < 64 ? ((u64)1 << (_width)) - 1 : \
+ (u64)-1)) \
+
+static inline u64 mc_enc(int lsoffset, int width, u64 val)
+{
+ return (u64)(((u64)val & MAKE_UMASK64(width)) << lsoffset);
+}
+
+static inline u64 mc_dec(u64 val, int lsoffset, int width)
+{
+ return (u64)((val >> lsoffset) & MAKE_UMASK64(width));
+}
+
+#define MC_CMD_OP(_cmd, _param, _offset, _width, _type, _arg) \
+ ((_cmd).params[_param] |= mc_enc((_offset), (_width), _arg))
+
+#define MC_RSP_OP(_cmd, _param, _offset, _width, _type, _arg) \
+ (_arg = (_type)mc_dec(_cmd.params[_param], (_offset), (_width)))
+
+#define MC_CMD_HDR_READ_TOKEN(_hdr) \
+ ((u16)mc_dec((_hdr), MC_CMD_HDR_TOKEN_O, MC_CMD_HDR_TOKEN_S))
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_OPEN(cmd, dpdmai_id) \
+ MC_CMD_OP(cmd, 0, 0, 32, int, dpdmai_id)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_CREATE(cmd, cfg) \
+do { \
+ MC_CMD_OP(cmd, 0, 8, 8, u8, (cfg)->priorities[0]);\
+ MC_CMD_OP(cmd, 0, 16, 8, u8, (cfg)->priorities[1]);\
+} while (0)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_RSP_IS_ENABLED(cmd, en) \
+ MC_RSP_OP(cmd, 0, 0, 1, int, en)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_SET_IRQ(cmd, irq_index, irq_cfg) \
+do { \
+ MC_CMD_OP(cmd, 0, 0, 8, u8, irq_index);\
+ MC_CMD_OP(cmd, 0, 32, 32, u32, irq_cfg->val);\
+ MC_CMD_OP(cmd, 1, 0, 64, u64, irq_cfg->addr);\
+ MC_CMD_OP(cmd, 2, 0, 32, int, irq_cfg->irq_num); \
+} while (0)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_GET_IRQ(cmd, irq_index) \
+ MC_CMD_OP(cmd, 0, 32, 8, u8, irq_index)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_RSP_GET_IRQ(cmd, type, irq_cfg) \
+do { \
+ MC_RSP_OP(cmd, 0, 0, 32, u32, irq_cfg->val); \
+ MC_RSP_OP(cmd, 1, 0, 64, u64, irq_cfg->addr);\
+ MC_RSP_OP(cmd, 2, 0, 32, int, irq_cfg->irq_num); \
+ MC_RSP_OP(cmd, 2, 32, 32, int, type); \
+} while (0)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_SET_IRQ_ENABLE(cmd, irq_index, enable_state) \
+do { \
+ MC_CMD_OP(cmd, 0, 0, 8, u8, enable_state); \
+ MC_CMD_OP(cmd, 0, 32, 8, u8, irq_index); \
+} while (0)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_GET_IRQ_ENABLE(cmd, irq_index) \
+ MC_CMD_OP(cmd, 0, 32, 8, u8, irq_index)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_RSP_GET_IRQ_ENABLE(cmd, enable_state) \
+ MC_RSP_OP(cmd, 0, 0, 8, u8, enable_state)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_SET_IRQ_MASK(cmd, irq_index, mask) \
+do { \
+ MC_CMD_OP(cmd, 0, 0, 32, u32, mask); \
+ MC_CMD_OP(cmd, 0, 32, 8, u8, irq_index); \
+} while (0)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_GET_IRQ_MASK(cmd, irq_index) \
+ MC_CMD_OP(cmd, 0, 32, 8, u8, irq_index)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_RSP_GET_IRQ_MASK(cmd, mask) \
+ MC_RSP_OP(cmd, 0, 0, 32, u32, mask)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_GET_IRQ_STATUS(cmd, irq_index, status) \
+do { \
+ MC_CMD_OP(cmd, 0, 0, 32, u32, status);\
+ MC_CMD_OP(cmd, 0, 32, 8, u8, irq_index);\
+} while (0)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_RSP_GET_IRQ_STATUS(cmd, status) \
+ MC_RSP_OP(cmd, 0, 0, 32, u32, status)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_CLEAR_IRQ_STATUS(cmd, irq_index, status) \
+do { \
+ MC_CMD_OP(cmd, 0, 0, 32, u32, status); \
+ MC_CMD_OP(cmd, 0, 32, 8, u8, irq_index); \
+} while (0)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_RSP_GET_ATTR(cmd, attr) \
+do { \
+ MC_RSP_OP(cmd, 0, 0, 32, int, (attr)->id); \
+ MC_RSP_OP(cmd, 0, 32, 8, u8, (attr)->num_of_priorities); \
+ MC_RSP_OP(cmd, 1, 0, 16, u16, (attr)->version.major);\
+ MC_RSP_OP(cmd, 1, 16, 16, u16, (attr)->version.minor);\
+} while (0)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_SET_RX_QUEUE(cmd, priority, cfg) \
+do { \
+ MC_CMD_OP(cmd, 0, 0, 32, int, (cfg)->dest_cfg.dest_id); \
+ MC_CMD_OP(cmd, 0, 32, 8, u8, (cfg)->dest_cfg.priority); \
+ MC_CMD_OP(cmd, 0, 40, 8, u8, priority); \
+ MC_CMD_OP(cmd, 0, 48, 4, enum dpdmai_dest, \
+ (cfg)->dest_cfg.dest_type); \
+ MC_CMD_OP(cmd, 1, 0, 64, u64, (cfg)->user_ctx); \
+ MC_CMD_OP(cmd, 2, 0, 32, u32, (cfg)->options);\
+} while (0)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_GET_RX_QUEUE(cmd, priority) \
+ MC_CMD_OP(cmd, 0, 40, 8, u8, priority)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_RSP_GET_RX_QUEUE(cmd, attr) \
+do { \
+ MC_RSP_OP(cmd, 0, 0, 32, int, (attr)->dest_cfg.dest_id);\
+ MC_RSP_OP(cmd, 0, 32, 8, u8, (attr)->dest_cfg.priority);\
+ MC_RSP_OP(cmd, 0, 48, 4, enum dpdmai_dest, \
+ (attr)->dest_cfg.dest_type);\
+ MC_RSP_OP(cmd, 1, 0, 64, u64, (attr)->user_ctx);\
+ MC_RSP_OP(cmd, 2, 0, 32, u32, (attr)->fqid);\
+} while (0)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_GET_TX_QUEUE(cmd, priority) \
+ MC_CMD_OP(cmd, 0, 40, 8, u8, priority)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_RSP_GET_TX_QUEUE(cmd, attr) \
+ MC_RSP_OP(cmd, 1, 0, 32, u32, (attr)->fqid)
+
+#endif /* _FSL_DPDMAI_CMD_H */
--
1.7.1
^ permalink raw reply related
* [V3 2/2] dmaengine: fsl-dpaa2-qdma: Add NXP dpaa2 qDMA controller driver for Layerscape SoCs
From: Peng Ma @ 2019-04-09 7:22 UTC (permalink / raw)
To: vkoul, dan.j.williams, leoyang.li; +Cc: linux-kernel, dmaengine, Peng Ma
In-Reply-To: <20190409072212.15860-1-peng.ma@nxp.com>
DPPA2(Data Path Acceleration Architecture 2) qDMA
The qDMA supports channel virtualization by allowing DMA jobs to be enqueued
into different frame queues. Core can initiate a DMA transaction by preparing
a frame descriptor(FD) for each DMA job and enqueuing this job to a frame queue.
through a hardware portal. The qDMA prefetches DMA jobs from the frame queues.
It then schedules and dispatches to internal DMA hardware engines, which
generate read and write requests. Both qDMA source data and destination data can
be either contiguous or non-contiguous using one or more scatter/gather tables.
The qDMA supports global bandwidth flow control where all DMA transactions are
stalled if the bandwidth threshold has been reached. Also supported are
transaction based read throttling.
Add NXP dppa2 qDMA to support some of Layerscape SoCs.
such as: LS1088A, LS208xA, LX2, etc.
Signed-off-by: Peng Ma <peng.ma@nxp.com>
---
changed for v3:
- Add depends on arm64 for dpaa2 qdma driver
- The dpaa2_io_service_[de]register functions have a new parameter
So update all calls to some functions
drivers/dma/Kconfig | 2 +
drivers/dma/Makefile | 1 +
drivers/dma/fsl-dpaa2-qdma/Kconfig | 9 +
drivers/dma/fsl-dpaa2-qdma/Makefile | 3 +
drivers/dma/fsl-dpaa2-qdma/dpaa2-qdma.c | 782 +++++++++++++++++++++++++++++++
drivers/dma/fsl-dpaa2-qdma/dpaa2-qdma.h | 152 ++++++
6 files changed, 949 insertions(+), 0 deletions(-)
create mode 100644 drivers/dma/fsl-dpaa2-qdma/Kconfig
create mode 100644 drivers/dma/fsl-dpaa2-qdma/Makefile
create mode 100644 drivers/dma/fsl-dpaa2-qdma/dpaa2-qdma.c
create mode 100644 drivers/dma/fsl-dpaa2-qdma/dpaa2-qdma.h
diff --git a/drivers/dma/Kconfig b/drivers/dma/Kconfig
index eaf78f4..08aae01 100644
--- a/drivers/dma/Kconfig
+++ b/drivers/dma/Kconfig
@@ -671,6 +671,8 @@ source "drivers/dma/sh/Kconfig"
source "drivers/dma/ti/Kconfig"
+source "drivers/dma/fsl-dpaa2-qdma/Kconfig"
+
# clients
comment "DMA Clients"
depends on DMA_ENGINE
diff --git a/drivers/dma/Makefile b/drivers/dma/Makefile
index 6126e1c..2499ed8 100644
--- a/drivers/dma/Makefile
+++ b/drivers/dma/Makefile
@@ -75,6 +75,7 @@ obj-$(CONFIG_UNIPHIER_MDMAC) += uniphier-mdmac.o
obj-$(CONFIG_XGENE_DMA) += xgene-dma.o
obj-$(CONFIG_ZX_DMA) += zx_dma.o
obj-$(CONFIG_ST_FDMA) += st_fdma.o
+obj-$(CONFIG_FSL_DPAA2_QDMA) += fsl-dpaa2-qdma/
obj-y += mediatek/
obj-y += qcom/
diff --git a/drivers/dma/fsl-dpaa2-qdma/Kconfig b/drivers/dma/fsl-dpaa2-qdma/Kconfig
new file mode 100644
index 0000000..258ed6b
--- /dev/null
+++ b/drivers/dma/fsl-dpaa2-qdma/Kconfig
@@ -0,0 +1,9 @@
+menuconfig FSL_DPAA2_QDMA
+ tristate "NXP DPAA2 QDMA"
+ depends on ARM64
+ depends on FSL_MC_BUS && FSL_MC_DPIO
+ select DMA_ENGINE
+ select DMA_VIRTUAL_CHANNELS
+ help
+ NXP Data Path Acceleration Architecture 2 QDMA driver,
+ using the NXP MC bus driver.
diff --git a/drivers/dma/fsl-dpaa2-qdma/Makefile b/drivers/dma/fsl-dpaa2-qdma/Makefile
new file mode 100644
index 0000000..c1d0226
--- /dev/null
+++ b/drivers/dma/fsl-dpaa2-qdma/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+# Makefile for the NXP DPAA2 qDMA controllers
+obj-$(CONFIG_FSL_DPAA2_QDMA) += dpaa2-qdma.o dpdmai.o
diff --git a/drivers/dma/fsl-dpaa2-qdma/dpaa2-qdma.c b/drivers/dma/fsl-dpaa2-qdma/dpaa2-qdma.c
new file mode 100644
index 0000000..0cdde0f
--- /dev/null
+++ b/drivers/dma/fsl-dpaa2-qdma/dpaa2-qdma.c
@@ -0,0 +1,782 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright 2014-2018 NXP
+
+/*
+ * Author: Changming Huang <jerry.huang@nxp.com>
+ *
+ * Driver for the NXP QDMA engine with QMan mode.
+ * Channel virtualization is supported through enqueuing of DMA jobs to,
+ * or dequeuing DMA jobs from different work queues with QMan portal.
+ * This module can be found on NXP LS2 SoCs.
+ *
+ */
+
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/dmapool.h>
+#include <linux/of_irq.h>
+#include <linux/iommu.h>
+#include <linux/sys_soc.h>
+#include <linux/fsl/mc.h>
+#include <soc/fsl/dpaa2-io.h>
+
+#include "../virt-dma.h"
+#include "dpdmai_cmd.h"
+#include "dpdmai.h"
+#include "dpaa2-qdma.h"
+
+static bool smmu_disable = true;
+
+static struct dpaa2_qdma_chan *to_dpaa2_qdma_chan(struct dma_chan *chan)
+{
+ return container_of(chan, struct dpaa2_qdma_chan, vchan.chan);
+}
+
+static struct dpaa2_qdma_comp *to_fsl_qdma_comp(struct virt_dma_desc *vd)
+{
+ return container_of(vd, struct dpaa2_qdma_comp, vdesc);
+}
+
+static int dpaa2_qdma_alloc_chan_resources(struct dma_chan *chan)
+{
+ struct dpaa2_qdma_chan *dpaa2_chan = to_dpaa2_qdma_chan(chan);
+ struct dpaa2_qdma_engine *dpaa2_qdma = dpaa2_chan->qdma;
+ struct device *dev = &dpaa2_qdma->priv->dpdmai_dev->dev;
+
+ dpaa2_chan->fd_pool = dma_pool_create("fd_pool", dev,
+ FD_POOL_SIZE, 32, 0);
+ if (!dpaa2_chan->fd_pool)
+ return -ENOMEM;
+
+ return dpaa2_qdma->desc_allocated++;
+}
+
+static void dpaa2_qdma_free_chan_resources(struct dma_chan *chan)
+{
+ struct dpaa2_qdma_chan *dpaa2_chan = to_dpaa2_qdma_chan(chan);
+ struct dpaa2_qdma_engine *dpaa2_qdma = dpaa2_chan->qdma;
+ unsigned long flags;
+
+ LIST_HEAD(head);
+
+ spin_lock_irqsave(&dpaa2_chan->vchan.lock, flags);
+ vchan_get_all_descriptors(&dpaa2_chan->vchan, &head);
+ spin_unlock_irqrestore(&dpaa2_chan->vchan.lock, flags);
+
+ vchan_dma_desc_free_list(&dpaa2_chan->vchan, &head);
+
+ dpaa2_dpdmai_free_comp(dpaa2_chan, &dpaa2_chan->comp_used);
+ dpaa2_dpdmai_free_comp(dpaa2_chan, &dpaa2_chan->comp_free);
+
+ dma_pool_destroy(dpaa2_chan->fd_pool);
+ dpaa2_qdma->desc_allocated--;
+}
+
+/*
+ * Request a command descriptor for enqueue.
+ */
+static struct dpaa2_qdma_comp *
+dpaa2_qdma_request_desc(struct dpaa2_qdma_chan *dpaa2_chan)
+{
+ struct dpaa2_qdma_comp *comp_temp = NULL;
+ unsigned long flags;
+
+ spin_lock_irqsave(&dpaa2_chan->queue_lock, flags);
+ if (list_empty(&dpaa2_chan->comp_free)) {
+ spin_unlock_irqrestore(&dpaa2_chan->queue_lock, flags);
+ comp_temp = kzalloc(sizeof(*comp_temp), GFP_KERNEL);
+ if (!comp_temp)
+ goto err;
+ comp_temp->fd_virt_addr =
+ dma_pool_alloc(dpaa2_chan->fd_pool, GFP_NOWAIT,
+ &comp_temp->fd_bus_addr);
+ if (!comp_temp->fd_virt_addr)
+ goto err;
+
+ comp_temp->fl_virt_addr =
+ (void *)((struct dpaa2_fd *)
+ comp_temp->fd_virt_addr + 1);
+ comp_temp->fl_bus_addr = comp_temp->fd_bus_addr +
+ sizeof(struct dpaa2_fd);
+ comp_temp->desc_virt_addr =
+ (void *)((struct dpaa2_fl_entry *)
+ comp_temp->fl_virt_addr + 3);
+ comp_temp->desc_bus_addr = comp_temp->fl_bus_addr +
+ sizeof(struct dpaa2_fl_entry) * 3;
+
+ comp_temp->qchan = dpaa2_chan;
+ return comp_temp;
+ }
+ comp_temp = list_first_entry(&dpaa2_chan->comp_free,
+ struct dpaa2_qdma_comp, list);
+ list_del(&comp_temp->list);
+ spin_unlock_irqrestore(&dpaa2_chan->queue_lock, flags);
+
+ comp_temp->qchan = dpaa2_chan;
+err:
+ return comp_temp;
+}
+
+static void
+dpaa2_qdma_populate_fd(u32 format, struct dpaa2_qdma_comp *dpaa2_comp)
+{
+ struct dpaa2_fd *fd;
+
+ fd = (struct dpaa2_fd *)dpaa2_comp->fd_virt_addr;
+ memset(fd, 0, sizeof(struct dpaa2_fd));
+
+ /* fd populated */
+ dpaa2_fd_set_addr(fd, dpaa2_comp->fl_bus_addr);
+ /* Bypass memory translation, Frame list format, short length disable */
+ /* we need to disable BMT if fsl-mc use iova addr */
+ if (smmu_disable)
+ dpaa2_fd_set_bpid(fd, QMAN_FD_BMT_ENABLE);
+ dpaa2_fd_set_format(fd, QMAN_FD_FMT_ENABLE | QMAN_FD_SL_DISABLE);
+
+ dpaa2_fd_set_frc(fd, format | QDMA_SER_CTX);
+}
+
+/* first frame list for descriptor buffer */
+static void
+dpaa2_qdma_populate_first_framel(struct dpaa2_fl_entry *f_list,
+ struct dpaa2_qdma_comp *dpaa2_comp,
+ bool wrt_changed)
+{
+ struct dpaa2_qdma_sd_d *sdd;
+
+ sdd = (struct dpaa2_qdma_sd_d *)dpaa2_comp->desc_virt_addr;
+ memset(sdd, 0, 2 * (sizeof(*sdd)));
+
+ /* source descriptor CMD */
+ sdd->cmd = cpu_to_le32(QDMA_SD_CMD_RDTTYPE_COHERENT);
+ sdd++;
+
+ /* dest descriptor CMD */
+ if (wrt_changed)
+ sdd->cmd = cpu_to_le32(LX2160_QDMA_DD_CMD_WRTTYPE_COHERENT);
+ else
+ sdd->cmd = cpu_to_le32(QDMA_DD_CMD_WRTTYPE_COHERENT);
+
+ memset(f_list, 0, sizeof(struct dpaa2_fl_entry));
+
+ /* first frame list to source descriptor */
+ dpaa2_fl_set_addr(f_list, dpaa2_comp->desc_bus_addr);
+ dpaa2_fl_set_len(f_list, 0x20);
+ dpaa2_fl_set_format(f_list, QDMA_FL_FMT_SBF | QDMA_FL_SL_LONG);
+
+ /* bypass memory translation */
+ if (smmu_disable)
+ f_list->bpid = cpu_to_le16(QDMA_FL_BMT_ENABLE);
+}
+
+/* source and destination frame list */
+static void
+dpaa2_qdma_populate_frames(struct dpaa2_fl_entry *f_list,
+ dma_addr_t dst, dma_addr_t src,
+ size_t len, uint8_t fmt)
+{
+ /* source frame list to source buffer */
+ memset(f_list, 0, sizeof(struct dpaa2_fl_entry));
+
+ dpaa2_fl_set_addr(f_list, src);
+ dpaa2_fl_set_len(f_list, len);
+
+ /* single buffer frame or scatter gather frame */
+ dpaa2_fl_set_format(f_list, (fmt | QDMA_FL_SL_LONG));
+
+ /* bypass memory translation */
+ if (smmu_disable)
+ f_list->bpid = cpu_to_le16(QDMA_FL_BMT_ENABLE);
+
+ f_list++;
+
+ /* destination frame list to destination buffer */
+ memset(f_list, 0, sizeof(struct dpaa2_fl_entry));
+
+ dpaa2_fl_set_addr(f_list, dst);
+ dpaa2_fl_set_len(f_list, len);
+ dpaa2_fl_set_format(f_list, (fmt | QDMA_FL_SL_LONG));
+ /* single buffer frame or scatter gather frame */
+ dpaa2_fl_set_final(f_list, QDMA_FL_F);
+ /* bypass memory translation */
+ if (smmu_disable)
+ f_list->bpid = cpu_to_le16(QDMA_FL_BMT_ENABLE);
+}
+
+static struct dma_async_tx_descriptor
+*dpaa2_qdma_prep_memcpy(struct dma_chan *chan, dma_addr_t dst,
+ dma_addr_t src, size_t len, ulong flags)
+{
+ struct dpaa2_qdma_chan *dpaa2_chan = to_dpaa2_qdma_chan(chan);
+ struct dpaa2_qdma_engine *dpaa2_qdma;
+ struct dpaa2_qdma_comp *dpaa2_comp;
+ struct dpaa2_fl_entry *f_list;
+ bool wrt_changed;
+ u32 format;
+
+ dpaa2_qdma = dpaa2_chan->qdma;
+ dpaa2_comp = dpaa2_qdma_request_desc(dpaa2_chan);
+ wrt_changed = (bool)dpaa2_qdma->qdma_wrtype_fixup;
+
+#ifdef LONG_FORMAT
+ format = QDMA_FD_LONG_FORMAT;
+#else
+ format = QDMA_FD_SHORT_FORMAT;
+#endif
+ /* populate Frame descriptor */
+ dpaa2_qdma_populate_fd(format, dpaa2_comp);
+
+ f_list = (struct dpaa2_fl_entry *)dpaa2_comp->fl_virt_addr;
+
+#ifdef LONG_FORMAT
+ /* first frame list for descriptor buffer (logn format) */
+ dpaa2_qdma_populate_first_framel(f_list, dpaa2_comp, wrt_changed);
+
+ f_list++;
+#endif
+
+ dpaa2_qdma_populate_frames(f_list, dst, src, len, QDMA_FL_FMT_SBF);
+
+ return vchan_tx_prep(&dpaa2_chan->vchan, &dpaa2_comp->vdesc, flags);
+}
+
+static enum
+dma_status dpaa2_qdma_tx_status(struct dma_chan *chan,
+ dma_cookie_t cookie,
+ struct dma_tx_state *txstate)
+{
+ return dma_cookie_status(chan, cookie, txstate);
+}
+
+static void dpaa2_qdma_issue_pending(struct dma_chan *chan)
+{
+ struct dpaa2_qdma_chan *dpaa2_chan = to_dpaa2_qdma_chan(chan);
+ struct dpaa2_qdma_engine *dpaa2_qdma = dpaa2_chan->qdma;
+ struct dpaa2_qdma_priv *priv = dpaa2_qdma->priv;
+ struct dpaa2_qdma_comp *dpaa2_comp;
+ struct virt_dma_desc *vdesc;
+ struct dpaa2_fd *fd;
+ unsigned long flags;
+ int err;
+
+ spin_lock_irqsave(&dpaa2_chan->queue_lock, flags);
+ spin_lock(&dpaa2_chan->vchan.lock);
+ if (vchan_issue_pending(&dpaa2_chan->vchan)) {
+ vdesc = vchan_next_desc(&dpaa2_chan->vchan);
+ if (!vdesc)
+ goto err_enqueue;
+ dpaa2_comp = to_fsl_qdma_comp(vdesc);
+
+ fd = (struct dpaa2_fd *)dpaa2_comp->fd_virt_addr;
+
+ list_del(&vdesc->node);
+ list_add_tail(&dpaa2_comp->list, &dpaa2_chan->comp_used);
+
+ /* TOBO: priority hard-coded to zero */
+ err = dpaa2_io_service_enqueue_fq(NULL,
+ priv->tx_queue_attr[0].fqid, fd);
+ if (err) {
+ list_del(&dpaa2_comp->list);
+ list_add_tail(&dpaa2_comp->list,
+ &dpaa2_chan->comp_free);
+ }
+ }
+err_enqueue:
+ spin_unlock(&dpaa2_chan->vchan.lock);
+ spin_unlock_irqrestore(&dpaa2_chan->queue_lock, flags);
+}
+
+static int __cold dpaa2_qdma_setup(struct fsl_mc_device *ls_dev)
+{
+ struct dpaa2_qdma_priv_per_prio *ppriv;
+ struct device *dev = &ls_dev->dev;
+ struct dpaa2_qdma_priv *priv;
+ u8 prio_def = DPDMAI_PRIO_NUM;
+ int err;
+ int i;
+
+ priv = dev_get_drvdata(dev);
+
+ priv->dev = dev;
+ priv->dpqdma_id = ls_dev->obj_desc.id;
+
+ /*Get the handle for the DPDMAI this interface is associate with */
+ err = dpdmai_open(priv->mc_io, 0, priv->dpqdma_id, &ls_dev->mc_handle);
+ if (err) {
+ dev_err(dev, "dpdmai_open() failed\n");
+ return err;
+ }
+ dev_info(dev, "Opened dpdmai object successfully\n");
+
+ err = dpdmai_get_attributes(priv->mc_io, 0, ls_dev->mc_handle,
+ &priv->dpdmai_attr);
+ if (err) {
+ dev_err(dev, "dpdmai_get_attributes() failed\n");
+ return err;
+ }
+
+ if (priv->dpdmai_attr.version.major > DPDMAI_VER_MAJOR) {
+ dev_err(dev, "DPDMAI major version mismatch\n"
+ "Found %u.%u, supported version is %u.%u\n",
+ priv->dpdmai_attr.version.major,
+ priv->dpdmai_attr.version.minor,
+ DPDMAI_VER_MAJOR, DPDMAI_VER_MINOR);
+ }
+
+ if (priv->dpdmai_attr.version.minor > DPDMAI_VER_MINOR) {
+ dev_err(dev, "DPDMAI minor version mismatch\n"
+ "Found %u.%u, supported version is %u.%u\n",
+ priv->dpdmai_attr.version.major,
+ priv->dpdmai_attr.version.minor,
+ DPDMAI_VER_MAJOR, DPDMAI_VER_MINOR);
+ }
+
+ priv->num_pairs = min(priv->dpdmai_attr.num_of_priorities, prio_def);
+ ppriv = kcalloc(priv->num_pairs, sizeof(*ppriv), GFP_KERNEL);
+ if (!ppriv) {
+ dev_err(dev, "kzalloc for ppriv failed\n");
+ return -1;
+ }
+ priv->ppriv = ppriv;
+
+ for (i = 0; i < priv->num_pairs; i++) {
+ err = dpdmai_get_rx_queue(priv->mc_io, 0, ls_dev->mc_handle,
+ i, &priv->rx_queue_attr[i]);
+ if (err) {
+ dev_err(dev, "dpdmai_get_rx_queue() failed\n");
+ return err;
+ }
+ ppriv->rsp_fqid = priv->rx_queue_attr[i].fqid;
+
+ err = dpdmai_get_tx_queue(priv->mc_io, 0, ls_dev->mc_handle,
+ i, &priv->tx_queue_attr[i]);
+ if (err) {
+ dev_err(dev, "dpdmai_get_tx_queue() failed\n");
+ return err;
+ }
+ ppriv->req_fqid = priv->tx_queue_attr[i].fqid;
+ ppriv->prio = i;
+ ppriv->priv = priv;
+ ppriv++;
+ }
+
+ return 0;
+}
+
+static void dpaa2_qdma_fqdan_cb(struct dpaa2_io_notification_ctx *ctx)
+{
+ struct dpaa2_qdma_priv_per_prio *ppriv = container_of(ctx,
+ struct dpaa2_qdma_priv_per_prio, nctx);
+ struct dpaa2_qdma_comp *dpaa2_comp, *_comp_tmp;
+ struct dpaa2_qdma_priv *priv = ppriv->priv;
+ u32 n_chans = priv->dpaa2_qdma->n_chans;
+ struct dpaa2_qdma_chan *qchan;
+ const struct dpaa2_fd *fd_eq;
+ const struct dpaa2_fd *fd;
+ struct dpaa2_dq *dq;
+ int is_last = 0;
+ int found;
+ u8 status;
+ int err;
+ int i;
+
+ do {
+ err = dpaa2_io_service_pull_fq(NULL, ppriv->rsp_fqid,
+ ppriv->store);
+ } while (err);
+
+ while (!is_last) {
+ do {
+ dq = dpaa2_io_store_next(ppriv->store, &is_last);
+ } while (!is_last && !dq);
+ if (!dq) {
+ dev_err(priv->dev, "FQID returned no valid frames!\n");
+ continue;
+ }
+
+ /* obtain FD and process the error */
+ fd = dpaa2_dq_fd(dq);
+
+ status = dpaa2_fd_get_ctrl(fd) & 0xff;
+ if (status)
+ dev_err(priv->dev, "FD error occurred\n");
+ found = 0;
+ for (i = 0; i < n_chans; i++) {
+ qchan = &priv->dpaa2_qdma->chans[i];
+ spin_lock(&qchan->queue_lock);
+ if (list_empty(&qchan->comp_used)) {
+ spin_unlock(&qchan->queue_lock);
+ continue;
+ }
+ list_for_each_entry_safe(dpaa2_comp, _comp_tmp,
+ &qchan->comp_used, list) {
+ fd_eq = (struct dpaa2_fd *)
+ dpaa2_comp->fd_virt_addr;
+
+ if (le64_to_cpu(fd_eq->simple.addr) ==
+ le64_to_cpu(fd->simple.addr)) {
+ spin_lock(&qchan->vchan.lock);
+ vchan_cookie_complete(&
+ dpaa2_comp->vdesc);
+ spin_unlock(&qchan->vchan.lock);
+ found = 1;
+ break;
+ }
+ }
+ spin_unlock(&qchan->queue_lock);
+ if (found)
+ break;
+ }
+ }
+
+ dpaa2_io_service_rearm(NULL, ctx);
+}
+
+static int __cold dpaa2_qdma_dpio_setup(struct dpaa2_qdma_priv *priv)
+{
+ struct dpaa2_qdma_priv_per_prio *ppriv;
+ struct device *dev = priv->dev;
+ int err, i, num;
+
+ num = priv->num_pairs;
+ ppriv = priv->ppriv;
+ for (i = 0; i < num; i++) {
+ ppriv->nctx.is_cdan = 0;
+ ppriv->nctx.desired_cpu = 1;
+ ppriv->nctx.id = ppriv->rsp_fqid;
+ ppriv->nctx.cb = dpaa2_qdma_fqdan_cb;
+ err = dpaa2_io_service_register(NULL, &ppriv->nctx, dev);
+ if (err) {
+ dev_err(dev, "Notification register failed\n");
+ goto err_service;
+ }
+
+ ppriv->store =
+ dpaa2_io_store_create(DPAA2_QDMA_STORE_SIZE, dev);
+ if (!ppriv->store) {
+ dev_err(dev, "dpaa2_io_store_create() failed\n");
+ goto err_store;
+ }
+
+ ppriv++;
+ }
+ return 0;
+
+err_store:
+ dpaa2_io_service_deregister(NULL, &ppriv->nctx, dev);
+err_service:
+ ppriv--;
+ while (ppriv >= priv->ppriv) {
+ dpaa2_io_service_deregister(NULL, &ppriv->nctx, dev);
+ dpaa2_io_store_destroy(ppriv->store);
+ ppriv--;
+ }
+ return -1;
+}
+
+static void dpaa2_dpmai_store_free(struct dpaa2_qdma_priv *priv)
+{
+ struct dpaa2_qdma_priv_per_prio *ppriv = priv->ppriv;
+ int i;
+
+ for (i = 0; i < priv->num_pairs; i++) {
+ dpaa2_io_store_destroy(ppriv->store);
+ ppriv++;
+ }
+}
+
+static void dpaa2_dpdmai_dpio_free(struct dpaa2_qdma_priv *priv)
+{
+ struct dpaa2_qdma_priv_per_prio *ppriv = priv->ppriv;
+ struct device *dev = priv->dev;
+ int i;
+
+ for (i = 0; i < priv->num_pairs; i++) {
+ dpaa2_io_service_deregister(NULL, &ppriv->nctx, dev);
+ ppriv++;
+ }
+}
+
+static int __cold dpaa2_dpdmai_bind(struct dpaa2_qdma_priv *priv)
+{
+ int err;
+ int i, num;
+ struct device *dev = priv->dev;
+ struct dpaa2_qdma_priv_per_prio *ppriv;
+ struct dpdmai_rx_queue_cfg rx_queue_cfg;
+ struct fsl_mc_device *ls_dev = to_fsl_mc_device(dev);
+
+ num = priv->num_pairs;
+ ppriv = priv->ppriv;
+ for (i = 0; i < num; i++) {
+ rx_queue_cfg.options = DPDMAI_QUEUE_OPT_USER_CTX |
+ DPDMAI_QUEUE_OPT_DEST;
+ rx_queue_cfg.user_ctx = ppriv->nctx.qman64;
+ rx_queue_cfg.dest_cfg.dest_type = DPDMAI_DEST_DPIO;
+ rx_queue_cfg.dest_cfg.dest_id = ppriv->nctx.dpio_id;
+ rx_queue_cfg.dest_cfg.priority = ppriv->prio;
+ err = dpdmai_set_rx_queue(priv->mc_io, 0, ls_dev->mc_handle,
+ rx_queue_cfg.dest_cfg.priority,
+ &rx_queue_cfg);
+ if (err) {
+ dev_err(dev, "dpdmai_set_rx_queue() failed\n");
+ return err;
+ }
+
+ ppriv++;
+ }
+
+ return 0;
+}
+
+static int __cold dpaa2_dpdmai_dpio_unbind(struct dpaa2_qdma_priv *priv)
+{
+ int i;
+ int err = 0;
+ struct device *dev = priv->dev;
+ struct fsl_mc_device *ls_dev = to_fsl_mc_device(dev);
+ struct dpaa2_qdma_priv_per_prio *ppriv = priv->ppriv;
+
+ for (i = 0; i < priv->num_pairs; i++) {
+ ppriv->nctx.qman64 = 0;
+ ppriv->nctx.dpio_id = 0;
+ ppriv++;
+ }
+
+ err = dpdmai_reset(priv->mc_io, 0, ls_dev->mc_handle);
+ if (err)
+ dev_err(dev, "dpdmai_reset() failed\n");
+
+ return err;
+}
+
+static void dpaa2_dpdmai_free_comp(struct dpaa2_qdma_chan *qchan,
+ struct list_head *head)
+{
+ struct dpaa2_qdma_comp *comp_tmp, *_comp_tmp;
+
+ list_for_each_entry_safe(comp_tmp, _comp_tmp,
+ head, list) {
+ dma_pool_free(qchan->fd_pool,
+ comp_tmp->fd_virt_addr,
+ comp_tmp->fd_bus_addr);
+ list_del(&comp_tmp->list);
+ kfree(comp_tmp);
+ }
+}
+
+static void dpaa2_dpdmai_free_channels(struct dpaa2_qdma_engine *dpaa2_qdma)
+{
+ struct dpaa2_qdma_chan *qchan;
+ int num, i;
+
+ num = dpaa2_qdma->n_chans;
+ for (i = 0; i < num; i++) {
+ qchan = &dpaa2_qdma->chans[i];
+ dpaa2_dpdmai_free_comp(qchan, &qchan->comp_used);
+ dpaa2_dpdmai_free_comp(qchan, &qchan->comp_free);
+ dma_pool_destroy(qchan->fd_pool);
+ }
+}
+
+static void dpaa2_qdma_free_desc(struct virt_dma_desc *vdesc)
+{
+ struct dpaa2_qdma_comp *dpaa2_comp;
+ struct dpaa2_qdma_chan *qchan;
+
+ dpaa2_comp = to_fsl_qdma_comp(vdesc);
+ qchan = dpaa2_comp->qchan;
+ list_del(&dpaa2_comp->list);
+ list_add_tail(&dpaa2_comp->list, &qchan->comp_free);
+}
+
+static int dpaa2_dpdmai_init_channels(struct dpaa2_qdma_engine *dpaa2_qdma)
+{
+ struct dpaa2_qdma_chan *dpaa2_chan;
+ int i;
+
+ INIT_LIST_HEAD(&dpaa2_qdma->dma_dev.channels);
+ for (i = 0; i < dpaa2_qdma->n_chans; i++) {
+ dpaa2_chan = &dpaa2_qdma->chans[i];
+ dpaa2_chan->qdma = dpaa2_qdma;
+ dpaa2_chan->vchan.desc_free = dpaa2_qdma_free_desc;
+ vchan_init(&dpaa2_chan->vchan, &dpaa2_qdma->dma_dev);
+ spin_lock_init(&dpaa2_chan->queue_lock);
+ INIT_LIST_HEAD(&dpaa2_chan->comp_used);
+ INIT_LIST_HEAD(&dpaa2_chan->comp_free);
+ }
+ return 0;
+}
+
+static int dpaa2_qdma_probe(struct fsl_mc_device *dpdmai_dev)
+{
+ struct dpaa2_qdma_priv *priv;
+ struct device *dev = &dpdmai_dev->dev;
+ struct dpaa2_qdma_engine *dpaa2_qdma;
+ int err;
+
+ priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+ dev_set_drvdata(dev, priv);
+ priv->dpdmai_dev = dpdmai_dev;
+
+ priv->iommu_domain = iommu_get_domain_for_dev(dev);
+ if (priv->iommu_domain)
+ smmu_disable = false;
+
+ /* obtain a MC portal */
+ err = fsl_mc_portal_allocate(dpdmai_dev, 0, &priv->mc_io);
+ if (err) {
+ if (err == -ENXIO)
+ err = -EPROBE_DEFER;
+ else
+ dev_err(dev, "MC portal allocation failed\n");
+ goto err_mcportal;
+ }
+
+ /* DPDMAI initialization */
+ err = dpaa2_qdma_setup(dpdmai_dev);
+ if (err) {
+ dev_err(dev, "dpaa2_dpdmai_setup() failed\n");
+ goto err_dpdmai_setup;
+ }
+
+ /* DPIO */
+ err = dpaa2_qdma_dpio_setup(priv);
+ if (err) {
+ dev_err(dev, "dpaa2_dpdmai_dpio_setup() failed\n");
+ goto err_dpio_setup;
+ }
+
+ /* DPDMAI binding to DPIO */
+ err = dpaa2_dpdmai_bind(priv);
+ if (err) {
+ dev_err(dev, "dpaa2_dpdmai_bind() failed\n");
+ goto err_bind;
+ }
+
+ /* DPDMAI enable */
+ err = dpdmai_enable(priv->mc_io, 0, dpdmai_dev->mc_handle);
+ if (err) {
+ dev_err(dev, "dpdmai_enable() faile\n");
+ goto err_enable;
+ }
+
+ dpaa2_qdma = kzalloc(sizeof(*dpaa2_qdma), GFP_KERNEL);
+ if (!dpaa2_qdma) {
+ err = -ENOMEM;
+ goto err_eng;
+ }
+
+ priv->dpaa2_qdma = dpaa2_qdma;
+ dpaa2_qdma->priv = priv;
+
+ dpaa2_qdma->desc_allocated = 0;
+ dpaa2_qdma->n_chans = NUM_CH;
+
+ dpaa2_dpdmai_init_channels(dpaa2_qdma);
+
+ if (soc_device_match(soc_fixup_tuning))
+ dpaa2_qdma->qdma_wrtype_fixup = true;
+ else
+ dpaa2_qdma->qdma_wrtype_fixup = false;
+
+ dma_cap_set(DMA_PRIVATE, dpaa2_qdma->dma_dev.cap_mask);
+ dma_cap_set(DMA_SLAVE, dpaa2_qdma->dma_dev.cap_mask);
+ dma_cap_set(DMA_MEMCPY, dpaa2_qdma->dma_dev.cap_mask);
+
+ dpaa2_qdma->dma_dev.dev = dev;
+ dpaa2_qdma->dma_dev.device_alloc_chan_resources =
+ dpaa2_qdma_alloc_chan_resources;
+ dpaa2_qdma->dma_dev.device_free_chan_resources =
+ dpaa2_qdma_free_chan_resources;
+ dpaa2_qdma->dma_dev.device_tx_status = dpaa2_qdma_tx_status;
+ dpaa2_qdma->dma_dev.device_prep_dma_memcpy = dpaa2_qdma_prep_memcpy;
+ dpaa2_qdma->dma_dev.device_issue_pending = dpaa2_qdma_issue_pending;
+
+ err = dma_async_device_register(&dpaa2_qdma->dma_dev);
+ if (err) {
+ dev_err(dev, "Can't register NXP QDMA engine.\n");
+ goto err_eng;
+ }
+
+ return 0;
+
+err_eng:
+ dpdmai_disable(priv->mc_io, 0, dpdmai_dev->mc_handle);
+err_enable:
+ dpaa2_dpdmai_dpio_unbind(priv);
+err_bind:
+ dpaa2_dpmai_store_free(priv);
+ dpaa2_dpdmai_dpio_free(priv);
+err_dpio_setup:
+ dpdmai_close(priv->mc_io, 0, dpdmai_dev->mc_handle);
+err_dpdmai_setup:
+ fsl_mc_portal_free(priv->mc_io);
+err_mcportal:
+ kfree(priv->ppriv);
+ kfree(priv);
+ dev_set_drvdata(dev, NULL);
+ return err;
+}
+
+static int dpaa2_qdma_remove(struct fsl_mc_device *ls_dev)
+{
+ struct device *dev;
+ struct dpaa2_qdma_priv *priv;
+ struct dpaa2_qdma_engine *dpaa2_qdma;
+
+ dev = &ls_dev->dev;
+ priv = dev_get_drvdata(dev);
+ dpaa2_qdma = priv->dpaa2_qdma;
+
+ dpdmai_disable(priv->mc_io, 0, ls_dev->mc_handle);
+ dpaa2_dpdmai_dpio_unbind(priv);
+ dpaa2_dpmai_store_free(priv);
+ dpaa2_dpdmai_dpio_free(priv);
+ dpdmai_close(priv->mc_io, 0, ls_dev->mc_handle);
+ fsl_mc_portal_free(priv->mc_io);
+ dev_set_drvdata(dev, NULL);
+ dpaa2_dpdmai_free_channels(dpaa2_qdma);
+
+ dma_async_device_unregister(&dpaa2_qdma->dma_dev);
+ kfree(priv);
+ kfree(dpaa2_qdma);
+
+ return 0;
+}
+
+static const struct fsl_mc_device_id dpaa2_qdma_id_table[] = {
+ {
+ .vendor = FSL_MC_VENDOR_FREESCALE,
+ .obj_type = "dpdmai",
+ },
+ { .vendor = 0x0 }
+};
+
+static struct fsl_mc_driver dpaa2_qdma_driver = {
+ .driver = {
+ .name = "dpaa2-qdma",
+ .owner = THIS_MODULE,
+ },
+ .probe = dpaa2_qdma_probe,
+ .remove = dpaa2_qdma_remove,
+ .match_id_table = dpaa2_qdma_id_table
+};
+
+static int __init dpaa2_qdma_driver_init(void)
+{
+ return fsl_mc_driver_register(&(dpaa2_qdma_driver));
+}
+late_initcall(dpaa2_qdma_driver_init);
+
+static void __exit fsl_qdma_exit(void)
+{
+ fsl_mc_driver_unregister(&(dpaa2_qdma_driver));
+}
+module_exit(fsl_qdma_exit);
+
+MODULE_ALIAS("platform:fsl-dpaa2-qdma");
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("NXP Layerscape DPAA2 qDMA engine driver");
diff --git a/drivers/dma/fsl-dpaa2-qdma/dpaa2-qdma.h b/drivers/dma/fsl-dpaa2-qdma/dpaa2-qdma.h
new file mode 100644
index 0000000..3fcade8
--- /dev/null
+++ b/drivers/dma/fsl-dpaa2-qdma/dpaa2-qdma.h
@@ -0,0 +1,152 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright 2014-2018 NXP */
+
+#ifndef __DPAA2_QDMA_H
+#define __DPAA2_QDMA_H
+
+#define LONG_FORMAT 1
+
+#define DPAA2_QDMA_STORE_SIZE 16
+#define NUM_CH 8
+
+struct dpaa2_qdma_sd_d {
+ u32 rsv:32;
+ union {
+ struct {
+ u32 ssd:12; /* souce stride distance */
+ u32 sss:12; /* souce stride size */
+ u32 rsv1:8;
+ } sdf;
+ struct {
+ u32 dsd:12; /* Destination stride distance */
+ u32 dss:12; /* Destination stride size */
+ u32 rsv2:8;
+ } ddf;
+ } df;
+ u32 rbpcmd; /* Route-by-port command */
+ u32 cmd;
+} __attribute__((__packed__));
+
+/* Source descriptor command read transaction type for RBP=0: */
+/* coherent copy of cacheable memory */
+#define QDMA_SD_CMD_RDTTYPE_COHERENT (0xb << 28)
+/* Destination descriptor command write transaction type for RBP=0: */
+/* coherent copy of cacheable memory */
+#define QDMA_DD_CMD_WRTTYPE_COHERENT (0x6 << 28)
+#define LX2160_QDMA_DD_CMD_WRTTYPE_COHERENT (0xb << 28)
+
+#define QMAN_FD_FMT_ENABLE BIT(0) /* frame list table enable */
+#define QMAN_FD_BMT_ENABLE BIT(15) /* bypass memory translation */
+#define QMAN_FD_BMT_DISABLE (0) /* bypass memory translation */
+#define QMAN_FD_SL_DISABLE (0) /* short lengthe disabled */
+#define QMAN_FD_SL_ENABLE BIT(14) /* short lengthe enabled */
+
+#define QDMA_FINAL_BIT_DISABLE (0) /* final bit disable */
+#define QDMA_FINAL_BIT_ENABLE BIT(31) /* final bit enable */
+
+#define QDMA_FD_SHORT_FORMAT BIT(11) /* short format */
+#define QDMA_FD_LONG_FORMAT (0) /* long format */
+#define QDMA_SER_DISABLE (8) /* no notification */
+#define QDMA_SER_CTX BIT(8) /* notification by FQD_CTX[fqid] */
+#define QDMA_SER_DEST (2 << 8) /* notification by destination desc */
+#define QDMA_SER_BOTH (3 << 8) /* soruce and dest notification */
+#define QDMA_FD_SPF_ENALBE BIT(30) /* source prefetch enable */
+
+#define QMAN_FD_VA_ENABLE BIT(14) /* Address used is virtual address */
+#define QMAN_FD_VA_DISABLE (0)/* Address used is a real address */
+/* Flow Context: 49bit physical address */
+#define QMAN_FD_CBMT_ENABLE BIT(15)
+#define QMAN_FD_CBMT_DISABLE (0) /* Flow Context: 64bit virtual address */
+#define QMAN_FD_SC_DISABLE (0) /* stashing control */
+
+#define QDMA_FL_FMT_SBF (0x0) /* Single buffer frame */
+#define QDMA_FL_FMT_SGE (0x2) /* Scatter gather frame */
+#define QDMA_FL_BMT_ENABLE BIT(15) /* enable bypass memory translation */
+#define QDMA_FL_BMT_DISABLE (0x0) /* enable bypass memory translation */
+#define QDMA_FL_SL_LONG (0x0)/* long length */
+#define QDMA_FL_SL_SHORT (0x1) /* short length */
+#define QDMA_FL_F (0x1)/* last frame list bit */
+
+/*Description of Frame list table structure*/
+struct dpaa2_qdma_chan {
+ struct virt_dma_chan vchan;
+ struct virt_dma_desc vdesc;
+ enum dma_status status;
+ struct dpaa2_qdma_engine *qdma;
+
+ struct mutex dpaa2_queue_mutex;
+ spinlock_t queue_lock;
+ struct dma_pool *fd_pool;
+
+ struct list_head comp_used;
+ struct list_head comp_free;
+
+};
+
+struct dpaa2_qdma_comp {
+ dma_addr_t fd_bus_addr;
+ dma_addr_t fl_bus_addr;
+ dma_addr_t desc_bus_addr;
+ void *fd_virt_addr;
+ void *fl_virt_addr;
+ void *desc_virt_addr;
+ struct dpaa2_qdma_chan *qchan;
+ struct virt_dma_desc vdesc;
+ struct list_head list;
+};
+
+struct dpaa2_qdma_engine {
+ struct dma_device dma_dev;
+ u32 n_chans;
+ struct dpaa2_qdma_chan chans[NUM_CH];
+ int qdma_wrtype_fixup;
+ int desc_allocated;
+
+ struct dpaa2_qdma_priv *priv;
+};
+
+/*
+ * dpaa2_qdma_priv - driver private data
+ */
+struct dpaa2_qdma_priv {
+ int dpqdma_id;
+
+ struct iommu_domain *iommu_domain;
+ struct dpdmai_attr dpdmai_attr;
+ struct device *dev;
+ struct fsl_mc_io *mc_io;
+ struct fsl_mc_device *dpdmai_dev;
+ u8 num_pairs;
+
+ struct dpaa2_qdma_engine *dpaa2_qdma;
+ struct dpaa2_qdma_priv_per_prio *ppriv;
+
+ struct dpdmai_rx_queue_attr rx_queue_attr[DPDMAI_PRIO_NUM];
+ struct dpdmai_tx_queue_attr tx_queue_attr[DPDMAI_PRIO_NUM];
+};
+
+struct dpaa2_qdma_priv_per_prio {
+ int req_fqid;
+ int rsp_fqid;
+ int prio;
+
+ struct dpaa2_io_store *store;
+ struct dpaa2_io_notification_ctx nctx;
+
+ struct dpaa2_qdma_priv *priv;
+};
+
+static struct soc_device_attribute soc_fixup_tuning[] = {
+ { .family = "QorIQ LX2160A"},
+ { },
+};
+
+/* FD pool size: one FD + 3 Frame list + 2 source/destination descriptor */
+#define FD_POOL_SIZE (sizeof(struct dpaa2_fd) + \
+ sizeof(struct dpaa2_fl_entry) * 3 + \
+ sizeof(struct dpaa2_qdma_sd_d) * 2)
+
+static void dpaa2_dpdmai_free_channels(struct dpaa2_qdma_engine *dpaa2_qdma);
+static void dpaa2_dpdmai_free_comp(struct dpaa2_qdma_chan *qchan,
+ struct list_head *head);
+#endif /* __DPAA2_QDMA_H */
--
1.7.1
^ permalink raw reply related
* [V3,2/2] dmaengine: fsl-dpaa2-qdma: Add NXP dpaa2 qDMA controller driver for Layerscape SoCs
From: Peng Ma @ 2019-04-09 7:22 UTC (permalink / raw)
To: vkoul, dan.j.williams, leoyang.li; +Cc: linux-kernel, dmaengine, Peng Ma
DPPA2(Data Path Acceleration Architecture 2) qDMA
The qDMA supports channel virtualization by allowing DMA jobs to be enqueued
into different frame queues. Core can initiate a DMA transaction by preparing
a frame descriptor(FD) for each DMA job and enqueuing this job to a frame queue.
through a hardware portal. The qDMA prefetches DMA jobs from the frame queues.
It then schedules and dispatches to internal DMA hardware engines, which
generate read and write requests. Both qDMA source data and destination data can
be either contiguous or non-contiguous using one or more scatter/gather tables.
The qDMA supports global bandwidth flow control where all DMA transactions are
stalled if the bandwidth threshold has been reached. Also supported are
transaction based read throttling.
Add NXP dppa2 qDMA to support some of Layerscape SoCs.
such as: LS1088A, LS208xA, LX2, etc.
Signed-off-by: Peng Ma <peng.ma@nxp.com>
---
changed for v3:
- Add depends on arm64 for dpaa2 qdma driver
- The dpaa2_io_service_[de]register functions have a new parameter
So update all calls to some functions
drivers/dma/Kconfig | 2 +
drivers/dma/Makefile | 1 +
drivers/dma/fsl-dpaa2-qdma/Kconfig | 9 +
drivers/dma/fsl-dpaa2-qdma/Makefile | 3 +
drivers/dma/fsl-dpaa2-qdma/dpaa2-qdma.c | 782 +++++++++++++++++++++++++++++++
drivers/dma/fsl-dpaa2-qdma/dpaa2-qdma.h | 152 ++++++
6 files changed, 949 insertions(+), 0 deletions(-)
create mode 100644 drivers/dma/fsl-dpaa2-qdma/Kconfig
create mode 100644 drivers/dma/fsl-dpaa2-qdma/Makefile
create mode 100644 drivers/dma/fsl-dpaa2-qdma/dpaa2-qdma.c
create mode 100644 drivers/dma/fsl-dpaa2-qdma/dpaa2-qdma.h
diff --git a/drivers/dma/Kconfig b/drivers/dma/Kconfig
index eaf78f4..08aae01 100644
--- a/drivers/dma/Kconfig
+++ b/drivers/dma/Kconfig
@@ -671,6 +671,8 @@ source "drivers/dma/sh/Kconfig"
source "drivers/dma/ti/Kconfig"
+source "drivers/dma/fsl-dpaa2-qdma/Kconfig"
+
# clients
comment "DMA Clients"
depends on DMA_ENGINE
diff --git a/drivers/dma/Makefile b/drivers/dma/Makefile
index 6126e1c..2499ed8 100644
--- a/drivers/dma/Makefile
+++ b/drivers/dma/Makefile
@@ -75,6 +75,7 @@ obj-$(CONFIG_UNIPHIER_MDMAC) += uniphier-mdmac.o
obj-$(CONFIG_XGENE_DMA) += xgene-dma.o
obj-$(CONFIG_ZX_DMA) += zx_dma.o
obj-$(CONFIG_ST_FDMA) += st_fdma.o
+obj-$(CONFIG_FSL_DPAA2_QDMA) += fsl-dpaa2-qdma/
obj-y += mediatek/
obj-y += qcom/
diff --git a/drivers/dma/fsl-dpaa2-qdma/Kconfig b/drivers/dma/fsl-dpaa2-qdma/Kconfig
new file mode 100644
index 0000000..258ed6b
--- /dev/null
+++ b/drivers/dma/fsl-dpaa2-qdma/Kconfig
@@ -0,0 +1,9 @@
+menuconfig FSL_DPAA2_QDMA
+ tristate "NXP DPAA2 QDMA"
+ depends on ARM64
+ depends on FSL_MC_BUS && FSL_MC_DPIO
+ select DMA_ENGINE
+ select DMA_VIRTUAL_CHANNELS
+ help
+ NXP Data Path Acceleration Architecture 2 QDMA driver,
+ using the NXP MC bus driver.
diff --git a/drivers/dma/fsl-dpaa2-qdma/Makefile b/drivers/dma/fsl-dpaa2-qdma/Makefile
new file mode 100644
index 0000000..c1d0226
--- /dev/null
+++ b/drivers/dma/fsl-dpaa2-qdma/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+# Makefile for the NXP DPAA2 qDMA controllers
+obj-$(CONFIG_FSL_DPAA2_QDMA) += dpaa2-qdma.o dpdmai.o
diff --git a/drivers/dma/fsl-dpaa2-qdma/dpaa2-qdma.c b/drivers/dma/fsl-dpaa2-qdma/dpaa2-qdma.c
new file mode 100644
index 0000000..0cdde0f
--- /dev/null
+++ b/drivers/dma/fsl-dpaa2-qdma/dpaa2-qdma.c
@@ -0,0 +1,782 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright 2014-2018 NXP
+
+/*
+ * Author: Changming Huang <jerry.huang@nxp.com>
+ *
+ * Driver for the NXP QDMA engine with QMan mode.
+ * Channel virtualization is supported through enqueuing of DMA jobs to,
+ * or dequeuing DMA jobs from different work queues with QMan portal.
+ * This module can be found on NXP LS2 SoCs.
+ *
+ */
+
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/dmapool.h>
+#include <linux/of_irq.h>
+#include <linux/iommu.h>
+#include <linux/sys_soc.h>
+#include <linux/fsl/mc.h>
+#include <soc/fsl/dpaa2-io.h>
+
+#include "../virt-dma.h"
+#include "dpdmai_cmd.h"
+#include "dpdmai.h"
+#include "dpaa2-qdma.h"
+
+static bool smmu_disable = true;
+
+static struct dpaa2_qdma_chan *to_dpaa2_qdma_chan(struct dma_chan *chan)
+{
+ return container_of(chan, struct dpaa2_qdma_chan, vchan.chan);
+}
+
+static struct dpaa2_qdma_comp *to_fsl_qdma_comp(struct virt_dma_desc *vd)
+{
+ return container_of(vd, struct dpaa2_qdma_comp, vdesc);
+}
+
+static int dpaa2_qdma_alloc_chan_resources(struct dma_chan *chan)
+{
+ struct dpaa2_qdma_chan *dpaa2_chan = to_dpaa2_qdma_chan(chan);
+ struct dpaa2_qdma_engine *dpaa2_qdma = dpaa2_chan->qdma;
+ struct device *dev = &dpaa2_qdma->priv->dpdmai_dev->dev;
+
+ dpaa2_chan->fd_pool = dma_pool_create("fd_pool", dev,
+ FD_POOL_SIZE, 32, 0);
+ if (!dpaa2_chan->fd_pool)
+ return -ENOMEM;
+
+ return dpaa2_qdma->desc_allocated++;
+}
+
+static void dpaa2_qdma_free_chan_resources(struct dma_chan *chan)
+{
+ struct dpaa2_qdma_chan *dpaa2_chan = to_dpaa2_qdma_chan(chan);
+ struct dpaa2_qdma_engine *dpaa2_qdma = dpaa2_chan->qdma;
+ unsigned long flags;
+
+ LIST_HEAD(head);
+
+ spin_lock_irqsave(&dpaa2_chan->vchan.lock, flags);
+ vchan_get_all_descriptors(&dpaa2_chan->vchan, &head);
+ spin_unlock_irqrestore(&dpaa2_chan->vchan.lock, flags);
+
+ vchan_dma_desc_free_list(&dpaa2_chan->vchan, &head);
+
+ dpaa2_dpdmai_free_comp(dpaa2_chan, &dpaa2_chan->comp_used);
+ dpaa2_dpdmai_free_comp(dpaa2_chan, &dpaa2_chan->comp_free);
+
+ dma_pool_destroy(dpaa2_chan->fd_pool);
+ dpaa2_qdma->desc_allocated--;
+}
+
+/*
+ * Request a command descriptor for enqueue.
+ */
+static struct dpaa2_qdma_comp *
+dpaa2_qdma_request_desc(struct dpaa2_qdma_chan *dpaa2_chan)
+{
+ struct dpaa2_qdma_comp *comp_temp = NULL;
+ unsigned long flags;
+
+ spin_lock_irqsave(&dpaa2_chan->queue_lock, flags);
+ if (list_empty(&dpaa2_chan->comp_free)) {
+ spin_unlock_irqrestore(&dpaa2_chan->queue_lock, flags);
+ comp_temp = kzalloc(sizeof(*comp_temp), GFP_KERNEL);
+ if (!comp_temp)
+ goto err;
+ comp_temp->fd_virt_addr =
+ dma_pool_alloc(dpaa2_chan->fd_pool, GFP_NOWAIT,
+ &comp_temp->fd_bus_addr);
+ if (!comp_temp->fd_virt_addr)
+ goto err;
+
+ comp_temp->fl_virt_addr =
+ (void *)((struct dpaa2_fd *)
+ comp_temp->fd_virt_addr + 1);
+ comp_temp->fl_bus_addr = comp_temp->fd_bus_addr +
+ sizeof(struct dpaa2_fd);
+ comp_temp->desc_virt_addr =
+ (void *)((struct dpaa2_fl_entry *)
+ comp_temp->fl_virt_addr + 3);
+ comp_temp->desc_bus_addr = comp_temp->fl_bus_addr +
+ sizeof(struct dpaa2_fl_entry) * 3;
+
+ comp_temp->qchan = dpaa2_chan;
+ return comp_temp;
+ }
+ comp_temp = list_first_entry(&dpaa2_chan->comp_free,
+ struct dpaa2_qdma_comp, list);
+ list_del(&comp_temp->list);
+ spin_unlock_irqrestore(&dpaa2_chan->queue_lock, flags);
+
+ comp_temp->qchan = dpaa2_chan;
+err:
+ return comp_temp;
+}
+
+static void
+dpaa2_qdma_populate_fd(u32 format, struct dpaa2_qdma_comp *dpaa2_comp)
+{
+ struct dpaa2_fd *fd;
+
+ fd = (struct dpaa2_fd *)dpaa2_comp->fd_virt_addr;
+ memset(fd, 0, sizeof(struct dpaa2_fd));
+
+ /* fd populated */
+ dpaa2_fd_set_addr(fd, dpaa2_comp->fl_bus_addr);
+ /* Bypass memory translation, Frame list format, short length disable */
+ /* we need to disable BMT if fsl-mc use iova addr */
+ if (smmu_disable)
+ dpaa2_fd_set_bpid(fd, QMAN_FD_BMT_ENABLE);
+ dpaa2_fd_set_format(fd, QMAN_FD_FMT_ENABLE | QMAN_FD_SL_DISABLE);
+
+ dpaa2_fd_set_frc(fd, format | QDMA_SER_CTX);
+}
+
+/* first frame list for descriptor buffer */
+static void
+dpaa2_qdma_populate_first_framel(struct dpaa2_fl_entry *f_list,
+ struct dpaa2_qdma_comp *dpaa2_comp,
+ bool wrt_changed)
+{
+ struct dpaa2_qdma_sd_d *sdd;
+
+ sdd = (struct dpaa2_qdma_sd_d *)dpaa2_comp->desc_virt_addr;
+ memset(sdd, 0, 2 * (sizeof(*sdd)));
+
+ /* source descriptor CMD */
+ sdd->cmd = cpu_to_le32(QDMA_SD_CMD_RDTTYPE_COHERENT);
+ sdd++;
+
+ /* dest descriptor CMD */
+ if (wrt_changed)
+ sdd->cmd = cpu_to_le32(LX2160_QDMA_DD_CMD_WRTTYPE_COHERENT);
+ else
+ sdd->cmd = cpu_to_le32(QDMA_DD_CMD_WRTTYPE_COHERENT);
+
+ memset(f_list, 0, sizeof(struct dpaa2_fl_entry));
+
+ /* first frame list to source descriptor */
+ dpaa2_fl_set_addr(f_list, dpaa2_comp->desc_bus_addr);
+ dpaa2_fl_set_len(f_list, 0x20);
+ dpaa2_fl_set_format(f_list, QDMA_FL_FMT_SBF | QDMA_FL_SL_LONG);
+
+ /* bypass memory translation */
+ if (smmu_disable)
+ f_list->bpid = cpu_to_le16(QDMA_FL_BMT_ENABLE);
+}
+
+/* source and destination frame list */
+static void
+dpaa2_qdma_populate_frames(struct dpaa2_fl_entry *f_list,
+ dma_addr_t dst, dma_addr_t src,
+ size_t len, uint8_t fmt)
+{
+ /* source frame list to source buffer */
+ memset(f_list, 0, sizeof(struct dpaa2_fl_entry));
+
+ dpaa2_fl_set_addr(f_list, src);
+ dpaa2_fl_set_len(f_list, len);
+
+ /* single buffer frame or scatter gather frame */
+ dpaa2_fl_set_format(f_list, (fmt | QDMA_FL_SL_LONG));
+
+ /* bypass memory translation */
+ if (smmu_disable)
+ f_list->bpid = cpu_to_le16(QDMA_FL_BMT_ENABLE);
+
+ f_list++;
+
+ /* destination frame list to destination buffer */
+ memset(f_list, 0, sizeof(struct dpaa2_fl_entry));
+
+ dpaa2_fl_set_addr(f_list, dst);
+ dpaa2_fl_set_len(f_list, len);
+ dpaa2_fl_set_format(f_list, (fmt | QDMA_FL_SL_LONG));
+ /* single buffer frame or scatter gather frame */
+ dpaa2_fl_set_final(f_list, QDMA_FL_F);
+ /* bypass memory translation */
+ if (smmu_disable)
+ f_list->bpid = cpu_to_le16(QDMA_FL_BMT_ENABLE);
+}
+
+static struct dma_async_tx_descriptor
+*dpaa2_qdma_prep_memcpy(struct dma_chan *chan, dma_addr_t dst,
+ dma_addr_t src, size_t len, ulong flags)
+{
+ struct dpaa2_qdma_chan *dpaa2_chan = to_dpaa2_qdma_chan(chan);
+ struct dpaa2_qdma_engine *dpaa2_qdma;
+ struct dpaa2_qdma_comp *dpaa2_comp;
+ struct dpaa2_fl_entry *f_list;
+ bool wrt_changed;
+ u32 format;
+
+ dpaa2_qdma = dpaa2_chan->qdma;
+ dpaa2_comp = dpaa2_qdma_request_desc(dpaa2_chan);
+ wrt_changed = (bool)dpaa2_qdma->qdma_wrtype_fixup;
+
+#ifdef LONG_FORMAT
+ format = QDMA_FD_LONG_FORMAT;
+#else
+ format = QDMA_FD_SHORT_FORMAT;
+#endif
+ /* populate Frame descriptor */
+ dpaa2_qdma_populate_fd(format, dpaa2_comp);
+
+ f_list = (struct dpaa2_fl_entry *)dpaa2_comp->fl_virt_addr;
+
+#ifdef LONG_FORMAT
+ /* first frame list for descriptor buffer (logn format) */
+ dpaa2_qdma_populate_first_framel(f_list, dpaa2_comp, wrt_changed);
+
+ f_list++;
+#endif
+
+ dpaa2_qdma_populate_frames(f_list, dst, src, len, QDMA_FL_FMT_SBF);
+
+ return vchan_tx_prep(&dpaa2_chan->vchan, &dpaa2_comp->vdesc, flags);
+}
+
+static enum
+dma_status dpaa2_qdma_tx_status(struct dma_chan *chan,
+ dma_cookie_t cookie,
+ struct dma_tx_state *txstate)
+{
+ return dma_cookie_status(chan, cookie, txstate);
+}
+
+static void dpaa2_qdma_issue_pending(struct dma_chan *chan)
+{
+ struct dpaa2_qdma_chan *dpaa2_chan = to_dpaa2_qdma_chan(chan);
+ struct dpaa2_qdma_engine *dpaa2_qdma = dpaa2_chan->qdma;
+ struct dpaa2_qdma_priv *priv = dpaa2_qdma->priv;
+ struct dpaa2_qdma_comp *dpaa2_comp;
+ struct virt_dma_desc *vdesc;
+ struct dpaa2_fd *fd;
+ unsigned long flags;
+ int err;
+
+ spin_lock_irqsave(&dpaa2_chan->queue_lock, flags);
+ spin_lock(&dpaa2_chan->vchan.lock);
+ if (vchan_issue_pending(&dpaa2_chan->vchan)) {
+ vdesc = vchan_next_desc(&dpaa2_chan->vchan);
+ if (!vdesc)
+ goto err_enqueue;
+ dpaa2_comp = to_fsl_qdma_comp(vdesc);
+
+ fd = (struct dpaa2_fd *)dpaa2_comp->fd_virt_addr;
+
+ list_del(&vdesc->node);
+ list_add_tail(&dpaa2_comp->list, &dpaa2_chan->comp_used);
+
+ /* TOBO: priority hard-coded to zero */
+ err = dpaa2_io_service_enqueue_fq(NULL,
+ priv->tx_queue_attr[0].fqid, fd);
+ if (err) {
+ list_del(&dpaa2_comp->list);
+ list_add_tail(&dpaa2_comp->list,
+ &dpaa2_chan->comp_free);
+ }
+ }
+err_enqueue:
+ spin_unlock(&dpaa2_chan->vchan.lock);
+ spin_unlock_irqrestore(&dpaa2_chan->queue_lock, flags);
+}
+
+static int __cold dpaa2_qdma_setup(struct fsl_mc_device *ls_dev)
+{
+ struct dpaa2_qdma_priv_per_prio *ppriv;
+ struct device *dev = &ls_dev->dev;
+ struct dpaa2_qdma_priv *priv;
+ u8 prio_def = DPDMAI_PRIO_NUM;
+ int err;
+ int i;
+
+ priv = dev_get_drvdata(dev);
+
+ priv->dev = dev;
+ priv->dpqdma_id = ls_dev->obj_desc.id;
+
+ /*Get the handle for the DPDMAI this interface is associate with */
+ err = dpdmai_open(priv->mc_io, 0, priv->dpqdma_id, &ls_dev->mc_handle);
+ if (err) {
+ dev_err(dev, "dpdmai_open() failed\n");
+ return err;
+ }
+ dev_info(dev, "Opened dpdmai object successfully\n");
+
+ err = dpdmai_get_attributes(priv->mc_io, 0, ls_dev->mc_handle,
+ &priv->dpdmai_attr);
+ if (err) {
+ dev_err(dev, "dpdmai_get_attributes() failed\n");
+ return err;
+ }
+
+ if (priv->dpdmai_attr.version.major > DPDMAI_VER_MAJOR) {
+ dev_err(dev, "DPDMAI major version mismatch\n"
+ "Found %u.%u, supported version is %u.%u\n",
+ priv->dpdmai_attr.version.major,
+ priv->dpdmai_attr.version.minor,
+ DPDMAI_VER_MAJOR, DPDMAI_VER_MINOR);
+ }
+
+ if (priv->dpdmai_attr.version.minor > DPDMAI_VER_MINOR) {
+ dev_err(dev, "DPDMAI minor version mismatch\n"
+ "Found %u.%u, supported version is %u.%u\n",
+ priv->dpdmai_attr.version.major,
+ priv->dpdmai_attr.version.minor,
+ DPDMAI_VER_MAJOR, DPDMAI_VER_MINOR);
+ }
+
+ priv->num_pairs = min(priv->dpdmai_attr.num_of_priorities, prio_def);
+ ppriv = kcalloc(priv->num_pairs, sizeof(*ppriv), GFP_KERNEL);
+ if (!ppriv) {
+ dev_err(dev, "kzalloc for ppriv failed\n");
+ return -1;
+ }
+ priv->ppriv = ppriv;
+
+ for (i = 0; i < priv->num_pairs; i++) {
+ err = dpdmai_get_rx_queue(priv->mc_io, 0, ls_dev->mc_handle,
+ i, &priv->rx_queue_attr[i]);
+ if (err) {
+ dev_err(dev, "dpdmai_get_rx_queue() failed\n");
+ return err;
+ }
+ ppriv->rsp_fqid = priv->rx_queue_attr[i].fqid;
+
+ err = dpdmai_get_tx_queue(priv->mc_io, 0, ls_dev->mc_handle,
+ i, &priv->tx_queue_attr[i]);
+ if (err) {
+ dev_err(dev, "dpdmai_get_tx_queue() failed\n");
+ return err;
+ }
+ ppriv->req_fqid = priv->tx_queue_attr[i].fqid;
+ ppriv->prio = i;
+ ppriv->priv = priv;
+ ppriv++;
+ }
+
+ return 0;
+}
+
+static void dpaa2_qdma_fqdan_cb(struct dpaa2_io_notification_ctx *ctx)
+{
+ struct dpaa2_qdma_priv_per_prio *ppriv = container_of(ctx,
+ struct dpaa2_qdma_priv_per_prio, nctx);
+ struct dpaa2_qdma_comp *dpaa2_comp, *_comp_tmp;
+ struct dpaa2_qdma_priv *priv = ppriv->priv;
+ u32 n_chans = priv->dpaa2_qdma->n_chans;
+ struct dpaa2_qdma_chan *qchan;
+ const struct dpaa2_fd *fd_eq;
+ const struct dpaa2_fd *fd;
+ struct dpaa2_dq *dq;
+ int is_last = 0;
+ int found;
+ u8 status;
+ int err;
+ int i;
+
+ do {
+ err = dpaa2_io_service_pull_fq(NULL, ppriv->rsp_fqid,
+ ppriv->store);
+ } while (err);
+
+ while (!is_last) {
+ do {
+ dq = dpaa2_io_store_next(ppriv->store, &is_last);
+ } while (!is_last && !dq);
+ if (!dq) {
+ dev_err(priv->dev, "FQID returned no valid frames!\n");
+ continue;
+ }
+
+ /* obtain FD and process the error */
+ fd = dpaa2_dq_fd(dq);
+
+ status = dpaa2_fd_get_ctrl(fd) & 0xff;
+ if (status)
+ dev_err(priv->dev, "FD error occurred\n");
+ found = 0;
+ for (i = 0; i < n_chans; i++) {
+ qchan = &priv->dpaa2_qdma->chans[i];
+ spin_lock(&qchan->queue_lock);
+ if (list_empty(&qchan->comp_used)) {
+ spin_unlock(&qchan->queue_lock);
+ continue;
+ }
+ list_for_each_entry_safe(dpaa2_comp, _comp_tmp,
+ &qchan->comp_used, list) {
+ fd_eq = (struct dpaa2_fd *)
+ dpaa2_comp->fd_virt_addr;
+
+ if (le64_to_cpu(fd_eq->simple.addr) ==
+ le64_to_cpu(fd->simple.addr)) {
+ spin_lock(&qchan->vchan.lock);
+ vchan_cookie_complete(&
+ dpaa2_comp->vdesc);
+ spin_unlock(&qchan->vchan.lock);
+ found = 1;
+ break;
+ }
+ }
+ spin_unlock(&qchan->queue_lock);
+ if (found)
+ break;
+ }
+ }
+
+ dpaa2_io_service_rearm(NULL, ctx);
+}
+
+static int __cold dpaa2_qdma_dpio_setup(struct dpaa2_qdma_priv *priv)
+{
+ struct dpaa2_qdma_priv_per_prio *ppriv;
+ struct device *dev = priv->dev;
+ int err, i, num;
+
+ num = priv->num_pairs;
+ ppriv = priv->ppriv;
+ for (i = 0; i < num; i++) {
+ ppriv->nctx.is_cdan = 0;
+ ppriv->nctx.desired_cpu = 1;
+ ppriv->nctx.id = ppriv->rsp_fqid;
+ ppriv->nctx.cb = dpaa2_qdma_fqdan_cb;
+ err = dpaa2_io_service_register(NULL, &ppriv->nctx, dev);
+ if (err) {
+ dev_err(dev, "Notification register failed\n");
+ goto err_service;
+ }
+
+ ppriv->store =
+ dpaa2_io_store_create(DPAA2_QDMA_STORE_SIZE, dev);
+ if (!ppriv->store) {
+ dev_err(dev, "dpaa2_io_store_create() failed\n");
+ goto err_store;
+ }
+
+ ppriv++;
+ }
+ return 0;
+
+err_store:
+ dpaa2_io_service_deregister(NULL, &ppriv->nctx, dev);
+err_service:
+ ppriv--;
+ while (ppriv >= priv->ppriv) {
+ dpaa2_io_service_deregister(NULL, &ppriv->nctx, dev);
+ dpaa2_io_store_destroy(ppriv->store);
+ ppriv--;
+ }
+ return -1;
+}
+
+static void dpaa2_dpmai_store_free(struct dpaa2_qdma_priv *priv)
+{
+ struct dpaa2_qdma_priv_per_prio *ppriv = priv->ppriv;
+ int i;
+
+ for (i = 0; i < priv->num_pairs; i++) {
+ dpaa2_io_store_destroy(ppriv->store);
+ ppriv++;
+ }
+}
+
+static void dpaa2_dpdmai_dpio_free(struct dpaa2_qdma_priv *priv)
+{
+ struct dpaa2_qdma_priv_per_prio *ppriv = priv->ppriv;
+ struct device *dev = priv->dev;
+ int i;
+
+ for (i = 0; i < priv->num_pairs; i++) {
+ dpaa2_io_service_deregister(NULL, &ppriv->nctx, dev);
+ ppriv++;
+ }
+}
+
+static int __cold dpaa2_dpdmai_bind(struct dpaa2_qdma_priv *priv)
+{
+ int err;
+ int i, num;
+ struct device *dev = priv->dev;
+ struct dpaa2_qdma_priv_per_prio *ppriv;
+ struct dpdmai_rx_queue_cfg rx_queue_cfg;
+ struct fsl_mc_device *ls_dev = to_fsl_mc_device(dev);
+
+ num = priv->num_pairs;
+ ppriv = priv->ppriv;
+ for (i = 0; i < num; i++) {
+ rx_queue_cfg.options = DPDMAI_QUEUE_OPT_USER_CTX |
+ DPDMAI_QUEUE_OPT_DEST;
+ rx_queue_cfg.user_ctx = ppriv->nctx.qman64;
+ rx_queue_cfg.dest_cfg.dest_type = DPDMAI_DEST_DPIO;
+ rx_queue_cfg.dest_cfg.dest_id = ppriv->nctx.dpio_id;
+ rx_queue_cfg.dest_cfg.priority = ppriv->prio;
+ err = dpdmai_set_rx_queue(priv->mc_io, 0, ls_dev->mc_handle,
+ rx_queue_cfg.dest_cfg.priority,
+ &rx_queue_cfg);
+ if (err) {
+ dev_err(dev, "dpdmai_set_rx_queue() failed\n");
+ return err;
+ }
+
+ ppriv++;
+ }
+
+ return 0;
+}
+
+static int __cold dpaa2_dpdmai_dpio_unbind(struct dpaa2_qdma_priv *priv)
+{
+ int i;
+ int err = 0;
+ struct device *dev = priv->dev;
+ struct fsl_mc_device *ls_dev = to_fsl_mc_device(dev);
+ struct dpaa2_qdma_priv_per_prio *ppriv = priv->ppriv;
+
+ for (i = 0; i < priv->num_pairs; i++) {
+ ppriv->nctx.qman64 = 0;
+ ppriv->nctx.dpio_id = 0;
+ ppriv++;
+ }
+
+ err = dpdmai_reset(priv->mc_io, 0, ls_dev->mc_handle);
+ if (err)
+ dev_err(dev, "dpdmai_reset() failed\n");
+
+ return err;
+}
+
+static void dpaa2_dpdmai_free_comp(struct dpaa2_qdma_chan *qchan,
+ struct list_head *head)
+{
+ struct dpaa2_qdma_comp *comp_tmp, *_comp_tmp;
+
+ list_for_each_entry_safe(comp_tmp, _comp_tmp,
+ head, list) {
+ dma_pool_free(qchan->fd_pool,
+ comp_tmp->fd_virt_addr,
+ comp_tmp->fd_bus_addr);
+ list_del(&comp_tmp->list);
+ kfree(comp_tmp);
+ }
+}
+
+static void dpaa2_dpdmai_free_channels(struct dpaa2_qdma_engine *dpaa2_qdma)
+{
+ struct dpaa2_qdma_chan *qchan;
+ int num, i;
+
+ num = dpaa2_qdma->n_chans;
+ for (i = 0; i < num; i++) {
+ qchan = &dpaa2_qdma->chans[i];
+ dpaa2_dpdmai_free_comp(qchan, &qchan->comp_used);
+ dpaa2_dpdmai_free_comp(qchan, &qchan->comp_free);
+ dma_pool_destroy(qchan->fd_pool);
+ }
+}
+
+static void dpaa2_qdma_free_desc(struct virt_dma_desc *vdesc)
+{
+ struct dpaa2_qdma_comp *dpaa2_comp;
+ struct dpaa2_qdma_chan *qchan;
+
+ dpaa2_comp = to_fsl_qdma_comp(vdesc);
+ qchan = dpaa2_comp->qchan;
+ list_del(&dpaa2_comp->list);
+ list_add_tail(&dpaa2_comp->list, &qchan->comp_free);
+}
+
+static int dpaa2_dpdmai_init_channels(struct dpaa2_qdma_engine *dpaa2_qdma)
+{
+ struct dpaa2_qdma_chan *dpaa2_chan;
+ int i;
+
+ INIT_LIST_HEAD(&dpaa2_qdma->dma_dev.channels);
+ for (i = 0; i < dpaa2_qdma->n_chans; i++) {
+ dpaa2_chan = &dpaa2_qdma->chans[i];
+ dpaa2_chan->qdma = dpaa2_qdma;
+ dpaa2_chan->vchan.desc_free = dpaa2_qdma_free_desc;
+ vchan_init(&dpaa2_chan->vchan, &dpaa2_qdma->dma_dev);
+ spin_lock_init(&dpaa2_chan->queue_lock);
+ INIT_LIST_HEAD(&dpaa2_chan->comp_used);
+ INIT_LIST_HEAD(&dpaa2_chan->comp_free);
+ }
+ return 0;
+}
+
+static int dpaa2_qdma_probe(struct fsl_mc_device *dpdmai_dev)
+{
+ struct dpaa2_qdma_priv *priv;
+ struct device *dev = &dpdmai_dev->dev;
+ struct dpaa2_qdma_engine *dpaa2_qdma;
+ int err;
+
+ priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+ dev_set_drvdata(dev, priv);
+ priv->dpdmai_dev = dpdmai_dev;
+
+ priv->iommu_domain = iommu_get_domain_for_dev(dev);
+ if (priv->iommu_domain)
+ smmu_disable = false;
+
+ /* obtain a MC portal */
+ err = fsl_mc_portal_allocate(dpdmai_dev, 0, &priv->mc_io);
+ if (err) {
+ if (err == -ENXIO)
+ err = -EPROBE_DEFER;
+ else
+ dev_err(dev, "MC portal allocation failed\n");
+ goto err_mcportal;
+ }
+
+ /* DPDMAI initialization */
+ err = dpaa2_qdma_setup(dpdmai_dev);
+ if (err) {
+ dev_err(dev, "dpaa2_dpdmai_setup() failed\n");
+ goto err_dpdmai_setup;
+ }
+
+ /* DPIO */
+ err = dpaa2_qdma_dpio_setup(priv);
+ if (err) {
+ dev_err(dev, "dpaa2_dpdmai_dpio_setup() failed\n");
+ goto err_dpio_setup;
+ }
+
+ /* DPDMAI binding to DPIO */
+ err = dpaa2_dpdmai_bind(priv);
+ if (err) {
+ dev_err(dev, "dpaa2_dpdmai_bind() failed\n");
+ goto err_bind;
+ }
+
+ /* DPDMAI enable */
+ err = dpdmai_enable(priv->mc_io, 0, dpdmai_dev->mc_handle);
+ if (err) {
+ dev_err(dev, "dpdmai_enable() faile\n");
+ goto err_enable;
+ }
+
+ dpaa2_qdma = kzalloc(sizeof(*dpaa2_qdma), GFP_KERNEL);
+ if (!dpaa2_qdma) {
+ err = -ENOMEM;
+ goto err_eng;
+ }
+
+ priv->dpaa2_qdma = dpaa2_qdma;
+ dpaa2_qdma->priv = priv;
+
+ dpaa2_qdma->desc_allocated = 0;
+ dpaa2_qdma->n_chans = NUM_CH;
+
+ dpaa2_dpdmai_init_channels(dpaa2_qdma);
+
+ if (soc_device_match(soc_fixup_tuning))
+ dpaa2_qdma->qdma_wrtype_fixup = true;
+ else
+ dpaa2_qdma->qdma_wrtype_fixup = false;
+
+ dma_cap_set(DMA_PRIVATE, dpaa2_qdma->dma_dev.cap_mask);
+ dma_cap_set(DMA_SLAVE, dpaa2_qdma->dma_dev.cap_mask);
+ dma_cap_set(DMA_MEMCPY, dpaa2_qdma->dma_dev.cap_mask);
+
+ dpaa2_qdma->dma_dev.dev = dev;
+ dpaa2_qdma->dma_dev.device_alloc_chan_resources =
+ dpaa2_qdma_alloc_chan_resources;
+ dpaa2_qdma->dma_dev.device_free_chan_resources =
+ dpaa2_qdma_free_chan_resources;
+ dpaa2_qdma->dma_dev.device_tx_status = dpaa2_qdma_tx_status;
+ dpaa2_qdma->dma_dev.device_prep_dma_memcpy = dpaa2_qdma_prep_memcpy;
+ dpaa2_qdma->dma_dev.device_issue_pending = dpaa2_qdma_issue_pending;
+
+ err = dma_async_device_register(&dpaa2_qdma->dma_dev);
+ if (err) {
+ dev_err(dev, "Can't register NXP QDMA engine.\n");
+ goto err_eng;
+ }
+
+ return 0;
+
+err_eng:
+ dpdmai_disable(priv->mc_io, 0, dpdmai_dev->mc_handle);
+err_enable:
+ dpaa2_dpdmai_dpio_unbind(priv);
+err_bind:
+ dpaa2_dpmai_store_free(priv);
+ dpaa2_dpdmai_dpio_free(priv);
+err_dpio_setup:
+ dpdmai_close(priv->mc_io, 0, dpdmai_dev->mc_handle);
+err_dpdmai_setup:
+ fsl_mc_portal_free(priv->mc_io);
+err_mcportal:
+ kfree(priv->ppriv);
+ kfree(priv);
+ dev_set_drvdata(dev, NULL);
+ return err;
+}
+
+static int dpaa2_qdma_remove(struct fsl_mc_device *ls_dev)
+{
+ struct device *dev;
+ struct dpaa2_qdma_priv *priv;
+ struct dpaa2_qdma_engine *dpaa2_qdma;
+
+ dev = &ls_dev->dev;
+ priv = dev_get_drvdata(dev);
+ dpaa2_qdma = priv->dpaa2_qdma;
+
+ dpdmai_disable(priv->mc_io, 0, ls_dev->mc_handle);
+ dpaa2_dpdmai_dpio_unbind(priv);
+ dpaa2_dpmai_store_free(priv);
+ dpaa2_dpdmai_dpio_free(priv);
+ dpdmai_close(priv->mc_io, 0, ls_dev->mc_handle);
+ fsl_mc_portal_free(priv->mc_io);
+ dev_set_drvdata(dev, NULL);
+ dpaa2_dpdmai_free_channels(dpaa2_qdma);
+
+ dma_async_device_unregister(&dpaa2_qdma->dma_dev);
+ kfree(priv);
+ kfree(dpaa2_qdma);
+
+ return 0;
+}
+
+static const struct fsl_mc_device_id dpaa2_qdma_id_table[] = {
+ {
+ .vendor = FSL_MC_VENDOR_FREESCALE,
+ .obj_type = "dpdmai",
+ },
+ { .vendor = 0x0 }
+};
+
+static struct fsl_mc_driver dpaa2_qdma_driver = {
+ .driver = {
+ .name = "dpaa2-qdma",
+ .owner = THIS_MODULE,
+ },
+ .probe = dpaa2_qdma_probe,
+ .remove = dpaa2_qdma_remove,
+ .match_id_table = dpaa2_qdma_id_table
+};
+
+static int __init dpaa2_qdma_driver_init(void)
+{
+ return fsl_mc_driver_register(&(dpaa2_qdma_driver));
+}
+late_initcall(dpaa2_qdma_driver_init);
+
+static void __exit fsl_qdma_exit(void)
+{
+ fsl_mc_driver_unregister(&(dpaa2_qdma_driver));
+}
+module_exit(fsl_qdma_exit);
+
+MODULE_ALIAS("platform:fsl-dpaa2-qdma");
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("NXP Layerscape DPAA2 qDMA engine driver");
diff --git a/drivers/dma/fsl-dpaa2-qdma/dpaa2-qdma.h b/drivers/dma/fsl-dpaa2-qdma/dpaa2-qdma.h
new file mode 100644
index 0000000..3fcade8
--- /dev/null
+++ b/drivers/dma/fsl-dpaa2-qdma/dpaa2-qdma.h
@@ -0,0 +1,152 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright 2014-2018 NXP */
+
+#ifndef __DPAA2_QDMA_H
+#define __DPAA2_QDMA_H
+
+#define LONG_FORMAT 1
+
+#define DPAA2_QDMA_STORE_SIZE 16
+#define NUM_CH 8
+
+struct dpaa2_qdma_sd_d {
+ u32 rsv:32;
+ union {
+ struct {
+ u32 ssd:12; /* souce stride distance */
+ u32 sss:12; /* souce stride size */
+ u32 rsv1:8;
+ } sdf;
+ struct {
+ u32 dsd:12; /* Destination stride distance */
+ u32 dss:12; /* Destination stride size */
+ u32 rsv2:8;
+ } ddf;
+ } df;
+ u32 rbpcmd; /* Route-by-port command */
+ u32 cmd;
+} __attribute__((__packed__));
+
+/* Source descriptor command read transaction type for RBP=0: */
+/* coherent copy of cacheable memory */
+#define QDMA_SD_CMD_RDTTYPE_COHERENT (0xb << 28)
+/* Destination descriptor command write transaction type for RBP=0: */
+/* coherent copy of cacheable memory */
+#define QDMA_DD_CMD_WRTTYPE_COHERENT (0x6 << 28)
+#define LX2160_QDMA_DD_CMD_WRTTYPE_COHERENT (0xb << 28)
+
+#define QMAN_FD_FMT_ENABLE BIT(0) /* frame list table enable */
+#define QMAN_FD_BMT_ENABLE BIT(15) /* bypass memory translation */
+#define QMAN_FD_BMT_DISABLE (0) /* bypass memory translation */
+#define QMAN_FD_SL_DISABLE (0) /* short lengthe disabled */
+#define QMAN_FD_SL_ENABLE BIT(14) /* short lengthe enabled */
+
+#define QDMA_FINAL_BIT_DISABLE (0) /* final bit disable */
+#define QDMA_FINAL_BIT_ENABLE BIT(31) /* final bit enable */
+
+#define QDMA_FD_SHORT_FORMAT BIT(11) /* short format */
+#define QDMA_FD_LONG_FORMAT (0) /* long format */
+#define QDMA_SER_DISABLE (8) /* no notification */
+#define QDMA_SER_CTX BIT(8) /* notification by FQD_CTX[fqid] */
+#define QDMA_SER_DEST (2 << 8) /* notification by destination desc */
+#define QDMA_SER_BOTH (3 << 8) /* soruce and dest notification */
+#define QDMA_FD_SPF_ENALBE BIT(30) /* source prefetch enable */
+
+#define QMAN_FD_VA_ENABLE BIT(14) /* Address used is virtual address */
+#define QMAN_FD_VA_DISABLE (0)/* Address used is a real address */
+/* Flow Context: 49bit physical address */
+#define QMAN_FD_CBMT_ENABLE BIT(15)
+#define QMAN_FD_CBMT_DISABLE (0) /* Flow Context: 64bit virtual address */
+#define QMAN_FD_SC_DISABLE (0) /* stashing control */
+
+#define QDMA_FL_FMT_SBF (0x0) /* Single buffer frame */
+#define QDMA_FL_FMT_SGE (0x2) /* Scatter gather frame */
+#define QDMA_FL_BMT_ENABLE BIT(15) /* enable bypass memory translation */
+#define QDMA_FL_BMT_DISABLE (0x0) /* enable bypass memory translation */
+#define QDMA_FL_SL_LONG (0x0)/* long length */
+#define QDMA_FL_SL_SHORT (0x1) /* short length */
+#define QDMA_FL_F (0x1)/* last frame list bit */
+
+/*Description of Frame list table structure*/
+struct dpaa2_qdma_chan {
+ struct virt_dma_chan vchan;
+ struct virt_dma_desc vdesc;
+ enum dma_status status;
+ struct dpaa2_qdma_engine *qdma;
+
+ struct mutex dpaa2_queue_mutex;
+ spinlock_t queue_lock;
+ struct dma_pool *fd_pool;
+
+ struct list_head comp_used;
+ struct list_head comp_free;
+
+};
+
+struct dpaa2_qdma_comp {
+ dma_addr_t fd_bus_addr;
+ dma_addr_t fl_bus_addr;
+ dma_addr_t desc_bus_addr;
+ void *fd_virt_addr;
+ void *fl_virt_addr;
+ void *desc_virt_addr;
+ struct dpaa2_qdma_chan *qchan;
+ struct virt_dma_desc vdesc;
+ struct list_head list;
+};
+
+struct dpaa2_qdma_engine {
+ struct dma_device dma_dev;
+ u32 n_chans;
+ struct dpaa2_qdma_chan chans[NUM_CH];
+ int qdma_wrtype_fixup;
+ int desc_allocated;
+
+ struct dpaa2_qdma_priv *priv;
+};
+
+/*
+ * dpaa2_qdma_priv - driver private data
+ */
+struct dpaa2_qdma_priv {
+ int dpqdma_id;
+
+ struct iommu_domain *iommu_domain;
+ struct dpdmai_attr dpdmai_attr;
+ struct device *dev;
+ struct fsl_mc_io *mc_io;
+ struct fsl_mc_device *dpdmai_dev;
+ u8 num_pairs;
+
+ struct dpaa2_qdma_engine *dpaa2_qdma;
+ struct dpaa2_qdma_priv_per_prio *ppriv;
+
+ struct dpdmai_rx_queue_attr rx_queue_attr[DPDMAI_PRIO_NUM];
+ struct dpdmai_tx_queue_attr tx_queue_attr[DPDMAI_PRIO_NUM];
+};
+
+struct dpaa2_qdma_priv_per_prio {
+ int req_fqid;
+ int rsp_fqid;
+ int prio;
+
+ struct dpaa2_io_store *store;
+ struct dpaa2_io_notification_ctx nctx;
+
+ struct dpaa2_qdma_priv *priv;
+};
+
+static struct soc_device_attribute soc_fixup_tuning[] = {
+ { .family = "QorIQ LX2160A"},
+ { },
+};
+
+/* FD pool size: one FD + 3 Frame list + 2 source/destination descriptor */
+#define FD_POOL_SIZE (sizeof(struct dpaa2_fd) + \
+ sizeof(struct dpaa2_fl_entry) * 3 + \
+ sizeof(struct dpaa2_qdma_sd_d) * 2)
+
+static void dpaa2_dpdmai_free_channels(struct dpaa2_qdma_engine *dpaa2_qdma);
+static void dpaa2_dpdmai_free_comp(struct dpaa2_qdma_chan *qchan,
+ struct list_head *head);
+#endif /* __DPAA2_QDMA_H */
^ permalink raw reply related
* [V3,1/2] dmaengine: fsl-dpaa2-qdma: Add the DPDMAI(Data Path DMA Interface) support
From: Peng Ma @ 2019-04-09 7:22 UTC (permalink / raw)
To: vkoul, dan.j.williams, leoyang.li; +Cc: linux-kernel, dmaengine, Peng Ma
The MC exports the DPDMAI object as an interface to operate the DPAA2 QDMA
Engine. The DPDMAI enables sending frame-based requests to QDMA and receiving
back confirmation response on transaction completion, utilizing the DPAA2 QBMan
infrastructure. DPDMAI object provides up to two priorities for processing QDMA
requests.
The following list summarizes the DPDMAI main features and capabilities:
1. Supports up to two scheduling priorities for processing service
requests.
- Each DPDMAI transmit queue is mapped to one of two service priorities,
allowing further prioritization in hardware between requests from
different DPDMAI objects.
2. Supports up to two receive queues for incoming transaction completion
confirmations.
- Each DPDMAI receive queue is mapped to one of two receive priorities,
allowing further prioritization between other interfaces when associating
the DPDMAI receive queues to DPIO or DPCON objects.
3. Supports different scheduling options for processing received packets:
- Queues can be configured either in 'parked' mode (default), oattached
to a DPIO object, or attached to DPCON object.
4. Allows interaction with one or more DPIO objects for dequeueing/enqueueing
frame descriptors(FD) and for acquiring/releasing buffers.
5. Supports enable, disable, and reset operations.
Add dpdmai to support some platforms with dpaa2 qdma engine.
Signed-off-by: Peng Ma <peng.ma@nxp.com>
---
changed for v3:
- no changed
drivers/dma/fsl-dpaa2-qdma/dpdmai.c | 483 ++++++++++++++++++++++++++++
drivers/dma/fsl-dpaa2-qdma/dpdmai.h | 524 +++++++++++++++++++++++++++++++
drivers/dma/fsl-dpaa2-qdma/dpdmai_cmd.h | 197 ++++++++++++
3 files changed, 1204 insertions(+), 0 deletions(-)
create mode 100644 drivers/dma/fsl-dpaa2-qdma/dpdmai.c
create mode 100644 drivers/dma/fsl-dpaa2-qdma/dpdmai.h
create mode 100644 drivers/dma/fsl-dpaa2-qdma/dpdmai_cmd.h
diff --git a/drivers/dma/fsl-dpaa2-qdma/dpdmai.c b/drivers/dma/fsl-dpaa2-qdma/dpdmai.c
new file mode 100644
index 0000000..685eabe
--- /dev/null
+++ b/drivers/dma/fsl-dpaa2-qdma/dpdmai.c
@@ -0,0 +1,483 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright 2014-2018 NXP
+
+#include <linux/types.h>
+#include <linux/io.h>
+#include "dpdmai.h"
+#include "dpdmai_cmd.h"
+#include <linux/fsl/mc.h>
+
+struct dpdmai_cmd_open {
+ __le32 dpdmai_id;
+};
+
+struct dpdmai_rsp_get_attributes {
+ __le32 id;
+ u8 num_of_priorities;
+ u8 pad0[3];
+ __le16 major;
+ __le16 minor;
+};
+
+struct dpdmai_cmd_queue {
+ __le32 dest_id;
+ u8 priority;
+ u8 queue;
+ u8 dest_type;
+ u8 pad;
+ __le64 user_ctx;
+ union {
+ __le32 options;
+ __le32 fqid;
+ };
+};
+
+struct dpdmai_rsp_get_tx_queue {
+ __le64 pad;
+ __le32 fqid;
+};
+
+int dpdmai_open(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ int dpdmai_id,
+ u16 *token)
+{
+ struct fsl_mc_command cmd = { 0 };
+ struct dpdmai_cmd_open *cmd_params;
+ int err;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_OPEN,
+ cmd_flags,
+ 0);
+
+ cmd_params = (struct dpdmai_cmd_open *)cmd.params;
+ cmd_params->dpdmai_id = cpu_to_le32(dpdmai_id);
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+ *token = mc_cmd_hdr_read_token(&cmd);
+ return 0;
+}
+
+int dpdmai_close(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token)
+{
+ struct fsl_mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_CLOSE,
+ cmd_flags, token);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpdmai_create(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ const struct dpdmai_cfg *cfg,
+ u16 *token)
+{
+ struct fsl_mc_command cmd = { 0 };
+ int err;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_CREATE,
+ cmd_flags,
+ 0);
+ DPDMAI_CMD_CREATE(cmd, cfg);
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+ *token = MC_CMD_HDR_READ_TOKEN(cmd.header);
+
+ return 0;
+}
+
+int dpdmai_destroy(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token)
+{
+ struct fsl_mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_DESTROY,
+ cmd_flags,
+ token);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpdmai_enable(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token)
+{
+ struct fsl_mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_ENABLE,
+ cmd_flags,
+ token);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpdmai_disable(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token)
+{
+ struct fsl_mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_DISABLE,
+ cmd_flags,
+ token);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpdmai_is_enabled(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ int *en)
+{
+ struct fsl_mc_command cmd = { 0 };
+ int err;
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_IS_ENABLED,
+ cmd_flags,
+ token);
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+ DPDMAI_RSP_IS_ENABLED(cmd, *en);
+
+ return 0;
+}
+
+int dpdmai_reset(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token)
+{
+ struct fsl_mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_RESET,
+ cmd_flags,
+ token);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpdmai_get_irq(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ int *type,
+ struct dpdmai_irq_cfg *irq_cfg)
+{
+ struct fsl_mc_command cmd = { 0 };
+ int err;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_GET_IRQ,
+ cmd_flags,
+ token);
+ DPDMAI_CMD_GET_IRQ(cmd, irq_index);
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+ DPDMAI_RSP_GET_IRQ(cmd, *type, irq_cfg);
+
+ return 0;
+}
+
+int dpdmai_set_irq(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ struct dpdmai_irq_cfg *irq_cfg)
+{
+ struct fsl_mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_SET_IRQ,
+ cmd_flags,
+ token);
+ DPDMAI_CMD_SET_IRQ(cmd, irq_index, irq_cfg);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpdmai_get_irq_enable(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u8 *en)
+{
+ struct fsl_mc_command cmd = { 0 };
+ int err;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_GET_IRQ_ENABLE,
+ cmd_flags,
+ token);
+ DPDMAI_CMD_GET_IRQ_ENABLE(cmd, irq_index);
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+ DPDMAI_RSP_GET_IRQ_ENABLE(cmd, *en);
+
+ return 0;
+}
+
+int dpdmai_set_irq_enable(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u8 en)
+{
+ struct fsl_mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_SET_IRQ_ENABLE,
+ cmd_flags,
+ token);
+ DPDMAI_CMD_SET_IRQ_ENABLE(cmd, irq_index, en);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpdmai_get_irq_mask(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u32 *mask)
+{
+ struct fsl_mc_command cmd = { 0 };
+ int err;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_GET_IRQ_MASK,
+ cmd_flags,
+ token);
+ DPDMAI_CMD_GET_IRQ_MASK(cmd, irq_index);
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+ DPDMAI_RSP_GET_IRQ_MASK(cmd, *mask);
+
+ return 0;
+}
+
+int dpdmai_set_irq_mask(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u32 mask)
+{
+ struct fsl_mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_SET_IRQ_MASK,
+ cmd_flags,
+ token);
+ DPDMAI_CMD_SET_IRQ_MASK(cmd, irq_index, mask);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpdmai_get_irq_status(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u32 *status)
+{
+ struct fsl_mc_command cmd = { 0 };
+ int err;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_GET_IRQ_STATUS,
+ cmd_flags,
+ token);
+ DPDMAI_CMD_GET_IRQ_STATUS(cmd, irq_index, *status);
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+ DPDMAI_RSP_GET_IRQ_STATUS(cmd, *status);
+
+ return 0;
+}
+
+int dpdmai_clear_irq_status(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u32 status)
+{
+ struct fsl_mc_command cmd = { 0 };
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_CLEAR_IRQ_STATUS,
+ cmd_flags,
+ token);
+ DPDMAI_CMD_CLEAR_IRQ_STATUS(cmd, irq_index, status);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpdmai_get_attributes(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ struct dpdmai_attr *attr)
+{
+ struct fsl_mc_command cmd = { 0 };
+ int err;
+ struct dpdmai_rsp_get_attributes *rsp_params;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_GET_ATTR,
+ cmd_flags,
+ token);
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+ rsp_params = (struct dpdmai_rsp_get_attributes *)cmd.params;
+ attr->id = le32_to_cpu(rsp_params->id);
+ attr->version.major = le16_to_cpu(rsp_params->major);
+ attr->version.minor = le16_to_cpu(rsp_params->minor);
+ attr->num_of_priorities = rsp_params->num_of_priorities;
+
+ return 0;
+}
+
+int dpdmai_set_rx_queue(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 priority,
+ const struct dpdmai_rx_queue_cfg *cfg)
+{
+ struct fsl_mc_command cmd = { 0 };
+ struct dpdmai_cmd_queue *cmd_params;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_SET_RX_QUEUE,
+ cmd_flags,
+ token);
+
+ cmd_params = (struct dpdmai_cmd_queue *)cmd.params;
+ cmd_params->dest_id = cpu_to_le32(cfg->dest_cfg.dest_id);
+ cmd_params->priority = cfg->dest_cfg.priority;
+ cmd_params->queue = priority;
+ cmd_params->dest_type = cfg->dest_cfg.dest_type;
+ cmd_params->user_ctx = cpu_to_le64(cfg->user_ctx);
+ cmd_params->options = cpu_to_le32(cfg->options);
+
+ /* send command to mc*/
+ return mc_send_command(mc_io, &cmd);
+}
+
+int dpdmai_get_rx_queue(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 priority, struct dpdmai_rx_queue_attr *attr)
+{
+ struct fsl_mc_command cmd = { 0 };
+ struct dpdmai_cmd_queue *cmd_params;
+ int err;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_GET_RX_QUEUE,
+ cmd_flags,
+ token);
+
+ cmd_params = (struct dpdmai_cmd_queue *)cmd.params;
+ cmd_params->queue = priority;
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+ attr->dest_cfg.dest_id = le32_to_cpu(cmd_params->dest_id);
+ attr->dest_cfg.priority = cmd_params->priority;
+ attr->dest_cfg.dest_type = cmd_params->dest_type;
+ attr->user_ctx = le64_to_cpu(cmd_params->user_ctx);
+ attr->fqid = le32_to_cpu(cmd_params->fqid);
+
+ return 0;
+}
+
+int dpdmai_get_tx_queue(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 priority,
+ struct dpdmai_tx_queue_attr *attr)
+{
+ struct fsl_mc_command cmd = { 0 };
+ struct dpdmai_cmd_queue *cmd_params;
+ struct dpdmai_rsp_get_tx_queue *rsp_params;
+ int err;
+
+ /* prepare command */
+ cmd.header = mc_encode_cmd_header(DPDMAI_CMDID_GET_TX_QUEUE,
+ cmd_flags,
+ token);
+
+ cmd_params = (struct dpdmai_cmd_queue *)cmd.params;
+ cmd_params->queue = priority;
+
+ /* send command to mc*/
+ err = mc_send_command(mc_io, &cmd);
+ if (err)
+ return err;
+
+ /* retrieve response parameters */
+
+ rsp_params = (struct dpdmai_rsp_get_tx_queue *)cmd.params;
+ attr->fqid = le32_to_cpu(rsp_params->fqid);
+
+ return 0;
+}
diff --git a/drivers/dma/fsl-dpaa2-qdma/dpdmai.h b/drivers/dma/fsl-dpaa2-qdma/dpdmai.h
new file mode 100644
index 0000000..c8a7b7f
--- /dev/null
+++ b/drivers/dma/fsl-dpaa2-qdma/dpdmai.h
@@ -0,0 +1,524 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright 2014-2018 NXP */
+
+/*
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of the above-listed copyright holders nor the
+ * names of any contributors may be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ *
+ * ALTERNATIVELY, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") as published by the Free Software
+ * Foundation, either version 2 of that License or (at your option) any
+ * later version.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+#ifndef __FSL_DPDMAI_H
+#define __FSL_DPDMAI_H
+
+struct fsl_mc_io;
+
+/* Data Path DMA Interface API
+ * Contains initialization APIs and runtime control APIs for DPDMAI
+ */
+
+/* General DPDMAI macros */
+
+/**
+ * Maximum number of Tx/Rx priorities per DPDMAI object
+ */
+#define DPDMAI_PRIO_NUM 2
+
+/**
+ * All queues considered; see dpdmai_set_rx_queue()
+ */
+#define DPDMAI_ALL_QUEUES (u8)(-1)
+
+/**
+ * dpdmai_open() - Open a control session for the specified object
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @dpdmai_id: DPDMAI unique ID
+ * @token: Returned token; use in subsequent API calls
+ *
+ * This function can be used to open a control session for an
+ * already created object; an object may have been declared in
+ * the DPL or by calling the dpdmai_create() function.
+ * This function returns a unique authentication token,
+ * associated with the specific object ID and the specific MC
+ * portal; this token must be used in all subsequent commands for
+ * this specific object.
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_open(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ int dpdmai_id,
+ u16 *token);
+
+/**
+ * dpdmai_close() - Close the control session of the object
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ *
+ * After this function is called, no further operations are
+ * allowed on the object without opening a new control session.
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_close(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token);
+
+/**
+ * struct dpdmai_cfg - Structure representing DPDMAI configuration
+ * @priorities: Priorities for the DMA hardware processing; valid priorities are
+ * configured with values 1-8; the entry following last valid entry
+ * should be configured with 0
+ */
+struct dpdmai_cfg {
+ u8 priorities[DPDMAI_PRIO_NUM];
+};
+
+/**
+ * dpdmai_create() - Create the DPDMAI object
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @cfg: Configuration structure
+ * @token: Returned token; use in subsequent API calls
+ *
+ * Create the DPDMAI object, allocate required resources and
+ * perform required initialization.
+ *
+ * The object can be created either by declaring it in the
+ * DPL file, or by calling this function.
+ *
+ * This function returns a unique authentication token,
+ * associated with the specific object ID and the specific MC
+ * portal; this token must be used in all subsequent calls to
+ * this specific object. For objects that are created using the
+ * DPL file, call dpdmai_open() function to get an authentication
+ * token first.
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_create(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ const struct dpdmai_cfg *cfg,
+ u16 *token);
+
+/**
+ * dpdmai_destroy() - Destroy the DPDMAI object and release all its resources.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ *
+ * Return: '0' on Success; error code otherwise.
+ */
+int dpdmai_destroy(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token);
+
+/**
+ * dpdmai_enable() - Enable the DPDMAI, allow sending and receiving frames.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_enable(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token);
+
+/**
+ * dpdmai_disable() - Disable the DPDMAI, stop sending and receiving frames.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_disable(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token);
+
+/**
+ * dpdmai_is_enabled() - Check if the DPDMAI is enabled.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @en: Returns '1' if object is enabled; '0' otherwise
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_is_enabled(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ int *en);
+
+/**
+ * dpdmai_reset() - Reset the DPDMAI, returns the object to initial state.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_reset(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token);
+
+/**
+ * struct dpdmai_irq_cfg - IRQ configuration
+ * @addr: Address that must be written to signal a message-based interrupt
+ * @val: Value to write into irq_addr address
+ * @irq_num: A user defined number associated with this IRQ
+ */
+struct dpdmai_irq_cfg {
+ u64 addr;
+ u32 val;
+ int irq_num;
+};
+
+/**
+ * dpdmai_set_irq() - Set IRQ information for the DPDMAI to trigger
+ * an interrupt.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @irq_index: Identifies the interrupt index to configure
+ * @irq_cfg: IRQ configuration
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_set_irq(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ struct dpdmai_irq_cfg *irq_cfg);
+
+/**
+ * dpdmai_get_irq() - Get IRQ information from the DPDMAI
+ *
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @irq_index: The interrupt index to configure
+ * @type: Interrupt type: 0 represents message interrupt
+ * type (both irq_addr and irq_val are valid)
+ * @irq_cfg: IRQ attributes
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_get_irq(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ int *type,
+ struct dpdmai_irq_cfg *irq_cfg);
+
+/**
+ * dpdmai_set_irq_enable() - Set overall interrupt state.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @irq_index: The interrupt index to configure
+ * @en: Interrupt state - enable = 1, disable = 0
+ *
+ * Allows GPP software to control when interrupts are generated.
+ * Each interrupt can have up to 32 causes. The enable/disable control's the
+ * overall interrupt state. if the interrupt is disabled no causes will cause
+ * an interrupt
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_set_irq_enable(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u8 en);
+
+/**
+ * dpdmai_get_irq_enable() - Get overall interrupt state
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @irq_index: The interrupt index to configure
+ * @en: Returned Interrupt state - enable = 1, disable = 0
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_get_irq_enable(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u8 *en);
+
+/**
+ * dpdmai_set_irq_mask() - Set interrupt mask.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @irq_index: The interrupt index to configure
+ * @mask: event mask to trigger interrupt;
+ * each bit:
+ * 0 = ignore event
+ * 1 = consider event for asserting IRQ
+ *
+ * Every interrupt can have up to 32 causes and the interrupt model supports
+ * masking/unmasking each cause independently
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_set_irq_mask(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u32 mask);
+
+/**
+ * dpdmai_get_irq_mask() - Get interrupt mask.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @irq_index: The interrupt index to configure
+ * @mask: Returned event mask to trigger interrupt
+ *
+ * Every interrupt can have up to 32 causes and the interrupt model supports
+ * masking/unmasking each cause independently
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_get_irq_mask(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u32 *mask);
+
+/**
+ * dpdmai_get_irq_status() - Get the current status of any pending interrupts
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @irq_index: The interrupt index to configure
+ * @status: Returned interrupts status - one bit per cause:
+ * 0 = no interrupt pending
+ * 1 = interrupt pending
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_get_irq_status(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u32 *status);
+
+/**
+ * dpdmai_clear_irq_status() - Clear a pending interrupt's status
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @irq_index: The interrupt index to configure
+ * @status: bits to clear (W1C) - one bit per cause:
+ * 0 = don't change
+ * 1 = clear status bit
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_clear_irq_status(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 irq_index,
+ u32 status);
+
+/**
+ * struct dpdmai_attr - Structure representing DPDMAI attributes
+ * @id: DPDMAI object ID
+ * @version: DPDMAI version
+ * @num_of_priorities: number of priorities
+ */
+struct dpdmai_attr {
+ int id;
+ /**
+ * struct version - DPDMAI version
+ * @major: DPDMAI major version
+ * @minor: DPDMAI minor version
+ */
+ struct {
+ u16 major;
+ u16 minor;
+ } version;
+ u8 num_of_priorities;
+};
+
+/**
+ * dpdmai_get_attributes() - Retrieve DPDMAI attributes.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @attr: Returned object's attributes
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_get_attributes(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ struct dpdmai_attr *attr);
+
+/**
+ * enum dpdmai_dest - DPDMAI destination types
+ * @DPDMAI_DEST_NONE: Unassigned destination; The queue is set in parked mode
+ * and does not generate FQDAN notifications; user is expected to dequeue
+ * from the queue based on polling or other user-defined method
+ * @DPDMAI_DEST_DPIO: The queue is set in schedule mode and generates FQDAN
+ * notifications to the specified DPIO; user is expected to dequeue
+ * from the queue only after notification is received
+ * @DPDMAI_DEST_DPCON: The queue is set in schedule mode and does not generate
+ * FQDAN notifications, but is connected to the specified DPCON object;
+ * user is expected to dequeue from the DPCON channel
+ */
+enum dpdmai_dest {
+ DPDMAI_DEST_NONE = 0,
+ DPDMAI_DEST_DPIO = 1,
+ DPDMAI_DEST_DPCON = 2
+};
+
+/**
+ * struct dpdmai_dest_cfg - Structure representing DPDMAI destination parameters
+ * @dest_type: Destination type
+ * @dest_id: Either DPIO ID or DPCON ID, depending on the destination type
+ * @priority: Priority selection within the DPIO or DPCON channel; valid values
+ * are 0-1 or 0-7, depending on the number of priorities in that
+ * channel; not relevant for 'DPDMAI_DEST_NONE' option
+ */
+struct dpdmai_dest_cfg {
+ enum dpdmai_dest dest_type;
+ int dest_id;
+ u8 priority;
+};
+
+/* DPDMAI queue modification options */
+
+/**
+ * Select to modify the user's context associated with the queue
+ */
+#define DPDMAI_QUEUE_OPT_USER_CTX 0x00000001
+
+/**
+ * Select to modify the queue's destination
+ */
+#define DPDMAI_QUEUE_OPT_DEST 0x00000002
+
+/**
+ * struct dpdmai_rx_queue_cfg - DPDMAI RX queue configuration
+ * @options: Flags representing the suggested modifications to the queue;
+ * Use any combination of 'DPDMAI_QUEUE_OPT_<X>' flags
+ * @user_ctx: User context value provided in the frame descriptor of each
+ * dequeued frame;
+ * valid only if 'DPDMAI_QUEUE_OPT_USER_CTX' is contained in 'options'
+ * @dest_cfg: Queue destination parameters;
+ * valid only if 'DPDMAI_QUEUE_OPT_DEST' is contained in 'options'
+ */
+struct dpdmai_rx_queue_cfg {
+ u32 options;
+ u64 user_ctx;
+ struct dpdmai_dest_cfg dest_cfg;
+
+};
+
+/**
+ * dpdmai_set_rx_queue() - Set Rx queue configuration
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @priority: Select the queue relative to number of
+ * priorities configured at DPDMAI creation; use
+ * DPDMAI_ALL_QUEUES to configure all Rx queues
+ * identically.
+ * @cfg: Rx queue configuration
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_set_rx_queue(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 priority,
+ const struct dpdmai_rx_queue_cfg *cfg);
+
+/**
+ * struct dpdmai_rx_queue_attr - Structure representing attributes of Rx queues
+ * @user_ctx: User context value provided in the frame descriptor of each
+ * dequeued frame
+ * @dest_cfg: Queue destination configuration
+ * @fqid: Virtual FQID value to be used for dequeue operations
+ */
+struct dpdmai_rx_queue_attr {
+ u64 user_ctx;
+ struct dpdmai_dest_cfg dest_cfg;
+ u32 fqid;
+};
+
+/**
+ * dpdmai_get_rx_queue() - Retrieve Rx queue attributes.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @priority: Select the queue relative to number of
+ * priorities configured at DPDMAI creation
+ * @attr: Returned Rx queue attributes
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_get_rx_queue(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 priority,
+ struct dpdmai_rx_queue_attr *attr);
+
+/**
+ * struct dpdmai_tx_queue_attr - Structure representing attributes of Tx queues
+ * @fqid: Virtual FQID to be used for sending frames to DMA hardware
+ */
+
+struct dpdmai_tx_queue_attr {
+ u32 fqid;
+};
+
+/**
+ * dpdmai_get_tx_queue() - Retrieve Tx queue attributes.
+ * @mc_io: Pointer to MC portal's I/O object
+ * @cmd_flags: Command flags; one or more of 'MC_CMD_FLAG_'
+ * @token: Token of DPDMAI object
+ * @priority: Select the queue relative to number of
+ * priorities configured at DPDMAI creation
+ * @attr: Returned Tx queue attributes
+ *
+ * Return: '0' on Success; Error code otherwise.
+ */
+int dpdmai_get_tx_queue(struct fsl_mc_io *mc_io,
+ u32 cmd_flags,
+ u16 token,
+ u8 priority,
+ struct dpdmai_tx_queue_attr *attr);
+
+#endif /* __FSL_DPDMAI_H */
diff --git a/drivers/dma/fsl-dpaa2-qdma/dpdmai_cmd.h b/drivers/dma/fsl-dpaa2-qdma/dpdmai_cmd.h
new file mode 100644
index 0000000..071a1a7
--- /dev/null
+++ b/drivers/dma/fsl-dpaa2-qdma/dpdmai_cmd.h
@@ -0,0 +1,197 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright 2014-2018 NXP */
+
+#ifndef _FSL_DPDMAI_CMD_H
+#define _FSL_DPDMAI_CMD_H
+
+/* DPDMAI Version */
+#define DPDMAI_VER_MAJOR 2
+#define DPDMAI_VER_MINOR 2
+
+#define DPDMAI_CMD_BASE_VERSION 0
+#define DPDMAI_CMD_ID_OFFSET 4
+
+#define DPDMAI_CMDID_FORMAT(x) (((x) << DPDMAI_CMD_ID_OFFSET) | \
+ DPDMAI_CMD_BASE_VERSION)
+
+/* Command IDs */
+#define DPDMAI_CMDID_CLOSE DPDMAI_CMDID_FORMAT(0x800)
+#define DPDMAI_CMDID_OPEN DPDMAI_CMDID_FORMAT(0x80E)
+#define DPDMAI_CMDID_CREATE DPDMAI_CMDID_FORMAT(0x90E)
+#define DPDMAI_CMDID_DESTROY DPDMAI_CMDID_FORMAT(0x900)
+
+#define DPDMAI_CMDID_ENABLE DPDMAI_CMDID_FORMAT(0x002)
+#define DPDMAI_CMDID_DISABLE DPDMAI_CMDID_FORMAT(0x003)
+#define DPDMAI_CMDID_GET_ATTR DPDMAI_CMDID_FORMAT(0x004)
+#define DPDMAI_CMDID_RESET DPDMAI_CMDID_FORMAT(0x005)
+#define DPDMAI_CMDID_IS_ENABLED DPDMAI_CMDID_FORMAT(0x006)
+
+#define DPDMAI_CMDID_SET_IRQ DPDMAI_CMDID_FORMAT(0x010)
+#define DPDMAI_CMDID_GET_IRQ DPDMAI_CMDID_FORMAT(0x011)
+#define DPDMAI_CMDID_SET_IRQ_ENABLE DPDMAI_CMDID_FORMAT(0x012)
+#define DPDMAI_CMDID_GET_IRQ_ENABLE DPDMAI_CMDID_FORMAT(0x013)
+#define DPDMAI_CMDID_SET_IRQ_MASK DPDMAI_CMDID_FORMAT(0x014)
+#define DPDMAI_CMDID_GET_IRQ_MASK DPDMAI_CMDID_FORMAT(0x015)
+#define DPDMAI_CMDID_GET_IRQ_STATUS DPDMAI_CMDID_FORMAT(0x016)
+#define DPDMAI_CMDID_CLEAR_IRQ_STATUS DPDMAI_CMDID_FORMAT(0x017)
+
+#define DPDMAI_CMDID_SET_RX_QUEUE DPDMAI_CMDID_FORMAT(0x1A0)
+#define DPDMAI_CMDID_GET_RX_QUEUE DPDMAI_CMDID_FORMAT(0x1A1)
+#define DPDMAI_CMDID_GET_TX_QUEUE DPDMAI_CMDID_FORMAT(0x1A2)
+
+#define MC_CMD_HDR_TOKEN_O 32 /* Token field offset */
+#define MC_CMD_HDR_TOKEN_S 16 /* Token field size */
+
+#define MAKE_UMASK64(_width) \
+ ((u64)((_width) < 64 ? ((u64)1 << (_width)) - 1 : \
+ (u64)-1)) \
+
+static inline u64 mc_enc(int lsoffset, int width, u64 val)
+{
+ return (u64)(((u64)val & MAKE_UMASK64(width)) << lsoffset);
+}
+
+static inline u64 mc_dec(u64 val, int lsoffset, int width)
+{
+ return (u64)((val >> lsoffset) & MAKE_UMASK64(width));
+}
+
+#define MC_CMD_OP(_cmd, _param, _offset, _width, _type, _arg) \
+ ((_cmd).params[_param] |= mc_enc((_offset), (_width), _arg))
+
+#define MC_RSP_OP(_cmd, _param, _offset, _width, _type, _arg) \
+ (_arg = (_type)mc_dec(_cmd.params[_param], (_offset), (_width)))
+
+#define MC_CMD_HDR_READ_TOKEN(_hdr) \
+ ((u16)mc_dec((_hdr), MC_CMD_HDR_TOKEN_O, MC_CMD_HDR_TOKEN_S))
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_OPEN(cmd, dpdmai_id) \
+ MC_CMD_OP(cmd, 0, 0, 32, int, dpdmai_id)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_CREATE(cmd, cfg) \
+do { \
+ MC_CMD_OP(cmd, 0, 8, 8, u8, (cfg)->priorities[0]);\
+ MC_CMD_OP(cmd, 0, 16, 8, u8, (cfg)->priorities[1]);\
+} while (0)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_RSP_IS_ENABLED(cmd, en) \
+ MC_RSP_OP(cmd, 0, 0, 1, int, en)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_SET_IRQ(cmd, irq_index, irq_cfg) \
+do { \
+ MC_CMD_OP(cmd, 0, 0, 8, u8, irq_index);\
+ MC_CMD_OP(cmd, 0, 32, 32, u32, irq_cfg->val);\
+ MC_CMD_OP(cmd, 1, 0, 64, u64, irq_cfg->addr);\
+ MC_CMD_OP(cmd, 2, 0, 32, int, irq_cfg->irq_num); \
+} while (0)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_GET_IRQ(cmd, irq_index) \
+ MC_CMD_OP(cmd, 0, 32, 8, u8, irq_index)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_RSP_GET_IRQ(cmd, type, irq_cfg) \
+do { \
+ MC_RSP_OP(cmd, 0, 0, 32, u32, irq_cfg->val); \
+ MC_RSP_OP(cmd, 1, 0, 64, u64, irq_cfg->addr);\
+ MC_RSP_OP(cmd, 2, 0, 32, int, irq_cfg->irq_num); \
+ MC_RSP_OP(cmd, 2, 32, 32, int, type); \
+} while (0)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_SET_IRQ_ENABLE(cmd, irq_index, enable_state) \
+do { \
+ MC_CMD_OP(cmd, 0, 0, 8, u8, enable_state); \
+ MC_CMD_OP(cmd, 0, 32, 8, u8, irq_index); \
+} while (0)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_GET_IRQ_ENABLE(cmd, irq_index) \
+ MC_CMD_OP(cmd, 0, 32, 8, u8, irq_index)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_RSP_GET_IRQ_ENABLE(cmd, enable_state) \
+ MC_RSP_OP(cmd, 0, 0, 8, u8, enable_state)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_SET_IRQ_MASK(cmd, irq_index, mask) \
+do { \
+ MC_CMD_OP(cmd, 0, 0, 32, u32, mask); \
+ MC_CMD_OP(cmd, 0, 32, 8, u8, irq_index); \
+} while (0)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_GET_IRQ_MASK(cmd, irq_index) \
+ MC_CMD_OP(cmd, 0, 32, 8, u8, irq_index)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_RSP_GET_IRQ_MASK(cmd, mask) \
+ MC_RSP_OP(cmd, 0, 0, 32, u32, mask)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_GET_IRQ_STATUS(cmd, irq_index, status) \
+do { \
+ MC_CMD_OP(cmd, 0, 0, 32, u32, status);\
+ MC_CMD_OP(cmd, 0, 32, 8, u8, irq_index);\
+} while (0)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_RSP_GET_IRQ_STATUS(cmd, status) \
+ MC_RSP_OP(cmd, 0, 0, 32, u32, status)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_CLEAR_IRQ_STATUS(cmd, irq_index, status) \
+do { \
+ MC_CMD_OP(cmd, 0, 0, 32, u32, status); \
+ MC_CMD_OP(cmd, 0, 32, 8, u8, irq_index); \
+} while (0)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_RSP_GET_ATTR(cmd, attr) \
+do { \
+ MC_RSP_OP(cmd, 0, 0, 32, int, (attr)->id); \
+ MC_RSP_OP(cmd, 0, 32, 8, u8, (attr)->num_of_priorities); \
+ MC_RSP_OP(cmd, 1, 0, 16, u16, (attr)->version.major);\
+ MC_RSP_OP(cmd, 1, 16, 16, u16, (attr)->version.minor);\
+} while (0)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_SET_RX_QUEUE(cmd, priority, cfg) \
+do { \
+ MC_CMD_OP(cmd, 0, 0, 32, int, (cfg)->dest_cfg.dest_id); \
+ MC_CMD_OP(cmd, 0, 32, 8, u8, (cfg)->dest_cfg.priority); \
+ MC_CMD_OP(cmd, 0, 40, 8, u8, priority); \
+ MC_CMD_OP(cmd, 0, 48, 4, enum dpdmai_dest, \
+ (cfg)->dest_cfg.dest_type); \
+ MC_CMD_OP(cmd, 1, 0, 64, u64, (cfg)->user_ctx); \
+ MC_CMD_OP(cmd, 2, 0, 32, u32, (cfg)->options);\
+} while (0)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_GET_RX_QUEUE(cmd, priority) \
+ MC_CMD_OP(cmd, 0, 40, 8, u8, priority)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_RSP_GET_RX_QUEUE(cmd, attr) \
+do { \
+ MC_RSP_OP(cmd, 0, 0, 32, int, (attr)->dest_cfg.dest_id);\
+ MC_RSP_OP(cmd, 0, 32, 8, u8, (attr)->dest_cfg.priority);\
+ MC_RSP_OP(cmd, 0, 48, 4, enum dpdmai_dest, \
+ (attr)->dest_cfg.dest_type);\
+ MC_RSP_OP(cmd, 1, 0, 64, u64, (attr)->user_ctx);\
+ MC_RSP_OP(cmd, 2, 0, 32, u32, (attr)->fqid);\
+} while (0)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_CMD_GET_TX_QUEUE(cmd, priority) \
+ MC_CMD_OP(cmd, 0, 40, 8, u8, priority)
+
+/* cmd, param, offset, width, type, arg_name */
+#define DPDMAI_RSP_GET_TX_QUEUE(cmd, attr) \
+ MC_RSP_OP(cmd, 1, 0, 32, u32, (attr)->fqid)
+
+#endif /* _FSL_DPDMAI_CMD_H */
^ permalink raw reply related
* RE: Issues with i.MX SPI DMA transfers
From: Robin Gong @ 2019-04-09 3:26 UTC (permalink / raw)
To: Igor Plyatov, Uwe Kleine-König
Cc: linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org, linux-spi@vger.kernel.org,
dl-linux-imx, Fabio Estevam, Pengutronix Kernel Team,
Sascha Hauer, Shawn Guo, Mark Brown, dmaengine@vger.kernel.org,
Vinod Koul, Dan Williams, Andy Duan, Han Xu, Clark Wang
In-Reply-To: <VI1PR04MB454305A56372466A38E9F81989560@VI1PR04MB4543.eurprd04.prod.outlook.com>
[-- Attachment #1: Type: text/plain, Size: 3911 bytes --]
Hi Igor,
Please have a try with the attached patches, and revert 25aaa75df1e6, ad0d92d7ba6a
, dd4b487b32a3, df07101e1c4a before apply. Besides XCH, tx thresh should be set to 0 ,
now no failure caught on ecspi5.
> -----Original Message-----
> From: Robin Gong
> Sent: 2019年4月2日 16:33
> To: 'Igor Plyatov' <plyatov@gmail.com>; Uwe Kleine-König
> <u.kleine-koenig@pengutronix.de>
> Cc: linux-kernel@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> linux-spi@vger.kernel.org; dl-linux-imx <linux-imx@nxp.com>; Fabio Estevam
> <festevam@gmail.com>; Pengutronix Kernel Team <kernel@pengutronix.de>;
> Sascha Hauer <s.hauer@pengutronix.de>; Shawn Guo
> <shawnguo@kernel.org>; Mark Brown <broonie@kernel.org>;
> dmaengine@vger.kernel.org; Vinod Koul <vkoul@kernel.org>; Dan Williams
> <dan.j.williams@intel.com>; Andy Duan <fugang.duan@nxp.com>; Han Xu
> <han.xu@nxp.com>; Clark Wang <xiaoning.wang@nxp.com>
> Subject: RE: Issues with i.MX SPI DMA transfers
>
> > -----Original Message-----
> > From: Igor Plyatov <plyatov@gmail.com>
> > Sent: 2019年4月2日 15:20
> > To: Robin Gong <yibin.gong@nxp.com>; Uwe Kleine-König
> > <u.kleine-koenig@pengutronix.de>
> > Cc: linux-kernel@vger.kernel.org;
> > linux-arm-kernel@lists.infradead.org;
> > linux-spi@vger.kernel.org; dl-linux-imx <linux-imx@nxp.com>; Fabio
> > Estevam <festevam@gmail.com>; Pengutronix Kernel Team
> > <kernel@pengutronix.de>; Sascha Hauer <s.hauer@pengutronix.de>; Shawn
> > Guo <shawnguo@kernel.org>; Mark Brown <broonie@kernel.org>;
> > dmaengine@vger.kernel.org; Vinod Koul <vkoul@kernel.org>; Dan Williams
> > <dan.j.williams@intel.com>; Andy Duan <fugang.duan@nxp.com>; Han Xu
> > <han.xu@nxp.com>; Clark Wang <xiaoning.wang@nxp.com>
> > Subject: Re: Issues with i.MX SPI DMA transfers
> >
> > Dear Robin Gong,
> >
> >
> > >> Sorry...below another sdma patch(ad0d92d7ba6a) need to be reverted,
> > >> because spi driver may dynamically change burst length.
> >
> >
> > now I have reverted patch ad0d92d7ba6a.
> >
> > Patches 0001-dma-engine-imx-sdma-add-mcu_2_ecspi-script.patch and
> > 0002-spi-spi-imx-fix-ERR009165.patch are applied.
> >
> >
> > Kernel log show messages
> >
> > [ 29.202639] imx-sdma 20ec000.sdma: loaded firmware 3.3 [
> > 29.238595] spi_imx 2008000.spi: probed [ 29.242802] spi_imx
> > 200c000.spi: probed [ 29.245217] spi_imx 2018000.spi: probed
> >
> > SPI DMA transfers still not work.
> >
> > If I test 32 byte transfers, then they work fine. But 64 byte
> > transfers fails always and I see error messages
> >
> > root@cr7:~# spidev_test -D /dev/spidev4.1 -s 1200000 -b 8 -S 64 -I 1
> > -l spi mode: 0x20 bits per word: 8 max speed: 1200000 Hz (1200 KHz) [
> > 423.686736] spi_master spi4: I/O Error in DMA RX [ 423.691392] spidev
> > spi4.1: SPI transfer failed: -110 [ 423.696382] spi_master spi4:
> > failed to transfer one message from queue can't send spi message:
> > Connection timed out Aborted (core dumped)
> >
> > I suppose, transfers shorter then 64 bytes made with help of PIO.
> >
> > Robin, is there any chance for you to find some time and look at this
> > issue again?
> I have quick test with spidev_test loopback, but didn't meet your error, Is your
> code the almost latest code in linux-next as mine?
>
> root@imx6qpdlsolox:~# cat /proc/interrupts | grep sdma
> 48: 37 GPC 2 Level sdma
> -lt@imx6qpdlsolox:~# ./spidev_test -D /dev/spidev0.0 -s 1200000 -b 8 -S 64 -I
> 1 -l spi mode: 0x20 bits per word: 8 max speed: 1200000 Hz (1200 KHz)
> root@imx6qpdlsolox:~# cat /proc/interrupts | grep sdma
> 48: 43 GPC 2 Level sdma
> ./spidev_test -D /dev/spidev0.0 -s 1200000 -b 8 -S 512 -I 1 -l spi mode: 0x20 bits
> per word: 8 max speed: 1200000 Hz (1200 KHz)
> total: tx 0.5KB, rx 0.5KB
> >
> > Best wishes.
> > --
> > Igor Plyatov
[-- Attachment #2: 0005-dma-engine-imx-sdma-add-mcu_2_ecspi-script.patch --]
[-- Type: application/octet-stream, Size: 1369 bytes --]
From fbdf4153b4790a43a7396b5c78a848b4d0735f80 Mon Sep 17 00:00:00 2001
From: Robin Gong <yibin.gong@nxp.com>
Date: Thu, 28 Mar 2019 16:36:12 +0800
Subject: [PATCH 5/6] dma: engine: imx-sdma: add mcu_2_ecspi script
Add mcu_2_ecspi script to fix ecspi errata ERR009165.
Signed-off-by: Robin Gong <yibin.gong@nxp.com>
---
drivers/dma/imx-sdma.c | 3 +++
include/linux/platform_data/dma-imx-sdma.h | 1 +
2 files changed, 4 insertions(+)
diff --git a/drivers/dma/imx-sdma.c b/drivers/dma/imx-sdma.c
index adc82d9..0cc52db 100644
--- a/drivers/dma/imx-sdma.c
+++ b/drivers/dma/imx-sdma.c
@@ -907,6 +907,9 @@ static void sdma_get_pc(struct sdma_channel *sdmac,
emi_2_per = sdma->script_addrs->mcu_2_ata_addr;
break;
case IMX_DMATYPE_CSPI:
+ per_2_emi = sdma->script_addrs->app_2_mcu_addr;
+ emi_2_per = sdma->script_addrs->mcu_2_ecspi_addr;
+ break;
case IMX_DMATYPE_EXT:
case IMX_DMATYPE_SSI:
case IMX_DMATYPE_SAI:
diff --git a/include/linux/platform_data/dma-imx-sdma.h b/include/linux/platform_data/dma-imx-sdma.h
index 6eaa53c..f794fee 100644
--- a/include/linux/platform_data/dma-imx-sdma.h
+++ b/include/linux/platform_data/dma-imx-sdma.h
@@ -51,6 +51,7 @@ struct sdma_script_start_addrs {
/* End of v2 array */
s32 zcanfd_2_mcu_addr;
s32 zqspi_2_mcu_addr;
+ s32 mcu_2_ecspi_addr;
/* End of v3 array */
};
--
2.7.4
[-- Attachment #3: 0006-spi-spi-imx-fix-ERR009165.patch --]
[-- Type: application/octet-stream, Size: 1635 bytes --]
From aa198c3265fb54d42715c5fa4c25d15a73ad934c Mon Sep 17 00:00:00 2001
From: Robin Gong <yibin.gong@nxp.com>
Date: Thu, 28 Mar 2019 16:38:13 +0800
Subject: [PATCH 6/6] spi: spi-imx: fix ERR009165
Change to XCH mode even in dma mode, please refer to below
errata:
https://www.nxp.com/docs/en/errata/IMX6DQCE.pdf
Signed-off-by: Robin Gong <yibin.gong@nxp.com>
---
drivers/spi/spi-imx.c | 8 ++------
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/drivers/spi/spi-imx.c b/drivers/spi/spi-imx.c
index 09c9a1e..04af84d 100644
--- a/drivers/spi/spi-imx.c
+++ b/drivers/spi/spi-imx.c
@@ -585,8 +585,9 @@ static int mx51_ecspi_prepare_transfer(struct spi_imx_data *spi_imx,
ctrl |= mx51_ecspi_clkdiv(spi_imx, t->speed_hz, &clk);
spi_imx->spi_bus_clk = clk;
+ /* ERR009165: work in XHC mode as PIO */
if (spi_imx->usedma)
- ctrl |= MX51_ECSPI_CTRL_SMC;
+ ctrl &= ~MX51_ECSPI_CTRL_SMC;
writel(ctrl, spi_imx->base + MX51_ECSPI_CTRL);
@@ -617,7 +618,6 @@ static void mx51_setup_wml(struct spi_imx_data *spi_imx)
* and enable DMA request.
*/
writel(MX51_ECSPI_DMA_RX_WML(spi_imx->wml - 1) |
- MX51_ECSPI_DMA_TX_WML(spi_imx->wml) |
MX51_ECSPI_DMA_RXT_WML(spi_imx->wml) |
MX51_ECSPI_DMA_TEDEN | MX51_ECSPI_DMA_RXDEN |
MX51_ECSPI_DMA_RXTDEN, spi_imx->base + MX51_ECSPI_DMA);
@@ -1265,10 +1265,6 @@ static int spi_imx_sdma_init(struct device *dev, struct spi_imx_data *spi_imx,
{
int ret;
- /* use pio mode for i.mx6dl chip TKT238285 */
- if (of_machine_is_compatible("fsl,imx6dl"))
- return 0;
-
spi_imx->wml = spi_imx->devtype_data->fifo_size / 2;
/* Prepare for TX DMA: */
--
2.7.4
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox