* [5/6] dmaengine: xilinx_dma: autodetect whether the HW supports scatter-gather
From: Andrea Merello @ 2018-06-20 14:57 UTC (permalink / raw)
To: Radhey Shyam Pandey
Cc: vkoul@kernel.org, dan.j.williams@intel.com, Michal Simek,
Appana Durga Kedareswara Rao, dmaengine@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org
On Wed, Jun 20, 2018 at 4:43 PM, Radhey Shyam Pandey <radheys@xilinx.com> wrote:
>> -----Original Message-----
>> From: dmaengine-owner@vger.kernel.org [mailto:dmaengine-
>> owner@vger.kernel.org] On Behalf Of Andrea Merello
>> Sent: Wednesday, June 20, 2018 2:07 PM
>> To: vkoul@kernel.org; dan.j.williams@intel.com; Michal Simek
>> <michals@xilinx.com>; Appana Durga Kedareswara Rao
>> <appanad@xilinx.com>; dmaengine@vger.kernel.org
>> Cc: linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org;
>> Andrea Merello <andrea.merello@gmail.com>
>> Subject: [PATCH 5/6] dmaengine: xilinx_dma: autodetect whether the HW
>> supports scatter-gather
>>
>> The HW can be either direct-access or scatter-gather version. These are
>> SW incompatible.
>>
>> The driver can handle both version: a DT property was used to
>> tell the driver whether to assume the HW is is scatter-gather mode.
>>
>> This patch makes the driver to autodetect this information. The DT
>> property is not required anymore.
>>
>> Signed-off-by: Andrea Merello <andrea.merello@gmail.com>
>> ---
>> drivers/dma/xilinx/xilinx_dma.c | 12 ++++++++----
>> 1 file changed, 8 insertions(+), 4 deletions(-)
>>
>> diff --git a/drivers/dma/xilinx/xilinx_dma.c b/drivers/dma/xilinx/xilinx_dma.c
>> index bdbc8ba9092a..8c6e818e596f 100644
>> --- a/drivers/dma/xilinx/xilinx_dma.c
>> +++ b/drivers/dma/xilinx/xilinx_dma.c
>> @@ -86,6 +86,7 @@
>> #define XILINX_DMA_DMASR_DMA_DEC_ERR BIT(6)
>> #define XILINX_DMA_DMASR_DMA_SLAVE_ERR BIT(5)
>> #define XILINX_DMA_DMASR_DMA_INT_ERR BIT(4)
>> +#define XILINX_DMA_DMASR_SG_MASK BIT(3)
>> #define XILINX_DMA_DMASR_IDLE BIT(1)
>> #define XILINX_DMA_DMASR_HALTED BIT(0)
>> #define XILINX_DMA_DMASR_DELAY_MASK GENMASK(31, 24)
>> @@ -407,7 +408,6 @@ struct xilinx_dma_config {
>> * @dev: Device Structure
>> * @common: DMA device structure
>> * @chan: Driver specific DMA channel
>> - * @has_sg: Specifies whether Scatter-Gather is present or not
>> * @mcdma: Specifies whether Multi-Channel is present or not
>> * @flush_on_fsync: Flush on frame sync
>> * @ext_addr: Indicates 64 bit addressing is supported by dma device
>> @@ -426,7 +426,6 @@ struct xilinx_dma_device {
>> struct device *dev;
>> struct dma_device common;
>> struct xilinx_dma_chan
>> *chan[XILINX_DMA_MAX_CHANS_PER_DEVICE];
>> - bool has_sg;
>> bool mcdma;
>> u32 flush_on_fsync;
>> bool ext_addr;
>> @@ -2391,7 +2390,6 @@ static int xilinx_dma_chan_probe(struct
>> xilinx_dma_device *xdev,
>>
>> chan->dev = xdev->dev;
>> chan->xdev = xdev;
>> - chan->has_sg = xdev->has_sg;
>> chan->desc_pendingcount = 0x0;
>> chan->ext_addr = xdev->ext_addr;
>> /* This variable ensures that descriptors are not
>> @@ -2488,6 +2486,13 @@ static int xilinx_dma_chan_probe(struct
>> xilinx_dma_device *xdev,
>> chan->stop_transfer = xilinx_dma_stop_transfer;
>> }
>>
>> + /* check if SG is enabled */
>> + if (dma_ctrl_read(chan, XILINX_DMA_REG_DMASR) &
>> + XILINX_DMA_DMASR_SG_MASK)
> I think SGIncld mask is only applicable for AXI DMA and CDMA IP.
> For VDMA IP this bit is reserved.
OK. I can make it conditional wrt the IP type. As far as I can see
VDMA IP has not the two (SG vs no-SG) variant at all, so all should be
still OK.
> .
>> + chan->has_sg = true;
>> + dev_dbg(chan->dev, "ch %d: SG %s\n", chan->id,
>> + chan->has_sg ? "enabled" : "disabled");
>> +
>> /* Initialize the tasklet */
>> tasklet_init(&chan->tasklet, xilinx_dma_do_tasklet,
>> (unsigned long)chan);
>> @@ -2626,7 +2631,6 @@ static int xilinx_dma_probe(struct platform_device
>> *pdev)
>> return PTR_ERR(xdev->regs);
>>
>> /* Retrieve the DMA engine properties from the device tree */
>> -
> Unrelated change
Oops.. Sorry.
>
>> if (xdev->dma_config->dmatype == XDMA_TYPE_AXIDMA) {
>> xdev->mcdma = of_property_read_bool(node, "xlnx,mcdma");
>> err = of_property_read_u32(node, "xlnx,lengthregwidth",
>> --
>> 2.17.1
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe dmaengine" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
---
To unsubscribe from this list: send the line "unsubscribe dmaengine" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [5/6] dmaengine: xilinx_dma: autodetect whether the HW supports scatter-gather
From: Radhey Shyam Pandey @ 2018-06-20 14:43 UTC (permalink / raw)
To: Andrea Merello, vkoul@kernel.org, dan.j.williams@intel.com,
Michal Simek, Appana Durga Kedareswara Rao,
dmaengine@vger.kernel.org
Cc: linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org
> -----Original Message-----
> From: dmaengine-owner@vger.kernel.org [mailto:dmaengine-
> owner@vger.kernel.org] On Behalf Of Andrea Merello
> Sent: Wednesday, June 20, 2018 2:07 PM
> To: vkoul@kernel.org; dan.j.williams@intel.com; Michal Simek
> <michals@xilinx.com>; Appana Durga Kedareswara Rao
> <appanad@xilinx.com>; dmaengine@vger.kernel.org
> Cc: linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org;
> Andrea Merello <andrea.merello@gmail.com>
> Subject: [PATCH 5/6] dmaengine: xilinx_dma: autodetect whether the HW
> supports scatter-gather
>
> The HW can be either direct-access or scatter-gather version. These are
> SW incompatible.
>
> The driver can handle both version: a DT property was used to
> tell the driver whether to assume the HW is is scatter-gather mode.
>
> This patch makes the driver to autodetect this information. The DT
> property is not required anymore.
>
> Signed-off-by: Andrea Merello <andrea.merello@gmail.com>
> ---
> drivers/dma/xilinx/xilinx_dma.c | 12 ++++++++----
> 1 file changed, 8 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/dma/xilinx/xilinx_dma.c b/drivers/dma/xilinx/xilinx_dma.c
> index bdbc8ba9092a..8c6e818e596f 100644
> --- a/drivers/dma/xilinx/xilinx_dma.c
> +++ b/drivers/dma/xilinx/xilinx_dma.c
> @@ -86,6 +86,7 @@
> #define XILINX_DMA_DMASR_DMA_DEC_ERR BIT(6)
> #define XILINX_DMA_DMASR_DMA_SLAVE_ERR BIT(5)
> #define XILINX_DMA_DMASR_DMA_INT_ERR BIT(4)
> +#define XILINX_DMA_DMASR_SG_MASK BIT(3)
> #define XILINX_DMA_DMASR_IDLE BIT(1)
> #define XILINX_DMA_DMASR_HALTED BIT(0)
> #define XILINX_DMA_DMASR_DELAY_MASK GENMASK(31, 24)
> @@ -407,7 +408,6 @@ struct xilinx_dma_config {
> * @dev: Device Structure
> * @common: DMA device structure
> * @chan: Driver specific DMA channel
> - * @has_sg: Specifies whether Scatter-Gather is present or not
> * @mcdma: Specifies whether Multi-Channel is present or not
> * @flush_on_fsync: Flush on frame sync
> * @ext_addr: Indicates 64 bit addressing is supported by dma device
> @@ -426,7 +426,6 @@ struct xilinx_dma_device {
> struct device *dev;
> struct dma_device common;
> struct xilinx_dma_chan
> *chan[XILINX_DMA_MAX_CHANS_PER_DEVICE];
> - bool has_sg;
> bool mcdma;
> u32 flush_on_fsync;
> bool ext_addr;
> @@ -2391,7 +2390,6 @@ static int xilinx_dma_chan_probe(struct
> xilinx_dma_device *xdev,
>
> chan->dev = xdev->dev;
> chan->xdev = xdev;
> - chan->has_sg = xdev->has_sg;
> chan->desc_pendingcount = 0x0;
> chan->ext_addr = xdev->ext_addr;
> /* This variable ensures that descriptors are not
> @@ -2488,6 +2486,13 @@ static int xilinx_dma_chan_probe(struct
> xilinx_dma_device *xdev,
> chan->stop_transfer = xilinx_dma_stop_transfer;
> }
>
> + /* check if SG is enabled */
> + if (dma_ctrl_read(chan, XILINX_DMA_REG_DMASR) &
> + XILINX_DMA_DMASR_SG_MASK)
I think SGIncld mask is only applicable for AXI DMA and CDMA IP.
For VDMA IP this bit is reserved.
.
> + chan->has_sg = true;
> + dev_dbg(chan->dev, "ch %d: SG %s\n", chan->id,
> + chan->has_sg ? "enabled" : "disabled");
> +
> /* Initialize the tasklet */
> tasklet_init(&chan->tasklet, xilinx_dma_do_tasklet,
> (unsigned long)chan);
> @@ -2626,7 +2631,6 @@ static int xilinx_dma_probe(struct platform_device
> *pdev)
> return PTR_ERR(xdev->regs);
>
> /* Retrieve the DMA engine properties from the device tree */
> -
Unrelated change
> if (xdev->dma_config->dmatype == XDMA_TYPE_AXIDMA) {
> xdev->mcdma = of_property_read_bool(node, "xlnx,mcdma");
> err = of_property_read_u32(node, "xlnx,lengthregwidth",
> --
> 2.17.1
>
> --
> To unsubscribe from this list: send the line "unsubscribe dmaengine" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
---
To unsubscribe from this list: send the line "unsubscribe dmaengine" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [3/6] dt-bindings: xilinx_dma: add required xlnx,lengthregwidth property
From: Radhey Shyam Pandey @ 2018-06-20 14:37 UTC (permalink / raw)
To: 'Andrea Merello', 'vkoul@kernel.org',
'dan.j.williams@intel.com', Michal Simek,
Appana Durga Kedareswara Rao, 'dmaengine@vger.kernel.org'
Cc: 'linux-arm-kernel@lists.infradead.org',
'linux-kernel@vger.kernel.org'
> -----Original Message-----
> From: Radhey Shyam Pandey
> Sent: Wednesday, June 20, 2018 7:13 PM
> To: Andrea Merello <andrea.merello@gmail.com>; vkoul@kernel.org;
> dan.j.williams@intel.com; Michal Simek <michals@xilinx.com>; Appana Durga
> Kedareswara Rao <appanad@xilinx.com>; dmaengine@vger.kernel.org
> Cc: linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org
> Subject: RE: [PATCH 3/6] dt-bindings: xilinx_dma: add required
> xlnx,lengthregwidth property
>
>
> > -----Original Message-----
> > From: dmaengine-owner@vger.kernel.org [mailto:dmaengine-
> > owner@vger.kernel.org] On Behalf Of Andrea Merello
> > Sent: Wednesday, June 20, 2018 2:07 PM
> > To: vkoul@kernel.org; dan.j.williams@intel.com; Michal Simek
> > <michals@xilinx.com>; Appana Durga Kedareswara Rao
> > <appanad@xilinx.com>; dmaengine@vger.kernel.org
> > Cc: linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org;
> > Andrea Merello <andrea.merello@gmail.com>
> > Subject: [PATCH 3/6] dt-bindings: xilinx_dma: add required
> > xlnx,lengthregwidth property
>
> dt-bindings: dmaengine: xilinx_dma
>
> Please also include DT folks.
> >
> > The width of the "length register" cannot be autodetected, and it is now
> > specified with a DT property. Add DOC for it.
> >
> > Signed-off-by: Andrea Merello <andrea.merello@gmail.com>
> > ---
> > Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt | 2 ++
> > 1 file changed, 2 insertions(+)
> >
> > diff --git a/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt
> > b/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt
> > index a2b8bfaec43c..acecdc5d8d47 100644
> > --- a/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt
> > +++ b/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt
> > @@ -36,6 +36,8 @@ Required properties:
> >
> > Required properties for VDMA:
> > - xlnx,num-fstores: Should be the number of framebuffers as configured in
> > h/w.
> > +Required properties for AXI DMA:
> > +- xlnx,lengthregwidth: Should be the width of the length register as
> > configured in h/w.
>
> One suggestion to be inline with IP property naming we can rename
> this prop to "xlnx,sg-length-width"? Please take a look at Xilinx tree
> we have this feature added in the master branch. It would be good
> to consolidate both implementations and upstream. Let me know
> if there are any followup queries.
It should be ok to cherrypick 3/6 and 4/6 (xlnx,sg-length-width)
from Xilinx tree and include it in your v2 patch series.
>
> >
> > Optional properties:
> > - xlnx,include-sg: Tells configured for Scatter-mode in
> > --
> > 2.17.1
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe dmaengine" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
---
To unsubscribe from this list: send the line "unsubscribe dmaengine" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [3/6] dt-bindings: xilinx_dma: add required xlnx,lengthregwidth property
From: Radhey Shyam Pandey @ 2018-06-20 13:42 UTC (permalink / raw)
To: Andrea Merello, vkoul@kernel.org, dan.j.williams@intel.com,
Michal Simek, Appana Durga Kedareswara Rao,
dmaengine@vger.kernel.org
Cc: linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org
> -----Original Message-----
> From: dmaengine-owner@vger.kernel.org [mailto:dmaengine-
> owner@vger.kernel.org] On Behalf Of Andrea Merello
> Sent: Wednesday, June 20, 2018 2:07 PM
> To: vkoul@kernel.org; dan.j.williams@intel.com; Michal Simek
> <michals@xilinx.com>; Appana Durga Kedareswara Rao
> <appanad@xilinx.com>; dmaengine@vger.kernel.org
> Cc: linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org;
> Andrea Merello <andrea.merello@gmail.com>
> Subject: [PATCH 3/6] dt-bindings: xilinx_dma: add required
> xlnx,lengthregwidth property
dt-bindings: dmaengine: xilinx_dma
Please also include DT folks.
>
> The width of the "length register" cannot be autodetected, and it is now
> specified with a DT property. Add DOC for it.
>
> Signed-off-by: Andrea Merello <andrea.merello@gmail.com>
> ---
> Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt
> b/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt
> index a2b8bfaec43c..acecdc5d8d47 100644
> --- a/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt
> +++ b/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt
> @@ -36,6 +36,8 @@ Required properties:
>
> Required properties for VDMA:
> - xlnx,num-fstores: Should be the number of framebuffers as configured in
> h/w.
> +Required properties for AXI DMA:
> +- xlnx,lengthregwidth: Should be the width of the length register as
> configured in h/w.
One suggestion to be inline with IP property naming we can rename
this prop to "xlnx,sg-length-width"? Please take a look at Xilinx tree
we have this feature added in the master branch. It would be good
to consolidate both implementations and upstream. Let me know
if there are any followup queries.
>
> Optional properties:
> - xlnx,include-sg: Tells configured for Scatter-mode in
> --
> 2.17.1
>
> --
> To unsubscribe from this list: send the line "unsubscribe dmaengine" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
---
To unsubscribe from this list: send the line "unsubscribe dmaengine" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [2/6] dmaengine: xilinx_dma: fix completion callback is not invoked for each DMA operation
From: Andrea Merello @ 2018-06-20 13:32 UTC (permalink / raw)
To: Radhey Shyam Pandey
Cc: vkoul@kernel.org, dan.j.williams@intel.com, Michal Simek,
Appana Durga Kedareswara Rao, dmaengine@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org
On Wed, Jun 20, 2018 at 2:36 PM, Radhey Shyam Pandey <radheys@xilinx.com> wrote:
>> -----Original Message-----
>> From: dmaengine-owner@vger.kernel.org [mailto:dmaengine-
>> owner@vger.kernel.org] On Behalf Of Andrea Merello
>> Sent: Wednesday, June 20, 2018 2:07 PM
>> To: vkoul@kernel.org; dan.j.williams@intel.com; Michal Simek
>> <michals@xilinx.com>; Appana Durga Kedareswara Rao
>> <appanad@xilinx.com>; dmaengine@vger.kernel.org
>> Cc: linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org;
>> Andrea Merello <andrea.merello@gmail.com>
>> Subject: [PATCH 2/6] dmaengine: xilinx_dma: fix completion callback is not
>> invoked for each DMA operation
>>
>> API specification says: "On completion of each DMA operation, the next in
>> queue is started and a tasklet triggered. The tasklet will then call the
>> client driver completion callback routine for notification, if set."
>>
>> Currently the driver keeps a "desc_pendingcount" counter of the total
>> descriptor pending, and it uses as IRQ coalesce threshold, as result it
>> only calls the CBs after ALL pending operations are completed, which is
>> wrong.
> I think IRQ coalescing enable/disable should be configurable.
> Performance related usecases will need this support.
I didn't intend this (only) wrt performances; my concern was mostly
wrt correctness. If my point of view is wrong then I'll drop this
patch from the series.
(.. I might respin it again in future: I had a patch wrt an old driver
version that allowed submitting new descriptors to the HW while the
DMA is running, and in this case disabling coalesce is needed i.e. in
order to submit a new empty buffer whenever the DMA finishes a
transfer without waiting the DMA to stop).
BTW, is there any dmaengine API suitable for setting interrupt coalesce?
>>
>> This patch uses disable IRQ coalesce and checks for the completion flag
>> for the descriptors (which is further divided in segments).
>>
>> Possibly a better optimization could be using proper IRQ coalesce
>> threshold to get an IRQ after all segments of the descriptors are done.
>> But we don't do that yet..
>>
>> NOTE: for now we do this only for AXI DMA, other DMA flavors are
>> untested/untouched.
>> This is loosely based on
>> commit 65df81a6dc74 ("xilinx_dma: IrqThreshold set incorrectly, unreliable.")
>> in my linux-4.6-zynq tree
> NOTE description doesn't help much.
>
>>
>> From: Jeremy Trimble [original patch]
>> Signed-off-by: Andrea Merello <andrea.merello@gmail.com>
>> ---
>> drivers/dma/xilinx/xilinx_dma.c | 39 +++++++++++++++++++++------------
>> 1 file changed, 25 insertions(+), 14 deletions(-)
>>
>> diff --git a/drivers/dma/xilinx/xilinx_dma.c b/drivers/dma/xilinx/xilinx_dma.c
>> index a516e7ffef21..cf12f7147f07 100644
>> --- a/drivers/dma/xilinx/xilinx_dma.c
>> +++ b/drivers/dma/xilinx/xilinx_dma.c
>> @@ -164,6 +164,7 @@
>> #define XILINX_DMA_CR_COALESCE_SHIFT 16
>> #define XILINX_DMA_BD_SOP BIT(27)
>> #define XILINX_DMA_BD_EOP BIT(26)
>> +#define XILINX_DMA_BD_CMPLT BIT(31)
>> #define XILINX_DMA_COALESCE_MAX 255
>> #define XILINX_DMA_NUM_DESCS 255
>> #define XILINX_DMA_NUM_APP_WORDS 5
>> @@ -1274,12 +1275,9 @@ static void xilinx_dma_start_transfer(struct
>> xilinx_dma_chan *chan)
>>
>> reg = dma_ctrl_read(chan, XILINX_DMA_REG_DMACR);
>>
>> - if (chan->desc_pendingcount <= XILINX_DMA_COALESCE_MAX) {
>> - reg &= ~XILINX_DMA_CR_COALESCE_MAX;
>> - reg |= chan->desc_pendingcount <<
>> - XILINX_DMA_CR_COALESCE_SHIFT;
>> - dma_ctrl_write(chan, XILINX_DMA_REG_DMACR, reg);
>> - }
>> + reg &= ~XILINX_DMA_CR_COALESCE_MAX;
>> + reg |= 1 << XILINX_DMA_CR_COALESCE_SHIFT;
>> + dma_ctrl_write(chan, XILINX_DMA_REG_DMACR, reg);
>>
>> if (chan->has_sg && !chan->xdev->mcdma)
>> xilinx_write(chan, XILINX_DMA_REG_CURDESC,
>> @@ -1378,6 +1376,20 @@ static void xilinx_dma_complete_descriptor(struct
>> xilinx_dma_chan *chan)
>> return;
>>
>> list_for_each_entry_safe(desc, next, &chan->active_list, node) {
>> + if (chan->xdev->dma_config->dmatype ==
>> XDMA_TYPE_AXIDMA) {
>> + /*
>> + * Check whether the last segment in this descriptor
>> + * has been completed.
>> + */
>> + const struct xilinx_axidma_tx_segment *const tail_seg
>> =
>> + list_last_entry(&desc->segments,
>> + struct
>> xilinx_axidma_tx_segment,
>> + node);
>> +
>> + /* we've processed all the completed descriptors */
>> + if (!(tail_seg->hw.status & XILINX_DMA_BD_CMPLT))
>> + break;
>> + }
>> list_del(&desc->node);
>> if (!desc->cyclic)
>> dma_cookie_complete(&desc->async_tx);
>> @@ -1826,14 +1838,13 @@ static struct dma_async_tx_descriptor
>> *xilinx_dma_prep_slave_sg(
>> struct xilinx_axidma_tx_segment, node);
>> desc->async_tx.phys = segment->phys;
>>
>> - /* For the last DMA_MEM_TO_DEV transfer, set EOP */
>> - if (chan->direction == DMA_MEM_TO_DEV) {
>> - segment->hw.control |= XILINX_DMA_BD_SOP;
>> - segment = list_last_entry(&desc->segments,
>> - struct xilinx_axidma_tx_segment,
>> - node);
>> - segment->hw.control |= XILINX_DMA_BD_EOP;
>> - }
>> + /* For the first transfer, set SOP */
>> + segment->hw.control |= XILINX_DMA_BD_SOP;
>> + /* For the last transfer, set EOP */
>> + segment = list_last_entry(&desc->segments,
>> + struct xilinx_axidma_tx_segment,
>> + node);
>> + segment->hw.control |= XILINX_DMA_BD_EOP;
>>
>> return &desc->async_tx;
>>
>> --
>> 2.17.1
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe dmaengine" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
---
To unsubscribe from this list: send the line "unsubscribe dmaengine" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [1/6] dmaengine: xilinx_dma: fix splitting transfer causes misalignments
From: Andrea Merello @ 2018-06-20 13:15 UTC (permalink / raw)
To: Radhey Shyam Pandey
Cc: vkoul@kernel.org, dan.j.williams@intel.com, Michal Simek,
Appana Durga Kedareswara Rao, dmaengine@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org
On Wed, Jun 20, 2018 at 1:37 PM, Radhey Shyam Pandey <radheys@xilinx.com> wrote:
>> -----Original Message-----
>> From: dmaengine-owner@vger.kernel.org [mailto:dmaengine-
>> owner@vger.kernel.org] On Behalf Of Andrea Merello
>> Sent: Wednesday, June 20, 2018 2:07 PM
>> To: vkoul@kernel.org; dan.j.williams@intel.com; Michal Simek
>> <michals@xilinx.com>; Appana Durga Kedareswara Rao
>> <appanad@xilinx.com>; dmaengine@vger.kernel.org
>> Cc: linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org;
>> Andrea Merello <andrea.merello@gmail.com>
>> Subject: [PATCH 1/6] dmaengine: xilinx_dma: fix splitting transfer causes
>> misalignments
>
> We should rephrase commit message to something like "In axidma
> slave_sg and dma_cylic mode align split descriptors"
OK
>>
>> Whenever a single or cyclic transaction is prepared, the driver
>> could eventually split it over several SG descriptors in order
>> to deal with the HW maximum transfer length.
>>
>> This could end up in DMA operations starting from a misaligned
>> address. This seems fatal for the HW.
> This seems fatal for the HW if DRE is not enabled.
OK
>>
>> This patch eventually adjusts the transfer size in order to make sure
>> all operations start from an aligned address.
>>
>> Signed-off-by: Andrea Merello <andrea.merello@gmail.com>
>> ---
>> drivers/dma/xilinx/xilinx_dma.c | 27 ++++++++++++++++++++-------
>> 1 file changed, 20 insertions(+), 7 deletions(-)
>>
>> diff --git a/drivers/dma/xilinx/xilinx_dma.c b/drivers/dma/xilinx/xilinx_dma.c
>> index 27b523530c4a..a516e7ffef21 100644
>> --- a/drivers/dma/xilinx/xilinx_dma.c
>> +++ b/drivers/dma/xilinx/xilinx_dma.c
>> @@ -376,6 +376,7 @@ struct xilinx_dma_chan {
>> void (*start_transfer)(struct xilinx_dma_chan *chan);
>> int (*stop_transfer)(struct xilinx_dma_chan *chan);
>> u16 tdest;
>> + u32 copy_mask;
> We can reuse copy_align itself. See below.
OK
>> };
>>
>> /**
>> @@ -1789,10 +1790,14 @@ static struct dma_async_tx_descriptor
>> *xilinx_dma_prep_slave_sg(
>>
>> /*
>> * Calculate the maximum number of bytes to transfer,
>> - * making sure it is less than the hw limit
>> + * making sure it is less than the hw limit and that
>> + * the next chuck start address is aligned
>
> /s/chuck/chunk
OK
>> */
>> - copy = min_t(size_t, sg_dma_len(sg) - sg_used,
>> - XILINX_DMA_MAX_TRANS_LEN);
>> + copy = sg_dma_len(sg) - sg_used;
>> + if (copy > XILINX_DMA_MAX_TRANS_LEN)
>> + copy = XILINX_DMA_MAX_TRANS_LEN &
>> + chan->copy_mask;
>> +
>
>
> In below implementation, we can reuse copy_align.
> Same for dma_cyclic.
>
> if ((copy + sg_used < sg_dma_len(sg)) &&
> chan->xdev->common.copy_align) {
> /* If this is not the last descriptor, make sure
> * the next one will be properly aligned
> */
> copy = rounddown(copy,
> (1 << chan->xdev->common.copy_align));
> }
OK for the general idea. But to me it seems a bit more complicated than needed:
What's the point in setting 'copy' with min_t, performing also the
subtraction sg_dma_len(sg) - sg_used, and then add sg_used again? What
about something like:
- copy = min_t(size_t, sg_dma_len(sg) - sg_used,
- XILINX_DMA_MAX_TRANS_LEN);
+ copy = sg_dma_len(sg) - sg_used;
+ if (copy > XILINX_DMA_MAX_TRANS_LEN &&
+ chan->xdev->common.copy_align)
+ copy = rounddown(XILINX_DMA_MAX_TRANS_LEN,
+ (1 << chan->xdev->common.copy_align));
+
>> hw = &segment->hw;
>>
>> /* Fill in the descriptor */
>> @@ -1894,10 +1899,14 @@ static struct dma_async_tx_descriptor
>> *xilinx_dma_prep_dma_cyclic(
>>
>> /*
>> * Calculate the maximum number of bytes to transfer,
>> - * making sure it is less than the hw limit
>> + * making sure it is less than the hw limit and that
>> + * the next chuck start address is aligned
>> */
>> - copy = min_t(size_t, period_len - sg_used,
>> - XILINX_DMA_MAX_TRANS_LEN);
>> + copy = period_len - sg_used;
>> + if (copy > XILINX_DMA_MAX_TRANS_LEN)
>> + copy = XILINX_DMA_MAX_TRANS_LEN &
>> + chan->copy_mask;
>> +
>> hw = &segment->hw;
>> xilinx_axidma_buf(chan, hw, buf_addr, sg_used,
>> period_len * i);
>> @@ -2402,8 +2411,12 @@ static int xilinx_dma_chan_probe(struct
>> xilinx_dma_device *xdev,
>> if (width > 8)
>> has_dre = false;
>>
>> - if (!has_dre)
>> + if (has_dre) {
>> + chan->copy_mask = ~0;
>> + } else {
>> xdev->common.copy_align = fls(width - 1);
>> + chan->copy_mask = ~(width - 1);
>> + }
>
> As mentioned above we don't need this additional field.
OK
>>
>> if (of_device_is_compatible(node, "xlnx,axi-vdma-mm2s-channel") ||
>> of_device_is_compatible(node, "xlnx,axi-dma-mm2s-channel") ||
>> --
>> 2.17.1
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe dmaengine" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
---
To unsubscribe from this list: send the line "unsubscribe dmaengine" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [2/6] dmaengine: xilinx_dma: fix completion callback is not invoked for each DMA operation
From: Radhey Shyam Pandey @ 2018-06-20 12:36 UTC (permalink / raw)
To: Andrea Merello, vkoul@kernel.org, dan.j.williams@intel.com,
Michal Simek, Appana Durga Kedareswara Rao,
dmaengine@vger.kernel.org
Cc: linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org
> -----Original Message-----
> From: dmaengine-owner@vger.kernel.org [mailto:dmaengine-
> owner@vger.kernel.org] On Behalf Of Andrea Merello
> Sent: Wednesday, June 20, 2018 2:07 PM
> To: vkoul@kernel.org; dan.j.williams@intel.com; Michal Simek
> <michals@xilinx.com>; Appana Durga Kedareswara Rao
> <appanad@xilinx.com>; dmaengine@vger.kernel.org
> Cc: linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org;
> Andrea Merello <andrea.merello@gmail.com>
> Subject: [PATCH 2/6] dmaengine: xilinx_dma: fix completion callback is not
> invoked for each DMA operation
>
> API specification says: "On completion of each DMA operation, the next in
> queue is started and a tasklet triggered. The tasklet will then call the
> client driver completion callback routine for notification, if set."
>
> Currently the driver keeps a "desc_pendingcount" counter of the total
> descriptor pending, and it uses as IRQ coalesce threshold, as result it
> only calls the CBs after ALL pending operations are completed, which is
> wrong.
I think IRQ coalescing enable/disable should be configurable.
Performance related usecases will need this support.
>
> This patch uses disable IRQ coalesce and checks for the completion flag
> for the descriptors (which is further divided in segments).
>
> Possibly a better optimization could be using proper IRQ coalesce
> threshold to get an IRQ after all segments of the descriptors are done.
> But we don't do that yet..
>
> NOTE: for now we do this only for AXI DMA, other DMA flavors are
> untested/untouched.
> This is loosely based on
> commit 65df81a6dc74 ("xilinx_dma: IrqThreshold set incorrectly, unreliable.")
> in my linux-4.6-zynq tree
NOTE description doesn't help much.
>
> From: Jeremy Trimble [original patch]
> Signed-off-by: Andrea Merello <andrea.merello@gmail.com>
> ---
> drivers/dma/xilinx/xilinx_dma.c | 39 +++++++++++++++++++++------------
> 1 file changed, 25 insertions(+), 14 deletions(-)
>
> diff --git a/drivers/dma/xilinx/xilinx_dma.c b/drivers/dma/xilinx/xilinx_dma.c
> index a516e7ffef21..cf12f7147f07 100644
> --- a/drivers/dma/xilinx/xilinx_dma.c
> +++ b/drivers/dma/xilinx/xilinx_dma.c
> @@ -164,6 +164,7 @@
> #define XILINX_DMA_CR_COALESCE_SHIFT 16
> #define XILINX_DMA_BD_SOP BIT(27)
> #define XILINX_DMA_BD_EOP BIT(26)
> +#define XILINX_DMA_BD_CMPLT BIT(31)
> #define XILINX_DMA_COALESCE_MAX 255
> #define XILINX_DMA_NUM_DESCS 255
> #define XILINX_DMA_NUM_APP_WORDS 5
> @@ -1274,12 +1275,9 @@ static void xilinx_dma_start_transfer(struct
> xilinx_dma_chan *chan)
>
> reg = dma_ctrl_read(chan, XILINX_DMA_REG_DMACR);
>
> - if (chan->desc_pendingcount <= XILINX_DMA_COALESCE_MAX) {
> - reg &= ~XILINX_DMA_CR_COALESCE_MAX;
> - reg |= chan->desc_pendingcount <<
> - XILINX_DMA_CR_COALESCE_SHIFT;
> - dma_ctrl_write(chan, XILINX_DMA_REG_DMACR, reg);
> - }
> + reg &= ~XILINX_DMA_CR_COALESCE_MAX;
> + reg |= 1 << XILINX_DMA_CR_COALESCE_SHIFT;
> + dma_ctrl_write(chan, XILINX_DMA_REG_DMACR, reg);
>
> if (chan->has_sg && !chan->xdev->mcdma)
> xilinx_write(chan, XILINX_DMA_REG_CURDESC,
> @@ -1378,6 +1376,20 @@ static void xilinx_dma_complete_descriptor(struct
> xilinx_dma_chan *chan)
> return;
>
> list_for_each_entry_safe(desc, next, &chan->active_list, node) {
> + if (chan->xdev->dma_config->dmatype ==
> XDMA_TYPE_AXIDMA) {
> + /*
> + * Check whether the last segment in this descriptor
> + * has been completed.
> + */
> + const struct xilinx_axidma_tx_segment *const tail_seg
> =
> + list_last_entry(&desc->segments,
> + struct
> xilinx_axidma_tx_segment,
> + node);
> +
> + /* we've processed all the completed descriptors */
> + if (!(tail_seg->hw.status & XILINX_DMA_BD_CMPLT))
> + break;
> + }
> list_del(&desc->node);
> if (!desc->cyclic)
> dma_cookie_complete(&desc->async_tx);
> @@ -1826,14 +1838,13 @@ static struct dma_async_tx_descriptor
> *xilinx_dma_prep_slave_sg(
> struct xilinx_axidma_tx_segment, node);
> desc->async_tx.phys = segment->phys;
>
> - /* For the last DMA_MEM_TO_DEV transfer, set EOP */
> - if (chan->direction == DMA_MEM_TO_DEV) {
> - segment->hw.control |= XILINX_DMA_BD_SOP;
> - segment = list_last_entry(&desc->segments,
> - struct xilinx_axidma_tx_segment,
> - node);
> - segment->hw.control |= XILINX_DMA_BD_EOP;
> - }
> + /* For the first transfer, set SOP */
> + segment->hw.control |= XILINX_DMA_BD_SOP;
> + /* For the last transfer, set EOP */
> + segment = list_last_entry(&desc->segments,
> + struct xilinx_axidma_tx_segment,
> + node);
> + segment->hw.control |= XILINX_DMA_BD_EOP;
>
> return &desc->async_tx;
>
> --
> 2.17.1
>
> --
> To unsubscribe from this list: send the line "unsubscribe dmaengine" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
---
To unsubscribe from this list: send the line "unsubscribe dmaengine" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [resend] dmaengine: rcar-dmac: Document R8A77990 bindings
From: Ulrich Hecht @ 2018-06-20 12:01 UTC (permalink / raw)
To: linux-renesas-soc
Cc: horms, dmaengine, devicetree, vinod.koul, Hiroyuki Yokoyama,
Ulrich Hecht
From: Hiroyuki Yokoyama <hiroyuki.yokoyama.vx@renesas.com>
Renesas R-Car E3 (R8A77990) SoC also has the R-Car gen2/3 compatible DMA
controllers, so document the SoC specific binding.
Signed-off-by: Hiroyuki Yokoyama <hiroyuki.yokoyama.vx@renesas.com>
Signed-off-by: Ulrich Hecht <ulrich.hecht+renesas@gmail.com>
Reviewed-by: Simon Horman <horms+renesas@verge.net.au>
Acked-by: Rob Herring <robh@kernel.org>
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
---
This didn't apply for Vinod before, but now it seems to. If it still does
not, please tell me what tree to base this on.
CU
Uli
Documentation/devicetree/bindings/dma/renesas,rcar-dmac.txt | 1 +
1 file changed, 1 insertion(+)
diff --git a/Documentation/devicetree/bindings/dma/renesas,rcar-dmac.txt b/Documentation/devicetree/bindings/dma/renesas,rcar-dmac.txt
index b1ba6395..946229c 100644
--- a/Documentation/devicetree/bindings/dma/renesas,rcar-dmac.txt
+++ b/Documentation/devicetree/bindings/dma/renesas,rcar-dmac.txt
@@ -29,6 +29,7 @@ Required Properties:
- "renesas,dmac-r8a77965" (R-Car M3-N)
- "renesas,dmac-r8a77970" (R-Car V3M)
- "renesas,dmac-r8a77980" (R-Car V3H)
+ - "renesas,dmac-r8a77990" (R-Car E3)
- "renesas,dmac-r8a77995" (R-Car D3)
- reg: base address and length of the registers block for the DMAC
^ permalink raw reply related
* [1/6] dmaengine: xilinx_dma: fix splitting transfer causes misalignments
From: Radhey Shyam Pandey @ 2018-06-20 11:37 UTC (permalink / raw)
To: Andrea Merello, vkoul@kernel.org, dan.j.williams@intel.com,
Michal Simek, Appana Durga Kedareswara Rao,
dmaengine@vger.kernel.org
Cc: linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org
> -----Original Message-----
> From: dmaengine-owner@vger.kernel.org [mailto:dmaengine-
> owner@vger.kernel.org] On Behalf Of Andrea Merello
> Sent: Wednesday, June 20, 2018 2:07 PM
> To: vkoul@kernel.org; dan.j.williams@intel.com; Michal Simek
> <michals@xilinx.com>; Appana Durga Kedareswara Rao
> <appanad@xilinx.com>; dmaengine@vger.kernel.org
> Cc: linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org;
> Andrea Merello <andrea.merello@gmail.com>
> Subject: [PATCH 1/6] dmaengine: xilinx_dma: fix splitting transfer causes
> misalignments
We should rephrase commit message to something like "In axidma
slave_sg and dma_cylic mode align split descriptors"
>
> Whenever a single or cyclic transaction is prepared, the driver
> could eventually split it over several SG descriptors in order
> to deal with the HW maximum transfer length.
>
> This could end up in DMA operations starting from a misaligned
> address. This seems fatal for the HW.
This seems fatal for the HW if DRE is not enabled.
>
> This patch eventually adjusts the transfer size in order to make sure
> all operations start from an aligned address.
>
> Signed-off-by: Andrea Merello <andrea.merello@gmail.com>
> ---
> drivers/dma/xilinx/xilinx_dma.c | 27 ++++++++++++++++++++-------
> 1 file changed, 20 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/dma/xilinx/xilinx_dma.c b/drivers/dma/xilinx/xilinx_dma.c
> index 27b523530c4a..a516e7ffef21 100644
> --- a/drivers/dma/xilinx/xilinx_dma.c
> +++ b/drivers/dma/xilinx/xilinx_dma.c
> @@ -376,6 +376,7 @@ struct xilinx_dma_chan {
> void (*start_transfer)(struct xilinx_dma_chan *chan);
> int (*stop_transfer)(struct xilinx_dma_chan *chan);
> u16 tdest;
> + u32 copy_mask;
We can reuse copy_align itself. See below.
> };
>
> /**
> @@ -1789,10 +1790,14 @@ static struct dma_async_tx_descriptor
> *xilinx_dma_prep_slave_sg(
>
> /*
> * Calculate the maximum number of bytes to transfer,
> - * making sure it is less than the hw limit
> + * making sure it is less than the hw limit and that
> + * the next chuck start address is aligned
/s/chuck/chunk
> */
> - copy = min_t(size_t, sg_dma_len(sg) - sg_used,
> - XILINX_DMA_MAX_TRANS_LEN);
> + copy = sg_dma_len(sg) - sg_used;
> + if (copy > XILINX_DMA_MAX_TRANS_LEN)
> + copy = XILINX_DMA_MAX_TRANS_LEN &
> + chan->copy_mask;
> +
In below implementation, we can reuse copy_align.
Same for dma_cyclic.
if ((copy + sg_used < sg_dma_len(sg)) &&
chan->xdev->common.copy_align) {
/* If this is not the last descriptor, make sure
* the next one will be properly aligned
*/
copy = rounddown(copy,
(1 << chan->xdev->common.copy_align));
}
> hw = &segment->hw;
>
> /* Fill in the descriptor */
> @@ -1894,10 +1899,14 @@ static struct dma_async_tx_descriptor
> *xilinx_dma_prep_dma_cyclic(
>
> /*
> * Calculate the maximum number of bytes to transfer,
> - * making sure it is less than the hw limit
> + * making sure it is less than the hw limit and that
> + * the next chuck start address is aligned
> */
> - copy = min_t(size_t, period_len - sg_used,
> - XILINX_DMA_MAX_TRANS_LEN);
> + copy = period_len - sg_used;
> + if (copy > XILINX_DMA_MAX_TRANS_LEN)
> + copy = XILINX_DMA_MAX_TRANS_LEN &
> + chan->copy_mask;
> +
> hw = &segment->hw;
> xilinx_axidma_buf(chan, hw, buf_addr, sg_used,
> period_len * i);
> @@ -2402,8 +2411,12 @@ static int xilinx_dma_chan_probe(struct
> xilinx_dma_device *xdev,
> if (width > 8)
> has_dre = false;
>
> - if (!has_dre)
> + if (has_dre) {
> + chan->copy_mask = ~0;
> + } else {
> xdev->common.copy_align = fls(width - 1);
> + chan->copy_mask = ~(width - 1);
> + }
As mentioned above we don't need this additional field.
>
> if (of_device_is_compatible(node, "xlnx,axi-vdma-mm2s-channel") ||
> of_device_is_compatible(node, "xlnx,axi-dma-mm2s-channel") ||
> --
> 2.17.1
>
> --
> To unsubscribe from this list: send the line "unsubscribe dmaengine" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
---
To unsubscribe from this list: send the line "unsubscribe dmaengine" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [6/6] dt-bindings: xilinx_dma: drop has-sg property
From: Andrea Merello @ 2018-06-20 8:36 UTC (permalink / raw)
To: vkoul, dan.j.williams, michal.simek, appana.durga.rao, dmaengine
Cc: linux-arm-kernel, linux-kernel, Andrea Merello
This property is not needed anymore, because the driver now autodetects it.
Delete references in documentation.
Signed-off-by: Andrea Merello <andrea.merello@gmail.com>
---
Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt | 3 ---
1 file changed, 3 deletions(-)
diff --git a/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt b/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt
index acecdc5d8d47..53341314eeb8 100644
--- a/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt
+++ b/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt
@@ -39,9 +39,6 @@ Required properties for VDMA:
Required properties for AXI DMA:
- xlnx,lengthregwidth: Should be the width of the length register as configured in h/w.
-Optional properties:
-- xlnx,include-sg: Tells configured for Scatter-mode in
- the hardware.
Optional properties for AXI DMA:
- xlnx,mcdma: Tells whether configured for multi-channel mode in the hardware.
Optional properties for VDMA:
^ permalink raw reply related
* [5/6] dmaengine: xilinx_dma: autodetect whether the HW supports scatter-gather
From: Andrea Merello @ 2018-06-20 8:36 UTC (permalink / raw)
To: vkoul, dan.j.williams, michal.simek, appana.durga.rao, dmaengine
Cc: linux-arm-kernel, linux-kernel, Andrea Merello
The HW can be either direct-access or scatter-gather version. These are
SW incompatible.
The driver can handle both version: a DT property was used to
tell the driver whether to assume the HW is is scatter-gather mode.
This patch makes the driver to autodetect this information. The DT
property is not required anymore.
Signed-off-by: Andrea Merello <andrea.merello@gmail.com>
---
drivers/dma/xilinx/xilinx_dma.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/drivers/dma/xilinx/xilinx_dma.c b/drivers/dma/xilinx/xilinx_dma.c
index bdbc8ba9092a..8c6e818e596f 100644
--- a/drivers/dma/xilinx/xilinx_dma.c
+++ b/drivers/dma/xilinx/xilinx_dma.c
@@ -86,6 +86,7 @@
#define XILINX_DMA_DMASR_DMA_DEC_ERR BIT(6)
#define XILINX_DMA_DMASR_DMA_SLAVE_ERR BIT(5)
#define XILINX_DMA_DMASR_DMA_INT_ERR BIT(4)
+#define XILINX_DMA_DMASR_SG_MASK BIT(3)
#define XILINX_DMA_DMASR_IDLE BIT(1)
#define XILINX_DMA_DMASR_HALTED BIT(0)
#define XILINX_DMA_DMASR_DELAY_MASK GENMASK(31, 24)
@@ -407,7 +408,6 @@ struct xilinx_dma_config {
* @dev: Device Structure
* @common: DMA device structure
* @chan: Driver specific DMA channel
- * @has_sg: Specifies whether Scatter-Gather is present or not
* @mcdma: Specifies whether Multi-Channel is present or not
* @flush_on_fsync: Flush on frame sync
* @ext_addr: Indicates 64 bit addressing is supported by dma device
@@ -426,7 +426,6 @@ struct xilinx_dma_device {
struct device *dev;
struct dma_device common;
struct xilinx_dma_chan *chan[XILINX_DMA_MAX_CHANS_PER_DEVICE];
- bool has_sg;
bool mcdma;
u32 flush_on_fsync;
bool ext_addr;
@@ -2391,7 +2390,6 @@ static int xilinx_dma_chan_probe(struct xilinx_dma_device *xdev,
chan->dev = xdev->dev;
chan->xdev = xdev;
- chan->has_sg = xdev->has_sg;
chan->desc_pendingcount = 0x0;
chan->ext_addr = xdev->ext_addr;
/* This variable ensures that descriptors are not
@@ -2488,6 +2486,13 @@ static int xilinx_dma_chan_probe(struct xilinx_dma_device *xdev,
chan->stop_transfer = xilinx_dma_stop_transfer;
}
+ /* check if SG is enabled */
+ if (dma_ctrl_read(chan, XILINX_DMA_REG_DMASR) &
+ XILINX_DMA_DMASR_SG_MASK)
+ chan->has_sg = true;
+ dev_dbg(chan->dev, "ch %d: SG %s\n", chan->id,
+ chan->has_sg ? "enabled" : "disabled");
+
/* Initialize the tasklet */
tasklet_init(&chan->tasklet, xilinx_dma_do_tasklet,
(unsigned long)chan);
@@ -2626,7 +2631,6 @@ static int xilinx_dma_probe(struct platform_device *pdev)
return PTR_ERR(xdev->regs);
/* Retrieve the DMA engine properties from the device tree */
-
if (xdev->dma_config->dmatype == XDMA_TYPE_AXIDMA) {
xdev->mcdma = of_property_read_bool(node, "xlnx,mcdma");
err = of_property_read_u32(node, "xlnx,lengthregwidth",
^ permalink raw reply related
* [4/6] dmaengine: xilinx_dma: fix hardcoded maximum transfer length may be wrong
From: Andrea Merello @ 2018-06-20 8:36 UTC (permalink / raw)
To: vkoul, dan.j.williams, michal.simek, appana.durga.rao, dmaengine
Cc: linux-arm-kernel, linux-kernel, Andrea Merello
The maximum transfer length is currently hardcoded in the driver, but
it depends by how the soft-IP is actually configured.
This seems to affect also max possible length for SG transfers.
This patch introduce a new DT property in order to operate with proper
maximum transfer length.
Signed-off-by: Andrea Merello <andrea.merello@gmail.com>
---
drivers/dma/xilinx/xilinx_dma.c | 24 +++++++++++++++++-------
1 file changed, 17 insertions(+), 7 deletions(-)
diff --git a/drivers/dma/xilinx/xilinx_dma.c b/drivers/dma/xilinx/xilinx_dma.c
index cf12f7147f07..bdbc8ba9092a 100644
--- a/drivers/dma/xilinx/xilinx_dma.c
+++ b/drivers/dma/xilinx/xilinx_dma.c
@@ -439,6 +439,7 @@ struct xilinx_dma_device {
struct clk *rxs_clk;
u32 nr_channels;
u32 chan_id;
+ int max_transfer;
};
/* Macros */
@@ -1806,8 +1807,8 @@ static struct dma_async_tx_descriptor *xilinx_dma_prep_slave_sg(
* the next chuck start address is aligned
*/
copy = sg_dma_len(sg) - sg_used;
- if (copy > XILINX_DMA_MAX_TRANS_LEN)
- copy = XILINX_DMA_MAX_TRANS_LEN &
+ if (copy > chan->xdev->max_transfer)
+ copy = chan->xdev->max_transfer &
chan->copy_mask;
hw = &segment->hw;
@@ -1914,8 +1915,8 @@ static struct dma_async_tx_descriptor *xilinx_dma_prep_dma_cyclic(
* the next chuck start address is aligned
*/
copy = period_len - sg_used;
- if (copy > XILINX_DMA_MAX_TRANS_LEN)
- copy = XILINX_DMA_MAX_TRANS_LEN &
+ if (copy > chan->xdev->max_transfer)
+ copy = chan->xdev->max_transfer &
chan->copy_mask;
hw = &segment->hw;
@@ -2594,7 +2595,7 @@ static int xilinx_dma_probe(struct platform_device *pdev)
struct xilinx_dma_device *xdev;
struct device_node *child, *np = pdev->dev.of_node;
struct resource *io;
- u32 num_frames, addr_width;
+ u32 num_frames, addr_width, lenreg_width;
int i, err;
/* Allocate and initialize the DMA engine structure */
@@ -2625,9 +2626,18 @@ static int xilinx_dma_probe(struct platform_device *pdev)
return PTR_ERR(xdev->regs);
/* Retrieve the DMA engine properties from the device tree */
- xdev->has_sg = of_property_read_bool(node, "xlnx,include-sg");
- if (xdev->dma_config->dmatype == XDMA_TYPE_AXIDMA)
+
+ if (xdev->dma_config->dmatype == XDMA_TYPE_AXIDMA) {
xdev->mcdma = of_property_read_bool(node, "xlnx,mcdma");
+ err = of_property_read_u32(node, "xlnx,lengthregwidth",
+ &lenreg_width);
+ if (err < 0) {
+ dev_err(xdev->dev,
+ "missing xlnx,lengthregwidth property\n");
+ return err;
+ }
+ xdev->max_transfer = GENMASK(lenreg_width - 1, 0);
+ }
if (xdev->dma_config->dmatype == XDMA_TYPE_VDMA) {
err = of_property_read_u32(node, "xlnx,num-fstores",
^ permalink raw reply related
* [3/6] dt-bindings: xilinx_dma: add required xlnx,lengthregwidth property
From: Andrea Merello @ 2018-06-20 8:36 UTC (permalink / raw)
To: vkoul, dan.j.williams, michal.simek, appana.durga.rao, dmaengine
Cc: linux-arm-kernel, linux-kernel, Andrea Merello
The width of the "length register" cannot be autodetected, and it is now
specified with a DT property. Add DOC for it.
Signed-off-by: Andrea Merello <andrea.merello@gmail.com>
---
Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt | 2 ++
1 file changed, 2 insertions(+)
diff --git a/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt b/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt
index a2b8bfaec43c..acecdc5d8d47 100644
--- a/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt
+++ b/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt
@@ -36,6 +36,8 @@ Required properties:
Required properties for VDMA:
- xlnx,num-fstores: Should be the number of framebuffers as configured in h/w.
+Required properties for AXI DMA:
+- xlnx,lengthregwidth: Should be the width of the length register as configured in h/w.
Optional properties:
- xlnx,include-sg: Tells configured for Scatter-mode in
^ permalink raw reply related
* [2/6] dmaengine: xilinx_dma: fix completion callback is not invoked for each DMA operation
From: Andrea Merello @ 2018-06-20 8:36 UTC (permalink / raw)
To: vkoul, dan.j.williams, michal.simek, appana.durga.rao, dmaengine
Cc: linux-arm-kernel, linux-kernel, Andrea Merello
API specification says: "On completion of each DMA operation, the next in
queue is started and a tasklet triggered. The tasklet will then call the
client driver completion callback routine for notification, if set."
Currently the driver keeps a "desc_pendingcount" counter of the total
descriptor pending, and it uses as IRQ coalesce threshold, as result it
only calls the CBs after ALL pending operations are completed, which is
wrong.
This patch uses disable IRQ coalesce and checks for the completion flag
for the descriptors (which is further divided in segments).
Possibly a better optimization could be using proper IRQ coalesce
threshold to get an IRQ after all segments of the descriptors are done.
But we don't do that yet..
NOTE: for now we do this only for AXI DMA, other DMA flavors are
untested/untouched.
This is loosely based on
commit 65df81a6dc74 ("xilinx_dma: IrqThreshold set incorrectly, unreliable.")
in my linux-4.6-zynq tree
From: Jeremy Trimble [original patch]
Signed-off-by: Andrea Merello <andrea.merello@gmail.com>
---
drivers/dma/xilinx/xilinx_dma.c | 39 +++++++++++++++++++++------------
1 file changed, 25 insertions(+), 14 deletions(-)
diff --git a/drivers/dma/xilinx/xilinx_dma.c b/drivers/dma/xilinx/xilinx_dma.c
index a516e7ffef21..cf12f7147f07 100644
--- a/drivers/dma/xilinx/xilinx_dma.c
+++ b/drivers/dma/xilinx/xilinx_dma.c
@@ -164,6 +164,7 @@
#define XILINX_DMA_CR_COALESCE_SHIFT 16
#define XILINX_DMA_BD_SOP BIT(27)
#define XILINX_DMA_BD_EOP BIT(26)
+#define XILINX_DMA_BD_CMPLT BIT(31)
#define XILINX_DMA_COALESCE_MAX 255
#define XILINX_DMA_NUM_DESCS 255
#define XILINX_DMA_NUM_APP_WORDS 5
@@ -1274,12 +1275,9 @@ static void xilinx_dma_start_transfer(struct xilinx_dma_chan *chan)
reg = dma_ctrl_read(chan, XILINX_DMA_REG_DMACR);
- if (chan->desc_pendingcount <= XILINX_DMA_COALESCE_MAX) {
- reg &= ~XILINX_DMA_CR_COALESCE_MAX;
- reg |= chan->desc_pendingcount <<
- XILINX_DMA_CR_COALESCE_SHIFT;
- dma_ctrl_write(chan, XILINX_DMA_REG_DMACR, reg);
- }
+ reg &= ~XILINX_DMA_CR_COALESCE_MAX;
+ reg |= 1 << XILINX_DMA_CR_COALESCE_SHIFT;
+ dma_ctrl_write(chan, XILINX_DMA_REG_DMACR, reg);
if (chan->has_sg && !chan->xdev->mcdma)
xilinx_write(chan, XILINX_DMA_REG_CURDESC,
@@ -1378,6 +1376,20 @@ static void xilinx_dma_complete_descriptor(struct xilinx_dma_chan *chan)
return;
list_for_each_entry_safe(desc, next, &chan->active_list, node) {
+ if (chan->xdev->dma_config->dmatype == XDMA_TYPE_AXIDMA) {
+ /*
+ * Check whether the last segment in this descriptor
+ * has been completed.
+ */
+ const struct xilinx_axidma_tx_segment *const tail_seg =
+ list_last_entry(&desc->segments,
+ struct xilinx_axidma_tx_segment,
+ node);
+
+ /* we've processed all the completed descriptors */
+ if (!(tail_seg->hw.status & XILINX_DMA_BD_CMPLT))
+ break;
+ }
list_del(&desc->node);
if (!desc->cyclic)
dma_cookie_complete(&desc->async_tx);
@@ -1826,14 +1838,13 @@ static struct dma_async_tx_descriptor *xilinx_dma_prep_slave_sg(
struct xilinx_axidma_tx_segment, node);
desc->async_tx.phys = segment->phys;
- /* For the last DMA_MEM_TO_DEV transfer, set EOP */
- if (chan->direction == DMA_MEM_TO_DEV) {
- segment->hw.control |= XILINX_DMA_BD_SOP;
- segment = list_last_entry(&desc->segments,
- struct xilinx_axidma_tx_segment,
- node);
- segment->hw.control |= XILINX_DMA_BD_EOP;
- }
+ /* For the first transfer, set SOP */
+ segment->hw.control |= XILINX_DMA_BD_SOP;
+ /* For the last transfer, set EOP */
+ segment = list_last_entry(&desc->segments,
+ struct xilinx_axidma_tx_segment,
+ node);
+ segment->hw.control |= XILINX_DMA_BD_EOP;
return &desc->async_tx;
^ permalink raw reply related
* [1/6] dmaengine: xilinx_dma: fix splitting transfer causes misalignments
From: Andrea Merello @ 2018-06-20 8:36 UTC (permalink / raw)
To: vkoul, dan.j.williams, michal.simek, appana.durga.rao, dmaengine
Cc: linux-arm-kernel, linux-kernel, Andrea Merello
Whenever a single or cyclic transaction is prepared, the driver
could eventually split it over several SG descriptors in order
to deal with the HW maximum transfer length.
This could end up in DMA operations starting from a misaligned
address. This seems fatal for the HW.
This patch eventually adjusts the transfer size in order to make sure
all operations start from an aligned address.
Signed-off-by: Andrea Merello <andrea.merello@gmail.com>
---
drivers/dma/xilinx/xilinx_dma.c | 27 ++++++++++++++++++++-------
1 file changed, 20 insertions(+), 7 deletions(-)
diff --git a/drivers/dma/xilinx/xilinx_dma.c b/drivers/dma/xilinx/xilinx_dma.c
index 27b523530c4a..a516e7ffef21 100644
--- a/drivers/dma/xilinx/xilinx_dma.c
+++ b/drivers/dma/xilinx/xilinx_dma.c
@@ -376,6 +376,7 @@ struct xilinx_dma_chan {
void (*start_transfer)(struct xilinx_dma_chan *chan);
int (*stop_transfer)(struct xilinx_dma_chan *chan);
u16 tdest;
+ u32 copy_mask;
};
/**
@@ -1789,10 +1790,14 @@ static struct dma_async_tx_descriptor *xilinx_dma_prep_slave_sg(
/*
* Calculate the maximum number of bytes to transfer,
- * making sure it is less than the hw limit
+ * making sure it is less than the hw limit and that
+ * the next chuck start address is aligned
*/
- copy = min_t(size_t, sg_dma_len(sg) - sg_used,
- XILINX_DMA_MAX_TRANS_LEN);
+ copy = sg_dma_len(sg) - sg_used;
+ if (copy > XILINX_DMA_MAX_TRANS_LEN)
+ copy = XILINX_DMA_MAX_TRANS_LEN &
+ chan->copy_mask;
+
hw = &segment->hw;
/* Fill in the descriptor */
@@ -1894,10 +1899,14 @@ static struct dma_async_tx_descriptor *xilinx_dma_prep_dma_cyclic(
/*
* Calculate the maximum number of bytes to transfer,
- * making sure it is less than the hw limit
+ * making sure it is less than the hw limit and that
+ * the next chuck start address is aligned
*/
- copy = min_t(size_t, period_len - sg_used,
- XILINX_DMA_MAX_TRANS_LEN);
+ copy = period_len - sg_used;
+ if (copy > XILINX_DMA_MAX_TRANS_LEN)
+ copy = XILINX_DMA_MAX_TRANS_LEN &
+ chan->copy_mask;
+
hw = &segment->hw;
xilinx_axidma_buf(chan, hw, buf_addr, sg_used,
period_len * i);
@@ -2402,8 +2411,12 @@ static int xilinx_dma_chan_probe(struct xilinx_dma_device *xdev,
if (width > 8)
has_dre = false;
- if (!has_dre)
+ if (has_dre) {
+ chan->copy_mask = ~0;
+ } else {
xdev->common.copy_align = fls(width - 1);
+ chan->copy_mask = ~(width - 1);
+ }
if (of_device_is_compatible(node, "xlnx,axi-vdma-mm2s-channel") ||
of_device_is_compatible(node, "xlnx,axi-dma-mm2s-channel") ||
^ permalink raw reply related
* [v4,2/2] dmaengine: fsl-edma: add ColdFire mcf5441x edma support
From: Angelo Dureghello @ 2018-06-19 19:41 UTC (permalink / raw)
To: vinod.koul; +Cc: dmaengine, linux-m68k, Angelo Dureghello
This patch adds support for ColdFire mcf5441x-family edma
module.
The ColdFire edma module is slightly different from fsl-edma,
so a new driver is added. But most of the code is common
between fsl-edma and mcf-edma so it has been collected into a
separate common module fsl-edma-common (patch 2/4).
Signed-off-by: Angelo Dureghello <angelo@sysam.it>
---
Changes for v2:
- patch splitted into 4
- add mcf-edma as minimal different parts from fsl-edma
Changes for v3:
none
Changes for v4:
- patch simplified from 4/4 into 2/2
- collecting all the mcf-edma-related changes
---
drivers/dma/Kconfig | 11 +
drivers/dma/Makefile | 1 +
drivers/dma/mcf-edma.c | 299 +++++++++++++++++++++
include/linux/platform_data/dma-mcf-edma.h | 38 +++
4 files changed, 349 insertions(+)
create mode 100644 drivers/dma/mcf-edma.c
create mode 100644 include/linux/platform_data/dma-mcf-edma.h
diff --git a/drivers/dma/Kconfig b/drivers/dma/Kconfig
index ca1680afa20a..23f444608514 100644
--- a/drivers/dma/Kconfig
+++ b/drivers/dma/Kconfig
@@ -320,6 +320,17 @@ config LPC18XX_DMAMUX
Enable support for DMA on NXP LPC18xx/43xx platforms
with PL080 and multiplexed DMA request lines.
+config MCF_EDMA
+ tristate "Freescale eDMA engine support, ColdFire mcf5441x SoCs"
+ depends on M5441x
+ select DMA_ENGINE
+ select DMA_VIRTUAL_CHANNELS
+ help
+ Support the Freescale ColdFire eDMA engine, 64-channel
+ implementation that performs complex data transfers with
+ minimal intervention from a host processor.
+ This module can be found on Freescale ColdFire mcf5441x SoCs.
+
config MMP_PDMA
bool "MMP PDMA support"
depends on ARCH_MMP || ARCH_PXA || COMPILE_TEST
diff --git a/drivers/dma/Makefile b/drivers/dma/Makefile
index 66022f59fca4..d97f317f4b34 100644
--- a/drivers/dma/Makefile
+++ b/drivers/dma/Makefile
@@ -32,6 +32,7 @@ obj-$(CONFIG_DW_DMAC_CORE) += dw/
obj-$(CONFIG_EP93XX_DMA) += ep93xx_dma.o
obj-$(CONFIG_FSL_DMA) += fsldma.o
obj-$(CONFIG_FSL_EDMA) += fsl-edma.o fsl-edma-common.o
+obj-$(CONFIG_MCF_EDMA) += mcf-edma.o fsl-edma-common.o
obj-$(CONFIG_FSL_RAID) += fsl_raid.o
obj-$(CONFIG_HSU_DMA) += hsu/
obj-$(CONFIG_IMG_MDC_DMA) += img-mdc-dma.o
diff --git a/drivers/dma/mcf-edma.c b/drivers/dma/mcf-edma.c
new file mode 100644
index 000000000000..9e1d55a5cc90
--- /dev/null
+++ b/drivers/dma/mcf-edma.c
@@ -0,0 +1,299 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2013-2014 Freescale Semiconductor, Inc
+// Copyright (c) 2017 Sysam, Angelo Dureghello <angelo@sysam.it>
+/*
+ * drivers/dma/mcf-edma.c
+ *
+ * Driver for the Freescale ColdFire 64-ch eDMA implementation,
+ * derived from drivers/dma/fsl-edma.c.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#include <linux/module.h>
+#include <linux/interrupt.h>
+#include <linux/dmaengine.h>
+#include <linux/platform_device.h>
+#include <linux/platform_data/dma-mcf-edma.h>
+
+#include "fsl-edma-common.h"
+
+#define EDMA_CHANNELS 64
+#define EDMA_MASK_CH(x) ((x) & GENMASK(5, 0))
+
+static irqreturn_t mcf_edma_tx_handler(int irq, void *dev_id)
+{
+ struct fsl_edma_engine *mcf_edma = dev_id;
+ struct edma_regs *regs = &mcf_edma->regs;
+ unsigned int ch;
+ struct fsl_edma_chan *mcf_chan;
+ u64 intmap;
+
+ intmap = ioread32(regs->inth);
+ intmap <<= 32;
+ intmap |= ioread32(regs->intl);
+ if (!intmap)
+ return IRQ_NONE;
+
+ for (ch = 0; ch < mcf_edma->n_chans; ch++) {
+ if (intmap & (0x1 << ch)) {
+ iowrite8(EDMA_MASK_CH(ch), regs->cint);
+
+ mcf_chan = &mcf_edma->chans[ch];
+
+ spin_lock(&mcf_chan->vchan.lock);
+ if (!mcf_chan->edesc->iscyclic) {
+ list_del(&mcf_chan->edesc->vdesc.node);
+ vchan_cookie_complete(&mcf_chan->edesc->vdesc);
+ mcf_chan->edesc = NULL;
+ mcf_chan->status = DMA_COMPLETE;
+ mcf_chan->idle = true;
+ } else {
+ vchan_cyclic_callback(&mcf_chan->edesc->vdesc);
+ }
+
+ if (!mcf_chan->edesc)
+ fsl_edma_xfer_desc(mcf_chan);
+
+ spin_unlock(&mcf_chan->vchan.lock);
+ }
+ }
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t mcf_edma_err_handler(int irq, void *dev_id)
+{
+ struct fsl_edma_engine *mcf_edma = dev_id;
+ struct edma_regs *regs = &mcf_edma->regs;
+ unsigned int err, ch;
+
+ err = ioread32(regs->errl);
+ if (!err)
+ return IRQ_NONE;
+
+ for (ch = 0; ch < (EDMA_CHANNELS / 2); ch++) {
+ if (err & (0x1 << ch)) {
+ fsl_edma_disable_request(&mcf_edma->chans[ch]);
+ iowrite8(EDMA_CERR_CERR(ch), regs->cerr);
+ mcf_edma->chans[ch].status = DMA_ERROR;
+ mcf_edma->chans[ch].idle = true;
+ }
+ }
+
+ err = ioread32(regs->errh);
+ if (!err)
+ return IRQ_NONE;
+
+ for (ch = (EDMA_CHANNELS / 2); ch < EDMA_CHANNELS; ch++) {
+ if (err & (0x1 << (ch - (EDMA_CHANNELS / 2)))) {
+ fsl_edma_disable_request(&mcf_edma->chans[ch]);
+ iowrite8(EDMA_CERR_CERR(ch), regs->cerr);
+ mcf_edma->chans[ch].status = DMA_ERROR;
+ mcf_edma->chans[ch].idle = true;
+ }
+ }
+
+ return IRQ_HANDLED;
+}
+
+static int mcf_edma_irq_init(struct platform_device *pdev,
+ struct fsl_edma_engine *mcf_edma)
+{
+ int ret = 0, i;
+ struct resource *res;
+
+ res = platform_get_resource_byname(pdev,
+ IORESOURCE_IRQ, "edma-tx-00-15");
+ if (!res)
+ return -1;
+
+ for (ret = 0, i = res->start; i <= res->end; ++i) {
+ ret |= devm_request_irq(&pdev->dev, i,
+ mcf_edma_tx_handler, 0, "eDMA", mcf_edma);
+ }
+ if (ret)
+ return ret;
+
+ res = platform_get_resource_byname(pdev,
+ IORESOURCE_IRQ, "edma-tx-16-55");
+ if (!res)
+ return -1;
+
+ for (ret = 0, i = res->start; i <= res->end; ++i) {
+ ret |= devm_request_irq(&pdev->dev, i,
+ mcf_edma_tx_handler, 0, "eDMA", mcf_edma);
+ }
+ if (ret)
+ return ret;
+
+ ret = platform_get_irq_byname(pdev, "edma-tx-56-63");
+ if (ret != -ENXIO) {
+ ret = devm_request_irq(&pdev->dev, ret,
+ mcf_edma_tx_handler, 0, "eDMA", mcf_edma);
+ if (ret)
+ return ret;
+ }
+
+ ret = platform_get_irq_byname(pdev, "edma-err");
+ if (ret != -ENXIO) {
+ ret = devm_request_irq(&pdev->dev, ret,
+ mcf_edma_err_handler, 0, "eDMA", mcf_edma);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int mcf_edma_probe(struct platform_device *pdev)
+{
+ struct mcf_edma_platform_data *pdata;
+ struct fsl_edma_engine *mcf_edma;
+ struct fsl_edma_chan *mcf_chan;
+ struct edma_regs *regs;
+ struct resource *res;
+ int ret, i, len, chans;
+
+ pdata = dev_get_platdata(&pdev->dev);
+ if (!pdata)
+ return PTR_ERR(pdata);
+
+ chans = pdata->dma_channels;
+ len = sizeof(*mcf_edma) + sizeof(*mcf_chan) * chans;
+ mcf_edma = devm_kzalloc(&pdev->dev, len, GFP_KERNEL);
+ if (!mcf_edma)
+ return -ENOMEM;
+
+ mcf_edma->n_chans = chans;
+
+ /* Set up version for ColdFire edma */
+ mcf_edma->version = v2;
+ mcf_edma->big_endian = 1;
+
+ if (!mcf_edma->n_chans) {
+ dev_info(&pdev->dev, "setting default channel number to 64");
+ mcf_edma->n_chans = 64;
+ }
+
+ mutex_init(&mcf_edma->fsl_edma_mutex);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+
+ mcf_edma->membase = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(mcf_edma->membase))
+ return PTR_ERR(mcf_edma->membase);
+
+ fsl_edma_setup_regs(mcf_edma);
+ regs = &mcf_edma->regs;
+
+ INIT_LIST_HEAD(&mcf_edma->dma_dev.channels);
+ for (i = 0; i < mcf_edma->n_chans; i++) {
+ struct fsl_edma_chan *mcf_chan = &mcf_edma->chans[i];
+
+ mcf_chan->edma = mcf_edma;
+ mcf_chan->slave_id = i;
+ mcf_chan->idle = true;
+ mcf_chan->vchan.desc_free = fsl_edma_free_desc;
+ vchan_init(&mcf_chan->vchan, &mcf_edma->dma_dev);
+ iowrite32(0x0, ®s->tcd[i].csr);
+ }
+
+ iowrite32(~0, regs->inth);
+ iowrite32(~0, regs->intl);
+
+ ret = mcf_edma_irq_init(pdev, mcf_edma);
+ if (ret)
+ return ret;
+
+ dma_cap_set(DMA_PRIVATE, mcf_edma->dma_dev.cap_mask);
+ dma_cap_set(DMA_SLAVE, mcf_edma->dma_dev.cap_mask);
+ dma_cap_set(DMA_CYCLIC, mcf_edma->dma_dev.cap_mask);
+
+ mcf_edma->dma_dev.dev = &pdev->dev;
+ mcf_edma->dma_dev.device_alloc_chan_resources =
+ fsl_edma_alloc_chan_resources;
+ mcf_edma->dma_dev.device_free_chan_resources =
+ fsl_edma_free_chan_resources;
+ mcf_edma->dma_dev.device_config = fsl_edma_slave_config;
+ mcf_edma->dma_dev.device_prep_dma_cyclic =
+ fsl_edma_prep_dma_cyclic;
+ mcf_edma->dma_dev.device_prep_slave_sg = fsl_edma_prep_slave_sg;
+ mcf_edma->dma_dev.device_tx_status = fsl_edma_tx_status;
+ mcf_edma->dma_dev.device_pause = fsl_edma_pause;
+ mcf_edma->dma_dev.device_resume = fsl_edma_resume;
+ mcf_edma->dma_dev.device_terminate_all = fsl_edma_terminate_all;
+ mcf_edma->dma_dev.device_issue_pending = fsl_edma_issue_pending;
+
+ mcf_edma->dma_dev.src_addr_widths = FSL_EDMA_BUSWIDTHS;
+ mcf_edma->dma_dev.dst_addr_widths = FSL_EDMA_BUSWIDTHS;
+ mcf_edma->dma_dev.directions =
+ BIT(DMA_DEV_TO_MEM) | BIT(DMA_MEM_TO_DEV);
+
+ mcf_edma->dma_dev.filter.fn = mcf_edma_filter_fn;
+ mcf_edma->dma_dev.filter.map = pdata->slave_map;
+ mcf_edma->dma_dev.filter.mapcnt = pdata->slavecnt;
+
+ platform_set_drvdata(pdev, mcf_edma);
+
+ ret = dma_async_device_register(&mcf_edma->dma_dev);
+ if (ret) {
+ dev_err(&pdev->dev,
+ "Can't register Freescale eDMA engine. (%d)\n", ret);
+ return ret;
+ }
+
+ /* Enable round robin arbitration */
+ iowrite32(EDMA_CR_ERGA | EDMA_CR_ERCA, regs->cr);
+
+ return 0;
+}
+
+static int mcf_edma_remove(struct platform_device *pdev)
+{
+ struct fsl_edma_engine *mcf_edma = platform_get_drvdata(pdev);
+
+ fsl_edma_cleanup_vchan(&mcf_edma->dma_dev);
+ dma_async_device_unregister(&mcf_edma->dma_dev);
+
+ return 0;
+}
+
+static struct platform_driver mcf_edma_driver = {
+ .driver = {
+ .name = "mcf-edma",
+ },
+ .probe = mcf_edma_probe,
+ .remove = mcf_edma_remove,
+};
+
+bool mcf_edma_filter_fn(struct dma_chan *chan, void *param)
+{
+ if (chan->device->dev->driver == &mcf_edma_driver.driver) {
+ struct fsl_edma_chan *mcf_chan = to_fsl_edma_chan(chan);
+
+ return (mcf_chan->slave_id == (int)param);
+ }
+
+ return false;
+}
+EXPORT_SYMBOL(mcf_edma_filter_fn);
+
+static int __init mcf_edma_init(void)
+{
+ return platform_driver_register(&mcf_edma_driver);
+}
+subsys_initcall(mcf_edma_init);
+
+static void __exit mcf_edma_exit(void)
+{
+ platform_driver_unregister(&mcf_edma_driver);
+}
+module_exit(mcf_edma_exit);
+
+MODULE_ALIAS("platform:mcf-edma");
+MODULE_DESCRIPTION("Freescale eDMA engine driver, ColdFire family");
+MODULE_LICENSE("GPL v2");
diff --git a/include/linux/platform_data/dma-mcf-edma.h b/include/linux/platform_data/dma-mcf-edma.h
new file mode 100644
index 000000000000..4f45d0d40aa7
--- /dev/null
+++ b/include/linux/platform_data/dma-mcf-edma.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Freescale eDMA platform data, ColdFire SoC's family.
+ *
+ * Copyright (c) 2017 Angelo Dureghello <angelo@xxxxxxxx>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef __MACH_MCF_EDMA_H__
+#define __MACH_MCF_EDMA_H__
+
+struct dma_slave_map;
+
+bool mcf_edma_filter_fn(struct dma_chan *chan, void *param);
+
+#define MCF_EDMA_FILTER_PARAM(ch) ((void *)ch)
+
+/**
+ * struct mcf_edma_platform_data - platform specific data for eDMA engine
+ *
+ * @ver The eDMA module version.
+ * @dma_channels The number of eDMA channels.
+ */
+struct mcf_edma_platform_data {
+ int dma_channels;
+ const struct dma_slave_map *slave_map;
+ int slavecnt;
+};
+
+#endif /* __MACH_MCF_EDMA_H__ */
^ permalink raw reply related
* [v4,1/2] dmaengine: fsl-edma: extract common fsl-edma code (no changes in behavior intended)
From: Angelo Dureghello @ 2018-06-19 19:41 UTC (permalink / raw)
To: vinod.koul; +Cc: dmaengine, linux-m68k, Angelo Dureghello
This patch adds a new fsl-edma-common module to allow new
mcf-edma module code to use most of the fsl-edma code.
Due to some differences between ColdFire edma (64 channels) and
fsl-edma (32 channels), as register set offsets and some other
points as the different interrupt organization and other minor
things, a common module can collect most of the code for both
32 and 64 channel edma module versions.
Signed-off-by: Angelo Dureghello <angelo@sysam.it>
---
Changes for v2:
- patch splitted into 4
- add fsl-edma-common module
Changes for v3:
none
Changes for v4:
- patch simplified from 4/4 into 2/2
- collecting all the fsl-edma-related changes
---
drivers/dma/Makefile | 2 +-
drivers/dma/fsl-edma-common.c | 662 ++++++++++++++++++++++++++++++
drivers/dma/fsl-edma-common.h | 175 ++++++++
drivers/dma/fsl-edma.c | 739 ++--------------------------------
4 files changed, 868 insertions(+), 710 deletions(-)
create mode 100644 drivers/dma/fsl-edma-common.c
create mode 100644 drivers/dma/fsl-edma-common.h
diff --git a/drivers/dma/Makefile b/drivers/dma/Makefile
index 203a99d68315..66022f59fca4 100644
--- a/drivers/dma/Makefile
+++ b/drivers/dma/Makefile
@@ -31,7 +31,7 @@ obj-$(CONFIG_DW_AXI_DMAC) += dw-axi-dmac/
obj-$(CONFIG_DW_DMAC_CORE) += dw/
obj-$(CONFIG_EP93XX_DMA) += ep93xx_dma.o
obj-$(CONFIG_FSL_DMA) += fsldma.o
-obj-$(CONFIG_FSL_EDMA) += fsl-edma.o
+obj-$(CONFIG_FSL_EDMA) += fsl-edma.o fsl-edma-common.o
obj-$(CONFIG_FSL_RAID) += fsl_raid.o
obj-$(CONFIG_HSU_DMA) += hsu/
obj-$(CONFIG_IMG_MDC_DMA) += img-mdc-dma.o
diff --git a/drivers/dma/fsl-edma-common.c b/drivers/dma/fsl-edma-common.c
new file mode 100644
index 000000000000..c205cb07f133
--- /dev/null
+++ b/drivers/dma/fsl-edma-common.c
@@ -0,0 +1,662 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2013-2014 Freescale Semiconductor, Inc
+// Copyright (c) 2017 Sysam, Angelo Dureghello <angelo@sysam.it>
+/*
+ * drivers/dma/fsl-edma-common.c
+ *
+ * Common code for Freescale the edma 32 or 64 channel version.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation version 2.
+ *
+ * This program is distributed "as is" WITHOUT ANY WARRANTY of any
+ * kind, whether express or implied; without even the implied warranty
+ * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/dmapool.h>
+#include <linux/slab.h>
+
+#include "fsl-edma-common.h"
+
+#define EDMA_CR 0x00
+#define EDMA_ES 0x04
+#define EDMA_ERQ 0x0C
+#define EDMA_EEI 0x14
+#define EDMA_CEEI 0x18
+#define EDMA_SEEI 0x19
+#define EDMA_CERQ 0x1A
+#define EDMA_SERQ 0x1B
+#define EDMA_CINT 0x1F
+#define EDMA_CERR 0x1E
+#define EDMA_SSRT 0x1D
+#define EDMA_CDNE 0x1C
+#define EDMA_INTR 0x24
+#define EDMA_ERR 0x2C
+
+#define EDMA64_ERQH 0x08
+#define EDMA64_EEIH 0x10
+#define EDMA64_SERQ 0x18
+#define EDMA64_CERQ 0x19
+#define EDMA64_SEEI 0x1a
+#define EDMA64_CEEI 0x1b
+#define EDMA64_CINT 0x1c
+#define EDMA64_CERR 0x1d
+#define EDMA64_SSRT 0x1e
+#define EDMA64_CDNE 0x1f
+#define EDMA64_INTH 0x20
+#define EDMA64_INTL 0x24
+#define EDMA64_ERRH 0x28
+#define EDMA64_ERRL 0x2c
+
+#define EDMA_TCD 0x1000
+#define EDMA_TCD_SIZE 32
+#define EDMA_TCD_MEM_ALIGN EDMA_TCD_SIZE
+
+#define EDMA_TCD_ATTR_SSIZE_8BIT 0
+#define EDMA_TCD_ATTR_SSIZE_16BIT BIT(0)
+#define EDMA_TCD_ATTR_SSIZE_32BIT BIT(1)
+#define EDMA_TCD_ATTR_SSIZE_64BIT (BIT(1) | BIT(0))
+#define EDMA_TCD_ATTR_SSIZE_16BYTE BIT(2)
+#define EDMA_TCD_ATTR_SSIZE_32BYTE (BIT(2) | BIT(0))
+#define EDMA_TCD_ATTR_DSIZE_8BIT (EDMA_TCD_ATTR_SSIZE_8BIT << 8)
+#define EDMA_TCD_ATTR_DSIZE_16BIT (EDMA_TCD_ATTR_SSIZE_16BIT << 8)
+#define EDMA_TCD_ATTR_DSIZE_32BIT (EDMA_TCD_ATTR_SSIZE_32BIT << 8)
+#define EDMA_TCD_ATTR_DSIZE_64BIT (EDMA_TCD_ATTR_SSIZE_64BIT << 8)
+#define EDMA_TCD_ATTR_DSIZE_16BYTE (EDMA_TCD_ATTR_SSIZE_16BYTE << 8)
+#define EDMA_TCD_ATTR_DSIZE_32BYTE (EDMA_TCD_ATTR_SSIZE_32BYTE << 8)
+
+#define EDMA_TCD_CITER_CITER(x) ((x) & GENMASK(14, 0))
+#define EDMA_TCD_BITER_BITER(x) ((x) & GENMASK(14, 0))
+
+#define EDMA_TCD_CSR_START BIT(0)
+#define EDMA_TCD_CSR_INT_MAJOR BIT(1)
+#define EDMA_TCD_CSR_INT_HALF BIT(2)
+#define EDMA_TCD_CSR_D_REQ BIT(3)
+#define EDMA_TCD_CSR_E_SG BIT(4)
+#define EDMA_TCD_CSR_E_LINK BIT(5)
+#define EDMA_TCD_CSR_ACTIVE BIT(6)
+#define EDMA_TCD_CSR_DONE BIT(7)
+
+struct fsl_edma_chan *to_fsl_edma_chan(struct dma_chan *chan)
+{
+ return container_of(chan, struct fsl_edma_chan, vchan.chan);
+}
+
+struct fsl_edma_desc *to_fsl_edma_desc(struct virt_dma_desc *vd)
+{
+ return container_of(vd, struct fsl_edma_desc, vdesc);
+}
+
+/*
+ * R/W functions for big- or little-endian registers:
+ * The eDMA controller's endian is independent of the CPU core's endian.
+ * For the big-endian IP module, the offset for 8-bit or 16-bit registers
+ * should also be swapped opposite to that in little-endian IP.
+ */
+u32 edma_readl(struct fsl_edma_engine *edma,
+ void __iomem *addr)
+{
+ if (edma->big_endian)
+ return ioread32be(addr);
+ else
+ return ioread32(addr);
+}
+
+void edma_writeb(struct fsl_edma_engine *edma, u8 val,
+ void __iomem *addr)
+{
+ /* swap the reg offset for these in big-endian mode */
+ if (edma->big_endian)
+ iowrite8(val, (void __iomem *)((unsigned long)addr ^ 0x3));
+ else
+ iowrite8(val, addr);
+}
+
+void edma_writew(struct fsl_edma_engine *edma, u16 val,
+ void __iomem *addr)
+{
+ /* swap the reg offset for these in big-endian mode */
+ if (edma->big_endian)
+ iowrite16be(val, (void __iomem *)((unsigned long)addr ^ 0x2));
+ else
+ iowrite16(val, addr);
+}
+
+void edma_writel(struct fsl_edma_engine *edma, u32 val,
+ void __iomem *addr)
+{
+ if (edma->big_endian)
+ iowrite32be(val, addr);
+ else
+ iowrite32(val, addr);
+}
+
+static void fsl_edma_fill_tcd(struct fsl_edma_hw_tcd *tcd,
+ u32 src, u32 dst, u16 attr, u16 soff,
+ u32 nbytes, u32 slast, u16 citer, u16 biter,
+ u16 doff, u32 dlast_sga, bool major_int,
+ bool disable_req, bool enable_sg)
+{
+ u16 csr = 0;
+
+ /*
+ * eDMA hardware SGs require the TCDs to be stored in little
+ * endian format irrespective of the register endian model.
+ * So we put the value in little endian in memory, waiting
+ * for fsl_edma_set_tcd_regs doing the swap.
+ */
+ tcd->saddr = cpu_to_le32(src);
+ tcd->daddr = cpu_to_le32(dst);
+ tcd->attr = cpu_to_le16(attr);
+ tcd->soff = cpu_to_le16(soff);
+ tcd->nbytes = cpu_to_le32(nbytes);
+ tcd->slast = cpu_to_le32(slast);
+ tcd->citer = cpu_to_le16(EDMA_TCD_CITER_CITER(citer));
+ tcd->doff = cpu_to_le16(doff);
+ tcd->dlast_sga = cpu_to_le32(dlast_sga);
+ tcd->biter = cpu_to_le16(EDMA_TCD_BITER_BITER(biter));
+
+ if (major_int)
+ csr |= EDMA_TCD_CSR_INT_MAJOR;
+ if (disable_req)
+ csr |= EDMA_TCD_CSR_D_REQ;
+ if (enable_sg)
+ csr |= EDMA_TCD_CSR_E_SG;
+
+ tcd->csr = cpu_to_le16(csr);
+}
+
+void fsl_edma_enable_request(struct fsl_edma_chan *fsl_chan)
+{
+ struct edma_regs *regs = &fsl_chan->edma->regs;
+ u32 ch = fsl_chan->vchan.chan.chan_id;
+
+ if (fsl_chan->edma->version == v1) {
+ edma_writeb(fsl_chan->edma, EDMA_SEEI_SEEI(ch), regs->seei);
+ edma_writeb(fsl_chan->edma, ch, regs->serq);
+ } else {
+ /* ColdFire is big endian, and accesses natively
+ * big endian I/O peripherals
+ */
+ iowrite8(EDMA_SEEI_SEEI(ch), regs->seei);
+ iowrite8(ch, regs->serq);
+ }
+}
+
+void fsl_edma_disable_request(struct fsl_edma_chan *fsl_chan)
+{
+ struct edma_regs *regs = &fsl_chan->edma->regs;
+ u32 ch = fsl_chan->vchan.chan.chan_id;
+
+ if (fsl_chan->edma->version == v1) {
+ edma_writeb(fsl_chan->edma, ch, regs->cerq);
+ edma_writeb(fsl_chan->edma, EDMA_CEEI_CEEI(ch), regs->ceei);
+ } else {
+ /* ColdFire is big endian, and accesses natively
+ * big endian I/O peripherals
+ */
+ iowrite8(ch, regs->cerq);
+ iowrite8(EDMA_CEEI_CEEI(ch), regs->ceei);
+ }
+}
+
+void fsl_edma_cleanup_vchan(struct dma_device *dmadev)
+{
+ struct fsl_edma_chan *chan, *_chan;
+
+ list_for_each_entry_safe(chan, _chan,
+ &dmadev->channels, vchan.chan.device_node) {
+ list_del(&chan->vchan.chan.device_node);
+ tasklet_kill(&chan->vchan.task);
+ }
+}
+
+static unsigned int fsl_edma_get_tcd_attr(enum dma_slave_buswidth addr_width)
+{
+ switch (addr_width) {
+ case 1:
+ return EDMA_TCD_ATTR_SSIZE_8BIT | EDMA_TCD_ATTR_DSIZE_8BIT;
+ case 2:
+ return EDMA_TCD_ATTR_SSIZE_16BIT | EDMA_TCD_ATTR_DSIZE_16BIT;
+ case 4:
+ return EDMA_TCD_ATTR_SSIZE_32BIT | EDMA_TCD_ATTR_DSIZE_32BIT;
+ case 8:
+ return EDMA_TCD_ATTR_SSIZE_64BIT | EDMA_TCD_ATTR_DSIZE_64BIT;
+ default:
+ return EDMA_TCD_ATTR_SSIZE_32BIT | EDMA_TCD_ATTR_DSIZE_32BIT;
+ }
+}
+
+static struct fsl_edma_desc *fsl_edma_alloc_desc(struct fsl_edma_chan *fsl_chan,
+ int sg_len)
+{
+ struct fsl_edma_desc *fsl_desc;
+ int i;
+
+ fsl_desc = kzalloc(sizeof(*fsl_desc) +
+ sizeof(struct fsl_edma_sw_tcd) * sg_len, GFP_NOWAIT);
+ if (!fsl_desc)
+ return NULL;
+
+ fsl_desc->echan = fsl_chan;
+ fsl_desc->n_tcds = sg_len;
+ for (i = 0; i < sg_len; i++) {
+ fsl_desc->tcd[i].vtcd = dma_pool_alloc(fsl_chan->tcd_pool,
+ GFP_NOWAIT, &fsl_desc->tcd[i].ptcd);
+ if (!fsl_desc->tcd[i].vtcd)
+ goto err;
+ }
+ return fsl_desc;
+
+err:
+ while (--i >= 0)
+ dma_pool_free(fsl_chan->tcd_pool, fsl_desc->tcd[i].vtcd,
+ fsl_desc->tcd[i].ptcd);
+ kfree(fsl_desc);
+ return NULL;
+}
+
+static void fsl_edma_set_tcd_regs(struct fsl_edma_chan *fsl_chan,
+ struct fsl_edma_hw_tcd *tcd)
+{
+ struct fsl_edma_engine *edma = fsl_chan->edma;
+ struct fsl_edma_hw_tcd *mtcd = edma->regs.tcd
+ + fsl_chan->vchan.chan.chan_id;
+
+ /*
+ * TCD parameters are stored in struct fsl_edma_hw_tcd in little
+ * endian format. However, we need to load the TCD registers in
+ * big- or little-endian obeying the eDMA engine model endian.
+ */
+ edma_writel(edma, le32_to_cpu(tcd->saddr), &mtcd->saddr);
+ edma_writel(edma, le32_to_cpu(tcd->daddr), &mtcd->daddr);
+ edma_writew(edma, le16_to_cpu(tcd->attr), &mtcd->attr);
+ edma_writew(edma, le16_to_cpu(tcd->soff), &mtcd->soff);
+ edma_writel(edma, le32_to_cpu(tcd->nbytes), &mtcd->nbytes);
+ edma_writel(edma, le32_to_cpu(tcd->slast), &mtcd->slast);
+ edma_writew(edma, le16_to_cpu(tcd->citer), &mtcd->citer);
+ edma_writew(edma, le16_to_cpu(tcd->biter), &mtcd->biter);
+ edma_writew(edma, le16_to_cpu(tcd->doff), &mtcd->doff);
+ edma_writel(edma, le32_to_cpu(tcd->dlast_sga), &mtcd->dlast_sga);
+ edma_writew(edma, le16_to_cpu(tcd->csr), &mtcd->csr);
+}
+
+static size_t fsl_edma_desc_residue(struct fsl_edma_chan *fsl_chan,
+ struct virt_dma_desc *vdesc, bool in_progress)
+{
+ struct fsl_edma_desc *edesc = fsl_chan->edesc;
+ struct edma_regs *regs = &fsl_chan->edma->regs;
+ u32 ch = fsl_chan->vchan.chan.chan_id;
+ enum dma_transfer_direction dir = fsl_chan->dir;
+ dma_addr_t cur_addr, dma_addr;
+ size_t len, size;
+ int i;
+
+ /* calculate the total size in this desc */
+ for (len = i = 0; i < fsl_chan->edesc->n_tcds; i++)
+ len += le32_to_cpu(edesc->tcd[i].vtcd->nbytes)
+ * le16_to_cpu(edesc->tcd[i].vtcd->biter);
+
+ if (!in_progress)
+ return len;
+
+ if (dir == DMA_MEM_TO_DEV)
+ cur_addr = edma_readl(fsl_chan->edma, ®s->tcd[ch].saddr);
+ else
+ cur_addr = edma_readl(fsl_chan->edma, ®s->tcd[ch].daddr);
+
+ /* figure out the finished and calculate the residue */
+ for (i = 0; i < fsl_chan->edesc->n_tcds; i++) {
+ size = le32_to_cpu(edesc->tcd[i].vtcd->nbytes)
+ * le16_to_cpu(edesc->tcd[i].vtcd->biter);
+ if (dir == DMA_MEM_TO_DEV)
+ dma_addr = le32_to_cpu(edesc->tcd[i].vtcd->saddr);
+ else
+ dma_addr = le32_to_cpu(edesc->tcd[i].vtcd->daddr);
+
+ len -= size;
+ if (cur_addr >= dma_addr && cur_addr < dma_addr + size) {
+ len += dma_addr + size - cur_addr;
+ break;
+ }
+ }
+
+ return len;
+}
+
+void fsl_edma_free_desc(struct virt_dma_desc *vdesc)
+{
+ struct fsl_edma_desc *mcf_desc;
+ int i;
+
+ mcf_desc = to_fsl_edma_desc(vdesc);
+
+ for (i = 0; i < mcf_desc->n_tcds; i++)
+ dma_pool_free(mcf_desc->echan->tcd_pool, mcf_desc->tcd[i].vtcd,
+ mcf_desc->tcd[i].ptcd);
+ kfree(mcf_desc);
+}
+
+int fsl_edma_alloc_chan_resources(struct dma_chan *chan)
+{
+ struct fsl_edma_chan *mcf_chan = to_fsl_edma_chan(chan);
+
+ mcf_chan->tcd_pool = dma_pool_create("tcd_pool", chan->device->dev,
+ sizeof(struct fsl_edma_hw_tcd), EDMA_TCD_MEM_ALIGN, 0);
+
+ return 0;
+}
+
+void fsl_edma_free_chan_resources(struct dma_chan *chan)
+{
+ struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan);
+ unsigned long flags;
+ LIST_HEAD(head);
+
+ spin_lock_irqsave(&fsl_chan->vchan.lock, flags);
+ fsl_edma_disable_request(fsl_chan);
+ fsl_chan->edesc = NULL;
+ vchan_get_all_descriptors(&fsl_chan->vchan, &head);
+ spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags);
+
+ vchan_dma_desc_free_list(&fsl_chan->vchan, &head);
+ dma_pool_destroy(fsl_chan->tcd_pool);
+ fsl_chan->tcd_pool = NULL;
+}
+
+int fsl_edma_slave_config(struct dma_chan *chan,
+ struct dma_slave_config *config)
+{
+ struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan);
+
+ memcpy(&fsl_chan->cfg, config, sizeof(*config));
+
+ return 0;
+}
+
+struct dma_async_tx_descriptor *fsl_edma_prep_dma_cyclic(
+ struct dma_chan *chan, dma_addr_t dma_addr, size_t buf_len,
+ size_t period_len, enum dma_transfer_direction direction,
+ unsigned long flags)
+{
+ struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan);
+ struct fsl_edma_desc *fsl_desc;
+ dma_addr_t dma_buf_next;
+ int sg_len, i;
+ u32 src_addr, dst_addr, last_sg, nbytes, attr;
+ u16 soff, doff, iter;
+
+ if (!is_slave_direction(direction))
+ return NULL;
+
+ sg_len = buf_len / period_len;
+ fsl_desc = fsl_edma_alloc_desc(fsl_chan, sg_len);
+ if (!fsl_desc)
+ return NULL;
+
+ fsl_desc->iscyclic = true;
+ fsl_chan->dir = direction;
+
+ dma_buf_next = dma_addr;
+
+ for (i = 0; i < sg_len; i++) {
+ if (dma_buf_next >= dma_addr + buf_len)
+ dma_buf_next = dma_addr;
+
+ /* get next sg's physical address */
+ last_sg = fsl_desc->tcd[(i + 1) % sg_len].ptcd;
+
+ if (direction == DMA_MEM_TO_DEV) {
+ nbytes = fsl_chan->cfg.dst_addr_width *
+ fsl_chan->cfg.dst_maxburst;
+ src_addr = dma_buf_next;
+ dst_addr = fsl_chan->cfg.dst_addr;
+ soff = fsl_chan->cfg.dst_addr_width;
+ doff = 0;
+ attr = fsl_edma_get_tcd_attr(
+ fsl_chan->cfg.dst_addr_width);
+ } else {
+ nbytes = fsl_chan->cfg.src_addr_width *
+ fsl_chan->cfg.src_maxburst;
+ src_addr = fsl_chan->cfg.src_addr;
+ dst_addr = dma_buf_next;
+ soff = 0;
+ doff = fsl_chan->cfg.src_addr_width;
+ attr = fsl_edma_get_tcd_attr(
+ fsl_chan->cfg.src_addr_width);
+ }
+
+ iter = period_len / nbytes;
+
+ fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd,
+ src_addr, dst_addr,
+ attr, soff, nbytes, 0, iter,
+ iter, doff, last_sg, true, false, true);
+ dma_buf_next += period_len;
+ }
+
+ return vchan_tx_prep(&fsl_chan->vchan, &fsl_desc->vdesc, flags);
+}
+
+struct dma_async_tx_descriptor *fsl_edma_prep_slave_sg(
+ struct dma_chan *chan, struct scatterlist *sgl,
+ unsigned int sg_len, enum dma_transfer_direction direction,
+ unsigned long flags, void *context)
+{
+ struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan);
+ struct fsl_edma_desc *fsl_desc;
+ struct scatterlist *sg;
+ u32 src_addr, dst_addr, last_sg, nbytes, attr;
+ u16 soff, doff, iter;
+ int i;
+
+ if (!is_slave_direction(direction))
+ return NULL;
+
+ fsl_desc = fsl_edma_alloc_desc(fsl_chan, sg_len);
+ if (!fsl_desc)
+ return NULL;
+
+ fsl_desc->iscyclic = false;
+ fsl_chan->dir = direction;
+
+ for_each_sg(sgl, sg, sg_len, i) {
+ /* get next sg's physical address */
+ last_sg = fsl_desc->tcd[(i + 1) % sg_len].ptcd;
+
+ if (direction == DMA_MEM_TO_DEV) {
+ nbytes = fsl_chan->cfg.dst_addr_width *
+ fsl_chan->cfg.dst_maxburst;
+ src_addr = sg_dma_address(sg);
+ dst_addr = fsl_chan->cfg.dst_addr;
+ soff = fsl_chan->cfg.dst_addr_width;
+ doff = 0;
+ attr = fsl_edma_get_tcd_attr(
+ fsl_chan->cfg.dst_addr_width);
+
+ } else {
+ nbytes = fsl_chan->cfg.src_addr_width *
+ fsl_chan->cfg.src_maxburst;
+ src_addr = fsl_chan->cfg.src_addr;
+ dst_addr = sg_dma_address(sg);
+ soff = 0;
+ doff = fsl_chan->cfg.src_addr_width;
+ attr = fsl_edma_get_tcd_attr(
+ fsl_chan->cfg.src_addr_width);
+ }
+
+ iter = sg_dma_len(sg) / nbytes;
+
+ if (i < sg_len - 1) {
+ last_sg = fsl_desc->tcd[(i + 1)].ptcd;
+ fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd,
+ src_addr,
+ dst_addr, attr, soff,
+ nbytes, 0, iter, iter, doff, last_sg,
+ false, false, true);
+ } else {
+ last_sg = 0;
+ fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd,
+ src_addr,
+ dst_addr, attr, soff,
+ nbytes, 0, iter, iter, doff, last_sg,
+ true, true, false);
+ }
+ }
+
+ return vchan_tx_prep(&fsl_chan->vchan, &fsl_desc->vdesc, flags);
+}
+
+enum dma_status fsl_edma_tx_status(struct dma_chan *chan,
+ dma_cookie_t cookie, struct dma_tx_state *txstate)
+{
+ struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan);
+ struct virt_dma_desc *vdesc;
+ enum dma_status status;
+ unsigned long flags;
+
+ status = dma_cookie_status(chan, cookie, txstate);
+ if (status == DMA_COMPLETE)
+ return status;
+
+ if (!txstate)
+ return fsl_chan->status;
+
+ spin_lock_irqsave(&fsl_chan->vchan.lock, flags);
+ vdesc = vchan_find_desc(&fsl_chan->vchan, cookie);
+ if (fsl_chan->edesc && cookie == fsl_chan->edesc->vdesc.tx.cookie)
+ txstate->residue =
+ fsl_edma_desc_residue(fsl_chan, vdesc, true);
+ else if (vdesc)
+ txstate->residue =
+ fsl_edma_desc_residue(fsl_chan, vdesc, false);
+ else
+ txstate->residue = 0;
+
+ spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags);
+
+ return fsl_chan->status;
+}
+
+int fsl_edma_pause(struct dma_chan *chan)
+{
+ struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan);
+ unsigned long flags;
+
+ spin_lock_irqsave(&fsl_chan->vchan.lock, flags);
+ if (fsl_chan->edesc) {
+ fsl_edma_disable_request(fsl_chan);
+ fsl_chan->status = DMA_PAUSED;
+ fsl_chan->idle = true;
+ }
+ spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags);
+ return 0;
+}
+
+int fsl_edma_resume(struct dma_chan *chan)
+{
+ struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan);
+ unsigned long flags;
+
+ spin_lock_irqsave(&fsl_chan->vchan.lock, flags);
+ if (fsl_chan->edesc) {
+ fsl_edma_enable_request(fsl_chan);
+ fsl_chan->status = DMA_IN_PROGRESS;
+ fsl_chan->idle = false;
+ }
+ spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags);
+ return 0;
+}
+
+int fsl_edma_terminate_all(struct dma_chan *chan)
+{
+ struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan);
+ unsigned long flags;
+ LIST_HEAD(head);
+
+ spin_lock_irqsave(&fsl_chan->vchan.lock, flags);
+ fsl_edma_disable_request(fsl_chan);
+ fsl_chan->edesc = NULL;
+ fsl_chan->idle = true;
+ vchan_get_all_descriptors(&fsl_chan->vchan, &head);
+ spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags);
+ vchan_dma_desc_free_list(&fsl_chan->vchan, &head);
+ return 0;
+}
+
+void fsl_edma_issue_pending(struct dma_chan *chan)
+{
+ struct fsl_edma_chan *mcf_chan = to_fsl_edma_chan(chan);
+ unsigned long flags;
+
+ spin_lock_irqsave(&mcf_chan->vchan.lock, flags);
+
+ if (vchan_issue_pending(&mcf_chan->vchan) && !mcf_chan->edesc)
+ fsl_edma_xfer_desc(mcf_chan);
+
+ spin_unlock_irqrestore(&mcf_chan->vchan.lock, flags);
+}
+
+void fsl_edma_xfer_desc(struct fsl_edma_chan *fsl_chan)
+{
+ struct virt_dma_desc *vdesc;
+
+ vdesc = vchan_next_desc(&fsl_chan->vchan);
+ if (!vdesc)
+ return;
+ fsl_chan->edesc = to_fsl_edma_desc(vdesc);
+ fsl_edma_set_tcd_regs(fsl_chan, fsl_chan->edesc->tcd[0].vtcd);
+ fsl_edma_enable_request(fsl_chan);
+ fsl_chan->status = DMA_IN_PROGRESS;
+ fsl_chan->idle = false;
+}
+
+/*
+ * On the 32 channels Vybrid/mpc577x edma version (here called "v1"),
+ * register offsets are different compared to ColdFire mcf5441x 64 channels
+ * edma (here called "v2").
+ *
+ * This function sets up register offsets as per proper declared version
+ * so must be called in xxx_edma_probe() just after setting the
+ * edma "version" and "membase" appropriately.
+ */
+void fsl_edma_setup_regs(struct fsl_edma_engine *edma)
+{
+ edma->regs.cr = edma->membase + EDMA_CR;
+ edma->regs.es = edma->membase + EDMA_ES;
+ edma->regs.erql = edma->membase + EDMA_ERQ;
+ edma->regs.eeil = edma->membase + EDMA_EEI;
+
+ edma->regs.serq = edma->membase + ((edma->version == v1) ?
+ EDMA_SERQ : EDMA64_SERQ);
+ edma->regs.cerq = edma->membase + ((edma->version == v1) ?
+ EDMA_CERQ : EDMA64_CERQ);
+ edma->regs.seei = edma->membase + ((edma->version == v1) ?
+ EDMA_SEEI : EDMA64_SEEI);
+ edma->regs.ceei = edma->membase + ((edma->version == v1) ?
+ EDMA_CEEI : EDMA64_CEEI);
+ edma->regs.cint = edma->membase + ((edma->version == v1) ?
+ EDMA_CINT : EDMA64_CINT);
+ edma->regs.cerr = edma->membase + ((edma->version == v1) ?
+ EDMA_CERR : EDMA64_CERR);
+ edma->regs.ssrt = edma->membase + ((edma->version == v1) ?
+ EDMA_SSRT : EDMA64_SSRT);
+ edma->regs.cdne = edma->membase + ((edma->version == v1) ?
+ EDMA_CDNE : EDMA64_CDNE);
+ edma->regs.intl = edma->membase + ((edma->version == v1) ?
+ EDMA_INTR : EDMA64_INTL);
+ edma->regs.errl = edma->membase + ((edma->version == v1) ?
+ EDMA_ERR : EDMA64_ERRL);
+
+ if (edma->version == v2) {
+ edma->regs.erqh = edma->membase + EDMA64_ERQH;
+ edma->regs.eeih = edma->membase + EDMA64_EEIH;
+ edma->regs.errh = edma->membase + EDMA64_ERRH;
+ edma->regs.inth = edma->membase + EDMA64_INTH;
+ }
+
+ edma->regs.tcd = edma->membase + EDMA_TCD;
+}
diff --git a/drivers/dma/fsl-edma-common.h b/drivers/dma/fsl-edma-common.h
new file mode 100644
index 000000000000..e56f9e59e50b
--- /dev/null
+++ b/drivers/dma/fsl-edma-common.h
@@ -0,0 +1,175 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * drivers/dma/fsl-edma-common.c
+ *
+ * Copyright 2013-2014 Freescale Semiconductor, Inc.
+ * Copyright 2018 Angelo Dureghello <angelo@sysam.it>
+ *
+ * Common code for Freescale the edma 32 or 64 channel version.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation version 2.
+ *
+ * This program is distributed "as is" WITHOUT ANY WARRANTY of any
+ * kind, whether express or implied; without even the implied warranty
+ * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+#ifndef _FSL_EDMA_COMMON_H_
+#define _FSL_EDMA_COMMON_H_
+
+#include <linux/types.h>
+
+#include "virt-dma.h"
+
+#define DMAMUX_NR 2
+
+#define EDMA_CR_EDBG BIT(1)
+#define EDMA_CR_ERCA BIT(2)
+#define EDMA_CR_ERGA BIT(3)
+#define EDMA_CR_HOE BIT(4)
+#define EDMA_CR_HALT BIT(5)
+#define EDMA_CR_CLM BIT(6)
+#define EDMA_CR_EMLM BIT(7)
+#define EDMA_CR_ECX BIT(16)
+#define EDMA_CR_CX BIT(17)
+
+#define EDMA_SEEI_SEEI(x) ((x) & GENMASK(6, 0))
+#define EDMA_CEEI_CEEI(x) ((x) & GENMASK(6, 0))
+#define EDMA_CINT_CINT(x) ((x) & GENMASK(6, 0))
+#define EDMA_CERR_CERR(x) ((x) & GENMASK(6, 0))
+
+#define FSL_EDMA_BUSWIDTHS (BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) | \
+ BIT(DMA_SLAVE_BUSWIDTH_2_BYTES) | \
+ BIT(DMA_SLAVE_BUSWIDTH_4_BYTES) | \
+ BIT(DMA_SLAVE_BUSWIDTH_8_BYTES))
+
+enum fsl_edma_pm_state {
+ RUNNING = 0,
+ SUSPENDED,
+};
+
+/*
+ * This are tcd regs, equal for both v32 and v64
+ */
+struct fsl_edma_hw_tcd {
+ __le32 saddr;
+ __le16 soff;
+ __le16 attr;
+ __le32 nbytes;
+ __le32 slast;
+ __le32 daddr;
+ __le16 doff;
+ __le16 citer;
+ __le32 dlast_sga;
+ __le16 csr;
+ __le16 biter;
+};
+
+/*
+ * This are iomem pointers, for both v32 and v64.
+ */
+struct edma_regs {
+ void __iomem *cr;
+ void __iomem *es;
+ void __iomem *erq;
+ void __iomem *erqh;
+ void __iomem *erql; /* aka erq on v32 */
+ void __iomem *eeih;
+ void __iomem *eeil; /* aka eei on v32 */
+ void __iomem *seei;
+ void __iomem *ceei;
+ void __iomem *serq;
+ void __iomem *cerq;
+ void __iomem *cint;
+ void __iomem *cerr;
+ void __iomem *ssrt;
+ void __iomem *cdne;
+ void __iomem *inth;
+ void __iomem *intl;
+ void __iomem *errh;
+ void __iomem *errl;
+ struct fsl_edma_hw_tcd __iomem *tcd;
+};
+
+struct fsl_edma_sw_tcd {
+ dma_addr_t ptcd;
+ struct fsl_edma_hw_tcd *vtcd;
+};
+
+struct fsl_edma_chan {
+ struct virt_dma_chan vchan;
+ enum dma_status status;
+ enum fsl_edma_pm_state pm_state;
+ bool idle;
+ u32 slave_id;
+ struct fsl_edma_engine *edma;
+ struct fsl_edma_desc *edesc;
+ struct dma_pool *tcd_pool;
+ struct dma_slave_config cfg;
+ enum dma_transfer_direction dir;
+};
+
+struct fsl_edma_desc {
+ struct virt_dma_desc vdesc;
+ struct fsl_edma_chan *echan;
+ bool iscyclic;
+ unsigned int n_tcds;
+ struct fsl_edma_sw_tcd tcd[];
+};
+
+enum edma_version {
+ v1, /* 32ch, Vybdir, mpc57x, etc */
+ v2, /* 64ch Coldfire */
+};
+
+struct fsl_edma_engine {
+ struct dma_device dma_dev;
+ struct edma_regs regs;
+ void __iomem *membase;
+ void __iomem *muxbase[DMAMUX_NR];
+ struct clk *muxclk[DMAMUX_NR];
+ struct mutex fsl_edma_mutex;
+ u32 n_chans;
+ int txirq;
+ int errirq;
+ bool big_endian;
+ enum edma_version version;
+ struct fsl_edma_chan chans[];
+};
+
+u32 edma_readl(struct fsl_edma_engine *edma, void __iomem *addr);
+void edma_writeb(struct fsl_edma_engine *edma, u8 val, void __iomem *addr);
+void edma_writew(struct fsl_edma_engine *edma, u16 val, void __iomem *addr);
+void edma_writel(struct fsl_edma_engine *edma, u32 val, void __iomem *addr);
+struct fsl_edma_chan *to_fsl_edma_chan(struct dma_chan *chan);
+struct fsl_edma_desc *to_fsl_edma_desc(struct virt_dma_desc *vd);
+void fsl_edma_setup_regs(struct fsl_edma_engine *edma);
+void fsl_edma_disable_request(struct fsl_edma_chan *fsl_chan);
+void fsl_edma_free_desc(struct virt_dma_desc *vdesc);
+void fsl_edma_xfer_desc(struct fsl_edma_chan *fsl_chan);
+void fsl_edma_cleanup_vchan(struct dma_device *dmadev);
+
+/* Operations */
+int fsl_edma_alloc_chan_resources(struct dma_chan *chan);
+void fsl_edma_free_chan_resources(struct dma_chan *chan);
+int fsl_edma_slave_config(struct dma_chan *chan,
+ struct dma_slave_config *config);
+struct dma_async_tx_descriptor *fsl_edma_prep_dma_cyclic(
+ struct dma_chan *chan, dma_addr_t dma_addr, size_t buf_len,
+ size_t period_len, enum dma_transfer_direction direction,
+ unsigned long flags);
+struct dma_async_tx_descriptor *fsl_edma_prep_slave_sg(
+ struct dma_chan *chan, struct scatterlist *sgl,
+ unsigned int sg_len, enum dma_transfer_direction direction,
+ unsigned long flags, void *context);
+enum dma_status fsl_edma_tx_status(struct dma_chan *chan,
+ dma_cookie_t cookie, struct dma_tx_state *txstate);
+int fsl_edma_pause(struct dma_chan *chan);
+int fsl_edma_resume(struct dma_chan *chan);
+int fsl_edma_terminate_all(struct dma_chan *chan);
+void fsl_edma_issue_pending(struct dma_chan *chan);
+
+#endif /* _FSL_EDMA_COMMON_H_ */
+
diff --git a/drivers/dma/fsl-edma.c b/drivers/dma/fsl-edma.c
index c7568869284e..2f6e4a65d054 100644
--- a/drivers/dma/fsl-edma.c
+++ b/drivers/dma/fsl-edma.c
@@ -1,8 +1,8 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright 2013-2014 Freescale Semiconductor, Inc.
/*
* drivers/dma/fsl-edma.c
*
- * Copyright 2013-2014 Freescale Semiconductor, Inc.
- *
* Driver for the Freescale eDMA engine with flexible channel multiplexing
* capability for DMA request sources. The eDMA block can be found on some
* Vybrid and Layerscape SoCs.
@@ -13,242 +13,20 @@
* option) any later version.
*/
-#include <linux/init.h>
#include <linux/module.h>
#include <linux/interrupt.h>
#include <linux/clk.h>
-#include <linux/dma-mapping.h>
-#include <linux/dmapool.h>
-#include <linux/slab.h>
-#include <linux/spinlock.h>
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/of_address.h>
#include <linux/of_irq.h>
#include <linux/of_dma.h>
-#include "virt-dma.h"
-
-#define EDMA_CR 0x00
-#define EDMA_ES 0x04
-#define EDMA_ERQ 0x0C
-#define EDMA_EEI 0x14
-#define EDMA_SERQ 0x1B
-#define EDMA_CERQ 0x1A
-#define EDMA_SEEI 0x19
-#define EDMA_CEEI 0x18
-#define EDMA_CINT 0x1F
-#define EDMA_CERR 0x1E
-#define EDMA_SSRT 0x1D
-#define EDMA_CDNE 0x1C
-#define EDMA_INTR 0x24
-#define EDMA_ERR 0x2C
-
-#define EDMA_TCD_SADDR(x) (0x1000 + 32 * (x))
-#define EDMA_TCD_SOFF(x) (0x1004 + 32 * (x))
-#define EDMA_TCD_ATTR(x) (0x1006 + 32 * (x))
-#define EDMA_TCD_NBYTES(x) (0x1008 + 32 * (x))
-#define EDMA_TCD_SLAST(x) (0x100C + 32 * (x))
-#define EDMA_TCD_DADDR(x) (0x1010 + 32 * (x))
-#define EDMA_TCD_DOFF(x) (0x1014 + 32 * (x))
-#define EDMA_TCD_CITER_ELINK(x) (0x1016 + 32 * (x))
-#define EDMA_TCD_CITER(x) (0x1016 + 32 * (x))
-#define EDMA_TCD_DLAST_SGA(x) (0x1018 + 32 * (x))
-#define EDMA_TCD_CSR(x) (0x101C + 32 * (x))
-#define EDMA_TCD_BITER_ELINK(x) (0x101E + 32 * (x))
-#define EDMA_TCD_BITER(x) (0x101E + 32 * (x))
-
-#define EDMA_CR_EDBG BIT(1)
-#define EDMA_CR_ERCA BIT(2)
-#define EDMA_CR_ERGA BIT(3)
-#define EDMA_CR_HOE BIT(4)
-#define EDMA_CR_HALT BIT(5)
-#define EDMA_CR_CLM BIT(6)
-#define EDMA_CR_EMLM BIT(7)
-#define EDMA_CR_ECX BIT(16)
-#define EDMA_CR_CX BIT(17)
-
-#define EDMA_SEEI_SEEI(x) ((x) & 0x1F)
-#define EDMA_CEEI_CEEI(x) ((x) & 0x1F)
-#define EDMA_CINT_CINT(x) ((x) & 0x1F)
-#define EDMA_CERR_CERR(x) ((x) & 0x1F)
-
-#define EDMA_TCD_ATTR_DSIZE(x) (((x) & 0x0007))
-#define EDMA_TCD_ATTR_DMOD(x) (((x) & 0x001F) << 3)
-#define EDMA_TCD_ATTR_SSIZE(x) (((x) & 0x0007) << 8)
-#define EDMA_TCD_ATTR_SMOD(x) (((x) & 0x001F) << 11)
-#define EDMA_TCD_ATTR_SSIZE_8BIT (0x0000)
-#define EDMA_TCD_ATTR_SSIZE_16BIT (0x0100)
-#define EDMA_TCD_ATTR_SSIZE_32BIT (0x0200)
-#define EDMA_TCD_ATTR_SSIZE_64BIT (0x0300)
-#define EDMA_TCD_ATTR_SSIZE_32BYTE (0x0500)
-#define EDMA_TCD_ATTR_DSIZE_8BIT (0x0000)
-#define EDMA_TCD_ATTR_DSIZE_16BIT (0x0001)
-#define EDMA_TCD_ATTR_DSIZE_32BIT (0x0002)
-#define EDMA_TCD_ATTR_DSIZE_64BIT (0x0003)
-#define EDMA_TCD_ATTR_DSIZE_32BYTE (0x0005)
-
-#define EDMA_TCD_SOFF_SOFF(x) (x)
-#define EDMA_TCD_NBYTES_NBYTES(x) (x)
-#define EDMA_TCD_SLAST_SLAST(x) (x)
-#define EDMA_TCD_DADDR_DADDR(x) (x)
-#define EDMA_TCD_CITER_CITER(x) ((x) & 0x7FFF)
-#define EDMA_TCD_DOFF_DOFF(x) (x)
-#define EDMA_TCD_DLAST_SGA_DLAST_SGA(x) (x)
-#define EDMA_TCD_BITER_BITER(x) ((x) & 0x7FFF)
-
-#define EDMA_TCD_CSR_START BIT(0)
-#define EDMA_TCD_CSR_INT_MAJOR BIT(1)
-#define EDMA_TCD_CSR_INT_HALF BIT(2)
-#define EDMA_TCD_CSR_D_REQ BIT(3)
-#define EDMA_TCD_CSR_E_SG BIT(4)
-#define EDMA_TCD_CSR_E_LINK BIT(5)
-#define EDMA_TCD_CSR_ACTIVE BIT(6)
-#define EDMA_TCD_CSR_DONE BIT(7)
-
-#define EDMAMUX_CHCFG_DIS 0x0
-#define EDMAMUX_CHCFG_ENBL 0x80
-#define EDMAMUX_CHCFG_SOURCE(n) ((n) & 0x3F)
-
-#define DMAMUX_NR 2
-
-#define FSL_EDMA_BUSWIDTHS BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) | \
- BIT(DMA_SLAVE_BUSWIDTH_2_BYTES) | \
- BIT(DMA_SLAVE_BUSWIDTH_4_BYTES) | \
- BIT(DMA_SLAVE_BUSWIDTH_8_BYTES)
-enum fsl_edma_pm_state {
- RUNNING = 0,
- SUSPENDED,
-};
-
-struct fsl_edma_hw_tcd {
- __le32 saddr;
- __le16 soff;
- __le16 attr;
- __le32 nbytes;
- __le32 slast;
- __le32 daddr;
- __le16 doff;
- __le16 citer;
- __le32 dlast_sga;
- __le16 csr;
- __le16 biter;
-};
-
-struct fsl_edma_sw_tcd {
- dma_addr_t ptcd;
- struct fsl_edma_hw_tcd *vtcd;
-};
-
-struct fsl_edma_slave_config {
- enum dma_transfer_direction dir;
- enum dma_slave_buswidth addr_width;
- u32 dev_addr;
- u32 burst;
- u32 attr;
-};
-
-struct fsl_edma_chan {
- struct virt_dma_chan vchan;
- enum dma_status status;
- enum fsl_edma_pm_state pm_state;
- bool idle;
- u32 slave_id;
- struct fsl_edma_engine *edma;
- struct fsl_edma_desc *edesc;
- struct fsl_edma_slave_config fsc;
- struct dma_pool *tcd_pool;
-};
-
-struct fsl_edma_desc {
- struct virt_dma_desc vdesc;
- struct fsl_edma_chan *echan;
- bool iscyclic;
- unsigned int n_tcds;
- struct fsl_edma_sw_tcd tcd[];
-};
-
-struct fsl_edma_engine {
- struct dma_device dma_dev;
- void __iomem *membase;
- void __iomem *muxbase[DMAMUX_NR];
- struct clk *muxclk[DMAMUX_NR];
- struct mutex fsl_edma_mutex;
- u32 n_chans;
- int txirq;
- int errirq;
- bool big_endian;
- struct fsl_edma_chan chans[];
-};
-
-/*
- * R/W functions for big- or little-endian registers:
- * The eDMA controller's endian is independent of the CPU core's endian.
- * For the big-endian IP module, the offset for 8-bit or 16-bit registers
- * should also be swapped opposite to that in little-endian IP.
- */
+#include "fsl-edma-common.h"
-static u32 edma_readl(struct fsl_edma_engine *edma, void __iomem *addr)
-{
- if (edma->big_endian)
- return ioread32be(addr);
- else
- return ioread32(addr);
-}
-
-static void edma_writeb(struct fsl_edma_engine *edma, u8 val, void __iomem *addr)
-{
- /* swap the reg offset for these in big-endian mode */
- if (edma->big_endian)
- iowrite8(val, (void __iomem *)((unsigned long)addr ^ 0x3));
- else
- iowrite8(val, addr);
-}
-
-static void edma_writew(struct fsl_edma_engine *edma, u16 val, void __iomem *addr)
-{
- /* swap the reg offset for these in big-endian mode */
- if (edma->big_endian)
- iowrite16be(val, (void __iomem *)((unsigned long)addr ^ 0x2));
- else
- iowrite16(val, addr);
-}
-
-static void edma_writel(struct fsl_edma_engine *edma, u32 val, void __iomem *addr)
-{
- if (edma->big_endian)
- iowrite32be(val, addr);
- else
- iowrite32(val, addr);
-}
-
-static struct fsl_edma_chan *to_fsl_edma_chan(struct dma_chan *chan)
-{
- return container_of(chan, struct fsl_edma_chan, vchan.chan);
-}
-
-static struct fsl_edma_desc *to_fsl_edma_desc(struct virt_dma_desc *vd)
-{
- return container_of(vd, struct fsl_edma_desc, vdesc);
-}
-
-static void fsl_edma_enable_request(struct fsl_edma_chan *fsl_chan)
-{
- void __iomem *addr = fsl_chan->edma->membase;
- u32 ch = fsl_chan->vchan.chan.chan_id;
-
- edma_writeb(fsl_chan->edma, EDMA_SEEI_SEEI(ch), addr + EDMA_SEEI);
- edma_writeb(fsl_chan->edma, ch, addr + EDMA_SERQ);
-}
-
-static void fsl_edma_disable_request(struct fsl_edma_chan *fsl_chan)
-{
- void __iomem *addr = fsl_chan->edma->membase;
- u32 ch = fsl_chan->vchan.chan.chan_id;
-
- edma_writeb(fsl_chan->edma, ch, addr + EDMA_CERQ);
- edma_writeb(fsl_chan->edma, EDMA_CEEI_CEEI(ch), addr + EDMA_CEEI);
-}
+#define EDMAMUX_CHCFG_DIS 0
+#define EDMAMUX_CHCFG_ENBL BIT(7)
+#define EDMAMUX_CHCFG_SOURCE(n) ((n) & GENMASK(6, 0))
static void fsl_edma_chan_mux(struct fsl_edma_chan *fsl_chan,
unsigned int slot, bool enable)
@@ -268,416 +46,20 @@ static void fsl_edma_chan_mux(struct fsl_edma_chan *fsl_chan,
iowrite8(EDMAMUX_CHCFG_DIS, muxaddr + ch_off);
}
-static unsigned int fsl_edma_get_tcd_attr(enum dma_slave_buswidth addr_width)
-{
- switch (addr_width) {
- case 1:
- return EDMA_TCD_ATTR_SSIZE_8BIT | EDMA_TCD_ATTR_DSIZE_8BIT;
- case 2:
- return EDMA_TCD_ATTR_SSIZE_16BIT | EDMA_TCD_ATTR_DSIZE_16BIT;
- case 4:
- return EDMA_TCD_ATTR_SSIZE_32BIT | EDMA_TCD_ATTR_DSIZE_32BIT;
- case 8:
- return EDMA_TCD_ATTR_SSIZE_64BIT | EDMA_TCD_ATTR_DSIZE_64BIT;
- default:
- return EDMA_TCD_ATTR_SSIZE_32BIT | EDMA_TCD_ATTR_DSIZE_32BIT;
- }
-}
-
-static void fsl_edma_free_desc(struct virt_dma_desc *vdesc)
-{
- struct fsl_edma_desc *fsl_desc;
- int i;
-
- fsl_desc = to_fsl_edma_desc(vdesc);
- for (i = 0; i < fsl_desc->n_tcds; i++)
- dma_pool_free(fsl_desc->echan->tcd_pool, fsl_desc->tcd[i].vtcd,
- fsl_desc->tcd[i].ptcd);
- kfree(fsl_desc);
-}
-
-static int fsl_edma_terminate_all(struct dma_chan *chan)
-{
- struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan);
- unsigned long flags;
- LIST_HEAD(head);
-
- spin_lock_irqsave(&fsl_chan->vchan.lock, flags);
- fsl_edma_disable_request(fsl_chan);
- fsl_chan->edesc = NULL;
- fsl_chan->idle = true;
- vchan_get_all_descriptors(&fsl_chan->vchan, &head);
- spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags);
- vchan_dma_desc_free_list(&fsl_chan->vchan, &head);
- return 0;
-}
-
-static int fsl_edma_pause(struct dma_chan *chan)
-{
- struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan);
- unsigned long flags;
-
- spin_lock_irqsave(&fsl_chan->vchan.lock, flags);
- if (fsl_chan->edesc) {
- fsl_edma_disable_request(fsl_chan);
- fsl_chan->status = DMA_PAUSED;
- fsl_chan->idle = true;
- }
- spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags);
- return 0;
-}
-
-static int fsl_edma_resume(struct dma_chan *chan)
-{
- struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan);
- unsigned long flags;
-
- spin_lock_irqsave(&fsl_chan->vchan.lock, flags);
- if (fsl_chan->edesc) {
- fsl_edma_enable_request(fsl_chan);
- fsl_chan->status = DMA_IN_PROGRESS;
- fsl_chan->idle = false;
- }
- spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags);
- return 0;
-}
-
-static int fsl_edma_slave_config(struct dma_chan *chan,
- struct dma_slave_config *cfg)
-{
- struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan);
-
- fsl_chan->fsc.dir = cfg->direction;
- if (cfg->direction == DMA_DEV_TO_MEM) {
- fsl_chan->fsc.dev_addr = cfg->src_addr;
- fsl_chan->fsc.addr_width = cfg->src_addr_width;
- fsl_chan->fsc.burst = cfg->src_maxburst;
- fsl_chan->fsc.attr = fsl_edma_get_tcd_attr(cfg->src_addr_width);
- } else if (cfg->direction == DMA_MEM_TO_DEV) {
- fsl_chan->fsc.dev_addr = cfg->dst_addr;
- fsl_chan->fsc.addr_width = cfg->dst_addr_width;
- fsl_chan->fsc.burst = cfg->dst_maxburst;
- fsl_chan->fsc.attr = fsl_edma_get_tcd_attr(cfg->dst_addr_width);
- } else {
- return -EINVAL;
- }
- return 0;
-}
-
-static size_t fsl_edma_desc_residue(struct fsl_edma_chan *fsl_chan,
- struct virt_dma_desc *vdesc, bool in_progress)
-{
- struct fsl_edma_desc *edesc = fsl_chan->edesc;
- void __iomem *addr = fsl_chan->edma->membase;
- u32 ch = fsl_chan->vchan.chan.chan_id;
- enum dma_transfer_direction dir = fsl_chan->fsc.dir;
- dma_addr_t cur_addr, dma_addr;
- size_t len, size;
- int i;
-
- /* calculate the total size in this desc */
- for (len = i = 0; i < fsl_chan->edesc->n_tcds; i++)
- len += le32_to_cpu(edesc->tcd[i].vtcd->nbytes)
- * le16_to_cpu(edesc->tcd[i].vtcd->biter);
-
- if (!in_progress)
- return len;
-
- if (dir == DMA_MEM_TO_DEV)
- cur_addr = edma_readl(fsl_chan->edma, addr + EDMA_TCD_SADDR(ch));
- else
- cur_addr = edma_readl(fsl_chan->edma, addr + EDMA_TCD_DADDR(ch));
-
- /* figure out the finished and calculate the residue */
- for (i = 0; i < fsl_chan->edesc->n_tcds; i++) {
- size = le32_to_cpu(edesc->tcd[i].vtcd->nbytes)
- * le16_to_cpu(edesc->tcd[i].vtcd->biter);
- if (dir == DMA_MEM_TO_DEV)
- dma_addr = le32_to_cpu(edesc->tcd[i].vtcd->saddr);
- else
- dma_addr = le32_to_cpu(edesc->tcd[i].vtcd->daddr);
-
- len -= size;
- if (cur_addr >= dma_addr && cur_addr < dma_addr + size) {
- len += dma_addr + size - cur_addr;
- break;
- }
- }
-
- return len;
-}
-
-static enum dma_status fsl_edma_tx_status(struct dma_chan *chan,
- dma_cookie_t cookie, struct dma_tx_state *txstate)
-{
- struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan);
- struct virt_dma_desc *vdesc;
- enum dma_status status;
- unsigned long flags;
-
- status = dma_cookie_status(chan, cookie, txstate);
- if (status == DMA_COMPLETE)
- return status;
-
- if (!txstate)
- return fsl_chan->status;
-
- spin_lock_irqsave(&fsl_chan->vchan.lock, flags);
- vdesc = vchan_find_desc(&fsl_chan->vchan, cookie);
- if (fsl_chan->edesc && cookie == fsl_chan->edesc->vdesc.tx.cookie)
- txstate->residue = fsl_edma_desc_residue(fsl_chan, vdesc, true);
- else if (vdesc)
- txstate->residue = fsl_edma_desc_residue(fsl_chan, vdesc, false);
- else
- txstate->residue = 0;
-
- spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags);
-
- return fsl_chan->status;
-}
-
-static void fsl_edma_set_tcd_regs(struct fsl_edma_chan *fsl_chan,
- struct fsl_edma_hw_tcd *tcd)
-{
- struct fsl_edma_engine *edma = fsl_chan->edma;
- void __iomem *addr = fsl_chan->edma->membase;
- u32 ch = fsl_chan->vchan.chan.chan_id;
-
- /*
- * TCD parameters are stored in struct fsl_edma_hw_tcd in little
- * endian format. However, we need to load the TCD registers in
- * big- or little-endian obeying the eDMA engine model endian.
- */
- edma_writew(edma, 0, addr + EDMA_TCD_CSR(ch));
- edma_writel(edma, le32_to_cpu(tcd->saddr), addr + EDMA_TCD_SADDR(ch));
- edma_writel(edma, le32_to_cpu(tcd->daddr), addr + EDMA_TCD_DADDR(ch));
-
- edma_writew(edma, le16_to_cpu(tcd->attr), addr + EDMA_TCD_ATTR(ch));
- edma_writew(edma, le16_to_cpu(tcd->soff), addr + EDMA_TCD_SOFF(ch));
-
- edma_writel(edma, le32_to_cpu(tcd->nbytes), addr + EDMA_TCD_NBYTES(ch));
- edma_writel(edma, le32_to_cpu(tcd->slast), addr + EDMA_TCD_SLAST(ch));
-
- edma_writew(edma, le16_to_cpu(tcd->citer), addr + EDMA_TCD_CITER(ch));
- edma_writew(edma, le16_to_cpu(tcd->biter), addr + EDMA_TCD_BITER(ch));
- edma_writew(edma, le16_to_cpu(tcd->doff), addr + EDMA_TCD_DOFF(ch));
-
- edma_writel(edma, le32_to_cpu(tcd->dlast_sga), addr + EDMA_TCD_DLAST_SGA(ch));
-
- edma_writew(edma, le16_to_cpu(tcd->csr), addr + EDMA_TCD_CSR(ch));
-}
-
-static inline
-void fsl_edma_fill_tcd(struct fsl_edma_hw_tcd *tcd, u32 src, u32 dst,
- u16 attr, u16 soff, u32 nbytes, u32 slast, u16 citer,
- u16 biter, u16 doff, u32 dlast_sga, bool major_int,
- bool disable_req, bool enable_sg)
-{
- u16 csr = 0;
-
- /*
- * eDMA hardware SGs require the TCDs to be stored in little
- * endian format irrespective of the register endian model.
- * So we put the value in little endian in memory, waiting
- * for fsl_edma_set_tcd_regs doing the swap.
- */
- tcd->saddr = cpu_to_le32(src);
- tcd->daddr = cpu_to_le32(dst);
-
- tcd->attr = cpu_to_le16(attr);
-
- tcd->soff = cpu_to_le16(EDMA_TCD_SOFF_SOFF(soff));
-
- tcd->nbytes = cpu_to_le32(EDMA_TCD_NBYTES_NBYTES(nbytes));
- tcd->slast = cpu_to_le32(EDMA_TCD_SLAST_SLAST(slast));
-
- tcd->citer = cpu_to_le16(EDMA_TCD_CITER_CITER(citer));
- tcd->doff = cpu_to_le16(EDMA_TCD_DOFF_DOFF(doff));
-
- tcd->dlast_sga = cpu_to_le32(EDMA_TCD_DLAST_SGA_DLAST_SGA(dlast_sga));
-
- tcd->biter = cpu_to_le16(EDMA_TCD_BITER_BITER(biter));
- if (major_int)
- csr |= EDMA_TCD_CSR_INT_MAJOR;
-
- if (disable_req)
- csr |= EDMA_TCD_CSR_D_REQ;
-
- if (enable_sg)
- csr |= EDMA_TCD_CSR_E_SG;
-
- tcd->csr = cpu_to_le16(csr);
-}
-
-static struct fsl_edma_desc *fsl_edma_alloc_desc(struct fsl_edma_chan *fsl_chan,
- int sg_len)
-{
- struct fsl_edma_desc *fsl_desc;
- int i;
-
- fsl_desc = kzalloc(sizeof(*fsl_desc) + sizeof(struct fsl_edma_sw_tcd) * sg_len,
- GFP_NOWAIT);
- if (!fsl_desc)
- return NULL;
-
- fsl_desc->echan = fsl_chan;
- fsl_desc->n_tcds = sg_len;
- for (i = 0; i < sg_len; i++) {
- fsl_desc->tcd[i].vtcd = dma_pool_alloc(fsl_chan->tcd_pool,
- GFP_NOWAIT, &fsl_desc->tcd[i].ptcd);
- if (!fsl_desc->tcd[i].vtcd)
- goto err;
- }
- return fsl_desc;
-
-err:
- while (--i >= 0)
- dma_pool_free(fsl_chan->tcd_pool, fsl_desc->tcd[i].vtcd,
- fsl_desc->tcd[i].ptcd);
- kfree(fsl_desc);
- return NULL;
-}
-
-static struct dma_async_tx_descriptor *fsl_edma_prep_dma_cyclic(
- struct dma_chan *chan, dma_addr_t dma_addr, size_t buf_len,
- size_t period_len, enum dma_transfer_direction direction,
- unsigned long flags)
-{
- struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan);
- struct fsl_edma_desc *fsl_desc;
- dma_addr_t dma_buf_next;
- int sg_len, i;
- u32 src_addr, dst_addr, last_sg, nbytes;
- u16 soff, doff, iter;
-
- if (!is_slave_direction(fsl_chan->fsc.dir))
- return NULL;
-
- sg_len = buf_len / period_len;
- fsl_desc = fsl_edma_alloc_desc(fsl_chan, sg_len);
- if (!fsl_desc)
- return NULL;
- fsl_desc->iscyclic = true;
-
- dma_buf_next = dma_addr;
- nbytes = fsl_chan->fsc.addr_width * fsl_chan->fsc.burst;
- iter = period_len / nbytes;
-
- for (i = 0; i < sg_len; i++) {
- if (dma_buf_next >= dma_addr + buf_len)
- dma_buf_next = dma_addr;
-
- /* get next sg's physical address */
- last_sg = fsl_desc->tcd[(i + 1) % sg_len].ptcd;
-
- if (fsl_chan->fsc.dir == DMA_MEM_TO_DEV) {
- src_addr = dma_buf_next;
- dst_addr = fsl_chan->fsc.dev_addr;
- soff = fsl_chan->fsc.addr_width;
- doff = 0;
- } else {
- src_addr = fsl_chan->fsc.dev_addr;
- dst_addr = dma_buf_next;
- soff = 0;
- doff = fsl_chan->fsc.addr_width;
- }
-
- fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, src_addr, dst_addr,
- fsl_chan->fsc.attr, soff, nbytes, 0, iter,
- iter, doff, last_sg, true, false, true);
- dma_buf_next += period_len;
- }
-
- return vchan_tx_prep(&fsl_chan->vchan, &fsl_desc->vdesc, flags);
-}
-
-static struct dma_async_tx_descriptor *fsl_edma_prep_slave_sg(
- struct dma_chan *chan, struct scatterlist *sgl,
- unsigned int sg_len, enum dma_transfer_direction direction,
- unsigned long flags, void *context)
-{
- struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan);
- struct fsl_edma_desc *fsl_desc;
- struct scatterlist *sg;
- u32 src_addr, dst_addr, last_sg, nbytes;
- u16 soff, doff, iter;
- int i;
-
- if (!is_slave_direction(fsl_chan->fsc.dir))
- return NULL;
-
- fsl_desc = fsl_edma_alloc_desc(fsl_chan, sg_len);
- if (!fsl_desc)
- return NULL;
- fsl_desc->iscyclic = false;
-
- nbytes = fsl_chan->fsc.addr_width * fsl_chan->fsc.burst;
- for_each_sg(sgl, sg, sg_len, i) {
- /* get next sg's physical address */
- last_sg = fsl_desc->tcd[(i + 1) % sg_len].ptcd;
-
- if (fsl_chan->fsc.dir == DMA_MEM_TO_DEV) {
- src_addr = sg_dma_address(sg);
- dst_addr = fsl_chan->fsc.dev_addr;
- soff = fsl_chan->fsc.addr_width;
- doff = 0;
- } else {
- src_addr = fsl_chan->fsc.dev_addr;
- dst_addr = sg_dma_address(sg);
- soff = 0;
- doff = fsl_chan->fsc.addr_width;
- }
-
- iter = sg_dma_len(sg) / nbytes;
- if (i < sg_len - 1) {
- last_sg = fsl_desc->tcd[(i + 1)].ptcd;
- fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, src_addr,
- dst_addr, fsl_chan->fsc.attr, soff,
- nbytes, 0, iter, iter, doff, last_sg,
- false, false, true);
- } else {
- last_sg = 0;
- fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, src_addr,
- dst_addr, fsl_chan->fsc.attr, soff,
- nbytes, 0, iter, iter, doff, last_sg,
- true, true, false);
- }
- }
-
- return vchan_tx_prep(&fsl_chan->vchan, &fsl_desc->vdesc, flags);
-}
-
-static void fsl_edma_xfer_desc(struct fsl_edma_chan *fsl_chan)
-{
- struct virt_dma_desc *vdesc;
-
- vdesc = vchan_next_desc(&fsl_chan->vchan);
- if (!vdesc)
- return;
- fsl_chan->edesc = to_fsl_edma_desc(vdesc);
- fsl_edma_set_tcd_regs(fsl_chan, fsl_chan->edesc->tcd[0].vtcd);
- fsl_edma_enable_request(fsl_chan);
- fsl_chan->status = DMA_IN_PROGRESS;
- fsl_chan->idle = false;
-}
-
static irqreturn_t fsl_edma_tx_handler(int irq, void *dev_id)
{
struct fsl_edma_engine *fsl_edma = dev_id;
unsigned int intr, ch;
- void __iomem *base_addr;
struct fsl_edma_chan *fsl_chan;
+ struct edma_regs *regs = &fsl_edma->regs;
- base_addr = fsl_edma->membase;
-
- intr = edma_readl(fsl_edma, base_addr + EDMA_INTR);
+ intr = edma_readl(fsl_edma, regs->intl);
if (!intr)
return IRQ_NONE;
for (ch = 0; ch < fsl_edma->n_chans; ch++) {
if (intr & (0x1 << ch)) {
- edma_writeb(fsl_edma, EDMA_CINT_CINT(ch),
- base_addr + EDMA_CINT);
+ edma_writeb(fsl_edma, EDMA_CINT_CINT(ch), regs->cint);
fsl_chan = &fsl_edma->chans[ch];
@@ -705,16 +87,16 @@ static irqreturn_t fsl_edma_err_handler(int irq, void *dev_id)
{
struct fsl_edma_engine *fsl_edma = dev_id;
unsigned int err, ch;
+ struct edma_regs *regs = &fsl_edma->regs;
- err = edma_readl(fsl_edma, fsl_edma->membase + EDMA_ERR);
+ err = edma_readl(fsl_edma, regs->errl);
if (!err)
return IRQ_NONE;
for (ch = 0; ch < fsl_edma->n_chans; ch++) {
if (err & (0x1 << ch)) {
fsl_edma_disable_request(&fsl_edma->chans[ch]);
- edma_writeb(fsl_edma, EDMA_CERR_CERR(ch),
- fsl_edma->membase + EDMA_CERR);
+ edma_writeb(fsl_edma, EDMA_CERR_CERR(ch), regs->cerr);
fsl_edma->chans[ch].status = DMA_ERROR;
fsl_edma->chans[ch].idle = true;
}
@@ -730,25 +112,6 @@ static irqreturn_t fsl_edma_irq_handler(int irq, void *dev_id)
return fsl_edma_err_handler(irq, dev_id);
}
-static void fsl_edma_issue_pending(struct dma_chan *chan)
-{
- struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan);
- unsigned long flags;
-
- spin_lock_irqsave(&fsl_chan->vchan.lock, flags);
-
- if (unlikely(fsl_chan->pm_state != RUNNING)) {
- spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags);
- /* cannot submit due to suspend */
- return;
- }
-
- if (vchan_issue_pending(&fsl_chan->vchan) && !fsl_chan->edesc)
- fsl_edma_xfer_desc(fsl_chan);
-
- spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags);
-}
-
static struct dma_chan *fsl_edma_xlate(struct of_phandle_args *dma_spec,
struct of_dma *ofdma)
{
@@ -761,7 +124,8 @@ static struct dma_chan *fsl_edma_xlate(struct of_phandle_args *dma_spec,
return NULL;
mutex_lock(&fsl_edma->fsl_edma_mutex);
- list_for_each_entry_safe(chan, _chan, &fsl_edma->dma_dev.channels, device_node) {
+ list_for_each_entry_safe(chan, _chan,
+ &fsl_edma->dma_dev.channels, device_node) {
if (chan->client_count)
continue;
if ((chan->chan_id / chans_per_mux) == dma_spec->args[0]) {
@@ -778,39 +142,12 @@ static struct dma_chan *fsl_edma_xlate(struct of_phandle_args *dma_spec,
}
}
mutex_unlock(&fsl_edma->fsl_edma_mutex);
- return NULL;
-}
-
-static int fsl_edma_alloc_chan_resources(struct dma_chan *chan)
-{
- struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan);
-
- fsl_chan->tcd_pool = dma_pool_create("tcd_pool", chan->device->dev,
- sizeof(struct fsl_edma_hw_tcd),
- 32, 0);
- return 0;
-}
-static void fsl_edma_free_chan_resources(struct dma_chan *chan)
-{
- struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan);
- unsigned long flags;
- LIST_HEAD(head);
-
- spin_lock_irqsave(&fsl_chan->vchan.lock, flags);
- fsl_edma_disable_request(fsl_chan);
- fsl_edma_chan_mux(fsl_chan, 0, false);
- fsl_chan->edesc = NULL;
- vchan_get_all_descriptors(&fsl_chan->vchan, &head);
- spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags);
-
- vchan_dma_desc_free_list(&fsl_chan->vchan, &head);
- dma_pool_destroy(fsl_chan->tcd_pool);
- fsl_chan->tcd_pool = NULL;
+ return NULL;
}
-static int
-fsl_edma_irq_init(struct platform_device *pdev, struct fsl_edma_engine *fsl_edma)
+static int fsl_edma_irq_init(struct platform_device *pdev,
+ struct fsl_edma_engine *fsl_edma)
{
int ret;
@@ -852,17 +189,6 @@ fsl_edma_irq_init(struct platform_device *pdev, struct fsl_edma_engine *fsl_edma
return 0;
}
-static void fsl_edma_irq_exit(
- struct platform_device *pdev, struct fsl_edma_engine *fsl_edma)
-{
- if (fsl_edma->txirq == fsl_edma->errirq) {
- devm_free_irq(&pdev->dev, fsl_edma->txirq, fsl_edma);
- } else {
- devm_free_irq(&pdev->dev, fsl_edma->txirq, fsl_edma);
- devm_free_irq(&pdev->dev, fsl_edma->errirq, fsl_edma);
- }
-}
-
static void fsl_disable_clocks(struct fsl_edma_engine *fsl_edma, int nr_clocks)
{
int i;
@@ -876,6 +202,7 @@ static int fsl_edma_probe(struct platform_device *pdev)
struct device_node *np = pdev->dev.of_node;
struct fsl_edma_engine *fsl_edma;
struct fsl_edma_chan *fsl_chan;
+ struct edma_regs *regs;
struct resource *res;
int len, chans;
int ret, i;
@@ -891,6 +218,8 @@ static int fsl_edma_probe(struct platform_device *pdev)
if (!fsl_edma)
return -ENOMEM;
+ fsl_edma->version = v1;
+
fsl_edma->n_chans = chans;
mutex_init(&fsl_edma->fsl_edma_mutex);
@@ -899,6 +228,9 @@ static int fsl_edma_probe(struct platform_device *pdev)
if (IS_ERR(fsl_edma->membase))
return PTR_ERR(fsl_edma->membase);
+ fsl_edma_setup_regs(fsl_edma);
+ regs = &fsl_edma->regs;
+
for (i = 0; i < DMAMUX_NR; i++) {
char clkname[32];
@@ -939,11 +271,11 @@ static int fsl_edma_probe(struct platform_device *pdev)
fsl_chan->vchan.desc_free = fsl_edma_free_desc;
vchan_init(&fsl_chan->vchan, &fsl_edma->dma_dev);
- edma_writew(fsl_edma, 0x0, fsl_edma->membase + EDMA_TCD_CSR(i));
+ edma_writew(fsl_edma, 0x0, ®s->tcd[i].csr);
fsl_edma_chan_mux(fsl_chan, 0, false);
}
- edma_writel(fsl_edma, ~0, fsl_edma->membase + EDMA_INTR);
+ edma_writel(fsl_edma, ~0, regs->intl);
ret = fsl_edma_irq_init(pdev, fsl_edma);
if (ret)
return ret;
@@ -990,28 +322,16 @@ static int fsl_edma_probe(struct platform_device *pdev)
}
/* enable round robin arbitration */
- edma_writel(fsl_edma, EDMA_CR_ERGA | EDMA_CR_ERCA, fsl_edma->membase + EDMA_CR);
+ edma_writel(fsl_edma, EDMA_CR_ERGA | EDMA_CR_ERCA, regs->cr);
return 0;
}
-static void fsl_edma_cleanup_vchan(struct dma_device *dmadev)
-{
- struct fsl_edma_chan *chan, *_chan;
-
- list_for_each_entry_safe(chan, _chan,
- &dmadev->channels, vchan.chan.device_node) {
- list_del(&chan->vchan.chan.device_node);
- tasklet_kill(&chan->vchan.task);
- }
-}
-
static int fsl_edma_remove(struct platform_device *pdev)
{
struct device_node *np = pdev->dev.of_node;
struct fsl_edma_engine *fsl_edma = platform_get_drvdata(pdev);
- fsl_edma_irq_exit(pdev, fsl_edma);
fsl_edma_cleanup_vchan(&fsl_edma->dma_dev);
of_dma_controller_free(np);
dma_async_device_unregister(&fsl_edma->dma_dev);
@@ -1048,18 +368,19 @@ static int fsl_edma_resume_early(struct device *dev)
{
struct fsl_edma_engine *fsl_edma = dev_get_drvdata(dev);
struct fsl_edma_chan *fsl_chan;
+ struct edma_regs *regs = &fsl_edma->regs;
int i;
for (i = 0; i < fsl_edma->n_chans; i++) {
fsl_chan = &fsl_edma->chans[i];
fsl_chan->pm_state = RUNNING;
- edma_writew(fsl_edma, 0x0, fsl_edma->membase + EDMA_TCD_CSR(i));
+ edma_writew(fsl_edma, 0x0, ®s->tcd[i].csr);
+
if (fsl_chan->slave_id != 0)
fsl_edma_chan_mux(fsl_chan, fsl_chan->slave_id, true);
}
- edma_writel(fsl_edma, EDMA_CR_ERGA | EDMA_CR_ERCA,
- fsl_edma->membase + EDMA_CR);
+ edma_writel(fsl_edma, EDMA_CR_ERGA | EDMA_CR_ERCA, regs->cr);
return 0;
}
^ permalink raw reply related
* [v5,7/7] dmaengine: imx-sdma: alloclate bd memory from dma pool
From: Robin Gong @ 2018-06-19 16:57 UTC (permalink / raw)
To: vkoul, s.hauer, l.stach, dan.j.williams, gregkh, jslaby
Cc: linux-serial, dmaengine, linux-kernel, linux-arm-kernel,
linux-imx
dma_terminate_all maybe called in interrupt context which means
WARN_ON() will be triggered as below when bd memory freed. Allocat
bd memory from dma pool instead.
[ 29.161079] WARNING: CPU: 1 PID: 533 at ./include/linux/dma-mapping.h:541 sdma_free_bd+0xa4/0xb4
[ 29.169883] Modules linked in:
[ 29.172990] CPU: 1 PID: 533 Comm: mpegaudioparse0 Not tainted 4.18.0-rc1-next-20180618-00009-gf79f22c #20
[ 29.182597] Hardware name: Freescale i.MX6 Quad/DualLite (Device Tree)
[ 29.189163] Backtrace:
[ 29.191685] [<c010d1e0>] (dump_backtrace) from [<c010d4a0>] (show_stack+0x18/0x1c)
[ 29.199306] r7:00000000 r6:600f0093 r5:00000000 r4:c107db7c
[ 29.205029] [<c010d488>] (show_stack) from [<c0a5bba0>] (dump_stack+0xb4/0xe8)
[ 29.212312] [<c0a5baec>] (dump_stack) from [<c012703c>] (__warn+0x104/0x130)
[ 29.219411] r9:ec3e817c r8:0000021d r7:00000009 r6:c0d1d440 r5:00000000 r4:00000000
[ 29.227204] [<c0126f38>] (__warn) from [<c0127180>] (warn_slowpath_null+0x44/0x50)
[ 29.234821] r8:ed129dc4 r7:c0b01978 r6:c04d4e90 r5:0000021d r4:c0d1d440
[ 29.241574] [<c012713c>] (warn_slowpath_null) from [<c04d4e90>] (sdma_free_bd+0xa4/0xb4)
[ 29.249706] r6:4c001000 r5:f082e000 r4:00000024
[ 29.254376] [<c04d4dec>] (sdma_free_bd) from [<c04d4eb4>] (sdma_desc_free+0x14/0x20)
[ 29.262163] r7:ec3e8110 r6:00000100 r5:00000200 r4:ecf89a00
[ 29.267873] [<c04d4ea0>] (sdma_desc_free) from [<c04d229c>] (vchan_dma_desc_free_list+0xa4/0xac)
[ 29.276697] r5:00000200 r4:ed129d9c
[ 29.280326] [<c04d21f8>] (vchan_dma_desc_free_list) from [<c04d482c>] (sdma_disable_channel_with_delay+0x14c/0x188)
[ 29.290808] r9:ecae560c r8:ec3e815c r7:00000000 r6:c1008908 r5:ed129dc4 r4:ec3e8110
[ 29.298605] [<c04d46e0>] (sdma_disable_channel_with_delay) from [<c07c5c84>] (snd_dmaengine_pcm_trigger+0x90/0x1b0)
[ 29.309087] r8:ecae5000 r7:ec940800 r6:ed31bd80 r5:ecadb200 r4:ec26a700
[ 29.315855] [<c07c5bf4>] (snd_dmaengine_pcm_trigger) from [<c07dd800>] (soc_pcm_trigger+0xb4/0x130)
[ 29.324953] r8:ecae5000 r7:ec940800 r6:00000000 r5:ecadb200 r4:ec26a700
[ 29.331716] [<c07dd74c>] (soc_pcm_trigger) from [<c07bc008>] (snd_pcm_do_stop+0x58/0x5c)
[ 29.339859] r9:ecaed5a8 r8:ed31bdc0 r7:00000000 r6:00000001 r5:ecadb200 r4:c0b9c4d0
[ 29.347652] [<c07bbfb0>] (snd_pcm_do_stop) from [<c07bbde8>] (snd_pcm_action_single+0x40/0x80)
[ 29.356315] [<c07bbda8>] (snd_pcm_action_single) from [<c07bbf1c>] (snd_pcm_action+0xf4/0xfc)
[ 29.364883] r7:00000001 r6:c0b9c4d0 r5:ecadb2d4 r4:ecadb200
[ 29.370593] [<c07bbe28>] (snd_pcm_action) from [<c07bc8dc>] (snd_pcm_drop+0x58/0x9c)
Signed-off-by: Robin Gong <yibin.gong@nxp.com>
---
drivers/dma/imx-sdma.c | 18 ++++++++++++------
1 file changed, 12 insertions(+), 6 deletions(-)
diff --git a/drivers/dma/imx-sdma.c b/drivers/dma/imx-sdma.c
index f8becaf..7dab7e9 100644
--- a/drivers/dma/imx-sdma.c
+++ b/drivers/dma/imx-sdma.c
@@ -24,6 +24,7 @@
#include <linux/spinlock.h>
#include <linux/device.h>
#include <linux/dma-mapping.h>
+#include <linux/dmapool.h>
#include <linux/firmware.h>
#include <linux/slab.h>
#include <linux/platform_device.h>
@@ -343,6 +344,7 @@ struct sdma_channel {
u32 shp_addr, per_addr;
enum dma_status status;
struct imx_dma_data data;
+ struct dma_pool *bd_pool;
};
#define IMX_DMA_SG_LOOP BIT(0)
@@ -1153,11 +1155,10 @@ static int sdma_request_channel0(struct sdma_engine *sdma)
static int sdma_alloc_bd(struct sdma_desc *desc)
{
- u32 bd_size = desc->num_bd * sizeof(struct sdma_buffer_descriptor);
int ret = 0;
- desc->bd = dma_zalloc_coherent(NULL, bd_size, &desc->bd_phys,
- GFP_ATOMIC);
+ desc->bd = dma_pool_alloc(desc->sdmac->bd_pool, GFP_ATOMIC,
+ &desc->bd_phys);
if (!desc->bd) {
ret = -ENOMEM;
goto out;
@@ -1168,9 +1169,7 @@ static int sdma_alloc_bd(struct sdma_desc *desc)
static void sdma_free_bd(struct sdma_desc *desc)
{
- u32 bd_size = desc->num_bd * sizeof(struct sdma_buffer_descriptor);
-
- dma_free_coherent(NULL, bd_size, desc->bd, desc->bd_phys);
+ dma_pool_free(desc->sdmac->bd_pool, desc->bd, desc->bd_phys);
}
static void sdma_desc_free(struct virt_dma_desc *vd)
@@ -1218,6 +1217,10 @@ static int sdma_alloc_chan_resources(struct dma_chan *chan)
if (ret)
goto disable_clk_ahb;
+ sdmac->bd_pool = dma_pool_create("bd_pool", chan->device->dev,
+ sizeof(struct sdma_buffer_descriptor),
+ 32, 0);
+
return 0;
disable_clk_ahb:
@@ -1246,6 +1249,9 @@ static void sdma_free_chan_resources(struct dma_chan *chan)
clk_disable(sdma->clk_ipg);
clk_disable(sdma->clk_ahb);
+
+ dma_pool_destroy(sdmac->bd_pool);
+ sdmac->bd_pool = NULL;
}
static struct sdma_desc *sdma_transfer_init(struct sdma_channel *sdmac,
^ permalink raw reply related
* [v5,6/7] dmaengine: imx-sdma: add sdma_transfer_init to decrease code overlap
From: Robin Gong @ 2018-06-19 16:57 UTC (permalink / raw)
To: vkoul, s.hauer, l.stach, dan.j.williams, gregkh, jslaby
Cc: linux-serial, dmaengine, linux-kernel, linux-arm-kernel,
linux-imx
There are lot of codes overlap between prep_sg and prep_cyclic function.
Add sdma_transfer_init() function to elimated the code overlap.
Signed-off-by: Robin Gong <yibin.gong@nxp.com>
---
drivers/dma/imx-sdma.c | 83 ++++++++++++++++++++++----------------------------
1 file changed, 37 insertions(+), 46 deletions(-)
diff --git a/drivers/dma/imx-sdma.c b/drivers/dma/imx-sdma.c
index 9f1a462..f8becaf 100644
--- a/drivers/dma/imx-sdma.c
+++ b/drivers/dma/imx-sdma.c
@@ -1248,6 +1248,40 @@ static void sdma_free_chan_resources(struct dma_chan *chan)
clk_disable(sdma->clk_ahb);
}
+static struct sdma_desc *sdma_transfer_init(struct sdma_channel *sdmac,
+ enum dma_transfer_direction direction, u32 bds)
+{
+ struct sdma_desc *desc;
+
+ desc = kzalloc((sizeof(*desc)), GFP_NOWAIT);
+ if (!desc)
+ goto err_out;
+
+ sdmac->status = DMA_IN_PROGRESS;
+ sdmac->direction = direction;
+ sdmac->flags = 0;
+
+ desc->chn_count = 0;
+ desc->chn_real_count = 0;
+ desc->buf_tail = 0;
+ desc->buf_ptail = 0;
+ desc->sdmac = sdmac;
+ desc->num_bd = bds;
+
+ if (sdma_alloc_bd(desc))
+ goto err_desc_out;
+
+ if (sdma_load_context(sdmac))
+ goto err_desc_out;
+
+ return desc;
+
+err_desc_out:
+ kfree(desc);
+err_out:
+ return NULL;
+}
+
static struct dma_async_tx_descriptor *sdma_prep_slave_sg(
struct dma_chan *chan, struct scatterlist *sgl,
unsigned int sg_len, enum dma_transfer_direction direction,
@@ -1260,36 +1294,13 @@ static struct dma_async_tx_descriptor *sdma_prep_slave_sg(
struct scatterlist *sg;
struct sdma_desc *desc;
- if (sdmac->status == DMA_IN_PROGRESS)
- return NULL;
- sdmac->status = DMA_IN_PROGRESS;
-
- sdmac->flags = 0;
-
- desc = kzalloc((sizeof(*desc)), GFP_NOWAIT);
+ desc = sdma_transfer_init(sdmac, direction, sg_len);
if (!desc)
goto err_out;
- desc->buf_tail = 0;
- desc->buf_ptail = 0;
- desc->sdmac = sdmac;
- desc->num_bd = sg_len;
- desc->chn_real_count = 0;
-
- if (sdma_alloc_bd(desc)) {
- kfree(desc);
- goto err_out;
- }
-
dev_dbg(sdma->dev, "setting up %d entries for channel %d.\n",
sg_len, channel);
- sdmac->direction = direction;
- ret = sdma_load_context(sdmac);
- if (ret)
- goto err_bd_out;
-
- desc->chn_count = 0;
for_each_sg(sgl, sg, sg_len, i) {
struct sdma_buffer_descriptor *bd = &desc->bd[i];
int param;
@@ -1365,38 +1376,18 @@ static struct dma_async_tx_descriptor *sdma_prep_dma_cyclic(
struct sdma_engine *sdma = sdmac->sdma;
int num_periods = buf_len / period_len;
int channel = sdmac->channel;
- int ret, i = 0, buf = 0;
+ int i = 0, buf = 0;
struct sdma_desc *desc;
dev_dbg(sdma->dev, "%s channel: %d\n", __func__, channel);
- if (sdmac->status == DMA_IN_PROGRESS)
- return NULL;
-
- sdmac->status = DMA_IN_PROGRESS;
-
- desc = kzalloc((sizeof(*desc)), GFP_NOWAIT);
+ desc = sdma_transfer_init(sdmac, direction, num_periods);
if (!desc)
goto err_out;
- desc->buf_tail = 0;
- desc->buf_ptail = 0;
- desc->sdmac = sdmac;
- desc->num_bd = num_periods;
- desc->chn_real_count = 0;
desc->period_len = period_len;
sdmac->flags |= IMX_DMA_SG_LOOP;
- sdmac->direction = direction;
-
- if (sdma_alloc_bd(desc)) {
- kfree(desc);
- goto err_bd_out;
- }
-
- ret = sdma_load_context(sdmac);
- if (ret)
- goto err_bd_out;
if (period_len > 0xffff) {
dev_err(sdma->dev, "SDMA channel %d: maximum period size exceeded: %zu > %d\n",
^ permalink raw reply related
* [v5,5/7] dmaengine: imx-sdma: remove the maximum limitation for bd numbers
From: Robin Gong @ 2018-06-19 16:57 UTC (permalink / raw)
To: vkoul, s.hauer, l.stach, dan.j.williams, gregkh, jslaby
Cc: linux-serial, dmaengine, linux-kernel, linux-arm-kernel,
linux-imx
No this limitation now after virtual dma used since bd is allocated
dynamically instead of static.
Signed-off-by: Robin Gong <yibin.gong@nxp.com>
---
drivers/dma/imx-sdma.c | 14 --------------
1 file changed, 14 deletions(-)
diff --git a/drivers/dma/imx-sdma.c b/drivers/dma/imx-sdma.c
index d1d3494..9f1a462 100644
--- a/drivers/dma/imx-sdma.c
+++ b/drivers/dma/imx-sdma.c
@@ -285,7 +285,6 @@ struct sdma_context_data {
u32 scratch7;
} __attribute__ ((packed));
-#define NUM_BD (int)(PAGE_SIZE / sizeof(struct sdma_buffer_descriptor))
struct sdma_engine;
@@ -1290,13 +1289,6 @@ static struct dma_async_tx_descriptor *sdma_prep_slave_sg(
if (ret)
goto err_bd_out;
- if (sg_len > NUM_BD) {
- dev_err(sdma->dev, "SDMA channel %d: maximum number of sg exceeded: %d > %d\n",
- channel, sg_len, NUM_BD);
- ret = -EINVAL;
- goto err_bd_out;
- }
-
desc->chn_count = 0;
for_each_sg(sgl, sg, sg_len, i) {
struct sdma_buffer_descriptor *bd = &desc->bd[i];
@@ -1406,12 +1398,6 @@ static struct dma_async_tx_descriptor *sdma_prep_dma_cyclic(
if (ret)
goto err_bd_out;
- if (num_periods > NUM_BD) {
- dev_err(sdma->dev, "SDMA channel %d: maximum number of sg exceeded: %d > %d\n",
- channel, num_periods, NUM_BD);
- goto err_bd_out;
- }
-
if (period_len > 0xffff) {
dev_err(sdma->dev, "SDMA channel %d: maximum period size exceeded: %zu > %d\n",
channel, period_len, 0xffff);
^ permalink raw reply related
* [v5,4/7] dmaengine: imx-sdma: remove useless 'lock' and 'enabled' in 'struct sdma_channel'
From: Robin Gong @ 2018-06-19 16:57 UTC (permalink / raw)
To: vkoul, s.hauer, l.stach, dan.j.williams, gregkh, jslaby
Cc: linux-serial, dmaengine, linux-kernel, linux-arm-kernel,
linux-imx
Since 'sdmac->vc.lock' and 'sdmac->desc' can be used as 'lock' and
'enabled' in 'struct sdma_channel sdmac', remove them.
Signed-off-by: Robin Gong <yibin.gong@nxp.com>
---
drivers/dma/imx-sdma.c | 23 -----------------------
1 file changed, 23 deletions(-)
diff --git a/drivers/dma/imx-sdma.c b/drivers/dma/imx-sdma.c
index 86fa799..d1d3494 100644
--- a/drivers/dma/imx-sdma.c
+++ b/drivers/dma/imx-sdma.c
@@ -342,10 +342,8 @@ struct sdma_channel {
unsigned long event_mask[2];
unsigned long watermark_level;
u32 shp_addr, per_addr;
- spinlock_t lock;
enum dma_status status;
struct imx_dma_data data;
- bool enabled;
};
#define IMX_DMA_SG_LOOP BIT(0)
@@ -606,14 +604,7 @@ static int sdma_config_ownership(struct sdma_channel *sdmac,
static void sdma_enable_channel(struct sdma_engine *sdma, int channel)
{
- unsigned long flags;
- struct sdma_channel *sdmac = &sdma->channel[channel];
-
writel(BIT(channel), sdma->regs + SDMA_H_START);
-
- spin_lock_irqsave(&sdmac->lock, flags);
- sdmac->enabled = true;
- spin_unlock_irqrestore(&sdmac->lock, flags);
}
/*
@@ -731,14 +722,6 @@ static void sdma_update_channel_loop(struct sdma_channel *sdmac)
struct sdma_buffer_descriptor *bd;
int error = 0;
enum dma_status old_status = sdmac->status;
- unsigned long flags;
-
- spin_lock_irqsave(&sdmac->lock, flags);
- if (!sdmac->enabled) {
- spin_unlock_irqrestore(&sdmac->lock, flags);
- return;
- }
- spin_unlock_irqrestore(&sdmac->lock, flags);
/*
* loop mode. Iterate over descriptors, re-setup them and
@@ -1000,15 +983,10 @@ static int sdma_disable_channel(struct dma_chan *chan)
struct sdma_channel *sdmac = to_sdma_chan(chan);
struct sdma_engine *sdma = sdmac->sdma;
int channel = sdmac->channel;
- unsigned long flags;
writel_relaxed(BIT(channel), sdma->regs + SDMA_H_STATSTOP);
sdmac->status = DMA_ERROR;
- spin_lock_irqsave(&sdmac->lock, flags);
- sdmac->enabled = false;
- spin_unlock_irqrestore(&sdmac->lock, flags);
-
return 0;
}
@@ -1922,7 +1900,6 @@ static int sdma_probe(struct platform_device *pdev)
struct sdma_channel *sdmac = &sdma->channel[i];
sdmac->sdma = sdma;
- spin_lock_init(&sdmac->lock);
sdmac->channel = i;
sdmac->vc.desc_free = sdma_desc_free;
^ permalink raw reply related
* [v5,3/7] dmaengine: imx-sdma: add virt-dma support
From: Robin Gong @ 2018-06-19 16:57 UTC (permalink / raw)
To: vkoul, s.hauer, l.stach, dan.j.williams, gregkh, jslaby
Cc: linux-serial, dmaengine, linux-kernel, linux-arm-kernel,
linux-imx
The legacy sdma driver has below limitations or drawbacks:
1. Hardcode the max BDs number as "PAGE_SIZE / sizeof(*)", and alloc
one page size for one channel regardless of only few BDs needed
most time. But in few cases, the max PAGE_SIZE maybe not enough.
2. One SDMA channel can't stop immediatley once channel disabled which
means SDMA interrupt may come in after this channel terminated.There
are some patches for this corner case such as commit "2746e2c389f9",
but not cover non-cyclic.
The common virt-dma overcomes the above limitations. It can alloc bd
dynamically and free bd once this tx transfer done. No memory wasted or
maximum limititation here, only depends on how many memory can be requested
from kernel. For No.2, such issue can be workaround by checking if there
is available descript("sdmac->desc") now once the unwanted interrupt
coming. At last the common virt-dma is easier for sdma driver maintain.
Signed-off-by: Robin Gong <yibin.gong@nxp.com>
---
drivers/dma/Kconfig | 1 +
drivers/dma/imx-sdma.c | 263 ++++++++++++++++++++++++++++++++-----------------
2 files changed, 171 insertions(+), 93 deletions(-)
diff --git a/drivers/dma/Kconfig b/drivers/dma/Kconfig
index ca1680a..d4a4230 100644
--- a/drivers/dma/Kconfig
+++ b/drivers/dma/Kconfig
@@ -250,6 +250,7 @@ config IMX_SDMA
tristate "i.MX SDMA support"
depends on ARCH_MXC
select DMA_ENGINE
+ select DMA_VIRTUAL_CHANNELS
help
Support the i.MX SDMA engine. This engine is integrated into
Freescale i.MX25/31/35/51/53/6 chips.
diff --git a/drivers/dma/imx-sdma.c b/drivers/dma/imx-sdma.c
index 19c351f..86fa799 100644
--- a/drivers/dma/imx-sdma.c
+++ b/drivers/dma/imx-sdma.c
@@ -41,6 +41,7 @@
#include <linux/mfd/syscon/imx6q-iomuxc-gpr.h>
#include "dmaengine.h"
+#include "virt-dma.h"
/* SDMA registers */
#define SDMA_H_C0PTR 0x000
@@ -301,6 +302,7 @@ struct sdma_engine;
* @bd pointer of alloced bd
*/
struct sdma_desc {
+ struct virt_dma_desc vd;
unsigned int num_bd;
dma_addr_t bd_phys;
unsigned int buf_tail;
@@ -324,8 +326,8 @@ struct sdma_desc {
* @word_size peripheral access size
*/
struct sdma_channel {
+ struct virt_dma_chan vc;
struct sdma_desc *desc;
- struct sdma_desc _desc;
struct sdma_engine *sdma;
unsigned int channel;
enum dma_transfer_direction direction;
@@ -340,11 +342,8 @@ struct sdma_channel {
unsigned long event_mask[2];
unsigned long watermark_level;
u32 shp_addr, per_addr;
- struct dma_chan chan;
spinlock_t lock;
- struct dma_async_tx_descriptor txdesc;
enum dma_status status;
- struct tasklet_struct tasklet;
struct imx_dma_data data;
bool enabled;
};
@@ -698,6 +697,35 @@ static void sdma_event_disable(struct sdma_channel *sdmac, unsigned int event)
writel_relaxed(val, sdma->regs + chnenbl);
}
+static struct sdma_desc *to_sdma_desc(struct dma_async_tx_descriptor *t)
+{
+ return container_of(t, struct sdma_desc, vd.tx);
+}
+
+static void sdma_start_desc(struct sdma_channel *sdmac)
+{
+ struct virt_dma_desc *vd = vchan_next_desc(&sdmac->vc);
+ struct sdma_desc *desc;
+ struct sdma_engine *sdma = sdmac->sdma;
+ int channel = sdmac->channel;
+
+ if (!vd) {
+ sdmac->desc = NULL;
+ return;
+ }
+ sdmac->desc = desc = to_sdma_desc(&vd->tx);
+ /*
+ * Do not delete the node in desc_issued list in cyclic mode, otherwise
+ * the desc alloced will never be freed in vchan_dma_desc_free_list
+ */
+ if (!(sdmac->flags & IMX_DMA_SG_LOOP))
+ list_del(&vd->node);
+
+ sdma->channel_control[channel].base_bd_ptr = desc->bd_phys;
+ sdma->channel_control[channel].current_bd_ptr = desc->bd_phys;
+ sdma_enable_channel(sdma, sdmac->channel);
+}
+
static void sdma_update_channel_loop(struct sdma_channel *sdmac)
{
struct sdma_buffer_descriptor *bd;
@@ -716,7 +744,7 @@ static void sdma_update_channel_loop(struct sdma_channel *sdmac)
* loop mode. Iterate over descriptors, re-setup them and
* call callback function.
*/
- while (1) {
+ while (sdmac->desc) {
struct sdma_desc *desc = sdmac->desc;
bd = &desc->bd[desc->buf_tail];
@@ -747,15 +775,16 @@ static void sdma_update_channel_loop(struct sdma_channel *sdmac)
* SDMA transaction status by the time the client tasklet is
* executed.
*/
-
- dmaengine_desc_get_callback_invoke(&sdmac->txdesc, NULL);
+ spin_unlock(&sdmac->vc.lock);
+ dmaengine_desc_get_callback_invoke(&desc->vd.tx, NULL);
+ spin_lock(&sdmac->vc.lock);
if (error)
sdmac->status = old_status;
}
}
-static void mxc_sdma_handle_channel_normal(unsigned long data)
+static void mxc_sdma_handle_channel_normal(struct sdma_channel *data)
{
struct sdma_channel *sdmac = (struct sdma_channel *) data;
struct sdma_buffer_descriptor *bd;
@@ -778,10 +807,6 @@ static void mxc_sdma_handle_channel_normal(unsigned long data)
sdmac->status = DMA_ERROR;
else
sdmac->status = DMA_COMPLETE;
-
- dma_cookie_complete(&sdmac->txdesc);
-
- dmaengine_desc_get_callback_invoke(&sdmac->txdesc, NULL);
}
static irqreturn_t sdma_int_handler(int irq, void *dev_id)
@@ -797,12 +822,21 @@ static irqreturn_t sdma_int_handler(int irq, void *dev_id)
while (stat) {
int channel = fls(stat) - 1;
struct sdma_channel *sdmac = &sdma->channel[channel];
+ struct sdma_desc *desc;
+
+ spin_lock(&sdmac->vc.lock);
+ desc = sdmac->desc;
+ if (desc) {
+ if (sdmac->flags & IMX_DMA_SG_LOOP) {
+ sdma_update_channel_loop(sdmac);
+ } else {
+ mxc_sdma_handle_channel_normal(sdmac);
+ vchan_cookie_complete(&desc->vd);
+ sdma_start_desc(sdmac);
+ }
+ }
- if (sdmac->flags & IMX_DMA_SG_LOOP)
- sdma_update_channel_loop(sdmac);
- else
- tasklet_schedule(&sdmac->tasklet);
-
+ spin_unlock(&sdmac->vc.lock);
__clear_bit(channel, &stat);
}
@@ -958,7 +992,7 @@ static int sdma_load_context(struct sdma_channel *sdmac)
static struct sdma_channel *to_sdma_chan(struct dma_chan *chan)
{
- return container_of(chan, struct sdma_channel, chan);
+ return container_of(chan, struct sdma_channel, vc.chan);
}
static int sdma_disable_channel(struct dma_chan *chan)
@@ -980,7 +1014,16 @@ static int sdma_disable_channel(struct dma_chan *chan)
static int sdma_disable_channel_with_delay(struct dma_chan *chan)
{
+ struct sdma_channel *sdmac = to_sdma_chan(chan);
+ unsigned long flags;
+ LIST_HEAD(head);
+
sdma_disable_channel(chan);
+ spin_lock_irqsave(&sdmac->vc.lock, flags);
+ vchan_get_all_descriptors(&sdmac->vc, &head);
+ sdmac->desc = NULL;
+ spin_unlock_irqrestore(&sdmac->vc.lock, flags);
+ vchan_dma_desc_free_list(&sdmac->vc, &head);
/*
* According to NXP R&D team a delay of one BD SDMA cost time
@@ -1109,46 +1152,56 @@ static int sdma_set_channel_priority(struct sdma_channel *sdmac,
return 0;
}
-static int sdma_request_channel(struct sdma_channel *sdmac)
+static int sdma_request_channel0(struct sdma_engine *sdma)
{
- struct sdma_engine *sdma = sdmac->sdma;
- struct sdma_desc *desc;
- int channel = sdmac->channel;
int ret = -EBUSY;
- sdmac->desc = &sdmac->_desc;
- desc = sdmac->desc;
-
- desc->bd = dma_zalloc_coherent(NULL, PAGE_SIZE, &desc->bd_phys,
- GFP_KERNEL);
- if (!desc->bd) {
+ sdma->bd0 = dma_zalloc_coherent(NULL, PAGE_SIZE, &sdma->bd0_phys,
+ GFP_NOWAIT);
+ if (!sdma->bd0) {
ret = -ENOMEM;
goto out;
}
- sdma->channel_control[channel].base_bd_ptr = desc->bd_phys;
- sdma->channel_control[channel].current_bd_ptr = desc->bd_phys;
+ sdma->channel_control[0].base_bd_ptr = sdma->bd0_phys;
+ sdma->channel_control[0].current_bd_ptr = sdma->bd0_phys;
- sdma_set_channel_priority(sdmac, MXC_SDMA_DEFAULT_PRIORITY);
+ sdma_set_channel_priority(&sdma->channel[0], MXC_SDMA_DEFAULT_PRIORITY);
return 0;
out:
return ret;
}
-static dma_cookie_t sdma_tx_submit(struct dma_async_tx_descriptor *tx)
+
+static int sdma_alloc_bd(struct sdma_desc *desc)
{
- unsigned long flags;
- struct sdma_channel *sdmac = to_sdma_chan(tx->chan);
- dma_cookie_t cookie;
+ u32 bd_size = desc->num_bd * sizeof(struct sdma_buffer_descriptor);
+ int ret = 0;
- spin_lock_irqsave(&sdmac->lock, flags);
+ desc->bd = dma_zalloc_coherent(NULL, bd_size, &desc->bd_phys,
+ GFP_ATOMIC);
+ if (!desc->bd) {
+ ret = -ENOMEM;
+ goto out;
+ }
+out:
+ return ret;
+}
- cookie = dma_cookie_assign(tx);
+static void sdma_free_bd(struct sdma_desc *desc)
+{
+ u32 bd_size = desc->num_bd * sizeof(struct sdma_buffer_descriptor);
- spin_unlock_irqrestore(&sdmac->lock, flags);
+ dma_free_coherent(NULL, bd_size, desc->bd, desc->bd_phys);
+}
- return cookie;
+static void sdma_desc_free(struct virt_dma_desc *vd)
+{
+ struct sdma_desc *desc = container_of(vd, struct sdma_desc, vd);
+
+ sdma_free_bd(desc);
+ kfree(desc);
}
static int sdma_alloc_chan_resources(struct dma_chan *chan)
@@ -1184,19 +1237,10 @@ static int sdma_alloc_chan_resources(struct dma_chan *chan)
if (ret)
goto disable_clk_ipg;
- ret = sdma_request_channel(sdmac);
- if (ret)
- goto disable_clk_ahb;
-
ret = sdma_set_channel_priority(sdmac, prio);
if (ret)
goto disable_clk_ahb;
- dma_async_tx_descriptor_init(&sdmac->txdesc, chan);
- sdmac->txdesc.tx_submit = sdma_tx_submit;
- /* txd.flags will be overwritten in prep funcs */
- sdmac->txdesc.flags = DMA_CTRL_ACK;
-
return 0;
disable_clk_ahb:
@@ -1210,9 +1254,8 @@ static void sdma_free_chan_resources(struct dma_chan *chan)
{
struct sdma_channel *sdmac = to_sdma_chan(chan);
struct sdma_engine *sdma = sdmac->sdma;
- struct sdma_desc *desc = sdmac->desc;
- sdma_disable_channel(chan);
+ sdma_disable_channel_with_delay(chan);
if (sdmac->event_id0)
sdma_event_disable(sdmac, sdmac->event_id0);
@@ -1224,8 +1267,6 @@ static void sdma_free_chan_resources(struct dma_chan *chan)
sdma_set_channel_priority(sdmac, 0);
- dma_free_coherent(NULL, PAGE_SIZE, desc->bd, desc->bd_phys);
-
clk_disable(sdma->clk_ipg);
clk_disable(sdma->clk_ahb);
}
@@ -1240,7 +1281,7 @@ static struct dma_async_tx_descriptor *sdma_prep_slave_sg(
int ret, i, count;
int channel = sdmac->channel;
struct scatterlist *sg;
- struct sdma_desc *desc = sdmac->desc;
+ struct sdma_desc *desc;
if (sdmac->status == DMA_IN_PROGRESS)
return NULL;
@@ -1248,23 +1289,34 @@ static struct dma_async_tx_descriptor *sdma_prep_slave_sg(
sdmac->flags = 0;
+ desc = kzalloc((sizeof(*desc)), GFP_NOWAIT);
+ if (!desc)
+ goto err_out;
+
desc->buf_tail = 0;
desc->buf_ptail = 0;
+ desc->sdmac = sdmac;
+ desc->num_bd = sg_len;
desc->chn_real_count = 0;
+ if (sdma_alloc_bd(desc)) {
+ kfree(desc);
+ goto err_out;
+ }
+
dev_dbg(sdma->dev, "setting up %d entries for channel %d.\n",
sg_len, channel);
sdmac->direction = direction;
ret = sdma_load_context(sdmac);
if (ret)
- goto err_out;
+ goto err_bd_out;
if (sg_len > NUM_BD) {
dev_err(sdma->dev, "SDMA channel %d: maximum number of sg exceeded: %d > %d\n",
channel, sg_len, NUM_BD);
ret = -EINVAL;
- goto err_out;
+ goto err_bd_out;
}
desc->chn_count = 0;
@@ -1280,7 +1332,7 @@ static struct dma_async_tx_descriptor *sdma_prep_slave_sg(
dev_err(sdma->dev, "SDMA channel %d: maximum bytes for sg entry exceeded: %d > %d\n",
channel, count, 0xffff);
ret = -EINVAL;
- goto err_out;
+ goto err_bd_out;
}
bd->mode.count = count;
@@ -1288,25 +1340,25 @@ static struct dma_async_tx_descriptor *sdma_prep_slave_sg(
if (sdmac->word_size > DMA_SLAVE_BUSWIDTH_4_BYTES) {
ret = -EINVAL;
- goto err_out;
+ goto err_bd_out;
}
switch (sdmac->word_size) {
case DMA_SLAVE_BUSWIDTH_4_BYTES:
bd->mode.command = 0;
if (count & 3 || sg->dma_address & 3)
- return NULL;
+ goto err_bd_out;
break;
case DMA_SLAVE_BUSWIDTH_2_BYTES:
bd->mode.command = 2;
if (count & 1 || sg->dma_address & 1)
- return NULL;
+ goto err_bd_out;
break;
case DMA_SLAVE_BUSWIDTH_1_BYTE:
bd->mode.command = 1;
break;
default:
- return NULL;
+ goto err_bd_out;
}
param = BD_DONE | BD_EXTD | BD_CONT;
@@ -1325,10 +1377,10 @@ static struct dma_async_tx_descriptor *sdma_prep_slave_sg(
bd->mode.status = param;
}
- desc->num_bd = sg_len;
- sdma->channel_control[channel].current_bd_ptr = desc->bd_phys;
-
- return &sdmac->txdesc;
+ return vchan_tx_prep(&sdmac->vc, &desc->vd, flags);
+err_bd_out:
+ sdma_free_bd(desc);
+ kfree(desc);
err_out:
sdmac->status = DMA_ERROR;
return NULL;
@@ -1344,7 +1396,7 @@ static struct dma_async_tx_descriptor *sdma_prep_dma_cyclic(
int num_periods = buf_len / period_len;
int channel = sdmac->channel;
int ret, i = 0, buf = 0;
- struct sdma_desc *desc = sdmac->desc;
+ struct sdma_desc *desc;
dev_dbg(sdma->dev, "%s channel: %d\n", __func__, channel);
@@ -1353,27 +1405,39 @@ static struct dma_async_tx_descriptor *sdma_prep_dma_cyclic(
sdmac->status = DMA_IN_PROGRESS;
+ desc = kzalloc((sizeof(*desc)), GFP_NOWAIT);
+ if (!desc)
+ goto err_out;
+
desc->buf_tail = 0;
desc->buf_ptail = 0;
+ desc->sdmac = sdmac;
+ desc->num_bd = num_periods;
desc->chn_real_count = 0;
desc->period_len = period_len;
sdmac->flags |= IMX_DMA_SG_LOOP;
sdmac->direction = direction;
+
+ if (sdma_alloc_bd(desc)) {
+ kfree(desc);
+ goto err_bd_out;
+ }
+
ret = sdma_load_context(sdmac);
if (ret)
- goto err_out;
+ goto err_bd_out;
if (num_periods > NUM_BD) {
dev_err(sdma->dev, "SDMA channel %d: maximum number of sg exceeded: %d > %d\n",
channel, num_periods, NUM_BD);
- goto err_out;
+ goto err_bd_out;
}
if (period_len > 0xffff) {
dev_err(sdma->dev, "SDMA channel %d: maximum period size exceeded: %zu > %d\n",
channel, period_len, 0xffff);
- goto err_out;
+ goto err_bd_out;
}
while (buf < buf_len) {
@@ -1385,7 +1449,7 @@ static struct dma_async_tx_descriptor *sdma_prep_dma_cyclic(
bd->mode.count = period_len;
if (sdmac->word_size > DMA_SLAVE_BUSWIDTH_4_BYTES)
- goto err_out;
+ goto err_bd_out;
if (sdmac->word_size == DMA_SLAVE_BUSWIDTH_4_BYTES)
bd->mode.command = 0;
else
@@ -1408,10 +1472,10 @@ static struct dma_async_tx_descriptor *sdma_prep_dma_cyclic(
i++;
}
- desc->num_bd = num_periods;
- sdma->channel_control[channel].current_bd_ptr = desc->bd_phys;
-
- return &sdmac->txdesc;
+ return vchan_tx_prep(&sdmac->vc, &desc->vd, flags);
+err_bd_out:
+ sdma_free_bd(desc);
+ kfree(desc);
err_out:
sdmac->status = DMA_ERROR;
return NULL;
@@ -1450,14 +1514,31 @@ static enum dma_status sdma_tx_status(struct dma_chan *chan,
struct dma_tx_state *txstate)
{
struct sdma_channel *sdmac = to_sdma_chan(chan);
- struct sdma_desc *desc = sdmac->desc;
+ struct sdma_desc *desc;
u32 residue;
+ struct virt_dma_desc *vd;
+ enum dma_status ret;
+ unsigned long flags;
- if (sdmac->flags & IMX_DMA_SG_LOOP)
- residue = (desc->num_bd - desc->buf_ptail) *
- desc->period_len - desc->chn_real_count;
- else
- residue = desc->chn_count - desc->chn_real_count;
+ ret = dma_cookie_status(chan, cookie, txstate);
+ if (ret == DMA_COMPLETE || !txstate)
+ return ret;
+
+ spin_lock_irqsave(&sdmac->vc.lock, flags);
+ vd = vchan_find_desc(&sdmac->vc, cookie);
+ if (vd) {
+ desc = to_sdma_desc(&vd->tx);
+ if (sdmac->flags & IMX_DMA_SG_LOOP)
+ residue = (desc->num_bd - desc->buf_ptail) *
+ desc->period_len - desc->chn_real_count;
+ else
+ residue = desc->chn_count - desc->chn_real_count;
+ } else if (sdmac->desc && sdmac->desc->vd.tx.cookie == cookie) {
+ residue = sdmac->desc->chn_count - sdmac->desc->chn_real_count;
+ } else {
+ residue = 0;
+ }
+ spin_unlock_irqrestore(&sdmac->vc.lock, flags);
dma_set_tx_state(txstate, chan->completed_cookie, chan->cookie,
residue);
@@ -1468,10 +1549,12 @@ static enum dma_status sdma_tx_status(struct dma_chan *chan,
static void sdma_issue_pending(struct dma_chan *chan)
{
struct sdma_channel *sdmac = to_sdma_chan(chan);
- struct sdma_engine *sdma = sdmac->sdma;
+ unsigned long flags;
- if (sdmac->status == DMA_IN_PROGRESS)
- sdma_enable_channel(sdma, sdmac->channel);
+ spin_lock_irqsave(&sdmac->vc.lock, flags);
+ if (vchan_issue_pending(&sdmac->vc) && !sdmac->desc)
+ sdma_start_desc(sdmac);
+ spin_unlock_irqrestore(&sdmac->vc.lock, flags);
}
#define SDMA_SCRIPT_ADDRS_ARRAY_SIZE_V1 34
@@ -1677,12 +1760,10 @@ static int sdma_init(struct sdma_engine *sdma)
for (i = 0; i < MAX_DMA_CHANNELS; i++)
writel_relaxed(0, sdma->regs + SDMA_CHNPRI_0 + i * 4);
- ret = sdma_request_channel(&sdma->channel[0]);
+ ret = sdma_request_channel0(sdma);
if (ret)
goto err_dma_alloc;
- sdma->bd0 = sdma->channel[0].desc->bd;
-
sdma_config_ownership(&sdma->channel[0], false, true, false);
/* Set Command Channel (Channel Zero) */
@@ -1843,20 +1924,15 @@ static int sdma_probe(struct platform_device *pdev)
sdmac->sdma = sdma;
spin_lock_init(&sdmac->lock);
- sdmac->chan.device = &sdma->dma_device;
- dma_cookie_init(&sdmac->chan);
sdmac->channel = i;
-
- tasklet_init(&sdmac->tasklet, mxc_sdma_handle_channel_normal,
- (unsigned long) sdmac);
+ sdmac->vc.desc_free = sdma_desc_free;
/*
* Add the channel to the DMAC list. Do not add channel 0 though
* because we need it internally in the SDMA driver. This also means
* that channel 0 in dmaengine counting matches sdma channel 1.
*/
if (i)
- list_add_tail(&sdmac->chan.device_node,
- &sdma->dma_device.channels);
+ vchan_init(&sdmac->vc, &sdma->dma_device);
}
ret = sdma_init(sdma);
@@ -1961,7 +2037,8 @@ static int sdma_remove(struct platform_device *pdev)
for (i = 0; i < MAX_DMA_CHANNELS; i++) {
struct sdma_channel *sdmac = &sdma->channel[i];
- tasklet_kill(&sdmac->tasklet);
+ tasklet_kill(&sdmac->vc.task);
+ sdma_free_chan_resources(&sdmac->vc.chan);
}
platform_set_drvdata(pdev, NULL);
^ permalink raw reply related
* [v5,2/7] dmaengine: imx-sdma: factor out a struct sdma_desc from struct sdma_channel
From: Robin Gong @ 2018-06-19 16:56 UTC (permalink / raw)
To: vkoul, s.hauer, l.stach, dan.j.williams, gregkh, jslaby
Cc: linux-serial, dmaengine, linux-kernel, linux-arm-kernel,
linux-imx
From: Sascha Hauer <s.hauer@pengutronix.de>
This is a preparation step to make the adding of virt-dma easier.
We create a struct sdma_desc, move some fields from struct sdma_channel
there and add a pointer from the former to the latter. For now we
allocate the data statically in struct sdma_channel, but with
virt-dma support it will be dynamically allocated.
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Robin Gong <yibin.gong@nxp.com>
---
drivers/dma/imx-sdma.c | 137 ++++++++++++++++++++++++++++++-------------------
1 file changed, 83 insertions(+), 54 deletions(-)
diff --git a/drivers/dma/imx-sdma.c b/drivers/dma/imx-sdma.c
index f0779926..19c351f 100644
--- a/drivers/dma/imx-sdma.c
+++ b/drivers/dma/imx-sdma.c
@@ -289,6 +289,30 @@ struct sdma_context_data {
struct sdma_engine;
/**
+ * struct sdma_desc - descriptor structor for one transfer
+ * @vd descriptor for virt dma
+ * @num_bd max NUM_BD. number of descriptors currently handling
+ * @buf_tail ID of the buffer that was processed
+ * @buf_ptail ID of the previous buffer that was processed
+ * @period_len period length, used in cyclic.
+ * @chn_real_count the real count updated from bd->mode.count
+ * @chn_count the transfer count setuped
+ * @sdmac sdma_channel pointer
+ * @bd pointer of alloced bd
+ */
+struct sdma_desc {
+ unsigned int num_bd;
+ dma_addr_t bd_phys;
+ unsigned int buf_tail;
+ unsigned int buf_ptail;
+ unsigned int period_len;
+ unsigned int chn_real_count;
+ unsigned int chn_count;
+ struct sdma_channel *sdmac;
+ struct sdma_buffer_descriptor *bd;
+};
+
+/**
* struct sdma_channel - housekeeping for a SDMA channel
*
* @sdma pointer to the SDMA engine for this channel
@@ -298,11 +322,10 @@ struct sdma_engine;
* @event_id0 aka dma request line
* @event_id1 for channels that use 2 events
* @word_size peripheral access size
- * @buf_tail ID of the buffer that was processed
- * @buf_ptail ID of the previous buffer that was processed
- * @num_bd max NUM_BD. number of descriptors currently handling
*/
struct sdma_channel {
+ struct sdma_desc *desc;
+ struct sdma_desc _desc;
struct sdma_engine *sdma;
unsigned int channel;
enum dma_transfer_direction direction;
@@ -310,12 +333,6 @@ struct sdma_channel {
unsigned int event_id0;
unsigned int event_id1;
enum dma_slave_buswidth word_size;
- unsigned int buf_tail;
- unsigned int buf_ptail;
- unsigned int num_bd;
- unsigned int period_len;
- struct sdma_buffer_descriptor *bd;
- dma_addr_t bd_phys;
unsigned int pc_from_device, pc_to_device;
unsigned int device_to_device;
unsigned long flags;
@@ -325,10 +342,8 @@ struct sdma_channel {
u32 shp_addr, per_addr;
struct dma_chan chan;
spinlock_t lock;
- struct dma_async_tx_descriptor desc;
+ struct dma_async_tx_descriptor txdesc;
enum dma_status status;
- unsigned int chn_count;
- unsigned int chn_real_count;
struct tasklet_struct tasklet;
struct imx_dma_data data;
bool enabled;
@@ -391,6 +406,8 @@ struct sdma_engine {
u32 spba_start_addr;
u32 spba_end_addr;
unsigned int irq;
+ dma_addr_t bd0_phys;
+ struct sdma_buffer_descriptor *bd0;
};
static struct sdma_driver_data sdma_imx31 = {
@@ -625,7 +642,7 @@ static int sdma_run_channel0(struct sdma_engine *sdma)
static int sdma_load_script(struct sdma_engine *sdma, void *buf, int size,
u32 address)
{
- struct sdma_buffer_descriptor *bd0 = sdma->channel[0].bd;
+ struct sdma_buffer_descriptor *bd0 = sdma->bd0;
void *buf_virt;
dma_addr_t buf_phys;
int ret;
@@ -700,7 +717,9 @@ static void sdma_update_channel_loop(struct sdma_channel *sdmac)
* call callback function.
*/
while (1) {
- bd = &sdmac->bd[sdmac->buf_tail];
+ struct sdma_desc *desc = sdmac->desc;
+
+ bd = &desc->bd[desc->buf_tail];
if (bd->mode.status & BD_DONE)
break;
@@ -716,11 +735,11 @@ static void sdma_update_channel_loop(struct sdma_channel *sdmac)
* the number of bytes present in the current buffer descriptor.
*/
- sdmac->chn_real_count = bd->mode.count;
+ desc->chn_real_count = bd->mode.count;
bd->mode.status |= BD_DONE;
- bd->mode.count = sdmac->period_len;
- sdmac->buf_ptail = sdmac->buf_tail;
- sdmac->buf_tail = (sdmac->buf_tail + 1) % sdmac->num_bd;
+ bd->mode.count = desc->period_len;
+ desc->buf_ptail = desc->buf_tail;
+ desc->buf_tail = (desc->buf_tail + 1) % desc->num_bd;
/*
* The callback is called from the interrupt context in order
@@ -729,7 +748,7 @@ static void sdma_update_channel_loop(struct sdma_channel *sdmac)
* executed.
*/
- dmaengine_desc_get_callback_invoke(&sdmac->desc, NULL);
+ dmaengine_desc_get_callback_invoke(&sdmac->txdesc, NULL);
if (error)
sdmac->status = old_status;
@@ -742,17 +761,17 @@ static void mxc_sdma_handle_channel_normal(unsigned long data)
struct sdma_buffer_descriptor *bd;
int i, error = 0;
- sdmac->chn_real_count = 0;
+ sdmac->desc->chn_real_count = 0;
/*
* non loop mode. Iterate over all descriptors, collect
* errors and call callback function
*/
- for (i = 0; i < sdmac->num_bd; i++) {
- bd = &sdmac->bd[i];
+ for (i = 0; i < sdmac->desc->num_bd; i++) {
+ bd = &sdmac->desc->bd[i];
if (bd->mode.status & (BD_DONE | BD_RROR))
error = -EIO;
- sdmac->chn_real_count += bd->mode.count;
+ sdmac->desc->chn_real_count += bd->mode.count;
}
if (error)
@@ -760,9 +779,9 @@ static void mxc_sdma_handle_channel_normal(unsigned long data)
else
sdmac->status = DMA_COMPLETE;
- dma_cookie_complete(&sdmac->desc);
+ dma_cookie_complete(&sdmac->txdesc);
- dmaengine_desc_get_callback_invoke(&sdmac->desc, NULL);
+ dmaengine_desc_get_callback_invoke(&sdmac->txdesc, NULL);
}
static irqreturn_t sdma_int_handler(int irq, void *dev_id)
@@ -890,7 +909,7 @@ static int sdma_load_context(struct sdma_channel *sdmac)
int channel = sdmac->channel;
int load_address;
struct sdma_context_data *context = sdma->context;
- struct sdma_buffer_descriptor *bd0 = sdma->channel[0].bd;
+ struct sdma_buffer_descriptor *bd0 = sdma->bd0;
int ret;
unsigned long flags;
@@ -1093,18 +1112,22 @@ static int sdma_set_channel_priority(struct sdma_channel *sdmac,
static int sdma_request_channel(struct sdma_channel *sdmac)
{
struct sdma_engine *sdma = sdmac->sdma;
+ struct sdma_desc *desc;
int channel = sdmac->channel;
int ret = -EBUSY;
- sdmac->bd = dma_zalloc_coherent(NULL, PAGE_SIZE, &sdmac->bd_phys,
+ sdmac->desc = &sdmac->_desc;
+ desc = sdmac->desc;
+
+ desc->bd = dma_zalloc_coherent(NULL, PAGE_SIZE, &desc->bd_phys,
GFP_KERNEL);
- if (!sdmac->bd) {
+ if (!desc->bd) {
ret = -ENOMEM;
goto out;
}
- sdma->channel_control[channel].base_bd_ptr = sdmac->bd_phys;
- sdma->channel_control[channel].current_bd_ptr = sdmac->bd_phys;
+ sdma->channel_control[channel].base_bd_ptr = desc->bd_phys;
+ sdma->channel_control[channel].current_bd_ptr = desc->bd_phys;
sdma_set_channel_priority(sdmac, MXC_SDMA_DEFAULT_PRIORITY);
return 0;
@@ -1169,10 +1192,10 @@ static int sdma_alloc_chan_resources(struct dma_chan *chan)
if (ret)
goto disable_clk_ahb;
- dma_async_tx_descriptor_init(&sdmac->desc, chan);
- sdmac->desc.tx_submit = sdma_tx_submit;
+ dma_async_tx_descriptor_init(&sdmac->txdesc, chan);
+ sdmac->txdesc.tx_submit = sdma_tx_submit;
/* txd.flags will be overwritten in prep funcs */
- sdmac->desc.flags = DMA_CTRL_ACK;
+ sdmac->txdesc.flags = DMA_CTRL_ACK;
return 0;
@@ -1187,6 +1210,7 @@ static void sdma_free_chan_resources(struct dma_chan *chan)
{
struct sdma_channel *sdmac = to_sdma_chan(chan);
struct sdma_engine *sdma = sdmac->sdma;
+ struct sdma_desc *desc = sdmac->desc;
sdma_disable_channel(chan);
@@ -1200,7 +1224,7 @@ static void sdma_free_chan_resources(struct dma_chan *chan)
sdma_set_channel_priority(sdmac, 0);
- dma_free_coherent(NULL, PAGE_SIZE, sdmac->bd, sdmac->bd_phys);
+ dma_free_coherent(NULL, PAGE_SIZE, desc->bd, desc->bd_phys);
clk_disable(sdma->clk_ipg);
clk_disable(sdma->clk_ahb);
@@ -1216,6 +1240,7 @@ static struct dma_async_tx_descriptor *sdma_prep_slave_sg(
int ret, i, count;
int channel = sdmac->channel;
struct scatterlist *sg;
+ struct sdma_desc *desc = sdmac->desc;
if (sdmac->status == DMA_IN_PROGRESS)
return NULL;
@@ -1223,9 +1248,9 @@ static struct dma_async_tx_descriptor *sdma_prep_slave_sg(
sdmac->flags = 0;
- sdmac->buf_tail = 0;
- sdmac->buf_ptail = 0;
- sdmac->chn_real_count = 0;
+ desc->buf_tail = 0;
+ desc->buf_ptail = 0;
+ desc->chn_real_count = 0;
dev_dbg(sdma->dev, "setting up %d entries for channel %d.\n",
sg_len, channel);
@@ -1242,9 +1267,9 @@ static struct dma_async_tx_descriptor *sdma_prep_slave_sg(
goto err_out;
}
- sdmac->chn_count = 0;
+ desc->chn_count = 0;
for_each_sg(sgl, sg, sg_len, i) {
- struct sdma_buffer_descriptor *bd = &sdmac->bd[i];
+ struct sdma_buffer_descriptor *bd = &desc->bd[i];
int param;
bd->buffer_addr = sg->dma_address;
@@ -1259,7 +1284,7 @@ static struct dma_async_tx_descriptor *sdma_prep_slave_sg(
}
bd->mode.count = count;
- sdmac->chn_count += count;
+ desc->chn_count += count;
if (sdmac->word_size > DMA_SLAVE_BUSWIDTH_4_BYTES) {
ret = -EINVAL;
@@ -1300,10 +1325,10 @@ static struct dma_async_tx_descriptor *sdma_prep_slave_sg(
bd->mode.status = param;
}
- sdmac->num_bd = sg_len;
- sdma->channel_control[channel].current_bd_ptr = sdmac->bd_phys;
+ desc->num_bd = sg_len;
+ sdma->channel_control[channel].current_bd_ptr = desc->bd_phys;
- return &sdmac->desc;
+ return &sdmac->txdesc;
err_out:
sdmac->status = DMA_ERROR;
return NULL;
@@ -1319,6 +1344,7 @@ static struct dma_async_tx_descriptor *sdma_prep_dma_cyclic(
int num_periods = buf_len / period_len;
int channel = sdmac->channel;
int ret, i = 0, buf = 0;
+ struct sdma_desc *desc = sdmac->desc;
dev_dbg(sdma->dev, "%s channel: %d\n", __func__, channel);
@@ -1327,10 +1353,10 @@ static struct dma_async_tx_descriptor *sdma_prep_dma_cyclic(
sdmac->status = DMA_IN_PROGRESS;
- sdmac->buf_tail = 0;
- sdmac->buf_ptail = 0;
- sdmac->chn_real_count = 0;
- sdmac->period_len = period_len;
+ desc->buf_tail = 0;
+ desc->buf_ptail = 0;
+ desc->chn_real_count = 0;
+ desc->period_len = period_len;
sdmac->flags |= IMX_DMA_SG_LOOP;
sdmac->direction = direction;
@@ -1351,7 +1377,7 @@ static struct dma_async_tx_descriptor *sdma_prep_dma_cyclic(
}
while (buf < buf_len) {
- struct sdma_buffer_descriptor *bd = &sdmac->bd[i];
+ struct sdma_buffer_descriptor *bd = &desc->bd[i];
int param;
bd->buffer_addr = dma_addr;
@@ -1382,10 +1408,10 @@ static struct dma_async_tx_descriptor *sdma_prep_dma_cyclic(
i++;
}
- sdmac->num_bd = num_periods;
- sdma->channel_control[channel].current_bd_ptr = sdmac->bd_phys;
+ desc->num_bd = num_periods;
+ sdma->channel_control[channel].current_bd_ptr = desc->bd_phys;
- return &sdmac->desc;
+ return &sdmac->txdesc;
err_out:
sdmac->status = DMA_ERROR;
return NULL;
@@ -1424,13 +1450,14 @@ static enum dma_status sdma_tx_status(struct dma_chan *chan,
struct dma_tx_state *txstate)
{
struct sdma_channel *sdmac = to_sdma_chan(chan);
+ struct sdma_desc *desc = sdmac->desc;
u32 residue;
if (sdmac->flags & IMX_DMA_SG_LOOP)
- residue = (sdmac->num_bd - sdmac->buf_ptail) *
- sdmac->period_len - sdmac->chn_real_count;
+ residue = (desc->num_bd - desc->buf_ptail) *
+ desc->period_len - desc->chn_real_count;
else
- residue = sdmac->chn_count - sdmac->chn_real_count;
+ residue = desc->chn_count - desc->chn_real_count;
dma_set_tx_state(txstate, chan->completed_cookie, chan->cookie,
residue);
@@ -1654,6 +1681,8 @@ static int sdma_init(struct sdma_engine *sdma)
if (ret)
goto err_dma_alloc;
+ sdma->bd0 = sdma->channel[0].desc->bd;
+
sdma_config_ownership(&sdma->channel[0], false, true, false);
/* Set Command Channel (Channel Zero) */
^ permalink raw reply related
* [v5,1/7] tty: serial: imx: correct dma cookie status
From: Robin Gong @ 2018-06-19 16:56 UTC (permalink / raw)
To: vkoul, s.hauer, l.stach, dan.j.williams, gregkh, jslaby
Cc: linux-serial, dmaengine, linux-kernel, linux-arm-kernel,
linux-imx
Correct to check the right rx dma cookie status in spit of it
works because only one cookie is running in the current sdma.
But it will not once sdma driver support multi cookies
running based on virt-dma.
Signed-off-by: Robin Gong <yibin.gong@nxp.com>
---
drivers/tty/serial/imx.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/tty/serial/imx.c b/drivers/tty/serial/imx.c
index 4e85357..2879407 100644
--- a/drivers/tty/serial/imx.c
+++ b/drivers/tty/serial/imx.c
@@ -1051,7 +1051,7 @@ static void imx_uart_dma_rx_callback(void *data)
unsigned int r_bytes;
unsigned int bd_size;
- status = dmaengine_tx_status(chan, (dma_cookie_t)0, &state);
+ status = dmaengine_tx_status(chan, sport->rx_cookie, &state);
if (status == DMA_ERROR) {
imx_uart_clear_rx_errors(sport);
^ permalink raw reply related
* dmaengine: pl330: report BURST residue granularity
From: Marek Szyprowski @ 2018-06-19 13:20 UTC (permalink / raw)
To: linux-samsung-soc, dmaengine, linux-arm-kernel, linux-kernel
Cc: Marek Szyprowski, Krzysztof Kozlowski, Bartlomiej Zolnierkiewicz,
Vinod Koul
The reported residue is already calculated in BURST unit granularity, so
advertise this capability properly to other devices in the system.
Fixes: aee4d1fac887 ("dmaengine: pl330: improve pl330_tx_status() function")
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
---
drivers/dma/pl330.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/dma/pl330.c b/drivers/dma/pl330.c
index defcdde4d358..de0957fe9668 100644
--- a/drivers/dma/pl330.c
+++ b/drivers/dma/pl330.c
@@ -3033,7 +3033,7 @@ pl330_probe(struct amba_device *adev, const struct amba_id *id)
pd->src_addr_widths = PL330_DMA_BUSWIDTHS;
pd->dst_addr_widths = PL330_DMA_BUSWIDTHS;
pd->directions = BIT(DMA_DEV_TO_MEM) | BIT(DMA_MEM_TO_DEV);
- pd->residue_granularity = DMA_RESIDUE_GRANULARITY_SEGMENT;
+ pd->residue_granularity = DMA_RESIDUE_GRANULARITY_BURST;
pd->max_burst = ((pl330->quirks & PL330_QUIRK_BROKEN_NO_FLUSHP) ?
1 : PL330_MAX_BURST);
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox