[PATCH] dmaengine: rcar-dmac: Wait for IRQs completion when freeing channel

[PATCH] dmaengine: rcar-dmac: Wait for IRQs completion when freeing channel

[Laurent Pinchart]

When freeing channel resources we need to ensure that no IRQ will occur
for the given channel. In order to do so the driver stops the hardware,
but an IRQ handler could still be running, especially given that the
channel IRQ is threaded. Fix it by waiting for pending IRQs completion
with synchronize_irq()

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/dma/sh/rcar-dmac.c | 37 +++++++++++++++++++------------------
 1 file changed, 19 insertions(+), 18 deletions(-)

Hi Morimoto-san,

This patch is compile-tested only. Would you be able to check if it fixes the
issue you've reported with SDHI and rcar-dmac ?

diff --git a/drivers/dma/sh/rcar-dmac.c b/drivers/dma/sh/rcar-dmac.c
index 7820d07e7bee..da50c8f4d533 100644
--- a/drivers/dma/sh/rcar-dmac.c
+++ b/drivers/dma/sh/rcar-dmac.c
@@ -121,6 +121,7 @@ struct rcar_dmac_desc_page {
  * struct rcar_dmac_chan - R-Car Gen2 DMA Controller Channel
  * @chan: base DMA channel object
  * @iomem: channel I/O memory base
+ * @irq: channel IRQ number
  * @index: index of this channel in the controller
  * @src_xfer_size: size (in bytes) of hardware transfers on the source side
  * @dst_xfer_size: size (in bytes) of hardware transfers on the destination side
@@ -140,6 +141,7 @@ struct rcar_dmac_desc_page {
 struct rcar_dmac_chan {
 	struct dma_chan chan;
 	void __iomem *iomem;
+	int irq;
 	unsigned int index;
 
 	unsigned int src_xfer_size;
@@ -726,6 +728,15 @@ static void rcar_dmac_chan_halt(struct rcar_dmac_chan *chan)
 	rcar_dmac_chan_write(chan, RCAR_DMACHCR, chcr);
 }
 
+static void rcar_dmac_chan_halt_and_wait(struct rcar_dmac_chan *chan)
+{
+	spin_lock_irq(&chan->lock);
+	rcar_dmac_chan_halt(chan);
+	spin_unlock_irq(&chan->lock);
+
+	synchronize_irq(chan->irq);
+}
+
 static void rcar_dmac_chan_reinit(struct rcar_dmac_chan *chan)
 {
 	struct rcar_dmac_desc *desc, *_desc;
@@ -972,9 +983,7 @@ static void rcar_dmac_free_chan_resources(struct dma_chan *chan)
 	LIST_HEAD(list);
 
 	/* Protect against ISR */
-	spin_lock_irq(&rchan->lock);
-	rcar_dmac_chan_halt(rchan);
-	spin_unlock_irq(&rchan->lock);
+	rcar_dmac_chan_halt_and_wait(rchan);
 
 	/* Now no new interrupts will occur */
 
@@ -1121,17 +1130,8 @@ static int rcar_dmac_device_config(struct dma_chan *chan,
 static int rcar_dmac_chan_terminate_all(struct dma_chan *chan)
 {
 	struct rcar_dmac_chan *rchan = to_rcar_dmac_chan(chan);
-	unsigned long flags;
-
-	spin_lock_irqsave(&rchan->lock, flags);
-	rcar_dmac_chan_halt(rchan);
-	spin_unlock_irqrestore(&rchan->lock, flags);
-
-	/*
-	 * FIXME: No new interrupt can occur now, but the IRQ thread might still
-	 * be running.
-	 */
 
+	rcar_dmac_chan_halt_and_wait(rchan);
 	rcar_dmac_chan_reinit(rchan);
 
 	return 0;
@@ -1524,7 +1524,6 @@ static int rcar_dmac_chan_probe(struct rcar_dmac *dmac,
 	struct dma_chan *chan = &rchan->chan;
 	char pdev_irqname[5];
 	char *irqname;
-	int irq;
 	int ret;
 
 	rchan->index = index;
@@ -1541,8 +1540,8 @@ static int rcar_dmac_chan_probe(struct rcar_dmac *dmac,
 
 	/* Request the channel interrupt. */
 	sprintf(pdev_irqname, "ch%u", index);
-	irq = platform_get_irq_byname(pdev, pdev_irqname);
-	if (irq < 0) {
+	rchan->irq = platform_get_irq_byname(pdev, pdev_irqname);
+	if (rchan->irq < 0) {
 		dev_err(dmac->dev, "no IRQ specified for channel %u\n", index);
 		return -ENODEV;
 	}
@@ -1552,11 +1551,13 @@ static int rcar_dmac_chan_probe(struct rcar_dmac *dmac,
 	if (!irqname)
 		return -ENOMEM;
 
-	ret = devm_request_threaded_irq(dmac->dev, irq, rcar_dmac_isr_channel,
+	ret = devm_request_threaded_irq(dmac->dev, rchan->irq,
+					rcar_dmac_isr_channel,
 					rcar_dmac_isr_channel_thread, 0,
 					irqname, rchan);
 	if (ret) {
-		dev_err(dmac->dev, "failed to request IRQ %u (%d)\n", irq, ret);
+		dev_err(dmac->dev, "failed to request IRQ %u (%d)\n",
+			rchan->irq, ret);
 		return ret;
 	}
 
-- 
Regards,

Laurent Pinchart

Re: [PATCH] dmaengine: rcar-dmac: Wait for IRQs completion when freeing channel

[Kuninori Morimoto]

Hi Laurent

> When freeing channel resources we need to ensure that no IRQ will occur
> for the given channel. In order to do so the driver stops the hardware,
> but an IRQ handler could still be running, especially given that the
> channel IRQ is threaded. Fix it by waiting for pending IRQs completion
> with synchronize_irq()
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> ---
>  drivers/dma/sh/rcar-dmac.c | 37 +++++++++++++++++++------------------
>  1 file changed, 19 insertions(+), 18 deletions(-)
> 
> Hi Morimoto-san,
> 
> This patch is compile-tested only. Would you be able to check if it fixes the
> issue you've reported with SDHI and rcar-dmac ?

Thanks. I will ask to BSP team

Re: [PATCH] dmaengine: rcar-dmac: Wait for IRQs completion when freeing channel

[Vinod Koul]

On Tue, Sep 15, 2015 at 12:11:33AM +0000, Kuninori Morimoto wrote:
> 
> Hi Laurent
> 
> > When freeing channel resources we need to ensure that no IRQ will occur
> > for the given channel. In order to do so the driver stops the hardware,
> > but an IRQ handler could still be running, especially given that the
> > channel IRQ is threaded. Fix it by waiting for pending IRQs completion
> > with synchronize_irq()
> > 
> > Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> > ---
> >  drivers/dma/sh/rcar-dmac.c | 37 +++++++++++++++++++------------------
> >  1 file changed, 19 insertions(+), 18 deletions(-)
> > 
> > Hi Morimoto-san,
> > 
> > This patch is compile-tested only. Would you be able to check if it fixes the
> > issue you've reported with SDHI and rcar-dmac ?
> 
> Thanks. I will ask to BSP team

Patch looks okay, but if we get this confirmation it would be great.

-- 
~Vinod

Re: [PATCH] dmaengine: rcar-dmac: Wait for IRQs completion when freeing channel

[Laurent Pinchart]

Hi Vinod,

On Monday 05 October 2015 15:11:55 Vinod Koul wrote:
> On Tue, Sep 15, 2015 at 12:11:33AM +0000, Kuninori Morimoto wrote:
> > Hi Laurent
> > 
> > > When freeing channel resources we need to ensure that no IRQ will occur
> > > for the given channel. In order to do so the driver stops the hardware,
> > > but an IRQ handler could still be running, especially given that the
> > > channel IRQ is threaded. Fix it by waiting for pending IRQs completion
> > > with synchronize_irq()
> > > 
> > > Signed-off-by: Laurent Pinchart
> > > <laurent.pinchart+renesas@ideasonboard.com>
> > > ---
> > > 
> > >  drivers/dma/sh/rcar-dmac.c | 37 +++++++++++++++++++------------------
> > >  1 file changed, 19 insertions(+), 18 deletions(-)
> > > 
> > > Hi Morimoto-san,
> > > 
> > > This patch is compile-tested only. Would you be able to check if it
> > > fixes the issue you've reported with SDHI and rcar-dmac ?
> > 
> > Thanks. I will ask to BSP team
> 
> Patch looks okay, but if we get this confirmation it would be great.

I've just received a report that the patch has been successfully tested to get 
rid of the oops, but it introduced another issue in that synchronize_irq() 
gets called from non-sleeptable context through terminate_all(). 

The DMA engine API states that

   * device_terminate_all
     - Aborts all the pending and ongoing transfers on the channel
     - This command should operate synchronously on the channel,
       terminating right away all the channels

I wonder how to interpret "synchronously" here, should terminate_all() wait 
for termination to be complete ? In that case it wouldn't be valid to call it 
from non-sleepable context.

-- 
Regards,

Laurent Pinchart

Re: [PATCH] dmaengine: rcar-dmac: Wait for IRQs completion when freeing channel

[Lars-Peter Clausen]

On 10/08/2015 04:22 PM, Laurent Pinchart wrote:
> Hi Vinod,
> 
> On Monday 05 October 2015 15:11:55 Vinod Koul wrote:
>> On Tue, Sep 15, 2015 at 12:11:33AM +0000, Kuninori Morimoto wrote:
>>> Hi Laurent
>>>
>>>> When freeing channel resources we need to ensure that no IRQ will occur
>>>> for the given channel. In order to do so the driver stops the hardware,
>>>> but an IRQ handler could still be running, especially given that the
>>>> channel IRQ is threaded. Fix it by waiting for pending IRQs completion
>>>> with synchronize_irq()
>>>>
>>>> Signed-off-by: Laurent Pinchart
>>>> <laurent.pinchart+renesas@ideasonboard.com>
>>>> ---
>>>>
>>>>  drivers/dma/sh/rcar-dmac.c | 37 +++++++++++++++++++------------------
>>>>  1 file changed, 19 insertions(+), 18 deletions(-)
>>>>
>>>> Hi Morimoto-san,
>>>>
>>>> This patch is compile-tested only. Would you be able to check if it
>>>> fixes the issue you've reported with SDHI and rcar-dmac ?
>>>
>>> Thanks. I will ask to BSP team
>>
>> Patch looks okay, but if we get this confirmation it would be great.
> 
> I've just received a report that the patch has been successfully tested to get 
> rid of the oops, but it introduced another issue in that synchronize_irq() 
> gets called from non-sleeptable context through terminate_all(). 
> 
> The DMA engine API states that
> 
>    * device_terminate_all
>      - Aborts all the pending and ongoing transfers on the channel
>      - This command should operate synchronously on the channel,
>        terminating right away all the channels
> 
> I wonder how to interpret "synchronously" here, should terminate_all() wait 
> for termination to be complete ? In that case it wouldn't be valid to call it 
> from non-sleepable context.

We need to extend the DMAengine API to allow synchronization. The issue is
not only the IRQ itself but also the tasklet that can be scheduled from the
IRQ. Since we in some cases (e.g. audio underrun) call terminate_all() from
within the completion callback that runs in the in the tasklet we can't
synchronize to the tasklet in dmaengine_terminate_all(). We need a separate
API call to handle this. And then maybe have a helper like
dmaengine_terminate_all_sync() that terminates and synchronizes. And in
cases where terminate_all is called from a context where it can't
synchronize the new API needs to be called separately before freeing the
resources.

I've talked about this in the past here:
http://mailman.alsa-project.org/pipermail/alsa-devel/2013-October/067155.html

- Lars

Re: [PATCH] dmaengine: rcar-dmac: Wait for IRQs completion when freeing channel

[Vinod Koul]

On Thu, Oct 08, 2015 at 04:51:30PM +0200, Lars-Peter Clausen wrote:
> > The DMA engine API states that
> > 
> >    * device_terminate_all
> >      - Aborts all the pending and ongoing transfers on the channel
> >      - This command should operate synchronously on the channel,
> >        terminating right away all the channels
> > 
> > I wonder how to interpret "synchronously" here, should terminate_all() wait 
> > for termination to be complete ? In that case it wouldn't be valid to call it 
> > from non-sleepable context.
> 
> We need to extend the DMAengine API to allow synchronization. The issue is
> not only the IRQ itself but also the tasklet that can be scheduled from the
> IRQ. Since we in some cases (e.g. audio underrun) call terminate_all() from
> within the completion callback that runs in the in the tasklet we can't
> synchronize to the tasklet in dmaengine_terminate_all(). We need a separate
> API call to handle this. And then maybe have a helper like
> dmaengine_terminate_all_sync() that terminates and synchronizes. And in
> cases where terminate_all is called from a context where it can't
> synchronize the new API needs to be called separately before freeing the
> resources.

Right now the terminate_all() is intended for syncronous behaviour which
prevents it from being invoked in the callback.

I do agree that we should add separate call to abort the txn only and use
this for abort behaviour when we want to close down..

> 
> I've talked about this in the past here:
> http://mailman.alsa-project.org/pipermail/alsa-devel/2013-October/067155.html

-- 
~Vinod

Re: [PATCH] dmaengine: rcar-dmac: Wait for IRQs completion when freeing channel

[Lars-Peter Clausen]

On 10/14/2015 12:50 PM, Vinod Koul wrote:
> On Thu, Oct 08, 2015 at 04:51:30PM +0200, Lars-Peter Clausen wrote:
>>> The DMA engine API states that
>>>
>>>    * device_terminate_all
>>>      - Aborts all the pending and ongoing transfers on the channel
>>>      - This command should operate synchronously on the channel,
>>>        terminating right away all the channels
>>>
>>> I wonder how to interpret "synchronously" here, should terminate_all() wait 
>>> for termination to be complete ? In that case it wouldn't be valid to call it 
>>> from non-sleepable context.
>>
>> We need to extend the DMAengine API to allow synchronization. The issue is
>> not only the IRQ itself but also the tasklet that can be scheduled from the
>> IRQ. Since we in some cases (e.g. audio underrun) call terminate_all() from
>> within the completion callback that runs in the in the tasklet we can't
>> synchronize to the tasklet in dmaengine_terminate_all(). We need a separate
>> API call to handle this. And then maybe have a helper like
>> dmaengine_terminate_all_sync() that terminates and synchronizes. And in
>> cases where terminate_all is called from a context where it can't
>> synchronize the new API needs to be called separately before freeing the
>> resources.
> 
> Right now the terminate_all() is intended for syncronous behaviour which
> prevents it from being invoked in the callback.

That does not match reality though. Which means the documentation is wrong.
Pretty much all drivers implement a non-synchronous terminate function and
there are users that rely on this.

Re: [PATCH] dmaengine: rcar-dmac: Wait for IRQs completion when freeing channel

[Vinod Koul]

On Wed, Oct 14, 2015 at 01:02:22PM +0200, Lars-Peter Clausen wrote:
> On 10/14/2015 12:50 PM, Vinod Koul wrote:
> > On Thu, Oct 08, 2015 at 04:51:30PM +0200, Lars-Peter Clausen wrote:
> >>> The DMA engine API states that
> >>>
> >>>    * device_terminate_all
> >>>      - Aborts all the pending and ongoing transfers on the channel
> >>>      - This command should operate synchronously on the channel,
> >>>        terminating right away all the channels
> >>>
> >>> I wonder how to interpret "synchronously" here, should terminate_all() wait 
> >>> for termination to be complete ? In that case it wouldn't be valid to call it 
> >>> from non-sleepable context.
> >>
> >> We need to extend the DMAengine API to allow synchronization. The issue is
> >> not only the IRQ itself but also the tasklet that can be scheduled from the
> >> IRQ. Since we in some cases (e.g. audio underrun) call terminate_all() from
> >> within the completion callback that runs in the in the tasklet we can't
> >> synchronize to the tasklet in dmaengine_terminate_all(). We need a separate
> >> API call to handle this. And then maybe have a helper like
> >> dmaengine_terminate_all_sync() that terminates and synchronizes. And in
> >> cases where terminate_all is called from a context where it can't
> >> synchronize the new API needs to be called separately before freeing the
> >> resources.
> > 
> > Right now the terminate_all() is intended for syncronous behaviour which
> > prevents it from being invoked in the callback.
> 
> That does not match reality though. Which means the documentation is wrong.
> Pretty much all drivers implement a non-synchronous terminate function and
> there are users that rely on this.

Lets fix that then :)

We should have both option IMHO, as I think we have both types of usages...
Care to send a patch?

-- 
~Vinod

Re: [PATCH] dmaengine: rcar-dmac: Wait for IRQs completion when freeing channel

[Lars-Peter Clausen]

On 10/15/2015 05:56 AM, Vinod Koul wrote:
> On Wed, Oct 14, 2015 at 01:02:22PM +0200, Lars-Peter Clausen wrote:
>> On 10/14/2015 12:50 PM, Vinod Koul wrote:
>>> On Thu, Oct 08, 2015 at 04:51:30PM +0200, Lars-Peter Clausen wrote:
>>>>> The DMA engine API states that
>>>>>
>>>>>    * device_terminate_all
>>>>>      - Aborts all the pending and ongoing transfers on the channel
>>>>>      - This command should operate synchronously on the channel,
>>>>>        terminating right away all the channels
>>>>>
>>>>> I wonder how to interpret "synchronously" here, should terminate_all() wait 
>>>>> for termination to be complete ? In that case it wouldn't be valid to call it 
>>>>> from non-sleepable context.
>>>>
>>>> We need to extend the DMAengine API to allow synchronization. The issue is
>>>> not only the IRQ itself but also the tasklet that can be scheduled from the
>>>> IRQ. Since we in some cases (e.g. audio underrun) call terminate_all() from
>>>> within the completion callback that runs in the in the tasklet we can't
>>>> synchronize to the tasklet in dmaengine_terminate_all(). We need a separate
>>>> API call to handle this. And then maybe have a helper like
>>>> dmaengine_terminate_all_sync() that terminates and synchronizes. And in
>>>> cases where terminate_all is called from a context where it can't
>>>> synchronize the new API needs to be called separately before freeing the
>>>> resources.
>>>
>>> Right now the terminate_all() is intended for syncronous behaviour which
>>> prevents it from being invoked in the callback.
>>
>> That does not match reality though. Which means the documentation is wrong.
>> Pretty much all drivers implement a non-synchronous terminate function and
>> there are users that rely on this.
> 
> Lets fix that then :)
> 
> We should have both option IMHO, as I think we have both types of usages...
> Care to send a patch?
> 

Yeah, it's on my TODO list for the next month, since I need the synchronous
terminate elsewhere as well.

Re: [PATCH] dmaengine: rcar-dmac: Wait for IRQs completion when freeing channel

[Laurent Pinchart]

On Wednesday 14 October 2015 13:02:22 Lars-Peter Clausen wrote:
> On 10/14/2015 12:50 PM, Vinod Koul wrote:
> > On Thu, Oct 08, 2015 at 04:51:30PM +0200, Lars-Peter Clausen wrote:
> >>> The DMA engine API states that
> >>> 
> >>>    * device_terminate_all
> >>>    
> >>>      - Aborts all the pending and ongoing transfers on the channel
> >>>      - This command should operate synchronously on the channel,
> >>>      
> >>>        terminating right away all the channels
> >>> 
> >>> I wonder how to interpret "synchronously" here, should terminate_all()
> >>> wait for termination to be complete ? In that case it wouldn't be valid
> >>> to call it from non-sleepable context.
> >> 
> >> We need to extend the DMAengine API to allow synchronization. The issue
> >> is not only the IRQ itself but also the tasklet that can be scheduled
> >> from the IRQ. Since we in some cases (e.g. audio underrun) call
> >> terminate_all() from within the completion callback that runs in the in
> >> the tasklet we can't synchronize to the tasklet in
> >> dmaengine_terminate_all(). We need a separate API call to handle this.
> >> And then maybe have a helper like dmaengine_terminate_all_sync() that
> >> terminates and synchronizes. And in cases where terminate_all is called
> >> from a context where it can't synchronize the new API needs to be called
> >> separately before freeing the resources.
> > 
> > Right now the terminate_all() is intended for syncronous behaviour which
> > prevents it from being invoked in the callback.
> 
> That does not match reality though. Which means the documentation is wrong.
> Pretty much all drivers implement a non-synchronous terminate function and
> there are users that rely on this.

Most notably audio drivers seem to call terminate_all() from a non-sleepable 
context.

"We" (volunteers needed...) need to fix the API, the callers and the drivers. 
In the meantime I'll resubmit this patch without making terminate_all() 
synchronous to avoid introducing breakages.

-- 
Regards,

Laurent Pinchart

Re: [PATCH] dmaengine: rcar-dmac: Wait for IRQs completion when freeing channel

[Laurent Pinchart]

Hi Lars,

On Thursday 15 October 2015 09:35:31 Lars-Peter Clausen wrote:
> On 10/15/2015 05:56 AM, Vinod Koul wrote:
> > On Wed, Oct 14, 2015 at 01:02:22PM +0200, Lars-Peter Clausen wrote:
> >> On 10/14/2015 12:50 PM, Vinod Koul wrote:
> >>> On Thu, Oct 08, 2015 at 04:51:30PM +0200, Lars-Peter Clausen wrote:
> >>>>> The DMA engine API states that
> >>>>> 
> >>>>>    * device_terminate_all
> >>>>>    
> >>>>>      - Aborts all the pending and ongoing transfers on the channel
> >>>>>      - This command should operate synchronously on the channel,
> >>>>>        terminating right away all the channels
> >>>>> 
> >>>>> I wonder how to interpret "synchronously" here, should terminate_all()
> >>>>> wait for termination to be complete ? In that case it wouldn't be
> >>>>> valid to call it from non-sleepable context.
> >>>> 
> >>>> We need to extend the DMAengine API to allow synchronization. The issue
> >>>> is not only the IRQ itself but also the tasklet that can be scheduled
> >>>> from the IRQ. Since we in some cases (e.g. audio underrun) call
> >>>> terminate_all() from within the completion callback that runs in the in
> >>>> the tasklet we can't synchronize to the tasklet in
> >>>> dmaengine_terminate_all(). We need a separate API call to handle this.
> >>>> And then maybe have a helper like dmaengine_terminate_all_sync() that
> >>>> terminates and synchronizes. And in cases where terminate_all is called
> >>>> from a context where it can't synchronize the new API needs to be
> >>>> called separately before freeing the resources.
> >>> 
> >>> Right now the terminate_all() is intended for syncronous behaviour which
> >>> prevents it from being invoked in the callback.
> >> 
> >> That does not match reality though. Which means the documentation is
> >> wrong. Pretty much all drivers implement a non-synchronous terminate
> >> function and there are users that rely on this.
> > 
> > Lets fix that then :)
> > 
> > We should have both option IMHO, as I think we have both types of
> > usages... Care to send a patch?
> 
> Yeah, it's on my TODO list for the next month, since I need the synchronous
> terminate elsewhere as well.

The DMA engine API needs lots of love. Should we try to synchronize on all the 
open issues to define a coherent direction ?

-- 
Regards,

Laurent Pinchart

Re: [PATCH] dmaengine: rcar-dmac: Wait for IRQs completion when freeing channel

[Lars-Peter Clausen]

On 10/15/2015 07:04 PM, Laurent Pinchart wrote:
> On Wednesday 14 October 2015 13:02:22 Lars-Peter Clausen wrote:
>> On 10/14/2015 12:50 PM, Vinod Koul wrote:
>>> On Thu, Oct 08, 2015 at 04:51:30PM +0200, Lars-Peter Clausen wrote:
>>>>> The DMA engine API states that
>>>>>
>>>>>    * device_terminate_all
>>>>>    
>>>>>      - Aborts all the pending and ongoing transfers on the channel
>>>>>      - This command should operate synchronously on the channel,
>>>>>      
>>>>>        terminating right away all the channels
>>>>>
>>>>> I wonder how to interpret "synchronously" here, should terminate_all()
>>>>> wait for termination to be complete ? In that case it wouldn't be valid
>>>>> to call it from non-sleepable context.
>>>>
>>>> We need to extend the DMAengine API to allow synchronization. The issue
>>>> is not only the IRQ itself but also the tasklet that can be scheduled
>>>> from the IRQ. Since we in some cases (e.g. audio underrun) call
>>>> terminate_all() from within the completion callback that runs in the in
>>>> the tasklet we can't synchronize to the tasklet in
>>>> dmaengine_terminate_all(). We need a separate API call to handle this.
>>>> And then maybe have a helper like dmaengine_terminate_all_sync() that
>>>> terminates and synchronizes. And in cases where terminate_all is called
>>>> from a context where it can't synchronize the new API needs to be called
>>>> separately before freeing the resources.
>>>
>>> Right now the terminate_all() is intended for syncronous behaviour which
>>> prevents it from being invoked in the callback.
>>
>> That does not match reality though. Which means the documentation is wrong.
>> Pretty much all drivers implement a non-synchronous terminate function and
>> there are users that rely on this.
> 
> Most notably audio drivers seem to call terminate_all() from a non-sleepable 
> context.
> 
> "We" (volunteers needed...) need to fix the API, the callers and the drivers. 
> In the meantime I'll resubmit this patch without making terminate_all() 
> synchronous to avoid introducing breakages.

I've started working on it. The plan is to introduce a separate per driver
device_synchronize() callback which should make sure that all callbacks have
finished running. Then introduce a dmaengine_terminate_all_async() which
does what dmaengine_terminate_all() does at the moment. A user of this
function needs to make sure to call dmaengine_synchronize() before freeing
any memory accessed by the DMA transfer itself or in the complete callback.

That part is basically done and I've updated a few drivers where I have
access to a device.

The final step then is to make dmaengine_terminate_all() synchronous by
making it call both the device_terminate_all() and device_synchronize()
callback. This will require careful review of all existing
dmaengine_terminate_all() callers to make sure that they can handle
synchronization (which might sleep).

- Lars

Re: [PATCH] dmaengine: rcar-dmac: Wait for IRQs completion when freeing channel

[Lars-Peter Clausen]

On 10/15/2015 07:09 PM, Laurent Pinchart wrote:
> Hi Lars,
> 
> On Thursday 15 October 2015 09:35:31 Lars-Peter Clausen wrote:
>> On 10/15/2015 05:56 AM, Vinod Koul wrote:
>>> On Wed, Oct 14, 2015 at 01:02:22PM +0200, Lars-Peter Clausen wrote:
>>>> On 10/14/2015 12:50 PM, Vinod Koul wrote:
>>>>> On Thu, Oct 08, 2015 at 04:51:30PM +0200, Lars-Peter Clausen wrote:
>>>>>>> The DMA engine API states that
>>>>>>>
>>>>>>>    * device_terminate_all
>>>>>>>    
>>>>>>>      - Aborts all the pending and ongoing transfers on the channel
>>>>>>>      - This command should operate synchronously on the channel,
>>>>>>>        terminating right away all the channels
>>>>>>>
>>>>>>> I wonder how to interpret "synchronously" here, should terminate_all()
>>>>>>> wait for termination to be complete ? In that case it wouldn't be
>>>>>>> valid to call it from non-sleepable context.
>>>>>>
>>>>>> We need to extend the DMAengine API to allow synchronization. The issue
>>>>>> is not only the IRQ itself but also the tasklet that can be scheduled
>>>>>> from the IRQ. Since we in some cases (e.g. audio underrun) call
>>>>>> terminate_all() from within the completion callback that runs in the in
>>>>>> the tasklet we can't synchronize to the tasklet in
>>>>>> dmaengine_terminate_all(). We need a separate API call to handle this.
>>>>>> And then maybe have a helper like dmaengine_terminate_all_sync() that
>>>>>> terminates and synchronizes. And in cases where terminate_all is called
>>>>>> from a context where it can't synchronize the new API needs to be
>>>>>> called separately before freeing the resources.
>>>>>
>>>>> Right now the terminate_all() is intended for syncronous behaviour which
>>>>> prevents it from being invoked in the callback.
>>>>
>>>> That does not match reality though. Which means the documentation is
>>>> wrong. Pretty much all drivers implement a non-synchronous terminate
>>>> function and there are users that rely on this.
>>>
>>> Lets fix that then :)
>>>
>>> We should have both option IMHO, as I think we have both types of
>>> usages... Care to send a patch?
>>
>> Yeah, it's on my TODO list for the next month, since I need the synchronous
>> terminate elsewhere as well.
> 
> The DMA engine API needs lots of love. Should we try to synchronize on all the 
> open issues to define a coherent direction ?

Sure. Stuff I have on my wish list:
 * Better status reporting on completed transfers. Errors,
underflow/overflow, number of bytes transferred.
 * Support for making all transfers cyclic. Most controllers support this
but it is currently not exposed through the API. This is e.g. useful for
video (cyclic 2d transfers) as well as audio when we do not want to or can
not use a continuous buffer.

- Lars