Re: [PATCH v2] dmaengine: fsl-edma: add PM suspend/resume support

Re: [PATCH v2] dmaengine: fsl-edma: add PM suspend/resume support

[Li Yang]

On Tue, Jul 21, 2015 at 3:56 AM, Yuan Yao <yao.yuan@freescale.com> wrote:
> This add power management suspend/resume support for the fsl-edma
> driver.
>
> eDMA acted as a basic function used by others. What it needs to do is
> the two steps below to support power management.
>
> In fsl_edma_suspend_late:
> Check whether the DMA chan is idle and if it is not idle, stop PM
> operation.

You should try to quiesce the device on suspend instead of depending
on itself to be happen in idle and failing if it is not.

Regards,
Leo

RE: [PATCH v2] dmaengine: fsl-edma: add PM suspend/resume support

[Yao Yuan]

Hi Leo,

Thanks for your review.
About those two methods for DMA suspend that you have mentioned. We have a lot of the discussions in other DMA driver like DMA for Freescale PowerPC.

Finally, we think the device which used the DMA transmission service should cancel the transmission service in its suspend. 
So DMA in suspend should be idle.

Once the DMA in late_suspend is not be idle, we think some driver haven't canceled the DMA transmission. So maybe something is error when other driver in suspend.

In the case, we should return failed to stop PM. DMA should not make a choice for other drivers(which used DMA) to force stop DMA transmission.

Thanks.

Best Regards,
Yuan Yao

> -----Original Message-----
> From: pku.leo@gmail.com [mailto:pku.leo@gmail.com] On Behalf Of Li Yang
> Sent: Friday, August 14, 2015 4:58 AM
> To: Yuan Yao-B46683
> Cc: Vinod Koul; stefan@agner.ch; Arnd Bergmann; Dan Williams;
> dmaengine@vger.kernel.org; lkml; linux-arm-kernel@lists.infradead.org; linux-
> pm@vger.kernel.org
> Subject: Re: [PATCH v2] dmaengine: fsl-edma: add PM suspend/resume
> support
> 
> On Tue, Jul 21, 2015 at 3:56 AM, Yuan Yao <yao.yuan@freescale.com> wrote:
> > This add power management suspend/resume support for the fsl-edma
> > driver.
> >
> > eDMA acted as a basic function used by others. What it needs to do is
> > the two steps below to support power management.
> >
> > In fsl_edma_suspend_late:
> > Check whether the DMA chan is idle and if it is not idle, stop PM
> > operation.
> 
> You should try to quiesce the device on suspend instead of depending on itself
> to be happen in idle and failing if it is not.
> 
> Regards,
> Leo

Re: [PATCH v2] dmaengine: fsl-edma: add PM suspend/resume support

[Li Yang]

On Fri, Aug 14, 2015 at 1:24 AM, Yao Yuan <yao.yuan@freescale.com> wrote:
> Hi Leo,
>
> Thanks for your review.
> About those two methods for DMA suspend that you have mentioned. We have a lot of the discussions in other DMA driver like DMA for Freescale PowerPC.
>
> Finally, we think the device which used the DMA transmission service should cancel the transmission service in its suspend.
> So DMA in suspend should be idle.

If that's the case you should clearly state this in the commit message
and in code, although I don't know if it is safe to make such
assumption.  There could be user of the DMA that doesn't track the
completion of transfers.

>
> Once the DMA in late_suspend is not be idle, we think some driver haven't canceled the DMA transmission. So maybe something is error when other driver in suspend.
>
> In the case, we should return failed to stop PM. DMA should not make a choice for other drivers(which used DMA) to force stop DMA transmission.

The suspend entrance should be terminated by wakeup events and only
critical issues.  I don't think we should just terminate the suspend
entrance just because having on-going I/O without even try to stop it.

>
> Thanks.
>
> Best Regards,
> Yuan Yao
>
>> -----Original Message-----
>> From: pku.leo@gmail.com [mailto:pku.leo@gmail.com] On Behalf Of Li Yang
>> Sent: Friday, August 14, 2015 4:58 AM
>> To: Yuan Yao-B46683
>> Cc: Vinod Koul; stefan@agner.ch; Arnd Bergmann; Dan Williams;
>> dmaengine@vger.kernel.org; lkml; linux-arm-kernel@lists.infradead.org; linux-
>> pm@vger.kernel.org
>> Subject: Re: [PATCH v2] dmaengine: fsl-edma: add PM suspend/resume
>> support
>>
>> On Tue, Jul 21, 2015 at 3:56 AM, Yuan Yao <yao.yuan@freescale.com> wrote:
>> > This add power management suspend/resume support for the fsl-edma
>> > driver.
>> >
>> > eDMA acted as a basic function used by others. What it needs to do is
>> > the two steps below to support power management.
>> >
>> > In fsl_edma_suspend_late:
>> > Check whether the DMA chan is idle and if it is not idle, stop PM
>> > operation.
>>
>> You should try to quiesce the device on suspend instead of depending on itself
>> to be happen in idle and failing if it is not.
>>
>> Regards,
>> Leo



-- 
- Leo

RE: [PATCH v2] dmaengine: fsl-edma: add PM suspend/resume support

[Yao Yuan]

On Sat, Aug 15, 2015 at 7:48 AM, pku.leo < pku.leo@gmail.com > wrote:
> On Fri, Aug 14, 2015 at 1:24 AM, Yao Yuan <yao.yuan@freescale.com> wrote:
> > Hi Leo,
> >
> > Thanks for your review.
> > About those two methods for DMA suspend that you have mentioned. We
> have a lot of the discussions in other DMA driver like DMA for Freescale
> PowerPC.
> >
> > Finally, we think the device which used the DMA transmission service should
> cancel the transmission service in its suspend.
> > So DMA in suspend should be idle.
> 
> If that's the case you should clearly state this in the commit message and in
> code, although I don't know if it is safe to make such assumption.  There could
> be user of the DMA that doesn't track the completion of transfers.

I think it should be safe. In my opinion, even some client(the user of the DMA) forget to cancel its DMA transmission,
It will just lead to PM failed but no other system and data risk.
Although we should first fix the behavior of the client.
Once you are no need the DMA transmission, why not stop it?

Is it right?

> >
> > Once the DMA in late_suspend is not be idle, we think some driver haven't
> canceled the DMA transmission. So maybe something is error when other
> driver in suspend.
> >
> > In the case, we should return failed to stop PM. DMA should not make a
> choice for other drivers(which used DMA) to force stop DMA transmission.
> 
> The suspend entrance should be terminated by wakeup events and only critical
> issues.  I don't think we should just terminate the suspend entrance just
> because having on-going I/O without even try to stop it.

The graceful behavior would be to for client to PAUSE or terminate and then suspend
followed by DMA suspend.
We need to rely on client doing the right thing here. 
The DMA should not make a decision instead of client.
If the DMA is not idle in DMA suspend, it should be the client's issue.
We don't know what the client really want to do, so just return the non-success value.

I'm not sure my description is clear.  So we may refer the discussion about the DMA PM support before.
Such as "DMA: Freescale: add suspend resume functions for DMA driver"

Like:https://lkml.org/lkml/2014/5/21/1

> >
> > Thanks.
> >
> > Best Regards,
> > Yuan Yao
> >
> >> -----Original Message-----
> >> From: pku.leo@gmail.com [mailto:pku.leo@gmail.com] On Behalf Of Li
> >> Yang
> >> Sent: Friday, August 14, 2015 4:58 AM
> >> To: Yuan Yao-B46683
> >> Cc: Vinod Koul; stefan@agner.ch; Arnd Bergmann; Dan Williams;
> >> dmaengine@vger.kernel.org; lkml;
> >> linux-arm-kernel@lists.infradead.org; linux- pm@vger.kernel.org
> >> Subject: Re: [PATCH v2] dmaengine: fsl-edma: add PM suspend/resume
> >> support
> >>
> >> On Tue, Jul 21, 2015 at 3:56 AM, Yuan Yao <yao.yuan@freescale.com> wrote:
> >> > This add power management suspend/resume support for the fsl-edma
> >> > driver.
> >> >
> >> > eDMA acted as a basic function used by others. What it needs to do
> >> > is the two steps below to support power management.
> >> >
> >> > In fsl_edma_suspend_late:
> >> > Check whether the DMA chan is idle and if it is not idle, stop PM
> >> > operation.
> >>
> >> You should try to quiesce the device on suspend instead of depending
> >> on itself to be happen in idle and failing if it is not.
> >>
> >> Regards,
> >> Leo
> 
> 
> 
> --
> - Leo

Re: [PATCH v2] dmaengine: fsl-edma: add PM suspend/resume support

[Nigel Cunningham]

Hi Yao.

On 17/08/15 13:59, Yao Yuan wrote:
> On Sat, Aug 15, 2015 at 7:48 AM, pku.leo < pku.leo@gmail.com > wrote:
>> On Fri, Aug 14, 2015 at 1:24 AM, Yao Yuan <yao.yuan@freescale.com> wrote:
>>> Hi Leo,
>>>
>>> Thanks for your review.
>>> About those two methods for DMA suspend that you have mentioned. We
>> have a lot of the discussions in other DMA driver like DMA for Freescale
>> PowerPC.
>>> Finally, we think the device which used the DMA transmission service should
>> cancel the transmission service in its suspend.
>>> So DMA in suspend should be idle.
>> If that's the case you should clearly state this in the commit message and in
>> code, although I don't know if it is safe to make such assumption.  There could
>> be user of the DMA that doesn't track the completion of transfers.
> I think it should be safe. In my opinion, even some client(the user of the DMA) forget to cancel its DMA transmission,
> It will just lead to PM failed but no other system and data risk.
> Although we should first fix the behavior of the client.
> Once you are no need the DMA transmission, why not stop it?
>
> Is it right?
Think of it from the end user perspective. Would you like your laptop (or whatever) to refuse to suspend because of this condition? The user may well expect that closing the lid on their laptop will reliably lead to it suspending to ram. Returning a failure here could result in a loss of data if the condition is not detected and the machine subsequently runs out of power.

I do agree that whatever is submitting DMA should be stopped first; ideally this driver would always be idle because whatever producers of work exist would already have been quiesced and output flushed.

Regards,

Nigel

RE: [PATCH v2] dmaengine: fsl-edma: add PM suspend/resume support

[Yao Yuan]

Hi Nigel,

On Mon, Aug 17, 2015 at 2:49 PM, Nigel Cunningham < nigel@nigelcunningham.com.au > wrote:
> On 17/08/15 13:59, Yao Yuan wrote:
> > On Sat, Aug 15, 2015 at 7:48 AM, pku.leo < pku.leo@gmail.com > wrote:
> >> On Fri, Aug 14, 2015 at 1:24 AM, Yao Yuan <yao.yuan@freescale.com> wrote:
> >>> Hi Leo,
> >>>
> >>> Thanks for your review.
> >>> About those two methods for DMA suspend that you have mentioned.
> We
> >> have a lot of the discussions in other DMA driver like DMA for
> >> Freescale PowerPC.
> >>> Finally, we think the device which used the DMA transmission service
> >>> should
> >> cancel the transmission service in its suspend.
> >>> So DMA in suspend should be idle.
> >> If that's the case you should clearly state this in the commit
> >> message and in code, although I don't know if it is safe to make such
> >> assumption.  There could be user of the DMA that doesn't track the
> completion of transfers.
> > I think it should be safe. In my opinion, even some client(the user of
> > the DMA) forget to cancel its DMA transmission, It will just lead to PM failed
> but no other system and data risk.
> > Although we should first fix the behavior of the client.
> > Once you are no need the DMA transmission, why not stop it?
> >
> > Is it right?
> Think of it from the end user perspective. Would you like your laptop (or
> whatever) to refuse to suspend because of this condition? The user may well
> expect that closing the lid on their laptop will reliably lead to it suspending to
> ram. Returning a failure here could result in a loss of data if the condition is not
> detected and the machine subsequently runs out of power.
> 

Yes, the user may well expect that closing the lid on their laptop will reliably lead to it suspending to ram.
So the client(the user of the DMA) must  to PAUSE or terminate the DMA transmission.

We need to rely on client doing the right thing here. 
The DMA should not make a decision instead of client.
If the DMA is not idle in DMA suspend, it should be the client's issue.
We don't know what the client really want to do, so just return the non-success value.

> I do agree that whatever is submitting DMA should be stopped first; ideally this
> driver would always be idle because whatever producers of work exist would
> already have been quiesced and output flushed.
> 

Re: [PATCH v2] dmaengine: fsl-edma: add PM suspend/resume support

[Li Yang]

On Mon, Aug 17, 2015 at 2:22 AM, Yao Yuan <yao.yuan@freescale.com> wrote:
> Hi Nigel,
>
> On Mon, Aug 17, 2015 at 2:49 PM, Nigel Cunningham < nigel@nigelcunningham.com.au > wrote:
>> On 17/08/15 13:59, Yao Yuan wrote:
>> > On Sat, Aug 15, 2015 at 7:48 AM, pku.leo < pku.leo@gmail.com > wrote:
>> >> On Fri, Aug 14, 2015 at 1:24 AM, Yao Yuan <yao.yuan@freescale.com> wrote:
>> >>> Hi Leo,
>> >>>
>> >>> Thanks for your review.
>> >>> About those two methods for DMA suspend that you have mentioned.
>> We
>> >> have a lot of the discussions in other DMA driver like DMA for
>> >> Freescale PowerPC.
>> >>> Finally, we think the device which used the DMA transmission service
>> >>> should
>> >> cancel the transmission service in its suspend.
>> >>> So DMA in suspend should be idle.
>> >> If that's the case you should clearly state this in the commit
>> >> message and in code, although I don't know if it is safe to make such
>> >> assumption.  There could be user of the DMA that doesn't track the
>> completion of transfers.
>> > I think it should be safe. In my opinion, even some client(the user of
>> > the DMA) forget to cancel its DMA transmission, It will just lead to PM failed
>> but no other system and data risk.
>> > Although we should first fix the behavior of the client.
>> > Once you are no need the DMA transmission, why not stop it?
>> >
>> > Is it right?
>> Think of it from the end user perspective. Would you like your laptop (or
>> whatever) to refuse to suspend because of this condition? The user may well
>> expect that closing the lid on their laptop will reliably lead to it suspending to
>> ram. Returning a failure here could result in a loss of data if the condition is not
>> detected and the machine subsequently runs out of power.
>>
>
> Yes, the user may well expect that closing the lid on their laptop will reliably lead to it suspending to ram.
> So the client(the user of the DMA) must  to PAUSE or terminate the DMA transmission.
>
> We need to rely on client doing the right thing here.
> The DMA should not make a decision instead of client.
> If the DMA is not idle in DMA suspend, it should be the client's issue.
> We don't know what the client really want to do, so just return the non-success value.

The problem here is that neither the client nor the DMA controller
driver should easily decide to stop the suspend entrance and rollback.
I don't think the non-idle situation is serious enough to cause a
rollback.  You should do whatever can be done with the DMA
controller(such as stop the controller and leave whatever to be done
to the wake up) and continue with the suspend.

Regards,
Leo

Re: [PATCH v2] dmaengine: fsl-edma: add PM suspend/resume support

[Vinod Koul]

On Mon, Aug 17, 2015 at 02:10:46PM -0500, Li Yang wrote:
> >> Think of it from the end user perspective. Would you like your laptop (or
> >> whatever) to refuse to suspend because of this condition? The user may well
> >> expect that closing the lid on their laptop will reliably lead to it suspending to
> >> ram. Returning a failure here could result in a loss of data if the condition is not
> >> detected and the machine subsequently runs out of power.
> >>
> >
> > Yes, the user may well expect that closing the lid on their laptop will reliably lead to it suspending to ram.
> > So the client(the user of the DMA) must  to PAUSE or terminate the DMA transmission.
> >
> > We need to rely on client doing the right thing here.
> > The DMA should not make a decision instead of client.
> > If the DMA is not idle in DMA suspend, it should be the client's issue.
> > We don't know what the client really want to do, so just return the non-success value.
> 
> The problem here is that neither the client nor the DMA controller
> driver should easily decide to stop the suspend entrance and rollback.
> I don't think the non-idle situation is serious enough to cause a
> rollback.  You should do whatever can be done with the DMA
> controller(such as stop the controller and leave whatever to be done
> to the wake up) and continue with the suspend.

Ideally yes client should suspend first and dmaengine driver then being idle
when suspend is invoked. But we know we are in idle world!
So, driver should ensure it suspends the active channels and then goes to
suspend and restores that on resume

-- 
~Vinod

RE: [PATCH v2] dmaengine: fsl-edma: add PM suspend/resume support

[Yao Yuan]

On Thu, Aug 20, 2015 at 12:08:00PM +0800, Vinod Koul wrote:
> On Mon, Aug 17, 2015 at 02:10:46PM -0500, Li Yang wrote:
> > >> Think of it from the end user perspective. Would you like your
> > >> laptop (or
> > >> whatever) to refuse to suspend because of this condition? The user
> > >> may well expect that closing the lid on their laptop will reliably
> > >> lead to it suspending to ram. Returning a failure here could result
> > >> in a loss of data if the condition is not detected and the machine
> subsequently runs out of power.
> > >>
> > >
> > > Yes, the user may well expect that closing the lid on their laptop will reliably
> lead to it suspending to ram.
> > > So the client(the user of the DMA) must  to PAUSE or terminate the DMA
> transmission.
> > >
> > > We need to rely on client doing the right thing here.
> > > The DMA should not make a decision instead of client.
> > > If the DMA is not idle in DMA suspend, it should be the client's issue.
> > > We don't know what the client really want to do, so just return the non-
> success value.
> >
> > The problem here is that neither the client nor the DMA controller
> > driver should easily decide to stop the suspend entrance and rollback.
> > I don't think the non-idle situation is serious enough to cause a
> > rollback.  You should do whatever can be done with the DMA
> > controller(such as stop the controller and leave whatever to be done
> > to the wake up) and continue with the suspend.
> 
> Ideally yes client should suspend first and dmaengine driver then being idle
> when suspend is invoked. But we know we are in idle world!
> So, driver should ensure it suspends the active channels and then goes to
> suspend and restores that on resume
> 

Hi Vinod, 
Hi Leo,

Is there any other discussions?

I think we can have the following solutions for DMA driver:
1, Force terminate the active channels in its suspend and then return.
2, DMA have to wait until the active channels idle.
3, Don't care about the active channels and direct return.
4, Return BUS BUSY error and stop PM progress.


I prefer the last one, because I think client should make sure the channel is idle. 

But also the first one should be works.

Can we have a conclusion and suggestion for which one is better?

Thanks.

RE: [PATCH v2] dmaengine: fsl-edma: add PM suspend/resume support

[Li Leo]

> On Thu, Aug 20, 2015 at 12:08:00PM +0800, Vinod Koul wrote:
> > On Mon, Aug 17, 2015 at 02:10:46PM -0500, Li Yang wrote:
> > > >> Think of it from the end user perspective. Would you like your
> > > >> laptop (or
> > > >> whatever) to refuse to suspend because of this condition? The
> > > >> user may well expect that closing the lid on their laptop will
> > > >> reliably lead to it suspending to ram. Returning a failure here
> > > >> could result in a loss of data if the condition is not detected
> > > >> and the machine
> > subsequently runs out of power.
> > > >>
> > > >
> > > > Yes, the user may well expect that closing the lid on their laptop
> > > > will reliably
> > lead to it suspending to ram.
> > > > So the client(the user of the DMA) must  to PAUSE or terminate the
> > > > DMA
> > transmission.
> > > >
> > > > We need to rely on client doing the right thing here.
> > > > The DMA should not make a decision instead of client.
> > > > If the DMA is not idle in DMA suspend, it should be the client's issue.
> > > > We don't know what the client really want to do, so just return
> > > > the non-
> > success value.
> > >
> > > The problem here is that neither the client nor the DMA controller
> > > driver should easily decide to stop the suspend entrance and rollback.
> > > I don't think the non-idle situation is serious enough to cause a
> > > rollback.  You should do whatever can be done with the DMA
> > > controller(such as stop the controller and leave whatever to be done
> > > to the wake up) and continue with the suspend.
> >
> > Ideally yes client should suspend first and dmaengine driver then
> > being idle when suspend is invoked. But we know we are in idle world!
> > So, driver should ensure it suspends the active channels and then goes
> > to suspend and restores that on resume
> >
> 
> Hi Vinod,
> Hi Leo,
> 
> Is there any other discussions?
> 
> I think we can have the following solutions for DMA driver:
> 1, Force terminate the active channels in its suspend and then return.
> 2, DMA have to wait until the active channels idle.
> 3, Don't care about the active channels and direct return.
> 4, Return BUS BUSY error and stop PM progress.
> 
> 
> I prefer the last one, because I think client should make sure the channel is idle.
> 
> But also the first one should be works.

I agree that client should be responsible for cleaning up the pending requests.  My concern is that we shouldn't be triggering rewind by returning error if any client failed to do so.  I would suggest to complain about it with a message and continue to do whatever we can do at the DMA driver level and continue with the suspend.  So I would prefer the solution one.

Regards,
Leo