[linux-pm Query] Power Management Device Suspend/ Resume

[linux-pm Query] Power Management Device Suspend/ Resume

[Viresh Kumar]

Hi,

Can somebody please help us understand suspend/resume.
It would be a great help, even if you can point us to some earlier fruitful
discussion.

--
viresh

On 9/22/2011 4:19 PM, Deepak Sikri wrote:
> Hi,
>  
> We are in the process of adding Suspend/ Resume call backs for the devices on SPEAr platform.
> The platform makes use of various power domains which could be turned off (during suspend, implying
> the devices in those power domains lose their context).
>  
> *There are certain queries*.
>  
> *Query- 1*. Suppose my platform has a ADC driver under char framework, which internally uses the
> DMA driver (channels). Now, ADC driver can be used by other kernel drivers or directly from user application.
>  
> *1. a.* What are the expectations from the suspend and resume routines of both the devices, DMA & ADC ?
>  
> */-- Given that few of the options are/*
> */1.a.1. On suspend,/*
> In ADC suspend function: ADC releases all the DMA channels, latches its registers; and
> DMA suspend function: DMA does nothing except for lataching of registers if required.
>  
> */1.a.2. On suspend, /*
> ADC suspend function: ADC just latches its registers and stops R/W through DMA, no DMA channel release;
> DMA suspend function: DMA halts its operating channels, and latches its registers if required.
>  
> */1.a.3 Some other alternative/*
> *//* 
>  
> *1.b* Will the suspend resume for the dependent drivers follow sequencing, i.e. Suspend of ADC followed by
> Suspend of DMA? (Assume both of these devices are hooked on to Platform bus, given that ADC uses DMA)
>  
> *Query-2*
>  
> 2.1 The user space threads freeze first followed by kernel space.  In case the user space process issues a a system
> call (lets assume an ioctl system call), how will the user space thread respond in case of suspend to ram?
>  
> 2.2 Will the kernel thread running on behalf of user space process complete the ioctl call OR 
>  freeze upon in the middle of execution?
>  
> *Query-3*
> ** 
> Are there any modalities w.r.t latching and restoring the contents at the driver level ?
> that is: Should this process be handled at drier level or platform specific code can also handle this
>  
>  
> Thanks & Regards
> Deepak

Re: [linux-pm Query] Power Management Device Suspend/ Resume

[Viresh Kumar]

Hi,

Can somebody please help us understand suspend/resume.
It would be a great help, even if you can point us to some earlier fruitful
discussion.

--
viresh

On 9/22/2011 4:19 PM, Deepak Sikri wrote:
> Hi,
>  
> We are in the process of adding Suspend/ Resume call backs for the devices on SPEAr platform.
> The platform makes use of various power domains which could be turned off (during suspend, implying
> the devices in those power domains lose their context).
>  
> *There are certain queries*.
>  
> *Query- 1*. Suppose my platform has a ADC driver under char framework, which internally uses the
> DMA driver (channels). Now, ADC driver can be used by other kernel drivers or directly from user application.
>  
> *1. a.* What are the expectations from the suspend and resume routines of both the devices, DMA & ADC ?
>  
> */-- Given that few of the options are/*
> */1.a.1. On suspend,/*
> In ADC suspend function: ADC releases all the DMA channels, latches its registers; and
> DMA suspend function: DMA does nothing except for lataching of registers if required.
>  
> */1.a.2. On suspend, /*
> ADC suspend function: ADC just latches its registers and stops R/W through DMA, no DMA channel release;
> DMA suspend function: DMA halts its operating channels, and latches its registers if required.
>  
> */1.a.3 Some other alternative/*
> *//* 
>  
> *1.b* Will the suspend resume for the dependent drivers follow sequencing, i.e. Suspend of ADC followed by
> Suspend of DMA? (Assume both of these devices are hooked on to Platform bus, given that ADC uses DMA)
>  
> *Query-2*
>  
> 2.1 The user space threads freeze first followed by kernel space.  In case the user space process issues a a system
> call (lets assume an ioctl system call), how will the user space thread respond in case of suspend to ram?
>  
> 2.2 Will the kernel thread running on behalf of user space process complete the ioctl call OR 
>  freeze upon in the middle of execution?
>  
> *Query-3*
> ** 
> Are there any modalities w.r.t latching and restoring the contents at the driver level ?
> that is: Should this process be handled at drier level or platform specific code can also handle this
>  
>  
> Thanks & Regards
> Deepak

[linux-pm Query] Power Management Device Suspend/ Resume

[Vinod Koul]

On Wed, 2011-09-28 at 08:55 +0530, Viresh Kumar wrote:
> Hi,
> 
> Can somebody please help us understand suspend/resume.
> It would be a great help, even if you can point us to some earlier fruitful
> discussion.
are you trying to do runtime power management or traditional suspend and
resume? IMO both should be done...
> 
> --
> viresh
> 
> On 9/22/2011 4:19 PM, Deepak Sikri wrote:
> > Hi,
> >  
> > We are in the process of adding Suspend/ Resume call backs for the devices on SPEAr platform.
> > The platform makes use of various power domains which could be turned off (during suspend, implying
> > the devices in those power domains lose their context).
> >  
> > *There are certain queries*.
> >  
> > *Query- 1*. Suppose my platform has a ADC driver under char framework, which internally uses the
> > DMA driver (channels). Now, ADC driver can be used by other kernel drivers or directly from user application.
> >  
> > *1. a.* What are the expectations from the suspend and resume routines of both the devices, DMA & ADC ?
> >  
> > */-- Given that few of the options are/*
> > */1.a.1. On suspend,/*
> > In ADC suspend function: ADC releases all the DMA channels, latches its registers; and
> > DMA suspend function: DMA does nothing except for lataching of registers if required.
> >  
> > */1.a.2. On suspend, /*
> > ADC suspend function: ADC just latches its registers and stops R/W through DMA, no DMA channel release;
> > DMA suspend function: DMA halts its operating channels, and latches its registers if required.
Well channel release/alloc is not really required. You can goto suspend
in between as well. if you have queued but not active descriptors on all
channels then dmac is idle and you can use this time to goto suspend.

> >  
> > */1.a.3 Some other alternative/*
> > *//* 
> >  
> > *1.b* Will the suspend resume for the dependent drivers follow sequencing, i.e. Suspend of ADC followed by
> > Suspend of DMA? (Assume both of these devices are hooked on to Platform bus, given that ADC uses DMA)
why do you care?
Each driver is independent and should manage its own resources properly
> >  
> > *Query-2*
> >  
> > 2.1 The user space threads freeze first followed by kernel space.  In case the user space process issues a a system
> > call (lets assume an ioctl system call), how will the user space thread respond in case of suspend to ram?
all user space processes are frozen on STR
> >  
> > 2.2 Will the kernel thread running on behalf of user space process complete the ioctl call OR 
> >  freeze upon in the middle of execution?
IIRC, It will be frozen

> >  
> > *Query-3*
> > ** 
> > Are there any modalities w.r.t latching and restoring the contents at the driver level ?
> > that is: Should this process be handled at drier level or platform specific code can also handle this
kernel context is not lost during suspend, so I don't think you need to
do much apart from resuming your device properly


-- 
~Vinod

Re: [linux-pm Query] Power Management Device Suspend/ Resume

[Vinod Koul]

On Wed, 2011-09-28 at 08:55 +0530, Viresh Kumar wrote:
> Hi,
> 
> Can somebody please help us understand suspend/resume.
> It would be a great help, even if you can point us to some earlier fruitful
> discussion.
are you trying to do runtime power management or traditional suspend and
resume? IMO both should be done...
> 
> --
> viresh
> 
> On 9/22/2011 4:19 PM, Deepak Sikri wrote:
> > Hi,
> >  
> > We are in the process of adding Suspend/ Resume call backs for the devices on SPEAr platform.
> > The platform makes use of various power domains which could be turned off (during suspend, implying
> > the devices in those power domains lose their context).
> >  
> > *There are certain queries*.
> >  
> > *Query- 1*. Suppose my platform has a ADC driver under char framework, which internally uses the
> > DMA driver (channels). Now, ADC driver can be used by other kernel drivers or directly from user application.
> >  
> > *1. a.* What are the expectations from the suspend and resume routines of both the devices, DMA & ADC ?
> >  
> > */-- Given that few of the options are/*
> > */1.a.1. On suspend,/*
> > In ADC suspend function: ADC releases all the DMA channels, latches its registers; and
> > DMA suspend function: DMA does nothing except for lataching of registers if required.
> >  
> > */1.a.2. On suspend, /*
> > ADC suspend function: ADC just latches its registers and stops R/W through DMA, no DMA channel release;
> > DMA suspend function: DMA halts its operating channels, and latches its registers if required.
Well channel release/alloc is not really required. You can goto suspend
in between as well. if you have queued but not active descriptors on all
channels then dmac is idle and you can use this time to goto suspend.

> >  
> > */1.a.3 Some other alternative/*
> > *//* 
> >  
> > *1.b* Will the suspend resume for the dependent drivers follow sequencing, i.e. Suspend of ADC followed by
> > Suspend of DMA? (Assume both of these devices are hooked on to Platform bus, given that ADC uses DMA)
why do you care?
Each driver is independent and should manage its own resources properly
> >  
> > *Query-2*
> >  
> > 2.1 The user space threads freeze first followed by kernel space.  In case the user space process issues a a system
> > call (lets assume an ioctl system call), how will the user space thread respond in case of suspend to ram?
all user space processes are frozen on STR
> >  
> > 2.2 Will the kernel thread running on behalf of user space process complete the ioctl call OR 
> >  freeze upon in the middle of execution?
IIRC, It will be frozen

> >  
> > *Query-3*
> > ** 
> > Are there any modalities w.r.t latching and restoring the contents at the driver level ?
> > that is: Should this process be handled at drier level or platform specific code can also handle this
kernel context is not lost during suspend, so I don't think you need to
do much apart from resuming your device properly


-- 
~Vinod

[linux-pm Query] Power Management Device Suspend/ Resume

[Viresh Kumar]

Vinod,

Thanks for mailing us back.

On 9/28/2011 9:12 AM, Vinod Koul wrote:
> On Wed, 2011-09-28 at 08:55 +0530, Viresh Kumar wrote:
>> On 9/22/2011 4:19 PM, Deepak Sikri wrote:
> are you trying to do runtime power management or traditional suspend and
> resume? IMO both should be done...

We will first be testing traditional suspend/resume, but obviously we would
try to design drivers in order to support runtime PM too.

>>> *Query- 1*. Suppose my platform has a ADC driver under char framework, which internally uses the
>>> DMA driver (channels). Now, ADC driver can be used by other kernel drivers or directly from user application.
>>>  
>>> *1. a.* What are the expectations from the suspend and resume routines of both the devices, DMA & ADC ?
>>>  
>>> */-- Given that few of the options are/*
>>> */1.a.1. On suspend,/*
>>> In ADC suspend function: ADC releases all the DMA channels, latches its registers; and
>>> DMA suspend function: DMA does nothing except for lataching of registers if required.
>>>  
>>> */1.a.2. On suspend, /*
>>> ADC suspend function: ADC just latches its registers and stops R/W through DMA, no DMA channel release;
>>> DMA suspend function: DMA halts its operating channels, and latches its registers if required.
> Well channel release/alloc is not really required. You can goto suspend
> in between as well. if you have queued but not active descriptors on all
> channels then dmac is idle and you can use this time to goto suspend.

But the time suspend is called for DMA, we may in middle of DMA transfers
on few channels. So should we wait for them to finish or simply disable
channel transfer? And in case of resume what will happen to transfers that
were active during suspend?

>>> *1.b* Will the suspend resume for the dependent drivers follow sequencing, i.e. Suspend of ADC followed by
>>> Suspend of DMA? (Assume both of these devices are hooked on to Platform bus, given that ADC uses DMA)
> why do you care?

Suppose suspend of DMA occurs first and we disable its clock in the suspend
routine. Now before the suspend of ADC is called, ADC requests for DMA
transfers and because not every routine of DMA keeps track of clk is on/off,
it may try to access DMA register.

So, we must ensure that all children are suspended before the parent is
requested to suspend. And probably this is the correct way.
So, here we need ADC to suspend before DMA. How is this sequencing ensured?

>>> 2.1 The user space threads freeze first followed by kernel space.  In case the user space process issues a a system
>>> call (lets assume an ioctl system call), how will the user space thread respond in case of suspend to ram?
> all user space processes are frozen on STR

Yes, but what will be its state. Will the system calls complete? Will it
close all devices opened by it before freezing?

Suppose a user applications is accessing few ADC channels. ADC controller
and its channel are all configured. Now, suspend is requested. What should
ADC driver do? As power will be turned off, its register will loose their
value.

Should ADC reconfigure all its channel registers by keeping their state?
Also it is not very straight forward to reconfigure channels directly, as
registers are required to be programmed in certain sequence, so there
needs to be good amount of intelligence put into driver for that.

IIRC, i couldn't find any such stuff in other mainline drivers. And that's
why we are confused on this topic.

-- 
viresh

Re: [linux-pm Query] Power Management Device Suspend/ Resume

[Viresh Kumar]

Vinod,

Thanks for mailing us back.

On 9/28/2011 9:12 AM, Vinod Koul wrote:
> On Wed, 2011-09-28 at 08:55 +0530, Viresh Kumar wrote:
>> On 9/22/2011 4:19 PM, Deepak Sikri wrote:
> are you trying to do runtime power management or traditional suspend and
> resume? IMO both should be done...

We will first be testing traditional suspend/resume, but obviously we would
try to design drivers in order to support runtime PM too.

>>> *Query- 1*. Suppose my platform has a ADC driver under char framework, which internally uses the
>>> DMA driver (channels). Now, ADC driver can be used by other kernel drivers or directly from user application.
>>>  
>>> *1. a.* What are the expectations from the suspend and resume routines of both the devices, DMA & ADC ?
>>>  
>>> */-- Given that few of the options are/*
>>> */1.a.1. On suspend,/*
>>> In ADC suspend function: ADC releases all the DMA channels, latches its registers; and
>>> DMA suspend function: DMA does nothing except for lataching of registers if required.
>>>  
>>> */1.a.2. On suspend, /*
>>> ADC suspend function: ADC just latches its registers and stops R/W through DMA, no DMA channel release;
>>> DMA suspend function: DMA halts its operating channels, and latches its registers if required.
> Well channel release/alloc is not really required. You can goto suspend
> in between as well. if you have queued but not active descriptors on all
> channels then dmac is idle and you can use this time to goto suspend.

But the time suspend is called for DMA, we may in middle of DMA transfers
on few channels. So should we wait for them to finish or simply disable
channel transfer? And in case of resume what will happen to transfers that
were active during suspend?

>>> *1.b* Will the suspend resume for the dependent drivers follow sequencing, i.e. Suspend of ADC followed by
>>> Suspend of DMA? (Assume both of these devices are hooked on to Platform bus, given that ADC uses DMA)
> why do you care?

Suppose suspend of DMA occurs first and we disable its clock in the suspend
routine. Now before the suspend of ADC is called, ADC requests for DMA
transfers and because not every routine of DMA keeps track of clk is on/off,
it may try to access DMA register.

So, we must ensure that all children are suspended before the parent is
requested to suspend. And probably this is the correct way.
So, here we need ADC to suspend before DMA. How is this sequencing ensured?

>>> 2.1 The user space threads freeze first followed by kernel space.  In case the user space process issues a a system
>>> call (lets assume an ioctl system call), how will the user space thread respond in case of suspend to ram?
> all user space processes are frozen on STR

Yes, but what will be its state. Will the system calls complete? Will it
close all devices opened by it before freezing?

Suppose a user applications is accessing few ADC channels. ADC controller
and its channel are all configured. Now, suspend is requested. What should
ADC driver do? As power will be turned off, its register will loose their
value.

Should ADC reconfigure all its channel registers by keeping their state?
Also it is not very straight forward to reconfigure channels directly, as
registers are required to be programmed in certain sequence, so there
needs to be good amount of intelligence put into driver for that.

IIRC, i couldn't find any such stuff in other mainline drivers. And that's
why we are confused on this topic.

-- 
viresh

[linux-pm Query] Power Management Device Suspend/ Resume

[Vinod Koul]

On Wed, 2011-09-28 at 12:15 +0530, Viresh Kumar wrote:
> Vinod,
> 
> Thanks for mailing us back.
> 
> On 9/28/2011 9:12 AM, Vinod Koul wrote:
> > On Wed, 2011-09-28 at 08:55 +0530, Viresh Kumar wrote:
> >> On 9/22/2011 4:19 PM, Deepak Sikri wrote:
> > are you trying to do runtime power management or traditional suspend and
> > resume? IMO both should be done...
> 
> We will first be testing traditional suspend/resume, but obviously we would
> try to design drivers in order to support runtime PM too.
> 
> >>> *Query- 1*. Suppose my platform has a ADC driver under char framework, which internally uses the
> >>> DMA driver (channels). Now, ADC driver can be used by other kernel drivers or directly from user application.
> >>>  
> >>> *1. a.* What are the expectations from the suspend and resume routines of both the devices, DMA & ADC ?
> >>>  
> >>> */-- Given that few of the options are/*
> >>> */1.a.1. On suspend,/*
> >>> In ADC suspend function: ADC releases all the DMA channels, latches its registers; and
> >>> DMA suspend function: DMA does nothing except for lataching of registers if required.
> >>>  
> >>> */1.a.2. On suspend, /*
> >>> ADC suspend function: ADC just latches its registers and stops R/W through DMA, no DMA channel release;
> >>> DMA suspend function: DMA halts its operating channels, and latches its registers if required.
> > Well channel release/alloc is not really required. You can goto suspend
> > in between as well. if you have queued but not active descriptors on all
> > channels then dmac is idle and you can use this time to goto suspend.
> 
> But the time suspend is called for DMA, we may in middle of DMA transfers
> on few channels. So should we wait for them to finish or simply disable
> channel transfer? And in case of resume what will happen to transfers that
> were active during suspend?
In traditional suspend, you should pause the channel, go to suspend.
Basically any active work needs to paused while you suspend to disk/ram.
But in case of embedded devices I would be surprised to see this even
being the case.
> 
> >>> *1.b* Will the suspend resume for the dependent drivers follow sequencing, i.e. Suspend of ADC followed by
> >>> Suspend of DMA? (Assume both of these devices are hooked on to Platform bus, given that ADC uses DMA)
> > why do you care?
> 
> Suppose suspend of DMA occurs first and we disable its clock in the suspend
> routine. Now before the suspend of ADC is called, ADC requests for DMA
> transfers and because not every routine of DMA keeps track of clk is on/off,
> it may try to access DMA register.
> 
> So, we must ensure that all children are suspended before the parent is
> requested to suspend. And probably this is the correct way.
> So, here we need ADC to suspend before DMA. How is this sequencing ensured?
No your driver should handle it well
Btw if you do runtime_pm implementation first, things will be lot easier
for you


> >>> 2.1 The user space threads freeze first followed by kernel space.  In case the user space process issues a a system
> >>> call (lets assume an ioctl system call), how will the user space thread respond in case of suspend to ram?
> > all user space processes are frozen on STR
> 
> Yes, but what will be its state. Will the system calls complete? Will it
> close all devices opened by it before freezing?
See the talks by Rafael in various conferences, they have lot of info
covering these questions.
IMO all processes are frozen on traditional suspend and later thawed on
resume. 
> 
> Suppose a user applications is accessing few ADC channels. ADC controller
> and its channel are all configured. Now, suspend is requested. What should
> ADC driver do? As power will be turned off, its register will loose their
> value.
It need to pause/stop current operation or return EBUSY/EAGAIN. Btw does
it happen for you while system is trying to do suspend.

> 
> Should ADC reconfigure all its channel registers by keeping their state?
> Also it is not very straight forward to reconfigure channels directly, as
> registers are required to be programmed in certain sequence, so there
> needs to be good amount of intelligence put into driver for that.
yup, you should be doing that on device probe anyway
> 
> IIRC, i couldn't find any such stuff in other mainline drivers. And that's
> why we are confused on this topic.

-- 
~Vinod

Re: [linux-pm Query] Power Management Device Suspend/ Resume

[Vinod Koul]

On Wed, 2011-09-28 at 12:15 +0530, Viresh Kumar wrote:
> Vinod,
> 
> Thanks for mailing us back.
> 
> On 9/28/2011 9:12 AM, Vinod Koul wrote:
> > On Wed, 2011-09-28 at 08:55 +0530, Viresh Kumar wrote:
> >> On 9/22/2011 4:19 PM, Deepak Sikri wrote:
> > are you trying to do runtime power management or traditional suspend and
> > resume? IMO both should be done...
> 
> We will first be testing traditional suspend/resume, but obviously we would
> try to design drivers in order to support runtime PM too.
> 
> >>> *Query- 1*. Suppose my platform has a ADC driver under char framework, which internally uses the
> >>> DMA driver (channels). Now, ADC driver can be used by other kernel drivers or directly from user application.
> >>>  
> >>> *1. a.* What are the expectations from the suspend and resume routines of both the devices, DMA & ADC ?
> >>>  
> >>> */-- Given that few of the options are/*
> >>> */1.a.1. On suspend,/*
> >>> In ADC suspend function: ADC releases all the DMA channels, latches its registers; and
> >>> DMA suspend function: DMA does nothing except for lataching of registers if required.
> >>>  
> >>> */1.a.2. On suspend, /*
> >>> ADC suspend function: ADC just latches its registers and stops R/W through DMA, no DMA channel release;
> >>> DMA suspend function: DMA halts its operating channels, and latches its registers if required.
> > Well channel release/alloc is not really required. You can goto suspend
> > in between as well. if you have queued but not active descriptors on all
> > channels then dmac is idle and you can use this time to goto suspend.
> 
> But the time suspend is called for DMA, we may in middle of DMA transfers
> on few channels. So should we wait for them to finish or simply disable
> channel transfer? And in case of resume what will happen to transfers that
> were active during suspend?
In traditional suspend, you should pause the channel, go to suspend.
Basically any active work needs to paused while you suspend to disk/ram.
But in case of embedded devices I would be surprised to see this even
being the case.
> 
> >>> *1.b* Will the suspend resume for the dependent drivers follow sequencing, i.e. Suspend of ADC followed by
> >>> Suspend of DMA? (Assume both of these devices are hooked on to Platform bus, given that ADC uses DMA)
> > why do you care?
> 
> Suppose suspend of DMA occurs first and we disable its clock in the suspend
> routine. Now before the suspend of ADC is called, ADC requests for DMA
> transfers and because not every routine of DMA keeps track of clk is on/off,
> it may try to access DMA register.
> 
> So, we must ensure that all children are suspended before the parent is
> requested to suspend. And probably this is the correct way.
> So, here we need ADC to suspend before DMA. How is this sequencing ensured?
No your driver should handle it well
Btw if you do runtime_pm implementation first, things will be lot easier
for you


> >>> 2.1 The user space threads freeze first followed by kernel space.  In case the user space process issues a a system
> >>> call (lets assume an ioctl system call), how will the user space thread respond in case of suspend to ram?
> > all user space processes are frozen on STR
> 
> Yes, but what will be its state. Will the system calls complete? Will it
> close all devices opened by it before freezing?
See the talks by Rafael in various conferences, they have lot of info
covering these questions.
IMO all processes are frozen on traditional suspend and later thawed on
resume. 
> 
> Suppose a user applications is accessing few ADC channels. ADC controller
> and its channel are all configured. Now, suspend is requested. What should
> ADC driver do? As power will be turned off, its register will loose their
> value.
It need to pause/stop current operation or return EBUSY/EAGAIN. Btw does
it happen for you while system is trying to do suspend.

> 
> Should ADC reconfigure all its channel registers by keeping their state?
> Also it is not very straight forward to reconfigure channels directly, as
> registers are required to be programmed in certain sequence, so there
> needs to be good amount of intelligence put into driver for that.
yup, you should be doing that on device probe anyway
> 
> IIRC, i couldn't find any such stuff in other mainline drivers. And that's
> why we are confused on this topic.

-- 
~Vinod