how to enable suspend to ram for arm-64 bits

how to enable suspend to ram for arm-64 bits

[yoma sophian]

hi all:
After I finish porting aarch64 linux kernel on my platform and enable
CONFIG_PM and CONFIG_SUSPEND,  I found there is no output when I cat
/sys/power/state.

After tracing kernel code, kernel/power/suspend.c, it seems we need to
create global suspend method table with suspend_set_ops then we can
get valid pm_states.
Is any dts or place like arm/match-xxx for registering global suspend method?

appreciate your kind help in advance,

how to enable suspend to ram for arm-64 bits

[Sudeep Holla]

Hi Yoma,

On 17/08/16 18:26, yoma sophian wrote:
> hi all: After I finish porting aarch64 linux kernel on my platform
> and enable CONFIG_PM and CONFIG_SUSPEND,  I found there is no output
> when I cat /sys/power/state.
>
> After tracing kernel code, kernel/power/suspend.c, it seems we need
> to create global suspend method table with suspend_set_ops then we
> can get valid pm_states.

Correct. What's the cpu enable method on your platform ? Is it PSCI ?
If yes, does you PSCI implementation support SYSTEM_SUSPEND ? If yes,
it should work. Check your PSCI implementation otherwise.

> Is any dts or place like arm/match-xxx for registering global suspend
> method?
>

No, currently only PSCI is supported and the firmware is queried and the
suspend_ops is registered automatically.

-- 
Regards,
Sudeep

-- 
Regards,
Sudeep

how to enable suspend to ram for arm-64 bits

[yoma sophian]

hi Sudeep

> Correct. What's the cpu enable method on your platform ? Is it PSCI ?
No, the enable method is "spin-table"

> If yes, does you PSCI implementation support SYSTEM_SUSPEND ? If yes,
> it should work. Check your PSCI implementation otherwise.

if so, there are 2 things make me curious:
a. I trace arch/arm64/kernel/psci.c even arch/arm64/*, but I sill
cannot find where it create global suspend method table with
suspend_set_ops.
# grep -rnw 'suspend_set_ops' ../linux-4.1/arch/arm64/
#
except arch/xxx folder, there are
../linux-4.1/drivers/macintosh/via-pmu.c:2276:    suspend_set_ops(&pmu_pm_ops);
../linux-4.1/drivers/acpi/sleep.c:666:    suspend_set_ops(old_suspend_ordering ?
Does that mean aarch64 register suspend_set_ops by apci flow when
adopt PSCI implementation?

b. in arm64, if some platform has its own suspend flow,  couldn't it
adopts arm/match-xxx to register its own global suspend method?

appreciate your kind help,

how to enable suspend to ram for arm-64 bits

[Sudeep Holla]

Hi Yoma,

On 18/08/16 03:07, yoma sophian wrote:
> hi Sudeep
>
>> Correct. What's the cpu enable method on your platform ? Is it PSCI  ?
> No, the enable method is "spin-table"
>

Generally spin-table is used for the initial bring up. If one is
interested in full power management support on a platform, PSCI is the
recommended standard on ARM64.

>> If yes, does you PSCI implementation support SYSTEM_SUSPEND ? If
>> yes, it should work. Check your PSCI implementation otherwise.
>
> if so, there are 2 things make me curious: a. I trace
> arch/arm64/kernel/psci.c even arch/arm64/*, but I sill cannot find
> where it create global suspend method table with suspend_set_ops. #
> grep -rnw 'suspend_set_ops' ../linux-4.1/arch/arm64/ #

drivers/firmware/psci.c

> except arch/xxx folder, there are
> ../linux-4.1/drivers/macintosh/via-pmu.c:2276:
> suspend_set_ops(&pmu_pm_ops); ../linux-4.1/drivers/acpi/sleep.c:666:
> suspend_set_ops(old_suspend_ordering ? Does that mean aarch64
> register suspend_set_ops by apci flow when adopt PSCI
> implementation?
>
Not yet, but may choose that from ACPI boot.


> b. in arm64, if some platform has its own suspend flow,  couldn't it
> adopts arm/match-xxx to register its own global suspend method?
>

No, PSCI is highly recommended.

-- 
Regards,
Sudeep

how to enable suspend to ram for arm-64 bits

[yoma sophian]

hi Sudeep:

2016-08-18 16:47 GMT+08:00 Sudeep Holla <sudeep.holla@arm.com>:
>
>> b. in arm64, if some platform has its own suspend flow,  couldn't it
>> adopts arm/match-xxx to register its own global suspend method?
>>
>
> No, PSCI is highly recommended.
Yes, I agree with you.
But it may not mean the ONLY way for all armv8 platform that support
suspend method is PSCI.

Sincerely appreciate your kind help,

how to enable suspend to ram for arm-64 bits

[Sudeep Holla]

Hi,

On 18/08/16 11:43, yoma sophian wrote:
> hi Sudeep:
>
> 2016-08-18 16:47 GMT+08:00 Sudeep Holla <sudeep.holla@arm.com>:
>>
>>> b. in arm64, if some platform has its own suspend flow,  couldn't it
>>> adopts arm/match-xxx to register its own global suspend method?
>>>
>>
>> No, PSCI is highly recommended.
> Yes, I agree with you.
> But it may not mean the ONLY way for all armv8 platform that support
> suspend method is PSCI.

I would say yes but if you have a real good reason not to adopt PSCI on
your platform then you can try present that. It should be convincing
enough to deviate from the standards :)

-- 
Regards,
Sudeep

how to enable suspend to ram for arm-64 bits

[yoma sophian]

hi Sudeep:

>> Yes, I agree with you.
>> But it may not mean the ONLY way for all armv8 platform that support
>> suspend method is PSCI.
>
> I would say yes but if you have a real good reason not to adopt PSCI on
> your platform then you can try present that. It should be convincing
> enough to deviate from the standards :)

Since we don't implement full features of PSCI on our platform, that
is why we wants to know whether there is registration of platform
suspend method.

BTW, from the git log, it seems psci.c doesn't register system suspend
until Linux 4.4.
if so, before 4.4, how aarch64 support such as str, echo mem >
/sys/power/state , suspend flow?

appreciate your kind explanation.

how to enable suspend to ram for arm-64 bits

[Sudeep Holla]

On 22/08/16 15:41, yoma sophian wrote:
> hi Sudeep:
>
>>> Yes, I agree with you.
>>> But it may not mean the ONLY way for all armv8 platform that support
>>> suspend method is PSCI.
>>
>> I would say yes but if you have a real good reason not to adopt PSCI on
>> your platform then you can try present that. It should be convincing
>> enough to deviate from the standards :)
>
> Since we don't implement full features of PSCI on our platform, that
> is why we wants to know whether there is registration of platform
> suspend method.
>

For adding support for your platform in the mainline kernel, you need to
support SYSTEM_SUSPEND in psci. There's no other way currently with the
mainline.

> BTW, from the git log, it seems psci.c doesn't register system suspend
> until Linux 4.4.

Correct.

> if so, before 4.4, how aarch64 support such as str, echo mem >
> /sys/power/state , suspend flow?
>

You can back-port those patches, they are quite trivial if you need
it on pre-v4.4 kernel.

Moreover, how do you plan to support system system without PSCI ?
How will you let the higher exception softwares know that you are
entering system suspend. How will their context resumed back ?

-- 
Regards,
Sudeep

how to enable suspend to ram for arm-64 bits

[Pavel Machek]

Hi!

> >b. in arm64, if some platform has its own suspend flow,  couldn't it
> >adopts arm/match-xxx to register its own global suspend method?
> >
> 
> No, PSCI is highly recommended.

Relying on firmware for suspend on x86 was a great disaster, lets not repeat
that mistake. arm32 has better powermanagement than x86 ever will (see Nokia N900
for example) -- feel free to copy that code from arm32.

Best regards,

									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

how to enable suspend to ram for arm-64 bits

[Sudeep Holla]

On 18/10/16 11:00, Pavel Machek wrote:
> Hi!
>
>>> b. in arm64, if some platform has its own suspend flow,  couldn't it
>>> adopts arm/match-xxx to register its own global suspend method?
>>>
>>
>> No, PSCI is highly recommended.
>
> Relying on firmware for suspend on x86 was a great disaster, lets not repeat
> that mistake.

Could you be more elaborate on this ?

  arm32 has better powermanagement than x86 ever will (see Nokia N900
> for example) -- feel free to copy that code from arm32.

OK are you suggesting that pull in all the low level assembly code that
are very platform specific in to the kernel ?

Sorry, no thanks. We don't want that in the kernel and IMO that's one of
the reason why many platforms lacked PM support in the upstream kernel
as they were too platform specific and hinders the progress towards
single kernel.

-- 
Regards,
Sudeep

how to enable suspend to ram for arm-64 bits

[Lorenzo Pieralisi]

On Tue, Oct 18, 2016 at 12:00:02PM +0200, Pavel Machek wrote:
> Hi!
> 
> > >b. in arm64, if some platform has its own suspend flow,  couldn't it
> > >adopts arm/match-xxx to register its own global suspend method?
> > >
> > 
> > No, PSCI is highly recommended.
> 
> Relying on firmware for suspend on x86 was a great disaster, lets not
> repeat that mistake. arm32 has better powermanagement than x86 ever
> will (see Nokia N900 for example) -- feel free to copy that code from
> arm32.

Yes sure, feel free to copy my:

NACKed-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

to all the resulting patches then. On ARM64 if he wants to implement
suspend-to-RAM his PSCI firmware will have to implement PSCI system
suspend method, end of this discussion.

By the way, why are you advising people on this subject ?

What do you know about the PSCI firmware interface to state what
you are saying above ?

Lorenzo

how to enable suspend to ram for arm-64 bits

[Mark Rutland]

On Tue, Oct 18, 2016 at 12:00:02PM +0200, Pavel Machek wrote:
> > >b. in arm64, if some platform has its own suspend flow,  couldn't it
> > >adopts arm/match-xxx to register its own global suspend method?
> > 
> > No, PSCI is highly recommended.
> 
> Relying on firmware for suspend on x86 was a great disaster, lets not repeat
> that mistake. arm32 has better powermanagement than x86 ever will (see Nokia N900
> for example) -- feel free to copy that code from arm32.

Quite frankly, copying hundreds of lines of board-specific code
(including assembly that won't compile) is unlikely to help.

So far arm64 requires well-defined, standard, reusable interfaces (e.g.
PSCI). That cleanly separates concerns (e.g. anyone can implement the
backend without mandatory changes to the kernel), and keeps things
maintainable.

ARM publishes and maintains the ARM Trusted Firmware [1], which anyone
can use and build atop of. It's open source (three-clause BSD with DCO),
and accepts board ports. You can have a completely open stack,
regardless of whether part of that stack is firmware.

Thanks,
Mark.

how to enable suspend to ram for arm-64 bits

[Sudeep Holla]

On 18/10/16 11:45, Mark Rutland wrote:
> On Tue, Oct 18, 2016 at 12:00:02PM +0200, Pavel Machek wrote:
>>>> b. in arm64, if some platform has its own suspend flow,  couldn't it
>>>> adopts arm/match-xxx to register its own global suspend method?
>>>
>>> No, PSCI is highly recommended.
>>
>> Relying on firmware for suspend on x86 was a great disaster, lets not repeat
>> that mistake. arm32 has better powermanagement than x86 ever will (see Nokia N900
>> for example) -- feel free to copy that code from arm32.
>
> Quite frankly, copying hundreds of lines of board-specific code
> (including assembly that won't compile) is unlikely to help.
>
> So far arm64 requires well-defined, standard, reusable interfaces (e.g.
> PSCI). That cleanly separates concerns (e.g. anyone can implement the
> backend without mandatory changes to the kernel), and keeps things
> maintainable.
>
> ARM publishes and maintains the ARM Trusted Firmware [1], which anyone
> can use and build atop of. It's open source (three-clause BSD with DCO),
> and accepts board ports. You can have a completely open stack,
> regardless of whether part of that stack is firmware.
>

I think you missed to add the link[1]

-- 
Regards,
Sudeep

[1] https://github.com/ARM-software/arm-trusted-firmware

how to enable suspend to ram for arm-64 bits

[yoma sophian]

hi Sudeep:

2016-10-18 18:59 GMT+08:00 Sudeep Holla <sudeep.holla@arm.com>:
>
>
> On 18/10/16 11:45, Mark Rutland wrote:
>>
>> On Tue, Oct 18, 2016 at 12:00:02PM +0200, Pavel Machek wrote:
>>>>>
>>>>> b. in arm64, if some platform has its own suspend flow,  couldn't it
>>>>> adopts arm/match-xxx to register its own global suspend method?
>>>>
>>>>
>>>> No, PSCI is highly recommended.
>>>
>>>
>>> Relying on firmware for suspend on x86 was a great disaster, lets not
>>> repeat
>>> that mistake. arm32 has better powermanagement than x86 ever will (see
>>> Nokia N900
>>> for example) -- feel free to copy that code from arm32.
>>
>>
>> Quite frankly, copying hundreds of lines of board-specific code
>> (including assembly that won't compile) is unlikely to help.
>>
>> So far arm64 requires well-defined, standard, reusable interfaces (e.g.
>> PSCI). That cleanly separates concerns (e.g. anyone can implement the
>> backend without mandatory changes to the kernel), and keeps things
>> maintainable.
>>
>> ARM publishes and maintains the ARM Trusted Firmware [1], which anyone
>> can use and build atop of. It's open source (three-clause BSD with DCO),
>> and accepts board ports. You can have a completely open stack,
>> regardless of whether part of that stack is firmware.
>>
>
> I think you missed to add the link[1]
> [1] https://github.com/ARM-software/arm-trusted-firmware
thanks for your kind information ^^

how to enable suspend to ram for arm-64 bits

[Pavel Machek]

On Tue 2016-10-18 11:45:39, Mark Rutland wrote:
> On Tue, Oct 18, 2016 at 12:00:02PM +0200, Pavel Machek wrote:
> > > >b. in arm64, if some platform has its own suspend flow,  couldn't it
> > > >adopts arm/match-xxx to register its own global suspend method?
> > > 
> > > No, PSCI is highly recommended.
> > 
> > Relying on firmware for suspend on x86 was a great disaster, lets not repeat
> > that mistake. arm32 has better powermanagement than x86 ever will (see Nokia N900
> > for example) -- feel free to copy that code from arm32.
> 
> Quite frankly, copying hundreds of lines of board-specific code
> (including assembly that won't compile) is unlikely to help.
> 
> So far arm64 requires well-defined, standard, reusable interfaces (e.g.
> PSCI). That cleanly separates concerns (e.g. anyone can implement the
> backend without mandatory changes to the kernel), and keeps things
> maintainable.

Either the lowlevel suspend code is stable and bug free, and then
having that code is not a problem. Or the lowlevel suspend code is
complex enough to contain some bugs, and in such case it is better to
debug and update it with kernel.

> ARM publishes and maintains the ARM Trusted Firmware [1], which anyone
> can use and build atop of. It's open source (three-clause BSD with DCO),
> and accepts board ports. You can have a completely open stack,
> regardless of whether part of that stack is firmware.

If something is called "Trusted", it is not trustworthy.

BSD is better than closed source, but it also means that you will not
get the sources from your hw vendor.

Being separate module means it will be never updated.

Being separate module means it will be hard to debug, in area where
debugging is already pretty hard.

Can it do advanced stuff like deep powersaving on N900 idle?

Just don't do it.
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20161019/05d8de63/attachment.sig>

how to enable suspend to ram for arm-64 bits

[Mark Rutland]

On Wed, Oct 19, 2016 at 11:42:27AM +0200, Pavel Machek wrote:
> On Tue 2016-10-18 11:45:39, Mark Rutland wrote:

> Either the lowlevel suspend code is stable and bug free, and then
> having that code is not a problem.

This ignores the cost of maintaining that code. Kernel APIs change over
time, and no code is ever completely stable, even if at one point in
time it happens to be bug-free.

> Or the lowlevel suspend code is complex enough to contain some bugs,
> and in such case it is better to debug and update it with kernel.

It is better for that code to be debuggable and updateable. That is not
the same as being part of the kernel.
 
> > ARM publishes and maintains the ARM Trusted Firmware [1], which anyone
> > can use and build atop of. It's open source (three-clause BSD with DCO),
> > and accepts board ports. You can have a completely open stack,
> > regardless of whether part of that stack is firmware.
> 
> If something is called "Trusted", it is not trustworthy.

Certainly we shouldn't blindly trust anything.

I object to ATF being called "not trustworthy"; the aims of the project
are certainly not dishonest.

> BSD is better than closed source, but it also means that you will not
> get the sources from your hw vendor.

That depends on your hardware vendor, as always. There are a number of
platform ports in the upstream ATF repo.

It's also worth considering that a number of 32-bit arm parts require
closed firmware (as far as I can tell, including the N900).

> Being separate module means it will be never updated.

That's certainly not true as a blanket statement. The Juno FW (including
an open-source UEFI!) is periodically updated, and mechanisms like
UpdateCapsule() should make this easier in future.

> Being separate module means it will be hard to debug, in area where
> debugging is already pretty hard.

It can be harder, yes. There are also benefits, given the same code can
be tested on a variety of platforms.

> Can it do advanced stuff like deep powersaving on N900 idle?

Sorry, I don't know precisely what you're referring to.

It can do things like shutting down entire CPU clusters (and IIRC
associated interconnect) when all relevant CPUs are idle, if that's what
you mean.

Thanks,
Mark.

how to enable suspend to ram for arm-64 bits

[Pavel Machek]

On Wed 2016-10-19 14:11:45, Mark Rutland wrote:
> On Wed, Oct 19, 2016 at 11:42:27AM +0200, Pavel Machek wrote:
> > On Tue 2016-10-18 11:45:39, Mark Rutland wrote:
> 
> > Either the lowlevel suspend code is stable and bug free, and then
> > having that code is not a problem.
> 
> This ignores the cost of maintaining that code. Kernel APIs change over
> time, and no code is ever completely stable, even if at one point in
> time it happens to be bug-free.

Well, kernel interfaces only change when there's good reason for a
change, and if you force stable binary interface to external
component, and there _is_ need for a change, we are all screwed.

> > BSD is better than closed source, but it also means that you will not
> > get the sources from your hw vendor.
> 
> That depends on your hardware vendor, as always. There are a number of
> platform ports in the upstream ATF repo.
> 
> It's also worth considering that a number of 32-bit arm parts require
> closed firmware (as far as I can tell, including the N900).

Yeah, but we are trying to remove closed firmware. Don't make people
add more of it...

> > Being separate module means it will be hard to debug, in area where
> > debugging is already pretty hard.
> 
> It can be harder, yes. There are also benefits, given the same code can
> be tested on a variety of platforms.

What benefits? You are able to share code between platforms in kernel,
too.

> > Can it do advanced stuff like deep powersaving on N900 idle?
> 
> Sorry, I don't know precisely what you're referring to.
> 
> It can do things like shutting down entire CPU clusters (and IIRC
> associated interconnect) when all relevant CPUs are idle, if that's what
> you mean.

I mean equivalent power savings between idle system and system in
s2ram.

Best regards,
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20161103/b3b419d1/attachment.sig>