Re: RFC: Proper suspend-to-ram implementation of Ingenic SoCs

[Paul Cercueil <paul@crapouillou.net>]

Le mer., juil. 13 2022 at 04:51:05 +0800, Mike Yang 
<reimu@sudomaker.com> a écrit :
> Hi Paul,
> 
> Thanks for the information.
> 
> 
> On 7/13/22 04:28, Paul Cercueil wrote:
>>  Hi Mike,
>> 
>>  Le mer., juil. 13 2022 at 03:19:32 +0800, Mike Yang 
>> <reimu@sudomaker.com> a écrit :
>>>  The suspend-to-ram implementation of Ingenic SoCs in the current 
>>> kernel is nowhere near usable, especially for the X series SoCs. 
>>> Since it involves turning off CPU core power and putting DRAM into 
>>> self-refresh mode, things are a bit complicated. Turning off CPU 
>>> core power means all register files and cache contents are lost. 
>>> Putting DRAM into self-refresh mode means it will no longer respond 
>>> to bus transactions.
>> 
>>  Suspend-to-RAM is well-tested and has been working fine for ages on 
>> all JZ SoCs, so I wouldn't call it "nowhere near usable". Zhou also 
>> implemented it on X-series SoCs.
> 
> With the vanilla 5.18 kernel, the system will simply become 
> unresponsive after typing "echo mem > /sys/power/state". It won't 
> respond to WKUP and other interrupt-enabled GPIO pins. The power 
> consumption is a bit lower, but nowhere near 0.045W. The behavior is 
> the same for X1000(E) and X1501.

Well, do you know why it fails? Did you try to debug it? Does it still 
become unresponsive if you comment the "wait" instruction?

It's a bit early to talk about power consumption if it doesn't suspend 
properly yet.

> I asked Dr. Zhou about this in person and he said he never tested the 
> suspend-to-ram, nor he confirmed it working.
> 
>> 
>>>  I ported the implementation from Ingenic's 3.10 kernel to 5.18, 
>>> and it worked. But it involves a separate piece of executable code, 
>>> and apparently there's no way to eliminate it. During pm_enter(), 
>>> various CPM registers are configured to turn off CPU core and put 
>>> DRAM into self-refresh upon issuing the "wait" instruction, this 
>>> piece of executable code will be copied to the on-chip SRAM, and 
>>> its entry address will be written into the CPM.SLPC register. Then, 
>>> cache will be flushed and CPU register files (incl. CP0, CP1 stuff) 
>>> will also be saved in the SRAM. Finally, the "wait" instruction 
>>> will be issued, and the suspend procedure completed. When any 
>>> external events trigger a resume, the CPU is powered on, and 
>>> immediately jumps to the PC stored in CPM.SLPC, and starts 
>>> executing the piece of code. The code will perform the usual crt0 
>>> stuff on MIPS machines, reconfigure the DRAM into normal mode, and 
>>> finally restore the register files. Then the control flow goes back 
>>> to
>>>  pm_enter(), and the resume procedure is completed.
>> 
>>  This sounds extremely complex and way overkill. But you don't need 
>> any of this.
>> 
>>>  The suspend-to-ram really saves a lot of power. For my particular 
>>> board, the idle power consumption is about 0.24W (1.25V Vcore, 
>>> 1.2GHz, 1000Hz, preempt). After suspend-to-ram, it drops to only 
>>> 0.045W.
>> 
>>  Yes, doesn't surprise me. The RG-350 (JZ4770 based) can last about 
>> ~6 hours of up-time, and when put  to sleep it will survive a few 
>> weeks.
>> 
>>>  So here are my questions:
>>>  1. I don't see a way to eliminate the piece of executable code in 
>>> SRAM. Is there any other ways?
>> 
>>  There is what's already implemented, yes. When triggering a 
>> suspend, the CPM.LCR.LPM setting is set to SLEEP mode 
>> (drivers/clk/ingenic/pm.c), then the ingenic_pm_enter() function 
>> (arch/mips/generic/board-ingenic.c) just executes the "wait" CPU 
>> instruction to put the CPU to sleep. All clocks but the RTC one are 
>> disabled until an interrupt is raised.
>> 
>>>  2. If we can't eliminate the code in SRAM, what's the accepted way 
>>> of integrating it into the kernel tree?
>> 
>>  Already upstream :)
>> 
>>>  3. If the hardware doesn't have 32k crystal connected, or the RTC 
>>> is stripped off (e.g. X1501), some CPM registers need to be 
>>> configured differently. How could we provide this configuration?
>> 
>>  It's already supported. The RTC clock can be re-parented (in device 
>> tree) to the EXT/512 clock, which is (as its name suggests) derived 
>> from the external EXT oscillator.
>> 
>>  Hopefully I answered all your questions.
>> 
>>  Cheers,
>>  -Paul
>> 
>> 
> 
> I'm afraid the above didn't work for me. Have you tested 
> suspend-to-ram in person on a X series SoC?

I didn't test on X-series, I mostly work with JZ. But that part of the 
design didn't change since the JZ4740.

Cheers,
-Paul