[RFC 0/3] experimental G5 powermac suspend implementation

[RFC 0/3] experimental G5 powermac suspend implementation

[Johannes Berg]

This is a rather hacky suspend implementation for G5s.

I'm only posting it for the fun of it and because it makes no sense to
continue sitting on it (I've had parts of this code more than half a year
ago...)

Don't blame me if it breaks your machine, filesystems or similar :)

johannes

[RFC 1/3] support powermac G5 CPU hotplug

[Johannes Berg]

This patch allows "hotplugging" of CPUs on G5 machines. CPUs that are
disabled are put into an idle loop with interrupts hard-disabled, to wake
them up again we kick them just like when bringing them up.

Maybe there's some way to actually turn off those CPUs even more?

Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>

--- linux-2.6-git.orig/arch/powerpc/platforms/powermac/smp.c	2006-12-11 23:34:47.612716759 +0100
+++ linux-2.6-git/arch/powerpc/platforms/powermac/smp.c	2006-12-11 23:37:54.971716759 +0100
@@ -898,7 +898,7 @@ void smp_core99_cpu_die(unsigned int cpu
 	cpu_dead[cpu] = 0;
 }
 
-#endif
+#endif /* CONFIG_HOTPLUG_CPU && CONFIG_PP32 */
 
 /* Core99 Macs (dual G4s and G5s) */
 struct smp_ops_t core99_smp_ops = {
@@ -908,8 +908,16 @@ struct smp_ops_t core99_smp_ops = {
 	.setup_cpu	= smp_core99_setup_cpu,
 	.give_timebase	= smp_core99_give_timebase,
 	.take_timebase	= smp_core99_take_timebase,
-#if defined(CONFIG_HOTPLUG_CPU) && defined(CONFIG_PPC32)
+#if defined(CONFIG_HOTPLUG_CPU)
+# if defined(CONFIG_PPC32)
 	.cpu_disable	= smp_core99_cpu_disable,
 	.cpu_die	= smp_core99_cpu_die,
+# endif
+# if defined(CONFIG_PPC64)
+	.cpu_disable	= generic_cpu_disable,
+	.cpu_die	= generic_cpu_die,
+	/* intentionally do *NOT* assign cpu_enable,
+	 * the generic code will use kick_cpu then! */
+# endif
 #endif
 };
--- linux-2.6-git.orig/arch/powerpc/platforms/powermac/setup.c	2006-12-11 23:34:47.648716759 +0100
+++ linux-2.6-git/arch/powerpc/platforms/powermac/setup.c	2006-12-11 23:37:54.974716759 +0100
@@ -490,6 +490,9 @@ static int pmac_late_init(void)
 #ifdef CONFIG_SOFTWARE_SUSPEND
 	pm_set_ops(&pmac_pm_ops);
 #endif /* CONFIG_SOFTWARE_SUSPEND */
+	/* this is udbg (which is __init) and we can later use it during
+	 * cpu hotplug (in smp_core99_kick_cpu) */
+	ppc_md.progress = NULL;
 	return 0;
 }
 
@@ -716,6 +719,32 @@ static int pmac_pci_probe_mode(struct pc
 		return PCI_PROBE_NORMAL;
 	return PCI_PROBE_DEVTREE;
 }
+
+#ifdef CONFIG_HOTPLUG_CPU
+/* access per cpu vars from generic smp.c */
+DECLARE_PER_CPU(int, cpu_state);
+
+static void pmac_cpu_die(void)
+{
+	/* turn off as much as possible, we'll be
+	 * kicked out as this will only be invoked
+	 * on core99 platforms for now ... */
+	hard_irq_disable();
+	printk(KERN_DEBUG "CPU#%d offline\n", smp_processor_id());
+	__get_cpu_var(cpu_state) = CPU_DEAD;
+	smp_wmb();
+	while (1) {
+		ppc64_runlatch_off();
+		if (ppc_md.power_save) {
+			ppc_md.power_save();
+		} else {
+			HMT_low();
+			HMT_very_low();
+		}
+	}
+}
+#endif
+
 #endif
 
 define_machine(powermac) {
@@ -754,6 +783,6 @@ define_machine(powermac) {
 	.phys_mem_access_prot	= pci_phys_mem_access_prot,
 #endif
 #if defined(CONFIG_HOTPLUG_CPU) && defined(CONFIG_PPC64)
-	.cpu_die		= generic_mach_cpu_die,
+	.cpu_die		= pmac_cpu_die,
 #endif
 };
--- linux-2.6-git.orig/arch/powerpc/kernel/smp.c	2006-12-11 23:34:47.693716759 +0100
+++ linux-2.6-git/arch/powerpc/kernel/smp.c	2006-12-11 23:37:55.017716759 +0100
@@ -558,6 +558,9 @@ int __devinit start_secondary(void *unus
 
 	local_irq_enable();
 
+	/* who knows what context the processor might have died in... */
+	task_thread_info(paca[cpu].__current)->preempt_count = 0;
+
 	cpu_idle();
 	return 0;
 }

--

Re: [linux-pm] [RFC 1/3] support powermac G5 CPU hotplug

[Pavel Machek]

Hi!

> This patch allows "hotplugging" of CPUs on G5 machines. CPUs that are
> disabled are put into an idle loop with interrupts hard-disabled, to wake
> them up again we kick them just like when bringing them up.
> 
> Maybe there's some way to actually turn off those CPUs even more?
> 
> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> 


> --- linux-2.6-git.orig/arch/powerpc/kernel/smp.c	2006-12-11 23:34:47.693716759 +0100
> +++ linux-2.6-git/arch/powerpc/kernel/smp.c	2006-12-11 23:37:55.017716759 +0100
> @@ -558,6 +558,9 @@ int __devinit start_secondary(void *unus
>  
>  	local_irq_enable();
>  
> +	/* who knows what context the processor might have died in... */
> +	task_thread_info(paca[cpu].__current)->preempt_count = 0;
> +

?? Context for downing cpu should better be well-defined. Just do this
in cpu_die if it is really required.
							Pavel
-- 
Thanks for all the (sleeping) penguins.

Re: [linux-pm] [RFC 1/3] support powermac G5 CPU hotplug

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 590 bytes --]

On Tue, 2006-12-12 at 13:09 +0000, Pavel Machek wrote:

> > +	/* who knows what context the processor might have died in... */
> > +	task_thread_info(paca[cpu].__current)->preempt_count = 0;
> > +
> 
> ?? Context for downing cpu should better be well-defined. Just do this
> in cpu_die if it is really required.

It is actually required but I'm not exactly sure why. When I don't do
this, I end up with preempt_count() 0x100000 (IIRC, notes not here at
the moment).

I think this is because the suspend code has interrupts disabled already
when downing the CPUs.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: [linux-pm] [RFC 1/3] support powermac G5 CPU hotplug

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 669 bytes --]

On Tue, 2006-12-12 at 14:18 +0100, Johannes Berg wrote:

> > ?? Context for downing cpu should better be well-defined. Just do this
> > in cpu_die if it is really required.
> 
> It is actually required but I'm not exactly sure why. When I don't do
> this, I end up with preempt_count() 0x100000 (IIRC, notes not here at
                                      ^^^^^^^^ typo, should be 0x10000
> the moment).
> 
> I think this is because the suspend code has interrupts disabled already
> when downing the CPUs.

Also, the /sys/devices/system/cpu/cpuN/online thing doesn't disable
interrupts before downing the CPU thus that works without this.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: [linux-pm] [RFC 1/3] support powermac G5 CPU hotplug

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 503 bytes --]

On Tue, 2006-12-12 at 13:09 +0000, Pavel Machek wrote:

> ?? Context for downing cpu should better be well-defined. Just do this
> in cpu_die if it is really required.

Hmm. Just tried, but that doesn't help either!

I'm confused. Really need to check where that value comes from. It looks
as though we disable the CPU inside an interrupt but I can't see how
that should happen during resume code. Or maybe preempt count is
initialised non-zero during boot or something.. Digging

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

[RFC 2/3] windfarm: dont die on suspend thread signal

[Johannes Berg]

When the windfarm thread gets a suspend signal it will die instead of
freezing. This fixes it.

Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>

--- linux-2.6-git.orig/drivers/macintosh/windfarm_core.c	2006-12-11 23:34:47.082716759 +0100
+++ linux-2.6-git/drivers/macintosh/windfarm_core.c	2006-12-11 23:37:58.181716759 +0100
@@ -93,8 +93,6 @@ static int wf_thread_func(void *data)
 	DBG("wf: thread started\n");
 
 	while(!kthread_should_stop()) {
-		try_to_freeze();
-
 		if (time_after_eq(jiffies, next)) {
 			wf_notify(WF_EVENT_TICK, NULL);
 			if (wf_overtemp) {
@@ -117,8 +115,8 @@ static int wf_thread_func(void *data)
 		if (delay <= HZ)
 			schedule_timeout_interruptible(delay);
 
-		/* there should be no signal, but oh well */
-		if (signal_pending(current)) {
+		/* there should be no non-suspend signal, but oh well */
+		if (signal_pending(current) && !try_to_freeze()) {
 			printk(KERN_WARNING "windfarm: thread got sigl !\n");
 			break;
 		}

--

Re: [linux-pm] [RFC 2/3] windfarm: dont die on suspend thread signal

[Pavel Machek]

On Tue 12-12-06 00:02:10, Johannes Berg wrote:
> When the windfarm thread gets a suspend signal it will die instead of
> freezing. This fixes it.
> 
> Signed-off-by: Johannes Berg <johannes@sipsolutions.net>
> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>

ACK.

-- 
Thanks for all the (sleeping) penguins.

Re: [RFC 2/3] windfarm: dont die on suspend thread signal

[Benjamin Herrenschmidt]

On Tue, 2006-12-12 at 00:02 +0100, Johannes Berg wrote:
> plain text document attachment (windfarm-dont-die-on-signals.patch)
> When the windfarm thread gets a suspend signal it will die instead of
> freezing. This fixes it.
> 
> Signed-off-by: Johannes Berg <johannes@sipsolutions.net>

Acked-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>

> --- linux-2.6-git.orig/drivers/macintosh/windfarm_core.c	2006-12-11 23:34:47.082716759 +0100
> +++ linux-2.6-git/drivers/macintosh/windfarm_core.c	2006-12-11 23:37:58.181716759 +0100
> @@ -93,8 +93,6 @@ static int wf_thread_func(void *data)
>  	DBG("wf: thread started\n");
>  
>  	while(!kthread_should_stop()) {
> -		try_to_freeze();
> -
>  		if (time_after_eq(jiffies, next)) {
>  			wf_notify(WF_EVENT_TICK, NULL);
>  			if (wf_overtemp) {
> @@ -117,8 +115,8 @@ static int wf_thread_func(void *data)
>  		if (delay <= HZ)
>  			schedule_timeout_interruptible(delay);
>  
> -		/* there should be no signal, but oh well */
> -		if (signal_pending(current)) {
> +		/* there should be no non-suspend signal, but oh well */
> +		if (signal_pending(current) && !try_to_freeze()) {
>  			printk(KERN_WARNING "windfarm: thread got sigl !\n");
>  			break;
>  		}
> 
> --

[RFC 3/3] suspend to disk on powermac G5

[Johannes Berg]

This patch implements a very hacky suspend to disk for powermac G5 machines. I
know it's broken in many places.

Built and tested against the tree after commit
620034c84d1d939717bdfbe02c51a3fee43541c3.

For some reason usb doesn't survive suspend even if I try to restore
interrupts now... I use this script:

>>>>>>
#!/bin/sh

set -e

echo reboot > /sys/power/disk

rmmod ohci_hcd
rmmod ehci_hcd || true
sync
echo disk > /sys/power/state || true
modprobe ohci_hcd
modprobe ehci_hcd
sleep 30 && reboot -f
<<<<<

and simply press ctrl-c within 30 seconds if my keyboard comes back up :)

Oh and did I say this is hacky? MMIO save/restore, the need to hook many
places, no proper MPIC suspend/resume functionality, snd-aoa doesn't work
after resume (it plays but no sound comes out, interrupts are coming in
though... go figure, eh wait, since I wrote it I guess I'll check it too).

I also ifdef'ed out

	set_context(current->active_mm->context.id, current->active_mm->pgd);

because I didn't know what it is doing and it works without :)

I've suspended and resumed a few times in a row with this now, with and
without longer breaks with the machine up and with the machine off.

Ah who am I kidding... This is pretty much guaranteed to break. Parts of the
assembly code are quite old (written like half a year ago) so I don't
remember some things... Why, for example, the save area has holes. I shall
fix these some time I guess.

Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>

--- linux-2.6-git.orig/arch/powerpc/kernel/Makefile	2006-12-11 23:34:46.458716759 +0100
+++ linux-2.6-git/arch/powerpc/kernel/Makefile	2006-12-11 23:38:00.127716759 +0100
@@ -36,6 +36,7 @@ obj-$(CONFIG_CRASH_DUMP)	+= crash_dump.o
 obj-$(CONFIG_6xx)		+= idle_6xx.o l2cr_6xx.o cpu_setup_6xx.o
 obj-$(CONFIG_TAU)		+= tau_6xx.o
 obj32-$(CONFIG_SOFTWARE_SUSPEND) += swsusp_32.o
+obj64-$(CONFIG_SOFTWARE_SUSPEND) += swsusp_64.o swsusp_asm64.o
 obj32-$(CONFIG_MODULES)		+= module_32.o
 
 ifeq ($(CONFIG_PPC_MERGE),y)
--- linux-2.6-git.orig/kernel/power/Kconfig	2006-12-11 23:34:46.682716759 +0100
+++ linux-2.6-git/kernel/power/Kconfig	2006-12-11 23:38:00.129716759 +0100
@@ -78,7 +78,7 @@ config PM_SYSFS_DEPRECATED
 
 config SOFTWARE_SUSPEND
 	bool "Software Suspend"
-	depends on PM && SWAP && ((X86 && (!SMP || SUSPEND_SMP) && !X86_PAE) || ((FRV || PPC32) && !SMP))
+	depends on PM && SWAP && (((X86 || PPC64) && (!SMP || SUSPEND_SMP) && !X86_PAE) || ((FRV || PPC32) && !SMP))
 	---help---
 	  Enable the possibility of suspending the machine.
 	  It doesn't need ACPI or APM.
@@ -128,5 +128,5 @@ config PM_STD_PARTITION
 
 config SUSPEND_SMP
 	bool
-	depends on HOTPLUG_CPU && X86 && PM
+	depends on HOTPLUG_CPU && (X86 || PPC64) && PM
 	default y
--- linux-2.6-git.orig/arch/powerpc/platforms/powermac/setup.c	2006-12-11 23:37:54.974716759 +0100
+++ linux-2.6-git/arch/powerpc/platforms/powermac/setup.c	2006-12-11 23:38:00.132716759 +0100
@@ -455,8 +455,10 @@ static int pmac_pm_finish(suspend_state_
 {
 	printk(KERN_DEBUG "%s(%d)\n", __FUNCTION__, state);
 
+#ifdef CONFIG_PPC32
 	/* Restore userland MMU context */
 	set_context(current->active_mm->context.id, current->active_mm->pgd);
+#endif
 
 	return 0;
 }
@@ -488,6 +490,7 @@ static int pmac_late_init(void)
 {
 	initializing = 0;
 #ifdef CONFIG_SOFTWARE_SUSPEND
+	iommu_init_late();
 	pm_set_ops(&pmac_pm_ops);
 #endif /* CONFIG_SOFTWARE_SUSPEND */
 	/* this is udbg (which is __init) and we can later use it during
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-git/include/asm/suspend.h	2006-12-11 23:38:00.135716759 +0100
@@ -0,0 +1,9 @@
+#ifndef __ASM_POWERPC_SUSPEND_H
+#define __ASM_POWERPC_SUSPEND_H
+
+static inline int arch_prepare_suspend(void)
+{
+	return 0;
+}
+
+#endif /* __ASM_POWERPC_SUSPEND_H */
--- linux-2.6-git.orig/include/linux/suspend.h	2006-12-11 23:34:46.802716759 +0100
+++ linux-2.6-git/include/linux/suspend.h	2006-12-11 23:38:00.136716759 +0100
@@ -1,7 +1,7 @@
 #ifndef _LINUX_SWSUSP_H
 #define _LINUX_SWSUSP_H
 
-#if defined(CONFIG_X86) || defined(CONFIG_FRV) || defined(CONFIG_PPC32)
+#if defined(CONFIG_X86) || defined(CONFIG_FRV) || defined(CONFIG_PPC32) || defined(CONFIG_PPC64)
 #include <asm/suspend.h>
 #endif
 #include <linux/swap.h>
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-git/arch/powerpc/kernel/swsusp_64.c	2006-12-11 23:38:00.138716759 +0100
@@ -0,0 +1,54 @@
+/*
+ * PowerPC 64-bit swsusp implementation
+ *
+ * Copyright 2006 Johannes Berg <johannes@sipsolutions.net>
+ *
+ * GPLv2
+ */
+
+#include <asm/system.h>
+#include <asm/iommu.h>
+#include <linux/irq.h>
+#include <linux/interrupt.h>
+
+void save_processor_state(void)
+{
+	hard_irq_disable();
+}
+
+void restore_processor_state(void)
+{
+	hard_irq_enable();
+}
+
+static void irqs_resume(void)
+{
+	int irq;
+
+	for (irq = 1; irq < NR_IRQS; irq++) {
+		irq_desc_t *desc = get_irq_desc(irq);
+		if (!desc->action || !desc->action->handler)
+			continue;
+		/* umm, this must be borked... */
+		if (desc->chip && desc->chip->set_type)
+			desc->chip->set_type(irq, (desc->status & IRQ_LEVEL) ? IRQF_TRIGGER_LOW : IRQF_TRIGGER_FALLING);
+		if (desc->chip->startup)
+			desc->chip->startup(irq);
+		else
+			desc->chip->enable(irq);
+	}
+}
+
+void do_after_copyback(void)
+{
+#ifdef CONFIG_U3_DART
+	iommu_dart_restore();
+#endif
+	irqs_resume();
+	touch_softlockup_watchdog();
+	mb();
+	hard_irq_enable();
+	/* all that happens after this is a return
+	 * to the location restore_image() came from
+	 */
+}
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6-git/arch/powerpc/kernel/swsusp_asm64.S	2006-12-11 23:38:00.142716759 +0100
@@ -0,0 +1,231 @@
+/*
+ * PowerPC 64-bit swsusp implementation
+ *
+ * Copyright 2006 Johannes Berg <johannes@sipsolutions.net>
+ *
+ * GPLv2
+ */
+
+#include <linux/threads.h>
+#include <asm/processor.h>
+#include <asm/page.h>
+#include <asm/cputable.h>
+#include <asm/thread_info.h>
+#include <asm/ppc_asm.h>
+#include <asm/asm-offsets.h>
+
+/*
+ * Structure for storing CPU registers on the save area.
+ */
+#define SL_r1		0x00	/* stack pointer */
+#define SL_PC		0x08
+#define SL_MSR		0x10
+#define SL_SDR1		0x18
+#define SL_XER		0x20
+#define SL_TB		0x40
+#define SL_r2		0x48
+#define SL_CR		0x50
+#define SL_LR		0x58
+#define SL_r12		0x60
+#define SL_r13		0x68
+#define SL_r14		0x70
+#define SL_r15		0x78
+#define SL_r16		0x80
+#define SL_r17		0x88
+#define SL_r18		0x90
+#define SL_r19		0x98
+#define SL_r20		0xa0
+#define SL_r21		0xa8
+#define SL_r22		0xb0
+#define SL_r23		0xb8
+#define SL_r24		0xc0
+#define SL_r25		0xc8
+#define SL_r26		0xd0
+#define SL_r27		0xd8
+#define SL_r28		0xe0
+#define SL_r29		0xe8
+#define SL_r30		0xf0
+#define SL_r31		0xf8
+#define SL_SIZE		SL_r31+8
+
+/* these macros rely on the save area being
+ * pointed to by r11 */
+#define SAVE_SPECIAL(special)		\
+	mf##special	r0		;\
+	std	r0, SL_##special(r11)
+#define RESTORE_SPECIAL(special)	\
+	ld	r0, SL_##special(r11)	;\
+	mt##special	r0
+#define SAVE_REGISTER(reg)		\
+	std	reg, SL_##reg(r11)
+#define RESTORE_REGISTER(reg)		\
+	ld	reg, SL_##reg(r11)
+
+/* space for storing cpu state */
+	.section .data
+	.align  5
+swsusp_save_area:
+	.space SL_SIZE
+
+	.section ".toc","aw"
+swsusp_save_area_ptr:
+	.tc	swsusp_save_area[TC],swsusp_save_area
+restore_pblist_ptr:
+	.tc	restore_pblist[TC],restore_pblist
+
+	.section .text
+	.align  5
+_GLOBAL(swsusp_arch_suspend)
+	ld	r11,swsusp_save_area_ptr@toc(r2)
+	SAVE_SPECIAL(LR)
+	SAVE_REGISTER(r1)
+	SAVE_SPECIAL(CR)
+	SAVE_SPECIAL(TB)
+	SAVE_REGISTER(r2)
+	SAVE_REGISTER(r12)
+	SAVE_REGISTER(r13)
+	SAVE_REGISTER(r14)
+	SAVE_REGISTER(r15)
+	SAVE_REGISTER(r16)
+	SAVE_REGISTER(r17)
+	SAVE_REGISTER(r18)
+	SAVE_REGISTER(r19)
+	SAVE_REGISTER(r20)
+	SAVE_REGISTER(r21)
+	SAVE_REGISTER(r22)
+	SAVE_REGISTER(r23)
+	SAVE_REGISTER(r24)
+	SAVE_REGISTER(r25)
+	SAVE_REGISTER(r26)
+	SAVE_REGISTER(r27)
+	SAVE_REGISTER(r28)
+	SAVE_REGISTER(r29)
+	SAVE_REGISTER(r30)
+	SAVE_REGISTER(r31)
+	SAVE_SPECIAL(MSR)
+	SAVE_SPECIAL(SDR1)
+	SAVE_SPECIAL(XER)
+
+	/* we push the stack up 128 bytes but don't store the
+	 * stack pointer on the stack like a real stackframe */
+	addi	r1,r1,-128
+
+#ifdef	CONFIG_U3_DART
+	bl iommu_dart_save
+#endif
+
+	bl swsusp_save
+
+	/* restore LR */
+	ld	r11,swsusp_save_area_ptr@toc(r2)
+	RESTORE_SPECIAL(LR)
+	addi	r1,r1,128
+
+	blr
+
+/* Resume code */
+_GLOBAL(swsusp_arch_resume)
+	/* Stop pending alitvec streams and memory accesses */
+BEGIN_FTR_SECTION
+	DSSALL
+END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC)
+	sync
+
+	ld	r12,restore_pblist_ptr@toc(r2)
+	ld	r12,0(r12)
+
+	cmpdi	r12,0
+	beq-	nothing_to_copy
+	li	r15,512
+copyloop:
+	ld	r13,pbe_address(r12)
+	ld	r14,pbe_orig_address(r12)
+
+	mtctr	r15
+	li	r10,0
+copy_page_loop:
+	ldx	r0,r10,r13
+	stdx	r0,r10,r14
+	addi	r10,r10,8
+	bdnz copy_page_loop
+
+	ld	r12,pbe_next(r12)
+	cmpdi	r12,0
+	bne+	copyloop
+nothing_to_copy:
+
+	/* flush caches */
+	lis	r3, 0x10
+	mtctr	r3
+	li	r3, 0
+	ori	r3, r3, CONFIG_KERNEL_START>>48
+	li	r0, 48
+	sld	r3, r3, r0
+	li	r0, 0
+1:
+	dcbf	r0,r3
+	addi	r3,r3,0x20
+	bdnz	1b
+
+	sync
+
+	tlbia
+
+	ld	r11,swsusp_save_area_ptr@toc(r2)
+
+	RESTORE_SPECIAL(CR)
+
+	/* restore timebase */
+	/* load saved tb */
+	ld	r1, SL_TB(r11)
+	/* get upper 32 bits of it */
+	srdi	r2, r1, 32
+	/* clear tb lower to avoid wrap */
+	li	r0, 0
+	mttbl	r0
+	/* set tb upper */
+	mttbu	r2
+	/* set tb lower */
+	mttbl	r1
+
+	/* restore registers */
+	RESTORE_REGISTER(r1)
+	RESTORE_REGISTER(r2)
+	RESTORE_REGISTER(r12)
+	RESTORE_REGISTER(r13)
+	RESTORE_REGISTER(r14)
+	RESTORE_REGISTER(r15)
+	RESTORE_REGISTER(r16)
+	RESTORE_REGISTER(r17)
+	RESTORE_REGISTER(r18)
+	RESTORE_REGISTER(r19)
+	RESTORE_REGISTER(r20)
+	RESTORE_REGISTER(r21)
+	RESTORE_REGISTER(r22)
+	RESTORE_REGISTER(r23)
+	RESTORE_REGISTER(r24)
+	RESTORE_REGISTER(r25)
+	RESTORE_REGISTER(r26)
+	RESTORE_REGISTER(r27)
+	RESTORE_REGISTER(r28)
+	RESTORE_REGISTER(r29)
+	RESTORE_REGISTER(r30)
+	RESTORE_REGISTER(r31)
+	/* can't use RESTORE_SPECIAL(MSR) */
+	ld	r0, SL_MSR(r11)
+	mtmsrd	r0, 0
+	RESTORE_SPECIAL(SDR1)
+	RESTORE_SPECIAL(XER)
+
+	sync
+
+	addi	r1,r1,-128
+	bl	slb_flush_and_rebolt
+	bl	do_after_copyback
+	addi	r1,r1,128
+
+	ld	r11,swsusp_save_area_ptr@toc(r2)
+	RESTORE_SPECIAL(LR)
+
+	li	r3, 0
+	blr
--- linux-2.6-git.orig/arch/powerpc/sysdev/dart_iommu.c	2006-12-11 23:34:46.639716759 +0100
+++ linux-2.6-git/arch/powerpc/sysdev/dart_iommu.c	2006-12-11 23:38:00.144716759 +0100
@@ -54,6 +54,9 @@ static unsigned long dart_tablesize;
 
 /* Virtual base address of the DART table */
 static u32 *dart_vbase;
+#ifdef CONFIG_SOFTWARE_SUSPEND
+static u32 *dart_copy;
+#endif
 
 /* Mapped base address for the dart */
 static unsigned int __iomem *dart;
@@ -346,6 +349,45 @@ void iommu_init_early_dart(void)
 	pci_dma_ops = &dma_direct_ops;
 }
 
+#ifdef CONFIG_SOFTWARE_SUSPEND
+void iommu_init_late(void)
+{
+	unsigned long i;
+	struct page *p;
+
+	/* phew. suckers. this 16MB area is left unmapped
+	 * at another place but they don't bother to mark it so */
+	for (i = 0; i < (1<<24); i+= PAGE_SIZE)
+		SetPageNosave(virt_to_page((void*)((unsigned long)dart_tablebase + i)));
+
+	if (dart_tablebase == 0 || dart_tablesize == 0)
+		return;
+
+	p = alloc_pages(GFP_KERNEL, 9);
+	BUG_ON(!p);
+	dart_copy = page_address(p);
+}
+
+void iommu_dart_save(void)
+{
+	if (dart_tablebase == 0 || dart_tablesize == 0)
+		return;
+
+	memcpy(dart_copy, dart_vbase, 2*1024*1024);
+	printk(KERN_INFO "dart copied\n");
+}
+
+void iommu_dart_restore(void)
+{
+	if (dart_tablebase == 0 || dart_tablesize == 0)
+		return;
+
+	printk("going to restore dart\n");
+	memcpy(dart_vbase, dart_copy, 2*1024*1024);
+	printk("copied over dart entries\n");
+	dart_tlb_invalidate_all();
+}
+#endif
 
 void __init alloc_dart_table(void)
 {
--- linux-2.6-git.orig/include/asm-powerpc/iommu.h	2006-12-11 23:34:46.861716759 +0100
+++ linux-2.6-git/include/asm-powerpc/iommu.h	2006-12-11 23:38:00.146716759 +0100
@@ -108,6 +108,11 @@ static inline void pci_iommu_init(void) 
 #endif
 
 extern void alloc_dart_table(void);
+#if defined(CONFIG_SOFTWARE_SUSPEND) && defined(CONFIG_U3_DART)
+extern void iommu_init_late(void);
+extern void iommu_dart_save(void);
+extern void iommu_dart_restore(void);
+#endif
 
 #endif /* __KERNEL__ */
 #endif /* _ASM_IOMMU_H */
--- linux-2.6-git.orig/arch/powerpc/kernel/asm-offsets.c	2006-12-11 23:34:46.529716759 +0100
+++ linux-2.6-git/arch/powerpc/kernel/asm-offsets.c	2006-12-11 23:38:00.148716759 +0100
@@ -21,12 +21,12 @@
 #include <linux/types.h>
 #include <linux/mman.h>
 #include <linux/mm.h>
+#include <linux/suspend.h>
 #ifdef CONFIG_PPC64
 #include <linux/time.h>
 #include <linux/hardirq.h>
 #else
 #include <linux/ptrace.h>
-#include <linux/suspend.h>
 #endif
 
 #include <asm/io.h>
@@ -257,11 +257,11 @@ int main(void)
 	DEFINE(CPU_SPEC_SETUP, offsetof(struct cpu_spec, cpu_setup));
 	DEFINE(CPU_SPEC_RESTORE, offsetof(struct cpu_spec, cpu_restore));
 
-#ifndef CONFIG_PPC64
 	DEFINE(pbe_address, offsetof(struct pbe, address));
 	DEFINE(pbe_orig_address, offsetof(struct pbe, orig_address));
 	DEFINE(pbe_next, offsetof(struct pbe, next));
 
+#ifndef CONFIG_PPC64
 	DEFINE(TASK_SIZE, TASK_SIZE);
 	DEFINE(NUM_USER_SEGMENTS, TASK_SIZE>>28);
 #endif /* ! CONFIG_PPC64 */
--- linux-2.6-git.orig/arch/powerpc/kernel/idle.c	2006-12-11 23:34:46.561716759 +0100
+++ linux-2.6-git/arch/powerpc/kernel/idle.c	2006-12-11 23:38:00.149716759 +0100
@@ -33,8 +33,11 @@
 #include <asm/smp.h>
 
 #ifdef CONFIG_HOTPLUG_CPU
+/* this is used for software suspend, and that shuts down
+ * CPUs even while the system is still booting... */
 #define cpu_should_die()	(cpu_is_offline(smp_processor_id()) && \
-				 system_state == SYSTEM_RUNNING)
+				   (system_state == SYSTEM_RUNNING     \
+				 || system_state == SYSTEM_BOOTING))
 #else
 #define cpu_should_die()	0
 #endif
--- linux-2.6-git.orig/kernel/power/snapshot.c	2006-12-11 23:34:46.718716759 +0100
+++ linux-2.6-git/kernel/power/snapshot.c	2006-12-11 23:38:00.152716759 +0100
@@ -707,6 +707,8 @@ static struct page *saveable_page(unsign
 		return NULL;
 	if (PageNosaveFree(page))
 		return NULL;
+	if (!page_is_ram(pfn))
+		return NULL;
 
 	return page;
 }

--

Re: [linux-pm] [RFC 3/3] suspend to disk on powermac G5

[Pavel Machek]

Hi!

> This patch implements a very hacky suspend to disk for powermac G5 machines. I
> know it's broken in many places.
> 
> Built and tested against the tree after commit
> 620034c84d1d939717bdfbe02c51a3fee43541c3.
> 
> For some reason usb doesn't survive suspend even if I try to restore
> interrupts now... I use this script:
...
> Oh and did I say this is hacky? MMIO save/restore, the need to hook many
> places, no proper MPIC suspend/resume functionality, snd-aoa doesn't work
> after resume (it plays but no sound comes out, interrupts are coming in
> though... go figure, eh wait, since I wrote it I guess I'll check it too).

Does aoa module load/unload help? Does making it build-in help? Also
try playing with aumix after resume.

> I also ifdef'ed out
> 
> 	set_context(current->active_mm->context.id, current->active_mm->pgd);
> 
> because I didn't know what it is doing and it works without :)
> 
> I've suspended and resumed a few times in a row with this now, with and
> without longer breaks with the machine up and with the machine off.
> 
> Ah who am I kidding... This is pretty much guaranteed to break. Parts of the
> assembly code are quite old (written like half a year ago) so I don't
> remember some things... Why, for example, the save area has holes. I shall
> fix these some time I guess.
> 


> +	/* phew. suckers. this 16MB area is left unmapped
> +	 * at another place but they don't bother to mark it so */
> +	for (i = 0; i < (1<<24); i+= PAGE_SIZE)
> +		SetPageNosave(virt_to_page((void*)((unsigned long)dart_tablebase + i)));
> +

Can you move that marking where it belongs?

> +	if (dart_tablebase == 0 || dart_tablesize == 0)
> +		return;
> +
> +	p = alloc_pages(GFP_KERNEL, 9);
> +	BUG_ON(!p);
> +	dart_copy = page_address(p);
> +}

Not sure how likely is 2MB allocation to succeed. You may need to
prealocate it... And you definitely will need to handle error here.


> --- linux-2.6-git.orig/kernel/power/snapshot.c	2006-12-11 23:34:46.718716759 +0100
> +++ linux-2.6-git/kernel/power/snapshot.c	2006-12-11 23:38:00.152716759 +0100
> @@ -707,6 +707,8 @@ static struct page *saveable_page(unsign
>  		return NULL;
>  	if (PageNosaveFree(page))
>  		return NULL;
> +	if (!page_is_ram(pfn))
> +		return NULL;
>  
>  	return page;
>  }

Could this get some testing in -mm?

Overall, it looks pretty much okay. Thanks...

							Pavel
-- 
Thanks for all the (sleeping) penguins.

Re: [linux-pm] [RFC 3/3] suspend to disk on powermac G5

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 2271 bytes --]

Hi,

> Does aoa module load/unload help? Does making it build-in help? Also
> try playing with aumix after resume.

Yeah, I tried all of that except making it built-in. It's rather strange
but I suppose I'm to blame since I wrote aoa :) I'll need to look a bit
further. The odd thing is that everything works, I can play, change
volumes etc. just don't get any sound. I'm thinking something is wrong
with the onyx codec programming but haven't really looked yet. i2sbus
should be fine since it suspends/resume ok on my powerbook.

> > +	/* phew. suckers. this 16MB area is left unmapped
> > +	 * at another place but they don't bother to mark it so */
> > +	for (i = 0; i < (1<<24); i+= PAGE_SIZE)
> > +		SetPageNosave(virt_to_page((void*)((unsigned long)dart_tablebase + i)));
> > +
> 
> Can you move that marking where it belongs?

Where does it belong? The reservation is done so early that
SetPageNosave can't work yet (IIRC, I've been sitting on this code too
long.)

> > +	if (dart_tablebase == 0 || dart_tablesize == 0)
> > +		return;
> > +
> > +	p = alloc_pages(GFP_KERNEL, 9);
> > +	BUG_ON(!p);
> > +	dart_copy = page_address(p);
> > +}
> 
> Not sure how likely is 2MB allocation to succeed. You may need to
> prealocate it... And you definitely will need to handle error here.

Good points. I suppose I'll just preallocate it or try mapping the dart
table after all and not doing anything special with it during
suspend/resume... need to think about it more.

> > --- linux-2.6-git.orig/kernel/power/snapshot.c	2006-12-11 23:34:46.718716759 +0100
> > +++ linux-2.6-git/kernel/power/snapshot.c	2006-12-11 23:38:00.152716759 +0100
> > @@ -707,6 +707,8 @@ static struct page *saveable_page(unsign
> >  		return NULL;
> >  	if (PageNosaveFree(page))
> >  		return NULL;
> > +	if (!page_is_ram(pfn))
> > +		return NULL;
> >  
> >  	return page;
> >  }
> 
> Could this get some testing in -mm?

I'd be much happier with it if I knew why it was necessary. On the
powermac we have a 2G memory hole: first 2G memory, then a 2G hole and
then the rest of the memory. It seems like the suspend code doesn't
handle any memory holes at the moment but that seems strange to me.

Thanks for your comments!

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: [RFC 0/3] experimental G5 powermac suspend implementation

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 198 bytes --]

Date/time is restored to wherever it was when suspending. Does anyone
know how to fix that? Does the RTC somehow warp the time on resume or do
we somehow mark the time/date __nosave?

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: [linux-pm] [RFC 0/3] experimental G5 powermac suspend implementation

[Pavel Machek]

On Tue 12-12-06 13:21:07, Johannes Berg wrote:
> Date/time is restored to wherever it was when suspending. Does anyone
> know how to fix that? Does the RTC somehow warp the time on resume or do
> we somehow mark the time/date __nosave?

Clock do not tick while suspended; I guess you need to copy time
suspend/resume from i386 or something.
							Pavel
-- 
Thanks for all the (sleeping) penguins.

Re: [linux-pm] [RFC 0/3] experimental G5 powermac suspend implementation

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 340 bytes --]

On Tue, 2006-12-12 at 13:22 +0000, Pavel Machek wrote:

> Clock do not tick while suspended; I guess you need to copy time
> suspend/resume from i386 or something.

Well the RTC of course ticks but the system clock obviously doesn't.
Yes, I'll have to look at the time suspend/resume code. Do you know
where to find it?

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: [linux-pm] [RFC 0/3] experimental G5 powermac suspend implementation

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 306 bytes --]

On Tue, 2006-12-12 at 14:36 +0100, Johannes Berg wrote:

> Well the RTC of course ticks but the system clock obviously doesn't.
> Yes, I'll have to look at the time suspend/resume code. Do you know
> where to find it?

Never mind, found it easily enough: it's in arch/i386/kernel/time.c

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: [linux-pm] [RFC 0/3] experimental G5 powermac suspend implementation

[Gabriel Paubert]

On Tue, Dec 12, 2006 at 03:07:43PM +0100, Johannes Berg wrote:
> On Tue, 2006-12-12 at 14:36 +0100, Johannes Berg wrote:
> 
> > Well the RTC of course ticks but the system clock obviously doesn't.
> > Yes, I'll have to look at the time suspend/resume code. Do you know
> > where to find it?
> 
> Never mind, found it easily enough: it's in arch/i386/kernel/time.c

Timer is also restored from RTC on wakeup on powerbooks, so 
there is already something under arch/powerpc...

	Gabriel

Re: [linux-pm] [RFC 0/3] experimental G5 powermac suspend implementation

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 230 bytes --]

On Tue, 2006-12-12 at 18:34 +0100, Gabriel Paubert wrote:

> Timer is also restored from RTC on wakeup on powerbooks, so 
> there is already something under arch/powerpc...

Right :) Sending a patch in a minute.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 190 bytes --]