Alternative to TIF_FREEZE -> a notifier in the task_struct?

Alternative to TIF_FREEZE -> a notifier in the task_struct?

[Christoph Lameter]

Hmmm. I am thinking about an alternative to PF_FREEZE and TIF_FREEZE. 
PF/TIF_FREEZE essentially makes a task call a certain function in the 
context of the task. Maybe there is a way to generalize that?

One could introducing a notifier chain (todo) in the task_struct that 
is then called in the current places where PF/TIF_FREEZE is checked?

This could be used to have a process execute any other piece that is 
required to run in the context of the thread. Maybe such a feature could 
help with PTRACE or/and get_user_pages that currently do ugly things to a 
process. It may also allow changing values that so far cannot be
changed from the outside in the task struct by running a function
in the context of the process.

The feature to execute a function via a notifier chain allows to localize
the suspend functionality in kernel/power/process.c. We may be able to 
take the definitions out of sched.h (while adding the functions to handle 
the todo list).

Here is a patch against Linus current tree that does this. Note that this 
patch is incomplete at this point and only a basis for further discussion. 
Not all software suspend checkpoints are useful for a notifier chain. We 
would need to inspect several cases where drivers/kernel threads have 
special functionality for software suspend.

Index: linux-2.6/include/linux/sched.h
===================================================================
--- linux-2.6.orig/include/linux/sched.h	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/include/linux/sched.h	2005-07-27 20:04:57.000000000 -0700
@@ -34,6 +34,7 @@
 #include <linux/percpu.h>
 #include <linux/topology.h>
 #include <linux/seccomp.h>
+#include <linux/notifier.h>
 
 struct exec_domain;
 
@@ -720,7 +721,10 @@
 	int (*notifier)(void *priv);
 	void *notifier_data;
 	sigset_t *notifier_mask;
-	
+
+	/* todo list to be executed in the context of this thread */
+	struct notifier_block *todo;
+
 	void *security;
 	struct audit_context *audit_context;
 	seccomp_t seccomp;
@@ -811,7 +815,6 @@
 #define PF_MEMALLOC	0x00000800	/* Allocating memory */
 #define PF_FLUSHER	0x00001000	/* responsible for disk writeback */
 #define PF_USED_MATH	0x00002000	/* if unset the fpu must be initialized before use */
-#define PF_FREEZE	0x00004000	/* this task is being frozen for suspend now */
 #define PF_NOFREEZE	0x00008000	/* this thread should not be frozen */
 #define PF_FROZEN	0x00010000	/* frozen for system suspend */
 #define PF_FSTRANS	0x00020000	/* inside a filesystem transaction */
@@ -1273,78 +1276,46 @@
 
 #endif
 
-#ifdef CONFIG_PM
-/*
- * Check if a process has been frozen
- */
-static inline int frozen(struct task_struct *p)
-{
-	return p->flags & PF_FROZEN;
-}
-
 /*
- * Check if there is a request to freeze a process
+ * Check if there is a todo list request
  */
-static inline int freezing(struct task_struct *p)
+static inline int todo_list_active(void)
 {
-	return p->flags & PF_FREEZE;
+	return current->todo != NULL;
 }
 
-/*
- * Request that a process be frozen
- * FIXME: SMP problem. We may not modify other process' flags!
- */
-static inline void freeze(struct task_struct *p)
+static inline void run_todo_list(void)
 {
-	p->flags |= PF_FREEZE;
+	notifier_call_chain(&current->todo, 0, current);
 }
 
-/*
- * Wake up a frozen process
- */
-static inline int thaw_process(struct task_struct *p)
+static inline int try_todo_list(void)
 {
-	if (frozen(p)) {
-		p->flags &= ~PF_FROZEN;
-		wake_up_process(p);
+	if (todo_list_active()) {
+		run_todo_list();
 		return 1;
-	}
-	return 0;
+	} else
+		return 0;
 }
 
-/*
- * freezing is complete, mark process as frozen
- */
-static inline void frozen_process(struct task_struct *p)
-{
-	p->flags = (p->flags & ~PF_FREEZE) | PF_FROZEN;
-}
+#ifdef CONFIG_PM
 
-extern void refrigerator(void);
 extern int freeze_processes(void);
 extern void thaw_processes(void);
 
-static inline int try_to_freeze(void)
+/*
+ * Check if a process has been frozen
+ */
+static inline int frozen(struct task_struct *p)
 {
-	if (freezing(current)) {
-		refrigerator();
-		return 1;
-	} else
-		return 0;
+	return p->flags & PF_FROZEN;
 }
+
 #else
 static inline int frozen(struct task_struct *p) { return 0; }
-static inline int freezing(struct task_struct *p) { return 0; }
-static inline void freeze(struct task_struct *p) { BUG(); }
-static inline int thaw_process(struct task_struct *p) { return 1; }
-static inline void frozen_process(struct task_struct *p) { BUG(); }
-
-static inline void refrigerator(void) {}
 static inline int freeze_processes(void) { BUG(); return 0; }
 static inline void thaw_processes(void) {}
 
-static inline int try_to_freeze(void) { return 0; }
-
 #endif /* CONFIG_PM */
 #endif /* __KERNEL__ */
 
Index: linux-2.6/kernel/power/process.c
===================================================================
--- linux-2.6.orig/kernel/power/process.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/kernel/power/process.c	2005-07-27 20:07:54.000000000 -0700
@@ -18,6 +18,8 @@
  */
 #define TIMEOUT	(6 * HZ)
 
+DECLARE_COMPLETION(thaw);
+static atomic_t nr_frozen;
 
 static inline int freezeable(struct task_struct * p)
 {
@@ -32,25 +34,38 @@
 }
 
 /* Refrigerator is place where frozen processes are stored :-). */
-void refrigerator(void)
+static int refrigerator(struct notifier_block *nl, unsigned long x, void *v)
 {
 	/* Hmm, should we be allowed to suspend when there are realtime
 	   processes around? */
 	long save;
 	save = current->state;
-	current->state = TASK_UNINTERRUPTIBLE;
 	pr_debug("%s entered refrigerator\n", current->comm);
 	printk("=");
 
-	frozen_process(current);
 	spin_lock_irq(&current->sighand->siglock);
+	current->flags |= PF_FROZEN;
+	notifier_chain_unregister(&current->todo, nl);
 	recalc_sigpending(); /* We sent fake signal, clean it up */
+	atomic_inc(&nr_frozen);
 	spin_unlock_irq(&current->sighand->siglock);
 
-	while (frozen(current))
-		schedule();
+	wait_for_completion(&thaw);
+	current->flags &= ~PF_FROZEN;
+	atomic_dec(&nr_frozen);
+	kfree(nl);
 	pr_debug("%s left refrigerator\n", current->comm);
 	current->state = save;
+	return 0;
+}
+
+void thaw_processes(void)
+{
+	printk( "Restarting tasks..." );
+	complete_all(&thaw);
+	while (atomic_read(&nr_frozen) > 0)
+		schedule();
+	printk( " done\n" );
 }
 
 /* 0 = success, else # of processes that we failed to stop */
@@ -61,19 +76,31 @@
 	struct task_struct *g, *p;
 	unsigned long flags;
 
+	atomic_set(&nr_frozen, 0);
+	INIT_COMPLETION(thaw);
+
 	printk( "Stopping tasks: " );
 	start_time = jiffies;
 	do {
 		todo = 0;
 		read_lock(&tasklist_lock);
 		do_each_thread(g, p) {
+			struct notifier_block *n;
+
 			if (!freezeable(p))
 				continue;
-			if (frozen(p))
+			if (p->flags & PF_FROZEN)
 				continue;
 
-			freeze(p);
+			n = kmalloc(sizeof(struct notifier_block), GFP_KERNEL);
+			n->notifier_call = refrigerator;
+			n->priority = 0;
 			spin_lock_irqsave(&p->sighand->siglock, flags);
+			if (!p->todo)
+				notifier_chain_register(&g->todo, n);
+			else
+				kfree(n);
+			recalc_sigpending();
 			signal_wake_up(p, 0);
 			spin_unlock_irqrestore(&p->sighand->siglock, flags);
 			todo++;
@@ -83,6 +110,7 @@
 		if (time_after(jiffies, start_time + TIMEOUT)) {
 			printk( "\n" );
 			printk(KERN_ERR " stopping tasks failed (%d tasks remaining)\n", todo );
+			thaw_processes();
 			return todo;
 		}
 	} while(todo);
@@ -92,22 +120,3 @@
 	return 0;
 }
 
-void thaw_processes(void)
-{
-	struct task_struct *g, *p;
-
-	printk( "Restarting tasks..." );
-	read_lock(&tasklist_lock);
-	do_each_thread(g, p) {
-		if (!freezeable(p))
-			continue;
-		if (!thaw_process(p))
-			printk(KERN_INFO " Strange, %s not stopped\n", p->comm );
-	} while_each_thread(g, p);
-
-	read_unlock(&tasklist_lock);
-	schedule();
-	printk( " done\n" );
-}
-
-EXPORT_SYMBOL(refrigerator);
Index: linux-2.6/kernel/signal.c
===================================================================
--- linux-2.6.orig/kernel/signal.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/kernel/signal.c	2005-07-27 20:01:49.000000000 -0700
@@ -213,7 +213,7 @@
 fastcall void recalc_sigpending_tsk(struct task_struct *t)
 {
 	if (t->signal->group_stop_count > 0 ||
-	    (freezing(t)) ||
+	    (t->todo) ||
 	    PENDING(&t->pending, &t->blocked) ||
 	    PENDING(&t->signal->shared_pending, &t->blocked))
 		set_tsk_thread_flag(t, TIF_SIGPENDING);
@@ -2231,7 +2231,7 @@
 			current->state = TASK_INTERRUPTIBLE;
 			timeout = schedule_timeout(timeout);
 
-			try_to_freeze();
+			try_todo_list();
 			spin_lock_irq(&current->sighand->siglock);
 			sig = dequeue_signal(current, &these, &info);
 			current->blocked = current->real_blocked;
Index: linux-2.6/Documentation/power/kernel_threads.txt
===================================================================
--- linux-2.6.orig/Documentation/power/kernel_threads.txt	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/Documentation/power/kernel_threads.txt	2005-07-27 19:59:59.000000000 -0700
@@ -4,15 +4,15 @@
 Freezer
 
 Upon entering a suspended state the system will freeze all
-tasks. This is done by delivering pseudosignals. This affects
-kernel threads, too. To successfully freeze a kernel thread
-the thread has to check for the pseudosignal and enter the
-refrigerator. Code to do this looks like this:
+tasks. This is done by making all processes execute a notifier.
+This affects kernel threads, too. To successfully freeze a kernel thread
+the thread has to check for the notifications and call the notifier
+chain for the process. Code to do this looks like this:
 
 	do {
 		hub_events();
 		wait_event_interruptible(khubd_wait, !list_empty(&hub_event_list));
-		try_to_freeze();
+		try_todo_list();
 	} while (!signal_pending(current));
 
 from drivers/usb/core/hub.c::hub_thread()
Index: linux-2.6/Documentation/power/swsusp.txt
===================================================================
--- linux-2.6.orig/Documentation/power/swsusp.txt	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/Documentation/power/swsusp.txt	2005-07-27 19:59:59.000000000 -0700
@@ -155,7 +155,8 @@
 website, and not to the Linux Kernel Mailing List. We are working
 toward merging suspend2 into the mainline kernel.
 
-Q: A kernel thread must voluntarily freeze itself (call 'refrigerator').
+Q: A kernel thread must work on the todo list (call 'run_todo_list')
+to enter the refrigerator.
 I found some kernel threads that don't do it, and they don't freeze
 so the system can't sleep. Is this a known behavior?
 
@@ -164,7 +165,7 @@
 should be held at that point and it must be safe to sleep there), and
 add:
 
-       try_to_freeze();
+       try_todo_list();
 
 If the thread is needed for writing the image to storage, you should
 instead set the PF_NOFREEZE process flag when creating the thread (and
Index: linux-2.6/arch/frv/kernel/signal.c
===================================================================
--- linux-2.6.orig/arch/frv/kernel/signal.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/arch/frv/kernel/signal.c	2005-07-27 19:59:59.000000000 -0700
@@ -536,7 +536,7 @@
 	if (!user_mode(regs))
 		return 1;
 
-	if (try_to_freeze())
+	if (try_todo_list())
 		goto no_signal;
 
 	if (!oldset)
Index: linux-2.6/arch/h8300/kernel/signal.c
===================================================================
--- linux-2.6.orig/arch/h8300/kernel/signal.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/arch/h8300/kernel/signal.c	2005-07-27 19:59:59.000000000 -0700
@@ -517,7 +517,7 @@
 	if ((regs->ccr & 0x10))
 		return 1;
 
-	if (try_to_freeze())
+	if (try_todo_list())
 		goto no_signal;
 
 	current->thread.esp0 = (unsigned long) regs;
Index: linux-2.6/arch/i386/kernel/io_apic.c
===================================================================
--- linux-2.6.orig/arch/i386/kernel/io_apic.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/arch/i386/kernel/io_apic.c	2005-07-27 19:59:59.000000000 -0700
@@ -574,7 +574,7 @@
 	for ( ; ; ) {
 		set_current_state(TASK_INTERRUPTIBLE);
 		time_remaining = schedule_timeout(time_remaining);
-		try_to_freeze();
+		try_todo_list();
 		if (time_after(jiffies,
 				prev_balance_time+balanced_irq_interval)) {
 			preempt_disable();
Index: linux-2.6/arch/i386/kernel/signal.c
===================================================================
--- linux-2.6.orig/arch/i386/kernel/signal.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/arch/i386/kernel/signal.c	2005-07-27 19:59:59.000000000 -0700
@@ -608,7 +608,7 @@
 	if (!user_mode(regs))
 		return 1;
 
-	if (try_to_freeze())
+	if (try_todo_list())
 		goto no_signal;
 
 	if (!oldset)
Index: linux-2.6/arch/m32r/kernel/signal.c
===================================================================
--- linux-2.6.orig/arch/m32r/kernel/signal.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/arch/m32r/kernel/signal.c	2005-07-27 19:59:59.000000000 -0700
@@ -371,7 +371,7 @@
 	if (!user_mode(regs))
 		return 1;
 
-	if (try_to_freeze()) 
+	if (try_todo_list())
 		goto no_signal;
 
 	if (!oldset)
Index: linux-2.6/arch/ppc/kernel/signal.c
===================================================================
--- linux-2.6.orig/arch/ppc/kernel/signal.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/arch/ppc/kernel/signal.c	2005-07-27 19:59:59.000000000 -0700
@@ -705,7 +705,7 @@
 	unsigned long frame, newsp;
 	int signr, ret;
 
-	if (try_to_freeze()) {
+	if (try_todo_list()) {
 		signr = 0;
 		if (!signal_pending(current))
 			goto no_signal;
Index: linux-2.6/arch/x86_64/kernel/signal.c
===================================================================
--- linux-2.6.orig/arch/x86_64/kernel/signal.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/arch/x86_64/kernel/signal.c	2005-07-27 19:59:59.000000000 -0700
@@ -425,7 +425,7 @@
 	if (!user_mode(regs))
 		return 1;
 
-	if (try_to_freeze())
+	if (try_todo_list())
 		goto no_signal;
 
 	if (!oldset)
Index: linux-2.6/drivers/block/pktcdvd.c
===================================================================
--- linux-2.6.orig/drivers/block/pktcdvd.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/drivers/block/pktcdvd.c	2005-07-27 19:59:59.000000000 -0700
@@ -1250,8 +1250,7 @@
 			residue = schedule_timeout(min_sleep_time);
 			VPRINTK("kcdrwd: wake up\n");
 
-			/* make swsusp happy with our thread */
-			try_to_freeze();
+			try_todo_list();
 
 			list_for_each_entry(pkt, &pd->cdrw.pkt_active_list, list) {
 				if (!pkt->sleep_time)
Index: linux-2.6/drivers/ieee1394/ieee1394_core.c
===================================================================
--- linux-2.6.orig/drivers/ieee1394/ieee1394_core.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/drivers/ieee1394/ieee1394_core.c	2005-07-27 19:59:59.000000000 -0700
@@ -1044,7 +1044,7 @@
 
 	while (1) {
 		if (down_interruptible(&khpsbpkt_sig)) {
-			if (try_to_freeze())
+			if (try_todo_list())
 				continue;
 			printk("khpsbpkt: received unexpected signal?!\n" );
 			break;
Index: linux-2.6/drivers/ieee1394/nodemgr.c
===================================================================
--- linux-2.6.orig/drivers/ieee1394/nodemgr.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/drivers/ieee1394/nodemgr.c	2005-07-27 19:59:59.000000000 -0700
@@ -1510,7 +1510,7 @@
 
 		if (down_interruptible(&hi->reset_sem) ||
 		    down_interruptible(&nodemgr_serialize)) {
-			if (try_to_freeze())
+			if (try_todo_list())
 				continue;
 			printk("NodeMgr: received unexpected signal?!\n" );
 			break;
Index: linux-2.6/drivers/input/gameport/gameport.c
===================================================================
--- linux-2.6.orig/drivers/input/gameport/gameport.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/drivers/input/gameport/gameport.c	2005-07-27 19:59:59.000000000 -0700
@@ -435,7 +435,7 @@
 		gameport_handle_events();
 		wait_event_interruptible(gameport_wait,
 			kthread_should_stop() || !list_empty(&gameport_event_list));
-		try_to_freeze();
+		try_todo_list();
 	} while (!kthread_should_stop());
 
 	printk(KERN_DEBUG "gameport: kgameportd exiting\n");
Index: linux-2.6/drivers/input/serio/serio.c
===================================================================
--- linux-2.6.orig/drivers/input/serio/serio.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/drivers/input/serio/serio.c	2005-07-27 19:59:59.000000000 -0700
@@ -371,7 +371,7 @@
 		serio_handle_events();
 		wait_event_interruptible(serio_wait,
 			kthread_should_stop() || !list_empty(&serio_event_list));
-		try_to_freeze();
+		try_todo_list();
 	} while (!kthread_should_stop());
 
 	printk(KERN_DEBUG "serio: kseriod exiting\n");
Index: linux-2.6/drivers/media/dvb/dvb-core/dvb_frontend.c
===================================================================
--- linux-2.6.orig/drivers/media/dvb/dvb-core/dvb_frontend.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/drivers/media/dvb/dvb-core/dvb_frontend.c	2005-07-27 19:59:59.000000000 -0700
@@ -394,7 +394,7 @@
 			break;
 		}
 
-		try_to_freeze();
+		try_todo_list();
 
 		if (down_interruptible(&fepriv->sem))
 			break;
Index: linux-2.6/drivers/net/irda/stir4200.c
===================================================================
--- linux-2.6.orig/drivers/net/irda/stir4200.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/drivers/net/irda/stir4200.c	2005-07-27 19:59:59.000000000 -0700
@@ -763,7 +763,7 @@
 	{
 #ifdef CONFIG_PM
 		/* if suspending, then power off and wait */
-		if (unlikely(freezing(current))) {
+		if (unlikely(todo_list_active())) {
 			if (stir->receiving)
 				receive_stop(stir);
 			else
@@ -771,7 +771,7 @@
 
 			write_reg(stir, REG_CTRL1, CTRL1_TXPWD|CTRL1_RXPWD);
 
-			refrigerator();
+			run_todo_list();
 
 			if (change_speed(stir, stir->speed))
 				break;
Index: linux-2.6/drivers/pcmcia/cs.c
===================================================================
--- linux-2.6.orig/drivers/pcmcia/cs.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/drivers/pcmcia/cs.c	2005-07-27 19:59:59.000000000 -0700
@@ -683,7 +683,7 @@
 		}
 
 		schedule();
-		try_to_freeze();
+		try_todo_list();
 
 		if (!skt->thread)
 			break;
Index: linux-2.6/drivers/usb/core/hub.c
===================================================================
--- linux-2.6.orig/drivers/usb/core/hub.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/drivers/usb/core/hub.c	2005-07-27 19:59:59.000000000 -0700
@@ -2812,7 +2812,7 @@
 		wait_event_interruptible(khubd_wait,
 				!list_empty(&hub_event_list) ||
 				kthread_should_stop());
-		try_to_freeze();
+		try_todo_list();
 	} while (!kthread_should_stop() || !list_empty(&hub_event_list));
 
 	pr_debug("%s: khubd exiting\n", usbcore_name);
Index: linux-2.6/drivers/usb/storage/usb.c
===================================================================
--- linux-2.6.orig/drivers/usb/storage/usb.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/drivers/usb/storage/usb.c	2005-07-27 19:59:59.000000000 -0700
@@ -847,7 +847,7 @@
 		wait_event_interruptible_timeout(us->delay_wait,
 				test_bit(US_FLIDX_DISCONNECTING, &us->flags),
 				delay_use * HZ);
-		if (try_to_freeze())
+		if (try_todo_list())
 			goto retry;
 	}
 
Index: linux-2.6/fs/afs/kafsasyncd.c
===================================================================
--- linux-2.6.orig/fs/afs/kafsasyncd.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/fs/afs/kafsasyncd.c	2005-07-27 19:59:59.000000000 -0700
@@ -116,7 +116,7 @@
 		remove_wait_queue(&kafsasyncd_sleepq, &myself);
 		set_current_state(TASK_RUNNING);
 
-		try_to_freeze();
+		try_todo_list();
 
 		/* discard pending signals */
 		afs_discard_my_signals();
Index: linux-2.6/fs/afs/kafstimod.c
===================================================================
--- linux-2.6.orig/fs/afs/kafstimod.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/fs/afs/kafstimod.c	2005-07-27 19:59:59.000000000 -0700
@@ -91,7 +91,7 @@
 			complete_and_exit(&kafstimod_dead, 0);
 		}
 
-		try_to_freeze();
+		try_todo_list();
 
 		/* discard pending signals */
 		afs_discard_my_signals();
Index: linux-2.6/fs/jbd/journal.c
===================================================================
--- linux-2.6.orig/fs/jbd/journal.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/fs/jbd/journal.c	2005-07-27 19:59:59.000000000 -0700
@@ -167,7 +167,7 @@
 	}
 
 	wake_up(&journal->j_wait_done_commit);
-	if (freezing(current)) {
+	if (todo_list_active()) {
 		/*
 		 * The simpler the better. Flushing journal isn't a
 		 * good idea, because that depends on threads that may
@@ -175,7 +175,7 @@
 		 */
 		jbd_debug(1, "Now suspending kjournald\n");
 		spin_unlock(&journal->j_state_lock);
-		refrigerator();
+		run_todo_list();
 		spin_lock(&journal->j_state_lock);
 	} else {
 		/*
Index: linux-2.6/fs/jfs/jfs_logmgr.c
===================================================================
--- linux-2.6.orig/fs/jfs/jfs_logmgr.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/fs/jfs/jfs_logmgr.c	2005-07-27 19:59:59.000000000 -0700
@@ -2360,9 +2360,9 @@
 			lbmStartIO(bp);
 			spin_lock_irq(&log_redrive_lock);
 		}
-		if (freezing(current)) {
+		if (todo_list_active()) {
 			spin_unlock_irq(&log_redrive_lock);
-			refrigerator();
+			run_todo_list();
 		} else {
 			add_wait_queue(&jfs_IO_thread_wait, &wq);
 			set_current_state(TASK_INTERRUPTIBLE);
Index: linux-2.6/fs/lockd/clntproc.c
===================================================================
--- linux-2.6.orig/fs/lockd/clntproc.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/fs/lockd/clntproc.c	2005-07-27 19:59:59.000000000 -0700
@@ -313,7 +313,7 @@
 	prepare_to_wait(queue, &wait, TASK_INTERRUPTIBLE);
 	if (!signalled ()) {
 		schedule_timeout(NLMCLNT_GRACE_WAIT);
-		try_to_freeze();
+		try_todo_list();
 		if (!signalled ())
 			status = 0;
 	}
Index: linux-2.6/fs/xfs/linux-2.6/xfs_buf.c
===================================================================
--- linux-2.6.orig/fs/xfs/linux-2.6/xfs_buf.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/fs/xfs/linux-2.6/xfs_buf.c	2005-07-27 19:59:59.000000000 -0700
@@ -1771,9 +1771,9 @@
 
 	INIT_LIST_HEAD(&tmp);
 	do {
-		if (unlikely(freezing(current))) {
+		if (unlikely(todo_list_active())) {
 			xfsbufd_force_sleep = 1;
-			refrigerator();
+			run_todo_list();
 		} else {
 			xfsbufd_force_sleep = 0;
 		}
Index: linux-2.6/fs/xfs/linux-2.6/xfs_super.c
===================================================================
--- linux-2.6.orig/fs/xfs/linux-2.6/xfs_super.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/fs/xfs/linux-2.6/xfs_super.c	2005-07-27 19:59:59.000000000 -0700
@@ -482,8 +482,8 @@
 	for (;;) {
 		set_current_state(TASK_INTERRUPTIBLE);
 		timeleft = schedule_timeout(timeleft);
-		/* swsusp */
-		try_to_freeze();
+
+		try_todo_list();
 		if (vfsp->vfs_flag & VFS_UMOUNT)
 			break;
 
Index: linux-2.6/kernel/sched.c
===================================================================
--- linux-2.6.orig/kernel/sched.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/kernel/sched.c	2005-07-27 19:59:59.000000000 -0700
@@ -4343,7 +4343,7 @@
 		struct list_head *head;
 		migration_req_t *req;
 
-		try_to_freeze();
+		try_todo_list();
 
 		spin_lock_irq(&rq->lock);
 
Index: linux-2.6/mm/pdflush.c
===================================================================
--- linux-2.6.orig/mm/pdflush.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/mm/pdflush.c	2005-07-27 19:59:59.000000000 -0700
@@ -105,7 +105,7 @@
 		spin_unlock_irq(&pdflush_lock);
 
 		schedule();
-		if (try_to_freeze()) {
+		if (try_todo_list()) {
 			spin_lock_irq(&pdflush_lock);
 			continue;
 		}
Index: linux-2.6/mm/vmscan.c
===================================================================
--- linux-2.6.orig/mm/vmscan.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/mm/vmscan.c	2005-07-27 19:59:59.000000000 -0700
@@ -1217,7 +1217,7 @@
 	for ( ; ; ) {
 		unsigned long new_order;
 
-		try_to_freeze();
+		try_todo_list();
 
 		prepare_to_wait(&pgdat->kswapd_wait, &wait, TASK_INTERRUPTIBLE);
 		new_order = pgdat->kswapd_max_order;
Index: linux-2.6/net/rxrpc/krxiod.c
===================================================================
--- linux-2.6.orig/net/rxrpc/krxiod.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/net/rxrpc/krxiod.c	2005-07-27 19:59:59.000000000 -0700
@@ -138,7 +138,7 @@
 
 		_debug("### End Work");
 
-		try_to_freeze();
+		try_todo_list();
 
                 /* discard pending signals */
 		rxrpc_discard_my_signals();
Index: linux-2.6/net/rxrpc/krxtimod.c
===================================================================
--- linux-2.6.orig/net/rxrpc/krxtimod.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/net/rxrpc/krxtimod.c	2005-07-27 19:59:59.000000000 -0700
@@ -90,7 +90,7 @@
 			complete_and_exit(&krxtimod_dead, 0);
 		}
 
-		try_to_freeze();
+		try_todo_list();
 
 		/* discard pending signals */
 		rxrpc_discard_my_signals();
Index: linux-2.6/net/sunrpc/svcsock.c
===================================================================
--- linux-2.6.orig/net/sunrpc/svcsock.c	2005-07-27 19:59:28.000000000 -0700
+++ linux-2.6/net/sunrpc/svcsock.c	2005-07-27 19:59:59.000000000 -0700
@@ -1186,7 +1186,7 @@
 	arg->len = (pages-1)*PAGE_SIZE;
 	arg->tail[0].iov_len = 0;
 
-	try_to_freeze();
+	try_todo_list();
 	if (signalled())
 		return -EINTR;
 
@@ -1227,7 +1227,7 @@
 
 		schedule_timeout(timeout);
 
-		try_to_freeze();
+		try_todo_list();
 
 		spin_lock_bh(&serv->sv_lock);
 		remove_wait_queue(&rqstp->rq_wait, &wait);
Index: linux-2.6/drivers/net/8139too.c
===================================================================
--- linux-2.6.orig/drivers/net/8139too.c	2005-07-27 18:29:20.000000000 -0700
+++ linux-2.6/drivers/net/8139too.c	2005-07-27 20:11:28.000000000 -0700
@@ -1605,8 +1605,7 @@
 		timeout = next_tick;
 		do {
 			timeout = interruptible_sleep_on_timeout (&tp->thr_wait, timeout);
-			/* make swsusp happy with our thread */
-			try_to_freeze();
+			try_todo_list();
 		} while (!signal_pending (current) && (timeout > 0));
 
 		if (signal_pending (current)) {

Re: Alternative to TIF_FREEZE -> a notifier in the task_struct?

[Pavel Machek]

Hi!

> This could be used to have a process execute any other piece that is 
> required to run in the context of the thread. Maybe such a feature could 
> help with PTRACE or/and get_user_pages that currently do ugly things to a 
> process. It may also allow changing values that so far cannot be
> changed from the outside in the task struct by running a function
> in the context of the process.

Well, we really need slightly more from "running in process context":
we also need "no locks held" for swsusp. (But other uses probably do,
too?)

> Here is a patch against Linus current tree that does this. Note that this 
> patch is incomplete at this point and only a basis for further discussion. 
> Not all software suspend checkpoints are useful for a notifier chain. We 
> would need to inspect several cases where drivers/kernel threads have 
> special functionality for software suspend.

I guess I'd prefer if you left "refrigerator()" and "try_to_freeze()"
functions in; there are about 1000 drivers that know/use them, and
some patches are probably in the queue....

								Pavel
-- 
teflon -- maybe it is a trademark, but it should not be.

Re: Alternative to TIF_FREEZE -> a notifier in the task_struct?

[Christoph Lameter]

On Thu, 28 Jul 2005, Pavel Machek wrote:

> > process. It may also allow changing values that so far cannot be
> > changed from the outside in the task struct by running a function
> > in the context of the process.
> 
> Well, we really need slightly more from "running in process context":
> we also need "no locks held" for swsusp. (But other uses probably do,
> too?)

Other uses also need "no locks held".

> I guess I'd prefer if you left "refrigerator()" and "try_to_freeze()"
> functions in; there are about 1000 drivers that know/use them, and
> some patches are probably in the queue....

Yeah but then other uses also could benefit from that.

Re: Alternative to TIF_FREEZE -> a notifier in the task_struct?

[Pavel Machek]

Hi!

> > I guess I'd prefer if you left "refrigerator()" and "try_to_freeze()"
> > functions in; there are about 1000 drivers that know/use them, and
> > some patches are probably in the queue....
> 
> Yeah but then other uses also could benefit from that.

Just don't rename try_to_freeze() to whatever... make it do whatever
you need, but don't change the name so that we don't have to fix it at
1000 places...

Or introduce new name, but keep try_to_freeze in, just calling "new"
function, so all 1000 usews don't have to be changed at the same time.

								Pavel
-- 
teflon -- maybe it is a trademark, but it should not be.

Re: Alternative to TIF_FREEZE -> a notifier in the task_struct?

[Christoph Lameter]

On Thu, 28 Jul 2005, Pavel Machek wrote:

> > > I guess I'd prefer if you left "refrigerator()" and "try_to_freeze()"
> > > functions in; there are about 1000 drivers that know/use them, and
> > > some patches are probably in the queue....
> > 
> > Yeah but then other uses also could benefit from that.
> 
> Just don't rename try_to_freeze() to whatever... make it do whatever
> you need, but don't change the name so that we don't have to fix it at
> 1000 places...

Ok. Patchset will follow soon.

[PATCH 1/4] Task notifier: Allow the removal of a notifier from the notifier handler

[Christoph Lameter]

Allow a notifier to remove itself from the notifier list.

This is done by retrieving the pointer to the next notifier from the list before the
notifier call. If a notifier removes itself in the current kernel then the 
pointer to the current notifier is invalid and notifier_call_chain 
will fail.

Signed-off-by: Christoph Lameter <christoph@lameter.com>

Index: linux-2.6.13-rc3/kernel/sys.c
===================================================================
--- linux-2.6.13-rc3.orig/kernel/sys.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/kernel/sys.c	2005-07-28 11:26:00.000000000 -0700
@@ -171,15 +171,18 @@
 {
 	int ret=NOTIFY_DONE;
 	struct notifier_block *nb = *n;
+	struct notifier_block *next;
 
 	while(nb)
 	{
-		ret=nb->notifier_call(nb,val,v);
+		/* Determining next here allows the notifier to unregister itself */
+		next = nb->next;
+		ret = nb->notifier_call(nb,val,v);
 		if(ret&NOTIFY_STOP_MASK)
 		{
 			return ret;
 		}
-		nb=nb->next;
+		nb = next;
 	}
 	return ret;
 }

[PATCH 2/4] Task notifier: Implement todo list in task_struct

[Christoph Lameter]

Introduce a todo notifier in the task_struct so that a task can be told to do
certain things. Abuse the suspend hooks try_to_freeze, freezing and refrigerator
to establish checkpoints where the todo list is processed. This will break software
suspend (next patch fixes and cleans up software suspend).

Signed-off-by: Christoph Lameter <christoph@lameter.com>

Index: linux-2.6.13-rc3/include/linux/sched.h
===================================================================
--- linux-2.6.13-rc3.orig/include/linux/sched.h	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/include/linux/sched.h	2005-07-28 11:54:40.000000000 -0700
@@ -34,6 +34,7 @@
 #include <linux/percpu.h>
 #include <linux/topology.h>
 #include <linux/seccomp.h>
+#include <linux/notifier.h>
 
 struct exec_domain;
 
@@ -720,7 +721,10 @@
 	int (*notifier)(void *priv);
 	void *notifier_data;
 	sigset_t *notifier_mask;
-	
+
+	/* todo list to be executed in the context of this thread */
+	struct notifier_block *todo;
+
 	void *security;
 	struct audit_context *audit_context;
 	seccomp_t seccomp;
@@ -1273,79 +1275,37 @@
 
 #endif
 
-#ifdef CONFIG_PM
 /*
- * Check if a process has been frozen
+ * Check if there is a todo list request
  */
-static inline int frozen(struct task_struct *p)
+static inline int todo_list_active(void)
 {
-	return p->flags & PF_FROZEN;
+	return current->todo != NULL;
 }
 
-/*
- * Check if there is a request to freeze a process
- */
-static inline int freezing(struct task_struct *p)
+static inline void run_todo_list(void)
 {
-	return p->flags & PF_FREEZE;
+	notifier_call_chain(&current->todo, 0, current);
 }
 
-/*
- * Request that a process be frozen
- * FIXME: SMP problem. We may not modify other process' flags!
- */
-static inline void freeze(struct task_struct *p)
+static inline int try_todo_list(void)
 {
-	p->flags |= PF_FREEZE;
-}
-
-/*
- * Wake up a frozen process
- */
-static inline int thaw_process(struct task_struct *p)
-{
-	if (frozen(p)) {
-		p->flags &= ~PF_FROZEN;
-		wake_up_process(p);
-		return 1;
-	}
-	return 0;
-}
-
-/*
- * freezing is complete, mark process as frozen
- */
-static inline void frozen_process(struct task_struct *p)
-{
-	p->flags = (p->flags & ~PF_FREEZE) | PF_FROZEN;
-}
-
-extern void refrigerator(void);
-extern int freeze_processes(void);
-extern void thaw_processes(void);
-
-static inline int try_to_freeze(void)
-{
-	if (freezing(current)) {
-		refrigerator();
+	if (todo_list_active()) {
+		run_todo_list();
 		return 1;
 	} else
 		return 0;
 }
-#else
-static inline int frozen(struct task_struct *p) { return 0; }
-static inline int freezing(struct task_struct *p) { return 0; }
-static inline void freeze(struct task_struct *p) { BUG(); }
-static inline int thaw_process(struct task_struct *p) { return 1; }
-static inline void frozen_process(struct task_struct *p) { BUG(); }
-
-static inline void refrigerator(void) {}
-static inline int freeze_processes(void) { BUG(); return 0; }
-static inline void thaw_processes(void) {}
 
-static inline int try_to_freeze(void) { return 0; }
+/*
+ * Compatibility definitions to use the suspend checkpoints for the task todo list.
+ * These may be removed once all uses of try_to_free,  refrigerator and freezing
+ * have been removed.
+ */
+#define try_to_freeze try_todo_list
+#define refrigerator run_todo_list
+#define freezing(p) todo_list_active()
 
-#endif /* CONFIG_PM */
 #endif /* __KERNEL__ */
 
 #endif
Index: linux-2.6.13-rc3/kernel/signal.c
===================================================================
--- linux-2.6.13-rc3.orig/kernel/signal.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/kernel/signal.c	2005-07-28 10:46:49.000000000 -0700
@@ -213,7 +213,7 @@
 fastcall void recalc_sigpending_tsk(struct task_struct *t)
 {
 	if (t->signal->group_stop_count > 0 ||
-	    (freezing(t)) ||
+	    (t->todo) ||
 	    PENDING(&t->pending, &t->blocked) ||
 	    PENDING(&t->signal->shared_pending, &t->blocked))
 		set_tsk_thread_flag(t, TIF_SIGPENDING);
@@ -2231,7 +2231,7 @@
 			current->state = TASK_INTERRUPTIBLE;
 			timeout = schedule_timeout(timeout);
 
-			try_to_freeze();
+			try_todo_list();
 			spin_lock_irq(&current->sighand->siglock);
 			sig = dequeue_signal(current, &these, &info);
 			current->blocked = current->real_blocked;
Index: linux-2.6.13-rc3/arch/frv/kernel/signal.c
===================================================================
--- linux-2.6.13-rc3.orig/arch/frv/kernel/signal.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/arch/frv/kernel/signal.c	2005-07-28 10:46:49.000000000 -0700
@@ -536,7 +536,7 @@
 	if (!user_mode(regs))
 		return 1;
 
-	if (try_to_freeze())
+	if (try_todo_list())
 		goto no_signal;
 
 	if (!oldset)
Index: linux-2.6.13-rc3/arch/h8300/kernel/signal.c
===================================================================
--- linux-2.6.13-rc3.orig/arch/h8300/kernel/signal.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/arch/h8300/kernel/signal.c	2005-07-28 10:46:49.000000000 -0700
@@ -517,7 +517,7 @@
 	if ((regs->ccr & 0x10))
 		return 1;
 
-	if (try_to_freeze())
+	if (try_todo_list())
 		goto no_signal;
 
 	current->thread.esp0 = (unsigned long) regs;
Index: linux-2.6.13-rc3/arch/i386/kernel/io_apic.c
===================================================================
--- linux-2.6.13-rc3.orig/arch/i386/kernel/io_apic.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/arch/i386/kernel/io_apic.c	2005-07-28 10:46:49.000000000 -0700
@@ -574,7 +574,7 @@
 	for ( ; ; ) {
 		set_current_state(TASK_INTERRUPTIBLE);
 		time_remaining = schedule_timeout(time_remaining);
-		try_to_freeze();
+		try_todo_list();
 		if (time_after(jiffies,
 				prev_balance_time+balanced_irq_interval)) {
 			preempt_disable();
Index: linux-2.6.13-rc3/arch/i386/kernel/signal.c
===================================================================
--- linux-2.6.13-rc3.orig/arch/i386/kernel/signal.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/arch/i386/kernel/signal.c	2005-07-28 10:46:49.000000000 -0700
@@ -608,7 +608,7 @@
 	if (!user_mode(regs))
 		return 1;
 
-	if (try_to_freeze())
+	if (try_todo_list())
 		goto no_signal;
 
 	if (!oldset)
Index: linux-2.6.13-rc3/arch/m32r/kernel/signal.c
===================================================================
--- linux-2.6.13-rc3.orig/arch/m32r/kernel/signal.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/arch/m32r/kernel/signal.c	2005-07-28 10:46:49.000000000 -0700
@@ -371,7 +371,7 @@
 	if (!user_mode(regs))
 		return 1;
 
-	if (try_to_freeze()) 
+	if (try_todo_list())
 		goto no_signal;
 
 	if (!oldset)
Index: linux-2.6.13-rc3/arch/ppc/kernel/signal.c
===================================================================
--- linux-2.6.13-rc3.orig/arch/ppc/kernel/signal.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/arch/ppc/kernel/signal.c	2005-07-28 10:46:49.000000000 -0700
@@ -705,7 +705,7 @@
 	unsigned long frame, newsp;
 	int signr, ret;
 
-	if (try_to_freeze()) {
+	if (try_todo_list()) {
 		signr = 0;
 		if (!signal_pending(current))
 			goto no_signal;
Index: linux-2.6.13-rc3/arch/x86_64/kernel/signal.c
===================================================================
--- linux-2.6.13-rc3.orig/arch/x86_64/kernel/signal.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/arch/x86_64/kernel/signal.c	2005-07-28 10:46:49.000000000 -0700
@@ -425,7 +425,7 @@
 	if (!user_mode(regs))
 		return 1;
 
-	if (try_to_freeze())
+	if (try_todo_list())
 		goto no_signal;
 
 	if (!oldset)

[PATCH 3/4] Task notifier: Make suspend code SMP safe using the todo list in the task struct

[Christoph Lameter]

Make the suspend code use the todo list in the task_struct

This patch makes the suspend code SMP clean by removing PF_FREEZE and PF_FROZEN. Instead
it relies on the new notification handler in the task_struct, a completion handler and an
atomic counter for the number of processes frozen. All the logic is local to the
suspend code and no longer requires changes to other kernel components.

Signed-off-by: Christoph Lameter <christoph@lameter.com>

Index: linux-2.6.13-rc3/include/linux/sched.h
===================================================================
--- linux-2.6.13-rc3.orig/include/linux/sched.h	2005-07-28 12:21:11.000000000 -0700
+++ linux-2.6.13-rc3/include/linux/sched.h	2005-07-28 12:39:37.000000000 -0700
@@ -815,9 +815,7 @@
 #define PF_MEMALLOC	0x00000800	/* Allocating memory */
 #define PF_FLUSHER	0x00001000	/* responsible for disk writeback */
 #define PF_USED_MATH	0x00002000	/* if unset the fpu must be initialized before use */
-#define PF_FREEZE	0x00004000	/* this task is being frozen for suspend now */
 #define PF_NOFREEZE	0x00008000	/* this thread should not be frozen */
-#define PF_FROZEN	0x00010000	/* frozen for system suspend */
 #define PF_FSTRANS	0x00020000	/* inside a filesystem transaction */
 #define PF_KSWAPD	0x00040000	/* I am kswapd */
 #define PF_SWAPOFF	0x00080000	/* I am in swapoff */
Index: linux-2.6.13-rc3/kernel/power/process.c
===================================================================
--- linux-2.6.13-rc3.orig/kernel/power/process.c	2005-07-28 12:07:15.000000000 -0700
+++ linux-2.6.13-rc3/kernel/power/process.c	2005-07-28 12:45:09.000000000 -0700
@@ -18,6 +18,8 @@
  */
 #define TIMEOUT	(6 * HZ)
 
+DECLARE_COMPLETION(thaw);
+static atomic_t nr_frozen;
 
 static inline int freezeable(struct task_struct * p)
 {
@@ -31,26 +33,36 @@
 	return 1;
 }
 
-/* Refrigerator is place where frozen processes are stored :-). */
-void refrigerator(void)
+static int freeze_process(struct notifier_block *nl, unsigned long x, void *v)
 {
 	/* Hmm, should we be allowed to suspend when there are realtime
 	   processes around? */
 	long save;
 	save = current->state;
-	current->state = TASK_UNINTERRUPTIBLE;
-	pr_debug("%s entered refrigerator\n", current->comm);
+	pr_debug("%s frozen\n", current->comm);
 	printk("=");
 
-	frozen_process(current);
 	spin_lock_irq(&current->sighand->siglock);
 	recalc_sigpending(); /* We sent fake signal, clean it up */
+	atomic_inc(&nr_frozen);
 	spin_unlock_irq(&current->sighand->siglock);
 
-	while (frozen(current))
-		schedule();
-	pr_debug("%s left refrigerator\n", current->comm);
+	wait_for_completion(&thaw);
+	atomic_dec(&nr_frozen);
+	notifier_chain_unregister(&current->todo, nl);
+	kfree(nl);
+	pr_debug("%s thawed\n", current->comm);
 	current->state = save;
+	return 0;
+}
+
+void thaw_processes(void)
+{
+	printk( "Restarting tasks..." );
+	complete_all(&thaw);
+	while (atomic_read(&nr_frozen) > 0)
+		schedule();
+	printk( " done\n" );
 }
 
 /* 0 = success, else # of processes that we failed to stop */
@@ -61,19 +73,32 @@
 	struct task_struct *g, *p;
 	unsigned long flags;
 
+	atomic_set(&nr_frozen, 0);
+	INIT_COMPLETION(thaw);
+
 	printk( "Stopping tasks: " );
 	start_time = jiffies;
 	do {
 		todo = 0;
 		read_lock(&tasklist_lock);
 		do_each_thread(g, p) {
+			struct notifier_block *n;
+
 			if (!freezeable(p))
 				continue;
-			if (frozen(p))
-				continue;
 
-			freeze(p);
+			/* If there is nothing on the todo list then get the process to freeze itself */
+			if (!todo_list_active()) {
+				n = kmalloc(sizeof(struct notifier_block), GFP_ATOMIC);
+				if (n) {
+					n->notifier_call = freeze_process;
+					n->priority = 0;
+					notifier_chain_register(&g->todo, n);
+				}
+		 	}
+			/* Make the process work on its todo list */
 			spin_lock_irqsave(&p->sighand->siglock, flags);
+			recalc_sigpending();
 			signal_wake_up(p, 0);
 			spin_unlock_irqrestore(&p->sighand->siglock, flags);
 			todo++;
@@ -83,31 +108,13 @@
 		if (time_after(jiffies, start_time + TIMEOUT)) {
 			printk( "\n" );
 			printk(KERN_ERR " stopping tasks failed (%d tasks remaining)\n", todo );
+			thaw_processes();
 			return todo;
 		}
-	} while(todo);
+	} while(todo < atomic_read(&nr_frozen));
 
 	printk( "|\n" );
 	BUG_ON(in_atomic());
 	return 0;
 }
 
-void thaw_processes(void)
-{
-	struct task_struct *g, *p;
-
-	printk( "Restarting tasks..." );
-	read_lock(&tasklist_lock);
-	do_each_thread(g, p) {
-		if (!freezeable(p))
-			continue;
-		if (!thaw_process(p))
-			printk(KERN_INFO " Strange, %s not stopped\n", p->comm );
-	} while_each_thread(g, p);
-
-	read_unlock(&tasklist_lock);
-	schedule();
-	printk( " done\n" );
-}
-
-EXPORT_SYMBOL(refrigerator);
Index: linux-2.6.13-rc3/Documentation/power/kernel_threads.txt
===================================================================
--- linux-2.6.13-rc3.orig/Documentation/power/kernel_threads.txt	2005-07-28 12:07:15.000000000 -0700
+++ linux-2.6.13-rc3/Documentation/power/kernel_threads.txt	2005-07-28 12:39:37.000000000 -0700
@@ -4,15 +4,15 @@
 Freezer
 
 Upon entering a suspended state the system will freeze all
-tasks. This is done by delivering pseudosignals. This affects
-kernel threads, too. To successfully freeze a kernel thread
-the thread has to check for the pseudosignal and enter the
-refrigerator. Code to do this looks like this:
+tasks. This is done by making all processes execute a notifier.
+This affects kernel threads, too. To successfully freeze a kernel thread
+the thread has to check for the notifications and call the notifier
+chain for the process. Code to do this looks like this:
 
 	do {
 		hub_events();
 		wait_event_interruptible(khubd_wait, !list_empty(&hub_event_list));
-		try_to_freeze();
+		try_todo_list();
 	} while (!signal_pending(current));
 
 from drivers/usb/core/hub.c::hub_thread()
Index: linux-2.6.13-rc3/Documentation/power/swsusp.txt
===================================================================
--- linux-2.6.13-rc3.orig/Documentation/power/swsusp.txt	2005-07-28 12:07:15.000000000 -0700
+++ linux-2.6.13-rc3/Documentation/power/swsusp.txt	2005-07-28 12:39:37.000000000 -0700
@@ -155,7 +155,8 @@
 website, and not to the Linux Kernel Mailing List. We are working
 toward merging suspend2 into the mainline kernel.
 
-Q: A kernel thread must voluntarily freeze itself (call 'refrigerator').
+Q: A kernel thread must work on the todo list (call 'run_todo_list')
+to enter the refrigerator.
 I found some kernel threads that don't do it, and they don't freeze
 so the system can't sleep. Is this a known behavior?
 
@@ -164,7 +165,7 @@
 should be held at that point and it must be safe to sleep there), and
 add:
 
-       try_to_freeze();
+       try_todo_list();
 
 If the thread is needed for writing the image to storage, you should
 instead set the PF_NOFREEZE process flag when creating the thread (and

[PATCH 4/4] Task notifier: s/try_to_freeze/try_todo_list/ in some drivers [optional]

[Christoph Lameter]

Patch to replace try_to_freeze with try_todo_list 

Replaces:

try_to_freeze -> try_todo_list
freezing -> todo_listactive
refrigerator -> run_todo_list

This patch is incomplete. Drivers may continue using try_to_freeze, freezing
and refrigerators since the above mapping is also provided by macros in
include/linux/sched.h.

At some point--when all drivers have been changed--the macros in include/linux/sched.h
may be removed.

Signed-off-by: Christoph Lameter <christoph@lameter.com>

Index: linux-2.6.13-rc3/drivers/block/pktcdvd.c
===================================================================
--- linux-2.6.13-rc3.orig/drivers/block/pktcdvd.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/drivers/block/pktcdvd.c	2005-07-28 12:45:13.000000000 -0700
@@ -1250,8 +1250,7 @@
 			residue = schedule_timeout(min_sleep_time);
 			VPRINTK("kcdrwd: wake up\n");
 
-			/* make swsusp happy with our thread */
-			try_to_freeze();
+			try_todo_list();
 
 			list_for_each_entry(pkt, &pd->cdrw.pkt_active_list, list) {
 				if (!pkt->sleep_time)
Index: linux-2.6.13-rc3/drivers/ieee1394/ieee1394_core.c
===================================================================
--- linux-2.6.13-rc3.orig/drivers/ieee1394/ieee1394_core.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/drivers/ieee1394/ieee1394_core.c	2005-07-28 12:45:13.000000000 -0700
@@ -1044,7 +1044,7 @@
 
 	while (1) {
 		if (down_interruptible(&khpsbpkt_sig)) {
-			if (try_to_freeze())
+			if (try_todo_list())
 				continue;
 			printk("khpsbpkt: received unexpected signal?!\n" );
 			break;
Index: linux-2.6.13-rc3/drivers/ieee1394/nodemgr.c
===================================================================
--- linux-2.6.13-rc3.orig/drivers/ieee1394/nodemgr.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/drivers/ieee1394/nodemgr.c	2005-07-28 12:45:13.000000000 -0700
@@ -1510,7 +1510,7 @@
 
 		if (down_interruptible(&hi->reset_sem) ||
 		    down_interruptible(&nodemgr_serialize)) {
-			if (try_to_freeze())
+			if (try_todo_list())
 				continue;
 			printk("NodeMgr: received unexpected signal?!\n" );
 			break;
Index: linux-2.6.13-rc3/drivers/input/gameport/gameport.c
===================================================================
--- linux-2.6.13-rc3.orig/drivers/input/gameport/gameport.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/drivers/input/gameport/gameport.c	2005-07-28 12:45:13.000000000 -0700
@@ -435,7 +435,7 @@
 		gameport_handle_events();
 		wait_event_interruptible(gameport_wait,
 			kthread_should_stop() || !list_empty(&gameport_event_list));
-		try_to_freeze();
+		try_todo_list();
 	} while (!kthread_should_stop());
 
 	printk(KERN_DEBUG "gameport: kgameportd exiting\n");
Index: linux-2.6.13-rc3/drivers/input/serio/serio.c
===================================================================
--- linux-2.6.13-rc3.orig/drivers/input/serio/serio.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/drivers/input/serio/serio.c	2005-07-28 12:45:13.000000000 -0700
@@ -371,7 +371,7 @@
 		serio_handle_events();
 		wait_event_interruptible(serio_wait,
 			kthread_should_stop() || !list_empty(&serio_event_list));
-		try_to_freeze();
+		try_todo_list();
 	} while (!kthread_should_stop());
 
 	printk(KERN_DEBUG "serio: kseriod exiting\n");
Index: linux-2.6.13-rc3/drivers/media/dvb/dvb-core/dvb_frontend.c
===================================================================
--- linux-2.6.13-rc3.orig/drivers/media/dvb/dvb-core/dvb_frontend.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/drivers/media/dvb/dvb-core/dvb_frontend.c	2005-07-28 12:45:13.000000000 -0700
@@ -394,7 +394,7 @@
 			break;
 		}
 
-		try_to_freeze();
+		try_todo_list();
 
 		if (down_interruptible(&fepriv->sem))
 			break;
Index: linux-2.6.13-rc3/drivers/net/irda/stir4200.c
===================================================================
--- linux-2.6.13-rc3.orig/drivers/net/irda/stir4200.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/drivers/net/irda/stir4200.c	2005-07-28 12:45:13.000000000 -0700
@@ -763,7 +763,7 @@
 	{
 #ifdef CONFIG_PM
 		/* if suspending, then power off and wait */
-		if (unlikely(freezing(current))) {
+		if (unlikely(todo_list_active())) {
 			if (stir->receiving)
 				receive_stop(stir);
 			else
@@ -771,7 +771,7 @@
 
 			write_reg(stir, REG_CTRL1, CTRL1_TXPWD|CTRL1_RXPWD);
 
-			refrigerator();
+			run_todo_list();
 
 			if (change_speed(stir, stir->speed))
 				break;
Index: linux-2.6.13-rc3/drivers/pcmcia/cs.c
===================================================================
--- linux-2.6.13-rc3.orig/drivers/pcmcia/cs.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/drivers/pcmcia/cs.c	2005-07-28 12:45:14.000000000 -0700
@@ -683,7 +683,7 @@
 		}
 
 		schedule();
-		try_to_freeze();
+		try_todo_list();
 
 		if (!skt->thread)
 			break;
Index: linux-2.6.13-rc3/drivers/usb/core/hub.c
===================================================================
--- linux-2.6.13-rc3.orig/drivers/usb/core/hub.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/drivers/usb/core/hub.c	2005-07-28 12:45:14.000000000 -0700
@@ -2812,7 +2812,7 @@
 		wait_event_interruptible(khubd_wait,
 				!list_empty(&hub_event_list) ||
 				kthread_should_stop());
-		try_to_freeze();
+		try_todo_list();
 	} while (!kthread_should_stop() || !list_empty(&hub_event_list));
 
 	pr_debug("%s: khubd exiting\n", usbcore_name);
Index: linux-2.6.13-rc3/drivers/usb/storage/usb.c
===================================================================
--- linux-2.6.13-rc3.orig/drivers/usb/storage/usb.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/drivers/usb/storage/usb.c	2005-07-28 12:45:14.000000000 -0700
@@ -847,7 +847,7 @@
 		wait_event_interruptible_timeout(us->delay_wait,
 				test_bit(US_FLIDX_DISCONNECTING, &us->flags),
 				delay_use * HZ);
-		if (try_to_freeze())
+		if (try_todo_list())
 			goto retry;
 	}
 
Index: linux-2.6.13-rc3/fs/afs/kafsasyncd.c
===================================================================
--- linux-2.6.13-rc3.orig/fs/afs/kafsasyncd.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/fs/afs/kafsasyncd.c	2005-07-28 12:45:14.000000000 -0700
@@ -116,7 +116,7 @@
 		remove_wait_queue(&kafsasyncd_sleepq, &myself);
 		set_current_state(TASK_RUNNING);
 
-		try_to_freeze();
+		try_todo_list();
 
 		/* discard pending signals */
 		afs_discard_my_signals();
Index: linux-2.6.13-rc3/fs/afs/kafstimod.c
===================================================================
--- linux-2.6.13-rc3.orig/fs/afs/kafstimod.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/fs/afs/kafstimod.c	2005-07-28 12:45:14.000000000 -0700
@@ -91,7 +91,7 @@
 			complete_and_exit(&kafstimod_dead, 0);
 		}
 
-		try_to_freeze();
+		try_todo_list();
 
 		/* discard pending signals */
 		afs_discard_my_signals();
Index: linux-2.6.13-rc3/fs/jbd/journal.c
===================================================================
--- linux-2.6.13-rc3.orig/fs/jbd/journal.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/fs/jbd/journal.c	2005-07-28 12:45:14.000000000 -0700
@@ -167,7 +167,7 @@
 	}
 
 	wake_up(&journal->j_wait_done_commit);
-	if (freezing(current)) {
+	if (todo_list_active()) {
 		/*
 		 * The simpler the better. Flushing journal isn't a
 		 * good idea, because that depends on threads that may
@@ -175,7 +175,7 @@
 		 */
 		jbd_debug(1, "Now suspending kjournald\n");
 		spin_unlock(&journal->j_state_lock);
-		refrigerator();
+		run_todo_list();
 		spin_lock(&journal->j_state_lock);
 	} else {
 		/*
Index: linux-2.6.13-rc3/fs/jfs/jfs_logmgr.c
===================================================================
--- linux-2.6.13-rc3.orig/fs/jfs/jfs_logmgr.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/fs/jfs/jfs_logmgr.c	2005-07-28 12:45:14.000000000 -0700
@@ -2359,9 +2359,9 @@
 			lbmStartIO(bp);
 			spin_lock_irq(&log_redrive_lock);
 		}
-		if (freezing(current)) {
+		if (todo_list_active()) {
 			spin_unlock_irq(&log_redrive_lock);
-			refrigerator();
+			run_todo_list();
 		} else {
 			add_wait_queue(&jfs_IO_thread_wait, &wq);
 			set_current_state(TASK_INTERRUPTIBLE);
Index: linux-2.6.13-rc3/fs/lockd/clntproc.c
===================================================================
--- linux-2.6.13-rc3.orig/fs/lockd/clntproc.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/fs/lockd/clntproc.c	2005-07-28 12:45:14.000000000 -0700
@@ -313,7 +313,7 @@
 	prepare_to_wait(queue, &wait, TASK_INTERRUPTIBLE);
 	if (!signalled ()) {
 		schedule_timeout(NLMCLNT_GRACE_WAIT);
-		try_to_freeze();
+		try_todo_list();
 		if (!signalled ())
 			status = 0;
 	}
Index: linux-2.6.13-rc3/fs/xfs/linux-2.6/xfs_buf.c
===================================================================
--- linux-2.6.13-rc3.orig/fs/xfs/linux-2.6/xfs_buf.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/fs/xfs/linux-2.6/xfs_buf.c	2005-07-28 12:45:14.000000000 -0700
@@ -1771,9 +1771,9 @@
 
 	INIT_LIST_HEAD(&tmp);
 	do {
-		if (unlikely(freezing(current))) {
+		if (unlikely(todo_list_active())) {
 			xfsbufd_force_sleep = 1;
-			refrigerator();
+			run_todo_list();
 		} else {
 			xfsbufd_force_sleep = 0;
 		}
Index: linux-2.6.13-rc3/fs/xfs/linux-2.6/xfs_super.c
===================================================================
--- linux-2.6.13-rc3.orig/fs/xfs/linux-2.6/xfs_super.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/fs/xfs/linux-2.6/xfs_super.c	2005-07-28 12:45:14.000000000 -0700
@@ -482,8 +482,8 @@
 	for (;;) {
 		set_current_state(TASK_INTERRUPTIBLE);
 		timeleft = schedule_timeout(timeleft);
-		/* swsusp */
-		try_to_freeze();
+
+		try_todo_list();
 		if (vfsp->vfs_flag & VFS_UMOUNT)
 			break;
 
Index: linux-2.6.13-rc3/kernel/sched.c
===================================================================
--- linux-2.6.13-rc3.orig/kernel/sched.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/kernel/sched.c	2005-07-28 12:45:14.000000000 -0700
@@ -4341,7 +4341,7 @@
 		struct list_head *head;
 		migration_req_t *req;
 
-		try_to_freeze();
+		try_todo_list();
 
 		spin_lock_irq(&rq->lock);
 
Index: linux-2.6.13-rc3/mm/pdflush.c
===================================================================
--- linux-2.6.13-rc3.orig/mm/pdflush.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/mm/pdflush.c	2005-07-28 12:45:14.000000000 -0700
@@ -105,7 +105,7 @@
 		spin_unlock_irq(&pdflush_lock);
 
 		schedule();
-		if (try_to_freeze()) {
+		if (try_todo_list()) {
 			spin_lock_irq(&pdflush_lock);
 			continue;
 		}
Index: linux-2.6.13-rc3/mm/vmscan.c
===================================================================
--- linux-2.6.13-rc3.orig/mm/vmscan.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/mm/vmscan.c	2005-07-28 12:45:14.000000000 -0700
@@ -1217,7 +1217,7 @@
 	for ( ; ; ) {
 		unsigned long new_order;
 
-		try_to_freeze();
+		try_todo_list();
 
 		prepare_to_wait(&pgdat->kswapd_wait, &wait, TASK_INTERRUPTIBLE);
 		new_order = pgdat->kswapd_max_order;
Index: linux-2.6.13-rc3/net/rxrpc/krxiod.c
===================================================================
--- linux-2.6.13-rc3.orig/net/rxrpc/krxiod.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/net/rxrpc/krxiod.c	2005-07-28 12:45:14.000000000 -0700
@@ -138,7 +138,7 @@
 
 		_debug("### End Work");
 
-		try_to_freeze();
+		try_todo_list();
 
                 /* discard pending signals */
 		rxrpc_discard_my_signals();
Index: linux-2.6.13-rc3/net/rxrpc/krxtimod.c
===================================================================
--- linux-2.6.13-rc3.orig/net/rxrpc/krxtimod.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/net/rxrpc/krxtimod.c	2005-07-28 12:45:14.000000000 -0700
@@ -90,7 +90,7 @@
 			complete_and_exit(&krxtimod_dead, 0);
 		}
 
-		try_to_freeze();
+		try_todo_list();
 
 		/* discard pending signals */
 		rxrpc_discard_my_signals();
Index: linux-2.6.13-rc3/net/sunrpc/svcsock.c
===================================================================
--- linux-2.6.13-rc3.orig/net/sunrpc/svcsock.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/net/sunrpc/svcsock.c	2005-07-28 12:45:14.000000000 -0700
@@ -1186,7 +1186,7 @@
 	arg->len = (pages-1)*PAGE_SIZE;
 	arg->tail[0].iov_len = 0;
 
-	try_to_freeze();
+	try_todo_list();
 	if (signalled())
 		return -EINTR;
 
@@ -1227,7 +1227,7 @@
 
 		schedule_timeout(timeout);
 
-		try_to_freeze();
+		try_todo_list();
 
 		spin_lock_bh(&serv->sv_lock);
 		remove_wait_queue(&rqstp->rq_wait, &wait);
Index: linux-2.6.13-rc3/drivers/net/8139too.c
===================================================================
--- linux-2.6.13-rc3.orig/drivers/net/8139too.c	2005-07-12 21:46:46.000000000 -0700
+++ linux-2.6.13-rc3/drivers/net/8139too.c	2005-07-28 12:45:14.000000000 -0700
@@ -1605,8 +1605,7 @@
 		timeout = next_tick;
 		do {
 			timeout = interruptible_sleep_on_timeout (&tp->thr_wait, timeout);
-			/* make swsusp happy with our thread */
-			try_to_freeze();
+			try_todo_list();
 		} while (!signal_pending (current) && (timeout > 0));
 
 		if (signal_pending (current)) {

Re: [PATCH 2/4] Task notifier: Implement todo list in task_struct

[Pavel Machek]

Hi!

> Introduce a todo notifier in the task_struct so that a task can be told to do
> certain things. Abuse the suspend hooks try_to_freeze, freezing and refrigerator
> to establish checkpoints where the todo list is processed. This will break software
> suspend (next patch fixes and cleans up software suspend).

Ugh, this conflicts with stuff in -mm tree rather badly... In part
thanks to patch by you that was already applied.

I fixed up rejects manually (but probably lost fix or two in
progress), and will test.

Uff, and then it breaks suspend completely:

Jul 28 23:21:12 amd kernel: Stopping tasks:
=====================================Restarting tasks...khpsbpkt:
received unexpected signal?!
Jul 28 23:21:12 amd kernel: NodeMgr: received unexpected signal?!
Jul 28 23:21:22 amd kernel:  done
Jul 28 23:21:32 amd pam_limits[1547]: wrong limit value 'unlimited'

(Left me with basically all kernel threads going crazy:)

top - 23:22:34 up 3 min,  4 users,  load average: 5.10, 1.90, 0.69
Tasks:  48 total,   2 running,  46 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0% us, 100.0% sy,  0.0% ni,  0.0% id,  0.0% wa,  0.0% hi,
0.0% si
Mem:   2064480k total,   155664k used,  1908816k free,    10280k
buffers
Swap:  2136448k total,        0k used,  2136448k free,   114624k
cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
  273 root      25   0     0    0    0 S 21.1  0.0   0:11.24 kswapd0
  938 root      25   0     0    0    0 S 21.1  0.0   0:11.24 pccardd
  940 root      25   0     0    0    0 S 21.1  0.0   0:11.30 pccardd
 1060 root      25   0     0    0    0 R 21.1  0.0   0:21.34 kjournald
 1409 root      25   0     0    0    0 S 17.3  0.0   0:19.71 kjournald

You are doing rather intrusive changes. Is testing them too much to
ask? I'm pretty sure you can get i386 machine to test swsusp on...
								Pavel
-- 
teflon -- maybe it is a trademark, but it should not be.

Re: [PATCH 2/4] Task notifier: Implement todo list in task_struct

[Pavel Machek]

Hi!

> > Introduce a todo notifier in the task_struct so that a task can be told to do
> > certain things. Abuse the suspend hooks try_to_freeze, freezing and refrigerator
> > to establish checkpoints where the todo list is processed. This will break software
> > suspend (next patch fixes and cleans up software suspend).
> 
> Ugh, this conflicts with stuff in -mm tree rather badly... In part
> thanks to patch by you that was already applied.
> 
> I fixed up rejects manually (but probably lost fix or two in
> progress), and will test.
> 
> Uff, and then it breaks suspend completely:
> 
> Jul 28 23:21:12 amd kernel: Stopping tasks:
> =====================================Restarting tasks...khpsbpkt:
> received unexpected signal?!
> Jul 28 23:21:12 amd kernel: NodeMgr: received unexpected signal?!
> Jul 28 23:21:22 amd kernel:  done
> Jul 28 23:21:32 amd pam_limits[1547]: wrong limit value 'unlimited'
> 
> (Left me with basically all kernel threads going crazy:)
> 
> top - 23:22:34 up 3 min,  4 users,  load average: 5.10, 1.90, 0.69
> Tasks:  48 total,   2 running,  46 sleeping,   0 stopped,   0 zombie
> Cpu(s):  0.0% us, 100.0% sy,  0.0% ni,  0.0% id,  0.0% wa,  0.0% hi,
> 0.0% si
> Mem:   2064480k total,   155664k used,  1908816k free,    10280k
> buffers
> Swap:  2136448k total,        0k used,  2136448k free,   114624k
> cached
> 
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>   273 root      25   0     0    0    0 S 21.1  0.0   0:11.24 kswapd0
>   938 root      25   0     0    0    0 S 21.1  0.0   0:11.24 pccardd
>   940 root      25   0     0    0    0 S 21.1  0.0   0:11.30 pccardd
>  1060 root      25   0     0    0    0 R 21.1  0.0   0:21.34 kjournald
>  1409 root      25   0     0    0    0 S 17.3  0.0   0:19.71 kjournald
> 
> You are doing rather intrusive changes. Is testing them too much to
> ask? I'm pretty sure you can get i386 machine to test swsusp on...

(And yes, I did apply the whole series. It would be nice if next
series was relative to -mm, it already contains some of your changes).

								Pavel
-- 
teflon -- maybe it is a trademark, but it should not be.

Re: [PATCH 2/4] Task notifier: Implement todo list in task_struct

[Christoph Lameter]

On Thu, 28 Jul 2005, Pavel Machek wrote:

> Hi!
> 
> > > Introduce a todo notifier in the task_struct so that a task can be told to do
> > > certain things. Abuse the suspend hooks try_to_freeze, freezing and refrigerator
> > > to establish checkpoints where the todo list is processed. This will break software
> > > suspend (next patch fixes and cleans up software suspend).
> > 
> > Ugh, this conflicts with stuff in -mm tree rather badly... In part
> > thanks to patch by you that was already applied.
> > 
> > I fixed up rejects manually (but probably lost fix or two in
> > progress), and will test.

Dont fix it up. Remove the ealier patch.

> > 
> > You are doing rather intrusive changes. Is testing them too much to
> > ask? I'm pretty sure you can get i386 machine to test swsusp on...
> 
> (And yes, I did apply the whole series. It would be nice if next
> series was relative to -mm, it already contains some of your changes).

mm contains the TIF_FREEZE changes that need to be undone. And yes I 
tested it on a i386 with suspend to disk and it worked fine.

Re: [PATCH 2/4] Task notifier: Implement todo list in task_struct

[Pavel Machek]

Hi!

> > > > Introduce a todo notifier in the task_struct so that a task can be told to do
> > > > certain things. Abuse the suspend hooks try_to_freeze, freezing and refrigerator
> > > > to establish checkpoints where the todo list is processed. This will break software
> > > > suspend (next patch fixes and cleans up software suspend).
> > > 
> > > Ugh, this conflicts with stuff in -mm tree rather badly... In part
> > > thanks to patch by you that was already applied.
> > > 
> > > I fixed up rejects manually (but probably lost fix or two in
> > > progress), and will test.
> 
> Dont fix it up. Remove the ealier patch.

Oops. Do you happen to have patch relative to -mm or something? I'd
prefer not to mess it up second time...

> > > You are doing rather intrusive changes. Is testing them too much to
> > > ask? I'm pretty sure you can get i386 machine to test swsusp on...
> > 
> > (And yes, I did apply the whole series. It would be nice if next
> > series was relative to -mm, it already contains some of your changes).
> 
> mm contains the TIF_FREEZE changes that need to be undone. And yes I 
> tested it on a i386 with suspend to disk and it worked fine.

Sorry.
								Pavel
-- 
teflon -- maybe it is a trademark, but it should not be.

Re: [PATCH 2/4] Task notifier: Implement todo list in task_struct

[Christoph Lameter]

On Fri, 29 Jul 2005, Pavel Machek wrote:

> > Dont fix it up. Remove the ealier patch.
> 
> Oops. Do you happen to have patch relative to -mm or something? I'd
> prefer not to mess it up second time...

Ok. I will make a patch against mm tomorrow. Patches 
are typically against Linus latest and if you test against mm then you may 
see breakage from other patches. Plus there will be dependencies with 
other patches in mm.

Re: [PATCH 2/4] Task notifier: Implement todo list in task_struct

[Andrew Morton]

Christoph Lameter <christoph@lameter.com> wrote:
>
> On Fri, 29 Jul 2005, Pavel Machek wrote:
> 
> > > Dont fix it up. Remove the ealier patch.
> > 
> > Oops. Do you happen to have patch relative to -mm or something? I'd
> > prefer not to mess it up second time...
> 
> Ok. I will make a patch against mm tomorrow.

Well if you want to go this way I can just drop the TIF_FREEZE stuff and
use the patches-relative-to-mainline.

Re: [PATCH 2/4] Task notifier: Implement todo list in task_struct

[Christoph Lameter]

On Thu, 28 Jul 2005, Andrew Morton wrote:

> Well if you want to go this way I can just drop the TIF_FREEZE stuff and
> use the patches-relative-to-mainline.

I would appreciate that.\

Re: [PATCH 2/4] Task notifier: Implement todo list in task_struct

[Nigel Cunningham]

Hi.

Just to let you know that I have it working with Suspend2. One thing I
am concerned about is that we still need a way of determining whether a
process has been signalled but not yet frozen. At the moment you just
check p->todo, but if/when other functionality begins to use the todo
list, this will be unreliable.

I'm happy to supply the patches I'm using if you want to compare. (I
retained the other freezer improvements, so it wouldn't just be bug
fixes against your patches).

Regards,

Nigel
-- 
Evolution.
Enumerate the requirements.
Consider the interdependencies.
Calculate the probabilities.

Re: [PATCH 2/4] Task notifier: Implement todo list in task_struct

[Christoph Lameter]

On Tue, 9 Aug 2005, Nigel Cunningham wrote:

> Just to let you know that I have it working with Suspend2. One thing I
> am concerned about is that we still need a way of determining whether a
> process has been signalled but not yet frozen. At the moment you just
> check p->todo, but if/when other functionality begins to use the todo
> list, this will be unreliable.
> 

No it wont. A process that has notifications to process should do that 
before being put into the freezer. Only after the notification list is 
empty will the process be notified and as long as the notification is 
pending no second notification should happen.

> I'm happy to supply the patches I'm using if you want to compare. (I
> retained the other freezer improvements, so it wouldn't just be bug
> fixes against your patches).

I am not sure how to sort that out. I guess post the current patches that 
you got and then we see how to continue from there.

Re: [PATCH 2/4] Task notifier: Implement todo list in task_struct

[Nigel Cunningham]

Hi.

On Tue, 2005-08-09 at 12:01, Christoph Lameter wrote:
> On Tue, 9 Aug 2005, Nigel Cunningham wrote:
> 
> > Just to let you know that I have it working with Suspend2. One thing I
> > am concerned about is that we still need a way of determining whether a
> > process has been signalled but not yet frozen. At the moment you just
> > check p->todo, but if/when other functionality begins to use the todo
> > list, this will be unreliable.
> > 
> 
> No it wont. A process that has notifications to process should do that 
> before being put into the freezer. Only after the notification list is 
> empty will the process be notified and as long as the notification is 
> pending no second notification should happen.

Sorry I wasn't clear. I was thinking of the case where a broken process
doesn't process its todo list. (May it never be, but still...). How do
we find out which one is broken? We need to traverse the todo list of
every process, checking for outstanding freeze requests.

Your reply leads me to another issue. It seems to me that you shouldn't
wait until the todo list is empty before putting the freeze request on
the todo list. If the todo list is ever used for something where it
becomes hot, you might never see the todo list empty before your
timeout, and freezing will be unreliable. Even if you do see it empty
and insert your freeze request, it might be that more work is added
afterwards, so you may as well just add the request whether or not the
queue is empty.

Of course if you address this, you then have the problem that the
calling routine hammers on routines until they enter the fridge, you can
end up with multiple freeze requests per process. That could be
addressed by splitting it into two loops - the first adds the todo work
for each thread to be frozen, and the second wakes them until they
process the notification. I reckon that we shouldn't need to wake them
repeatedly like this, but I've never taken the time to see why a process
could need multiple wakeups. (I believe it's not bogus).

> > I'm happy to supply the patches I'm using if you want to compare. (I
> > retained the other freezer improvements, so it wouldn't just be bug
> > fixes against your patches).
> 
> I am not sure how to sort that out. I guess post the current patches that 
> you got and then we see how to continue from there.

Will do shortly. Just have to go talk to my boss first :> (Separate
issue).

Nigel
-- 
Evolution.
Enumerate the requirements.
Consider the interdependencies.
Calculate the probabilities.

Re: [PATCH 2/4] Task notifier: Implement todo list in task_struct

[Nigel Cunningham]

(Sorry everyone else for emailing you too. I'm only doing so to honour
the convention of not removing people from replies.)

Hi Christoph.

As I look at the patch in preparation for sending it, I don't think I
really changed anything significant. (I didn't address the issues I
mentioned in the previous email).

The only thing I think might be worth mentioning is that I added
freeze_processes and thaw_processes declarations to
include/linux/suspend.h. I got warnings without them there - perhaps,
though, it's just due to differences in Suspend2.

Having said this, I promised to send you my version, so I've inlined it
below. As you'll see, I'm making some changes just to minimise the
differences with swsusp.

Regards,

Nigel

 Documentation/power/kernel_threads.txt |   10 +-
 Documentation/power/swsusp.txt         |    5 -
 include/linux/sched.h                  |    1 
 kernel/power/process.c                 |  114 ++++++++++++++++++---------------
 4 files changed, 72 insertions(+), 58 deletions(-)
diff -ruNp 992-suspend_fix.patch-old/Documentation/power/kernel_threads.txt 992-suspend_fix.patch-new/Documentation/power/kernel_threads.txt
--- 992-suspend_fix.patch-old/Documentation/power/kernel_threads.txt	2005-08-08 17:23:31.000000000 +1000
+++ 992-suspend_fix.patch-new/Documentation/power/kernel_threads.txt	2005-08-09 08:11:41.000000000 +1000
@@ -4,15 +4,15 @@ KERNEL THREADS
 Freezer
 
 Upon entering a suspended state the system will freeze all
-tasks. This is done by delivering pseudosignals. This affects
-kernel threads, too. To successfully freeze a kernel thread
-the thread has to check for the pseudosignal and enter the
-refrigerator. Code to do this looks like this:
+tasks. This is done by making all processes execute a notifier.
+This affects kernel threads, too. To successfully freeze a kernel thread
+the thread has to check for the notifications and call the notifier
+chain for the process. Code to do this looks like this:
 
 	do {
 		hub_events();
 		wait_event_interruptible(khubd_wait, !list_empty(&hub_event_list));
-		try_to_freeze();
+		try_todo_list();
 	} while (!signal_pending(current));
 
 from drivers/usb/core/hub.c::hub_thread()
diff -ruNp 992-suspend_fix.patch-old/Documentation/power/swsusp.txt 992-suspend_fix.patch-new/Documentation/power/swsusp.txt
--- 992-suspend_fix.patch-old/Documentation/power/swsusp.txt	2005-08-08 17:23:31.000000000 +1000
+++ 992-suspend_fix.patch-new/Documentation/power/swsusp.txt	2005-08-09 08:11:41.000000000 +1000
@@ -155,7 +155,8 @@ should be sent to the mailing list avail
 website, and not to the Linux Kernel Mailing List. We are working
 toward merging suspend2 into the mainline kernel.
 
-Q: A kernel thread must voluntarily freeze itself (call 'refrigerator').
+Q: A kernel thread must work on the todo list (call 'run_todo_list')
+to enter the refrigerator.
 I found some kernel threads that don't do it, and they don't freeze
 so the system can't sleep. Is this a known behavior?
 
@@ -164,7 +165,7 @@ place where the thread is safe to be fro
 should be held at that point and it must be safe to sleep there), and
 add:
 
-       try_to_freeze();
+       try_todo_list();
 
 If the thread is needed for writing the image to storage, you should
 instead set the PF_NOFREEZE process flag when creating the thread (and
diff -ruNp 992-suspend_fix.patch-old/include/linux/sched.h 992-suspend_fix.patch-new/include/linux/sched.h
--- 992-suspend_fix.patch-old/include/linux/sched.h	2005-08-09 11:17:49.000000000 +1000
+++ 992-suspend_fix.patch-new/include/linux/sched.h	2005-08-09 08:11:41.000000000 +1000
@@ -835,7 +835,6 @@ do { if (atomic_dec_and_test(&(tsk)->usa
 #define PF_MEMALLOC	0x00000800	/* Allocating memory */
 #define PF_FLUSHER	0x00001000	/* responsible for disk writeback */
 #define PF_USED_MATH	0x00002000	/* if unset the fpu must be initialized before use */
-#define PF_FREEZE	0x00004000	/* this task is being frozen for suspend now */
 #define PF_NOFREEZE	0x00008000	/* this thread should not be frozen */
 #define PF_FROZEN	0x00010000	/* frozen for system suspend */
 #define PF_FSTRANS	0x00020000	/* inside a filesystem transaction */
diff -ruNp 992-suspend_fix.patch-old/kernel/power/process.c 992-suspend_fix.patch-new/kernel/power/process.c
--- 992-suspend_fix.patch-old/kernel/power/process.c	2005-08-09 11:17:48.000000000 +1000
+++ 992-suspend_fix.patch-new/kernel/power/process.c	2005-08-09 11:10:59.000000000 +1000
@@ -61,6 +61,11 @@ extern void suspend_relinquish_console(v
 static void freeze_lru(void);
 extern void thaw_lru(void);
 
+DECLARE_COMPLETION(kernelspace_thaw);
+DECLARE_COMPLETION(userspace_thaw);
+static atomic_t nr_userspace_frozen;
+static atomic_t nr_kernelspace_frozen;
+
 /*
  * to_be_frozen
  *
@@ -72,7 +77,7 @@ extern void thaw_lru(void);
 static int to_be_frozen(struct task_struct * p, int type_being_frozen) {
 
 	if ((p == current) ||
-	    (frozen(p)) ||
+	    (p->flags & PF_FROZEN) ||
 	    (p->flags & PF_NOFREEZE) ||
 	    (p->exit_state == EXIT_ZOMBIE) ||
 	    (p->exit_state == EXIT_DEAD) ||
@@ -86,47 +91,68 @@ static int to_be_frozen(struct task_stru
 	return 1;
 }
 
-/**
- * refrigerator - idle routine for frozen processes
- * @flag: unsigned long, non zero if signals to be flushed.
- *
- * A routine for kernel threads which should not do work during suspend
- * to enter and spin in until the process is finished.
+/*
+ * Invoked by the task todo list notifier when the task to be
+ * frozen is running.
  */
-
-void refrigerator(void)
+static int freeze_process(struct notifier_block *nl, unsigned long x, void *v)
 {
-	unsigned long flags;
+  	/* Hmm, should we be allowed to suspend when there are realtime
+  	   processes around? */
 	long save;
+
+	save = current->state;
+	current->flags |= PF_FROZEN;
+	notifier_chain_unregister(&current->todo, nl);
+	kfree(nl);
 	
-	spin_lock_irqsave(&current->sighand->siglock, flags);
+	spin_lock_irq(&current->sighand->siglock);
 	recalc_sigpending();
-	spin_unlock_irqrestore(&current->sighand->siglock, flags);
+	spin_unlock_irq(&current->sighand->siglock);
 
-	if (unlikely(current->flags & PF_NOFREEZE)) {
-		current->flags &= ~PF_FREEZE;
-		return;
+	if (current->mm) {
+		atomic_inc(&nr_userspace_frozen);
+		wait_for_completion(&userspace_thaw);
+		atomic_dec(&nr_userspace_frozen);
+	} else {
+		atomic_inc(&nr_kernelspace_frozen);
+		wait_for_completion(&kernelspace_thaw);
+		atomic_dec(&nr_kernelspace_frozen);
 	}
 
-	save = current->state;
-	suspend_message(SUSPEND_FREEZER, SUSPEND_VERBOSE, 0,
-		"\n%s (%d) refrigerated and sigpending recalculated.",
-		current->comm, current->pid);
+	current->flags &= ~PF_FROZEN;
+	current->state = save;
 
-	frozen_process(current);
+	return 0;
+}
 
-	while (test_suspend_state(SUSPEND_FREEZER_ON) && frozen(current)) {
-		current->state = TASK_STOPPED;
-		schedule();
-		spin_lock_irqsave(&current->sighand->siglock, flags);
-		recalc_sigpending();
-		spin_unlock_irqrestore(&current->sighand->siglock, flags);
-	}
+static inline void freeze(struct task_struct *p)
+{
+	unsigned long flags;
+
+	/*
+	 * We only freeze if the todo list is empty to avoid
+	 * putting multiple freeze handlers on the todo list.
+	 */
 	
-	current->state = save;
-}
+	if (!p->todo) {
+		struct notifier_block *n;
 
+		n = kmalloc(sizeof(struct notifier_block),
+					GFP_ATOMIC);
+		if (n) {
+			n->notifier_call = freeze_process;
+			n->priority = 0;
+			notifier_chain_register(&p->todo, n);
+		}
+ 	}
 
+	/* Make the process work on its todo list */
+	spin_lock_irqsave(&p->sighand->siglock, flags);
+	recalc_sigpending();
+	signal_wake_up(p, 0);
+	spin_unlock_irqrestore(&p->sighand->siglock, flags);
+}
 
 /*
  * num_to_be_frozen
@@ -204,6 +230,12 @@ static int freeze_threads(int type, int 
 	unsigned long start_time = jiffies;
 	int result = 0, still_to_do;
 
+	if (!atomic_read(&nr_userspace_frozen))
+		init_completion(&userspace_thaw);
+	
+	if (!atomic_read(&nr_kernelspace_frozen))
+		init_completion(&kernelspace_thaw);
+
 	suspend_message(SUSPEND_FREEZER, SUSPEND_VERBOSE, 1,
 		"\n STARTING TO FREEZE TYPE %d THREADS.\n",
 		type);
@@ -388,7 +420,6 @@ aborting:
 
 void thaw_processes(int which_threads)
 {
-	struct task_struct *p, *g;
 	suspend_message(SUSPEND_FREEZER, SUSPEND_LOW, 1, "Thawing tasks\n");
 	
 	suspend_task = 0;
@@ -397,27 +428,10 @@ void thaw_processes(int which_threads)
 
 	clear_suspend_state(SUSPEND_DISABLE_SYNCING);
 	
-	preempt_disable();
-	
-	local_irq_disable();
+	complete_all(&kernelspace_thaw);
 
-	read_lock(&tasklist_lock);
-
-	do_each_thread(g, p) {
-		if (frozen(p)) {
-			if ((which_threads == FREEZER_KERNEL_THREADS) && p->mm)
-				continue;
-			suspend_message(SUSPEND_FREEZER, SUSPEND_VERBOSE, 0,
-					"Waking %5d: %s.\n", p->pid, p->comm);
-			thaw_process(p);
-		}
-	} while_each_thread(g, p);
-
-	read_unlock(&tasklist_lock);
-
-	preempt_enable();
-
-	local_irq_enable();
+	if (which_threads != FREEZER_KERNEL_THREADS)
+		complete_all(&userspace_thaw);
 
 	schedule();
 }

Re: [PATCH 2/4] Task notifier: Implement todo list in task_struct

[Christoph Lameter]

On Tue, 9 Aug 2005, Nigel Cunningham wrote:

> > No it wont. A process that has notifications to process should do that 
> > before being put into the freezer. Only after the notification list is 
> > empty will the process be notified and as long as the notification is 
> > pending no second notification should happen.
> 
> Sorry I wasn't clear. I was thinking of the case where a broken process
> doesn't process its todo list. (May it never be, but still...). How do
> we find out which one is broken? We need to traverse the todo list of
> every process, checking for outstanding freeze requests.

If a process does not handle its todo then you cannot freeze the process 
at all. Something very baaaad is going on. The current version gives up if 
a process cannot be frozen. This is still doing the same.

> Your reply leads me to another issue. It seems to me that you shouldn't
> wait until the todo list is empty before putting the freeze request on
> the todo list. If the todo list is ever used for something where it
> becomes hot, you might never see the todo list empty before your
> timeout, and freezing will be unreliable. Even if you do see it empty
> and insert your freeze request, it might be that more work is added
> afterwards, so you may as well just add the request whether or not the
> queue is empty.

Yes other tasks can be put on the notifier later. But the freeze notifier 
was first so the task is put to sleep and will run the other notifiers 
upon wakeup. The check is still okay. No additional notification should be 
added if the freeze notifier is already queued.

> > I am not sure how to sort that out. I guess post the current patches that 
> > you got and then we see how to continue from there.
> 
> Will do shortly. Just have to go talk to my boss first :> (Separate
> issue).

The patch you sent looks fine to me.