[PATCH] rcu: force all adopted callbacks are invoked before next cpu-offline

[PATCH] rcu: force all adopted callbacks are invoked before next cpu-offline

[Lai Jiangshan]

When cpu-offline/online happen continuously, a CPU may
adopts too much callbacks or some callbacks are just
moved back and forth without invoked.

It will cause a CPU has a lot of burden or some callbacks
are invoked very late or starvation.

Signed-off-by: Lai Jiangshan <laijs@cn.fujitsu.com>
---
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index 669d7fe..8af45d6 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -1811,6 +1811,23 @@ static int __cpuinit rcu_cpu_notify(struct notifier_block *self,
 	case CPU_UP_CANCELED_FROZEN:
 		rcu_offline_cpu(cpu);
 		break;
+	case CPU_POST_DEAD:
+	case CPU_POST_DEAD | CPU_TASKS_FROZEN:
+		/*
+		 * When a cpu is offline, the callbacks in the offline CPU
+		 * are orphaned and adopted by a online CPU. These callbacks
+		 * are processed as new inserted one. So in the worst case,
+		 * if the offline/online happen continuously, a CPU may
+		 * adopts too much callbacks or some callbacks are just
+		 * moved back and forth without invoked.
+		 *
+		 * To fix it, we must force all adopted callbacks are invoked
+		 * before next cpu-offline.
+		 */
+		rcu_barrier_bh();
+		rcu_barrier_sched();
+		rcu_barrier();
+		break;
 	default:
 		break;
 	}