[PATCH 1/3] sched: Add new API wake_up_if_idle() to wake up the idle cpu

[PATCH 1/3] sched: Add new API wake_up_if_idle() to wake up the idle cpu

[Chuansheng Liu]

Implementing one new API wake_up_if_idle(), which is used to
wake up the idle CPU.

Suggested-by: Andy Lutomirski <luto@amacapital.net>
Signed-off-by: Chuansheng Liu <chuansheng.liu@intel.com>
---
 include/linux/sched.h |    1 +
 kernel/sched/core.c   |   16 ++++++++++++++++
 2 files changed, 17 insertions(+)

diff --git a/include/linux/sched.h b/include/linux/sched.h
index 857ba40..3f89ac1 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1024,6 +1024,7 @@ struct sched_domain_topology_level {
 extern struct sched_domain_topology_level *sched_domain_topology;
 
 extern void set_sched_topology(struct sched_domain_topology_level *tl);
+extern void wake_up_if_idle(int cpu);
 
 #ifdef CONFIG_SCHED_DEBUG
 # define SD_INIT_NAME(type)		.name = #type
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 1211575..adf104f 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -1620,6 +1620,22 @@ static void ttwu_queue_remote(struct task_struct *p, int cpu)
 	}
 }
 
+void wake_up_if_idle(int cpu)
+{
+	struct rq *rq = cpu_rq(cpu);
+	unsigned long flags;
+
+	if (set_nr_if_polling(rq->idle)) {
+		trace_sched_wake_idle_without_ipi(cpu);
+	} else {
+		raw_spin_lock_irqsave(&rq->lock, flags);
+		if (rq->curr == rq->idle)
+			smp_send_reschedule(cpu);
+		/* Else cpu is not in idle, do nothing here */
+		raw_spin_unlock_irqrestore(&rq->lock, flags);
+	}
+}
+
 bool cpus_share_cache(int this_cpu, int that_cpu)
 {
 	return per_cpu(sd_llc_id, this_cpu) == per_cpu(sd_llc_id, that_cpu);
-- 
1.7.9.5

[PATCH 2/3] smp: re-implement the kick_all_cpus_sync() with wake_up_if_idle()

[Chuansheng Liu]

Currently using smp_call_function() just woke up the corresponding
cpu, but can not break the polling idle loop.

Here using the new sched API wake_up_if_idle() to implement it.

Signed-off-by: Chuansheng Liu <chuansheng.liu@intel.com>
---
 kernel/smp.c |   18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/kernel/smp.c b/kernel/smp.c
index aff8aa1..0b647c3 100644
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -13,6 +13,7 @@
 #include <linux/gfp.h>
 #include <linux/smp.h>
 #include <linux/cpu.h>
+#include <linux/sched.h>
 
 #include "smpboot.h"
 
@@ -677,10 +678,6 @@ void on_each_cpu_cond(bool (*cond_func)(int cpu, void *info),
 }
 EXPORT_SYMBOL(on_each_cpu_cond);
 
-static void do_nothing(void *unused)
-{
-}
-
 /**
  * kick_all_cpus_sync - Force all cpus out of idle
  *
@@ -694,8 +691,15 @@ static void do_nothing(void *unused)
  */
 void kick_all_cpus_sync(void)
 {
-	/* Make sure the change is visible before we kick the cpus */
-	smp_mb();
-	smp_call_function(do_nothing, NULL, 1);
+	int cpu;
+
+	preempt_disable();
+	for_each_online_cpu(cpu) {
+		if (cpu == smp_processor_id())
+			continue;
+
+		wake_up_if_idle(cpu);
+	}
+	preempt_enable();
 }
 EXPORT_SYMBOL_GPL(kick_all_cpus_sync);
-- 
1.7.9.5

[PATCH 3/3] cpuidle: Using the kick_all_cpus_sync() to wake up all cpus

[Chuansheng Liu]

Current latency notify callback has the same function with
kick_all_cpus_sync().

Here use it directly to remove the redundant code.

Signed-off-by: Chuansheng Liu <chuansheng.liu@intel.com>
---
 drivers/cpuidle/cpuidle.c |    7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c
index ee9df5e..7827375 100644
--- a/drivers/cpuidle/cpuidle.c
+++ b/drivers/cpuidle/cpuidle.c
@@ -530,11 +530,6 @@ EXPORT_SYMBOL_GPL(cpuidle_register);
 
 #ifdef CONFIG_SMP
 
-static void smp_callback(void *v)
-{
-	/* we already woke the CPU up, nothing more to do */
-}
-
 /*
  * This function gets called when a part of the kernel has a new latency
  * requirement.  This means we need to get all processors out of their C-state,
@@ -544,7 +539,7 @@ static void smp_callback(void *v)
 static int cpuidle_latency_notify(struct notifier_block *b,
 		unsigned long l, void *v)
 {
-	smp_call_function(smp_callback, NULL, 1);
+	kick_all_cpus_sync();
 	return NOTIFY_OK;
 }
 
-- 
1.7.9.5

Re: [PATCH 2/3] smp: re-implement the kick_all_cpus_sync() with wake_up_if_idle()

[Andy Lutomirski]

On Fri, Aug 15, 2014 at 12:01 AM, Chuansheng Liu
<chuansheng.liu@intel.com> wrote:
> Currently using smp_call_function() just woke up the corresponding
> cpu, but can not break the polling idle loop.
>
> Here using the new sched API wake_up_if_idle() to implement it.

kick_all_cpus_sync has other callers, and those other callers want the
old behavior.  I think this should be a new function.

--Andy

>
> Signed-off-by: Chuansheng Liu <chuansheng.liu@intel.com>
> ---
>  kernel/smp.c |   18 +++++++++++-------
>  1 file changed, 11 insertions(+), 7 deletions(-)
>
> diff --git a/kernel/smp.c b/kernel/smp.c
> index aff8aa1..0b647c3 100644
> --- a/kernel/smp.c
> +++ b/kernel/smp.c
> @@ -13,6 +13,7 @@
>  #include <linux/gfp.h>
>  #include <linux/smp.h>
>  #include <linux/cpu.h>
> +#include <linux/sched.h>
>
>  #include "smpboot.h"
>
> @@ -677,10 +678,6 @@ void on_each_cpu_cond(bool (*cond_func)(int cpu, void *info),
>  }
>  EXPORT_SYMBOL(on_each_cpu_cond);
>
> -static void do_nothing(void *unused)
> -{
> -}
> -
>  /**
>   * kick_all_cpus_sync - Force all cpus out of idle
>   *
> @@ -694,8 +691,15 @@ static void do_nothing(void *unused)
>   */
>  void kick_all_cpus_sync(void)
>  {
> -       /* Make sure the change is visible before we kick the cpus */
> -       smp_mb();
> -       smp_call_function(do_nothing, NULL, 1);
> +       int cpu;
> +
> +       preempt_disable();
> +       for_each_online_cpu(cpu) {
> +               if (cpu == smp_processor_id())
> +                       continue;
> +
> +               wake_up_if_idle(cpu);
> +       }
> +       preempt_enable();
>  }
>  EXPORT_SYMBOL_GPL(kick_all_cpus_sync);
> --
> 1.7.9.5
>



-- 
Andy Lutomirski
AMA Capital Management, LLC

RE: [PATCH 2/3] smp: re-implement the kick_all_cpus_sync() with wake_up_if_idle()

[Liu, Chuansheng]

Hello Andy,

> -----Original Message-----
> From: Andy Lutomirski [mailto:luto@amacapital.net]
> Sent: Friday, August 15, 2014 11:41 PM
> To: Liu, Chuansheng
> Cc: Peter Zijlstra; Daniel Lezcano; Rafael J. Wysocki; Ingo Molnar;
> linux-pm@vger.kernel.org; linux-kernel@vger.kernel.org; Liu, Changcheng;
> Wang, Xiaoming; Chakravarty, Souvik K
> Subject: Re: [PATCH 2/3] smp: re-implement the kick_all_cpus_sync() with
> wake_up_if_idle()
> 
> On Fri, Aug 15, 2014 at 12:01 AM, Chuansheng Liu
> <chuansheng.liu@intel.com> wrote:
> > Currently using smp_call_function() just woke up the corresponding
> > cpu, but can not break the polling idle loop.
> >
> > Here using the new sched API wake_up_if_idle() to implement it.
> 
> kick_all_cpus_sync has other callers, and those other callers want the
> old behavior.  I think this should be a new function.
> 
Yes, seems some current users of kick_all_cpus_sync() need IPI indeed,
will try to send out patch V2 with one new function.