Re: [tip:sched/core] sched/core: Check for available DL bandwidth in cpuset_cpu_inactive()

[Borislav Petkov <bp@alien8.de>]

On Thu, Apr 02, 2015 at 11:47:04AM -0700, tip-bot for Juri Lelli wrote:
> Commit-ID:  3c18d447b3b36a8d3c90dc37dfbd363cdb685d0a
> Gitweb:     http://git.kernel.org/tip/3c18d447b3b36a8d3c90dc37dfbd363cdb685d0a
> Author:     Juri Lelli <juri.lelli@arm.com>
> AuthorDate: Tue, 31 Mar 2015 09:53:37 +0100
> Committer:  Ingo Molnar <mingo@kernel.org>
> CommitDate: Thu, 2 Apr 2015 17:42:56 +0200
> 
> sched/core: Check for available DL bandwidth in cpuset_cpu_inactive()
> 
> Hotplug operations are destructive w.r.t. cpusets. In case such an
> operation is performed on a CPU belonging to an exlusive cpuset, the
> DL bandwidth information associated with the corresponding root
> domain is gone even if the operation fails (in sched_cpu_inactive()).
> 
> For this reason we need to move the check we currently have in
> sched_cpu_inactive() to cpuset_cpu_inactive() to prevent useless
> cpusets reconfiguration in the CPU_DOWN_FAILED path.
> 
> Signed-off-by: Juri Lelli <juri.lelli@arm.com>
> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> Cc: Juri Lelli <juri.lelli@gmail.com>
> Link: http://lkml.kernel.org/r/1427792017-7356-2-git-send-email-juri.lelli@arm.com
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
> ---
>  kernel/sched/core.c | 56 ++++++++++++++++++++++++++---------------------------
>  1 file changed, 28 insertions(+), 28 deletions(-)
> 
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 4c49e75..28b0d75 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -5337,36 +5337,13 @@ static int sched_cpu_active(struct notifier_block *nfb,
>  static int sched_cpu_inactive(struct notifier_block *nfb,
>  					unsigned long action, void *hcpu)
>  {
> -	unsigned long flags;
> -	long cpu = (long)hcpu;
> -	struct dl_bw *dl_b;
> -
>  	switch (action & ~CPU_TASKS_FROZEN) {
>  	case CPU_DOWN_PREPARE:
> -		set_cpu_active(cpu, false);
> -
> -		/* explicitly allow suspend */
> -		if (!(action & CPU_TASKS_FROZEN)) {
> -			bool overflow;
> -			int cpus;
> -
> -			rcu_read_lock_sched();
> -			dl_b = dl_bw_of(cpu);
> -
> -			raw_spin_lock_irqsave(&dl_b->lock, flags);
> -			cpus = dl_bw_cpus(cpu);
> -			overflow = __dl_overflow(dl_b, cpus, 0, 0);
> -			raw_spin_unlock_irqrestore(&dl_b->lock, flags);
> -
> -			rcu_read_unlock_sched();
> -
> -			if (overflow)
> -				return notifier_from_errno(-EBUSY);
> -		}
> +		set_cpu_active((long)hcpu, false);
>  		return NOTIFY_OK;
> +	default:
> +		return NOTIFY_DONE;
>  	}
> -
> -	return NOTIFY_DONE;
>  }
>  
>  static int __init migration_init(void)
> @@ -7006,7 +6983,6 @@ static int cpuset_cpu_active(struct notifier_block *nfb, unsigned long action,
>  		 */
>  
>  	case CPU_ONLINE:
> -	case CPU_DOWN_FAILED:
>  		cpuset_update_active_cpus(true);
>  		break;
>  	default:
> @@ -7018,8 +6994,32 @@ static int cpuset_cpu_active(struct notifier_block *nfb, unsigned long action,
>  static int cpuset_cpu_inactive(struct notifier_block *nfb, unsigned long action,
>  			       void *hcpu)
>  {
> -	switch (action) {
> +	unsigned long flags;
> +	long cpu = (long)hcpu;
> +	struct dl_bw *dl_b;
> +
> +	switch (action & ~CPU_TASKS_FROZEN) {
>  	case CPU_DOWN_PREPARE:
> +		/* explicitly allow suspend */
> +		if (!(action & CPU_TASKS_FROZEN)) {
> +			bool overflow;
> +			int cpus;
> +
> +			rcu_read_lock_sched();
> +			dl_b = dl_bw_of(cpu);
> +
> +			raw_spin_lock_irqsave(&dl_b->lock, flags);
> +			cpus = dl_bw_cpus(cpu);
> +			overflow = __dl_overflow(dl_b, cpus, 0, 0);
> +			raw_spin_unlock_irqrestore(&dl_b->lock, flags);
> +
> +			rcu_read_unlock_sched();
> +
> +			if (overflow) {
> +				trace_printk("hotplug failed for cpu %lu", cpu);

Look ma,

someone forgot debugging code:

...
[    0.000000] 	RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=2.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[    0.000000] 
[    0.000000] **********************************************************
[    0.000000] **   NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE   **
[    0.000000] **                                                      **
[    0.000000] ** trace_printk() being used. Allocating extra memory.  **
[    0.000000] **                                                      **
[    0.000000] ** This means that this is a DEBUG kernel and it is     **
[    0.000000] ** unsafe for production use.                           **
[    0.000000] **                                                      **
[    0.000000] ** If you see this message and you are not debugging    **
[    0.000000] ** the kernel, report this immediately to your vendor!  **
[    0.000000] **                                                      **
[    0.000000] **   NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE   **
[    0.000000] **********************************************************

Fix coming up.

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--