From mboxrd@z Thu Jan  1 00:00:00 1970
From: Peter Zijlstra <peterz@infradead.org>
Subject: Re: [PATCH] sched: fix the intention to re-evalute tick dependency
 for offline cpu
Date: Thu, 11 Aug 2016 16:45:11 +0200
Message-ID: <20160811144511.GR6879@twins.programming.kicks-ass.net>
References: <1470304280-3917-1-git-send-email-wanpeng.li@hotmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
	Wanpeng Li <wanpeng.li@hotmail.com>,
	Ingo Molnar <mingo@redhat.com>,
	Juri Lelli <juri.lelli@arm.com>,
	Luca Abeni <luca.abeni@unitn.it>,
	Frederic Weisbecker <fweisbec@gmail.com>
To: Wanpeng Li <kernellwp@gmail.com>
Return-path: <kvm-owner@vger.kernel.org>
Received: from merlin.infradead.org ([205.233.59.134]:36522 "EHLO
	merlin.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1751031AbcHKOpV (ORCPT <rfc822;kvm@vger.kernel.org>);
	Thu, 11 Aug 2016 10:45:21 -0400
Content-Disposition: inline
In-Reply-To: <1470304280-3917-1-git-send-email-wanpeng.li@hotmail.com>
Sender: kvm-owner@vger.kernel.org
List-ID: <kvm.vger.kernel.org>

On Thu, Aug 04, 2016 at 05:51:20PM +0800, Wanpeng Li wrote:
> From: Wanpeng Li <wanpeng.li@hotmail.com>
> 
> The dl task will be replenished after dl task timer fire and start a new 
> period. It will be enqueued and to re-evaluate its dependency on the tick 
> in order to restart it. However, if cpu is hot-unplug, irq_work_queue will 
> splash since the target cpu is offline.
> 
> As a result:
> 
>     WARNING: CPU: 2 PID: 0 at kernel/irq_work.c:69 irq_work_queue_on+0xad/0xe0
>     Call Trace:
>      dump_stack+0x99/0xd0
>      __warn+0xd1/0xf0
>      warn_slowpath_null+0x1d/0x20
>      irq_work_queue_on+0xad/0xe0
>      tick_nohz_full_kick_cpu+0x44/0x50
>      tick_nohz_dep_set_cpu+0x74/0xb0
>      enqueue_task_dl+0x226/0x480
>      activate_task+0x5c/0xa0
>      dl_task_timer+0x19b/0x2c0
>      ? push_dl_task.part.31+0x190/0x190

Hurm, so this is after hot-unplug succeeded. We get a timer (which is
also already migrated), but we enqueue the dl task on the offline CPU,
because we need to do replenish because start_dl_timer() -- see the
comment in dl_task_timer() at #ifdef CONFIG_SMP.

Then, once we've enqueued the task on the offline cpu, do we migrate it.

Bit icky that, but I don't immediately see a better way.

And I think you're right in that we don't leak the nohz state, the
migration, which we do immediately after this, takes care of that.

Juri, any opinions?