From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1755282AbdKCOrc (ORCPT <rfc822;w@1wt.eu>);
        Fri, 3 Nov 2017 10:47:32 -0400
Received: from foss.arm.com ([217.140.101.70]:44162 "EHLO foss.arm.com"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S1752076AbdKCOrb (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Fri, 3 Nov 2017 10:47:31 -0400
Message-ID: <59FC8119.8030608@arm.com>
Date: Fri, 03 Nov 2017 14:45:45 +0000
From: James Morse <james.morse@arm.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Icedove/31.6.0
MIME-Version: 1.0
To: Peter Zijlstra <peterz@infradead.org>,
        Thomas Gleixner <tglx@linutronix.de>
CC: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: get_online_cpus() from a  preemptible() context (bug?)
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Thomas, Peter,

I'm trying to work out what stops a thread being pre-empted and migrated between
calling get_online_cpus() and put_online_cpus().

According to __percpu_down_read(), its the pre-empt count:
>  * Due to having preemption disabled the decrement happens on
>  * the same CPU as the increment, avoiding the
>  * increment-on-one-CPU-and-decrement-on-another problem.


So this:
> void cpus_read_lock(void)
> {
>        percpu_down_read(&cpu_hotplug_lock);
> +
> +       /* Can we migrated before we release this per-cpu lock? */
> +       WARN_ON(preemptible());
>  }

should never fire?

It does, some of the offenders:
> kmem_cache_create
> apply_workqueue_attrs
> stop_machine
> static_key_enable
> lru_add_drain_all
> __cpuhp_setup_state
> kmem_cache_shrink
> vmstat_shepherd
> __cpuhp_state_add_instance


Trying to leave preempt disabled between the down/up leads to
scheduling-while-atomic instead.

Can you point out what I've missed here?


Thanks,

James