CPU hotplug lockdep trace during offline.

CPU hotplug lockdep trace during offline.

[Dave Jones]

>From Linus' tree as of a half hour ago.

echo 0 > /sys/devices/system/cpu/cpu1/online



[   67.675171] ======================================================
[   67.676121] [ INFO: possible circular locking dependency detected ]
[   67.677084] 3.7.0+ #34 Not tainted
[   67.677641] -------------------------------------------------------
[   67.678614] bash/604 is trying to acquire lock:
[   67.679326] blocked:  (cgroup_mutex){+.+.+.}, instance: ffffffff81c38278, at: [<ffffffff810cef77>] cgroup_lock+0x17/0x20
[   67.681220] 
but task is already holding lock:
[   67.682115] held:     (cpu_hotplug.lock){+.+.+.}, instance: ffffffff81c1f700, at: [<ffffffff81049ddf>] cpu_hotplug_begin+0x2f/0x60
[   67.684087] 
which lock already depends on the new lock.

[   67.685316] 
the existing dependency chain (in reverse order) is:
[   67.686450] 
-> #2 (cpu_hotplug.lock){+.+.+.}:
[   67.687183]        [<ffffffff810b84e2>] lock_acquire+0x92/0x1d0
[   67.688165]        [<ffffffff816a7713>] mutex_lock_nested+0x73/0x3b0
[   67.689186]        [<ffffffff81049ddf>] cpu_hotplug_begin+0x2f/0x60
[   67.690217]        [<ffffffff816986df>] _cpu_up+0x29/0x14b
[   67.691134]        [<ffffffff8169884f>] cpu_up+0x4e/0x5e
[   67.692025]        [<ffffffff81edf551>] smp_init+0x82/0xa3
[   67.692950]        [<ffffffff8168d214>] kernel_init+0xd4/0x2d0
[   67.693924]        [<ffffffff816b2b5c>] ret_from_fork+0x7c/0xb0
[   67.694921] 
-> #1 (cpu_add_remove_lock){+.+.+.}:
[   67.695791]        [<ffffffff810b84e2>] lock_acquire+0x92/0x1d0
[   67.696770]        [<ffffffff816a7713>] mutex_lock_nested+0x73/0x3b0
[   67.697814]        [<ffffffff81049f47>] cpu_maps_update_begin+0x17/0x20
[   67.698897]        [<ffffffff8168e406>] register_cpu_notifier+0x16/0x40
[   67.699973]        [<ffffffff8168f450>] mem_cgroup_css_alloc+0x450/0x6a0
[   67.701071]        [<ffffffff81edfde7>] cgroup_init_subsys+0x5e/0xf8
[   67.702112]        [<ffffffff81ee00ad>] cgroup_init+0x46/0x129
[   67.703076]        [<ffffffff81ec8bd9>] start_kernel+0x3cb/0x413
[   67.704071]        [<ffffffff81ec832d>] x86_64_start_reservations+0x131/0x135
[   67.705206]        [<ffffffff81ec8409>] x86_64_start_kernel+0xd8/0xdc
[   67.706266] 
-> #0 (cgroup_mutex){+.+.+.}:
[   67.707091]        [<ffffffff810b7daf>] __lock_acquire+0x1a7f/0x1b30
[   67.708133]        [<ffffffff810b84e2>] lock_acquire+0x92/0x1d0
[   67.709111]        [<ffffffff816a7713>] mutex_lock_nested+0x73/0x3b0
[   67.710129]        [<ffffffff810cef77>] cgroup_lock+0x17/0x20
[   67.711081]        [<ffffffff810d9599>] cpuset_update_active_cpus+0x19/0x160
[   67.712225]        [<ffffffff810891e7>] cpuset_cpu_inactive+0x47/0x50
[   67.713286]        [<ffffffff816ae6d6>] notifier_call_chain+0x66/0x150
[   67.714358]        [<ffffffff81078a2e>] __raw_notifier_call_chain+0xe/0x10
[   67.715425]        [<ffffffff81049d70>] __cpu_notify+0x20/0x40
[   67.716389]        [<ffffffff8168e0bd>] _cpu_down+0x8d/0x330
[   67.717306]        [<ffffffff8168e396>] cpu_down+0x36/0x50
[   67.718230]        [<ffffffff81691b1d>] store_online+0x5d/0xd0
[   67.719194]        [<ffffffff8143b1b8>] dev_attr_store+0x18/0x30
[   67.720186]        [<ffffffff81238440>] sysfs_write_file+0xe0/0x150
[   67.721224]        [<ffffffff811b8fef>] vfs_write+0xaf/0x180
[   67.722163]        [<ffffffff811b9335>] sys_write+0x55/0xa0
[   67.723095]        [<ffffffff816b2c02>] system_call_fastpath+0x16/0x1b
[   67.724142] 
other info that might help us debug this:

[   67.725345] Chain exists of:
  cgroup_mutex --> cpu_add_remove_lock --> cpu_hotplug.lock

[   67.745352]  Possible unsafe locking scenario:

[   67.758710]        CPU0                    CPU1
[   67.765527]        ----                    ----
[   67.772259]   lock(cpu_hotplug.lock);
[   67.778901]                                lock(cpu_add_remove_lock);
[   67.785983]                                lock(cpu_hotplug.lock);
[   67.792943]   lock(cgroup_mutex);
[   67.799358] 
 *** DEADLOCK ***

[   67.816718] 5 locks on stack by bash/604:
[   67.822496]  #0: held:     (&buffer->mutex){+.+.+.}, instance: ffff880108886518, at: [<ffffffff812383a7>] sysfs_write_file+0x47/0x150
[   67.829905]  #1: blocked:  (s_active#69){.+.+.+}, instance: ffff88012ce6cd60, at: [<ffffffff81238428>] sysfs_write_file+0xc8/0x150
[   67.837370]  #2: held:     (x86_cpu_hotplug_driver_mutex){+.+.+.}, instance: ffffffff81c1aef8, at: [<ffffffff8101e4b7>] cpu_hotplug_driver_lock+0x17/0x20
[   67.845512]  #3: held:     (cpu_add_remove_lock){+.+.+.}, instance: ffffffff81c1f638, at: [<ffffffff81049f47>] cpu_maps_update_begin+0x17/0x20
[   67.853640]  #4: held:     (cpu_hotplug.lock){+.+.+.}, instance: ffffffff81c1f700, at: [<ffffffff81049ddf>] cpu_hotplug_begin+0x2f/0x60
[   67.861874] 
stack backtrace:
[   67.874926] Pid: 604, comm: bash Not tainted 3.7.0+ #34
[   67.881886] Call Trace:
[   67.888481]  [<ffffffff8169f49d>] print_circular_bug+0x1fe/0x20f
[   67.895623]  [<ffffffff810b7daf>] __lock_acquire+0x1a7f/0x1b30
[   67.902779]  [<ffffffff810b8e6e>] ? mark_held_locks+0xae/0x110
[   67.909957]  [<ffffffff810b2ffe>] ? put_lock_stats.isra.23+0xe/0x40
[   67.917206]  [<ffffffff816aacb5>] ? _raw_spin_unlock_irqrestore+0x65/0x80
[   67.924565]  [<ffffffff810b84e2>] lock_acquire+0x92/0x1d0
[   67.931733]  [<ffffffff810cef77>] ? cgroup_lock+0x17/0x20
[   67.938894]  [<ffffffff816a7713>] mutex_lock_nested+0x73/0x3b0
[   67.946061]  [<ffffffff810cef77>] ? cgroup_lock+0x17/0x20
[   67.953171]  [<ffffffff810b907d>] ? trace_hardirqs_on+0xd/0x10
[   67.960330]  [<ffffffff810cef77>] ? cgroup_lock+0x17/0x20
[   67.967439]  [<ffffffff8101df26>] ? native_send_call_func_single_ipi+0x36/0x40
[   67.974840]  [<ffffffff810c04c1>] ? generic_exec_single+0xb1/0xc0
[   67.982097]  [<ffffffff810cef77>] cgroup_lock+0x17/0x20
[   67.989262]  [<ffffffff810d9599>] cpuset_update_active_cpus+0x19/0x160
[   67.996640]  [<ffffffff810c3ff2>] ? __module_address+0xf2/0x130
[   68.003933]  [<ffffffff810b394e>] ? __lock_is_held+0x5e/0x90
[   68.011206]  [<ffffffff810891e7>] cpuset_cpu_inactive+0x47/0x50
[   68.018485]  [<ffffffff816ae6d6>] notifier_call_chain+0x66/0x150
[   68.025765]  [<ffffffff81078a2e>] __raw_notifier_call_chain+0xe/0x10
[   68.033047]  [<ffffffff81049d70>] __cpu_notify+0x20/0x40
[   68.040181]  [<ffffffff8168e0bd>] _cpu_down+0x8d/0x330
[   68.047281]  [<ffffffff8168e396>] cpu_down+0x36/0x50
[   68.054328]  [<ffffffff81691b1d>] store_online+0x5d/0xd0
[   68.061400]  [<ffffffff8143b1b8>] dev_attr_store+0x18/0x30
[   68.068489]  [<ffffffff81238440>] sysfs_write_file+0xe0/0x150
[   68.075618]  [<ffffffff811b8fef>] vfs_write+0xaf/0x180
[   68.082647]  [<ffffffff811b9335>] sys_write+0x55/0xa0
[   68.089623]  [<ffffffff816b2c02>] system_call_fastpath+0x16/0x1b