Re: [Patch 0/7] Per cpu thread hotplug infrastructure - V3

["Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com>]

On 07/19/2012 05:24 AM, Paul E. McKenney wrote:
> On Wed, Jul 18, 2012 at 11:06:52PM +0530, Srivatsa S. Bhat wrote:
>> On 07/16/2012 08:52 PM, Paul E. McKenney wrote:
>>> On Mon, Jul 16, 2012 at 10:42:34AM -0000, Thomas Gleixner wrote:
>>>> The following series implements the infrastructure for parking and
>>>> unparking kernel threads to avoid the full teardown and fork on cpu
>>>> hotplug operations along with management infrastructure for hotplug
>>>> and users.
>>>>
>>>> Changes vs. V2:
>>>>
>>>>  Use callbacks for all functionality. Thanks to Rusty for pointing
>>>>  that out. It makes the use sites nice and simple and keeps all the
>>>>  code which would be duplicated otherwise on the core.
>>>
>>> Hello, Thomas,
>>>
>>> What version should I apply this patchset to?  I tried v3.5-rc7, but
>>> got lots of warnings (one shown below) and the watchdog patch did not
>>> apply.
>>>
>>
>> Hi Paul,
>>
>> This patchset applies cleanly on Thomas' smp/hotplug branch in the -tip
>> tree.
> 
> Thank you, Srivatsa, works much better.  Still get "scheduling while
> atomic", looking into that.
> 

Got a chance to run this patchset now.. Even I am getting "scheduling while
atomic" messages like shown below..  Hmmm...

Regards,
Srivatsa S. Bhat

-----------------

BUG: sleeping function called from invalid context at kernel/softirq.c:748
in_atomic(): 1, irqs_disabled(): 0, pid: 3, name: ksoftirqd/0
INFO: lockdep is turned off.
Pid: 3, comm: ksoftirqd/0 Tainted: G        W    3.5.0-rc1-threadhotplugv3-debug #2
Call Trace:
 [<ffffffff8108e7ac>] __might_sleep+0x18c/0x250
 [<ffffffff8105ea4f>] run_ksoftirqd+0x4f/0x70
 [<ffffffff810c8b97>] smpboot_thread_fn+0x157/0x1f0
 [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80
 [<ffffffff8107e7ee>] kthread+0xee/0x100
 [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10
 [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13
 [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70
 [<ffffffff8158eb70>] ? gs_change+0x13/0x13
BUG: scheduling while atomic: ksoftirqd/0/3/0x1000000b
INFO: lockdep is turned off.
Modules linked in: ext3 jbd mbcache sd_mod crc_t10dif qla2xxx scsi_transport_fc scsi_tgt mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan]
Pid: 3, comm: ksoftirqd/0 Tainted: G        W    3.5.0-rc1-threadhotplugv3-debug #2
Call Trace:
 [<ffffffff8108d1cc>] __schedule_bug+0x6c/0x90
 [<ffffffff81582818>] __schedule+0x7c8/0x880
 [<ffffffff81092cea>] __cond_resched+0x2a/0x40
 [<ffffffff81582960>] _cond_resched+0x30/0x40
SELinux: 2048 avtab hash slots, 250818 rules.
 [<ffffffff8105ea54>] run_ksoftirqd+0x54/0x70
 [<ffffffff810c8b97>] smpboot_thread_fn+0x157/0x1f0
 [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80
 [<ffffffff8107e7ee>] kthread+0xee/0x100
 [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10
 [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13
 [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70
 [<ffffffff8158eb70>] ? gs_change+0x13/0x13

[...]

BUG: sleeping function called from invalid context at kernel/softirq.c:748
in_atomic(): 1, irqs_disabled(): 0, pid: 37, name: ksoftirqd/8
INFO: lockdep is turned off.
Pid: 37, comm: ksoftirqd/8 Tainted: G        W    3.5.0-rc1-threadhotplugv3-debug #2
Call Trace:
 [<ffffffff8108e7ac>] __might_sleep+0x18c/0x250
 [<ffffffff8105ea4f>] run_ksoftirqd+0x4f/0x70
 [<ffffffff810c8b97>] smpboot_thread_fn+0x157/0x1f0
		Welcome to  [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80
 [<ffffffff8107e7ee>] kthread+0xee/0x100
Red Hat [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10
 [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13
 [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70
 [<ffffffff8158eb70>] ? gs_change+0x13/0x13
BUG: scheduling while atomic: ksoftirqd/8/37/0x10000004
INFO: lockdep is turned off.
Modules linked in: ext3 jbd mbcache sd_mod crc_t10dif qla2xxx scsi_transport_fc scsi_tgt mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan]
Pid: 37, comm: ksoftirqd/8 Tainted: G        W    3.5.0-rc1-threadhotplugv3-debug #2
Call Trace:
 Enterprise Linu [<ffffffff8108d1cc>] __schedule_bug+0x6c/0x90
x Server
 [<ffffffff81582818>] __schedule+0x7c8/0x880
 [<ffffffff81092cea>] __cond_resched+0x2a/0x40
 [<ffffffff81582960>] _cond_resched+0x30/0x40
 [<ffffffff8105ea54>] run_ksoftirqd+0x54/0x70
 [<ffffffff810c8b97>] smpboot_thread_fn+0x157/0x1f0
 [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80
 [<ffffffff8107e7ee>] kthread+0xee/0x100
 [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10
 [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13
 [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70
 [<ffffffff8158eb70>] ? gs_change+0x13/0x13

[...]

BUG: scheduling while atomic: watchdog/15/64/0x0000009b
INFO: lockdep is turned off.
Modules linked in: ebtable_nat ebtables ipt_MASQUERADE iptable_nat nf_nat xt_CHECKSUM iptable_mangle bridge stp llc autofs4 cpufreq_powersave sunrpc cpufreq_ondemand acpi_cpufreq freq_table mperf ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 dm_mirror dm_region_hash dm_log dm_round_robin scsi_dh_rdac dm_multipath vhost_net macvtap macvlan tun uinput cdc_ether usbnet mii serio_raw kvm_intel kvm microcode i2c_i801 i2c_core lpc_ich mfd_core shpchp ioatdma dca i7core_edac edac_core bnx2 dm_mod sg ext3 jbd mbcache sd_mod crc_t10dif qla2xxx scsi_transport_fc scsi_tgt mptsas mptscsih mptbase scsi_transport_sas [last unloaded: scsi_wait_scan]
Pid: 64, comm: watchdog/15 Tainted: G        W    3.5.0-rc1-threadhotplugv3-debug #2
Call Trace:
 [<ffffffff8108d1cc>] __schedule_bug+0x6c/0x90
 [<ffffffff81582818>] __schedule+0x7c8/0x880
 [<ffffffff81582bf9>] schedule+0x29/0x70
 [<ffffffff81582f77>] schedule_preempt_disabled+0x27/0x40
 [<ffffffff810c8b33>] smpboot_thread_fn+0xf3/0x1f0
 [<ffffffff810c8a40>] ? smpboot_create_threads+0x80/0x80
 [<ffffffff8107e7ee>] kthread+0xee/0x100
 [<ffffffff8158eb74>] kernel_thread_helper+0x4/0x10
 [<ffffffff815844f0>] ? retint_restore_args+0x13/0x13
 [<ffffffff8107e700>] ? __init_kthread_worker+0x70/0x70
 [<ffffffff8158eb70>] ? gs_change+0x13/0x13