3.3.0-rc1 kernel becomes unresponsive

3.3.0-rc1 kernel becomes unresponsive

[Arend van Spriel]

I moved my development work to 3.3.0-rc1 from wireless-testing
repository. After loading the brcmfmac driver (12 sec.) I suddenly see
the BUG message 'scheduling while atomic' and it fills up the entire log
with the machine becoming totally unresponsive.

Any suggestions on where to start looking. What type of problems could
cause this?

Gr. AvS
----------------------------------------------------------------------------------------

Jan 24 04:53:07 lb-bun-10 kernel: [63122.211870] usbcore: registered new
interface driver brcmfmac
Jan 24 04:53:08 lb-bun-10 kernel: [63122.300516] udev[7158]: renamed
network interface wlan0 to wlan1
Jan 24 04:53:08 lb-bun-10 kernel: [63122.325482] ERROR @brcmf_run_iscan
: error (-3)
Jan 24 04:53:09 lb-bun-10 kernel: [63123.695387] cfg80211: Found new
beacon on frequency: 5745 MHz (Ch 149) on
 phy0
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106731] BUG: scheduling while
atomic: swapper/0/0/0x10000100
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106737] Modules linked in:
brcmfmac(O) brcmutil(O) cfg80211(O) sdhci_pci sdhci mmc_core aes_generic
arc4 binfmt_misc snd_hda_codec_hdmi snd_hda_codec_idt snd_hda_intel
snd_hda_codec intel_agp nouveau snd_hwdep snd_pcm snd_seq_midi
snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd
soundcore snd_page_alloc dell_laptop psmouse intel_gtt dell_wmi ttm
drm_kms_helper drm serio_raw intel_ips mxm_wmi dcdbas agpgart
sparse_keymap video firewire_ohci ahci libahci firewire_core crc_itu_t
e1000e [last unloaded: mmc_core]
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106794] Modules linked in:
brcmfmac(O) brcmutil(O) cfg80211(O) sdhci_pci sdhci mmc_core aes_generic
arc4 binfmt_misc snd_hda_codec_hdmi snd_hda_codec_idt snd_hda_intel
snd_hda_codec intel_agp nouveau snd_hwdep snd_pcm snd_seq_midi
snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device snd
soundcore snd_page_alloc dell_laptop psmouse intel_gtt dell_wmi ttm
drm_kms_helper drm serio_raw intel_ips mxm_wmi dcdbas agpgart
sparse_keymap video firewire_ohci ahci libahci firewire_core crc_itu_t
e1000e [last unloaded: mmc_core]
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106840]
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106845] Pid: 0, comm: swapper/0
Tainted: G           O 3.3.0-rc1-wl-testing #2 Dell Inc. Latitude
E6410/07XJP9
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106853] EIP: 0060:[<c12be284>]
EFLAGS: 00000202 CPU: 0
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106863] EIP is at
intel_idle+0xb4/0x110
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106867] EAX: 00000000 EBX:
000018ec ECX: 00000000 EDX: 000018ec
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106870] ESI: 00000000 EDI:
00000008 EBP: c169df70 ESP: c169df54
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106874]  DS: 007b ES: 007b FS:
00d8 GS: 00e0 SS: 0068
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106878] Process swapper/0 (pid:
0, ti=f500c000 task=c16a1de0 task.ti=
c169c000)
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106882] Stack:
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106884]  00000020 f5409e5c
00000003 00000000 000000cc c16d2000 f5409e
5c c169df94
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106892]  c13beb8b 00000000
00000000 00000003 c169f000 c16f1010 000000
00 c169f000
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106900]  c169dfb4 c1001d84
0dc4807d 0000000f 39547204 0c134087 c172d0
60 0008d800
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106909] Call Trace:
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106919]  [<c13beb8b>]
cpuidle_idle_call+0xab/0x1d0
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106926]  [<c1001d84>]
cpu_idle+0xa4/0xf0
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106933]  [<c149d2cd>]
rest_init+0x5d/0x70
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106942]  [<c16f5853>]
start_kernel+0x38a/0x390
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106947]  [<c16f52f3>] ?
kernel_init+0x145/0x145
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106953]  [<c16f5075>]
i386_start_kernel+0x75/0x79
Jan 24 04:53:20 lb-bun-10 kernel: [63135.106955] Code: f6 89 e0 25 00 e0
ff ff 8b 40 08 a8 08 75 08 b1 01 8b 4
5 e4 0f 01 c9 e8 7b 85 db ff 29 d8 19 f2 e8 d2 86 d7 ff 89 c6 89 d3 fb
90 <8d> 74 26 00 85 3d a0 28 6d c1 75 0
d 8d 55 f0 b8 05 00 00 00 e8
Jan 24 04:53:20 lb-bun-10 kernel: [63135.107009] Call Trace:
Jan 24 04:53:20 lb-bun-10 kernel: [63135.107014]  [<c13beb8b>]
cpuidle_idle_call+0xab/0x1d0
Jan 24 04:53:20 lb-bun-10 kernel: [63135.107018]  [<c1001d84>]
cpu_idle+0xa4/0xf0
Jan 24 04:53:20 lb-bun-10 kernel: [63135.107023]  [<c149d2cd>]
rest_init+0x5d/0x70
Jan 24 04:53:20 lb-bun-10 kernel: [63135.107028]  [<c16f5853>]
start_kernel+0x38a/0x390
Jan 24 04:53:20 lb-bun-10 kernel: [63135.107033]  [<c16f52f3>] ?
kernel_init+0x145/0x145
Jan 24 04:53:20 lb-bun-10 kernel: [63135.107038]  [<c16f5075>]
i386_start_kernel+0x75/0x79

Re: 3.3.0-rc1 kernel becomes unresponsive

[Stanislaw Gruszka]

[-- Attachment #1: Type: text/plain, Size: 802 bytes --]

On Tue, Jan 24, 2012 at 11:34:25AM +0100, Arend van Spriel wrote:
> I moved my development work to 3.3.0-rc1 from wireless-testing
> repository. After loading the brcmfmac driver (12 sec.) I suddenly see
> the BUG message 'scheduling while atomic' and it fills up the entire log
> with the machine becoming totally unresponsive.
> 
> Any suggestions on where to start looking. What type of problems could
> cause this?

There are various debug config options which allow to narrow such problems
down. CONFIG_LOCKDEP seems to be most useful, but there are other quite
helpful too. Try to add options from attached file to your .config (do not
forgot to "make oldconfig" and check if these options ware not wiped out),
and after recompile see if kernel will print some more valuable messages.

Stanislaw

[-- Attachment #2: debug.config --]
[-- Type: text/plain, Size: 4798 bytes --]

#
# Kernel hacking
#
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
CONFIG_PRINTK_TIME=y
CONFIG_DEFAULT_MESSAGE_LOGLEVEL=4
# CONFIG_ENABLE_WARN_DEPRECATED is not set
CONFIG_ENABLE_MUST_CHECK=y
CONFIG_FRAME_WARN=1024
CONFIG_MAGIC_SYSRQ=y
CONFIG_STRIP_ASM_SYMS=y
# CONFIG_UNUSED_SYMBOLS is not set
CONFIG_DEBUG_FS=y
CONFIG_HEADERS_CHECK=y
# CONFIG_DEBUG_SECTION_MISMATCH is not set
CONFIG_DEBUG_KERNEL=y
CONFIG_DEBUG_SHIRQ=y
# CONFIG_LOCKUP_DETECTOR is not set
# CONFIG_HARDLOCKUP_DETECTOR is not set
CONFIG_DETECT_HUNG_TASK=y
CONFIG_DEFAULT_HUNG_TASK_TIMEOUT=120
# CONFIG_BOOTPARAM_HUNG_TASK_PANIC is not set
CONFIG_BOOTPARAM_HUNG_TASK_PANIC_VALUE=0
CONFIG_SCHED_DEBUG=y
CONFIG_SCHEDSTATS=y
CONFIG_TIMER_STATS=y
CONFIG_DEBUG_OBJECTS=y
# CONFIG_DEBUG_OBJECTS_SELFTEST is not set
CONFIG_DEBUG_OBJECTS_FREE=y
CONFIG_DEBUG_OBJECTS_TIMERS=y
CONFIG_DEBUG_OBJECTS_WORK=y
CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
# CONFIG_DEBUG_OBJECTS_PERCPU_COUNTER is not set
CONFIG_DEBUG_OBJECTS_ENABLE_DEFAULT=1
# CONFIG_SLUB_STATS is not set
CONFIG_DEBUG_RT_MUTEXES=y
CONFIG_DEBUG_PI_LIST=y
# CONFIG_RT_MUTEX_TESTER is not set
CONFIG_DEBUG_SPINLOCK=y
CONFIG_DEBUG_MUTEXES=y
CONFIG_DEBUG_LOCK_ALLOC=y
CONFIG_PROVE_LOCKING=y
# CONFIG_PROVE_RCU is not set
# CONFIG_SPARSE_RCU_POINTER is not set
CONFIG_LOCKDEP=y
# CONFIG_LOCK_STAT is not set
# CONFIG_DEBUG_LOCKDEP is not set
CONFIG_TRACE_IRQFLAGS=y
# CONFIG_DEBUG_ATOMIC_SLEEP is not set
# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
CONFIG_STACKTRACE=y
CONFIG_DEBUG_STACK_USAGE=y
# CONFIG_DEBUG_KOBJECT is not set
CONFIG_DEBUG_HIGHMEM=y
# CONFIG_DEBUG_INFO is not set
CONFIG_DEBUG_VM=y
# CONFIG_DEBUG_VIRTUAL is not set
CONFIG_DEBUG_WRITECOUNT=y
# CONFIG_DEBUG_MEMORY_INIT is not set
CONFIG_DEBUG_LIST=y
# CONFIG_TEST_LIST_SORT is not set
CONFIG_DEBUG_SG=y
CONFIG_DEBUG_NOTIFIERS=y
CONFIG_DEBUG_CREDENTIALS=y
CONFIG_ARCH_WANT_FRAME_POINTERS=y
CONFIG_FRAME_POINTER=y
CONFIG_BOOT_PRINTK_DELAY=y
# CONFIG_RCU_TORTURE_TEST is not set
CONFIG_RCU_CPU_STALL_TIMEOUT=60
# CONFIG_KPROBES_SANITY_TEST is not set
# CONFIG_BACKTRACE_SELF_TEST is not set
# CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
# CONFIG_DEBUG_FORCE_WEAK_PER_CPU is not set
# CONFIG_DEBUG_PER_CPU_MAPS is not set
# CONFIG_LKDTM is not set
# CONFIG_CPU_NOTIFIER_ERROR_INJECT is not set
CONFIG_FAULT_INJECTION=y
CONFIG_FAILSLAB=y
CONFIG_FAIL_PAGE_ALLOC=y
CONFIG_FAIL_MAKE_REQUEST=y
CONFIG_FAIL_IO_TIMEOUT=y
CONFIG_FAULT_INJECTION_DEBUG_FS=y
CONFIG_FAULT_INJECTION_STACKTRACE_FILTER=y
CONFIG_LATENCYTOP=y
CONFIG_SYSCTL_SYSCALL_CHECK=y
CONFIG_DEBUG_PAGEALLOC=y
CONFIG_USER_STACKTRACE_SUPPORT=y
CONFIG_NOP_TRACER=y
CONFIG_HAVE_FUNCTION_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_FP_TEST=y
CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
CONFIG_HAVE_C_RECORDMCOUNT=y
CONFIG_TRACER_MAX_TRACE=y
CONFIG_RING_BUFFER=y
CONFIG_EVENT_TRACING=y
CONFIG_EVENT_POWER_TRACING_DEPRECATED=y
CONFIG_CONTEXT_SWITCH_TRACER=y
CONFIG_RING_BUFFER_ALLOW_SWAP=y
CONFIG_TRACING=y
CONFIG_GENERIC_TRACER=y
CONFIG_TRACING_SUPPORT=y
CONFIG_FTRACE=y
# CONFIG_FUNCTION_TRACER is not set
# CONFIG_IRQSOFF_TRACER is not set
CONFIG_SCHED_TRACER=y
CONFIG_FTRACE_SYSCALLS=y
CONFIG_BRANCH_PROFILE_NONE=y
# CONFIG_PROFILE_ANNOTATED_BRANCHES is not set
# CONFIG_PROFILE_ALL_BRANCHES is not set
# CONFIG_STACK_TRACER is not set
CONFIG_BLK_DEV_IO_TRACE=y
CONFIG_KPROBE_EVENT=y
# CONFIG_FTRACE_STARTUP_TEST is not set
CONFIG_MMIOTRACE=y
# CONFIG_MMIOTRACE_TEST is not set
CONFIG_RING_BUFFER_BENCHMARK=m
# CONFIG_PROVIDE_OHCI1394_DMA_INIT is not set
CONFIG_BUILD_DOCSRC=y
CONFIG_DYNAMIC_DEBUG=y
CONFIG_DMA_API_DEBUG=y
# CONFIG_ATOMIC64_SELFTEST is not set
# CONFIG_SAMPLES is not set
CONFIG_HAVE_ARCH_KGDB=y
CONFIG_KGDB=y
CONFIG_KGDB_SERIAL_CONSOLE=y
CONFIG_KGDB_TESTS=y
# CONFIG_KGDB_TESTS_ON_BOOT is not set
# CONFIG_KGDB_LOW_LEVEL_TRAP is not set
# CONFIG_KGDB_KDB is not set
CONFIG_HAVE_ARCH_KMEMCHECK=y
# CONFIG_KMEMCHECK is not set
# CONFIG_TEST_KSTRTOX is not set
CONFIG_STRICT_DEVMEM=y
# CONFIG_X86_VERBOSE_BOOTUP is not set
CONFIG_EARLY_PRINTK=y
CONFIG_EARLY_PRINTK_DBGP=y
CONFIG_DEBUG_STACKOVERFLOW=y
CONFIG_X86_PTDUMP=y
# CONFIG_DEBUG_RODATA is not set
# CONFIG_DEBUG_SET_MODULE_RONX is not set
CONFIG_DEBUG_NX_TEST=m
CONFIG_DOUBLEFAULT=y
# CONFIG_IOMMU_STRESS is not set
CONFIG_HAVE_MMIOTRACE_SUPPORT=y
# CONFIG_X86_DECODER_SELFTEST is not set
CONFIG_IO_DELAY_TYPE_0X80=0
CONFIG_IO_DELAY_TYPE_0XED=1
CONFIG_IO_DELAY_TYPE_UDELAY=2
CONFIG_IO_DELAY_TYPE_NONE=3
CONFIG_IO_DELAY_0X80=y
# CONFIG_IO_DELAY_0XED is not set
# CONFIG_IO_DELAY_UDELAY is not set
# CONFIG_IO_DELAY_NONE is not set
CONFIG_DEFAULT_IO_DELAY_TYPE=0
CONFIG_DEBUG_BOOT_PARAMS=y
# CONFIG_CPA_DEBUG is not set
CONFIG_OPTIMIZE_INLINING=y
# CONFIG_DEBUG_STRICT_USER_COPY_CHECKS is not set