Top 10 kernel oopses/warnings for the week of December 21st 2007

Top 10 kernel oopses/warnings for the week of December 21st 2007

[Arjan van de Ven]

The http://www.kerneloops.org website collects kernel oops and
warning reports from various mailing lists and bugzillas as well as
with a client users can install to auto-submit oopses.
Below is a top 10 list of the oopses collected in the last 7 days.
(Reports prior to 2.6.23 have been omitted in collecting the top 10)

This week a total of 100 oopses have been collected, up from 59 last week.
(This does not mean that the kernel quality has gone down; rather it means
that there's more effective data collection)

Rank 1: ieee80211_tx
	Warning at net/mac80211/tx.c:1093 in ieee80211_tx()
	Reported 30 times
	Only reported for 2.6.24-rc5 so far
	Probably specific to the rt2x00pci driver
	More info: http://www.kerneloops.org/search.php?search=ieee80211_tx
	
Rank 2: uart_flush_buffer
	Warning at drivers/serial/serial_core.c:544 in uart_flush_buffer()
	Reported 16 times
	No specific version information reported; bug present in 2.6.24-rc5
	Caused by a bug in the Bluetooth line discipline/tty code
	More info: http://lkml.org/lkml/2007/12/20/314 (analysis)
	More info: http://lkml.org/lkml/2007/12/20/414 (patch)
	More info: http://www.kerneloops.org/search.php?search=uart_flush_buffer
	
Rank 3: anon_vma_link
	Kernel page fault
	Reported 4 times
	Reported for 2.6.23.9 and 2.6.23.11
	So far only reported for reiserfs4 patched kernels
	More info: http://www.kerneloops.org/search.php?search=anon_vma_link
	
Rank 4: acpi_idle_enter_bm
	Soft lockup / crash
	Reported 3 times
	Reported for 2.6.23.8 (FC8) and 2.6.24-rc4
	Idle path: could be a hardware issue
	More info: http://www.kerneloops.org/search.php?search=acpi_idle_enter_bm
	
Rank 5:	iput
	Null pointer
	Reported 3 times
	No specific version information
	ISOFS specific; could be caused by an invalid iso formatted disk (security issue)
	More info: http://www.kerneloops.org/oops.php?number=2493
	More info: http://www.kerneloops.org/guilty.php?version=<unknown>&guilty=iput&start=0&end=0

Rank 6: acpi_idle_enter_simple
	Soft lockup
	Reported 2 times
	Reported only for 2.6.23.8 (FC8)
	Idle path: could be a hardware issue
	Possibly related to this weeks Rank 4 issue
	More info: http://www.kerneloops.org/search.php?search=acpi_idle_enter_simple
	
Rank 7: uart_write
	Warning at drivers/serial/serial_core.c:490 in uart_write()
	Reported 2 times
	No specific version information
	Appears to be a similar bug to the one in this weeks Rank 2
	More info: http://www.kerneloops.org/search.php?search=uart_write
	
Rank 8: __change_page_attr
	BUG at arch/x86/mm/pageattr_64.c:176
	Reported 2 times
	Reported this week for 2.6.24-rc5; history goes back to 2.6.15
	More info: http://www.kerneloops.org/guilty.php?version=2.6.24-rc5&guilty=__change_page_attr&start=1574144&end=1574144

Rank 9: r_show
	Kernel NULL pointer
	Reported 2 times
	Reported only for the -mm tree: 2.6.24-rc5-mm1
	More info: http://www.kerneloops.org/search.php?search=r_show
	
Rank 10: set_dentry_child_flags
	Warning at fs/inotify.c:172 in set_dentry_child_flags()
	Reported 2 times
	No specific version information
	Inotify related
	More info: http://www.kerneloops.org/search.php?search=set_dentry_child_flags
	
	
	
Website news / improvements:
    * Various user feedback is now incorportated; keep this coming!
		* Improved banner
		* BUG/WARN_ON's now have a direct link to the specific line of code in git-web
    * The oops submit client is now included in the Debian (unstable) and Gentoo distributions

Re: Top 10 kernel oopses/warnings for the week of December 21st 2007

[Andi Kleen]

Arjan van de Ven <arjan@linux.intel.com> writes:

> 	
> Rank 8: __change_page_attr
> 	BUG at arch/x86/mm/pageattr_64.c:176
> 	Reported 2 times
> 	Reported this week for 2.6.24-rc5; history goes back to 2.6.15

There is no BUG on this line on 2.6.24-rc5.   Since there are many
BUG_ONs in this file it is unclear which you mean.

Could you always include the version of the kernel where the actual
oops in the line came from?

Anyways there are a lot of third party modules who do strange 
things with c_p_a(), not always legal, so you might look up out for that 
pattern too. Perhaps report the out of tree modules loaded in the 
summary too?

-Andi

Re: Top 10 kernel oopses/warnings for the week of December 21st 2007

[Alan Cox]

> Rank 2: uart_flush_buffer
> 	Warning at drivers/serial/serial_core.c:544 in uart_flush_buffer()
> 	Reported 16 times
> 	No specific version information reported; bug present in 2.6.24-rc5
> 	Caused by a bug in the Bluetooth line discipline/tty code

> Rank 7: uart_write
> 	Warning at drivers/serial/serial_core.c:490 in uart_write()
> 	Reported 2 times

Seems to be more of the Bluetooth stuf


Alan
--
	                "DASD is not really IBM - it has a vowel"
                        -Arjan van de Ven

Re: Top 10 kernel oopses/warnings for the week of December 21st 2007

[Arjan van de Ven]

Andi Kleen wrote:
> Arjan van de Ven <arjan@linux.intel.com> writes:
> 
>> 	
>> Rank 8: __change_page_attr
>> 	BUG at arch/x86/mm/pageattr_64.c:176
>> 	Reported 2 times
>> 	Reported this week for 2.6.24-rc5; history goes back to 2.6.15
> 
> There is no BUG on this line on 2.6.24-rc5.   Since there are many
> BUG_ONs in this file it is unclear which you mean.
> 
> Could you always include the version of the kernel where the actual
> oops in the line came from?

in this case this is really all the version information available ;(
it seems to be a patched kernel without patched EXTRAVERSION.
But in the future if I have more specific information (eg it's only 1 kernel version) I'll mention it in more detail.
It gets unwieldy if there's 500 reports for an oops of course ;)

> 
> Anyways there are a lot of third party modules who do strange 
> things with c_p_a(), not always legal, so you might look up out for that 
> pattern too. Perhaps report the out of tree modules loaded in the 
> summary too?

I already always will mention if the oops is tainted or not (that I track specifically);
I'll keep an eye out for other non-tainting out of tree modules as well.

Thanks for the suggestions.

Re: Top 10 kernel oopses/warnings for the week of December 21st 2007

[Andi Kleen]

> in this case this is really all the version information available ;(
> it seems to be a patched kernel without patched EXTRAVERSION.
> But in the future if I have more specific information (eg it's only 1 
> kernel version) I'll mention it in more detail.
> It gets unwieldy if there's 500 reports for an oops of course ;)

Hmm would there be an automatic way to check out the file of the 
kernel version and then check if the BUG_ON/WARN_ON is on that line?
Maybe it could be done using git.

> 
> >
> >Anyways there are a lot of third party modules who do strange 
> >things with c_p_a(), not always legal, so you might look up out for that 
> >pattern too. Perhaps report the out of tree modules loaded in the 
> >summary too?
> 
> I already always will mention if the oops is tainted or not (that I track 
> specifically);

I don't necessarily mean tainted, just out of tree modules in general.
There are some GPL modules who do strange things too. Not saying
that these oopses should be all ignored -- they might be legitimate
kernel bugs that they just trigger -- just it should be visible somehow
in the summary in case there is a pattern. 

Especially for c_p_a() i'm quite suspicious
because it depends a lot on what the caller did.

One way perhaps would be also to check if there is an out of tree
module inside the backtrace. I suppose you could keep a list 
of in tree modules and do this automatically. Of course there could
be false positives too with the standard inexact backtrace.

-Andi