an oops possibly due to an SMP related bug in netfilter

an oops possibly due to an SMP related bug in netfilter

[Joshua N Pritikin]

[-- Attachment #1: Type: text/plain, Size: 3523 bytes --]

With a big thanks to netconsole, I am able to present you with an oops
in the netfilter code.  This occurred in a vanilla 2.6.7 kernel.  Let
me know what more details I can provide about the system in question.

(Perhaps I am one of the few people crazy enough to run a firewall on
an SMP machine.  ;-)

Unable to handle kernel NULL pointer dereference
 at virtual address 00000000 
 printing eip: 
c8895955 
*pde = 00000000 
Oops: 0000 [#1] 
SMP 
 
Modules linked in:
 netconsole
 mga
 nbd
 ipt_MASQUERADE
 iptable_nat
 ipt_LOG
 ipt_limit
 ipt_state
 iptable_filter
 ip_tables
 dm_mod
 reiserfs
 snd_dummy
 snd_pcm_oss
 snd_mixer_oss
 snd_pcm
 snd_timer
 snd
 snd_page_alloc
 sctp
 via_agp
 agpgart
 af_key
 rtc
 8139too
 mii
 crc32
 psmouse
 uhci_hcd
 ip_conntrack_ftp
 ip_conntrack
 
CPU:    0 
EIP:    0060:[<c8895955>]    Not tainted 
EFLAGS: 00010246   (2.6.7)  
EIP is at __ip_conntrack_find+0x179/0x1a0 [ip_conntrack] 
eax: 00000001   ebx: 00000000   ecx: c0353cc0   edx: 00000000 
esi: 00000000   edi: 00000000   ebp: c0353c88   esp: c0353c6c 
ds: 007b   es: 007b   ss: 0068 
Process swapper (pid: 0, threadinfo=c0352000 task=c0300980)
 
Stack: 
c0352000 
c7ced540 
00000000 
c600ce2c 
00000768 
c0352000 
00000000 
c0353ca0 
 
       
c889600c 
c0353cc0 
c7ced540 
c0353cc0 
c7ced540 
c0353cd0 
c89a8e9f 
c0353cc0 
 
       
c7ced540 
c0353cc0 
c0353d8c 
c0353dbc 
c0353d8c 
02130644 
c0356e00 
0400a8c0 
 
Call Trace: 
 [<c01068e3>] 
show_stack+0x83/0x90
 
 [<c0106a22>] 
show_registers+0x112/0x17c
 
 [<c0106b90>] 
die+0x7c/0xe8
 
 [<c011385b>] 
do_page_fault+0x337/0x48e
 
 [<c0106571>] 
error_code+0x2d/0x38
 
 [<c889600c>] 
ip_conntrack_tuple_taken+0x90/0xec [ip_conntrack]
 
 [<c89a8e9f>] 
ip_nat_used_tuple+0x1f/0x28 [iptable_nat]
 
 [<c89a957b>] 
get_unique_tuple+0xe7/0x1e8 [iptable_nat]
 
 [<c89a9706>] 
ip_nat_setup_info+0x8a/0x350 [iptable_nat]
 
 [<c89a89ee>] 
ip_nat_rule_find+0x8e/0x9c [iptable_nat]
 
 [<c89a81f7>] 
gcc2_compiled.+0x1f7/0x2cc [iptable_nat]
 
 [<c0240df6>] 
nf_iterate+0x3a/0xb0
 
 [<c0241128>] 
nf_hook_slow+0xa0/0x128
 
 [<c024cd40>] 
ip_rcv+0x1b8/0x204
 
 [<c0238954>] 
netif_receive_skb+0x150/0x180
 
 [<c88c9c8d>] 
rtl8139_rx+0x191/0x240 [8139too]
 
 [<c88c9eb2>] 
rtl8139_poll+0x4e/0xd8 [8139too]
 
 [<c0238b1b>] 
net_rx_action+0x7f/0x120
 
 [<c011d6ed>] 
__do_softirq+0x5d/0xbc
 
 [<c011d777>] 
do_softirq+0x2b/0x3c
 
 [<c0107d45>] 
do_IRQ+0x111/0x120
 
 [<c0106474>] 
common_interrupt+0x18/0x20
 
 [<c0103fb6>] 
cpu_idle+0x3a/0x48
 
 [<c01002b9>] 
rest_init+0x49/0x50
 
 [<c0354968>] 
start_kernel+0x1a4/0x1a8
 
 [<c01001e0>] 
0xc01001e0
 
 
Code: 
8b 
03 
0f 
18 
00 
90 
8b 
45 
f4 
03 
05 
0c 
f0 
89 
c8 
39 
c3 
0f 
85 
39 
 
 
Kernel panic: Fatal exception in interrupt 
In interrupt handler - not syncing 
 
-------
Here is my NAT table:

always:~# iptables -t nat -L -v
Chain PREROUTING (policy ACCEPT 10 packets, 626 bytes)
 pkts bytes target     prot opt in     out     source               destination 

Chain POSTROUTING (policy ACCEPT 41 packets, 3413 bytes)
 pkts bytes target     prot opt in     out     source               destination 
   98  6388 MASQUERADE  all  --  any    eth1    anywhere             anywhere 

Chain OUTPUT (policy ACCEPT 136 packets, 9621 bytes)
 pkts bytes target     prot opt in     out     source               destination 

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: an oops possibly due to an SMP related bug in netfilter

[KOVACS Krisztian]

  Hi,

On Mon, Aug 30, 2004 at 05:38:09PM +0530, Joshua N Pritikin wrote:
> (Perhaps I am one of the few people crazy enough to run a firewall on
> an SMP machine.  ;-)
>  
> CPU:    0 
> EIP:    0060:[<c8895955>]    Not tainted 
> EFLAGS: 00010246   (2.6.7)  
> EIP is at __ip_conntrack_find+0x179/0x1a0 [ip_conntrack] 
> eax: 00000001   ebx: 00000000   ecx: c0353cc0   edx: 00000000 
> esi: 00000000   edi: 00000000   ebp: c0353c88   esp: c0353c6c 
> ds: 007b   es: 007b   ss: 0068 
> Process swapper (pid: 0, threadinfo=c0352000 task=c0300980)

  I don't think you're the only one running iptables on SMP... This looks
like a conntrack hash table corruption, so the first thing you should
check is your memory, of course. Are you 100 percent sure that it is ok?

-- 
 KOVACS Krisztian

Re: an oops possibly due to an SMP related bug in netfilter

[Joshua N Pritikin]

[-- Attachment #1: Type: text/plain, Size: 1091 bytes --]

On Mon, Aug 30, 2004 at 06:57:53PM +0200, KOVACS Krisztian wrote:
> On Mon, Aug 30, 2004 at 05:38:09PM +0530, Joshua N Pritikin wrote:
> > (Perhaps I am one of the few people crazy enough to run a firewall on
> > an SMP machine.  ;-)
> >  
> > CPU:    0 
> > EIP:    0060:[<c8895955>]    Not tainted 
> > EFLAGS: 00010246   (2.6.7)  
> > EIP is at __ip_conntrack_find+0x179/0x1a0 [ip_conntrack] 
> > eax: 00000001   ebx: 00000000   ecx: c0353cc0   edx: 00000000 
> > esi: 00000000   edi: 00000000   ebp: c0353c88   esp: c0353c6c 
> > ds: 007b   es: 007b   ss: 0068 
> > Process swapper (pid: 0, threadinfo=c0352000 task=c0300980)
> 
>   I don't think you're the only one running iptables on SMP... This looks
> like a conntrack hash table corruption, so the first thing you should
> check is your memory, of course. Are you 100 percent sure that it is ok?

Fair enough.

Memtest86 doesn't spot anything BUT it could be due to voltage
fluctuation.  I guess I can't run this motherboard without a UPS.

-- 
A new cognitive theory of emotion, http://openheartlogic.org

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: an oops possibly due to an SMP related bug in netfilter

[Bill Davidsen]

Joshua N Pritikin wrote:
> On Mon, Aug 30, 2004 at 06:57:53PM +0200, KOVACS Krisztian wrote:
> 
>>On Mon, Aug 30, 2004 at 05:38:09PM +0530, Joshua N Pritikin wrote:
>>
>>>(Perhaps I am one of the few people crazy enough to run a firewall on
>>>an SMP machine.  ;-)
>>> 
>>>CPU:    0 
>>>EIP:    0060:[<c8895955>]    Not tainted 
>>>EFLAGS: 00010246   (2.6.7)  
>>>EIP is at __ip_conntrack_find+0x179/0x1a0 [ip_conntrack] 
>>>eax: 00000001   ebx: 00000000   ecx: c0353cc0   edx: 00000000 
>>>esi: 00000000   edi: 00000000   ebp: c0353c88   esp: c0353c6c 
>>>ds: 007b   es: 007b   ss: 0068 
>>>Process swapper (pid: 0, threadinfo=c0352000 task=c0300980)
>>
>>  I don't think you're the only one running iptables on SMP... This looks
>>like a conntrack hash table corruption, so the first thing you should
>>check is your memory, of course. Are you 100 percent sure that it is ok?
> 
> 
> Fair enough.
> 
> Memtest86 doesn't spot anything BUT it could be due to voltage
> fluctuation.  I guess I can't run this motherboard without a UPS.
> 
Bad to guess, there's a program called memburn which may also be used. 
It's doing testing in a different way and I just grabbed it because 
someone in comp.sys.intel reported finding problems which memtest didn't 
show.

I run firewall on a LOT of dual Xeon+HT and ran iptables firewall on 
dual P5-166 until it died. I don't think there are problems.

-- 
    -bill davidsen (davidsen@tmr.com)
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me