Re: kernel BUG at net/core/skbuff.c:1028!

Re: kernel BUG at net/core/skbuff.c:1028!

[David S. Miller]

   From: Jens Axboe <axboe@suse.de>
   Date: Wed, 7 May 2003 14:14:12 +0200

   Booting 2.5-BK on my little router BUG's out before the login is
   reached. 100% reproduceable. Let me know if you want more detail.

I forwarded this to Rusty, I think netfilter changes he made
recently have caused this.

kernel BUG at net/core/skbuff.c:1028!

[Jens Axboe]

Hi,

Booting 2.5-BK on my little router BUG's out before the login is
reached. 100% reproduceable. Let me know if you want more detail.

kernel BUG at net/core/skbuff.c:1028!
invalid operand: 0000 [#1]
CPU:    0
EIP:    0060:[<c0260774>]    Not tainted
EFLAGS: 00010206
EIP is at skb_checksum+0x244/0x260
eax: 00000000   ebx: 00000035   ecx: cee3a980   edx: cdbcfa80
esi: 00000014   edi: 00000049   ebp: c036dc9c   esp: c036dc78
ds: 007b   es: 007b   ss: 0068
Process swapper (pid: 0, threadinfo=c036c000 task=c0326ca0)
Stack: cee3a8c0 00000206 cec76444 c036dc90 00000035 00000000 cdbcfa10
00000003 
       c036dda0 c036dcc8 c02ba71e cee3a980 00000049 00000049 b02e4fd1
cec764c0 
       cdbcfa24 cec76420 00000003 c036dda0 c036dd00 c02b9240 c036dda0
cec76420 
Call Trace:
 [<c02ba71e>] icmp_reply_translation+0x7e/0x220
 [<c02b9240>] ip_nat_fn+0x1e0/0x230
 [<c02b938f>] ip_nat_local_fn+0x5f/0xb0
 [<c0289010>] dst_output+0x0/0x30
 [<c026920c>] nf_iterate+0x5c/0xb0
 [<c0289010>] dst_output+0x0/0x30
 [<c02694f9>] nf_hook_slow+0x69/0x100
 [<c0289010>] dst_output+0x0/0x30
 [<c0288a19>] ip_push_pending_frames+0x329/0x3b0
 [<c0289010>] dst_output+0x0/0x30
 [<c02a87cf>] icmp_send+0x2bf/0x3b0
 [<c023c245>] __ide_dma_read+0xc5/0xe0
 [<c0234308>] do_rw_disk+0x6e8/0x800
 [<c022910f>] start_request+0x11f/0x180
 [<c0281323>] ipv4_link_failure+0x13/0x50
 [<c02a6753>] arp_error_report+0x63/0x70
 [<c0265e76>] neigh_timer_handler+0x96/0x180
 [<c0265de0>] neigh_timer_handler+0x0/0x180
 [<c011e2db>] run_timer_softirq+0x9b/0x150
 [<c010a731>] handle_IRQ_event+0x31/0xf0
 [<c011a96f>] do_softirq+0x6f/0xd0
 [<c010a9a5>] do_IRQ+0xc5/0xe0
 [<c01070d0>] default_idle+0x0/0x50
 [<c0109208>] common_interrupt+0x18/0x20
 [<c01070d0>] default_idle+0x0/0x50
 [<c01070f6>] default_idle+0x26/0x50
 [<c0107192>] cpu_idle+0x32/0x50
 [<c0105000>] _stext+0x0/0x20
 [<c036e6ca>] start_kernel+0x12a/0x130

Code: 0f 0b 04 04 99 f4 30 c0 8b 45 14 8d 65 f4 5b 5e 5f 5d c3 89 
 <0>Kernel panic: Fatal exception in interrupt
In interrupt handler - not syncing


-- 
Jens Axboe

Re: kernel BUG at net/core/skbuff.c:1028!

[Jens Axboe]

On Wed, May 07 2003, David S. Miller wrote:
>    From: Jens Axboe <axboe@suse.de>
>    Date: Wed, 7 May 2003 14:14:12 +0200
> 
>    Booting 2.5-BK on my little router BUG's out before the login is
>    reached. 100% reproduceable. Let me know if you want more detail.
> 
> I forwarded this to Rusty, I think netfilter changes he made
> recently have caused this.

Backing out this puppy:

http://www.kernel.org/pub/linux/kernel/v2.5/testing/cset/cset-rusty@rustcorp.com.au|ChangeSet|20030506080426|32903.txt

makes it work. Ruuuuusty?

-- 
Jens Axboe

Re: Fw: kernel BUG at net/core/skbuff.c:1028!

[Rusty Russell]

In message <20030507.042003.26512841.davem@redhat.com> you write:
> It has to be from some of the skb linearization changes.
> I can't think of any other change we've made that would
> make this start to happen.

Yep, culprit is obvious stupid bug.  This indicates a serious lack of
testing on my part 8(

Jens, does this help?
Rusty.

diff -urpN --exclude TAGS -X /home/rusty/devel/kernel/kernel-patches/current-dontdiff --minimal linux-2.5.69-bk2/net/ipv4/netfilter/ip_nat_core.c working-2.5.69-bk2-fix-nat/net/ipv4/netfilter/ip_nat_core.c
--- linux-2.5.69-bk2/net/ipv4/netfilter/ip_nat_core.c	2003-05-08 10:31:08.000000000 +1000
+++ working-2.5.69-bk2-fix-nat/net/ipv4/netfilter/ip_nat_core.c	2003-05-08 11:19:04.000000000 +1000
@@ -870,7 +870,8 @@ icmp_reply_translation(struct sk_buff **
 	   adjustment, so make sure the current checksum is correct. */
 	if ((*pskb)->ip_summed != CHECKSUM_UNNECESSARY
 	    && (u16)csum_fold(skb_checksum(*pskb, (*pskb)->nh.iph->ihl*4,
-					   (*pskb)->len, 0)))
+					   (*pskb)->len
+					   - (*pskb)->nh.iph->ihl*4, 0)))
 		return 0;
 
 	/* Must be RELATED */


--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Re: Fw: kernel BUG at net/core/skbuff.c:1028!

[Jens Axboe]

On Thu, May 08 2003, Rusty Russell wrote:
> In message <20030507.042003.26512841.davem@redhat.com> you write:
> > It has to be from some of the skb linearization changes.
> > I can't think of any other change we've made that would
> > make this start to happen.
> 
> Yep, culprit is obvious stupid bug.  This indicates a serious lack of
> testing on my part 8(

One would think so, since it doesn't even get to the login :)

> Jens, does this help?

[snip]

Nope, it still dies hard. I didn't log the oops this time (box is
headless and I need to move it to do so), but it hung hard before it was
done booting.

Want me to log a new oops?

-- 
Jens Axboe

Re: kernel BUG at net/core/skbuff.c:1028!

[David S. Miller]

   From: Rusty Russell <rusty@rustcorp.com.au>
   Date: Thu, 08 May 2003 11:20:27 +1000

   Yep, culprit is obvious stupid bug.  This indicates a serious lack of
   testing on my part 8(
   
   Jens, does this help?

There were two cases of the same bug, you fixed only one
instance :-)

Jens, try this patch instead.

--- net/ipv4/netfilter/ip_nat_core.c.~1~	Thu May  8 11:23:22 2003
+++ net/ipv4/netfilter/ip_nat_core.c	Thu May  8 11:25:56 2003
@@ -861,6 +861,7 @@
 	} *inside;
 	unsigned int i;
 	struct ip_nat_info *info = &conntrack->nat.info;
+	int hdrlen;
 
 	if (!skb_ip_make_writable(pskb,(*pskb)->nh.iph->ihl*4+sizeof(*inside)))
 		return 0;
@@ -868,10 +869,12 @@
 
 	/* We're actually going to mangle it beyond trivial checksum
 	   adjustment, so make sure the current checksum is correct. */
-	if ((*pskb)->ip_summed != CHECKSUM_UNNECESSARY
-	    && (u16)csum_fold(skb_checksum(*pskb, (*pskb)->nh.iph->ihl*4,
-					   (*pskb)->len, 0)))
-		return 0;
+	if ((*pskb)->ip_summed != CHECKSUM_UNNECESSARY) {
+		hdrlen = (*pskb)->nh.iph->ihl * 4;
+		if ((u16)csum_fold(skb_checksum(*pskb, hdrlen,
+						(*pskb)->len - hdrlen, 0)))
+			return 0;
+	}
 
 	/* Must be RELATED */
 	IP_NF_ASSERT((*pskb)->nfct
@@ -948,10 +951,12 @@
 	}
 	READ_UNLOCK(&ip_nat_lock);
 
+	hdrlen = (*pskb)->nh.iph->ihl * 4;
+
 	inside->icmp.checksum = 0;
-	inside->icmp.checksum = csum_fold(skb_checksum(*pskb,
-						       (*pskb)->nh.iph->ihl*4,
-						       (*pskb)->len, 0));
+	inside->icmp.checksum = csum_fold(skb_checksum(*pskb, hdrlen,
+						       (*pskb)->len - hdrlen,
+						       0));
 	return 1;
 
  unlock_fail:

Re: kernel BUG at net/core/skbuff.c:1028!

[Jens Axboe]

On Thu, May 08 2003, David S. Miller wrote:
>    From: Rusty Russell <rusty@rustcorp.com.au>
>    Date: Thu, 08 May 2003 11:20:27 +1000
> 
>    Yep, culprit is obvious stupid bug.  This indicates a serious lack of
>    testing on my part 8(
>    
>    Jens, does this help?
> 
> There were two cases of the same bug, you fixed only one
> instance :-)
> 
> Jens, try this patch instead.

I went to apply it to bk-current as of this morning, but I see it's
already in. And bk-current does indeed boot and (appears to :) work,
thanks Dave!

-- 
Jens Axboe