[IPv6] BUG: NULL pointer dereference in(?) ip6_flush_pending_frames

[IPv6] BUG: NULL pointer dereference in(?) ip6_flush_pending_frames

[Bernhard Schmidt]

Hi,

I'm running a public Teredo relay (IPv4-to-IPv6 migration protocol)
using Miredo. Every once in a while (a few minutes to days after
daemon restart) it becomes unusable and I see the following kernel
message:

BUG: unable to handle kernel NULL pointer dereference at virtual address
0000008c
 printing eip:
c02640e6
*pde = 00000000
Oops: 0000 [#17]
SMP
Modules linked in: ip6table_filter ip6_tables af_packet tun bitrev crc32
ipt_LOG xt_tcpudp iptable_filter iptable_mangle ip_tables x_tables
dm_mod capability commoncap iTCO_wdt floppy e1000 rtc unix
CPU:    0
EIP:    0060:[<c02640e6>]    Not tainted VLI
EFLAGS: 00210246   (2.6.21.3-iabg-pe750 #1)
EIP is at ip6_flush_pending_frames+0x97/0x121
eax: 00000000   ebx: d3e3ca80   ecx: db590380   edx: d3e3caf0
esi: d3e3cc80   edi: db590380   ebp: 00000002   esp: d4af7cd4
ds: 007b   es: 007b   fs: 00d8  gs: 0033  ss: 0068
Process miredo (pid: 17615, ti=d4af6000 task=cfd60030 task.ti=d4af6000)
Stack: 000005d0 00000000 d4af7d44 d4af7d54 d4af7d54 00000000 db590380
c0275ab5
       00000000 00000000 00000040 00000000 00000000 d4af7d48 df4c6780
00000040
       d4af7f44 d3e3ca80 3a000000 00000000 0000001c 003a0000 00000000
00000000
Call Trace:
 [<c0275ab5>] rawv6_sendmsg+0x840/0xa63
 [<c0258a09>] inet_sendmsg+0x3b/0x45
 [<c021df73>] sock_sendmsg+0xbc/0xd4
 [<c0123f99>] autoremove_wake_function+0x0/0x35
 [<e087c911>] tun_chr_aio_read+0x29e/0x2a8 [tun]
 [<c011025a>] default_wake_function+0x0/0xc
 [<c021e29c>] sys_sendto+0x118/0x138
 [<c014d03c>] do_readv_writev+0x17d/0x187
 [<e087c673>] tun_chr_aio_read+0x0/0x2a8 [tun]
 [<c021ef2e>] sys_socketcall+0x15e/0x242
 [<c0102560>] syscall_call+0x7/0xb
 =======================
Code: 8d 43 70 8b 48 04 39 c1 74 31 85 c9 74 2d ff 48 08 8b 11 8b 41 04
c7 41 04 00 00 00 00 c7 01 00 00 00 00 89 42 04 89 10 8b 41 28 <8b> b8
8c 00 00 00 85 ff 0f 85 6b ff ff ff eb 94 83 a3 84 01 00
EIP: [<c02640e6>] ip6_flush_pending_frames+0x97/0x121 SS:ESP
0068:d4af7cd4

I have not found anything related on netdev, I'll try a new kernel to be
sure. Do you need any more information to debug this issue?

Hardware is a Dell PowerEdge 750 (i386 P4 HT), vanilla kernel 2.6.21.3
running Debian testing.

Thanks,
Bernhard

Re: [IPv6] BUG: NULL pointer dereference in(?) ip6_flush_pending_frames

[YOSHIFUJI Hideaki / 吉藤英明]

In article <20070909222400.GC7591@obelix.birkenwald.de> (at Mon, 10 Sep 2007 00:24:00 +0200), Bernhard Schmidt <berni@birkenwald.de> says:

> I'm running a public Teredo relay (IPv4-to-IPv6 migration protocol)
> using Miredo. Every once in a while (a few minutes to days after
> daemon restart) it becomes unusable and I see the following kernel
> message:
> 
> BUG: unable to handle kernel NULL pointer dereference at virtual address
> 0000008c
:
> EIP is at ip6_flush_pending_frames+0x97/0x121

I think I've found a bug.

Some of skbs in sk->write_queue do not have skb->dst because
we do not fill skb->dst when we allocate new skb in append_data().

Miyazawa-san, am I right?

BTW, I think we may not need to (or we should not) increment some stats
when using corking; if 100 sendmsg() (with MSG_MORE) result in 2 packets,
how many should we increment?

If 100, we should set skb->dst for every queued skbs.

If 1 (or 2 (*)), we increment the stats for the first queued skb and
we should just skip incrementing OutDiscards for the rest of queued skbs,
adn we should also impelement this semantics in other places;
e.g., we should increment other stats just once, not 100 times.

*: depends on the place we are discarding the datagram.

I guess should just increment by 1 (or 2).

Anyway, please try this.

Signed-off-by: YOSHIFUJI Hideaki <yoshfuji@linux-ipv6.org>

diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c
index 4704b5f..e489499 100644
--- a/net/ipv6/ip6_output.c
+++ b/net/ipv6/ip6_output.c
@@ -1423,8 +1423,9 @@ void ip6_flush_pending_frames(struct sock *sk)
 	struct sk_buff *skb;
 
 	while ((skb = __skb_dequeue_tail(&sk->sk_write_queue)) != NULL) {
-		IP6_INC_STATS(ip6_dst_idev(skb->dst),
-			      IPSTATS_MIB_OUTDISCARDS);
+		if (skb->dst)
+			IP6_INC_STATS(ip6_dst_idev(skb->dst),
+				      IPSTATS_MIB_OUTDISCARDS);
 		kfree_skb(skb);
 	}
 


--yoshfuji

Re: [IPv6] BUG: NULL pointer dereference in(?) ip6_flush_pending_frames

[Bernhard Schmidt]

YOSHIFUJI Hideaki / 吉藤英明:

Hi,

>> BUG: unable to handle kernel NULL pointer dereference at virtual address
>> 0000008c
> :
>> EIP is at ip6_flush_pending_frames+0x97/0x121
> 
> I think I've found a bug.
[...]
> Anyway, please try this.

FTR, I tried 2.6.22.6 without the patch and it failed as well. The 
patched kernel is running since yesterday evening (about 8h now) and 
seems to be stable so far. Too early to tell for sure, but I guess we 
have a fix.

Thanks Yoshifuji!

Regards,
Bernhard

Re: [IPv6] BUG: NULL pointer dereference in(?) ip6_flush_pending_frames

[David Miller]

From: Bernhard Schmidt <berni@birkenwald.de>
Date: Mon, 10 Sep 2007 10:18:14 +0200

> FTR, I tried 2.6.22.6 without the patch and it failed as well. The 
> patched kernel is running since yesterday evening (about 8h now) and 
> seems to be stable so far. Too early to tell for sure, but I guess we 
> have a fix.
> 
> Thanks Yoshifuji!

I've applied this fix, thanks!