oops on 2.6.9 in 'ip ro del'

oops on 2.6.9 in 'ip ro del'

[Alexander Vodomerov]

  Hello!

After upgrading kernel from 2.6.9-rc2 to 2.6.9, deconfiguring network
interfaces started to cause oops.

Machine: Athlon XP 2500+, 1Gb RAM (no highmem), all hardware is linux-friendly,
working fine for more than year.
Kernel: stock vanilla 2.6.9 kernel, no special patches, except PAGE_OFFSET
moved to 0x80000000.
Distro: Debian unstable/sid i386 (with the most recent updates).
GCC: Debian 3.3.5-1 from Debian package.
Oops: 100% reproducible.

I will be glad to provide any additional information.
Please Cc: me beacuse I'm not subscribed to mailing list.

Here is screenshot (copied by hand because system was unusable):

Deconfiguring network interfaces... Unable to handle kernel NULL pointer derefer
ence at virtual address 00000000
 printing eip:
802c5101
*pde = 00000000
Oops: 0002 [#1]
Modules linked in: ieee1394 mga_vid 8139too mii crc32 tg3 snd_pcm_oss snd_pcm sn
d_timer snd_page_alloc snd_mixer_oss snd soundcore psmouse usbhid evdev usb_stor
age ehci_hcd uhci_hcd usbcore sd_mod sg scsi_mod asb100 i2c_sensor i2c_viapro i2
c_dev i2c_core tun mga via_agp agpgart
CPU:	0
EIP:	0060:[<802c5101>]	Not tainted VLI
EFLAGS:	00010246   (2.6.9)
EIP is at fib_release_info+0x71/0xc0
eax: 00000000	ebx: bf5271c0	ecx: 00000000	edx:bf527160
esi: bf5271c4	edi: 5e2fb280	ebp: bffd0650	esp:bf203c24
ds: 007b   es: 007b   ss:0068
Process ip (pid: 1304, threadinfo=bf202000 task=bf7b1020)
Stack: 00000001 be730268 802c71c3 bf527160 be730260 be2fb280 00000000 000000cd
       bffd0640 bed52f58 00143653 bf3211e0 00000000 be730260 bf32c77c bffd0650
       bffd0640 bfff24a0 bffd0640 802c48aa bee0a8c0 bffd0650 bfff24a0 bffd0640
Call Trace:
 [<802c71c3>] fn_hash_delete+0x1c3/0x270
 [<802c48aa>] inet_rtm_delroute+0x6a/0x80
 [<8028e390>] rtnetlink_rcv+0x2d0/0x390
 [<802933c1>] netlink_data_ready+0x61/0x70
 [<802929a3>] netlink_sendskb+0xa3/0xb0
 [<80293062>] netlink_sendmsg+0x202/0x2f0
 [<8027cf05>] sock_sendmsg+0xe5/0x100
 [<80104a45>] error_code+0x2d/0x38
 [<80168c12>] update_atime+0x92/0xe0
 [<801f0142>] copy_from_user+0x42/0x80
 [<8011b1d0>] autoremove_wake_function+0x0/0x60
 [<8027e8d9>] sys_sendmsg+0x189/0x1f0
 [<80143a71>] handle_mm_fault+0xd1/0x140
 [<80115962>] do_page_fault+0x192/0x585
 [<8027db47>] __sock_create+0xa7/0x190
 [<801f0142>] copy_from_user+0x42/0x80
 [<8027ed76>] sys_socketcall+0x236/0x260
 [<80109d5a>] do_gettimeofday+0x1a/0xb0
 [<801157d0>] do_page_fault+0x0/0x585
 [<80104a45>] error_code+0x2d/0x38
 [<8010403b>] syscall_call+0x7/0xb
Code: 8d 5a 08 8b 4b 04 85 c0 89 01 74 03 89 48 04 c7 42 08 00 01 10 00 c7 43 04
 00 02 20 00 8d 5a 60 8d 72 64 8b 43 04 8b 4e 04 85 c0 <89> 01 74 03 89 48 04 c7
 43 04 00 01 10 00 c7 46 04 00 02 20 00
 /bin/sh: line 1:  1304 Segmentation fault      ip ro del table mh.hackers unrea
chable default

oops on 2.6.9 in 'ip ro del': debug results

[Alexander Vodomerov]

On Sun, Oct 24, 2004 at 01:08:04PM +0300, Denis Vlasenko wrote:
> Okay, so you've got an oops and want to find out what happened?
Thank you for nice docs!

I recompiled kernel with frame pointers. Here is stack dump and results
of my investigation.

Error occurs in fib_release_info (net/ipv4/fib_semantics.c:164) in the
following lines:

fib_release_info:
	...
	change_nexthops(fi) {
		hlist_del(&nh->nh_hash);
	} endfor_nexthops(fi)

hlist_del calls __hlist_del: 

__hlist_del:
        struct hlist_node *next = n->next;
	struct hlist_node **pprev = n->pprev;
	*pprev = next;           <===  Oops here: pprev = NULL
	if (next) {
		next->pprev = pprev;
	}

Oops is occuring, becuase pprev is NULL (may be it is hlist head?) in
line **prev = next;

I can make any additional debugging. I'm not a kernel hacker, may be you
can provide a link to explation how all this fib/hlist struct works?

BTW, this time I managed to get oops without hang, so it was just copied
from dmesg output:

Unable to handle kernel NULL pointer dereference at virtual address 00000000
 printing eip:
802c1c83
*pde = 00000000
Oops: 0002 [#1]
Modules linked in: ieee1394 8139too mii crc32 tg3 snd_pcm_oss snd_pcm snd_timer snd_page_alloc snd_mixer_oss snd soundcore psmouse usbhid evdev usb_storage ehci_hcd uhci_hcd usbcore sd_mod sg scsi_mod asb100 i2c_sensor i2c_viapro i2c_dev i2c_core tun mga via_agp agpgart
CPU:    0
EIP:    0060:[<802c1c83>]    Not tainted VLI
EFLAGS: 00010246   (2.6.9) 
EIP is at fib_release_info+0x73/0xc0
eax: 00000000   ebx: beaf2974   ecx: 00000000   edx: beaf2914
esi: beaf2978   edi: be33726c   ebp: befc9bf8   esp: befc9bf0
ds: 007b   es: 007b   ss: 0068
Process ip (pid: 1295, threadinfo=befc8000 task=bf01e020)
Stack: 00000001 be337274 befc9c40 802c3d78 beaf2914 be33726c be31b260 00000000 
       000000c8 bec7c254 be32c9a4 be31b260 001bcbe4 bea753b8 00000000 be33726c 
       befc9c74 bec7c264 bec7c254 bfff170c befc9c68 802c1437 bffe3ac0 bec7c264 
Call Trace:
 [<80105d1f>] show_stack+0x7f/0xa0
 [<80105eba>] show_registers+0x15a/0x1c0
 [<8010607e>] die+0xce/0x150
 [<80116aaf>] do_page_fault+0x2df/0x5c7
 [<8010592d>] error_code+0x2d/0x38
 [<802c3d78>] fn_hash_delete+0x1c8/0x280
 [<802c1437>] inet_rtm_delroute+0x67/0x80
 [<8028b720>] rtnetlink_rcv+0x2e0/0x3b0
 [<8029068a>] netlink_data_ready+0x5a/0x70
 [<8028fc53>] netlink_sendskb+0x93/0xa0
 [<80290327>] netlink_sendmsg+0x1f7/0x2f0
 [<8027a4df>] sock_sendmsg+0xbf/0xe0
 [<8027bdac>] sys_sendmsg+0x1bc/0x240
 [<8027c245>] sys_socketcall+0x215/0x240
 [<80104f23>] syscall_call+0x7/0xb
Code: 8d 5a 08 8b 4b 04 85 c0 89 01 74 03 89 48 04 c7 42 08 00 01 10 00 c7 43 04 00 02 20 00 8d 5a 60 8d 72 64 8b 43 04 8b 4e 04 85 c0 <89> 01 74 03 89 48 04 c7 43 04 00 01 10 00 c7 46 04 00 02 20 00