* Fw: [Bug 32772] New: PROBLEM: kernel BUG at net/ipv4/inetpeer.c:386
@ 2011-04-06 15:18 Stephen Hemminger
2011-04-06 16:05 ` Eric Dumazet
0 siblings, 1 reply; 9+ messages in thread
From: Stephen Hemminger @ 2011-04-06 15:18 UTC (permalink / raw)
To: netdev
Begin forwarded message:
Date: Wed, 6 Apr 2011 07:39:54 GMT
From: bugzilla-daemon@bugzilla.kernel.org
To: shemminger@linux-foundation.org
Subject: [Bug 32772] New: PROBLEM: kernel BUG at net/ipv4/inetpeer.c:386
https://bugzilla.kernel.org/show_bug.cgi?id=32772
Summary: PROBLEM: kernel BUG at net/ipv4/inetpeer.c:386
Product: Networking
Version: 2.5
Kernel Version: 2.6.38
Platform: All
OS/Version: Linux
Tree: Mainline
Status: NEW
Severity: normal
Priority: P1
Component: IPV4
AssignedTo: shemminger@linux-foundation.org
ReportedBy: dimetrios@gmail.com
Regression: No
Kernel oopses periodically with 'kernel BUG at net/ipv4/inetpeer.c:386'
message. Machine is used as BGP router and runs Quagga. Nonordinary kernel
config option set: CONFIG_IP_FIB_TRIE=y.
Two traces:
--------------------trace begin--------------
[625279.329241] kernel BUG at net/ipv4/inetpeer.c:386!
[625279.329241] invalid opcode: 0000 [#1] SMP
[625279.329241] last sysfs file: /sys/module/ip_tables/initstate
[625279.329241] Modules linked in: nf_nat_pptp nf_nat_proto_gre
nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_ftp nf_conntrack_ftp ipt_REJECT
xt_state xt_tcpudp xt_multiport ip_set iptable_filter iptable_mangle
iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ip_tables
x_tables act_police cls_u32 sch_ingress sch_tbf 8021q garp bridge ipv6 stp llc
loop i2c_i801 intel_agp parport_pc i2c_core intel_gtt rng_core agpgart
processor parport button evdev pcspkr thermal_sys serio_raw tpm_tis tpm
tpm_bios ext3 jbd mbcache sd_mod crc_t10dif ata_generic ata_piix libata
scsi_mod uhci_hcd ide_pci_generic e1000e ehci_hcd r8169 ide_core igb dca mii
usbcore nls_base [last unloaded: scsi_wait_scan]
[625279.329241]
[625279.329241] Pid: 0, comm: kworker/0:0 Not tainted 2.6.38-demyan-1.1demyan
#1 Gigabyte Technology Co., Ltd. G41MT-ES2L/G41MT-ES2L
[625279.329241] EIP: 0060:[<c11e0caa>] EFLAGS: 00010283 CPU: 1
[625279.329241] EIP is at unlink_from_pool+0x85/0x14a
[625279.329241] EAX: c125ff04 EBX: ed21cd40 ECX: c351ce70 EDX: e8db5b40
[625279.329241] ESI: c1333338 EDI: f4c91ca0 EBP: c351b55e ESP: f4c91c48
[625279.329241] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
[625279.329241] Process kworker/0:0 (pid: 0, ti=f4c90000 task=f4c6a400
task.ti=f4c8c000)
[625279.329241] Stack:
[625279.329241] f1be9b00 00000001 c351ce70 c133333c c1333338 ed21a684 f11a3f84
f0146f00
[625279.329241] ed2dca80 ed21a900 f0146644 ec4c2f40 f0146280 ec701dc0 f0467fc0
eea79604
[625279.329241] f16c12c0 ef727900 ec865784 e721a3c0 ee859cc4 e8db5b40 00000640
00000014
[625279.329241] Call Trace:
[625279.329241] [<c11ea34a>] ? tcp_tso_segment+0x24d/0x25c
[625279.329241] [<f820048a>] ? tcp_packet+0xb8e/0xbb8 [nf_conntrack]
[625279.329241] [<c11e0de9>] ? cleanup_once+0x7a/0x7f
[625279.329241] [<c11e0fa9>] ? inet_getpeer+0x1bb/0x1dc
[625279.329241] [<c11d0001>] ? store_xps_map+0xa1/0x2b8
[625279.329241] [<c11c3477>] ? dev_hard_start_xmit+0x36f/0x454
[625279.329241] [<c1021ef1>] ? get_nohz_timer_target+0x47/0x64
[625279.329241] [<c11e1cb0>] ? ip4_frag_init+0x66/0x71
[625279.329241] [<c120bb54>] ? inet_frag_find+0x80/0x18d
[625279.329241] [<c11e1dec>] ? ip_defrag+0x131/0x955
[625279.329241] [<f81be0b1>] ? ipv4_conntrack_defrag+0xb0/0xd3
[nf_defrag_ipv4]
[625279.329241] [<c11dc036>] ? nf_iterate+0x32/0x5d
[625279.329241] [<c11e10e0>] ? ip_rcv_finish+0x0/0x31f
[625279.329241] [<c11dc13d>] ? nf_hook_slow+0x40/0xb5
[625279.329241] [<c11e10e0>] ? ip_rcv_finish+0x0/0x31f
[625279.329241] [<c11e164c>] ? ip_rcv+0x24d/0x293
[625279.329241] [<c11e10e0>] ? ip_rcv_finish+0x0/0x31f
[625279.329241] [<c11c1b3c>] ? __netif_receive_skb+0x405/0x42c
[625279.329241] [<c11c1a63>] ? __netif_receive_skb+0x32c/0x42c
[625279.329241] [<c1047585>] ? ktime_get_real+0x10/0x2d
[625279.329241] [<c11c2547>] ? netif_receive_skb+0x5a/0x5f
[625279.329241] [<c11c25ff>] ? napi_skb_finish+0x1b/0x30
[625279.329241] [<f80a9723>] ? igb_poll+0x649/0x94a [igb]
[625279.329241] [<c1007765>] ? sched_clock+0x9/0xd
[625279.329241] [<c1030582>] ? do_exit+0x2e/0x60c
[625279.329241] [<c104438f>] ? sched_clock_local+0x17/0x13d
[625279.329241] [<c11c2b7b>] ? net_rx_action+0x90/0x150
[625279.329241] [<c1031f12>] ? __do_softirq+0x75/0x10e
[625279.329241] [<c1031e9d>] ? __do_softirq+0x0/0x10e
[625279.329241] <IRQ>
[625279.329241] [<c1031df3>] ? irq_exit+0x31/0x64
[625279.329241] [<c1004397>] ? do_IRQ+0x73/0x84
[625279.329241] [<c1003429>] ? common_interrupt+0x29/0x30
[625279.329241] [<c10089b4>] ? mwait_idle+0x4f/0x59
[625279.329241] [<c10021ef>] ? cpu_idle+0x46/0x63
[625279.329241] Code: 24 08 39 cd 75 09 42 3b 54 24 04 7c e9 eb 18 3b 6c 24 08
8d 50 04 0f 42 d0 89 17 83 c7 04 8b 02 3d 04 ff 25 c1 75 bb 39 d8 74 04 <0f> 0b
eb fe 8d 6f fc 81 3b 04 ff 25 c1 89 6c 24 08 75 0d 8b 47
[625279.329241] EIP: [<c11e0caa>] unlink_from_pool+0x85/0x14a SS:ESP
0068:f4c91c48
[625280.416294] ---[ end trace b75ce593ad6cbee7 ]---
[625280.430422] Kernel panic - not syncing: Fatal exception in interrupt
[625280.449739] Pid: 0, comm: kworker/0:0 Tainted: G D
2.6.38-demyan-1.1demyan #1
[625280.473762] Call Trace:
[625280.481380] [<c1231f71>] ? panic+0x4d/0x137
[625280.494457] [<c1005722>] ? oops_end+0x8e/0x99
[625280.508054] [<c1003a0e>] ? do_invalid_op+0x0/0x75
[625280.522693] [<c1003a7a>] ? do_invalid_op+0x6c/0x75
[625280.537588] [<c11e0caa>] ? unlink_from_pool+0x85/0x14a
[625280.553527] [<c11e0bbd>] ? inet_putpeer+0x15/0x47
[625280.568165] [<c11e0d64>] ? unlink_from_pool+0x13f/0x14a
[625280.584367] [<f80a9ed6>] ? igb_xmit_frame_ring_adv+0x4b2/0x795 [igb]
[625280.603941] [<c1007765>] ? sched_clock+0x9/0xd
[625280.617797] [<c123464e>] ? error_code+0x5a/0x60
[625280.631913] [<c1003a0e>] ? do_invalid_op+0x0/0x75
[625280.646552] [<c11e0caa>] ? unlink_from_pool+0x85/0x14a
[625280.662490] [<c11ea34a>] ? tcp_tso_segment+0x24d/0x25c
[625280.678427] [<f820048a>] ? tcp_packet+0xb8e/0xbb8 [nf_conntrack]
[625280.696964] [<c11e0de9>] ? cleanup_once+0x7a/0x7f
[625280.711600] [<c11e0fa9>] ? inet_getpeer+0x1bb/0x1dc
[625280.726758] [<c11d0001>] ? store_xps_map+0xa1/0x2b8
[625280.741916] [<c11c3477>] ? dev_hard_start_xmit+0x36f/0x454
[625280.758894] [<c1021ef1>] ? get_nohz_timer_target+0x47/0x64
[625280.775870] [<c11e1cb0>] ? ip4_frag_init+0x66/0x71
[625280.790768] [<c120bb54>] ? inet_frag_find+0x80/0x18d
[625280.806184] [<c11e1dec>] ? ip_defrag+0x131/0x955
[625280.820562] [<f81be0b1>] ? ipv4_conntrack_defrag+0xb0/0xd3
[nf_defrag_ipv4]
[625280.841961] [<c11dc036>] ? nf_iterate+0x32/0x5d
[625280.856078] [<c11e10e0>] ? ip_rcv_finish+0x0/0x31f
[625280.870975] [<c11dc13d>] ? nf_hook_slow+0x40/0xb5
[625280.885612] [<c11e10e0>] ? ip_rcv_finish+0x0/0x31f
[625280.900510] [<c11e164c>] ? ip_rcv+0x24d/0x293
[625280.914107] [<c11e10e0>] ? ip_rcv_finish+0x0/0x31f
[625280.929005] [<c11c1b3c>] ? __netif_receive_skb+0x405/0x42c
[625280.945982] [<c11c1a63>] ? __netif_receive_skb+0x32c/0x42c
[625280.962960] [<c1047585>] ? ktime_get_real+0x10/0x2d
[625280.978121] [<c11c2547>] ? netif_receive_skb+0x5a/0x5f
[625280.994055] [<c11c25ff>] ? napi_skb_finish+0x1b/0x30
[625281.009473] [<f80a9723>] ? igb_poll+0x649/0x94a [igb]
[625281.025150] [<c1007765>] ? sched_clock+0x9/0xd
[625281.039005] [<c1030582>] ? do_exit+0x2e/0x60c
[625281.052603] [<c104438f>] ? sched_clock_local+0x17/0x13d
[625281.068800] [<c11c2b7b>] ? net_rx_action+0x90/0x150
[625281.083958] [<c1031f12>] ? __do_softirq+0x75/0x10e
[625281.098857] [<c1031e9d>] ? __do_softirq+0x0/0x10e
[625281.113493] <IRQ> [<c1031df3>] ? irq_exit+0x31/0x64
[625281.128963] [<c1004397>] ? do_IRQ+0x73/0x84
[625281.142040] [<c1003429>] ? common_interrupt+0x29/0x30
[625281.157718] [<c10089b4>] ? mwait_idle+0x4f/0x59
[625281.171836] [<c10021ef>] ? cpu_idle+0x46/0x63
[625281.185435] Rebooting in 5 seconds..
--------------------trace end--------------
--------------------trace begin--------------
[237684.673906] kernel BUG at net/ipv4/inetpeer.c:386!
[237684.673906] invalid opcode: 0000 [#1] SMP
[237684.673906] last sysfs file: /sys/module/nf_conntrack_pptp/initstate
[237684.673906] Modules linked in: nf_nat_pptp nf_nat_proto_gre
nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_ftp nf_conntrack_ftp ipt_REJECT
xt_state xt_tcpudp xt_multiport ip_set iptable_filter iptable_mangle
iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ip_tables
x_tables act_police cls_u32 sch_ingress sch_tbf 8021q garp bridge ipv6 stp llc
loop i2c_i801 rng_core intel_agp intel_gtt agpgart i2c_core tpm_tis evdev
pcspkr parport_pc processor parport button tpm thermal_sys tpm_bios serio_raw
ext3 jbd mbcache sd_mod crc_t10dif ata_generic ata_piix libata scsi_mod
uhci_hcd ide_pci_generic r8169 ehci_hcd e1000e mii igb dca ide_core usbcore
nls_base [last unloaded: scsi_wait_scan]
[237684.673906]
[237684.673906] Pid: 0, comm: swapper Not tainted 2.6.38-demyan-1.1demyan #1
Gigabyte Technology Co., Ltd. G41MT-ES2L/G41MT-ES2L
[237684.673906] EIP: 0060:[<c11e0caa>] EFLAGS: 00010287 CPU: 0
[237684.673906] EIP is at unlink_from_pool+0x85/0x14a
[237684.673906] EAX: c125ff04 EBX: ed76d180 ECX: 75c219bc EDX: e8de9444
[237684.673906] ESI: c1333338 EDI: f4c0bbfc EBP: 75c25152 ESP: f4c0bba8
[237684.673906] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
[237684.673906] Process swapper (pid: 0, ti=f4c0a000 task=c1315f20
task.ti=c1300000)
[237684.673906] Stack:
[237684.673906] ef42e49c 00000001 75c219bc c133333c c1333338 f4780d80 ed76d744
f19e3744
[237684.673906] ed71f980 f452a404 f183fc04 f452afc4 f4780ac0 f18ca680 f474fe84
f1871180
[237684.673906] ee2a9884 edef3844 f1cf3e04 edf15284 e8de9444 f4c0bcb4 ef42e49c
f4c0bc78
[237684.673906] Call Trace:
[237684.673906] [<c120f068>] ? fib4_rule_action+0x40/0x4d
[237684.673906] [<c11d1be3>] ? fib_rules_lookup+0x8d/0xe4
[237684.673906] [<c109bf68>] ? cache_alloc_refill+0x75/0x3dc
[237684.673906] [<c11e0de9>] ? cleanup_once+0x7a/0x7f
[237684.673906] [<c11e0fa9>] ? inet_getpeer+0x1bb/0x1dc
[237684.673906] [<c11dc073>] ? nf_ct_attach+0x12/0x13
[237684.673906] [<c1202404>] ? icmp_glue_bits+0x65/0x6a
[237684.673906] [<c11e4109>] ? ip_append_data+0x595/0x850
[237684.673906] [<c11e025d>] ? rt_bind_peer+0x1d/0x3d
[237684.673906] [<c11e029f>] ? __ip_select_ident+0x22/0xa6
[237684.673906] [<c11e4f60>] ? ip_push_pending_frames+0x206/0x2cb
[237684.673906] [<c120301b>] ? icmp_send+0x4fe/0x523
[237684.673906] [<f8270b09>] ? ____nf_conntrack_find+0xfa/0x142 [nf_conntrack]
[237684.673906] [<f8272069>] ? nf_conntrack_in+0x4f3/0x5e3 [nf_conntrack]
[237684.673906] [<f81ef536>] ? ipt_do_table+0x4bc/0x4eb [ip_tables]
[237684.673906] [<c11e2949>] ? ip_forward+0x2ef/0x316
[237684.673906] [<c11e13da>] ? ip_rcv_finish+0x2fa/0x31f
[237684.673906] [<c11c1b3c>] ? __netif_receive_skb+0x405/0x42c
[237684.673906] [<c11c1a63>] ? __netif_receive_skb+0x32c/0x42c
[237684.673906] [<c1047585>] ? ktime_get_real+0x10/0x2d
[237684.673906] [<c11c2547>] ? netif_receive_skb+0x5a/0x5f
[237684.673906] [<c11c25ff>] ? napi_skb_finish+0x1b/0x30
[237684.673906] [<f80e1723>] ? igb_poll+0x649/0x94a [igb]
[237684.673906] [<c1007765>] ? sched_clock+0x9/0xd
[237684.673906] [<c1030091>] ? wait_consider_task+0x974/0xa91
[237684.673906] [<c104438f>] ? sched_clock_local+0x17/0x13d
[237684.673906] [<c11c2b7b>] ? net_rx_action+0x90/0x150
[237684.673906] [<c1031f12>] ? __do_softirq+0x75/0x10e
[237684.673906] [<c1031e9d>] ? __do_softirq+0x0/0x10e
[237684.673906] <IRQ>
[237684.673906] [<c1031df3>] ? irq_exit+0x31/0x64
[237684.673906] [<c1004397>] ? do_IRQ+0x73/0x84
[237684.673906] [<c1003429>] ? common_interrupt+0x29/0x30
[237684.673906] [<c10089b4>] ? mwait_idle+0x4f/0x59
[237684.673906] [<c10021ef>] ? cpu_idle+0x46/0x63
[237684.673906] [<c133b85c>] ? start_kernel+0x2e2/0x2e5
[237684.673906] Code: 24 08 39 cd 75 09 42 3b 54 24 04 7c e9 eb 18 3b 6c 24 08
8d 50 04 0f 42 d0 89 17 83 c7 04 8b 02 3d 04 ff 25 c1 75 bb 39 d8 74 04 <0f> 0b
eb fe 8d 6f fc 81 3b 04 ff 25 c1 89 6c 24 08 75 0d 8b 47
[237684.673906] EIP: [<c11e0caa>] unlink_from_pool+0x85/0x14a SS:ESP
0068:f4c0bba8
[237685.787747] ---[ end trace e3c73323a4e3b283 ]---
[237685.801876] Kernel panic - not syncing: Fatal exception in interrupt
[237685.821194] Pid: 0, comm: swapper Tainted: G D
2.6.38-demyan-1.1demyan #1
[237685.844177] Call Trace:
[237685.851797] [<c1231f71>] ? panic+0x4d/0x137
[237685.864874] [<c1005722>] ? oops_end+0x8e/0x99
[237685.878471] [<c1003a0e>] ? do_invalid_op+0x0/0x75
[237685.893109] [<c1003a7a>] ? do_invalid_op+0x6c/0x75
[237685.908005] [<c11e0caa>] ? unlink_from_pool+0x85/0x14a
[237685.923942] [<c1007765>] ? sched_clock+0x9/0xd
[237685.937801] [<c1007765>] ? sched_clock+0x9/0xd
[237685.951658] [<c104438f>] ? sched_clock_local+0x17/0x13d
[237685.967856] [<c123464e>] ? error_code+0x5a/0x60
[237685.981973] [<c1003a0e>] ? do_invalid_op+0x0/0x75
[237685.996610] [<c11e0caa>] ? unlink_from_pool+0x85/0x14a
[237686.012548] [<c120f068>] ? fib4_rule_action+0x40/0x4d
[237686.028225] [<c11d1be3>] ? fib_rules_lookup+0x8d/0xe4
[237686.043902] [<c109bf68>] ? cache_alloc_refill+0x75/0x3dc
[237686.060359] [<c11e0de9>] ? cleanup_once+0x7a/0x7f
[237686.074997] [<c11e0fa9>] ? inet_getpeer+0x1bb/0x1dc
[237686.090156] [<c11dc073>] ? nf_ct_attach+0x12/0x13
[237686.104792] [<c1202404>] ? icmp_glue_bits+0x65/0x6a
[237686.119949] [<c11e4109>] ? ip_append_data+0x595/0x850
[237686.135626] [<c11e025d>] ? rt_bind_peer+0x1d/0x3d
[237686.150264] [<c11e029f>] ? __ip_select_ident+0x22/0xa6
[237686.166202] [<c11e4f60>] ? ip_push_pending_frames+0x206/0x2cb
[237686.183959] [<c120301b>] ? icmp_send+0x4fe/0x523
[237686.198338] [<f8270b09>] ? ____nf_conntrack_find+0xfa/0x142 [nf_conntrack]
[237686.219474] [<f8272069>] ? nf_conntrack_in+0x4f3/0x5e3 [nf_conntrack]
[237686.239311] [<f81ef536>] ? ipt_do_table+0x4bc/0x4eb [ip_tables]
[237686.257589] [<c11e2949>] ? ip_forward+0x2ef/0x316
[237686.272227] [<c11e13da>] ? ip_rcv_finish+0x2fa/0x31f
[237686.287643] [<c11c1b3c>] ? __netif_receive_skb+0x405/0x42c
[237686.304620] [<c11c1a63>] ? __netif_receive_skb+0x32c/0x42c
[237686.321599] [<c1047585>] ? ktime_get_real+0x10/0x2d
[237686.336760] [<c11c2547>] ? netif_receive_skb+0x5a/0x5f
[237686.352692] [<c11c25ff>] ? napi_skb_finish+0x1b/0x30
[237686.368111] [<f80e1723>] ? igb_poll+0x649/0x94a [igb]
[237686.383788] [<c1007765>] ? sched_clock+0x9/0xd
[237686.397645] [<c1030091>] ? wait_consider_task+0x974/0xa91
[237686.414362] [<c104438f>] ? sched_clock_local+0x17/0x13d
[237686.430559] [<c11c2b7b>] ? net_rx_action+0x90/0x150
[237686.445718] [<c1031f12>] ? __do_softirq+0x75/0x10e
[237686.460614] [<c1031e9d>] ? __do_softirq+0x0/0x10e
[237686.475251] <IRQ> [<c1031df3>] ? irq_exit+0x31/0x64
[237686.490722] [<c1004397>] ? do_IRQ+0x73/0x84
[237686.503799] [<c1003429>] ? common_interrupt+0x29/0x30
[237686.519476] [<c10089b4>] ? mwait_idle+0x4f/0x59
[237686.533593] [<c10021ef>] ? cpu_idle+0x46/0x63
[237686.547191] [<c133b85c>] ? start_kernel+0x2e2/0x2e5
[237686.562350] Rebooting in 5 seconds..
--------------------trace end--------------
--
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
--
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Fw: [Bug 32772] New: PROBLEM: kernel BUG at net/ipv4/inetpeer.c:386
2011-04-06 15:18 Fw: [Bug 32772] New: PROBLEM: kernel BUG at net/ipv4/inetpeer.c:386 Stephen Hemminger
@ 2011-04-06 16:05 ` Eric Dumazet
2011-04-06 16:25 ` Eric Dumazet
0 siblings, 1 reply; 9+ messages in thread
From: Eric Dumazet @ 2011-04-06 16:05 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: netdev, dimetrios, David Miller
Le mercredi 06 avril 2011 à 08:18 -0700, Stephen Hemminger a écrit :
>
> Begin forwarded message:
>
> Date: Wed, 6 Apr 2011 07:39:54 GMT
> From: bugzilla-daemon@bugzilla.kernel.org
> To: shemminger@linux-foundation.org
> Subject: [Bug 32772] New: PROBLEM: kernel BUG at net/ipv4/inetpeer.c:386
>
>
> https://bugzilla.kernel.org/show_bug.cgi?id=32772
>
> Summary: PROBLEM: kernel BUG at net/ipv4/inetpeer.c:386
> Product: Networking
> Version: 2.5
> Kernel Version: 2.6.38
> Platform: All
> OS/Version: Linux
> Tree: Mainline
> Status: NEW
> Severity: normal
> Priority: P1
> Component: IPV4
> AssignedTo: shemminger@linux-foundation.org
> ReportedBy: dimetrios@gmail.com
> Regression: No
>
>
> Kernel oopses periodically with 'kernel BUG at net/ipv4/inetpeer.c:386'
> message. Machine is used as BGP router and runs Quagga. Nonordinary kernel
> config option set: CONFIG_IP_FIB_TRIE=y.
> Two traces:
> --------------------trace begin--------------
> [625279.329241] kernel BUG at net/ipv4/inetpeer.c:386!
Hmm...
if (atomic_cmpxchg(&p->refcnt, 1, -1) == 1) {
struct inet_peer __rcu **stack[PEER_MAXDEPTH];
struct inet_peer __rcu ***stackptr, ***delp;
if (lookup(&p->daddr, stack, base) != p)
BUG();
So we cant find a peer in AVL tree, while we really should at this stage.
This reminds me a possible memory corruption (from another layer)
Could Dmitry try to boot with boot parameter "slub_nomerge" , to make sure
inetpeer layer doesnt share its kmem_cache with a corrupter ?
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Fw: [Bug 32772] New: PROBLEM: kernel BUG at net/ipv4/inetpeer.c:386
2011-04-06 16:05 ` Eric Dumazet
@ 2011-04-06 16:25 ` Eric Dumazet
2011-04-06 18:15 ` Dmitry Novikov
0 siblings, 1 reply; 9+ messages in thread
From: Eric Dumazet @ 2011-04-06 16:25 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: netdev, dimetrios, David Miller
Le mercredi 06 avril 2011 à 18:05 +0200, Eric Dumazet a écrit :
>
> This reminds me a possible memory corruption (from another layer)
>
I found the reference of a past bug report, where slub_nomerge was used
too.
http://www.spinics.net/lists/netdev/msg154206.html
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Fw: [Bug 32772] New: PROBLEM: kernel BUG at net/ipv4/inetpeer.c:386
2011-04-06 16:25 ` Eric Dumazet
@ 2011-04-06 18:15 ` Dmitry Novikov
2011-04-06 18:16 ` David Miller
0 siblings, 1 reply; 9+ messages in thread
From: Dmitry Novikov @ 2011-04-06 18:15 UTC (permalink / raw)
To: Eric Dumazet; +Cc: Stephen Hemminger, netdev, David Miller
2011/4/6 Eric Dumazet <eric.dumazet@gmail.com>
>
> Le mercredi 06 avril 2011 à 18:05 +0200, Eric Dumazet a écrit :
>
> >
> > This reminds me a possible memory corruption (from another layer)
> >
>
> I found the reference of a past bug report, where slub_nomerge was used
> too.
>
> http://www.spinics.net/lists/netdev/msg154206.html
>
>
>
I will schedule reboot in 6 hours so that slub_nomerge parmeter been applied
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Bug 32772] New: PROBLEM: kernel BUG at net/ipv4/inetpeer.c:386
2011-04-06 18:15 ` Dmitry Novikov
@ 2011-04-06 18:16 ` David Miller
2011-04-13 20:14 ` Dmitry Novikov
0 siblings, 1 reply; 9+ messages in thread
From: David Miller @ 2011-04-06 18:16 UTC (permalink / raw)
To: dimetrios; +Cc: eric.dumazet, shemminger, netdev
From: Dmitry Novikov <dimetrios@gmail.com>
Date: Wed, 6 Apr 2011 21:15:17 +0300
> 2011/4/6 Eric Dumazet <eric.dumazet@gmail.com>
>>
>> Le mercredi 06 avril 2011 à 18:05 +0200, Eric Dumazet a écrit :
>>
>> >
>> > This reminds me a possible memory corruption (from another layer)
>> >
>>
>> I found the reference of a past bug report, where slub_nomerge was used
>> too.
>>
>> http://www.spinics.net/lists/netdev/msg154206.html
>>
>>
>>
> I will schedule reboot in 6 hours so that slub_nomerge parmeter been applied
Thanks for helping us track this down.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Bug 32772] New: PROBLEM: kernel BUG at net/ipv4/inetpeer.c:386
2011-04-06 18:16 ` David Miller
@ 2011-04-13 20:14 ` Dmitry Novikov
2011-04-13 20:24 ` David Miller
0 siblings, 1 reply; 9+ messages in thread
From: Dmitry Novikov @ 2011-04-13 20:14 UTC (permalink / raw)
To: David Miller; +Cc: eric.dumazet, shemminger, netdev
Hello.
Crash again after 7 days of uptime. slub_nomerge is set
[559353.216526] ------------[ cut here ]------------
[559353.217494] kernel BUG at net/ipv4/inetpeer.c:386!
[559353.217494] invalid opcode: 0000 [#1] SMP
[559353.217494] last sysfs file: /sys/module/nf_conntrack_pptp/initstate
[559353.217494] Modules linked in: nf_nat_pptp nf_nat_proto_gre
nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_ftp nf_conntrack_ftp
ipt_REJECT xt_state xt_tcpudp xt_multiport ip_set iptable_filter
iptable_mangle iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack
nf_defrag_ipv4 ip_tables x_tables act_police cls_u32 sch_ingress
sch_tbf 8021q garp bridge ipv6 stp llc loop intel_agp intel_gtt
agpgart rng_core pcspkr i2c_i801 i2c_core processor thermal_sys
parport_pc evdev parport serio_raw tpm_tis tpm button tpm_bios ext3
jbd mbcache sd_mod crc_t10dif ata_generic ata_piix libata scsi_mod
uhci_hcd ide_pci_generic e1000e ehci_hcd igb r8169 ide_core dca mii
usbcore nls_base [last unloaded: scsi_wait_scan]
[559353.217494]
[559353.217494] Pid: 0, comm: kworker/0:0 Not tainted
2.6.38-demyan-1.1demyan #1 Gigabyte Technology Co., Ltd.
G41MT-ES2L/G41MT-ES2L
[559353.217494] EIP: 0060:[<c11e0caa>] EFLAGS: 00010287 CPU: 1
[559353.217494] EIP is at unlink_from_pool+0x85/0x14a
[559353.217494] EAX: c125ff04 EBX: efcb09c0 ECX: abfd6970 EDX: ee6d77c4
[559353.217494] ESI: c1333338 EDI: f4c91bfc EBP: abfea42e ESP: f4c91ba8
[559353.217494] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
[559353.217494] Process kworker/0:0 (pid: 0, ti=f4c90000 task=f4c6a400
task.ti=f4c8c000)
[559353.217494] Stack:
[559353.217494] f351c790 00000001 abfd6970 c133333c c1333338 efc6b384
efe2af80 efcd1c04
[559353.217494] f3cc2784 ef3f62c4 f251da80 ef054b40 efcdc300 f0373dc0
f429a144 ef254a80
[559353.217494] ed4e6340 f0705f40 efcdb580 f05261c0 ee6d77c4 f4c91cb4
f351c790 f4c91c78
[559353.217494] Call Trace:
[559353.217494] [<c120f068>] ? fib4_rule_action+0x40/0x4d
[559353.217494] [<c11d1be3>] ? fib_rules_lookup+0x8d/0xe4
[559353.217494] [<c11e0de9>] ? cleanup_once+0x7a/0x7f
[559353.217494] [<c11e0fa9>] ? inet_getpeer+0x1bb/0x1dc
[559353.217494] [<c11dc073>] ? nf_ct_attach+0x12/0x13
[559353.217494] [<c1202404>] ? icmp_glue_bits+0x65/0x6a
[559353.217494] [<c11e4109>] ? ip_append_data+0x595/0x850
[559353.217494] [<c11e025d>] ? rt_bind_peer+0x1d/0x3d
[559353.217494] [<c11e029f>] ? __ip_select_ident+0x22/0xa6
[559353.217494] [<c11e4f60>] ? ip_push_pending_frames+0x206/0x2cb
[559353.217494] [<c120301b>] ? icmp_send+0x4fe/0x523
[559353.217494] [<f81a6b09>] ? ____nf_conntrack_find+0xfa/0x142 [nf_conntrack]
[559353.217494] [<f81a8069>] ? nf_conntrack_in+0x4f3/0x5e3 [nf_conntrack]
[559353.217494] [<f815c536>] ? ipt_do_table+0x4bc/0x4eb [ip_tables]
[559353.217494] [<c11e2949>] ? ip_forward+0x2ef/0x316
[559353.217494] [<c11e13da>] ? ip_rcv_finish+0x2fa/0x31f
[559353.217494] [<c11c1b3c>] ? __netif_receive_skb+0x405/0x42c
[559353.217494] [<c11c1a63>] ? __netif_receive_skb+0x32c/0x42c
[559353.217494] [<c1047585>] ? ktime_get_real+0x10/0x2d
[559353.217494] [<c11c2547>] ? netif_receive_skb+0x5a/0x5f
[559353.217494] [<c11c25ff>] ? napi_skb_finish+0x1b/0x30
[559353.217494] [<f8104723>] ? igb_poll+0x649/0x94a [igb]
[559353.217494] [<c1007765>] ? sched_clock+0x9/0xd
[559353.217494] [<c1030094>] ? wait_consider_task+0x977/0xa91
[559353.217494] [<c104438f>] ? sched_clock_local+0x17/0x13d
[559353.217494] [<c11c2b7b>] ? net_rx_action+0x90/0x150
[559353.217494] [<c1031f12>] ? __do_softirq+0x75/0x10e
[559353.217494] [<c1031e9d>] ? __do_softirq+0x0/0x10e
[559353.217494] <IRQ>
[559353.217494] [<c1031df3>] ? irq_exit+0x31/0x64
[559353.217494] [<c1004397>] ? do_IRQ+0x73/0x84
[559353.217494] [<c1003429>] ? common_interrupt+0x29/0x30
[559353.217494] [<c10089b4>] ? mwait_idle+0x4f/0x59
[559353.217494] [<c10021ef>] ? cpu_idle+0x46/0x63
[559353.217494] Code: 24 08 39 cd 75 09 42 3b 54 24 04 7c e9 eb 18 3b
6c 24 08 8d 50 04 0f 42 d0 89 17 83 c7 04 8b 02 3d 04 ff 25 c1 75 bb
39 d8 74 04 <0f> 0b eb fe 8d 6f fc 81 3b 04 ff 25 c1 89 6c 24 08 75 0d
8b 47
[559353.217494] EIP: [<c11e0caa>] unlink_from_pool+0x85/0x14a SS:ESP
0068:f4c91ba8
[559354.302112] ---[ end trace 55cdab910854890a ]---
[559354.316239] Kernel panic - not syncing: Fatal exception in interrupt
[559354.335557] Pid: 0, comm: kworker/0:0 Tainted: G D
2.6.38-demyan-1.1demyan #1
[559354.359578] Call Trace:
[559354.367198] [<c1231f71>] ? panic+0x4d/0x137
[559354.380274] [<c1005722>] ? oops_end+0x8e/0x99
[559354.393871] [<c1003a0e>] ? do_invalid_op+0x0/0x75
[559354.408509] [<c1003a7a>] ? do_invalid_op+0x6c/0x75
[559354.423407] [<c11e0caa>] ? unlink_from_pool+0x85/0x14a
[559354.439345] [<c120f068>] ? fib4_rule_action+0x40/0x4d
[559354.455022] [<c11d1be3>] ? fib_rules_lookup+0x8d/0xe4
[559354.470700] [<c120f122>] ? fib_lookup+0x31/0x3f
[559354.484818] [<c11ca4f1>] ? neigh_lookup+0x8e/0x96
[559354.499454] [<c123464e>] ? error_code+0x5a/0x60
[559354.513571] [<c1003a0e>] ? do_invalid_op+0x0/0x75
[559354.528208] [<c11e0caa>] ? unlink_from_pool+0x85/0x14a
[559354.544146] [<c120f068>] ? fib4_rule_action+0x40/0x4d
[559354.559823] [<c11d1be3>] ? fib_rules_lookup+0x8d/0xe4
[559354.575500] [<c11e0de9>] ? cleanup_once+0x7a/0x7f
[559354.590137] [<c11e0fa9>] ? inet_getpeer+0x1bb/0x1dc
[559354.605297] [<c11dc073>] ? nf_ct_attach+0x12/0x13
[559354.619934] [<c1202404>] ? icmp_glue_bits+0x65/0x6a
[559354.635090] [<c11e4109>] ? ip_append_data+0x595/0x850
[559354.650767] [<c11e025d>] ? rt_bind_peer+0x1d/0x3d
[559354.665405] [<c11e029f>] ? __ip_select_ident+0x22/0xa6
[559354.681344] [<c11e4f60>] ? ip_push_pending_frames+0x206/0x2cb
[559354.699099] [<c120301b>] ? icmp_send+0x4fe/0x523
[559354.713479] [<f81a6b09>] ? ____nf_conntrack_find+0xfa/0x142 [nf_conntrack]
[559354.734615] [<f81a8069>] ? nf_conntrack_in+0x4f3/0x5e3 [nf_conntrack]
[559354.754452] [<f815c536>] ? ipt_do_table+0x4bc/0x4eb [ip_tables]
[559354.772731] [<c11e2949>] ? ip_forward+0x2ef/0x316
[559354.787366] [<c11e13da>] ? ip_rcv_finish+0x2fa/0x31f
[559354.802785] [<c11c1b3c>] ? __netif_receive_skb+0x405/0x42c
[559354.819762] [<c11c1a63>] ? __netif_receive_skb+0x32c/0x42c
[559354.836738] [<c1047585>] ? ktime_get_real+0x10/0x2d
[559354.851901] [<c11c2547>] ? netif_receive_skb+0x5a/0x5f
[559354.867835] [<c11c25ff>] ? napi_skb_finish+0x1b/0x30
[559354.883254] [<f8104723>] ? igb_poll+0x649/0x94a [igb]
[559354.898930] [<c1007765>] ? sched_clock+0x9/0xd
[559354.912786] [<c1030094>] ? wait_consider_task+0x977/0xa91
[559354.929502] [<c104438f>] ? sched_clock_local+0x17/0x13d
[559354.945701] [<c11c2b7b>] ? net_rx_action+0x90/0x150
[559354.960857] [<c1031f12>] ? __do_softirq+0x75/0x10e
[559354.975756] [<c1031e9d>] ? __do_softirq+0x0/0x10e
[559354.990393] <IRQ> [<c1031df3>] ? irq_exit+0x31/0x64
[559355.005862] [<c1004397>] ? do_IRQ+0x73/0x84
[559355.018941] [<c1003429>] ? common_interrupt+0x29/0x30
[559355.034618] [<c10089b4>] ? mwait_idle+0x4f/0x59
[559355.048734] [<c10021ef>] ? cpu_idle+0x46/0x63
[559355.062333] Rebooting in 5 seconds..
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Bug 32772] New: PROBLEM: kernel BUG at net/ipv4/inetpeer.c:386
2011-04-13 20:14 ` Dmitry Novikov
@ 2011-04-13 20:24 ` David Miller
2011-04-14 13:02 ` Dmitry Novikov
0 siblings, 1 reply; 9+ messages in thread
From: David Miller @ 2011-04-13 20:24 UTC (permalink / raw)
To: dimetrios; +Cc: eric.dumazet, shemminger, netdev
From: Dmitry Novikov <dimetrios@gmail.com>
Date: Wed, 13 Apr 2011 23:14:03 +0300
> Crash again after 7 days of uptime. slub_nomerge is set
Looks like too deep stack, try this patch which is in net-2.6:
--------------------
inetpeer: reduce stack usage
On 64bit arches, we use 752 bytes of stack when cleanup_once() is called
from inet_getpeer().
Lets share the avl stack to save ~376 bytes.
Before patch :
# objdump -d net/ipv4/inetpeer.o | scripts/checkstack.pl
0x000006c3 unlink_from_pool [inetpeer.o]: 376
0x00000721 unlink_from_pool [inetpeer.o]: 376
0x00000cb1 inet_getpeer [inetpeer.o]: 376
0x00000e6d inet_getpeer [inetpeer.o]: 376
0x0004 inet_initpeers [inetpeer.o]: 112
# size net/ipv4/inetpeer.o
text data bss dec hex filename
5320 432 21 5773 168d net/ipv4/inetpeer.o
After patch :
objdump -d net/ipv4/inetpeer.o | scripts/checkstack.pl
0x00000c11 inet_getpeer [inetpeer.o]: 376
0x00000dcd inet_getpeer [inetpeer.o]: 376
0x00000ab9 peer_check_expire [inetpeer.o]: 328
0x00000b7f peer_check_expire [inetpeer.o]: 328
0x0004 inet_initpeers [inetpeer.o]: 112
# size net/ipv4/inetpeer.o
text data bss dec hex filename
5163 432 21 5616 15f0 net/ipv4/inetpeer.o
Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Scot Doyle <lkml@scotdoyle.com>
Cc: Stephen Hemminger <shemminger@vyatta.com>
Cc: Hiroaki SHIMODA <shimoda.hiroaki@gmail.com>
Reviewed-by: Hiroaki SHIMODA <shimoda.hiroaki@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
---
net/ipv4/inetpeer.c | 13 +++++++------
1 files changed, 7 insertions(+), 6 deletions(-)
diff --git a/net/ipv4/inetpeer.c b/net/ipv4/inetpeer.c
index dd1b20e..9df4e63 100644
--- a/net/ipv4/inetpeer.c
+++ b/net/ipv4/inetpeer.c
@@ -354,7 +354,8 @@ static void inetpeer_free_rcu(struct rcu_head *head)
}
/* May be called with local BH enabled. */
-static void unlink_from_pool(struct inet_peer *p, struct inet_peer_base *base)
+static void unlink_from_pool(struct inet_peer *p, struct inet_peer_base *base,
+ struct inet_peer __rcu **stack[PEER_MAXDEPTH])
{
int do_free;
@@ -368,7 +369,6 @@ static void unlink_from_pool(struct inet_peer *p, struct inet_peer_base *base)
* We use refcnt=-1 to alert lockless readers this entry is deleted.
*/
if (atomic_cmpxchg(&p->refcnt, 1, -1) == 1) {
- struct inet_peer __rcu **stack[PEER_MAXDEPTH];
struct inet_peer __rcu ***stackptr, ***delp;
if (lookup(&p->daddr, stack, base) != p)
BUG();
@@ -422,7 +422,7 @@ static struct inet_peer_base *peer_to_base(struct inet_peer *p)
}
/* May be called with local BH enabled. */
-static int cleanup_once(unsigned long ttl)
+static int cleanup_once(unsigned long ttl, struct inet_peer __rcu **stack[PEER_MAXDEPTH])
{
struct inet_peer *p = NULL;
@@ -454,7 +454,7 @@ static int cleanup_once(unsigned long ttl)
* happen because of entry limits in route cache. */
return -1;
- unlink_from_pool(p, peer_to_base(p));
+ unlink_from_pool(p, peer_to_base(p), stack);
return 0;
}
@@ -524,7 +524,7 @@ struct inet_peer *inet_getpeer(struct inetpeer_addr *daddr, int create)
if (base->total >= inet_peer_threshold)
/* Remove one less-recently-used entry. */
- cleanup_once(0);
+ cleanup_once(0, stack);
return p;
}
@@ -540,6 +540,7 @@ static void peer_check_expire(unsigned long dummy)
{
unsigned long now = jiffies;
int ttl, total;
+ struct inet_peer __rcu **stack[PEER_MAXDEPTH];
total = compute_total();
if (total >= inet_peer_threshold)
@@ -548,7 +549,7 @@ static void peer_check_expire(unsigned long dummy)
ttl = inet_peer_maxttl
- (inet_peer_maxttl - inet_peer_minttl) / HZ *
total / inet_peer_threshold * HZ;
- while (!cleanup_once(ttl)) {
+ while (!cleanup_once(ttl, stack)) {
if (jiffies != now)
break;
}
--
1.7.4.3
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [Bug 32772] New: PROBLEM: kernel BUG at net/ipv4/inetpeer.c:386
2011-04-13 20:24 ` David Miller
@ 2011-04-14 13:02 ` Dmitry Novikov
2011-04-14 13:32 ` Eric Dumazet
0 siblings, 1 reply; 9+ messages in thread
From: Dmitry Novikov @ 2011-04-14 13:02 UTC (permalink / raw)
To: David Miller; +Cc: eric.dumazet, shemminger, netdev
Thanks. Patch applied . Will wait results
2011/4/13 David Miller <davem@davemloft.net>:
> From: Dmitry Novikov <dimetrios@gmail.com>
> Date: Wed, 13 Apr 2011 23:14:03 +0300
>
>> Crash again after 7 days of uptime. slub_nomerge is set
>
> Looks like too deep stack, try this patch which is in net-2.6:
>
> --------------------
> inetpeer: reduce stack usage
>
> On 64bit arches, we use 752 bytes of stack when cleanup_once() is called
> from inet_getpeer().
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Bug 32772] New: PROBLEM: kernel BUG at net/ipv4/inetpeer.c:386
2011-04-14 13:02 ` Dmitry Novikov
@ 2011-04-14 13:32 ` Eric Dumazet
0 siblings, 0 replies; 9+ messages in thread
From: Eric Dumazet @ 2011-04-14 13:32 UTC (permalink / raw)
To: Dmitry Novikov; +Cc: David Miller, shemminger, netdev
Le jeudi 14 avril 2011 à 16:02 +0300, Dmitry Novikov a écrit :
> Thanks. Patch applied . Will wait results
Thanks to you Dmitry, for your patience
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2011-04-14 13:32 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-04-06 15:18 Fw: [Bug 32772] New: PROBLEM: kernel BUG at net/ipv4/inetpeer.c:386 Stephen Hemminger
2011-04-06 16:05 ` Eric Dumazet
2011-04-06 16:25 ` Eric Dumazet
2011-04-06 18:15 ` Dmitry Novikov
2011-04-06 18:16 ` David Miller
2011-04-13 20:14 ` Dmitry Novikov
2011-04-13 20:24 ` David Miller
2011-04-14 13:02 ` Dmitry Novikov
2011-04-14 13:32 ` Eric Dumazet
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).