* Re: 2.4.30-hf1 do_IRQ stack overflows [not found] <5.2.1.1.2.20050511093252.01c887b0@mail.meteodat.ch> @ 2005-05-11 12:46 ` Marcelo Tosatti 2005-05-11 21:38 ` Herbert Xu ` (2 more replies) 0 siblings, 3 replies; 9+ messages in thread From: Marcelo Tosatti @ 2005-05-11 12:46 UTC (permalink / raw) To: Manfred Schwarb; +Cc: linux-kernel, davem, netdev [-- Attachment #1: Type: text/plain, Size: 1371 bytes --] Hi Manfred, On Wed, May 11, 2005 at 10:15:02AM +0200, Manfred Schwarb wrote: > Hi, > with recent versions of the 2.4 kernel (Vanilla), I get an increasing amount of do_IRQ stack overflows. > This night, I got 3 of them. > With 2.4.28 I got an overflow about twice a year, with 2.4.29 nearly once a month and with > 2.4.30 nearly every day 8-(( The system is getting dangerously close to an actual stack overflow, which would crash the system. "do_IRQ: stack overflow: " indicates how many bytes are still available. The traces show huge networking execution paths. It seems you are using some packet scheduler (CONFIG_NET_SCHED)? Pretty much all traces show functions from sch_generic.c. Can you disable that for a test? > My layout: Pentium4 HT SMP, raid1 on a promise card, driven by libata_promise, > everything using reiserfs, heavy nfs and network traffic, with a Linksys (tulip) and a Realtek (8139too) NIC. > The used 2.4.30-hf1 is pure Vanilla. > > Below my three overflow messages. Would the stack reduction patches of Badari Pulavarty > help in my case? If so, I would strongly vote for inclusion into 2.4 series!! It has been decided that the stack reduction patches were too intrusive to be merged at this stage of v2.4 life. We did not receive any report either - v2.4's 8KB kernel stack seems to be enough for vast majority of situations. [-- Attachment #2: stack_ovf_2430.log --] [-- Type: text/plain, Size: 21830 bytes --] May 11 04:22:09 server kernel: do_IRQ: stack overflow: 628 May 11 04:22:09 server kernel: f3668824 00000274 e9a40380 00000000 00000003 f7e5d280 f3668000 0000000e May 11 04:22:09 server kernel: c010d948 00000003 f7e5d318 f7e5d308 f7e5d280 f3668000 0000000e 00000006 May 11 04:22:09 server kernel: f1960018 f1a00018 ffffff14 c024893e 00000010 00000202 f7e5d280 f196cc00 May 11 04:22:09 server kernel: Call Trace: [call_do_IRQ+5/13] [pfifo_fast_dequeue+62/80] [qdisc_restart+31/432] [dev_queue_xmit+383/880] [neigh_resolve_output+321/624] May 11 04:22:09 server kernel: Call Trace: [<c010d948>] [<c024893e>] [<c02483df>] [<c023ccaf>] [<c0243af1>] May 11 04:22:09 server kernel: [ip_finish_output2+184/336] [ip_finish_output2+0/336] [ip_finish_output2+0/336] [nf_hook_slow+478/528] [ip_finish_output2+0/336] [ip_output+334/480] May 11 04:22:09 server kernel: [<c025a2d8>] [<c025a220>] [<c025a220>] [<c024763e>] [<c025a220>] [<c0258a4e>] May 11 04:22:09 server kernel: [ip_finish_output2+0/336] [ip_queue_xmit2+213/671] [ip_queue_xmit2+0/671] [ip_queue_xmit2+0/671] [nf_hook_slow+478/528] [ip_queue_xmit2+0/671] May 11 04:22:09 server kernel: [<c025a220>] [<c025a445>] [<c025a370>] [<c025a370>] [<c024763e>] [<c025a370>] May 11 04:22:09 server kernel: [ip_queue_xmit+845/1536] [ip_queue_xmit2+0/671] [dev_queue_xmit+383/880] [tcp_v4_send_check+160/240] [tcp_transmit_skb+1001/1792] [tcp_send_ack+132/208] May 11 04:22:09 server kernel: [<c0258e2d>] [<c025a370>] [<c023ccaf>] [<c0271c20>] [<c026bdc9>] [<c026e824>] May 11 04:22:09 server kernel: [tcp_rfree+0/32] [tcp_rfree+0/32] [tcp_rcv_established+2042/2640] [tcp_v4_do_rcv+314/352] [tcp_v4_rcv+1726/1952] [ip_local_deliver_finish+351/416] May 11 04:22:09 server kernel: [<c025d4a0>] [<c025d4a0>] [<c026a2fa>] [<c0272eda>] [<c02735be>] [<c025581f>] May 11 04:22:09 server kernel: [ip_local_deliver_finish+0/416] [nf_hook_slow+478/528] [ip_local_deliver_finish+0/416] [ip_local_deliver+399/576] [ip_local_deliver_finish+0/416] [ip_rcv_finish+553/700] May 11 04:22:09 server kernel: [<c02556c0>] [<c024763e>] [<c02556c0>] [<c025514f>] [<c02556c0>] [<c0255a89>] May 11 04:22:09 server kernel: [ip_rcv_finish+0/700] [ip_rcv_finish+0/700] [nf_hook_slow+478/528] [ip_rcv_finish+0/700] [ip_rcv+905/1216] [ip_rcv_finish+0/700] May 11 04:22:09 server kernel: [<c0255860>] [<c0255860>] [<c024763e>] [<c0255860>] [<c0255589>] [<c0255860>] May 11 04:22:09 server kernel: [netif_receive_skb+485/544] [process_backlog+147/304] [net_rx_action+250/368] [do_softirq+118/224] [do_IRQ+244/304] [call_do_IRQ+5/13] May 11 04:22:09 server kernel: [<c023d505>] [<c023d5d3>] [<c023d76a>] [<c01254c6>] [<c010b094>] [<c010d948>] May 11 04:22:09 server kernel: [__switch_to+82/256] [schedule+738/1344] [schedule_timeout+84/160] [process_timeout+0/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294702743/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294703097/96] May 11 04:22:09 server kernel: [<c0107722>] [<c011cee2>] [<c011cb14>] [<c011ca60>] [<f90f0697>] [<f90f07f9>] May 11 04:22:09 server kernel: [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294703275/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294709446/96] [<f914d588>] [<f914f255>] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288825300/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288879050/96] May 11 04:22:09 server kernel: [<f90f08ab>] [<f90f20c6>] [<f914d588>] [<f914f255>] [<f8b557d4>] [<f8b629ca>] May 11 04:22:09 server kernel: [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288713668/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288789620/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288713668/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288712232/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288755907/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288791678/96] May 11 04:22:09 server kernel: [<f8b3a3c4>] [<f8b4cc74>] [<f8b3a3c4>] [<f8b39e28>] [<f8b448c3>] [<f8b4d47e>] May 11 04:22:09 server kernel: [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288800963/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288763515/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288713480/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288763309/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288815612/96] [__kfree_skb+254/352] May 11 04:22:09 server kernel: [<f8b4f8c3>] [<f8b4667b>] [<f8b3a308>] [<f8b465ad>] [<f8b531fc>] [<c02387be>] May 11 04:22:09 server kernel: [__kfree_skb+254/352] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294624756/96] [qdisc_restart+31/432] [dev_queue_xmit+383/880] [neigh_resolve_output+321/624] [__kfree_skb+254/352] May 11 04:22:09 server kernel: [<c02387be>] [<f90dd5f4>] [<c02483df>] [<c023ccaf>] [<c0243af1>] [<c02387be>] May 11 04:22:09 server kernel: [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294624756/96] [qdisc_restart+31/432] [dev_queue_xmit+383/880] [neigh_resolve_output+321/624] [ip_finish_output2+184/336] [ip_finish_output2+0/336] May 11 04:22:09 server kernel: [<f90dd5f4>] [<c02483df>] [<c023ccaf>] [<c0243af1>] [<c025a2d8>] [<c025a220>] May 11 04:22:09 server kernel: [ip_finish_output2+0/336] [nf_hook_slow+478/528] [ip_finish_output2+0/336] [ip_output+334/480] [ip_finish_output2+0/336] [ip_queue_xmit2+213/671] May 11 04:22:09 server kernel: [<c025a220>] [<c024763e>] [<c025a220>] [<c0258a4e>] [<c025a220>] [<c025a445>] May 11 04:22:09 server kernel: [ip_queue_xmit2+0/671] [ip_queue_xmit2+0/671] [nf_hook_slow+478/528] [ip_queue_xmit2+0/671] [ip_queue_xmit+845/1536] [ip_queue_xmit2+0/671] May 11 04:22:09 server kernel: [<c025a370>] [<c025a370>] [<c024763e>] [<c025a370>] [<c0258e2d>] [<c025a370>] May 11 04:22:09 server kernel: [skb_checksum+92/800] [sock_def_readable+99/128] [tcp_rfree+0/32] [tcp_rfree+0/32] [tcp_rcv_established+1981/2640] [tcp_v4_do_rcv+314/352] May 11 04:22:09 server kernel: [<c02399dc>] [<c0237f63>] [<c025d4a0>] [<c025d4a0>] [<c026a2bd>] [<c0272eda>] May 11 04:22:09 server kernel: [tcp_v4_rcv+1726/1952] [locate_hd_struct+56/160] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288606771/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288606771/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288607559/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288679225/96] May 11 04:22:09 server kernel: [<c02735be>] [<c01e4308>] [<f8b20233>] [<f8b20233>] [<f8b20547>] [<f8b31d39>] May 11 04:22:09 server kernel: [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288574821/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288906935/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288935563/96] [bh_action+106/112] [tasklet_hi_action+83/160] [sys_ioctl+677/785] May 11 04:22:09 server kernel: [<f8b18565>] [<f8b696b7>] [<f8b7068b>] [<c01258ca>] [<c0125753>] [<c0158905>] May 11 04:22:09 server kernel: [system_call+51/56] May 11 04:22:09 server kernel: [<c0108fa7>] ####################################################### May 11 04:26:28 server kernel: do_IRQ: stack overflow: 1000 May 11 04:26:28 server kernel: f3668998 000003e8 00000000 00000000 f3668000 f3668ac0 f3668a34 00000000 May 11 04:26:28 server kernel: c010d948 f3668000 0001961a 00000944 f3668ac0 f3668a34 00000000 00000944 May 11 04:26:28 server kernel: 00000018 f1960018 ffffff14 f90820a7 00000010 00000246 f3668a34 f40a9ec0 May 11 04:26:28 server kernel: Call Trace: [call_do_IRQ+5/13] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294250663/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294268736/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294250842/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294254302/96] May 11 04:26:28 server kernel: Call Trace: [<c010d948>] [<f90820a7>] [<f9086740>] [<f908215a>] [<f9082ede>] May 11 04:26:28 server kernel: [ip_queue_xmit2+0/671] [nf_iterate+85/160] [ip_queue_xmit2+0/671] [ip_queue_xmit2+0/671] [nf_hook_slow+247/528] [ip_queue_xmit2+0/671] May 11 04:26:28 server kernel: [<c025a370>] [<c0247185>] [<c025a370>] [<c025a370>] [<c0247557>] [<c025a370>] May 11 04:26:28 server kernel: [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294266808/96] [ip_queue_xmit+845/1536] [ip_queue_xmit2+0/671] [tcp_v4_rcv+1726/1952] [tcp_v4_send_check+160/240] [tcp_transmit_skb+1001/1792] May 11 04:26:28 server kernel: [<f9085fb8>] [<c0258e2d>] [<c025a370>] [<c02735be>] [<c0271c20>] [<c026bdc9>] May 11 04:26:28 server kernel: [tcp_send_ack+132/208] [tcp_rfree+0/32] [tcp_rfree+0/32] [tcp_rcv_established+2042/2640] [tcp_v4_do_rcv+314/352] [tcp_v4_rcv+1726/1952] May 11 04:26:28 server kernel: [<c026e824>] [<c025d4a0>] [<c025d4a0>] [<c026a2fa>] [<c0272eda>] [<c02735be>] May 11 04:26:28 server kernel: [ip_local_deliver_finish+351/416] [ip_local_deliver_finish+0/416] [nf_hook_slow+478/528] [ip_local_deliver_finish+0/416] [ip_local_deliver+399/576] [ip_local_deliver_finish+0/416] May 11 04:26:28 server kernel: [<c025581f>] [<c02556c0>] [<c024763e>] [<c02556c0>] [<c025514f>] [<c02556c0>] May 11 04:26:28 server kernel: [ip_rcv_finish+553/700] [ip_rcv_finish+0/700] [ip_rcv_finish+0/700] [nf_hook_slow+478/528] [ip_rcv_finish+0/700] [ip_rcv+905/1216] May 11 04:26:28 server kernel: [<c0255a89>] [<c0255860>] [<c0255860>] [<c024763e>] [<c0255860>] [<c0255589>] May 11 04:26:28 server kernel: [ip_rcv_finish+0/700] [netif_receive_skb+485/544] [process_backlog+147/304] [net_rx_action+250/368] [do_softirq+118/224] [do_IRQ+244/304] May 11 04:26:28 server kernel: [<c0255860>] [<c023d505>] [<c023d5d3>] [<c023d76a>] [<c01254c6>] [<c010b094>] May 11 04:26:28 server kernel: [call_do_IRQ+5/13] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294701080/96] [<f914e620>] [<f914e7b6>] [<f914d473>] [<f914d5a6>] May 11 04:26:28 server kernel: [<c010d948>] [<f90f0018>] [<f914e620>] [<f914e7b6>] [<f914d473>] [<f914d5a6>] May 11 04:26:28 server kernel: [isofs_read_super+1131/2048] [<f914f255>] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288825300/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288879050/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288713668/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288789620/96] May 11 04:26:28 server kernel: [<c018480b>] [<f914f255>] [<f8b557d4>] [<f8b629ca>] [<f8b3a3c4>] [<f8b4cc74>] May 11 04:26:28 server kernel: [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288713668/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288712232/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288755907/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288791678/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288800963/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288763515/96] May 11 04:26:28 server kernel: [<f8b3a3c4>] [<f8b39e28>] [<f8b448c3>] [<f8b4d47e>] [<f8b4f8c3>] [<f8b4667b>] May 11 04:26:28 server kernel: [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288713480/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288763309/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288815612/96] [__kfree_skb+254/352] [__kfree_skb+254/352] [__kfree_skb+254/352] May 11 04:26:28 server kernel: [<f8b3a308>] [<f8b465ad>] [<f8b531fc>] [<c02387be>] [<c02387be>] [<c02387be>] May 11 04:26:28 server kernel: [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294624756/96] [qdisc_restart+31/432] [dev_queue_xmit+383/880] [neigh_resolve_output+321/624] [ip_finish_output2+184/336] [ip_finish_output2+0/336] May 11 04:26:28 server kernel: [<f90dd5f4>] [<c02483df>] [<c023ccaf>] [<c0243af1>] [<c025a2d8>] [<c025a220>] May 11 04:26:28 server kernel: [ip_finish_output2+0/336] [nf_hook_slow+478/528] [ip_finish_output2+0/336] [ip_output+334/480] [ip_finish_output2+0/336] [ip_queue_xmit2+213/671] May 11 04:26:28 server kernel: [<c025a220>] [<c024763e>] [<c025a220>] [<c0258a4e>] [<c025a220>] [<c025a445>] May 11 04:26:28 server kernel: [ip_queue_xmit2+0/671] [ip_queue_xmit2+0/671] [nf_hook_slow+478/528] [ip_queue_xmit2+0/671] [ip_queue_xmit+845/1536] [ip_queue_xmit2+0/671] May 11 04:26:28 server kernel: [<c025a370>] [<c025a370>] [<c024763e>] [<c025a370>] [<c0258e2d>] [<c025a370>] May 11 04:26:28 server kernel: [nf_hook_slow+478/528] [tcp_v4_send_check+160/240] [tcp_transmit_skb+1001/1792] [sock_def_readable+99/128] [tcp_rfree+0/32] [tcp_rfree+0/32] May 11 04:26:28 server kernel: [<c024763e>] [<c0271c20>] [<c026bdc9>] [<c0237f63>] [<c025d4a0>] [<c025d4a0>] May 11 04:26:28 server kernel: [tcp_rcv_established+1981/2640] [tcp_v4_do_rcv+314/352] [tcp_v4_rcv+1726/1952] [ip_local_deliver_finish+351/416] [ip_local_deliver_finish+0/416] [nf_hook_slow+478/528] May 11 04:26:28 server kernel: [<c026a2bd>] [<c0272eda>] [<c02735be>] [<c025581f>] [<c02556c0>] [<c024763e>] May 11 04:26:28 server kernel: [ip_local_deliver_finish+0/416] [ip_local_deliver+399/576] [ip_local_deliver_finish+0/416] [ip_rcv_finish+553/700] [ip_rcv_finish+0/700] [ip_rcv_finish+0/700] May 11 04:26:28 server kernel: [<c02556c0>] [<c025514f>] [<c02556c0>] [<c0255a89>] [<c0255860>] [<c0255860>] May 11 04:26:28 server kernel: [nf_hook_slow+478/528] [schedule+738/1344] [schedule_timeout+94/160] [poll_freewait+68/80] [do_select+569/592] [sys_select+660/1248] May 11 04:26:28 server kernel: [<c024763e>] [<c011cee2>] [<c011cb1e>] [<c01594d4>] [<c01598d9>] [<c0159bb4>] May 11 04:26:28 server kernel: [sys_ioctl+677/785] [sys_time+21/80] [system_call+51/56] May 11 04:26:28 server kernel: [<c0158905>] [<c0124bb5>] [<c0108fa7>] ############################################################# May 11 04:30:06 server kernel: do_IRQ: stack overflow: 640 May 11 04:30:06 server kernel: f3668830 00000280 f2c73080 00000000 f196cd80 d2029180 d2029180 00000042 May 11 04:30:06 server kernel: c010d948 f196cd80 f008a0ec d2029180 d2029180 d2029180 00000042 f008a080 May 11 04:30:06 server kernel: f1960018 f5fd0018 ffffff14 c0238723 00000010 00000202 d2029180 d2029180 May 11 04:30:06 server kernel: Call Trace: [call_do_IRQ+5/13] [__kfree_skb+99/352] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294624756/96] [qdisc_restart+114/432] [dev_queue_xmit+383/880] May 11 04:30:06 server kernel: Call Trace: [<c010d948>] [<c0238723>] [<f90dd5f4>] [<c0248432>] [<c023ccaf>] May 11 04:30:06 server kernel: [ip_finish_output2+184/336] [ip_finish_output2+0/336] [ip_finish_output2+0/336] [nf_hook_slow+478/528] [ip_finish_output2+0/336] [ip_output+334/480] May 11 04:30:06 server kernel: [<c025a2d8>] [<c025a220>] [<c025a220>] [<c024763e>] [<c025a220>] [<c0258a4e>] May 11 04:30:06 server kernel: [ip_finish_output2+0/336] [ip_queue_xmit2+213/671] [ip_queue_xmit2+0/671] [ip_queue_xmit2+0/671] [nf_hook_slow+478/528] [ip_queue_xmit2+0/671] May 11 04:30:06 server kernel: [<c025a220>] [<c025a445>] [<c025a370>] [<c025a370>] [<c024763e>] [<c025a370>] May 11 04:30:06 server kernel: [ip_queue_xmit+845/1536] [ip_queue_xmit2+0/671] [tcp_v4_send_check+160/240] [tcp_transmit_skb+1001/1792] [tcp_send_ack+132/208] [tcp_rfree+0/32] May 11 04:30:06 server kernel: [<c0258e2d>] [<c025a370>] [<c0271c20>] [<c026bdc9>] [<c026e824>] [<c025d4a0>] May 11 04:30:06 server kernel: [tcp_rfree+0/32] [tcp_rcv_established+2042/2640] [tcp_v4_do_rcv+314/352] [tcp_v4_rcv+1726/1952] [ip_local_deliver_finish+351/416] [ip_local_deliver_finish+0/416] May 11 04:30:06 server kernel: [<c025d4a0>] [<c026a2fa>] [<c0272eda>] [<c02735be>] [<c025581f>] [<c02556c0>] May 11 04:30:06 server kernel: [nf_hook_slow+478/528] [ip_local_deliver_finish+0/416] [ip_local_deliver+399/576] [ip_local_deliver_finish+0/416] [ip_rcv_finish+553/700] [ip_rcv_finish+0/700] May 11 04:30:06 server kernel: [<c024763e>] [<c02556c0>] [<c025514f>] [<c02556c0>] [<c0255a89>] [<c0255860>] May 11 04:30:06 server kernel: [ip_rcv_finish+0/700] [nf_hook_slow+478/528] [ip_rcv_finish+0/700] [ip_rcv+905/1216] [ip_rcv_finish+0/700] [netif_receive_skb+485/544] May 11 04:30:06 server kernel: [<c0255860>] [<c024763e>] [<c0255860>] [<c0255589>] [<c0255860>] [<c023d505>] May 11 04:30:06 server kernel: [process_backlog+147/304] [net_rx_action+250/368] [do_softirq+118/224] [do_IRQ+244/304] [call_do_IRQ+5/13] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294806243/96] May 11 04:30:06 server kernel: [<c023d5d3>] [<c023d76a>] [<c01254c6>] [<c010b094>] [<c010d948>] [<f9109ae3>] May 11 04:30:06 server kernel: [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294808580/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294815256/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294702477/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294702701/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294703097/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294703275/96] May 11 04:30:06 server kernel: [<f910a404>] [<f910be18>] [<f90f058d>] [<f90f066d>] [<f90f07f9>] [<f90f08ab>] May 11 04:30:06 server kernel: [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294709446/96] [<f914d588>] [isofs_read_super+1131/2048] [<f914f255>] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288825300/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288879050/96] May 11 04:30:06 server kernel: [<f90f20c6>] [<f914d588>] [<c018480b>] [<f914f255>] [<f8b557d4>] [<f8b629ca>] May 11 04:30:06 server kernel: [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288713668/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288789620/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288713668/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288712232/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288755907/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288791678/96] May 11 04:30:06 server kernel: [<f8b3a3c4>] [<f8b4cc74>] [<f8b3a3c4>] [<f8b39e28>] [<f8b448c3>] [<f8b4d47e>] May 11 04:30:06 server kernel: [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288800963/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288763515/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288713480/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288763309/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4288815612/96] [__kfree_skb+254/352] May 11 04:30:06 server kernel: [<f8b4f8c3>] [<f8b4667b>] [<f8b3a308>] [<f8b465ad>] [<f8b531fc>] [<c02387be>] May 11 04:30:06 server kernel: [netif_rx+147/496] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294626081/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294624756/96] [__kfree_skb+254/352] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294624756/96] [__kfree_skb+254/352] May 11 04:30:06 server kernel: [<c023cfb3>] [<f90ddb21>] [<f90dd5f4>] [<c02387be>] [<f90dd5f4>] [<c02387be>] May 11 04:30:06 server kernel: [__kfree_skb+254/352] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294624756/96] [qdisc_restart+31/432] [dev_queue_xmit+383/880] [ip_finish_output2+184/336] [ip_finish_output2+0/336] May 11 04:30:06 server kernel: [<c02387be>] [<f90dd5f4>] [<c02483df>] [<c023ccaf>] [<c025a2d8>] [<c025a220>] May 11 04:30:06 server kernel: [ip_finish_output2+0/336] [nf_hook_slow+478/528] [ip_finish_output2+0/336] [ip_output+334/480] [ip_finish_output2+0/336] [ip_queue_xmit2+213/671] May 11 04:30:06 server kernel: [<c025a220>] [<c024763e>] [<c025a220>] [<c0258a4e>] [<c025a220>] [<c025a445>] May 11 04:30:06 server kernel: [ip_queue_xmit2+0/671] [ip_queue_xmit2+0/671] [nf_hook_slow+478/528] [ip_queue_xmit2+0/671] [ip_queue_xmit+845/1536] [ip_queue_xmit2+0/671] May 11 04:30:06 server kernel: [<c025a370>] [<c025a370>] [<c024763e>] [<c025a370>] [<c0258e2d>] [<c025a370>] May 11 04:30:06 server kernel: [sock_def_readable+99/128] [tcp_rfree+0/32] [tcp_rfree+0/32] [tcp_rcv_established+1981/2640] [tcp_v4_do_rcv+314/352] [tcp_v4_rcv+1726/1952] May 11 04:30:06 server kernel: [<c0237f63>] [<c025d4a0>] [<c025d4a0>] [<c026a2bd>] [<c0272eda>] [<c02735be>] May 11 04:30:06 server kernel: [ip_local_deliver_finish+351/416] [ip_local_deliver_finish+0/416] [nf_hook_slow+478/528] [ip_local_deliver_finish+0/416] [ip_local_deliver+399/576] [ip_local_deliver_finish+0/416] May 11 04:30:06 server kernel: [<c025581f>] [<c02556c0>] [<c024763e>] [<c02556c0>] [<c025514f>] [<c02556c0>] May 11 04:30:06 server kernel: [ip_rcv_finish+553/700] [__alloc_pages+100/640] [poll_freewait+68/80] [do_select+569/592] [sys_select+660/1248] [sys_ioctl+677/785] May 11 04:30:06 server kernel: [<c0255a89>] [<c013e624>] [<c01594d4>] [<c01598d9>] [<c0159bb4>] [<c0158905>] May 11 04:30:06 server kernel: [sys_time+21/80] [system_call+51/56] May 11 04:30:06 server kernel: [<c0124bb5>] [<c0108fa7>] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 2.4.30-hf1 do_IRQ stack overflows 2005-05-11 12:46 ` 2.4.30-hf1 do_IRQ stack overflows Marcelo Tosatti @ 2005-05-11 21:38 ` Herbert Xu 2005-05-12 8:44 ` Manfred Schwarb 2005-05-12 7:59 ` Manfred Schwarb 2005-06-07 12:38 ` Manfred Schwarb 2 siblings, 1 reply; 9+ messages in thread From: Herbert Xu @ 2005-05-11 21:38 UTC (permalink / raw) To: Marcelo Tosatti; +Cc: manfred99, linux-kernel, davem, netdev Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote: > > May 11 04:22:09 server kernel: [__switch_to+82/256] [schedule+738/1344] [schedule_timeout+84/160] [process_timeout+0/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294702743/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294703097/96] The stack trace becomes unreadable at this point. Please run klogd with -X and then decode the messages with ksymoops. Alternatively run ksymoops on the output of dmesg directly. Thanks, -- Visit Openswan at http://www.openswan.org/ Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 2.4.30-hf1 do_IRQ stack overflows 2005-05-11 21:38 ` Herbert Xu @ 2005-05-12 8:44 ` Manfred Schwarb 2005-05-14 11:05 ` Herbert Xu 0 siblings, 1 reply; 9+ messages in thread From: Manfred Schwarb @ 2005-05-12 8:44 UTC (permalink / raw) To: Herbert Xu; +Cc: marcelo.tosatti, linux-kernel, davem, netdev > Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote: > > > > May 11 04:22:09 server kernel: [__switch_to+82/256] > [schedule+738/1344] [schedule_timeout+84/160] [process_timeout+0/96] > [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294702743/96] > [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294703097/96] > > The stack trace becomes unreadable at this point. Please run klogd > with -X and then decode the messages with ksymoops. Alternatively > run ksymoops on the output of dmesg directly. > Here are the 3 ksymoopsified traces (ksymoops -i): May 11 04:22:09 tp-meteodat7 kernel: f3668824 00000274 e9a40380 00000000 00000003 f7e5d280 f3668000 0000000e May 11 04:22:09 tp-meteodat7 kernel: c010d948 00000003 f7e5d318 f7e5d308 f7e5d280 f3668000 0000000e 00000006 May 11 04:22:09 tp-meteodat7 kernel: f1960018 f1a00018 ffffff14 c024893e 00000010 00000202 f7e5d280 f196cc00 May 11 04:22:09 tp-meteodat7 kernel: Call Trace: [call_do_IRQ+5/13] [pfifo_fast_dequeue+62/80] [qdisc_restart+31/432] [dev_queue_xmit+383/880] [neigh_resolve_output+321/624] May 11 04:22:09 tp-meteodat7 kernel: Call Trace: [<c010d948>] [<c024893e>] [<c02483df>] [<c023ccaf>] [<c0243af1>] May 11 04:22:09 tp-meteodat7 kernel: [<c025a2d8>] [<c025a220>] [<c025a220>] [<c024763e>] [<c025a220>] [<c0258a4e>] May 11 04:22:09 tp-meteodat7 kernel: [<c025a220>] [<c025a445>] [<c025a370>] [<c025a370>] [<c024763e>] [<c025a370>] May 11 04:22:09 tp-meteodat7 kernel: [<c0258e2d>] [<c025a370>] [<c023ccaf>] [<c0271c20>] [<c026bdc9>] [<c026e824>] May 11 04:22:09 tp-meteodat7 kernel: [<c025d4a0>] [<c025d4a0>] [<c026a2fa>] [<c0272eda>] [<c02735be>] [<c025581f>] May 11 04:22:09 tp-meteodat7 kernel: [<c02556c0>] [<c024763e>] [<c02556c0>] [<c025514f>] [<c02556c0>] [<c0255a89>] May 11 04:22:09 tp-meteodat7 kernel: [<c0255860>] [<c0255860>] [<c024763e>] [<c0255860>] [<c0255589>] [<c0255860>] May 11 04:22:09 tp-meteodat7 kernel: [<c023d505>] [<c023d5d3>] [<c023d76a>] [<c01254c6>] [<c010b094>] [<c010d948>] May 11 04:22:09 tp-meteodat7 kernel: [<c0107722>] [<c011cee2>] [<c011cb14>] [<c011ca60>] [<f90f0697>] [<f90f07f9>] May 11 04:22:09 tp-meteodat7 kernel: [<f90f08ab>] [<f90f20c6>] [<f914d588>] [<f914f255>] [<f8b557d4>] [<f8b629ca>] May 11 04:22:09 tp-meteodat7 kernel: [<f8b3a3c4>] [<f8b4cc74>] [<f8b3a3c4>] [<f8b39e28>] [<f8b448c3>] [<f8b4d47e>] May 11 04:22:09 tp-meteodat7 kernel: [<f8b4f8c3>] [<f8b4667b>] [<f8b3a308>] [<f8b465ad>] [<f8b531fc>] [<c02387be>] May 11 04:22:09 tp-meteodat7 kernel: [<c02387be>] [<f90dd5f4>] [<c02483df>] [<c023ccaf>] [<c0243af1>] [<c02387be>] May 11 04:22:09 tp-meteodat7 kernel: [<f90dd5f4>] [<c02483df>] [<c023ccaf>] [<c0243af1>] [<c025a2d8>] [<c025a220>] May 11 04:22:09 tp-meteodat7 kernel: [<c025a220>] [<c024763e>] [<c025a220>] [<c0258a4e>] [<c025a220>] [<c025a445>] May 11 04:22:09 tp-meteodat7 kernel: [<c025a370>] [<c025a370>] [<c024763e>] [<c025a370>] [<c0258e2d>] [<c025a370>] May 11 04:22:09 tp-meteodat7 kernel: [<c02399dc>] [<c0237f63>] [<c025d4a0>] [<c025d4a0>] [<c026a2bd>] [<c0272eda>] May 11 04:22:09 tp-meteodat7 kernel: [<c02735be>] [<c01e4308>] [<f8b20233>] [<f8b20233>] [<f8b20547>] [<f8b31d39>] May 11 04:22:09 tp-meteodat7 kernel: [<f8b18565>] [<f8b696b7>] [<f8b7068b>] [<c01258ca>] [<c0125753>] [<c0158905>] May 11 04:22:09 tp-meteodat7 kernel: [<c0108fa7>] Warning (Oops_read): Code line not seen, dumping what data is available Trace; c010d948 <call_do_IRQ+5/d> Trace; c024893e <pfifo_fast_dequeue+3e/50> Trace; c02483df <qdisc_restart+1f/1b0> Trace; c023ccaf <dev_queue_xmit+17f/370> Trace; c0243af1 <neigh_resolve_output+141/270> Trace; c025a2d8 <ip_finish_output2+b8/150> Trace; c025a220 <ip_finish_output2+0/150> Trace; c025a220 <ip_finish_output2+0/150> Trace; c024763e <nf_hook_slow+1de/210> Trace; c025a220 <ip_finish_output2+0/150> Trace; c0258a4e <ip_output+14e/1e0> Trace; c025a220 <ip_finish_output2+0/150> Trace; c025a445 <ip_queue_xmit2+d5/29f> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c024763e <nf_hook_slow+1de/210> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c0258e2d <ip_queue_xmit+34d/600> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c023ccaf <dev_queue_xmit+17f/370> Trace; c0271c20 <tcp_v4_send_check+a0/f0> Trace; c026bdc9 <tcp_transmit_skb+3e9/700> Trace; c026e824 <tcp_send_ack+84/d0> Trace; c025d4a0 <tcp_rfree+0/20> Trace; c025d4a0 <tcp_rfree+0/20> Trace; c026a2fa <tcp_rcv_established+7fa/a50> Trace; c0272eda <tcp_v4_do_rcv+13a/160> Trace; c02735be <tcp_v4_rcv+6be/7a0> Trace; c025581f <ip_local_deliver_finish+15f/1a0> Trace; c02556c0 <ip_local_deliver_finish+0/1a0> Trace; c024763e <nf_hook_slow+1de/210> Trace; c02556c0 <ip_local_deliver_finish+0/1a0> Trace; c025514f <ip_local_deliver+18f/240> Trace; c02556c0 <ip_local_deliver_finish+0/1a0> Trace; c0255a89 <ip_rcv_finish+229/2bc> Trace; c0255860 <ip_rcv_finish+0/2bc> Trace; c0255860 <ip_rcv_finish+0/2bc> Trace; c024763e <nf_hook_slow+1de/210> Trace; c0255860 <ip_rcv_finish+0/2bc> Trace; c0255589 <ip_rcv+389/4c0> Trace; c0255860 <ip_rcv_finish+0/2bc> Trace; c023d505 <netif_receive_skb+1e5/220> Trace; c023d5d3 <process_backlog+93/130> Trace; c023d76a <net_rx_action+fa/170> Trace; c01254c6 <do_softirq+76/e0> Trace; c010b094 <do_IRQ+f4/130> Trace; c010d948 <call_do_IRQ+5/d> Trace; c0107722 <__switch_to+52/100> Trace; c011cee2 <schedule+2e2/540> Trace; c011cb14 <schedule_timeout+54/a0> Trace; c011ca60 <process_timeout+0/60> Trace; f90f0697 <[usbcore]usb_start_wait_urb+a7/1a0> Trace; f90f07f9 <[usbcore]usb_internal_control_msg+69/80> Trace; f90f08ab <[usbcore]usb_control_msg+9b/b0> Trace; f90f20c6 <[usbcore]usb_get_report+76/80> Trace; f914d588 <[hid]hid_read_report+98/c0> Trace; f914f255 <[hid]hiddev_ioctl+565/cd0> Trace; f8b557d4 <[reiserfs]get_cnode+14/80> Trace; f8b629ca <[reiserfs].LC32+57e7/5a7d> Trace; f8b3a3c4 <[reiserfs]do_balance_mark_leaf_dirty+64/a0> Trace; f8b4cc74 <[reiserfs]leaf_insert_into_buf+1c4/2c0> Trace; f8b3a3c4 <[reiserfs]do_balance_mark_leaf_dirty+64/a0> Trace; f8b39e28 <[reiserfs]balance_leaf+30a8/3120> Trace; f8b448c3 <[reiserfs]ip_check_balance+2e3/bf0> Trace; f8b4d47e <[reiserfs]leaf_cut_from_buffer+28e/4b0> Trace; f8b4f8c3 <[reiserfs]pathrelse_and_restore+43/60> Trace; f8b4667b <[reiserfs]unfix_nodes+9b/180> Trace; f8b3a308 <[reiserfs]do_balance+d8/130> Trace; f8b465ad <[reiserfs]fix_nodes+3fd/430> Trace; f8b531fc <[reiserfs]reiserfs_insert_item+14c/150> Trace; c02387be <__kfree_skb+fe/160> Trace; c02387be <__kfree_skb+fe/160> Trace; f90dd5f4 <[8139too]rtl8139_start_xmit+84/180> Trace; c02483df <qdisc_restart+1f/1b0> Trace; c023ccaf <dev_queue_xmit+17f/370> Trace; c0243af1 <neigh_resolve_output+141/270> Trace; c02387be <__kfree_skb+fe/160> Trace; f90dd5f4 <[8139too]rtl8139_start_xmit+84/180> Trace; c02483df <qdisc_restart+1f/1b0> Trace; c023ccaf <dev_queue_xmit+17f/370> Trace; c0243af1 <neigh_resolve_output+141/270> Trace; c025a2d8 <ip_finish_output2+b8/150> Trace; c025a220 <ip_finish_output2+0/150> Trace; c025a220 <ip_finish_output2+0/150> Trace; c024763e <nf_hook_slow+1de/210> Trace; c025a220 <ip_finish_output2+0/150> Trace; c0258a4e <ip_output+14e/1e0> Trace; c025a220 <ip_finish_output2+0/150> Trace; c025a445 <ip_queue_xmit2+d5/29f> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c024763e <nf_hook_slow+1de/210> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c0258e2d <ip_queue_xmit+34d/600> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c02399dc <skb_checksum+5c/320> Trace; c0237f63 <sock_def_readable+63/80> Trace; c025d4a0 <tcp_rfree+0/20> Trace; c025d4a0 <tcp_rfree+0/20> Trace; c026a2bd <tcp_rcv_established+7bd/a50> Trace; c0272eda <tcp_v4_do_rcv+13a/160> Trace; c02735be <tcp_v4_rcv+6be/7a0> Trace; c01e4308 <locate_hd_struct+38/a0> Trace; f8b20233 <[scsi_mod]__scsi_end_request+233/240> Trace; f8b20233 <[scsi_mod]__scsi_end_request+233/240> Trace; f8b20547 <[scsi_mod]scsi_io_completion+1d7/4c0> Trace; f8b31d39 <[sd_mod]rw_intr+89/2c0> Trace; f8b18565 <[scsi_mod]scsi_finish_command+c5/110> Trace; f8b696b7 <[libata]ata_qc_complete+37/b0> Trace; f8b7068b <[sata_promise]pdc_interrupt+16b/1b0> Trace; c01258ca <bh_action+6a/70> Trace; c0125753 <tasklet_hi_action+53/a0> Trace; c0158905 <sys_ioctl+2a5/311> Trace; c0108fa7 <system_call+33/38> ############################################################### May 11 04:26:28 tp-meteodat7 kernel: f3668998 000003e8 00000000 00000000 f3668000 f3668ac0 f3668a34 00000000 May 11 04:26:28 tp-meteodat7 kernel: c010d948 f3668000 0001961a 00000944 f3668ac0 f3668a34 00000000 00000944 May 11 04:26:28 tp-meteodat7 kernel: 00000018 f1960018 ffffff14 f90820a7 00000010 00000246 f3668a34 f40a9ec0 May 11 04:26:28 tp-meteodat7 kernel: Call Trace: [call_do_IRQ+5/13] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294250663/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294268736/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294250842/96] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294254302/96] May 11 04:26:28 tp-meteodat7 kernel: Call Trace: [<c010d948>] [<f90820a7>] [<f9086740>] [<f908215a>] [<f9082ede>] May 11 04:26:28 tp-meteodat7 kernel: [<c025a370>] [<c0247185>] [<c025a370>] [<c025a370>] [<c0247557>] [<c025a370>] May 11 04:26:28 tp-meteodat7 kernel: [<f9085fb8>] [<c0258e2d>] [<c025a370>] [<c02735be>] [<c0271c20>] [<c026bdc9>] May 11 04:26:28 tp-meteodat7 kernel: [<c026e824>] [<c025d4a0>] [<c025d4a0>] [<c026a2fa>] [<c0272eda>] [<c02735be>] May 11 04:26:28 tp-meteodat7 kernel: [<c025581f>] [<c02556c0>] [<c024763e>] [<c02556c0>] [<c025514f>] [<c02556c0>] May 11 04:26:28 tp-meteodat7 kernel: [<c0255a89>] [<c0255860>] [<c0255860>] [<c024763e>] [<c0255860>] [<c0255589>] May 11 04:26:28 tp-meteodat7 kernel: [<c0255860>] [<c023d505>] [<c023d5d3>] [<c023d76a>] [<c01254c6>] [<c010b094>] May 11 04:26:28 tp-meteodat7 kernel: [<c010d948>] [<f90f0018>] [<f914e620>] [<f914e7b6>] [<f914d473>] [<f914d5a6>] May 11 04:26:28 tp-meteodat7 kernel: [<c018480b>] [<f914f255>] [<f8b557d4>] [<f8b629ca>] [<f8b3a3c4>] [<f8b4cc74>] May 11 04:26:28 tp-meteodat7 kernel: [<f8b3a3c4>] [<f8b39e28>] [<f8b448c3>] [<f8b4d47e>] [<f8b4f8c3>] [<f8b4667b>] May 11 04:26:28 tp-meteodat7 kernel: [<f8b3a308>] [<f8b465ad>] [<f8b531fc>] [<c02387be>] [<c02387be>] [<c02387be>] May 11 04:26:28 tp-meteodat7 kernel: [<f90dd5f4>] [<c02483df>] [<c023ccaf>] [<c0243af1>] [<c025a2d8>] [<c025a220>] May 11 04:26:28 tp-meteodat7 kernel: [<c025a220>] [<c024763e>] [<c025a220>] [<c0258a4e>] [<c025a220>] [<c025a445>] May 11 04:26:28 tp-meteodat7 kernel: [<c025a370>] [<c025a370>] [<c024763e>] [<c025a370>] [<c0258e2d>] [<c025a370>] May 11 04:26:28 tp-meteodat7 kernel: [<c024763e>] [<c0271c20>] [<c026bdc9>] [<c0237f63>] [<c025d4a0>] [<c025d4a0>] May 11 04:26:28 tp-meteodat7 kernel: [<c026a2bd>] [<c0272eda>] [<c02735be>] [<c025581f>] [<c02556c0>] [<c024763e>] May 11 04:26:28 tp-meteodat7 kernel: [<c02556c0>] [<c025514f>] [<c02556c0>] [<c0255a89>] [<c0255860>] [<c0255860>] May 11 04:26:28 tp-meteodat7 kernel: [<c024763e>] [<c011cee2>] [<c011cb1e>] [<c01594d4>] [<c01598d9>] [<c0159bb4>] May 11 04:26:28 tp-meteodat7 kernel: [<c0158905>] [<c0124bb5>] [<c0108fa7>] Warning (Oops_read): Code line not seen, dumping what data is available Trace; c010d948 <call_do_IRQ+5/d> Trace; f90820a7 <[ip_conntrack]__ip_conntrack_find+17/90> Trace; f9086740 <[ip_conntrack]ip_conntrack_protocol_tcp+0/40> Trace; f908215a <[ip_conntrack]ip_conntrack_find_get+3a/80> Trace; f9082ede <[ip_conntrack]ip_conntrack_in+1ae/2e0> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c0247185 <nf_iterate+55/a0> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c0247557 <nf_hook_slow+f7/210> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; f9085fb8 <[ip_conntrack]ip_conntrack_local_out_ops+0/18> Trace; c0258e2d <ip_queue_xmit+34d/600> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c02735be <tcp_v4_rcv+6be/7a0> Trace; c0271c20 <tcp_v4_send_check+a0/f0> Trace; c026bdc9 <tcp_transmit_skb+3e9/700> Trace; c026e824 <tcp_send_ack+84/d0> Trace; c025d4a0 <tcp_rfree+0/20> Trace; c025d4a0 <tcp_rfree+0/20> Trace; c026a2fa <tcp_rcv_established+7fa/a50> Trace; c0272eda <tcp_v4_do_rcv+13a/160> Trace; c02735be <tcp_v4_rcv+6be/7a0> Trace; c025581f <ip_local_deliver_finish+15f/1a0> Trace; c02556c0 <ip_local_deliver_finish+0/1a0> Trace; c024763e <nf_hook_slow+1de/210> Trace; c02556c0 <ip_local_deliver_finish+0/1a0> Trace; c025514f <ip_local_deliver+18f/240> Trace; c02556c0 <ip_local_deliver_finish+0/1a0> Trace; c0255a89 <ip_rcv_finish+229/2bc> Trace; c0255860 <ip_rcv_finish+0/2bc> Trace; c0255860 <ip_rcv_finish+0/2bc> Trace; c024763e <nf_hook_slow+1de/210> Trace; c0255860 <ip_rcv_finish+0/2bc> Trace; c0255589 <ip_rcv+389/4c0> Trace; c0255860 <ip_rcv_finish+0/2bc> Trace; c023d505 <netif_receive_skb+1e5/220> Trace; c023d5d3 <process_backlog+93/130> Trace; c023d76a <net_rx_action+fa/170> Trace; c01254c6 <do_softirq+76/e0> Trace; c010b094 <do_IRQ+f4/130> Trace; c010d948 <call_do_IRQ+5/d> Trace; f90f0018 <[usbcore]call_policy_interface+8/260> Trace; f914e620 <[hid]hiddev_send_event+0/b0> Trace; f914e7b6 <[hid]hiddev_report_event+66/80> Trace; f914d473 <[hid]hid_input_report+133/170> Trace; f914d5a6 <[hid]hid_read_report+b6/c0> Trace; c018480b <isofs_read_super+46b/800> Trace; f914f255 <[hid]hiddev_ioctl+565/cd0> Trace; f8b557d4 <[reiserfs]get_cnode+14/80> Trace; f8b629ca <[reiserfs].LC32+57e7/5a7d> Trace; f8b3a3c4 <[reiserfs]do_balance_mark_leaf_dirty+64/a0> Trace; f8b4cc74 <[reiserfs]leaf_insert_into_buf+1c4/2c0> Trace; f8b3a3c4 <[reiserfs]do_balance_mark_leaf_dirty+64/a0> Trace; f8b39e28 <[reiserfs]balance_leaf+30a8/3120> Trace; f8b448c3 <[reiserfs]ip_check_balance+2e3/bf0> Trace; f8b4d47e <[reiserfs]leaf_cut_from_buffer+28e/4b0> Trace; f8b4f8c3 <[reiserfs]pathrelse_and_restore+43/60> Trace; f8b4667b <[reiserfs]unfix_nodes+9b/180> Trace; f8b3a308 <[reiserfs]do_balance+d8/130> Trace; f8b465ad <[reiserfs]fix_nodes+3fd/430> Trace; f8b531fc <[reiserfs]reiserfs_insert_item+14c/150> Trace; c02387be <__kfree_skb+fe/160> Trace; c02387be <__kfree_skb+fe/160> Trace; c02387be <__kfree_skb+fe/160> Trace; f90dd5f4 <[8139too]rtl8139_start_xmit+84/180> Trace; c02483df <qdisc_restart+1f/1b0> Trace; c023ccaf <dev_queue_xmit+17f/370> Trace; c0243af1 <neigh_resolve_output+141/270> Trace; c025a2d8 <ip_finish_output2+b8/150> Trace; c025a220 <ip_finish_output2+0/150> Trace; c025a220 <ip_finish_output2+0/150> Trace; c024763e <nf_hook_slow+1de/210> Trace; c025a220 <ip_finish_output2+0/150> Trace; c0258a4e <ip_output+14e/1e0> Trace; c025a220 <ip_finish_output2+0/150> Trace; c025a445 <ip_queue_xmit2+d5/29f> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c024763e <nf_hook_slow+1de/210> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c0258e2d <ip_queue_xmit+34d/600> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c024763e <nf_hook_slow+1de/210> Trace; c0271c20 <tcp_v4_send_check+a0/f0> Trace; c026bdc9 <tcp_transmit_skb+3e9/700> Trace; c0237f63 <sock_def_readable+63/80> Trace; c025d4a0 <tcp_rfree+0/20> Trace; c025d4a0 <tcp_rfree+0/20> Trace; c026a2bd <tcp_rcv_established+7bd/a50> Trace; c0272eda <tcp_v4_do_rcv+13a/160> Trace; c02735be <tcp_v4_rcv+6be/7a0> Trace; c025581f <ip_local_deliver_finish+15f/1a0> Trace; c02556c0 <ip_local_deliver_finish+0/1a0> Trace; c024763e <nf_hook_slow+1de/210> Trace; c02556c0 <ip_local_deliver_finish+0/1a0> Trace; c025514f <ip_local_deliver+18f/240> Trace; c02556c0 <ip_local_deliver_finish+0/1a0> Trace; c0255a89 <ip_rcv_finish+229/2bc> Trace; c0255860 <ip_rcv_finish+0/2bc> Trace; c0255860 <ip_rcv_finish+0/2bc> Trace; c024763e <nf_hook_slow+1de/210> Trace; c011cee2 <schedule+2e2/540> Trace; c011cb1e <schedule_timeout+5e/a0> Trace; c01594d4 <poll_freewait+44/50> Trace; c01598d9 <do_select+239/250> Trace; c0159bb4 <sys_select+294/4e0> Trace; c0158905 <sys_ioctl+2a5/311> Trace; c0124bb5 <sys_time+15/50> Trace; c0108fa7 <system_call+33/38> ################################################################ May 11 04:30:06 tp-meteodat7 kernel: f3668830 00000280 f2c73080 00000000 f196cd80 d2029180 d2029180 00000042 May 11 04:30:06 tp-meteodat7 kernel: c010d948 f196cd80 f008a0ec d2029180 d2029180 d2029180 00000042 f008a080 May 11 04:30:06 tp-meteodat7 kernel: f1960018 f5fd0018 ffffff14 c0238723 00000010 00000202 d2029180 d2029180 May 11 04:30:06 tp-meteodat7 kernel: Call Trace: [call_do_IRQ+5/13] [__kfree_skb+99/352] [st:__insmod_st_O/lib/modules/2.4.30-hf1/kernel/drivers/scsi/st+4294624756/96] [qdisc_restart+114/432] [dev_queue_xmit+383/880] May 11 04:30:06 tp-meteodat7 kernel: Call Trace: [<c010d948>] [<c0238723>] [<f90dd5f4>] [<c0248432>] [<c023ccaf>] May 11 04:30:06 tp-meteodat7 kernel: [<c025a2d8>] [<c025a220>] [<c025a220>] [<c024763e>] [<c025a220>] [<c0258a4e>] May 11 04:30:06 tp-meteodat7 kernel: [<c025a220>] [<c025a445>] [<c025a370>] [<c025a370>] [<c024763e>] [<c025a370>] May 11 04:30:06 tp-meteodat7 kernel: [<c0258e2d>] [<c025a370>] [<c0271c20>] [<c026bdc9>] [<c026e824>] [<c025d4a0>] May 11 04:30:06 tp-meteodat7 kernel: [<c025d4a0>] [<c026a2fa>] [<c0272eda>] [<c02735be>] [<c025581f>] [<c02556c0>] May 11 04:30:06 tp-meteodat7 kernel: [<c024763e>] [<c02556c0>] [<c025514f>] [<c02556c0>] [<c0255a89>] [<c0255860>] May 11 04:30:06 tp-meteodat7 kernel: [<c0255860>] [<c024763e>] [<c0255860>] [<c0255589>] [<c0255860>] [<c023d505>] May 11 04:30:06 tp-meteodat7 kernel: [<c023d5d3>] [<c023d76a>] [<c01254c6>] [<c010b094>] [<c010d948>] [<f9109ae3>] May 11 04:30:06 tp-meteodat7 kernel: [<f910a404>] [<f910be18>] [<f90f058d>] [<f90f066d>] [<f90f07f9>] [<f90f08ab>] May 11 04:30:06 tp-meteodat7 kernel: [<f90f20c6>] [<f914d588>] [<c018480b>] [<f914f255>] [<f8b557d4>] [<f8b629ca>] May 11 04:30:06 tp-meteodat7 kernel: [<f8b3a3c4>] [<f8b4cc74>] [<f8b3a3c4>] [<f8b39e28>] [<f8b448c3>] [<f8b4d47e>] May 11 04:30:06 tp-meteodat7 kernel: [<f8b4f8c3>] [<f8b4667b>] [<f8b3a308>] [<f8b465ad>] [<f8b531fc>] [<c02387be>] May 11 04:30:06 tp-meteodat7 kernel: [<c023cfb3>] [<f90ddb21>] [<f90dd5f4>] [<c02387be>] [<f90dd5f4>] [<c02387be>] May 11 04:30:06 tp-meteodat7 kernel: [<c02387be>] [<f90dd5f4>] [<c02483df>] [<c023ccaf>] [<c025a2d8>] [<c025a220>] May 11 04:30:06 tp-meteodat7 kernel: [<c025a220>] [<c024763e>] [<c025a220>] [<c0258a4e>] [<c025a220>] [<c025a445>] May 11 04:30:06 tp-meteodat7 kernel: [<c025a370>] [<c025a370>] [<c024763e>] [<c025a370>] [<c0258e2d>] [<c025a370>] May 11 04:30:06 tp-meteodat7 kernel: [<c0237f63>] [<c025d4a0>] [<c025d4a0>] [<c026a2bd>] [<c0272eda>] [<c02735be>] May 11 04:30:06 tp-meteodat7 kernel: [<c025581f>] [<c02556c0>] [<c024763e>] [<c02556c0>] [<c025514f>] [<c02556c0>] May 11 04:30:06 tp-meteodat7 kernel: [<c0255a89>] [<c013e624>] [<c01594d4>] [<c01598d9>] [<c0159bb4>] [<c0158905>] May 11 04:30:06 tp-meteodat7 kernel: [<c0124bb5>] [<c0108fa7>] Warning (Oops_read): Code line not seen, dumping what data is available Trace; c010d948 <call_do_IRQ+5/d> Trace; c0238723 <__kfree_skb+63/160> Trace; f90dd5f4 <[8139too]rtl8139_start_xmit+84/180> Trace; c0248432 <qdisc_restart+72/1b0> Trace; c023ccaf <dev_queue_xmit+17f/370> Trace; c025a2d8 <ip_finish_output2+b8/150> Trace; c025a220 <ip_finish_output2+0/150> Trace; c025a220 <ip_finish_output2+0/150> Trace; c024763e <nf_hook_slow+1de/210> Trace; c025a220 <ip_finish_output2+0/150> Trace; c0258a4e <ip_output+14e/1e0> Trace; c025a220 <ip_finish_output2+0/150> Trace; c025a445 <ip_queue_xmit2+d5/29f> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c024763e <nf_hook_slow+1de/210> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c0258e2d <ip_queue_xmit+34d/600> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c0271c20 <tcp_v4_send_check+a0/f0> Trace; c026bdc9 <tcp_transmit_skb+3e9/700> Trace; c026e824 <tcp_send_ack+84/d0> Trace; c025d4a0 <tcp_rfree+0/20> Trace; c025d4a0 <tcp_rfree+0/20> Trace; c026a2fa <tcp_rcv_established+7fa/a50> Trace; c0272eda <tcp_v4_do_rcv+13a/160> Trace; c02735be <tcp_v4_rcv+6be/7a0> Trace; c025581f <ip_local_deliver_finish+15f/1a0> Trace; c02556c0 <ip_local_deliver_finish+0/1a0> Trace; c024763e <nf_hook_slow+1de/210> Trace; c02556c0 <ip_local_deliver_finish+0/1a0> Trace; c025514f <ip_local_deliver+18f/240> Trace; c02556c0 <ip_local_deliver_finish+0/1a0> Trace; c0255a89 <ip_rcv_finish+229/2bc> Trace; c0255860 <ip_rcv_finish+0/2bc> Trace; c0255860 <ip_rcv_finish+0/2bc> Trace; c024763e <nf_hook_slow+1de/210> Trace; c0255860 <ip_rcv_finish+0/2bc> Trace; c0255589 <ip_rcv+389/4c0> Trace; c0255860 <ip_rcv_finish+0/2bc> Trace; c023d505 <netif_receive_skb+1e5/220> Trace; c023d5d3 <process_backlog+93/130> Trace; c023d76a <net_rx_action+fa/170> Trace; c01254c6 <do_softirq+76/e0> Trace; c010b094 <do_IRQ+f4/130> Trace; c010d948 <call_do_IRQ+5/d> Trace; f9109ae3 <[usb-uhci]insert_qh+83/c0> Trace; f910a404 <[usb-uhci]uhci_submit_control_urb+244/370> Trace; f910be18 <[usb-uhci]uhci_submit_urb+3c8/400> Trace; f90f058d <[usbcore]usb_submit_urb+3d/40> Trace; f90f066d <[usbcore]usb_start_wait_urb+7d/1a0> Trace; f90f07f9 <[usbcore]usb_internal_control_msg+69/80> Trace; f90f08ab <[usbcore]usb_control_msg+9b/b0> Trace; f90f20c6 <[usbcore]usb_get_report+76/80> Trace; f914d588 <[hid]hid_read_report+98/c0> Trace; c018480b <isofs_read_super+46b/800> Trace; f914f255 <[hid]hiddev_ioctl+565/cd0> Trace; f8b557d4 <[reiserfs]get_cnode+14/80> Trace; f8b629ca <[reiserfs].LC32+57e7/5a7d> Trace; f8b3a3c4 <[reiserfs]do_balance_mark_leaf_dirty+64/a0> Trace; f8b4cc74 <[reiserfs]leaf_insert_into_buf+1c4/2c0> Trace; f8b3a3c4 <[reiserfs]do_balance_mark_leaf_dirty+64/a0> Trace; f8b39e28 <[reiserfs]balance_leaf+30a8/3120> Trace; f8b448c3 <[reiserfs]ip_check_balance+2e3/bf0> Trace; f8b4d47e <[reiserfs]leaf_cut_from_buffer+28e/4b0> Trace; f8b4f8c3 <[reiserfs]pathrelse_and_restore+43/60> Trace; f8b4667b <[reiserfs]unfix_nodes+9b/180> Trace; f8b3a308 <[reiserfs]do_balance+d8/130> Trace; f8b465ad <[reiserfs]fix_nodes+3fd/430> Trace; f8b531fc <[reiserfs]reiserfs_insert_item+14c/150> Trace; c02387be <__kfree_skb+fe/160> Trace; c023cfb3 <netif_rx+93/1f0> Trace; f90ddb21 <[8139too]rtl8139_rx_interrupt+131/320> Trace; f90dd5f4 <[8139too]rtl8139_start_xmit+84/180> Trace; c02387be <__kfree_skb+fe/160> Trace; f90dd5f4 <[8139too]rtl8139_start_xmit+84/180> Trace; c02387be <__kfree_skb+fe/160> Trace; c02387be <__kfree_skb+fe/160> Trace; f90dd5f4 <[8139too]rtl8139_start_xmit+84/180> Trace; c02483df <qdisc_restart+1f/1b0> Trace; c023ccaf <dev_queue_xmit+17f/370> Trace; c025a2d8 <ip_finish_output2+b8/150> Trace; c025a220 <ip_finish_output2+0/150> Trace; c025a220 <ip_finish_output2+0/150> Trace; c024763e <nf_hook_slow+1de/210> Trace; c025a220 <ip_finish_output2+0/150> Trace; c0258a4e <ip_output+14e/1e0> Trace; c025a220 <ip_finish_output2+0/150> Trace; c025a445 <ip_queue_xmit2+d5/29f> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c024763e <nf_hook_slow+1de/210> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c0258e2d <ip_queue_xmit+34d/600> Trace; c025a370 <ip_queue_xmit2+0/29f> Trace; c0237f63 <sock_def_readable+63/80> Trace; c025d4a0 <tcp_rfree+0/20> Trace; c025d4a0 <tcp_rfree+0/20> Trace; c026a2bd <tcp_rcv_established+7bd/a50> Trace; c0272eda <tcp_v4_do_rcv+13a/160> Trace; c02735be <tcp_v4_rcv+6be/7a0> Trace; c025581f <ip_local_deliver_finish+15f/1a0> Trace; c02556c0 <ip_local_deliver_finish+0/1a0> Trace; c024763e <nf_hook_slow+1de/210> Trace; c02556c0 <ip_local_deliver_finish+0/1a0> Trace; c025514f <ip_local_deliver+18f/240> Trace; c02556c0 <ip_local_deliver_finish+0/1a0> Trace; c0255a89 <ip_rcv_finish+229/2bc> Trace; c013e624 <__alloc_pages+64/280> Trace; c01594d4 <poll_freewait+44/50> Trace; c01598d9 <do_select+239/250> Trace; c0159bb4 <sys_select+294/4e0> Trace; c0158905 <sys_ioctl+2a5/311> Trace; c0124bb5 <sys_time+15/50> Trace; c0108fa7 <system_call+33/38> -- +++ Lassen Sie Ihren Gedanken freien Lauf... z.B. per FreeSMS +++ GMX bietet bis zu 100 FreeSMS/Monat: http://www.gmx.net/de/go/mail ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 2.4.30-hf1 do_IRQ stack overflows 2005-05-12 8:44 ` Manfred Schwarb @ 2005-05-14 11:05 ` Herbert Xu 2005-05-17 16:05 ` Manfred Schwarb 0 siblings, 1 reply; 9+ messages in thread From: Herbert Xu @ 2005-05-14 11:05 UTC (permalink / raw) To: Manfred Schwarb; +Cc: marcelo.tosatti, linux-kernel, davem, netdev On Thu, May 12, 2005 at 10:44:37AM +0200, Manfred Schwarb wrote: > > Trace; f8b531fc <[reiserfs]reiserfs_insert_item+14c/150> > Trace; c02387be <__kfree_skb+fe/160> > Trace; c02387be <__kfree_skb+fe/160> > Trace; f90dd5f4 <[8139too]rtl8139_start_xmit+84/180> Do you have any funky netfilter/iptables modules loaded? -- Visit Openswan at http://www.openswan.org/ Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 2.4.30-hf1 do_IRQ stack overflows 2005-05-14 11:05 ` Herbert Xu @ 2005-05-17 16:05 ` Manfred Schwarb 0 siblings, 0 replies; 9+ messages in thread From: Manfred Schwarb @ 2005-05-17 16:05 UTC (permalink / raw) To: Herbert Xu; +Cc: marcelo.tosatti, linux-kernel, davem, netdev > > > > Trace; f8b531fc <[reiserfs]reiserfs_insert_item+14c/150> > > Trace; c02387be <__kfree_skb+fe/160> > > Trace; c02387be <__kfree_skb+fe/160> > > Trace; f90dd5f4 <[8139too]rtl8139_start_xmit+84/180> > > Do you have any funky netfilter/iptables modules loaded? I use a iptables based firewall, but no additional netfilter modules are loaded. The network configuration is as it is shipped by SuSE (i.e. no CONFIG_IP_NF_* modules, but most of the "Networking options" are set to y). And as I told in an earlier mail, I had scheduling built in the kernel (CONFIG_NET_SCHED=y, CONFIG_NET_QOS=y, CONFIG_NET_ESTIMATOR=y, CONFIG_NET_CLS=y, CONFIG_NET_CLS_ROUTE=y). I have disabled NET_SCHED now (as Marcelo suggested), and I got no overflows since then (4 days uptime). Seems to work so far. Thanks and regards, Manfred -- Weitersagen: GMX DSL-Flatrates mit Tempo-Garantie! Ab 4,99 Euro/Monat: http://www.gmx.net/de/go/dsl ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 2.4.30-hf1 do_IRQ stack overflows 2005-05-11 12:46 ` 2.4.30-hf1 do_IRQ stack overflows Marcelo Tosatti 2005-05-11 21:38 ` Herbert Xu @ 2005-05-12 7:59 ` Manfred Schwarb 2005-06-07 12:38 ` Manfred Schwarb 2 siblings, 0 replies; 9+ messages in thread From: Manfred Schwarb @ 2005-05-12 7:59 UTC (permalink / raw) To: Marcelo Tosatti; +Cc: linux-kernel, davem, netdev > > The traces show huge networking execution paths. > > It seems you are using some packet scheduler (CONFIG_NET_SCHED)? Pretty > much all > traces show functions from sch_generic.c. Can you disable that for a test? > Yes, indeed, I made some experiments with it, but do not need it urgently. I will disable it, thanks for the hint. And I will report back. > > > > Below my three overflow messages. Would the stack reduction patches of > Badari Pulavarty > > help in my case? If so, I would strongly vote for inclusion into 2.4 > series!! > > It has been decided that the stack reduction patches were too intrusive to > be merged > at this stage of v2.4 life. > OK, I got the message, I will consider upgrading to 2.6 sometime this year... -- +++ Lassen Sie Ihren Gedanken freien Lauf... z.B. per FreeSMS +++ GMX bietet bis zu 100 FreeSMS/Monat: http://www.gmx.net/de/go/mail ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 2.4.30-hf1 do_IRQ stack overflows 2005-05-11 12:46 ` 2.4.30-hf1 do_IRQ stack overflows Marcelo Tosatti 2005-05-11 21:38 ` Herbert Xu 2005-05-12 7:59 ` Manfred Schwarb @ 2005-06-07 12:38 ` Manfred Schwarb 2005-06-09 15:00 ` Marcelo Tosatti 2 siblings, 1 reply; 9+ messages in thread From: Manfred Schwarb @ 2005-06-07 12:38 UTC (permalink / raw) To: Marcelo Tosatti; +Cc: linux-kernel, davem, netdev, herbert > > Hi Manfred, > > On Wed, May 11, 2005 at 10:15:02AM +0200, Manfred Schwarb wrote: > > Hi, > > with recent versions of the 2.4 kernel (Vanilla), I get an increasing > amount of do_IRQ stack overflows. > > This night, I got 3 of them. > > With 2.4.28 I got an overflow about twice a year, with 2.4.29 nearly > once a month and with > > 2.4.30 nearly every day 8-(( > > The system is getting dangerously close to an actual stack overflow, which > would > crash the system. > > "do_IRQ: stack overflow: " indicates how many bytes are still available. > > The traces show huge networking execution paths. > > It seems you are using some packet scheduler (CONFIG_NET_SCHED)? Pretty > much all > traces show functions from sch_generic.c. Can you disable that for a test? > Sorry to bother you again, but the problem didn't vanish completely. This morning, I caught another one. I built a new kernel with CONFIG_NET_SCHED=n as suggested, uptime is now 25 days, and the following is the first do_IRQ since then (ksymoops -i): Jun 7 03:55:01 tp-meteodat7 kernel: f3238830 00000280 f49e7b80 00000000 00000042 cca1388e f4116980 f17aa000 Jun 7 03:55:01 tp-meteodat7 kernel: c010d948 00000042 f4116980 00000000 cca1388e f4116980 f17aa000 00000042 Jun 7 03:55:01 tp-meteodat7 kernel: 00000018 f61d0018 ffffff14 c023a039 00000010 00000246 ee5ea480 00000000 Jun 7 03:55:01 tp-meteodat7 kernel: Call Trace: [call_do_IRQ+5/13] [skb_copy_and_csum_dev+73/256] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4256445916/96] [qdisc_restart+114/432] [dev_queue_xmit+383/880] Jun 7 03:55:01 tp-meteodat7 kernel: Call Trace: [<c010d948>] [<c023a039>] [<f90df5dc>] [<c0248402>] [<c023cc7f>] Jun 7 03:55:01 tp-meteodat7 kernel: [<c02561a8>] [<c02560f0>] [<c02560f0>] [<c024760e>] [<c02560f0>] [<c025492e>] Jun 7 03:55:01 tp-meteodat7 kernel: [<c02560f0>] [<c0256315>] [<c0256240>] [<c0256240>] [<c024760e>] [<c0256240>] Jun 7 03:55:01 tp-meteodat7 kernel: [<c0254d0d>] [<c0256240>] [<c026daf0>] [<c0267c99>] [<c026a6f4>] [<c0259370>] Jun 7 03:55:01 tp-meteodat7 kernel: [<c0259370>] [<c02661ca>] [<c026edaa>] [<c026f48e>] [<c025174f>] [<c02515f0>] Jun 7 03:55:01 tp-meteodat7 kernel: [<c024760e>] [<c02515f0>] [<c0251790>] [<c02510df>] [<c02515f0>] [<c0251790>] Jun 7 03:55:01 tp-meteodat7 kernel: [<c0251969>] [<c0251790>] [<c024760e>] [<c0251790>] [<c02514b8>] [<c0251790>] Jun 7 03:55:01 tp-meteodat7 kernel: [<c023d4d5>] [<c023d5a3>] [<c023d73a>] [<c01254c6>] [<c010b094>] [<c010d948>] Jun 7 03:55:01 tp-meteodat7 kernel: [<c010a625>] [<c011ce8a>] [<c011cb14>] [<c011ca60>] [<f90f2697>] [<f90f27f9>] Jun 7 03:55:01 tp-meteodat7 kernel: [<f90f28ab>] [<f90f40c6>] [<f914f588>] [<f915053e>] [<f9151255>] [<f8b3a3c4>] Jun 7 03:55:01 tp-meteodat7 kernel: [<f8b4bcf0>] [<f8b3a3c4>] [<f8b4d80f>] [<f8b3a3c4>] [<f8b3a3c4>] [<f8b4cc74>] Jun 7 03:55:01 tp-meteodat7 kernel: [<f8b3a3c4>] [<f8b39e28>] [<f8b448c3>] [<f8b4f8c3>] [<f8b4667b>] [<f8b3a308>] Jun 7 03:55:01 tp-meteodat7 kernel: [<f8b465ad>] [<f8b531fc>] [<c02387b6>] [<f90df5f4>] [<c02483af>] [<c023cc7f>] Jun 7 03:55:01 tp-meteodat7 kernel: [<c02561a8>] [<c02387b6>] [<f90df5f4>] [<c02387b6>] [<f90df5f4>] [<c02483af>] Jun 7 03:55:01 tp-meteodat7 kernel: [<c02560f0>] [<c02560f0>] [<c024760e>] [<c02560f0>] [<c025492e>] [<c02560f0>] Jun 7 03:55:01 tp-meteodat7 kernel: [<c0256315>] [<c0237f63>] [<c0259370>] [<c0259370>] [<c026618d>] [<c026edaa>] Jun 7 03:55:01 tp-meteodat7 kernel: [<c026f48e>] [<c025174f>] [<c02515f0>] [<c024760e>] [<c02515f0>] [<c0251790>] Jun 7 03:55:01 tp-meteodat7 kernel: [<c02510df>] [<c02515f0>] [<c0251790>] [<c0251969>] [<c0251790>] [<c024760e>] Jun 7 03:55:01 tp-meteodat7 kernel: [<c0251790>] [<c013e624>] [<c01594d4>] [<c01598d9>] [<c0159bb4>] [<c0158905>] Warning (Oops_read): Code line not seen, dumping what data is available Trace; c010d948 <call_do_IRQ+5/d> Trace; c023a039 <skb_copy_and_csum_dev+49/100> Trace; f90df5dc <[8139too]rtl8139_start_xmit+6c/180> Trace; c0248402 <qdisc_restart+72/1b0> Trace; c023cc7f <dev_queue_xmit+17f/370> Trace; c02561a8 <ip_finish_output2+b8/150> Trace; c02560f0 <ip_finish_output2+0/150> Trace; c02560f0 <ip_finish_output2+0/150> Trace; c024760e <nf_hook_slow+1de/210> Trace; c02560f0 <ip_finish_output2+0/150> Trace; c025492e <ip_output+14e/1e0> Trace; c02560f0 <ip_finish_output2+0/150> Trace; c0256315 <ip_queue_xmit2+d5/29f> Trace; c0256240 <ip_queue_xmit2+0/29f> Trace; c0256240 <ip_queue_xmit2+0/29f> Trace; c024760e <nf_hook_slow+1de/210> Trace; c0256240 <ip_queue_xmit2+0/29f> Trace; c0254d0d <ip_queue_xmit+34d/600> Trace; c0256240 <ip_queue_xmit2+0/29f> Trace; c026daf0 <tcp_v4_send_check+a0/f0> Trace; c0267c99 <tcp_transmit_skb+3e9/700> Trace; c026a6f4 <tcp_send_ack+84/d0> Trace; c0259370 <tcp_rfree+0/20> Trace; c0259370 <tcp_rfree+0/20> Trace; c02661ca <tcp_rcv_established+7fa/a50> Trace; c026edaa <tcp_v4_do_rcv+13a/160> Trace; c026f48e <tcp_v4_rcv+6be/7a0> Trace; c025174f <ip_local_deliver_finish+15f/1a0> Trace; c02515f0 <ip_local_deliver_finish+0/1a0> Trace; c024760e <nf_hook_slow+1de/210> Trace; c02515f0 <ip_local_deliver_finish+0/1a0> Trace; c0251790 <ip_rcv_finish+0/268> Trace; c02510df <ip_local_deliver+18f/240> Trace; c02515f0 <ip_local_deliver_finish+0/1a0> Trace; c0251790 <ip_rcv_finish+0/268> Trace; c0251969 <ip_rcv_finish+1d9/268> Trace; c0251790 <ip_rcv_finish+0/268> Trace; c024760e <nf_hook_slow+1de/210> Trace; c0251790 <ip_rcv_finish+0/268> Trace; c02514b8 <ip_rcv+328/460> Trace; c0251790 <ip_rcv_finish+0/268> Trace; c023d4d5 <netif_receive_skb+1e5/220> Trace; c023d5a3 <process_backlog+93/130> Trace; c023d73a <net_rx_action+fa/170> Trace; c01254c6 <do_softirq+76/e0> Trace; c010b094 <do_IRQ+f4/130> Trace; c010d948 <call_do_IRQ+5/d> Trace; c010a625 <set_ldt_desc+5/3b> Trace; c011ce8a <schedule+28a/540> Trace; c011cb14 <schedule_timeout+54/a0> Trace; c011ca60 <process_timeout+0/60> Trace; f90f2697 <[usbcore]usb_start_wait_urb+a7/1a0> Trace; f90f27f9 <[usbcore]usb_internal_control_msg+69/80> Trace; f90f28ab <[usbcore]usb_control_msg+9b/b0> Trace; f90f40c6 <[usbcore]usb_get_report+76/80> Trace; f914f588 <[hid]hid_read_report+98/c0> Trace; f915053e <[hid]hiddev_lookup_report+9e/c0> Trace; f9151255 <[hid]hiddev_ioctl+565/cd0> Trace; f8b3a3c4 <[reiserfs]do_balance_mark_leaf_dirty+64/a0> Trace; f8b4bcf0 <[reiserfs]leaf_copy_items_entirely+1c0/250> Trace; f8b3a3c4 <[reiserfs]do_balance_mark_leaf_dirty+64/a0> Trace; f8b4d80f <[reiserfs]leaf_delete_items_entirely+16f/250> Trace; f8b3a3c4 <[reiserfs]do_balance_mark_leaf_dirty+64/a0> Trace; f8b3a3c4 <[reiserfs]do_balance_mark_leaf_dirty+64/a0> Trace; f8b4cc74 <[reiserfs]leaf_insert_into_buf+1c4/2c0> Trace; f8b3a3c4 <[reiserfs]do_balance_mark_leaf_dirty+64/a0> Trace; f8b39e28 <[reiserfs]balance_leaf+30a8/3120> Trace; f8b448c3 <[reiserfs]ip_check_balance+2e3/bf0> Trace; f8b4f8c3 <[reiserfs]pathrelse_and_restore+43/60> Trace; f8b4667b <[reiserfs]unfix_nodes+9b/180> Trace; f8b3a308 <[reiserfs]do_balance+d8/130> Trace; f8b465ad <[reiserfs]fix_nodes+3fd/430> Trace; f8b531fc <[reiserfs]reiserfs_insert_item+14c/150> Trace; c02387b6 <__kfree_skb+f6/150> Trace; f90df5f4 <[8139too]rtl8139_start_xmit+84/180> Trace; c02483af <qdisc_restart+1f/1b0> Trace; c023cc7f <dev_queue_xmit+17f/370> Trace; c02561a8 <ip_finish_output2+b8/150> Trace; c02387b6 <__kfree_skb+f6/150> Trace; f90df5f4 <[8139too]rtl8139_start_xmit+84/180> Trace; c02387b6 <__kfree_skb+f6/150> Trace; f90df5f4 <[8139too]rtl8139_start_xmit+84/180> Trace; c02483af <qdisc_restart+1f/1b0> Trace; c02560f0 <ip_finish_output2+0/150> Trace; c02560f0 <ip_finish_output2+0/150> Trace; c024760e <nf_hook_slow+1de/210> Trace; c02560f0 <ip_finish_output2+0/150> Trace; c025492e <ip_output+14e/1e0> Trace; c02560f0 <ip_finish_output2+0/150> Trace; c0256315 <ip_queue_xmit2+d5/29f> Trace; c0237f63 <sock_def_readable+63/80> Trace; c0259370 <tcp_rfree+0/20> Trace; c0259370 <tcp_rfree+0/20> Trace; c026618d <tcp_rcv_established+7bd/a50> Trace; c026edaa <tcp_v4_do_rcv+13a/160> Trace; c026f48e <tcp_v4_rcv+6be/7a0> Trace; c025174f <ip_local_deliver_finish+15f/1a0> Trace; c02515f0 <ip_local_deliver_finish+0/1a0> Trace; c024760e <nf_hook_slow+1de/210> Trace; c02515f0 <ip_local_deliver_finish+0/1a0> Trace; c0251790 <ip_rcv_finish+0/268> Trace; c02510df <ip_local_deliver+18f/240> Trace; c02515f0 <ip_local_deliver_finish+0/1a0> Trace; c0251790 <ip_rcv_finish+0/268> Trace; c0251969 <ip_rcv_finish+1d9/268> Trace; c0251790 <ip_rcv_finish+0/268> Trace; c024760e <nf_hook_slow+1de/210> Trace; c0251790 <ip_rcv_finish+0/268> Trace; c013e624 <__alloc_pages+64/280> Trace; c01594d4 <poll_freewait+44/50> Trace; c01598d9 <do_select+239/250> Trace; c0159bb4 <sys_select+294/4e0> Trace; c0158905 <sys_ioctl+2a5/311> -- Weitersagen: GMX DSL-Flatrates mit Tempo-Garantie! Ab 4,99 Euro/Monat: http://www.gmx.net/de/go/dsl ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 2.4.30-hf1 do_IRQ stack overflows 2005-06-07 12:38 ` Manfred Schwarb @ 2005-06-09 15:00 ` Marcelo Tosatti 2005-06-10 8:10 ` Manfred Schwarb 0 siblings, 1 reply; 9+ messages in thread From: Marcelo Tosatti @ 2005-06-09 15:00 UTC (permalink / raw) To: Manfred Schwarb; +Cc: linux-kernel, davem, netdev, herbert Hi, On Tue, Jun 07, 2005 at 02:38:01PM +0200, Manfred Schwarb wrote: > > > > > > Hi Manfred, > > > > On Wed, May 11, 2005 at 10:15:02AM +0200, Manfred Schwarb wrote: > > > Hi, > > > with recent versions of the 2.4 kernel (Vanilla), I get an increasing > > amount of do_IRQ stack overflows. > > > This night, I got 3 of them. > > > With 2.4.28 I got an overflow about twice a year, with 2.4.29 nearly > > once a month and with > > > 2.4.30 nearly every day 8-(( > > > > The system is getting dangerously close to an actual stack overflow, which > > would > > crash the system. > > > > "do_IRQ: stack overflow: " indicates how many bytes are still available. > > > > The traces show huge networking execution paths. > > > > It seems you are using some packet scheduler (CONFIG_NET_SCHED)? Pretty > > much all > > traces show functions from sch_generic.c. Can you disable that for a test? > > > > Sorry to bother you again, but the problem didn't vanish completely. > This morning, I caught another one. I built a new kernel with > CONFIG_NET_SCHED=n as suggested, uptime is now 25 days, and the following > is the first do_IRQ since then (ksymoops -i): > > Jun 7 03:55:01 tp-meteodat7 kernel: f3238830 00000280 f49e7b80 00000000 > 00000042 cca1388e f4116980 f17aa000 > Jun 7 03:55:01 tp-meteodat7 kernel: c010d948 00000042 f4116980 > 00000000 cca1388e f4116980 f17aa000 00000042 > Jun 7 03:55:01 tp-meteodat7 kernel: 00000018 f61d0018 ffffff14 > c023a039 00000010 00000246 ee5ea480 00000000 > Jun 7 03:55:01 tp-meteodat7 kernel: Call Trace: [call_do_IRQ+5/13] > [skb_copy_and_csum_dev+73/256] > [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4256445916/96] > [qdisc_restart+114/432] [dev_queue_xmit+383/880] > Jun 7 03:55:01 tp-meteodat7 kernel: Call Trace: [<c010d948>] > [<c023a039>] [<f90df5dc>] [<c0248402>] [<c023cc7f>] > Jun 7 03:55:01 tp-meteodat7 kernel: [<c02561a8>] [<c02560f0>] > [<c02560f0>] [<c024760e>] [<c02560f0>] [<c025492e>] > Jun 7 03:55:01 tp-meteodat7 kernel: [<c02560f0>] [<c0256315>] > [<c0256240>] [<c0256240>] [<c024760e>] [<c0256240>] > Jun 7 03:55:01 tp-meteodat7 kernel: [<c0254d0d>] [<c0256240>] > [<c026daf0>] [<c0267c99>] [<c026a6f4>] [<c0259370>] > Jun 7 03:55:01 tp-meteodat7 kernel: [<c0259370>] [<c02661ca>] > [<c026edaa>] [<c026f48e>] [<c025174f>] [<c02515f0>] > Jun 7 03:55:01 tp-meteodat7 kernel: [<c024760e>] [<c02515f0>] > [<c0251790>] [<c02510df>] [<c02515f0>] [<c0251790>] > Jun 7 03:55:01 tp-meteodat7 kernel: [<c0251969>] [<c0251790>] > [<c024760e>] [<c0251790>] [<c02514b8>] [<c0251790>] > Jun 7 03:55:01 tp-meteodat7 kernel: [<c023d4d5>] [<c023d5a3>] > [<c023d73a>] [<c01254c6>] [<c010b094>] [<c010d948>] > Jun 7 03:55:01 tp-meteodat7 kernel: [<c010a625>] [<c011ce8a>] > [<c011cb14>] [<c011ca60>] [<f90f2697>] [<f90f27f9>] > Jun 7 03:55:01 tp-meteodat7 kernel: [<f90f28ab>] [<f90f40c6>] > [<f914f588>] [<f915053e>] [<f9151255>] [<f8b3a3c4>] > Jun 7 03:55:01 tp-meteodat7 kernel: [<f8b4bcf0>] [<f8b3a3c4>] > [<f8b4d80f>] [<f8b3a3c4>] [<f8b3a3c4>] [<f8b4cc74>] > Jun 7 03:55:01 tp-meteodat7 kernel: [<f8b3a3c4>] [<f8b39e28>] > [<f8b448c3>] [<f8b4f8c3>] [<f8b4667b>] [<f8b3a308>] > Jun 7 03:55:01 tp-meteodat7 kernel: [<f8b465ad>] [<f8b531fc>] > [<c02387b6>] [<f90df5f4>] [<c02483af>] [<c023cc7f>] > Jun 7 03:55:01 tp-meteodat7 kernel: [<c02561a8>] [<c02387b6>] > [<f90df5f4>] [<c02387b6>] [<f90df5f4>] [<c02483af>] > Jun 7 03:55:01 tp-meteodat7 kernel: [<c02560f0>] [<c02560f0>] > [<c024760e>] [<c02560f0>] [<c025492e>] [<c02560f0>] > Jun 7 03:55:01 tp-meteodat7 kernel: [<c0256315>] [<c0237f63>] > [<c0259370>] [<c0259370>] [<c026618d>] [<c026edaa>] > Jun 7 03:55:01 tp-meteodat7 kernel: [<c026f48e>] [<c025174f>] > [<c02515f0>] [<c024760e>] [<c02515f0>] [<c0251790>] > Jun 7 03:55:01 tp-meteodat7 kernel: [<c02510df>] [<c02515f0>] > [<c0251790>] [<c0251969>] [<c0251790>] [<c024760e>] > Jun 7 03:55:01 tp-meteodat7 kernel: [<c0251790>] [<c013e624>] > [<c01594d4>] [<c01598d9>] [<c0159bb4>] [<c0158905>] > Warning (Oops_read): Code line not seen, dumping what data is available Do you have the "do_IRQ stack overflow" output and the amount of bytes left it informs? > Trace; c010d948 <call_do_IRQ+5/d> > Trace; c023a039 <skb_copy_and_csum_dev+49/100> > Trace; f90df5dc <[8139too]rtl8139_start_xmit+6c/180> > Trace; c0248402 <qdisc_restart+72/1b0> > Trace; c023cc7f <dev_queue_xmit+17f/370> > Trace; c02561a8 <ip_finish_output2+b8/150> > Trace; c02560f0 <ip_finish_output2+0/150> > Trace; c02560f0 <ip_finish_output2+0/150> I can't explain the "ip_finish_output2+0" entries. Odd. > Trace; c024760e <nf_hook_slow+1de/210> > Trace; c02560f0 <ip_finish_output2+0/150> > Trace; c025492e <ip_output+14e/1e0> > Trace; c02560f0 <ip_finish_output2+0/150> > Trace; c0256315 <ip_queue_xmit2+d5/29f> > Trace; c0256240 <ip_queue_xmit2+0/29f> > Trace; c0256240 <ip_queue_xmit2+0/29f> > Trace; c024760e <nf_hook_slow+1de/210> > Trace; c0256240 <ip_queue_xmit2+0/29f> > Trace; c0254d0d <ip_queue_xmit+34d/600> > Trace; c0256240 <ip_queue_xmit2+0/29f> > Trace; c026daf0 <tcp_v4_send_check+a0/f0> > Trace; c0267c99 <tcp_transmit_skb+3e9/700> > Trace; c026a6f4 <tcp_send_ack+84/d0> > Trace; c0259370 <tcp_rfree+0/20> > Trace; c0259370 <tcp_rfree+0/20> > Trace; c02661ca <tcp_rcv_established+7fa/a50> > Trace; c026edaa <tcp_v4_do_rcv+13a/160> > Trace; c026f48e <tcp_v4_rcv+6be/7a0> > Trace; c025174f <ip_local_deliver_finish+15f/1a0> > Trace; c02515f0 <ip_local_deliver_finish+0/1a0> > Trace; c024760e <nf_hook_slow+1de/210> > Trace; c02515f0 <ip_local_deliver_finish+0/1a0> > Trace; c0251790 <ip_rcv_finish+0/268> > Trace; c02510df <ip_local_deliver+18f/240> > Trace; c02515f0 <ip_local_deliver_finish+0/1a0> > Trace; c0251790 <ip_rcv_finish+0/268> > Trace; c0251969 <ip_rcv_finish+1d9/268> > Trace; c0251790 <ip_rcv_finish+0/268> > Trace; c024760e <nf_hook_slow+1de/210> > Trace; c0251790 <ip_rcv_finish+0/268> > Trace; c02514b8 <ip_rcv+328/460> > Trace; c0251790 <ip_rcv_finish+0/268> > Trace; c023d4d5 <netif_receive_skb+1e5/220> > Trace; c023d5a3 <process_backlog+93/130> > Trace; c023d73a <net_rx_action+fa/170> > Trace; c01254c6 <do_softirq+76/e0> > Trace; c010b094 <do_IRQ+f4/130> > Trace; c010d948 <call_do_IRQ+5/d> I dont see any huge stack consumers on this callchain. David, Herbert, any clues what might be going on here? ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 2.4.30-hf1 do_IRQ stack overflows 2005-06-09 15:00 ` Marcelo Tosatti @ 2005-06-10 8:10 ` Manfred Schwarb 0 siblings, 0 replies; 9+ messages in thread From: Manfred Schwarb @ 2005-06-10 8:10 UTC (permalink / raw) To: Marcelo Tosatti; +Cc: linux-kernel, davem, netdev, herbert > Hi, > > On Tue, Jun 07, 2005 at 02:38:01PM +0200, Manfred Schwarb wrote: > > > > > > > > > > Hi Manfred, > > > > > > On Wed, May 11, 2005 at 10:15:02AM +0200, Manfred Schwarb wrote: > > > > Hi, > > > > with recent versions of the 2.4 kernel (Vanilla), I get an > increasing > > > amount of do_IRQ stack overflows. > > > > This night, I got 3 of them. > > > > With 2.4.28 I got an overflow about twice a year, with 2.4.29 nearly > > > once a month and with > > > > 2.4.30 nearly every day 8-(( > > > > > > The system is getting dangerously close to an actual stack overflow, > which > > > would > > > crash the system. > > > > > > "do_IRQ: stack overflow: " indicates how many bytes are still > available. > > > > > > The traces show huge networking execution paths. > > > > > > It seems you are using some packet scheduler (CONFIG_NET_SCHED)? > Pretty > > > much all > > > traces show functions from sch_generic.c. Can you disable that for a > test? > > > > > > > Sorry to bother you again, but the problem didn't vanish completely. > > This morning, I caught another one. I built a new kernel with > > CONFIG_NET_SCHED=n as suggested, uptime is now 25 days, and the > following > > is the first do_IRQ since then (ksymoops -i): > > > > Jun 7 03:55:01 tp-meteodat7 kernel: f3238830 00000280 f49e7b80 00000000 <---------snip--------> > > [<c01594d4>] [<c01598d9>] [<c0159bb4>] [<c0158905>] > > Warning (Oops_read): Code line not seen, dumping what data is available > > Do you have the "do_IRQ stack overflow" output and the amount of bytes > left it informs? > Yes, it was a close one, 640. I append the original output to the end of this email. Thanks for looking at this. > > Trace; c010d948 <call_do_IRQ+5/d> > > Trace; c023a039 <skb_copy_and_csum_dev+49/100> <---------snip--------> > > Trace; c010d948 <call_do_IRQ+5/d> > > I dont see any huge stack consumers on this callchain. > > David, Herbert, any clues what might be going on here? > > Jun 7 03:55:01 tp-meteodat7 kernel: do_IRQ: stack overflow: 640 Jun 7 03:55:01 tp-meteodat7 kernel: f3238830 00000280 f49e7b80 00000000 00000042 cca1388e f4116980 f17aa000 Jun 7 03:55:01 tp-meteodat7 kernel: c010d948 00000042 f4116980 00000000 cca1388e f4116980 f17aa000 00000042 Jun 7 03:55:01 tp-meteodat7 kernel: 00000018 f61d0018 ffffff14 c023a039 00000010 00000246 ee5ea480 00000000 Jun 7 03:55:01 tp-meteodat7 kernel: Call Trace: [call_do_IRQ+5/13] [skb_copy_and_csum_dev+73/256] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4256445916/96] [qdisc_restart+114/432] [dev_queue_xmit+383/880] Jun 7 03:55:01 tp-meteodat7 kernel: Call Trace: [<c010d948>] [<c023a039>] [<f90df5dc>] [<c0248402>] [<c023cc7f>] Jun 7 03:55:01 tp-meteodat7 kernel: [ip_finish_output2+184/336] [ip_finish_output2+0/336] [ip_finish_output2+0/336] [nf_hook_slow+478/528] [ip_finish_output2+0/336] [ip_output+334/480] Jun 7 03:55:01 tp-meteodat7 kernel: [<c02561a8>] [<c02560f0>] [<c02560f0>] [<c024760e>] [<c02560f0>] [<c025492e>] Jun 7 03:55:01 tp-meteodat7 kernel: [ip_finish_output2+0/336] [ip_queue_xmit2+213/671] [ip_queue_xmit2+0/671] [ip_queue_xmit2+0/671] [nf_hook_slow+478/528] [ip_queue_xmit2+0/671] Jun 7 03:55:01 tp-meteodat7 kernel: [<c02560f0>] [<c0256315>] [<c0256240>] [<c0256240>] [<c024760e>] [<c0256240>] Jun 7 03:55:01 tp-meteodat7 kernel: [ip_queue_xmit+845/1536] [ip_queue_xmit2+0/671] [tcp_v4_send_check+160/240] [tcp_transmit_skb+1001/1792] [tcp_send_ack+132/208] [tcp_rfree+0/32] Jun 7 03:55:01 tp-meteodat7 kernel: [<c0254d0d>] [<c0256240>] [<c026daf0>] [<c0267c99>] [<c026a6f4>] [<c0259370>] Jun 7 03:55:01 tp-meteodat7 kernel: [tcp_rfree+0/32] [tcp_rcv_established+2042/2640] [tcp_v4_do_rcv+314/352] [tcp_v4_rcv+1726/1952] [ip_local_deliver_finish+351/416] [ip_local_deliver_finish+0/416] Jun 7 03:55:01 tp-meteodat7 kernel: [<c0259370>] [<c02661ca>] [<c026edaa>] [<c026f48e>] [<c025174f>] [<c02515f0>] Jun 7 03:55:01 tp-meteodat7 kernel: [nf_hook_slow+478/528] [ip_local_deliver_finish+0/416] [ip_rcv_finish+0/616] [ip_local_deliver+399/576] [ip_local_deliver_finish+0/416] [ip_rcv_finish+0/616] Jun 7 03:55:01 tp-meteodat7 kernel: [<c024760e>] [<c02515f0>] [<c0251790>] [<c02510df>] [<c02515f0>] [<c0251790>] Jun 7 03:55:01 tp-meteodat7 kernel: [ip_rcv_finish+473/616] [ip_rcv_finish+0/616] [nf_hook_slow+478/528] [ip_rcv_finish+0/616] [ip_rcv+808/1120] [ip_rcv_finish+0/616] Jun 7 03:55:01 tp-meteodat7 kernel: [<c0251969>] [<c0251790>] [<c024760e>] [<c0251790>] [<c02514b8>] [<c0251790>] Jun 7 03:55:01 tp-meteodat7 kernel: [netif_receive_skb+485/544] [process_backlog+147/304] [net_rx_action+250/368] [do_softirq+118/224] [do_IRQ+244/304] [call_do_IRQ+5/13] Jun 7 03:55:01 tp-meteodat7 kernel: [<c023d4d5>] [<c023d5a3>] [<c023d73a>] [<c01254c6>] [<c010b094>] [<c010d948>] Jun 7 03:55:01 tp-meteodat7 kernel: [set_ldt_desc+5/59] [schedule+650/1344] [schedule_timeout+84/160] [process_timeout+0/96] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4256523927/96] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4256524281/96] Jun 7 03:55:01 tp-meteodat7 kernel: [<c010a625>] [<c011ce8a>] [<c011cb14>] [<c011ca60>] [<f90f2697>] [<f90f27f9>] Jun 7 03:55:01 tp-meteodat7 kernel: [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4256524459/96] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4256530630/96] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4256904584/96] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4256908606/96] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4256911957/96] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4250526660/96] Jun 7 03:55:01 tp-meteodat7 kernel: [<f90f28ab>] [<f90f40c6>] [<f914f588>] [<f915053e>] [<f9151255>] [<f8b3a3c4>] Jun 7 03:55:01 tp-meteodat7 kernel: [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4250598640/96] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4250526660/96] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4250605583/96] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4250526660/96] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4250526660/96] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4250602612/96] Jun 7 03:55:01 tp-meteodat7 kernel: [<f8b4bcf0>] [<f8b3a3c4>] [<f8b4d80f>] [<f8b3a3c4>] [<f8b3a3c4>] [<f8b4cc74>] Jun 7 03:55:01 tp-meteodat7 kernel: [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4250526660/96] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4250525224/96] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4250568899/96] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4250613955/96] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4250576507/96] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4250526472/96] Jun 7 03:55:01 tp-meteodat7 kernel: [<f8b3a3c4>] [<f8b39e28>] [<f8b448c3>] [<f8b4f8c3>] [<f8b4667b>] [<f8b3a308>] Jun 7 03:55:01 tp-meteodat7 kernel: [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4250576301/96] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4250628604/96] [__kfree_skb+246/336] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4256445940/96] [qdisc_restart+31/432] [dev_queue_xmit+383/880] Jun 7 03:55:01 tp-meteodat7 kernel: [<f8b465ad>] [<f8b531fc>] [<c02387b6>] [<f90df5f4>] [<c02483af>] [<c023cc7f>] Jun 7 03:55:01 tp-meteodat7 kernel: [ip_finish_output2+184/336] [__kfree_skb+246/336] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4256445940/96] [__kfree_skb+246/336] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4256445940/96] [qdisc_restart+31/432] Jun 7 03:55:01 tp-meteodat7 kernel: [<c02561a8>] [<c02387b6>] [<f90df5f4>] [<c02387b6>] [<f90df5f4>] [<c02483af>] Jun 7 03:55:01 tp-meteodat7 kernel: [dev_queue_xmit+383/880] [__kfree_skb+246/336] [nfsd:__insmod_nfsd_O/lib/modules/2.4.30-hf1/kernel/fs/nfsd/nfsd.+4256445940/96] [qdisc_restart+31/432] [dev_queue_xmit+383/880] [ip_finish_output2+184/336] Jun 7 03:55:01 tp-meteodat7 kernel: [ip_finish_output2+0/336] [ip_finish_output2+0/336] [nf_hook_slow+478/528] [ip_finish_output2+0/336] [ip_output+334/480] [ip_finish_output2+0/336] Jun 7 03:55:01 tp-meteodat7 kernel: [<c02560f0>] [<c02560f0>] [<c024760e>] [<c02560f0>] [<c025492e>] [<c02560f0>] Jun 7 03:55:01 tp-meteodat7 kernel: [ip_queue_xmit2+213/671] [sock_def_readable+99/128] [tcp_rfree+0/32] [tcp_rfree+0/32] [tcp_rcv_established+1981/2640] [tcp_v4_do_rcv+314/352] Jun 7 03:55:01 tp-meteodat7 kernel: [<c0256315>] [<c0237f63>] [<c0259370>] [<c0259370>] [<c026618d>] [<c026edaa>] Jun 7 03:55:01 tp-meteodat7 kernel: [tcp_v4_rcv+1726/1952] [ip_local_deliver_finish+351/416] [ip_local_deliver_finish+0/416] [nf_hook_slow+478/528] [ip_local_deliver_finish+0/416] [ip_rcv_finish+0/616] Jun 7 03:55:01 tp-meteodat7 kernel: [<c026f48e>] [<c025174f>] [<c02515f0>] [<c024760e>] [<c02515f0>] [<c0251790>] Jun 7 03:55:01 tp-meteodat7 kernel: [ip_local_deliver+399/576] [ip_local_deliver_finish+0/416] [ip_rcv_finish+0/616] [ip_rcv_finish+473/616] [ip_rcv_finish+0/616] [nf_hook_slow+478/528] Jun 7 03:55:01 tp-meteodat7 kernel: [<c02510df>] [<c02515f0>] [<c0251790>] [<c0251969>] [<c0251790>] [<c024760e>] Jun 7 03:55:01 tp-meteodat7 kernel: [ip_rcv_finish+0/616] [__alloc_pages+100/640] [poll_freewait+68/80] [do_select+569/592] [sys_select+660/1248] [sys_ioctl+677/785] Jun 7 03:55:01 tp-meteodat7 kernel: [<c0251790>] [<c013e624>] [<c01594d4>] [<c01598d9>] [<c0159bb4>] [<c0158905>] Jun 7 03:55:01 tp-meteodat7 kernel: [sys_time+21/80] [system_call+51/56] Jun 7 03:55:01 tp-meteodat7 kernel: [<c0124bb5>] [<c0108fa7>] In my previous posting, I snipped /var/log/messages at the wrong place, and zapped the last line. This line results in two additional lines of the ksymoops output: Trace; c0124bb5 <sys_time+15/50> Trace; c0108fa7 <system_call+33/38> -- Geschenkt: 3 Monate GMX ProMail gratis + 3 Ausgaben stern gratis ++ Jetzt anmelden & testen ++ http://www.gmx.net/de/go/promail ++ ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2005-06-10 8:10 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <5.2.1.1.2.20050511093252.01c887b0@mail.meteodat.ch>
2005-05-11 12:46 ` 2.4.30-hf1 do_IRQ stack overflows Marcelo Tosatti
2005-05-11 21:38 ` Herbert Xu
2005-05-12 8:44 ` Manfred Schwarb
2005-05-14 11:05 ` Herbert Xu
2005-05-17 16:05 ` Manfred Schwarb
2005-05-12 7:59 ` Manfred Schwarb
2005-06-07 12:38 ` Manfred Schwarb
2005-06-09 15:00 ` Marcelo Tosatti
2005-06-10 8:10 ` Manfred Schwarb
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).