re-formated /Re: Hi!Couldn't you take a look at kernel stack's trace? (ip_route_input+0x91a/0xcc9)/

re-formated /Re: Hi!Couldn't you take a look at kernel stack's trace? (ip_route_input+0x91a/0xcc9)/

[Igor Podlesny]

   Hm, not too easy to read. Sorry! Re-trying with prefixes removed:
----------X8------------------X8------------------X8------------------X8--------
BUG: unable to handle kernel NULL pointer dereference at 00000019
 IP: [<c0313d75>] ip_route_input+0x91a/0xcc9
 *pde = 00000000
 Oops: 0002 [#1] PREEMPT SMP
 Modules linked in: snd_rtctimer rtc ip_queue xt_owner ipt_recent
ipt_ttl ipt_addrtype ipt_REJECT iptable_filter ipt_LOG xt_limit
xt_DSCP xt_TCPMSS iptable_mangle it87 hwmon_vid hwmon snd_usb_audio
snd_usb_lib usbhid hid parport_pc 8250_pnp 8250 serial_core lp
ppp_generic parport slhc forcedeth ohci_hcd ehci_hcd i2c_nforce2 fan
button fuse via_rhine 3c59x mii tun powernow_k8 thermal processor
8021q tuner tea5767 tda8290 tuner_xc2028 tda9887 tuner_simple mt20xx
tea5761 saa7134 videodev v4l1_compat compat_ioctl32 v4l2_common
videobuf_dma_sg videobuf_core ir_kbd_i2c ir_common tveeprom i2c_core
snd_seq_oss snd_seq_midi_event snd_seq snd_emu10k1 firmware_class
snd_util_mem snd_hwdep snd_pcm_oss snd_mixer_oss snd_via82xx
snd_ac97_codec ac97_bus snd_pcm snd_timer snd_page_alloc
snd_mpu401_uart snd_rawmidi snd_seq_device snd soundcore loop pcspkr
evdev usbcore

 Pid: 8, comm: sirq-net-rx/0 Not tainted (2.6.25.4-rt4aa-gcc43 #1)
 EIP: 0060:[<c0313d75>] EFLAGS: 00010202 CPU: 0
 EIP is at ip_route_input+0x91a/0xcc9
 EAX: 00000000 EBX: f5a69e00 ECX: f7477e14 EDX: 00000001
 ESI: 0f6ea8c0 EDI: f7477ea4 EBP: ffffff8f ESP: f7477e38
  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 preempt:00000001
 Process sirq-net-rx/0 (pid: 8, ti=f7476000 task=f746c690 task.ti=f7476000)
 Stack: 00000000 016ea8c0 f6e2f180 00000001 c04f1964 00000001 f5a69e00 016ea8c0
        00000000 00000000 c04851cc f786d640 00000000 00000006 00000000 016ea8c0
        0f6ea8c0 00000000 00000000 00000000 00000000 00000000 00000000 00000000
 Call Trace:
  [<c024d578>] __next_cpu+0x12/0x21
  [<c0333ee8>] arp_process+0x1fe/0x5fe
  [<c01277f5>] wake_up_process+0xe/0x10
  [<c0172e80>] __kmalloc+0x97/0xcb
  [<c02f16be>] __alloc_skb+0x45/0xf2
  [<c02f58da>] netif_receive_skb+0x2dd/0x30b
  [<c02f5975>] process_backlog+0x6d/0xc1
  [<c02f46a3>] net_rx_action+0x65/0x14c
  [<c01304dd>] ksoftirqd+0x138/0x214
  [<c01303a5>] ksoftirqd+0x0/0x214
  [<c013c30e>] kthread+0x38/0x5d
  [<c013c2d6>] kthread+0x0/0x5d
  [<c0113f3f>] kernel_thread_helper+0x7/0x10
  =======================
 Code: ff ff 8b 5c 24 18 f0 ff 4b 04 0f 94 c0 84 c0 74 07 89 d8 e8 32
1b 02 00 83 7c 24 14 00 0f 84 98 03 00 00 8b 54 24 70 85 d2 74 12 <f0>
ff 4a 18 0f 94 c0 84 c0 74 07 89 d0 e8 e7 92 02 00 8b 54 24
 EIP: [<c0313d75>] ip_route_input+0x91a/0xcc9 SS:ESP 0068:f7477e38
 ---[ end trace 0f990b633467f554 ]---
----------X8------------------X8------------------X8------------------X8--------

	Plus, it's been bin-pasted: http://pastebin.com/d209e7256
--
End of message. Next message?

Re: re-formated /Re: Hi!Couldn't you take a look at kernel stack's trace? (ip_route_input+0x91a/0xcc9)/

[Patrick McHardy]

Igor Podlesny wrote:
>  Pid: 8, comm: sirq-net-rx/0 Not tainted (2.6.25.4-rt4aa-gcc43 #1)
>  EIP: 0060:[<c0313d75>] EFLAGS: 00010202 CPU: 0
>  EIP is at ip_route_input+0x91a/0xcc9
>  EAX: 00000000 EBX: f5a69e00 ECX: f7477e14 EDX: 00000001
>  ESI: 0f6ea8c0 EDI: f7477ea4 EBP: ffffff8f ESP: f7477e38
>   DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 preempt:00000001
>  Process sirq-net-rx/0 (pid: 8, ti=f7476000 task=f746c690 task.ti=f7476000)
>  Stack: 00000000 016ea8c0 f6e2f180 00000001 c04f1964 00000001 f5a69e00 016ea8c0
>         00000000 00000000 c04851cc f786d640 00000000 00000006 00000000 016ea8c0
>         0f6ea8c0 00000000 00000000 00000000 00000000 00000000 00000000 00000000
>  Call Trace:
>   [<c024d578>] __next_cpu+0x12/0x21
>   [<c0333ee8>] arp_process+0x1fe/0x5fe
>   [<c01277f5>] wake_up_process+0xe/0x10
>   [<c0172e80>] __kmalloc+0x97/0xcb
>   [<c02f16be>] __alloc_skb+0x45/0xf2
>   [<c02f58da>] netif_receive_skb+0x2dd/0x30b
>   [<c02f5975>] process_backlog+0x6d/0xc1
>   [<c02f46a3>] net_rx_action+0x65/0x14c
>   [<c01304dd>] ksoftirqd+0x138/0x214
>   [<c01303a5>] ksoftirqd+0x0/0x214
>   [<c013c30e>] kthread+0x38/0x5d
>   [<c013c2d6>] kthread+0x0/0x5d
>   [<c0113f3f>] kernel_thread_helper+0x7/0x10

Whats sirq-net-rx? Doesn't seem to be present in the mainline tree.

Re: re-formated /Re: Hi!Couldn't you take a look at kernel stack's trace? (ip_route_input+0x91a/0xcc9)/

[Igor Podlesny]

2008/6/12 Patrick McHardy <kaber@trash.net>:
> > Igor Podlesny wrote:
>> >>
>> >>  Pid: 8, comm: sirq-net-rx/0 Not tainted (2.6.25.4-rt4aa-gcc43 #1)
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^

	(2.6.25.4-rt4aa-gcc43)
	
	-RT is real-time: http://rt.wiki.kernel.org/index.php/Main_Page

>> >>  EIP: 0060:[<c0313d75>] EFLAGS: 00010202 CPU: 0
[...]
> > Whats sirq-net-rx? Doesn't seem to be present in the mainline tree.
> >

-- 
End of message. Next message?

Re: re-formated /Re: Hi!Couldn't you take a look at kernel stack's trace? (ip_route_input+0x91a/0xcc9)/

[Patrick McHardy]

Igor Podlesny wrote:
> 2008/6/12 Patrick McHardy <kaber@trash.net>:
>>> Igor Podlesny wrote:
>>>>>  Pid: 8, comm: sirq-net-rx/0 Not tainted (2.6.25.4-rt4aa-gcc43 #1)
>                                             ^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> 	(2.6.25.4-rt4aa-gcc43)
> 	
> 	-RT is real-time: http://rt.wiki.kernel.org/index.php/Main_Page

Thanks, I missed that.

Re: re-formated /Re: Hi!Couldn't you take a look at kernel stack's trace? (ip_route_input+0x91a/0xcc9)/

[Jarek Poplawski]

Patrick McHardy wrote, On 06/12/2008 02:26 PM:

> Igor Podlesny wrote:
>> 2008/6/12 Patrick McHardy <kaber@trash.net>:
>>>> Igor Podlesny wrote:
>>>>>>  Pid: 8, comm: sirq-net-rx/0 Not tainted (2.6.25.4-rt4aa-gcc43 #1)
>>                                             ^^^^^^^^^^^^^^^^^^^^^^^^^
>>
>> 	(2.6.25.4-rt4aa-gcc43)
>> 	
>> 	-RT is real-time: http://rt.wiki.kernel.org/index.php/Main_Page

Hi,

Maybe I got something wrong but it seems to be this place:

static int ip_route_input_slow(struct sk_buff *skb, __be32 daddr, __be32 saddr,
                               u8 tos, struct net_device *dev)
...
        if (!IN_DEV_FORWARD(in_dev))
                goto e_hostunreach;
        if (res.type != RTN_UNICAST)
                goto martian_destination;

        err = ip_mkroute_input(skb, &res, &fl, in_dev, daddr, saddr, tos);
done:
        in_dev_put(in_dev);
        if (free_res)
                fib_res_put(&res);	<------------
out:    return err;
=====
static inline void fib_res_put(struct fib_result *res)
{
        if (res->fi)
                fib_info_put(res->fi);	<------------
#ifdef CONFIG_IP_MULTIPLE_TABLES
        if (res->r)
                fib_rule_put(res->r);
#endif
}
=====
static inline void fib_info_put(struct fib_info *fi)
{
        if (atomic_dec_and_test(&fi->fib_clntref))	<---------
                free_fib_info(fi);
}

Looks like EDX: 00000001 is fi, and fi + 0x18 is for fib_clntref,
but I didn't track it further.

Igor, probably for this list it's more interesting if it's reproducible
with unpatched kernels, and if you tried other than 2.6.25 versions
for this?

Thanks,
Jarek P.

> with unpatched kernels, and if you tried other than 2.6.25 versions

[Igor M Podlesny]

On 2008-06-13 03:36, Jarek Poplawski wrote:
> Patrick McHardy wrote, On 06/12/2008 02:26 PM:
[...]
> 
> Igor, probably for this list it's more interesting if it's reproducible
> with unpatched kernels, and if you tried other than 2.6.25 versions
> for this?
> 
> Thanks,
> Jarek P.

	Recently I got another one (several times, actually):

[105373.451522] swapper: page allocation failure. order:2, mode:0x4020
[105373.451527] Pid: 0, comm: swapper Tainted: P         2.6.25.6aa-gcc43 #1
[105373.451540]  [<c0157670>] __alloc_pages+0x2af/0x2c7
[105373.451550]  [<c015770f>] __get_free_pages+0x3c/0x4d
[105373.451553]  [<c02ec2e1>] __alloc_skb+0x49/0xf6
[105373.451558]  [<f8b65b9d>] nv_alloc_rx_optimized+0x45/0x15d [forcedeth]
[105373.451567]  [<f8b6a768>] nv_napi_poll+0x50f/0x5e7 [forcedeth]
[105373.451577]  [<c02ef173>] net_rx_action+0x65/0x158
[105373.451582]  [<c012fa8a>] __do_softirq+0x5a/0xa8
[105373.451588]  [<c012fb0a>] do_softirq+0x32/0x36
[105373.451591]  [<c012fc5f>] irq_exit+0x35/0x76
[105373.451594]  [<c01157c7>] do_IRQ+0x54/0x65
[105373.451599]  [<c0113c0f>] common_interrupt+0x23/0x28
[105373.451607]  [<c011238c>] default_idle+0x36/0x5a
[105373.451610]  [<c0112356>] default_idle+0x0/0x5a
[105373.451613]  [<c0112326>] cpu_idle+0x8c/0xbc
[105373.451620]  =======================

$ uname -sr
Linux 2.6.25.6aa-gcc43

BTW, it's unclear, would it be better to have rebooted the comp. immediately after
getting those messages?

-- 

Re: > with unpatched kernels, and if you tried other than 2.6.25 versions

[Jarek Poplawski]

On Fri, Jun 13, 2008 at 03:46:36AM +0800, Igor M Podlesny wrote:
> On 2008-06-13 03:36, Jarek Poplawski wrote:
> > Patrick McHardy wrote, On 06/12/2008 02:26 PM:
> [...]
> > 
> > Igor, probably for this list it's more interesting if it's reproducible
> > with unpatched kernels, and if you tried other than 2.6.25 versions
> > for this?
> > 
> > Thanks,
> > Jarek P.
> 
> 	Recently I got another one (several times, actually):
> 
> [105373.451522] swapper: page allocation failure. order:2, mode:0x4020
> [105373.451527] Pid: 0, comm: swapper Tainted: P         2.6.25.6aa-gcc43 #1
> [105373.451540]  [<c0157670>] __alloc_pages+0x2af/0x2c7
> [105373.451550]  [<c015770f>] __get_free_pages+0x3c/0x4d
> [105373.451553]  [<c02ec2e1>] __alloc_skb+0x49/0xf6
> [105373.451558]  [<f8b65b9d>] nv_alloc_rx_optimized+0x45/0x15d [forcedeth]
> [105373.451567]  [<f8b6a768>] nv_napi_poll+0x50f/0x5e7 [forcedeth]
> [105373.451577]  [<c02ef173>] net_rx_action+0x65/0x158
> [105373.451582]  [<c012fa8a>] __do_softirq+0x5a/0xa8
> [105373.451588]  [<c012fb0a>] do_softirq+0x32/0x36
> [105373.451591]  [<c012fc5f>] irq_exit+0x35/0x76
> [105373.451594]  [<c01157c7>] do_IRQ+0x54/0x65
> [105373.451599]  [<c0113c0f>] common_interrupt+0x23/0x28
> [105373.451607]  [<c011238c>] default_idle+0x36/0x5a
> [105373.451610]  [<c0112356>] default_idle+0x0/0x5a
> [105373.451613]  [<c0112326>] cpu_idle+0x8c/0xbc
> [105373.451620]  =======================
> 
> $ uname -sr
> Linux 2.6.25.6aa-gcc43
> 
> BTW, it's unclear, would it be better to have rebooted the comp. immediately after
> getting those messages?

Hm... hard to tell. This shouldn't happen and rebooting isn't a solution.
Try to eliminate this (is it patched kernel BTW?). Check your hardware
and maybe try a more current kernel (e.g. 2.6.26-rc5).

BTW, is it your mailer or some memory problems: this message has strange
"Subject"...

Jarek P.

> Hm... hard to tell. This shouldn't happen and rebooting isn't a solution.

[Igor M Podlesny]

On 2008-06-13 04:15, Jarek Poplawski wrote:
> On Fri, Jun 13, 2008 at 03:46:36AM +0800, Igor M Podlesny wrote:
	[...]
> Hm... hard to tell. This shouldn't happen and rebooting isn't a solution.

	I was asking not bout whether reboot was a solution but data corruption/loss
prevention...

-- 

> try a more current kernel (e.g. 2.6.26-rc5).

[Igor M Podlesny]

On 2008-06-13 04:15, Jarek Poplawski wrote:
> On Fri, Jun 13, 2008 at 03:46:36AM +0800, Igor M Podlesny wrote:
>> On 2008-06-13 03:36, Jarek Poplawski wrote:
	[...]
> Hm... hard to tell. This shouldn't happen and rebooting isn't a solution.
> Try to eliminate this (is it patched kernel BTW?). Check your hardware
> and maybe try a more current kernel (e.g. 2.6.26-rc5).

	2.6.26 has changed something in sensors interface and Nvidia's driver could not be
compiled for it as well. As far as I use TwinView I can afford just switching to not
Nvidia's driver.

-- 

> this message has strange "Subject"...

[Igor M Podlesny]

On 2008-06-13 04:15, Jarek Poplawski wrote:
	[...]
> BTW, is it your mailer or some memory problems: this message has strange
> "Subject"...

	Neither of above. It's just my habbit to keep Subjects related to current state of
mail, not original one.

-- 

Re: > try a more current kernel (e.g. 2.6.26-rc5).

[Stephen Hemminger]

On Fri, 13 Jun 2008 04:21:34 +0800
Igor M Podlesny <for.poige+linux@gmail.com> wrote:

> On 2008-06-13 04:15, Jarek Poplawski wrote:
> > On Fri, Jun 13, 2008 at 03:46:36AM +0800, Igor M Podlesny wrote:
> >> On 2008-06-13 03:36, Jarek Poplawski wrote:
> 	[...]
> > Hm... hard to tell. This shouldn't happen and rebooting isn't a solution.
> > Try to eliminate this (is it patched kernel BTW?). Check your hardware
> > and maybe try a more current kernel (e.g. 2.6.26-rc5).
> 
> 	2.6.26 has changed something in sensors interface and Nvidia's driver could not be
> compiled for it as well. As far as I use TwinView I can afford just switching to not
> Nvidia's driver.
> 

And kernel developer's can not afford debugging closed source drivers.

> And kernel developer's can not afford debugging closed source drivers.

[Igor M Podlesny]

On 2008-06-13 04:39, Stephen Hemminger wrote:
[...]
> And kernel developer's can not afford debugging closed source drivers.

	If you were more focused on attentiveness instead of such kind of bravado then you
could have noticed that the original stack trace happened before nvidia's kernel
module were loaded at all.

	What it means regarding kernel open source code quality is left as exercise to a
reader. Have fun.

-- 

Re: > Hm... hard to tell. This shouldn't happen and rebooting isn't a solution.

[Jarek Poplawski]

On Fri, Jun 13, 2008 at 04:19:09AM +0800, Igor M Podlesny wrote:
> On 2008-06-13 04:15, Jarek Poplawski wrote:
> > On Fri, Jun 13, 2008 at 03:46:36AM +0800, Igor M Podlesny wrote:
> 	[...]
> > Hm... hard to tell. This shouldn't happen and rebooting isn't a solution.
> 
> 	I was asking not bout whether reboot was a solution but data corruption/loss
> prevention...
> 
> -- 

OK, a reboot after a bug can sometimes prevent further data corruption/loss.

> OK, a reboot after a bug can sometimes prevent further data corruption/loss.

[Igor M Podlesny]

On 2008-06-13 04:49, Jarek Poplawski wrote:
	[...]
> 
> OK, a reboot after a bug can sometimes prevent further data corruption/loss.

	:-) In general ("sometimes") it's obvious. I was asking bout that very case.

-- 

Re: > OK, a reboot after a bug can sometimes prevent further data corruption/loss.

[Jarek Poplawski]

Igor M Podlesny wrote, On 06/12/2008 10:51 PM:

> On 2008-06-13 04:49, Jarek Poplawski wrote:
> 	[...]
>> OK, a reboot after a bug can sometimes prevent further data corruption/loss.
> 
> 	:-) In general ("sometimes") it's obvious. I was asking bout that very case.
> 


This case looks helpless to me...