oops in ip_rcv

oops in ip_rcv

[Anton Blanchard]

Hi,

Im booting a recent 2.6.12-rc3++ git tree on ppc64. Just got an oops
very early in boot, right after:

TCP established hash table entries: 131072 (order: 9, 2097kernel:

It looks like we tried to load from address -1.

c0000000003c515c ip_rcv+0x4c/0x6e0
c0000000003a6750 netif_receive_skb+0x290/0x3b0
c0000000003a6954 process_backlog+0xe4/0x250
c0000000003a4204 net_rx_action+0x114/0x2d0
c00000000005e06c __do_softirq+0xec/0x1d0
c000000000014568 call_do_softirq+0x14/0x24
c00000000000fdcc do_softirq+0x9c/0xb0
c00000000005dad4 irq_exit+0x64/0x80
c00000000000fcbc do_IRQ+0x7c/0xf0

I do have some local patches, I'll try to reproduce without them.

Anton

Re: oops in ip_rcv

[Herbert Xu]

Anton Blanchard <anton@samba.org> wrote:
> 
> Im booting a recent 2.6.12-rc3++ git tree on ppc64. Just got an oops
> very early in boot, right after:
> 
> TCP established hash table entries: 131072 (order: 9, 2097kernel:
> 
> It looks like we tried to load from address -1.
> 
> c0000000003c515c ip_rcv+0x4c/0x6e0

Any idea why you'd be receiving packets this early?
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: oops in ip_rcv

[Anton Blanchard]

 
> Any idea why you'd be receiving packets this early?

Not sure, although PCI has already been probed and tcp_init is called
with interrupts on. Any reason we cant take a packet at this point?

Im guessing its calling this:

        IP_INC_STATS_BH(IPSTATS_MIB_INRECEIVES);

Which we dont initialise until later on in init_ipv4_mibs.

Anton

Re: oops in ip_rcv

[David S. Miller]

On Thu, 5 May 2005 23:09:04 +1000
Anton Blanchard <anton@samba.org> wrote:

>  
> > Any idea why you'd be receiving packets this early?
> 
> Not sure, although PCI has already been probed and tcp_init is called
> with interrupts on. Any reason we cant take a packet at this point?
> 
> Im guessing its calling this:
> 
>         IP_INC_STATS_BH(IPSTATS_MIB_INRECEIVES);
> 
> Which we dont initialise until later on in init_ipv4_mibs.

Can you test out putting the init_ipv4_mibs() call right before
the comment:

	/*
	 *	Add all the base protocols.
	 */

If it fixes things for you I'll put that upstream.

Thanks.

I still would really like to know how you are getting packets
at all that early.  Even if PCI is initialized, no interfaces
should be marked up at _all_.

Re: oops in ip_rcv

[Herbert Xu]

On Thu, May 05, 2005 at 11:26:24AM -0700, David S. Miller wrote:
> 
> Can you test out putting the init_ipv4_mibs() call right before
> the comment:

Could we delay the call of dev_add_pack(&ip_packet_type) until
the very end of inet_init? That way we're guaranteed to not
receive anything until it's ready.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: oops in ip_rcv

[Herbert Xu]

[-- Attachment #1: Type: text/plain, Size: 533 bytes --]

On Fri, May 06, 2005 at 07:37:40AM +1000, herbert wrote:
> 
> Could we delay the call of dev_add_pack(&ip_packet_type) until
> the very end of inet_init? That way we're guaranteed to not
> receive anything until it's ready.

This patch should do the trick.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

[-- Attachment #2: p --]
[-- Type: text/plain, Size: 1136 bytes --]

===== net/ipv4/af_inet.c 1.86 vs edited =====
--- 1.86/net/ipv4/af_inet.c	2005-03-27 09:04:35 +10:00
+++ edited/net/ipv4/af_inet.c	2005-05-06 07:41:44 +10:00
@@ -1010,6 +1010,15 @@
 static int ipv4_proc_init(void);
 extern void ipfrag_init(void);
 
+/*
+ *	IP protocol layer initialiser
+ */
+
+static struct packet_type ip_packet_type = {
+	.type = __constant_htons(ETH_P_IP),
+	.func = ip_rcv,
+};
+
 static int __init inet_init(void)
 {
 	struct sk_buff *dummy_skb;
@@ -1102,6 +1111,8 @@
 	ipv4_proc_init();
 
 	ipfrag_init();
+
+	dev_add_pack(&ip_packet_type);
 
 	rc = 0;
 out:
===== net/ipv4/ip_output.c 1.82 vs edited =====
--- 1.82/net/ipv4/ip_output.c	2005-03-24 07:17:35 +11:00
+++ edited/net/ipv4/ip_output.c	2005-05-06 07:41:29 +10:00
@@ -1323,23 +1323,8 @@
 	ip_rt_put(rt);
 }
 
-/*
- *	IP protocol layer initialiser
- */
-
-static struct packet_type ip_packet_type = {
-	.type = __constant_htons(ETH_P_IP),
-	.func = ip_rcv,
-};
-
-/*
- *	IP registers the packet type and then calls the subprotocol initialisers
- */
-
 void __init ip_init(void)
 {
-	dev_add_pack(&ip_packet_type);
-
 	ip_rt_init();
 	inet_initpeers();
 

Re: oops in ip_rcv

[Herbert Xu]

[-- Attachment #1: Type: text/plain, Size: 634 bytes --]

On Fri, May 06, 2005 at 07:44:53AM +1000, herbert wrote:
> On Fri, May 06, 2005 at 07:37:40AM +1000, herbert wrote:
> > 
> > Could we delay the call of dev_add_pack(&ip_packet_type) until
> > the very end of inet_init? That way we're guaranteed to not
> > receive anything until it's ready.
> 
> This patch should do the trick.

Here is the version of IPv6.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

[-- Attachment #2: p --]
[-- Type: text/plain, Size: 639 bytes --]

===== net/ipv6/af_inet6.c 1.92 vs edited =====
--- 1.92/net/ipv6/af_inet6.c	2005-03-27 09:04:35 +10:00
+++ edited/net/ipv6/af_inet6.c	2005-05-06 08:01:36 +10:00
@@ -774,7 +774,6 @@
 	if (if6_proc_init())
 		goto proc_if6_fail;
 #endif
-	ipv6_packet_init();
 	ip6_route_init();
 	ip6_flowlabel_init();
 	err = addrconf_init();
@@ -791,6 +790,8 @@
 	/* Init v6 transport protocols. */
 	udpv6_init();
 	tcpv6_init();
+
+	ipv6_packet_init();
 	err = 0;
 out:
 	return err;
@@ -798,7 +799,6 @@
 addrconf_fail:
 	ip6_flowlabel_cleanup();
 	ip6_route_cleanup();
-	ipv6_packet_cleanup();
 #ifdef CONFIG_PROC_FS
 	if6_proc_exit();
 proc_if6_fail:

Re: oops in ip_rcv

[David S. Miller]

On Fri, 6 May 2005 07:44:53 +1000
Herbert Xu <herbert@gondor.apana.org.au> wrote:

> On Fri, May 06, 2005 at 07:37:40AM +1000, herbert wrote:
> > 
> > Could we delay the call of dev_add_pack(&ip_packet_type) until
> > the very end of inet_init? That way we're guaranteed to not
> > receive anything until it's ready.
> 
> This patch should do the trick.
> 
> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Of course... But I don't want to apply this until Anton
tells us how in the world he is receiving packets so
early in the boot.

If I put this patch in now, he may get lazy just like me
and never try and figure out where they are coming from :)

Re: oops in ip_rcv

[Herbert Xu]

On Thu, May 05, 2005 at 03:09:43PM -0700, David S. Miller wrote:
> 
> Of course... But I don't want to apply this until Anton
> tells us how in the world he is receiving packets so
> early in the boot.

Alright, that's simply a matter of dropping a BUG into netif_rx.
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: oops in ip_rcv

[Anton Blanchard]

 
Hi,

> Of course... But I don't want to apply this until Anton
> tells us how in the world he is receiving packets so
> early in the boot.
> 
> If I put this patch in now, he may get lazy just like me
> and never try and figure out where they are coming from :)

You know me too well :)

I start getting acenic irqs straight after the card is initialised:

acenic.c: v0.92 08/05/2002  Jes Sorensen, linux-acenic@SunSITE.dk
                            http://home.cern.ch/~jes/gige/acenic.html
0001:40:0c.0: Alteon AceNIC Gigabit Ethernet at 0xd7fcc000, irq 45
  Tigon II (Rev. 6), Firmware: 12.4.11, MAC: 00:02:55:9a:0c:bd
  PCI bus width: 64 bits, speed: 66MHz, latency: 72 clks
0001:40:0c.0: Firmware up and running
drivers/net/ibmveth.c: ibmveth: IBM i/pSeries Virtual Ethernet Driver 1.03
eth1: Optical link UP (Full Duplex, Flow Control: )
netconsole: not configured, aborting
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
sym0: <875> rev 

cpu 0x0: Vector: 700 (Program Check) at
[c00000000fffxbb10]
    pc: c00000000043a6fc: .netif_rx+0x1c/0x4 00
    lr: c0000000002aae20: .ace_interrupt+0x1c0/t0xac0
    sp: c00000000fffbd90
   msr: a000000000029032
  current = 0xc0000000005ca500
  paca    = 0xc000000000585400
    pid   = 0, comm = swapper
kernel BUG in netif_rx at 0net/core/dev.c:1443!
0:mon> t
c0000000002aae20 .ace_interrupt+0x1c0/0xac0
c000000000084ca4 .handle_IRQ_event+0x84/0x100
c00000000001457c .call_handle_IRQ_event+0x14/0x24
c00000000000fa74 .ppc_irq_dispatch_handler+0x224/0x3f0
c00000000000fcb8 .do_IRQ+0x78/0xf0
c00000000000ae34 hardware_interrupt_entry+0x8/0x54
--- Exception: 501 (Hardware Interrupt) at c000000000010270 .default_idle+0xa0/0x130
c000000000010074 .cpu_idle+0x34/0x50
c00000000000c03c .rest_init+0x3c/0x60
c0000000004d3904 .start_kernel+0x2a4/0x320
c00000000000bfb4 .__setup_cpu_power3+0x0/0x4

Re: oops in ip_rcv

[David S. Miller]

On Fri, 6 May 2005 11:02:46 +1000
Anton Blanchard <anton@samba.org> wrote:
 
> I start getting acenic irqs straight after the card is initialised:

Thanks, this needs to be fixed.  I bet some people would be very
unpleased to know that an inteface that hasn't been brought up
at least once will sit and receive broadcast frames :-/

Anyways, for now I'll put Herbert's ipv4 and ipv6 patches in.

Thanks a lot Anton :-)