Oops in percpu_modalloc in 2.6.9-rc1-mm1

Oops in percpu_modalloc in 2.6.9-rc1-mm1

[Pete Zaitcev]

Here's an oops on boot, happens every time when iptables-restore is run:

Linux version 2.6.9-rc1-mm1-ub (zaitcev@lembas) (gcc version 3.3.3 20040412 (Red Hat Linux 3.3.3-7)) #1 SMP Tue Aug 31 19:07:35 PDT 2004
.........
ip_tables: (C) 2000-2002 Netfilter core team
------------[ cut here ]------------
kernel BUG at kernel/module.c:264!
invalid operand: 0000 [#1]
SMP 
Modules linked in: iptable_filter ip_tables ide_cd cdrom sg scsi_mod microcode uhci_hcd usbcore thermal processor fan button battery asus_acpi ac rtc
CPU:    0
EIP:    0060:[<c013b495>]    Not tainted VLI
EFLAGS: 00010202   (2.6.9-rc1-mm1-ub) 
EIP is at percpu_modalloc+0x115/0x130
eax: 0000003c   ebx: e08fe000   ecx: 00000017   edx: 00000020
esi: 00000017   edi: 00000398   ebp: 00000000   esp: decb3ef8
ds: 007b   es: 007b   ss: 0068
Process modprobe (pid: 2004, threadinfo=decb2000 task=df5ed9b0)
Stack: 0807a218 00000001 e08fe000 00000017 00000398 00000000 c013d344 0000003c 
       00000020 c1673430 00000292 c014b53f dffab980 c1673430 00000292 00000000 
       dfcef8b4 00000000 e0904f20 00000017 00000000 00000000 00000000 00000011 
Call Trace:
 [<c013d344>] load_module+0x3f4/0xa50
 [<c014b53f>] kmem_cache_free+0x2f/0x70
 [<c013da0a>] sys_init_module+0x6a/0x260
 [<c0155e54>] sys_munmap+0x34/0x50
 [<c01050b9>] sysenter_past_esp+0x52/0x79
Code: 59 74 a4 8b 35 94 d7 3e c0 eb d9 29 c8 eb b5 8d b4 26 00 00 00 00 85 c9 0f 84 51 ff ff ff 0f 0b 0e 01 2d de 30 c0 e9 44 ff ff ff <0f> 0b 08 01 2d de 30 c0 e9 f0 fe ff ff 8d b4 26 00 00 00 00 8d 

I'm not sure who may be interested, maybe Rusty? Or AKPM?
This didn't happen in linux-2.6.8.1-mm3, so it's new.

The name of the kernel ends in -ub, but it hasn't got any patches.
I was just about to look at an oops report in ub, rebuilt new kernel
and woops...

-- Pete

Re: Oops in percpu_modalloc in 2.6.9-rc1-mm1

[Rusty Russell]

On Thu, 2004-09-02 at 16:35, Pete Zaitcev wrote:
> On Wed, 01 Sep 2004 15:21:45 +1000
> Rusty Russell <rusty@rustcorp.com.au> wrote:
> 
> > Pete, can you figure out what module it is which goes boom, send me the
> > .ko file, and tell me the value of CONFIG_X86_L1_CACHE_SHIFT?
> 
> Two modules have percpu data segment: net/ipv4/netfilter/ipchains.o
> and net/ipv4/netfilter/ip_conntrack_core.o.
> The CONFIG_X86_L1_CACHE_SHIFT is 4.

...

>  12 .data         000006e4  00000000  00000000  0000ba20  2**5
>                   CONTENTS, ALLOC, LOAD, RELOC, DATA
>  13 .data.percpu  0000003c  00000000  00000000  0000c120  2**5
>                   CONTENTS, ALLOC, LOAD, DATA

Well, here's the problem.  alloc_bootmem will return something
SMP_CACHE_BYTES aligned, so we can't meet this.  If we knew in advance
what the alignment requirements were, we could do so.

Note also, that archs currently use vmalloc() or variants for
module_alloc, but they could easily use kmalloc() for small sizes.  This
would make your life painful, since kmalloc() also returns L1
cache-aligned memory.

I'd suggest that for a 486 configuration, GCC shouldn't be asking for
32-byte alignment for anything.  Still, we probably have to remove the
BUG_ON and simply accept that alignment > SMP_CACHE_BYTES is actually
taking pot luck (depends if the original allocation was aligned).

Rusty.
-- 
Anyone who quotes me in their signature is an idiot -- Rusty Russell