OOPS at boot in 2.4.17-rc[12] (kernel BUG at slab.c:815) maybe devfs

OOPS at boot in 2.4.17-rc[12] (kernel BUG at slab.c:815) maybe devfs

[Gregor Suhr]

[-- Attachment #1: Type: text/plain, Size: 4490 bytes --]

  Hi,
I tried to upgrade my system from 2.4.16 to 2.4.17-rc2 but I'am always 
got the following OOPS:

ksymoops 2.4.3 on i686 2.4.16.  Options used
     -v /usr/src/linux/vmlinux (specified)
     -K (specified)
     -L (specified)
     -o /lib/modules/2.4.17-rc2 (specified)
     -m /usr/src/linux/System.map (specified)

No modules in ksyms, skipping objects
kernel BUG at slab.c:815!
invalid operand: 0000
CPU:    0
EIP:    0010:[<c012960a>]    Not tainted
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010286
eax: 0000001a   ebx: dfe38698   ecx: 00000001   edx: 00002a56
esi: dfe38691   edi: c0324117   ebp: 00000000   esp: c181dee4
ds: 0018   es: 0018   ss: 0018
Process swapper (pid: 1, stackpage=c181d000)
Stack: c0316083 0000032f c181def4 dfe386b8 00000018 c180f368 c180f324 
c180f324
       c030c113 c03a7ad4 c032410a 0000001c 00000020 00000000 00000000 
00000000
       c03a72f3 c0317020 00000002 00000000 00000000 dfe80e24 00003a00 
00000009
Call Trace: [<c0118206>] [<c0118226>] [<c010522c>] [<c0105000>] [<c010525e>]
   [<c0105000>] [<c0105726>] [<c0105250>]
Code: 0f 0b 5f 8b 13 5d 89 d3 8b 03 89 c2 0f 18 02 81 fb c8 dc 37

 >>EIP; c012960a <kmem_cache_create+40a/470>   <=====
Trace; c0118206 <sys_wait4+3a6/3b0>
Trace; c0118226 <sys_waitpid+16/20>
Trace; c010522c <prepare_namespace+11c/140>
Trace; c0105000 <_stext+0/0>
Trace; c010525e <init+e/140>
Trace; c0105000 <_stext+0/0>
Trace; c0105726 <kernel_thread+26/30>
Trace; c0105250 <init+0/140>
Code;  c012960a <kmem_cache_create+40a/470>
00000000 <_EIP>:
Code;  c012960a <kmem_cache_create+40a/470>   <=====
   0:   0f 0b                     ud2a      <=====
Code;  c012960c <kmem_cache_create+40c/470>
   2:   5f                        pop    %edi
Code;  c012960c <kmem_cache_create+40c/470>
   3:   8b 13                     mov    (%ebx),%edx
Code;  c012960e <kmem_cache_create+40e/470>
   5:   5d                        pop    %ebp
Code;  c0129610 <kmem_cache_create+410/470>
   6:   89 d3                     mov    %edx,%ebx
Code;  c0129612 <kmem_cache_create+412/470>
   8:   8b 03                     mov    (%ebx),%eax
Code;  c0129614 <kmem_cache_create+414/470>
   a:   89 c2                     mov    %eax,%edx
Code;  c0129616 <kmem_cache_create+416/470>
   c:   0f 18 02                  prefetchnta (%edx)
Code;  c0129618 <kmem_cache_create+418/470>
   f:   81 fb c8 dc 37 00         cmp    $0x37dcc8,%ebx

 <0>Kernel panic: Attempted to kill init!

Before the kernel gives up i get the following error messages while 
trying to find my LVM drives:

Mounted devfs on /dev
modprobe: Can't open dependencies file 
/lib/modules/2.4.17-rc2/modules.dep (No such file odevfs: 
devfs_mk_dir(vg0): using old entry in dir: c1808724 ""
r directory)
vgdevfs: devfs_register(group): could not append to parent, err: -17
scan -- reading devfs: devfs_register(root): could not append to parent, 
err: -17
all physical voldevfs: devfs_register(tmp): could not append to parent, 
err: -17
umes (this may tdevfs: devfs_register(var): could not append to parent, 
err: -17
ake a while...)devfs: devfs_register(squid): could not append to parent, 
err: -17

vgscan -- founddevfs: devfs_register(usr): could not append to parent, 
err: -17
 inactive volumedevfs: devfs_register(home): could not append to parent, 
err: -17
 group "vg0"
vgdevfs: devfs_register(swap): could not append to parent, err: -17
scan -- "/etc/lvdevfs: devfs_register(redhat): could not append to 
parent, err: -17
mtab" and "/etc/devfs: devfs_register(mp3): could not append to parent, 
err: -17
lvmtab.d" succesdevfs: devfs_register(appdata): could not append to 
parent, err: -17
sfully created
devfs: devfs_register(oracle): could not append to parent, err: -17
vgscan -- WARNING: This program does not do a VGDA backup of your volume 
group

vgchange -- volume group "vg0" successfully activated


System Configuration:
PIII 700
512MB RAM
Gnu C                  2.96
Gnu make               3.79.1
binutils               2.11.90.0.8
util-linux             2.11f
mount                  2.11l
modutils               2.4.12
e2fsprogs              1.23
PPP                    2.4.1
isdn4k-utils           3.1pre1
Linux C Library        2.2.4
Dynamic linker (ldd)   2.2.4
Procps                 2.0.7
Net-tools              1.60
Console-tools          0.3.3
Sh-utils               2.0.11

All partitions are on  a LVM drive, and should be found by the initrd.

I hope the problem will be solved before 2.4.17.

Best regards,

Gregor Suhr



[-- Attachment #2: linuxconfig.gz --]
[-- Type: application/x-gzip, Size: 0 bytes --]

Re: OOPS at boot in 2.4.17-rc[12] (kernel BUG at slab.c:815) maybe devfs

[Richard Gooch]

Gregor Suhr writes:
> I tried to upgrade my system from 2.4.16 to 2.4.17-rc2 but I'am always 
> got the following OOPS:
> 
> kernel BUG at slab.c:815!
> invalid operand: 0000
> CPU:    0
> EIP:    0010:[<c012960a>]    Not tainted
> Using defaults from ksymoops -t elf32-i386 -a i386
> EFLAGS: 00010286
> eax: 0000001a   ebx: dfe38698   ecx: 00000001   edx: 00002a56
> esi: dfe38691   edi: c0324117   ebp: 00000000   esp: c181dee4
> ds: 0018   es: 0018   ss: 0018
> Process swapper (pid: 1, stackpage=c181d000)
> Stack: c0316083 0000032f c181def4 dfe386b8 00000018 c180f368 c180f324 
> c180f324
>        c030c113 c03a7ad4 c032410a 0000001c 00000020 00000000 00000000 
> 00000000
>        c03a72f3 c0317020 00000002 00000000 00000000 dfe80e24 00003a00 
> 00000009
> Call Trace: [<c0118206>] [<c0118226>] [<c010522c>] [<c0105000>] [<c010525e>]
>    [<c0105000>] [<c0105726>] [<c0105250>]
> Code: 0f 0b 5f 8b 13 5d 89 d3 8b 03 89 c2 0f 18 02 81 fb c8 dc 37
> 
>  >>EIP; c012960a <kmem_cache_create+40a/470>   <=====
> Trace; c0118206 <sys_wait4+3a6/3b0>
> Trace; c0118226 <sys_waitpid+16/20>
> Trace; c010522c <prepare_namespace+11c/140>
> Trace; c0105000 <_stext+0/0>
> Trace; c010525e <init+e/140>
> Trace; c0105000 <_stext+0/0>
> Trace; c0105726 <kernel_thread+26/30>
> Trace; c0105250 <init+0/140>

No mention of devfs in the traceback. It doesn't look like a devfs
problem. 

> Before the kernel gives up i get the following error messages while 
> trying to find my LVM drives:
> 
> Mounted devfs on /dev
> modprobe: Can't open dependencies file 
> /lib/modules/2.4.17-rc2/modules.dep (No such file or directory)

You've got a problem with your modules configuration.

> devfs: devfs_mk_dir(vg0): using old entry in dir: c1808724 ""
> vgdevfs: devfs_register(group): could not append to parent, err: -17
> scan -- reading devfs: devfs_register(root): could not append to parent, 
> err: -17
> all physical voldevfs: devfs_register(tmp): could not append to parent, 
> err: -17
> umes (this may tdevfs: devfs_register(var): could not append to parent, 
> err: -17
> ake a while...)devfs: devfs_register(squid): could not append to parent, 
> err: -17
> 
> vgscan -- founddevfs: devfs_register(usr): could not append to parent, 
> err: -17
>  inactive volumedevfs: devfs_register(home): could not append to parent, 
> err: -17
>  group "vg0"
> vgdevfs: devfs_register(swap): could not append to parent, err: -17
> scan -- "/etc/lvdevfs: devfs_register(redhat): could not append to 
> parent, err: -17
> mtab" and "/etc/devfs: devfs_register(mp3): could not append to parent, 
> err: -17
> lvmtab.d" succesdevfs: devfs_register(appdata): could not append to 
> parent, err: -17
> sfully created
> devfs: devfs_register(oracle): could not append to parent, err: -17

These devfs-related messages are due to configuration problems
(i.e. boot scripts untarring a pile of crap into devfs), but they are
unlikely to be the cause of your Oops.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: OOPS at boot in 2.4.17-rc[12] (kernel BUG at slab.c:815) maybe devfs

[Andrew Morton]

Richard Gooch wrote:
> 
> Gregor Suhr writes:
> > ...
> > kernel BUG at slab.c:815!

Somebody tried to create the same cache twice.  This can
happen when loading a buggy module the second time, various
things.

Please, apply this patch and run it again.

--- linux-2.4.17-rc2/mm/slab.c	Tue Dec 18 19:37:31 2001
+++ linux-akpm/mm/slab.c	Thu Dec 20 16:14:19 2001
@@ -811,8 +811,10 @@ next:
 			kmem_cache_t *pc = list_entry(p, kmem_cache_t, next);
 
 			/* The name field is constant - no lock needed. */
-			if (!strcmp(pc->name, name))
+			if (!strcmp(pc->name, name)) {
+				printk(__FUNCTION__ ": %s\n", name);
 				BUG();
+			}
 		}
 	}

Re: OOPS at boot in 2.4.17-rc[12] (kernel BUG at slab.c:815) maybe devfs

[Richard Gooch]

Gregor Suhr writes:
> Richard Gooch wrote:
> 
> >Gregor Suhr writes:
> >
> >>>>
> >>vgchange -- volume group "vg0" successfully activated
> >>
> >>kjournald starting.  Commit interval 5 seconds
> >>EXT3-fs: mounted filesystem with ordered data mode.
> >>VFS: Mounted root (ext3 filesystem) readonly.
> >>devfs: devfs_do_symlink(root): could not append to parent, err: -17
> >>change_root: old root has d_count=2
> >>kmem_cache_create: devfsd_event
> >>
> >
> >Now this is useful information! I see what's caused this: the second
> >mount of devfs (because you're using initrd) is creating the
> >devfsd_event slab cache again, which is a bug. I've appended a patch
> >which fixes this. Please test it out and let me know how it goes.
> >
> I will try the patch, but it seem to fit only on plain 2.4.17 and not on 
> my 2.4.17-rc2 (is it right??).

It will be fine on 2.4.17-rc2 as well. The patch is localised to
devfs, and there are no devfs changes between 2.4.17-rc2 and 2.4.17.

> >>Code: 0f 0b 5f 8b 13 5d 89 d3 8b 03 89 c2 0f 18 02 81 fb 08 dd 37
> >> <0>Kernel panic: Attempted to kill init!
> >>
> >
> >Where is the ksymoops decoding of this? It looks like you didn't run
> >ksymoops on each Oops trace.
> >
> It is the same trace as above (now in the last mail), but i thougt I is 
> better to append a part of the bootlog.
> As Andrew Morton pointed out the oops happens then devfs trys to create 
> an second slab cache.

Are you sure you mean "then devfs trys...", or perhaps you mean "when
devfs trys..."? If you actually mean "then", that implies that there
are two things trying to create a duplicate cache. One of them being
devfs (which is fixed in the patch I sent you), and the other is some
random driver. It would be good to know for sure.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: OOPS at boot in 2.4.17-rc[12] (kernel BUG at slab.c:815) maybe devfs

[Richard Gooch]

Gregor Suhr writes:
> >
> >
> >
> >Now this is useful information! I see what's caused this: the second
> >mount of devfs (because you're using initrd) is creating the
> >devfsd_event slab cache again, which is a bug. I've appended a patch
> >which fixes this. Please test it out and let me know how it goes.
> >
> I applied the patch to 2.4.17 and it seems to work well (the system came 
> up).

Great.

> Do you have an idea how I can solve the err. -17 error or may i realy 
> ignore it?

I've added a section to the devfs FAQ to answer this.
http://www.atnf.csiro.au/~rgooch/linux/docs/devfs.html

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: OOPS at boot in 2.4.17-rc[12] (kernel BUG at slab.c:815) maybe devfs

[Richard Gooch]

Gregor Suhr writes:
>   Richard Gooch wrote:
> 
> >Gregor Suhr writes:
> >
> >>
> >>Trace; c0105000 <_stext+0/0>
> >>Trace; c010525e <init+e/140>
> >>Trace; c0105000 <_stext+0/0>
> >>Trace; c0105726 <kernel_thread+26/30>
> >>Trace; c0105250 <init+0/140>
> >>
> >
> >No mention of devfs in the traceback. It doesn't look like a devfs
> >problem. 
> >
> OK

However, on seeing your other message, I take that back. See my other
reply.

> >>mtab" and "/etc/devfs: devfs_register(mp3): could not append to parent, 
> >>err: -17
> >>lvmtab.d" succesdevfs: devfs_register(appdata): could not append to 
> >>parent, err: -17
> >>sfully created
> >>devfs: devfs_register(oracle): could not append to parent, err: -17
> >>
> >
> >These devfs-related messages are due to configuration problems
> >(i.e. boot scripts untarring a pile of crap into devfs), but they are
> >unlikely to be the cause of your Oops.
> >
> The linuxrc script  of the initrd include:
> 
>    #!/bin/sh
>    /bin/mount /proc
>    /sbin/vgscan
>    /sbin/vgchange -a y
>    /bin/umount /proc
> 
> But it is using a mountd devfs without runnig devfsd.
> I used this inird (without any change or rebuild) since 2.4 without any 
> problem, so something in the kernel must been changed.

Yes, something in the kernel has changed. New devfs core, which
doesn't work with broken drivers, and generates warning messages for
bad configurations. But those are warnings, and broken configurations
should continue to work (in 2.4, in 2.5 they won't). The problem you
are having is unrelated to a broken configuration. See other message.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca