Oops in prune_dcache (2.4.0-prerelease)

Oops in prune_dcache (2.4.0-prerelease)

[Udo A. Steinberg]

Hi Linus et. all

While under massive disk and cpu load, 2.4.0-prerelease produced
the following oops (decode see below)

Keith, I've read the FAQ about having been bitten by Makefile bugs
with certain symbols and such, yet I still get these symbol warnings
even after a mrproper rebuild. Any clues?

-Udo.

ksymoops 2.3.5 on i686 2.4.0-prerelease.  Options used
     -V (default)
     -k /proc/ksyms (default)
     -l /proc/modules (default)
     -o /lib/modules/2.4.0-prerelease/ (default)
     -m /boot/System.map-2.4.0-prerelease (specified)

Warning (compare_maps): ksyms_base symbol acpi_clear_event_R__ver_acpi_clear_event not found in System.map.  Ignoring ksyms_base entry
Warning (compare_maps): ksyms_base symbol acpi_cm_memcpy_R__ver_acpi_cm_memcpy not found in System.map.  Ignoring ksyms_base entry

[ ** many other warnings snipped to reduce spam ** ]

Unable to handle kernel paging request at virtual address 01000014
c01419cc
*pde = 00000000
Oops: 0000
CPU:    0
EIP:    0010:[<c01419cc>]
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010206
eax: 01000000   ebx: c20847e0   ecx: c2081d10   edx: c2081d10
esi: c20847c0   edi: c2081d00   ebp: 00002c79   esp: c147bfa4
ds: 0018   es: 0018   ss: 0018
Process kswapd (pid: 3, stackpage=c147b000)
Stack: 00010f00 00000003 00000004 00000000 c0141cc1 00006ea6 c012a0d3 00000006 
       00000004 00010f00 c023e1f1 c147a239 0008e000 c012a19a 00000004 00000000 
       cffe5fbc c0105000 ffffff9c c01074d3 00000000 c02d6d64 c02a5fdc 
Call Trace: [<c0141cc1>] [<c012a0d3>] [<c023e1f1>] [<c012a19a>] [<c0105000>] [<c01074d3>] 
Code: 8b 40 14 85 c0 74 09 57 56 ff d0 83 c4 08 eb 0c 57 e8 be 1b 

>>EIP; c01419cc <prune_dcache+9c/120>   <=====
Trace; c0141cc1 <shrink_dcache_memory+21/30>
Trace; c012a0d3 <do_try_to_free_pages+53/90>
Trace; c023e1f1 <tvecs+2169/1b124>
Trace; c012a19a <kswapd+8a/140>
Trace; c0105000 <empty_bad_page+0/1000>
Trace; c01074d3 <kernel_thread+23/30>
Code;  c01419cc <prune_dcache+9c/120>
00000000 <_EIP>:
Code;  c01419cc <prune_dcache+9c/120>   <=====
   0:   8b 40 14                  movl   0x14(%eax),%eax   <=====
Code;  c01419cf <prune_dcache+9f/120>
   3:   85 c0                     testl  %eax,%eax
Code;  c01419d1 <prune_dcache+a1/120>
   5:   74 09                     je     10 <_EIP+0x10> c01419dc <prune_dcache+ac/120>
Code;  c01419d3 <prune_dcache+a3/120>
   7:   57                        pushl  %edi
Code;  c01419d4 <prune_dcache+a4/120>
   8:   56                        pushl  %esi
Code;  c01419d5 <prune_dcache+a5/120>
   9:   ff d0                     call   *%eax
Code;  c01419d7 <prune_dcache+a7/120>
   b:   83 c4 08                  addl   $0x8,%esp
Code;  c01419da <prune_dcache+aa/120>
   e:   eb 0c                     jmp    1c <_EIP+0x1c> c01419e8 <prune_dcache+b8/120>
Code;  c01419dc <prune_dcache+ac/120>
  10:   57                        pushl  %edi
Code;  c01419dd <prune_dcache+ad/120>
  11:   e8 be 1b 00 00            call   1bd4 <_EIP+0x1bd4> c01435a0 <iput+0/150>


45 warnings issued.  Results may not be reliable.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Oops in prune_dcache (2.4.0-prerelease)

[Linus Torvalds]

On Wed, 3 Jan 2001, Udo A. Steinberg wrote:
> 
> While under massive disk and cpu load, 2.4.0-prerelease produced
> the following oops (decode see below)

Hmm.. If I'm not mistaken, this is in dentry_iput() (inline function
called by prune_one_dentry(), which is _also_ an inline function, which
is why it gets reported as being in prune_dcache):

	if (dentry->d_op && dentry->d_op->d_iput)
		dentry->d_op->d_iput(dentry, inode);

and it looks like your dentry->d_op has a value of 0x01000000, so when we
load the d_op->d_iput pointer, we get a page fault.

The strange thing is that 0x01000000 value, which almost certainly should
just be NULL. A one-bit error.

Now, I assume this machine has been historically stable, with no history
of memory corruption problems.. It's entirely possible (and likely) that
the one-bit error is due to some wild kernel pointer. Which makes this
_really_ hard to debug.

I'll try to think about it some more, but I'd love to have more reports to
go on to try to find a pattern..

		Linus

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Oops in prune_dcache (2.4.0-prerelease)

[Alexander Viro]

On Wed, 3 Jan 2001, Udo A. Steinberg wrote:

> 
> Hi Linus et. all
> 
> While under massive disk and cpu load, 2.4.0-prerelease produced
> the following oops (decode see below)
 
> Unable to handle kernel paging request at virtual address 01000014
 
> Code;  c01419cc <prune_dcache+9c/120>   <=====
>    0:   8b 40 14                  movl   0x14(%eax),%eax   <=====
> Code;  c01419cf <prune_dcache+9f/120>
>    3:   85 c0                     testl  %eax,%eax
> Code;  c01419d1 <prune_dcache+a1/120>
>    5:   74 09                     je     10 <_EIP+0x10> c01419dc <prune_dcache+ac/120>

dentry->d_op == 0x1000000 in dentry_iput(). 9:1 that you've got bit 24 flipped
(i.e. it was supposed to be NULL and you are seeing an effect of hardware
problem).

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Oops in prune_dcache (2.4.0-prerelease)

[Udo A. Steinberg]

Hi,

Linus Torvalds wrote:
> 
> The strange thing is that 0x01000000 value, which almost certainly should
> just be NULL. A one-bit error.
> 
> Now, I assume this machine has been historically stable, with no history
> of memory corruption problems.. It's entirely possible (and likely) that
> the one-bit error is due to some wild kernel pointer. Which makes this
> _really_ hard to debug.

Yes the machine is otherwise rock stable, not overclocked and memory timings
are rather conservative. Before the oops the machine had been compiling some
major application for like 5 hours and maybe the excessive stress kicked a
bit somewhere - who knows.

> I'll try to think about it some more, but I'd love to have more reports to
> go on to try to find a pattern..

That's one I can't reproduce. I've just run memtest86 over the entire ram
and it doesn't show any oddities - which doesn't really rule out an
occassional bit-flip due to neutrino storms though ;-)
If someone else has seen something similar lately, it's time to speak up.

-Udo.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Oops in prune_dcache (2.4.0-prerelease)

[Dan Aloni]

On Tue, 2 Jan 2001, Linus Torvalds wrote:

> On Wed, 3 Jan 2001, Udo A. Steinberg wrote:
> > 
> > While under massive disk and cpu load, 2.4.0-prerelease produced
> > the following oops (decode see below)

[..]

> Now, I assume this machine has been historically stable, with no history
> of memory corruption problems.. It's entirely possible (and likely) that
> the one-bit error is due to some wild kernel pointer. Which makes this
> _really_ hard to debug.

After a bit of few code reviewing, it looks like the only code that
assigns stuff to ->d_op in a nonstandard way is in fs/vfat/namei.c. 

Udo, are you using vfat?

-- 
Dan Aloni 
dax@karrde.org

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

[patch] 2.4.0-prerelease acpi exported symbols

[Keith Owens]

On Wed, 03 Jan 2001 04:39:50 +0100, 
"Udo A. Steinberg" <sorisor@Hell.WH8.TU-Dresden.De> wrote:
>Keith, I've read the FAQ about having been bitten by Makefile bugs
>with certain symbols and such, yet I still get these symbol warnings
>even after a mrproper rebuild. Any clues?
>Warning (compare_maps): ksyms_base symbol acpi_clear_event_R__ver_acpi_clear_event not found in System.map.  Ignoring ksyms_base entry

Two separate problems.

(1) acpi used a file called ksyms.c which conflicted with the kernel's
    export file of the same name.  The symbol version code uses a flat
    namespace for its files so drivers/acpi/ksyms.c has to be renamed
    to acpi_ksyms.c.

(2) Even with the rename, symbol generation for acpi fails because the
    genksyms pass does not honour EXTRA_CFLAGS.  The correct fix is to
    add EXTRA_CFLAGS to the genksyms pass in Rules.make.  However that
    hits every single compile and has with unknown side effects.  I do
    not have time to test a kernel wide change and at this late stage
    in the release cycle I am not going to risk it, especially since
    only acpi has a problem.  So this patch includes a bandaid to fix
    just acpi.  After 2.4.0 is released I will look at doing and
    verifying the correct fix.

Most of the patch is caused by the rename from ksyms.c to acpi_ksyms.c,
91 deletes and 91 inserts.

Index: 0-prerelease.1/drivers/acpi/Makefile
--- 0-prerelease.1/drivers/acpi/Makefile Fri, 22 Dec 2000 15:44:33 +1100 kaos (linux-2.4/Q/c/28_Makefile 1.2.3.1.2.2 644)
+++ 0-prerelease.1(w)/drivers/acpi/Makefile Wed, 03 Jan 2001 19:43:52 +1100 kaos (linux-2.4/Q/c/28_Makefile 1.2.3.1.2.2 644)
@@ -4,7 +4,7 @@
 
 O_TARGET := acpi.o
 
-export-objs := ksyms.o
+export-objs := acpi_ksyms.o
 
 export ACPI_CFLAGS
 ACPI_CFLAGS := -D_LINUX
@@ -20,18 +20,26 @@
 
 EXTRA_CFLAGS += $(ACPI_CFLAGS)
 
+# genksyms only reads $(CFLAGS), it should really read $(EXTRA_CFLAGS) as well.
+# Without EXTRA_CFLAGS the gcc pass for genksyms fails, resulting in an empty
+# include/linux/modules/acpi_ksyms.ver.  Changing genkyms to use EXTRA_CFLAGS
+# will hit everything, too risky in 2.4.0-prerelease.  Bandaid by tweaking
+# CFLAGS only for .ver targets.  Review after 2.4.0 release.  KAO
+
+$(MODINCL)/%.ver: CFLAGS = -I./include $(CFLAGS)
+
 acpi-subdirs := common dispatcher events hardware \
 		interpreter namespace parser resources tables
 
 subdir-$(CONFIG_ACPI) += $(acpi-subdirs)
 
 obj-$(CONFIG_ACPI) := $(patsubst %,%.o,$(acpi-subdirs))
-obj-$(CONFIG_ACPI) += os.o ksyms.o
+obj-$(CONFIG_ACPI) += os.o acpi_ksyms.o
 
 ifdef CONFIG_ACPI_KERNEL_CONFIG
   obj-$(CONFIG_ACPI) += acpiconf.o osconf.o
 else
-  obj-$(CONFIG_ACPI) += driver.o cmbatt.o cpu.o ec.o ksyms.o sys.o table.o power.o
+  obj-$(CONFIG_ACPI) += driver.o cmbatt.o cpu.o ec.o acpi_ksyms.o sys.o table.o power.o
 endif
 
 include $(TOPDIR)/Rules.make
Index: 0-prerelease.1(w)/drivers/acpi/ksyms.c
--- 0-prerelease.1/drivers/acpi/ksyms.c Sat, 30 Dec 2000 13:21:14 +1100 kaos (linux-2.4/o/d/49_ksyms.c 1.2 644)
+++ 0-prerelease.1(w)/drivers/acpi/ksyms.c Wed, 03 Jan 2001 20:05:59 +1100 kaos ()
@@ -1,91 +0,0 @@
-/*
- *  ksyms.c - ACPI exported symbols
- *
- *  Copyright (C) 2000 Andrew Grover
- *
- *  This program is free software; you can redistribute it and/or modify
- *  it under the terms of the GNU General Public License as published by
- *  the Free Software Foundation; either version 2 of the License, or
- *  (at your option) any later version.
- *
- *  This program is distributed in the hope that it will be useful,
- *  but WITHOUT ANY WARRANTY; without even the implied warranty of
- *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- *  GNU General Public License for more details.
- *
- *  You should have received a copy of the GNU General Public License
- *  along with this program; if not, write to the Free Software
- *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
- */
-
-#include <linux/module.h>
-#include <linux/init.h>
-#include <linux/kernel.h>
-#include <linux/types.h>
-#include <linux/acpi.h>
-#include "acpi.h"
-#include "acdebug.h"
-
-extern int acpi_in_debugger;
-
-#define _COMPONENT	OS_DEPENDENT
-	MODULE_NAME	("symbols")
-
-#ifdef ENABLE_DEBUGGER
-EXPORT_SYMBOL(acpi_in_debugger);
-EXPORT_SYMBOL(acpi_db_user_commands);
-#endif
-
-EXPORT_SYMBOL(acpi_os_free);
-EXPORT_SYMBOL(acpi_os_breakpoint);
-EXPORT_SYMBOL(acpi_os_printf);
-EXPORT_SYMBOL(acpi_os_callocate);
-EXPORT_SYMBOL(acpi_os_sleep);
-EXPORT_SYMBOL(acpi_os_sleep_usec);
-EXPORT_SYMBOL(acpi_os_in8);
-EXPORT_SYMBOL(acpi_os_out8);
-EXPORT_SYMBOL(acpi_os_queue_for_execution);
-
-EXPORT_SYMBOL(acpi_dbg_layer);
-EXPORT_SYMBOL(acpi_dbg_level);
-EXPORT_SYMBOL(function_exit);
-EXPORT_SYMBOL(function_trace);
-EXPORT_SYMBOL(function_status_exit);
-EXPORT_SYMBOL(function_value_exit);
-EXPORT_SYMBOL(debug_print_raw);
-EXPORT_SYMBOL(debug_print_prefix);
-
-EXPORT_SYMBOL(acpi_cm_strncmp);
-EXPORT_SYMBOL(acpi_cm_memcpy);
-EXPORT_SYMBOL(acpi_cm_memset);
-
-EXPORT_SYMBOL(acpi_get_handle);
-EXPORT_SYMBOL(acpi_get_parent);
-EXPORT_SYMBOL(acpi_get_type);
-EXPORT_SYMBOL(acpi_get_name);
-EXPORT_SYMBOL(acpi_get_object_info);
-EXPORT_SYMBOL(acpi_get_next_object);
-EXPORT_SYMBOL(acpi_evaluate_object);
-
-EXPORT_SYMBOL(acpi_install_notify_handler);
-EXPORT_SYMBOL(acpi_remove_notify_handler);
-EXPORT_SYMBOL(acpi_install_gpe_handler);
-EXPORT_SYMBOL(acpi_remove_gpe_handler);
-EXPORT_SYMBOL(acpi_install_address_space_handler);
-EXPORT_SYMBOL(acpi_remove_address_space_handler);
-
-EXPORT_SYMBOL(acpi_get_current_resources);
-EXPORT_SYMBOL(acpi_get_possible_resources);
-EXPORT_SYMBOL(acpi_set_current_resources);
-
-EXPORT_SYMBOL(acpi_enable_event);
-EXPORT_SYMBOL(acpi_disable_event);
-EXPORT_SYMBOL(acpi_clear_event);
-
-EXPORT_SYMBOL(acpi_get_processor_throttling_info);
-EXPORT_SYMBOL(acpi_get_processor_throttling_state);
-EXPORT_SYMBOL(acpi_set_processor_throttling_state);
-
-EXPORT_SYMBOL(acpi_get_processor_cx_info);
-EXPORT_SYMBOL(acpi_set_processor_sleep_state);
-EXPORT_SYMBOL(acpi_processor_sleep);
Index: 0-prerelease.1/drivers/acpi/acpi_ksyms.c
--- 0-prerelease.1/drivers/acpi/acpi_ksyms.c Wed, 03 Jan 2001 20:05:59 +1100 kaos ()
+++ 0-prerelease.1(w)/drivers/acpi/acpi_ksyms.c Sat, 30 Dec 2000 13:21:14 +1100 kaos (linux-2.4/t/d/19_acpi_ksyms  644)
@@ -0,0 +1,91 @@
+/*
+ *  ksyms.c - ACPI exported symbols
+ *
+ *  Copyright (C) 2000 Andrew Grover
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; either version 2 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program; if not, write to the Free Software
+ *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+ */
+
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/types.h>
+#include <linux/acpi.h>
+#include "acpi.h"
+#include "acdebug.h"
+
+extern int acpi_in_debugger;
+
+#define _COMPONENT	OS_DEPENDENT
+	MODULE_NAME	("symbols")
+
+#ifdef ENABLE_DEBUGGER
+EXPORT_SYMBOL(acpi_in_debugger);
+EXPORT_SYMBOL(acpi_db_user_commands);
+#endif
+
+EXPORT_SYMBOL(acpi_os_free);
+EXPORT_SYMBOL(acpi_os_breakpoint);
+EXPORT_SYMBOL(acpi_os_printf);
+EXPORT_SYMBOL(acpi_os_callocate);
+EXPORT_SYMBOL(acpi_os_sleep);
+EXPORT_SYMBOL(acpi_os_sleep_usec);
+EXPORT_SYMBOL(acpi_os_in8);
+EXPORT_SYMBOL(acpi_os_out8);
+EXPORT_SYMBOL(acpi_os_queue_for_execution);
+
+EXPORT_SYMBOL(acpi_dbg_layer);
+EXPORT_SYMBOL(acpi_dbg_level);
+EXPORT_SYMBOL(function_exit);
+EXPORT_SYMBOL(function_trace);
+EXPORT_SYMBOL(function_status_exit);
+EXPORT_SYMBOL(function_value_exit);
+EXPORT_SYMBOL(debug_print_raw);
+EXPORT_SYMBOL(debug_print_prefix);
+
+EXPORT_SYMBOL(acpi_cm_strncmp);
+EXPORT_SYMBOL(acpi_cm_memcpy);
+EXPORT_SYMBOL(acpi_cm_memset);
+
+EXPORT_SYMBOL(acpi_get_handle);
+EXPORT_SYMBOL(acpi_get_parent);
+EXPORT_SYMBOL(acpi_get_type);
+EXPORT_SYMBOL(acpi_get_name);
+EXPORT_SYMBOL(acpi_get_object_info);
+EXPORT_SYMBOL(acpi_get_next_object);
+EXPORT_SYMBOL(acpi_evaluate_object);
+
+EXPORT_SYMBOL(acpi_install_notify_handler);
+EXPORT_SYMBOL(acpi_remove_notify_handler);
+EXPORT_SYMBOL(acpi_install_gpe_handler);
+EXPORT_SYMBOL(acpi_remove_gpe_handler);
+EXPORT_SYMBOL(acpi_install_address_space_handler);
+EXPORT_SYMBOL(acpi_remove_address_space_handler);
+
+EXPORT_SYMBOL(acpi_get_current_resources);
+EXPORT_SYMBOL(acpi_get_possible_resources);
+EXPORT_SYMBOL(acpi_set_current_resources);
+
+EXPORT_SYMBOL(acpi_enable_event);
+EXPORT_SYMBOL(acpi_disable_event);
+EXPORT_SYMBOL(acpi_clear_event);
+
+EXPORT_SYMBOL(acpi_get_processor_throttling_info);
+EXPORT_SYMBOL(acpi_get_processor_throttling_state);
+EXPORT_SYMBOL(acpi_set_processor_throttling_state);
+
+EXPORT_SYMBOL(acpi_get_processor_cx_info);
+EXPORT_SYMBOL(acpi_set_processor_sleep_state);
+EXPORT_SYMBOL(acpi_processor_sleep);

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Oops in prune_dcache (2.4.0-prerelease)

[Alexander Viro]

On Wed, 3 Jan 2001, Dan Aloni wrote:

> After a bit of few code reviewing, it looks like the only code that
> assigns stuff to ->d_op in a nonstandard way is in fs/vfat/namei.c. 
> 
> Udo, are you using vfat?

	If it was assigned by something that was supposed to set ->d_op
it would not get such value. Whatever had done that had no idea of the
->d_op or struct dentry in the first place.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Oops in prune_dcache (2.4.0-prerelease)

[Udo A. Steinberg]

Dan Aloni wrote:
> 
> After a bit of few code reviewing, it looks like the only code that
> assigns stuff to ->d_op in a nonstandard way is in fs/vfat/namei.c.
> 
> Udo, are you using vfat?

Yes.

-Udo.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Oops in prune_dcache (2.4.0-prerelease)

[Alexander Viro]

On Wed, 3 Jan 2001, Udo A. Steinberg wrote:

> Dan Aloni wrote:
> > 
> > After a bit of few code reviewing, it looks like the only code that
> > assigns stuff to ->d_op in a nonstandard way is in fs/vfat/namei.c.
> > 
> > Udo, are you using vfat?
> 
> Yes.

In principle, it might be that d_find_alias() is broken. I don't see where
it could happen, but then I'm half-asleep right now...  While we are at it,
do you have
	* autofs
	* knfsd
	* ncpfs

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Oops in prune_dcache (2.4.0-prerelease)

[Udo A. Steinberg]

Hi,

Alexander Viro wrote:
>
> In principle, it might be that d_find_alias() is broken. I don't see where
> it could happen, but then I'm half-asleep right now...  While we are at it,
> do you have

>         * autofs

Yes.

>         * knfsd
>         * ncpfs

No, neither of these two.

-Udo.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/