Two 2.6.20-rc5-rt2 issues

Two 2.6.20-rc5-rt2 issues

[Rui Nuno Capela]

[-- Attachment #1: Type: text/plain, Size: 3479 bytes --]

Hi,

Sorry to interrupt :) I have two issues to report here, while running
2.6.20-rc5-rt2.

First one is about building for UP (CONFIG_SMP not set) on my old P4
laptop. As it seems, all my build attempts failed at the final link stage,
with undefined references to paravirt_enable. After disabling
CONFIG_PARAVIRT I get a similar failure, but this time for a couple kvm*
symbols. I could only get a clean build when CONFIG_SMP is set, which is
(IMHO) overkill for a machine which is neither HyperThread/SMT enabled nor
multi-core. Its plain dead UP and it used to run PREEMPT_RT kernels for a
long time now.

Second one is already about running SMP, on a Dual Core2 T7200, for which
the build goes fine but run-time is haunted by a crippling BUG:

BUG: unable to handle kernel NULL pointer dereference at virtual address
0000001c
 printing eip:
c01181a5
*pde = 2e68e067
stopped custom tracer.
Oops: 0000 [#1]
PREEMPT SMP
Modules linked in: arc4 ecb blkcipher ieee80211_crypt_wep af_packet i915
drm rfcomm hidp l2cap bluetooth cpufreq_conservative cpufreq_ondemand
cpufreq_userspace cpufreq_powersave speedstep_centrino freq_table
snd_pcm_oss snd_mixer_oss snd_seq snd_seq_device button battery ac loop
dm_mod ohci1394 ieee1394 ipw3945 e100 mii sdhci ieee80211 mmc_core
ieee80211_crypt firmware_class ehci_hcd ide_cd snd_hda_intel snd_hda_codec
snd_pcm snd_timer i2c_i801 snd iTCO_wdt cdrom soundcore snd_page_alloc
intel_agp i2c_core agpgart uhci_hcd usbcore ext3 mbcache jbd sg edd fan
ahci libata piix thermal processor sd_mod scsi_mod ide_disk ide_core
CPU:    0
EIP:    0060:[<c01181a5>]    Not tainted VLI
EFLAGS: 00010282   (2.6.20-rc5-rt2.0-smp #1)
EIP is at try_to_wake_up+0x11/0x395
eax: 00000000   ebx: dfc7f5c0   ecx: 00000000   edx: dfc5fee0
esi: 00000000   edi: dfc7fb76   ebp: dfc5fef0   esp: dfc5fea4
ds: 007b   es: 007b   ss: 0068   preempt: 00000001
Process IRQ-9 (pid: 56, ti=dfc5e000 task=dfc541f0 task.ti=dfc5e000)
Stack: 00003000 00000061 0000001f 00000100 00000000 00000000 00000000
00000000
       00000002 00000001 00000000 00000000 dfc541f0 c03764c0 dfc541f0
c0102dad
       dfc7f5c0 00000017 dfc7fb76 dfc5fefc c01185c8 00000000 c19faa40
c01fe568
Call Trace:
 [<c0102dad>] __switch_to+0xcc/0x176
 [<c01185c8>] wake_up_process+0x19/0x1b
 [<c01fe568>] acpi_ec_gpe_handler+0x1f/0x53
 [<c01ec6c6>] acpi_ev_gpe_dispatch+0x64/0x163
 [<c01eca06>] acpi_ev_gpe_detect+0x94/0xd7
 [<c01eb182>] acpi_ev_sci_xrupt_handler+0x12/0x19
 [<c01e6091>] acpi_irq+0xb/0x14
 [<c0142cf5>] handle_IRQ_event+0x4a/0xd2
 [<c0143188>] thread_simple_irq+0x31/0x5f
 [<c0143820>] do_irqd+0xe2/0x29e
 [<c014373e>] do_irqd+0x0/0x29e
 [<c012e2e3>] kthread+0xb2/0xdb
 [<c012e231>] kthread+0x0/0xdb
 [<c0104b13>] kernel_thread_helper+0x7/0x10
 =======================
Code: 8b 5d f0 89 4f 40 89 5f 44 8b 55 e8 89 f8 83 c4 0c 5b 5e 5f 5d e9 24
e1 ff ff 55 89 e5 57 56 89 c6 53 83 ec 40 89 55 bc 8d 55 f0 <83> 78 1c 63
b8 00 00 00 00 0f 4f c1 89 45 b8 89 f0 e8 64 e1 ff
EIP: [<c01181a5>] try_to_wake_up+0x11/0x395 SS:ESP 0068:dfc5fea4
 <3>hub 5-0:1.0: connect-debounce failed, port 6 disabled


Complete dmesg is attached.

I think this one is specific to this hardware combination, most probably
to some ACPI idiosyncrasy. Note that the system only gets partially
unresponsive but to my knowledge the only reasonable action to take on
this situation is forcing a reboot.

Any ideas on where to look would be much appreciated, as always.

Cheers.
-- 
rncbc aka Rui Nuno Capela
rncbc@rncbc.org

[-- Attachment #2: dmesg-2.6.20-rc5-rt2.0-smp.0.gz --]
[-- Type: application/x-gzip-compressed, Size: 8634 bytes --]

Re: Two 2.6.20-rc5-rt2 issues

[Ingo Molnar]

* Rui Nuno Capela <rncbc@rncbc.org> wrote:

> First one is about building for UP (CONFIG_SMP not set) on my old P4 
> laptop. As it seems, all my build attempts failed at the final link 
> stage, with undefined references to paravirt_enable. After disabling 
> CONFIG_PARAVIRT I get a similar failure, but this time for a couple 
> kvm* symbols. [...]

ok, i think i have managed to fix both bugs. I have released -rt3, 
please re-check whether it works any better. If it still doesnt then 
please send me the exact .config that fails.

> [...] I could only get a clean build when CONFIG_SMP is set, which is 
> (IMHO) overkill for a machine which is neither HyperThread/SMT enabled 
> nor multi-core. Its plain dead UP and it used to run PREEMPT_RT 
> kernels for a long time now.

btw., latest SMP kernels (2.6.18 and later) "patch themselves back" to 
UP instructions to a high degree if they run on a single CPU - that's 
why for example Fedora uses only an SMP kernel these days. Running a 
genuine UP kernel is still more efficient - but the difference shouldnt 
be /that/ large anymore. (if someone would like to measure it that would 
be interesting to see)

> Second one is already about running SMP, on a Dual Core2 T7200, for 
> which the build goes fine but run-time is haunted by a crippling BUG:

> Call Trace:
>  [<c0102dad>] __switch_to+0xcc/0x176
>  [<c01185c8>] wake_up_process+0x19/0x1b
>  [<c01fe568>] acpi_ec_gpe_handler+0x1f/0x53
>  [<c01ec6c6>] acpi_ev_gpe_dispatch+0x64/0x163
>  [<c01eca06>] acpi_ev_gpe_detect+0x94/0xd7

hm, this is a -rt specific thing that i hoped to have worked around but 
apparently not. The ACPI code uses a waitqueue in its idle routine 
(argh!) which cannot by done sanely on PREEMPT_RT. In -rt3 i've added a 
more conservative (but still ugly and incorrect) hack - could you try 
it, does -rt3 work any better?

	Ingo

Re: Two 2.6.20-rc5-rt2 issues

[Rui Nuno Capela]

[-- Attachment #1: Type: text/plain, Size: 1933 bytes --]

On Tue, January 16, 2007 11:56, Ingo Molnar wrote:
>

> * Rui Nuno Capela <rncbc@rncbc.org> wrote:
>
>> First one is about building for UP (CONFIG_SMP not set) on my old P4
>> laptop. As it seems, all my build attempts failed at the final link
>> stage, with undefined references to paravirt_enable. After disabling
>> CONFIG_PARAVIRT I get a similar failure, but this time for a couple
>> kvm* symbols. [...]
>
> ok, i think i have managed to fix both bugs. I have released -rt3,
> please re-check whether it works any better. If it still doesnt then
> please send me the exact .config that fails.
>

Building this already with -rt5, still gives:
...
  LD      arch/i386/boot/compressed/vmlinux
  OBJCOPY arch/i386/boot/vmlinux.bin
  BUILD   arch/i386/boot/bzImage
Root device is (3, 2)
Boot sector 512 bytes.
Setup is 7407 bytes.
System is 1427 kB
Kernel: arch/i386/boot/bzImage is ready  (#1)
WARNING: "profile_hits" [drivers/kvm/kvm-intel.ko] undefined!
WARNING: "profile_hits" [drivers/kvm/kvm-amd.ko] undefined!
make[1]: *** [__modpost] Error 1
make: *** [modules] Error 2
...

.config \x7fas .gz is attached.


>
>> Second one is already about running SMP, on a Dual Core2 T7200, for
>> which the build goes fine but run-time is haunted by a crippling BUG:
>
>> Call Trace:
>> [<c0102dad>] __switch_to+0xcc/0x176
>> [<c01185c8>] wake_up_process+0x19/0x1b
>> [<c01fe568>] acpi_ec_gpe_handler+0x1f/0x53
>> [<c01ec6c6>] acpi_ev_gpe_dispatch+0x64/0x163
>> [<c01eca06>] acpi_ev_gpe_detect+0x94/0xd7
>>
>
> hm, this is a -rt specific thing that i hoped to have worked around but
> apparently not. The ACPI code uses a waitqueue in its idle routine
> (argh!) which cannot by done sanely on PREEMPT_RT. In -rt3 i've added
> a more conservative (but still ugly and incorrect) hack - could you try
> it, does -rt3 work any better?
>

Yes it does. No BUG has been spotted on -rt3, yet.

Cheers.
-- 
rncbc aka Rui Nuno Capela
rncbc@rncbc.org

[-- Attachment #2: config.gz --]
[-- Type: application/x-gzip, Size: 20296 bytes --]

Re: Two 2.6.20-rc5-rt2 issues

[Ingo Molnar]

* Rui Nuno Capela <rncbc@rncbc.org> wrote:

> Building this already with -rt5, still gives:
> ...
>   LD      arch/i386/boot/compressed/vmlinux
>   OBJCOPY arch/i386/boot/vmlinux.bin
>   BUILD   arch/i386/boot/bzImage
> Root device is (3, 2)
> Boot sector 512 bytes.
> Setup is 7407 bytes.
> System is 1427 kB
> Kernel: arch/i386/boot/bzImage is ready  (#1)
> WARNING: "profile_hits" [drivers/kvm/kvm-intel.ko] undefined!
> WARNING: "profile_hits" [drivers/kvm/kvm-amd.ko] undefined!

ok - in my test-config i didnt have KVM modular - the patch below should 
fix this problem.

	Ingo

Index: linux/kernel/profile.c
===================================================================
--- linux.orig/kernel/profile.c
+++ linux/kernel/profile.c
@@ -332,7 +332,6 @@ out:
 	local_irq_restore(flags);
 	put_cpu();
 }
-EXPORT_SYMBOL_GPL(profile_hits);
 
 static int __devinit profile_cpu_callback(struct notifier_block *info,
 					unsigned long action, void *__cpu)
@@ -402,6 +401,8 @@ void profile_hits(int type, void *__pc, 
 }
 #endif /* !CONFIG_SMP */
 
+EXPORT_SYMBOL_GPL(profile_hits);
+
 void __profile_tick(int type, struct pt_regs *regs)
 {
 	if (type == CPU_PROFILING && timer_hook)

Re: Two 2.6.20-rc5-rt2 issues

[Rui Nuno Capela]

On Wed, January 17, 2007 06:31, Ingo Molnar wrote:
>
> * Rui Nuno Capela <rncbc@rncbc.org> wrote:
>
>> Building this already with -rt5, still gives:
>> ...
>> LD      arch/i386/boot/compressed/vmlinux
>> OBJCOPY arch/i386/boot/vmlinux.bin
>> BUILD   arch/i386/boot/bzImage
>> Root device is (3, 2)
>> Boot sector 512 bytes.
>> Setup is 7407 bytes.
>> System is 1427 kB
>> Kernel: arch/i386/boot/bzImage is ready  (#1)
>> WARNING: "profile_hits" [drivers/kvm/kvm-intel.ko] undefined!
>> WARNING: "profile_hits" [drivers/kvm/kvm-amd.ko] undefined!
>>
>
> ok - in my test-config i didnt have KVM modular - the patch below should
> fix this problem.
>
> Ingo
>
>
> Index: linux/kernel/profile.c
> ===================================================================
> --- linux.orig/kernel/profile.c
> +++ linux/kernel/profile.c
> @@ -332,7 +332,6 @@ out:
> local_irq_restore(flags); put_cpu(); }
> -EXPORT_SYMBOL_GPL(profile_hits);
>
>
> static int __devinit profile_cpu_callback(struct notifier_block *info,
> unsigned long action, void *__cpu) @@ -402,6 +401,8 @@ void
> profile_hits(int type, void *__pc, }
> #endif /* !CONFIG_SMP */
>
>
> +EXPORT_SYMBOL_GPL(profile_hits);
> +
> void __profile_tick(int type, struct pt_regs *regs) {
> if (type == CPU_PROFILING && timer_hook)
>

OK, now it builds alright.

Both issues seem to be fixed now, thanks.

Bye.
-- 
rncbc aka Rui Nuno Capela
rncbc@rncbc.org

compilation error

[Alessio Igor Bogani]

Hi,

I have this error trying to compile a minimal 2.6.20-rc5-rt7 i386
kernel:

[...]
  CHK     include/linux/compile.h
  UPD     include/linux/compile.h
  CC      init/version.o
  LD      init/built-in.o
  LD      .tmp_vmlinux1
arch/i386/kernel/built-in.o: In function `init_intel':
intel.c:(.init.text+0x3c01): undefined reference to `paravirt_enabled'
make: *** [.tmp_vmlinux1] Error 1

With this trivial patch i can compile kernel successfully:

--- intel.c.orig	2007-01-22 11:03:06.000000000 +0100
+++ intel.c	2007-01-22 11:03:36.000000000 +0100
@@ -6,6 +6,7 @@
 #include <linux/smp.h>
 #include <linux/thread_info.h>
 #include <linux/module.h>
+#include <linux/paravirt.h>
 
 #include <asm/processor.h>
 #include <asm/msr.h>


But i don't know if it is the best way to resolve this problem.

Ciao,
Alessio

Re: compilation error

[Ingo Molnar]

* Alessio Igor Bogani <abogani@texware.it> wrote:

> With this trivial patch i can compile kernel successfully:
> 
> --- intel.c.orig	2007-01-22 11:03:06.000000000 +0100
> +++ intel.c	2007-01-22 11:03:36.000000000 +0100
> @@ -6,6 +6,7 @@
>  #include <linux/smp.h>
>  #include <linux/thread_info.h>
>  #include <linux/module.h>
> +#include <linux/paravirt.h>
>  
>  #include <asm/processor.h>
>  #include <asm/msr.h>
> 
> But i don't know if it is the best way to resolve this problem.

that's the best way i think - i've applied your patch.

	Ingo