Re: GCC4 fun.

Re: GCC4 fun.

[Joerg Dorchain]

[-- Attachment #1: Type: text/plain, Size: 2109 bytes --]

Hi list,

I just booted a self-compiled 2.6.11 on my iBook G4.

The dmesg shows a hickup, where I am wondering what's the cause.

Total memory = 768MB; using 2048kB for hash table (at c0400000)
Linux version 2.6.11 (root@whisker) (gcc version 3.3.5 (Debian 1:3.3.5-8)) #29 Wed Mar 2 20:15:15 CET 2005
Found UniNorth memory controller & host bridge, revision: 210
Mapped at 0xfdf00000
Found a Intrepid mac-io controller, rev: 0, mapped at 0xfde80000
Processor NAP mode on idle enabled.
PowerMac motherboard: iBook G4
Found UniNorth PCI host bridge at 0xf0000000. Firmware bus number: 0->0
Found UniNorth PCI host bridge at 0xf2000000. Firmware bus number: 0->0
Found UniNorth PCI host bridge at 0xf4000000. Firmware bus number: 0->0
via-pmu: Server Mode is disabled
PMU driver 2 initialized for Core99, firmware: 0c
On node 0 totalpages: 196608
  DMA zone: 196608 pages, LIFO batch:16
  Normal zone: 0 pages, LIFO batch:1
  HighMem zone: 0 pages, LIFO batch:1
Built 1 zonelists
Kernel command line: root=/dev/hda5 ro 
PowerMac using OpenPIC irq controller at 0x80040000
OpenPIC Version 1.2 (4 CPUs and 64 IRQ sources) at fc633000
OpenPIC timer frequency is 4.166666 MHz
PID hash table entries: 4096 (order: 12, 65536 bytes)
time_init: decrementer frequency = 18.432000 MHz
Console: colour dummy device 80x25
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Memory: 773632k available (1928k kernel code, 1044k data, 144k init, 0k highmem)
AGP special page: 0xeffff000
Calibrating delay loop... 663.55 BogoMIPS (lpj=331776)
Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
scheduling while atomic: swapper/0x00000002/0
Call trace:
 [c0007620] dump_stack+0x18/0x28
 [c01de704] schedule+0x678/0x67c
 [c0004500] syscall_exit_work+0x108/0x10c
 [c02a97b4] proc_root_init+0x168/0x174
 [ff847288] 0xff847288
 [c02945e8] start_kernel+0x144/0x170
 [00003a30] 0x3a30

Anyone an idea where this comes from? (Or, what else is needed for a
qualified statement about it?)

TIA,

Joerg

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: GCC4 fun.

[Marc Leeman]

[-- Attachment #1: Type: text/plain, Size: 713 bytes --]

> scheduling while atomic: swapper/0x00000002/0
> Call trace:
>  [c0007620] dump_stack+0x18/0x28
>  [c01de704] schedule+0x678/0x67c
>  [c0004500] syscall_exit_work+0x108/0x10c
>  [c02a97b4] proc_root_init+0x168/0x174
>  [ff847288] 0xff847288
>  [c02945e8] start_kernel+0x144/0x170
>  [00003a30] 0x3a30

We have something similar with PPC 8245, I was planning to have a look
at it, but didn't get round to it. Since our system is crunching numbers
since yesterday evening without fault, I guess it's not that serious...

-- 
  greetz, marc
Aeryn died so you could live John. She would want you to keep fighting.
	D'Argo - Season of Death
scorpius 2.6.11 #1 Wed Mar 2 16:38:38 CET 2005 GNU/Linux

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: GCC4 fun.

[evilninja]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Joerg Dorchain wrote:
> Hi list,
> 
> I just booted a self-compiled 2.6.11 on my iBook G4.
> 
> The dmesg shows a hickup, where I am wondering what's the cause.
> 
> Total memory = 768MB; using 2048kB for hash table (at c0400000)
> Linux version 2.6.11 (root@whisker) (gcc version 3.3.5 (Debian 1:3.3.5-8))
- ---------------------------------------------------^

what does this have to do with your subject? ("GCC4 fun.")

> scheduling while atomic: swapper/0x00000002/0
> Call trace:
>  [c0007620] dump_stack+0x18/0x28
>  [c01de704] schedule+0x678/0x67c
>  [c0004500] syscall_exit_work+0x108/0x10c
>  [c02a97b4] proc_root_init+0x168/0x174
>  [ff847288] 0xff847288
>  [c02945e8] start_kernel+0x144/0x170
>  [00003a30] 0x3a30

did you try the various -rc kernels before 2.6.11? did it happen with
these too?

i don't know why you're getting this error, but seeing "swapper" to be
involved here: can you compile with CONFIG_SWAP=n and boot this one?

Christian.
- --
BOFH excuse #440:

Cache miss - please take better aim next time
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCJxU8C/PVm5+NVoYRAkYTAKCqSqJyhy2yGYPEG6ZQWs3oaqI7KQCfeGCW
rfBeUbJ5EhS4Fx0L5kvh4a8=
=vIIE
-----END PGP SIGNATURE-----

Re: boot time scheduling hile atomic (was: GCC4 fun.)

[Joerg Dorchain]

[-- Attachment #1: Type: text/plain, Size: 833 bytes --]

On Thu, Mar 03, 2005 at 02:46:36PM +0100, evilninja wrote:
> 
> what does this have to do with your subject? ("GCC4 fun.")

My stupidity. Mixed up threads.
> 
> > scheduling while atomic: swapper/0x00000002/0
> > Call trace:
> >  [c0007620] dump_stack+0x18/0x28
> >  [c01de704] schedule+0x678/0x67c
> >  [c0004500] syscall_exit_work+0x108/0x10c
> >  [c02a97b4] proc_root_init+0x168/0x174
> >  [ff847288] 0xff847288
> >  [c02945e8] start_kernel+0x144/0x170
> >  [00003a30] 0x3a30
> 
> did you try the various -rc kernels before 2.6.11? did it happen with
> these too?

No. Unfortunately, my mac has no regular internet connection.
> 
> i don't know why you're getting this error, but seeing "swapper" to be
> involved here: can you compile with CONFIG_SWAP=n and boot this one?

I'll do tonight.


Joerg


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: boot time scheduling hile atomic (was: GCC4 fun.)

[Joerg Dorchain]

[-- Attachment #1: Type: text/plain, Size: 832 bytes --]

On Thu, Mar 03, 2005 at 03:18:32PM +0100, Joerg Dorchain wrote:
> > 
> > > scheduling while atomic: swapper/0x00000002/0
> > > Call trace:
> > >  [c0007620] dump_stack+0x18/0x28
> > >  [c01de704] schedule+0x678/0x67c
> > >  [c0004500] syscall_exit_work+0x108/0x10c
> > >  [c02a97b4] proc_root_init+0x168/0x174
> > >  [ff847288] 0xff847288
> > >  [c02945e8] start_kernel+0x144/0x170
> > >  [00003a30] 0x3a30
> > 
> > i don't know why you're getting this error, but seeing "swapper" to be
> > involved here: can you compile with CONFIG_SWAP=n and boot this one?
> 
> I'll do tonight.

Did so, no effect on this. IMHO "swapper" is a somewhat misguiding name
for the idle task with pid 0.

I am more wondering what in proc_root_init or immediately after it
causes a call to the exit_work syscall.

Bye,

Joerg

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: boot time scheduling hile atomic

[_evil]

Joerg Dorchain wrote:
>>>
>>>i don't know why you're getting this error, but seeing "swapper" to be
>>>involved here: can you compile with CONFIG_SWAP=n and boot this one?
>>
>>I'll do tonight.
> 
> Did so, no effect on this. IMHO "swapper" is a somewhat misguiding name
> for the idle task with pid 0.

yes, i just guessed. sadly, it did not help :-\
and i can't decode the backtrace either.

does the system halt after the message? if not, we a re not in a hurry 
and if you have time you could possbily track it down a bit to 1) the 
kernelversion when it happened first and/or 2) find the .config option, 
that causes this error.

if the box halts after this, and no guru from linuxppc-dev is available, 
maybe you should try and post on linux-kernel too.

just my 2c,
Christian.

Re: boot time scheduling hile atomic

[Takeharu KATO]

Hi

Joerg Dorchain wrote:
> On Thu, Mar 03, 2005 at 03:18:32PM +0100, Joerg Dorchain wrote:
> 
>>>>scheduling while atomic: swapper/0x00000002/0
>>>>Call trace:
>>>> [c0007620] dump_stack+0x18/0x28
>>>> [c01de704] schedule+0x678/0x67c
>>>> [c0004500] syscall_exit_work+0x108/0x10c
>>>> [c02a97b4] proc_root_init+0x168/0x174
>>>> [ff847288] 0xff847288
>>>> [c02945e8] start_kernel+0x144/0x170
>>>> [00003a30] 0x3a30
>>
>>I'll do tonight.
> 
> I am more wondering what in proc_root_init or immediately after it
> causes a call to the exit_work syscall.
> 
This is not called by proc_root_init.
It seems be called by timer interrupt
(I confirmed it by viewing trap number in
interrupt context with ICE).

As long as I investigate the problem,
proc_root_init is called twice.
At first time, it's called from
start_kernel in init/main.c.
Second case, it seems to be called by some kind of
init call facility.

proc_root_init is not init_call function, I could not
figure out concretely why it called twice.

I will investigate the problem as long as I can.

I wishes this report to become your help.

Regards,
-- 
Takeharu KATO
Fujitsu Limited
Email:kato.takeharu at jp.fujitsu.com

Re: boot time scheduling hile atomic

[Joerg Dorchain]

[-- Attachment #1: Type: text/plain, Size: 586 bytes --]

On Thu, Mar 10, 2005 at 01:51:27PM +0900, Takeharu KATO wrote:
> > I am more wondering what in proc_root_init or immediately after it
> > causes a call to the exit_work syscall.
> > 
> This is not called by proc_root_init.
> It seems be called by timer interrupt
> (I confirmed it by viewing trap number in
> interrupt context with ICE).

I came to the same idea. This explains as well why the i386 party does
not notice the effect, as their first interrupt happen later (different
HZ value)

At this point, I was out of clue, so thank you for your help.

Bye,

Joerg

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: boot time scheduling hile atomic

[Takeharu KATO]

Hi Joerg:

> I came to the same idea. This explains as well why the i386 party does
> not notice the effect, as their first interrupt happen later (different
> HZ value)
> 
As far as I know, current HZ value for PPC is 1000
as same as i386 family.

Please see following:

-- include/asm-i386/param.h
   1 #ifndef _ASMi386_PARAM_H
   2 #define _ASMi386_PARAM_H
   3
   4 #ifdef __KERNEL__
   5 # define HZ             1000            /* Internal kernel timer 
frequency */
-- include/asm-i386/param.h

-- include/asm-ppc/param.h

   1 #ifndef _ASM_PPC_PARAM_H
   2 #define _ASM_PPC_PARAM_H
   3
   4 #ifdef __KERNEL__
   5 #define HZ              1000            /* internal timer frequency */
   6 #define USER_HZ         100             /* for user interfaces in 
"ticks" */
-- include/asm-ppc/param.h

FYI, Kernel preemption ought not to be taken place at this point,
because preemption count is a positive number.

According to my observation, the problem occur on second call of
proc_root_init.But I could not who call it.
IIRC,the value of LR(link register) pointed at the start of BSS.

Regards,



-- 
Takeharu KATO

Re: boot time scheduling hile atomic

[Takeharu KATO]

Hi Joerg:

First of all, I am sorry to making mistake on my stack analysis.
After all, I found what is wrong.

The syscall_exit_work is called(jumped) by fork, strictly speaking,
this is caused by kernel_thead for init process.
At this point, kernel set a need_resched flag in thread_info struct
(flags member).
So, the system executes syscall_exit_work when
the swapper thread exited from syscall.

However, the syscall_exit_work routine does not check
preempt counts when it calls schedule function.

FYI, I post the patch which fix this problem.
I've confirmed the problem has been removed by this patch,
but I can not figure out whether it is truely
correct or not.

Please review this.

Signed-off-by: Takeharu KATO <kato.takeharu@jp.fujitsu.com>
--- linux-2.6.11/arch/ppc/kernel/entry.S        2005-03-09 12:57:32.000000000 +0900
+++ linux-2.6.11-wdt/arch/ppc/kernel/entry.S    2005-03-11 14:39:16.345490784 +0900
@@ -314,10 +314,16 @@ syscall_exit_work:
         SYNC
         MTMSRD(r10)             /* disable interrupts again */
         rlwinm  r12,r1,0,0,18   /* current_thread_info() */
-       lwz     r9,TI_FLAGS(r12)
  5:
+#ifdef CONFIG_PREEMPT
+       lwz     r0,TI_PREEMPT(r12)
+       cmpwi   0,r0,0          /* if non-zero, we can not reschedule here */
+       bne     6f
+#endif
+       lwz     r9,TI_FLAGS(r12)
         andi.   r0,r9,_TIF_NEED_RESCHED
         bne     1f
+6:
         lwz     r5,_MSR(r1)
         andi.   r5,r5,MSR_PR
         beq     syscall_exit_cont

-- 
Takeharu KATO
Fujitsu Limited
Email:kato.takeharu at jp.fujitsu.com

Re: boot time scheduling while atomic

[Takeharu KATO]

Hi

> 
> I came to the same idea. This explains as well why the i386 party does
> not notice the effect, as their first interrupt happen later (different
> HZ value)
> 

FYI, i386 party's kernel_thread function calls do_fork directly, 
however, do_fork is called via syscall trap(clone syscall) in powerpc.
So, this difference may be taken place.

IMHO, from this point of view, the real fix for this problem is change 
the way to implement kernel_thread function of PowerPC like as other 
architecture does.

Regards,


-- 
Takeharu KATO

Re: boot time scheduling hile atomic

[Benjamin Herrenschmidt]

On Fri, 2005-03-11 at 15:19 +0900, Takeharu KATO wrote:
> Hi Joerg:
> 
> First of all, I am sorry to making mistake on my stack analysis.
> After all, I found what is wrong.
> 
> The syscall_exit_work is called(jumped) by fork, strictly speaking,
> this is caused by kernel_thead for init process.
> At this point, kernel set a need_resched flag in thread_info struct
> (flags member).
> So, the system executes syscall_exit_work when
> the swapper thread exited from syscall.
> 
> However, the syscall_exit_work routine does not check
> preempt counts when it calls schedule function.
> 
> FYI, I post the patch which fix this problem.
> I've confirmed the problem has been removed by this patch,
> but I can not figure out whether it is truely
> correct or not.

I think the problem is more why are we calling a syscall while the
preempt count is up ?

Ben.

Re: boot time scheduling hile atomic

[Takeharu KATO]

Hi Benjamin:

Thank you for your response.

> I think the problem is more why are we calling a syscall while the
> preempt count is up ?
> 
Because, start_kernel(line:455) in init/main.c count up this.
-- start_kernel (init/main.c)
451         /*
452          * Disable preemption - early bootup scheduling is extremely
453          * fragile until we cpu_idle() for the first time.
454          */
455         preempt_disable();
456         build_all_zonelists();
-- start_kernel(init/main.c)

Preemption is disabled until it enters the body of idle task from rest_init.
Please see below:
-- rest_init (init/main.c)
379 static void noinline rest_init(void)
380         __releases(kernel_lock)
381 {
382         kernel_thread(init, NULL, CLONE_FS | CLONE_SIGHAND);
383         numa_default_policy();
384         unlock_kernel();
385         preempt_enable_no_resched();
386         cpu_idle();
387 }
--
I think that the kernel assumes that init thread(the kernel thread which become to
init process later) is kicked by idle process(cpu_idle).

But, the kernel invoke a system call trap when it execute kernel_thread call
(at rest_init line:382) on PowerPC.
The preemption count has positive value here.
So, PowerPC linux kernel invoke system call when the preempt count is up.

Regards,

-- 
Takeharu KATO
Fujitsu Limited
Email:kato.takeharu at jp.fujitsu.com

Re: boot time scheduling hile atomic

[Takeharu KATO]

Hi Benjamin:

I forgot writing that I've sent the revised edition of
the patch to the PPC-Dev ML.
Please see the patch as ``boot time scheduling w hile atomic fix''
which I posted in 2005-03-15 from Patch tracking
system for PowerPC Linux as follows.

http://ozlabs.org/ppc32-patches/

It changes the implementation of kernel_thread function
to be performed with function call.

Regards,

Takeharu KATO wrote:

> But, the kernel invoke a system call trap when it execute kernel_thread call
> (at rest_init line:382) on PowerPC.
> The preemption count has positive value here.
> So, PowerPC linux kernel invoke system call when the preempt count is up.
> 

-- 
Takeharu KATO
Fujitsu Limited
Email:kato.takeharu at jp.fujitsu.com

Re: boot time scheduling hile atomic

[Benjamin Herrenschmidt]

On Wed, 2005-03-16 at 14:49 +0900, Takeharu KATO wrote:
> Hi Benjamin:
> 
> I forgot writing that I've sent the revised edition of
> the patch to the PPC-Dev ML.
> Please see the patch as ``boot time scheduling w hile atomic fix''
> which I posted in 2005-03-15 from Patch tracking
> system for PowerPC Linux as follows.
> 
> http://ozlabs.org/ppc32-patches/
> 
> It changes the implementation of kernel_thread function
> to be performed with function call.

I prefer your previous patch. Paul hacked it a bit and will commit his
modified version.

Ben.

Re: boot time scheduling hile atomic

[Takeharu KATO]

Hi Benjamin:

I can not figure out why my previous patch is more preferable than
later one.
Could you explain the reason of that please.

Regards,

>>
>>http://ozlabs.org/ppc32-patches/
>>
>>It changes the implementation of kernel_thread function
>>to be performed with function call.
> 
> 
> I prefer your previous patch. Paul hacked it a bit and will commit his
> modified version.
> 
> Ben.
> 

-- 
Takeharu KATO

Re: boot time scheduling hile atomic

[Benjamin Herrenschmidt]

On Wed, 2005-03-16 at 23:07 +0900, Takeharu KATO wrote:
> Hi Benjamin:
> 
> I can not figure out why my previous patch is more preferable than
> later one.
> Could you explain the reason of that please.

There may be other kernel bits calling syscall, it sounds saner to
always make sure we don't reschedule when in atomic (though it's dubious
that it's allowed to do syscalls in atomic regions, let's be safe
there).

In fact, we are thinking about consolidating the syscall return path and
the normal irq return path like we did on ppc64.

Ben.

Re: boot time scheduling hile atomic

[Takeharu KATO]

Hi Benjamin:

Benjamin Herrenschmidt wrote:

> 
> There may be other kernel bits calling syscall, it sounds saner to
> always make sure we don't reschedule when in atomic (though it's dubious
> that it's allowed to do syscalls in atomic regions, let's be safe
> there).
> 
> In fact, we are thinking about consolidating the syscall return path and
> the normal irq return path like we did on ppc64.
> 
Thkans, I make sense.
Would you send me Paul's patch?

Regards,

-- 
Takeharu KATO
Fujitsu Limited
Email:kato.takeharu at jp.fujitsu.com

RE: boot time scheduling hile atomic

[Rune Torgersen]

Any patch forthcomming soon?=20

> -----Original Message-----
> From: linuxppc-dev-bounces@ozlabs.org=20
> [mailto:linuxppc-dev-bounces@ozlabs.org] On Behalf Of=20
> Benjamin Herrenschmidt
> Sent: Wednesday, March 16, 2005 16:29
> To: Takeharu KATO
> Cc: linuxppc-dev list
> Subject: Re: boot time scheduling hile atomic
>=20
> On Wed, 2005-03-16 at 23:07 +0900, Takeharu KATO wrote:
> > Hi Benjamin:
> >=20
> > I can not figure out why my previous patch is more preferable than
> > later one.
> > Could you explain the reason of that please.
>=20
> There may be other kernel bits calling syscall, it sounds saner to
> always make sure we don't reschedule when in atomic (though=20
> it's dubious
> that it's allowed to do syscalls in atomic regions, let's be safe
> there).
>=20
> In fact, we are thinking about consolidating the syscall=20
> return path and
> the normal irq return path like we did on ppc64.
>=20
> Ben.
>=20
>=20
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-dev
>=20
>=20

Re: boot time scheduling hile atomic

[Takeharu KATO]

Hi

IMHO, it is more preferable to change kernel thread creation mechanism adapting to
other architecture(in stead of tweaking exception handling) to solve the issue at least.
Of course, cleaning up the exception codes to adapt to ppc64  is also preferable from
code maintenance view.

Anyway, please try to apply the patch as follow:
http://ozlabs.org/ppc32-patches/patch.pl?id=318

Rune Torgersen wrote:
> Any patch forthcomming soon? 
> 
> 
>>-----Original Message-----
>>From: linuxppc-dev-bounces@ozlabs.org 
>>[mailto:linuxppc-dev-bounces@ozlabs.org] On Behalf Of 
>>Benjamin Herrenschmidt
>>Sent: Wednesday, March 16, 2005 16:29
>>To: Takeharu KATO
>>Cc: linuxppc-dev list
>>Subject: Re: boot time scheduling hile atomic
>>
>>On Wed, 2005-03-16 at 23:07 +0900, Takeharu KATO wrote:
>>
>>>Hi Benjamin:
>>>
>>>I can not figure out why my previous patch is more preferable than
>>>later one.
>>>Could you explain the reason of that please.
>>
>>There may be other kernel bits calling syscall, it sounds saner to
>>always make sure we don't reschedule when in atomic (though 
>>it's dubious
>>that it's allowed to do syscalls in atomic regions, let's be safe
>>there).
>>
>>In fact, we are thinking about consolidating the syscall 
>>return path and
>>the normal irq return path like we did on ppc64.
>>
>>Ben.
>>
>>
>>_______________________________________________
>>Linuxppc-dev mailing list
>>Linuxppc-dev@ozlabs.org
>>https://ozlabs.org/mailman/listinfo/linuxppc-dev
>>
>>
> 
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-dev
>