[Linux-ia64] where to set rr for uncached region

[Linux-ia64] where to set rr for uncached region

[nomura]

Hello,

Current code sets rr for region 6 (kernel uncached region) in ia64_mmu_init().
However, thinking about application processors, there are some printk()s
before ia64_mmu_init() to be called.
Since printk() ends up in vt_console_print() and store to uncached region
will happen, I think this cause a problem.

In reality, MCAs always happen on our machine after AP prints
"CPU 1: mapping PAL code ..." to serial console. The processor log
shows that MCA happend in vt_console_print() on CPU1.

I can work around the problem by applying this patch.

I've noticed the problem when EARLY_PRINTK was deleted from config option.
So I'm curious about whether anyone has not experienced same kind of problem.


diff -u -r1.1.1.8.6.1 head.S
--- arch/ia64/kernel/head.S	2001/10/04 10:31:46	1.1.1.8.6.1
+++ arch/ia64/kernel/head.S	2001/10/05 02:18:17
@@ -111,7 +111,6 @@
 	mov ar.fpsr=r2
 	;;
 
-#ifdef CONFIG_IA64_EARLY_PRINTK
 	mov r3=(6<<8) | (KERNEL_PG_SHIFT<<2)
 	movl r2=6<<61
 	;;
@@ -119,7 +118,6 @@
 	;;
 	srlz.i
 	;;
-#endif
 
 #define isAP	p2	// are we an Application Processor?
 #define isBP	p3	// are we the Bootstrap Processor?


Best regards.
--
NOMURA, Jun'ichi <j-nomura@ce.jp.nec.com, nomura@hpc.bs1.fc.nec.co.jp>
HPC Operating System Group, 1st Computers Software Division,
Computers Software Operations Unit, NEC Solutions.

Re: [Linux-ia64] where to set rr for uncached region

[David Mosberger]

>>>>> On Thu, 22 Nov 2001 20:36:47 +0900, nomura@hpc.bs1.fc.nec.co.jp said:

  >> Hello, Current code sets rr for region 6 (kernel uncached region)
  >> in ia64_mmu_init().  However, thinking about application
  >> processors, there are some printk()s before ia64_mmu_init() to be
  >> called.  Since printk() ends up in vt_console_print() and store
  >> to uncached region will happen, I think this cause a problem.

  >> In reality, MCAs always happen on our machine after AP prints
  >> "CPU 1: mapping PAL code ..." to serial console. The processor
  >> log shows that MCA happend in vt_console_print() on CPU1.

  >> I can work around the problem by applying this patch.

  >> I've noticed the problem when EARLY_PRINTK was deleted from
  >> config option.  So I'm curious about whether anyone has not
  >> experienced same kind of problem.

Hmmh, this is a bit of a nasty problem.  We shouldn't be calling any
console drivers until an AP is fully initialized.  But the platform
independent part of Linux currently only delays printing on the
boostrap processor.  Perhaps printk() should be enhanced with a check
to ensure that a processor is ready for calling the console driver.
For example, in __call_console_drivers() we could have something along
the lines of:

	for (con = console_drivers; con; con = con->next) {
		if ((con->flags & CON_ENABLED) && con->write
		    && (cpu_online_map & 1UL << smp_processor_id()))
			con->write(con, &LOG_BUF(start), end - start);
	}

Do you want to try this?

	--david

Re: [Linux-ia64] where to set rr for uncached region

[nomura]

Hi,

Thank you for your suggestion. It's a better approach.
Though putting the check in __call_console_drivers() does fix the problem,
it's not satisfactory. Because, until cpu_online_map is set up, log_buf is
flushed without printing.

So I put the check in release_console_sem().
How about this fix?

From: David Mosberger <davidm@hpl.hp.com>
Subject: Re: [Linux-ia64] where to set rr for uncached region
Date: Mon, 26 Nov 2001 15:41:31 -0800

> Hmmh, this is a bit of a nasty problem.  We shouldn't be calling any
> console drivers until an AP is fully initialized.  But the platform
> independent part of Linux currently only delays printing on the
> boostrap processor.  Perhaps printk() should be enhanced with a check
> to ensure that a processor is ready for calling the console driver.
> For example, in __call_console_drivers() we could have something along
> the lines of:
> 
> 	for (con = console_drivers; con; con = con->next) {
> 		if ((con->flags & CON_ENABLED) && con->write
> 		    && (cpu_online_map & 1UL << smp_processor_id()))
> 			con->write(con, &LOG_BUF(start), end - start);
> 	}
> 
> Do you want to try this?

Index: kernel/printk.c
=================================RCS file: /home/cvsadm/cvsroot/linux/kernel/printk.c,v
retrieving revision 1.1.1.8.2.1
diff -u -r1.1.1.8.2.1 printk.c
--- kernel/printk.c	2001/11/27 08:28:04	1.1.1.8.2.1
+++ kernel/printk.c	2001/11/29 06:20:35
@@ -505,6 +505,10 @@
 	for ( ; ; ) {
 		spin_lock_irqsave(&logbuf_lock, flags);
 		must_wake_klogd |= log_start - log_end;
+#ifndef CONFIG_IA64_EARLY_PRINTK
+		if (!(cpu_online_map & 1UL << smp_processor_id()))
+			break;
+#endif
 		if (con_start = log_end)
 			break;			/* Nothing to print */
 		_con_start = con_start;


Best regards.
--
NOMURA, Jun'ichi <j-nomura@ce.jp.nec.com, nomura@hpc.bs1.fc.nec.co.jp>
HPC Operating System Group, 1st Computers Software Division,
Computers Software Operations Unit, NEC Solutions.

Re: [Linux-ia64] where to set rr for uncached region

[David Mosberger]

>>>>> On Thu, 29 Nov 2001 16:34:21 +0900, nomura@hpc.bs1.fc.nec.co.jp said:

  >> Hi, Thank you for your suggestion. It's a better approach.
  >> Though putting the check in __call_console_drivers() does fix the
  >> problem, it's not satisfactory. Because, until cpu_online_map is
  >> set up, log_buf is flushed without printing.

  >> So I put the check in release_console_sem().  How about this fix?

Works for me.

Thanks,

	--david