From: Ben Woodard <woodard@redhat.com>
To: Vivek Goyal <vgoyal@redhat.com>
Cc: Neil Horman <nhorman@redhat.com>,
Andi Kleen <andi@firstfloor.org>,
kexec@lists.infradead.org, linux-kernel@vger.kernel.org,
Andi Kleen <ak@suse.de>,
hbabu@us.ibm.com, "Eric W. Biederman" <ebiederm@xmission.com>
Subject: Re: [PATCH] kexec: force x86_64 arches to boot kdump kernels on boot cpu
Date: Thu, 29 Nov 2007 18:12:07 -0800 [thread overview]
Message-ID: <474F7177.7050306@redhat.com> (raw)
In-Reply-To: <20071128190525.GD3192@redhat.com>
Vivek Goyal wrote:
> On Wed, Nov 28, 2007 at 11:02:06AM -0500, Neil Horman wrote:
>> On Wed, Nov 28, 2007 at 10:36:49AM -0500, Vivek Goyal wrote:
>>> On Tue, Nov 27, 2007 at 03:24:35PM -0800, Ben Woodard wrote:
>>>> Andi Kleen wrote:
>>>>>> Are we putting the system back in PIC mode or virtual wire mode? I have
>>>>>> not seen systems which support PIC mode. All latest systems seems
>>>>>> to be having virtual wire mode. I think in case of PIC mode, interrupts
>>>>> Yes it's probably virtual wire. For real PIC mode we would need really
>>>>> old systems without APIC.
>>>>>
>>>>>> can be delivered to cpu0 only. In virt wire mode, one can program IOAPIC
>>>>>> to deliver interrupt to any of the cpus and that's what we have been
>>>>> The code doesn't try to program anything specific, it just restores the state
>>>>> that was left over originally by the BIOS.
>>>>>
>>>> So if the BIOS originally left the IOAPIC in a state where the timer
>>>> interrupts were only going to CPU0 then by restoring that state we could be
>>>> bringing this problem upon ourselves when we restore that state.
>>>>
>>> Hi Ben,
>>>
>>> Apart from restoring the original state (Bring APICS back to virtual wire
>>> mode), we also reprogram IOAPIC so that timer interrupt can go to crashing
>>> cpu (and not necessarily cpu0). Look at following code in disable_IO_APIC.
>>>
>>> entry.dest.physical.physical_dest =
>>> GET_APIC_ID(apic_read(APIC_ID));
>>>
>>> Here we read the apic id of crashing cpu and program IOAPIC accordingly.
>>> This will make sure that even in virtual wire mode, timer interrupts
>>> will be delivered to crashing cpu APIC.
>>>
>> Yes, but according to Bens last debug effort, the APIC printout regarding the
>> timer setup, indicates that ioapic_i8259.pin == -1, meaning that the 8259 is not
>> routed through the ioapic. In those cases, disable_IO_APIC does not take us
>> through the path you reference above, and does not revert to virtual wire mode.
>> Instead, it simply disables legacy vector 0, which if I understand this
>> correctly, simply tells the ioapic to not handle timer interrupts, trusting that
>> the 8259 in the system will deliver that interrupt where it needs to be. If the
>> 8259 is wired to deliver timer interrupts to cpu0 only, then you get the problem
>> that we have, do you?
>>
>
> Ok. Got it. So in this case we route the interrupts directly through LAPIC
> and put LVT0 in ExtInt mode and IOAPIC is bypassed.
>
> I am looking at Intel Multiprocessor specification v1.4 and as per figure
> 3-3 on page 3-9, 8259 is connected to LINTIN0 line, which in turn is
> connected to LINTIN0 pin on all processors. If that is the case, even in
> this mode, all the CPU should see the timer interrupts (which is coming
> from 8259)?
>
> Can you print the LAPIC registers (print_local_APIC) during normal boot
> and during kdump boot and paste here?
Here are the ones from a normal bootup.
I was unable to get info from a kdump boot. I haven't figured out why
yet. With the same patch that I used to capture this, when I tried to
kdump the kernel, it paused a second or two after the backtrace and then
dropped to BIOS and came up normally.
Here is a little trick, at the point where we are trying to get the info
to print out, the kernel command line hasn't been completely parsed yet.
That tricked me for part of the day. I had apic=debug on the command
line but the logic in print_local_APIC saw the default value because the
kernel command line had yet to be parsed.
2007-11-29 17:58:07 ***Here is the info you requested
2007-11-29 17:58:07
2007-11-29 17:58:07 printing local APIC contents on CPU#0/0:
2007-11-29 17:58:07 ... APIC ID: 00000000 (0)
2007-11-29 17:58:07 ... APIC VERSION: 80050010
2007-11-29 17:58:07 ... APIC TASKPRI: 00000000 (00)
2007-11-29 17:58:07 ... APIC ARBPRI: 00000000 (00)
2007-11-29 17:58:07 ... APIC PROCPRI: 00000000
2007-11-29 17:58:07 ... APIC EOI: 00000000
2007-11-29 17:58:07 ... APIC RRR: 00000002
2007-11-29 17:58:07 ... APIC LDR: 00000000
2007-11-29 17:58:07 ... APIC DFR: ffffffff
2007-11-29 17:58:07 ... APIC SPIV: 0000010f
2007-11-29 17:58:07 ... APIC ISR field:
2007-11-29 17:58:07 ... APIC TMR field:
2007-11-29 17:58:07 ... APIC IRR field:
2007-11-29 17:58:07 ... APIC ESR: 00000000
2007-11-29 17:58:07 ... APIC ICR: 00004630
2007-11-29 17:58:07 ... APIC ICR2: 07000000
2007-11-29 17:58:07 ... APIC LVTT: 00010000
2007-11-29 17:58:07 ... APIC LVTPC: 00010000
2007-11-29 17:58:07 ... APIC LVT0: 00000700
2007-11-29 17:58:07 ... APIC LVT1: 00000400
2007-11-29 17:58:07 ... APIC LVTERR: 0001000f
2007-11-29 17:58:07 ... APIC TMICT: 80000000
2007-11-29 17:58:07 ... APIC TMCCT: 00000000
2007-11-29 17:58:07 ... APIC TDCR: 00000000
2007-11-29 17:58:07
2007-11-29 17:58:07 number of MP IRQ sources: 15.
2007-11-29 17:58:07 number of IO-APIC #8 registers: 0.
2007-11-29 17:58:07 number of IO-APIC #9 registers: 0.
2007-11-29 17:58:07 number of IO-APIC #10 registers: 0.
2007-11-29 17:58:07 testing the IO APIC.......................
2007-11-29 17:58:07
2007-11-29 17:58:07 IO APIC #8......
2007-11-29 17:58:07 .... register #00: 08000000
2007-11-29 17:58:07 ....... : physical APIC id: 08
2007-11-29 17:58:07 .... register #01: 00170011
2007-11-29 17:58:07 ....... : max redirection entries: 0017
2007-11-29 17:58:07 ....... : PRQ implemented: 0
2007-11-29 17:58:07 ....... : IO APIC version: 0011
2007-11-29 17:58:07 .... register #02: 08000000
2007-11-29 17:58:07 ....... : arbitration: 08
2007-11-29 17:58:07 .... IRQ redirection table:
2007-11-29 17:58:07 NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
2007-11-29 17:58:07 00 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 01 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 02 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 03 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 04 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 05 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 06 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 07 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 08 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 09 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 0a 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 0b 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 0c 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 0d 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 0e 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 0f 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 10 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 11 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 12 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 13 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 14 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 15 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 16 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 17 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07
2007-11-29 17:58:07 IO APIC #9......
2007-11-29 17:58:07 .... register #00: 09000000
2007-11-29 17:58:07 ....... : physical APIC id: 09
2007-11-29 17:58:07 .... register #01: 00060011
2007-11-29 17:58:07 ....... : max redirection entries: 0006
2007-11-29 17:58:07 ....... : PRQ implemented: 0
2007-11-29 17:58:07 ....... : IO APIC version: 0011
2007-11-29 17:58:07 .... register #02: 00000000
2007-11-29 17:58:07 ....... : arbitration: 00
2007-11-29 17:58:07 .... IRQ redirection table:
2007-11-29 17:58:07 NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
2007-11-29 17:58:07 00 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:07 01 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:08 02 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:08 03 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:08 04 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:08 05 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:08 06 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:08
2007-11-29 17:58:08 IO APIC #10......
2007-11-29 17:58:08 .... register #00: 0A000000
2007-11-29 17:58:08 ....... : physical APIC id: 0A
2007-11-29 17:58:08 .... register #01: 00060011
2007-11-29 17:58:08 ....... : max redirection entries: 0006
2007-11-29 17:58:08 ....... : PRQ implemented: 0
2007-11-29 17:58:08 ....... : IO APIC version: 0011
2007-11-29 17:58:08 .... register #02: 00000000
2007-11-29 17:58:08 ....... : arbitration: 00
2007-11-29 17:58:08 .... IRQ redirection table:
2007-11-29 17:58:08 NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
2007-11-29 17:58:08 00 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:08 01 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:08 02 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:08 03 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:08 04 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:08 05 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:08 06 000 00 1 0 0 0 0 0 0 00
2007-11-29 17:58:08 Using vector-based indexing
2007-11-29 17:58:08 IRQ to pin mappings:
2007-11-29 17:58:08 IRQ0 -> 0:0
2007-11-29 17:58:08 IRQ1 -> 0:0
2007-11-29 17:58:08 IRQ2 -> 0:0
2007-11-29 17:58:08 IRQ3 -> 0:0
2007-11-29 17:58:08 IRQ4 -> 0:0
2007-11-29 17:58:08 IRQ5 -> 0:0
2007-11-29 17:58:08 IRQ6 -> 0:0
2007-11-29 17:58:08 IRQ7 -> 0:0
2007-11-29 17:58:08 IRQ8 -> 0:0
2007-11-29 17:58:08 IRQ9 -> 0:0
2007-11-29 17:58:08 IRQ10 -> 0:0
2007-11-29 17:58:08 IRQ11 -> 0:0
2007-11-29 17:58:08 IRQ12 -> 0:0
2007-11-29 17:58:08 IRQ13 -> 0:0
2007-11-29 17:58:08 IRQ14 -> 0:0
2007-11-29 17:58:08 IRQ15 -> 0:0
2007-11-29 17:58:08 IRQ0 -> 0:0
<snip a whole bunch of identical lines>
2007-11-29 17:58:08 .................................... done.
2007-11-29 17:58:08 Built 4 zonelists. Total pages: 3937654
The patch that I was using to get this is as follows:
--- linux/init/main.c (revision 763)
+++ linux/init/main.c (working copy)
@@ -484,6 +484,8 @@
{
}
+extern void print_local_APIC(void);
+
asmlinkage void __init start_kernel(void)
{
char * command_line;
@@ -515,6 +517,11 @@
setup_per_cpu_areas();
smp_prepare_boot_cpu(); /* arch-specific boot-cpu hooks */
+ console_loglevel=10;
+ printk(KERN_DEBUG "***Here is the info you requested\n");
+ print_local_APIC();
+ print_IO_APIC();
+
/*
* Set up the scheduler prior starting any interrupts (such as the
* timer interrupt). Full topology setup happens at smp_init()
Index: linux/arch/x86_64/kernel/io_apic.c
===================================================================
--- linux/arch/x86_64/kernel/io_apic.c (revision 763)
+++ linux/arch/x86_64/kernel/io_apic.c (working copy)
@@ -1012,9 +1012,9 @@
union IO_APIC_reg_02 reg_02;
unsigned long flags;
- if (apic_verbosity == APIC_QUIET)
+/* if (apic_verbosity == APIC_QUIET)
return;
-
+*/
printk(KERN_DEBUG "number of MP IRQ sources: %d.\n",
mp_irq_entries);
for (i = 0; i < nr_ioapics; i++)
printk(KERN_DEBUG "number of IO-APIC #%d registers: %d.\n",
@@ -1131,8 +1131,6 @@
return;
}
-#if 0
-
static __apicdebuginit void print_APIC_bitfield (int base)
{
unsigned int v;
@@ -1158,9 +1156,9 @@
{
unsigned int v, ver, maxlvt;
- if (apic_verbosity == APIC_QUIET)
+/* if (apic_verbosity == APIC_QUIET)
return;
-
+*/
printk("\n" KERN_DEBUG "printing local APIC contents on
CPU#%d/%d:\n",
smp_processor_id(), hard_smp_processor_id());
v = apic_read(APIC_ID);
@@ -1268,8 +1266,6 @@
printk(KERN_DEBUG "... PIC ELCR: %04x\n", v);
}
-#endif /* 0 */
-
static void __init enable_IO_APIC(void)
{
union IO_APIC_reg_01 reg_01;
>
> Thanks
> Vivek
--
-ben
-=-
next prev parent reply other threads:[~2007-11-30 2:11 UTC|newest]
Thread overview: 100+ messages / expand[flat|nested] mbox.gz Atom feed top
2007-11-27 1:47 [PATCH] kexec: force x86_64 arches to boot kdump kernels on boot cpu Neil Horman
2007-11-27 4:12 ` Eric W. Biederman
2007-11-27 13:13 ` Neil Horman
2007-11-27 13:28 ` Eric W. Biederman
2007-11-27 13:45 ` Andi Kleen
2007-11-27 14:28 ` Neil Horman
2007-11-27 14:43 ` Andi Kleen
2007-11-27 14:48 ` Neil Horman
2007-11-27 15:24 ` Andi Kleen
2007-11-27 15:30 ` Eric W. Biederman
2007-11-27 16:45 ` Neil Horman
2007-11-27 20:50 ` Ben Woodard
2007-11-27 21:05 ` Neil Horman
2007-11-27 22:38 ` Eric W. Biederman
2007-11-27 23:15 ` Ben Woodard
2007-11-28 0:15 ` Eric W. Biederman
2007-11-27 23:40 ` Neil Horman
2007-11-28 0:43 ` Eric W. Biederman
2007-11-28 15:54 ` Neil Horman
2007-11-27 14:56 ` Eric W. Biederman
2007-11-27 15:34 ` Neil Horman
2007-11-27 18:41 ` Ben Woodard
2007-11-27 19:42 ` Neil Horman
2007-11-27 20:00 ` Vivek Goyal
2007-11-27 20:52 ` Neil Horman
2007-11-27 22:24 ` Andi Kleen
2007-11-27 23:24 ` Ben Woodard
2007-11-27 23:56 ` Andi Kleen
2007-11-28 15:36 ` Vivek Goyal
2007-11-28 16:02 ` Neil Horman
2007-11-28 17:36 ` Eric W. Biederman
2007-11-28 18:16 ` Neil Horman
2007-11-28 19:05 ` Vivek Goyal
2007-11-28 19:42 ` Eric W. Biederman
2007-11-28 21:09 ` Neil Horman
2007-11-28 23:27 ` Eric W. Biederman
2007-11-30 2:16 ` Ben Woodard
2007-11-30 2:54 ` Eric W. Biederman
2007-11-30 8:59 ` Yinghai Lu
2007-11-30 14:35 ` Vivek Goyal
2007-11-30 14:32 ` Neil Horman
2007-11-30 2:12 ` Ben Woodard [this message]
2007-11-30 14:42 ` Vivek Goyal
2007-11-30 14:51 ` Neil Horman
2007-12-06 21:39 ` Neil Horman
2007-12-06 22:11 ` Vivek Goyal
2007-12-07 0:10 ` Neil Horman
2007-12-07 14:39 ` Vivek Goyal
2007-12-07 14:53 ` Neil Horman
2007-12-07 15:16 ` Vivek Goyal
2007-12-07 15:53 ` Neil Horman
2007-12-07 18:46 ` Eric W. Biederman
2007-12-07 0:33 ` Eric W. Biederman
2007-12-07 2:04 ` Neil Horman
2007-12-07 8:50 ` Yinghai Lu
2007-12-07 9:22 ` Yinghai Lu
2007-12-07 14:21 ` Neil Horman
2007-12-07 17:58 ` Neil Horman
2007-12-07 19:19 ` yhlu
2007-12-07 20:13 ` Neil Horman
2007-12-10 15:39 ` Neil Horman
2007-12-10 16:20 ` Vivek Goyal
2007-12-11 1:17 ` Eric W. Biederman
2007-12-11 1:08 ` Eric W. Biederman
2007-12-11 3:43 ` Neil Horman
2007-12-11 4:48 ` Eric W. Biederman
2007-12-11 6:31 ` Yinghai Lu
2007-12-11 14:39 ` Neil Horman
2007-12-11 15:29 ` Eric W. Biederman
2007-12-11 18:00 ` Yinghai Lu
2007-12-11 18:29 ` Neil Horman
2007-12-11 18:45 ` Yinghai Lu
2007-12-11 18:22 ` Neil Horman
2007-12-11 18:46 ` Eric W. Biederman
2007-12-11 19:24 ` Neil Horman
2007-12-11 19:51 ` Yinghai Lu
2007-12-11 20:59 ` Neil Horman
2007-12-12 0:16 ` Ben Woodard
2007-12-12 0:52 ` Neil Horman
2007-12-12 1:07 ` Yinghai Lu
2007-12-12 8:43 ` [PATCH] k8: Enable legacy irqs with extended cpu ids Eric W. Biederman
2007-12-12 14:21 ` [PATCH] kexec: force x86_64 arches to boot kdump kernels on boot cpu Andi Kleen
2007-12-12 15:55 ` Neil Horman
2007-12-12 16:07 ` Andi Kleen
2007-12-12 19:43 ` Eric W. Biederman
2007-12-12 20:22 ` Neil Horman
2007-12-12 21:32 ` Eric W. Biederman
2007-12-13 14:39 ` Neil Horman
2007-12-13 15:16 ` Andi Kleen
2007-12-13 15:32 ` Neil Horman
2007-12-17 11:38 ` Neil Horman
2007-12-18 0:13 ` Eric W. Biederman
2007-12-17 15:16 ` Ingo Molnar
2007-12-17 15:47 ` Neil Horman
2007-12-07 18:36 ` Eric W. Biederman
2007-12-07 18:48 ` Neil Horman
2007-11-27 13:53 ` Neil Horman
2007-11-27 10:55 ` Andi Kleen
2007-11-27 11:19 ` Eric W. Biederman
2007-11-27 13:28 ` Neil Horman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=474F7177.7050306@redhat.com \
--to=woodard@redhat.com \
--cc=ak@suse.de \
--cc=andi@firstfloor.org \
--cc=ebiederm@xmission.com \
--cc=hbabu@us.ibm.com \
--cc=kexec@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=nhorman@redhat.com \
--cc=vgoyal@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox