kdump on non-boot cpu

kdump on non-boot cpu

[Itsuro Oda]

Hi,

I found the following in an old mail:

>From vgoyal at in.ibm.com  Thu Jan  6 07:20:43 2005
...
>2. Kdump can possibly fail on SMP machines if crash occurs on non-boot
>cpu. Hari is finalizing the stop gap patch to handle this problem.

Is this finished ?  (It seems it is not in 2.6.11-rc2-mm1.)
-- 
Itsuro ODA <oda@valinux.co.jp>

Re: kdump on non-boot cpu

[Vivek Goyal]

On Thu, 2005-02-03 at 10:42, Itsuro Oda wrote:
> Hi,
> 
> I found the following in an old mail:
> 
> >From vgoyal at in.ibm.com  Thu Jan  6 07:20:43 2005
> ...
> >2. Kdump can possibly fail on SMP machines if crash occurs on non-boot
> >cpu. Hari is finalizing the stop gap patch to handle this problem.
> 
> Is this finished ?  (It seems it is not in 2.6.11-rc2-mm1.)

Not yet. For the time being focus got shifted to other kdump issues. I
am not even sure if this is a problem. See below a clip from discussions
on fastboot.

> Hi Eric,
> > 
> > I had a quick look at kexec3. Had some queries.
> > 
> > 1. Code for relocating to boot cpu or enabling boot from non-boot cpu is
> > required.
> 
> Actually I just looked and it appears this snippet from smp_boot_cpus
> already handles that case.
> 
>         boot_cpu_physical_apicid = GET_APIC_ID(apic_read(APIC_ID));
>         boot_cpu_logical_apicid = logical_smp_processor_id();
>         x86_cpu_to_apicid[0] = boot_cpu_physical_apicid;
> 
> While looking I certainly did not see anything still in the
> kernel that would complain if we get this wrong.
> 
> Although I  am not really comfortable with a capture kernel using
> multiprocessors. 
> 
> Eric


Do you see a problem in the code flow somewhere?

Vivek

Re: kdump on non-boot cpu

[Itsuro Oda]

Hi,

This is not for kdump but an experience of our project(mkdump).
The dump kernel(not SMP config) boot hangs if machine_kexec()
excutes on non-boot CPU on x86_64 platform.
We don't found why. (Please let me know if you know why.)
but fix that the boot-cpu excutes machine_kexec() in the nmi handler. 
(It becomes OK after that)

See attached patch (this is only for explanation). (this is for
arch/i386/kernel/crash.c in the 2.6.11-rc2-mm1)

> Do you see a problem in the code flow somewhere?

I point some concerns in the attached patch too. (not related above)

Thanks.

On 03 Feb 2005 12:12:32 +0530
Vivek Goyal <vgoyal@in.ibm.com> wrote:

> On Thu, 2005-02-03 at 10:42, Itsuro Oda wrote:
> > Hi,
> > 
> > I found the following in an old mail:
> > 
> > >From vgoyal at in.ibm.com  Thu Jan  6 07:20:43 2005
> > ...
> > >2. Kdump can possibly fail on SMP machines if crash occurs on non-boot
> > >cpu. Hari is finalizing the stop gap patch to handle this problem.
> > 
> > Is this finished ?  (It seems it is not in 2.6.11-rc2-mm1.)
> 
> Not yet. For the time being focus got shifted to other kdump issues. I
> am not even sure if this is a problem. See below a clip from discussions
> on fastboot.
> 
> > Hi Eric,
> > > 
> > > I had a quick look at kexec3. Had some queries.
> > > 
> > > 1. Code for relocating to boot cpu or enabling boot from non-boot cpu is
> > > required.
> > 
> > Actually I just looked and it appears this snippet from smp_boot_cpus
> > already handles that case.
> > 
> >         boot_cpu_physical_apicid = GET_APIC_ID(apic_read(APIC_ID));
> >         boot_cpu_logical_apicid = logical_smp_processor_id();
> >         x86_cpu_to_apicid[0] = boot_cpu_physical_apicid;
> > 
> > While looking I certainly did not see anything still in the
> > kernel that would complain if we get this wrong.
> > 
> > Although I  am not really comfortable with a capture kernel using
> > multiprocessors. 
> > 
> > Eric
> 
> 
> Do you see a problem in the code flow somewhere?
> 
> Vivek

-- 
Itsuro ODA <oda@valinux.co.jp>

---
--- crash.c     2005-01-28 23:53:30.000000000 +0900
+++ crash.c.new 2005-02-04 00:49:12.599084080 +0900
@@ -111,6 +111,26 @@

 #ifdef CONFIG_SMP
 static atomic_t waiting_for_crash_ipi;
+static int reboot_cpu = 0;
+
+static void wait_and_reboot(void)
+{
+       unsigned long msecs;
+
+       msecs = 1000; /* Wait at most a second for the other cpus to stop */
+       while ((atomic_read(&waiting_for_crash_ipi) > 0) && msecs) {
+               mdelay(1);
+               msecs--;
+       }
+#if defined(CONFIG_X86_IO_APIC)
+       /* XXX: it is necessary... to avoid deadlock
+            - export "spinlock_t ioapic_lock"
+            - call spin_lock_init(&ioapic_lock) here
+         */
+       disable_IO_APIC();
+#endif
+       machine_kexec(...); /* XXX: needs image */
+}

 static int crash_nmi_callback(struct pt_regs *regs, int cpu)
 {
@@ -118,9 +138,13 @@
        crash_save_this_cpu(regs, cpu);
        disable_local_APIC();
        atomic_dec(&waiting_for_crash_ipi);
-       /* Assume hlt works */
-       __asm__("hlt");
-       for(;;);
+
+       if (cpu == reboot_cpu) {
+               wait_and_reboot();
+       } else {
+               __asm__("hlt");
+               for(;;);
+       }
        return 1;
 }

@@ -136,7 +160,6 @@

 static void nmi_shootdown_cpus(void)
 {
-       unsigned long msecs;
        atomic_set(&waiting_for_crash_ipi, num_online_cpus() - 1);

        /* Would it be better to replace the trap vector here? */
@@ -148,19 +171,15 @@

        smp_send_nmi_allbutself();

-       msecs = 1000; /* Wait at most a second for the other cpus to stop */
-       while ((atomic_read(&waiting_for_crash_ipi) > 0) && msecs) {
-               mdelay(1);
-               msecs--;
-       }
-
-       /* Leave the nmi callback set */
+       crash_save_self();
        disable_local_APIC();
-}
-#else
-static void nmi_shootdown_cpus(void)
-{
-       /* There are no cpus to shootdown */
+
+       if (smp_processor_id() == reboot_cpu) {
+               wait_and_reboot();
+       } else {
+               __asm__("hlt");
+               for(;;);
+       }
 }
 #endif

@@ -176,10 +195,13 @@
         */
        /* The kernel is broken so disable interrupts */
        local_irq_disable();
+#ifdef CONFIG_SMP
        nmi_shootdown_cpus();
-       lapic_shutdown();
+#else
+       lapic_shutdown();       /* XXX: this calls local_irq_enable. is it OK ? */
 #if defined(CONFIG_X86_IO_APIC)
        disable_IO_APIC();
 #endif
        crash_save_self();
+#endif
 }
---

Re: [Fastboot] Re: kdump on non-boot cpu

[Eric W. Biederman]

Itsuro Oda <oda@valinux.co.jp> writes:

> Hi,
> 
> This is not for kdump but an experience of our project(mkdump).
> The dump kernel(not SMP config) boot hangs if machine_kexec()
> excutes on non-boot CPU on x86_64 platform.

?? x86_64 is Opteron cpu, amd64, Intel cpu?
Are the kernels running in 32bit or 64bit mode. I'm guessing
SMP Opterons running in 32bit mode.

Anyway one thing I want to do is actually drop the apic shutdown
code altogether in this code path.  I threw it in there to
ease the transition from the old code base to the new, but
if that code is causing issues....  So this is probably a good time
to start testing that.

> We don't found why. (Please let me know if you know why.)
> but fix that the boot-cpu excutes machine_kexec() in the nmi handler. 
> (It becomes OK after that)

My best hunch is that your UP kernel is not getting interrupts.
Any chance on getting a serial console boot log?  

I suspect it can be made to work if you compile your UP
kernel with IOAPIC support.  I do know the table parsers
no longer complain about the configuration.

Eric

Re: [Fastboot] Re: kdump on non-boot cpu

[Itsuro Oda]

Hi,

On 03 Feb 2005 02:58:02 -0700
ebiederm@xmission.com (Eric W. Biederman) wrote:

> Itsuro Oda <oda@valinux.co.jp> writes:
> 
> > Hi,
> > 
> > This is not for kdump but an experience of our project(mkdump).
> > The dump kernel(not SMP config) boot hangs if machine_kexec()
> > excutes on non-boot CPU on x86_64 platform.
> 
> ?? x86_64 is Opteron cpu, amd64, Intel cpu?
> Are the kernels running in 32bit or 64bit mode. I'm guessing
> SMP Opterons running in 32bit mode.

SMP Opterons running in 64bit mode. (The normal kernel is SMP configed.
The dump kernel is not SMP configed.)

> 
> Anyway one thing I want to do is actually drop the apic shutdown
> code altogether in this code path.  

It sounds nice. (if available)

> 
> My best hunch is that your UP kernel is not getting interrupts.

yes. It seems the timer interrupts is not comming up.

> Any chance on getting a serial console boot log?  
> 
attached. both success case and unsuccess case.

> I suspect it can be made to work if you compile your UP
> kernel with IOAPIC support.  I do know the table parsers
> no longer complain about the configuration.

with IOAPIC support. the config is attached.

Thanks.
-- 
Itsuro ODA <oda@valinux.co.jp>

--- unsuccess case (machine_kexec is executed on CPU #1) ---
No mptable found.
No mptable found.
On node 0 totalpages: 2048
  DMA zone: 2048 pages, LIFO batch:1
  Normal zone: 0 pages, LIFO batch:1
  HighMem zone: 0 pages, LIFO batch:1
ACPI: RSDP (v002 PTLTD                                     ) @ 0x00000000000f69a
0
ACPI: XSDT (v001 PTLTD           XSDT   0x06040000  LTP 0x00000000) @ 0x00000000
fbf734d8
ACPI: FADT (v003 AMD    HAMMER   0x06040000 PTEC 0x000f4240) @ 0x00000000fbf75e4
6
ACPI: MADT (v001 PTLTD           APIC   0x06040000  LTP 0x00000000) @ 0x00000000
fbf75f3a
ACPI: SPCR (v001 PTLTD  $UCRTBL$ 0x06040000 PTL  0x00000001) @ 0x00000000fbf75fb
0
ACPI: DSDT (v001 AMD-K8  AMDACPI 0x06040000 MSFT 0x0100000e) @ 0x000000000000000
0
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
Processor #0 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
Processor #1 15:5 APIC version 16
WARNING: NR_CPUS limit of 1 reached. Processor ignored.
ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
Using ACPI for processor (LAPIC) configuration information
Intel MultiProcessor Specification v1.4
    Virtual Wire compatibility mode.
SMP mptable: bad signature [@ツス  ]!
BIOS bug, MP table errors detected!...
... disabling SMP support. (tell your hw vendor)
Checking aperture...
CPU 0: aperture @ 0 size 32 MB
No AGP bridge found
Built 1 zonelists
Kernel command line: ro console=tty0 console=ttyS0,38400n8r dump_dev=0x300008 me
m=7M
Initializing CPU#0
PID hash table entries: 16 (order 4: 256 bytes)
time.c: Using 1.193182 MHz PIT timer.
time.c: Detected 1403.226 MHz processor.
Console: colour VGA+ 80x25
unexpected IRQ trap at vector 11
Memory: 5724k/8192k available (1080k kernel code, 2040k reserved, 432k data, 140
k init)
Calibrating delay loop...
###### hang #########

--- success case (machine_kexec is executed on CPU #0) ---
No mptable found.
No mptable found.
On node 0 totalpages: 2048
  DMA zone: 2048 pages, LIFO batch:1
  Normal zone: 0 pages, LIFO batch:1
  HighMem zone: 0 pages, LIFO batch:1
ACPI: RSDP (v002 PTLTD                                     ) @ 0x00000000000f69a
0
ACPI: XSDT (v001 PTLTD           XSDT   0x06040000  LTP 0x00000000) @ 0x00000000
fbf734d8
ACPI: FADT (v003 AMD    HAMMER   0x06040000 PTEC 0x000f4240) @ 0x00000000fbf75e4
6
ACPI: MADT (v001 PTLTD           APIC   0x06040000  LTP 0x00000000) @ 0x00000000
fbf75f3a
ACPI: SPCR (v001 PTLTD  $UCRTBL$ 0x06040000 PTL  0x00000001) @ 0x00000000fbf75fb
0
ACPI: DSDT (v001 AMD-K8  AMDACPI 0x06040000 MSFT 0x0100000e) @ 0x000000000000000
0
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
Processor #0 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
Processor #1 15:5 APIC version 16
WARNING: NR_CPUS limit of 1 reached. Processor ignored.
ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
Using ACPI for processor (LAPIC) configuration information
Intel MultiProcessor Specification v1.4
    Virtual Wire compatibility mode.
SMP mptable: bad signature [@ツス  ]!
BIOS bug, MP table errors detected!...
... disabling SMP support. (tell your hw vendor)
Checking aperture...
CPU 0: aperture @ 0 size 32 MB
No AGP bridge found
Built 1 zonelists
Kernel command line: ro console=tty0 console=ttyS0,38400n8r dump_dev=0x300008 me
m=7M
Initializing CPU#0
PID hash table entries: 16 (order 4: 256 bytes)
time.c: Using 1.193182 MHz PIT timer.
time.c: Detected 1403.204 MHz processor.
Console: colour VGA+ 80x25
Memory: 5724k/8192k available (1080k kernel code, 2040k reserved, 432k data, 140
k init)
Calibrating delay loop... 2760.70 BogoMIPS
##### OK ##########

--- config (the dump kernel) ----
#
# Automatically generated make config: don't edit
#
CONFIG_X86_64=y
CONFIG_64BIT=y
CONFIG_X86=y
CONFIG_MMU=y
CONFIG_RWSEM_GENERIC_SPINLOCK=y
CONFIG_X86_CMPXCHG=y
CONFIG_EARLY_PRINTK=y
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
CONFIG_GENERIC_ISA_DMA=y

#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
CONFIG_CLEAN_COMPILE=y
CONFIG_STANDALONE=y
CONFIG_BROKEN_ON_SMP=y

#
# General setup
#
CONFIG_DUMP_MINI_KERNEL=y
CONFIG_X86_PAE=y
# CONFIG_SWAP is not set
# CONFIG_SYSVIPC is not set
# CONFIG_POSIX_MQUEUE is not set
# CONFIG_BSD_PROCESS_ACCT is not set
CONFIG_SYSCTL=y
# CONFIG_AUDIT is not set
CONFIG_LOG_BUF_SHIFT=14
# CONFIG_HOTPLUG is not set
# CONFIG_IKCONFIG is not set
# CONFIG_EMBEDDED is not set
CONFIG_KALLSYMS=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set

#
# Loadable module support
#
# CONFIG_MODULES is not set

#
# Processor type and features
#
CONFIG_MK8=y
# CONFIG_MPSC is not set
# CONFIG_GENERIC_CPU is not set
CONFIG_X86_L1_CACHE_BYTES=64
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_X86_TSC=y
CONFIG_X86_GOOD_APIC=y
# CONFIG_MICROCODE is not set
CONFIG_X86_MSR=y
CONFIG_X86_CPUID=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_MTRR=y
# CONFIG_SMP is not set
# CONFIG_PREEMPT is not set
CONFIG_GART_IOMMU=y
CONFIG_SWIOTLB=y
CONFIG_X86_MCE=y

#
# Power management options
#
CONFIG_PM=y

#
# ACPI (Advanced Configuration and Power Interface) Support
#
# CONFIG_ACPI is not set
CONFIG_ACPI_BOOT=y

#
# CPU Frequency scaling
#
# CONFIG_CPU_FREQ is not set

#
# Bus options (PCI etc.)
#
CONFIG_PCI=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
# CONFIG_PCI_LEGACY_PROC is not set
# CONFIG_PCI_NAMES is not set

#
# Executable file formats / Emulations
#
CONFIG_BINFMT_ELF=y
# CONFIG_BINFMT_MISC is not set
# CONFIG_IA32_EMULATION is not set

#
# Device Drivers
#

#
# Generic Driver Options
#

#
# Memory Technology Devices (MTD)
#
# CONFIG_MTD is not set

#
# Parallel port support
#
# CONFIG_PARPORT is not set

#
# Plug and Play support
#

#
# Block devices
#
CONFIG_BLK_DEV_FD=y
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
CONFIG_BLK_DEV_LOOP=y
# CONFIG_BLK_DEV_CRYPTOLOOP is not set
# CONFIG_BLK_DEV_CARMEL is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_SIZE=4096
CONFIG_BLK_DEV_INITRD=y
CONFIG_LBD=y

#
# ATA/ATAPI/MFM/RLL support
#
CONFIG_IDE=y
CONFIG_BLK_DEV_IDE=y

#
# Please see Documentation/ide.txt for help/info on IDE drives
#
# CONFIG_BLK_DEV_HD_IDE is not set
CONFIG_BLK_DEV_IDEDISK=y
CONFIG_IDEDISK_MULTI_MODE=y
# CONFIG_IDEDISK_STROKE is not set
CONFIG_BLK_DEV_IDECD=y
# CONFIG_BLK_DEV_IDETAPE is not set
# CONFIG_BLK_DEV_IDEFLOPPY is not set
# CONFIG_BLK_DEV_IDESCSI is not set
# CONFIG_IDE_TASK_IOCTL is not set
# CONFIG_IDE_TASKFILE_IO is not set

#
# IDE chipset support/bugfixes
#
CONFIG_IDE_GENERIC=y
# CONFIG_BLK_DEV_CMD640 is not set
CONFIG_BLK_DEV_IDEPCI=y
# CONFIG_IDEPCI_SHARE_IRQ is not set
# CONFIG_BLK_DEV_OFFBOARD is not set
# CONFIG_BLK_DEV_GENERIC is not set
# CONFIG_BLK_DEV_OPTI621 is not set
# CONFIG_BLK_DEV_RZ1000 is not set
CONFIG_BLK_DEV_IDEDMA_PCI=y
# CONFIG_BLK_DEV_IDEDMA_FORCED is not set
CONFIG_IDEDMA_PCI_AUTO=y
# CONFIG_IDEDMA_ONLYDISK is not set
CONFIG_BLK_DEV_ADMA=y
# CONFIG_BLK_DEV_AEC62XX is not set
# CONFIG_BLK_DEV_ALI15X3 is not set
CONFIG_BLK_DEV_AMD74XX=y
# CONFIG_BLK_DEV_ATIIXP is not set
# CONFIG_BLK_DEV_CMD64X is not set
# CONFIG_BLK_DEV_TRIFLEX is not set
# CONFIG_BLK_DEV_CY82C693 is not set
# CONFIG_BLK_DEV_CS5520 is not set
# CONFIG_BLK_DEV_CS5530 is not set
# CONFIG_BLK_DEV_HPT34X is not set
# CONFIG_BLK_DEV_HPT366 is not set
# CONFIG_BLK_DEV_SC1200 is not set
CONFIG_BLK_DEV_PIIX=y
# CONFIG_BLK_DEV_NS87415 is not set
# CONFIG_BLK_DEV_PDC202XX_OLD is not set
# CONFIG_BLK_DEV_PDC202XX_NEW is not set
# CONFIG_BLK_DEV_SVWKS is not set
# CONFIG_BLK_DEV_SIIMAGE is not set
# CONFIG_BLK_DEV_SIS5513 is not set
# CONFIG_BLK_DEV_SLC90E66 is not set
# CONFIG_BLK_DEV_TRM290 is not set
# CONFIG_BLK_DEV_VIA82CXXX is not set
CONFIG_BLK_DEV_IDEDMA=y
# CONFIG_IDEDMA_IVB is not set
CONFIG_IDEDMA_AUTO=y
# CONFIG_BLK_DEV_HD is not set

#
# SCSI device support
#
CONFIG_SCSI=y
# CONFIG_SCSI_PROC_FS is not set

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
# CONFIG_BLK_DEV_SR is not set
# CONFIG_CHR_DEV_SG is not set

#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs
#
# CONFIG_SCSI_MULTI_LUN is not set
# CONFIG_SCSI_REPORT_LUNS is not set
# CONFIG_SCSI_CONSTANTS is not set
# CONFIG_SCSI_LOGGING is not set

#
# SCSI Transport Attributes
#
# CONFIG_SCSI_SPI_ATTRS is not set
# CONFIG_SCSI_FC_ATTRS is not set

#
# SCSI low-level drivers
#
CONFIG_BLK_DEV_3W_XXXX_RAID=y
# CONFIG_SCSI_ACARD is not set
# CONFIG_SCSI_AACRAID is not set
# CONFIG_SCSI_AIC7XXX is not set
# CONFIG_SCSI_AIC7XXX_OLD is not set
# CONFIG_SCSI_AIC79XX is not set
# CONFIG_SCSI_ADVANSYS is not set
# CONFIG_SCSI_MEGARAID is not set
CONFIG_SCSI_SATA=y
# CONFIG_SCSI_SATA_SVW is not set
CONFIG_SCSI_ATA_PIIX=y
# CONFIG_SCSI_SATA_PROMISE is not set
# CONFIG_SCSI_SATA_SIL is not set
# CONFIG_SCSI_SATA_SIS is not set
CONFIG_SCSI_SATA_VIA=y
# CONFIG_SCSI_SATA_VITESSE is not set
# CONFIG_SCSI_BUSLOGIC is not set
# CONFIG_SCSI_CPQFCTS is not set
# CONFIG_SCSI_DMX3191D is not set
# CONFIG_SCSI_EATA is not set
# CONFIG_SCSI_EATA_PIO is not set
# CONFIG_SCSI_FUTURE_DOMAIN is not set
# CONFIG_SCSI_GDTH is not set
# CONFIG_SCSI_IPS is not set
# CONFIG_SCSI_INIA100 is not set
# CONFIG_SCSI_SYM53C8XX_2 is not set
# CONFIG_SCSI_QLOGIC_ISP is not set
# CONFIG_SCSI_QLOGIC_FC is not set
# CONFIG_SCSI_QLOGIC_1280 is not set
CONFIG_SCSI_QLA2XXX=y
# CONFIG_SCSI_QLA21XX is not set
# CONFIG_SCSI_QLA22XX is not set
# CONFIG_SCSI_QLA2300 is not set
# CONFIG_SCSI_QLA2322 is not set
# CONFIG_SCSI_QLA6312 is not set
# CONFIG_SCSI_QLA6322 is not set
# CONFIG_SCSI_DC395x is not set
# CONFIG_SCSI_DC390T is not set
# CONFIG_SCSI_DEBUG is not set

#
# Multi-device support (RAID and LVM)
#
# CONFIG_MD is not set

#
# Fusion MPT device support
#
CONFIG_FUSION=y
CONFIG_FUSION_MAX_SGE=40

#
# IEEE 1394 (FireWire) support
#
# CONFIG_IEEE1394 is not set

#
# I2O device support
#

#
# Networking support
#
# CONFIG_NET is not set
# CONFIG_NETPOLL is not set
# CONFIG_NET_POLL_CONTROLLER is not set

#
# ISDN subsystem
#

#
# Telephony Support
#
# CONFIG_PHONE is not set

#
# Input device support
#
CONFIG_INPUT=y

#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
# CONFIG_INPUT_JOYDEV is not set
# CONFIG_INPUT_TSDEV is not set
CONFIG_INPUT_EVDEV=y
# CONFIG_INPUT_EVBUG is not set

#
# Input I/O drivers
#
# CONFIG_GAMEPORT is not set
CONFIG_SOUND_GAMEPORT=y
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
# CONFIG_SERIO_SERPORT is not set
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PCIPS2 is not set

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_KEYBOARD_NEWTON is not set
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=y
# CONFIG_MOUSE_SERIAL is not set
# CONFIG_MOUSE_VSXXXAA is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
# CONFIG_INPUT_MISC is not set

#
# Character devices
#
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_SERIAL_NONSTANDARD is not set

#
# Serial drivers
#
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_8250_NR_UARTS=4
# CONFIG_SERIAL_8250_EXTENDED is not set

#
# Non-8250 serial port support
#
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
CONFIG_UNIX98_PTYS=y
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256
# CONFIG_QIC02_TAPE is not set

#
# IPMI
#
# CONFIG_IPMI_HANDLER is not set

#
# Watchdog Cards
#
# CONFIG_WATCHDOG is not set
CONFIG_HW_RANDOM=y
# CONFIG_NVRAM is not set
CONFIG_RTC=y
# CONFIG_DTLK is not set
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set

#
# Ftape, the floppy tape device driver
#
# CONFIG_FTAPE is not set
CONFIG_AGP=y
CONFIG_AGP_AMD64=y
# CONFIG_AGP_INTEL_MCH is not set
# CONFIG_DRM is not set
# CONFIG_MWAVE is not set
CONFIG_RAW_DRIVER=y
CONFIG_MAX_RAW_DEVS=256
CONFIG_HANGCHECK_TIMER=y

#
# I2C support
#
# CONFIG_I2C is not set

#
# Misc devices
#
# CONFIG_IBM_ASM is not set

#
# Multimedia devices
#
# CONFIG_VIDEO_DEV is not set

#
# Digital Video Broadcasting Devices
#

#
# Graphics support
#
# CONFIG_FB is not set
# CONFIG_VIDEO_SELECT is not set

#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
# CONFIG_MDA_CONSOLE is not set
CONFIG_DUMMY_CONSOLE=y

#
# Sound
#
CONFIG_SOUND=y

#
# Advanced Linux Sound Architecture
#
# CONFIG_SND is not set

#
# Open Sound System
#
CONFIG_SOUND_PRIME=y
# CONFIG_SOUND_BT878 is not set
# CONFIG_SOUND_CMPCI is not set
# CONFIG_SOUND_EMU10K1 is not set
# CONFIG_SOUND_FUSION is not set
# CONFIG_SOUND_CS4281 is not set
# CONFIG_SOUND_ES1370 is not set
# CONFIG_SOUND_ES1371 is not set
# CONFIG_SOUND_ESSSOLO1 is not set
# CONFIG_SOUND_MAESTRO is not set
# CONFIG_SOUND_MAESTRO3 is not set
CONFIG_SOUND_ICH=y
# CONFIG_SOUND_SONICVIBES is not set
# CONFIG_SOUND_TRIDENT is not set
# CONFIG_SOUND_VIA82CXXX is not set
# CONFIG_SOUND_OSS is not set
# CONFIG_SOUND_ALI5455 is not set
# CONFIG_SOUND_FORTE is not set
# CONFIG_SOUND_RME96XX is not set
# CONFIG_SOUND_AD1980 is not set

#
# USB support
#
# CONFIG_USB is not set

#
# USB Gadget Support
#
# CONFIG_USB_GADGET is not set

#
# Firmware Drivers
#
# CONFIG_EDD is not set

#
# File systems
#
# CONFIG_EXT2_FS is not set
# CONFIG_EXT3_FS is not set
# CONFIG_JBD is not set
# CONFIG_REISERFS_FS is not set
# CONFIG_JFS_FS is not set
# CONFIG_XFS_FS is not set
# CONFIG_MINIX_FS is not set
# CONFIG_ROMFS_FS is not set
# CONFIG_QUOTA is not set
# CONFIG_AUTOFS_FS is not set
# CONFIG_AUTOFS4_FS is not set

#
# CD-ROM/DVD Filesystems
#
# CONFIG_ISO9660_FS is not set
# CONFIG_UDF_FS is not set

#
# DOS/FAT/NT Filesystems
#
# CONFIG_FAT_FS is not set
# CONFIG_NTFS_FS is not set

#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_SYSFS=y
# CONFIG_DEVFS_FS is not set
# CONFIG_DEVPTS_FS_XATTR is not set
CONFIG_TMPFS=y
# CONFIG_HUGETLBFS is not set
# CONFIG_HUGETLB_PAGE is not set
CONFIG_RAMFS=y

#
# Miscellaneous filesystems
#
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_CRAMFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set

#
# Partition Types
#
# CONFIG_PARTITION_ADVANCED is not set
CONFIG_MSDOS_PARTITION=y

#
# Native Language Support
#
# CONFIG_NLS is not set

#
# Profiling support
#
# CONFIG_PROFILING is not set

#
# Kernel hacking
#
# CONFIG_DEBUG_KERNEL is not set
# CONFIG_MAGIC_SYSRQ is not set
CONFIG_FRAME_POINTER=y

#
# Security options
#
# CONFIG_SECURITY is not set

#
# Cryptographic options
#
# CONFIG_CRYPTO is not set

#
# Library routines
#
CONFIG_CRC32=y
# CONFIG_LIBCRC32C is not set
---

Re: [Fastboot] Re: kdump on non-boot cpu

[Eric W. Biederman]

Itsuro Oda <oda@valinux.co.jp> writes:

> Hi,
> 
> On 03 Feb 2005 02:58:02 -0700
> ebiederm@xmission.com (Eric W. Biederman) wrote:
> 
> > Itsuro Oda <oda@valinux.co.jp> writes:
> > 
> > > Hi,
> > > 
> > > This is not for kdump but an experience of our project(mkdump).
> > > The dump kernel(not SMP config) boot hangs if machine_kexec()
> > > excutes on non-boot CPU on x86_64 platform.
> > 
> > ?? x86_64 is Opteron cpu, amd64, Intel cpu?
> > Are the kernels running in 32bit or 64bit mode. I'm guessing
> > SMP Opterons running in 32bit mode.
> 
> SMP Opterons running in 64bit mode. (The normal kernel is SMP configed.
> The dump kernel is not SMP configed.)

The reason I was asking and assuming you had a 32bit kernel is that
you were quoting pieces of arch/i386/kernel/crash.c instead of
arch/x86_64/kernel/crash.c

While the functionality can easily be transfered I didn't think
any one had done that yet.

> > Anyway one thing I want to do is actually drop the apic shutdown
> > code altogether in this code path.  
> 
> It sounds nice. (if available)

All that has to happen is to drop the x86-crash_shutdown-apic-shutdown.patch
from Andrews tree.  That patch just added the handful of lines that
disabled the apics.   Once I get a test case that someone can boot a
kernel without disabling apics I will ask Andrew to drop the above
mentioned patch.

> > My best hunch is that your UP kernel is not getting interrupts.
> 
> yes. It seems the timer interrupts is not comming up.
> 
> > Any chance on getting a serial console boot log?  
> > 
> attached. both success case and unsuccess case.

Thanks.

> > I suspect it can be made to work if you compile your UP
> > kernel with IOAPIC support.  I do know the table parsers
> > no longer complain about the configuration.
> 
> with IOAPIC support. the config is attached.

Ok. Thanks.  This is a legitimate bug.  And it is probably the reason
I even care about the non-SMP interrupt case some days.  The problem
is that the kernel just assumes interrupts are setup in non-APIC mode
when it starts booting, and quite possibly only the bootstrap cpu can
see those interrupts. 

So I believe the fix needs to be to enable apics before we calibrate
the delay timer.  I'm not certain off the top of my head what that
patch will look like but it should not be fundamentally hard.  
With that code in place we also don't need to do any APIC shutdown
as the kernel knows enough to completely setup the apics.

Eric

Re: [Fastboot] Re: kdump on non-boot cpu

[Itsuro Oda]

Hi,

> The reason I was asking and assuming you had a 32bit kernel is that
> you were quoting pieces of arch/i386/kernel/crash.c instead of
> arch/x86_64/kernel/crash.c

Using "arch/i386/kernel/crash.c" is just for explanation how we avoid
the hang. (I found x86_64 kdump is not supported in 2.6.11-rc2-mm1 :-))

The attached log is a log of running mkdump (it supports x86_64). not kdump.
The basic procedure before jumping new kernel is almost same as kdump.
So I inform this infromation to you since I think it may be helpfull 
for kdump development.

> Ok. Thanks.  This is a legitimate bug.  And it is probably the reason
> I even care about the non-SMP interrupt case some days.  The problem
> is that the kernel just assumes interrupts are setup in non-APIC mode
> when it starts booting, and quite possibly only the bootstrap cpu can
> see those interrupts. 
> 
> So I believe the fix needs to be to enable apics before we calibrate
> the delay timer.  I'm not certain off the top of my head what that
> patch will look like but it should not be fundamentally hard.  
> With that code in place we also don't need to do any APIC shutdown
> as the kernel knows enough to completely setup the apics.

I see. Thank you for your explanation.

Thanks.
-- 
Itsuro ODA <oda@valinux.co.jp>

Re: [Fastboot] Re: kdump on non-boot cpu

[Eric W. Biederman]

Itsuro Oda <oda@valinux.co.jp> writes:

> > So I believe the fix needs to be to enable apics before we calibrate
> > the delay timer.  I'm not certain off the top of my head what that
> > patch will look like but it should not be fundamentally hard.  
> > With that code in place we also don't need to do any APIC shutdown
> > as the kernel knows enough to completely setup the apics.
> 
> I see. Thank you for your explanation.

I have done a bit more digging.

For some reason, likely historical we don't initialize the
IO_APIC in init_IRQ().  Instead we wait until smp_prepare_cpus() or
smp_init().  Both functions are called much later in the init process
than calibrate_delay().

Given the separation that has happened between apics and SMP it should
be possible to initialize the local apic and the IO_APIC of the boot
cpu much earlier in the game.  It looks like it may take some heavy
lifting though.

Eric