Re: Linux as 32-bit Dom0?

Re: Linux as 32-bit Dom0?

[Juergen Gross]

On 22/11/17 15:05, Jan Beulich wrote:
> Jürgen, Boris,
> 
> am I trying something that's not allowed, but selectable via Kconfig?
> On system with multiple IO-APICs (I assume that's what triggers the
> problem) I get
> 
> Kernel panic - not syncing: Max apic_id exceeded!

Generally I don't think 32 bit dom0 is forbidden, but rarely used. I
wouldn't be too sad in case we'd decide to drop that support. ;-)

Can you please be a little bit more specific?

How many IOAPICs? From the code I guess this is an INTEL system with not
too recent IOAPIC versions (<0x14)?

Having a little bit more of the boot log might help, too.


Juergen

> 
> CPU: 0 PID: 0 Comm: swapper Not tainted 4.14.1-2017-11-21-xen0 #6
> Hardware name: ...
> Call Trace:
>  ? show_stack+0x20/0x50
>  ? dump_stack+0x7e/0xc0
>  ? panic+0x99/0x220
>  ? io_apic_get_unique_id+0x207/0x210
>  ? __raw_callee_save_xen_restore_fl+0x6/0x8
>  ? xen_flush_tlb_single+0x6f/0x80
>  ? set_pte_vaddr+0xef/0x110
>  ? xen_io_apic_read+0x36/0x90
>  ? mp_register_ioapic+0x2b7/0x410
>  ? acpi_os_map_iomem+0x14d/0x210
>  ? acpi_parse_ioapic+0x6b/0x6f
>  ? acpi_parse_entries_array+0xa1/0x16d
>  ? acpi_tb_get_table+0x95/0x9d
>  ? acpi_ut_release_mutex+0x109/0x10e
>  ? acpi_table_parse_entries_array+0x98/0xa8
>  ? acpi_table_parse_entries+0x30/0x35
>  ? acpi_boot_init+0x65/0x65
>  ? acpi_table_parse_madt+0x1b/0x1f
>  ? acpi_boot_init+0x65/0x65
>  ? acpi_parse_madt_ioapic_entries+0x4a/0x119
>  ? mutex_lock+0x8/0x30
>  ? acpi_process_madt+0xbe/0x105
>  ? acpi_boot_init+0x3d/0x65
>  ? setup_arch+0x67a/0x83f
>  ? 0xc1000000
>  ? start_kernel+0x46/0x361
>  ? x86_early_init_platform_quirks+0x4d/0x90
>  ? i386_start_kernel+0x22/0x88
>  ? xen_start_kernel+0x453/0x639
> (XEN) Hardware Dom0 crashed: 'noreboot' set - not rebooting.
> 
> I admit I have a few custom patches in that tree, but I'm reasonably
> certain that none of them comes even close to having such an effect.
> 
> Jan
> 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Linux as 32-bit Dom0?

[Jan Beulich]

[-- Attachment #1: Type: text/plain, Size: 837 bytes --]

>>> On 22.11.17 at 15:40, <jgross@suse.com> wrote:
> On 22/11/17 15:05, Jan Beulich wrote:
>> Jürgen, Boris,
>> 
>> am I trying something that's not allowed, but selectable via Kconfig?
>> On system with multiple IO-APICs (I assume that's what triggers the
>> problem) I get
>> 
>> Kernel panic - not syncing: Max apic_id exceeded!
> 
> Generally I don't think 32 bit dom0 is forbidden, but rarely used. I
> wouldn't be too sad in case we'd decide to drop that support. ;-)
> 
> Can you please be a little bit more specific?
> 
> How many IOAPICs? From the code I guess this is an INTEL system with not
> too recent IOAPIC versions (<0x14)?
> 
> Having a little bit more of the boot log might help, too.

Full log attached, which should answer all questions. This is
a Haswell system, so not too old an IO-APIC flavor I would say.

Jan

[-- Attachment #2: minicom.Linux-4.14 --]
[-- Type: application/octet-stream, Size: 22258 bytes --]

 __  __            _  _    _  ___                     _        _     _      
 \ \/ /___ _ __   | || |  / |/ _ \    _   _ _ __  ___| |_ __ _| |__ | | ___ 
  \  // _ \ '_ \  | || |_ | | | | |__| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
  /  \  __/ | | | |__   _|| | |_| |__| |_| | | | \__ \ || (_| | |_) | |  __/
 /_/\_\___|_| |_|    |_|(_)_|\___/    \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
                                                                            
(XEN) Xen version 4.10-unstable (jbeulich@dus.novell.com) (gccx (GCC) 7.2.0) debug=n  Wed Nov 22 13:54:24 CET 2017
(XEN) Latest ChangeSet: 36560
(XEN) Bootloader: GNU GRUB 0.97
(XEN) Command line: com1=115200,,pci,18 serial_tx_buffer=128k conswitch=qx vga=mode-0x31a dom0_mem=-255M ucode=2 loglvl=all guest_loglvl=all noreboot cpufreq=xen:up_threshold=40 console=vga,com1
(XEN) Xen image load base address: 0
(XEN) Video information:
(XEN)  VGA is graphics mode 1280x1024, 16 bpp
(XEN)  VBE/DDC methods: none; EDID transfer time: 0 seconds
(XEN)  EDID info not retrieved because no DDC retrieval method detected
(XEN) Disc information:
(XEN)  Found 2 MBR signatures
(XEN)  Found 2 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000009e800 (usable)
(XEN)  000000000009e800 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 00000000c93f0000 (usable)
(XEN)  00000000c93f0000 - 00000000c9f8d000 (reserved)
(XEN)  00000000c9f8d000 - 00000000c9fdf000 (ACPI data)
(XEN)  00000000c9fdf000 - 00000000cac83000 (ACPI NVS)
(XEN)  00000000cac83000 - 00000000cb173000 (reserved)
(XEN)  00000000cb173000 - 00000000cb174000 (usable)
(XEN)  00000000cb174000 - 00000000cb182000 (reserved)
(XEN)  00000000cb182000 - 00000000cd000000 (usable)
(XEN)  00000000cd000000 - 00000000ce000000 (reserved)
(XEN)  00000000d0000000 - 00000000e0000000 (reserved)
(XEN)  00000000fed1c000 - 00000000fed20000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 0000000430000000 (usable)
(XEN) New Xen image base address: 0xcca00000
(XEN) ACPI: RSDP 000F0540, 0024 (r2 DELL  )
(XEN) ACPI: XSDT C9F9D098, 00B4 (r1 DELL    CBX3     1072009 AMI     10013)
(XEN) ACPI: FACP C9FC8EC8, 010C (r5 DELL    CBX3     1072009 AMI     10013)
(XEN) ACPI: DSDT C9F9D1E0, 2BCE1 (r2 DELL    CBX3     1072009 INTL 20091013)
(XEN) ACPI: FACS CAC7FF80, 0040
(XEN) ACPI: APIC C9FC8FD8, 0224 (r3 DELL    CBX3     1072009 AMI     10013)
(XEN) ACPI: FPDT C9FC9200, 0044 (r1 DELL    CBX3     1072009 AMI     10013)
(XEN) ACPI: FIDT C9FC9248, 009C (r1 DELL    CBX3     1072009 AMI     10013)
(XEN) ACPI: MCFG C9FC92E8, 003C (r1 DELL    CBX3     1072009 MSFT       97)
(XEN) ACPI: UEFI C9FC9328, 0042 (r1 INTEL  EDK2            2       1000013)
(XEN) ACPI: HPET C9FC9370, 0038 (r1 DELL    CBX3           1 INTL 20091013)
(XEN) ACPI: MSCT C9FC93A8, 0090 (r1 DELL    CBX3           1 INTL 20091013)
(XEN) ACPI: SLIT C9FC9438, 0030 (r1 DELL    CBX3           1 INTL 20091013)
(XEN) ACPI: SRAT C9FC9468, 1158 (r3 DELL    CBX3           1 INTL 20091013)
(XEN) ACPI: SRAT C9FCA5C0, 2358 (r3 DELL    CBX3           2 INTL 20091013)
(XEN) ACPI: WDDT C9FCC918, 0040 (r1 DELL    CBX3           0 INTL 20091013)
(XEN) ACPI: SSDT C9FCC958, 11CD4 (r1 DELL      PmMgt        1 INTL 20120913)
(XEN) ACPI: NITR C9FDE630, 0071 (r2 DELL    CBX3           1 INTL 20091013)
(XEN) ACPI: SLIC C9FDE6A8, 0176 (r3 DELL    CBX3     1072009 MSFT    10013)
(XEN) ACPI: MSDM C9FDE820, 0055 (r3 DELL    CBX3     6222004 AMI     10013)
(XEN) ACPI: DMAR C9FDE878, 0118 (r1 DELL    CBX3           1 INTL 20091013)
(XEN) ACPI: ASF! C9FDE990, 00A0 (r32 INTEL       HCG        1 TFSM    F4240)
(XEN) System RAM: 16306MB (16697396kB)
(XEN) SRAT: PXM 0 -> APIC 00 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 02 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 04 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 06 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 08 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 0a -> Node 0
(XEN) SRAT: PXM 0 -> APIC 0c -> Node 0
(XEN) SRAT: PXM 0 -> APIC 0e -> Node 0
(XEN) SRAT: PXM 1 -> APIC 10 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 12 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 14 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 16 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 18 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 1a -> Node 1
(XEN) SRAT: PXM 1 -> APIC 1c -> Node 1
(XEN) SRAT: PXM 1 -> APIC 1e -> Node 1
(XEN) SRAT: PXM 0 -> APIC 01 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 03 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 05 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 07 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 09 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 0b -> Node 0
(XEN) SRAT: PXM 0 -> APIC 0d -> Node 0
(XEN) SRAT: PXM 0 -> APIC 0f -> Node 0
(XEN) SRAT: PXM 1 -> APIC 11 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 13 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 15 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 17 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 19 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 1b -> Node 1
(XEN) SRAT: PXM 1 -> APIC 1d -> Node 1
(XEN) SRAT: PXM 1 -> APIC 1f -> Node 1
(XEN) SRAT: Node 0 PXM 0 0-d0000000
(XEN) SRAT: Node 0 PXM 0 100000000-230000000
(XEN) SRAT: Node 1 PXM 1 230000000-430000000
(XEN) NUMA: Allocated memnodemap from 42ec6e000 - 42ec6f000
(XEN) NUMA: Using 16 for the hash shift.
(XEN) Domain heap initialised DMA width 32 bits
(XEN) vesafb: framebuffer at 0xf1000000, mapped to 0xffff82c000201000, using 4096k, total 14336k
(XEN) vesafb: mode is 1280x1024x16, linelength=2560, font 8x16
(XEN) vesafb: Truecolor: size=0:5:6:5, shift=0:11:5:0
(XEN) CPU Vendor: Intel, Family 6 (0x6), Model 63 (0x3f), Stepping 2 (raw 000306f2)
(XEN) found SMP MP-table at 000fdcd0
(XEN) DMI 2.7 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x408 (32 bits)
(XEN) ACPI: v5 SLEEP INFO: control[0:0], status[0:0]
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:404,1:0], pm1x_evt[1:400,1:0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT - cac7ff80/0000000000000000, using 32
(XEN) ACPI:             wakeup_vec[cac7ff8c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x04] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x06] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x08] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x0a] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x0c] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x0e] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x10] lapic_id[0x10] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x12] lapic_id[0x12] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x14] lapic_id[0x14] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x16] lapic_id[0x16] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x18] lapic_id[0x18] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x1a] lapic_id[0x1a] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x1c] lapic_id[0x1c] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x1e] lapic_id[0x1e] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x05] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x07] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x09] lapic_id[0x09] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x0b] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x0d] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x0f] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x11] lapic_id[0x11] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x13] lapic_id[0x13] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x15] lapic_id[0x15] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x17] lapic_id[0x17] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x19] lapic_id[0x19] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x1b] lapic_id[0x1b] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x1d] lapic_id[0x1d] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x1f] lapic_id[0x1f] enabled)
(XEN) Overriding APIC driver with bigsmp
(XEN) ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 1, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec01000] gsi_base[24])
(XEN) IOAPIC[1]: apic_id 2, version 32, address 0xfec01000, GSI 24-47
(XEN) ACPI: IOAPIC (id[0x03] address[0xfec40000] gsi_base[48])
(XEN) IOAPIC[2]: apic_id 3, version 32, address 0xfec40000, GSI 48-71
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Phys.  Using 3 I/O APICs
(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 32 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 72 GSI, 6088 MSI/MSI-X
(XEN) Not enabling x2APIC (upon firmware request)
(XEN) microcode: CPU0 updated from revision 0x39 to 0x3a, date = 2017-01-30 
(XEN) xstate: size: 0x340 and states: 0x7
(XEN) CMCI: threshold 0x2 too large for CPU0 bank 17, using 0x1
(XEN) CMCI: threshold 0x2 too large for CPU0 bank 18, using 0x1
(XEN) CMCI: threshold 0x2 too large for CPU0 bank 19, using 0x1
(XEN) CPU0: Intel machine check reporting enabled
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Platform timer is 14.318MHz HPET
(XEN) Detected 2394.461 MHz processor.
(XEN) Initing memory sharing.
(XEN) alt table ffff82d080400ed8 -> ffff82d0804034d0
(XEN) PCI: MCFG configuration 0: base d0000000 segment 0000 buses 00 - ff
(XEN) PCI: MCFG area at d0000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-ff
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d iommu 2 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d Snoop Control not enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Posted Interrupt not enabled.
(XEN) Intel VT-d Shared EPT tables enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) nr_sockets: 2
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) TSC deadline timer enabled
(XEN) Defaulting to alternative key handling; send 'A' to switch to normal mode.
(XEN) Allocated console ring of 256 KiB.
(XEN) mwait-idle: MWAIT substates: 0x2120
(XEN) mwait-idle: v0.4.1 model 0x3f
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN)  - APIC Register Virtualization
(XEN)  - Virtual Interrupt Delivery
(XEN)  - Posted Interrupt Processing
(XEN)  - VMCS shadowing
(XEN)  - VM Functions
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) microcode: CPU2 updated from revision 0x39 to 0x3a, date = 2017-01-30 
(XEN) microcode: CPU4 updated from revision 0x39 to 0x3a, date = 2017-01-30 
(XEN) microcode: CPU6 updated from revision 0x39 to 0x3a, date = 2017-01-30 
(XEN) microcode: CPU8 updated from revision 0x39 to 0x3a, date = 2017-01-30 
(XEN) microcode: CPU10 updated from revision 0x39 to 0x3a, date = 2017-01-30 
(XEN) microcode: CPU12 updated from revision 0x39 to 0x3a, date = 2017-01-30 
(XEN) microcode: CPU14 updated from revision 0x39 to 0x3a, date = 2017-01-30 
(XEN) microcode: CPU16 updated from revision 0x39 to 0x3a, date = 2017-01-30 
(XEN) CMCI: threshold 0x2 too large for CPU16 bank 17, using 0x1
(XEN) CMCI: threshold 0x2 too large for CPU16 bank 18, using 0x1
(XEN) CMCI: threshold 0x2 too large for CPU16 bank 19, using 0x1
(XEN) microcode: CPU18 updated from revision 0x39 to 0x3a, date = 2017-01-30 
(XEN) microcode: CPU20 updated from revision 0x39 to 0x3a, date = 2017-01-30 
(XEN) microcode: CPU22 updated from revision 0x39 to 0x3a, date = 2017-01-30 
(XEN) microcode: CPU24 updated from revision 0x39 to 0x3a, date = 2017-01-30 
(XEN) microcode: CPU26 updated from revision 0x39 to 0x3a, date = 2017-01-30 
(XEN) microcode: CPU28 updated from revision 0x39 to 0x3a, date = 2017-01-30 
(XEN) microcode: CPU30 updated from revision 0x39 to 0x3a, date = 2017-01-30 
(XEN) Brought up 32 CPUs
(XEN) build-id: fa9c939e0ef8c24ca0b42aed0a797d5770fd37a3
(XEN) TSC warp detected, disabling TSC_RELIABLE
(XEN) ACPI sleep modes: S3
(XEN) VPMU: disabled
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) CMOS aliased at 74, index r/w
(XEN) Dom0 has maximum 1224 PIRQs
(XEN) NX (Execute Disable) protection active
(XEN) *** LOADING DOMAIN 0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 32-bit, PAE, lsb, paddr 0x1000000 -> 0x18a2000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000420000000->0000000424000000 (4032665 pages to be allocated)
(XEN)  Init. ramdisk: 000000042f765000->000000042fff7088
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: 00000000c1000000->00000000c18a2000
(XEN)  Init. ramdisk: 0000000000000000->0000000000000000
(XEN)  Phys-Mach map: 00000000c18a2000->00000000c28164b0
(XEN)  Start info:    00000000c2817000->00000000c28174b4
(XEN)  Page tables:   00000000c2818000->00000000c2833000
(XEN)  Boot stack:    00000000c2833000->00000000c2834000
(XEN)  TOTAL:         00000000c0000000->00000000c2c00000
(XEN)  ENTRY ADDRESS: 00000000c15971b8
(XEN) Dom0 has maximum 32 VCPUs
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Scrubbing Free RAM on 2 nodes using 16 CPUs
(XEN) .........done.
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> Xen (type 'CTRL-q' three times to switch input to DOM0)
(XEN) Freed 468kB init memory
mapping kernel into physical memory
about to get started...
Reserving virtual address space above 0xfec00000 (rounded to 0xfec00000)
Linux version 4.14.1-2017-11-21-xen0 (jbeulich@dus-dev-sles11d) (gcc version 4.3.4 [gcc-4_3-branch revision 152973] (SUSE Linux)) #6 SMP Wed Nov 22 14:47:29 CET 2017
KERNEL supported cpus:
  Intel GenuineIntel
x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
Released 0 page(s)
e820: BIOS-provided physical RAM map:
Xen: [mem 0x0000000000000000-0x000000000009dfff] usable
Xen: [mem 0x000000000009e800-0x00000000000fffff] reserved
Xen: [mem 0x0000000000100000-0x00000000c93effff] usable
Xen: [mem 0x00000000c93f0000-0x00000000c9f8cfff] reserved
Xen: [mem 0x00000000c9f8d000-0x00000000c9fdefff] ACPI data
Xen: [mem 0x00000000c9fdf000-0x00000000cac82fff] ACPI NVS
Xen: [mem 0x00000000cac83000-0x00000000cb172fff] reserved
Xen: [mem 0x00000000cb173000-0x00000000cb173fff] usable
Xen: [mem 0x00000000cb174000-0x00000000cb181fff] reserved
Xen: [mem 0x00000000cb182000-0x00000000ccffffff] usable
Xen: [mem 0x00000000cd000000-0x00000000cdffffff] reserved
Xen: [mem 0x00000000d0000000-0x00000000dfffffff] reserved
Xen: [mem 0x00000000f3ffc000-0x00000000f3ffdfff] reserved
Xen: [mem 0x00000000f7ffc000-0x00000000f7ffcfff] reserved
Xen: [mem 0x00000000fec00000-0x00000000fec01fff] reserved
Xen: [mem 0x00000000fec40000-0x00000000fec40fff] reserved
Xen: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved
Xen: [mem 0x00000000fee00000-0x00000000feefffff] reserved
Xen: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
Xen: [mem 0x0000000100000000-0x000000042fffffff] usable
bootconsole [xenboot0] enabled
NX (Execute Disable) protection: active
e820: user-defined physical RAM map:
user: [mem 0x0000000000000000-0x000000000009dfff] usable
user: [mem 0x000000000009e800-0x00000000000fffff] reserved
user: [mem 0x0000000000100000-0x00000000c93effff] usable
user: [mem 0x00000000c93f0000-0x00000000c9f8cfff] reserved
user: [mem 0x00000000c9f8d000-0x00000000c9fdefff] ACPI data
user: [mem 0x00000000c9fdf000-0x00000000cac82fff] ACPI NVS
user: [mem 0x00000000cac83000-0x00000000cb172fff] reserved
user: [mem 0x00000000cb173000-0x00000000cb173fff] usable
user: [mem 0x00000000cb174000-0x00000000cb181fff] reserved
user: [mem 0x00000000cb182000-0x00000000ccffffff] usable
user: [mem 0x00000000cd000000-0x00000000cdffffff] reserved
user: [mem 0x00000000d0000000-0x00000000dfffffff] reserved
user: [mem 0x00000000f3ffc000-0x00000000f3ffdfff] reserved
user: [mem 0x00000000f7ffc000-0x00000000f7ffcfff] reserved
user: [mem 0x00000000fec00000-0x00000000fec01fff] reserved
user: [mem 0x00000000fec40000-0x00000000fec40fff] reserved
user: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved
user: [mem 0x00000000fee00000-0x00000000feefffff] reserved
user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
user: [mem 0x0000000100000000-0x00000003ffffffff] usable
random: fast init done
SMBIOS 2.8 present.
DMI: Dell Inc. Precision Tower 7810/0GWHMW, BIOS A20 07/26/2017
Hypervisor detected: Xen PV
tsc: Fast TSC calibration using PIT
e820: last_pfn = 0x400000 max_arch_pfn = 0x1000000
MTRR: Disabled
x86/PAT: MTRRs disabled, skipping PAT initialization too.
x86/PAT: Configuration [0-7]: WB  WT  UC- UC  WC  WP  UC  UC  
RAMDISK: [mem 0x04000000-0x04892fff]
ACPI: Early table checksum verification disabled
ACPI: RSDP 0x00000000000F0540 000024 (v02 DELL  )
ACPI: XSDT 0x00000000C9F9D098 0000B4 (v01 DELL   CBX3     01072009 AMI  00010013)
ACPI: FACP 0x00000000C9FC8EC8 00010C (v05 DELL   CBX3     01072009 AMI  00010013)
ACPI: DSDT 0x00000000C9F9D1E0 02BCE1 (v02 DELL   CBX3     01072009 INTL 20091013)
ACPI: FACS 0x00000000CAC7FF80 000040
ACPI: APIC 0x00000000C9FC8FD8 000224 (v03 DELL   CBX3     01072009 AMI  00010013)
ACPI: FPDT 0x00000000C9FC9200 000044 (v01 DELL   CBX3     01072009 AMI  00010013)
ACPI: FIDT 0x00000000C9FC9248 00009C (v01 DELL   CBX3     01072009 AMI  00010013)
ACPI: MCFG 0x00000000C9FC92E8 00003C (v01 DELL   CBX3     01072009 MSFT 00000097)
ACPI: UEFI 0x00000000C9FC9328 000042 (v01 INTEL  EDK2     00000002      01000013)
ACPI: HPET 0x00000000C9FC9370 000038 (v01 DELL   CBX3     00000001 INTL 20091013)
ACPI: MSCT 0x00000000C9FC93A8 000090 (v01 DELL   CBX3     00000001 INTL 20091013)
ACPI: SLIT 0x00000000C9FC9438 000030 (v01 DELL   CBX3     00000001 INTL 20091013)
ACPI: SRAT 0x00000000C9FC9468 001158 (v03 DELL   CBX3     00000001 INTL 20091013)
ACPI: SRAT 0x00000000C9FCA5C0 002358 (v03 DELL   CBX3     00000002 INTL 20091013)
ACPI: WDDT 0x00000000C9FCC918 000040 (v01 DELL   CBX3     00000000 INTL 20091013)
ACPI: SSDT 0x00000000C9FCC958 011CD4 (v01 DELL   PmMgt    00000001 INTL 20120913)
ACPI: NITR 0x00000000C9FDE630 000071 (v02 DELL   CBX3     00000001 INTL 20091013)
ACPI: SLIC 0x00000000C9FDE6A8 000176 (v03 DELL   CBX3     01072009 MSFT 00010013)
ACPI: MSDM 0x00000000C9FDE820 000055 (v03 DELL   CBX3     06222004 AMI  00010013)
ACPI: RMAD 0x00000000C9FDE878 000118 (v01 DELL   CBX3     00000001 INTL 20091013)
ACPI: ASF! 0x00000000C9FDE990 0000A0 (v32 INTEL   HCG     00000001 TFSM 000F4240)
Switched to APIC driver `Xen PV'.
NUMA turned off
Faking a node at [mem 0x0000000000000000-0x00000003ffffffff]
NODE_DATA(0) allocated [mem 0x365f8000-0x365f8fff]
15514MB HIGHMEM available.
869MB LOWMEM available.
  mapped low ram: 0 - 365fe000
  low ram: 0 - 365fe000
Node: 0, start_pfn: 1, end_pfn: 9e
Node: 0, start_pfn: 100, end_pfn: c93f0
Node: 0, start_pfn: cb173, end_pfn: cb174
Node: 0, start_pfn: cb182, end_pfn: cd000
Node: 0, start_pfn: 100000, end_pfn: 400000
2c8000 2cc000 2d0000 2d4000 2d8000 2dc000 2e0000 2e4000 2e8000 2ec000 2f0000 2f4000 2f8000 2fc000 300000 304000 308000 30c000 310000 314000 318000 31c000 320000 324000 328000 32c000 330000 334000 338000 33c000 340000 344000 348000 34c000 350000 354000 358000 35c000 360000 364000 368000 36c000 370000 374000 378000 37c000 380000 384000 388000 38c000 390000 394000 398000 39c000 3a0000 3a4000 3a8000 3ac000 3b0000 3b4000 3b8000 3bc000 3c0000 3c4000 3c8000 3cc000 3d0000 3d4000 3d8000 3dc000 3e0000 3e4000 3e8000 3ec000 3f0000 3f4000 3f8000 3fc000 
Zone ranges:
  DMA      [mem 0x0000000000001000-0x0000000000ffffff]
  Normal   [mem 0x0000000001000000-0x00000000365fdfff]
  HighMem  [mem 0x00000000365fe000-0x00000003ffffffff]
Movable zone start for each node
Early memory node ranges
  node   0: [mem 0x0000000000001000-0x000000000009dfff]
  node   0: [mem 0x0000000000100000-0x00000000c93effff]
  node   0: [mem 0x00000000cb173000-0x00000000cb173fff]
  node   0: [mem 0x00000000cb182000-0x00000000ccffffff]
  node   0: [mem 0x0000000100000000-0x00000003ffffffff]
Initmem setup node 0 [mem 0x0000000000001000-0x00000003ffffffff]
p2m virtual area at f7000000, size is 1400000
Remapped 216563 page(s)
Using APIC driver Xen PV
ACPI: PM-Timer IO Port: 0x408
Kernel panic - not syncing: Max apic_id exceeded!

CPU: 0 PID: 0 Comm: swapper Not tainted 4.14.1-2017-11-21-xen0 #6
Hardware name: Dell Inc. Precision Tower 7810/0GWHMW, BIOS A20 07/26/2017
Call Trace:
 ? show_stack+0x20/0x50
 ? dump_stack+0x7e/0xc0
 ? panic+0x99/0x220
 ? io_apic_get_unique_id+0x207/0x210
 ? __raw_callee_save_xen_restore_fl+0x6/0x8
 ? xen_flush_tlb_single+0x6f/0x80
 ? set_pte_vaddr+0xef/0x110
 ? xen_io_apic_read+0x36/0x90
 ? mp_register_ioapic+0x2b7/0x410
 ? acpi_os_map_iomem+0x14d/0x210
 ? acpi_parse_ioapic+0x6b/0x6f
 ? acpi_parse_entries_array+0xa1/0x16d
 ? acpi_tb_get_table+0x95/0x9d
 ? acpi_ut_release_mutex+0x109/0x10e
 ? acpi_table_parse_entries_array+0x98/0xa8
 ? acpi_table_parse_entries+0x30/0x35
 ? acpi_boot_init+0x65/0x65
 ? acpi_table_parse_madt+0x1b/0x1f
 ? acpi_boot_init+0x65/0x65
 ? acpi_parse_madt_ioapic_entries+0x4a/0x119
 ? mutex_lock+0x8/0x30
 ? acpi_process_madt+0xbe/0x105
 ? acpi_boot_init+0x3d/0x65
 ? setup_arch+0x67a/0x83f
 ? 0xc1000000
 ? start_kernel+0x46/0x361
 ? x86_early_init_platform_quirks+0x4d/0x90
 ? i386_start_kernel+0x22/0x88
 ? xen_start_kernel+0x453/0x639
(XEN) Hardware Dom0 crashed: 'noreboot' set - not rebooting.
ÿÿÿÿ

[-- Attachment #3: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Linux as 32-bit Dom0?

[Boris Ostrovsky]

On 11/22/2017 09:40 AM, Juergen Gross wrote:
> On 22/11/17 15:05, Jan Beulich wrote:
>> Jürgen, Boris,
>>
>> am I trying something that's not allowed, but selectable via Kconfig?
>> On system with multiple IO-APICs (I assume that's what triggers the
>> problem) I get
>>
>> Kernel panic - not syncing: Max apic_id exceeded!
> Generally I don't think 32 bit dom0 is forbidden, but rarely used. I
> wouldn't be too sad in case we'd decide to drop that support. ;-)
>
> Can you please be a little bit more specific?
>
> How many IOAPICs? From the code I guess this is an INTEL system with not
> too recent IOAPIC versions (<0x14)?
>
> Having a little bit more of the boot log might help, too.


I just booted a 3-IOAPIC Intel box with a 32-bit 4.13 kernel.

(well, it actually didn't quite boot as it hang somewhere further down
the line, so that's a problem).

I do see

...
[    0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[    0.000000] IOAPIC[0]: apic_id 0 already used, trying 1
[    0.000000] IOAPIC[0]: Unable to change apic_id!
[    0.000000] IOAPIC[0]: apic_id 255, version 32, address 0xfec00000,
GSI 0-23
[    0.000000] IOAPIC[1]: apic_id 1 already used, trying 3
[    0.000000] IOAPIC[1]: Unable to change apic_id!
[    0.000000] IOAPIC[1]: apic_id 255, version 32, address 0xfec3f000,
GSI 24-47
[    0.000000] IOAPIC[2]: apic_id 2 already used, trying 5
[    0.000000] IOAPIC[2]: Unable to change apic_id!
[    0.000000] IOAPIC[2]: apic_id 255, version 32, address 0xfec7f000,
GSI 48-71


-boris



>
>
> Juergen
>
>> CPU: 0 PID: 0 Comm: swapper Not tainted 4.14.1-2017-11-21-xen0 #6
>> Hardware name: ...
>> Call Trace:
>>  ? show_stack+0x20/0x50
>>  ? dump_stack+0x7e/0xc0
>>  ? panic+0x99/0x220
>>  ? io_apic_get_unique_id+0x207/0x210
>>  ? __raw_callee_save_xen_restore_fl+0x6/0x8
>>  ? xen_flush_tlb_single+0x6f/0x80
>>  ? set_pte_vaddr+0xef/0x110
>>  ? xen_io_apic_read+0x36/0x90
>>  ? mp_register_ioapic+0x2b7/0x410
>>  ? acpi_os_map_iomem+0x14d/0x210
>>  ? acpi_parse_ioapic+0x6b/0x6f
>>  ? acpi_parse_entries_array+0xa1/0x16d
>>  ? acpi_tb_get_table+0x95/0x9d
>>  ? acpi_ut_release_mutex+0x109/0x10e
>>  ? acpi_table_parse_entries_array+0x98/0xa8
>>  ? acpi_table_parse_entries+0x30/0x35
>>  ? acpi_boot_init+0x65/0x65
>>  ? acpi_table_parse_madt+0x1b/0x1f
>>  ? acpi_boot_init+0x65/0x65
>>  ? acpi_parse_madt_ioapic_entries+0x4a/0x119
>>  ? mutex_lock+0x8/0x30
>>  ? acpi_process_madt+0xbe/0x105
>>  ? acpi_boot_init+0x3d/0x65
>>  ? setup_arch+0x67a/0x83f
>>  ? 0xc1000000
>>  ? start_kernel+0x46/0x361
>>  ? x86_early_init_platform_quirks+0x4d/0x90
>>  ? i386_start_kernel+0x22/0x88
>>  ? xen_start_kernel+0x453/0x639
>> (XEN) Hardware Dom0 crashed: 'noreboot' set - not rebooting.
>>
>> I admit I have a few custom patches in that tree, but I'm reasonably
>> certain that none of them comes even close to having such an effect.
>>
>> Jan
>>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Linux as 32-bit Dom0?

[Juergen Gross]

On 22/11/17 15:48, Jan Beulich wrote:
>>>> On 22.11.17 at 15:40, <jgross@suse.com> wrote:
>> On 22/11/17 15:05, Jan Beulich wrote:
>>> Jürgen, Boris,
>>>
>>> am I trying something that's not allowed, but selectable via Kconfig?
>>> On system with multiple IO-APICs (I assume that's what triggers the
>>> problem) I get
>>>
>>> Kernel panic - not syncing: Max apic_id exceeded!
>>
>> Generally I don't think 32 bit dom0 is forbidden, but rarely used. I
>> wouldn't be too sad in case we'd decide to drop that support. ;-)
>>
>> Can you please be a little bit more specific?
>>
>> How many IOAPICs? From the code I guess this is an INTEL system with not
>> too recent IOAPIC versions (<0x14)?
>>
>> Having a little bit more of the boot log might help, too.
> 
> Full log attached, which should answer all questions. This is
> a Haswell system, so not too old an IO-APIC flavor I would say.

From this data I can't explain why the system is crashing.

Right now I have 3 possible explanations, all could be proofed by
adding some printk statements in io_apic_get_unique_id(). Could you
please print the value returned by get_physical_broadcast() and the
complete apic_id_map right before the panic() call?

The possibilities IMHO are:
- the LAPIC version is limiting the number of available apicids
- apic_id_map is somehow filled up completely with all bits set
- a compiler bug leading to a false positive


Juergen



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Linux as 32-bit Dom0?

[Jan Beulich]

Jürgen, Boris,

am I trying something that's not allowed, but selectable via Kconfig?
On system with multiple IO-APICs (I assume that's what triggers the
problem) I get

Kernel panic - not syncing: Max apic_id exceeded!

CPU: 0 PID: 0 Comm: swapper Not tainted 4.14.1-2017-11-21-xen0 #6
Hardware name: ...
Call Trace:
 ? show_stack+0x20/0x50
 ? dump_stack+0x7e/0xc0
 ? panic+0x99/0x220
 ? io_apic_get_unique_id+0x207/0x210
 ? __raw_callee_save_xen_restore_fl+0x6/0x8
 ? xen_flush_tlb_single+0x6f/0x80
 ? set_pte_vaddr+0xef/0x110
 ? xen_io_apic_read+0x36/0x90
 ? mp_register_ioapic+0x2b7/0x410
 ? acpi_os_map_iomem+0x14d/0x210
 ? acpi_parse_ioapic+0x6b/0x6f
 ? acpi_parse_entries_array+0xa1/0x16d
 ? acpi_tb_get_table+0x95/0x9d
 ? acpi_ut_release_mutex+0x109/0x10e
 ? acpi_table_parse_entries_array+0x98/0xa8
 ? acpi_table_parse_entries+0x30/0x35
 ? acpi_boot_init+0x65/0x65
 ? acpi_table_parse_madt+0x1b/0x1f
 ? acpi_boot_init+0x65/0x65
 ? acpi_parse_madt_ioapic_entries+0x4a/0x119
 ? mutex_lock+0x8/0x30
 ? acpi_process_madt+0xbe/0x105
 ? acpi_boot_init+0x3d/0x65
 ? setup_arch+0x67a/0x83f
 ? 0xc1000000
 ? start_kernel+0x46/0x361
 ? x86_early_init_platform_quirks+0x4d/0x90
 ? i386_start_kernel+0x22/0x88
 ? xen_start_kernel+0x453/0x639
(XEN) Hardware Dom0 crashed: 'noreboot' set - not rebooting.

I admit I have a few custom patches in that tree, but I'm reasonably
certain that none of them comes even close to having such an effect.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel