Xen supported LAPIC features

Xen supported LAPIC features

[Hoyer, David]

I am working at getting VxWorks to boot as a guest OS on Xen/Debian as an SMP guest (HVM).   I have run into a couple problems that I was hoping could be addressed by this group.   None of the other groups seemed to discuss at a low enough level which is why I am starting here.

I noticed that the LAPIC timer does not seem to be supported by Xen.  Is this correct and if so, are there plans of supporting it?   VxWorks defaults to LAPIC timer for the system timer so it would be beneficial to have that support in XEN if we are to be able to run with an unmodified VxWorks kernel.

I also noticed that reading from address 0xfee00020 always returns 0 when it should be returning which core is running.   I found a way around that by using a CPUID command but this begs the question of what else is not supported of the LAPIC commands.


David Hoyer
  

Re: Xen supported LAPIC features

[Andrew Cooper]

On 08/10/13 20:03, Hoyer, David wrote:
> I am working at getting VxWorks to boot as a guest OS on Xen/Debian as an SMP guest (HVM).   I have run into a couple problems that I was hoping could be addressed by this group.   None of the other groups seemed to discuss at a low enough level which is why I am starting here.
>
> I noticed that the LAPIC timer does not seem to be supported by Xen.  Is this correct and if so, are there plans of supporting it?   VxWorks defaults to LAPIC timer for the system timer so it would be beneficial to have that support in XEN if we are to be able to run with an unmodified VxWorks kernel.
>
> I also noticed that reading from address 0xfee00020 always returns 0 when it should be returning which core is running.   I found a way around that by using a CPUID command but this begs the question of what else is not supported of the LAPIC commands.
>
>
> David Hoyer
>   

Which version of Xen are you running on?  That sounds like a broken
domain build; both of those should be working.

Do you have your domain configuration and boot logs?

~Andrew

Re: Xen supported LAPIC features

[Hoyer, David]

Output from xl dmesg:   Note that I have modified Xen to partition off some of my DRAM as RAID CACHE in my E820 tables due to a specific need that we have.   This is under early development by us.


(XEN) Xen version 4.2.2 (Debian 4.2.2-1) (waldi@debian.org) (gcc (Debian 4.7.2-5) 4.7.2) Tue Sep 17 10:04:12 CDT 2013
(XEN) Bootloader: SYSLINUX 4.05 20130605
(XEN) Command line: dom0_mem=512M,max:512M e820_raid_cache=2G iommu=1 console=hvc0
(XEN) Video information:
(XEN)  No VGA detected
(XEN) Disc information:
(XEN)  Found 1 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) Add E820_RAID_CACHE 2048MB to memory map
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000009ac00 (usable)
(XEN)  000000000009ac00 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 000000007e4dd000 (usable)
(XEN)  000000007e4dd000 - 000000007e556000 (ACPI data)
(XEN)  000000007e556000 - 000000007e665000 (ACPI NVS)
(XEN)  000000007e665000 - 000000007f265000 (reserved)
(XEN)  000000007f265000 - 000000007f2dd000 (usable)
(XEN)  000000007f2dd000 - 000000007f363000 (reserved)
(XEN)  000000007f363000 - 000000007f364000 (usable)
(XEN)  000000007f364000 - 000000007f365000 (ACPI NVS)
(XEN)  000000007f365000 - 000000007f36b000 (reserved)
(XEN)  000000007f36b000 - 000000007f373000 (ACPI NVS)
(XEN)  000000007f373000 - 000000007f39c000 (reserved)
(XEN)  000000007f39c000 - 000000007f41f000 (ACPI NVS)
(XEN)  000000007f41f000 - 000000007f800000 (usable)
(XEN)  00000000d0000000 - 00000000e0000000 (reserved)
(XEN)  00000000fed1c000 - 00000000fed20000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 0000000300000000 (usable)
(XEN)  0000000300000000 - 0000000380000000 (RAID CACHE)
(XEN) ACPI: RSDP 000F0490, 0024 (r2 ALASKA)
(XEN) ACPI: XSDT 7E4DD078, 0074 (r1 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: FACP 7E4E4B20, 00F4 (r4 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: DSDT 7E4DD180, 799F (r2 ALASKA    A M I      120 INTL 20051117)
(XEN) ACPI: FACS 7F370F80, 0040
(XEN) ACPI: APIC 7E4E4C18, 00C8 (r3 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: MCFG 7E4E4CE0, 003C (r1 ALASKA OEMMCFG.  1072009 MSFT       97)
(XEN) ACPI: HPET 7E4E4D20, 0038 (r1 ALASKA    A M I  1072009 AMI.        5)
(XEN) ACPI: SSDT 7E4E4D58, 70104 (r2  INTEL    CpuPm     4000 INTL 20051117)
(XEN) ACPI: DMAR 7E554E60, 00B4 (r1 A M I   OEMDMAR        1 INTL        1)
(XEN) ACPI: EINJ 7E554F18, 0130 (r1    AMI AMI EINJ        0             0)
(XEN) ACPI: ERST 7E555048, 0210 (r1  AMIER AMI ERST        0             0)
(XEN) ACPI: HEST 7E555258, 00A8 (r1    AMI AMI HEST        0             0)
(XEN) ACPI: BERT 7E555300, 0030 (r1    AMI AMI BERT        0             0)
(XEN) System RAM: 10216MB (10462020kB)
(XEN) Domain heap initialised
(XEN) ACPI: 32/64X FACS address mismatch in FADT - 7f370f80/0000000000000000, using 32
(XEN) Processor #0 6:13 APIC version 21
(XEN) Processor #2 6:13 APIC version 21
(XEN) Processor #4 6:13 APIC version 21
(XEN) Processor #6 6:13 APIC version 21
(XEN) Processor #1 6:13 APIC version 21
(XEN) Processor #3 6:13 APIC version 21
(XEN) Processor #5 6:13 APIC version 21
(XEN) Processor #7 6:13 APIC version 21
(XEN) IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
(XEN) IOAPIC[1]: apic_id 2, version 32, address 0xfec01000, GSI 24-47
(XEN) Enabling APIC mode:  Flat.  Using 2 I/O APICs
(XEN) Switched to APIC driver x2apic_cluster.
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 1995.266 MHz processor.
(XEN) Initing memory sharing.
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d Snoop Control enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Shared EPT tables enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 16 KiB.
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) Brought up 8 CPUs
(XEN) *** LOADING DOMAIN 0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1dfb000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   00000002ec000000->00000002f0000000 (96656 pages to be allocated)
(XEN)  Init. ramdisk: 00000002fb990000->00000002fffff600
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff81dfb000
(XEN)  Init. ramdisk: ffffffff81dfb000->ffffffff8646a600
(XEN)  Phys-Mach map: ffffffff8646b000->ffffffff8656b000
(XEN)  Start info:    ffffffff8656b000->ffffffff8656b4b4
(XEN)  Page tables:   ffffffff8656c000->ffffffff865a3000
(XEN)  Boot stack:    ffffffff865a3000->ffffffff865a4000
(XEN)  TOTAL:         ffffffff80000000->ffffffff86800000
(XEN)  ENTRY ADDRESS: ffffffff816e21e0
(XEN) Dom0 has maximum 8 VCPUs
(XEN) Scrubbing Free RAM: ................................................................................................done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 264kB init memory.
(XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
(XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
(XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
(XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
(XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
(XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
(XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
(XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
(XEN) domctl.c:1581: E820 RAID CACHE from Domain-1
(XEN) domctl.c:1597: E820 RAID CACHE Addr=0x0000000300000000 Size=0x0000000080000000
(XEN) physdev.c:178: dom1: 17:-1 already mapped to 17
(XEN) physdev.c:178: dom1: 31:-1 already mapped to 21
(XEN) physdev.c:178: dom1: 39:-1 already mapped to 22
(XEN) physdev.c:178: dom1: 31:-1 already mapped to 21
(XEN) physdev.c:178: dom1: 39:-1 already mapped to 22
(XEN) physdev.c:178: dom1: 31:-1 already mapped to 21
(XEN) physdev.c:178: dom1: 39:-1 already mapped to 22
(XEN) vioapic.c:197:d1 vioapic_write_indirect error register 3


-----Original Message-----
From: Andrew Cooper [mailto:andrew.cooper3@citrix.com] 
Sent: Tuesday, October 08, 2013 2:13 PM
To: Hoyer, David
Cc: xen-devel@lists.xen.org
Subject: Re: [Xen-devel] Xen supported LAPIC features

On 08/10/13 20:03, Hoyer, David wrote:
> I am working at getting VxWorks to boot as a guest OS on Xen/Debian as an SMP guest (HVM).   I have run into a couple problems that I was hoping could be addressed by this group.   None of the other groups seemed to discuss at a low enough level which is why I am starting here.
>
> I noticed that the LAPIC timer does not seem to be supported by Xen.  Is this correct and if so, are there plans of supporting it?   VxWorks defaults to LAPIC timer for the system timer so it would be beneficial to have that support in XEN if we are to be able to run with an unmodified VxWorks kernel.
>
> I also noticed that reading from address 0xfee00020 always returns 0 when it should be returning which core is running.   I found a way around that by using a CPUID command but this begs the question of what else is not supported of the LAPIC commands.
>
>
> David Hoyer
>   

The domain config file is as follows:

firmware_override = "/usr/lib/xen-4.2/boot/hvmloader"
builder = "hvm"
device_model_version = "qemu-xen"
device_model_override = "/usr/bin/qemu-system-i386"
disk = ["phy:/dev/vg0/vxworks,hda,w"]
name = "IOVM"
boot = "c"
videoram=32
keymap = "en-us"
serial = "pty"
pae=1
pci = ['0000:0c:0b.0@5', '0000:03:00.0@A', '0000:05:00.0@B', '0000:06:00.0@7', '0000:07:00.0@8', '0000:08:00.0@9', '0000:00:04.0@E', '0000:00:04.1@F', '0000:00:04.2@10', '0000:00:04.3@11', '0000:00:04.4@12', '0000:00:04.5@13', '0000:00:04.6@14', '0000:00:04.7@15']
vif = [ "type=ioemu, model=e1000, mac=00:80:e5:11:11:11, bridge=xenbr0" ]
maxmem = 1024
memory = 1024
vcpus = 1
cpu="2"
acpi = 1
apic = 1
e820_raid_cache = 1
on_poweroff = 'destroy'
on_reboot = 'destroy'
on_crash = 'destroy'


Which version of Xen are you running on?  That sounds like a broken domain build; both of those should be working.

Do you have your domain configuration and boot logs?

~Andrew

Re: Xen supported LAPIC features

[Andrew Cooper]

On 08/10/13 20:37, Hoyer, David wrote:
> Output from xl dmesg:   Note that I have modified Xen to partition off some of my DRAM as RAID CACHE in my E820 tables due to a specific need that we have.   This is under early development by us.
>
>
> (XEN) Freed 264kB init memory.
> (XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
> (XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
> (XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
> (XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
> (XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
> (XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
> (XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
> (XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
> (XEN) domctl.c:1581: E820 RAID CACHE from Domain-1
> (XEN) domctl.c:1597: E820 RAID CACHE Addr=0x0000000300000000 Size=0x0000000080000000
> (XEN) physdev.c:178: dom1: 17:-1 already mapped to 17
> (XEN) physdev.c:178: dom1: 31:-1 already mapped to 21
> (XEN) physdev.c:178: dom1: 39:-1 already mapped to 22
> (XEN) physdev.c:178: dom1: 31:-1 already mapped to 21
> (XEN) physdev.c:178: dom1: 39:-1 already mapped to 22
> (XEN) physdev.c:178: dom1: 31:-1 already mapped to 21
> (XEN) physdev.c:178: dom1: 39:-1 already mapped to 22
> (XEN) vioapic.c:197:d1 vioapic_write_indirect error register 3

Without seeing the changes you have made for the raid cache, I cant tell
whether you have broken something in Xen.

However, this final error message from vioapic.c proves that you have a
buggy IOAPIC driver dom1 (which I presume is your domU); It is
attempting to read IOAPIC register 0x3 which doesn't exist in real
hardware (or certainly not in any spec I have encountered).

The errors from physdev.c are from physdev_map_pirq().  Given the number
of PCI devices you are trying to pass through, you might have run out of
pirqs for domU, although I would expect a rather more direct error
message.  Try booting xen with "extra_guest_irqs=64" or so.  The default
is for a limit of 32.

~Andrew

Re: Xen supported LAPIC features

[Hoyer, David]

That message below only shows up when I kick off my domU (the vxWorks kernel).   I bumped the counter as you mentioned below but that did not help.   It will take awhile to see where the vxWorks kernel might be attempting the register 3 operation on ioapic.  But since this works just fine when I run the vxWorks kernel directly on HW, I am not inclined to believe there is a big problem with the ioapic driver.

-----Original Message-----
From: Andrew Cooper [mailto:andrew.cooper3@citrix.com] 
Sent: Tuesday, October 08, 2013 3:33 PM
To: Hoyer, David
Cc: xen-devel@lists.xen.org
Subject: Re: [Xen-devel] Xen supported LAPIC features

On 08/10/13 20:37, Hoyer, David wrote:
> Output from xl dmesg:   Note that I have modified Xen to partition off some of my DRAM as RAID CACHE in my E820 tables due to a specific need that we have.   This is under early development by us.
>
>
> (XEN) Freed 264kB init memory.
> (XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
> (XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
> (XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
> (XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
> (XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
> (XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
> (XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
> (XEN) traps.c:2600:d0 Domain attempted WRMSR 00000000000001fc from 0x000000002504005b to 0x0000000025040059.
> (XEN) domctl.c:1581: E820 RAID CACHE from Domain-1
> (XEN) domctl.c:1597: E820 RAID CACHE Addr=0x0000000300000000 
> Size=0x0000000080000000
> (XEN) physdev.c:178: dom1: 17:-1 already mapped to 17
> (XEN) physdev.c:178: dom1: 31:-1 already mapped to 21
> (XEN) physdev.c:178: dom1: 39:-1 already mapped to 22
> (XEN) physdev.c:178: dom1: 31:-1 already mapped to 21
> (XEN) physdev.c:178: dom1: 39:-1 already mapped to 22
> (XEN) physdev.c:178: dom1: 31:-1 already mapped to 21
> (XEN) physdev.c:178: dom1: 39:-1 already mapped to 22
> (XEN) vioapic.c:197:d1 vioapic_write_indirect error register 3

Without seeing the changes you have made for the raid cache, I cant tell whether you have broken something in Xen.

However, this final error message from vioapic.c proves that you have a buggy IOAPIC driver dom1 (which I presume is your domU); It is attempting to read IOAPIC register 0x3 which doesn't exist in real hardware (or certainly not in any spec I have encountered).

The errors from physdev.c are from physdev_map_pirq().  Given the number of PCI devices you are trying to pass through, you might have run out of pirqs for domU, although I would expect a rather more direct error message.  Try booting xen with "extra_guest_irqs=64" or so.  The default is for a limit of 32.

~Andrew

Re: Xen supported LAPIC features

[Andrew Cooper]

On 09/10/13 15:04, Hoyer, David wrote:
> That message below only shows up when I kick off my domU (the vxWorks kernel).   I bumped the counter as you mentioned below but that did not help.   It will take awhile to see where the vxWorks kernel might be attempting the register 3 operation on ioapic.  But since this works just fine when I run the vxWorks kernel directly on HW, I am not inclined to believe there is a big problem with the ioapic driver.

On Xen, as well as real hardware, you will read 0xffffffff from that
attempted access.  Xen is merely also making a note that you have a
buggy ioapic driver.  My point was that, given you definitely have a
buggy ioapic driver, are you certain you have no bugs in your lapic driver?

~Andrew

Re: Xen supported LAPIC features

[Hoyer, David]

There is the distinct possibility of that... but since this is WindRiver code, it would require a bit of investigation on my part.

The fact that you believe that the loapic timer should be able to run on a guest OS leads me to believe that I need to look into this further on the vxWorks side.

-----Original Message-----
From: Andrew Cooper [mailto:andrew.cooper3@citrix.com] 
Sent: Wednesday, October 09, 2013 9:11 AM
To: Hoyer, David
Cc: xen-devel@lists.xen.org
Subject: Re: [Xen-devel] Xen supported LAPIC features

On 09/10/13 15:04, Hoyer, David wrote:
> That message below only shows up when I kick off my domU (the vxWorks kernel).   I bumped the counter as you mentioned below but that did not help.   It will take awhile to see where the vxWorks kernel might be attempting the register 3 operation on ioapic.  But since this works just fine when I run the vxWorks kernel directly on HW, I am not inclined to believe there is a big problem with the ioapic driver.

On Xen, as well as real hardware, you will read 0xffffffff from that attempted access.  Xen is merely also making a note that you have a buggy ioapic driver.  My point was that, given you definitely have a buggy ioapic driver, are you certain you have no bugs in your lapic driver?

~Andrew

Re: Xen supported LAPIC features

[Hoyer, David]

I wanted to follow up on this discussion just so that the community knows what I have discovered.

Previously I was running with Xen 4.2.2/Debian(sid) and vxWorks 6.9.3 and was having the problems previously described.   Based on some feedback, I decided to upgrade to vxWorks 6.9.3.2 (their latest) and ran into some new yet different problems.   I debugged it down to an issue with how they were terminating early in the MP table scan if the MP tables did not have certain information in it.   When I removed the early termination and allowed it to resume in spite of this particular data being missing, it suddenly was able to boot up successfully using the loapic timer!  I am also no longer getting the error message that Andrew pointed out earlier.

Thanks for the help, Andrew.

-----Original Message-----
From: Andrew Cooper [mailto:andrew.cooper3@citrix.com] 
Sent: Wednesday, October 09, 2013 9:11 AM
To: Hoyer, David
Cc: xen-devel@lists.xen.org
Subject: Re: [Xen-devel] Xen supported LAPIC features

On 09/10/13 15:04, Hoyer, David wrote:
> That message below only shows up when I kick off my domU (the vxWorks kernel).   I bumped the counter as you mentioned below but that did not help.   It will take awhile to see where the vxWorks kernel might be attempting the register 3 operation on ioapic.  But since this works just fine when I run the vxWorks kernel directly on HW, I am not inclined to believe there is a big problem with the ioapic driver.

On Xen, as well as real hardware, you will read 0xffffffff from that attempted access.  Xen is merely also making a note that you have a buggy ioapic driver.  My point was that, given you definitely have a buggy ioapic driver, are you certain you have no bugs in your lapic driver?

~Andrew

Re: Xen supported LAPIC features

[Andrew Cooper]

On 25/10/2013 20:45, Hoyer, David wrote:
> I wanted to follow up on this discussion just so that the community knows what I have discovered.
>
> Previously I was running with Xen 4.2.2/Debian(sid) and vxWorks 6.9.3 and was having the problems previously described.   Based on some feedback, I decided to upgrade to vxWorks 6.9.3.2 (their latest) and ran into some new yet different problems.   I debugged it down to an issue with how they were terminating early in the MP table scan if the MP tables did not have certain information in it.   When I removed the early termination and allowed it to resume in spite of this particular data being missing, it suddenly was able to boot up successfully using the loapic timer!  I am also no longer getting the error message that Andrew pointed out earlier.
>
> Thanks for the help, Andrew.

Thats good news that you have managed to make it work.

Which information is vxWorks expecting from the MP tables which is not
present?  If it is useful information then we might as well be
presenting it.

~Andrew

Re: Xen supported LAPIC features

[Hoyer, David]

In this case, they were looking for the number of "Local interrupts" and if it was zero, they were skipping the rest of the initialization.   They also will skip the remainder of the initialization if the IO interrupts is zero but I didn't trip over that case.

-----Original Message-----
From: Andrew Cooper [mailto:andrew.cooper3@citrix.com] 
Sent: Friday, October 25, 2013 7:19 PM
To: Hoyer, David
Cc: xen-devel@lists.xen.org
Subject: Re: [Xen-devel] Xen supported LAPIC features

On 25/10/2013 20:45, Hoyer, David wrote:
> I wanted to follow up on this discussion just so that the community knows what I have discovered.
>
> Previously I was running with Xen 4.2.2/Debian(sid) and vxWorks 6.9.3 and was having the problems previously described.   Based on some feedback, I decided to upgrade to vxWorks 6.9.3.2 (their latest) and ran into some new yet different problems.   I debugged it down to an issue with how they were terminating early in the MP table scan if the MP tables did not have certain information in it.   When I removed the early termination and allowed it to resume in spite of this particular data being missing, it suddenly was able to boot up successfully using the loapic timer!  I am also no longer getting the error message that Andrew pointed out earlier.
>
> Thanks for the help, Andrew.

Thats good news that you have managed to make it work.

Which information is vxWorks expecting from the MP tables which is not present?  If it is useful information then we might as well be presenting it.

~Andrew