* Xen, MCFG acpi table and E820 address map @ 2013-09-04 1:13 Santosh Jodh 2013-09-04 9:16 ` Jan Beulich 0 siblings, 1 reply; 11+ messages in thread From: Santosh Jodh @ 2013-09-04 1:13 UTC (permalink / raw) To: xen-devel@lists.xen.org Xen will use information from MCFG acpi table to access PCIe extended configuration space. However, Xen validates MCFG table by making sure that the addresses specified in the MCFG table is correctly marked as reserved in the E820 address map. If it is not, the MCFG table is ignored - thereby preventing Xen from accessing PCIe extended configuration space. I recently came across a workstation class system that supports VT-d. This system BIOS has a valid MCFG table. The BIOS does NOT report the MCFG addresses as reserved in the E820 address map. However, the addresses ARE claimed as reserved via the ACPI motherboard resource devnode (PNP0C01) mechanism. On this system, Xen ignores the MCFG table as it fails the E820 check. dom0 during early boot does the same thing. However, once ACPI driver is online, dom0 validates the MCFG table against the motherboard resource devnode and then accepts the MCFG table. You now end up with a situation where dom0 can correctly enumerate and use PCIe devices but Xen cannot access extended configuration space. If an attempt is now made to pass-through a PCIe SRIOV VF with MSI interrupt for e.g., you see following warning (and IIRC, the passed through device did not work): (XEN) Xen WARN at msi.c:688 (XEN) ----[ Xen-4.1.1 x86_64 debug=n Not tainted ]---- (XEN) CPU: 7 (XEN) RIP: e008:[<ffff82c48015a9c5>] pci_enable_msi+0x645/0x8f0 (XEN) RFLAGS: 0000000000010206 CONTEXT: hypervisor (XEN) rax: 0000000000000000 rbx: 00000000fa160000 rcx: 0000000000000000 (XEN) rdx: 0000000000000cfe rsi: 0000000000000246 rdi: ffff82c48023db80 (XEN) rbp: ffff830413e97ea8 rsp: ffff830413e97d28 r8: 0000000000000100 (XEN) r9: ffff830413e97c7c r10: 0000000000000000 r11: 0000000000000070 (XEN) r12: 0000000000000481 r13: ffff8304079b6250 r14: ffff830413e97e28 (XEN) r15: 0000000000000003 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 0000000413e69000 cr2: 00000000d84341b8 (XEN) ds: 007b es: 007b fs: 00d8 gs: 0033 ss: 0000 cs: e008 (XEN) Xen stack trace from rsp=ffff830413e97d28: (XEN) ffff830407bcdf50 ffff8300de982000 ffff8304079b62e8 0000000000000040 (XEN) ffff82c400000003 0000000000000064 ffff83041f480fd8 00000070c0027631 (XEN) ffff8303b1471e80 0000001000000004 0000007200000001 00000000000fa160 (XEN) 00000000000fa160 0000000000000000 00000000fa160000 0000000000000033 (XEN) 0000000100000000 0000000300000000 ffff8304108441d8 ffff830413e97ea8 (XEN) 0000000000000033 ffff830407a84000 000000000000002a 00000000ffffffed (XEN) ffff8304079b6250 ffff82c48015c63c 0000000000000000 00000000000000cc (XEN) 000000000000002a 00000000000000a8 ffff83041f402a80 0000000000000000 (XEN) 00000000ffffffff ffff8304123840b8 00000000088c6004 0000000000000033 (XEN) 000000000000002a ffff830413e97e78 ffff830413e97ea8 ffff82c4801fb561 (XEN) 0000000400000000 00000000c02f6361 0000000000000001 ffffffffffffffff (XEN) 0000008100000004 fa16000000000000 ffff830400000000 0000000000000000 (XEN) 0000008100000004 000000000000002a 00000000fa160000 ffff830407a84000 (XEN) 00000000000002ed 00c0bb0013e97ee0 0000000000000000 ffff8300deff2000 (XEN) 00000000d6bbfed4 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000000 ffff82c4801fd324 0000000000000000 0000000000000000 (XEN) 0000000000000000 0000000000000000 00000000d6bbfed4 000000000000000d (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000 (XEN) 0000000000000021 00000000088c6004 00000000bf9006f8 00000000b773f658 (XEN) 00000000b7730c28 0000010000000000 00000000c0101427 0000000000000061 (XEN) Xen call trace: (XEN) [<ffff82c48015a9c5>] pci_enable_msi+0x645/0x8f0 (XEN) [<ffff82c48015c63c>] map_domain_pirq+0x25c/0x350 (XEN) [<ffff82c4801fb561>] compat_physdev_op+0xe31/0x12f0 (XEN) [<ffff82c4801fd324>] compat_hypercall+0x74/0x80 Is this machine an exception? As we move forward, are we likely to see more such systems (relying more on ACPI and less on E820 for reserving the MCFG range)? Is it worth adding a mmcfg=force option to xen to ignore E820 validation result? Regards, Santosh ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Xen, MCFG acpi table and E820 address map 2013-09-04 1:13 Xen, MCFG acpi table and E820 address map Santosh Jodh @ 2013-09-04 9:16 ` Jan Beulich 2013-09-04 15:01 ` Konrad Rzeszutek Wilk 0 siblings, 1 reply; 11+ messages in thread From: Jan Beulich @ 2013-09-04 9:16 UTC (permalink / raw) To: Santosh Jodh; +Cc: xen-devel, Boris Ostrovsky, David Vrabel [-- Attachment #1: Type: text/plain, Size: 1936 bytes --] >>> On 04.09.13 at 03:13, Santosh Jodh <Santosh.Jodh@citrix.com> wrote: > Xen will use information from MCFG acpi table to access PCIe extended > configuration space. However, Xen validates MCFG table by making sure that > the addresses specified in the MCFG table is correctly marked as reserved in > the E820 address map. If it is not, the MCFG table is ignored - thereby > preventing Xen from accessing PCIe extended configuration space. > > I recently came across a workstation class system that supports VT-d. This > system BIOS has a valid MCFG table. The BIOS does NOT report the MCFG > addresses as reserved in the E820 address map. However, the addresses ARE > claimed as reserved via the ACPI motherboard resource devnode (PNP0C01) > mechanism. > > On this system, Xen ignores the MCFG table as it fails the E820 check. dom0 > during early boot does the same thing. However, once ACPI driver is online, > dom0 validates the MCFG table against the motherboard resource devnode and > then accepts the MCFG table. You now end up with a situation where dom0 can > correctly enumerate and use PCIe devices but Xen cannot access extended > configuration space. > [...] > Is this machine an exception? As we move forward, are we likely to see more > such systems (relying more on ACPI and less on E820 for reserving the MCFG > range)? Is it worth adding a mmcfg=force option to xen to ignore E820 > validation result? No, this is quite normal, and Xen has code to deal with that (PHYSDEVOP_pci_mmcfg_reserved). While our (forward ported) kernels have been making use of this since 2.6.27, I suppose upstream still doesn't (perhaps not the least because integration of this might once again raise concerns about Xen needing code changes in too many different places). For reference I'm attaching the diff of the respective source file, in case someone finds this useful. Jan [-- Attachment #2: mmcfg-reserved.patch --] [-- Type: text/plain, Size: 1862 bytes --] --- linux-3.11/arch/x86/pci/mmconfig-shared.c +++ head/arch/x86/pci/mmconfig-shared.c @@ -23,6 +23,10 @@ #include <asm/pci_x86.h> #include <asm/acpi.h> +#ifdef CONFIG_XEN +#include <xen/interface/physdev.h> +#endif + #define PREFIX "PCI: " /* Indicate if the mmcfg resources have been placed into the resource table. */ @@ -437,6 +441,36 @@ static int is_acpi_reserved(u64 start, u return mcfg_res.flags; } +static int xen_report_mmconf_reserved(const struct pci_mmcfg_region *cfg, + int valid, int with_e820) +{ +#ifdef CONFIG_XEN + if (!with_e820) { + struct physdev_pci_mmcfg_reserved r = { + .address = cfg->address, + .segment = cfg->segment, + .start_bus = cfg->start_bus, + .end_bus = cfg->end_bus, + .flags = valid ? XEN_PCI_MMCFG_RESERVED : 0 + }; + int rc; + + rc = HYPERVISOR_physdev_op(PHYSDEVOP_pci_mmcfg_reserved, &r); + switch (rc) { + case 0: case -ENOSYS: + break; + default: + pr_warn(PREFIX "Failed to report MMCONFIG reservation" + " state for %04x [bus%02x-%02x] to hypervisor" + " (%d)\n", + cfg->segment, cfg->start_bus, cfg->end_bus, + rc); + } + } +#endif + return valid; +} + typedef int (*check_reserved_t)(u64 start, u64 end, unsigned type); static int __ref is_mmconf_reserved(check_reserved_t is_reserved, @@ -456,7 +490,7 @@ static int __ref is_mmconf_reserved(chec } if (size < (16UL<<20) && size != old_size) - return 0; + return xen_report_mmconf_reserved(cfg, 0, with_e820); if (dev) dev_info(dev, "MMCONFIG at %pR reserved in %s\n", @@ -488,7 +522,7 @@ static int __ref is_mmconf_reserved(chec &cfg->res, (unsigned long) cfg->address); } - return 1; + return xen_report_mmconf_reserved(cfg, 1, with_e820); } static int __ref pci_mmcfg_check_reserved(struct device *dev, [-- Attachment #3: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Xen, MCFG acpi table and E820 address map 2013-09-04 9:16 ` Jan Beulich @ 2013-09-04 15:01 ` Konrad Rzeszutek Wilk 2013-09-04 15:12 ` Jan Beulich 2013-09-04 16:14 ` Santosh Jodh 0 siblings, 2 replies; 11+ messages in thread From: Konrad Rzeszutek Wilk @ 2013-09-04 15:01 UTC (permalink / raw) To: Jan Beulich; +Cc: xen-devel, Boris Ostrovsky, David Vrabel, Santosh Jodh On Wed, Sep 04, 2013 at 10:16:26AM +0100, Jan Beulich wrote: > >>> On 04.09.13 at 03:13, Santosh Jodh <Santosh.Jodh@citrix.com> wrote: > > Xen will use information from MCFG acpi table to access PCIe extended > > configuration space. However, Xen validates MCFG table by making sure that > > the addresses specified in the MCFG table is correctly marked as reserved in > > the E820 address map. If it is not, the MCFG table is ignored - thereby > > preventing Xen from accessing PCIe extended configuration space. > > > > I recently came across a workstation class system that supports VT-d. This > > system BIOS has a valid MCFG table. The BIOS does NOT report the MCFG > > addresses as reserved in the E820 address map. However, the addresses ARE > > claimed as reserved via the ACPI motherboard resource devnode (PNP0C01) > > mechanism. Could you tell me what machine this is? It would be good to know to develop a patch against it. > > > > On this system, Xen ignores the MCFG table as it fails the E820 check. dom0 > > during early boot does the same thing. However, once ACPI driver is online, > > dom0 validates the MCFG table against the motherboard resource devnode and > > then accepts the MCFG table. You now end up with a situation where dom0 can > > correctly enumerate and use PCIe devices but Xen cannot access extended > > configuration space. > > [...] > > Is this machine an exception? As we move forward, are we likely to see more > > such systems (relying more on ACPI and less on E820 for reserving the MCFG > > range)? Is it worth adding a mmcfg=force option to xen to ignore E820 > > validation result? > > No, this is quite normal, and Xen has code to deal with that > (PHYSDEVOP_pci_mmcfg_reserved). While our (forward ported) > kernels have been making use of this since 2.6.27, I suppose > upstream still doesn't (perhaps not the least because integration > of this might once again raise concerns about Xen needing code > changes in too many different places). > > For reference I'm attaching the diff of the respective source file, > in case someone finds this useful. Thank you for providing the patch. Is there a nicer way of doing this? Inserting Xen codepaths right there is on the high list of no-no. > > Jan > > --- linux-3.11/arch/x86/pci/mmconfig-shared.c > +++ head/arch/x86/pci/mmconfig-shared.c > @@ -23,6 +23,10 @@ > #include <asm/pci_x86.h> > #include <asm/acpi.h> > > +#ifdef CONFIG_XEN > +#include <xen/interface/physdev.h> > +#endif > + > #define PREFIX "PCI: " > > /* Indicate if the mmcfg resources have been placed into the resource table. */ > @@ -437,6 +441,36 @@ static int is_acpi_reserved(u64 start, u > return mcfg_res.flags; > } > > +static int xen_report_mmconf_reserved(const struct pci_mmcfg_region *cfg, > + int valid, int with_e820) > +{ > +#ifdef CONFIG_XEN > + if (!with_e820) { > + struct physdev_pci_mmcfg_reserved r = { > + .address = cfg->address, > + .segment = cfg->segment, > + .start_bus = cfg->start_bus, > + .end_bus = cfg->end_bus, > + .flags = valid ? XEN_PCI_MMCFG_RESERVED : 0 > + }; > + int rc; > + > + rc = HYPERVISOR_physdev_op(PHYSDEVOP_pci_mmcfg_reserved, &r); > + switch (rc) { > + case 0: case -ENOSYS: > + break; > + default: > + pr_warn(PREFIX "Failed to report MMCONFIG reservation" > + " state for %04x [bus%02x-%02x] to hypervisor" > + " (%d)\n", > + cfg->segment, cfg->start_bus, cfg->end_bus, > + rc); > + } > + } > +#endif > + return valid; > +} > + > typedef int (*check_reserved_t)(u64 start, u64 end, unsigned type); > > static int __ref is_mmconf_reserved(check_reserved_t is_reserved, > @@ -456,7 +490,7 @@ static int __ref is_mmconf_reserved(chec > } > > if (size < (16UL<<20) && size != old_size) > - return 0; > + return xen_report_mmconf_reserved(cfg, 0, with_e820); > > if (dev) > dev_info(dev, "MMCONFIG at %pR reserved in %s\n", > @@ -488,7 +522,7 @@ static int __ref is_mmconf_reserved(chec > &cfg->res, (unsigned long) cfg->address); > } > > - return 1; > + return xen_report_mmconf_reserved(cfg, 1, with_e820); > } > > static int __ref pci_mmcfg_check_reserved(struct device *dev, ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Xen, MCFG acpi table and E820 address map 2013-09-04 15:01 ` Konrad Rzeszutek Wilk @ 2013-09-04 15:12 ` Jan Beulich 2013-09-04 16:14 ` Santosh Jodh 1 sibling, 0 replies; 11+ messages in thread From: Jan Beulich @ 2013-09-04 15:12 UTC (permalink / raw) To: Konrad Rzeszutek Wilk Cc: xen-devel, Boris Ostrovsky, David Vrabel, Santosh Jodh >>> On 04.09.13 at 17:01, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote: > On Wed, Sep 04, 2013 at 10:16:26AM +0100, Jan Beulich wrote: >> For reference I'm attaching the diff of the respective source file, >> in case someone finds this useful. > > Thank you for providing the patch. > > Is there a nicer way of doing this? Inserting Xen codepaths right > there is on the high list of no-no. I guess xen_report_mmconf_reserved() could become a pv op, pointing to a no-op on native, returning back just the "valid" flag it gets passed. Jan >> --- linux-3.11/arch/x86/pci/mmconfig-shared.c >> +++ head/arch/x86/pci/mmconfig-shared.c >> @@ -23,6 +23,10 @@ >> #include <asm/pci_x86.h> >> #include <asm/acpi.h> >> >> +#ifdef CONFIG_XEN >> +#include <xen/interface/physdev.h> >> +#endif >> + >> #define PREFIX "PCI: " >> >> /* Indicate if the mmcfg resources have been placed into the resource table. */ >> @@ -437,6 +441,36 @@ static int is_acpi_reserved(u64 start, u >> return mcfg_res.flags; >> } >> >> +static int xen_report_mmconf_reserved(const struct pci_mmcfg_region *cfg, >> + int valid, int with_e820) >> +{ >> +#ifdef CONFIG_XEN >> + if (!with_e820) { >> + struct physdev_pci_mmcfg_reserved r = { >> + .address = cfg->address, >> + .segment = cfg->segment, >> + .start_bus = cfg->start_bus, >> + .end_bus = cfg->end_bus, >> + .flags = valid ? XEN_PCI_MMCFG_RESERVED : 0 >> + }; >> + int rc; >> + >> + rc = HYPERVISOR_physdev_op(PHYSDEVOP_pci_mmcfg_reserved, &r); >> + switch (rc) { >> + case 0: case -ENOSYS: >> + break; >> + default: >> + pr_warn(PREFIX "Failed to report MMCONFIG reservation" >> + " state for %04x [bus%02x-%02x] to hypervisor" >> + " (%d)\n", >> + cfg->segment, cfg->start_bus, cfg->end_bus, >> + rc); >> + } >> + } >> +#endif >> + return valid; >> +} >> + >> typedef int (*check_reserved_t)(u64 start, u64 end, unsigned type); >> >> static int __ref is_mmconf_reserved(check_reserved_t is_reserved, >> @@ -456,7 +490,7 @@ static int __ref is_mmconf_reserved(chec >> } >> >> if (size < (16UL<<20) && size != old_size) >> - return 0; >> + return xen_report_mmconf_reserved(cfg, 0, with_e820); >> >> if (dev) >> dev_info(dev, "MMCONFIG at %pR reserved in %s\n", >> @@ -488,7 +522,7 @@ static int __ref is_mmconf_reserved(chec >> &cfg->res, (unsigned long) cfg->address); >> } >> >> - return 1; >> + return xen_report_mmconf_reserved(cfg, 1, with_e820); >> } >> >> static int __ref pci_mmcfg_check_reserved(struct device *dev, ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Xen, MCFG acpi table and E820 address map 2013-09-04 15:01 ` Konrad Rzeszutek Wilk 2013-09-04 15:12 ` Jan Beulich @ 2013-09-04 16:14 ` Santosh Jodh 2013-09-04 16:24 ` Gordan Bobic 1 sibling, 1 reply; 11+ messages in thread From: Santosh Jodh @ 2013-09-04 16:14 UTC (permalink / raw) To: Konrad Rzeszutek Wilk, Jan Beulich Cc: xen-devel, Boris Ostrovsky, David Vrabel -----Original Message----- From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com] Sent: Wednesday, September 04, 2013 8:01 AM To: Jan Beulich Cc: Santosh Jodh; David Vrabel; xen-devel; Boris Ostrovsky Subject: Re: [Xen-devel] Xen, MCFG acpi table and E820 address map On Wed, Sep 04, 2013 at 10:16:26AM +0100, Jan Beulich wrote: > >>> On 04.09.13 at 03:13, Santosh Jodh <Santosh.Jodh@citrix.com> wrote: > > Xen will use information from MCFG acpi table to access PCIe > > extended configuration space. However, Xen validates MCFG table by > > making sure that the addresses specified in the MCFG table is > > correctly marked as reserved in the E820 address map. If it is not, > > the MCFG table is ignored - thereby preventing Xen from accessing PCIe extended configuration space. > > > > I recently came across a workstation class system that supports > > VT-d. This system BIOS has a valid MCFG table. The BIOS does NOT > > report the MCFG addresses as reserved in the E820 address map. > > However, the addresses ARE claimed as reserved via the ACPI > > motherboard resource devnode (PNP0C01) mechanism. Could you tell me what machine this is? It would be good to know to develop a patch against it. [Santosh Jodh] Unfortunately, this is a new platform I cannot disclose much about. However, it seems like this issue has been seen on the following: Asus S5A laptop, Asus P5RD1-VM. There are few IBM systems as well - http://www-947.ibm.com/support/entry/portal/docdisplay?lndocid=MIGR-5080398 I should be able to test the patch and provide feedback on this particular system. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Xen, MCFG acpi table and E820 address map 2013-09-04 16:14 ` Santosh Jodh @ 2013-09-04 16:24 ` Gordan Bobic 2013-09-04 16:26 ` Santosh Jodh 2013-09-05 7:44 ` Jan Beulich 0 siblings, 2 replies; 11+ messages in thread From: Gordan Bobic @ 2013-09-04 16:24 UTC (permalink / raw) To: Santosh Jodh; +Cc: xen-devel, Boris Ostrovsky, David Vrabel, Jan Beulich On Wed, 4 Sep 2013 16:14:55 +0000, Santosh Jodh <Santosh.Jodh@citrix.com> wrote: > -----Original Message----- > From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com] > Sent: Wednesday, September 04, 2013 8:01 AM > To: Jan Beulich > Cc: Santosh Jodh; David Vrabel; xen-devel; Boris Ostrovsky > Subject: Re: [Xen-devel] Xen, MCFG acpi table and E820 address map > > On Wed, Sep 04, 2013 at 10:16:26AM +0100, Jan Beulich wrote: >> >>> On 04.09.13 at 03:13, Santosh Jodh <Santosh.Jodh@citrix.com> >> wrote: >> > Xen will use information from MCFG acpi table to access PCIe >> > extended configuration space. However, Xen validates MCFG table by >> > making sure that the addresses specified in the MCFG table is >> > correctly marked as reserved in the E820 address map. If it is >> not, >> > the MCFG table is ignored - thereby preventing Xen from accessing >> PCIe extended configuration space. >> > >> > I recently came across a workstation class system that supports >> > VT-d. This system BIOS has a valid MCFG table. The BIOS does NOT >> > report the MCFG addresses as reserved in the E820 address map. >> > However, the addresses ARE claimed as reserved via the ACPI >> > motherboard resource devnode (PNP0C01) mechanism. > > Could you tell me what machine this is? It would be good to know to > develop a patch against it. > [Santosh Jodh] Unfortunately, this is a new platform I cannot > disclose much about. If this is pre-release/experimental hardware, isn't the right thing to do to talk to the manufacturer and get them to fix the BIOS so it exposes the correct e820 map in the first place? Gordan ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Xen, MCFG acpi table and E820 address map 2013-09-04 16:24 ` Gordan Bobic @ 2013-09-04 16:26 ` Santosh Jodh 2013-09-05 7:47 ` Jan Beulich 2013-09-05 7:44 ` Jan Beulich 1 sibling, 1 reply; 11+ messages in thread From: Santosh Jodh @ 2013-09-04 16:26 UTC (permalink / raw) To: Gordan Bobic; +Cc: xen-devel, Boris Ostrovsky, David Vrabel, Jan Beulich -----Original Message----- From: Gordan Bobic [mailto:gordan@bobich.net] Sent: Wednesday, September 04, 2013 9:24 AM To: Santosh Jodh Cc: Konrad Rzeszutek Wilk; Jan Beulich; xen-devel; Boris Ostrovsky; David Vrabel Subject: Re: [Xen-devel] Xen, MCFG acpi table and E820 address map On Wed, 4 Sep 2013 16:14:55 +0000, Santosh Jodh <Santosh.Jodh@citrix.com> wrote: > -----Original Message----- > From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com] > Sent: Wednesday, September 04, 2013 8:01 AM > To: Jan Beulich > Cc: Santosh Jodh; David Vrabel; xen-devel; Boris Ostrovsky > Subject: Re: [Xen-devel] Xen, MCFG acpi table and E820 address map > > On Wed, Sep 04, 2013 at 10:16:26AM +0100, Jan Beulich wrote: >> >>> On 04.09.13 at 03:13, Santosh Jodh <Santosh.Jodh@citrix.com> >> wrote: >> > Xen will use information from MCFG acpi table to access PCIe >> > extended configuration space. However, Xen validates MCFG table by >> > making sure that the addresses specified in the MCFG table is >> > correctly marked as reserved in the E820 address map. If it is >> not, >> > the MCFG table is ignored - thereby preventing Xen from accessing >> PCIe extended configuration space. >> > >> > I recently came across a workstation class system that supports >> > VT-d. This system BIOS has a valid MCFG table. The BIOS does NOT >> > report the MCFG addresses as reserved in the E820 address map. >> > However, the addresses ARE claimed as reserved via the ACPI >> > motherboard resource devnode (PNP0C01) mechanism. > > Could you tell me what machine this is? It would be good to know to > develop a patch against it. > [Santosh Jodh] Unfortunately, this is a new platform I cannot disclose > much about. If this is pre-release/experimental hardware, isn't the right thing to do to talk to the manufacturer and get them to fix the BIOS so it exposes the correct e820 map in the first place? [Santosh Jodh] yes - BIOS fix is in progress. My original query was - is this an exception? If it is a trend, then Xen should handle this better. Gordan ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Xen, MCFG acpi table and E820 address map 2013-09-04 16:26 ` Santosh Jodh @ 2013-09-05 7:47 ` Jan Beulich 2013-09-05 16:47 ` Santosh Jodh 0 siblings, 1 reply; 11+ messages in thread From: Jan Beulich @ 2013-09-05 7:47 UTC (permalink / raw) To: Santosh Jodh; +Cc: BorisOstrovsky, xen-devel, Gordan Bobic, David Vrabel >>> On 04.09.13 at 18:26, Santosh Jodh <Santosh.Jodh@citrix.com> wrote: > > -----Original Message----- > From: Gordan Bobic [mailto:gordan@bobich.net] > Sent: Wednesday, September 04, 2013 9:24 AM > To: Santosh Jodh > Cc: Konrad Rzeszutek Wilk; Jan Beulich; xen-devel; Boris Ostrovsky; David > Vrabel > Subject: Re: [Xen-devel] Xen, MCFG acpi table and E820 address map > > On Wed, 4 Sep 2013 16:14:55 +0000, Santosh Jodh <Santosh.Jodh@citrix.com> > wrote: >> -----Original Message----- >> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com] >> Sent: Wednesday, September 04, 2013 8:01 AM >> To: Jan Beulich >> Cc: Santosh Jodh; David Vrabel; xen-devel; Boris Ostrovsky >> Subject: Re: [Xen-devel] Xen, MCFG acpi table and E820 address map >> >> On Wed, Sep 04, 2013 at 10:16:26AM +0100, Jan Beulich wrote: >>> >>> On 04.09.13 at 03:13, Santosh Jodh <Santosh.Jodh@citrix.com> >>> wrote: >>> > Xen will use information from MCFG acpi table to access PCIe >>> > extended configuration space. However, Xen validates MCFG table by >>> > making sure that the addresses specified in the MCFG table is >>> > correctly marked as reserved in the E820 address map. If it is >>> not, >>> > the MCFG table is ignored - thereby preventing Xen from accessing >>> PCIe extended configuration space. >>> > >>> > I recently came across a workstation class system that supports >>> > VT-d. This system BIOS has a valid MCFG table. The BIOS does NOT >>> > report the MCFG addresses as reserved in the E820 address map. >>> > However, the addresses ARE claimed as reserved via the ACPI >>> > motherboard resource devnode (PNP0C01) mechanism. >> >> Could you tell me what machine this is? It would be good to know to >> develop a patch against it. >> [Santosh Jodh] Unfortunately, this is a new platform I cannot disclose >> much about. > > If this is pre-release/experimental hardware, isn't the right thing to do > to talk to the manufacturer and get them to fix the BIOS so it exposes the > correct e820 map in the first place? > [Santosh Jodh] yes - BIOS fix is in progress. My original query was - is > this an exception? If it is a trend, then Xen should handle this better. You seem to ignore my earlier response: Xen _does_ handle this. It's the pv-ops kernel that doesn't make use of the interface the hypervisor provides. And a remark to your mail response style - please try to set up your mail client such that it properly quotes the original text. The way you did your responses on this thread so far makes it pretty hard to spot your additions without first re-reading most of what was already said. Jan ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Xen, MCFG acpi table and E820 address map 2013-09-05 7:47 ` Jan Beulich @ 2013-09-05 16:47 ` Santosh Jodh 2013-09-06 6:32 ` Jan Beulich 0 siblings, 1 reply; 11+ messages in thread From: Santosh Jodh @ 2013-09-05 16:47 UTC (permalink / raw) To: Jan Beulich; +Cc: BorisOstrovsky, xen-devel, Gordan Bobic, David Vrabel > -----Original Message----- > From: Jan Beulich [mailto:JBeulich@suse.com] > Sent: Thursday, September 05, 2013 12:48 AM > To: Santosh Jodh > Cc: Gordan Bobic; David Vrabel; xen-devel; BorisOstrovsky; Konrad Rzeszutek > Wilk > Subject: RE: [Xen-devel] Xen, MCFG acpi table and E820 address map > > >>> On 04.09.13 at 18:26, Santosh Jodh <Santosh.Jodh@citrix.com> wrote: > > > > > -----Original Message----- > > From: Gordan Bobic [mailto:gordan@bobich.net] > > Sent: Wednesday, September 04, 2013 9:24 AM > > To: Santosh Jodh > > Cc: Konrad Rzeszutek Wilk; Jan Beulich; xen-devel; Boris Ostrovsky; > > David Vrabel > > Subject: Re: [Xen-devel] Xen, MCFG acpi table and E820 address map > > > > On Wed, 4 Sep 2013 16:14:55 +0000, Santosh Jodh > > <Santosh.Jodh@citrix.com> > > wrote: > >> -----Original Message----- > >> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com] > >> Sent: Wednesday, September 04, 2013 8:01 AM > >> To: Jan Beulich > >> Cc: Santosh Jodh; David Vrabel; xen-devel; Boris Ostrovsky > >> Subject: Re: [Xen-devel] Xen, MCFG acpi table and E820 address map > >> > >> On Wed, Sep 04, 2013 at 10:16:26AM +0100, Jan Beulich wrote: > >>> >>> On 04.09.13 at 03:13, Santosh Jodh <Santosh.Jodh@citrix.com> > >>> wrote: > >>> > Xen will use information from MCFG acpi table to access PCIe > >>> > extended configuration space. However, Xen validates MCFG table by > >>> > making sure that the addresses specified in the MCFG table is > >>> > correctly marked as reserved in the E820 address map. If it is > >>> not, > >>> > the MCFG table is ignored - thereby preventing Xen from accessing > >>> PCIe extended configuration space. > >>> > > >>> > I recently came across a workstation class system that supports > >>> > VT-d. This system BIOS has a valid MCFG table. The BIOS does NOT > >>> > report the MCFG addresses as reserved in the E820 address map. > >>> > However, the addresses ARE claimed as reserved via the ACPI > >>> > motherboard resource devnode (PNP0C01) mechanism. > >> > >> Could you tell me what machine this is? It would be good to know to > >> develop a patch against it. > >> [Santosh Jodh] Unfortunately, this is a new platform I cannot > >> disclose much about. > > > > If this is pre-release/experimental hardware, isn't the right thing > > to do to talk to the manufacturer and get them to fix the BIOS so it > > exposes the correct e820 map in the first place? > > [Santosh Jodh] yes - BIOS fix is in progress. My original query was - > > is this an exception? If it is a trend, then Xen should handle this better. > > You seem to ignore my earlier response: Xen _does_ handle this. It's the pv- > ops kernel that doesn't make use of the interface the hypervisor provides. [Santosh Jodh] I was not trying ignore your response, sorry if it came across like that. I was simply trying to explain reason for my original email. It's great to know Xen already handles this and look forward to dom0 fix going in as well. > > And a remark to your mail response style - please try to set up your mail > client such that it properly quotes the original text. The way you did your > responses on this thread so far makes it pretty hard to spot your additions > without first re-reading most of what was already said. [Santosh Jodh] Does this work better? ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Xen, MCFG acpi table and E820 address map 2013-09-05 16:47 ` Santosh Jodh @ 2013-09-06 6:32 ` Jan Beulich 0 siblings, 0 replies; 11+ messages in thread From: Jan Beulich @ 2013-09-06 6:32 UTC (permalink / raw) To: Santosh Jodh; +Cc: BorisOstrovsky, xen-devel, Gordan Bobic, David Vrabel >>> On 05.09.13 at 18:47, Santosh Jodh <Santosh.Jodh@citrix.com> wrote: >> And a remark to your mail response style - please try to set up your mail >> client such that it properly quotes the original text. The way you did your >> responses on this thread so far makes it pretty hard to spot your additions >> without first re-reading most of what was already said. > [Santosh Jodh] Does this work better? Yes. Tagging it with your name then also becomes unnecessary. In general - just see how (almost) everyone else does it... Jan ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Xen, MCFG acpi table and E820 address map 2013-09-04 16:24 ` Gordan Bobic 2013-09-04 16:26 ` Santosh Jodh @ 2013-09-05 7:44 ` Jan Beulich 1 sibling, 0 replies; 11+ messages in thread From: Jan Beulich @ 2013-09-05 7:44 UTC (permalink / raw) To: Gordan Bobic, Santosh Jodh; +Cc: xen-devel, BorisOstrovsky, David Vrabel >>> On 04.09.13 at 18:24, Gordan Bobic <gordan@bobich.net> wrote: > On Wed, 4 Sep 2013 16:14:55 +0000, Santosh Jodh > <Santosh.Jodh@citrix.com> wrote: >> -----Original Message----- >> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com] >> Sent: Wednesday, September 04, 2013 8:01 AM >> To: Jan Beulich >> Cc: Santosh Jodh; David Vrabel; xen-devel; Boris Ostrovsky >> Subject: Re: [Xen-devel] Xen, MCFG acpi table and E820 address map >> >> On Wed, Sep 04, 2013 at 10:16:26AM +0100, Jan Beulich wrote: >>> >>> On 04.09.13 at 03:13, Santosh Jodh <Santosh.Jodh@citrix.com> >>> wrote: >>> > Xen will use information from MCFG acpi table to access PCIe >>> > extended configuration space. However, Xen validates MCFG table by >>> > making sure that the addresses specified in the MCFG table is >>> > correctly marked as reserved in the E820 address map. If it is >>> not, >>> > the MCFG table is ignored - thereby preventing Xen from accessing >>> PCIe extended configuration space. >>> > >>> > I recently came across a workstation class system that supports >>> > VT-d. This system BIOS has a valid MCFG table. The BIOS does NOT >>> > report the MCFG addresses as reserved in the E820 address map. >>> > However, the addresses ARE claimed as reserved via the ACPI >>> > motherboard resource devnode (PNP0C01) mechanism. >> >> Could you tell me what machine this is? It would be good to know to >> develop a patch against it. >> [Santosh Jodh] Unfortunately, this is a new platform I cannot >> disclose much about. > > If this is pre-release/experimental hardware, isn't the > right thing to do to talk to the manufacturer and get them > to fix the BIOS so it exposes the correct e820 map in the > first place? No. As said in an earlier reply, this is legitimate behavior that - as shown - Linux can deal with. It's just the lack of the use of the intended interface by the pv-ops kernel that gets in the way here. In fact I have at least one system that also exhibits such behavior; whether that's also a pre-release specific situation (most of the systems I use for regular testing found their way here by other than purchase) I can't tell, but it at least got me to spot the Linux side solution and integrate it with Xen. Jan ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2013-09-06 6:32 UTC | newest] Thread overview: 11+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2013-09-04 1:13 Xen, MCFG acpi table and E820 address map Santosh Jodh 2013-09-04 9:16 ` Jan Beulich 2013-09-04 15:01 ` Konrad Rzeszutek Wilk 2013-09-04 15:12 ` Jan Beulich 2013-09-04 16:14 ` Santosh Jodh 2013-09-04 16:24 ` Gordan Bobic 2013-09-04 16:26 ` Santosh Jodh 2013-09-05 7:47 ` Jan Beulich 2013-09-05 16:47 ` Santosh Jodh 2013-09-06 6:32 ` Jan Beulich 2013-09-05 7:44 ` Jan Beulich
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.