* ACPI fixmap overflow @ 2016-11-15 19:34 Boris Ostrovsky 2016-11-15 19:45 ` Andrew Cooper 0 siblings, 1 reply; 8+ messages in thread From: Boris Ostrovsky @ 2016-11-15 19:34 UTC (permalink / raw) To: xen-devel; +Cc: Andrew Cooper, Alex Thorlton, Jan Beulich In addition to running out of e820 entries on that large machine that Alex was referring to in [0] he is also running out of ACPI fixmap space while parsing MADT (the box has *lots* of processors). The quick-and-dirty solution is to increase NUM_FIXMAP_ACPI_PAGES but I wonder whether we should move to dynamic memory allocation. -boris [0] https://lists.xenproject.org/archives/html/xen-devel/2016-11/msg01077.html _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: ACPI fixmap overflow 2016-11-15 19:34 ACPI fixmap overflow Boris Ostrovsky @ 2016-11-15 19:45 ` Andrew Cooper 2016-11-15 20:39 ` Boris Ostrovsky 0 siblings, 1 reply; 8+ messages in thread From: Andrew Cooper @ 2016-11-15 19:45 UTC (permalink / raw) To: Boris Ostrovsky, xen-devel; +Cc: Alex Thorlton, Jan Beulich On 15/11/16 19:34, Boris Ostrovsky wrote: > In addition to running out of e820 entries on that large machine that > Alex was referring to in [0] he is also running out of ACPI fixmap space > while parsing MADT (the box has *lots* of processors). The > quick-and-dirty solution is to increase NUM_FIXMAP_ACPI_PAGES but I > wonder whether we should move to dynamic memory allocation. Why do we use fixmap anyway? It doesn't look too hard to reorder vm_init() slightly higher, and be able to use ioremap() for all APCI tables. ~Andrew _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: ACPI fixmap overflow 2016-11-15 19:45 ` Andrew Cooper @ 2016-11-15 20:39 ` Boris Ostrovsky 2016-11-15 20:44 ` Andrew Cooper 0 siblings, 1 reply; 8+ messages in thread From: Boris Ostrovsky @ 2016-11-15 20:39 UTC (permalink / raw) To: Andrew Cooper, xen-devel; +Cc: Alex Thorlton, Jan Beulich On 11/15/2016 02:45 PM, Andrew Cooper wrote: > On 15/11/16 19:34, Boris Ostrovsky wrote: >> In addition to running out of e820 entries on that large machine that >> Alex was referring to in [0] he is also running out of ACPI fixmap space >> while parsing MADT (the box has *lots* of processors). The >> quick-and-dirty solution is to increase NUM_FIXMAP_ACPI_PAGES but I >> wonder whether we should move to dynamic memory allocation. > Why do we use fixmap anyway? It doesn't look too hard to reorder > vm_init() slightly higher, and be able to use ioremap() for all APCI tables. Hmm... Let me see how possible this is. Just moving it up won't work since heap allocator is initialized after ACPI tables. -boris _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: ACPI fixmap overflow 2016-11-15 20:39 ` Boris Ostrovsky @ 2016-11-15 20:44 ` Andrew Cooper 2016-11-15 20:51 ` Boris Ostrovsky 0 siblings, 1 reply; 8+ messages in thread From: Andrew Cooper @ 2016-11-15 20:44 UTC (permalink / raw) To: Boris Ostrovsky, xen-devel; +Cc: Alex Thorlton, Jan Beulich On 15/11/2016 20:39, Boris Ostrovsky wrote: > On 11/15/2016 02:45 PM, Andrew Cooper wrote: >> On 15/11/16 19:34, Boris Ostrovsky wrote: >>> In addition to running out of e820 entries on that large machine that >>> Alex was referring to in [0] he is also running out of ACPI fixmap space >>> while parsing MADT (the box has *lots* of processors). The >>> quick-and-dirty solution is to increase NUM_FIXMAP_ACPI_PAGES but I >>> wonder whether we should move to dynamic memory allocation. >> Why do we use fixmap anyway? It doesn't look too hard to reorder >> vm_init() slightly higher, and be able to use ioremap() for all APCI tables. > Hmm... Let me see how possible this is. Just moving it up won't work > since heap allocator is initialized after ACPI tables. We have plenty of usable PTEs already allocated at boot, mainly from the init pagetables. Given a static __init vm_bitmap, a new boot-time-only vm range should be usable without any heap allocations at all. ~Andrew _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: ACPI fixmap overflow 2016-11-15 20:44 ` Andrew Cooper @ 2016-11-15 20:51 ` Boris Ostrovsky 2016-11-16 10:08 ` Jan Beulich 0 siblings, 1 reply; 8+ messages in thread From: Boris Ostrovsky @ 2016-11-15 20:51 UTC (permalink / raw) To: Andrew Cooper, xen-devel; +Cc: Alex Thorlton, Jan Beulich On 11/15/2016 03:44 PM, Andrew Cooper wrote: > On 15/11/2016 20:39, Boris Ostrovsky wrote: >> On 11/15/2016 02:45 PM, Andrew Cooper wrote: >>> On 15/11/16 19:34, Boris Ostrovsky wrote: >>>> In addition to running out of e820 entries on that large machine that >>>> Alex was referring to in [0] he is also running out of ACPI fixmap space >>>> while parsing MADT (the box has *lots* of processors). The >>>> quick-and-dirty solution is to increase NUM_FIXMAP_ACPI_PAGES but I >>>> wonder whether we should move to dynamic memory allocation. >>> Why do we use fixmap anyway? It doesn't look too hard to reorder >>> vm_init() slightly higher, and be able to use ioremap() for all APCI tables. >> Hmm... Let me see how possible this is. Just moving it up won't work >> since heap allocator is initialized after ACPI tables. > We have plenty of usable PTEs already allocated at boot, mainly from the > init pagetables. Given a static __init vm_bitmap, a new boot-time-only > vm range should be usable without any heap allocations at all. Wouldn't that (using pre-allocated PTEs), in a way, be equivalent to increasing fixmap size? -boris _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: ACPI fixmap overflow 2016-11-15 20:51 ` Boris Ostrovsky @ 2016-11-16 10:08 ` Jan Beulich 2016-11-17 15:01 ` Boris Ostrovsky 0 siblings, 1 reply; 8+ messages in thread From: Jan Beulich @ 2016-11-16 10:08 UTC (permalink / raw) To: Andrew Cooper, Boris Ostrovsky; +Cc: Alex Thorlton, xen-devel >>> On 15.11.16 at 21:51, <boris.ostrovsky@oracle.com> wrote: > On 11/15/2016 03:44 PM, Andrew Cooper wrote: >> On 15/11/2016 20:39, Boris Ostrovsky wrote: >>> On 11/15/2016 02:45 PM, Andrew Cooper wrote: >>>> On 15/11/16 19:34, Boris Ostrovsky wrote: >>>>> In addition to running out of e820 entries on that large machine that >>>>> Alex was referring to in [0] he is also running out of ACPI fixmap space >>>>> while parsing MADT (the box has *lots* of processors). The >>>>> quick-and-dirty solution is to increase NUM_FIXMAP_ACPI_PAGES but I >>>>> wonder whether we should move to dynamic memory allocation. >>>> Why do we use fixmap anyway? It doesn't look too hard to reorder >>>> vm_init() slightly higher, and be able to use ioremap() for all APCI tables. >>> Hmm... Let me see how possible this is. Just moving it up won't work >>> since heap allocator is initialized after ACPI tables. >> We have plenty of usable PTEs already allocated at boot, mainly from the >> init pagetables. Given a static __init vm_bitmap, a new boot-time-only >> vm range should be usable without any heap allocations at all. > > Wouldn't that (using pre-allocated PTEs), in a way, be equivalent to > increasing fixmap size? Indeed. For the time being I think growing the fixmap should be fine. Clearly it being a fixed 4 pages has been wrong for a long time - there's no point in it being smaller than MAX_LOCAL_APIC x2apic entries, plus min(MAX_LOCAL_APIC, 256) lapic ones, plus MAX_IO_APICS ioapic ones etc. Otoh the actual parsing of MADT happens after the heap allocator has been initialized. The only earlier use is via acpi_table_init() -> check_multiple_madt(); acpi_initialize_tables() doesn't appear to map full tables. And check_multiple_madt() not being able to map the full table would not prevent boot from continuing afaics. So Andrew's suggestion to switch to dynamic mapping earlier would still seem to be possible (and then preferred). In fact acpi_boot_init() already gets called after vm_init(), so switching acpi_os_map_memory() to use dynamic mappings when >= SYS_STATE_boot should already work today (on x86 at least, not sure about ARM). Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: ACPI fixmap overflow 2016-11-16 10:08 ` Jan Beulich @ 2016-11-17 15:01 ` Boris Ostrovsky 2016-11-17 16:04 ` Alex Thorlton 0 siblings, 1 reply; 8+ messages in thread From: Boris Ostrovsky @ 2016-11-17 15:01 UTC (permalink / raw) To: Jan Beulich, Andrew Cooper; +Cc: Alex Thorlton, xen-devel On 11/16/2016 05:08 AM, Jan Beulich wrote: >>>> On 15.11.16 at 21:51, <boris.ostrovsky@oracle.com> wrote: >> On 11/15/2016 03:44 PM, Andrew Cooper wrote: >>> On 15/11/2016 20:39, Boris Ostrovsky wrote: >>>> On 11/15/2016 02:45 PM, Andrew Cooper wrote: >>>>> On 15/11/16 19:34, Boris Ostrovsky wrote: >>>>>> In addition to running out of e820 entries on that large machine that >>>>>> Alex was referring to in [0] he is also running out of ACPI fixmap space >>>>>> while parsing MADT (the box has *lots* of processors). The >>>>>> quick-and-dirty solution is to increase NUM_FIXMAP_ACPI_PAGES but I >>>>>> wonder whether we should move to dynamic memory allocation. >>>>> Why do we use fixmap anyway? It doesn't look too hard to reorder >>>>> vm_init() slightly higher, and be able to use ioremap() for all APCI tables. >>>> Hmm... Let me see how possible this is. Just moving it up won't work >>>> since heap allocator is initialized after ACPI tables. >>> We have plenty of usable PTEs already allocated at boot, mainly from the >>> init pagetables. Given a static __init vm_bitmap, a new boot-time-only >>> vm range should be usable without any heap allocations at all. >> Wouldn't that (using pre-allocated PTEs), in a way, be equivalent to >> increasing fixmap size? > Indeed. For the time being I think growing the fixmap should be > fine. Clearly it being a fixed 4 pages has been wrong for a long > time - there's no point in it being smaller than MAX_LOCAL_APIC > x2apic entries, plus min(MAX_LOCAL_APIC, 256) lapic ones, plus > MAX_IO_APICS ioapic ones etc. > > Otoh the actual parsing of MADT happens after the heap allocator > has been initialized. The only earlier use is via acpi_table_init() -> > check_multiple_madt(); acpi_initialize_tables() doesn't appear to > map full tables. And check_multiple_madt() not being able to map > the full table would not prevent boot from continuing afaics. So > Andrew's suggestion to switch to dynamic mapping earlier would > still seem to be possible (and then preferred). In fact > acpi_boot_init() already gets called after vm_init(), so switching > acpi_os_map_memory() to use dynamic mappings when >> = SYS_STATE_boot should already work today (on x86 at least, > not sure about ARM). Yes, switching to ioremap on SYS_STATE_boot seems to work. I asked Alex to test this on OVM (which is 4.4-based). -boris _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: ACPI fixmap overflow 2016-11-17 15:01 ` Boris Ostrovsky @ 2016-11-17 16:04 ` Alex Thorlton 0 siblings, 0 replies; 8+ messages in thread From: Alex Thorlton @ 2016-11-17 16:04 UTC (permalink / raw) To: Boris Ostrovsky; +Cc: Andrew Cooper, Alex Thorlton, Jan Beulich, xen-devel On Thu, Nov 17, 2016 at 10:01:15AM -0500, Boris Ostrovsky wrote: > On 11/16/2016 05:08 AM, Jan Beulich wrote: > >>>> On 15.11.16 at 21:51, <boris.ostrovsky@oracle.com> wrote: > >> On 11/15/2016 03:44 PM, Andrew Cooper wrote: > >>> On 15/11/2016 20:39, Boris Ostrovsky wrote: > >>>> On 11/15/2016 02:45 PM, Andrew Cooper wrote: > >>>>> On 15/11/16 19:34, Boris Ostrovsky wrote: > >>>>>> In addition to running out of e820 entries on that large machine that > >>>>>> Alex was referring to in [0] he is also running out of ACPI fixmap space > >>>>>> while parsing MADT (the box has *lots* of processors). The > >>>>>> quick-and-dirty solution is to increase NUM_FIXMAP_ACPI_PAGES but I > >>>>>> wonder whether we should move to dynamic memory allocation. > >>>>> Why do we use fixmap anyway? It doesn't look too hard to reorder > >>>>> vm_init() slightly higher, and be able to use ioremap() for all APCI tables. > >>>> Hmm... Let me see how possible this is. Just moving it up won't work > >>>> since heap allocator is initialized after ACPI tables. > >>> We have plenty of usable PTEs already allocated at boot, mainly from the > >>> init pagetables. Given a static __init vm_bitmap, a new boot-time-only > >>> vm range should be usable without any heap allocations at all. > >> Wouldn't that (using pre-allocated PTEs), in a way, be equivalent to > >> increasing fixmap size? > > Indeed. For the time being I think growing the fixmap should be > > fine. Clearly it being a fixed 4 pages has been wrong for a long > > time - there's no point in it being smaller than MAX_LOCAL_APIC > > x2apic entries, plus min(MAX_LOCAL_APIC, 256) lapic ones, plus > > MAX_IO_APICS ioapic ones etc. > > > > Otoh the actual parsing of MADT happens after the heap allocator > > has been initialized. The only earlier use is via acpi_table_init() -> > > check_multiple_madt(); acpi_initialize_tables() doesn't appear to > > map full tables. And check_multiple_madt() not being able to map > > the full table would not prevent boot from continuing afaics. So > > Andrew's suggestion to switch to dynamic mapping earlier would > > still seem to be possible (and then preferred). In fact > > acpi_boot_init() already gets called after vm_init(), so switching > > acpi_os_map_memory() to use dynamic mappings when > >> = SYS_STATE_boot should already work today (on x86 at least, > > not sure about ARM). > > Yes, switching to ioremap on SYS_STATE_boot seems to work. I asked Alex > to test this on OVM (which is 4.4-based). FYI, I'm going to get this done a little later today. I'll let you know how it worked out this evening! - Alex _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2016-11-17 16:04 UTC | newest] Thread overview: 8+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2016-11-15 19:34 ACPI fixmap overflow Boris Ostrovsky 2016-11-15 19:45 ` Andrew Cooper 2016-11-15 20:39 ` Boris Ostrovsky 2016-11-15 20:44 ` Andrew Cooper 2016-11-15 20:51 ` Boris Ostrovsky 2016-11-16 10:08 ` Jan Beulich 2016-11-17 15:01 ` Boris Ostrovsky 2016-11-17 16:04 ` Alex Thorlton
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).