Re: [RFC 0/4] Transparent on-demand struct page initialization embedded in the buddy allocator

Robin Holt
2013-07-15T15:16:25Z

On Fri, Jul 12, 2013 at 10:27:56AM +0200, Ingo Molnar wrote:
>
> * Robin Holt <holt@sgi.com> wrote:
>
> > [...]
> >
> > With this patch, we did boot a 16TiB machine. Without the patches, the
> > v3.10 kernel with the same configuration took 407 seconds for
> > free_all_bootmem. With the patches and operating on 2MiB pages instead
> > of 1GiB, it took 26 seconds so performance was improved. I have no feel
> > for how the 1GiB chunk size will perform.
>
> That's pretty impressive.

And WRONG!

That is a 15x speedup in the freeing of memory at the free_all_bootmem
point. That is _NOT_ the speedup from memmap_init_zone. I forgot to
take that into account as Nate pointed out this morning in a hallway
discussion. Before, on the 16TiB machine, memmap_init_zone took 1152
seconds. After, it took 50. If it were a straight 1/512th, we would
have expected that 1152 to be something more on the line of 2-3 so there
is still significant room for improvement.

Sorry for the confusion.

> It's still a 15x speedup instead of a 512x speedup, so I'd say there's
> something else being the current bottleneck, besides page init
> granularity.
>
> Can you boot with just a few gigs of RAM and stuff the rest into hotplug
> memory, and then hot-add that memory? That would allow easy profiling of
> remaining overhead.

Nate and I will be working on other things for the next few hours hoping
there is a better answer to the first question we asked about there
being a way to test a page other than comparing against all zeroes to
see if it has been initialized.

Thanks,
Robin