Re: [rfc][patch 0/3] bootmem2: a memory block-oriented boot time allocator

Robin Holt
2008-05-05T16:04:43Z

On Mon, May 05, 2008 at 08:23:34AM -0700, Linus Torvalds wrote:
>
>
> On Mon, 5 May 2008, Johannes Weiner wrote:
> >
> > here is a bootmem allocator replacement that uses one bitmap for all
> > available pages and works with a model of contiguous memory blocks
> > that reside on nodes instead of nodes only as the current allocator
> > does.
>
> Won't this have problems with huge non-contiguous areas?
>
> Some setups have traditionally had node memory separated in physical space
> by the high bits of the memory address, and using a single bitmap for such
> things would potentially be basically impossible - even with a single bit
> per page, the "span" of possible pages is potentially just too high, even
> if the nodes themselves don't have tons of memory, because the memory is
> just very spread out - and allocating the initial bitmap may not work
> reliably.
>
> Now, admittedly I don't know if we even support that kind of thing or if
> people really do things that way any more, so maybe it's not an issue.

SGI sn2 architecture does. Each DIMM bank is allocated a 16GB range
of physical addresses. There are up to four banks per node. The node
number is stuck into higher portions of the address, giving a gap between
nodes of 256GB. With a potential of 1024 nodes, you would have a very
large array.

Additionally on our upcoming UV systems, there will potentially be a
hole between the bulk of memory and a small amount addressable at the
high end of the address range (slightly short of 16TB) with the typical
gap being on the order of 15TB.

Thanks,
Robin Holt