[CFT] tree-based bootmem

[CFT] tree-based bootmem

[William Lee Irwin III]

I've waited a week or two for bug reports to roll in from early testers
(including myself), and no reports have come in.

So now I humbly request the assistance of a larger userbase in testing
the bootmem patch.

The bootmem subsystem is used for tracking and reserving usable physical
memory during early boot. A patch such as this, which tracks extents
explicitly, was requested by several people. The patch is also intended
to improve discontiguous memory support in that subsystem.

The only user-visible effect of the patch should be to save small
amounts of memory by virtue of somewhat more accurate tracking of the
boundaries of allocated regions, on the order of 4-12KB on i386.

The patch has been tested successfully on i386, IA64, mipsel, Super-H,
ppc64, sparc64, and sparc32, when applied against 2.4.15.

The patch (against various kernel versions) is available from

	ftp://ftp.kernel.org/pub/linux/kernel/people/wli/bootmem/

Those using the 2.4.15 patch will need to apply overflow-2.4.15 in
addition to bootmem-2.4.15


Thanks,
Bill

Re: [CFT] tree-based bootmem

[Robert Love]

On Sat, 2001-12-08 at 18:40, William Lee Irwin III wrote:
> I've waited a week or two for bug reports to roll in from early testers
> (including myself), and no reports have come in.
> 
> So now I humbly request the assistance of a larger userbase in testing
> the bootmem patch.

Patch successfully tested on (3) i386 machines, an SH4 machine, and a
sparc32 machine.  And on the low memory SH4 machine, the
finer-granularity of addressing is welcome.

Now, when do we see some interface changes to bootmem that take
advantage of your patch? :-)

	Robert Love

Re: [CFT] tree-based bootmem

[William Lee Irwin III]

On Sun, Dec 09, 2001 at 06:29:26PM -0500, Robert Love wrote:
> Patch successfully tested on (3) i386 machines, an SH4 machine, and a
> sparc32 machine.  And on the low memory SH4 machine, the
> finer-granularity of addressing is welcome.

Excellent!

On Sun, Dec 09, 2001 at 06:29:26PM -0500, Robert Love wrote:
> Now, when do we see some interface changes to bootmem that take
> advantage of your patch? :-)

Every bootmem caller, that is, setup_arch() for every architecture,
would need to be adjusted. The changes are not particularly difficult
to make, but they require the consent and/or assistance of various arch
maintainers, and I have only a limited selection of target architectures
to test and develop things on.

The interface changes I am interested in making are those that make
bootmem easier to use and eliminate duplicated code across various
architectures. For instance, to set up the bootmem allocator, one must
now build up an extent-based representation of memory fragments, and
then initialize separate bootmem state structures for each fragment in
order to accommodate the direct-mapped table representation now in use.
An alternative interface would be to pass in an initial tree node pool
for early reservations, finalizing the memory map to notify bootmem that
allocations may now be done, and afterward performing page stealing
when the node pool is exhausted by fragmentation from dynamic allocations.
(Personally, I'd very much like to get rid of NR_SEGMENTS in my code.)

I'd like to see the mechanism integrated before trying to bring in
the interface changes, and I'd also like to bring in some of the
other people involved in discontiguous memory support essentially
for having consensus around what the interface should be and to
integrate the patch with existing ones, e.g. discontig support for SN1.
Another important aspect of it is making sure they like this, too.
The discontig support for SN1 does not revolve around bootmem, but
requires some bootmem changes. I'm quite unaware of how discontiguous
memory is dealt with for Wildfire, so if someone could point me to
what's going on there, I'd be much obliged.

Another cause for my extreme conservatism with respect to testing
and "release schedule" is that the architectures requiring extreme
measures for discontiguous memory support already have something in
place just to work at all, and thus far, I've heard very little back
from those who maintain them, aside from some early comments from
discontig-devel members mentioning that the mechanism was elegant, and
that it would be good to bring a large change to common code to lkml
for review and testing. As these are the people the patch is intended
to benefit most, their feedback has much importance.


For the moment, I'll cc: this to discontig-devel and see what feedback
comes from there.

For those to whom this mail comes out of context, my current patches
may be found at

	ftp://ftp.kernel.org/pub/linux/kernel/people/wli/bootmem/

The bootmem-2.4.15 patch will also require the overflow-2.4.15 patch
for an important bug fix involving integer overflows.

Admittedly, little has changed in several weeks. I chose to leave
things largely as they are for the sake of additional testing as
opposed to adding in more features immediately.


Thanks,
Bill