Memory fragmentation, order>0 allocation, and 4.0 dynamic RAM optimization features

[Dan Magenheimer <dan.magenheimer@oracle.com>]

In a recent thread:

http://lists.xensource.com/archives/html/xen-devel/2010-02/msg00295.html

Jan Beulich points out that the memory fragmentation that results
from Transcendent Memory ("tmem") sometimes causes problems for
domain creation and PV migration because the shadow code requires
order=2 allocations and the domain struct is order=4.

Though tmem accelerates fragmentation, I *think* this fragmentation
can occur with page sharing/swapping, and possibly PoD.  In fact,
I think it can occur even with just ballooning.

I think the domain struct issue should be relatively easy to
resolve (though maybe with a large patch), but the shadow code
may be much harder.

But unless the shadow code is also fixed, theoretically 75% of RAM
could be "free" but domain creation/migration failures may occur,
reported only as insufficient memory.

Clearly it's too late to fix this for 4.0 but, given that 4.0-based
product announcements are likely to emphasize the new 4.0 memory
optimization technologies, it might be good to resolve it very
early in 4.1/xen-unstable development.

Comments?

Are there other known order>0 allocations that might result
in similar issues?

Thanks,
Dan