RE: [Lhms-devel] [PATCH 0/7] Fragmentation Avoidance V19

Rohit Seth
2005-11-07T21:31:11Z

On Mon, 2005-11-07 at 15:11 -0600, Adam Litke wrote:
> On Mon, 2005-11-07 at 12:51 -0800, Rohit Seth wrote:
>
> > Isn't it true that most of the times we'll need to be worrying about
> > run-time allocation of memory (using malloc or such) as compared to
> > static.
>
> It really depends on the workload. I've run HPC apps with 10+GB data
> segments. I've also worked with applications that would benefit from a
> hugetlb-enabled morecore (glibc malloc/sbrk). I'd like to see one
> standard hugetlb preload library that handles every different "memory
> object" we care about (static and dynamic). That's what I'm working on
> now.
>

As said below, we will need this functionality even for code pages. I
would rather have the changes absorbed in run-time loader rather than
having a preload library. Makes it easy to manage.

malloc/sbrks are the interesting part that does pose some challenges (as
in some archs different address space is reserved hugetlb). Moreover,
it will also be critical that existing semantics of normal pages is
maintained even when the application ends up using hugepages.

> > We'll need a similar flag for even code pages to start using hugetlb
> > pages. In this case to keep the kernel changes to minimum, RTLD will
> > need to modified.
>
> Yes, I foresee the functionality currently in my preload lib to exist in
> RTLD at some point way down the road.
>

It will be much sooner...

-rohit