Re: [Lse-tech] Re: hugetlb demand paging patch part [2/3]

'David Gibson'
2004-04-19T00:47:36Z

On Sun, Apr 18, 2004 at 12:36:05PM -0500, Ray Bryant wrote:
>
>
> 'David Gibson' wrote:
>
> >
> >
> >My main interest in it is as a prerequisite for various methods of
> >"automatically" using hugepages for programs where it is difficult to
> >manually code them to use hugetlbfs. In particular, think HPC
> >monsters written in FORTRAN. e.g. automatically putting suitable
> >aligned anonymous mmap()s in hugepages under some circumstances (I
> >can't say I like that idea much), using an LD_PRELOAD to put
> >malloc()ated memory into hugepages, or using a hacked ELF loader to
> >put the BSS section (again, think FORTRAN) into hugepages (actually
> >easier and less ugly than it sounds).
> >
>
> Well, that certainly is a laudable goal. At the moment, one usually has to
> resort to such things as POINTER variables and the like to get access to
> hugetlbpage segments. Unfortunately, some of our experiments with the
> Intel compiler for ia64 have indicated that the generated code can be
> significantly slower when arrays are referenced off of POINTER variables
> than when the same arrays are referenced out of COMMON, thus eliminating
> the performance gain of HUGETLB pages.

Well, that's one problem with using POINTERs, but I think perhaps the
more serious one is that a lot of HPC code is written by scientists
who aren't programmers, and who still think in FORTRAN77.

> My question was really intended to address applying development effort to
> things that the users of hugetlbpages will likely actually use. For
> example, it seems pointless to worry too much about demand paging of
> hugetlbpages out to disk. Anyone who uses hugetlbpages for the performance
> boost they give will also likely have rightsized their problem or machine
> configuration to eliminate any swapping.

Well, indeed. Note that this "demand paging" set of patches don't
actually do paging out to disk - they just do on-demand allocation of
physical hugepages, rather than prefaulting them all. My only real
interest in it is that part of the mechanism is identical to that
needed for COW (handle_hugetlb_mm_fault(), in particular).

> >In any of these cases having the memory have different semantics
> >(MAP_SHARED) to normal anonymous memory would clearly be a Bad Thing.