Re: [PATCH 2/2] mm: thp: fix transparent_hugepage/defrag = madvise || always

[Andrea Arcangeli <aarcange@redhat.com>]

Hello,

On Mon, Aug 20, 2018 at 08:35:17AM -0400, Zi Yan wrote:
> I think this can also be triggered in khugepaged. In collapse_huge_page(), khugepaged_alloc_page()
> would also cause DIRECT_RECLAIM if defrag==always, since GFP_TRANSHUGE implies __GFP_DIRECT_RECLAIM.
> 
> But is it an expected behavior of khugepaged?

That's a good point, and answer it not obvious. It's not an apple to
apple comparison because khugepaged is not increasing the overall
memory footprint of the app. The pages that gets compacted gets freed
later. However not all memory of the node may be possible to compact
100% so it's not perfectly a 1:1 exchange and it could require some
swap to succeed compaction.

So we may want to look also at khugepaged later, but it's not obvious
it needs fixing too. It'd weaken a bit khugepaged to add
__GFP_COMPACT_ONLY to it, if compaction returns COMPACT_SKIPPED
especially.

As opposed in the main allocation path (when memory footprint
definitely increases) I even tried to still allow reclaim only for
COMPACT_SKIPPED and it still caused swapout storms because new THP
kept being added to the local node as the old memory was swapped out
to make more free memory to compact more 4k pages. Otherwise I could
have gotten away with using __GFP_NORETRY instead of
__GFP_COMPACT_ONLY but it wasn't nearly enough.

Similarly to khugepaged NUMA balancing also uses __GFP_THISNODE but if
it decided to migrate a page to such node supposedly there's a good
reason to call reclaim to allocate the page there if needed. That also
is freeing memory on node and adding memory to another node, and it's
not increasing the memory footprint overall (unlike khugepaged, it
increases the footprint cross-node though).

Thanks,
Andrea