Re: [PATCH v10 00/35] kmemcg shrinkers

Andrew Morton
2013-06-05T23:07:21Z

On Mon, 3 Jun 2013 23:29:29 +0400 Glauber Costa <glommer@openvz.org> wrote:

> Andrew,
>
> This submission contains one small bug fix over the last one. I have been
> testing it regularly and believe this is ready for merging. I have follow up
> patches for this series, with a few improvements (namely: dynamic sized
> list_lru node arrays, memcg flush-at-destruction, kmemcg shrinking setting
> limit < usage). But since this series is already quite mature - and very
> extensive, I don't believe that adding new patches would make them receive the
> appropriate level of review. So please advise me if there is anything crucial
> missing in here. Thanks!
>
> Hi,
>
> This patchset implements targeted shrinking for memcg when kmem limits are
> present. So far, we've been accounting kernel objects but failing allocations
> when short of memory. This is because our only option would be to call the
> global shrinker, depleting objects from all caches and breaking isolation.
>
> The main idea is to associate per-memcg lists with each of the LRUs. The main
> LRU still provides a single entry point and when adding or removing an element
> from the LRU, we use the page information to figure out which memcg it belongs
> to and relay it to the right list.
>
> Base work:
> ==========
>
> Please note that this builds upon the recent work from Dave Chinner that
> sanitizes the LRU shrinking API and make the shrinkers node aware. Node
> awareness is not *strictly* needed for my work, but I still perceive it
> as an advantage. The API unification is a major need, and I build upon it
> heavily. That allows us to manipulate the LRUs without knowledge of the
> underlying objects with ease. This time, I am including that work here as
> a baseline.

This patchset is huge.

My overall take is that the patchset is massive and intrusive and scary
:( I'd like to see more evidence that the memcg people (mhocko, hannes,
kamezawa etc) have spent quality time reviewing and testing this code.
There really is a lot of it!

I haven't seen any show-stoppers yet so I guess I'll slam it all into
-next and cross fingers. I would ask that the relevant developers set
aside a solid day to read and runtime test it all. Realistically, it's
likely to take considerably more time that that.

I do expect that I'll drop the entire patchset again for the next
version, if only because the next version should withdraw all the
switch-random-code-to-xfs-coding-style changes...

I'm thinking that we should approach this in two stages: all the new
shrinker stuff separated from the memcg_kmem work. So we merge
everything up to "shrinker: Kill old ->shrink API" and then continue to
work on the memcg things?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: email@kvack.org