Re: [RFC][PATCH 0/5] Memory controller soft limit patches (v8)

[Balbir Singh <balbir@linux.vnet.ibm.com>]

On Fri, Jul 10, 2009 at 10:23 AM, KAMEZAWA
Hiroyuki<kamezawa.hiroyu@jp.fujitsu.com> wrote:
> On Thu, 09 Jul 2009 22:44:41 +0530
> Balbir Singh <balbir@linux.vnet.ibm.com> wrote:
>
>>
>> From: Balbir Singh <balbir@linux.vnet.ibm.com>
>>
>> New Feature: Soft limits for memory resource controller.
>>
>> Here is v8 of the new soft limit implementation. Soft limits is a new feature
>> for the memory resource controller, something similar has existed in the
>> group scheduler in the form of shares. The CPU controllers interpretation
>> of shares is very different though.
>>
>> Soft limits are the most useful feature to have for environments where
>> the administrator wants to overcommit the system, such that only on memory
>> contention do the limits become active. The current soft limits implementation
>> provides a soft_limit_in_bytes interface for the memory controller and not
>> for memory+swap controller. The implementation maintains an RB-Tree of groups
>> that exceed their soft limit and starts reclaiming from the group that
>> exceeds this limit by the maximum amount.
>>
>> v8 has come out after a long duration, we were held back by bug fixes
>> (most notably swap cache leak fix) and Kamezawa-San has his series of
>> patches for soft limits. Kamezawa-San asked me to refactor these patches
>> to make the data structure per-node-per-zone.
>>
>> TODOs
>>
>> 1. The current implementation maintains the delta from the soft limit
>>    and pushes back groups to their soft limits, a ratio of delta/soft_limit
>>    might be more useful
>> 2. Small optimizations that I intend to push in v9, if the v8 design looks
>>    good and acceptable.
>>
>> Tests
>> -----
>>
>> I've run two memory intensive workloads with differing soft limits and
>> seen that they are pushed back to their soft limit on contention. Their usage
>> was their soft limit plus additional memory that they were able to grab
>> on the system. Soft limit can take a while before we see the expected
>> results.
>>
>
> Before pointing out nitpicks, here are my impressions.
>
>  1. seems good in general.
>

Thanks

>  2. Documentation is not enough. I think it's necessary to write "excuse" as
>    "soft-limit is built on complex memory management system's behavior, then,
>     this may not work as you expect. But in many case, this works well.
>     please take this as best-effort service" or some.
>

Sure, I'll revisit it and update.

>  3. Using "jiffies" again is not good. plz use other check or event counter.
>

Yes, I considered event based sampling and update. I wrote the code,
but then realized that it works really well if I keep the sampling per
cpu, otherwise it does not scale well. My problem with per-cpu
sampling is that the view we get could vary drastically if we migrated
or the task migrated to a different node and allocated memory.

>  4. I think it's better to limit soltlimit only against root of hierarcy node.
>    (use_hierarchy=1) I can't explain how the system works if several soft limits
>    are set to root and its children under a hierarchy.
>

The idea is that if we add a node and it has children and that node
goes above the soft limit, we'll do hierarchical reclaim from the
children underneath almost like a normal reclaim, where the unused
pages would be reclaimed/ Having said that I am open to your
suggestion, my concern is that semantics can get a bit confusing as to
when the administrator can setup soft limits. We can come up with
guidelines and recommend your suggestion.

>  5. I'm glad if you extract patch 4/5 as an independent clean up patch.
>

Thanks,

>  6. no overheads ?
>

I ran some tests and saw no additional overheads, I'll test some more
and post results. There are some cleanups pending like the ones you
pointed, where we can use page_to_* instead of pc_* routines. I did
not clean them up as I wanted to get out the RFC soon with working
functionality and post v9 with those cleaned up.

Thanks for the review.
Balbir

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>