Re: [PATCH V5 06/10] Per-memcg background reclaim.

[Minchan Kim <minchan.kim@gmail.com>]

On Tue, Apr 19, 2011 at 11:42 AM, Ying Han <yinghan@google.com> wrote:
>
>
> On Mon, Apr 18, 2011 at 4:32 PM, Minchan Kim <minchan.kim@gmail.com> wrote:
>>
>> On Tue, Apr 19, 2011 at 6:38 AM, Ying Han <yinghan@google.com> wrote:
>> >
>> >
>> > On Sun, Apr 17, 2011 at 8:51 PM, Minchan Kim <minchan.kim@gmail.com>
>> > wrote:
>> >>
>> >> On Sat, Apr 16, 2011 at 8:23 AM, Ying Han <yinghan@google.com> wrote:
>> >> > +
>> >> > +               sc->nr_scanned = 0;
>> >> > +               shrink_zone(priority, zone, sc);
>> >> > +               total_scanned += sc->nr_scanned;
>> >> > +
>> >> > +               /*
>> >> > +                * If we've done a decent amount of scanning and
>> >> > +                * the reclaim ratio is low, start doing writepage
>> >> > +                * even in laptop mode
>> >> > +                */
>> >> > +               if (total_scanned > SWAP_CLUSTER_MAX * 2 &&
>> >> > +                   total_scanned > sc->nr_reclaimed +
>> >> > sc->nr_reclaimed
>> >> > / 2) {
>> >> > +                       sc->may_writepage = 1;
>> >>
>> >> I don't want to add more random write any more although we don't have
>> >> a trouble of real memory shortage.
>> >
>> >
>> >>
>> >> Do you have any reason to reclaim memory urgently as writing dirty
>> >> pages?
>> >> Maybe if we wait a little bit of time, flusher would write out the
>> >> page.
>> >
>> > We would like to reduce the writing dirty pages from page reclaim,
>> > especially from direct reclaim. AFAIK,
>> > the try_to_free_mem_cgroup_pages()
>> > still need to write dirty pages when there is a need. removing this from
>> > the
>> > per-memcg kswap will only add more pressure to the per-memcg direct
>> > reclaim,
>> > which seems to be worse. (stack overflow as one example which we would
>> > like
>> > to get rid of)
>> >
>>
>> Stack overflow would be another topic.
>>
>> Normal situation :
>>
>> The softlimit memory pressure of memcg isn't real memory shortage and
>> if we have gap between hardlimit and softlimit, periodic writeback of
>> flusher would write it out before reaching the hardlimit. In the end,
>> direct reclaim don't need to write it out.
>>
>> Exceptional situation :
>>
>> Of course, it doesn't work well in congestion of bdi, sudden big
>> memory consumption in memcg in wrong [hard/soft]limit(small gap)
>> configuration of administrator.
>>
>> I think we have to design it by normal situation.
>> The point is that softlimit isn't real memory shortage so that we are
>> not urgent.
>
> This patch is not dealing with soft_limit, but hard_limit. The soft_limit
> reclaim which we talked about during LSF
> is something i am currently looking at right now. This patch is doing the
> per-memcg background reclaim which
> based on the watermarks calculated on the hard_limit. We don't have the
> memcg entering the direct reclaim each
> time it is reaching the hard_limit, so we add the background reclaim which
> reclaiming pages proactively.
>>
>> How about adding new function which checks global memory pressure and
>> if we have a trouble by global memory pressure, we can change
>> may_write with 1 dynamically in memcg_kswapd?
>
>
> Like I mentioned, the may_write is still needed in this case otherwise we
> are just put this further to per-memcg
> direct reclaim.

Totally, you're right. I misunderstood some point.
Thanks.

-- 
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>