From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752725AbcEPHRQ (ORCPT ); Mon, 16 May 2016 03:17:16 -0400 Received: from mx2.suse.de ([195.135.220.15]:45823 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750787AbcEPHRP (ORCPT ); Mon, 16 May 2016 03:17:15 -0400 Subject: Re: [RFC 11/13] mm, compaction: add the ultimate direct compaction priority To: Michal Hocko References: <1462865763-22084-1-git-send-email-vbabka@suse.cz> <1462865763-22084-12-git-send-email-vbabka@suse.cz> <20160513133851.GP20141@dhcp22.suse.cz> Cc: linux-mm@kvack.org, Andrew Morton , Joonsoo Kim , Rik van Riel , David Rientjes , Mel Gorman , Johannes Weiner , Tetsuo Handa , linux-kernel@vger.kernel.org, Linus Torvalds From: Vlastimil Babka Message-ID: <573973F7.7070202@suse.cz> Date: Mon, 16 May 2016 09:17:11 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: <20160513133851.GP20141@dhcp22.suse.cz> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 05/13/2016 03:38 PM, Michal Hocko wrote: > On Tue 10-05-16 09:36:01, Vlastimil Babka wrote: >> During reclaim/compaction loop, it's desirable to get a final answer from >> unsuccessful compaction so we can either fail the allocation or invoke the OOM >> killer. However, heuristics such as deferred compaction or pageblock skip bits >> can cause compaction to skip parts or whole zones and lead to premature OOM's, >> failures or excessive reclaim/compaction retries. >> >> To remedy this, we introduce a new direct compaction priority called >> COMPACT_PRIO_SYNC_FULL, which instructs direct compaction to: >> >> - ignore deferred compaction status for a zone >> - ignore pageblock skip hints >> - ignore cached scanner positions and scan the whole zone >> - use MIGRATE_SYNC migration mode > > I do not think we can do MIGRATE_SYNC because fallback_migrate_page > would trigger pageout and we are in the allocation path and so we > could blow up the stack. Ah, I thought it was just waiting for the writeout to complete, and you wanted to introduce another migrate mode to actually do the writeout. But looks like I misremembered. >> The new priority should get eventually picked up by should_compact_retry() and >> this should improve success rates for costly allocations using __GFP_RETRY, > > s@__GFP_RETRY@__GFP_REPEAT@ Ah thanks. Depending on the patch timing it might be __GFP_RETRY_HARD in the end, right :) >> such as hugetlbfs allocations, and reduce some corner-case OOM's for non-costly >> allocations. > > My testing has shown that even with the current implementation with > deferring, skip hints and cached positions had (close to) 100% success > rate even with close to OOM conditions. Hmm, I thought you at one point said that ignoring skip hints was a large improvement, because the current resetting of them is just too fuzzy. > I am wondering whether this strongest priority should be done only for > !costly high order pages. But we probably want less special cases > between costly and !costly orders. Yeah, if somebody wants to retry hard, let him. >> Signed-off-by: Vlastimil Babka > > Acked-by: Michal Hocko > >> --- >> include/linux/compaction.h | 1 + >> mm/compaction.c | 15 ++++++++++++--- >> 2 files changed, 13 insertions(+), 3 deletions(-) >> > [...] >> @@ -1631,7 +1639,8 @@ enum compact_result try_to_compact_pages(gfp_t gfp_mask, unsigned int order, >> ac->nodemask) { >> enum compact_result status; >> >> - if (compaction_deferred(zone, order)) { >> + if (prio > COMPACT_PRIO_SYNC_FULL >> + && compaction_deferred(zone, order)) { >> rc = max_t(enum compact_result, COMPACT_DEFERRED, rc); >> continue; >> } > > Wouldn't it be better to pull the prio check into compaction_deferred > directly? There are more callers and I am not really sure all of them > would behave consistently. I'll check, thanks.