All of lore.kernel.org
 help / color / mirror / Atom feed
From: Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
To: Alex Shi <alex.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>
Cc: akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org,
	mgorman-3eNAlZScCAx27rWaFMvyedHuzzzSOjJt@public.gmane.org,
	tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org,
	hughd-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org,
	khlebnikov-XoJtRXgx1JseBXzfvpsJ4g@public.gmane.org,
	daniel.m.jordan-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org,
	willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org,
	lkp-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org,
	linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org,
	iamjoonsoo.kim-Hm3cg6mZ9cc@public.gmane.org,
	richard.weiyang-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org,
	kirill-oKw7cIdHH8eLwutG50LtGA@public.gmane.org,
	alexander.duyck-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org,
	rong.a.chen-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org,
	mhocko-IBi9RG/b67k@public.gmane.org,
	vdavydov.dev-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org,
	shy828301-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Subject: Re: [PATCH v20 16/20] mm/compaction: do page isolation first in compaction
Date: Mon, 2 Nov 2020 10:18:07 -0500	[thread overview]
Message-ID: <20201102151807.GI724984@cmpxchg.org> (raw)
In-Reply-To: <1603968305-8026-17-git-send-email-alex.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>

On Thu, Oct 29, 2020 at 06:45:01PM +0800, Alex Shi wrote:
> Currently, compaction would get the lru_lock and then do page isolation
> which works fine with pgdat->lru_lock, since any page isoltion would
> compete for the lru_lock. If we want to change to memcg lru_lock, we
> have to isolate the page before getting lru_lock, thus isoltion would
> block page's memcg change which relay on page isoltion too. Then we
> could safely use per memcg lru_lock later.
> 
> The new page isolation use previous introduced TestClearPageLRU() +
> pgdat lru locking which will be changed to memcg lru lock later.
> 
> Hugh Dickins <hughd-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> fixed following bugs in this patch's
> early version:
> 
> Fix lots of crashes under compaction load: isolate_migratepages_block()
> must clean up appropriately when rejecting a page, setting PageLRU again
> if it had been cleared; and a put_page() after get_page_unless_zero()
> cannot safely be done while holding locked_lruvec - it may turn out to
> be the final put_page(), which will take an lruvec lock when PageLRU.
> And move __isolate_lru_page_prepare back after get_page_unless_zero to
> make trylock_page() safe:
> trylock_page() is not safe to use at this time: its setting PG_locked
> can race with the page being freed or allocated ("Bad page"), and can
> also erase flags being set by one of those "sole owners" of a freshly
> allocated page who use non-atomic __SetPageFlag().
> 
> Suggested-by: Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
> Signed-off-by: Alex Shi <alex.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>
> Acked-by: Hugh Dickins <hughd-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
> Cc: Andrew Morton <akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>
> Cc: Matthew Wilcox <willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
> Cc: linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> Cc: linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org

Acked-by: Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>

WARNING: multiple messages have this Message-ID (diff)
From: Johannes Weiner <hannes@cmpxchg.org>
To: Alex Shi <alex.shi@linux.alibaba.com>
Cc: akpm@linux-foundation.org, mgorman@techsingularity.net,
	tj@kernel.org, hughd@google.com, khlebnikov@yandex-team.ru,
	daniel.m.jordan@oracle.com, willy@infradead.org, lkp@intel.com,
	linux-mm@kvack.org, linux-kernel@vger.kernel.org,
	cgroups@vger.kernel.org, shakeelb@google.com,
	iamjoonsoo.kim@lge.com, richard.weiyang@gmail.com,
	kirill@shutemov.name, alexander.duyck@gmail.com,
	rong.a.chen@intel.com, mhocko@suse.com, vdavydov.dev@gmail.com,
	shy828301@gmail.com
Subject: Re: [PATCH v20 16/20] mm/compaction: do page isolation first in compaction
Date: Mon, 2 Nov 2020 10:18:07 -0500	[thread overview]
Message-ID: <20201102151807.GI724984@cmpxchg.org> (raw)
In-Reply-To: <1603968305-8026-17-git-send-email-alex.shi@linux.alibaba.com>

On Thu, Oct 29, 2020 at 06:45:01PM +0800, Alex Shi wrote:
> Currently, compaction would get the lru_lock and then do page isolation
> which works fine with pgdat->lru_lock, since any page isoltion would
> compete for the lru_lock. If we want to change to memcg lru_lock, we
> have to isolate the page before getting lru_lock, thus isoltion would
> block page's memcg change which relay on page isoltion too. Then we
> could safely use per memcg lru_lock later.
> 
> The new page isolation use previous introduced TestClearPageLRU() +
> pgdat lru locking which will be changed to memcg lru lock later.
> 
> Hugh Dickins <hughd@google.com> fixed following bugs in this patch's
> early version:
> 
> Fix lots of crashes under compaction load: isolate_migratepages_block()
> must clean up appropriately when rejecting a page, setting PageLRU again
> if it had been cleared; and a put_page() after get_page_unless_zero()
> cannot safely be done while holding locked_lruvec - it may turn out to
> be the final put_page(), which will take an lruvec lock when PageLRU.
> And move __isolate_lru_page_prepare back after get_page_unless_zero to
> make trylock_page() safe:
> trylock_page() is not safe to use at this time: its setting PG_locked
> can race with the page being freed or allocated ("Bad page"), and can
> also erase flags being set by one of those "sole owners" of a freshly
> allocated page who use non-atomic __SetPageFlag().
> 
> Suggested-by: Johannes Weiner <hannes@cmpxchg.org>
> Signed-off-by: Alex Shi <alex.shi@linux.alibaba.com>
> Acked-by: Hugh Dickins <hughd@google.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Matthew Wilcox <willy@infradead.org>
> Cc: linux-kernel@vger.kernel.org
> Cc: linux-mm@kvack.org

Acked-by: Johannes Weiner <hannes@cmpxchg.org>


  parent reply	other threads:[~2020-11-02 15:18 UTC|newest]

Thread overview: 109+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-29 10:44 [PATCH v20 00/20] per memcg lru lock Alex Shi
2020-10-29 10:44 ` Alex Shi
2020-10-29 10:44 ` [PATCH v20 04/20] mm/thp: use head for head page in lru_add_page_tail Alex Shi
     [not found]   ` <1603968305-8026-5-git-send-email-alex.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>
2020-10-29 13:50     ` Johannes Weiner
2020-10-29 13:50       ` Johannes Weiner
     [not found]       ` <20201029135047.GE599825-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2020-10-30  2:46         ` Alex Shi
2020-10-30  2:46           ` Alex Shi
     [not found]           ` <06a5b7d8-bbf2-51b7-1352-2b630186e15f-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>
2020-10-30 13:52             ` Johannes Weiner
2020-10-30 13:52               ` Johannes Weiner
2020-10-31  1:14               ` Alex Shi
2020-11-02 16:03           ` Matthew Wilcox
     [not found]             ` <20201102160357.GP27442-FZi0V3Vbi30CUdFEqe4BF2D2FQJk+8+b@public.gmane.org>
2020-11-03  2:43               ` Alex Shi
2020-11-03  2:43                 ` Alex Shi
2020-10-29 10:44 ` [PATCH v20 05/20] mm/thp: Simplify lru_add_page_tail() Alex Shi
2020-10-29 10:44   ` Alex Shi
2020-10-29 14:00   ` Johannes Weiner
2020-10-30  2:48   ` Alex Shi
2020-10-30  2:48     ` Alex Shi
2020-10-29 10:44 ` [PATCH v20 06/20] mm/thp: narrow lru locking Alex Shi
2020-10-29 10:44 ` [PATCH v20 08/20] mm: page_idle_get_page() does not need lru_lock Alex Shi
2020-11-02 14:41   ` Johannes Weiner
     [not found]     ` <20201102144110.GB724984-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2020-11-02 14:49       ` Matthew Wilcox
2020-11-02 14:49         ` Matthew Wilcox
2020-11-02 20:20         ` Johannes Weiner
     [not found]           ` <20201102202003.GA740958-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2020-11-04 11:27             ` Alex Shi
2020-11-04 11:27               ` Alex Shi
     [not found]               ` <b4038b87-cf5a-fcb7-06f4-b98874029615-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>
2020-11-04 17:46                 ` Johannes Weiner
2020-11-04 17:46                   ` Johannes Weiner
2020-11-05  4:52                   ` Alex Shi
     [not found]                     ` <6eea82d8-e406-06ee-2333-eb6e2f1944e5-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>
2020-11-05  4:57                       ` Matthew Wilcox
2020-11-05  4:57                         ` Matthew Wilcox
2020-11-05  5:03                         ` Alex Shi
2020-11-05  5:03                           ` Alex Shi
     [not found]                           ` <1e8f0162-cf2e-03eb-e7e0-ccc9f6a3eaf2-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>
2020-11-05 15:36                             ` Johannes Weiner
2020-11-05 15:36                               ` Johannes Weiner
     [not found]                               ` <20201105153649.GC744831-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2020-11-05 15:43                                 ` Matthew Wilcox
2020-11-05 15:43                                   ` Matthew Wilcox
2020-11-06  1:11                                 ` Alex Shi
2020-11-06  1:11                                   ` Alex Shi
2020-11-11  7:27     ` Hugh Dickins
2020-10-29 10:44 ` [PATCH v20 09/20] mm/memcg: add debug checking in lock_page_memcg Alex Shi
     [not found]   ` <1603968305-8026-10-git-send-email-alex.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>
2020-11-02 14:45     ` Johannes Weiner
2020-11-02 14:45       ` Johannes Weiner
2020-10-29 10:44 ` [PATCH v20 11/20] mm/lru: move lock into lru_note_cost Alex Shi
     [not found]   ` <1603968305-8026-12-git-send-email-alex.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>
2020-10-29 13:42     ` Johannes Weiner
2020-10-29 13:42       ` Johannes Weiner
2020-10-29 10:44 ` [PATCH v20 13/20] mm/mlock: remove lru_lock on TestClearPageMlocked Alex Shi
     [not found]   ` <1603968305-8026-14-git-send-email-alex.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>
2020-11-02 14:55     ` Johannes Weiner
2020-11-02 14:55       ` Johannes Weiner
     [not found] ` <1603968305-8026-1-git-send-email-alex.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>
2020-10-29 10:44   ` [PATCH v20 01/20] mm/memcg: warning on !memcg after readahead page charged Alex Shi
2020-10-29 10:44     ` Alex Shi
2020-10-29 13:43     ` Johannes Weiner
2020-10-29 10:44   ` [PATCH v20 02/20] mm/memcg: bail early from swap accounting if memcg disabled Alex Shi
2020-10-29 10:44     ` Alex Shi
2020-10-29 13:46     ` Johannes Weiner
2020-10-30  2:27       ` Alex Shi
2020-10-30  2:27         ` Alex Shi
     [not found]         ` <96b6d122-df0e-dfb0-368c-6bd714fab116-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>
2020-10-30 14:04           ` Johannes Weiner
2020-10-30 14:04             ` Johannes Weiner
     [not found]             ` <20201030140420.GB666074-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2020-10-31  1:13               ` Alex Shi
2020-10-31  1:13                 ` Alex Shi
2020-10-29 10:44   ` [PATCH v20 03/20] mm/thp: move lru_add_page_tail func to huge_memory.c Alex Shi
2020-10-29 10:44     ` Alex Shi
2020-10-29 13:47     ` Johannes Weiner
2020-10-29 10:44   ` [PATCH v20 07/20] mm/vmscan: remove unnecessary lruvec adding Alex Shi
2020-10-29 10:44     ` Alex Shi
2020-11-02 14:20     ` Johannes Weiner
2020-10-29 10:44   ` [PATCH v20 10/20] mm/swap.c: fold vm event PGROTATED into pagevec_move_tail_fn Alex Shi
2020-10-29 10:44     ` Alex Shi
2020-11-02 14:48     ` Johannes Weiner
2020-10-29 10:44   ` [PATCH v20 12/20] mm/vmscan: remove lruvec reget in move_pages_to_lru Alex Shi
2020-10-29 10:44     ` Alex Shi
     [not found]     ` <1603968305-8026-13-git-send-email-alex.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>
2020-11-02 14:52       ` Johannes Weiner
2020-11-02 14:52         ` Johannes Weiner
     [not found]         ` <20201102145220.GE724984-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2020-11-03  2:51           ` Alex Shi
2020-11-03  2:51             ` Alex Shi
2020-10-29 10:44   ` [PATCH v20 14/20] mm/mlock: remove __munlock_isolate_lru_page Alex Shi
2020-10-29 10:44     ` Alex Shi
     [not found]     ` <1603968305-8026-15-git-send-email-alex.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>
2020-11-02 14:56       ` Johannes Weiner
2020-11-02 14:56         ` Johannes Weiner
2020-10-29 10:45   ` [PATCH v20 15/20] mm/lru: introduce TestClearPageLRU Alex Shi
2020-10-29 10:45     ` Alex Shi
2020-11-02 15:10     ` Johannes Weiner
2020-11-03  3:02       ` Alex Shi
2020-11-03  3:02         ` Alex Shi
2020-10-29 10:45   ` [PATCH v20 16/20] mm/compaction: do page isolation first in compaction Alex Shi
2020-10-29 10:45     ` Alex Shi
     [not found]     ` <1603968305-8026-17-git-send-email-alex.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>
2020-11-02 15:18       ` Johannes Weiner [this message]
2020-11-02 15:18         ` Johannes Weiner
2020-11-04 11:55   ` [PATCH v20 00/20] per memcg lru lock Alex Shi
2020-11-04 11:55     ` Alex Shi
     [not found]     ` <811216d4-4972-4721-d6b9-1028c02f4290-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>
2020-11-04 16:59       ` Johannes Weiner
2020-11-04 16:59         ` Johannes Weiner
     [not found]         ` <20201104165935.GA744831-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2020-11-05  5:07           ` Alex Shi
2020-11-05  5:07             ` Alex Shi
2020-10-29 10:45 ` [PATCH v20 17/20] mm/swap.c: serialize memcg changes in pagevec_lru_move_fn Alex Shi
     [not found]   ` <1603968305-8026-18-git-send-email-alex.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>
2020-11-02 15:20     ` Johannes Weiner
2020-11-02 15:20       ` Johannes Weiner
2020-10-29 10:45 ` [PATCH v20 18/20] mm/lru: replace pgdat lru_lock with lruvec lock Alex Shi
2020-10-30  2:49   ` Alex Shi
     [not found]     ` <ef279dfe-afa5-45cb-4013-6c34169ff55e-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>
2020-11-02 20:41       ` Johannes Weiner
2020-11-02 20:41         ` Johannes Weiner
2020-11-03  4:58         ` Alex Shi
2020-11-03  4:58           ` Alex Shi
2020-10-29 10:45 ` [PATCH v20 19/20] mm/lru: introduce the relock_page_lruvec function Alex Shi
     [not found]   ` <1603968305-8026-20-git-send-email-alex.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org>
2020-11-02 20:44     ` Johannes Weiner
2020-11-02 20:44       ` Johannes Weiner
2020-10-29 10:45 ` [PATCH v20 20/20] mm/lru: revise the comments of lru_lock Alex Shi
2020-11-02 20:46   ` Johannes Weiner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201102151807.GI724984@cmpxchg.org \
    --to=hannes-druugvl0lcnafugrpc6u6w@public.gmane.org \
    --cc=akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org \
    --cc=alex.shi-KPsoFbNs7GizrGE5bRqYAgC/G2K4zDHf@public.gmane.org \
    --cc=alexander.duyck-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
    --cc=cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=daniel.m.jordan-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org \
    --cc=hughd-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
    --cc=iamjoonsoo.kim-Hm3cg6mZ9cc@public.gmane.org \
    --cc=khlebnikov-XoJtRXgx1JseBXzfvpsJ4g@public.gmane.org \
    --cc=kirill-oKw7cIdHH8eLwutG50LtGA@public.gmane.org \
    --cc=linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org \
    --cc=lkp-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org \
    --cc=mgorman-3eNAlZScCAx27rWaFMvyedHuzzzSOjJt@public.gmane.org \
    --cc=mhocko-IBi9RG/b67k@public.gmane.org \
    --cc=richard.weiyang-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
    --cc=rong.a.chen-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org \
    --cc=shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
    --cc=shy828301-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
    --cc=tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org \
    --cc=vdavydov.dev-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
    --cc=willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.