[PATCH] mm/pagewalk: fix stale walk->action escaping walk_pmd_range()

[Hyunwoo Kim <imv4bel@gmail.com>]

walk_pmd_range() resets walk->action in only one place in its loop body,
and that place is after the pmd_none() branch. For a walker with no
->install_pte, that branch continues to the next entry without passing the
reset. So if ->pmd_entry() sets ACTION_AGAIN and returns 0, and the PMD has
become none by the time the loop restarts at the again label, the reset is
skipped. If the remaining entries are all none too, the loop returns 0 with
ACTION_AGAIN still set. The ACTION_AGAIN that walk_pte_range() sets when
pte_offset_map_lock() fails escapes the same way.

Nothing looked at that value after walk_pmd_range() returned until commit
3b89863c3fa4 ("mm/pagewalk: fix race between concurrent split and refault")
turned that into a problem. It added both the PUD check that sets
ACTION_AGAIN before the loop is entered and the test in walk_pud_range()
that picks the value up right after walk_pmd_range() returns and walks
[addr, pud_addr_end(addr, end)) again. That is fine for the PUD check,
since none of walk_pmd_range()'s own callbacks have run at that point, but
a value that escaped as described above arrives after those callbacks have
already covered the range.

For mincore(2) this becomes an out-of-bounds write. ->pmd_entry() and
->pte_hole() advance the walk->private cursor by one byte per page, the
buffer is a single page from __get_free_page(), and mincore(2) asks for at
most PAGE_SIZE entries at a time, so there is no room to spare. Walking
the range a second time pushes the cursor past the end of the buffer, and
it does so again every time the race is hit. Reproducing this needs no
privileges: run mincore(2) over a 16 MiB anonymous mapping marked
MADV_NOHUGEPAGE while another thread repeatedly faults in a PMD-aligned
2 MiB range inside it and then drops it with madvise(MADV_DONTNEED).

Move the reset to the first statement of the loop body. walk_pud_range()
has the same shape and gets the same change; walk_p4d_range() never looks
at walk->action, so that hunk keeps the two functions in sync rather than
fixing a second bug.

Fixes: 3b89863c3fa4 ("mm/pagewalk: fix race between concurrent split and refault")
Cc: stable@vger.kernel.org
Signed-off-by: Hyunwoo Kim <imv4bel@gmail.com>
---
 mm/pagewalk.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/mm/pagewalk.c b/mm/pagewalk.c
index 5d87c632a25507..d3bfece3193366 100644
--- a/mm/pagewalk.c
+++ b/mm/pagewalk.c
@@ -126,6 +126,7 @@ static int walk_pmd_range(pud_t *pud, unsigned long addr, unsigned long end,
 	pmd = pmd_offset(pud, addr);
 	do {
 again:
+		walk->action = ACTION_SUBTREE;
 		next = pmd_addr_end(addr, end);
 		if (pmd_none(*pmd)) {
 			if (has_install)
@@ -138,8 +139,6 @@ static int walk_pmd_range(pud_t *pud, unsigned long addr, unsigned long end,
 				continue;
 		}
 
-		walk->action = ACTION_SUBTREE;
-
 		/*
 		 * This implies that each ->pmd_entry() handler
 		 * needs to know about pmd_trans_huge() pmds
@@ -196,6 +195,7 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end,
 	pud = pud_offset(p4d, addr);
 	do {
  again:
+		walk->action = ACTION_SUBTREE;
 		next = pud_addr_end(addr, end);
 		if (pud_none(*pud)) {
 			if (has_install)
@@ -208,8 +208,6 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end,
 				continue;
 		}
 
-		walk->action = ACTION_SUBTREE;
-
 		if (ops->pud_entry)
 			err = ops->pud_entry(pud, addr, next, walk);
 		if (err)
-- 
2.43.0