All of lore.kernel.org
 help / color / mirror / Atom feed
From: David Hildenbrand <david@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: linux-mm@kvack.org, David Hildenbrand <david@redhat.com>,
	Yiqian Wei <yiwei@redhat.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Kirill Tkhai <ktkhai@virtuozzo.com>,
	Shile Zhang <shile.zhang@linux.alibaba.com>,
	Pavel Tatashin <pasha.tatashin@soleen.com>,
	Daniel Jordan <daniel.m.jordan@oracle.com>,
	Michal Hocko <mhocko@kernel.org>,
	Alexander Duyck <alexander.duyck@gmail.com>,
	Baoquan He <bhe@redhat.com>, Oscar Salvador <osalvador@suse.de>
Subject: [PATCH v1 1/2] mm/page_alloc: fix RCU stalls during deferred page initialization
Date: Wed,  1 Apr 2020 12:41:55 +0200	[thread overview]
Message-ID: <20200401104156.11564-2-david@redhat.com> (raw)
In-Reply-To: <20200401104156.11564-1-david@redhat.com>

With CONFIG_DEFERRED_STRUCT_PAGE_INIT and without CONFIG_PREEMPT, it can
happen that we get RCU stalls detected when booting up.

[   60.474005] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[   60.475000] rcu:  1-...0: (0 ticks this GP) idle=02a/1/0x4000000000000000 softirq=1/1 fqs=15000
[   60.475000] rcu:  (detected by 0, t=60002 jiffies, g=-1199, q=1)
[   60.475000] Sending NMI from CPU 0 to CPUs 1:
[    1.760091] NMI backtrace for cpu 1
[    1.760091] CPU: 1 PID: 20 Comm: pgdatinit0 Not tainted 4.18.0-147.9.1.el8_1.x86_64 #1
[    1.760091] Hardware name: Red Hat KVM, BIOS 1.13.0-1.module+el8.2.0+5520+4e5817f3 04/01/2014
[    1.760091] RIP: 0010:__init_single_page.isra.65+0x10/0x4f
[    1.760091] Code: 48 83 cf 63 48 89 f8 0f 1f 40 00 48 89 c6 48 89 d7 e8 6b 18 80 ff 66 90 5b c3 31 c0 b9 10 00 00 00 49 89 f8 48 c1 e6 33 f3 ab <b8> 07 00 00 00 48 c1 e2 36 41 c7 40 34 01 00 00 00 48 c1 e0 33 41
[    1.760091] RSP: 0000:ffffba783123be40 EFLAGS: 00000006
[    1.760091] RAX: 0000000000000000 RBX: fffffad34405e300 RCX: 0000000000000000
[    1.760091] RDX: 0000000000000000 RSI: 0010000000000000 RDI: fffffad34405e340
[    1.760091] RBP: 0000000033f3177e R08: fffffad34405e300 R09: 0000000000000002
[    1.760091] R10: 000000000000002b R11: ffff98afb691a500 R12: 0000000000000002
[    1.760091] R13: 0000000000000000 R14: 000000003f03ea00 R15: 000000003e10178c
[    1.760091] FS:  0000000000000000(0000) GS:ffff9c9ebeb00000(0000) knlGS:0000000000000000
[    1.760091] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    1.760091] CR2: 00000000ffffffff CR3: 000000a1cf20a001 CR4: 00000000003606e0
[    1.760091] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    1.760091] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[    1.760091] Call Trace:
[    1.760091]  deferred_init_pages+0x8f/0xbf
[    1.760091]  deferred_init_memmap+0x184/0x29d
[    1.760091]  ? deferred_free_pages.isra.97+0xba/0xba
[    1.760091]  kthread+0x112/0x130
[    1.760091]  ? kthread_flush_work_fn+0x10/0x10
[    1.760091]  ret_from_fork+0x35/0x40
[   89.123011] node 0 initialised, 1055935372 pages in 88650ms

The issue becomes visible when having a lot of memory (e.g., 4TB)
assigned to a single NUMA node - a system that can easily be created
using QEMU. Inside VMs on a hypervisor with quite some memory
overcommit, this is fairly easy to trigger.

Adding the cond_resched() makes RCU happy.

Reported-by: Yiqian Wei <yiwei@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Kirill Tkhai <ktkhai@virtuozzo.com>
Cc: Shile Zhang <shile.zhang@linux.alibaba.com>
Cc: Pavel Tatashin <pasha.tatashin@soleen.com>
Cc: Daniel Jordan <daniel.m.jordan@oracle.com>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Alexander Duyck <alexander.duyck@gmail.com>
Cc: Baoquan He <bhe@redhat.com>
Cc: Oscar Salvador <osalvador@suse.de>
Signed-off-by: David Hildenbrand <david@redhat.com>
---
 mm/page_alloc.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index ca1453204e66..084cabffc90d 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1877,6 +1877,7 @@ static int __init deferred_init_memmap(void *data)
 			prev_nr_pages = nr_pages;
 			pgdat->first_deferred_pfn = spfn;
 			pgdat_resize_unlock(pgdat, &flags);
+			cond_resched();
 			goto again;
 		}
 	}
-- 
2.25.1



  reply	other threads:[~2020-04-01 10:42 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-04-01 10:41 [PATCH v1 0/2] mm/page_alloc: fix stalls/soft lockups with huge VMs David Hildenbrand
2020-04-01 10:41 ` David Hildenbrand [this message]
2020-04-01 13:18   ` [PATCH v1 1/2] mm/page_alloc: fix RCU stalls during deferred page initialization Pavel Tatashin
2020-04-01 13:49   ` Baoquan He
2020-04-01 13:58   ` Shile Zhang
2020-04-01 14:09   ` Pankaj Gupta
2020-04-01 15:45   ` Michal Hocko
2020-04-01 15:47     ` David Hildenbrand
2020-04-01 10:41 ` [PATCH v1 2/2] mm/page_alloc: fix watchdog soft lockups during set_zone_contiguous() David Hildenbrand
2020-04-01 13:17   ` Pavel Tatashin
2020-04-01 13:45   ` Pankaj Gupta
2020-04-01 13:50   ` Baoquan He
2020-04-01 13:59   ` Shile Zhang
2020-04-01 15:45   ` Michal Hocko
2020-04-01 14:10 ` [PATCH v1 0/2] mm/page_alloc: fix stalls/soft lockups with huge VMs David Hildenbrand
2020-04-01 14:31   ` Pankaj Gupta
2020-04-01 14:45     ` Daniel Jordan
2020-04-01 15:54       ` David Hildenbrand
2020-04-01 16:10         ` Daniel Jordan
2020-04-01 18:06       ` Andrew Morton
2020-04-01 18:29         ` David Hildenbrand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200401104156.11564-2-david@redhat.com \
    --to=david@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=alexander.duyck@gmail.com \
    --cc=bhe@redhat.com \
    --cc=daniel.m.jordan@oracle.com \
    --cc=ktkhai@virtuozzo.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@kernel.org \
    --cc=osalvador@suse.de \
    --cc=pasha.tatashin@soleen.com \
    --cc=shile.zhang@linux.alibaba.com \
    --cc=yiwei@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.