stable.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org, Shaohua Li <shli@fb.com>,
	Tejun Heo <tj@kernel.org>
Subject: [PATCH 4.1 31/46] workqueue: make sure delayed work run in local cpu
Date: Fri, 23 Oct 2015 10:46:17 -0700	[thread overview]
Message-ID: <20151023174621.720803981@linuxfoundation.org> (raw)
In-Reply-To: <20151023174620.779720995@linuxfoundation.org>

4.1-stable review patch.  If anyone has any objections, please let me know.

------------------

From: Shaohua Li <shli@fb.com>

commit 874bbfe600a660cba9c776b3957b1ce393151b76 upstream.

My system keeps crashing with below message. vmstat_update() schedules a delayed
work in current cpu and expects the work runs in the cpu.
schedule_delayed_work() is expected to make delayed work run in local cpu. The
problem is timer can be migrated with NO_HZ. __queue_work() queues work in
timer handler, which could run in a different cpu other than where the delayed
work is scheduled. The end result is the delayed work runs in different cpu.
The patch makes __queue_delayed_work records local cpu earlier. Where the timer
runs doesn't change where the work runs with the change.

[   28.010131] ------------[ cut here ]------------
[   28.010609] kernel BUG at ../mm/vmstat.c:1392!
[   28.011099] invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC KASAN
[   28.011860] Modules linked in:
[   28.012245] CPU: 0 PID: 289 Comm: kworker/0:3 Tainted: G        W4.3.0-rc3+ #634
[   28.013065] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140709_153802- 04/01/2014
[   28.014160] Workqueue: events vmstat_update
[   28.014571] task: ffff880117682580 ti: ffff8800ba428000 task.ti: ffff8800ba428000
[   28.015445] RIP: 0010:[<ffffffff8115f921>]  [<ffffffff8115f921>]vmstat_update+0x31/0x80
[   28.016282] RSP: 0018:ffff8800ba42fd80  EFLAGS: 00010297
[   28.016812] RAX: 0000000000000000 RBX: ffff88011a858dc0 RCX:0000000000000000
[   28.017585] RDX: ffff880117682580 RSI: ffffffff81f14d8c RDI:ffffffff81f4df8d
[   28.018366] RBP: ffff8800ba42fd90 R08: 0000000000000001 R09:0000000000000000
[   28.019169] R10: 0000000000000000 R11: 0000000000000121 R12:ffff8800baa9f640
[   28.019947] R13: ffff88011a81e340 R14: ffff88011a823700 R15:0000000000000000
[   28.020071] FS:  0000000000000000(0000) GS:ffff88011a800000(0000)knlGS:0000000000000000
[   28.020071] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[   28.020071] CR2: 00007ff6144b01d0 CR3: 00000000b8e93000 CR4:00000000000006f0
[   28.020071] Stack:
[   28.020071]  ffff88011a858dc0 ffff8800baa9f640 ffff8800ba42fe00ffffffff8106bd88
[   28.020071]  ffffffff8106bd0b 0000000000000096 0000000000000000ffffffff82f9b1e8
[   28.020071]  ffffffff829f0b10 0000000000000000 ffffffff81f18460ffff88011a81e340
[   28.020071] Call Trace:
[   28.020071]  [<ffffffff8106bd88>] process_one_work+0x1c8/0x540
[   28.020071]  [<ffffffff8106bd0b>] ? process_one_work+0x14b/0x540
[   28.020071]  [<ffffffff8106c214>] worker_thread+0x114/0x460
[   28.020071]  [<ffffffff8106c100>] ? process_one_work+0x540/0x540
[   28.020071]  [<ffffffff81071bf8>] kthread+0xf8/0x110
[   28.020071]  [<ffffffff81071b00>] ?kthread_create_on_node+0x200/0x200
[   28.020071]  [<ffffffff81a6522f>] ret_from_fork+0x3f/0x70
[   28.020071]  [<ffffffff81071b00>] ?kthread_create_on_node+0x200/0x200

Signed-off-by: Shaohua Li <shli@fb.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

---
 kernel/workqueue.c |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -1451,13 +1451,13 @@ static void __queue_delayed_work(int cpu
 	timer_stats_timer_set_start_info(&dwork->timer);
 
 	dwork->wq = wq;
+	/* timer isn't guaranteed to run in this cpu, record earlier */
+	if (cpu == WORK_CPU_UNBOUND)
+		cpu = raw_smp_processor_id();
 	dwork->cpu = cpu;
 	timer->expires = jiffies + delay;
 
-	if (unlikely(cpu != WORK_CPU_UNBOUND))
-		add_timer_on(timer, cpu);
-	else
-		add_timer(timer);
+	add_timer_on(timer, cpu);
 }
 
 /**



  parent reply	other threads:[~2015-10-23 17:46 UTC|newest]

Thread overview: 54+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-10-23 17:45 [PATCH 4.1 00/46] 4.1.12-stable review Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 01/46] net/ibm/emac: bump version numbers for correct work with ethtool Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 02/46] l2tp: protect tunnel->del_work by ref_count Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 03/46] skbuff: Fix skb checksum flag on skb pull Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 04/46] skbuff: Fix skb checksum partial check Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 05/46] inet: fix races in reqsk_queue_hash_req() Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 06/46] net: add pfmemalloc check in sk_add_backlog() Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 07/46] ppp: dont override sk->sk_state in pppoe_flush_dev() Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 08/46] inet: fix race in reqsk_queue_unlink() Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 09/46] bpf: fix panic in SO_GET_FILTER with native ebpf programs Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 10/46] ovs: do not allocate memory from offline numa node Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 11/46] act_mirred: clear sender cpu before sending to tx Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 12/46] ethtool: Use kcalloc instead of kmalloc for ethtool_get_strings Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 13/46] tipc: move fragment importance field to new header position Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 14/46] netlink: Trim skb to alloc size to avoid MSG_TRUNC Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 15/46] af_unix: Convert the unix_sk macro to an inline function for type safety Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 16/46] af_unix: return data from multiple SKBs on recv() with MSG_PEEK flag Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 17/46] net/unix: fix logic about sk_peek_offset Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 18/46] drm: Fix locking for sysfs dpms file Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 19/46] crypto: sparc - initialize blkcipher.ivsize Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 20/46] crypto: ahash - ensure statesize is non-zero Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 21/46] memcg: convert threshold to bytes Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 22/46] btrfs: check unsupported filters in balance arguments Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 23/46] btrfs: fix use after free iterating extrefs Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 24/46] arm64: errata: use KBUILD_CFLAGS_MODULE for erratum #843419 Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 25/46] nfsd/blocklayout: accept any minlength Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 26/46] mfd: max77843: Fix max77843_chg_init() return on error Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 27/46] i2c: rcar: enable RuntimePM before registering to the core Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 28/46] i2c: s3c2410: " Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 29/46] i2c: designware: Do not use parameters from ACPI on Dell Inspiron 7348 Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 30/46] i2c: designware-platdrv: enable RuntimePM before registering to the core Greg Kroah-Hartman
2015-10-23 17:46 ` Greg Kroah-Hartman [this message]
2015-10-23 17:46 ` [PATCH 4.1 32/46] drm/nouveau/fbcon: take runpm reference when userspace has an open fd Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 33/46] drm/dp/mst: make mst i2c transfer code more robust Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 34/46] drm/radeon: attach tile property to mst connector Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 35/46] drm/radeon: add pm sysfs files late Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 36/46] dm thin: fix missing pool reference count decrement in pool_ctr error path Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 37/46] rbd: fix double free on rbd_dev->header_name Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 38/46] sched/preempt: Rename PREEMPT_CHECK_OFFSET to PREEMPT_DISABLE_OFFSET Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 39/46] sched/preempt: Fix cond_resched_lock() and cond_resched_softirq() Greg Kroah-Hartman
2015-10-23 20:14   ` Thomas Backlund
2015-10-23 23:21     ` Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 41/46] arm64: Fix THP protection change logic Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 42/46] svcrdma: handle rdma read with a non-zero initial page offset Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 43/46] locks: have flock_lock_file take an inode pointer instead of a filp Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 44/46] locks: new helpers - flock_lock_inode_wait and posix_lock_inode_wait Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 45/46] locks: inline posix_lock_file_wait and flock_lock_file_wait Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 46/46] nfs4: have do_vfs_lock take an inode pointer Greg Kroah-Hartman
2015-10-23 20:34 ` [PATCH 4.1 00/46] 4.1.12-stable review Shuah Khan
2015-10-23 23:22   ` Greg Kroah-Hartman
2015-10-24  1:11     ` Guenter Roeck
2015-10-24  3:15 ` Guenter Roeck
2015-10-24 13:20   ` Greg Kroah-Hartman
     [not found] ` <562b9ffc.e8acc20a.c45a5.08fd@mx.google.com>
2015-10-24 15:16   ` Kevin Hilman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20151023174621.720803981@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=shli@fb.com \
    --cc=stable@vger.kernel.org \
    --cc=tj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).