All of lore.kernel.org
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org, Shaohua Li <shli@fb.com>,
	Tejun Heo <tj@kernel.org>
Subject: [PATCH 4.1 31/46] workqueue: make sure delayed work run in local cpu
Date: Fri, 23 Oct 2015 10:46:17 -0700	[thread overview]
Message-ID: <20151023174621.720803981@linuxfoundation.org> (raw)
In-Reply-To: <20151023174620.779720995@linuxfoundation.org>

4.1-stable review patch.  If anyone has any objections, please let me know.

------------------

From: Shaohua Li <shli@fb.com>

commit 874bbfe600a660cba9c776b3957b1ce393151b76 upstream.

My system keeps crashing with below message. vmstat_update() schedules a delayed
work in current cpu and expects the work runs in the cpu.
schedule_delayed_work() is expected to make delayed work run in local cpu. The
problem is timer can be migrated with NO_HZ. __queue_work() queues work in
timer handler, which could run in a different cpu other than where the delayed
work is scheduled. The end result is the delayed work runs in different cpu.
The patch makes __queue_delayed_work records local cpu earlier. Where the timer
runs doesn't change where the work runs with the change.

[   28.010131] ------------[ cut here ]------------
[   28.010609] kernel BUG at ../mm/vmstat.c:1392!
[   28.011099] invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC KASAN
[   28.011860] Modules linked in:
[   28.012245] CPU: 0 PID: 289 Comm: kworker/0:3 Tainted: G        W4.3.0-rc3+ #634
[   28.013065] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140709_153802- 04/01/2014
[   28.014160] Workqueue: events vmstat_update
[   28.014571] task: ffff880117682580 ti: ffff8800ba428000 task.ti: ffff8800ba428000
[   28.015445] RIP: 0010:[<ffffffff8115f921>]  [<ffffffff8115f921>]vmstat_update+0x31/0x80
[   28.016282] RSP: 0018:ffff8800ba42fd80  EFLAGS: 00010297
[   28.016812] RAX: 0000000000000000 RBX: ffff88011a858dc0 RCX:0000000000000000
[   28.017585] RDX: ffff880117682580 RSI: ffffffff81f14d8c RDI:ffffffff81f4df8d
[   28.018366] RBP: ffff8800ba42fd90 R08: 0000000000000001 R09:0000000000000000
[   28.019169] R10: 0000000000000000 R11: 0000000000000121 R12:ffff8800baa9f640
[   28.019947] R13: ffff88011a81e340 R14: ffff88011a823700 R15:0000000000000000
[   28.020071] FS:  0000000000000000(0000) GS:ffff88011a800000(0000)knlGS:0000000000000000
[   28.020071] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[   28.020071] CR2: 00007ff6144b01d0 CR3: 00000000b8e93000 CR4:00000000000006f0
[   28.020071] Stack:
[   28.020071]  ffff88011a858dc0 ffff8800baa9f640 ffff8800ba42fe00ffffffff8106bd88
[   28.020071]  ffffffff8106bd0b 0000000000000096 0000000000000000ffffffff82f9b1e8
[   28.020071]  ffffffff829f0b10 0000000000000000 ffffffff81f18460ffff88011a81e340
[   28.020071] Call Trace:
[   28.020071]  [<ffffffff8106bd88>] process_one_work+0x1c8/0x540
[   28.020071]  [<ffffffff8106bd0b>] ? process_one_work+0x14b/0x540
[   28.020071]  [<ffffffff8106c214>] worker_thread+0x114/0x460
[   28.020071]  [<ffffffff8106c100>] ? process_one_work+0x540/0x540
[   28.020071]  [<ffffffff81071bf8>] kthread+0xf8/0x110
[   28.020071]  [<ffffffff81071b00>] ?kthread_create_on_node+0x200/0x200
[   28.020071]  [<ffffffff81a6522f>] ret_from_fork+0x3f/0x70
[   28.020071]  [<ffffffff81071b00>] ?kthread_create_on_node+0x200/0x200

Signed-off-by: Shaohua Li <shli@fb.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

---
 kernel/workqueue.c |    8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -1451,13 +1451,13 @@ static void __queue_delayed_work(int cpu
 	timer_stats_timer_set_start_info(&dwork->timer);
 
 	dwork->wq = wq;
+	/* timer isn't guaranteed to run in this cpu, record earlier */
+	if (cpu == WORK_CPU_UNBOUND)
+		cpu = raw_smp_processor_id();
 	dwork->cpu = cpu;
 	timer->expires = jiffies + delay;
 
-	if (unlikely(cpu != WORK_CPU_UNBOUND))
-		add_timer_on(timer, cpu);
-	else
-		add_timer(timer);
+	add_timer_on(timer, cpu);
 }
 
 /**



  parent reply	other threads:[~2015-10-23 17:55 UTC|newest]

Thread overview: 54+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-10-23 17:45 [PATCH 4.1 00/46] 4.1.12-stable review Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 01/46] net/ibm/emac: bump version numbers for correct work with ethtool Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 02/46] l2tp: protect tunnel->del_work by ref_count Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 03/46] skbuff: Fix skb checksum flag on skb pull Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 04/46] skbuff: Fix skb checksum partial check Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 05/46] inet: fix races in reqsk_queue_hash_req() Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 06/46] net: add pfmemalloc check in sk_add_backlog() Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 07/46] ppp: dont override sk->sk_state in pppoe_flush_dev() Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 08/46] inet: fix race in reqsk_queue_unlink() Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 09/46] bpf: fix panic in SO_GET_FILTER with native ebpf programs Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 10/46] ovs: do not allocate memory from offline numa node Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 11/46] act_mirred: clear sender cpu before sending to tx Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 12/46] ethtool: Use kcalloc instead of kmalloc for ethtool_get_strings Greg Kroah-Hartman
2015-10-23 17:45 ` [PATCH 4.1 13/46] tipc: move fragment importance field to new header position Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 14/46] netlink: Trim skb to alloc size to avoid MSG_TRUNC Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 15/46] af_unix: Convert the unix_sk macro to an inline function for type safety Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 16/46] af_unix: return data from multiple SKBs on recv() with MSG_PEEK flag Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 17/46] net/unix: fix logic about sk_peek_offset Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 18/46] drm: Fix locking for sysfs dpms file Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 19/46] crypto: sparc - initialize blkcipher.ivsize Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 20/46] crypto: ahash - ensure statesize is non-zero Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 21/46] memcg: convert threshold to bytes Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 22/46] btrfs: check unsupported filters in balance arguments Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 23/46] btrfs: fix use after free iterating extrefs Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 24/46] arm64: errata: use KBUILD_CFLAGS_MODULE for erratum #843419 Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 25/46] nfsd/blocklayout: accept any minlength Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 26/46] mfd: max77843: Fix max77843_chg_init() return on error Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 27/46] i2c: rcar: enable RuntimePM before registering to the core Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 28/46] i2c: s3c2410: " Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 29/46] i2c: designware: Do not use parameters from ACPI on Dell Inspiron 7348 Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 30/46] i2c: designware-platdrv: enable RuntimePM before registering to the core Greg Kroah-Hartman
2015-10-23 17:46 ` Greg Kroah-Hartman [this message]
2015-10-23 17:46 ` [PATCH 4.1 32/46] drm/nouveau/fbcon: take runpm reference when userspace has an open fd Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 33/46] drm/dp/mst: make mst i2c transfer code more robust Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 34/46] drm/radeon: attach tile property to mst connector Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 35/46] drm/radeon: add pm sysfs files late Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 36/46] dm thin: fix missing pool reference count decrement in pool_ctr error path Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 37/46] rbd: fix double free on rbd_dev->header_name Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 38/46] sched/preempt: Rename PREEMPT_CHECK_OFFSET to PREEMPT_DISABLE_OFFSET Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 39/46] sched/preempt: Fix cond_resched_lock() and cond_resched_softirq() Greg Kroah-Hartman
2015-10-23 20:14   ` Thomas Backlund
2015-10-23 23:21     ` Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 41/46] arm64: Fix THP protection change logic Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 42/46] svcrdma: handle rdma read with a non-zero initial page offset Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 43/46] locks: have flock_lock_file take an inode pointer instead of a filp Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 44/46] locks: new helpers - flock_lock_inode_wait and posix_lock_inode_wait Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 45/46] locks: inline posix_lock_file_wait and flock_lock_file_wait Greg Kroah-Hartman
2015-10-23 17:46 ` [PATCH 4.1 46/46] nfs4: have do_vfs_lock take an inode pointer Greg Kroah-Hartman
2015-10-23 20:34 ` [PATCH 4.1 00/46] 4.1.12-stable review Shuah Khan
2015-10-23 23:22   ` Greg Kroah-Hartman
2015-10-24  1:11     ` Guenter Roeck
2015-10-24  3:15 ` Guenter Roeck
2015-10-24 13:20   ` Greg Kroah-Hartman
     [not found] ` <562b9ffc.e8acc20a.c45a5.08fd@mx.google.com>
2015-10-24 15:16   ` Kevin Hilman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20151023174621.720803981@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=shli@fb.com \
    --cc=stable@vger.kernel.org \
    --cc=tj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.