public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
From: Sasha Levin <sashal@kernel.org>
To: stable@vger.kernel.org, linux-kernel@vger.kernel.org
Cc: Stephen Hemminger <stephen@networkplumber.org>,
	Stephen Hemminger <sthemmin@microsoft.com>,
	"David S . Miller" <davem@davemloft.net>,
	Sasha Levin <alexander.levin@microsoft.com>
Subject: [PATCH AUTOSEL 4.14 14/23] hv_netvsc: fix schedule in RCU context
Date: Fri,  5 Oct 2018 12:16:25 -0400	[thread overview]
Message-ID: <20181005161634.20631-14-sashal@kernel.org> (raw)
In-Reply-To: <20181005161634.20631-1-sashal@kernel.org>

From: Stephen Hemminger <stephen@networkplumber.org>

[ Upstream commit 018349d70f28a78d5343b3660cb66e1667005f8a ]

When netvsc device is removed it can call reschedule in RCU context.
This happens because canceling the subchannel setup work could (in theory)
cause a reschedule when manipulating the timer.

To reproduce, run with lockdep enabled kernel and unbind
a network device from hv_netvsc (via sysfs).

[  160.682011] WARNING: suspicious RCU usage
[  160.707466] 4.19.0-rc3-uio+ #2 Not tainted
[  160.709937] -----------------------------
[  160.712352] ./include/linux/rcupdate.h:302 Illegal context switch in RCU read-side critical section!
[  160.723691]
[  160.723691] other info that might help us debug this:
[  160.723691]
[  160.730955]
[  160.730955] rcu_scheduler_active = 2, debug_locks = 1
[  160.762813] 5 locks held by rebind-eth.sh/1812:
[  160.766851]  #0: 000000008befa37a (sb_writers#6){.+.+}, at: vfs_write+0x184/0x1b0
[  160.773416]  #1: 00000000b097f236 (&of->mutex){+.+.}, at: kernfs_fop_write+0xe2/0x1a0
[  160.783766]  #2: 0000000041ee6889 (kn->count#3){++++}, at: kernfs_fop_write+0xeb/0x1a0
[  160.787465]  #3: 0000000056d92a74 (&dev->mutex){....}, at: device_release_driver_internal+0x39/0x250
[  160.816987]  #4: 0000000030f6031e (rcu_read_lock){....}, at: netvsc_remove+0x1e/0x250 [hv_netvsc]
[  160.828629]
[  160.828629] stack backtrace:
[  160.831966] CPU: 1 PID: 1812 Comm: rebind-eth.sh Not tainted 4.19.0-rc3-uio+ #2
[  160.832952] Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS Hyper-V UEFI Release v1.0 11/26/2012
[  160.832952] Call Trace:
[  160.832952]  dump_stack+0x85/0xcb
[  160.832952]  ___might_sleep+0x1a3/0x240
[  160.832952]  __flush_work+0x57/0x2e0
[  160.832952]  ? __mutex_lock+0x83/0x990
[  160.832952]  ? __kernfs_remove+0x24f/0x2e0
[  160.832952]  ? __kernfs_remove+0x1b2/0x2e0
[  160.832952]  ? mark_held_locks+0x50/0x80
[  160.832952]  ? get_work_pool+0x90/0x90
[  160.832952]  __cancel_work_timer+0x13c/0x1e0
[  160.832952]  ? netvsc_remove+0x1e/0x250 [hv_netvsc]
[  160.832952]  ? __lock_is_held+0x55/0x90
[  160.832952]  netvsc_remove+0x9a/0x250 [hv_netvsc]
[  160.832952]  vmbus_remove+0x26/0x30
[  160.832952]  device_release_driver_internal+0x18a/0x250
[  160.832952]  unbind_store+0xb4/0x180
[  160.832952]  kernfs_fop_write+0x113/0x1a0
[  160.832952]  __vfs_write+0x36/0x1a0
[  160.832952]  ? rcu_read_lock_sched_held+0x6b/0x80
[  160.832952]  ? rcu_sync_lockdep_assert+0x2e/0x60
[  160.832952]  ? __sb_start_write+0x141/0x1a0
[  160.832952]  ? vfs_write+0x184/0x1b0
[  160.832952]  vfs_write+0xbe/0x1b0
[  160.832952]  ksys_write+0x55/0xc0
[  160.832952]  do_syscall_64+0x60/0x1b0
[  160.832952]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  160.832952] RIP: 0033:0x7fe48f4c8154

Resolve this by getting RTNL earlier. This is safe because the subchannel
work queue does trylock on RTNL and will detect the race.

Fixes: 7b2ee50c0cd5 ("hv_netvsc: common detach logic")
Signed-off-by: Stephen Hemminger <sthemmin@microsoft.com>
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Sasha Levin <alexander.levin@microsoft.com>
---
 drivers/net/hyperv/netvsc_drv.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c
index aba16d81e9bb..2d90cffae9ff 100644
--- a/drivers/net/hyperv/netvsc_drv.c
+++ b/drivers/net/hyperv/netvsc_drv.c
@@ -2110,17 +2110,15 @@ static int netvsc_remove(struct hv_device *dev)
 
 	cancel_delayed_work_sync(&ndev_ctx->dwork);
 
-	rcu_read_lock();
-	nvdev = rcu_dereference(ndev_ctx->nvdev);
-
-	if  (nvdev)
+	rtnl_lock();
+	nvdev = rtnl_dereference(ndev_ctx->nvdev);
+	if (nvdev)
 		cancel_work_sync(&nvdev->subchan_work);
 
 	/*
 	 * Call to the vsc driver to let it know that the device is being
 	 * removed. Also blocks mtu and channel changes.
 	 */
-	rtnl_lock();
 	vf_netdev = rtnl_dereference(ndev_ctx->vf_netdev);
 	if (vf_netdev)
 		netvsc_unregister_vf(vf_netdev);
@@ -2132,7 +2130,6 @@ static int netvsc_remove(struct hv_device *dev)
 	list_del(&ndev_ctx->list);
 
 	rtnl_unlock();
-	rcu_read_unlock();
 
 	hv_set_drvdata(dev, NULL);
 
-- 
2.17.1


  parent reply	other threads:[~2018-10-05 16:16 UTC|newest]

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-10-05 16:16 [PATCH AUTOSEL 4.14 01/23] ASoC: rt5514: Fix the issue of the delay volume applied again Sasha Levin
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 02/23] ASoC: wm8804: Add ACPI support Sasha Levin
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 03/23] ASoC: sigmadsp: safeload should not have lower byte limit Sasha Levin
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 04/23] selftests/efivarfs: add required kernel configs Sasha Levin
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 05/23] selftests: memory-hotplug: add required configs Sasha Levin
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 06/23] ASoC: rsnd: adg: care clock-frequency size Sasha Levin
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 07/23] ASoC: rsnd: don't fallback to PIO mode when -EPROBE_DEFER Sasha Levin
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 08/23] Bluetooth: hci_ldisc: Free rw_semaphore on close Sasha Levin
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 09/23] mfd: omap-usb-host: Fix dts probe of children Sasha Levin
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 10/23] scsi: iscsi: target: Don't use stack buffer for scatterlist Sasha Levin
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 11/23] scsi: qla2xxx: Fix an endian bug in fcpcmd_is_corrupted() Sasha Levin
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 12/23] sound: enable interrupt after dma buffer initialization Sasha Levin
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 13/23] sound: don't call skl_init_chip() to reset intel skl soc Sasha Levin
2018-10-05 16:16 ` Sasha Levin [this message]
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 15/23] stmmac: fix valid numbers of unicast filter entries Sasha Levin
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 16/23] net: macb: disable scatter-gather for macb on sama5d3 Sasha Levin
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 17/23] ARM: dts: at91: add new compatibility string " Sasha Levin
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 18/23] PCI: hv: support reporting serial number as slot information Sasha Levin
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 19/23] clk: x86: add "ether_clk" alias for Bay Trail / Cherry Trail Sasha Levin
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 20/23] clk: x86: Stop marking clocks as CLK_IS_CRITICAL Sasha Levin
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 21/23] x86/kvm/lapic: always disable MMIO interface in x2APIC mode Sasha Levin
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 22/23] drm/amdgpu: Fix SDMA HQD destroy error on gfx_v7 Sasha Levin
2018-10-05 16:16 ` [PATCH AUTOSEL 4.14 23/23] ubifs: Check for name being NULL while mounting Sasha Levin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20181005161634.20631-14-sashal@kernel.org \
    --to=sashal@kernel.org \
    --cc=alexander.levin@microsoft.com \
    --cc=davem@davemloft.net \
    --cc=linux-kernel@vger.kernel.org \
    --cc=stable@vger.kernel.org \
    --cc=stephen@networkplumber.org \
    --cc=sthemmin@microsoft.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox