From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
stable@vger.kernel.org, Yaju Cao <yacao@redhat.com>,
"K. Y. Srinivasan" <kys@microsoft.com>,
Haiyang Zhang <haiyangz@microsoft.com>,
Jason Wang <jasowang@redhat.com>,
"David S. Miller" <davem@davemloft.net>
Subject: [PATCH 3.12 19/77] netvsc: dont flush peers notifying work during setting mtu
Date: Mon, 13 Jan 2014 16:27:40 -0800 [thread overview]
Message-ID: <20140114002753.046760713@linuxfoundation.org> (raw)
In-Reply-To: <20140114002752.497010554@linuxfoundation.org>
3.12-stable review patch. If anyone has any objections, please let me know.
------------------
From: Jason Wang <jasowang@redhat.com>
[ Upstream commit 50dc875f2e6e2e04aed3b3033eb0ac99192d6d02 ]
There's a possible deadlock if we flush the peers notifying work during setting
mtu:
[ 22.991149] ======================================================
[ 22.991173] [ INFO: possible circular locking dependency detected ]
[ 22.991198] 3.10.0-54.0.1.el7.x86_64.debug #1 Not tainted
[ 22.991219] -------------------------------------------------------
[ 22.991243] ip/974 is trying to acquire lock:
[ 22.991261] ((&(&net_device_ctx->dwork)->work)){+.+.+.}, at: [<ffffffff8108af95>] flush_work+0x5/0x2e0
[ 22.991307]
but task is already holding lock:
[ 22.991330] (rtnl_mutex){+.+.+.}, at: [<ffffffff81539deb>] rtnetlink_rcv+0x1b/0x40
[ 22.991367]
which lock already depends on the new lock.
[ 22.991398]
the existing dependency chain (in reverse order) is:
[ 22.991426]
-> #1 (rtnl_mutex){+.+.+.}:
[ 22.991449] [<ffffffff810dfdd9>] __lock_acquire+0xb19/0x1260
[ 22.991477] [<ffffffff810e0d12>] lock_acquire+0xa2/0x1f0
[ 22.991501] [<ffffffff81673659>] mutex_lock_nested+0x89/0x4f0
[ 22.991529] [<ffffffff815392b7>] rtnl_lock+0x17/0x20
[ 22.991552] [<ffffffff815230b2>] netdev_notify_peers+0x12/0x30
[ 22.991579] [<ffffffffa0340212>] netvsc_send_garp+0x22/0x30 [hv_netvsc]
[ 22.991610] [<ffffffff8108d251>] process_one_work+0x211/0x6e0
[ 22.991637] [<ffffffff8108d83b>] worker_thread+0x11b/0x3a0
[ 22.991663] [<ffffffff81095e5d>] kthread+0xed/0x100
[ 22.991686] [<ffffffff81681c6c>] ret_from_fork+0x7c/0xb0
[ 22.991715]
-> #0 ((&(&net_device_ctx->dwork)->work)){+.+.+.}:
[ 22.991715] [<ffffffff810de817>] check_prevs_add+0x967/0x970
[ 22.991715] [<ffffffff810dfdd9>] __lock_acquire+0xb19/0x1260
[ 22.991715] [<ffffffff810e0d12>] lock_acquire+0xa2/0x1f0
[ 22.991715] [<ffffffff8108afde>] flush_work+0x4e/0x2e0
[ 22.991715] [<ffffffff8108e1b5>] __cancel_work_timer+0x95/0x130
[ 22.991715] [<ffffffff8108e303>] cancel_delayed_work_sync+0x13/0x20
[ 22.991715] [<ffffffffa03404e4>] netvsc_change_mtu+0x84/0x200 [hv_netvsc]
[ 22.991715] [<ffffffff815233d4>] dev_set_mtu+0x34/0x80
[ 22.991715] [<ffffffff8153bc2a>] do_setlink+0x23a/0xa00
[ 22.991715] [<ffffffff8153d054>] rtnl_newlink+0x394/0x5e0
[ 22.991715] [<ffffffff81539eac>] rtnetlink_rcv_msg+0x9c/0x260
[ 22.991715] [<ffffffff8155cdd9>] netlink_rcv_skb+0xa9/0xc0
[ 22.991715] [<ffffffff81539dfa>] rtnetlink_rcv+0x2a/0x40
[ 22.991715] [<ffffffff8155c41d>] netlink_unicast+0xdd/0x190
[ 22.991715] [<ffffffff8155c807>] netlink_sendmsg+0x337/0x750
[ 22.991715] [<ffffffff8150d219>] sock_sendmsg+0x99/0xd0
[ 22.991715] [<ffffffff8150d63e>] ___sys_sendmsg+0x39e/0x3b0
[ 22.991715] [<ffffffff8150eba2>] __sys_sendmsg+0x42/0x80
[ 22.991715] [<ffffffff8150ebf2>] SyS_sendmsg+0x12/0x20
[ 22.991715] [<ffffffff81681d19>] system_call_fastpath+0x16/0x1b
This is because we hold the rtnl_lock() before ndo_change_mtu() and try to flush
the work in netvsc_change_mtu(), in the mean time, netdev_notify_peers() may be
called from worker and also trying to hold the rtnl_lock. This will lead the
flush won't succeed forever. Solve this by not canceling and flushing the work,
this is safe because the transmission done by NETDEV_NOTIFY_PEERS was
synchronized with the netif_tx_disable() called by netvsc_change_mtu().
Reported-by: Yaju Cao <yacao@redhat.com>
Tested-by: Yaju Cao <yacao@redhat.com>
Cc: K. Y. Srinivasan <kys@microsoft.com>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
Acked-by: Haiyang Zhang <haiyangz@microsoft.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
drivers/net/hyperv/netvsc_drv.c | 1 -
1 file changed, 1 deletion(-)
--- a/drivers/net/hyperv/netvsc_drv.c
+++ b/drivers/net/hyperv/netvsc_drv.c
@@ -327,7 +327,6 @@ static int netvsc_change_mtu(struct net_
return -EINVAL;
nvdev->start_remove = true;
- cancel_delayed_work_sync(&ndevctx->dwork);
cancel_work_sync(&ndevctx->work);
netif_tx_disable(ndev);
rndis_filter_device_remove(hdev);
next prev parent reply other threads:[~2014-01-14 0:27 UTC|newest]
Thread overview: 83+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-01-14 0:27 [PATCH 3.12 00/77] 3.12.8-stable review Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 01/77] IPv6: Fixed support for blackhole and prohibit routes Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 02/77] net: do not pretend FRAGLIST support Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 03/77] rds: prevent BUG_ON triggered on congestion update to loopback Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 04/77] net: clear local_df when passing skb between namespaces Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 05/77] macvtap: update file current position Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 06/77] tun: " Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 07/77] tun: unbreak truncated packet signalling Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 08/77] macvtap: Do not double-count received packets Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 09/77] macvtap: signal truncated packets Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 10/77] virtio: delete napi structures from netdev before releasing memory Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 11/77] br: fix use of ->rx_handler_data in code executed on non-rx_handler path Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 12/77] packet: fix send path when running with proto == 0 Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 13/77] ipv6: dont count addrconf generated routes against gc limit Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 14/77] net: drop_monitor: fix the value of maxattr Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 15/77] inet: fix NULL pointer Oops in fib(6)_rule_suppress Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 16/77] net: unix: allow set_peek_off to fail Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 17/77] vxlan: release rt when found circular route Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 18/77] tg3: Initialize REG_BASE_ADDR at PCI config offset 120 to 0 Greg Kroah-Hartman
2014-01-14 0:27 ` Greg Kroah-Hartman [this message]
2014-01-14 0:27 ` [PATCH 3.12 20/77] ipv6: fix illegal mac_header comparison on 32bit Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 21/77] net: unix: allow bind to fail on mutex lock Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 23/77] net: inet_diag: zero out uninitialized idiag_{src,dst} fields Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 24/77] drivers/net/hamradio: Integer overflow in hdlcdrv_ioctl() Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 26/77] net: fec: fix potential use after free Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 27/77] ipv6: always set the new created dsts from in ip6_rt_copy Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 28/77] rds: prevent dereference of a NULL device Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 29/77] arc_emac: fix potential use after free Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 30/77] net: rose: restore old recvmsg behavior Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 31/77] vlan: Fix header ops passthru when doing TX VLAN offload Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 32/77] virtio_net: fix error handling for mergeable buffers Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 33/77] virtio-net: make all RX paths handle errors consistently Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 34/77] virtio_net: dont leak memory or block when too many frags Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 35/77] ipv4: fix tunneled VM traffic over hw VXLAN/GRE GSO NIC Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 36/77] virtio-net: fix refill races during restore Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 37/77] net: llc: fix use after free in llc_ui_recvmsg Greg Kroah-Hartman
2014-01-14 0:27 ` [PATCH 3.12 38/77] netpoll: Fix missing TXQ unlock and and OOPS Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 39/77] bridge: use spin_lock_bh() in br_multicast_set_hash_max Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 40/77] sfc: Add length checks to efx_xmit_with_hwtstamp() and efx_ptp_is_ptp_tx() Greg Kroah-Hartman
2014-01-14 0:45 ` Ben Hutchings
2014-01-16 10:50 ` Luis Henriques
2014-01-16 19:42 ` David Miller
2014-01-16 20:51 ` Luis Henriques
2014-01-16 21:15 ` Ben Hutchings
2014-01-14 0:28 ` [PATCH 3.12 41/77] sfc: PTP: Moderate log message on event queue overflow Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 42/77] sfc: Rate-limit log message for PTP packets without a matching timestamp event Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 43/77] sfc: Stop/re-start PTP when stopping/starting the datapath Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 44/77] sfc: Maintain current frequency adjustment when applying a time offset Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 45/77] sfc: RX buffer allocation takes prefix size into account in IP header alignment Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 46/77] sfc: Refactor efx_mcdi_poll() by introducing efx_mcdi_poll_once() Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 47/77] sfc: Poll for MCDI completion once before timeout occurs Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 48/77] ARM: fix footbridge clockevent device Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 49/77] ARM: fix "bad mode in ... handler" message for undefined instructions Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 50/77] ARM: 7923/1: mm: fix dcache flush logic for compound high pages Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 51/77] ARM: dts: exynos5250: Fix MDMA0 clock number Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 52/77] ARM: shmobile: kzm9g: Fix coherent DMA mask Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 53/77] ARM: shmobile: armadillo: " Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 54/77] ARM: shmobile: mackerel: " Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 56/77] parisc: Ensure full cache coherency for kmap/kunmap Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 57/77] ahci: add PCI ID for Marvell 88SE9170 SATA controller Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 58/77] clk: clk-divider: fix divisor > 255 bug Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 59/77] clk: samsung: exynos4: Correct SRC_MFC register Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 60/77] clk: samsung: exynos5250: Fix ACP gate register offset Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 61/77] clk: samsung: exynos5250: Add MDMA0 clocks Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 62/77] clk: samsung: exynos5250: Add CLK_IGNORE_UNUSED flag for the sysreg clock Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 63/77] clk: exynos5250: fix sysmmu_mfc{l,r} gate clocks Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 65/77] mfd: rtsx_pcr: Disable interrupts before cancelling delayed works Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 66/77] ACPI / TPM: fix memory leak when walking ACPI namespace Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 67/77] ACPI / Battery: Add a _BIX quirk for NEC LZ750/LS Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 68/77] mac80211: move "bufferable MMPDU" check to fix AP mode scan Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 69/77] intel_pstate: Add X86_FEATURE_APERFMPERF to cpu match parameters Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 70/77] SCSI: sd: Reduce buffer size for vpd request Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 71/77] netfilter: fix wrong byte order in nf_ct_seqadj_set internal information Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 72/77] netfilter: nf_nat: fix access to uninitialized buffer in IRC NAT helper Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 73/77] x86, fpu, amd: Clear exceptions in AMD FXSAVE workaround Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 74/77] sched: Fix race on toggling cfs_bandwidth_used Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 75/77] sched: Fix cfs_bandwidth misuse of hrtimer_expires_remaining Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 76/77] sched: Fix hrtimer_cancel()/rq->lock deadlock Greg Kroah-Hartman
2014-01-14 0:28 ` [PATCH 3.12 77/77] sched: Guarantee new group-entities always have weight Greg Kroah-Hartman
2014-01-14 3:03 ` [PATCH 3.12 00/77] 3.12.8-stable review Guenter Roeck
2014-01-14 14:42 ` Satoru Takeuchi
2014-01-14 23:12 ` Greg Kroah-Hartman
2014-01-14 19:31 ` Shuah Khan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140114002753.046760713@linuxfoundation.org \
--to=gregkh@linuxfoundation.org \
--cc=davem@davemloft.net \
--cc=haiyangz@microsoft.com \
--cc=jasowang@redhat.com \
--cc=kys@microsoft.com \
--cc=linux-kernel@vger.kernel.org \
--cc=stable@vger.kernel.org \
--cc=yacao@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).