From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
stable@vger.kernel.org, Eric Dumazet <edumazet@google.com>,
"David S. Miller" <davem@davemloft.net>
Subject: [PATCH 4.11 10/84] Fix an intermittent pr_emerg warning about lo becoming free.
Date: Mon, 3 Jul 2017 15:34:50 +0200 [thread overview]
Message-ID: <20170703133403.641849974@linuxfoundation.org> (raw)
In-Reply-To: <20170703133402.874816941@linuxfoundation.org>
4.11-stable review patch. If anyone has any objections, please let me know.
------------------
From: Krister Johansen <kjlx@templeofstupid.com>
[ Upstream commit f186ce61bb8235d80068c390dc2aad7ca427a4c2 ]
It looks like this:
Message from syslogd@flamingo at Apr 26 00:45:00 ...
kernel:unregister_netdevice: waiting for lo to become free. Usage count = 4
They seem to coincide with net namespace teardown.
The message is emitted by netdev_wait_allrefs().
Forced a kdump in netdev_run_todo, but found that the refcount on the lo
device was already 0 at the time we got to the panic.
Used bcc to check the blocking in netdev_run_todo. The only places
where we're off cpu there are in the rcu_barrier() and msleep() calls.
That behavior is expected. The msleep time coincides with the amount of
time we spend waiting for the refcount to reach zero; the rcu_barrier()
wait times are not excessive.
After looking through the list of callbacks that the netdevice notifiers
invoke in this path, it appears that the dst_dev_event is the most
interesting. The dst_ifdown path places a hold on the loopback_dev as
part of releasing the dev associated with the original dst cache entry.
Most of our notifier callbacks are straight-forward, but this one a)
looks complex, and b) places a hold on the network interface in
question.
I constructed a new bcc script that watches various events in the
liftime of a dst cache entry. Note that dst_ifdown will take a hold on
the loopback device until the invalidated dst entry gets freed.
[ __dst_free] on DST: ffff883ccabb7900 IF tap1008300eth0 invoked at 1282115677036183
__dst_free
rcu_nocb_kthread
kthread
ret_from_fork
Acked-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
net/core/dst.c | 14 ++++++++++++++
1 file changed, 14 insertions(+)
--- a/net/core/dst.c
+++ b/net/core/dst.c
@@ -469,6 +469,20 @@ static int dst_dev_event(struct notifier
spin_lock_bh(&dst_garbage.lock);
dst = dst_garbage.list;
dst_garbage.list = NULL;
+ /* The code in dst_ifdown places a hold on the loopback device.
+ * If the gc entry processing is set to expire after a lengthy
+ * interval, this hold can cause netdev_wait_allrefs() to hang
+ * out and wait for a long time -- until the the loopback
+ * interface is released. If we're really unlucky, it'll emit
+ * pr_emerg messages to console too. Reset the interval here,
+ * so dst cleanups occur in a more timely fashion.
+ */
+ if (dst_garbage.timer_inc > DST_GC_INC) {
+ dst_garbage.timer_inc = DST_GC_INC;
+ dst_garbage.timer_expires = DST_GC_MIN;
+ mod_delayed_work(system_wq, &dst_gc_work,
+ dst_garbage.timer_expires);
+ }
spin_unlock_bh(&dst_garbage.lock);
if (last)
next prev parent reply other threads:[~2017-07-03 13:47 UTC|newest]
Thread overview: 81+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-07-03 13:34 [PATCH 4.11 00/84] 4.11.9-stable review Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 4.11 01/84] net: dont call strlen on non-terminated string in dev_set_alias() Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 4.11 02/84] net: Fix inconsistent teardown and release of private netdev state Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 4.11 03/84] net: s390: fix up for "Fix inconsistent teardown and release of private netdev state" Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 4.11 04/84] mac80211: free netdev on dev_alloc_name() error Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 4.11 05/84] decnet: dn_rtmsg: Improve input length sanitization in dnrmg_receive_user_skb Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 4.11 06/84] net: Zero ifla_vf_info in rtnl_fill_vfinfo() Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 4.11 07/84] net: ipv6: Release route when device is unregistering Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 4.11 08/84] net: vrf: Make add_fib_rules per network namespace flag Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 4.11 09/84] af_unix: Add sockaddr length checks before accessing sa_family in bind and connect handlers Greg Kroah-Hartman
2017-07-03 13:34 ` Greg Kroah-Hartman [this message]
2017-07-03 13:34 ` [PATCH 4.11 11/84] sctp: disable BH in sctp_for_each_endpoint Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 4.11 12/84] net: caif: Fix a sleep-in-atomic bug in cfpkt_create_pfx Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 4.11 13/84] net: tipc: Fix a sleep-in-atomic bug in tipc_msg_reverse Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 4.11 14/84] net/mlx5: Remove several module events out of ethtool stats Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 4.11 15/84] net/mlx5e: Added BW check for DIM decision mechanism Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 4.11 16/84] net/mlx5e: Fix wrong indications in DIM due to counter wraparound Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 4.11 17/84] net/mlx5: Enable 4K UAR only when page size is bigger than 4K Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 4.11 18/84] proc: snmp6: Use correct type in memset Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 4.11 19/84] igmp: acquire pmc lock for ip_mc_clear_src() Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 20/84] igmp: add a missing spin_lock_init() Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 22/84] net: dont global ICMP rate limit packets originating from loopback Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 23/84] ipv6: fix calling in6_ifa_hold incorrectly for dad work Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 24/84] sctp: return next obj by passing pos + 1 into sctp_transport_get_idx Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 25/84] net/mlx5e: Fix min inline value for VF rep SQs Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 26/84] net/mlx5e: Avoid doing a cleanup call if the profile doesnt have it Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 27/84] net/mlx5: Wait for FW readiness before initializing command interface Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 28/84] net/mlx5e: Fix timestamping capabilities reporting Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 29/84] decnet: always not take dst->__refcnt when inserting dst into hash table Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 30/84] net: 8021q: Fix one possible panic caused by BUG_ON in free_netdev Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 31/84] ipv6: Do not leak throw route references Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 32/84] rtnetlink: add IFLA_GROUP to ifla_policy Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 33/84] netfilter: synproxy: fix conntrackd interaction Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 34/84] NFSv4.x/callback: Create the callback service through svc_create_pooled Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 36/84] MIPS: head: Reorder instructions missing a delay slot Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 37/84] MIPS: Avoid accidental raw backtrace Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 38/84] MIPS: pm-cps: Drop manual cache-line alignment of ready_count Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 39/84] MIPS: Fix IRQ tracing & lockdep when rescheduling Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 40/84] ALSA: hda - Fix endless loop of codec configure Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 41/84] ALSA: hda - set input_path bitmap to zero after moving it to new place Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 42/84] NFSv4.2: Dont send mode again in post-EXCLUSIVE4_1 SETATTR with umask Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 43/84] NFSv4.1: Fix a race in nfs4_proc_layoutget Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 44/84] Revert "NFS: nfs_rename() handle -ERESTARTSYS dentry left behind" Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 45/84] ovl: copy-up: dont unlock between lookup and link Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 46/84] gpiolib: fix filtering out unwanted events Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 47/84] x86/intel_rdt: Fix memory leak on mount failure Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 48/84] perf/x86/intel/uncore: Fix wrong box pointer check Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 49/84] drm/vmwgfx: Free hash table allocated by cmdbuf managed res mgr Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 50/84] dm thin: do not queue freed thin mapping for next stage processing Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 51/84] x86/mm: Fix boot crash caused by incorrect loop count calculation in sync_global_pgds() Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 52/84] pinctrl/amd: Use regular interrupt instead of chained Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 53/84] mm/vmalloc.c: huge-vmap: fail gracefully on unexpected huge vmap mappings Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 55/84] xfrm6: Fix IPv6 payload_len in xfrm6_transport_finish Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 56/84] xfrm: move xfrm_garbage_collect out of xfrm_policy_flush Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 57/84] xfrm: fix stack access out of bounds with CONFIG_XFRM_SUB_POLICY Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 58/84] xfrm: NULL dereference on allocation failure Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 59/84] xfrm: Oops on error in pfkey_msg2xfrm_state() Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 60/84] watchdog: bcm281xx: Fix use of uninitialized spinlock Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 61/84] ARM64: PCI: Fix struct acpi_pci_root_ops allocation failure path Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 62/84] ARM64/ACPI: Fix BAD_MADT_GICC_ENTRY() macro implementation Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 63/84] ARM: 8685/1: ensure memblock-limit is pmd-aligned Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 64/84] ARM: davinci: PM: Free resources in error handling path in davinci_pm_init Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 65/84] ARM: davinci: PM: Do not free useful resources in normal " Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 66/84] tools arch: Sync arch/x86/lib/memcpy_64.S with the kernel Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 67/84] Revert "x86/entry: Fix the end of the stack for newly forked tasks" Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 68/84] x86/mshyperv: Remove excess #includes from mshyperv.h Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 69/84] x86/boot/KASLR: Fix kexec crash due to virt_addr calculation bug Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 70/84] perf/x86: Fix spurious NMI with PEBS Load Latency event Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 71/84] x86/mpx: Correctly report do_mpx_bt_fault() failures to user-space Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 72/84] x86/mm: Fix flush_tlb_page() on Xen Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 73/84] ocfs2: o2hb: revert hb threshold to keep compatible Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 74/84] ocfs2: fix deadlock caused by recursive locking in xattr Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 75/84] iommu/dma: Dont reserve PCI I/O windows Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 76/84] iommu/amd: Fix incorrect error handling in amd_iommu_bind_pasid() Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 77/84] iommu/amd: Fix interrupt remapping when disable guest_mode Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 78/84] infiniband: hns: avoid gcc-7.0.1 warning for uninitialized data Greg Kroah-Hartman
2017-07-03 13:35 ` [PATCH 4.11 79/84] mtd: nand: brcmnand: Check flash #WP pin status before nand erase/program Greg Kroah-Hartman
2017-07-03 13:36 ` [PATCH 4.11 80/84] mtd: nand: fsmc: fix NAND width handling Greg Kroah-Hartman
2017-07-03 13:36 ` [PATCH 4.11 81/84] KVM: x86: fix emulation of RSM and IRET instructions Greg Kroah-Hartman
2017-07-03 19:55 ` [PATCH 4.11 00/84] 4.11.9-stable review Guenter Roeck
2017-07-04 7:59 ` Greg Kroah-Hartman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170703133403.641849974@linuxfoundation.org \
--to=gregkh@linuxfoundation.org \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.