From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
stable@vger.kernel.org, Eric Dumazet <edumazet@google.com>,
"David S. Miller" <davem@davemloft.net>
Subject: [PATCH 3.18 07/36] Fix an intermittent pr_emerg warning about lo becoming free.
Date: Mon, 3 Jul 2017 15:34:04 +0200 [thread overview]
Message-ID: <20170703133256.608537557@linuxfoundation.org> (raw)
In-Reply-To: <20170703133256.260692013@linuxfoundation.org>
3.18-stable review patch. If anyone has any objections, please let me know.
------------------
From: Krister Johansen <kjlx@templeofstupid.com>
[ Upstream commit f186ce61bb8235d80068c390dc2aad7ca427a4c2 ]
It looks like this:
Message from syslogd@flamingo at Apr 26 00:45:00 ...
kernel:unregister_netdevice: waiting for lo to become free. Usage count = 4
They seem to coincide with net namespace teardown.
The message is emitted by netdev_wait_allrefs().
Forced a kdump in netdev_run_todo, but found that the refcount on the lo
device was already 0 at the time we got to the panic.
Used bcc to check the blocking in netdev_run_todo. The only places
where we're off cpu there are in the rcu_barrier() and msleep() calls.
That behavior is expected. The msleep time coincides with the amount of
time we spend waiting for the refcount to reach zero; the rcu_barrier()
wait times are not excessive.
After looking through the list of callbacks that the netdevice notifiers
invoke in this path, it appears that the dst_dev_event is the most
interesting. The dst_ifdown path places a hold on the loopback_dev as
part of releasing the dev associated with the original dst cache entry.
Most of our notifier callbacks are straight-forward, but this one a)
looks complex, and b) places a hold on the network interface in
question.
I constructed a new bcc script that watches various events in the
liftime of a dst cache entry. Note that dst_ifdown will take a hold on
the loopback device until the invalidated dst entry gets freed.
[ __dst_free] on DST: ffff883ccabb7900 IF tap1008300eth0 invoked at 1282115677036183
__dst_free
rcu_nocb_kthread
kthread
ret_from_fork
Acked-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
net/core/dst.c | 14 ++++++++++++++
1 file changed, 14 insertions(+)
--- a/net/core/dst.c
+++ b/net/core/dst.c
@@ -397,6 +397,20 @@ static int dst_dev_event(struct notifier
spin_lock_bh(&dst_garbage.lock);
dst = dst_garbage.list;
dst_garbage.list = NULL;
+ /* The code in dst_ifdown places a hold on the loopback device.
+ * If the gc entry processing is set to expire after a lengthy
+ * interval, this hold can cause netdev_wait_allrefs() to hang
+ * out and wait for a long time -- until the the loopback
+ * interface is released. If we're really unlucky, it'll emit
+ * pr_emerg messages to console too. Reset the interval here,
+ * so dst cleanups occur in a more timely fashion.
+ */
+ if (dst_garbage.timer_inc > DST_GC_INC) {
+ dst_garbage.timer_inc = DST_GC_INC;
+ dst_garbage.timer_expires = DST_GC_MIN;
+ mod_delayed_work(system_wq, &dst_gc_work,
+ dst_garbage.timer_expires);
+ }
spin_unlock_bh(&dst_garbage.lock);
if (last)
next prev parent reply other threads:[~2017-07-03 13:36 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-07-03 13:33 [PATCH 3.18 00/36] 3.18.60-stable review Greg Kroah-Hartman
2017-07-03 13:33 ` [PATCH 3.18 01/36] xhci: fix deadlock at host remove by running watchdog correctly Greg Kroah-Hartman
2017-07-03 13:33 ` [PATCH 3.18 02/36] ipv6: release dst on error in ip6_dst_lookup_tail Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 03/36] netfilter: xt_TCPMSS: add more sanity tests on tcph->doff Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 04/36] netfilter: synproxy: fix conntrackd interaction Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 05/36] net: dont call strlen on non-terminated string in dev_set_alias() Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 06/36] decnet: dn_rtmsg: Improve input length sanitization in dnrmg_receive_user_skb Greg Kroah-Hartman
2017-07-03 13:34 ` Greg Kroah-Hartman [this message]
2017-07-03 13:34 ` [PATCH 3.18 08/36] net: caif: Fix a sleep-in-atomic bug in cfpkt_create_pfx Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 09/36] igmp: acquire pmc lock for ip_mc_clear_src() Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 10/36] igmp: add a missing spin_lock_init() Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 11/36] ipv6: fix calling in6_ifa_hold incorrectly for dad work Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 12/36] decnet: always not take dst->__refcnt when inserting dst into hash table Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 13/36] net: 8021q: Fix one possible panic caused by BUG_ON in free_netdev Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 14/36] NFSv4: fix a reference leak caused WARNING messages Greg Kroah-Hartman
2017-07-03 14:33 ` Trond Myklebust
2017-07-03 15:02 ` gregkh
2017-07-03 15:02 ` gregkh
2017-07-03 13:34 ` [PATCH 3.18 15/36] arm64: cpuinfo: Missing NULL terminator in compat_hwcap_str Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 16/36] MIPS: Avoid accidental raw backtrace Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 17/36] MIPS: pm-cps: Drop manual cache-line alignment of ready_count Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 18/36] MIPS: Fix IRQ tracing & lockdep when rescheduling Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 19/36] ALSA: hda - set input_path bitmap to zero after moving it to new place Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 20/36] drm/vmwgfx: Free hash table allocated by cmdbuf managed res mgr Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 21/36] usb: gadget: f_fs: Fix possibe deadlock Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 22/36] sysctl: enable strict writes Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 23/36] mm: numa: avoid waiting on freed migrated pages Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 25/36] net: korina: Fix NAPI versus resources freeing Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 27/36] xfrm: fix stack access out of bounds with CONFIG_XFRM_SUB_POLICY Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 28/36] xfrm: NULL dereference on allocation failure Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 29/36] xfrm: Oops on error in pfkey_msg2xfrm_state() Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 30/36] watchdog: bcm281xx: Fix use of uninitialized spinlock Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 31/36] ARM: 8685/1: ensure memblock-limit is pmd-aligned Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 32/36] iommu/vt-d: Dont over-free page table directories Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 33/36] iommu/amd: Fix incorrect error handling in amd_iommu_bind_pasid() Greg Kroah-Hartman
2017-07-03 13:34 ` [PATCH 3.18 34/36] cpufreq: s3c2416: double free on driver init error path Greg Kroah-Hartman
2017-07-03 19:34 ` [PATCH 3.18 00/36] 3.18.60-stable review Guenter Roeck
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170703133256.608537557@linuxfoundation.org \
--to=gregkh@linuxfoundation.org \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.