From: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
To: Linus Torvalds <torvalds@linux-foundation.org>,
Andrey Konovalov <andreyknvl@google.com>,
Eric Dumazet <edumazet@google.com>
Cc: "David S. Miller" <davem@davemloft.net>,
Alexey Kuznetsov <kuznet@ms2.inr.ac.ru>,
James Morris <jmorris@namei.org>,
Hideaki YOSHIFUJI <yoshfuji@linux-ipv6.org>,
Patrick McHardy <kaber@trash.net>,
netdev <netdev@vger.kernel.org>,
LKML <linux-kernel@vger.kernel.org>,
Alexander Duyck <alexander.h.duyck@intel.com>,
David Ahern <dsa@cumulusnetworks.com>,
Daniel Borkmann <daniel@iogearbox.net>, tcharding <me@tobin.cc>,
Jiri Pirko <jiri@mellanox.com>,
stephen hemminger <stephen@networkplumber.org>,
Dmitry Vyukov <dvyukov@google.com>,
Kostya Serebryany <kcc@google.com>,
syzkaller <syzkaller@googlegroups.com>
Subject: Re: net/core: BUG in unregister_netdevice_many
Date: Fri, 21 Apr 2017 21:30:30 +0300 [thread overview]
Message-ID: <2e044c75-c0e5-f5d5-1d94-7609d7af86fa@cumulusnetworks.com> (raw)
In-Reply-To: <CA+55aFyTFzML+_8E_kDjX=YFkaDi_ejGo15UgGtAT29244UwgA@mail.gmail.com>
On 21/04/17 20:42, Linus Torvalds wrote:
> On Fri, Apr 21, 2017 at 10:25 AM, Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
>>
>> I'm assuming that the real cause is simply that "dev->reg_state" ends
>> up being NETREG_UNREGISTERING or something. Maybe the BUG_ON() could
>> be just removed, and replaced by the previous warning about
>> NETREG_UNINITIALIZED.
>>
>> Something like the attached (TOTALLY UNTESTED) patch.
>
> .. might as well test it.
>
> That patch doesn't fix the problem, but it does show that yes, it was
> NETREG_UNREGISTERING:
>
> unregister_netdevice: device pim6reg/ffff962dc4606000 was not registered (2)
>
> but then immediately afterwards we get
>
> general protection fault: 0000 [#1] SMP
> Workqueue: netns cleanup_net
> RIP: 0010:dev_shutdown+0xe/0xc0
> Call Trace:
> rollback_registered_many+0x2a5/0x440
> unregister_netdevice_many+0x1e/0xb0
> default_device_exit_batch+0x145/0x170
>
> which is due to a
>
> mov 0x388(%rdi),%eax
>
> where %rdi is 0xdead000000000090. That is at the very beginning of
> dev_shutdown, it's "dev" itself that has that value, so it comes from
> (_another_) invocation of rollback_registered_many(), when it does
> that
>
> list_for_each_entry(dev, head, unreg_list) {
>
> so it seems to be a case of another "list_del() leaves list in bad
> state", and it was the added test for "dev->reg_state !=
> NETREG_REGISTERED" that did that
>
> list_del(&dev->unreg_list);
>
> and left random contents in the unreg_list.
>
> So that "handle error case" was almost certainly just buggy too.
>
> And the bug seems to be that we're trying to unregister a netdevice
> that has already been unregistered.
>
> Over to Eric and networking people. This oops is user-triggerable, and
> leaves the machine in a bad state (the original BUG_ON() and the new
> GP fault both happen while holding the RTNL, so networking is not
> healthy afterwards.
>
> Linus
>
Right, I've already posted a patch for ip6mr that should fix the issue.
CCed you and LKML just now.
Thanks,
Nik
next prev parent reply other threads:[~2017-04-21 18:30 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-04-21 12:48 net/core: BUG in unregister_netdevice_many Andrey Konovalov
2017-04-21 17:25 ` Linus Torvalds
2017-04-21 17:42 ` Linus Torvalds
2017-04-21 18:30 ` Nikolay Aleksandrov [this message]
2017-04-21 19:37 ` David Miller
2017-04-21 18:55 ` Cong Wang
2017-04-21 19:38 ` David Miller
2017-04-21 17:42 ` [PATCH net] ip6mr: fix notification device destruction Nikolay Aleksandrov
2017-04-21 18:30 ` Nikolay Aleksandrov
2017-04-21 18:46 ` Andrey Konovalov
2017-04-21 19:36 ` David Miller
2017-04-21 19:50 ` Nikolay Aleksandrov
2017-04-21 19:55 ` David Miller
2017-04-21 19:56 ` Nikolay Aleksandrov
2017-04-21 19:58 ` David Miller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2e044c75-c0e5-f5d5-1d94-7609d7af86fa@cumulusnetworks.com \
--to=nikolay@cumulusnetworks.com \
--cc=alexander.h.duyck@intel.com \
--cc=andreyknvl@google.com \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=dsa@cumulusnetworks.com \
--cc=dvyukov@google.com \
--cc=edumazet@google.com \
--cc=jiri@mellanox.com \
--cc=jmorris@namei.org \
--cc=kaber@trash.net \
--cc=kcc@google.com \
--cc=kuznet@ms2.inr.ac.ru \
--cc=linux-kernel@vger.kernel.org \
--cc=me@tobin.cc \
--cc=netdev@vger.kernel.org \
--cc=stephen@networkplumber.org \
--cc=syzkaller@googlegroups.com \
--cc=torvalds@linux-foundation.org \
--cc=yoshfuji@linux-ipv6.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox