All of lore.kernel.org
 help / color / mirror / Atom feed
From: Grant Zhang <gzhang@fastly.com>
To: Eric Dumazet <eric.dumazet@gmail.com>, Patrick Schaaf <kernelorg@bof.de>
Cc: NETDEV <netdev@vger.kernel.org>,
	linux-kernel <linux-kernel@vger.kernel.org>
Subject: Re: Kernel 4.1 hang, apparently in __inet_lookup_established
Date: Sun, 15 Nov 2015 16:58:33 -0800	[thread overview]
Message-ID: <56492A39.2060307@fastly.com> (raw)
In-Reply-To: <1443025879.29850.116.camel@edumazet-glaptop2.roam.corp.google.com>

Hi Patrick,

Have you tried the two patches Eric mentioned? One of my 4.1.11 server 
just hanged with very similar stack trace and I am wondering whether the 
aforementioned patches would help.

Thanks,

Grant

On 23/09/2015 09:31, Eric Dumazet wrote:
> On Wed, 2015-09-23 at 10:25 +0200, Patrick Schaaf wrote:
>> Dear kernel developers,
>>
>> I recently started to upgrade my production hosts and VMs from the 3.14 series
>> to 4.1 kernels, starting with 4.1.6. Yesterday, for the second time after I
>> started these upgrades, I experienced one of our webserver VMs hanging.
>>
>> The first time this happened, the VM hung completely, all 5 virtual cores
>> spinning at 100%, ping still worked, but nothing else, including no virsh
>> console reaction - I had to destroy and restart that VM. No messages were to
>> be found.
>>
>> Yesterday, when it happened the second time, I found the VM spinning on a
>> single core only, and could still connect to it via ssh - but it stopped
>> accepting apache connections. The core it spun on showed 100% time used in
>> "si", with top, and it produced the messages appended below. The VM did not
>> shutdown properly when told to, and had to be destroyed again.
>>
>> If I read that dmesg output correctly it spins in __inet_lookup_established,
>> which indeed reads like it has infinite spin potential. But that code itself
>> did not change relative to the 3.14 series we've been running for a long time
>> without the issues - so the root cause would be something else.
>>
>> For our production systems I'll revert to the 3.14 series, but maybe this
>> report may help somebody understand what's going on.
>>
>> best regards
>>    Patrick
>
>
> You could try following commits :
>
> http://git.kernel.org/cgit/linux/kernel/git/davem/net.git/commit/?id=ed2e923945892a8372ab70d2f61d364b0b6d9054
>
> http://git.kernel.org/cgit/linux/kernel/git/davem/net.git/commit/?id=29c6852602e259d2c1882f320b29d5c3fec0de04
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

  reply	other threads:[~2015-11-16  0:58 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-09-23  8:25 Kernel 4.1 hang, apparently in __inet_lookup_established Patrick Schaaf
2015-09-23 16:31 ` Eric Dumazet
2015-11-16  0:58   ` Grant Zhang [this message]
2015-11-16 11:30     ` Patrick Schaaf
2015-11-16 15:07     ` Eric Dumazet
2015-11-16 17:30       ` Grant Zhang
2016-01-11 20:27         ` Grant Zhang
2016-01-11 21:11           ` Eric Dumazet
2016-01-11 21:47             ` Eric Dumazet
2016-01-13 16:01             ` David Miller
2016-01-13 16:19               ` Eric Dumazet
2016-01-13 19:38                 ` David Miller
2016-01-13 22:16                   ` Eric Dumazet

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=56492A39.2060307@fastly.com \
    --to=gzhang@fastly.com \
    --cc=eric.dumazet@gmail.com \
    --cc=kernelorg@bof.de \
    --cc=linux-kernel@vger.kernel.org \
    --cc=netdev@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.