From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756902AbcAJKXt (ORCPT ); Sun, 10 Jan 2016 05:23:49 -0500 Received: from vulcan.natalenko.name ([104.207.131.136]:52856 "EHLO vulcan.natalenko.name" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754518AbcAJKXr convert rfc822-to-8bit (ORCPT ); Sun, 10 Jan 2016 05:23:47 -0500 DMARC-Filter: OpenDMARC Filter v1.3.1 vulcan.natalenko.name 2550078ACE Authentication-Results: vulcan.natalenko.name; dmarc=none header.from=natalenko.name From: Oleksandr Natalenko To: Yuchung Cheng Cc: netdev , "linux-kernel@vger.kernel.org" , Patrick McHardy , Hideaki YOSHIFUJI , James Morris , Alexey Kuznetsov , "David S. Miller" Subject: Re: [REGRESSION] tcp/ipv4: kernel panic because of (possible) division by zero Date: Sun, 10 Jan 2016 12:23:43 +0200 Message-ID: <8830013.bJO8vnIv9N@spock> User-Agent: KMail/5.1 (Linux/4.3.0-pf3; KDE/5.17.0; x86_64; ; ) In-Reply-To: References: <26396443.iDTTxgChSj@spock> MIME-Version: 1.0 Content-Transfer-Encoding: 8BIT Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org With the patch queued for upstream and ECN enabled I get WARN_ON_ONCE() triggered. Here is the stacktrace: https://gist.github.com/89203e77bfcb051f269a It seems that tp->prior_cwnd is zero. Ideas? On середа, 6 січня 2016 р. 10:43:45 EET Yuchung Cheng wrote: > On Wed, Jan 6, 2016 at 10:19 AM, Yuchung Cheng wrote: > > On Wed, Jan 6, 2016 at 8:50 AM, Oleksandr Natalenko > > > > wrote: > >> Unfortunately, the patch didn't help -- I've got the same stacktrace with > >> slightly different offset (+3) within the function. > >> > >> Now trying to get full stacktrace via netconsole. Need more time. > >> > >> Meanwhile, any other ideas on what went wrong? > > > > That's odd b/c the patch already checks and avoids div0. Can post me > > the stacktrace and kernel warnings if any ... > > > > One possibility is that tcp_cwnd_reduction() may set a cwnd of 0, > > which then gets used to start another recovery phase. This may or may > > not be the culprit of this div0 issue because I wasn't able to > > reproduce exactly your issue on our servers. But I will post the fix > > today and CC you. > > Could you turn off ecn (sysctl net.ipv4.tcp_ecn=0) to see if this still > happen? > >> On December 22, 2015 4:10:32 AM EET, Yuchung Cheng wrote: > >> >On Mon, Dec 21, 2015 at 12:25 PM, Oleksandr Natalenko > >> > > >> > wrote: > >> >> Commit 3759824da87b30ce7a35b4873b62b0ba38905ef5 (tcp: PRR uses CRB > >> > > >> >mode by > >> > > >> >> default and SS mode conditionally) introduced changes to > >> > > >> >net/ipv4/tcp_input.c > >> > > >> >> tcp_cwnd_reduction() that, possibly, cause division by zero, and > >> > > >> >therefore, > >> > > >> >> kernel panic in interrupt handler [1]. > >> >> > >> >> Reverting 3759824da87b30ce7a35b4873b62b0ba38905ef5 seems to fix the > >> > > >> >issue. > >> > > >> >> I'm able to reproduce the issue on 4.3.0–4.3.3 once per several day > >> >> (occasionally). > >> >> > >> >> What could be done to help in debugging this issue? > >> > > >> >Do you have ECN enabled (i.e. sysctl net.ipv4.tcp_ecn > 0)? > >> > > >> >If so I suspect an ACK carrying ECE during CA_Loss causes entering CWR > >> >state w/o calling tcp_init_cwnd_reduct() to set tp->prior_cwnd. Can > >> >you try this debug / quick-fix patch and send me the error message if > >> >any? > >> > > >> >> Regards, > >> >> > >> >> Oleksandr. > >> >> > >> >> [1] http://i.piccy.info/ > >> > > >> >i9/6f5cb187c4ff282d189f78c63f95af43/1450729403/283985/951663/panic.jpg > >> > >> -- > >> Sent from my Android device with K-9 Mail. Please excuse my brevity.