From mboxrd@z Thu Jan 1 00:00:00 1970 From: Eric Dumazet Subject: Re: Performance regression between 4.13 and 4.14 Date: Tue, 8 May 2018 10:10:27 -0700 Message-ID: References: <9115910b-dd8b-e7f3-be53-f739b8382032@candelatech.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit To: Ben Greear , netdev Return-path: Received: from mail-pl0-f41.google.com ([209.85.160.41]:46897 "EHLO mail-pl0-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752123AbeEHRK2 (ORCPT ); Tue, 8 May 2018 13:10:28 -0400 Received: by mail-pl0-f41.google.com with SMTP id 59-v6so2597371plc.13 for ; Tue, 08 May 2018 10:10:28 -0700 (PDT) In-Reply-To: <9115910b-dd8b-e7f3-be53-f739b8382032@candelatech.com> Content-Language: en-US Sender: netdev-owner@vger.kernel.org List-ID: On 05/08/2018 09:44 AM, Ben Greear wrote: > Hello, > > I am trying to track down a performance regression that appears to be between 4.13 > and 4.14. > > I first saw the problem with a hacked version of pktgen on some ixgbe NICs.  4.13 can do > right at 10G bi-directional on two ports, and 4.14 and later can do only about 6Gbps. > > I also tried with user-space UDP traffic on a stock kernel, and I can get about 3.2Gbps combined tx+rx > on 4.14 and about 4.4Gbps on 4.13. > > Attempting to bisect seems to be triggering a weirdness in git, and also lots of commits > crash or do not bring up networking, which makes the bisect difficult. > > Looking at perf top, it would appear that some lock is probably to blame. perf record -a -g -e cycles:pp sleep 5 perf report Then you'll be able to tell us which lock (or call graph) is killing your perf.