From mboxrd@z Thu Jan  1 00:00:00 1970
From: Stephen Hemminger <stephen@networkplumber.org>
Subject: Re: Significant capacity drop on loopback interface
Date: Thu, 10 May 2018 07:58:40 -0700
Message-ID: <20180510075840.12840865@xeon-e3>
References: <CANpxKHFZPEOgE21oPM2WHF97jhD8w3-N-wdycSPEnOeMPwREsQ@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Cc: netdev@vger.kernel.org
To: Naruto Nguyen <narutonguyen2018@gmail.com>
Return-path: <netdev-owner@vger.kernel.org>
Received: from mail-pf0-f179.google.com ([209.85.192.179]:34104 "EHLO
        mail-pf0-f179.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S935624AbeEJO6n (ORCPT
        <rfc822;netdev@vger.kernel.org>); Thu, 10 May 2018 10:58:43 -0400
Received: by mail-pf0-f179.google.com with SMTP id a14-v6so1211732pfi.1
        for <netdev@vger.kernel.org>; Thu, 10 May 2018 07:58:43 -0700 (PDT)
In-Reply-To: <CANpxKHFZPEOgE21oPM2WHF97jhD8w3-N-wdycSPEnOeMPwREsQ@mail.gmail.com>
Sender: netdev-owner@vger.kernel.org
List-ID: <netdev.vger.kernel.org>

On Thu, 10 May 2018 15:35:59 +0700
Naruto Nguyen <narutonguyen2018@gmail.com> wrote:

> Hello everyone,
> 
> Recently, I used netperf to test the TCP performance on loopback
> interface on my 2 nodes, one is installed kernel 4.4.103 and the other
> is 3.12.61
> 
> netperf -l 100 -t TCP_RR
> netperf -l 100 -t TCP_RR -- -D
> 
> In both cases, I see that the throughput on 4.4.103 is about just 1/2
> in comparing with 3.12.61 node
> 
> # netperf -l 100 -t TCP_RR
> MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0
> AF_INET to localhost () port 0 AF_INET : first burst 0
> Local /Remote
> Socket Size   Request  Resp.   Elapsed  Trans.
> Send   Recv   Size     Size    Time     Rate
> bytes  Bytes  bytes    bytes   secs.    per sec
> 
> 16384  87380  1        1       100.00   37714.68
> 16384  87380
> 
> 
> netperf -l 100 -t TCP_RR
> MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0
> AF_INET to localhost () port 0 AF_INET : first burst 0
> Local /Remote
> Socket Size   Request  Resp.   Elapsed  Trans.
> Send   Recv   Size     Size    Time     Rate
> bytes  Bytes  bytes    bytes   secs.    per sec
> 
> 16384  87380  1        1       100.00   64038.41
> 16384  87380
> 
> 
> When running tcpdump to capture all packets in loopback interface, I
> see that during 200s capture, the number of packets on loopback of
> 4.4.103 is double the number of packets in 3.12.61? Could you please
> let me know if it can cause the low throughput as above? Do we have
> any tuning for TCP on loopback to improve the performace (actually the
> low throughput also happens with UDP) or if we have any known
> performance issue in 4.4 kernel on loopback?
> 
> Thanks a lot,
> Brs,
> Naruto

This might just be the increased overhead of KPTI to fix Spectre/Meltdown.
Loopback is very sensitive to syscall overhead.