From mboxrd@z Thu Jan 1 00:00:00 1970 From: Eric Dumazet Subject: Re: SACK compression patch causing performance drop Date: Thu, 8 Nov 2018 07:14:46 -0800 Message-ID: References: <89214271-4496-c438-c4e8-b39f0ae72af7@dupond.be> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit To: Jean-Louis Dupond , netdev@vger.kernel.org, edumazet@google.com Return-path: Received: from mail-pg1-f170.google.com ([209.85.215.170]:32950 "EHLO mail-pg1-f170.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726421AbeKIAuq (ORCPT ); Thu, 8 Nov 2018 19:50:46 -0500 Received: by mail-pg1-f170.google.com with SMTP id q5-v6so8988367pgv.0 for ; Thu, 08 Nov 2018 07:14:48 -0800 (PST) In-Reply-To: Content-Language: en-US Sender: netdev-owner@vger.kernel.org List-ID: On 11/08/2018 12:23 AM, Jean-Louis Dupond wrote: > Hi, > > Was somebody able to check this? > Really think this should be fixed :) > > Thanks > Jean-Louis > I somehow missed this email. Packet captures might help, please send me tcpdump -s 128 -i ethX -w sack.pcap of some samples with or without the sack compression enabled. > On 3/11/18 16:59, Jean-Louis Dupond wrote: >> Hi All, >> >> On recent kernels we noticed a way lower throughput to our SAN system than before. >> While on pre 4.18 kernels we had 400-700MB/sec read speed, on 4.18+ we only had 70-120MB/sec. >> >> The SAN is connected via iSCSI over a 10G network (ixgbe/X520 NICS if it matters). >> >> After some debugging, I tried to bisect between 4.17 and 4.18 to see what commit caused the slowdown. >> It showed that the addition of the SACK compression (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5d9f4262b7ea41ca9981cc790e37cca6e37c789e) was the cause. >> >> And indeed, if I set net.ipv4.tcp_comp_sack_nr to 0 on 4.19 for example, the throughput is (almost) back to normal again. >> So it seems like this change causes quite some performance issues. >> >> Any ideas? >> >> Thanks >> Jean-Louis >>