From mboxrd@z Thu Jan 1 00:00:00 1970 From: Eric Dumazet Subject: Re: [RFC PATCH] ip: re-introduce fragments cache worker Date: Fri, 6 Jul 2018 07:20:45 -0700 Message-ID: <8480556a-b0a3-5e1a-d1b7-726c52b52046@gmail.com> References: <6512d94713d40f1d572d2023168c48990f0d9cf0.1530798211.git.pabeni@redhat.com> <51ef14ac-1d98-ad75-d282-eb6cb177fe7a@gmail.com> <1df6b0ea-885b-7d5e-a0c9-e01a5a33a4f2@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Cc: "David S. Miller" , Eric Dumazet , Florian Westphal , NeilBrown To: Paolo Abeni , Eric Dumazet , netdev@vger.kernel.org Return-path: Received: from mail-pl0-f67.google.com ([209.85.160.67]:45089 "EHLO mail-pl0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753638AbeGFOUs (ORCPT ); Fri, 6 Jul 2018 10:20:48 -0400 Received: by mail-pl0-f67.google.com with SMTP id bi1-v6so2967521plb.12 for ; Fri, 06 Jul 2018 07:20:47 -0700 (PDT) In-Reply-To: Content-Language: en-US Sender: netdev-owner@vger.kernel.org List-ID: On 07/06/2018 06:56 AM, Paolo Abeni wrote: > On Fri, 2018-07-06 at 05:09 -0700, Eric Dumazet wrote: >> On 07/06/2018 04:56 AM, Paolo Abeni wrote: >>> With your setting, you need a bit more concurrent connections (400 ?) >>> to saturate the ipfrag cache. Above that number, performances will >>> still sink. >> >> Maybe, but IP defrag can not be 'perfect'. >> >> For this particular use case I could still bump high_thresh to 6 GB and all would be good :) > > Understood. > > I'd like to be sure I stated the problem I see clearly. With the > current code the "goodput" goes to almost 0 as soon as the ipfrag cache > load goes above it's capacity. Before the worker removal, after > reaching high_thresh, the "goodput" degratated slowly and even with a > load more than an order of magnitude higher, the performances were > still quite good. I think we can't ask customers to add more memory for > a kernel upgrade; even changing the default sysfs configuration is > somewhat troubling. > >>> This looks nice, I'll try to test it in my use case and I'll report >>> here. > > I tried the patch, but the result are not encouraging: > > ./super_netperf.sh 200 -H 192.168.101.2 -t UDP_STREAM -l 60 > 34.94 > > # on the receiver side: > echo 2 > /proc/sys/net/ipv4/ipfrag_time > > # on the sender side: > ./super_netperf.sh 200 -H 192.168.101.2 -t UDP_STREAM -l 60 > 85.8 > > # still on receiver side, while the test is running: > nstat>/dev/null ;sleep 1; nstat |grep IpReasm > IpReasmTimeout 2128 0.0 > IpReasmReqds 754770 0.0 > IpReasmOKs 135 0.0 > IpReasmFails 752811 0.0 > > grep FRAG /proc/net/sockstat > FRAG: inuse 124 memory 5286144 > > The patch has some effect, as I basically saw no timeout without it, > but still does not look aggressive enough. Or possibly it's evicting > the fragments that are more likely to be used/completed (the most > recents one). Hey, that was simply an idea (not even compiled), not the final patch. I will test/polish it later, I am coming back from vacations and have a backlog. Here are my results : (Note that I have _not_ changed /proc/sys/net/ipv4/ipfrag_time ) lpaa6:~# grep . /proc/sys/net/ipv4/ipfrag_* ; grep FRAG /proc/net/sockstat /proc/sys/net/ipv4/ipfrag_high_thresh:104857600 /proc/sys/net/ipv4/ipfrag_low_thresh:78643200 /proc/sys/net/ipv4/ipfrag_max_dist:0 /proc/sys/net/ipv4/ipfrag_secret_interval:0 /proc/sys/net/ipv4/ipfrag_time:30 FRAG: inuse 1379 memory 105006776 lpaa5:/export/hda3/google/edumazet# ./super_netperf 400 -H 10.246.7.134 -t UDP_STREAM -l 60 netperf: send_omni: send_data failed: No route to host netperf: send_omni: send_data failed: No route to host 9063 I would say that it looks pretty good to me.