From mboxrd@z Thu Jan 1 00:00:00 1970 From: Eric Dumazet Subject: Re: Strange packet drops with heavy firewalling Date: Thu, 15 Apr 2010 15:42:53 +0200 Message-ID: <1271338973.16881.2593.camel@edumazet-laptop> References: <1271083479.2858.377.camel@ursa.amorsen.dk> <1271091990.2858.409.camel@ursa.amorsen.dk> <1271138186.16881.168.camel@edumazet-laptop> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: Changli Gao , zhigang gong , netdev@vger.kernel.org To: Benny Amorsen Return-path: Received: from mail-bw0-f225.google.com ([209.85.218.225]:45535 "EHLO mail-bw0-f225.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753455Ab0DONnD (ORCPT ); Thu, 15 Apr 2010 09:43:03 -0400 Received: by bwz25 with SMTP id 25so1608697bwz.28 for ; Thu, 15 Apr 2010 06:43:01 -0700 (PDT) In-Reply-To: Sender: netdev-owner@vger.kernel.org List-ID: Le jeudi 15 avril 2010 =C3=A0 15:23 +0200, Benny Amorsen a =C3=A9crit : > Benny Amorsen writes: >=20 > > I'll keep monitoring the server, and if it starts dropping packets = again > > or load increases I'll check whether irqbalanced does the right thi= ng, > > and if not I'll implement your suggestion. >=20 > It did start dropping packets (although very few, a few packets dropp= ed > at once perhaps every ten minutes). Irqbalanced didn't move the > interrupts. >=20 > Doing >=20 > echo 01 >/proc/irq/99/smp_affinity > echo 02 >/proc/irq/100/smp_affinity > echo 04 >/proc/irq/101/smp_affinity >=20 > and so on like Erik Dumazet suggested seems to have helped, but not > entirely solved the problem. >=20 > The problem now manifests itself this way in ethtool -S: > rx_no_buffer_count: 270 > rx_queue_drop_packet_count: 270 >=20 > I can't be sure that I'm not just getting hit by a 1Gbps traffic spik= e, > of course, but it is a bit strange that a machine which can do 200Mbp= s > at 92% idle can't handle subsecond peaks close to 1Gbps... >=20 Even with multiqueue, its quite possible one queue gets more than one packet per micro second. Time to process a packet might be greater then 1 us even on recent hardware. So bursts of 1000 small packets with same flow information, hit one queue, one cpu, and fill rx ring. Loosing these packets is OK, its very likely its an attack :) > I wish ifstat could report errors so I could see what the traffic rat= e > was when the problem occurred... yes, it could be added I guess.