From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jean-Michel Hautbois Subject: Re: Difficulties to get 1Gbps on be2net ethernet card Date: Wed, 6 Jun 2012 15:07:46 +0200 Message-ID: References: <1338360536.2760.85.camel@edumazet-glaptop> <1338361587.2760.94.camel@edumazet-glaptop> <3367B80B08154D42A3B2BC708B5D41F647C678B73F@EXMAIL.ad.emulex.com> <1338980484.2760.4219.camel@edumazet-glaptop> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: Sathya.Perla@emulex.com, netdev@vger.kernel.org To: Eric Dumazet Return-path: Received: from mail-we0-f174.google.com ([74.125.82.174]:60483 "EHLO mail-we0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753294Ab2FFNII convert rfc822-to-8bit (ORCPT ); Wed, 6 Jun 2012 09:08:08 -0400 Received: by weyu7 with SMTP id u7so4307787wey.19 for ; Wed, 06 Jun 2012 06:08:07 -0700 (PDT) In-Reply-To: Sender: netdev-owner@vger.kernel.org List-ID: 2012/6/6 Jean-Michel Hautbois : > 2012/6/6 Eric Dumazet : >> On Wed, 2012-06-06 at 12:04 +0200, Jean-Michel Hautbois wrote: >> >>> Well, well, well, after having tested several configurations, sever= al >>> drivers, I have a big difference between an old 2.6.26 kernel and a >>> newer one (I tried 3.2 and 3.4). >>> >>> Here is my stream : UDP packets (multicast), 4000 bytes length, MTU >>> set to 4096. I am sending packets only, nothing on RX. >>> I send from 1Gbps upto 2.4Gbps and I see no drops in tc with 2.6.26 >>> kernel, but a lot of drops with a newer kernel. >>> So, I don't know if I missed something in my kernel configuration, = but >>> I have used the 2.6.26 one as a reference, in order to set the same >>> options (DMA related, etc). >>> >>> I easily reproduce this problem and setting a bigger txqueuelen sol= ves >>> it partially. >>> 1Gbps requires a txqueulen of 9000, 2.4Gbps requires more than 2000= 0 ! >>> >>> If you have any idea, I am interested, as this is a big issue for m= y use case. >>> >> >> Yep. >> >> This driver wants to limit number of tx completions, thats just wron= g. >> >> Fix and dirty patch: >> >> >> diff --git a/drivers/net/ethernet/emulex/benet/be.h b/drivers/net/et= hernet/emulex/benet/be.h >> index c5c4c0e..1e8f8a6 100644 >> --- a/drivers/net/ethernet/emulex/benet/be.h >> +++ b/drivers/net/ethernet/emulex/benet/be.h >> @@ -105,7 +105,7 @@ static inline char *nic_name(struct pci_dev *pde= v) >> =C2=A0#define MAX_TX_QS =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A08 >> =C2=A0#define MAX_ROCE_EQS =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 5 >> =C2=A0#define MAX_MSIX_VECTORS =C2=A0 =C2=A0 =C2=A0 (MAX_RSS_QS + MA= X_ROCE_EQS) /* RSS qs + RoCE */ >> -#define BE_TX_BUDGET =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 256 >> +#define BE_TX_BUDGET =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 65535 >> =C2=A0#define BE_NAPI_WEIGHT =C2=A0 =C2=A0 =C2=A0 =C2=A0 64 >> =C2=A0#define MAX_RX_POST =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0B= E_NAPI_WEIGHT /* Frags posted at a time */ >> =C2=A0#define RX_FRAGS_REFILL_WM =C2=A0 =C2=A0 (RX_Q_LEN - MAX_RX_PO= ST) >> > > I will try that in a few minutes. > I also have a mlx4 driver (mlx4_en) which has a similar behaviour, an= d > a broadcom (bnx2x). > And it is not really better, still need about 18000 at 2.4Gbps in order to avoid drops... I really think there is something in the networking stack or in my configuration (DMA ? Something else ?)... As it doesn't seem to be driver related as I said... JM