Brust data send problem on gigabit NIC on Linux

Brust data send problem on gigabit NIC on Linux

[Lk Overrun]

Hi, I am seeking advice on how to best send out huge
number of packets on a gigabit ethernet interface.  I
am using kernel 2.4.19.  I try to send out as many as
possible 15Kbyte-long ethernet packets to try to
utilize the giga-bit/sec bandwidth.  My CPU is really
fast (2 GHz) amd I dump the packets to the interface
in a  tight loop in user space.  However, I can only
reach around 400 Mbits/sec before the packets get
dropped.  The queue discipline (qdisc) seems to be
responsible because the queue length (txqueuelen) is
only 100 by default, and the queue just cannot store
so many packets at once.  I can eliminate the packet
drop by raising the queue length to somewhere like
60000 but that is not practical because it uses too
much memory. It seems I need some delay between
sending packets but I cannot sleep for less than 10 ms
(1/Hz) in user space and 10 ms is too long.

I am using raw socket bypassing the IP stack and my
NIC is the Intel Pro1000 (using the e1000.o driver).

What is the best way to send raw ethernet packets,
reaching gigabit range withuut packet drop on Linux? 
Thanks for any advice.

 

__________________________________________________
Do you Yahoo!?
Y! Web Hosting - Let the expert host your web site
http://webhosting.yahoo.com/

Re: Brust data send problem on gigabit NIC on Linux

[Ben Greear]

Lk Overrun wrote:
> Hi, I am seeking advice on how to best send out huge
> number of packets on a gigabit ethernet interface.  I
> am using kernel 2.4.19.  I try to send out as many as
> possible 15Kbyte-long ethernet packets to try to
> utilize the giga-bit/sec bandwidth.  My CPU is really
> fast (2 GHz) amd I dump the packets to the interface
> in a  tight loop in user space.  However, I can only
> reach around 400 Mbits/sec before the packets get
> dropped.  The queue discipline (qdisc) seems to be
> responsible because the queue length (txqueuelen) is
> only 100 by default, and the queue just cannot store
> so many packets at once.  I can eliminate the packet
> drop by raising the queue length to somewhere like
> 60000 but that is not practical because it uses too
> much memory. It seems I need some delay between
> sending packets but I cannot sleep for less than 10 ms
> (1/Hz) in user space and 10 ms is too long.
> 
> I am using raw socket bypassing the IP stack and my
> NIC is the Intel Pro1000 (using the e1000.o driver).
> 
> What is the best way to send raw ethernet packets,
> reaching gigabit range withuut packet drop on Linux? 
> Thanks for any advice.

Make sure the e1000 driver is tuned, try insmodding it with:
TxDescriptors=1024 RxDescriptors=4096

Also, in your user-space app, check the return code of your
packet-sending call.  That can let you know that the kernel
dropped it, and that you need to re-send.

How do you know you are dropping packets?  (Ie, are you also
reading on another machine?)  Usually it's read that drops
more packets than write.

Good luck,
Ben

> 
>  
> 
> __________________________________________________
> Do you Yahoo!?
> Y! Web Hosting - Let the expert host your web site
> http://webhosting.yahoo.com/
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 


-- 
Ben Greear <greearb@candelatech.com>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear

Re: Brust data send problem on gigabit NIC on Linux

[Jon Burgess]

>I try to send out as many as
 >possible 15Kbyte-long ethernet packets to try to
> utilize the giga-bit/sec bandwidth

If you just want to generate packets then try pktgen in Linux-2.4.19:

CONFIG_NET_PKTGEN
  This module will inject preconfigured packets, at a configurable
  rate, out of a given interface.  It is used for network interface
  stress testing and performance analysis.  If you don't understand
  what was just said, you don't need it: say N.

  Documentation on how to use the packet generator can be found
  at <file:Documentation/networking/pktgen.txt>.

  This code is also available as a module called pktgen.o ( = code
  which can be inserted in and removed from the running kernel
  whenever you want).  If you want to compile it as a module, say M
  here and read <file:Documentation/modules.txt>.