From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andy Furniss Date: Tue, 06 Dec 2005 01:55:14 +0000 Subject: Re: [LARTC] The effects of queueing on delay...(TX Ring Buffer the Message-Id: <4394EF82.6000602@dsl.pipex.com> List-Id: References: <1129064647.13493.41.camel@pgala.it.nuigalway.ie> In-Reply-To: <1129064647.13493.41.camel@pgala.it.nuigalway.ie> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: lartc@vger.kernel.org Jonathan Lynch wrote: > Quoting Andy Furniss : > > >>Jonathan Lynch wrote: >> >>>This was down to the tx buffer size on the network card i was using. It >>>was an Intel 82547EI gigabit Card using the e1000 driver and operating >>>at 100mbit. The tx buffer was set to 256 which caused this huge delay. >>>The minimum the driver lets me reduce the tx buffer size using ethtool >>>is 80. By reducing the tx ring buffer to 80, the delay when there is >>>full link utilisation and a maximum queue of 10 packets was reduced from >>>30ms to 10ms. >>> >>>The 3com 3c59x vortex driver uses a tx buffer of 16. I reduced the tx to >>>16 on the e1000 driver, but the max throughput i could achieve on the >>>interface went down. >>> >>>Has anyone experimented with reducing the size of the tx buffer on this >>>card to get a good balance between delay and throughput ? >> >>Strange - I thought that as long as you are under rate for the link then >>the most htb should burst per tick is the burst size specified. >> >>That assumes one bulk class - more will make it worse. >> >>Andy. >> > > > Just noticed your reply there, havnt been very busy lately and havnt been checked LARTC in a while. > > say for example with a htb qdisc configured with a ceil of 100 Mbit (overhead 24 mpu 84 mtu 1600 > burst 12k cburst 12k quantum 1500) or a queue discipline that doesnt rate limit such as prio or red > there was a delay of 30 ms imposed when the outgoing interface was saturated and the tx ring size > was 256. when the tx ring size was reduced to 80 the delay was around 9ms. Ahh I see what you mean - reducing the buffer beyond htb, but I don't really see why you need to, rather than reducing htb rate so you only have one htb burst of packets in it at a time (that assumes you only have two classes like in your other tests - more bulk classes would be worse). > > The tx ring is a fifo structure. The NIC driver uses DMA to transmit packets from the tx ring. these > are worst case delays when The tx ring is full of maximum size FTP packets with the VoIP packet at > the end. The VoIP has to wait for all the FTP packets to be transmitted. > > When the rate was reduced to 99Mbit the maximum delay imposed is about 2ms. It seems that with the > reduced rate there is time to clear more packets from the TX ring...there are less packets in the > ring resulting in a lower delay. But the delay increases linearly. I agree from our previous discussion and tests that even with overheads added you need to back off a bit more than expected, but I assumed this was to do with either : The 8/16 byte granularity of the lookup tables. The fact that overhead was not designed into htb, but added later. Timers maybe being a bit out. Me not knowing the overhead of ethernet properly. Making tx buffer smaller is just a workaround for htb being over rate for whatever reason. > > Also a question when defining the following parameters (overhead 24 mpu 84 mtu 1600 burst 12k cburst > 12k quantum 1500) I suppose quantum should be 1514 - as you pointed out to me previously as it's eth - maybe more if you are still testing on vlans. I don't think it will make any difference in this test - I see the overhead allows for it already (I am just stating for the list as it was down during our previous discussions about overheads). mtu 1600 - It's a bit late to check now - but there is a size makes the table go from 8 to 16 and 1600 seems familiar - but 2048 comes to mind aswell :-) I would check with tc -s -d class ls and reduce it a bit if you see /16 after burst. i have them defined on all classes and on the htb qdisc itself. Is there a minimum > place where they can be specified...ie just on the htb qdisc itself, or do they have to be > specified on all I would think so, htb has a lookup table for each rate and ceil. Andy. _______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc