From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andy Furniss Date: Thu, 28 Jul 2005 21:49:54 +0000 Subject: Re: [LARTC] HTB and PRIO qdiscs introducing extra latency when output Message-Id: <42E95302.2020303@dsl.pipex.com> List-Id: References: <1122460110.8454.11.camel@pgala.it.nuigalway.ie> In-Reply-To: <1122460110.8454.11.camel@pgala.it.nuigalway.ie> MIME-Version: 1.0 Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: quoted-printable To: lartc@vger.kernel.org Jonathan Lynch wrote: > Andy, thanks again for your help. Yes, HZ is still 1000 in 2.6.12. I > tried your suggestions are here are the results. >=20 > ASCII diagram >=20 > (network A) --> (eth1) core router (eth0) --> (network C) > (eth2)=20 > ^ > | > | > (network B) >=20 > Looking at the following graphics >=20 > http://140.203.56.30/~jlynch/htb/core_router.png > http://140.203.56.30/~jlynch/htb/voip_stream_23691.png >=20 > voip_stream_23691.png is a graph of the delay of the voice stream > travelling from network A to network C in test 2. Notice from the core > router graph that there is only voip traffic passing through the core > router until time 07:55 and the delay in voip stream is 0.25 ms until > then. After this time tcp traffic is introduced saturating the outgoing > interface of the core router (eth0). The delay increases to a maximum of > 2.75 ms , which is a considerable improvement on 30ms when I was using > the ceil value of 100mbit. But there is a lot of jitter. I suppose you could hope for a bit less jitter 12k burst is about 1ms at=20 100mbit. There is a tweak you can do for htb which may help - in=20 net/sched/sch_htb.c there is a #define HYSTERESIS 1 - changing it to 0=20 and recompiling kernel/the module makes things more accurate. >=20 >=20 > With the ceil at 90Mbit, the outgoing bit rate of eth0 has gone from > 98mbit to approx 90Mbit as can be seen from the core router graph for > eth0 bytes out. Note that with the tcp traffic is all http downloads, so > most Ethernet frames will be of maximum size, 1518 bytes, so 98mbits is > the maximum throughput possible on a 100mbit card, taking into account > the overheads of ethernet such as the interframe gap, preamble and start > frame delimiter. >=20 > Im not sure how to configure some of the htb parameters. The following > is my understanding of them and a few questions I have as well. >=20 > How exactly does the HZ value have a bearing on the ceil value ? How can > I calculate a maximum for the ceil value ? It's more to do with burst/cburst than ceil. >=20 > 12kb is the minimum burst size for a 100 mbit NIC with a timer > resolution of 1ms (1000hz) and tc calculates the smallest possible burst > when it is not specified, right ?.=20 It seems not, I think hysteresis may be involved again here (but then=20 one of my tcs is hacked about a bit). You can see what htb is using as defaults by doing tc -s -d class ls .. If I do that on similar kernels one with hysteresis 0 and one with 1 I=20 see quite different values. I chose 12k as big enough for the 90mbit test 12000*8*1000=96mbit at ip=20 level and it seemed like a nice multiple of 1500mtu :-) >=20 > cburst is the number of bytes that can be burst as fast as the interface > can transmit them. It is smaller than burst can is ideally one packet > size, right ? Ideally 1 packet but not achievable with htb at lan speed and hz 1000,=20 also AIUI the way htb does drr means with mixed packet sizes things=20 aren't packet perfect even at low rates. Saying that I use htb at low rates and can apparently get packet perfect=20 with my traffic mix. I think hfsc can do it perfectly on both counts. > =20 > quantum determines the ratio at which the classes share their parents > bandwidth. Each class is given quantum number of bytes before serving > the next class, right ? Yea setting 1500 probably makes no difference for this test. >=20 > Is there any way I can limit the jitter of the VoIP traffic passing > through the htb class ? Try the hysteresis and/or setting the rate for interactive way higher=20 than it's traffic rate. I did a quick test to see how things were for me at 100mbit. Because my=20 other pcs are slow I needed to use two as receivers for netperf. I noticed something I didn't expect with red or the settings you use -=20 one of the pcs is slower and has less memory thus smaller tcp socket=20 size. Using 4 streams two to each unshaped they get about the same,=20 though with txqueuelen =3D 1000 there are no drops (with window scalng off = there is a difference). With red and wscale on, the red really favoured=20 the fast pc - I tried a 40k bfifo so that I got drops, expecting to see=20 the same, but it was still far more even than the red. I couldn't really simulate the voip traffic in theory I should be able=20 to use ping with -i < 1 sec, but using the latest inetutils you get a=20 flood ping if you do that. I reported this about 18 months ago and it's=20 supposedly fixed in the cvs (though I don't know if fixed means it just=20 says invalid argument rather than actually does what's asked, because I=20 have failed to build it so far). So if anyone reading this has a i386 ping that -i 0.5 works on, please=20 mail me the binary :-) Andy. _______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc