Re: [LARTC] HTB and PRIO qdiscs introducing extra latency when output

[Andy Furniss <andy.furniss@dsl.pipex.com>]

Jonathan Lynch wrote:
> Andy, thanks again for your help. Yes, HZ is still 1000 in 2.6.12. I
> tried your suggestions are here are the results.
> 
> ASCII diagram
> 
> (network A)  -->  (eth1) core router (eth0) --> (network C)
>                             (eth2) 
>                              ^
>                              |
>                              |
>                          (network B)
> 
> Looking at the following graphics
> 
> http://140.203.56.30/~jlynch/htb/core_router.png
> http://140.203.56.30/~jlynch/htb/voip_stream_23691.png
> 
> voip_stream_23691.png is a graph of the delay of the voice stream
> travelling from network A to network C in test 2. Notice from the core
> router graph that there is only voip traffic passing through the core
> router until time 07:55 and the delay in voip stream is 0.25 ms until
> then. After this time tcp traffic is introduced saturating the outgoing
> interface of the core router (eth0). The delay increases to a maximum of
> 2.75 ms , which is a considerable improvement on 30ms when I was using
> the ceil value of 100mbit. But there is a lot of jitter.

I suppose you could hope for a bit less jitter 12k burst is about 1ms at 
100mbit.

There is a tweak you can do for htb which may help - in 
net/sched/sch_htb.c there is a #define HYSTERESIS 1 - changing it to 0 
and recompiling kernel/the module makes things more accurate.

> 
> 
> With the ceil at 90Mbit, the outgoing bit rate of eth0 has gone from
> 98mbit to approx 90Mbit as can be seen from the core router graph for
> eth0 bytes out. Note that with the tcp traffic is all http downloads, so
> most Ethernet frames will be of maximum size, 1518 bytes, so 98mbits is
> the maximum throughput possible on a 100mbit card, taking into account
> the overheads of ethernet such as the interframe gap, preamble and start
> frame delimiter.
> 
> Im not sure how to configure some of the htb parameters. The following
> is my understanding of them and a few questions I have as well.
> 
> How exactly does the HZ value have a bearing on the ceil value ? How can
> I calculate a maximum for the ceil value ?

It's more to do with burst/cburst than ceil.

> 
> 12kb is the minimum burst size for a 100 mbit NIC with a timer
> resolution of 1ms (1000hz) and tc calculates the smallest possible burst
> when it is not specified, right ?. 

It seems not, I think hysteresis may be involved again here (but then 
one of my tcs is hacked about a bit).

You can see what htb is using as defaults by doing tc -s -d class ls ..

If I do that on similar kernels one with hysteresis 0 and one with 1 I 
see quite different values.

I chose 12k as big enough for the 90mbit test 12000*8*1000–mbit at ip 
level and it seemed like a nice multiple of 1500mtu :-)


> 
> cburst is the number of bytes that can be burst as fast as the interface
> can transmit them. It is smaller than burst can is ideally one packet
> size, right ?

Ideally 1 packet but not achievable with htb at lan speed and hz 1000, 
also AIUI the way htb does drr means with mixed packet sizes things 
aren't packet perfect even at low rates.

Saying that I use htb at low rates and can apparently get packet perfect 
with my traffic mix.

I think hfsc can do it perfectly on both counts.

>  
> quantum determines the ratio at which the classes share their parents
> bandwidth. Each class is given quantum number of bytes before serving
> the next class, right ?

Yea setting 1500 probably makes no difference for this test.

> 
> Is there any way I can limit the jitter of the VoIP traffic passing
> through the htb class ?

Try the hysteresis and/or setting the rate for interactive way higher 
than it's traffic rate.

I did a quick test to see how things were for me at 100mbit. Because my 
other pcs are slow I needed to use two as receivers for netperf.

I noticed something I didn't expect with red or the settings you use - 
one of the pcs is slower and has less memory thus smaller tcp socket 
size. Using 4 streams two to each unshaped they get about the same, 
though with txqueuelen = 1000 there are no drops (with window scalng off 
there is a difference). With red and wscale on, the red really favoured 
the fast pc - I tried a 40k bfifo so that I got drops, expecting to see 
the same, but it was still far more even than the red.

I couldn't really simulate the voip traffic in theory I should be able 
to use ping with -i < 1 sec, but using the latest inetutils you get a 
flood ping if you do that. I reported this about 18 months ago and it's 
supposedly fixed in the cvs (though I don't know if fixed means it just 
says invalid argument rather than actually does what's asked, because I 
have failed to build it so far).

So if anyone reading this has a i386 ping that -i 0.5 works on, please 
mail me the binary :-)

Andy.
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc