Re: HTB accuracy for high speed

[Jarek Poplawski <jarkao2@gmail.com>]

On Thu, May 21, 2009 at 12:46:16AM +0200, Eric Dumazet wrote:
> Vladimir Ivashchenko a écrit :
> >>>> I guess you should send some logs. Your previous report seem to show
> >>> Can you give some hints on which logs you would like to see?
> >> Similarly to Antonio's: ifconfigs and tc -s for qdiscs and classes at
> >> the beginning and at the end of testing.
> > 
> > Ok, it seems that I finally found what is causing my HTB on 2.6.29 not
> > to reach full throughput: dst hashing on sfq with high divisor value.
> > 
> > 2.6.21 esfq divisor 13 depth 4096 hash dst - 680 mbps
> > 2.6.29 sfq WITHOUT "flow hash keys dst ... " (default sfq) - 680 mbps
> > 2.6.29 sfq + "flow hash keys dst divisor 64" filter - 680 mbps
> > 2.6.29 sfq + "flow hash keys dst divisor 256" filter - 660 mbps
> > 2.6.29 sfq + "flow hash keys dst divisor 2048" filters - 460 mbps
> > 
> > I'm using high sfq hash divisor in order to decrease the number of
> > collisions, there are several thousands of hosts behind each of the
> > classes. 
> > 
> > Any ideas why increasing the sfq divisor size results in drop of
> > throughput ?
> > 
> > Attached are diagnostics gathered in case of divisor 2048.
> > 
> 
> 
> But... it appears sfq currently supports a fixed divisor of 1024
> 
> net/sched/sch_sfq.c
> 
>  IMPLEMENTATION:
>  This implementation limits maximal queue length to 128;
>  maximal mtu to 2^15-1; number of hash buckets to 1024.
>  The only goal of this restrictions was that all data
>  fit into one 4K page :-). Struct sfq_sched_data is
>  organized in anti-cache manner: all the data for a bucket
>  are scattered over different locations. This is not good,
>  but it allowed me to put it into 4K.
> 
>  It is easy to increase these values, but not in flight.  */
> 
> #define SFQ_DEPTH   128
> #define SFQ_HASH_DIVISOR    1024
> 
> 
> Apparently Corey Hickey 2007 work on SFQ was not merged.
> 
> http://kerneltrap.org/mailarchive/linux-netdev/2007/9/28/325048

Yes, sfq has its design limits, and as a matter of fact, because of
max length (127) it should be treated as a toy or "personal" qdisc.

I don't know why more of esfq wasn't merged, anyway similar
functionality could be achieved in current kernels with sch_drr +
cls_flow, alas not enough documented. Here is some hint:
http://markmail.org/message/h24627xkrxyqxn4k

Jarek P.

PS: I guess, you wasn't very consistent if your main problem was
exceeding or not reaching htb rate, and there is quite a difference.

Vladimir Ivashchenko wrote, On 05/08/2009 10:46 PM:

> Exporting HZ=1000 doesn't help. However, even if I recompile the kernel
> to 1000 Hz and the burst is calculated correctly, for some reason HTB on
> 2.6.29 is still worse at rate control than 2.6.21.
> 
> With 2.6.21, ceil of 775 mbits, burst 99425b -> actual rate 825 mbits.
> With 2.6.29, same ceil/burst -> actual rate 890 mbits.
...

Vladimir Ivashchenko wrote, On 05/17/2009 10:29 PM:

> Hi Antonio,
> 
> FYI, these are exactly the same problems I get in real life.
> Check the later posts in "bond + tc regression" thread.
...