[LARTC] Extremely inaccurate results using either TBF or HTB

[LARTC] Extremely inaccurate results using either TBF or HTB

[Dan Farino]

[-- Attachment #1: Type: text/plain, Size: 1460 bytes --]

Hi everyone,

 

I am getting extremely inaccurate results from my setup:

- RedHat 8.0 on a Compaq ProLiant 1600, dual PII/450, Intel Dual 100 NIC

- I've tried both SMP and non-SMP kernels.

- I'm using the updated tc from the HTB home page.

- I'm using the HTB that comes with the RH8 kernel.

 

Here's what's happening. If I, for instance:

     tc qdisc add dev eth2 root tbf rate 1.9mbit buffer 20Kb/8 limit
15Kb

 

Then my download rate goes at 360,000 bytes/s, which, by my
calculations, is 2.77mbit. I am using Win2k perfmon to test the speed.
The line is flat and the speed is very consistent (consistently *wrong*,
unfortunately...)

 

Immediately after, I do:

     tc qdisc change dev eth2 root tbf rate 1.8mbit buffer 20Kb/8 limit
15Kb

 

My download speed is then 243,000 bytes/s, or 1.85mbit. Again, very
flat, consistent line.

 

I have played with the parameters on both TBF and HTB in many different
configurations. I always seem to have a large jump between 1.8-1.9. I
can't find any number between those two that will actually produce a
2mbit limit. I know 1.85 is close to 2.0, but is this the expected
margin of error? (Then again, 2.77 isn't close to 1.9 at all, so I hope
not!)

 

Has anyone else experienced anything like this?

 

Thank you very much for any help you can provide!

 

Dan Farino

Sr. System Engineer

Stamps.com, Inc.

Santa Monica, CA

 


[-- Attachment #2: Type: text/html, Size: 6342 bytes --]

Re: [LARTC] Extremely inaccurate results using either TBF or HTB

[Stef Coene]

>
> I am getting extremely inaccurate results from my setup:
>
> - RedHat 8.0 on a Compaq ProLiant 1600, dual PII/450, Intel Dual 100 NIC
> - I've tried both SMP and non-SMP kernels.
> - I'm using the updated tc from the HTB home page.
> - I'm using the HTB that comes with the RH8 kernel.
>
> Here's what's happening. If I, for instance:
>
>      tc qdisc add dev eth2 root tbf rate 1.9mbit buffer 20Kb/8 limit
> 15Kb
>
> Then my download rate goes at 360,000 bytes/s, which, by my
> calculations, is 2.77mbit. I am using Win2k perfmon to test the speed.
> The line is flat and the speed is very consistent (consistently *wrong*,
> unfortunately...)
>
> Immediately after, I do:
>
>      tc qdisc change dev eth2 root tbf rate 1.8mbit buffer 20Kb/8 limit
> 15Kb
>
> My download speed is then 243,000 bytes/s, or 1.85mbit. Again, very
> flat, consistent line.
>
>
> I have played with the parameters on both TBF and HTB in many different
> configurations. I always seem to have a large jump between 1.8-1.9. I
> can't find any number between those two that will actually produce a
> 2mbit limit. I know 1.85 is close to 2.0, but is this the expected
> margin of error? (Then again, 2.77 isn't close to 1.9 at all, so I hope
> not!)
>
> Has anyone else experienced anything like this?
I did some tests with htb and tbf :
http://www.docum.org/stef.coene/qos/tests/tbf/
http://www.docum.org/stef.coene/qos/tests/htb/index.html
And the accuracy was very good.

I did the tests on a debian system with a custom kernel and tc command.

How did you measure the throughput?  I use the iptables counters so I know 
exactly what passes the box.  If you download a file from the internet with a 
ftp client, there can be overhead due to retransmission, control packets, 
....  So the speed you measure with the ftp client may be lower then the real 
connection speed.


Stef

-- 

stef.coene@docum.org
 "Using Linux as bandwidth manager"
     http://www.docum.org/
     #lartc @ irc.oftc.net

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

RE: [LARTC] Extremely inaccurate results using either TBF or HTB

[Dan Farino]

Hi Stef,

Thanks for the reply! However, I think the gap is way too large to be
explained by overhead, etc.

Here are the results from some other tests. I started with:
tc qdisc add dev eth2 root tbf rate 1.9mbit buffer 20Kb/8 limit 15Kb

Here are the average transfer rates measured over approx 20 seconds
(again according to PerfMon) as I change the "1.9mbit" to other values
while performing a large download:

Mbit	Kb/s
-----	------
2.5	360
2.0	360
1.9	360
1.8	240
1.5	240
1.4	240
1.3	180
1.2	180

The lines really are that flat. There is nothing resembling the graphs
on your site. Mine has the low-res, stair-step effect that you can see
by graphing the above numbers. 

I can't seem to find any "mbit" number that will actually produce a cap
at exactly 2mbit.

I will try a both different type NIC and Debian at some point today.

Thanks for your help!

Dan Farino
Sr. System Engineer
Stamps.com, Inc.
Santa Monica, CA


-----Original Message-----
From: Stef Coene [mailto:stef.coene@docum.org] 
Sent: Tuesday, November 12, 2002 8:09 AM
To: Dan Farino; lartc@mailman.ds9a.nl
Subject: Re: [LARTC] Extremely inaccurate results using either TBF or
HTB

>
> I am getting extremely inaccurate results from my setup:
>
> - RedHat 8.0 on a Compaq ProLiant 1600, dual PII/450, Intel Dual 100
NIC
> - I've tried both SMP and non-SMP kernels.
> - I'm using the updated tc from the HTB home page.
> - I'm using the HTB that comes with the RH8 kernel.
>
> Here's what's happening. If I, for instance:
>
>      tc qdisc add dev eth2 root tbf rate 1.9mbit buffer 20Kb/8 limit
> 15Kb
>
> Then my download rate goes at 360,000 bytes/s, which, by my
> calculations, is 2.77mbit. I am using Win2k perfmon to test the speed.
> The line is flat and the speed is very consistent (consistently
*wrong*,
> unfortunately...)
>
> Immediately after, I do:
>
>      tc qdisc change dev eth2 root tbf rate 1.8mbit buffer 20Kb/8
limit
> 15Kb
>
> My download speed is then 243,000 bytes/s, or 1.85mbit. Again, very
> flat, consistent line.
>
>
> I have played with the parameters on both TBF and HTB in many
different
> configurations. I always seem to have a large jump between 1.8-1.9. I
> can't find any number between those two that will actually produce a
> 2mbit limit. I know 1.85 is close to 2.0, but is this the expected
> margin of error? (Then again, 2.77 isn't close to 1.9 at all, so I
hope
> not!)
>
> Has anyone else experienced anything like this?
I did some tests with htb and tbf :
http://www.docum.org/stef.coene/qos/tests/tbf/
http://www.docum.org/stef.coene/qos/tests/htb/index.html
And the accuracy was very good.

I did the tests on a debian system with a custom kernel and tc command.

How did you measure the throughput?  I use the iptables counters so I
know 
exactly what passes the box.  If you download a file from the internet
with a 
ftp client, there can be overhead due to retransmission, control
packets, 
....  So the speed you measure with the ftp client may be lower then the
real 
connection speed.


Stef

-- 

stef.coene@docum.org
 "Using Linux as bandwidth manager"
     http://www.docum.org/
     #lartc @ irc.oftc.net

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Extremely inaccurate results using either TBF or HTB

[Stef Coene]

On Tuesday 12 November 2002 17:58, Dan Farino wrote:
> Hi Stef,
>
> Thanks for the reply! However, I think the gap is way too large to be
> explained by overhead, etc.
>
> Here are the results from some other tests. I started with:
> tc qdisc add dev eth2 root tbf rate 1.9mbit buffer 20Kb/8 limit 15Kb
>
> Here are the average transfer rates measured over approx 20 seconds
> (again according to PerfMon) as I change the "1.9mbit" to other values
> while performing a large download:
>
> Mbit	Kb/s
> -----	------
> 2.5	360
> 2.0	360
> 1.9	360
> 1.8	240
> 1.5	240
> 1.4	240
> 1.3	180
> 1.2	180
>
> The lines really are that flat. There is nothing resembling the graphs
> on your site. Mine has the low-res, stair-step effect that you can see
> by graphing the above numbers.
>
> I can't seem to find any "mbit" number that will actually produce a cap
> at exactly 2mbit.
>
> I will try a both different type NIC and Debian at some point today.
What NIC are you using ?

> Thanks for your help!
No problem.  I can send you my scripts that I use the record the bandwidth 
usage.  Some of them can be found on www.docum.org.  Basically, I create a 
iptables chain for each traffic stream I want to monitor.  I record the 
counters each second and calculates the bandwdith.  
I have some scripts that executes a htb script, starts the needed flows, 
calculates the bandwidth, if the bandwidth is stable : record the bandwidth, 
restart the proces with a different setting.
After some time, you have it for each rate so you can create a nice graph.


Stef

-- 

stef.coene@docum.org
 "Using Linux as bandwidth manager"
     http://www.docum.org/
     #lartc @ irc.oftc.net

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Extremely inaccurate results using either TBF or HTB

[sufcrusher]

[-- Attachment #1: Type: text/plain, Size: 1943 bytes --]

This worked miracles for me:

change /usr/src/linux/include/net/pkt_sched.h
PSCHED_CLOCK_SOURCE to PSCHED_CPU if you have a cpu with timestamp
counter (TSC) that will give you Mhz timer granularity.

(don't forget to recompile of course)


Jannes Faber
  ----- Original Message ----- 
  From: Dan Farino 
  To: lartc@mailman.ds9a.nl 
  Sent: Tuesday, November 12, 2002 9:57 AM
  Subject: [LARTC] Extremely inaccurate results using either TBF or HTB


  Hi everyone,



  I am getting extremely inaccurate results from my setup:

  - RedHat 8.0 on a Compaq ProLiant 1600, dual PII/450, Intel Dual 100 NIC

  - I've tried both SMP and non-SMP kernels.

  - I'm using the updated tc from the HTB home page.

  - I'm using the HTB that comes with the RH8 kernel.



  Here's what's happening. If I, for instance:

       tc qdisc add dev eth2 root tbf rate 1.9mbit buffer 20Kb/8 limit 15Kb



  Then my download rate goes at 360,000 bytes/s, which, by my calculations, is 2.77mbit. I am using Win2k perfmon to test the speed. The line is flat and the speed is very consistent (consistently *wrong*, unfortunately.)



  Immediately after, I do:

       tc qdisc change dev eth2 root tbf rate 1.8mbit buffer 20Kb/8 limit 15Kb



  My download speed is then 243,000 bytes/s, or 1.85mbit. Again, very flat, consistent line.



  I have played with the parameters on both TBF and HTB in many different configurations. I always seem to have a large jump between 1.8-1.9. I can't find any number between those two that will actually produce a 2mbit limit. I know 1.85 is close to 2.0, but is this the expected margin of error? (Then again, 2.77 isn't close to 1.9 at all, so I hope not!)



  Has anyone else experienced anything like this?



  Thank you very much for any help you can provide!



  Dan Farino

  Sr. System Engineer

  Stamps.com, Inc.

  Santa Monica, CA




[-- Attachment #2: Type: text/html, Size: 8004 bytes --]

RE: [LARTC] Extremely inaccurate results using either TBF or HTB

[Dan Farino]

[-- Attachment #1: Type: text/plain, Size: 2456 bytes --]

Jannes,

 

That worked great! Everything is dead-on accurate now. 

 

Thank you very much for your help!

 

Dan Farino

Sr. System Engineer

Stamps.com, Inc.

Santa Monica, CA

 

-----Original Message-----
From: sufcrusher [mailto:sufcrusher@zonnet.nl] 
Sent: Tuesday, November 12, 2002 3:11 PM
To: Dan Farino; lartc@mailman.ds9a.nl
Subject: Re: [LARTC] Extremely inaccurate results using either TBF or
HTB

 

This worked miracles for me:

 

change /usr/src/linux/include/net/pkt_sched.h
PSCHED_CLOCK_SOURCE to PSCHED_CPU if you have a cpu with timestamp
counter (TSC) that will give you Mhz timer granularity.

 

(don't forget to recompile of course)

 

Jannes Faber

	----- Original Message ----- 

	From: Dan Farino <mailto:DFarino@Stamps.com>  

	To: lartc@mailman.ds9a.nl 

	Sent: Tuesday, November 12, 2002 9:57 AM

	Subject: [LARTC] Extremely inaccurate results using either TBF
or HTB

	 

	Hi everyone,

	 

	I am getting extremely inaccurate results from my setup:

	- RedHat 8.0 on a Compaq ProLiant 1600, dual PII/450, Intel Dual
100 NIC

	- I've tried both SMP and non-SMP kernels.

	- I'm using the updated tc from the HTB home page.

	- I'm using the HTB that comes with the RH8 kernel.

	 

	Here's what's happening. If I, for instance:

	     tc qdisc add dev eth2 root tbf rate 1.9mbit buffer 20Kb/8
limit 15Kb

	 

	Then my download rate goes at 360,000 bytes/s, which, by my
calculations, is 2.77mbit. I am using Win2k perfmon to test the speed.
The line is flat and the speed is very consistent (consistently *wrong*,
unfortunately...)

	 

	Immediately after, I do:

	     tc qdisc change dev eth2 root tbf rate 1.8mbit buffer
20Kb/8 limit 15Kb

	 

	My download speed is then 243,000 bytes/s, or 1.85mbit. Again,
very flat, consistent line.

	 

	I have played with the parameters on both TBF and HTB in many
different configurations. I always seem to have a large jump between
1.8-1.9. I can't find any number between those two that will actually
produce a 2mbit limit. I know 1.85 is close to 2.0, but is this the
expected margin of error? (Then again, 2.77 isn't close to 1.9 at all,
so I hope not!)

	 

	Has anyone else experienced anything like this?

	 

	Thank you very much for any help you can provide!

	 

	Dan Farino

	Sr. System Engineer

	Stamps.com, Inc.

	Santa Monica, CA

	 


[-- Attachment #2: Type: text/html, Size: 12850 bytes --]

RE: [LARTC] Extremely inaccurate results using either TBF or HTB

[Dan Farino]

Stef,

I solved my problem by changing PSCHED_CLOCK_SOURCE to PSCHED_CPU in
/usr/src/linux/include/net/pkt_sched.h

My graph is now a nice smooth diagonal line as I increase the bandwidth
in increments as I did before.

Thanks for you help on this one. You site has been a tremendous help in
getting me up and running.

Take care,

Dan Farino
Sr. System Engineer
Stamps.com, Inc.
Santa Monica, CA


-----Original Message-----
From: Stef Coene [mailto:stef.coene@docum.org] 
Sent: Tuesday, November 12, 2002 10:07 AM
To: Dan Farino; lartc@mailman.ds9a.nl
Subject: Re: [LARTC] Extremely inaccurate results using either TBF or
HTB

On Tuesday 12 November 2002 17:58, Dan Farino wrote:
> Hi Stef,
>
> Thanks for the reply! However, I think the gap is way too large to be
> explained by overhead, etc.
>
> Here are the results from some other tests. I started with:
> tc qdisc add dev eth2 root tbf rate 1.9mbit buffer 20Kb/8 limit 15Kb
>
> Here are the average transfer rates measured over approx 20 seconds
> (again according to PerfMon) as I change the "1.9mbit" to other values
> while performing a large download:
>
> Mbit	Kb/s
> -----	------
> 2.5	360
> 2.0	360
> 1.9	360
> 1.8	240
> 1.5	240
> 1.4	240
> 1.3	180
> 1.2	180
>
> The lines really are that flat. There is nothing resembling the graphs
> on your site. Mine has the low-res, stair-step effect that you can see
> by graphing the above numbers.
>
> I can't seem to find any "mbit" number that will actually produce a
cap
> at exactly 2mbit.
>
> I will try a both different type NIC and Debian at some point today.
What NIC are you using ?

> Thanks for your help!
No problem.  I can send you my scripts that I use the record the
bandwidth 
usage.  Some of them can be found on www.docum.org.  Basically, I create
a 
iptables chain for each traffic stream I want to monitor.  I record the 
counters each second and calculates the bandwdith.  
I have some scripts that executes a htb script, starts the needed flows,

calculates the bandwidth, if the bandwidth is stable : record the
bandwidth, 
restart the proces with a different setting.
After some time, you have it for each rate so you can create a nice
graph.


Stef

-- 

stef.coene@docum.org
 "Using Linux as bandwidth manager"
     http://www.docum.org/
     #lartc @ irc.oftc.net

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Extremely inaccurate results using either TBF or HTB

[Stef Coene]

On Wednesday 13 November 2002 00:11, sufcrusher wrote:
> This worked miracles for me:
>
> change /usr/src/linux/include/net/pkt_sched.h
> PSCHED_CLOCK_SOURCE to PSCHED_CPU if you have a cpu with timestamp
> counter (TSC) that will give you Mhz timer granularity.
Is this only needed if you have multiple CPU's ?  Or is your setup a 1 CPU 
setup?

Stef

-- 

stef.coene@docum.org
 "Using Linux as bandwidth manager"
     http://www.docum.org/
     #lartc @ irc.oftc.net

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/