[LARTC] patch: HTB update for ADSL users

[LARTC] patch: HTB update for ADSL users

[Ed Wildgoose]

OK, here it is.  Near perfect bandwidth calculation for ADSL users.  
Patch iproute2 with the HTB stuff and then this:

It's still a hack (as far as I can tell) because we are patching the 
rates tables, and hence I think it is only loosly coupled with the 
actual calculation of bytes in each bucket. 

However, it works very nicely for me!  I have only been lightly testing 
with downloading stuff (hence packet dropping to slow rates), and I can 
set the max download rate to within a few kbyte/sec of the maximum and 
still keep near min ping times.  I assume that the remaining sliver of 
bandwidth is taken up passing packets which arrive in a slight cluster, 
and for packets which I later need to drop (since I'm testing on an 
incoming interface and dropped packets don't count for bandwidth used 
calcs).  However, I seem to be able to get *extremely* close to the max 
with this patch

Obviously all the numbers are hard coded, but they should be suitable 
for all ATM users.  PPoE users will need to do something different (if 
someone can supply the details then I will see what we can do to make a 
more generic patch and use module params.

Note: That this code will probably affect the policer and CBQ modules in 
the same way as HTB, however, I don't have such a setup, so I can't test 
effectiveness (or detriment...).  Feedback appreciated

Note also that rates in your scripts will now be expressed in terms of 
the ATM bandwidth, ie you put in something like the bandwidth you paid 
for, but (of course) you get roughly (bw * 48/53) passing through (this 
is normal, it's the overhead of running ATM).


--- iproute2-2.4.7.20020116/tc/tc_core.c        2000-04-16 
18:42:55.000000000 +0100
+++ iproute2/tc/tc_core.c       2004-06-18 12:20:39.912974518 +0100
@@ -59,10 +59,19 @@
                while ((mtu>>cell_log) > 255)
                        cell_log++;
        }
+
+       // HACK - UK ATM Params
+       int encaps_cell_sz = 53;
+       int encaps_cell_overhead = 5;
+       int encaps_data_sz = encaps_cell_sz - encaps_cell_overhead;
+       int proto_overhead = 10; // PPP Overhead
+
        for (i=0; i<256; i++) {
-               unsigned sz = (i<<cell_log);
-               if (sz < mpu)
-                       sz = mpu;
+               unsigned sz = ((i+1)<<cell_log)-1;
+                sz = sz + proto_overhead;
+               sz = (  (int)((sz-1)/encaps_data_sz) + 1) * encaps_cell_sz;
+//             if (sz < mpu)
+//                     sz = mpu;
                rtab[i] = tc_core_usec2tick(1000000*((double)sz/bps));
        }
        return cell_log;

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] patch: HTB update for ADSL users

[Andy Furniss]

Ed Wildgoose wrote:
> OK, here it is.  Near perfect bandwidth calculation for ADSL users.  
> Patch iproute2 with the HTB stuff and then this:
> 
> It's still a hack (as far as I can tell) because we are patching the 
> rates tables, and hence I think it is only loosly coupled with the 
> actual calculation of bytes in each bucket.
> However, it works very nicely for me!  I have only been lightly testing 
> with downloading stuff (hence packet dropping to slow rates), and I can 
> set the max download rate to within a few kbyte/sec of the maximum and 
> still keep near min ping times.  I assume that the remaining sliver of 
> bandwidth is taken up passing packets which arrive in a slight cluster, 
> and for packets which I later need to drop (since I'm testing on an 
> incoming interface and dropped packets don't count for bandwidth used 
> calcs).  However, I seem to be able to get *extremely* close to the max 
> with this patch
> 
> Obviously all the numbers are hard coded, but they should be suitable 
> for all ATM users.  PPoE users will need to do something different (if 
> someone can supply the details then I will see what we can do to make a 
> more generic patch and use module params.
> 
> Note: That this code will probably affect the policer and CBQ modules in 
> the same way as HTB, however, I don't have such a setup, so I can't test 
> effectiveness (or detriment...).  Feedback appreciated
> 
> Note also that rates in your scripts will now be expressed in terms of 
> the ATM bandwidth, ie you put in something like the bandwidth you paid 
> for, but (of course) you get roughly (bw * 48/53) passing through (this 
> is normal, it's the overhead of running ATM).
> 
> 
> --- iproute2-2.4.7.20020116/tc/tc_core.c        2000-04-16 
> 18:42:55.000000000 +0100
> +++ iproute2/tc/tc_core.c       2004-06-18 12:20:39.912974518 +0100
> @@ -59,10 +59,19 @@
>                while ((mtu>>cell_log) > 255)
>                        cell_log++;
>        }
> +
> +       // HACK - UK ATM Params
> +       int encaps_cell_sz = 53;
> +       int encaps_cell_overhead = 5;
> +       int encaps_data_sz = encaps_cell_sz - encaps_cell_overhead;
> +       int proto_overhead = 10; // PPP Overhead
> +
>        for (i=0; i<256; i++) {
> -               unsigned sz = (i<<cell_log);
> -               if (sz < mpu)
> -                       sz = mpu;
> +               unsigned sz = ((i+1)<<cell_log)-1;
> +                sz = sz + proto_overhead;
> +               sz = (  (int)((sz-1)/encaps_data_sz) + 1) * encaps_cell_sz;
> +//             if (sz < mpu)
> +//                     sz = mpu;
>                rtab[i] = tc_core_usec2tick(1000000*((double)sz/bps));
>        }
>        return cell_log;
> 
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
> 


Nice one Ed :-)

  After a bit of messing about -
Patch would't apply and I couldn't see why. Then did it by hand and had 
to move vars to top of function to get it to compile.

I set my uprate to 280kbit in TC = 286720 bit/s I am synced at 288000 - 
as you probably are, in UK, on what BT call 250/500 and isps call 
256/512. I left a bit of slack just to let buffer empty if the odd 
packet extra slips through. FWIW maxing downlink (576000 for me) will 
probably mess up - you need to be slower or you don't get to build up 
queues and will often be using your isp's buffer.

I've been maxing uplink with bt for the last couple of hours and it's 
working fine -

100 packets transmitted, 100 packets received, 0% packet loss
round-trip min/avg/max/stddev = 15.586/44.518/67.984/13.367 ms

It's just as it should be for my MTU.

When I get some time later I'll start hitting it with lots of small 
packets aswell.

Andy.

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] patch: HTB update for ADSL users

[Ed Wildgoose]

>  After a bit of messing about -
> Patch would't apply and I couldn't see why. Then did it by hand and 
> had to move vars to top of function to get it to compile.


Hmm, perhaps it got corrupted because of the change in line endings when 
I pasted it in on a windows machine?  Piece of cake to apply manually.  
If I can get some PPoE settings then I will make a more generic patch 
and stick it on a website.

Can you paste the compile errors and tell me versions of gcc please?  I 
can't see any probs with that code though?  (They will be params passed 
in later anyway)

> I set my uprate to 280kbit in TC = 286720 bit/s I am synced at 288000 
> - as you probably are, in UK, on what BT call 250/500 and isps call 
> 256/512. I left a bit of slack just to let buffer empty if the odd 
> packet extra slips through. FWIW maxing downlink (576000 for me) will 
> probably mess up - you need to be slower or you don't get to build up 
> queues and will often be using your isp's buffer.
>
> I've been maxing uplink with bt for the last couple of hours and it's 
> working fine -


Yes, excellent isn't it!  I tested download rates (bearing in mind the 
difficulty of controlling those, and could get within a sliver of full 
bandwidth before the rate rises!

I see a two stage rise in ping times.  First it stays on 30ms, then it 
rises to 60ms-90ms, then it queues like crazy.  Interesting the kind of 
three step ramp up.  I have a hunch that packets don't arrive smoothly 
and queuing occurs at the ISP end (once we get near the limit) even 
though the average rate is below the max rate...?  (ie from time to time 
you start to see two packets ahead of you instead of just one)

> 100 packets transmitted, 100 packets received, 0% packet loss
> round-trip min/avg/max/stddev = 15.586/44.518/67.984/13.367 ms
>
> It's just as it should be for my MTU.


Hmm, what's your MTU?  Those numbers look extremely low for 1500 byte 
packets (at least if you have a little downlink congestion as well?)

> When I get some time later I'll start hitting it with lots of small 
> packets aswell.


I have a 1meg downstream with 256 upstream, and I turned on bittorrent 
to try and flog the connection a bit.  Upstream was maxed out, but 
downstream was only half full.  However, ping times are 20-110.  I think 
they ought to be only 20-80 ish, and I'm trying to work out why there is 
some excess queuing (1500 ish mtu).  My QOS is based on the the 
(excellent) script from:
http://digriz.org.uk/jdg-qos-script/

Basically, HTB in both directions.  RED on the incoming stream (works 
nicely).  Outgoing classifies into 10 buckets, and ACK + ping are 
definitely going out ok in the top prio bucket, and the rest is going 
out in the prio 10 bucket....  But still these high pings...  Hmm


I would be interested to hear if anyone has a CBQ based setup and can 
tell me if that patch works for them?  Or even whether it works on the 
incoming policer properly?

It looks as though this is an adequate way to tackle the problem.  The 
alternative would be to hook into the enqueue side of the qdisk, 
calculate a new size value then and fix the code to refer to this value 
from then on.  It would be quite invasive though because it modifies 
kernel headers.  I would need someone who understands the scheduler in 
more detail to guide me as to whether it was neccessary

Ed W

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] patch: HTB update for ADSL users

[Andy Furniss]

Ed Wildgoose wrote:
> 
>>  After a bit of messing about -
>> Patch would't apply and I couldn't see why. Then did it by hand and 
>> had to move vars to top of function to get it to compile.
> 
> 
> 
> Hmm, perhaps it got corrupted because of the change in line endings when 
> I pasted it in on a windows machine?  Piece of cake to apply manually.  
> If I can get some PPoE settings then I will make a more generic patch 
> and stick it on a website.
> 
> Can you paste the compile errors and tell me versions of gcc please?  I 
> can't see any probs with that code though?  (They will be params passed 
> in later anyway)

GCC 2.95.3 - Moved the var declarations back after the

while ((mtu>>cell_log) > 255)
                        cell_log++;
        }

and again get -

tc_core.c: In function `tc_calc_rtable':
tc_core.c:65: parse error before `int'
tc_core.c:77: `proto_overhead' undeclared (first use in this function)
tc_core.c:77: (Each undeclared identifier is reported only once
tc_core.c:77: for each function it appears in.)
tc_core.c:78: `encaps_data_sz' undeclared (first use in this function)
tc_core.c:78: `encaps_cell_sz' undeclared (first use in this function)
make[1]: *** [tc_core.o] Error 1

Works if I move them to the top of the function - maybe the formatting 
is messed up (I copy n pasted).

> 
>> I set my uprate to 280kbit in TC = 286720 bit/s I am synced at 288000 
>> - as you probably are, in UK, on what BT call 250/500 and isps call 
>> 256/512. I left a bit of slack just to let buffer empty if the odd 
>> packet extra slips through. FWIW maxing downlink (576000 for me) will 
>> probably mess up - you need to be slower or you don't get to build up 
>> queues and will often be using your isp's buffer.
>>
>> I've been maxing uplink with bt for the last couple of hours and it's 
>> working fine -
> 
> 
> 
> Yes, excellent isn't it!  I tested download rates (bearing in mind the 
> difficulty of controlling those, and could get within a sliver of full 
> bandwidth before the rate rises!

I still think that you need to throttle back on down rates - when really 
full you may find that new/bursty connections mean that you loose 
control of the queue. Of course having twice as much bandwidth always 
helps.


> 
> I see a two stage rise in ping times.  First it stays on 30ms, then it 
> rises to 60ms-90ms, then it queues like crazy.  Interesting the kind of 
> three step ramp up.  I have a hunch that packets don't arrive smoothly 
> and queuing occurs at the ISP end (once we get near the limit) even 
> though the average rate is below the max rate...?  (ie from time to time 
> you start to see two packets ahead of you instead of just one)

I am only testing uploads - looking at some more pings, it does appear 
that they are not quite as random as they were - but apart from the odd 
double dequeue (in a way I think you can expect this with HTB using 
quanta/DRR rather than per byte), the max is right. I suspect this is 
nothing to do with ISP/Teleco end. I could actually do slightly better 
on my latency, but I am running at 100Hz - and I can tell with pings - 
they slowly rise then snap down by 10ms. This is nothing to do with TC, 
I normally run 500 but forgot to set it for this kernel.


> 
>> 100 packets transmitted, 100 packets received, 0% packet loss
>> round-trip min/avg/max/stddev = 15.586/44.518/67.984/13.367 ms
>>
>> It's just as it should be for my MTU.
> 
> 
> 
> Hmm, what's your MTU?  Those numbers look extremely low for 1500 byte 
> packets (at least if you have a little downlink congestion as well?)
> 

My downlink is clear, I am using 1478 MTU (so I don't waste half a cell 
per packet). Just did another hundred to my first hop -

100 packets transmitted, 100 packets received, 0% packet loss
round-trip min/avg/max/stddev = 16.661/43.673/70.841/14.634 ms

>>When I get some time later I'll start hitting it with lots of small 
>> packets aswell.

Been going for 7 hours now - I started a couple of games and 60 small 
pps didn't cause any problems.

> 
> 
> 
> I have a 1meg downstream with 256 upstream, and I turned on bittorrent 
> to try and flog the connection a bit.  Upstream was maxed out, but 
> downstream was only half full.  However, ping times are 20-110.  I think 
> they ought to be only 20-80 ish, and I'm trying to work out why there is 
> some excess queuing (1500 ish mtu).

I worked out once bitrate latency is about 44ms for 1478. Your figures 
look like you may be sending packets in pairs - or have you already done 
the tweak which fixed this for me.

set HTB_HYSTERESIS 0 in net/sched/sch_htb.c


   My QOS is based on the the
> (excellent) script from:
> http://digriz.org.uk/jdg-qos-script/

Yea it's nice - mine is a modified version.

> 
> Basically, HTB in both directions.  RED on the incoming stream (works 
> nicely).  Outgoing classifies into 10 buckets, and ACK + ping are 
> definitely going out ok in the top prio bucket, and the rest is going 
> out in the prio 10 bucket....  But still these high pings...  Hmm
> 

I did find in the worst case I could do better than RED (not much 
though) and now I do per IP for bulk so Its harder to get the right 
settings with more than one RED, that may have different bandwidths at 
any given time. I also reduced the number of classes so each could have 
a higher rate.

Andy.

> 
> I would be interested to hear if anyone has a CBQ based setup and can 
> tell me if that patch works for them?  Or even whether it works on the 
> incoming policer properly?
> 
> It looks as though this is an adequate way to tackle the problem.  The 
> alternative would be to hook into the enqueue side of the qdisk, 
> calculate a new size value then and fix the code to refer to this value 
> from then on.  It would be quite invasive though because it modifies 
> kernel headers.  I would need someone who understands the scheduler in 
> more detail to guide me as to whether it was neccessary
> 
> Ed W
> 
> 


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] patch: HTB update for ADSL users

[Ed Wildgoose]

> I still think that you need to throttle back on down rates - when 
> really full you may find that new/bursty connections mean that you 
> loose control of the queue. Of course having twice as much bandwidth 
> always helps.


Agree that it might be best in practice, but baring the initial surge 
you can now really crank everything up to the limit.  Actually, just 
starting one or two connections (wget), I didn't notice more than a 
slight blip on ping speeds on startup - for single users you could 
probably leave it quite close to the max (as long as you can tolerate 
very brief increases in latency)


> I am only testing uploads - looking at some more pings, it does appear 
> that they are not quite as random as they were - but apart from the 
> odd double dequeue (in a way I think you can expect this with HTB 
> using quanta/DRR rather than per byte), the max is right. I suspect 
> this is nothing to do with ISP/Teleco end. I could actually do 
> slightly better on my latency, but I am running at 100Hz - and I can 
> tell with pings - they slowly rise then snap down by 10ms. This is 
> nothing to do with TC, I normally run 500 but forgot to set it for 
> this kernel.


Aha, I switched off the HTB_HYSTERIS option and now my pings are nailed 
to the floor.  I'm also on a 2.6 kernel which I *think* means 1000Hz 
scheduling by default? 

> My downlink is clear, I am using 1478 MTU (so I don't waste half a 
> cell per packet). Just did another hundred to my first hop -


Hmm, quick question.  I'm using Nildram as ISP, and I can't ping at more 
than 1430 packet sizes and non-fragment bit set.  However, according to 
iptraf on the bridge (behind adsl router), I am receiving 1492 byte 
packets quite regularly...  Since the ADSL router is doing NAT, does 
this likely imply that it is doing packet reassembly before the bridge 
sees each packet?

I just changed the MTU on the router, but I don't see any difference in 
the size of packets.  Shouldn't the router MTU be enough to trigger the 
initial connection to drop to a smaller MTU?  Do I need to apply MTU 
changes to all host machines here..? (My understanding is not)


> Been going for 7 hours now - I started a couple of games and 60 small 
> pps didn't cause any problems.


Perhaps a small perl script to do something like contact your local SMTP 
server, setup the connection to start the body of the message, then feed 
the message in 1 byte pieces (perhaps a random message with crlf every 
100 chars or so).  This should give lots of mostly empty packets.  
Perhaps this can be done with the ping flood option as well...?


>> Basically, HTB in both directions.  RED on the incoming stream (works 
>> nicely).  Outgoing classifies into 10 buckets, and ACK + ping are 
>> definitely going out ok in the top prio bucket, and the rest is going 
>> out in the prio 10 bucket....  But still these high pings...  Hmm
>>
>
> I did find in the worst case I could do better than RED (not much 
> though) and now I do per IP for bulk so Its harder to get the right 
> settings with more than one RED, that may have different bandwidths at 
> any given time. I also reduced the number of classes so each could 
> have a higher rate.


Can you describe a little more about how you are improving on RED? 

I would quite like to tweak the incoming to have 3 buckets.  One for 
ACK's etc, the second for normal incoming and the last for bulk 
incoming.  The idea would be to drop the bulk in preference to the 
normal incoming.  I appreciate this won't work all that well, but it 
would be nice to say that my web browsing takes slight preference over a 
bulk ftp download. Or perhaps that incoming connections to a server 
machine take big preference over downloads to a local client machine 
(think office environment).

I would be interested on your ideas to achieve that.

Thanks Andy.

Ed W


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] patch: HTB update for ADSL users

[syrius.ml]

Ed Wildgoose <lists@wildgooses.com> writes:

> I would be interested to hear if anyone has a CBQ based setup and can
> tell me if that patch works for them?  Or even whether it works on the
> incoming policer properly?

Would that work with HFSC ?
what about PPPoE ? i didn't really understand what I have to change if
I'm using PPPoE and MTU\x1492.

TIA

-- 
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] patch: HTB update for ADSL users

[Ed Wildgoose]

syrius.ml@no-log.org wrote:

>Ed Wildgoose <lists@wildgooses.com> writes:
>
>  
>
>>I would be interested to hear if anyone has a CBQ based setup and can
>>tell me if that patch works for them?  Or even whether it works on the
>>incoming policer properly?
>>    
>>
>
>Would that work with HFSC ?
>  
>

To be honest, I can't see these bits of code being used in HFSC, so I 
doubt it.  However, if performance isn't an issue, it would be fairly 
easy to patch the enqueue function in any of these schedulers to just 
lie about the size of the packet.

>what about PPPoE ? i didn't really understand what I have to change if
>I'm using PPPoE and MTU\x1492.
>  
>

I'm not really sure how PPPoE is working?  Is it as simple as wrapping 
IP in IP? 

The patch is trivial though, the first interesting line is this one:
    + sz = sz + proto_overhead;

Here we just lie about the size of the packet and add the overhead PPP 
causes (this is per original packet)

The next interesting line is this:
    +  sz = (  (int)((sz-1)/encaps_data_sz) + 1) * encaps_cell_sz;

All we are doing is dividing by the size of the packet that our data is 
encapsulated in, (perhaps in your case, encaps_data_sz should be 1492 
?).  Then we multiply up by the actual size of the packet when it 
crosses the link "encaps_cell_size".  Perhaps in your case this should 
be 1492 + 40 = 1532 ?

The other lines were just trying to make it look a little neater.

I will look up how PPPoE works and suggest some new values

Ed W

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] patch: HTB update for ADSL users

[Jason Boxman]

On Friday 18 June 2004 07:45, Ed Wildgoose wrote:
> OK, here it is.  Near perfect bandwidth calculation for ADSL users.
> Patch iproute2 with the HTB stuff and then this:

Did I miss this the first time around?  There's a patch to sch_htb.c as well, 
or not?  If there is, I haven't applied it which would explain why the 
original tc patch did not have any effect for me.

I'm eager to try this out as soon as I have some time.

Thanks for the patch!

<snip>

-- 

Jason Boxman
Perl Programmer / *NIX Systems Administrator
Shimberg Center for Affordable Housing | University of Florida
http://edseek.com/ - Linux and FOSS stuff

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] patch: HTB update for ADSL users

[Andy Furniss]

Ed Wildgoose wrote:
> 
>> I still think that you need to throttle back on down rates - when 
>> really full you may find that new/bursty connections mean that you 
>> loose control of the queue. Of course having twice as much bandwidth 
>> always helps.
> 
> 
> 
> Agree that it might be best in practice, but baring the initial surge 
> you can now really crank everything up to the limit.  Actually, just 
> starting one or two connections (wget), I didn't notice more than a 
> slight blip on ping speeds on startup - for single users you could 
> probably leave it quite close to the max (as long as you can tolerate 
> very brief increases in latency)

Latency is what I care about most - 300ms blip is really annoying for 
me, I suppose what I would see as 300 you would see as 150 which I could 
live with.

If I ran close on ingress it would hurt too much - but two long 
connections would be OK. It's new connections and many bursty 
connections that are the problem.

If you let things get really bad and you start having ISP drop packets 
then you will loose fairness/control aswell.

Can you test how big your ISP/BT buffer is - Mine is 600ms I wonder if 
yours is half or the same.

> 
> 
>> I am only testing uploads - looking at some more pings, it does appear 
>> that they are not quite as random as they were - but apart from the 
>> odd double dequeue (in a way I think you can expect this with HTB 
>> using quanta/DRR rather than per byte), the max is right. I suspect 
>> this is nothing to do with ISP/Teleco end. I could actually do 
>> slightly better on my latency, but I am running at 100Hz - and I can 
>> tell with pings - they slowly rise then snap down by 10ms. This is 
>> nothing to do with TC, I normally run 500 but forgot to set it for 
>> this kernel.
> 
> 
> 
> Aha, I switched off the HTB_HYSTERIS option and now my pings are nailed 
> to the floor.  I'm also on a 2.6 kernel which I *think* means 1000Hz 
> scheduling by default?
> 
>> My downlink is clear, I am using 1478 MTU (so I don't waste half a 
>> cell per packet). Just did another hundred to my first hop -
> 
> 
> 
> Hmm, quick question.  I'm using Nildram as ISP, and I can't ping at more 
> than 1430 packet sizes and non-fragment bit set. 

Hmm ISTR reading of an ISP, maybe nildram, that enforced tweaks - 
thought they chose 1478 but alot of people like 1430.

  However, according to
> iptraf on the bridge (behind adsl router), I am receiving 1492 byte 
> packets quite regularly...  Since the ADSL router is doing NAT, does 
> this likely imply that it is doing packet reassembly before the bridge 
> sees each packet?

I know that for me, if I set ppp0 to say 576 and eth0 and LAN machines 
to 1500, then linux will frag and reconstruct ICMP - but let larger 
packets through.


> 
> I just changed the MTU on the router, but I don't see any difference in 
> the size of packets.  Shouldn't the router MTU be enough to trigger the 
> initial connection to drop to a smaller MTU?

No - well it won't for me. You can use mss clamping to make tcp comply, 
but I change LAN machines.


>> I did find in the worst case I could do better than RED (not much 
>> though) and now I do per IP for bulk so Its harder to get the right 
>> settings with more than one RED, that may have different bandwidths at 
>> any given time. I also reduced the number of classes so each could 
>> have a higher rate.
> 
> 
> 
> Can you describe a little more about how you are improving on RED?
> I would quite like to tweak the incoming to have 3 buckets.  One for 
> ACK's etc, the second for normal incoming and the last for bulk 
> incoming.  The idea would be to drop the bulk in preference to the 
> normal incoming.  I appreciate this won't work all that well, but it 
> would be nice to say that my web browsing takes slight preference over a 
> bulk ftp download. Or perhaps that incoming connections to a server 
> machine take big preference over downloads to a local client machine 
> (think office environment).
> 
> I would be interested on your ideas to achieve that.

I'll answer this later - it's hard to get on a PC in my house on a 
Sunday - "Not my turn" apparently :-)

Andy.

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] patch: HTB update for ADSL users

[Ed Wildgoose]

Jason Boxman wrote:

>On Friday 18 June 2004 07:45, Ed Wildgoose wrote:
>  
>
>>OK, here it is.  Near perfect bandwidth calculation for ADSL users.
>>Patch iproute2 with the HTB stuff and then this:
>>    
>>
>
>Did I miss this the first time around?  There's a patch to sch_htb.c as well, 
>or not?  If there is, I haven't applied it which would explain why the 
>original tc patch did not have any effect for me.
>  
>

Nope, it's just a case of patching the TC prog.  Make sure it is all up 
to date with the latest HTB patches first though (not the HTB 2 stuff 
for example).

It works by changing the "rate" calculations which are then later used 
in the HTB, policer and CBQ modules.  I haven't tested the effect in 
anything other than HTB though, and it seems to be pretty reliable.  I 
*think* it might be a hack because now "rates" are measured correctly, 
but bytes per bucket are measured in the original units.  However, it 
seems to be very accurate, so I think it is "good enough"

Basically, just lie slightly about the rates that the device can achieve 
for a given packet size...  It's actually just a one line patch done in 
a slightly verbose way (you might need to patch manually if line endings 
got mangled in my email - easy though)

Ed W
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] patch: HTB update for ADSL users

[Ed Wildgoose]

syrius.ml@no-log.org wrote:

>Ed Wildgoose <lists@wildgooses.com> writes:
>
>  
>
>>I would be interested to hear if anyone has a CBQ based setup and can
>>tell me if that patch works for them?  Or even whether it works on the
>>incoming policer properly?
>>    
>>
>
>Would that work with HFSC ?
>what about PPPoE ? i didn't really understand what I have to change if
>I'm using PPPoE and MTU\x1492.
>  
>

After further investigation I think you need the patch a little more 
like this for PPPoE and the params you gave (Andy do you think this 
looks OK?)  Please let me know if this works for you.

(As near as I can tell, there is no PPP overhead(?), and you then 
basically encapsulate your packets in IP packets of max size 1492, with 
an 8 byte overhead - Perhaps Andy can correct me as to whether there is 
still a PPP overhead?)

--- iproute2-2.4.7.20020116/tc/tc_core.c        2000-04-16 
18:42:55.000000000 +0100
+++ iproute2/tc/tc_core.c       2004-06-18 12:20:39.912974518 +0100
@@ -59,10 +59,19 @@
               while ((mtu>>cell_log) > 255)
                       cell_log++;
       }
+
+       // HACK - UK ATM Params
+       int encaps_cell_sz = 1492;
+       int encaps_cell_overhead = 8;
+       int encaps_data_sz = encaps_cell_sz - encaps_cell_overhead;
+       int proto_overhead = 0; // PPP Overhead
+
       for (i=0; i<256; i++) {
-               unsigned sz = (i<<cell_log);
-               if (sz < mpu)
-                       sz = mpu;
+               unsigned sz = ((i+1)<<cell_log)-1;
+                sz = sz + proto_overhead;
+               sz = (  (int)((sz-1)/encaps_data_sz) + 1) * encaps_cell_sz;
+//             if (sz < mpu)
+//                     sz = mpu;
               rtab[i] = tc_core_usec2tick(1000000*((double)sz/bps));
       }
       return cell_log;
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] patch: HTB update for ADSL users

[Ed Wildgoose]

> After further investigation I think you need the patch a little more 
> like this for PPPoE and the params you gave (Andy do you think this 
> looks OK?)  Please let me know if this works for you.


Duh, scratch that.  I think the original patch will work for PPPoE (give 
or take).

Can someone help work out how the heck you calculate the real usage of a 
packet over PPPoE?  Reading a bit further I get the impression that you 
have an 8 byte overhead for PPPoE, then it still travels down an ATM 
circuit (just like PPPoA)?  Can anyone confirm or deny?  I have seen 
some reports that you actually have two ethernet frames wrapped inside 
each other (which is very inefficient), but I can't see which is right?

If it's the former, then I think the original patch will work well 
enough for PPPoE people, it's 2 bytes out, but that shouldn't really 
show up.

Can someone please try and report back

Thanks

Ed W
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] patch: HTB update for ADSL users

[Jason Boxman]

On Sunday 20 June 2004 09:16, Ed Wildgoose wrote:
> Jason Boxman wrote:
> >On Friday 18 June 2004 07:45, Ed Wildgoose wrote:
> >>OK, here it is.  Near perfect bandwidth calculation for ADSL users.
> >>Patch iproute2 with the HTB stuff and then this:
> >
> >Did I miss this the first time around?  There's a patch to sch_htb.c as
> > well, or not?  If there is, I haven't applied it which would explain why
> > the original tc patch did not have any effect for me.
>
> Nope, it's just a case of patching the TC prog.  Make sure it is all up
> to date with the latest HTB patches first though (not the HTB 2 stuff
> for example).

Interesting.

> It works by changing the "rate" calculations which are then later used
> in the HTB, policer and CBQ modules.  I haven't tested the effect in
> anything other than HTB though, and it seems to be pretty reliable.  I
> *think* it might be a hack because now "rates" are measured correctly,
> but bytes per bucket are measured in the original units.  However, it
> seems to be very accurate, so I think it is "good enough"

So does it just let you be honest about your rated speed, or is there an added 
performance bonus from being able to specify your true ATM rate?

<snip>

-- 

Jason Boxman
Perl Programmer / *NIX Systems Administrator
Shimberg Center for Affordable Housing | University of Florida
http://edseek.com/ - Linux and FOSS stuff

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] patch: HTB update for ADSL users

[Ed Wildgoose]

>So does it just let you be honest about your rated speed, or is there an added 
>performance bonus from being able to specify your true ATM rate?
>  
>

The idea is that your ethernet packets get broken up and encapsulated in 
53 byte ATM packets.  Each ATM packet is fixed size and has a 5 byte 
header = 48 bytes of data.

Straight off that means that the 512Kbit bandwidth you bought is only 
good for 512/53 * 48 = 463 Kbit or 58 Kbytes/s.  However, unless your 
packets are exactly a multiple of 48 then you will be wasting bandwidth 
big time (think about your MTU).  Consider small ack packets which will 
take up two cells, but the second cell is basically empty (bandwidth 
slashed in half).  Unless all your packets are the same size, this is 
why the current mutliply by some fixed constant approach is not working 
perfectly with ADSL.

So the patch simply works out how many cells you will need and 
multiplies up by the real size of those cells to work out bw used. 

It's also more conservative than before.  Previously it was working out 
an effective rate for 8 byte packets, and using that rate for 8-15 byte 
packets.  Now it works out the rate for 15 byte packets and uses *that* 
for 8-15 byte packets.

It's probably not perfect, but it's extremely accurate for me.  I can 
bump up the bandwidth on my interfaces to basically 99% and for stable 
streams there is no queuing.  P2P still seems to cause so many new 
incoming connections that you need to throttle down the incoming to 
leave some space for an unexpected incoming rush - but for most users I 
think it will work nicely

Ed W
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] patch: HTB update for ADSL users

[Damion de Soto]

> After further investigation I think you need the patch a little more 
> like this for PPPoE and the params you gave (Andy do you think this 
> looks OK?)  Please let me know if this works for you.
> 
> (As near as I can tell, there is no PPP overhead(?), and you then 
> basically encapsulate your packets in IP packets of max size 1492, with 
> an 8 byte overhead - Perhaps Andy can correct me as to whether there is 
> still a PPP overhead?)
I wonder if there's any easy way we can get the ATM/PPPoE support enabled
at runtime.
I guess a tc option would be needed to enable ADSL (atm) numbers,
but the HTB qdisc should be able to detect PPP packets for PPPoE, right ?
at least egress on any pppX interface.

Are you using this patch just for inbound shaping on pppX, or does it still make 
sense to use it on an IMQ device, or on a LAN-side ethX device to do the equivalent 
of ingress shaping ?

regards

-- 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Damion de Soto - Software Engineer  email:     damion@snapgear.com
SnapGear - A CyberGuard Company ---    ph:         +61 7 3435 2809
  | Custom Embedded Solutions          fax:         +61 7 3891 3630
  | and Security Appliances            web: http://www.snapgear.com
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  ---  Free Embedded Linux Distro at   http://www.snapgear.org  ---
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] patch: HTB update for ADSL users

[Jason Boxman]

[-- Attachment #1: Type: text/plain, Size: 2420 bytes --]

On Monday 21 June 2004 06:04, Ed Wildgoose wrote:
> >So does it just let you be honest about your rated speed, or is there an
> > added performance bonus from being able to specify your true ATM rate?
>
<snip>
>
> So the patch simply works out how many cells you will need and
> multiplies up by the real size of those cells to work out bw used.

Interesting.

> It's also more conservative than before.  Previously it was working out
> an effective rate for 8 byte packets, and using that rate for 8-15 byte
> packets.  Now it works out the rate for 15 byte packets and uses *that*
> for 8-15 byte packets.

10 byte PPP overhead + 5 byte ATM overhead?

> It's probably not perfect, but it's extremely accurate for me.  I can
> bump up the bandwidth on my interfaces to basically 99% and for stable
> streams there is no queuing.  P2P still seems to cause so many new
> incoming connections that you need to throttle down the incoming to
> leave some space for an unexpected incoming rush - but for most users I
> think it will work nicely

I think we have a winner.

I had to futz with the patch some, as Andy did.  Once I got it to compile, I 
kicked my rate up too 256.  That didn't work, but 224 seems stable.  I 
started to `scp` a copy of linux-2.6.3.tar.bz2 since it's all I could find on 
short notice to a remote host.  Both machines have host CPUs fast enough for 
3DES encryption not be the limiting factor.

I ping out around 15ms - 70ms to my gateway.  That's about what I was seeing 
at 160kbit before.

I'm running PPPoEoATM here.  I don't know what my actual PPPoE overhead is, 
but I guess 10 bytes is reasonably close enough.  PPPoE is handled by my 
Westell Wirespeed, which doesn't provide any useful cell information.  At the 
moment I cannot easily obtain a cell count to determine my actual PPP 
overhead.

Perhaps I need to idle everything and do one of those 'speed tests' to see 
what my actual upstream is.  Could be it's really around 224, since I'm not 
guaranteed 256 by my ISP anyway.

218 packets transmitted, 218 packets received, 0% packet loss
round-trip min/avg/max = 11.5/41.3/70.6 ms

I have attached a unified `diff` with both the HTB `tc` patch and Ed 
Wildgoose's new PPPoA overhead patch as a unified patch.

-- 

Jason Boxman
Perl Programmer / *NIX Systems Administrator
Shimberg Center for Affordable Housing | University of Florida
http://edseek.com/ - Linux and FOSS stuff


[-- Attachment #2: tc_htb_pppoa_unified.diff --]
[-- Type: text/x-diff, Size: 4984 bytes --]

diff -uNr iproute-20010824/tc/q_htb.c iproute-20010824-mod/tc/q_htb.c
--- iproute-20010824/tc/q_htb.c	2004-06-22 01:09:00.000000000 -0400
+++ iproute-20010824-mod/tc/q_htb.c	2004-06-22 00:27:44.000000000 -0400
@@ -34,10 +34,14 @@
 		" default  minor id of class to which unclassified packets are sent {0}\n"
 		" r2q      DRR quantums are computed as rate in Bps/r2q {10}\n"
 		" debug    string of 16 numbers each 0-3 {0}\n\n"
-		"... class add ... htb rate R1 burst B1 [prio P] [slot S] [pslot PS]\n"
+		"... class add ... htb rate R1 [burst B1] [mpu B] [overhead O]\n"
+		"                      [prio P] [slot S] [pslot PS]\n"
 		"                      [ceil R2] [cburst B2] [mtu MTU] [quantum Q]\n"
 		" rate     rate allocated to this class (class can still borrow)\n"
 		" burst    max bytes burst which can be accumulated during idle period {computed}\n"
+		" mpu      minimum packet size used in rate computations\n"
+		" overhead per-packet size overhead used in rate computations\n"
+
 		" ceil     definite upper class rate (no borrows) {rate}\n"
 		" cburst   burst but for ceil {computed}\n"
 		" mtu      max packet size we create rate map for {1600}\n"
@@ -102,7 +106,9 @@
 	struct tc_htb_opt opt;
 	__u32 rtab[256],ctab[256];
 	unsigned buffer=0,cbuffer=0;
-	int cell_log=-1,ccell_log = -1,mtu;
+	int cell_log=-1,ccell_log = -1;
+	unsigned mtu, mpu;
+	unsigned char mpu8 = 0, overhead = 0;
 	struct rtattr *tail;
 
 	memset(&opt, 0, sizeof(opt)); mtu = 1600; /* eth packet len */
@@ -119,6 +125,16 @@
 			if (get_u32(&mtu, *argv, 10)) {
 				explain1("mtu"); return -1;
 			}
+		} else if (matches(*argv, "mpu") == 0) {
+			NEXT_ARG();
+			if (get_u8(&mpu8, *argv, 10)) {
+				explain1("mpu"); return -1;
+			}
+		} else if (matches(*argv, "overhead") == 0) {
+			NEXT_ARG();
+			if (get_u8(&overhead, *argv, 10)) {
+				explain1("overhead"); return -1;
+			}
 		} else if (matches(*argv, "quantum") == 0) {
 			NEXT_ARG();
 			if (get_u32(&opt.quantum, *argv, 10)) {
@@ -190,14 +206,18 @@
 	if (!buffer) buffer = opt.rate.rate / HZ + mtu;
 	if (!cbuffer) cbuffer = opt.ceil.rate / HZ + mtu;
 
-	if ((cell_log = tc_calc_rtable(opt.rate.rate, rtab, cell_log, mtu, 0)) < 0) {
+/* encode overhead and mpu, 8 bits each, into lower 16 bits */
+	mpu = (unsigned)mpu8 | (unsigned)overhead << 8;
+	opt.ceil.mpu = mpu; opt.rate.mpu = mpu;
+
+	if ((cell_log = tc_calc_rtable(opt.rate.rate, rtab, cell_log, mtu, mpu)) < 0) {
 		fprintf(stderr, "htb: failed to calculate rate table.\n");
 		return -1;
 	}
 	opt.buffer = tc_calc_xmittime(opt.rate.rate, buffer);
 	opt.rate.cell_log = cell_log;
 	
-	if ((ccell_log = tc_calc_rtable(opt.ceil.rate, ctab, cell_log, mtu, 0)) < 0) {
+	if ((ccell_log = tc_calc_rtable(opt.ceil.rate, ctab, cell_log, mtu, mpu)) < 0) {
 		fprintf(stderr, "htb: failed to calculate ceil rate table.\n");
 		return -1;
 	}
@@ -221,6 +241,7 @@
 	double buffer,cbuffer;
 	SPRINT_BUF(b1);
 	SPRINT_BUF(b2);
+	SPRINT_BUF(b3);
 
 	if (opt == NULL)
 		return 0;
@@ -243,10 +264,16 @@
 	    fprintf(f, "ceil %s ", sprint_rate(hopt->ceil.rate, b1));
 	    cbuffer = ((double)hopt->ceil.rate*tc_core_tick2usec(hopt->cbuffer))/1000000;
 	    if (show_details) {
-		fprintf(f, "burst %s/%u mpu %s ", sprint_size(buffer, b1),
-			1<<hopt->rate.cell_log, sprint_size(hopt->rate.mpu, b2));
-		fprintf(f, "cburst %s/%u mpu %s ", sprint_size(cbuffer, b1),
-			1<<hopt->ceil.cell_log, sprint_size(hopt->ceil.mpu, b2));
+		fprintf(f, "burst %s/%u mpu %s overhead %s ",
+			sprint_size(buffer, b1),
+			1<<hopt->rate.cell_log,
+			sprint_size(hopt->rate.mpu&0xFF, b2),
+			sprint_size((hopt->rate.mpu>>8)&0xFF, b3));
+		fprintf(f, "cburst %s/%u mpu %s overhead %s ",
+			sprint_size(cbuffer, b1),
+			1<<hopt->ceil.cell_log,
+			sprint_size(hopt->ceil.mpu&0xFF, b2),
+			sprint_size((hopt->ceil.mpu>>8)&0xFF, b3));
 		fprintf(f, "level %d ", (int)hopt->level);
 	    } else {
 		fprintf(f, "burst %s ", sprint_size(buffer, b1));
diff -uNr iproute-20010824/tc/tc_core.c iproute-20010824-mod/tc/tc_core.c
--- iproute-20010824/tc/tc_core.c	2000-04-16 13:42:55.000000000 -0400
+++ iproute-20010824-mod/tc/tc_core.c	2004-06-22 00:41:52.000000000 -0400
@@ -50,6 +50,9 @@
 		   unsigned mpu)
 {
 	int i;
+	unsigned overhead = (mpu >> 8) & 0xFF;
+	mpu = mpu & 0xFF;
+	unsigned sz;
 
 	if (mtu == 0)
 		mtu = 2047;
@@ -59,10 +62,22 @@
 		while ((mtu>>cell_log) > 255)
 			cell_log++;
 	}
+
+	// HACK - UK ATM Params
+	int encaps_cell_sz = 53;
+	int encaps_cell_overhead = 5;
+	int encaps_data_sz = encaps_cell_sz - encaps_cell_overhead;
+	int proto_overhead = 10; // PPP Overhead
+
 	for (i=0; i<256; i++) {
-		unsigned sz = (i<<cell_log);
-		if (sz < mpu)
-			sz = mpu;
+		if (overhead)
+			sz += overhead;
+		sz = ((i+1)<<cell_log)-1;
+//		unsigned sz = ((i+1)<<cell_log)-1;
+		sz = sz + proto_overhead;
+		sz = (  (int)((sz-1)/encaps_data_sz) + 1) * encaps_cell_sz;
+//		if (sz < mpu)
+//			sz = mpu;
 		rtab[i] = tc_core_usec2tick(1000000*((double)sz/bps));
 	}
 	return cell_log;

Re: [LARTC] patch: HTB update for ADSL users

[Ed Wildgoose]

Damion de Soto wrote:

>> After further investigation I think you need the patch a little more 
>> like this for PPPoE and the params you gave (Andy do you think this 
>> looks OK?)  Please let me know if this works for you.
>>
>> (As near as I can tell, there is no PPP overhead(?), and you then 
>> basically encapsulate your packets in IP packets of max size 1492, 
>> with an 8 byte overhead - Perhaps Andy can correct me as to whether 
>> there is still a PPP overhead?)
>
> I wonder if there's any easy way we can get the ATM/PPPoE support enabled
> at runtime.


Yeah, I was thinking of adding a module param in the final version, 
something like "encaps PPPoA" or similar

> I guess a tc option would be needed to enable ADSL (atm) numbers,
> but the HTB qdisc should be able to detect PPP packets for PPPoE, right ?
> at least egress on any pppX interface.


Now that's a very interesting question.  The size is passed in as the 
size of the packet that the kernel sees (including headers).  We mangle 
that to get the size sent over the ATM link.  However, I have ethernet 
on both sides in my setup.  I wonder what the kernel sees if one 
interface is PPP?  Is it already encapsulated in PPP before QOS gets it, 
or not?

> Are you using this patch just for inbound shaping on pppX, or does it 
> still make sense to use it on an IMQ device, or on a LAN-side ethX 
> device to do the equivalent of ingress shaping ?


Use it for both.

Ed W
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] patch: HTB update for ADSL users

[Ed Wildgoose]

>I'm running PPPoEoATM here.  I don't know what my actual PPPoE overhead is, 
>but I guess 10 bytes is reasonably close enough.  PPPoE is handled by my 
>Westell Wirespeed, which doesn't provide any useful cell information.  At the 
>moment I cannot easily obtain a cell count to determine my actual PPP 
>overhead.
>  
>

Try bumping the protocol_overhead up to 16 for PPPoE (from 10).  You 
should also make sure that your MTU is lower than that required by your 
PPPoE provider or else you will get ethernet packet fragmentation and I 
doubt we want to extend the patch to cover those situations anyway.

The patch should actually have most benefit when you are doing transfers 
with smaller packets.  I think with larger constant streams like the one 
you tested, there will be little difference between bumping up the 
interface speed with the patch or leaving it all as it was (at the end 
of the day we are mostly just shifting the calculation of interface 
speed somewhere else).

>Perhaps I need to idle everything and do one of those 'speed tests' to see 
>what my actual upstream is.  Could be it's really around 224, since I'm not 
>guaranteed 256 by my ISP anyway.
>  
>

Your upstream will be 256Kbits of ATM bandwidth.   This consists of 53 
byte packets with 48 bytes of data.  So you already only have 256 * 
48/53 of real bandwidth.  We then have to take off PPP headers and PPPoE 
headers.

We are obviously still a few bytes out with this patch or else you 
should be able to crank up the speed to 250 ish and still see your ping 
speeds stay low.  I will investigate further

Ed W

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] patch: HTB update for ADSL users

[Jason Boxman]

On Tuesday 22 June 2004 06:00, Ed Wildgoose wrote:
> >I'm running PPPoEoATM here.  I don't know what my actual PPPoE overhead
> > is, but I guess 10 bytes is reasonably close enough.  PPPoE is handled by
> > my Westell Wirespeed, which doesn't provide any useful cell information. 
> > At the moment I cannot easily obtain a cell count to determine my actual
> > PPP overhead.
>
> Try bumping the protocol_overhead up to 16 for PPPoE (from 10).  You
> should also make sure that your MTU is lower than that required by your
> PPPoE provider or else you will get ethernet packet fragmentation and I
> doubt we want to extend the patch to cover those situations anyway.

The largest MTU I could use is 1492.  I have the Westell Wirespeed handling 
PPPoE, so I speak through eth0 locally.  Until recently that was fine, but 
now I need to use 1492 instead of 1500 on eth0 due to strange SSH hangs that 
haven't happened in a year at 1500 with the same configuration.  It's odd.  
All other machines are using 1500 without incident.

> The patch should actually have most benefit when you are doing transfers
> with smaller packets.  I think with larger constant streams like the one
> you tested, there will be little difference between bumping up the
> interface speed with the patch or leaving it all as it was (at the end
> of the day we are mostly just shifting the calculation of interface
> speed somewhere else).

Without the patch, if I set my rate to 256 * 0.8, I die.  The connection is 
not completely unusable, but gaming is extremely laggy and Web traffic is 
noticeably laggy, although pages still load with about 2s (versus a few 
hundred ms without the patch at 160kbit).  With the patch I can set it to 
224, so there's obviously a large improvement even with mostly large TCP 
packets going out doing a bulk `scp` copy.

> >Perhaps I need to idle everything and do one of those 'speed tests' to see
> >what my actual upstream is.  Could be it's really around 224, since I'm
> > not guaranteed 256 by my ISP anyway.
>
> Your upstream will be 256Kbits of ATM bandwidth.   This consists of 53
> byte packets with 48 bytes of data.  So you already only have 256 *
> 48/53 of real bandwidth.  We then have to take off PPP headers and PPPoE
> headers.

That's the maximum promised speed from my ISP.  In reality, of course, line 
conditions might result in my true speed not being that high, before 
accounting for overhead and things.

> We are obviously still a few bytes out with this patch or else you
> should be able to crank up the speed to 250 ish and still see your ping
> speeds stay low.  I will investigate further

Okay, if you're sure it's not just my line having a true upstream less than 
the consumer rated speed it was assigned. ;)

> Ed W
>

-- 

Jason Boxman
Perl Programmer / *NIX Systems Administrator
Shimberg Center for Affordable Housing | University of Florida
http://edseek.com/ - Linux and FOSS stuff

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] patch: HTB update for ADSL users

[Jason Boxman]

On Tuesday 22 June 2004 06:00, Ed Wildgoose wrote:
<snip>
> Your upstream will be 256Kbits of ATM bandwidth.   This consists of 53
> byte packets with 48 bytes of data.  So you already only have 256 *
> 48/53 of real bandwidth.  We then have to take off PPP headers and PPPoE
> headers.
<snip>
> We are obviously still a few bytes out with this patch or else you
> should be able to crank up the speed to 250 ish and still see your ping
> speeds stay low.  I will investigate further

I guess I'm still missing the point.  Isn't my _effective_ bandwidth still 
entirely dependent on how big my PPP encapsulated packets are just prior to 
being broken into streams of 53 byte ATM cells?

With the patch does HTB behave appropriately if it gets hit with mostly ~ 100 
byte packets where you'd encounter around 30% of your ATM rate lost due to 
the overhead of null byte padding?

> Ed W

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

RE: [LARTC] patch: HTB update for ADSL users

[ThE LinuX_KiD]

hi !

this patch can work good with last tc release ?
which performance haves with rp-pppoe ?

bests
andres.




-> -----Mensaje original-----
-> De: lartc-admin@mailman.ds9a.nl [mailto:lartc-admin@mailman.ds9a.nl]En
-> nombre de Jason Boxman
-> Enviado el: Martes, 22 de Junio de 2004 02:17 p.m.
-> Para: lartc@mailman.ds9a.nl
-> Asunto: Re: [LARTC] patch: HTB update for ADSL users
->
->
-> On Tuesday 22 June 2004 06:00, Ed Wildgoose wrote:
-> > >I'm running PPPoEoATM here.  I don't know what my actual
-> PPPoE overhead
-> > > is, but I guess 10 bytes is reasonably close enough.  PPPoE
-> is handled by
-> > > my Westell Wirespeed, which doesn't provide any useful cell
-> information.
-> > > At the moment I cannot easily obtain a cell count to
-> determine my actual
-> > > PPP overhead.
-> >
-> > Try bumping the protocol_overhead up to 16 for PPPoE (from 10).  You
-> > should also make sure that your MTU is lower than that required by your
-> > PPPoE provider or else you will get ethernet packet fragmentation and I
-> > doubt we want to extend the patch to cover those situations anyway.
->
-> The largest MTU I could use is 1492.  I have the Westell
-> Wirespeed handling
-> PPPoE, so I speak through eth0 locally.  Until recently that was
-> fine, but
-> now I need to use 1492 instead of 1500 on eth0 due to strange
-> SSH hangs that
-> haven't happened in a year at 1500 with the same configuration.
-> It's odd.
-> All other machines are using 1500 without incident.
->
-> > The patch should actually have most benefit when you are doing
-> transfers
-> > with smaller packets.  I think with larger constant streams
-> like the one
-> > you tested, there will be little difference between bumping up the
-> > interface speed with the patch or leaving it all as it was (at the end
-> > of the day we are mostly just shifting the calculation of interface
-> > speed somewhere else).
->
-> Without the patch, if I set my rate to 256 * 0.8, I die.  The
-> connection is
-> not completely unusable, but gaming is extremely laggy and Web
-> traffic is
-> noticeably laggy, although pages still load with about 2s (versus a few
-> hundred ms without the patch at 160kbit).  With the patch I can
-> set it to
-> 224, so there's obviously a large improvement even with mostly large TCP
-> packets going out doing a bulk `scp` copy.
->
-> > >Perhaps I need to idle everything and do one of those 'speed
-> tests' to see
-> > >what my actual upstream is.  Could be it's really around 224,
-> since I'm
-> > > not guaranteed 256 by my ISP anyway.
-> >
-> > Your upstream will be 256Kbits of ATM bandwidth.   This consists of 53
-> > byte packets with 48 bytes of data.  So you already only have 256 *
-> > 48/53 of real bandwidth.  We then have to take off PPP headers
-> and PPPoE
-> > headers.
->
-> That's the maximum promised speed from my ISP.  In reality, of
-> course, line
-> conditions might result in my true speed not being that high, before
-> accounting for overhead and things.
->
-> > We are obviously still a few bytes out with this patch or else you
-> > should be able to crank up the speed to 250 ish and still see your ping
-> > speeds stay low.  I will investigate further
->
-> Okay, if you're sure it's not just my line having a true
-> upstream less than
-> the consumer rated speed it was assigned. ;)
->
-> > Ed W
-> >
->
-> --
->
-> Jason Boxman
-> Perl Programmer / *NIX Systems Administrator
-> Shimberg Center for Affordable Housing | University of Florida
-> http://edseek.com/ - Linux and FOSS stuff
->
-> _______________________________________________
-> LARTC mailing list / LARTC@mailman.ds9a.nl
-> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] patch: HTB update for ADSL users

[Ed Wildgoose]

ThE LinuX_KiD wrote:

>hi !
>
>this patch can work good with last tc release ?
>which performance haves with rp-pppoe ?
>  
>

I guess so.

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/