Re: [LARTC] Question about how TC enforces bandwidth limiting

[Vadtec <vadtec@vadtec.net>]

Martin A. Brown wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Greetings again Vadtec,
>
>  : After another two days of trying to get this to work like I think 
>  : it should, I've still hit a brick wall.
>
> This can be frustrating, I know.
>
> [ snip ]  (I admit, I didn't look at the rules very closely, but 
>            fundamentally the rules themselves look good.)
>
> You do use an ingress filter with a policer.  Since the policer 
> applies equally to all inbound traffic flows, it doesn't really do 
> you any good here.
>
>  : Regardless of the values I use for any of the rates, I still have 
>  : the same problem. I do not classify ANY traffic with iptables for 
>  : this set of tc filters, so any traffic that is generated is 
>  : solely shaped by tc in this case.
>  : 
>  : The problem I have is this, whenever I allow any torrent client 
>  : to use above 20k of outgoing bandwidth, *everything* becomes 
>  : laggy for some reason. Most notable is SSH. While I have no 
>  : accurate way to time the lag, as near as I can tell it lags about 
>  : 2 seconds. When I press a key on my keyboard it takes ~2 seconds 
>  : to show up in the SSH client. Or, if I enter lets say "ls" and 
>  : press enter, it takes roughly 2 seconds for the ls output to 
>  : reach me, and while its being displayed, its choppy appearing (as 
>  : in it comes in chunks rather than a nice stream).
>  : 
>  : I see absolutely no reason for this to be happening. Why should 
>  : anything lag when I'm using more outgoing bandwidth? Why would 
>  : more outgoing bandwidth cause a slow down on incoming bandwidth. 
>  : Or, why would more outgoing bandwidth slow down the filters/tc? 
>  : I've verified that this happens on more than just my modem. I've 
>  : used two different routers and a cable modem. All suffer from the 
>  : same symptoms.
>
> You are neglecting one important consideration, and that is the 
> download traffic.  You may well be shaping the upstream traffic, but 
> the queues transmitting downstream are also full.
>
> So, once again--I'd suggest that you consider what it takes for your 
> "ls" keystrokes and then the output to make it back to you.
>
>   * you press a key ("l"), your ssh client transmits a packet
>   * this packet goes across your congested link (probably 
>     significantly delayed, because there's congestion here)
>   * the packet arrives on the ultimate destination
>   * the sshd on the remote side receives the packet, passes it to
>     the application layer (blah blah, bash, fork, exec, ls)
>   * the sshd receives data from the application layer and transmits 
>     a packet
>   * your traffic control structures take this inbound ssh packet and 
>     give it its dedicated slot (however you have built your queues)
>   * packet returns quickly to your system
>
> So, your shaping is great, but you aren't shaping all of the 
> paths through which your application flow must pass.
>   
Ok, so answer me this: Does tc use the same general queue for egress and 
ingress traffic?
Or is it a matter of tc just using a default ingress filter that is 
filling for some reason?

Either way, it still doesn't explain why everything starts to lag as 
soon as one applications outgoing
bandwidth climbs anything above 20.5k (I tested to see when the break 
was). If the ingress queue
works just fine at 20k, why does it not work just fine at >21k? That's 
my central issue. If I can figure
that out, I can most likely filter differently to insure proper 
performance is maintained.
>  : For the record, my router PC is built as such:
>  : CentOS 5 64bit
>  : AMD Sempron 2800+ (64bit, 1.6Ghz)
>  : 2GB of DDR
>  : 2GB of swap
>  : 
>  : As you can see, this PC is more than capable of acting as my router.
>
> And then some.  Very reasonable looking box.
>
> Good luck,
>
> - -Martin
>
> - -- 
> Martin A. Brown
> http://linux-ip.net/
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2 (GNU/Linux)
> Comment: pgf-0.72 (http://linux-ip.net/sw/pine-gpg-filter/)
>
> iD8DBQFG32pfHEoZD1iZ+YcRAp8DAKDDb7eO6/cNZp+lLg8tyO07QffzuQCfcTA3
> DnHkDHC/Ea09qNsuEBhi+vs> =VBjg
> -----END PGP SIGNATURE-----
>
>   

Thanks for the input Martin. I really do appreciate the time you have 
taken with trying to help me.

In an effort to make this work better, I am currently working on setting 
up both egress and ingress qdiscs.
(You're e-mail prompted me to take a break. :P) For the sake of 
simplicity, I am going to setup my ingress
qdiscs (practically) the same way as I do my egress filters (even though 
they may not get used, I will still
set them up for the time being just in case I need them). Then, using fw 
marking, I'm going to filter using
iptables on both egress and ingress (though to be honest I really have 
no idea how I'm going to do the ingress
filtering and get it back to the right place, though I assume it will 
take care of it self, just like egress).

Maybe once I do this I will start to see improvements. You can expect 
that in a few days I will post again when
I have exhausted all my (misguided or otherwise) ideas about how to make 
this work.

Thanks again,


Vadtec
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc