From mboxrd@z Thu Jan 1 00:00:00 1970 From: devik Date: Mon, 07 Jul 2003 18:38:04 +0000 Subject: Re: [LARTC] HTB doesn't respect rate values MIME-Version: 1 Content-Type: multipart/mixed; boundary="8323328-908300613-1057603084=:18108" Message-Id: List-Id: References: In-Reply-To: To: lartc@vger.kernel.org This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. Send mail to mime@docserver.cac.washington.edu for more info. --8323328-908300613-1057603084=:18108 Content-Type: TEXT/PLAIN; charset=ISO-8859-2 Content-Transfer-Encoding: QUOTED-PRINTABLE Well. The right way for your case would be to limit single subqueue in SFQ. See line 24 of attached patch - and try patch itself. devik On Mon, 7 Jul 2003, Sergiusz Brzezi=F1ski wrote: > U=BFytkownik devik napisa=B3: > >>I did pfifo with limit 10 and HTB started to work. I noticed drops and > >>rate was OK. Sometimes (for 10-40 seconds) but seldom it worked bad (1:= 2 > >>got less, than it should) but there wasn't drops during this time. I > >>tried this also with 12kbit at it was similar. > >> > >>If I good understand, there is something (TCP stack or what) whitch > >>works BEVORE htb and this makes some connections slower or faster, so > >>HTB has later nothing to do. > >> > >>The question for me is: how can I set it (the mechanism bevore HTB) to > >>give HTB full control over the bandwitch? I don't wont to use pfifo (no= t > >>everywhere). I would like to use sfq for some classes. > > > > > > you can try to play with /proc/sys/net/ipv4/tcp_{w,r,}mem. If wmem is > > smaller than space in qdisc (15kB for 10 pfifo, approx 200kB for SFQ) > > then TCP will back off before it fill the qdisc... > > Also with sfq it should work - it is like pfifo with limit 128 - you > > might need to increase wmem. > > Note that the problem occur only when you have qdisc on the same machin= e > > as sending app. > > - I did this: > # echo "4096 2048000 5120000" > /proc/sys/net/ipv4/tcp_wmem > than I tried if in /proc/sys/net/ipv4/tcp_wmem there are really the new > values (i set so high values because i wanted really be sure that the > amount of memory will be OK :) - defaults where: 4096 16384 131072) > > Well, it helped in 80%. Why only in 80? I repeated my test with SFQ and: > > - it worked better than bevore, there where long time periods > (15-20sec.) with right rate-values > - but class whitch should become 98kbit still got sometimes only 38kbit; > it happen seldom and short but it was a fact! > - what was very strange: there were still no drops and no overlimits > (!!!) in stats ("tc -s class show dev eth0"); in a test with "pfifo > limit 10" I could see: when there were drops - the rate was OK, when > there were no drops - the rate got lower than it should; with SFQ there > were NO DROPS AT ALL so the question is: who (or what) really makes the > whole work? It doesn't look like HTB-work.(have I right?) > > Should I also do something with "/proc/sys/net/ipv4/tcp_mem"? > Is the min value in tcp_wmem (4096) OK? > Do you have some more ideas? > > I would make some experiments but I'm really not familiar with this > theme. So the only thing I can do is to ask YOU or someone else from the > group. > > Sergiusz > > >>U=BFytkownik devik napisa=B3: > >> > >>>Interestingly from what I see HTB didn't come into play. > >>>All drop and overlimits counters are zero. It seems that > >>>www server haven't managed to send more. > >>>Please try to add pfifo with limit 10 under both classes. > >>>Because you are sending from the same computer, your > >>>TCP stack uses send queue management which counts packets > >>>in qdisc and backs off. It MIGHT cause problems ... > >>> > >>>------------------------------- > >>> Martin Devera aka devik > >>>Linux kernel QoS/HTB maintainer > >>> http://luxik.cdi.cz/~devik/ > >>> > >>>On Sat, 5 Jul 2003, Sergiusz Brzezi=F1ski wrote: > >>> > >>> > >>> > >>>>Thak you for your hints! > >>>> > >>>> > >>>>>1) 6kbit is really too small it should be at least 10 .. > >>>> > >>>>I tried with 12, 20 and even with 30kbit for 1:3 > >>>> > >>>>I noticed, that it work for some seconds (or 1-2 minutes) but than th= e > >>>>1:3 class gets more then it should get :(. > >>>> > >>>> > >>>> > >>>>>2) it should workeven with 6k: > >>>>> - look at stats (tc -s class show dev eth0) before and > >>>>> after the test - you are interested in drops. Also try > >>>>> it during the test to look whether queues are build up. > >>>>> > >>>> > >>>>I made a test with settings: > >>>>--------------------------------- > >>>> > >>>>tc qdisc del root dev eth0 > >>>>tc qdisc add dev eth0 root handle 1:0 htb default 3 > >>>> > >>>>tc class add dev eth0 parent 1:0 classid 1:1 htb rate 128kbit ceil > >>>>128kbit burst 20kbit > >>>> > >>>>tc class add dev eth0 parent 1:1 classid 1:2 htb rate 98kbit ceil > >>>>128kbit quantum 4900 burst 20kbit > >>>> > >>>>tc class add dev eth0 parent 1:1 classid 1:3 htb rate 30kbit ceil > >>>>128kbit quantum 1500 > >>>> > >>>>tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip spo= rt > >>>>80 0xffff flowid 1:2 > >>>> > >>>>Bevore test: (reseted htb) > >>>>-------------------------------- > >>>># tc -s class show dev eth0 > >>>> > >>>>class htb 1:1 root rate 128Kbit ceil 128Kbit burst 2559b cburst 1762b > >>>> Sent 0 bytes 0 pkts (dropped 0, overlimits 0) > >>>> lended: 0 borrowed: 0 giants: 0 > >>>> tokens: 244140 ctokens: 168131 > >>>> > >>>>class htb 1:2 parent 1:1 prio 0 rate 98Kbit ceil 128Kbit burst 2559b > >>>>cburst 1762b > >>>> Sent 0 bytes 0 pkts (dropped 0, overlimits 0) > >>>> lended: 0 borrowed: 0 giants: 0 > >>>> tokens: 318876 ctokens: 168131 > >>>> > >>>>class htb 1:3 parent 1:1 prio 0 rate 30Kbit ceil 128Kbit burst 1637b > >>>>cburst 1762b > >>>> Sent 0 bytes 0 pkts (dropped 0, overlimits 0) > >>>> lended: 0 borrowed: 0 giants: 0 > >>>> tokens: 666503 ctokens: 168131 > >>>> > >>>>After test: > >>>>------------ > >>>>class htb 1:1 root rate 128Kbit ceil 128Kbit burst 2559b cburst 1762b > >>>> Sent 5843869 bytes 4715 pkts (dropped 0, overlimits 0) > >>>> rate 15427bps 12pps > >>>> lended: 1461 borrowed: 0 giants: 0 > >>>> tokens: -21142 ctokens: -97151 > >>>> > >>>>class htb 1:2 parent 1:1 prio 0 rate 98Kbit ceil 128Kbit burst 2559b > >>>>cburst 1762b > >>>> Sent 2735702 bytes 1811 pkts (dropped 0, overlimits 0) > >>>> rate 6397bps 4pps > >>>> lended: 1802 borrowed: 9 giants: 0 > >>>> tokens: 312898 ctokens: 163555 > >>>> > >>>>class htb 1:3 parent 1:1 prio 0 rate 30Kbit ceil 128Kbit burst 1637b > >>>>cburst 1762b > >>>> Sent 3108167 bytes 2904 pkts (dropped 0, overlimits 0) > >>>> rate 9488bps 8pps > >>>> lended: 1452 borrowed: 1452 giants: 0 > >>>> tokens: -561135 ctokens: -97151 > >>>> > >>>>Description of the test: > >>>>------------------------ > >>>>On the beginning it was everything OK, after 1 min, 1:2 lost his 98kb= it. > >>>>Than he got sometimes his 98kbit again and sometimes he got even 30kb= it. > >>>> > >>>> > >>>>1. Can I do something more to find out what happen? > >>>>2. What does mean: "queues are build up" ? > >>>> > >>>>Sergiusz > >>>> > >> > >> > >> > > > > > > > > --8323328-908300613-1057603084=:18108 Content-Type: TEXT/PLAIN; charset=US-ASCII; name="sfq.diff" Content-Transfer-Encoding: BASE64 Content-ID: Content-Description: Content-Disposition: attachment; filename="sfq.diff" ZGlmZiAtcnViQiAvdXNyL3NyYy9saW51eC0yLjIuMTYvbmV0L3NjaGVkL3Nj aF9zZnEuYyBnYXRlay9uZXQvc2NoZWQvc2NoX3NmcS5jDQotLS0gL3Vzci9z cmMvbGludXgtMi4yLjE2L25ldC9zY2hlZC9zY2hfc2ZxLmMJU3VuIEFwciAy NSAwMjo1MTo0OCAxOTk5DQorKysgZ2F0ZWsvbmV0L3NjaGVkL3NjaF9zZnEu YwlTdW4gU2VwIDI0IDE1OjQ1OjUwIDIwMDANCkBAIC0xMTEsNiArMTExLDgg QEANCiAJaW50CQlwZXJ0dXJiYXRpb247DQogCXNmcV9pbmRleAl0YWlsOwkJ LyogSW5kZXggb2YgY3VycmVudCBzbG90IGluIHJvdW5kICovDQogCXNmcV9p bmRleAltYXhfZGVwdGg7CS8qIE1heGltYWwgZGVwdGggKi8NCisJc2hvcnQJ CWZsb3dzOwkJLyogTnVtIG9mIGZsb3dzICovDQorCXNob3J0CQltYXhfZmxv d3M7CS8qIE1heCBudW0gb2YgZmxvd3MgKi8NCiANCiAJc2ZxX2luZGV4CWh0 W1NGUV9IQVNIX0RJVklTT1JdOwkvKiBIYXNoIHRhYmxlICovDQogCXNmcV9p bmRleAluZXh0W1NGUV9ERVBUSF07CS8qIEFjdGl2ZSBzbG90cyBsaW5rICov DQpAQCAtMjY1LDYgKzI2Nyw3IEBADQogCV9fc2tiX3F1ZXVlX3RhaWwoJnEt PnFzW3hdLCBza2IpOw0KIAlzZnFfaW5jKHEsIHgpOw0KIAlpZiAocS0+cXNb eF0ucWxlbiA9PSAxKSB7CQkvKiBUaGUgZmxvdyBpcyBuZXcgKi8NCisJICAg IAlpZiAoKytxLT5mbG93cyA+IHEtPm1heF9mbG93cykgcS0+bWF4X2Zsb3dz ID0gcS0+Zmxvd3M7DQogCQlpZiAocS0+dGFpbCA9PSBTRlFfREVQVEgpIHsJ LyogSXQgaXMgdGhlIGZpcnN0IGZsb3cgKi8NCiAJCQlxLT50YWlsID0geDsN CiAJCQlxLT5uZXh0W3hdID0geDsNCkBAIC0yNzYsMTEgKzI3OSwxMyBAQA0K IAkJfQ0KIAl9DQogCWlmICgrK3NjaC0+cS5xbGVuIDwgU0ZRX0RFUFRILTEp IHsNCisJCS8vIGFyYml0cmFyeSBoYWNrIHRvIGxpbWl0IG1heGltYWwgZmxv dydzIGJhY2tsb2cNCisJCWlmIChxLT5xc1t4XS5xbGVuID4gNSkgZ290byBk cm9wOw0KIAkJc2NoLT5zdGF0cy5ieXRlcyArPSBza2ItPmxlbjsNCiAJCXNj aC0+c3RhdHMucGFja2V0cysrOw0KIAkJcmV0dXJuIDE7DQogCX0NCi0NCitk cm9wOg0KIAlzZnFfZHJvcChzY2gpOw0KIAlyZXR1cm4gMDsNCiB9DQpAQCAt MzAwLDYgKzMwNSw3IEBADQogCV9fc2tiX3F1ZXVlX2hlYWQoJnEtPnFzW3hd LCBza2IpOw0KIAlzZnFfaW5jKHEsIHgpOw0KIAlpZiAocS0+cXNbeF0ucWxl biA9PSAxKSB7CQkvKiBUaGUgZmxvdyBpcyBuZXcgKi8NCisJICAgIAlpZiAo KytxLT5mbG93cyA+IHEtPm1heF9mbG93cykgcS0+bWF4X2Zsb3dzID0gcS0+ Zmxvd3M7DQogCQlpZiAocS0+dGFpbCA9PSBTRlFfREVQVEgpIHsJLyogSXQg aXMgdGhlIGZpcnN0IGZsb3cgKi8NCiAJCQlxLT50YWlsID0geDsNCiAJCQlx LT5uZXh0W3hdID0geDsNCkBAIC0zNDEsNiArMzQ3LDcgQEANCiANCiAJLyog SXMgdGhlIHNsb3QgZW1wdHk/ICovDQogCWlmIChxLT5xc1thXS5xbGVuID09 IDApIHsNCisJICAgIAlxLT5mbG93cy0tOw0KIAkJYSA9IHEtPm5leHRbYV07 DQogCQlpZiAoYSA9PSBvbGRfYSkgew0KIAkJCXEtPnRhaWwgPSBTRlFfREVQ VEg7DQpAQCAtNDE3LDYgKzQyNCw4IEBADQogCQlxLT5kZXBbaStTRlFfREVQ VEhdLnByZXYgPSBpK1NGUV9ERVBUSDsNCiAJfQ0KIAlxLT5tYXhfZGVwdGgg PSAwOw0KKwlxLT5tYXhfZmxvd3MgPSAwOw0KKwlxLT5mbG93cyA9IDA7DQog CXEtPnRhaWwgPSBTRlFfREVQVEg7DQogCWlmIChvcHQgPT0gTlVMTCkgew0K IAkJcS0+cXVhbnR1bSA9IHBzY2hlZF9tdHUoc2NoLT5kZXYpOw0KQEAgLTQ1 MCw4ICs0NTksOCBAQA0KIAlvcHQucGVydHVyYl9wZXJpb2QgPSBxLT5wZXJ0 dXJiX3BlcmlvZC9IWjsNCiANCiAJb3B0LmxpbWl0ID0gU0ZRX0RFUFRIOw0K LQlvcHQuZGl2aXNvciA9IFNGUV9IQVNIX0RJVklTT1I7DQotCW9wdC5mbG93 cyA9IFNGUV9ERVBUSDsNCisJb3B0LmRpdmlzb3IgPSBxLT5tYXhfZmxvd3M7 DQorCW9wdC5mbG93cyA9IHEtPmZsb3dzOw0KIA0KIAlSVEFfUFVUKHNrYiwg VENBX09QVElPTlMsIHNpemVvZihvcHQpLCAmb3B0KTsNCiANCg== --8323328-908300613-1057603084=:18108-- _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/