Using Unix TC to shape high bandwidth traffic

Using Unix TC to shape high bandwidth traffic

[LAMBRUSCHI Yannick (EXT)]

SGVsbG8sDQoNCg0KV2UgYWN0dWFsbHkgaGF2ZSBhIDEwR2IvcyBzZXJ2ZXJzIGFuZCAxR2IvcyBz
ZXJ2ZXJzIHRoYXQgY29leGlzdCB0b2dldGhlciAodGVtcG9yYXJ5IG1pZ3JhdGluZyBzb2x1dGlv
bikgW1VEUCB0cmFmZmljXS4gV2Ugd291bGQgbGlrZSB0byBzaGFwZSB0aGUgdHJhZmZpYyBjb21p
bmcgZnJvbSB0aGUgMTBHYi9zIHNlcnZlcnMgaW4gb3JkZXIgdG8gYXZvaWQgYmlnIGJ1cnN0cyB0
aGF0IHRoZSAxRyBzZXJ2ZXJzIGNvdWxkIG5vdCBoYW5kbGUuDQoNCkl0IHNlZW1zIHRoYXQgInRj
IiBjYW5ub3QgZG8gdGhlIGpvYiB3aXRoIGEgdGJmIChvciBtYXliZSB3ZSB1c2UgaXQgdGhlIHdy
b25nIHdheSkuIEZvciBpbnN0YW5jZSBvbiBvdXIgMTBHIHNlcnZlcnMgd2UgdHJpZWQgdGhlIGZv
bGxvd2luZzoNCg0Kc3VkbyB0YyBxZGlzYyBhZGQgZGV2IGV0aDUgcm9vdCB0YmYgcmF0ZSA5NTBt
Yml0IGxhdGVuY3kgMXMgYnVyc3QgNTBtYml0IHBlYWtyYXRlIDEwMDBtYml0IG10dSAxNTAwDQoN
CkhlcmUgd2Ugbm9ybWFsbHkgc2V0IHRoZSBwZWFrcmF0ZSBhdCAxbWIgKHdoaWNoIG5vcm1hbGx5
IGNhbid0IGdlbmVyYXRlIGJ1cnN0ID4gMW1iL3MpLiBVbmZvcnR1bmF0ZWx5LCB0aGF0IGRvZXMg
bm90IHdvcmssIGluIGZhY3QgYWZ0ZXIgdXNpbmcgdGhpcyB0YyBjb25maWcsIHdlIGxvd2VyIG91
ciBtYWluIGJhbmR3aWR0aCB0byBhdCBtYXggMk1iL3MuLg0KDQpPdXIgb25seSBjbHVlIGZvciB0
aGlzIHN0cmFuZ2UgYmVoYXZpb3IgaXMgdGhhdCBzZW50ZW5jZSBpbiB0aGUgdGMgbWFudWFsOg0K
DQoiVG8gYWNoaWV2ZSBwZXJmZWN0aW9uLCB0aGUgc2Vjb25kIGJ1Y2tldCBtYXkgY29udGFpbiBv
bmx5IGEgc2luZ2xlIHBhY2tldCwgd2hpY2ggbGVhZHMgdG8gdGhlIGVhcmxpZXIgbWVudGlvbmVk
IDFtYml0L3MgbGltaXQuDQoNClRoaXMgbGltaXQgaXMgY2F1c2VkIGJ5IHRoZSBmYWN0IHRoYXQg
dGhlIGtlcm5lbCBjYW4gb25seSB0aHJvdHRsZSBmb3IgYXQgbWluaW11bSAxICdqaWZmeScsIHdo
aWNoIGRlcGVuZHMgb24gSFogYXMgMS9IWi4gRm9yIHBlcmZlY3Qgc2hhcGluZywgb25seSBhIHNp
bmdsZSBwYWNrZXQgY2FuIGdldCBzZW50IHBlciBqaWZmeSAtIGZvciBIWj0xMDAsIHRoaXMgbWVh
bnMgMTAwIHBhY2tldHMgb2Ygb24gYXZlcmFnZSAxMDAwIGJ5dGVzIGVhY2gsIHdoaWNoIHJvdWdo
bHkgY29ycmVzcG9uZHMgdG8gMW1iaXQvcy4gIg0KDQpTbywgaXQncyBzdXJlIHdlIGNhbid0IGhh
dmUgYSBwZWFrcmF0ZSA+IDFNYml0L3MgPw0KDQpNYXliZSwgdGhlcmUgaXMgYW5vdGhlciBjb21w
bGV0ZWx5IGRpZmZlcmVudCB3YXkgdG8gYWNoaWV2ZSBvdXIgZ29hbCwgaWYgYW55b25lIGhhcyBh
IHN1Z2dlc3Rpb24gdGhhdCB3b3VsZCBoZWxwIG1lIGFjaGlldmUgb3VyIGdvYWwuLiA9KSA/DQoN
CktpbmQgcmVnYXJkcw0K

Re: Using Unix TC to shape high bandwidth traffic

[Andy Furniss]

LAMBRUSCHI Yannick (EXT) wrote:
> Hello,
>
>
> We actually have a 10Gb/s servers and 1Gb/s servers that coexist
> together (temporary migrating solution) [UDP traffic]. We would like
> to shape the traffic coming from the 10Gb/s servers in order to avoid
> big bursts that the 1G servers could not handle.
>
> It seems that "tc" cannot do the job with a tbf (or maybe we use it
> the wrong way). For instance on our 10G servers we tried the
> following:

I don't know about shaping high bandwidth.

> sudo tc qdisc add dev eth5 root tbf rate 950mbit latency 1s burst
> 50mbit peakrate 1000mbit mtu 1500

I can test this though and it's actually broken for me = no full size 
traffic.

You should read man tc-tbf - the mtu setting + peakrate is an issue.

mtu on eth when playing with tc is 1514 to allow for 1500 ip level as
that's the size tc sees the packets as. Of course it may be more or less
eg. on vlan (1518) or ip level on ppp.

The mtu using your command actually prevents 1500 ip level traffic 
totally (for me).

>
> Here we normally set the peakrate at 1mb (which normally can't
> generate burst > 1mb/s). Unfortunately, that does not work, in fact
> after using this tc config, we lower our main bandwidth to at max
> 2Mb/s..
>
> Our only clue for this strange behavior is that sentence in the tc
> manual:
>
> "To achieve perfection, the second bucket may contain only a single
> packet, which leads to the earlier mentioned 1mbit/s limit.
>
> This limit is caused by the fact that the kernel can only throttle
> for at minimum 1 'jiffy', which depends on HZ as 1/HZ. For perfect
> shaping, only a single packet can get sent per jiffy - for HZ\x100,
> this means 100 packets of on average 1000 bytes each, which roughly
> corresponds to 1mbit/s. "
>
> So, it's sure we can't have a peakrate > 1Mbit/s ?
>
> Maybe, there is another completely different way to achieve our goal,
> if anyone has a suggestion that would help me achieve our goal.. =)
> ?
>
> Kind regards
> N�����r��y���b�X��ǧv�^�)޺{.n�+���j�\�{ay�\x1dʇڙ�,j\a��f���h���z�\x1e�w���\f���j:+v���w�j�m����\a����zZ+�����ݢj"��!tml>

RE: Using Unix TC to shape high bandwidth traffic

[LAMBRUSCHI Yannick (EXT)]

SGVsbG8sDQoNCg0KVGhhbmtzIGZvciB0aGUgcG9zdC4gV2UgYWxzbyBwbGF5ZWQgd2l0aCBtdHUg
MTUxNCAoYW5kIGV2ZW4gZ3JlYXRlcikgYnV0IGRpZCBub3QgY2hhbmdlIGFueXRoaW5nLiBJIHJl
YWxseSB0aGluayBpdCBjb21lcyBmcm9tIHRoZSBqaWZmeSBsaW1pdCBidXQgY2FuJ3QgZmluZCBh
bm90aGVyIHdheS4uDQoNCllhbm5pY2sNCg0KLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCkZy
b206IEFuZHkgRnVybmlzcyBbbWFpbHRvOmFkZi5saXN0c0BnbWFpbC5jb21dIA0KU2VudDogVGh1
cnNkYXkgNiBBdWd1c3QgMjAxNSAxNDoxMA0KVG86IExBTUJSVVNDSEkgWWFubmljayAoRVhUKSBS
ZXNnR3RzTWt0TWRtOyBsYXJ0Yw0KU3ViamVjdDogUmU6IFVzaW5nIFVuaXggVEMgdG8gc2hhcGUg
aGlnaCBiYW5kd2lkdGggdHJhZmZpYw0KDQpMQU1CUlVTQ0hJIFlhbm5pY2sgKEVYVCkgd3JvdGU6
DQo+IEhlbGxvLA0KPg0KPg0KPiBXZSBhY3R1YWxseSBoYXZlIGEgMTBHYi9zIHNlcnZlcnMgYW5k
IDFHYi9zIHNlcnZlcnMgdGhhdCBjb2V4aXN0DQo+IHRvZ2V0aGVyICh0ZW1wb3JhcnkgbWlncmF0
aW5nIHNvbHV0aW9uKSBbVURQIHRyYWZmaWNdLiBXZSB3b3VsZCBsaWtlDQo+IHRvIHNoYXBlIHRo
ZSB0cmFmZmljIGNvbWluZyBmcm9tIHRoZSAxMEdiL3Mgc2VydmVycyBpbiBvcmRlciB0byBhdm9p
ZA0KPiBiaWcgYnVyc3RzIHRoYXQgdGhlIDFHIHNlcnZlcnMgY291bGQgbm90IGhhbmRsZS4NCj4N
Cj4gSXQgc2VlbXMgdGhhdCAidGMiIGNhbm5vdCBkbyB0aGUgam9iIHdpdGggYSB0YmYgKG9yIG1h
eWJlIHdlIHVzZSBpdA0KPiB0aGUgd3Jvbmcgd2F5KS4gRm9yIGluc3RhbmNlIG9uIG91ciAxMEcg
c2VydmVycyB3ZSB0cmllZCB0aGUNCj4gZm9sbG93aW5nOg0KDQpJIGRvbid0IGtub3cgYWJvdXQg
c2hhcGluZyBoaWdoIGJhbmR3aWR0aC4NCg0KPiBzdWRvIHRjIHFkaXNjIGFkZCBkZXYgZXRoNSBy
b290IHRiZiByYXRlIDk1MG1iaXQgbGF0ZW5jeSAxcyBidXJzdA0KPiA1MG1iaXQgcGVha3JhdGUg
MTAwMG1iaXQgbXR1IDE1MDANCg0KSSBjYW4gdGVzdCB0aGlzIHRob3VnaCBhbmQgaXQncyBhY3R1
YWxseSBicm9rZW4gZm9yIG1lID0gbm8gZnVsbCBzaXplIA0KdHJhZmZpYy4NCg0KWW91IHNob3Vs
ZCByZWFkIG1hbiB0Yy10YmYgLSB0aGUgbXR1IHNldHRpbmcgKyBwZWFrcmF0ZSBpcyBhbiBpc3N1
ZS4NCg0KbXR1IG9uIGV0aCB3aGVuIHBsYXlpbmcgd2l0aCB0YyBpcyAxNTE0IHRvIGFsbG93IGZv
ciAxNTAwIGlwIGxldmVsIGFzDQp0aGF0J3MgdGhlIHNpemUgdGMgc2VlcyB0aGUgcGFja2V0cyBh
cy4gT2YgY291cnNlIGl0IG1heSBiZSBtb3JlIG9yIGxlc3MNCmVnLiBvbiB2bGFuICgxNTE4KSBv
ciBpcCBsZXZlbCBvbiBwcHAuDQoNClRoZSBtdHUgdXNpbmcgeW91ciBjb21tYW5kIGFjdHVhbGx5
IHByZXZlbnRzIDE1MDAgaXAgbGV2ZWwgdHJhZmZpYyANCnRvdGFsbHkgKGZvciBtZSkuDQoNCj4N
Cj4gSGVyZSB3ZSBub3JtYWxseSBzZXQgdGhlIHBlYWtyYXRlIGF0IDFtYiAod2hpY2ggbm9ybWFs
bHkgY2FuJ3QNCj4gZ2VuZXJhdGUgYnVyc3QgPiAxbWIvcykuIFVuZm9ydHVuYXRlbHksIHRoYXQg
ZG9lcyBub3Qgd29yaywgaW4gZmFjdA0KPiBhZnRlciB1c2luZyB0aGlzIHRjIGNvbmZpZywgd2Ug
bG93ZXIgb3VyIG1haW4gYmFuZHdpZHRoIHRvIGF0IG1heA0KPiAyTWIvcy4uDQo+DQo+IE91ciBv
bmx5IGNsdWUgZm9yIHRoaXMgc3RyYW5nZSBiZWhhdmlvciBpcyB0aGF0IHNlbnRlbmNlIGluIHRo
ZSB0Yw0KPiBtYW51YWw6DQo+DQo+ICJUbyBhY2hpZXZlIHBlcmZlY3Rpb24sIHRoZSBzZWNvbmQg
YnVja2V0IG1heSBjb250YWluIG9ubHkgYSBzaW5nbGUNCj4gcGFja2V0LCB3aGljaCBsZWFkcyB0
byB0aGUgZWFybGllciBtZW50aW9uZWQgMW1iaXQvcyBsaW1pdC4NCj4NCj4gVGhpcyBsaW1pdCBp
cyBjYXVzZWQgYnkgdGhlIGZhY3QgdGhhdCB0aGUga2VybmVsIGNhbiBvbmx5IHRocm90dGxlDQo+
IGZvciBhdCBtaW5pbXVtIDEgJ2ppZmZ5Jywgd2hpY2ggZGVwZW5kcyBvbiBIWiBhcyAxL0haLiBG
b3IgcGVyZmVjdA0KPiBzaGFwaW5nLCBvbmx5IGEgc2luZ2xlIHBhY2tldCBjYW4gZ2V0IHNlbnQg
cGVyIGppZmZ5IC0gZm9yIEhaPTEwMCwNCj4gdGhpcyBtZWFucyAxMDAgcGFja2V0cyBvZiBvbiBh
dmVyYWdlIDEwMDAgYnl0ZXMgZWFjaCwgd2hpY2ggcm91Z2hseQ0KPiBjb3JyZXNwb25kcyB0byAx
bWJpdC9zLiAiDQo+DQo+IFNvLCBpdCdzIHN1cmUgd2UgY2FuJ3QgaGF2ZSBhIHBlYWtyYXRlID4g
MU1iaXQvcyA/DQo+DQo+IE1heWJlLCB0aGVyZSBpcyBhbm90aGVyIGNvbXBsZXRlbHkgZGlmZmVy
ZW50IHdheSB0byBhY2hpZXZlIG91ciBnb2FsLA0KPiBpZiBhbnlvbmUgaGFzIGEgc3VnZ2VzdGlv
biB0aGF0IHdvdWxkIGhlbHAgbWUgYWNoaWV2ZSBvdXIgZ29hbC4uID0pDQo+ID8NCj4NCj4gS2lu
ZCByZWdhcmRzDQo+IE7vv73vv73vv73vv73vv71y77+977+9ee+/ve+/ve+/vWLvv71Y77+977+9
x6d277+9Xu+/vSneunsubu+/vSvvv73vv73vv71q77+9XO+/vXthee+/vR3Kh9qZ77+9LGoN77+9
77+9Zu+/ve+/ve+/vWjvv73vv73vv71677+9Hu+/vXfvv73vv73vv70NCu+/ve+/ve+/vWo6K3bv
v73vv73vv71377+9au+/vW3vv73vv73vv73vv70N77+977+977+977+9elor77+977+977+977+9
77+93aJqIu+/ve+/vSF0bWw9DQo+DQo

Re: Using Unix TC to shape high bandwidth traffic

[Andy Furniss]

LAMBRUSCHI Yannick (EXT) wrote:
> Hello,
>
>
> Thanks for the post. We also played with mtu 1514 (and even greater)
> but did not change anything. I really think it comes from the jiffy
> limit but can't find another way..

Strangely I have now tested with your command and mtu 1514 and I can get
gig speed OK.

I was thinking possibly you could just omit the peak rate and mtu (which
seems to be another buffer size) and see if it was OK.

I don't know why I am not limited like you but -

I am using tcp netperf.

My mobo gig nic is dumb so doesn't offload. I guess a 10 gig nic will
offload, so unless you already turned it off with ethtool it's possible
the kernel is sending huge packets to the nic.

Maybe try far bigger mtu or not bother with peakrate/mtu - FWIW setting
1000mbit peakrate is still going to be faster than a gig nic can go as
there are overheads above what tc sees.

It could be kernel settings - you mention jiffies, tbf is ancient and I
don't know whether it was ever changed, or if kernel settings can change
things.

I don't know why I am not limited like you grepping hz from my config
looks like -

ph4[1080i25]$ grep -i hz /boot/config-4.2.0-rc4-g194a336
CONFIG_HZ_PERIODIC=y
# CONFIG_NO_HZ_IDLE is not set
# CONFIG_NO_HZ_FULL is not set
# CONFIG_NO_HZ is not set
# CONFIG_HZ_100 is not set
# CONFIG_HZ_250 is not set
# CONFIG_HZ_300 is not set
CONFIG_HZ_1000=y
CONFIG_HZ\x1000
# CONFIG_MACHZ_WDT is not set

RE: Using Unix TC to shape high bandwidth traffic

[LAMBRUSCHI Yannick (EXT)]

SGkgQW5keSwNCg0KDQpJIGZvcmdvdCB0byBtZW50aW9uIHRoYXQgYnV0IEknbSBub3QgbGltaXRl
ZCBhdCBhbGwgd2hlbiBJIGRvbid0IHNldCBhIHBlYWtyYXRlLCBpdCB3b3JrcyB3ZWxsLiBCdXQg
ZG9lc24ndCBzZXR0aW5nIGEgcGVhayByYXRlIG1lYW5zIHRoYXQgaXQgd2lsbCBhbGxvdyBiaWcg
MTBHIGJ1cnN0cyBhdCBzb21lIGluc3RhbnQgKHRvbyBtdWNoIGZvciB0aGUgMUcgc2VydmVycyBJ
IHRoaW5rKS4gSSBvbmx5IGdldCBpc3N1ZSB3aGVuIEkgc2V0IHRoZSBwZWFrIHJhdGUgKHdoaWNo
IGtpbmQgb2YgbGlua2VkIHdpdGggdGhlIGppZmZ5IGlmIEknbSBub3QgbWlzdGFrZW4pLg0KDQoN
ClJlZ2FyZHMsDQoNCllhbm5pY2sNCg0KLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCkZyb206
IEFuZHkgRnVybmlzcyBbbWFpbHRvOmFkZi5saXN0c0BnbWFpbC5jb21dIA0KU2VudDogVGh1cnNk
YXkgNiBBdWd1c3QgMjAxNSAxOTowMQ0KVG86IExBTUJSVVNDSEkgWWFubmljayAoRVhUKSBSZXNn
R3RzTWt0TWRtOyBsYXJ0Yw0KU3ViamVjdDogUmU6IFVzaW5nIFVuaXggVEMgdG8gc2hhcGUgaGln
aCBiYW5kd2lkdGggdHJhZmZpYw0KDQpMQU1CUlVTQ0hJIFlhbm5pY2sgKEVYVCkgd3JvdGU6DQo+
IEhlbGxvLA0KPg0KPg0KPiBUaGFua3MgZm9yIHRoZSBwb3N0LiBXZSBhbHNvIHBsYXllZCB3aXRo
IG10dSAxNTE0IChhbmQgZXZlbiBncmVhdGVyKQ0KPiBidXQgZGlkIG5vdCBjaGFuZ2UgYW55dGhp
bmcuIEkgcmVhbGx5IHRoaW5rIGl0IGNvbWVzIGZyb20gdGhlIGppZmZ5DQo+IGxpbWl0IGJ1dCBj
YW4ndCBmaW5kIGFub3RoZXIgd2F5Li4NCg0KU3RyYW5nZWx5IEkgaGF2ZSBub3cgdGVzdGVkIHdp
dGggeW91ciBjb21tYW5kIGFuZCBtdHUgMTUxNCBhbmQgSSBjYW4gZ2V0DQpnaWcgc3BlZWQgT0su
DQoNCkkgd2FzIHRoaW5raW5nIHBvc3NpYmx5IHlvdSBjb3VsZCBqdXN0IG9taXQgdGhlIHBlYWsg
cmF0ZSBhbmQgbXR1ICh3aGljaA0Kc2VlbXMgdG8gYmUgYW5vdGhlciBidWZmZXIgc2l6ZSkgYW5k
IHNlZSBpZiBpdCB3YXMgT0suDQoNCkkgZG9uJ3Qga25vdyB3aHkgSSBhbSBub3QgbGltaXRlZCBs
aWtlIHlvdSBidXQgLQ0KDQpJIGFtIHVzaW5nIHRjcCBuZXRwZXJmLg0KDQpNeSBtb2JvIGdpZyBu
aWMgaXMgZHVtYiBzbyBkb2Vzbid0IG9mZmxvYWQuIEkgZ3Vlc3MgYSAxMCBnaWcgbmljIHdpbGwN
Cm9mZmxvYWQsIHNvIHVubGVzcyB5b3UgYWxyZWFkeSB0dXJuZWQgaXQgb2ZmIHdpdGggZXRodG9v
bCBpdCdzIHBvc3NpYmxlDQp0aGUga2VybmVsIGlzIHNlbmRpbmcgaHVnZSBwYWNrZXRzIHRvIHRo
ZSBuaWMuDQoNCk1heWJlIHRyeSBmYXIgYmlnZ2VyIG10dSBvciBub3QgYm90aGVyIHdpdGggcGVh
a3JhdGUvbXR1IC0gRldJVyBzZXR0aW5nDQoxMDAwbWJpdCBwZWFrcmF0ZSBpcyBzdGlsbCBnb2lu
ZyB0byBiZSBmYXN0ZXIgdGhhbiBhIGdpZyBuaWMgY2FuIGdvIGFzDQp0aGVyZSBhcmUgb3Zlcmhl
YWRzIGFib3ZlIHdoYXQgdGMgc2Vlcy4NCg0KSXQgY291bGQgYmUga2VybmVsIHNldHRpbmdzIC0g
eW91IG1lbnRpb24gamlmZmllcywgdGJmIGlzIGFuY2llbnQgYW5kIEkNCmRvbid0IGtub3cgd2hl
dGhlciBpdCB3YXMgZXZlciBjaGFuZ2VkLCBvciBpZiBrZXJuZWwgc2V0dGluZ3MgY2FuIGNoYW5n
ZQ0KdGhpbmdzLg0KDQpJIGRvbid0IGtub3cgd2h5IEkgYW0gbm90IGxpbWl0ZWQgbGlrZSB5b3Ug
Z3JlcHBpbmcgaHogZnJvbSBteSBjb25maWcNCmxvb2tzIGxpa2UgLQ0KDQpwaDRbMTA4MGkyNV0k
IGdyZXAgLWkgaHogL2Jvb3QvY29uZmlnLTQuMi4wLXJjNC1nMTk0YTMzNg0KQ09ORklHX0haX1BF
UklPRElDPXkNCiMgQ09ORklHX05PX0haX0lETEUgaXMgbm90IHNldA0KIyBDT05GSUdfTk9fSFpf
RlVMTCBpcyBub3Qgc2V0DQojIENPTkZJR19OT19IWiBpcyBub3Qgc2V0DQojIENPTkZJR19IWl8x
MDAgaXMgbm90IHNldA0KIyBDT05GSUdfSFpfMjUwIGlzIG5vdCBzZXQNCiMgQ09ORklHX0haXzMw
MCBpcyBub3Qgc2V0DQpDT05GSUdfSFpfMTAwMD15DQpDT05GSUdfSFo9MTAwMA0KIyBDT05GSUdf
TUFDSFpfV0RUIGlzIG5vdCBzZXQNCg=

Re: Using Unix TC to shape high bandwidth traffic

[Dave Taht]

We are certainly looking for more high-speed testers of the new "cake"
qdisc which we hope to be a successor to htb + fq_codel in many
respects. It is still out of tree but can be made to compile on most
kernels between 3.13 and 4.0. It solves the burst problem in
particular, and also runs pretty fast.

A recent preso:

http://battlemesh.org/BattleMeshV8/Agenda?action=AttachFile&do=view&target ke-battlemesh-v8.pdf

More info and where to get it:

http://www.bufferbloat.net/projects/codel/wiki/Cake

On Fri, Aug 7, 2015 at 10:30 AM, LAMBRUSCHI Yannick (EXT)
<yannick.lambruschi-ext@socgen.com> wrote:
> Hi Andy,
>
>
> I forgot to mention that but I'm not limited at all when I don't set a peakrate, it works well. But doesn't setting a peak rate means that it will allow big 10G bursts at some instant (too much for the 1G servers I think). I only get issue when I set the peak rate (which kind of linked with the jiffy if I'm not mistaken).
>
>
> Regards,
>
> Yannick
>
> -----Original Message-----
> From: Andy Furniss [mailto:adf.lists@gmail.com]
> Sent: Thursday 6 August 2015 19:01
> To: LAMBRUSCHI Yannick (EXT) ResgGtsMktMdm; lartc
> Subject: Re: Using Unix TC to shape high bandwidth traffic
>
> LAMBRUSCHI Yannick (EXT) wrote:
>> Hello,
>>
>>
>> Thanks for the post. We also played with mtu 1514 (and even greater)
>> but did not change anything. I really think it comes from the jiffy
>> limit but can't find another way..
>
> Strangely I have now tested with your command and mtu 1514 and I can get
> gig speed OK.
>
> I was thinking possibly you could just omit the peak rate and mtu (which
> seems to be another buffer size) and see if it was OK.
>
> I don't know why I am not limited like you but -
>
> I am using tcp netperf.
>
> My mobo gig nic is dumb so doesn't offload. I guess a 10 gig nic will
> offload, so unless you already turned it off with ethtool it's possible
> the kernel is sending huge packets to the nic.
>
> Maybe try far bigger mtu or not bother with peakrate/mtu - FWIW setting
> 1000mbit peakrate is still going to be faster than a gig nic can go as
> there are overheads above what tc sees.
>
> It could be kernel settings - you mention jiffies, tbf is ancient and I
> don't know whether it was ever changed, or if kernel settings can change
> things.
>
> I don't know why I am not limited like you grepping hz from my config
> looks like -
>
> ph4[1080i25]$ grep -i hz /boot/config-4.2.0-rc4-g194a336
> CONFIG_HZ_PERIODIC=y
> # CONFIG_NO_HZ_IDLE is not set
> # CONFIG_NO_HZ_FULL is not set
> # CONFIG_NO_HZ is not set
> # CONFIG_HZ_100 is not set
> # CONFIG_HZ_250 is not set
> # CONFIG_HZ_300 is not set
> CONFIG_HZ_1000=y
> CONFIG_HZ\x1000
> # CONFIG_MACHZ_WDT is not set



-- 
Dave Täht
worldwide bufferbloat report:
http://www.dslreports.com/speedtest/results/bufferbloat
And:
What will it take to vastly improve wifi for everyone?
https://plus.google.com/u/0/explore/makewififast
--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Using Unix TC to shape high bandwidth traffic

[LAMBRUSCHI Yannick (EXT)]

SGVsbG8sDQoNCg0KVGhhbmtzIGZvciB0aGUgaWRlYSwgd2Ugd2lsbCB0cnkgdG9vIGxvb2sgaW50
byBpdCwgc2VlbXMgYSBiaXQgY29tcGxpY2F0ZWQgdG8gbWUgPSkuDQoNCg0KWWFubmljaw0KDQot
LS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KRnJvbTogbGFydGMtb3duZXJAdmdlci5rZXJuZWwu
b3JnIFttYWlsdG86bGFydGMtb3duZXJAdmdlci5rZXJuZWwub3JnXSBPbiBCZWhhbGYgT2YgRGF2
ZSBUYWh0DQpTZW50OiBGcmlkYXkgNyBBdWd1c3QgMjAxNSAxMDo0NQ0KVG86IExBTUJSVVNDSEkg
WWFubmljayAoRVhUKSBSZXNnR3RzTWt0TWRtDQpDYzogQW5keSBGdXJuaXNzOyBsYXJ0Yw0KU3Vi
amVjdDogUmU6IFVzaW5nIFVuaXggVEMgdG8gc2hhcGUgaGlnaCBiYW5kd2lkdGggdHJhZmZpYw0K
DQpXZSBhcmUgY2VydGFpbmx5IGxvb2tpbmcgZm9yIG1vcmUgaGlnaC1zcGVlZCB0ZXN0ZXJzIG9m
IHRoZSBuZXcgImNha2UiDQpxZGlzYyB3aGljaCB3ZSBob3BlIHRvIGJlIGEgc3VjY2Vzc29yIHRv
IGh0YiArIGZxX2NvZGVsIGluIG1hbnkNCnJlc3BlY3RzLiBJdCBpcyBzdGlsbCBvdXQgb2YgdHJl
ZSBidXQgY2FuIGJlIG1hZGUgdG8gY29tcGlsZSBvbiBtb3N0DQprZXJuZWxzIGJldHdlZW4gMy4x
MyBhbmQgNC4wLiBJdCBzb2x2ZXMgdGhlIGJ1cnN0IHByb2JsZW0gaW4NCnBhcnRpY3VsYXIsIGFu
ZCBhbHNvIHJ1bnMgcHJldHR5IGZhc3QuDQoNCkEgcmVjZW50IHByZXNvOg0KDQpodHRwOi8vYmF0
dGxlbWVzaC5vcmcvQmF0dGxlTWVzaFY4L0FnZW5kYT9hY3Rpb249QXR0YWNoRmlsZSZkbz12aWV3
JnRhcmdldD1jYWtlLWJhdHRsZW1lc2gtdjgucGRmDQoNCk1vcmUgaW5mbyBhbmQgd2hlcmUgdG8g
Z2V0IGl0Og0KDQpodHRwOi8vd3d3LmJ1ZmZlcmJsb2F0Lm5ldC9wcm9qZWN0cy9jb2RlbC93aWtp
L0Nha2UNCg0KT24gRnJpLCBBdWcgNywgMjAxNSBhdCAxMDozMCBBTSwgTEFNQlJVU0NISSBZYW5u
aWNrIChFWFQpDQo8eWFubmljay5sYW1icnVzY2hpLWV4dEBzb2NnZW4uY29tPiB3cm90ZToNCj4g
SGkgQW5keSwNCj4NCj4NCj4gSSBmb3Jnb3QgdG8gbWVudGlvbiB0aGF0IGJ1dCBJJ20gbm90IGxp
bWl0ZWQgYXQgYWxsIHdoZW4gSSBkb24ndCBzZXQgYSBwZWFrcmF0ZSwgaXQgd29ya3Mgd2VsbC4g
QnV0IGRvZXNuJ3Qgc2V0dGluZyBhIHBlYWsgcmF0ZSBtZWFucyB0aGF0IGl0IHdpbGwgYWxsb3cg
YmlnIDEwRyBidXJzdHMgYXQgc29tZSBpbnN0YW50ICh0b28gbXVjaCBmb3IgdGhlIDFHIHNlcnZl
cnMgSSB0aGluaykuIEkgb25seSBnZXQgaXNzdWUgd2hlbiBJIHNldCB0aGUgcGVhayByYXRlICh3
aGljaCBraW5kIG9mIGxpbmtlZCB3aXRoIHRoZSBqaWZmeSBpZiBJJ20gbm90IG1pc3Rha2VuKS4N
Cj4NCj4NCj4gUmVnYXJkcywNCj4NCj4gWWFubmljaw0KPg0KPiAtLS0tLU9yaWdpbmFsIE1lc3Nh
Z2UtLS0tLQ0KPiBGcm9tOiBBbmR5IEZ1cm5pc3MgW21haWx0bzphZGYubGlzdHNAZ21haWwuY29t
XQ0KPiBTZW50OiBUaHVyc2RheSA2IEF1Z3VzdCAyMDE1IDE5OjAxDQo+IFRvOiBMQU1CUlVTQ0hJ
IFlhbm5pY2sgKEVYVCkgUmVzZ0d0c01rdE1kbTsgbGFydGMNCj4gU3ViamVjdDogUmU6IFVzaW5n
IFVuaXggVEMgdG8gc2hhcGUgaGlnaCBiYW5kd2lkdGggdHJhZmZpYw0KPg0KPiBMQU1CUlVTQ0hJ
IFlhbm5pY2sgKEVYVCkgd3JvdGU6DQo+PiBIZWxsbywNCj4+DQo+Pg0KPj4gVGhhbmtzIGZvciB0
aGUgcG9zdC4gV2UgYWxzbyBwbGF5ZWQgd2l0aCBtdHUgMTUxNCAoYW5kIGV2ZW4gZ3JlYXRlcikN
Cj4+IGJ1dCBkaWQgbm90IGNoYW5nZSBhbnl0aGluZy4gSSByZWFsbHkgdGhpbmsgaXQgY29tZXMg
ZnJvbSB0aGUgamlmZnkNCj4+IGxpbWl0IGJ1dCBjYW4ndCBmaW5kIGFub3RoZXIgd2F5Li4NCj4N
Cj4gU3RyYW5nZWx5IEkgaGF2ZSBub3cgdGVzdGVkIHdpdGggeW91ciBjb21tYW5kIGFuZCBtdHUg
MTUxNCBhbmQgSSBjYW4gZ2V0DQo+IGdpZyBzcGVlZCBPSy4NCj4NCj4gSSB3YXMgdGhpbmtpbmcg
cG9zc2libHkgeW91IGNvdWxkIGp1c3Qgb21pdCB0aGUgcGVhayByYXRlIGFuZCBtdHUgKHdoaWNo
DQo+IHNlZW1zIHRvIGJlIGFub3RoZXIgYnVmZmVyIHNpemUpIGFuZCBzZWUgaWYgaXQgd2FzIE9L
Lg0KPg0KPiBJIGRvbid0IGtub3cgd2h5IEkgYW0gbm90IGxpbWl0ZWQgbGlrZSB5b3UgYnV0IC0N
Cj4NCj4gSSBhbSB1c2luZyB0Y3AgbmV0cGVyZi4NCj4NCj4gTXkgbW9ibyBnaWcgbmljIGlzIGR1
bWIgc28gZG9lc24ndCBvZmZsb2FkLiBJIGd1ZXNzIGEgMTAgZ2lnIG5pYyB3aWxsDQo+IG9mZmxv
YWQsIHNvIHVubGVzcyB5b3UgYWxyZWFkeSB0dXJuZWQgaXQgb2ZmIHdpdGggZXRodG9vbCBpdCdz
IHBvc3NpYmxlDQo+IHRoZSBrZXJuZWwgaXMgc2VuZGluZyBodWdlIHBhY2tldHMgdG8gdGhlIG5p
Yy4NCj4NCj4gTWF5YmUgdHJ5IGZhciBiaWdnZXIgbXR1IG9yIG5vdCBib3RoZXIgd2l0aCBwZWFr
cmF0ZS9tdHUgLSBGV0lXIHNldHRpbmcNCj4gMTAwMG1iaXQgcGVha3JhdGUgaXMgc3RpbGwgZ29p
bmcgdG8gYmUgZmFzdGVyIHRoYW4gYSBnaWcgbmljIGNhbiBnbyBhcw0KPiB0aGVyZSBhcmUgb3Zl
cmhlYWRzIGFib3ZlIHdoYXQgdGMgc2Vlcy4NCj4NCj4gSXQgY291bGQgYmUga2VybmVsIHNldHRp
bmdzIC0geW91IG1lbnRpb24gamlmZmllcywgdGJmIGlzIGFuY2llbnQgYW5kIEkNCj4gZG9uJ3Qg
a25vdyB3aGV0aGVyIGl0IHdhcyBldmVyIGNoYW5nZWQsIG9yIGlmIGtlcm5lbCBzZXR0aW5ncyBj
YW4gY2hhbmdlDQo+IHRoaW5ncy4NCj4NCj4gSSBkb24ndCBrbm93IHdoeSBJIGFtIG5vdCBsaW1p
dGVkIGxpa2UgeW91IGdyZXBwaW5nIGh6IGZyb20gbXkgY29uZmlnDQo+IGxvb2tzIGxpa2UgLQ0K
Pg0KPiBwaDRbMTA4MGkyNV0kIGdyZXAgLWkgaHogL2Jvb3QvY29uZmlnLTQuMi4wLXJjNC1nMTk0
YTMzNg0KPiBDT05GSUdfSFpfUEVSSU9ESUM9eQ0KPiAjIENPTkZJR19OT19IWl9JRExFIGlzIG5v
dCBzZXQNCj4gIyBDT05GSUdfTk9fSFpfRlVMTCBpcyBub3Qgc2V0DQo+ICMgQ09ORklHX05PX0ha
IGlzIG5vdCBzZXQNCj4gIyBDT05GSUdfSFpfMTAwIGlzIG5vdCBzZXQNCj4gIyBDT05GSUdfSFpf
MjUwIGlzIG5vdCBzZXQNCj4gIyBDT05GSUdfSFpfMzAwIGlzIG5vdCBzZXQNCj4gQ09ORklHX0ha
XzEwMDA9eQ0KPiBDT05GSUdfSFo9MTAwMA0KPiAjIENPTkZJR19NQUNIWl9XRFQgaXMgbm90IHNl
dA0KDQoNCg0KLS0gDQpEYXZlIFTDpGh0DQp3b3JsZHdpZGUgYnVmZmVyYmxvYXQgcmVwb3J0Og0K
aHR0cDovL3d3dy5kc2xyZXBvcnRzLmNvbS9zcGVlZHRlc3QvcmVzdWx0cy9idWZmZXJibG9hdA0K
QW5kOg0KV2hhdCB3aWxsIGl0IHRha2UgdG8gdmFzdGx5IGltcHJvdmUgd2lmaSBmb3IgZXZlcnlv
bmU/DQpodHRwczovL3BsdXMuZ29vZ2xlLmNvbS91LzAvZXhwbG9yZS9tYWtld2lmaWZhc3QNCi0t
DQpUbyB1bnN1YnNjcmliZSBmcm9tIHRoaXMgbGlzdDogc2VuZCB0aGUgbGluZSAidW5zdWJzY3Jp
YmUgbGFydGMiIGluDQp0aGUgYm9keSBvZiBhIG1lc3NhZ2UgdG8gbWFqb3Jkb21vQHZnZXIua2Vy
bmVsLm9yZw0KTW9yZSBtYWpvcmRvbW8gaW5mbyBhdCAgaHR0cDovL3ZnZXIua2VybmVsLm9yZy9t
YWpvcmRvbW8taW5mby5odG1sDQo

Re: Using Unix TC to shape high bandwidth traffic

[Marco Gaiarin]

Mandi! Dave Taht
  In chel di` si favelave...

> More info and where to get it:
> http://www.bufferbloat.net/projects/codel/wiki/Cake

Someone have just recompiled kernel modules and iproute utility for
debian AS debian packages?

Thanks.

-- 
dott. Marco Gaiarin				        GNUPG Key ID: 240A3D66
  Associazione ``La Nostra Famiglia''                    http://www.sv.lnf.it/
  Polo FVG   -   Via della Bontà, 7 - 33078   -   San Vito al Tagliamento (PN)
  marco.gaiarin(at)lanostrafamiglia.it   t +39-0434-842711   f +39-0434-842797

		Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA!
    http://www.lanostrafamiglia.it/25/index.php/component/k2/item/123
	(cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA)