From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alan Goodman Date: Mon, 22 Sep 2014 13:09:31 +0000 Subject: Re: [Cerowrt-devel] Correctly calculating overheads on unknown connections Message-Id: <54201F8B.70408@yescomputersolutions.com> List-Id: References: <6DF5DFA0-D88E-470E-ACB6-37703EA964E7@gmx.de> In-Reply-To: <6DF5DFA0-D88E-470E-ACB6-37703EA964E7@gmx.de> MIME-Version: 1.0 Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: quoted-printable To: lartc@vger.kernel.org Hello all once again, I tried running the attached ping sweeper yesterday evening as is and=20 didnt get particularly plausible looking results. I therefore decided=20 to increase the upper limit of the size of ping packets sent and let the=20 script run over night while the connection was quiet. Here is a screen shot of the resulting graph which does appear to have a=20 stepped appearance, but perhaps not as expected? http://imgur.com/RjmT8Qh This test was ran on a BT Infinity VDSL/FTTC connection with the modem=20 plugged directly into a CentOS 6 machine which is doing PPPoE. The=20 connection is synced at 80mbit down and 20mbit up. BT restrict=20 downstream speed to 77.44Mbps IP traffic. I can run the test on a slower BT connection over the week end if anyone=20 is interested in the results? Alan On 21/09/14 19:35, Sebastian Moeller wrote: > Hi Dave, hi Andy, > > > > > On Sep 20, 2014, at 19:55 , Dave Taht wrote: > >> We'd had a very long thread on cerowrt-devel and in the end sebastian >> (I think) had developed some scripts to exaustively (it took hours) >> derive the right encapsulation frame size on a link. I can't find the >> relevant link right now, ccing that list=85 > > I am certainly not the first to have looked at ATM encapsulation effects= on DSL-links, e.g. Jesper Dangaard Brouer wrote a thesis about this topic = (see http://www.adsl-optimizer.dk) and together with Russel Stuart (http://= ace-host.stuart.id.au/russell/files/tc/tc-atm/) I believe they taught the = linux kernel about how to account for encapsulation. What you need to tell = the kernel is whether or not you have ATM encapsulation (ATM is weird in th= at each ip Packet gets chopped into 48 byte cells, with the last partially = full cell padded) and the per packet overhead on your link. You can either = get this information from your ISP and/or from the DSL-modem=92s informatio= n page, but both are not guaranteed to be available/useful. So I set out t= o empirically deduce this information from measurements on my own link. I n= aively started out with using ICMP echo requests as probes (as I easily cou= ld generate probe packets with different sizes with the linux/macosx ping b= inary), as it tur ned out,=20 this works well enough, at least for relatively slow ADSL-links. So ping_sw= eeper6.sh (attached) is the program I use (on an otherwise idle link, typic= ally over night) to collect ~1000 repetitions of time stamped ping packets = spanning two (potential) ATM cells. I then use tc_stab_parameter_guide.m (a= matlab/octave program) to read in the output of the ping_sweeper script an= d process the data. In short if the link runs ATM encapsulation the plot of= the data needs to look like a stair with 48 byte step width, if it is just= smoothly increasing the carrier is not ATM. For ATM links and only ATM lin= ks, the script also tries to figure out the per packet overhead which alway= s worked well for me. (My home-link got recently a silent upgrade where the= encapsulation changed from 40 bytes to 44 bytes (probably due to the intro= duction of VLAN tags), which caused some disturbances in link capacity meas= urements I was running at the time; so I ran my code again and lo and behol= d the overhead=20 had incre ased, which caused the issues with the measurements, as after taking the re= al overhead into account the disturbances went away, but I guess I digress = ;) ) > > > > > Best Regards > Sebastian > > >> >> On Sat, Sep 20, 2014 at 7:17 PM, Andy Furniss wrot= e: >>> Alan Goodman wrote: >>>> >>>> Hi, >>>> >>>> I am looking to figure out the most fool proof way to calculate stab >>>> overheads for ADSL/VDSL connections. >>>> >>>> ppp0 Link encap:Point-to-Point Protocol inet addr:81.149.38.69 >>>> P-t-P:81.139.160.1 Mask:255.255.255.255 UP POINTOPOINT RUNNING NOARP >>>> MULTICAST MTU:1492 Metric:1 RX packets:17368223 errors:0 dropped:0 >>>> overruns:0 frame:0 TX packets:12040295 errors:0 dropped:0 overruns:0 >>>> carrier:0 collisions:0 txqueuelen:100 RX bytes:17420109286 (16.2 GiB) >>>> TX bytes:3611007028 (3.3 GiB) >>>> >>>> I am setting a longer txqueuelen as I am not currently using any fair >>>> queuing (buffer bloat issues with sfq) >>> >>> >>> Whatever is txqlen is on ppp there is likely some other buffer after it >>> - the default can hurt with eg, htb as if you don't add qdiscs to >>> classes it takes (last time I looked) its qlen from that. >>> >>> Sfq was only ever meant for bulk, so should really be in addition to >>> some classification to separate interactive - I don't really get the >> >> Hmm? sfq separates bulk from interactive pretty nicely. It tends to do >> bad things to bulk as it doesn't manage queue length. >> >> A little bit of prioritization or deprioritization for some traffic is >> helpful, but most traffic is hard to classify. >> >>> bufferbloat bit, you could make the default 128 limit lower if you want= ed. >> >> htb + fq_codel, if available, is the right thing here.... >> >> http://www.bufferbloat.net/projects/cerowrt/wiki/Wondershaper_Must_Die >> >>>> The connection is a BT Infinity FTTC VDSL connection synced at >>>> 80mbit/20mbit. The modem is connected directly to the ethernet port >>>> on a server running a slightly tweaked HFSC setup that you folks >>>> helped me set up in July - back when I was on ADSL. I am still >>>> running pppoe I believe from my server. >>> >>> >>> I have similar since May 2013 and I still haven't got round to reading >>> up on everything yet :-) >>> >>> I have extra geek score for using mini jumbos =3D running pppoe with mtu >>> 1500 which works for me on plusnet. You need a recent pppd for this and >>> a nic that works with mtu >=3D 1508. >>> >>> As for overheads, initial searching indicated that it's not easy or >>> maybe even truly possible like adsl. >>> >>>> The largest ping packet that I can fit out onto the wire is 1464 >>>> bytes: >>>> >>>> # ping -c 2 -s 1464 -M do google.com PING google.com (31.55.166.216) >>>> 1464(1492) bytes of data. 1472 bytes from 31.55.166.216: icmp_seq=3D1 >>>> ttlX time=11.7 ms 1472 bytes from 31.55.166.216: icmp_seq=3D2 ttlX >>>> time=11.9 ms >>>> >>>> # ping -c 2 -s 1465 -M do google.com PING google.com (31.55.166.212) >>>> 1465(1493) bytes of data. From >>>> host81-149-38-69.in-addr.btopenworld.com (81.149.38.69) icmp_seq=3D1 >>>> Frag needed and DF set (mtu =3D 1492) From >>>> host81-149-38-69.in-addr.btopenworld.com (81.149.38.69) icmp_seq=3D1 >>>> Frag needed and DF set (mtu =3D 1492) >>> >>> >>> You can't work out your overheads like this. >>> >>> On slow uplink adsl it was possible with ping to infer the fixed part >>> but you needed to send loads of pings increasing in size and plot the >>> best time for each to make a stepped graph. >>> >>> >>>> Based on this I believe overhead should be set to 28, however with 28 >>>> set as my overhead and hfsc ls m2 20000kbit ul m2 20000kbit I seem >>>> to be loosing about 1.5mbit of upload... >>> >>> >>> Even if you could do things perfectly I would back off a few kbit just >>> to be safe. Timers may be different or there may be OAM/Reporting data >>> going up, albeit rarely. >>> >>>> >>>> No traffic manager enabled: >>>> >>>> http://www.thinkbroadband.com/speedtest/results.html?id=14111608942488= 3990118 >>>> >>>> >>>> HFSC traffic manager: >>>> >>>> http://www.thinkbroadband.com/speedtest/results.html?id=14111621662109= 3133034 >>>> >>>> >>>> >>>> Am I calculating overhead incorrectly? >>> >>> >>> VDSL doesn't use ATM I think the PTM it uses is 64/65 - so don't specify >>> atm with stab. Unfortunately stab doesn't do 64/65. >>> >>> As for the fixed part - I am not sure, but roughly starting with IP as >>> that's what tc sees on ppp (as opposed to ip + 14 on eth) >>> >>> IP >>> +8 for PPPOE >>> +14 for ethertype and macs >>> +4 because Openreach modem uses vlan >>> +2 CRC ?? >>> + "a few" 64/65 >>> >>> That's it for fixed - of course 64/65 adds another one for every 64 TBH >>> I didn't get the precice detail from the spec and not having looked >>> recently I can't remember. >>> >>> BT Sin 498 does give some of this info and a couple of examples of >>> throughput for different frame sizes - but it's rounded to kbit which >>> means I couldn't work out to the byte what the overheads were. >>> >>> Worse still VDSL can use link layer retransmits and the sin says that >>> though currently (2013) not enabled, they would be in due course. I have >>> no clue how these work. >>> >>> >>> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe lartc" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> >> >> -- >> Dave T=E4ht >> >> https://www.bufferbloat.net/projects/make-wifi-fast >> _______________________________________________ >> Cerowrt-devel mailing list >> Cerowrt-devel@lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/cerowrt-devel >