[OT] testing internet performance, esp latency/drops?

[OT] testing internet performance, esp latency/drops?

[Larry McVoy]

Merry kernel hackers, we recently installed a T1 line at bitmover.com and
expected improved performance.  In some places we got it, pings to places
in the silicon valley are a respectable 5-9 milliseconds.  FTP performance
is a predictable 180KB/sec, as expected.

However, web browsing sucks.  On about 80% of all links, there is a noticable
hesitation, between 1-15 seconds, as it looks up the name and as it fetches
the first page.  After that point, that site will appear to be OK.

It sounds to me like return packets are getting dropped a lot.  Which is 
possible, but I'd like to know for sure.

Before I wander off to write a test for this, I'm wondering if anyone 
knows of a test suite or a methodology which works.  I was thinking 
about just coding every reference in bookmarks/history into a driver
file which drove a connect-o-matic program that timed how fast it 
could connect to each of those sites.  Any comments on that idea?
-- 
---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

Re: [OT] testing internet performance, esp latency/drops?

[Rob Landley]

On Monday 08 October 2001 12:02, Larry McVoy wrote:
> Merry kernel hackers, we recently installed a T1 line at bitmover.com and
> expected improved performance.  In some places we got it, pings to places
> in the silicon valley are a respectable 5-9 milliseconds.  FTP performance
> is a predictable 180KB/sec, as expected.
>
> However, web browsing sucks.  On about 80% of all links, there is a
> noticable hesitation, between 1-15 seconds, as it looks up the name and as
> it fetches the first page.  After that point, that site will appear to be
> OK.

You're having a DNS problem then.  (You even say it yourself, "as it looks up 
the name"...)  Unless you've got explicit congestion notification support 
turned on, randomly dropped packets would kill your throughput (or at the 
very least the predictability of it).

Where does your DNS server live?  Can you try using another one, or setting 
up a local cacheing one in your subnet and have your system use that?  (I 
think Red Hat 7.1 can do this pretty easily.  I'd make sure it's only bound 
to an address in your local subnet, though.  BIND is a perpetual security 
nightmare, I keep meaning to write something better in Python (it's a UDP 
protocol, how hard could it be?) but I haven't gotten mad enough to make time 
to do it properly...)

For a DNS server with an empty cache, DNS lookup times of 5 seconds or more 
actually aren't that unusual at ALL, considering that an uncached full resove 
from the root nameservers can bounce you off something like five different 
servers along the way, each adding a second or two of latency...  (Gotta love 
nested zone delegations...)

> It sounds to me like return packets are getting dropped a lot.  Which is
> possible, but I'd like to know for sure.
>
> Before I wander off to write a test for this, I'm wondering if anyone
> knows of a test suite or a methodology which works.  I was thinking
> about just coding every reference in bookmarks/history into a driver
> file which drove a connect-o-matic program that timed how fast it
> could connect to each of those sites.  Any comments on that idea?

Seperate out connecting to the sites (by IP) from doing the DNS lookups.  If 
your DNS lookups are the problem, the actually connect is just noise in your 
test.

Try playing around with "dig" for a bit.  It's a DNS swiss army knife, with a 
man page that would do justice to the Acme Kitchen Wonder...

Rob

Re: [OT] testing internet performance, esp latency/drops?

[Alan Cox]

> about just coding every reference in bookmarks/history into a driver
> file which drove a connect-o-matic program that timed how fast it 
> could connect to each of those sites.  Any comments on that idea?

Try connecting to them by ip number - picking a page that doesn't contain
references to other DNS records. Then find a site or a friend who can put
a site on a port other than 80 or 8080 and see if that is mysteriously 
fast even using DNS.

If its the former then its probably DNS issues. If it goes away with non
port 80 its someones transparent proxy

Re: [OT] testing internet performance, esp latency/drops?

[Pedro M. Rodrigues]

   DNS maybe? What is your DNS servers topology? Are you using 
any caching DNS server? I've seen that happen in a company that 
was using an external DNS server from their isp, and in times of 
extreme internet link usage the problem would appear. Heck, even 
their sql queries would block for around 60 seconds in the 
Windows machines they used. It must have left lots of SAP 
consultants scratching their heads. :)


Pedro

On 8 Oct 2001, at 9:02, Larry McVoy wrote:


> However, web browsing sucks.  On about 80% of all links, there is a noticable
> hesitation, between 1-15 seconds, as it looks up the name and as it fetches
> the first page.  After that point, that site will appear to be OK.
> 

Re: [OT] testing internet performance, esp latency/drops?

[Henning P. Schmiedehausen]

Larry McVoy <lm@bitmover.com> writes:

>However, web browsing sucks.  On about 80% of all links, there is a noticable
>hesitation, between 1-15 seconds, as it looks up the name and as it fetches
>the first page.  After that point, that site will appear to be OK.

This means, that 80% of all web site operators are too dumb to
configure a web server. Is it possible that your brand spanking new T1
addresses have no reverse name resolution? Are you using a proxy?

Look:

lm@bitmover   IP 1.2.3.4
    |
    |
"i want to surf to http://www.surf.xxx/"
    |
    |
    v
Website "www.surf.xxx"

"connect from 1.2.3.4" -> "Lookup for 4.3.2.1.IN-ADDR.ARPA"
                                       |
                                       |
                            1-15 Second timeout, "addr not found"
                                       |
                                       |
                                       v
                            send HTTP page to 1.2.3.4 anyway.


>Before I wander off to write a test for this, I'm wondering if anyone 
>knows of a test suite or a methodology which works.  I was thinking 

This has nothing to do with OSes, Linux or even a kernel. It is just
that most people on this net (and also most people who operate servers
or work at ISPs) are complete idiots that neither know how to operate
a webserver (set it to "name lookup off" and do address resolving for
web statistics offline) or an ip-space (how to setup a reverse
resolution).

I did consulting for an ISP that has > 50 Class C networks and not
_one_ had right forward and reverse resolution [1] as I arrived there.

	Regards
		Henning

[1] Now it has. ;-)      


-- 
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen       -- Geschaeftsfuehrer
INTERMETA - Gesellschaft fuer Mehrwertdienste mbH     hps@intermeta.de

Am Schwabachgrund 22  Fon.: 09131 / 50654-0   info@intermeta.de
D-91054 Buckenhof     Fax.: 09131 / 50654-20