Re: FW: Performance evaluation of high speed TCPs

Re: FW: Performance evaluation of high speed TCPs

[Douglas Leith]

[-- Attachment #1: Type: text/plain, Size: 742 bytes --]

>Seriously, we can't run the tests for every fix and bug report. 

Perhaps best to view it as returning a favour.  You may recall that we re-ran all our own experimental tests last year (all data and code available online at www.hamilton.ie/net/eval/) on discovering a previously unreported bug introduced by the linux folks when implementing bic.  Something similar has happened with importing htcp into linux.  

Seriously, where's the value in comparing buggy implementations - isn't that just a waste of all our time ?  If we are genuine about wanting to understand tcp performance then I think we just have to take the hit from issues such as this that are outside all of our control.

Doug

Hamilton Institute
www.hamilton.ie

[-- Attachment #2: Type: text/html, Size: 1200 bytes --]

Re: FW: Performance evaluation of high speed TCPs

[rhee]

Sure. Your comments about running the buggy implementation are well taken.
That is
why this type of reporting is helpful and we are committed to keep this
effort. Just that
it takes time to run the tests, and before we run a new set of tests, we
have to do some
batch of patches to reduce our effort level (but in this case of the HTCP
bug, rest assured
that we are running it now..it is just that there are a lot of other
things going on
that we have to catch a breath a little).

Then again, if we don't do the test and keep the report
up-to-date then it is difficult to find bugs as well...so these reportings
help us find
bugs and also improve TCP algorithms. (I hope our report did the same for
you).  Also
sometimes we are not motivated to find the bugs ourselves.

In fact, i contacted your student "Baruch" one month and half before  we
posted our
report -- it was CCed in the netdev mailing list as well and we gave him
login and
passwd on our result website (at that time we were just about to write the
report)
and we have not heard from your guys until just one week ago. At least we
did try to
make sure we are running a buggy version.




>>Seriously, we can't run the tests for every fix and bug report.
>
> Perhaps best to view it as returning a favour.  You may recall that we
> re-ran all our own experimental tests last year (all data and code
> available online at www.hamilton.ie/net/eval/) on discovering a previously
> unreported bug introduced by the linux folks when implementing bic.
> Something similar has happened with importing htcp into linux.
>
> Seriously, where's the value in comparing buggy implementations - isn't
> that just a waste of all our time ?  If we are genuine about wanting to
> understand tcp performance then I think we just have to take the hit from
> issues such as this that are outside all of our control.
>
> Doug
>
> Hamilton Institute
> www.hamilton.ie
>

Re: FW: Performance evaluation of high speed TCPs

[Douglas Leith]

[-- Attachment #1: Type: text/plain, Size: 1262 bytes --]

Injong,

>In fact, i contacted your student "Baruch" one month and half before  we posted our
report -- it was CCed in the netdev mailing list as well and we gave him login and
passwd on our result website (at that time we were just about to write the report)
and we have not heard from your guys until just one week ago. At least we did try to
make sure we are running a buggy version.

We have no record of receiving such an email.  Just a mix-up I guess.

Doug


>>Seriously, we can't run the tests for every fix and bug report.
>
> Perhaps best to view it as returning a favour.  You may recall that we
> re-ran all our own experimental tests last year (all data and code
> available online at www.hamilton.ie/net/eval/) on discovering a previously
> unreported bug introduced by the linux folks when implementing bic.
> Something similar has happened with importing htcp into linux.
>
> Seriously, where's the value in comparing buggy implementations - isn't
> that just a waste of all our time ?  If we are genuine about wanting to
> understand tcp performance then I think we just have to take the hit from
> issues such as this that are outside all of our control.
>
> Doug
>
> Hamilton Institute
> www.hamilton.ie
>




[-- Attachment #2: Type: text/html, Size: 1885 bytes --]

Re: FW: Performance evaluation of high speed TCPs

[rhee]

Let's get off this e2e list for this discussion. It is really unnecessary
to use this list for
this discussion. I don't understand why you keep sending your email to
this list even
though we are seating next to each other in the same conference. Isn't
this amusing
 or abusing of this mailing list? :-)
-- 
to others in the list, Doug and I are attending pfldnet in japan now..




> Injong,
>
>>In fact, i contacted your student "Baruch" one month and half before  we
>> posted our
> report -- it was CCed in the netdev mailing list as well and we gave him
> login and
> passwd on our result website (at that time we were just about to write the
> report)
> and we have not heard from your guys until just one week ago. At least we
> did try to
> make sure we are running a buggy version.
>
> We have no record of receiving such an email.  Just a mix-up I guess.
>
> Doug
>
>
>>>Seriously, we can't run the tests for every fix and bug report.
>>
>> Perhaps best to view it as returning a favour.  You may recall that we
>> re-ran all our own experimental tests last year (all data and code
>> available online at www.hamilton.ie/net/eval/) on discovering a
>> previously
>> unreported bug introduced by the linux folks when implementing bic.
>> Something similar has happened with importing htcp into linux.
>>
>> Seriously, where's the value in comparing buggy implementations - isn't
>> that just a waste of all our time ?  If we are genuine about wanting to
>> understand tcp performance then I think we just have to take the hit
>> from
>> issues such as this that are outside all of our control.
>>
>> Doug
>>
>> Hamilton Institute
>> www.hamilton.ie
>>
>
>
>
>

Re: FW: Performance evaluation of high speed TCPs

[Ian McDonald]

>  Seriously, where's the value in comparing buggy implementations - isn't
> that just a waste of all our time ?  If we are genuine about wanting to
> understand tcp performance then I think we just have to take the hit from
> issues such as this that are outside all of our control.
>
A real part of the problem here is that the Linux doesn't have a full
TCP testing suite and doesn't have build checking to check for
regressions in TCP variants. As I understand the only thing tested in
nightly builds is throughput for the default TCP.

Stephen Hemminger has done some work on TCP Probes but this is where I
think real progress could be made in improving Linux TCP. I may get
around to doing this myself at some point in my research but would
welcome other people doing it also!

Ian
--
Ian McDonald
http://wand.net.nz/~iam4
WAND Network Research Group
University of Waikato
New Zealand