RFC: encourage switch away from heanet.sf.net

RFC: encourage switch away from heanet.sf.net

[Rolf Leggewie]

Hi,

it occurred to me today that Sourceforge is not using that intermediate 
page anymore where it lists all the mirrors for a file to be downloaded. 
    It used to be necessary that you click on one of the mirrors there 
to actually start the download from the browser. Instead they 
automatically 302 you now.

Mirrors are there for a reason.  Unfortunately, sf.net forced us in the 
past to use heanet.sourceforge.net for download since it was the only 
complete mirror.  This is not the case any more, we can use 
downloads.sf.net but still be sure to get the files we need (assuming 
that sf.net will only redirect us to mirrors that *do* have the file).

I therefore suggest to change bitbake.conf in the following way and 
request your comments on the matter (especially from hrw since RP 
suggested you are the most knowledgeable about sf.net and the 
associated, past problems).  This would set the IMHO sane default from 
which users can still deviate via local.conf and site.conf

- SOURCEFORGE_MIRROR = "http://heanet.dl.sourceforge.net/sourceforge"
+ SOURCEFORGE_MIRROR = "http://downloads.sf.net"

For what it is worth, I for one have switched (via site.conf so far).

Regards

Rolf

Re: RFC: encourage switch away from heanet.sf.net

[Michael 'Mickey' Lauer]

Rolf Leggewie wrote:
> it occurred to me today that Sourceforge is not using that intermediate
> page anymore where it lists all the mirrors for a file to be downloaded.
>     It used to be necessary that you click on one of the mirrors there
> to actually start the download from the browser. Instead they 
> automatically 302 you now.

If that's true, then +1 from me.

Regards,

:M:
-- 
Michael 'Mickey' Lauer | IT-Freelancer | http://www.vanille-media.de

Re: RFC: encourage switch away from heanet.sf.net

[Rolf Leggewie]

Michael 'Mickey' Lauer wrote:
>> Instead they automatically 302 you now.
> 
> If that's true, then +1 from me.

It seems they do.  And a sample session I wanted to provide as proof 
apparently also revealed how they deal with the situation that a mirror 
does not have the file.  I post only the relevant lines here.

$ wget http://downloads.sf.net/gakusei/qpf-unismall-100_1.0.0-r0_all.ipk
HTTP Anforderung gesendet, warte auf Antwort... 302 Found
Platz: 
http://downloads.sourceforge.net/gakusei/qpf-unismall-100_1.0.0-r0_all.ipk 
HTTP Anforderung gesendet, warte auf Antwort... 302 Found
Platz: 
http://ovh.dl.sourceforge.net/sourceforge/gakusei/qpf-unismall-100_1.0.0-r0_all.ipk 
HTTP Anforderung gesendet, warte auf Antwort... 200 OK

302 twice, then 200 from ovh.  Download successful.

$ rm qpf-unismall-100_1.0.0-r0_all.ipk
$ wget http://downloads.sf.net/gakusei/qpf-unismall-100_1.0.0-r0_all.ipk
HTTP Anforderung gesendet, warte auf Antwort... 302 Found
Platz: 
http://downloads.sourceforge.net/gakusei/qpf-unismall-100_1.0.0-r0_all.ipk 
HTTP Anforderung gesendet, warte auf Antwort... 302 Found
Platz: 
http://puzzle.dl.sourceforge.net/sourceforge/gakusei/qpf-unismall-100_1.0.0-r0_all.ipk 
HTTP Anforderung gesendet, warte auf Antwort... 302 Found
Platz: 
http://prdownloads.sourceforge.net/gakusei/qpf-unismall-100_1.0.0-r0_all.ipk?download&failedmirror=puzzle.dl.sourceforge.net 
HTTP Anforderung gesendet, warte auf Antwort... 301 Moved Permanently
Platz: 
http://downloads.sourceforge.net/gakusei/qpf-unismall-100_1.0.0-r0_all.ipk?download&failedmirror=puzzle.dl.sourceforge.net 
HTTP Anforderung gesendet, warte auf Antwort... 302 Found
Platz: 
http://osdn.dl.sourceforge.net/sourceforge/gakusei/qpf-unismall-100_1.0.0-r0_all.ipk 
HTTP Anforderung gesendet, warte auf Antwort... 200 OK

302 twice to puzzle which does not have the file and thus 302's again to 
prdownloads.  There is an added attribute that this file failed for the 
mirror and I assume that downloads.sf.net will act on that information 
for future downloads.  prdownloads sets a 301, we get another 302 and 
then osdn answers the request.  In some cases quite a few hops but it 
seems to work.

Out of curiousity, I tested out what happens with files that are not on 
the sf.net network.  It seems that one of two things happens.  If heanet 
is the first mirror being redirected to and it cannot find the file, it 
will answer with a 404 itself.  Otherwise, if some other mirror is used 
first, prdownloads will redirect to osdn which will redirect again to 
prdownloads with &failedmirror=osdn.dl.sourceforge.net.  A 404 is the 
answer.

Re: RFC: encourage switch away from heanet.sf.net

[Richard Purdie]

On Sat, 2006-12-09 at 11:35 +0100, Rolf Leggewie wrote:
> Michael 'Mickey' Lauer wrote:
> >> Instead they automatically 302 you now.
> > 
> > If that's true, then +1 from me.
> 
> It seems they do.  And a sample session I wanted to provide as proof 
> apparently also revealed how they deal with the situation that a mirror 
> does not have the file.  I post only the relevant lines here.

I'm ok with trying this. I suggest checking with hrw and if he's ok with
it, apply it to .dev. Thanks for looking into it.

Cheers,

Richard

Re: RFC: encourage switch away from heanet.sf.net

[Marcin Juszkiewicz]

Dnia niedziela, 10 grudnia 2006 23:30, Richard Purdie napisał:

> I suggest checking with hrw and if he's ok with it, apply it to .dev.

Thx for checking - I just started fetch build for every recipe which fetch 
from SOURCEFORGE_MIRROR with your change applied.

-- 
JID: hrw-jabber.org
OpenEmbedded developer/consultant

          Cats are like people: the females are the prickly ones
          and the males are good-natured idiots. 

Re: RFC: encourage switch away from heanet.sf.net

[Rolf Leggewie]

Rolf Leggewie wrote:

> - SOURCEFORGE_MIRROR = "http://heanet.dl.sourceforge.net/sourceforge"
> + SOURCEFORGE_MIRROR = "http://downloads.sf.net"

pushed after successful tests from Marcin.