Re: resume downloads

[Sitaram Chamarty <sitaramc@gmail.com>]

On 05/11/2015 03:49 AM, Junio C Hamano wrote:
> The current thinking is to model this after the "repo" tool.
> Prepare a reasonably up-to-date bundle file on the server side,

<shameless plug (but not "commercial")>

For people using gitolite, the server side issues of generating a
reasonably up-to-date bundle *and* enabling it for resumable download
using rsync (with the same ssh key used to gain gitolite access), can
all be handled by gitolite.

</shameless plug>

Of course the client side issues still remain; gitolite can't help
there.

> add a protocol capability to advertise the URL to download that
> bundle from upload-pack, and have "git clone" to pay attention to it.
> 
> Then, a "git clone" could become:
> 
>  - If the capability advertises such a prebuilt bundle, spawn "curl"
>    or "wget" internally to fetch it. This can be resumed when the
>    connection goes down and will grab majority of the data necessary.
> 
>  - Extract the bundle into temporary area inside .git/refs/ to help
>    the next step.
> 
>  - Internally do a "git fetch" to the original server. Thanks to the
>    bundle transfer that has already happened, this step will become
>    a small incremental update.
> 
>  - Then prune away the temporary .git/refs/ refs that were in the
>    bundle, as these are not the up-to-date refs that exist on the
>    server side.
> 
> A few points that need to be considered by whoever is doing this
> are:
> 
>  - Where to download the bundle, so that after killing "git clone"
>    that is still in the bundle-download phase, the next invocation
>    of "git clone" can notice and resume the bundle-download;
> 
>  - What kind of transfer protocols do we want to support? Is http
>    and https from CDN sufficient? In other words, what exactly
>    should the new capability say to point at the prebuilt bundle?
> 
> These (and probably there are several others) are not something
> that "repo" does not have to worry about, but would become
> issues when we try to fold this into "git clone".
> 
> 
> 
> On Sun, May 10, 2015 at 2:55 PM, Thiago Farina <tfransosi@gmail.com> wrote:
>> Hi,
>>
>> Is there links to discussion on this? I mean, is resume downloads a
>> feature that is still being considered?
>>
>> Being able to download huge repos like WebKit, Linux, LibreOffice in
>> small parts seems like a good feature to me.
>>
>> --
>> Thiago Farina
>> --
>> To unsubscribe from this list: send the line "unsubscribe git" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>