All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jakub Narebski <jnareb@gmail.com>
To: Shawn Pearce <spearce@spearce.org>
Cc: Johannes Sixt <j.sixt@viscovery.net>,
	Clemens Buchacher <drizzd@aon.at>, Jeff King <peff@peff.net>,
	Junio C Hamano <gitster@pobox.com>,
	Jonathan Nieder <jrnieder@gmail.com>,
	netroby <hufeng1987@gmail.com>,
	Git Mail List <git@vger.kernel.org>,
	Tomas Carnecky <tom@dbservice.com>
Subject: Re: New Feature wanted: Is it possible to let git clone continue last break point?
Date: Fri, 04 Nov 2011 08:55:32 -0700 (PDT)	[thread overview]
Message-ID: <m3k47f51f0.fsf@localhost.localdomain> (raw)
In-Reply-To: <CAJo=hJtsiEEHA33CQn1MCvb7vFv7uEF+U292YgBa7EWv7P8Jng@mail.gmail.com>

Shawn Pearce <spearce@spearce.org> writes:
> On Fri, Nov 4, 2011 at 02:35, Johannes Sixt <j.sixt@viscovery.net> wrote:
> > Am 11/4/2011 9:56, schrieb Clemens Buchacher:

> > > Cache ... not the pack but the information
> > >    to re-create it...
> >
> > It has been discussed. It doesn't work. Because with threaded pack
> > generation, the resulting pack is not deterministic.
> 
> The information to create a pack for a repository with 2M objects
> (e.g. Linux kernel tree) is *at least* 152M of data. This is just a
> first order approximation of what it takes to write out the 2M SHA-1s,
> along with say a 4 byte length so you can find given an offset
> provided by the client roughly where to resumse in the object stream.
> This is like 25% of the pack size itself. Ouch.

Well, perhaps caching a few most popular packs in some kind of cache
(packfile is saved to disk as it is streamed if we detect that it will
be large), indexing by WANT / HAVE?
 
> This data is still insufficient to resume from. A correct solution
> would allow you to resume in the middle of an object, which means we
> also need to store some sort of indicator of which representation was
> chosen from an existing pack file for object reuse. Which adds more
> data to the stream. And then there is the not so simple problem of how
> to resume in the middle of an object that was being recompressed on
> the fly, such as a large loose object.

Well, so you wouldn't be able to just concatenate packs^W received
data.  Still it should be possible to "repair" halfway downloaded
partial pack...
 
Just my 2 eurocents^W groszy.
-- 
Jakub Narębski

  reply	other threads:[~2011-11-04 15:55 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <CAEZo+gfKVY-YgMjd=bEYzRV4-460kqDik-yVcQ9Xs=DoCZOMDg@mail.gmail.com>
2011-10-31  2:28 ` New Feature wanted: Is it possible to let git clone continue last break point? netroby
2011-10-31  4:00   ` Tay Ray Chuan
2011-10-31  9:07   ` Jonathan Nieder
2011-10-31  9:16     ` netroby
2011-11-02 22:06     ` Jeff King
2011-11-02 22:41       ` Junio C Hamano
2011-11-02 23:27         ` Jeff King
2011-11-03  0:06           ` Shawn Pearce
2011-11-03  2:42             ` Jeff King
2011-11-03  4:19               ` Shawn Pearce
2011-11-04  8:56                 ` Clemens Buchacher
2011-11-04  9:35                   ` Johannes Sixt
2011-11-04 14:22                     ` Shawn Pearce
2011-11-04 15:55                       ` Jakub Narebski [this message]
2011-11-04 16:05                       ` Nguyen Thai Ngoc Duy
2011-11-05 10:00                       ` Clemens Buchacher
2011-10-31  9:14   ` Jakub Narebski
2011-10-31 12:49     ` Michael Schubert

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=m3k47f51f0.fsf@localhost.localdomain \
    --to=jnareb@gmail.com \
    --cc=drizzd@aon.at \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=hufeng1987@gmail.com \
    --cc=j.sixt@viscovery.net \
    --cc=jrnieder@gmail.com \
    --cc=peff@peff.net \
    --cc=spearce@spearce.org \
    --cc=tom@dbservice.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.