Re: git subtree as a solution to partial cloning?

[Avery Pennarun <apenwarr@gmail.com>]

On Mon, May 25, 2009 at 2:28 PM, Asger Ottar Alstrup <asger@area9.dk> wrote:
> On Mon, May 25, 2009 at 7:54 PM, Avery Pennarun <apenwarr@gmail.com> wrote:
>> On Mon, May 25, 2009 at 1:35 PM, Asger Ottar Alstrup <asger@area9.dk> wrote:
>>> So a poor mans system could work like this:
>>>
>>> - A reduced repository is defined by a list of paths in a file, I
>>> guess with a format similar to .gitignore
>>
>> Are you sure you want to define the list with exclusions instead of
>> inclusions?  I don't really know your use case.
>
> Since the .gitignore format supports !, I believe that should not make
> much of a difference.
>
>> Anyway, if you're using git filter-branch, it'll be up to you to fix
>> the index to contain the list of files you want. (See man
>> git-filter-branch)
>
> Yes, sure, and that is why I asked whether there is some tool in git
> that can give a list of concrete files surviving a .gitignore list of
> patterns.

Well, the problem here is with the definition of "concrete file."  If
you're using git filter-branch --index-filter (which is much faster
than --tree-filter), then your trees won't be checked out at all.  And
thus there is the open question of exactly what list of files you want
to use.  It's unlikely that any existing tool will do it exactly the
way you want (although I could be wrong).

In any case, what you'd probably do is something like git ls-files
--cached | perlscript, where your perlscript does whatever you want to
the file list.

> Thanks. OK, I see now that filter-branch will not destroy the original
> repository. That is not at all obvious from reading the man page, when
> the very first sentence says that it will rewrite history. But the
> main point of this exercise is to reduce the size of the reduced
> repository so that it can be transferred effectively. So after
> filter-branch, I guess I would run clone afterwards to make the new,
> smaller repository, and then the question becomes: Will clone reuse
> and prune packs intelligently?

filter-branch will destroy the history of the current branch.  But if
you make a new branch first, you'll be okay.

You seem to be giving the concept of "packs" a bit too much weight.
Packs are just groups of objects.  AFAIK, cloning and fetching
generally produces entirely new packs for each client.

clone is quite intelligent; in fact, if you clone the repository on
your local machine, it's so intelligent that it'll hardlink the packs
instead of copying them and it'll take virtually no space at all!

But you don't need to copy the whole repository unless you want to.
You can retrieve just the one, stripped-down branch from a client with
something like this:

   mkdir myproj
   cd myproj
   git init
   git fetch server:whatever.git my-stripped-down-branchname
   git checkout -b master FETCH_HEAD

Have fun,

Avery