Sparse checkouts

Sparse checkouts

[Uri Moszkowicz]

Hi,
I'm working with a repository with a very large number of files and Git is 
appealing because it officially supports sparse checkouts unlike the other 
DVCS tools. However, all of the usage examples that I've come across have 
you checkout the full repository and then prune the undesired files as such:

  git clone <dir>
  git config core.sparsecheckout true
  echo "<dir>/" > .git/info/sparse-checkout
  git read-tree -m -u HEAD

I tried adding "-n" to the clone command but then none of the missing 
directories are checked out by read-tree. I can manually check them out 
and everything seems to work fine but I can also manually check out a bunch 
of other directories and read-tree, reset, etc all seem to ignore the extra 
directory even though they aren't specified in the sparse-checkout file. Is 
this use mode just not well supported by git yet or am I missing something? 
I'm new to Git so I don't expect to be able to figure it out easily and I 
haven't been able to find the answer elsewhere.

Thanks,
Uri

Re: Sparse checkouts

[Nguyen Thai Ngoc Duy]

On Sun, Nov 7, 2010 at 9:04 AM, Uri Moszkowicz <uri@4refs.com> wrote:
> Hi,
> I'm working with a repository with a very large number of files and Git is
> appealing because it officially supports sparse checkouts unlike the other
> DVCS tools. However, all of the usage examples that I've come across have
> you checkout the full repository and then prune the undesired files as such:
>
>  git clone <dir>
>  git config core.sparsecheckout true
>  echo "<dir>/" > .git/info/sparse-checkout
>  git read-tree -m -u HEAD

You can replace the last command with "git checkout" (no additional argument)

> I tried adding "-n" to the clone command but then none of the missing
> directories are checked out by read-tree.

Hmm.. it works fine here. Can you post the exact sequence of commands you used?

> I can manually check them out
> and everything seems to work fine but I can also manually check out a bunch
> of other directories and read-tree, reset, etc all seem to ignore the extra
> directory even though they aren't specified in the sparse-checkout file

Sparse checkout mode does not stop you from checking out non-sparse
files. But all those files are not "seen" by git because of sparse
checkout mode. They are ignored most of the time, except when git is
going to overwrite them.

> Is this use mode just not well supported by git yet or am I missing something?

Sparse checkout support is pretty basic. Any suggestions are welcome.
-- 
Duy

Re: Sparse checkouts

[Nguyen Thai Ngoc Duy]

On Mon, Nov 8, 2010 at 11:29 PM, Uri Moszkowicz <uri@4refs.com> wrote:
> Thanks for the reply. "git checkout" does seem to work. The exact commands
> that I used are shown below:
>>  git clone -n <dir>
>>  git config core.sparsecheckout true
>>  echo "<dir>/" > .git/info/sparse-checkout
>>  git read-tree -m -u HEAD
>
> It would be nice if sparse checkout mode were more intuitive. A start would
> be making it on by default. I don't know why the config bit has to be set.

Because there is overhead in this feature and I guess only a tiny
percent of users needs it. So it's not enabled by default. When I make
the overhead really small, maybe I can convince Junio to enable it by
default.

> Second, when it is enabled, then checkout commands could add the given
> directories to the sparse checkout list. A checkout command with no sparse
> checkout file would result in "*" being added to the file.

Earlier versions of sparse checkout supported that, sparse rules were
much simpler back then. But because you can now set complex rules in
sparse checkout file (keep directory a, but not a/b, but keep a/b/c,
for example), command line support to edit the file automatically may
give unexpected result.

But I think command line support to just clear sparse checkout file
and replace it with "*" is good.

> Third, when
> directories are removed it would be nice if they were removed from the
> sparse checkout file. I don't know if you can trap "rm" or if you need to
> add a new command like "git remove".

That can't be done reliably. If you put pattern "a/*" in sparse
checkout file, then remove a/b, git is not smart enough to put "!a/b"
in sparse checkout file.

> Third, other commands should try to
> bring the tree in sync with the sparse checkout list. read-tree and checkout
> should remove directories which aren't in the list, perhaps throwing an
> error if there were changes in those directories.

Let me think about this. Not all commands should try to sync, but
perhaps some of them should.

Thank you for your feedback.
-- 
Duy