[PATCH 15/16] checkout: add new options to support narrow checkout

[PATCH 15/16] checkout: add new options to support narrow checkout

[Nguyễn Thái Ngọc Duy]

These options include:

 --full: return to full checkout (default)
 --path: narrow checkout to some areas according to given spec
 --add-path/--remove-path: adjust current narrow checkout

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git-checkout.txt |   68 +++++++++++++++++++++++++++++++++-
 builtin-checkout.c             |   44 ++++++++++++++++++++++
 t/t2011-checkout-narrow.sh     |   80 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 190 insertions(+), 2 deletions(-)
 create mode 100755 t/t2011-checkout-narrow.sh

diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
index 82e154d..2ae483b 100644
--- a/Documentation/git-checkout.txt
+++ b/Documentation/git-checkout.txt
@@ -8,8 +8,10 @@ git-checkout - Checkout a branch or paths to the working tree
 SYNOPSIS
 --------
 [verse]
-'git checkout' [-q] [-f] [--track | --no-track] [-b <new_branch> [-l]] [-m] [<branch>]
-'git checkout' [-f|--ours|--theirs|-m|--conflict=<style>] [<tree-ish>] [--] <paths>...
+'git checkout' [-q] [-f] [--track | --no-track] [-b <new_branch> [-l]] [-m]
+	  [<narrow options>] [<branch>]
+'git checkout' [-f|--ours|--theirs|-m|--conflict=<style>] [<tree-ish>]
+	  [<narrow options>] [--] <paths>...
 
 DESCRIPTION
 -----------
@@ -34,6 +36,10 @@ used to specify a specific tree-ish (i.e. commit, tag or tree)
 to update the index for the given paths before updating the
 working tree.
 
+<narrow options> include --full, --path, --add-path and --remove-path.
+The last three require narrow spec. Please refer to "narrow checkout"
+section to have more information about this mode.
+
 The index may contain unmerged entries after a failed merge.  By
 default, if you try to check out such an entry from the index, the
 checkout operation will fail and nothing will be checked out.
@@ -108,6 +114,22 @@ exlicitly give a name with '-b' in such a case.
 	However, with this option, a three-way merge between the current
 	branch, your working tree contents, and the new branch
 	is done, and you will be on the new branch.
+
+--full::
+	Quit narrow checkout mode. Return to full checkout.
+
+--path=<narrow_spec>::
+	Re-apply new narrow spec on current working directory to
+	form new checkout area.
+
+--add-path=<narrow_spec>::
+	Checkout more areas specified by narrow spec to current
+	checkout area.
+
+--remove-path=<narrow_spec>::
+	Narrow checkout area by removing files specified by narrow spec
+	from current checkout area. This operation will fail if there
+	is unmerged or modified files in the removing areas.
 +
 When a merge conflict happens, the index entries for conflicting
 paths are left unmerged, and you need to resolve the conflicts
@@ -171,6 +193,48 @@ the reflog for HEAD where you were, e.g.
 $ git log -g -2 HEAD
 ------------
 
+Narrow checkout
+---------------
+
+Normally when you do checkout a branch, your working directory
+will be fully populated. In some situations, you just need to
+work on certain files, no full checkout is required. Narrow
+checkout is a mode that limits checkout area according to your
+rules.
+
+Because narrow checkout uses new index format, it will be
+incompatible with git prior to 1.6.0. In order to make your
+working directory work with those versions, you can use `git
+checkout --full` to return to normal mode (and compatible index
+format).
+
+Narrow works by applying your rules to the index, marking which
+file you need and which file you need not. Modified/Unmerged
+files cannot be marked unneeded. Unnecessary files will be
+removed from working directory.  Note that after this step,
+removed files can still be added to working directory when they
+are needed by any git command. For example, when you have a merge
+conflict, conflicted files will be checked out on working
+directory and will no longer be marked "unneeded".
+
+New files after merges will always be "needed". You can also
+apply rules when switching branches to avoid unwanted new files.
+
+Files that are marked "no-checkout" will be treated like entries
+with "assume-unchanged bit" (see linkgit:git-update-index[1]). In
+short, Git will never look for those files in working directory
+no matter whether they exist in working directory.
+
+You can apply your rules at once with --path option, or do it
+incrementally with --add-path and --remove-path.
+
+Narrow spec will be used to specify how you want to narrow your
+checkout. It is a list of pathspecs separated by colons. Each
+patchspec specifies what files should be checked out on working
+directory. Pathspec can contain wildcards and is relative to
+current working directory. Usually asterisk (*) does not match
+slashes. If a pathspec is prefixed by a plus sign (+), then
+any asterisk will match anything, even slashes.
 
 EXAMPLES
 --------
diff --git a/builtin-checkout.c b/builtin-checkout.c
index c7b0aad..b6bc3c5 100644
--- a/builtin-checkout.c
+++ b/builtin-checkout.c
@@ -33,6 +33,12 @@ struct checkout_opts {
 	const char *new_branch;
 	int new_branch_log;
 	enum branch_track track;
+
+	const char *prefix;
+	char *new_path;
+	char *add_path;
+	char *remove_path;
+	int no_narrow_checkout;
 };
 
 static int post_checkout_hook(struct commit *old, struct commit *new,
@@ -412,6 +418,24 @@ static int merge_working_tree(struct checkout_opts *opts,
 		tree = parse_tree_indirect(new->commit->object.sha1);
 		init_tree_desc(&trees[1], tree->buffer, tree->size);
 
+		topts.narrow_prefix = opts->prefix;
+		if (opts->no_narrow_checkout) {
+			/* leave narrow_spec NULL */
+			topts.new_narrow_path = 1;
+		}
+		else if (opts->new_path) {
+			topts.narrow_spec = opts->new_path;
+			topts.new_narrow_path = 1;
+		}
+		else if (opts->add_path) {
+			topts.narrow_spec = opts->add_path;
+			topts.add_narrow_path = 1;
+		}
+		else if (opts->remove_path) {
+			topts.narrow_spec = opts->remove_path;
+			topts.remove_narrow_path = 1;
+		}
+
 		ret = unpack_trees(2, trees, &topts);
 		if (ret == -1) {
 			/*
@@ -600,6 +624,10 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 		OPT_BOOLEAN('m', "merge", &opts.merge, "merge"),
 		OPT_STRING(0, "conflict", &conflict_style, "style",
 			   "conflict style (merge or diff3)"),
+		OPT_BOOLEAN(0, "full", &opts.no_narrow_checkout, "quit sparse checkout"),
+		OPT_STRING(0, "path", &opts.new_path, "prefixes", "apply new narrow checkout path"),
+		OPT_STRING(0, "add-path", &opts.add_path, "prefixes", "add more checkout area"),
+		OPT_STRING(0, "remove-path", &opts.remove_path, "prefixes", "narrow checkout area"),
 		OPT_END(),
 	};
 	int has_dash_dash;
@@ -610,6 +638,7 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 	git_config(git_checkout_config, NULL);
 
 	opts.track = BRANCH_TRACK_UNSPECIFIED;
+	opts.prefix = prefix;
 
 	argc = parse_options(argc, argv, options, checkout_usage,
 			     PARSE_OPT_KEEP_DASHDASH);
@@ -639,6 +668,18 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 	if (!opts.new_branch && (opts.track != git_branch_track))
 		die("git checkout: --track and --no-track require -b");
 
+	if (((opts.no_narrow_checkout ? 1 : 0) +
+	     (opts.new_path ? 1 : 0) +
+	     (opts.add_path ? 1 : 0) +
+	     (opts.remove_path ? 1 : 0)) > 1)
+		die("git checkout: --path, --full, --add-path and --remove-path are incompatible");
+
+	if (opts.new_branch && (opts.add_path || opts.remove_path))
+		die("git checkout: --add-path and --remove-path should only be used on current branch");
+
+	if (opts.new_branch && opts.no_narrow_checkout)
+		die("git checkout: switching branch with --full does not make sense");
+
 	if (opts.force && opts.merge)
 		die("git checkout: -f and -m are incompatible");
 
@@ -732,6 +773,9 @@ no_reference:
 		if (1 < !!opts.writeout_stage + !!opts.force + !!opts.merge)
 			die("git checkout: --ours/--theirs, --force and --merge are incompatible when\nchecking out of the index.");
 
+		if (opts.no_narrow_checkout || opts.new_path || opts.add_path || opts.remove_path)
+			die("git checkout: updating paths is incompatible with setting sparse checkout");
+
 		return checkout_paths(source_tree, pathspec, &opts);
 	}
 
diff --git a/t/t2011-checkout-narrow.sh b/t/t2011-checkout-narrow.sh
new file mode 100755
index 0000000..77b99e1
--- /dev/null
+++ b/t/t2011-checkout-narrow.sh
@@ -0,0 +1,80 @@
+#!/bin/sh
+
+test_description='narrow checkout'
+
+. ./test-lib.sh
+
+test_expect_success setup '
+	mkdir work1 work2 work3
+	touch one two three
+	touch work1/one work2/two work3/three
+	git add one work1/one
+	git commit -m work1
+	git add two work2/two
+	git commit -m work2
+	git add three work3/three
+	git commit -m work3
+'
+
+test_expect_success '--full on no-narrow checkout' '
+	git checkout --full
+'
+
+test_expect_success '--full and --path incompatible' '
+	test_must_fail git checkout --full --path=work1
+'
+
+test_expect_success 'limit worktree to work1 and work2' '
+	git checkout --path="work1/*:work2/*" &&
+	test -f work1/one &&
+	test -f work2/two &&
+	! test -f work3/three
+'
+
+test_expect_success 'update worktree to work2 and work3' '
+	git checkout --path="work2/*:work3/*" &&
+	! test -f work1/one &&
+	test -f work2/two &&
+	test -f work3/three
+'
+
+test_expect_success 'update narrow prefix with modification' '
+	echo modified >> work2/two &&
+	git checkout --path="work1/*:work2/*" &&
+	test -f work1/one &&
+	test -f work2/two &&
+	! test -f work3/three &&
+	grep -q modified work2/two
+'
+
+test_expect_success 'update narrow should not lose modification' '
+	! git checkout --path="work1/*:work3/*" &&
+	test -f work1/one &&
+	test -f work2/two &&
+	! test -f work3/three &&
+	grep -q modified work2/two
+'
+
+test_expect_success 'widen checkout area' '
+	git checkout --add-path="work3/*" &&
+	test -f work1/one &&
+	test -f work2/two &&
+	test -f work3/three
+'
+
+test_expect_success 'narrow checkout area' '
+	git checkout --remove-path="work3/*" &&
+	test -f work1/one &&
+	test -f work2/two &&
+	! test -f work3/three
+'
+
+test_expect_success 'exit narrow checkout' '
+	git checkout --full &&
+	test -f work1/one &&
+	test -f work2/two &&
+	test -f work3/three &&
+	test one
+'
+
+test_done
-- 
1.6.0.96.g2fad1.dirty

Re: [PATCH 15/16] checkout: add new options to support narrow checkout

[Jakub Narebski]

I will comment only on the documentation...

Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:

> These options include:
> 
>  --full: return to full checkout (default)
>  --path: narrow checkout to some areas according to given spec
>  --add-path/--remove-path: adjust current narrow checkout

Hmmm... I'm not sure if such formatting of commit message, with
body being not independent on the subject (first line of commit
message) is a good idea.

> diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
[...]
> +--full::
> +	Quit narrow checkout mode. Return to full checkout.
> +
> +--path=<narrow_spec>::
> +	Re-apply new narrow spec on current working directory to
> +	form new checkout area.
> +
> +--add-path=<narrow_spec>::
> +	Checkout more areas specified by narrow spec to current
> +	checkout area.
> +
> +--remove-path=<narrow_spec>::
> +	Narrow checkout area by removing files specified by narrow spec
> +	from current checkout area. This operation will fail if there
> +	is unmerged or modified files in the removing areas.

Subversion from version 1.5 supports something called "Sparse directories",
http://blogs.open.collab.net/svn/2007/06/sparse-director.html (as pointed
out by Bjorn Steinbrink (doener_) on #git).  Subversion has the following
options for 'svn checkout':

  -N::
  --non-recursive::
         Checkout only the main directory of the trunk and not the
         sub-directories
   --depth=empty::
         Updates will not pull in any files or subdirectories not
         already present.
   --depth=files::
         Updates will pull in any files not already present,
         but not subdirectories.
   --depth=immediates::  
         Updates will pull in any files or subdirectories not
         already present; the subdirectories will have depth=empty.
   --depth=infinity::
         Updates will pull in any files or subdirectories not
         already present; the subdirectories will have depth=infinity.
         Equivalent to today's default update behavior.

I'm not sure if those ways of limiting are worth implementing, but
I guess that they are at least worth thinking about.

> +Narrow checkout
> +---------------
> +
> +Normally when you do checkout a branch, your working directory
> +will be fully populated. In some situations, you just need to
> +work on certain files, no full checkout is required. Narrow
> +checkout is a mode that limits checkout area according to your
> +rules.

Perhaps s/required/needed/?

> +
> +Because narrow checkout uses new index format, it will be
> +incompatible with git prior to 1.6.0. In order to make your
> +working directory work with those versions, you can use `git
> +checkout --full` to return to normal mode (and compatible index
> +format).

Very nice to have those compatibility concerns mentioned upfront.
In short: new feature, wouldn't work with git prior to 1.6.0.

I hope that nobody mistakes _working_ with repository with 
partial / sparse / narrow checkout, which requires >= 1.6.0,
with being able to clone / fetch such repository, where there
are no limitations, contrary for example to what was for submodule
support.

> +
> +Narrow works by applying your rules to the index, marking which
> +file you need and which file you need not. Modified/Unmerged
> +files cannot be marked unneeded. Unnecessary files will be
> +removed from working directory.  Note that after this step,
> +removed files can still be added to working directory when they
> +are needed by any git command. For example, when you have a merge
> +conflict, conflicted files will be checked out on working
> +directory and will no longer be marked "unneeded".

This paragraph I think need some more love...

So the "checkout rules" are meant to mark which paths are "wanted"
or "needed", and we would like to have in the checkout, and which
files are "unwanted" or "not needed" ("unneeded"?) and we want to
not have them present in working directory; something akin to accept
and deny rules, isn't it?

What are the rules, does all files except those marked explicitely
as needed are unneeded, or do you have to first mark all files as
unneeded?

How would the following table look like:

  working directory  || needed       | not needed    |
  ----------------------------------------------------
  file is absent     || checkout     | no change     |
  file is present    || no change    | removed       |
  file is modified   || conflict     | conflict?     |

> +
> +New files after merges will always be "needed". You can also
> +apply rules when switching branches to avoid unwanted new files.

Does it mean that if merge brings some new files, then those
files would be "needed" (without "no checkout" bit)?

What does it mean this sentence about switching branches:
how does partial/sparse/narrow checkout rules change when
switching to other branch (which, like 'html' and 'todo'
branches in git repository, can be completely unrelated)?

> +
> +Files that are marked "no-checkout" will be treated like entries
> +with "assume-unchanged bit" (see linkgit:git-update-index[1]). In
> +short, Git will never look for those files in working directory
> +no matter whether they exist in working directory.

Perhaps add that they would be marked with "no checkout" bit, and
refer to --no-checkout flag of git-update-index?  I'm not sure here
about that...

> +
> +You can apply your rules at once with --path option, or do it
> +incrementally with --add-path and --remove-path.

Nice.

> +
> +Narrow spec will be used to specify how you want to narrow your
> +checkout. It is a list of pathspecs separated by colons. Each
> +patchspec specifies what files should be checked out on working
> +directory. Pathspec can contain wildcards and is relative to
> +current working directory. Usually asterisk (*) does not match
> +slashes. If a pathspec is prefixed by a plus sign (+), then
> +any asterisk will match anything, even slashes.

First, does this mean that you can specify paths containing colons
(':') only using --add-path and --remove-path, or does it mean that
you cannot specify paths containg colon ':' (which should be rare)
at all as checkout limiter / checkout narrowing rule?

Second, wouldn't it be better to use '**' to match also '/'?
Changing meaning of '*' using per-path flag seems a bit bad.

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: [PATCH 15/16] checkout: add new options to support narrow checkout

[Jakub Narebski]

[I don't see mails I am replying to on GMane interface to git mailing
 list, so threads might be broken. Strange... Perhaps too long lines
 were cause of rejection by anti-SPAM vger filter?]

On Mon, 15 Sep 2008, Duy Nguyen wrote:
> On 09/15/2008 "Jakub Narebski" <jnareb@gmail.com> wrote:

> > > +Narrow works by applying your rules to the index, marking which
> > > +file you need and which file you need not. Modified/Unmerged
> > > +files cannot be marked unneeded. Unnecessary files will be
> > > +removed from working directory.  Note that after this step,
> > > +removed files can still be added to working directory when they
> > > +are needed by any git command. For example, when you have a merge
> > > +conflict, conflicted files will be checked out on working
> > > +directory and will no longer be marked "unneeded".
> > 
> > This paragraph I think need some more love...
> > 
> > So the "checkout rules" are meant to mark which paths are "wanted"
> > or "needed", and we would like to have in the checkout, and which
> > files are "unwanted" or "not needed" ("unneeded"?) and we want to
> > not have them present in working directory; something akin to accept
> > and deny rules, isn't it?
> 
> Yes. But rules will be gone, only the results remain. I don't
> save/reuse rules. 

Ah. I understand. The options are only to select which files to check-out
(which are "wanted"), and which we do not want and mark with "no-checkout"
bit ("unwanted").
 
> > What are the rules, does all files except those marked explicitely
> > as needed are unneeded, or do you have to first mark all files as
> > unneeded?
> > 
> > How would the following table look like:
> > 
> >   working directory  || needed       | not needed    |
> >   ----------------------------------------------------
> >   file is absent     || checkout     | no change     |
> >   file is present    || no change    | removed       |
> >   file is modified   || conflict     | conflict?     |
> 
> Looks better than my description. Though it would be "no change" for
> "file is modified/needed" case. There should be another line for
> unmerged entries.  

Now I am not sure about the line with 'file is modified', because even
in simple full checkout case there are different situations dealing with
checking out of index (and '-f' option), and switching to other branch
(and '-m' option).
 
Doesn't unmerged simply mean ignore "no-checkout" bit?

> > > +
> > > +New files after merges will always be "needed". You can also
> > > +apply rules when switching branches to avoid unwanted new files.
> > 
> > Does it mean that if merge brings some new files, then those
> > files would be "needed" (without "no checkout" bit)?

And as far as I understand the same for simple checkout, and for "2-way
merge" checkout.

> Yes.

Perhaps: new entries appearing in index have "no-checkout" bit unset
(cleared).  Or perhaps in addition, as clarification.

> > What does it mean this sentence about switching branches:
> > how does partial/sparse/narrow checkout rules change when
> > switching to other branch (which, like 'html' and 'todo'
> > branches in git repository, can be completely unrelated)?
> 
> Recall above I say rules are not saved. When you switch branches,
> files that are needed will still be and stay in workdir. New files
> will always appear in workdir ("needed"). If two branches are
> completely unrelated, all files will be new so you get full workdir.   

Thanks. Now I understand: new entries in index are always with
"no-checkout" bit unset, regardless how they got there.
  
> > > +Narrow spec will be used to specify how you want to narrow your
> > > +checkout. It is a list of pathspecs separated by colons. Each
> > > +patchspec specifies what files should be checked out on working
> > > +directory. Pathspec can contain wildcards and is relative to
> > > +current working directory. Usually asterisk (*) does not match
> > > +slashes. If a pathspec is prefixed by a plus sign (+), then
> > > +any asterisk will match anything, even slashes.
> > 
> > First, does this mean that you can specify paths containing colons
> > (':') only using --add-path and --remove-path, or does it mean that
> > you cannot specify paths containg colon ':' (which should be rare)
> > at all as checkout limiter / checkout narrowing rule?
> 
> You cannot do othat explicitly unfortunately. You can work-around using
> wildcard though. 

Couldn't you simply escape ':', i.e. write for example Git\:\:Tag.3pm,
or Eichten_PRD21\:313,1980_erratum.tex, or \:0.log, or perhaps
kmail/jnareb@gmail.com\:@pop.gmail.com\:995, or even something like
Mail/inbox/cur/1194202360.32296.mprnq\:2,S, in the same way like you
can escape other special characters, for example wildcard characters
like '\*' for '*' and '\?' for '?', and of course '\\' for '\'?
 
> > Second, wouldn't it be better to use '**' to match also '/'?
> > Changing meaning of '*' using per-path flag seems a bit bad.
> 
> It would be better. But I don't see any way but duplicating fnmatch()
> implementation and modify it to support '**' so I made a compromise.
> Will make another patch for '**' support and see how bloat the code
> will be.   

Well, the alternative would be to tell in commit message _why_ you
choose that (for implementation reasons), and perhaps give an example.

BTW. does '+' prefixed pathspec means that '?' can match '/', directory
separator?
 
-- 
Jakub Narebski
Poland

Re: [PATCH 15/16] checkout: add new options to support narrow checkout

[Nguyen Thai Ngoc Duy]

On 9/15/08, Jakub Narebski <jnareb@gmail.com> wrote:
> [I don't see mails I am replying to on GMane interface to git mailing
>   list, so threads might be broken. Strange... Perhaps too long lines
>   were cause of rejection by anti-SPAM vger filter?]

It was because I used zenbe.com (limited choices at work) to send
mails and did not realize until too late that zenbe.com did not encode
UTF-8 properly in mail headers. It made vger.k.o nervous and refuse to
accept my mails.

>  On Mon, 15 Sep 2008, Duy Nguyen wrote:
>  > On 09/15/2008 "Jakub Narebski" <jnareb@gmail.com> wrote:
>
>
> > > > +Narrow works by applying your rules to the index, marking which
>  > > > +file you need and which file you need not. Modified/Unmerged
>  > > > +files cannot be marked unneeded. Unnecessary files will be
>  > > > +removed from working directory.  Note that after this step,
>  > > > +removed files can still be added to working directory when they
>  > > > +are needed by any git command. For example, when you have a merge
>  > > > +conflict, conflicted files will be checked out on working
>  > > > +directory and will no longer be marked "unneeded".
>  > >
>  > > This paragraph I think need some more love...
>  > >
>  > > So the "checkout rules" are meant to mark which paths are "wanted"
>  > > or "needed", and we would like to have in the checkout, and which
>  > > files are "unwanted" or "not needed" ("unneeded"?) and we want to
>  > > not have them present in working directory; something akin to accept
>  > > and deny rules, isn't it?
>  >
>  > Yes. But rules will be gone, only the results remain. I don't
>  > save/reuse rules.
>
>
> Ah. I understand. The options are only to select which files to check-out
>  (which are "wanted"), and which we do not want and mark with "no-checkout"
>  bit ("unwanted").
>
>
>  > > What are the rules, does all files except those marked explicitely
>  > > as needed are unneeded, or do you have to first mark all files as
>  > > unneeded?
>  > >
>  > > How would the following table look like:
>  > >
>  > >   working directory  || needed       | not needed    |
>  > >   ----------------------------------------------------
>  > >   file is absent     || checkout     | no change     |
>  > >   file is present    || no change    | removed       |
>  > >   file is modified   || conflict     | conflict?     |
>  >
>  > Looks better than my description. Though it would be "no change" for
>  > "file is modified/needed" case. There should be another line for
>  > unmerged entries.
>
>
> Now I am not sure about the line with 'file is modified', because even
>  in simple full checkout case there are different situations dealing with
>  checking out of index (and '-f' option), and switching to other branch
>  (and '-m' option).
>
>  Doesn't unmerged simply mean ignore "no-checkout" bit?

Yes.

>  > > > +Narrow spec will be used to specify how you want to narrow your
>  > > > +checkout. It is a list of pathspecs separated by colons. Each
>  > > > +patchspec specifies what files should be checked out on working
>  > > > +directory. Pathspec can contain wildcards and is relative to
>  > > > +current working directory. Usually asterisk (*) does not match
>  > > > +slashes. If a pathspec is prefixed by a plus sign (+), then
>  > > > +any asterisk will match anything, even slashes.
>  > >
>  > > First, does this mean that you can specify paths containing colons
>  > > (':') only using --add-path and --remove-path, or does it mean that
>  > > you cannot specify paths containg colon ':' (which should be rare)
>  > > at all as checkout limiter / checkout narrowing rule?
>  >
>  > You cannot do othat explicitly unfortunately. You can work-around using
>  > wildcard though.
>
>
> Couldn't you simply escape ':', i.e. write for example Git\:\:Tag.3pm,
>  or Eichten_PRD21\:313,1980_erratum.tex, or \:0.log, or perhaps
>  kmail/jnareb@gmail.com\:@pop.gmail.com\:995, or even something like
>  Mail/inbox/cur/1194202360.32296.mprnq\:2,S, in the same way like you
>  can escape other special characters, for example wildcard characters
>  like '\*' for '*' and '\?' for '?', and of course '\\' for '\'?
>

Sure. Somehow I forgot that.

>  > > Second, wouldn't it be better to use '**' to match also '/'?
>  > > Changing meaning of '*' using per-path flag seems a bit bad.
>  >
>  > It would be better. But I don't see any way but duplicating fnmatch()
>  > implementation and modify it to support '**' so I made a compromise.
>  > Will make another patch for '**' support and see how bloat the code
>  > will be.
>
>
> Well, the alternative would be to tell in commit message _why_ you
>  choose that (for implementation reasons), and perhaps give an example.
>
>  BTW. does '+' prefixed pathspec means that '?' can match '/', directory
>  separator?

No it only affects '*'.

So how about this? Note that document changes that also require code
change like narrow spec escape or '**' do not get in. I will change it
when I update code.

--<--
Narrow checkout
---------------

Normally when you do checkout a branch, your working directory
will be fully populated. In some situations, you just need to
work on certain files, no full checkout is needed. Narrow
checkout is a mode that limits checkout area according to your
needs.

Because narrow checkout uses new index format, it will be
incompatible with git prior to 1.6.0 regarding worktree operations.
Repository-only operations such as clone, push, pull... should not be
affected by narrow checkout. In order to make your working directory
work again with those versions, you can use `git checkout --full` to
return to normal mode (and compatible index format).

In narrow checkout mode, checkout status of every files in your
working directory will be recorded in index. If a file is marked
"no-checkout", it means that file is not needed to be present in
working directory by user or any git command. When a new file is added
to index, it will be marked "checkout" unless narrow spec is applied.
Unmerged files are always "checkout". linkgit:git-update-index[1] can
be used to update "checkout/no-checkout" status in index. When you
checkout new files using "git checkout <file>" they will be
automatically marked "checkout". Other commands such as "git apply"
can also checkout new files if they are needed.

"No-checkout" status is very similar to "assume-unchanged bit"
(see linkgit:git-update-index[1]). The main difference between them
is "assume unchanged" bit just ignores corresponding files in working
directory while narrow checkout goes a bit farther, remove those files
when it is safe to do so.

Narrow spec
-----------

Narrow spec is used to easily specify how do you want to form your
checkout area.  It is a list of pathspecs separated by colons. Each
patchspec specifies what files should or should not be checked out on
working directory. Pathspec can contain wildcards and is relative to
current working directory. Usually asterisk (*) does not match
slashes. If a pathspec is prefixed by a plus sign (+), then any
asterisk will match anything, even slashes.

When you apply new narrow spec to your working directory using either
--path, --add-path or --remove-path, it will update "checkout" status
in index accordingly. Moreover, if a file is marked "no-checkout" and
is present in working directory, it will be removed. If a file is
turned from "no-checkout" to "checkout", then it will be added again
to working directory. Modified and unmerged entries can't bear
"no-checkout" status, if narrow spec applies to them, "git checkout"
will refuse to update working directory.

Narrow spec is not saved by "git checkout". You can form your checkout
area on one go with --path option, or do it incrementally
with --add-path and --remove-path.
--<--

-- 
Duy

Re: [PATCH 15/16] checkout: add new options to support narrow checkout

[Jakub Narebski]

On Mon, 15 Sep 2008 16:01, Nguyen Thai Ngoc Duy wrote:
> On 9/15/08, Jakub Narebski <jnareb@gmail.com> wrote:
>> [I don't see mails I am replying to on GMane interface to git mailing
>>   list, so threads might be broken. Strange... Perhaps too long lines
>>   were cause of rejection by anti-SPAM vger filter?]
> 
> It was because I used zenbe.com (limited choices at work) to send
> mails and did not realize until too late that zenbe.com did not encode
> UTF-8 properly in mail headers. It made vger.k.o nervous and refuse to
> accept my mails.

Yes, vger anti-SPAM filter is quite "trigger happy"...

>>  On Mon, 15 Sep 2008, Duy Nguyen wrote:
>>> On 09/15/2008 "Jakub Narebski" <jnareb@gmail.com> wrote:

>>>>> +Narrow spec will be used to specify how you want to narrow your
>>>>> +checkout. It is a list of pathspecs separated by colons. Each
>>>>> +patchspec specifies what files should be checked out on working
>>>>> +directory. Pathspec can contain wildcards and is relative to
>>>>> +current working directory. Usually asterisk (*) does not match
>>>>> +slashes. If a pathspec is prefixed by a plus sign (+), then
>>>>> +any asterisk will match anything, even slashes.
>>>>
>>>> First, does this mean that you can specify paths containing colons
>>>> (':') only using --add-path and --remove-path, or does it mean that
>>>> you cannot specify paths containg colon ':' (which should be rare)
>>>> at all as checkout limiter / checkout narrowing rule?
>>>
>>> You cannot do othat explicitly unfortunately. You can work-around using
>>> wildcard though.
>>
>> Couldn't you simply escape ':', i.e. write for example Git\:\:Tag.3pm,
>> or Eichten_PRD21\:313,1980_erratum.tex, or \:0.log, or perhaps
>> kmail/jnareb@gmail.com\:@pop.gmail.com\:995, or even something like
>> Mail/inbox/cur/1194202360.32296.mprnq\:2,S, in the same way like you
>> can escape other special characters, for example wildcard characters
>> like '\*' for '*' and '\?' for '?', and of course '\\' for '\'?
> 
> Sure. Somehow I forgot that.

Well, if it is possible, it should be stated in documentation.
Even if it is obvious.


> [...] how about this? Note that document changes that also require code
> change like narrow spec escape or '**' do not get in. I will change it
> when I update code.

> --<--
> Narrow checkout
> ---------------
> 
> Normally when you do checkout a branch, your working directory
> will be fully populated. In some situations, you just need to
> work on certain files, no full checkout is needed. Narrow
> checkout is a mode that limits checkout area according to your
> needs.

You have decided then on the term 'narrow checkout', and not
'partial checkout' or 'sparse checkout', then?
 
It would be nice if git mailing list came with examples of usage for
this feature.  It was quite frequently requested.  I can think off-hand
about two reasons one might want to use 'narrow checkout':

 1.) when you are not interested in files outside some selected
     set, and you don't want to mess files outside this set, even
     by accident.  For example for end-user documentation writers
     it would be Documentation/* (non-recursive).

 2.) when you work on some large monolithic repository which doesn't
     use submodules, and you have limited and small quota (or disk size).
     Usually repository size is smaller or comparable to checkout size,
     and even if it isn't you can use alternates to reduce size of
     repository (assuming that reference repository is hosted outside
     user quota).


> Because narrow checkout uses new index format, it will be
> incompatible with git prior to 1.6.0 regarding worktree operations.
> Repository-only operations such as clone, push, pull... should not be

s/pull/fetch/. pull affects working repository, and it can affect
narrow checkout unexpectedly by conflicts during merge part of pull.

> affected by narrow checkout. In order to make your working directory
> work again with those versions, you can use `git checkout --full` to
> return to normal mode (and compatible index format).

By the way, you have made "git checkout <file>" get file and mark
it "wanted", i.e. clear/zero "no-checkout" bit.  Wouldn't then
"git checkout ." be shorter equivalent to "git checkout --full"?
I'm not saying that '--full' option should be abandoned...

>
> In narrow checkout mode, checkout status of every files in your
> working directory will be recorded in index. If a file is marked
> "no-checkout", it means that file is not needed to be present in
> working directory by user or any git command. When a new file is added
> to index, it will be marked "checkout" unless narrow spec is applied.
> Unmerged files are always "checkout". linkgit:git-update-index[1] can
> be used to update "checkout/no-checkout" status in index. When you
> checkout new files using "git checkout <file>" they will be
> automatically marked "checkout". Other commands such as "git apply"
> can also checkout new files if they are needed.
> 
> "No-checkout" status is very similar to "assume-unchanged bit"
> (see linkgit:git-update-index[1]). The main difference between them
> is "assume unchanged" bit just ignores corresponding files in working
> directory while narrow checkout goes a bit farther, remove those files
> when it is safe to do so.

Good description (although probably could be improved even further).

>
> Narrow spec
> -----------
> 
> Narrow spec is used to easily specify how do you want to form your
> checkout area.  It is a list of pathspecs separated by colons. Each
> patchspec specifies what files should or should not be checked out on
> working directory. Pathspec can contain wildcards and is relative to
> current working directory. Usually asterisk (*) does not match
> slashes. If a pathspec is prefixed by a plus sign (+), then any
> asterisk will match anything, even slashes.

Here I think you should give some examples, perhaps together with
wildcard escaping, escape (backslash) escaping and colon escaping.

> 
> When you apply new narrow spec to your working directory using either
> --path, --add-path or --remove-path, it will update "checkout" status
> in index accordingly. Moreover, if a file is marked "no-checkout" and
> is present in working directory, it will be removed. If a file is
> turned from "no-checkout" to "checkout", then it will be added again
> to working directory. Modified and unmerged entries can't bear
> "no-checkout" status, if narrow spec applies to them, "git checkout"
> will refuse to update working directory.

Do I understand correctly, that if one uses --path=<colon separated list>
_only_ filenames matching one of patterns specified would be checked out,
--add-path=<path> would additionally checkout given path and mark "wanted",
and  --remove-path=<path> would additionally mark "no-checkout" and remove
path?  

What --add-path starts from, i.e. does

  $ git checkout --add-path=.gitignore

starts from empty set if --add-path is first, or from full set as without
--add-path?

And is <pathspec> matched against full pathname, or like .gitignore
rules, i.e. as full pathname if it has at least one '/' in it?

> 
> Narrow spec is not saved by "git checkout". You can form your checkout
> area on one go with --path option, or do it incrementally
> with --add-path and --remove-path.
> --<--

I would probably say that specification used to select which paths to
check out is not saved anywhere, and used only to mark paths in index
as "no-checkout" or not.  Or somehing like that.


Thank you very much for your continued work on this feature!
-- 
Jakub Narebski
Poland

Re: [PATCH 15/16] checkout: add new options to support narrow checkout

[Baz]

Just a few minor grammar nits and typos in the docs, and one leftover
mention of 'sparse' that should be 'narrow'.

On Sun, Sep 14, 2008 at 2:08 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:
> These options include:
>
>  --full: return to full checkout (default)
>  --path: narrow checkout to some areas according to given spec
>  --add-path/--remove-path: adjust current narrow checkout
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>  Documentation/git-checkout.txt |   68 +++++++++++++++++++++++++++++++++-
>  builtin-checkout.c             |   44 ++++++++++++++++++++++
>  t/t2011-checkout-narrow.sh     |   80 ++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 190 insertions(+), 2 deletions(-)
>  create mode 100755 t/t2011-checkout-narrow.sh
>
> diff --git a/Documentation/git-checkout.txt b/Documentation/git-checkout.txt
> index 82e154d..2ae483b 100644
> --- a/Documentation/git-checkout.txt
> +++ b/Documentation/git-checkout.txt
> @@ -8,8 +8,10 @@ git-checkout - Checkout a branch or paths to the working tree
>  SYNOPSIS
>  --------
>  [verse]
> -'git checkout' [-q] [-f] [--track | --no-track] [-b <new_branch> [-l]] [-m] [<branch>]
> -'git checkout' [-f|--ours|--theirs|-m|--conflict=<style>] [<tree-ish>] [--] <paths>...
> +'git checkout' [-q] [-f] [--track | --no-track] [-b <new_branch> [-l]] [-m]
> +         [<narrow options>] [<branch>]
> +'git checkout' [-f|--ours|--theirs|-m|--conflict=<style>] [<tree-ish>]
> +         [<narrow options>] [--] <paths>...
>
>  DESCRIPTION
>  -----------
> @@ -34,6 +36,10 @@ used to specify a specific tree-ish (i.e. commit, tag or tree)
>  to update the index for the given paths before updating the
>  working tree.
>
> +<narrow options> include --full, --path, --add-path and --remove-path.
> +The last three require narrow spec. Please refer to "narrow checkout"
> +section to have more information about this mode.

s/to have/for/

> +
>  The index may contain unmerged entries after a failed merge.  By
>  default, if you try to check out such an entry from the index, the
>  checkout operation will fail and nothing will be checked out.
> @@ -108,6 +114,22 @@ exlicitly give a name with '-b' in such a case.
>        However, with this option, a three-way merge between the current
>        branch, your working tree contents, and the new branch
>        is done, and you will be on the new branch.
> +
> +--full::
> +       Quit narrow checkout mode. Return to full checkout.
> +
> +--path=<narrow_spec>::
> +       Re-apply new narrow spec on current working directory to
> +       form new checkout area.
> +
> +--add-path=<narrow_spec>::
> +       Checkout more areas specified by narrow spec to current
> +       checkout area.
> +
> +--remove-path=<narrow_spec>::
> +       Narrow checkout area by removing files specified by narrow spec
> +       from current checkout area. This operation will fail if there
> +       is unmerged or modified files in the removing areas.

s/is/are/

>  +
>  When a merge conflict happens, the index entries for conflicting
>  paths are left unmerged, and you need to resolve the conflicts
> @@ -171,6 +193,48 @@ the reflog for HEAD where you were, e.g.
>  $ git log -g -2 HEAD
>  ------------
>
> +Narrow checkout
> +---------------
> +
> +Normally when you do checkout a branch, your working directory

s/do//

> +will be fully populated. In some situations, you just need to
> +work on certain files, no full checkout is required. Narrow
> +checkout is a mode that limits checkout area according to your

s/limits checkout area/limits the checkout area/

> +rules.
> +
> +Because narrow checkout uses new index format, it will be

s/uses new index format/uses a new index format/

> +incompatible with git prior to 1.6.0. In order to make your
> +working directory work with those versions, you can use `git
> +checkout --full` to return to normal mode (and compatible index
> +format).
> +
> +Narrow works by applying your rules to the index, marking which
> +file you need and which file you need not. Modified/Unmerged

s/file you need and file you need not/files you need and which you do not/

> +files cannot be marked unneeded. Unnecessary files will be

This reads a bit awkwardly. 'marked as unneeded' maybe, or 'marked
"unneeded"' as you've done elsewhere.

> +removed from working directory.  Note that after this step,
> +removed files can still be added to working directory when they
> +are needed by any git command. For example, when you have a merge
> +conflict, conflicted files will be checked out on working
> +directory and will no longer be marked "unneeded".
> +
> +New files after merges will always be "needed". You can also
> +apply rules when switching branches to avoid unwanted new files.
> +
> +Files that are marked "no-checkout" will be treated like entries
> +with "assume-unchanged bit" (see linkgit:git-update-index[1]). In
> +short, Git will never look for those files in working directory
> +no matter whether they exist in working directory.
> +
> +You can apply your rules at once with --path option, or do it
> +incrementally with --add-path and --remove-path.
> +
> +Narrow spec will be used to specify how you want to narrow your

"The narrow spec specifies how you want to narrow your..."

> +checkout. It is a list of pathspecs separated by colons. Each
> +patchspec specifies what files should be checked out on working

pathspec here, not patchspec.

> +directory. Pathspec can contain wildcards and is relative to
> +current working directory. Usually asterisk (*) does not match
> +slashes. If a pathspec is prefixed by a plus sign (+), then
> +any asterisk will match anything, even slashes.
>
>  EXAMPLES
>  --------
> diff --git a/builtin-checkout.c b/builtin-checkout.c
> index c7b0aad..b6bc3c5 100644
> --- a/builtin-checkout.c
> +++ b/builtin-checkout.c
> @@ -33,6 +33,12 @@ struct checkout_opts {
>        const char *new_branch;
>        int new_branch_log;
>        enum branch_track track;
> +
> +       const char *prefix;
> +       char *new_path;
> +       char *add_path;
> +       char *remove_path;
> +       int no_narrow_checkout;
>  };
>
>  static int post_checkout_hook(struct commit *old, struct commit *new,
> @@ -412,6 +418,24 @@ static int merge_working_tree(struct checkout_opts *opts,
>                tree = parse_tree_indirect(new->commit->object.sha1);
>                init_tree_desc(&trees[1], tree->buffer, tree->size);
>
> +               topts.narrow_prefix = opts->prefix;
> +               if (opts->no_narrow_checkout) {
> +                       /* leave narrow_spec NULL */
> +                       topts.new_narrow_path = 1;
> +               }
> +               else if (opts->new_path) {
> +                       topts.narrow_spec = opts->new_path;
> +                       topts.new_narrow_path = 1;
> +               }
> +               else if (opts->add_path) {
> +                       topts.narrow_spec = opts->add_path;
> +                       topts.add_narrow_path = 1;
> +               }
> +               else if (opts->remove_path) {
> +                       topts.narrow_spec = opts->remove_path;
> +                       topts.remove_narrow_path = 1;
> +               }
> +
>                ret = unpack_trees(2, trees, &topts);
>                if (ret == -1) {
>                        /*
> @@ -600,6 +624,10 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
>                OPT_BOOLEAN('m', "merge", &opts.merge, "merge"),
>                OPT_STRING(0, "conflict", &conflict_style, "style",
>                           "conflict style (merge or diff3)"),
> +               OPT_BOOLEAN(0, "full", &opts.no_narrow_checkout, "quit sparse checkout"),

s/quit sparse checkout/quit narrow checkout mode/

As per your documentation above. I noticed this because I was
searching for 'sparse checkout' and was surprised when only one mail
in a patch series matched :)

Hope this helps,
Baz

> +               OPT_STRING(0, "path", &opts.new_path, "prefixes", "apply new narrow checkout path"),
> +               OPT_STRING(0, "add-path", &opts.add_path, "prefixes", "add more checkout area"),
> +               OPT_STRING(0, "remove-path", &opts.remove_path, "prefixes", "narrow checkout area"),
>                OPT_END(),
>        };
>        int has_dash_dash;
> @@ -610,6 +638,7 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
>        git_config(git_checkout_config, NULL);
>
>        opts.track = BRANCH_TRACK_UNSPECIFIED;
> +       opts.prefix = prefix;
>
>        argc = parse_options(argc, argv, options, checkout_usage,
>                             PARSE_OPT_KEEP_DASHDASH);
> @@ -639,6 +668,18 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
>        if (!opts.new_branch && (opts.track != git_branch_track))
>                die("git checkout: --track and --no-track require -b");
>
> +       if (((opts.no_narrow_checkout ? 1 : 0) +
> +            (opts.new_path ? 1 : 0) +
> +            (opts.add_path ? 1 : 0) +
> +            (opts.remove_path ? 1 : 0)) > 1)
> +               die("git checkout: --path, --full, --add-path and --remove-path are incompatible");
> +
> +       if (opts.new_branch && (opts.add_path || opts.remove_path))
> +               die("git checkout: --add-path and --remove-path should only be used on current branch");
> +
> +       if (opts.new_branch && opts.no_narrow_checkout)
> +               die("git checkout: switching branch with --full does not make sense");
> +
>        if (opts.force && opts.merge)
>                die("git checkout: -f and -m are incompatible");
>
> @@ -732,6 +773,9 @@ no_reference:
>                if (1 < !!opts.writeout_stage + !!opts.force + !!opts.merge)
>                        die("git checkout: --ours/--theirs, --force and --merge are incompatible when\nchecking out of the index.");
>
> +               if (opts.no_narrow_checkout || opts.new_path || opts.add_path || opts.remove_path)
> +                       die("git checkout: updating paths is incompatible with setting sparse checkout");
> +
>                return checkout_paths(source_tree, pathspec, &opts);
>        }
>
> diff --git a/t/t2011-checkout-narrow.sh b/t/t2011-checkout-narrow.sh
> new file mode 100755
> index 0000000..77b99e1
> --- /dev/null
> +++ b/t/t2011-checkout-narrow.sh
> @@ -0,0 +1,80 @@
> +#!/bin/sh
> +
> +test_description='narrow checkout'
> +
> +. ./test-lib.sh
> +
> +test_expect_success setup '
> +       mkdir work1 work2 work3
> +       touch one two three
> +       touch work1/one work2/two work3/three
> +       git add one work1/one
> +       git commit -m work1
> +       git add two work2/two
> +       git commit -m work2
> +       git add three work3/three
> +       git commit -m work3
> +'
> +
> +test_expect_success '--full on no-narrow checkout' '
> +       git checkout --full
> +'
> +
> +test_expect_success '--full and --path incompatible' '
> +       test_must_fail git checkout --full --path=work1
> +'
> +
> +test_expect_success 'limit worktree to work1 and work2' '
> +       git checkout --path="work1/*:work2/*" &&
> +       test -f work1/one &&
> +       test -f work2/two &&
> +       ! test -f work3/three
> +'
> +
> +test_expect_success 'update worktree to work2 and work3' '
> +       git checkout --path="work2/*:work3/*" &&
> +       ! test -f work1/one &&
> +       test -f work2/two &&
> +       test -f work3/three
> +'
> +
> +test_expect_success 'update narrow prefix with modification' '
> +       echo modified >> work2/two &&
> +       git checkout --path="work1/*:work2/*" &&
> +       test -f work1/one &&
> +       test -f work2/two &&
> +       ! test -f work3/three &&
> +       grep -q modified work2/two
> +'
> +
> +test_expect_success 'update narrow should not lose modification' '
> +       ! git checkout --path="work1/*:work3/*" &&
> +       test -f work1/one &&
> +       test -f work2/two &&
> +       ! test -f work3/three &&
> +       grep -q modified work2/two
> +'
> +
> +test_expect_success 'widen checkout area' '
> +       git checkout --add-path="work3/*" &&
> +       test -f work1/one &&
> +       test -f work2/two &&
> +       test -f work3/three
> +'
> +
> +test_expect_success 'narrow checkout area' '
> +       git checkout --remove-path="work3/*" &&
> +       test -f work1/one &&
> +       test -f work2/two &&
> +       ! test -f work3/three
> +'
> +
> +test_expect_success 'exit narrow checkout' '
> +       git checkout --full &&
> +       test -f work1/one &&
> +       test -f work2/two &&
> +       test -f work3/three &&
> +       test one
> +'
> +
> +test_done
> --
> 1.6.0.96.g2fad1.dirty
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: [PATCH 15/16] checkout: add new options to support narrow checkout

[Johannes Sixt]

Nguyễn Thái Ngọc Duy schrieb:
> +--full::
> +	Quit narrow checkout mode. Return to full checkout.
> +
> +--path=<narrow_spec>::
> +	Re-apply new narrow spec on current working directory to
> +	form new checkout area.
> +
> +--add-path=<narrow_spec>::
> +	Checkout more areas specified by narrow spec to current
> +	checkout area.
> +
> +--remove-path=<narrow_spec>::
> +	Narrow checkout area by removing files specified by narrow spec
> +	from current checkout area. This operation will fail if there
> +	is unmerged or modified files in the removing areas.

The option --path is named a bit too generic for my taste. How about
--narrow=...? And how would you like --more= instead of --add-path= and
--less= instead of --remove-path=? I do think that --full is OK.

-- Hannes

Re: [PATCH 15/16] checkout: add new options to support narrow checkout

[Nguyen Thai Ngoc Duy]

On 9/16/08, Jakub Narebski <jnareb@gmail.com> wrote:
>  >> Couldn't you simply escape ':', i.e. write for example Git\:\:Tag.3pm,
>  >> or Eichten_PRD21\:313,1980_erratum.tex, or \:0.log, or perhaps
>  >> kmail/jnareb@gmail.com\:@pop.gmail.com\:995, or even something like
>  >> Mail/inbox/cur/1194202360.32296.mprnq\:2,S, in the same way like you
>  >> can escape other special characters, for example wildcard characters
>  >> like '\*' for '*' and '\?' for '?', and of course '\\' for '\'?
>  >
>  > Sure. Somehow I forgot that.
>
>
> Well, if it is possible, it should be stated in documentation.
>  Even if it is obvious.

I mean it should be possible, but not yet implemented. Next time
document will be updated when '\' escape is implemented.

>  > --<--
>  > Narrow checkout
>  > ---------------
>  >
>  > Normally when you do checkout a branch, your working directory
>  > will be fully populated. In some situations, you just need to
>  > work on certain files, no full checkout is needed. Narrow
>  > checkout is a mode that limits checkout area according to your
>  > needs.
>
>
> You have decided then on the term 'narrow checkout', and not
>  'partial checkout' or 'sparse checkout', then?

Not yet. I tend to prefer partial/sparse checkout. Probably should
have a look at how other SCMs do and what term they use. If Git's
functionality is so different, a different term might notice people
about that.

>  > Because narrow checkout uses new index format, it will be
>  > incompatible with git prior to 1.6.0 regarding worktree operations.
>  > Repository-only operations such as clone, push, pull... should not be
>
>
> s/pull/fetch/. pull affects working repository, and it can affect
>  narrow checkout unexpectedly by conflicts during merge part of pull.

Bad writing. I mean pull/fetch from a narrow-checkout-ed repository to
another fully populated one. Will fix.

>
>  > affected by narrow checkout. In order to make your working directory
>  > work again with those versions, you can use `git checkout --full` to
>  > return to normal mode (and compatible index format).
>
>
> By the way, you have made "git checkout <file>" get file and mark
>  it "wanted", i.e. clear/zero "no-checkout" bit.  Wouldn't then
>  "git checkout ." be shorter equivalent to "git checkout --full"?
>  I'm not saying that '--full' option should be abandoned...

It is not equivalent. "git checkout ." will happily overwrite any
modified files in your working directory.

>
>  >
>  > In narrow checkout mode, checkout status of every files in your
>  > working directory will be recorded in index. If a file is marked
>  > "no-checkout", it means that file is not needed to be present in
>  > working directory by user or any git command. When a new file is added
>  > to index, it will be marked "checkout" unless narrow spec is applied.
>  > Unmerged files are always "checkout". linkgit:git-update-index[1] can
>  > be used to update "checkout/no-checkout" status in index. When you
>  > checkout new files using "git checkout <file>" they will be
>  > automatically marked "checkout". Other commands such as "git apply"
>  > can also checkout new files if they are needed.
>  >
>  > "No-checkout" status is very similar to "assume-unchanged bit"
>  > (see linkgit:git-update-index[1]). The main difference between them
>  > is "assume unchanged" bit just ignores corresponding files in working
>  > directory while narrow checkout goes a bit farther, remove those files
>  > when it is safe to do so.
>
>
> Good description (although probably could be improved even further).

Contributions are welcome   ;)

>
>  >
>  > When you apply new narrow spec to your working directory using either
>  > --path, --add-path or --remove-path, it will update "checkout" status
>  > in index accordingly. Moreover, if a file is marked "no-checkout" and
>  > is present in working directory, it will be removed. If a file is
>  > turned from "no-checkout" to "checkout", then it will be added again
>  > to working directory. Modified and unmerged entries can't bear
>  > "no-checkout" status, if narrow spec applies to them, "git checkout"
>  > will refuse to update working directory.
>
>
> Do I understand correctly, that if one uses --path=<colon separated list>
>  _only_ filenames matching one of patterns specified would be checked out,
>  --add-path=<path> would additionally checkout given path and mark "wanted",
>  and  --remove-path=<path> would additionally mark "no-checkout" and remove
>  path?

--add-path/--remove-path also use narrow spec, which could be more
than one pattern.

>  What --add-path starts from, i.e. does
>
>   $ git checkout --add-path=.gitignore
>
>  starts from empty set if --add-path is first, or from full set as without
>  --add-path?

I'm not sure I understand this.

>  And is <pathspec> matched against full pathname, or like .gitignore
>  rules, i.e. as full pathname if it has at least one '/' in it?

like shell wildcard, full pathname must match. On my way back home, I
thought I should have removed mention of "pathspec", which behaves a
little bit different.

Also those specs are relative to working directory though, so if you
are in b/c and want to match d, you only need to type --add-path=d
instead of --add-path=b/c/d. Will add this to doc.

>
>  >
>  > Narrow spec is not saved by "git checkout". You can form your checkout
>  > area on one go with --path option, or do it incrementally
>  > with --add-path and --remove-path.
>  > --<--
>
>
> I would probably say that specification used to select which paths to
>  check out is not saved anywhere, and used only to mark paths in index
>  as "no-checkout" or not.  Or somehing like that.
>

Thanks (as for other parts of your comments stripped by me)

-- 
Duy

Re: [PATCH 15/16] checkout: add new options to support narrow checkout

[Nguyen Thai Ngoc Duy]

On 9/16/08, Johannes Sixt <j.sixt@viscovery.net> wrote:
> Nguyễn Thái Ngọc Duy schrieb:
>
> > +--full::
>  > +     Quit narrow checkout mode. Return to full checkout.
>  > +
>  > +--path=<narrow_spec>::
>  > +     Re-apply new narrow spec on current working directory to
>  > +     form new checkout area.
>  > +
>  > +--add-path=<narrow_spec>::
>  > +     Checkout more areas specified by narrow spec to current
>  > +     checkout area.
>  > +
>  > +--remove-path=<narrow_spec>::
>  > +     Narrow checkout area by removing files specified by narrow spec
>  > +     from current checkout area. This operation will fail if there
>  > +     is unmerged or modified files in the removing areas.
>
>
> The option --path is named a bit too generic for my taste. How about
>  --narrow=...?

--narrow or --narrow-template (or --checkout-template?) seems good.

> And how would you like --more= instead of --add-path= and
>  --less= instead of --remove-path=?

Now these --more/--less are too generic to me ;)

> I do think that --full is OK.
>
>  -- Hannes
>
>


-- 
Duy

Re: [PATCH 15/16] checkout: add new options to support narrow checkout

[Jakub Narebski]

On Tue, 16 Sep 2008, Nguyen Thai Ngoc Duy wrote:
> On 9/16/08, Jakub Narebski <jnareb@gmail.com> wrote:

>>>> Couldn't you simply escape ':', i.e. write for example Git\:\:Tag.3pm,
>>>> or Eichten_PRD21\:313,1980_erratum.tex, or \:0.log, or perhaps
>>>> kmail/jnareb@gmail.com\:@pop.gmail.com\:995, or even something like
>>>> Mail/inbox/cur/1194202360.32296.mprnq\:2,S, in the same way like you
>>>> can escape other special characters, for example wildcard characters
>>>> like '\*' for '*' and '\?' for '?', and of course '\\' for '\'?
>>>
>>> Sure. Somehow I forgot that.
>>
>> Well, if it is possible, it should be stated in documentation.
>>  Even if it is obvious.
> 
> I mean it should be possible, but not yet implemented. Next time
> document will be updated when '\' escape is implemented.

I was thinking about explicitly stating this limitation (that ':' in
paths are not permitted/can be not escaped), or put it in "Bugs" aka
"Known Limitations" ;-) (sub)section.

>>> --<--
>>> Narrow checkout
>>> ---------------
>>>
>>> Normally when you do checkout a branch, your working directory
>>> will be fully populated. In some situations, you just need to
>>> work on certain files, no full checkout is needed. Narrow
>>> checkout is a mode that limits checkout area according to your
>>> needs.
>>
>> You have decided then on the term 'narrow checkout', and not
>>  'partial checkout' or 'sparse checkout', then?
> 
> Not yet. I tend to prefer partial/sparse checkout. Probably should
> have a look at how other SCMs do and what term they use. If Git's
> functionality is so different, a different term might notice people
> about that.

Perhaps whip up a survey? Just kidding (or not).

>>> Because narrow checkout uses new index format, it will be
>>> incompatible with git prior to 1.6.0 regarding worktree operations.
>>> Repository-only operations such as clone, push, pull... should not be
>>
>> s/pull/fetch/. pull affects working repository, and it can affect
>>  narrow checkout unexpectedly by conflicts during merge part of pull.
> 
> Bad writing. I mean pull/fetch from a narrow-checkout-ed repository to
> another fully populated one. Will fix.

Well, I though that you were referring to operations done _inside_
given repository, not on repository from outside. So I would say
something like repository-only operations (operations which do
not affect index and/or working area) such as fetch and pull, and
by definition all remote operations such as fetching from repository,
or cloning this repository... but this also isn't IMHO best way to
say it.

>>> affected by narrow checkout. In order to make your working directory
>>> work again with those versions, you can use `git checkout --full` to
>>> return to normal mode (and compatible index format).
>>
>>
>> By the way, you have made "git checkout <file>" get file and mark
>>  it "wanted", i.e. clear/zero "no-checkout" bit.  Wouldn't then
>>  "git checkout ." be shorter equivalent to "git checkout --full"?
>>  I'm not saying that '--full' option should be abandoned...
> 
> It is not equivalent. "git checkout ." will happily overwrite any
> modified files in your working directory.

True, I fortgot about this. Sorry, my mistake.
 
>>> When you apply new narrow spec to your working directory using either
>>> --path, --add-path or --remove-path, it will update "checkout" status
>>> in index accordingly. Moreover, if a file is marked "no-checkout" and
>>> is present in working directory, it will be removed. If a file is
>>> turned from "no-checkout" to "checkout", then it will be added again
>>> to working directory. Modified and unmerged entries can't bear
>>> "no-checkout" status, if narrow spec applies to them, "git checkout"
>>> will refuse to update working directory.
>>
>>
>> Do I understand correctly, that if one uses --path=<colon separated list>
>>  _only_ filenames matching one of patterns specified would be checked out,
>>  --add-path=<path> would additionally checkout given path and mark "wanted",
>>  and  --remove-path=<path> would additionally mark "no-checkout" and remove
>>  path?
> 
> --add-path/--remove-path also use narrow spec, which could be more
> than one pattern.

What is the difference then between --add-path and --path? The fact
that --add-path can be incremental, and --path starts always from
an empty set?
 
>>  What --add-path starts from, i.e. does
>>
>>   $ git checkout --add-path=.gitignore
>>
>>  starts from empty set if --add-path is first, or from full set as without
>>  --add-path?
> 
> I'm not sure I understand this.

Well, what I wanted to ask is if --remove-path starts from fully
checked out repository, for example if

  $ git checkout --remove-path=some_large_file

would checkout all files _except_ 'some_large_file'.

>>  And is <pathspec> matched against full pathname, or like .gitignore
>>  rules, i.e. as full pathname if it has at least one '/' in it?
> 
> like shell wildcard, full pathname must match. On my way back home, I
> thought I should have removed mention of "pathspec", which behaves a
> little bit different.
> 
> Also those specs are relative to working directory though, so if you
> are in b/c and want to match d, you only need to type --add-path=d
> instead of --add-path=b/c/d. Will add this to doc.

I would have thought that you follow the same rules (perhaps with
exception of !path excluding rule) like for gitignore and gitattributes.

-- 
Jakub Narebski
Poland

Re: [PATCH 15/16] checkout: add new options to support narrow checkout

[Nguyen Thai Ngoc Duy]

On 9/17/08, Jakub Narebski <jnareb@gmail.com> wrote:
>  >> You have decided then on the term 'narrow checkout', and not
>  >>  'partial checkout' or 'sparse checkout', then?
>  >
>  > Not yet. I tend to prefer partial/sparse checkout. Probably should
>  > have a look at how other SCMs do and what term they use. If Git's
>  > functionality is so different, a different term might notice people
>  > about that.
>
>
> Perhaps whip up a survey? Just kidding (or not).

Well I've been asking about the name on this list long enough. I'm
going with 'sparse checkout' as svn' sparse directories does not look
too different from git's.

>  >>> When you apply new narrow spec to your working directory using either
>  >>> --path, --add-path or --remove-path, it will update "checkout" status
>  >>> in index accordingly. Moreover, if a file is marked "no-checkout" and
>  >>> is present in working directory, it will be removed. If a file is
>  >>> turned from "no-checkout" to "checkout", then it will be added again
>  >>> to working directory. Modified and unmerged entries can't bear
>  >>> "no-checkout" status, if narrow spec applies to them, "git checkout"
>  >>> will refuse to update working directory.
>  >>
>  >>
>  >> Do I understand correctly, that if one uses --path=<colon separated list>
>  >>  _only_ filenames matching one of patterns specified would be checked out,
>  >>  --add-path=<path> would additionally checkout given path and mark "wanted",
>  >>  and  --remove-path=<path> would additionally mark "no-checkout" and remove
>  >>  path?
>  >
>  > --add-path/--remove-path also use narrow spec, which could be more
>  > than one pattern.
>
>
> What is the difference then between --add-path and --path? The fact
>  that --add-path can be incremental, and --path starts always from
>  an empty set?

--path clears out all no-checkout bits and set again based on the
given spec. --add-path adds more checkout entries based on the given
spec, think of widening checkout area. --remove-path does narrow the
checkout area. They are like '=', '+=' and '-=' operators
respectively.

>
>  >>  What --add-path starts from, i.e. does
>  >>
>  >>   $ git checkout --add-path=.gitignore
>  >>
>  >>  starts from empty set if --add-path is first, or from full set as without
>  >>  --add-path?
>  >
>  > I'm not sure I understand this.
>
>
> Well, what I wanted to ask is if --remove-path starts from fully
>  checked out repository, for example if
>
>   $ git checkout --remove-path=some_large_file
>
>  would checkout all files _except_ 'some_large_file'.

No, there is no negative spec. Like I said above, --remove-path is to
remove some files based on the given spec.

>
>  >>  And is <pathspec> matched against full pathname, or like .gitignore
>  >>  rules, i.e. as full pathname if it has at least one '/' in it?
>  >
>  > like shell wildcard, full pathname must match. On my way back home, I
>  > thought I should have removed mention of "pathspec", which behaves a
>  > little bit different.
>  >
>  > Also those specs are relative to working directory though, so if you
>  > are in b/c and want to match d, you only need to type --add-path=d
>  > instead of --add-path=b/c/d. Will add this to doc.
>
>
> I would have thought that you follow the same rules (perhaps with
>  exception of !path excluding rule) like for gitignore and gitattributes.
>

Um.. never thought of gitignores/gitattributes rules before. It's a
good idea all narrowspec/gitignore/gitattributes using the same rules.
Thanks.
-- 
Duy

Re: [PATCH 15/16] checkout: add new options to support narrow checkout

[Johannes Sixt]

Nguyen Thai Ngoc Duy schrieb:
> Well I've been asking about the name on this list long enough. I'm
> going with 'sparse checkout' as svn' sparse directories does not look
> too different from git's.

FWIW, /me likes 'sparse checkout'.

> --path clears out all no-checkout bits and set again based on the
> given spec. --add-path adds more checkout entries based on the given
> spec, think of widening checkout area. --remove-path does narrow the
> checkout area. They are like '=', '+=' and '-=' operators
> respectively.

The still un-answered question was: In a full checkout, i.e. in a
repository where the narrow/sparse checkout feature has never been used so
far, is

   $ git checkout --add-path=foo

a no-op, or is it equivalent to

   $ git checkout --path=foo

Or stated differently: In the sequence

   $ git checkout --full
   $ git checkout --add-path=foo

is the second statement redundant?

-- Hannes

Re: [PATCH 15/16] checkout: add new options to support narrow checkout

[Nguyen Thai Ngoc Duy]

On 9/17/08, Johannes Sixt <j.sixt@viscovery.net> wrote:
> The still un-answered question was: In a full checkout, i.e. in a
>  repository where the narrow/sparse checkout feature has never been used so
>  far, is
>
>    $ git checkout --add-path=foo
>
>  a no-op, or is it equivalent to
>
>    $ git checkout --path=foo

no-op.
-- 
Duy

Re: [PATCH 15/16] checkout: add new options to support narrow checkout

[Jakub Narebski]

On Wed, 17 Sep 2008, Nguyen Thai Ngoc Duy wrote:
> On 9/17/08, Jakub Narebski <jnareb@gmail.com> wrote:

>>
>> Well, what I wanted to ask is if --remove-path starts from fully
>>  checked out repository, for example if
>>
>>   $ git checkout --remove-path=some_large_file
>>
>>  would checkout all files _except_ 'some_large_file'.
> 
> No, there is no negative spec. Like I said above, --remove-path is to
> remove some files based on the given spec.

So "git checkout --path/--add-path/--remove-path" is porcelain
interface to use (git-update-index is plumbing)?
 
>>>>  And is <pathspec> matched against full pathname, or like .gitignore
>>>>  rules, i.e. as full pathname if it has at least one '/' in it?
>>>
>>> like shell wildcard, full pathname must match. On my way back home, I
>>> thought I should have removed mention of "pathspec", which behaves a
>>> little bit different.
>>>
>>> Also those specs are relative to working directory though, so if you
>>> are in b/c and want to match d, you only need to type --add-path=d
>>> instead of --add-path=b/c/d. Will add this to doc.
>>
>> I would have thought that you follow the same rules (perhaps with
>>  exception of !path excluding rule) like for gitignore and gitattributes.
> 
> Um.. never thought of gitignores/gitattributes rules before. It's a
> good idea all narrowspec/gitignore/gitattributes using the same rules.

Perhaps you would be able to even reuse some of implementation?...

-- 
Jakub Narebski
Poland