Re: [PATCH 14/16] clone: support narrow checkout with --path option

Re: [PATCH 14/16] clone: support narrow checkout with --path option

[Jakub Narebski]

Duy Nguyen wrote:
> On 09/15/2008 "Jakub Narebski" <jnareb@gmail.com> wrote:

> > 
> > Second, how `--path=<narrow_spec>` interacts with `-n' (no checkout).
> > Does it even have any sense? Perhaps those options should be mutually
> > incompatibile (and --bare and this option too)?
> 
> Yes they are incompatible and won't work if either -n or --bare is provided with --path.

Perhaps that should be stated explicitly? It is quite obvious, but...
-- 
Jakub Narebski
Poland

[PATCH 00/16] Narrow/Partial/Sparse checkout

[Nguyễn Thái Ngọc Duy]

I hope this series is now ready to be reviewed. Documentation is in
place so no more explanation here (main document is in git-checkout.txt).
The series could be splitted into:

Nguyễn Thái Ngọc Duy (6):
  Extend index to save more flags
  Introduce CE_NO_CHECKOUT bit
  update-index: refactor mark_valid() in preparation for new options
  update-index: add --checkout/--no-checkout to update CE_NO_CHECKOUT bit
  ls-files: add --narrow-checkout option to "will checkout" entries
  Add tests for updating no-checkout entries in index

 .gitignore                         |    1 +
 Documentation/git-ls-files.txt     |    6 +++
 Documentation/git-update-index.txt |   12 ++++++
 Makefile                           |    2 +-
 builtin-ls-files.c                 |   11 ++++++
 builtin-update-index.c             |   40 +++++++++++++--------
 cache.h                            |   66 +++++++++++++++++++++++++++++++++--
 read-cache.c                       |   57 ++++++++++++++++++++++++-------
 t/t2104-update-index-narrow.sh     |   36 +++++++++++++++++++
 test-index-version.c               |   14 ++++++++
 10 files changed, 212 insertions(+), 33 deletions(-)

This adds CE_NO_CHECKOUT bit and plumbing support to manipulate it.
This bit is to mark what entry will be checked out.

Nguyễn Thái Ngọc Duy (4):
  Prevent diff machinery from examining worktree outside narrow checkout
  checkout_entry(): CE_NO_CHECKOUT on checked out entries.
  ls-files: apply --deleted on narrow area only
  grep: skip files that have not been checked out

 builtin-grep.c                 |    7 ++++++-
 builtin-ls-files.c             |    2 +-
 diff-lib.c                     |    5 +++--
 diff.c                         |    4 +++-
 entry.c                        |    1 +
 t/t2104-update-index-narrow.sh |    6 ++++++
 6 files changed, 20 insertions(+), 5 deletions(-)

Various fixes to make sure it works once in this mode.

Nguyễn Thái Ngọc Duy (5):
  unpack_trees(): add support for narrow checkout
  narrow spec: put '+' before a spec will change semantic of '*'
  ls-files: add --narrow-match=spec option for testing narrow matching
  clone: support narrow checkout with --path option
  checkout: add new options to support narrow checkout

 Documentation/git-checkout.txt   |   68 ++++++++++++++++++++++++-
 Documentation/git-clone.txt      |    8 +++-
 Documentation/git-ls-files.txt   |    7 ++-
 builtin-checkout.c               |   44 ++++++++++++++++
 builtin-clone.c                  |   13 +++++
 builtin-ls-files.c               |   14 +++++-
 cache.h                          |    3 +
 t/t2011-checkout-narrow.sh       |   80 +++++++++++++++++++++++++++++
 t/t3003-ls-files-narrow-match.sh |   45 ++++++++++++++++
 t/t5703-clone-narrow.sh          |   39 ++++++++++++++
 unpack-trees.c                   |  105 ++++++++++++++++++++++++++++++++++++++
 unpack-trees.h                   |    6 ++
 12 files changed, 427 insertions(+), 5 deletions(-)

Support for "git checkout" and "git clone" to enter/update/leave
narrow/sparse/partial checkout.

Nguyễn Thái Ngọc Duy (1):
  ls-files: add --overlay option

 Documentation/git-ls-files.txt |    4 ++++
 builtin-ls-files.c             |   16 +++++++++++++---
 t/t2104-update-index-narrow.sh |    4 ++++
 3 files changed, 21 insertions(+), 3 deletions(-)

Bonus stuff, not strictly needed.

[PATCH 01/16] Extend index to save more flags

[Nguyễn Thái Ngọc Duy]

The on-disk format of index only saves 16 bit flags, nearly all have
been used. The last bit (CE_EXTENDED) is used to for future extension.

This patch extends index entry format to save more flags in future.
The new entry format will be used when CE_EXTENDED bit is 1.

Because older implementation may not understand CE_EXTENDED bit and
misread the new format, if there is any extended entry in index, index
header version will turn 3, which makes it incompatible for older git.
If there is none, header version will return to 2 again.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 cache.h      |   58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++----
 read-cache.c |   51 +++++++++++++++++++++++++++++++++++++++++----------
 2 files changed, 95 insertions(+), 14 deletions(-)

diff --git a/cache.h b/cache.h
index f725783..1e572e4 100644
--- a/cache.h
+++ b/cache.h
@@ -109,6 +109,26 @@ struct ondisk_cache_entry {
 	char name[FLEX_ARRAY]; /* more */
 };
 
+/*
+ * This struct is used when CE_EXTENDED bit is 1
+ * The struct must match ondisk_cache_entry exactly from
+ * ctime till flags
+ */
+struct ondisk_cache_entry_extended {
+	struct cache_time ctime;
+	struct cache_time mtime;
+	unsigned int dev;
+	unsigned int ino;
+	unsigned int mode;
+	unsigned int uid;
+	unsigned int gid;
+	unsigned int size;
+	unsigned char sha1[20];
+	unsigned short flags;
+	unsigned short flags2;
+	char name[FLEX_ARRAY]; /* more */
+};
+
 struct cache_entry {
 	unsigned int ce_ctime;
 	unsigned int ce_mtime;
@@ -130,7 +150,15 @@ struct cache_entry {
 #define CE_VALID     (0x8000)
 #define CE_STAGESHIFT 12
 
-/* In-memory only */
+/*
+ * Range 0xFFFF0000 in ce_flags is divided into
+ * two parts: in-memory flags and on-disk ones.
+ * Flags in CE_EXTENDED_FLAGS will get saved on-disk
+ * if you want to save a new flag, add it in
+ * CE_EXTENDED_FLAGS
+ *
+ * In-memory only flags
+ */
 #define CE_UPDATE    (0x10000)
 #define CE_REMOVE    (0x20000)
 #define CE_UPTODATE  (0x40000)
@@ -140,6 +168,24 @@ struct cache_entry {
 #define CE_UNHASHED  (0x200000)
 
 /*
+ * Extended on-disk flags
+ */
+/* CE_EXTENDED2 is for future extension */
+#define CE_EXTENDED2 0x80000000
+
+#define CE_EXTENDED_FLAGS (0)
+
+/*
+ * Safeguard to avoid saving wrong flags:
+ *  - CE_EXTENDED2 won't get saved until its semantic is known
+ *  - Bits in 0x0000FFFF have been saved in ce_flags already
+ *  - Bits in 0x003F0000 are currently in-memory flags
+ */
+#if CE_EXTENDED_FLAGS & 0x80CFFFFF
+#error "CE_EXTENDED_FLAGS out of range"
+#endif
+
+/*
  * Copy the sha1 and stat state of a cache entry from one to
  * another. But we never change the name, or the hash state!
  */
@@ -171,7 +217,9 @@ static inline size_t ce_namelen(const struct cache_entry *ce)
 }
 
 #define ce_size(ce) cache_entry_size(ce_namelen(ce))
-#define ondisk_ce_size(ce) ondisk_cache_entry_size(ce_namelen(ce))
+#define ondisk_ce_size(ce) (((ce)->ce_flags & CE_EXTENDED) ? \
+			    ondisk_cache_entry_extended_size(ce_namelen(ce)) : \
+			    ondisk_cache_entry_size(ce_namelen(ce)))
 #define ce_stage(ce) ((CE_STAGEMASK & (ce)->ce_flags) >> CE_STAGESHIFT)
 #define ce_uptodate(ce) ((ce)->ce_flags & CE_UPTODATE)
 #define ce_mark_uptodate(ce) ((ce)->ce_flags |= CE_UPTODATE)
@@ -214,8 +262,10 @@ static inline int ce_to_dtype(const struct cache_entry *ce)
 	(S_ISREG(mode) ? (S_IFREG | ce_permissions(mode)) : \
 	S_ISLNK(mode) ? S_IFLNK : S_ISDIR(mode) ? S_IFDIR : S_IFGITLINK)
 
-#define cache_entry_size(len) ((offsetof(struct cache_entry,name) + (len) + 8) & ~7)
-#define ondisk_cache_entry_size(len) ((offsetof(struct ondisk_cache_entry,name) + (len) + 8) & ~7)
+#define flexible_size(STRUCT,len) ((offsetof(struct STRUCT,name) + (len) + 8) & ~7)
+#define cache_entry_size(len) flexible_size(cache_entry,len)
+#define ondisk_cache_entry_size(len) flexible_size(ondisk_cache_entry,len)
+#define ondisk_cache_entry_extended_size(len) flexible_size(ondisk_cache_entry_extended,len)
 
 struct index_state {
 	struct cache_entry **cache;
diff --git a/read-cache.c b/read-cache.c
index c5a8659..667c36b 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1096,7 +1096,7 @@ static int verify_hdr(struct cache_header *hdr, unsigned long size)
 
 	if (hdr->hdr_signature != htonl(CACHE_SIGNATURE))
 		return error("bad signature");
-	if (hdr->hdr_version != htonl(2))
+	if (hdr->hdr_version != htonl(2) && hdr->hdr_version != htonl(3))
 		return error("bad index version");
 	SHA1_Init(&c);
 	SHA1_Update(&c, hdr, size - 20);
@@ -1131,6 +1131,7 @@ int read_index(struct index_state *istate)
 static void convert_from_disk(struct ondisk_cache_entry *ondisk, struct cache_entry *ce)
 {
 	size_t len;
+	const char *name;
 
 	ce->ce_ctime = ntohl(ondisk->ctime.sec);
 	ce->ce_mtime = ntohl(ondisk->mtime.sec);
@@ -1143,19 +1144,31 @@ static void convert_from_disk(struct ondisk_cache_entry *ondisk, struct cache_en
 	/* On-disk flags are just 16 bits */
 	ce->ce_flags = ntohs(ondisk->flags);
 
-	/* For future extension: we do not understand this entry yet */
-	if (ce->ce_flags & CE_EXTENDED)
-		die("Unknown index entry format");
 	hashcpy(ce->sha1, ondisk->sha1);
 
 	len = ce->ce_flags & CE_NAMEMASK;
+
+	if (ce->ce_flags & CE_EXTENDED) {
+		struct ondisk_cache_entry_extended *ondisk2;
+		int extended_flags;
+		ondisk2 = (struct ondisk_cache_entry_extended *)ondisk;
+		extended_flags = ntohs(ondisk2->flags2) << 16;
+		/* We do not yet understand any bit out of CE_EXTENDED_FLAGS */
+		if (extended_flags & ~CE_EXTENDED_FLAGS)
+			die("Unknown index entry format %08x", extended_flags);
+		ce->ce_flags |= extended_flags;
+		name = ondisk2->name;
+	}
+	else
+		name = ondisk->name;
+
 	if (len == CE_NAMEMASK)
-		len = strlen(ondisk->name);
+		len = strlen(name);
 	/*
 	 * NEEDSWORK: If the original index is crafted, this copy could
 	 * go unchecked.
 	 */
-	memcpy(ce->name, ondisk->name, len + 1);
+	memcpy(ce->name, name, len + 1);
 }
 
 static inline size_t estimate_cache_size(size_t ondisk_size, unsigned int entries)
@@ -1415,6 +1428,7 @@ static int ce_write_entry(SHA_CTX *c, int fd, struct cache_entry *ce)
 {
 	int size = ondisk_ce_size(ce);
 	struct ondisk_cache_entry *ondisk = xcalloc(1, size);
+	char *name;
 
 	ondisk->ctime.sec = htonl(ce->ce_ctime);
 	ondisk->ctime.nsec = 0;
@@ -1428,7 +1442,15 @@ static int ce_write_entry(SHA_CTX *c, int fd, struct cache_entry *ce)
 	ondisk->size = htonl(ce->ce_size);
 	hashcpy(ondisk->sha1, ce->sha1);
 	ondisk->flags = htons(ce->ce_flags);
-	memcpy(ondisk->name, ce->name, ce_namelen(ce));
+	if (ce->ce_flags & CE_EXTENDED) {
+		struct ondisk_cache_entry_extended *ondisk2;
+		ondisk2 = (struct ondisk_cache_entry_extended *)ondisk;
+		ondisk2->flags2 = htons((ce->ce_flags & CE_EXTENDED_FLAGS) >> 16);
+		name = ondisk2->name;
+	}
+	else
+		name = ondisk->name;
+	memcpy(name, ce->name, ce_namelen(ce));
 
 	return ce_write(c, fd, ondisk, size);
 }
@@ -1437,16 +1459,25 @@ int write_index(const struct index_state *istate, int newfd)
 {
 	SHA_CTX c;
 	struct cache_header hdr;
-	int i, err, removed;
+	int i, err, removed, extended;
 	struct cache_entry **cache = istate->cache;
 	int entries = istate->cache_nr;
 
-	for (i = removed = 0; i < entries; i++)
+	for (i = removed = extended = 0; i < entries; i++) {
 		if (cache[i]->ce_flags & CE_REMOVE)
 			removed++;
 
+		/* reduce extended entries if possible */
+		cache[i]->ce_flags &= ~CE_EXTENDED;
+		if (cache[i]->ce_flags & CE_EXTENDED_FLAGS) {
+			extended++;
+			cache[i]->ce_flags |= CE_EXTENDED;
+		}
+	}
+
 	hdr.hdr_signature = htonl(CACHE_SIGNATURE);
-	hdr.hdr_version = htonl(2);
+	/* for extended format, increase version so older git won't try to read it */
+	hdr.hdr_version = htonl(extended ? 3 : 2);
 	hdr.hdr_entries = htonl(entries - removed);
 
 	SHA1_Init(&c);
-- 
1.6.0.96.g2fad1.dirty

[PATCH 02/16] Introduce CE_NO_CHECKOUT bit

[Nguyễn Thái Ngọc Duy]

This bit is the basis of narrow checkout. If this bit is on, the entry
is outside narrow checkout and therefore should be ignored (similar
to CE_VALID)

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 cache.h      |   10 +++++++++-
 read-cache.c |    6 +++---
 2 files changed, 12 insertions(+), 4 deletions(-)

diff --git a/cache.h b/cache.h
index 1e572e4..2b2c90f 100644
--- a/cache.h
+++ b/cache.h
@@ -170,10 +170,11 @@ struct cache_entry {
 /*
  * Extended on-disk flags
  */
+#define CE_NO_CHECKOUT 0x40000000
 /* CE_EXTENDED2 is for future extension */
 #define CE_EXTENDED2 0x80000000
 
-#define CE_EXTENDED_FLAGS (0)
+#define CE_EXTENDED_FLAGS (CE_NO_CHECKOUT)
 
 /*
  * Safeguard to avoid saving wrong flags:
@@ -185,6 +186,9 @@ struct cache_entry {
 #error "CE_EXTENDED_FLAGS out of range"
 #endif
 
+/* "Assume unchanged" mask */
+#define CE_VALID_MASK (CE_VALID | CE_NO_CHECKOUT)
+
 /*
  * Copy the sha1 and stat state of a cache entry from one to
  * another. But we never change the name, or the hash state!
@@ -222,6 +226,10 @@ static inline size_t ce_namelen(const struct cache_entry *ce)
 			    ondisk_cache_entry_size(ce_namelen(ce)))
 #define ce_stage(ce) ((CE_STAGEMASK & (ce)->ce_flags) >> CE_STAGESHIFT)
 #define ce_uptodate(ce) ((ce)->ce_flags & CE_UPTODATE)
+#define ce_no_checkout(ce) ((ce)->ce_flags & CE_NO_CHECKOUT)
+#define ce_checkout(ce) (!ce_no_checkout(ce))
+#define ce_mark_no_checkout(ce) ((ce)->ce_flags |= CE_NO_CHECKOUT)
+#define ce_mark_checkout(ce) ((ce)->ce_flags &= ~CE_NO_CHECKOUT)
 #define ce_mark_uptodate(ce) ((ce)->ce_flags |= CE_UPTODATE)
 
 #define ce_permissions(mode) (((mode) & 0100) ? 0755 : 0644)
diff --git a/read-cache.c b/read-cache.c
index 667c36b..e965a4c 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -254,7 +254,7 @@ int ie_match_stat(const struct index_state *istate,
 	 * If it's marked as always valid in the index, it's
 	 * valid whatever the checked-out copy says.
 	 */
-	if (!ignore_valid && (ce->ce_flags & CE_VALID))
+	if (!ignore_valid && (ce->ce_flags & CE_VALID_MASK))
 		return 0;
 
 	changed = ce_match_stat_basic(ce, st);
@@ -962,10 +962,10 @@ static struct cache_entry *refresh_cache_ent(struct index_state *istate,
 		return ce;
 
 	/*
-	 * CE_VALID means the user promised us that the change to
+	 * CE_VALID_MASK means the user promised us that the change to
 	 * the work tree does not matter and told us not to worry.
 	 */
-	if (!ignore_valid && (ce->ce_flags & CE_VALID)) {
+	if (!ignore_valid && (ce->ce_flags & CE_VALID_MASK)) {
 		ce_mark_uptodate(ce);
 		return ce;
 	}
-- 
1.6.0.96.g2fad1.dirty

[PATCH 03/16] update-index: refactor mark_valid() in preparation for new options

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin-update-index.c |   24 ++++++++++--------------
 1 files changed, 10 insertions(+), 14 deletions(-)

diff --git a/builtin-update-index.c b/builtin-update-index.c
index ce83224..f63a34e 100644
--- a/builtin-update-index.c
+++ b/builtin-update-index.c
@@ -24,8 +24,8 @@ static int info_only;
 static int force_remove;
 static int verbose;
 static int mark_valid_only;
-#define MARK_VALID 1
-#define UNMARK_VALID 2
+#define MARK_FLAG 1
+#define UNMARK_FLAG 2
 
 static void report(const char *fmt, ...)
 {
@@ -40,19 +40,15 @@ static void report(const char *fmt, ...)
 	va_end(vp);
 }
 
-static int mark_valid(const char *path)
+static int mark_ce_flags(const char *path, int flag, int mark)
 {
 	int namelen = strlen(path);
 	int pos = cache_name_pos(path, namelen);
 	if (0 <= pos) {
-		switch (mark_valid_only) {
-		case MARK_VALID:
-			active_cache[pos]->ce_flags |= CE_VALID;
-			break;
-		case UNMARK_VALID:
-			active_cache[pos]->ce_flags &= ~CE_VALID;
-			break;
-		}
+		if (mark)
+			active_cache[pos]->ce_flags |= flag;
+		else
+			active_cache[pos]->ce_flags &= ~flag;
 		cache_tree_invalidate_path(active_cache_tree, path);
 		active_cache_changed = 1;
 		return 0;
@@ -276,7 +272,7 @@ static void update_one(const char *path, const char *prefix, int prefix_length)
 		goto free_return;
 	}
 	if (mark_valid_only) {
-		if (mark_valid(p))
+		if (mark_ce_flags(p, CE_VALID, mark_valid_only == MARK_FLAG))
 			die("Unable to mark file %s", path);
 		goto free_return;
 	}
@@ -649,11 +645,11 @@ int cmd_update_index(int argc, const char **argv, const char *prefix)
 				continue;
 			}
 			if (!strcmp(path, "--assume-unchanged")) {
-				mark_valid_only = MARK_VALID;
+				mark_valid_only = MARK_FLAG;
 				continue;
 			}
 			if (!strcmp(path, "--no-assume-unchanged")) {
-				mark_valid_only = UNMARK_VALID;
+				mark_valid_only = UNMARK_FLAG;
 				continue;
 			}
 			if (!strcmp(path, "--info-only")) {
-- 
1.6.0.96.g2fad1.dirty

[PATCH 04/16] update-index: add --checkout/--no-checkout to update CE_NO_CHECKOUT bit

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git-update-index.txt |   12 ++++++++++++
 builtin-update-index.c             |   16 +++++++++++++++-
 2 files changed, 27 insertions(+), 1 deletions(-)

diff --git a/Documentation/git-update-index.txt b/Documentation/git-update-index.txt
index 1d9d81a..707c33f 100644
--- a/Documentation/git-update-index.txt
+++ b/Documentation/git-update-index.txt
@@ -15,6 +15,7 @@ SYNOPSIS
 	     [--cacheinfo <mode> <object> <file>]\*
 	     [--chmod=(+|-)x]
 	     [--assume-unchanged | --no-assume-unchanged]
+	     [--checkout | --no-checkout]
 	     [--ignore-submodules]
 	     [--really-refresh] [--unresolve] [--again | -g]
 	     [--info-only] [--index-info]
@@ -88,6 +89,17 @@ OPTIONS
 	sometimes helpful when working with a big project on a
 	filesystem that has very slow lstat(2) system call
 	(e.g. cifs).
+--checkout::
+--no-checkout::
+	When these flags are specified, the object name recorded
+	for the paths are not updated. Instead, these options
+	set and unset the "no-checkout" bit for the paths. This
+	bit is used for marking files for narrow checkout. If
+	a path is marked "no-checkout", then it should not be
+	checked out unless requested by user or needed for a git
+	command to function.
+	See linkgit:git-checkout[1] for more information about
+	narrow checkout.
 +
 This option can be also used as a coarse file-level mechanism
 to ignore uncommitted changes in tracked files (akin to what
diff --git a/builtin-update-index.c b/builtin-update-index.c
index f63a34e..6ed883d 100644
--- a/builtin-update-index.c
+++ b/builtin-update-index.c
@@ -24,6 +24,7 @@ static int info_only;
 static int force_remove;
 static int verbose;
 static int mark_valid_only;
+static int mark_no_checkout_only;
 #define MARK_FLAG 1
 #define UNMARK_FLAG 2
 
@@ -276,6 +277,11 @@ static void update_one(const char *path, const char *prefix, int prefix_length)
 			die("Unable to mark file %s", path);
 		goto free_return;
 	}
+	if (mark_no_checkout_only) {
+		if (mark_ce_flags(p, CE_NO_CHECKOUT, mark_no_checkout_only == MARK_FLAG))
+			die("Unable to mark file %s", path);
+		goto free_return;
+	}
 
 	if (force_remove) {
 		if (remove_file_from_cache(p))
@@ -386,7 +392,7 @@ static void read_index_info(int line_termination)
 }
 
 static const char update_index_usage[] =
-"git update-index [-q] [--add] [--replace] [--remove] [--unmerged] [--refresh] [--really-refresh] [--cacheinfo] [--chmod=(+|-)x] [--assume-unchanged] [--info-only] [--force-remove] [--stdin] [--index-info] [--unresolve] [--again | -g] [--ignore-missing] [-z] [--verbose] [--] <file>...";
+"git update-index [-q] [--add] [--replace] [--remove] [--unmerged] [--refresh] [--really-refresh] [--cacheinfo] [--chmod=(+|-)x] [--assume-unchanged] [--checkout|--no-checkout] [--info-only] [--force-remove] [--stdin] [--index-info] [--unresolve] [--again | -g] [--ignore-missing] [-z] [--verbose] [--] <file>...";
 
 static unsigned char head_sha1[20];
 static unsigned char merge_head_sha1[20];
@@ -652,6 +658,14 @@ int cmd_update_index(int argc, const char **argv, const char *prefix)
 				mark_valid_only = UNMARK_FLAG;
 				continue;
 			}
+			if (!strcmp(path, "--checkout")) {
+				mark_no_checkout_only = UNMARK_FLAG;
+				continue;
+			}
+			if (!strcmp(path, "--no-checkout")) {
+				mark_no_checkout_only = MARK_FLAG;
+				continue;
+			}
 			if (!strcmp(path, "--info-only")) {
 				info_only = 1;
 				continue;
-- 
1.6.0.96.g2fad1.dirty

[PATCH 05/16] ls-files: add --narrow-checkout option to "will checkout" entries

[Nguyễn Thái Ngọc Duy]

Even in narrow checkout mode, "git ls-files --cached" (and --stage)
will show all entries in index. When those options are used together
with --narrow-checkout, no-checkout entries will be skipped.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git-ls-files.txt |    6 ++++++
 builtin-ls-files.c             |   11 +++++++++++
 2 files changed, 17 insertions(+), 0 deletions(-)

diff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt
index 9f85d60..f74b212 100644
--- a/Documentation/git-ls-files.txt
+++ b/Documentation/git-ls-files.txt
@@ -12,6 +12,7 @@ SYNOPSIS
 'git ls-files' [-z] [-t] [-v]
 		(--[cached|deleted|others|ignored|stage|unmerged|killed|modified])\*
 		(-[c|d|o|i|s|u|k|m])\*
+		[--narrow-checkout]
 		[-x <pattern>|--exclude=<pattern>]
 		[-X <file>|--exclude-from=<file>]
 		[--exclude-per-directory=<file>]
@@ -72,6 +73,11 @@ OPTIONS
 	to file/directory conflicts for checkout-index to
 	succeed.
 
+--narrow-checkout::
+	When narrow checkout is being used, this option together with other
+	index-based selection options like --cached or --stage, only narrowed
+	portion will be printed out.
+
 -z::
 	\0 line termination on output.
 
diff --git a/builtin-ls-files.c b/builtin-ls-files.c
index 068f424..456d41c 100644
--- a/builtin-ls-files.c
+++ b/builtin-ls-files.c
@@ -20,6 +20,7 @@ static int show_unmerged;
 static int show_modified;
 static int show_killed;
 static int show_valid_bit;
+static int narrow_checkout;
 static int line_terminator = '\n';
 
 static int prefix_len;
@@ -245,6 +246,8 @@ static void show_files(struct dir_struct *dir, const char *prefix)
 				continue;
 			if (ce->ce_flags & CE_UPDATE)
 				continue;
+			if (narrow_checkout && ce_no_checkout(ce))
+				continue;
 			show_ce_entry(ce_stage(ce) ? tag_unmerged : tag_cached, ce);
 		}
 	}
@@ -424,6 +427,7 @@ int report_path_error(const char *ps_matched, const char **pathspec, int prefix_
 
 static const char ls_files_usage[] =
 	"git ls-files [-z] [-t] [-v] (--[cached|deleted|others|stage|unmerged|killed|modified])* "
+	"[ --narrow-checkout ] "
 	"[ --ignored ] [--exclude=<pattern>] [--exclude-from=<file>] "
 	"[ --exclude-per-directory=<filename> ] [--exclude-standard] "
 	"[--full-name] [--abbrev] [--] [<file>]*";
@@ -465,6 +469,10 @@ int cmd_ls_files(int argc, const char **argv, const char *prefix)
 			show_cached = 1;
 			continue;
 		}
+		if (!strcmp(arg, "--narrow-checkout")) {
+			narrow_checkout = 1;
+			continue;
+		}
 		if (!strcmp(arg, "-d") || !strcmp(arg, "--deleted")) {
 			show_deleted = 1;
 			continue;
@@ -596,6 +604,9 @@ int cmd_ls_files(int argc, const char **argv, const char *prefix)
 	      show_killed | show_modified))
 		show_cached = 1;
 
+	if (narrow_checkout && !show_cached && !show_stage)
+		die("ls-files: --narrow-checkout can only be used with either --cached or --stage");
+
 	read_cache();
 	if (prefix)
 		prune_cache(prefix);
-- 
1.6.0.96.g2fad1.dirty

[PATCH 06/16] Add tests for updating no-checkout entries in index

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 .gitignore                     |    1 +
 Makefile                       |    2 +-
 t/t2104-update-index-narrow.sh |   36 ++++++++++++++++++++++++++++++++++++
 test-index-version.c           |   14 ++++++++++++++
 4 files changed, 52 insertions(+), 1 deletions(-)
 create mode 100755 t/t2104-update-index-narrow.sh
 create mode 100644 test-index-version.c

diff --git a/.gitignore b/.gitignore
index bbaf9de..0c35577 100644
--- a/.gitignore
+++ b/.gitignore
@@ -147,6 +147,7 @@ test-date
 test-delta
 test-dump-cache-tree
 test-genrandom
+test-index-version
 test-match-trees
 test-parse-options
 test-path-utils
diff --git a/Makefile b/Makefile
index 716161d..7a22bde 100644
--- a/Makefile
+++ b/Makefile
@@ -1323,7 +1323,7 @@ endif
 
 ### Testing rules
 
-TEST_PROGRAMS = test-chmtime$X test-genrandom$X test-date$X test-delta$X test-sha1$X test-match-trees$X test-parse-options$X test-path-utils$X
+TEST_PROGRAMS = test-chmtime$X test-genrandom$X test-date$X test-delta$X test-sha1$X test-match-trees$X test-parse-options$X test-path-utils$X test-index-version$X
 
 all:: $(TEST_PROGRAMS)
 
diff --git a/t/t2104-update-index-narrow.sh b/t/t2104-update-index-narrow.sh
new file mode 100755
index 0000000..2683929
--- /dev/null
+++ b/t/t2104-update-index-narrow.sh
@@ -0,0 +1,36 @@
+#!/bin/sh
+#
+# Copyright (c) 2008 Nguyễn Thái Ngọc Duy
+#
+
+test_description='git update-index narrow checkout update'
+
+. ./test-lib.sh
+
+test_expect_success 'setup' '
+	mkdir sub &&
+	touch 1 2 sub/1 sub/2 &&
+	git add 1 2 sub/1 sub/2
+'
+
+test_expect_success 'index is at version 2' '
+	test "$(test-index-version < .git/index)" = 2
+'
+
+test_expect_success 'update-index --no-checkout' '
+	git update-index --no-checkout 1 sub/1 &&
+	test -z "$(git ls-files --narrow-checkout|grep 1)"'
+
+test_expect_success 'index is at version 3 after having some no-checkout entries' '
+	test "$(test-index-version < .git/index)" = 3
+'
+
+test_expect_success 'update-index --checkout' '
+	git update-index --checkout 1 sub/1 &&
+	test "$(git ls-files)" = "$(git ls-files --narrow-checkout)"'
+
+test_expect_success 'index version is back to 2 when there is no no-checkout entry' '
+	test "$(test-index-version < .git/index)" = 2
+'
+
+test_done
diff --git a/test-index-version.c b/test-index-version.c
new file mode 100644
index 0000000..bfaad9e
--- /dev/null
+++ b/test-index-version.c
@@ -0,0 +1,14 @@
+#include "cache.h"
+
+int main(int argc, const char **argv)
+{
+	struct cache_header hdr;
+	int version;
+
+	memset(&hdr,0,sizeof(hdr));
+	if (read(0, &hdr, sizeof(hdr)) != sizeof(hdr))
+		return 0;
+	version = ntohl(hdr.hdr_version);
+	printf("%d\n", version);
+	return 0;
+}
-- 
1.6.0.96.g2fad1.dirty

[PATCH 07/16] Prevent diff machinery from examining worktree outside narrow checkout

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 diff-lib.c |    5 +++--
 diff.c     |    4 +++-
 2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/diff-lib.c b/diff-lib.c
index ae96c64..992280b 100644
--- a/diff-lib.c
+++ b/diff-lib.c
@@ -161,7 +161,7 @@ int run_diff_files(struct rev_info *revs, unsigned int option)
 				continue;
 		}
 
-		if (ce_uptodate(ce))
+		if (ce_uptodate(ce) || ce_no_checkout(ce))
 			continue;
 
 		changed = check_removed(ce, &st);
@@ -348,6 +348,8 @@ static void do_oneway_diff(struct unpack_trees_options *o,
 	struct rev_info *revs = cbdata->revs;
 	int match_missing, cached;
 
+	/* if the entry is not checked out, don't examine work tree */
+	cached = o->index_only || (idx && ce_no_checkout(idx));
 	/*
 	 * Backward compatibility wart - "diff-index -m" does
 	 * not mean "do not ignore merges", but "match_missing".
@@ -355,7 +357,6 @@ static void do_oneway_diff(struct unpack_trees_options *o,
 	 * But with the revision flag parsing, that's found in
 	 * "!revs->ignore_merges".
 	 */
-	cached = o->index_only;
 	match_missing = !revs->ignore_merges;
 
 	if (cached && idx && ce_stage(idx)) {
diff --git a/diff.c b/diff.c
index a2f4850..7a8f520 100644
--- a/diff.c
+++ b/diff.c
@@ -1784,8 +1784,10 @@ static int reuse_worktree_file(const char *name, const unsigned char *sha1, int
 
 	/*
 	 * If ce matches the file in the work tree, we can reuse it.
+	 * For narrow checkout case, ce_uptodate() may be true although
+	 * the file may or may not exist in the work tree.
 	 */
-	if (ce_uptodate(ce) ||
+	if ((ce_uptodate(ce) && ce_checkout(ce)) ||
 	    (!lstat(name, &st) && !ce_match_stat(ce, &st, 0)))
 		return 1;
 
-- 
1.6.0.96.g2fad1.dirty

[PATCH 08/16] checkout_entry(): CE_NO_CHECKOUT on checked out entries.

[Nguyễn Thái Ngọc Duy]

With this you can just do "git checkout some-files" to
widen your checkout. One caveat though: caller must save
the index.

For all of its callers (unpack_trees(), checkout-index, checkout
and apply), only "git apply" does not write index back.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 entry.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/entry.c b/entry.c
index aa2ee46..305f8d3 100644
--- a/entry.c
+++ b/entry.c
@@ -230,5 +230,6 @@ int checkout_entry(struct cache_entry *ce, const struct checkout *state, char *t
 	} else if (state->not_new)
 		return 0;
 	create_directories(path, state);
+	ce_mark_checkout(ce);
 	return write_entry(ce, path, state, 0);
 }
-- 
1.6.0.96.g2fad1.dirty

[PATCH 09/16] ls-files: apply --deleted on narrow area only

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin-ls-files.c             |    2 +-
 t/t2104-update-index-narrow.sh |    6 ++++++
 2 files changed, 7 insertions(+), 1 deletions(-)

diff --git a/builtin-ls-files.c b/builtin-ls-files.c
index 456d41c..02fa00b 100644
--- a/builtin-ls-files.c
+++ b/builtin-ls-files.c
@@ -260,7 +260,7 @@ static void show_files(struct dir_struct *dir, const char *prefix)
 			if (excluded(dir, ce->name, &dtype) != dir->show_ignored)
 				continue;
 			err = lstat(ce->name, &st);
-			if (show_deleted && err)
+			if (show_deleted && err && ce_checkout(ce))
 				show_ce_entry(tag_removed, ce);
 			if (show_modified && ce_modified(ce, &st, 0))
 				show_ce_entry(tag_modified, ce);
diff --git a/t/t2104-update-index-narrow.sh b/t/t2104-update-index-narrow.sh
index 2683929..1a3acdd 100755
--- a/t/t2104-update-index-narrow.sh
+++ b/t/t2104-update-index-narrow.sh
@@ -25,6 +25,12 @@ test_expect_success 'index is at version 3 after having some no-checkout entries
 	test "$(test-index-version < .git/index)" = 3
 '
 
+test_expect_success 'ls-files --deleted ignores no-checkout entries' '
+	rm 1 &&
+	test -z "$(git ls-files --deleted)" &&
+	touch 1
+'
+
 test_expect_success 'update-index --checkout' '
 	git update-index --checkout 1 sub/1 &&
 	test "$(git ls-files)" = "$(git ls-files --narrow-checkout)"'
-- 
1.6.0.96.g2fad1.dirty

[PATCH 10/16] grep: skip files that have not been checked out

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin-grep.c |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)

diff --git a/builtin-grep.c b/builtin-grep.c
index 3a51662..d5507d7 100644
--- a/builtin-grep.c
+++ b/builtin-grep.c
@@ -343,6 +343,8 @@ static int external_grep(struct grep_opt *opt, const char **paths, int cached)
 			continue;
 		if (!pathspec_matches(paths, ce->name))
 			continue;
+		if (ce_no_checkout(ce))
+			continue;
 		name = ce->name;
 		if (name[0] == '-') {
 			int len = ce_namelen(ce);
@@ -404,8 +406,11 @@ static int grep_cache(struct grep_opt *opt, const char **paths, int cached)
 				continue;
 			hit |= grep_sha1(opt, ce->sha1, ce->name, 0);
 		}
-		else
+		else {
+			if (ce_no_checkout(ce))
+				continue;
 			hit |= grep_file(opt, ce->name);
+		}
 		if (ce_stage(ce)) {
 			do {
 				nr++;
-- 
1.6.0.96.g2fad1.dirty

[PATCH 11/16] unpack_trees(): add support for narrow checkout

[Nguyễn Thái Ngọc Duy]

This patch teaches unpack_trees() to checkout/remove entries
on working directories appropriately when narrow area is
changed. There are three kind of changes:

 - new_narrow_path: reset workdir to a new narrow checkout
 - add_narrow_path: keep current narrow areas and add more entries
 - remove_narrow_path: remove some entries from current narrow areas

A simple "narrow spec" is introduced to specify what entries is in
narrow area. For now it is just a list of prefix separated by colon.

CE_WD_REMOVE is introduced to remove entries from working directories,
but still keep them in index

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 cache.h        |    3 ++
 unpack-trees.c |  100 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 unpack-trees.h |    6 +++
 3 files changed, 109 insertions(+), 0 deletions(-)

diff --git a/cache.h b/cache.h
index 2b2c90f..1fc0f83 100644
--- a/cache.h
+++ b/cache.h
@@ -167,6 +167,9 @@ struct cache_entry {
 #define CE_HASHED    (0x100000)
 #define CE_UNHASHED  (0x200000)
 
+/* Only remove in work directory, not index */
+#define CE_WD_REMOVE (0x400000)
+
 /*
  * Extended on-disk flags
  */
diff --git a/unpack-trees.c b/unpack-trees.c
index e59d144..968cc98 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -96,6 +96,15 @@ static int check_updates(struct unpack_trees_options *o)
 	if (o->update && o->verbose_update) {
 		for (total = cnt = 0; cnt < index->cache_nr; cnt++) {
 			struct cache_entry *ce = index->cache[cnt];
+
+			if (ce->ce_flags & CE_WD_REMOVE) {
+				total++;
+				continue;
+			}
+
+			if (ce_no_checkout(ce))
+				continue;
+
 			if (ce->ce_flags & (CE_UPDATE | CE_REMOVE))
 				total++;
 		}
@@ -108,6 +117,16 @@ static int check_updates(struct unpack_trees_options *o)
 	for (i = 0; i < index->cache_nr; i++) {
 		struct cache_entry *ce = index->cache[i];
 
+		if (ce->ce_flags & CE_WD_REMOVE) {
+			display_progress(progress, ++cnt);
+			if (o->update)
+				unlink_entry(ce);
+			continue;
+		}
+
+		if (ce_no_checkout(ce))
+			continue;
+
 		if (ce->ce_flags & CE_REMOVE) {
 			display_progress(progress, ++cnt);
 			if (o->update)
@@ -121,6 +140,9 @@ static int check_updates(struct unpack_trees_options *o)
 	for (i = 0; i < index->cache_nr; i++) {
 		struct cache_entry *ce = index->cache[i];
 
+		if (ce_no_checkout(ce))
+			continue;
+
 		if (ce->ce_flags & CE_UPDATE) {
 			display_progress(progress, ++cnt);
 			ce->ce_flags &= ~CE_UPDATE;
@@ -133,6 +155,47 @@ static int check_updates(struct unpack_trees_options *o)
 	return errs != 0;
 }
 
+static int verify_uptodate(struct cache_entry *ce, struct unpack_trees_options *o);
+static int apply_narrow_checkout(struct unpack_trees_options *o)
+{
+	struct index_state *index = &o->result;
+	int i;
+
+	if (!(o->new_narrow_path | o->add_narrow_path | o->remove_narrow_path))
+		return 0;
+
+	for (i = 0; i < index->cache_nr; i++) {
+		struct cache_entry *ce = index->cache[i];
+		int was_checkout = ce_checkout(ce);
+		int match = match_narrow_spec(o->narrow_spec, ce->name, o->narrow_prefix);
+
+		if (ce_stage(ce))
+			continue;
+
+		if (o->new_narrow_path) {
+			if (match)
+				ce_mark_checkout(ce);
+			else
+				ce_mark_no_checkout(ce);
+		}
+
+		if (o->add_narrow_path && match)
+			ce_mark_checkout(ce);
+
+		if (o->remove_narrow_path && match)
+			ce_mark_no_checkout(ce);
+
+		if (was_checkout && ce_no_checkout(ce)) {
+			if (verify_uptodate(ce, o))
+				return -1;
+			ce->ce_flags |= CE_WD_REMOVE;
+		}
+		if (!was_checkout && ce_checkout(ce))
+			ce->ce_flags |= CE_UPDATE;
+	}
+	return 0;
+}
+
 static inline int call_unpack_fn(struct cache_entry **src, struct unpack_trees_options *o)
 {
 	int ret = o->fn(src, o);
@@ -409,6 +472,9 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options
 	if (o->trivial_merges_only && o->nontrivial_merge)
 		return unpack_failed(o, "Merge requires file-level merging");
 
+	if (apply_narrow_checkout(o))
+		return unpack_failed(o, NULL);
+
 	o->src_index = NULL;
 	ret = check_updates(o) ? (-2) : 0;
 	if (o->dst_index)
@@ -677,6 +743,8 @@ static int merged_entry(struct cache_entry *merge, struct cache_entry *old,
 				return -1;
 			invalidate_ce_path(old, o);
 		}
+		if (ce_no_checkout(old))
+			update |= CE_NO_CHECKOUT;
 	}
 	else {
 		if (verify_absent(merge, "overwritten", o))
@@ -726,6 +794,38 @@ static void show_stage_entry(FILE *o,
 }
 #endif
 
+int match_narrow_spec(const char *spec_, const char *path, const char *prefix)
+{
+	int match = 0;
+	char *spec, *cur_spec;
+	int prefix_len = 0;
+
+	if (!spec_)
+		return 1; /* always match if spec_ is NULL */
+	if (prefix) {
+		if (prefixcmp(path, prefix))
+			return 0;
+		prefix_len = strlen(prefix);
+	}
+
+	spec = cur_spec = xstrdup(spec_);
+
+	while (!match) {
+		char *next_spec = strchr(cur_spec, ':');
+		if (next_spec)
+			*next_spec = '\0';
+
+		if (!fnmatch(cur_spec, path+prefix_len, 0))
+			match = 1;
+
+		if (!next_spec)
+			break;
+		cur_spec = next_spec+1;
+	}
+	free(spec);
+	return match;
+}
+
 int threeway_merge(struct cache_entry **stages, struct unpack_trees_options *o)
 {
 	struct cache_entry *index;
diff --git a/unpack-trees.h b/unpack-trees.h
index 0d26f3d..942a007 100644
--- a/unpack-trees.h
+++ b/unpack-trees.h
@@ -27,6 +27,9 @@ struct unpack_trees_options {
 		     aggressive:1,
 		     skip_unmerged:1,
 		     initial_checkout:1,
+		     new_narrow_path:1,
+		     add_narrow_path:2,
+		     remove_narrow_path:2,
 		     gently:1;
 	const char *prefix;
 	int pos;
@@ -38,6 +41,8 @@ struct unpack_trees_options {
 	int merge_size;
 
 	struct cache_entry *df_conflict_entry;
+	const char *narrow_spec;
+	const char *narrow_prefix;
 	void *unpack_data;
 
 	struct index_state *dst_index;
@@ -48,6 +53,7 @@ struct unpack_trees_options {
 extern int unpack_trees(unsigned n, struct tree_desc *t,
 		struct unpack_trees_options *options);
 
+int match_narrow_spec(const char *spec_, const char *path, const char *prefix);
 int threeway_merge(struct cache_entry **stages, struct unpack_trees_options *o);
 int twoway_merge(struct cache_entry **src, struct unpack_trees_options *o);
 int bind_merge(struct cache_entry **src, struct unpack_trees_options *o);
-- 
1.6.0.96.g2fad1.dirty

[PATCH 12/16] narrow spec: put '+' before a spec will change semantic of '*'

[Nguyễn Thái Ngọc Duy]

Previously '*' in narrow spec means 'match everything, even slashes'.
With this change, '*' is now 'match everything except slashes' so it is
more like shell wildcard one. Putting '+' before a spec will make
'*' as powerful as before.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 unpack-trees.c |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)

diff --git a/unpack-trees.c b/unpack-trees.c
index 968cc98..3782de6 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -812,10 +812,15 @@ int match_narrow_spec(const char *spec_, const char *path, const char *prefix)
 
 	while (!match) {
 		char *next_spec = strchr(cur_spec, ':');
+		int flag = 0;
 		if (next_spec)
 			*next_spec = '\0';
 
-		if (!fnmatch(cur_spec, path+prefix_len, 0))
+		if (*cur_spec == '+')
+			cur_spec++;
+		else
+			flag = FNM_PATHNAME;
+		if (!fnmatch(cur_spec, path+prefix_len, flag))
 			match = 1;
 
 		if (!next_spec)
-- 
1.6.0.96.g2fad1.dirty

[PATCH 13/16] ls-files: add --narrow-match=spec option for testing narrow matching

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git-ls-files.txt   |    7 +++++-
 builtin-ls-files.c               |   14 +++++++++++-
 t/t3003-ls-files-narrow-match.sh |   45 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 64 insertions(+), 2 deletions(-)
 create mode 100755 t/t3003-ls-files-narrow-match.sh

diff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt
index f74b212..4f5a37e 100644
--- a/Documentation/git-ls-files.txt
+++ b/Documentation/git-ls-files.txt
@@ -12,7 +12,7 @@ SYNOPSIS
 'git ls-files' [-z] [-t] [-v]
 		(--[cached|deleted|others|ignored|stage|unmerged|killed|modified])\*
 		(-[c|d|o|i|s|u|k|m])\*
-		[--narrow-checkout]
+		[--narrow-checkout] [--narrow-match=<narrow_spec>]
 		[-x <pattern>|--exclude=<pattern>]
 		[-X <file>|--exclude-from=<file>]
 		[--exclude-per-directory=<file>]
@@ -78,6 +78,11 @@ OPTIONS
 	index-based selection options like --cached or --stage, only narrowed
 	portion will be printed out.
 
+--narrow-match=<narrow_spec>::
+	Narrow spec can be applied on ls-files output so that you can test
+	your spec. Can only be used with --cached. See linkgit:git-checkout[1]
+	for more information about narrow spec.
+
 -z::
 	\0 line termination on output.
 
diff --git a/builtin-ls-files.c b/builtin-ls-files.c
index 02fa00b..f48a157 100644
--- a/builtin-ls-files.c
+++ b/builtin-ls-files.c
@@ -10,6 +10,8 @@
 #include "dir.h"
 #include "builtin.h"
 #include "tree.h"
+#include "tree-walk.h"
+#include "unpack-trees.h"
 
 static int abbrev;
 static int show_deleted;
@@ -29,6 +31,7 @@ static const char **pathspec;
 static int error_unmatch;
 static char *ps_matched;
 static const char *with_tree;
+static const char *narrow_spec;
 
 static const char *tag_cached = "";
 static const char *tag_unmerged = "";
@@ -248,6 +251,8 @@ static void show_files(struct dir_struct *dir, const char *prefix)
 				continue;
 			if (narrow_checkout && ce_no_checkout(ce))
 				continue;
+			if (narrow_spec && !match_narrow_spec(narrow_spec, ce->name, prefix))
+				continue;
 			show_ce_entry(ce_stage(ce) ? tag_unmerged : tag_cached, ce);
 		}
 	}
@@ -427,7 +432,7 @@ int report_path_error(const char *ps_matched, const char **pathspec, int prefix_
 
 static const char ls_files_usage[] =
 	"git ls-files [-z] [-t] [-v] (--[cached|deleted|others|stage|unmerged|killed|modified])* "
-	"[ --narrow-checkout ] "
+	"[ --narrow-checkout ] [--narrow-match=<narrow_spec>] "
 	"[ --ignored ] [--exclude=<pattern>] [--exclude-from=<file>] "
 	"[ --exclude-per-directory=<filename> ] [--exclude-standard] "
 	"[--full-name] [--abbrev] [--] [<file>]*";
@@ -473,6 +478,10 @@ int cmd_ls_files(int argc, const char **argv, const char *prefix)
 			narrow_checkout = 1;
 			continue;
 		}
+		if (!prefixcmp(arg, "--narrow-match=")) {
+			narrow_spec = arg+15;
+			continue;
+		}
 		if (!strcmp(arg, "-d") || !strcmp(arg, "--deleted")) {
 			show_deleted = 1;
 			continue;
@@ -607,6 +616,9 @@ int cmd_ls_files(int argc, const char **argv, const char *prefix)
 	if (narrow_checkout && !show_cached && !show_stage)
 		die("ls-files: --narrow-checkout can only be used with either --cached or --stage");
 
+	if (narrow_spec && !show_cached && !show_stage)
+		die("ls-files: --narrow-match can only be used with either --cached or --stage");
+
 	read_cache();
 	if (prefix)
 		prune_cache(prefix);
diff --git a/t/t3003-ls-files-narrow-match.sh b/t/t3003-ls-files-narrow-match.sh
new file mode 100755
index 0000000..b48fdb2
--- /dev/null
+++ b/t/t3003-ls-files-narrow-match.sh
@@ -0,0 +1,45 @@
+#!/bin/sh
+
+test_description='This test is for narrow spec matching'
+
+. test-lib.sh
+
+test_expect_success 'setup' '
+	touch 1 2 3 &&
+	mkdir -p sub/subsub &&
+	touch sub/1 sub/2 sub/3 &&
+	touch sub/subsub/1 sub/subsub/2 sub/subsub/3 &&
+	git add .
+'
+
+test_expect_success 'exact matches' '
+	test "$(git ls-files --narrow-match=1)" = 1 &&
+	test -z "$(git ls-files --narrow-match=sub)" && # exact match on a directory gives nothing
+	(cd sub &&
+	test "$(git ls-files --full-name --narrow-match=subsub/1)" = sub/subsub/1 &&
+	test "$(git ls-files --narrow-match=subsub/1)" = subsub/1)
+'
+
+test_expect_success 'star matches' '
+	test "$(git ls-files --narrow-match=\*1)" = 1 &&
+	(cd sub &&
+	test "$(git ls-files --full-name --narrow-match=subsub/\*1)" = sub/subsub/1 &&
+	test "$(git ls-files --narrow-match=subsub/\*1)" = subsub/1)
+'
+
+cat >starplus-1.expected <<\EOF
+1
+sub/1
+sub/subsub/1
+EOF
+cat >starplus-2.expected <<\EOF
+sub/1
+sub/subsub/1
+EOF
+test_expect_success 'starplus matches' '
+	git ls-files --narrow-match=+\*1 | cmp starplus-1.expected
+	(cd sub &&
+	git ls-files --full-name --narrow-match=+\*1 | cmp ../starplus-2.expected)
+'
+
+test_done
-- 
1.6.0.96.g2fad1.dirty

[PATCH 14/16] clone: support narrow checkout with --path option

[Nguyễn Thái Ngọc Duy]

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git-clone.txt |    8 +++++++-
 builtin-clone.c             |   13 +++++++++++++
 t/t5703-clone-narrow.sh     |   39 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 59 insertions(+), 1 deletions(-)
 create mode 100755 t/t5703-clone-narrow.sh

diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 0e14e73..0a7d11d 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -12,7 +12,8 @@ SYNOPSIS
 'git clone' [--template=<template_directory>]
 	  [-l] [-s] [--no-hardlinks] [-q] [-n] [--bare] [--mirror]
 	  [-o <name>] [-u <upload-pack>] [--reference <repository>]
-	  [--depth <depth>] [--] <repository> [<directory>]
+	  [--depth <depth>] [--path=<narrow_spec>] [--]
+	  <repository> [<directory>]
 
 DESCRIPTION
 -----------
@@ -94,6 +95,11 @@ then the cloned repository will become corrupt.
 -n::
 	No checkout of HEAD is performed after the clone is complete.
 
+--path=<narrow_spec>::
+	A narrow checkout of HEAD is performed after clone is complete
+	with given narrow spec. Please refer to linkgit:git-checkout[1]
+	for more detail on narrow checkout and narrow spec.
+
 --bare::
 	Make a 'bare' GIT repository.  That is, instead of
 	creating `<directory>` and placing the administrative
diff --git a/builtin-clone.c b/builtin-clone.c
index c843529..37d44ca 100644
--- a/builtin-clone.c
+++ b/builtin-clone.c
@@ -36,6 +36,7 @@ static const char * const builtin_clone_usage[] = {
 static int option_quiet, option_no_checkout, option_bare, option_mirror;
 static int option_local, option_no_hardlinks, option_shared;
 static char *option_template, *option_reference, *option_depth;
+static char *option_narrow_path;
 static char *option_origin = NULL;
 static char *option_upload_pack = "git-upload-pack";
 
@@ -43,6 +44,8 @@ static struct option builtin_clone_options[] = {
 	OPT__QUIET(&option_quiet),
 	OPT_BOOLEAN('n', "no-checkout", &option_no_checkout,
 		    "don't create a checkout"),
+	OPT_STRING(0, "path", &option_narrow_path, "prefixes",
+		    "limit checkout to specified paths (narrow checkout)"),
 	OPT_BOOLEAN(0, "bare", &option_bare, "create a bare repository"),
 	OPT_BOOLEAN(0, "naked", &option_bare, "create a bare repository"),
 	OPT_BOOLEAN(0, "mirror", &option_mirror,
@@ -385,10 +388,15 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 		if (option_origin)
 			die("--bare and --origin %s options are incompatible.",
 			    option_origin);
+		if (option_narrow_path)
+			die("--bare and --path options are incompatible.");
 		option_no_checkout = 1;
 		use_separate_remote = 0;
 	}
 
+	if (option_no_checkout && option_narrow_path)
+		die("--no-checkout and --path options are incompatible.");
+
 	if (!option_origin)
 		option_origin = "origin";
 
@@ -597,6 +605,11 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 		opts.src_index = &the_index;
 		opts.dst_index = &the_index;
 
+		if (option_narrow_path) {
+			opts.new_narrow_path = 1;
+			opts.narrow_spec = option_narrow_path;
+		}
+
 		tree = parse_tree_indirect(remote_head->old_sha1);
 		parse_tree(tree);
 		init_tree_desc(&t, tree->buffer, tree->size);
diff --git a/t/t5703-clone-narrow.sh b/t/t5703-clone-narrow.sh
new file mode 100755
index 0000000..3ab9eaf
--- /dev/null
+++ b/t/t5703-clone-narrow.sh
@@ -0,0 +1,39 @@
+#!/bin/sh
+
+test_description='narrow clone'
+
+. ./test-lib.sh
+
+test_expect_success setup '
+	rm -fr .git &&
+	test_create_repo src &&
+	(
+		cd src
+		mkdir -p work/sub/dir
+		touch untracked tracked modified added
+		touch work/untracked work/tracked work/modified work/added
+		git add tracked work/tracked
+		git add modified work/modified
+		git commit -m initial
+	)
+
+'
+
+test_expect_success 'narrow clone incompatible with --bare' '
+	rm -fr dst &&
+	test_must_fail git clone --path=work --bare src dst
+'
+
+test_expect_success 'narrow clone incompatible with --no-checkout' '
+	rm -fr dst &&
+	test_must_fail git clone --path=work -n src dst
+'
+
+test_expect_success 'clone with --path' '
+	rm -fr dst &&
+	git clone --path=work src dst &&
+	cd dst &&
+	test -z "$(git ls-files --checkout | grep -v ^work/)"
+'
+
+test_done
-- 
1.6.0.96.g2fad1.dirty

Re: [PATCH 14/16] clone: support narrow checkout with --path option

[Jakub Narebski]

I will comment only on the documentation...

Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:

> +--path=<narrow_spec>::
> +	A narrow checkout of HEAD is performed after clone is complete
> +	with given narrow spec. Please refer to linkgit:git-checkout[1]
> +	for more detail on narrow checkout and narrow spec.

Two questions. First, can you repeat those specifiers, i.e. use
"git clone --path=Documentation/ --path=.gitignore"?

Second, how `--path=<narrow_spec>` interacts with `-n' (no checkout).
Does it even have any sense? Perhaps those options should be mutually
incompatibile (and --bare and this option too)?

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: [PATCH 14/16] clone: support narrow checkout with --path option

[Junio C Hamano]

Jakub Narebski <jnareb@gmail.com> writes:

> Second, how `--path=<narrow_spec>` interacts with `-n' (no checkout).
> Does it even have any sense? Perhaps those options should be mutually
> incompatibile (and --bare and this option too)?

Technically you could mark the index entries with no-checkout bits even
with -n, so that later checkout or branch switching would honor that, but
the branch switching would make paths that do not exist in the remote HEAD
branch but do exist in the switched-to branch appear, which would be
confusing.  So instead of supporting:

	$ git clone -n --path=Documentation/ $there work
        $ cd work
        $ git checkout next

we can support:

	$ git clone -n $there work
        $ cd work
        $ git checkout --path=Documentation/ next

which is much easier to explain and understand.