[PATCH 0/2] Introducing git-run-with-user-path program.

[PATCH 0/2] Introducing git-run-with-user-path program.

[Junio C Hamano]

This is a new series I've mentioned earlier today.

 [PATCH 1/2] Introduce git-run-with-user-path helper program.
 [PATCH 2/2] Add sample ignore logic to git-run-with-user-path command.

The first one adds a path canonicalization helper with path
ignore hooks but no ignore logic implementation (it passes
everything that passes verify_path()).  The second one adds a
sample ignore logic implementation using PCRE.

Although the second one is done primarily as an example and to
start a mailing list discussion, it should be also safe to merge
if you decide to take patch 1, because the logic is used only by
git-run-with-user-path which is a new program. no Porcelain uses
right now.

[PATCH 0/2] Introducing git-run-with-user-path program.

[Junio C Hamano]

I've polished the driver part a bit more, so that it can do
something similar to "git-ls-files --others" can do, but with
the (to-be-defined) "ignore rules Porcelain layers would agree
upon". 

 [PATCH 1/2] Introduce git-run-with-user-path helper program.
 [PATCH 2/2] Add sample ignore logic to git-run-with-user-path command.

The first one adds a path canonicalization helper with path
ignore hooks but no ignore logic implementation (it passes
everything that passes verify_path()).  The second one adds a
sample ignore logic implementation using PCRE.

Although the second one is done primarily as an example and to
start a mailing list discussion, it should be also safe to merge
if you decide to take patch 1, because the logic is used only by
git-run-with-user-path which is a new program. no Porcelain uses
right now.

Re: [PATCH 1/2] Introduce git-run-with-user-path helper program.

[Junio C Hamano]

A new command git-run-with-user-path takes a command and paths
that are filesystem paths (either relative to the cwd or
absolute pathname).  It canonicalizes these paths to be usable
by the core GIT commands, filters using the ignore pattern rule,
chdir(2)'s to the top level of the tree and runs the given
command with these canonicalizd paths as its arguments.

This version contains necessary hooks to implement the ignore
pattern rule, but it does not implement any ignore pattern
rules, waiting for more mailing list discussions.

Signed-off-by: Junio C Hamano <junkio@cox.net>
---

Documentation/git-run-with-user-path.txt |  100 +++++++++++++++
Makefile                                 |    7 -
paths.c                                  |  199 +++++++++++++++++++++++++++++++
paths.h                                  |   14 ++
run-with-user-path.c                     |   87 +++++++++++++
t/README                                 |    1 
t/t7000-git-run-with-user-path-basic.sh  |   87 +++++++++++++
update-cache.c                           |   29 ----
8 files changed, 495 insertions(+), 29 deletions(-)
Documentation/git-run-with-user-path.txt (. --> 100644)
paths.c (. --> 100644)
paths.h (. --> 100644)
run-with-user-path.c (. --> 100644)
t/t7000-git-run-with-user-path-basic.sh (. --> 100755)

--- /dev/null
+++ b/Documentation/git-run-with-user-path.txt
@@ -0,0 +1,100 @@
+git-run-with-user-path(1)
+=========================
+v0.1, May 2005
+
+NAME
+----
+git-run-with-user-path - Run command from the top after canonicalizing paths.
+
+
+SYNOPSIS
+--------
+'git-run-with-user-path' [options] <command> <argument>... '--' <path>...
+'git-run-with-user-path' [options] ( '--echo-z' | '--echo' ) <path>...
+
+DESCRIPTION
+-----------
+In the first form, this command takes a <command>, zero or more
+<argument> and zero or more <path> arguments.  <path> arguments name
+objects on the filesystem, <command> is typically a core GIT command,
+and <argument> are the initial arguments to the <command>.
+
+It first finds the project top directory (the directory that corresponds
+to the top of the tree structure GIT_INDEX_FILE describes).  When the
+environment variable GIT_PROJECT_TOP is set, the value of the variable
+is used.  Then the <path> parameters are canonicalized to be relative to
+the project top.  It then chdir(2)'s to the project top directory and
+runs the given <command>, with <argument> and these canonicalized <path>
+arguments.
+
+This is useful for the Porcelain layer to run core GIT commands from
+subdirectories.  For example, if linux-2.6.git tree is checked out in
+/usr/src/linux, you can do:
+
+    $ cd /usr/src/linux/fs
+    $ ... work in fs directory making changes ...
+    $ git-run-with-user-path git-diff-tree -r HEAD -- ext? ../include/linux
+    $ find ext? ../include/linux ! -type d -print0 |
+      xargs -0 git-run-with-user-path git-update-cache --add -- --
+
+The above is roughly equivalent to:
+
+    $ cd /usr/src/linux
+    $ git-diff-tree -r HEAD fs/ext? include/linux
+    $ find fs/ext? include/linux ! -type d -print0 |
+      xargs git-update-cache --add --
+
+In the second form, this command just canonicalizes and filters paths,
+and writes the resulting paths to the standard output.  '--echo'
+separates each output record with a newline while '--echo-z' uses NUL as
+the record separator.
+
+    $ find ext? ../include/linux ! -type d -print0 |
+      xargs -0 git-run-with-user-path --echo --show-ignore
+
+This example gives you the list of ignored files, one per line.  It is
+roughly equivalent to:
+
+    $ find ext? ../include/linux ! -type d -print0 |
+      xargs -0 -n1 git-run-with-user-path --show-ignore echo --
+
+
+OPTIONS
+-------
+
+--no-ignore::
+
+	By default, the path arguments are filtered with the same ignore
+	rules Porcelain layers use.  With --no-ignore flag, there is no
+	such filtering done.
+
+--show-ignore::
+
+	This causes the filter to be applied in reverse.  Only the paths
+	ignored by the ignore rules are passed to the command (or
+	output).
+
+
+ENVIRONMENT VARIABLES
+---------------------
+
+'GIT_PROJECT_TOP'::
+	If the 'GIT_PROJECT_TOP' environment variable is set then it
+	specifies the directory that corresponds to the top level of the
+	tree structure GIT_INDEX_FILE describes.  When this environment
+	variable is not defined, the closest parent directory that has
+	.git/ subdirectory in it is looked for and used.
+
+
+Author
+------
+Written by Junio C Hamano <junkio@cox.net>
+
+Documentation
+--------------
+Documentation by Junio C Hamano.
+
+GIT
+---
+Part of the link:git.html[git] suite
+
--- a/Makefile
+++ b/Makefile
@@ -28,7 +28,7 @@
 	git-unpack-file git-export git-diff-cache git-convert-cache \
 	git-http-pull git-rpush git-rpull git-rev-list git-mktag \
 	git-diff-helper git-tar-tree git-local-pull git-write-blob \
-	git-get-tar-commit-id
+	git-get-tar-commit-id git-run-with-user-path
 
 all: $(PROG)
 
@@ -46,6 +46,9 @@
 LIB_H += diff.h
 LIB_OBJS += diff.o
 
+LIB_H += paths.h
+LIB_OBJS += paths.o
+
 LIB_OBJS += gitenv.o
 
 LIBS = $(LIB_FILE)
@@ -100,6 +103,7 @@
 git-rpush: rsh.c
 git-rpull: rsh.c pull.c
 git-rev-list: rev-list.c
+git-run-with-user-path: run-with-user-path.c 
 git-mktag: mktag.c
 git-diff-helper: diff-helper.c
 git-tar-tree: tar-tree.c
@@ -117,6 +121,7 @@
 sha1_file.o: $(LIB_H)
 usage.o: $(LIB_H)
 diff.o: $(LIB_H)
+paths.o: $(LIB_H)
 strbuf.o: $(LIB_H)
 gitenv.o: $(LIB_H)
 
--- /dev/null
+++ b/paths.c
@@ -0,0 +1,199 @@
+/*
+ * Copyright (c) 2005 Junio C Hamano
+ */
+#include <string.h>
+#include "cache.h"
+#include "paths.h"
+
+/****************************************************************/
+
+/* Ignore list handling part */
+
+/*
+ * We fundamentally don't like some paths: we don't want
+ * dot or dot-dot anywhere, and in fact, we don't even want
+ * any other dot-files (.git or anything else). They
+ * are hidden, for chist sake.
+ *
+ * Also, we don't want double slashes or slashes at the
+ * end that can make pathnames ambiguous.
+ */
+int verify_path(const char *path)
+{
+	char c;
+
+	goto inside;
+	for (;;) {
+		if (!c)
+			return 1;
+		if (c == '/') {
+inside:
+			c = *path++;
+			if (c != '/' && c != '.' && c != '\0')
+				continue;
+			return 0;
+		}
+		c = *path++;
+	}
+}
+
+static int initialize_ignore_list(void)
+{
+	/* Put the Porcelain layer ignore logic initialization here.
+	 * Return non-zero after issuing appropriate error message
+	 * if initialization fails.
+	 */
+	return 0;
+}
+
+int path_ignored(const char *path)
+{
+	if (!verify_path(path))
+		return 1;
+
+	/* Put the Porcelain layer ignore logic here.
+	 * Return non-zero if path is to be ignored.
+	 */
+	return 0;
+}
+
+
+/****************************************************************/
+
+/* Path canonicalization part */
+
+char *git_project_top = NULL;
+static char git_cwd[PATH_MAX];
+
+static int find_project_top(void)
+{
+	char path[PATH_MAX];
+	int dir_length;
+
+	if (!getcwd(git_cwd, sizeof(git_cwd)))
+		return error("cannot get cwd to find GIT_PROJECT_TOP");
+
+	git_project_top = gitenv("GIT_PROJECT_TOP");
+	if (git_project_top)
+		return 0;
+
+	strcpy(path, git_cwd);
+	while (path[0] && strcmp(path, "/") && !git_project_top) {
+		char *cp;
+		struct stat st;
+		dir_length = strlen(path);
+		path[dir_length] = '/';
+
+		strcpy(path + dir_length + 1, ".git");
+		if (stat(path, &st) < 0) {
+			if (errno != ENOENT)
+				return error("%s: %s", path, strerror(errno));
+			/* notfound */
+		}
+		else if (S_ISDIR(st.st_mode)) {
+			path[dir_length] = 0;
+			git_project_top = strdup(path);
+			break;
+		}
+		else
+			return error("%s: not a directory", path);
+		path[dir_length] = 0;
+		cp = strrchr(path, '/');
+		if (cp)
+			*cp = 0;
+	}
+	if (!git_project_top)
+		return error("cannot find GIT_PROJECT_TOP");
+
+	return 0;
+}
+
+char *canon_path(const char *path)
+{
+	/* path is either absolute path from root fs or
+	 * relative to the git_cwd.  What is the relative path
+	 * for that thing, viewed from GIT_PROJECT_TOP?
+	 */
+	char *cp, *op, *endp, *result = NULL;
+	char *work = xmalloc(strlen(git_cwd) + strlen(path) + 2);
+	int pfxlen = strlen(git_project_top);
+
+	if (path[0] == '/')
+		strcpy(work, path);
+	else
+		sprintf(work, "%s/%s", git_cwd, path);
+	/* We will copy to *op starting from *cp while removing
+	 * nonsense.  Initially op and cp are both set to one
+	 * past the root-level '/'.
+	 */
+	op = cp = work + 1;
+	endp = cp + strlen(cp);
+	while (cp < endp) {
+		char *ep = strchr(cp, '/');
+		if (!ep)
+			ep = endp; /* at terminating NUL */
+		/* Now look at what is between cp and ep. */
+		if (ep == cp) {
+			/* Remove double slashes.
+			 * "/xxx//foo" ==> "/xxx//foo"
+			 *    cp^              cp^
+			 */
+			cp++;
+			continue;
+		}
+		if (*cp == '.') {
+			/* dot something.  What is it? */
+			if (cp[1] == 0 || cp[1] == '/') {
+				/* Remove trailing dot.
+				 * "/xxx/." ==> "/xxx/."
+				 *     cp^           cp^
+				 * "/xxx/./foo" ==> "/xxx/./foo"
+				 *     cp^               cp^
+				 */
+				cp = ep;
+				continue;
+			}
+			if (cp[1] == '.' && (cp[2] == 0 || cp[2] == '/')) {
+				/* Uplevel.
+				 * "/xxx/../foo" ==> "/xxx/../foo"
+				 *     cp^                  cp^
+				 * while backspacing "xxx" in the op
+				 */
+				cp = cp + 3;
+				op -= 2;
+				if (op < work)
+					op = work + 1;
+				while (*op != '/' && work < op)
+					op--;
+				op++;
+				continue;
+			}
+		}
+		/* otherwise there is no funnies */
+		while (cp <= ep && *cp)
+			*op++ = *cp++;
+	}
+	*op = 0;
+	if (op[-1] == '/' && op != work)
+		op[-1] = 0;
+
+	if (!strncmp(git_project_top, work, pfxlen) &&
+	    (work[pfxlen] == '/' || work[pfxlen] == 0))
+		result = strdup(work + pfxlen + 1);
+	/* otherwise, path is outside of git-project-top and we ignore it. */
+
+	free(work);
+	return result;
+}
+
+/****************************************************************/
+
+int setup_paths(void)
+{
+	if (find_project_top())
+		return -1;
+	if (initialize_ignore_list())
+		return -1;
+	return 0;
+}
+
--- /dev/null
+++ b/paths.h
@@ -0,0 +1,14 @@
+/*
+ * Copyright (c) 2005 Junio C Hamano
+ */
+#ifndef _PATHS_H_
+#define _PATHS_H_
+
+int setup_paths(void);
+extern char *git_project_top;
+
+char *canon_path(const char *);
+int path_ignored(const char *);
+int verify_path(const char *);
+
+#endif
--- /dev/null
+++ b/run-with-user-path.c
@@ -0,0 +1,87 @@
+/*
+ * Copyright (c) 2005 Junio C Hamano
+ */
+#include <unistd.h>
+#include "cache.h"
+#include "paths.h"
+
+static int no_ignore = 0, show_ignore = 0, echo = 0, line_termination = '\n';
+
+static const char *usage_rwup = 
+"git-run-with-user-path [ --no-ignore ] [ --show-ignore ] "
+"( --echo-z | --echo | <command> <argument>... -- <path>... )";
+
+static int prepare_path_args(char **exec_param, char **path)
+{
+	int i, cnt;
+	char *canon;
+
+	for (i = cnt = 0; path[i]; i++) {
+		canon = canon_path(path[i]);
+		if (no_ignore ||
+		    !!path_ignored(canon) == !!show_ignore)
+			exec_param[cnt++] = canon;
+	}
+	return cnt;
+}
+
+int main(int ac, char **av)
+{
+	char **exec_param;
+	int i, command_end, cnt_path;
+
+	if (setup_paths())
+		exit(1);
+
+	while (1 < ac && av[1][0] == '-') {
+		if (!strcmp(av[1], "--no-ignore"))
+			no_ignore = 1;
+		else if (!strcmp(av[1], "--show-ignore"))
+			show_ignore = 1;
+		else if (!strcmp(av[1], "--echo"))
+			echo = 1;
+		else if (!strcmp(av[1], "--echo-z")) {
+			echo = 1;
+			line_termination = 0;
+		}
+		else
+			break;
+		ac--; av++;
+	}
+
+	exec_param = xcalloc(ac, sizeof(char *));
+
+	if (echo) {
+		cnt_path = prepare_path_args(exec_param, av + 1);
+		for (i = 0; exec_param[i]; i++)
+			printf("%s%c", exec_param[i], line_termination);
+		exit(0);
+	}
+
+	for (i = 1; i < ac; i++)
+		if (!strcmp(av[i], "--"))
+			break;
+	if (ac <= i)
+		die(usage_rwup); /* no -- to start path */
+
+	command_end = i; /* pointing at -- */
+	/* command command arg1 arg2 ... path1 path2 ... NULL */
+	for (i = 1; i < command_end; i++)
+		exec_param[i - 1] = av[i];
+
+	/* We need to special case -- (end of command) followed
+	 * immediately by -- (beginning of paths); otherwise
+	 * "git-run-with-user-path git-update-cache --add -- -- foo"
+	 * would try canonicalize and filter path arguments starting
+	 * from -- (beginning of paths), which is not what we want.
+	 */
+	if (!strcmp(av[command_end + 1], "--")) {
+		exec_param[command_end-1] = av[command_end + 1];
+		command_end++;
+	}
+	cnt_path = prepare_path_args(exec_param + command_end - 1,
+				     av + command_end + 1);
+	chdir(git_project_top);
+	execvp(exec_param[0], exec_param);
+	exit(1);
+}
--- a/t/README
+++ b/t/README
@@ -73,6 +73,7 @@
 	4 - the diff commands
 	5 - the pull and exporting commands
 	6 - the revision tree commands (even e.g. merge-base)
+	7 - the non-core commands and helpers
 
 Second digit tells the particular command we are testing.
 
--- /dev/null
+++ b/t/t7000-git-run-with-user-path-basic.sh
@@ -0,0 +1,87 @@
+#!/bin/sh
+#
+# Copyright (c) 2005, Junio C Hamano
+#
+
+test_description='git-run-with-user-path basic test.
+
+The command is used to help running core GIT commands that always
+expect to be run from the top level directory (i.e. the directory
+that corresponds to the top of tree GIT_INDEX_FILE describes).
+'
+
+. ./test-lib.sh
+
+LF='
+'
+HERE=$(pwd)
+
+test_expect_success \
+setup '
+mkdir path0 path1 path1/path2
+for p in path0/file0 path1/file1 path1/path2/file2
+do
+    echo hello >$p
+    git-update-cache --add -- $p
+done
+'
+
+test_expect_success \
+'finding paths from a subdirectory' '
+    case "$(cd path0 &&
+            git-run-with-user-path --no-ignore cat -- \
+	    file0 ../path1/path2/file2)" in
+    "hello${LF}hello") : ;;
+    *) (exit 1) ;;
+    esac
+'
+
+test_expect_success \
+'feeding find output via xargs from a subdirectory' '
+    case "$(cd path0 &&
+	    find . ../path1 -type f -print0 |
+	    xargs -r -0 git-run-with-user-path --no-ignore cat --)" in
+    "hello${LF}hello${LF}hello") : ;;
+    *) (exit 1) ;;
+    esac
+'
+
+cd $HERE
+test_expect_success \
+'handling special case -- at the end of command' '
+    case "$(cd path0 &&
+            git-run-with-user-path --no-ignore echo -- \
+	    -- ../path1/file1)" in
+    "-- path1/file1") : ;;
+    *) (exit 1) ;;
+    esac	    
+'
+
+cd $HERE
+test_expect_success \
+'the --echo option' '
+    case "$(cd path0 &&
+            git-run-with-user-path --no-ignore --echo ../path1/file1)" in
+    "path1/file1") : ;;
+    *) (exit 1) ;;
+    esac	    
+'
+
+cd $HERE
+mv .git .svn
+GIT_DIR=$(pwd)/.svn
+GIT_PROJECT_TOP=$(pwd)
+export GIT_DIR GIT_PROJECT_TOP
+
+test_expect_success \
+'feeding find output via xargs from a subdirectory (with GIT_PROJECT_TOP)' '
+    case "$(cd path0 &&
+            find . ../path1 -type f -print0 |
+	    xargs -r -0 git-run-with-user-path --no-ignore cat --)" in
+    "hello${LF}hello${LF}hello") : ;;
+    *) (exit 1) ;;
+    esac
+    cd ..
+'
+
+test_done
--- a/update-cache.c
+++ b/update-cache.c
@@ -5,6 +5,7 @@
  */
 #include <signal.h>
 #include "cache.h"
+#include "paths.h"
 
 /*
  * Default to not allowing changes to the list of files. The
@@ -257,34 +258,6 @@
 	return has_errors;
 }
 
-/*
- * We fundamentally don't like some paths: we don't want
- * dot or dot-dot anywhere, and in fact, we don't even want
- * any other dot-files (.git or anything else). They
- * are hidden, for chist sake.
- *
- * Also, we don't want double slashes or slashes at the
- * end that can make pathnames ambiguous.
- */
-static int verify_path(char *path)
-{
-	char c;
-
-	goto inside;
-	for (;;) {
-		if (!c)
-			return 1;
-		if (c == '/') {
-inside:
-			c = *path++;
-			if (c != '/' && c != '.' && c != '\0')
-				continue;
-			return 0;
-		}
-		c = *path++;
-	}
-}
-
 static int add_cacheinfo(char *arg1, char *arg2, char *arg3)
 {
 	int size, len, option;
------------------------------------------------

Re: [PATCH 1/2] Introduce git-run-with-user-path helper program.

[Junio C Hamano]

*Blush*.  I sent out a bad copy for this patch.

In run-with-user-path.c, in main():

+	/* We need to special case -- (end of command) followed
+	 * immediately by -- (beginning of paths); otherwise
+	 * "git-run-with-user-path git-update-cache --add -- -- foo"
+	 * would try canonicalize and filter path arguments starting
+	 * from -- (beginning of paths), which is not what we want.
+	 */
+	if (!strcmp(av[command_end + 1], "--")) {
+		exec_param[command_end-1] = av[command_end + 1];
+		command_end++;
+	}

The above if() statement should be:

    if (command_end + 1 < ac && !strcmp(av[command_end + 1], "--")) {

Otherwise "./git-run-with-user-path echo --" would segfault.
Sorry about the noise.

[PATCH 2/2] Add sample ignore logic to git-run-with-user-path command.

[Junio C Hamano]

This adds a sample ignore file logic to git-run-with-user-path
command.  This is primarily to serve as an example for plugging
ignore file logic to the previously introduced framework, and to
spur mailing list discussions on what the final ignore file
logic should be, and where the information should come from.

Signed-off-by: Junio C Hamano <junkio@cox.net>
---

Documentation/git-run-with-user-path.txt |   32 ++++++++
Makefile                                 |    6 +
paths.c                                  |  117 +++++++++++++++++++++++++++++--
t/t7001-git-run-with-user-path-ignore.sh |   67 +++++++++++++++++
4 files changed, 215 insertions(+), 7 deletions(-)
t/t7001-git-run-with-user-path-ignore.sh (. --> 100755)

--- a/Documentation/git-run-with-user-path.txt
+++ b/Documentation/git-run-with-user-path.txt
@@ -75,6 +75,38 @@
 	output).
 
 
+IGNORE FILES
+------------
+
+This command currently uses a pcre based implementation to express
+ignore patterns.  The purpose of this implementation is to primarily
+serve as an example and to start GIT mailing list discussions, and by no
+means is cast in stone.  This section describes what this sample
+implementation does.
+
+The information used to define which paths to ignore is read from two
+files.  Both files use the same syntax.
+
+First, $CIT_DIR/ignore is read.  Then, the file whose path (relative to
+the project top) recorded in $GIT_DIR/info/ignore-file is read next.
+The latter file is expected to be revision controlled with GIT.
+
+These two files should record one ignore record per line.  A line that
+is empty, and a line that starts with a '#' are ignored and used as
+comments.
+
+Each ignore record is a pcre regular expression, optionally prefixed
+with a '!'.  To determine if a path is to be ignored, the path is
+matched against each ignore record in the order they appear in the
+ignore file.  If the ignore record matches the path, and it does not
+have the optional '!', then the path is ignored.  Otherwise, the path is
+not ignored.  In either case, the rest of ignore records are not used
+after the first match.  This means (1) an earlier entry in an ignore
+file has precedence over later ones, and (2) the entries in
+$GIT_DIR/ignore file have precedence over the ones in the file named
+by $GIT_DIR/info/ignore-file.
+
+
 ENVIRONMENT VARIABLES
 ---------------------
 
--- a/Makefile
+++ b/Makefile
@@ -54,6 +54,12 @@
 LIBS = $(LIB_FILE)
 LIBS += -lz
 
+IGNORE_USING_PCRE=1
+
+ifdef IGNORE_USING_PCRE
+  LIBS += -lpcreposix
+endif
+
 ifdef MOZILLA_SHA1
   SHA1_HEADER="mozilla-sha1/sha1.h"
   LIB_OBJS += mozilla-sha1/sha1.o
--- a/paths.c
+++ b/paths.c
@@ -2,6 +2,7 @@
  * Copyright (c) 2005 Junio C Hamano
  */
 #include <string.h>
+#include <pcreposix.h>
 #include "cache.h"
 #include "paths.h"
 
@@ -37,23 +38,125 @@
 	}
 }
 
+static struct ignore_entry {
+	int negate;
+	regex_t regexp;
+} **ignore_list;
+static int ignore_nr;
+static int ignore_alloc;
+
+static void add_ignore(const char *buf)
+{
+	struct ignore_entry *ie = xmalloc(sizeof(*ie));
+	if (buf[0] == '!') {
+		ie->negate = 1;
+		buf++;
+	}
+	else
+		ie->negate = 0;
+
+	if (regcomp(&(ie->regexp), buf, 0)) {
+		fprintf(stderr, "bad regexp <%s>\n", buf);
+		free(ie);
+		return;
+	}
+	if (ignore_alloc <= ignore_nr) {
+		ignore_alloc = alloc_nr(ignore_alloc);
+		ignore_list = xrealloc(ignore_list,
+				       ignore_alloc * sizeof(ie));
+	}
+	ignore_list[ignore_nr++] = ie;
+}
+
+static void read_ignore_list(const char *path)
+{
+	FILE *in;
+	char buf[1024];
+	in = fopen(path, "r");
+	if (!in)
+		return;
+	while (fgets(buf, sizeof(buf), in) != NULL) {
+		int l = strlen(buf);
+		/* An empty line and a line that starts with # is comment. */
+		if (buf[0] != '#' && buf[0] != '\n' && buf[l-1] == '\n') {
+			buf[l-1] = 0;
+			add_ignore(buf);
+		}
+	}
+	fclose(in);
+}
+
+static void read_ignore_list_from_file(const char *path)
+{
+	char filename[PATH_MAX];
+	int len;
+	FILE *in;
+
+	in = fopen(path, "r");
+	if (!in)
+		return;
+	strcpy(filename, git_project_top);
+	len = strlen(filename);
+	filename[len++] = '/';
+	if (fgets(filename + len, sizeof(filename) - len, in) == NULL) {
+		fclose(in);
+		return;
+	}
+	fclose(in);
+	len = strlen(filename);
+	if (filename[len-1] != '\n')
+		return;
+	filename[len-1] = 0;
+	read_ignore_list(filename);
+}
+
 static int initialize_ignore_list(void)
 {
-	/* Put the Porcelain layer ignore logic initialization here.
-	 * Return non-zero after issuing appropriate error message
-	 * if initialization fails.
-	 */
+	char *git_dir = gitenv("GIT_DIR");
+	char path[PATH_MAX];
+	int git_dir_len;
+
+	if (! git_dir)
+		sprintf(path, "%s/.git", git_project_top);
+	else
+		strcpy(path, git_dir);
+	git_dir_len = strlen(path);
+	path[git_dir_len++] = '/';
+
+	/* read private list first, and then shared list. */
+	strcpy(path + git_dir_len, "ignore");
+	read_ignore_list(path);
+
+	strcpy(path + git_dir_len, "info/ignore-file");
+	read_ignore_list_from_file(path);
+
 	return 0;
 }
 
 int path_ignored(const char *path)
 {
+	int i;
+
 	if (!verify_path(path))
 		return 1;
 
-	/* Put the Porcelain layer ignore logic here.
-	 * Return non-zero if path is to be ignored.
-	 */
+	for (i = 0; i < ignore_nr; i++) {
+		int status;
+		regmatch_t pmatch[10];
+		char errbuf[1024];
+
+		status = regexec(&(ignore_list[i]->regexp), path,
+				 sizeof(pmatch)/sizeof(pmatch[0]),
+				 pmatch, 0);
+		if (!status)
+			return !ignore_list[i]->negate;
+		if (status == REG_NOMATCH)
+			continue;
+
+		regerror(status, &(ignore_list[i]->regexp), errbuf,
+			 sizeof(errbuf));
+		fprintf(stderr, "pcre regexp execution error <%s>\n", errbuf);
+	}
 	return 0;
 }
 
--- /dev/null
+++ b/t/t7001-git-run-with-user-path-ignore.sh
@@ -0,0 +1,67 @@
+#!/bin/sh
+#
+# Copyright (c) 2005, Junio C Hamano
+#
+
+test_description='git-run-with-user-path basic test (part #2).
+
+The command is used to help running core GIT commands that always
+expect to be run from the top level directory (i.e. the directory
+that corresponds to the top of tree GIT_INDEX_FILE describes).
+
+It knows how to handle ignore files convention used by the Porcelain
+layer implementation.
+'
+
+. ./test-lib.sh
+
+LF='
+'
+HERE=$(pwd)
+
+test_expect_success \
+setup '
+echo ".*1\$" >.git/ignore &&
+echo ".*0\$" >dontdiff &&
+mkdir .git/info &&
+echo "dontdiff" >.git/info/ignore-file &&
+mkdir path0 path1 path1/path2 &&
+for p in path0/file0 path1/file1 path1/path2/file2
+do
+    echo hello >$p || exit 1
+done
+'
+
+cd $HERE
+test_expect_success \
+'finding paths from a subdirectory' '
+    case "$(cd path0 &&
+            git-run-with-user-path echo -- \
+	    file0 ../path1/path2/file2)" in
+    "path1/path2/file2") : ;;
+    *) (exit 1) ;;
+    esac
+'
+
+cd $HERE
+test_expect_success \
+'feeding find output via xargs from a subdirectory' '
+    case "$(cd path0 &&
+	    find . ../path1 -type f -print0 |
+	    xargs -r -0 git-run-with-user-path ls -1 --)" in
+    "path1/path2/file2") : ;;
+    *) (exit 1) ;;
+    esac
+'
+
+cd $HERE
+test_expect_success \
+'using !negate pattern' '
+    echo "!path0/file0$" >>.git/ignore &&
+    case "$(git-run-with-user-path ls -1 -- path0/* path1/file1)" in
+    "path0/file0") : ;;
+    *) (exit 1) ;;
+    esac
+'
+
+test_done
------------------------------------------------

[PATCH 1/2] Introduce git-run-with-user-path helper program.

[Junio C Hamano]

Introduce git-run-with-user-path helper program.

A new command git-run-with-user-path takes a command and paths
that are filesystem paths (either relative to the cwd or
absolute pathname).  It canonicalizes these paths to be usable
by the core GIT commands, filters using the ignore pattern rule,
chdir(2)'s to the top level of the tree and runs the given
command with these canonicalizd paths as its arguments.

This version contains necessary hooks to implement the ignore
pattern rule, but it does not implement any ignore pattern
rules, waiting for more mailing list discussions.

Signed-off-by: Junio C Hamano <junkio@cox.net>
---

Documentation/git-run-with-user-path.txt |   79 ++++++++++++
Makefile                                 |    7 -
paths.c                                  |  199 +++++++++++++++++++++++++++++++
paths.h                                  |   14 ++
run-with-user-path.c                     |   61 +++++++++
t/README                                 |    1 
t/t7000-git-run-with-user-path-basic.sh  |   66 ++++++++++
update-cache.c                           |   29 ----
8 files changed, 427 insertions(+), 29 deletions(-)
Documentation/git-run-with-user-path.txt (. --> 100644)
paths.c (. --> 100644)
paths.h (. --> 100644)
run-with-user-path.c (. --> 100644)
t/t7000-git-run-with-user-path-basic.sh (. --> 100755)

--- a/Documentation/git-run-with-user-path.txt
+++ b/Documentation/git-run-with-user-path.txt
@@ -0,0 +1,79 @@
+git-run-with-user-path(1)
+=========================
+v0.1, May 2005
+
+NAME
+----
+git-run-with-user-path - Run command from the top after canonicalizing paths.
+
+
+SYNOPSIS
+--------
+'git-run-with-user-path' [options] <command> <argument>... '--' <path>...
+
+DESCRIPTION
+-----------
+This command takes a <command>, zero or more <argument> and zero
+or more <path> arguments.  <path> arguments name objects on the
+filesystem, <command> is typically a core GIT command, and
+<argument> are the initial arguments to the <command>.
+
+It first finds the project top directory (the directory that corresponds
+to the top of the tree structure GIT_INDEX_FILE describes).  When the
+environment variable GIT_PROJECT_TOP is set, the value of the variable
+is used.  Then the <path> parameters are canonicalized to be relative to
+the project top.  It then chdir(2)'s to the project top directory and
+runs the given <command>, with <argument> and these canonicalized <path>
+arguments.
+
+This is useful for the Porcelain layer to run core GIT commands from
+subdirectories.  For example, if linux-2.6.git tree is checked out in
+/usr/src/linux, you can do:
+
+    $ cd /usr/src/linux/fs
+    $ ... work in fs directory making changes ...
+    $ git-run-with-user-path git-diff-tree -r HEAD -- ext? ../include/linux
+    $ find ext? ../include/linux ! -type d -print0 |
+      xargs -0 git-run-with-user-path git-update-cache --add -- --
+
+The above is roughly equivalent to:
+
+    $ cd /usr/src/linux
+    $ git-diff-tree -r HEAD fs/ext? include/linux
+    $ find fs/ext? include/linux ! -type d -print0 |
+      xargs git-update-cache --add --
+
+
+OPTIONS
+-------
+--no-ignore::
+
+	By default, the path arguments are filtered with the
+	same ignore rules Porcelain layers use.  With
+	--no-ignore flag, there is no such filtering done.
+
+
+ENVIRONMENT VARIABLES
+---------------------
+
+'GIT_PROJECT_TOP'::
+	If the 'GIT_PROJECT_TOP' environment variable is set
+	then it specifies the directory that corresponds to the
+	top level of the tree structure GIT_INDEX_FILE describes.
+	When this environment variable is not defined, the
+	closest parent directory that has .git/ subdirectory in
+	it is looked for and used.
+
+
+Author
+------
+Written by Junio C Hamano <junkio@cox.net>
+
+Documentation
+--------------
+Documentation by Junio C Hamano.
+
+GIT
+---
+Part of the link:git.html[git] suite
+
--- a/Makefile
+++ b/Makefile
@@ -28,7 +28,7 @@
 	git-unpack-file git-export git-diff-cache git-convert-cache \
 	git-http-pull git-rpush git-rpull git-rev-list git-mktag \
 	git-diff-helper git-tar-tree git-local-pull git-write-blob \
-	git-get-tar-commit-id
+	git-get-tar-commit-id git-run-with-user-path
 
 all: $(PROG)
 
@@ -46,6 +46,9 @@
 LIB_H += diff.h
 LIB_OBJS += diff.o
 
+LIB_H += paths.h
+LIB_OBJS += paths.o
+
 LIB_OBJS += gitenv.o
 
 LIBS = $(LIB_FILE)
@@ -100,6 +103,7 @@
 git-rpush: rsh.c
 git-rpull: rsh.c pull.c
 git-rev-list: rev-list.c
+git-run-with-user-path: run-with-user-path.c 
 git-mktag: mktag.c
 git-diff-helper: diff-helper.c
 git-tar-tree: tar-tree.c
@@ -117,6 +121,7 @@
 sha1_file.o: $(LIB_H)
 usage.o: $(LIB_H)
 diff.o: $(LIB_H)
+paths.o: $(LIB_H)
 strbuf.o: $(LIB_H)
 gitenv.o: $(LIB_H)
 
--- a/paths.c
+++ b/paths.c
@@ -0,0 +1,199 @@
+/*
+ * Copyright (c) 2005 Junio C Hamano
+ */
+#include <string.h>
+#include "cache.h"
+#include "paths.h"
+
+/****************************************************************/
+
+/* Ignore list handling part */
+
+/*
+ * We fundamentally don't like some paths: we don't want
+ * dot or dot-dot anywhere, and in fact, we don't even want
+ * any other dot-files (.git or anything else). They
+ * are hidden, for chist sake.
+ *
+ * Also, we don't want double slashes or slashes at the
+ * end that can make pathnames ambiguous.
+ */
+int verify_path(const char *path)
+{
+	char c;
+
+	goto inside;
+	for (;;) {
+		if (!c)
+			return 1;
+		if (c == '/') {
+inside:
+			c = *path++;
+			if (c != '/' && c != '.' && c != '\0')
+				continue;
+			return 0;
+		}
+		c = *path++;
+	}
+}
+
+static int initialize_ignore_list(void)
+{
+	/* Put the Porcelain layer ignore logic initialization here.
+	 * Return non-zero after issuing appropriate error message
+	 * if initialization fails.
+	 */
+	return 0;
+}
+
+int path_ignored(const char *path)
+{
+	if (!verify_path(path))
+		return 1;
+
+	/* Put the Porcelain layer ignore logic here.
+	 * Return non-zero if path is to be ignored.
+	 */
+	return 0;
+}
+
+
+/****************************************************************/
+
+/* Path canonicalization part */
+
+char *git_project_top = NULL;
+static char git_cwd[PATH_MAX];
+
+static int find_project_top(void)
+{
+	char path[PATH_MAX];
+	int dir_length;
+
+	if (!getcwd(git_cwd, sizeof(git_cwd)))
+		return error("cannot get cwd to find GIT_PROJECT_TOP");
+
+	git_project_top = gitenv("GIT_PROJECT_TOP");
+	if (git_project_top)
+		return 0;
+
+	strcpy(path, git_cwd);
+	while (path[0] && strcmp(path, "/") && !git_project_top) {
+		char *cp;
+		struct stat st;
+		dir_length = strlen(path);
+		path[dir_length] = '/';
+
+		strcpy(path + dir_length + 1, ".git");
+		if (stat(path, &st) < 0) {
+			if (errno != ENOENT)
+				return error("%s: %s", path, strerror(errno));
+			/* notfound */
+		}
+		else if (S_ISDIR(st.st_mode)) {
+			path[dir_length] = 0;
+			git_project_top = strdup(path);
+			break;
+		}
+		else
+			return error("%s: not a directory", path);
+		path[dir_length] = 0;
+		cp = strrchr(path, '/');
+		if (cp)
+			*cp = 0;
+	}
+	if (!git_project_top)
+		return error("cannot find GIT_PROJECT_TOP");
+
+	return 0;
+}
+
+char *canon_path(const char *path)
+{
+	/* path is either absolute path from root fs or
+	 * relative to the git_cwd.  What is the relative path
+	 * for that thing, viewed from GIT_PROJECT_TOP?
+	 */
+	char *cp, *op, *endp, *result = NULL;
+	char *work = xmalloc(strlen(git_cwd) + strlen(path) + 2);
+	int pfxlen = strlen(git_project_top);
+
+	if (path[0] == '/')
+		strcpy(work, path);
+	else
+		sprintf(work, "%s/%s", git_cwd, path);
+	/* We will copy to *op starting from *cp while removing
+	 * nonsense.  Initially op and cp are both set to one
+	 * past the root-level '/'.
+	 */
+	op = cp = work + 1;
+	endp = cp + strlen(cp);
+	while (cp < endp) {
+		char *ep = strchr(cp, '/');
+		if (!ep)
+			ep = endp; /* at terminating NUL */
+		/* Now look at what is between cp and ep. */
+		if (ep == cp) {
+			/* Remove double slashes.
+			 * "/xxx//foo" ==> "/xxx//foo"
+			 *    cp^              cp^
+			 */
+			cp++;
+			continue;
+		}
+		if (*cp == '.') {
+			/* dot something.  What is it? */
+			if (cp[1] == 0 || cp[1] == '/') {
+				/* Remove trailing dot.
+				 * "/xxx/." ==> "/xxx/."
+				 *     cp^           cp^
+				 * "/xxx/./foo" ==> "/xxx/./foo"
+				 *     cp^               cp^
+				 */
+				cp = ep;
+				continue;
+			}
+			if (cp[1] == '.' && (cp[2] == 0 || cp[2] == '/')) {
+				/* Uplevel.
+				 * "/xxx/../foo" ==> "/xxx/../foo"
+				 *     cp^                  cp^
+				 * while backspacing "xxx" in the op
+				 */
+				cp = cp + 3;
+				op -= 2;
+				if (op < work)
+					op = work + 1;
+				while (*op != '/' && work < op)
+					op--;
+				op++;
+				continue;
+			}
+		}
+		/* otherwise there is no funnies */
+		while (cp <= ep && *cp)
+			*op++ = *cp++;
+	}
+	*op = 0;
+	if (op[-1] == '/' && op != work)
+		op[-1] = 0;
+
+	if (!strncmp(git_project_top, work, pfxlen) &&
+	    (work[pfxlen] == '/' || work[pfxlen] == 0))
+		result = strdup(work + pfxlen + 1);
+	/* otherwise, path is outside of git-project-top and we ignore it. */
+
+	free(work);
+	return result;
+}
+
+/****************************************************************/
+
+int setup_paths(void)
+{
+	if (find_project_top())
+		return -1;
+	if (initialize_ignore_list())
+		return -1;
+	return 0;
+}
+
--- a/paths.h
+++ b/paths.h
@@ -0,0 +1,14 @@
+/*
+ * Copyright (c) 2005 Junio C Hamano
+ */
+#ifndef _PATHS_H_
+#define _PATHS_H_
+
+int setup_paths(void);
+extern char *git_project_top;
+
+char *canon_path(const char *);
+int path_ignored(const char *);
+int verify_path(const char *);
+
+#endif
--- a/run-with-user-path.c
+++ b/run-with-user-path.c
@@ -0,0 +1,61 @@
+/*
+ * Copyright (c) 2005 Junio C Hamano
+ */
+#include <unistd.h>
+#include "cache.h"
+#include "paths.h"
+
+static int no_ignore = 0;
+
+static const char *usage_rwup = 
+"git-run-with-user-path [ --no-ignore ] <command> <argument>... -- <path>...";
+
+static int prepare_path_args(char **exec_param, char **path)
+{
+	int i, cnt;
+	char *canon;
+
+	for (i = cnt = 0; path[i]; i++) {
+		canon = canon_path(path[i]);
+		if (no_ignore || !path_ignored(canon))
+			exec_param[cnt++] = canon;
+	}
+	return cnt;
+}
+
+int main(int ac, char **av)
+{
+	char **exec_param;
+	int i, command_end, cnt_path;
+
+	if (setup_paths())
+		exit(1);
+
+	while (1 < ac && av[1][0] == '-') {
+		if (!strcmp(av[1], "--no-ignore"))
+			no_ignore = 1;
+		else
+			break;
+		ac--; av++;
+	}
+	for (i = 1; i < ac; i++)
+		if (!strcmp(av[i], "--"))
+			break;
+	if (ac <= i)
+		die(usage_rwup); /* no -- to start path */
+
+	command_end = i; /* pointing at -- */
+
+	/* command command arg1 arg2 ... path1 path2 ... NULL */
+	exec_param = xcalloc(ac, sizeof(char *));
+	exec_param[ac - 1] = 0;
+	for (i = 1; i < command_end; i++)
+		exec_param[i - 1] = av[i];
+	cnt_path = prepare_path_args(exec_param + command_end - 1,
+				     av + command_end + 1);
+
+	chdir(git_project_top);
+	execvp(exec_param[0], exec_param);
+
+	exit(0);
+}
--- a/t/README
+++ b/t/README
@@ -73,6 +73,7 @@
 	4 - the diff commands
 	5 - the pull and exporting commands
 	6 - the revision tree commands (even e.g. merge-base)
+	7 - the non-core commands and helpers
 
 Second digit tells the particular command we are testing.
 
--- a/t/t7000-git-run-with-user-path-basic.sh
+++ b/t/t7000-git-run-with-user-path-basic.sh
@@ -0,0 +1,66 @@
+#!/bin/sh
+#
+# Copyright (c) 2005, Junio C Hamano
+#
+
+test_description='git-run-with-user-path basic test.
+
+The command is used to help running core GIT commands that always
+expect to be run from the top level directory (i.e. the directory
+that corresponds to the top of tree GIT_INDEX_FILE describes).
+'
+
+. ./test-lib.sh
+
+LF='
+'
+HERE=$(pwd)
+
+test_expect_success \
+setup '
+mkdir path0 path1 path1/path2
+for p in path0/file0 path1/file1 path1/path2/file2
+do
+    echo hello >$p
+    git-update-cache --add -- $p
+done
+'
+
+test_expect_success \
+'finding paths from a subdirectory' '
+    case "$(cd path0 &&
+            git-run-with-user-path --no-ignore cat -- \
+	    file0 ../path1/path2/file2)" in
+    "hello${LF}hello") : ;;
+    *) (exit 1) ;;
+    esac
+'
+
+test_expect_success \
+'feeding find output via xargs from a subdirectory' '
+    case "$(cd path0 &&
+	    find . ../path1 -type f -print0 |
+	    xargs -r -0 git-run-with-user-path --no-ignore cat --)" in
+    "hello${LF}hello${LF}hello") : ;;
+    *) (exit 1) ;;
+    esac
+'
+
+cd $HERE
+mv .git .svn
+GIT_DIR=$(pwd)/.svn
+GIT_PROJECT_TOP=$(pwd)
+export GIT_DIR GIT_PROJECT_TOP
+
+test_expect_success \
+'feeding find output via xargs from a subdirectory (with GIT_PROJECT_TOP)' '
+    case "$(cd path0 &&
+            find . ../path1 -type f -print0 |
+	    xargs -r -0 git-run-with-user-path --no-ignore cat --)" in
+    "hello${LF}hello${LF}hello") : ;;
+    *) (exit 1) ;;
+    esac
+    cd ..
+'
+
+test_done
--- a/update-cache.c
+++ b/update-cache.c
@@ -5,6 +5,7 @@
  */
 #include <signal.h>
 #include "cache.h"
+#include "paths.h"
 
 /*
  * Default to not allowing changes to the list of files. The
@@ -257,34 +258,6 @@
 	return has_errors;
 }
 
-/*
- * We fundamentally don't like some paths: we don't want
- * dot or dot-dot anywhere, and in fact, we don't even want
- * any other dot-files (.git or anything else). They
- * are hidden, for chist sake.
- *
- * Also, we don't want double slashes or slashes at the
- * end that can make pathnames ambiguous.
- */
-static int verify_path(char *path)
-{
-	char c;
-
-	goto inside;
-	for (;;) {
-		if (!c)
-			return 1;
-		if (c == '/') {
-inside:
-			c = *path++;
-			if (c != '/' && c != '.' && c != '\0')
-				continue;
-			return 0;
-		}
-		c = *path++;
-	}
-}
-
 static int add_cacheinfo(char *arg1, char *arg2, char *arg3)
 {
 	int size, len, option;
------------------------------------------------

Re: [PATCH 1/2] Introduce git-run-with-user-path helper program.

[Petr Baudis]

Dear diary, on Mon, May 16, 2005 at 08:05:19AM CEST, I got a letter
where Junio C Hamano <junkio@cox.net> told me that...
> --- a/paths.c
> +++ b/paths.c
> @@ -0,0 +1,199 @@
> +static int initialize_ignore_list(void)
> +{
> +	/* Put the Porcelain layer ignore logic initialization here.
> +	 * Return non-zero after issuing appropriate error message
> +	 * if initialization fails.
> +	 */
> +	return 0;
> +}
> +
> +int path_ignored(const char *path)
> +{
> +	if (!verify_path(path))
> +		return 1;
> +
> +	/* Put the Porcelain layer ignore logic here.
> +	 * Return non-zero if path is to be ignored.
> +	 */
> +	return 0;
> +}

I actually think you shouldn't. All the Porcelain layers should
hopefully use the same git toolkit layer, not each one shipping own due
to differences in things like this.

If we don't agree on something common (implemented in a way to be
still circumventable by a porcelain layer if desired), I wouldn't put
the ignore logic inside at all.

> +/****************************************************************/
> +
> +/* Path canonicalization part */

And why is this in the library?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: [PATCH 1/2] Introduce git-run-with-user-path helper program.

[Junio C Hamano]

>>>>> "PB" == Petr Baudis <pasky@ucw.cz> writes:

>> +int path_ignored(const char *path)
>> +{
>> +	if (!verify_path(path))
>> +		return 1;
>> +
>> +	/* Put the Porcelain layer ignore logic here.
>> +	 * Return non-zero if path is to be ignored.
>> +	 */
>> +	return 0;
>> +}

PB> I actually think you shouldn't. All the Porcelain layers should
PB> hopefully use the same git toolkit layer, not each one shipping own due
PB> to differences in things like this.

What you said above _is_ exactly my intention.  I phrased that
comment very badly.  It should have said:

    /* We _will_ put the "ignore logic Porcelain layers agree upon"
     * here, once we have a concensus.
     *
     * The code should return non-zero if path is to be ignored.
     */

I did not put any implementation there because I do not think we
have agreed upon anything yet.  This patch is to establish
the framework.  

The second patch is separate, because it is _my_ version of the
ignore logic proposal, to serve as a sample.  Whatever ignore
logic is agreed upon, that _will_ be in the place you pointed
out and there will be no choice.  Everybody _will_ use the
ignore logic.

>> +/****************************************************************/
>> +
>> +/* Path canonicalization part */

PB> And why is this in the library?

Why not?  It is something other programs would eventually find
useful.

Also the second patch, a sample implementation of ignore logic I
proposed, wants to know GIT_PROJECT_TOP to figure out the file
pointed at by GIT_DIR/.git/info/ignore-file.

Also it would not hurt if you are always running from the
project top and give only verify_path() approved paths.  Then
canon_path would become identity function.

git-run-with-user-path is useful both in implementing
porcelain-add if the porcelain's policy is to take filesystem
paths not GIT paths, like this:

    #!/bin/sh
    # porcelain-add
    exec git-run-with-user-path git-update-cache --add -- -- "$@"

Also if the porcelain's policy is to take GIT paths not
filesystem paths, then users can say:

    $ find . ! -type d -print0 |
      xargs -0 git-run-with-user-path cg-add --

You cannot use both for obvious reasons.

Re: [PATCH 1/2] Introduce git-run-with-user-path helper program.

[Petr Baudis]

Dear diary, on Tue, May 17, 2005 at 09:27:03PM CEST, I got a letter
where Junio C Hamano <junkio@cox.net> told me that...
> >>>>> "PB" == Petr Baudis <pasky@ucw.cz> writes:
> 
> >> +int path_ignored(const char *path)
> >> +{
> >> +	if (!verify_path(path))
> >> +		return 1;
> >> +
> >> +	/* Put the Porcelain layer ignore logic here.
> >> +	 * Return non-zero if path is to be ignored.
> >> +	 */
> >> +	return 0;
> >> +}
> 
> PB> I actually think you shouldn't. All the Porcelain layers should
> PB> hopefully use the same git toolkit layer, not each one shipping own due
> PB> to differences in things like this.
> 
> What you said above _is_ exactly my intention.  I phrased that
> comment very badly.  It should have said:
> 
>     /* We _will_ put the "ignore logic Porcelain layers agree upon"
>      * here, once we have a concensus.
>      *
>      * The code should return non-zero if path is to be ignored.
>      */
> 
> I did not put any implementation there because I do not think we
> have agreed upon anything yet.  This patch is to establish
> the framework.  

Ok, so this just bad comment. :-) No problem then.


Regarding having the code in the library, well, I'm thinking about why
not to just put this logic into all the git commands. Unfortunately I
can't find the email with Linus' argumentation against that right now.
:-(

> git-run-with-user-path is useful both in implementing
> porcelain-add if the porcelain's policy is to take filesystem
> paths not GIT paths, like this:

Actually, my doubts about general usefulness of this wrapper are
growing. Cogito is unlikely to ever make use of it since it has to
figure out the .git location anyway for own use (it keeps plenty of own
files there). But that's likely what any other porcelain layer would
have to do as well, isn't it? The wrapper could still be useful for the
standalone users, though.

Another thing is, I don't think git-run-with-user-path is the right name.
I think it doesn't make much sense on its own, and the wrapper is
actually doing more anyway, applying the ignore rules. What about
calling it just git-run-wrapper?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: [PATCH 1/2] Introduce git-run-with-user-path helper program.

[Junio C Hamano]

>>>>> "PB" == Petr Baudis <pasky@ucw.cz> writes:

PB> Actually, my doubts about general usefulness of this wrapper are
PB> growing. Cogito is unlikely to ever make use of it since it has to
PB> figure out the .git location anyway for own use (it keeps plenty of own
PB> files there).

I think "having to figure out .git anyway" is backwards, if your
plan is to make Cogito take filesystem paths as opposed to GIT
paths.  If the plan for Cogito is to take always GIT paths,
which is a sensible way as well, then it is irrelevant for the
implementation of Cogito, but then it becomes useful for users
of Cogito).

If your plan is to make Cogito take filesystem paths, then you
can move bulk of the code currently in cg-blah, except the part
that picks up non-path parameters, to cg-Xblah, and reduce
cg-blah implementation down to just:

    ... parse options by shifting "$@" out.
    ... then
    git-run-with-user-path cg-Xblah $non-path-opts -- "$@"

and you can rip "the code to figure out .git" out from cg-Xblah.
There is nothing to figure out at that point; it always is
${GIT_DIR-.git}/.

Re: [PATCH 1/2] Introduce git-run-with-user-path helper program.

[Petr Baudis]

Dear diary, on Tue, May 17, 2005 at 11:18:18PM CEST, I got a letter
where Junio C Hamano <junio@siamese.dyndns.org> told me that...
> If your plan is to make Cogito take filesystem paths, then you

Yes, that's my plan.

> can move bulk of the code currently in cg-blah, except the part
> that picks up non-path parameters, to cg-Xblah, and reduce
> cg-blah implementation down to just:
> 
>     ... parse options by shifting "$@" out.
>     ... then
>     git-run-with-user-path cg-Xblah $non-path-opts -- "$@"
> 
> and you can rip "the code to figure out .git" out from cg-Xblah.
> There is nothing to figure out at that point; it always is
> ${GIT_DIR-.git}/.

But that won't work good enough for me. E.g. when committing in a
subdirectory, I want to commit only changes made in the subdirectory,
etc.

Not even talking about much uglier implementation (that could be
remedied by calling myself recursively with some special argument).

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: [PATCH 1/2] Introduce git-run-with-user-path helper program.

[Junio C Hamano]

>>>>> "PB" == Petr Baudis <pasky@ucw.cz> writes:

PB> But that won't work good enough for me. E.g. when committing in a
PB> subdirectory, I want to commit only changes made in the subdirectory,
PB> etc.

Assuming that you have something that lets you commit selected
files when you are at the top level (say cg-commit), and further
assuming that today it only works from the toplevel, that is:

    $ pwd
    /usr/src/linux
    $ cg-commit fs/ext?/Makefile

works today, what I am saying is:

    $ pwd
    /usr/src/linux/fs
    $ git-run-with-user-path cg-commit -- ext?/Makefile

would work.

Usually the command like cg-commit would take non-path
parameters, so if this works today:

    $ pwd
    /usr/src/linux
    $ cg-commit -m 'Changed Makefile' fs/ext?/Makefile

then:

    $ pwd
    /usr/src/linux/fs
    $ git-run-with-user-path cg-commit -m 'Changed Makefile' -- ext?/Makefile

would work.

Once you have a core that works well but only at the top
directory level, then you can make a thin wrapper using
git-run-with-user-path to make that work equally well with the
filesystem path from subdirectories.  And the core-ish thing
that only works at the top directory level does not need to
worry about finding .git/ anymore, which is the whole point of
what this helper is giving you.



BTW, I am wondering if your choice of cg-commit as an example
(as opposed to something else like diff or add) is a flamebait
or just an innocent random example ;-)?

Re: [PATCH 1/2] Introduce git-run-with-user-path helper program.

[Petr Baudis]

Dear diary, on Wed, May 18, 2005 at 12:13:32AM CEST, I got a letter
where Junio C Hamano <junkio@cox.net> told me that...
> >>>>> "PB" == Petr Baudis <pasky@ucw.cz> writes:
> 
> PB> But that won't work good enough for me. E.g. when committing in a
> PB> subdirectory, I want to commit only changes made in the subdirectory,
> PB> etc.
> 
> Assuming that you have something that lets you commit selected
> files when you are at the top level (say cg-commit), and further
> assuming that today it only works from the toplevel, that is:
> 
>     $ pwd
>     /usr/src/linux
>     $ cg-commit fs/ext?/Makefile
> 
> works today, what I am saying is:
> 
>     $ pwd
>     /usr/src/linux/fs
>     $ git-run-with-user-path cg-commit -- ext?/Makefile
> 
> would work.

Yes. But if you do just cg-commit in the subdirectory, it won't work.
You could pass the original directory in some environment variable or
whatever, but I think that's just not worth the trouble for Cogito -
it's much easier for it when you just stay in the directory you are in
and instead set the environment variables so that the git toolkit DTRT.
(I like this acronym. :-)

> BTW, I am wondering if your choice of cg-commit as an example
> (as opposed to something else like diff or add) is a flamebait
> or just an innocent random example ;-)?

It was completely innocent. :-) How would it be a flamebait?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: [PATCH 1/2] Introduce git-run-with-user-path helper program.

[Junio C Hamano]

>>>>> "PB" == Petr Baudis <pasky@ucw.cz> writes:

>> $ pwd
>> /usr/src/linux/fs
>> $ git-run-with-user-path cg-commit -- ext?/Makefile
>> 
>> would work.

PB> Yes. But if you do just cg-commit in the subdirectory, it won't work.

The point of git-run-with-user-path is that it canonicalizes and
filters the paths, chdir(2)'s to GIT_PROJECT_TOP before running
cg-commit.  So when cg-commit starts in the above example,

    (1) its $cwd is /usr/src/linux and your .git subdirectory is
        right there in ./.git/
    (2) it gets fs/ext2/Makefile and fs/ext3/Makefile as arguments.

>> BTW, I am wondering if your choice of cg-commit as an example
>> (as opposed to something else like diff or add) is a flamebait
>> or just an innocent random example ;-)?

PB> It was completely innocent. :-) How would it be a flamebait?

<http://members.cox.net/junkio/per-file-commit.txt> ;-).

Re: [PATCH 1/2] Introduce git-run-with-user-path helper program.

[Petr Baudis]

Dear diary, on Thu, May 19, 2005 at 12:41:38AM CEST, I got a letter
where Junio C Hamano <junkio@cox.net> told me that...
> >>>>> "PB" == Petr Baudis <pasky@ucw.cz> writes:
> 
> >> $ pwd
> >> /usr/src/linux/fs
> >> $ git-run-with-user-path cg-commit -- ext?/Makefile
> >> 
> >> would work.
> 
> PB> Yes. But if you do just cg-commit in the subdirectory, it won't work.
> 
> The point of git-run-with-user-path is that it canonicalizes and
> filters the paths, chdir(2)'s to GIT_PROJECT_TOP before running
> cg-commit.  So when cg-commit starts in the above example,
> 
>     (1) its $cwd is /usr/src/linux and your .git subdirectory is
>         right there in ./.git/
>     (2) it gets fs/ext2/Makefile and fs/ext3/Makefile as arguments.

Yes. My point is that sometimes the Cogito commands have
directory-specific functionality even when called without any arguments.

$ pwd
/usr/src/linux
$ date >>README
$ cd fs
$ date >>Makefile
$ cg-commit

will commit only the fs/Makefile change.

> >> BTW, I am wondering if your choice of cg-commit as an example
> >> (as opposed to something else like diff or add) is a flamebait
> >> or just an innocent random example ;-)?
> 
> PB> It was completely innocent. :-) How would it be a flamebait?
> 
> <http://members.cox.net/junkio/per-file-commit.txt> ;-).

JIT's snapshotting makes up for it, I think. It has some beauty. :-)

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: [PATCH 1/2] Introduce git-run-with-user-path helper program.

[Junio C Hamano]

>>>>> "PB" == Petr Baudis <pasky@ucw.cz> writes:

PB> Yes. My point is that sometimes the Cogito commands have
PB> directory-specific functionality even when called without any arguments.

PB> $ pwd
PB> /usr/src/linux
PB> $ date >>README
PB> $ cd fs
PB> $ date >>Makefile
PB> $ cg-commit

PB> will commit only the fs/Makefile change.

Ah, thanks.  That what I missed.

Re: [PATCH 1/2] Introduce git-run-with-user-path helper program.

[Linus Torvalds]

On Wed, 18 May 2005, Junio C Hamano wrote:
> 
> PB> Yes. My point is that sometimes the Cogito commands have
> PB> directory-specific functionality even when called without any arguments.
> 
> PB> $ pwd
> PB> /usr/src/linux
> PB> $ date >>README
> PB> $ cd fs
> PB> $ date >>Makefile
> PB> $ cg-commit
> 
> PB> will commit only the fs/Makefile change.
> 
> Ah, thanks.  That what I missed.

Note that if git-run-with-user-path just has some way to tell what the
relative pathname of the original program was (say $DEF_SUBDIRECTORY),
this could still fairly easily be handled: having the cg-Xcommit program
say "if there are no arguments, we default to $DEF_SUBDIRECTORY" rather
than "with no arguments, default to '.'".

I don't personally much care, since this is all porcelain, but basically I
don't think these things are in any way mutually incompatible, and I do
believe that git-run-with-user-path _could_ be a good way to abstract out
the "where the heck in the tree am I?" issues.

		Linus

Re: [PATCH 1/2] Introduce git-run-with-user-path helper program.

[Junio C Hamano]

>>>>> "LT" == Linus Torvalds <torvalds@osdl.org> writes:

LT> ... I do believe that git-run-with-user-path _could_ be a
LT> good way to abstract out the "where the heck in the tree am
LT> I?" issues.

Yes, I am still in search of a good way to abstract that issue
out and I myself is not yet convinced that the command in its
current form _is_ a good enough way yet.

What I am most unhappy about with it lies elsewhere, though.
There needs to be a better way to tell it how the underlying
command handles non-paths arguments, so that I can just say

    git-run-with-user-path <some option spec for the command> \
        command arg1 arg2 arg3 ...

and if arg1 through argO is non-path options then have it
canonicalize and filter only starting from argO+1.  That would
alleviate one issue I have with the current implementation.

Re: [PATCH 1/2] Introduce git-run-with-user-path helper program.

[Thomas Glanzmann]

Hello,

> > <http://members.cox.net/junkio/per-file-commit.txt> ;-).

> I think the workflow that involves per-file commit is
> fundamentally broken at two levels.

I disagree here. We at FAUmachine often have FLAGS to turn on specific
features or debugging output by tweaking a headerfile. However we commit
often and work in small steps (at the moment using CVS because every
developer has write access). And we definitely don't want to tweak the
header file every time we commit, so I often do a:

cvs diff > diff
vim diff
lsdiff diff
cvs commit <interesting files here>

	Thomas

Re: [PATCH 1/2] Introduce git-run-with-user-path helper program.

[Junio C Hamano]

>>>>> "TG" == Thomas Glanzmann <sithglan@stud.uni-erlangen.de> writes:

TG> Hello,
>> > <http://members.cox.net/junkio/per-file-commit.txt> ;-).

>> I think the workflow that involves per-file commit is
>> fundamentally broken at two levels.

TG> I disagree here. We at FAUmachine often have FLAGS to turn on specific
TG> features or debugging output by tweaking a headerfile.

So what?  I would understand that in your workflow, the nature
of that never-committed headerfile (the fact it only has
debugging tweaks and contains nothing substantially risky)
practically minimizes the risk to the level everybody in the
group feels acceptable.

That does not, however, change what I stated in the document:
what you have in the repository is something that never existed
in a work tree as a consistent whole and tested.

You are only saying is that it does not practically matter in
your workflow, only because what is floating (not checked in)
are things you feel safe to drift.  I would not dare say that is
a wrong way to work.

However, I feel fairly strong about this after being burned many
times by careless coleagues who forgot to check in either newly
created files or locally modified files and finding problems
only after customer installation happened.