Tracking branch history

Tracking branch history

[Daniel Barkalow]

One feature that might make git more intuitive to people is if we were to 
additionally track the history of what commit was the head of each branch 
over time. This is only vaguely related to the history of the content, but 
it's well-defined and sometimes significant.

E.g., if you know that two weeks ago, what you had worked, but it doesn't 
work now, you can use git-bisect to figure out what happened, but first 
you have to figure out what commit it was that you were using two weeks 
ago. Two weeks ago, we had that information, but we didn't keep it.

It would probably also be useful for showing changelogs in a 
locally-useful order. If you merge in a tree that's been in separate 
development for a long time, the commits in that tree will be interleaved 
in commit date with the commits you did locally. You tend to want to 
attribute all of the changes that happened in the merge to the time of the 
merge, but that commit object isn't going to tell you anything useful, 
because it may have been done by the other tree (and you fast-forwarded to 
the merge). In fact, you may want to attribute the changes to the 
fast-forward, which can't recorded in the content history, because nothing 
happened to the content. On the other hand, if we were to also record the 
branch history, we could give output which shows changes in the order they 
reached the local tree (then ordered by the commit tree), just by having 
it do:

<time now>

git log <head-as-of-before>..<head-of-of-now>

<time before>

git log <head-as-of-before-that>..<head-as-of-before>

<time before-that>

and so forth.

	-Daniel
*This .sig left intentionally blank*

Re: Tracking branch history

[Linus Torvalds]

On Fri, 12 May 2006, Daniel Barkalow wrote:
>
> One feature that might make git more intuitive to people is if we were to 
> additionally track the history of what commit was the head of each branch 
> over time. This is only vaguely related to the history of the content, but 
> it's well-defined and sometimes significant.
> 
> E.g., if you know that two weeks ago, what you had worked, but it doesn't 
> work now, you can use git-bisect to figure out what happened, but first 
> you have to figure out what commit it was that you were using two weeks 
> ago. Two weeks ago, we had that information, but we didn't keep it.

Note that this is possible, but it must be done literally as a separate 
history from the commit history. 

IOW, a good (?) way to do it is to literally have a commit hook that 
basically just does

	echo $new >> .git/$branch-commit-history

possibly together with a datestamp thing (ie it could be something like
"echo $new "$USER" $(date)" rather than just the commit SHA1).

Make sure that not just "git commit", but anything else that changes the 
branch (notably, "git fetch" and a fast-forward merge as a result of an 
explicit merge or a "git pull") would also do this same thing.

But realize that this is really purely a per-repository logging thing, and 
not really bound to the actual git history any way.

			Linus

Re: Tracking branch history

[Linus Torvalds]

On Fri, 12 May 2006, Linus Torvalds wrote:
> 
> IOW, a good (?) way to do it is to literally have a commit hook that 
> basically just does
> 
> 	echo $new >> .git/$branch-commit-history
> 
> possibly together with a datestamp thing (ie it could be something like
> "echo $new "$USER" $(date)" rather than just the commit SHA1).

Btw, the real problem with this is how to use it.

The only really valid use I see is to use it for date-based things, ie if 
given a date, look up the most recent commit ID that is older than the 
date in question. No other op seems to really make sense, but that one 
does.

Now, the one other operation that is semantically sensible is to use the 
list of commits to figure out a "path" through the commit space. However, 
that path won't actually even be well-defined (a fast-forward pull/merge 
can and often /will/ update the history in a way where it's impossible to 
select one particular path to the previous commit listed in the commit 
log).

The other thing that makes the "path" thing hard is that it's just 
fundamentally a pretty hard thing to calculate, even when it would result 
in one unambiguous path. I _believe_ that it comes close to what "git 
bisect" does, and that the bisect algorithm could probably be used to 
always create _a_ path between each commit (is just pick successive 
half-way-points - the commit list _should_ always have a direct dominance 
relationship, but the bisection algorithm should do something half-way 
sane even if you "jump about" by "git reset" or something).

It might be interesting to see if it's somethign that can be done 
reasonably efficiently.

			Linus

Re: Tracking branch history

[Daniel Barkalow]

On Fri, 12 May 2006, Linus Torvalds wrote:

> Btw, the real problem with this is how to use it.
> 
> The only really valid use I see is to use it for date-based things, ie if 
> given a date, look up the most recent commit ID that is older than the 
> date in question. No other op seems to really make sense, but that one 
> does.
> 
> Now, the one other operation that is semantically sensible is to use the 
> list of commits to figure out a "path" through the commit space. However, 
> that path won't actually even be well-defined (a fast-forward pull/merge 
> can and often /will/ update the history in a way where it's impossible to 
> select one particular path to the previous commit listed in the commit 
> log).

I think that jumping around with reset is necessary to make this actually 
complicated; a fast-forward only happens if the new value descends from 
the old value, and a merge obviously descends from the old value. Sure, 
the non-linear history added by a merge will still be non-linear, but 
from the local user's point of view, it was all added in bulk by the 
merge.

I think the program creating the history should note the tricky cases, 
where the new value doesn't descend from the old value, which should be 
easy to identify. I'm not sure what should actually be done to report a 
reset in a changelog, either. The section of the log just before the reset 
is clearly a false start of some sort, and you probably want to do 
something special to list the commits which don't actually lead to the 
current state, but you probably want to report them, in case the reason 
you'd looking through this is that there was some benefit to a version 
that you ended up discarding, and you want to revisit that attempt.

I think in the always-forward case, there's a useful optimization to be 
had by having the rev-list-equivalent actually binning commits by the 
earliest points that descend from them, so you don't trace the local 
branch back to where other branches forked off over and over. But it seems 
to me otherwise pretty simple.

	-Daniel
*This .sig left intentionally blank*

Re: Tracking branch history

[Shawn Pearce]

Linus Torvalds <torvalds@osdl.org> wrote:
> 
> On Fri, 12 May 2006, Daniel Barkalow wrote:
> >
> > One feature that might make git more intuitive to people is if we were to 
> > additionally track the history of what commit was the head of each branch 
> > over time. This is only vaguely related to the history of the content, but 
> > it's well-defined and sometimes significant.
> > 
> > E.g., if you know that two weeks ago, what you had worked, but it doesn't 
> > work now, you can use git-bisect to figure out what happened, but first 
> > you have to figure out what commit it was that you were using two weeks 
> > ago. Two weeks ago, we had that information, but we didn't keep it.
> 
> Note that this is possible, but it must be done literally as a separate 
> history from the commit history. 
> 
> IOW, a good (?) way to do it is to literally have a commit hook that 
> basically just does
> 
> 	echo $new >> .git/$branch-commit-history
> 
> possibly together with a datestamp thing (ie it could be something like
> "echo $new "$USER" $(date)" rather than just the commit SHA1).

Why not intergrate this into git-update-ref.  Almost every tool which
deals with a GIT repository (aside from my pure-Java Eclipse plugin
which is still a major work-in-process) performs ref changes through
git-udpate-ref.  So just have it append the ref's history to a file:

	.git/log/refs/heads/$branch

where the history records are stored as:

	40 byte commit-ish SHA1
	<SP>
	<committer>
	<LF>

e.g.:

	cbb6d91d95e523c2b6a6b52577c4be28d18ace83 Shawn O. Pearce <spearce@spearce.org> 1137039378 -0500
	ae8c74e96a1e02bbfb7f1a9669b77d6f8ee6c3cf Shawn O. Pearce <spearce@spearce.org> 1136921470 -0500

Of course a major issue here is locking the log file during the ref
update, but it looks like it might just be safe to append the entry
to the log file right after the re_verify and before the rename.

I wouldn't have git-update-ref create the log file. I'd would only
have it append if the log already exists.

Hmm, this actually looks like it would be really easy.  Maybe I'll
hack up an RFC patch this evening after dinner.

-- 
Shawn.

Re: Tracking branch history

[Linus Torvalds]

On Fri, 12 May 2006, Shawn Pearce wrote:
> 
> Why not intergrate this into git-update-ref.  Almost every tool which
> deals with a GIT repository (aside from my pure-Java Eclipse plugin
> which is still a major work-in-process) performs ref changes through
> git-udpate-ref.  So just have it append the ref's history to a file:
> 
> 	.git/log/refs/heads/$branch
> 
> where the history records are stored as:
> 
> 	40 byte commit-ish SHA1
> 	<SP>
> 	<committer>
> 	<LF>

Sure. Except it's not really "committer", in the ordinary sense (there's 
no "commit" for a fast-forward). But yes, re-using that format (with date 
and all) makes sense.

> Of course a major issue here is locking the log file during the ref
> update, but it looks like it might just be safe to append the entry
> to the log file right after the re_verify and before the rename.

I'd suggest just opening it with O_APPEND, and doing the update with a 
single write() system call. Let the OS do the locking for you. 

		Linus

Re: Tracking branch history

[Junio C Hamano]

Shawn Pearce <spearce@spearce.org> writes:

> git-udpate-ref.  So just have it append the ref's history to a file:
>
> 	.git/log/refs/heads/$branch
>
> where the history records are stored as:
>
> 	40 byte commit-ish SHA1
> 	<SP>
> 	<committer>
> 	<LF>
>
> e.g.:
>
> 	cbb6d91d95e523c2b6a6b52577c4be28d18ace83 Shawn O. Pearce <spearce@spearce.org> 1137039378 -0500
> 	ae8c74e96a1e02bbfb7f1a9669b77d6f8ee6c3cf Shawn O. Pearce <spearce@spearce.org> 1136921470 -0500
>

Because the question we often would want to ask is "two days ago
my tip worked but today it does not", recording the timestamp
makes sense, but I do not know what the point is for the name
and e-mail.  If it is in your local repository (i.e. the program
that updates the tip ref is not receive-pack which is invoked by
your pushing into a remote repo), it will always be you.  And in
the receive-pack case, the information is not available to begin
with (you may have a UNIX UID but that is about it).

Re: Tracking branch history

[Shawn Pearce]

Junio C Hamano <junkio@cox.net> wrote:
> Shawn Pearce <spearce@spearce.org> writes:
> 
> > git-udpate-ref.  So just have it append the ref's history to a file:
> >
> > 	.git/log/refs/heads/$branch
> >
> > where the history records are stored as:
> >
> > 	40 byte commit-ish SHA1
> > 	<SP>
> > 	<committer>
> > 	<LF>
> >
> > e.g.:
> >
> > 	cbb6d91d95e523c2b6a6b52577c4be28d18ace83 Shawn O. Pearce <spearce@spearce.org> 1137039378 -0500
> > 	ae8c74e96a1e02bbfb7f1a9669b77d6f8ee6c3cf Shawn O. Pearce <spearce@spearce.org> 1136921470 -0500
> >
> 
> Because the question we often would want to ask is "two days ago
> my tip worked but today it does not", recording the timestamp
> makes sense, but I do not know what the point is for the name
> and e-mail.  If it is in your local repository (i.e. the program
> that updates the tip ref is not receive-pack which is invoked by
> your pushing into a remote repo), it will always be you.  And in
> the receive-pack case, the information is not available to begin
> with (you may have a UNIX UID but that is about it).

Agreed.  Prototype patch below.

While writing this I discovered at least two chunks of GIT which
don't use git-update-ref: fetch.c and upload-pack.c.  fetch.c uses
the APIs in refs.c but upload-pack.c doesn't.  I spent a couple of
hours trying to convert update-ref.c to use the APIs in refs.c so
I could just put the logging change there, but that turned out to
be more difficult than expected for a simple prototype so it all
went out the window.

-- >-
Log ref updates to logs/refs/<refname>

If .git/logs/refs/<refname> exists then append a line to it whenever
git-update-ref <refname> is executed.  Each log line contains the
following information:

  40 byte tree-ish SHA1
  <SP>
  date/time
  <LF>

where date/time is the current date, time and timezone in the
standard GIT date format.  If the caller is unable to append to
the log file and the log file exists then git-update-ref will fail
without updating <refname>.

---

 Documentation/git-update-ref.txt |   15 ++++++++++++++
 update-ref.c                     |   41 +++++++++++++++++++++++++++++++++++---
 2 files changed, 53 insertions(+), 3 deletions(-)

8f1ccd3b0eda9d54eca37af178113c91174e81ca
diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 475237f..d314786 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -49,6 +49,21 @@ for reading but not for writing (so we'l
 ref symlink to some other tree, if you have copied a whole
 archive by creating a symlink tree).
 
+Logging Updates
+---------------
+If "$GIT_DIR/logs/<ref>" (possibly dereferencing symbolic refs)
+exists then `git-update-ref` will append a line to the log file
+describing the change in ref value.  Log lines are formatted as:
+
+    . sha1 SP date LF
++
+Where "sha1" is the 40 character hexadecimal value of <newvalue>
+and "date" is the current date/time and timezone in the standard
+GIT date format.
+
+An update will fail (without changing <ref>) if the log file
+exists but the current user is unable to append to the file.
+
 Author
 ------
 Written by Linus Torvalds <torvalds@osdl.org>.
diff --git a/update-ref.c b/update-ref.c
index fd48742..bffe5f9 100644
--- a/update-ref.c
+++ b/update-ref.c
@@ -20,9 +20,9 @@ int main(int argc, char **argv)
 {
 	char *hex;
 	const char *refname, *value, *oldval, *path;
-	char *lockpath;
+	char *lockpath, *logpath;
 	unsigned char sha1[20], oldsha1[20], currsha1[20];
-	int fd, written;
+	int fd, logfd, written, pfxlen;
 
 	setup_git_directory();
 	git_config(git_default_config);
@@ -38,7 +38,9 @@ int main(int argc, char **argv)
 	if (oldval && get_sha1(oldval, oldsha1))
 		die("%s: not a valid old SHA1", oldval);
 
-	path = resolve_ref(git_path("%s", refname), currsha1, !!oldval);
+	path = git_path("%s", refname);
+	pfxlen = strlen(path) - strlen(refname);
+	path = resolve_ref(path, currsha1, !!oldval);
 	if (!path)
 		die("No such ref: %s", refname);
 
@@ -50,6 +52,17 @@ int main(int argc, char **argv)
 			exit(0);
 	}
 	path = strdup(path);
+
+	/*
+	 * If logging is required make sure we can append to the log.
+	 */
+	logpath = strdup(git_path("logs/%s", path + pfxlen));
+	logfd = open(logpath, O_APPEND | O_WRONLY, 0);
+	if (logfd < 0 && errno != ENOENT)
+		die("Unable to append to log %s", logpath);
+	if (logfd >= 0)
+		setup_ident();
+
 	lockpath = mkpath("%s.lock", path);
 	if (safe_create_leading_directories(lockpath) < 0)
 		die("Unable to create all of %s", lockpath);
@@ -75,6 +88,28 @@ int main(int argc, char **argv)
 	}
 
 	/*
+	 * Write to the log, if it was opened.
+	 */
+	if (logfd >= 0) {
+		char now[50];
+		char logrec[100];
+		unsigned len;
+
+		datestamp(now, sizeof(now));
+		len = snprintf(logrec, sizeof(logrec), "%s %s\n", sha1_to_hex(sha1), now);
+		if (len >= sizeof(logrec)) {
+			unlink(lockpath);
+			die("Internal error formatting log record.");
+		}
+		written = write(logfd, logrec, len);
+		if (written != len) {
+			unlink(lockpath);
+			die("Unable to append to %s", logpath);
+		}
+		close(logfd);
+	}
+
+	/*
 	 * Finally, replace the old ref with the new one
 	 */
 	if (rename(lockpath, path) < 0) {
-- 
1.3.2.g7278

Re: Tracking branch history

[Shawn Pearce]

Shawn Pearce <spearce@spearce.org> wrote:
> Junio C Hamano <junkio@cox.net> wrote:
> > Shawn Pearce <spearce@spearce.org> writes:
> > 
> > > git-udpate-ref.  So just have it append the ref's history to a file:
> > >
> > > 	.git/log/refs/heads/$branch
> > >
> > > where the history records are stored as:
> > >
> > > 	40 byte commit-ish SHA1
> > > 	<SP>
> > > 	<committer>
> > > 	<LF>
> > >
> > > e.g.:
> > >
> > > 	cbb6d91d95e523c2b6a6b52577c4be28d18ace83 Shawn O. Pearce <spearce@spearce.org> 1137039378 -0500
> > > 	ae8c74e96a1e02bbfb7f1a9669b77d6f8ee6c3cf Shawn O. Pearce <spearce@spearce.org> 1136921470 -0500
> > >
> > 
> > Because the question we often would want to ask is "two days ago
> > my tip worked but today it does not", recording the timestamp
> > makes sense, but I do not know what the point is for the name
> > and e-mail.  If it is in your local repository (i.e. the program
> > that updates the tip ref is not receive-pack which is invoked by
> > your pushing into a remote repo), it will always be you.  And in
> > the receive-pack case, the information is not available to begin
> > with (you may have a UNIX UID but that is about it).

Forget my last patch.  This one automatically creates the log file
by looking for a config value of 'core.logRefUpdates=true'.

--> -
Log ref updates to logs/refs/<ref>

If config parameter core.logRefUpdates is true then append a line
to .git/logs/refs/<ref> whenever git-update-ref <ref> is executed.
Each log line contains the following information:

  40 byte tree-ish SHA1
  <SP>
  date/time
  <LF>

where date/time is the current date, time and timezone in the
standard GIT date format.  If the caller is unable to append to
the log file then git-update-ref will fail without updating <ref>.

---

 Documentation/config.txt         |    7 ++++++
 Documentation/git-update-ref.txt |   17 +++++++++++++++
 cache.h                          |    1 +
 config.c                         |    5 ++++
 environment.c                    |    1 +
 update-ref.c                     |   43 +++++++++++++++++++++++++++++++++++---
 6 files changed, 71 insertions(+), 3 deletions(-)

cac86f2df9a52d94cb03038e267934c67f04122b
diff --git a/Documentation/config.txt b/Documentation/config.txt
index d1a4bec..f06695c 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -70,6 +70,13 @@ core.preferSymlinkRefs::
 	This is sometimes needed to work with old scripts that
 	expect HEAD to be a symbolic link.
 
+core.logRefUpdates::
+	If true, `git-update-ref` will append a line to
+	"$GIT_DIR/logs/<ref>" listing the new SHA1 and the date/time
+	of the update.	This information can be used to determine
+	what commit was the tip of a branch "2 days ago".  This value
+	is false by default (no logging).
+
 core.repositoryFormatVersion::
 	Internal variable identifying the repository format and layout
 	version.
diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 475237f..8c46263 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -49,6 +49,23 @@ for reading but not for writing (so we'l
 ref symlink to some other tree, if you have copied a whole
 archive by creating a symlink tree).
 
+Logging Updates
+---------------
+If config parameter "core.logRefUpdates" is true then
+`git-update-ref` will append a line to the log file
+"$GIT_DIR/logs/<ref>" (dereferencing all symbolic refs before
+creating the log name) describing the change in ref value.  Log lines
+are formatted as:
+
+    . sha1 SP date LF
++
+Where "sha1" is the 40 character hexadecimal value of <newvalue>
+and "date" is the current date/time and timezone in the standard
+GIT date format.
+
+An update will fail (without changing <ref>) if the current user is
+unable to create a new log file or append to the existing log file.
+
 Author
 ------
 Written by Linus Torvalds <torvalds@osdl.org>.
diff --git a/cache.h b/cache.h
index b1300cd..887ce20 100644
--- a/cache.h
+++ b/cache.h
@@ -171,6 +171,7 @@ extern void rollback_index_file(struct c
 extern int trust_executable_bit;
 extern int assume_unchanged;
 extern int prefer_symlink_refs;
+extern int log_ref_updates;
 extern int warn_ambiguous_refs;
 extern int diff_rename_limit_default;
 extern int shared_repository;
diff --git a/config.c b/config.c
index 0f518c9..f8a814e 100644
--- a/config.c
+++ b/config.c
@@ -232,6 +232,11 @@ int git_default_config(const char *var, 
 		return 0;
 	}
 
+	if (!strcmp(var, "core.logrefupdates")) {
+		log_ref_updates = git_config_bool(var, value);
+		return 0;
+	}
+
 	if (!strcmp(var, "core.warnambiguousrefs")) {
 		warn_ambiguous_refs = git_config_bool(var, value);
 		return 0;
diff --git a/environment.c b/environment.c
index 444c99e..437266e 100644
--- a/environment.c
+++ b/environment.c
@@ -14,6 +14,7 @@ char git_default_name[MAX_GITNAME];
 int trust_executable_bit = 1;
 int assume_unchanged = 0;
 int prefer_symlink_refs = 0;
+int log_ref_updates = 0;
 int warn_ambiguous_refs = 1;
 int repository_format_version = 0;
 char git_commit_encoding[MAX_ENCODING_LENGTH] = "utf-8";
diff --git a/update-ref.c b/update-ref.c
index fd48742..c231760 100644
--- a/update-ref.c
+++ b/update-ref.c
@@ -22,7 +22,7 @@ int main(int argc, char **argv)
 	const char *refname, *value, *oldval, *path;
 	char *lockpath;
 	unsigned char sha1[20], oldsha1[20], currsha1[20];
-	int fd, written;
+	int fd, written, pfxlen;
 
 	setup_git_directory();
 	git_config(git_default_config);
@@ -38,7 +38,9 @@ int main(int argc, char **argv)
 	if (oldval && get_sha1(oldval, oldsha1))
 		die("%s: not a valid old SHA1", oldval);
 
-	path = resolve_ref(git_path("%s", refname), currsha1, !!oldval);
+	path = git_path("%s", refname);
+	pfxlen = strlen(path) - strlen(refname);
+	path = resolve_ref(path, currsha1, !!oldval);
 	if (!path)
 		die("No such ref: %s", refname);
 
@@ -50,7 +52,7 @@ int main(int argc, char **argv)
 			exit(0);
 	}
 	path = strdup(path);
-	lockpath = mkpath("%s.lock", path);
+	lockpath = strdup(mkpath("%s.lock", path));
 	if (safe_create_leading_directories(lockpath) < 0)
 		die("Unable to create all of %s", lockpath);
 
@@ -75,6 +77,41 @@ int main(int argc, char **argv)
 	}
 
 	/*
+	 * Write to the log if logging of ref updates is enabled
+	 */
+	if (log_ref_updates) {
+		char *logpath;
+		char now[50];
+		char logrec[100];
+		unsigned len;
+		int logfd;
+
+		datestamp(now, sizeof(now));
+		len = snprintf(logrec, sizeof(logrec), "%s %s\n", sha1_to_hex(sha1), now);
+		if (len >= sizeof(logrec)) {
+			unlink(lockpath);
+			die("Internal error formatting log record.");
+		}
+
+		logpath = git_path("logs/%s", path + pfxlen);
+		if (safe_create_leading_directories(logpath) < 0) {
+			unlink(lockpath);
+			die("Unable to create all of %s", logpath);
+		}
+		logfd = open(logpath, O_CREAT | O_APPEND | O_WRONLY, 0666);
+		if (logfd < 0) {
+			unlink(lockpath);
+			die("Unable to append to log %s", logpath);
+		}
+		written = write(logfd, logrec, len);
+		if (written != len) {
+			unlink(lockpath);
+			die("Unable to append to %s", logpath);
+		}
+		close(logfd);
+	}
+
+	/*
 	 * Finally, replace the old ref with the new one
 	 */
 	if (rename(lockpath, path) < 0) {
-- 
1.3.2.g7278

Re: Tracking branch history

[Linus Torvalds]

On Sat, 13 May 2006, Shawn Pearce wrote:
> 
> Forget my last patch.  This one automatically creates the log file
> by looking for a config value of 'core.logRefUpdates=true'.

Looks better.

I actually disagree with Junio - I think the logging the name made sense. 
If you have a shared repo with multiple people pushing to the same branch, 
it's interesting to see who does the pushing.

I also think it might be good to save the oldsha1 value. Yes, it _should_ 
always be the previous sha1 logged, but it's interesting to see in case it 
isn't (ie the ref was updated some other way), and it's also interesting 
for the first entry after logging has been enabled.

Hmm?

		Linue

Re: Tracking branch history

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> I actually disagree with Junio - I think the logging the name made sense. 
> If you have a shared repo with multiple people pushing to the same branch, 
> it's interesting to see who does the pushing.

Yes, I agree recording that information is interesting.  

My comment was about how it is recorded.  In the local case
GIT_COMMITTER_IDENT is what you want as Shawn did in his
original proposal, but remote "pushing into shared repository"
case I do not think it is available in general.

Well, at least that was what I was thinking when I made that
comment.  However,

If the pushers arrange to have appropriate environment variables
while receive-pack does its work (.git/config in the target
repository is not an appropriate place to get user.* settings
from in a shared setting), what you say makes perfect sense.

Having the usual enviornment available would be handy to make
the hooks on the receiving end to do useful things anyway, so I
retract that suggestion.

> I also think it might be good to save the oldsha1 value. Yes, it _should_ 
> always be the previous sha1 logged, but it's interesting to see in case it 
> isn't (ie the ref was updated some other way), and it's also interesting 
> for the first entry after logging has been enabled.
>
> 		Linue

Yes, and also what user-level command was used to cause update
the ref; was it a merge from remote, own commit, rewind/rebase?

	Junia

;-)

Re: Tracking branch history

[Shawn Pearce]

Junio C Hamano <junkio@cox.net> wrote:
> Linus Torvalds <torvalds@osdl.org> writes:
> 
> > I actually disagree with Junio - I think the logging the name made sense. 
> > If you have a shared repo with multiple people pushing to the same branch, 
> > it's interesting to see who does the pushing.
> 
> Yes, I agree recording that information is interesting.  
> 
> My comment was about how it is recorded.  In the local case
> GIT_COMMITTER_IDENT is what you want as Shawn did in his
> original proposal, but remote "pushing into shared repository"
> case I do not think it is available in general.
> 
> Well, at least that was what I was thinking when I made that
> comment.  However,
> 
> If the pushers arrange to have appropriate environment variables
> while receive-pack does its work (.git/config in the target
> repository is not an appropriate place to get user.* settings
> from in a shared setting), what you say makes perfect sense.
> 
> Having the usual enviornment available would be handy to make
> the hooks on the receiving end to do useful things anyway, so I
> retract that suggestion.
> 
> > I also think it might be good to save the oldsha1 value. Yes, it _should_ 
> > always be the previous sha1 logged, but it's interesting to see in case it 
> > isn't (ie the ref was updated some other way), and it's also interesting 
> > for the first entry after logging has been enabled.
> >
> > 		Linue
> 
> Yes, and also what user-level command was used to cause update
> the ref; was it a merge from remote, own commit, rewind/rebase?
> 
> 	Junia
> 
> ;-)
> 

New version of the patch below; this patch superceeds the prior two.
The format now includes the old sha1 and committer ident, and does
not write a log record if the update-ref is pointless.

Hmm, why does update-ref do a pointless update?

Now the problem with this patch is it doesn't append to the log
in the case of upload-pack or fetch; nor does it record what user
level command caused the update.

  [spearce@pb15 git]$ grep update-ref *.sh *.perl *.py | wc -l
      20

Hmm.  That's easy enough to fix.  Anyone up for a '-m foo' comment
switch to update-ref from the higher-level tools such that foo
appears as part of the log line?

I'll be offline for a few hours but can work on this some more
later tonight.

--> -
Log ref updates to logs/refs/<ref>

If config parameter core.logRefUpdates is true then append a line
to .git/logs/refs/<ref> whenever git-update-ref <ref> is executed.
Each log line contains the following information:

  oldsha1 <SP> newsha1 <SP> committer <LF>

where committer is the current user, date, time and timezone in
the standard GIT ident format.  If the caller is unable to append
to the log file then git-update-ref will fail without updating <ref>.

---

 Documentation/config.txt         |    7 ++++++
 Documentation/git-update-ref.txt |   19 +++++++++++++++
 cache.h                          |    1 +
 config.c                         |    5 ++++
 environment.c                    |    1 +
 update-ref.c                     |   48 ++++++++++++++++++++++++++++++++++++--
 6 files changed, 78 insertions(+), 3 deletions(-)

898e33d9c2def73296697d65dff0676d96adb5d8
diff --git a/Documentation/config.txt b/Documentation/config.txt
index d1a4bec..f06695c 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -70,6 +70,13 @@ core.preferSymlinkRefs::
 	This is sometimes needed to work with old scripts that
 	expect HEAD to be a symbolic link.
 
+core.logRefUpdates::
+	If true, `git-update-ref` will append a line to
+	"$GIT_DIR/logs/<ref>" listing the new SHA1 and the date/time
+	of the update.	This information can be used to determine
+	what commit was the tip of a branch "2 days ago".  This value
+	is false by default (no logging).
+
 core.repositoryFormatVersion::
 	Internal variable identifying the repository format and layout
 	version.
diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 475237f..edd39d3 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -49,6 +49,25 @@ for reading but not for writing (so we'l
 ref symlink to some other tree, if you have copied a whole
 archive by creating a symlink tree).
 
+Logging Updates
+---------------
+If config parameter "core.logRefUpdates" is true then
+`git-update-ref` will append a line to the log file
+"$GIT_DIR/logs/<ref>" (dereferencing all symbolic refs before
+creating the log name) describing the change in ref value.  Log lines
+are formatted as:
+
+    . oldsha1 SP newsha1 SP committer LF
++
+Where "oldsha1" is the 40 character hexadecimal value previously
+stored in <ref>, "newsha1" is the 40 character hexadecimal value of
+<newvalue> and "committer" is the committer's name, email address
+and date in the standard GIT committer ident format.
+
+An update will fail (without changing <ref>) if the current user is
+unable to create a new log file, append to the existing log file
+or does not have committer information available.
+
 Author
 ------
 Written by Linus Torvalds <torvalds@osdl.org>.
diff --git a/cache.h b/cache.h
index b1300cd..887ce20 100644
--- a/cache.h
+++ b/cache.h
@@ -171,6 +171,7 @@ extern void rollback_index_file(struct c
 extern int trust_executable_bit;
 extern int assume_unchanged;
 extern int prefer_symlink_refs;
+extern int log_ref_updates;
 extern int warn_ambiguous_refs;
 extern int diff_rename_limit_default;
 extern int shared_repository;
diff --git a/config.c b/config.c
index 0f518c9..f8a814e 100644
--- a/config.c
+++ b/config.c
@@ -232,6 +232,11 @@ int git_default_config(const char *var, 
 		return 0;
 	}
 
+	if (!strcmp(var, "core.logrefupdates")) {
+		log_ref_updates = git_config_bool(var, value);
+		return 0;
+	}
+
 	if (!strcmp(var, "core.warnambiguousrefs")) {
 		warn_ambiguous_refs = git_config_bool(var, value);
 		return 0;
diff --git a/environment.c b/environment.c
index 444c99e..437266e 100644
--- a/environment.c
+++ b/environment.c
@@ -14,6 +14,7 @@ char git_default_name[MAX_GITNAME];
 int trust_executable_bit = 1;
 int assume_unchanged = 0;
 int prefer_symlink_refs = 0;
+int log_ref_updates = 0;
 int warn_ambiguous_refs = 1;
 int repository_format_version = 0;
 char git_commit_encoding[MAX_ENCODING_LENGTH] = "utf-8";
diff --git a/update-ref.c b/update-ref.c
index fd48742..64fc3db 100644
--- a/update-ref.c
+++ b/update-ref.c
@@ -22,7 +22,7 @@ int main(int argc, char **argv)
 	const char *refname, *value, *oldval, *path;
 	char *lockpath;
 	unsigned char sha1[20], oldsha1[20], currsha1[20];
-	int fd, written;
+	int fd, written, pfxlen;
 
 	setup_git_directory();
 	git_config(git_default_config);
@@ -38,7 +38,9 @@ int main(int argc, char **argv)
 	if (oldval && get_sha1(oldval, oldsha1))
 		die("%s: not a valid old SHA1", oldval);
 
-	path = resolve_ref(git_path("%s", refname), currsha1, !!oldval);
+	path = git_path("%s", refname);
+	pfxlen = strlen(path) - strlen(refname);
+	path = resolve_ref(path, currsha1, !!oldval);
 	if (!path)
 		die("No such ref: %s", refname);
 
@@ -50,7 +52,7 @@ int main(int argc, char **argv)
 			exit(0);
 	}
 	path = strdup(path);
-	lockpath = mkpath("%s.lock", path);
+	lockpath = strdup(mkpath("%s.lock", path));
 	if (safe_create_leading_directories(lockpath) < 0)
 		die("Unable to create all of %s", lockpath);
 
@@ -75,6 +77,46 @@ int main(int argc, char **argv)
 	}
 
 	/*
+	 * Write to the log if logging of ref updates is enabled
+	 */
+	if (log_ref_updates && memcmp(currsha1, sha1, 20)) {
+		char *logrec, *logpath;
+		const char *comitter;
+		unsigned maxlen, len;
+		int logfd;
+
+		setup_ident();
+		comitter = git_committer_info(1);
+		maxlen = strlen(comitter) + 2*40 + 4;
+		logrec = xmalloc(maxlen);
+		len = snprintf(logrec, maxlen, "%s %s %s\n",
+			sha1_to_hex(currsha1),
+			sha1_to_hex(sha1),
+			comitter);
+		if (len >= maxlen) {
+			unlink(lockpath);
+			die("Internal error formatting log record.");
+		}
+
+		logpath = git_path("logs/%s", path + pfxlen);
+		if (safe_create_leading_directories(logpath) < 0) {
+			unlink(lockpath);
+			die("Unable to create all of %s", logpath);
+		}
+		logfd = open(logpath, O_CREAT | O_APPEND | O_WRONLY, 0666);
+		if (logfd < 0) {
+			unlink(lockpath);
+			die("Unable to append to %s", logpath);
+		}
+		written = write(logfd, logrec, len);
+		if (written != len) {
+			unlink(lockpath);
+			die("Unable to append to %s", logpath);
+		}
+		close(logfd);
+	}
+
+	/*
 	 * Finally, replace the old ref with the new one
 	 */
 	if (rename(lockpath, path) < 0) {
-- 
1.3.2.g7278

Re: Tracking branch history

[Junio C Hamano]

Shawn Pearce <spearce@spearce.org> writes:

> Log ref updates to logs/refs/<ref>
>
> If config parameter core.logRefUpdates is true then append a line
> to .git/logs/refs/<ref> whenever git-update-ref <ref> is executed.

I cannot decide if a parameter makes more sense, or just making
the existence of such a file a cue is better.  For example, I do
not much care about when I updated each of my topic branch head,
while I do care about master, next, and pu branches.  A global
parameter would make this black-or-white choice, while opening
the log without O_CREAT and write things out only when the log
file exists might make things as easy and controllable.

I could "touch" the ones I care about to prime the process.

Hmm?

Re: Tracking branch history

[Shawn Pearce]

Junio C Hamano <junkio@cox.net> wrote:
> Shawn Pearce <spearce@spearce.org> writes:
> 
> > Log ref updates to logs/refs/<ref>
> >
> > If config parameter core.logRefUpdates is true then append a line
> > to .git/logs/refs/<ref> whenever git-update-ref <ref> is executed.
> 
> I cannot decide if a parameter makes more sense, or just making
> the existence of such a file a cue is better.  For example, I do
> not much care about when I updated each of my topic branch head,
> while I do care about master, next, and pu branches.  A global
> parameter would make this black-or-white choice, while opening
> the log without O_CREAT and write things out only when the log
> file exists might make things as easy and controllable.
> 
> I could "touch" the ones I care about to prime the process.

I'm as torn as you are on this.  My `pg' would have some serious
issues with update-ref and the logRefUpdates config being true as it
uses a large number of refs to track the patch stack.  StGIT might
also be in a somewhat similiar position, but I think it uses slightly
less refs.  Opening without O_CREAT and letting the user touch the
log file into existance would fix this problem for most users.

But I can also see the case where the user has most branches logged
then infrequently creates a new branch but expects that branch
to also be logged.  What happens when they later find out its not
logging but they were counting on that log data to be there?

Perhaps a mix of the two solutions is ideal.  Use O_CREAT if
logRefUpdates is true and otherwise log only if the file already
exists.  This is now in the patch below.

I have also added an optional '-m <reason>' flag to update-ref,
for use in e.g. git-commit.  I have also moved the logging code to
refs.c where it could be used by fetch or upload-pack, but those
changes are not yet included in this patch.

--> -
Log ref updates to logs/refs/<ref>

If config parameter core.logAllRefUpdates is true or the log
file already exists then append a line to ".git/logs/refs/<ref>"
whenever git-update-ref <ref> is executed.  Each log line contains
the following information:

  oldsha1 <SP> newsha1 <SP> committer <LF>

where committer is the current user, date, time and timezone in
the standard GIT ident format.  If the caller is unable to append
to the log file then git-update-ref will fail without updating <ref>.

An optional message may be included in the log line with the -m flag.

---

 Documentation/config.txt         |    8 ++++++
 Documentation/git-update-ref.txt |   28 +++++++++++++++++++-
 cache.h                          |    1 +
 config.c                         |    5 ++++
 environment.c                    |    1 +
 refs.c                           |   50 ++++++++++++++++++++++++++++++++++++
 refs.h                           |    3 ++
 update-ref.c                     |   53 ++++++++++++++++++++++++++++++++------
 8 files changed, 139 insertions(+), 10 deletions(-)

bbac000fbc779c7434d66b10639eaa5ea218f62e
diff --git a/Documentation/config.txt b/Documentation/config.txt
index d1a4bec..e178ee2 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -70,6 +70,14 @@ core.preferSymlinkRefs::
 	This is sometimes needed to work with old scripts that
 	expect HEAD to be a symbolic link.
 
+core.logAllRefUpdates::
+	If true, `git-update-ref` will append a line to
+	"$GIT_DIR/logs/<ref>" listing the new SHA1 and the date/time
+	of the update.	If the file does not exist it will be
+	created automatically.	This information can be used to
+	determine what commit was the tip of a branch "2 days ago".
+	This value is false by default (no logging).
+
 core.repositoryFormatVersion::
 	Internal variable identifying the repository format and layout
 	version.
diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 475237f..dfbd886 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -7,7 +7,7 @@ git-update-ref - update the object name 
 
 SYNOPSIS
 --------
-'git-update-ref' <ref> <newvalue> [<oldvalue>]
+'git-update-ref' <ref> <newvalue> [<oldvalue>] [-m <reason>]
 
 DESCRIPTION
 -----------
@@ -49,6 +49,32 @@ for reading but not for writing (so we'l
 ref symlink to some other tree, if you have copied a whole
 archive by creating a symlink tree).
 
+Logging Updates
+---------------
+If config parameter "core.logAllRefUpdates" is true or the file
+"$GIT_DIR/logs/<ref>" exists then `git-update-ref` will append
+a line to the log file "$GIT_DIR/logs/<ref>" (dereferencing all
+symbolic refs before creating the log name) describing the change
+in ref value.  Log lines are formatted as:
+
+    . oldsha1 SP newsha1 SP committer LF
++
+Where "oldsha1" is the 40 character hexadecimal value previously
+stored in <ref>, "newsha1" is the 40 character hexadecimal value of
+<newvalue> and "committer" is the committer's name, email address
+and date in the standard GIT committer ident format.
+
+Optionally with -m:
+
+    . oldsha1 SP newsha1 SP committer TAB message LF
++
+Where all fields are as described above and "message" is the
+value supplied to the -m option.
+
+An update will fail (without changing <ref>) if the current user is
+unable to create a new log file, append to the existing log file
+or does not have committer information available.
+
 Author
 ------
 Written by Linus Torvalds <torvalds@osdl.org>.
diff --git a/cache.h b/cache.h
index b1300cd..82adbba 100644
--- a/cache.h
+++ b/cache.h
@@ -171,6 +171,7 @@ extern void rollback_index_file(struct c
 extern int trust_executable_bit;
 extern int assume_unchanged;
 extern int prefer_symlink_refs;
+extern int log_all_ref_updates;
 extern int warn_ambiguous_refs;
 extern int diff_rename_limit_default;
 extern int shared_repository;
diff --git a/config.c b/config.c
index 0f518c9..fa6db4b 100644
--- a/config.c
+++ b/config.c
@@ -232,6 +232,11 @@ int git_default_config(const char *var, 
 		return 0;
 	}
 
+	if (!strcmp(var, "core.logallrefupdates")) {
+		log_all_ref_updates = git_config_bool(var, value);
+		return 0;
+	}
+
 	if (!strcmp(var, "core.warnambiguousrefs")) {
 		warn_ambiguous_refs = git_config_bool(var, value);
 		return 0;
diff --git a/environment.c b/environment.c
index 444c99e..2e79eab 100644
--- a/environment.c
+++ b/environment.c
@@ -14,6 +14,7 @@ char git_default_name[MAX_GITNAME];
 int trust_executable_bit = 1;
 int assume_unchanged = 0;
 int prefer_symlink_refs = 0;
+int log_all_ref_updates = 0;
 int warn_ambiguous_refs = 1;
 int repository_format_version = 0;
 char git_commit_encoding[MAX_ENCODING_LENGTH] = "utf-8";
diff --git a/refs.c b/refs.c
index 275b914..691a8ba 100644
--- a/refs.c
+++ b/refs.c
@@ -375,3 +375,53 @@ int write_ref_sha1_unlocked(const char *
 	free(lock_filename);
 	return retval;
 }
+
+int log_ref_update(const char *ref, const unsigned char *currsha1, const unsigned char *newsha1, const char *msg)
+{
+	char *logpath, *logrec;
+	const char *comitter;
+	unsigned maxlen, len;
+	int oflags, logfd, written;
+
+	oflags = O_APPEND | O_WRONLY;
+	logpath = git_path("logs/%s", ref);
+	if (log_all_ref_updates) {
+		if (safe_create_leading_directories(logpath) < 0)
+			return error("Unable to create all of %s", logpath);
+		oflags |= O_CREAT;
+	}
+
+	logfd = open(logpath, oflags, 0666);
+	if (logfd < 0) {
+		if (!log_all_ref_updates && errno == ENOENT)
+			return 0;
+		return error("Unable to append to %s: %s", logpath, strerror(errno));
+	}
+
+	setup_ident();
+	comitter = git_committer_info(1);
+	if (msg) {
+		maxlen = strlen(comitter) + strlen(msg) + 2*40 + 5;
+		logrec = xmalloc(maxlen);
+		len = snprintf(logrec, maxlen, "%s %s %s\t%s\n",
+			sha1_to_hex(currsha1),
+			sha1_to_hex(newsha1),
+			comitter,
+			msg);
+	} else {
+		maxlen = strlen(comitter) + 2*40 + 4;
+		logrec = xmalloc(maxlen);
+		len = snprintf(logrec, maxlen, "%s %s %s\n",
+			sha1_to_hex(currsha1),
+			sha1_to_hex(newsha1),
+			comitter);
+	}
+	if (len >= maxlen)
+		return error("Internal error formatting log record.");
+	written = write(logfd, logrec, len);
+	if (written != len)
+		return error("Unable to append to %s", logpath);
+	free(logrec);
+	close(logfd);
+	return 0;
+}
diff --git a/refs.h b/refs.h
index 2625596..9345942 100644
--- a/refs.h
+++ b/refs.h
@@ -25,4 +25,7 @@ extern int write_ref_sha1_unlocked(const
 /** Returns 0 if target has the right format for a ref. **/
 extern int check_ref_format(const char *target);
 
+/** If logging is enabled logs the change made to the ref. **/
+extern int log_ref_update(const char *ref, const unsigned char *currsha1, const unsigned char *newsha1, const char *msg);
+
 #endif /* REFS_H */
diff --git a/update-ref.c b/update-ref.c
index fd48742..95940dd 100644
--- a/update-ref.c
+++ b/update-ref.c
@@ -1,7 +1,8 @@
 #include "cache.h"
 #include "refs.h"
 
-static const char git_update_ref_usage[] = "git-update-ref <refname> <value> [<oldval>]";
+static const char git_update_ref_usage[] =
+"git-update-ref <refname> <value> [<oldval>] [-m <reason>]";
 
 static int re_verify(const char *path, unsigned char *oldsha1, unsigned char *currsha1)
 {
@@ -19,26 +20,50 @@ static int re_verify(const char *path, u
 int main(int argc, char **argv)
 {
 	char *hex;
-	const char *refname, *value, *oldval, *path;
+	const char *refname=NULL, *value=NULL, *oldval=NULL, *msg=NULL, *path;
 	char *lockpath;
 	unsigned char sha1[20], oldsha1[20], currsha1[20];
-	int fd, written;
+	int i, fd, written, pfxlen;
 
 	setup_git_directory();
 	git_config(git_default_config);
-	if (argc < 3 || argc > 4)
+
+	for (i = 1; i < argc; i++) {
+		if (!strcmp("-m", argv[i])) {
+			if (i+1 >= argc)
+				usage(git_update_ref_usage);
+			msg = argv[++i];
+			if (!*msg)
+				die("Refusing to perform update with empty message.");
+			if (strchr(msg, '\n'))
+				die("Refusing to perform update with \\n in message.");
+			continue;
+		}
+		if (!refname) {
+			refname = argv[i];
+			continue;
+		}
+		if (!value) {
+			value = argv[i];
+			continue;
+		}
+		if (!oldval) {
+			oldval = argv[i];
+			continue;
+		}
+	}
+	if (!refname || !value)
 		usage(git_update_ref_usage);
 
-	refname = argv[1];
-	value = argv[2];
-	oldval = argv[3];
 	if (get_sha1(value, sha1))
 		die("%s: not a valid SHA1", value);
 	memset(oldsha1, 0, 20);
 	if (oldval && get_sha1(oldval, oldsha1))
 		die("%s: not a valid old SHA1", oldval);
 
-	path = resolve_ref(git_path("%s", refname), currsha1, !!oldval);
+	path = git_path("%s", refname);
+	pfxlen = strlen(path) - strlen(refname);
+	path = resolve_ref(path, currsha1, !!oldval);
 	if (!path)
 		die("No such ref: %s", refname);
 
@@ -49,8 +74,13 @@ int main(int argc, char **argv)
 		if (!memcmp(oldsha1, sha1, 20))
 			exit(0);
 	}
+
+	/* No change in ref.  Don't do anything. */
+	if (!memcmp(currsha1, sha1, 20))
+		exit(0);
+
 	path = strdup(path);
-	lockpath = mkpath("%s.lock", path);
+	lockpath = strdup(mkpath("%s.lock", path));
 	if (safe_create_leading_directories(lockpath) < 0)
 		die("Unable to create all of %s", lockpath);
 
@@ -74,6 +104,11 @@ int main(int argc, char **argv)
 		die("Ref lock failed");
 	}
 
+	if (log_ref_update(path + pfxlen, currsha1, sha1, msg) < 0) {
+		unlink(lockpath);
+		die("Logging ref update failed, update aborted.");
+	}
+
 	/*
 	 * Finally, replace the old ref with the new one
 	 */
-- 
1.3.2.g7278

Re: Tracking branch history

[Shawn Pearce]

Shawn Pearce <spearce@spearce.org> wrote:
> Log ref updates to logs/refs/<ref>
> 
> If config parameter core.logAllRefUpdates is true or the log
> file already exists then append a line to ".git/logs/refs/<ref>"
> whenever git-update-ref <ref> is executed.  Each log line contains
> the following information:
> 
>   oldsha1 <SP> newsha1 <SP> committer <LF>
> 
> where committer is the current user, date, time and timezone in
> the standard GIT ident format.  If the caller is unable to append
> to the log file then git-update-ref will fail without updating <ref>.
> 
> An optional message may be included in the log line with the -m flag.

This is all well and good but its sort of useless without the diffcore
being able to lookup what SHA1 was valid on a given branch at a given
point in time.  :-)

I'm thinking about extending the 'extended SHA1' syntax to accept
a date (or date expression) as a suffix:

	HEAD@'2 hours ago'
	HEAD@'2006-04-20'
	HEAD@'2006-04-20 14:12'

etc... This would be merged into get_sha1 (sha1_name.c) so its
usable pretty much anywhere.  Does this seem reasonable?  If so
I'll work up a patch for it.

-- 
Shawn.

Re: Tracking branch history

[Junio C Hamano]

Shawn Pearce <spearce@spearce.org> writes:

> This is all well and good but its sort of useless without the diffcore
> being able to lookup what SHA1 was valid on a given branch at a given
> point in time.  :-)
>
> I'm thinking about extending the 'extended SHA1' syntax to accept
> a date (or date expression) as a suffix:
>
> 	HEAD@'2 hours ago'
> 	HEAD@'2006-04-20'
> 	HEAD@'2006-04-20 14:12'
>
> etc... This would be merged into get_sha1 (sha1_name.c) so its
> usable pretty much anywhere.  Does this seem reasonable?  If so
> I'll work up a patch for it.

HEAD?

Are you going to hook into symbolic-ref as well to track branch
switching?

Since there is no reverse pointer to tell which symbolic
reference is pointing at branch heads,and there are symbolic
references like refs/remotes/origin/HEAD that point at
refs/remotes/origin/master, detecting that such and such
symbolic refs are pointing at a branch that is advanced by a
call to update-ref and update the log for the symbolic refs that
point at it becomes rather expensive.
So probably you would need a separate log format that tracks
which concrete ref a symbolic ref was pointing at at any given
time and use that to keep track of them.

I personally doubt it is worth the trouble.  I switch branches
between master, next and the topics all the time, and never is
interested in which branch I happened to be on 30 minutes ago.

The time-warp format would make sense for individual branches,
like refs/heads/master, though.

sha1_name.c and sha1_file.c were supposed to be real core, but
get_sha1() is looking more and more Porcelainish these days, and
I do not have much problem with being able to say "tip of this
branch, two hours ago".

I am not sure about the syntax though.  We would want to be able
to say "start from the commit that was at the tip of 'master'
branch two days ago, grab its tree and look at arch/sparc64
directory", so things like

            "master@2006-05-14 14:12"
            "master@2006-05-14 14:12^{tree}"
            "master@two days ago:arch/sparc64"

would need to be supported.

Re: Tracking branch history

[Shawn Pearce]

Junio C Hamano <junkio@cox.net> wrote:
> Shawn Pearce <spearce@spearce.org> writes:
> 
> > This is all well and good but its sort of useless without the diffcore
> > being able to lookup what SHA1 was valid on a given branch at a given
> > point in time.  :-)
> >
> > I'm thinking about extending the 'extended SHA1' syntax to accept
> > a date (or date expression) as a suffix:
> >
> > 	HEAD@'2 hours ago'
> > 	HEAD@'2006-04-20'
> > 	HEAD@'2006-04-20 14:12'
> >
> > etc... This would be merged into get_sha1 (sha1_name.c) so its
> > usable pretty much anywhere.  Does this seem reasonable?  If so
> > I'll work up a patch for it.
> 
> HEAD?
> 
> Are you going to hook into symbolic-ref as well to track branch
> switching?

I hadn't planned on it.  I was going to resolve the symref HEAD
down to the real ref (e.g. refs/heads/sp/ref-log) and then do the
date range searching on the real branch.  I didn't think it was
interesting to track what HEAD is.  But I think it would be very
common for the user to use HEAD rather than their actual branch
ref names when forming an expression.
 
[snip]
> The time-warp format would make sense for individual branches,
> like refs/heads/master, though.
> 
> sha1_name.c and sha1_file.c were supposed to be real core, but
> get_sha1() is looking more and more Porcelainish these days, and
> I do not have much problem with being able to say "tip of this
> branch, two hours ago".
> 
> I am not sure about the syntax though.  We would want to be able
> to say "start from the commit that was at the tip of 'master'
> branch two days ago, grab its tree and look at arch/sparc64
> directory", so things like
> 
>             "master@2006-05-14 14:12"
>             "master@2006-05-14 14:12^{tree}"
>             "master@two days ago:arch/sparc64"
> 
> would need to be supported.

Yea, I realize that.  I'm currently looking at get_sha1_1 and how
I can put the date resolution in before the ^, ~ and :.  :-)

-- 
Shawn.

Re: Tracking branch history

[Shawn Pearce]

Shawn Pearce <spearce@spearce.org> wrote:
> Junio C Hamano <junkio@cox.net> wrote:
> > Shawn Pearce <spearce@spearce.org> writes:
> > 
> > > This is all well and good but its sort of useless without the diffcore
> > > being able to lookup what SHA1 was valid on a given branch at a given
> > > point in time.  :-)
> > >
> > > I'm thinking about extending the 'extended SHA1' syntax to accept
> > > a date (or date expression) as a suffix:
> > >
> > > 	HEAD@'2 hours ago'
> > > 	HEAD@'2006-04-20'
> > > 	HEAD@'2006-04-20 14:12'
> > >
> > > etc... This would be merged into get_sha1 (sha1_name.c) so its
> > > usable pretty much anywhere.  Does this seem reasonable?  If so
> > > I'll work up a patch for it.

This is a preliminary patch for this syntax.  I haven't handled the
absolute date parsing yet; I was hoping to use the same syntax
accepted by GIT_AUTHOR_DATE/GIT_COMMITTER_DATE but looking at the
code in date.c it wasn't going to be easily reused.  I'll work
on it more tomorrow, right now I have to go do my day job to pay
the rent.  :-)

Hmm... A quick look at date.c indicates I should be able to clean up
this parse_date_spec function quite a bit by using code from date.c.
I'll look at it more later.

-- >8 -
Support 'master@2 hours ago' syntax

Extended sha1 expressions may now include date specifications
which indicate a point in time within the local repository's
history.  If the ref indicated to the left of '@' has a log in
$GIT_DIR/logs/<ref> then the value of the ref at the time indicated
by the specification is obtained from the ref's log.

---

 Documentation/git-rev-parse.txt |    6 ++
 refs.c                          |   52 ++++++++++++++
 refs.h                          |    3 +
 sha1_name.c                     |  145 ++++++++++++++++++++++++++++++++++-----
 4 files changed, 189 insertions(+), 17 deletions(-)

1f16364fd8cadb6cdeb0b14a6f5439f02b578924
diff --git a/Documentation/git-rev-parse.txt b/Documentation/git-rev-parse.txt
index ab896fc..df308c3 100644
--- a/Documentation/git-rev-parse.txt
+++ b/Documentation/git-rev-parse.txt
@@ -124,6 +124,12 @@ syntax.
   happen to have both heads/master and tags/master, you can
   explicitly say 'heads/master' to tell git which one you mean.
 
+* A suffix '@' followed by a date specification such as 'yesterday'
+  (24 hours ago) or '1 month 2 weeks 3 days 1 hour 1 second ago'
+  to specify the value of the ref at a prior point in time.
+  This suffix may only be used immediately following a ref name
+  and the ref must have an existing log ($GIT_DIR/logs/<ref>).
+
 * A suffix '{caret}' to a revision parameter means the first parent of
   that commit object.  '{caret}<n>' means the <n>th parent (i.e.
   'rev{caret}'
diff --git a/refs.c b/refs.c
index a50ea8f..da009ac 100644
--- a/refs.c
+++ b/refs.c
@@ -440,3 +440,55 @@ int log_ref_update(const char *ref, cons
 	close(logfd);
 	return 0;
 }
+
+int read_ref_at(const char *ref, unsigned long at_time, unsigned char *sha1)
+{
+	const char *logfile, *logdata, *rec, *c;
+	char *tz_c;
+	int logfd, tz;
+	struct stat st;
+	unsigned long date;
+	unsigned char oldsha1[20];
+
+	logfile = git_path("logs/%s", ref);
+	logfd = open(logfile, O_RDONLY, 0);
+	if (logfd < 0)
+		die("Unable to read log %s: %s", logfile, strerror(errno));
+	fstat(logfd, &st);
+	if (!st.st_size)
+		die("Log %s is empty.", logfile);
+	logdata = mmap(NULL, st.st_size, PROT_READ, MAP_PRIVATE, logfd, 0);
+	close(logfd);
+
+	rec = logdata + st.st_size;
+	while (logdata < rec) {
+		if (logdata < rec && *(rec-1) == '\n')
+			rec--;
+		while (logdata < rec && *(rec-1) != '\n')
+			rec--;
+		c = rec;
+		while (*c++ != '>')
+			/* nada */;
+		date = strtoul(c, NULL, 10);
+		if (date <= at_time) {
+			if (get_sha1_hex(rec, oldsha1))
+				die("Log %s is corrupt.", logfile);
+			if (get_sha1_hex(rec + 41, sha1))
+				die("Log %s is corrupt.", logfile);
+			munmap((void*)logdata, st.st_size);
+			return 0;
+		}
+	}
+
+	c = logdata;
+	while (*c++ != '>')
+		/* nada */;
+	date = strtoul(c, &tz_c, 10);
+	tz = strtoul(tz_c, NULL, 10);
+	if (get_sha1_hex(logdata, sha1))
+		die("Log %s is corrupt.", logfile);
+	munmap((void*)logdata, st.st_size);
+	fprintf(stderr, "warning: Log %s only goes back to %s.\n",
+		logfile, show_rfc2822_date(date, tz));
+	return 0;
+}
diff --git a/refs.h b/refs.h
index de3cb92..4831cdb 100644
--- a/refs.h
+++ b/refs.h
@@ -31,4 +31,7 @@ extern int check_ref_format(const char *
 /** If logging is enabled logs the change made to the ref. **/
 extern int log_ref_update(const char *ref, const unsigned char *currsha1, const unsigned char *newsha1, const char *msg);
 
+/** Reads log for the value of ref during at_time. **/
+extern int read_ref_at(const char *ref, unsigned long at_time, unsigned char *sha1);
+
 #endif /* REFS_H */
diff --git a/sha1_name.c b/sha1_name.c
index dc68355..5f33aea 100644
--- a/sha1_name.c
+++ b/sha1_name.c
@@ -4,6 +4,7 @@ #include "commit.h"
 #include "tree.h"
 #include "blob.h"
 #include "tree-walk.h"
+#include "refs.h"
 
 static int find_short_object_filename(int len, const char *name, unsigned char *sha1)
 {
@@ -234,6 +235,98 @@ static int ambiguous_path(const char *pa
 	return slash;
 }
 
+static unsigned long parse_date_spec(const char *str, int len)
+{
+	long delta;
+	time_t now;
+
+	time(&now);
+	if (len == 9 && !strncasecmp("yesterday", str, 9))
+		return now - 24 * 60 * 60;
+	if (len > 4 && !strncasecmp(" ago", str + (len - 4), 4)) {
+		len -= 4;
+		while (len) {
+			if (len > 2 && !strncasecmp("a ", str, 2)) {
+				delta = 1;
+				len -= 2;
+				str += 2;
+			}
+			else if (len > 3 && !strncasecmp("an ", str, 3)) {
+				delta = 1;
+				len -= 2;
+				str += 2;
+			} else {
+				delta = 0;
+				while (len && isdigit(*str)) {
+					if (delta)
+						delta *= 10;
+					delta += *str++ - '0';
+					len--;
+				}
+				if (!delta)
+					return (time_t)-1;
+				while (len && isspace(*str)) {
+					str++;
+					len--;
+				}
+			}
+
+			if (len >= 5 && !strncasecmp("month", str, 5)) {
+				len -= 5;
+				str += 5;
+				now -= 30 * 24 * 60 * 60 * delta;
+			}
+			else if (len >= 4 && !strncasecmp("week", str, 4)) {
+				len -= 4;
+				str += 4;
+				now -= 7 * 24 * 60 * 60 * delta;
+			}
+			else if (len >= 3 && !strncasecmp("day", str, 3)) {
+				len -= 3;
+				str += 3;
+				now -= 24 * 60 * 60 * delta;
+			}
+			else if (len >= 4 && !strncasecmp("hour", str, 4)) {
+				len -= 4;
+				str += 4;
+				now -= 60 * 60 * delta;
+			}
+			else if (len >= 6 && !strncasecmp("minute", str, 6)) {
+				len -= 6;
+				str += 6;
+				now -= 60 * delta;
+			}
+			else if (len >= 3 && !strncasecmp("min", str, 3)) {
+				len -= 3;
+				str += 3;
+				now -= 60 * delta;
+			}
+			else if (len >= 6 && !strncasecmp("second", str, 6)) {
+				len -= 6;
+				str += 6;
+				now -= delta;
+			}
+			else if (len >= 3 && !strncasecmp("sec", str, 3)) {
+				len -= 3;
+				str += 3;
+				now -= delta;
+			}
+
+			if (len && *str == 's') {
+				len -= 1;
+				str += 1;
+			}
+
+			while (len && isspace(*str)) {
+				str++;
+				len--;
+			}
+		}
+		return now;
+	}
+	return (time_t)-1;
+}
+
 static int get_sha1_basic(const char *str, int len, unsigned char *sha1)
 {
 	static const char *fmt[] = {
@@ -245,36 +338,54 @@ static int get_sha1_basic(const char *st
 		"refs/remotes/%.*s/HEAD",
 		NULL
 	};
-	const char **p;
-	const char *warning = "warning: refname '%.*s' is ambiguous.\n";
-	char *pathname;
-	int already_found = 0;
+	static const char *warning = "warning: refname '%.*s' is ambiguous.\n";
+	const char **p, *pathname;
+	char *real_path = NULL;
+	int refs_found = 0, at_mark;
+	unsigned long at_time = (unsigned long)-1;
 	unsigned char *this_result;
 	unsigned char sha1_from_ref[20];
 
 	if (len == 40 && !get_sha1_hex(str, sha1))
 		return 0;
 
+	/* At a given period of time? "@2 hours ago" */
+	for (at_mark = 1; at_mark < len; at_mark++) {
+		if (str[at_mark] == '@') {
+			at_time = parse_date_spec(str + at_mark + 1, len - at_mark - 1);
+			if (at_time == (unsigned long)-1)
+				die("Invalid date spec after @ in '%.*s'", len, str);
+			len = at_mark;
+		}
+	}
+
 	/* Accept only unambiguous ref paths. */
 	if (ambiguous_path(str, len))
 		return -1;
 
 	for (p = fmt; *p; p++) {
-		this_result = already_found ? sha1_from_ref : sha1;
-		pathname = git_path(*p, len, str);
-		if (!read_ref(pathname, this_result)) {
-			if (warn_ambiguous_refs) {
-				if (already_found)
-					fprintf(stderr, warning, len, str);
-				already_found++;
-			}
-			else
-				return 0;
+		this_result = refs_found ? sha1_from_ref : sha1;
+		pathname = resolve_ref(git_path(*p, len, str), this_result, 1);
+		if (pathname) {
+			if (!refs_found++)
+				real_path = strdup(pathname);
+			if (!warn_ambiguous_refs)
+				break;
 		}
 	}
-	if (already_found)
-		return 0;
-	return -1;
+
+	if (!refs_found)
+		return -1;
+
+	if (warn_ambiguous_refs && refs_found > 1)
+		fprintf(stderr, warning, len, str);
+
+	if (at_time != (unsigned long)-1) {
+		read_ref_at(real_path + strlen(git_path(".")) - 1, at_time, sha1);
+	}
+
+	free(real_path);
+	return 0;
 }
 
 static int get_sha1_1(const char *name, int len, unsigned char *sha1);
-- 
1.3.2.g7278

Re: Tracking branch history

[Linus Torvalds]

On Sat, 13 May 2006, Shawn Pearce wrote:
>
> +
> +	/*
> +	 * If logging is required make sure we can append to the log.
> +	 */
> +	logpath = strdup(git_path("logs/%s", path + pfxlen));

I don't think you need the strdup().

I also think you might as well just let the logging silently fail, but 
hey, that's up to you.

		Linus

Re: Tracking branch history

[Elrond]

Daniel Barkalow <barkalow <at> iabervon.org> writes:

> 
> One feature that might make git more intuitive to people is if we were to 
> additionally track the history of what commit was the head of each branch 
> over time. This is only vaguely related to the history of the content, but 
> it's well-defined and sometimes significant.
> 
> E.g., if you know that two weeks ago, what you had worked, but it doesn't 
> work now, you can use git-bisect to figure out what happened, but first 
> you have to figure out what commit it was that you were using two weeks 
> ago. Two weeks ago, we had that information, but we didn't keep it.

On a related issue:

Looking at a commit:
   commit id-commit
   parent id-1
   parent id-2
   parent id-3

       Merge branch 'branch-2', 'branch-3'

One can tell the name of the branches for id-2 and id-3 (branch-2, 3),
but one can't tell the name of id-1.

At the time, those branches were not yet merged, this information was
available easily, even remotely via git-clone.

Re: Tracking branch history

[Junio C Hamano]

Elrond <elrond+kernel.org@samba-tng.org> writes:

> Daniel Barkalow <barkalow <at> iabervon.org> writes:
>
>> 
>> One feature that might make git more intuitive to people is if we were to 
>> additionally track the history of what commit was the head of each branch 
>> over time. This is only vaguely related to the history of the content, but 
>> it's well-defined and sometimes significant.
>> 
>> E.g., if you know that two weeks ago, what you had worked, but it doesn't 
>> work now, you can use git-bisect to figure out what happened, but first 
>> you have to figure out what commit it was that you were using two weeks 
>> ago. Two weeks ago, we had that information, but we didn't keep it.
>
> On a related issue:
>
> Looking at a commit:
>    commit id-commit
>    parent id-1
>    parent id-2
>    parent id-3
>
>        Merge branch 'branch-2', 'branch-3'
>
> One can tell the name of the branches for id-2 and id-3 (branch-2, 3),
> but one can't tell the name of id-1.

That's deliberate.  If you are merging into a branch other than
"master", the message would say:

        commit ea892b27b15fbc46a3bb3ad2ddce737dc6590ae5
        Merge: 7278a29... 8d48ad6...
        Author: Junio C Hamano <junkio@cox.net>
        Date:   Sat May 13 18:49:54 2006 -0700

            Merge branch 'lt/config' into next

            * lt/config:
              git config syntax updates
              Another config file parsing fix.
              checkout: use --aggressive when running a 3-way merge (-m).
              Fix git-pack-objects for 64-bit platforms
              fix diff-delta bad memory access

The point is to keep the punch line as short and meaningful for
the most common case.