Re: git and time

Re: git and time

[apodtele]

Matthew wrote:
> No. I merely think git should try harder to ensure that commit order is consistent
> with time order

Matthew doesn't seem to grasp the idea that commit order is
conceptually MORE fundamental than time, NOT LESS. Hence, the time
shall not dictate the order for git. If anything, git may try harder
to ensure that time is consistent with order, not vise versa. :)

Re: git and time

[apodtele]

On Thu, 28 Sep 2006, Matthew L Foster wrote:
>
> It should be possible to export git data, through say a web interface,
> in a such a way that local time order is consistent with commit order.

You are proposing to sort your genealogy tree by your ancestor's birthdays...
Yeah, that is really cool and useful too! Look ma! The order is pretty random!

Re: git and time

[Matthew L Foster]

--- Tom Prince <tom.prince@ualberta.net> wrote:

> On Thu, Sep 28, 2006 at 09:55:08AM -0700, Matthew L Foster wrote:
> > 
> > --- Rogan Dawes <discard@dawes.za.net> wrote:
> > 
> > > I just don't think that any of the kernel developers feel the need to 
> > > police any one else's clocks . . . they're more interested in the 
> > > contents of the patch.
> > 
> > I am not saying git should "police any one else's clocks", I am saying git should be designed
> or
> > configured in such a way, using local time, that it obviates the current reliance on everyone
> > else's clock being set correctly. 
> > 
> > -Matt
> 
> The point is, it DOES NOT rely on anyone's clock being set correctly.

gitweb.cgi does. It should be possible to export git data, through say a web interface, in a such
a way that local time order is consistent with commit order. 

[responding to Linus' mail here:] I understand that git doesn't care about time internally as it
treats it as a random number for people to care about or not on their own, I am only saying that
local time is more likely to be cared about than disparate remote creation times. 

-Matt


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[apodtele]

Shall we summarize? Time is a very important concept in physics.
Yet time is pretty much useless in math and git for that matter.

GIT == Git Ignores Time

Re: git and time

[Linus Torvalds]

On Wed, 27 Sep 2006, apodtele wrote:
>
> Shall we summarize? Time is a very important concept in physics.

Actually, even in physics (or very _much_ in physics), time doesn't 
follow the nice procession that Matthew is kind of asking for.

In physics, particles can go backwards in time or forwards in time (the 
_likelihood_ that a photon moves at exactly the speed of light is higher, 
but on a quantum scale, strange things happen). 

And everybody should by now know what relativity says: time does not 
impose an "ordering" of events outside of the so-called "cone of light". 
There is only an ordering imposed by _causality_, not by time.

That, btw, is very similar to git. The only _true_ ordering in git is 
causality. Time itself tends to have certain properties that makes it 
_look_ like it is about causality, but in real life there is just a strong 
correlation.

Nature is sometimes stranger than our everyday experiences would have us 
believe. And "time" is a hell of a lot more complicated than just a global 
one-dimensional entity, steadily ticking away.

			Linus

git and time

[Matthew L Foster]

After seeing how git currently accepts a remote repository's timestamp it occurred to me that
git should probably instead prefer the time a particular changeset was committed to _this_
repository. Perhaps I don't know enough about git but it seems to me the important information is
when a particular changeset was committed to this repository, all other remote/sub/parent
repositories' timestamps are secondary (or at least should be tracked separately).

-Matt



__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Johannes Schindelin]

Hi,

On Tue, 26 Sep 2006, Matthew L Foster wrote:

> After seeing how git currently accepts a remote repository's timestamp 
> it occurred to me that git should probably instead prefer the time a 
> particular changeset was committed to _this_ repository.

Git accepts it, but does not rely on it. Instead, it relies on 
parent-child relations.

Hth,
Dscho

Re: git and time

[Jakub Narebski]

Matthew L Foster wrote:

> After seeing how git currently accepts a remote repository's timestamp
> it occurred to me that git should probably instead prefer the time
> a particular changeset was committed to _this_ repository. Perhaps
> I don't know enough about git but it seems to me the important
> information is  when a particular changeset was committed to this
> repository, all other remote/sub/parent repositories' timestamps
> are secondary (or at least should be tracked separately). 

First, the information you want is contained in reflog. Dates the head tip
got the specified value.

Second, git cannot rewrite commits (and commits contain timestamp) when
fetching commit from remote repository for performance reasons.

Third, git uses timestams as heuristics, but relies on parent information.
-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: git and time

[Jeff King]

On Tue, Sep 26, 2006 at 04:23:16PM -0700, Matthew L Foster wrote:

> After seeing how git currently accepts a remote repository's timestamp
> it occurred to me that git should probably instead prefer the time a
> particular changeset was committed to _this_ repository. Perhaps I

If you fetch a commit from a remote repository, it does not get
"committed" to the local repository. It is simply copied. Keep in mind
that the act of making a commit means making an immutable SHA1 object.
If, when you fetched that commit, you changed some aspect of it (like
the timestamp), it would cease to have the same SHA1, and thus the DAG
of your history would differ from the remote end.

During operations where the original commit isn't preserved (e.g.,
applying patches from an email), git applies the current timestamp as
the committer timestamp (but uses the email date as the author
timestamp).

> don't know enough about git but it seems to me the important
> information is when a particular changeset was committed to this
> repository, all other remote/sub/parent repositories' timestamps are
> secondary (or at least should be tracked separately).

That information is not tracked in the commit objects (because they are
never "committed" in the local repository, only copied); however, Shawn's
reflog implementation gives some indication of when each ref changed,
which shows when some (but not all) commits made it into the local
repository.

Keep in mind that git doesn't really CARE about timestamps to do most
operations; it operates on the graph created by parentage. Think of the
timestamps more as comments; when a commit is created, we comment who
did it and when, both accordinging to their local information.

-Peff

PS Nit: Git doesn't work with changesets, it works with snapshots,
building a directed graph of snapshots. Maybe that is the source of your
confusion?

Re: git and time

[Matthew L Foster]

> Keep in mind that git doesn't really CARE about timestamps to do most
> operations; it operates on the graph created by parentage. Think of the
> timestamps more as comments; when a commit is created, we comment who
> did it and when, both accordinging to their local information.
> 
> -Peff
> 
> PS Nit: Git doesn't work with changesets, it works with snapshots,
> building a directed graph of snapshots. Maybe that is the source of your
> confusion

It's true I don't know much about git, what is the difference between a changeset and a snapshot?
Are you saying timestamps should be tracked separately or tracked by an scm system built on top of
git? Does/should git care about the when of a snapshot?

Perhaps my question is directed more toward gitweb.cgi, it seems to me the timestamp of when a
snapshot was merged into this repository should somehow be tracked and that is what gitweb.cgi
should default to display. For example, if someone wants to know if security bugfix X was merged
into linus' kernel tree they also want to know when that happened, don't they? 

-Matt


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Junio C Hamano]

Matthew L Foster <mfoster167@yahoo.com> writes:

>> PS Nit: Git doesn't work with changesets, it works with snapshots,
>> building a directed graph of snapshots. Maybe that is the source of your
>> confusion
>
> It's true I don't know much about git, what is the difference
> between a changeset and a snapshot?  Are you saying timestamps
> should be tracked separately or tracked by an scm system built
> on top of git? Does/should git care about the when of a
> snapshot?

I do not know what Jeff meant by snapshot vs changeset, so I
would not comment on this part.

> Perhaps my question is directed more toward gitweb.cgi, it
> seems to me the timestamp of when a snapshot was merged into
> this repository should somehow be tracked and that is what
> gitweb.cgi should default to display. For example, if someone
> wants to know if security bugfix X was merged into linus'
> kernel tree they also want to know when that happened, don't
> they?

Each commit object in git records two timestamps.  When the
author made that change, and when the change was made into a
commit object in _some_ repository.  I _think_ gitweb shows the
latter, but I haven't checked, so Jakub is CC'ed.

What you want to know, when a particular change has become part
of the history of one branch in one repository, is not something
a git commit object records.  Enough people wanted to know that
information, so ref-log was introduced.  When it is enabled on a
branch, ref-log records when the tip of the branch changed from
what commit to what other commit.  But it primarily is meant to
answer this question: "what commit was at the tip of this branch
at time T?"

So if Linus had enabled ref-log in his public repository, and if
gitweb knew to look at ref-log, then gitweb _could_ iterate over
ref-log records for the "master" branch of Linus's repository,
find the earliest one that makes the tip of the "master" branch
a descendant of that security fix X, and report the time of that
change.  It could certainly do that.

I have to warn you that this is fairly expensive, and also I
happen to know that Linus does not have ref-log enabled on his
public repository.

Having said that, I doubt the question "when did Linus's tree
saw this security fix X commit?" has much practical value.  For
one thing, the time he merges the side branch that has the
security fix and the time he pushes out the resulting mess^Wtree
out to the public repository is different.  After pushed out to
the public repository, it takes time for that to mirror out for
public view.  From the consumer's point of view, the time it
finishes mirroring out _is_ the only timestamp that matters, but
that mirroing happens outside of git so even if ref-log showed
timestamp from the repository Linus pushes to, it would not
reflect the time the general public first saw "Linus's official
version that contained that fix".

What people would often want to know is "Does v2.6.18-rc5
include that fix?" which is a similar but different question.
This is something gitweb _could_ answer without using ref-log,
and gitk already knows how to answer it.

I somehow thought that it was possible to get "the latest tag
that precedes this commit" (aka "git describe") for each commit
by visiting its commitdiff_plain page, but I do not see it now.
Can somebody tell me if I am hallucinating?

Re: git and time

[Andreas Ericsson]

Junio C Hamano wrote:
> Matthew L Foster <mfoster167@yahoo.com> writes:
> 
>>> PS Nit: Git doesn't work with changesets, it works with snapshots,
>>> building a directed graph of snapshots. Maybe that is the source of your
>>> confusion
>> It's true I don't know much about git, what is the difference
>> between a changeset and a snapshot?  Are you saying timestamps
>> should be tracked separately or tracked by an scm system built
>> on top of git? Does/should git care about the when of a
>> snapshot?
> 
> I do not know what Jeff meant by snapshot vs changeset, so I
> would not comment on this part.
> 

Me neither, but I've seen this distinction before on the mailing-list.

To my mind, a changeset is the patch that brings some form of data from 
one state (snapshot) to another. In this respect, git is certainly both 
snapshot- and changeset-based.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: git and time

[Jeff King]

On Wed, Sep 27, 2006 at 10:42:52AM +0200, Andreas Ericsson wrote:

> >>It's true I don't know much about git, what is the difference
> >>between a changeset and a snapshot?  Are you saying timestamps
> >
> >I do not know what Jeff meant by snapshot vs changeset, so I
> >would not comment on this part.
>
> Me neither, but I've seen this distinction before on the mailing-list.
> 
> To my mind, a changeset is the patch that brings some form of data from 
> one state (snapshot) to another. In this respect, git is certainly both 
> snapshot- and changeset-based.

I was talking specifically about the core data structure of git. The
commit object doesn't say "I'm based on commit X, and the deltas are Y."
It says "Here are the complete contents of the tree at this point, and
the previous complete contents were X."

Of course, git often shows changesets (patches, git-whatchanged, etc)
because that's what's useful to users. But the context of the discussion
was fetching commits to a repository. In that case it's important to
note that you're just grabbing the new state (albeit optimizing the
process by skipping things you have) and not "re-committing" changesets
(which is what the OP seemed to think was happening).

-Peff

Re: git and time

[Matthew L Foster]

> Each commit object in git records two timestamps.  When the
> author made that change, and when the change was made into a
> commit object in _some_ repository. 

Perhaps git should record three(+) timestamps, adding when the change was committed into this
repository? Last weekend there was a committ in Linus' kernel tree with a timestamp ~2 days into
the future, which could be a problem in a scenario involving when an important bug fix was
merged/published in Linus' kernel tree, situations like that should be impossible. How can git be
said to keep an accurate record of history if time is uncertain?

-Matt

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Andreas Ericsson]

Matthew L Foster wrote:
> How can git be
> said to keep an accurate record of history if time is uncertain?
> 

Because git doesn't care about timestamps. It stores them as comments 
(albeit auto-formatted comments) and relies on the dependency chain to 
provide history.

In the same way that contributors are expected to write clear and 
concise commit-messages, they are also expected to keep their system 
clocks somewhat in sync. Sometimes one or the other fails, and this is 
as inevitable as it can be annoying (although commit-messages along the 
line of "fixed some bugs causing some random crashes" for a commit that 
touches 2384 lines are indefinitely worse than a bad timestamp).

What's beautiful about git is that it's designed to present a correct 
history even if random-contributor-X's system clock is out of sync with 
the rest of the world, as it inevitably will be at one point or another. 
It handles content, and the order in which each piece of content was 
added/removed/mutilated/transformed into something else, and it does a 
good job at that.

All that aside though, would you rather have that fix pronto with a bad 
timestamp, or three days later when the contributor had time to set up 
ntp properly?

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: git and time

[Matthew L Foster]

> Because git doesn't care about timestamps. It stores them as comments 
> (albeit auto-formatted comments) and relies on the dependency chain to 
> provide history.

Ok, the word "history" in the context of git primarily means the order of changes not the when?
Would it be a conceptual or technical issue for git to directly track the local time of
merges/changesets?

-Matt

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Linus Torvalds]

On Wed, 27 Sep 2006, Matthew L Foster wrote:
> 
> Ok, the word "history" in the context of git primarily means the order of changes not the when?
> Would it be a conceptual or technical issue for git to directly track the local time of
> merges/changesets?

True merges _get_ tracked - they are commits too (they just have multiple 
parents).

But it's only the time the merge was done that gets tracked, not the time 
the merge was then pushed out to somebody else.

		Linus

Re: git and time

[Matthew L Foster]

--- Linus Torvalds <torvalds@osdl.org> wrote:

> > Ok, the word "history" in the context of git primarily means the order of changes not the
> when?
> > Would it be a conceptual or technical issue for git to directly track the local time of
> > merges/changesets?
> 
> True merges _get_ tracked - they are commits too (they just have multiple 
> parents).
> 
> But it's only the time the merge was done that gets tracked, not the time 
> the merge was then pushed out to somebody else.

What is the difference between a merge and a "merge then pushed out"? There are at least some
situations where a repo would prefer to know its local time of a merge or pulled in merge and
anyway a local repo probably should not in any way be dependent on nor _trust_ all remote repos
timestamps...?

-Matt


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Linus Torvalds]

On Wed, 27 Sep 2006, Matthew L Foster wrote:
> 
> What is the difference between a merge and a "merge then pushed out"? There are at least some
> situations where a repo would prefer to know its local time of a merge or pulled in merge and
> anyway a local repo probably should not in any way be dependent on nor _trust_ all remote repos
> timestamps...?

Look into the ref-logging. It's exactly what you ask for.

The fact is, in a distributed system, you can _never_ make sense of 
"time". Just live with it. That's basic "distributed programming 101", and 
it's the one thing every such course should start with on the very first 
day.

So in short, you _cannot_ depend on time in a distributed environment. 
Really. Stop even asking. Please.

You can ask when some local reference was changed, and we support that 
already, and I pointed you to how to enable it in a repository you care 
about. But it's _always_ going to be just about your local repository, the 
whole question doesn't make sense any other way.

And no, it's _never_ going to tag individual merges or commits, since the 
same merge or commit can show up at DIFFERENT times in different branches, 
even within the same local repository.

So as long as you continue to ask for "commit times", you cannot get what 
you ask for. The _only_ commit time that makes sense is the time ON THE 
MACHINE that the commit was made. That's the time that git already saves 
in the commit itself. And if you don't trust that timeframe, then tough 
luck.

Git itself doesn't trust it, because git knows better. But it's there.

		Linus

Re: git and time

[Matthew L Foster]

--- Linus Torvalds <torvalds@osdl.org> wrote:

> The fact is, in a distributed system, you can _never_ make sense of 
> "time". Just live with it. That's basic "distributed programming 101", and 
> it's the one thing every such course should start with on the very first 
> day.

I agree which is exactly why I think git should conceptually prefer a repo's _local_ time. 

Commit/merge times could be specific to each repo and not generally distributed?  
 
-Matt

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Linus Torvalds]

On Wed, 27 Sep 2006, Matthew L Foster wrote:
> 
> What is the difference between a merge and a "merge then pushed out"?

To answer just this technical question, which came up today:

 - I might do a merge at 8:30 in the morning, but since I do it on my 
   machine, and nobody else actually saw my merge.

 - at 9:30, somebody sends me a patch that I already have (through the 
   merge), and I reply saying "I got this already", but I realize I 
   haven't pushed it out.

 - so I replicate my home machine tree to the one on master.kernel.org, 
   and now others can see it.

When did the merge happen? It happened at 8:30 on my machine, and that's 
what is recorded. End of story. No ifs, buts, maybes about it. That's the 
only time you can _ever_ see for that merge.

When did everybody else see it? It only became _visible_ in the git 
archive on kernel.org at 9:30, when I pushed out _all_ the merges that I 
had done (some of them at 8:30, others at 8:00, yet others perhaps at some 
other time).

So all those different merges, which were done at different times, only 
became visible in another git repository all at the same time, at 9:30.

There's _zero_ commits or merges that themselves happened at 9:30. There's 
no development at all that happened then. The only actual thing that 
changed at 9:30 was that the _reference_ that made all those old changes 
visible finally made it to the repository at kernel.org.

This is why you can use the ref-logging code to say "Ok, how did my 
repository look at 9:25 vs 9:35" and you could see the difference. BUT NO 
ACTUAL COMMIT OR MERGE WILL EVER HAVE THAT TIMESTAMP. That's purely a 
"when did it show up" question, and the whole question only makes sense 
within a particular repository (ie the answers are different: on _my_ 
machine, a merge showed up on 8:30 when it was made. On other machines, 
the time will always be different).

		Linus

Re: git and time

[Matthew L Foster]

--- Linus Torvalds <torvalds@osdl.org> wrote:

>  - so I replicate my home machine tree to the one on master.kernel.org, 
>    and now others can see it.
> 
> When did the merge happen? It happened at 8:30 on my machine, and that's 
> what is recorded. End of story. No ifs, buts, maybes about it. That's the 
> only time you can _ever_ see for that merge.

Ok, so it's more complex because of the workflow issue of delayed/pseudo mirroring/replication
between private and public repos? This cloning/replication is not done through git? Are you saying
it's impossible for master.kernel.org's git to track the local time of each
commit/merge/replication? Perhaps replication time is precisely what should/could be tracked
(locally)? 

>From an integrity or at least gitweb.cgi's viewpoint it seems very important to me that commit
order also be per repo consistent with time order.

-Matt

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Linus Torvalds]

On Wed, 27 Sep 2006, Matthew L Foster wrote:
> > When did the merge happen? It happened at 8:30 on my machine, and that's 
> > what is recorded. End of story. No ifs, buts, maybes about it. That's the 
> > only time you can _ever_ see for that merge.
> 
> Ok, so it's more complex because of the workflow issue of delayed/pseudo mirroring/replication
> between private and public repos? This cloning/replication is not done through git?

No, it very much happened with git.

But git will _refuse_ to rewrite history. That means that if a commit says 
it happened at 8:30AM on machine X, git will _not_ rewrite history to say 
that it happened at 9:30AM on machine Y just because that's when it made 
it to that machine.

> Are you saying it's impossible for master.kernel.org's git to track the 
> local time of each commit/merge/replication?

No, I'm saying that that would be _lying_.

The actual action happened at 8:30. And git tracks only truth. It doesn't 
rewrite the truth afterwards.

		Linus

Re: git and time

[Matthew L Foster]

--- Linus Torvalds <torvalds@osdl.org> wrote:

> The actual action happened at 8:30. And git tracks only truth. It doesn't 
> rewrite the truth afterward.

So the separate action of replication is not tracked? Replication/sub merges are denied the
possibility of "truth"?

To be clear I think there are actually two separate though semi-related issues:

- 1. release/replication time of a mirrored private --> public repo

- 2. A repo's commit order being inconsistent with local time order

My questions are primarily focused on #2. Last weekend even your private repo's commit order was
out of sync with your local time order because a remote git server's time was grossly
misconfigured, right? Integrity wise that shouldn't happen. I am merely asking if tracking
commits/merges using local repo time could solve both issues? If the local merge time information
is already available in the ref-log then gitweb.cgi might only need to be made aware of it.

-Matt


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Linus Torvalds]

On Wed, 27 Sep 2006, Matthew L Foster wrote:
>
> So the separate action of replication is not tracked?

Correct. Replication without changes is a no-op.

> Replication/sub merges are denied the possibility of "truth"?

No, it's actually much deeper than that.

To git, pure replication simply isn't an action at all, so trying to track 
it would be like trying to track all the voices in my head - something 
that doesn't exist. It wouldn't be "truth", it would be insanity.

And the thing is, _not_ tracking it is really fundamental. If you actually 
track the issue of copying a git repository, you'd end up in a technically 
untenable and insane situation. You could never "merge" two git trees ever 
again without going into an infinite bouncing back-and-forth of "A merged 
the changes from B" and "B merged the fact that A merged the changes from 
B" and "A merged the fact that B merged the fact that A merged the changes 
from B" and so on ad infinitum.

There's another reason too, namely that if you track where things came 
from and when, suddenly it matters whether you cloned from the _original_ 
repository or from somewhere else. And that's also fundamnetally wrong, 
since I don't actually want to give _anybody_ access to the actual 
original repository on my machine, so everything always has to go through 
an intermediate repository. If we tracked that, we'd just confuse 
everything, and it wouldn't be seamless any more.

There's one final reason, namely that I wanted to design git to just track 
_contents_. So the design philosophy is very much against tracking exactly 
which repository something has been in, since that has nothing to do with 
the deeper issue of what you are actually tracking.

		Linus

Re: git and time

[Shawn Pearce]

Linus Torvalds <torvalds@osdl.org> wrote:
> 
> 
> On Wed, 27 Sep 2006, Matthew L Foster wrote:
> >
> > So the separate action of replication is not tracked?
> 
> Correct. Replication without changes is a no-op.

I think it could be reasonably argued that knowing when the head(s)
of your public repository on a mirror of kernel.org changed to a
given value is useful.  Especially when there is a mirroring lag.

-- 
Shawn.

Re: git and time

[Linus Torvalds]

On Wed, 27 Sep 2006, Linus Torvalds wrote:
> 
> No, it's actually much deeper than that.
> 
> To git, pure replication simply isn't an action at all, so trying to track 
> it would be like trying to track all the voices in my head - something 
> that doesn't exist. It wouldn't be "truth", it would be insanity.

Another reason it's not an action at all: git in many ways does not 
actually care at all about the difference of a "local branch" and a 
"remote branch on another host".

Of course there _is_ a difference, in that the remote branch has to be 
fetched from that other repository, but it's possible (and some of the 
original design came from this) to share the repository data between 
multiple separate repositories. They can even be on different machines, if 
there is a networked filesystem in between (and, unlike most systems, the 
git database format should even be happy about _disconnected_ networked 
filesystems).

So git from the ground up is designed so that there is no real difference 
between "remote branch" and "local branch", other than simply physically 
where the data might be.

By that token, "cloning" a repository is pretty much by definition a 
no-op as far as the repository contents is concerned. In fact, if you use 
"git clone -l -s", all the cost is just checking out the new copy (so if 
you add "-n" to avoid checking out the new state, you basically have a 
zero-cost clone).

	[torvalds@g5 ~]$ time git clone -n -l -s v2.6/linux empty-clone

	real    0m0.129s
	user    0m0.084s
	sys     0m0.048s

That's it. I created a "clone" of the whole kernel repo in 0.129 seconds.

Exactly because cloning doesn't actually _do_ something (of course, 0.129 
seconds in git speak is pretty slow, so I suspect we are doing something 
stupid here with shell-script).

		Linus

Re: git and time

[Matthew L Foster]

Ignoring the separate issue of replication for a momment, can someone respond to my time integrity
question about whether a future version of git could trust/prefer its local time rather than a
remote/sub/parent (non replicated) git server's timestamp? How do we fix gitweb.cgi, ref-log? How
useful is gitweb.cgi if timestamps are all over the place? It does not make sense that commit
order is currently out of sync with time order in the main linux kernel tree git repo on
kernel.org. Why must each and every repo be dependent on time being set properly on all other git
servers? How useful is change history or commit order without some concept of (local) time order?

-Matt 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Jeff King]

On Wed, Sep 27, 2006 at 05:12:41PM -0700, Matthew L Foster wrote:

> Ignoring the separate issue of replication for a momment, can someone
> respond to my time integrity question about whether a future version
> of git could trust/prefer its local time rather than a
> remote/sub/parent (non replicated) git server's timestamp?

Yes, it could. But it would not involve rewriting the commit object or
adding a new commit object (for reasons I hope have been adequately
explained by Linus). Instead, an external mapping would be made between
a commit SHA1 and a timestamp (or a (branch,sha1) pair and a timestamp).

In fact, this is how reflogs work (but they make a map only when the
head changes, not marking each commit that enters the repo).

> How do we fix gitweb.cgi, ref-log?

To "fix" gitweb, keep a database in each local repository as described
above (either based on reflog, or one that is more comprehensive). Have
gitweb report that date rather than the timestamp contained in the
commit.

Nobody has created such a patch; you can always try and see what the
response is.

> How useful is gitweb.cgi if timestamps are all over the place? It does

Quite useful, according to many git users.

-Peff

Re: git and time

[Shawn Pearce]

Matthew L Foster <mfoster167@yahoo.com> wrote:
> 
> Ignoring the separate issue of replication for a momment, can someone respond to my time integrity
> question about whether a future version of git could trust/prefer its local time rather than a
> remote/sub/parent (non replicated) git server's timestamp? How do we fix gitweb.cgi, ref-log? How
> useful is gitweb.cgi if timestamps are all over the place? It does not make sense that commit
> order is currently out of sync with time order in the main linux kernel tree git repo on
> kernel.org. Why must each and every repo be dependent on time being set properly on all other git
> servers? How useful is change history or commit order without some concept of (local) time order?

Dependency order is all that matters.

It doesn't matter if K. Hacker makes a bug fix at 8 am his local
time or 3 days ago.  All that matters is that K. Hacker made it by
changing version A to version B.  Therefore commit B (containing
the bug fix) depends on commit A and only commit A (which may in
turn depend on commit A^, etc.).

That dependency in turn implies that you can't have bug fix B without
whatever feature/bug fix was A, and what that dependend on, etc.
Thus you know you have some particular chain of events as a result
of having B.  That's all that's interesting.


It _may_ matter to me that I received commit B (and maybe commit A)
at 3 pm my local time.  It may not.

In general I don't care too much about when a commit comes to me and
when it doesn't or when it was written, though I do look at

	git log next@{yesterday}..next

to see what Junio has pushed out recently.  Since I tend to
fetch only once per day (and usually around the same time of day)
this works reasonably well.

-- 
Shawn.

Re: git and time

[Matthew L Foster]

--- Shawn Pearce <spearce@spearce.org> wrote:

> Dependency order is all that matters.
> 
> It doesn't matter if K. Hacker makes a bug fix at 8 am his local
> time or 3 days ago.  All that matters is that K. Hacker made it by
> changing version A to version B.  Therefore commit B (containing
> the bug fix) depends on commit A and only commit A (which may in
> turn depend on commit A^, etc.).

>From a web display/generic notion of integrity perspective time order matters to me but it looks
like I am the only one. Keeping track of _local_ commit time would not add any dependencies. 

-Matt


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Linus Torvalds]

On Wed, 27 Sep 2006, Matthew L Foster wrote:
> 
> From a web display/generic notion of integrity perspective time order 
> matters to me but it looks like I am the only one. Keeping track of 
> _local_ commit time would not add any dependencies.

Actually, I think one problem here is that anybody why looks at just the 
gitweb interface may not realize how git works.

If you use gitk as your primary way of learning about a git problem, the 
whole time issue just goes away, because gitk shows the _real_ 
relationships so well.

I used gitk in all my initial explanations of git, because it turned a 
fairly abstract "here, let me explain how it works" into a "See? Look at 
this" kind of situation.

I think gitweb is great (in a way I have _never_ felt about any of the CVS 
web interfaces I have ever seen), but gitweb doesn't really explain how 
things work as well as gitk does.

		Linus

Re: git and time

[Andreas Ericsson]

Linus Torvalds wrote:
> 
> On Wed, 27 Sep 2006, Matthew L Foster wrote:
>> From a web display/generic notion of integrity perspective time order 
>> matters to me but it looks like I am the only one. Keeping track of 
>> _local_ commit time would not add any dependencies.
> 
> Actually, I think one problem here is that anybody why looks at just the 
> gitweb interface may not realize how git works.
> 
> If you use gitk as your primary way of learning about a git problem, the 
> whole time issue just goes away, because gitk shows the _real_ 
> relationships so well.
> 
> I used gitk in all my initial explanations of git, because it turned a 
> fairly abstract "here, let me explain how it works" into a "See? Look at 
> this" kind of situation.
> 

True that. I would have had a hard time introducing git as The SCM in 
the company if it hadn't been for gitk and qgit. They both let you just 
skip over 90% of that initial steep part of the learning curve and jump 
straight to work.

> I think gitweb is great (in a way I have _never_ felt about any of the CVS 
> web interfaces I have ever seen), but gitweb doesn't really explain how 
> things work as well as gitk does.
> 

Someone started hacking on a web-thingie to show the graph. Whatever 
happened to that? If it's no longer alive, perhaps we could add some 
qgit/gitk screenshots to the git wiki/docs so the people who spend most 
of their lives in browsers can get some visual aid in understanding the 
way git works.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: git and time

[Johannes Schindelin]

Hi,

On Fri, 29 Sep 2006, Andreas Ericsson wrote:

> Someone started hacking on a web-thingie to show the graph. Whatever 
> happened to that?

It is called git-browser, and was done by Artem Khodush. See 
http://straytree.com/.

I asked Artem what the plans are, since some features are not yet 
implemented, but he said that the thing is too slow, and he'll probably 
not continue to work on it.

Ciao,
Dscho

Re: git and time

[Andreas Ericsson]

Johannes Schindelin wrote:
> Hi,
> 
> On Fri, 29 Sep 2006, Andreas Ericsson wrote:
> 
>> Someone started hacking on a web-thingie to show the graph. Whatever 
>> happened to that?
> 
> It is called git-browser, and was done by Artem Khodush. See 
> http://straytree.com/.
> 
> I asked Artem what the plans are, since some features are not yet 
> implemented, but he said that the thing is too slow, and he'll probably 
> not continue to work on it.
> 

Ah well. I hope he keeps that page running though, and I hope the 
"time-is-importan" people find it.

For reference, it gives a crude (and indeed slow) picture of what gitk 
and qgit does.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: git and time

[Junio C Hamano]

Matthew L Foster <mfoster167@yahoo.com> writes:

> Ignoring the separate issue of replication for a momment, can
> someone respond to my time integrity question about whether a
> future version of git could trust/prefer its local time rather
> than a remote/sub/parent (non replicated) git server's
> timestamp?

This must be a trick question.  git does not trust _any_
timestamp, so there is no local "rather than" remote.

So, no there is no chance for that.

Having said that, what Porcelains like gitweb and gitk do are
completely different issue.  I already outlined how you can do
that using reflog last night, so I am not going to repeat it
here.  Also Linus stressed that why the "order that commits
appeared on a particular branch in one particular repository" is
strictly local issue today at least three times, and I hope
everybody understood it by now.

As Shawn said in a nearby thread, in a public and prominent
repository like kernel.org repository it may sometimes be
interesting and useful to know when each commit became available
on each branch.  I am reasonably sure that it would not however
make gitweb output easier to read to order its output by that
timestamp.   Linus pulls from subsystem maintainers all the time
and one pull may bring in dozens of commits, and they will get
the same timestamp if you did so.  Actually it is worse than
that.  He tends to batch, so he would have many such pulls and
patch applications in his private repository, perhaps over a few
hour, but the result will be pushed out to kernel.org with one
push operation.  To show the "truthful" time, your gitweb would
give the timestamp of that push operation for hundreds of
commits pushed out during that operation.

I do not personally think that would be useful at all.  And I
happen to know how expensive to teach gitweb to produce such an
output, so I would not seriously suggest anybody to try it.

Can we please close this topic?

Re: git and time

[Matthew L Foster]

--- Junio C Hamano <junkio@cox.net> wrote:

> This must be a trick question.  git does not trust _any_
> timestamp, so there is no local "rather than" remote.

I actually understand that and agree. All I've been saying is it (git or gitweb.cgi) should prefer
the local timestamp rather than any "remote" timestamps for no other reason than to minimize the
possibility of timestamps being grossly inaccurate.

-Matt

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Sean]

On Wed, 27 Sep 2006 18:48:11 -0700 (PDT)
Matthew L Foster <mfoster167@yahoo.com> wrote:

> I actually understand that and agree. All I've been saying is it (git or gitweb.cgi) should prefer
> the local timestamp rather than any "remote" timestamps for no other reason than to minimize the
> possibility of timestamps being grossly inaccurate.

But any local time stamp would be a _lie_.  The time stamp in the commit records
when it was actually created.  And as Junio has pointed out, hundreds of commits
will typically arrive in a repo at the exact same time.  Your suggestion would
have them all showing the exact same time.  That's not helpful, and it loses
important factual information.

You'd lose the information of when that commit was actually _in truth_ created.
The vast majority of the time, everyone has their clock set to a reasonable
value and this all just works.  In the rare case when someone has their clock
set seriously out of whack, Git accurately records and reports that fact too.

Really.. this just isn't a problem.  Everything in Git continues to work
exactly as it should.   It may be interesting to see when a commit arrived
locally, but it shouldn't be a substitute for the _real_ accurate time
recorded in each commit that tells us when it was _really_, _in fact_,
_in truth_ created.

Sean

Re: git and time

[Matthew L Foster]

--- Sean <seanlkml@sympatico.ca> wrote:

> On Wed, 27 Sep 2006 18:48:11 -0700 (PDT)
> Matthew L Foster <mfoster167@yahoo.com> wrote:
> 
> > I actually understand that and agree. All I've been saying is it (git or gitweb.cgi) should
> prefer
> > the local timestamp rather than any "remote" timestamps for no other reason than to minimize
> the
> > possibility of timestamps being grossly inaccurate.
> 
> But any local time stamp would be a _lie_.  The time stamp in the commit records
> when it was actually created.  And as Junio has pointed out, hundreds of commits
> will typically arrive in a repo at the exact same time.  Your suggestion would
> have them all showing the exact same time.  That's not helpful, and it loses
> important factual information.

How does git ensure that the timestamp in a commit records when it was actually created? I am not
saying throw away creation time, just that local time is more preferable and relevant and
git/gitweb.cgi should not in any way depend on time being configured correctly on each and every
git server. I think users of kernel.org's repo (or web interface) care more about when change X
was commited to it than when an author created/emailed change X, but perhaps I am wrong or don't
understand git or both. 

-Matt


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[David Lang]

On Wed, 27 Sep 2006, Matthew L Foster wrote:

> --- Sean <seanlkml@sympatico.ca> wrote:
>
>> On Wed, 27 Sep 2006 18:48:11 -0700 (PDT)
>> Matthew L Foster <mfoster167@yahoo.com> wrote:
>>
>>> I actually understand that and agree. All I've been saying is it (git or gitweb.cgi) should
>> prefer
>>> the local timestamp rather than any "remote" timestamps for no other reason than to minimize
>> the
>>> possibility of timestamps being grossly inaccurate.
>>
>> But any local time stamp would be a _lie_.  The time stamp in the commit records
>> when it was actually created.  And as Junio has pointed out, hundreds of commits
>> will typically arrive in a repo at the exact same time.  Your suggestion would
>> have them all showing the exact same time.  That's not helpful, and it loses
>> important factual information.
>
> How does git ensure that the timestamp in a commit records when it was actually created? I am not
> saying throw away creation time, just that local time is more preferable and relevant and
> git/gitweb.cgi should not in any way depend on time being configured correctly on each and every
> git server. I think users of kernel.org's repo (or web interface) care more about when change X
> was commited to it than when an author created/emailed change X, but perhaps I am wrong or don't
> understand git or both.

what you are missing is that there is no one true time 'when the change was 
commited' to record.

the closest that you can get in a distributed environement is 'when was this 
change created' and that is the locally defined time on the box that created it 
(or as Ted stated, some much more complicated process requireing network access 
to a certified time server)

after it has been created it has been commited (on that box). the same change 
could be commited on 50 other boxes, either by receiving it through git, or by 
receiving it via other methods.

all 51 of the boxes above are equally important to git. the fact that one or two 
of those boxes happen to have a user named Linus doesn't matter.

however, if what you want to know is 'in what order did this change get into the 
tree I am looking at compared to other changes' git can tell you that and what 
it will tell you is accurate, no matter what the clocks are set to on the 
various systems.

David Lang

Re: git and time

[Sean]

On Wed, 27 Sep 2006 19:49:37 -0700 (PDT)
Matthew L Foster <mfoster167@yahoo.com> wrote:

> How does git ensure that the timestamp in a commit records when it was actually created? I am not
> saying throw away creation time, just that local time is more preferable and relevant and
> git/gitweb.cgi should not in any way depend on time being configured correctly on each and every

The time stamp held in each Git commit accurately records the system time
of the machine on which it was created.  That's about the only guarantee 
offered.

The vast majority of the time this accurately records the proper time.
In the rare case when the system time is set incorrectly Git will
"accurately" record this value.  The good news is that Git continues
to work without any problem even if this happens.  Hopefully, you
can put your mind at ease over that issue.

> git server. I think users of kernel.org's repo (or web interface) care more about when change X
> was commited to it than when an author created/emailed change X, but perhaps I am wrong or don't
> understand git or both. 

Well no, the time each commit was actually created and imported into its
initial repository is also interesting to people.

But I happen to agree with you that it would be nice to _also_ show the
time that each commit appeared in the kernel.org repo.  Unfortunately, the
way Git is designed this just isn't easy.  In fact, Junio has gone so far
as to dare everyone not to even try to implement that feature.  So we might
just have to accept that we'll never see that information on kernel.org.

You will likely become a lot more comfortable with this, the more you get
to know and use Git.  Perhaps playing with gitk is worthwhile, as Linus
suggested.  Hopefully you can let this issue go, at least until you've
played with git some more.

Sean

Re: git and time

[Tom Prince]

On Wed, Sep 27, 2006 at 07:49:37PM -0700, Matthew L Foster wrote:
> How does git ensure that the timestamp in a commit records when it was
> actually created?
Git doesn't try. Git would be perfectly happy if time(2) returned a
random value each time. No part of git uses the time in any meaningful
way, it just keeps a record of it to display.

> I am not saying throw away creation time, just that local time is more
> preferable and relevant and git/gitweb.cgi should not in any way
> depend on time being configured correctly on each and every git
> server.
git and gitweb.cgi DO NOT in any way depend on time being configured
correctly. If some machine has incorrectly configured time, then they
will *correctly* display the strange time. 

> I think users of kernel.org's repo (or web interface) care
> more about when change X was commited to it than when an author
> created/emailed change X, but perhaps I am wrong or don't understand
> git or both. 
Based on comments on this list, it appears that most people don't aren't
interested in when exactly a given change hits kernel.org. Several
people have pointed out ways to get at the information you want, but
none of have seen a need for the information, and so haven't the
motivation to implement.

It might help if you gave a use case for the information you want. If
you give us a problem that you think the information will solve, the
list will probably be able to you how to get at the information you
want.

I think you have some misconception of how git works. Git is designed to
not lose information. Once a commit is created it is immutable,
regardless of whether the time is wrong, or the author, or the commit
message, or any bugs in the commit. This is a *feature*. It means you
need to go out of your way to destroy history.

> -Matt

  Tom

Re: git and time

[Theodore Tso]

On Wed, Sep 27, 2006 at 10:04:04PM -0400, Sean wrote:

> > I actually understand that and agree. All I've been saying is it
> > (git or gitweb.cgi) should prefer the local timestamp rather than
> > any "remote" timestamps for no other reason than to minimize the
> > possibility of timestamps being grossly inaccurate.
> 
> But any local time stamp would be a _lie_.  The time stamp in the
> commit records when it was actually created.  And as Junio has
> pointed out, hundreds of commits will typically arrive in a repo at
> the exact same time.  Your suggestion would have them all showing
> the exact same time.  That's not helpful, and it loses important
> factual information.

There are two issues here.  Could a git tree record the local time
that each commit entered the repository?  Sure.  Someone who wanted to
hack up git so that it created a local db file associating the SHA
hash name of the commit with when it arrived in its local repository.
It would not be part of the "true" git repository information, and it
could be something which is optional in the sense that if the
information is lost, it's not the end of the world, vis-a-vis correct
git functioning.

The second question, though, is would it be *useful*?  Presumably this
would be an option, so the user could request to see the time when a
commit hit a particular repository, as well as when the commit was
first created --- or perhaps both.

The problem though is that this could easily get confusing, since it
adds a distinction (which repository am I talking to?) which normally
doesn't exist in git.  And it forces us to make explicit things that
normally are kept hidden --- such as whether or not Linus does his
work directly on the git tree which is exposed by master.kernel.org,
or whether he does the work on his own local tree, and then pushes the
changes to master.kernel.org.  

In git, we believe that all repositories are equal, and that any sense
that a particular repository is the "master" or the "mainline" is
strictly speaking, a matter of convention.  What Matthew I think is
asking for is direct support in git for that notion.

So it *could* be done, but whether or not it is a good idea or worth
the complexity is a different question.  One could imagine a
completely different protocol, which exported the contents of the
hypothetical db database described above.  So for maintainers that
were *important*, and who were willing to make this information
available, given a particular SHA hash of a commit, one could ask the
question, "when did this commit first eneter your repository"?

Matthew seems hung up on on desperately wanting to know this
information as it relates to a particular repository --- Linus's.
This is in fact different from when the commit hit the repository
which is master.kernel.org which is why the local commit times on
master.kernel.org wouldn't be useful.  

HOWEVER, if it was judged important enough, and worth the complexity
that it would add to git, the ability to track when a commit entered a
particular repository and the ability to export it via some kind of
web interface certainly be implemented.  Linus would then have to
decide whether this was information he felt like making available to
inquisitive seekers wanting to know this sort of info.

						- Ted

Re: git and time

[Matthew L Foster]

--- Theodore Tso <tytso@mit.edu> wrote:

> In git, we believe that all repositories are equal, and that any sense
> that a particular repository is the "master" or the "mainline" is
> strictly speaking, a matter of convention.  What Matthew I think is
> asking for is direct support in git for that notion.

No. I merely think git should try harder to ensure that commit order is consistent with time
order, it really should (somehow) be impossible for git and gitweb.cgi to have commit dates ~2
days in the future. I think replication is a separate issue. In a distributed system the only
"time" that makes any sense or is the most relevant in many situations and most importantly is the
only thing that can be semi-trusted is local time. "Creation date" is basically just random text
someone entered, there is no guarantee and you are tempting inconsistent time order. And git
shouldn't be so fragile as to need each and every git server to have time set semi-correctly, but
I guess it's a bigger deal to non-developers as we actually use time and also believe even web
interfaces should have consistency and integrity. 

-Matt

-Matt

-Matt
 

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Rogan Dawes]

Matthew L Foster wrote:
> --- Theodore Tso <tytso@mit.edu> wrote:
> 
>> In git, we believe that all repositories are equal, and that any sense
>> that a particular repository is the "master" or the "mainline" is
>> strictly speaking, a matter of convention.  What Matthew I think is
>> asking for is direct support in git for that notion.
> 
> No. I merely think git should try harder to ensure that commit order is consistent with time
> order, it really should (somehow) be impossible for git and gitweb.cgi to have commit dates ~2
> days in the future. I think replication is a separate issue. In a distributed system the only
> "time" that makes any sense or is the most relevant in many situations and most importantly is the
> only thing that can be semi-trusted is local time. "Creation date" is basically just random text
> someone entered, there is no guarantee and you are tempting inconsistent time order. And git
> shouldn't be so fragile as to need each and every git server to have time set semi-correctly, but
> I guess it's a bigger deal to non-developers as we actually use time and also believe even web
> interfaces should have consistency and integrity. 
> 
> -Matt
> 

See, the confusion here seems to be that you think that not caring about 
the time attached to a commit for anything more than a heuristic makes 
git fragile.

I think that the rest of the git developers prefer to see this as a 
feature that makes git more robust in the face of something that they 
might not have any control over (or desire to control).

That said, if *you* are managing a repository, it shouldn't be difficult 
to enforce the kind of rule that you are asking for. Simply implement a 
pre-commit hook that checks all commits that will be added to *your* 
repo to make sure that time is monotonically increasing from the last 
commit, and that it does not pass "now".

So if you receive a commit from a developer that violates this rule, you 
can send the commit back to them with a request to fix their system 
time, and recreate the patch/series.

I just don't think that any of the kernel developers feel the need to 
police any one else's clocks . . . they're more interested in the 
contents of the patch.

Regards,

Rogan

Re: git and time

[Matthew L Foster]

--- Rogan Dawes <discard@dawes.za.net> wrote:

> I just don't think that any of the kernel developers feel the need to 
> police any one else's clocks . . . they're more interested in the 
> contents of the patch.

I am not saying git should "police any one else's clocks", I am saying git should be designed or
configured in such a way, using local time, that it obviates the current reliance on everyone
else's clock being set correctly. 

-Matt

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Linus Torvalds]

On Thu, 28 Sep 2006, Matthew L Foster wrote:
> 
> I am not saying git should "police any one else's clocks", I am saying git should be designed or
> configured in such a way, using local time, that it obviates the current reliance on everyone
> else's clock being set correctly. 

Matt!

THERE IS NO SUCH RELIANCE! NONE.

Trust me. When we say that git ignores time, WE MEAN IT. Git does not rely 
on time, git does not use time, git does not CARE!

Please stop looking at gitweb _immediately_. If you think time has some 
meaning for git, stop. It doesn't. We've told you over and over and over 
again that there is absolutely _zero_ reliance on everybody else's clock 
being set correctly. The damn clock could go _backwards_, or make huge 
jumping purple leaps of imagination, and git wouldn't care.

The time that git records is purely a random number. It's a random number 
that _humans_ can choose to care about or not, and it's a random number 
that git itself uses only in the sense of "ok, I've got two equal choices, 
let's toss a coin to select which one I'll look at next", BUT IT IS A 
RANDOM NUMBER.

Please.

Download a local git archive, and run "gitk" on that archive. Then _hide_ 
the time column (just so that it doesn't confuse you), and then _look_ at 
the leftmost column which shows the CAUSALITY DAG!

THAT is what git actually cares about. It's the _only_ thing that git 
cares about. Git cares about _causality_, not time. Nothing else matters.

Really. Truly, pretty please, I promise you, cross my heart and hope to 
die! Scouts honor. Lock it up and throw away the key. It's TRUE.

		Linus

Re: git and time

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> On Thu, 28 Sep 2006, Matthew L Foster wrote:
>> 
>> I am not saying git should "police any one else's clocks", I am saying git should be designed or
>> configured in such a way, using local time, that it obviates the current reliance on everyone
>> else's clock being set correctly. 
>
> Matt!
>
> THERE IS NO SUCH RELIANCE! NONE.
>
> Trust me. When we say that git ignores time, WE MEAN IT. Git does not rely 
> on time, git does not use time, git does not CARE!
>
> Please stop looking at gitweb _immediately_. If you think time has some 
> meaning for git, stop. It doesn't. We've told you over and over and over 
> again that there is absolutely _zero_ reliance on everybody else's clock 
> being set correctly. The damn clock could go _backwards_, or make huge 
> jumping purple leaps of imagination, and git wouldn't care.

I think Matthew means (by "relying") that everybody's clock must
be set correctly in order for us to show the commits in gitweb
or rev-list output so that their timestamps are monotonically
decreasing (because we list things from newer to older).  I
sympathise.

We order things by causality (i.e. ancestry order), but that
unfortunately (!!)  happens to match timestamp order for simple
history made on a single machine.  This can easily lead to such
a misunderstanding that we are somehow trying to show things in
linear time order, hence we subscribe to the notion of global,
uniform and monotonic time.

Of course, we don't.

Re: git and time

[Matthew L Foster]

> Trust me. When we say that git ignores time, WE MEAN IT. Git does not rely 
> on time, git does not use time, git does not CARE!

I understand that git doesn't care about time internally as it treats it as a random number for
people to care about or not on their own, I am only saying that local time is more likely to be
cared about than disparate remote creation times. 

It should be possible to export git data, through say a web interface, in a such a way that local
time order is consistent with commit order. 

-Matt


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Johannes Schindelin]

On Thu, 28 Sep 2006, Matthew L Foster wrote:

> It should be possible to export git data, through say a web interface, 
> in a such a way that local time order is consistent with commit order.

Why?

Re: git and time

[Matthew L Foster]

--- Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

> On Thu, 28 Sep 2006, Matthew L Foster wrote:
> 
> > It should be possible to export git data, through say a web interface, 
> > in a such a way that local time order is consistent with commit order.
> 
> Why?

- So exported data is never/rarely in an inconsistent state with respect to commit order and local
time order (data integrity).

- To encourage people to care about/prefer local commit time rather than remote creation/emailed
time

- So people that user repo X, or binaries from repo X, know when bug fix Y/fancy new feature Z was
committed/merged locally

- In many situations "history" is incomplete without local commit time. If a company has a new
driver they would probably prefer to know when the main kernel repo has it, not when they
created/emailed it or when a remote repo committed it.

-Matt

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Shawn Pearce]

Matthew L Foster <mfoster167@yahoo.com> wrote:
> --- Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> 
> > On Thu, 28 Sep 2006, Matthew L Foster wrote:
> > 
> > > It should be possible to export git data, through say a web interface, 
> > > in a such a way that local time order is consistent with commit order.
> > 
> > Why?
> 
> - So exported data is never/rarely in an inconsistent state with respect to commit order and local
> time order (data integrity).

Pick one.  You can't have "never" and "rarely".
 
> - To encourage people to care about/prefer local commit time rather than remote creation/emailed
> time

Why?  In general I don't care about time in Git.  Maybe I care about
when I authored something if my wife wants to know why I was up until
6 am the night before ("Look honey, I was coding until 6:00 am! See
the commit!") but otherwise I don't find time to be that interesting.

Then again I don't find time in the real world that intersting
either.  I'm either where I'm supposed to be or I'm not and I'll
get there when I get there.

> - So people that user repo X, or binaries from repo X, know when bug fix Y/fancy new feature Z was
> committed/merged locally

Track it by version, not timestamp.  Know what commit or tag SHA1
was used to produce that binary.  Ask GIT if the fix is in that
SHA1 ancestory or not.  I've already said that on this thread.
 
> - In many situations "history" is incomplete without local commit time. If a company has a new
> driver they would probably prefer to know when the main kernel repo has it, not when they
> created/emailed it or when a remote repo committed it.

I think they care more about what release of the kernel will have
that driver.  That can easily be determined by the DAG and by
understanding what branch(es) will wind up in the next release and
doing simple math: "Lets see, current release is version 2.6.9000,
so it will be in 2.6.9001."

-- 
Shawn.

Re: git and time

[Matthew L Foster]

--- Shawn Pearce <spearce@spearce.org> wrote:

> > - So exported data is never/rarely in an inconsistent state with respect to commit order and
> > local time order (data integrity).
> 
> Pick one.  You can't have "never" and "rarely".

I mean "rarely" in the sense that there is no guarantee that local time is exact but any
inexactness would be confined locally.
 
> Track it by version, not timestamp.  Know what commit or tag SHA1
> was used to produce that binary.  Ask GIT if the fix is in that
> SHA1 ancestory or not.  I've already said that on this thread.

So you are saying time, even local commit time, is completely unnecessary? I disagree. Git doesn't
need to keep track of any times in a distributed way, it just might be worthwhile to keep track of
local commit timestamps internally per repo.

> I think they care more about what release of the kernel will have
> that driver.  That can easily be determined by the DAG and by
> understanding what branch(es) will wind up in the next release and
> doing simple math: "Lets see, current release is version 2.6.9000,
> so it will be in 2.6.9001."

Even if people care more about "what release" that doesn't mean they don't care about (local
commit) time.

-Matt


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Johannes Schindelin]

Matt,

what exactly are you trying to achieve here? Is this a sort of contest who 
can have the longest thread on git@vger.kernel.org?

If you really want to understand why git does not rely on timestamps, and 
why it should not, and why you can still be happy nevertheless, there are 
enough answers in this thread.

Hth,
Dscho

Re: git and time

[Matthew L Foster]

--- Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

> what exactly are you trying to achieve here? 

Timestamps not being all over the place in gitweb.cgi. 

> If you really want to understand why git does not rely on timestamps, and 
> why it should not, and why you can still be happy nevertheless, there are 
> enough answers in this thread.

I agree and understand that distributed git should not and does not rely on timestamps, I am just
suggesting that it might be worthwhile to _locally_ track local commit time more efficiently for
local use in things like gitweb.cgi. 

-Matt

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Andreas Ericsson]

Matthew L Foster wrote:
> --- Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> 
>> On Thu, 28 Sep 2006, Matthew L Foster wrote:
>>
>>> It should be possible to export git data, through say a web interface, 
>>> in a such a way that local time order is consistent with commit order.
>> Why?
> 
> - So exported data is never/rarely in an inconsistent state with respect to commit order and local
> time order (data integrity).
> 

Moot point (it has been iterated so many times that I can't be asked to 
repeat it again).

> - To encourage people to care about/prefer local commit time rather than remote creation/emailed
> time
> 

Most people use ntp, and are in general concerned with keeping their 
clocks in sync as lots of other software depend on it (calender 
functions, fe). It shouldn't be the task of project leaders to make sure 
that the ~50000 random people around the world that submit patches to 
opensource projects every day all have their clocks in sync.

> - So people that user repo X, or binaries from repo X, know when bug fix Y/fancy new feature Z was
> committed/merged locally
> 

Can be done using reflog. Feel free to submit patches. Make sure you 
sync your clock to whatever ntp-server or other timekeeping mechanism 
Junio uses before you commit and send your patch though. ;-)

> - In many situations "history" is incomplete without local commit time. If a company has a new
> driver they would probably prefer to know when the main kernel repo has it, not when they
> created/emailed it or when a remote repo committed it.
> 

See the reflog response and, again, feel free to submit patches.

To get you started, I think the easiest way would be to teach gitweb 
about the reflog, and then insert a line saying
"--- pushed to this repo $date ---"
or something like that in the summary page whenever a commit is found 
that is also in the reflog. This should also be fairly CPU efficient if 
my guesses on how gitweb and the reflog works are correct. CBA to check, 
since I sincerely and whole-heartedly don't care about it myself.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: git and time

[Linus Torvalds]

On Thu, 28 Sep 2006, Matthew L Foster wrote:
> 
> It should be possible to export git data, through say a web interface, 
> in a such a way that local time order is consistent with commit order.

I really don't think the thing you ask for exists.

Don't get me wrong. You _can_ have a local time for each commit that 
tracks "when did this commit show up in this particular branch". Git 
already supports that, even if gitweb cannot show it, and in fact showing 
it would be very hard (since the exact same commit can often exist in 
multiple different branches, you'd have to show multiple times: in Junios 
"git" tree you often have a commit that showed up in the "next" branch 
three weeks ago, but in the "master" branch only yesterday).

The _problem_ with this is that it makes the whole concept of time 
meaningless. It's pointless. You can do it, but I guarantee you that once 
you actually use it for a while, you'll want to go back. There are several 
reasons for that:

 - it means that the -same- exact project, when looked at frm two 
   different sites that mirror it, have totally different times. In other 
   words, the times have become pointless for something like gitweb.

 - it means that all times will be seriously "compressed", in that you'll 
   find hundreds (or thousands) of commits that just have the same 
   timestamp. You could try to "spread them out" by just making up some 
   totally arbitrary mapping function, but that would basically have 
   absolutely no basis in anything that has any relationship to "reality"

So it just doesn't make any sense.

The only thing that makes sense is that in your private repository (that 
is _not_ exported to others through "gitweb" or something like that), you 
can ask yourself the question:

	"What did my tree look like yesterday before I went out for a 
	 beer, and came back drunk as a toad, and screwed everything up?"

And the thing is, you can do that already. Just say

	git log "master@{18 hours ago}"

and git will hopefully show you (assuming you had enabled ref-logging as 
described earlier in this thread) exactly what you wanted.

See? 

But it does not make sense in _any_ other setting. Certainly not gitweb.

			Linus

Re: git and time

[Matthew L Foster]

--- Linus Torvalds <torvalds@osdl.org> wrote:
 
>  - it means that the -same- exact project, when looked at frm two 
>    different sites that mirror it, have totally different times. In other 
>    words, the times have become pointless for something like gitweb.

Mirrored git repos probably should use the same timestamp, e.g. the "master" or "private" git
server's local time. Replicated repos have a delay compared to when you made changes in your
private repo, that is ok, replication is not what makes commit order inconsistent with time, it's
the act of pulling/merging from a server with misconfigured time and gitweb.cgi trusting "creation
time", right?  Or is replication the same thing as merging/pulling? 

-Matt 

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Theodore Tso]

On Thu, Sep 28, 2006 at 12:18:15PM -0700, Matthew L Foster wrote:
> --- Linus Torvalds <torvalds@osdl.org> wrote:
>  
> >  - it means that the -same- exact project, when looked at frm two 
> >    different sites that mirror it, have totally different times. In other 
> >    words, the times have become pointless for something like gitweb.
> 
> Mirrored git repos probably should use the same timestamp, e.g. the
> "master" or "private" git server's local time. Replicated repos have
> a delay compared to when you made changes in your private repo, that
> is ok, replication is not what makes commit order inconsistent with
> time, it's the act of pulling/merging from a server with
> misconfigured time and gitweb.cgi trusting "creation time", right?
> Or is replication the same thing as merging/pulling?

Git mirroring takes place using the same pushing and pulling that are
used with multiple repositories.  That's why if Linus does a huge
amount of work over the period of a day or two in his private
repository, and then publishes it master.kernel.org, the "local" time
on master.kernel.org will be the time when they are pushed to
master.kernel.org, because it's done via the same operation as any
other repository push or pull.

That's what everyone has been trying to tell you for this entire
thread....

						- Ted

Re: git and time

[Matthew L Foster]

--- Theodore Tso <tytso@mit.edu> wrote:

> Git mirroring takes place using the same pushing and pulling that are
> used with multiple repositories.  That's why if Linus does a huge
> amount of work over the period of a day or two in his private
> repository, and then publishes it master.kernel.org, the "local" time
> on master.kernel.org will be the time when they are pushed to
> master.kernel.org, because it's done via the same operation as any
> other repository push or pull.
> 
> That's what everyone has been trying to tell you for this entire
> thread....

Ok, I was wondering about that. In your example above the internally unnecessary timestamp will be
from Linus' private repo, not master.kernel.org's? Even now knowing replication = merge/push/pull
I still think it worthwile for local commit time to be effiently tracked locally and/or if already
available in the local ref-log used by things like gitweb.cgi if feasible but I might be the only
one that cares this much about time. Gitweb.cgi currently relies at least display/visually on
those internally unnecessary and potentially misleading timestamps...

-Matt


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Junio C Hamano]

Matthew L Foster <mfoster167@yahoo.com> writes:

> Ok, I was wondering about that. In your example above the
> internally unnecessary timestamp will be from Linus' private
> repo, not master.kernel.org's?

Yes, and it is stronger than that.  If somebody did a sata
patch, sent that to Jeff over e-mail, and if the patch was
accepted, Jeff will make a commit in his private repository,
recording local time on his machine.  Later Linus may pull from
Jeff and the commit object is transferred during that.

To Linus, the time he first saw the commit was the time he
pulled from Jeff, so being able to tell when it came into his
repository may help him if he pulled from other people too and
then suddenly realizes his sata disk does not respond at all.

He COULD say "it was working yesterday, I pulled from Jeff 3
hours ago, and then David 2 hours ago, I did not do my own
development during that time.  Did the breakage come when I
pulled from Jeff or when I pulled from David?"  ref-log would
let him do something like:

	git checkout -b trythis master@{4.hours.ago}

to make sure the state before he pulled from Jeff was a working
state and then still on the trythis branch he cuold do

	git reset --hard master@{2.hours.30.minutes.ago}

to see if the state after he pulled from Jeff was broken.

	Side note: in reality he does not have to care.  He can
	just bisect it without using any of his "pull boundary"
	time.

So that was discussion about the time Linus first saw the
commit.  What about us, general public?  Until Linus pushes out
the merge result, we would not see it.  Anyway, eventually he
will push his tip of the branch out to kernel.org and rsync will
mirror to public git:// and gitweb machines.

What's the local time the general public sees the commit for the
first time?  It's (forgetting for now the rsync mirroring delay)
the time Linus pushed the tip of the branch out.  Along with all
other hundreds of commits he acquired since the last time he
pushed his tree out.

It is sometimes useful to know when a particular commit has
become available to the general public.  I do not think anybody
is denying it.

But it is a completely separate issue if it is useful to label
the commits that happened to be pushed out together at the same
time with the same timestamp on the gitweb short-log page (or
short-log corner on the summary page).  Most of the time you
will see commits pushed out by the same push operation and
having exactly the same timestamp.  That's not very useful way
to present the information.

	Side note: some commits arrive kernel.org machine
	earlier than others because Linus does not have infinite
	bandwidth to kernel.org, but these hundreds of commits
	become visible to the general public exactly the same
	time, because we send them and as the last operation we
	update the tip of the branch.  So you cannot even say
	"record the time down to the second they arrived the
	kernel.org repository" -- until the branch tip is
	updated the general public cannot see them so the
	arrival time (mtime of .git/objects/??/???...?? files)
	does not even matter.

If somebody feels strongly about it, I would suggest adding that
information on the commit page of gitweb, where the program
needs to deal with only one commit.  That would help somebody
who is interested in _one_ particular commit and wants to know
when it has become available to the general public.

-

Re: git and time

[Robin Rosenberg]

torsdag 28 september 2006 19:11 skrev Linus Torvalds:
> The time that git records is purely a random number. It's a random number
> that _humans_ can choose to care about or not, and it's a random number
> that git itself uses only in the sense of "ok, I've got two equal choices,
> let's toss a coin to select which one I'll look at next", BUT IT IS A
> RANDOM NUMBER.

I'd think of it as comment, about as (un)reliable as the author field or the 
descriptive free-form comment people enter when they commit. It's not even 
necessarily the local system time if GIT_AUTHOR_DATE has been set.

-- robin

Re: git and time

[A Large Angry SCM]

Matthew L Foster wrote:
> --- Rogan Dawes <discard@dawes.za.net> wrote:
> 
>> I just don't think that any of the kernel developers feel the need to 
>> police any one else's clocks . . . they're more interested in the 
>> contents of the patch.
> 
> I am not saying git should "police any one else's clocks", I am saying git should be designed or
> configured in such a way, using local time, that it obviates the current reliance on everyone
> else's clock being set correctly. 

Sounds like you're suggesting that Git should not record any timestamps 
at all. After all, Git doesn't need them.

Re: git and time

[Matthew L Foster]

--- A Large Angry SCM <gitzilla@gmail.com> wrote:

> Sounds like you're suggesting that Git should not record any timestamps 
> at all. After all, Git doesn't need them.

Yeah kind of, since distributed git doesn't need timestamps and can't guarantee them the only time
that might make any sense to use locally is local commit time. 

-Matt

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[A Large Angry SCM]

Matthew L Foster wrote:
> --- A Large Angry SCM <gitzilla@gmail.com> wrote:
> 
>> Sounds like you're suggesting that Git should not record any timestamps 
>> at all. After all, Git doesn't need them.
> 
> Yeah kind of, since distributed git doesn't need timestamps and can't guarantee them the only time
> that might make any sense to use locally is local commit time. 

There is no local commit time for things you get from a remote repository.

When I wrote "Sounds like you're suggesting that Git should not record 
any timestamps at all", I meant _you_ don't think Git should record 
_any_ timestamps since they can't be guaranteed to match the DAG.

Re: git and time

[Matthew L Foster]

--- A Large Angry SCM <gitzilla@gmail.com> wrote:

> There is no local commit time for things you get from a remote repository.

Are you saying it's impossible to internally record/track local commit/merge time for things you
get from a remote repository, ref-log?

-Matt

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Matthew L Foster]

--- A Large Angry SCM <gitzilla@gmail.com> wrote:

> When I wrote "Sounds like you're suggesting that Git should not record 
> any timestamps at all", I meant _you_ don't think Git should record 
> _any_ timestamps since they can't be guaranteed to match the DAG.

Well, I mean since there is no time order matching commit order guarantee for distributed git the
only timestamp to use for anything locally is local time (if timestamps are used at all). The
current remote creation or merge time could stay as is (I don't care either way), but at least
gitweb.cgi should become local commit time/ref-log aware. Though if it's infeasible or requires
some rearchitecting it might not happen.

-Matt 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Nicolas Pitre]

On Wed, 27 Sep 2006, Junio C Hamano wrote:

> As Shawn said in a nearby thread, in a public and prominent
> repository like kernel.org repository it may sometimes be
> interesting and useful to know when each commit became available
> on each branch.  I am reasonably sure that it would not however
> make gitweb output easier to read to order its output by that
> timestamp.   Linus pulls from subsystem maintainers all the time
> and one pull may bring in dozens of commits, and they will get
> the same timestamp if you did so.  Actually it is worse than
> that.  He tends to batch, so he would have many such pulls and
> patch applications in his private repository, perhaps over a few
> hour, but the result will be pushed out to kernel.org with one
> push operation.  To show the "truthful" time, your gitweb would
> give the timestamp of that push operation for hundreds of
> commits pushed out during that operation.
> 
> I do not personally think that would be useful at all.  And I
> happen to know how expensive to teach gitweb to produce such an
> output, so I would not seriously suggest anybody to try it.

I beg to differ.  Such information might be really useful.  I agree 
though that this is an expensive operation and gitweb might not be the 
best place for it at all.

For example... some times I look at git-log output and finds about a 
certain bug fix that was apparently committed a month ago.  And 
incidentally I recall having been bitten by that bug not really long 
ago, say last week.  Although the bug fix was committed _somewhere_ last 
month, what I would really want to know is just when _i_ received that 
bug fix in my own repository to determine if it was before or after last 
week.  So if it was before last week then I could conclude that the bug 
fix didn't actually fix my bug.  Knowing that it has been committed last 
month is absolutely useless to me in this case.


Nicolas

Re: git and time

[Tom Prince]

On Wed, Sep 27, 2006 at 11:07:35PM -0400, Nicolas Pitre wrote:
> I beg to differ.  Such information might be really useful.  I agree 
> though that this is an expensive operation and gitweb might not be the 
> best place for it at all.
> 
> For example... some times I look at git-log output and finds about a 
> certain bug fix that was apparently committed a month ago.  And 
> incidentally I recall having been bitten by that bug not really long 
> ago, say last week.  Although the bug fix was committed _somewhere_ last 
> month, what I would really want to know is just when _i_ received that 
> bug fix in my own repository to determine if it was before or after last 
> week.  So if it was before last week then I could conclude that the bug 
> fix didn't actually fix my bug.  Knowing that it has been committed last 
> month is absolutely useless to me in this case.
> 

But even knowing when the commit arrived in your local repository does
you no good unless you are compiling every time you pull, in which case,
the reflog support on you local machine will give you the information
you need. Otherwise, you need to know the name of the commit you were
running when you got bitten by the bug which is a separate issue.

> Nicolas

  Tom

Re: git and time

[Shawn Pearce]

Tom Prince <tom.prince@ualberta.net> wrote:
> On Wed, Sep 27, 2006 at 11:07:35PM -0400, Nicolas Pitre wrote:
> > I beg to differ.  Such information might be really useful.  I agree 
> > though that this is an expensive operation and gitweb might not be the 
> > best place for it at all.
> > 
> > For example... some times I look at git-log output and finds about a 
> > certain bug fix that was apparently committed a month ago.  And 
> > incidentally I recall having been bitten by that bug not really long 
> > ago, say last week.  Although the bug fix was committed _somewhere_ last 
> > month, what I would really want to know is just when _i_ received that 
> > bug fix in my own repository to determine if it was before or after last 
> > week.  So if it was before last week then I could conclude that the bug 
> > fix didn't actually fix my bug.  Knowing that it has been committed last 
> > month is absolutely useless to me in this case.
> > 
> 
> But even knowing when the commit arrived in your local repository does
> you no good unless you are compiling every time you pull, in which case,
> the reflog support on you local machine will give you the information
> you need. Otherwise, you need to know the name of the commit you were
> running when you got bitten by the bug which is a separate issue.

Which is why Git embeds its version number from git-describe as
part of its build process.  So "git --version" can give you back
an identifier.

And given that identifier (and the new schooling given to the
sha1 expression parser) you can easily ask git to determine if the
given bug fix commit was an ancestor of that identifier, or not.
No reflog involved and you have your answer.

As it happens I build and install `next` every time I fetch from
Junio's public tree.  But I never rely on my reflog to associate
back to a commit; I always use `git --version`.

-- 
Shawn.

Re: git and time

[Junio C Hamano]

Nicolas Pitre <nico@cam.org> writes:

> On Wed, 27 Sep 2006, Junio C Hamano wrote:
>
>> ....  He tends to batch, so he would have many such pulls and
>> patch applications in his private repository, perhaps over a few
>> hour, but the result will be pushed out to kernel.org with one
>> push operation.  To show the "truthful" time, your gitweb would
>> give the timestamp of that push operation for hundreds of
>> commits pushed out during that operation.
>> 
>> I do not personally think that would be useful at all.  And I
>> happen to know how expensive to teach gitweb to produce such an
>> output, so I would not seriously suggest anybody to try it.
>
> I beg to differ.  Such information might be really useful.  I agree 
> though that this is an expensive operation and gitweb might not be the 
> best place for it at all.
>
> For example... some times I look at git-log output and finds about a 
> certain bug fix that was apparently committed a month ago.  And 
> incidentally I recall having been bitten by that bug not really long 
> ago, say last week.  Although the bug fix was committed _somewhere_ last 
> month, what I would really want to know is just when _i_ received that 
> bug fix in my own repository to determine if it was before or after last 
> week.  So if it was before last week then I could conclude that the bug 
> fix didn't actually fix my bug.  Knowing that it has been committed last 
> month is absolutely useless to me in this case.

I beg to agree ;-).  Being able to inspect a particular commit
to find out when it hit the branch is probably useful.  I do not
have much objection to have it on the commit page.

It is totally a separate matter to use that timestamp to order
the commits listed on the shortlog page, which Matthew seemed to
be after.  I would say that _is_ wasteful and loses more
interesting information by listing all several dozen commits
Linus pushed out the last time with the same timestamps.

Re: git and time

[Theodore Tso]

On Wed, Sep 27, 2006 at 05:12:41PM -0700, Matthew L Foster wrote:
> 
> Ignoring the separate issue of replication for a momment, can
> someone respond to my time integrity question about whether a future
> version of git could trust/prefer its local time rather than a
> remote/sub/parent (non replicated) git server's timestamp? 

No, it can't.  In order to do that it would have to change the commit,
and that would be rewriting history.

Consider a more complicated case of replication, where git is used
exclusively, so that a patch gets commited by developer A, pushed to
device driver maintainer B, which then gets pushed to subsystem
maintainer C, which then pushed to Linus's private tree, and then
Linus pushes it to the publically visible repository.  Now assume for
the sake of argument that developers A, B, and C all keep publically
visible repositories, all of which are accessible via gitweb.cgi, and
assume for the sake of argument that the patch is pushed exclusively
via git.  

In that case, the time that the changeset was made is the time that
developer A created the commit.  That is, of course, the time on his
local machine, which may or may not be accurate.  And of course, the
e-mail address which he gives may or may not be accurate as well.
There can be no integrity guarantees because we are not using
cryptography.  There are no time notaries, no public key signatures.
Both the time and the author name/e-mail are merely metadata which is
attached to the patch, and the time, author name/e-mail, and patch are
hashed together to form a SHA checksum of the commit.  So while we
don't have any kind of assurance about the time or author e-mail, what
we *do* have is an assurance that a commit's name --- which is the SHA
checksum --- refers to a specific patch, and with a specific claimed
timestamp and claimed e-mail name.  So if we refer to a particular
commit by that SHA hash value, we do know globally that we are always
referring to the same commit.  Changing the time would change the SHA
hash, which would make it impossible for two machines to know whether
or not a commit is the same or not.

More to the point, that particular commit will travel from developer
A, to B, to C, to Linus, to public repository on master.kernel.org.
During that whole time, it is invariant.  It cannot change, including
the commit timestamp.  Now, at each stage, as the patch gets pushed
along, all of the information ---- the patch itself, the claimed time,
and the claimed author date --- are used to evalute the patch and
decide whether or not the patch should be pushed higher up the trust
hierarchy.  In Linus, the most important thing which is used is an
evaluation of the patch itself.  If the patch is good, the fact that
the commit time might be six months in the past isn't necessarily a
problem.  

And of course, the commit time might be *right*.  It could be that
developer A did do the work six months ago, but merely sat on it for a
while before submitting it to maintainer B, and maybe maintainer B sat
on it because rc1 had passed and he was being a good doobie and not
forwarding patches on so that tree could stablize, and by the time rc7
was released, he had gotten tired of waiting, so pushed it on to
lieutenant C, and it finally got pushed to Linus after 2.6.18 was
released, many months later.  

> How do we fix gitweb.cgi, ref-log? 
> How useful is gitweb.cgi if timestamps are
> all over the place? It does not make sense that commit order is
> currently out of sync with time order in the main linux kernel tree
> git repo on kernel.org. 

Your problem is that you are putting too much faith in the time stamp.
It's there as a help, but it is no more trustworthy than the author
name or the patch description --- which is to say, if people in the
git hierarchy are happy with the commit, it will get pushed, and if
they aren't it won't.  And most people don't care about the time; they
care about whether the patch is correct.  Now, the time is mostly
correct, so if you look at the patch dependencies you can usually
determine when a patch was likely committed to the first local
repoistory to within a few hours in all likelihood.

Now, I could imagine a system where every changeset also had metadata
associated with it that involved a receipt from a time notary, and a
digital signature so that each patch could be cryptographically shown
to have come form some individual (or at least someone who posses that
particular individual's key).  This would add a lot more overhead to
the SCM, but at least it's theoretically possible.  The question is it
worth it?

At least within the Linux commmunity, the most important thing is
whether or not the patch is good, not whether it is cryptographically
signed by some particular developer.  And times are considered even
less important --- and given that a time notary is complex, and
requires out of band facilities such as publishing lists of checksums
and checksums of checksums in publically norepudiable places such as
classifie dadds in the New York Times, LA Times, and Washington Post,
it won't be something which is zero cost, either.  And given that no
one cares enough about times to spend that kind of resources, I very
much doubt git will ever support time notary support.

> Why must each and every repo be dependent on
> time being set properly on all other git servers? 

They don't.

> How useful is change history or commit order without some concept of
> (local) time order?

Very useful.  :-)

							- Ted

Re: git and time

[Matthew L Foster]

--- Theodore Tso <tytso@mit.edu> wrote:

> On Wed, Sep 27, 2006 at 05:12:41PM -0700, Matthew L Foster wrote:
> > 
> > Ignoring the separate issue of replication for a momment, can
> > someone respond to my time integrity question about whether a future
> > version of git could trust/prefer its local time rather than a
> > remote/sub/parent (non replicated) git server's timestamp? 
> 
> No, it can't.  In order to do that it would have to change the commit,
> and that would be rewriting history.

Perhaps the actual change itself should not contain a "commit time", only "local commit time"
should matter or be tracked locally (if time is tracked/matters any). To repeat from a previous
mail, I am not saying timestamps (local or other) should be tracked in a git distributed way,
quite the opposite, local commit time should be tracked locally. 

Replication is a separate issue if I understand git any. Please correct me if I misunderstand: the
kernel.org gitweb.cgi linux repo time inconsistencies happened when Linus pulled into his
_private_ repo from a remote git server with misconfigured time, _not_ when he later replicated
those errant timestamps from his private repo to the public kernel.org one. I don't care nearly as
much about replication not being time aware, I care about a _merge_ from a remote misconfigured
git server making timestamps inconsistent with commit order. Using/prefering local time could
solve this but perhaps I am the only one that thinks local time is most important.

-Matt


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Nicolas Pitre]

On Wed, 27 Sep 2006, Matthew L Foster wrote:

> --- Theodore Tso <tytso@mit.edu> wrote:
> 
> > On Wed, Sep 27, 2006 at 05:12:41PM -0700, Matthew L Foster wrote:
> > > 
> > > Ignoring the separate issue of replication for a momment, can
> > > someone respond to my time integrity question about whether a future
> > > version of git could trust/prefer its local time rather than a
> > > remote/sub/parent (non replicated) git server's timestamp? 
> > 
> > No, it can't.  In order to do that it would have to change the commit,
> > and that would be rewriting history.
> 
> Perhaps the actual change itself should not contain a "commit time", 
> only "local commit time" should matter or be tracked locally (if time 
> is tracked/matters any). To repeat from a previous mail, I am not 
> saying timestamps (local or other) should be tracked in a git 
> distributed way, quite the opposite, local commit time should be 
> tracked locally.

What I think you want and what you should talk about is that you're 
interested into the "local appearance time" for a given commit and not 
"local commit time".  Using that terminology is probably much less 
confusing in the GIT world.

To do so you'll need a GIT command that doesn'T exist yet.  Let's call 
it git-local-arrival.  It could be defined as follows:

SYNOPSIS

	git-local-arrival <committish>

DESCRIPTION

	The command displays the time when given commit appeared in the 
	local repository.

Is that what you want?  That's certainly something _I_ would be 
interested in.  But such a command would have to do some commit graph 
walking, based on the recorded reflog data, (there is not much 
documentation about reflog unfortunately) to find out exactly when given 
commit actually was fetched into the local repository.  While that would 
be perfectly acceptable to use on your own machine, I don't think it 
would be a good idea to let gitweb use it due to the computing cost 
required.

But again that's something possible but for which there is currently no 
code.

[ thinking out loud: maybe git-rev-list could provide that local 
  appearance time quite easily though... ]


Nicolas

Re: git and time

[Junio C Hamano]

Nicolas Pitre <nico@cam.org> writes:

> What I think you want and what you should talk about is that you're 
> interested into the "local appearance time" for a given commit and not 
> "local commit time".  Using that terminology is probably much less 
> confusing in the GIT world.
>
> To do so you'll need a GIT command that doesn'T exist yet.  Let's call 
> it git-local-arrival.  It could be defined as follows:
>
> SYNOPSIS
>
> 	git-local-arrival <committish>
>
> DESCRIPTION
>
> 	The command displays the time when given commit appeared in the 
> 	local repository.

This should be certainly doable, but local-arrival may not be
interesting if the repository has more than one branches.  Maybe

	git-local-arrival <committish> [<branch>]

which defaults to the current branch?

Re: git and time

[Nicolas Pitre]

On Wed, 27 Sep 2006, Junio C Hamano wrote:

> Nicolas Pitre <nico@cam.org> writes:
> 
> > SYNOPSIS
> >
> > 	git-local-arrival <committish>
> >
> > DESCRIPTION
> >
> > 	The command displays the time when given commit appeared in the 
> > 	local repository.
> 
> This should be certainly doable, but local-arrival may not be
> interesting if the repository has more than one branches.  Maybe
> 
> 	git-local-arrival <committish> [<branch>]
> 
> which defaults to the current branch?

Indeed.  I didn't mention it initially because it is really easy to do 
once you have it working for the current branch.  The technical 
challenge is about making it efficient to find out which reflog entry 
with a path to given commit is the oldest.


Nicolas

Re: git and time

[Junio C Hamano]

Nicolas Pitre <nico@cam.org> writes:

>> Nicolas Pitre <nico@cam.org> writes:
>> 
>> > SYNOPSIS
>> >
>> > 	git-local-arrival <committish>
>> >
>> > DESCRIPTION
>> >
>> > 	The command displays the time when given commit appeared in the 
>> > 	local repository.
>> 
>> This should be certainly doable, but local-arrival may not be
>> interesting if the repository has more than one branches.  Maybe
>> 
>> 	git-local-arrival <committish> [<branch>]
>> 
>> which defaults to the current branch?
>
> Indeed.  I didn't mention it initially because it is really easy to do 
> once you have it working for the current branch.  The technical 
> challenge is about making it efficient to find out which reflog entry 
> with a path to given commit is the oldest.

Perhaps bisect it like this.  This assumes you never rewind the
branch in question and the tip already contains the commit in
question.

-- >8 --

#!/bin/sh

. git-sh-setup
commit=`git rev-parse "$1"` &&
branch="${2-`git symbolic-ref HEAD`} || exit

contains () {
	contains=`git merge-base --all $1 $commit`
	case "$LF$contains$LF" in
	*"$LF$contains$LF"*)
		: ;;
	*)
		false ;;
	esac
}

LF='
'
script='{
	s/	.*//
	p
}' ;# strip things after TAB if a TAB exists
reflog="$GIT_DIR/logs/refs/heads/$branch";
lo=1;
hi=`wc -l <$reflog`
ld=$hi
while expr "$lo" \< "$hi" >/dev/null
do
	mi=`expr \( $hi \+ $lo \) / 2`
	line=`sed -ne "$mi$script" "$reflog"`
	to=`expr "$line" : '[^ ]* \([^ ]*\) '`
	if contains $to
	then
		ld=$hi
		hi=$mi
	else
		lo=`expr "$mi" + 1`
	fi
done

case "$ld" in	
'')
	echo >&2 Oops
	exit 1;;
?*)	
	line=`sed -ne "$ld$script" "$reflog"`
	time=`expr "$line" : '.*> \([0-9]*\) '`
	zone=`expr "$line" : '.*> [0-9]* \(.[0-9][0-9][0-9][0-9]\)$'`
	echo "$time $zone"
esac

Re: git and time

[Junio C Hamano]

Nicolas Pitre <nico@cam.org> writes:

> On Wed, 27 Sep 2006, Junio C Hamano wrote:
>
>> Nicolas Pitre <nico@cam.org> writes:
>> 
>> > SYNOPSIS
>> >
>> > 	git-local-arrival <committish>
>> >
>> > DESCRIPTION
>> >
>> > 	The command displays the time when given commit appeared in the 
>> > 	local repository.
>> 
>> This should be certainly doable, but local-arrival may not be
>> interesting if the repository has more than one branches.  Maybe
>> 
>> 	git-local-arrival <committish> [<branch>]
>> 
>> which defaults to the current branch?
>
> Indeed.  I didn't mention it initially because it is really easy to do 
> once you have it working for the current branch.  The technical 
> challenge is about making it efficient to find out which reflog entry 
> with a path to given commit is the oldest.

The more I think about this, if we were to add yet another
command, I think it should be a command that lets us inspect
ref-log.  We do not have an UI other than @{time} syntax to
interact with it right now.

What are the things we would want?  Here is a strawman.

 - List when and how a branch was changed.

   git ref-log --list --type=merge next (when did I merge into my 'next'?)
   git ref-log --list --type=merge (ditto but any branches)
   git ref-log --list next (any changes not just 'merge')

   I expect the output would give timestamp and reason comment;
   in addition the branch name when no branch is specified.
   Type does not have to be a concrete thing -- it could just be
   a substring match in the reason comment string.

   Also we would limit output with -n <limit>.  The output
   should be sorted by the timestamp of ref-log entry -- we are
   talking about a particular repository's ref-log, so its
   timestamp has more sane meaning than in distributed case.
   
 - Find which branches currently contains a commit, and find the
   earliest time that the commit became part of each of them.

   git ref-log $commit next master (when did it enter 'next' and
                                    when did it graduate to 'master'?)
   git ref-log $commit (ditto but any branches)

   I expect the output to be the timestamp and reason comment;
   in addition the branch name when no branch is specified.

Also for a shared repository, the person who made the change
would be a reasonable thing to report.

So for consistency, in all cases we could make the output
format like this:

    branch SP time-and-zone SP name SP email SP reason-comment LF

where time-and-zone is human-readable timestamp as we see in
git-log output.

Re: git and time

[Shawn Pearce]

Junio C Hamano <junkio@cox.net> wrote:
> The more I think about this, if we were to add yet another
> command, I think it should be a command that lets us inspect
> ref-log.  We do not have an UI other than @{time} syntax to
> interact with it right now.

Agreed.  I've been missing such a command and have wanted to add
one but it wasn't important enough to me to actually code it.  :)
 
> What are the things we would want?  Here is a strawman.
> 
>  - List when and how a branch was changed.
> 
>    git ref-log --list --type=merge next (when did I merge into my 'next'?)
>    git ref-log --list --type=merge (ditto but any branches)
>    git ref-log --list next (any changes not just 'merge')
> 
>    I expect the output would give timestamp and reason comment;
>    in addition the branch name when no branch is specified.
>    Type does not have to be a concrete thing -- it could just be
>    a substring match in the reason comment string.

What about --grep=pat instead of --type?

You are talking about essentially the same behavior as
`git log --grep=pat` except applying it to the message
in the reflog rather than to message in the commits.

Also I think that this should be the default behavior
and thus --list shouldn't be an option.  This matches
git-log's default behavior to just show whatever is
in the named branches.

>    Also we would limit output with -n <limit>.

I'd limit with "--max-count=<n>" like we do with git-log.

>    The output
>    should be sorted by the timestamp of ref-log entry -- we are
>    talking about a particular repository's ref-log, so its
>    timestamp has more sane meaning than in distributed case.

Agreed, sorting newest -> oldest so newest displays first, much as
git-log does.  This way its order of operation, much as git-log is
order of operation.

If multiple branches are specified we really should interleave the
various reflogs according to timestamps, to show the "global picture"
of what happened in this repository.

>  - Find which branches currently contains a commit, and find the
>    earliest time that the commit became part of each of them.
> 
>    git ref-log $commit next master (when did it enter 'next' and
>                                     when did it graduate to 'master'?)
>    git ref-log $commit (ditto but any branches)

Since I'm suggesting above that this behavior not be the default
what about:

    git ref-log --arrive=$commit next master
    git ref-log --arrive=$commit
?

>    I expect the output to be the timestamp and reason comment;
>    in addition the branch name when no branch is specified.

Agreed.

> Also for a shared repository, the person who made the change
> would be a reasonable thing to report.

I think that should be shown no matter what; even if
core.sharedRepository is false.

> So for consistency, in all cases we could make the output
> format like this:
> 
>     branch SP time-and-zone SP name SP email SP reason-comment LF

That's too long of a line with most reason-comments in the ref-log.
Especially ones that come from git-commit, and especially if they
were human written commit messages.

I'd like to see the output be more like git-log.  Allow a --pretty
option with a few useful formats:

  --pretty=full:
     branch branchname LF
     from old
     to new
     Modifier: name SP email
     Date: time-and-zone

         reason-comment

  --pretty=oneline:
     branchname SP time-and-zone SP name SP email SP reason-comment LF

  --pretty=raw is obviously the exact line in the reflog but with
  the branch name preceeding it if more than one branch was specified
  or none were specified.

And --pretty=full should be the default, much as --pretty=medium
is with git-log.

-- 
Shawn.

Re: git and time

[Junio C Hamano]

Shawn Pearce <spearce@spearce.org> writes:

> Agreed.  I've been missing such a command and have wanted to add
> one but it wasn't important enough to me to actually code it.  :)

Everything you said in your message sounds sane and makes sense
to me.  Now we have to find a sucker^Wvolunteer to implement it
;-).

Re: git and time

[Linus Torvalds]

On Sat, 30 Sep 2006, Junio C Hamano wrote:

> Shawn Pearce <spearce@spearce.org> writes:
> 
> > Agreed.  I've been missing such a command and have wanted to add
> > one but it wasn't important enough to me to actually code it.  :)
> 
> Everything you said in your message sounds sane and makes sense
> to me.  Now we have to find a sucker^Wvolunteer to implement it
> ;-).

Ehh. As far as I can see it's 
 - a damn hard thing to do efficiently
 - essentially exactly the same problem you already solved with "git 
   describe"

In other words, I think you could make git describe do it, by simply 
making it parse not just all tags, but also walking the branch log.

		Linus

Re: git and time

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> On Sat, 30 Sep 2006, Junio C Hamano wrote:
>
>> Shawn Pearce <spearce@spearce.org> writes:
>> 
>> > Agreed.  I've been missing such a command and have wanted to add
>> > one but it wasn't important enough to me to actually code it.  :)
>> 
>> Everything you said in your message sounds sane and makes sense
>> to me.  Now we have to find a sucker^Wvolunteer to implement it
>> ;-).
>
> Ehh. As far as I can see it's 
>  - a damn hard thing to do efficiently
>  - essentially exactly the same problem you already solved with "git 
>    describe"
>
> In other words, I think you could make git describe do it, by simply 
> making it parse not just all tags, but also walking the branch log.

As a user interface, I think it makes a lot of sense to have
"git describe" do it without introducing a new command.

However, I think the traditional "find the closest ancestor"
behaviour and ref-log behaviour are mutually incompatible, while
they both return information to help address similar issues to
the end user when viewed at a very high level.

Especially, "find the closest ancestor" behaviour means when you
get "tag-gXXXX" as an answer, the tag proper does _not_ contain
the given commit (e.g. commit v1.4.2-g4839bd8 is not part of
v1.4.2).  To answer "when did the fix deadbeef go into master
branch", reporting "master@{yesterday}-gdeadbeef" with the same
logic and format is misleading; "master@{yesterday}" may be the
closest ancestor of commit deadbeef, but that means it does
_not_ contain the fix.  When walking ref-log, we want it the
other way around: "find the earliest descendant among the
entries in ref-log for a particular branch".

The internal logic for doing that may be somewhat different and
I suspect you may not be able to share much code with the
existing logic..

Re: git and time

[Linus Torvalds]

On Sat, 30 Sep 2006, Junio C Hamano wrote:
> 
> However, I think the traditional "find the closest ancestor"
> behaviour and ref-log behaviour are mutually incompatible, while
> they both return information to help address similar issues to
> the end user when viewed at a very high level.
> 
> Especially, "find the closest ancestor" behaviour means when you
> get "tag-gXXXX" as an answer, the tag proper does _not_ contain
> the given commit (e.g. commit v1.4.2-g4839bd8 is not part of
> v1.4.2).

Correct.

But that just means that we should take the _next_ one in the time-ordered 
list, no?

		Linus

Re: git and time

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> On Sat, 30 Sep 2006, Junio C Hamano wrote:
>> 
>> However, I think the traditional "find the closest ancestor"
>> behaviour and ref-log behaviour are mutually incompatible, while
>> they both return information to help address similar issues to
>> the end user when viewed at a very high level.
>> 
>> Especially, "find the closest ancestor" behaviour means when you
>> get "tag-gXXXX" as an answer, the tag proper does _not_ contain
>> the given commit (e.g. commit v1.4.2-g4839bd8 is not part of
>> v1.4.2).
>
> Correct.
>
> But that just means that we should take the _next_ one in the time-ordered 
> list, no?

I do not think so.

Extending the example (sorry for doing the same topic on two
separate threads) I just gave Jeff on "fix based on v0.99",
after finding that the fix is based on v0.99, finding another
commit that immediately followed the v0.99 commit on my master
branch does not help finding out that I very recently merged the
fix in at all.  I think we cannot get away without honestly
doing the first descendant, which is unfortunately a lot more
expensive.

Re: git and time

[Junio C Hamano]

Junio C Hamano <junkio@cox.net> writes:

> Linus Torvalds <torvalds@osdl.org> writes:
>
>> On Sat, 30 Sep 2006, Junio C Hamano wrote:
>>> 
>>> Especially, "find the closest ancestor" behaviour means when you
>>> get "tag-gXXXX" as an answer, the tag proper does _not_ contain
>>> the given commit (e.g. commit v1.4.2-g4839bd8 is not part of
>>> v1.4.2).
>>
>> Correct.
>>
>> But that just means that we should take the _next_ one in the time-ordered 
>> list, no?
>
> I do not think so.
>
> Extending the example (sorry for doing the same topic on two
> separate threads) I just gave Jeff on "fix based on v0.99",
> after finding that the fix is based on v0.99, finding another
> commit that immediately followed the v0.99 commit on my master
> branch does not help finding out that I very recently merged the
> fix in at all.  I think we cannot get away without honestly
> doing the first descendant, which is unfortunately a lot more
> expensive.

Maybe not *that* expensive.  Here is an outline, thinking aloud.

When describing a commit and a ref, we first run the ancestry
traversal algorithm merge-base uses internally.  If the tip of
the ref is not a descendant of the commit, abort (I'll justify
this in a moment).

Otherwise, we would already have parsed all the necessary
commits we need to determine which commits on the given ref's
ancestry is the first one that is a descendant of the target
commit at this point.  We collect these commits in a set, and
then mark the ones that are immediate children of the target
commit, then the ones that are children of them, etc. until we
find all the descendant of the target commit.

After that, we can bisect the reflog for the ref to find the
first commit that we have marked as a descendant of the target
commit in the above process.

If the tip of the ref is not a descendant of the commit to begin
with, that does not automatically mean that the target commit
has never been part of the ref -- the ref _could_ have contained
it and then later got rewound.  But then the question "when did
the commit has become part of this branch" itself stops being
interesting.  It would not do us much good if we know it was
part of the branch for only two days last week but it is not
contained in the branch anymore.

Re: git and time

[Johannes Schindelin]

Hi,

On Sat, 30 Sep 2006, Junio C Hamano wrote:

> I think we cannot get away without honestly doing the first descendant, 
> which is unfortunately a lot more expensive.

... and which happens to be almost implemented in git-name-rev.

Ciao,
Dscho

Re: git and time

[Jakub Narebski]

Junio C Hamano wrote:

> Linus Torvalds <torvalds@osdl.org> writes:
> 
>> On Sat, 30 Sep 2006, Junio C Hamano wrote:
>>
>>> Shawn Pearce <spearce@spearce.org> writes:
>>> 
>>> > Agreed.  I've been missing such a command and have wanted to add
>>> > one but it wasn't important enough to me to actually code it.  :)
>>> 
>>> Everything you said in your message sounds sane and makes sense
>>> to me.  Now we have to find a sucker^Wvolunteer to implement it
>>> ;-).
>>
>> Ehh. As far as I can see it's 
>>  - a damn hard thing to do efficiently
>>  - essentially exactly the same problem you already solved with "git 
>>    describe"
>>
>> In other words, I think you could make git describe do it, by simply 
>> making it parse not just all tags, but also walking the branch log.
> 
> As a user interface, I think it makes a lot of sense to have
> "git describe" do it without introducing a new command.
> 
> However, I think the traditional "find the closest ancestor"
> behaviour and ref-log behaviour are mutually incompatible, while
> they both return information to help address similar issues to
> the end user when viewed at a very high level.
> 
> Especially, "find the closest ancestor" behaviour means when you
> get "tag-gXXXX" as an answer, the tag proper does _not_ contain
> the given commit (e.g. commit v1.4.2-g4839bd8 is not part of
> v1.4.2).  To answer "when did the fix deadbeef go into master
> branch", reporting "master@{yesterday}-gdeadbeef" with the same
> logic and format is misleading; "master@{yesterday}" may be the
> closest ancestor of commit deadbeef, but that means it does
> _not_ contain the fix.  When walking ref-log, we want it the
> other way around: "find the earliest descendant among the
> entries in ref-log for a particular branch".
> 
> The internal logic for doing that may be somewhat different and
> I suspect you may not be able to share much code with the
> existing logic..

Isn't git-name-rev doing kind of earliest descendant among refs?
Well, I'm not sure if name-rev does not use shortest description
instead of earliest (closest) descendant... but we could extend it.
I'd like also to limit refs used to given pattern, and perhaps
also to only tag objects.

This is command to extend it using it together with ref-log to know
when given fix appeared in repository.
-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: git and time

[Jakub Narebski]

Matthew L. Foster wrote:
> If the local merge time information is already available in the
> ref-log then gitweb.cgi might only need to be made aware of it. 

It is planned to add reflog support (view) to gitweb.

But of course the repository that is under gitweb has to have reflog 
_enabled_ to be able to view it.
-- 
Jakub Narebski
Poland

Re: git and time

[Shawn Pearce]

Matthew L Foster <mfoster167@yahoo.com> wrote:
> --- Linus Torvalds <torvalds@osdl.org> wrote:
> 
> >  - so I replicate my home machine tree to the one on master.kernel.org, 
> >    and now others can see it.
> > 
> > When did the merge happen? It happened at 8:30 on my machine, and that's 
> > what is recorded. End of story. No ifs, buts, maybes about it. That's the 
> > only time you can _ever_ see for that merge.
> 
> Ok, so it's more complex because of the workflow issue of delayed/pseudo mirroring/replication
> between private and public repos?

No.  Its no different than anything else.  Linus' personal repository
is just as visible to you as say Andrew Morton's personal repository
(read: neither one is visible to you).  Therefore the date/time
that a given commit hits either one of those repositories is only
of interest to the people with direct access to it.  Which is Linus
and Andrew respectively.  And probably nobody else.

When Linus pushes all of those merges out to kernel.org those
commits are arriving at "his" kernel.org repository at a date/time
that is absolutely after when they first arrived in Linus' personal
repository.  Note that the kernel.org repository is a completely
different repository from Linus' personal work repository!

But due to clock skew, time zone differences, etc. between systems
those commits may actually appear to arrive at kernel.org before,
at the same time as, or after they arrived on Linus' personal system.
:-)

> This cloning/replication is not done through git?

I don't know.  I'd wager Linus is probably using Git to push changes
to some repository on a master system behind the kernel.org domain
name, and that master then gets replicated out to mirror systems
through some form of replication.

> Are you saying it's impossible for master.kernel.org's git to track the local time of each
> commit/merge/replication? Perhaps replication time is precisely what should/could be tracked
> (locally)? 

I don't think the replication time is really important here.  If it
was then Git should be used for the replication of Git repositories
being mirrored on kernel.org.  I doubt you'd get every mirror
operator to do that however.

> From an integrity or at least gitweb.cgi's viewpoint it seems very important to me that commit
> order also be per repo consistent with time order.

I'm not sure I follow you.  Time order has nothing to do with
anything.  That's largely been the entire point of this thread.

Because of the potentical for clock skew even on a single system
you can't take much stock in a timestamp.  But with Git you can at
least completely trust the commit graph, provided that you trust
those who made commits before your own commit.  Of course this
trust is only possible because the commit graph cannot be altered
once a node has been added into it.

As such the commit graph is consistent between repositories (assuming
they have the same head commits), but the timestamps of the reflogs
within each will widely differ.  They could widely differ even on
the same system due to ntpd updating the clock at the exact wrong
moment for example.  :)

-- 
Shawn.

Re: git and time

[Matthew L Foster]

--- Shawn Pearce <spearce@spearce.org> wrote:

> Because of the potentical for clock skew even on a single system
> you can't take much stock in a timestamp.  But with Git you can at
> least completely trust the commit graph, provided that you trust
> those who made commits before your own commit.  Of course this
> trust is only possible because the commit graph cannot be altered
> once a node has been added into it.
> 
> As such the commit graph is consistent between repositories (assuming
> they have the same head commits), but the timestamps of the reflogs
> within each will widely differ.  They could widely differ even on
> the same system due to ntpd updating the clock at the exact wrong
> moment for example.  :)

I am not arguing for git to try to achieve "exact" time just merely locally time consistent commit
order. This might all just be a gitweb.cgi time display issue, it should be more impossible for a
commit to appear as being made 2 days in the future and impossible for local time order to be out
of sync with commit order. If each git repo used local time to track commits/merges you wouldn't
have to worry if any remote git server's time was grossly misconfigured. Time doesn't need to be
exact, all I am saying is each git repo should trust/prefer its local time rather than a remote
git server's timestamp.

-Matt


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: git and time

[Shawn Pearce]

Matthew L Foster <mfoster167@yahoo.com> wrote:
> --- Shawn Pearce <spearce@spearce.org> wrote:
> 
> > Because of the potentical for clock skew even on a single system
> > you can't take much stock in a timestamp.  But with Git you can at
> > least completely trust the commit graph, provided that you trust
> > those who made commits before your own commit.  Of course this
> > trust is only possible because the commit graph cannot be altered
> > once a node has been added into it.
> > 
> > As such the commit graph is consistent between repositories (assuming
> > they have the same head commits), but the timestamps of the reflogs
> > within each will widely differ.  They could widely differ even on
> > the same system due to ntpd updating the clock at the exact wrong
> > moment for example.  :)
> 
> I am not arguing for git to try to achieve "exact" time just merely locally time consistent commit
> order. This might all just be a gitweb.cgi time display issue, it should be more impossible for a
> commit to appear as being made 2 days in the future and impossible for local time order to be out
> of sync with commit order. If each git repo used local time to track commits/merges you wouldn't
> have to worry if any remote git server's time was grossly misconfigured. Time doesn't need to be
> exact, all I am saying is each git repo should trust/prefer its local time rather than a remote
> git server's timestamp.

Git does has local time order, so long as the user doesn't screw
around with their clock.

Each time a commit gets made (or a merge gets performed) Git takes
the local system clock and dumps into the new commit as part of
the "committer" comment.  But as has been stated many times before
in this thread, this is no more trustworthy than the username and
email address also appearing in that "committer" line and its not
relied upon by Git.  Some interfaces may try to sort based on this
timestamp but only to help it break ties.  The dependency graph
always wins as that's always correct.

I have a Git repository from which I'm tracking an SVN repository.
Recently that SVN repository gave me "(no date)" as a datestamp.
Git converted that to Jan 1, 1970.  That commit has a parent and
has a child, both of which have sane timestamps.  But viewing this
in gitk or git log its obvious that the Jan 1, 1970 commit is in
the right position within the graph, it just has a timestamp that
I can't trust as its 36 years in the past.

There's nothing I can do about that timestamp.  It came from another
system that I have no control over.  But Git accurately recorded
what that other system provided it.  The "truth" here is that other
system provided a bogus timestamp.

-- 
Shawn.

Re: git and time

[Andy Whitcroft]

Matthew L Foster wrote:
>> Because git doesn't care about timestamps. It stores them as comments 
>> (albeit auto-formatted comments) and relies on the dependency chain to 
>> provide history.
> 
> Ok, the word "history" in the context of git primarily means the order of changes not the when?
> Would it be a conceptual or technical issue for git to directly track the local time of
> merges/changesets?

It is tracking the local times of each change as it is added to the
dependancy chain.  This chain then moves about between repositories
carrying its stamp with it.  When we merge a set of changes into a trunk
such as Linus does that merge will be stamped by him saying when he
merged it.  So there is plenty of time stuff in there.

Of course none of it tells you when the kernel you are running has it
in.  The only way to know that is to know when the thing was released,
under what version#, and what version you are running.

Now when we make a signed tag, doen't that make a new object too and I
assume that has a tagged date in it.  That time might really actually
mean something and a fix's relation ship to those tags might also mean
something.

-apw

Re: git and time

[Linus Torvalds]

On Wed, 27 Sep 2006, Matthew L Foster wrote:
> 
> Perhaps git should record three(+) timestamps, adding when the change was committed into this
> repository?

If you ask for logging, that's exactly what you get.

Well, almost. 

You cannot connect the time to a _commit_, since a commit (very much by 
design) is totally immutable. Once you have created any git object, it's 
done. It can never be changed ever again, and this is not just a small 
detail, it's what the whole system builds up on, and it's where the 
security and trustworthiness fundamentally comes from (it's also where the 
naming scheme comes from - things are named by their contents, so if you 
were to ever change them again, they'd have to be renamed - they 
effectively _become_ something different).

Also, you fundamentally couldn't do it anyway, since the third time isn't 
actually even well-defined in a distributed manner. It only makes sense 
very much in a local way, and as such can never be part of the distributed 
data - and trying to add something like that to the commit would be 
fundamentally incorrect.

BUT.

What you CAN do is to connect (in any particular private repository) a 
_branch_update_ with the time it was done. That is Shawn Pierces "reflog" 
work - you can track a particular branch _locally_. It's purely local to 
that _one_ repository, though. It by definition makes no sense anywhere 
else, and it's not tracking commits, it's literally tracking how branches 
changed in a local copy.

To enable it, just add a

	[core]
		logAllRefUpdates=true

thing to your .git/config file (or, if you want to do it for _all_ the 
projects you track, you can just do it in your ~/.gitconfig file, and it 
should be the default for everything you do).

(Althernatively, you can choose to log just a _single_ branch by just 
creating the ".git/logs/refs/heads/<branchname>" file - git should start 
logging that branch automatically)

This is a reasonably new feature, so old git versions need not apply.

			Linus

Re: git and time

[Edgar Toernig]

Linus Torvalds wrote:
>
> On Wed, 27 Sep 2006, Matthew L Foster wrote:
> > 
> > Perhaps git should record three(+) timestamps, adding when the change was committed into this
> > repository?
> 
> What you CAN do is to connect (in any particular private repository) a 
> _branch_update_ with the time it was done. That is Shawn Pierces "reflog" 
> work - you can track a particular branch _locally_. It's purely local to 
> that _one_ repository, though. It by definition makes no sense anywhere 
> else, and it's not tracking commits, it's literally tracking how branches 
> changed in a local copy.

Well, I would simply look at the filesystem's mtime of the commit object
resp. the pack containing the commit.  IMHO good enough most of the time.

Ciao, ET.

Re: git and time

[Linus Torvalds]

On Wed, 27 Sep 2006, Edgar Toernig wrote:
> 
> Well, I would simply look at the filesystem's mtime of the commit object
> resp. the pack containing the commit.  IMHO good enough most of the time.

Nope. The moment you repack, you're toast. And if you don't repack, please 
don't use large repositories.

Here's a real-world example: in the week since 2.6.18 was released, the 
kernel has gotten over twenty _thousand_ new objects. If you don't repack, 
you'll have lost about 40MB of diskspace. It adds up.

So yes, mtime works for a bit. And then it stops working ;)

		Linus

Re: git and time

[Jakub Narebski]

Junio C. Hamano wrote:
> I somehow thought that it was possible to get "the latest tag
> that precedes this commit" (aka "git describe") for each commit
> by visiting its commitdiff_plain page, but I do not see it now.
> Can somebody tell me if I am hallucinating?

No, as of now "commitdiff_plain" or "commit_plain" view shows either 
git-name-rev information, or just tag if the tag points exactly at 
given [child] commit, not git-describe information. Although it would 
be fairly easy to add this information, though...
-- 
Jakub Narebski
Poland

Re: git and time

[Linus Torvalds]

On Tue, 26 Sep 2006, Matthew L Foster wrote:
> 
> It's true I don't know much about git, what is the difference between a 
> changeset and a snapshot?

Some of it is just semantic, but a lot of it has real user-visible meaning 
simply because of the "mental model" difference, so the semantics actually 
have some meaning.

A lot of systems think of commits as "what changed", and thus the 
"changeset" mentality. A "commit" is just the combination of all changes 
that that commit introduced.

Git very fundamentally does not think like that at all.

Git thinks of a commit as a _state_, and the history that led up to that 
state. So instead of the commit actually containing pointers to what 
changed, it very much contains a pointer directly to the actual state that 
was committed (a "tree" in git parlance), and then a set of pointers to 
the "parent" commits - the commits that explain where we came from.

Now, in some sense, you can ignore the difference between the two models, 
since you'd think that they are totally equivalent: from the git model, 
you can always get the "changeset" by just diffing the current state with 
the previous state, and conversely from the "changeset" model you can 
always get the "current state" by just applying the changeset to the 
previous state.

So in that sense, it's just two different ways of looking at exactly the 
same thing.

HOWEVER. The fact that git internally thinks in terms of "snapshots" means 
that it makes no sense to (for example) record a "file rename". Git 
figures it out on its own, by just looking at the state before and after. 
The great thing about that is that the exact same logic actually works 
even for _unconnected_ states/snapshots, in a way that a "changeset" based 
situation would find very hard.

So this is when the otherwise semantic difference actually shows itself. 
You can diff between two arbitrary points in time, and git will figure out 
renames on its own, without actually ever looking at the changesets in 
between (in fact, there may not even _be_ a straight, unbroken chain of 
changesets between the two states).

> Are you saying timestamps should be tracked separately or tracked by an 
> scm system built on top of git? Does/should git care about the when of a 
> snapshot?

Git does record the timestamp, but it records it in the same way it 
records the "username" - in that it doesn't really _matter_ to git. It 
never actually affects any meaning (well, since you can query for it, it 
has a meaning of sorts, but it's strictly limited to any explicit queries, 
so if you do "git log --since=2.weeks.ago" it will use the timestamp to 
give you what you want, but it doesn't actually affect anything 
important).

So think of the timestamps as just comments with a very specific format.

		Linus

Re: git and time

[Jakub Narebski]

Linus Torvalds wrote:

> Now, in some sense, you can ignore the difference between the two models, 
> since you'd think that they are totally equivalent: from the git model, 
> you can always get the "changeset" by just diffing the current state with 
> the previous state, and conversely from the "changeset" model you can 
> always get the "current state" by just applying the changeset to the 
> previous state.

And if I understand correctly, that is how StGit and pg (Patchy Git), which
are patch management applications similar in the purpose to the Quilt, and
are based on Git, works.

http://wiki.procode.org/cgi-bin/wiki.cgi/StGITtheory
-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: git and time

[Jeff King]

On Tue, Sep 26, 2006 at 05:27:45PM -0700, Matthew L Foster wrote:

> It's true I don't know much about git, what is the difference between
> a changeset and a snapshot?

I think Linus' explanation covered what I meant, but please ask for
clarification if there was something that didn't make sense.

> Are you saying timestamps should be tracked separately or tracked by
> an scm system built on top of git?  Does/should git care about the
> when of a snapshot?

Yes, they could be tracked separately. My point was that git deals in
immutable snapshot objects (commits) and you don't want to change the
commit objects after the fact. You could certainly make an external
mapping of "this commit object entered the repo at local time T" but I
doubt it would be of much use. See below.

> Perhaps my question is directed more toward gitweb.cgi, it seems to me
> the timestamp of when a snapshot was merged into this repository
> should somehow be tracked and that is what gitweb.cgi should default
> to display. For example, if someone wants to know if security bugfix X
> was merged into linus' kernel tree they also want to know when that
> happened, don't they?

Right. So you really want to know not "when did this commit enter this
repo" but rather "when did this head/branch first contain this commit"
(since there may be multiple branches within a repo).  We can find out
"does commit X contain commit Y" by looking at the commit graph. The
reflog system records "head H contained commit X at time T" so between
the two you can find the answer to your question (but it takes some
computation).

I think Junio's email explained this better than I could, but again,
please ask if something is unclear.

-Peff

Re: git and time

[Sean]

On Tue, 26 Sep 2006 23:34:59 -0400
Jeff King <peff@peff.net> wrote:

> Right. So you really want to know not "when did this commit enter this
> repo" but rather "when did this head/branch first contain this commit"
> (since there may be multiple branches within a repo).

Even though it's being a bit pedantic, I have to disagree with you here.
The question the user is asking is exactly, "When did this commit enter
_this_ repo?".

Because of the design of git, such a question must be converted into a
question regarding reflogs and head/branch values etc...  But the user
doesn't care anything about all that.  They're just interested in the
date/time the commit was published in the repository in question, not
the date time the commit was originally created in some distant
repo.

Sean

Re: git and time

[Junio C Hamano]

Sean <seanlkml@sympatico.ca> writes:

> On Tue, 26 Sep 2006 23:34:59 -0400
> Jeff King <peff@peff.net> wrote:
>
>> Right. So you really want to know not "when did this commit enter this
>> repo" but rather "when did this head/branch first contain this commit"
>> (since there may be multiple branches within a repo).
>
> Even though it's being a bit pedantic, I have to disagree with you here.
> The question the user is asking is exactly, "When did this commit enter
> _this_ repo?".
>
> Because of the design of git, such a question must be converted into a
> question regarding reflogs and head/branch values etc...  But the user
> doesn't care anything about all that.  They're just interested in the
> date/time the commit was published in the repository in question, not
> the date time the commit was originally created in some distant
> repo.

I disagree.

For somebody who is tracking my "master" branch, it does not
matter when some critical fix appeared on my "next" branch.  The
user will be vulnerable until that fix makes its way to the
"master" branch.  The user _can_ switch to track my "next"
branch, but that is like arguing that the user can apply the
same patch to his local copy that tracks my "master".

I may try-pull from Paul to get updates from gitk, but usually I
do that with "git fetch", not "git pull".  So my repository may
contain commits and blobs for the latest and greatest gitk, but
until I merge it and push the result out nobody would benefit
from it through my repository.

See?  Having a commit somewhere in the repository does not make
any difference unless that commit is on some branches you care
about.

Re: git and time

[Sean]

On Tue, 26 Sep 2006 21:50:51 -0700
Junio C Hamano <junkio@cox.net> wrote:

> For somebody who is tracking my "master" branch, it does not
> matter when some critical fix appeared on my "next" branch.  The
> user will be vulnerable until that fix makes its way to the
> "master" branch.  The user _can_ switch to track my "next"
> branch, but that is like arguing that the user can apply the
> same patch to his local copy that tracks my "master".
> 
> I may try-pull from Paul to get updates from gitk, but usually I
> do that with "git fetch", not "git pull".  So my repository may
> contain commits and blobs for the latest and greatest gitk, but
> until I merge it and push the result out nobody would benefit
> from it through my repository.
> 
> See?  Having a commit somewhere in the repository does not make
> any difference unless that commit is on some branches you care
> about.

Well, yes.  And all of my examples have assumed the example of
Linus' repository where there is only one branch.  So yes, in the
case where there are more than one branch, you want to be able
to ask the more specific question, when did this commit arrive
into this repo-branch.

But that is really the minutia of the issue.  First we have to
agree that users _do_ want to know the date of commits beyond
just those recorded inside the commit itself.

If we do agree on that point, then the rest is just the details
of how plausible it is to provide those answers.   Shawn has made
it clear that the reflog doesn't really have all the information
we need yet.  On top of which it would be expensive to compute
etc.

Sean

Re: git and time

[Jeff King]

On Wed, Sep 27, 2006 at 01:04:37AM -0400, Sean wrote:

> Well, yes.  And all of my examples have assumed the example of
> Linus' repository where there is only one branch.  So yes, in the
> case where there are more than one branch, you want to be able
> to ask the more specific question, when did this commit arrive
> into this repo-branch.

Yes, that is what I was trying to point out by making the branch/repo
distinction in my previous mail.

> If we do agree on that point, then the rest is just the details
> of how plausible it is to provide those answers.   Shawn has made
> it clear that the reflog doesn't really have all the information
> we need yet.  On top of which it would be expensive to compute
> etc.

We should be able to make a naive space-time tradeoff: whenever a ref is
updated from X to Y at time T, for each commit C in X..Y, mark the tuple
(ref, C) with time T. Assuming a reasonably packed format (20 bytes of
SHA1, 4 bytes of time, sorted into .git/time-cache/master) the git
repository would require less than 1M per branch. Updating the ref
becomes much more expensive, but looking up the value is quite cheap.

Of course, rewinding would make this more complicated. I'm still not
convinced this approach is worth the effort.

-Peff

Re: git and time

[Sean]

On Wed, 27 Sep 2006 01:52:16 -0400
Jeff King <peff@peff.net> wrote:

> Yes, that is what I was trying to point out by making the branch/repo
> distinction in my previous mail.

Yeah, Junio's response made me realize that.  Apologies.

> We should be able to make a naive space-time tradeoff: whenever a ref is
> updated from X to Y at time T, for each commit C in X..Y, mark the tuple
> (ref, C) with time T. Assuming a reasonably packed format (20 bytes of
> SHA1, 4 bytes of time, sorted into .git/time-cache/master) the git
> repository would require less than 1M per branch. Updating the ref
> becomes much more expensive, but looking up the value is quite cheap.

Yes.  It should be relatively straight forward, and removes the need to
use the reflog at all.  It's the right tradeoff since the value would
be queried much more often than updated.
 
> Of course, rewinding would make this more complicated. I'm still not
> convinced this approach is worth the effort.

You could almost ignore rewinding without too big a problem, or at least
only deal with it when performing a prune.

As for being worth the effort, we'll have to see if anyone steps up to
actually offer a patch.  If not, you're right again :o)

Sean

Re: git and time

[Shawn Pearce]

Sean <seanlkml@sympatico.ca> wrote:
> On Tue, 26 Sep 2006 23:34:59 -0400
> Jeff King <peff@peff.net> wrote:
> 
> > Right. So you really want to know not "when did this commit enter this
> > repo" but rather "when did this head/branch first contain this commit"
> > (since there may be multiple branches within a repo).
> 
> Even though it's being a bit pedantic, I have to disagree with you here.
> The question the user is asking is exactly, "When did this commit enter
> _this_ repo?".
> 
> Because of the design of git, such a question must be converted into a
> question regarding reflogs and head/branch values etc...  But the user
> doesn't care anything about all that.  They're just interested in the
> date/time the commit was published in the repository in question, not
> the date time the commit was originally created in some distant
> repo.

And with the completely distributed nature of Git I have to say
that's a useful question to get an answer to.  Though I believe
Junio pointed out that may be incorrect by several hours for
Linus' public tree due to the mirroring that occurs on kernel.org.
However so long as the existance of that lag is publically stated
I don't really see a problem.

As Junio correctly pointed out answering that question is rather
compute intensive (as Git things go) as you need to compare both
the reflog for the branch in question (or all branches!) against
the commit chain(s) and find where that commit became visible.
Not cheap certainly but possible with the data we have.


However people are ignoring the fact that receive-pack doesn't
update the reflog.  As of current `next` its *still* doing the ref
updates by hand, rather than going through the common library code
in refs.c.  As a consequence its bypassing the reflog implementation.
This means that `git-push` does not update the reflog.  Which means
there's no way Linus' public repository could have a reflog.

Maybe if I (or someone else) fixed receive-pack.c Linus might
consider enabling reflog, and maybe someone could fix gitweb.cgi to
perform the computation - but possibly server owners wouldn't want
that feature running due to its high computation cost.  Especially
if gitweb.cgi doesn't have a native C executable to do most of the
work for it.

For what its worth I keep meaning to get around to fix receive-pack.c
but I just haven't done it yet.

-- 
Shawn.

Re: git and time

[Junio C Hamano]

Shawn Pearce <spearce@spearce.org> writes:

> For what its worth I keep meaning to get around to fix receive-pack.c
> but I just haven't done it yet.

I've been futzing around that code because I wanted to make sure
we get the locking right on creation and deletion of refs.  I'll
try to remember when I revisit the code tomorrow.

Re: git and time

[Junio C Hamano]

Shawn Pearce <spearce@spearce.org> writes:

> However people are ignoring the fact that receive-pack doesn't
> update the reflog.  As of current `next` its *still* doing the ref
> updates by hand, rather than going through the common library code
> in refs.c.

This is unfortunately on top of many things, but judging from
the number of deleted lines and added lines, I think it is going
in the right direction.

One thing that makes "the common library code" less useful is
that lock_ref_sha1() and its cousin lock_any_ref_for_update() do
not let the caller to tell why a ref could not be locked ("did
it not exist?  did the old_sha1 not match?"  and in
lock_ref_sha1()'s case "did the ref have funny characters?").

-- >8 --
[PATCH] Teach receive-pack about ref-log

This converts receive-pack to use the standard ref locking code
instead of its own.  As a side effect, it automatically records
the "push" event to ref-log if enabled.

Signed-off-by: Junio C Hamano <junkio@cox.net>
---
 receive-pack.c |   88 ++++++++++----------------------------------------------
 1 files changed, 15 insertions(+), 73 deletions(-)

diff --git a/receive-pack.c b/receive-pack.c
index abbcb6a..f0b4cb4 100644
--- a/receive-pack.c
+++ b/receive-pack.c
@@ -41,34 +41,6 @@ struct command {
 
 static struct command *commands;
 
-static int is_all_zeroes(const char *hex)
-{
-	int i;
-	for (i = 0; i < 40; i++)
-		if (*hex++ != '0')
-			return 0;
-	return 1;
-}
-
-static int verify_old_ref(const char *name, char *hex_contents)
-{
-	int fd, ret;
-	char buffer[60];
-
-	if (is_all_zeroes(hex_contents))
-		return 0;
-	fd = open(name, O_RDONLY);
-	if (fd < 0)
-		return -1;
-	ret = read(fd, buffer, 40);
-	close(fd);
-	if (ret != 40)
-		return -1;
-	if (memcmp(buffer, hex_contents, 40))
-		return -1;
-	return 0;
-}
-
 static char update_hook[] = "hooks/update";
 
 static int run_update_hook(const char *refname,
@@ -105,8 +77,8 @@ static int update(struct command *cmd)
 	const char *name = cmd->ref_name;
 	unsigned char *old_sha1 = cmd->old_sha1;
 	unsigned char *new_sha1 = cmd->new_sha1;
-	char new_hex[60], *old_hex, *lock_name;
-	int newfd, namelen, written;
+	char new_hex[41], old_hex[41];
+	struct ref_lock *lock;
 
 	cmd->error_string = NULL;
 	if (!strncmp(name, "refs/", 5) && check_ref_format(name + 5)) {
@@ -115,59 +87,27 @@ static int update(struct command *cmd)
 			     name);
 	}
 
-	namelen = strlen(name);
-	lock_name = xmalloc(namelen + 10);
-	memcpy(lock_name, name, namelen);
-	memcpy(lock_name + namelen, ".lock", 6);
-
 	strcpy(new_hex, sha1_to_hex(new_sha1));
-	old_hex = sha1_to_hex(old_sha1);
+	strcpy(old_hex, sha1_to_hex(old_sha1));
 	if (!has_sha1_file(new_sha1)) {
 		cmd->error_string = "bad pack";
 		return error("unpack should have generated %s, "
 			     "but I can't find it!", new_hex);
 	}
-	safe_create_leading_directories(lock_name);
-
-	newfd = open(lock_name, O_CREAT | O_EXCL | O_WRONLY, 0666);
-	if (newfd < 0) {
-		cmd->error_string = "can't lock";
-		return error("unable to create %s (%s)",
-			     lock_name, strerror(errno));
-	}
-
-	/* Write the ref with an ending '\n' */
-	new_hex[40] = '\n';
-	new_hex[41] = 0;
-	written = write(newfd, new_hex, 41);
-	/* Remove the '\n' again */
-	new_hex[40] = 0;
-
-	close(newfd);
-	if (written != 41) {
-		unlink(lock_name);
-		cmd->error_string = "can't write";
-		return error("unable to write %s", lock_name);
-	}
-	if (verify_old_ref(name, old_hex) < 0) {
-		unlink(lock_name);
-		cmd->error_string = "raced";
-		return error("%s changed during push", name);
-	}
 	if (run_update_hook(name, old_hex, new_hex)) {
-		unlink(lock_name);
 		cmd->error_string = "hook declined";
 		return error("hook declined to update %s", name);
 	}
-	else if (rename(lock_name, name) < 0) {
-		unlink(lock_name);
-		cmd->error_string = "can't rename";
-		return error("unable to replace %s", name);
-	}
-	else {
-		fprintf(stderr, "%s: %s -> %s\n", name, old_hex, new_hex);
-		return 0;
+
+	lock = lock_any_ref_for_update(name, old_sha1);
+	if (!lock) {
+		cmd->error_string = "failed to lock";
+		return error("failed to lock %s", name);
 	}
+	write_ref_sha1(lock, new_sha1, "push");
+
+	fprintf(stderr, "%s: %s -> %s\n", name, old_hex, new_hex);
+	return 0;
 }
 
 static char update_post_hook[] = "hooks/post-update";
@@ -318,9 +258,11 @@ int main(int argc, char **argv)
 	if (!dir)
 		usage(receive_pack_usage);
 
-	if(!enter_repo(dir, 0))
+	if (!enter_repo(dir, 0))
 		die("'%s': unable to chdir or not a git archive", dir);
 
+	git_config(git_default_config);
+
 	write_head_info();
 
 	/* EOF */
-- 
1.4.2.1.gf80a

Re: git and time

[Shawn Pearce]

Junio C Hamano <junkio@cox.net> wrote:
> One thing that makes "the common library code" less useful is
> that lock_ref_sha1() and its cousin lock_any_ref_for_update() do
> not let the caller to tell why a ref could not be locked ("did
> it not exist?  did the old_sha1 not match?"  and in
> lock_ref_sha1()'s case "did the ref have funny characters?").

Yes.  But I thought that in all such cases we use error or die so
the message is sent to STDERR before the process either terminates
or the function returns NULL.  So although the caller doesn't know
why the lock failed the end user does.

Git hasn't exactly structured its error messages in the past.
If we are talking about going down the path of having functions
return why they failed to the caller so the caller can react to
the failure as they see fit then we've got some work to do.  :)

> diff --git a/receive-pack.c b/receive-pack.c
> @@ -318,9 +258,11 @@ int main(int argc, char **argv)
>  	if (!dir)
>  		usage(receive_pack_usage);
>  
> -	if(!enter_repo(dir, 0))
> +	if (!enter_repo(dir, 0))
>  		die("'%s': unable to chdir or not a git archive", dir);
>  

You are missing:
+	setup_ident();

Without that reflog can't get the proper committer data from the
host's gecos information.  This is probably what is desired for
most pushes over SSH.

> +	git_config(git_default_config);
> +
>  	write_head_info();

-- 
Shawn.

Re: git and time

[Junio C Hamano]

Shawn Pearce <spearce@spearce.org> writes:

> You are missing:
> +	setup_ident();
>
> Without that reflog can't get the proper committer data from the
> host's gecos information.  This is probably what is desired for
> most pushes over SSH.

What's even more interesting is when there is .git/config file
and you do not override it with environment variables; the log
entry will be made under the name of the repository user.name in
such a case.

>> +	git_config(git_default_config);
>> +
>>  	write_head_info();

Often setup_ident() needs to go together with git_config(), and
you need to remember that setup must come before config.  These
rules are a bit cumbersome to follow and I often forget.

I wonder if we can have a simpler start-up sequence perhaps to
avoid future mistakes like this?

Re: git and time

[Shawn Pearce]

Junio C Hamano <junkio@cox.net> wrote:
> Shawn Pearce <spearce@spearce.org> writes:
> 
> > You are missing:
> > +	setup_ident();
> >
> > Without that reflog can't get the proper committer data from the
> > host's gecos information.  This is probably what is desired for
> > most pushes over SSH.
> 
> What's even more interesting is when there is .git/config file
> and you do not override it with environment variables; the log
> entry will be made under the name of the repository user.name in
> such a case.

Yes.  :-)

Shared repository owners might want to not set user.name in their
config files.

Another way around this would be to create a variation of
git_default_config for use in receive-pack.c that doesn't recognize
user.name/user.email.
 
> >> +	git_config(git_default_config);
> >> +
> >>  	write_head_info();
> 
> Often setup_ident() needs to go together with git_config(), and
> you need to remember that setup must come before config.  These
> rules are a bit cumbersome to follow and I often forget.
> 
> I wonder if we can have a simpler start-up sequence perhaps to
> avoid future mistakes like this?

I think most places we are "starting up" we invoke both, in that
order.  Which sort of implies maybe we could just fold setup_ident
into git_config.

-- 
Shawn.

Re: git and time

[Sean]

On Tue, 26 Sep 2006 17:27:45 -0700 (PDT)
Matthew L Foster <mfoster167@yahoo.com> wrote:


> It's true I don't know much about git, what is the difference between a changeset and a snapshot?
> Are you saying timestamps should be tracked separately or tracked by an scm system built on top of
> git? Does/should git care about the when of a snapshot?
> 
> Perhaps my question is directed more toward gitweb.cgi, it seems to me the timestamp of when a
> snapshot was merged into this repository should somehow be tracked and that is what gitweb.cgi
> should default to display. For example, if someone wants to know if security bugfix X was merged
> into linus' kernel tree they also want to know when that happened, don't they? 

You are right that a "Merged Date:" in gitweb would be useful information to
show for each commit, but it's not straightforward given the design of git.

Each commit contains the date and time it was first created.  Because this value
is used as part of each commits' unique hash value, it can not be changed without
breaking a very fundamental part of Git.  This means that Git can not easily
answer the question of which date any particular commit was merged with the
local repository.

To help address this, the "reflog" feature was added (i believe by Shawn Pearce)
which records a local time stamp when pulling in changes from other repositories.
It should be possible to query this log to get the information you desire, but I
don't think it would be efficient enough to do in gitweb unless the values were
cached instead of queried each time.

Sean

Re: git and time

[David Lang]

On Tue, 26 Sep 2006, Sean wrote:

>> It's true I don't know much about git, what is the difference between a changeset and a snapshot?
>> Are you saying timestamps should be tracked separately or tracked by an scm system built on top of
>> git? Does/should git care about the when of a snapshot?
>>
>> Perhaps my question is directed more toward gitweb.cgi, it seems to me the timestamp of when a
>> snapshot was merged into this repository should somehow be tracked and that is what gitweb.cgi
>> should default to display. For example, if someone wants to know if security bugfix X was merged
>> into linus' kernel tree they also want to know when that happened, don't they?
>
> You are right that a "Merged Date:" in gitweb would be useful information to
> show for each commit, but it's not straightforward given the design of git.

would it?

remember that a pach could be merged to many trees in any order. which merge 
date do you want to know about?

David Lang

Re: git and time

[Sean]

On Tue, 26 Sep 2006 17:47:27 -0700 (PDT)
David Lang <dlang@digitalinsight.com> wrote:

> would it?
> 
> remember that a pach could be merged to many trees in any order. which merge 
> date do you want to know about?

I *think* it would be useful information to know when it was merged with the
_local_ repository.   So in the case of kernel.org gitweb, you'd essentially
be able to find out when _Linus_ received the commit and published it to the
world.  Doesn't that sound worthwhile?

Sean

Re: git and time

[Junio C Hamano]

David Lang <dlang@digitalinsight.com> writes:

>> You are right that a "Merged Date:" in gitweb would be useful information to
>> show for each commit, but it's not straightforward given the design of git.
>
> would it?
>
> remember that a pach could be merged to many trees in any order. which
> merge date do you want to know about?

The date that repository I am looking at with gitweb first had
that commit, of course.  What other dates did you have in mind?

Re: git and time

[David Lang]

On Tue, 26 Sep 2006, Junio C Hamano wrote:

> David Lang <dlang@digitalinsight.com> writes:
>
>>> You are right that a "Merged Date:" in gitweb would be useful information to
>>> show for each commit, but it's not straightforward given the design of git.
>>
>> would it?
>>
>> remember that a pach could be merged to many trees in any order. which
>> merge date do you want to know about?
>
> The date that repository I am looking at with gitweb first had
> that commit, of course.  What other dates did you have in mind?
>

if that repository is then merged into another one, what date would that second 
one record for that commit? the date it was pulled there?

in many cases this would seem to be useless or distracting information (you 
already display where in the history the merge took place, do you really need to 
attach that date to all changes that arrive from the branch?)

if you have something like

a-b-c-d-e-f-g
              \
h------i------j

or worse

     a
      \
b-----c-d
  \   /   \
   e-f-----g
            \
h-i---------j

do you really want the date of j attached to all the changes a-g?

and if this information really is important, wouldn't you want to export the 
info out (as j then gets merged into m somewhere else)

David Lang

Re: git and time

[Sean]

On Tue, 26 Sep 2006 18:31:59 -0700 (PDT)
David Lang <dlang@digitalinsight.com> wrote:

> if that repository is then merged into another one, what date would that second 
> one record for that commit? the date it was pulled there?

Yes of course.  The question being posed and answered is.. when did the local
repository first get any specific commit.  For the sake of such a question
its merge history and path into the local repository doesn't matter, only
when the commit arrived locallly.

Sean

Re: git and time

[Junio C Hamano]

David Lang <dlang@digitalinsight.com> writes:

>>> remember that a pach could be merged to many trees in any order. which
>>> merge date do you want to know about?
>>
>> The date that repository I am looking at with gitweb first had
>> that commit, of course.  What other dates did you have in mind?
>
> if that repository is then merged into another one, what date would
> that second one record for that commit? the date it was pulled there?
>
> in many cases this would seem to be useless or distracting information
> (you already display where in the history the merge took place, do you
> really need to attach that date to all changes that arrive from the
> branch?)

Usually, but not on fast forward.

Only when somebody is interested in the particular question
"when did this commit has become part of this branch" it becomes
relevant.  And do not get me wrong.  I am in principle agreeing
with you that this is an extra information for most of the time
-- I even doubt "when did this commit has become part of this
branch" is all that useful.

Re: git and time

[Sean]

On Tue, 26 Sep 2006 19:31:47 -0700
Junio C Hamano <junkio@cox.net> wrote:

> Only when somebody is interested in the particular question
> "when did this commit has become part of this branch" it becomes
> relevant.  And do not get me wrong.  I am in principle agreeing
> with you that this is an extra information for most of the time
> -- I even doubt "when did this commit has become part of this
> branch" is all that useful.

It is interesting information for some people though.  For instance
someone wondering how long ago Linus published a certain security fix.
To be able to say to easily query gitweb and be able to report,
"Linus published that security fix X day ago etc.."

Of course, I agree with you that also knowing the revision number
such a commit appeared in is at least if not more important.  But
still I don't dismiss this date/time based question as completely
as you do.  The very fact that a gitweb user has brought this issue
forward (and this isn't the first time) suggests the information
is at least of casual interest to some people.

Sean

Re: git and time

[Jan Harkes]

On Tue, Sep 26, 2006 at 10:41:33PM -0400, Sean wrote:
> It is interesting information for some people though.  For instance
> someone wondering how long ago Linus published a certain security fix.
> To be able to say to easily query gitweb and be able to report,
> "Linus published that security fix X day ago etc.."

I don't see the point in knowing how many days ago the security fix was
published, since I'd really care if my machine is running a kernel that
contains the fix.

So I can see how I might want to know which branches (and/or tags) in my
repository contain the security fix. And this is pretty easy,

#!/bin/sh
fix="$(git-rev-parse --verify $1)"
git ls-remote . | while read sha ref ; do
    [ "$fix" == "$(git-merge-base "$fix" "$sha")" ] && echo $ref
done

Of course this could be cleaned up and extended quite a bit, possibly
allowing a user to specify if he cares about only branches, or tags or
some specific branch.

Jan

Re: git and time

[Sean]

On Fri, 29 Sep 2006 13:37:36 -0400
Jan Harkes <jaharkes@cs.cmu.edu> wrote:

> I don't see the point in knowing how many days ago the security fix was
> published, since I'd really care if my machine is running a kernel that
> contains the fix.

It was just a single example, one maybe not important to you.  In fact
it might be interesting to someone who isn't even running _any_ Linux
kernel themselves; a security researcher or journalist for instance.
There are other examples as well.  Maybe none of them would apply to
your needs, but there are people who would find it interesting and
convenient.

Sean