How to (re-)create .git/logs/refs

How to (re-)create .git/logs/refs

[Holger Schurig]

Hi !

I have an old SVN project with lots of commits from 2005 to 2008
converted. This was a month ago. In the mean-time, I've made
some commits from git as well.

"git log" shows both types of commit quite nicely.

"git checkout <sha1>" works also correctly, for arbitrary ancient
SHA1s.

"git checkout @{2007-04-01}" doesn't work. I get an error like
this:

  warning: Log for '' only goes back to Thu, 8 May 2008 09:35:38 +0000.
  fatal: bad object @{2007-04-01}

However, "git checkout @{2008-05-01}" works, because this commit
come from git direct, not implicitly from SVN. It seems that
neiver "git svn clone" nor "examples/git-svnimport.perl"
creates/updates .git/logs/refs when it creates commits from an
old SVN repository.


So my question:

a) is there a way to re-create .git/logs/refs ?

b) is there a simple way to get a list of ISO-dates and
   associated SHA1?  Then I could use a little python code to
   determine the SHA1 id that corresponds to some date (I need
   to access old versions of the source via bitbake and/or
   www.openembedded.org, so adding a custom fetch method with
   some python magic is a non-brainer).

How to (re-)create .git/logs/refs

[Holger Schurig]

Hi !

I have an old SVN project with lots of commits from 2005 to 2008
converted. This was a month ago. In the mean-time, I've made
some commits from git as well.

"git log" shows both types of commit quite nicely.

"git checkout <sha1>" works also correctly, for arbitrary ancient
SHA1s.

"git checkout @{2007-04-01}" doesn't work. I get an error like
this:

  warning: Log for '' only goes back to Thu, 8 May 2008 09:35:38 +0000.
  fatal: bad object @{2007-04-01}

However, "git checkout @{2008-05-01}" works, because this commit
was a git commit, not an SVN-converted one. It seems that
neiver "git svn clone" nor "examples/git-svnimport.perl"
creates/updates .git/logs/refs and that therefore @{date} is
useless here.


So my question:

a) is there a way to re-create .git/logs/refs ?

b) is there a simple way to get a list of ISO-dates and
   associated SHA1?  Then I could use a little python code to
   determine the SHA1 id that corresponds to some date (I need
   to access old versions of the source via bitbake and/or
   www.openembedded.org, so adding a custom fetch method with
   some python magic is a non-brainer).

c) any plan to fix git-svn?

Re: How to (re-)create .git/logs/refs

[Holger Schurig]

>   warning: Log for '' only goes back to Thu, 8 May 2008
> 09:35:38 +0000. fatal: bad object @{2007-04-01}

Oops, this was from another test that I made today. My original
git repository shows of course a different date:

  warning: Log for '' only goes back to Wed, 23 Apr 2008 08:36:26 +0200.
  fatal: bad object @{2007-05-01}

Also, "git log @{2007-05-01}" | cat  gives me this error
message, "git checkout @{2007-05-01}" gives

  error: pathspec '@{2007-05-01}' did not match any file(s) known to git.
  Did you forget to 'git add'?

which is completely misleading.



Sorry for being unclear in my first mail.

Re: How to (re-)create .git/logs/refs

[Nicolas Pitre]

On Thu, 8 May 2008, Holger Schurig wrote:

> So my question:
> 
> a) is there a way to re-create .git/logs/refs ?

No.

The reflog records how _you_ modified your various branches locally.  
This has no direct relationship with the dates the commits in those 
branches were created.  It records what was your own view of the 
repository in time.  So if you performed an import of a SVN repository 
last week, then it is normal that you don't have any view into that 
particular repository's state before last week.

> b) is there a simple way to get a list of ISO-dates and
>    associated SHA1?  Then I could use a little python code to
>    determine the SHA1 id that corresponds to some date (I need
>    to access old versions of the source via bitbake and/or
>    www.openembedded.org, so adding a custom fetch method with
>    some python magic is a non-brainer).

Have a look at the documentation for the --pretty=format option in the 
git-log man page.  For example:

	git log --pretty=format:"%H %ai"

might be just what you're looking for.


Nicolas

Re: How to (re-)create .git/logs/refs

[Holger Schurig]

> > a) is there a way to re-create .git/logs/refs ?
>
> No.
>
> The reflog records how _you_ modified your various branches
> locally.

Okay, then I presume it's a shortcoming that the @{date} syntax 
uses .git/logs/ref. It could do otherwise, e.g. git bisect seems 
to able to go via bisection arbitrarily back in time without 
using log-refs.

> 	git log --pretty=format:"%H %ai"

Thanks.

Re: How to (re-)create .git/logs/refs

[Holger Schurig]

> Okay, then I presume it's a shortcoming that the @{date}
> syntax uses .git/logs/ref.

Is there any reason sha1_name.c (which is responsible for the @{} 
syntax) doesn't include list-objects.h and uses 
traverse_commit_list() for it's purposes?  It wouldn't then need 
to use .git/logs/ref anymore.

If that would be ok, then maybe I find time (in my spare time) 
and prepare a patch in this direction.

Re: How to (re-)create .git/logs/refs

[Andreas Ericsson]

Holger Schurig wrote:
>> Okay, then I presume it's a shortcoming that the @{date}
>> syntax uses .git/logs/ref.
> 
> Is there any reason sha1_name.c (which is responsible for the @{} 
> syntax) doesn't include list-objects.h and uses 
> traverse_commit_list() for it's purposes?


It would be alot slower for large projects. Otoh it would also mean
it'd work on a freshly cloned repository, so perhaps it's worth
the trade-off.


>  It wouldn't then need 
> to use .git/logs/ref anymore.
> 
> If that would be ok, then maybe I find time (in my spare time) 
> and prepare a patch in this direction.
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: How to (re-)create .git/logs/refs

[Johannes Schindelin]

Hi,

On Tue, 13 May 2008, Holger Schurig wrote:

> > > a) is there a way to re-create .git/logs/refs ?
> >
> > No.
> >
> > The reflog records how _you_ modified your various branches
> > locally.
> 
> Okay, then I presume it's a shortcoming that the @{date} syntax 
> uses .git/logs/ref.

Are you sure you understand what @{date} does?  It shows you the state 
_this_ repository was in at that date.

There is _no way_ to get this information from somewhere else than the 
reflogs.

It is a common misconception that you can reference anything by date in a 
distributed setup.  (Before you ask, I will just mention "clock skew" and 
"parallel branches" to give you an idea why this is a misconception.)

Ciao,
Dscho

Re: How to (re-)create .git/logs/refs

[Holger Schurig]

> Are you sure you understand what @{date} does?  It shows you
> the state _this_ repository was in at that date.

Who says so?  "man git-rev-parse" just says:

o  A ref followed by the suffix @ with a date specification
   enclosed in a brace pair (e.g. {yesterday}, {1 month 2 weeks 3
   days 1 hour 1 second ago} or {1979-02-26 18:30:00}) to specify
   the value of the ref at a prior point in time. This suffix may
   only be used immediately following a ref name and the ref
   must have an existing log ($GIT_DIR/logs/<ref>).


It just states "at a prior point in time", not "at a prior point 
in time in your git repository/checkout".


> It is a common misconception that you can reference anything
> by date in a distributed setup.  (Before you ask, I will just
> mention "clock skew" and "parallel branches" to give you an
> idea why this is a misconception.)

My idea was to use the first commit (from git-rev-log) where the 
date is below the specified date.

To give you some context: Bitbake (from www.openembedded.org) is 
a tool that can download software, patch it, configure it, 
compile it and create packages out of it. Nothing fancy, except 
that it can do this for a huge amount of embedded devices, 
usually using a cross-compiler for ARM, MIPS, whatever.

And it can download not just tar files, it can also use CVS, SVN, 
hg, git and so on.

In Openembedded, many patch recipes specify the version to use. 
If you deal with versioned stuff, e.g. 
filename-frob-3.14.tar.bz, this is fine. Sometimes the version 
is a SVN revision number. And sometimes it's a git, mercury or 
monotone hash. However, the latter beast are almost 
non-describing. So many bitbake recipes specify a SRCDATE, and 
you can immediately see if SRCDATE=20070501 that it uses an 
ancient version of the software, from 1st May 2007. You won't 
see that if it would specify 
GIT_REV=6e2df4fd066c450b0b3c8e0f1769d4163e2b52c4. Of course you 
can do

  GIT_REV=6e2df4fd066c450b0b3c8e0f1769d4163e2b52c4
  # This is from 2007 May 1st

but then you're redundant and chances are high that those two 
lines get out-of-sync.


So, when I have SRCDATE=20070501, I'd be happy if git would, for 
me, find out that this is 
6e2df4fd066c450b0b3c8e0f1769d4163e2b52c4, even when the commit 
6e2df4fd066c450b0b3c8e0f1769d4163e2b52c4 was not done on my 
local machine, but is a commit that I pulled from the outside.

Also, when I do "git checkout @{20070501}", I don't usually mind 
if, e.g. because of time-skew or multi-heads, I'm not at the 
17th commit in this area, but on the top-most for which this 
condition is true. I can use "git log" and "git reset" or 
whatever to dig my way to the state I want, but git would do the 
dull work for me.

Re: How to (re-)create .git/logs/refs

[Jakub Narebski]

Holger Schurig <hs4233@mail.mn-solutions.de> writes:

> > Are you sure you understand what @{date} does?  It shows you
> > the state _this_ repository was in at that date.
> 
> Who says so? "man git-rev-parse" just says:
> 
> o  A ref followed by the suffix @ with a date specification
>    enclosed in a brace pair (e.g. {yesterday}, {1 month 2 weeks 3
>    days 1 hour 1 second ago} or {1979-02-26 18:30:00}) to specify
>    the value of the ref at a prior point in time. This suffix may
>    only be used immediately following a ref name and the ref
>    must have an existing log ($GIT_DIR/logs/<ref>).
> 
> It just states "at a prior point in time", not "at a prior point 
> in time in your git repository/checkout".

Please read carefully: "value of the _ref_" at a given time.
It means value where ref pointed to at given time.  That of
course depends on what you did with repository: did you just fetch,
or perhaps you made those commits, or perhaps you rebased branch,
or decided that the work you did is wrong and rewound to previous
version using reset.

As this is purely local matter, it matter only for _your_
repository.

> > It is a common misconception that you can reference anything
> > by date in a distributed setup.  (Before you ask, I will just
> > mention "clock skew" and "parallel branches" to give you an
> > idea why this is a misconception.)
> 
> My idea was to use the first commit (from git-rev-log) where the 
> date is below the specified date.

Which might be not what you want, as you can get revision which
was on some side branch.
 
> To give you some context: Bitbake (from www.openembedded.org) is 
> a tool that can download software, patch it, configure it, 
> compile it and create packages out of it. Nothing fancy, except 
> that it can do this for a huge amount of embedded devices, 
> usually using a cross-compiler for ARM, MIPS, whatever.
>
> And it can download not just tar files, it can also use CVS, SVN, 
> hg, git and so on.
> 
> In OpenEmbedded, many patch recipes specify the version to use. 
> If you deal with versioned stuff, e.g. 
> filename-frob-3.14.tar.bz, this is fine. Sometimes the version 
> is a SVN revision number. And sometimes it's a git, mercury or 
> monotone hash. However, the latter beast are almost 
> non-describing. So many bitbake recipes specify a SRCDATE, and 
> you can immediately see if SRCDATE=20070501 that it uses an 
> ancient version of the software, from 1st May 2007. You won't 
> see that if it would specify 
> GIT_REV=6e2df4fd066c450b0b3c8e0f1769d4163e2b52c4. Of course you 
> can do
> 
>   GIT_REV=6e2df4fd066c450b0b3c8e0f1769d4163e2b52c4
>   # This is from 2007 May 1st
> 
> but then you're redundant and chances are high that those two 
> lines get out-of-sync.
 
Thanks for explanation.  This is most useful when proposing solutions
to real problem...

The solution, of course, is to use git-describe output instead of
full SHA-1 (this of course assumes that you tag your releases).
Something like "v1.5.5.1-215-gc853afb" is, I think, quite well
self-described.  There is still place for ambiguity due without
the shortened sha-1 part due to parallel development, but I think
no less than using date.
 
> So, when I have SRCDATE=20070501, I'd be happy if git would, for 
> me, find out that this is 
> 6e2df4fd066c450b0b3c8e0f1769d4163e2b52c4, even when the commit 
> 6e2df4fd066c450b0b3c8e0f1769d4163e2b52c4 was not done on my 
> local machine, but is a commit that I pulled from the outside.
> 
> Also, when I do "git checkout @{20070501}", I don't usually mind 
> if, e.g. because of time-skew or multi-heads, I'm not at the 
> 17th commit in this area, but on the top-most for which this 
> condition is true. I can use "git log" and "git reset" or 
> whatever to dig my way to the state I want, but git would do the 
> dull work for me.

You can try 

 $ git checkout "$(git log -1 --pretty=format:%H --before=2007-05-01)"

but YMMV, i.e. if you hit upon correct revision or not would depend
on luck.

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: How to (re-)create .git/logs/refs

[Holger Schurig]

Hi Jakub !

Thanks for your trick. That completely solves my problem *)



*) I'm fully aware that that "git log" might show something like:

2 days ago: Blah
3 days ago: Muh
3 days ago: Blech
2 days ago: Huwa!

But this won't happen in my case --- at least not in repositories 
freshly converted from SVN.