Re: [WISH] Store also tag dereferences in packed-refs

[Junio C Hamano <junkio@cox.net>]

Linus Torvalds <torvalds@osdl.org> writes:

> So the optimizations that _can_ be done are:
>
>  - add dereference info to .git/packed-refs
>
>    This would allow us to simply not do the expensive object lookup for 
>    every single tag. We'd still have to do it for non-packed objects, of 
>    course, but the cost here tends to be that over time you might have 
>    hundreds of tags, and even if each tag only takes 0.02s to look up, 
>    you're going to be slow.
>
>  - avoid the references for "heads/" (which we know are supposed to be 
>    commits, and cannot be tags) and when not specifying "-d". This won't 
>    help your case very much, though. If you want "-d", you want it, and 
>    the _big_ number of refs tends to be in tags, not branches, anyway.
>
>  - using a filesystem wih nicer locality behaviour for directory entries 
>    and inodes. This can cut down costs of cold-cache case by a factor of 
>    two, but right now there are no good filesystems that do this (but see 
>    for example "spadfs" that Mikulas Patocka announced a few weeks ago on 
>    linux-kernel - it would seem to have the possibility of being better in 
>    this area. I looked at the code and it looked like it could become 
>    very reasonable, but I've not actually _tested_ it, soo...)
>
> Anyway, I think that if we really want to make "git show-refs" go fast 
> when things are cold in the cache, and with lots ot tags and "-d" (which 
> is a reasonable case to optimize for: it's probably exactly what we end up 
> doing both for gitweb _and_ for "git-send-pack"), we'd need to expand the 
> packed-refs file with the deref cache.
>
> Junio?

Yes; the "for discussion only" WIP patch from yesterday was
about the first point and half of the second point.

One downside about storing the peeled refs in .git/packed-refs
is that the code in the wild does not have safety guard against
them, so the version of show-ref from v1.4.4 recently released
will list tags/v2.6.19^{} as if it is a true tag and worse yet
it might even show tags/v2.6.19^{}^{} under -d option after the
repository's refs are packed in the new format.

But I think we can avoid that problem by using a format slightly
different from the WIP patch.  We can for example use two spaces
between SHA-1 and the name for them.  refs.c::parse_ref_line()
in existing code will say "oh, this line is nonsense" and skip
them, while the updated code can say "ok, this is a peeled ref",
since it has a seemingly incomplete "check for valid refname"
there that only checks "isspace(*line)".

Did you do this "if (isspace(*line)) return NULL" to have
extensibility later, I wonder...