How to fetch missing pack

How to fetch missing pack

[Jörg Sommer]

Hi,

I'm missing a pack. My history of the git repository is broken:
% git show --pretty=raw e0fda6ab|head -7
commit e0fda6abd11c567b72f29ec0ee06c541404a9cb7
tree 77a7e4a849bbec646d88ae863f80ea3f519e26bd
parent 1ab58e8d6f728cdde0057f7ee88daab3a1c2d06f
author Wincent Colaiuta <win@wincent.com> 1196066088 +0100
committer Junio C Hamano <gitster@pobox.com> 1196109124 -0800

    Fix typo in draft 1.5.4 release notes
% git log --pretty=oneline e0fda6ab | wc -l
1
% git show e0fda6ab~1|cat
fatal: ambiguous argument 'e0fda6ab~1': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions

Can I somehow tell git fetch to check the whole history for holes and
fetch missing packs?

Bye, Jörg.
-- 
Der kommt den Göttern am nächsten, der auch dann schweigen kann,
wenn er im Recht ist.                         (Cato; 234–149 v. Chr.)

Re: How to fetch missing pack

[Peter Baumann]

On Sat, Apr 12, 2008 at 12:07:12PM +0000, Jörg Sommer wrote:
> Hi,
> 
> I'm missing a pack. My history of the git repository is broken:
> % git show --pretty=raw e0fda6ab|head -7
> commit e0fda6abd11c567b72f29ec0ee06c541404a9cb7
> tree 77a7e4a849bbec646d88ae863f80ea3f519e26bd
> parent 1ab58e8d6f728cdde0057f7ee88daab3a1c2d06f
> author Wincent Colaiuta <win@wincent.com> 1196066088 +0100
> committer Junio C Hamano <gitster@pobox.com> 1196109124 -0800
> 
>     Fix typo in draft 1.5.4 release notes
> % git log --pretty=oneline e0fda6ab | wc -l
> 1
> % git show e0fda6ab~1|cat
> fatal: ambiguous argument 'e0fda6ab~1': unknown revision or path not in the working tree.
> Use '--' to separate paths from revisions
> 
> Can I somehow tell git fetch to check the whole history for holes and
> fetch missing packs?
> 
> Bye, Jörg.

Try using git fsck --full to check you repo. If you are missing some
objects, you could just grab them from another valid repo which still has
them. (e.g. copy all packs/objects from the other repo into yours and then
do a git-repack -a -d -f)

-Peter

Re: How to fetch missing pack

[Jörg Sommer]

Hi Peter,

Peter Baumann <waste.manager@gmx.de> wrote:
> On Sat, Apr 12, 2008 at 12:07:12PM +0000, Jörg Sommer wrote:
>> I'm missing a pack. My history of the git repository is broken:
>> % git show --pretty=raw e0fda6ab|head -7
>> commit e0fda6abd11c567b72f29ec0ee06c541404a9cb7
>> tree 77a7e4a849bbec646d88ae863f80ea3f519e26bd
>> parent 1ab58e8d6f728cdde0057f7ee88daab3a1c2d06f
>> author Wincent Colaiuta <win@wincent.com> 1196066088 +0100
>> committer Junio C Hamano <gitster@pobox.com> 1196109124 -0800
>> 
>>     Fix typo in draft 1.5.4 release notes
>> % git log --pretty=oneline e0fda6ab | wc -l
>> 1
>> % git show e0fda6ab~1|cat
>> fatal: ambiguous argument 'e0fda6ab~1': unknown revision or path not in the working tree.
>> Use '--' to separate paths from revisions
>> 
>> Can I somehow tell git fetch to check the whole history for holes and
>> fetch missing packs?
>> 
>> Bye, Jörg.
>
> Try using git fsck --full to check you repo. If you are missing some
> objects, you could just grab them from another valid repo which still has
> them.

How can I exctract single objects from a pack?

> (e.g. copy all packs/objects from the other repo into yours and then do
> a git-repack -a -d -f)

This didn't work. I've cloned my git repo and copied the pack into
packs/objects without the .keep file and removed objects/info/packs.
After that I ran repack -a -d -f, but nothing changed. I still can't
access the parent of e0fda6ab. What else can I do?

Bye, Jörg.
-- 
The social dynamics of the net are a direct consequence of the fact
that nobody has yet developed a Remote Strangulation Protocol.
            (Larry Wall)

Re: How to fetch missing pack

[Christian Couder]

Le mercredi 23 avril 2008, Jörg Sommer a écrit :
> Hi Peter,
>
> Peter Baumann <waste.manager@gmx.de> wrote:
> >
> > Try using git fsck --full to check you repo. If you are missing some
> > objects, you could just grab them from another valid repo which still
> > has them.
>
> How can I exctract single objects from a pack?

Please have a look at this:

http://git.or.cz/gitwiki/GitFaq#head-ac11406480d09e2df98588e800e41b7256602074

Regards,
Christian.

Re: How to fetch missing pack

[Shawn O. Pearce]

Christian Couder <chriscool@tuxfamily.org> wrote:
> Le mercredi 23 avril 2008, Jörg Sommer a écrit :
> > Peter Baumann <waste.manager@gmx.de> wrote:
> > >
> > > Try using git fsck --full to check you repo. If you are missing some
> > > objects, you could just grab them from another valid repo which still
> > > has them.
> >
> > How can I exctract single objects from a pack?
> 
> Please have a look at this:
> 
> http://git.or.cz/gitwiki/GitFaq#head-ac11406480d09e2df98588e800e41b7256602074

If you know the name of the objects you are missing, you can
make a temporary pack to transfer the list of objects:

	# at good repo
	#
	git pack-objects --stdout <object.list >save.pack

	# at bad repo
	#
	git index-pack --stdin <save.pack

Make object.list just one SHA-1 per line.  The resulting pack is
not likely to be highly compressed, as it will probably contain
a non-delta version of each object, but its faster to create than
calling git-cat-file for each object, and may give you a smaller
pack than just copying everything as the FAQ entry recommends.

Note that you may need to do this several times; e.g. if you are
missing a tree you won't know what blobs you are missing until
you load the tree and run fsck --full again to parse the tree and
discover the missing blobs.  This iterative copying approach is
what the FAQ avoids when it suggests you copy everything.

-- 
Shawn.

git doesn't finds the parent of a commit (was: How to fetch missing pack)

[Jörg Sommer]

[-- Attachment #1: Type: text/plain, Size: 2595 bytes --]

Hi,

Shawn O. Pearce schrieb am Thu 24. Apr, 02:08 (-0400):
> If you know the name of the objects you are missing, you can
> make a temporary pack to transfer the list of objects:
> 
> 	# at good repo
> 	#
> 	git pack-objects --stdout <object.list >save.pack
> 
> 	# at bad repo
> 	#
> 	git index-pack --stdin <save.pack

I didn't know the list of objects I'm missing, but I used this little
script do find them:

anchor=7cb192eab0251911e2ca77d4ecceb621dd2d34f5
while true
do
    outp=$(git log --pretty=raw --raw $anchor 2>&1 >/dev/null)
    echo $outp
    [ $#outp -eq 0 ] && break
    cmt=${${outp##* \(}%\)}
    git cat-file -t $cmt >/dev/null 2>&1 && \
      cmt=$(git cat-file commit $cmt |grep '^tree ')
    echo ${cmt##* } | ssh server cd git\; git pack-objects --stdout | \
      git index-pack --stdin >/dev/null
done

Maybe it helps someone else. :)

But now, there's something else broken.

% git cat-file commit b63e99500137c913bd801a2f22b6cf88c63b95c5
tree 68a58fb97935f35c6fb7bcbcfed73b1697db000a
parent 3f061887c562b20d3ed3d1f764462cf986a1ad12
author Wincent Colaiuta <win@wincent.com> 1195996542 +0100
committer Junio C Hamano <gitster@pobox.com> 1196019475 -0800

Add "--patch" option to git-add--interactive

When the "--patch" option is supplied, the patch_update_cmd() function is
…

% git cat-file commit 3f061887c562b20d3ed3d1f764462cf986a1ad12
tree 636a0f9c085aac7ba539ef2ace03b3081481891a
parent 324ccbd6a09816af830b22b02bbeb06349141849
author Junio C Hamano <gitster@pobox.com> 1196014210 -0800
committer Junio C Hamano <gitster@pobox.com> 1196014993 -0800

add -i: Fix running from a subdirectory

This fixes the pathspec interactive_add() passes to the underlying
…

% git rev-parse b63e99500137c913bd801a2f22b6cf88c63b95c5~1
b63e99500137c913bd801a2f22b6cf88c63b95c5~1
fatal: ambiguous argument 'b63e99500137c913bd801a2f22b6cf88c63b95c5~1': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions

Can someone tell me what I'm doing wrong?

> Note that you may need to do this several times; e.g. if you are
> missing a tree you won't know what blobs you are missing until
> you load the tree and run fsck --full again to parse the tree and
> discover the missing blobs.

fsck --full reports only dangling objects, no breakage.

Bye, Jörg.
-- 
Die zehn Gebote Gottes enthalten 172 Wörter, die amerikanische
Unabhängigkeitserklärung 300 Wörter, die Verordnung der europäischen
Gemeinschaft über den Import von Karamelbonbons exakt 25911 Wörter.

[-- Attachment #2: Digital signature http://en.wikipedia.org/wiki/OpenPGP --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: git doesn't finds the parent of a commit (was: How to fetch missing pack)

[Stephen R. van den Berg]

J?rg Sommer wrote:
>% git rev-parse b63e99500137c913bd801a2f22b6cf88c63b95c5~1
>b63e99500137c913bd801a2f22b6cf88c63b95c5~1
>fatal: ambiguous argument 'b63e99500137c913bd801a2f22b6cf88c63b95c5~1': unknown revision or path not in the working tree.
>Use '--' to separate paths from revisions

>Can someone tell me what I'm doing wrong?

I've had similar symptoms when I had circular references in the
repository.  They're not reported by any of the existing checks, I've
submitted a patch (resent it just now) which causes git to check for
(and report) circular references when using --topo-order on e.g.
git-rev-list.
-- 
Sincerely,                                                          srb@cuci.nl
           Stephen R. van den Berg.

Re: git doesn't finds the parent of a commit

[Junio C Hamano]

"Stephen R. van den Berg" <srb@cuci.nl> writes:

> J?rg Sommer wrote:
>>% git rev-parse b63e99500137c913bd801a2f22b6cf88c63b95c5~1
>>b63e99500137c913bd801a2f22b6cf88c63b95c5~1
>>fatal: ambiguous argument 'b63e99500137c913bd801a2f22b6cf88c63b95c5~1': unknown revision or path not in the working tree.
>>Use '--' to separate paths from revisions
>
>>Can someone tell me what I'm doing wrong?
>
> I've had similar symptoms when I had circular references in the
> repository.  They're not reported by any of the existing checks, I've
> submitted a patch (resent it just now) which causes git to check for
> (and report) circular references when using --topo-order on e.g.
> git-rev-list.

Assuming that we never have SHA-1 hash collisions, the graft mechansim is
practically the only way to get yourself into the circular reference
situation.

Perhaps we should check this circularity when we install grafts instead of
special casing the topo-order codepath?  How expensive would that
alternative approach be?

Re: git doesn't finds the parent of a commit

[Stephen R. van den Berg]

Junio C Hamano wrote:
>"Stephen R. van den Berg" <srb@cuci.nl> writes:
>> I've had similar symptoms when I had circular references in the
>> repository.  They're not reported by any of the existing checks, I've
>> submitted a patch (resent it just now) which causes git to check for
>> (and report) circular references when using --topo-order on e.g.
>> git-rev-list.

>Assuming that we never have SHA-1 hash collisions, the graft mechansim is
>practically the only way to get yourself into the circular reference
>situation.

True, I think.

>Perhaps we should check this circularity when we install grafts instead of
>special casing the topo-order codepath?  How expensive would that
>alternative approach be?

Not practical in its current form.
Checking for circular references is O(n) in CPU and memory use relative
to the number of commits in the entire repository.

Consider:
- The proposed check in the topo-order path is very low-cost, it costs a
  single decrement/increment per commit (and will detect other circular
  references not caused by the grafts mechanism, if they ever should occur).
- If it is being done during the grafts install, then there should be a
  flag-file (at least), which indicates when the grafts file has changed
  since the last check.
- It could/should be added to git-fsck.
-- 
Sincerely,                                                          srb@cuci.nl
           Stephen R. van den Berg.

Re: git doesn't finds the parent of a commit

[Jörg Sommer]

Hallo,

"Stephen R. van den Berg" <srb@cuci.nl> wrote:
> J?rg Sommer wrote:
>>% git rev-parse b63e99500137c913bd801a2f22b6cf88c63b95c5~1
>>b63e99500137c913bd801a2f22b6cf88c63b95c5~1
>>fatal: ambiguous argument 'b63e99500137c913bd801a2f22b6cf88c63b95c5~1': unknown revision or path not in the working tree.
>>Use '--' to separate paths from revisions
>
>>Can someone tell me what I'm doing wrong?
>
> I've had similar symptoms when I had circular references in the
> repository.

No, this was not my problem. But I found the reason with bisect:
.git/shallow. I've started with a shallow clone, but forgot it. Is it
possible that git warns about this or ignores the shallow entry if the
parent of the commit is available?

Bye, Jörg.
-- 
Dein Gesicht wird dir geschenkt. Lächeln musst du selber! (Inga Hermann)