git 1.5.4.3 push incorrectly honors grafts file

git 1.5.4.3 push incorrectly honors grafts file

[elupus]

Hi, 

I recently had a problem with git push honoring the grafts file. It caused
it not to push all data required for a branch to the remote repository,
rendering it impossible to clone the remote repository (missing blobs)

This was with an not so fresh git version of 1.5.4.3 (ubuntu hardy).

Has this issue been fixed in later git version? I saw a thread talking
about it a long time ago (long before my release in question) on this
mailing list, but nothing was mentioned about if it was actually solved.

Regards
Joakim Plate

Re: git 1.5.4.3 push incorrectly honors grafts file

[elupus]

On Mon, 27 Apr 2009 17:51:05 +0200, elupus wrote:

> Hi, 
> 
> I recently had a problem with git push honoring the grafts file. It caused
> it not to push all data required for a branch to the remote repository,
> rendering it impossible to clone the remote repository (missing blobs)
> 
> This was with an not so fresh git version of 1.5.4.3 (ubuntu hardy).
> 
> Has this issue been fixed in later git version? I saw a thread talking
> about it a long time ago (long before my release in question) on this
> mailing list, but nothing was mentioned about if it was actually solved.
> 
> Regards
> Joakim Plate

Bump, anybody know of a way to avoid this? The problem even occurs on the
local machine in that git gc will cleanup stuff that isn't required due to
the grafts file, rendering the repo invalid if the graft file is removed.

I don't think running filter-branch on the git svn imported branches seems
like a good idea. since that would also wreak havoc on any repo that pulls
from mine (ie still private repo like usb stick or other dev machine). 

Imho, grafts shouldn't be honored on either push/pull/gc operations. 

Joakim

Re: git 1.5.4.3 push incorrectly honors grafts file

[Johannes Sixt]

elupus schrieb:
> On Mon, 27 Apr 2009 17:51:05 +0200, elupus wrote:
> 
>> Hi, 
>>
>> I recently had a problem with git push honoring the grafts file. It caused
>> it not to push all data required for a branch to the remote repository,
>> rendering it impossible to clone the remote repository (missing blobs)
>>
>> This was with an not so fresh git version of 1.5.4.3 (ubuntu hardy).
>>
>> Has this issue been fixed in later git version? I saw a thread talking
>> about it a long time ago (long before my release in question) on this
>> mailing list, but nothing was mentioned about if it was actually solved.
>>
>> Regards
>> Joakim Plate
> 
> Bump, anybody know of a way to avoid this? The problem even occurs on the
> local machine in that git gc will cleanup stuff that isn't required due to
> the grafts file, rendering the repo invalid if the graft file is removed.
> 
> I don't think running filter-branch on the git svn imported branches seems
> like a good idea. since that would also wreak havoc on any repo that pulls
> from mine (ie still private repo like usb stick or other dev machine). 
> 
> Imho, grafts shouldn't be honored on either push/pull/gc operations. 

Right.

It's a know issue. But it was nobody's itch, yet, perhaps because too few 
people use grafts. There is a topic, cc/replace, in Junio's pu branch that 
introduces "replacement objects"; these are a generalization of grafts. If 
effort is invested, then it's best to nurse this topic.

-- Hannes

Re: git 1.5.4.3 push incorrectly honors grafts file

[elupus]

On Mon, 04 May 2009 19:25:27 +0200, Johannes Sixt wrote:

> elupus schrieb:
>> [26 quoted lines suppressed]
> 
> Right.
> 
> It's a know issue. But it was nobody's itch, yet, perhaps because too few 
> people use grafts. There is a topic, cc/replace, in Junio's pu branch that 
> introduces "replacement objects"; these are a generalization of grafts. If 
> effort is invested, then it's best to nurse this topic.
> 
> -- Hannes

Ah okey. Atleast then i know to thread carfully. 

The ugglies thing about this was that git gc had actually pruned a commit
from my repo due to an invalid graft's file. That push/pull didn't work
wasn't soo bad, but that my repo got corrupted due to an invalid graft file
is abit scary. 

Luckily in my case it was a git svn repo, and I even think i found a way to
just rebuild that branch where it occured.

It's quite probable that the problem's I had with the push/pull was due to
this missing commit. Since all my grafts (part from the wrong one) are only
additions of parents to commits. A cloned repo should still be okey even if
it does honor the grafts file (i think without knowning too much about the
inards of git).

Joakim

Re: git 1.5.4.3 push incorrectly honors grafts file

[Dmitry Potapov]

On Sun, May 03, 2009 at 07:14:22PM +0200, elupus wrote:
> On Mon, 27 Apr 2009 17:51:05 +0200, elupus wrote:
> 
> > Hi, 
> > 
> > I recently had a problem with git push honoring the grafts file. It caused
> > it not to push all data required for a branch to the remote repository,
> > rendering it impossible to clone the remote repository (missing blobs)
> > 
> > This was with an not so fresh git version of 1.5.4.3 (ubuntu hardy).
> > 
> > Has this issue been fixed in later git version? I saw a thread talking
> > about it a long time ago (long before my release in question) on this
> > mailing list, but nothing was mentioned about if it was actually solved.
> > 
> > Regards
> > Joakim Plate
> 
> Bump, anybody know of a way to avoid this?

Don't use the grafts file. It is primary useful immediately after importing
some repository to Git when you need to clean up it a bit before fixing the
result by running git-filter-branch. Another usage of it is to add old history,
which is not part of the official repository. In the last case, you only add
a new parent, so it should not be a problem, except this addition is purely
local and any cloned repo will not have it.

> The problem even occurs on the
> local machine in that git gc will cleanup stuff that isn't required due to
> the grafts file, rendering the repo invalid if the graft file is removed.

It may happen only if your graft makes some part of history unreachable. And
as I said above, using the grafts file should be avoided wherever possible,
and graft that replaces or removes some parents should only be used if you
are going to run git-filter-branch after that.

> 
> I don't think running filter-branch on the git svn imported branches seems
> like a good idea. since that would also wreak havoc on any repo that pulls
> from mine (ie still private repo like usb stick or other dev machine).

If you migrate from SVN to Git and want to edit history after importing, you
should run filter-branch before making this repository public.  Re-writing
public history is always a bad idea, regardless how it is done. If you use
git-svn for bidirectional synchronization then you most like use grafts in
the way it was not intended to use...

> Imho, grafts shouldn't be honored on either push/pull/gc operations.

If git gc will not honor grafts then it may delete those that are referred
by grafts, which is clearly wrong. So, perhaps, what you really want is that
git-gc should consider grafts as an additional source of information rather
than replacement. (Actually, git-gc is high level wrapper over git-repack,
git-prune, etc, which should be changed.) Also, git-fsck needs to be changed
to consider grafts as an additional source of information...

Whether grafts should be completely ignored by push/pull is not completely
clear. Though it may be the best course of action, it is also confusing,
because git log and other commands show you one history, but something
different gets pulled or pushed. (So, anyone who inherited such a repo from
another co-worker is bound to a big surprise as to why pull/push do not
work correctly). Also, we have a single logic that creates packages, whether
it is packages for network transfer or local storage, but it should be
changed to work different based on whether it is local or non-local package.
So, it appears to me a lot of changes, but the end result will be still not
something what you really want to use for reliable storage...

IMHO, grafts should not be used at all except very rare cases like editing
an imported history before making it public.


Dmitry

Re: git 1.5.4.3 push incorrectly honors grafts file

[elupus]

On Tue, 5 May 2009 00:40:22 +0400, Dmitry Potapov wrote:

> 
> It may happen only if your graft makes some part of history unreachable. And
> as I said above, using the grafts file should be avoided wherever possible,
> and graft that replaces or removes some parents should only be used if you
> are going to run git-filter-branch after that.

Understandable, but not really an option in my case. (see below)

> If you migrate from SVN to Git and want to edit history after importing, you
> should run filter-branch before making this repository public.  Re-writing
> public history is always a bad idea, regardless how it is done. If you use
> git-svn for bidirectional synchronization then you most like use grafts in
> the way it was not intended to use...

In my case i'm not migrating. I'm following a svn repo, but want to do my
own development in git.

> If git gc will not honor grafts then it may delete those that are referred
> by grafts, which is clearly wrong. So, perhaps, what you really want is that
> git-gc should consider grafts as an additional source of information rather
> than replacement. (Actually, git-gc is high level wrapper over git-repack,
> git-prune, etc, which should be changed.) Also, git-fsck needs to be changed
> to consider grafts as an additional source of information...

Yea, sounds like the correct approach.

> IMHO, grafts should not be used at all except very rare cases like editing
> an imported history before making it public.

Suppose so, but they are also very usefull for git svn repo's where they
can be used to correct for merge history. For example a feature branch
getting resynced to the trunk multiple times during it's lifetime. Adding
grafts for atleast the latest such occurance, makes it way easier to git
diff / git merge --squash and so on between the branches.

Also after the branch have been merged back into trunk, there could be a
point to add a graft as the previous branch will likely be deleted from the
svn repo, and thus would end up unreachable in the git repo should one
decide to remove the ref to the tip since there is not branch tracking in
the git svn client.

In a perfect world (not a likely one as i'll not spend time on implementing
it) I would like to be able to add new parents to commits and have them
become permanent through something like filter-branch, without this ruining
anything for anybody that pulls from this repo.

Joakim

Re: git 1.5.4.3 push incorrectly honors grafts file

[Michael J Gruber]

elupus venit, vidit, dixit 27.04.2009 17:51:
> Hi, 
> 
> I recently had a problem with git push honoring the grafts file. It caused
> it not to push all data required for a branch to the remote repository,
> rendering it impossible to clone the remote repository (missing blobs)
> 
> This was with an not so fresh git version of 1.5.4.3 (ubuntu hardy).
> 
> Has this issue been fixed in later git version? I saw a thread talking

How about testing this yourself, since you have a test case in your
hands and only need to build a more current git? Alternatively,
providing a simple test script would allow others to test this for you.

Cheers,
Michael

> about it a long time ago (long before my release in question) on this
> mailing list, but nothing was mentioned about if it was actually solved.
> 
> Regards
> Joakim Plate
>