git-push of new, tiny branch doing more work than necessary?

git-push of new, tiny branch doing more work than necessary?

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 1072 bytes --]

I've run through something quite like the following scenario a few
times lately:

	git pull origin
	git checkout -b experiment
	# hack a file or two
	git commit -a -m "new experiment"
	git push origin experiment

What I expect at this point is for git to push the few newly created
objects out to the repository. Instead it talks about generating,
deltifying, transferring, and unpacking thousands of objects (see
below).

Shouldn't the same WANT/HAVE hand shaking that makes git-fetch
efficient be doing something similar here? Or am I just doing
something wrong?

-Carl

$ git --version
git version 1.4.2.rc2.gef1d9
$ git push cworth 8379
updating 'refs/heads/8379'
  from 0000000000000000000000000000000000000000
  to   44379202da96faf632b73cdcdd6c0a74fb4b54f4
Generating pack...
Done counting 4900 objects.
Result has 4622 objects.
Deltifying 4622 objects.
 100% (4622/4622) done
Unpacking 4622 objects
Total 4622, written 4622 (delta 3411), reused 1259 (delta 946)
refs/heads/8379: 0000000000000000000000000000000000000000 ->
44379202da96faf632b73cdcdd6c0a74fb4b54f4

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: git-push of new, tiny branch doing more work than necessary?

[Johannes Schindelin]

Hi,

On Thu, 21 Sep 2006, Carl Worth wrote:

> I've run through something quite like the following scenario a few
> times lately:
> 
> 	git pull origin
> 	git checkout -b experiment
> 	# hack a file or two
> 	git commit -a -m "new experiment"
> 	git push origin experiment
> 
> What I expect at this point is for git to push the few newly created
> objects out to the repository. Instead it talks about generating,
> deltifying, transferring, and unpacking thousands of objects (see
> below).

I experienced something like this, too, but did not have the time to debug 
it. But alas, some time (I think last week) it started working as expected 
again.

> git version 1.4.2.rc2.gef1d9

Please try a more recent version (yours is from Jul 27).

Ciao,
Dscho

Re: git-push of new, tiny branch doing more work than necessary?

[Andy Whitcroft]

Johannes Schindelin wrote:
> Hi,
> 
> On Thu, 21 Sep 2006, Carl Worth wrote:
> 
>> I've run through something quite like the following scenario a few
>> times lately:
>>
>> 	git pull origin
>> 	git checkout -b experiment
>> 	# hack a file or two
>> 	git commit -a -m "new experiment"
>> 	git push origin experiment
>>
>> What I expect at this point is for git to push the few newly created
>> objects out to the repository. Instead it talks about generating,
>> deltifying, transferring, and unpacking thousands of objects (see
>> below).
> 
> I experienced something like this, too, but did not have the time to debug 
> it. But alas, some time (I think last week) it started working as expected 
> again.

I hope this started working again because we fixed it this time.  Two
bugs were fixed in this area.  First any branch names over about 40
characters in length were being ignored completely due to a validation
bug.  Second in an attempt to ensure we didn't bust the 900 entry
command line limit for rev-list we were restricting ourselves to 16
remote refs as possible bases.  master now has these changes in it and I
am no longer seeing this behavior.

> 
>> git version 1.4.2.rc2.gef1d9
> 
> Please try a more recent version (yours is from Jul 27).

-apw

Re: git-push of new, tiny branch doing more work than necessary?

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 1077 bytes --]

On Fri, 22 Sep 2006 08:40:41 +0100, Andy Whitcroft wrote:
> I hope this started working again because we fixed it this time.  Two
> bugs were fixed in this area.  First any branch names over about 40
> characters in length were being ignored completely due to a validation
> bug.  Second in an attempt to ensure we didn't bust the 900 entry
> command line limit for rev-list we were restricting ourselves to 16
> remote refs as possible bases.  master now has these changes in it and I
> am no longer seeing this behavior.

Aha! Thanks.

It doesn't appear I had the 40-character ref problem, nor was I
exceeding the 900 ref limit. But I was definitely well over the 16 ref
limit.

> >> git version 1.4.2.rc2.gef1d9
> >
> > Please try a more recent version (yours is from Jul 27).

I updated as soon as I got your reply, but just waited until I hit a
repeat of the use case today, (I was just too lazy to try to roll
things back and duplicate the exact same conditions).

Sure enough, things seem to be much better now (git version
1.4.2.1.g16fd), so thanks for the fix.

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: git-push of new, tiny branch doing more work than necessary?

[Junio C Hamano]

Carl Worth <cworth@cworth.org> writes:

> I've run through something quite like the following scenario a few
> times lately:
>
> 	git pull origin
> 	git checkout -b experiment
> 	# hack a file or two
> 	git commit -a -m "new experiment"
> 	git push origin experiment
>
> What I expect at this point is for git to push the few newly created
> objects out to the repository. Instead it talks about generating,
> deltifying, transferring, and unpacking thousands of objects (see
> below).
>
> Shouldn't the same WANT/HAVE hand shaking that makes git-fetch
> efficient be doing something similar here? Or am I just doing
> something wrong?

Depending on the version of git you are running, there are two
possibilities.

There has been a change to send-pack last week pushed out to
"master", in which Andy Whitcroft fixed a problem when there are
tons of refs on either end.  Earlier we were limiting ourselves
to use 900 or so refs when generating the pack, but the new code
uses all refs it knows about (so it excludes commits pointed by
all remote refs, not just the ones that fits in the arbitrary
900 limit).

If you are seeing problem in a pair of repository with tons of
refs, it may be the problem Andy fixed with the above change, so
using the recent "master" version would help you.

If you are on the other han seeing the problem to start after
installing the recent "master", it could be that the above
change broke things for you.  If you do not see the problem with
v1.4.2.1 but do see the problem with the current "master" that
would be an indication this is the case.

Another thing to watch out for is that send-pack/receive-pack
protocol does NOT do the full "want/have" conversation like
fetch-pack/upload-pack protocol does.  It merely say "receiving
end has these at tips of branches and tags", "sending side has
these at tips of branches and tags", and does not ask about
what's befind the tips.  So if you are in this situation, with
or without Andy's fix I can imagine you would see some redundant
sending of things.

 - The origin repository has an ancient tag and the master
   branch.  Nothing else.

 - Clone origin to victim repository. The victim repository
   would now have origin and master (the same), and the tag.

 - Somebody creates a commit in the "origin" repository on the
   master branch.

 - You create a commit on the "master" in the victim
   repository.

 - You run send-pack in the victim repository and try to push
   your master to the master in the origin repository.

In this case, the update is not a fast forward, so you may need
to force it.  When you do force, send-pack running on your end
learns from the other end about the ancient tag and its
"master", the latter of which it does not know about at all
(because you haven't pulled from the origin).  So all it has to
work with as the common commit is the ancient tag -- which means
you would end up sending everything since that ancient tag til
your tip of "master".

I do not know if that is what is happening to you in this case,
though.  While I think not doing the discovery harder like
upload-pack is a bug in send-pack, it is not an issue when you
are not forcing a non-fast forward, so it probably is a lower
priority bug than other things.  To fix it properly I think we
would need a minor protocol update.