[PATCH 0/8] git-repack --max-pack-size

[PATCH 0/8] git-repack --max-pack-size

[Dana How]

The three most common ways of making large packfiles
are git-fast-import, the first git-repack, or git-repack -a.
The first already supports a "--max-pack-size=N" option,
which limits the resulting packfiles to N megabytes.
This patchset adds the same option, with the same
behavior, to git-repack to handle the other two cases.

This iteration fixes miscellaneous issues discussed on the list
and introduces no behavior not seen elsewhere in git.
It is based on "next" in order to incorporate Nicolas Pitre's work.

Re: [PATCH 0/8] git-repack --max-pack-size

[Junio C Hamano]

Dana How <danahow@gmail.com> writes:

> It is based on "next" in order to incorporate Nicolas Pitre's work.

Heh, since last night, 'master' and 'next' points at exactly the
same tree object, although their commit ancestry are vastly
different for obvious reasons.

Which leaves 'master' right now at v1.5.2-rc1 while 'next' at
v1.5.2-rc1-687-gcb3892c; we might want to do something about
this apparent discrepancy.

Re: [PATCH 0/8] git-repack --max-pack-size

[Shawn O. Pearce]

Junio C Hamano <junkio@cox.net> wrote:
> Heh, since last night, 'master' and 'next' points at exactly the
> same tree object, although their commit ancestry are vastly
> different for obvious reasons.
> 
> Which leaves 'master' right now at v1.5.2-rc1 while 'next' at
> v1.5.2-rc1-687-gcb3892c; we might want to do something about
> this apparent discrepancy.

Are you considering rewinding next to master?  I'm not sure how
else we could cancel out all of those merges and reverts of bad
ideas from next.

One option might be to teach git-describe to look for matching trees,
and see if git-describe could match the tree that v1.5.2-rc1 points
at with the tree that cb3892c points and, and just call the latter
the former...  But that sounds a little risky to me in some projects,
I'm not convinced that is always the right way to describe a history.

-- 
Shawn.

Re: [PATCH 0/8] git-repack --max-pack-size

[Andy Parkins]

On Tuesday 2007 May 01, Junio C Hamano wrote:

> Which leaves 'master' right now at v1.5.2-rc1 while 'next' at
> v1.5.2-rc1-687-gcb3892c; we might want to do something about
> this apparent discrepancy.

It's perfect - I'd say that it's exactly right.

git-describe is for making unique - human readable names for points in 
history, not for describing the tree.  It makes no difference that A and B 
have the same tree, they are different points.

I've always thought of git-describe as being a way of mapping a commit hash to 
a nicer looking name.  If A and B are different commits then they should have 
different names.



Andy
-- 
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com

Re: [PATCH 0/8] git-repack --max-pack-size

[Junio C Hamano]

Andy Parkins <andyparkins@gmail.com> writes:

> On Tuesday 2007 May 01, Junio C Hamano wrote:
>
>> Which leaves 'master' right now at v1.5.2-rc1 while 'next' at
>> v1.5.2-rc1-687-gcb3892c; we might want to do something about
>> this apparent discrepancy.
>
> It's perfect - I'd say that it's exactly right.
>
> git-describe is for making unique - human readable names for points in 
> history, not for describing the tree.  It makes no difference that A and B 
> have the same tree, they are different points.
>
> I've always thought of git-describe as being a way of mapping a commit hash to 
> a nicer looking name.  If A and B are different commits then they should have 
> different names.

Although 'next' and 'master' have vastly different histories
behind them, among 685 commits between master and next, only 27
are non-merges, among which 20 are "real" commits that tried to
advance various topics that failed, and 7 are reverts against
these 20 commits [*1*].

Even though hstory of failed attempts (why they initially seemed
good ideas, how they tried to solve problems, and why they
turned out not to be so good ideas in the end) are interesting,
if we ignore the failed attempts, iow, if we view the history
from the point of view of "surviving features", the development
history of 'next' and 'master' are moral equivalents.  Yes, most
of them were merged way earlier in 'next' than 'master', many of
them were merged in multiple steps of two and three to 'next'
and then finally merged to 'master' with a single merge, but the
changes did hit both 'master' and 'next' (obviously, that is why
we ended up with the same trees).

So the equivalence of 'master' and 'next' tonight is not quite
the same as equating two random commits that happen to have the
same tree.  v1.5.2-rc1 and v1.5.2-rc1-687-gcb3892c should
naturally have the same tree because they share conceptually the
same history.

But I was not talking about changing describe output because of
the above argument.  What I was wondering was that it might be a
good idea to loosen the promise of never rewinding 'next'.  It
might be easier to view the history of 'next' during development
for each cycle, if it started afresh after a feature release.

Since now we are in a stabilization freeze, I expect that
'master' and 'next' will always have identical trees until
v1.5.2 final.  We _could_ declare now that 'next' will be reset
to 'master' when v1.5.2 happens, and people who forked from
'next' to do their own customization can rebase any time that is
convenient for them between tonight and v1.5.2 final.

I was not sure if that is even a good idea, and I am now
inclined to think that keeping the failed attempt history is
probably better than potentially causing confusion to people who
follow 'next'.  But it _is_ a possibility to reset 'next' to
'master'.


[Footnote]

*1* Some topics I did not use "git revert" to revert them
one-by-one; instead, I reverted a whole topic with one commit.

Re: [PATCH 0/8] git-repack --max-pack-size

[Andy Parkins]

On Tuesday 2007 May 01, Junio C Hamano wrote:

> But I was not talking about changing describe output because of
> the above argument.  What I was wondering was that it might be a
> good idea to loosen the promise of never rewinding 'next'.  It
> might be easier to view the history of 'next' during development
> for each cycle, if it started afresh after a feature release.

This is an interesting philosophy-of-version-control question.  If two 
identical trees fall in the forest and there is no one there to diff them, 
was a release made? :-)

It's been my experience that failed attempts and dead-end branches are often 
of equal value to the successful branches.  It's very handy when someone 
asks "why can't we do it like this", to be able to answer "look at revision 
xyz onwards".  Even just for your own reference, I've often looked back on 
abandoned paths and thought "that wasn't as bad as I thought, I just need to 
fix it here and here" - if I'd discarded that work it would be gone forever.

It's certainly true in academia, a large part of my doctoral thesis was 
about "things that don't work" :-)  Documenting failure is as important as 
documenting success.



Andy
-- 
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com

Re: [PATCH 0/8] git-repack --max-pack-size

[Nicolas Pitre]

On Tue, 1 May 2007, Junio C Hamano wrote:

> I was not sure if that is even a good idea, and I am now
> inclined to think that keeping the failed attempt history is
> probably better than potentially causing confusion to people who
> follow 'next'.  But it _is_ a possibility to reset 'next' to
> 'master'.

And what is the advantage of doing that, exactly?


Nicolas

Re: [PATCH 0/8] git-repack --max-pack-size

[Junio C Hamano]

Nicolas Pitre <nico@cam.org> writes:

> On Tue, 1 May 2007, Junio C Hamano wrote:
>
>> I was not sure if that is even a good idea, and I am now
>> inclined to think that keeping the failed attempt history is
>> probably better than potentially causing confusion to people who
>> follow 'next'.  But it _is_ a possibility to reset 'next' to
>> 'master'.
>
> And what is the advantage of doing that, exactly?

Not much and that is why I am not sure if that is a good idea,
but one thing is:

	$ git log master..next
	$ git log --no-merges master..next

would start showing what are still pending in next without
noise.

With the current 'next', the output of the former is almost
useless, and the latter is useful only until 'next' starts
diverging again from the master and if you are interested in
what will never merged to 'master'.  Once 'next' starts growing
again, you will get mixture of what's cooking and what's killed;
the output from the latter would become useless again.

Re: [PATCH 0/8] git-repack --max-pack-size

[Nicolas Pitre]

On Tue, 1 May 2007, Junio C Hamano wrote:

> Nicolas Pitre <nico@cam.org> writes:
> 
> > On Tue, 1 May 2007, Junio C Hamano wrote:
> >
> >> I was not sure if that is even a good idea, and I am now
> >> inclined to think that keeping the failed attempt history is
> >> probably better than potentially causing confusion to people who
> >> follow 'next'.  But it _is_ a possibility to reset 'next' to
> >> 'master'.
> >
> > And what is the advantage of doing that, exactly?
> 
> Not much and that is why I am not sure if that is a good idea,
> but one thing is:
> 
> 	$ git log master..next
> 	$ git log --no-merges master..next
> 
> would start showing what are still pending in next without
> noise.

If you do so, then please tag the branch before resetting it.  I think 
it contains valuable data that would get lost otherwise.


Nicolas

Re: [PATCH 0/8] git-repack --max-pack-size

[Johannes Schindelin]

Hi,

On Tue, 1 May 2007, Junio C Hamano wrote:

> Nicolas Pitre <nico@cam.org> writes:
> 
> > On Tue, 1 May 2007, Junio C Hamano wrote:
> >
> >> I was not sure if that is even a good idea, and I am now
> >> inclined to think that keeping the failed attempt history is
> >> probably better than potentially causing confusion to people who
> >> follow 'next'.  But it _is_ a possibility to reset 'next' to
> >> 'master'.
> >
> > And what is the advantage of doing that, exactly?
> 
> Not much and that is why I am not sure if that is a good idea,
> but one thing is:
> 
> 	$ git log master..next
> 	$ git log --no-merges master..next
> 
> would start showing what are still pending in next without
> noise.

Of course, you could graft a fake relationship there. Something like

echo $(git rev-parse master master^ next) >> .git/info/grafts

should do. And you can do it _locally_, since all you care about is the 
higher-level relationship that _you_ know about. And this is the way to 
teach Git about it.

Ciao,
Dscho

Re: [PATCH 0/8] git-repack --max-pack-size

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi,
>
> On Tue, 1 May 2007, Junio C Hamano wrote:
>
>> Nicolas Pitre <nico@cam.org> writes:
>> 
>> > On Tue, 1 May 2007, Junio C Hamano wrote:
>> >
>> >> I was not sure if that is even a good idea, and I am now
>> >> inclined to think that keeping the failed attempt history is
>> >> probably better than potentially causing confusion to people who
>> >> follow 'next'.  But it _is_ a possibility to reset 'next' to
>> >> 'master'.
>> >
>> > And what is the advantage of doing that, exactly?
>> 
>> Not much and that is why I am not sure if that is a good idea,
>> but one thing is:
>> 
>> 	$ git log master..next
>> 	$ git log --no-merges master..next
>> 
>> would start showing what are still pending in next without
>> noise.
>
> Of course, you could graft a fake relationship there. Something like
>
> echo $(git rev-parse master master^ next) >> .git/info/grafts
>
> should do. And you can do it _locally_, since all you care about is the 
> higher-level relationship that _you_ know about. And this is the way to 
> teach Git about it.

In case you did not get it, making the above usable is not about _me_.

[PATCH 0/8] git-repack --max-pack-size

[Dana How]

The three most common ways of making large packfiles
are git-fast-import, the first git-repack, or git-repack -a.
The first already supports a "--max-pack-size=N" option,
which limits the resulting packfiles to N megabytes.
This patchset adds the same option, with the same
behavior, to git-repack to handle the other two cases.

After reviewing others' comments,  this latest patchset
touches 2/3 fewer files and 1/6 fewer lines,  and
introduces no behavior not seen elsewhere in git.

Re: [PATCH 0/8] git-repack --max-pack-size

[Nicolas Pitre]

On Sun, 8 Apr 2007, Dana How wrote:

> After reviewing others' comments,  this latest patchset
> touches 2/3 fewer files and 1/6 fewer lines,  and
> introduces no behavior not seen elsewhere in git.

I'm starting to like it, but...

I have some extensive changes that touch the same area more or less.  
And so far I think it might be easier to rebase your changes on top of 
mine rather than the other way around.

So I'd like if my changes would be merged first and then yours to be 
applied afterwards.  They still have some issues left anyway.

I'm trying to format my changes now and send them tonight.


Nicolas

Re: [PATCH 0/8] git-repack --max-pack-size

[Dana How]

On 4/8/07, Nicolas Pitre <nico@cam.org> wrote:
> On Sun, 8 Apr 2007, Dana How wrote:
>
> > After reviewing others' comments,  this latest patchset
> > touches 2/3 fewer files and 1/6 fewer lines,  and
> > introduces no behavior not seen elsewhere in git.
>
> I'm starting to like it, but...
>
> I have some extensive changes that touch the same area more or less.
> And so far I think it might be easier to rebase your changes on top of
> mine rather than the other way around.
>
> So I'd like if my changes would be merged first and then yours to be
> applied afterwards.  They still have some issues left anyway.
>
> I'm trying to format my changes now and send them tonight.
Thanks for sending them out.
I guess I'll update mine to apply after yours once
yours show up in the appropriate branch.

Thanks,
-- 
Dana L. How  danahow@gmail.com  +1 650 804 5991 cell

Re: [PATCH 0/8] git-repack --max-pack-size

[Dana How]

On 4/8/07, Dana How <danahow@gmail.com> wrote:
> The three most common ways of making large packfiles
> are git-fast-import, the first git-repack, or git-repack -a.
> The first already supports a "--max-pack-size=N" option,
> which limits the resulting packfiles to N megabytes.
> This patchset adds the same option, with the same
> behavior, to git-repack to handle the other two cases.
>
> After reviewing others' comments,  this latest patchset
> touches 2/3 fewer files and 1/6 fewer lines,  and
> introduces no behavior not seen elsewhere in git.

At the moment,  I plan the following changes to this patchset:
* Redo after 64b index applied (Nicolas)
* Bad re-use of header around sha1write (Nicolas)
* Set object_entry.offset before write_object (Dana)
* Finish reducing struct object_entry (Junio)
* Remove spaces from inside () (Junio)

Concerning refactorings,
I'd prefer to keep those in a separate (follow-on?) patchset.

Thanks,
-- 
Dana L. How  danahow@gmail.com  +1 650 804 5991 cell