Removal of "--merge-order"?

Removal of "--merge-order"?

[Linus Torvalds]

I just tested it again, and

	git-rev-list --merge-order HEAD

takes an inordinate amount of time:

	real    5m1.139s
	user    4m59.504s
	sys     0m1.220s

and that's on a reasonably fast machine (not my fastest, but no slouch by 
any measure - my fastest machine I'm not allowed to really benchmark 
publicly ;)

It may be a cool algorithm, but it's essentially useless on any bigger 
tree. And nobody uses it, since "--topo-order" gives the guarantees that 
people really care about, and finishes in 0.537 seconds on the same 
machine with the same tree.

It also depends on the openssh "bignum" stuff, which means that any 
machine where we just rely on our own SHA1 implementation and don't use 
openssh doesn't have the flag anyway.

In other words, I'd really prefer if it was gone. Some of the things I 
might do to git-rev-list would be much simpler if I didn't have to worry 
about merge-order, and the way it interfaces with the rest of 
git-rev-list.

Comments?

			Linus

Re: Removal of "--merge-order"?

[Randy.Dunlap]

On Fri, 24 Feb 2006, Linus Torvalds wrote:

>
> I just tested it again, and
>
> 	git-rev-list --merge-order HEAD
>
> takes an inordinate amount of time:
>
> 	real    5m1.139s
> 	user    4m59.504s
> 	sys     0m1.220s

That's too bad.

> and that's on a reasonably fast machine (not my fastest, but no slouch by
> any measure - my fastest machine I'm not allowed to really benchmark
> publicly ;)
>
> It may be a cool algorithm, but it's essentially useless on any bigger
> tree. And nobody uses it, since "--topo-order" gives the guarantees that
> people really care about, and finishes in 0.537 seconds on the same
> machine with the same tree.
>
> It also depends on the openssh "bignum" stuff, which means that any
> machine where we just rely on our own SHA1 implementation and don't use
> openssh doesn't have the flag anyway.
>
> In other words, I'd really prefer if it was gone. Some of the things I
> might do to git-rev-list would be much simpler if I didn't have to worry
> about merge-order, and the way it interfaces with the rest of
> git-rev-list.
>
> Comments?

I'm just a lowly user, but I see people trying to export git
trees to other SCMs, and they seem to prefer merge-order.
This is your chance to correct me about:
(a) how I am wrong; (b) how they are wrong.  8;)

I've heard/seen you say that merge-order is not interesting,
but I still believe that *your* merge order of the Linux kernel
tree is almost all that people really care about.
Apparently I needed to go to LCA to hear you discuss git.

-- 
~Randy

Re: Removal of "--merge-order"?

[Linus Torvalds]

On Fri, 24 Feb 2006, Randy.Dunlap wrote:
> 
> I'm just a lowly user, but I see people trying to export git
> trees to other SCMs, and they seem to prefer merge-order.
> This is your chance to correct me about:
> (a) how I am wrong; (b) how they are wrong.  8;)

Well, I didn't even realize anybody at all was using it. I've never seen 
any mention of it, and considering how ungodly slow it is, I would have 
expected somebody to pipe up about it..

I did a google search for "git" and "merge-order", and the only actual use 
(as opposed to mention in a man-page) I found in the 20 hits google showed 
was an old version of gitk.

> I've heard/seen you say that merge-order is not interesting,
> but I still believe that *your* merge order of the Linux kernel
> tree is almost all that people really care about.

Could you actually point to somebody using it? They're hiding it well.

> Apparently I needed to go to LCA to hear you discuss git.

I certainly never delved into any of that.. 

		Linus

Re: Removal of "--merge-order"?

[Ryan Anderson]

On Fri, Feb 24, 2006 at 09:23:24AM -0800, Linus Torvalds wrote:
> On Fri, 24 Feb 2006, Randy.Dunlap wrote:
> > 
> > I'm just a lowly user, but I see people trying to export git
> > trees to other SCMs, and they seem to prefer merge-order.
> > This is your chance to correct me about:
> > (a) how I am wrong; (b) how they are wrong.  8;)
> 
> Well, I didn't even realize anybody at all was using it. I've never seen 
> any mention of it, and considering how ungodly slow it is, I would have 
> expected somebody to pipe up about it..
> 
> I did a google search for "git" and "merge-order", and the only actual use 
> (as opposed to mention in a man-page) I found in the 20 hits google showed 
> was an old version of gitk.

http://www.gelato.unsw.edu.au/archives/git/0511/12965.html

But topo-order would probably work as well, the default ordering just
didn't work correctly in my tests.

Certainly not a case that votes *against* removal, just noting an actual
user at one point.

-- 

Ryan Anderson
  sometimes Pug Majere

Re: Removal of "--merge-order"?

[Randy.Dunlap]

On Fri, 24 Feb 2006, Linus Torvalds wrote:

>
>
> On Fri, 24 Feb 2006, Randy.Dunlap wrote:
> >
> > I'm just a lowly user, but I see people trying to export git
> > trees to other SCMs, and they seem to prefer merge-order.
> > This is your chance to correct me about:
> > (a) how I am wrong; (b) how they are wrong.  8;)
>
> Well, I didn't even realize anybody at all was using it. I've never seen
> any mention of it, and considering how ungodly slow it is, I would have
> expected somebody to pipe up about it..
>
> I did a google search for "git" and "merge-order", and the only actual use
> (as opposed to mention in a man-page) I found in the 20 hits google showed
> was an old version of gitk.
>
> > I've heard/seen you say that merge-order is not interesting,
> > but I still believe that *your* merge order of the Linux kernel
> > tree is almost all that people really care about.
>
> Could you actually point to somebody using it? They're hiding it well.

Other than Ryan's reply, I found 2 users in a quick search,
but they have already stated that they are willing to change, so I
don't see objections unless someone else comes forward.

(Martin Langhoff, archimport)
http://marc.theaimsgroup.com/?l=git&m=112682069025547&w=2
Jon Seymour:
http://marc.theaimsgroup.com/?l=git&m=112998877717814&w=2

> > Apparently I needed to go to LCA to hear you discuss git.
>
> I certainly never delved into any of that..

Darn.

-- 
~Randy

Re: Removal of "--merge-order"?

[Linus Torvalds]

On Fri, 24 Feb 2006, Randy.Dunlap wrote:
>
> Other than Ryan's reply, I found 2 users in a quick search,
> but they have already stated that they are willing to change, so I
> don't see objections unless someone else comes forward.

One thing we could do - and might be simpler - is to make the merge-order 
thing be a post-processing phase of git-rev-list.

IOW, instead of

	git-rev-list --merge-order

we could perhaps do

	git-rev-list --parents [--topo-order?] | git-merge-order

so that the merge-order code wouldn't impact git-rev-list itself.

As it is, the merge-order code ends up hooking into the "process_commit" 
thing (and thus to "filter_commit" which does the parent rewriting, and 
then show_commit), which makes it harder to work with.

Now, rev-list.c is not the biggest file (apply.c is about twice the size), 
but in many ways it's the most complex one by far. It's also the most 
performance-critical one, and the one that it would be really nice if we 
were to be able to libify it.

For example, instead of the horrid scriping language, I _think_ I could 
almost libify it by just hooking into "show_commit", and using a callback 
function for that (and then the stand-alone program would just make the 
callback function be one that prints out the commit). 

With some care, we might be able to make things like "git diff" be small C 
programs (or, more likely, to save space and not replicate the binaries 
many times - make the "git" binary able to do all the simple things on its 
own: "git-diff" would be just a link to "git").

That would possibly be a simpler way to get away from using nonportable 
scripts. Plain C really does remain one of the most portable things out 
there.

			Linus

Re: Removal of "--merge-order"?

[Randy.Dunlap]

On Fri, 24 Feb 2006, Linus Torvalds wrote:

>
>
> On Fri, 24 Feb 2006, Randy.Dunlap wrote:
> >
> > Other than Ryan's reply, I found 2 users in a quick search,
> > but they have already stated that they are willing to change, so I
> > don't see objections unless someone else comes forward.
>
> One thing we could do - and might be simpler - is to make the merge-order
> thing be a post-processing phase of git-rev-list.
>
> IOW, instead of
>
> 	git-rev-list --merge-order
>
> we could perhaps do
>
> 	git-rev-list --parents [--topo-order?] | git-merge-order
>
> so that the merge-order code wouldn't impact git-rev-list itself.

Makes sense to me... thanks.
But even that may not be needed if noone else really needs it.

> As it is, the merge-order code ends up hooking into the "process_commit"
> thing (and thus to "filter_commit" which does the parent rewriting, and
> then show_commit), which makes it harder to work with.
>
> Now, rev-list.c is not the biggest file (apply.c is about twice the size),
> but in many ways it's the most complex one by far. It's also the most
> performance-critical one, and the one that it would be really nice if we
> were to be able to libify it.
>
> For example, instead of the horrid scriping language, I _think_ I could
> almost libify it by just hooking into "show_commit", and using a callback
> function for that (and then the stand-alone program would just make the
> callback function be one that prints out the commit).
>
> With some care, we might be able to make things like "git diff" be small C
> programs (or, more likely, to save space and not replicate the binaries
> many times - make the "git" binary able to do all the simple things on its
> own: "git-diff" would be just a link to "git").
>
> That would possibly be a simpler way to get away from using nonportable
> scripts. Plain C really does remain one of the most portable things out
> there.

-- 
~Randy

Re: Removal of "--merge-order"?

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> In other words, I'd really prefer if it was gone. Some of the things I 
> might do to git-rev-list would be much simpler if I didn't have to worry 
> about merge-order, and the way it interfaces with the rest of 
> git-rev-list.
>
> Comments?
>
> 			Linus

I am really glad you brought it up.  I would not miss it at all.

Re: Removal of "--merge-order"?

[Johannes Schindelin]

Hi,

On Fri, 24 Feb 2006, Linus Torvalds wrote:

> Now, rev-list.c is not the biggest file (apply.c is about twice the size), 
> but in many ways it's the most complex one by far. It's also the most 
> performance-critical one, and the one that it would be really nice if we 
> were to be able to libify it.

This is what I wanted to try today, but unfortunately I had to do real 
work :-(

> For example, instead of the horrid scriping language, I _think_ I could 
> almost libify it by just hooking into "show_commit", and using a callback 
> function for that (and then the stand-alone program would just make the 
> callback function be one that prints out the commit). 

I don't find the scripting language you invented particularly horrid. 
Maybe some odd things (like "if" branching to the "else" block whenever 
*any* argument was passed), but not horrid.

But in the end I would prefer a libified git, if only to get rid of 
double parsing (if you pipe the output of git-rev-list to another git 
program, chances are that you parse the commit objects at least twice).

> That would possibly be a simpler way to get away from using nonportable 
> scripts. Plain C really does remain one of the most portable things out 
> there.

Yes.

Ciao,
Dscho