new features in gitk

new features in gitk

[Paul Mackerras]

New features that I have added recently to the gitk repository at

rsync://rsync.kernel.org/pub/scm/gitk/gitk.git

include:

* Added a right-click context menu on the headlines displayed in the
  top-left pane with the following entries:
  - Display diff between two commits
  - Generate a patch file containing the diff between two commits
  - Create a local direct tag for a commit

* Left-click on a graph line now displays the parent and the children
  connected by that line in the bottom-left pane, with buttons
  allowing you to jump to the parent or a child.

* Pasting into the SHA1 ID field clears it first if it already
  contains a SHA1 ID.

* Checks for a .git (or $GIT_DIR) directory at startup.

I'm interested to know if people find the diff/patch generation
options useful.

Paul.

Re: new features in gitk

[Luc Van Oostenryck]

Paul Mackerras wrote:
> 
> I'm interested to know if people find the diff/patch generation
> options useful.
> 
> Paul.


I find the patch generation very usefull (when I've seen the tool a few days ago, I've said to myself:
"what a wonderfull tool, if only I could create a patch from this")
but it doesn't work for me (the first three entries stay always in grey, only the last one "Create tag" is in black).
I'm missing something ?


Luc

Re: new features in gitk

[Paul Mackerras]

Luc Van Oostenryck writes:

> I find the patch generation very usefull (when I've seen the tool a few days ago, I've said to myself:
> "what a wonderfull tool, if only I could create a patch from this")
> but it doesn't work for me (the first three entries stay always in grey, only the last one "Create tag" is in black).
> I'm missing something ?

The diffs and patches are between the selected commit (one that you
clicked on with the left button and which is shown with a grey
background) and the commit where you click the right button.  That was
the best way I could think of to indicate which were the two commits
to diff.  If you (or anyone) has a better suggestion, let me know.

Paul.

Re: new features in gitk

[Linus Torvalds]

On Tue, 28 Jun 2005, Paul Mackerras wrote:
>
> New features that I have added recently to the gitk repository at

Hey, cool, the automatic merge actually worked.

I knew it would, of course, but still.. It's nice when theory and practice 
end up matching.

		Linus

Re: new features in gitk

[Greg KH]

On Tue, Jun 28, 2005 at 08:22:46AM +1000, Paul Mackerras wrote:
> I'm interested to know if people find the diff/patch generation
> options useful.

Well, I find them nice to have, in case I need to generate a real patch.
Nice job.

thanks,

greg k-h

Re: new features in gitk

[Luc Van Oostenryck]

Paul Mackerras wrote:
> Luc Van Oostenryck writes:
> 
> 
>>I find the patch generation very usefull (when I've seen the tool a few days ago, I've said to myself:
>>"what a wonderfull tool, if only I could create a patch from this")
>>but it doesn't work for me (the first three entries stay always in grey, only the last one "Create tag" is in black).
>>I'm missing something ?
> 
> 
> The diffs and patches are between the selected commit (one that you
> clicked on with the left button and which is shown with a grey
> background) and the commit where you click the right button.  That was
> the best way I could think of to indicate which were the two commits
> to diff.  If you (or anyone) has a better suggestion, let me know.
> 
> Paul.

OK I see it now, it works nicely.
I didn't found the tric yesterday because I was thinking: one commit -> one patch,
but it's probably more usefull like it is now.

Thanks.

Re: new features in gitk

[Paul Mackerras]

Linus Torvalds writes:

> Hey, cool, the automatic merge actually worked.

Yes, very cool.  It'll get more interesting if you make changes to
gitk in your repository, of course. :)

Paul.

Re: new features in gitk

[Junio C Hamano]

Finally I started converting my workflow of day-job I do on my
home machine from darcs to git, so I started running cvs2git
over ssh slurping from my office environment (sloooooooooow).

While I was watching the import going, I started up gitk on the
halfway imported repository, and it showed Japanese characters
in the source correctly without any special configuration (the
only thing I have is LC_CTYPE=ja_JP).  Very often GUIish
software would not work properly for me with Japanese
characters, and this was a pleasant surprise.

The credit is probably owed more to tk, not something you do
special in gitk, but nevertheless I am impressed and happy ;-).

New features in gitk

[Paul Mackerras]

I just pulled the dev branch of gitk into the master branch, so the
master branch now has the new features and improvements that I have
been working on, namely:

* The find and highlight functions have been combined into a single
  function, and there is now a button for finding backwards as well as
  a find forwards button.  Thus you can now search for commits that
  modify certain files or directories, or commits that add/remove a
  given string, as well as searching for commits by commit message,
  author, committer or headline.

* Combining the find and highlight functions freed up space that is
  now used for a progress bar and a status window.

* There is now a font chooser accessible from the edit/preferences
  window.

* Gitk now uses a new graph layout algorithm, which means it doesn't
  have to generate the whole layout from top to bottom at startup
  time, making startup faster.  Gitk also uses a new style for drawing
  short diagonal line segments that join an existing vertical line,
  which is visually clearer when the segment crosses another line.

* Gitk caches the topology information used for the previous/next tag
  and branch information, making startup faster.

Tk 8.5 is now in beta, meaning that some distros now have it
packaged.  Gitk looks much nicer in Tk8.5 since it supports
anti-aliased fonts, so I strongly suggest that people install and use
Tk8.5 if possible.

Paul.

Re: New features in gitk

[Linus Torvalds]

On Sun, 28 Oct 2007, Paul Mackerras wrote:
>
> I just pulled the dev branch of gitk into the master branch, so the
> master branch now has the new features and improvements that I have
> been working on, namely:

*Huge* improvements. It is now really nice to start up gitk even on the 
full kernel history.

However, that crazy green bar chasing back-and-forth int he "reading" 
phase is really quite visually distracting. Maybe it looks better in 
Tk8.5, but it does look pretty annoying in the version I have. Can you 
tone that down a bit? 

But this has both the layout performance improvements and the fixes to 
only show selected files in the diff view by default, so I hope this gets 
merged into standard git soon..

		Linus

Re: New features in gitk

[Paul Mackerras]

Linus Torvalds writes:

> However, that crazy green bar chasing back-and-forth int he "reading" 
> phase is really quite visually distracting. Maybe it looks better in 
> Tk8.5, but it does look pretty annoying in the version I have. Can you 
> tone that down a bit? 

Yeah.  Actually what I'd like is to know how many commits git log is
going to give me, so that I can do a normal progress bar whose length
is proportional to commits_read / total_commits.  With --topo-order
(or --date-order) it has to get to the last commit before it outputs
the first commit, doesn't it?  So could it print the total number of
commits on a line by itself at the start of its output?  (Presumably
it would need a --commit-count flag to enable that behaviour.)

Other than that, I could slow the progress bar down, or do a bar of
moving diagonal stripes, or something.

Paul.

Re: New features in gitk

[Steffen Prohaska]

On Oct 28, 2007, at 8:11 AM, Paul Mackerras wrote:

> Linus Torvalds writes:
>
>> However, that crazy green bar chasing back-and-forth int he "reading"
>> phase is really quite visually distracting. Maybe it looks better in
>> Tk8.5, but it does look pretty annoying in the version I have. Can  
>> you
>> tone that down a bit?

I have the same impression.


> Yeah.  Actually what I'd like is to know how many commits git log is
> going to give me, so that I can do a normal progress bar whose length
> is proportional to commits_read / total_commits.

Can you use something like a rotating wheel, if the total size
of the task is unknown.

Or if you know an upper bound on the number of expected commits,
you could use this as total_commits. And adjust the upper
bound if more information becomes available.

Or you just print the number of commits already read and the
user is happy because something is changing.

	Steffen

Re: New features in gitk

[Linus Torvalds]

On Sun, 28 Oct 2007, Paul Mackerras wrote:
>
> Yeah.  Actually what I'd like is to know how many commits git log is
> going to give me

That's not known until later.

> With --topo-order (or --date-order) it has to get to the last commit 
> before it outputs the first commit, doesn't it?

The cost is not generally in outputting the commits. The real cost is in 
traversing them in the first place. 

So yes, we could output the number of commits once we know it, but 
generally, by that time, it's not an interesting number any more! You 
might as well just read the list, because git is going to feed it to you 
as fast as it can (which is plenty fast - you'll probably get hundreds of 
megabytes of SHA1 values per second at that point).

So basically, by the time you start getting SHA1's from --topo-order, the 
best thing you can do is just lay back and think of England. The *last* 
thing you want to do is bother with any graphics and updates, because it's 
just going to slow things down.

It's before you even start getting the SHA1's, _or_ if you don't use 
"--date/topo-order" in the first place, that you want to have a "wait, I'm 
thinking" thing. And at neither time do you know how long it's going to 
be.

(And as mentioned many times earlier - if you can avoid topo-order and 
date-order entirely, you are going to perform a million times better at 
startup for the cold-cache case. Since you seem to be doing the graph 
layout lazily now, maybe you could aim for that some day? It does mean 
that you might - occasionally - end up having to add a commit to 
*before* one you already laid out).

		Linus

Re: New features in gitk

[Pierre Habouzit]

[-- Attachment #1: Type: text/plain, Size: 2629 bytes --]

On dim, oct 28, 2007 at 01:39:34 +0000, Paul Mackerras wrote:
> I just pulled the dev branch of gitk into the master branch, so the
> master branch now has the new features and improvements that I have
> been working on, namely: [...]

As you seem to be the guy to ask for, I've a couple of requests wrt
gitk.

  * the diff window is quite bad with merge commits, the colorization is
    rather poor, and the last version you just merged isn't especially
    better.

  * the 'sha1' input field is a major pain in the UI: the cut&paste
    interaction is very poor. I don't know why, but it's often very very
    hard to really copy the sha id, probably because it's selected by
    default.

  * hjkl in the history list do very very very curious things, whereas I
    would expect j/k to do the same as (resp) down/up. Note that in
    [Help->Key bindings] it's said it should work that way, but it
    doesn't here at least. A way to customize bindings would be much
    appreciated (I like vi bindings, and I miss ^U/^D, and ^E/^Y e.g.).

  * I really really really miss an option to ignore whitespaces in
    diffs, a small checkbox to view the full blown diff, or the one
    without spaces changes (-w -b) would be _really_ great.

  * the fact that it remembers the position where it was in the WM when
    it was closed is really annoying. the WM is supposed to place the
    window. With at least ion3 and xinerama it often shows up on the
    wrong screen. Remembering the window size though is fine.

  * wrt the layout, when the gitk window is resized, the resizing of the
    three columns (subjects, commiter, date) is really cumbersome. I
    would expect that the subject one would be the sole one to be
    resized.

  * still wrt the layout, the focus is quite cumbersome. Gitk would be
    really really really nice to be used only from the keyboard, but
    because of a very unclear focus policy, it really isn't for me.
    Maybe it's just me, and I know this may not be 100% helpful, but I
    never know which part of gitk will receive my keys (history part,
    diff part, tree, ...).

  * in the diff [lines of context] input, if you hit "down" it
    decrements the number of lines which is okay, but _also_ moves the
    selected history line which is not.


  This list may sound harsh, I hope not, I love gitk, it's one of the
10 git commands I use the most.

Cheers,
-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: New features in gitk

[Mike Hommey]

On Sun, Oct 28, 2007 at 07:32:16PM +0100, Pierre Habouzit wrote:
> On dim, oct 28, 2007 at 01:39:34 +0000, Paul Mackerras wrote:
> > I just pulled the dev branch of gitk into the master branch, so the
> > master branch now has the new features and improvements that I have
> > been working on, namely: [...]
> 
> As you seem to be the guy to ask for, I've a couple of requests wrt
> gitk.
(...)
    * When running gitk --all, it would be nice if the current branch
      was selected, instead of the topmost commit.

Mike

Re: New features in gitk

[Paul Mackerras]

Pierre Habouzit writes:

> As you seem to be the guy to ask for, I've a couple of requests wrt
> gitk.
> 
>   * the diff window is quite bad with merge commits, the colorization is
>     rather poor, and the last version you just merged isn't especially
>     better.

That's not a request, that's a grizzle. :)  What would you like it to
look like?

>   * the 'sha1' input field is a major pain in the UI: the cut&paste
>     interaction is very poor. I don't know why, but it's often very very
>     hard to really copy the sha id, probably because it's selected by
>     default.

It's selected so that the contents are in the cut buffer and you can
paste them in an xterm with middle-button.  Possibly I need to check
that control-C (or command-C under macos) is properly bound to copy.

>   * the fact that it remembers the position where it was in the WM when
>     it was closed is really annoying. the WM is supposed to place the
>     window. With at least ion3 and xinerama it often shows up on the
>     wrong screen. Remembering the window size though is fine.

That came in with some changes that make gitk start up correctly under
windows.  I could see about making it set the window position only
under windows.

>   * still wrt the layout, the focus is quite cumbersome. Gitk would be
>     really really really nice to be used only from the keyboard, but
>     because of a very unclear focus policy, it really isn't for me.
>     Maybe it's just me, and I know this may not be 100% helpful, but I
>     never know which part of gitk will receive my keys (history part,
>     diff part, tree, ...).

What focus policy would you like?

Paul.

Re: New features in gitk

[Pierre Habouzit]

[-- Attachment #1: Type: text/plain, Size: 3758 bytes --]

On Sun, Oct 28, 2007 at 11:13:43PM +0000, Paul Mackerras wrote:
> Pierre Habouzit writes:
> 
> > As you seem to be the guy to ask for, I've a couple of requests wrt
> > gitk.
> > 
> >   * the diff window is quite bad with merge commits, the colorization is
> >     rather poor, and the last version you just merged isn't especially
> >     better.
> 
> That's not a request, that's a grizzle. :)  What would you like it to
> look like?

  I believe that git show/diff has it right: lines with a + should be in
the "added" color, and lines with a '-' in the "removed" one. gitk only
take the first "column" of +/- into account or sth I find awkward at
best, and I often go to the console to see a merge commit because of
that.

> >   * the 'sha1' input field is a major pain in the UI: the cut&paste
> >     interaction is very poor. I don't know why, but it's often very very
> >     hard to really copy the sha id, probably because it's selected by
> >     default.
>
> It's selected so that the contents are in the cut buffer and you can
> paste them in an xterm with middle-button.  Possibly I need to check
> that control-C (or command-C under macos) is properly bound to copy.

  Well, doing ^C doesn't always copy it (probably a glitch wrt which
input has the focus), and it certainly doesn't synchronize with the cut
buffer for me. And it doesn't work for anyone at work either. I use ion
with the KDE clipboard manager (klipper -- because I never managed to
find a clipboard manager that is as good yet, not depending upon KDE),
and at work most people use KDE, with the same klipper. Maybe it's a bad
interaction, I should try to use it under gnome or so to see if it is.

> >   * the fact that it remembers the position where it was in the WM when
> >     it was closed is really annoying. the WM is supposed to place the
> >     window. With at least ion3 and xinerama it often shows up on the
> >     wrong screen. Remembering the window size though is fine.
> 
> That came in with some changes that make gitk start up correctly under
> windows.  I could see about making it set the window position only
> under windows.

  That'd be really great.

> >   * still wrt the layout, the focus is quite cumbersome. Gitk would be
> >     really really really nice to be used only from the keyboard, but
> >     because of a very unclear focus policy, it really isn't for me.
> >     Maybe it's just me, and I know this may not be 100% helpful, but I
> >     never know which part of gitk will receive my keys (history part,
> >     diff part, tree, ...).
>
> What focus policy would you like?

  Well, what would make sense (to _me_ at least) would be some shortcuts
to move to the history panel (say e.g. using F1), or to the diff view
(using e.g. F2), or in the file list (say F3).
  That would hilight with a black 1px line (like it does for other
inputs fields) to say that this is the primary window part that takes
the keyboard inputs atm. And when doing that, if you press 'down' or
'up' it would scroll the adequate panel. It's really confusing that the
keyboard (or hjkl) right now always make the history change.

  This way you can make the difference between the keyboard shortcuts
that apply to the focused part of the window (up/down, pgup/pgdown are
IMHO of that kind), and the one that the user (or the default gitk) has
associated to a specific part, no matter if it has the focus. E.g. J/K
(or for emacsish people ^N/^P) could always move the history, that would
make sense.

Cheers,
-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: New features in gitk

[Pierre Habouzit]

[-- Attachment #1: Type: text/plain, Size: 698 bytes --]

On dim, oct 28, 2007 at 11:13:43 +0000, Paul Mackerras wrote:
> Pierre Habouzit writes:
> 
> > As you seem to be the guy to ask for, I've a couple of requests wrt
> > gitk.
> > 
> >   * the diff window is quite bad with merge commits, the colorization is
> >     rather poor, and the last version you just merged isn't especially
> >     better.
> 
> That's not a request, that's a grizzle. :)

  Right, would have I known a single word of Tcl, I would have provided
patches for that long time ago btw :P

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: New features in gitk

[Jonathan del Strother]

On 29 Oct 2007, at 06:20, Pierre Habouzit wrote:

> On Sun, Oct 28, 2007 at 11:13:43PM +0000, Paul Mackerras wrote:
>>>   * the 'sha1' input field is a major pain in the UI: the cut&paste
>>>     interaction is very poor. I don't know why, but it's often  
>>> very very
>>>     hard to really copy the sha id, probably because it's  
>>> selected by
>>>     default.
>>
>> It's selected so that the contents are in the cut buffer and you can
>> paste them in an xterm with middle-button.  Possibly I need to check
>> that control-C (or command-C under macos) is properly bound to copy.
>
>   Well, doing ^C doesn't always copy it (probably a glitch wrt which
> input has the focus), and it certainly doesn't synchronize with the  
> cut
> buffer for me. And it doesn't work for anyone at work either. I use  
> ion
> with the KDE clipboard manager (klipper -- because I never managed to
> find a clipboard manager that is as good yet, not depending upon KDE),
> and at work most people use KDE, with the same klipper. Maybe it's  
> a bad
> interaction, I should try to use it under gnome or so to see if it is.
>

FWIW, I have exactly the same problem under OS X.  I've never figured  
out a pattern that gives a guaranteed copy - I'll try playing around  
today and see what I can find.

Actually, while I'm here, gitk semi-regularly ignores ⌘Q, which  
ought to quit on OS X.

Re: New features in gitk

[Han-Wen Nienhuys]

Paul Mackerras escreveu:
> I just pulled the dev branch of gitk into the master branch, so the
> master branch now has the new features and improvements that I have
> been working on, namely:
> 
> * The find and highlight functions have been combined into a single

sound extremely cool; I didn't know someone was working on it actively.

Can I misuse this thread to bring a ancient bug under your attention? 
It is affecting me regularly; see here for the report:

  http://article.gmane.org/gmane.comp.version-control.git/48789


-- 
 Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen

Re: New features in gitk

[Michele Ballabio]

On Sunday 28 October 2007, Paul Mackerras wrote:
> * Gitk now uses a new graph layout algorithm, which means it doesn't
>   have to generate the whole layout from top to bottom at startup
>   time, making startup faster.

This is probably causing gitk to eat my (admittedly old) CPU, sometimes.

For example, a

	gitk --all v1.5.2..v1.5.3

gives me problems when I scroll down to about half of the shown history:
that is when I reach the sequence of hundreds of "merge topic branch
into next" commits, and gitk tries hard to display the graph as best
as it can.

It can become unresponsive for one second at every PgDown.

Otherwise, gitk is way faster in other (non-pathological)
circumstances.

Re: New features in gitk

[Paul Mackerras]

Linus Torvalds writes:

> (And as mentioned many times earlier - if you can avoid topo-order and 
> date-order entirely, you are going to perform a million times better at 
> startup for the cold-cache case. Since you seem to be doing the graph 
> layout lazily now, maybe you could aim for that some day? It does mean 
> that you might - occasionally - end up having to add a commit to 
> *before* one you already laid out).

The other thing --topo-order does is reorder the commits so that
related commits come together.  So far, doing that in Tcl has turned
out to be much slower than having it done in C (within git log) for
the hot-cache case (which I expect is the common case).

I'm now thinking that the best approach would be to have gitk cache
the topology, and on startup only read in the part of the graph that
isn't in the cache.  Mostly that will be small and so git log should
be fast even in the cold-cache case with --topo-order.

Paul.

Re: New features in gitk

[Paul Mackerras]

Linus Torvalds writes:

> The cost is not generally in outputting the commits. The real cost is in 
> traversing them in the first place. 

Actually, the biggest cost is still gitk reading in the commits from
git log and doing the processing that gitk needs to do on each commit
(which I have tried to minimize, and is way smaller than it used to
be, but is still significant).

In fact that would go significantly faster if git log could output the
data for each commit in a slightly different format.  What would be
good is to get one header line for each commit in the form:

id flag {parent parent parent...} length

where:

id is the 40-char SHA1 for the commit
flag is normally 1, but is 0 for "boundary" commits, 2 for "left-side"
    commits (with --merge), or 3 for "right-side" commits
length is the number of characters of commit data that follow
    (which may differ from the number of bytes, so there would need
     to be agreement on the encoding)

followed by the body of the commit (with no null or other separator
character between commits).

That would be easier to parse in Tcl, and looks like it would knock
another 1.5 seconds off the gitk startup time (for the kernel
repository on my G5).

Paul.

Re: New features in gitk

[Linus Torvalds]

On Thu, 1 Nov 2007, Paul Mackerras wrote:
> 
> The other thing --topo-order does is reorder the commits so that
> related commits come together.

If that's the only reason for using it, then please stop, and use 
"--first-parent" instead.

		Linus

Re: New features in gitk

[Linus Torvalds]

On Thu, 1 Nov 2007, Paul Mackerras wrote:
>
> Linus Torvalds writes: 
> > The cost is not generally in outputting the commits. The real cost is in 
> > traversing them in the first place. 
> 
> Actually, the biggest cost is still gitk reading in the commits from
> git log and doing the processing that gitk needs to do on each commit
> (which I have tried to minimize, and is way smaller than it used to
> be, but is still significant).

Umm. I think you're basing all your timings on hot-cache numbers.

The hot-cache numbers are already pretty damn good. But try this:

	echo 3 > /proc/sys/vm/drop_caches
	gitk

on a big repository, _especially_ one that isn't totally packed, or on a 
machine with a slow laptop disk. Just following the commit history is 
really quite expensive.

THAT is the problem with --topo-order.

			Linus

Re: New features in gitk

[Linus Torvalds]

On Thu, 1 Nov 2007, Linus Torvalds wrote:
> 
> The hot-cache numbers are already pretty damn good. But try this:
> 
> 	echo 3 > /proc/sys/vm/drop_caches
> 	gitk

Actually, do the above with a path limiter, to make it more obvious.

So try this on the kernel, and you'll see the difference even with a fast 
disk, and even if it's fully packed:

	echo 3 > /proc/sys/vm/drop_caches
	time git rev-list HEAD drivers/scsi | head -10

and now try it with --topo-order.

I get ten seconds with --topo-order (because it has to traverse the 
*whole* history even to just generate the first ten lines), while the 
non-topo-order case is *three*times* faster.

On my laptop, it's even more noticeable. I don't know quite why, but the 
non-topo-order case is actually faster on my laptop than on my desktop 
(will have to see what's up, but I suspect it's a result of they being at 
different points in history, and just bad luck wrt the top-of-history 
having happened to change drivers/scsi or not):

	[torvalds@t40 linux]$ time git rev-list HEAD drivers/scsi | head -10
	..
	real    0m0.688s

but with --topo-order it's much slower:

	[torvalds@t40 linux]$ time git rev-list --topo-order HEAD drivers/scsi | head -10
	..
	real    0m17.458s

See? You shouldn't care about CPU usage, you should care about IO costs! 
Those are the first-order effects.

In other words, if you can be incremental, we're talking about performance 
differences that are orders-of-magnitude. Not ten percent or something 
piddling like that! And the performance improvemens come for the cases 
that are the _problem_, rather than the cases that already work perfectly 
well.

				Linus

Re: New features in gitk

[Paul Mackerras]

Linus Torvalds writes:

> If that's the only reason for using it, then please stop, and use 
> "--first-parent" instead.

How would that help?  That doesn't list about 2/3 of the commits at
all.

In any case, no that's not the only reason.  The main reason is that
it (i.e. --topo-order) spits out the commits in exactly the order that
gitk wants to display them (of which the bit about parents coming
after all their children is a part), and thus reduces the amount of
processing I need to do in Tcl.

Paul.

Re: New features in gitk

[Marco Costalba]

On 11/2/07, Paul Mackerras <paulus@samba.org> wrote:
>
> In any case, no that's not the only reason.  The main reason is that
> it (i.e. --topo-order) spits out the commits in exactly the order that
> gitk wants to display them (of which the bit about parents coming
> after all their children is a part), and thus reduces the amount of
> processing I need to do in Tcl.
>

I have tried to overcome --topo-order in qgit but I found it very
difficult, too much for me.

Lazily drawing the layout it doesn't mean that you lazy load the data
from git, indeed you load all the git-log output as soon as it
arrives.

And if the revisions arrive "in order", i.e. if revision A arrive
before revision B it means that A is NOT an ancestor of B, this is of
great help.

When drawing the graph assuming that the vector/list of the arrived
sha is already ordered greatly simplify the whole thing, if we relax
this hypothesis then a lot of work should be done before to draw a
graph chunk, essentially the GUI tool needs to walk the _entire_  list
and reorder it by itself _before_ to draw any graph chunk also if very
small.

So at the end you end up transferring the complete revision walk from
git-log to the GUI tool, and (this is the important thing) to be sure
graph is always correct you need to perform the walk _before_ drawing
any stuff.

The only possible _trick_ I was able to find is to optimistically draw
the graph chunk _assuming_ that it is ordered.

Then reorder the list in the background and finally check if the graph
is correct, if not redraw with correct data.

If the out of order revisions are rare you end up mimic a fast correct
drawing. If are not user will see some flickering at the end of the
load.

IMHO the above scheme is very complicated and fragile.

Just my two cents.

Marco

Re: New features in gitk

[Linus Torvalds]

On Fri, 2 Nov 2007, Paul Mackerras wrote:
> 
> How would that help?  That doesn't list about 2/3 of the commits at
> all.

Yeah, you'd have to do all the parent processing on your own, I guess that 
would be too slow.

> In any case, no that's not the only reason.  The main reason is that
> it (i.e. --topo-order) spits out the commits in exactly the order that
> gitk wants to display them (of which the bit about parents coming
> after all their children is a part), and thus reduces the amount of
> processing I need to do in Tcl.

The thing is, you shouldn't *care* how long it takes to get 50,000+ 
commits.

You're only visualizing ~20 commits at a time. Ignore the rest.

THAT is the number that is timing-critical. You're optimizing for the 
wrong case - the "whole history" thing doesn't matter as much as the 
"recent history" does.

So I bet from a usability standpoint, you'd be *much* better off with 
something that might take ten times as long for the whole thing, if the 
first thirty lines show up in a third of the time.

In fact, if you want to really optimize parsing and that is a big issue, 
use

	git log --left-right --parents --pretty=format:"%m %H %P %an <%ae> %aD"

which will give you a single line per commit. I bet tk is good at reading 
single lines. Don't even read anythign else - until somebody actually 
*selects* the commit, at which point you do the diff *and* the full thing.

Yes, it will make things slower over-all. And no, the above won't work for 
the "search everywhere", which means that once people start searching for 
everything, you'll be screwed, but with somethign like the above, you'll 
get the first commits immediately and can start showing them.

And yes, if they come in the wrong order, you'll have to recalculate the 
display, but I thought you had something incremental already - ie you can 
always do it for just the current window of 100 commits or whatever.

			Linus

Re: New features in gitk

[Linus Torvalds]

On Fri, 2 Nov 2007, Marco Costalba wrote:
> 
> I have tried to overcome --topo-order in qgit but I found it very
> difficult, too much for me.
> 
> Lazily drawing the layout it doesn't mean that you lazy load the data
> from git, indeed you load all the git-log output as soon as it
> arrives.

Would it be more palatable if I tried to write some visualization-specific 
front-end that acted kind of like "git rev-list", but would have some way 
of "resetting" its output?

The thing is, I'm pretty sure I can feed you commits really quickly if I 
don't sort them, and if I don't do the full and careful "oops, this commit 
was reachable from a commit that was marked uninteresting", but while the 
fast-and-stupid approach will work well enough for most things, it will 
occasionally get the wrong answer.

But it will *notice* when it gets the wrong answer, though, and can reset 
and start over!

IOW, I might be able to do something that

 - prints out the commit info per line

 - prepends each line with a line number

 - goes back to an earlier line 'n' when it notices that it needs to 
   output a commit before a previous commit (or when it notices that a 
   commit that it had already output was actually not supposed to show up)

and with something like that, I could make git give you incremental 
output.

The thing is, any revision information that requires "global knowledge" 
simply cannot scale. And "git rev-list --topo-order" may be fast as hell, 
and I can do it in one second on the kernel archive on my machine, but 
that's really only true when it's all cached. 

If it's not cached, it will inevitably have to read in every single commit 
if you want a "final and unchanging ordering". Which inevitably gets you a 
really irritating startup latency. That's just fundamental.

On the other hand, if there is some way to say "oops, restart", I can 
optimistically give you a list that is always properly sorted on a *local* 
scale, but then based on later data I might notice that it wasn't right 
globally and that I need to re-do all or part of it.

But as mentioned, that requires that side-band data of "uhhuh, I screwed 
up, let me go back and fix it".

			Linus

Re: New features in gitk

[Marco Costalba]

On 11/2/07, Linus Torvalds <torvalds@linux-foundation.org> wrote:
>
> But it will *notice* when it gets the wrong answer, though, and can reset
> and start over!
>
> IOW, I might be able to do something that
>
>  - prints out the commit info per line
>
>  - prepends each line with a line number
>
>  - goes back to an earlier line 'n' when it notices that it needs to
>    output a commit before a previous commit (or when it notices that a
>    commit that it had already output was actually not supposed to show up)
>
> and with something like that, I could make git give you incremental
> output.
>

Yes. That's would be easier to implement. Better yet do not give line
numbers I already push back each revision sha in a vector according to
arrival order. It's a stack like storing.

So would be nice if 'git log --restarting' would work like this:

- Output the normal stream of commits according to git log arguments.
No line numbers, no fancy additional stuff.

- If '--topo-order' or something similar was given git log checks if a
wrong output occurs, as example because it founds a revisions that
should have been already put out say 'n' revisions before the last
outputted one.

- In the above case git log outputs a side-band data of "uhhuh, I screwed
up, I restart from 'n' revisions before the last one outputted".

- Then ouput _again_ the stream starting from 'n' revisions earlier.
Note that not only the new offending revision is trasmitted but *all
the revisions* from the out of order one to the remaining.

Given a vector of 'k' arrived revisions, for me it's far easier simply
to flush the 'n' tail items in the sha vector and restart again then
_insert_ in the vector the new out of order one.

This is because parsing alghoritm is based on an 'append new stuff'
approach, not 'insert in the middle', so better flush all the tail
also if probably the big part of retrasmitted revisions would remain
the same.


Marco

P.S: The out of bound information should be commit data aligned and
could take advantage of the fact that an sha always starts with an
alphanumeric char value [0..9 a..f]

IOW instead of the commit sha this signal could write something like

'Restarting  from -12'

and parsing knows that an sha cannot start with an 'R'. Please note
that 'instead of the commit sha' it means in the _exact_ place where
sha is expected and this is not predefined but depends on the git-log
arguments, so that as example

$git log --with-restart

would output:

commit 6959893b0b65ebc68ce2fb524a8ec15a26ca4972
Merge: 452b800... d279fc1...
Author: Junio C Hamano <gitster@pobox.com>
Date:   Wed Oct 31 23:53:55 2007 -0700

    Merge branch 'sp/mergetool'

    * sp/mergetool:
      mergetool: avoid misleading message "Resetting to default..."
      mergetool: add support for ECMerge
      mergetool: use path to mergetool in config var mergetool.<tool>.path

commit Restart from -7
commit 3e4bb087a18435b12eb82116e93af2887578e816
Merge: 5fb1948... 136e631...
Author: Junio C Hamano <gitster@pobox.com>
Date:   Thu Nov 1 17:09:08 2007 -0700

    Merge branch 'maint'



while

$git log --with-restart --pretty=oneline

would output

6959893b0b65ebc68ce2fb524a8ec15a26ca4972 Merge branch 'sp/mergetool'
Restart from -7
3e4bb087a18435b12eb82116e93af2887578e816 Merge branch 'maint'
5fb19486e6f4b6d31f33f5a1eab970b244fa2d08 Merge branch 'bk/maint-cvsexportcommit'


In this way this side band info became compatible with _any_ git-log
output format as long as the format foreseens the output of the
revision sha.

Re: New features in gitk

[Linus Torvalds]

On Fri, 2 Nov 2007, Linus Torvalds wrote:
> 
> The thing is, I'm pretty sure I can feed you commits really quickly if I 
> don't sort them, and if I don't do the full and careful "oops, this commit 
> was reachable from a commit that was marked uninteresting", but while the 
> fast-and-stupid approach will work well enough for most things, it will 
> occasionally get the wrong answer.

Ok, I have a trial patch that doesn't do the replay yet, but at least 
notices when it outputs a parent that has already been shown.

In the whole git archive, it happens three times.

In the kernel archive, it happens twelve times.

I'll try to get it into some kind of usable shape, and send out 
experimental patches for people to play with.

		Linus

Re: New features in gitk

[Johannes Schindelin]

Hi,

On Fri, 2 Nov 2007, Linus Torvalds wrote:

> On Fri, 2 Nov 2007, Marco Costalba wrote:
> > 
> > I have tried to overcome --topo-order in qgit but I found it very 
> > difficult, too much for me.
> > 
> > Lazily drawing the layout it doesn't mean that you lazy load the data 
> > from git, indeed you load all the git-log output as soon as it 
> > arrives.
> 
> Would it be more palatable if I tried to write some 
> visualization-specific front-end that acted kind of like "git rev-list", 
> but would have some way of "resetting" its output?

Heh, Shawn and I were discussing this when we met in San Jose earlier this 
month.  The application we had in mind was a common backend for graphical 
representation of the commit graph, which could be used by git gui to show 
(part of) the history.  The ultimate goal was a graphical rebase -i.

I would have _loved_ to implement this.  Alas, as it appears my choice of 
job was less than brilliant, and even when I have some spare moments at 
the end of the day, I watch movies to forget the day, instead of 
implementing this fascinating and useful feature.

Ciao,
Dscho