git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* faster egit history page and a pure java "gitk"
@ 2008-03-24  9:27 Shawn O. Pearce
  2008-03-24 13:33 ` Roger C. Soares
                   ` (3 more replies)
  0 siblings, 4 replies; 16+ messages in thread
From: Shawn O. Pearce @ 2008-03-24  9:27 UTC (permalink / raw)
  To: Robin Rosenberg, Roger C. Soares; +Cc: git

OK, so I decided a few weeks back that the history page was not fast
enough.  I think I've spent the past 3 weeks writing true revision
machinary for jgit, and now connecting it up to a UI visualizer.

  git://repo.or.cz/egit/spearce.git plotter

The history page has been completely replaced.  I saw Roger has
some patches against the current history page.  :-|

There are huge benefits to this infrastructure:

 * Fast as snot.  We are literally just 10-20 ms slower than C Git
   on the same hardware, same repository, for the same tree.  That
   is pretty damn good, given that we are in Java.

 * Faster than gitk.  Yes, really.  My plot algorithm is not nearly
   as good as Paulus' work, but I think its better than what we
   have right now.

 * Nearly instant results without path limiter(s).  If the graph
   doesn't require parent rewrites we are able to show results
   almost immediately, and fill in the rest incrementally from
   the background job.

 * Lower memory usage.  By massive amounts.  I can't even begin to
   tell you how much different it is.  Histories that we could not
   show before can now be shown in <20M.  Our memory usage is much
   lower than that of gitk.

 * Multiple path limiters.  You can select more than one resource
   (or directory!) at once and get the combined history for
   all of them at once.  This is basically the same path limiter
   algorithm that C Git/gitk rely upon for the same sort of query.
   It is still limited to a single-repository, but I think we could
   easily extend it to allow multiple-repository unions.  :-)

 * Parent/child hyperlinks in message viewer.  Like gitk you can
   now click on the parent and child commit SHA-1s to jump up and
   down the graph.

 * Grep options available.  Not implemented in the UI yet, but
   the lower level machinary supports author/committer/log message
   grep functions.

 * Interesting/not-interesting flags available.  Not implemented
   in the UI yet, but the lower level machinary can do things like
   "^foo bar" to show all commits in bar that are not in foo.

 * More modular code.  Its broken apart much better. We don't have
   3000 lines in a single class.

 * Common AWT and SWT drawing.  Most of the UI visualization code
   is implemented in shared code that has no AWT or SWT specifics
   about it.  This makes the renderer completely portable.  I have
   both an AWT and an SWT implementation running (compare from the
   command line "jgit glog HEAD" to the history view in Eclipse).

 * Faster "RevCommit" class.  This class mostly replaces the older
   "Commit" class.  Accessing data out of a RevCommit can be much
   quicker than out of a Commit, plus a RevCommit gets the encoding
   of the author, committer and message correct more of the time.
   The downside is you can only get a RevCommit from a RevWalk,
   or one of its subclasses.

-- 
Shawn.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: faster egit history page and a pure java "gitk"
  2008-03-24  9:27 faster egit history page and a pure java "gitk" Shawn O. Pearce
@ 2008-03-24 13:33 ` Roger C. Soares
  2008-03-24 14:10   ` Robin Rosenberg
  2008-03-25  4:43   ` Shawn O. Pearce
  2008-03-24 14:06 ` Robin Rosenberg
                   ` (2 subsequent siblings)
  3 siblings, 2 replies; 16+ messages in thread
From: Roger C. Soares @ 2008-03-24 13:33 UTC (permalink / raw)
  To: Shawn O. Pearce, Robin Rosenberg; +Cc: git


Shawn O. Pearce escreveu:
> OK, so I decided a few weeks back that the history page was not fast
> enough.  I think I've spent the past 3 weeks writing true revision
> machinary for jgit, and now connecting it up to a UI visualizer.
>
>   git://repo.or.cz/egit/spearce.git plotter
>
> The history page has been completely replaced.  I saw Roger has
> some patches against the current history page.  :-|
>   
Hi Shawn. This is awesome, I can't wait to see this integrated in the 
main repo :)

I don't spend much time working on egit so I usually take a while to 
make small things, but I can certainly merge my patches on top of yours.

Robin, how should I proceed, resend all my patches from the weekend on 
top of Shawn's tree?

[]s,
Roger.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: faster egit history page and a pure java "gitk"
  2008-03-24  9:27 faster egit history page and a pure java "gitk" Shawn O. Pearce
  2008-03-24 13:33 ` Roger C. Soares
@ 2008-03-24 14:06 ` Robin Rosenberg
  2008-03-25  5:10   ` Shawn O. Pearce
  2008-03-24 14:31 ` faster egit history page and a pure java "gitk" so Robin Rosenberg
  2008-03-25  5:07 ` faster egit history page and a pure java "gitk" Roger C. Soares
  3 siblings, 1 reply; 16+ messages in thread
From: Robin Rosenberg @ 2008-03-24 14:06 UTC (permalink / raw)
  To: Shawn O. Pearce; +Cc: Roger C. Soares, git

Den Monday 24 March 2008 09.27.26 skrev Shawn O. Pearce:
> OK, so I decided a few weeks back that the history page was not fast
> enough.  I think I've spent the past 3 weeks writing true revision
> machinary for jgit, and now connecting it up to a UI visualizer.
>
>   git://repo.or.cz/egit/spearce.git plotter
>
> The history page has been completely replaced.  I saw Roger has
> some patches against the current history page.  :-|
The page was very messy. It was my first attempt at anything ui in
Eclipse at all. Like a child drawing his first picture of a person :)

> There are huge benefits to this infrastructure:
>
>  * Fast as snot.  We are literally just 10-20 ms slower than C Git
>    on the same hardware, same repository, for the same tree.  That
>    is pretty damn good, given that we are in Java.

On what repo is that measured?

As for Java speed, it is some two to three times slower than C on array 
intensive stuff. On just about anything else the difference is less.

The "Micro-optimize pack index v2 findOffset routine" commit suprises me
a little. The rearranged ObjectId layout does not. Could we do even better
using two long's and one int?

>  * Faster than gitk.  Yes, really.  My plot algorithm is not nearly
>    as good as Paulus' work, but I think its better than what we
>    have right now.
gitk has to talk to externa processess and parse things so I'm not surprised
we'd come to it some day.

>  * Nearly instant results without path limiter(s).  If the graph
>    doesn't require parent rewrites we are able to show results
>    almost immediately, and fill in the rest incrementally from
>    the background job.
>
>  * Lower memory usage.  By massive amounts.  I can't even begin to
>    tell you how much different it is.  Histories that we could not
>    show before can now be shown in <20M.  Our memory usage is much
>    lower than that of gitk.
This probably is related to speed too because the gc got to do a lot
of work.

>  * Multiple path limiters.  You can select more than one resource
>    (or directory!) at once and get the combined history for
>    all of them at once.  This is basically the same path limiter
>    algorithm that C Git/gitk rely upon for the same sort of query.
>    It is still limited to a single-repository, but I think we could
>    easily extend it to allow multiple-repository unions.  :-)
You mean submodules, real and "virtual" ?

>  * Common AWT and SWT drawing.  Most of the UI visualization code
>    is implemented in shared code that has no AWT or SWT specifics
>    about it.  This makes the renderer completely portable.  I have
>    both an AWT and an SWT implementation running (compare from the
>    command line "jgit glog HEAD" to the history view in Eclipse).
Sweet. Needs some polishing though. At least under Linux and small screens
with lots of pixels.

>  * Faster "RevCommit" class.  This class mostly replaces the older
>    "Commit" class.  Accessing data out of a RevCommit can be much
>    quicker than out of a Commit, plus a RevCommit gets the encoding
>    of the author, committer and message correct more of the time.
>    The downside is you can only get a RevCommit from a RevWalk,
>    or one of its subclasses.

Very impressive work Mr Shawn. I'll walk through and publish.

-- robin

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: faster egit history page and a pure java "gitk"
  2008-03-24 13:33 ` Roger C. Soares
@ 2008-03-24 14:10   ` Robin Rosenberg
  2008-03-25  4:43   ` Shawn O. Pearce
  1 sibling, 0 replies; 16+ messages in thread
From: Robin Rosenberg @ 2008-03-24 14:10 UTC (permalink / raw)
  To: Roger C. Soares; +Cc: Shawn O. Pearce, git

Den Monday 24 March 2008 13.33.23 skrev Roger C. Soares:
> Robin, how should I proceed, resend all my patches from the weekend on
> top of Shawn's tree?

Please, do.

-- robin

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: faster egit history page and a pure java "gitk" so
  2008-03-24  9:27 faster egit history page and a pure java "gitk" Shawn O. Pearce
  2008-03-24 13:33 ` Roger C. Soares
  2008-03-24 14:06 ` Robin Rosenberg
@ 2008-03-24 14:31 ` Robin Rosenberg
  2008-03-25  4:48   ` Shawn O. Pearce
  2008-03-25  5:07 ` faster egit history page and a pure java "gitk" Roger C. Soares
  3 siblings, 1 reply; 16+ messages in thread
From: Robin Rosenberg @ 2008-03-24 14:31 UTC (permalink / raw)
  To: Shawn O. Pearce; +Cc: Roger C. Soares, git

Måndagen 24 March 2008 09.27.26 skrev Shawn O. Pearce:
> OK, so I decided a few weeks back that the history page was not fast
> enough.  I think I've spent the past 3 weeks writing true revision
> machinary for jgit, and now connecting it up to a UI visualizer.
>
>   git://repo.or.cz/egit/spearce.git plotter
>
> The history page has been completely replaced.  I saw Roger has
> some patches against the current history page.  :-|

Did you lose compare?

Ok, found it, but it only shows one file in the compare editor, so I cannot 
walk through all changes in the commit with Ctrl-. and Ctrl-,

-- robin

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: faster egit history page and a pure java "gitk"
  2008-03-24 13:33 ` Roger C. Soares
  2008-03-24 14:10   ` Robin Rosenberg
@ 2008-03-25  4:43   ` Shawn O. Pearce
  2008-03-25 12:33     ` Roger C. Soares
  1 sibling, 1 reply; 16+ messages in thread
From: Shawn O. Pearce @ 2008-03-25  4:43 UTC (permalink / raw)
  To: Roger C. Soares; +Cc: Robin Rosenberg, git

"Roger C. Soares" <rogersoares@intelinet.com.br> wrote:
> Shawn O. Pearce escreveu:
> >OK, so I decided a few weeks back that the history page was not fast
> >enough.  I think I've spent the past 3 weeks writing true revision
> >machinary for jgit, and now connecting it up to a UI visualizer.
> >
> >  git://repo.or.cz/egit/spearce.git plotter
> >
> >The history page has been completely replaced.  I saw Roger has
> >some patches against the current history page.  :-|
> 
> Hi Shawn. This is awesome, I can't wait to see this integrated in the 
> main repo :)
> 
> I don't spend much time working on egit so I usually take a while to 
> make small things, but I can certainly merge my patches on top of yours.
> 
> Robin, how should I proceed, resend all my patches from the weekend on 
> top of Shawn's tree?

I just pushed a newer version out that contains support for showing
commits using a bold font if the commit has the highlightFlag set
on it.  I assume you were trying to work on a feature like gitk
has where you can type in a string and have gitk bold all of the
commits that contain that string in the message, right?

If you look at the newer internal.history.GitHistoryPage class there
is a historyFlag available as an instance member.  We also have an
SWTCommitList allocated in the inputSet() method.  The SWTCommitList
has an applyFlag method that accepts a RevFilter and adds the passed
RevFlag onto any commit that matches the filter.

There are a lot of RevFilter implementations, take a look at the
revwalk.filter package, or see the command line parsing code in
RevWalkTextBuiltin.  We have ones for author, committer, message...

So a good part of the code is there to do a bold/unbold thing.
There's also indexOf and lastIndexOf methods on SWTCommitList that
can search for the highlightFlag in either direction from a given
row index, making a prev/next feature pretty easy.

I decided to leave some low hanging fruit.  There's bigger and
more complex issues still lurking in the revwalk/treewalk features.

-- 
Shawn.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: faster egit history page and a pure java "gitk" so
  2008-03-24 14:31 ` faster egit history page and a pure java "gitk" so Robin Rosenberg
@ 2008-03-25  4:48   ` Shawn O. Pearce
  0 siblings, 0 replies; 16+ messages in thread
From: Shawn O. Pearce @ 2008-03-25  4:48 UTC (permalink / raw)
  To: Robin Rosenberg; +Cc: Roger C. Soares, git

Robin Rosenberg <robin.rosenberg@dewire.com> wrote:
> Måndagen 24 March 2008 09.27.26 skrev Shawn O. Pearce:
> > OK, so I decided a few weeks back that the history page was not fast
> > enough.  I think I've spent the past 3 weeks writing true revision
> > machinary for jgit, and now connecting it up to a UI visualizer.
> >
> >   git://repo.or.cz/egit/spearce.git plotter
> >
> > The history page has been completely replaced.  I saw Roger has
> > some patches against the current history page.  :-|
> 
> Did you lose compare?
> 
> Ok, found it, but it only shows one file in the compare editor, so I cannot 
> walk through all changes in the commit with Ctrl-. and Ctrl-,

Yea.  The compare linkage in the prior history view was a little
unclear and I was trying to put something together.  So I went
with a really quick and dirty linkage that worked for the simple
single file case.  Ideally it should get fully ported over, but
the underlying concepts like GitFileRevision are probably going
to go by the wayside with RevWalk and TreeWalk being available.
So its probably trivial to port.

-- 
Shawn.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: faster egit history page and a pure java "gitk"
  2008-03-24  9:27 faster egit history page and a pure java "gitk" Shawn O. Pearce
                   ` (2 preceding siblings ...)
  2008-03-24 14:31 ` faster egit history page and a pure java "gitk" so Robin Rosenberg
@ 2008-03-25  5:07 ` Roger C. Soares
  2008-03-25  5:36   ` Shawn O. Pearce
  3 siblings, 1 reply; 16+ messages in thread
From: Roger C. Soares @ 2008-03-25  5:07 UTC (permalink / raw)
  To: Shawn O. Pearce; +Cc: Robin Rosenberg, git

Shawn O. Pearce escreveu:
> OK, so I decided a few weeks back that the history page was not fast
> enough.  I think I've spent the past 3 weeks writing true revision
> machinary for jgit, and now connecting it up to a UI visualizer.
>
>   git://repo.or.cz/egit/spearce.git plotter
>
> The history page has been completely replaced.  I saw Roger has
> some patches against the current history page.  :-|
>   
Hi Guys,

I saw it running now, here are my initial comments/impressions:

I have a git.git clone on my eclipse and now egit can open it, cool! :) 
But it wasn't that fast, it took some minutes to finish building the 
whole tree. Also, changing projects (different git repos) makes the cpu 
go very high, and what opened fast the first time takes minutes after...

When reading the email I thought the new history page would have the 
same features from the current, but it doesn't. It looks promissing thought.

My first impression is that I like the current way of showing files in 
the strutucture compare better. In the future I would like the option to 
make it open on the left pane (where package explorer is) and have the 
options to collapse/expand directories, present in flat/hierarchical 
modes, have the fast view, etc.

Comments now can't be automatically wrapped. I actually prefer to leave 
wrapping to the computer, I don't like having to manually fix line 
lenghts when amending a comment, I think eclipse should do it. Also, 
currently, the history page usually has little space allocated for it 
and currently the first lines of the comment are visible. In the new 
page I have to scroll to see the comment when the history page is not 
maximized.

[]s,
Roger.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: faster egit history page and a pure java "gitk"
  2008-03-24 14:06 ` Robin Rosenberg
@ 2008-03-25  5:10   ` Shawn O. Pearce
  0 siblings, 0 replies; 16+ messages in thread
From: Shawn O. Pearce @ 2008-03-25  5:10 UTC (permalink / raw)
  To: Robin Rosenberg; +Cc: Roger C. Soares, git

Robin Rosenberg <robin.rosenberg@dewire.com> wrote:
> Den Monday 24 March 2008 09.27.26 skrev Shawn O. Pearce:
> >   git://repo.or.cz/egit/spearce.git plotter
> >
> > The history page has been completely replaced.
>
> The page was very messy. It was my first attempt at anything ui in
> Eclipse at all. Like a child drawing his first picture of a person :)

It was pretty good.  I actually learned from your work before
replacing it.  My motivation wasn't about the look, but about
the speed of it.  :)
 
> > There are huge benefits to this infrastructure:
> >
> >  * Fast as snot.  We are literally just 10-20 ms slower than C Git
> >    on the same hardware, same repository, for the same tree.  That
> >    is pretty damn good, given that we are in Java.
> 
> On what repo is that measured?

git.git, starting from 0e545f75169e2c260dfb4445203c23cafdfc76ef, which
is dated Dec 22 2007, so its a bit dated, but not too far back to be
completely useless.

git.git is a toy compared to other projects, yes, but its what I
had on hand in my egit work area.  I haven't tried larger projects
(like say linux-2.6 or OOo).

> As for Java speed, it is some two to three times slower than C on array 
> intensive stuff. On just about anything else the difference is less.

During this work I've found that even with Java 6's HotSpot the way
you write the code in Java affects how the JIT optimizes it.  In some
cases hand-unrolling loops (like say a 20 byte SHA-1 compare routine)
makes a big impact.  Using "p + 1", "p + 2", "p + 3" to access into the
array can be faster than "p++", "p++', "p++".
 
> The "Micro-optimize pack index v2 findOffset routine" commit suprises me
> a little.

Me too.  I assumed the compiler+JIT would be able to figure out that
"a * 5" can be rewritten as "a << 2 + a", but alas they didn't.  The
rewriting I did there was part of the boost I got.

> The rearranged ObjectId layout does not. Could we do even better
> using two long's and one int?

I actually started with the two long and one int layout but it was
quite a bit slower than the five int case.  Keep in mind this was
on a 32 bit Windows system; on an x86-64 or SPARC the two longs may
be faster due to the 64 bit registers being available to the JIT.
 
> >  * Lower memory usage.  By massive amounts.  I can't even begin to
> >    tell you how much different it is.  Histories that we could not
> >    show before can now be shown in <20M.  Our memory usage is much
> >    lower than that of gitk.
>
> This probably is related to speed too because the gc got to do a lot
> of work.

Yup.  The massive memory usage of the prior history view is actually
what drove me to do this work.  I was watching the memory meter on
the workbench while trying to open the history of my git.git clone
(HEAD is just pegged on the commit above) and realized something was
really wrong if we were spiking up through 150M and onward just to
pull up the history.  It also took several minutes.  :-|
 
> >  * Multiple path limiters.  You can select more than one resource
> >    (or directory!) at once and get the combined history for
> >    all of them at once.  This is basically the same path limiter
> >    algorithm that C Git/gitk rely upon for the same sort of query.
> >    It is still limited to a single-repository, but I think we could
> >    easily extend it to allow multiple-repository unions.  :-)
>
> You mean submodules, real and "virtual" ?

Yes.  Or if projects are in different repositories and the user
selects files in both and asks for history we could (in theory) show
a combined graph showing the commits from both repositories.  If they
have no sort of connected history you'll still get interleaved
timestamps, but there will be at least two lanes running down the
entire graph.

Most of the abstractions are there.  It shouldn't be difficult to
change RevWalk and TreeWalk to take some sort of new interface
instead of the Repository, where that interface can delegate
to either one Repository, or search through a set of packfiles
from multiple repositories, and then search through their loose
objects.  That's really all that should be needed to union together
repositories.
 
> >  * Common AWT and SWT drawing.  Most of the UI visualization code
> >    is implemented in shared code that has no AWT or SWT specifics
> >    about it.  This makes the renderer completely portable.  I have
> >    both an AWT and an SWT implementation running (compare from the
> >    command line "jgit glog HEAD" to the history view in Eclipse).
>
> Sweet. Needs some polishing though. At least under Linux and small screens
> with lots of pixels.

Yea, I agree.  I tried drawing it with 3 pixel wide rectangles
today but it looked like c**p^H^H^H^Hnot the best thing I have
ever written.  But it is more readable, so we'll have to come back
to that and see if we can't do better.
 
-- 
Shawn.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: faster egit history page and a pure java "gitk"
  2008-03-25  5:07 ` faster egit history page and a pure java "gitk" Roger C. Soares
@ 2008-03-25  5:36   ` Shawn O. Pearce
  2008-03-25  8:09     ` Shawn O. Pearce
                       ` (2 more replies)
  0 siblings, 3 replies; 16+ messages in thread
From: Shawn O. Pearce @ 2008-03-25  5:36 UTC (permalink / raw)
  To: Roger C. Soares; +Cc: Robin Rosenberg, git

"Roger C. Soares" <rogersoares@intelinet.com.br> wrote:
> Shawn O. Pearce escreveu:
> >The history page has been completely replaced.
> 
> I have a git.git clone on my eclipse and now egit can open it, cool! :) 

Yea, that was my impression too.  "Yay, it can open git.git!"  :)

> But it wasn't that fast, it took some minutes to finish building the 
> whole tree. Also, changing projects (different git repos) makes the cpu 
> go very high, and what opened fast the first time takes minutes after...

Hmm.  How long does C Git take for "git rev-list HEAD >/dev/null" ?
I have thus far only tuned the lower level machinary, and there
may still be tuning left there, but I _really_ have not tried to
tune the plotting portion yet.

I did push something out a few minutes ago (b66eae Limit the number
of UI refreshes ...) that may help improve performance on larger
histories.

Another thing is how many pack files/loose objects do you have?
The loose objects are harder to access, and jgit is currently
lacking some of the pack search tricks that C Git uses to get
good performance.  As such all of my testing has been working on
a fully packed repository that has exactly one packfile in it,
with no alternates.

Its _almost_ acceptable to me.  We can still do better, but there's
a point at which we just can't get past.  We probably can't beat
rev-list, but I think we should be able to beat gitk almost always.
In the simple fully packed case "jgit glog" beats gitk at opening
the full graph and displaying it.

> When reading the email I thought the new history page would have the 
> same features from the current, but it doesn't. It looks promissing thought.

Nope.  Sorry.  Its basically a rewrite.
 
> My first impression is that I like the current way of showing files in 
> the strutucture compare better.

I actually prefer the individual files in the right pane, like gitk
does, as I like to browse up and down sometimes looking at changes.
Maybe we can make it an option to show that right side panel,
and give a button (or IOpenListener on the graph table?) to let
you open the whole commit in the structure compare view.

> In the future I would like the option to 
> make it open on the left pane (where package explorer is) and have the 
> options to collapse/expand directories, present in flat/hierarchical 
> modes, have the fast view, etc.

Oooh.  Sounds fun.
 
> Comments now can't be automatically wrapped.

Oversight/planned loss of feature.  I'm a strong believer of showing
the commit message *exactly* as recorded, which means don't do
line wrapping of it.  Things like character encoding translation
and indenting the left side 2-4 spaces to keep it unambiguous from
headers is fine when showing it to a human, but otherwise it should
match what the user wrote.

I forgot to offer a wrap option.  If we do enable line wrapping I
think we should give the user a way to toggle it on/off for the
message area viewer so that if line wrapping is enabled and its
borking the current message (e.g. a nice pretty ASCII diagram)
you can disable it.

> I actually prefer to leave 
> wrapping to the computer, I don't like having to manually fix line 
> lenghts when amending a comment, I think eclipse should do it. Also, 
> currently, the history page usually has little space allocated for it 
> and currently the first lines of the comment are visible. In the new 
> page I have to scroll to see the comment when the history page is not 
> maximized.

Yea, I started to notice that myself today.  The area isn't quite
wide enough, and I was using Eclipse on a very high resolution CRT
and it was taking up a good 80% of the display.  Most users can't
sanely devote the area I did to the History view, and it was still
not wide enough for some git.git messages.

Thanks for the feedback.  Obviously its still got a lot of areas
for improvements.

-- 
Shawn.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: faster egit history page and a pure java "gitk"
  2008-03-25  5:36   ` Shawn O. Pearce
@ 2008-03-25  8:09     ` Shawn O. Pearce
  2008-03-25 13:46     ` Roger C. Soares
  2008-03-26  1:37     ` Roger C. Soares
  2 siblings, 0 replies; 16+ messages in thread
From: Shawn O. Pearce @ 2008-03-25  8:09 UTC (permalink / raw)
  To: Roger C. Soares; +Cc: Robin Rosenberg, git

"Shawn O. Pearce" <spearce@spearce.org> wrote:
> "Roger C. Soares" <rogersoares@intelinet.com.br> wrote:
> 
> > But it wasn't that fast, it took some minutes to finish building the 
> > whole tree. Also, changing projects (different git repos) makes the cpu 
> > go very high, and what opened fast the first time takes minutes after...

Something else I noticed - the core.packedGit* settings make a
difference on performance.  On Windows XP with Java 6 I am getting
much better performance (200 ms lower running time) by using a much
smaller window size and disabling mmap:

	[core]
		packedGitWindowSize = 8k
		packedGitLimit = 10m
		packedGitMMAP = false

by default jgit is using mmap, as Robin has reported it runs faster
that way for him.  But I have never been able to reproduce that on
Mac OS 10.4/Java 5 or Windows XP/Java 6.  In both systems setting
mmap to false has performed better, even on the initial first set
of hits to the cache where we have to read in the blocks vs. mmap
them.  The ByteBuffer API is just that much slower than accessing
a byte[] directly when shoving it through inflate.  We spend at
least 30% of our time in inflate.

-- 
Shawn.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: faster egit history page and a pure java "gitk"
  2008-03-25  4:43   ` Shawn O. Pearce
@ 2008-03-25 12:33     ` Roger C. Soares
  0 siblings, 0 replies; 16+ messages in thread
From: Roger C. Soares @ 2008-03-25 12:33 UTC (permalink / raw)
  To: Shawn O. Pearce; +Cc: Robin Rosenberg, git

Shawn O. Pearce escreveu:
> I just pushed a newer version out that contains support for showing
> commits using a bold font if the commit has the highlightFlag set
> on it.  I assume you were trying to work on a feature like gitk
> has where you can type in a string and have gitk bold all of the
> commits that contain that string in the message, right?
>   
Yep, the find toolbar highlights the commits in bold. It's probably the 
first thing that I'll try to port. Thanks for the info.

[]s,
Roger.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: faster egit history page and a pure java "gitk"
  2008-03-25  5:36   ` Shawn O. Pearce
  2008-03-25  8:09     ` Shawn O. Pearce
@ 2008-03-25 13:46     ` Roger C. Soares
  2008-03-25 19:48       ` Robin Rosenberg
  2008-03-26  1:37     ` Roger C. Soares
  2 siblings, 1 reply; 16+ messages in thread
From: Roger C. Soares @ 2008-03-25 13:46 UTC (permalink / raw)
  To: Shawn O. Pearce; +Cc: Robin Rosenberg, git


Shawn O. Pearce escreveu:
> Hmm.  How long does C Git take for "git rev-list HEAD >/dev/null" ?
> I have thus far only tuned the lower level machinary, and there
> may still be tuning left there, but I _really_ have not tried to
> tune the plotting portion yet.
>   
I'll do that when I get back home, but I think it should be fast. At 
least gitk was showing the repo fast enough, from calling it from the 
command line and gitk stoping visible activity, I'd say around 2 or 3 
seconds.

Maybe my problem was with the plotting part. I was running on linux.

> Another thing is how many pack files/loose objects do you have?
> The loose objects are harder to access, and jgit is currently
> lacking some of the pack search tricks that C Git uses to get
> good performance.  As such all of my testing has been working on
> a fully packed repository that has exactly one packfile in it,
> with no alternates.
>   
I made a clone of it and never changed it. Don't recall making fetches 
either, so it should be in good shape. I can confirm later.

> Oversight/planned loss of feature.  I'm a strong believer of showing
> the commit message *exactly* as recorded, which means don't do
> line wrapping of it.  Things like character encoding translation
> and indenting the left side 2-4 spaces to keep it unambiguous from
> headers is fine when showing it to a human, but otherwise it should
> match what the user wrote.
>
> I forgot to offer a wrap option.  If we do enable line wrapping I
> think we should give the user a way to toggle it on/off for the
> message area viewer so that if line wrapping is enabled and its
> borking the current message (e.g. a nice pretty ASCII diagram)
> you can disable it.
>   
I understand that you guys use a lot of ASCII art and wrapping can mess 
this. But here we track more things in bugzilla and there's some 
copy&pasting going on, so wrapping makes comments more readable. 
Currently it's a toogle preference in the local toolbar menu (like the 
CVS plugin).

I left the comment on the right side because it's easy to set/unset 
wrapping for the whole viewer, and also for consistency with the CVS/SVN 
plugins, I still use them :)

My last patches also added the changed files in the left pane as text. 
The next step would be to add links. Before doing this I thought about 
adding a table there (like what you did) but I chose text with links 
because of copy&paste, I find it convenient to paste selected commit 
info into IM or email.

[]s,
Roger.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: faster egit history page and a pure java "gitk"
  2008-03-25 13:46     ` Roger C. Soares
@ 2008-03-25 19:48       ` Robin Rosenberg
  0 siblings, 0 replies; 16+ messages in thread
From: Robin Rosenberg @ 2008-03-25 19:48 UTC (permalink / raw)
  To: Roger C. Soares; +Cc: Shawn O. Pearce, git

Den Tuesday 25 March 2008 14.46.46 skrev Roger C. Soares:
> I understand that you guys use a lot of ASCII art and wrapping can mess
> this. But here we track more things in bugzilla and there's some
> copy&pasting going on, so wrapping makes comments more readable.
> Currently it's a toogle preference in the local toolbar menu (like the
> CVS plugin).

It seems to me ASCII art should be auto-detectable... Maybe enough to 
make the wrap flag unneeded in most cases. (wrap/nowrap/auto). At first
I was thinking about looking for special chars, but then it may be simpler
and just as useful to look for comment lines start with spaces ane a few
special characters.

Try this command: git log | sed 's/^    [\t |\\\/>]/|    /' | less. It seems 
to detect lines that I do not want wrapped pretty well.

-- robin

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: faster egit history page and a pure java "gitk"
  2008-03-25  5:36   ` Shawn O. Pearce
  2008-03-25  8:09     ` Shawn O. Pearce
  2008-03-25 13:46     ` Roger C. Soares
@ 2008-03-26  1:37     ` Roger C. Soares
  2008-03-26  4:52       ` Shawn O. Pearce
  2 siblings, 1 reply; 16+ messages in thread
From: Roger C. Soares @ 2008-03-26  1:37 UTC (permalink / raw)
  To: Shawn O. Pearce; +Cc: Robin Rosenberg, git

Shawn O. Pearce escreveu:
> Hmm.  How long does C Git take for "git rev-list HEAD >/dev/null" ?
> I have thus far only tuned the lower level machinary, and there
> may still be tuning left there, but I _really_ have not tried to
> tune the plotting portion yet.
>
> I did push something out a few minutes ago (b66eae Limit the number
> of UI refreshes ...) that may help improve performance on larger
> histories.
>   
"git rev-list HEAD >/dev/null" returns very fast, around 1 sec I'd say. 
My git clone has 0 loose objects and 1 pack.

I updated from your repo some minutes ago and it's pretty decent now. 
The history appears very fast, even changing projects, and for the git 
clone the progress bar disapears in around 7 seconds. :)

[]s,
Roger.

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: faster egit history page and a pure java "gitk"
  2008-03-26  1:37     ` Roger C. Soares
@ 2008-03-26  4:52       ` Shawn O. Pearce
  0 siblings, 0 replies; 16+ messages in thread
From: Shawn O. Pearce @ 2008-03-26  4:52 UTC (permalink / raw)
  To: Roger C. Soares; +Cc: Robin Rosenberg, git

"Roger C. Soares" <rogersoares@intelinet.com.br> wrote:
> Shawn O. Pearce escreveu:
> >Hmm.  How long does C Git take for "git rev-list HEAD >/dev/null" ?
> >I have thus far only tuned the lower level machinary, and there
> >may still be tuning left there, but I _really_ have not tried to
> >tune the plotting portion yet.
> >
> >I did push something out a few minutes ago (b66eae Limit the number
> >of UI refreshes ...) that may help improve performance on larger
> >histories.
>  
> "git rev-list HEAD >/dev/null" returns very fast, around 1 sec I'd say. 
> My git clone has 0 loose objects and 1 pack.

OK, so its well packed and C Git behaves nicely.  :)

> I updated from your repo some minutes ago and it's pretty decent now. 
> The history appears very fast, even changing projects, and for the git 
> clone the progress bar disapears in around 7 seconds. :)

So it must have been the massive flurry of UI updates that I used to
be doing during revision walking.  I backed it off to at most 4 times
per second, which seems to help.

FWIW I just pushed another update out:

 * re-activates the old preferences for hiding/showing the commit
   message and file list;

 * word wrap setting for the comment viewer area;

 * saves the geometry (split pane positions) of the history page
   in a hidden preference;

 * copy and select all global actions (Edit->Copy aka Ctrl-C) now
   works to copy:
     - selected text in comment area;
	 - selected path names in the file list;
	 - commit SHA-1s of selected commits in DAG;

 * the window cache is now managed by Eclipse workspace settings
   when inside Eclipse;

 * the window cache now defaults to 8k/10m/10m/no-mmap as that is
   working very well for me on multiple systems;

 * global workspace preferences (Team -> Git) now shows the history
   preferences and the window cache settings

-- 
Shawn.

^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2008-03-26  4:53 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-03-24  9:27 faster egit history page and a pure java "gitk" Shawn O. Pearce
2008-03-24 13:33 ` Roger C. Soares
2008-03-24 14:10   ` Robin Rosenberg
2008-03-25  4:43   ` Shawn O. Pearce
2008-03-25 12:33     ` Roger C. Soares
2008-03-24 14:06 ` Robin Rosenberg
2008-03-25  5:10   ` Shawn O. Pearce
2008-03-24 14:31 ` faster egit history page and a pure java "gitk" so Robin Rosenberg
2008-03-25  4:48   ` Shawn O. Pearce
2008-03-25  5:07 ` faster egit history page and a pure java "gitk" Roger C. Soares
2008-03-25  5:36   ` Shawn O. Pearce
2008-03-25  8:09     ` Shawn O. Pearce
2008-03-25 13:46     ` Roger C. Soares
2008-03-25 19:48       ` Robin Rosenberg
2008-03-26  1:37     ` Roger C. Soares
2008-03-26  4:52       ` Shawn O. Pearce

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).