git tree browsing in redmine

git tree browsing in redmine

[Burt Culver]

Hi,

I'm trying to find some ideas for a better way to implement browsing
of a git repository within the redmine application.

My top level directory in my git repository has 300 files. Redmine
wants to display each file name, its most recent revision, and the
comment for that revision. Currently it does a git log -1 on each file
to find the latest revision.  Is there a quicker way of doing this for
a whole directory?  git log runs from .2 seconds to 3 seconds
depending on the file on my server.

Here is the open issue at redmine.org which has more details:
http://www.redmine.org/issues/show/1435

A fix for this would help a whole bunch of redmine / git users including myself.

-- 
Regards,

Burt Culver
Fishpond.co.nz & Fishpond.com.au
New Zealand & Australias Biggest Bookstore

Re: git tree browsing in redmine

[Andreas Ericsson]

Burt Culver wrote:
> Hi,
> 
> I'm trying to find some ideas for a better way to implement browsing
> of a git repository within the redmine application.
> 
> My top level directory in my git repository has 300 files. Redmine
> wants to display each file name, its most recent revision, and the
> comment for that revision. Currently it does a git log -1 on each file
> to find the latest revision.  Is there a quicker way of doing this for
> a whole directory?

Not yet, but if you clone libgit2 and start helping out on that, things
like this will become fairly trivial to do in a single pass instead of,
as redmine does it today, using one call to "git log" for every file in
the directory (which is just insane).

You should be aware that git is fully snapshot based though, while svn
is file-revision based (a bunch of file-revisions make up a "changeset"),
so it's fundamentally easy to get the file-revisions for a particular
changeset in svn and fundamentally hard to get it in git.

>  git log runs from .2 seconds to 3 seconds
> depending on the file on my server.
> 

That's because the command used by redmine ("git log <cruft> -1 -- $path")
stops as soon as it finds the first commit that touches the path. Some
paths haven't been touched in a long time, so more revisions have to
be loaded, parsed, analyzed and discarded before the correct one is
found.

> Here is the open issue at redmine.org which has more details:
> http://www.redmine.org/issues/show/1435
> 
> A fix for this would help a whole bunch of redmine / git users including myself.
> 

I have no idea what redmine is, but one solution for them would be to
run "git log --pretty=fuller -p $branch" and then parse the diffs to get
the latest commit that touched all files (in one go). This would be
slightly slower for small repos (where it shouldn't matter anyway), but
time should increase linearly with the number of commits in the repository.
It's almost certainly required to be able to turn this feature off, btw,
as it won't work at all for super-huge repositories (think kde or OO.org).

You should also make sure to have your repository properly packed. Run
"git gc" on it every once in a while.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: git tree browsing in redmine

[Jakub Narebski]

"Burt Culver" <burt@fishpond.co.nz> writes:

> I'm trying to find some ideas for a better way to implement browsing
> of a git repository within the redmine application.
> 
> My top level directory in my git repository has 300 files. Redmine
> wants to display each file name, its most recent revision, and the
> comment for that revision. Currently it does a git log -1 on each file
> to find the latest revision.  Is there a quicker way of doing this for
> a whole directory?  git log runs from .2 seconds to 3 seconds
> depending on the file on my server.
> 
> Here is the open issue at redmine.org which has more details:
> http://www.redmine.org/issues/show/1435
> 
> A fix for this would help a whole bunch of redmine / git users
> including myself.

First, while it might be common and cheap view (information) for
Subversion, it is not the case for Git.  Git is whole project snapshot
based, not per-file snapshot based, or patch based.  Take a look at
one of git web interfaces: gitweb, or cgit, or ViewGit to see how
repository browsing is implemented there: it is log-like view, then
tree, not tree, then history of files.  Note also that for example
history of two files is more than union of histories of individual
files (history simplification, including merge simplification).

Second, I tried to implement such view (which I named 'tree blame'
view, because it looks a bit like blame/annotate, but for
directories/trees) for gitweb.  You can view my attempts in
'gitweb/tree_blame' branch of my git fork:

  http://repo.or.cz/w/git/jnareb-git.git?a=shortlog;h=refs/heads/gitweb/tree_blame

But I think the correct solution, if you decide that you absolutely
need so unnatural (and I think not that useful) view, would be to
implement such view in git core, allowing 'git blame' to work for
trees (directories).  You would probably need to implement also some
kind of '--porcelain' output for script/Redmine.

Or as suggested add your work to libgit2 library, and make use of it
in Redmine...
-- 
Jakub Narebski
Poland
ShadeHawk on #git