RFC: Very useful script to SVG graph the git commits from a file orientated view

RFC: Very useful script to SVG graph the git commits from a file orientated view

[John Tapsell]

Hi,
  I made this script to help me see the logical connections between
commits.  It produces a .svg graph showing the commits that affected a
file.

For example, say you have the commits:

commit1 - modify hello.c
commit2 - modify goodbye.c
commit3 - modify hello.c and goodbye.c

It will draw a graph showing the first two commits as siblings, and
commit3 as a child of commit1 and commit2.

I have found this very useful when squashing and rebasing development
branches that have got a lot of "fix typo" and "fix"  type commit
messages.  From the graph you can quickly see which commit they were
fixing (the parent, in the graph).

Here is an example output, running it on kwin for the last 100 commits:

$ graph_git.pl --nofiles -100

http://imagebin.org/252754

And again with files for the last 10 commits:

$ graph_git.pl -10

http://imagebin.org/252756

(Note that it has tooltips)

JohnFlux

Re: RFC: Very useful script to SVG graph the git commits from a file orientated view

[Jeremy Rosen]

very usefull indeed, where can I find it ? I have a big rebase/merge/reorganise work that is comming soon and that is going to be tremendously usefull...

    Cordialement

    Jérémy Rosen

fight key loggers : write some perl using vim

----- Mail original -----
> Hi,
>   I made this script to help me see the logical connections between
> commits.  It produces a .svg graph showing the commits that affected
> a
> file.
> 
> For example, say you have the commits:
> 
> commit1 - modify hello.c
> commit2 - modify goodbye.c
> commit3 - modify hello.c and goodbye.c
> 
> It will draw a graph showing the first two commits as siblings, and
> commit3 as a child of commit1 and commit2.
> 
> I have found this very useful when squashing and rebasing development
> branches that have got a lot of "fix typo" and "fix"  type commit
> messages.  From the graph you can quickly see which commit they were
> fixing (the parent, in the graph).
> 
> Here is an example output, running it on kwin for the last 100
> commits:
> 
> $ graph_git.pl --nofiles -100
> 
> http://imagebin.org/252754
> 
> And again with files for the last 10 commits:
> 
> $ graph_git.pl -10
> 
> http://imagebin.org/252756
> 
> (Note that it has tooltips)
> 
> JohnFlux
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: RFC: Very useful script to SVG graph the git commits from a file orientated view

[John Tapsell]

[-- Attachment #1: Type: text/plain, Size: 65 bytes --]

Opps, somehow I forgot to actually attach it.

It's now attached

[-- Attachment #2: graph_git.pl --]
[-- Type: application/octet-stream, Size: 3851 bytes --]

#!/usr/bin/perl

use strict;
use warnings;
eval "require Git::Repository; require IPC::Run; 1" or die "Please do:  sudo apt-get install libgit-repository-perl graphviz libipc-run-perl    - Missing libraries";
use Git::Repository;
use IPC::Run qw( run );
my $svg_filename = "plot.svg";

if (!defined($ARGV[0]) ||  $ARGV[0] eq "-h" ||  $ARGV[0] eq "--help") {
	print <<"EOT";
JohnFlux's nifty git graphing tool
==================================

This program produces a $svg_filename file in the current folder, for the
commits in the given git range in the current git repository.

In the SVG, a commit's parents are the commits that last touched the same files
that the commit touched.  This graph is particularly useful when trying to squash.

Multiple lines between the same commits means that those commits both modified the same
multiple set of files.

Run as $0 [-h|--help]
       $0 [--stdout] [--nofiles] <git log options>.

Examples:

  #show the last 100 commits:
  $0 -100 && firefox plot.svg

  #show all the commits back to branch foo:
  $0 HEAD...foo && firefox plot.svg

  #show all the commits back to branch foo and don't show the files in seperate boxes:
  $0 --nofiles HEAD...foo && firefox plot.svg

EOT
	exit;
}

my $onlyPrintResult = 0;
my $noFiles = 0;

if ($ARGV[0] eq "--stdout") {
	shift;
	$onlyPrintResult = 1;
}
if ($ARGV[0] eq "--nofiles") {
	shift;
	$noFiles = 1;
}
my $git  = Git::Repository->new( work_tree => '.' );
my @lines = $git->run( log=> '--name-only', '--pretty=format:%h %s', @ARGV);
	
my %filesToSha = ();
my %filesToColorIndex = ();
my $nextColorIndex = 1; 
my $sha = "";
my $title = "";
my $nextLineIsSha = 1;
my $output ="";
$output .= 'digraph "git" {' . "\n";
for my $line ( @lines )  {
	#Empty line moves us on to the next commit
	if ($line =~ /^\s*$/) {
		$nextLineIsSha = 1;
	} elsif ($nextLineIsSha) {
		$nextLineIsSha = 0;
		($sha, $title) = split ' ', $line, 2;
		my $fullinfo = $git->run( show => '--stat', $sha );
		$title =~ s/"/'/g;
		$fullinfo =~ s/"/'/g;
		$fullinfo =~ s/\n/&#10;/g;
		$fullinfo =~ s/  +/ /g; #spaces eat up a lot space since they get encoded to &nbsp; = 6 letters

		# Firefox can't cope with it being longer than about 3000 characters
		# but character encodings mean that a single character can expand to many letters.  So we just have to guess at
		# a maximum length here
		$fullinfo = substr($fullinfo, 0, 1000);

		$output .= "\"$sha\" [label=\"$sha $title\", tooltip=\"$fullinfo\"]\n";
	} else {
		#  $line is a filename that the current $sha commit modified 

		# See http://www.graphviz.org/doc/info/colors.html for a list of color schemes
		my $colorIndex;
		if( defined( my $parentSha = $filesToSha{ $line } ) ) {
			$colorIndex = $filesToColorIndex{ $line };
			$output .= "\"$parentSha\" -> { \"$sha\" } ";# no newline - we append the edge properties next
			$output .= " [edgetooltip=\"$line\", colorscheme=\"paired12\", color=\"$colorIndex\"]\n";
		} else {
			$filesToColorIndex{ $line } = $colorIndex = $nextColorIndex;
			$nextColorIndex = ($nextColorIndex % 12) +1; # mod 12 because we're using the paired12 colorscheme, which has 12 colors
			if ($noFiles == 0) {
				$output .= "\"$line\" [shape=box]\n";
				$output .= "\"$line\" -> { \"$sha\" } "; # no newline - we append the edge properties next
				$output .= " [edgetooltip=\"$line\", colorscheme=\"paired12\", color=\"$colorIndex\"]\n";
			}
		}

		$filesToSha{ $line } = $sha;
	}
}
$output .= "}\n";

if ($onlyPrintResult) {
	print $output;
	# Example use of output:
	#   cat tmp.dot | dot -Tsvg -o plot.svg
	#
	# Or to make the result tall instead of wide: 
	#   cat tmp.dot | ccomps -Cx | dot  | gvpack -array_1 | neato -n2 -Tsvg -o plot.svg
} else {
	run [ qw(ccomps -Cx) ], '<', \$output, "|", [ qw(dot) ], "|", [ qw(gvpack -array_1) ], "|", [ qw( neato -n2 -Tsvg -o ), $svg_filename ];
}

Re: RFC: Very useful script to SVG graph the git commits from a file orientated view

[Jeremy Rosen]

so, I started using it this WE for my big rebase 

I had aproximately 130 non-merge commits in my branch, a feature branch in which I had regularly merged master, but I needed to rebase everything and then reorganise most commits to make the whole thing reviewable

* merge bug-fix with the commit that introduced the bug
* change sloppy commit messages
* separate sloppy commit into multiple commits

I did my initial rebase and I am now doing repetitive "git rebase -i" to get everything going

your script is very usefull to me because it allows me to easily see the overall layout of things and figure what commit are "suspect" (touching files from different areas that have no reasons to be touched at the same time)

it also allows me to easily find commits that are highly connected to other ones and are the most likely to be problematic when reordering commits. Overall that script is awesome.

couple of ideas to refine things

* The tooltips are very handy, but it would be nice if the tooltip would activate on the whole commit ellipsis, not just the text inside the ellipsis
* I would love to have tooltips on the arrows too. when trying to follow what arrow is what file it makes things really handi

It would be nice if there were a way to filter only some files in the output... there probably is with the git-log like syntax but i'm not a master of that... hints would be welcome

is there a public repo for this script so I can point other people to it ?


    Cordialement

    Jérémy Rosen

fight key loggers : write some perl using vim

----- Mail original -----
> Opps, somehow I forgot to actually attach it.
> 
> It's now attached
> 

Re: RFC: Very useful script to SVG graph the git commits from a file orientated view

[Jeremy Rosen]

looking a little bit more into this, I was very suprised....

there seems to be little/no tools in the git ecosystem that studies the dependencies between commits based on the file they modified and/or the conflict they would cause.

Is there any pre-existing tool to do that ? It can be done with git-log --name-only(the graph_git.pl is just a graphing layer above that command) but i'm suprised that I couldn't find anything else

And that was at the file level, is there any tool to help find what commits can be reordered without causing conflicts ? I am not sure if there is an easy way to extract potential conflict information from git...

    Regards

    Jérémy Rosen

fight key loggers : write some perl using vim

----- Mail original -----
> Hi Jeremy,
> 
>   It would be great if you could send your email again to the list,
>   so
> that other people can see that there is interest in my script :)
> Makes it easier for me to get it included.
> 
> > * The tooltips are very handy, but it would be nice if the tooltip
> > would activate on the whole commit ellipsis, not just the text
> > inside the ellipsis
> 
> Yes, tooltips are a real pain.  I don't know how to do that.
> 
> > * I would love to have tooltips on the arrows too. when trying to
> > follow what arrow is what file it makes things really handi
> 
> Actually they do have tooltips - it's just that the arrows are really
> thin so you have to be very accurate with your mouse pointing!
> 
> > It would be nice if there were a way to filter only some files in
> > the output... there probably is with the git-log like syntax but
> > i'm not a master of that... hints would be welcome
> 
> Try just specifying the files:
> 
> $ graph_git.pl -10  filename1 filename2
> 
> I haven't tested, but it should work
> 
> > is there a public repo for this script so I can point other people
> > to it ?
> 
> No - any suggestions as to where to put it are welcome :)
> 
> John
> 

Re: RFC: Very useful script to SVG graph the git commits from a file orientated view

[Christian Couder]

Hi,

On Tue, Apr 9, 2013 at 10:55 AM, Jeremy Rosen <jeremy.rosen@openwide.fr> wrote:
> looking a little bit more into this, I was very suprised....
>
> there seems to be little/no tools in the git ecosystem that studies the dependencies between commits based on the file they modified and/or the conflict they would cause.
>
> Is there any pre-existing tool to do that ? It can be done with git-log --name-only(the graph_git.pl is just a graphing layer above that command) but i'm suprised that I couldn't find anything else
>
> And that was at the file level, is there any tool to help find what commits can be reordered without causing conflicts ? I am not sure if there is an easy way to extract potential conflict information from git...

It looks like this tool will do "Proactive Merge Conflicts Detection":

http://commitq.com/

But it's true that it would be nice if there was something in git itself.

Thanks,
Christian.