[PATCH] Let git-svnimport clean up SVK commit messages.

[PATCH] Let git-svnimport clean up SVK commit messages.

[Dave O'Neill]

SVK likes to begin all commit messages with a line of the format:
   r12345@hostname: user | YYYY-MM-DD HH:MM:SS -ZZZZ
which makes the import desperately ugly in git.  This adds a -k option to move
this extra SVK commit line to the end of the commit message, rather than
keeping it at the beginning.
---
 Documentation/git-svnimport.txt |    6 +++++-
 git-svnimport.perl              |   18 +++++++++++++++---
 2 files changed, 20 insertions(+), 4 deletions(-)

diff --git a/Documentation/git-svnimport.txt b/Documentation/git-svnimport.txt
index e97d15e..23a54b0 100644
--- a/Documentation/git-svnimport.txt
+++ b/Documentation/git-svnimport.txt
@@ -11,7 +11,7 @@ SYNOPSIS
 --------
 [verse]
 'git-svnimport' [ -o <branch-for-HEAD> ] [ -h ] [ -v ] [ -d | -D ]
-		[ -C <GIT_repository> ] [ -i ] [ -u ] [-l limit_rev]
+		[ -C <GIT_repository> ] [ -i ] [ -k ] [ -u ] [-l limit_rev]
 		[ -b branch_subdir ] [ -T trunk_subdir ] [ -t tag_subdir ]
 		[ -s start_chg ] [ -m ] [ -r ] [ -M regex ]
 		[ -I <ignorefile_name> ] [ -A <author_file> ]
@@ -68,6 +68,10 @@ When importing incrementally, you might need to edit the .git/svn2git file.
 	Prepend 'rX: ' to commit messages, where X is the imported
 	subversion revision.
 
+-k::
+	Remove SVK-style revision/commiter info from the top of commit
+	messages and move it to the bottom.
+
 -I <ignorefile_name>::
 	Import the svn:ignore directory property to files with this
 	name in each directory. (The Subversion and GIT ignore
diff --git a/git-svnimport.perl b/git-svnimport.perl
index f459762..7076f41 100755
--- a/git-svnimport.perl
+++ b/git-svnimport.perl
@@ -32,20 +32,20 @@ $ENV{'TZ'}="UTC";
 
 our($opt_h,$opt_o,$opt_v,$opt_u,$opt_C,$opt_i,$opt_m,$opt_M,$opt_t,$opt_T,
     $opt_b,$opt_r,$opt_I,$opt_A,$opt_s,$opt_l,$opt_d,$opt_D,$opt_S,$opt_F,
-    $opt_P,$opt_R);
+    $opt_P,$opt_R,$opt_k);
 
 sub usage() {
 	print STDERR <<END;
 Usage: ${\basename $0}     # fetch/update GIT from SVN
        [-o branch-for-HEAD] [-h] [-v] [-l max_rev] [-R repack_each_revs]
        [-C GIT_repository] [-t tagname] [-T trunkname] [-b branchname]
-       [-d|-D] [-i] [-u] [-r] [-I ignorefilename] [-s start_chg]
+       [-d|-D] [-i] [-k] [-u] [-r] [-I ignorefilename] [-s start_chg]
        [-m] [-M regex] [-A author_file] [-S] [-F] [-P project_name] [SVN_URL]
 END
 	exit(1);
 }
 
-getopts("A:b:C:dDFhiI:l:mM:o:rs:t:T:SP:R:uv") or usage();
+getopts("A:b:C:dDFhiI:kl:mM:o:rs:t:T:SP:R:uv") or usage();
 usage if $opt_h;
 
 my $tag_name = $opt_t || "tags";
@@ -835,7 +835,19 @@ sub commit {
 		$pw->writer();
 		$pr->reader();
 
+
+		if ( $opt_k ) {
+			# Remove SVK cruft in commit message
+			my ($svk_info) = $message =~ s{(r\d+\@[^:]+:\s+\S+\s+\|\s+\d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}:\d{2}\s[\+-]\d{4})\s*}{};
+			if( $svk_info ) {
+				# and stick it on the end so we don't lose it
+				$message .= "\n$svk_info"
+			}
+		}
+
+
 		$message =~ s/[\s\n]+\z//;
+
 		$message = "r$revision: $message" if $opt_r;
 
 		print $pw "$message\n"
-- 
1.5.2.2.239.g89630

Re: [PATCH] Let git-svnimport clean up SVK commit messages.

[Steven Grimm]

Dave O'Neill wrote:
> SVK likes to begin all commit messages with a line of the format:
>    r12345@hostname: user | YYYY-MM-DD HH:MM:SS -ZZZZ
> which makes the import desperately ugly in git.  This adds a -k option to move
> this extra SVK commit line to the end of the commit message, rather than
> keeping it at the beginning.
>   

Any chance of applying this to git-svn instead? There has been talk of 
deprecating git-svnimport since git-svn now does everything 
git-svnimport does, and more. (If you believe that's not the case, 
please describe what you're doing with git-svnimport that you can't do 
with git-svn.)

-Steve

Re: [PATCH] Let git-svnimport clean up SVK commit messages.

[Sam Vilain]

Dave O'Neill wrote:
> SVK likes to begin all commit messages with a line of the format:
>    r12345@hostname: user | YYYY-MM-DD HH:MM:SS -ZZZZ
> which makes the import desperately ugly in git.  This adds a -k option to move
> this extra SVK commit line to the end of the commit message, rather than
> keeping it at the beginning.

This is a good idea, of course if somebody didn't specify the magic -I
switch to their 'svk sm' incantation then there will be multiple changes
listed in a single revision

some examples

http://dev.catalystframework.org/svnweb/Catalyst/revision/?rev=6477
svn log -r 7190:7190 http://svn.pugscode.org/pugs

There was also a pretty nasty bug in SVK which pushed huge commitlogs
with no changes

see for example

http://utsl.gen.nz/gitweb/?p=pugs;a=commit;h=817b73f
(or:
svn log -r 14734:14734 http://svn.pugscode.org/pugs
svn diff -r 14733:14734 http://svn.pugscode.org/pugs
)

That occurred often enough that it might even be worth detecting and
dealing with specially.  ie, if multiple SVK changesets are seen in a
commit with no changes, mark it as likely bogus.

Yeah, I'm not sure what to say about all this other than "lolsvn".

Sam.

Re: [PATCH] Let git-svnimport clean up SVK commit messages.

[Dave O'Neill]

On Wed, Jun 20, 2007 at 08:19:33PM -0700, Steven Grimm wrote:

> Any chance of applying this to git-svn instead? There has been talk of 
> deprecating git-svnimport since git-svn now does everything 
> git-svnimport does, and more. (If you believe that's not the case, 
> please describe what you're doing with git-svnimport that you can't do 
> with git-svn.)

Sure, I can probably apply it to git-svn as well, but based on the
testing I've done, git-svnimport still works better for what I'm doing.
I'm trying to do a straight conversion from SVN to git, so we can do
away with our Subversion repositories, so I won't need the bidirectional
support of git-svn.  

If git-svn had a --one-way option that could 
  - know that I'm converting, and import my SVN tags and branches to
    local tags and heads rather than remotes
  - turn off the git-svn-id: tags in the commit
  - avoid preserving a .git/svn/ directory (a git-svn conversion was 50%
    larger than one from git-svnimport because of this)
  - convert svn:ignore attrs to .gitignore (like the -I option of
    git-svnignore)

then I wouldn't need git-svnimport.  Alternatively, if there's some way
to postprocess my clone to do all of the above, that would probably be
good enough.

Cheers,
Dave

Re: [PATCH] Let git-svnimport clean up SVK commit messages.

[Dave O'Neill]

On Thu, Jun 21, 2007 at 06:01:01PM +1200, Sam Vilain wrote:
> This is a good idea, of course if somebody didn't specify the magic -I
> switch to their 'svk sm' incantation then there will be multiple changes
> listed in a single revision

Yeah, I don't think there's much hope of fully cleaning up those
messages.  My goal was just to make 'git log --pretty=oneline' show
something meaningful from the commit message, rather than:

edfbf48adabb68f9b3973f5950570c355f3ee811  r6211@oxygen:  dmo | 2007-04-17 17:28:23 -0400
d5b3268e815ce9e88dd617a5b62df80da263f396  r6210@oxygen:  dmo | 2007-04-17 15:54:04 -0400

Cheers,
Dave

Re: [PATCH] Let git-svnimport clean up SVK commit messages.

[Jeffrey C. Ollie]

[-- Attachment #1: Type: text/plain, Size: 1847 bytes --]

On Thu, 2007-06-21 at 10:14 -0400, Dave O'Neill wrote:
> On Wed, Jun 20, 2007 at 08:19:33PM -0700, Steven Grimm wrote:
> 
> > Any chance of applying this to git-svn instead? There has been talk of 
> > deprecating git-svnimport since git-svn now does everything 
> > git-svnimport does, and more. (If you believe that's not the case, 
> > please describe what you're doing with git-svnimport that you can't do 
> > with git-svn.)
> 
> Sure, I can probably apply it to git-svn as well, but based on the
> testing I've done, git-svnimport still works better for what I'm doing.
> I'm trying to do a straight conversion from SVN to git, so we can do
> away with our Subversion repositories, so I won't need the bidirectional
> support of git-svn.  
> 
> If git-svn had a --one-way option that could 
>   - know that I'm converting, and import my SVN tags and branches to
>     local tags and heads rather than remotes

This quickie Python script should convert the "tag branches" to regular
Git tags:

        import os
        refs = os.popen('git-for-each-ref --format="%(objectname)%00%(refname)%00" refs/remotes/tags')
        for line in refs:
            [sha, ref] = line.split('\0')[:2]
            os.system('git tag %s %s' % (ref[18:], sha))


>   - turn off the git-svn-id: tags in the commit

git svn fetch --no-metadata

>   - avoid preserving a .git/svn/ directory (a git-svn conversion was 50%
>     larger than one from git-svnimport because of this)

Once you are done importing, you can delete that directory.

>   - convert svn:ignore attrs to .gitignore (like the -I option of
>     git-svnignore)

git svn show-ignore

> then I wouldn't need git-svnimport.  Alternatively, if there's some way
> to postprocess my clone to do all of the above, that would probably be
> good enough.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[PATCH] Let git-svnimport and git-svn clean up SVK commit messages

[Dave O'Neill]

SVN repositories that have been committed to with SVK will have commits that
contain SVK metadata on the first line of the message:

   r12345@hostname: user | YYYY-MM-DD HH:MM:SS -ZZZZ

After importing, git will then pick up this first as the commit title, which
isn't terribly informative in 'git log --pretty=oneline' or gitk.

This patch adds a -k option to git-svnimport and a --fix-svk-message option
(and fixSvkMessage config option) to git-svn.  Enabling this option allows
git-svn to try and move that metadata to the end of the commit message,
possibly giving us a more meaningful first line.
---
 Documentation/git-svn.txt       |   13 +++++++++++++
 Documentation/git-svnimport.txt |    6 +++++-
 git-svn.perl                    |   23 ++++++++++++++++++++---
 git-svnimport.perl              |   15 ++++++++++++---
 4 files changed, 50 insertions(+), 7 deletions(-)

diff --git a/Documentation/git-svn.txt b/Documentation/git-svn.txt
index c0d7d95..5cd89f9 100644
--- a/Documentation/git-svn.txt
+++ b/Documentation/git-svn.txt
@@ -54,6 +54,8 @@ COMMANDS
 	Set the 'useSvmProps' option in the [svn-remote] config.
 --use-svnsync-props;;
 	Set the 'useSvnsyncProps' option in the [svn-remote] config.
+--fix-svk-message;;
+	Set the 'fixSvkMessage' option in the [svn-remote] config.
 --rewrite-root=<URL>;;
 	Set the 'rewriteRoot' option in the [svn-remote] config.
 --username=<USER>;;
@@ -385,6 +387,14 @@ svn-remote.<name>.rewriteRoot::
 	the repository with a public http:// or svn:// URL in the
 	metadata so users of it will see the public URL.
 
+svn-remote.<name>.fixSvkMessage::
+	SVN repositories that have been committed to with SVK will have
+	commits that contain SVK metadata on the first line of the
+	message, which git will then pick up as the commit title.  Enabling
+	this option allows git-svn to try and move that metadata to the end
+	of the commit message, possibly giving us a more meaningful first
+	line.
+
 Since the noMetadata, rewriteRoot, useSvnsyncProps and useSvmProps
 options all affect the metadata generated and used by git-svn; they
 *must* be set in the configuration file before any history is imported
@@ -393,6 +403,9 @@ and these settings should never be changed once they are set.
 Additionally, only one of these four options can be used per-svn-remote
 section because they affect the 'git-svn-id:' metadata line.
 
+The fixSvkMessage option may be set in combination with any of these, and
+can be set after import if desired.
+
 --
 
 BASIC EXAMPLES
diff --git a/Documentation/git-svnimport.txt b/Documentation/git-svnimport.txt
index e97d15e..23a54b0 100644
--- a/Documentation/git-svnimport.txt
+++ b/Documentation/git-svnimport.txt
@@ -11,7 +11,7 @@ SYNOPSIS
 --------
 [verse]
 'git-svnimport' [ -o <branch-for-HEAD> ] [ -h ] [ -v ] [ -d | -D ]
-		[ -C <GIT_repository> ] [ -i ] [ -u ] [-l limit_rev]
+		[ -C <GIT_repository> ] [ -i ] [ -k ] [ -u ] [-l limit_rev]
 		[ -b branch_subdir ] [ -T trunk_subdir ] [ -t tag_subdir ]
 		[ -s start_chg ] [ -m ] [ -r ] [ -M regex ]
 		[ -I <ignorefile_name> ] [ -A <author_file> ]
@@ -68,6 +68,10 @@ When importing incrementally, you might need to edit the .git/svn2git file.
 	Prepend 'rX: ' to commit messages, where X is the imported
 	subversion revision.
 
+-k::
+	Remove SVK-style revision/commiter info from the top of commit
+	messages and move it to the bottom.
+
 -I <ignorefile_name>::
 	Import the svn:ignore directory property to files with this
 	name in each directory. (The Subversion and GIT ignore
diff --git a/git-svn.perl b/git-svn.perl
index 50128d7..1451f0d 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -83,6 +83,7 @@ my %init_opts = ( 'template=s' => \$_template, 'shared:s' => \$_shared,
                   'trunk|T=s' => \$_trunk, 'tags|t=s' => \$_tags,
                   'branches|b=s' => \$_branches, 'prefix=s' => \$_prefix,
                   'minimize-url|m' => \$Git::SVN::_minimize_url,
+		  'fix-svk-message' => \$Git::SVN::_fix_svk_message,
 		  'no-metadata' => sub { $icv{noMetadata} = 1 },
 		  'use-svm-props' => sub { $icv{useSvmProps} = 1 },
 		  'use-svnsync-props' => sub { $icv{useSvnsyncProps} = 1 },
@@ -280,6 +281,11 @@ sub do_git_init_db {
 		command_noisy('config', "$pfx.$i", $icv{$i});
 		$set = $i;
 	}
+
+	if( defined $Git::SVN::_fix_svk_message ) {
+		command_noisy('config', "$pfx.fixSvkMessage", $Git::SVN::_fix_svk_message);
+	}
+
 }
 
 sub init_subdir {
@@ -826,7 +832,7 @@ sub working_head_info {
 package Git::SVN;
 use strict;
 use warnings;
-use vars qw/$default_repo_id $default_ref_id $_no_metadata $_follow_parent
+use vars qw/$default_repo_id $default_ref_id $_no_metadata $_follow_parent $_fix_svk_message
             $_repack $_repack_flags $_use_svm_props $_head
             $_use_svnsync_props $no_reuse_existing $_minimize_url/;
 use Carp qw/croak/;
@@ -849,7 +855,7 @@ BEGIN {
 	# per [svn-remote "..."] section.  Command-line options will *NOT*
 	# override options set in an [svn-remote "..."] section
 	no strict 'refs';
-	for my $option (qw/follow_parent no_metadata use_svm_props
+	for my $option (qw/follow_parent no_metadata use_svm_props fix_svk_message
 			   use_svnsync_props/) {
 		my $key = $option;
 		$key =~ tr/_//d;
@@ -1613,7 +1619,18 @@ sub do_git_commit {
 	}
 	defined(my $pid = open3(my $msg_fh, my $out_fh, '>&STDERR', @exec))
 	                                                           or croak $!;
-	print $msg_fh $log_entry->{log} or croak $!;
+	my $log_msg = $log_entry->{log};
+
+	if( $self->fix_svk_message ) {
+		# Remove SVK cruft in commit message
+		$log_msg =~ s{(r\d+\@[^:]+:\s+\S+\s+\|\s+\d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}:\d{2}\s[\+-]\d{4})\s*}{};
+		if( $1 ) {
+			# and stick it on the end so we don't lose it
+			$log_msg .= "\n$1"
+		}
+	}
+
+	print $msg_fh $log_msg or croak $!;
 	unless ($self->no_metadata) {
 		print $msg_fh "\ngit-svn-id: $log_entry->{metadata}\n"
 		              or croak $!;
diff --git a/git-svnimport.perl b/git-svnimport.perl
index f459762..89afdd8 100755
--- a/git-svnimport.perl
+++ b/git-svnimport.perl
@@ -32,20 +32,20 @@ $ENV{'TZ'}="UTC";
 
 our($opt_h,$opt_o,$opt_v,$opt_u,$opt_C,$opt_i,$opt_m,$opt_M,$opt_t,$opt_T,
     $opt_b,$opt_r,$opt_I,$opt_A,$opt_s,$opt_l,$opt_d,$opt_D,$opt_S,$opt_F,
-    $opt_P,$opt_R);
+    $opt_P,$opt_R,$opt_k);
 
 sub usage() {
 	print STDERR <<END;
 Usage: ${\basename $0}     # fetch/update GIT from SVN
        [-o branch-for-HEAD] [-h] [-v] [-l max_rev] [-R repack_each_revs]
        [-C GIT_repository] [-t tagname] [-T trunkname] [-b branchname]
-       [-d|-D] [-i] [-u] [-r] [-I ignorefilename] [-s start_chg]
+       [-d|-D] [-i] [-k] [-u] [-r] [-I ignorefilename] [-s start_chg]
        [-m] [-M regex] [-A author_file] [-S] [-F] [-P project_name] [SVN_URL]
 END
 	exit(1);
 }
 
-getopts("A:b:C:dDFhiI:l:mM:o:rs:t:T:SP:R:uv") or usage();
+getopts("A:b:C:dDFhiI:kl:mM:o:rs:t:T:SP:R:uv") or usage();
 usage if $opt_h;
 
 my $tag_name = $opt_t || "tags";
@@ -835,6 +835,15 @@ sub commit {
 		$pw->writer();
 		$pr->reader();
 
+		if ( $opt_k ) {
+			# Remove SVK cruft in commit message
+			$message =~ s{(r\d+\@[^:]+:\s+\S+\s+\|\s+\d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}:\d{2}\s[\+-]\d{4})\s*}{};
+			if( $1 ) {
+				# and stick it on the end so we don't lose it
+				$message .= "\n$1"
+			}
+		}
+
 		$message =~ s/[\s\n]+\z//;
 		$message = "r$revision: $message" if $opt_r;
 
-- 
1.5.2.2.239.g89630

Re: [PATCH] Let git-svnimport clean up SVK commit messages.

[Dave O'Neill]

On Thu, Jun 21, 2007 at 10:00:00AM -0500, Jeffrey C. Ollie wrote:
> This quickie Python script should convert the "tag branches" to regular
> Git tags:
> 
>         import os
>         refs = os.popen('git-for-each-ref --format="%(objectname)%00%(refname)%00" refs/remotes/tags')
>         for line in refs:
>             [sha, ref] = line.split('\0')[:2]
>             os.system('git tag %s %s' % (ref[18:], sha))

Thanks!  I'll give that a try.  It would be nice if this sort of cleanup
were part of git-svn in some way, though.  I doubt I'll be the last
person wanting an easy conversion from SVN.

Cheers,
Dave