[PATCH/RFC 0/2] git-svn: Allow certain refs to be ignored

[PATCH/RFC 0/2] git-svn: Allow certain refs to be ignored

[Michael Olson]

This change allows certain refs to be ignored while importing svn
history.  This is useful for:

 - Performing overlapping transforms from svn's branch namespace into
   git's namespace.  The overlapping refs can be ignored.  It might
   potentially be a better idea in this case to make git-svn assign
   precedence according to the order in which the branch patterns are
   specified, perhaps with an option to trigger that behavior (which
   is out of scope for this patch series).

 - Avoiding some refs that represent bad svn operations which cause
   git-svn to take a very long time.  Example: copying one module
   accidentally into another module's branch namespace.

A new config directive called "ignore-refs" implements this feature by
means of a regex of refs to ignore.  I haven't written the necessary
git-svn documentation updates for it yet.

In addition, I ran across a problem where git-svn would die if a
parent ref did not exist.  This might possibly have been exposed as a
result of the first patch.  I threw an eval around the offending code,
and that seems to work, though some review would be appreciated.

Here is an example ~/.git/config file which works with these changes.
The example upstream svn repository has branches in tags in 2
different namespaces.  Originally all tags/branches were directly
under /root/mod/branches and /root/mod/tags.  Later on, they created
new branches and tags in subdirectories such as
/root/mod/branches/myorg/bugs/BUGID.  This config file forces
old-style branches and tags into the "old/" namespace in git, and puts
new-style branches and tags into (for example) "myorg/bugs/BUGID".
The "ignore-refs" directive prevents the first level of the new
namespace ("myorg", in particular) from being replicated underneath
"old/" in git.

[core]
       repositoryformatversion = 0
       filemode = true
       bare = false
       logallrefupdates = true
[svn-remote "svn"]
       url = https://svn.my.org/svn/root
       fetch = mod/trunk:refs/remotes/trunk
       branches = mod/branches/*:refs/remotes/old/*
       tags = mod/tags/*:refs/remotes/tags/old/*
       branches = mod/branches/myorg/bugs/*:refs/remotes/myorg/bugs/*
       tags = mod/tags/myorg/bugs/*:refs/remotes/tags/myorg/bugs/*
       branches = mod/branches/myorg/projects/*:refs/remotes/myorg/projects/*
       tags = mod/tags/myorg/projects/*:refs/remotes/tags/myorg/projects/*
       branches = mod/branches/myorg/releases/*:refs/remotes/myorg/releases/*
       tags = mod/tags/myorg/releases/*:refs/remotes/tags/myorg/releases/*
       ignore-refs = ^refs/remotes/(tags/)?old/myorg

Michael Olson (2):
  git-svn: Allow certain refs to be ignored
  git-svn: Don't allow missing commit parent to stop git-svn

 git-svn.perl |   40 ++++++++++++++++++++++++++++++++++------
 1 files changed, 34 insertions(+), 6 deletions(-)

Re: [PATCH/RFC 0/2] git-svn: Allow certain refs to be ignored

[Junio C Hamano]

Michael Olson <mwolson@gnu.org> writes:

> [svn-remote "svn"]
>        url = https://svn.my.org/svn/root
> ...
>        ignore-refs = ^refs/remotes/(tags/)?old/myorg

Traditionally configuration variable names are spelled camelCase without
dashes.  You probably would want to be consistent.

Also "refs" and any pathname-like things are traditionally matched using
globs and not regexes.  It is Ok to deviate if you have a strong reason to
(and I suspect it would make it easier to write "exclude" patterns like
the above example to allow a regex here), but that needs to be prominently
documented (e.g. "Unlike any other ref-matching configuration variable,
this alone uses regex, not glob") to avoid end user confusion.

Re: [PATCH/RFC 0/2] git-svn: Allow certain refs to be ignored

[Eric Wong]

Junio C Hamano <gitster@pobox.com> wrote:
> Michael Olson <mwolson@gnu.org> writes:
> 
> > [svn-remote "svn"]
> >        url = https://svn.my.org/svn/root
> > ...
> >        ignore-refs = ^refs/remotes/(tags/)?old/myorg
> 
> Traditionally configuration variable names are spelled camelCase without
> dashes.  You probably would want to be consistent.

Configuration variables should definitely be camelCase in
examples/documentation (and I even dislike camelCase).  No dashes or
underscores here.

> Also "refs" and any pathname-like things are traditionally matched using
> globs and not regexes.  It is Ok to deviate if you have a strong reason to
> (and I suspect it would make it easier to write "exclude" patterns like
> the above example to allow a regex here), but that needs to be prominently
> documented (e.g. "Unlike any other ref-matching configuration variable,
> this alone uses regex, not glob") to avoid end user confusion.

I favor globs for more consistent/natural for path matching.

Another thing to keep in mind is that these would be Perl regular
expressions exposed to a user interface.  If git-svn were ever be
reimplemented in something other than Perl, PCRE (or similar) would be
required to interpret them consistently.

-- 
Eric Wong

Re: [PATCH/RFC 0/2] git-svn: Allow certain refs to be ignored

[Michael Olson]

On Tue, Apr 20, 2010 at 4:58 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Michael Olson <mwolson@gnu.org> writes:
>
>> [svn-remote "svn"]
>>        url = https://svn.my.org/svn/root
>> ...
>>        ignore-refs = ^refs/remotes/(tags/)?old/myorg
>
> Traditionally configuration variable names are spelled camelCase without
> dashes.  You probably would want to be consistent.

I was looking at the existing syntax for "ignore-paths", which uses
dashes.  Should that be changed as well?

> Also "refs" and any pathname-like things are traditionally matched using
> globs and not regexes.  It is Ok to deviate if you have a strong reason to
> (and I suspect it would make it easier to write "exclude" patterns like
> the above example to allow a regex here), but that needs to be prominently
> documented (e.g. "Unlike any other ref-matching configuration variable,
> this alone uses regex, not glob") to avoid end user confusion.

ignore-paths also uses a regex.  I'm concerned that using globs will
not be expressive enough to represent a regex like (a combination of
the 2 use cases I posted initially):

^refs/remotes/((tags/)?old/myorg|old/bad_branch|old/unlabeled[^/]*|tags/(old/bad_tag|releases/another_tag))$

-- 
Michael Olson  |  http://mwolson.org/

Re: [PATCH/RFC 0/2] git-svn: Allow certain refs to be ignored

[Eric Wong]

Michael Olson <mwolson@gnu.org> wrote:
> On Tue, Apr 20, 2010 at 4:58 PM, Junio C Hamano <gitster@pobox.com> wrote:
> > Michael Olson <mwolson@gnu.org> writes:
> >
> >> [svn-remote "svn"]
> >>        url = https://svn.my.org/svn/root
> >> ...
> >>        ignore-refs = ^refs/remotes/(tags/)?old/myorg
> >
> > Traditionally configuration variable names are spelled camelCase without
> > dashes.  You probably would want to be consistent.
> 
> I was looking at the existing syntax for "ignore-paths", which uses
> dashes.  Should that be changed as well?

Yes, I missed that the first time around :x

I suppose that should be changed to work both ways.

> > Also "refs" and any pathname-like things are traditionally matched using
> > globs and not regexes.  It is Ok to deviate if you have a strong reason to
> > (and I suspect it would make it easier to write "exclude" patterns like
> > the above example to allow a regex here), but that needs to be prominently
> > documented (e.g. "Unlike any other ref-matching configuration variable,
> > this alone uses regex, not glob") to avoid end user confusion.
> 
> ignore-paths also uses a regex.  I'm concerned that using globs will
> not be expressive enough to represent a regex like (a combination of
> the 2 use cases I posted initially):

I also missed that (I never used that feature :x)

> ^refs/remotes/((tags/)?old/myorg|old/bad_branch|old/unlabeled[^/]*|tags/(old/bad_tag|releases/another_tag))$

I wonder if both would be better as a series of globs would be
easier to read (maybe more verbose to write):

	ignoreRefs = refs/remotes/tags/old/myorg
	ignoreRefs = refs/remotes/old/myorg
	ignoreRefs = refs/remotes/old/unlabeled/*

But I suppose the regexp route is fine.  I'll ack and push them
out with Sam's Ack on 2/2

-- 
Eric Wong

Re: [PATCH/RFC 0/2] git-svn: Allow certain refs to be ignored

[Michael Olson]

On Wed, Apr 21, 2010 at 9:54 AM, Eric Wong <normalperson@yhbt.net> wrote:
> Michael Olson <mwolson@gnu.org> wrote:
>> ^refs/remotes/((tags/)?old/myorg|old/bad_branch|old/unlabeled[^/]*|tags/(old/bad_tag|releases/another_tag))$
>
> I wonder if both would be better as a series of globs would be
> easier to read (maybe more verbose to write):
>
>        ignoreRefs = refs/remotes/tags/old/myorg
>        ignoreRefs = refs/remotes/old/myorg
>        ignoreRefs = refs/remotes/old/unlabeled/*

That does look cleaner, and kind of reminiscent of .gitignore.

-- 
Michael Olson  |  http://mwolson.org/

Re: [PATCH/RFC 0/2] git-svn: Allow certain refs to be ignored

[Eric Wong]

Michael Olson <mwolson@gnu.org> wrote:
> This change allows certain refs to be ignored while importing svn
> history.  This is useful for:
> 
>  - Performing overlapping transforms from svn's branch namespace into
>    git's namespace.  The overlapping refs can be ignored.  It might
>    potentially be a better idea in this case to make git-svn assign
>    precedence according to the order in which the branch patterns are
>    specified, perhaps with an option to trigger that behavior (which
>    is out of scope for this patch series).
> 
>  - Avoiding some refs that represent bad svn operations which cause
>    git-svn to take a very long time.  Example: copying one module
>    accidentally into another module's branch namespace.

Hi Michael, these are definitely good things to have.  Thanks!

> A new config directive called "ignore-refs" implements this feature by
> means of a regex of refs to ignore.  I haven't written the necessary
> git-svn documentation updates for it yet.

See Junio's and my other email.

> In addition, I ran across a problem where git-svn would die if a
> parent ref did not exist.  This might possibly have been exposed as a
> result of the first patch.  I threw an eval around the offending code,
> and that seems to work, though some review would be appreciated.

I'm cc-ing Sam for this one.  It looks good to me given the first patch
would cause more refs to not exist.

-- 
Eric Wong