Re: GSOC Proposal draft: git-remote-svn

[Jonathan Nieder <jrnieder@gmail.com>]

Florian Achleitner wrote:

> If the remote-svn-alpha script is really all that needs to be done, you're 
> right. It just pipes through svn-fe. I thought svn-fe could only import an svn 
> repo initially, and there would be some difference between importing the whole 
> history and fetching new revisions later, (?).

Yes, Dmitry's script (not the first version, but a later one) supports
incremental imports without trouble if I remember correctly.

[...]
> Listing patches and planing all details in the submitted proposal would 
> require me to know what I do and how I will do it all before last Friday! As 
> I'm not yet an expert on this topic, I don't know how I could have known all 
> details a-priori.

Oh, I didn't mean you would need to do that alone. :)  Dmitry, David,
Ram, Sverre, and I should be able to answer any questions you have
about how git, vcs-svn, svnrdump, and the transport-helper currently
work in the importer.

I've marked the proposal editable to allow details to be filled in.

[...]
> I planned to implement a remote-helper using the existing interface 
> specification to communicate over pipes with git's transport-helper. 
> Instead of invoking svn-fe as a subprocess, I want to call vcs-svn/ functions 
> directly from the remote-helper and place new functions in this directory (?).

Ah, this is a good place to start.  In my diagram I lumped everything
under vcs-svn/ together as svn-fe for convenience, but in fact the
vcs-svn lib is made up of multiple components:

	caller
	 .
	 :
	 |
	public interface (svndump_init, svndump_read, etc)
	 |
	 |
	 |
	dump file parser (svndump_read body)
	 |
	 |
	 |
	fast-export interface (fast_export_*, repo_*) --------- svndiff0 parser
	 |
	 :
	 .
	git fast-import

Each component has a narrow interface.  For each action in the dump,
svndump_read() calls some appropriate function from the fast-export
interface to bring about the corresponding change on the git side.
Details of svndump syntax and the state needed to parse it are
isolated in svndump.c and details of fast-import syntax are in
fast-export.c and repo_tree.c.

(The structure used to be more complicated when the repo_* functions
had to keep track of the repository state instead of relying on
fast-import for that.)

Where would the branch mapping go?  What kind of state needs to be
maintained as it occurs?  What steps would I follow to imitate the
code and work out a branch mapping on paper?  How do I invoke the code
if I want to try it out (i.e., what functions form the public
interface needed to support branch mapping)?

I don't expect you to have answers to all these questions already; I
understand that getting used to what's already there and trying out
ideas will take time.  However, I do think we have a much better
chance of this going well if there are answers to these questions by
the time the coding period starts.

[...]
> Additionally the remote-helper will read a configuration file containing 
> additional information about branch-mapping, this should be closely related to 
> Andrew's SBL.

That sounds reasonable to me.  I am somewhat unconvinced (but
convinceable) about the need to use a configuration scheme that
handles all the edge cases right away.  Shouldn't it be enough to tell
the importer the following?

 - the path to the repository (from which it can deduce $SVNROOT
   and the path within there to the subproject of interest)

 - a single bit of information on top of that: "this repository uses
   the standard layout"

Once that works, the tools could easily be tweaked to respect a
configuration file that describes more complex situations, and as a
bonus the SBL tools for making sense of those situations would have
time to become more mature in the meantime.

Thanks for some useful clarifications.
Jonathan