[PATCH v2 00/12] vcs-svn: incremental import

[Jonathan Nieder <jrnieder@gmail.com>]

Hi again,

Jonathan Nieder wrote:

> Using David's "ls" command we can eliminate the in-memory repo_tree
> and rely on the target repository for information about old revs.

Here's a reroll.  Aside from the aspects already mentioned (which
avoid a dependency on the mostly orthogonal topic of support for text
deltas), the original patch #10 has been split into smaller, more
easily digestible pieces.

Most of the credit for this incarnation of the series belongs to
David, who heroically streamlined it and untangled it from other
topics.

Patch 1 changes the mark numbers for blobs to be ridiculously high,
to make room for memorable commit marks (:1 for r1, :2 for r2, etc).
Patch 2 brings those commit marks into existence, as mentioned before.

Patches 3-5 simplify the repo-tree API somewhat.  They are somewhat
minimal; patches on top of this series offering further simplification
would be very welcome.

Patch 6 is a bit sneaky.  We want svn-fe's output to change from

	<import blob>
	<import blob>
	...
	<import blob>
	<import commit, using blobs>

to

	<commit header>
	M 100644 inline one/path
	<import blob>
	M 100644 inline another/path
	...
	<commit footer (progress update)>

since the latter allows svn-fe to maintain much less state.  But
that's a big change, so patch 6 introduces a stepping stone on the way
there:

	<comment that will become commit header>
	<import blob>
	...
	<import commit; this will become the commit footer>

That paves the way for patches 7-11, which teach svn-fe to rely
on the fast-import backend for information about previously
imported blobs, at long last.

The visible effects should be:

 - svn-fe _requires_ a backchannel from the fast-import
   backend now.  You can't do

	svn-fe <dump >stream &&
	fast-import <stream

   in two steps any more.

 - Given one dump that picks up where another left off, svn-fe
   can continue the import.  Use

	git fast-import --relative-marks \
		--import-marks-if-exists=svn-revs \
		--export-marks=svn-revs \
		--cat-blob-fd=3 3>backchannel

   for both imports.

I'm not happy about the loss of usability but I'm happy about the gain
in functionality.  A good next step might be to build a simple remote
helper to make this comfortable to use.

Thoughts?  Improvements?  Complaints?  Despite the deficiencies just
mentioned I'm tempted to push this out soon.  Feedback in either
direction would be welcome.

David Barr (3):
  vcs-svn: set up channel to read fast-import cat-blob response
  vcs-svn: quote paths correctly for ls command
  vcs-svn: use mark from previous import for parent commit

Jonathan Nieder (9):
  vcs-svn: use higher mark numbers for blobs
  vcs-svn: save marks for imported commits
  vcs-svn: introduce repo_read_path to check the content at a path
  vcs-svn: handle_node: use repo_read_path
  vcs-svn: simplify repo_modify_path and repo_copy
  vcs-svn: add a comment before each commit
  vcs-svn: allow input errors to be detected promptly
  vcs-svn: eliminate repo_tree structure
  vcs-svn: handle filenames with dq correctly

 contrib/svn-fe/svn-fe.txt |    6 +-
 t/t9010-svn-fe.sh         |  217 +++++++++++++++++++------
 vcs-svn/fast_export.c     |  145 +++++++++++++++--
 vcs-svn/fast_export.h     |   39 +++--
 vcs-svn/line_buffer.c     |    5 +
 vcs-svn/line_buffer.h     |    1 +
 vcs-svn/repo_tree.c       |  386 ++++++++-------------------------------------
 vcs-svn/repo_tree.h       |    5 +-
 vcs-svn/string_pool.c     |   13 ++-
 vcs-svn/string_pool.h     |    3 +-
 vcs-svn/svndump.c         |  106 +++++++++----
 11 files changed, 490 insertions(+), 436 deletions(-)
 rewrite vcs-svn/fast_export.h (75%)
 rewrite vcs-svn/repo_tree.c (96%)