git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Matteo Bertini <naufraghi@develer.com>
To: git@vger.kernel.org
Cc: Matteo Bertini <naufraghi@develer.com>,
	normalperson@yhbt.net, gitster@pobox.com
Subject: [PATCH/RFC 0/4] Add option to enable filters in git-svn
Date: Tue, 31 May 2016 17:07:45 +0200	[thread overview]
Message-ID: <20160531150749.24840-1-naufraghi@develer.com> (raw)

===========
Description
===========

This is a RFC for a patch that allows to enable filters in the repositories
created/managed by git-svn.


==================
Example scenario
==================

The usage scenario I’m facing is the migration of a large SVN repository to a
git repository, with big files being transparently handled through git-lfs

To recap, these migrations are usually handled through a transitioning repo
created by git-svn, which is then pushed to a remote git server; this server
acts as a read-only mirror for a while, until the main SVN server is shut down.
For large repos, the process can take several weeks to adjust the migration
scripts, branch mapping, authorship, etc., and it’s also common to update the
mirror incrementally.


Without this patch
==================

Without this patch, the migration needs to be setup with a pipeline with:

* incremental git svn fetch,
* bfg-repo-cleaner with —convert-to-git-lfs option (which puts invalid
  .gitattributes files, see https://github.com/rtyley/bfg-repo-cleaner/issues/143)
* git-filter-branch (to put in place the correct .gitattributes files)

A minimal change in the pipeline results in unrelated branches after
the rebases.

Each deviation from the svn sources must be committed back or saved
as patch and reapplied after the rebase.


With this patch
===============

Everything is much simpler: filters can be triggered during the fetch, so that
git-lfs already runs while the repository is fetched.
At the end of the process, the local git repository is already in its final
form, and no further git-filter-branch/rebases is required.

The transitioning git repo can diverge partially from the svn source
thanks to git svn rebase.


Details
=======

In the current implementation git-svn uses the hash-object command
with the --no-filters option. There is no gain in removing that option
because the git-svn uses temporary file names.

In this patch hash-object accepts a new format for the streaming input,
files and paths, tab separated. git-svn can provide the real path
to hash-object and thanks to that we can enable the filters.


Status
======

The patch is in RFC, no new test where written (yet), just runned for
regressions (https://travis-ci.org/naufraghi/git/builds/134170350).

I'd like to gain some feedback before spending more time on the
feature, mainly:

 * feature comments,
 * coding comments (neither Perl or C are my main languages),
 * user interface (option names, config placements) comments.

Best,
Matteo


Matteo Bertini (4):
  hash-object.c: Allow distinct file/path in stdin mode too.
  Git.pm: Add $path and $enable_filters arguments to
    hash_and_insert_object.
  SVN/Fetcher.pm: Add svn-remote.<id>.enable-filters to enable the
    filters.
  git-svn.perl: Add git svn init --enable-filters option.

 builtin/hash-object.c   | 29 +++++++++++++++++++++++++++--
 git-svn.perl            |  4 ++++
 perl/Git.pm             | 19 +++++++++++++------
 perl/Git/SVN/Fetcher.pm | 16 ++++++++++++----
 4 files changed, 56 insertions(+), 12 deletions(-)

-- 
2.9.0.rc0.39.gb9f310b.dirty

             reply	other threads:[~2016-05-31 15:15 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-05-31 15:07 Matteo Bertini [this message]
2016-05-31 15:07 ` [PATCH 1/4] hash-object.c: Allow distinct file/path in stdin mode too Matteo Bertini
2016-05-31 15:07 ` [PATCH 2/4] Git.pm: Add $path and $enable_filters arguments to hash_and_insert_object Matteo Bertini
2016-05-31 15:07 ` [PATCH 3/4] SVN/Fetcher.pm: Add svn-remote.<id>.enable-filters to enable the filters Matteo Bertini
2016-05-31 15:07 ` [PATCH 4/4] git-svn.perl: Add git svn init --enable-filters option Matteo Bertini
2016-05-31 15:34 ` [PATCH/RFC 0/4] Add option to enable filters in git-svn Matteo Bertini
2016-05-31 18:12   ` Eric Wong
2016-05-31 20:34     ` Matteo Bertini
2016-06-02 21:13       ` Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20160531150749.24840-1-naufraghi@develer.com \
    --to=naufraghi@develer.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=normalperson@yhbt.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).