Switch from svn to git and modify repo completely

Switch from svn to git and modify repo completely

[Matthias Fechner]

Dear Git List,

I just started to migrate a svn repo to a git repo and was successfully
with the "git svn" command and waiting a long time :)

I have now some confidential documents in the repository I must remove
completely (including the complete history). These are single files
spread over the complete repository. Is this possible with git?

The next step I would like to do is separate the repo and smaller ones.
The problem here is it is not only take this directory and move it out,
it is a complete mess and the files are spread over the complete repo
and over all the revisions (files were moved around in the old
subversion repo from folder to folder). Is there a possibility to say
take file1, file2, fileN and directory1, directoryN and move it to
another repo or remove all not matching files/directories completely?

Thanks a lot for some hints which commands I could use here or if it is
possible.

Bye
Matthias

Re: Switch from svn to git and modify repo completely

[Alexey Shumkin]

> Dear Git List,
> 
> I just started to migrate a svn repo to a git repo and was
> successfully with the "git svn" command and waiting a long time :)
> 
> I have now some confidential documents in the repository I must remove
> completely (including the complete history). These are single files
> spread over the complete repository. Is this possible with git?

take a look at "git filter-branch"

Re: Switch from svn to git and modify repo completely

[Matthias Fechner]

Am 19.11.2011 19:50, schrieb Alexey Shumkin:
> take a look at "git filter-branch"

thanks a lot for your answer. I checked the manual page, but it does not
work like I expected it or it is described. What I did:

1. I searched the path name with the command:
git show --pretty="format:" --name-only firstrev..endrev | sort | uniq
|grep sdcc

2. I tried to remove the file with the command:
git filter-branch -f --tree-filter 'rm -f
software/tools/net.sourceforge.eclipsesdcc-1.0.0-win32.x86.zip'
--prune-empty -- --all

git told me:
Ref 'refs/heads/master' was rewritten
Ref 'refs/remotes/origin/master' was rewritten
WARNING: Ref 'refs/remotes/origin/master' is unchanged

3. I cloned the repository with:
git clone --no-hardlinks repo-orig.git repo-filtered.git

4. After this I verified in the repo-filtered.git that the file was
really completely removed, but it was not the case. I was able to
checkout the revision which holds the file.

Regarding the manual page it should work... but reality is different.
Has here anyone an idea what I did wrong?

Bye
Matthias

-- 

"Programming today is a race between software engineers striving to
build bigger and better idiot-proof programs, and the universe trying to
produce bigger and better idiots. So far, the universe is winning." --
Rich Cook

Re: Switch from svn to git and modify repo completely

[Jonathan Nieder]

(re-populating cc list)
Hi Matthias,

Matthias Fechner wrote:

> git told me:
> Ref 'refs/heads/master' was rewritten
> Ref 'refs/remotes/origin/master' was rewritten
> WARNING: Ref 'refs/remotes/origin/master' is unchanged
>
> 3. I cloned the repository with:
> git clone --no-hardlinks repo-orig.git repo-filtered.git
>
> 4. After this I verified in the repo-filtered.git that the file was
> really completely removed, but it was not the case.

The section "CHECKLIST FOR SHRINKING A REPOSITORY" from the
git-filter-branch(1) manual page has some hints.  In particular, "git
clone --no-hardlinks" still _copies_ all objects --- you probably
would want "git clone file://$(pwd)/repo-orig" to make sure the
ordinary transfer negotiation kicks in.

It's very important that the documentation not be misleading, so if
you can point to places where the wording can be less confusing, that
would be very welcome.

Thanks and hope that helps,
Jonathan

Re: Switch from svn to git and modify repo completely

[Matthias Fechner]

Am 21.11.2011 08:02, schrieb Jonathan Nieder:
> The section "CHECKLIST FOR SHRINKING A REPOSITORY" from the
> git-filter-branch(1) manual page has some hints.  In particular, "git
> clone --no-hardlinks" still _copies_ all objects --- you probably
> would want "git clone file://$(pwd)/repo-orig" to make sure the
> ordinary transfer negotiation kicks in.
> 
> It's very important that the documentation not be misleading, so if
> you can point to places where the wording can be less confusing, that
> would be very welcome.

it would be fantastic if exactly this information is noted in the
manual. For me it is not really clear what is the difference between a
git clone with and without --no-hardlinks.

I was now able to convert my svn repo to a git repo and split the new
git repo in several different ones, including the complete history.

Here a short summary for all how need to do the same.
At first convert the svn into a git reposity:
git svn clone file:///path/to/svn -A authors -s gitrepo.git

To create the authors file, check the man page.

To get a list of all files:
git show --pretty="format:" --name-only startrev..endrev | sort | uniq

Maybe you want to remove some branches from the repo because svn users
used it in a completely wrong way:
git branch -rd badbranch

Now clone the repo and remove files from it you do not want anymore:
git clone gitrepo.git tofilter.git
cd tofilter.git
git filter-branch -f --tree-filter 'rm -Rf file1 file2.bla dir1 \
projekt1/dir2' --prune-empty -- --all
git gc
cd ..

Repeat this step for all combination you need.

Now create a bare repository and enable some options to use it as a
central repo:
git clone --bare tofilter.git tofilter_bare.git
cd tofilter_bare.git
git config core.sharedRepository 1
git config receive.denyNonFastForwards true
git gc
cd ..

Now you can copy the repo to the target server/directory and clone it
from there.

Bye
Matthias

-- 

"Programming today is a race between software engineers striving to
build bigger and better idiot-proof programs, and the universe trying to
produce bigger and better idiots. So far, the universe is winning." --
Rich Cook

Re: Switch from svn to git and modify repo completely

[Matthias Fechner]

Am 27.11.11 11:56, schrieb Matthias Fechner:
> git clone gitrepo.git tofilter.git

one additional note.
To get it really working you should execute:
git clone file://gitrepo.git tofilter.git

And later if you copy it to a bare repo:
git clone --bare file://tofiler.git barerepo.git

This will ensure that removed files are not in the cloned repository.

Bye,
Matthias

-- 
"Programming today is a race between software engineers striving to 
build bigger and better idiot-proof programs, and the universe trying to 
produce bigger and better idiots. So far, the universe is winning." -- 
Rich Cook