How to use git-svn to clone from a mirror?

How to use git-svn to clone from a mirror?

[David Kastrup]

Hi, I seem to be too stupid to get this right.

I have used something like

git-svn clone -T trunk -b branches -t tags file:///tmp/rsync-mirror

to clone an rsync mirror of an SVN repository.  Now I want to have
fetch revert to pulling from the upstream repository in future.
However, if I change the respective line in .git/config to
svn://the.svn.link/whatever, git-rebase will fetch the right updates,
but then says that it can't work with the objects in the git
repository.

Changing the config back will make git-rebase -l work.

So what would be the right procedure to shift the SVN source from an
rsync mirror to the original, without git-svn breaking?

-- 
David Kastrup

Re: How to use git-svn to clone from a mirror?

[Raja R Harinath]

Hi,

David Kastrup <dak@gnu.org> writes:

> Hi, I seem to be too stupid to get this right.
>
> I have used something like
>
> git-svn clone -T trunk -b branches -t tags file:///tmp/rsync-mirror
>
> to clone an rsync mirror of an SVN repository.  Now I want to have
> fetch revert to pulling from the upstream repository in future.
> However, if I change the respective line in .git/config to
> svn://the.svn.link/whatever, git-rebase will fetch the right updates,
> but then says that it can't work with the objects in the git
> repository.
>
> Changing the config back will make git-rebase -l work.
>
> So what would be the right procedure to shift the SVN source from an
> rsync mirror to the original, without git-svn breaking?

I think you'll have to

-------------8<------------
  # remove stored revision db, since we're going to change all the commit ids
  rm .git/svn/git-svn/.rev_db.*

  # rewrite git-svn-id: lines
  cg-admin-rewritehist \
	--msg-filter \
	'sed "s,file:///tmp/rsync-mirror,svn://the.svn.link/whatever,"'

  # recreate new revision db, and fetch updates, if any
  git-svn rebase
-------------8<------------

- Hari

Does git-svn-id need repository path? (Re: How to use git-svn to clone from a mirror?)

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 816 bytes --]

On Mon, Jun 04, 2007 at 20:41:29 +0530, Raja R Harinath wrote:
> -------------8<------------
>   # remove stored revision db, since we're going to change all the commit ids
>   rm .git/svn/git-svn/.rev_db.*
> 
>   # rewrite git-svn-id: lines
>   cg-admin-rewritehist \
> 	--msg-filter \
> 	'sed "s,file:///tmp/rsync-mirror,svn://the.svn.link/whatever,"'
> 
>   # recreate new revision db, and fetch updates, if any
>   git-svn rebase
> -------------8<------------

Which leads me to a question. Should the url really be part of the
git-svn-id? A subversion repository is uniquely identified by it's uuid, so
the uuid + revno is a good identifier. Is it possible to change that now, or
is it necessary to keep the format now for backward compatibility?

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: How to use git-svn to clone from a mirror?

[Johannes Schindelin]

Hi,

On Mon, 4 Jun 2007, Raja R Harinath wrote:

> David Kastrup <dak@gnu.org> writes:
> 
> > I have used something like
> >
> > git-svn clone -T trunk -b branches -t tags file:///tmp/rsync-mirror
> >
> > to clone an rsync mirror of an SVN repository.  Now I want to have 
> > fetch revert to pulling from the upstream repository in future. 
> > However, if I change the respective line in .git/config to 
> > svn://the.svn.link/whatever, git-rebase will fetch the right updates, 
> > but then says that it can't work with the objects in the git 
> > repository.
> >
> > Changing the config back will make git-rebase -l work.
> >
> > So what would be the right procedure to shift the SVN source from an
> > rsync mirror to the original, without git-svn breaking?
> 
> I think you'll have to
> 
> -------------8<------------
>   # remove stored revision db, since we're going to change all the commit ids
>   rm .git/svn/git-svn/.rev_db.*
> 
>   # rewrite git-svn-id: lines
>   cg-admin-rewritehist \
> 	--msg-filter \
> 	'sed "s,file:///tmp/rsync-mirror,svn://the.svn.link/whatever,"'
> 
>   # recreate new revision db, and fetch updates, if any
>   git-svn rebase
> -------------8<------------

<shameless plug>
	Or you use the just-rewritten version of it, git-filter-branch.
</shameless>

Ciao,
Dscho

Re: How to use git-svn to clone from a mirror?

[David Kastrup]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> On Mon, 4 Jun 2007, Raja R Harinath wrote:
>
>> David Kastrup <dak@gnu.org> writes:
>> 
>> > I have used something like
>> >
>> > git-svn clone -T trunk -b branches -t tags file:///tmp/rsync-mirror
>> >
>> > to clone an rsync mirror of an SVN repository.  Now I want to have 
>> > fetch revert to pulling from the upstream repository in future. 
>> > However, if I change the respective line in .git/config to 
>> > svn://the.svn.link/whatever, git-rebase will fetch the right updates, 
>> > but then says that it can't work with the objects in the git 
>> > repository.
>> >
>> > Changing the config back will make git-rebase -l work.
>> >
>> > So what would be the right procedure to shift the SVN source from an
>> > rsync mirror to the original, without git-svn breaking?
>> 
>> I think you'll have to
>> 
>> -------------8<------------
>>   # remove stored revision db, since we're going to change all the commit ids
>>   rm .git/svn/git-svn/.rev_db.*
>> 
>>   # rewrite git-svn-id: lines
>>   cg-admin-rewritehist \
>> 	--msg-filter \
>> 	'sed "s,file:///tmp/rsync-mirror,svn://the.svn.link/whatever,"'
>> 
>>   # recreate new revision db, and fetch updates, if any
>>   git-svn rebase
>> -------------8<------------
>
> <shameless plug>
> 	Or you use the just-rewritten version of it, git-filter-branch.
> </shameless>

Well, part of the reason I worked from an rsynced copy was to be able
to repeat the experiment by just wasting a few hours of time each
time, without wasting more bandwidth.

What I arrived at was to use
git-svn init -T trunk -t tags -b branches
  --rewrite-root svn://tug.org/texlive file:///mirror/texlive
git-svn fetch --all
[edit .git/config and replace the url and rewrite-root lines with a
 single url line pointing to the root]
git-reset --hard   [don't ask me why]
and afterwards fetches worked online.

I liked the commit messages when using --no-metadata better than with
--rewrite-root, but I found no way to get the resulting archive
operative for git-svn rebase afterwards.

Could someone explain to me why git needs to know the upstream URL
history, whether by --rewrite-root or rewrite-hist or
git-filter-branch?

I find this rather hard to understand, so I would like to get an idea
where this fits naturally into the overall design of git, and how it
makes sense.

Thanks,

-- 
David Kastrup

Re: How to use git-svn to clone from a mirror?

[Eric Wong]

David Kastrup <dak@gnu.org> wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > On Mon, 4 Jun 2007, Raja R Harinath wrote:
> >
> >> David Kastrup <dak@gnu.org> writes:
> >> 
> >> > I have used something like
> >> >
> >> > git-svn clone -T trunk -b branches -t tags file:///tmp/rsync-mirror
> >> >
> >> > to clone an rsync mirror of an SVN repository.  Now I want to have 
> >> > fetch revert to pulling from the upstream repository in future. 
> >> > However, if I change the respective line in .git/config to 
> >> > svn://the.svn.link/whatever, git-rebase will fetch the right updates, 
> >> > but then says that it can't work with the objects in the git 
> >> > repository.
> >> >
> >> > Changing the config back will make git-rebase -l work.
> >> >
> >> > So what would be the right procedure to shift the SVN source from an
> >> > rsync mirror to the original, without git-svn breaking?
> >> 
> >> I think you'll have to
> >> 
> >> -------------8<------------
> >>   # remove stored revision db, since we're going to change all the commit ids
> >>   rm .git/svn/git-svn/.rev_db.*
> >> 
> >>   # rewrite git-svn-id: lines
> >>   cg-admin-rewritehist \
> >> 	--msg-filter \
> >> 	'sed "s,file:///tmp/rsync-mirror,svn://the.svn.link/whatever,"'
> >> 
> >>   # recreate new revision db, and fetch updates, if any
> >>   git-svn rebase
> >> -------------8<------------
> >
> > <shameless plug>
> > 	Or you use the just-rewritten version of it, git-filter-branch.
> > </shameless>
> 
> Well, part of the reason I worked from an rsynced copy was to be able
> to repeat the experiment by just wasting a few hours of time each
> time, without wasting more bandwidth.
> 
> What I arrived at was to use
> git-svn init -T trunk -t tags -b branches
>   --rewrite-root svn://tug.org/texlive file:///mirror/texlive
> git-svn fetch --all
> [edit .git/config and replace the url and rewrite-root lines with a
>  single url line pointing to the root]
> git-reset --hard   [don't ask me why]
> and afterwards fetches worked online.

Yes, sorry for the late response, but --rewrite-root was written
with this purpose in mind.

> I liked the commit messages when using --no-metadata better than with
> --rewrite-root, but I found no way to get the resulting archive
> operative for git-svn rebase afterwards.

You should be able to use git-svn fetch and then plain git-rebase <remote>

> Could someone explain to me why git needs to know the upstream URL
> history, whether by --rewrite-root or rewrite-hist or
> git-filter-branch?
> 
> I find this rather hard to understand, so I would like to get an idea
> where this fits naturally into the overall design of git, and how it
> makes sense.

Having the git-svn-id: lines in commits allows commits to be made to the
upstream SVN repository more easily and without user interaction or
configuraton.  I put git-svn-id: in the commit objects themselves
because they're immutable and robust.

The URL portion of git-svn-id: is useful when I'm using many throwaway
branches and I've forgotten which upstream (SVN) branch I need to
dcommit against, and git-svn can easily figure it out for me
without needing to remember to use git-checkout --track on my part
or memorization.

It's possible to clone the refs and objects over to a new repository
over the native git:// protocol and have it fully usable from git-svn
without needing additional setup or having to copy the .git/config or
.git/svn (neither of which are transferred over the git protocol).
Cloning the refs is a bit of a pain these days because of the "remotes/"
convention in the name, but still possible.

The .rev_db files in .git/svn can get corrupted or deleted, and since
they're not managed by git, they're next to impossible to recover
if the git-svn-id: lines didn't exist.

I'll be the first to admit that the git-svn-id: lines are ugly, but
that's why I wrote "git svn log" :)

-- 
Eric Wong