[RFC] git pull and importers

[RFC] git pull and importers

[Daniel Barkalow]

There's an SVN project I'm trying to track with git-svn. "git svn fetch" 
fetches and imports the commits perfectly, but I can't figure out a way to 
merge upstream commits into my branch automatically.

It seems like the right solution should be:

[remote "origin"]
	importer = svn
	url = svn://ixion.tartarus.org/main
	fetch = puzzles:refs/remotes/puzzles
[branch "master"]
	remote = origin
	merge = puzzles

Which would mean that it would use "git svn fetch" instead of "git fetch" 
for that remote, and "git svn fetch" would use that config section instead 
of its current config section.

Any reason this couldn't be made to work?

	-Daniel
*This .sig left intentionally blank*

Re: [RFC] git pull and importers

[Junio C Hamano]

Daniel Barkalow <barkalow@iabervon.org> writes:

> Which would mean that it would use "git svn fetch" instead of "git fetch" 
> for that remote, and "git svn fetch" would use that config section instead 
> of its current config section.
>
> Any reason this couldn't be made to work?

No reason.  Making git-$(scm)import to behave more like
git-fetch-pack backend was one of the longstanding items on my
wishlist.  I think "cvsimport -i" could mostly be usable in the
current shape for that, but I do not know about others.

Especially I do not interoperate with any SVN repositories
myself, so I cannot scratch your itch.

However, I suspect you could help me with a code I've been
struggling with on-and-off lately.  Do you still remember
unpack-trees code?

Re: [RFC] git pull and importers

[Daniel Barkalow]

On Sat, 7 Apr 2007, Junio C Hamano wrote:

> Daniel Barkalow <barkalow@iabervon.org> writes:
> 
> > Which would mean that it would use "git svn fetch" instead of "git fetch" 
> > for that remote, and "git svn fetch" would use that config section instead 
> > of its current config section.
> >
> > Any reason this couldn't be made to work?
> 
> No reason.  Making git-$(scm)import to behave more like
> git-fetch-pack backend was one of the longstanding items on my
> wishlist.  I think "cvsimport -i" could mostly be usable in the
> current shape for that, but I do not know about others.

I picked git-svn in part because it looks especially obvious; it even 
calls fetch "fetch". Not that that *really* counts for much, but I can 
pretend.

> Especially I do not interoperate with any SVN repositories
> myself, so I cannot scratch your itch.

I'll look into it, if the config option scheme looks reasonable to you.

> However, I suspect you could help me with a code I've been
> struggling with on-and-off lately.  Do you still remember
> unpack-trees code?

I'm not sure how similar it is these days to the code I worked on (which 
was called "read-tree", I think), but I'll take a look.

	-Daniel
*This .sig left intentionally blank*

Re: [RFC] git pull and importers

[Eric Wong]

Daniel Barkalow <barkalow@iabervon.org> wrote:
> There's an SVN project I'm trying to track with git-svn. "git svn fetch" 
> fetches and imports the commits perfectly, but I can't figure out a way to 
> merge upstream commits into my branch automatically.

I don't suggest using merge with git-svn since it can generate
non-linear history.  Non-linear history does not map well to SVN.

"git svn rebase" (in 1.5.1) is handy for getting upstream commits into
your branch (it's a wrapper around "git rebase")

> It seems like the right solution should be:
> 
> [remote "origin"]
> 	importer = svn
> 	url = svn://ixion.tartarus.org/main
> 	fetch = puzzles:refs/remotes/puzzles
> [branch "master"]
> 	remote = origin
> 	merge = puzzles

git-svn in 1.5.1 already allows you to define:

[svn-remote "svn"]
	url = svn://ixion.tartarus.org/main
	fetch = puzzles:refs/remotes/puzzles

And then "git svn rebase" should automatically be able to figure out to
rebase against refs/remotes/puzzles without needing a [branch "master"]
section.

I don't think having the "importer = svn" in [remote "..."] is a good
idea since it would be incompatible with older versions of git and the
documentation would confuse users who don't track the latest version.

> Which would mean that it would use "git svn fetch" instead of "git fetch" 
> for that remote, and "git svn fetch" would use that config section instead 
> of its current config section.

Here's what I'd like git-fetch to do someday[1]:

When git-fetch is called without any remote arguments, it would look for
[remote "origin"] as it does now.  However, if no [remote "..."]
sections are found (as is common with importer-created repos), it would
try other importers: [svn-remote "svn"], (and hopefully one day
[cvs-remote "cvs"], [arch-remote "arch"], ...).

Of course, git-fetch would also be able to handle --svn (and later
--cvs/--arch/...) flags if the [*remote "..."] section names are
ambiguous.

I intentionally named the default svn-remote section "svn" instead of
"origin" for this reason, too; I didn't want to confuse git-fetch.

[1] - patches would be very much welcome (*nudge*nudge*wink*wink),
      I have a lot on my plate and this isn't a high priority.

-- 
Eric Wong

Re: [RFC] git pull and importers

[Junio C Hamano]

Eric Wong <normalperson@yhbt.net> writes:

> When git-fetch is called without any remote arguments, it would look for
> [remote "origin"] as it does now.  However, if no [remote "..."]
> sections are found (as is common with importer-created repos), it would
> try other importers: [svn-remote "svn"], (and hopefully one day
> [cvs-remote "cvs"], [arch-remote "arch"], ...).

I wonder why wouldn't the alternative of noticing the URL scheme
of '[remote "svn"] url' variable is "svn://".  That is...

	[remote "gitrepo"]
        	url = git://example.com/repo.git/
		fetch = refs/heads/*:refs/remotes/gitrepo/*
	[remote "svnrepo"]
        	url = svn://example.com/repo.svn/
                fetch = trunk:refs/remotes/svnrepo/trunk
	[remote "cvsrepo"]
        	url = cvs://example.com/repo.cvs/
                fetch = HEAD:refs/remotes/cvsrepo/

The sections would probably can have importer specific options
besides usual url/fetch.

Re: [RFC] git pull and importers

[Eric Wong]

Junio C Hamano <junkio@cox.net> wrote:
> Eric Wong <normalperson@yhbt.net> writes:
> 
> > When git-fetch is called without any remote arguments, it would look for
> > [remote "origin"] as it does now.  However, if no [remote "..."]
> > sections are found (as is common with importer-created repos), it would
> > try other importers: [svn-remote "svn"], (and hopefully one day
> > [cvs-remote "cvs"], [arch-remote "arch"], ...).
> 
> I wonder why wouldn't the alternative of noticing the URL scheme
> of '[remote "svn"] url' variable is "svn://".  That is...

SVN repositories can also be http:// or https://, too.

-- 
Eric Wong

Re: [RFC] git pull and importers

[Daniel Barkalow]

On Sat, 7 Apr 2007, Eric Wong wrote:

> Daniel Barkalow <barkalow@iabervon.org> wrote:
> > There's an SVN project I'm trying to track with git-svn. "git svn fetch" 
> > fetches and imports the commits perfectly, but I can't figure out a way to 
> > merge upstream commits into my branch automatically.
> 
> I don't suggest using merge with git-svn since it can generate
> non-linear history.  Non-linear history does not map well to SVN.

That depends entirely on your merge strategy. In this case, I'm only 
really interested in fast-forwards, which can't create, well, anything. If 
I did any development on this particular project, I'd be emailing patch 
series, at least for now, so mapping back into SVN isn't a concern for me.

> "git svn rebase" (in 1.5.1) is handy for getting upstream commits into
> your branch (it's a wrapper around "git rebase")

Possibly "pull" should be able to invoke rebase instead of merge, also
(or merge should be able to actually use rebase). Handwaving, it would be 
good if a branch config could say that the history has to remain linear 
for some policy reason.

> > It seems like the right solution should be:
> > 
> > [remote "origin"]
> > 	importer = svn
> > 	url = svn://ixion.tartarus.org/main
> > 	fetch = puzzles:refs/remotes/puzzles
> > [branch "master"]
> > 	remote = origin
> > 	merge = puzzles
> 
> git-svn in 1.5.1 already allows you to define:
> 
> [svn-remote "svn"]
> 	url = svn://ixion.tartarus.org/main
> 	fetch = puzzles:refs/remotes/puzzles
> 
> And then "git svn rebase" should automatically be able to figure out to
> rebase against refs/remotes/puzzles without needing a [branch "master"]
> section.

That's what I've got right now.

> I don't think having the "importer = svn" in [remote "..."] is a good
> idea since it would be incompatible with older versions of git and the
> documentation would confuse users who don't track the latest version.

The current version of git just ignores it, which means that a regular 
"git fetch" fails, but actually somewhat less confusingly than if you 
don't have the section at all. (See below)

> > Which would mean that it would use "git svn fetch" instead of "git fetch" 
> > for that remote, and "git svn fetch" would use that config section instead 
> > of its current config section.
> 
> Here's what I'd like git-fetch to do someday[1]:
> 
> When git-fetch is called without any remote arguments, it would look for
> [remote "origin"] as it does now.  However, if no [remote "..."]
> sections are found (as is common with importer-created repos), it would
> try other importers: [svn-remote "svn"], (and hopefully one day
> [cvs-remote "cvs"], [arch-remote "arch"], ...).
> 
> Of course, git-fetch would also be able to handle --svn (and later
> --cvs/--arch/...) flags if the [*remote "..."] section names are
> ambiguous.
> 
> I intentionally named the default svn-remote section "svn" instead of
> "origin" for this reason, too; I didn't want to confuse git-fetch.

git-fetch is confused worse by getting invoked without any remotes than by 
getting a URL that doesn't work. I think it's trying to fetch from 
"./origin/.git" effectively. At least if you've got a section with a URL 
that doesn't have a git repository at it, it tells you it couldn't find a 
git repository at the svn URL (or it tells you it doesn't understand 
svn://...), at which point you know what's wrong.

For future versions, it would be just as easy to have a setting to tell it 
what interfacing thing it needs in order to handle a particular remote.

But I could see it being helpful to be able to have a ton of different 
remotes, some of which are through importers, where you don't much care 
how you get the content. E.g., Linus could just do "git pull alsa", to 
import from ALSA's repository, ignoring the fact that it's hg (and used 
to be cvs). It would definitely be helpful for cases where changes only 
come from the foriegn SCM (or go back as patches). So, at the very least, 
you should be able to use:

[hg-remote "alsa"]

Also, subprojects in the "vast superproject" model would often have 
import-only links to upstream repositories in other systems, as a backbone 
for non-linear local development.

> [1] - patches would be very much welcome (*nudge*nudge*wink*wink),
>       I have a lot on my plate and this isn't a high priority.

I've been poking at the git-fetch stuff some. Certainly doing better than 
looking at git-svn, since I don't know perl...

	-Daniel
*This .sig left intentionally blank*