[RFC PATCH 2/3] Add specification of git-vcs helpers

[RFC PATCH 2/3] Add specification of git-vcs helpers

[Daniel Barkalow]

So far, I've actually used list and import; I've also implemented an 
export, but haven't figured out exactly how the user should cause it to be 
used.

Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
---
 Documentation/git-vcs.txt |   93 +++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 93 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/git-vcs.txt

diff --git a/Documentation/git-vcs.txt b/Documentation/git-vcs.txt
new file mode 100644
index 0000000..fa02b76
--- /dev/null
+++ b/Documentation/git-vcs.txt
@@ -0,0 +1,93 @@
+git-vcs-*(1)
+============
+
+NAME
+----
+git-vcs-* - Helper programs for interoperation with foreign systems
+
+SYNOPSIS
+--------
+'git vcs-<system>' <command> [options] [arguments]
+
+DESCRIPTION
+-----------
+
+These programs are normally not used directly by end users, but are
+invoked by various git programs that interact with remote repositories
+when the repository they would operate on is part of a foreign
+system.
+
+Each 'git vcs-<system>' is a helper for interoperating with a
+particular version control system. Different helpers have different
+capabilities (limited both be the particular helper and by the
+capabilities of the system they connect to), and they report what
+capabilities they support.
+
+In general, these programs interact with a portion of the refs
+namespace that isn't used by the rest of git. The configuration will
+then (generally) map these refs into the remotes namespace. This
+allows the importer to do whatever it wants with its refs without
+affecting the state visible to normal git programs.
+
+COMMANDS
+--------
+
+'capabilities'::
+	Prints the capabilities of the helper, one per line. These are:
+	 - import: the basic import command
+	 - marks: import should be done with a saved marks file
+	 - find-new-branches: detect new branches
+	 - export: the general export command
+	 - fork: create a new branch and export to it
+	 - anonymous-fork: make commits on a branch without an inherent name
+	 - merge: merge branches (of whatever type the system supports)
+
+	If the helper doesn't support "merge", the default for pull is
+	to rebase instead of merging.
+
+'list'::
+	Takes the remote name, and outputs the names of refs. These
+	may be followed, after a single space, by "changed" or
+	"unchanged", indicating whether the foreign repository has
+	changed from the state in the ref. If the helper doesn't know,
+	it doesn't have to provide a value. (In particular, it
+	shouldn't do expensive operations, such as importing the
+	content, to see whether it matches.)
+
+'import'::
+	Takes the remote name and a list of names of refs, and imports
+	whatever it describes, by outputting it in git-fast-import
+	format.
+
+'export'::
+	Sends the branch to the foreign system and reimports it in
+	fast-import format.
+
+	Reads a list of commits from stdin, where each commit has no
+	parents which were neither produced by an earlier import nor
+	appearing earlier in the list, where some commit has the old
+	value of the branch as a parent, and where all commits listed
+	are ancestors of the last one. Furthermore:
+
+	 - if the system doesn't support merges, each of these commits
+	   has only a single parent;
+
+	 - if the system doesn't support anonymous branches, the first
+	   commit has the old value of the branch as a parent (if the
+	   branch already had a value), and all parents are either the
+	   commit listed immediately before or produced by an earlier
+	   import;
+
+	 - if the system doesn't support many-way merges, each commit
+	   has at most two parents.
+
+	export produces output in fast-import format giving the
+	content after a round-trip through the foreign system. This
+	also contains extra headers to report the mapping of original
+	git commits to reimported git commits (to facilitate rewriting
+	local branches to use the history-as-reimported instead of the
+	git-only version).
+
+	export reports how much it managed to export by producing
+	commits in the fast-import stream that replace the listed
+	items that were successfully exported.
-- 
1.6.0.6

Re: [RFC PATCH 2/3] Add specification of git-vcs helpers

[Junio C Hamano]

Daniel Barkalow <barkalow@iabervon.org> writes:

> So far, I've actually used list and import; I've also implemented an 
> export, but haven't figured out exactly how the user should cause it to be 
> used.
> ...
> +'capabilities'::
> +	Prints the capabilities of the helper, one per line. These are:
> +	 - import: the basic import command
> +	 - marks: import should be done with a saved marks file
> +	 - find-new-branches: detect new branches
> +	 - export: the general export command
> +	 - fork: create a new branch and export to it
> +	 - anonymous-fork: make commits on a branch without an inherent name
> +	 - merge: merge branches (of whatever type the system supports)
> +
> +	If the helper doesn't support "merge", the default for pull is
> +	to rebase instead of merging.

Have you tried formatting this to both html and man?  I think you need to
dedent the second paragraph, and have '+' as the sole character on a line
immediately before, without any blank lines.  The formatted output of the
description of 'export' is probably more troublesome as it is much longer.

This list feels like overengineered yet without enough thinking behind it.

Your 'list' allows to list what can become branches on the git end, so as
long as you have tracking information on the git side, find-new-branches
seems unnecessary, for examle.  Worse, find-new-branches needs its own
tracking mechansim anyway, but that means how the namespace for remote
tracking branches is managed is left to the vcs backends, even though that
namespace is directly visiblt to git --- my gut feeling is that it would
probably better to arrange things similar to what git native transport
does already.  'list' would say "The remote has what can become refs on
the git end and calls them A, B, C", and the built-in code, using ref
mapping specification, maps them to refs/remotes/somename/{A,B,C}.  You do
it again and if you see the remote has D but does not have B anymore, you
know D is a new branch and B was deleted.

What does 'merge' mean?  It cannot mean to fetch their changes from
foreign vcs and create a merge locally (you only need import from the
backend, and merge will be a usual git merge).  Perhaps you meant if you
can export a merge back?  Some foreign systems may support importing
merges but not octopus, so it may not be just a black-or-white boolean.

Also "the default is to rebase instead of merge" is probably too weak.
You would want to error out if the user tried to merge, wouldn't you?

> +'list'::
> +	Takes the remote name, and outputs the names of refs. These
> +	may be followed, after a single space, by "changed" or
> +	"unchanged", indicating whether the foreign repository has
> +	changed from the state in the ref. If the helper doesn't know,
> +	it doesn't have to provide a value. (In particular, it
> +	shouldn't do expensive operations, such as importing the
> +	content, to see whether it matches.)

I am guessing by 'a value' you mean 'changed/unchanged', iow you are not
saying if the helper does not know it can omit such refs from the list,
but it is unclear.

Re: [RFC PATCH 2/3] Add specification of git-vcs helpers

[Daniel Barkalow]

On Sun, 11 Jan 2009, Junio C Hamano wrote:

> Daniel Barkalow <barkalow@iabervon.org> writes:
> 
> > So far, I've actually used list and import; I've also implemented an 
> > export, but haven't figured out exactly how the user should cause it to be 
> > used.
> > ...
> > +'capabilities'::
> > +	Prints the capabilities of the helper, one per line. These are:
> > +	 - import: the basic import command
> > +	 - marks: import should be done with a saved marks file
> > +	 - find-new-branches: detect new branches
> > +	 - export: the general export command
> > +	 - fork: create a new branch and export to it
> > +	 - anonymous-fork: make commits on a branch without an inherent name
> > +	 - merge: merge branches (of whatever type the system supports)
> > +
> > +	If the helper doesn't support "merge", the default for pull is
> > +	to rebase instead of merging.
> 
> Have you tried formatting this to both html and man?  I think you need to
> dedent the second paragraph, and have '+' as the sole character on a line
> immediately before, without any blank lines.  The formatted output of the
> description of 'export' is probably more troublesome as it is much longer.

Nope, I've just been referring to it in the original text, so I've been 
careless with markup.

> This list feels like overengineered yet without enough thinking behind it.

Yeah, I haven't really nailed this one down; it's hard to get really right 
without having a bunch of different helpers which can do different amounts 
of stuff and need different git-side help.

> Your 'list' allows to list what can become branches on the git end, so as
> long as you have tracking information on the git side, find-new-branches
> seems unnecessary, for examle.

I was unclear about what that one meant, I guess. It's supposed to handle 
systems where it's possible to create things that work like branches but 
are hard to find. If a helper doesn't report the capability, then the user 
may need to say, "I also want some/branch/location", or the helper 
wouldn't know that's a branch.

For example, in order to find all of the branches in Perforce, you have 
to figure out both directions of integration, in order to find branches 
that haven't been integrated back into the location you know about, and 
that's kind of expensive to determine and a bit tricky to parse.

It's not intended to indicate that the helper will tell you which ones are 
new or not, just that, if the helper doesn't have it, it will only tell 
you about branches that it's been told about.

> What does 'merge' mean?  It cannot mean to fetch their changes from
> foreign vcs and create a merge locally (you only need import from the
> backend, and merge will be a usual git merge).  Perhaps you meant if you
> can export a merge back?  Some foreign systems may support importing
> merges but not octopus, so it may not be just a black-or-white boolean.

If the helper supports "merge", it means that it is able to create a merge 
in the foreign vcs. You're right about needing to separately list an 
"octopus" capability, and maybe further refinements.

> Also "the default is to rebase instead of merge" is probably too weak.
> You would want to error out if the user tried to merge, wouldn't you?

I think users should be able to create merges in git if they want to, but 
they'd have to linearize the history in order to push it back. Hopefully, 
at some point, we'll have a tool to help with this (which would be helpful 
for generating patch series, anyway).

> > +'list'::
> > +	Takes the remote name, and outputs the names of refs. These
> > +	may be followed, after a single space, by "changed" or
> > +	"unchanged", indicating whether the foreign repository has
> > +	changed from the state in the ref. If the helper doesn't know,
> > +	it doesn't have to provide a value. (In particular, it
> > +	shouldn't do expensive operations, such as importing the
> > +	content, to see whether it matches.)
> 
> I am guessing by 'a value' you mean 'changed/unchanged', iow you are not
> saying if the helper does not know it can omit such refs from the list,
> but it is unclear.

Yeah, that's what I meant.

	-Daniel
*This .sig left intentionally blank*

Re: [RFC PATCH 2/3] Add specification of git-vcs helpers

[Johannes Sixt]

Daniel Barkalow schrieb:
> +NAME
> +----
> +git-vcs-* - Helper programs for interoperation with foreign systems
> +
> +SYNOPSIS
> +--------
> +'git vcs-<system>' <command> [options] [arguments]

The current rule is that helper programs have double-dash in the name:
git-rebase--interactive, git-web--browse, etc.

-- Hannes

Re: [RFC PATCH 2/3] Add specification of git-vcs helpers

[Daniel Barkalow]

On Mon, 12 Jan 2009, Johannes Sixt wrote:

> Daniel Barkalow schrieb:
> > +NAME
> > +----
> > +git-vcs-* - Helper programs for interoperation with foreign systems
> > +
> > +SYNOPSIS
> > +--------
> > +'git vcs-<system>' <command> [options] [arguments]
> 
> The current rule is that helper programs have double-dash in the name:
> git-rebase--interactive, git-web--browse, etc.

I'm not entirely sure if these are helper programs in that sense, or 
really what the distinction is between these and regular plumbing. 
Multiple programs will be calling them, and they'll have a specified API, 
in any case. (I think the double-dash programs are ones where all of the 
call sites are internal, so both the API and all the users can be changed 
together.)

	-Daniel
*This .sig left intentionally blank*