[PATCH] Third try at documenting command integration requirements.

[PATCH] Third try at documenting command integration requirements.

[Eric S. Raymond]

This document contains no new policies or proposals; it attempts
to document established practices and interface requirements.

Signed-off-by: Eric S. Raymond <esr@thyrsus.com>
---
 Documentation/technical/api-command.txt |   91 +++++++++++++++++++++++++++++++
 1 file changed, 91 insertions(+)
 create mode 100644 Documentation/technical/api-command.txt

diff --git a/Documentation/technical/api-command.txt b/Documentation/technical/api-command.txt
new file mode 100644
index 0000000..c1c1afb
--- /dev/null
+++ b/Documentation/technical/api-command.txt
@@ -0,0 +1,91 @@
+= Integrating new subcommands =
+
+This is how-to documentation for people who want to add extension
+commands to git.  It should be read alongside api-builtin.txt.
+
+== Runtime environment ==
+
+git subcommands are standalone executables that live in the git
+execution directory, normally /usr/lib/git-core.  The git executable itself
+is a thin wrapper that sets GIT_DIR and passes command-line arguments
+to the subcommand.
+
+(If "git foo" is not found in the git exec path, the wrapper
+will look in the rest of your $PATH for it.  Thus, it's possible
+to write local git extensions that don't live in system space.)
+
+== Implementation languages ==
+
+Most subcommands are written in C or shell.  A few are written in
+Perl.  A tiny minority are written in Python.
+
+While we strongly encourage coding in portable C for portability, these
+specific scripting languages are also acceptable. We won't accept more
+without a very strong technical case, as we don't want to broaden the
+git suite's required dependencies.
+
+Python is fine for import utilities, surgical tools, remote helpers
+and other code at the edges of the git suite - but it should not yet
+be used for core functions. This may change in the future; the problem
+is that we need better Python integration in the git Windows installer
+before we can be confident people in that environment won't
+experience an unacceptably large loss of capability.
+
+C commands are normally written as single modules, named after the
+command, that link a collection of functions called libgit.  Thus,
+your command 'git-foo' would normally be implemented as a single
+"git-foo.c"; this organization makes it easy for people reading the
+code to find things.
+
+See the CodingGuidelines document for other guidance on what we consider
+good practice in C and shell, and api-builtin.txt for the support
+functions available to built-in commands written in C.
+
+== What every extension command needs ==
+
+You must have a man page, written in asciidoc (this is what git help
+followed by your subcommand name will display).  Be aware that there is
+a local asciidoc configuration and macros which you should use.  It's
+often helpful to start by cloning an existing page and replacing the
+text content.
+
+You must have a test, written to report in TAP (Test Anything Protocol).
+Tests are executables (usually shell scripts) that live in the 't' 
+subdirectory of the tree.  Each test name begins with 't' and a sequence
+number that controls where in the test sequence it will be executed;
+conventionally the rest of the name stem is that of the command 
+being tested.
+
+Read the file t/README to learn more about the conventions to be used
+in writing tests, and the test support library.
+
+== Integrating a command ==
+
+Here are the things you need to do when you want to merge a new 
+subcommand into the git tree.
+
+0. Don't forget to sign off your patch!
+
+1. Append your command name to one of the variables BUILTIN_OBJS,
+EXTRA_PROGRAMS, SCRIPT_SH, SCRIPT_PERL or SCRIPT_PYTHON.
+
+2. Drop its test in the t directory.
+
+3. If your command is implemented in an interpreted language with a 
+p-code intermediate form, make sure .gitignore in the main directory
+includes a pattern entry that ignores such files.  Python .pyc and
+.pyo files will already be covered.
+
+4. If your command has any dependency on a a particular version of
+your language, document it in the INSTALL file.
+
+5. There is a file command-list.txt in the distribution main directory
+that categorizes commands by type, so they can be listed in appropriate
+subsections in the documentation's summary command list.  Add an entry 
+for yours.  To understand the categories, look at git-cmmands.txt
+in the main directory.
+
+6. When your patch is merged, remind the maintainer to add something
+about it in the RelNotes file.
+
+That's all there is to it.
-- 
1.7.9.5

-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Whether the authorities be invaders or merely local tyrants, the
effect of such [gun control] laws is to place the individual at the 
mercy of the state, unable to resist.
        -- Robert Anson Heinlein, 1949

Re: [PATCH] Third try at documenting command integration requirements.

[Perry Hutchison]

esr@thyrsus.com (Eric S. Raymond) wrote:

> This document contains no new policies or proposals; it attempts
> to document established practices and interface requirements.
...
> +4. If your command has any dependency on a a particular version of
                                            ^^^

typo.  (granted this is an extreme nit)

Re: [PATCH] Third try at documenting command integration requirements.

[Junio C Hamano]

esr@thyrsus.com (Eric S. Raymond) writes:

> This document contains no new policies or proposals; it attempts
> to document established practices and interface requirements.
>
> Signed-off-by: Eric S. Raymond <esr@thyrsus.com>

I'll reword the title (readers of "git log" output 6 months down the
road will not care if this is the third try or the first one) and
tweak things here and there before queuing.

> diff --git a/Documentation/technical/api-command.txt b/Documentation/technical/api-command.txt
> new file mode 100644
> index 0000000..c1c1afb
> --- /dev/null
> +++ b/Documentation/technical/api-command.txt
> @@ -0,0 +1,91 @@
> += Integrating new subcommands =
> +
> +This is how-to documentation for people who want to add extension
> +commands to git.  It should be read alongside api-builtin.txt.
> +
> +== Runtime environment ==
> +
> +git subcommands are standalone executables that live in the git
> +execution directory, normally /usr/lib/git-core.  The git executable itself
> +is a thin wrapper that sets GIT_DIR and passes command-line arguments
> +to the subcommand.

    $ echo >$HOME/bin/git-showenv '#!/bin/sh
    exec env'
    $ chmod +x $HOME/bin/git-showenv
    $ git showenv | grep GIT_

gives me emptyness.  I rewrote the above to:

    git subcommands are standalone executables that live in the git exec
    path, normally /usr/lib/git-core.  The git executable itself is a
    thin wrapper that knows where the subcommands live, and runs them by
    passing command-line arguments to them.

FYI, a builtin command _can_ ask the git wrapper to set up the
execution environment by setting RUN_SETUP bit in its cmd_struct
entry, but it is not done by default.

> +== Implementation languages ==
> +
> +Most subcommands are written in C or shell.  A few are written in
> +Perl.  A tiny minority are written in Python.
> +
> +While we strongly encourage coding in portable C for portability, these
> +specific scripting languages are also acceptable. We won't accept more
> +without a very strong technical case, as we don't want to broaden the
> +git suite's required dependencies.
> +
> +Python is fine for import utilities, surgical tools, remote helpers
> +and other code at the edges of the git suite - but it should not yet
> +be used for core functions. This may change in the future; the problem
> +is that we need better Python integration in the git Windows installer
> +before we can be confident people in that environment won't
> +experience an unacceptably large loss of capability.

As Felipe and others said in the discussion, Python is not *that*
special over other languages (and I think we have a Go in contrib/).

I rewrote the above to:

    Most subcommands are written in C or shell.  A few are written in
    Perl.

    While we strongly encourage coding in portable C for portability,
    these specific scripting languages are also acceptable.  We won't
    accept more without a very strong technical case, as we don't want
    to broaden the git suite's required dependencies.  Import utilities,
    surgical tools, remote helpers and other code at the edges of the
    git suite are more lenient and we allow Python (and even Tcl/tk),
    but they should not be used for core functions.

    This may change in the future.  Especially Python is not allowed in
    core because we need better Python integration in the git Windows
    installer before we can be confident people in that environment
    won't experience an unacceptably large loss of capability.

> +C commands are normally written as single modules, named after the
> +command, that link a collection of functions called libgit.  Thus,
> +your command 'git-foo' would normally be implemented as a single
> +"git-foo.c"; this organization makes it easy for people reading the

    "git-foo.c" (or "builtin/foo.c" if it is to be linked to the main
    binary);

> +4. If your command has any dependency on a a particular version of
> +your language, document it in the INSTALL file.

    s/a a/a/;

> +6. When your patch is merged, remind the maintainer to add something
> +about it in the RelNotes file.

    6. Give the maintainer a one paragraph to include in the RelNotes
    file to describe the new feature; a good place to do so is in the
    cover letter [PATCH 0/n].

Thanks.

Re: [PATCH] Third try at documenting command integration requirements.

[Eric S. Raymond]

Junio C Hamano <gitster@pobox.com>:
> I'll reword the title (readers of "git log" output 6 months down the
> road will not care if this is the third try or the first one) and
> tweak things here and there before queuing.

Result looks good from here.
 
The next things on my git to-do list are 

1. Audit the in-tree Python for version dependencies.  Add floor-version checks.

2. Submit a doc patch containing guidelines that (a) Python scripts should
   check for their floor version and error out gracefully if they won't
   run with the host's interpreter, and (b) Python scripts sbould be
   2.6-compatible.

3. Submit the git-weave integration patch.  I could do that now, but while my
   regression test speaks TAP it doesn't presently use the test library. I plan
   to re-work it to do that.

Do you have any other pending tasks for which you think my expertise would
be useful?  I refer specifically to the facts that (a) I find writing and 
editing documentation easy and can do it rapidly, (b) I'm a Python expert, 
and (c) I am very interested in, and know a lot about, tools for repository
surgery and import/export.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: [PATCH] Third try at documenting command integration requirements.

[Michael Haggerty]

On 11/26/2012 10:41 PM, Eric S. Raymond wrote:
> The next things on my git to-do list are 
> [...]
> 2. Submit a doc patch containing guidelines that (a) Python scripts should
>    check for their floor version and error out gracefully if they won't
>    run with the host's interpreter, and (b) Python scripts sbould be
>    2.6-compatible.

OK, now let's discuss *which* minimum Python version that git should
support in the hypothetical new world...

Data point: Mercurial supports Python 2.4 - 2.7 with the following
explanation [1]:

    We will continue to support Python 2.4 as long as it doesn't
    present a significant barrier to development. Given that Python 2.5
    and later don't contain any features that we're dying to use, that
    may be a long time off. [...]

    We also will continue to support Python 2.x as long as there is a
    significant installed base in the form of Red Hat Enterprise Linux
    and Ubuntu LTS users. RHEL 5, which uses Python 2.4, will reach the
    end of the "production 2" portion of its lifecycle in Q1 2014 and
    the end of its regular lifecycle in 2017.

It would be a shame to leave RHEL 5 users behind if Python is used to
implement important git functionality.  Python 2.4 is missing some of
Python's shiny new features, but still quite OK.  What features would
you miss the most if we were to target Python 2.4 instead of 2.6?

Michael

[1] http://mercurial.selenic.com/wiki/SupportedPythonVersions

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

Re: [PATCH] Third try at documenting command integration requirements.

[Eric S. Raymond]

Michael Haggerty <mhagger@alum.mit.edu>:
> OK, now let's discuss *which* minimum Python version that git should
> support in the hypothetical new world...

By all means!
 
> It would be a shame to leave RHEL 5 users behind if Python is used to
> implement important git functionality.  Python 2.4 is missing some of
> Python's shiny new features, but still quite OK.  What features would
> you miss the most if we were to target Python 2.4 instead of 2.6?

Off the top of my head...the 'with' statement, the conditional
expression, and built-in JSON support.  Other developers would be
likely to kick about the string format() method; personally I'm
cheerfully old-school about that.

I agree that 2.4 is still quite OK.  I'm a little concerned that dropping that
far back might store up some transition problems for the day we decide to
make the jump to Python 3.

On the other hand, I think gating features on RHEL5 might be
excessively cautious.  According to [1], RHEL will red-zone within 30
days if it hasn't done so already ([1] says "Q4").  And RHEL6 (with
Python 2.6) has been shipping for two years.

Policy suggestion: we aim to stay friendly for every version of RHEL that
is still in Support 1.  I doubt anyone will code anything critical 
in Python before Dec 31st - I'm certainly not planning to!

[1] http://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux RHEL5 is going
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: [PATCH] Third try at documenting command integration requirements.

[Junio C Hamano]

"Eric S. Raymond" <esr@thyrsus.com> writes:

> I agree that 2.4 is still quite OK.  I'm a little concerned that dropping that
> far back might store up some transition problems for the day we decide to
> make the jump to Python 3.
>
> On the other hand, I think gating features on RHEL5 might be
> excessively cautious.  According to [1], RHEL will red-zone within 30
> days if it hasn't done so already ([1] says "Q4").  And RHEL6 (with
> Python 2.6) has been shipping for two years.

I won't worry about Python 3 yet; in what timeframe did Python's
i18n/unicode support become usable?  In 2.4, or 2.6?

Re: [PATCH] Third try at documenting command integration requirements.

[Eric S. Raymond]

Junio C Hamano <gitster@pobox.com>:
> I won't worry about Python 3 yet; in what timeframe did Python's
> i18n/unicode support become usable?  In 2.4, or 2.6?

Er, it depends on what you consider "usable".

Unicode integration turned out to have a lot messier edge cases than
anyone understood going in.  First-cut support was in 1.6, but I'd say
it still has some pretty sharp edges *today*.  Which is why 3.0 has
gone all-Unicode-all-the-time.  The problems mostly come from having
two different notions of "string" that don't really mix well.

Me, I still avoid the hell out of Unicode in Python.  And occasionally
fund myself cursing a library maintainer who didn't.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: [PATCH] Third try at documenting command integration requirements.

[Junio C Hamano]

esr@thyrsus.com (Eric S. Raymond) writes:

> @@ -0,0 +1,91 @@
> += Integrating new subcommands =
> +
> +This is how-to documentation for people who want to add extension
> +commands to git.  It should be read alongside api-builtin.txt.
> +
> +== Runtime environment ==
> +
> +git subcommands are standalone executables that live in the git

Even though "={n} title ={n}" is a valid AsciiDoc heading, all other
files use (older) underscored titles; please refrain from being
original.

Especially, this interferes with the way the api-index.txt file in
the same directory is autogenerated.

Re: [PATCH] Third try at documenting command integration requirements.

[Eric S. Raymond]

Junio C Hamano <gitster@pobox.com>:
> Even though "={n} title ={n}" is a valid AsciiDoc heading, all other
> files use (older) underscored titles; please refrain from being
> original.
> 
> Especially, this interferes with the way the api-index.txt file in
> the same directory is autogenerated.

Noted for the future, thank you.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>