What to expect after 0.99.8

What to expect after 0.99.8

[Junio C Hamano]

As I mentioned in teh 0.99.8 announcement, let's start aiming
for 1.0, really this time.  From now on, brown paper bags,
bugfixes, portability fixes, usability enhancements including
documentation updates take precedence over any new features.
One exception area is probably merge strategy modules -- they
are like adding new device drivers or adding new filesystem, and
can come in anytime as long as they do not touch the coreish.


The GIT To-Do File
==================

  The latest copy of this document is found at 

    http://kernel.org/git/?p=git/git.git;a=blob;hb=todo;f=TODO


Tool Renames Plan
=================

 - In 0.99.8, we still install the backward compatible symbolic
   links in $(bindir).  These will however be removed before 1.0
   happens.

   git-ssh-push and git-ssh-pull pair is not going away within
   this timeframe, if ever.  Each of these old-name commands
   continues to invoke its old-name counterpart on the other
   end.


What to expect after 0.99.8
===========================

This is written in a form of to-do list for me, so if I say
"accept patch", it means I do not currently plan to do that
myself.  People interested in seeing it materialize please take
a hint.


Documentation
-------------

* Accept patches from people who actually have done CVS
  migration and update the cvs-migration documentation.
  Link the documentation from the main git.txt page.

* Talk about using rsync just once at the beginning when
  initializing a remote repository so that local packs do not
  need to be expanded.  I personally do not think we need tool
  support for this (but see below about optimized cloning).

* Maybe update tutorial with a toy project that involves two or
  three developers..

* Update tutorial to cover setting up repository hooks to do
  common tasks.

* Accept patches to finish missing docs.

* Accept patches to talk about "Whoops, it broke.  What's
  next?".

* Accept patches to make formatted tables in asciidoc to work
  well in both html and man pages (see git-diff(1)).


Technical (heavier)
-------------------

* We might want to optimize cloning with GIT native transport
  not to explode the pack, and store it in objects/pack instead.
  We would need a tool to generate an idx file out of a pack
  file for this.  Also this itself may turn out to be a bad
  idea, making the set of packs in repositories everybody has
  different from each other.

* Git daemon, when deployed at kernel.org, might turn out to be
  quite a burden, since it needs to generate customized packs
  every time a new request comes in.  It may be worthwhile to
  precompute some packs for popular sets of heads downloaders
  have and serve that, even if that could give more than the
  client asks for in some cases.  We will know about this soon
  enough.

* Libification.  There are many places "run once" mentality is
  ingrained in the management of basic data structures, which
  need to be fixed.

* Maybe a pack optimizer.

* Maybe an Emacs VC backend.

* 'git split-projects'?  This requires updated 'git-rev-list' to
  skip irrelevant commits.
  Message-ID: <Pine.LNX.4.63.0509221617300.23242@iabervon.org>

* Look at libified GNU diff CVS seems to use, or libxdiff.


Technical (milder)
------------------

* Encourage concrete proposals to commit log message templates
  we discussed some time ago.

* Accept patches to cause "read-tree -u" delete a directory when
  it makes it empty.

* Perhaps accept patches to introduce the concept of "patch flow
  expressed as ref mappings" Josef has been advocating about.

* Perhaps accept patches to do undo/redo.

* Perhaps accept patch to optionally allow '--fuzz' in
  'git-apply'.

* Allow 'git apply' to accept GNU diff 2.7 output that forgets
  to say '\No newline' if both input ends with incomplete
  lines.

* Maybe grok PGP signed text/plain in applymbox as well.

* Perhaps a tool to revert a single file to pre-modification
  state?  People with BK background know this operation as
  'clean'.  'git checkout [-f] ent [path...]' was suggested by
  Matthias Urlichs which sounds a natural extention to what the
  command currently does.

* Enhance "git repack" to not always use --all; this would be
  handy if the repository contains wagging heads like "pu" in
  git.git repository.

* Internally split the project into non-doc and doc parts; add
  an extra root for the doc part and merge from it; move the
  internal doc source to a separate repository, like the +Meta
  repository; experiment if this results in a reasonable
  workflow, and document it in howto form if it does.

* Make rebase restartable; instead of skipping what cannot be
  automatically forward ported, leave the conflicts in the work
  tree, have the user resolve it, and then restart from where it
  left off.

* Output full path in the "git-rev-list --objects" output, not
  just the basename, and see the improved clustering results in
  better packing [Tried, but did not work out well].

* Updated git-changes-script Jeff Garzik needs [Inquiry for
  external spec sent out with a quick hack.  Will know if that
  is what he needs soon enough].


Technical (trivial)
-------------------

* short SHA1 naming is not enforcing uniqueness.  Should fix.

* 'git repack' can be DOSed.  Should fix.

* Stop installing the old-name symlinks [POSTPONED].

* 'git merge-projects'?

* 'git lost-and-found'?  Link dangling commits found by
  fsck-objects under $GIT_DIR/refs/lost-found/.  Then
  show-branch or gitk can be used to find any lost commit. [A
  feeler patch sent out. Very underwhelming response X-<.]

  Do not name it /lost+found/; that would probably confuse
  things that mistake it a mount point (not our code but
  somebody else's).

* Add simple globbing rules to git-show-branch so that I can
  say 'git show-branch --heads "ko-*"' (ko-master, ko-pu, and
  ko-rc are in refs/tags/).

* We would want test scripts for the relative directory path
  stuff Linus has been working on.  So far, the following
  commands should be usable with relative directory paths:

    git-update-index
    git-ls-files
    git-diff-files
    git-diff-index
    git-diff-tree
    git-rev-list
    git-rev-parse

* In a freashly created empty repository, `git fetch foo:bar`
  works OK, but `git checkout bar` afterwards does not (missing
  `.git/HEAD`).

\f
Local Variables:
mode: text
End:

Re: What to expect after 0.99.8

[A Large Angry SCM]

Junio C Hamano wrote:
> As I mentioned in teh 0.99.8 announcement, let's start aiming
> for 1.0, really this time.  From now on, brown paper bags,
> bugfixes, portability fixes, usability enhancements including
> documentation updates take precedence over any new features.
> One exception area is probably merge strategy modules -- they
> are like adding new device drivers or adding new filesystem, and
> can come in anytime as long as they do not touch the coreish.
> 
> 
> The GIT To-Do File
> ==================
> 
>   The latest copy of this document is found at 
> 
>     http://kernel.org/git/?p=git/git.git;a=blob;hb=todo;f=TODO

If you were to publish the ToDo to the mailing list once a week it might 
encourage more of those patches you want to accept.

> 
...
> 
> * Maybe update tutorial with a toy project that involves two or
>   three developers..

This is *important* to have for 1.0.

> 
> * Update tutorial to cover setting up repository hooks to do
>   common tasks.

This is nice to have for 1.0.

> 
> * Accept patches to finish missing docs.

A list of missing, incomplete, and/or wrong docs in the ToDo file would 
help focus effort when people (like me) have space cycles.

> 
> * Accept patches to talk about "Whoops, it broke.  What's
>   next?".

This is *important* to have for 1.0.

> 
> * Accept patches to make formatted tables in asciidoc to work
>   well in both html and man pages (see git-diff(1)).

A Git documentation asciidoc style guide and howto would be _very_ 
useful. The current Git documentation is not all that consistent. (and I 
accept the blame for the docs I wrote)

> 
> 
> Technical (heavier)
> -------------------
...
> 
> * Maybe a pack optimizer.

Huh?

...

> 
> * Internally split the project into non-doc and doc parts; add
>   an extra root for the doc part and merge from it; move the
>   internal doc source to a separate repository, like the +Meta
>   repository; experiment if this results in a reasonable
>   workflow, and document it in howto form if it does.

I think this is a bad idea. The docs should be part of the project 
(repository and head) as the code. Otherwise, they'll become even more 
out-of-sync.

> 
...
> 
> Technical (trivial)
> -------------------
> 
...
> 
> * 'git merge-projects'?

Huh?

> 
> * 'git lost-and-found'?  Link dangling commits found by
>   fsck-objects under $GIT_DIR/refs/lost-found/.  Then
>   show-branch or gitk can be used to find any lost commit. [A
>   feeler patch sent out. Very underwhelming response X-<.]

The response may have been underwhelming but it's still a good idea.

> 
>   Do not name it /lost+found/; that would probably confuse
>   things that mistake it a mount point (not our code but
>   somebody else's).

I the this concern is overblown.

> 
...

Re: What to expect after 0.99.8

[Junio C Hamano]

A Large Angry SCM <gitzilla@gmail.com> writes:

> If you were to publish the ToDo to the mailing list once a week it might 
> encourage more of those patches you want to accept.

Hmph.  I tend to dislike periodical posting that is more often
than once a month.

>> * Accept patches to finish missing docs.
>
> A list of missing, incomplete, and/or wrong docs in the ToDo file would 
> help focus effort when people (like me) have space cycles.

Well, the thing is, I am not good at documentation, especially
when I have other interests, and once I start writing a list of
missing or incomplete docs, my interests _will_ shift to fill in
those gaps and I will end up doing them myself, which means I
would not have a chance to place the list in the TODO file.

>> Technical (heavier)
>> -------------------
> ...
>> * Maybe a pack optimizer.
>
> Huh?

Given a set of objects and a set of refs (probably a handful
branch heads and point release tags), find a set of packs to
allow reasonably minimum download for all of these classes of
people: (1) somebody cloning the repository from scratch, (2)
somebody who tends to follow the master branch head reasonably
closely, (3) somebody who tends to follow only the point
releases.

>> * Internally split the project into non-doc and doc parts; add
>>   an extra root for the doc part and merge from it; move the
>>   internal doc source to a separate repository, like the +Meta
>>   repository; experiment if this results in a reasonable
>>   workflow, and document it in howto form if it does.
>
> I think this is a bad idea. The docs should be part of the project 
> (repository and head) as the code. Otherwise, they'll become even more 
> out-of-sync.

The point was to make it possible to fork that part off to
somebody else; then I do not have to maintain Documentation
directory myself anymore, just like I simply slurp the latest
gitk from Paul and not worry about it ;-).

>> Technical (trivial)
>> -------------------
>>
> ...
>> * 'git merge-projects'?
>
> Huh?

    Subject: Re: Merges without bases
    References: <1125004228.4110.20.camel@localhost.localdomain>
    Date: Thu, 25 Aug 2005 15:26:36 -0700
    Message-ID: <7vvf1tps9v.fsf@assigned-by-dhcp.cox.net>

[PATCH] Enable and fix support for base less merges.

[Fredrik Kuivinen]

Let the merge strategies handle the base less case if they are able to
do it. It also fixes git-resolve.sh to die if no common ancestors
exists, instead of doing the wrong thing. Furthermore, it contains a
small independent fix for git-merge.sh and a fix for a base less code
path in gitMergeCommon.py.

With this it's possible to use
    git merge -s recursive 'merge message' A B
to do a base less merge of A and B.

Signed-off-by: Fredrik Kuivinen <freku045@student.liu.se>


---

 git-merge-resolve.sh |    6 ++++++
 git-merge.sh         |    4 ++--
 gitMergeCommon.py    |    2 +-
 3 files changed, 9 insertions(+), 3 deletions(-)

d298deec60a0fac7a16f43135f390c285251f613
diff --git a/git-merge-resolve.sh b/git-merge-resolve.sh
--- a/git-merge-resolve.sh
+++ b/git-merge-resolve.sh
@@ -31,6 +31,12 @@ case "$remotes" in
 	exit 2 ;;
 esac
 
+# Give up if this is a baseless merge.
+if test '' == "$bases"
+then
+	exit 2
+fi
+
 git-update-index --refresh 2>/dev/null
 git-read-tree -u -m $bases $head $remotes || exit 2
 echo "Trying simple merge."
diff --git a/git-merge.sh b/git-merge.sh
--- a/git-merge.sh
+++ b/git-merge.sh
@@ -26,7 +26,7 @@ dropsave() {
 savestate() {
 	# Stash away any local modifications.
 	git-diff-index -r -z --name-only $head |
-	cpio -0 -o >"$GIR_DIR/MERGE_SAVE"
+	cpio -0 -o >"$GIT_DIR/MERGE_SAVE"
 }
 
 restorestate() {
@@ -103,7 +103,7 @@ echo "$head" >"$GIT_DIR/ORIG_HEAD"
 
 case "$#,$common" in
 *,'')
-	die "Unable to find common commit between $head_arg and $*"
+	# No common ancestors found. We need a real merge.
 	;;
 1,"$1")
 	# If head can reach all the merge then we are up to date.
diff --git a/gitMergeCommon.py b/gitMergeCommon.py
--- a/gitMergeCommon.py
+++ b/gitMergeCommon.py
@@ -213,7 +213,7 @@ def buildGraph(heads):
 
 # Write the empty tree to the object database and return its SHA1
 def writeEmptyTree():
-    tmpIndex = os.environ['GIT_DIR'] + '/merge-tmp-index'
+    tmpIndex = os.environ.get('GIT_DIR', '.git') + '/merge-tmp-index'
     def delTmpIndex():
         try:
             os.unlink(tmpIndex)

Re: [PATCH] Enable and fix support for base less merges.

[Fredrik Kuivinen]

On Mon, Oct 03, 2005 at 07:34:19PM +1300, Martin Langhoff wrote:
> > With this it's possible to use
> >     git merge -s recursive 'merge message' A B
> > to do a base less merge of A and B.
> 
> Would it be possible/useful to teach git-apply about this?
> 

I don't really understand what you mean. In what way could git-apply
use this? Is there a specific use case you are thinking about?

- Fredrik

Re: [PATCH] Enable and fix support for base less merges.

[Fredrik Kuivinen]

On Tue, Oct 04, 2005 at 07:22:04PM +1300, Martin Langhoff wrote:
> On 10/4/05, Fredrik Kuivinen <freku045@student.liu.se> wrote:
> > I don't really understand what you mean. In what way could git-apply
> > use this? Is there a specific use case you are thinking about?
> 
> Hmmm, perhaps I'm not understanding what a 'base less' merge is.
>

A base less merge is a merge of two branches which do not have a
common ancestor. That is, git-merge-base --all branch-A branch-B will
not return any results.

> Lately, I've been doing some "merges" where there was no common
> ancestor (known to git) and doing some lightweight cherrypicking by
> using `git-format-patch --mbox -o tmpdir` and then using
> git-applymbox. This is very useful to "replay" history against a
> different git repo (or branch) that doesn't share a common ancestor.
>
> But this has no support from the new smart merging mechanisms, which
> could potentially help by applying a patch to a renamed file. I'm not
> sure whether the "recursive" strategy needs 2 parents to figure this
> out, but if it doesn't, this'd be interesting to have. But at the time
> git-apply is cold, limited and unhelpful.

A base less merge is handled exactly as if there was a common ancestor
for the two branches with an empty tree. Renames are detected by
executing git-diff-tree -M --diff-filter=R -r <common ancestor>
<branch-A> and the analogous command for <branch-B>. Hence, if <common
ancestor> corresponds to an empty tree no renames will be detected.

I guess the major difference between cherrypicking with
git-format-patch and merging is that a merge is pretty much an all or
nothing thing. If you merge a branch you will get every commit from
that branch (and if you don't merge you will obviously not get any
commits at all).

> If baseless merges support what I am doing without resorting to
> patches, I'd be a really happy camper. Using mbox patchruns sucks,
> thank you very much for asking, because they don't support binary
> files.

It's unfortunate that binary files aren't supported. I have been
thinking about doing something about it, there isn't any code yet
though.

Anyway, the idea I have thought about is to use (probably
base64-encoded) xdelta diffs for the binary files. With this approach
git diffs could look like:

diff --git --xdelta a/foo b/foo
<base64-encoded xdelta data>

Is this approach reasonable?

- Fredrik

Re: [PATCH] Enable and fix support for base less merges.

[Junio C Hamano]

Fredrik Kuivinen <freku045@student.liu.se> writes:

> Anyway, the idea I have thought about is to use (probably
> base64-encoded) xdelta diffs for the binary files. With this approach
> git diffs could look like:
>
> diff --git --xdelta a/foo b/foo
> <base64-encoded xdelta data>
>
> Is this approach reasonable?

If we continue to operate within 3-way merge paradigm of
"merging his changes to common ancestor into mine", more
appropriate would be to make the use of "merge" a bit more
configurable.  As HPA mentioned, XML for example is a text file
not binary, but textual merge of it often produces less than
usable results, especially when it is machine generated, meant
to be consumed by machines, and optimized for compactness.  Just
like we made merge strategy backends pluggable to git-merge,
strategy backends, after making their own decision on what 3
blobs to use for 3-way file-level merge, can be told which
low-level merge program to use (merge, diff3, wiggle, ...).

Re: What to expect after 0.99.8

[Horst von Brand]

Junio C Hamano <junkio@cox.net> wrote:
> A Large Angry SCM <gitzilla@gmail.com> writes:
> 
> > If you were to publish the ToDo to the mailing list once a week it might 
> > encourage more of those patches you want to accept.
> 
> Hmph.  I tend to dislike periodical posting that is more often
> than once a month.
> 
> >> * Accept patches to finish missing docs.
> >
> > A list of missing, incomplete, and/or wrong docs in the ToDo file would 
> > help focus effort when people (like me) have space cycles.

> Well, the thing is, I am not good at documentation, especially
> when I have other interests, and once I start writing a list of
> missing or incomplete docs, my interests _will_ shift to fill in
> those gaps and I will end up doing them myself, which means I
> would not have a chance to place the list in the TODO file.

Then put the following on the TODO list:

* Accept patches to the TODO list for missing/incomplete/... documentation

;-)
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: What to expect after 0.99.8

[Josef Weidendorfer]

On Monday 03 October 2005 02:14, Junio C Hamano wrote:
> * Perhaps accept patches to introduce the concept of "patch flow
>   expressed as ref mappings" Josef has been advocating about.

This was about a central place to store local/remote ref mappings.
I proposed the Pull/Push lines in remote/ files changing to hold
these mappings, and putting the defaults into Pull-Default and
Push-Default.

I changed my mind: IMHO for simplicity, porcelain commands should
mostly deal with refspecs (default: current head), and thus,
above mappings should be stored per ref/head, not per remote repository.
I.e. I won't provide a patch for this.

But why did we choose to make git-pull/git-push
to accept a remote repository as first argument, and not a head/refspec
in the first place? [Of course, this needs the remote repository be
retrievable by head name: see branches/ files. And currently missing here is 
the distinction between fetch and push direction].

In the current state, it would be better to get rid of branches/
parsing in GIT at all: By keeping it in, we force Cogito to keep the current
format.
[BTW: the git-push man page is wrong about branches/: the name of the file
in branches/ corresponds to a local refspec, and not to a remote name]

Josef

Re: What to expect after 0.99.8

[Junio C Hamano]

Josef Weidendorfer <Josef.Weidendorfer@gmx.de> writes:

> But why did we choose to make git-pull/git-push
> to accept a remote repository as first argument, and not a head/refspec
> in the first place? [Of course, this needs the remote repository be
> retrievable by head name: see branches/ files. And currently missing here is 
> the distinction between fetch and push direction].

I do not understand this comment.  I do not think you are just
talking about the syntax: 

        $ git-push <remote> <refspec1> <refspec2>...
    vs
        $ git-push' <refspec1> <refspec2>... <remote>

There must be something deeper you are talking about I am
missing.  The branches/ file allowed optionally specifying one
single refname (otherwise defaulting HEAD) since we tried to
become compatible with what Cogito did.

It appears that cg-push uses the same branches information for
pushing into remote, which is what I missed when I did
git-parse-remote --- we do not use this information to decide
the default ref to push into, and not to break expectations of
Cogito users we may need to fix this.  Is this what you are
discussing here?

> In the current state, it would be better to get rid of branches/
> parsing in GIT at all: By keeping it in, we force Cogito to keep the current
> format.

Hmph.  The intent was to keep people's existing Cogito derived
configuration working.

Re: What to expect after 0.99.8

[Josef Weidendorfer]

On Tuesday 04 October 2005 07:57, Junio C Hamano wrote:
> I do not understand this comment.

I talk about configuration per remote repository vs. configuration per
local head, and a consistent user interface regarding these.

Cogito is using configuration per head: If there is a remote fetch mapping,
this is stored in branches/<headname>. There is no new shortcut to
remember, because every file in branches corresponds to a local head.

On the other hand, a remotes/<remote> file is configuration per remote 
repository, and it introduces a new shortname <remote> for a repository, 
which by itself has nothing to do with a head name.

You look for a file in branches/ if there is nothing in remotes/.
I think this is quite confusing for newbies, and blurs the understanding
of remotes/ files, as a file name in branches/ (which is a existing head),
and the file name in remotes/ (which is an arbitrary choosen shortcut for a 
remote repository) are quite different things.

If GIT makes use of per-remote-repository only (as it is now), I
suggest getting rid of parsing branches/. Cogito can do its
per-head configuration in addition/layered on top of GITs 
per-remote-repository configuration, e.g. by refering to a repository
shortcut name in its files in branches/.


> I do not think you are just 
> talking about the syntax:
>
>         $ git-push <remote> <refspec1> <refspec2>...
>     vs
>         $ git-push' <refspec1> <refspec2>... <remote>

No. It is about specifying local head names only, without remote.
Please forget for the moment the current command lines, and
and suppose it is about

	$ git-fetch <localhead>

Given per-head configuration for <localhead>, the <remote> would
be already available (in branches/<localhead>); it does not
need to be specified at all. And a

	$ git-fetch <localhead1> <localhead2> ...

would potentially fetch from different remote repositories.
The nice thing here is that without a head, you can use the
current HEAD as default.

I do not advocate to change this in GIT at all; but what about
git-fetch-head/git-pull-head/git-push-head ?

> It appears that cg-push uses the same branches information for
> pushing into remote, which is what I missed when I did
> git-parse-remote --- we do not use this information to decide
> the default ref to push into, and not to break expectations of
> Cogito users we may need to fix this.

This even would be the wrong behavior: cg-clone creates local <origin>
head, branches off <master>, and checks out <master>. So when running
cg-push afterwards, you are in fact on the <master> branch, and there
is no remote specified for <master>. Cogito
currently is hardcoded to push to the remote head specified in <origin>
when on <master>.

I suggested Pasky to get rid of this hardcoding, support a "Push:"
line in branches/ files, and create a branches/master in cg-clone.
Similar, a configuration to specify the relation of origin and master
is missing.

Then, to be correct, you would have to use the remote head
in the "Push:" line of the current HEAD.

> Is this what you are 
> discussing here?

No. We can not do this, as Cogito currently only stores half of its
behavior in configuration files.

> > In the current state, it would be better to get rid of branches/
> > parsing in GIT at all: By keeping it in, we force Cogito to keep the
> > current format.
>
> Hmph.  The intent was to keep people's existing Cogito derived
> configuration working.

If you really want to still parse branches/ files, perhaps it would be
better to create a remotes/ file the first time after parsing it, and
give out a warning that a new repository shortcut was created, to be
used with git-pull etc.

I still think it is wrong to use one head name of a repository as the
default for the repository's name.

Josef

Re: What to expect after 0.99.8

[Junio C Hamano]

Josef Weidendorfer <Josef.Weidendorfer@gmx.de> writes:

> On Tuesday 04 October 2005 07:57, Junio C Hamano wrote:
>> I do not understand this comment.
>
> I talk about configuration per remote repository vs. configuration per
> local head, and a consistent user interface regarding these.
>
> Cogito is using configuration per head: If there is a remote fetch mapping,
> this is stored in branches/<headname>. There is no new shortcut to
> remember, because every file in branches corresponds to a local head.

Thanks.  This is "lightbulb" moment for me.  I did not realize
branches/ files are "per head configuration" -- I just thought
they are arbitrary tokens, and by convention it can only slurp
into the branch with the same name as the file.  I even thought
about mentioning the possibility to enhance the current

    url://to/repo#rembranch

to
    url://to/repo#rembranch1:localbranch1,rembranch2:localbranch2,...

but I am glad I didn't -- localbranch cannot be anything other
than the filename because by definition the file is to describe
the local branch of the same name, and describing more than one
localbranch does not make sense.  OTOH,

    url://to/repo#rembranch1,rembranch2,...

may make sense now --- you are telling this local branch pulls
(and makes Octopus) from multiple remote branches.

Anyway, I think I understand what you meant to say now.

> I still think it is wrong to use one head name of a repository as the
> default for the repository's name.

I think this is about branches/ file: "One head name of the
local repository is used as a shorthand to name the remote
repository described by the file, inside the local repository".  

[PATCH] Random documentation fixes

[Jonas Fonseca]

Junio C Hamano <junkio@cox.net> wrote Sun, Oct 02, 2005:
> Documentation
> -------------
> * Accept patches to make formatted tables in asciidoc to work
>   well in both html and man pages (see git-diff(1)).

I went through the docs and have attached a patch fixing some
formattings that was causing bad rendering. Note, I have mostly
concentrated on getting the HTML better.

In order to fix the manpages we probably have add our own docbook hooks
to convert everything to verbatim form instead of tables, which
apparently cannot be handled well by xmlto. I haven't tried it though,
just an idea.

Another thing, a lot of the manpages have long synopsis lines spanning
multiple lines. In both the man and HTML pages they are turned into one
long line. I found that wrapping them in something like:

  SYNOPSIS
  --------
 +[verse]
 +.........................................................................
  'git-cvsimport' [ -o <branch-for-HEAD> ] [ -h ] [ -v ]
  			[ -d <CVSROOT> ] [ -p <options-for-cvsps> ]
  			[ -C <GIT_repository> ] [ -i ] [ -k ]
  			[ -s <subst> ] [ -m ] [ -M regex ] [ <CVS_module> ]
 +.........................................................................

will make it look like intended. I don't know if sacrificing readability
of the *.txt files for improved generated docs is acceptable. If not the
patch below should probably also be filtered a bit.

Another thing (sorry for putting it all in one mail), what about also
documenting all Makefile configuration variables, such as:

 diff --git a/Makefile b/Makefile
 --- a/Makefile
 +++ b/Makefile
 @@ -29,6 +29,8 @@
  #
  # Define WITH_OWN_SUBPROCESS_PY if you want to use with python 2.3.
  #
 +# Define WITH_SEND_EMAIL if you have the Perl Sendmail module installed
 +#
  # Define COLLISION_CHECK below if you believe that SHA1's
  # 1461501637330902918203684832716283019655932542976 hashes do not give you
  # sufficient guarantee that no collisions between objects will ever happen.

---

The fixes focuses on improving the HTML output. Most noteworthy:

 - Fix the Makefile to also make various *.html files depend on
   included files.

 - Consistently use 'NOTE: ...' instead of '[ ... ]' for additional
   info.

 - Fix ending '::' for description lists in OPTION section etc.

 - Fix paragraphs in description lists ending up as preformated text.

 - Always use listingblocks (preformatted text wrapped in lines with -----)
   for examples that span empty lines, so they are put in only one HTML
   block.

 - Use '1.' instead of '(1)' for numbered lists.

 - Fix linking to other GIT docs.

 - git-rev-list.txt: put option descriptions in an OPTION section.

Signed-off-by: Jonas Fonseca <fonseca@diku.dk>

---

 Documentation/Makefile             |    3 +
 Documentation/cvs-migration.txt    |    8 +-
 Documentation/diffcore.txt         |   12 ++-
 Documentation/git-applypatch.txt   |    2 -
 Documentation/git-bisect.txt       |    2 -
 Documentation/git-cherry-pick.txt  |    6 +-
 Documentation/git-commit-tree.txt  |    6 +-
 Documentation/git-cvsimport.txt    |   10 +--
 Documentation/git-diff-index.txt   |    8 +-
 Documentation/git-diff-tree.txt    |   22 +++---
 Documentation/git-fetch.txt        |    2 -
 Documentation/git-fsck-objects.txt |    8 +-
 Documentation/git-ls-files.txt     |   20 +++--
 Documentation/git-pack-objects.txt |    3 +
 Documentation/git-prune-packed.txt |    3 +
 Documentation/git-read-tree.txt    |    4 +
 Documentation/git-repack.txt       |    3 +
 Documentation/git-rev-list.txt     |   72 ++++++++++---------
 Documentation/git-revert.txt       |    6 +-
 Documentation/git-send-email.txt   |   16 ++--
 Documentation/git-send-pack.txt    |    8 +-
 Documentation/git-sh-setup.txt     |    9 +-
 Documentation/git-update-index.txt |    2 -
 Documentation/git-var.txt          |   14 ++--
 Documentation/git-verify-pack.txt  |   14 ++--
 Documentation/pull-fetch-param.txt |  138 +++++++++++++++++-------------------
 27 files changed, 211 insertions(+), 192 deletions(-)

diff --git a/Documentation/Makefile b/Documentation/Makefile
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -53,6 +53,9 @@ install: man
 $(patsubst %.txt,%.1,$(wildcard git-diff-*.txt)): \
 	diff-format.txt diff-options.txt
 $(patsubst %,%.1,git-fetch git-pull git-push): pull-fetch-param.txt
+$(patsubst %.txt,%.html,$(wildcard git-diff-*.txt)): \
+	diff-format.txt diff-options.txt
+$(patsubst %,%.html,git-fetch git-pull git-push): pull-fetch-param.txt
 git.7: ../README
 
 clean:
diff --git a/Documentation/cvs-migration.txt b/Documentation/cvs-migration.txt
--- a/Documentation/cvs-migration.txt
+++ b/Documentation/cvs-migration.txt
@@ -229,10 +229,10 @@ does rename or copy would not show in th
 "o-file.c", it would find the commit that changed the statement
 when it was in "o-file.c".
 
-[ BTW, the current versions of "git-diff-tree -C" is not eager
+NOTE: The current versions of "git-diff-tree -C" is not eager
   enough to find copies, and it will miss the fact that a-file.c
   was created by copying o-file.c unless o-file.c was somehow
-  changed in the same commit.]
+  changed in the same commit.
 
 You can use the --pickaxe-all flag in addition to the -S flag.
 This causes the differences from all the files contained in
@@ -243,6 +243,6 @@ that contain this changed "if" statement
 		nitfol();
 	}' --pickaxe-all
 
-[ Side note.  This option is called "--pickaxe-all" because -S
+NOTE: This option is called "--pickaxe-all" because -S
   option is internally called "pickaxe", a tool for software
-  archaeologists.]
+  archaeologists.
diff --git a/Documentation/diffcore.txt b/Documentation/diffcore.txt
--- a/Documentation/diffcore.txt
+++ b/Documentation/diffcore.txt
@@ -254,11 +254,11 @@ As an example, typical orderfile for the
 would look like this:
 
 ------------------------------------------------
-    README
-    Makefile
-    Documentation
-    *.h
-    *.c
-    t
+README
+Makefile
+Documentation
+*.h
+*.c
+t
 ------------------------------------------------
 
diff --git a/Documentation/git-applypatch.txt b/Documentation/git-applypatch.txt
--- a/Documentation/git-applypatch.txt
+++ b/Documentation/git-applypatch.txt
@@ -30,7 +30,7 @@ OPTIONS
 <patch>::
 	The patch to apply.
 
-<info>:
+<info>::
 	Author and subject information extracted from e-mail,
 	used on "author" line and as the first line of the
 	commit log message.
diff --git a/Documentation/git-bisect.txt b/Documentation/git-bisect.txt
--- a/Documentation/git-bisect.txt
+++ b/Documentation/git-bisect.txt
@@ -91,7 +91,7 @@ Author
 Written by Linus Torvalds <torvalds@osdl.org>
 
 Documentation
---------------
+-------------
 Documentation by Junio C Hamano and the git-list <git@vger.kernel.org>.
 
 GIT
diff --git a/Documentation/git-cherry-pick.txt b/Documentation/git-cherry-pick.txt
--- a/Documentation/git-cherry-pick.txt
+++ b/Documentation/git-cherry-pick.txt
@@ -38,9 +38,9 @@ OPTIONS
 	option is used, your working tree does not have to match
 	the HEAD commit.  The cherry-pick is done against the
 	beginning state of your working tree.
-
-	This is useful when cherry-picking more than one commits'
-	effect to your working tree in a row.
++
+This is useful when cherry-picking more than one commits'
+effect to your working tree in a row.
 
 
 Author
diff --git a/Documentation/git-commit-tree.txt b/Documentation/git-commit-tree.txt
--- a/Documentation/git-commit-tree.txt
+++ b/Documentation/git-commit-tree.txt
@@ -58,11 +58,11 @@ following environment variables.
 	GIT_COMMITTER_NAME
 	GIT_COMMITTER_EMAIL
 
-(nb <,> and '\n's are stripped)
+(nb "<", ">" and "\n"s are stripped)
 
 A commit comment is read from stdin (max 999 chars). If a changelog
-entry is not provided via '<' redirection, "git-commit-tree" will just wait
-for one to be entered and terminated with ^D
+entry is not provided via "<" redirection, "git-commit-tree" will just wait
+for one to be entered and terminated with ^D.
 
 Diagnostics
 -----------
diff --git a/Documentation/git-cvsimport.txt b/Documentation/git-cvsimport.txt
--- a/Documentation/git-cvsimport.txt
+++ b/Documentation/git-cvsimport.txt
@@ -51,15 +51,15 @@ OPTIONS
 	The 'HEAD' branch from CVS is imported to the 'origin' branch within
 	the git repository, as 'HEAD' already has a special meaning for git.
 	Use this option if you want to import into a different branch.
-
-	Use '-o master' for continuing an import that was initially done by
-	the old cvs2git tool.
++
+Use '-o master' for continuing an import that was initially done by
+the old cvs2git tool.
 
 -p <options-for-cvsps>::
 	Additional options for cvsps.
 	The options '-u' and '-A' are implicit and should not be used here.
-
-	If you need to pass multiple options, separate them with a comma.
++
+If you need to pass multiple options, separate them with a comma.
 
 -m::    
 	Attempt to detect merges based on the commit message. This option
diff --git a/Documentation/git-diff-index.txt b/Documentation/git-diff-index.txt
--- a/Documentation/git-diff-index.txt
+++ b/Documentation/git-diff-index.txt
@@ -86,8 +86,8 @@ the more useful of the two in that what 
 a "git-write-tree" + "git-diff-tree". Thus that's the default mode.
 The non-cached version asks the question:
 
-   show me the differences between HEAD and the currently checked out
-   tree - index contents _and_ files that aren't up-to-date
+  show me the differences between HEAD and the currently checked out
+  tree - index contents _and_ files that aren't up-to-date
 
 which is obviously a very useful question too, since that tells you what
 you *could* commit. Again, the output matches the "git-diff-tree -r"
@@ -107,13 +107,13 @@ not up-to-date and may contain new stuff
 get the real diff, you need to look at the object in the working directory
 directly rather than do an object-to-object diff.
 
-NOTE! As with other commands of this type, "git-diff-index" does not
+NOTE: As with other commands of this type, "git-diff-index" does not
 actually look at the contents of the file at all. So maybe
 `kernel/sched.c` hasn't actually changed, and it's just that you
 touched it. In either case, it's a note that you need to
 "git-upate-cache" it to make the cache be in sync.
 
-NOTE 2! You can have a mixture of files show up as "has been updated"
+NOTE: You can have a mixture of files show up as "has been updated"
 and "is still dirty in the working directory" together. You can always
 tell which file is in which state, since the "has been updated" ones
 show a valid sha1, and the "not in sync with the index" ones will
diff --git a/Documentation/git-diff-tree.txt b/Documentation/git-diff-tree.txt
--- a/Documentation/git-diff-tree.txt
+++ b/Documentation/git-diff-tree.txt
@@ -101,16 +101,18 @@ An example of normal usage is:
 which tells you that the last commit changed just one file (it's from
 this one:
 
-  commit 3c6f7ca19ad4043e9e72fa94106f352897e651a8
-  tree 5319e4d609cdd282069cc4dce33c1db559539b03
-  parent b4e628ea30d5ab3606119d2ea5caeab141d38df7
-  author Linus Torvalds <torvalds@ppc970.osdl.org> Sat Apr 9 12:02:30 2005
-  committer Linus Torvalds <torvalds@ppc970.osdl.org> Sat Apr 9 12:02:30 2005
-
-  Make "git-fsck-objects" print out all the root commits it finds.
-
-  Once I do the reference tracking, I'll also make it print out all the
-  HEAD commits it finds, which is even more interesting.
+-----------------------------------------------------------------------------
+commit 3c6f7ca19ad4043e9e72fa94106f352897e651a8
+tree 5319e4d609cdd282069cc4dce33c1db559539b03
+parent b4e628ea30d5ab3606119d2ea5caeab141d38df7
+author Linus Torvalds <torvalds@ppc970.osdl.org> Sat Apr 9 12:02:30 2005
+committer Linus Torvalds <torvalds@ppc970.osdl.org> Sat Apr 9 12:02:30 2005
+
+Make "git-fsck-objects" print out all the root commits it finds.
+
+Once I do the reference tracking, I'll also make it print out all the
+HEAD commits it finds, which is even more interesting.
+-----------------------------------------------------------------------------
 
 in case you care).
 
diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
--- a/Documentation/git-fetch.txt
+++ b/Documentation/git-fetch.txt
@@ -39,7 +39,7 @@ Written by Linus Torvalds <torvalds@osdl
 Junio C Hamano <junkio@cox.net>
 
 Documentation
---------------
+-------------
 Documentation by David Greaves, Junio C Hamano and the git-list <git@vger.kernel.org>.
 
 GIT
diff --git a/Documentation/git-fsck-objects.txt b/Documentation/git-fsck-objects.txt
--- a/Documentation/git-fsck-objects.txt
+++ b/Documentation/git-fsck-objects.txt
@@ -19,9 +19,9 @@ OPTIONS
 -------
 <object>::
 	An object to treat as the head of an unreachability trace.
-
-	If no objects are given, git-fsck-objects defaults to using the
-	index file and all SHA1 references in .git/refs/* as heads.
++
+If no objects are given, git-fsck-objects defaults to using the
+index file and all SHA1 references in .git/refs/* as heads.
 
 --unreachable::
 	Print out objects that exist but that aren't readable from any
@@ -128,7 +128,7 @@ GIT_OBJECT_DIRECTORY::
 GIT_INDEX_FILE::
 	used to specify the index file of the cache
 
-GIT_ALTERNATE_OBJECT_DIRECTORIES:
+GIT_ALTERNATE_OBJECT_DIRECTORIES::
 	used to specify additional object database roots (usually unset)
 
 Author
diff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt
--- a/Documentation/git-ls-files.txt
+++ b/Documentation/git-ls-files.txt
@@ -70,11 +70,11 @@ OPTIONS
 -t::
 	Identify the file status with the following tags (followed by
 	a space) at the start of each line:
-	H	cached
-	M	unmerged
-	R	removed/deleted
-	C	modifed/changed
-	K	to be killed
+	H::	cached
+	M::	unmerged
+	R::	removed/deleted
+	C::	modifed/changed
+	K::	to be killed
 	?	other
 
 --::
@@ -110,13 +110,13 @@ flags --others or --ignored are specifie
 
 These exclude patterns come from these places:
 
- (1) command line flag --exclude=<pattern> specifies a single
+  1. command line flag --exclude=<pattern> specifies a single
      pattern.
 
- (2) command line flag --exclude-from=<file> specifies a list of
+  2. command line flag --exclude-from=<file> specifies a list of
      patterns stored in a file.
 
- (3) command line flag --exclude-per-directory=<name> specifies
+  3. command line flag --exclude-per-directory=<name> specifies
      a name of the file in each directory 'git-ls-files'
      examines, and if exists, its contents are used as an
      additional list of patterns.
@@ -168,12 +168,13 @@ An exclude pattern is of the following f
  - otherwise, it is a shell glob pattern, suitable for
    consumption by fnmatch(3) with FNM_PATHNAME flag.  I.e. a
    slash in the pattern must match a slash in the pathname.
-   "Documentation/*.html" matches "Documentation/git.html" but
+   "Documentation/\*.html" matches "Documentation/git.html" but
    not "ppc/ppc.html".  As a natural exception, "/*.c" matches
    "cat-file.c" but not "mozilla-sha1/sha1.c".
 
 An example:
 
+--------------------------------------------------------------
     $ cat .git/ignore
     # ignore objects and archives, anywhere in the tree.
     *.[oa]
@@ -186,6 +187,7 @@ An example:
         --exclude='Documentation/*.[0-9]' \
         --exclude-from=.git/ignore \
         --exclude-per-directory=.gitignore
+--------------------------------------------------------------
 
 
 See Also
diff --git a/Documentation/git-pack-objects.txt b/Documentation/git-pack-objects.txt
--- a/Documentation/git-pack-objects.txt
+++ b/Documentation/git-pack-objects.txt
@@ -76,7 +76,8 @@ Documentation by Junio C Hamano
 
 See-Also
 --------
-git-repack(1) git-prune-packed(1)
+gitlink:git-repack[1]
+gitlink:git-prune-packed[1]
 
 GIT
 ---
diff --git a/Documentation/git-prune-packed.txt b/Documentation/git-prune-packed.txt
--- a/Documentation/git-prune-packed.txt
+++ b/Documentation/git-prune-packed.txt
@@ -34,7 +34,8 @@ Documentation by Ryan Anderson <ryan@mic
 
 See-Also
 --------
-git-pack-objects(1) git-repack(1)
+gitlink:git-pack-objects[1]
+gitlink:git-repack[1]
 
 GIT
 ---
diff --git a/Documentation/git-read-tree.txt b/Documentation/git-read-tree.txt
--- a/Documentation/git-read-tree.txt
+++ b/Documentation/git-read-tree.txt
@@ -84,10 +84,10 @@ fast forward situation).
 When two trees are specified, the user is telling git-read-tree
 the following:
 
-    (1) The current index and work tree is derived from $H, but
+     1. The current index and work tree is derived from $H, but
         the user may have local changes in them since $H;
 
-    (2) The user wants to fast-forward to $M.
+     2. The user wants to fast-forward to $M.
 
 In this case, the "git-read-tree -m $H $M" command makes sure
 that no local change is lost as the result of this "merge".
diff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt
--- a/Documentation/git-repack.txt
+++ b/Documentation/git-repack.txt
@@ -51,7 +51,8 @@ Documentation by Ryan Anderson <ryan@mic
 
 See-Also
 --------
-git-pack-objects(1) git-prune-packed(1)
+gitlink:git-pack-objects[1]
+gitlink:git-prune-packed[1]
 
 GIT
 ---
diff --git a/Documentation/git-rev-list.txt b/Documentation/git-rev-list.txt
--- a/Documentation/git-rev-list.txt
+++ b/Documentation/git-rev-list.txt
@@ -22,65 +22,69 @@ that point. Their parents are implied. "
 means "list all the commits which are included in 'foo' and 'bar', but
 not in 'baz'".
 
-If *--pretty* is specified, print the contents of the commit changesets
-in human-readable form.
-
-The *--objects* flag causes 'git-rev-list' to print the object IDs of
-any object referenced by the listed commits. 'git-rev-list --objects foo
-^bar' thus means "send me all object IDs which I need to download if
-I have the commit object 'bar', but not 'foo'".
-
-The *--bisect* flag limits output to the one commit object which is
-roughly halfway between the included and excluded commits. Thus,
-if 'git-rev-list --bisect foo ^bar
-^baz' outputs 'midpoint', the output
-of 'git-rev-list foo ^midpoint' and 'git-rev-list midpoint
-^bar
-^baz'
-would be of roughly the same length. Finding the change which introduces
-a regression is thus reduced to a binary search: repeatedly generate and
-test new 'midpoint's until the commit chain is of length one.
-
-If *--merge-order* is specified, the commit history is decomposed into a
-unique sequence of minimal, non-linear epochs and maximal, linear epochs.
-Non-linear epochs are then linearised by sorting them into merge order, which
-is described below.
-
+OPTIONS
+-------
+--pretty::
+	Print the contents of the commit changesets in human-readable form.
+
+--objects::
+	Print the object IDs of any object referenced by the listed commits.
+	'git-rev-list --objects foo ^bar' thus means "send me all object IDs
+	which I need to download if I have the commit object 'bar', but
+	not 'foo'".
+
+--bisect::
+	Limit output to the one commit object which is roughly halfway
+	between the included and excluded commits. Thus, if 'git-rev-list
+	--bisect foo ^bar ^baz' outputs 'midpoint', the output
+	of 'git-rev-list foo ^midpoint' and 'git-rev-list midpoint
+	^bar ^baz' would be of roughly the same length. Finding the change
+	which introduces a regression is thus reduced to a binary search:
+	repeatedly generate and test new 'midpoint's until the commit chain
+	is of length one.
+
+--merge-order::
+	When specified the commit history is decomposed into a unique
+	sequence of minimal, non-linear epochs and maximal, linear epochs.
+	Non-linear epochs are then linearised by sorting them into merge
+	order, which is described below.
++
 Maximal, linear epochs correspond to periods of sequential development.
 Minimal, non-linear epochs correspond to periods of divergent development
 followed by a converging merge. The theory of epochs is described in more
 detail at
 link:http://blackcubes.dyndns.org/epoch/[http://blackcubes.dyndns.org/epoch/].
-
++
 The merge order for a non-linear epoch is defined as a linearisation for which
 the following invariants are true:
-
++
     1. if a commit P is reachable from commit N, commit P sorts after commit N
        in the linearised list.
     2. if Pi and Pj are any two parents of a merge M (with i < j), then any
        commit N, such that N is reachable from Pj but not reachable from Pi,
        sorts before all commits reachable from Pi.
-
++
 Invariant 1 states that later commits appear before earlier commits they are
 derived from.
-
++
 Invariant 2 states that commits unique to "later" parents in a merge, appear
 before all commits from "earlier" parents of a merge.
 
-If *--show-breaks* is specified, each item of the list is output with a
-2-character prefix consisting of one of: (|), (^), (=) followed by a space.
-
+--show-breaks::
+	Each item of the list is output with a 2-character prefix consisting
+	of one of: (|), (^), (=) followed by a space.
++
 Commits marked with (=) represent the boundaries of minimal, non-linear epochs
 and correspond either to the start of a period of divergent development or to
 the end of such a period.
-
++
 Commits marked with (|) are direct parents of commits immediately preceding
 the marked commit in the list.
-
++
 Commits marked with (^) are not parents of the immediately preceding commit.
 These "breaks" represent necessary discontinuities implied by trying to
 represent an arbtirary DAG in a linear form.
-
++
 *--show-breaks* is only valid if *--merge-order* is also specified.
 
 Author
diff --git a/Documentation/git-revert.txt b/Documentation/git-revert.txt
--- a/Documentation/git-revert.txt
+++ b/Documentation/git-revert.txt
@@ -29,9 +29,9 @@ OPTIONS
 	working tree does not have to match the HEAD commit.
 	The revert is done against the beginning state of your
 	working tree.
-
-	This is useful when reverting more than one commits'
-	effect to your working tree in a row.
++
+This is useful when reverting more than one commits'
+effect to your working tree in a row.
 
 
 Author
diff --git a/Documentation/git-send-email.txt b/Documentation/git-send-email.txt
--- a/Documentation/git-send-email.txt
+++ b/Documentation/git-send-email.txt
@@ -21,35 +21,37 @@ The header of the email is configurable 
 specified on the command line, the user will be prompted with a ReadLine
 enabled interface to provide the necessary information.
 
+OPTIONS
+-------
 The options available are:
 
-  --to
+--to::
 	Specify the primary recipient of the emails generated.
 	Generally, this will be the upstream maintainer of the
 	project involved.
 
-   --from
+--from::
 	Specify the sender of the emails.  This will default to
 	the value GIT_COMMITTER_IDENT, as returned by "git-var -l".
 	The user will still be prompted to confirm this entry.
 
-   --compose
+--compose::
 	Use \$EDITOR to edit an introductory message for the
 	patch series.
 
-   --subject
+--subject::
    	Specify the initial subject of the email thread.
 	Only necessary if --compose is also set.  If --compose
 	is not set, this will be prompted for.
 
-   --in-reply-to
+--in-reply-to::
 	Specify the contents of the first In-Reply-To header.
 	Subsequent emails will refer to the previous email 
 	instead of this if --chain-reply-to is set (the default)
 	Only necessary if --compose is also set.  If --compose
 	is not set, this will be prompted for.
 
-   --chain-reply-to, --no-chain-reply-to
+--chain-reply-to, --no-chain-reply-to::
 	If this is set, each email will be sent as a reply to the previous
 	email sent.  If disabled with "--no-chain-reply-to", all emails after
 	the first will be sent as replies to the first email sent.  When using
@@ -57,7 +59,7 @@ The options available are:
 	entire patch series.
 	Default is --chain-reply-to
 
-   --smtp-server
+--smtp-server::
 	If set, specifies the outgoing SMTP server to use.  Defaults to
 	localhost.
 
diff --git a/Documentation/git-send-pack.txt b/Documentation/git-send-pack.txt
--- a/Documentation/git-send-pack.txt
+++ b/Documentation/git-send-pack.txt
@@ -61,9 +61,9 @@ this flag.
 Without '--all' and without any '<ref>', the refs that exist
 both on the local side and on the remote side are updated.
 
-When '<ref>'s are specified explicitly, it can be either a
+When one or more '<ref>' are specified explicitly, it can be either a
 single pattern, or a pair of such pattern separated by a colon
-':' (this means that a ref name cannot have a colon in it).  A
+":" (this means that a ref name cannot have a colon in it).  A
 single pattern '<name>' is just a shorthand for '<name>:<name>'.
 
 Each pattern pair consists of the source side (before the colon)
@@ -79,10 +79,10 @@ destination side.
 
  - If <dst> does not match any remote ref, either
 
-   - it has to start with "refs/"; <dst> is used as the
+   * it has to start with "refs/"; <dst> is used as the
      destination literally in this case.
 
-   - <src> == <dst> and the ref that matched the <src> must not
+   * <src> == <dst> and the ref that matched the <src> must not
      exist in the set of remote refs; the ref matched <src>
      locally is used as the name of the destination.
 
diff --git a/Documentation/git-sh-setup.txt b/Documentation/git-sh-setup.txt
--- a/Documentation/git-sh-setup.txt
+++ b/Documentation/git-sh-setup.txt
@@ -14,11 +14,12 @@ DESCRIPTION
 
 Sets up the normal git environment variables and a few helper functions
 (currently just "die()"), and returns ok if it all looks like a git archive.
-So use it something like
+So, to make the rest of the git scripts more careful and readable,
+use it as follows:
 
-	. git-sh-setup || die "Not a git archive"
-
-to make the rest of the git scripts more careful and readable.
+-------------------------------------------------
+. git-sh-setup || die "Not a git archive"
+-------------------------------------------------
 
 Author
 ------
diff --git a/Documentation/git-update-index.txt b/Documentation/git-update-index.txt
--- a/Documentation/git-update-index.txt
+++ b/Documentation/git-update-index.txt
@@ -101,7 +101,7 @@ Using --cacheinfo or --info-only
 current working directory.  This is useful for minimum-checkout
 merging.
 
-  To pretend you have a file with mode and sha1 at path, say:
+To pretend you have a file with mode and sha1 at path, say:
 
    $ git-update-index --cacheinfo mode sha1 path
 
diff --git a/Documentation/git-var.txt b/Documentation/git-var.txt
--- a/Documentation/git-var.txt
+++ b/Documentation/git-var.txt
@@ -15,21 +15,23 @@ DESCRIPTION
 -----------
 Prints a git logical variable.
 
--l causes the logical variables to be listed.
+OPTIONS
+-------
+-l::
+	Cause the logical variables to be listed.
 
 EXAMPLE
 --------
-$git-var GIT_AUTHOR_IDENT
-
-Eric W. Biederman <ebiederm@lnxi.com> 1121223278 -0600
+	$ git-var GIT_AUTHOR_IDENT
+	Eric W. Biederman <ebiederm@lnxi.com> 1121223278 -0600
 
 
 VARIABLES
 ----------
-GIT_AUTHOR_IDENT
+GIT_AUTHOR_IDENT::
     The author of a piece of code.
 
-GIT_COMMITTER_IDENT
+GIT_COMMITTER_IDENT::
     The person who put a piece of code into git.
 
 Diagnostics
diff --git a/Documentation/git-verify-pack.txt b/Documentation/git-verify-pack.txt
--- a/Documentation/git-verify-pack.txt
+++ b/Documentation/git-verify-pack.txt
@@ -25,15 +25,19 @@ OPTIONS
 
 -v::
 	After verifying the pack, show list of objects contained
-	in the pack.  The format used is:
+	in the pack.
 
-		SHA1 type size offset-in-packfile
+OUTPUT FORMAT
+-------------
+When specifying the -v option the format used is:
 
-	for objects that are not deltified in the pack, and
+	SHA1 type size offset-in-packfile
 
-		SHA1 type size offset-in-packfile depth base-SHA1
+for objects that are not deltified in the pack, and
 
-	for objects that are deltified.
+	SHA1 type size offset-in-packfile depth base-SHA1
+
+for objects that are deltified.
 
 Author
 ------
diff --git a/Documentation/pull-fetch-param.txt b/Documentation/pull-fetch-param.txt
--- a/Documentation/pull-fetch-param.txt
+++ b/Documentation/pull-fetch-param.txt
@@ -2,84 +2,78 @@
 	The "remote" repository to pull from.  One of the
 	following notations can be used to name the repository
 	to pull from:
-
-		Rsync URL
-			rsync://remote.machine/path/to/repo.git/
-
-		HTTP(s) URL
-			http://remote.machine/path/to/repo.git/
-
-		GIT URL
-			git://remote.machine/path/to/repo.git/
-			remote.machine:/path/to/repo.git/
-
-		Local directory
-			/path/to/repo.git/
-
-	In addition to the above, as a short-hand, the name of a
-	file in $GIT_DIR/remotes directory can be given; the
-	named file should be in the following format:
-
-		URL: one of the above URL format
-		Push: <refspec>...
-		Pull: <refspec>...
-
-	When such a short-hand is specified in place of
-	<repository> without <refspec> parameters on the command
-	line, <refspec>... specified on Push lines or Pull lines
-	are used for "git push" and "git fetch/pull",
-	respectively.
-
-	The name of a file in $GIT_DIR/branches directory can be
-	specified as an older notation short-hand; the named
-	file should contain a single line, a URL in one of the
-	above formats, optionally followed by a hash '#' and the
-	name of remote head (URL fragment notation).
-	$GIT_DIR/branches/<remote> file that stores a <url>
-	without the fragment is equivalent to have this in the
-	corresponding file in the $GIT_DIR/remotes/ directory
-
-		URL: <url>
-		Pull: refs/heads/master:<remote>
-
-	while having <url>#<head> is equivalent to
-
-		URL: <url>
-		Pull: refs/heads/<head>:<remote>
++
+===============================================================
+- Rsync URL:		rsync://remote.machine/path/to/repo.git/
+- HTTP(s) URL:		http://remote.machine/path/to/repo.git/
+- GIT URL:		git://remote.machine/path/to/repo.git/
+			or remote.machine:/path/to/repo.git/
+- Local directory:	/path/to/repo.git/
+===============================================================
++
+In addition to the above, as a short-hand, the name of a
+file in $GIT_DIR/remotes directory can be given; the
+named file should be in the following format:
++
+	URL: one of the above URL format
+	Push: <refspec>...
+	Pull: <refspec>...
++
+When such a short-hand is specified in place of
+<repository> without <refspec> parameters on the command
+line, <refspec>... specified on Push lines or Pull lines
+are used for "git push" and "git fetch/pull",
+respectively.
++
+The name of a file in $GIT_DIR/branches directory can be
+specified as an older notation short-hand; the named
+file should contain a single line, a URL in one of the
+above formats, optionally followed by a hash '#' and the
+name of remote head (URL fragment notation).
+$GIT_DIR/branches/<remote> file that stores a <url>
+without the fragment is equivalent to have this in the
+corresponding file in the $GIT_DIR/remotes/ directory
++
+	URL: <url>
+	Pull: refs/heads/master:<remote>
++
+while having <url>#<head> is equivalent to
++
+	URL: <url>
+	Pull: refs/heads/<head>:<remote>
 
 <refspec>::
 	The canonical format of a <refspec> parameter is
 	'+?<src>:<dst>'; that is, an optional plus '+', followed
 	by the source ref, followed by a colon ':', followed by
 	the destination ref.
-
-	When used in "git push", the <src> side can be an
-	arbitrary "SHA1 expression" that can be used as an
-	argument to "git-cat-file -t".  E.g. "master~4" (push
-	four parents before the current master head).
-
-        For "git push", the local ref that matches <src> is used
-        to fast forward the remote ref that matches <dst>.  If
-        the optional plus '+' is used, the remote ref is updated
-        even if it does not result in a fast forward update.
-
-	For "git fetch/pull", the remote ref that matches <src>
-	is fetched, and if <dst> is not empty string, the local
-	ref that matches it is fast forwarded using <src>.
-	Again, if the optional plus '+' is used, the local ref
-	is updated even if it does not result in a fast forward
-	update.
-
-	Some short-cut notations are also supported.
-
-	* For backward compatibility, "tag" is almost ignored;
-          it just makes the following parameter <tag> to mean a
-          refspec "refs/tags/<tag>:refs/tags/<tag>".
-
-        * A parameter <ref> without a colon is equivalent to
-          <ref>: when pulling/fetching, and <ref>:<ref> when
-          pushing.  That is, do not store it locally if
-          fetching, and update the same name if pushing.
++
+When used in "git push", the <src> side can be an
+arbitrary "SHA1 expression" that can be used as an
+argument to "git-cat-file -t".  E.g. "master~4" (push
+four parents before the current master head).
++
+For "git push", the local ref that matches <src> is used
+to fast forward the remote ref that matches <dst>.  If
+the optional plus '+' is used, the remote ref is updated
+even if it does not result in a fast forward update.
++
+For "git fetch/pull", the remote ref that matches <src>
+is fetched, and if <dst> is not empty string, the local
+ref that matches it is fast forwarded using <src>.
+Again, if the optional plus '+' is used, the local ref
+is updated even if it does not result in a fast forward
+update.
++
+Some short-cut notations are also supported.
++
+* For backward compatibility, "tag" is almost ignored;
+  it just makes the following parameter <tag> to mean a
+  refspec "refs/tags/<tag>:refs/tags/<tag>".
+* A parameter <ref> without a colon is equivalent to
+  <ref>: when pulling/fetching, and <ref>:<ref> when
+  pushing.  That is, do not store it locally if
+  fetching, and update the same name if pushing.
 
 -a, \--append::
 	Append ref names and object names of fetched refs to the

-- 
Jonas Fonseca

Re: What to expect after 0.99.8

[Daniel Barkalow]

On Sun, 2 Oct 2005, Junio C Hamano wrote:

> What to expect after 0.99.8
> ===========================
> 
> This is written in a form of to-do list for me, so if I say
> "accept patch", it means I do not currently plan to do that
> myself.  People interested in seeing it materialize please take
> a hint.

Are these all before 1.0, or are some of them supposed to happen 
eventually but later?

> Technical (heavier)
> -------------------
> 
> * Libification.  There are many places "run once" mentality is
>   ingrained in the management of basic data structures, which
>   need to be fixed.

I think this should be a post-1.0 thing; I think after 1.0, we should 
rearrange a lot of the code to make more sense from a programmer 
perspective.

> * 'git split-projects'?  This requires updated 'git-rev-list' to
>   skip irrelevant commits.
>   Message-ID: <Pine.LNX.4.63.0509221617300.23242@iabervon.org>

I thought about this some more, and realized that the operation to get 
additions to the gitk history out of a git repository is going to be slow, 
because you don't know the correct hashes for parents until you go all the 
way back. Still worth doing, but less exciting than I'd hoped.

> * Look at libified GNU diff CVS seems to use, or libxdiff.

I've almost got a suffix-tree-based diff that works reasonably well, 
that's built as a library, and outputs unified diff. I need to merge it 
with git, hook up input from trees and blobs, and test it on a wider set 
of data.

I'd also like to add:

 * Accept patches to fetch multiple objects by HTTP in parallel.

I think this may be necessary to get good performance without rsync for 
repositories hosted without specific git support.

	-Daniel
*This .sig left intentionally blank*

Re: What to expect after 0.99.8

[Martin Coxall]

> I'd also like to add:
>
>  * Accept patches to fetch multiple objects by HTTP in parallel.
>

Indeed. As a humble user of git for our inhouse development, I can 
vouch that HTTP pulling on unpacked repositories (i.e. all of them, it 
seems) is appalling. Ship a 1.0 without addressing this, then I imagine 
the Mercurial devs will haemorrage with their Pythonesque mocking. ;p

Martin

Re: What to expect after 0.99.8

[Nick Hengeveld]

On Mon, Oct 03, 2005 at 03:43:02PM -0400, Daniel Barkalow wrote:

>  * Accept patches to fetch multiple objects by HTTP in parallel.
> 
> I think this may be necessary to get good performance without rsync for 
> repositories hosted without specific git support.

I have a version of this almost working, but it's a nontrivial patch
that feels more like a post-1.0 thing to me.

-- 
For a successful technology, reality must take precedence over public
relations, for nature cannot be fooled.

Re: What to expect after 0.99.8

[Daniel Barkalow]

On Mon, 3 Oct 2005, Nick Hengeveld wrote:

> On Mon, Oct 03, 2005 at 03:43:02PM -0400, Daniel Barkalow wrote:
> 
> >  * Accept patches to fetch multiple objects by HTTP in parallel.
> > 
> > I think this may be necessary to get good performance without rsync for 
> > repositories hosted without specific git support.
> 
> I have a version of this almost working, but it's a nontrivial patch
> that feels more like a post-1.0 thing to me.

I'd be interested to see it anyway.

	-Daniel
*This .sig left intentionally blank*

Re: What to expect after 0.99.8

[Junio C Hamano]

Daniel Barkalow <barkalow@iabervon.org> writes:

> Are these all before 1.0, or are some of them supposed to happen 
> eventually but later?

The latter.

>> * Libification.  There are many places "run once" mentality is
>>   ingrained in the management of basic data structures, which
>>   need to be fixed.
>
> I think this should be a post-1.0 thing; I think after 1.0, we should 
> rearrange a lot of the code to make more sense from a programmer 
> perspective.

I agree.

>> * Look at libified GNU diff CVS seems to use, or libxdiff.
>
> I've almost got a suffix-tree-based diff that works reasonably well, 
> that's built as a library, and outputs unified diff. I need to merge it 
> with git, hook up input from trees and blobs, and test it on a wider set 
> of data.

Sounds like fun.

> I'd also like to add:
>
>  * Accept patches to fetch multiple objects by HTTP in parallel.
>
> I think this may be necessary to get good performance without rsync for 
> repositories hosted without specific git support.

Surely.  I'd love to see you work with Nick Hengeveld on this
one, perhaps even before 1.0.  Looking out the fetch.c code, I
think you already have set up a reasonable "work queue" during
the last round, with a lot of simplification thanks to Sergey,
and parallel fetching would be a very nice addition.

Re: What to expect after 0.99.8

[Daniel Barkalow]

On Mon, 3 Oct 2005, Junio C Hamano wrote:

> Daniel Barkalow <barkalow@iabervon.org> writes:
> 
> > Are these all before 1.0, or are some of them supposed to happen 
> > eventually but later?
> 
> The latter.

Good. It would be nice to mark things as being post-1.0 or not.

> > I'd also like to add:
> >
> >  * Accept patches to fetch multiple objects by HTTP in parallel.
> >
> > I think this may be necessary to get good performance without rsync for 
> > repositories hosted without specific git support.
> 
> Surely.  I'd love to see you work with Nick Hengeveld on this
> one, perhaps even before 1.0.  Looking out the fetch.c code, I
> think you already have set up a reasonable "work queue" during
> the last round, with a lot of simplification thanks to Sergey,
> and parallel fetching would be a very nice addition.

The code in ssh-fetch already does parallel fetching, actually (only over 
one connection, but requests are sent before responses are read), so 
multiple requests are in progress at the same time.

	-Daniel
*This .sig left intentionally blank*

Re: What to expect after 0.99.8

[Junio C Hamano]

Daniel Barkalow <barkalow@iabervon.org> writes:

> The code in ssh-fetch already does parallel fetching, actually (only over 
> one connection, but requests are sent before responses are read), so 
> multiple requests are in progress at the same time.

This reminds me of one patch:

    From: Dan Aloni <da-x@monatomic.org>
    Subject: [PATCH] Fix git+ssh's indefinite halts during long fetches
    Date: Sat, 1 Oct 2005 21:39:42 +0300
    Message-ID: <20051001183942.GA2099@localdomain>

I'd appreciate it if you had a chance to take a look at it and
comment on it.

The change is isolated to ssh-fetch [*1*], so even if it were to
break something it would only break ssh-fetch and in that sense
it is a safer change.

But it still is a lot of code, and I felt there might be a
simpler way to do this.  That is why I am deliberately holding
it off.

[Footnote]

*1* The patch touches sha1_file.c and cache.h but that is to
update write_sha1_from_fd(), which is used only by ssh-fetch
AFAICT.  We may want to move it from sha1_file.c to sha1-fetch.c
and make it static, removing it from cache.h.

Re: What to expect after 0.99.8

[Linus Torvalds]

On Mon, 3 Oct 2005, Junio C Hamano wrote:
> 
> This reminds me of one patch:
> 
>     From: Dan Aloni <da-x@monatomic.org>
>     Subject: [PATCH] Fix git+ssh's indefinite halts during long fetches
>     Date: Sat, 1 Oct 2005 21:39:42 +0300
>     Message-ID: <20051001183942.GA2099@localdomain>
> 
> I'd appreciate it if you had a chance to take a look at it and
> comment on it.

I personally hate it.

It adds horrible patches to fairly core stuff, all because the prefetching 
is not limited.

As far as I can tell, it should be much easier to just limit the 
prefetching to some reasonable limit (say, a few objects deep), which 
guarantees that the prefetching doesn't fill up the write queues on the 
fetching side.

It's not like prefetching improves performance once you get to the point 
where you can stream. I suspect having more than two or three objects "in 
flight" really only helps with

 - lots of small objects
 - high latency
 - high bandwidth

and the thing is, high latency together with high bandwidth is really 
quite uncommon - usually high latency goes along with _low_ bandwidth (the 
one exception is things like satellite links, which can have latencies in 
the seconds, even with good throughput).

It should be pretty easy to benchmark, but my _suspicion_ is that limiting 
the read-ahead to even just five is likely to get you 99% of the way, and 
that the performance impact of going higher is very limited.

(It might need some extra code to make the synchronous receiving side 
re-start the prefetching if the prefetching has stopped after a few 
entries - but at that point the extra code is where it is supposed to be, 
rather than having core code work around problems in the fetching. I also 
suspect that the prefetch limiting can happily be done in the generic 
"pull" code, rather than separately for each protocol, so it would need to 
be done in just one place).

		Linus

Re: What to expect after 0.99.8

[Dan Aloni]

On Mon, Oct 03, 2005 at 04:16:27PM -0700, Linus Torvalds wrote:
> 
> 
> On Mon, 3 Oct 2005, Junio C Hamano wrote:
> > 
> > This reminds me of one patch:
> > 
> >     From: Dan Aloni <da-x@monatomic.org>
> >     Subject: [PATCH] Fix git+ssh's indefinite halts during long fetches
> >     Date: Sat, 1 Oct 2005 21:39:42 +0300
> >     Message-ID: <20051001183942.GA2099@localdomain>
> > 
> > I'd appreciate it if you had a chance to take a look at it and
> > comment on it.
> 
> I personally hate it.
> 
> It adds horrible patches to fairly core stuff, all because the prefetching 
> is not limited.

Well it can be reworked to be more clean...
 
> As far as I can tell, it should be much easier to just limit the 
> prefetching to some reasonable limit (say, a few objects deep), which 
> guarantees that the prefetching doesn't fill up the write queues on the 
> fetching side.

I'm not sure how this will be completely reliable, even if you limit the
prefetching to one object.

Suppose that this one object's size is larger than the receiving queues of 
the receiving end (like 1 MB?) and the bandwidth is high, wouldn't that 
break?

-- 
Dan Aloni
da-x@monatomic.org, da-x@colinux.org, da-x@gmx.net

Re: What to expect after 0.99.8

[Daniel Barkalow]

On Tue, 4 Oct 2005, Dan Aloni wrote:

> On Mon, Oct 03, 2005 at 04:16:27PM -0700, Linus Torvalds wrote:
> > 
> > 
> > On Mon, 3 Oct 2005, Junio C Hamano wrote:
> > > 
> > > This reminds me of one patch:
> > > 
> > >     From: Dan Aloni <da-x@monatomic.org>
> > >     Subject: [PATCH] Fix git+ssh's indefinite halts during long fetches
> > >     Date: Sat, 1 Oct 2005 21:39:42 +0300
> > >     Message-ID: <20051001183942.GA2099@localdomain>
> > > 
> > > I'd appreciate it if you had a chance to take a look at it and
> > > comment on it.
> > 
> > I personally hate it.
> > 
> > It adds horrible patches to fairly core stuff, all because the prefetching 
> > is not limited.
> 
> Well it can be reworked to be more clean...
>  
> > As far as I can tell, it should be much easier to just limit the 
> > prefetching to some reasonable limit (say, a few objects deep), which 
> > guarantees that the prefetching doesn't fill up the write queues on the 
> > fetching side.
> 
> I'm not sure how this will be completely reliable, even if you limit the
> prefetching to one object.
> 
> Suppose that this one object's size is larger than the receiving queues of 
> the receiving end (like 1 MB?) and the bandwidth is high, wouldn't that 
> break?

It shouldn't cause any problem, unless there isn't a 4K buffer between the 
git-ssh-fetch and ssh; the fetch side would have to fill this buffer 
before getting stuck, even if ssh can't send out any more data until the 
object has been read, and 100 requests (each 21 bytes) wouldn't be enough. 
I remember that there's a lot that depends on being able to put 4K into an 
empty pipe without blocking, and I'd guess that UNIX sockets have a 
similar capacity (although I'm not going to look it up tonight).

	-Daniel
*This .sig left intentionally blank*

Re: What to expect after 0.99.8

[Matthias Urlichs]

Hi, Daniel Barkalow wrote:

> I'd guess that UNIX sockets have a
> similar capacity (although I'm not going to look it up tonight).

You can set TCP options to change the buffer sizes.

I would however assume that *nobody* sets both the send and receive
buffers such that their cumulative size is <4k, so 99 object IDs
at 41 bytes definitely should be OK.

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
I was going to include an ethnic slur in here, but I couldn't figure out how
to get you into this file.

Re: What to expect after 0.99.8

[H. Peter Anvin]

Matthias Urlichs wrote:
> Hi, Daniel Barkalow wrote:
> 
>>I'd guess that UNIX sockets have a
>>similar capacity (although I'm not going to look it up tonight).
> 
> You can set TCP options to change the buffer sizes.
> 
> I would however assume that *nobody* sets both the send and receive
> buffers such that their cumulative size is <4k, so 99 object IDs
> at 41 bytes definitely should be OK.
> 

For TCP, I think we should simply get our own (or set) packet buffer 
size and conform to it.  Problem solved...

	-hpa

Re: What to expect after 0.99.8

[Matthias Urlichs]

Hi,

H. Peter Anvin:
> >I would however assume that *nobody* sets both the send and receive
> >buffers such that their cumulative size is <4k, so 99 object IDs
> >at 41 bytes definitely should be OK.
> 
> For TCP, I think we should simply get our own (or set) packet buffer 
> size and conform to it.  Problem solved...

Actually, it isn't -- if you have a ssh connection, you don't have
access to the raw TCP socket.

Just limit the number of objects in flight to 20 or so.
Problem also solved.  ;-)

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
He hated to set precedents; those who did so were sometimes promoted, more
frequently they joined their ancestors.
					-- Robert A. Heinlein

Re: What to expect after 0.99.8

[H. Peter Anvin]

Matthias Urlichs wrote:
> 
> Actually, it isn't -- if you have a ssh connection, you don't have
> access to the raw TCP socket.
> 

If you have an ssh connection, you're writing over a pipe to the ssh 
process, and your local buffer is that pipe, which is PIPE_BUF size.

	-hpa

Re: What to expect after 0.99.8

[Junio C Hamano]

"H. Peter Anvin" <hpa@zytor.com> writes:

> If you have an ssh connection, you're writing over a pipe to the ssh 
> process, and your local buffer is that pipe, which is PIPE_BUF size.

I vaguely recall there was an interesting regression in recent
kernel history when the implementation of the pipe buffer was
changed, with which, writing the same amount of data with
different number of writes made things behave differently and
making the worst case buffer size less than traditional 4K.

I wonder if we are going to be bitten by that one...

Re: What to expect after 0.99.8

[Linus Torvalds]

On Tue, 4 Oct 2005, Junio C Hamano wrote:
> 
> I vaguely recall there was an interesting regression in recent
> kernel history when the implementation of the pipe buffer was
> changed, with which, writing the same amount of data with
> different number of writes made things behave differently and
> making the worst case buffer size less than traditional 4K.

Just for performance reasons, I ended up doing merging anyway, so in fact 
regardless of how you write the current pipe buffer size is up to 16 
pages.

I think you can safely assume that pretty much any file descriptor you use 
has at least 1kB of buffer. Even 4kB is likely a "safe assumption", and in 
reality, most of them end up having even more.

With something like ssh, you probably end up having even deeper ones, 
since you end up having the local pty or socket to ssh, and then ssh has 
the TCP buffer to the network..

			Linus

Re: What to expect after 0.99.8

[H. Peter Anvin]

Junio C Hamano wrote:
> "H. Peter Anvin" <hpa@zytor.com> writes:
> 
>>If you have an ssh connection, you're writing over a pipe to the ssh 
>>process, and your local buffer is that pipe, which is PIPE_BUF size.
> 
> I vaguely recall there was an interesting regression in recent
> kernel history when the implementation of the pipe buffer was
> changed, with which, writing the same amount of data with
> different number of writes made things behave differently and
> making the worst case buffer size less than traditional 4K.
> 
> I wonder if we are going to be bitten by that one...
> 

The definition of PIPE_BUF is that a write to a pipe of no more than 
PIPE_BUF bytes will either succeed immediately or block; it will not be 
broken up into multiple writes (with potential interlace problems.)  It 
says *nothing* about what happens with multiple writes.

	-hpa

Re: What to expect after 0.99.8

[Daniel Barkalow]

On Tue, 4 Oct 2005, Matthias Urlichs wrote:

> Hi, Daniel Barkalow wrote:
> 
> > I'd guess that UNIX sockets have a
> > similar capacity (although I'm not going to look it up tonight).
> 
> You can set TCP options to change the buffer sizes.
> 
> I would however assume that *nobody* sets both the send and receive
> buffers such that their cumulative size is <4k, so 99 object IDs
> at 41 bytes definitely should be OK.

I actually mean UNIX (a.k.a. PF_LOCAL) sockets; git-ssh-fetch is connected 
to ssh via sockets from "socketpair()". Looks like you can set the buffer 
size with a socket option here, too, but I doubt ssh will try setting 
socket options on standard in and out, and git-ssh-fetch leaves them at 
their defaults. I'm not clear where the defaults for these come from in 
general.

	-Daniel
*This .sig left intentionally blank*

Re: What to expect after 0.99.8

[H. Peter Anvin]

Daniel Barkalow wrote:
> 
> I actually mean UNIX (a.k.a. PF_LOCAL) sockets; git-ssh-fetch is connected 
> to ssh via sockets from "socketpair()". Looks like you can set the buffer 
> size with a socket option here, too, but I doubt ssh will try setting 
> socket options on standard in and out, and git-ssh-fetch leaves them at 
> their defaults. I'm not clear where the defaults for these come from in 
> general.
> 

Well, git-ssh-fetch could set both SO_SNDBUF and SO_RCVBUF if it cared.

For portability, it would be a good thing to explicitly set the buffer 
size rather than just blindly assume it can hold a specific amount of data.

	-hpa

Re: What to expect after 0.99.8

[Daniel Barkalow]

On Mon, 3 Oct 2005, Junio C Hamano wrote:

> Daniel Barkalow <barkalow@iabervon.org> writes:
> 
> > The code in ssh-fetch already does parallel fetching, actually (only over 
> > one connection, but requests are sent before responses are read), so 
> > multiple requests are in progress at the same time.
> 
> This reminds me of one patch:
> 
>     From: Dan Aloni <da-x@monatomic.org>
>     Subject: [PATCH] Fix git+ssh's indefinite halts during long fetches
>     Date: Sat, 1 Oct 2005 21:39:42 +0300
>     Message-ID: <20051001183942.GA2099@localdomain>
> 
> I'd appreciate it if you had a chance to take a look at it and
> comment on it.

I think it's overly hacky; we should be able, in prefetch, to check 
whether we've stuffed in a lot of hashes already, and actually read an 
object out before requesting another; there's nothing in the fetch 
contract that says that an object can't become available at some random 
time between the start of the fetch and when it gets requested with 
fetch(). (In fact, I had a pack-exchange version of the ssh stuff which 
would notice that you have certain things and you're looking for a commit, 
and get a bunch of stuff you probably want as a pack before you actually 
ask for it, but Linus beat me on that one with send-pack/upload-pack.)

I think that a limit of 100 objects in transit is about right, because the 
requests for 100 objects fits well within 4K and I expect that we 
commonly have small enough objects that we need to queue up a relatively 
large number of requests to maintain streaming.

I've got a patch, which I'll send in the next email.

	-Daniel
*This .sig left intentionally blank*

Re: What to expect after 0.99.8

[Alan Chandler]

On Monday 03 Oct 2005 01:14, Junio C Hamano wrote:
...
>
> Technical (heavier)
> -------------------
...
> * Look at libified GNU diff CVS seems to use, or libxdiff.
>

I did wonder today in response to a discussion at work about version control 
of documentation, whether git could in some way understand OpenDocument 
format and allow some form of merging of concurrent changes to the same 
document.  I am not sure if just unzipping the files and applying standard 
merge strategies the resultant xml would be good enough.

Sadly just a thought at the moment.


-- 
Alan Chandler
http://www.chandlerfamily.org.uk

Re: What to expect after 0.99.8

[H. Peter Anvin]

Alan Chandler wrote:
> 
> I did wonder today in response to a discussion at work about version control 
> of documentation, whether git could in some way understand OpenDocument 
> format and allow some form of merging of concurrent changes to the same 
> document.  I am not sure if just unzipping the files and applying standard 
> merge strategies the resultant xml would be good enough.
> 
> Sadly just a thought at the moment.
> 

I believe in the medium-to-long term a plugin architecture for merging 
is imperative.  It's not even different media types, but some *files* 
have specific merging policies.  Think, for example, of pci.ids in the 
Linux kernel tree.

	-hpa

Re: What to expect after 0.99.8

[Greg KH]

On Mon, Oct 03, 2005 at 02:08:36PM -0700, H. Peter Anvin wrote:
> 
> I believe in the medium-to-long term a plugin architecture for merging 
> is imperative.  It's not even different media types, but some *files* 
> have specific merging policies.  Think, for example, of pci.ids in the 
> Linux kernel tree.

You mean the file that is now removed from the kernel tree?  Sure,
that's a good merge policy :)

greg k-h

Re: What to expect after 0.99.8

[H. Peter Anvin]

Greg KH wrote:
> On Mon, Oct 03, 2005 at 02:08:36PM -0700, H. Peter Anvin wrote:
> 
>>I believe in the medium-to-long term a plugin architecture for merging 
>>is imperative.  It's not even different media types, but some *files* 
>>have specific merging policies.  Think, for example, of pci.ids in the 
>>Linux kernel tree.
> 
> You mean the file that is now removed from the kernel tree?  Sure,
> that's a good merge policy :)
> 

*Plonk*... well, the point was that even text files can be structured so 
that application-specific merge rules can be made to apply.  ChangeLogs 
are another good example.

	-hpa

Re: What to expect after 0.99.8

[Horst von Brand]

Alan Chandler <alan@chandlerfamily.org.uk> wrote:
> On Monday 03 Oct 2005 01:14, Junio C Hamano wrote:
> ...
> >
> > Technical (heavier)
> > -------------------
> ...
> > * Look at libified GNU diff CVS seems to use, or libxdiff.

> I did wonder today in response to a discussion at work about version control 
> of documentation, whether git could in some way understand OpenDocument 
> format and allow some form of merging of concurrent changes to the same 
> document.  I am not sure if just unzipping the files and applying standard 
> merge strategies the resultant xml would be good enough.

A student of mine worked on layering SCM under a code design tool that uses
XML as its representation of the project it is working on. He started with
CVS, and ended up grafting a problem-specific merge strategy on top. The
standard merge from CVS just messed everything up. Dunno if this is very
relevant for OASIS, but I suspect you'd see the same.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: What to expect after 0.99.8

[Matthias Urlichs]

Hi, Junio C Hamano wrote:

> * Libification.  There are many places "run once" mentality is
>   ingrained in the management of basic data structures, which need to be
>   fixed.

I have started work on doing this "the right way"
(as per earlier discussion).

Current status: There's a toplevel "struct git_env", an associated "struct
git_objdb", and (thread-safe and globals-free) library code to read
sha1-identified object (meta)data, including packs and all.

http://netz.smurf.noris.de/git/git.git#libize

Next on my TODO list: introduce a "struct git_obj" which represents
exactly one sha1 and the metadata associated with it, rename the
accessor functions to be more consistent, add SWIG interface code and
Python testcases, submit to everybody's scrutinity.

After that, the task can hopefully be parallelized.

Definitely a post-1.0 job; the job is too big, and shipping 1.0 with a
partial library that doesn't do much that's useful does not make sense.

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
We've sent a man to the moon, and that's 29,000 miles away.  The center
of the Earth is only 4,000 miles away.  You could drive that in a week,
but for some reason nobody's ever done it.
		-- Andy Rooney

Re: What to expect after 0.99.8

[Chuck Lever]

[-- Attachment #1: Type: text/plain, Size: 7260 bytes --]

two quick notes:

1.  git-update-ref has no documentation (i don't have time to sit down 
and construct it, otherwise i'd post a patch).

2.  what is your thinking about including the cache abstraction layer 
after 1.0 ?  i think it would help the libification effort.


Junio C Hamano wrote:
> As I mentioned in teh 0.99.8 announcement, let's start aiming
> for 1.0, really this time.  From now on, brown paper bags,
> bugfixes, portability fixes, usability enhancements including
> documentation updates take precedence over any new features.
> One exception area is probably merge strategy modules -- they
> are like adding new device drivers or adding new filesystem, and
> can come in anytime as long as they do not touch the coreish.
> 
> 
> The GIT To-Do File
> ==================
> 
>   The latest copy of this document is found at 
> 
>     http://kernel.org/git/?p=git/git.git;a=blob;hb=todo;f=TODO
> 
> 
> Tool Renames Plan
> =================
> 
>  - In 0.99.8, we still install the backward compatible symbolic
>    links in $(bindir).  These will however be removed before 1.0
>    happens.
> 
>    git-ssh-push and git-ssh-pull pair is not going away within
>    this timeframe, if ever.  Each of these old-name commands
>    continues to invoke its old-name counterpart on the other
>    end.
> 
> 
> What to expect after 0.99.8
> ===========================
> 
> This is written in a form of to-do list for me, so if I say
> "accept patch", it means I do not currently plan to do that
> myself.  People interested in seeing it materialize please take
> a hint.
> 
> 
> Documentation
> -------------
> 
> * Accept patches from people who actually have done CVS
>   migration and update the cvs-migration documentation.
>   Link the documentation from the main git.txt page.
> 
> * Talk about using rsync just once at the beginning when
>   initializing a remote repository so that local packs do not
>   need to be expanded.  I personally do not think we need tool
>   support for this (but see below about optimized cloning).
> 
> * Maybe update tutorial with a toy project that involves two or
>   three developers..
> 
> * Update tutorial to cover setting up repository hooks to do
>   common tasks.
> 
> * Accept patches to finish missing docs.
> 
> * Accept patches to talk about "Whoops, it broke.  What's
>   next?".
> 
> * Accept patches to make formatted tables in asciidoc to work
>   well in both html and man pages (see git-diff(1)).
> 
> 
> Technical (heavier)
> -------------------
> 
> * We might want to optimize cloning with GIT native transport
>   not to explode the pack, and store it in objects/pack instead.
>   We would need a tool to generate an idx file out of a pack
>   file for this.  Also this itself may turn out to be a bad
>   idea, making the set of packs in repositories everybody has
>   different from each other.
> 
> * Git daemon, when deployed at kernel.org, might turn out to be
>   quite a burden, since it needs to generate customized packs
>   every time a new request comes in.  It may be worthwhile to
>   precompute some packs for popular sets of heads downloaders
>   have and serve that, even if that could give more than the
>   client asks for in some cases.  We will know about this soon
>   enough.
> 
> * Libification.  There are many places "run once" mentality is
>   ingrained in the management of basic data structures, which
>   need to be fixed.
> 
> * Maybe a pack optimizer.
> 
> * Maybe an Emacs VC backend.
> 
> * 'git split-projects'?  This requires updated 'git-rev-list' to
>   skip irrelevant commits.
>   Message-ID: <Pine.LNX.4.63.0509221617300.23242@iabervon.org>
> 
> * Look at libified GNU diff CVS seems to use, or libxdiff.
> 
> 
> Technical (milder)
> ------------------
> 
> * Encourage concrete proposals to commit log message templates
>   we discussed some time ago.
> 
> * Accept patches to cause "read-tree -u" delete a directory when
>   it makes it empty.
> 
> * Perhaps accept patches to introduce the concept of "patch flow
>   expressed as ref mappings" Josef has been advocating about.
> 
> * Perhaps accept patches to do undo/redo.
> 
> * Perhaps accept patch to optionally allow '--fuzz' in
>   'git-apply'.
> 
> * Allow 'git apply' to accept GNU diff 2.7 output that forgets
>   to say '\No newline' if both input ends with incomplete
>   lines.
> 
> * Maybe grok PGP signed text/plain in applymbox as well.
> 
> * Perhaps a tool to revert a single file to pre-modification
>   state?  People with BK background know this operation as
>   'clean'.  'git checkout [-f] ent [path...]' was suggested by
>   Matthias Urlichs which sounds a natural extention to what the
>   command currently does.
> 
> * Enhance "git repack" to not always use --all; this would be
>   handy if the repository contains wagging heads like "pu" in
>   git.git repository.
> 
> * Internally split the project into non-doc and doc parts; add
>   an extra root for the doc part and merge from it; move the
>   internal doc source to a separate repository, like the +Meta
>   repository; experiment if this results in a reasonable
>   workflow, and document it in howto form if it does.
> 
> * Make rebase restartable; instead of skipping what cannot be
>   automatically forward ported, leave the conflicts in the work
>   tree, have the user resolve it, and then restart from where it
>   left off.
> 
> * Output full path in the "git-rev-list --objects" output, not
>   just the basename, and see the improved clustering results in
>   better packing [Tried, but did not work out well].
> 
> * Updated git-changes-script Jeff Garzik needs [Inquiry for
>   external spec sent out with a quick hack.  Will know if that
>   is what he needs soon enough].
> 
> 
> Technical (trivial)
> -------------------
> 
> * short SHA1 naming is not enforcing uniqueness.  Should fix.
> 
> * 'git repack' can be DOSed.  Should fix.
> 
> * Stop installing the old-name symlinks [POSTPONED].
> 
> * 'git merge-projects'?
> 
> * 'git lost-and-found'?  Link dangling commits found by
>   fsck-objects under $GIT_DIR/refs/lost-found/.  Then
>   show-branch or gitk can be used to find any lost commit. [A
>   feeler patch sent out. Very underwhelming response X-<.]
> 
>   Do not name it /lost+found/; that would probably confuse
>   things that mistake it a mount point (not our code but
>   somebody else's).
> 
> * Add simple globbing rules to git-show-branch so that I can
>   say 'git show-branch --heads "ko-*"' (ko-master, ko-pu, and
>   ko-rc are in refs/tags/).
> 
> * We would want test scripts for the relative directory path
>   stuff Linus has been working on.  So far, the following
>   commands should be usable with relative directory paths:
> 
>     git-update-index
>     git-ls-files
>     git-diff-files
>     git-diff-index
>     git-diff-tree
>     git-rev-list
>     git-rev-parse
> 
> * In a freashly created empty repository, `git fetch foo:bar`
>   works OK, but `git checkout bar` afterwards does not (missing
>   `.git/HEAD`).
> 
> \f
> Local Variables:
> mode: text
> End:
> 
> -
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


[-- Attachment #2: cel.vcf --]
[-- Type: text/x-vcard, Size: 439 bytes --]

begin:vcard
fn:Chuck Lever
n:Lever;Charles
org:Network Appliance, Incorporated;Linux NFS Client Development
adr:535 West William Street, Suite 3100;;Center for Information Technology Integration;Ann Arbor;MI;48103-4943;USA
email;internet:cel@citi.umich.edu
title:Member of Technical Staff
tel;work:+1 734 763-4415
tel;fax:+1 734 763 4434
tel;home:+1 734 668-1089
x-mozilla-html:FALSE
url:http://www.monkey.org/~cel/
version:2.1
end:vcard

Re: What to expect after 0.99.8

[Junio C Hamano]

Chuck Lever <cel@citi.umich.edu> writes:

> two quick notes:
>
> 1.  git-update-ref has no documentation (i don't have time to sit down 
> and construct it, otherwise i'd post a patch).

Thanks; neither git-symbolic-ref.  I'll write them up.

> 2.  what is your thinking about including the cache abstraction layer 
> after 1.0 ?  i think it would help the libification effort.

I haven't had a chance to look at the code Smurf is working on,
but I suspect that your cache abstraction work would interact
badly with it if done independently and made into mainline
first, and would end up requiring some parts of libification to
be redone.

    From: Matthias Urlichs <smurf@smurf.noris.de>
    Date: Mon, 03 Oct 2005 22:48:54 +0200
    Message-ID: <pan.2005.10.03.20.48.52.132570@smurf.noris.de>

    I have started work on doing this "the right way"
    (as per earlier discussion).

    Current status: There's a toplevel "struct git_env", an associated "struct
    git_objdb", and (thread-safe and globals-free) library code to read
    sha1-identified object (meta)data, including packs and all.

    http://netz.smurf.noris.de/git/git.git#libize

    Next on my TODO list: introduce a "struct git_obj" which represents
    exactly one sha1 and the metadata associated with it, rename the
    accessor functions to be more consistent, add SWIG interface code and
    Python testcases, submit to everybody's scrutinity.

    After that, the task can hopefully be parallelized.

    Definitely a post-1.0 job; the job is too big, and shipping 1.0 with a
    partial library that doesn't do much that's useful does not make sense.

So if you have energy and inclination, I'd like to see you take
a look at the smurf tree, and if you can work with him and add
cache abstraction as part of the libification that would be the
ideal approach.  Then hopefully nobody has to do work twice.