What's cooking in git.git (Aug 2008, #07; Sat, 23)

What's cooking in git.git (Aug 2008, #07; Sat, 23)

[Junio C Hamano]

Here are the topics that have been cooking.  Commits prefixed
with '-' are only in 'pu' while commits prefixed with '+' are
in 'next'.

The topics list the commits in reverse chronological order.  The topics
meant to be merged to the maintenance series have "maint-" in their names.

Here is a list of issues/topics we saw on the mailing list but haven't
resulted in anything queuable in 'pu' yet.  They need further work by
interested parties:

 * Windows relocatable install

   Steffen Prohaska ($gmane/92605), Johannes Sixt.

 * Haiku port

   Andreas Färber ($gmane/92582)

 * ksh "trap foo EXIT" triggers on function return, loses exit status

   Brandon Casey ($gmane/92873)

 * document webdav debugging tip with davfs2

   Giovanni Funchal ($gmane/92745)

 * update "rebase -i" documentation with examples

   Eric Hanchrow ($gmane/92669)

 * pre-push hook

   Scott Chacon ($gmane/92900, $gmane/92936)

 * "git commit --author=nickname" expanding nickname from somewhere

   Michael J Gruber ($gmane/93274)

 * Handling (possibly $HOME-) relative paths in config files

   Karl Chen ($gmane/93250)

 * "submodule sync"

   David Aguilar ($gmane/93265)

 * "rev-list --bisect --first-parent"

   Avery Pennarun, me ($gmane/93420)

 * "apply --include"

   Joe Perches ($gmane/93505)

----------------------------------------------------------------
[New Topics]

* bd/blame (Thu Aug 21 18:22:01 2008 -0500) 5 commits
 . Use xdiff caching to improve git blame performance
 . Allow xdiff machinery to cache hash results for a file
 . Always initialize xpparam_t to 0
 . Bypass textual patch generation and parsing in git blame
 . Allow alternate "low-level" emit function from xdl_diff

Réne had a good comments on how the callback is structured.

* jc/maint-name-hash-clear (Sat Aug 23 13:05:10 2008 -0700) 1 commit
 - discard_cache: reset lazy name_hash bit

I spotted this by accident while working on something unrelated.

When a program calls discard_cache() to read the index again, we do not
properly re-initialize the name_hash structure that is used by the case
insensitivitly logic.  This _might_ improve issues people may be having on
case insensitive filesystems.  I dunno.

* mv/maint-merge-fix (Sat Aug 23 12:56:57 2008 -0700) 1 commit
 + merge: fix numerus bugs around "trivial merge" area

* ml/submodule (Thu Aug 21 19:54:01 2008 -0400) 2 commits
 + git-submodule.sh - Remove trailing / from URL if found
 + git-submodule.sh - Remove trailing / from URL if found

Soon to be in 'master', I guess.

* np/verify-pack (Fri Aug 22 15:45:53 2008 -0400) 1 commit
 + discard revindex data when pack list changes

* jc/add-ita (Thu Aug 21 01:44:53 2008 -0700) 3 commits
 - git-add --intent-to-add (-N)
 - cached_object: learn empty blob
 - sha1_object_info(): pay attention to cached objects

Teaches "git add" to record only the intent to add a path later.
I think this is better done without the hardcoded empty blob object.

----------------------------------------------------------------
[Stalled -- Needs Action to Proceed (or to be dropped)]

* sb/daemon (Thu Aug 14 20:02:20 2008 +0200) 4 commits
 - git-daemon: rewrite kindergarden, new option --max-connections
 - git-daemon: Simplify dead-children reaping logic
 - git-daemon: use LOG_PID, simplify logging code
 - git-daemon: call logerror() instead of error()

Can somebody who actually runs the daemon standalone comment on this one?

* jc/cc-ld-dynpath (Sat Aug 16 15:01:23 2008 +0200) 2 commits
 - configure: auto detect dynamic library path switches
 - Makefile: Allow CC_LD_DYNPATH to be overriden

Needs success reports from people who do use user-defined dynamic library
path when they build their "git" before this series can go anywhere.

* lt/time-reject-fractional-seconds (Sat Aug 16 21:25:40 2008 -0700) 1 commit
 - date/time: do not get confused by fractional seconds

Linus hints further enhancements as "the right way", so let's see if
somebody else steps up and tries it before merging this to 'next'.

* sp/smart-http (Sun Aug 3 00:25:17 2008 -0700) 2 commits
 - [do not merge -- original version] Add Git-aware CGI for Git-aware
   smart HTTP transport
 - Add backdoor options to receive-pack for use in Git-aware CGI

The "magic" detection protocol was revised to use POST to info/refs; the
top one queued is from before that discussion.

----------------------------------------------------------------
[Actively Cooking]

* cc/bisect (Fri Aug 22 05:52:29 2008 +0200) 2 commits
 - bisect: only check merge bases when needed
 - bisect: test merge base if good rev is not an ancestor of bad rev

* mv/merge-recursive (Tue Aug 12 22:14:00 2008 +0200) 3 commits
 . Make builtin-revert.c use merge_recursive_generic()
 . merge-recursive.c: Add more generic merge_recursive_generic()
 . Split out merge_recursive() to merge-recursive.c

Miklos will be working on updates based on comments.

* lw/gitweb (Mon Aug 18 21:39:49 2008 +0200) 3 commits
 . gitweb: use new Git::Repo API, and add optional caching
 . add new Perl API: Git::Repo, Git::Commit, Git::Tag, and
   Git::RepoRoot
 . gitweb: add test suite with Test::WWW::Mechanize::CGI

Tentatively dropped as its tests do not seem to pass and I have no time to
look at them.

* jc/diff-prefix (Mon Aug 18 20:08:09 2008 -0700) 1 commit
 - diff: vary default prefix depending on what are compared

As some people may have noticed, I've been running with this one when
sending out "How about this" patches to the discussion threads.

* sp/missing-thin-base (Tue Aug 12 11:31:06 2008 -0700) 1 commit
 + pack-objects: Allow missing base objects when creating thin packs

* tr/filter-branch (Tue Aug 12 10:45:59 2008 +0200) 3 commits
 + filter-branch: use --simplify-merges
 + filter-branch: fix ref rewriting with --subdirectory-filter
 + filter-branch: Extend test to show rewriting bug

Fixes a longstanding filter branch bug.  Success stories?

* jc/post-simplify (Fri Aug 15 01:34:51 2008 -0700) 8 commits
 - revision --simplify-merges: incremental simplification
 - revision --simplify-merges: prepare for incremental simplification
 - revision --simplify-merges: make it a no-op without pathspec
 + revision --simplify-merges: do not leave commits unprocessed
 + revision --simplify-merges: use decoration instead of commit->util
   field
 + Topo-sort before --simplify-merges
 + revision traversal: show full history with merge simplification
 + revision.c: whitespace fix

"log --full-history" is with too much clutter, "log" itself is too cleverer
than some people, and here is the middle level of merge simplification.

I started making this incremental but the progress is not so great.

* tr/rev-list-docs (Tue Aug 12 01:55:37 2008 +0200) 1 commit
 + Documentation: rev-list-options: move --simplify-merges
   documentation

----------------------------------------------------------------
[Will merge to master soon]

* jc/no-slim-shell (Tue Aug 19 18:05:43 2008 -0700) 2 commits
 + Build-in "git-shell"
 + shell: do not play duplicated definition games to shrink the
   executable

* mv/merge-custom (Sat Aug 23 19:23:22 2008 -0700) 9 commits
 + t7606: fix custom merge test
 + Fix "git-merge -s bogo" help text
 + Update .gitignore to ignore git-help
 + Builtin git-help.
 + builtin-help: always load_command_list() in cmd_help()
 + Add a second testcase for handling invalid strategies in git-merge
 + Add a new test for using a custom merge strategy
 + builtin-merge: allow using a custom strategy
 + builtin-help: make some internal functions available to other
   builtins

The one at the tip fixes a test that assumed git-merge has a broken
"trivial merge" implementation.

* jc/add-addremove (Tue Jul 22 22:30:40 2008 -0700) 2 commits
 + builtin-add.c: optimize -A option and "git add ."
 + builtin-add.c: restructure the code for maintainability

* am/cherry-pick-rerere (Sun Aug 10 17:18:55 2008 +0530) 1 commit
 + Make cherry-pick use rerere for conflict resolution.

----------------------------------------------------------------
[On Hold]

* jc/stripspace (Sun Mar 9 00:30:35 2008 -0800) 6 commits
 - git-am --forge: add Signed-off-by: line for the author
 - git-am: clean-up Signed-off-by: lines
 - stripspace: add --log-clean option to clean up signed-off-by:
   lines
 - stripspace: use parse_options()
 - Add "git am -s" test
 - git-am: refactor code to add signed-off-by line for the committer

* jc/send-pack-tell-me-more (Thu Mar 20 00:44:11 2008 -0700) 1 commit
 - "git push": tellme-more protocol extension

* jc/merge-whitespace (Sun Feb 24 23:29:36 2008 -0800) 1 commit
 - WIP: start teaching the --whitespace=fix to merge machinery

* jc/blame (Wed Jun 4 22:58:40 2008 -0700) 2 commits
 - blame: show "previous" information in --porcelain/--incremental
   format
 - git-blame: refactor code to emit "porcelain format" output

* sg/merge-options (Sun Apr 6 03:23:47 2008 +0200) 1 commit
 + merge: remove deprecated summary and diffstat options and config
   variables

This was previously in "will be in master soon" category, but it turns out
that the synonyms to the ones this one deletes are fairly new invention
that happend in 1.5.6 timeframe, and we cannot do this just yet.  Perhaps
in 1.7.0.

* jc/dashless (Wed Jun 25 15:55:11 2008 -0700) 1 commit
 - Make clients ask for "git program" over ssh and local transport

This is the "botched" one.  Will be resurrected during 1.7.0 or 1.8.0
timeframe.

* jk/renamelimit (Sat May 3 13:58:42 2008 -0700) 1 commit
 - diff: enable "too large a rename" warning when -M/-C is explicitly
   asked for

This would be the right thing to do for command line use, but gitk will be
hit due to tcl/tk's limitation, so I am holding this back for now.

----------------------------------------------------------------
[Graduated to "master"]

* ml/submodule-foreach (Sun Aug 10 19:10:04 2008 -0400) 1 commit
 + git-submodule - Add 'foreach' subcommand

* pm/log-exit-code (Mon Aug 11 08:46:25 2008 +0200) 2 commits
 + Teach git log --exit-code to return an appropriate exit code
 + Teach git log --check to return an appropriate exit code

* sb/commit-tree-minileak (Tue Aug 12 00:35:11 2008 +0200) 1 commit
 + Fix commit_tree() buffer leak

* pb/reflog-dwim (Sun Aug 10 22:22:21 2008 +0200) 1 commit
 + builtin-reflog: Allow reflog expire to name partial ref

* jc/add-stop-at-symlink (Mon Aug 4 00:52:37 2008 -0700) 2 commits
 + add: refuse to add working tree items beyond symlinks
 + update-index: refuse to add working tree items beyond symlinks

Fix for a longstanding bug that allows "git add" and "git update-index" to
add a path "a/b" to the index when "a" is a symbolic link.  We would need
a similar fix for the case where "a" is a submodule.

* kh/diff-tree (Sun Aug 10 18:13:04 2008 +0200) 4 commits
 + Add test for diff-tree --stdin with two trees
 + Teach git diff-tree --stdin to diff trees
 + diff-tree: Note that the commit ID is printed with --stdin
 + Refactoring: Split up diff_tree_stdin

* mg/count-objects (Fri Aug 15 00:20:20 2008 -0400) 1 commit
 + count-objects: Add total pack size to verbose output

This one is without the human readable bits.

* mz/push-verbose (Sat Aug 16 19:58:32 2008 +0200) 1 commit
 + Make push more verbose about illegal combination of options

* jc/index-extended-flags (Sat Aug 16 23:02:08 2008 -0700) 1 commit
 + index: future proof for "extended" index entries

* cc/merge-base-many (Sun Jul 27 13:47:22 2008 -0700) 4 commits
 + git-merge-octopus: use (merge-base A (merge B C D E...)) for
   stepwise merge
 + merge-base-many: add trivial tests based on the documentation
 + documentation: merge-base: explain "git merge-base" with more than
   2 args
 + merge-base: teach "git merge-base" to drive underlying
   merge_bases_many()

* js/parallel-test (Mon Aug 18 12:25:40 2008 -0400) 4 commits
 + Update t/.gitignore to ignore all trash directories
 + Enable parallel tests
 + tests: Clarify dependencies between tests, 'aggregate-results' and
   'clean'
 + t9700: remove useless check

* jc/test-deeper (Fri Aug 8 02:26:28 2008 -0700) 1 commit
 + tests: use $TEST_DIRECTORY to refer to the t/ directory

This does not actually move "t/test directory" any deeper, but fixes test
scripts that assume they run immediately below "t/" to use TEST_DIRECTORY
variable.

* js/mingw-stat (Mon Aug 18 22:01:06 2008 +0200) 2 commits
 + Revert "Windows: Use a customized struct stat that also has the
   st_blocks member."
 + compat: introduce on_disk_bytes()

This gets rid of use of st_blocks member (which is XSI but not POSIX
proper), which was originally prompted by recent Haiku port but it turns
out MinGW has the same issue as well.  Queued on 'pu' just to have a
chance to make sure I munged the version j6t sent me correctly before
merging it upwards.

* js/checkout-dwim-local (Sat Aug 9 16:00:12 2008 +0200) 1 commit
 + checkout --track: make up a sensible branch name if '-b' was
   omitted

Alex has update to dwim "checkout --track remotes/origin/hack" as well.

* bd/diff-strbuf (Wed Aug 13 23:18:22 2008 -0700) 3 commits
 + xdiff-interface: hide the whole "xdiff_emit_state" business from
   the caller
 + Use strbuf for struct xdiff_emit_state's remainder
 + Make xdi_diff_outf interface for running xdiff_outf diffs

Gives measurable performance improvement to textual diff generation.  For
improving "blame" performance, it might be more effective to hook directly
to lower level of xdiff machinery so that we do not even have to generate
patch only to discard after reading "@@ -l,k +m,n @@" lines, but that
would be a separate topic.

* dp/hash-literally (Sun Aug 3 18:36:22 2008 +0400) 6 commits
 + add --no-filters option to git hash-object
 + add --path option to git hash-object
 + use parse_options() in git hash-object
 + correct usage help string for git-hash-object
 + correct argument checking test for git hash-object
 + teach index_fd to work with pipes

Gives a bit more flexibility to hash-objects by allowing us to lie about
the path the contents comes from.

* rs/imap (Wed Jul 9 22:29:02 2008 +0100) 5 commits
 + Documentation: Improve documentation for git-imap-send(1)
 + imap-send.c: more style fixes
 + imap-send.c: style fixes
 + git-imap-send: Support SSL
 + git-imap-send: Allow the program to be run from subdirectories of
   a git tree

Some people seem to prefer having this feature available also with gnutls.
Such an enhancement can be done in-tree on top of this series if they are
so inclined.

* jk/pager-swap (Tue Jul 22 03:14:12 2008 -0400) 2 commits
 + spawn pager via run_command interface
 + run-command: add pre-exec callback

This changes the parent-child relationship between the pager and the git
process.  We used to make pager the parent which meant that the exit
status from git is lost from the caller.

* ph/enable-threaded (Mon Jul 21 11:23:43 2008 +0200) 1 commit
 + Enable threaded delta search on *BSD and Linux.

Maintaining "needswork" section of "What's (not) cooking"

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Here is a list of issues/topics we saw on the mailing list but haven't
> resulted in anything queuable in 'pu' yet.  They need further work by
> interested parties:
> ...
> [Stalled -- Needs Action to Proceed (or to be dropped)]
> ...

I am still experimenting with my own workflow with these parts.

With more people on the list sending patches recently, I still have enough
mental bandwidth to reject the initial rounds with comments/suggestions
for improvements, but I do not think it is realistic to expect me to keep
track of all of their progress on re-submission of improvements anymore (I
used to prod people privately asking how much they are making progress
after not hearing from people who received review comments).

It is a large loss for all of us when people do not come back with updated
patches after receiving review comments.  Lost are their good idea based
on the real world needs.  Time other people have volunteered to review
their initial submissions are also wasted when that happens.

I however do not intend to lower the patch acceptance standards.  It is
not a workable approach to accept many new features implemented in a
substandard way or designed incoherently, expecting they will eventually
be cleaned up in-tree.  Nobody will clean anything up after it is merged,
and when we give something to the end users, it becomes harder to change
its behaviour.

If a few people can volunteer to maintain a list that looks like the above
(not-quite) quoted ones, we may be able to give people more incentive to
keep polishing their patches in response to the reviews they received.
Other people can also offer help in topics they are interested in, and we
may see more topics through their completion as the result.

Thoughts?
.

Re: What's cooking in git.git (Aug 2008, #07; Sat, 23)

[Johannes Schindelin]

Hi,

On Sat, 23 Aug 2008, Junio C Hamano wrote:

> ----------------------------------------------------------------
> [Stalled -- Needs Action to Proceed (or to be dropped)]
> 
> * sb/daemon (Thu Aug 14 20:02:20 2008 +0200) 4 commits
>  - git-daemon: rewrite kindergarden, new option --max-connections
>  - git-daemon: Simplify dead-children reaping logic
>  - git-daemon: use LOG_PID, simplify logging code
>  - git-daemon: call logerror() instead of error()
> 
> Can somebody who actually runs the daemon standalone comment on this 
> one?

I am somewhat uneasy about running my production machine with these 
changes, since the last commit (the one introducing a kindergarden with 
cradles) is too unobvious for me.

Ciao,
Dscho

Re: What's cooking in git.git (Aug 2008, #07; Sat, 23)

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> * sb/daemon (Thu Aug 14 20:02:20 2008 +0200) 4 commits
>>  - git-daemon: rewrite kindergarden, new option --max-connections
>>  - git-daemon: Simplify dead-children reaping logic
>>  - git-daemon: use LOG_PID, simplify logging code
>>  - git-daemon: call logerror() instead of error()
>> 
>> Can somebody who actually runs the daemon standalone comment on this 
>> one?
>
> I am somewhat uneasy about running my production machine with these 
> changes, since the last commit (the one introducing a kindergarden with 
> cradles) is too unobvious for me.

Well, I didn't ask anybody to _run_ it.

I asked people who care about their daemons to comment on the change, so
that if there are any issues in the code that I didn't see, the breakage
gets caught before it propagates to their daemons they build from 'next'
or 'master'.

Having said that, I do agree that the kindergarden change is very involved
(it removes more code than it adds --- the reduction of lines is somewhat
inflated because quite a lot of comments that have become stale gets
removed by the patch).

Re: What's cooking in git.git (Aug 2008, #07; Sat, 23)

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
>>> * sb/daemon (Thu Aug 14 20:02:20 2008 +0200) 4 commits
>>>  - git-daemon: rewrite kindergarden, new option --max-connections
>>>  - git-daemon: Simplify dead-children reaping logic
>>>  - git-daemon: use LOG_PID, simplify logging code
>>>  - git-daemon: call logerror() instead of error()
>>> 
>>> Can somebody who actually runs the daemon standalone comment on this 
>>> one?
>>
>> I am somewhat uneasy about running my production machine with these 
>> changes, since the last commit (the one introducing a kindergarden with 
>> cradles) is too unobvious for me.
>
> Well, I didn't ask anybody to _run_ it.
>
> I asked people who care about their daemons to comment on the change, so
> that if there are any issues in the code that I didn't see, the breakage
> gets caught before it propagates to their daemons they build from 'next'
> or 'master'.
>
> Having said that, I do agree that the kindergarden change is very involved
> (it removes more code than it adds --- the reduction of lines is somewhat
> inflated because quite a lot of comments that have become stale gets
> removed by the patch).

Ok, so I looked at the patch again (sorry, Stephen, for keeping this in a
limbo for very long).

First of all, I have to say that I think this patch is reasonably well
done.  I do see an unnecessary check of xcalloc() return value and some
style differences from what git uses, but that is only an indication of
the author not very familiar with git code, and every other aspect of the
change shows that this is quite competently done.  The list traversal with
cradle/blanket/newborn, except using perhaps too cute names for some
people's tastes, makes sense.

I however do have one issue with the logic, though.

It seems that kill_some_child() will not kill anything if nobody else is
coming from the same address, while the old code did kill some.  Is this
intended?

By the way, add_child() compares the whole "struct sockaddr_storage" in
order to queue the newborn in front of an existing connection from the
same address, and kill_some_child() takes advanrage of this to kill the
newest connection ("We kill the newest" comment should probably be moved
to add_child() to describe why the queuing is done this way).  If you
simplify add_child() to queue the newborn always at the front of the list,
your kill_some_child() will continue to do so, so I do not see the point
of the loop in add_child().  Am I mistaken?

[PATCH 1/3] daemon.c: minor style fixup

[Junio C Hamano]

 * "else" on the same line as "}" that closes corresponding "if (...) {";

 * multi-line comments begin with "/*\n";

 * sizeof, even it is not a function, is written as "sizeof(...)";

 * no need to check x?alloc() return value -- it would have died;

 * "if (...) { ... }" that covers the whole function body can be dedented
   by returning from the function early with "if (!...) return;";

 * SP on each side of an operator, i.e. "a > 0", not "a>0";

Also removes stale comment describing how remove_child() used to do its
thing.

Signed-off-by: Junio C Hamano <gitster@pobox.com>:
---

Junio C Hamano <gitster@pobox.com> writes:

> I however do have one issue with the logic, though.
>
> It seems that kill_some_child() will not kill anything if nobody else is
> coming from the same address, while the old code did kill some.  Is this
> intended?
>
> By the way, add_child() compares the whole "struct sockaddr_storage" in
> order to queue the newborn in front of an existing connection from the
> same address, and kill_some_child() takes advanrage of this to kill the
> newest connection ("We kill the newest" comment should probably be moved
> to add_child() to describe why the queuing is done this way).  If you
> simplify add_child() to queue the newborn always at the front of the list,
> your kill_some_child() will continue to do so, so I do not see the point
> of the loop in add_child().  Am I mistaken?

So here is a three-patch series on top.  I am not sure about the last one.

 daemon.c |   72 +++++++++++++++++++++++++------------------------------------
 1 files changed, 30 insertions(+), 42 deletions(-)

diff --git a/daemon.c b/daemon.c
index 79f0388..35bd34a 100644
--- a/daemon.c
+++ b/daemon.c
@@ -81,9 +81,9 @@ static void logreport(int priority, const char *err, va_list params)
 		char buf[1024];
 		vsnprintf(buf, sizeof(buf), err, params);
 		syslog(priority, "%s", buf);
-	}
-	else {
-		/* Since stderr is set to linebuffered mode, the
+	} else {
+		/*
+		 * Since stderr is set to linebuffered mode, the
 		 * logging of different processes will not overlap
 		 */
 		fprintf(stderr, "[%d] ", (int)getpid());
@@ -596,31 +596,20 @@ static struct child {
 
 static void add_child(pid_t pid, struct sockaddr *addr, int addrlen)
 {
-	struct child *newborn;
-	newborn = xcalloc(1, sizeof *newborn);
-	if (newborn) {
-		struct child **cradle;
-
-		live_children++;
-		newborn->pid = pid;
-		memcpy(&newborn->address, addr, addrlen);
-		for (cradle = &firstborn; *cradle; cradle = &(*cradle)->next)
-			if (!memcmp(&(*cradle)->address, &newborn->address,
-				   sizeof newborn->address))
-				break;
-		newborn->next = *cradle;
-		*cradle = newborn;
-	}
-	else
-		logerror("Out of memory spawning new child");
+	struct child *newborn, **cradle;
+	newborn = xcalloc(1, sizeof(*newborn));
+
+	live_children++;
+	newborn->pid = pid;
+	memcpy(&newborn->address, addr, addrlen);
+	for (cradle = &firstborn; *cradle; cradle = &(*cradle)->next)
+		if (!memcmp(&(*cradle)->address, &newborn->address,
+			    sizeof(newborn->address)))
+			break;
+	newborn->next = *cradle;
+	*cradle = newborn;
 }
 
-/*
- * Walk from "deleted" to "spawned", and remove child "pid".
- *
- * We move everything up by one, since the new "deleted" will
- * be one higher.
- */
 static void remove_child(pid_t pid)
 {
 	struct child **cradle, *blanket;
@@ -642,18 +631,17 @@ static void remove_child(pid_t pid)
  */
 static void kill_some_child(void)
 {
-	const struct child *blanket;
+	const struct child *blanket, *next;
 
-	if ((blanket = firstborn)) {
-		const struct child *next;
+	if (!(blanket = firstborn))
+		return;
 
-		for (; (next = blanket->next); blanket = next)
-			if (!memcmp(&blanket->address, &next->address,
-				   sizeof next->address)) {
-				kill(blanket->pid, SIGTERM);
-				break;
-			}
-	}
+	for (; (next = blanket->next); blanket = next)
+		if (!memcmp(&blanket->address, &next->address,
+			    sizeof(next->address))) {
+			kill(blanket->pid, SIGTERM);
+			break;
+		}
 }
 
 static void check_dead_children(void)
@@ -661,10 +649,10 @@ static void check_dead_children(void)
 	int status;
 	pid_t pid;
 
-	while ((pid = waitpid(-1, &status, WNOHANG))>0) {
+	while ((pid = waitpid(-1, &status, WNOHANG)) > 0) {
 		const char *dead = "";
 		remove_child(pid);
-		if (!WIFEXITED(status) || WEXITSTATUS(status) > 0)
+		if (!WIFEXITED(status) || (WEXITSTATUS(status) > 0))
 			dead = " (with error)";
 		loginfo("[%d] Disconnected%s", (int)pid, dead);
 	}
@@ -676,7 +664,7 @@ static void handle(int incoming, struct sockaddr *addr, int addrlen)
 
 	if (max_connections && live_children >= max_connections) {
 		kill_some_child();
-		sleep(1);			 /* give it some time to die */
+		sleep(1);  /* give it some time to die */
 		check_dead_children();
 		if (live_children >= max_connections) {
 			close(incoming);
@@ -705,7 +693,8 @@ static void handle(int incoming, struct sockaddr *addr, int addrlen)
 
 static void child_handler(int signo)
 {
-	/* Otherwise empty handler because systemcalls will get interrupted
+	/*
+	 * Otherwise empty handler because systemcalls will get interrupted
 	 * upon signal receipt
 	 * SysV needs the handler to be rearmed
 	 */
@@ -1089,8 +1078,7 @@ int main(int argc, char **argv)
 	if (log_syslog) {
 		openlog("git-daemon", LOG_PID, LOG_DAEMON);
 		set_die_routine(daemon_die);
-	}
-	else
+	} else
 		setlinebuf(stderr); /* avoid splitting a message in the middle */
 
 	if (inetd_mode && (group_name || user_name))
-- 
1.6.0.129.ge10d2

[PATCH 2/3] daemon.c: simplify add_child() and kill_some_child() logic

[Junio C Hamano]

kill_some_child() function walks the list of connections and kills the
first connection it finds from the same address.  There is not much point
to find the existing connection from the same address in add_child() to
insert the new one in front of it.  At the front of the whole queue is
just as good.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 daemon.c |   19 ++++++++++---------
 1 files changed, 10 insertions(+), 9 deletions(-)

diff --git a/daemon.c b/daemon.c
index 35bd34a..8d2755a 100644
--- a/daemon.c
+++ b/daemon.c
@@ -596,18 +596,18 @@ static struct child {
 
 static void add_child(pid_t pid, struct sockaddr *addr, int addrlen)
 {
-	struct child *newborn, **cradle;
-	newborn = xcalloc(1, sizeof(*newborn));
+	struct child *newborn;
 
+	/*
+	 * This must be xcalloc() -- we'll compare the whole sockaddr_storage
+	 * later in kill_some_child().
+	 */
+	newborn = xcalloc(1, sizeof(*newborn));
 	live_children++;
 	newborn->pid = pid;
 	memcpy(&newborn->address, addr, addrlen);
-	for (cradle = &firstborn; *cradle; cradle = &(*cradle)->next)
-		if (!memcmp(&(*cradle)->address, &newborn->address,
-			    sizeof(newborn->address)))
-			break;
-	newborn->next = *cradle;
-	*cradle = newborn;
+	newborn->next = firstborn;
+	firstborn = newborn;
 }
 
 static void remove_child(pid_t pid)
@@ -627,7 +627,8 @@ static void remove_child(pid_t pid)
  * This gets called if the number of connections grows
  * past "max_connections".
  *
- * We kill the newest connection from a duplicate IP.
+ * We kill the newest connection from the same address (add_child() queues
+ * new ones at the front).
  */
 static void kill_some_child(void)
 {
-- 
1.6.0.129.ge10d2

[PATCH 3/3] daemon.c: make sure kill_some_child() really kills somebody

[Junio C Hamano]

We used to kill nobody if there is no existing connection from the same
address the new connection we are trying to handle, and dropped the new
connection.  Make sure we at least kill one.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 * I am not sure about this one, but it may be more in spirit with the old
   behaviour that made sure at max connection limit we killed some to
   handle new ones.

   Actually I do think this is probably a bad idea.  What we really want
   to do is to detect an old one that is not making any progress instead.
   "old" we can detect by looking at its position in the queue (or we
   could even add an explicit timestamp to the child structure), but it is
   harder to measure "not making any progress", especially without going
   too platform specific, e.g. monitoring rusage or somesuch, which we
   would want to avoid.

 daemon.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/daemon.c b/daemon.c
index 8d2755a..a0d8f65 100644
--- a/daemon.c
+++ b/daemon.c
@@ -641,8 +641,11 @@ static void kill_some_child(void)
 		if (!memcmp(&blanket->address, &next->address,
 			    sizeof(next->address))) {
 			kill(blanket->pid, SIGTERM);
-			break;
+			return;
 		}
+
+	/* Nobody from the same address?  Kill the youngest one, then. */
+	kill(firstborn->pid, SIGTERM);
 }
 
 static void check_dead_children(void)
-- 
1.6.0.129.ge10d2

Re: What's cooking in git.git (Aug 2008, #07; Sat, 23)

[Stephen R. van den Berg]

Junio C Hamano wrote:
>Junio C Hamano <gitster@pobox.com> writes:
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>>>> * sb/daemon (Thu Aug 14 20:02:20 2008 +0200) 4 commits
>>>>  - git-daemon: rewrite kindergarden, new option --max-connections
>>>>  - git-daemon: Simplify dead-children reaping logic
>>>>  - git-daemon: use LOG_PID, simplify logging code
>>>>  - git-daemon: call logerror() instead of error()

>>>> Can somebody who actually runs the daemon standalone comment on this 
>>>> one?

>> Well, I didn't ask anybody to _run_ it.

Actually I *am* running it in production (obviously).

>It seems that kill_some_child() will not kill anything if nobody else is
>coming from the same address, while the old code did kill some.  Is this
>intended?

This is intended.
The reasoning goes a bit like this:
- We already allow a --timeout to be specified, that is taken care of
  by the child process.
- The child process is well-behaved, so it should honour its own timeout
  setting.
- If the daemon would need to cater for child processes failing to
  honour their own timeout setting, we have some serious bugs in the
  child-code, and *that* should be fixed ASAP instead of letting that
  kind of problems linger by not getting noticed because some daemon
  tries to clean up after badly-behaving children.
- Therefore, the only two problems that are left are:
  + Many connections in total.  That would mean that we can't handle the
    load.  Well, the best way to respond to that is to diligently
    process the ones that *did* manage to connect, and *not* to randomly
    disconnect sessions which are already halfway there.
  + Many connections in total, but also multiple connections from the
    same IP-address.  In this case, we could try and evenly distribute
    the resource and kill off the *newest* connections from the same IP
    addresses, to allow everyone an equal share to the daemon.

That means:
- Random killing is not necessary.
- Since we know exactly how many clients are running, we can keep tabs
  on when to fork and when not (further reducing the load problem on
  a busy site).  Which means that at most we need to
  kill one client for every new connection.
- Since we don't want to force old connections to die (we trust our
  children), we simply refuse further connections as soon as we've
  reached our limit and don't have resource-eaters (duplicate sessions
  from the same IP address).

>By the way, add_child() compares the whole "struct sockaddr_storage" in
>order to queue the newborn in front of an existing connection from the
>same address, and kill_some_child() takes advanrage of this to kill the
>newest connection ("We kill the newest" comment should probably be moved
>to add_child() to describe why the queuing is done this way).  If you
>simplify add_child() to queue the newborn always at the front of the list,
>your kill_some_child() will continue to do so, so I do not see the point
>of the loop in add_child().  Am I mistaken?

You are mistaken.
The point is that we need to find out *duplicate* IP-adresses.
I.e. when looking for the child to be killed, the IP-address we're looking
for is not the IP-address of the new connection, but we're looking for
two consecutive duplicate addresses.  In order for this to work, we need
to cluster the connections of the same IP-address, and the newest
connection needs to be inserted in front, in order for kill_child to
kill the most recent connection.

Shall I incorporate your suggested changes (as far as I consider them ok)
into my patches and resubmit them?
-- 
Sincerely,
           Stephen R. van den Berg.
Auto repair rates: basic labor $40/hour; if you wait, $60; if you watch, $80;
if you ask questions, $100; if you help, $120; if you laugh, $140.

Re: What's cooking in git.git (Aug 2008, #07; Sat, 23)

[Junio C Hamano]

"Stephen R. van den Berg" <srb@cuci.nl> writes:

> Junio C Hamano wrote:
> ...
>>It seems that kill_some_child() will not kill anything if nobody else is
>>coming from the same address, while the old code did kill some.  Is this
>>intended?
>
> This is intended.
> ...
>>...  If you
>>simplify add_child() to queue the newborn always at the front of the list,
>>your kill_some_child() will continue to do so, so I do not see the point
>>of the loop in add_child().  Am I mistaken?
> ...
> You are mistaken.
> The point is that we need to find out *duplicate* IP-adresses.

Lightbulb goes on on both counts.  Thanks.

> Shall I incorporate your suggested changes (as far as I consider them ok)
> into my patches and resubmit them?

I think your response makes the only remaining is the style fixup, which
was my [1/3] from yesterday, so I think we are Ok with what already we
have.

Thanks.

Re: What's cooking in git.git (Aug 2008, #07; Sat, 23)

[Stephen R. van den Berg]

Junio C Hamano wrote:
>"Stephen R. van den Berg" <srb@cuci.nl> writes:
>I think your response makes the only remaining is the style fixup, which
>was my [1/3] from yesterday, so I think we are Ok with what already we
>have.

Checked that one, looked fine.  Thanks.
-- 
Sincerely,
           Stephen R. van den Berg.
Auto repair rates: basic labor $40/hour; if you wait, $60; if you watch, $80;
if you ask questions, $100; if you help, $120; if you laugh, $140.