[PATCH 000/160] [PULL] ab/i18n rebased on ab/i18n-prereqs

[PATCH 000/160] [PULL] ab/i18n rebased on ab/i18n-prereqs

[Ævar Arnfjörð Bjarmason]

Here's an updated ab/i18n rebased on the new ab/i18n-prereqs
series. The major change here is this patch:

    gettext: fix bug in git-sh-i18n's eval_gettext() by using envsubst(1)

Which fixes eval_gettext to work using our own copy of envsubst(1). And:

    Makefile: Set NO_GETTEXT=YesPlease on Windows & MinGW

Which implements Johannes Six's suggestion in "[PATCH] Do not build
i18n on Windows." to not build with gettext on Windows by default.

This series can be fetched from:

    git://github.com/avar/git.git ab/i18n

Erik Faye-Lund (1):
  gettext.c: use libcharset.h instead of langinfo.h when available

Jan Engelhardt (1):
  po/de.po: complete German translation

Jan Krüger (1):
  po/de.po: add German translation

Marcin Cieślak (1):
  po/pl.po: add Polish translation

Peter Krefting (1):
  po/sv.po: add Swedish translation

Ramkumar Ramachandra (1):
  po/hi.po: add Hindi Translation

Sam Reed (1):
  po/en_GB.po: add British English translation

Thomas Rast (1):
  gettext tests: locate i18n lib&data correctly under --valgrind

Ævar Arnfjörð Bjarmason (152):
  gettext: add infrastructure for translating Git with gettext
  Makefile: Set NO_GETTEXT=YesPlease on Windows & MinGW
  gettext: fix bug in git-sh-i18n's eval_gettext() by using envsubst(1)
  gettext tests: rename test to work around GNU gettext bug
  gettext: setlocale(LC_CTYPE, "") breaks Git's C function assumptions
  Makefile: A variable for options used by xgettext(1) calls
  Makefile: provide a --msgid-bugs-address to xgettext(1)
  Makefile: tell xgettext(1) that our source is in UTF-8
  Makefile: use variables and shorter lines for xgettext
  builtin.h: Include gettext.h
  gettext.c: work around us not using setlocale(LC_CTYPE, "")
  gettext tests: add GETTEXT_POISON=YesPlease Makefile parameter
  gettext tests: skip lib-gettext.sh tests under GETTEXT_POISON
  gettextize: git-init basic messages
  gettextize: git-init "Initialized [...] repository" message
  gettext tests: test if $VERSION exists before using it
  gettext tests: add detection for is_IS.ISO-8859-1 locale
  gettext tests: test message re-encoding under Shell
  gettext tests: test re-encoding with a UTF-8 msgid under Shell
  gettext tests: mark a test message as not needing translation
  po/is.po: add Icelandic translation
  gettext tests: test message re-encoding under C
  gettextize: git-clone basic messages
  gettextize: git-clone "Cloning into" message
  gettextize: git-add basic messages
  gettextize: git-add "no files added" message
  gettextize: git-add "Use -f if you really want" message
  gettextize: git-add "pathspec [...] did not match" message
  gettextize: git-add "remove '%s'" message
  gettextize: git-add refresh_index message
  gettextize: git-branch basic messages
  gettextize: git-branch "remote branch '%s' not found" message
  gettextize: git-branch "Deleted branch [...]" message
  gettextize: git-branch "git branch -v" messages
  gettextize: git-branch "(no branch)" message
  gettextize: git-checkout basic messages
  gettextize: git-checkout: our/their version message
  gettextize: git-checkout describe_detached_head messages
  gettextize: git-checkout "HEAD is now at" message
  gettextize: git-checkout "Switched to a .. branch" message
  gettextize: git-commit basic messages
  gettextize: git-commit "middle of a merge" message
  gettextize: git-commit formatting messages
  gettextize: git-commit print_summary messages
  gettextize: git-commit "enter the commit message" message
  gettextize: git-commit advice messages
  gettextize: git-diff basic messages
  gettextize: git-fetch basic messages
  gettextize: git-fetch formatting messages
  gettextize: git-fetch update_local_ref messages
  gettextize: git-fetch split up "(non-fast-forward)" message
  gettextize: git-grep basic messages
  gettextize: git-grep "--open-files-in-pager" message
  gettextize: git-log basic messages
  gettextize: git-log "--OPT does not make sense" messages
  gettextize: git-merge basic messages
  gettextize: git-merge "Updating %s..%s" message
  gettextize: git-merge "You have not concluded your merge" messages
  gettextize: git-merge "Wonderful" message
  gettextize: git-mv basic messages
  gettextize: git-mv "bad" messages
  gettextize: git-rm basic messages
  gettextize: git-reset basic messages
  gettextize: git-reset reset_type_names messages
  gettextize: git-reset "Unstaged changes after reset" message
  gettextize: git-tag basic messages
  gettextize: git-tag tag_template message
  gettextize: git-push basic messages
  gettextize: git-push "prevent you from losing" message
  gettextize: git-status basic messages
  gettextize: git-status "nothing to commit" messages
  gettextize: git-status shortstatus messages
  gettextize: git-status "Changes to be committed" message
  gettextize: git-status "Initial commit" message
  gettextize: git-status "renamed: " message
  gettextize: git-archive basic messages
  gettextize: git-bundle basic messages
  gettextize: git-clean basic messages
  gettextize: git-clean clean.requireForce messages
  gettextize: git-describe basic messages
  gettextize: git-gc basic messages
  gettextize: git-gc "Auto packing the repository" message
  gettextize: git-notes basic commands
  gettextize: git-notes GIT_NOTES_REWRITE_MODE error message
  gettextize: git-notes "Refusing to %s notes in %s" message
  gettextize: git-revert basic messages
  gettextize: git-revert "Your local changes" message
  gettextize: git-revert literal "me" messages
  gettextize: git-revert split up "could not revert/apply" message
  gettextize: git-shortlog basic messages
  Makefile: add GNU_GETTEXT, set when we expect GNU gettext
  Makefile: MSGFMT="msgfmt --check" under GNU_GETTEXT
  gettext tests: add GETTEXT_POISON support for shell scripts
  gettext tests: add GETTEXT_POISON tests for shell scripts
  gettextize: git-am add git-sh-i18n
  gettextize: git-am one-line gettext $msg; echo
  gettextize: git-am multi-line getttext $msg; echo
  gettextize: git-am eval_gettext messages
  gettextize: git-am die messages
  gettextize: git-am cannot_fallback messages
  gettextize: git-am clean_abort messages
  gettextize: git-am "Apply?" message
  gettextize: git-am core say messages
  gettextize: git-am printf(1) message to eval_gettext
  gettext docs: add po/README file documenting Git's gettext
  Makefile: only add gettext tests on XGETTEXT_INCLUDE_TESTS=YesPlease
  gettext docs: add a "Testing your changes" section to po/README
  gettext docs: add "Marking strings for translation" section in
    po/README
  gettext docs: the gettext.h C interface
  gettext docs: the git-sh-i18n.sh Shell interface
  gettext docs: the Git::I18N Perl interface
  gettext docs: add "Testing marked strings" section to po/README
  gettextize: git-pull add git-sh-i18n
  gettextize: git-pull die messages
  gettextize: git-pull eval_gettext + die message
  gettextize: git-pull eval_gettext + warning message
  gettextize: git-pull split up "no candidate" message
  gettextize: git-pull "You asked to pull" message
  gettextize: git-pull "[...] not currently on a branch" message
  gettextize: git-pull "rebase against" / "merge with" messages
  gettextize: git-submodule add git-sh-i18n
  gettextize: git-submodule echo + eval_gettext messages
  gettextize: git-submodule say + eval_gettext messages
  gettextize: git-submodule die + eval_gettext messages
  gettextize: git-submodule $update_module say + die messages
  gettextize: git-submodule "cached cannot be used" message
  gettextize: git-submodule "Submodule change[...]" messages
  gettextize: git-submodule $errmsg messages
  gettextize: git-submodule "Entering [...]" message
  gettextize: git-submodule "[...] path is ignored" message
  gettextize: git-submodule "path not initialized" message
  gettextize: git-submodule "blob" and "submodule" messages
  gettextize: git-stash add git-sh-i18n
  gettextize: git-stash echo + gettext message
  gettextize: git-stash say + gettext messages
  gettextize: git-stash die + gettext messages
  gettextize: git-stash die + eval_gettext messages
  gettextize: git-stash die + eval_gettext $* messages
  gettextize: git-stash die + eval_gettext $1 messages
  gettextize: git-stash "unknown option" message
  gettextize: git-stash drop_stash say/die messages
  gettextize: git-bisect add git-sh-i18n
  gettextize: git-bisect gettext + echo message
  gettextize: git-bisect echo + gettext messages
  gettextize: git-bisect echo + eval_gettext message
  gettextize: git-bisect die + gettext messages
  gettextize: git-bisect die + eval_gettext messages
  gettextize: git-bisect bisect_run + $@ messages
  gettextize: git-bisect bisect_reset + $1 messages
  gettextize: git-bisect bisect_replay + $1 messages
  gettextize: git-bisect [Y/n] messages
  gettextize: git-bisect bisect_next_check "You need to" message

 .gitignore                          |    3 +
 Documentation/CodingGuidelines      |    7 +
 INSTALL                             |   12 +
 Makefile                            |  135 ++-
 builtin.h                           |    1 +
 builtin/add.c                       |   46 +-
 builtin/archive.c                   |   14 +-
 builtin/branch.c                    |   69 +-
 builtin/bundle.c                    |    6 +-
 builtin/checkout.c                  |   93 +-
 builtin/clean.c                     |   33 +-
 builtin/clone.c                     |   63 +-
 builtin/commit.c                    |  156 +-
 builtin/describe.c                  |   36 +-
 builtin/diff.c                      |   20 +-
 builtin/fetch.c                     |   82 +-
 builtin/gc.c                        |   24 +-
 builtin/grep.c                      |   34 +-
 builtin/init-db.c                   |   58 +-
 builtin/log.c                       |   68 +-
 builtin/merge.c                     |  124 +-
 builtin/mv.c                        |   32 +-
 builtin/notes.c                     |  134 +-
 builtin/push.c                      |   42 +-
 builtin/reset.c                     |   42 +-
 builtin/revert.c                    |   75 +-
 builtin/rm.c                        |   22 +-
 builtin/shortlog.c                  |    8 +-
 builtin/tag.c                       |   66 +-
 config.mak.in                       |    3 +
 configure.ac                        |   18 +
 daemon.c                            |    3 +
 fast-import.c                       |    3 +
 gettext.c                           |   35 +
 gettext.h                           |   22 +
 git-am.sh                           |   68 +-
 git-bisect.sh                       |   84 +-
 git-pull.sh                         |  147 +-
 git-sh-i18n.sh                      |   80 +
 git-stash.sh                        |   75 +-
 git-submodule.sh                    |   94 +-
 git.c                               |    3 +
 http-backend.c                      |    3 +
 http-fetch.c                        |    3 +
 http-push.c                         |    3 +
 imap-send.c                         |    3 +
 perl/Git/I18N.pm                    |   91 +
 perl/Makefile                       |    3 +-
 perl/Makefile.PL                    |   14 +-
 po/.gitignore                       |    1 +
 po/README                           |  209 ++
 po/de.po                            | 3609 +++++++++++++++++++++++++++++++++++
 po/en_GB.po                         | 2784 +++++++++++++++++++++++++++
 po/hi.po                            | 2787 +++++++++++++++++++++++++++
 po/is.po                            |  194 ++
 po/pl.po                            | 2793 +++++++++++++++++++++++++++
 po/sv.po                            | 3492 +++++++++++++++++++++++++++++++++
 sh-i18n--envsubst.c                 |  444 +++++
 shell.c                             |    3 +
 show-index.c                        |    3 +
 t/lib-gettext.sh                    |   92 +
 t/lib-httpd.sh                      |    2 +-
 t/t0001-init.sh                     |    2 +-
 t/t0200-gettext-basic.sh            |  111 ++
 t/t0200/test.c                      |   23 +
 t/t0200/test.perl                   |   14 +
 t/t0200/test.sh                     |   14 +
 t/t0201-gettext-fallbacks.sh        |   52 +
 t/t0202-gettext-perl.sh             |   27 +
 t/t0202/test.pl                     |  109 ++
 t/t0203-gettext-setlocale-sanity.sh |   26 +
 t/t0204-gettext-reencode-sanity.sh  |   78 +
 t/t0205-gettext-poison.sh           |   36 +
 t/t1200-tutorial.sh                 |    5 +-
 t/t2200-add-update.sh               |    2 +-
 t/t2204-add-ignored.sh              |   37 +-
 t/t3030-merge-recursive.sh          |    2 +-
 t/t3200-branch.sh                   |    2 +-
 t/t3203-branch-output.sh            |    2 +-
 t/t3501-revert-cherry-pick.sh       |    2 +-
 t/t3507-cherry-pick-conflict.sh     |    2 +-
 t/t3700-add.sh                      |   11 +-
 t/t4001-diff-rename.sh              |    4 +-
 t/t4014-format-patch.sh             |    2 +-
 t/t4150-am.sh                       |    5 +-
 t/t4151-am-abort.sh                 |    2 +-
 t/t5541-http-push.sh                |    7 +-
 t/t5601-clone.sh                    |    2 +-
 t/t6040-tracking-info.sh            |    2 +-
 t/t6120-describe.sh                 |    2 +-
 t/t7004-tag.sh                      |    1 +
 t/t7012-skip-worktree-writing.sh    |    4 +-
 t/t7060-wtstatus.sh                 |    2 +-
 t/t7102-reset.sh                    |    2 +-
 t/t7110-reset-merge.sh              |    6 +-
 t/t7201-co.sh                       |   10 +-
 t/t7300-clean.sh                    |    6 +-
 t/t7400-submodule-basic.sh          |    4 +-
 t/t7401-submodule-summary.sh        |   12 +-
 t/t7407-submodule-foreach.sh        |    4 +-
 t/t7500-commit.sh                   |    6 +-
 t/t7501-commit.sh                   |    7 +-
 t/t7502-commit.sh                   |   60 +-
 t/t7506-status-submodule.sh         |   28 +-
 t/t7508-status.sh                   |  115 +-
 t/t7600-merge.sh                    |    2 +-
 t/t7811-grep-open.sh                |    2 +-
 t/test-lib.sh                       |   10 +
 upload-pack.c                       |    3 +
 wt-status.c                         |  116 +-
 110 files changed, 18581 insertions(+), 1075 deletions(-)
 create mode 100644 gettext.c
 create mode 100644 gettext.h
 create mode 100644 git-sh-i18n.sh
 create mode 100644 perl/Git/I18N.pm
 create mode 100644 po/.gitignore
 create mode 100644 po/README
 create mode 100644 po/de.po
 create mode 100644 po/en_GB.po
 create mode 100644 po/hi.po
 create mode 100644 po/is.po
 create mode 100644 po/pl.po
 create mode 100644 po/sv.po
 create mode 100644 sh-i18n--envsubst.c
 create mode 100644 t/lib-gettext.sh
 create mode 100755 t/t0200-gettext-basic.sh
 create mode 100644 t/t0200/test.c
 create mode 100644 t/t0200/test.perl
 create mode 100644 t/t0200/test.sh
 create mode 100755 t/t0201-gettext-fallbacks.sh
 create mode 100755 t/t0202-gettext-perl.sh
 create mode 100644 t/t0202/test.pl
 create mode 100755 t/t0203-gettext-setlocale-sanity.sh
 create mode 100755 t/t0204-gettext-reencode-sanity.sh
 create mode 100755 t/t0205-gettext-poison.sh

-- 
1.7.2.3

[PATCH 002/160] Makefile: Set NO_GETTEXT=YesPlease on Windows & MinGW

[Ævar Arnfjörð Bjarmason]

Change the Windows and MinGW build defaults to not build the gettext
tools by default. Gettext hasn't yet been ported to the default
environments on those systems, although Erik Faye-Lund is working on
MinGW support.

Reported-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/Makefile b/Makefile
index 426e0b1..67d94d3 100644
--- a/Makefile
+++ b/Makefile
@@ -826,6 +826,8 @@ ifeq ($(uname_S),Windows)
 	NO_REGEX = YesPlease
 	NO_CURL = YesPlease
 	NO_PYTHON = YesPlease
+	NO_GETTEXT = YesPlease
+	NEEDS_LIBINTL =
 	BLK_SHA1 = YesPlease
 	NATIVE_CRLF = YesPlease
 
@@ -878,6 +880,8 @@ ifneq (,$(findstring MINGW,$(uname_S)))
 	OBJECT_CREATION_USES_RENAMES = UnfortunatelyNeedsTo
 	NO_REGEX = YesPlease
 	NO_PYTHON = YesPlease
+	NO_GETTEXT = YesPlease
+	NEEDS_LIBINTL =
 	BLK_SHA1 = YesPlease
 	ETAGS_TARGET = ETAGS
 	COMPAT_CFLAGS += -D__USE_MINGW_ACCESS -DNOGDI -Icompat -Icompat/fnmatch -Icompat/win32
-- 
1.7.2.3

[PATCH 003/160] gettext: fix bug in git-sh-i18n's eval_gettext() by using envsubst(1)

[Ævar Arnfjörð Bjarmason]

Change eval_gettext(1) in git-sh-i18n.sh to use a modified version of
gettext's envsubst(1) program, instead of using a clever (but broken)
printf + eval + printf trick.

Our previous fallback would incorrectly handle cases where the
variable being interpolated contained spaces. E.g.:

    cmd="git foo"; eval_gettext "command: \$cmd"

Would emit "command: gitfoo", instead of the correct "command: git
foo". This happened with a message in git-am.sh that used the $cmdline
variable.

To work around this, and to improve our variable expansion behavior
(eval has security issues) I've imported a stripped-down version of
gettext's envsubst(1) program.

Using it we pass the latter of the two tests added along with this
patch (the first one was just added for completeness).

Since we want to test both our fallback eval_gettext() and the one
we'll end up using (i.e. on Solaris) the new tests are executed in
both t0200-gettext-basic.sh and t0201-gettext-fallbacks.sh.

These are the modifications I made to envsubst.c as I turned it into
sh-i18n--envsubst.c:

 * Added our git-compat-util.h header for xrealloc() and friends.

 * Removed inclusion of gettext-specific headers.

 * Removed most of main() and replaced it with my own. The modified
   version only does option parsing for --variables. That's all it
   needs.

 * Modified error() invocations to use our error() instead of
   error(3).

 * Replaced the gettext XNMALLOC(n, size) macro with just
   xmalloc(n). Since XNMALLOC() only allocated char's.

 * Removed the string_list_destroy function. It's redundant (also in
   the upstream code).

 * Replaced the use of stdbool.h (a C99 header) by doing the following
   replacements on the code:

    * s/bool/unsigned short int/g
    * s/true/1/g
    * s/false/0/g

Reported-by: Johannes Sixt <j.sixt@viscovery.net>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .gitignore                   |    1 +
 Makefile                     |    1 +
 git-sh-i18n.sh               |   14 +-
 sh-i18n--envsubst.c          |  444 ++++++++++++++++++++++++++++++++++++++++++
 t/lib-gettext.sh             |   24 +++
 t/t0200-gettext.sh           |    3 +
 t/t0201-gettext-fallbacks.sh |    3 +
 7 files changed, 485 insertions(+), 5 deletions(-)
 create mode 100644 sh-i18n--envsubst.c

diff --git a/.gitignore b/.gitignore
index 80ca718..d2d9ec2 100644
--- a/.gitignore
+++ b/.gitignore
@@ -126,6 +126,7 @@
 /git-send-email
 /git-send-pack
 /git-sh-i18n
+/git-sh-i18n--envsubst
 /git-sh-setup
 /git-shell
 /git-shortlog
diff --git a/Makefile b/Makefile
index 67d94d3..1ef4648 100644
--- a/Makefile
+++ b/Makefile
@@ -420,6 +420,7 @@ PROGRAM_OBJS += shell.o
 PROGRAM_OBJS += show-index.o
 PROGRAM_OBJS += upload-pack.o
 PROGRAM_OBJS += http-backend.o
+PROGRAM_OBJS += sh-i18n--envsubst.o
 
 PROGRAMS += $(patsubst %.o,git-%$X,$(PROGRAM_OBJS))
 
diff --git a/git-sh-i18n.sh b/git-sh-i18n.sh
index 698a000..3aac21c 100644
--- a/git-sh-i18n.sh
+++ b/git-sh-i18n.sh
@@ -48,10 +48,12 @@ then
 
 	# Solaris has a gettext(1) but no eval_gettext(1)
 	eval_gettext () {
-		gettext_out=$(gettext "$1")
-		gettext_eval="printf '%s' \"$gettext_out\""
-		printf "%s" "`eval \"$gettext_eval\"`"
+		gettext "$1" | (
+			export PATH $(git sh-i18n--envsubst --variables "$1");
+			git sh-i18n--envsubst "$1"
+		)
 	}
+
 else
 	# Since gettext.sh isn't available we'll have to define our own
 	# dummy pass-through functions.
@@ -65,7 +67,9 @@ else
 	}
 
 	eval_gettext () {
-		gettext_eval="printf '%s' \"$1\""
-		printf "%s" "`eval \"$gettext_eval\"`"
+		printf "%s" "$1" | (
+			export PATH $(git sh-i18n--envsubst --variables "$1");
+			git sh-i18n--envsubst "$1"
+		)
 	}
 fi
diff --git a/sh-i18n--envsubst.c b/sh-i18n--envsubst.c
new file mode 100644
index 0000000..8db71b1
--- /dev/null
+++ b/sh-i18n--envsubst.c
@@ -0,0 +1,444 @@
+/*
+ * sh-i18n--envsubst.c - a stripped-down version of gettext's envsubst(1)
+ *
+ * Copyright (C) 2010 Ævar Arnfjörð Bjarmason
+ *
+ * This is a modified version of
+ * 67d0871a8c:gettext-runtime/src/envsubst.c from the gettext.git
+ * repository. It has been stripped down to only implement the
+ * envsubst(1) features that we need in the git-sh-i18n fallbacks.
+ *
+ * The "Close standard error" part in main() is from
+ * 8dac033df0:gnulib-local/lib/closeout.c. The copyright notices for
+ * both files are reproduced immediately below.
+ */
+
+#include "git-compat-util.h"
+
+/* Substitution of environment variables in shell format strings.
+   Copyright (C) 2003-2007 Free Software Foundation, Inc.
+   Written by Bruno Haible <bruno@clisp.org>, 2003.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 2, or (at your option)
+   any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program; if not, write to the Free Software Foundation,
+   Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.  */
+
+/* closeout.c - close standard output and standard error
+   Copyright (C) 1998-2007 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 2, or (at your option)
+   any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program; if not, write to the Free Software Foundation,
+   Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.  */
+
+#include <errno.h>
+#include <getopt.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+/* If true, substitution shall be performed on all variables.  */
+static unsigned short int all_variables;
+
+/* Forward declaration of local functions.  */
+static void print_variables (const char *string);
+static void note_variables (const char *string);
+static void subst_from_stdin (void);
+
+int
+main (int argc, char *argv[])
+{
+  /* Default values for command line options.  */
+  unsigned short int show_variables = 0;
+
+  switch (argc)
+	{
+	case 1:
+	  error ("we won't substitute all variables on stdin for you");
+	  /*
+	  all_variables = 1;
+      subst_from_stdin ();
+	  */
+	case 2:
+	  /* echo '$foo and $bar' | git sh-i18n--envsubst --variables '$foo and $bar' */
+	  all_variables = 0;
+	  note_variables (argv[1]);
+      subst_from_stdin ();
+	  break;
+	case 3:
+	  /* git sh-i18n--envsubst --variables '$foo and $bar' */
+	  if (strcmp(argv[1], "--variables"))
+		error ("first argument must be --variables when two are given");
+	  show_variables = 1;
+      print_variables (argv[2]);
+	  break;
+	default:
+	  error ("too many arguments");
+	  break;
+	}
+
+  /* Close standard error.  This is simpler than fwriteerror_no_ebadf, because
+     upon failure we don't need an errno - all we can do at this point is to
+     set an exit status.  */
+  errno = 0;
+  if (ferror (stderr) || fflush (stderr))
+    { 
+      fclose (stderr);
+      exit (EXIT_FAILURE);
+    }
+  if (fclose (stderr) && errno != EBADF)
+    exit (EXIT_FAILURE);
+
+  exit (EXIT_SUCCESS);
+}
+
+/* Parse the string and invoke the callback each time a $VARIABLE or
+   ${VARIABLE} construct is seen, where VARIABLE is a nonempty sequence
+   of ASCII alphanumeric/underscore characters, starting with an ASCII
+   alphabetic/underscore character.
+   We allow only ASCII characters, to avoid dependencies w.r.t. the current
+   encoding: While "${\xe0}" looks like a variable access in ISO-8859-1
+   encoding, it doesn't look like one in the BIG5, BIG5-HKSCS, GBK, GB18030,
+   SHIFT_JIS, JOHAB encodings, because \xe0\x7d is a single character in these
+   encodings.  */
+static void
+find_variables (const char *string,
+		void (*callback) (const char *var_ptr, size_t var_len))
+{
+  for (; *string != '\0';)
+    if (*string++ == '$')
+      {
+	const char *variable_start;
+	const char *variable_end;
+	unsigned short int valid;
+	char c;
+
+	if (*string == '{')
+	  string++;
+
+	variable_start = string;
+	c = *string;
+	if ((c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || c == '_')
+	  {
+	    do
+	      c = *++string;
+	    while ((c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z')
+		   || (c >= '0' && c <= '9') || c == '_');
+	    variable_end = string;
+
+	    if (variable_start[-1] == '{')
+	      {
+		if (*string == '}')
+		  {
+		    string++;
+		    valid = 1;
+		  }
+		else
+		  valid = 0;
+	      }
+	    else
+	      valid = 1;
+
+	    if (valid)
+	      callback (variable_start, variable_end - variable_start);
+	  }
+      }
+}
+
+
+/* Print a variable to stdout, followed by a newline.  */
+static void
+print_variable (const char *var_ptr, size_t var_len)
+{
+  fwrite (var_ptr, var_len, 1, stdout);
+  putchar ('\n');
+}
+
+/* Print the variables contained in STRING to stdout, each one followed by a
+   newline.  */
+static void
+print_variables (const char *string)
+{
+  find_variables (string, &print_variable);
+}
+
+
+/* Type describing list of immutable strings,
+   implemented using a dynamic array.  */
+typedef struct string_list_ty string_list_ty;
+struct string_list_ty
+{
+  const char **item;
+  size_t nitems;
+  size_t nitems_max;
+};
+
+/* Initialize an empty list of strings.  */
+static inline void
+string_list_init (string_list_ty *slp)
+{
+  slp->item = NULL;
+  slp->nitems = 0;
+  slp->nitems_max = 0;
+}
+
+/* Append a single string to the end of a list of strings.  */
+static inline void
+string_list_append (string_list_ty *slp, const char *s)
+{
+  /* Grow the list.  */
+  if (slp->nitems >= slp->nitems_max)
+    {
+      size_t nbytes;
+
+      slp->nitems_max = slp->nitems_max * 2 + 4;
+      nbytes = slp->nitems_max * sizeof (slp->item[0]);
+      slp->item = (const char **) xrealloc (slp->item, nbytes);
+    }
+
+  /* Add the string to the end of the list.  */
+  slp->item[slp->nitems++] = s;
+}
+
+/* Compare two strings given by reference.  */
+static int
+cmp_string (const void *pstr1, const void *pstr2)
+{
+  const char *str1 = *(const char **)pstr1;
+  const char *str2 = *(const char **)pstr2;
+
+  return strcmp (str1, str2);
+}
+
+/* Sort a list of strings.  */
+static inline void
+string_list_sort (string_list_ty *slp)
+{
+  if (slp->nitems > 0)
+    qsort (slp->item, slp->nitems, sizeof (slp->item[0]), cmp_string);
+}
+
+/* Test whether a string list contains a given string.  */
+static inline int
+string_list_member (const string_list_ty *slp, const char *s)
+{
+  size_t j;
+
+  for (j = 0; j < slp->nitems; ++j)
+    if (strcmp (slp->item[j], s) == 0)
+      return 1;
+  return 0;
+}
+
+/* Test whether a sorted string list contains a given string.  */
+static int
+sorted_string_list_member (const string_list_ty *slp, const char *s)
+{
+  size_t j1, j2;
+
+  j1 = 0;
+  j2 = slp->nitems;
+  if (j2 > 0)
+    {
+      /* Binary search.  */
+      while (j2 - j1 > 1)
+	{
+	  /* Here we know that if s is in the list, it is at an index j
+	     with j1 <= j < j2.  */
+	  size_t j = (j1 + j2) >> 1;
+	  int result = strcmp (slp->item[j], s);
+
+	  if (result > 0)
+	    j2 = j;
+	  else if (result == 0)
+	    return 1;
+	  else
+	    j1 = j + 1;
+	}
+      if (j2 > j1)
+	if (strcmp (slp->item[j1], s) == 0)
+	  return 1;
+    }
+  return 0;
+}
+
+
+/* Set of variables on which to perform substitution.
+   Used only if !all_variables.  */
+static string_list_ty variables_set;
+
+/* Adds a variable to variables_set.  */
+static void
+note_variable (const char *var_ptr, size_t var_len)
+{
+  char *string = xmalloc (var_len + 1);
+  memcpy (string, var_ptr, var_len);
+  string[var_len] = '\0';
+
+  string_list_append (&variables_set, string);
+}
+
+/* Stores the variables occurring in the string in variables_set.  */
+static void
+note_variables (const char *string)
+{
+  string_list_init (&variables_set);
+  find_variables (string, &note_variable);
+  string_list_sort (&variables_set);
+}
+
+
+static int
+do_getc ()
+{
+  int c = getc (stdin);
+
+  if (c == EOF)
+    {
+      if (ferror (stdin))
+	error ("error while reading standard input");
+    }
+
+  return c;
+}
+
+static inline void
+do_ungetc (int c)
+{
+  if (c != EOF)
+    ungetc (c, stdin);
+}
+
+/* Copies stdin to stdout, performing substitutions.  */
+static void
+subst_from_stdin ()
+{
+  static char *buffer;
+  static size_t bufmax;
+  static size_t buflen;
+  int c;
+
+  for (;;)
+    {
+      c = do_getc ();
+      if (c == EOF)
+	break;
+      /* Look for $VARIABLE or ${VARIABLE}.  */
+      if (c == '$')
+	{
+	  unsigned short int opening_brace = 0;
+	  unsigned short int closing_brace = 0;
+
+	  c = do_getc ();
+	  if (c == '{')
+	    {
+	      opening_brace = 1;
+	      c = do_getc ();
+	    }
+	  if ((c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || c == '_')
+	    {
+	      unsigned short int valid;
+
+	      /* Accumulate the VARIABLE in buffer.  */
+	      buflen = 0;
+	      do
+		{
+		  if (buflen >= bufmax)
+		    {
+		      bufmax = 2 * bufmax + 10;
+		      buffer = xrealloc (buffer, bufmax);
+		    }
+		  buffer[buflen++] = c;
+
+		  c = do_getc ();
+		}
+	      while ((c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z')
+		     || (c >= '0' && c <= '9') || c == '_');
+
+	      if (opening_brace)
+		{
+		  if (c == '}')
+		    {
+		      closing_brace = 1;
+		      valid = 1;
+		    }
+		  else
+		    {
+		      valid = 0;
+		      do_ungetc (c);
+		    }
+		}
+	      else
+		{
+		  valid = 1;
+		  do_ungetc (c);
+		}
+
+	      if (valid)
+		{
+		  /* Terminate the variable in the buffer.  */
+		  if (buflen >= bufmax)
+		    {
+		      bufmax = 2 * bufmax + 10;
+		      buffer = xrealloc (buffer, bufmax);
+		    }
+		  buffer[buflen] = '\0';
+
+		  /* Test whether the variable shall be substituted.  */
+		  if (!all_variables
+		      && !sorted_string_list_member (&variables_set, buffer))
+		    valid = 0;
+		}
+
+	      if (valid)
+		{
+		  /* Substitute the variable's value from the environment.  */
+		  const char *env_value = getenv (buffer);
+
+		  if (env_value != NULL)
+		    fputs (env_value, stdout);
+		}
+	      else
+		{
+		  /* Perform no substitution at all.  Since the buffered input
+		     contains no other '$' than at the start, we can just
+		     output all the buffered contents.  */
+		  putchar ('$');
+		  if (opening_brace)
+		    putchar ('{');
+		  fwrite (buffer, buflen, 1, stdout);
+		  if (closing_brace)
+		    putchar ('}');
+		}
+	    }
+	  else
+	    {
+	      do_ungetc (c);
+	      putchar ('$');
+	      if (opening_brace)
+		putchar ('{');
+	    }
+	}
+      else
+	putchar (c);
+    }
+}
diff --git a/t/lib-gettext.sh b/t/lib-gettext.sh
index 831ee38..f0101fc 100644
--- a/t/lib-gettext.sh
+++ b/t/lib-gettext.sh
@@ -40,3 +40,27 @@ else
 	test_set_prereq NO_GETTEXT
 	say "# lib-gettext: No GETTEXT support available"
 fi
+
+test_eval_gettext_interpolation() {
+	test_expect_success NO_GETTEXT_POISON 'eval_gettext: our eval_gettext() fallback can interpolate whitespace variables' '
+	    git_am_cmdline="git am" &&
+	    printf "test git am" >expect &&
+	    eval_gettext "test \$git_am_cmdline" >actual &&
+	    test_cmp expect actual
+	'
+
+	test_expect_success NO_GETTEXT_POISON 'eval_gettext: git am $cmdline bug' '
+	    cmdline="git am -3" &&
+	    export cmdline &&
+	    cat >expect <<EOF &&
+When you have resolved this problem run "git am -3 --resolved".
+If you would prefer to skip this patch, instead run "git am -3 --skip".
+To restore the original branch and stop patching run "git am -3 --abort".
+EOF
+	    eval_gettext "When you have resolved this problem run \"\$cmdline --resolved\".
+If you would prefer to skip this patch, instead run \"\$cmdline --skip\".
+To restore the original branch and stop patching run \"\$cmdline --abort\"." >actual &&
+	    echo >>actual &&
+	    test_cmp expect actual
+	'
+}
diff --git a/t/t0200-gettext.sh b/t/t0200-gettext.sh
index 8853d8a..58948fa 100755
--- a/t/t0200-gettext.sh
+++ b/t/t0200-gettext.sh
@@ -105,4 +105,7 @@ test_expect_success GETTEXT_LOCALE 'sanity: Some gettext("") data for real local
     test -s real-locale
 '
 
+# Test eval_gettext() interpolation with the actual eval_gettext function
+test_eval_gettext_interpolation
+
 test_done
diff --git a/t/t0201-gettext-fallbacks.sh b/t/t0201-gettext-fallbacks.sh
index 47ce4f6..d2dc4af 100755
--- a/t/t0201-gettext-fallbacks.sh
+++ b/t/t0201-gettext-fallbacks.sh
@@ -46,4 +46,7 @@ test_expect_success 'eval_gettext: our eval_gettext() fallback can interpolate v
     test_cmp expect actual
 '
 
+# Test eval_gettext() interpolation with the fallback eval_gettext function
+test_eval_gettext_interpolation
+
 test_done
-- 
1.7.2.3

Re: [PATCH 003/160] gettext: fix bug in git-sh-i18n's eval_gettext() by using envsubst(1)

[Jonathan Nieder]

Ævar Arnfjörð Bjarmason wrote:

> These are the modifications I made to envsubst.c as I turned it into
> sh-i18n--envsubst.c:
> 
>  * Added our git-compat-util.h header for xrealloc() and friends.
> 
>  * Removed inclusion of gettext-specific headers.
> 
>  * Removed most of main() and replaced it with my own. The modified
>    version only does option parsing for --variables. That's all it
>    needs.
> 
>  * Modified error() invocations to use our error() instead of
>    error(3).
> 
>  * Replaced the gettext XNMALLOC(n, size) macro with just
>    xmalloc(n). Since XNMALLOC() only allocated char's.
> 
>  * Removed the string_list_destroy function. It's redundant (also in
>    the upstream code).
> 
>  * Replaced the use of stdbool.h (a C99 header) by doing the following
>    replacements on the code:

Nit: can this be moved to a subdir?  Maybe po or i18n, or even
envsubst.  This could make it clearer that the code has an upstream
and should be updated when upstream is.

Jonathan

Re: [PATCH 003/160] gettext: fix bug in git-sh-i18n's eval_gettext() by using envsubst(1)

[Junio C Hamano]

Jonathan Nieder <jrnieder@gmail.com> writes:

> Nit: can this be moved to a subdir?  Maybe po or i18n, or even
> envsubst.  This could make it clearer that the code has an upstream
> and should be updated when upstream is.

IOW, just like "xdiff/" is a separate directory.  I think that is a sane
thing to do.

Thanks.

Re: [PATCH 003/160] gettext: fix bug in git-sh-i18n's eval_gettext() by using envsubst(1)

[Ævar Arnfjörð Bjarmason]

On Sun, Nov 14, 2010 at 20:15, Junio C Hamano <gitster@pobox.com> wrote:
> Jonathan Nieder <jrnieder@gmail.com> writes:
>> Nit: can this be moved to a subdir?  Maybe po or i18n, or even
>> envsubst.  This could make it clearer that the code has an upstream
>> and should be updated when upstream is.
>
> IOW, just like "xdiff/" is a separate directory.  I think that is a sane
> thing to do.

Thanks for the suggestion. I'll put it in envsubst/ then.

Any idea about the issue of it being linked to too many things? I
tried to add another entry to this section:

    git-%$X: %.o $(GITLIBS)
        $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter
%.o,$^) $(LIBS)

    git-imap-send$X: imap-send.o $(GITLIBS)
        $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
            $(LIBS) $(OPENSSL_LINK) $(OPENSSL_LIBSSL)

But it wasn't actually used. But then again I don't know Makefile
hacking that much.

Re: [PATCH 003/160] gettext: fix bug in git-sh-i18n's eval_gettext() by using envsubst(1)

[Jonathan Nieder]

Ævar Arnfjörð Bjarmason wrote:

> Thanks for the suggestion. I'll put it in envsubst/ then.
> 
> Any idea about the issue of it being linked to too many things?

Maybe http://thread.gmane.org/gmane.comp.version-control.git/160842
is related?

> tried to add another entry to this section:
> 
>     git-%$X: %.o $(GITLIBS)
>         $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter
> %.o,$^) $(LIBS)
> 
>     git-imap-send$X: imap-send.o $(GITLIBS)
>         $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
>             $(LIBS) $(OPENSSL_LINK) $(OPENSSL_LIBSSL)
> 
> But it wasn't actually used. But then again I don't know Makefile
> hacking that much.

If you send output from "make envsubst" from your attempt with V=1,
I can try to debug it.

Re: [PATCH 003/160] gettext: fix bug in git-sh-i18n's eval_gettext() by using envsubst(1)

[Ævar Arnfjörð Bjarmason]

On Sun, Nov 14, 2010 at 22:27, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Ævar Arnfjörð Bjarmason wrote:
>
>> Thanks for the suggestion. I'll put it in envsubst/ then.
>>
>> Any idea about the issue of it being linked to too many things?
>
> Maybe http://thread.gmane.org/gmane.comp.version-control.git/160842
> is related?
>
>> tried to add another entry to this section:
>>
>>     git-%$X: %.o $(GITLIBS)
>>         $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter
>> %.o,$^) $(LIBS)
>>
>>     git-imap-send$X: imap-send.o $(GITLIBS)
>>         $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
>>             $(LIBS) $(OPENSSL_LINK) $(OPENSSL_LIBSSL)
>>
>> But it wasn't actually used. But then again I don't know Makefile
>> hacking that much.
>
> If you send output from "make envsubst" from your attempt with V=1,
> I can try to debug it.

This is how it currently gets compiled:

    $ rm git-sh-i18n--envsubst sh-i18n--envsubst.o; make -j 5 all V=1
    gcc -o sh-i18n--envsubst.o -c   -g -O2 -Wall -I.  -DHAVE_PATHS_H
-DSHA1_HEADER='<openssl/sha.h>'  -DNO_STRLCPY -DNO_MKSTEMPS
sh-i18n--envsubst.c
    gcc  -g -O2 -Wall -I.  -DHAVE_PATHS_H
-DSHA1_HEADER='<openssl/sha.h>'  -DNO_STRLCPY -DNO_MKSTEMPS -o
git-sh-i18n--envsubst   sh-i18n--envsubst.o libgit.a xdiff/lib.a  -lz
-lcrypto -lpthread

I tried to remove some of those library includes. But it'll
complain. However if I do:

    diff --git a/sh-i18n--envsubst.c b/sh-i18n--envsubst.c
    index 8db71b1..7d4c870 100644
    --- a/sh-i18n--envsubst.c
    +++ b/sh-i18n--envsubst.c
    @@ -215 +215 @@ string_list_append (string_list_ty *slp, const char *s)
    -      slp->item = (const char **) xrealloc (slp->item, nbytes);
    +      slp->item = (const char **) realloc (slp->item, nbytes);
    @@ -293 +293 @@ note_variable (const char *var_ptr, size_t var_len)
    -  char *string = xmalloc (var_len + 1);
    +  char *string = malloc (var_len + 1);
    @@ -368 +368 @@ subst_from_stdin ()
    -                     buffer = xrealloc (buffer, bufmax);
    +                     buffer = realloc (buffer, bufmax);
    @@ -402 +402 @@ subst_from_stdin ()
    -                     buffer = xrealloc (buffer, bufmax);
    +                     buffer = realloc (buffer, bufmax);

It'll compile stand-alone with no library includes at all. Maybe I
should just modify it to have basic versions of those two x* functions
so we can compile it completely stand-alone.

Re: [PATCH 003/160] gettext: fix bug in git-sh-i18n's eval_gettext() by using envsubst(1)

[Jonathan Nieder]

Ævar Arnfjörð Bjarmason wrote:

> This is how it currently gets compiled:
> 
>     $ rm git-sh-i18n--envsubst sh-i18n--envsubst.o; make -j 5 all V=1
>     gcc -o sh-i18n--envsubst.o -c   -g -O2 -Wall -I.  -DHAVE_PATHS_H
> -DSHA1_HEADER='<openssl/sha.h>'  -DNO_STRLCPY -DNO_MKSTEMPS
> sh-i18n--envsubst.c
>     gcc  -g -O2 -Wall -I.  -DHAVE_PATHS_H
> -DSHA1_HEADER='<openssl/sha.h>'  -DNO_STRLCPY -DNO_MKSTEMPS -o
> git-sh-i18n--envsubst   sh-i18n--envsubst.o libgit.a xdiff/lib.a  -lz
> -lcrypto -lpthread
> 
> I tried to remove some of those library includes. But it'll
> complain. However if I do:
> 
>     diff --git a/sh-i18n--envsubst.c b/sh-i18n--envsubst.c
>     index 8db71b1..7d4c870 100644
>     --- a/sh-i18n--envsubst.c
>     +++ b/sh-i18n--envsubst.c
>     @@ -215 +215 @@ string_list_append (string_list_ty *slp, const char *s)
>     -      slp->item = (const char **) xrealloc (slp->item, nbytes);
>     +      slp->item = (const char **) realloc (slp->item, nbytes);
[...]
> It'll compile stand-alone with no library includes at all.

Yes, that is what $gmane/160842 is about (also see $gmane/137738 for
background).  In this case, a simple local stub xmalloc/xrealloc etc
to take care of checking for NULL seems fine, indeed.

Re: [PATCH 003/160] gettext: fix bug in git-sh-i18n's eval_gettext() by using envsubst(1)

[Ævar Arnfjörð Bjarmason]

On Sun, Nov 14, 2010 at 22:47, Jonathan Nieder <jrnieder@gmail.com> wrote:

> Yes, that is what $gmane/160842 is about (also see $gmane/137738 for
> background).  In this case, a simple local stub xmalloc/xrealloc etc
> to take care of checking for NULL seems fine, indeed.

Indeed. The problem is that even if I did that I couldn't convince the
Makefile in cooperating with creating my binary in a special
way. That's what I needed help with.

Then again I'm not a Makefile wiz and only tried to poke it for 20
minutes or so. So I might be missing something obvious.

Re: [PATCH 002/160] Makefile: Set NO_GETTEXT=YesPlease on Windows & MinGW

[Erik Faye-Lund]

(msysgit mailing list CC'ed)

On Sun, Nov 14, 2010 at 4:07 PM, Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> Change the Windows and MinGW build defaults to not build the gettext
> tools by default. Gettext hasn't yet been ported to the default
> environments on those systems, although Erik Faye-Lund is working on
> MinGW support.
>
> Reported-by: Johannes Sixt <j6t@kdbg.org>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

I don't think this is exactly what Hannes reported; IIRC he reported
that setting NO_GETTEXT wasn't sufficient to get it to build. Just
unsetting NEEDS_LIBINTL for a platform that doesn't have gettext is
curing the symptoms rather than the decease, shouldn't the build
system and/or the code be fixed so setting NO_GETTEXT is sufficient
instead?

As a side-note, I think it might be time to merge the gettext-branch
into msysGit soon. If no one complains, I'll get it done one of the
next days.

Re: [PATCH 000/160] [PULL] ab/i18n rebased on ab/i18n-prereqs

[Ævar Arnfjörð Bjarmason]

On Sun, Nov 14, 2010 at 16:07, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> Here's an updated ab/i18n rebased on the new ab/i18n-prereqs
> series. The major change here is this patch:

I've rebased ab/i18n and ab/i18n-prereqs again to account for the
master Junio pushed out which contains two of the ab/i18n-prereqs
patches.