* Re: [PATCH] tests: turn on test-lint-shell-syntax by default
From: Torsten Bögershausen @ 2013-01-27 7:43 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Torsten Bögershausen, Jonathan Nieder, git, kraai
In-Reply-To: <7v1ud71uys.fsf@alter.siamese.dyndns.org>
On 26.01.13 22:43, Junio C Hamano wrote:
> Torsten Bögershausen <tboegi@web.de> writes:
>
>> Do we really need "which" to detect if frotz is installed?
> I think we all know the answer to that question is no, but why is
> that a relevant question in the context of this discussion? One of
> us may be very confused.
>
> I thought the topic of this discussion was that, already knowing
> that "which" should never be used anywhere in our scripts, you are
> trying to devise a mechanical way to catch newcomers' attempts to
> use it in their changes, in order to prevent patches that add use of
> "which" to be sent for review to waste our time. I was illustrating
> that the approach to override "which" in a shell function for test
> scripts will not be a useful solution for that goal.
Yes, the diskussion went away.
I would rather see the check-non-portable-shell.pl enabled per default.
It looks as if the "which" command is hard to find, when we want a minimal
risk of false positves.
I can see different solutions:
1) We can make a much simpler expression which only catches the most
common usage of which, like
"if whitch foo".
This will not catch lines like
if test -x "$(which foo 2>/dev/null)"
But I think the -x is not a useful anyway, because which should only
list command which have the executable bit set.
2) We drop the which from check-non-portable-shell.pl
I'll send a patch for 1)
/Torsten
^ permalink raw reply
* [PATCH v2] Reduce false positive in check-non-portable-shell.pl
From: Torsten Bögershausen @ 2013-01-27 7:46 UTC (permalink / raw)
To: git; +Cc: tboegi
check-non-portable-shell.pl is using simple regular expressions to
find illegal shell syntax.
Improve the expressions and reduce the chance for false positves:
"sed -i" must be followed by 1..n whitespace and 1 non whitespace
"declare" must be followed by 1..n whitespace and 1 non whitespace
"echo -n" must be followed by 1..n whitespace and 1 non whitespace
"which": catch lines like "if which foo"
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
---
t/check-non-portable-shell.pl | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
index 8b5a71d..d9ddcdf 100755
--- a/t/check-non-portable-shell.pl
+++ b/t/check-non-portable-shell.pl
@@ -16,10 +16,10 @@ sub err {
while (<>) {
chomp;
- /^\s*sed\s+-i/ and err 'sed -i is not portable';
- /^\s*echo\s+-n/ and err 'echo -n is not portable (please use printf)';
- /^\s*declare\s+/ and err 'arrays/declare not portable';
- /^\s*[^#]\s*which\s/ and err 'which is not portable (please use type)';
+ /^\s*sed\s+-i\s+\S/ and err 'sed -i is not portable';
+ /^\s*echo\s+-n\s+\S/ and err 'echo -n is not portable (please use printf)';
+ /^\s*declare\s+\S/ and err 'arrays/declare not portable';
+ /^\s*if\s+which\s+\S/ and err 'which is not portable (please use type)';
/test\s+[^=]*==/ and err '"test a == b" is not portable (please use =)';
# this resets our $. for each file
close ARGV if eof;
--
1.8.1.1
^ permalink raw reply related
* [RFC v2] git-multimail: a replacement for post-receive-email
From: Michael Haggerty @ 2013-01-27 8:37 UTC (permalink / raw)
To: git discussion list
Cc: Andy Parkins, Sitaram Chamarty, Matthieu Moy, Junio C Hamano,
Marc Branchaud, Ævar Arnfjörð Bjarmason,
Chris Hiestand
A while ago, I submitted an RFC for adding a new email notification
script to "contrib" [1]. The reaction seemed favorable and it was
suggested that the new script should replace post-receive-email rather
than be added separately, ideally with some kind of migration support.
I've been working on this on and off since then and I think it is time
for another iteration. I think I have addressed most of the points
raised earlier, including a migration script and specific migration
instructions.
Review of main advantages of git-multimail over post-receive-email:
* Can (optionally) send a separate email for each new commit (in
addition to the emails for each reference change).
* More flexible configuration, including out-of-the-box support for
running under gitolite.
* Fixed algorithm for detecting "new" commits.
* More information in emails (e.g., commit log subject lines, telling
when a push discards old commits)j.
* Written in Python rather than shell. Easier to extend.
Summary of improvements since the first version:
* Rename the project from the cumbersome "post-receive-multimail.py" to
"git-multimail".
* Push the project into a subdirectory and break it into multiple files
(script, docs, etc).
* Vastly improve the documentation and separate it out of the script
into a README file.
* Add a migration script, migrate-mailhook-config, that converts a
post-receive-email configuration into a git-multimail configuration.
Document the migration procedure and differences between the two scripts
in README.migrate-from-post-receive-email.
* Store the configuration options in namespace "multimailhook.*" rather
than "hooks.*". (The post-receive-email script's use of a too-generic
top-level name was IMHO a bad idea, so fix it now.)
* Allow the feature of sending a separate email for each individual
commit to be turned off via a configuration option (to better support
post-receive-email migrants).
* Re-implement the feature of showing a short log of commits in
announcement emails, configurable via an option.
* Make it possible to import the main code as a Python module to allow
most customization to be done via Python code without the need to edit
the original file. (Note for existing users: the Environment API has
changed since the original RFC, but I will try to keep it stable from
now on.)
* Allow the config settings that define recipient lists to be multivalued.
* Added some testing infrastructure (though the tests are still very
limited).
* Add "Auto-Submitted" headers to emails (as implemented for
post-receive-email by Chris Hiestand).
* Add option to truncate email lines to a specified length (suggested by
Matthieu Moy). By default, this option is *on* and set to 500 characters.
* Add option to force the main part of the email body to be valid UTF-8,
with invalid characters turned into the Unicode replacement character,
U+FFFD. By default, this option is *on* (arguments for turning it off
by default are welcome).
The code is in its own GitHub repository:
https://github.com/mhagger/git-multimail
The script should work with any Python 2.x starting with 2.4, though I
haven't actually tested older Python versions. It does not yet support
Python 3.x.
If it is accepted for the git project, then I would prepare a patch that
drops the git-multimail project's "git-multimail" subdirectory into the
git project as contrib/hooks/git-multimail and optionally deletes the
old post-receive-email script. I am flexible about whether future
development should occur directly in the git project's repository or in
the git-multimail repo with occasional code drops to the git project. I
am also flexible about whether the rough little test scripts should be
included in the git project or kept separate.
It would be very helpful if people would test this script in their own
environments and give me feedback/bug reports. It is rather awkward to
simulate all of the possible environment scenarios myself.
Michael
[1] http://thread.gmane.org/gmane.comp.version-control.git/201433
--
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
^ permalink raw reply
* Re: [PATCH v3 6/8] git-remote-testpy: hash bytes explicitly
From: Michael Haggerty @ 2013-01-27 8:41 UTC (permalink / raw)
To: Sverre Rabbelier; +Cc: Junio C Hamano, John Keeping, Git List
In-Reply-To: <CAGdFq_icLDEdJJKHZsht8bXpZzSNProLt3F_u=0en2rFBvxLKw@mail.gmail.com>
On 01/27/2013 06:30 AM, Sverre Rabbelier wrote:
> On Sat, Jan 26, 2013 at 8:44 PM, Michael Haggerty <mhagger@alum.mit.edu> wrote:
>> So to handle all of the cases across Python versions as closely as
>> possible to the old 2.x code, it might be necessary to make the code
>> explicitly depend on the Python version number, like:
>
> Does this all go away if we restrict ourselves to python 2.6 and just
> use the b prefix?
repo.path ultimately comes from the command line, which means that it is
a bytestring under Python 2.x and a Unicode string under Python 3.x. It
does not come from a literal that could be changed to b"value". (Nor is
a six.b()-like function helpful, if that is what you meant; that is also
intended to wrap literal strings.)
Michael
--
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
^ permalink raw reply
* Re: [PATCH 2/2] mergetools: Make tortoisemerge work with
From: Sven Strickroth @ 2013-01-27 9:14 UTC (permalink / raw)
To: git; +Cc: David Aguilar, Junio C Hamano, Sebastian Schuberth, Jeff King
In-Reply-To: <CAJDDKr6OhZOitTdDkHWnhVhdAis0U+95xUtaNn6nwkQ-k+bA+w@mail.gmail.com>
Am 26.01.2013 08:10 schrieb David Aguilar:
> These patches look correct (I do not have the tool to test)
> but I think we should fixup this commit message.
>
> How about something like...
>
> mergetools: Teach tortoisemerge about TortoiseGitMerge
>
> TortoiseGitMerge improved its syntax to allow for file paths
> with spaces. Detect when it is installed and prefer it over
> TortoiseMerge.
This message implies, that I have to combine two patches again?!
--
Best regards,
Sven Strickroth
PGP key id F5A9D4C4 @ any key-server
^ permalink raw reply
* Re: [PATCH] tests: turn on test-lint-shell-syntax by default
From: Jonathan Nieder @ 2013-01-27 9:31 UTC (permalink / raw)
To: Torsten Bögershausen; +Cc: Junio C Hamano, git, kraai
In-Reply-To: <51037E5F.8090506@web.de>
Hi,
Torsten Bögershausen wrote:
> On 15.01.13 21:38, Junio C Hamano wrote:
>> Torsten Bögershausen <tboegi@web.de> writes:
>>> What do we think about something like this for fishing for which:
[...]
>>> +which () {
>>> + echo >&2 "which is not portable (please use type)"
>>> + exit 1
>>> +}
[...]
>> if (
>> which frotz &&
>> test $(frobonitz --version" -le 2.0
>> )
With the above definition of "which", the only sign of a mistake would
be some extra output to stderr (which is quelled when running tests in
the normal way). The "exit" is caught by the subshell and just makes
the "if" condition false.
That's not so terrible --- it could still dissuade new test authors
from using "which". The downside I'd worry about is that it provides
a false sense of security despite not catching problems like
write_script x <<-EOF &&
# Use "foo" if possible. Otherwise use "bar".
if which foo && test $(foo --version) -le 2.0
then
...
...
EOF
./x
That's not a great tradeoff relative to the impact of the problem
being solved.
Don't get me wrong. I really do want to see more static or dynamic
analysis of git's shell scripts in the future. I fear that for the
tradeoffs to make sense, though, the analysis needs to be more
sophisticated:
* A very common error in test scripts is leaving out the "&&"
connecting adjacent statements, which causes early errors
in a test assertion to be missed and tests to pass by mistake.
Unfortunately the grammar of the dialect of shell used in tests is
not regular enough to make this easily detectable using regexps.
* Another common mistake is using "cd" without entering a subshell.
Detecting this requires counting nested parentheses and noticing
when a parenthesis is quoted.
* Another common mistake is relying on the semantics of variable
assignments in front of function calls. Detecting this requires
recognizing which commands are function calls.
In the end the analysis that works best would probably involve a
full-fledged shell script parser. Something like "sparse", except for
shell command language.
Sorry I don't have more practical advice in the short term.
My two cents,
Jonathan
^ permalink raw reply
* Re: [PATCH 2/2] fetch-pack: avoid repeatedly re-scanning pack directory
From: Jonathan Nieder @ 2013-01-27 10:27 UTC (permalink / raw)
To: Jeff King; +Cc: git, Junio C Hamano
In-Reply-To: <20130126224043.GB20849@sigill.intra.peff.net>
Hi,
Jeff King wrote:
> When we look up a sha1 object for reading, we first check
> packfiles, and then loose objects. If we still haven't found
> it, we re-scan the list of packfiles in `objects/pack`. This
> final step ensures that we can co-exist with a simultaneous
> repack process which creates a new pack and then prunes the
> old object.
I like the context above and what follows it, but I think you forgot
to mention what the patch actually does. :)
I guess it is:
However, in the first scan over refs in fetch-pack.c::everything_local,
this double-check of packfiles is not necessary since we are only
trying to get a rough estimate of the last time we fetched from this
remote repository in order to find good candidate common commits ---
a missed object would only result in a slightly slower fetch.
Avoid that slow second scan in the common case by guarding the object
lookup with has_sha1_file().
Sounds like it would not affect most fetches except by making them
a lot faster in the many-refs case, so for what it's worth I like it.
I had not read this codepath before. I'm left with a few questions:
* Why is 49bb805e ("Do not ask for objects known to be complete",
2005-10-19) trying to do? Are we hurting that in any way?
For the sake of an example, suppose in my stalled project I
maintain 20 topic branches against an unmoving mainline I do not
advertise and you regularly fetch from me. The cutoff is the
*newest* commit date of any of my topic branches you already have.
By declaring you have that topic branch you avoid a complicated
negotiation to discover that we both have the mainline. Is that
the goal?
* Is has_sha1_file() generally succeptible to the race against repack
you mentioned? How is that normally dealt with?
* Can a slow operation get confused if an object is incorporated into
a pack and then expelled again by two repacks in sequence?
Thanks,
Jonathan
^ permalink raw reply
* Behavior of stash apply vs merge
From: Robin Rosenberg @ 2013-01-27 11:35 UTC (permalink / raw)
To: Git List
In-Reply-To: <1192924141.1697155.1359285809347.JavaMail.root@dewire.com>
Hi,
What good reason is it that 'git stash apply' gives hairy conflict markers, while 'git merge stash' does not. No renames involved.
-- robin
^ permalink raw reply
* Re: [PATCH v2 2/3] branch: give a more helpful message on redundant arguments
From: Jonathan Nieder @ 2013-01-27 11:58 UTC (permalink / raw)
To: Nguyễn Thái Ngọc Duy; +Cc: git, Junio C Hamano, Matthieu Moy
In-Reply-To: <1359118225-14356-2-git-send-email-pclouds@gmail.com>
Nguyễn Thái Ngọc Duy wrote:
> --- a/builtin/branch.c
> +++ b/builtin/branch.c
> @@ -852,14 +852,14 @@ int cmd_branch(int argc, const char **argv, const char *prefix)
> const char *branch_name;
> struct strbuf branch_ref = STRBUF_INIT;
>
> - if (detached)
> - die("Cannot give description to detached HEAD");
> - if (!argc)
> + if (!argc) {
> + if (detached)
> + die("Cannot give description to detached HEAD");
Good catch. Shouldn't this bugfix be a separate patch, so it can also
be included in maint?
^ permalink raw reply
* Re: [PATCH v2 3/3] branch: mark more strings for translation
From: Jonathan Nieder @ 2013-01-27 11:55 UTC (permalink / raw)
To: Nguyễn Thái Ngọc Duy; +Cc: git, Junio C Hamano, Matthieu Moy
In-Reply-To: <1359118225-14356-3-git-send-email-pclouds@gmail.com>
Nguyễn Thái Ngọc Duy wrote:
> --- a/builtin/branch.c
> +++ b/builtin/branch.c
> @@ -466,7 +466,7 @@ static void add_verbose_info(struct strbuf *out, struct ref_item *item,
> int verbose, int abbrev)
> {
> struct strbuf subject = STRBUF_INIT, stat = STRBUF_INIT;
> - const char *sub = " **** invalid ref ****";
> + const char *sub = _(" **** invalid ref ****");
This worried me for a second --- is it an actual message that gets
emitted, a placeholder used only in code, or some combination of
the two?
Luckily it really is just a message (or rather, a value for the commit
message column in the " f7c0c00 [ahead 58, behind 197] vcs-svn: drop
obj_pool.h" message).
For what it's worth, assuming this passes tests,
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
^ permalink raw reply
* Re: [PATCH v2] add: warn when -u or -A is used without filepattern
From: Jonathan Nieder @ 2013-01-27 12:22 UTC (permalink / raw)
To: Matthieu Moy
Cc: git, gitster, Robin Rosenberg, Piotr Krukowiecki,
Eric James Michael Ritz, Tomas Carnecky
In-Reply-To: <1359110978-20054-1-git-send-email-Matthieu.Moy@imag.fr>
Hi Matthieu,
Matthieu Moy wrote:
> --- a/builtin/add.c
> +++ b/builtin/add.c
[...]
> @@ -392,8 +420,14 @@ int cmd_add(int argc, const char **argv, const char *prefix)
> die(_("-A and -u are mutually incompatible"));
> if (!show_only && ignore_missing)
> die(_("Option --ignore-missing can only be used together with --dry-run"));
> - if ((addremove || take_worktree_changes) && !argc) {
> + if (addremove)
> + option_with_implicit_dot = "--all";
> + if (take_worktree_changes)
> + option_with_implicit_dot = "--update";
I agree with Junio that these are most often spelled as "-A" and "-u".
> + if (option_with_implicit_dot && !argc) {
> static const char *here[2] = { ".", NULL };
> + if (prefix)
> + warn_pathless_add(option_with_implicit_dot);
For what it's worth, with or without s/--all/-A/ and s/--update/-u/,
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Thanks. If someone wants to preserve the spelling of the option name
passed by the user, that can happen as a patch on top.
^ permalink raw reply
* [PATCH v4 0/2] for-each-repo: new command for multi-repo operations
From: Lars Hjemli @ 2013-01-27 12:46 UTC (permalink / raw)
To: git; +Cc: Lars Hjemli
Changes since v3:
* option -x used to execute non-git commands
* option -z used to NUL-terminate paths
* write_name_quoted() used to print repo paths
* repos are handled in sorted order (as defined by strcmp(3)) to get
predictable output from the command
* unsetenv() reintroduced to avoid problems from GIT_DIR/WORK_TREE
* more tests
Lars Hjemli (2):
for-each-repo: new command used for multi-repo operations
git: rewrite `git -a` to become a git-for-each-repo command
.gitignore | 1 +
Documentation/git-for-each-repo.txt | 71 ++++++++++++
Makefile | 1 +
builtin.h | 1 +
builtin/for-each-repo.c | 145 ++++++++++++++++++++++++
git.c | 37 +++++++
t/t6400-for-each-repo.sh | 213 ++++++++++++++++++++++++++++++++++++
7 files changed, 469 insertions(+)
create mode 100644 Documentation/git-for-each-repo.txt
create mode 100644 builtin/for-each-repo.c
create mode 100755 t/t6400-for-each-repo.sh
--
1.8.1.1.349.g4cdd23e
^ permalink raw reply
* [PATCH v4 1/2] for-each-repo: new command used for multi-repo operations
From: Lars Hjemli @ 2013-01-27 12:46 UTC (permalink / raw)
To: git; +Cc: Lars Hjemli
In-Reply-To: <1359290777-5483-1-git-send-email-hjemli@gmail.com>
When working with multiple, unrelated (or loosly related) git repos,
there is often a need to locate all repos with uncommitted work and
perform some action on them (say, commit and push). Before this patch,
such tasks would require manually visiting all repositories, running
`git status` within each one and then decide what to do next.
This mundane task can now be automated by e.g. `git for-each-repo --dirty
status`, which will find all non-bare git repositories below the current
directory (even nested ones), check if they are dirty (as defined by
`git diff --quiet && git diff --cached --quiet`), and for each dirty repo
print the path to the repo and then execute `git status` within the repo.
The command also honours the option '--clean' which restricts the set of
repos to those which '--dirty' would skip, and '-x' which is used to
execute non-git commands.
Finally, the command to execute within each repo is optional. If none is
given, git-for-each-repo will just print the path to each repo found. And
since the command supports -z, this can be used for more advanced scripting
needs.
Note: since git-for-each-repo can execute both git- and nongit commands, it
must cd into the worktree of each repository before executing the command.
It is then no need for the environment variables $GIT_WORK_TREE and $GIT_DIR
to be specified, so git-for-each-repo will instead unset these variables to
stop them from interfering with the executed commands.
Signed-off-by: Lars Hjemli <hjemli@gmail.com>
---
.gitignore | 1 +
Documentation/git-for-each-repo.txt | 71 +++++++++++++++++
Makefile | 1 +
builtin.h | 1 +
builtin/for-each-repo.c | 145 ++++++++++++++++++++++++++++++++++
git.c | 1 +
t/t6400-for-each-repo.sh | 150 ++++++++++++++++++++++++++++++++++++
7 files changed, 370 insertions(+)
create mode 100644 Documentation/git-for-each-repo.txt
create mode 100644 builtin/for-each-repo.c
create mode 100755 t/t6400-for-each-repo.sh
diff --git a/.gitignore b/.gitignore
index aa258a6..0c27981 100644
--- a/.gitignore
+++ b/.gitignore
@@ -56,6 +56,7 @@
/git-filter-branch
/git-fmt-merge-msg
/git-for-each-ref
+/git-for-each-repo
/git-format-patch
/git-fsck
/git-fsck-objects
diff --git a/Documentation/git-for-each-repo.txt b/Documentation/git-for-each-repo.txt
new file mode 100644
index 0000000..fb12b3f
--- /dev/null
+++ b/Documentation/git-for-each-repo.txt
@@ -0,0 +1,71 @@
+git-for-each-repo(1)
+====================
+
+NAME
+----
+git-for-each-repo - Execute a git command in multiple non-bare repositories
+
+SYNOPSIS
+--------
+[verse]
+'git for-each-repo' [-acdxz] [command]
+
+DESCRIPTION
+-----------
+The git-for-each-repo command is used to locate all non-bare git
+repositories within the current directory tree, and optionally
+execute a git command in each of the found repos.
+
+OPTIONS
+-------
+-a::
+--all::
+ Include both clean and dirty repositories (this is the default
+ behaviour of `git-for-each-repo`).
+
+-c::
+--clean::
+ Only include repositories with a clean worktree.
+
+-d::
+--dirty::
+ Only include repositories with a dirty worktree.
+
+-x::
+ Execute a genric (non-git) command in each repo.
+
+-z::
+ Terminate each path name with the NUL character.
+
+EXAMPLES
+--------
+
+Various ways to exploit this command::
++
+------------
+$ git for-each-repo <1>
+$ git for-each-repo fetch <2>
+$ git for-each-repo -d gui <3>
+$ git for-each-repo -c push <4>
+$ git for-each-repo -x du -sh <5>
+------------
++
+<1> Print the path to all repos found below the current directory.
+
+<2> Fetch updates from default remote in all repos.
+
+<3> Start linkgit:git-gui[1] in each repo containing uncommitted changes.
+
+<4> Push the current branch in each repo with no uncommited changes.
+
+<5> Print disk-usage for each repository.
+
+NOTES
+-----
+
+For the purpose of `git-for-each-repo`, a dirty worktree is defined as a
+worktree with uncommitted changes.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index a786d4c..8c42c17 100644
--- a/Makefile
+++ b/Makefile
@@ -870,6 +870,7 @@ BUILTIN_OBJS += builtin/fetch-pack.o
BUILTIN_OBJS += builtin/fetch.o
BUILTIN_OBJS += builtin/fmt-merge-msg.o
BUILTIN_OBJS += builtin/for-each-ref.o
+BUILTIN_OBJS += builtin/for-each-repo.o
BUILTIN_OBJS += builtin/fsck.o
BUILTIN_OBJS += builtin/gc.o
BUILTIN_OBJS += builtin/grep.o
diff --git a/builtin.h b/builtin.h
index 7e7bbd6..02fc712 100644
--- a/builtin.h
+++ b/builtin.h
@@ -73,6 +73,7 @@ extern int cmd_fetch(int argc, const char **argv, const char *prefix);
extern int cmd_fetch_pack(int argc, const char **argv, const char *prefix);
extern int cmd_fmt_merge_msg(int argc, const char **argv, const char *prefix);
extern int cmd_for_each_ref(int argc, const char **argv, const char *prefix);
+extern int cmd_for_each_repo(int argc, const char **argv, const char *prefix);
extern int cmd_format_patch(int argc, const char **argv, const char *prefix);
extern int cmd_fsck(int argc, const char **argv, const char *prefix);
extern int cmd_gc(int argc, const char **argv, const char *prefix);
diff --git a/builtin/for-each-repo.c b/builtin/for-each-repo.c
new file mode 100644
index 0000000..9333ae0
--- /dev/null
+++ b/builtin/for-each-repo.c
@@ -0,0 +1,145 @@
+/*
+ * "git for-each-repo" builtin command.
+ *
+ * Copyright (c) 2013 Lars Hjemli <hjemli@gmail.com>
+ */
+#include "cache.h"
+#include "color.h"
+#include "quote.h"
+#include "builtin.h"
+#include "run-command.h"
+#include "parse-options.h"
+
+#define ALL 0
+#define DIRTY 1
+#define CLEAN 2
+
+static char *color = GIT_COLOR_NORMAL;
+static int eol = '\n';
+static int match;
+static int runopt = RUN_GIT_CMD;
+
+static const char * const builtin_foreachrepo_usage[] = {
+ N_("git for-each-repo [-acdxz] [cmd]"),
+ NULL
+};
+
+static struct option builtin_foreachrepo_options[] = {
+ OPT_SET_INT('a', "all", &match, N_("match both clean and dirty repositories"), ALL),
+ OPT_SET_INT('c', "clean", &match, N_("only show clean repositories"), CLEAN),
+ OPT_SET_INT('d', "dirty", &match, N_("only show dirty repositories"), DIRTY),
+ OPT_SET_INT('x', NULL, &runopt, N_("execute generic (non-git) command"), 0),
+ OPT_SET_INT('z', NULL, &eol, N_("terminate each repo path with NUL character"), 0),
+ OPT_END(),
+};
+
+static int get_repo_state(const char *dir)
+{
+ const char *diffidx[] = {"diff", "--quiet", "--cached", NULL};
+ const char *diffwd[] = {"diff", "--quiet", NULL};
+
+ if (run_command_v_opt_cd_env(diffidx, RUN_GIT_CMD, dir, NULL) != 0)
+ return DIRTY;
+ if (run_command_v_opt_cd_env(diffwd, RUN_GIT_CMD, dir, NULL) != 0)
+ return DIRTY;
+ return CLEAN;
+}
+
+static void print_repo_path(const char *path, unsigned pretty)
+{
+ if (path[0] == '.' && path[1] == '/')
+ path += 2;
+ if (pretty)
+ color_fprintf_ln(stdout, color, "[%s]", path);
+ else
+ write_name_quoted(path, stdout, eol);
+}
+
+static void handle_repo(struct strbuf *path, const char **argv)
+{
+ const char *gitdir;
+ int len;
+
+ len = path->len;
+ strbuf_addstr(path, ".git");
+ gitdir = resolve_gitdir(path->buf);
+ strbuf_setlen(path, len - 1);
+ if (!gitdir)
+ goto done;
+ if (match != ALL && match != get_repo_state(path->buf))
+ goto done;
+ print_repo_path(path->buf, *argv != NULL);
+ if (*argv)
+ run_command_v_opt_cd_env(argv, runopt, path->buf, NULL);
+done:
+ strbuf_addstr(path, "/");
+}
+
+static int walk(struct strbuf *path, int argc, const char **argv)
+{
+ DIR *dir;
+ struct dirent *ent;
+ struct stat st;
+ size_t len;
+ int has_dotgit = 0;
+ struct string_list list = STRING_LIST_INIT_DUP;
+ struct string_list_item *item;
+
+ dir = opendir(path->buf);
+ if (!dir)
+ return errno;
+ strbuf_addstr(path, "/");
+ len = path->len;
+ while ((ent = readdir(dir))) {
+ if (!strcmp(ent->d_name, ".") || !strcmp(ent->d_name, ".."))
+ continue;
+ if (!strcmp(ent->d_name, ".git")) {
+ has_dotgit = 1;
+ continue;
+ }
+ switch (DTYPE(ent)) {
+ case DT_UNKNOWN:
+ case DT_LNK:
+ /* Use stat() to figure out if this path leads
+ * to a directory - it's not important if it's
+ * a symlink which gets us there.
+ */
+ strbuf_setlen(path, len);
+ strbuf_addstr(path, ent->d_name);
+ if (stat(path->buf, &st) || !S_ISDIR(st.st_mode))
+ break;
+ /* fallthrough */
+ case DT_DIR:
+ string_list_append(&list, ent->d_name);
+ break;
+ }
+ }
+ closedir(dir);
+ strbuf_setlen(path, len);
+ if (has_dotgit)
+ handle_repo(path, argv);
+ sort_string_list(&list);
+ for_each_string_list_item(item, &list) {
+ strbuf_setlen(path, len);
+ strbuf_addstr(path, item->string);
+ walk(path, argc, argv);
+ }
+ string_list_clear(&list, 0);
+ return 0;
+}
+
+int cmd_for_each_repo(int argc, const char **argv, const char *prefix)
+{
+ struct strbuf path = STRBUF_INIT;
+
+ unsetenv(GIT_DIR_ENVIRONMENT);
+ unsetenv(GIT_WORK_TREE_ENVIRONMENT);
+ argc = parse_options(argc, argv, prefix,
+ builtin_foreachrepo_options,
+ builtin_foreachrepo_usage,
+ PARSE_OPT_STOP_AT_NON_OPTION);
+ if (want_color(GIT_COLOR_AUTO))
+ color = GIT_COLOR_YELLOW;
+ strbuf_addstr(&path, ".");
+ return walk(&path, argc, argv);
+}
diff --git a/git.c b/git.c
index ed66c66..6b53169 100644
--- a/git.c
+++ b/git.c
@@ -337,6 +337,7 @@ static void handle_internal_command(int argc, const char **argv)
{ "fetch-pack", cmd_fetch_pack, RUN_SETUP },
{ "fmt-merge-msg", cmd_fmt_merge_msg, RUN_SETUP },
{ "for-each-ref", cmd_for_each_ref, RUN_SETUP },
+ { "for-each-repo", cmd_for_each_repo },
{ "format-patch", cmd_format_patch, RUN_SETUP },
{ "fsck", cmd_fsck, RUN_SETUP },
{ "fsck-objects", cmd_fsck, RUN_SETUP },
diff --git a/t/t6400-for-each-repo.sh b/t/t6400-for-each-repo.sh
new file mode 100755
index 0000000..af02c0c
--- /dev/null
+++ b/t/t6400-for-each-repo.sh
@@ -0,0 +1,150 @@
+#!/bin/sh
+#
+# Copyright (c) 2013 Lars Hjemli
+#
+
+test_description='Test the git-for-each-repo command'
+
+. ./test-lib.sh
+
+qname="with\"quote"
+qqname="\"with\\\"quote\""
+
+test_expect_success "setup" '
+ test_create_repo clean &&
+ (cd clean && test_commit foo1) &&
+ git init --separate-git-dir=.cleansub clean/gitfile &&
+ (cd clean/gitfile && test_commit foo2 && echo bar >>foo2.t) &&
+ test_create_repo dirty-idx &&
+ (cd dirty-idx && test_commit foo3 && git rm foo3.t) &&
+ test_create_repo dirty-wt &&
+ (cd dirty-wt && mv .git .linkedgit && ln -s .linkedgit .git &&
+ test_commit foo4 && rm foo4.t) &&
+ test_create_repo "$qname" &&
+ (cd "$qname" && test_commit foo5) &&
+ mkdir fakedir && mkdir fakedir/.git
+'
+
+test_expect_success "without filtering, all repos are included" '
+ echo "." >expect &&
+ echo "clean" >>expect &&
+ echo "clean/gitfile" >>expect &&
+ echo "dirty-idx" >>expect &&
+ echo "dirty-wt" >>expect &&
+ echo "$qqname" >>expect &&
+ git for-each-repo >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success "-z NUL-terminates each path" '
+ echo "(.)" >expect &&
+ echo "(clean)" >>expect &&
+ echo "(clean/gitfile)" >>expect &&
+ echo "(dirty-idx)" >>expect &&
+ echo "(dirty-wt)" >>expect &&
+ echo "($qname)" >>expect &&
+ git for-each-repo -z | xargs -0 printf "(%s)\n" >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success "--dirty only includes dirty repos" '
+ echo "clean/gitfile" >expect &&
+ echo "dirty-idx" >>expect &&
+ echo "dirty-wt" >>expect &&
+ git for-each-repo --dirty >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success "--clean only includes clean repos" '
+ echo "." >expect &&
+ echo "clean" >>expect &&
+ echo "$qqname" >>expect &&
+ git for-each-repo --clean >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success "run a git-command in all repos" '
+ echo "[.]" >expect &&
+ echo "[clean]" >>expect &&
+ echo "[clean/gitfile]" >>expect &&
+ echo " M foo2.t" >>expect &&
+ echo "[dirty-idx]" >>expect &&
+ echo "D foo3.t" >>expect &&
+ echo "[dirty-wt]" >>expect &&
+ echo " D foo4.t" >> expect
+ echo "[$qname]" >>expect &&
+ git for-each-repo status -suno >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success "run a git-command in dirty repos only" '
+ echo "[clean/gitfile]" >expect &&
+ echo " M foo2.t" >>expect &&
+ echo "[dirty-idx]" >>expect &&
+ echo "D foo3.t" >>expect &&
+ echo "[dirty-wt]" >>expect &&
+ echo " D foo4.t" >> expect
+ git for-each-repo -d status -suno >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success "run a git-command in clean repos only" '
+ echo "[.]" >expect &&
+ echo "[clean]" >>expect &&
+ echo "foo1.t" >>expect &&
+ echo "[$qname]" >>expect &&
+ echo "foo5.t" >>expect &&
+ git for-each-repo -c ls-files >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success "-z is disabled when a command is run" '
+ echo "[.]" >expect &&
+ echo "[clean]" >>expect &&
+ echo "foo1.t" >>expect &&
+ echo "[$qname]" >>expect &&
+ echo "foo5.t" >>expect &&
+ git for-each-repo -cz ls-files >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success "-x executes any command in each repo" '
+ echo "[.]" >expect &&
+ echo "$HOME" >>expect &&
+ echo "[clean]" >>expect &&
+ echo "$HOME/clean" >>expect &&
+ echo "[clean/gitfile]" >>expect &&
+ echo "$HOME/clean/gitfile" >>expect &&
+ echo "[dirty-idx]" >>expect &&
+ echo "$HOME/dirty-idx" >>expect &&
+ echo "[dirty-wt]" >>expect &&
+ echo "$HOME/dirty-wt" >> expect
+ echo "[$qname]" >>expect &&
+ echo "$HOME/$qname" >>expect &&
+ git for-each-repo -x pwd >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success "-cx executes any command in clean repos" '
+ echo "[.]" >expect &&
+ echo "$HOME" >>expect &&
+ echo "[clean]" >>expect &&
+ echo "$HOME/clean" >>expect &&
+ echo "[$qname]" >>expect &&
+ echo "$HOME/$qname" >>expect &&
+ git for-each-repo -cx pwd >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success "-dx executes any command in dirty repos" '
+ echo "[clean/gitfile]" >expect &&
+ echo "$HOME/clean/gitfile" >>expect &&
+ echo "[dirty-idx]" >>expect &&
+ echo "$HOME/dirty-idx" >>expect &&
+ echo "[dirty-wt]" >>expect &&
+ echo "$HOME/dirty-wt" >> expect
+ git for-each-repo -dx pwd >actual &&
+ test_cmp expect actual
+'
+
+test_done
--
1.8.1.1.349.g4cdd23e
^ permalink raw reply related
* [PATCH v4 2/2] git: rewrite `git -a` to become a git-for-each-repo command
From: Lars Hjemli @ 2013-01-27 12:46 UTC (permalink / raw)
To: git; +Cc: Lars Hjemli
In-Reply-To: <1359290777-5483-1-git-send-email-hjemli@gmail.com>
With this rewriting, it is now possible to run e.g. `git -ad gui` to
start up git-gui in each repo within the current directory which
contains uncommited work.
Signed-off-by: Lars Hjemli <hjemli@gmail.com>
---
git.c | 36 +++++++++++++++++++++++++++
t/t6400-for-each-repo.sh | 63 ++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 99 insertions(+)
diff --git a/git.c b/git.c
index 6b53169..f933b5d 100644
--- a/git.c
+++ b/git.c
@@ -31,8 +31,42 @@ static void commit_pager_choice(void) {
}
}
+/*
+ * Rewrite 'git -ad status' to 'git for-each-repo -d status'
+ */
+static int rewrite_foreach_repo(const char ***orig_argv,
+ const char **curr_argv,
+ int *curr_argc)
+{
+ const char **new_argv;
+ char *tmp;
+ int new_argc, curr_pos, i, j;
+
+ curr_pos = curr_argv - *orig_argv;
+ if (strlen(curr_argv[0]) == 2) {
+ curr_argv[0] = "for-each-repo";
+ return curr_pos - 1;
+ }
+
+ new_argc = curr_pos + *curr_argc + 1;
+ new_argv = xmalloc(new_argc * sizeof(void *));
+ for (i = j = 0; j < new_argc; i++, j++) {
+ if (i == curr_pos) {
+ asprintf(&tmp, "-%s", (*orig_argv)[i] + 2);
+ new_argv[j] = "for-each-repo";
+ new_argv[++j] = tmp;
+ } else {
+ new_argv[j] = (*orig_argv)[i];
+ }
+ }
+ *orig_argv = new_argv;
+ (*curr_argc)++;
+ return curr_pos;
+}
+
static int handle_options(const char ***argv, int *argc, int *envchanged)
{
+ const char ***pargv = argv;
const char **orig_argv = *argv;
while (*argc > 0) {
@@ -143,6 +177,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
setenv(GIT_LITERAL_PATHSPECS_ENVIRONMENT, "0", 1);
if (envchanged)
*envchanged = 1;
+ } else if (!strncmp(cmd, "-a", 2)) {
+ return rewrite_foreach_repo(pargv, *argv, argc);
} else {
fprintf(stderr, "Unknown option: %s\n", cmd);
usage(git_usage_string);
diff --git a/t/t6400-for-each-repo.sh b/t/t6400-for-each-repo.sh
index af02c0c..eaa4518 100755
--- a/t/t6400-for-each-repo.sh
+++ b/t/t6400-for-each-repo.sh
@@ -147,4 +147,67 @@ test_expect_success "-dx executes any command in dirty repos" '
test_cmp expect actual
'
+test_expect_success "rewrite 'git -a'" '
+ echo "." >expect &&
+ echo "clean" >>expect &&
+ echo "clean/gitfile" >>expect &&
+ echo "dirty-idx" >>expect &&
+ echo "dirty-wt" >>expect &&
+ echo "$qqname" >>expect &&
+ git -a >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success "rewrite 'git -az'" '
+ echo "(.)" >expect &&
+ echo "(clean)" >>expect &&
+ echo "(clean/gitfile)" >>expect &&
+ echo "(dirty-idx)" >>expect &&
+ echo "(dirty-wt)" >>expect &&
+ echo "($qname)" >>expect &&
+ git -az | xargs -0 printf "(%s)\n" >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success "rewrite 'git -ad'" '
+ echo "clean/gitfile" >expect &&
+ echo "dirty-idx" >>expect &&
+ echo "dirty-wt" >>expect &&
+ git -ad >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success "rewrite 'git -ac'" '
+ echo "." >expect &&
+ echo "clean" >>expect &&
+ echo "$qqname" >>expect &&
+ git -ac >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success "rewrite 'git -a status -suno'" '
+ echo "[.]" >expect &&
+ echo "[clean]" >>expect &&
+ echo "[clean/gitfile]" >>expect &&
+ echo " M foo2.t" >>expect &&
+ echo "[dirty-idx]" >>expect &&
+ echo "D foo3.t" >>expect &&
+ echo "[dirty-wt]" >>expect &&
+ echo " D foo4.t" >> expect
+ echo "[$qname]" >>expect &&
+ git -a status -suno >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success "rewrite 'git -acx pwd'" '
+ echo "[.]" >expect &&
+ echo "$HOME" >>expect &&
+ echo "[clean]" >>expect &&
+ echo "$HOME/clean" >>expect &&
+ echo "[$qname]" >>expect &&
+ echo "$HOME/$qname" >>expect &&
+ git -acx pwd >actual &&
+ test_cmp expect actual
+'
+
test_done
--
1.8.1.1.349.g4cdd23e
^ permalink raw reply related
* Re: [PATCH] tests: turn on test-lint-shell-syntax by default
From: Torsten Bögershausen @ 2013-01-27 13:13 UTC (permalink / raw)
To: Jonathan Nieder; +Cc: Torsten Bögershausen, Junio C Hamano, git, kraai
In-Reply-To: <20130127093121.GA4228@elie.Belkin>
On 27.01.13 10:31, Jonathan Nieder wrote:
> Hi,
>
> Torsten Bögershausen wrote:
>> On 15.01.13 21:38, Junio C Hamano wrote:
>>> Torsten Bögershausen <tboegi@web.de> writes:
>
>>>> What do we think about something like this for fishing for which:
> [...]
>>>> +which () {
>>>> + echo >&2 "which is not portable (please use type)"
>>>> + exit 1
>>>> +}
> [...]
>>> if (
>>> which frotz &&
>>> test $(frobonitz --version" -le 2.0
>>> )
>
> With the above definition of "which", the only sign of a mistake would
> be some extra output to stderr (which is quelled when running tests in
> the normal way). The "exit" is caught by the subshell and just makes
> the "if" condition false.
>
> That's not so terrible --- it could still dissuade new test authors
> from using "which". The downside I'd worry about is that it provides
> a false sense of security despite not catching problems like
>
> write_script x <<-EOF &&
> # Use "foo" if possible. Otherwise use "bar".
> if which foo && test $(foo --version) -le 2.0
> then
> ...
> ...
> EOF
> ./x
>
> That's not a great tradeoff relative to the impact of the problem
> being solved.
>
> Don't get me wrong. I really do want to see more static or dynamic
> analysis of git's shell scripts in the future. I fear that for the
> tradeoffs to make sense, though, the analysis needs to be more
> sophisticated:
>
> * A very common error in test scripts is leaving out the "&&"
> connecting adjacent statements, which causes early errors
> in a test assertion to be missed and tests to pass by mistake.
> Unfortunately the grammar of the dialect of shell used in tests is
> not regular enough to make this easily detectable using regexps.
>
> * Another common mistake is using "cd" without entering a subshell.
> Detecting this requires counting nested parentheses and noticing
> when a parenthesis is quoted.
>
> * Another common mistake is relying on the semantics of variable
> assignments in front of function calls. Detecting this requires
> recognizing which commands are function calls.
>
> In the end the analysis that works best would probably involve a
> full-fledged shell script parser. Something like "sparse", except for
> shell command language.
>
> Sorry I don't have more practical advice in the short term.
>
> My two cents,
> Jonathan
Jonathan,
thanks for the review.
My ambition is to get the check-non-portable-shell.pl into a shape
that we can enable it by default.
This may be with or without checking for "which".
As a first step we will hopefully see less breakage for e.g. Mac OS
caused by "echo -n" or "sed -i" usage.
On the longer run, we may be able to have something more advanced.
Back to the which:
Writing a t0009-no-which.sh like this:
--------------------
#!/bin/sh
test_description='Test the which'
. ./test-lib.sh
which () {
echo >&2 "which is not portable (please use type)"
exit 1
}
test_expect_success 'which is not portable' '
if which frotz
then
say "frotz does not exist"
else
say "frotz does exist"
fi
'
test_done
--------------
and running "make test" gives the following, at least in my system:
[snip]
*** t0009-no-which.sh ***
FATAL: Unexpected exit with code 1
make[2]: *** [t0009-no-which.sh] Error 1
make[1]: *** [test] Error 2
make: *** [test] Error 2
-----------------------
running inside t/
./t0009-no-which.sh --verbose
Initialized empty Git repository in /Users/tb/projects/git/tb/t/trash directory.t0009-no-which/.git/
expecting success:
if which frotz
then
say "frotz does not exist"
else
say "frotz does exist"
fi
which is not portable (please use type)
FATAL: Unexpected exit with code 1
/Torsten
^ permalink raw reply
* Re: git-svn problems with white-space in tag names
From: Hans-Juergen Euler @ 2013-01-27 14:12 UTC (permalink / raw)
To: git
In-Reply-To: <CAK3CF+4GPKBfAmgsHYnf_6nCCOCe-1d31cGsWp4jkKC28cZr0g@mail.gmail.com>
This seems to be a problem of the windows version. At least with its
complete severity. Installed git on Ubuntu in a virtual machine was
able to clone the subversion repos past the tag with the white-space
at the end. I am not sure but apparently this tag has not been
converted.
The git repos I could copy from Ubuntu to windows. So far no problems
seen in this copy on windows.
2013/1/23 Hans-Juergen Euler <waas.nett@gmail.com>:
> I have discussed already the problem a bit more in this thread
> groups.google.com/d/topic/git-users/kfMFZ3uEFsM/discussion
>
> -----Operating system (specifically which version)
> windows 7 64 bit
>
> ------Git version (git --version)
> Git version 1.8.0 for windows obviously.
> git bash and git gui installed and using
>
> ------Git configuration (system, home, repository)
> hmm guess is covered with git bash and git gui. Using the standard config stuff
>
> using subversion
> TortoiseSVN 1.7.11
> Subversion 1.7.8
> Was typically always up-to-date (within 2 months or so) with previous versions
>
> using an external subversion provider for storing the information
> externally. guess the version there is older but do not know
>
>
> I have tried to convert some of my external subversion data bases with
> git-svn clone
>
> I have encountered a problem with one of my subversion repos. I have
> obviously introduced about 2 years ago a problem with an additional
> white space at the end of tag name.
>
> So there is an entry "tags/blabla "
>
> in the subversion repos. The sequential handling of the svn repos with
> git-svn gets stuck there. I could not find a way around this. My guess
> is that the white-space was introduced by accident on windows by
> Tortoise-SVN.
> Unfortunately this occurs at revision 90 something and I have almost
> 1000 revisions stored.
>
> Let me know if you need more details.Thanks.
^ permalink raw reply
* Re: [PATCH v3 6/8] git-remote-testpy: hash bytes explicitly
From: John Keeping @ 2013-01-27 14:13 UTC (permalink / raw)
To: Michael Haggerty; +Cc: Junio C Hamano, git, Sverre Rabbelier
In-Reply-To: <5104B0B5.1030501@alum.mit.edu>
On Sun, Jan 27, 2013 at 05:44:37AM +0100, Michael Haggerty wrote:
> On 01/26/2013 10:44 PM, Junio C Hamano wrote:
> > John Keeping <john@keeping.me.uk> writes:
> >> @@ -45,7 +45,7 @@ def get_repo(alias, url):
> >> repo.get_head()
> >>
> >> hasher = _digest()
> >> - hasher.update(repo.path)
> >> + hasher.update(repo.path.encode('utf-8'))
> >> repo.hash = hasher.hexdigest()
> >>
> >> repo.get_base_path = lambda base: os.path.join(
>
> This will still fail under Python 2.x if repo.path is a byte string that
> contains non-ASCII characters.
I had forgotten about Python 2 while doing this.
> And it will fail under Python 3.1 and
> later if repo.path contains characters using the surrogateescape
> encoding option [1], as it will if the original command-line argument
> contained bytes that cannot be decoded into Unicode using the user's
> default encoding:
Interesting. I wasn't aware of the "surrogateescape" error handler.
> 'surrogateescape' is not supported in Python 3.0, but I think it would
> be quite acceptable only to support Python 3.x for x >= 1.
I agree.
> But 'surrogateescape' doesn't seem to be supported at all in Python 2.x
> (I tested 2.7.3 and it's not there).
>
> Here you don't really need byte-for-byte correctness; it would be enough
> to get *some* byte string that is unique for a given input (ideally,
> consistent with ASCII or UTF-8 for backwards compatibility). So you
> could use
>
> b = s.encode('utf-8', 'backslashreplace')
>
> Unfortunately, this doesn't work under Python 2.x:
>
> $ python2 -c "
> import sys
> print(repr(sys.argv[1]))
> print(repr(sys.argv[1].encode('utf-8', 'backslashreplace')))
> " $(echo français|iconv -t latin1)
> 'fran\xe7ais'
> Traceback (most recent call last):
> File "<string>", line 4, in <module>
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe7 in position
> 4: ordinal not in range(128)
>
> Apparently when you call bytestring.encode(), Python first tries to
> decode the string to Unicode using the 'ascii' encoding.
Actually it appears to use sys.getdefaultencoding() to do this initial
decode. Not that it makes much difference here since the failure is the
same.
> So to handle all of the cases across Python versions as closely as
> possible to the old 2.x code, it might be necessary to make the code
> explicitly depend on the Python version number, like:
>
> hasher = _digest()
> if sys.hexversion < 0x03000000:
> pathbytes = repo.path
> elif sys.hexversion < 0x03010000:
> # If support for Python 3.0.x is desired (note: result can
> # be different in this case than under 2.x or 3.1+):
> pathbytes = repo.path.encode(sys.getfilesystemencoding(),
> 'backslashreplace')
> else
> pathbytes = repo.path.encode(sys.getfilesystemencoding(),
> 'surrogateescape')
> hasher.update(pathbytes)
> repo.hash = hasher.hexdigest()
If we don't want to put a version check in it probably wants to look
like this (ignoring Python 3.0 since I don't think we need to support
it):
hasher = _digest()
try:
codecs.lookup_error('surrogateescape')
except LookupError:
pathbytes = repo.path
else:
pathbytes = repo.path.encode(sys.getfilesystemencoding(),
'surrogateescape')
hasher.update(pathbytes)
repo.hash = hasher.hexdigest()
The version with a version check seems better to me, although this
should probably be a utility function.
John
^ permalink raw reply
* Re: [PATCH v3 6/8] git-remote-testpy: hash bytes explicitly
From: John Keeping @ 2013-01-27 14:21 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Michael Haggerty, git, Sverre Rabbelier
In-Reply-To: <7vy5ffxkfb.fsf@alter.siamese.dyndns.org>
On Sat, Jan 26, 2013 at 09:30:00PM -0800, Junio C Hamano wrote:
> Michael Haggerty <mhagger@alum.mit.edu> writes:
>
> > This will still fail under Python 2.x if repo.path is a byte string that
> > contains non-ASCII characters. And it will fail under Python 3.1 and
> > later if repo.path contains characters using the surrogateescape
> > encoding option [1],...
> > Here you don't really need byte-for-byte correctness; it would be enough
> > to get *some* byte string that is unique for a given input ...
>
> Yeek.
>
> As we do not care about the actual value at all, how about doing
> something like this instead?
>
> + hasher.update(".".join([str(ord(c)) for c in repo.path]))
This doesn't solve the original problem since we're still ending up with
a Unicode string. If we wanted something like this it would need to be:
hasher.update(b'.'.join([b'%X' % ord(c) for c in repo.path]))
which limits us to Python 2.6 and later and seems to me to be less clear
than introducing an "encode_filepath" helper function using Michael's
suggestion.
John
^ permalink raw reply
* Re: [PATCH 0/7] guilt patches, including git 1.8 support
From: Josef 'Jeff' Sipek @ 2013-01-27 14:38 UTC (permalink / raw)
To: Jonathan Nieder
Cc: git, Per Cederqvist, Theodore Ts'o, Iulian Udrea,
Axel Beckert
In-Reply-To: <20130116022606.GI12524@google.com>
On Tue, Jan 15, 2013 at 06:26:06PM -0800, Jonathan Nieder wrote:
> Hi Jeff and other guilty parties,
>
> I collected all the guilt patches I could find on-list and added one
> of my own. Completely untested, except for running the regression
> tests. These are also available via git protocol from
>
> git://repo.or.cz/guilt/mob.git mob
>
> Thoughts?
Sorry for taking so long. I finally reclaimed access to
git://repo.or.cz/guilt.git. I pulled from mob, and merged it with what I
had locally.
http://repo.or.cz/w/guilt.git
Thanks for collecting all these in one place!
Jeff.
> Jonathan Nieder (1):
> Drop unneeded git version check.
>
> Per Cederqvist (6):
> get rid of "cat: write error: Broken pipe" error message
> The tests should not fail if log.date or log.decorate are set.
> Testsuite: get rid of "Broken pipe" errors from yes.
> Handle empty patches and patches with only a header.
> Fix fatal "guilt graph" error in sha1sum invocation.
> Change git branch when patches are applied.
>
> Documentation/guilt.7 | 4 +
> guilt | 73 +++++---
> guilt-branch | 12 +-
> guilt-commit | 7 +
> guilt-import-commit | 4 +-
> guilt-repair | 7 +-
> os.Darwin | 7 +-
> os.Linux | 7 +-
> os.SunOS | 7 +-
> regression/scaffold | 7 +-
> regression/t-029.sh | 6 +-
> regression/t-052.out | 24 +--
> regression/t-052.sh | 7 +-
> regression/t-061.out | 468 ++++++++++++++++++++++++++++++++++++++++++++++++++
> regression/t-061.sh | 148 ++++++++++++++++
> 15 files changed, 743 insertions(+), 45 deletions(-)
> create mode 100644 regression/t-061.out
> create mode 100755 regression/t-061.sh
--
I have always wished for my computer to be as easy to use as my telephone;
my wish has come true because I can no longer figure out how to use my
telephone.
- Bjarne Stroustrup
^ permalink raw reply
* [PATCH] git-remote-testpy: fix patch hashing on Python 3
From: John Keeping @ 2013-01-27 14:50 UTC (permalink / raw)
To: Michael Haggerty; +Cc: Junio C Hamano, git, Sverre Rabbelier
In-Reply-To: <20130127141329.GN7498@serenity.lan>
When this change was originally made (0846b0c - git-remote-testpy: hash
bytes explicitly , I didn't realised that the "hex" encoding we chose is
a "bytes to bytes" encoding so it just fails with an error on Python 3
in the same way as the original code.
It is not possible to provide a single code path that works on Python 2
and Python 3 since Python 2.x will attempt to decode the string before
encoding it, which fails for strings that are not valid in the default
encoding. Python 3.1 introduced the "surrogateescape" error handler
which handles this correctly and permits a bytes -> unicode -> bytes
round-trip to be lossless.
At this point Python 3.0 is unsupported so we don't go out of our way to
try to support it.
Helped-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: John Keeping <john@keeping.me.uk>
---
On Sun, Jan 27, 2013 at 02:13:29PM +0000, John Keeping wrote:
> On Sun, Jan 27, 2013 at 05:44:37AM +0100, Michael Haggerty wrote:
> > So to handle all of the cases across Python versions as closely as
> > possible to the old 2.x code, it might be necessary to make the code
> > explicitly depend on the Python version number, like:
> >
> > hasher = _digest()
> > if sys.hexversion < 0x03000000:
> > pathbytes = repo.path
> > elif sys.hexversion < 0x03010000:
> > # If support for Python 3.0.x is desired (note: result can
> > # be different in this case than under 2.x or 3.1+):
> > pathbytes = repo.path.encode(sys.getfilesystemencoding(),
> > 'backslashreplace')
> > else
> > pathbytes = repo.path.encode(sys.getfilesystemencoding(),
> > 'surrogateescape')
> > hasher.update(pathbytes)
> > repo.hash = hasher.hexdigest()
How about this?
git-remote-testpy.py | 18 +++++++++++++++++-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/git-remote-testpy.py b/git-remote-testpy.py
index c7a04ec..16b0c52 100644
--- a/git-remote-testpy.py
+++ b/git-remote-testpy.py
@@ -36,6 +36,22 @@ if sys.hexversion < 0x02000000:
sys.stderr.write("git-remote-testgit: requires Python 2.0 or later.\n")
sys.exit(1)
+
+def _encode_filepath(path):
+ """Encodes a Unicode file path to a byte string.
+
+ On Python 2 this is a no-op; on Python 3 we encode the string as
+ suggested by [1] which allows an exact round-trip from the command line
+ to the filesystem.
+
+ [1] http://docs.python.org/3/c-api/unicode.html#file-system-encoding
+
+ """
+ if sys.hexversion < 0x03000000:
+ return path
+ return path.encode('utf-8', 'surrogateescape')
+
+
def get_repo(alias, url):
"""Returns a git repository object initialized for usage.
"""
@@ -45,7 +61,7 @@ def get_repo(alias, url):
repo.get_head()
hasher = _digest()
- hasher.update(repo.path.encode('hex'))
+ hasher.update(_encode_filepath(repo.path))
repo.hash = hasher.hexdigest()
repo.get_base_path = lambda base: os.path.join(
--
1.8.1.1
^ permalink raw reply related
* [PATCH v2] Reduce false positive in check-non-portable-shell.pl
From: Torsten Bögershausen @ 2013-01-27 7:47 UTC (permalink / raw)
To: git; +Cc: tboegi
check-non-portable-shell.pl is using simple regular expressions to
find illegal shell syntax.
Improve the expressions and reduce the chance for false positves:
"sed -i" must be followed by 1..n whitespace and 1 non whitespace
"declare" must be followed by 1..n whitespace and 1 non whitespace
"echo -n" must be followed by 1..n whitespace and 1 non whitespace
"which": catch lines like "if which foo"
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
---
t/check-non-portable-shell.pl | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
index 8b5a71d..d9ddcdf 100755
--- a/t/check-non-portable-shell.pl
+++ b/t/check-non-portable-shell.pl
@@ -16,10 +16,10 @@ sub err {
while (<>) {
chomp;
- /^\s*sed\s+-i/ and err 'sed -i is not portable';
- /^\s*echo\s+-n/ and err 'echo -n is not portable (please use printf)';
- /^\s*declare\s+/ and err 'arrays/declare not portable';
- /^\s*[^#]\s*which\s/ and err 'which is not portable (please use type)';
+ /^\s*sed\s+-i\s+\S/ and err 'sed -i is not portable';
+ /^\s*echo\s+-n\s+\S/ and err 'echo -n is not portable (please use printf)';
+ /^\s*declare\s+\S/ and err 'arrays/declare not portable';
+ /^\s*if\s+which\s+\S/ and err 'which is not portable (please use type)';
/test\s+[^=]*==/ and err '"test a == b" is not portable (please use =)';
# this resets our $. for each file
close ARGV if eof;
--
1.8.1.1
^ permalink raw reply related
* [RFC] test-lib.sh: No POSIXPERM for cygwin
From: Torsten Bögershausen @ 2013-01-27 14:57 UTC (permalink / raw)
To: ramsay, git; +Cc: j6t, tboegi
t0070 and t1301 fail when running the test suite under cygwin.
Skip the failing tests by unsetting POSIXPERM.
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
---
t/test-lib.sh | 1 -
1 file changed, 1 deletion(-)
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 1a6c4ab..94b097e 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -669,7 +669,6 @@ case $(uname -s) in
test_set_prereq SED_STRIPS_CR
;;
*CYGWIN*)
- test_set_prereq POSIXPERM
test_set_prereq EXECKEEPSPID
test_set_prereq NOT_MINGW
test_set_prereq SED_STRIPS_CR
--
1.8.1.1
^ permalink raw reply related
* Re: [PATCH v2] add: warn when -u or -A is used without filepattern
From: Matthieu Moy @ 2013-01-27 16:10 UTC (permalink / raw)
To: Junio C Hamano
Cc: git, Jonathan Nieder, Robin Rosenberg, Piotr Krukowiecki,
Eric James Michael Ritz, Tomas Carnecky
In-Reply-To: <7v8v7h3vx8.fsf@alter.siamese.dyndns.org>
Junio C Hamano <gitster@pobox.com> writes:
> Matthieu Moy <Matthieu.Moy@imag.fr> writes:
>
>> Most git commands that can be used with our without a filepattern are
>> tree-wide by default, the filepattern being used to restrict their scope.
>> A few exceptions are: 'git grep', 'git clean', 'git add -u' and 'git add -A'.
>>
>> The inconsistancy of 'git add -u' and 'git add -A' are particularly
>
> s/consistan/consisten/;
Thanks, will fix.
> I wonder if we want to say in the message
>
> The behaviour of 'git add --all (or -A)'...
>
> otherwise people who typed "git add -A" and got this message with
> just "--all" may go "Huh?" for a brief moment. I however do not
> think replacing these strings to
>
> option_with_implicit_dot = "--all (-A)";
>
> is a solution, given they are goven to _("l10n template %s").
Plus, option_with_implicit_dot is used in cut-and-paste ready commands
below. I can easily add another variable short_option or so to display
both. Ideally, we should use whatever the user had typed, but that does
not seem easy to do with parse-option so I'd say it's overkill.
--
Matthieu Moy
http://www-verimag.imag.fr/~moy/
^ permalink raw reply
* mergetool: include custom tools in '--tool-help'
From: John Keeping @ 2013-01-27 16:34 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, David Aguilar
'git mergetool --tool-help' only lists builtin tools, not those that the
user has configured via a 'mergetool.<tool>.cmd' config value. Fix this
by inspecting the tools configured in this way and adding them to the
available and unavailable lists before displaying them.
Signed-off-by: John Keeping <john@keeping.me.uk>
---
After the recent changes to mergetool, do we want to do something like
this as well, so that 'git mergetool --tool-help' will display any tools
configured by the user/system administrator?
This is on top of jk/mergetool.
git-mergetool--lib.sh | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/git-mergetool--lib.sh b/git-mergetool--lib.sh
index 1d0fb12..f9a617c 100644
--- a/git-mergetool--lib.sh
+++ b/git-mergetool--lib.sh
@@ -206,6 +206,29 @@ list_merge_tool_candidates () {
esac
}
+# Adds tools from git-config to the available and unavailable lists.
+# The tools are found in "$1.<tool>.cmd".
+add_config_tools() {
+ section=$1
+
+ eval $(git config --get-regexp $section'\..*\.cmd' |
+ while read -r key value
+ do
+ tool=${key#mergetool.}
+ tool=${tool%.cmd}
+
+ tool=$(echo "$tool" |sed -e 's/'\''/'\''\\'\'\''/g')
+
+ cmd=$(eval -- "set -- $value"; echo "$1")
+ if type "$cmd" >/dev/null 2>&1
+ then
+ echo "available=\"\${available}\"'$tool'\"\$LF\""
+ else
+ echo "unavailable=\"\${unavailable}\"'$tool'\"\$LF\""
+ fi
+ done)
+}
+
show_tool_help () {
unavailable= available= LF='
'
@@ -223,6 +246,12 @@ show_tool_help () {
fi
done
+ add_config_tools mergetool
+ if diff_mode
+ then
+ add_config_tools difftool
+ fi
+
cmd_name=${TOOL_MODE}tool
if test -n "$available"
then
--
1.8.1.1
^ permalink raw reply related
* Re: [PATCH] tests: turn on test-lint-shell-syntax by default
From: Junio C Hamano @ 2013-01-27 17:15 UTC (permalink / raw)
To: Jonathan Nieder; +Cc: Torsten Bögershausen, git, kraai
In-Reply-To: <20130127093121.GA4228@elie.Belkin>
Jonathan Nieder <jrnieder@gmail.com> writes:
> ...
> With the above definition of "which", the only sign of a mistake would
> be some extra output to stderr (which is quelled when running tests in
> the normal way). The "exit" is caught by the subshell and just makes
> the "if" condition false.
>
> That's not so terrible --- it could still dissuade new test authors
> from using "which". The downside I'd worry about is that it provides
> a false sense of security despite not catching problems ...
> ...
> In the end the analysis that works best would probably involve a
> full-fledged shell script parser. Something like "sparse", except for
> shell command language.
Exactly.
That is why I keep saying that whole test-lint-shell-syntax should
stay outside the default until it gets more robust by becoming a
reasonable shell parser; it may not necessarily have to become
"full" parser though.
As we discourage the use of tricky features of the language like
eval in individual test scripts to implement their own mini test
framework, the "something like sparse" parser may initialy start
small and still be useful; for example it can learn to exclude
anything inside <<HERE_DOCUMENT from getting inspected.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox