* Re: [PATCH v3 1/6] submodules: add helper functions to determine presence of submodules
From: Stefan Beller @ 2016-11-15 23:49 UTC (permalink / raw)
To: Brandon Williams; +Cc: git@vger.kernel.org, Jonathan Tan, Junio C Hamano
In-Reply-To: <1478908273-190166-2-git-send-email-bmwill@google.com>
On Fri, Nov 11, 2016 at 3:51 PM, Brandon Williams <bmwill@google.com> wrote:
> Add two helper functions to submodules.c.
> `is_submodule_initialized()` checks if a submodule has been initialized
> at a given path and `is_submodule_populated()` check if a submodule
> has been checked out at a given path.
This reminds me to write the documentation patch explaining the
concepts of submodules (specifically that overview page would state
all the possible states of submodules)
This patch looks good,
Stefan
> +
> + if (module) {
> + char *key = xstrfmt("submodule.%s.url", module->name);
> + char *value = NULL;
minor nit:
In case a reroll is needed, you could replace `value` by
`not_needed` or `unused` to make it easier to follow.
Hence it also doesn't need initialization (Doh, it does
for free() to work, nevermind).
^ permalink raw reply
* Re: [PATCH 09/16] update submodules: add scheduling to update submodules
From: Brandon Williams @ 2016-11-16 0:02 UTC (permalink / raw)
To: Stefan Beller; +Cc: git, gitster, jrnieder, mogulguy10, David.Turner
In-Reply-To: <20161115230651.23953-10-sbeller@google.com>
On 11/15, Stefan Beller wrote:
> +static int update_submodule(const char *path, const struct object_id *oid,
> + int force, int is_new)
> +{
> + const char *git_dir;
> + struct child_process cp = CHILD_PROCESS_INIT;
> + const struct submodule *sub = submodule_from_path(null_sha1, path);
> +
> + if (!sub || !sub->name)
> + return -1;
> +
> + git_dir = resolve_gitdir(git_common_path("modules/%s", sub->name));
> +
> + if (!git_dir)
> + return -1;
> +
> + if (is_new)
> + connect_work_tree_and_git_dir(path, git_dir);
> +
> + /* update index via `read-tree --reset sha1` */
> + argv_array_pushl(&cp.args, "read-tree",
> + force ? "--reset" : "-m",
> + "-u", sha1_to_hex(oid->hash), NULL);
> + prepare_submodule_repo_env(&cp.env_array);
> + cp.git_cmd = 1;
> + cp.no_stdin = 1;
> + cp.dir = path;
> + if (run_command(&cp)) {
> + warning(_("reading the index in submodule '%s' failed"), path);
> + child_process_clear(&cp);
> + return -1;
> + }
> +
> + /* write index to working dir */
> + child_process_clear(&cp);
> + child_process_init(&cp);
> + argv_array_pushl(&cp.args, "checkout-index", "-a", NULL);
> + cp.git_cmd = 1;
> + cp.no_stdin = 1;
> + cp.dir = path;
> + if (force)
> + argv_array_push(&cp.args, "-f");
> +
> + if (run_command(&cp)) {
> + warning(_("populating the working directory in submodule '%s' failed"), path);
> + child_process_clear(&cp);
> + return -1;
> + }
> +
> + /* get the HEAD right */
> + child_process_clear(&cp);
> + child_process_init(&cp);
> + argv_array_pushl(&cp.args, "checkout", "--recurse-submodules", NULL);
> + cp.git_cmd = 1;
> + cp.no_stdin = 1;
> + cp.dir = path;
> + if (force)
> + argv_array_push(&cp.args, "-f");
> + argv_array_push(&cp.args, sha1_to_hex(oid->hash));
> +
> + if (run_command(&cp)) {
> + warning(_("setting the HEAD in submodule '%s' failed"), path);
> + child_process_clear(&cp);
> + return -1;
> + }
> +
> + child_process_clear(&cp);
> + return 0;
> +}
If run command is successful then it handles the clearing of the child
process struct, correct? Is there a negative to having all the explicit
clears when the child was successful?
> +
> int depopulate_submodule(const char *path)
> {
> int ret = 0;
> @@ -1336,3 +1405,51 @@ void prepare_submodule_repo_env(struct argv_array *out)
> }
> argv_array_push(out, "GIT_DIR=.git");
> }
> +
> +struct scheduled_submodules_update_type {
> + const char *path;
> + const struct object_id *oid;
> + /*
> + * Do we need to perform a complete checkout or just incremental
> + * update?
> + */
> + unsigned is_new:1;
> +} *scheduled_submodules;
> +#define SCHEDULED_SUBMODULES_INIT {NULL, NULL}
I may not know enough about these types of initializors but that Init
macro only has 2 entries while there are three entries in the struct
itself.
> +
> +int scheduled_submodules_nr, scheduled_submodules_alloc;
Should these globals be static since they should be scoped to only this
file?
> +
> +void schedule_submodule_for_update(const struct cache_entry *ce, int is_new)
> +{
> + struct scheduled_submodules_update_type *ssu;
> + ALLOC_GROW(scheduled_submodules,
> + scheduled_submodules_nr + 1,
> + scheduled_submodules_alloc);
> + ssu = &scheduled_submodules[scheduled_submodules_nr++];
> + ssu->path = ce->name;
> + ssu->oid = &ce->oid;
> + ssu->is_new = !!is_new;
> +}
> +
> +int update_submodules(int force)
> +{
> + int i;
> + gitmodules_config();
> +
> + /*
> + * NEEDSWORK: As submodule updates can potentially take some
> + * time each and they do not overlap (i.e. no d/f conflicts;
> + * this can be parallelized using the run_commands.h API.
> + */
> + for (i = 0; i < scheduled_submodules_nr; i++) {
> + struct scheduled_submodules_update_type *ssu =
> + &scheduled_submodules[i];
> +
> + if (submodule_is_interesting(ssu->path, null_sha1))
> + update_submodule(ssu->path, ssu->oid,
> + force, ssu->is_new);
> + }
> +
> + scheduled_submodules_nr = 0;
> + return 0;
> +}
nit: organization wise it makes more sense to me to have the
'update_submodule' helper function be located more closely to the
'update_submodules' function.
--
Brandon Williams
^ permalink raw reply
* Re: gitweb html validation
From: Junio C Hamano @ 2016-11-16 0:05 UTC (permalink / raw)
To: Ralf Thielow, Jakub Narębski; +Cc: Raphaël Gertz, git
In-Reply-To: <20161115182632.GA17539@gmail.com>
Ralf Thielow <ralf.thielow@gmail.com> writes:
> Only block level elements are
> allowed to be inside form tags, according to
> https://www.w3.org/2010/04/xhtml10-strict.html#elem_form
> ...
> I think it's better to just move the <form>-Tag outside of the
> surrounding div?
> Something like this perhaps, I didn't test it myself yet.
That sounds like a sensible update to me (no, I do not run gitweb
myself). Is this the only <form> we have in the UI, or is it the
only one that is problematic?
> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
> index 7cf68f07b..33d7c154f 100755
> --- a/gitweb/gitweb.perl
> +++ b/gitweb/gitweb.perl
> @@ -5531,8 +5531,8 @@ sub git_project_search_form {
> $limit = " in '$project_filter/'";
> }
>
> - print "<div class=\"projsearch\">\n";
> print $cgi->start_form(-method => 'get', -action => $my_uri) .
> + "<div class=\"projsearch\">\n" .
> $cgi->hidden(-name => 'a', -value => 'project_list') . "\n";
> print $cgi->hidden(-name => 'pf', -value => $project_filter). "\n"
> if (defined $project_filter);
> @@ -5544,11 +5544,11 @@ sub git_project_search_form {
> -checked => $search_use_regexp) .
> "</span>\n" .
> $cgi->submit(-name => 'btnS', -value => 'Search') .
> - $cgi->end_form() . "\n" .
> $cgi->a({-href => href(project => undef, searchtext => undef,
> project_filter => $project_filter)},
> esc_html("List all projects$limit")) . "<br />\n";
> - print "</div>\n";
> + print "</div>\n" .
> + $cgi->end_form() . "\n";
> }
>
> # entry for given @keys needs filling if at least one of keys in list
> diff --git a/gitweb/static/gitweb.css b/gitweb/static/gitweb.css
> index 321260103..507740b6a 100644
> --- a/gitweb/static/gitweb.css
> +++ b/gitweb/static/gitweb.css
> @@ -539,7 +539,7 @@ div.projsearch {
> margin: 20px 0px;
> }
>
> -div.projsearch form {
> +form div.projsearch {
> margin-bottom: 2px;
> }
>
^ permalink raw reply
* Re: [PATCH 10/16] update submodules: is_submodule_checkout_safe
From: Brandon Williams @ 2016-11-16 0:06 UTC (permalink / raw)
To: Stefan Beller; +Cc: git, gitster, jrnieder, mogulguy10, David.Turner
In-Reply-To: <20161115230651.23953-11-sbeller@google.com>
On 11/15, Stefan Beller wrote:
> In later patches we introduce the options and flag for commands
> that modify the working directory, e.g. git-checkout.
>
> This piece of code will answer the question:
> "Is it safe to change the submodule to this new state?"
> e.g. is it overwriting untracked files or are there local
> changes that would be overwritten?
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
> submodule.c | 22 ++++++++++++++++++++++
> submodule.h | 2 ++
> 2 files changed, 24 insertions(+)
>
> diff --git a/submodule.c b/submodule.c
> index 2773151..2149ef7 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -1201,6 +1201,28 @@ int ok_to_remove_submodule(const char *path)
> return ok_to_remove;
> }
>
> +int is_submodule_checkout_safe(const char *path, const struct object_id *oid)
> +{
> + struct child_process cp = CHILD_PROCESS_INIT;
> +
> + if (!is_submodule_populated(path))
> + /* The submodule is not populated, it's safe to check it out */
> + /*
> + * TODO: When git learns to re-populate submodules, a check must be
> + * added here to assert that no local files will be overwritten.
> + */
When you mean local files do you mean in the situation where we want to
checkout a submodule at path 'foo' but there already exists a file at
path 'foo'?
> + return 1;
> +
> + argv_array_pushl(&cp.args, "read-tree", "-n", "-m", "HEAD",
> + sha1_to_hex(oid->hash), NULL);
> +
> + prepare_submodule_repo_env(&cp.env_array);
> + cp.git_cmd = 1;
> + cp.no_stdin = 1;
> + cp.dir = path;
> + return run_command(&cp) == 0;
> +}
> +
> static int find_first_merges(struct object_array *result, const char *path,
> struct commit *a, struct commit *b)
> {
> diff --git a/submodule.h b/submodule.h
> index f01f87c..a7fa634 100644
> --- a/submodule.h
> +++ b/submodule.h
> @@ -74,6 +74,8 @@ extern unsigned is_submodule_modified(const char *path, int ignore_untracked);
> extern int is_submodule_populated(const char *path);
> extern int submodule_uses_gitfile(const char *path);
> extern int ok_to_remove_submodule(const char *path);
> +extern int is_submodule_checkout_safe(const char *path,
> + const struct object_id *oid);
> extern int merge_submodule(unsigned char result[20], const char *path,
> const unsigned char base[20],
> const unsigned char a[20],
> --
> 2.10.1.469.g00a8914
>
--
Brandon Williams
^ permalink raw reply
* Re: [PATCH v3 4/6] grep: optionally recurse into submodules
From: Stefan Beller @ 2016-11-16 0:07 UTC (permalink / raw)
To: Brandon Williams; +Cc: git@vger.kernel.org, Jonathan Tan, Junio C Hamano
In-Reply-To: <1478908273-190166-5-git-send-email-bmwill@google.com>
On Fri, Nov 11, 2016 at 3:51 PM, Brandon Williams <bmwill@google.com> wrote:
> Allow grep to recognize submodules and recursively search for patterns in
> each submodule. This is done by forking off a process to recursively
> call grep on each submodule. The top level --super-prefix option is
> used to pass a path to the submodule which can in turn be used to
> prepend to output or in pathspec matching logic.
>
> Recursion only occurs for submodules which have been initialized and
> checked out by the parent project. If a submodule hasn't been
> initialized and checked out it is simply skipped.
>
> In order to support the existing multi-threading infrastructure in grep,
> output from each child process is captured in a strbuf so that it can be
> later printed to the console in an ordered fashion.
>
> To limit the number of theads that are created, each child process has
> half the number of threads as its parents (minimum of 1), otherwise we
> potentailly have a fork-bomb.
>
> Signed-off-by: Brandon Williams <bmwill@google.com>
> ---
> Documentation/git-grep.txt | 5 +
> builtin/grep.c | 300 ++++++++++++++++++++++++++++++++++---
> git.c | 2 +-
> t/t7814-grep-recurse-submodules.sh | 99 ++++++++++++
> 4 files changed, 385 insertions(+), 21 deletions(-)
> create mode 100755 t/t7814-grep-recurse-submodules.sh
>
> diff --git a/Documentation/git-grep.txt b/Documentation/git-grep.txt
> index 0ecea6e..17aa1ba 100644
> --- a/Documentation/git-grep.txt
> +++ b/Documentation/git-grep.txt
> @@ -26,6 +26,7 @@ SYNOPSIS
> [--threads <num>]
> [-f <file>] [-e] <pattern>
> [--and|--or|--not|(|)|-e <pattern>...]
> + [--recurse-submodules]
> [ [--[no-]exclude-standard] [--cached | --no-index | --untracked] | <tree>...]
> [--] [<pathspec>...]
>
> @@ -88,6 +89,10 @@ OPTIONS
> mechanism. Only useful when searching files in the current directory
> with `--no-index`.
>
> +--recurse-submodules::
> + Recursively search in each submodule that has been initialized and
> + checked out in the repository.
> +
> -a::
> --text::
> Process binary files as if they were text.
> diff --git a/builtin/grep.c b/builtin/grep.c
> index 8887b6a..1fd292f 100644
> --- a/builtin/grep.c
> +++ b/builtin/grep.c
> @@ -18,12 +18,20 @@
> #include "quote.h"
> #include "dir.h"
> #include "pathspec.h"
> +#include "submodule.h"
>
> static char const * const grep_usage[] = {
> N_("git grep [<options>] [-e] <pattern> [<rev>...] [[--] <path>...]"),
> NULL
> };
>
> +static const char *super_prefix;
> +static int recurse_submodules;
> +static struct argv_array submodule_options = ARGV_ARRAY_INIT;
> +
> +static int grep_submodule_launch(struct grep_opt *opt,
> + const struct grep_source *gs);
> +
> #define GREP_NUM_THREADS_DEFAULT 8
> static int num_threads;
>
> @@ -174,7 +182,10 @@ static void *run(void *arg)
> break;
>
> opt->output_priv = w;
> - hit |= grep_source(opt, &w->source);
> + if (w->source.type == GREP_SOURCE_SUBMODULE)
> + hit |= grep_submodule_launch(opt, &w->source);
> + else
> + hit |= grep_source(opt, &w->source);
> grep_source_clear_data(&w->source);
> work_done(w);
> }
> @@ -300,6 +311,10 @@ static int grep_sha1(struct grep_opt *opt, const unsigned char *sha1,
> if (opt->relative && opt->prefix_length) {
> quote_path_relative(filename + tree_name_len, opt->prefix, &pathbuf);
> strbuf_insert(&pathbuf, 0, filename, tree_name_len);
> + } else if (super_prefix) {
> + strbuf_add(&pathbuf, filename, tree_name_len);
> + strbuf_addstr(&pathbuf, super_prefix);
> + strbuf_addstr(&pathbuf, filename + tree_name_len);
> } else {
> strbuf_addstr(&pathbuf, filename);
> }
> @@ -328,10 +343,13 @@ static int grep_file(struct grep_opt *opt, const char *filename)
> {
> struct strbuf buf = STRBUF_INIT;
>
> - if (opt->relative && opt->prefix_length)
> + if (opt->relative && opt->prefix_length) {
> quote_path_relative(filename, opt->prefix, &buf);
> - else
> + } else {
> + if (super_prefix)
> + strbuf_addstr(&buf, super_prefix);
> strbuf_addstr(&buf, filename);
> + }
>
> #ifndef NO_PTHREADS
> if (num_threads) {
> @@ -378,31 +396,258 @@ static void run_pager(struct grep_opt *opt, const char *prefix)
> exit(status);
> }
>
> -static int grep_cache(struct grep_opt *opt, const struct pathspec *pathspec, int cached)
> +static void compile_submodule_options(const struct grep_opt *opt,
> + const struct pathspec *pathspec,
> + int cached, int untracked,
> + int opt_exclude, int use_index,
> + int pattern_type_arg)
> +{
> + struct grep_pat *pattern;
> + int i;
> +
> + if (recurse_submodules)
> + argv_array_push(&submodule_options, "--recurse-submodules");
> +
> + if (cached)
> + argv_array_push(&submodule_options, "--cached");
> + if (!use_index)
> + argv_array_push(&submodule_options, "--no-index");
> + if (untracked)
> + argv_array_push(&submodule_options, "--untracked");
> + if (opt_exclude > 0)
> + argv_array_push(&submodule_options, "--exclude-standard");
> +
> + if (opt->invert)
> + argv_array_push(&submodule_options, "-v");
> + if (opt->ignore_case)
> + argv_array_push(&submodule_options, "-i");
> + if (opt->word_regexp)
> + argv_array_push(&submodule_options, "-w");
> + switch (opt->binary) {
> + case GREP_BINARY_NOMATCH:
> + argv_array_push(&submodule_options, "-I");
> + break;
> + case GREP_BINARY_TEXT:
> + argv_array_push(&submodule_options, "-a");
> + break;
> + default:
> + break;
> + }
> + if (opt->allow_textconv)
> + argv_array_push(&submodule_options, "--textconv");
> + if (opt->max_depth != -1)
> + argv_array_pushf(&submodule_options, "--max-depth=%d",
> + opt->max_depth);
> + if (opt->linenum)
> + argv_array_push(&submodule_options, "-n");
> + if (!opt->pathname)
> + argv_array_push(&submodule_options, "-h");
> + if (!opt->relative)
> + argv_array_push(&submodule_options, "--full-name");
> + if (opt->name_only)
> + argv_array_push(&submodule_options, "-l");
> + if (opt->unmatch_name_only)
> + argv_array_push(&submodule_options, "-L");
> + if (opt->null_following_name)
> + argv_array_push(&submodule_options, "-z");
> + if (opt->count)
> + argv_array_push(&submodule_options, "-c");
> + if (opt->file_break)
> + argv_array_push(&submodule_options, "--break");
> + if (opt->heading)
> + argv_array_push(&submodule_options, "--heading");
> + if (opt->pre_context)
> + argv_array_pushf(&submodule_options, "--before-context=%d",
> + opt->pre_context);
> + if (opt->post_context)
> + argv_array_pushf(&submodule_options, "--after-context=%d",
> + opt->post_context);
> + if (opt->funcname)
> + argv_array_push(&submodule_options, "-p");
> + if (opt->funcbody)
> + argv_array_push(&submodule_options, "-W");
> + if (opt->all_match)
> + argv_array_push(&submodule_options, "--all-match");
> + if (opt->debug)
> + argv_array_push(&submodule_options, "--debug");
> + if (opt->status_only)
> + argv_array_push(&submodule_options, "-q");
> +
> + switch (pattern_type_arg) {
> + case GREP_PATTERN_TYPE_BRE:
> + argv_array_push(&submodule_options, "-G");
> + break;
> + case GREP_PATTERN_TYPE_ERE:
> + argv_array_push(&submodule_options, "-E");
> + break;
> + case GREP_PATTERN_TYPE_FIXED:
> + argv_array_push(&submodule_options, "-F");
> + break;
> + case GREP_PATTERN_TYPE_PCRE:
> + argv_array_push(&submodule_options, "-P");
> + break;
> + case GREP_PATTERN_TYPE_UNSPECIFIED:
> + break;
> + }
> +
> + for (pattern = opt->pattern_list; pattern != NULL;
> + pattern = pattern->next) {
> + switch (pattern->token) {
> + case GREP_PATTERN:
> + argv_array_pushf(&submodule_options, "-e%s",
> + pattern->pattern);
> + break;
> + case GREP_AND:
> + case GREP_OPEN_PAREN:
> + case GREP_CLOSE_PAREN:
> + case GREP_NOT:
> + case GREP_OR:
> + argv_array_push(&submodule_options, pattern->pattern);
> + break;
> + /* BODY and HEAD are not used by git-grep */
> + case GREP_PATTERN_BODY:
> + case GREP_PATTERN_HEAD:
> + break;
> + }
> + }
> +
> + /*
> + * Limit number of threads for child process to use.
> + * This is to prevent potential fork-bomb behavior of git-grep as each
> + * submodule process has its own thread pool.
> + */
> + if (num_threads)
> + argv_array_pushf(&submodule_options, "--threads=%d",
> + (num_threads + 1) / 2);
I think you would want to pass --threads=%d unconditionally,
as it also serves as a weak defusal for fork bombs. Is it possible to come here
with num_threads=0? (i.e. what happens if the user doesn't specify the number
of threads or such, do we fall back to some default or is it just 0?)
I have seen some other places that check for num_threads unequal to 0,
as e.g. no mutex needs to be locked then (assuming we don't have any
thread but grep within the main process), but as you intend to use this also
as a helper to not blow up the number of threads recursively, we'd need to
pass at a number != 0 here?
> +
> + git grep -e "bar" --and -e "foo" --recurse-submodules > actual &&
nit here and in the tests below:
We prefer to have no white space between > and the file piped to.
> + test_cmp expect actual
> +'
> +
> +test_expect_success 'grep and multiple patterns' '
> + cat >expect <<-\EOF &&
> + b/b:bar
> + EOF
> +
> + git grep -e "bar" --and --not -e "foo" --recurse-submodules > actual &&
Otherwise, this patch looks good.
Thanks,
Stefan
^ permalink raw reply
* RE: [PATCH 09/16] update submodules: add scheduling to update submodules
From: David Turner @ 2016-11-16 0:07 UTC (permalink / raw)
To: 'Brandon Williams', Stefan Beller
Cc: git@vger.kernel.org, gitster@pobox.com, jrnieder@gmail.com,
mogulguy10@gmail.com
In-Reply-To: <20161116000236.GF66382@google.com>
> -----Original Message-----
> From: Brandon Williams [mailto:bmwill@google.com]
> > +struct scheduled_submodules_update_type {
> > + const char *path;
> > + const struct object_id *oid;
> > + /*
> > + * Do we need to perform a complete checkout or just incremental
> > + * update?
> > + */
> > + unsigned is_new:1;
> > +} *scheduled_submodules;
> > +#define SCHEDULED_SUBMODULES_INIT {NULL, NULL}
>
> I may not know enough about these types of initializors but that Init
> macro only has 2 entries while there are three entries in the struct
> itself.
In fact, only the first NULL is necessary; unspecified initializer entries in C default to zero.
^ permalink raw reply
* Re: [PATCH v3 4/4] submodule_needs_pushing() NEEDSWORK when we can not answer this question
From: Junio C Hamano @ 2016-11-16 0:13 UTC (permalink / raw)
To: Stefan Beller
Cc: Heiko Voigt, git@vger.kernel.org, Jeff King, Jens Lehmann,
Fredrik Gustafsson, Leandro Lucarella
In-Reply-To: <CAGZ79kYyyjP7W7gWq6WomVSkhRtMbZZMKYQPFszko4_f9oprgg@mail.gmail.com>
Stefan Beller <sbeller@google.com> writes:
>> "We do not know" ...
>
> ... because there is no way to check for us as we don't have the
> submodule commits.
>
> " We do consider it safe as no one in their sane mind would
> have changed the submodule pointers without having the
> submodule around. If a user did however change the submodules
> without having the submodule commits around, this indicates an
> expert who knows what they were doing."
I didn't think it through myself to arrive at such a conclusion, but
to me the above sounds like a sensible reasoning [*1*].
>> We currently
>> + * proceed pushing here as if the submodules commits are
>> + * available on a remote. Since we can not check the
>> + * remote availability for this submodule we should
>> + * consider changing this behavior to: Stop here and
>> + * tell the user how to skip this check if wanted.
>> + */
>> return 0;
>
> Thanks for adding the NEEDSWORK, I just wrote the above lines
> to clarify my thought process, not as a suggestion for change.
One thing I would suggest would be "Stop here, explain the situation
and then tell the user how to skip". I am not convinced that it
would _help_ users and make it _safer_ if we stopped here, though.
> Overall the series looks good to me; the nits are minor IMHO.
Ditto.
[Footnote]
*1* My version was more like "we do not know if they would get into
a situation where they do not have enough submodule commits if
we pushed our superproject, but more importantly, we DO KNOW
that it would not help an iota if we pushed our submodule to
them, so there is no point stopping the push of superproject
saying 'no, no, no, you must push the submodule first'".
^ permalink raw reply
* Re: [PATCH 11/16] teach unpack_trees() to remove submodule contents
From: Brandon Williams @ 2016-11-16 0:14 UTC (permalink / raw)
To: Stefan Beller; +Cc: git, gitster, jrnieder, mogulguy10, David.Turner
In-Reply-To: <20161115230651.23953-12-sbeller@google.com>
On 11/15, Stefan Beller wrote:
> Extend rmdir_or_warn() to remove the directories of those submodules which
> are scheduled for removal. Also teach verify_clean_submodule() to check
> that a submodule configured to be removed is not modified before scheduling
> it for removal.
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
> unpack-trees.c | 6 ++----
> wrapper.c | 4 ++++
> 2 files changed, 6 insertions(+), 4 deletions(-)
>
> diff --git a/unpack-trees.c b/unpack-trees.c
> index ea6bdd2..576e1d5 100644
> --- a/unpack-trees.c
> +++ b/unpack-trees.c
> @@ -9,6 +9,7 @@
> #include "refs.h"
> #include "attr.h"
> #include "split-index.h"
> +#include "submodule.h"
> #include "dir.h"
>
> /*
> @@ -1361,15 +1362,12 @@ static void invalidate_ce_path(const struct cache_entry *ce,
> /*
> * Check that checking out ce->sha1 in subdir ce->name is not
> * going to overwrite any working files.
> - *
> - * Currently, git does not checkout subprojects during a superproject
> - * checkout, so it is not going to overwrite anything.
> */
> static int verify_clean_submodule(const struct cache_entry *ce,
> enum unpack_trees_error_types error_type,
> struct unpack_trees_options *o)
> {
> - return 0;
> + return submodule_is_interesting(ce->name, null_sha1) && is_submodule_modified(ce->name, 0);
> }
So what does the return value from this function meant to mean? Is '1'
mean the submodule is clean while '0' indicates it is dirty or is it the
reverse of that? Reading this it seems to me a value of '1' means "yes
the submodule is clean!" but the way the return value is calculated
tells a different story. Either I'm understanding it incorrectly or I
think the return should be something like this:
return submodule_is_interesting(ce->name, null_sha1) && !is_submodule_modified(ce->name, 0);
Where we return '1' if the submodule is interesting and it hasn't been
modified.
>
> static int verify_clean_subdirectory(const struct cache_entry *ce,
> diff --git a/wrapper.c b/wrapper.c
> index e7f1979..17c08de 100644
> --- a/wrapper.c
> +++ b/wrapper.c
> @@ -2,6 +2,7 @@
> * Various trivial helper wrappers around standard functions
> */
> #include "cache.h"
> +#include "submodule.h"
>
> static void do_nothing(size_t size)
> {
> @@ -592,6 +593,9 @@ int unlink_or_warn(const char *file)
>
> int rmdir_or_warn(const char *file)
> {
> + if (submodule_is_interesting(file, null_sha1)
> + && depopulate_submodule(file))
> + return -1;
> return warn_if_unremovable("rmdir", file, rmdir(file));
> }
It seems weird to me that rmdir is doing checks to see if the file being
removed is a submodule. Shouldn't those checks have occurred before
calling rmdir?
--
Brandon Williams
^ permalink raw reply
* RE: [PATCH 07/16] update submodules: introduce submodule_is_interesting
From: David Turner @ 2016-11-16 0:14 UTC (permalink / raw)
To: 'Stefan Beller'
Cc: git@vger.kernel.org, bmwill@google.com, gitster@pobox.com,
jrnieder@gmail.com, mogulguy10@gmail.com
In-Reply-To: <20161115230651.23953-8-sbeller@google.com>
> -----Original Message-----
> From: Stefan Beller [mailto:sbeller@google.com]
> Sent: Tuesday, November 15, 2016 6:07 PM
> Cc: git@vger.kernel.org; bmwill@google.com; gitster@pobox.com;
> jrnieder@gmail.com; mogulguy10@gmail.com; David Turner; Stefan Beller
> Subject: [PATCH 07/16] update submodules: introduce
> submodule_is_interesting
> +int submodule_is_interesting(const char *path, const unsigned char
> +*sha1) {
This is apparently only ever (in this series) called with null_sha1. So either this arg is unnecessary, or there are bugs elsewhere in the code.
^ permalink raw reply
* Re: [PATCH v15 01/27] bisect--helper: use OPT_CMDMODE instead of OPT_BOOL
From: Junio C Hamano @ 2016-11-16 0:18 UTC (permalink / raw)
To: Stephan Beyer
Cc: git, Pranit Bauva, Christian Couder, Matthieu Moy, Alex Henrie,
Antoine Delaite
In-Reply-To: <20ffa616-765d-ef73-4133-977561105eff@gmx.net>
Stephan Beyer <s-beyer@gmx.net> writes:
> Besides the things I'm mentioning in respective patch e-mails, I wonder
> why several bisect--helper commands are prefixed by "bisect"; I'm
> talking about:
>
> git bisect--helper --bisect-clean-state
>...
> git bisect--helper --bisect-start
> etc.
>
> instead of
>
> git bisect--helper --clean-state
>...
> git bisect--helper --start
> etc.
>
> Well, I know *why* they have these names: because the shell function
> names are simply reused. But I don't know why these prefixes are kept in
> the bisect--helper command options. On the other hand, these command
> names are not exposed to the user and may hence not be that important.(?)
That's a good point ;-)
These are not intended to be end-user entry points, so names that
are bit longer than necessary does not bother me too much.
Hopefully the longer-term endgame would be not to need a separate
"bisect-helper" binary at all but to have a "git bisect" binary
making these requests as subroutine calls, and at that point, the
names of the functions would want to have "bisect" prefix.
^ permalink raw reply
* Bug with disabling compression and 'binary' files.
From: Douglas Cox @ 2016-11-16 0:21 UTC (permalink / raw)
To: git
I was doing some experiments today with large-ish (100-200MB) binary
files and was trying to determine the best configuration for Git. Here
are the steps and timings I saw:
git init Test
cp .../largemovie.mp4 .
time git add largemovie.mp4
This took 6.5s for a 200MB file.
This file compressed a little bit, but not enough to matter, so I
wanted to see how much faster the add would be with compression
disabled. So I ran:
git config core.compression = 0
I then completely reran the test above starting with a clean
repository. This time the add took only 2.08s. I repeated these two
tests about 10 times using the same file each time to verify it wasn't
related to disk caching, etc.
At this point I decided that this was likely a good setting for this
repository, so I also created a .gitattributes file and added a few
entries often seen in suggestions for dealing with binary files:
*.mp4 binary -delta
The goal here was to disable any delta compression done during a
gc/pack and use the other settings 'binary' applies. Unfortunately
when I ran the test again (still using compression = 0), the test was
back to taking 6.5s and the file inside the .git/objects/ folder was
compressed again.
I narrowed this down to the '-text' attribute that is set when
specifying 'binary'. For some reason this flag is cancelling out the
core.compression = 0 setting and I think this is a bug?
Unfortunately core.compression = 0 is also global. Ideally it would be
great if there was a separate 'compression' attribute that could be
specified in .gitattributes per wildcard similar to how -delta can be
used. This way we would still be able to get compression for
text/source files, while still getting the speed of skipping
compression for binary files that do not compress well.
Has there been any discussion on having an attribute similar to this?
Thanks,
-Doug
^ permalink raw reply
* RE: [PATCH 13/16] submodule: teach unpack_trees() to update submodules
From: David Turner @ 2016-11-16 0:22 UTC (permalink / raw)
To: 'Stefan Beller'
Cc: git@vger.kernel.org, bmwill@google.com, gitster@pobox.com,
jrnieder@gmail.com, mogulguy10@gmail.com
In-Reply-To: <20161115230651.23953-14-sbeller@google.com>
[I've reviewed up-to and including 13; I'll look at 14-16 tomorrow-ish]
> -----Original Message-----
> From: Stefan Beller [mailto:sbeller@google.com]
> Sent: Tuesday, November 15, 2016 6:07 PM
> Cc: git@vger.kernel.org; bmwill@google.com; gitster@pobox.com;
> jrnieder@gmail.com; mogulguy10@gmail.com; David Turner; Stefan Beller
> Subject: [PATCH 13/16] submodule: teach unpack_trees() to update
> submodules
...
> msgs[ERROR_NOT_UPTODATE_DIR] =
> _("Updating the following directories would lose untracked
> files in it:\n%s");
> + msgs[ERROR_NOT_UPTODATE_SUBMODULE] =
> + _("Updating the following submodules would lose modifications
> in
> +it:\n%s");
s/it/them/
> if (!strcmp(cmd, "checkout"))
> msg = advice_commit_before_merge
> @@ -1315,19 +1320,18 @@ static int verify_uptodate_1(const struct
> cache_entry *ce,
> return 0;
>
> if (!lstat(ce->name, &st)) {
> - int flags =
> CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE;
> - unsigned changed = ie_match_stat(o->src_index, ce, &st,
> flags);
> - if (!changed)
> - return 0;
> - /*
> - * NEEDSWORK: the current default policy is to allow
> - * submodule to be out of sync wrt the superproject
> - * index. This needs to be tightened later for
> - * submodules that are marked to be automatically
> - * checked out.
> - */
> - if (S_ISGITLINK(ce->ce_mode))
> - return 0;
> + if (!S_ISGITLINK(ce->ce_mode)) {
I generally prefer to avoid if (!x) { A } else { B } -- I would rather just see if (x) { B } else { A }.
> + if (!changed) {
> + /* old is always a submodule */
> + if (S_ISGITLINK(new->ce_mode)) {
> + /*
> + * new is also a submodule, so check if we care
> + * and then if can checkout the new sha1 safely
> + */
> + if (submodule_is_interesting(old->name, null_sha1)
> + && is_submodule_checkout_safe(new->name, &new-
> >oid))
> + return 0;
> + } else {
> + /*
> + * new is not a submodule any more, so only
> + * care if we care:
> + */
> + if (submodule_is_interesting(old->name, null_sha1)
> + && ok_to_remove_submodule(old->name))
> + return 0;
> + }
Do we need a return 1 in here somewhere? Because otherwise, we fall through and return 0 later.
^ permalink raw reply
* Re: [PATCH 13/16] submodule: teach unpack_trees() to update submodules
From: Brandon Williams @ 2016-11-16 0:25 UTC (permalink / raw)
To: Stefan Beller; +Cc: git, gitster, jrnieder, mogulguy10, David.Turner
In-Reply-To: <20161115230651.23953-14-sbeller@google.com>
On 11/15, Stefan Beller wrote:
> + if (!S_ISGITLINK(ce->ce_mode)) {
> + int flags = CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE;
For readability you may want to have spaces between the two flags
> + if (o->index_only
> + || (!((old->ce_flags & CE_VALID) || ce_skip_worktree(old))
> + && (o->reset || ce_uptodate(old))))
> + return 0;
The coding guidelines say that git prefers to have the logical operators
on the right side like this:
if (o->index_only ||
(!((old->ce_flags & CE_VALID) || ce_skip_worktree(old)) &&
(o->reset || ce_uptodate(old))))
return 0;
It also says the other way is ok too, just a thought :)
> + if (submodule_is_interesting(old->name, null_sha1)
> + && is_submodule_checkout_safe(new->name, &new->oid))
> + return 0;
here too.
> + } else {
> + /*
> + * new is not a submodule any more, so only
> + * care if we care:
> + */
> + if (submodule_is_interesting(old->name, null_sha1)
> + && ok_to_remove_submodule(old->name))
> + return 0;
and here
--
Brandon Williams
^ permalink raw reply
* Re: [PATCH 14/16] checkout: recurse into submodules if asked to
From: Brandon Williams @ 2016-11-16 0:33 UTC (permalink / raw)
To: Stefan Beller; +Cc: git, gitster, jrnieder, mogulguy10, David.Turner
In-Reply-To: <20161115230651.23953-15-sbeller@google.com>
On 11/15, Stefan Beller wrote:
> +int option_parse_recurse_submodules(const struct option *opt,
> + const char *arg, int unset)
> +{
> + if (unset) {
> + recurse_submodules = RECURSE_SUBMODULES_OFF;
> + return 0;
> + }
> + if (arg)
> + recurse_submodules =
> + parse_update_recurse_submodules_arg(opt->long_name,
> + arg);
> + else
> + recurse_submodules = RECURSE_SUBMODULES_ON;
> +
> + return 0;
> +}
I assume it is ok to always return 0 from this function? Also, I know
we don't like having braces on single statement if's but it is a bit
hard to read that multi-line statement without braces. But that's just
my opinion :)
--
Brandon Williams
^ permalink raw reply
* Re: [PATCH v3 5/6] grep: enable recurse-submodules to work on <tree> objects
From: Stefan Beller @ 2016-11-16 1:09 UTC (permalink / raw)
To: Brandon Williams; +Cc: git@vger.kernel.org, Jonathan Tan, Junio C Hamano
In-Reply-To: <1478908273-190166-6-git-send-email-bmwill@google.com>
On Fri, Nov 11, 2016 at 3:51 PM, Brandon Williams <bmwill@google.com> wrote:
> to:
> HEAD:file
> HEAD:sub/file
Maybe indent this ;)
> static struct argv_array submodule_options = ARGV_ARRAY_INIT;
> +static const char *parent_basename;
>
> static int grep_submodule_launch(struct grep_opt *opt,
> const struct grep_source *gs);
> @@ -535,19 +537,53 @@ static int grep_submodule_launch(struct grep_opt *opt,
> {
> struct child_process cp = CHILD_PROCESS_INIT;
> int status, i;
> + const char *end_of_base;
> + const char *name;
> struct work_item *w = opt->output_priv;
>
> + end_of_base = strchr(gs->name, ':');
> + if (end_of_base)
> + name = end_of_base + 1;
> + else
> + name = gs->name;
> +
> prepare_submodule_repo_env(&cp.env_array);
>
> /* Add super prefix */
> argv_array_pushf(&cp.args, "--super-prefix=%s%s/",
> super_prefix ? super_prefix : "",
> - gs->name);
> + name);
> argv_array_push(&cp.args, "grep");
>
> + /*
> + * Add basename of parent project
> + * When performing grep on a <tree> object the filename is prefixed
> + * with the object's name: '<tree-name>:filename'.
This comment is hard to read as it's unclear what the <angle brackets> mean.
(Are the supposed to indicate a variable? If so why is file name not marked up?)
> In order to
> + * provide uniformity of output we want to pass the name of the
> + * parent project's object name to the submodule so the submodule can
> + * prefix its output with the parent's name and not its own SHA1.
> + */
> + if (end_of_base)
> + argv_array_pushf(&cp.args, "--parent-basename=%.*s",
> + (int) (end_of_base - gs->name),
> + gs->name);
Do we pass this only with the tree-ish?
What if we are grepping the working tree and the file name contains a colon?
> +test_expect_success 'grep tree HEAD^' '
> + cat >expect <<-\EOF &&
> + HEAD^:a:foobar
> + HEAD^:b/b:bar
> + HEAD^:submodule/a:foobar
> + EOF
> +
> + git grep -e "bar" --recurse-submodules HEAD^ > actual &&
> + test_cmp expect actual
> +'
> +
> +test_expect_success 'grep tree HEAD^^' '
> + cat >expect <<-\EOF &&
> + HEAD^^:a:foobar
> + HEAD^^:b/b:bar
> + EOF
> +
> + git grep -e "bar" --recurse-submodules HEAD^^ > actual &&
> + test_cmp expect actual
> +'
> +
> +test_expect_success 'grep tree and pathspecs' '
> + cat >expect <<-\EOF &&
> + HEAD:submodule/a:foobar
> + HEAD:submodule/sub/a:foobar
> + EOF
> +
> + git grep -e "bar" --recurse-submodules HEAD -- submodule > actual &&
> + test_cmp expect actual
> +'
Mind to add tests for
* recursive submodules (say 2 levels), preferrably not having the
gitlink at the root each, i.e. root has a sub1 at path subs/sub1 and
sub1 has a sub2
at path subs/sub2, such that recursing would produce a path like
HEAD:subs/sub1/subs/sub2/dir/file ?
* file names with a colon in it
* instead of just HEAD referencing trees, maybe a sha1 referenced test as well
(though it is not immediately clear what the benefit would be)
* what if the submodule doesn't have the commit referenced in the given sha1
Thanks,
Stefan
^ permalink raw reply
* Re: [PATCH v7 13/17] ref-filter: add `:dir` and `:base` options for ref printing atoms
From: Jacob Keller @ 2016-11-15 21:19 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Karthik Nayak, Git mailing list
In-Reply-To: <xmqq4m38vdw4.fsf@gitster.mtv.corp.google.com>
On November 15, 2016 9:42:03 AM PST, Junio C Hamano <gitster@pobox.com> wrote:
>I think you are going in the right direction. I had a similar
>thought but built around a different axis. I.e. if strip=1 strips
>one from the left, perhaps we want to have rstrip=1 that strips one
>from the right, and also strip=-1 to mean strip everything except
>one from the left and so on?. I think this and your keep (and
>perhaps you'll have rkeep for completeness) have the same expressive
>power. I do not offhand have a preference one over the other.
I prefer strip implemented with negative numbers. That is simple and expressive and if we need we can implement rstrip if necessary.
>
>Somehow it sounds a bit strange to me to treat 'remotes' as the same
>class of token as 'heads' and 'tags' (I'd expect 'heads' and
>'remotes/origin' would be at the same level in end-user's mind), but
>that is probably an unrelated tangent. The reason this series wants
>to introduce :base must be to emulate an existing feature, so that
>existing feature is a concrete counter-example that argues against
>my "it sounds a bit strange" reaction.
It may be a bit strange indeed. What is the requirement for this?
I think implementing a strip and rstrip ( if necessary ) with negative numbers would be most ideal.
Thanks
Jake
^ permalink raw reply
* Re: Bug with disabling compression and 'binary' files.
From: Junio C Hamano @ 2016-11-16 1:42 UTC (permalink / raw)
To: Douglas Cox; +Cc: git
In-Reply-To: <CA+i4re65SsxcaLcpGyMDnJygQFmAq4X_x_uxrkqB0yqQkEYPUQ@mail.gmail.com>
Douglas Cox <ziflin@gmail.com> writes:
> I narrowed this down to the '-text' attribute that is set when
> specifying 'binary'. For some reason this flag is cancelling out the
> core.compression = 0 setting and I think this is a bug?
>
> Unfortunately core.compression = 0 is also global. Ideally it would be
> great if there was a separate 'compression' attribute that could be
> specified in .gitattributes per wildcard similar to how -delta can be
> used. This way we would still be able to get compression for
> text/source files, while still getting the speed of skipping
> compression for binary files that do not compress well.
>
> Has there been any discussion on having an attribute similar to this?
Nope.
I do not offhand think of a way for '-text' attribute (or any
attribute for what matter) to interfere with compression level, but
while reading the various parts of the system that futz with the
compression level configuration, I noticed one thing. When we do an
initial "bulk-checkin" optimization that sends all objects to a
single packfile upon "git add", the packfile creation uses its own
compression level that is not affected by any configuration or
command line option. This may or may not be related to the symptom
you are observing (if it is, then you would see a packfile created
in objects/pack/, not in loose objects in object/??/ directories).
^ permalink raw reply
* [PATCH] compression: unify pack.compression configuration parsing
From: Junio C Hamano @ 2016-11-16 2:48 UTC (permalink / raw)
To: git
There are three codepaths that use a variable whose name is
pack_compression_level to affect how objects and deltas sent to a
packfile is compressed. Unlike zlib_compression_level that controls
the loose object compression, however, this variable was static to
each of these codepaths. Two of them read the pack.compression
configuration variable, using core.compression as the default, and
one of them also allowed overriding it from the command line.
The other codepath in bulk-checkin did not pay any attention to the
configuration.
Unify the configuration parsing to git_default_config(), where we
implement the parsing of core.loosecompression and core.compression
and make the former override the latter, by moving code to parse
pack.compression and also allow core.compression to give default to
this variable.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
* I was primarily interested in teaching the configuration to
bulk-checkin codepath, and added a test that succeeds only with
the code change. The handling of confugraiton in other two
codepaths, pack-objects and fast-import, may be totally broken
with this change if there is no existing test. A similar set of
tests for them are very much welcomed, as without them this
change won't be 'next' worthy yet.
builtin/pack-objects.c | 14 --------------
bulk-checkin.c | 2 --
cache.h | 2 ++
config.c | 14 ++++++++++++++
environment.c | 2 ++
fast-import.c | 13 -------------
t/t1050-large.sh | 30 ++++++++++++++++++++++++++++++
7 files changed, 48 insertions(+), 29 deletions(-)
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 0fd52bd6b4..8841f8b366 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -61,8 +61,6 @@ static int delta_search_threads;
static int pack_to_stdout;
static int num_preferred_base;
static struct progress *progress_state;
-static int pack_compression_level = Z_DEFAULT_COMPRESSION;
-static int pack_compression_seen;
static struct packed_git *reuse_packfile;
static uint32_t reuse_packfile_objects;
@@ -2368,16 +2366,6 @@ static int git_pack_config(const char *k, const char *v, void *cb)
depth = git_config_int(k, v);
return 0;
}
- if (!strcmp(k, "pack.compression")) {
- int level = git_config_int(k, v);
- if (level == -1)
- level = Z_DEFAULT_COMPRESSION;
- else if (level < 0 || level > Z_BEST_COMPRESSION)
- die("bad pack compression level %d", level);
- pack_compression_level = level;
- pack_compression_seen = 1;
- return 0;
- }
if (!strcmp(k, "pack.deltacachesize")) {
max_delta_cache_size = git_config_int(k, v);
return 0;
@@ -2869,8 +2857,6 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
reset_pack_idx_option(&pack_idx_opts);
git_config(git_pack_config, NULL);
- if (!pack_compression_seen && core_compression_seen)
- pack_compression_level = core_compression_level;
progress = isatty(2);
argc = parse_options(argc, argv, prefix, pack_objects_options,
diff --git a/bulk-checkin.c b/bulk-checkin.c
index 4347f5c76a..991b4a13e2 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -7,8 +7,6 @@
#include "pack.h"
#include "strbuf.h"
-static int pack_compression_level = Z_DEFAULT_COMPRESSION;
-
static struct bulk_checkin_state {
unsigned plugged:1;
diff --git a/cache.h b/cache.h
index a50a61a197..a5d689f9d3 100644
--- a/cache.h
+++ b/cache.h
@@ -671,6 +671,8 @@ extern const char *git_hooks_path;
extern int zlib_compression_level;
extern int core_compression_level;
extern int core_compression_seen;
+extern int pack_compression_level;
+extern int pack_compression_seen;
extern size_t packed_git_window_size;
extern size_t packed_git_limit;
extern size_t delta_base_cache_limit;
diff --git a/config.c b/config.c
index 83fdecb1bc..0477386f27 100644
--- a/config.c
+++ b/config.c
@@ -865,6 +865,8 @@ static int git_default_core_config(const char *var, const char *value)
core_compression_seen = 1;
if (!zlib_compression_seen)
zlib_compression_level = level;
+ if (!pack_compression_seen)
+ pack_compression_level = level;
return 0;
}
@@ -1125,6 +1127,18 @@ int git_default_config(const char *var, const char *value, void *dummy)
pack_size_limit_cfg = git_config_ulong(var, value);
return 0;
}
+
+ if (!strcmp(var, "pack.compression")) {
+ int level = git_config_int(var, value);
+ if (level == -1)
+ level = Z_DEFAULT_COMPRESSION;
+ else if (level < 0 || level > Z_BEST_COMPRESSION)
+ die(_("bad pack compression level %d"), level);
+ pack_compression_level = level;
+ pack_compression_seen = 1;
+ return 0;
+ }
+
/* Add other config variables here and to Documentation/config.txt. */
return 0;
}
diff --git a/environment.c b/environment.c
index 0935ec696e..88ab1c38c6 100644
--- a/environment.c
+++ b/environment.c
@@ -35,6 +35,8 @@ const char *git_hooks_path;
int zlib_compression_level = Z_BEST_SPEED;
int core_compression_level;
int core_compression_seen;
+int pack_compression_level = Z_DEFAULT_COMPRESSION;
+int pack_compression_seen;
int fsync_object_files;
size_t packed_git_window_size = DEFAULT_PACKED_GIT_WINDOW_SIZE;
size_t packed_git_limit = DEFAULT_PACKED_GIT_LIMIT;
diff --git a/fast-import.c b/fast-import.c
index cb545d7df5..f561ba833b 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -284,8 +284,6 @@ static unsigned long max_depth = 10;
static off_t max_packsize;
static int unpack_limit = 100;
static int force_update;
-static int pack_compression_level = Z_DEFAULT_COMPRESSION;
-static int pack_compression_seen;
/* Stats and misc. counters */
static uintmax_t alloc_count;
@@ -3381,15 +3379,6 @@ static void git_pack_config(void)
if (max_depth > MAX_DEPTH)
max_depth = MAX_DEPTH;
}
- if (!git_config_get_int("pack.compression", &pack_compression_level)) {
- if (pack_compression_level == -1)
- pack_compression_level = Z_DEFAULT_COMPRESSION;
- else if (pack_compression_level < 0 ||
- pack_compression_level > Z_BEST_COMPRESSION)
- git_die_config("pack.compression",
- "bad pack compression level %d", pack_compression_level);
- pack_compression_seen = 1;
- }
if (!git_config_get_int("pack.indexversion", &indexversion_value)) {
pack_idx_opts.version = indexversion_value;
if (pack_idx_opts.version > 2)
@@ -3454,8 +3443,6 @@ int cmd_main(int argc, const char **argv)
setup_git_directory();
reset_pack_idx_option(&pack_idx_opts);
git_pack_config();
- if (!pack_compression_seen && core_compression_seen)
- pack_compression_level = core_compression_level;
alloc_objects(object_entry_alloc);
strbuf_init(&command_buf, 0);
diff --git a/t/t1050-large.sh b/t/t1050-large.sh
index 096dbffecc..f800af1451 100755
--- a/t/t1050-large.sh
+++ b/t/t1050-large.sh
@@ -5,6 +5,13 @@ test_description='adding and checking out large blobs'
. ./test-lib.sh
+# This should be moved to test-lib.sh together with the
+# copy in t0021 after both topics have graduated to 'master'.
+
+file_size () {
+ perl -e 'print -s $ARGV[0]' "$1"
+}
+
test_expect_success setup '
# clone does not allow us to pass core.bigfilethreshold to
# new repos, so set core.bigfilethreshold globally
@@ -17,6 +24,29 @@ test_expect_success setup '
export GIT_ALLOC_LIMIT
'
+# add a large file with different settings
+while read expect config
+do
+ test_expect_success "add with $config" '
+ test_when_finished "rm -f .git/objects/pack/pack-*.* .git/index" &&
+ git $config add large1 &&
+ sz=$(file_size .git/objects/pack/pack-*.pack) &&
+ case "$expect" in
+ small) test "$sz" -le 100000 ;;
+ large) test "$sz" -ge 100000 ;;
+ esac
+ '
+done <<-EOF
+large -c core.compression=0
+small -c core.compression=9
+large -c core.compression=0 -c pack.compression=0
+large -c core.compression=9 -c pack.compression=0
+small -c core.compression=0 -c pack.compression=9
+small -c core.compression=9 -c pack.compression=9
+large -c pack.compression=0
+small -c pack.compression=9
+EOF
+
test_expect_success 'add a large file or two' '
git add large1 huge large2 &&
# make sure we got a single packfile and no loose objects
--
2.11.0-rc1-154-g4118e37061
^ permalink raw reply related
* Re: Bug with disabling compression and 'binary' files.
From: Douglas Cox @ 2016-11-16 3:12 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <xmqqk2c4ryj8.fsf@gitster.mtv.corp.google.com>
> This may or may not be related to the symptom
> you are observing (if it is, then you would see a packfile created
> in objects/pack/, not in loose objects in object/??/ directories).
No, the file is loose (it's in .git/objects/eb in this case). This is
seen immediately after the add, though I believe it's the same way
when doing a commit on a changed file.
On Tue, Nov 15, 2016 at 8:42 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Douglas Cox <ziflin@gmail.com> writes:
>
>> I narrowed this down to the '-text' attribute that is set when
>> specifying 'binary'. For some reason this flag is cancelling out the
>> core.compression = 0 setting and I think this is a bug?
>>
>> Unfortunately core.compression = 0 is also global. Ideally it would be
>> great if there was a separate 'compression' attribute that could be
>> specified in .gitattributes per wildcard similar to how -delta can be
>> used. This way we would still be able to get compression for
>> text/source files, while still getting the speed of skipping
>> compression for binary files that do not compress well.
>>
>> Has there been any discussion on having an attribute similar to this?
>
> Nope.
>
> I do not offhand think of a way for '-text' attribute (or any
> attribute for what matter) to interfere with compression level, but
> while reading the various parts of the system that futz with the
> compression level configuration, I noticed one thing. When we do an
> initial "bulk-checkin" optimization that sends all objects to a
> single packfile upon "git add", the packfile creation uses its own
> compression level that is not affected by any configuration or
> command line option. This may or may not be related to the symptom
> you are observing (if it is, then you would see a packfile created
> in objects/pack/, not in loose objects in object/??/ directories).
>
>
>
^ permalink raw reply
* [PATCH v2] compression: unify pack.compression configuration parsing
From: Junio C Hamano @ 2016-11-16 5:21 UTC (permalink / raw)
To: git
In-Reply-To: <xmqqfumsrvge.fsf@gitster.mtv.corp.google.com>
There are three codepaths that use a variable whose name is
pack_compression_level to affect how objects and deltas sent to a
packfile is compressed. Unlike zlib_compression_level that controls
the loose object compression, however, this variable was static to
each of these codepaths. Two of them read the pack.compression
configuration variable, using core.compression as the default, and
one of them also allowed overriding it from the command line.
The other codepath in bulk-checkin did not pay any attention to the
configuration.
Unify the configuration parsing to git_default_config(), where we
implement the parsing of core.loosecompression and core.compression
and make the former override the latter, by moving code to parse
pack.compression and also allow core.compression to give default to
this variable.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
* Now comes with new tests to cover pack-objects and fast-import,
which should not fail with or without code change because they
are only here to prevent regression with this change.
On the code side, *_compression_seen variables are also made
private to config.c, as these variables are used only by the
config reader to implement "core.compression gives the default to
*.compression" logic and the code is consolidated to this single
file.
builtin/pack-objects.c | 14 --------
bulk-checkin.c | 2 --
cache.h | 2 +-
config.c | 16 +++++++++
environment.c | 2 +-
fast-import.c | 13 -------
t/t1050-large.sh | 29 ++++++++++++++++
t/t5315-pack-objects-compression.sh | 44 ++++++++++++++++++++++++
t/t9303-fast-import-compression.sh | 67 +++++++++++++++++++++++++++++++++++++
9 files changed, 158 insertions(+), 31 deletions(-)
create mode 100755 t/t5315-pack-objects-compression.sh
create mode 100755 t/t9303-fast-import-compression.sh
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 0fd52bd6b4..8841f8b366 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -61,8 +61,6 @@ static int delta_search_threads;
static int pack_to_stdout;
static int num_preferred_base;
static struct progress *progress_state;
-static int pack_compression_level = Z_DEFAULT_COMPRESSION;
-static int pack_compression_seen;
static struct packed_git *reuse_packfile;
static uint32_t reuse_packfile_objects;
@@ -2368,16 +2366,6 @@ static int git_pack_config(const char *k, const char *v, void *cb)
depth = git_config_int(k, v);
return 0;
}
- if (!strcmp(k, "pack.compression")) {
- int level = git_config_int(k, v);
- if (level == -1)
- level = Z_DEFAULT_COMPRESSION;
- else if (level < 0 || level > Z_BEST_COMPRESSION)
- die("bad pack compression level %d", level);
- pack_compression_level = level;
- pack_compression_seen = 1;
- return 0;
- }
if (!strcmp(k, "pack.deltacachesize")) {
max_delta_cache_size = git_config_int(k, v);
return 0;
@@ -2869,8 +2857,6 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
reset_pack_idx_option(&pack_idx_opts);
git_config(git_pack_config, NULL);
- if (!pack_compression_seen && core_compression_seen)
- pack_compression_level = core_compression_level;
progress = isatty(2);
argc = parse_options(argc, argv, prefix, pack_objects_options,
diff --git a/bulk-checkin.c b/bulk-checkin.c
index 4347f5c76a..991b4a13e2 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -7,8 +7,6 @@
#include "pack.h"
#include "strbuf.h"
-static int pack_compression_level = Z_DEFAULT_COMPRESSION;
-
static struct bulk_checkin_state {
unsigned plugged:1;
diff --git a/cache.h b/cache.h
index a50a61a197..b14c0e8e38 100644
--- a/cache.h
+++ b/cache.h
@@ -670,7 +670,7 @@ extern const char *git_attributes_file;
extern const char *git_hooks_path;
extern int zlib_compression_level;
extern int core_compression_level;
-extern int core_compression_seen;
+extern int pack_compression_level;
extern size_t packed_git_window_size;
extern size_t packed_git_limit;
extern size_t delta_base_cache_limit;
diff --git a/config.c b/config.c
index 83fdecb1bc..2f1aef742e 100644
--- a/config.c
+++ b/config.c
@@ -66,6 +66,8 @@ static struct key_value_info *current_config_kvi;
*/
static enum config_scope current_parsing_scope;
+static int core_compression_seen;
+static int pack_compression_seen;
static int zlib_compression_seen;
/*
@@ -865,6 +867,8 @@ static int git_default_core_config(const char *var, const char *value)
core_compression_seen = 1;
if (!zlib_compression_seen)
zlib_compression_level = level;
+ if (!pack_compression_seen)
+ pack_compression_level = level;
return 0;
}
@@ -1125,6 +1129,18 @@ int git_default_config(const char *var, const char *value, void *dummy)
pack_size_limit_cfg = git_config_ulong(var, value);
return 0;
}
+
+ if (!strcmp(var, "pack.compression")) {
+ int level = git_config_int(var, value);
+ if (level == -1)
+ level = Z_DEFAULT_COMPRESSION;
+ else if (level < 0 || level > Z_BEST_COMPRESSION)
+ die(_("bad pack compression level %d"), level);
+ pack_compression_level = level;
+ pack_compression_seen = 1;
+ return 0;
+ }
+
/* Add other config variables here and to Documentation/config.txt. */
return 0;
}
diff --git a/environment.c b/environment.c
index 0935ec696e..4bce3eebfa 100644
--- a/environment.c
+++ b/environment.c
@@ -34,7 +34,7 @@ const char *git_attributes_file;
const char *git_hooks_path;
int zlib_compression_level = Z_BEST_SPEED;
int core_compression_level;
-int core_compression_seen;
+int pack_compression_level = Z_DEFAULT_COMPRESSION;
int fsync_object_files;
size_t packed_git_window_size = DEFAULT_PACKED_GIT_WINDOW_SIZE;
size_t packed_git_limit = DEFAULT_PACKED_GIT_LIMIT;
diff --git a/fast-import.c b/fast-import.c
index cb545d7df5..f561ba833b 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -284,8 +284,6 @@ static unsigned long max_depth = 10;
static off_t max_packsize;
static int unpack_limit = 100;
static int force_update;
-static int pack_compression_level = Z_DEFAULT_COMPRESSION;
-static int pack_compression_seen;
/* Stats and misc. counters */
static uintmax_t alloc_count;
@@ -3381,15 +3379,6 @@ static void git_pack_config(void)
if (max_depth > MAX_DEPTH)
max_depth = MAX_DEPTH;
}
- if (!git_config_get_int("pack.compression", &pack_compression_level)) {
- if (pack_compression_level == -1)
- pack_compression_level = Z_DEFAULT_COMPRESSION;
- else if (pack_compression_level < 0 ||
- pack_compression_level > Z_BEST_COMPRESSION)
- git_die_config("pack.compression",
- "bad pack compression level %d", pack_compression_level);
- pack_compression_seen = 1;
- }
if (!git_config_get_int("pack.indexversion", &indexversion_value)) {
pack_idx_opts.version = indexversion_value;
if (pack_idx_opts.version > 2)
@@ -3454,8 +3443,6 @@ int cmd_main(int argc, const char **argv)
setup_git_directory();
reset_pack_idx_option(&pack_idx_opts);
git_pack_config();
- if (!pack_compression_seen && core_compression_seen)
- pack_compression_level = core_compression_level;
alloc_objects(object_entry_alloc);
strbuf_init(&command_buf, 0);
diff --git a/t/t1050-large.sh b/t/t1050-large.sh
index 096dbffecc..6fd264cff0 100755
--- a/t/t1050-large.sh
+++ b/t/t1050-large.sh
@@ -5,6 +5,12 @@ test_description='adding and checking out large blobs'
. ./test-lib.sh
+# This should be moved to test-lib.sh together with the
+# copy in t0021 after both topics have graduated to 'master'.
+file_size () {
+ perl -e 'print -s $ARGV[0]' "$1"
+}
+
test_expect_success setup '
# clone does not allow us to pass core.bigfilethreshold to
# new repos, so set core.bigfilethreshold globally
@@ -17,6 +23,29 @@ test_expect_success setup '
export GIT_ALLOC_LIMIT
'
+# add a large file with different settings
+while read expect config
+do
+ test_expect_success "add with $config" '
+ test_when_finished "rm -f .git/objects/pack/pack-*.* .git/index" &&
+ git $config add large1 &&
+ sz=$(file_size .git/objects/pack/pack-*.pack) &&
+ case "$expect" in
+ small) test "$sz" -le 100000 ;;
+ large) test "$sz" -ge 100000 ;;
+ esac
+ '
+done <<\EOF
+large -c core.compression=0
+small -c core.compression=9
+large -c core.compression=0 -c pack.compression=0
+large -c core.compression=9 -c pack.compression=0
+small -c core.compression=0 -c pack.compression=9
+small -c core.compression=9 -c pack.compression=9
+large -c pack.compression=0
+small -c pack.compression=9
+EOF
+
test_expect_success 'add a large file or two' '
git add large1 huge large2 &&
# make sure we got a single packfile and no loose objects
diff --git a/t/t5315-pack-objects-compression.sh b/t/t5315-pack-objects-compression.sh
new file mode 100755
index 0000000000..34c47dae09
--- /dev/null
+++ b/t/t5315-pack-objects-compression.sh
@@ -0,0 +1,44 @@
+#!/bin/sh
+
+test_description='pack-object compression configuration'
+
+. ./test-lib.sh
+
+# This should be moved to test-lib.sh together with the
+# copy in t0021 after both topics have graduated to 'master'.
+file_size () {
+ perl -e 'print -s $ARGV[0]' "$1"
+}
+
+test_expect_success setup '
+ printf "%2000000s" X |
+ git hash-object -w --stdin >object-name &&
+ # make sure it resulted in a loose object
+ ob=$(sed -e "s/\(..\).*/\1/" object-name) &&
+ ject=$(sed -e "s/..\(.*\)/\1/" object-name) &&
+ test -f .git/objects/$ob/$ject
+'
+
+while read expect config
+do
+ test_expect_success "pack-objects with $config" '
+ test_when_finished "rm -f pack-*.*" &&
+ git $config pack-objects pack <object-name &&
+ sz=$(file_size pack-*.pack) &&
+ case "$expect" in
+ small) test "$sz" -le 100000 ;;
+ large) test "$sz" -ge 100000 ;;
+ esac
+ '
+done <<\EOF
+large -c core.compression=0
+small -c core.compression=9
+large -c core.compression=0 -c pack.compression=0
+large -c core.compression=9 -c pack.compression=0
+small -c core.compression=0 -c pack.compression=9
+small -c core.compression=9 -c pack.compression=9
+large -c pack.compression=0
+small -c pack.compression=9
+EOF
+
+test_done
diff --git a/t/t9303-fast-import-compression.sh b/t/t9303-fast-import-compression.sh
new file mode 100755
index 0000000000..856219f46a
--- /dev/null
+++ b/t/t9303-fast-import-compression.sh
@@ -0,0 +1,67 @@
+#!/bin/sh
+
+test_description='compression setting of fast-import utility'
+. ./test-lib.sh
+
+# This should be moved to test-lib.sh together with the
+# copy in t0021 after both topics have graduated to 'master'.
+file_size () {
+ perl -e 'print -s $ARGV[0]' "$1"
+}
+
+import_large () {
+ (
+ echo blob
+ echo "data <<EOD"
+ printf "%2000000s\n" "$*"
+ echo EOD
+ ) | git "$@" fast-import
+}
+
+while read expect config
+do
+ test_expect_success "fast-import (packed) with $config" '
+ test_when_finished "rm -f .git/objects/pack/pack-*.*" &&
+ test_when_finished "rm -rf .git/objects/??" &&
+ import_large -c fastimport.unpacklimit=0 $config &&
+ sz=$(file_size .git/objects/pack/pack-*.pack) &&
+ case "$expect" in
+ small) test "$sz" -le 100000 ;;
+ large) test "$sz" -ge 100000 ;;
+ esac
+ '
+done <<\EOF
+large -c core.compression=0
+small -c core.compression=9
+large -c core.compression=0 -c pack.compression=0
+large -c core.compression=9 -c pack.compression=0
+small -c core.compression=0 -c pack.compression=9
+small -c core.compression=9 -c pack.compression=9
+large -c pack.compression=0
+small -c pack.compression=9
+EOF
+
+while read expect config
+do
+ test_expect_success "fast-import (loose) with $config" '
+ test_when_finished "rm -f .git/objects/pack/pack-*.*" &&
+ test_when_finished "rm -rf .git/objects/??" &&
+ import_large -c fastimport.unpacklimit=9 $config &&
+ sz=$(file_size .git/objects/??/????*) &&
+ case "$expect" in
+ small) test "$sz" -le 100000 ;;
+ large) test "$sz" -ge 100000 ;;
+ esac
+ '
+done <<\EOF
+large -c core.compression=0
+small -c core.compression=9
+large -c core.compression=0 -c core.loosecompression=0
+large -c core.compression=9 -c core.loosecompression=0
+small -c core.compression=0 -c core.loosecompression=9
+small -c core.compression=9 -c core.loosecompression=9
+large -c core.loosecompression=0
+small -c core.loosecompression=9
+EOF
+
+test_done
--
2.11.0-rc1-154-g4118e37061
^ permalink raw reply related
* Re: Bug with disabling compression and 'binary' files.
From: Junio C Hamano @ 2016-11-16 5:23 UTC (permalink / raw)
To: Douglas Cox; +Cc: git
In-Reply-To: <CA+i4re4FZhshheftQnsogaY7601wgf_GfsFUxy+doLcOYPWv7Q@mail.gmail.com>
Douglas Cox <ziflin@gmail.com> writes:
>> This may or may not be related to the symptom
>> you are observing (if it is, then you would see a packfile created
>> in objects/pack/, not in loose objects in object/??/ directories).
>
> No, the file is loose (it's in .git/objects/eb in this case). This is
> seen immediately after the add, though I believe it's the same way
> when doing a commit on a changed file.
Then I do not have a guess as to where the symptom you are seeing is
coming from.
^ permalink raw reply
* Re: [PATCH v7 13/17] ref-filter: add `:dir` and `:base` options for ref printing atoms
From: Karthik Nayak @ 2016-11-16 7:58 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Jacob Keller, Git mailing list
In-Reply-To: <xmqq4m38vdw4.fsf@gitster.mtv.corp.google.com>
On Tue, Nov 15, 2016 at 11:12 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Jacob Keller <jacob.keller@gmail.com> writes:
>
>> dirname makes sense. What about implementing a reverse variant of
>> strip, which you could perform stripping of right-most components and
>> instead of stripping by a number, strip "to" a number, ie: keep the
>> left N most components, and then you could use something like
>> ...
>> I think that would be more general purpose than basename, and less confusing?
>
> I think you are going in the right direction. I had a similar
> thought but built around a different axis. I.e. if strip=1 strips
> one from the left, perhaps we want to have rstrip=1 that strips one
> from the right, and also strip=-1 to mean strip everything except
> one from the left and so on?. I think this and your keep (and
> perhaps you'll have rkeep for completeness) have the same expressive
> power. I do not offhand have a preference one over the other.
>
If we do implement strip with negative numbers, it definitely would be
neat, but to get
the desired feature which I've mentioned below, we'd need to call
strip twice, i.e
to get remotes from /refs/foo/abc/xyz we'd need to do
strip=1,strip=-1, which could be
done but would need a lot of tweaking, since we have the same
subatom/option having
multiple values.
On the other hand it would be easier maybe to just introduce rstrip,
where we strip from
right and ensure that strip and rstrip can be used together.
On Wed, Nov 16, 2016 at 2:49 AM, Jacob Keller <jacob.keller@gmail.com> wrote:
> On November 15, 2016 9:42:03 AM PST, Junio C Hamano <gitster@pobox.com> wrote:
>>Somehow it sounds a bit strange to me to treat 'remotes' as the same
>>class of token as 'heads' and 'tags' (I'd expect 'heads' and
>>'remotes/origin' would be at the same level in end-user's mind), but
>>that is probably an unrelated tangent. The reason this series wants
>>to introduce :base must be to emulate an existing feature, so that
>>existing feature is a concrete counter-example that argues against
>>my "it sounds a bit strange" reaction.
>
> It may be a bit strange indeed. What is the requirement for this?
>
> I think implementing a strip and rstrip ( if necessary ) with negative numbers would be most ideal.
The necessity is that we need to do different formatting as per the
ref type in branch -l. If you see the
last but one patch, we do
strbuf_addf(&fmt,
"%%(if:notequals=remotes)%%(refname:base)%%(then)%s%%(else)%s%%(end)",
local.buf, remote.buf);
where its using ':base' to check for the ref type and do conditional
printing along with the %(if)...%(end) atoms.
--
Regards,
Karthik Nayak
^ permalink raw reply
* Re: gitweb html validation
From: Raphaël Gertz @ 2016-11-16 9:25 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Ralf Thielow, Jakub Narębski, git
In-Reply-To: <xmqqy40ks301.fsf@gitster.mtv.corp.google.com>
Le 16.11.2016 01:05, Junio C Hamano a écrit :
> Ralf Thielow <ralf.thielow@gmail.com> writes:
>
>> Only block level elements are
>> allowed to be inside form tags, according to
>> https://www.w3.org/2010/04/xhtml10-strict.html#elem_form
>> ...
>> I think it's better to just move the <form>-Tag outside of the
>> surrounding div?
>> Something like this perhaps, I didn't test it myself yet.
>
> That sounds like a sensible update to me (no, I do not run gitweb
> myself). Is this the only <form> we have in the UI, or is it the
> only one that is problematic?
>
There is an other form in the cgi line 4110 :
print $cgi->start_form(-method => "get", -action => $action) .
"<div class=\"search\">\n" .
But this one has a <div class="search"> inside.
The problem with projsearch I want to change is that the div is around
the form without a container inside.
I agree with moving the <div class="projsearch"> inside the form if it's
a better option.
Best regards
^ permalink raw reply
* Re: [PATCH] t0021, t5615: use $PWD instead of $(pwd) in PATH-like shell variables
From: Johannes Schindelin @ 2016-11-16 9:47 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Lars Schneider, Johannes Sixt, git, Jeff King
In-Reply-To: <alpine.DEB.2.20.1611151753300.3746@virtualbox>
Hi,
On Tue, 15 Nov 2016, Johannes Schindelin wrote:
> On Mon, 14 Nov 2016, Junio C Hamano wrote:
>
> > Dscho's mention of 'still times out' may be an indiciation that
> > something unspecified on 'pu' is not ready to be merged to 'next',
> > but blocking all of 'pu' with a blanket statement is not useful,
> > and that was where my response comes from.
>
> Until the time when the test suite takes less than the insane three hours
> to run, I am afraid that a blanket statement on `pu` is the best I can do.
Well, I should add that the test suite does not take 3 hours to run for
`pu` these days.
It used to time out after four hours until two days ago (I think; I was a
bit too busy with other CI work to pay close attention to the constantly
failing `pu` job, with quite a few failing `next`s and even a couple of
failing `master`s thrown in).
As of two days ago, the test suite takes no time at all. The build already
fails (which makes me wonder why a couple of patch series I contributed
had such a hard time getting into `pu` when they compiled and tested
just fine, whereas some obviously non-building stuff gets into `pu`
already, makes no sense to me).
This is the offending part from last night's build:
-- snipsnap --
2016-11-16T00:31:57.5321220Z copy.c: In function 'copy_dir_1':
2016-11-16T00:31:57.5321220Z copy.c:369:8: error: implicit declaration of function 'lchown' [-Werror=implicit-function-declaration]
2016-11-16T00:31:57.5321220Z if (lchown(dest, source_stat.st_uid, source_stat.st_gid) < 0)
2016-11-16T00:31:57.5321220Z ^~~~~~
2016-11-16T00:31:57.5321220Z copy.c:391:7: error: implicit declaration of function 'mknod' [-Werror=implicit-function-declaration]
2016-11-16T00:31:57.5321220Z if (mknod(dest, source_stat.st_mode, source_stat.st_rdev) < 0)
2016-11-16T00:31:57.5321220Z ^~~~~
2016-11-16T00:31:57.5321220Z copy.c:405:7: error: implicit declaration of function 'utimes' [-Werror=implicit-function-declaration]
2016-11-16T00:31:57.5321220Z if (utimes(dest, times) < 0)
2016-11-16T00:31:57.5321220Z ^~~~~~
2016-11-16T00:31:57.5321220Z copy.c:407:7: error: implicit declaration of function 'chown' [-Werror=implicit-function-declaration]
2016-11-16T00:31:57.5321220Z if (chown(dest, source_stat.st_uid, source_stat.st_gid) < 0) {
2016-11-16T00:31:57.5321220Z ^~~~~
2016-11-16T00:31:57.7982432Z CC ctype.o
2016-11-16T00:31:58.1418929Z cc1.exe: all warnings being treated as errors
2016-11-16T00:31:58.6368128Z make: *** [Makefile:1988: copy.o] Error 1
^ permalink raw reply
* Re: [RFC/PATCH 0/2] git diff <(command1) <(command2)
From: Johannes Schindelin @ 2016-11-16 9:50 UTC (permalink / raw)
To: Junio C Hamano
Cc: Michael J Gruber, Jacob Keller, Dennis Kaarsemaker,
Git mailing list
In-Reply-To: <xmqqtwb9wywp.fsf@gitster.mtv.corp.google.com>
Hi Junio,
On Mon, 14 Nov 2016, Junio C Hamano wrote:
> I _think_ the no-index mode was primarily for those who want to use
> our diff as a replacement for GNU and other diffs, and from that
> point of view, I'd favour not doing the "comparing symbolic link?
> We'll show the difference between the link contents, not target"
> under no-index mode myself.
If I read this correctly, then we are in agreement that the default for
--no-index should be as it is right now, i.e. comparing symlink targets as
opposed to --follow-links.
> That is a lot closer to the diff other people implemented, not ours.
> Hence the knee-jerk reaction I gave in
>
> http://public-inbox.org/git/xmqqinrt1zcx.fsf@gitster.mtv.corp.google.com
Let me quote the knee-jerk reaction:
> My knee-jerk reaction is:
>
> * The --no-index mode should default to your --follow-symlinks
> behaviour, without any option to turn it on or off.
But this is the exact opposite of what I find reasonable.
Ciao,
Johannes
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox