* merge --no-ff is NOT mentioned in help
From: Vanderhoof, Tzadik @ 2016-11-16 15:16 UTC (permalink / raw)
To: git@vger.kernel.org
When I do: "git merge -h" to get help, the option "--no-ff" is left out of the list of options.
This e-mail, including attachments, may include confidential and/or
proprietary information, and may be used only by the person or entity
to which it is addressed. If the reader of this e-mail is not the intended
recipient or his or her authorized agent, the reader is hereby notified
that any dissemination, distribution or copying of this e-mail is
prohibited. If you have received this e-mail in error, please notify the
sender by replying to this message and delete this e-mail immediately.
^ permalink raw reply
* [PATCH v4 3/4] batch check whether submodule needs pushing into one call
From: Heiko Voigt @ 2016-11-16 15:11 UTC (permalink / raw)
To: Junio C Hamano
Cc: Brandon Williams, git, Jeff King, Stefan Beller, Jens.Lehmann,
Fredrik Gustafsson, Leandro Lucarella
In-Reply-To: <cover.1479308877.git.hvoigt@hvoigt.net>
We run a command for each sha1 change in a submodule. This is
unnecessary since we can simply batch all sha1's we want to check into
one command. Lets do it so we can speedup the check when many submodule
changes are in need of checking.
Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
---
submodule.c | 62 ++++++++++++++++++++++++++++++++-----------------------------
1 file changed, 33 insertions(+), 29 deletions(-)
diff --git a/submodule.c b/submodule.c
index 12ac1ea..11391fa 100644
--- a/submodule.c
+++ b/submodule.c
@@ -507,27 +507,49 @@ static int append_sha1_to_argv(const unsigned char sha1[20], void *data)
return 0;
}
-static int submodule_needs_pushing(const char *path, const unsigned char sha1[20])
+static int check_has_commit(const unsigned char sha1[20], void *data)
{
- if (add_submodule_odb(path) || !lookup_commit_reference(sha1))
+ int *has_commit = data;
+
+ if (!lookup_commit_reference(sha1))
+ *has_commit = 0;
+
+ return 0;
+}
+
+static int submodule_has_commits(const char *path, struct sha1_array *commits)
+{
+ int has_commit = 1;
+
+ if (add_submodule_odb(path))
+ return 0;
+
+ sha1_array_for_each_unique(commits, check_has_commit, &has_commit);
+ return has_commit;
+}
+
+static int submodule_needs_pushing(const char *path, struct sha1_array *commits)
+{
+ if (!submodule_has_commits(path, commits))
return 0;
if (for_each_remote_ref_submodule(path, has_remote, NULL) > 0) {
struct child_process cp = CHILD_PROCESS_INIT;
- const char *argv[] = {"rev-list", NULL, "--not", "--remotes", "-n", "1" , NULL};
struct strbuf buf = STRBUF_INIT;
int needs_pushing = 0;
- argv[1] = sha1_to_hex(sha1);
- cp.argv = argv;
+ argv_array_push(&cp.args, "rev-list");
+ sha1_array_for_each_unique(commits, append_sha1_to_argv, &cp.args);
+ argv_array_pushl(&cp.args, "--not", "--remotes", "-n", "1" , NULL);
+
prepare_submodule_repo_env(&cp.env_array);
cp.git_cmd = 1;
cp.no_stdin = 1;
cp.out = -1;
cp.dir = path;
if (start_command(&cp))
- die("Could not run 'git rev-list %s --not --remotes -n 1' command in submodule %s",
- sha1_to_hex(sha1), path);
+ die("Could not run 'git rev-list <commits> --not --remotes -n 1' command in submodule %s",
+ path);
if (strbuf_read(&buf, cp.out, 41))
needs_pushing = 1;
finish_command(&cp);
@@ -582,22 +604,6 @@ static void find_unpushed_submodule_commits(struct commit *commit,
diff_tree_combined_merge(commit, 1, &rev);
}
-struct collect_submodule_from_sha1s_data {
- char *submodule_path;
- struct string_list *needs_pushing;
-};
-
-static int collect_submodules_from_sha1s(const unsigned char sha1[20],
- void *data)
-{
- struct collect_submodule_from_sha1s_data *me = data;
-
- if (submodule_needs_pushing(me->submodule_path, sha1))
- string_list_insert(me->needs_pushing, me->submodule_path);
-
- return 0;
-}
-
static void free_submodules_sha1s(struct string_list *submodules)
{
struct string_list_item *item;
@@ -634,12 +640,10 @@ int find_unpushed_submodules(struct sha1_array *commits,
argv_array_clear(&argv);
for_each_string_list_item(submodule, &submodules) {
- struct collect_submodule_from_sha1s_data data;
- data.submodule_path = submodule->string;
- data.needs_pushing = needs_pushing;
- sha1_array_for_each_unique((struct sha1_array *) submodule->util,
- collect_submodules_from_sha1s,
- &data);
+ struct sha1_array *commits = (struct sha1_array *) submodule->util;
+
+ if (submodule_needs_pushing(submodule->string, commits))
+ string_list_insert(needs_pushing, submodule->string);
}
free_submodules_sha1s(&submodules);
--
2.10.1.386.gc503e45
^ permalink raw reply related
* [PATCH v4 2/4] serialize collection of refs that contain submodule changes
From: Heiko Voigt @ 2016-11-16 15:11 UTC (permalink / raw)
To: Junio C Hamano
Cc: Brandon Williams, git, Jeff King, Stefan Beller, Jens.Lehmann,
Fredrik Gustafsson, Leandro Lucarella
In-Reply-To: <cover.1479308877.git.hvoigt@hvoigt.net>
We are iterating over each pushed ref and want to check whether it
contains changes to submodules. Instead of immediately checking each ref
lets first collect them and then do the check for all of them in one
revision walk.
Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
---
submodule.c | 35 ++++++++++++++++++++---------------
submodule.h | 5 +++--
transport.c | 29 +++++++++++++++++++++--------
3 files changed, 44 insertions(+), 25 deletions(-)
diff --git a/submodule.c b/submodule.c
index b2908fe..12ac1ea 100644
--- a/submodule.c
+++ b/submodule.c
@@ -500,6 +500,13 @@ static int has_remote(const char *refname, const struct object_id *oid,
return 1;
}
+static int append_sha1_to_argv(const unsigned char sha1[20], void *data)
+{
+ struct argv_array *argv = data;
+ argv_array_push(argv, sha1_to_hex(sha1));
+ return 0;
+}
+
static int submodule_needs_pushing(const char *path, const unsigned char sha1[20])
{
if (add_submodule_odb(path) || !lookup_commit_reference(sha1))
@@ -599,25 +606,24 @@ static void free_submodules_sha1s(struct string_list *submodules)
string_list_clear(submodules, 1);
}
-int find_unpushed_submodules(unsigned char new_sha1[20],
+int find_unpushed_submodules(struct sha1_array *commits,
const char *remotes_name, struct string_list *needs_pushing)
{
struct rev_info rev;
struct commit *commit;
- const char *argv[] = {NULL, NULL, "--not", "NULL", NULL};
- int argc = ARRAY_SIZE(argv) - 1;
- char *sha1_copy;
struct string_list submodules = STRING_LIST_INIT_DUP;
struct string_list_item *submodule;
+ struct argv_array argv = ARGV_ARRAY_INIT;
- struct strbuf remotes_arg = STRBUF_INIT;
-
- strbuf_addf(&remotes_arg, "--remotes=%s", remotes_name);
init_revisions(&rev, NULL);
- sha1_copy = xstrdup(sha1_to_hex(new_sha1));
- argv[1] = sha1_copy;
- argv[3] = remotes_arg.buf;
- setup_revisions(argc, argv, &rev, NULL);
+
+ /* argv.argv[0] will be ignored by setup_revisions */
+ argv_array_push(&argv, "find_unpushed_submodules");
+ sha1_array_for_each_unique(commits, append_sha1_to_argv, &argv);
+ argv_array_push(&argv, "--not");
+ argv_array_pushf(&argv, "--remotes=%s", remotes_name);
+
+ setup_revisions(argv.argc, argv.argv, &rev, NULL);
if (prepare_revision_walk(&rev))
die("revision walk setup failed");
@@ -625,8 +631,7 @@ int find_unpushed_submodules(unsigned char new_sha1[20],
find_unpushed_submodule_commits(commit, &submodules);
reset_revision_walk();
- free(sha1_copy);
- strbuf_release(&remotes_arg);
+ argv_array_clear(&argv);
for_each_string_list_item(submodule, &submodules) {
struct collect_submodule_from_sha1s_data data;
@@ -663,12 +668,12 @@ static int push_submodule(const char *path)
return 1;
}
-int push_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_name)
+int push_unpushed_submodules(struct sha1_array *commits, const char *remotes_name)
{
int i, ret = 1;
struct string_list needs_pushing = STRING_LIST_INIT_DUP;
- if (!find_unpushed_submodules(new_sha1, remotes_name, &needs_pushing))
+ if (!find_unpushed_submodules(commits, remotes_name, &needs_pushing))
return 1;
for (i = 0; i < needs_pushing.nr; i++) {
diff --git a/submodule.h b/submodule.h
index d9e197a..9454806 100644
--- a/submodule.h
+++ b/submodule.h
@@ -3,6 +3,7 @@
struct diff_options;
struct argv_array;
+struct sha1_array;
enum {
RECURSE_SUBMODULES_CHECK = -4,
@@ -62,9 +63,9 @@ int submodule_uses_gitfile(const char *path);
int ok_to_remove_submodule(const char *path);
int merge_submodule(unsigned char result[20], const char *path, const unsigned char base[20],
const unsigned char a[20], const unsigned char b[20], int search);
-int find_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_name,
+int find_unpushed_submodules(struct sha1_array *commits, const char *remotes_name,
struct string_list *needs_pushing);
-int push_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_name);
+int push_unpushed_submodules(struct sha1_array *commits, const char *remotes_name);
void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);
int parallel_submodules(void);
diff --git a/transport.c b/transport.c
index d57e8de..f482869 100644
--- a/transport.c
+++ b/transport.c
@@ -949,23 +949,36 @@ int transport_push(struct transport *transport,
if ((flags & TRANSPORT_RECURSE_SUBMODULES_ON_DEMAND) && !is_bare_repository()) {
struct ref *ref = remote_refs;
+ struct sha1_array commits = SHA1_ARRAY_INIT;
+
for (; ref; ref = ref->next)
- if (!is_null_oid(&ref->new_oid) &&
- !push_unpushed_submodules(ref->new_oid.hash,
- transport->remote->name))
- die ("Failed to push all needed submodules!");
+ if (!is_null_oid(&ref->new_oid))
+ sha1_array_append(&commits, ref->new_oid.hash);
+
+ if (!push_unpushed_submodules(&commits, transport->remote->name)) {
+ sha1_array_clear(&commits);
+ die("Failed to push all needed submodules!");
+ }
+ sha1_array_clear(&commits);
}
if ((flags & (TRANSPORT_RECURSE_SUBMODULES_ON_DEMAND |
TRANSPORT_RECURSE_SUBMODULES_CHECK)) && !is_bare_repository()) {
struct ref *ref = remote_refs;
struct string_list needs_pushing = STRING_LIST_INIT_DUP;
+ struct sha1_array commits = SHA1_ARRAY_INIT;
for (; ref; ref = ref->next)
- if (!is_null_oid(&ref->new_oid) &&
- find_unpushed_submodules(ref->new_oid.hash,
- transport->remote->name, &needs_pushing))
- die_with_unpushed_submodules(&needs_pushing);
+ if (!is_null_oid(&ref->new_oid))
+ sha1_array_append(&commits, ref->new_oid.hash);
+
+ if (find_unpushed_submodules(&commits, transport->remote->name,
+ &needs_pushing)) {
+ sha1_array_clear(&commits);
+ die_with_unpushed_submodules(&needs_pushing);
+ }
+ string_list_clear(&needs_pushing, 0);
+ sha1_array_clear(&commits);
}
push_ret = transport->push_refs(transport, remote_refs, flags);
--
2.10.1.386.gc503e45
^ permalink raw reply related
* [PATCH v4 4/4] submodule_needs_pushing() NEEDSWORK when we can not answer this question
From: Heiko Voigt @ 2016-11-16 15:11 UTC (permalink / raw)
To: Junio C Hamano
Cc: Brandon Williams, git, Jeff King, Stefan Beller, Jens.Lehmann,
Fredrik Gustafsson, Leandro Lucarella
In-Reply-To: <cover.1479308877.git.hvoigt@hvoigt.net>
Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
---
submodule.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/submodule.c b/submodule.c
index 11391fa..00dd655 100644
--- a/submodule.c
+++ b/submodule.c
@@ -531,6 +531,17 @@ static int submodule_has_commits(const char *path, struct sha1_array *commits)
static int submodule_needs_pushing(const char *path, struct sha1_array *commits)
{
if (!submodule_has_commits(path, commits))
+ /*
+ * NOTE: We do consider it safe to return "no" here. The
+ * correct answer would be "We do not know" instead of
+ * "No push needed", but it is quite hard to change
+ * the submodule pointer without having the submodule
+ * around. If a user did however change the submodules
+ * without having the submodule around, this indicates
+ * an expert who knows what they are doing or a
+ * maintainer integrating work from other people. In
+ * both cases it should be safe to skip this check.
+ */
return 0;
if (for_each_remote_ref_submodule(path, has_remote, NULL) > 0) {
--
2.10.1.386.gc503e45
^ permalink raw reply related
* [PATCH v4 1/4] serialize collection of changed submodules
From: Heiko Voigt @ 2016-11-16 15:11 UTC (permalink / raw)
To: Junio C Hamano
Cc: Brandon Williams, git, Jeff King, Stefan Beller, Jens.Lehmann,
Fredrik Gustafsson, Leandro Lucarella
In-Reply-To: <cover.1479308877.git.hvoigt@hvoigt.net>
To check whether a submodule needs to be pushed we need to collect all
changed submodules. Lets collect them first and then execute the
possibly expensive test whether certain revisions are already pushed
only once per submodule.
There is further potential for optimization since we can assemble one
command and only issued that instead of one call for each remote ref in
the submodule.
Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
---
submodule.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++----
1 file changed, 55 insertions(+), 4 deletions(-)
diff --git a/submodule.c b/submodule.c
index 6f7d883..b2908fe 100644
--- a/submodule.c
+++ b/submodule.c
@@ -532,19 +532,34 @@ static int submodule_needs_pushing(const char *path, const unsigned char sha1[20
return 0;
}
+static struct sha1_array *submodule_commits(struct string_list *submodules,
+ const char *path)
+{
+ struct string_list_item *item;
+
+ item = string_list_insert(submodules, path);
+ if (item->util)
+ return (struct sha1_array *) item->util;
+
+ /* NEEDSWORK: should we have sha1_array_init()? */
+ item->util = xcalloc(1, sizeof(struct sha1_array));
+ return (struct sha1_array *) item->util;
+}
+
static void collect_submodules_from_diff(struct diff_queue_struct *q,
struct diff_options *options,
void *data)
{
int i;
- struct string_list *needs_pushing = data;
+ struct string_list *submodules = data;
for (i = 0; i < q->nr; i++) {
struct diff_filepair *p = q->queue[i];
+ struct sha1_array *commits;
if (!S_ISGITLINK(p->two->mode))
continue;
- if (submodule_needs_pushing(p->two->path, p->two->oid.hash))
- string_list_insert(needs_pushing, p->two->path);
+ commits = submodule_commits(submodules, p->two->path);
+ sha1_array_append(commits, p->two->oid.hash);
}
}
@@ -560,6 +575,30 @@ static void find_unpushed_submodule_commits(struct commit *commit,
diff_tree_combined_merge(commit, 1, &rev);
}
+struct collect_submodule_from_sha1s_data {
+ char *submodule_path;
+ struct string_list *needs_pushing;
+};
+
+static int collect_submodules_from_sha1s(const unsigned char sha1[20],
+ void *data)
+{
+ struct collect_submodule_from_sha1s_data *me = data;
+
+ if (submodule_needs_pushing(me->submodule_path, sha1))
+ string_list_insert(me->needs_pushing, me->submodule_path);
+
+ return 0;
+}
+
+static void free_submodules_sha1s(struct string_list *submodules)
+{
+ struct string_list_item *item;
+ for_each_string_list_item(item, submodules)
+ sha1_array_clear((struct sha1_array *) item->util);
+ string_list_clear(submodules, 1);
+}
+
int find_unpushed_submodules(unsigned char new_sha1[20],
const char *remotes_name, struct string_list *needs_pushing)
{
@@ -568,6 +607,8 @@ int find_unpushed_submodules(unsigned char new_sha1[20],
const char *argv[] = {NULL, NULL, "--not", "NULL", NULL};
int argc = ARRAY_SIZE(argv) - 1;
char *sha1_copy;
+ struct string_list submodules = STRING_LIST_INIT_DUP;
+ struct string_list_item *submodule;
struct strbuf remotes_arg = STRBUF_INIT;
@@ -581,12 +622,22 @@ int find_unpushed_submodules(unsigned char new_sha1[20],
die("revision walk setup failed");
while ((commit = get_revision(&rev)) != NULL)
- find_unpushed_submodule_commits(commit, needs_pushing);
+ find_unpushed_submodule_commits(commit, &submodules);
reset_revision_walk();
free(sha1_copy);
strbuf_release(&remotes_arg);
+ for_each_string_list_item(submodule, &submodules) {
+ struct collect_submodule_from_sha1s_data data;
+ data.submodule_path = submodule->string;
+ data.needs_pushing = needs_pushing;
+ sha1_array_for_each_unique((struct sha1_array *) submodule->util,
+ collect_submodules_from_sha1s,
+ &data);
+ }
+ free_submodules_sha1s(&submodules);
+
return needs_pushing->nr;
}
--
2.10.1.386.gc503e45
^ permalink raw reply related
* [PATCH v4 0/4] Speedup finding of unpushed submodules
From: Heiko Voigt @ 2016-11-16 15:11 UTC (permalink / raw)
To: Junio C Hamano
Cc: Brandon Williams, git, Jeff King, Stefan Beller, Jens.Lehmann,
Fredrik Gustafsson, Leandro Lucarella
You can find the third iteration of this series here:
http://public-inbox.org/git/cover.1479221071.git.hvoigt@hvoigt.net/
All comments from the last iteration should be addressed.
Cheers Heiko
Heiko Voigt (4):
serialize collection of changed submodules
serialize collection of refs that contain submodule changes
batch check whether submodule needs pushing into one call
submodule_needs_pushing() NEEDSWORK when we can not answer this
question
submodule.c | 123 +++++++++++++++++++++++++++++++++++++++++++++++-------------
submodule.h | 5 ++-
transport.c | 29 ++++++++++----
3 files changed, 121 insertions(+), 36 deletions(-)
--
2.10.1.386.gc503e45
^ permalink raw reply
* Re: [PATCH v1 2/2] travis-ci: disable GIT_TEST_HTTPD for macOS
From: Heiko Voigt @ 2016-11-16 14:39 UTC (permalink / raw)
To: Jeff King
Cc: Lars Schneider, Junio C Hamano, Torsten Bögershausen, git,
Eric Sunshine
In-Reply-To: <20161115153159.mxfl73dnhljad5so@sigill.intra.peff.net>
On Tue, Nov 15, 2016 at 10:31:59AM -0500, Jeff King wrote:
> On Tue, Nov 15, 2016 at 01:07:18PM +0100, Heiko Voigt wrote:
>
> > On Fri, Nov 11, 2016 at 09:22:51AM +0100, Lars Schneider wrote:
> > > To all macOS users on the list:
> > > Does anyone execute the tests with GIT_TEST_HTTPD enabled successfully?
> >
> > Nope. The following tests fail for me on master: 5539, 5540, 5541, 5542,
> > 5550, 5551, 5561, 5812.
>
> Failing how? Does apache fail to start up? Do tests fails? What does
> "-v" say? Is there anything interesting in httpd/error.log in the trash
> directory?
This is what I see for 5539:
$ GIT_TEST_HTTPD=1 ./t5539-fetch-http-shallow.sh -v
Initialized empty Git repository in /Users/hvoigt/Repository/git4/t/trash directory.t5539-fetch-http-shallow/.git/
checking prerequisite: NOT_ROOT
mkdir -p "$TRASH_DIRECTORY/prereq-test-dir" &&
(
cd "$TRASH_DIRECTORY/prereq-test-dir" &&
uid=$(id -u) &&
test "$uid" != 0
)
prerequisite NOT_ROOT ok
httpd: Syntax error on line 65 of /Users/hvoigt/Repository/git4/t/lib-httpd/apache.conf: Cannot load modules/mod_mpm_prefork.so into server: dlopen(/Users/hvoigt/Repository/git4/t/trash directory.t5539-fetch-http-shallow/httpd/modules/mod_mpm_prefork.so, 10): image not found
error: web server setup failed
It seems the other failures have the same cause.
Cheers Heiko
^ permalink raw reply
* Re: [PATCH v3 3/4] batch check whether submodule needs pushing into one call
From: Heiko Voigt @ 2016-11-16 14:29 UTC (permalink / raw)
To: Stefan Beller
Cc: Junio C Hamano, git@vger.kernel.org, Jeff King, Jens Lehmann,
Fredrik Gustafsson, Leandro Lucarella
In-Reply-To: <CAGZ79kZtQG5pE-6N-1yZKA95VRmrDRB3PnSd1gtqBM9fxD48Cg@mail.gmail.com>
On Tue, Nov 15, 2016 at 02:28:31PM -0800, Stefan Beller wrote:
> On Tue, Nov 15, 2016 at 6:56 AM, Heiko Voigt <hvoigt@hvoigt.net> wrote:
>
> > -static int submodule_needs_pushing(const char *path, const unsigned char sha1[20])
> > +static int check_has_commit(const unsigned char sha1[20], void *data)
> > {
> > - if (add_submodule_odb(path) || !lookup_commit_reference(sha1))
> > + int *has_commit = (int *) data;
>
> nit: just as prior patches ;) void* can be cast implicitly.
Even though its just a nit: Will remove all the void casts. :)
Cheers Heiko
^ permalink raw reply
* Re: [PATCH v3 4/4] submodule_needs_pushing() NEEDSWORK when we can not answer this question
From: Heiko Voigt @ 2016-11-16 14:26 UTC (permalink / raw)
To: Junio C Hamano
Cc: Stefan Beller, git@vger.kernel.org, Jeff King, Jens Lehmann,
Fredrik Gustafsson, Leandro Lucarella
In-Reply-To: <xmqqtwb8s2m8.fsf@gitster.mtv.corp.google.com>
On Tue, Nov 15, 2016 at 04:13:51PM -0800, Junio C Hamano wrote:
> Stefan Beller <sbeller@google.com> writes:
>
> >> "We do not know" ...
> >
> > ... because there is no way to check for us as we don't have the
> > submodule commits.
> >
> > " We do consider it safe as no one in their sane mind would
> > have changed the submodule pointers without having the
> > submodule around. If a user did however change the submodules
> > without having the submodule commits around, this indicates an
> > expert who knows what they were doing."
>
> I didn't think it through myself to arrive at such a conclusion, but
> to me the above sounds like a sensible reasoning [*1*].
I think you have a point here. If I rephrase it like this: "We do
consider it safe as no one in their sane mind *could* have changed the
submodule pointers without having the submodule around..."
Since its actually hard to create such a situation without the submodule
commit around I agree here.
> *1* My version was more like "we do not know if they would get into
> a situation where they do not have enough submodule commits if
> we pushed our superproject, but more importantly, we DO KNOW
> that it would not help an iota if we pushed our submodule to
> them, so there is no point stopping the push of superproject
> saying 'no, no, no, you must push the submodule first'".
Yes saying that would be wrong. I was rather suggesting that we tell the
user that we could not find the submodule commits to and that if he
wants to proceed he should either pass --recurse-submodules=no or
initialize the submodule.
But I think the above reasoning obsoletes my suggestion. I would adjust
the comment accordingly but still keep the patch so we have
documentation that this behavior is on purpose.
Cheers Heiko
^ permalink raw reply
* Re: [RFH] limiting ref advertisements
From: Duy Nguyen @ 2016-11-16 13:42 UTC (permalink / raw)
To: Jeff King; +Cc: Git Mailing List
In-Reply-To: <20161114212122.rkgeoh4rj5cxdadf@sigill.intra.peff.net>
On Tue, Nov 15, 2016 at 4:21 AM, Jeff King <peff@peff.net> wrote:
> Thanks for responding to this.
Glad to help (or more precisely annoy you somewhat :D)
> I've been meaning to get back to it with
> some code experiments, but they keep getting bumped down in priority. So
> let me at least outline some of my thoughts, without code. :)
>
> I was hoping to avoid right-anchoring because it's expensive to find all
> of the right-anchored cases (assuming that ref storage is generally
> hierarchical, which it is now and probably will be for future backends).
Urgh.. I completely forgot about future refs backends. Yeah this would
kick both wildmatch and right-anchoring out of the window.
For the record I almost suggested BPF as well (to keep the server side
simple, but at the super high cost of client side). That would also go
out of the window the same way wildmatch and right-anchoring does.
> But remember that these are "early capabilities", before the server
> has spoken at all. So the client doesn't know if we can handle v2.
> So we have to send _both_ (and v2-aware servers can ignore the v1).
>
> advertise-lookup-v1=master
> advertise-lookup-v2=master
>
> But that's not quite enough. A v1 server won't look in refs/notes
> at all. So we have to say that, too:
>
> advertise-lookup-v1=refs/notes/master
>
> And of course the v1 server has no idea that this isn't necessary
> if we already found refs/heads/master.
We discussed a bit about upgrading upload-pack version 1 to 2 in more
than one session: the first fetch just does v1 as normal, the server
returns v1 response to but also advertises that v2 is supported. The
client keeps this info and skips v1 and tries v2 right away in the
following fetches, falling back to v1 (new fetch session) if v2 is
unsupported. Can it work the same way here too?
I'm in favor of this option 2 (without trying to be absolute backward
compatible with older lookup versions) since it allows us to optimize
for common case and experiment a bit. Once we know better we can make
the next version that hopefully suits everybody.
> So I think you really do need the client to be able to say "also
> look at this pattern".
What about the order of patterns? Does it matter "this pattern" is in
the middle or the end of the pattern list? I suppose not, just
checking...
But does this call for the ability to remove a pattern from the
pattern list as well, as a way to narrow down the search scope and
avoid sending unwanted refs?
> Of course we do still want left-anchoring, too. Wildcards like
> "refs/heads/*" are always left-anchored. So I think we'd have two types,
> and a full request for
>
> git fetch origin +refs/heads/*:refs/remotes/origin/* master:foo
>
> would look like:
>
> (1) advertise-pattern-v1
> (2) advertise-pattern=refs/notes/%s
> (3) advertise-prefix=refs/heads
> (4) advertise-lookup=master
>
> where the lines mean:
>
> 1. Use the standard v1 patterns (we could spell them out, but this
> just saves bandwidth. In fact, it could just be implicit that v1
> patterns are included, and we could skip this line).
>
> 2. This is for our fictional future version where the client knows
> added refs/notes/* to its DWIM but the server hasn't yet.
>
> 3. Give me all of refs/heads/*
>
> 4. Look up "master" using the advertise patterns and give me the first
> one you find.
Well.. it sounds good to me. But I would not trust myself on refs matters :D
> So given that we can omit (1), and that (2) is just an example for the
> future, it could look like:
>
> advertise-prefix=refs/heads
> advertise-lookup=master
>
> which is pretty reasonable. It's not _completely_ bulletproof in terms
> of backwards compatibility. The "v1" thing means the client can't insert
> a new pattern in the middle (remember they're ordered by priority).
OK so pattern order probably matters...
> So
> maybe it is better to spell them all out (one thing that makes me
> hesitate is that these will probably end up as URL parameters for the
> HTTP version, which means our URL can start to get a little long).
>
> Anyway. That's the direction I'm thinking. I haven't written the code
> yet. The trickiest thing will probably be that the server would want to
> avoid advertising the same ref twice via two mechanisms (or perhaps the
> client just be tolerant of duplicates; that relieves the server of any
> duplicate-storage requirements).
Thanks for sharing.
--
Duy
^ permalink raw reply
* Re: [PATCH] worktree: fix a sparse 'Using plain integer as NULL pointer' warning
From: Duy Nguyen @ 2016-11-16 13:17 UTC (permalink / raw)
To: Ramsay Jones; +Cc: Junio C Hamano, GIT Mailing-list
In-Reply-To: <5b7d7d0b-8a6c-d516-4eb9-4e4ea13dce73@ramsayjones.plus.com>
On Wed, Nov 16, 2016 at 3:28 AM, Ramsay Jones
<ramsay@ramsayjones.plus.com> wrote:
>
> Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
> ---
>
> Hi Duy,
>
> If you need to re-roll your 'nd/worktree-move' branch, could you
> please squash this into the relevant patch [commit c49e92f5c
> ("worktree move: refuse to move worktrees with submodules", 12-11-2016)].
Will do, thanks (and it's also "thanks" for your other similar emails,
I just don't want to send a mail with just 'thanks' that adds nothing
else).
> Also, one of the new tests introduced by commit 31a8f3066 ("worktree move:
> new command", 12-11-2016), fails for me, thus:
>
> $ ./t2028-worktree-move.sh -i -v
> ...
> --- expected 2016-11-15 20:22:50.647241458 +0000
> +++ actual 2016-11-15 20:22:50.647241458 +0000
> @@ -1,3 +1,3 @@
> worktree /home/ramsay/git/t/trash directory.t2028-worktree-move
> -worktree /home/ramsay/git/t/trash directory.t2028-worktree-move/destination
> worktree /home/ramsay/git/t/trash directory.t2028-worktree-move/elsewhere
> +worktree /home/ramsay/git/t/trash directory.t2028-worktree-move/destination
> not ok 12 - move worktree
> #
> # git worktree move source destination &&
> # test_path_is_missing source &&
> # git worktree list --porcelain | grep "^worktree" >actual &&
> # cat <<-EOF >expected &&
> # worktree $TRASH_DIRECTORY
> # worktree $TRASH_DIRECTORY/destination
> # worktree $TRASH_DIRECTORY/elsewhere
> # EOF
> # test_cmp expected actual &&
> # git -C destination log --format=%s >actual2 &&
> # echo init >expected2 &&
> # test_cmp expected2 actual2
> #
> $
>
> Is there an expectation that the submodules will be listed in
> any particular order by 'git worktree list --porcelain' ?
I just sent a patch [1] to fix this before reading this mail. The
order so far has been determined by readdir() which is not great.
[1] https://public-inbox.org/git/CACsJy8DOT_4N_48UaoYK61G_8JUaXbEs7N=n24CH2q1GN=++5g@mail.gmail.com/T/#mfcf797219a1a143ed2ac45198015f19e82c70db2
--
Duy
^ permalink raw reply
* Re: Git status takes too long- How to improve the performance of git
From: Fredrik Gustafsson @ 2016-11-16 13:21 UTC (permalink / raw)
To: Renuka Pampana; +Cc: Fredrik Gustafsson, git
In-Reply-To: <CAEAva_1JAu+kWmk3MZDFK=4CgQB5M+JN8FwzMVr6zKgXTAhdXw@mail.gmail.com>
On Wed, Nov 16, 2016 at 05:13:57PM +0530, Renuka Pampana wrote:
> > On Tue, Nov 15, 2016 at 02:33:12AM -0700, ravalika wrote:
> > > It is an centralized server and git status takes too long
> >
> > A centralized server? How? git is designed to be runned locally. If
> > you're running git on a network file system, the performance will
> > suffer. Could you elaborate on how your environment is setup?
> >
> >
> We have setup main git repository in remote location on Linux server
> And created a git repository in local Linux server, as a reference for the
> remote git repository,
> And update the local git repository for every 15 min in local server
>
> Users will be able to access the local git repository through NFS
And each user will have their own copy of the repository locally on
their machine? That is having done a git clone?
>
> All users will clone the git repository from remote project url by using
> local git repo as reference
>
> For example : git clone --reference ~/gitcaches/reference user@drupal
> :/home/project/drupal.git
>
> All the users have ssh credentials for the remote server
Why are you using --reference for a 8.9MB big clone?
>
>
> What is the best way to implement remote git repo and able to access the
> git repo from other location, without any performance glitches?
> Users should be able to access git repo from different servers and from
> different locations.
The best way is to have it locally cloned. Yes the initial clone will be
expensive but operations after that will be fairly smooth. You do not(!)
want to execute git on one machine and having the repository beeing on
an other machine (for example via a network file system, except git
clone, git fetch, git push, etc.).
> >
> > > How to improve the performance of git status
> > >
> > > Git repo details:
> > >
> > > Size of the .git folder is 8.9MB
> > > Number of commits approx 53838 (git rev-list HEAD --count)
> > > Number of branches - 330
> > > Number of files - 63883
> > > Working tree clone size is 4.3GB
> >
> > .git folder of 8.9 MEGABYTE and working tree of 4.3 GIGABYTE? Is this a
> > typo?
> >
> > All git related information is stored in .git directory of the working
> directory
> It is 8.9M
> And size of the local workspace is 4.3G
Can you please elaborate on this? How can you store 8.9 MB of data that
will result in a 4.3 G workspace?
--
Fredrik Gustafsson
phone: +46 733-608274
e-mail: iveqy@iveqy.com
website: http://www.iveqy.com
^ permalink raw reply
* Re: [PATCH 00/11] git worktree (re)move
From: Duy Nguyen @ 2016-11-16 13:11 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Git Mailing List
In-Reply-To: <20161116130533.GA5393@ash>
On Wed, Nov 16, 2016 at 8:05 PM, Duy Nguyen <pclouds@gmail.com> wrote:
> diff --git a/worktree.c b/worktree.c
> index f7869f8..fe92d6f 100644
> --- a/worktree.c
> +++ b/worktree.c
> @@ -173,6 +173,13 @@ static void mark_current_worktree(struct worktree **worktrees)
> free(git_dir);
> }
>
> +static int compare_worktree(const void *a_, const void *b_)
> +{
> + const struct worktree *const *a = a_;
> + const struct worktree *const *b = b_;
> + return fspathcmp((*a)->path, (*b)->path);
> +}
> +
> struct worktree **get_worktrees(void)
> {
> struct worktree **list = NULL;
> @@ -205,6 +212,11 @@ struct worktree **get_worktrees(void)
> ALLOC_GROW(list, counter + 1, alloc);
> list[counter] = NULL;
>
> + /*
> + * don't sort the first item (main worktree), which will
> + * always be the first
> + */
Urgh.. I should review my patches more carefully before sending out :(
The main worktree could be missing (failing to parse HEAD) so I need a
better trick than simply assuming the first item is the main worktree
here. Tests did not catch this, naturally..
> + qsort(list + 1, counter - 1, sizeof(*list), compare_worktree);
> mark_current_worktree(list);
> return list;
> }
--
Duy
^ permalink raw reply
* Re: [PATCH 00/11] git worktree (re)move
From: Duy Nguyen @ 2016-11-16 13:05 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <xmqqa8d4yts7.fsf@gitster.mtv.corp.google.com>
On Sat, Nov 12, 2016 at 06:53:44PM -0800, Junio C Hamano wrote:
> not ok 12 - move worktree
> #
> # git worktree move source destination &&
> # test_path_is_missing source &&
> # git worktree list --porcelain | grep "^worktree" >actual &&
> # cat <<-EOF >expected &&
> # worktree $TRASH_DIRECTORY
> # worktree $TRASH_DIRECTORY/destination
> # worktree $TRASH_DIRECTORY/elsewhere
> # EOF
> # test_cmp expected actual &&
> # git -C destination log --format=%s >actual2 &&
> # echo init >expected2 &&
> # test_cmp expected2 actual2
I think I've seen this (i.e. 'expected' and 'actual' differ only in
the order of items) once after a rebase and ignored it, assuming
something was changed during the rebase that caused this.
The following patch should fix it if that's the same thing you saw. I
could pile it on worktree-move series, or you can make it a separate
one-patch series. What's your preference?
-- 8< --
Subject: [PATCH] worktree list: keep the list sorted
It makes it easier to write tests for. But it should also be good for
the user since locating a worktree by eye would be easier once they
notice this.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
worktree.c | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/worktree.c b/worktree.c
index f7869f8..fe92d6f 100644
--- a/worktree.c
+++ b/worktree.c
@@ -173,6 +173,13 @@ static void mark_current_worktree(struct worktree **worktrees)
free(git_dir);
}
+static int compare_worktree(const void *a_, const void *b_)
+{
+ const struct worktree *const *a = a_;
+ const struct worktree *const *b = b_;
+ return fspathcmp((*a)->path, (*b)->path);
+}
+
struct worktree **get_worktrees(void)
{
struct worktree **list = NULL;
@@ -205,6 +212,11 @@ struct worktree **get_worktrees(void)
ALLOC_GROW(list, counter + 1, alloc);
list[counter] = NULL;
+ /*
+ * don't sort the first item (main worktree), which will
+ * always be the first
+ */
+ qsort(list + 1, counter - 1, sizeof(*list), compare_worktree);
mark_current_worktree(list);
return list;
}
--
2.8.2.524.g6ff3d78
-- 8< --
^ permalink raw reply related
* Re: RFC: Enable delayed responses to Git clean/smudge filter requests
From: Lars Schneider @ 2016-11-16 9:53 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Eric Wong, git
In-Reply-To: <xmqqvavotych.fsf@gitster.mtv.corp.google.com>
On 15 Nov 2016, at 19:03, Junio C Hamano <gitster@pobox.com> wrote:
> Lars Schneider <larsxschneider@gmail.com> writes:
>
>>> The filter itself would need to be aware of parallelism
>>> if it lives for multiple objects, right?
>>
>> Correct. This way Git doesn't need to deal with threading...
>
> I think you need to be careful about three things (at least; there
> may be more):
>
> * Codepaths that check out multiple cache entries do rely on the
> order of checkout. We checkout removals first to make room so
> that creation of a path X can succeed if an existing path X/Y
> that used to want to see X as a directory can succeed (see the
> use of checkout_entry() by "git checkout", which does have two
> separate loops to explicitly guarantee this), for example. I
> think "remove all and then create" you do not specifically have
> to worry about with the proposed change, but you may need to
> inspect and verify there aren't other kind of order dependency.
OK
> * Done naively, it will lead to unmaintainable code, like this:
>
> + struct list_of_cache_entries *list = ...;
> for (i = 0; i < active_nr; i++)
> - checkout_entry(active_cache[i], state, NULL);
> + if (checkout_entry(active_cache[i], state, NULL) == DELAYED)
> + add_cache_to_queue(&list, active_cache[i]);
> + while (list) {
> + wait_for_checkout_to_finish(*list);
> + list = list->next;
> + }
>
> I do not think we want to see such a rewrite all over the
> codepaths. It might be OK to add such a "these entries are known
> to be delayed" list in struct checkout so that the above becomes
> more like this:
>
> for (i = 0; i < active_nr; i++)
> checkout_entry(active_cache[i], state, NULL);
> + checkout_entry_finish(state);
>
> That is, addition of a single "some of the checkout_entry() calls
> done so far might have been lazy, and I'll give them a chance to
> clean up" might be palatable. Anything more than that on the
> caller side is not.
I haven't thought hard about the implementation, yet, but I'll try
to stick to your suggestion and change as less code as possible on
the caller sides.
> * You'd need to rein in the maximum parallelism somehow, as you do
> not want to see hundreds of competing filter processes starting
> only to tell the main loop over an index with hundreds of entries
> that they are delayed checkouts.
I intend to implement this feature only for the new long running filter
process protocol. OK with you?
Thanks,
Lars
^ permalink raw reply
* Re: [RFC/PATCH 0/2] git diff <(command1) <(command2)
From: Johannes Schindelin @ 2016-11-16 9:50 UTC (permalink / raw)
To: Junio C Hamano
Cc: Michael J Gruber, Jacob Keller, Dennis Kaarsemaker,
Git mailing list
In-Reply-To: <xmqqtwb9wywp.fsf@gitster.mtv.corp.google.com>
Hi Junio,
On Mon, 14 Nov 2016, Junio C Hamano wrote:
> I _think_ the no-index mode was primarily for those who want to use
> our diff as a replacement for GNU and other diffs, and from that
> point of view, I'd favour not doing the "comparing symbolic link?
> We'll show the difference between the link contents, not target"
> under no-index mode myself.
If I read this correctly, then we are in agreement that the default for
--no-index should be as it is right now, i.e. comparing symlink targets as
opposed to --follow-links.
> That is a lot closer to the diff other people implemented, not ours.
> Hence the knee-jerk reaction I gave in
>
> http://public-inbox.org/git/xmqqinrt1zcx.fsf@gitster.mtv.corp.google.com
Let me quote the knee-jerk reaction:
> My knee-jerk reaction is:
>
> * The --no-index mode should default to your --follow-symlinks
> behaviour, without any option to turn it on or off.
But this is the exact opposite of what I find reasonable.
Ciao,
Johannes
^ permalink raw reply
* Re: [PATCH] t0021, t5615: use $PWD instead of $(pwd) in PATH-like shell variables
From: Johannes Schindelin @ 2016-11-16 9:47 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Lars Schneider, Johannes Sixt, git, Jeff King
In-Reply-To: <alpine.DEB.2.20.1611151753300.3746@virtualbox>
Hi,
On Tue, 15 Nov 2016, Johannes Schindelin wrote:
> On Mon, 14 Nov 2016, Junio C Hamano wrote:
>
> > Dscho's mention of 'still times out' may be an indiciation that
> > something unspecified on 'pu' is not ready to be merged to 'next',
> > but blocking all of 'pu' with a blanket statement is not useful,
> > and that was where my response comes from.
>
> Until the time when the test suite takes less than the insane three hours
> to run, I am afraid that a blanket statement on `pu` is the best I can do.
Well, I should add that the test suite does not take 3 hours to run for
`pu` these days.
It used to time out after four hours until two days ago (I think; I was a
bit too busy with other CI work to pay close attention to the constantly
failing `pu` job, with quite a few failing `next`s and even a couple of
failing `master`s thrown in).
As of two days ago, the test suite takes no time at all. The build already
fails (which makes me wonder why a couple of patch series I contributed
had such a hard time getting into `pu` when they compiled and tested
just fine, whereas some obviously non-building stuff gets into `pu`
already, makes no sense to me).
This is the offending part from last night's build:
-- snipsnap --
2016-11-16T00:31:57.5321220Z copy.c: In function 'copy_dir_1':
2016-11-16T00:31:57.5321220Z copy.c:369:8: error: implicit declaration of function 'lchown' [-Werror=implicit-function-declaration]
2016-11-16T00:31:57.5321220Z if (lchown(dest, source_stat.st_uid, source_stat.st_gid) < 0)
2016-11-16T00:31:57.5321220Z ^~~~~~
2016-11-16T00:31:57.5321220Z copy.c:391:7: error: implicit declaration of function 'mknod' [-Werror=implicit-function-declaration]
2016-11-16T00:31:57.5321220Z if (mknod(dest, source_stat.st_mode, source_stat.st_rdev) < 0)
2016-11-16T00:31:57.5321220Z ^~~~~
2016-11-16T00:31:57.5321220Z copy.c:405:7: error: implicit declaration of function 'utimes' [-Werror=implicit-function-declaration]
2016-11-16T00:31:57.5321220Z if (utimes(dest, times) < 0)
2016-11-16T00:31:57.5321220Z ^~~~~~
2016-11-16T00:31:57.5321220Z copy.c:407:7: error: implicit declaration of function 'chown' [-Werror=implicit-function-declaration]
2016-11-16T00:31:57.5321220Z if (chown(dest, source_stat.st_uid, source_stat.st_gid) < 0) {
2016-11-16T00:31:57.5321220Z ^~~~~
2016-11-16T00:31:57.7982432Z CC ctype.o
2016-11-16T00:31:58.1418929Z cc1.exe: all warnings being treated as errors
2016-11-16T00:31:58.6368128Z make: *** [Makefile:1988: copy.o] Error 1
^ permalink raw reply
* Re: gitweb html validation
From: Raphaël Gertz @ 2016-11-16 9:25 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Ralf Thielow, Jakub Narębski, git
In-Reply-To: <xmqqy40ks301.fsf@gitster.mtv.corp.google.com>
Le 16.11.2016 01:05, Junio C Hamano a écrit :
> Ralf Thielow <ralf.thielow@gmail.com> writes:
>
>> Only block level elements are
>> allowed to be inside form tags, according to
>> https://www.w3.org/2010/04/xhtml10-strict.html#elem_form
>> ...
>> I think it's better to just move the <form>-Tag outside of the
>> surrounding div?
>> Something like this perhaps, I didn't test it myself yet.
>
> That sounds like a sensible update to me (no, I do not run gitweb
> myself). Is this the only <form> we have in the UI, or is it the
> only one that is problematic?
>
There is an other form in the cgi line 4110 :
print $cgi->start_form(-method => "get", -action => $action) .
"<div class=\"search\">\n" .
But this one has a <div class="search"> inside.
The problem with projsearch I want to change is that the div is around
the form without a container inside.
I agree with moving the <div class="projsearch"> inside the form if it's
a better option.
Best regards
^ permalink raw reply
* Re: [PATCH v7 13/17] ref-filter: add `:dir` and `:base` options for ref printing atoms
From: Karthik Nayak @ 2016-11-16 7:58 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Jacob Keller, Git mailing list
In-Reply-To: <xmqq4m38vdw4.fsf@gitster.mtv.corp.google.com>
On Tue, Nov 15, 2016 at 11:12 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Jacob Keller <jacob.keller@gmail.com> writes:
>
>> dirname makes sense. What about implementing a reverse variant of
>> strip, which you could perform stripping of right-most components and
>> instead of stripping by a number, strip "to" a number, ie: keep the
>> left N most components, and then you could use something like
>> ...
>> I think that would be more general purpose than basename, and less confusing?
>
> I think you are going in the right direction. I had a similar
> thought but built around a different axis. I.e. if strip=1 strips
> one from the left, perhaps we want to have rstrip=1 that strips one
> from the right, and also strip=-1 to mean strip everything except
> one from the left and so on?. I think this and your keep (and
> perhaps you'll have rkeep for completeness) have the same expressive
> power. I do not offhand have a preference one over the other.
>
If we do implement strip with negative numbers, it definitely would be
neat, but to get
the desired feature which I've mentioned below, we'd need to call
strip twice, i.e
to get remotes from /refs/foo/abc/xyz we'd need to do
strip=1,strip=-1, which could be
done but would need a lot of tweaking, since we have the same
subatom/option having
multiple values.
On the other hand it would be easier maybe to just introduce rstrip,
where we strip from
right and ensure that strip and rstrip can be used together.
On Wed, Nov 16, 2016 at 2:49 AM, Jacob Keller <jacob.keller@gmail.com> wrote:
> On November 15, 2016 9:42:03 AM PST, Junio C Hamano <gitster@pobox.com> wrote:
>>Somehow it sounds a bit strange to me to treat 'remotes' as the same
>>class of token as 'heads' and 'tags' (I'd expect 'heads' and
>>'remotes/origin' would be at the same level in end-user's mind), but
>>that is probably an unrelated tangent. The reason this series wants
>>to introduce :base must be to emulate an existing feature, so that
>>existing feature is a concrete counter-example that argues against
>>my "it sounds a bit strange" reaction.
>
> It may be a bit strange indeed. What is the requirement for this?
>
> I think implementing a strip and rstrip ( if necessary ) with negative numbers would be most ideal.
The necessity is that we need to do different formatting as per the
ref type in branch -l. If you see the
last but one patch, we do
strbuf_addf(&fmt,
"%%(if:notequals=remotes)%%(refname:base)%%(then)%s%%(else)%s%%(end)",
local.buf, remote.buf);
where its using ':base' to check for the ref type and do conditional
printing along with the %(if)...%(end) atoms.
--
Regards,
Karthik Nayak
^ permalink raw reply
* Re: Bug with disabling compression and 'binary' files.
From: Junio C Hamano @ 2016-11-16 5:23 UTC (permalink / raw)
To: Douglas Cox; +Cc: git
In-Reply-To: <CA+i4re4FZhshheftQnsogaY7601wgf_GfsFUxy+doLcOYPWv7Q@mail.gmail.com>
Douglas Cox <ziflin@gmail.com> writes:
>> This may or may not be related to the symptom
>> you are observing (if it is, then you would see a packfile created
>> in objects/pack/, not in loose objects in object/??/ directories).
>
> No, the file is loose (it's in .git/objects/eb in this case). This is
> seen immediately after the add, though I believe it's the same way
> when doing a commit on a changed file.
Then I do not have a guess as to where the symptom you are seeing is
coming from.
^ permalink raw reply
* [PATCH v2] compression: unify pack.compression configuration parsing
From: Junio C Hamano @ 2016-11-16 5:21 UTC (permalink / raw)
To: git
In-Reply-To: <xmqqfumsrvge.fsf@gitster.mtv.corp.google.com>
There are three codepaths that use a variable whose name is
pack_compression_level to affect how objects and deltas sent to a
packfile is compressed. Unlike zlib_compression_level that controls
the loose object compression, however, this variable was static to
each of these codepaths. Two of them read the pack.compression
configuration variable, using core.compression as the default, and
one of them also allowed overriding it from the command line.
The other codepath in bulk-checkin did not pay any attention to the
configuration.
Unify the configuration parsing to git_default_config(), where we
implement the parsing of core.loosecompression and core.compression
and make the former override the latter, by moving code to parse
pack.compression and also allow core.compression to give default to
this variable.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
* Now comes with new tests to cover pack-objects and fast-import,
which should not fail with or without code change because they
are only here to prevent regression with this change.
On the code side, *_compression_seen variables are also made
private to config.c, as these variables are used only by the
config reader to implement "core.compression gives the default to
*.compression" logic and the code is consolidated to this single
file.
builtin/pack-objects.c | 14 --------
bulk-checkin.c | 2 --
cache.h | 2 +-
config.c | 16 +++++++++
environment.c | 2 +-
fast-import.c | 13 -------
t/t1050-large.sh | 29 ++++++++++++++++
t/t5315-pack-objects-compression.sh | 44 ++++++++++++++++++++++++
t/t9303-fast-import-compression.sh | 67 +++++++++++++++++++++++++++++++++++++
9 files changed, 158 insertions(+), 31 deletions(-)
create mode 100755 t/t5315-pack-objects-compression.sh
create mode 100755 t/t9303-fast-import-compression.sh
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 0fd52bd6b4..8841f8b366 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -61,8 +61,6 @@ static int delta_search_threads;
static int pack_to_stdout;
static int num_preferred_base;
static struct progress *progress_state;
-static int pack_compression_level = Z_DEFAULT_COMPRESSION;
-static int pack_compression_seen;
static struct packed_git *reuse_packfile;
static uint32_t reuse_packfile_objects;
@@ -2368,16 +2366,6 @@ static int git_pack_config(const char *k, const char *v, void *cb)
depth = git_config_int(k, v);
return 0;
}
- if (!strcmp(k, "pack.compression")) {
- int level = git_config_int(k, v);
- if (level == -1)
- level = Z_DEFAULT_COMPRESSION;
- else if (level < 0 || level > Z_BEST_COMPRESSION)
- die("bad pack compression level %d", level);
- pack_compression_level = level;
- pack_compression_seen = 1;
- return 0;
- }
if (!strcmp(k, "pack.deltacachesize")) {
max_delta_cache_size = git_config_int(k, v);
return 0;
@@ -2869,8 +2857,6 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
reset_pack_idx_option(&pack_idx_opts);
git_config(git_pack_config, NULL);
- if (!pack_compression_seen && core_compression_seen)
- pack_compression_level = core_compression_level;
progress = isatty(2);
argc = parse_options(argc, argv, prefix, pack_objects_options,
diff --git a/bulk-checkin.c b/bulk-checkin.c
index 4347f5c76a..991b4a13e2 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -7,8 +7,6 @@
#include "pack.h"
#include "strbuf.h"
-static int pack_compression_level = Z_DEFAULT_COMPRESSION;
-
static struct bulk_checkin_state {
unsigned plugged:1;
diff --git a/cache.h b/cache.h
index a50a61a197..b14c0e8e38 100644
--- a/cache.h
+++ b/cache.h
@@ -670,7 +670,7 @@ extern const char *git_attributes_file;
extern const char *git_hooks_path;
extern int zlib_compression_level;
extern int core_compression_level;
-extern int core_compression_seen;
+extern int pack_compression_level;
extern size_t packed_git_window_size;
extern size_t packed_git_limit;
extern size_t delta_base_cache_limit;
diff --git a/config.c b/config.c
index 83fdecb1bc..2f1aef742e 100644
--- a/config.c
+++ b/config.c
@@ -66,6 +66,8 @@ static struct key_value_info *current_config_kvi;
*/
static enum config_scope current_parsing_scope;
+static int core_compression_seen;
+static int pack_compression_seen;
static int zlib_compression_seen;
/*
@@ -865,6 +867,8 @@ static int git_default_core_config(const char *var, const char *value)
core_compression_seen = 1;
if (!zlib_compression_seen)
zlib_compression_level = level;
+ if (!pack_compression_seen)
+ pack_compression_level = level;
return 0;
}
@@ -1125,6 +1129,18 @@ int git_default_config(const char *var, const char *value, void *dummy)
pack_size_limit_cfg = git_config_ulong(var, value);
return 0;
}
+
+ if (!strcmp(var, "pack.compression")) {
+ int level = git_config_int(var, value);
+ if (level == -1)
+ level = Z_DEFAULT_COMPRESSION;
+ else if (level < 0 || level > Z_BEST_COMPRESSION)
+ die(_("bad pack compression level %d"), level);
+ pack_compression_level = level;
+ pack_compression_seen = 1;
+ return 0;
+ }
+
/* Add other config variables here and to Documentation/config.txt. */
return 0;
}
diff --git a/environment.c b/environment.c
index 0935ec696e..4bce3eebfa 100644
--- a/environment.c
+++ b/environment.c
@@ -34,7 +34,7 @@ const char *git_attributes_file;
const char *git_hooks_path;
int zlib_compression_level = Z_BEST_SPEED;
int core_compression_level;
-int core_compression_seen;
+int pack_compression_level = Z_DEFAULT_COMPRESSION;
int fsync_object_files;
size_t packed_git_window_size = DEFAULT_PACKED_GIT_WINDOW_SIZE;
size_t packed_git_limit = DEFAULT_PACKED_GIT_LIMIT;
diff --git a/fast-import.c b/fast-import.c
index cb545d7df5..f561ba833b 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -284,8 +284,6 @@ static unsigned long max_depth = 10;
static off_t max_packsize;
static int unpack_limit = 100;
static int force_update;
-static int pack_compression_level = Z_DEFAULT_COMPRESSION;
-static int pack_compression_seen;
/* Stats and misc. counters */
static uintmax_t alloc_count;
@@ -3381,15 +3379,6 @@ static void git_pack_config(void)
if (max_depth > MAX_DEPTH)
max_depth = MAX_DEPTH;
}
- if (!git_config_get_int("pack.compression", &pack_compression_level)) {
- if (pack_compression_level == -1)
- pack_compression_level = Z_DEFAULT_COMPRESSION;
- else if (pack_compression_level < 0 ||
- pack_compression_level > Z_BEST_COMPRESSION)
- git_die_config("pack.compression",
- "bad pack compression level %d", pack_compression_level);
- pack_compression_seen = 1;
- }
if (!git_config_get_int("pack.indexversion", &indexversion_value)) {
pack_idx_opts.version = indexversion_value;
if (pack_idx_opts.version > 2)
@@ -3454,8 +3443,6 @@ int cmd_main(int argc, const char **argv)
setup_git_directory();
reset_pack_idx_option(&pack_idx_opts);
git_pack_config();
- if (!pack_compression_seen && core_compression_seen)
- pack_compression_level = core_compression_level;
alloc_objects(object_entry_alloc);
strbuf_init(&command_buf, 0);
diff --git a/t/t1050-large.sh b/t/t1050-large.sh
index 096dbffecc..6fd264cff0 100755
--- a/t/t1050-large.sh
+++ b/t/t1050-large.sh
@@ -5,6 +5,12 @@ test_description='adding and checking out large blobs'
. ./test-lib.sh
+# This should be moved to test-lib.sh together with the
+# copy in t0021 after both topics have graduated to 'master'.
+file_size () {
+ perl -e 'print -s $ARGV[0]' "$1"
+}
+
test_expect_success setup '
# clone does not allow us to pass core.bigfilethreshold to
# new repos, so set core.bigfilethreshold globally
@@ -17,6 +23,29 @@ test_expect_success setup '
export GIT_ALLOC_LIMIT
'
+# add a large file with different settings
+while read expect config
+do
+ test_expect_success "add with $config" '
+ test_when_finished "rm -f .git/objects/pack/pack-*.* .git/index" &&
+ git $config add large1 &&
+ sz=$(file_size .git/objects/pack/pack-*.pack) &&
+ case "$expect" in
+ small) test "$sz" -le 100000 ;;
+ large) test "$sz" -ge 100000 ;;
+ esac
+ '
+done <<\EOF
+large -c core.compression=0
+small -c core.compression=9
+large -c core.compression=0 -c pack.compression=0
+large -c core.compression=9 -c pack.compression=0
+small -c core.compression=0 -c pack.compression=9
+small -c core.compression=9 -c pack.compression=9
+large -c pack.compression=0
+small -c pack.compression=9
+EOF
+
test_expect_success 'add a large file or two' '
git add large1 huge large2 &&
# make sure we got a single packfile and no loose objects
diff --git a/t/t5315-pack-objects-compression.sh b/t/t5315-pack-objects-compression.sh
new file mode 100755
index 0000000000..34c47dae09
--- /dev/null
+++ b/t/t5315-pack-objects-compression.sh
@@ -0,0 +1,44 @@
+#!/bin/sh
+
+test_description='pack-object compression configuration'
+
+. ./test-lib.sh
+
+# This should be moved to test-lib.sh together with the
+# copy in t0021 after both topics have graduated to 'master'.
+file_size () {
+ perl -e 'print -s $ARGV[0]' "$1"
+}
+
+test_expect_success setup '
+ printf "%2000000s" X |
+ git hash-object -w --stdin >object-name &&
+ # make sure it resulted in a loose object
+ ob=$(sed -e "s/\(..\).*/\1/" object-name) &&
+ ject=$(sed -e "s/..\(.*\)/\1/" object-name) &&
+ test -f .git/objects/$ob/$ject
+'
+
+while read expect config
+do
+ test_expect_success "pack-objects with $config" '
+ test_when_finished "rm -f pack-*.*" &&
+ git $config pack-objects pack <object-name &&
+ sz=$(file_size pack-*.pack) &&
+ case "$expect" in
+ small) test "$sz" -le 100000 ;;
+ large) test "$sz" -ge 100000 ;;
+ esac
+ '
+done <<\EOF
+large -c core.compression=0
+small -c core.compression=9
+large -c core.compression=0 -c pack.compression=0
+large -c core.compression=9 -c pack.compression=0
+small -c core.compression=0 -c pack.compression=9
+small -c core.compression=9 -c pack.compression=9
+large -c pack.compression=0
+small -c pack.compression=9
+EOF
+
+test_done
diff --git a/t/t9303-fast-import-compression.sh b/t/t9303-fast-import-compression.sh
new file mode 100755
index 0000000000..856219f46a
--- /dev/null
+++ b/t/t9303-fast-import-compression.sh
@@ -0,0 +1,67 @@
+#!/bin/sh
+
+test_description='compression setting of fast-import utility'
+. ./test-lib.sh
+
+# This should be moved to test-lib.sh together with the
+# copy in t0021 after both topics have graduated to 'master'.
+file_size () {
+ perl -e 'print -s $ARGV[0]' "$1"
+}
+
+import_large () {
+ (
+ echo blob
+ echo "data <<EOD"
+ printf "%2000000s\n" "$*"
+ echo EOD
+ ) | git "$@" fast-import
+}
+
+while read expect config
+do
+ test_expect_success "fast-import (packed) with $config" '
+ test_when_finished "rm -f .git/objects/pack/pack-*.*" &&
+ test_when_finished "rm -rf .git/objects/??" &&
+ import_large -c fastimport.unpacklimit=0 $config &&
+ sz=$(file_size .git/objects/pack/pack-*.pack) &&
+ case "$expect" in
+ small) test "$sz" -le 100000 ;;
+ large) test "$sz" -ge 100000 ;;
+ esac
+ '
+done <<\EOF
+large -c core.compression=0
+small -c core.compression=9
+large -c core.compression=0 -c pack.compression=0
+large -c core.compression=9 -c pack.compression=0
+small -c core.compression=0 -c pack.compression=9
+small -c core.compression=9 -c pack.compression=9
+large -c pack.compression=0
+small -c pack.compression=9
+EOF
+
+while read expect config
+do
+ test_expect_success "fast-import (loose) with $config" '
+ test_when_finished "rm -f .git/objects/pack/pack-*.*" &&
+ test_when_finished "rm -rf .git/objects/??" &&
+ import_large -c fastimport.unpacklimit=9 $config &&
+ sz=$(file_size .git/objects/??/????*) &&
+ case "$expect" in
+ small) test "$sz" -le 100000 ;;
+ large) test "$sz" -ge 100000 ;;
+ esac
+ '
+done <<\EOF
+large -c core.compression=0
+small -c core.compression=9
+large -c core.compression=0 -c core.loosecompression=0
+large -c core.compression=9 -c core.loosecompression=0
+small -c core.compression=0 -c core.loosecompression=9
+small -c core.compression=9 -c core.loosecompression=9
+large -c core.loosecompression=0
+small -c core.loosecompression=9
+EOF
+
+test_done
--
2.11.0-rc1-154-g4118e37061
^ permalink raw reply related
* Re: Bug with disabling compression and 'binary' files.
From: Douglas Cox @ 2016-11-16 3:12 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <xmqqk2c4ryj8.fsf@gitster.mtv.corp.google.com>
> This may or may not be related to the symptom
> you are observing (if it is, then you would see a packfile created
> in objects/pack/, not in loose objects in object/??/ directories).
No, the file is loose (it's in .git/objects/eb in this case). This is
seen immediately after the add, though I believe it's the same way
when doing a commit on a changed file.
On Tue, Nov 15, 2016 at 8:42 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Douglas Cox <ziflin@gmail.com> writes:
>
>> I narrowed this down to the '-text' attribute that is set when
>> specifying 'binary'. For some reason this flag is cancelling out the
>> core.compression = 0 setting and I think this is a bug?
>>
>> Unfortunately core.compression = 0 is also global. Ideally it would be
>> great if there was a separate 'compression' attribute that could be
>> specified in .gitattributes per wildcard similar to how -delta can be
>> used. This way we would still be able to get compression for
>> text/source files, while still getting the speed of skipping
>> compression for binary files that do not compress well.
>>
>> Has there been any discussion on having an attribute similar to this?
>
> Nope.
>
> I do not offhand think of a way for '-text' attribute (or any
> attribute for what matter) to interfere with compression level, but
> while reading the various parts of the system that futz with the
> compression level configuration, I noticed one thing. When we do an
> initial "bulk-checkin" optimization that sends all objects to a
> single packfile upon "git add", the packfile creation uses its own
> compression level that is not affected by any configuration or
> command line option. This may or may not be related to the symptom
> you are observing (if it is, then you would see a packfile created
> in objects/pack/, not in loose objects in object/??/ directories).
>
>
>
^ permalink raw reply
* [PATCH] compression: unify pack.compression configuration parsing
From: Junio C Hamano @ 2016-11-16 2:48 UTC (permalink / raw)
To: git
There are three codepaths that use a variable whose name is
pack_compression_level to affect how objects and deltas sent to a
packfile is compressed. Unlike zlib_compression_level that controls
the loose object compression, however, this variable was static to
each of these codepaths. Two of them read the pack.compression
configuration variable, using core.compression as the default, and
one of them also allowed overriding it from the command line.
The other codepath in bulk-checkin did not pay any attention to the
configuration.
Unify the configuration parsing to git_default_config(), where we
implement the parsing of core.loosecompression and core.compression
and make the former override the latter, by moving code to parse
pack.compression and also allow core.compression to give default to
this variable.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
* I was primarily interested in teaching the configuration to
bulk-checkin codepath, and added a test that succeeds only with
the code change. The handling of confugraiton in other two
codepaths, pack-objects and fast-import, may be totally broken
with this change if there is no existing test. A similar set of
tests for them are very much welcomed, as without them this
change won't be 'next' worthy yet.
builtin/pack-objects.c | 14 --------------
bulk-checkin.c | 2 --
cache.h | 2 ++
config.c | 14 ++++++++++++++
environment.c | 2 ++
fast-import.c | 13 -------------
t/t1050-large.sh | 30 ++++++++++++++++++++++++++++++
7 files changed, 48 insertions(+), 29 deletions(-)
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 0fd52bd6b4..8841f8b366 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -61,8 +61,6 @@ static int delta_search_threads;
static int pack_to_stdout;
static int num_preferred_base;
static struct progress *progress_state;
-static int pack_compression_level = Z_DEFAULT_COMPRESSION;
-static int pack_compression_seen;
static struct packed_git *reuse_packfile;
static uint32_t reuse_packfile_objects;
@@ -2368,16 +2366,6 @@ static int git_pack_config(const char *k, const char *v, void *cb)
depth = git_config_int(k, v);
return 0;
}
- if (!strcmp(k, "pack.compression")) {
- int level = git_config_int(k, v);
- if (level == -1)
- level = Z_DEFAULT_COMPRESSION;
- else if (level < 0 || level > Z_BEST_COMPRESSION)
- die("bad pack compression level %d", level);
- pack_compression_level = level;
- pack_compression_seen = 1;
- return 0;
- }
if (!strcmp(k, "pack.deltacachesize")) {
max_delta_cache_size = git_config_int(k, v);
return 0;
@@ -2869,8 +2857,6 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
reset_pack_idx_option(&pack_idx_opts);
git_config(git_pack_config, NULL);
- if (!pack_compression_seen && core_compression_seen)
- pack_compression_level = core_compression_level;
progress = isatty(2);
argc = parse_options(argc, argv, prefix, pack_objects_options,
diff --git a/bulk-checkin.c b/bulk-checkin.c
index 4347f5c76a..991b4a13e2 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -7,8 +7,6 @@
#include "pack.h"
#include "strbuf.h"
-static int pack_compression_level = Z_DEFAULT_COMPRESSION;
-
static struct bulk_checkin_state {
unsigned plugged:1;
diff --git a/cache.h b/cache.h
index a50a61a197..a5d689f9d3 100644
--- a/cache.h
+++ b/cache.h
@@ -671,6 +671,8 @@ extern const char *git_hooks_path;
extern int zlib_compression_level;
extern int core_compression_level;
extern int core_compression_seen;
+extern int pack_compression_level;
+extern int pack_compression_seen;
extern size_t packed_git_window_size;
extern size_t packed_git_limit;
extern size_t delta_base_cache_limit;
diff --git a/config.c b/config.c
index 83fdecb1bc..0477386f27 100644
--- a/config.c
+++ b/config.c
@@ -865,6 +865,8 @@ static int git_default_core_config(const char *var, const char *value)
core_compression_seen = 1;
if (!zlib_compression_seen)
zlib_compression_level = level;
+ if (!pack_compression_seen)
+ pack_compression_level = level;
return 0;
}
@@ -1125,6 +1127,18 @@ int git_default_config(const char *var, const char *value, void *dummy)
pack_size_limit_cfg = git_config_ulong(var, value);
return 0;
}
+
+ if (!strcmp(var, "pack.compression")) {
+ int level = git_config_int(var, value);
+ if (level == -1)
+ level = Z_DEFAULT_COMPRESSION;
+ else if (level < 0 || level > Z_BEST_COMPRESSION)
+ die(_("bad pack compression level %d"), level);
+ pack_compression_level = level;
+ pack_compression_seen = 1;
+ return 0;
+ }
+
/* Add other config variables here and to Documentation/config.txt. */
return 0;
}
diff --git a/environment.c b/environment.c
index 0935ec696e..88ab1c38c6 100644
--- a/environment.c
+++ b/environment.c
@@ -35,6 +35,8 @@ const char *git_hooks_path;
int zlib_compression_level = Z_BEST_SPEED;
int core_compression_level;
int core_compression_seen;
+int pack_compression_level = Z_DEFAULT_COMPRESSION;
+int pack_compression_seen;
int fsync_object_files;
size_t packed_git_window_size = DEFAULT_PACKED_GIT_WINDOW_SIZE;
size_t packed_git_limit = DEFAULT_PACKED_GIT_LIMIT;
diff --git a/fast-import.c b/fast-import.c
index cb545d7df5..f561ba833b 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -284,8 +284,6 @@ static unsigned long max_depth = 10;
static off_t max_packsize;
static int unpack_limit = 100;
static int force_update;
-static int pack_compression_level = Z_DEFAULT_COMPRESSION;
-static int pack_compression_seen;
/* Stats and misc. counters */
static uintmax_t alloc_count;
@@ -3381,15 +3379,6 @@ static void git_pack_config(void)
if (max_depth > MAX_DEPTH)
max_depth = MAX_DEPTH;
}
- if (!git_config_get_int("pack.compression", &pack_compression_level)) {
- if (pack_compression_level == -1)
- pack_compression_level = Z_DEFAULT_COMPRESSION;
- else if (pack_compression_level < 0 ||
- pack_compression_level > Z_BEST_COMPRESSION)
- git_die_config("pack.compression",
- "bad pack compression level %d", pack_compression_level);
- pack_compression_seen = 1;
- }
if (!git_config_get_int("pack.indexversion", &indexversion_value)) {
pack_idx_opts.version = indexversion_value;
if (pack_idx_opts.version > 2)
@@ -3454,8 +3443,6 @@ int cmd_main(int argc, const char **argv)
setup_git_directory();
reset_pack_idx_option(&pack_idx_opts);
git_pack_config();
- if (!pack_compression_seen && core_compression_seen)
- pack_compression_level = core_compression_level;
alloc_objects(object_entry_alloc);
strbuf_init(&command_buf, 0);
diff --git a/t/t1050-large.sh b/t/t1050-large.sh
index 096dbffecc..f800af1451 100755
--- a/t/t1050-large.sh
+++ b/t/t1050-large.sh
@@ -5,6 +5,13 @@ test_description='adding and checking out large blobs'
. ./test-lib.sh
+# This should be moved to test-lib.sh together with the
+# copy in t0021 after both topics have graduated to 'master'.
+
+file_size () {
+ perl -e 'print -s $ARGV[0]' "$1"
+}
+
test_expect_success setup '
# clone does not allow us to pass core.bigfilethreshold to
# new repos, so set core.bigfilethreshold globally
@@ -17,6 +24,29 @@ test_expect_success setup '
export GIT_ALLOC_LIMIT
'
+# add a large file with different settings
+while read expect config
+do
+ test_expect_success "add with $config" '
+ test_when_finished "rm -f .git/objects/pack/pack-*.* .git/index" &&
+ git $config add large1 &&
+ sz=$(file_size .git/objects/pack/pack-*.pack) &&
+ case "$expect" in
+ small) test "$sz" -le 100000 ;;
+ large) test "$sz" -ge 100000 ;;
+ esac
+ '
+done <<-EOF
+large -c core.compression=0
+small -c core.compression=9
+large -c core.compression=0 -c pack.compression=0
+large -c core.compression=9 -c pack.compression=0
+small -c core.compression=0 -c pack.compression=9
+small -c core.compression=9 -c pack.compression=9
+large -c pack.compression=0
+small -c pack.compression=9
+EOF
+
test_expect_success 'add a large file or two' '
git add large1 huge large2 &&
# make sure we got a single packfile and no loose objects
--
2.11.0-rc1-154-g4118e37061
^ permalink raw reply related
* Re: Bug with disabling compression and 'binary' files.
From: Junio C Hamano @ 2016-11-16 1:42 UTC (permalink / raw)
To: Douglas Cox; +Cc: git
In-Reply-To: <CA+i4re65SsxcaLcpGyMDnJygQFmAq4X_x_uxrkqB0yqQkEYPUQ@mail.gmail.com>
Douglas Cox <ziflin@gmail.com> writes:
> I narrowed this down to the '-text' attribute that is set when
> specifying 'binary'. For some reason this flag is cancelling out the
> core.compression = 0 setting and I think this is a bug?
>
> Unfortunately core.compression = 0 is also global. Ideally it would be
> great if there was a separate 'compression' attribute that could be
> specified in .gitattributes per wildcard similar to how -delta can be
> used. This way we would still be able to get compression for
> text/source files, while still getting the speed of skipping
> compression for binary files that do not compress well.
>
> Has there been any discussion on having an attribute similar to this?
Nope.
I do not offhand think of a way for '-text' attribute (or any
attribute for what matter) to interfere with compression level, but
while reading the various parts of the system that futz with the
compression level configuration, I noticed one thing. When we do an
initial "bulk-checkin" optimization that sends all objects to a
single packfile upon "git add", the packfile creation uses its own
compression level that is not affected by any configuration or
command line option. This may or may not be related to the symptom
you are observing (if it is, then you would see a packfile created
in objects/pack/, not in loose objects in object/??/ directories).
^ permalink raw reply
* Re: [PATCH v7 13/17] ref-filter: add `:dir` and `:base` options for ref printing atoms
From: Jacob Keller @ 2016-11-15 21:19 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Karthik Nayak, Git mailing list
In-Reply-To: <xmqq4m38vdw4.fsf@gitster.mtv.corp.google.com>
On November 15, 2016 9:42:03 AM PST, Junio C Hamano <gitster@pobox.com> wrote:
>I think you are going in the right direction. I had a similar
>thought but built around a different axis. I.e. if strip=1 strips
>one from the left, perhaps we want to have rstrip=1 that strips one
>from the right, and also strip=-1 to mean strip everything except
>one from the left and so on?. I think this and your keep (and
>perhaps you'll have rkeep for completeness) have the same expressive
>power. I do not offhand have a preference one over the other.
I prefer strip implemented with negative numbers. That is simple and expressive and if we need we can implement rstrip if necessary.
>
>Somehow it sounds a bit strange to me to treat 'remotes' as the same
>class of token as 'heads' and 'tags' (I'd expect 'heads' and
>'remotes/origin' would be at the same level in end-user's mind), but
>that is probably an unrelated tangent. The reason this series wants
>to introduce :base must be to emulate an existing feature, so that
>existing feature is a concrete counter-example that argues against
>my "it sounds a bit strange" reaction.
It may be a bit strange indeed. What is the requirement for this?
I think implementing a strip and rstrip ( if necessary ) with negative numbers would be most ideal.
Thanks
Jake
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox