From: "Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>
To: pclouds@gmail.com
Cc: Ben.Peart@microsoft.com, git@vger.kernel.org, gitster@pobox.com,
newren@gmail.com, peartben@gmail.com, peff@peff.net
Subject: [PATCH v5 6/7] unpack-trees: add missing cache invalidation
Date: Sat, 18 Aug 2018 16:41:27 +0200 [thread overview]
Message-ID: <20180818144128.19361-7-pclouds@gmail.com> (raw)
In-Reply-To: <20180818144128.19361-1-pclouds@gmail.com>
Any changes to the output index should be (confusingly) marked in the
source index with invalidate_ce_path(). This is used to make sure we
still have valid untracked cache and cache-tree extensions in the end.
We do a pretty good job of invalidating except in two places.
verify_clean_subdirectory() is part of verify_absent() and
verify_absent_sparse(). The former is usually called by merged_entry()
or directly in threeway_merge(). The latter is obviously used by
sparse checkout.
In these three call sites, only merged_entry() follows up with
invalidate_ce_path(). The other two don't, but they should not trigger
this ce removal because this is about D/F conflicts [1]. But let's be
safe and invalidate_ce_path() here as well.
The second place is keep_entry() which is also used by threeway_merge()
to keep higher stage entries. In order to reuse cache-tree we need to
invalidate these paths as well. It's not a problem in the past because
whenever a higher stage entry is present, cache-tree will not be
created [2]. Now we salvage cache-tree even when higher stage entries
are present, we need more invalidation.
[1] c81935348b (Fix switching to a branch with D/F when current branch
has file D. - 2007-03-15)
[2] This is probably too strict. We should be able to create and save
cache-tree for the directories that do not have conflict entries
in cache_tree_update(). And this becomes more important when
cache-tree plays bigger role in terms of performance.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
unpack-trees.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/unpack-trees.c b/unpack-trees.c
index aa80b65ee1..bc43922922 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -1774,6 +1774,7 @@ static int verify_clean_subdirectory(const struct cache_entry *ce,
if (verify_uptodate(ce2, o))
return -1;
add_entry(o, ce2, CE_REMOVE, 0);
+ invalidate_ce_path(ce, o);
mark_ce_used(ce2, o);
}
cnt++;
@@ -2033,6 +2034,8 @@ static int keep_entry(const struct cache_entry *ce,
struct unpack_trees_options *o)
{
add_entry(o, ce, 0, 0);
+ if (ce_stage(ce))
+ invalidate_ce_path(ce, o);
return 1;
}
--
2.18.0.1004.g6639190530
next prev parent reply other threads:[~2018-08-18 14:41 UTC|newest]
Thread overview: 121+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-07-18 20:45 [PATCH v1 0/3] [RFC] Speeding up checkout (and merge, rebase, etc) Ben Peart
2018-07-18 20:45 ` [PATCH v1 1/3] add unbounded Multi-Producer-Multi-Consumer queue Ben Peart
2018-07-18 20:57 ` Stefan Beller
2018-07-19 19:11 ` Junio C Hamano
2018-07-18 20:45 ` [PATCH v1 2/3] add performance tracing around traverse_trees() in unpack_trees() Ben Peart
2018-07-18 20:45 ` [PATCH v1 3/3] Add initial parallel version of unpack_trees() Ben Peart
2018-07-18 22:56 ` Junio C Hamano
2018-07-18 21:02 ` [PATCH v1 0/3] [RFC] Speeding up checkout (and merge, rebase, etc) Stefan Beller
2018-07-18 21:34 ` Jeff King
2018-07-23 15:48 ` Ben Peart
2018-07-23 17:03 ` Duy Nguyen
2018-07-23 20:51 ` Ben Peart
2018-07-24 4:20 ` Jeff King
2018-07-24 15:33 ` Duy Nguyen
2018-07-25 20:56 ` Ben Peart
2018-07-26 5:30 ` Duy Nguyen
2018-07-26 16:30 ` Duy Nguyen
2018-07-26 19:40 ` Junio C Hamano
2018-07-27 15:42 ` Duy Nguyen
2018-07-27 16:22 ` Ben Peart
2018-07-27 18:00 ` Duy Nguyen
2018-07-27 17:14 ` Junio C Hamano
2018-07-27 17:52 ` Duy Nguyen
2018-07-29 6:24 ` Duy Nguyen
2018-07-29 10:33 ` [PATCH v2 0/4] Speed up unpack_trees() Nguyễn Thái Ngọc Duy
2018-07-29 10:33 ` [PATCH v2 1/4] unpack-trees.c: add performance tracing Nguyễn Thái Ngọc Duy
2018-07-30 20:16 ` Ben Peart
2018-07-29 10:33 ` [PATCH v2 2/4] unpack-trees: optimize walking same trees with cache-tree Nguyễn Thái Ngọc Duy
2018-07-30 20:52 ` Ben Peart
2018-07-29 10:33 ` [PATCH v2 3/4] unpack-trees: reduce malloc in cache-tree walk Nguyễn Thái Ngọc Duy
2018-07-30 20:58 ` Ben Peart
2018-07-29 10:33 ` [PATCH v2 4/4] unpack-trees: cheaper index update when walking by cache-tree Nguyễn Thái Ngọc Duy
2018-08-08 18:46 ` Elijah Newren
2018-08-10 16:39 ` Duy Nguyen
2018-08-10 18:39 ` Elijah Newren
2018-08-10 19:30 ` Duy Nguyen
2018-08-10 19:40 ` Elijah Newren
2018-08-10 19:48 ` Duy Nguyen
2018-07-30 18:10 ` [PATCH v2 0/4] Speed up unpack_trees() Ben Peart
2018-07-31 15:31 ` Duy Nguyen
2018-07-31 16:50 ` Ben Peart
2018-07-31 17:31 ` Ben Peart
2018-08-01 16:38 ` Duy Nguyen
2018-08-08 20:53 ` Ben Peart
2018-08-09 8:16 ` Ben Peart
2018-08-10 16:08 ` Duy Nguyen
2018-08-10 15:51 ` Duy Nguyen
2018-07-30 21:04 ` Ben Peart
2018-08-04 5:37 ` [PATCH v3 " Nguyễn Thái Ngọc Duy
2018-08-04 5:37 ` [PATCH v3 1/4] unpack-trees: add performance tracing Nguyễn Thái Ngọc Duy
2018-08-04 5:37 ` [PATCH v3 2/4] unpack-trees: optimize walking same trees with cache-tree Nguyễn Thái Ngọc Duy
2018-08-08 18:23 ` Elijah Newren
2018-08-10 16:29 ` Duy Nguyen
2018-08-10 18:48 ` Elijah Newren
2018-08-04 5:37 ` [PATCH v3 3/4] unpack-trees: reduce malloc in cache-tree walk Nguyễn Thái Ngọc Duy
2018-08-08 18:30 ` Elijah Newren
2018-08-04 5:37 ` [PATCH v3 4/4] unpack-trees: cheaper index update when walking by cache-tree Nguyễn Thái Ngọc Duy
2018-08-06 15:48 ` [PATCH v3 0/4] Speed up unpack_trees() Junio C Hamano
2018-08-06 15:59 ` Duy Nguyen
2018-08-06 18:59 ` Junio C Hamano
2018-08-08 17:00 ` Ben Peart
2018-08-08 17:46 ` Junio C Hamano
2018-08-08 18:12 ` Junio C Hamano
2018-08-08 18:39 ` Junio C Hamano
2018-08-10 16:53 ` Duy Nguyen
2018-08-12 8:15 ` [PATCH v4 0/5] " Nguyễn Thái Ngọc Duy
2018-08-12 8:15 ` [PATCH v4 1/5] trace.h: support nested performance tracing Nguyễn Thái Ngọc Duy
2018-08-13 18:39 ` Ben Peart
2018-08-12 8:15 ` [PATCH v4 2/5] unpack-trees: add " Nguyễn Thái Ngọc Duy
2018-08-12 10:05 ` Thomas Adam
2018-08-13 18:50 ` Junio C Hamano
2018-08-13 18:44 ` Ben Peart
2018-08-13 19:25 ` Jeff King
2018-08-13 19:36 ` Stefan Beller
2018-08-13 20:11 ` Ben Peart
2018-08-13 19:52 ` Duy Nguyen
2018-08-13 21:47 ` Jeff King
2018-08-13 22:41 ` Junio C Hamano
2018-08-14 18:19 ` Jeff Hostetler
2018-08-14 18:32 ` Duy Nguyen
2018-08-14 18:44 ` Stefan Beller
2018-08-14 18:51 ` Duy Nguyen
2018-08-14 19:54 ` Jeff King
2018-08-14 20:52 ` Junio C Hamano
2018-08-15 16:32 ` Duy Nguyen
2018-08-15 18:28 ` Junio C Hamano
2018-08-14 20:14 ` Jeff Hostetler
2018-08-12 8:15 ` [PATCH v4 3/5] unpack-trees: optimize walking same trees with cache-tree Nguyễn Thái Ngọc Duy
2018-08-13 18:58 ` Ben Peart
2018-08-15 16:38 ` Duy Nguyen
2018-08-12 8:15 ` [PATCH v4 4/5] unpack-trees: reduce malloc in cache-tree walk Nguyễn Thái Ngọc Duy
2018-08-12 8:15 ` [PATCH v4 5/5] unpack-trees: reuse (still valid) cache-tree from src_index Nguyễn Thái Ngọc Duy
2018-08-13 15:48 ` Elijah Newren
2018-08-13 15:57 ` Duy Nguyen
2018-08-13 16:05 ` Ben Peart
2018-08-13 16:25 ` Duy Nguyen
2018-08-13 17:15 ` Ben Peart
2018-08-13 19:01 ` [PATCH v4 0/5] Speed up unpack_trees() Junio C Hamano
2018-08-14 19:19 ` Ben Peart
2018-08-18 14:41 ` [PATCH v5 0/7] " Nguyễn Thái Ngọc Duy
2018-08-18 14:41 ` [PATCH v5 1/7] trace.h: support nested performance tracing Nguyễn Thái Ngọc Duy
2018-08-18 14:41 ` [PATCH v5 2/7] unpack-trees: add " Nguyễn Thái Ngọc Duy
2018-08-18 14:41 ` [PATCH v5 3/7] unpack-trees: optimize walking same trees with cache-tree Nguyễn Thái Ngọc Duy
2018-08-20 12:43 ` Ben Peart
2018-08-18 14:41 ` [PATCH v5 4/7] unpack-trees: reduce malloc in cache-tree walk Nguyễn Thái Ngọc Duy
2018-08-18 14:41 ` [PATCH v5 5/7] unpack-trees: reuse (still valid) cache-tree from src_index Nguyễn Thái Ngọc Duy
2018-08-18 14:41 ` Nguyễn Thái Ngọc Duy [this message]
2018-08-18 14:41 ` [PATCH v5 7/7] cache-tree: verify valid cache-tree in the test suite Nguyễn Thái Ngọc Duy
2018-08-18 21:45 ` Elijah Newren
2018-08-18 22:01 ` [PATCH v5 0/7] Speed up unpack_trees() Elijah Newren
2018-08-19 5:09 ` Duy Nguyen
2018-08-25 12:18 ` [PATCH] Document update for nd/unpack-trees-with-cache-tree Nguyễn Thái Ngọc Duy
2018-08-25 12:31 ` Martin Ågren
2018-08-25 13:02 ` [PATCH v2] " Nguyễn Thái Ngọc Duy
2018-07-27 15:50 ` [PATCH v1 0/3] [RFC] Speeding up checkout (and merge, rebase, etc) Ben Peart
2018-07-26 16:35 ` Duy Nguyen
2018-07-24 5:54 ` Junio C Hamano
2018-07-24 15:13 ` Duy Nguyen
2018-07-24 21:21 ` Jeff King
2018-07-25 16:09 ` Ben Peart
2018-07-24 4:27 ` Jeff King
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180818144128.19361-7-pclouds@gmail.com \
--to=pclouds@gmail.com \
--cc=Ben.Peart@microsoft.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=newren@gmail.com \
--cc=peartben@gmail.com \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.