From: Taylor Blau <me@ttaylorr.com>
To: git@vger.kernel.org
Cc: Johannes.Schindelin@gmx.de, dstolee@microsoft.com,
jeffhost@microsoft.com, peff@peff.net
Subject: [PATCH 0/4] midx: avoid potential rename-while-mapped on Windows
Date: Fri, 8 Oct 2021 17:46:25 -0400 [thread overview]
Message-ID: <cover.1633729502.git.me@ttaylorr.com> (raw)
(This topic is based on tb/repack-write-midx).
While looking through some failures in t5319 when Git is compiled with
SANITIZE=leak, I noticed a curious pattern that basically looks like:
struct multi_pack_index *m = load_multi_pack_index(object_dir, 0);
/* ... */
write_midx_internal();
This can cause problems on platforms where a memory mapped file cannot be
overridden via a rename, because load_multi_pack_index() builds a new MIDX from
scratch each time it is called without adding that MIDX to the object store. But
write_midx_internal() loads a MIDX from the object store, causing the same
region to be mapped twice.
The details are fully spelled out in the second patch, but the gist is that
while write_midx_internal() unmaps *its* copy of the MIDX, the caller's copy is
still mapped in memory.
Strangely, I could not get this to produce a failure on Windows, even though I
would have expected one. I asked around off-list, and it sounds like it's
possible this behavior has changed, or Windows' mmap-equivalent may not lock
files that fit within a single page.
In any case, we let's side step the issue entirely by only interacting with
MIDXs that can be found from the object store.
Taylor Blau (4):
midx.c: extract MIDX lookup by object_dir
midx.c: lookup MIDX by object directory during expire
midx.c: lookup MIDX by object directory during repack
midx.c: guard against commit_lock_file() failures
midx.c | 42 +++++++++++++++++++++++-------------------
1 file changed, 23 insertions(+), 19 deletions(-)
--
2.33.0.96.g73915697e6
next reply other threads:[~2021-10-08 21:46 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-08 21:46 Taylor Blau [this message]
2021-10-08 21:46 ` [PATCH 1/4] midx.c: extract MIDX lookup by object_dir Taylor Blau
2021-10-08 21:46 ` [PATCH 2/4] midx.c: lookup MIDX by object directory during expire Taylor Blau
2021-10-13 13:28 ` Derrick Stolee
2021-10-08 21:46 ` [PATCH 3/4] midx.c: lookup MIDX by object directory during repack Taylor Blau
2021-10-08 21:46 ` [PATCH 4/4] midx.c: guard against commit_lock_file() failures Taylor Blau
2021-10-09 2:07 ` Ævar Arnfjörð Bjarmason
2021-10-13 13:29 ` [PATCH 0/4] midx: avoid potential rename-while-mapped on Windows Derrick Stolee
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1633729502.git.me@ttaylorr.com \
--to=me@ttaylorr.com \
--cc=Johannes.Schindelin@gmx.de \
--cc=dstolee@microsoft.com \
--cc=git@vger.kernel.org \
--cc=jeffhost@microsoft.com \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.