git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Eric Wong <e@80x24.org>
To: git@vger.kernel.org
Subject: [PATCH 2/4] map_loose_object_1: attempt to handle ENOMEM from mmap
Date: Tue, 29 Jun 2021 08:11:06 +0000	[thread overview]
Message-ID: <20210629081108.28657-3-e@80x24.org> (raw)
In-Reply-To: <20210629081108.28657-1-e@80x24.org>

This benefits unprivileged users who lack permissions to raise
the `sys.vm.max_map_count' sysctl and/or RLIMIT_DATA resource
limit.

Multi-threaded callers to map_loose_object_1 (e.g. "git grep")
appear to guard against thread-safety problems.  Other
multi-core aware pieces of code (e.g. parallel-checkout) uses
child processes

Forcing a call to unuse_one_window() once before xmmap_gently()
revealed no test suite failures:

	--- a/object-file.c
	+++ b/object-file.c
	@@ -1197,6 +1197,7 @@ static void *map_loose_object_1(struct repository *r, const char *path,
					return NULL;
				}
				do {
	+				unuse_one_window(NULL);
					map = xmmap_gently(NULL, *size, PROT_READ,
							MAP_PRIVATE, fd, 0);
				} while (map == MAP_FAILED && errno == ENOMEM

Signed-off-by: Eric Wong <e@80x24.org>
---
 object-file.c | 8 +++++++-
 packfile.c    | 2 +-
 packfile.h    | 2 ++
 3 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/object-file.c b/object-file.c
index f233b440b2..4c043f1f3c 100644
--- a/object-file.c
+++ b/object-file.c
@@ -1196,7 +1196,13 @@ static void *map_loose_object_1(struct repository *r, const char *path,
 				close(fd);
 				return NULL;
 			}
-			map = xmmap(NULL, *size, PROT_READ, MAP_PRIVATE, fd, 0);
+			do {
+				map = xmmap_gently(NULL, *size, PROT_READ,
+						MAP_PRIVATE, fd, 0);
+			} while (map == MAP_FAILED && errno == ENOMEM
+				&& unuse_one_window(NULL));
+			if (map == MAP_FAILED)
+				die_errno("%s cannot be mapped", path);
 		}
 		close(fd);
 	}
diff --git a/packfile.c b/packfile.c
index a0da790fb4..4bc7fa0f36 100644
--- a/packfile.c
+++ b/packfile.c
@@ -265,7 +265,7 @@ static void scan_windows(struct packed_git *p,
 	}
 }
 
-static int unuse_one_window(struct packed_git *current)
+int unuse_one_window(struct packed_git *current)
 {
 	struct packed_git *p, *lru_p = NULL;
 	struct pack_window *lru_w = NULL, *lru_l = NULL;
diff --git a/packfile.h b/packfile.h
index 3ae117a8ae..da9a061e30 100644
--- a/packfile.h
+++ b/packfile.h
@@ -196,4 +196,6 @@ int is_promisor_object(const struct object_id *oid);
 int load_idx(const char *path, const unsigned int hashsz, void *idx_map,
 	     size_t idx_size, struct packed_git *p);
 
+int unuse_one_window(struct packed_git *);
+
 #endif

  parent reply	other threads:[~2021-06-29  8:11 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-06-29  8:11 [PATCH 0/4] gracefully handling mmap failures Eric Wong
2021-06-29  8:11 ` [PATCH 1/4] use_pack: attempt to handle ENOMEM from mmap Eric Wong
2021-06-30  2:30   ` Jeff King
2021-06-29  8:11 ` Eric Wong [this message]
2021-06-30  2:41   ` [PATCH 2/4] map_loose_object_1: " Jeff King
2021-06-30  6:06   ` Ævar Arnfjörð Bjarmason
2021-06-29  8:11 ` [PATCH 3/4] check_packed_git_idx: " Eric Wong
2021-06-29 20:21   ` [HOLD " Eric Wong
2021-06-29 21:33     ` Junio C Hamano
2021-06-29 22:31       ` Eric Wong
2021-06-29  8:11 ` [PATCH 4/4] xmmap: inform Linux users of tuning knobs on ENOMEM Eric Wong
2021-06-30  0:01 ` [PATCH v2] " Eric Wong
2021-06-30  2:46   ` Jeff King
2021-06-30 10:07     ` Eric Wong
2021-06-30 17:18       ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210629081108.28657-3-e@80x24.org \
    --to=e@80x24.org \
    --cc=git@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).