[PATCH 3/6] wrapper: create safe_memory_limit_check()

["Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com>]

From: Derrick Stolee <stolee@gmail.com>

The existing memory_limit_check() is used in many places within wrapper.c,
but because it initializes the GIT_ALLOC_LIMIT environment variable _and_
can call die() when not in gentle mode, this method isn't appropriate for a
safe API.

Modify the implementation to be safe_memory_limit_check() and to keep
calling error() when there is an allocation problem. The original method
calls that version but will die() instead when failing and not gentle.

The one potential behavior change is that when git_alloc_limit is unset we
must assume SIZE_MAX instead of loading the environment variable. Since we
load this environment variable proactively in setup_environment(), this
should only matter for that brief window before setup_environment() and the
safe APIs that call this version. If such safe APIs are used in that window,
then they should allocate small enough amounts of memory to fit under any
reasonable values of GIT_ALLOC_LIMIT.

Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
 wrapper.c | 27 ++++++++++++++++++---------
 1 file changed, 18 insertions(+), 9 deletions(-)

diff --git a/wrapper.c b/wrapper.c
index 3de6b21cc2..97a29bda75 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -30,22 +30,31 @@ void initialize_git_alloc_limit(void)
 	}
 }
 
-static int memory_limit_check(size_t size, int gentle)
+static int safe_memory_limit_check(size_t size, int verbose)
 {
-	initialize_git_alloc_limit();
-
-	if (size > git_alloc_limit) {
-		if (gentle) {
+	size_t limit = git_alloc_limit ? git_alloc_limit : SIZE_MAX;
+	if (size > limit) {
+		if (verbose)
 			error("attempting to allocate %"PRIuMAX" over limit %"PRIuMAX,
 			      (uintmax_t)size, (uintmax_t)git_alloc_limit);
-			return -1;
-		} else
-			die("attempting to allocate %"PRIuMAX" over limit %"PRIuMAX,
-			    (uintmax_t)size, (uintmax_t)git_alloc_limit);
+		return -1;
 	}
 	return 0;
 }
 
+static int memory_limit_check(size_t size, int gentle)
+{
+	int res;
+	initialize_git_alloc_limit();
+
+	res = safe_memory_limit_check(size, gentle);
+	if (res && !gentle) {
+		die("attempting to allocate %"PRIuMAX" over limit %"PRIuMAX,
+		    (uintmax_t)size, (uintmax_t)git_alloc_limit);
+	}
+	return res;
+}
+
 char *xstrdup(const char *str)
 {
 	char *ret = strdup(str);
-- 
gitgitgadget