git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Martin Koegler <mkoegler@auto.tuwien.ac.at>
To: Nicolas Pitre <nico@cam.org>,
	Johannes Schindelin <Johannes.Schindelin@gmx.de>
Cc: git@vger.kernel.org, Martin Koegler <mkoegler@auto.tuwien.ac.at>
Subject: [PATCH] pack-objects: only throw away data during memory pressure
Date: Mon, 11 Feb 2008 08:26:25 +0100	[thread overview]
Message-ID: <120271478556-git-send-email-mkoegler@auto.tuwien.ac.at> (raw)

If pack-objects hit the memory limit, it deletes objects from the delta
window.

This patch make it only delete the data, which is recomputed, if needed again.

Signed-off-by: Martin Koegler <mkoegler@auto.tuwien.ac.at>
---
What about this not really tested patch for dealing with memory pressure in git-pack-objects?

It will slow down the repack in the case of memory pressure, but missing memory will not affect the results.

 builtin-pack-objects.c |   13 +++++++++++--
 1 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index 6f8f388..231d65f 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -1464,7 +1464,7 @@ static unsigned int check_delta_limit(struct object_entry *me, unsigned int n)
 	return m;
 }
 
-static unsigned long free_unpacked(struct unpacked *n)
+static unsigned long free_unpacked_data(struct unpacked *n)
 {
 	unsigned long freed_mem = sizeof_delta_index(n->index);
 	free_delta_index(n->index);
@@ -1474,6 +1474,12 @@ static unsigned long free_unpacked(struct unpacked *n)
 		free(n->data);
 		n->data = NULL;
 	}
+	return freed_mem;
+}
+
+static unsigned long free_unpacked(struct unpacked *n)
+{
+	unsigned long freed_mem = free_unpacked_data(n);
 	n->entry = NULL;
 	n->depth = 0;
 	return freed_mem;
@@ -1514,7 +1520,7 @@ static void find_deltas(struct object_entry **list, unsigned *list_size,
 		       mem_usage > window_memory_limit &&
 		       count > 1) {
 			uint32_t tail = (idx + window - count) % window;
-			mem_usage -= free_unpacked(array + tail);
+			mem_usage -= free_unpacked_data(array + tail);
 			count--;
 		}
 
@@ -1547,6 +1553,9 @@ static void find_deltas(struct object_entry **list, unsigned *list_size,
 			if (!m->entry)
 				break;
 			ret = try_delta(n, m, max_depth, &mem_usage);
+			if (window_memory_limit &&
+			    mem_usage > window_memory_limit)
+				mem_usage -= free_unpacked_data(m);
 			if (ret < 0)
 				break;
 			else if (ret > 0)
-- 
1.5.4.g42f90

             reply	other threads:[~2008-02-11  7:27 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-02-11  7:26 Martin Koegler [this message]
2008-02-11 15:20 ` [PATCH] pack-objects: only throw away data during memory pressure Johannes Schindelin
2008-02-11 16:00   ` Nicolas Pitre
2008-02-11 16:08     ` Johannes Schindelin
2008-02-11 16:00 ` Nicolas Pitre
2008-02-12  8:22   ` Brian Downing
2008-02-12 14:26     ` Nicolas Pitre
2008-02-12 17:14       ` Brian Downing
2008-02-12 17:17         ` Brian Downing
2008-02-12 17:28         ` Brian Downing
2008-02-12 17:39         ` Nicolas Pitre
2008-02-13  1:48   ` Nicolas Pitre

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=120271478556-git-send-email-mkoegler@auto.tuwien.ac.at \
    --to=mkoegler@auto.tuwien.ac.at \
    --cc=Johannes.Schindelin@gmx.de \
    --cc=git@vger.kernel.org \
    --cc=nico@cam.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).