git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Shawn O. Pearce" <spearce@spearce.org>
To: Junio C Hamano <junkio@cox.net>
Cc: git@vger.kernel.org
Subject: [PATCH 10/17] Unmap individual windows rather than entire files.
Date: Sat, 23 Dec 2006 02:34:23 -0500	[thread overview]
Message-ID: <20061223073423.GK9837@spearce.org> (raw)
In-Reply-To: <53b67707929c7f051f6d384c5d96e653bfa8419c.1166857884.git.spearce@spearce.org>

To support multiple windows per packfile we need to unmap only one
window at a time from that packfile, leaving any other windows in
place and available for reference.

We treat all windows from all packfiles equally; the least recently
used, not-in-use window across all packfiles will always be closed
first.

If we have unmapped all windows in a packfile then we can also close
the packfile's file descriptor as its possible we won't need to map
any window from that file in the near future.  This decision about
when to close the pack file descriptor may need to be revisited in
the future after additional testing on several different platforms
can be performed.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 sha1_file.c |   45 ++++++++++++++++++++++++++++++---------------
 1 files changed, 30 insertions(+), 15 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index 6c34482..49dd4b7 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -450,24 +450,39 @@ static int check_packed_git_idx(const char *path, unsigned long *idx_size_,
 	return 0;
 }
 
-static int unuse_one_packed_git(void)
+static int unuse_one_window(void)
 {
-	struct packed_git *p, *lru = NULL;
+	struct packed_git *p, *lru_p = NULL;
+	struct pack_window *w, *w_l, *lru_w = NULL, *lru_l = NULL;
 
 	for (p = packed_git; p; p = p->next) {
-		if (!p->windows || p->windows->inuse_cnt)
-			continue;
-		if (!lru || p->windows->last_used < lru->windows->last_used)
-			lru = p;
+		for (w_l = NULL, w = p->windows; w; w = w->next) {
+			if (!w->inuse_cnt) {
+				if (!lru_w || w->last_used < lru_w->last_used) {
+					lru_p = p;
+					lru_w = w;
+					lru_l = w_l;
+				}
+			}
+			w_l = w;
+		}
 	}
-	if (!lru)
-		return 0;
-	munmap(lru->windows->base, lru->windows->len);
-	free(lru->windows);
-	lru->windows = NULL;
-	close(p->pack_fd);
-	p->pack_fd = -1;
-	return 1;
+	if (lru_p) {
+		munmap(lru_w->base, lru_w->len);
+		pack_mapped -= lru_w->len;
+		if (lru_l)
+			lru_l->next = lru_w->next;
+		else {
+			lru_p->windows = lru_w->next;
+			if (!lru_p->windows) {
+				close(lru_p->pack_fd);
+				lru_p->pack_fd = -1;
+			}
+		}
+		free(lru_w);
+		return 1;
+	}
+	return 0;
 }
 
 void unuse_pack(struct pack_window **w_cursor)
@@ -532,7 +547,7 @@ unsigned char* use_pack(struct packed_git *p,
 		open_packed_git(p);
 	if (!win) {
 		pack_mapped += p->pack_size;
-		while (packed_git_limit < pack_mapped && unuse_one_packed_git())
+		while (packed_git_limit < pack_mapped && unuse_one_window())
 			; /* nothing */
 		win = xcalloc(1, sizeof(*win));
 		win->len = p->pack_size;
-- 
1.4.4.3.g87d8

  parent reply	other threads:[~2006-12-23  7:34 UTC|newest]

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <53b67707929c7f051f6d384c5d96e653bfa8419c.1166857884.git.spearce@spearce.org>
2006-12-23  7:33 ` [PATCH 1/17] Replace unpack_entry_gently with unpack_entry Shawn O. Pearce
2006-12-23  7:33 ` [PATCH 2/17] Introduce new config option for mmap limit Shawn O. Pearce
2006-12-23  7:33 ` [PATCH 3/17] Refactor packed_git to prepare for sliding mmap windows Shawn O. Pearce
2006-12-23  7:33 ` [PATCH 4/17] Use off_t for index and pack file lengths Shawn O. Pearce
2006-12-23  7:33 ` [PATCH 5/17] Create read_or_die utility routine Shawn O. Pearce
2006-12-23  7:34 ` [PATCH 6/17] Refactor how we open pack files to prepare for multiple windows Shawn O. Pearce
2006-12-23  7:34 ` [PATCH 7/17] Replace use_packed_git with window cursors Shawn O. Pearce
2006-12-23  7:34 ` [PATCH 8/17] Loop over pack_windows when inflating/accessing data Shawn O. Pearce
2006-12-23  7:34 ` [PATCH 9/17] Document why header parsing won't exceed a window Shawn O. Pearce
2006-12-23  7:34 ` Shawn O. Pearce [this message]
2006-12-23  7:34 ` [PATCH 11/17] Fully activate the sliding window pack access Shawn O. Pearce
2006-12-23 18:44   ` Linus Torvalds
2006-12-23 19:34     ` Eric Blake
2006-12-24  0:58       ` Johannes Schindelin
2006-12-23 19:45     ` Junio C Hamano
2006-12-23 20:10       ` Linus Torvalds
2006-12-24  1:23         ` Johannes Schindelin
2006-12-24  2:23       ` Shawn Pearce
2006-12-24  2:35       ` Shawn Pearce
2006-12-23  7:34 ` [PATCH 12/17] Load core configuration in git-verify-pack Shawn O. Pearce
2006-12-23  7:34 ` [PATCH 13/17] Ensure core.packedGitWindowSize cannot be less than 2 pages Shawn O. Pearce
2006-12-23  7:34 ` [PATCH 14/17] Improve error message when packfile mmap fails Shawn O. Pearce
2006-12-23  7:34 ` [PATCH 15/17] Support unmapping windows on 'temporary' packfiles Shawn O. Pearce
2006-12-23  7:34 ` [PATCH 16/17] Create pack_report() as a debugging aid Shawn O. Pearce
2006-12-23  7:34 ` [PATCH 17/17] Test suite for sliding window mmap implementation Shawn O. Pearce

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20061223073423.GK9837@spearce.org \
    --to=spearce@spearce.org \
    --cc=git@vger.kernel.org \
    --cc=junkio@cox.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).