git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Shawn O. Pearce" <spearce@spearce.org>
To: Robin Rosenberg <robin.rosenberg@dewire.com>
Cc: git@vger.kernel.org
Subject: [JGIT PATCH v2 08/10] Scan for new packs if GIT_DIR/objects/pack has been modified
Date: Mon, 20 Apr 2009 19:08:07 -0700	[thread overview]
Message-ID: <20090421020807.GA23604@spearce.org> (raw)
In-Reply-To: <20090421013911.GZ23604@spearce.org>

If GIT_DIR/objects/pack directory has changed then one or more packs
may have been added to the directory, or removed from it, due to a
concurrent application performing a repack.  In such cases we need
to reload the list of available packs before conducting a search.

As packs are infrequently created or modified relative to searches,
we only scan for new packs if our search through existing packs has
turned up no results.

Transports that create packs locally through IndexPack automatically
register their new PackFile, so even those cases will typically not
require scanning the object directory again.

As misses are fairly common during IndexPack (due to its automatic
collision detection logic) we avoid scanning for packs unless the
GIT_DIR/objects/pack directory has been modified since the last
scan of it.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---

 "Shawn O. Pearce" <spearce@spearce.org> wrote:
 > 
 > We could try to fudge it by saying < here, [...]
 > I think in real applications, its "good enough" to allow this amount
 > of fudging.

 And this version makes it so.  The interdiff is just <= changed to <.

 .../src/org/spearce/jgit/lib/ObjectDatabase.java   |   15 ++++++++++++++-
 .../src/org/spearce/jgit/lib/ObjectDirectory.java  |   13 +++++++++++++
 2 files changed, 27 insertions(+), 1 deletions(-)

diff --git a/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectDatabase.java b/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectDatabase.java
index ed1290f..ec228a1 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectDatabase.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectDatabase.java
@@ -137,7 +137,7 @@ private final boolean hasObjectImpl1(final AnyObjectId objectId) {
 				return true;
 			}
 		}
-		return false;
+		return tryAgain1() && hasObject1(objectId);
 	}
 
 	private final boolean hasObjectImpl2(final String objectId) {
@@ -216,6 +216,12 @@ private ObjectLoader openObjectImpl1(final WindowCursor curs,
 				return ldr;
 			}
 		}
+		if (tryAgain1()) {
+			ldr = openObject1(curs, objectId);
+			if (ldr != null) {
+				return ldr;
+			}
+		}
 		return null;
 	}
 
@@ -311,6 +317,13 @@ void openObjectInAllPacks1(Collection<PackedObjectLoader> out,
 	}
 
 	/**
+	 * @return true if the fast-half search should be tried again.
+	 */
+	protected boolean tryAgain1() {
+		return false;
+	}
+
+	/**
 	 * Get the alternate databases known to this database.
 	 * 
 	 * @return the alternate list. Never null, but may be an empty array.
diff --git a/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectDirectory.java b/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectDirectory.java
index 36f221e..6ba0180 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectDirectory.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectDirectory.java
@@ -75,6 +75,8 @@
 
 	private final AtomicReference<PackFile[]> packList;
 
+	private volatile long packDirectoryLastModified;
+
 	/**
 	 * Initialize a reference to an on-disk object directory.
 	 * 
@@ -257,6 +259,16 @@ protected ObjectLoader openObject2(final WindowCursor curs,
 		}
 	}
 
+	@Override
+	protected boolean tryAgain1() {
+		final PackFile[] old = packList.get();
+		if (packDirectoryLastModified < packDirectory.lastModified()) {
+			scanPacks(old);
+			return true;
+		}
+		return false;
+	}
+
 	private void insertPack(final PackFile pf) {
 		PackFile[] o, n;
 		do {
@@ -389,6 +401,7 @@ synchronized (packList) {
 	}
 
 	private String[] listPackIdx() {
+		packDirectoryLastModified = packDirectory.lastModified();
 		final String[] idxList = packDirectory.list(new FilenameFilter() {
 			public boolean accept(final File baseDir, final String n) {
 				// Must match "pack-[0-9a-f]{40}.idx" to be an index.
-- 
1.6.3.rc1.188.ga02b

      reply	other threads:[~2009-04-21  2:09 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2009-04-21  1:21 [JGIT PATCH 00/10] Object access improvements Shawn O. Pearce
2009-04-21  1:21 ` [JGIT PATCH 01/10] Safely handle closing an already closed WindowedFile Shawn O. Pearce
2009-04-21  1:21   ` [JGIT PATCH 02/10] Change empty tree test case to use a temporary repository Shawn O. Pearce
2009-04-21  1:21     ` [JGIT PATCH 03/10] Replace hand-coded read fully loop with NB.readFully Shawn O. Pearce
2009-04-21  1:21       ` [JGIT PATCH 04/10] Clear the reverse index when closing a PackFile Shawn O. Pearce
2009-04-21  1:21         ` [JGIT PATCH 05/10] Introduce a new exception type for an in-place pack modification Shawn O. Pearce
2009-04-21  1:21           ` [JGIT PATCH 06/10] Refactor object database access with new abstraction Shawn O. Pearce
2009-04-21  1:21             ` [JGIT PATCH 07/10] Rescan packs if a pack is modified in-place (part 1) Shawn O. Pearce
2009-04-21  1:21               ` [JGIT PATCH 08/10] Scan for new packs if GIT_DIR/objects/pack has been modified Shawn O. Pearce
2009-04-21  1:21                 ` [JGIT PATCH 09/10] Add test cases for loading new (or replaced) pack files Shawn O. Pearce
2009-04-21  1:21                   ` [JGIT PATCH 10/10] BROKEN TEST: ObjectLoader stays valid across repacks Shawn O. Pearce
2009-04-21 23:16                     ` Robin Rosenberg
2009-04-22 16:33                       ` Shawn O. Pearce
2009-04-22 23:02                       ` Shawn O. Pearce
2009-04-21  2:10                   ` [JGIT PATCH v2 09/10] Add test cases for loading new (or replaced) pack files Shawn O. Pearce
2009-04-21  1:39                 ` [JGIT PATCH 08/10] Scan for new packs if GIT_DIR/objects/pack has been modified Shawn O. Pearce
2009-04-21  2:08                   ` Shawn O. Pearce [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20090421020807.GA23604@spearce.org \
    --to=spearce@spearce.org \
    --cc=git@vger.kernel.org \
    --cc=robin.rosenberg@dewire.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).