[PATCH 0/9] Split up "object-file.c"

git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed

* [PATCH 0/9] Split up "object-file.c"
@ 2025-04-08 10:24 Patrick Steinhardt
  2025-04-08 10:24 ` [PATCH 1/9] object-file: move `safe_create_leading_directories()` into "dir.c" Patrick Steinhardt
                   ` (12 more replies)
  0 siblings, 13 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-08 10:24 UTC (permalink / raw)
  To: git

Hi,

"object-file.c" is quite a grab-bag of all kinds of different functions.
Many of these functions aren't really a good fit though and should be
owned by a different subsystem. This patch series tries to split up
concerns a bit better by splitting out this functionality into other
files:

  - `safe_create_leading_directories()` is moved into "dir.c".
  - `xmmap()` is moved into "wrapper.c".
  - `git_open_cloexec()` is moved into "compat/open.c".
  - Several functions attached to `struct index_state` are moved into
    "read-cache.c".
  - Several functions related to `struct object_store` are moved into a
    new file "object-store.c".

"object-file.c" now mostly contains logic to read and write loose object
files, whereas "object-store.c" contains the higher-level logic to
manage different object directories for a repository. Eventually, these
will become the loose object backend as well as the `struct ref_store`
equivalent for objects, respectively.

The series is built on top of 9d22ac51228 (The third batch, 2025-04-07)
with ps/object-wo-the-repository at 9442b1c919a (Merge remote-tracking
branch 'junio/ps/object-wo-the-repository' into HEAD, 2025-04-08) merged
into it.

Thanks!

Patrick

---
Patrick Steinhardt (9):
      object-file: move `safe_create_leading_directories()` into "dir.c"
      object-file: move `git_open_cloexec()` to "compat/open.c"
      object-file: move `xmmap()` into "wrapper.c"
      object-file: split out functions relating to object store subsystem
      object-file: split up concerns of `HASH_*` flags
      object-file: split out functions relating to index subsystem
      object: split out functions relating to object store subsystem
      object-store: remove global array of cached objects
      object-store: merge "object-store-ll.h" and "object-store.h"

 Makefile                           |    3 +-
 apply.c                            |    2 +-
 archive-tar.c                      |    2 +-
 archive-zip.c                      |    2 +-
 archive.c                          |    2 +-
 attr.c                             |    2 +-
 bisect.c                           |    2 +-
 blame.c                            |    4 +-
 builtin/backfill.c                 |    2 +-
 builtin/blame.c                    |    2 +-
 builtin/bugreport.c                |    2 +-
 builtin/cat-file.c                 |    2 +-
 builtin/checkout.c                 |    3 +-
 builtin/clone.c                    |    2 +-
 builtin/commit-graph.c             |    2 +-
 builtin/commit-tree.c              |    2 +-
 builtin/count-objects.c            |    2 +-
 builtin/credential-cache--daemon.c |    2 +-
 builtin/describe.c                 |    2 +-
 builtin/diagnose.c                 |    2 +-
 builtin/difftool.c                 |    4 +-
 builtin/fast-export.c              |    2 +-
 builtin/fast-import.c              |    2 +-
 builtin/fetch.c                    |    2 +-
 builtin/fsck.c                     |    3 +-
 builtin/gc.c                       |    4 +-
 builtin/grep.c                     |    2 +-
 builtin/hash-object.c              |   26 +-
 builtin/index-pack.c               |    2 +-
 builtin/init-db.c                  |    2 +-
 builtin/log.c                      |    4 +-
 builtin/ls-tree.c                  |    2 +-
 builtin/merge-file.c               |    1 +
 builtin/merge-tree.c               |    2 +-
 builtin/mktag.c                    |    2 +-
 builtin/mktree.c                   |    3 +-
 builtin/multi-pack-index.c         |    2 +-
 builtin/notes.c                    |    3 +-
 builtin/pack-objects.c             |    2 +-
 builtin/pack-redundant.c           |    2 +-
 builtin/prune.c                    |    2 +-
 builtin/receive-pack.c             |    3 +-
 builtin/remote.c                   |    2 +-
 builtin/repack.c                   |    2 +-
 builtin/replace.c                  |    5 +-
 builtin/rev-list.c                 |    2 +-
 builtin/show-ref.c                 |    2 +-
 builtin/submodule--helper.c        |    2 +-
 builtin/tag.c                      |    3 +-
 builtin/unpack-file.c              |    3 +-
 builtin/unpack-objects.c           |    3 +-
 builtin/update-index.c             |    2 +-
 bulk-checkin.c                     |    9 +-
 bundle-uri.c                       |    2 +-
 bundle.c                           |    2 +-
 cache-tree.c                       |    4 +-
 combine-diff.c                     |    2 +-
 commit-graph.c                     |    4 +-
 commit-graph.h                     |    2 +-
 commit.c                           |    3 +-
 compat/open.c                      |   29 +
 config.c                           |    2 +-
 connected.c                        |    2 +-
 convert.c                          |    2 +-
 diagnose.c                         |    2 +-
 diff.c                             |    4 +-
 diffcore-rename.c                  |    2 +-
 dir.c                              |  109 ++-
 dir.h                              |   35 +
 entry.c                            |    2 +-
 fetch-pack.c                       |    2 +-
 fmt-merge-msg.c                    |    2 +-
 fsck.c                             |    2 +-
 git-compat-util.h                  |    3 +
 grep.c                             |    2 +-
 http-backend.c                     |    2 +-
 http-push.c                        |    3 +-
 http-walker.c                      |    2 +-
 http.c                             |    2 +-
 list-objects-filter.c              |    2 +-
 list-objects.c                     |    2 +-
 log-tree.c                         |    2 +-
 mailmap.c                          |    2 +-
 match-trees.c                      |    3 +-
 merge-blobs.c                      |    2 +-
 merge-ort.c                        |    3 +-
 merge-recursive.c                  |    2 +-
 meson.build                        |    2 +
 midx-write.c                       |    1 +
 midx.c                             |    1 -
 notes-cache.c                      |    3 +-
 notes-merge.c                      |    5 +-
 notes.c                            |    3 +-
 object-file.c                      | 1420 +-----------------------------------
 object-file.h                      |  118 +--
 object-name.c                      |    2 +-
 object-store-ll.h                  |  556 --------------
 object-store.c                     | 1050 ++++++++++++++++++++++++++
 object-store.h                     |  516 ++++++++++++-
 object.c                           |   67 --
 oss-fuzz/fuzz-pack-idx.c           |    2 +-
 pack-bitmap-write.c                |    2 +-
 pack-bitmap.c                      |    3 +-
 pack-check.c                       |    2 +-
 pack-mtimes.c                      |    3 +-
 pack-objects.h                     |    2 +-
 pack-revindex.c                    |    3 +-
 packfile.c                         |    2 +-
 path.c                             |    2 +-
 promisor-remote.c                  |    2 +-
 protocol-caps.c                    |    2 +-
 prune-packed.c                     |    2 +-
 reachable.c                        |    2 +-
 read-cache.c                       |  230 +++++-
 read-cache.h                       |    9 +
 ref-filter.c                       |    2 +-
 reflog.c                           |    2 +-
 refs.c                             |    2 +-
 remote.c                           |    2 +-
 replace-object.c                   |    2 +-
 replace-object.h                   |    2 +-
 repository.c                       |    2 +-
 rerere.c                           |    2 +-
 revision.c                         |    2 +-
 send-pack.c                        |    2 +-
 sequencer.c                        |    2 +-
 server-info.c                      |    2 +-
 shallow.c                          |    2 +-
 streaming.c                        |    2 +-
 submodule-config.c                 |    2 +-
 submodule.c                        |    2 +-
 t/helper/test-pack-mtimes.c        |    2 +-
 t/helper/test-partial-clone.c      |    2 +-
 t/helper/test-read-graph.c         |    2 +-
 t/helper/test-read-midx.c          |    2 +-
 t/helper/test-ref-store.c          |    2 +-
 tag.c                              |    2 +-
 tmp-objdir.c                       |    2 +-
 tree-walk.c                        |    2 +-
 tree.c                             |    2 +-
 unpack-trees.c                     |    2 +-
 upload-pack.c                      |    2 +-
 walker.c                           |    2 +-
 wrapper.c                          |   48 ++
 xdiff-interface.c                  |    2 +-
 145 files changed, 2296 insertions(+), 2224 deletions(-)


---
base-commit: 9442b1c919af9aed513eb0a484fe96358a500cf5
change-id: 20250408-pks-split-object-file-c61d7cd2a21f


^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH 1/9] object-file: move `safe_create_leading_directories()` into "dir.c"
  2025-04-08 10:24 [PATCH 0/9] Split up "object-file.c" Patrick Steinhardt
@ 2025-04-08 10:24 ` Patrick Steinhardt
  2025-04-09 14:36   ` Elijah Newren
  2025-04-08 10:24 ` [PATCH 2/9] object-file: move `git_open_cloexec()` to "compat/open.c" Patrick Steinhardt
                   ` (11 subsequent siblings)
  12 siblings, 1 reply; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-08 10:24 UTC (permalink / raw)
  To: git

The `safe_create_leading_directories()` function and its relatives are
located in "object-file.c", which is not a good fit as they provide
generic functionality not related to objects at all. Move them into
"dir.c".

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/bugreport.c                |   2 +-
 builtin/credential-cache--daemon.c |   2 +-
 builtin/diagnose.c                 |   2 +-
 builtin/fsck.c                     |   1 +
 builtin/gc.c                       |   2 +-
 builtin/init-db.c                  |   2 +-
 builtin/log.c                      |   2 +-
 commit-graph.c                     |   1 +
 dir.c                              | 107 ++++++++++++++++++++++++++++++++++++-
 dir.h                              |  35 ++++++++++++
 midx-write.c                       |   1 +
 object-file.c                      | 106 ------------------------------------
 object-file.h                      |  35 ------------
 13 files changed, 150 insertions(+), 148 deletions(-)

diff --git a/builtin/bugreport.c b/builtin/bugreport.c
index 66d64bfd5ae..d07fa91c247 100644
--- a/builtin/bugreport.c
+++ b/builtin/bugreport.c
@@ -1,6 +1,7 @@
 #define USE_THE_REPOSITORY_VARIABLE
 #include "builtin.h"
 #include "abspath.h"
+#include "dir.h"
 #include "editor.h"
 #include "gettext.h"
 #include "parse-options.h"
@@ -10,7 +11,6 @@
 #include "hook.h"
 #include "hook-list.h"
 #include "diagnose.h"
-#include "object-file.h"
 #include "setup.h"
 #include "version.h"
 
diff --git a/builtin/credential-cache--daemon.c b/builtin/credential-cache--daemon.c
index e707618e743..80d29b4f5c0 100644
--- a/builtin/credential-cache--daemon.c
+++ b/builtin/credential-cache--daemon.c
@@ -1,8 +1,8 @@
 #define USE_THE_REPOSITORY_VARIABLE
 #include "builtin.h"
 #include "abspath.h"
+#include "dir.h"
 #include "gettext.h"
-#include "object-file.h"
 #include "parse-options.h"
 
 #ifndef NO_UNIX_SOCKETS
diff --git a/builtin/diagnose.c b/builtin/diagnose.c
index 33c39bd5981..d5dadd6a48b 100644
--- a/builtin/diagnose.c
+++ b/builtin/diagnose.c
@@ -2,8 +2,8 @@
 
 #include "builtin.h"
 #include "abspath.h"
+#include "dir.h"
 #include "gettext.h"
-#include "object-file.h"
 #include "parse-options.h"
 #include "diagnose.h"
 
diff --git a/builtin/fsck.c b/builtin/fsck.c
index 9c8a6d6a8df..32d40d8f9fc 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -1,5 +1,6 @@
 #define USE_THE_REPOSITORY_VARIABLE
 #include "builtin.h"
+#include "dir.h"
 #include "gettext.h"
 #include "hex.h"
 #include "config.h"
diff --git a/builtin/gc.c b/builtin/gc.c
index 99431fd4674..b069629676c 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -16,6 +16,7 @@
 #include "builtin.h"
 #include "abspath.h"
 #include "date.h"
+#include "dir.h"
 #include "environment.h"
 #include "hex.h"
 #include "config.h"
@@ -28,7 +29,6 @@
 #include "commit.h"
 #include "commit-graph.h"
 #include "packfile.h"
-#include "object-file.h"
 #include "object-store-ll.h"
 #include "pack.h"
 #include "pack-objects.h"
diff --git a/builtin/init-db.c b/builtin/init-db.c
index 196dccdd77a..39730c1b0ce 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -6,9 +6,9 @@
 #define USE_THE_REPOSITORY_VARIABLE
 #include "builtin.h"
 #include "abspath.h"
+#include "dir.h"
 #include "environment.h"
 #include "gettext.h"
-#include "object-file.h"
 #include "parse-options.h"
 #include "path.h"
 #include "refs.h"
diff --git a/builtin/log.c b/builtin/log.c
index 0d4c579dad7..06ffaa93e86 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -10,11 +10,11 @@
 #include "builtin.h"
 #include "abspath.h"
 #include "config.h"
+#include "dir.h"
 #include "environment.h"
 #include "gettext.h"
 #include "hex.h"
 #include "refs.h"
-#include "object-file.h"
 #include "object-name.h"
 #include "object-store-ll.h"
 #include "pager.h"
diff --git a/commit-graph.c b/commit-graph.c
index 8286d5dda24..3fae20dc21b 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -4,6 +4,7 @@
 #include "git-compat-util.h"
 #include "config.h"
 #include "csum-file.h"
+#include "dir.h"
 #include "gettext.h"
 #include "hex.h"
 #include "lockfile.h"
diff --git a/dir.c b/dir.c
index 28b0e03feb4..16ae3b5169d 100644
--- a/dir.c
+++ b/dir.c
@@ -17,7 +17,6 @@
 #include "environment.h"
 #include "gettext.h"
 #include "name-hash.h"
-#include "object-file.h"
 #include "object-store-ll.h"
 #include "path.h"
 #include "refs.h"
@@ -4132,3 +4131,109 @@ int path_match_flags(const char *const str, const enum path_match_flags flags)
 		return is_xplatform_dir_sep(*p);
 	BUG("unreachable");
 }
+
+int mkdir_in_gitdir(const char *path)
+{
+	if (mkdir(path, 0777)) {
+		int saved_errno = errno;
+		struct stat st;
+		struct strbuf sb = STRBUF_INIT;
+
+		if (errno != EEXIST)
+			return -1;
+		/*
+		 * Are we looking at a path in a symlinked worktree
+		 * whose original repository does not yet have it?
+		 * e.g. .git/rr-cache pointing at its original
+		 * repository in which the user hasn't performed any
+		 * conflict resolution yet?
+		 */
+		if (lstat(path, &st) || !S_ISLNK(st.st_mode) ||
+		    strbuf_readlink(&sb, path, st.st_size) ||
+		    !is_absolute_path(sb.buf) ||
+		    mkdir(sb.buf, 0777)) {
+			strbuf_release(&sb);
+			errno = saved_errno;
+			return -1;
+		}
+		strbuf_release(&sb);
+	}
+	return adjust_shared_perm(the_repository, path);
+}
+
+static enum scld_error safe_create_leading_directories_1(char *path, int share)
+{
+	char *next_component = path + offset_1st_component(path);
+	enum scld_error ret = SCLD_OK;
+
+	while (ret == SCLD_OK && next_component) {
+		struct stat st;
+		char *slash = next_component, slash_character;
+
+		while (*slash && !is_dir_sep(*slash))
+			slash++;
+
+		if (!*slash)
+			break;
+
+		next_component = slash + 1;
+		while (is_dir_sep(*next_component))
+			next_component++;
+		if (!*next_component)
+			break;
+
+		slash_character = *slash;
+		*slash = '\0';
+		if (!stat(path, &st)) {
+			/* path exists */
+			if (!S_ISDIR(st.st_mode)) {
+				errno = ENOTDIR;
+				ret = SCLD_EXISTS;
+			}
+		} else if (mkdir(path, 0777)) {
+			if (errno == EEXIST &&
+			    !stat(path, &st) && S_ISDIR(st.st_mode))
+				; /* somebody created it since we checked */
+			else if (errno == ENOENT)
+				/*
+				 * Either mkdir() failed because
+				 * somebody just pruned the containing
+				 * directory, or stat() failed because
+				 * the file that was in our way was
+				 * just removed.  Either way, inform
+				 * the caller that it might be worth
+				 * trying again:
+				 */
+				ret = SCLD_VANISHED;
+			else
+				ret = SCLD_FAILED;
+		} else if (share && adjust_shared_perm(the_repository, path)) {
+			ret = SCLD_PERMS;
+		}
+		*slash = slash_character;
+	}
+	return ret;
+}
+
+enum scld_error safe_create_leading_directories(char *path)
+{
+	return safe_create_leading_directories_1(path, 1);
+}
+
+enum scld_error safe_create_leading_directories_no_share(char *path)
+{
+	return safe_create_leading_directories_1(path, 0);
+}
+
+enum scld_error safe_create_leading_directories_const(const char *path)
+{
+	int save_errno;
+	/* path points to cache entries, so xstrdup before messing with it */
+	char *buf = xstrdup(path);
+	enum scld_error result = safe_create_leading_directories(buf);
+
+	save_errno = errno;
+	free(buf);
+	errno = save_errno;
+	return result;
+}
diff --git a/dir.h b/dir.h
index d7e71aa8daa..02c1f9420b0 100644
--- a/dir.h
+++ b/dir.h
@@ -676,4 +676,39 @@ static inline int starts_with_dot_dot_slash_native(const char *const path)
 	return path_match_flags(path, what | PATH_MATCH_NATIVE);
 }
 
+/*
+ * Create the directory containing the named path, using care to be
+ * somewhat safe against races. Return one of the scld_error values to
+ * indicate success/failure. On error, set errno to describe the
+ * problem.
+ *
+ * SCLD_VANISHED indicates that one of the ancestor directories of the
+ * path existed at one point during the function call and then
+ * suddenly vanished, probably because another process pruned the
+ * directory while we were working.  To be robust against this kind of
+ * race, callers might want to try invoking the function again when it
+ * returns SCLD_VANISHED.
+ *
+ * safe_create_leading_directories() temporarily changes path while it
+ * is working but restores it before returning.
+ * safe_create_leading_directories_const() doesn't modify path, even
+ * temporarily. Both these variants adjust the permissions of the
+ * created directories to honor core.sharedRepository, so they are best
+ * suited for files inside the git dir. For working tree files, use
+ * safe_create_leading_directories_no_share() instead, as it ignores
+ * the core.sharedRepository setting.
+ */
+enum scld_error {
+	SCLD_OK = 0,
+	SCLD_FAILED = -1,
+	SCLD_PERMS = -2,
+	SCLD_EXISTS = -3,
+	SCLD_VANISHED = -4
+};
+enum scld_error safe_create_leading_directories(char *path);
+enum scld_error safe_create_leading_directories_const(const char *path);
+enum scld_error safe_create_leading_directories_no_share(char *path);
+
+int mkdir_in_gitdir(const char *path);
+
 #endif
diff --git a/midx-write.c b/midx-write.c
index a628ac24dcb..e01a867c583 100644
--- a/midx-write.c
+++ b/midx-write.c
@@ -3,6 +3,7 @@
 #include "git-compat-util.h"
 #include "abspath.h"
 #include "config.h"
+#include "dir.h"
 #include "hex.h"
 #include "lockfile.h"
 #include "packfile.h"
diff --git a/object-file.c b/object-file.c
index 772c311f188..23b2c8560be 100644
--- a/object-file.c
+++ b/object-file.c
@@ -91,112 +91,6 @@ static int get_conv_flags(unsigned flags)
 }
 
 
-int mkdir_in_gitdir(const char *path)
-{
-	if (mkdir(path, 0777)) {
-		int saved_errno = errno;
-		struct stat st;
-		struct strbuf sb = STRBUF_INIT;
-
-		if (errno != EEXIST)
-			return -1;
-		/*
-		 * Are we looking at a path in a symlinked worktree
-		 * whose original repository does not yet have it?
-		 * e.g. .git/rr-cache pointing at its original
-		 * repository in which the user hasn't performed any
-		 * conflict resolution yet?
-		 */
-		if (lstat(path, &st) || !S_ISLNK(st.st_mode) ||
-		    strbuf_readlink(&sb, path, st.st_size) ||
-		    !is_absolute_path(sb.buf) ||
-		    mkdir(sb.buf, 0777)) {
-			strbuf_release(&sb);
-			errno = saved_errno;
-			return -1;
-		}
-		strbuf_release(&sb);
-	}
-	return adjust_shared_perm(the_repository, path);
-}
-
-static enum scld_error safe_create_leading_directories_1(char *path, int share)
-{
-	char *next_component = path + offset_1st_component(path);
-	enum scld_error ret = SCLD_OK;
-
-	while (ret == SCLD_OK && next_component) {
-		struct stat st;
-		char *slash = next_component, slash_character;
-
-		while (*slash && !is_dir_sep(*slash))
-			slash++;
-
-		if (!*slash)
-			break;
-
-		next_component = slash + 1;
-		while (is_dir_sep(*next_component))
-			next_component++;
-		if (!*next_component)
-			break;
-
-		slash_character = *slash;
-		*slash = '\0';
-		if (!stat(path, &st)) {
-			/* path exists */
-			if (!S_ISDIR(st.st_mode)) {
-				errno = ENOTDIR;
-				ret = SCLD_EXISTS;
-			}
-		} else if (mkdir(path, 0777)) {
-			if (errno == EEXIST &&
-			    !stat(path, &st) && S_ISDIR(st.st_mode))
-				; /* somebody created it since we checked */
-			else if (errno == ENOENT)
-				/*
-				 * Either mkdir() failed because
-				 * somebody just pruned the containing
-				 * directory, or stat() failed because
-				 * the file that was in our way was
-				 * just removed.  Either way, inform
-				 * the caller that it might be worth
-				 * trying again:
-				 */
-				ret = SCLD_VANISHED;
-			else
-				ret = SCLD_FAILED;
-		} else if (share && adjust_shared_perm(the_repository, path)) {
-			ret = SCLD_PERMS;
-		}
-		*slash = slash_character;
-	}
-	return ret;
-}
-
-enum scld_error safe_create_leading_directories(char *path)
-{
-	return safe_create_leading_directories_1(path, 1);
-}
-
-enum scld_error safe_create_leading_directories_no_share(char *path)
-{
-	return safe_create_leading_directories_1(path, 0);
-}
-
-enum scld_error safe_create_leading_directories_const(const char *path)
-{
-	int save_errno;
-	/* path points to cache entries, so xstrdup before messing with it */
-	char *buf = xstrdup(path);
-	enum scld_error result = safe_create_leading_directories(buf);
-
-	save_errno = errno;
-	free(buf);
-	errno = save_errno;
-	return result;
-}
-
 int odb_mkstemp(struct strbuf *temp_filename, const char *pattern)
 {
 	int fd;
diff --git a/object-file.h b/object-file.h
index 81b30d269c8..922f2bba8c9 100644
--- a/object-file.h
+++ b/object-file.h
@@ -21,41 +21,6 @@ extern int fetch_if_missing;
 int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);
 int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags);
 
-/*
- * Create the directory containing the named path, using care to be
- * somewhat safe against races. Return one of the scld_error values to
- * indicate success/failure. On error, set errno to describe the
- * problem.
- *
- * SCLD_VANISHED indicates that one of the ancestor directories of the
- * path existed at one point during the function call and then
- * suddenly vanished, probably because another process pruned the
- * directory while we were working.  To be robust against this kind of
- * race, callers might want to try invoking the function again when it
- * returns SCLD_VANISHED.
- *
- * safe_create_leading_directories() temporarily changes path while it
- * is working but restores it before returning.
- * safe_create_leading_directories_const() doesn't modify path, even
- * temporarily. Both these variants adjust the permissions of the
- * created directories to honor core.sharedRepository, so they are best
- * suited for files inside the git dir. For working tree files, use
- * safe_create_leading_directories_no_share() instead, as it ignores
- * the core.sharedRepository setting.
- */
-enum scld_error {
-	SCLD_OK = 0,
-	SCLD_FAILED = -1,
-	SCLD_PERMS = -2,
-	SCLD_EXISTS = -3,
-	SCLD_VANISHED = -4
-};
-enum scld_error safe_create_leading_directories(char *path);
-enum scld_error safe_create_leading_directories_const(const char *path);
-enum scld_error safe_create_leading_directories_no_share(char *path);
-
-int mkdir_in_gitdir(const char *path);
-
 int git_open_cloexec(const char *name, int flags);
 #define git_open(name) git_open_cloexec(name, O_RDONLY)
 

-- 
2.49.0.682.gc9b6a7b2b0.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH 2/9] object-file: move `git_open_cloexec()` to "compat/open.c"
  2025-04-08 10:24 [PATCH 0/9] Split up "object-file.c" Patrick Steinhardt
  2025-04-08 10:24 ` [PATCH 1/9] object-file: move `safe_create_leading_directories()` into "dir.c" Patrick Steinhardt
@ 2025-04-08 10:24 ` Patrick Steinhardt
  2025-04-08 10:24 ` [PATCH 3/9] object-file: move `xmmap()` into "wrapper.c" Patrick Steinhardt
                   ` (10 subsequent siblings)
  12 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-08 10:24 UTC (permalink / raw)
  To: git

The `git_open_cloexec()` wrapper function provides the ability to open a
file with `O_CLOEXEC` in a platform-agnostic way. This function is
provided by "object-file.c" even though it is not specific to the object
subsystem at all.

Move the file into "compat/open.c". This file already exists before this
commit, but has only been compiled conditionally depending on whether or
not open(3p) may return EINTR. With this change we now unconditionally
compile the object, but wrap `git_open_with_retry()` in an ifdef.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Makefile          |  2 +-
 commit-graph.c    |  1 -
 compat/open.c     | 29 +++++++++++++++++++++++++++++
 git-compat-util.h |  3 +++
 meson.build       |  1 +
 midx.c            |  1 -
 object-file.c     | 27 ---------------------------
 object-file.h     |  3 ---
 pack-bitmap.c     |  1 -
 pack-mtimes.c     |  1 -
 pack-revindex.c   |  1 -
 11 files changed, 34 insertions(+), 36 deletions(-)

diff --git a/Makefile b/Makefile
index c41fc41ef0e..bb5407b4703 100644
--- a/Makefile
+++ b/Makefile
@@ -994,6 +994,7 @@ LIB_OBJS += common-exit.o
 LIB_OBJS += common-init.o
 LIB_OBJS += compat/nonblock.o
 LIB_OBJS += compat/obstack.o
+LIB_OBJS += compat/open.o
 LIB_OBJS += compat/terminal.o
 LIB_OBJS += compiler-tricks/not-constant.o
 LIB_OBJS += config.o
@@ -1812,7 +1813,6 @@ ifdef FREAD_READS_DIRECTORIES
 endif
 ifdef OPEN_RETURNS_EINTR
 	COMPAT_CFLAGS += -DOPEN_RETURNS_EINTR
-	COMPAT_OBJS += compat/open.o
 endif
 ifdef NO_SYMLINK_HEAD
 	BASIC_CFLAGS += -DNO_SYMLINK_HEAD
diff --git a/commit-graph.c b/commit-graph.c
index 3fae20dc21b..8060c358b84 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -14,7 +14,6 @@
 #include "refs.h"
 #include "hash-lookup.h"
 #include "commit-graph.h"
-#include "object-file.h"
 #include "object-store-ll.h"
 #include "oid-array.h"
 #include "path.h"
diff --git a/compat/open.c b/compat/open.c
index eb3754a23b8..37ae2b1aeb9 100644
--- a/compat/open.c
+++ b/compat/open.c
@@ -1,5 +1,6 @@
 #include "git-compat-util.h"
 
+#ifdef OPEN_RETURNS_EINTR
 #undef open
 int git_open_with_retry(const char *path, int flags, ...)
 {
@@ -23,3 +24,31 @@ int git_open_with_retry(const char *path, int flags, ...)
 
 	return ret;
 }
+#endif
+
+int git_open_cloexec(const char *name, int flags)
+{
+	int fd;
+	static int o_cloexec = O_CLOEXEC;
+
+	fd = open(name, flags | o_cloexec);
+	if ((o_cloexec & O_CLOEXEC) && fd < 0 && errno == EINVAL) {
+		/* Try again w/o O_CLOEXEC: the kernel might not support it */
+		o_cloexec &= ~O_CLOEXEC;
+		fd = open(name, flags | o_cloexec);
+	}
+
+#if defined(F_GETFD) && defined(F_SETFD) && defined(FD_CLOEXEC)
+	{
+		static int fd_cloexec = FD_CLOEXEC;
+
+		if (!o_cloexec && 0 <= fd && fd_cloexec) {
+			/* Opened w/o O_CLOEXEC?  try with fcntl(2) to add it */
+			int flags = fcntl(fd, F_GETFD);
+			if (fcntl(fd, F_SETFD, flags | fd_cloexec))
+				fd_cloexec = 0;
+		}
+	}
+#endif
+	return fd;
+}
diff --git a/git-compat-util.h b/git-compat-util.h
index cf733b38acd..9273a8ee087 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -1000,6 +1000,9 @@ int git_vsnprintf(char *str, size_t maxsize,
 int git_open_with_retry(const char *path, int flag, ...);
 #endif
 
+int git_open_cloexec(const char *name, int flags);
+#define git_open(name) git_open_cloexec(name, O_RDONLY)
+
 #ifdef __GLIBC_PREREQ
 #if __GLIBC_PREREQ(2, 1)
 #define HAVE_STRCHRNUL
diff --git a/meson.build b/meson.build
index 145d2f7ff9e..a55e800b85b 100644
--- a/meson.build
+++ b/meson.build
@@ -263,6 +263,7 @@ libgit_sources = [
   'common-init.c',
   'compat/nonblock.c',
   'compat/obstack.c',
+  'compat/open.c',
   'compat/terminal.c',
   'compiler-tricks/not-constant.c',
   'config.c',
diff --git a/midx.c b/midx.c
index 807fdf72f7b..3d0015f7828 100644
--- a/midx.c
+++ b/midx.c
@@ -5,7 +5,6 @@
 #include "dir.h"
 #include "hex.h"
 #include "packfile.h"
-#include "object-file.h"
 #include "hash-lookup.h"
 #include "midx.h"
 #include "progress.h"
diff --git a/object-file.c b/object-file.c
index 23b2c8560be..1a20c7fa072 100644
--- a/object-file.c
+++ b/object-file.c
@@ -834,33 +834,6 @@ int stream_object_signature(struct repository *r, const struct object_id *oid)
 	return !oideq(oid, &real_oid) ? -1 : 0;
 }
 
-int git_open_cloexec(const char *name, int flags)
-{
-	int fd;
-	static int o_cloexec = O_CLOEXEC;
-
-	fd = open(name, flags | o_cloexec);
-	if ((o_cloexec & O_CLOEXEC) && fd < 0 && errno == EINVAL) {
-		/* Try again w/o O_CLOEXEC: the kernel might not support it */
-		o_cloexec &= ~O_CLOEXEC;
-		fd = open(name, flags | o_cloexec);
-	}
-
-#if defined(F_GETFD) && defined(F_SETFD) && defined(FD_CLOEXEC)
-	{
-		static int fd_cloexec = FD_CLOEXEC;
-
-		if (!o_cloexec && 0 <= fd && fd_cloexec) {
-			/* Opened w/o O_CLOEXEC?  try with fcntl(2) to add it */
-			int flags = fcntl(fd, F_GETFD);
-			if (fcntl(fd, F_SETFD, flags | fd_cloexec))
-				fd_cloexec = 0;
-		}
-	}
-#endif
-	return fd;
-}
-
 /*
  * Find "oid" as a loose object in the local repository or in an alternate.
  * Returns 0 on success, negative on failure.
diff --git a/object-file.h b/object-file.h
index 922f2bba8c9..353d8a85c33 100644
--- a/object-file.h
+++ b/object-file.h
@@ -21,9 +21,6 @@ extern int fetch_if_missing;
 int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);
 int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags);
 
-int git_open_cloexec(const char *name, int flags);
-#define git_open(name) git_open_cloexec(name, O_RDONLY)
-
 /**
  * unpack_loose_header() initializes the data stream needed to unpack
  * a loose object header.
diff --git a/pack-bitmap.c b/pack-bitmap.c
index 7fd78c634ef..0dbd7c4ffe1 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -17,7 +17,6 @@
 #include "packfile.h"
 #include "repository.h"
 #include "trace2.h"
-#include "object-file.h"
 #include "object-store-ll.h"
 #include "list-objects-filter-options.h"
 #include "midx.h"
diff --git a/pack-mtimes.c b/pack-mtimes.c
index cdf30b8d2b0..bcea28e521d 100644
--- a/pack-mtimes.c
+++ b/pack-mtimes.c
@@ -1,7 +1,6 @@
 #include "git-compat-util.h"
 #include "gettext.h"
 #include "pack-mtimes.h"
-#include "object-file.h"
 #include "object-store-ll.h"
 #include "packfile.h"
 #include "strbuf.h"
diff --git a/pack-revindex.c b/pack-revindex.c
index 038e0c96b1c..1ee7b49e206 100644
--- a/pack-revindex.c
+++ b/pack-revindex.c
@@ -1,7 +1,6 @@
 #include "git-compat-util.h"
 #include "gettext.h"
 #include "pack-revindex.h"
-#include "object-file.h"
 #include "object-store-ll.h"
 #include "packfile.h"
 #include "strbuf.h"

-- 
2.49.0.682.gc9b6a7b2b0.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH 3/9] object-file: move `xmmap()` into "wrapper.c"
  2025-04-08 10:24 [PATCH 0/9] Split up "object-file.c" Patrick Steinhardt
  2025-04-08 10:24 ` [PATCH 1/9] object-file: move `safe_create_leading_directories()` into "dir.c" Patrick Steinhardt
  2025-04-08 10:24 ` [PATCH 2/9] object-file: move `git_open_cloexec()` to "compat/open.c" Patrick Steinhardt
@ 2025-04-08 10:24 ` Patrick Steinhardt
  2025-04-09 14:36   ` Elijah Newren
  2025-04-08 10:24 ` [PATCH 4/9] object-file: split out functions relating to object store subsystem Patrick Steinhardt
                   ` (9 subsequent siblings)
  12 siblings, 1 reply; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-08 10:24 UTC (permalink / raw)
  To: git

The `xmmap()` function is provided by "object-file.c" even though its
functionality got nothing to do with the object file subsystem. Move it
into "wrapper.c", whose header already declares those functions.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 object-file.c | 48 ------------------------------------------------
 wrapper.c     | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 48 insertions(+), 48 deletions(-)

diff --git a/object-file.c b/object-file.c
index 1a20c7fa072..ea2ed7628e6 100644
--- a/object-file.c
+++ b/object-file.c
@@ -719,54 +719,6 @@ int has_loose_object(const struct object_id *oid)
 	return check_and_freshen(oid, 0);
 }
 
-static void mmap_limit_check(size_t length)
-{
-	static size_t limit = 0;
-	if (!limit) {
-		limit = git_env_ulong("GIT_MMAP_LIMIT", 0);
-		if (!limit)
-			limit = SIZE_MAX;
-	}
-	if (length > limit)
-		die(_("attempting to mmap %"PRIuMAX" over limit %"PRIuMAX),
-		    (uintmax_t)length, (uintmax_t)limit);
-}
-
-void *xmmap_gently(void *start, size_t length,
-		  int prot, int flags, int fd, off_t offset)
-{
-	void *ret;
-
-	mmap_limit_check(length);
-	ret = mmap(start, length, prot, flags, fd, offset);
-	if (ret == MAP_FAILED && !length)
-		ret = NULL;
-	return ret;
-}
-
-const char *mmap_os_err(void)
-{
-	static const char blank[] = "";
-#if defined(__linux__)
-	if (errno == ENOMEM) {
-		/* this continues an existing error message: */
-		static const char enomem[] =
-", check sys.vm.max_map_count and/or RLIMIT_DATA";
-		return enomem;
-	}
-#endif /* OS-specific bits */
-	return blank;
-}
-
-void *xmmap(void *start, size_t length,
-	int prot, int flags, int fd, off_t offset)
-{
-	void *ret = xmmap_gently(start, length, prot, flags, fd, offset);
-	if (ret == MAP_FAILED)
-		die_errno(_("mmap failed%s"), mmap_os_err());
-	return ret;
-}
-
 static int format_object_header_literally(char *str, size_t size,
 					  const char *type, size_t objsize)
 {
diff --git a/wrapper.c b/wrapper.c
index 8b985931490..3c79778055e 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -829,3 +829,51 @@ uint32_t git_rand(unsigned flags)
 
 	return result;
 }
+
+static void mmap_limit_check(size_t length)
+{
+	static size_t limit = 0;
+	if (!limit) {
+		limit = git_env_ulong("GIT_MMAP_LIMIT", 0);
+		if (!limit)
+			limit = SIZE_MAX;
+	}
+	if (length > limit)
+		die(_("attempting to mmap %"PRIuMAX" over limit %"PRIuMAX),
+		    (uintmax_t)length, (uintmax_t)limit);
+}
+
+void *xmmap_gently(void *start, size_t length,
+		  int prot, int flags, int fd, off_t offset)
+{
+	void *ret;
+
+	mmap_limit_check(length);
+	ret = mmap(start, length, prot, flags, fd, offset);
+	if (ret == MAP_FAILED && !length)
+		ret = NULL;
+	return ret;
+}
+
+const char *mmap_os_err(void)
+{
+	static const char blank[] = "";
+#if defined(__linux__)
+	if (errno == ENOMEM) {
+		/* this continues an existing error message: */
+		static const char enomem[] =
+", check sys.vm.max_map_count and/or RLIMIT_DATA";
+		return enomem;
+	}
+#endif /* OS-specific bits */
+	return blank;
+}
+
+void *xmmap(void *start, size_t length,
+	int prot, int flags, int fd, off_t offset)
+{
+	void *ret = xmmap_gently(start, length, prot, flags, fd, offset);
+	if (ret == MAP_FAILED)
+		die_errno(_("mmap failed%s"), mmap_os_err());
+	return ret;
+}

-- 
2.49.0.682.gc9b6a7b2b0.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH 4/9] object-file: split out functions relating to object store subsystem
  2025-04-08 10:24 [PATCH 0/9] Split up "object-file.c" Patrick Steinhardt
                   ` (2 preceding siblings ...)
  2025-04-08 10:24 ` [PATCH 3/9] object-file: move `xmmap()` into "wrapper.c" Patrick Steinhardt
@ 2025-04-08 10:24 ` Patrick Steinhardt
  2025-04-08 10:24 ` [PATCH 5/9] object-file: split up concerns of `HASH_*` flags Patrick Steinhardt
                   ` (8 subsequent siblings)
  12 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-08 10:24 UTC (permalink / raw)
  To: git

While we have the "object-store.h" header, most of the functionality for
object stores is actually hosted in "object-file.c". This makes it hard
to find relevant functions and causes us to mix up concerns.

Split out functions relating to the object store subsystem into a new
"object-store.c" file.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Makefile                 |   1 +
 builtin/checkout.c       |   1 +
 builtin/merge-file.c     |   1 +
 builtin/mktree.c         |   1 +
 builtin/notes.c          |   1 +
 builtin/receive-pack.c   |   1 +
 builtin/tag.c            |   1 +
 builtin/unpack-file.c    |   1 +
 builtin/unpack-objects.c |   1 +
 commit.c                 |   1 +
 http-push.c              |   1 +
 match-trees.c            |   3 +-
 merge-ort.c              |   3 +-
 meson.build              |   1 +
 notes-cache.c            |   3 +-
 notes.c                  |   3 +-
 object-file.c            | 991 +----------------------------------------------
 object-file.h            |  68 +++-
 object-store-ll.h        |  60 +--
 object-store.c           | 972 ++++++++++++++++++++++++++++++++++++++++++++++
 20 files changed, 1074 insertions(+), 1041 deletions(-)

diff --git a/Makefile b/Makefile
index bb5407b4703..d8ad76fb791 100644
--- a/Makefile
+++ b/Makefile
@@ -1086,6 +1086,7 @@ LIB_OBJS += notes.o
 LIB_OBJS += object-file-convert.o
 LIB_OBJS += object-file.o
 LIB_OBJS += object-name.o
+LIB_OBJS += object-store.o
 LIB_OBJS += object.o
 LIB_OBJS += oid-array.o
 LIB_OBJS += oidmap.o
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 2e7486cf658..3e68623838a 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -18,6 +18,7 @@
 #include "lockfile.h"
 #include "mem-pool.h"
 #include "merge-recursive.h"
+#include "object-file.h"
 #include "object-name.h"
 #include "object-store-ll.h"
 #include "parse-options.h"
diff --git a/builtin/merge-file.c b/builtin/merge-file.c
index 7e315f374b2..2b16b10d2ca 100644
--- a/builtin/merge-file.c
+++ b/builtin/merge-file.c
@@ -5,6 +5,7 @@
 #include "abspath.h"
 #include "diff.h"
 #include "hex.h"
+#include "object-file.h"
 #include "object-name.h"
 #include "object-store.h"
 #include "config.h"
diff --git a/builtin/mktree.c b/builtin/mktree.c
index 3c16faa40e3..0644f951161 100644
--- a/builtin/mktree.c
+++ b/builtin/mktree.c
@@ -11,6 +11,7 @@
 #include "strbuf.h"
 #include "tree.h"
 #include "parse-options.h"
+#include "object-file.h"
 #include "object-store-ll.h"
 
 static struct treeent {
diff --git a/builtin/notes.c b/builtin/notes.c
index ff61ec5f2da..0dbc233752d 100644
--- a/builtin/notes.c
+++ b/builtin/notes.c
@@ -14,6 +14,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "notes.h"
+#include "object-file.h"
 #include "object-name.h"
 #include "object-store-ll.h"
 #include "path.h"
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index b3e2a9d0c60..ee51bd76f60 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -31,6 +31,7 @@
 #include "tmp-objdir.h"
 #include "oidset.h"
 #include "packfile.h"
+#include "object-file.h"
 #include "object-name.h"
 #include "object-store-ll.h"
 #include "path.h"
diff --git a/builtin/tag.c b/builtin/tag.c
index 7c173535cb3..536a01ff3ae 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -17,6 +17,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "refs.h"
+#include "object-file.h"
 #include "object-name.h"
 #include "object-store-ll.h"
 #include "path.h"
diff --git a/builtin/unpack-file.c b/builtin/unpack-file.c
index fb5fcbc40a8..b19e5cabd03 100644
--- a/builtin/unpack-file.c
+++ b/builtin/unpack-file.c
@@ -2,6 +2,7 @@
 #include "builtin.h"
 #include "config.h"
 #include "hex.h"
+#include "object-file.h"
 #include "object-name.h"
 #include "object-store-ll.h"
 
diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c
index 3bbcaf2de9b..4078eab9252 100644
--- a/builtin/unpack-objects.c
+++ b/builtin/unpack-objects.c
@@ -8,6 +8,7 @@
 #include "gettext.h"
 #include "git-zlib.h"
 #include "hex.h"
+#include "object-file.h"
 #include "object-store-ll.h"
 #include "object.h"
 #include "delta.h"
diff --git a/commit.c b/commit.c
index 48aeefaad31..fbf4f8e87fd 100644
--- a/commit.c
+++ b/commit.c
@@ -29,6 +29,7 @@
 #include "tree.h"
 #include "hook.h"
 #include "parse.h"
+#include "object-file.h"
 #include "object-file-convert.h"
 
 static struct commit_extra_header *read_commit_extra_header_lines(const char *buf, size_t len, const char **);
diff --git a/http-push.c b/http-push.c
index 1b030d96f48..806eb67cf1b 100644
--- a/http-push.c
+++ b/http-push.c
@@ -19,6 +19,7 @@
 #include "tree-walk.h"
 #include "url.h"
 #include "packfile.h"
+#include "object-file.h"
 #include "object-store-ll.h"
 #include "commit-reach.h"
 
diff --git a/match-trees.c b/match-trees.c
index ef14ceb594c..72922d5d64e 100644
--- a/match-trees.c
+++ b/match-trees.c
@@ -6,7 +6,8 @@
 #include "strbuf.h"
 #include "tree.h"
 #include "tree-walk.h"
-#include "object-store-ll.h"
+#include "object-file.h"
+#include "object-store.h"
 #include "repository.h"
 
 static int score_missing(unsigned mode)
diff --git a/merge-ort.c b/merge-ort.c
index 1554900d984..5de3e2fc159 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -36,8 +36,9 @@
 #include "merge-ll.h"
 #include "match-trees.h"
 #include "mem-pool.h"
+#include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "path.h"
 #include "promisor-remote.h"
diff --git a/meson.build b/meson.build
index a55e800b85b..f656f978020 100644
--- a/meson.build
+++ b/meson.build
@@ -355,6 +355,7 @@ libgit_sources = [
   'object-file-convert.c',
   'object-file.c',
   'object-name.c',
+  'object-store.c',
   'object.c',
   'oid-array.c',
   'oidmap.c',
diff --git a/notes-cache.c b/notes-cache.c
index ecfdf6e43b5..150241b15e0 100644
--- a/notes-cache.c
+++ b/notes-cache.c
@@ -2,7 +2,8 @@
 
 #include "git-compat-util.h"
 #include "notes-cache.h"
-#include "object-store-ll.h"
+#include "object-file.h"
+#include "object-store.h"
 #include "pretty.h"
 #include "repository.h"
 #include "commit.h"
diff --git a/notes.c b/notes.c
index ce5a1006a83..d9645c4b5dc 100644
--- a/notes.c
+++ b/notes.c
@@ -6,8 +6,9 @@
 #include "environment.h"
 #include "hex.h"
 #include "notes.h"
+#include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "utf8.h"
 #include "strbuf.h"
 #include "tree-walk.h"
diff --git a/object-file.c b/object-file.c
index ea2ed7628e6..1c74aa239fb 100644
--- a/object-file.c
+++ b/object-file.c
@@ -11,75 +11,26 @@
 #define DISABLE_SIGN_COMPARE_WARNINGS
 
 #include "git-compat-util.h"
-#include "abspath.h"
-#include "config.h"
+#include "bulk-checkin.h"
 #include "convert.h"
 #include "environment.h"
+#include "fsck.h"
 #include "gettext.h"
 #include "hex.h"
-#include "string-list.h"
-#include "lockfile.h"
-#include "pack.h"
-#include "commit.h"
-#include "run-command.h"
-#include "refs.h"
-#include "bulk-checkin.h"
-#include "repository.h"
-#include "replace-object.h"
-#include "streaming.h"
-#include "dir.h"
-#include "list.h"
-#include "quote.h"
-#include "packfile.h"
+#include "loose.h"
+#include "object-file-convert.h"
 #include "object-file.h"
 #include "object-store.h"
 #include "oidtree.h"
+#include "pack.h"
+#include "packfile.h"
 #include "path.h"
-#include "promisor-remote.h"
 #include "setup.h"
-#include "submodule.h"
-#include "fsck.h"
-#include "loose.h"
-#include "object-file-convert.h"
+#include "streaming.h"
 
 /* The maximum size for an object header. */
 #define MAX_HEADER_LEN 32
 
-/*
- * This is meant to hold a *small* number of objects that you would
- * want repo_read_object_file() to be able to return, but yet you do not want
- * to write them into the object store (e.g. a browse-only
- * application).
- */
-static struct cached_object_entry {
-	struct object_id oid;
-	struct cached_object {
-		enum object_type type;
-		const void *buf;
-		unsigned long size;
-	} value;
-} *cached_objects;
-static int cached_object_nr, cached_object_alloc;
-
-static const struct cached_object *find_cached_object(const struct object_id *oid)
-{
-	static const struct cached_object empty_tree = {
-		.type = OBJ_TREE,
-		.buf = "",
-	};
-	int i;
-	const struct cached_object_entry *co = cached_objects;
-
-	for (i = 0; i < cached_object_nr; i++, co++) {
-		if (oideq(&co->oid, oid))
-			return &co->value;
-	}
-	if (oideq(oid, the_hash_algo->empty_tree))
-		return &empty_tree;
-	return NULL;
-}
-
-
 static int get_conv_flags(unsigned flags)
 {
 	if (flags & HASH_RENORMALIZE)
@@ -90,40 +41,6 @@ static int get_conv_flags(unsigned flags)
 		return 0;
 }
 
-
-int odb_mkstemp(struct strbuf *temp_filename, const char *pattern)
-{
-	int fd;
-	/*
-	 * we let the umask do its job, don't try to be more
-	 * restrictive except to remove write permission.
-	 */
-	int mode = 0444;
-	repo_git_path_replace(the_repository, temp_filename, "objects/%s", pattern);
-	fd = git_mkstemp_mode(temp_filename->buf, mode);
-	if (0 <= fd)
-		return fd;
-
-	/* slow path */
-	/* some mkstemp implementations erase temp_filename on failure */
-	repo_git_path_replace(the_repository, temp_filename, "objects/%s", pattern);
-	safe_create_leading_directories(temp_filename->buf);
-	return xmkstemp_mode(temp_filename->buf, mode);
-}
-
-int odb_pack_keep(const char *name)
-{
-	int fd;
-
-	fd = open(name, O_RDWR|O_CREAT|O_EXCL, 0600);
-	if (0 <= fd)
-		return fd;
-
-	/* slow path */
-	safe_create_leading_directories_const(name);
-	return open(name, O_RDWR|O_CREAT|O_EXCL, 0600);
-}
-
 static void fill_loose_path(struct strbuf *buf, const struct object_id *oid)
 {
 	int i;
@@ -137,9 +54,9 @@ static void fill_loose_path(struct strbuf *buf, const struct object_id *oid)
 	}
 }
 
-static const char *odb_loose_path(struct object_directory *odb,
-				  struct strbuf *buf,
-				  const struct object_id *oid)
+const char *odb_loose_path(struct object_directory *odb,
+			   struct strbuf *buf,
+			   const struct object_id *oid)
 {
 	strbuf_reset(buf);
 	strbuf_addstr(buf, odb->path);
@@ -148,513 +65,6 @@ static const char *odb_loose_path(struct object_directory *odb,
 	return buf->buf;
 }
 
-const char *loose_object_path(struct repository *r, struct strbuf *buf,
-			      const struct object_id *oid)
-{
-	return odb_loose_path(r->objects->odb, buf, oid);
-}
-
-/*
- * Return non-zero iff the path is usable as an alternate object database.
- */
-static int alt_odb_usable(struct raw_object_store *o,
-			  struct strbuf *path,
-			  const char *normalized_objdir, khiter_t *pos)
-{
-	int r;
-
-	/* Detect cases where alternate disappeared */
-	if (!is_directory(path->buf)) {
-		error(_("object directory %s does not exist; "
-			"check .git/objects/info/alternates"),
-		      path->buf);
-		return 0;
-	}
-
-	/*
-	 * Prevent the common mistake of listing the same
-	 * thing twice, or object directory itself.
-	 */
-	if (!o->odb_by_path) {
-		khiter_t p;
-
-		o->odb_by_path = kh_init_odb_path_map();
-		assert(!o->odb->next);
-		p = kh_put_odb_path_map(o->odb_by_path, o->odb->path, &r);
-		assert(r == 1); /* never used */
-		kh_value(o->odb_by_path, p) = o->odb;
-	}
-	if (fspatheq(path->buf, normalized_objdir))
-		return 0;
-	*pos = kh_put_odb_path_map(o->odb_by_path, path->buf, &r);
-	/* r: 0 = exists, 1 = never used, 2 = deleted */
-	return r == 0 ? 0 : 1;
-}
-
-/*
- * Prepare alternate object database registry.
- *
- * The variable alt_odb_list points at the list of struct
- * object_directory.  The elements on this list come from
- * non-empty elements from colon separated ALTERNATE_DB_ENVIRONMENT
- * environment variable, and $GIT_OBJECT_DIRECTORY/info/alternates,
- * whose contents is similar to that environment variable but can be
- * LF separated.  Its base points at a statically allocated buffer that
- * contains "/the/directory/corresponding/to/.git/objects/...", while
- * its name points just after the slash at the end of ".git/objects/"
- * in the example above, and has enough space to hold all hex characters
- * of the object ID, an extra slash for the first level indirection, and
- * the terminating NUL.
- */
-static void read_info_alternates(struct repository *r,
-				 const char *relative_base,
-				 int depth);
-static int link_alt_odb_entry(struct repository *r, const struct strbuf *entry,
-	const char *relative_base, int depth, const char *normalized_objdir)
-{
-	struct object_directory *ent;
-	struct strbuf pathbuf = STRBUF_INIT;
-	struct strbuf tmp = STRBUF_INIT;
-	khiter_t pos;
-	int ret = -1;
-
-	if (!is_absolute_path(entry->buf) && relative_base) {
-		strbuf_realpath(&pathbuf, relative_base, 1);
-		strbuf_addch(&pathbuf, '/');
-	}
-	strbuf_addbuf(&pathbuf, entry);
-
-	if (!strbuf_realpath(&tmp, pathbuf.buf, 0)) {
-		error(_("unable to normalize alternate object path: %s"),
-		      pathbuf.buf);
-		goto error;
-	}
-	strbuf_swap(&pathbuf, &tmp);
-
-	/*
-	 * The trailing slash after the directory name is given by
-	 * this function at the end. Remove duplicates.
-	 */
-	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
-		strbuf_setlen(&pathbuf, pathbuf.len - 1);
-
-	if (!alt_odb_usable(r->objects, &pathbuf, normalized_objdir, &pos))
-		goto error;
-
-	CALLOC_ARRAY(ent, 1);
-	/* pathbuf.buf is already in r->objects->odb_by_path */
-	ent->path = strbuf_detach(&pathbuf, NULL);
-
-	/* add the alternate entry */
-	*r->objects->odb_tail = ent;
-	r->objects->odb_tail = &(ent->next);
-	ent->next = NULL;
-	assert(r->objects->odb_by_path);
-	kh_value(r->objects->odb_by_path, pos) = ent;
-
-	/* recursively add alternates */
-	read_info_alternates(r, ent->path, depth + 1);
-	ret = 0;
- error:
-	strbuf_release(&tmp);
-	strbuf_release(&pathbuf);
-	return ret;
-}
-
-static const char *parse_alt_odb_entry(const char *string,
-				       int sep,
-				       struct strbuf *out)
-{
-	const char *end;
-
-	strbuf_reset(out);
-
-	if (*string == '#') {
-		/* comment; consume up to next separator */
-		end = strchrnul(string, sep);
-	} else if (*string == '"' && !unquote_c_style(out, string, &end)) {
-		/*
-		 * quoted path; unquote_c_style has copied the
-		 * data for us and set "end". Broken quoting (e.g.,
-		 * an entry that doesn't end with a quote) falls
-		 * back to the unquoted case below.
-		 */
-	} else {
-		/* normal, unquoted path */
-		end = strchrnul(string, sep);
-		strbuf_add(out, string, end - string);
-	}
-
-	if (*end)
-		end++;
-	return end;
-}
-
-static void link_alt_odb_entries(struct repository *r, const char *alt,
-				 int sep, const char *relative_base, int depth)
-{
-	struct strbuf objdirbuf = STRBUF_INIT;
-	struct strbuf entry = STRBUF_INIT;
-
-	if (!alt || !*alt)
-		return;
-
-	if (depth > 5) {
-		error(_("%s: ignoring alternate object stores, nesting too deep"),
-				relative_base);
-		return;
-	}
-
-	strbuf_realpath(&objdirbuf, r->objects->odb->path, 1);
-
-	while (*alt) {
-		alt = parse_alt_odb_entry(alt, sep, &entry);
-		if (!entry.len)
-			continue;
-		link_alt_odb_entry(r, &entry,
-				   relative_base, depth, objdirbuf.buf);
-	}
-	strbuf_release(&entry);
-	strbuf_release(&objdirbuf);
-}
-
-static void read_info_alternates(struct repository *r,
-				 const char *relative_base,
-				 int depth)
-{
-	char *path;
-	struct strbuf buf = STRBUF_INIT;
-
-	path = xstrfmt("%s/info/alternates", relative_base);
-	if (strbuf_read_file(&buf, path, 1024) < 0) {
-		warn_on_fopen_errors(path);
-		free(path);
-		return;
-	}
-
-	link_alt_odb_entries(r, buf.buf, '\n', relative_base, depth);
-	strbuf_release(&buf);
-	free(path);
-}
-
-void add_to_alternates_file(const char *reference)
-{
-	struct lock_file lock = LOCK_INIT;
-	char *alts = repo_git_path(the_repository, "objects/info/alternates");
-	FILE *in, *out;
-	int found = 0;
-
-	hold_lock_file_for_update(&lock, alts, LOCK_DIE_ON_ERROR);
-	out = fdopen_lock_file(&lock, "w");
-	if (!out)
-		die_errno(_("unable to fdopen alternates lockfile"));
-
-	in = fopen(alts, "r");
-	if (in) {
-		struct strbuf line = STRBUF_INIT;
-
-		while (strbuf_getline(&line, in) != EOF) {
-			if (!strcmp(reference, line.buf)) {
-				found = 1;
-				break;
-			}
-			fprintf_or_die(out, "%s\n", line.buf);
-		}
-
-		strbuf_release(&line);
-		fclose(in);
-	}
-	else if (errno != ENOENT)
-		die_errno(_("unable to read alternates file"));
-
-	if (found) {
-		rollback_lock_file(&lock);
-	} else {
-		fprintf_or_die(out, "%s\n", reference);
-		if (commit_lock_file(&lock))
-			die_errno(_("unable to move new alternates file into place"));
-		if (the_repository->objects->loaded_alternates)
-			link_alt_odb_entries(the_repository, reference,
-					     '\n', NULL, 0);
-	}
-	free(alts);
-}
-
-void add_to_alternates_memory(const char *reference)
-{
-	/*
-	 * Make sure alternates are initialized, or else our entry may be
-	 * overwritten when they are.
-	 */
-	prepare_alt_odb(the_repository);
-
-	link_alt_odb_entries(the_repository, reference,
-			     '\n', NULL, 0);
-}
-
-struct object_directory *set_temporary_primary_odb(const char *dir, int will_destroy)
-{
-	struct object_directory *new_odb;
-
-	/*
-	 * Make sure alternates are initialized, or else our entry may be
-	 * overwritten when they are.
-	 */
-	prepare_alt_odb(the_repository);
-
-	/*
-	 * Make a new primary odb and link the old primary ODB in as an
-	 * alternate
-	 */
-	new_odb = xcalloc(1, sizeof(*new_odb));
-	new_odb->path = xstrdup(dir);
-
-	/*
-	 * Disable ref updates while a temporary odb is active, since
-	 * the objects in the database may roll back.
-	 */
-	new_odb->disable_ref_updates = 1;
-	new_odb->will_destroy = will_destroy;
-	new_odb->next = the_repository->objects->odb;
-	the_repository->objects->odb = new_odb;
-	return new_odb->next;
-}
-
-void restore_primary_odb(struct object_directory *restore_odb, const char *old_path)
-{
-	struct object_directory *cur_odb = the_repository->objects->odb;
-
-	if (strcmp(old_path, cur_odb->path))
-		BUG("expected %s as primary object store; found %s",
-		    old_path, cur_odb->path);
-
-	if (cur_odb->next != restore_odb)
-		BUG("we expect the old primary object store to be the first alternate");
-
-	the_repository->objects->odb = restore_odb;
-	free_object_directory(cur_odb);
-}
-
-/*
- * Compute the exact path an alternate is at and returns it. In case of
- * error NULL is returned and the human readable error is added to `err`
- * `path` may be relative and should point to $GIT_DIR.
- * `err` must not be null.
- */
-char *compute_alternate_path(const char *path, struct strbuf *err)
-{
-	char *ref_git = NULL;
-	const char *repo;
-	int seen_error = 0;
-
-	ref_git = real_pathdup(path, 0);
-	if (!ref_git) {
-		seen_error = 1;
-		strbuf_addf(err, _("path '%s' does not exist"), path);
-		goto out;
-	}
-
-	repo = read_gitfile(ref_git);
-	if (!repo)
-		repo = read_gitfile(mkpath("%s/.git", ref_git));
-	if (repo) {
-		free(ref_git);
-		ref_git = xstrdup(repo);
-	}
-
-	if (!repo && is_directory(mkpath("%s/.git/objects", ref_git))) {
-		char *ref_git_git = mkpathdup("%s/.git", ref_git);
-		free(ref_git);
-		ref_git = ref_git_git;
-	} else if (!is_directory(mkpath("%s/objects", ref_git))) {
-		struct strbuf sb = STRBUF_INIT;
-		seen_error = 1;
-		if (get_common_dir(&sb, ref_git)) {
-			strbuf_addf(err,
-				    _("reference repository '%s' as a linked "
-				      "checkout is not supported yet."),
-				    path);
-			goto out;
-		}
-
-		strbuf_addf(err, _("reference repository '%s' is not a "
-					"local repository."), path);
-		goto out;
-	}
-
-	if (!access(mkpath("%s/shallow", ref_git), F_OK)) {
-		strbuf_addf(err, _("reference repository '%s' is shallow"),
-			    path);
-		seen_error = 1;
-		goto out;
-	}
-
-	if (!access(mkpath("%s/info/grafts", ref_git), F_OK)) {
-		strbuf_addf(err,
-			    _("reference repository '%s' is grafted"),
-			    path);
-		seen_error = 1;
-		goto out;
-	}
-
-out:
-	if (seen_error) {
-		FREE_AND_NULL(ref_git);
-	}
-
-	return ref_git;
-}
-
-struct object_directory *find_odb(struct repository *r, const char *obj_dir)
-{
-	struct object_directory *odb;
-	char *obj_dir_real = real_pathdup(obj_dir, 1);
-	struct strbuf odb_path_real = STRBUF_INIT;
-
-	prepare_alt_odb(r);
-	for (odb = r->objects->odb; odb; odb = odb->next) {
-		strbuf_realpath(&odb_path_real, odb->path, 1);
-		if (!strcmp(obj_dir_real, odb_path_real.buf))
-			break;
-	}
-
-	free(obj_dir_real);
-	strbuf_release(&odb_path_real);
-
-	if (!odb)
-		die(_("could not find object directory matching %s"), obj_dir);
-	return odb;
-}
-
-static void fill_alternate_refs_command(struct child_process *cmd,
-					const char *repo_path)
-{
-	const char *value;
-
-	if (!git_config_get_value("core.alternateRefsCommand", &value)) {
-		cmd->use_shell = 1;
-
-		strvec_push(&cmd->args, value);
-		strvec_push(&cmd->args, repo_path);
-	} else {
-		cmd->git_cmd = 1;
-
-		strvec_pushf(&cmd->args, "--git-dir=%s", repo_path);
-		strvec_push(&cmd->args, "for-each-ref");
-		strvec_push(&cmd->args, "--format=%(objectname)");
-
-		if (!git_config_get_value("core.alternateRefsPrefixes", &value)) {
-			strvec_push(&cmd->args, "--");
-			strvec_split(&cmd->args, value);
-		}
-	}
-
-	strvec_pushv(&cmd->env, (const char **)local_repo_env);
-	cmd->out = -1;
-}
-
-static void read_alternate_refs(const char *path,
-				alternate_ref_fn *cb,
-				void *data)
-{
-	struct child_process cmd = CHILD_PROCESS_INIT;
-	struct strbuf line = STRBUF_INIT;
-	FILE *fh;
-
-	fill_alternate_refs_command(&cmd, path);
-
-	if (start_command(&cmd))
-		return;
-
-	fh = xfdopen(cmd.out, "r");
-	while (strbuf_getline_lf(&line, fh) != EOF) {
-		struct object_id oid;
-		const char *p;
-
-		if (parse_oid_hex(line.buf, &oid, &p) || *p) {
-			warning(_("invalid line while parsing alternate refs: %s"),
-				line.buf);
-			break;
-		}
-
-		cb(&oid, data);
-	}
-
-	fclose(fh);
-	finish_command(&cmd);
-	strbuf_release(&line);
-}
-
-struct alternate_refs_data {
-	alternate_ref_fn *fn;
-	void *data;
-};
-
-static int refs_from_alternate_cb(struct object_directory *e,
-				  void *data)
-{
-	struct strbuf path = STRBUF_INIT;
-	size_t base_len;
-	struct alternate_refs_data *cb = data;
-
-	if (!strbuf_realpath(&path, e->path, 0))
-		goto out;
-	if (!strbuf_strip_suffix(&path, "/objects"))
-		goto out;
-	base_len = path.len;
-
-	/* Is this a git repository with refs? */
-	strbuf_addstr(&path, "/refs");
-	if (!is_directory(path.buf))
-		goto out;
-	strbuf_setlen(&path, base_len);
-
-	read_alternate_refs(path.buf, cb->fn, cb->data);
-
-out:
-	strbuf_release(&path);
-	return 0;
-}
-
-void for_each_alternate_ref(alternate_ref_fn fn, void *data)
-{
-	struct alternate_refs_data cb;
-	cb.fn = fn;
-	cb.data = data;
-	foreach_alt_odb(refs_from_alternate_cb, &cb);
-}
-
-int foreach_alt_odb(alt_odb_fn fn, void *cb)
-{
-	struct object_directory *ent;
-	int r = 0;
-
-	prepare_alt_odb(the_repository);
-	for (ent = the_repository->objects->odb->next; ent; ent = ent->next) {
-		r = fn(ent, cb);
-		if (r)
-			break;
-	}
-	return r;
-}
-
-void prepare_alt_odb(struct repository *r)
-{
-	if (r->objects->loaded_alternates)
-		return;
-
-	link_alt_odb_entries(r, r->objects->alternate_db, PATH_SEP, NULL, 0);
-
-	read_info_alternates(r, r->objects->odb->path, 0);
-	r->objects->loaded_alternates = 1;
-}
-
-int has_alt_odb(struct repository *r)
-{
-	prepare_alt_odb(r);
-	return !!r->objects->odb->next;
-}
-
 /* Returns 1 if we have successfully freshened the file, 0 otherwise. */
 static int freshen_file(const char *fn)
 {
@@ -1056,9 +466,9 @@ int parse_loose_header(const char *hdr, struct object_info *oi)
 	return 0;
 }
 
-static int loose_object_info(struct repository *r,
-			     const struct object_id *oid,
-			     struct object_info *oi, int flags)
+int loose_object_info(struct repository *r,
+		      const struct object_id *oid,
+		      struct object_info *oi, int flags)
 {
 	int status = 0;
 	int fd;
@@ -1154,345 +564,6 @@ static int loose_object_info(struct repository *r,
 	return status;
 }
 
-int obj_read_use_lock = 0;
-pthread_mutex_t obj_read_mutex;
-
-void enable_obj_read_lock(void)
-{
-	if (obj_read_use_lock)
-		return;
-
-	obj_read_use_lock = 1;
-	init_recursive_mutex(&obj_read_mutex);
-}
-
-void disable_obj_read_lock(void)
-{
-	if (!obj_read_use_lock)
-		return;
-
-	obj_read_use_lock = 0;
-	pthread_mutex_destroy(&obj_read_mutex);
-}
-
-int fetch_if_missing = 1;
-
-static int do_oid_object_info_extended(struct repository *r,
-				       const struct object_id *oid,
-				       struct object_info *oi, unsigned flags)
-{
-	static struct object_info blank_oi = OBJECT_INFO_INIT;
-	const struct cached_object *co;
-	struct pack_entry e;
-	int rtype;
-	const struct object_id *real = oid;
-	int already_retried = 0;
-
-
-	if (flags & OBJECT_INFO_LOOKUP_REPLACE)
-		real = lookup_replace_object(r, oid);
-
-	if (is_null_oid(real))
-		return -1;
-
-	if (!oi)
-		oi = &blank_oi;
-
-	co = find_cached_object(real);
-	if (co) {
-		if (oi->typep)
-			*(oi->typep) = co->type;
-		if (oi->sizep)
-			*(oi->sizep) = co->size;
-		if (oi->disk_sizep)
-			*(oi->disk_sizep) = 0;
-		if (oi->delta_base_oid)
-			oidclr(oi->delta_base_oid, the_repository->hash_algo);
-		if (oi->type_name)
-			strbuf_addstr(oi->type_name, type_name(co->type));
-		if (oi->contentp)
-			*oi->contentp = xmemdupz(co->buf, co->size);
-		oi->whence = OI_CACHED;
-		return 0;
-	}
-
-	while (1) {
-		if (find_pack_entry(r, real, &e))
-			break;
-
-		/* Most likely it's a loose object. */
-		if (!loose_object_info(r, real, oi, flags))
-			return 0;
-
-		/* Not a loose object; someone else may have just packed it. */
-		if (!(flags & OBJECT_INFO_QUICK)) {
-			reprepare_packed_git(r);
-			if (find_pack_entry(r, real, &e))
-				break;
-		}
-
-		/*
-		 * If r is the_repository, this might be an attempt at
-		 * accessing a submodule object as if it were in the_repository
-		 * (having called add_submodule_odb() on that submodule's ODB).
-		 * If any such ODBs exist, register them and try again.
-		 */
-		if (r == the_repository &&
-		    register_all_submodule_odb_as_alternates())
-			/* We added some alternates; retry */
-			continue;
-
-		/* Check if it is a missing object */
-		if (fetch_if_missing && repo_has_promisor_remote(r) &&
-		    !already_retried &&
-		    !(flags & OBJECT_INFO_SKIP_FETCH_OBJECT)) {
-			promisor_remote_get_direct(r, real, 1);
-			already_retried = 1;
-			continue;
-		}
-
-		if (flags & OBJECT_INFO_DIE_IF_CORRUPT) {
-			const struct packed_git *p;
-			if ((flags & OBJECT_INFO_LOOKUP_REPLACE) && !oideq(real, oid))
-				die(_("replacement %s not found for %s"),
-				    oid_to_hex(real), oid_to_hex(oid));
-			if ((p = has_packed_and_bad(r, real)))
-				die(_("packed object %s (stored in %s) is corrupt"),
-				    oid_to_hex(real), p->pack_name);
-		}
-		return -1;
-	}
-
-	if (oi == &blank_oi)
-		/*
-		 * We know that the caller doesn't actually need the
-		 * information below, so return early.
-		 */
-		return 0;
-	rtype = packed_object_info(r, e.p, e.offset, oi);
-	if (rtype < 0) {
-		mark_bad_packed_object(e.p, real);
-		return do_oid_object_info_extended(r, real, oi, 0);
-	} else if (oi->whence == OI_PACKED) {
-		oi->u.packed.offset = e.offset;
-		oi->u.packed.pack = e.p;
-		oi->u.packed.is_delta = (rtype == OBJ_REF_DELTA ||
-					 rtype == OBJ_OFS_DELTA);
-	}
-
-	return 0;
-}
-
-static int oid_object_info_convert(struct repository *r,
-				   const struct object_id *input_oid,
-				   struct object_info *input_oi, unsigned flags)
-{
-	const struct git_hash_algo *input_algo = &hash_algos[input_oid->algo];
-	int do_die = flags & OBJECT_INFO_DIE_IF_CORRUPT;
-	struct strbuf type_name = STRBUF_INIT;
-	struct object_id oid, delta_base_oid;
-	struct object_info new_oi, *oi;
-	unsigned long size;
-	void *content;
-	int ret;
-
-	if (repo_oid_to_algop(r, input_oid, the_hash_algo, &oid)) {
-		if (do_die)
-			die(_("missing mapping of %s to %s"),
-			    oid_to_hex(input_oid), the_hash_algo->name);
-		return -1;
-	}
-
-	/* Is new_oi needed? */
-	oi = input_oi;
-	if (input_oi && (input_oi->delta_base_oid || input_oi->sizep ||
-			 input_oi->contentp)) {
-		new_oi = *input_oi;
-		/* Does delta_base_oid need to be converted? */
-		if (input_oi->delta_base_oid)
-			new_oi.delta_base_oid = &delta_base_oid;
-		/* Will the attributes differ when converted? */
-		if (input_oi->sizep || input_oi->contentp) {
-			new_oi.contentp = &content;
-			new_oi.sizep = &size;
-			new_oi.type_name = &type_name;
-		}
-		oi = &new_oi;
-	}
-
-	ret = oid_object_info_extended(r, &oid, oi, flags);
-	if (ret)
-		return -1;
-	if (oi == input_oi)
-		return ret;
-
-	if (new_oi.contentp) {
-		struct strbuf outbuf = STRBUF_INIT;
-		enum object_type type;
-
-		type = type_from_string_gently(type_name.buf, type_name.len,
-					       !do_die);
-		if (type == -1)
-			return -1;
-		if (type != OBJ_BLOB) {
-			ret = convert_object_file(the_repository, &outbuf,
-						  the_hash_algo, input_algo,
-						  content, size, type, !do_die);
-			free(content);
-			if (ret == -1)
-				return -1;
-			size = outbuf.len;
-			content = strbuf_detach(&outbuf, NULL);
-		}
-		if (input_oi->sizep)
-			*input_oi->sizep = size;
-		if (input_oi->contentp)
-			*input_oi->contentp = content;
-		else
-			free(content);
-		if (input_oi->type_name)
-			*input_oi->type_name = type_name;
-		else
-			strbuf_release(&type_name);
-	}
-	if (new_oi.delta_base_oid == &delta_base_oid) {
-		if (repo_oid_to_algop(r, &delta_base_oid, input_algo,
-				 input_oi->delta_base_oid)) {
-			if (do_die)
-				die(_("missing mapping of %s to %s"),
-				    oid_to_hex(&delta_base_oid),
-				    input_algo->name);
-			return -1;
-		}
-	}
-	input_oi->whence = new_oi.whence;
-	input_oi->u = new_oi.u;
-	return ret;
-}
-
-int oid_object_info_extended(struct repository *r, const struct object_id *oid,
-			     struct object_info *oi, unsigned flags)
-{
-	int ret;
-
-	if (oid->algo && (hash_algo_by_ptr(r->hash_algo) != oid->algo))
-		return oid_object_info_convert(r, oid, oi, flags);
-
-	obj_read_lock();
-	ret = do_oid_object_info_extended(r, oid, oi, flags);
-	obj_read_unlock();
-	return ret;
-}
-
-
-/* returns enum object_type or negative */
-int oid_object_info(struct repository *r,
-		    const struct object_id *oid,
-		    unsigned long *sizep)
-{
-	enum object_type type;
-	struct object_info oi = OBJECT_INFO_INIT;
-
-	oi.typep = &type;
-	oi.sizep = sizep;
-	if (oid_object_info_extended(r, oid, &oi,
-				      OBJECT_INFO_LOOKUP_REPLACE) < 0)
-		return -1;
-	return type;
-}
-
-int pretend_object_file(void *buf, unsigned long len, enum object_type type,
-			struct object_id *oid)
-{
-	struct cached_object_entry *co;
-	char *co_buf;
-
-	hash_object_file(the_hash_algo, buf, len, type, oid);
-	if (repo_has_object_file_with_flags(the_repository, oid, OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT) ||
-	    find_cached_object(oid))
-		return 0;
-	ALLOC_GROW(cached_objects, cached_object_nr + 1, cached_object_alloc);
-	co = &cached_objects[cached_object_nr++];
-	co->value.size = len;
-	co->value.type = type;
-	co_buf = xmalloc(len);
-	memcpy(co_buf, buf, len);
-	co->value.buf = co_buf;
-	oidcpy(&co->oid, oid);
-	return 0;
-}
-
-/*
- * This function dies on corrupt objects; the callers who want to
- * deal with them should arrange to call oid_object_info_extended() and give
- * error messages themselves.
- */
-void *repo_read_object_file(struct repository *r,
-			    const struct object_id *oid,
-			    enum object_type *type,
-			    unsigned long *size)
-{
-	struct object_info oi = OBJECT_INFO_INIT;
-	unsigned flags = OBJECT_INFO_DIE_IF_CORRUPT | OBJECT_INFO_LOOKUP_REPLACE;
-	void *data;
-
-	oi.typep = type;
-	oi.sizep = size;
-	oi.contentp = &data;
-	if (oid_object_info_extended(r, oid, &oi, flags))
-		return NULL;
-
-	return data;
-}
-
-void *read_object_with_reference(struct repository *r,
-				 const struct object_id *oid,
-				 enum object_type required_type,
-				 unsigned long *size,
-				 struct object_id *actual_oid_return)
-{
-	enum object_type type;
-	void *buffer;
-	unsigned long isize;
-	struct object_id actual_oid;
-
-	oidcpy(&actual_oid, oid);
-	while (1) {
-		int ref_length = -1;
-		const char *ref_type = NULL;
-
-		buffer = repo_read_object_file(r, &actual_oid, &type, &isize);
-		if (!buffer)
-			return NULL;
-		if (type == required_type) {
-			*size = isize;
-			if (actual_oid_return)
-				oidcpy(actual_oid_return, &actual_oid);
-			return buffer;
-		}
-		/* Handle references */
-		else if (type == OBJ_COMMIT)
-			ref_type = "tree ";
-		else if (type == OBJ_TAG)
-			ref_type = "object ";
-		else {
-			free(buffer);
-			return NULL;
-		}
-		ref_length = strlen(ref_type);
-
-		if (ref_length + the_hash_algo->hexsz > isize ||
-		    memcmp(buffer, ref_type, ref_length) ||
-		    get_oid_hex((char *) buffer + ref_length, &actual_oid)) {
-			free(buffer);
-			return NULL;
-		}
-		free(buffer);
-		/* Now we have the ID of the referred-to object in
-		 * actual_oid.  Check again. */
-	}
-}
-
 static void hash_object_body(const struct git_hash_algo *algo, struct git_hash_ctx *c,
 			     const void *buf, unsigned long len,
 			     struct object_id *oid,
@@ -2154,32 +1225,6 @@ int force_object_loose(const struct object_id *oid, time_t mtime)
 	return ret;
 }
 
-int has_object(struct repository *r, const struct object_id *oid,
-	       unsigned flags)
-{
-	int quick = !(flags & HAS_OBJECT_RECHECK_PACKED);
-	unsigned object_info_flags = OBJECT_INFO_SKIP_FETCH_OBJECT |
-		(quick ? OBJECT_INFO_QUICK : 0);
-
-	if (!startup_info->have_repository)
-		return 0;
-	return oid_object_info_extended(r, oid, NULL, object_info_flags) >= 0;
-}
-
-int repo_has_object_file_with_flags(struct repository *r,
-				    const struct object_id *oid, int flags)
-{
-	if (!startup_info->have_repository)
-		return 0;
-	return oid_object_info_extended(r, oid, NULL, flags) >= 0;
-}
-
-int repo_has_object_file(struct repository *r,
-			 const struct object_id *oid)
-{
-	return repo_has_object_file_with_flags(r, oid, 0);
-}
-
 /*
  * We can't use the normal fsck_error_function() for index_mem(),
  * because we don't yet have a valid oid for it to report. Instead,
@@ -2407,16 +1452,6 @@ int read_pack_header(int fd, struct pack_header *header)
 	return 0;
 }
 
-void assert_oid_type(const struct object_id *oid, enum object_type expect)
-{
-	enum object_type type = oid_object_info(the_repository, oid, NULL);
-	if (type < 0)
-		die(_("%s is not a valid object"), oid_to_hex(oid));
-	if (type != expect)
-		die(_("%s is not a valid '%s' object"), oid_to_hex(oid),
-		    type_name(expect));
-}
-
 int for_each_file_in_obj_subdir(unsigned int subdir_nr,
 				struct strbuf *path,
 				each_loose_object_fn obj_cb,
diff --git a/object-file.h b/object-file.h
index 353d8a85c33..78c84d970a9 100644
--- a/object-file.h
+++ b/object-file.h
@@ -21,6 +21,29 @@ extern int fetch_if_missing;
 int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);
 int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags);
 
+struct object_directory;
+
+const char *odb_loose_path(struct object_directory *odb,
+			   struct strbuf *buf,
+			   const struct object_id *oid);
+
+/*
+ * Return true iff an alternate object database has a loose object
+ * with the specified name.  This function does not respect replace
+ * references.
+ */
+int has_loose_object_nonlocal(const struct object_id *);
+
+int has_loose_object(const struct object_id *);
+
+/**
+ * format_object_header() is a thin wrapper around s xsnprintf() that
+ * writes the initial "<type> <obj-len>" part of the loose object
+ * header. It returns the size that snprintf() returns + 1.
+ */
+int format_object_header(char *str, size_t size, enum object_type type,
+			 size_t objsize);
+
 /**
  * unpack_loose_header() initializes the data stream needed to unpack
  * a loose object header.
@@ -61,6 +84,29 @@ enum unpack_loose_header_result unpack_loose_header(git_zstream *stream,
 struct object_info;
 int parse_loose_header(const char *hdr, struct object_info *oi);
 
+int write_object_file_flags(const void *buf, unsigned long len,
+			    enum object_type type, struct object_id *oid,
+			    struct object_id *comapt_oid_in, unsigned flags);
+static inline int write_object_file(const void *buf, unsigned long len,
+				    enum object_type type, struct object_id *oid)
+{
+	return write_object_file_flags(buf, len, type, oid, NULL, 0);
+}
+
+struct input_stream {
+	const void *(*read)(struct input_stream *, unsigned long *len);
+	void *data;
+	int is_finished;
+};
+
+int write_object_file_literally(const void *buf, unsigned long len,
+				const char *type, struct object_id *oid,
+				unsigned flags);
+int stream_loose_object(struct input_stream *in_stream, size_t len,
+			struct object_id *oid);
+
+int force_object_loose(const struct object_id *oid, time_t mtime);
+
 /**
  * With in-core object data in "buf", rehash it to make sure the
  * object name actually matches "oid" to detect object corruption.
@@ -79,6 +125,10 @@ int check_object_signature(struct repository *r, const struct object_id *oid,
  */
 int stream_object_signature(struct repository *r, const struct object_id *oid);
 
+int loose_object_info(struct repository *r,
+		      const struct object_id *oid,
+		      struct object_info *oi, int flags);
+
 enum finalize_object_file_flags {
 	FOF_SKIP_COLLISION_CHECK = 1,
 };
@@ -90,10 +140,18 @@ int finalize_object_file_flags(const char *tmpfile, const char *filename,
 /* Helper to check and "touch" a file */
 int check_and_freshen_file(const char *fn, int freshen);
 
-void *read_object_with_reference(struct repository *r,
-				 const struct object_id *oid,
-				 enum object_type required_type,
-				 unsigned long *size,
-				 struct object_id *oid_ret);
+/*
+ * Open the loose object at path, check its hash, and return the contents,
+ * use the "oi" argument to assert things about the object, or e.g. populate its
+ * type, and size. If the object is a blob, then "contents" may return NULL,
+ * to allow streaming of large blobs.
+ *
+ * Returns 0 on success, negative on error (details may be written to stderr).
+ */
+int read_loose_object(const char *path,
+		      const struct object_id *expected_oid,
+		      struct object_id *real_oid,
+		      void **contents,
+		      struct object_info *oi);
 
 #endif /* OBJECT_FILE_H */
diff --git a/object-store-ll.h b/object-store-ll.h
index cd3bd5bd99f..8ae80b8a5fa 100644
--- a/object-store-ll.h
+++ b/object-store-ll.h
@@ -49,12 +49,6 @@ struct object_directory {
 	char *path;
 };
 
-struct input_stream {
-	const void *(*read)(struct input_stream *, unsigned long *len);
-	void *data;
-	int is_finished;
-};
-
 void prepare_alt_odb(struct repository *r);
 int has_alt_odb(struct repository *r);
 char *compute_alternate_path(const char *path, struct strbuf *err);
@@ -273,21 +267,6 @@ void hash_object_file(const struct git_hash_algo *algo, const void *buf,
 		      unsigned long len, enum object_type type,
 		      struct object_id *oid);
 
-int write_object_file_flags(const void *buf, unsigned long len,
-			    enum object_type type, struct object_id *oid,
-			    struct object_id *comapt_oid_in, unsigned flags);
-static inline int write_object_file(const void *buf, unsigned long len,
-				    enum object_type type, struct object_id *oid)
-{
-	return write_object_file_flags(buf, len, type, oid, NULL, 0);
-}
-
-int write_object_file_literally(const void *buf, unsigned long len,
-				const char *type, struct object_id *oid,
-				unsigned flags);
-int stream_loose_object(struct input_stream *in_stream, size_t len,
-			struct object_id *oid);
-
 /*
  * Add an object file to the in-memory object store, without writing it
  * to disk.
@@ -299,8 +278,6 @@ int stream_loose_object(struct input_stream *in_stream, size_t len,
 int pretend_object_file(void *, unsigned long, enum object_type,
 			struct object_id *oid);
 
-int force_object_loose(const struct object_id *oid, time_t mtime);
-
 struct object_info {
 	/* Request */
 	enum object_type *typep;
@@ -364,20 +341,6 @@ int oid_object_info_extended(struct repository *r,
 			     const struct object_id *,
 			     struct object_info *, unsigned flags);
 
-/*
- * Open the loose object at path, check its hash, and return the contents,
- * use the "oi" argument to assert things about the object, or e.g. populate its
- * type, and size. If the object is a blob, then "contents" may return NULL,
- * to allow streaming of large blobs.
- *
- * Returns 0 on success, negative on error (details may be written to stderr).
- */
-int read_loose_object(const char *path,
-		      const struct object_id *expected_oid,
-		      struct object_id *real_oid,
-		      void **contents,
-		      struct object_info *oi);
-
 /* Retry packed storage after checking packed and loose storage */
 #define HAS_OBJECT_RECHECK_PACKED 1
 
@@ -405,23 +368,6 @@ int repo_has_object_file(struct repository *r, const struct object_id *oid);
 int repo_has_object_file_with_flags(struct repository *r,
 				    const struct object_id *oid, int flags);
 
-/*
- * Return true iff an alternate object database has a loose object
- * with the specified name.  This function does not respect replace
- * references.
- */
-int has_loose_object_nonlocal(const struct object_id *);
-
-int has_loose_object(const struct object_id *);
-
-/**
- * format_object_header() is a thin wrapper around s xsnprintf() that
- * writes the initial "<type> <obj-len>" part of the loose object
- * header. It returns the size that snprintf() returns + 1.
- */
-int format_object_header(char *str, size_t size, enum object_type type,
-			 size_t objsize);
-
 void assert_oid_type(const struct object_id *oid, enum object_type expect);
 
 /*
@@ -553,4 +499,10 @@ int for_each_object_in_pack(struct packed_git *p,
 int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 			   void *data, enum for_each_object_flags flags);
 
+void *read_object_with_reference(struct repository *r,
+				 const struct object_id *oid,
+				 enum object_type required_type,
+				 unsigned long *size,
+				 struct object_id *oid_ret);
+
 #endif /* OBJECT_STORE_LL_H */
diff --git a/object-store.c b/object-store.c
new file mode 100644
index 00000000000..e5f1f00cdde
--- /dev/null
+++ b/object-store.c
@@ -0,0 +1,972 @@
+#define USE_THE_REPOSITORY_VARIABLE
+
+#include "git-compat-util.h"
+#include "abspath.h"
+#include "config.h"
+#include "environment.h"
+#include "gettext.h"
+#include "hex.h"
+#include "lockfile.h"
+#include "object-file-convert.h"
+#include "object-file.h"
+#include "object-store.h"
+#include "packfile.h"
+#include "path.h"
+#include "promisor-remote.h"
+#include "quote.h"
+#include "replace-object.h"
+#include "run-command.h"
+#include "setup.h"
+#include "strbuf.h"
+#include "strvec.h"
+#include "submodule.h"
+#include "write-or-die.h"
+
+/*
+ * This is meant to hold a *small* number of objects that you would
+ * want repo_read_object_file() to be able to return, but yet you do not want
+ * to write them into the object store (e.g. a browse-only
+ * application).
+ */
+static struct cached_object_entry {
+	struct object_id oid;
+	struct cached_object {
+		enum object_type type;
+		const void *buf;
+		unsigned long size;
+	} value;
+} *cached_objects;
+static int cached_object_nr, cached_object_alloc;
+
+static const struct cached_object *find_cached_object(const struct object_id *oid)
+{
+	static const struct cached_object empty_tree = {
+		.type = OBJ_TREE,
+		.buf = "",
+	};
+	int i;
+	const struct cached_object_entry *co = cached_objects;
+
+	for (i = 0; i < cached_object_nr; i++, co++) {
+		if (oideq(&co->oid, oid))
+			return &co->value;
+	}
+	if (oideq(oid, the_hash_algo->empty_tree))
+		return &empty_tree;
+	return NULL;
+}
+
+int odb_mkstemp(struct strbuf *temp_filename, const char *pattern)
+{
+	int fd;
+	/*
+	 * we let the umask do its job, don't try to be more
+	 * restrictive except to remove write permission.
+	 */
+	int mode = 0444;
+	repo_git_path_replace(the_repository, temp_filename, "objects/%s", pattern);
+	fd = git_mkstemp_mode(temp_filename->buf, mode);
+	if (0 <= fd)
+		return fd;
+
+	/* slow path */
+	/* some mkstemp implementations erase temp_filename on failure */
+	repo_git_path_replace(the_repository, temp_filename, "objects/%s", pattern);
+	safe_create_leading_directories(temp_filename->buf);
+	return xmkstemp_mode(temp_filename->buf, mode);
+}
+
+int odb_pack_keep(const char *name)
+{
+	int fd;
+
+	fd = open(name, O_RDWR|O_CREAT|O_EXCL, 0600);
+	if (0 <= fd)
+		return fd;
+
+	/* slow path */
+	safe_create_leading_directories_const(name);
+	return open(name, O_RDWR|O_CREAT|O_EXCL, 0600);
+}
+
+const char *loose_object_path(struct repository *r, struct strbuf *buf,
+			      const struct object_id *oid)
+{
+	return odb_loose_path(r->objects->odb, buf, oid);
+}
+
+/*
+ * Return non-zero iff the path is usable as an alternate object database.
+ */
+static int alt_odb_usable(struct raw_object_store *o,
+			  struct strbuf *path,
+			  const char *normalized_objdir, khiter_t *pos)
+{
+	int r;
+
+	/* Detect cases where alternate disappeared */
+	if (!is_directory(path->buf)) {
+		error(_("object directory %s does not exist; "
+			"check .git/objects/info/alternates"),
+		      path->buf);
+		return 0;
+	}
+
+	/*
+	 * Prevent the common mistake of listing the same
+	 * thing twice, or object directory itself.
+	 */
+	if (!o->odb_by_path) {
+		khiter_t p;
+
+		o->odb_by_path = kh_init_odb_path_map();
+		assert(!o->odb->next);
+		p = kh_put_odb_path_map(o->odb_by_path, o->odb->path, &r);
+		assert(r == 1); /* never used */
+		kh_value(o->odb_by_path, p) = o->odb;
+	}
+	if (fspatheq(path->buf, normalized_objdir))
+		return 0;
+	*pos = kh_put_odb_path_map(o->odb_by_path, path->buf, &r);
+	/* r: 0 = exists, 1 = never used, 2 = deleted */
+	return r == 0 ? 0 : 1;
+}
+
+/*
+ * Prepare alternate object database registry.
+ *
+ * The variable alt_odb_list points at the list of struct
+ * object_directory.  The elements on this list come from
+ * non-empty elements from colon separated ALTERNATE_DB_ENVIRONMENT
+ * environment variable, and $GIT_OBJECT_DIRECTORY/info/alternates,
+ * whose contents is similar to that environment variable but can be
+ * LF separated.  Its base points at a statically allocated buffer that
+ * contains "/the/directory/corresponding/to/.git/objects/...", while
+ * its name points just after the slash at the end of ".git/objects/"
+ * in the example above, and has enough space to hold all hex characters
+ * of the object ID, an extra slash for the first level indirection, and
+ * the terminating NUL.
+ */
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth);
+static int link_alt_odb_entry(struct repository *r, const struct strbuf *entry,
+	const char *relative_base, int depth, const char *normalized_objdir)
+{
+	struct object_directory *ent;
+	struct strbuf pathbuf = STRBUF_INIT;
+	struct strbuf tmp = STRBUF_INIT;
+	khiter_t pos;
+	int ret = -1;
+
+	if (!is_absolute_path(entry->buf) && relative_base) {
+		strbuf_realpath(&pathbuf, relative_base, 1);
+		strbuf_addch(&pathbuf, '/');
+	}
+	strbuf_addbuf(&pathbuf, entry);
+
+	if (!strbuf_realpath(&tmp, pathbuf.buf, 0)) {
+		error(_("unable to normalize alternate object path: %s"),
+		      pathbuf.buf);
+		goto error;
+	}
+	strbuf_swap(&pathbuf, &tmp);
+
+	/*
+	 * The trailing slash after the directory name is given by
+	 * this function at the end. Remove duplicates.
+	 */
+	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
+		strbuf_setlen(&pathbuf, pathbuf.len - 1);
+
+	if (!alt_odb_usable(r->objects, &pathbuf, normalized_objdir, &pos))
+		goto error;
+
+	CALLOC_ARRAY(ent, 1);
+	/* pathbuf.buf is already in r->objects->odb_by_path */
+	ent->path = strbuf_detach(&pathbuf, NULL);
+
+	/* add the alternate entry */
+	*r->objects->odb_tail = ent;
+	r->objects->odb_tail = &(ent->next);
+	ent->next = NULL;
+	assert(r->objects->odb_by_path);
+	kh_value(r->objects->odb_by_path, pos) = ent;
+
+	/* recursively add alternates */
+	read_info_alternates(r, ent->path, depth + 1);
+	ret = 0;
+ error:
+	strbuf_release(&tmp);
+	strbuf_release(&pathbuf);
+	return ret;
+}
+
+static const char *parse_alt_odb_entry(const char *string,
+				       int sep,
+				       struct strbuf *out)
+{
+	const char *end;
+
+	strbuf_reset(out);
+
+	if (*string == '#') {
+		/* comment; consume up to next separator */
+		end = strchrnul(string, sep);
+	} else if (*string == '"' && !unquote_c_style(out, string, &end)) {
+		/*
+		 * quoted path; unquote_c_style has copied the
+		 * data for us and set "end". Broken quoting (e.g.,
+		 * an entry that doesn't end with a quote) falls
+		 * back to the unquoted case below.
+		 */
+	} else {
+		/* normal, unquoted path */
+		end = strchrnul(string, sep);
+		strbuf_add(out, string, end - string);
+	}
+
+	if (*end)
+		end++;
+	return end;
+}
+
+static void link_alt_odb_entries(struct repository *r, const char *alt,
+				 int sep, const char *relative_base, int depth)
+{
+	struct strbuf objdirbuf = STRBUF_INIT;
+	struct strbuf entry = STRBUF_INIT;
+
+	if (!alt || !*alt)
+		return;
+
+	if (depth > 5) {
+		error(_("%s: ignoring alternate object stores, nesting too deep"),
+				relative_base);
+		return;
+	}
+
+	strbuf_realpath(&objdirbuf, r->objects->odb->path, 1);
+
+	while (*alt) {
+		alt = parse_alt_odb_entry(alt, sep, &entry);
+		if (!entry.len)
+			continue;
+		link_alt_odb_entry(r, &entry,
+				   relative_base, depth, objdirbuf.buf);
+	}
+	strbuf_release(&entry);
+	strbuf_release(&objdirbuf);
+}
+
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth)
+{
+	char *path;
+	struct strbuf buf = STRBUF_INIT;
+
+	path = xstrfmt("%s/info/alternates", relative_base);
+	if (strbuf_read_file(&buf, path, 1024) < 0) {
+		warn_on_fopen_errors(path);
+		free(path);
+		return;
+	}
+
+	link_alt_odb_entries(r, buf.buf, '\n', relative_base, depth);
+	strbuf_release(&buf);
+	free(path);
+}
+
+void add_to_alternates_file(const char *reference)
+{
+	struct lock_file lock = LOCK_INIT;
+	char *alts = repo_git_path(the_repository, "objects/info/alternates");
+	FILE *in, *out;
+	int found = 0;
+
+	hold_lock_file_for_update(&lock, alts, LOCK_DIE_ON_ERROR);
+	out = fdopen_lock_file(&lock, "w");
+	if (!out)
+		die_errno(_("unable to fdopen alternates lockfile"));
+
+	in = fopen(alts, "r");
+	if (in) {
+		struct strbuf line = STRBUF_INIT;
+
+		while (strbuf_getline(&line, in) != EOF) {
+			if (!strcmp(reference, line.buf)) {
+				found = 1;
+				break;
+			}
+			fprintf_or_die(out, "%s\n", line.buf);
+		}
+
+		strbuf_release(&line);
+		fclose(in);
+	}
+	else if (errno != ENOENT)
+		die_errno(_("unable to read alternates file"));
+
+	if (found) {
+		rollback_lock_file(&lock);
+	} else {
+		fprintf_or_die(out, "%s\n", reference);
+		if (commit_lock_file(&lock))
+			die_errno(_("unable to move new alternates file into place"));
+		if (the_repository->objects->loaded_alternates)
+			link_alt_odb_entries(the_repository, reference,
+					     '\n', NULL, 0);
+	}
+	free(alts);
+}
+
+void add_to_alternates_memory(const char *reference)
+{
+	/*
+	 * Make sure alternates are initialized, or else our entry may be
+	 * overwritten when they are.
+	 */
+	prepare_alt_odb(the_repository);
+
+	link_alt_odb_entries(the_repository, reference,
+			     '\n', NULL, 0);
+}
+
+struct object_directory *set_temporary_primary_odb(const char *dir, int will_destroy)
+{
+	struct object_directory *new_odb;
+
+	/*
+	 * Make sure alternates are initialized, or else our entry may be
+	 * overwritten when they are.
+	 */
+	prepare_alt_odb(the_repository);
+
+	/*
+	 * Make a new primary odb and link the old primary ODB in as an
+	 * alternate
+	 */
+	new_odb = xcalloc(1, sizeof(*new_odb));
+	new_odb->path = xstrdup(dir);
+
+	/*
+	 * Disable ref updates while a temporary odb is active, since
+	 * the objects in the database may roll back.
+	 */
+	new_odb->disable_ref_updates = 1;
+	new_odb->will_destroy = will_destroy;
+	new_odb->next = the_repository->objects->odb;
+	the_repository->objects->odb = new_odb;
+	return new_odb->next;
+}
+
+void restore_primary_odb(struct object_directory *restore_odb, const char *old_path)
+{
+	struct object_directory *cur_odb = the_repository->objects->odb;
+
+	if (strcmp(old_path, cur_odb->path))
+		BUG("expected %s as primary object store; found %s",
+		    old_path, cur_odb->path);
+
+	if (cur_odb->next != restore_odb)
+		BUG("we expect the old primary object store to be the first alternate");
+
+	the_repository->objects->odb = restore_odb;
+	free_object_directory(cur_odb);
+}
+
+/*
+ * Compute the exact path an alternate is at and returns it. In case of
+ * error NULL is returned and the human readable error is added to `err`
+ * `path` may be relative and should point to $GIT_DIR.
+ * `err` must not be null.
+ */
+char *compute_alternate_path(const char *path, struct strbuf *err)
+{
+	char *ref_git = NULL;
+	const char *repo;
+	int seen_error = 0;
+
+	ref_git = real_pathdup(path, 0);
+	if (!ref_git) {
+		seen_error = 1;
+		strbuf_addf(err, _("path '%s' does not exist"), path);
+		goto out;
+	}
+
+	repo = read_gitfile(ref_git);
+	if (!repo)
+		repo = read_gitfile(mkpath("%s/.git", ref_git));
+	if (repo) {
+		free(ref_git);
+		ref_git = xstrdup(repo);
+	}
+
+	if (!repo && is_directory(mkpath("%s/.git/objects", ref_git))) {
+		char *ref_git_git = mkpathdup("%s/.git", ref_git);
+		free(ref_git);
+		ref_git = ref_git_git;
+	} else if (!is_directory(mkpath("%s/objects", ref_git))) {
+		struct strbuf sb = STRBUF_INIT;
+		seen_error = 1;
+		if (get_common_dir(&sb, ref_git)) {
+			strbuf_addf(err,
+				    _("reference repository '%s' as a linked "
+				      "checkout is not supported yet."),
+				    path);
+			goto out;
+		}
+
+		strbuf_addf(err, _("reference repository '%s' is not a "
+					"local repository."), path);
+		goto out;
+	}
+
+	if (!access(mkpath("%s/shallow", ref_git), F_OK)) {
+		strbuf_addf(err, _("reference repository '%s' is shallow"),
+			    path);
+		seen_error = 1;
+		goto out;
+	}
+
+	if (!access(mkpath("%s/info/grafts", ref_git), F_OK)) {
+		strbuf_addf(err,
+			    _("reference repository '%s' is grafted"),
+			    path);
+		seen_error = 1;
+		goto out;
+	}
+
+out:
+	if (seen_error) {
+		FREE_AND_NULL(ref_git);
+	}
+
+	return ref_git;
+}
+
+struct object_directory *find_odb(struct repository *r, const char *obj_dir)
+{
+	struct object_directory *odb;
+	char *obj_dir_real = real_pathdup(obj_dir, 1);
+	struct strbuf odb_path_real = STRBUF_INIT;
+
+	prepare_alt_odb(r);
+	for (odb = r->objects->odb; odb; odb = odb->next) {
+		strbuf_realpath(&odb_path_real, odb->path, 1);
+		if (!strcmp(obj_dir_real, odb_path_real.buf))
+			break;
+	}
+
+	free(obj_dir_real);
+	strbuf_release(&odb_path_real);
+
+	if (!odb)
+		die(_("could not find object directory matching %s"), obj_dir);
+	return odb;
+}
+
+static void fill_alternate_refs_command(struct child_process *cmd,
+					const char *repo_path)
+{
+	const char *value;
+
+	if (!git_config_get_value("core.alternateRefsCommand", &value)) {
+		cmd->use_shell = 1;
+
+		strvec_push(&cmd->args, value);
+		strvec_push(&cmd->args, repo_path);
+	} else {
+		cmd->git_cmd = 1;
+
+		strvec_pushf(&cmd->args, "--git-dir=%s", repo_path);
+		strvec_push(&cmd->args, "for-each-ref");
+		strvec_push(&cmd->args, "--format=%(objectname)");
+
+		if (!git_config_get_value("core.alternateRefsPrefixes", &value)) {
+			strvec_push(&cmd->args, "--");
+			strvec_split(&cmd->args, value);
+		}
+	}
+
+	strvec_pushv(&cmd->env, (const char **)local_repo_env);
+	cmd->out = -1;
+}
+
+static void read_alternate_refs(const char *path,
+				alternate_ref_fn *cb,
+				void *data)
+{
+	struct child_process cmd = CHILD_PROCESS_INIT;
+	struct strbuf line = STRBUF_INIT;
+	FILE *fh;
+
+	fill_alternate_refs_command(&cmd, path);
+
+	if (start_command(&cmd))
+		return;
+
+	fh = xfdopen(cmd.out, "r");
+	while (strbuf_getline_lf(&line, fh) != EOF) {
+		struct object_id oid;
+		const char *p;
+
+		if (parse_oid_hex(line.buf, &oid, &p) || *p) {
+			warning(_("invalid line while parsing alternate refs: %s"),
+				line.buf);
+			break;
+		}
+
+		cb(&oid, data);
+	}
+
+	fclose(fh);
+	finish_command(&cmd);
+	strbuf_release(&line);
+}
+
+struct alternate_refs_data {
+	alternate_ref_fn *fn;
+	void *data;
+};
+
+static int refs_from_alternate_cb(struct object_directory *e,
+				  void *data)
+{
+	struct strbuf path = STRBUF_INIT;
+	size_t base_len;
+	struct alternate_refs_data *cb = data;
+
+	if (!strbuf_realpath(&path, e->path, 0))
+		goto out;
+	if (!strbuf_strip_suffix(&path, "/objects"))
+		goto out;
+	base_len = path.len;
+
+	/* Is this a git repository with refs? */
+	strbuf_addstr(&path, "/refs");
+	if (!is_directory(path.buf))
+		goto out;
+	strbuf_setlen(&path, base_len);
+
+	read_alternate_refs(path.buf, cb->fn, cb->data);
+
+out:
+	strbuf_release(&path);
+	return 0;
+}
+
+void for_each_alternate_ref(alternate_ref_fn fn, void *data)
+{
+	struct alternate_refs_data cb;
+	cb.fn = fn;
+	cb.data = data;
+	foreach_alt_odb(refs_from_alternate_cb, &cb);
+}
+
+int foreach_alt_odb(alt_odb_fn fn, void *cb)
+{
+	struct object_directory *ent;
+	int r = 0;
+
+	prepare_alt_odb(the_repository);
+	for (ent = the_repository->objects->odb->next; ent; ent = ent->next) {
+		r = fn(ent, cb);
+		if (r)
+			break;
+	}
+	return r;
+}
+
+void prepare_alt_odb(struct repository *r)
+{
+	if (r->objects->loaded_alternates)
+		return;
+
+	link_alt_odb_entries(r, r->objects->alternate_db, PATH_SEP, NULL, 0);
+
+	read_info_alternates(r, r->objects->odb->path, 0);
+	r->objects->loaded_alternates = 1;
+}
+
+int has_alt_odb(struct repository *r)
+{
+	prepare_alt_odb(r);
+	return !!r->objects->odb->next;
+}
+
+int obj_read_use_lock = 0;
+pthread_mutex_t obj_read_mutex;
+
+void enable_obj_read_lock(void)
+{
+	if (obj_read_use_lock)
+		return;
+
+	obj_read_use_lock = 1;
+	init_recursive_mutex(&obj_read_mutex);
+}
+
+void disable_obj_read_lock(void)
+{
+	if (!obj_read_use_lock)
+		return;
+
+	obj_read_use_lock = 0;
+	pthread_mutex_destroy(&obj_read_mutex);
+}
+
+int fetch_if_missing = 1;
+
+static int do_oid_object_info_extended(struct repository *r,
+				       const struct object_id *oid,
+				       struct object_info *oi, unsigned flags)
+{
+	static struct object_info blank_oi = OBJECT_INFO_INIT;
+	const struct cached_object *co;
+	struct pack_entry e;
+	int rtype;
+	const struct object_id *real = oid;
+	int already_retried = 0;
+
+
+	if (flags & OBJECT_INFO_LOOKUP_REPLACE)
+		real = lookup_replace_object(r, oid);
+
+	if (is_null_oid(real))
+		return -1;
+
+	if (!oi)
+		oi = &blank_oi;
+
+	co = find_cached_object(real);
+	if (co) {
+		if (oi->typep)
+			*(oi->typep) = co->type;
+		if (oi->sizep)
+			*(oi->sizep) = co->size;
+		if (oi->disk_sizep)
+			*(oi->disk_sizep) = 0;
+		if (oi->delta_base_oid)
+			oidclr(oi->delta_base_oid, the_repository->hash_algo);
+		if (oi->type_name)
+			strbuf_addstr(oi->type_name, type_name(co->type));
+		if (oi->contentp)
+			*oi->contentp = xmemdupz(co->buf, co->size);
+		oi->whence = OI_CACHED;
+		return 0;
+	}
+
+	while (1) {
+		if (find_pack_entry(r, real, &e))
+			break;
+
+		/* Most likely it's a loose object. */
+		if (!loose_object_info(r, real, oi, flags))
+			return 0;
+
+		/* Not a loose object; someone else may have just packed it. */
+		if (!(flags & OBJECT_INFO_QUICK)) {
+			reprepare_packed_git(r);
+			if (find_pack_entry(r, real, &e))
+				break;
+		}
+
+		/*
+		 * If r is the_repository, this might be an attempt at
+		 * accessing a submodule object as if it were in the_repository
+		 * (having called add_submodule_odb() on that submodule's ODB).
+		 * If any such ODBs exist, register them and try again.
+		 */
+		if (r == the_repository &&
+		    register_all_submodule_odb_as_alternates())
+			/* We added some alternates; retry */
+			continue;
+
+		/* Check if it is a missing object */
+		if (fetch_if_missing && repo_has_promisor_remote(r) &&
+		    !already_retried &&
+		    !(flags & OBJECT_INFO_SKIP_FETCH_OBJECT)) {
+			promisor_remote_get_direct(r, real, 1);
+			already_retried = 1;
+			continue;
+		}
+
+		if (flags & OBJECT_INFO_DIE_IF_CORRUPT) {
+			const struct packed_git *p;
+			if ((flags & OBJECT_INFO_LOOKUP_REPLACE) && !oideq(real, oid))
+				die(_("replacement %s not found for %s"),
+				    oid_to_hex(real), oid_to_hex(oid));
+			if ((p = has_packed_and_bad(r, real)))
+				die(_("packed object %s (stored in %s) is corrupt"),
+				    oid_to_hex(real), p->pack_name);
+		}
+		return -1;
+	}
+
+	if (oi == &blank_oi)
+		/*
+		 * We know that the caller doesn't actually need the
+		 * information below, so return early.
+		 */
+		return 0;
+	rtype = packed_object_info(r, e.p, e.offset, oi);
+	if (rtype < 0) {
+		mark_bad_packed_object(e.p, real);
+		return do_oid_object_info_extended(r, real, oi, 0);
+	} else if (oi->whence == OI_PACKED) {
+		oi->u.packed.offset = e.offset;
+		oi->u.packed.pack = e.p;
+		oi->u.packed.is_delta = (rtype == OBJ_REF_DELTA ||
+					 rtype == OBJ_OFS_DELTA);
+	}
+
+	return 0;
+}
+
+static int oid_object_info_convert(struct repository *r,
+				   const struct object_id *input_oid,
+				   struct object_info *input_oi, unsigned flags)
+{
+	const struct git_hash_algo *input_algo = &hash_algos[input_oid->algo];
+	int do_die = flags & OBJECT_INFO_DIE_IF_CORRUPT;
+	struct strbuf type_name = STRBUF_INIT;
+	struct object_id oid, delta_base_oid;
+	struct object_info new_oi, *oi;
+	unsigned long size;
+	void *content;
+	int ret;
+
+	if (repo_oid_to_algop(r, input_oid, the_hash_algo, &oid)) {
+		if (do_die)
+			die(_("missing mapping of %s to %s"),
+			    oid_to_hex(input_oid), the_hash_algo->name);
+		return -1;
+	}
+
+	/* Is new_oi needed? */
+	oi = input_oi;
+	if (input_oi && (input_oi->delta_base_oid || input_oi->sizep ||
+			 input_oi->contentp)) {
+		new_oi = *input_oi;
+		/* Does delta_base_oid need to be converted? */
+		if (input_oi->delta_base_oid)
+			new_oi.delta_base_oid = &delta_base_oid;
+		/* Will the attributes differ when converted? */
+		if (input_oi->sizep || input_oi->contentp) {
+			new_oi.contentp = &content;
+			new_oi.sizep = &size;
+			new_oi.type_name = &type_name;
+		}
+		oi = &new_oi;
+	}
+
+	ret = oid_object_info_extended(r, &oid, oi, flags);
+	if (ret)
+		return -1;
+	if (oi == input_oi)
+		return ret;
+
+	if (new_oi.contentp) {
+		struct strbuf outbuf = STRBUF_INIT;
+		enum object_type type;
+
+		type = type_from_string_gently(type_name.buf, type_name.len,
+					       !do_die);
+		if (type == -1)
+			return -1;
+		if (type != OBJ_BLOB) {
+			ret = convert_object_file(the_repository, &outbuf,
+						  the_hash_algo, input_algo,
+						  content, size, type, !do_die);
+			free(content);
+			if (ret == -1)
+				return -1;
+			size = outbuf.len;
+			content = strbuf_detach(&outbuf, NULL);
+		}
+		if (input_oi->sizep)
+			*input_oi->sizep = size;
+		if (input_oi->contentp)
+			*input_oi->contentp = content;
+		else
+			free(content);
+		if (input_oi->type_name)
+			*input_oi->type_name = type_name;
+		else
+			strbuf_release(&type_name);
+	}
+	if (new_oi.delta_base_oid == &delta_base_oid) {
+		if (repo_oid_to_algop(r, &delta_base_oid, input_algo,
+				 input_oi->delta_base_oid)) {
+			if (do_die)
+				die(_("missing mapping of %s to %s"),
+				    oid_to_hex(&delta_base_oid),
+				    input_algo->name);
+			return -1;
+		}
+	}
+	input_oi->whence = new_oi.whence;
+	input_oi->u = new_oi.u;
+	return ret;
+}
+
+int oid_object_info_extended(struct repository *r, const struct object_id *oid,
+			     struct object_info *oi, unsigned flags)
+{
+	int ret;
+
+	if (oid->algo && (hash_algo_by_ptr(r->hash_algo) != oid->algo))
+		return oid_object_info_convert(r, oid, oi, flags);
+
+	obj_read_lock();
+	ret = do_oid_object_info_extended(r, oid, oi, flags);
+	obj_read_unlock();
+	return ret;
+}
+
+
+/* returns enum object_type or negative */
+int oid_object_info(struct repository *r,
+		    const struct object_id *oid,
+		    unsigned long *sizep)
+{
+	enum object_type type;
+	struct object_info oi = OBJECT_INFO_INIT;
+
+	oi.typep = &type;
+	oi.sizep = sizep;
+	if (oid_object_info_extended(r, oid, &oi,
+				      OBJECT_INFO_LOOKUP_REPLACE) < 0)
+		return -1;
+	return type;
+}
+
+int pretend_object_file(void *buf, unsigned long len, enum object_type type,
+			struct object_id *oid)
+{
+	struct cached_object_entry *co;
+	char *co_buf;
+
+	hash_object_file(the_hash_algo, buf, len, type, oid);
+	if (repo_has_object_file_with_flags(the_repository, oid, OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT) ||
+	    find_cached_object(oid))
+		return 0;
+	ALLOC_GROW(cached_objects, cached_object_nr + 1, cached_object_alloc);
+	co = &cached_objects[cached_object_nr++];
+	co->value.size = len;
+	co->value.type = type;
+	co_buf = xmalloc(len);
+	memcpy(co_buf, buf, len);
+	co->value.buf = co_buf;
+	oidcpy(&co->oid, oid);
+	return 0;
+}
+
+/*
+ * This function dies on corrupt objects; the callers who want to
+ * deal with them should arrange to call oid_object_info_extended() and give
+ * error messages themselves.
+ */
+void *repo_read_object_file(struct repository *r,
+			    const struct object_id *oid,
+			    enum object_type *type,
+			    unsigned long *size)
+{
+	struct object_info oi = OBJECT_INFO_INIT;
+	unsigned flags = OBJECT_INFO_DIE_IF_CORRUPT | OBJECT_INFO_LOOKUP_REPLACE;
+	void *data;
+
+	oi.typep = type;
+	oi.sizep = size;
+	oi.contentp = &data;
+	if (oid_object_info_extended(r, oid, &oi, flags))
+		return NULL;
+
+	return data;
+}
+
+void *read_object_with_reference(struct repository *r,
+				 const struct object_id *oid,
+				 enum object_type required_type,
+				 unsigned long *size,
+				 struct object_id *actual_oid_return)
+{
+	enum object_type type;
+	void *buffer;
+	unsigned long isize;
+	struct object_id actual_oid;
+
+	oidcpy(&actual_oid, oid);
+	while (1) {
+		int ref_length = -1;
+		const char *ref_type = NULL;
+
+		buffer = repo_read_object_file(r, &actual_oid, &type, &isize);
+		if (!buffer)
+			return NULL;
+		if (type == required_type) {
+			*size = isize;
+			if (actual_oid_return)
+				oidcpy(actual_oid_return, &actual_oid);
+			return buffer;
+		}
+		/* Handle references */
+		else if (type == OBJ_COMMIT)
+			ref_type = "tree ";
+		else if (type == OBJ_TAG)
+			ref_type = "object ";
+		else {
+			free(buffer);
+			return NULL;
+		}
+		ref_length = strlen(ref_type);
+
+		if (ref_length + the_hash_algo->hexsz > isize ||
+		    memcmp(buffer, ref_type, ref_length) ||
+		    get_oid_hex((char *) buffer + ref_length, &actual_oid)) {
+			free(buffer);
+			return NULL;
+		}
+		free(buffer);
+		/* Now we have the ID of the referred-to object in
+		 * actual_oid.  Check again. */
+	}
+}
+
+int has_object(struct repository *r, const struct object_id *oid,
+	       unsigned flags)
+{
+	int quick = !(flags & HAS_OBJECT_RECHECK_PACKED);
+	unsigned object_info_flags = OBJECT_INFO_SKIP_FETCH_OBJECT |
+		(quick ? OBJECT_INFO_QUICK : 0);
+
+	if (!startup_info->have_repository)
+		return 0;
+	return oid_object_info_extended(r, oid, NULL, object_info_flags) >= 0;
+}
+
+int repo_has_object_file_with_flags(struct repository *r,
+				    const struct object_id *oid, int flags)
+{
+	if (!startup_info->have_repository)
+		return 0;
+	return oid_object_info_extended(r, oid, NULL, flags) >= 0;
+}
+
+int repo_has_object_file(struct repository *r,
+			 const struct object_id *oid)
+{
+	return repo_has_object_file_with_flags(r, oid, 0);
+}
+
+void assert_oid_type(const struct object_id *oid, enum object_type expect)
+{
+	enum object_type type = oid_object_info(the_repository, oid, NULL);
+	if (type < 0)
+		die(_("%s is not a valid object"), oid_to_hex(oid));
+	if (type != expect)
+		die(_("%s is not a valid '%s' object"), oid_to_hex(oid),
+		    type_name(expect));
+}

-- 
2.49.0.682.gc9b6a7b2b0.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH 5/9] object-file: split up concerns of `HASH_*` flags
  2025-04-08 10:24 [PATCH 0/9] Split up "object-file.c" Patrick Steinhardt
                   ` (3 preceding siblings ...)
  2025-04-08 10:24 ` [PATCH 4/9] object-file: split out functions relating to object store subsystem Patrick Steinhardt
@ 2025-04-08 10:24 ` Patrick Steinhardt
  2025-04-08 10:24 ` [PATCH 6/9] object-file: split out functions relating to index subsystem Patrick Steinhardt
                   ` (7 subsequent siblings)
  12 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-08 10:24 UTC (permalink / raw)
  To: git

The functions `hash_object_file()`, `write_object_file()` and
`index_fd()` reuse the same set of flags to alter their behaviour. This
not only adds confusion, but given that every function only supports a
subset of the flags it becomes very hard to see which flags can be
passed to what function. Last but not least, this entangles the
implementation of all three function families.

Split up concerns by creating separate flags for each of the function
families.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/hash-object.c  | 23 +++++++++++++++++------
 builtin/replace.c      |  2 +-
 builtin/update-index.c |  2 +-
 bulk-checkin.c         |  6 +++---
 cache-tree.c           |  2 +-
 notes-merge.c          |  2 +-
 object-file.c          | 18 +++++++++---------
 object-file.h          | 25 +++++++++++++++++++++----
 read-cache.c           |  4 ++--
 9 files changed, 56 insertions(+), 28 deletions(-)

diff --git a/builtin/hash-object.c b/builtin/hash-object.c
index a25f0403f44..e7c0d6afdef 100644
--- a/builtin/hash-object.c
+++ b/builtin/hash-object.c
@@ -19,6 +19,11 @@
 #include "strbuf.h"
 #include "write-or-die.h"
 
+enum {
+	HASH_OBJECT_CHECK = (1 << 0),
+	HASH_OBJECT_WRITE = (1 << 1),
+};
+
 /*
  * This is to create corrupt objects for debugging and as such it
  * needs to bypass the data conversion performed by, and the type
@@ -33,7 +38,7 @@ static int hash_literally(struct object_id *oid, int fd, const char *type, unsig
 		ret = -1;
 	else
 		ret = write_object_file_literally(buf.buf, buf.len, type, oid,
-						 flags);
+						  (flags & HASH_OBJECT_WRITE) ? WRITE_OBJECT_FILE_PERSIST : 0);
 	close(fd);
 	strbuf_release(&buf);
 	return ret;
@@ -42,15 +47,21 @@ static int hash_literally(struct object_id *oid, int fd, const char *type, unsig
 static void hash_fd(int fd, const char *type, const char *path, unsigned flags,
 		    int literally)
 {
+	unsigned int index_flags = 0;
 	struct stat st;
 	struct object_id oid;
 
+	if (flags & HASH_OBJECT_WRITE)
+		index_flags |= INDEX_WRITE_OBJECT;
+	if (flags & HASH_OBJECT_CHECK)
+		index_flags |= INDEX_FORMAT_CHECK;
+
 	if (fstat(fd, &st) < 0 ||
 	    (literally
 	     ? hash_literally(&oid, fd, type, flags)
 	     : index_fd(the_repository->index, &oid, fd, &st,
-			type_from_string(type), path, flags)))
-		die((flags & HASH_WRITE_OBJECT)
+			type_from_string(type), path, index_flags)))
+		die((flags & HASH_OBJECT_WRITE)
 		    ? "Unable to add %s to database"
 		    : "Unable to hash %s", path);
 	printf("%s\n", oid_to_hex(&oid));
@@ -102,13 +113,13 @@ int cmd_hash_object(int argc,
 	int no_filters = 0;
 	int literally = 0;
 	int nongit = 0;
-	unsigned flags = HASH_FORMAT_CHECK;
+	unsigned flags = HASH_OBJECT_CHECK;
 	const char *vpath = NULL;
 	char *vpath_free = NULL;
 	const struct option hash_object_options[] = {
 		OPT_STRING('t', NULL, &type, N_("type"), N_("object type")),
 		OPT_BIT('w', NULL, &flags, N_("write the object into the object database"),
-			HASH_WRITE_OBJECT),
+			HASH_OBJECT_WRITE),
 		OPT_COUNTUP( 0 , "stdin", &hashstdin, N_("read the object from stdin")),
 		OPT_BOOL( 0 , "stdin-paths", &stdin_paths, N_("read file names from stdin")),
 		OPT_BOOL( 0 , "no-filters", &no_filters, N_("store file as is without filters")),
@@ -122,7 +133,7 @@ int cmd_hash_object(int argc,
 	argc = parse_options(argc, argv, prefix, hash_object_options,
 			     hash_object_usage, 0);
 
-	if (flags & HASH_WRITE_OBJECT)
+	if (flags & HASH_OBJECT_WRITE)
 		prefix = setup_git_directory();
 	else
 		prefix = setup_git_directory_gently(&nongit);
diff --git a/builtin/replace.c b/builtin/replace.c
index 15ec0922ce1..2b4fc9a68b3 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -305,7 +305,7 @@ static int import_object(struct object_id *oid, enum object_type type,
 		strbuf_release(&result);
 	} else {
 		struct stat st;
-		int flags = HASH_FORMAT_CHECK | HASH_WRITE_OBJECT;
+		int flags = INDEX_FORMAT_CHECK | INDEX_WRITE_OBJECT;
 
 		if (fstat(fd, &st) < 0) {
 			error_errno(_("unable to fstat %s"), filename);
diff --git a/builtin/update-index.c b/builtin/update-index.c
index b2f6b1a3fbb..f0cf964294d 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -304,7 +304,7 @@ static int add_one_path(const struct cache_entry *old, const char *path, int len
 	ce->ce_mode = ce_mode_from_stat(old, st->st_mode);
 
 	if (index_path(the_repository->index, &ce->oid, path, st,
-		       info_only ? 0 : HASH_WRITE_OBJECT)) {
+		       info_only ? 0 : INDEX_WRITE_OBJECT)) {
 		discard_cache_entry(ce);
 		return -1;
 	}
diff --git a/bulk-checkin.c b/bulk-checkin.c
index 23ac00ea0a6..309201a76a6 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -171,7 +171,7 @@ static int stream_blob_to_pack(struct bulk_checkin_packfile *state,
 	unsigned char obuf[16384];
 	unsigned hdrlen;
 	int status = Z_OK;
-	int write_object = (flags & HASH_WRITE_OBJECT);
+	int write_object = (flags & INDEX_WRITE_OBJECT);
 	off_t offset = 0;
 
 	git_deflate_init(&s, pack_compression_level);
@@ -241,7 +241,7 @@ static int stream_blob_to_pack(struct bulk_checkin_packfile *state,
 static void prepare_to_stream(struct bulk_checkin_packfile *state,
 			      unsigned flags)
 {
-	if (!(flags & HASH_WRITE_OBJECT) || state->f)
+	if (!(flags & INDEX_WRITE_OBJECT) || state->f)
 		return;
 
 	state->f = create_tmp_packfile(the_repository, &state->pack_tmp_name);
@@ -275,7 +275,7 @@ static int deflate_blob_to_pack(struct bulk_checkin_packfile *state,
 	git_hash_update(&ctx, obuf, header_len);
 
 	/* Note: idx is non-NULL when we are writing */
-	if ((flags & HASH_WRITE_OBJECT) != 0) {
+	if ((flags & INDEX_WRITE_OBJECT) != 0) {
 		CALLOC_ARRAY(idx, 1);
 
 		prepare_to_stream(state, flags);
diff --git a/cache-tree.c b/cache-tree.c
index bcbcad3d61a..4c8167ea927 100644
--- a/cache-tree.c
+++ b/cache-tree.c
@@ -452,7 +452,7 @@ static int update_one(struct cache_tree *it,
 				 OBJ_TREE, &it->oid);
 	} else if (write_object_file_flags(buffer.buf, buffer.len, OBJ_TREE,
 					   &it->oid, NULL, flags & WRITE_TREE_SILENT
-					   ? HASH_SILENT : 0)) {
+					   ? WRITE_OBJECT_FILE_SILENT : 0)) {
 		strbuf_release(&buffer);
 		return -1;
 	}
diff --git a/notes-merge.c b/notes-merge.c
index 5008faef450..d9f2914c819 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -729,7 +729,7 @@ int notes_merge_commit(struct notes_merge_options *o,
 		/* write file as blob, and add to partial_tree */
 		if (stat(path.buf, &st))
 			die_errno("Failed to stat '%s'", path.buf);
-		if (index_path(o->repo->index, &blob_oid, path.buf, &st, HASH_WRITE_OBJECT))
+		if (index_path(o->repo->index, &blob_oid, path.buf, &st, INDEX_WRITE_OBJECT))
 			die("Failed to write blob object from '%s'", path.buf);
 		if (add_note(partial_tree, &obj_oid, &blob_oid, NULL))
 			die("Failed to add resolved note '%s' to notes tree",
diff --git a/object-file.c b/object-file.c
index 1c74aa239fb..dea5a70a331 100644
--- a/object-file.c
+++ b/object-file.c
@@ -33,9 +33,9 @@
 
 static int get_conv_flags(unsigned flags)
 {
-	if (flags & HASH_RENORMALIZE)
+	if (flags & INDEX_RENORMALIZE)
 		return CONV_EOL_RENORMALIZE;
-	else if (flags & HASH_WRITE_OBJECT)
+	else if (flags & INDEX_WRITE_OBJECT)
 		return global_conv_flags_eol | CONV_WRITE_OBJECT;
 	else
 		return 0;
@@ -835,7 +835,7 @@ static int start_loose_object_common(struct strbuf *tmp_file,
 
 	fd = create_tmpfile(tmp_file, filename);
 	if (fd < 0) {
-		if (flags & HASH_SILENT)
+		if (flags & WRITE_OBJECT_FILE_SILENT)
 			return -1;
 		else if (errno == EACCES)
 			return error(_("insufficient permission for adding "
@@ -967,7 +967,7 @@ static int write_loose_object(const struct object_id *oid, char *hdr,
 		utb.actime = mtime;
 		utb.modtime = mtime;
 		if (utime(tmp_file.buf, &utb) < 0 &&
-		    !(flags & HASH_SILENT))
+		    !(flags & WRITE_OBJECT_FILE_SILENT))
 			warning_errno(_("failed utime() on %s"), tmp_file.buf);
 	}
 
@@ -1178,7 +1178,7 @@ int write_object_file_literally(const void *buf, unsigned long len,
 	write_object_file_prepare_literally(the_hash_algo, buf, len, type,
 					    oid, header, &hdrlen);
 
-	if (!(flags & HASH_WRITE_OBJECT))
+	if (!(flags & WRITE_OBJECT_FILE_PERSIST))
 		goto cleanup;
 	if (freshen_packed_object(oid) || freshen_loose_object(oid))
 		goto cleanup;
@@ -1249,7 +1249,7 @@ static int index_mem(struct index_state *istate,
 {
 	struct strbuf nbuf = STRBUF_INIT;
 	int ret = 0;
-	int write_object = flags & HASH_WRITE_OBJECT;
+	int write_object = flags & INDEX_WRITE_OBJECT;
 
 	if (!type)
 		type = OBJ_BLOB;
@@ -1264,7 +1264,7 @@ static int index_mem(struct index_state *istate,
 			size = nbuf.len;
 		}
 	}
-	if (flags & HASH_FORMAT_CHECK) {
+	if (flags & INDEX_FORMAT_CHECK) {
 		struct fsck_options opts = FSCK_OPTIONS_DEFAULT;
 
 		opts.strict = 1;
@@ -1290,7 +1290,7 @@ static int index_stream_convert_blob(struct index_state *istate,
 				     unsigned flags)
 {
 	int ret = 0;
-	const int write_object = flags & HASH_WRITE_OBJECT;
+	const int write_object = flags & INDEX_WRITE_OBJECT;
 	struct strbuf sbuf = STRBUF_INIT;
 
 	assert(path);
@@ -1422,7 +1422,7 @@ int index_path(struct index_state *istate, struct object_id *oid,
 	case S_IFLNK:
 		if (strbuf_readlink(&sb, path, st->st_size))
 			return error_errno("readlink(\"%s\")", path);
-		if (!(flags & HASH_WRITE_OBJECT))
+		if (!(flags & INDEX_WRITE_OBJECT))
 			hash_object_file(the_hash_algo, sb.buf, sb.len,
 					 OBJ_BLOB, oid);
 		else if (write_object_file(sb.buf, sb.len, OBJ_BLOB, oid))
diff --git a/object-file.h b/object-file.h
index 78c84d970a9..c002fbe2345 100644
--- a/object-file.h
+++ b/object-file.h
@@ -14,10 +14,12 @@ struct index_state;
  */
 extern int fetch_if_missing;
 
-#define HASH_WRITE_OBJECT 1
-#define HASH_FORMAT_CHECK 2
-#define HASH_RENORMALIZE  4
-#define HASH_SILENT 8
+enum {
+	INDEX_WRITE_OBJECT = (1 << 0),
+	INDEX_FORMAT_CHECK = (1 << 1),
+	INDEX_RENORMALIZE  = (1 << 2),
+};
+
 int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);
 int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags);
 
@@ -84,6 +86,21 @@ enum unpack_loose_header_result unpack_loose_header(git_zstream *stream,
 struct object_info;
 int parse_loose_header(const char *hdr, struct object_info *oi);
 
+enum {
+	/*
+	 * By default, `write_object_file_literally()` does not actually write
+	 * anything into the object store, but only computes the object ID.
+	 * This flag changes that so that the object will be written as a loose
+	 * object and persisted.
+	 */
+	WRITE_OBJECT_FILE_PERSIST = (1 << 0),
+
+	/*
+	 * Do not print an error in case something gose wrong.
+	 */
+	WRITE_OBJECT_FILE_SILENT = (1 << 1),
+};
+
 int write_object_file_flags(const void *buf, unsigned long len,
 			    enum object_type type, struct object_id *oid,
 			    struct object_id *comapt_oid_in, unsigned flags);
diff --git a/read-cache.c b/read-cache.c
index 2f9e21c897d..23028f43a11 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -706,11 +706,11 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st,
 	int intent_only = flags & ADD_CACHE_INTENT;
 	int add_option = (ADD_CACHE_OK_TO_ADD|ADD_CACHE_OK_TO_REPLACE|
 			  (intent_only ? ADD_CACHE_NEW_ONLY : 0));
-	unsigned hash_flags = pretend ? 0 : HASH_WRITE_OBJECT;
+	unsigned hash_flags = pretend ? 0 : INDEX_WRITE_OBJECT;
 	struct object_id oid;
 
 	if (flags & ADD_CACHE_RENORMALIZE)
-		hash_flags |= HASH_RENORMALIZE;
+		hash_flags |= INDEX_RENORMALIZE;
 
 	if (!S_ISREG(st_mode) && !S_ISLNK(st_mode) && !S_ISDIR(st_mode))
 		return error(_("%s: can only add regular files, symbolic links or git-directories"), path);

-- 
2.49.0.682.gc9b6a7b2b0.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH 6/9] object-file: split out functions relating to index subsystem
  2025-04-08 10:24 [PATCH 0/9] Split up "object-file.c" Patrick Steinhardt
                   ` (4 preceding siblings ...)
  2025-04-08 10:24 ` [PATCH 5/9] object-file: split up concerns of `HASH_*` flags Patrick Steinhardt
@ 2025-04-08 10:24 ` Patrick Steinhardt
  2025-04-08 10:24 ` [PATCH 7/9] object: split out functions relating to object store subsystem Patrick Steinhardt
                   ` (6 subsequent siblings)
  12 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-08 10:24 UTC (permalink / raw)
  To: git

Split out functions relating to the index subsystem from "object-file.c"
to help us separate concerns.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/difftool.c    |   2 +-
 builtin/hash-object.c |   1 +
 builtin/replace.c     |   1 +
 bulk-checkin.c        |   1 +
 diff.c                |   2 +-
 notes-merge.c         |   1 +
 object-file.c         | 225 --------------------------------------------------
 object-file.h         |  11 ---
 read-cache.c          | 224 +++++++++++++++++++++++++++++++++++++++++++++++++
 read-cache.h          |   9 ++
 10 files changed, 239 insertions(+), 238 deletions(-)

diff --git a/builtin/difftool.c b/builtin/difftool.c
index 41cd00066cc..45d6ea8a801 100644
--- a/builtin/difftool.c
+++ b/builtin/difftool.c
@@ -22,7 +22,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "parse-options.h"
-#include "read-cache-ll.h"
+#include "read-cache.h"
 #include "repository.h"
 #include "sparse-index.h"
 #include "strvec.h"
diff --git a/builtin/hash-object.c b/builtin/hash-object.c
index e7c0d6afdef..7aa889b1b4a 100644
--- a/builtin/hash-object.c
+++ b/builtin/hash-object.c
@@ -15,6 +15,7 @@
 #include "blob.h"
 #include "quote.h"
 #include "parse-options.h"
+#include "read-cache.h"
 #include "setup.h"
 #include "strbuf.h"
 #include "write-or-die.h"
diff --git a/builtin/replace.c b/builtin/replace.c
index 2b4fc9a68b3..ce2948a9b2a 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -20,6 +20,7 @@
 #include "object-file.h"
 #include "object-name.h"
 #include "object-store-ll.h"
+#include "read-cache.h"
 #include "replace-object.h"
 #include "tag.h"
 #include "wildmatch.h"
diff --git a/bulk-checkin.c b/bulk-checkin.c
index 309201a76a6..0b9bad92868 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -11,6 +11,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "lockfile.h"
+#include "read-cache.h"
 #include "repository.h"
 #include "csum-file.h"
 #include "pack.h"
diff --git a/diff.c b/diff.c
index 3bcf5028831..c361199ab1b 100644
--- a/diff.c
+++ b/diff.c
@@ -42,7 +42,7 @@
 #include "dir.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "read-cache-ll.h"
+#include "read-cache.h"
 #include "setup.h"
 #include "strmap.h"
 #include "ws.h"
diff --git a/notes-merge.c b/notes-merge.c
index d9f2914c819..13750490eff 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -10,6 +10,7 @@
 #include "object-name.h"
 #include "object-store-ll.h"
 #include "path.h"
+#include "read-cache.h"
 #include "repository.h"
 #include "diff.h"
 #include "diffcore.h"
diff --git a/object-file.c b/object-file.c
index dea5a70a331..235d29858ff 100644
--- a/object-file.c
+++ b/object-file.c
@@ -12,9 +12,7 @@
 
 #include "git-compat-util.h"
 #include "bulk-checkin.h"
-#include "convert.h"
 #include "environment.h"
-#include "fsck.h"
 #include "gettext.h"
 #include "hex.h"
 #include "loose.h"
@@ -25,22 +23,11 @@
 #include "pack.h"
 #include "packfile.h"
 #include "path.h"
-#include "setup.h"
 #include "streaming.h"
 
 /* The maximum size for an object header. */
 #define MAX_HEADER_LEN 32
 
-static int get_conv_flags(unsigned flags)
-{
-	if (flags & INDEX_RENORMALIZE)
-		return CONV_EOL_RENORMALIZE;
-	else if (flags & INDEX_WRITE_OBJECT)
-		return global_conv_flags_eol | CONV_WRITE_OBJECT;
-	else
-		return 0;
-}
-
 static void fill_loose_path(struct strbuf *buf, const struct object_id *oid)
 {
 	int i;
@@ -1225,218 +1212,6 @@ int force_object_loose(const struct object_id *oid, time_t mtime)
 	return ret;
 }
 
-/*
- * We can't use the normal fsck_error_function() for index_mem(),
- * because we don't yet have a valid oid for it to report. Instead,
- * report the minimal fsck error here, and rely on the caller to
- * give more context.
- */
-static int hash_format_check_report(struct fsck_options *opts UNUSED,
-				    void *fsck_report UNUSED,
-				    enum fsck_msg_type msg_type UNUSED,
-				    enum fsck_msg_id msg_id UNUSED,
-				    const char *message)
-{
-	error(_("object fails fsck: %s"), message);
-	return 1;
-}
-
-static int index_mem(struct index_state *istate,
-		     struct object_id *oid,
-		     const void *buf, size_t size,
-		     enum object_type type,
-		     const char *path, unsigned flags)
-{
-	struct strbuf nbuf = STRBUF_INIT;
-	int ret = 0;
-	int write_object = flags & INDEX_WRITE_OBJECT;
-
-	if (!type)
-		type = OBJ_BLOB;
-
-	/*
-	 * Convert blobs to git internal format
-	 */
-	if ((type == OBJ_BLOB) && path) {
-		if (convert_to_git(istate, path, buf, size, &nbuf,
-				   get_conv_flags(flags))) {
-			buf = nbuf.buf;
-			size = nbuf.len;
-		}
-	}
-	if (flags & INDEX_FORMAT_CHECK) {
-		struct fsck_options opts = FSCK_OPTIONS_DEFAULT;
-
-		opts.strict = 1;
-		opts.error_func = hash_format_check_report;
-		if (fsck_buffer(null_oid(the_hash_algo), type, buf, size, &opts))
-			die(_("refusing to create malformed object"));
-		fsck_finish(&opts);
-	}
-
-	if (write_object)
-		ret = write_object_file(buf, size, type, oid);
-	else
-		hash_object_file(the_hash_algo, buf, size, type, oid);
-
-	strbuf_release(&nbuf);
-	return ret;
-}
-
-static int index_stream_convert_blob(struct index_state *istate,
-				     struct object_id *oid,
-				     int fd,
-				     const char *path,
-				     unsigned flags)
-{
-	int ret = 0;
-	const int write_object = flags & INDEX_WRITE_OBJECT;
-	struct strbuf sbuf = STRBUF_INIT;
-
-	assert(path);
-	assert(would_convert_to_git_filter_fd(istate, path));
-
-	convert_to_git_filter_fd(istate, path, fd, &sbuf,
-				 get_conv_flags(flags));
-
-	if (write_object)
-		ret = write_object_file(sbuf.buf, sbuf.len, OBJ_BLOB,
-					oid);
-	else
-		hash_object_file(the_hash_algo, sbuf.buf, sbuf.len, OBJ_BLOB,
-				 oid);
-	strbuf_release(&sbuf);
-	return ret;
-}
-
-static int index_pipe(struct index_state *istate, struct object_id *oid,
-		      int fd, enum object_type type,
-		      const char *path, unsigned flags)
-{
-	struct strbuf sbuf = STRBUF_INIT;
-	int ret;
-
-	if (strbuf_read(&sbuf, fd, 4096) >= 0)
-		ret = index_mem(istate, oid, sbuf.buf, sbuf.len, type, path, flags);
-	else
-		ret = -1;
-	strbuf_release(&sbuf);
-	return ret;
-}
-
-#define SMALL_FILE_SIZE (32*1024)
-
-static int index_core(struct index_state *istate,
-		      struct object_id *oid, int fd, size_t size,
-		      enum object_type type, const char *path,
-		      unsigned flags)
-{
-	int ret;
-
-	if (!size) {
-		ret = index_mem(istate, oid, "", size, type, path, flags);
-	} else if (size <= SMALL_FILE_SIZE) {
-		char *buf = xmalloc(size);
-		ssize_t read_result = read_in_full(fd, buf, size);
-		if (read_result < 0)
-			ret = error_errno(_("read error while indexing %s"),
-					  path ? path : "<unknown>");
-		else if (read_result != size)
-			ret = error(_("short read while indexing %s"),
-				    path ? path : "<unknown>");
-		else
-			ret = index_mem(istate, oid, buf, size, type, path, flags);
-		free(buf);
-	} else {
-		void *buf = xmmap(NULL, size, PROT_READ, MAP_PRIVATE, fd, 0);
-		ret = index_mem(istate, oid, buf, size, type, path, flags);
-		munmap(buf, size);
-	}
-	return ret;
-}
-
-/*
- * This creates one packfile per large blob unless bulk-checkin
- * machinery is "plugged".
- *
- * This also bypasses the usual "convert-to-git" dance, and that is on
- * purpose. We could write a streaming version of the converting
- * functions and insert that before feeding the data to fast-import
- * (or equivalent in-core API described above). However, that is
- * somewhat complicated, as we do not know the size of the filter
- * result, which we need to know beforehand when writing a git object.
- * Since the primary motivation for trying to stream from the working
- * tree file and to avoid mmaping it in core is to deal with large
- * binary blobs, they generally do not want to get any conversion, and
- * callers should avoid this code path when filters are requested.
- */
-static int index_blob_stream(struct object_id *oid, int fd, size_t size,
-			     const char *path,
-			     unsigned flags)
-{
-	return index_blob_bulk_checkin(oid, fd, size, path, flags);
-}
-
-int index_fd(struct index_state *istate, struct object_id *oid,
-	     int fd, struct stat *st,
-	     enum object_type type, const char *path, unsigned flags)
-{
-	int ret;
-
-	/*
-	 * Call xsize_t() only when needed to avoid potentially unnecessary
-	 * die() for large files.
-	 */
-	if (type == OBJ_BLOB && path && would_convert_to_git_filter_fd(istate, path))
-		ret = index_stream_convert_blob(istate, oid, fd, path, flags);
-	else if (!S_ISREG(st->st_mode))
-		ret = index_pipe(istate, oid, fd, type, path, flags);
-	else if (st->st_size <= repo_settings_get_big_file_threshold(the_repository) ||
-		 type != OBJ_BLOB ||
-		 (path && would_convert_to_git(istate, path)))
-		ret = index_core(istate, oid, fd, xsize_t(st->st_size),
-				 type, path, flags);
-	else
-		ret = index_blob_stream(oid, fd, xsize_t(st->st_size), path,
-					flags);
-	close(fd);
-	return ret;
-}
-
-int index_path(struct index_state *istate, struct object_id *oid,
-	       const char *path, struct stat *st, unsigned flags)
-{
-	int fd;
-	struct strbuf sb = STRBUF_INIT;
-	int rc = 0;
-
-	switch (st->st_mode & S_IFMT) {
-	case S_IFREG:
-		fd = open(path, O_RDONLY);
-		if (fd < 0)
-			return error_errno("open(\"%s\")", path);
-		if (index_fd(istate, oid, fd, st, OBJ_BLOB, path, flags) < 0)
-			return error(_("%s: failed to insert into database"),
-				     path);
-		break;
-	case S_IFLNK:
-		if (strbuf_readlink(&sb, path, st->st_size))
-			return error_errno("readlink(\"%s\")", path);
-		if (!(flags & INDEX_WRITE_OBJECT))
-			hash_object_file(the_hash_algo, sb.buf, sb.len,
-					 OBJ_BLOB, oid);
-		else if (write_object_file(sb.buf, sb.len, OBJ_BLOB, oid))
-			rc = error(_("%s: failed to insert into database"), path);
-		strbuf_release(&sb);
-		break;
-	case S_IFDIR:
-		return repo_resolve_gitlink_ref(the_repository, path, "HEAD", oid);
-	default:
-		return error(_("%s: unsupported file type"), path);
-	}
-	return rc;
-}
-
 int read_pack_header(int fd, struct pack_header *header)
 {
 	if (read_in_full(fd, header, sizeof(*header)) != sizeof(*header))
diff --git a/object-file.h b/object-file.h
index c002fbe2345..7d48994f1b6 100644
--- a/object-file.h
+++ b/object-file.h
@@ -4,8 +4,6 @@
 #include "git-zlib.h"
 #include "object.h"
 
-struct index_state;
-
 /*
  * Set this to 0 to prevent oid_object_info_extended() from fetching missing
  * blobs. This has a difference only if extensions.partialClone is set.
@@ -14,15 +12,6 @@ struct index_state;
  */
 extern int fetch_if_missing;
 
-enum {
-	INDEX_WRITE_OBJECT = (1 << 0),
-	INDEX_FORMAT_CHECK = (1 << 1),
-	INDEX_RENORMALIZE  = (1 << 2),
-};
-
-int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);
-int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags);
-
 struct object_directory;
 
 const char *odb_loose_path(struct object_directory *odb,
diff --git a/read-cache.c b/read-cache.c
index 23028f43a11..9909b56902f 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -10,9 +10,11 @@
 #include "git-compat-util.h"
 #include "bulk-checkin.h"
 #include "config.h"
+#include "convert.h"
 #include "date.h"
 #include "diff.h"
 #include "diffcore.h"
+#include "fsck.h"
 #include "hex.h"
 #include "tempfile.h"
 #include "lockfile.h"
@@ -4007,3 +4009,225 @@ int add_files_to_cache(struct repository *repo, const char *prefix,
 	release_revisions(&rev);
 	return !!data.add_errors;
 }
+
+static int get_conv_flags(unsigned flags)
+{
+	if (flags & INDEX_RENORMALIZE)
+		return CONV_EOL_RENORMALIZE;
+	else if (flags & INDEX_WRITE_OBJECT)
+		return global_conv_flags_eol | CONV_WRITE_OBJECT;
+	else
+		return 0;
+}
+
+/*
+ * We can't use the normal fsck_error_function() for index_mem(),
+ * because we don't yet have a valid oid for it to report. Instead,
+ * report the minimal fsck error here, and rely on the caller to
+ * give more context.
+ */
+static int hash_format_check_report(struct fsck_options *opts UNUSED,
+				    void *fsck_report UNUSED,
+				    enum fsck_msg_type msg_type UNUSED,
+				    enum fsck_msg_id msg_id UNUSED,
+				    const char *message)
+{
+	error(_("object fails fsck: %s"), message);
+	return 1;
+}
+
+static int index_mem(struct index_state *istate,
+		     struct object_id *oid,
+		     const void *buf, size_t size,
+		     enum object_type type,
+		     const char *path, unsigned flags)
+{
+	struct strbuf nbuf = STRBUF_INIT;
+	int ret = 0;
+	int write_object = flags & INDEX_WRITE_OBJECT;
+
+	if (!type)
+		type = OBJ_BLOB;
+
+	/*
+	 * Convert blobs to git internal format
+	 */
+	if ((type == OBJ_BLOB) && path) {
+		if (convert_to_git(istate, path, buf, size, &nbuf,
+				   get_conv_flags(flags))) {
+			buf = nbuf.buf;
+			size = nbuf.len;
+		}
+	}
+	if (flags & INDEX_FORMAT_CHECK) {
+		struct fsck_options opts = FSCK_OPTIONS_DEFAULT;
+
+		opts.strict = 1;
+		opts.error_func = hash_format_check_report;
+		if (fsck_buffer(null_oid(the_hash_algo), type, buf, size, &opts))
+			die(_("refusing to create malformed object"));
+		fsck_finish(&opts);
+	}
+
+	if (write_object)
+		ret = write_object_file(buf, size, type, oid);
+	else
+		hash_object_file(the_hash_algo, buf, size, type, oid);
+
+	strbuf_release(&nbuf);
+	return ret;
+}
+
+static int index_stream_convert_blob(struct index_state *istate,
+				     struct object_id *oid,
+				     int fd,
+				     const char *path,
+				     unsigned flags)
+{
+	int ret = 0;
+	const int write_object = flags & INDEX_WRITE_OBJECT;
+	struct strbuf sbuf = STRBUF_INIT;
+
+	assert(path);
+	assert(would_convert_to_git_filter_fd(istate, path));
+
+	convert_to_git_filter_fd(istate, path, fd, &sbuf,
+				 get_conv_flags(flags));
+
+	if (write_object)
+		ret = write_object_file(sbuf.buf, sbuf.len, OBJ_BLOB,
+					oid);
+	else
+		hash_object_file(the_hash_algo, sbuf.buf, sbuf.len, OBJ_BLOB,
+				 oid);
+	strbuf_release(&sbuf);
+	return ret;
+}
+
+static int index_pipe(struct index_state *istate, struct object_id *oid,
+		      int fd, enum object_type type,
+		      const char *path, unsigned flags)
+{
+	struct strbuf sbuf = STRBUF_INIT;
+	int ret;
+
+	if (strbuf_read(&sbuf, fd, 4096) >= 0)
+		ret = index_mem(istate, oid, sbuf.buf, sbuf.len, type, path, flags);
+	else
+		ret = -1;
+	strbuf_release(&sbuf);
+	return ret;
+}
+
+#define SMALL_FILE_SIZE (32*1024)
+
+static int index_core(struct index_state *istate,
+		      struct object_id *oid, int fd, size_t size,
+		      enum object_type type, const char *path,
+		      unsigned flags)
+{
+	int ret;
+
+	if (!size) {
+		ret = index_mem(istate, oid, "", size, type, path, flags);
+	} else if (size <= SMALL_FILE_SIZE) {
+		char *buf = xmalloc(size);
+		ssize_t read_result = read_in_full(fd, buf, size);
+		if (read_result < 0)
+			ret = error_errno(_("read error while indexing %s"),
+					  path ? path : "<unknown>");
+		else if (read_result != size)
+			ret = error(_("short read while indexing %s"),
+				    path ? path : "<unknown>");
+		else
+			ret = index_mem(istate, oid, buf, size, type, path, flags);
+		free(buf);
+	} else {
+		void *buf = xmmap(NULL, size, PROT_READ, MAP_PRIVATE, fd, 0);
+		ret = index_mem(istate, oid, buf, size, type, path, flags);
+		munmap(buf, size);
+	}
+	return ret;
+}
+
+/*
+ * This creates one packfile per large blob unless bulk-checkin
+ * machinery is "plugged".
+ *
+ * This also bypasses the usual "convert-to-git" dance, and that is on
+ * purpose. We could write a streaming version of the converting
+ * functions and insert that before feeding the data to fast-import
+ * (or equivalent in-core API described above). However, that is
+ * somewhat complicated, as we do not know the size of the filter
+ * result, which we need to know beforehand when writing a git object.
+ * Since the primary motivation for trying to stream from the working
+ * tree file and to avoid mmaping it in core is to deal with large
+ * binary blobs, they generally do not want to get any conversion, and
+ * callers should avoid this code path when filters are requested.
+ */
+static int index_blob_stream(struct object_id *oid, int fd, size_t size,
+			     const char *path,
+			     unsigned flags)
+{
+	return index_blob_bulk_checkin(oid, fd, size, path, flags);
+}
+
+int index_fd(struct index_state *istate, struct object_id *oid,
+	     int fd, struct stat *st,
+	     enum object_type type, const char *path, unsigned flags)
+{
+	int ret;
+
+	/*
+	 * Call xsize_t() only when needed to avoid potentially unnecessary
+	 * die() for large files.
+	 */
+	if (type == OBJ_BLOB && path && would_convert_to_git_filter_fd(istate, path))
+		ret = index_stream_convert_blob(istate, oid, fd, path, flags);
+	else if (!S_ISREG(st->st_mode))
+		ret = index_pipe(istate, oid, fd, type, path, flags);
+	else if (st->st_size <= repo_settings_get_big_file_threshold(the_repository) ||
+		 type != OBJ_BLOB ||
+		 (path && would_convert_to_git(istate, path)))
+		ret = index_core(istate, oid, fd, xsize_t(st->st_size),
+				 type, path, flags);
+	else
+		ret = index_blob_stream(oid, fd, xsize_t(st->st_size), path,
+					flags);
+	close(fd);
+	return ret;
+}
+
+int index_path(struct index_state *istate, struct object_id *oid,
+	       const char *path, struct stat *st, unsigned flags)
+{
+	int fd;
+	struct strbuf sb = STRBUF_INIT;
+	int rc = 0;
+
+	switch (st->st_mode & S_IFMT) {
+	case S_IFREG:
+		fd = open(path, O_RDONLY);
+		if (fd < 0)
+			return error_errno("open(\"%s\")", path);
+		if (index_fd(istate, oid, fd, st, OBJ_BLOB, path, flags) < 0)
+			return error(_("%s: failed to insert into database"),
+				     path);
+		break;
+	case S_IFLNK:
+		if (strbuf_readlink(&sb, path, st->st_size))
+			return error_errno("readlink(\"%s\")", path);
+		if (!(flags & INDEX_WRITE_OBJECT))
+			hash_object_file(the_hash_algo, sb.buf, sb.len,
+					 OBJ_BLOB, oid);
+		else if (write_object_file(sb.buf, sb.len, OBJ_BLOB, oid))
+			rc = error(_("%s: failed to insert into database"), path);
+		strbuf_release(&sb);
+		break;
+	case S_IFDIR:
+		return repo_resolve_gitlink_ref(the_repository, path, "HEAD", oid);
+	default:
+		return error(_("%s: unsupported file type"), path);
+	}
+	return rc;
+}
diff --git a/read-cache.h b/read-cache.h
index 043da1f1aae..d655eb64bb7 100644
--- a/read-cache.h
+++ b/read-cache.h
@@ -42,4 +42,13 @@ static inline int ce_path_match(struct index_state *istate,
 			      S_ISDIR(ce->ce_mode) || S_ISGITLINK(ce->ce_mode));
 }
 
+enum {
+	INDEX_WRITE_OBJECT = (1 << 0),
+	INDEX_FORMAT_CHECK = (1 << 1),
+	INDEX_RENORMALIZE  = (1 << 2),
+};
+
+int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);
+int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags);
+
 #endif /* READ_CACHE_H */

-- 
2.49.0.682.gc9b6a7b2b0.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH 7/9] object: split out functions relating to object store subsystem
  2025-04-08 10:24 [PATCH 0/9] Split up "object-file.c" Patrick Steinhardt
                   ` (5 preceding siblings ...)
  2025-04-08 10:24 ` [PATCH 6/9] object-file: split out functions relating to index subsystem Patrick Steinhardt
@ 2025-04-08 10:24 ` Patrick Steinhardt
  2025-04-08 10:24 ` [PATCH 8/9] object-store: remove global array of cached objects Patrick Steinhardt
                   ` (5 subsequent siblings)
  12 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-08 10:24 UTC (permalink / raw)
  To: git

Split out functions relating to the object store subsystem from
"object.c". This helps us to separate concerns.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 object-store-ll.h |  3 ---
 object-store.c    | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 object.c          | 67 -------------------------------------------------------
 3 files changed, 66 insertions(+), 70 deletions(-)

diff --git a/object-store-ll.h b/object-store-ll.h
index 8ae80b8a5fa..8bb0f33f9a8 100644
--- a/object-store-ll.h
+++ b/object-store-ll.h
@@ -92,9 +92,6 @@ struct oidtree *odb_loose_cache(struct object_directory *odb,
 /* Empty the loose object cache for the specified object directory. */
 void odb_clear_loose_cache(struct object_directory *odb);
 
-/* Clear and free the specified object directory */
-void free_object_directory(struct object_directory *odb);
-
 struct packed_git {
 	struct hashmap_entry packmap_ent;
 	struct packed_git *next;
diff --git a/object-store.c b/object-store.c
index e5f1f00cdde..ea2d86c429b 100644
--- a/object-store.c
+++ b/object-store.c
@@ -2,11 +2,13 @@
 
 #include "git-compat-util.h"
 #include "abspath.h"
+#include "commit-graph.h"
 #include "config.h"
 #include "environment.h"
 #include "gettext.h"
 #include "hex.h"
 #include "lockfile.h"
+#include "loose.h"
 #include "object-file-convert.h"
 #include "object-file.h"
 #include "object-store.h"
@@ -361,6 +363,14 @@ struct object_directory *set_temporary_primary_odb(const char *dir, int will_des
 	return new_odb->next;
 }
 
+static void free_object_directory(struct object_directory *odb)
+{
+	free(odb->path);
+	odb_clear_loose_cache(odb);
+	loose_object_map_clear(&odb->loose_map);
+	free(odb);
+}
+
 void restore_primary_odb(struct object_directory *restore_odb, const char *old_path)
 {
 	struct object_directory *cur_odb = the_repository->objects->odb;
@@ -970,3 +980,59 @@ void assert_oid_type(const struct object_id *oid, enum object_type expect)
 		die(_("%s is not a valid '%s' object"), oid_to_hex(oid),
 		    type_name(expect));
 }
+
+struct raw_object_store *raw_object_store_new(void)
+{
+	struct raw_object_store *o = xmalloc(sizeof(*o));
+
+	memset(o, 0, sizeof(*o));
+	INIT_LIST_HEAD(&o->packed_git_mru);
+	hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0);
+	pthread_mutex_init(&o->replace_mutex, NULL);
+	return o;
+}
+
+static void free_object_directories(struct raw_object_store *o)
+{
+	while (o->odb) {
+		struct object_directory *next;
+
+		next = o->odb->next;
+		free_object_directory(o->odb);
+		o->odb = next;
+	}
+	kh_destroy_odb_path_map(o->odb_by_path);
+	o->odb_by_path = NULL;
+}
+
+void raw_object_store_clear(struct raw_object_store *o)
+{
+	FREE_AND_NULL(o->alternate_db);
+
+	oidmap_free(o->replace_map, 1);
+	FREE_AND_NULL(o->replace_map);
+	pthread_mutex_destroy(&o->replace_mutex);
+
+	free_commit_graph(o->commit_graph);
+	o->commit_graph = NULL;
+	o->commit_graph_attempted = 0;
+
+	free_object_directories(o);
+	o->odb_tail = NULL;
+	o->loaded_alternates = 0;
+
+	INIT_LIST_HEAD(&o->packed_git_mru);
+	close_object_store(o);
+
+	/*
+	 * `close_object_store()` only closes the packfiles, but doesn't free
+	 * them. We thus have to do this manually.
+	 */
+	for (struct packed_git *p = o->packed_git, *next; p; p = next) {
+		next = p->next;
+		free(p);
+	}
+	o->packed_git = NULL;
+
+	hashmap_clear(&o->pack_map);
+}
diff --git a/object.c b/object.c
index 154525a4972..ccda798b75f 100644
--- a/object.c
+++ b/object.c
@@ -6,16 +6,13 @@
 #include "object.h"
 #include "replace-object.h"
 #include "object-file.h"
-#include "object-store.h"
 #include "blob.h"
 #include "statinfo.h"
 #include "tree.h"
 #include "commit.h"
 #include "tag.h"
 #include "alloc.h"
-#include "packfile.h"
 #include "commit-graph.h"
-#include "loose.h"
 
 unsigned int get_max_object_index(const struct repository *repo)
 {
@@ -567,70 +564,6 @@ struct parsed_object_pool *parsed_object_pool_new(struct repository *repo)
 	return o;
 }
 
-struct raw_object_store *raw_object_store_new(void)
-{
-	struct raw_object_store *o = xmalloc(sizeof(*o));
-
-	memset(o, 0, sizeof(*o));
-	INIT_LIST_HEAD(&o->packed_git_mru);
-	hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0);
-	pthread_mutex_init(&o->replace_mutex, NULL);
-	return o;
-}
-
-void free_object_directory(struct object_directory *odb)
-{
-	free(odb->path);
-	odb_clear_loose_cache(odb);
-	loose_object_map_clear(&odb->loose_map);
-	free(odb);
-}
-
-static void free_object_directories(struct raw_object_store *o)
-{
-	while (o->odb) {
-		struct object_directory *next;
-
-		next = o->odb->next;
-		free_object_directory(o->odb);
-		o->odb = next;
-	}
-	kh_destroy_odb_path_map(o->odb_by_path);
-	o->odb_by_path = NULL;
-}
-
-void raw_object_store_clear(struct raw_object_store *o)
-{
-	FREE_AND_NULL(o->alternate_db);
-
-	oidmap_free(o->replace_map, 1);
-	FREE_AND_NULL(o->replace_map);
-	pthread_mutex_destroy(&o->replace_mutex);
-
-	free_commit_graph(o->commit_graph);
-	o->commit_graph = NULL;
-	o->commit_graph_attempted = 0;
-
-	free_object_directories(o);
-	o->odb_tail = NULL;
-	o->loaded_alternates = 0;
-
-	INIT_LIST_HEAD(&o->packed_git_mru);
-	close_object_store(o);
-
-	/*
-	 * `close_object_store()` only closes the packfiles, but doesn't free
-	 * them. We thus have to do this manually.
-	 */
-	for (struct packed_git *p = o->packed_git, *next; p; p = next) {
-		next = p->next;
-		free(p);
-	}
-	o->packed_git = NULL;
-
-	hashmap_clear(&o->pack_map);
-}
-
 void parsed_object_pool_reset_commit_grafts(struct parsed_object_pool *o)
 {
 	for (int i = 0; i < o->grafts_nr; i++) {

-- 
2.49.0.682.gc9b6a7b2b0.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH 8/9] object-store: remove global array of cached objects
  2025-04-08 10:24 [PATCH 0/9] Split up "object-file.c" Patrick Steinhardt
                   ` (6 preceding siblings ...)
  2025-04-08 10:24 ` [PATCH 7/9] object: split out functions relating to object store subsystem Patrick Steinhardt
@ 2025-04-08 10:24 ` Patrick Steinhardt
  2025-04-08 10:24 ` [PATCH 9/9] object-store: merge "object-store-ll.h" and "object-store.h" Patrick Steinhardt
                   ` (4 subsequent siblings)
  12 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-08 10:24 UTC (permalink / raw)
  To: git

Cached objects are virtual objects that can be set up without writing
anything into the object store directly. This mechanism for example
allows us to create fake commits in git-blame(1).

The cached objects are stored in a global variable. Refactor the code so
that we instead store the array as part of the raw object store. This is
another step into the direction of libifying our object database.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 blame.c           |  2 +-
 object-store-ll.h | 14 +++++++++++++-
 object-store.c    | 39 +++++++++++++++++++++++----------------
 3 files changed, 37 insertions(+), 18 deletions(-)

diff --git a/blame.c b/blame.c
index 703dab43e78..b7c5bd692e6 100644
--- a/blame.c
+++ b/blame.c
@@ -277,7 +277,7 @@ static struct commit *fake_working_tree_commit(struct repository *r,
 	convert_to_git(r->index, path, buf.buf, buf.len, &buf, 0);
 	origin->file.ptr = buf.buf;
 	origin->file.size = buf.len;
-	pretend_object_file(buf.buf, buf.len, OBJ_BLOB, &origin->blob_oid);
+	pretend_object_file(the_repository, buf.buf, buf.len, OBJ_BLOB, &origin->blob_oid);
 
 	/*
 	 * Read the current index, replace the path entry with
diff --git a/object-store-ll.h b/object-store-ll.h
index 8bb0f33f9a8..bb5e8798a1b 100644
--- a/object-store-ll.h
+++ b/object-store-ll.h
@@ -151,6 +151,8 @@ static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
 	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
 }
 
+struct cached_object_entry;
+
 struct raw_object_store {
 	/*
 	 * Set of all object directories; the main directory is first (and
@@ -203,6 +205,15 @@ struct raw_object_store {
 		unsigned flags;
 	} kept_pack_cache;
 
+	/*
+	 * This is meant to hold a *small* number of objects that you would
+	 * want repo_read_object_file() to be able to return, but yet you do not want
+	 * to write them into the object store (e.g. a browse-only
+	 * application).
+	 */
+	struct cached_object_entry *cached_objects;
+	size_t cached_object_nr, cached_object_alloc;
+
 	/*
 	 * A map of packfiles to packed_git structs for tracking which
 	 * packs have been loaded already.
@@ -272,7 +283,8 @@ void hash_object_file(const struct git_hash_algo *algo, const void *buf,
  * object in persistent storage before writing any other new objects
  * that reference it.
  */
-int pretend_object_file(void *, unsigned long, enum object_type,
+int pretend_object_file(struct repository *repo,
+			void *buf, unsigned long len, enum object_type type,
 			struct object_id *oid);
 
 struct object_info {
diff --git a/object-store.c b/object-store.c
index ea2d86c429b..17fa06a86fa 100644
--- a/object-store.c
+++ b/object-store.c
@@ -30,31 +30,31 @@
  * to write them into the object store (e.g. a browse-only
  * application).
  */
-static struct cached_object_entry {
+struct cached_object_entry {
 	struct object_id oid;
 	struct cached_object {
 		enum object_type type;
 		const void *buf;
 		unsigned long size;
 	} value;
-} *cached_objects;
-static int cached_object_nr, cached_object_alloc;
+};
 
-static const struct cached_object *find_cached_object(const struct object_id *oid)
+static const struct cached_object *find_cached_object(struct raw_object_store *object_store,
+						      const struct object_id *oid)
 {
 	static const struct cached_object empty_tree = {
 		.type = OBJ_TREE,
 		.buf = "",
 	};
-	int i;
-	const struct cached_object_entry *co = cached_objects;
+	const struct cached_object_entry *co = object_store->cached_objects;
 
-	for (i = 0; i < cached_object_nr; i++, co++) {
+	for (size_t i = 0; i < object_store->cached_object_nr; i++, co++)
 		if (oideq(&co->oid, oid))
 			return &co->value;
-	}
-	if (oideq(oid, the_hash_algo->empty_tree))
+
+	if (oid->algo && oideq(oid, hash_algos[oid->algo].empty_tree))
 		return &empty_tree;
+
 	return NULL;
 }
 
@@ -650,7 +650,7 @@ static int do_oid_object_info_extended(struct repository *r,
 	if (!oi)
 		oi = &blank_oi;
 
-	co = find_cached_object(real);
+	co = find_cached_object(r->objects, real);
 	if (co) {
 		if (oi->typep)
 			*(oi->typep) = co->type;
@@ -853,18 +853,21 @@ int oid_object_info(struct repository *r,
 	return type;
 }
 
-int pretend_object_file(void *buf, unsigned long len, enum object_type type,
+int pretend_object_file(struct repository *repo,
+			void *buf, unsigned long len, enum object_type type,
 			struct object_id *oid)
 {
 	struct cached_object_entry *co;
 	char *co_buf;
 
-	hash_object_file(the_hash_algo, buf, len, type, oid);
-	if (repo_has_object_file_with_flags(the_repository, oid, OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT) ||
-	    find_cached_object(oid))
+	hash_object_file(repo->hash_algo, buf, len, type, oid);
+	if (repo_has_object_file_with_flags(repo, oid, OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT) ||
+	    find_cached_object(repo->objects, oid))
 		return 0;
-	ALLOC_GROW(cached_objects, cached_object_nr + 1, cached_object_alloc);
-	co = &cached_objects[cached_object_nr++];
+
+	ALLOC_GROW(repo->objects->cached_objects,
+		   repo->objects->cached_object_nr + 1, repo->objects->cached_object_alloc);
+	co = &repo->objects->cached_objects[repo->objects->cached_object_nr++];
 	co->value.size = len;
 	co->value.type = type;
 	co_buf = xmalloc(len);
@@ -1021,6 +1024,10 @@ void raw_object_store_clear(struct raw_object_store *o)
 	o->odb_tail = NULL;
 	o->loaded_alternates = 0;
 
+	for (size_t i = 0; i < o->cached_object_nr; i++)
+		free((char *) o->cached_objects[i].value.buf);
+	FREE_AND_NULL(o->cached_objects);
+
 	INIT_LIST_HEAD(&o->packed_git_mru);
 	close_object_store(o);
 

-- 
2.49.0.682.gc9b6a7b2b0.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH 9/9] object-store: merge "object-store-ll.h" and "object-store.h"
  2025-04-08 10:24 [PATCH 0/9] Split up "object-file.c" Patrick Steinhardt
                   ` (7 preceding siblings ...)
  2025-04-08 10:24 ` [PATCH 8/9] object-store: remove global array of cached objects Patrick Steinhardt
@ 2025-04-08 10:24 ` Patrick Steinhardt
  2025-04-08 23:29 ` [PATCH 0/9] Split up "object-file.c" Junio C Hamano
                   ` (3 subsequent siblings)
  12 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-08 10:24 UTC (permalink / raw)
  To: git

The "object-store-ll.h" header has been introduced to keep transitive
header dependendcies and compile times at bay. Now that we have created
a new "object-store.c" file though we can easily move the last remaining
additional bit of "object-store.h", the `odb_path_map`, out of the
header.

Do so. As the "object-store.h" header is now equivalent to its low-level
alternative we drop the latter and inline it into the former.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 apply.c                       |   2 +-
 archive-tar.c                 |   2 +-
 archive-zip.c                 |   2 +-
 archive.c                     |   2 +-
 attr.c                        |   2 +-
 bisect.c                      |   2 +-
 blame.c                       |   2 +-
 builtin/backfill.c            |   2 +-
 builtin/blame.c               |   2 +-
 builtin/cat-file.c            |   2 +-
 builtin/checkout.c            |   2 +-
 builtin/clone.c               |   2 +-
 builtin/commit-graph.c        |   2 +-
 builtin/commit-tree.c         |   2 +-
 builtin/count-objects.c       |   2 +-
 builtin/describe.c            |   2 +-
 builtin/difftool.c            |   2 +-
 builtin/fast-export.c         |   2 +-
 builtin/fast-import.c         |   2 +-
 builtin/fetch.c               |   2 +-
 builtin/fsck.c                |   2 +-
 builtin/gc.c                  |   2 +-
 builtin/grep.c                |   2 +-
 builtin/hash-object.c         |   2 +-
 builtin/index-pack.c          |   2 +-
 builtin/log.c                 |   2 +-
 builtin/ls-tree.c             |   2 +-
 builtin/merge-tree.c          |   2 +-
 builtin/mktag.c               |   2 +-
 builtin/mktree.c              |   2 +-
 builtin/multi-pack-index.c    |   2 +-
 builtin/notes.c               |   2 +-
 builtin/pack-objects.c        |   2 +-
 builtin/pack-redundant.c      |   2 +-
 builtin/prune.c               |   2 +-
 builtin/receive-pack.c        |   2 +-
 builtin/remote.c              |   2 +-
 builtin/repack.c              |   2 +-
 builtin/replace.c             |   2 +-
 builtin/rev-list.c            |   2 +-
 builtin/show-ref.c            |   2 +-
 builtin/submodule--helper.c   |   2 +-
 builtin/tag.c                 |   2 +-
 builtin/unpack-file.c         |   2 +-
 builtin/unpack-objects.c      |   2 +-
 bulk-checkin.c                |   2 +-
 bundle-uri.c                  |   2 +-
 bundle.c                      |   2 +-
 cache-tree.c                  |   2 +-
 combine-diff.c                |   2 +-
 commit-graph.c                |   2 +-
 commit-graph.h                |   2 +-
 commit.c                      |   2 +-
 config.c                      |   2 +-
 connected.c                   |   2 +-
 convert.c                     |   2 +-
 diagnose.c                    |   2 +-
 diff.c                        |   2 +-
 diffcore-rename.c             |   2 +-
 dir.c                         |   2 +-
 entry.c                       |   2 +-
 fetch-pack.c                  |   2 +-
 fmt-merge-msg.c               |   2 +-
 fsck.c                        |   2 +-
 grep.c                        |   2 +-
 http-backend.c                |   2 +-
 http-push.c                   |   2 +-
 http-walker.c                 |   2 +-
 http.c                        |   2 +-
 list-objects-filter.c         |   2 +-
 list-objects.c                |   2 +-
 log-tree.c                    |   2 +-
 mailmap.c                     |   2 +-
 merge-blobs.c                 |   2 +-
 merge-recursive.c             |   2 +-
 notes-merge.c                 |   2 +-
 object-file.c                 |   1 +
 object-name.c                 |   2 +-
 object-store-ll.h             | 517 ------------------------------------------
 object-store.c                |   5 +
 object-store.h                | 516 ++++++++++++++++++++++++++++++++++++++++-
 oss-fuzz/fuzz-pack-idx.c      |   2 +-
 pack-bitmap-write.c           |   2 +-
 pack-bitmap.c                 |   2 +-
 pack-check.c                  |   2 +-
 pack-mtimes.c                 |   2 +-
 pack-objects.h                |   2 +-
 pack-revindex.c               |   2 +-
 packfile.c                    |   2 +-
 path.c                        |   2 +-
 promisor-remote.c             |   2 +-
 protocol-caps.c               |   2 +-
 prune-packed.c                |   2 +-
 reachable.c                   |   2 +-
 read-cache.c                  |   2 +-
 ref-filter.c                  |   2 +-
 reflog.c                      |   2 +-
 refs.c                        |   2 +-
 remote.c                      |   2 +-
 replace-object.c              |   2 +-
 replace-object.h              |   2 +-
 repository.c                  |   2 +-
 rerere.c                      |   2 +-
 revision.c                    |   2 +-
 send-pack.c                   |   2 +-
 sequencer.c                   |   2 +-
 server-info.c                 |   2 +-
 shallow.c                     |   2 +-
 streaming.c                   |   2 +-
 submodule-config.c            |   2 +-
 submodule.c                   |   2 +-
 t/helper/test-pack-mtimes.c   |   2 +-
 t/helper/test-partial-clone.c |   2 +-
 t/helper/test-read-graph.c    |   2 +-
 t/helper/test-read-midx.c     |   2 +-
 t/helper/test-ref-store.c     |   2 +-
 tag.c                         |   2 +-
 tmp-objdir.c                  |   2 +-
 tree-walk.c                   |   2 +-
 tree.c                        |   2 +-
 unpack-trees.c                |   2 +-
 upload-pack.c                 |   2 +-
 walker.c                      |   2 +-
 xdiff-interface.c             |   2 +-
 124 files changed, 637 insertions(+), 642 deletions(-)

diff --git a/apply.c b/apply.c
index f274a379487..2b6f4d0af87 100644
--- a/apply.c
+++ b/apply.c
@@ -14,7 +14,7 @@
 #include "abspath.h"
 #include "base85.h"
 #include "config.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "delta.h"
 #include "diff.h"
 #include "dir.h"
diff --git a/archive-tar.c b/archive-tar.c
index 0edf13fba75..282b48196f9 100644
--- a/archive-tar.c
+++ b/archive-tar.c
@@ -11,7 +11,7 @@
 #include "hex.h"
 #include "tar.h"
 #include "archive.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "strbuf.h"
 #include "streaming.h"
 #include "run-command.h"
diff --git a/archive-zip.c b/archive-zip.c
index 9f32730181b..405da6f3d83 100644
--- a/archive-zip.c
+++ b/archive-zip.c
@@ -12,7 +12,7 @@
 #include "hex.h"
 #include "streaming.h"
 #include "utf8.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "strbuf.h"
 #include "userdiff.h"
 #include "write-or-die.h"
diff --git a/archive.c b/archive.c
index c95e3981524..014c312178c 100644
--- a/archive.c
+++ b/archive.c
@@ -14,7 +14,7 @@
 #include "pretty.h"
 #include "setup.h"
 #include "refs.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tree.h"
 #include "tree-walk.h"
diff --git a/attr.c b/attr.c
index 0bd2750528f..86b6109fc4e 100644
--- a/attr.c
+++ b/attr.c
@@ -22,7 +22,7 @@
 #include "read-cache-ll.h"
 #include "refs.h"
 #include "revision.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "setup.h"
 #include "thread-utils.h"
 #include "tree-walk.h"
diff --git a/bisect.c b/bisect.c
index 269a98bf978..a327468c75b 100644
--- a/bisect.c
+++ b/bisect.c
@@ -20,7 +20,7 @@
 #include "commit-slab.h"
 #include "commit-reach.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "dir.h"
 
diff --git a/blame.c b/blame.c
index b7c5bd692e6..57daa45e899 100644
--- a/blame.c
+++ b/blame.c
@@ -3,7 +3,7 @@
 
 #include "git-compat-util.h"
 #include "refs.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "cache-tree.h"
 #include "mergesort.h"
 #include "commit.h"
diff --git a/builtin/backfill.c b/builtin/backfill.c
index 33e1ea2f84f..aaa104bc91d 100644
--- a/builtin/backfill.c
+++ b/builtin/backfill.c
@@ -13,7 +13,7 @@
 #include "tree.h"
 #include "tree-walk.h"
 #include "object.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "oidset.h"
 #include "promisor-remote.h"
diff --git a/builtin/blame.c b/builtin/blame.c
index c470654c7ec..4e156bfd19d 100644
--- a/builtin/blame.c
+++ b/builtin/blame.c
@@ -28,7 +28,7 @@
 #include "line-log.h"
 #include "progress.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pager.h"
 #include "blame.h"
 #include "refs.h"
diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index b13561cf73b..c870fde260a 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -22,7 +22,7 @@
 #include "packfile.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "replace-object.h"
 #include "promisor-remote.h"
 #include "mailmap.h"
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 3e68623838a..8136962e2b3 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -20,7 +20,7 @@
 #include "merge-recursive.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "parse-options.h"
 #include "path.h"
 #include "preload-index.h"
diff --git a/builtin/clone.c b/builtin/clone.c
index 2993acb630e..d0423a2198b 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -25,7 +25,7 @@
 #include "refs.h"
 #include "refspec.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "tree.h"
 #include "tree-walk.h"
 #include "unpack-trees.h"
diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c
index 8ca75262c59..be06d0a811b 100644
--- a/builtin/commit-graph.c
+++ b/builtin/commit-graph.c
@@ -6,7 +6,7 @@
 #include "hex.h"
 #include "parse-options.h"
 #include "commit-graph.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "progress.h"
 #include "replace-object.h"
 #include "strbuf.h"
diff --git a/builtin/commit-tree.c b/builtin/commit-tree.c
index 38457600a4e..6f9975e7a88 100644
--- a/builtin/commit-tree.c
+++ b/builtin/commit-tree.c
@@ -9,7 +9,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 #include "commit.h"
 #include "parse-options.h"
diff --git a/builtin/count-objects.c b/builtin/count-objects.c
index 1e89148ed74..0bb5360b2f2 100644
--- a/builtin/count-objects.c
+++ b/builtin/count-objects.c
@@ -12,7 +12,7 @@
 #include "parse-options.h"
 #include "quote.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 static unsigned long garbage;
 static off_t size_garbage;
diff --git a/builtin/describe.c b/builtin/describe.c
index 23df333fd04..0f87fbceef3 100644
--- a/builtin/describe.c
+++ b/builtin/describe.c
@@ -19,7 +19,7 @@
 #include "setup.h"
 #include "strvec.h"
 #include "run-command.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "list-objects.h"
 #include "commit-slab.h"
 #include "wildmatch.h"
diff --git a/builtin/difftool.c b/builtin/difftool.c
index 45d6ea8a801..e6e45436531 100644
--- a/builtin/difftool.c
+++ b/builtin/difftool.c
@@ -29,7 +29,7 @@
 #include "strbuf.h"
 #include "lockfile.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "dir.h"
 #include "entry.h"
 #include "setup.h"
diff --git a/builtin/fast-export.c b/builtin/fast-export.c
index 170126d41ac..afacd228b5d 100644
--- a/builtin/fast-export.c
+++ b/builtin/fast-export.c
@@ -14,7 +14,7 @@
 #include "refs.h"
 #include "refspec.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "object.h"
 #include "tag.h"
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index 63880b595cc..0357211ea6c 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -24,7 +24,7 @@
 #include "packfile.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "mem-pool.h"
 #include "commit-reach.h"
 #include "khash.h"
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 02af5054690..b52a32a5e0d 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -14,7 +14,7 @@
 #include "refs.h"
 #include "refspec.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oidset.h"
 #include "oid-array.h"
 #include "commit.h"
diff --git a/builtin/fsck.c b/builtin/fsck.c
index 32d40d8f9fc..47055ebaf9a 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -18,7 +18,7 @@
 #include "packfile.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "read-cache-ll.h"
 #include "replace-object.h"
diff --git a/builtin/gc.c b/builtin/gc.c
index b069629676c..def922e7145 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -29,7 +29,7 @@
 #include "commit.h"
 #include "commit-graph.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pack.h"
 #include "pack-objects.h"
 #include "path.h"
diff --git a/builtin/grep.c b/builtin/grep.c
index 283d64cab80..bcfbe5be5ba 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -26,7 +26,7 @@
 #include "submodule-config.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 #include "pager.h"
 #include "path.h"
diff --git a/builtin/hash-object.c b/builtin/hash-object.c
index 7aa889b1b4a..e4eddd63bf4 100644
--- a/builtin/hash-object.c
+++ b/builtin/hash-object.c
@@ -11,7 +11,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "blob.h"
 #include "quote.h"
 #include "parse-options.h"
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index de127c0ff13..60a8ee05dbc 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -21,7 +21,7 @@
 #include "packfile.h"
 #include "pack-revindex.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "oidset.h"
 #include "path.h"
diff --git a/builtin/log.c b/builtin/log.c
index 06ffaa93e86..72ac45e9433 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -16,7 +16,7 @@
 #include "hex.h"
 #include "refs.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pager.h"
 #include "color.h"
 #include "commit.h"
diff --git a/builtin/ls-tree.c b/builtin/ls-tree.c
index 8542b5d53e4..8aafc30ca48 100644
--- a/builtin/ls-tree.c
+++ b/builtin/ls-tree.c
@@ -10,7 +10,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "tree.h"
 #include "path.h"
 #include "quote.h"
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 3ec7127b3a6..4aafa73c615 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -10,7 +10,7 @@
 #include "commit-reach.h"
 #include "merge-ort.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "parse-options.h"
 #include "blob.h"
 #include "merge-blobs.h"
diff --git a/builtin/mktag.c b/builtin/mktag.c
index 6e188dce50a..7ac11c46d53 100644
--- a/builtin/mktag.c
+++ b/builtin/mktag.c
@@ -6,7 +6,7 @@
 #include "strbuf.h"
 #include "replace-object.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "fsck.h"
 #include "config.h"
 
diff --git a/builtin/mktree.c b/builtin/mktree.c
index 0644f951161..7ffe6eefd8a 100644
--- a/builtin/mktree.c
+++ b/builtin/mktree.c
@@ -12,7 +12,7 @@
 #include "tree.h"
 #include "parse-options.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 static struct treeent {
 	unsigned mode;
diff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c
index 2a938466f53..d98410ca6c6 100644
--- a/builtin/multi-pack-index.c
+++ b/builtin/multi-pack-index.c
@@ -7,7 +7,7 @@
 #include "midx.h"
 #include "strbuf.h"
 #include "trace2.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "replace-object.h"
 #include "repository.h"
 
diff --git a/builtin/notes.c b/builtin/notes.c
index 0dbc233752d..a3f433ca4c0 100644
--- a/builtin/notes.c
+++ b/builtin/notes.c
@@ -16,7 +16,7 @@
 #include "notes.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 
 #include "pretty.h"
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 163aab547fe..488c80f2cf3 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -32,7 +32,7 @@
 #include "list.h"
 #include "packfile.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "replace-object.h"
 #include "dir.h"
 #include "midx.h"
diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c
index 3febe732f8e..5d1fc781761 100644
--- a/builtin/pack-redundant.c
+++ b/builtin/pack-redundant.c
@@ -13,7 +13,7 @@
 #include "hex.h"
 
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "strbuf.h"
 
 #define BLKSIZE 512
diff --git a/builtin/prune.c b/builtin/prune.c
index 8f52da8bd66..e930caa0c0a 100644
--- a/builtin/prune.c
+++ b/builtin/prune.c
@@ -17,7 +17,7 @@
 #include "replace-object.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "shallow.h"
 
 static const char * const prune_usage[] = {
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index ee51bd76f60..be314879e82 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -33,7 +33,7 @@
 #include "packfile.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "protocol.h"
 #include "commit-reach.h"
diff --git a/builtin/remote.c b/builtin/remote.c
index 1b7aad88380..59481b3a82c 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -14,7 +14,7 @@
 #include "rebase.h"
 #include "refs.h"
 #include "refspec.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "strvec.h"
 #include "commit-reach.h"
 #include "progress.h"
diff --git a/builtin/repack.c b/builtin/repack.c
index f3330ade7b8..1fd2874324a 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -17,7 +17,7 @@
 #include "midx.h"
 #include "packfile.h"
 #include "prune-packed.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "promisor-remote.h"
 #include "shallow.h"
 #include "pack.h"
diff --git a/builtin/replace.c b/builtin/replace.c
index ce2948a9b2a..4204ca3605b 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -19,7 +19,7 @@
 #include "run-command.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "read-cache.h"
 #include "replace-object.h"
 #include "tag.h"
diff --git a/builtin/rev-list.c b/builtin/rev-list.c
index bb26bee0d45..0170d79b62f 100644
--- a/builtin/rev-list.c
+++ b/builtin/rev-list.c
@@ -14,7 +14,7 @@
 #include "object.h"
 #include "object-name.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pack-bitmap.h"
 #include "log-tree.h"
 #include "graph.h"
diff --git a/builtin/show-ref.c b/builtin/show-ref.c
index 285cd3e4338..f81209f23c3 100644
--- a/builtin/show-ref.c
+++ b/builtin/show-ref.c
@@ -5,7 +5,7 @@
 #include "hex.h"
 #include "refs/refs-internal.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "object.h"
 #include "string-list.h"
 #include "parse-options.h"
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 570226ea166..a8d1200b7cb 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -28,7 +28,7 @@
 #include "diff.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "advice.h"
 #include "branch.h"
 #include "list-objects-filter-options.h"
diff --git a/builtin/tag.c b/builtin/tag.c
index 536a01ff3ae..e6b372cebf5 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -19,7 +19,7 @@
 #include "refs.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "tag.h"
 #include "parse-options.h"
diff --git a/builtin/unpack-file.c b/builtin/unpack-file.c
index b19e5cabd03..e33acfc4ee4 100644
--- a/builtin/unpack-file.c
+++ b/builtin/unpack-file.c
@@ -4,7 +4,7 @@
 #include "hex.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 static char *create_temp_file(struct object_id *oid)
 {
diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c
index 4078eab9252..661be789f13 100644
--- a/builtin/unpack-objects.c
+++ b/builtin/unpack-objects.c
@@ -9,7 +9,7 @@
 #include "git-zlib.h"
 #include "hex.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "object.h"
 #include "delta.h"
 #include "pack.h"
diff --git a/bulk-checkin.c b/bulk-checkin.c
index 0b9bad92868..b38937d739b 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -19,7 +19,7 @@
 #include "tmp-objdir.h"
 #include "packfile.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 static int odb_transaction_nesting;
 
diff --git a/bundle-uri.c b/bundle-uri.c
index 744257c49c1..96d2ba726d9 100644
--- a/bundle-uri.c
+++ b/bundle-uri.c
@@ -14,7 +14,7 @@
 #include "fetch-pack.h"
 #include "remote.h"
 #include "trace2.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 static struct {
 	enum bundle_list_heuristic heuristic;
diff --git a/bundle.c b/bundle.c
index d7ad6908433..d661c4ec214 100644
--- a/bundle.c
+++ b/bundle.c
@@ -7,7 +7,7 @@
 #include "environment.h"
 #include "gettext.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "repository.h"
 #include "object.h"
 #include "commit.h"
diff --git a/cache-tree.c b/cache-tree.c
index 4c8167ea927..c0e1e9ee1d4 100644
--- a/cache-tree.c
+++ b/cache-tree.c
@@ -10,7 +10,7 @@
 #include "cache-tree.h"
 #include "bulk-checkin.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "read-cache-ll.h"
 #include "replace-object.h"
 #include "repository.h"
diff --git a/combine-diff.c b/combine-diff.c
index 553bf59fed6..dfae9f7995d 100644
--- a/combine-diff.c
+++ b/combine-diff.c
@@ -2,7 +2,7 @@
 #define DISABLE_SIGN_COMPARE_WARNINGS
 
 #include "git-compat-util.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "convert.h"
 #include "diff.h"
diff --git a/commit-graph.c b/commit-graph.c
index 8060c358b84..a36c2f1e303 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -14,7 +14,7 @@
 #include "refs.h"
 #include "hash-lookup.h"
 #include "commit-graph.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "path.h"
 #include "alloc.h"
diff --git a/commit-graph.h b/commit-graph.h
index 67819401954..13f662827d4 100644
--- a/commit-graph.h
+++ b/commit-graph.h
@@ -1,7 +1,7 @@
 #ifndef COMMIT_GRAPH_H
 #define COMMIT_GRAPH_H
 
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oidset.h"
 
 #define GIT_TEST_COMMIT_GRAPH "GIT_TEST_COMMIT_GRAPH"
diff --git a/commit.c b/commit.c
index fbf4f8e87fd..00842678bd2 100644
--- a/commit.c
+++ b/commit.c
@@ -9,7 +9,7 @@
 #include "hex.h"
 #include "repository.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "utf8.h"
 #include "diff.h"
 #include "revision.h"
diff --git a/config.c b/config.c
index accb47e2d18..b18b5617fcd 100644
--- a/config.c
+++ b/config.c
@@ -31,7 +31,7 @@
 #include "hashmap.h"
 #include "string-list.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pager.h"
 #include "path.h"
 #include "utf8.h"
diff --git a/connected.c b/connected.c
index 3099da84f33..4415388beba 100644
--- a/connected.c
+++ b/connected.c
@@ -3,7 +3,7 @@
 #include "git-compat-util.h"
 #include "gettext.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "run-command.h"
 #include "sigchain.h"
 #include "connected.h"
diff --git a/convert.c b/convert.c
index 9cc0ca20ca0..8783e17941f 100644
--- a/convert.c
+++ b/convert.c
@@ -8,7 +8,7 @@
 #include "copy.h"
 #include "gettext.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "attr.h"
 #include "run-command.h"
 #include "quote.h"
diff --git a/diagnose.c b/diagnose.c
index bd485effea2..b1be74be983 100644
--- a/diagnose.c
+++ b/diagnose.c
@@ -7,7 +7,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "strvec.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 #include "parse-options.h"
 #include "repository.h"
diff --git a/diff.c b/diff.c
index c361199ab1b..9f2042fd1a5 100644
--- a/diff.c
+++ b/diff.c
@@ -23,7 +23,7 @@
 #include "color.h"
 #include "run-command.h"
 #include "utf8.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "userdiff.h"
 #include "submodule.h"
 #include "hashmap.h"
diff --git a/diffcore-rename.c b/diffcore-rename.c
index 5002e896aad..787a2cef5f4 100644
--- a/diffcore-rename.c
+++ b/diffcore-rename.c
@@ -8,7 +8,7 @@
 #include "git-compat-util.h"
 #include "diff.h"
 #include "diffcore.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "hashmap.h"
 #include "mem-pool.h"
 #include "oid-array.h"
diff --git a/dir.c b/dir.c
index 16ae3b5169d..1f8b6e2aa67 100644
--- a/dir.c
+++ b/dir.c
@@ -17,7 +17,7 @@
 #include "environment.h"
 #include "gettext.h"
 #include "name-hash.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "refs.h"
 #include "repository.h"
diff --git a/entry.c b/entry.c
index 81b321e53d1..f36ec5ad242 100644
--- a/entry.c
+++ b/entry.c
@@ -1,7 +1,7 @@
 #define USE_THE_REPOSITORY_VARIABLE
 
 #include "git-compat-util.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "dir.h"
 #include "environment.h"
 #include "gettext.h"
diff --git a/fetch-pack.c b/fetch-pack.c
index 1ed5e11dd56..210dc30d50f 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -24,7 +24,7 @@
 #include "oid-array.h"
 #include "oidset.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "connected.h"
 #include "fetch-negotiator.h"
diff --git a/fmt-merge-msg.c b/fmt-merge-msg.c
index 5b63c3b088a..501b5acdd44 100644
--- a/fmt-merge-msg.c
+++ b/fmt-merge-msg.c
@@ -6,7 +6,7 @@
 #include "environment.h"
 #include "refs.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "diff.h"
 #include "diff-merges.h"
 #include "hex.h"
diff --git a/fsck.c b/fsck.c
index 9fc4c25ffd5..8dc8472ceb3 100644
--- a/fsck.c
+++ b/fsck.c
@@ -4,7 +4,7 @@
 #include "date.h"
 #include "dir.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "repository.h"
 #include "object.h"
diff --git a/grep.c b/grep.c
index 9284b5741f5..f8d535182c3 100644
--- a/grep.c
+++ b/grep.c
@@ -5,7 +5,7 @@
 #include "gettext.h"
 #include "grep.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pretty.h"
 #include "userdiff.h"
 #include "xdiff-interface.h"
diff --git a/http-backend.c b/http-backend.c
index 50b2858fad6..0c575aa88aa 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -18,7 +18,7 @@
 #include "url.h"
 #include "strvec.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "protocol.h"
 #include "date.h"
 #include "write-or-die.h"
diff --git a/http-push.c b/http-push.c
index 806eb67cf1b..32e37565f4e 100644
--- a/http-push.c
+++ b/http-push.c
@@ -20,7 +20,7 @@
 #include "url.h"
 #include "packfile.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit-reach.h"
 
 #ifdef EXPAT_NEEDS_XMLPARSE_H
diff --git a/http-walker.c b/http-walker.c
index 7918ddc0968..882cae19c24 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -9,7 +9,7 @@
 #include "list.h"
 #include "transport.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 struct alt_base {
 	char *base;
diff --git a/http.c b/http.c
index 0c9a872809f..bc18ff83c4b 100644
--- a/http.c
+++ b/http.c
@@ -19,7 +19,7 @@
 #include "packfile.h"
 #include "string-list.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "tempfile.h"
 
 static struct trace_key trace_curl = TRACE_KEY_INIT(CURL);
diff --git a/list-objects-filter.c b/list-objects-filter.c
index dc598a081bb..7765761b3c6 100644
--- a/list-objects-filter.c
+++ b/list-objects-filter.c
@@ -12,7 +12,7 @@
 #include "oidmap.h"
 #include "oidset.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 /* Remember to update object flag allocation in object.h */
 /*
diff --git a/list-objects.c b/list-objects.c
index 943e62e868f..1e5512e1318 100644
--- a/list-objects.c
+++ b/list-objects.c
@@ -14,7 +14,7 @@
 #include "list-objects-filter.h"
 #include "list-objects-filter-options.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "trace.h"
 #include "environment.h"
 
diff --git a/log-tree.c b/log-tree.c
index 5dd1b63076f..a4d4ab59ca0 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -9,7 +9,7 @@
 #include "environment.h"
 #include "hex.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "repository.h"
 #include "tmp-objdir.h"
 #include "commit.h"
diff --git a/mailmap.c b/mailmap.c
index f35d20ed7fd..9e2642a043b 100644
--- a/mailmap.c
+++ b/mailmap.c
@@ -6,7 +6,7 @@
 #include "string-list.h"
 #include "mailmap.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "setup.h"
 
 char *git_mailmap_file;
diff --git a/merge-blobs.c b/merge-blobs.c
index 0ad0390fea5..53f36dbc175 100644
--- a/merge-blobs.c
+++ b/merge-blobs.c
@@ -4,7 +4,7 @@
 #include "merge-ll.h"
 #include "blob.h"
 #include "merge-blobs.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 static int fill_mmfile_blob(mmfile_t *f, struct blob *obj)
 {
diff --git a/merge-recursive.c b/merge-recursive.c
index 9aedffc546b..981e57698ed 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -27,7 +27,7 @@
 #include "name-hash.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "repository.h"
 #include "revision.h"
diff --git a/notes-merge.c b/notes-merge.c
index 13750490eff..96e59322a8b 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -8,7 +8,7 @@
 #include "refs.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "read-cache.h"
 #include "repository.h"
diff --git a/object-file.c b/object-file.c
index 235d29858ff..4abca6cb613 100644
--- a/object-file.c
+++ b/object-file.c
@@ -12,6 +12,7 @@
 
 #include "git-compat-util.h"
 #include "bulk-checkin.h"
+#include "dir.h"
 #include "environment.h"
 #include "gettext.h"
 #include "hex.h"
diff --git a/object-name.c b/object-name.c
index 91f731373a1..2c751a5352a 100644
--- a/object-name.c
+++ b/object-name.c
@@ -19,7 +19,7 @@
 #include "oidtree.h"
 #include "packfile.h"
 #include "pretty.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "read-cache-ll.h"
 #include "repo-settings.h"
 #include "repository.h"
diff --git a/object-store-ll.h b/object-store-ll.h
deleted file mode 100644
index bb5e8798a1b..00000000000
--- a/object-store-ll.h
+++ /dev/null
@@ -1,517 +0,0 @@
-#ifndef OBJECT_STORE_LL_H
-#define OBJECT_STORE_LL_H
-
-#include "hashmap.h"
-#include "object.h"
-#include "list.h"
-#include "thread-utils.h"
-#include "oidset.h"
-
-struct oidmap;
-struct oidtree;
-struct strbuf;
-struct repository;
-
-struct object_directory {
-	struct object_directory *next;
-
-	/*
-	 * Used to store the results of readdir(3) calls when we are OK
-	 * sacrificing accuracy due to races for speed. That includes
-	 * object existence with OBJECT_INFO_QUICK, as well as
-	 * our search for unique abbreviated hashes. Don't use it for tasks
-	 * requiring greater accuracy!
-	 *
-	 * Be sure to call odb_load_loose_cache() before using.
-	 */
-	uint32_t loose_objects_subdir_seen[8]; /* 256 bits */
-	struct oidtree *loose_objects_cache;
-
-	/* Map between object IDs for loose objects. */
-	struct loose_object_map *loose_map;
-
-	/*
-	 * This is a temporary object store created by the tmp_objdir
-	 * facility. Disable ref updates since the objects in the store
-	 * might be discarded on rollback.
-	 */
-	int disable_ref_updates;
-
-	/*
-	 * This object store is ephemeral, so there is no need to fsync.
-	 */
-	int will_destroy;
-
-	/*
-	 * Path to the alternative object store. If this is a relative path,
-	 * it is relative to the current working directory.
-	 */
-	char *path;
-};
-
-void prepare_alt_odb(struct repository *r);
-int has_alt_odb(struct repository *r);
-char *compute_alternate_path(const char *path, struct strbuf *err);
-struct object_directory *find_odb(struct repository *r, const char *obj_dir);
-typedef int alt_odb_fn(struct object_directory *, void *);
-int foreach_alt_odb(alt_odb_fn, void*);
-typedef void alternate_ref_fn(const struct object_id *oid, void *);
-void for_each_alternate_ref(alternate_ref_fn, void *);
-
-/*
- * Add the directory to the on-disk alternates file; the new entry will also
- * take effect in the current process.
- */
-void add_to_alternates_file(const char *dir);
-
-/*
- * Add the directory to the in-memory list of alternates (along with any
- * recursive alternates it points to), but do not modify the on-disk alternates
- * file.
- */
-void add_to_alternates_memory(const char *dir);
-
-/*
- * Replace the current writable object directory with the specified temporary
- * object directory; returns the former primary object directory.
- */
-struct object_directory *set_temporary_primary_odb(const char *dir, int will_destroy);
-
-/*
- * Restore a previous ODB replaced by set_temporary_main_odb.
- */
-void restore_primary_odb(struct object_directory *restore_odb, const char *old_path);
-
-/*
- * Populate and return the loose object cache array corresponding to the
- * given object ID.
- */
-struct oidtree *odb_loose_cache(struct object_directory *odb,
-				  const struct object_id *oid);
-
-/* Empty the loose object cache for the specified object directory. */
-void odb_clear_loose_cache(struct object_directory *odb);
-
-struct packed_git {
-	struct hashmap_entry packmap_ent;
-	struct packed_git *next;
-	struct list_head mru;
-	struct pack_window *windows;
-	off_t pack_size;
-	const void *index_data;
-	size_t index_size;
-	uint32_t num_objects;
-	size_t crc_offset;
-	struct oidset bad_objects;
-	int index_version;
-	time_t mtime;
-	int pack_fd;
-	int index;              /* for builtin/pack-objects.c */
-	unsigned pack_local:1,
-		 pack_keep:1,
-		 pack_keep_in_core:1,
-		 freshened:1,
-		 do_not_close:1,
-		 pack_promisor:1,
-		 multi_pack_index:1,
-		 is_cruft:1;
-	unsigned char hash[GIT_MAX_RAWSZ];
-	struct revindex_entry *revindex;
-	const uint32_t *revindex_data;
-	const uint32_t *revindex_map;
-	size_t revindex_size;
-	/*
-	 * mtimes_map points at the beginning of the memory mapped region of
-	 * this pack's corresponding .mtimes file, and mtimes_size is the size
-	 * of that .mtimes file
-	 */
-	const uint32_t *mtimes_map;
-	size_t mtimes_size;
-
-	/* repo denotes the repository this packfile belongs to */
-	struct repository *repo;
-
-	/* something like ".git/objects/pack/xxxxx.pack" */
-	char pack_name[FLEX_ARRAY]; /* more */
-};
-
-struct multi_pack_index;
-
-static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
-				     const struct hashmap_entry *entry,
-				     const struct hashmap_entry *entry2,
-				     const void *keydata)
-{
-	const char *key = keydata;
-	const struct packed_git *pg1, *pg2;
-
-	pg1 = container_of(entry, const struct packed_git, packmap_ent);
-	pg2 = container_of(entry2, const struct packed_git, packmap_ent);
-
-	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
-}
-
-struct cached_object_entry;
-
-struct raw_object_store {
-	/*
-	 * Set of all object directories; the main directory is first (and
-	 * cannot be NULL after initialization). Subsequent directories are
-	 * alternates.
-	 */
-	struct object_directory *odb;
-	struct object_directory **odb_tail;
-	struct kh_odb_path_map *odb_by_path;
-
-	int loaded_alternates;
-
-	/*
-	 * A list of alternate object directories loaded from the environment;
-	 * this should not generally need to be accessed directly, but will
-	 * populate the "odb" list when prepare_alt_odb() is run.
-	 */
-	char *alternate_db;
-
-	/*
-	 * Objects that should be substituted by other objects
-	 * (see git-replace(1)).
-	 */
-	struct oidmap *replace_map;
-	unsigned replace_map_initialized : 1;
-	pthread_mutex_t replace_mutex; /* protect object replace functions */
-
-	struct commit_graph *commit_graph;
-	unsigned commit_graph_attempted : 1; /* if loading has been attempted */
-
-	/*
-	 * private data
-	 *
-	 * should only be accessed directly by packfile.c and midx.c
-	 */
-	struct multi_pack_index *multi_pack_index;
-
-	/*
-	 * private data
-	 *
-	 * should only be accessed directly by packfile.c
-	 */
-
-	struct packed_git *packed_git;
-	/* A most-recently-used ordered version of the packed_git list. */
-	struct list_head packed_git_mru;
-
-	struct {
-		struct packed_git **packs;
-		unsigned flags;
-	} kept_pack_cache;
-
-	/*
-	 * This is meant to hold a *small* number of objects that you would
-	 * want repo_read_object_file() to be able to return, but yet you do not want
-	 * to write them into the object store (e.g. a browse-only
-	 * application).
-	 */
-	struct cached_object_entry *cached_objects;
-	size_t cached_object_nr, cached_object_alloc;
-
-	/*
-	 * A map of packfiles to packed_git structs for tracking which
-	 * packs have been loaded already.
-	 */
-	struct hashmap pack_map;
-
-	/*
-	 * A fast, rough count of the number of objects in the repository.
-	 * These two fields are not meant for direct access. Use
-	 * repo_approximate_object_count() instead.
-	 */
-	unsigned long approximate_object_count;
-	unsigned approximate_object_count_valid : 1;
-
-	/*
-	 * Whether packed_git has already been populated with this repository's
-	 * packs.
-	 */
-	unsigned packed_git_initialized : 1;
-};
-
-struct raw_object_store *raw_object_store_new(void);
-void raw_object_store_clear(struct raw_object_store *o);
-
-/*
- * Create a temporary file rooted in the object database directory, or
- * die on failure. The filename is taken from "pattern", which should have the
- * usual "XXXXXX" trailer, and the resulting filename is written into the
- * "template" buffer. Returns the open descriptor.
- */
-int odb_mkstemp(struct strbuf *temp_filename, const char *pattern);
-
-/*
- * Create a pack .keep file named "name" (which should generally be the output
- * of odb_pack_name). Returns a file descriptor opened for writing, or -1 on
- * error.
- */
-int odb_pack_keep(const char *name);
-
-/*
- * Put in `buf` the name of the file in the local object database that
- * would be used to store a loose object with the specified oid.
- */
-const char *loose_object_path(struct repository *r, struct strbuf *buf,
-			      const struct object_id *oid);
-
-void *map_loose_object(struct repository *r, const struct object_id *oid,
-		       unsigned long *size);
-
-void *repo_read_object_file(struct repository *r,
-			    const struct object_id *oid,
-			    enum object_type *type,
-			    unsigned long *size);
-
-/* Read and unpack an object file into memory, write memory to an object file */
-int oid_object_info(struct repository *r, const struct object_id *, unsigned long *);
-
-void hash_object_file(const struct git_hash_algo *algo, const void *buf,
-		      unsigned long len, enum object_type type,
-		      struct object_id *oid);
-
-/*
- * Add an object file to the in-memory object store, without writing it
- * to disk.
- *
- * Callers are responsible for calling write_object_file to record the
- * object in persistent storage before writing any other new objects
- * that reference it.
- */
-int pretend_object_file(struct repository *repo,
-			void *buf, unsigned long len, enum object_type type,
-			struct object_id *oid);
-
-struct object_info {
-	/* Request */
-	enum object_type *typep;
-	unsigned long *sizep;
-	off_t *disk_sizep;
-	struct object_id *delta_base_oid;
-	struct strbuf *type_name;
-	void **contentp;
-
-	/* Response */
-	enum {
-		OI_CACHED,
-		OI_LOOSE,
-		OI_PACKED,
-		OI_DBCACHED
-	} whence;
-	union {
-		/*
-		 * struct {
-		 * 	... Nothing to expose in this case
-		 * } cached;
-		 * struct {
-		 * 	... Nothing to expose in this case
-		 * } loose;
-		 */
-		struct {
-			struct packed_git *pack;
-			off_t offset;
-			unsigned int is_delta;
-		} packed;
-	} u;
-};
-
-/*
- * Initializer for a "struct object_info" that wants no items. You may
- * also memset() the memory to all-zeroes.
- */
-#define OBJECT_INFO_INIT { 0 }
-
-/* Invoke lookup_replace_object() on the given hash */
-#define OBJECT_INFO_LOOKUP_REPLACE 1
-/* Allow reading from a loose object file of unknown/bogus type */
-#define OBJECT_INFO_ALLOW_UNKNOWN_TYPE 2
-/* Do not retry packed storage after checking packed and loose storage */
-#define OBJECT_INFO_QUICK 8
-/*
- * Do not attempt to fetch the object if missing (even if fetch_is_missing is
- * nonzero).
- */
-#define OBJECT_INFO_SKIP_FETCH_OBJECT 16
-/*
- * This is meant for bulk prefetching of missing blobs in a partial
- * clone. Implies OBJECT_INFO_SKIP_FETCH_OBJECT and OBJECT_INFO_QUICK
- */
-#define OBJECT_INFO_FOR_PREFETCH (OBJECT_INFO_SKIP_FETCH_OBJECT | OBJECT_INFO_QUICK)
-
-/* Die if object corruption (not just an object being missing) was detected. */
-#define OBJECT_INFO_DIE_IF_CORRUPT 32
-
-int oid_object_info_extended(struct repository *r,
-			     const struct object_id *,
-			     struct object_info *, unsigned flags);
-
-/* Retry packed storage after checking packed and loose storage */
-#define HAS_OBJECT_RECHECK_PACKED 1
-
-/*
- * Returns 1 if the object exists. This function will not lazily fetch objects
- * in a partial clone.
- */
-int has_object(struct repository *r, const struct object_id *oid,
-	       unsigned flags);
-
-/*
- * These macros and functions are deprecated. If checking existence for an
- * object that is likely to be missing and/or whose absence is relatively
- * inconsequential (or is consequential but the caller is prepared to handle
- * it), use has_object(), which has better defaults (no lazy fetch in a partial
- * clone and no rechecking of packed storage). In the unlikely event that a
- * caller needs to assert existence of an object that it fully expects to
- * exist, and wants to trigger a lazy fetch in a partial clone, use
- * oid_object_info_extended() with a NULL struct object_info.
- *
- * These functions can be removed once all callers have migrated to
- * has_object() and/or oid_object_info_extended().
- */
-int repo_has_object_file(struct repository *r, const struct object_id *oid);
-int repo_has_object_file_with_flags(struct repository *r,
-				    const struct object_id *oid, int flags);
-
-void assert_oid_type(const struct object_id *oid, enum object_type expect);
-
-/*
- * Enabling the object read lock allows multiple threads to safely call the
- * following functions in parallel: repo_read_object_file(),
- * read_object_with_reference(), oid_object_info() and oid_object_info_extended().
- *
- * obj_read_lock() and obj_read_unlock() may also be used to protect other
- * section which cannot execute in parallel with object reading. Since the used
- * lock is a recursive mutex, these sections can even contain calls to object
- * reading functions. However, beware that in these cases zlib inflation won't
- * be performed in parallel, losing performance.
- *
- * TODO: oid_object_info_extended()'s call stack has a recursive behavior. If
- * any of its callees end up calling it, this recursive call won't benefit from
- * parallel inflation.
- */
-void enable_obj_read_lock(void);
-void disable_obj_read_lock(void);
-
-extern int obj_read_use_lock;
-extern pthread_mutex_t obj_read_mutex;
-
-static inline void obj_read_lock(void)
-{
-	if(obj_read_use_lock)
-		pthread_mutex_lock(&obj_read_mutex);
-}
-
-static inline void obj_read_unlock(void)
-{
-	if(obj_read_use_lock)
-		pthread_mutex_unlock(&obj_read_mutex);
-}
-
-/*
- * Iterate over the files in the loose-object parts of the object
- * directory "path", triggering the following callbacks:
- *
- *  - loose_object is called for each loose object we find.
- *
- *  - loose_cruft is called for any files that do not appear to be
- *    loose objects. Note that we only look in the loose object
- *    directories "objects/[0-9a-f]{2}/", so we will not report
- *    "objects/foobar" as cruft.
- *
- *  - loose_subdir is called for each top-level hashed subdirectory
- *    of the object directory (e.g., "$OBJDIR/f0"). It is called
- *    after the objects in the directory are processed.
- *
- * Any callback that is NULL will be ignored. Callbacks returning non-zero
- * will end the iteration.
- *
- * In the "buf" variant, "path" is a strbuf which will also be used as a
- * scratch buffer, but restored to its original contents before
- * the function returns.
- */
-typedef int each_loose_object_fn(const struct object_id *oid,
-				 const char *path,
-				 void *data);
-typedef int each_loose_cruft_fn(const char *basename,
-				const char *path,
-				void *data);
-typedef int each_loose_subdir_fn(unsigned int nr,
-				 const char *path,
-				 void *data);
-int for_each_file_in_obj_subdir(unsigned int subdir_nr,
-				struct strbuf *path,
-				each_loose_object_fn obj_cb,
-				each_loose_cruft_fn cruft_cb,
-				each_loose_subdir_fn subdir_cb,
-				void *data);
-int for_each_loose_file_in_objdir(const char *path,
-				  each_loose_object_fn obj_cb,
-				  each_loose_cruft_fn cruft_cb,
-				  each_loose_subdir_fn subdir_cb,
-				  void *data);
-int for_each_loose_file_in_objdir_buf(struct strbuf *path,
-				      each_loose_object_fn obj_cb,
-				      each_loose_cruft_fn cruft_cb,
-				      each_loose_subdir_fn subdir_cb,
-				      void *data);
-
-/* Flags for for_each_*_object() below. */
-enum for_each_object_flags {
-	/* Iterate only over local objects, not alternates. */
-	FOR_EACH_OBJECT_LOCAL_ONLY = (1<<0),
-
-	/* Only iterate over packs obtained from the promisor remote. */
-	FOR_EACH_OBJECT_PROMISOR_ONLY = (1<<1),
-
-	/*
-	 * Visit objects within a pack in packfile order rather than .idx order
-	 */
-	FOR_EACH_OBJECT_PACK_ORDER = (1<<2),
-
-	/* Only iterate over packs that are not marked as kept in-core. */
-	FOR_EACH_OBJECT_SKIP_IN_CORE_KEPT_PACKS = (1<<3),
-
-	/* Only iterate over packs that do not have .keep files. */
-	FOR_EACH_OBJECT_SKIP_ON_DISK_KEPT_PACKS = (1<<4),
-};
-
-/*
- * Iterate over all accessible loose objects without respect to
- * reachability. By default, this includes both local and alternate objects.
- * The order in which objects are visited is unspecified.
- *
- * Any flags specific to packs are ignored.
- */
-int for_each_loose_object(each_loose_object_fn, void *,
-			  enum for_each_object_flags flags);
-
-/*
- * Iterate over all accessible packed objects without respect to reachability.
- * By default, this includes both local and alternate packs.
- *
- * Note that some objects may appear twice if they are found in multiple packs.
- * Each pack is visited in an unspecified order. By default, objects within a
- * pack are visited in pack-idx order (i.e., sorted by oid).
- */
-typedef int each_packed_object_fn(const struct object_id *oid,
-				  struct packed_git *pack,
-				  uint32_t pos,
-				  void *data);
-int for_each_object_in_pack(struct packed_git *p,
-			    each_packed_object_fn, void *data,
-			    enum for_each_object_flags flags);
-int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
-			   void *data, enum for_each_object_flags flags);
-
-void *read_object_with_reference(struct repository *r,
-				 const struct object_id *oid,
-				 enum object_type required_type,
-				 unsigned long *size,
-				 struct object_id *oid_ret);
-
-#endif /* OBJECT_STORE_LL_H */
diff --git a/object-store.c b/object-store.c
index 17fa06a86fa..fc48925d230 100644
--- a/object-store.c
+++ b/object-store.c
@@ -4,9 +4,11 @@
 #include "abspath.h"
 #include "commit-graph.h"
 #include "config.h"
+#include "dir.h"
 #include "environment.h"
 #include "gettext.h"
 #include "hex.h"
+#include "khash.h"
 #include "lockfile.h"
 #include "loose.h"
 #include "object-file-convert.h"
@@ -24,6 +26,9 @@
 #include "submodule.h"
 #include "write-or-die.h"
 
+KHASH_INIT(odb_path_map, const char * /* key: odb_path */,
+	struct object_directory *, 1, fspathhash, fspatheq)
+
 /*
  * This is meant to hold a *small* number of objects that you would
  * want repo_read_object_file() to be able to return, but yet you do not want
diff --git a/object-store.h b/object-store.h
index 1b3e3d7d014..46961dc9542 100644
--- a/object-store.h
+++ b/object-store.h
@@ -1,11 +1,517 @@
 #ifndef OBJECT_STORE_H
 #define OBJECT_STORE_H
 
-#include "khash.h"
-#include "dir.h"
-#include "object-store-ll.h"
+#include "hashmap.h"
+#include "object.h"
+#include "list.h"
+#include "oidset.h"
+#include "thread-utils.h"
 
-KHASH_INIT(odb_path_map, const char * /* key: odb_path */,
-	struct object_directory *, 1, fspathhash, fspatheq)
+struct oidmap;
+struct oidtree;
+struct strbuf;
+struct repository;
+
+struct object_directory {
+	struct object_directory *next;
+
+	/*
+	 * Used to store the results of readdir(3) calls when we are OK
+	 * sacrificing accuracy due to races for speed. That includes
+	 * object existence with OBJECT_INFO_QUICK, as well as
+	 * our search for unique abbreviated hashes. Don't use it for tasks
+	 * requiring greater accuracy!
+	 *
+	 * Be sure to call odb_load_loose_cache() before using.
+	 */
+	uint32_t loose_objects_subdir_seen[8]; /* 256 bits */
+	struct oidtree *loose_objects_cache;
+
+	/* Map between object IDs for loose objects. */
+	struct loose_object_map *loose_map;
+
+	/*
+	 * This is a temporary object store created by the tmp_objdir
+	 * facility. Disable ref updates since the objects in the store
+	 * might be discarded on rollback.
+	 */
+	int disable_ref_updates;
+
+	/*
+	 * This object store is ephemeral, so there is no need to fsync.
+	 */
+	int will_destroy;
+
+	/*
+	 * Path to the alternative object store. If this is a relative path,
+	 * it is relative to the current working directory.
+	 */
+	char *path;
+};
+
+void prepare_alt_odb(struct repository *r);
+int has_alt_odb(struct repository *r);
+char *compute_alternate_path(const char *path, struct strbuf *err);
+struct object_directory *find_odb(struct repository *r, const char *obj_dir);
+typedef int alt_odb_fn(struct object_directory *, void *);
+int foreach_alt_odb(alt_odb_fn, void*);
+typedef void alternate_ref_fn(const struct object_id *oid, void *);
+void for_each_alternate_ref(alternate_ref_fn, void *);
+
+/*
+ * Add the directory to the on-disk alternates file; the new entry will also
+ * take effect in the current process.
+ */
+void add_to_alternates_file(const char *dir);
+
+/*
+ * Add the directory to the in-memory list of alternates (along with any
+ * recursive alternates it points to), but do not modify the on-disk alternates
+ * file.
+ */
+void add_to_alternates_memory(const char *dir);
+
+/*
+ * Replace the current writable object directory with the specified temporary
+ * object directory; returns the former primary object directory.
+ */
+struct object_directory *set_temporary_primary_odb(const char *dir, int will_destroy);
+
+/*
+ * Restore a previous ODB replaced by set_temporary_main_odb.
+ */
+void restore_primary_odb(struct object_directory *restore_odb, const char *old_path);
+
+/*
+ * Populate and return the loose object cache array corresponding to the
+ * given object ID.
+ */
+struct oidtree *odb_loose_cache(struct object_directory *odb,
+				  const struct object_id *oid);
+
+/* Empty the loose object cache for the specified object directory. */
+void odb_clear_loose_cache(struct object_directory *odb);
+
+struct packed_git {
+	struct hashmap_entry packmap_ent;
+	struct packed_git *next;
+	struct list_head mru;
+	struct pack_window *windows;
+	off_t pack_size;
+	const void *index_data;
+	size_t index_size;
+	uint32_t num_objects;
+	size_t crc_offset;
+	struct oidset bad_objects;
+	int index_version;
+	time_t mtime;
+	int pack_fd;
+	int index;              /* for builtin/pack-objects.c */
+	unsigned pack_local:1,
+		 pack_keep:1,
+		 pack_keep_in_core:1,
+		 freshened:1,
+		 do_not_close:1,
+		 pack_promisor:1,
+		 multi_pack_index:1,
+		 is_cruft:1;
+	unsigned char hash[GIT_MAX_RAWSZ];
+	struct revindex_entry *revindex;
+	const uint32_t *revindex_data;
+	const uint32_t *revindex_map;
+	size_t revindex_size;
+	/*
+	 * mtimes_map points at the beginning of the memory mapped region of
+	 * this pack's corresponding .mtimes file, and mtimes_size is the size
+	 * of that .mtimes file
+	 */
+	const uint32_t *mtimes_map;
+	size_t mtimes_size;
+
+	/* repo denotes the repository this packfile belongs to */
+	struct repository *repo;
+
+	/* something like ".git/objects/pack/xxxxx.pack" */
+	char pack_name[FLEX_ARRAY]; /* more */
+};
+
+struct multi_pack_index;
+
+static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
+				     const struct hashmap_entry *entry,
+				     const struct hashmap_entry *entry2,
+				     const void *keydata)
+{
+	const char *key = keydata;
+	const struct packed_git *pg1, *pg2;
+
+	pg1 = container_of(entry, const struct packed_git, packmap_ent);
+	pg2 = container_of(entry2, const struct packed_git, packmap_ent);
+
+	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
+}
+
+struct cached_object_entry;
+
+struct raw_object_store {
+	/*
+	 * Set of all object directories; the main directory is first (and
+	 * cannot be NULL after initialization). Subsequent directories are
+	 * alternates.
+	 */
+	struct object_directory *odb;
+	struct object_directory **odb_tail;
+	struct kh_odb_path_map *odb_by_path;
+
+	int loaded_alternates;
+
+	/*
+	 * A list of alternate object directories loaded from the environment;
+	 * this should not generally need to be accessed directly, but will
+	 * populate the "odb" list when prepare_alt_odb() is run.
+	 */
+	char *alternate_db;
+
+	/*
+	 * Objects that should be substituted by other objects
+	 * (see git-replace(1)).
+	 */
+	struct oidmap *replace_map;
+	unsigned replace_map_initialized : 1;
+	pthread_mutex_t replace_mutex; /* protect object replace functions */
+
+	struct commit_graph *commit_graph;
+	unsigned commit_graph_attempted : 1; /* if loading has been attempted */
+
+	/*
+	 * private data
+	 *
+	 * should only be accessed directly by packfile.c and midx.c
+	 */
+	struct multi_pack_index *multi_pack_index;
+
+	/*
+	 * private data
+	 *
+	 * should only be accessed directly by packfile.c
+	 */
+
+	struct packed_git *packed_git;
+	/* A most-recently-used ordered version of the packed_git list. */
+	struct list_head packed_git_mru;
+
+	struct {
+		struct packed_git **packs;
+		unsigned flags;
+	} kept_pack_cache;
+
+	/*
+	 * This is meant to hold a *small* number of objects that you would
+	 * want repo_read_object_file() to be able to return, but yet you do not want
+	 * to write them into the object store (e.g. a browse-only
+	 * application).
+	 */
+	struct cached_object_entry *cached_objects;
+	size_t cached_object_nr, cached_object_alloc;
+
+	/*
+	 * A map of packfiles to packed_git structs for tracking which
+	 * packs have been loaded already.
+	 */
+	struct hashmap pack_map;
+
+	/*
+	 * A fast, rough count of the number of objects in the repository.
+	 * These two fields are not meant for direct access. Use
+	 * repo_approximate_object_count() instead.
+	 */
+	unsigned long approximate_object_count;
+	unsigned approximate_object_count_valid : 1;
+
+	/*
+	 * Whether packed_git has already been populated with this repository's
+	 * packs.
+	 */
+	unsigned packed_git_initialized : 1;
+};
+
+struct raw_object_store *raw_object_store_new(void);
+void raw_object_store_clear(struct raw_object_store *o);
+
+/*
+ * Create a temporary file rooted in the object database directory, or
+ * die on failure. The filename is taken from "pattern", which should have the
+ * usual "XXXXXX" trailer, and the resulting filename is written into the
+ * "template" buffer. Returns the open descriptor.
+ */
+int odb_mkstemp(struct strbuf *temp_filename, const char *pattern);
+
+/*
+ * Create a pack .keep file named "name" (which should generally be the output
+ * of odb_pack_name). Returns a file descriptor opened for writing, or -1 on
+ * error.
+ */
+int odb_pack_keep(const char *name);
+
+/*
+ * Put in `buf` the name of the file in the local object database that
+ * would be used to store a loose object with the specified oid.
+ */
+const char *loose_object_path(struct repository *r, struct strbuf *buf,
+			      const struct object_id *oid);
+
+void *map_loose_object(struct repository *r, const struct object_id *oid,
+		       unsigned long *size);
+
+void *repo_read_object_file(struct repository *r,
+			    const struct object_id *oid,
+			    enum object_type *type,
+			    unsigned long *size);
+
+/* Read and unpack an object file into memory, write memory to an object file */
+int oid_object_info(struct repository *r, const struct object_id *, unsigned long *);
+
+void hash_object_file(const struct git_hash_algo *algo, const void *buf,
+		      unsigned long len, enum object_type type,
+		      struct object_id *oid);
+
+/*
+ * Add an object file to the in-memory object store, without writing it
+ * to disk.
+ *
+ * Callers are responsible for calling write_object_file to record the
+ * object in persistent storage before writing any other new objects
+ * that reference it.
+ */
+int pretend_object_file(struct repository *repo,
+			void *buf, unsigned long len, enum object_type type,
+			struct object_id *oid);
+
+struct object_info {
+	/* Request */
+	enum object_type *typep;
+	unsigned long *sizep;
+	off_t *disk_sizep;
+	struct object_id *delta_base_oid;
+	struct strbuf *type_name;
+	void **contentp;
+
+	/* Response */
+	enum {
+		OI_CACHED,
+		OI_LOOSE,
+		OI_PACKED,
+		OI_DBCACHED
+	} whence;
+	union {
+		/*
+		 * struct {
+		 * 	... Nothing to expose in this case
+		 * } cached;
+		 * struct {
+		 * 	... Nothing to expose in this case
+		 * } loose;
+		 */
+		struct {
+			struct packed_git *pack;
+			off_t offset;
+			unsigned int is_delta;
+		} packed;
+	} u;
+};
+
+/*
+ * Initializer for a "struct object_info" that wants no items. You may
+ * also memset() the memory to all-zeroes.
+ */
+#define OBJECT_INFO_INIT { 0 }
+
+/* Invoke lookup_replace_object() on the given hash */
+#define OBJECT_INFO_LOOKUP_REPLACE 1
+/* Allow reading from a loose object file of unknown/bogus type */
+#define OBJECT_INFO_ALLOW_UNKNOWN_TYPE 2
+/* Do not retry packed storage after checking packed and loose storage */
+#define OBJECT_INFO_QUICK 8
+/*
+ * Do not attempt to fetch the object if missing (even if fetch_is_missing is
+ * nonzero).
+ */
+#define OBJECT_INFO_SKIP_FETCH_OBJECT 16
+/*
+ * This is meant for bulk prefetching of missing blobs in a partial
+ * clone. Implies OBJECT_INFO_SKIP_FETCH_OBJECT and OBJECT_INFO_QUICK
+ */
+#define OBJECT_INFO_FOR_PREFETCH (OBJECT_INFO_SKIP_FETCH_OBJECT | OBJECT_INFO_QUICK)
+
+/* Die if object corruption (not just an object being missing) was detected. */
+#define OBJECT_INFO_DIE_IF_CORRUPT 32
+
+int oid_object_info_extended(struct repository *r,
+			     const struct object_id *,
+			     struct object_info *, unsigned flags);
+
+/* Retry packed storage after checking packed and loose storage */
+#define HAS_OBJECT_RECHECK_PACKED 1
+
+/*
+ * Returns 1 if the object exists. This function will not lazily fetch objects
+ * in a partial clone.
+ */
+int has_object(struct repository *r, const struct object_id *oid,
+	       unsigned flags);
+
+/*
+ * These macros and functions are deprecated. If checking existence for an
+ * object that is likely to be missing and/or whose absence is relatively
+ * inconsequential (or is consequential but the caller is prepared to handle
+ * it), use has_object(), which has better defaults (no lazy fetch in a partial
+ * clone and no rechecking of packed storage). In the unlikely event that a
+ * caller needs to assert existence of an object that it fully expects to
+ * exist, and wants to trigger a lazy fetch in a partial clone, use
+ * oid_object_info_extended() with a NULL struct object_info.
+ *
+ * These functions can be removed once all callers have migrated to
+ * has_object() and/or oid_object_info_extended().
+ */
+int repo_has_object_file(struct repository *r, const struct object_id *oid);
+int repo_has_object_file_with_flags(struct repository *r,
+				    const struct object_id *oid, int flags);
+
+void assert_oid_type(const struct object_id *oid, enum object_type expect);
+
+/*
+ * Enabling the object read lock allows multiple threads to safely call the
+ * following functions in parallel: repo_read_object_file(),
+ * read_object_with_reference(), oid_object_info() and oid_object_info_extended().
+ *
+ * obj_read_lock() and obj_read_unlock() may also be used to protect other
+ * section which cannot execute in parallel with object reading. Since the used
+ * lock is a recursive mutex, these sections can even contain calls to object
+ * reading functions. However, beware that in these cases zlib inflation won't
+ * be performed in parallel, losing performance.
+ *
+ * TODO: oid_object_info_extended()'s call stack has a recursive behavior. If
+ * any of its callees end up calling it, this recursive call won't benefit from
+ * parallel inflation.
+ */
+void enable_obj_read_lock(void);
+void disable_obj_read_lock(void);
+
+extern int obj_read_use_lock;
+extern pthread_mutex_t obj_read_mutex;
+
+static inline void obj_read_lock(void)
+{
+	if(obj_read_use_lock)
+		pthread_mutex_lock(&obj_read_mutex);
+}
+
+static inline void obj_read_unlock(void)
+{
+	if(obj_read_use_lock)
+		pthread_mutex_unlock(&obj_read_mutex);
+}
+
+/*
+ * Iterate over the files in the loose-object parts of the object
+ * directory "path", triggering the following callbacks:
+ *
+ *  - loose_object is called for each loose object we find.
+ *
+ *  - loose_cruft is called for any files that do not appear to be
+ *    loose objects. Note that we only look in the loose object
+ *    directories "objects/[0-9a-f]{2}/", so we will not report
+ *    "objects/foobar" as cruft.
+ *
+ *  - loose_subdir is called for each top-level hashed subdirectory
+ *    of the object directory (e.g., "$OBJDIR/f0"). It is called
+ *    after the objects in the directory are processed.
+ *
+ * Any callback that is NULL will be ignored. Callbacks returning non-zero
+ * will end the iteration.
+ *
+ * In the "buf" variant, "path" is a strbuf which will also be used as a
+ * scratch buffer, but restored to its original contents before
+ * the function returns.
+ */
+typedef int each_loose_object_fn(const struct object_id *oid,
+				 const char *path,
+				 void *data);
+typedef int each_loose_cruft_fn(const char *basename,
+				const char *path,
+				void *data);
+typedef int each_loose_subdir_fn(unsigned int nr,
+				 const char *path,
+				 void *data);
+int for_each_file_in_obj_subdir(unsigned int subdir_nr,
+				struct strbuf *path,
+				each_loose_object_fn obj_cb,
+				each_loose_cruft_fn cruft_cb,
+				each_loose_subdir_fn subdir_cb,
+				void *data);
+int for_each_loose_file_in_objdir(const char *path,
+				  each_loose_object_fn obj_cb,
+				  each_loose_cruft_fn cruft_cb,
+				  each_loose_subdir_fn subdir_cb,
+				  void *data);
+int for_each_loose_file_in_objdir_buf(struct strbuf *path,
+				      each_loose_object_fn obj_cb,
+				      each_loose_cruft_fn cruft_cb,
+				      each_loose_subdir_fn subdir_cb,
+				      void *data);
+
+/* Flags for for_each_*_object() below. */
+enum for_each_object_flags {
+	/* Iterate only over local objects, not alternates. */
+	FOR_EACH_OBJECT_LOCAL_ONLY = (1<<0),
+
+	/* Only iterate over packs obtained from the promisor remote. */
+	FOR_EACH_OBJECT_PROMISOR_ONLY = (1<<1),
+
+	/*
+	 * Visit objects within a pack in packfile order rather than .idx order
+	 */
+	FOR_EACH_OBJECT_PACK_ORDER = (1<<2),
+
+	/* Only iterate over packs that are not marked as kept in-core. */
+	FOR_EACH_OBJECT_SKIP_IN_CORE_KEPT_PACKS = (1<<3),
+
+	/* Only iterate over packs that do not have .keep files. */
+	FOR_EACH_OBJECT_SKIP_ON_DISK_KEPT_PACKS = (1<<4),
+};
+
+/*
+ * Iterate over all accessible loose objects without respect to
+ * reachability. By default, this includes both local and alternate objects.
+ * The order in which objects are visited is unspecified.
+ *
+ * Any flags specific to packs are ignored.
+ */
+int for_each_loose_object(each_loose_object_fn, void *,
+			  enum for_each_object_flags flags);
+
+/*
+ * Iterate over all accessible packed objects without respect to reachability.
+ * By default, this includes both local and alternate packs.
+ *
+ * Note that some objects may appear twice if they are found in multiple packs.
+ * Each pack is visited in an unspecified order. By default, objects within a
+ * pack are visited in pack-idx order (i.e., sorted by oid).
+ */
+typedef int each_packed_object_fn(const struct object_id *oid,
+				  struct packed_git *pack,
+				  uint32_t pos,
+				  void *data);
+int for_each_object_in_pack(struct packed_git *p,
+			    each_packed_object_fn, void *data,
+			    enum for_each_object_flags flags);
+int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
+			   void *data, enum for_each_object_flags flags);
+
+void *read_object_with_reference(struct repository *r,
+				 const struct object_id *oid,
+				 enum object_type required_type,
+				 unsigned long *size,
+				 struct object_id *oid_ret);
 
 #endif /* OBJECT_STORE_H */
diff --git a/oss-fuzz/fuzz-pack-idx.c b/oss-fuzz/fuzz-pack-idx.c
index 3e190214d14..609a343ee3e 100644
--- a/oss-fuzz/fuzz-pack-idx.c
+++ b/oss-fuzz/fuzz-pack-idx.c
@@ -1,5 +1,5 @@
 #include "git-compat-util.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 
 int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size);
diff --git a/pack-bitmap-write.c b/pack-bitmap-write.c
index 6a97b52b36d..4c7f7985e78 100644
--- a/pack-bitmap-write.c
+++ b/pack-bitmap-write.c
@@ -4,7 +4,7 @@
 #include "environment.h"
 #include "gettext.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "diff.h"
 #include "revision.h"
diff --git a/pack-bitmap.c b/pack-bitmap.c
index 0dbd7c4ffe1..fb68d2ae637 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -17,7 +17,7 @@
 #include "packfile.h"
 #include "repository.h"
 #include "trace2.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "list-objects-filter-options.h"
 #include "midx.h"
 #include "config.h"
diff --git a/pack-check.c b/pack-check.c
index 95dcbbe9852..874897d6cba 100644
--- a/pack-check.c
+++ b/pack-check.c
@@ -8,7 +8,7 @@
 #include "progress.h"
 #include "packfile.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 struct idx_entry {
 	off_t                offset;
diff --git a/pack-mtimes.c b/pack-mtimes.c
index bcea28e521d..20900ca88d3 100644
--- a/pack-mtimes.c
+++ b/pack-mtimes.c
@@ -1,7 +1,7 @@
 #include "git-compat-util.h"
 #include "gettext.h"
 #include "pack-mtimes.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 #include "strbuf.h"
 
diff --git a/pack-objects.h b/pack-objects.h
index d73e3843c92..d1c4ae7f9b6 100644
--- a/pack-objects.h
+++ b/pack-objects.h
@@ -1,7 +1,7 @@
 #ifndef PACK_OBJECTS_H
 #define PACK_OBJECTS_H
 
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "thread-utils.h"
 #include "pack.h"
 
diff --git a/pack-revindex.c b/pack-revindex.c
index 1ee7b49e206..37ad9c79268 100644
--- a/pack-revindex.c
+++ b/pack-revindex.c
@@ -1,7 +1,7 @@
 #include "git-compat-util.h"
 #include "gettext.h"
 #include "pack-revindex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 #include "strbuf.h"
 #include "trace2.h"
diff --git a/packfile.c b/packfile.c
index 9d09f8bc726..d91016f1c7f 100644
--- a/packfile.c
+++ b/packfile.c
@@ -19,7 +19,7 @@
 #include "tree-walk.h"
 #include "tree.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "midx.h"
 #include "commit-graph.h"
 #include "pack-revindex.h"
diff --git a/path.c b/path.c
index 910756c8b32..1f7a014322c 100644
--- a/path.c
+++ b/path.c
@@ -15,7 +15,7 @@
 #include "submodule-config.h"
 #include "path.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "lockfile.h"
 #include "exec-cmd.h"
 
diff --git a/promisor-remote.c b/promisor-remote.c
index 5801ebfd9b2..9d058586dfa 100644
--- a/promisor-remote.c
+++ b/promisor-remote.c
@@ -3,7 +3,7 @@
 #include "git-compat-util.h"
 #include "gettext.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "promisor-remote.h"
 #include "config.h"
 #include "trace2.h"
diff --git a/protocol-caps.c b/protocol-caps.c
index 855f279c2f7..9b8db37a210 100644
--- a/protocol-caps.c
+++ b/protocol-caps.c
@@ -6,7 +6,7 @@
 #include "hash.h"
 #include "hex.h"
 #include "object.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "repository.h"
 #include "string-list.h"
 #include "strbuf.h"
diff --git a/prune-packed.c b/prune-packed.c
index 7dad2fc0c16..c1d95a519d7 100644
--- a/prune-packed.c
+++ b/prune-packed.c
@@ -2,7 +2,7 @@
 
 #include "git-compat-util.h"
 #include "gettext.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 #include "progress.h"
 #include "prune-packed.h"
diff --git a/reachable.c b/reachable.c
index 1b26b9b1d76..16e23a38037 100644
--- a/reachable.c
+++ b/reachable.c
@@ -14,7 +14,7 @@
 #include "list-objects.h"
 #include "packfile.h"
 #include "worktree.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pack-bitmap.h"
 #include "pack-mtimes.h"
 #include "config.h"
diff --git a/read-cache.c b/read-cache.c
index 9909b56902f..9aef59cab4c 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -22,7 +22,7 @@
 #include "refs.h"
 #include "dir.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "tree.h"
 #include "commit.h"
diff --git a/ref-filter.c b/ref-filter.c
index 6da8d4c03b6..7a274633cfc 100644
--- a/ref-filter.c
+++ b/ref-filter.c
@@ -12,7 +12,7 @@
 #include "refs.h"
 #include "wildmatch.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "repo-settings.h"
 #include "repository.h"
diff --git a/reflog.c b/reflog.c
index 1b5f031f6d7..60834a124d9 100644
--- a/reflog.c
+++ b/reflog.c
@@ -3,7 +3,7 @@
 
 #include "git-compat-util.h"
 #include "gettext.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "reflog.h"
 #include "refs.h"
 #include "revision.h"
diff --git a/refs.c b/refs.c
index 1208f86629b..14f25c499cd 100644
--- a/refs.c
+++ b/refs.c
@@ -19,7 +19,7 @@
 #include "run-command.h"
 #include "hook.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "object.h"
 #include "path.h"
 #include "submodule.h"
diff --git a/remote.c b/remote.c
index e609cf5c56a..7fc657d98b8 100644
--- a/remote.c
+++ b/remote.c
@@ -12,7 +12,7 @@
 #include "refs.h"
 #include "refspec.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "commit.h"
 #include "diff.h"
diff --git a/replace-object.c b/replace-object.c
index 9a3cdd809a9..7b8a09b5cb4 100644
--- a/replace-object.c
+++ b/replace-object.c
@@ -2,7 +2,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "oidmap.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "replace-object.h"
 #include "refs.h"
 #include "repository.h"
diff --git a/replace-object.h b/replace-object.h
index 66c41b938b4..ba478eb30c4 100644
--- a/replace-object.h
+++ b/replace-object.h
@@ -3,7 +3,7 @@
 
 #include "oidmap.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 struct replace_object {
 	struct oidmap_entry original;
diff --git a/repository.c b/repository.c
index 6cbaf2e3daa..9b3d6665fc6 100644
--- a/repository.c
+++ b/repository.c
@@ -1,7 +1,7 @@
 #include "git-compat-util.h"
 #include "abspath.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "config.h"
 #include "object.h"
 #include "lockfile.h"
diff --git a/rerere.c b/rerere.c
index 740e8ad1a0b..28e59c42380 100644
--- a/rerere.c
+++ b/rerere.c
@@ -18,7 +18,7 @@
 #include "path.h"
 #include "pathspec.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "strmap.h"
 
 #define RESOLVED 0
diff --git a/revision.c b/revision.c
index b536c4a29ad..352e18b1f97 100644
--- a/revision.c
+++ b/revision.c
@@ -8,7 +8,7 @@
 #include "hex.h"
 #include "object-name.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oidset.h"
 #include "tag.h"
 #include "blob.h"
diff --git a/send-pack.c b/send-pack.c
index 856a65d5f5a..5005689cb55 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -4,7 +4,7 @@
 #include "date.h"
 #include "gettext.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pkt-line.h"
 #include "sideband.h"
 #include "run-command.h"
diff --git a/sequencer.c b/sequencer.c
index c112d2e1c43..925cfa1af68 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -13,7 +13,7 @@
 #include "dir.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "object.h"
 #include "pager.h"
 #include "commit.h"
diff --git a/server-info.c b/server-info.c
index 1ca0e00d51e..3048e5ee407 100644
--- a/server-info.c
+++ b/server-info.c
@@ -11,7 +11,7 @@
 #include "packfile.h"
 #include "path.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "server-info.h"
 #include "strbuf.h"
 #include "tempfile.h"
diff --git a/shallow.c b/shallow.c
index 06c3266a3e0..2f82ebd6e3f 100644
--- a/shallow.c
+++ b/shallow.c
@@ -5,7 +5,7 @@
 #include "repository.h"
 #include "tempfile.h"
 #include "lockfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tag.h"
 #include "pkt-line.h"
diff --git a/streaming.c b/streaming.c
index 018b794d252..127d6b5d6ac 100644
--- a/streaming.c
+++ b/streaming.c
@@ -10,7 +10,7 @@
 #include "streaming.h"
 #include "repository.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "replace-object.h"
 #include "packfile.h"
 
diff --git a/submodule-config.c b/submodule-config.c
index d82b404b73e..8630e27947d 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -13,7 +13,7 @@
 #include "submodule.h"
 #include "strbuf.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "parse-options.h"
 #include "thread-utils.h"
 #include "tree-walk.h"
diff --git a/submodule.c b/submodule.c
index 0821507ecaa..3e52ef3b036 100644
--- a/submodule.c
+++ b/submodule.c
@@ -27,7 +27,7 @@
 #include "parse-options.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit-reach.h"
 #include "read-cache-ll.h"
 #include "setup.h"
diff --git a/t/helper/test-pack-mtimes.c b/t/helper/test-pack-mtimes.c
index f8f9afbb5b1..50f5941bff7 100644
--- a/t/helper/test-pack-mtimes.c
+++ b/t/helper/test-pack-mtimes.c
@@ -3,7 +3,7 @@
 #include "test-tool.h"
 #include "hex.h"
 #include "strbuf.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 #include "pack-mtimes.h"
 #include "setup.h"
diff --git a/t/helper/test-partial-clone.c b/t/helper/test-partial-clone.c
index a1af9710c31..34f1aee5581 100644
--- a/t/helper/test-partial-clone.c
+++ b/t/helper/test-partial-clone.c
@@ -1,7 +1,7 @@
 #include "test-tool.h"
 #include "hex.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "setup.h"
 
 /*
diff --git a/t/helper/test-read-graph.c b/t/helper/test-read-graph.c
index 811dde1cb3c..8b413b644be 100644
--- a/t/helper/test-read-graph.c
+++ b/t/helper/test-read-graph.c
@@ -3,7 +3,7 @@
 #include "test-tool.h"
 #include "commit-graph.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "bloom.h"
 #include "setup.h"
 
diff --git a/t/helper/test-read-midx.c b/t/helper/test-read-midx.c
index fc632369618..ac81390899a 100644
--- a/t/helper/test-read-midx.c
+++ b/t/helper/test-read-midx.c
@@ -4,7 +4,7 @@
 #include "hex.h"
 #include "midx.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pack-bitmap.h"
 #include "packfile.h"
 #include "setup.h"
diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c
index 2ff67c067ac..4cfc7c90b59 100644
--- a/t/helper/test-ref-store.c
+++ b/t/helper/test-ref-store.c
@@ -5,7 +5,7 @@
 #include "refs.h"
 #include "setup.h"
 #include "worktree.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "repository.h"
 #include "strbuf.h"
diff --git a/tag.c b/tag.c
index 8d9e9e29304..05be39067cf 100644
--- a/tag.c
+++ b/tag.c
@@ -5,7 +5,7 @@
 #include "environment.h"
 #include "tag.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tree.h"
 #include "blob.h"
diff --git a/tmp-objdir.c b/tmp-objdir.c
index 31d16a4c2c5..c38fbeb5e8a 100644
--- a/tmp-objdir.c
+++ b/tmp-objdir.c
@@ -10,7 +10,7 @@
 #include "strbuf.h"
 #include "strvec.h"
 #include "quote.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "repository.h"
 
 struct tmp_objdir {
diff --git a/tree-walk.c b/tree-walk.c
index a0333979656..90655d52378 100644
--- a/tree-walk.c
+++ b/tree-walk.c
@@ -6,7 +6,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "trace2.h"
 #include "tree.h"
 #include "pathspec.h"
diff --git a/tree.c b/tree.c
index ad86ad1ba99..b85f56267fb 100644
--- a/tree.c
+++ b/tree.c
@@ -4,7 +4,7 @@
 #include "hex.h"
 #include "tree.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "alloc.h"
 #include "tree-walk.h"
diff --git a/unpack-trees.c b/unpack-trees.c
index cf5b73c84be..471837f0329 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -26,7 +26,7 @@
 #include "symlinks.h"
 #include "trace2.h"
 #include "fsmonitor.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "promisor-remote.h"
 #include "entry.h"
 #include "parallel-checkout.h"
diff --git a/upload-pack.c b/upload-pack.c
index 02ce6336028..30e4630f3a1 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -10,7 +10,7 @@
 #include "pkt-line.h"
 #include "sideband.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "object.h"
 #include "commit.h"
diff --git a/walker.c b/walker.c
index 1cf3da02193..4fedc19f346 100644
--- a/walker.c
+++ b/walker.c
@@ -5,7 +5,7 @@
 #include "hex.h"
 #include "walker.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "strbuf.h"
 #include "tree.h"
diff --git a/xdiff-interface.c b/xdiff-interface.c
index 77712811ff1..1edcd319e6e 100644
--- a/xdiff-interface.c
+++ b/xdiff-interface.c
@@ -5,7 +5,7 @@
 #include "gettext.h"
 #include "config.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "strbuf.h"
 #include "xdiff-interface.h"
 #include "xdiff/xtypes.h"

-- 
2.49.0.682.gc9b6a7b2b0.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* Re: [PATCH 0/9] Split up "object-file.c"
  2025-04-08 10:24 [PATCH 0/9] Split up "object-file.c" Patrick Steinhardt
                   ` (8 preceding siblings ...)
  2025-04-08 10:24 ` [PATCH 9/9] object-store: merge "object-store-ll.h" and "object-store.h" Patrick Steinhardt
@ 2025-04-08 23:29 ` Junio C Hamano
  2025-04-11  9:26   ` Patrick Steinhardt
  2025-04-09 14:42 ` Elijah Newren
                   ` (2 subsequent siblings)
  12 siblings, 1 reply; 53+ messages in thread
From: Junio C Hamano @ 2025-04-08 23:29 UTC (permalink / raw)
  To: Patrick Steinhardt; +Cc: git

Patrick Steinhardt <ps@pks.im> writes:

> The series is built on top of 9d22ac51228 (The third batch, 2025-04-07)
> with ps/object-wo-the-repository at 9442b1c919a (Merge remote-tracking
> branch 'junio/ps/object-wo-the-repository' into HEAD, 2025-04-08) merged
> into it.

As 9442b1c919a is not public, this description lacks a way to verify
the result I attempted to reproduce.  ps/object-wo-the-repository I
have ends with 7d70b29c (hash: stop depending on `the_repository` in
`null_oid()`, 2025-03-10) and it hasn't moved for a while, so
hopefully we are in agreement ;-)

There were a few unpleasant interactions with other topics in
flight; please check the conflict resolution I made.

Will queue.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH 1/9] object-file: move `safe_create_leading_directories()` into "dir.c"
  2025-04-08 10:24 ` [PATCH 1/9] object-file: move `safe_create_leading_directories()` into "dir.c" Patrick Steinhardt
@ 2025-04-09 14:36   ` Elijah Newren
  2025-04-11  9:27     ` Patrick Steinhardt
  0 siblings, 1 reply; 53+ messages in thread
From: Elijah Newren @ 2025-04-09 14:36 UTC (permalink / raw)
  To: Patrick Steinhardt; +Cc: git

On Tue, Apr 8, 2025 at 3:37 AM Patrick Steinhardt <ps@pks.im> wrote:
>
> The `safe_create_leading_directories()` function and its relatives

How is mkdir_in_gitdir() a relative of safe_create_leading_directories()?

I assumed the relation was "called by", but there is no such
relationship.  The rest of the patch looked fine, but I was puzzled
for a while trying to figure out what this relationship is.

> are
> located in "object-file.c", which is not a good fit as they provide
> generic functionality not related to objects at all. Move them into
> "dir.c".

Makes sense.


>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  builtin/bugreport.c                |   2 +-
>  builtin/credential-cache--daemon.c |   2 +-
>  builtin/diagnose.c                 |   2 +-
>  builtin/fsck.c                     |   1 +
>  builtin/gc.c                       |   2 +-
>  builtin/init-db.c                  |   2 +-
>  builtin/log.c                      |   2 +-
>  commit-graph.c                     |   1 +
>  dir.c                              | 107 ++++++++++++++++++++++++++++++++++++-
>  dir.h                              |  35 ++++++++++++
>  midx-write.c                       |   1 +
>  object-file.c                      | 106 ------------------------------------
>  object-file.h                      |  35 ------------
>  13 files changed, 150 insertions(+), 148 deletions(-)
>
> diff --git a/builtin/bugreport.c b/builtin/bugreport.c
> index 66d64bfd5ae..d07fa91c247 100644
> --- a/builtin/bugreport.c
> +++ b/builtin/bugreport.c
> @@ -1,6 +1,7 @@
>  #define USE_THE_REPOSITORY_VARIABLE
>  #include "builtin.h"
>  #include "abspath.h"
> +#include "dir.h"
>  #include "editor.h"
>  #include "gettext.h"
>  #include "parse-options.h"
> @@ -10,7 +11,6 @@
>  #include "hook.h"
>  #include "hook-list.h"
>  #include "diagnose.h"
> -#include "object-file.h"
>  #include "setup.h"
>  #include "version.h"
>
> diff --git a/builtin/credential-cache--daemon.c b/builtin/credential-cache--daemon.c
> index e707618e743..80d29b4f5c0 100644
> --- a/builtin/credential-cache--daemon.c
> +++ b/builtin/credential-cache--daemon.c
> @@ -1,8 +1,8 @@
>  #define USE_THE_REPOSITORY_VARIABLE
>  #include "builtin.h"
>  #include "abspath.h"
> +#include "dir.h"
>  #include "gettext.h"
> -#include "object-file.h"
>  #include "parse-options.h"
>
>  #ifndef NO_UNIX_SOCKETS
> diff --git a/builtin/diagnose.c b/builtin/diagnose.c
> index 33c39bd5981..d5dadd6a48b 100644
> --- a/builtin/diagnose.c
> +++ b/builtin/diagnose.c
> @@ -2,8 +2,8 @@
>
>  #include "builtin.h"
>  #include "abspath.h"
> +#include "dir.h"
>  #include "gettext.h"
> -#include "object-file.h"
>  #include "parse-options.h"
>  #include "diagnose.h"
>
> diff --git a/builtin/fsck.c b/builtin/fsck.c
> index 9c8a6d6a8df..32d40d8f9fc 100644
> --- a/builtin/fsck.c
> +++ b/builtin/fsck.c
> @@ -1,5 +1,6 @@
>  #define USE_THE_REPOSITORY_VARIABLE
>  #include "builtin.h"
> +#include "dir.h"
>  #include "gettext.h"
>  #include "hex.h"
>  #include "config.h"
> diff --git a/builtin/gc.c b/builtin/gc.c
> index 99431fd4674..b069629676c 100644
> --- a/builtin/gc.c
> +++ b/builtin/gc.c
> @@ -16,6 +16,7 @@
>  #include "builtin.h"
>  #include "abspath.h"
>  #include "date.h"
> +#include "dir.h"
>  #include "environment.h"
>  #include "hex.h"
>  #include "config.h"
> @@ -28,7 +29,6 @@
>  #include "commit.h"
>  #include "commit-graph.h"
>  #include "packfile.h"
> -#include "object-file.h"
>  #include "object-store-ll.h"
>  #include "pack.h"
>  #include "pack-objects.h"
> diff --git a/builtin/init-db.c b/builtin/init-db.c
> index 196dccdd77a..39730c1b0ce 100644
> --- a/builtin/init-db.c
> +++ b/builtin/init-db.c
> @@ -6,9 +6,9 @@
>  #define USE_THE_REPOSITORY_VARIABLE
>  #include "builtin.h"
>  #include "abspath.h"
> +#include "dir.h"
>  #include "environment.h"
>  #include "gettext.h"
> -#include "object-file.h"
>  #include "parse-options.h"
>  #include "path.h"
>  #include "refs.h"
> diff --git a/builtin/log.c b/builtin/log.c
> index 0d4c579dad7..06ffaa93e86 100644
> --- a/builtin/log.c
> +++ b/builtin/log.c
> @@ -10,11 +10,11 @@
>  #include "builtin.h"
>  #include "abspath.h"
>  #include "config.h"
> +#include "dir.h"
>  #include "environment.h"
>  #include "gettext.h"
>  #include "hex.h"
>  #include "refs.h"
> -#include "object-file.h"
>  #include "object-name.h"
>  #include "object-store-ll.h"
>  #include "pager.h"
> diff --git a/commit-graph.c b/commit-graph.c
> index 8286d5dda24..3fae20dc21b 100644
> --- a/commit-graph.c
> +++ b/commit-graph.c
> @@ -4,6 +4,7 @@
>  #include "git-compat-util.h"
>  #include "config.h"
>  #include "csum-file.h"
> +#include "dir.h"
>  #include "gettext.h"
>  #include "hex.h"
>  #include "lockfile.h"
> diff --git a/dir.c b/dir.c
> index 28b0e03feb4..16ae3b5169d 100644
> --- a/dir.c
> +++ b/dir.c
> @@ -17,7 +17,6 @@
>  #include "environment.h"
>  #include "gettext.h"
>  #include "name-hash.h"
> -#include "object-file.h"
>  #include "object-store-ll.h"
>  #include "path.h"
>  #include "refs.h"
> @@ -4132,3 +4131,109 @@ int path_match_flags(const char *const str, const enum path_match_flags flags)
>                 return is_xplatform_dir_sep(*p);
>         BUG("unreachable");
>  }
> +
> +int mkdir_in_gitdir(const char *path)
> +{
> +       if (mkdir(path, 0777)) {
> +               int saved_errno = errno;
> +               struct stat st;
> +               struct strbuf sb = STRBUF_INIT;
> +
> +               if (errno != EEXIST)
> +                       return -1;
> +               /*
> +                * Are we looking at a path in a symlinked worktree
> +                * whose original repository does not yet have it?
> +                * e.g. .git/rr-cache pointing at its original
> +                * repository in which the user hasn't performed any
> +                * conflict resolution yet?
> +                */
> +               if (lstat(path, &st) || !S_ISLNK(st.st_mode) ||
> +                   strbuf_readlink(&sb, path, st.st_size) ||
> +                   !is_absolute_path(sb.buf) ||
> +                   mkdir(sb.buf, 0777)) {
> +                       strbuf_release(&sb);
> +                       errno = saved_errno;
> +                       return -1;
> +               }
> +               strbuf_release(&sb);
> +       }
> +       return adjust_shared_perm(the_repository, path);
> +}
> +
> +static enum scld_error safe_create_leading_directories_1(char *path, int share)
> +{
> +       char *next_component = path + offset_1st_component(path);
> +       enum scld_error ret = SCLD_OK;
> +
> +       while (ret == SCLD_OK && next_component) {
> +               struct stat st;
> +               char *slash = next_component, slash_character;
> +
> +               while (*slash && !is_dir_sep(*slash))
> +                       slash++;
> +
> +               if (!*slash)
> +                       break;
> +
> +               next_component = slash + 1;
> +               while (is_dir_sep(*next_component))
> +                       next_component++;
> +               if (!*next_component)
> +                       break;
> +
> +               slash_character = *slash;
> +               *slash = '\0';
> +               if (!stat(path, &st)) {
> +                       /* path exists */
> +                       if (!S_ISDIR(st.st_mode)) {
> +                               errno = ENOTDIR;
> +                               ret = SCLD_EXISTS;
> +                       }
> +               } else if (mkdir(path, 0777)) {
> +                       if (errno == EEXIST &&
> +                           !stat(path, &st) && S_ISDIR(st.st_mode))
> +                               ; /* somebody created it since we checked */
> +                       else if (errno == ENOENT)
> +                               /*
> +                                * Either mkdir() failed because
> +                                * somebody just pruned the containing
> +                                * directory, or stat() failed because
> +                                * the file that was in our way was
> +                                * just removed.  Either way, inform
> +                                * the caller that it might be worth
> +                                * trying again:
> +                                */
> +                               ret = SCLD_VANISHED;
> +                       else
> +                               ret = SCLD_FAILED;
> +               } else if (share && adjust_shared_perm(the_repository, path)) {
> +                       ret = SCLD_PERMS;
> +               }
> +               *slash = slash_character;
> +       }
> +       return ret;
> +}
> +
> +enum scld_error safe_create_leading_directories(char *path)
> +{
> +       return safe_create_leading_directories_1(path, 1);
> +}
> +
> +enum scld_error safe_create_leading_directories_no_share(char *path)
> +{
> +       return safe_create_leading_directories_1(path, 0);
> +}
> +
> +enum scld_error safe_create_leading_directories_const(const char *path)
> +{
> +       int save_errno;
> +       /* path points to cache entries, so xstrdup before messing with it */
> +       char *buf = xstrdup(path);
> +       enum scld_error result = safe_create_leading_directories(buf);
> +
> +       save_errno = errno;
> +       free(buf);
> +       errno = save_errno;
> +       return result;
> +}
> diff --git a/dir.h b/dir.h
> index d7e71aa8daa..02c1f9420b0 100644
> --- a/dir.h
> +++ b/dir.h
> @@ -676,4 +676,39 @@ static inline int starts_with_dot_dot_slash_native(const char *const path)
>         return path_match_flags(path, what | PATH_MATCH_NATIVE);
>  }
>
> +/*
> + * Create the directory containing the named path, using care to be
> + * somewhat safe against races. Return one of the scld_error values to
> + * indicate success/failure. On error, set errno to describe the
> + * problem.
> + *
> + * SCLD_VANISHED indicates that one of the ancestor directories of the
> + * path existed at one point during the function call and then
> + * suddenly vanished, probably because another process pruned the
> + * directory while we were working.  To be robust against this kind of
> + * race, callers might want to try invoking the function again when it
> + * returns SCLD_VANISHED.
> + *
> + * safe_create_leading_directories() temporarily changes path while it
> + * is working but restores it before returning.
> + * safe_create_leading_directories_const() doesn't modify path, even
> + * temporarily. Both these variants adjust the permissions of the
> + * created directories to honor core.sharedRepository, so they are best
> + * suited for files inside the git dir. For working tree files, use
> + * safe_create_leading_directories_no_share() instead, as it ignores
> + * the core.sharedRepository setting.
> + */
> +enum scld_error {
> +       SCLD_OK = 0,
> +       SCLD_FAILED = -1,
> +       SCLD_PERMS = -2,
> +       SCLD_EXISTS = -3,
> +       SCLD_VANISHED = -4
> +};
> +enum scld_error safe_create_leading_directories(char *path);
> +enum scld_error safe_create_leading_directories_const(const char *path);
> +enum scld_error safe_create_leading_directories_no_share(char *path);
> +
> +int mkdir_in_gitdir(const char *path);
> +
>  #endif
> diff --git a/midx-write.c b/midx-write.c
> index a628ac24dcb..e01a867c583 100644
> --- a/midx-write.c
> +++ b/midx-write.c
> @@ -3,6 +3,7 @@
>  #include "git-compat-util.h"
>  #include "abspath.h"
>  #include "config.h"
> +#include "dir.h"
>  #include "hex.h"
>  #include "lockfile.h"
>  #include "packfile.h"
> diff --git a/object-file.c b/object-file.c
> index 772c311f188..23b2c8560be 100644
> --- a/object-file.c
> +++ b/object-file.c
> @@ -91,112 +91,6 @@ static int get_conv_flags(unsigned flags)
>  }
>
>
> -int mkdir_in_gitdir(const char *path)
> -{
> -       if (mkdir(path, 0777)) {
> -               int saved_errno = errno;
> -               struct stat st;
> -               struct strbuf sb = STRBUF_INIT;
> -
> -               if (errno != EEXIST)
> -                       return -1;
> -               /*
> -                * Are we looking at a path in a symlinked worktree
> -                * whose original repository does not yet have it?
> -                * e.g. .git/rr-cache pointing at its original
> -                * repository in which the user hasn't performed any
> -                * conflict resolution yet?
> -                */
> -               if (lstat(path, &st) || !S_ISLNK(st.st_mode) ||
> -                   strbuf_readlink(&sb, path, st.st_size) ||
> -                   !is_absolute_path(sb.buf) ||
> -                   mkdir(sb.buf, 0777)) {
> -                       strbuf_release(&sb);
> -                       errno = saved_errno;
> -                       return -1;
> -               }
> -               strbuf_release(&sb);
> -       }
> -       return adjust_shared_perm(the_repository, path);
> -}
> -
> -static enum scld_error safe_create_leading_directories_1(char *path, int share)
> -{
> -       char *next_component = path + offset_1st_component(path);
> -       enum scld_error ret = SCLD_OK;
> -
> -       while (ret == SCLD_OK && next_component) {
> -               struct stat st;
> -               char *slash = next_component, slash_character;
> -
> -               while (*slash && !is_dir_sep(*slash))
> -                       slash++;
> -
> -               if (!*slash)
> -                       break;
> -
> -               next_component = slash + 1;
> -               while (is_dir_sep(*next_component))
> -                       next_component++;
> -               if (!*next_component)
> -                       break;
> -
> -               slash_character = *slash;
> -               *slash = '\0';
> -               if (!stat(path, &st)) {
> -                       /* path exists */
> -                       if (!S_ISDIR(st.st_mode)) {
> -                               errno = ENOTDIR;
> -                               ret = SCLD_EXISTS;
> -                       }
> -               } else if (mkdir(path, 0777)) {
> -                       if (errno == EEXIST &&
> -                           !stat(path, &st) && S_ISDIR(st.st_mode))
> -                               ; /* somebody created it since we checked */
> -                       else if (errno == ENOENT)
> -                               /*
> -                                * Either mkdir() failed because
> -                                * somebody just pruned the containing
> -                                * directory, or stat() failed because
> -                                * the file that was in our way was
> -                                * just removed.  Either way, inform
> -                                * the caller that it might be worth
> -                                * trying again:
> -                                */
> -                               ret = SCLD_VANISHED;
> -                       else
> -                               ret = SCLD_FAILED;
> -               } else if (share && adjust_shared_perm(the_repository, path)) {
> -                       ret = SCLD_PERMS;
> -               }
> -               *slash = slash_character;
> -       }
> -       return ret;
> -}
> -
> -enum scld_error safe_create_leading_directories(char *path)
> -{
> -       return safe_create_leading_directories_1(path, 1);
> -}
> -
> -enum scld_error safe_create_leading_directories_no_share(char *path)
> -{
> -       return safe_create_leading_directories_1(path, 0);
> -}
> -
> -enum scld_error safe_create_leading_directories_const(const char *path)
> -{
> -       int save_errno;
> -       /* path points to cache entries, so xstrdup before messing with it */
> -       char *buf = xstrdup(path);
> -       enum scld_error result = safe_create_leading_directories(buf);
> -
> -       save_errno = errno;
> -       free(buf);
> -       errno = save_errno;
> -       return result;
> -}
> -
>  int odb_mkstemp(struct strbuf *temp_filename, const char *pattern)
>  {
>         int fd;
> diff --git a/object-file.h b/object-file.h
> index 81b30d269c8..922f2bba8c9 100644
> --- a/object-file.h
> +++ b/object-file.h
> @@ -21,41 +21,6 @@ extern int fetch_if_missing;
>  int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);
>  int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags);
>
> -/*
> - * Create the directory containing the named path, using care to be
> - * somewhat safe against races. Return one of the scld_error values to
> - * indicate success/failure. On error, set errno to describe the
> - * problem.
> - *
> - * SCLD_VANISHED indicates that one of the ancestor directories of the
> - * path existed at one point during the function call and then
> - * suddenly vanished, probably because another process pruned the
> - * directory while we were working.  To be robust against this kind of
> - * race, callers might want to try invoking the function again when it
> - * returns SCLD_VANISHED.
> - *
> - * safe_create_leading_directories() temporarily changes path while it
> - * is working but restores it before returning.
> - * safe_create_leading_directories_const() doesn't modify path, even
> - * temporarily. Both these variants adjust the permissions of the
> - * created directories to honor core.sharedRepository, so they are best
> - * suited for files inside the git dir. For working tree files, use
> - * safe_create_leading_directories_no_share() instead, as it ignores
> - * the core.sharedRepository setting.
> - */
> -enum scld_error {
> -       SCLD_OK = 0,
> -       SCLD_FAILED = -1,
> -       SCLD_PERMS = -2,
> -       SCLD_EXISTS = -3,
> -       SCLD_VANISHED = -4
> -};
> -enum scld_error safe_create_leading_directories(char *path);
> -enum scld_error safe_create_leading_directories_const(const char *path);
> -enum scld_error safe_create_leading_directories_no_share(char *path);
> -
> -int mkdir_in_gitdir(const char *path);
> -
>  int git_open_cloexec(const char *name, int flags);
>  #define git_open(name) git_open_cloexec(name, O_RDONLY)
>
>
> --
> 2.49.0.682.gc9b6a7b2b0.dirty
>
>

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH 3/9] object-file: move `xmmap()` into "wrapper.c"
  2025-04-08 10:24 ` [PATCH 3/9] object-file: move `xmmap()` into "wrapper.c" Patrick Steinhardt
@ 2025-04-09 14:36   ` Elijah Newren
  0 siblings, 0 replies; 53+ messages in thread
From: Elijah Newren @ 2025-04-09 14:36 UTC (permalink / raw)
  To: Patrick Steinhardt; +Cc: git

On Tue, Apr 8, 2025 at 3:33 AM Patrick Steinhardt <ps@pks.im> wrote:
>
> The `xmmap()` function is provided by "object-file.c" even though its
> functionality got nothing to do with the object file subsystem. Move it

s/got/has/ ?


> into "wrapper.c", whose header already declares those functions.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  object-file.c | 48 ------------------------------------------------
>  wrapper.c     | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 48 insertions(+), 48 deletions(-)
>
> diff --git a/object-file.c b/object-file.c
> index 1a20c7fa072..ea2ed7628e6 100644
> --- a/object-file.c
> +++ b/object-file.c
> @@ -719,54 +719,6 @@ int has_loose_object(const struct object_id *oid)
>         return check_and_freshen(oid, 0);
>  }
>
> -static void mmap_limit_check(size_t length)
> -{
> -       static size_t limit = 0;
> -       if (!limit) {
> -               limit = git_env_ulong("GIT_MMAP_LIMIT", 0);
> -               if (!limit)
> -                       limit = SIZE_MAX;
> -       }
> -       if (length > limit)
> -               die(_("attempting to mmap %"PRIuMAX" over limit %"PRIuMAX),
> -                   (uintmax_t)length, (uintmax_t)limit);
> -}
> -
> -void *xmmap_gently(void *start, size_t length,
> -                 int prot, int flags, int fd, off_t offset)
> -{
> -       void *ret;
> -
> -       mmap_limit_check(length);
> -       ret = mmap(start, length, prot, flags, fd, offset);
> -       if (ret == MAP_FAILED && !length)
> -               ret = NULL;
> -       return ret;
> -}
> -
> -const char *mmap_os_err(void)
> -{
> -       static const char blank[] = "";
> -#if defined(__linux__)
> -       if (errno == ENOMEM) {
> -               /* this continues an existing error message: */
> -               static const char enomem[] =
> -", check sys.vm.max_map_count and/or RLIMIT_DATA";
> -               return enomem;
> -       }
> -#endif /* OS-specific bits */
> -       return blank;
> -}
> -
> -void *xmmap(void *start, size_t length,
> -       int prot, int flags, int fd, off_t offset)
> -{
> -       void *ret = xmmap_gently(start, length, prot, flags, fd, offset);
> -       if (ret == MAP_FAILED)
> -               die_errno(_("mmap failed%s"), mmap_os_err());
> -       return ret;
> -}
> -
>  static int format_object_header_literally(char *str, size_t size,
>                                           const char *type, size_t objsize)
>  {
> diff --git a/wrapper.c b/wrapper.c
> index 8b985931490..3c79778055e 100644
> --- a/wrapper.c
> +++ b/wrapper.c
> @@ -829,3 +829,51 @@ uint32_t git_rand(unsigned flags)
>
>         return result;
>  }
> +
> +static void mmap_limit_check(size_t length)
> +{
> +       static size_t limit = 0;
> +       if (!limit) {
> +               limit = git_env_ulong("GIT_MMAP_LIMIT", 0);
> +               if (!limit)
> +                       limit = SIZE_MAX;
> +       }
> +       if (length > limit)
> +               die(_("attempting to mmap %"PRIuMAX" over limit %"PRIuMAX),
> +                   (uintmax_t)length, (uintmax_t)limit);
> +}
> +
> +void *xmmap_gently(void *start, size_t length,
> +                 int prot, int flags, int fd, off_t offset)
> +{
> +       void *ret;
> +
> +       mmap_limit_check(length);
> +       ret = mmap(start, length, prot, flags, fd, offset);
> +       if (ret == MAP_FAILED && !length)
> +               ret = NULL;
> +       return ret;
> +}
> +
> +const char *mmap_os_err(void)
> +{
> +       static const char blank[] = "";
> +#if defined(__linux__)
> +       if (errno == ENOMEM) {
> +               /* this continues an existing error message: */
> +               static const char enomem[] =
> +", check sys.vm.max_map_count and/or RLIMIT_DATA";
> +               return enomem;
> +       }
> +#endif /* OS-specific bits */
> +       return blank;
> +}
> +
> +void *xmmap(void *start, size_t length,
> +       int prot, int flags, int fd, off_t offset)
> +{
> +       void *ret = xmmap_gently(start, length, prot, flags, fd, offset);
> +       if (ret == MAP_FAILED)
> +               die_errno(_("mmap failed%s"), mmap_os_err());
> +       return ret;
> +}
>
> --
> 2.49.0.682.gc9b6a7b2b0.dirty
>
>

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH 0/9] Split up "object-file.c"
  2025-04-08 10:24 [PATCH 0/9] Split up "object-file.c" Patrick Steinhardt
                   ` (9 preceding siblings ...)
  2025-04-08 23:29 ` [PATCH 0/9] Split up "object-file.c" Junio C Hamano
@ 2025-04-09 14:42 ` Elijah Newren
  2025-04-11  9:27   ` Patrick Steinhardt
  2025-04-11  9:29 ` [PATCH v2 " Patrick Steinhardt
  2025-04-15  9:38 ` [PATCH v3 00/10] Split up "object-file.c" Patrick Steinhardt
  12 siblings, 1 reply; 53+ messages in thread
From: Elijah Newren @ 2025-04-09 14:42 UTC (permalink / raw)
  To: Patrick Steinhardt; +Cc: git

On Tue, Apr 8, 2025 at 3:35 AM Patrick Steinhardt <ps@pks.im> wrote:
>
> Hi,
>
> "object-file.c" is quite a grab-bag of all kinds of different functions.
> Many of these functions aren't really a good fit though and should be
> owned by a different subsystem. This patch series tries to split up
> concerns a bit better by splitting out this functionality into other
> files:
>
>   - `safe_create_leading_directories()` is moved into "dir.c".
>   - `xmmap()` is moved into "wrapper.c".
>   - `git_open_cloexec()` is moved into "compat/open.c".
>   - Several functions attached to `struct index_state` are moved into
>     "read-cache.c".
>   - Several functions related to `struct object_store` are moved into a
>     new file "object-store.c".
>
> "object-file.c" now mostly contains logic to read and write loose object
> files, whereas "object-store.c" contains the higher-level logic to
> manage different object directories for a repository. Eventually, these
> will become the loose object backend as well as the `struct ref_store`
> equivalent for objects, respectively.
>
> The series is built on top of 9d22ac51228 (The third batch, 2025-04-07)
> with ps/object-wo-the-repository at 9442b1c919a (Merge remote-tracking
> branch 'junio/ps/object-wo-the-repository' into HEAD, 2025-04-08) merged
> into it.

I was able to apply with these instructions, and looking over the
series locally with --color-moved was very helpful to verify that you
were moving functions without modifying them (or, to notice where you
did modify them and why, e.g. removing the static declaration).

I very much appreciate the grouping of functions by area that is done
here with read-cache.c, wrapper.c, compat/open.c, dir.c, and the new
object-store.c, as well as the new enums to make the purpose of the
HASH_* flags clearer, and the reduction in the number of globals.

I only had a few comments on a couple of the patch commit messages.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH 0/9] Split up "object-file.c"
  2025-04-08 23:29 ` [PATCH 0/9] Split up "object-file.c" Junio C Hamano
@ 2025-04-11  9:26   ` Patrick Steinhardt
  0 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-11  9:26 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Tue, Apr 08, 2025 at 04:29:38PM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > The series is built on top of 9d22ac51228 (The third batch, 2025-04-07)
> > with ps/object-wo-the-repository at 9442b1c919a (Merge remote-tracking
> > branch 'junio/ps/object-wo-the-repository' into HEAD, 2025-04-08) merged
> > into it.
> 
> As 9442b1c919a is not public, this description lacks a way to verify
> the result I attempted to reproduce.  ps/object-wo-the-repository I
> have ends with 7d70b29c (hash: stop depending on `the_repository` in
> `null_oid()`, 2025-03-10) and it hasn't moved for a while, so
> hopefully we are in agreement ;-)

Ugh, yeah, I meant to refer to the tip of that branch indeed, which does
point to the commit you mention.

> There were a few unpleasant interactions with other topics in
> flight; please check the conflict resolution I made.

I couldn't spot anything wrong. Thanks!

Patrick

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH 1/9] object-file: move `safe_create_leading_directories()` into "dir.c"
  2025-04-09 14:36   ` Elijah Newren
@ 2025-04-11  9:27     ` Patrick Steinhardt
  2025-04-11 17:11       ` Elijah Newren
  0 siblings, 1 reply; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-11  9:27 UTC (permalink / raw)
  To: Elijah Newren; +Cc: git

On Wed, Apr 09, 2025 at 07:36:47AM -0700, Elijah Newren wrote:
> On Tue, Apr 8, 2025 at 3:37 AM Patrick Steinhardt <ps@pks.im> wrote:
> >
> > The `safe_create_leading_directories()` function and its relatives
> 
> How is mkdir_in_gitdir() a relative of safe_create_leading_directories()?
> 
> I assumed the relation was "called by", but there is no such
> relationship.  The rest of the patch looked fine, but I was puzzled
> for a while trying to figure out what this relationship is.

It's more of a sibling than a child/parent in this case, true. I still
think it makes sense to move it around as it is rather generic in the
functionality it provides and doesn't have anything to do with objects.

Patrick

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH 0/9] Split up "object-file.c"
  2025-04-09 14:42 ` Elijah Newren
@ 2025-04-11  9:27   ` Patrick Steinhardt
  0 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-11  9:27 UTC (permalink / raw)
  To: Elijah Newren; +Cc: git

On Wed, Apr 09, 2025 at 07:42:38AM -0700, Elijah Newren wrote:
> On Tue, Apr 8, 2025 at 3:35 AM Patrick Steinhardt <ps@pks.im> wrote:
> >
> > Hi,
> >
> > "object-file.c" is quite a grab-bag of all kinds of different functions.
> > Many of these functions aren't really a good fit though and should be
> > owned by a different subsystem. This patch series tries to split up
> > concerns a bit better by splitting out this functionality into other
> > files:
> >
> >   - `safe_create_leading_directories()` is moved into "dir.c".
> >   - `xmmap()` is moved into "wrapper.c".
> >   - `git_open_cloexec()` is moved into "compat/open.c".
> >   - Several functions attached to `struct index_state` are moved into
> >     "read-cache.c".
> >   - Several functions related to `struct object_store` are moved into a
> >     new file "object-store.c".
> >
> > "object-file.c" now mostly contains logic to read and write loose object
> > files, whereas "object-store.c" contains the higher-level logic to
> > manage different object directories for a repository. Eventually, these
> > will become the loose object backend as well as the `struct ref_store`
> > equivalent for objects, respectively.
> >
> > The series is built on top of 9d22ac51228 (The third batch, 2025-04-07)
> > with ps/object-wo-the-repository at 9442b1c919a (Merge remote-tracking
> > branch 'junio/ps/object-wo-the-repository' into HEAD, 2025-04-08) merged
> > into it.
> 
> I was able to apply with these instructions, and looking over the
> series locally with --color-moved was very helpful to verify that you
> were moving functions without modifying them (or, to notice where you
> did modify them and why, e.g. removing the static declaration).
> 
> I very much appreciate the grouping of functions by area that is done
> here with read-cache.c, wrapper.c, compat/open.c, dir.c, and the new
> object-store.c, as well as the new enums to make the purpose of the
> HASH_* flags clearer, and the reduction in the number of globals.
> 
> I only had a few comments on a couple of the patch commit messages.

Thanks for your review!

Patrick

^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH v2 0/9] Split up "object-file.c"
  2025-04-08 10:24 [PATCH 0/9] Split up "object-file.c" Patrick Steinhardt
                   ` (10 preceding siblings ...)
  2025-04-09 14:42 ` Elijah Newren
@ 2025-04-11  9:29 ` Patrick Steinhardt
  2025-04-11  9:29   ` [PATCH v2 1/9] object-file: move `safe_create_leading_directories()` into "dir.c" Patrick Steinhardt
                     ` (8 more replies)
  2025-04-15  9:38 ` [PATCH v3 00/10] Split up "object-file.c" Patrick Steinhardt
  12 siblings, 9 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-11  9:29 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Junio C Hamano

Hi,

"object-file.c" is quite a grab-bag of all kinds of different functions.
Many of these functions aren't really a good fit though and should be
owned by a different subsystem. This patch series tries to split up
concerns a bit better by splitting out this functionality into other
files:

  - `safe_create_leading_directories()` is moved into "dir.c".
  - `xmmap()` is moved into "wrapper.c".
  - `git_open_cloexec()` is moved into "compat/open.c".
  - Several functions attached to `struct index_state` are moved into
    "read-cache.c".
  - Several functions related to `struct object_store` are moved into a
    new file "object-store.c".

"object-file.c" now mostly contains logic to read and write loose object
files, whereas "object-store.c" contains the higher-level logic to
manage different object directories for a repository. Eventually, these
will become the loose object backend as well as the `struct ref_store`
equivalent for objects, respectively.

The series is built on top of 9d22ac51228 (The third batch, 2025-04-07)
with ps/object-wo-the-repository at 9442b1c919a (Merge remote-tracking
branch 'junio/ps/object-wo-the-repository' into HEAD, 2025-04-08) merged
into it.

Changes in v2:
  - Fix a grammar issue in one of the commit messages.
  - Link to v1: https://lore.kernel.org/r/20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im

Thanks!

Patrick

---
Patrick Steinhardt (9):
      object-file: move `safe_create_leading_directories()` into "dir.c"
      object-file: move `git_open_cloexec()` to "compat/open.c"
      object-file: move `xmmap()` into "wrapper.c"
      object-file: split out functions relating to object store subsystem
      object-file: split up concerns of `HASH_*` flags
      object-file: split out functions relating to index subsystem
      object: split out functions relating to object store subsystem
      object-store: remove global array of cached objects
      object-store: merge "object-store-ll.h" and "object-store.h"

 Makefile                           |    3 +-
 apply.c                            |    2 +-
 archive-tar.c                      |    2 +-
 archive-zip.c                      |    2 +-
 archive.c                          |    2 +-
 attr.c                             |    2 +-
 bisect.c                           |    2 +-
 blame.c                            |    4 +-
 builtin/backfill.c                 |    2 +-
 builtin/blame.c                    |    2 +-
 builtin/bugreport.c                |    2 +-
 builtin/cat-file.c                 |    2 +-
 builtin/checkout.c                 |    3 +-
 builtin/clone.c                    |    2 +-
 builtin/commit-graph.c             |    2 +-
 builtin/commit-tree.c              |    2 +-
 builtin/count-objects.c            |    2 +-
 builtin/credential-cache--daemon.c |    2 +-
 builtin/describe.c                 |    2 +-
 builtin/diagnose.c                 |    2 +-
 builtin/difftool.c                 |    4 +-
 builtin/fast-export.c              |    2 +-
 builtin/fast-import.c              |    2 +-
 builtin/fetch.c                    |    2 +-
 builtin/fsck.c                     |    3 +-
 builtin/gc.c                       |    4 +-
 builtin/grep.c                     |    2 +-
 builtin/hash-object.c              |   26 +-
 builtin/index-pack.c               |    2 +-
 builtin/init-db.c                  |    2 +-
 builtin/log.c                      |    4 +-
 builtin/ls-tree.c                  |    2 +-
 builtin/merge-file.c               |    1 +
 builtin/merge-tree.c               |    2 +-
 builtin/mktag.c                    |    2 +-
 builtin/mktree.c                   |    3 +-
 builtin/multi-pack-index.c         |    2 +-
 builtin/notes.c                    |    3 +-
 builtin/pack-objects.c             |    2 +-
 builtin/pack-redundant.c           |    2 +-
 builtin/prune.c                    |    2 +-
 builtin/receive-pack.c             |    3 +-
 builtin/remote.c                   |    2 +-
 builtin/repack.c                   |    2 +-
 builtin/replace.c                  |    5 +-
 builtin/rev-list.c                 |    2 +-
 builtin/show-ref.c                 |    2 +-
 builtin/submodule--helper.c        |    2 +-
 builtin/tag.c                      |    3 +-
 builtin/unpack-file.c              |    3 +-
 builtin/unpack-objects.c           |    3 +-
 builtin/update-index.c             |    2 +-
 bulk-checkin.c                     |    9 +-
 bundle-uri.c                       |    2 +-
 bundle.c                           |    2 +-
 cache-tree.c                       |    4 +-
 combine-diff.c                     |    2 +-
 commit-graph.c                     |    4 +-
 commit-graph.h                     |    2 +-
 commit.c                           |    3 +-
 compat/open.c                      |   29 +
 config.c                           |    2 +-
 connected.c                        |    2 +-
 convert.c                          |    2 +-
 diagnose.c                         |    2 +-
 diff.c                             |    4 +-
 diffcore-rename.c                  |    2 +-
 dir.c                              |  109 ++-
 dir.h                              |   35 +
 entry.c                            |    2 +-
 fetch-pack.c                       |    2 +-
 fmt-merge-msg.c                    |    2 +-
 fsck.c                             |    2 +-
 git-compat-util.h                  |    3 +
 grep.c                             |    2 +-
 http-backend.c                     |    2 +-
 http-push.c                        |    3 +-
 http-walker.c                      |    2 +-
 http.c                             |    2 +-
 list-objects-filter.c              |    2 +-
 list-objects.c                     |    2 +-
 log-tree.c                         |    2 +-
 mailmap.c                          |    2 +-
 match-trees.c                      |    3 +-
 merge-blobs.c                      |    2 +-
 merge-ort.c                        |    3 +-
 merge-recursive.c                  |    2 +-
 meson.build                        |    2 +
 midx-write.c                       |    1 +
 midx.c                             |    1 -
 notes-cache.c                      |    3 +-
 notes-merge.c                      |    5 +-
 notes.c                            |    3 +-
 object-file.c                      | 1420 +-----------------------------------
 object-file.h                      |  118 +--
 object-name.c                      |    2 +-
 object-store-ll.h                  |  556 --------------
 object-store.c                     | 1050 ++++++++++++++++++++++++++
 object-store.h                     |  516 ++++++++++++-
 object.c                           |   67 --
 oss-fuzz/fuzz-pack-idx.c           |    2 +-
 pack-bitmap-write.c                |    2 +-
 pack-bitmap.c                      |    3 +-
 pack-check.c                       |    2 +-
 pack-mtimes.c                      |    3 +-
 pack-objects.h                     |    2 +-
 pack-revindex.c                    |    3 +-
 packfile.c                         |    2 +-
 path.c                             |    2 +-
 promisor-remote.c                  |    2 +-
 protocol-caps.c                    |    2 +-
 prune-packed.c                     |    2 +-
 reachable.c                        |    2 +-
 read-cache.c                       |  230 +++++-
 read-cache.h                       |    9 +
 ref-filter.c                       |    2 +-
 reflog.c                           |    2 +-
 refs.c                             |    2 +-
 remote.c                           |    2 +-
 replace-object.c                   |    2 +-
 replace-object.h                   |    2 +-
 repository.c                       |    2 +-
 rerere.c                           |    2 +-
 revision.c                         |    2 +-
 send-pack.c                        |    2 +-
 sequencer.c                        |    2 +-
 server-info.c                      |    2 +-
 shallow.c                          |    2 +-
 streaming.c                        |    2 +-
 submodule-config.c                 |    2 +-
 submodule.c                        |    2 +-
 t/helper/test-pack-mtimes.c        |    2 +-
 t/helper/test-partial-clone.c      |    2 +-
 t/helper/test-read-graph.c         |    2 +-
 t/helper/test-read-midx.c          |    2 +-
 t/helper/test-ref-store.c          |    2 +-
 tag.c                              |    2 +-
 tmp-objdir.c                       |    2 +-
 tree-walk.c                        |    2 +-
 tree.c                             |    2 +-
 unpack-trees.c                     |    2 +-
 upload-pack.c                      |    2 +-
 walker.c                           |    2 +-
 wrapper.c                          |   48 ++
 xdiff-interface.c                  |    2 +-
 145 files changed, 2296 insertions(+), 2224 deletions(-)

Range-diff versus v1:

 1:  f1b7bd0550e =  1:  ad5f2af9380 object-file: move `safe_create_leading_directories()` into "dir.c"
 2:  859e33f9b6e =  2:  2ae1de46a37 object-file: move `git_open_cloexec()` to "compat/open.c"
 3:  f676828ab77 !  3:  18145eb4a7d object-file: move `xmmap()` into "wrapper.c"
    @@ Commit message
         object-file: move `xmmap()` into "wrapper.c"
     
         The `xmmap()` function is provided by "object-file.c" even though its
    -    functionality got nothing to do with the object file subsystem. Move it
    +    functionality has nothing to do with the object file subsystem. Move it
         into "wrapper.c", whose header already declares those functions.
     
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
 4:  1ed615206a5 =  4:  944883c3ad0 object-file: split out functions relating to object store subsystem
 5:  2d6354ebb65 =  5:  fed8a395b4f object-file: split up concerns of `HASH_*` flags
 6:  1d97b65731d =  6:  292c4d911ff object-file: split out functions relating to index subsystem
 7:  47fb869121a =  7:  478209b9109 object: split out functions relating to object store subsystem
 8:  bd8ce7dcc57 =  8:  e446ade793b object-store: remove global array of cached objects
 9:  69e337e58f0 =  9:  d2f0e4e52a1 object-store: merge "object-store-ll.h" and "object-store.h"

---
base-commit: 9442b1c919af9aed513eb0a484fe96358a500cf5
change-id: 20250408-pks-split-object-file-c61d7cd2a21f


^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH v2 1/9] object-file: move `safe_create_leading_directories()` into "dir.c"
  2025-04-11  9:29 ` [PATCH v2 " Patrick Steinhardt
@ 2025-04-11  9:29   ` Patrick Steinhardt
  2025-04-11 20:09     ` Junio C Hamano
  2025-04-11  9:29   ` [PATCH v2 2/9] object-file: move `git_open_cloexec()` to "compat/open.c" Patrick Steinhardt
                     ` (7 subsequent siblings)
  8 siblings, 1 reply; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-11  9:29 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Junio C Hamano

The `safe_create_leading_directories()` function and its relatives are
located in "object-file.c", which is not a good fit as they provide
generic functionality not related to objects at all. Move them into
"dir.c".

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/bugreport.c                |   2 +-
 builtin/credential-cache--daemon.c |   2 +-
 builtin/diagnose.c                 |   2 +-
 builtin/fsck.c                     |   1 +
 builtin/gc.c                       |   2 +-
 builtin/init-db.c                  |   2 +-
 builtin/log.c                      |   2 +-
 commit-graph.c                     |   1 +
 dir.c                              | 107 ++++++++++++++++++++++++++++++++++++-
 dir.h                              |  35 ++++++++++++
 midx-write.c                       |   1 +
 object-file.c                      | 106 ------------------------------------
 object-file.h                      |  35 ------------
 13 files changed, 150 insertions(+), 148 deletions(-)

diff --git a/builtin/bugreport.c b/builtin/bugreport.c
index 66d64bfd5ae..d07fa91c247 100644
--- a/builtin/bugreport.c
+++ b/builtin/bugreport.c
@@ -1,6 +1,7 @@
 #define USE_THE_REPOSITORY_VARIABLE
 #include "builtin.h"
 #include "abspath.h"
+#include "dir.h"
 #include "editor.h"
 #include "gettext.h"
 #include "parse-options.h"
@@ -10,7 +11,6 @@
 #include "hook.h"
 #include "hook-list.h"
 #include "diagnose.h"
-#include "object-file.h"
 #include "setup.h"
 #include "version.h"
 
diff --git a/builtin/credential-cache--daemon.c b/builtin/credential-cache--daemon.c
index e707618e743..80d29b4f5c0 100644
--- a/builtin/credential-cache--daemon.c
+++ b/builtin/credential-cache--daemon.c
@@ -1,8 +1,8 @@
 #define USE_THE_REPOSITORY_VARIABLE
 #include "builtin.h"
 #include "abspath.h"
+#include "dir.h"
 #include "gettext.h"
-#include "object-file.h"
 #include "parse-options.h"
 
 #ifndef NO_UNIX_SOCKETS
diff --git a/builtin/diagnose.c b/builtin/diagnose.c
index 33c39bd5981..d5dadd6a48b 100644
--- a/builtin/diagnose.c
+++ b/builtin/diagnose.c
@@ -2,8 +2,8 @@
 
 #include "builtin.h"
 #include "abspath.h"
+#include "dir.h"
 #include "gettext.h"
-#include "object-file.h"
 #include "parse-options.h"
 #include "diagnose.h"
 
diff --git a/builtin/fsck.c b/builtin/fsck.c
index 9c8a6d6a8df..32d40d8f9fc 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -1,5 +1,6 @@
 #define USE_THE_REPOSITORY_VARIABLE
 #include "builtin.h"
+#include "dir.h"
 #include "gettext.h"
 #include "hex.h"
 #include "config.h"
diff --git a/builtin/gc.c b/builtin/gc.c
index 99431fd4674..b069629676c 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -16,6 +16,7 @@
 #include "builtin.h"
 #include "abspath.h"
 #include "date.h"
+#include "dir.h"
 #include "environment.h"
 #include "hex.h"
 #include "config.h"
@@ -28,7 +29,6 @@
 #include "commit.h"
 #include "commit-graph.h"
 #include "packfile.h"
-#include "object-file.h"
 #include "object-store-ll.h"
 #include "pack.h"
 #include "pack-objects.h"
diff --git a/builtin/init-db.c b/builtin/init-db.c
index 196dccdd77a..39730c1b0ce 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -6,9 +6,9 @@
 #define USE_THE_REPOSITORY_VARIABLE
 #include "builtin.h"
 #include "abspath.h"
+#include "dir.h"
 #include "environment.h"
 #include "gettext.h"
-#include "object-file.h"
 #include "parse-options.h"
 #include "path.h"
 #include "refs.h"
diff --git a/builtin/log.c b/builtin/log.c
index 0d4c579dad7..06ffaa93e86 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -10,11 +10,11 @@
 #include "builtin.h"
 #include "abspath.h"
 #include "config.h"
+#include "dir.h"
 #include "environment.h"
 #include "gettext.h"
 #include "hex.h"
 #include "refs.h"
-#include "object-file.h"
 #include "object-name.h"
 #include "object-store-ll.h"
 #include "pager.h"
diff --git a/commit-graph.c b/commit-graph.c
index 8286d5dda24..3fae20dc21b 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -4,6 +4,7 @@
 #include "git-compat-util.h"
 #include "config.h"
 #include "csum-file.h"
+#include "dir.h"
 #include "gettext.h"
 #include "hex.h"
 #include "lockfile.h"
diff --git a/dir.c b/dir.c
index 28b0e03feb4..16ae3b5169d 100644
--- a/dir.c
+++ b/dir.c
@@ -17,7 +17,6 @@
 #include "environment.h"
 #include "gettext.h"
 #include "name-hash.h"
-#include "object-file.h"
 #include "object-store-ll.h"
 #include "path.h"
 #include "refs.h"
@@ -4132,3 +4131,109 @@ int path_match_flags(const char *const str, const enum path_match_flags flags)
 		return is_xplatform_dir_sep(*p);
 	BUG("unreachable");
 }
+
+int mkdir_in_gitdir(const char *path)
+{
+	if (mkdir(path, 0777)) {
+		int saved_errno = errno;
+		struct stat st;
+		struct strbuf sb = STRBUF_INIT;
+
+		if (errno != EEXIST)
+			return -1;
+		/*
+		 * Are we looking at a path in a symlinked worktree
+		 * whose original repository does not yet have it?
+		 * e.g. .git/rr-cache pointing at its original
+		 * repository in which the user hasn't performed any
+		 * conflict resolution yet?
+		 */
+		if (lstat(path, &st) || !S_ISLNK(st.st_mode) ||
+		    strbuf_readlink(&sb, path, st.st_size) ||
+		    !is_absolute_path(sb.buf) ||
+		    mkdir(sb.buf, 0777)) {
+			strbuf_release(&sb);
+			errno = saved_errno;
+			return -1;
+		}
+		strbuf_release(&sb);
+	}
+	return adjust_shared_perm(the_repository, path);
+}
+
+static enum scld_error safe_create_leading_directories_1(char *path, int share)
+{
+	char *next_component = path + offset_1st_component(path);
+	enum scld_error ret = SCLD_OK;
+
+	while (ret == SCLD_OK && next_component) {
+		struct stat st;
+		char *slash = next_component, slash_character;
+
+		while (*slash && !is_dir_sep(*slash))
+			slash++;
+
+		if (!*slash)
+			break;
+
+		next_component = slash + 1;
+		while (is_dir_sep(*next_component))
+			next_component++;
+		if (!*next_component)
+			break;
+
+		slash_character = *slash;
+		*slash = '\0';
+		if (!stat(path, &st)) {
+			/* path exists */
+			if (!S_ISDIR(st.st_mode)) {
+				errno = ENOTDIR;
+				ret = SCLD_EXISTS;
+			}
+		} else if (mkdir(path, 0777)) {
+			if (errno == EEXIST &&
+			    !stat(path, &st) && S_ISDIR(st.st_mode))
+				; /* somebody created it since we checked */
+			else if (errno == ENOENT)
+				/*
+				 * Either mkdir() failed because
+				 * somebody just pruned the containing
+				 * directory, or stat() failed because
+				 * the file that was in our way was
+				 * just removed.  Either way, inform
+				 * the caller that it might be worth
+				 * trying again:
+				 */
+				ret = SCLD_VANISHED;
+			else
+				ret = SCLD_FAILED;
+		} else if (share && adjust_shared_perm(the_repository, path)) {
+			ret = SCLD_PERMS;
+		}
+		*slash = slash_character;
+	}
+	return ret;
+}
+
+enum scld_error safe_create_leading_directories(char *path)
+{
+	return safe_create_leading_directories_1(path, 1);
+}
+
+enum scld_error safe_create_leading_directories_no_share(char *path)
+{
+	return safe_create_leading_directories_1(path, 0);
+}
+
+enum scld_error safe_create_leading_directories_const(const char *path)
+{
+	int save_errno;
+	/* path points to cache entries, so xstrdup before messing with it */
+	char *buf = xstrdup(path);
+	enum scld_error result = safe_create_leading_directories(buf);
+
+	save_errno = errno;
+	free(buf);
+	errno = save_errno;
+	return result;
+}
diff --git a/dir.h b/dir.h
index d7e71aa8daa..02c1f9420b0 100644
--- a/dir.h
+++ b/dir.h
@@ -676,4 +676,39 @@ static inline int starts_with_dot_dot_slash_native(const char *const path)
 	return path_match_flags(path, what | PATH_MATCH_NATIVE);
 }
 
+/*
+ * Create the directory containing the named path, using care to be
+ * somewhat safe against races. Return one of the scld_error values to
+ * indicate success/failure. On error, set errno to describe the
+ * problem.
+ *
+ * SCLD_VANISHED indicates that one of the ancestor directories of the
+ * path existed at one point during the function call and then
+ * suddenly vanished, probably because another process pruned the
+ * directory while we were working.  To be robust against this kind of
+ * race, callers might want to try invoking the function again when it
+ * returns SCLD_VANISHED.
+ *
+ * safe_create_leading_directories() temporarily changes path while it
+ * is working but restores it before returning.
+ * safe_create_leading_directories_const() doesn't modify path, even
+ * temporarily. Both these variants adjust the permissions of the
+ * created directories to honor core.sharedRepository, so they are best
+ * suited for files inside the git dir. For working tree files, use
+ * safe_create_leading_directories_no_share() instead, as it ignores
+ * the core.sharedRepository setting.
+ */
+enum scld_error {
+	SCLD_OK = 0,
+	SCLD_FAILED = -1,
+	SCLD_PERMS = -2,
+	SCLD_EXISTS = -3,
+	SCLD_VANISHED = -4
+};
+enum scld_error safe_create_leading_directories(char *path);
+enum scld_error safe_create_leading_directories_const(const char *path);
+enum scld_error safe_create_leading_directories_no_share(char *path);
+
+int mkdir_in_gitdir(const char *path);
+
 #endif
diff --git a/midx-write.c b/midx-write.c
index a628ac24dcb..e01a867c583 100644
--- a/midx-write.c
+++ b/midx-write.c
@@ -3,6 +3,7 @@
 #include "git-compat-util.h"
 #include "abspath.h"
 #include "config.h"
+#include "dir.h"
 #include "hex.h"
 #include "lockfile.h"
 #include "packfile.h"
diff --git a/object-file.c b/object-file.c
index 772c311f188..23b2c8560be 100644
--- a/object-file.c
+++ b/object-file.c
@@ -91,112 +91,6 @@ static int get_conv_flags(unsigned flags)
 }
 
 
-int mkdir_in_gitdir(const char *path)
-{
-	if (mkdir(path, 0777)) {
-		int saved_errno = errno;
-		struct stat st;
-		struct strbuf sb = STRBUF_INIT;
-
-		if (errno != EEXIST)
-			return -1;
-		/*
-		 * Are we looking at a path in a symlinked worktree
-		 * whose original repository does not yet have it?
-		 * e.g. .git/rr-cache pointing at its original
-		 * repository in which the user hasn't performed any
-		 * conflict resolution yet?
-		 */
-		if (lstat(path, &st) || !S_ISLNK(st.st_mode) ||
-		    strbuf_readlink(&sb, path, st.st_size) ||
-		    !is_absolute_path(sb.buf) ||
-		    mkdir(sb.buf, 0777)) {
-			strbuf_release(&sb);
-			errno = saved_errno;
-			return -1;
-		}
-		strbuf_release(&sb);
-	}
-	return adjust_shared_perm(the_repository, path);
-}
-
-static enum scld_error safe_create_leading_directories_1(char *path, int share)
-{
-	char *next_component = path + offset_1st_component(path);
-	enum scld_error ret = SCLD_OK;
-
-	while (ret == SCLD_OK && next_component) {
-		struct stat st;
-		char *slash = next_component, slash_character;
-
-		while (*slash && !is_dir_sep(*slash))
-			slash++;
-
-		if (!*slash)
-			break;
-
-		next_component = slash + 1;
-		while (is_dir_sep(*next_component))
-			next_component++;
-		if (!*next_component)
-			break;
-
-		slash_character = *slash;
-		*slash = '\0';
-		if (!stat(path, &st)) {
-			/* path exists */
-			if (!S_ISDIR(st.st_mode)) {
-				errno = ENOTDIR;
-				ret = SCLD_EXISTS;
-			}
-		} else if (mkdir(path, 0777)) {
-			if (errno == EEXIST &&
-			    !stat(path, &st) && S_ISDIR(st.st_mode))
-				; /* somebody created it since we checked */
-			else if (errno == ENOENT)
-				/*
-				 * Either mkdir() failed because
-				 * somebody just pruned the containing
-				 * directory, or stat() failed because
-				 * the file that was in our way was
-				 * just removed.  Either way, inform
-				 * the caller that it might be worth
-				 * trying again:
-				 */
-				ret = SCLD_VANISHED;
-			else
-				ret = SCLD_FAILED;
-		} else if (share && adjust_shared_perm(the_repository, path)) {
-			ret = SCLD_PERMS;
-		}
-		*slash = slash_character;
-	}
-	return ret;
-}
-
-enum scld_error safe_create_leading_directories(char *path)
-{
-	return safe_create_leading_directories_1(path, 1);
-}
-
-enum scld_error safe_create_leading_directories_no_share(char *path)
-{
-	return safe_create_leading_directories_1(path, 0);
-}
-
-enum scld_error safe_create_leading_directories_const(const char *path)
-{
-	int save_errno;
-	/* path points to cache entries, so xstrdup before messing with it */
-	char *buf = xstrdup(path);
-	enum scld_error result = safe_create_leading_directories(buf);
-
-	save_errno = errno;
-	free(buf);
-	errno = save_errno;
-	return result;
-}
-
 int odb_mkstemp(struct strbuf *temp_filename, const char *pattern)
 {
 	int fd;
diff --git a/object-file.h b/object-file.h
index 81b30d269c8..922f2bba8c9 100644
--- a/object-file.h
+++ b/object-file.h
@@ -21,41 +21,6 @@ extern int fetch_if_missing;
 int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);
 int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags);
 
-/*
- * Create the directory containing the named path, using care to be
- * somewhat safe against races. Return one of the scld_error values to
- * indicate success/failure. On error, set errno to describe the
- * problem.
- *
- * SCLD_VANISHED indicates that one of the ancestor directories of the
- * path existed at one point during the function call and then
- * suddenly vanished, probably because another process pruned the
- * directory while we were working.  To be robust against this kind of
- * race, callers might want to try invoking the function again when it
- * returns SCLD_VANISHED.
- *
- * safe_create_leading_directories() temporarily changes path while it
- * is working but restores it before returning.
- * safe_create_leading_directories_const() doesn't modify path, even
- * temporarily. Both these variants adjust the permissions of the
- * created directories to honor core.sharedRepository, so they are best
- * suited for files inside the git dir. For working tree files, use
- * safe_create_leading_directories_no_share() instead, as it ignores
- * the core.sharedRepository setting.
- */
-enum scld_error {
-	SCLD_OK = 0,
-	SCLD_FAILED = -1,
-	SCLD_PERMS = -2,
-	SCLD_EXISTS = -3,
-	SCLD_VANISHED = -4
-};
-enum scld_error safe_create_leading_directories(char *path);
-enum scld_error safe_create_leading_directories_const(const char *path);
-enum scld_error safe_create_leading_directories_no_share(char *path);
-
-int mkdir_in_gitdir(const char *path);
-
 int git_open_cloexec(const char *name, int flags);
 #define git_open(name) git_open_cloexec(name, O_RDONLY)
 

-- 
2.49.0.777.g153de2bbd5.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH v2 2/9] object-file: move `git_open_cloexec()` to "compat/open.c"
  2025-04-11  9:29 ` [PATCH v2 " Patrick Steinhardt
  2025-04-11  9:29   ` [PATCH v2 1/9] object-file: move `safe_create_leading_directories()` into "dir.c" Patrick Steinhardt
@ 2025-04-11  9:29   ` Patrick Steinhardt
  2025-04-11  9:29   ` [PATCH v2 3/9] object-file: move `xmmap()` into "wrapper.c" Patrick Steinhardt
                     ` (6 subsequent siblings)
  8 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-11  9:29 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Junio C Hamano

The `git_open_cloexec()` wrapper function provides the ability to open a
file with `O_CLOEXEC` in a platform-agnostic way. This function is
provided by "object-file.c" even though it is not specific to the object
subsystem at all.

Move the file into "compat/open.c". This file already exists before this
commit, but has only been compiled conditionally depending on whether or
not open(3p) may return EINTR. With this change we now unconditionally
compile the object, but wrap `git_open_with_retry()` in an ifdef.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Makefile          |  2 +-
 commit-graph.c    |  1 -
 compat/open.c     | 29 +++++++++++++++++++++++++++++
 git-compat-util.h |  3 +++
 meson.build       |  1 +
 midx.c            |  1 -
 object-file.c     | 27 ---------------------------
 object-file.h     |  3 ---
 pack-bitmap.c     |  1 -
 pack-mtimes.c     |  1 -
 pack-revindex.c   |  1 -
 11 files changed, 34 insertions(+), 36 deletions(-)

diff --git a/Makefile b/Makefile
index c41fc41ef0e..bb5407b4703 100644
--- a/Makefile
+++ b/Makefile
@@ -994,6 +994,7 @@ LIB_OBJS += common-exit.o
 LIB_OBJS += common-init.o
 LIB_OBJS += compat/nonblock.o
 LIB_OBJS += compat/obstack.o
+LIB_OBJS += compat/open.o
 LIB_OBJS += compat/terminal.o
 LIB_OBJS += compiler-tricks/not-constant.o
 LIB_OBJS += config.o
@@ -1812,7 +1813,6 @@ ifdef FREAD_READS_DIRECTORIES
 endif
 ifdef OPEN_RETURNS_EINTR
 	COMPAT_CFLAGS += -DOPEN_RETURNS_EINTR
-	COMPAT_OBJS += compat/open.o
 endif
 ifdef NO_SYMLINK_HEAD
 	BASIC_CFLAGS += -DNO_SYMLINK_HEAD
diff --git a/commit-graph.c b/commit-graph.c
index 3fae20dc21b..8060c358b84 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -14,7 +14,6 @@
 #include "refs.h"
 #include "hash-lookup.h"
 #include "commit-graph.h"
-#include "object-file.h"
 #include "object-store-ll.h"
 #include "oid-array.h"
 #include "path.h"
diff --git a/compat/open.c b/compat/open.c
index eb3754a23b8..37ae2b1aeb9 100644
--- a/compat/open.c
+++ b/compat/open.c
@@ -1,5 +1,6 @@
 #include "git-compat-util.h"
 
+#ifdef OPEN_RETURNS_EINTR
 #undef open
 int git_open_with_retry(const char *path, int flags, ...)
 {
@@ -23,3 +24,31 @@ int git_open_with_retry(const char *path, int flags, ...)
 
 	return ret;
 }
+#endif
+
+int git_open_cloexec(const char *name, int flags)
+{
+	int fd;
+	static int o_cloexec = O_CLOEXEC;
+
+	fd = open(name, flags | o_cloexec);
+	if ((o_cloexec & O_CLOEXEC) && fd < 0 && errno == EINVAL) {
+		/* Try again w/o O_CLOEXEC: the kernel might not support it */
+		o_cloexec &= ~O_CLOEXEC;
+		fd = open(name, flags | o_cloexec);
+	}
+
+#if defined(F_GETFD) && defined(F_SETFD) && defined(FD_CLOEXEC)
+	{
+		static int fd_cloexec = FD_CLOEXEC;
+
+		if (!o_cloexec && 0 <= fd && fd_cloexec) {
+			/* Opened w/o O_CLOEXEC?  try with fcntl(2) to add it */
+			int flags = fcntl(fd, F_GETFD);
+			if (fcntl(fd, F_SETFD, flags | fd_cloexec))
+				fd_cloexec = 0;
+		}
+	}
+#endif
+	return fd;
+}
diff --git a/git-compat-util.h b/git-compat-util.h
index cf733b38acd..9273a8ee087 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -1000,6 +1000,9 @@ int git_vsnprintf(char *str, size_t maxsize,
 int git_open_with_retry(const char *path, int flag, ...);
 #endif
 
+int git_open_cloexec(const char *name, int flags);
+#define git_open(name) git_open_cloexec(name, O_RDONLY)
+
 #ifdef __GLIBC_PREREQ
 #if __GLIBC_PREREQ(2, 1)
 #define HAVE_STRCHRNUL
diff --git a/meson.build b/meson.build
index 145d2f7ff9e..a55e800b85b 100644
--- a/meson.build
+++ b/meson.build
@@ -263,6 +263,7 @@ libgit_sources = [
   'common-init.c',
   'compat/nonblock.c',
   'compat/obstack.c',
+  'compat/open.c',
   'compat/terminal.c',
   'compiler-tricks/not-constant.c',
   'config.c',
diff --git a/midx.c b/midx.c
index 807fdf72f7b..3d0015f7828 100644
--- a/midx.c
+++ b/midx.c
@@ -5,7 +5,6 @@
 #include "dir.h"
 #include "hex.h"
 #include "packfile.h"
-#include "object-file.h"
 #include "hash-lookup.h"
 #include "midx.h"
 #include "progress.h"
diff --git a/object-file.c b/object-file.c
index 23b2c8560be..1a20c7fa072 100644
--- a/object-file.c
+++ b/object-file.c
@@ -834,33 +834,6 @@ int stream_object_signature(struct repository *r, const struct object_id *oid)
 	return !oideq(oid, &real_oid) ? -1 : 0;
 }
 
-int git_open_cloexec(const char *name, int flags)
-{
-	int fd;
-	static int o_cloexec = O_CLOEXEC;
-
-	fd = open(name, flags | o_cloexec);
-	if ((o_cloexec & O_CLOEXEC) && fd < 0 && errno == EINVAL) {
-		/* Try again w/o O_CLOEXEC: the kernel might not support it */
-		o_cloexec &= ~O_CLOEXEC;
-		fd = open(name, flags | o_cloexec);
-	}
-
-#if defined(F_GETFD) && defined(F_SETFD) && defined(FD_CLOEXEC)
-	{
-		static int fd_cloexec = FD_CLOEXEC;
-
-		if (!o_cloexec && 0 <= fd && fd_cloexec) {
-			/* Opened w/o O_CLOEXEC?  try with fcntl(2) to add it */
-			int flags = fcntl(fd, F_GETFD);
-			if (fcntl(fd, F_SETFD, flags | fd_cloexec))
-				fd_cloexec = 0;
-		}
-	}
-#endif
-	return fd;
-}
-
 /*
  * Find "oid" as a loose object in the local repository or in an alternate.
  * Returns 0 on success, negative on failure.
diff --git a/object-file.h b/object-file.h
index 922f2bba8c9..353d8a85c33 100644
--- a/object-file.h
+++ b/object-file.h
@@ -21,9 +21,6 @@ extern int fetch_if_missing;
 int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);
 int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags);
 
-int git_open_cloexec(const char *name, int flags);
-#define git_open(name) git_open_cloexec(name, O_RDONLY)
-
 /**
  * unpack_loose_header() initializes the data stream needed to unpack
  * a loose object header.
diff --git a/pack-bitmap.c b/pack-bitmap.c
index 7fd78c634ef..0dbd7c4ffe1 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -17,7 +17,6 @@
 #include "packfile.h"
 #include "repository.h"
 #include "trace2.h"
-#include "object-file.h"
 #include "object-store-ll.h"
 #include "list-objects-filter-options.h"
 #include "midx.h"
diff --git a/pack-mtimes.c b/pack-mtimes.c
index cdf30b8d2b0..bcea28e521d 100644
--- a/pack-mtimes.c
+++ b/pack-mtimes.c
@@ -1,7 +1,6 @@
 #include "git-compat-util.h"
 #include "gettext.h"
 #include "pack-mtimes.h"
-#include "object-file.h"
 #include "object-store-ll.h"
 #include "packfile.h"
 #include "strbuf.h"
diff --git a/pack-revindex.c b/pack-revindex.c
index 038e0c96b1c..1ee7b49e206 100644
--- a/pack-revindex.c
+++ b/pack-revindex.c
@@ -1,7 +1,6 @@
 #include "git-compat-util.h"
 #include "gettext.h"
 #include "pack-revindex.h"
-#include "object-file.h"
 #include "object-store-ll.h"
 #include "packfile.h"
 #include "strbuf.h"

-- 
2.49.0.777.g153de2bbd5.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH v2 3/9] object-file: move `xmmap()` into "wrapper.c"
  2025-04-11  9:29 ` [PATCH v2 " Patrick Steinhardt
  2025-04-11  9:29   ` [PATCH v2 1/9] object-file: move `safe_create_leading_directories()` into "dir.c" Patrick Steinhardt
  2025-04-11  9:29   ` [PATCH v2 2/9] object-file: move `git_open_cloexec()` to "compat/open.c" Patrick Steinhardt
@ 2025-04-11  9:29   ` Patrick Steinhardt
  2025-04-11  9:29   ` [PATCH v2 4/9] object-file: split out functions relating to object store subsystem Patrick Steinhardt
                     ` (5 subsequent siblings)
  8 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-11  9:29 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Junio C Hamano

The `xmmap()` function is provided by "object-file.c" even though its
functionality has nothing to do with the object file subsystem. Move it
into "wrapper.c", whose header already declares those functions.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 object-file.c | 48 ------------------------------------------------
 wrapper.c     | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 48 insertions(+), 48 deletions(-)

diff --git a/object-file.c b/object-file.c
index 1a20c7fa072..ea2ed7628e6 100644
--- a/object-file.c
+++ b/object-file.c
@@ -719,54 +719,6 @@ int has_loose_object(const struct object_id *oid)
 	return check_and_freshen(oid, 0);
 }
 
-static void mmap_limit_check(size_t length)
-{
-	static size_t limit = 0;
-	if (!limit) {
-		limit = git_env_ulong("GIT_MMAP_LIMIT", 0);
-		if (!limit)
-			limit = SIZE_MAX;
-	}
-	if (length > limit)
-		die(_("attempting to mmap %"PRIuMAX" over limit %"PRIuMAX),
-		    (uintmax_t)length, (uintmax_t)limit);
-}
-
-void *xmmap_gently(void *start, size_t length,
-		  int prot, int flags, int fd, off_t offset)
-{
-	void *ret;
-
-	mmap_limit_check(length);
-	ret = mmap(start, length, prot, flags, fd, offset);
-	if (ret == MAP_FAILED && !length)
-		ret = NULL;
-	return ret;
-}
-
-const char *mmap_os_err(void)
-{
-	static const char blank[] = "";
-#if defined(__linux__)
-	if (errno == ENOMEM) {
-		/* this continues an existing error message: */
-		static const char enomem[] =
-", check sys.vm.max_map_count and/or RLIMIT_DATA";
-		return enomem;
-	}
-#endif /* OS-specific bits */
-	return blank;
-}
-
-void *xmmap(void *start, size_t length,
-	int prot, int flags, int fd, off_t offset)
-{
-	void *ret = xmmap_gently(start, length, prot, flags, fd, offset);
-	if (ret == MAP_FAILED)
-		die_errno(_("mmap failed%s"), mmap_os_err());
-	return ret;
-}
-
 static int format_object_header_literally(char *str, size_t size,
 					  const char *type, size_t objsize)
 {
diff --git a/wrapper.c b/wrapper.c
index 8b985931490..3c79778055e 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -829,3 +829,51 @@ uint32_t git_rand(unsigned flags)
 
 	return result;
 }
+
+static void mmap_limit_check(size_t length)
+{
+	static size_t limit = 0;
+	if (!limit) {
+		limit = git_env_ulong("GIT_MMAP_LIMIT", 0);
+		if (!limit)
+			limit = SIZE_MAX;
+	}
+	if (length > limit)
+		die(_("attempting to mmap %"PRIuMAX" over limit %"PRIuMAX),
+		    (uintmax_t)length, (uintmax_t)limit);
+}
+
+void *xmmap_gently(void *start, size_t length,
+		  int prot, int flags, int fd, off_t offset)
+{
+	void *ret;
+
+	mmap_limit_check(length);
+	ret = mmap(start, length, prot, flags, fd, offset);
+	if (ret == MAP_FAILED && !length)
+		ret = NULL;
+	return ret;
+}
+
+const char *mmap_os_err(void)
+{
+	static const char blank[] = "";
+#if defined(__linux__)
+	if (errno == ENOMEM) {
+		/* this continues an existing error message: */
+		static const char enomem[] =
+", check sys.vm.max_map_count and/or RLIMIT_DATA";
+		return enomem;
+	}
+#endif /* OS-specific bits */
+	return blank;
+}
+
+void *xmmap(void *start, size_t length,
+	int prot, int flags, int fd, off_t offset)
+{
+	void *ret = xmmap_gently(start, length, prot, flags, fd, offset);
+	if (ret == MAP_FAILED)
+		die_errno(_("mmap failed%s"), mmap_os_err());
+	return ret;
+}

-- 
2.49.0.777.g153de2bbd5.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH v2 4/9] object-file: split out functions relating to object store subsystem
  2025-04-11  9:29 ` [PATCH v2 " Patrick Steinhardt
                     ` (2 preceding siblings ...)
  2025-04-11  9:29   ` [PATCH v2 3/9] object-file: move `xmmap()` into "wrapper.c" Patrick Steinhardt
@ 2025-04-11  9:29   ` Patrick Steinhardt
  2025-04-11  9:29   ` [PATCH v2 5/9] object-file: split up concerns of `HASH_*` flags Patrick Steinhardt
                     ` (4 subsequent siblings)
  8 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-11  9:29 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Junio C Hamano

While we have the "object-store.h" header, most of the functionality for
object stores is actually hosted in "object-file.c". This makes it hard
to find relevant functions and causes us to mix up concerns.

Split out functions relating to the object store subsystem into a new
"object-store.c" file.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Makefile                 |   1 +
 builtin/checkout.c       |   1 +
 builtin/merge-file.c     |   1 +
 builtin/mktree.c         |   1 +
 builtin/notes.c          |   1 +
 builtin/receive-pack.c   |   1 +
 builtin/tag.c            |   1 +
 builtin/unpack-file.c    |   1 +
 builtin/unpack-objects.c |   1 +
 commit.c                 |   1 +
 http-push.c              |   1 +
 match-trees.c            |   3 +-
 merge-ort.c              |   3 +-
 meson.build              |   1 +
 notes-cache.c            |   3 +-
 notes.c                  |   3 +-
 object-file.c            | 991 +----------------------------------------------
 object-file.h            |  68 +++-
 object-store-ll.h        |  60 +--
 object-store.c           | 972 ++++++++++++++++++++++++++++++++++++++++++++++
 20 files changed, 1074 insertions(+), 1041 deletions(-)

diff --git a/Makefile b/Makefile
index bb5407b4703..d8ad76fb791 100644
--- a/Makefile
+++ b/Makefile
@@ -1086,6 +1086,7 @@ LIB_OBJS += notes.o
 LIB_OBJS += object-file-convert.o
 LIB_OBJS += object-file.o
 LIB_OBJS += object-name.o
+LIB_OBJS += object-store.o
 LIB_OBJS += object.o
 LIB_OBJS += oid-array.o
 LIB_OBJS += oidmap.o
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 2e7486cf658..3e68623838a 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -18,6 +18,7 @@
 #include "lockfile.h"
 #include "mem-pool.h"
 #include "merge-recursive.h"
+#include "object-file.h"
 #include "object-name.h"
 #include "object-store-ll.h"
 #include "parse-options.h"
diff --git a/builtin/merge-file.c b/builtin/merge-file.c
index 7e315f374b2..2b16b10d2ca 100644
--- a/builtin/merge-file.c
+++ b/builtin/merge-file.c
@@ -5,6 +5,7 @@
 #include "abspath.h"
 #include "diff.h"
 #include "hex.h"
+#include "object-file.h"
 #include "object-name.h"
 #include "object-store.h"
 #include "config.h"
diff --git a/builtin/mktree.c b/builtin/mktree.c
index 3c16faa40e3..0644f951161 100644
--- a/builtin/mktree.c
+++ b/builtin/mktree.c
@@ -11,6 +11,7 @@
 #include "strbuf.h"
 #include "tree.h"
 #include "parse-options.h"
+#include "object-file.h"
 #include "object-store-ll.h"
 
 static struct treeent {
diff --git a/builtin/notes.c b/builtin/notes.c
index ff61ec5f2da..0dbc233752d 100644
--- a/builtin/notes.c
+++ b/builtin/notes.c
@@ -14,6 +14,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "notes.h"
+#include "object-file.h"
 #include "object-name.h"
 #include "object-store-ll.h"
 #include "path.h"
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index b3e2a9d0c60..ee51bd76f60 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -31,6 +31,7 @@
 #include "tmp-objdir.h"
 #include "oidset.h"
 #include "packfile.h"
+#include "object-file.h"
 #include "object-name.h"
 #include "object-store-ll.h"
 #include "path.h"
diff --git a/builtin/tag.c b/builtin/tag.c
index 7c173535cb3..536a01ff3ae 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -17,6 +17,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "refs.h"
+#include "object-file.h"
 #include "object-name.h"
 #include "object-store-ll.h"
 #include "path.h"
diff --git a/builtin/unpack-file.c b/builtin/unpack-file.c
index fb5fcbc40a8..b19e5cabd03 100644
--- a/builtin/unpack-file.c
+++ b/builtin/unpack-file.c
@@ -2,6 +2,7 @@
 #include "builtin.h"
 #include "config.h"
 #include "hex.h"
+#include "object-file.h"
 #include "object-name.h"
 #include "object-store-ll.h"
 
diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c
index 3bbcaf2de9b..4078eab9252 100644
--- a/builtin/unpack-objects.c
+++ b/builtin/unpack-objects.c
@@ -8,6 +8,7 @@
 #include "gettext.h"
 #include "git-zlib.h"
 #include "hex.h"
+#include "object-file.h"
 #include "object-store-ll.h"
 #include "object.h"
 #include "delta.h"
diff --git a/commit.c b/commit.c
index 48aeefaad31..fbf4f8e87fd 100644
--- a/commit.c
+++ b/commit.c
@@ -29,6 +29,7 @@
 #include "tree.h"
 #include "hook.h"
 #include "parse.h"
+#include "object-file.h"
 #include "object-file-convert.h"
 
 static struct commit_extra_header *read_commit_extra_header_lines(const char *buf, size_t len, const char **);
diff --git a/http-push.c b/http-push.c
index 1b030d96f48..806eb67cf1b 100644
--- a/http-push.c
+++ b/http-push.c
@@ -19,6 +19,7 @@
 #include "tree-walk.h"
 #include "url.h"
 #include "packfile.h"
+#include "object-file.h"
 #include "object-store-ll.h"
 #include "commit-reach.h"
 
diff --git a/match-trees.c b/match-trees.c
index ef14ceb594c..72922d5d64e 100644
--- a/match-trees.c
+++ b/match-trees.c
@@ -6,7 +6,8 @@
 #include "strbuf.h"
 #include "tree.h"
 #include "tree-walk.h"
-#include "object-store-ll.h"
+#include "object-file.h"
+#include "object-store.h"
 #include "repository.h"
 
 static int score_missing(unsigned mode)
diff --git a/merge-ort.c b/merge-ort.c
index 1554900d984..5de3e2fc159 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -36,8 +36,9 @@
 #include "merge-ll.h"
 #include "match-trees.h"
 #include "mem-pool.h"
+#include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "path.h"
 #include "promisor-remote.h"
diff --git a/meson.build b/meson.build
index a55e800b85b..f656f978020 100644
--- a/meson.build
+++ b/meson.build
@@ -355,6 +355,7 @@ libgit_sources = [
   'object-file-convert.c',
   'object-file.c',
   'object-name.c',
+  'object-store.c',
   'object.c',
   'oid-array.c',
   'oidmap.c',
diff --git a/notes-cache.c b/notes-cache.c
index ecfdf6e43b5..150241b15e0 100644
--- a/notes-cache.c
+++ b/notes-cache.c
@@ -2,7 +2,8 @@
 
 #include "git-compat-util.h"
 #include "notes-cache.h"
-#include "object-store-ll.h"
+#include "object-file.h"
+#include "object-store.h"
 #include "pretty.h"
 #include "repository.h"
 #include "commit.h"
diff --git a/notes.c b/notes.c
index ce5a1006a83..d9645c4b5dc 100644
--- a/notes.c
+++ b/notes.c
@@ -6,8 +6,9 @@
 #include "environment.h"
 #include "hex.h"
 #include "notes.h"
+#include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "utf8.h"
 #include "strbuf.h"
 #include "tree-walk.h"
diff --git a/object-file.c b/object-file.c
index ea2ed7628e6..1c74aa239fb 100644
--- a/object-file.c
+++ b/object-file.c
@@ -11,75 +11,26 @@
 #define DISABLE_SIGN_COMPARE_WARNINGS
 
 #include "git-compat-util.h"
-#include "abspath.h"
-#include "config.h"
+#include "bulk-checkin.h"
 #include "convert.h"
 #include "environment.h"
+#include "fsck.h"
 #include "gettext.h"
 #include "hex.h"
-#include "string-list.h"
-#include "lockfile.h"
-#include "pack.h"
-#include "commit.h"
-#include "run-command.h"
-#include "refs.h"
-#include "bulk-checkin.h"
-#include "repository.h"
-#include "replace-object.h"
-#include "streaming.h"
-#include "dir.h"
-#include "list.h"
-#include "quote.h"
-#include "packfile.h"
+#include "loose.h"
+#include "object-file-convert.h"
 #include "object-file.h"
 #include "object-store.h"
 #include "oidtree.h"
+#include "pack.h"
+#include "packfile.h"
 #include "path.h"
-#include "promisor-remote.h"
 #include "setup.h"
-#include "submodule.h"
-#include "fsck.h"
-#include "loose.h"
-#include "object-file-convert.h"
+#include "streaming.h"
 
 /* The maximum size for an object header. */
 #define MAX_HEADER_LEN 32
 
-/*
- * This is meant to hold a *small* number of objects that you would
- * want repo_read_object_file() to be able to return, but yet you do not want
- * to write them into the object store (e.g. a browse-only
- * application).
- */
-static struct cached_object_entry {
-	struct object_id oid;
-	struct cached_object {
-		enum object_type type;
-		const void *buf;
-		unsigned long size;
-	} value;
-} *cached_objects;
-static int cached_object_nr, cached_object_alloc;
-
-static const struct cached_object *find_cached_object(const struct object_id *oid)
-{
-	static const struct cached_object empty_tree = {
-		.type = OBJ_TREE,
-		.buf = "",
-	};
-	int i;
-	const struct cached_object_entry *co = cached_objects;
-
-	for (i = 0; i < cached_object_nr; i++, co++) {
-		if (oideq(&co->oid, oid))
-			return &co->value;
-	}
-	if (oideq(oid, the_hash_algo->empty_tree))
-		return &empty_tree;
-	return NULL;
-}
-
-
 static int get_conv_flags(unsigned flags)
 {
 	if (flags & HASH_RENORMALIZE)
@@ -90,40 +41,6 @@ static int get_conv_flags(unsigned flags)
 		return 0;
 }
 
-
-int odb_mkstemp(struct strbuf *temp_filename, const char *pattern)
-{
-	int fd;
-	/*
-	 * we let the umask do its job, don't try to be more
-	 * restrictive except to remove write permission.
-	 */
-	int mode = 0444;
-	repo_git_path_replace(the_repository, temp_filename, "objects/%s", pattern);
-	fd = git_mkstemp_mode(temp_filename->buf, mode);
-	if (0 <= fd)
-		return fd;
-
-	/* slow path */
-	/* some mkstemp implementations erase temp_filename on failure */
-	repo_git_path_replace(the_repository, temp_filename, "objects/%s", pattern);
-	safe_create_leading_directories(temp_filename->buf);
-	return xmkstemp_mode(temp_filename->buf, mode);
-}
-
-int odb_pack_keep(const char *name)
-{
-	int fd;
-
-	fd = open(name, O_RDWR|O_CREAT|O_EXCL, 0600);
-	if (0 <= fd)
-		return fd;
-
-	/* slow path */
-	safe_create_leading_directories_const(name);
-	return open(name, O_RDWR|O_CREAT|O_EXCL, 0600);
-}
-
 static void fill_loose_path(struct strbuf *buf, const struct object_id *oid)
 {
 	int i;
@@ -137,9 +54,9 @@ static void fill_loose_path(struct strbuf *buf, const struct object_id *oid)
 	}
 }
 
-static const char *odb_loose_path(struct object_directory *odb,
-				  struct strbuf *buf,
-				  const struct object_id *oid)
+const char *odb_loose_path(struct object_directory *odb,
+			   struct strbuf *buf,
+			   const struct object_id *oid)
 {
 	strbuf_reset(buf);
 	strbuf_addstr(buf, odb->path);
@@ -148,513 +65,6 @@ static const char *odb_loose_path(struct object_directory *odb,
 	return buf->buf;
 }
 
-const char *loose_object_path(struct repository *r, struct strbuf *buf,
-			      const struct object_id *oid)
-{
-	return odb_loose_path(r->objects->odb, buf, oid);
-}
-
-/*
- * Return non-zero iff the path is usable as an alternate object database.
- */
-static int alt_odb_usable(struct raw_object_store *o,
-			  struct strbuf *path,
-			  const char *normalized_objdir, khiter_t *pos)
-{
-	int r;
-
-	/* Detect cases where alternate disappeared */
-	if (!is_directory(path->buf)) {
-		error(_("object directory %s does not exist; "
-			"check .git/objects/info/alternates"),
-		      path->buf);
-		return 0;
-	}
-
-	/*
-	 * Prevent the common mistake of listing the same
-	 * thing twice, or object directory itself.
-	 */
-	if (!o->odb_by_path) {
-		khiter_t p;
-
-		o->odb_by_path = kh_init_odb_path_map();
-		assert(!o->odb->next);
-		p = kh_put_odb_path_map(o->odb_by_path, o->odb->path, &r);
-		assert(r == 1); /* never used */
-		kh_value(o->odb_by_path, p) = o->odb;
-	}
-	if (fspatheq(path->buf, normalized_objdir))
-		return 0;
-	*pos = kh_put_odb_path_map(o->odb_by_path, path->buf, &r);
-	/* r: 0 = exists, 1 = never used, 2 = deleted */
-	return r == 0 ? 0 : 1;
-}
-
-/*
- * Prepare alternate object database registry.
- *
- * The variable alt_odb_list points at the list of struct
- * object_directory.  The elements on this list come from
- * non-empty elements from colon separated ALTERNATE_DB_ENVIRONMENT
- * environment variable, and $GIT_OBJECT_DIRECTORY/info/alternates,
- * whose contents is similar to that environment variable but can be
- * LF separated.  Its base points at a statically allocated buffer that
- * contains "/the/directory/corresponding/to/.git/objects/...", while
- * its name points just after the slash at the end of ".git/objects/"
- * in the example above, and has enough space to hold all hex characters
- * of the object ID, an extra slash for the first level indirection, and
- * the terminating NUL.
- */
-static void read_info_alternates(struct repository *r,
-				 const char *relative_base,
-				 int depth);
-static int link_alt_odb_entry(struct repository *r, const struct strbuf *entry,
-	const char *relative_base, int depth, const char *normalized_objdir)
-{
-	struct object_directory *ent;
-	struct strbuf pathbuf = STRBUF_INIT;
-	struct strbuf tmp = STRBUF_INIT;
-	khiter_t pos;
-	int ret = -1;
-
-	if (!is_absolute_path(entry->buf) && relative_base) {
-		strbuf_realpath(&pathbuf, relative_base, 1);
-		strbuf_addch(&pathbuf, '/');
-	}
-	strbuf_addbuf(&pathbuf, entry);
-
-	if (!strbuf_realpath(&tmp, pathbuf.buf, 0)) {
-		error(_("unable to normalize alternate object path: %s"),
-		      pathbuf.buf);
-		goto error;
-	}
-	strbuf_swap(&pathbuf, &tmp);
-
-	/*
-	 * The trailing slash after the directory name is given by
-	 * this function at the end. Remove duplicates.
-	 */
-	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
-		strbuf_setlen(&pathbuf, pathbuf.len - 1);
-
-	if (!alt_odb_usable(r->objects, &pathbuf, normalized_objdir, &pos))
-		goto error;
-
-	CALLOC_ARRAY(ent, 1);
-	/* pathbuf.buf is already in r->objects->odb_by_path */
-	ent->path = strbuf_detach(&pathbuf, NULL);
-
-	/* add the alternate entry */
-	*r->objects->odb_tail = ent;
-	r->objects->odb_tail = &(ent->next);
-	ent->next = NULL;
-	assert(r->objects->odb_by_path);
-	kh_value(r->objects->odb_by_path, pos) = ent;
-
-	/* recursively add alternates */
-	read_info_alternates(r, ent->path, depth + 1);
-	ret = 0;
- error:
-	strbuf_release(&tmp);
-	strbuf_release(&pathbuf);
-	return ret;
-}
-
-static const char *parse_alt_odb_entry(const char *string,
-				       int sep,
-				       struct strbuf *out)
-{
-	const char *end;
-
-	strbuf_reset(out);
-
-	if (*string == '#') {
-		/* comment; consume up to next separator */
-		end = strchrnul(string, sep);
-	} else if (*string == '"' && !unquote_c_style(out, string, &end)) {
-		/*
-		 * quoted path; unquote_c_style has copied the
-		 * data for us and set "end". Broken quoting (e.g.,
-		 * an entry that doesn't end with a quote) falls
-		 * back to the unquoted case below.
-		 */
-	} else {
-		/* normal, unquoted path */
-		end = strchrnul(string, sep);
-		strbuf_add(out, string, end - string);
-	}
-
-	if (*end)
-		end++;
-	return end;
-}
-
-static void link_alt_odb_entries(struct repository *r, const char *alt,
-				 int sep, const char *relative_base, int depth)
-{
-	struct strbuf objdirbuf = STRBUF_INIT;
-	struct strbuf entry = STRBUF_INIT;
-
-	if (!alt || !*alt)
-		return;
-
-	if (depth > 5) {
-		error(_("%s: ignoring alternate object stores, nesting too deep"),
-				relative_base);
-		return;
-	}
-
-	strbuf_realpath(&objdirbuf, r->objects->odb->path, 1);
-
-	while (*alt) {
-		alt = parse_alt_odb_entry(alt, sep, &entry);
-		if (!entry.len)
-			continue;
-		link_alt_odb_entry(r, &entry,
-				   relative_base, depth, objdirbuf.buf);
-	}
-	strbuf_release(&entry);
-	strbuf_release(&objdirbuf);
-}
-
-static void read_info_alternates(struct repository *r,
-				 const char *relative_base,
-				 int depth)
-{
-	char *path;
-	struct strbuf buf = STRBUF_INIT;
-
-	path = xstrfmt("%s/info/alternates", relative_base);
-	if (strbuf_read_file(&buf, path, 1024) < 0) {
-		warn_on_fopen_errors(path);
-		free(path);
-		return;
-	}
-
-	link_alt_odb_entries(r, buf.buf, '\n', relative_base, depth);
-	strbuf_release(&buf);
-	free(path);
-}
-
-void add_to_alternates_file(const char *reference)
-{
-	struct lock_file lock = LOCK_INIT;
-	char *alts = repo_git_path(the_repository, "objects/info/alternates");
-	FILE *in, *out;
-	int found = 0;
-
-	hold_lock_file_for_update(&lock, alts, LOCK_DIE_ON_ERROR);
-	out = fdopen_lock_file(&lock, "w");
-	if (!out)
-		die_errno(_("unable to fdopen alternates lockfile"));
-
-	in = fopen(alts, "r");
-	if (in) {
-		struct strbuf line = STRBUF_INIT;
-
-		while (strbuf_getline(&line, in) != EOF) {
-			if (!strcmp(reference, line.buf)) {
-				found = 1;
-				break;
-			}
-			fprintf_or_die(out, "%s\n", line.buf);
-		}
-
-		strbuf_release(&line);
-		fclose(in);
-	}
-	else if (errno != ENOENT)
-		die_errno(_("unable to read alternates file"));
-
-	if (found) {
-		rollback_lock_file(&lock);
-	} else {
-		fprintf_or_die(out, "%s\n", reference);
-		if (commit_lock_file(&lock))
-			die_errno(_("unable to move new alternates file into place"));
-		if (the_repository->objects->loaded_alternates)
-			link_alt_odb_entries(the_repository, reference,
-					     '\n', NULL, 0);
-	}
-	free(alts);
-}
-
-void add_to_alternates_memory(const char *reference)
-{
-	/*
-	 * Make sure alternates are initialized, or else our entry may be
-	 * overwritten when they are.
-	 */
-	prepare_alt_odb(the_repository);
-
-	link_alt_odb_entries(the_repository, reference,
-			     '\n', NULL, 0);
-}
-
-struct object_directory *set_temporary_primary_odb(const char *dir, int will_destroy)
-{
-	struct object_directory *new_odb;
-
-	/*
-	 * Make sure alternates are initialized, or else our entry may be
-	 * overwritten when they are.
-	 */
-	prepare_alt_odb(the_repository);
-
-	/*
-	 * Make a new primary odb and link the old primary ODB in as an
-	 * alternate
-	 */
-	new_odb = xcalloc(1, sizeof(*new_odb));
-	new_odb->path = xstrdup(dir);
-
-	/*
-	 * Disable ref updates while a temporary odb is active, since
-	 * the objects in the database may roll back.
-	 */
-	new_odb->disable_ref_updates = 1;
-	new_odb->will_destroy = will_destroy;
-	new_odb->next = the_repository->objects->odb;
-	the_repository->objects->odb = new_odb;
-	return new_odb->next;
-}
-
-void restore_primary_odb(struct object_directory *restore_odb, const char *old_path)
-{
-	struct object_directory *cur_odb = the_repository->objects->odb;
-
-	if (strcmp(old_path, cur_odb->path))
-		BUG("expected %s as primary object store; found %s",
-		    old_path, cur_odb->path);
-
-	if (cur_odb->next != restore_odb)
-		BUG("we expect the old primary object store to be the first alternate");
-
-	the_repository->objects->odb = restore_odb;
-	free_object_directory(cur_odb);
-}
-
-/*
- * Compute the exact path an alternate is at and returns it. In case of
- * error NULL is returned and the human readable error is added to `err`
- * `path` may be relative and should point to $GIT_DIR.
- * `err` must not be null.
- */
-char *compute_alternate_path(const char *path, struct strbuf *err)
-{
-	char *ref_git = NULL;
-	const char *repo;
-	int seen_error = 0;
-
-	ref_git = real_pathdup(path, 0);
-	if (!ref_git) {
-		seen_error = 1;
-		strbuf_addf(err, _("path '%s' does not exist"), path);
-		goto out;
-	}
-
-	repo = read_gitfile(ref_git);
-	if (!repo)
-		repo = read_gitfile(mkpath("%s/.git", ref_git));
-	if (repo) {
-		free(ref_git);
-		ref_git = xstrdup(repo);
-	}
-
-	if (!repo && is_directory(mkpath("%s/.git/objects", ref_git))) {
-		char *ref_git_git = mkpathdup("%s/.git", ref_git);
-		free(ref_git);
-		ref_git = ref_git_git;
-	} else if (!is_directory(mkpath("%s/objects", ref_git))) {
-		struct strbuf sb = STRBUF_INIT;
-		seen_error = 1;
-		if (get_common_dir(&sb, ref_git)) {
-			strbuf_addf(err,
-				    _("reference repository '%s' as a linked "
-				      "checkout is not supported yet."),
-				    path);
-			goto out;
-		}
-
-		strbuf_addf(err, _("reference repository '%s' is not a "
-					"local repository."), path);
-		goto out;
-	}
-
-	if (!access(mkpath("%s/shallow", ref_git), F_OK)) {
-		strbuf_addf(err, _("reference repository '%s' is shallow"),
-			    path);
-		seen_error = 1;
-		goto out;
-	}
-
-	if (!access(mkpath("%s/info/grafts", ref_git), F_OK)) {
-		strbuf_addf(err,
-			    _("reference repository '%s' is grafted"),
-			    path);
-		seen_error = 1;
-		goto out;
-	}
-
-out:
-	if (seen_error) {
-		FREE_AND_NULL(ref_git);
-	}
-
-	return ref_git;
-}
-
-struct object_directory *find_odb(struct repository *r, const char *obj_dir)
-{
-	struct object_directory *odb;
-	char *obj_dir_real = real_pathdup(obj_dir, 1);
-	struct strbuf odb_path_real = STRBUF_INIT;
-
-	prepare_alt_odb(r);
-	for (odb = r->objects->odb; odb; odb = odb->next) {
-		strbuf_realpath(&odb_path_real, odb->path, 1);
-		if (!strcmp(obj_dir_real, odb_path_real.buf))
-			break;
-	}
-
-	free(obj_dir_real);
-	strbuf_release(&odb_path_real);
-
-	if (!odb)
-		die(_("could not find object directory matching %s"), obj_dir);
-	return odb;
-}
-
-static void fill_alternate_refs_command(struct child_process *cmd,
-					const char *repo_path)
-{
-	const char *value;
-
-	if (!git_config_get_value("core.alternateRefsCommand", &value)) {
-		cmd->use_shell = 1;
-
-		strvec_push(&cmd->args, value);
-		strvec_push(&cmd->args, repo_path);
-	} else {
-		cmd->git_cmd = 1;
-
-		strvec_pushf(&cmd->args, "--git-dir=%s", repo_path);
-		strvec_push(&cmd->args, "for-each-ref");
-		strvec_push(&cmd->args, "--format=%(objectname)");
-
-		if (!git_config_get_value("core.alternateRefsPrefixes", &value)) {
-			strvec_push(&cmd->args, "--");
-			strvec_split(&cmd->args, value);
-		}
-	}
-
-	strvec_pushv(&cmd->env, (const char **)local_repo_env);
-	cmd->out = -1;
-}
-
-static void read_alternate_refs(const char *path,
-				alternate_ref_fn *cb,
-				void *data)
-{
-	struct child_process cmd = CHILD_PROCESS_INIT;
-	struct strbuf line = STRBUF_INIT;
-	FILE *fh;
-
-	fill_alternate_refs_command(&cmd, path);
-
-	if (start_command(&cmd))
-		return;
-
-	fh = xfdopen(cmd.out, "r");
-	while (strbuf_getline_lf(&line, fh) != EOF) {
-		struct object_id oid;
-		const char *p;
-
-		if (parse_oid_hex(line.buf, &oid, &p) || *p) {
-			warning(_("invalid line while parsing alternate refs: %s"),
-				line.buf);
-			break;
-		}
-
-		cb(&oid, data);
-	}
-
-	fclose(fh);
-	finish_command(&cmd);
-	strbuf_release(&line);
-}
-
-struct alternate_refs_data {
-	alternate_ref_fn *fn;
-	void *data;
-};
-
-static int refs_from_alternate_cb(struct object_directory *e,
-				  void *data)
-{
-	struct strbuf path = STRBUF_INIT;
-	size_t base_len;
-	struct alternate_refs_data *cb = data;
-
-	if (!strbuf_realpath(&path, e->path, 0))
-		goto out;
-	if (!strbuf_strip_suffix(&path, "/objects"))
-		goto out;
-	base_len = path.len;
-
-	/* Is this a git repository with refs? */
-	strbuf_addstr(&path, "/refs");
-	if (!is_directory(path.buf))
-		goto out;
-	strbuf_setlen(&path, base_len);
-
-	read_alternate_refs(path.buf, cb->fn, cb->data);
-
-out:
-	strbuf_release(&path);
-	return 0;
-}
-
-void for_each_alternate_ref(alternate_ref_fn fn, void *data)
-{
-	struct alternate_refs_data cb;
-	cb.fn = fn;
-	cb.data = data;
-	foreach_alt_odb(refs_from_alternate_cb, &cb);
-}
-
-int foreach_alt_odb(alt_odb_fn fn, void *cb)
-{
-	struct object_directory *ent;
-	int r = 0;
-
-	prepare_alt_odb(the_repository);
-	for (ent = the_repository->objects->odb->next; ent; ent = ent->next) {
-		r = fn(ent, cb);
-		if (r)
-			break;
-	}
-	return r;
-}
-
-void prepare_alt_odb(struct repository *r)
-{
-	if (r->objects->loaded_alternates)
-		return;
-
-	link_alt_odb_entries(r, r->objects->alternate_db, PATH_SEP, NULL, 0);
-
-	read_info_alternates(r, r->objects->odb->path, 0);
-	r->objects->loaded_alternates = 1;
-}
-
-int has_alt_odb(struct repository *r)
-{
-	prepare_alt_odb(r);
-	return !!r->objects->odb->next;
-}
-
 /* Returns 1 if we have successfully freshened the file, 0 otherwise. */
 static int freshen_file(const char *fn)
 {
@@ -1056,9 +466,9 @@ int parse_loose_header(const char *hdr, struct object_info *oi)
 	return 0;
 }
 
-static int loose_object_info(struct repository *r,
-			     const struct object_id *oid,
-			     struct object_info *oi, int flags)
+int loose_object_info(struct repository *r,
+		      const struct object_id *oid,
+		      struct object_info *oi, int flags)
 {
 	int status = 0;
 	int fd;
@@ -1154,345 +564,6 @@ static int loose_object_info(struct repository *r,
 	return status;
 }
 
-int obj_read_use_lock = 0;
-pthread_mutex_t obj_read_mutex;
-
-void enable_obj_read_lock(void)
-{
-	if (obj_read_use_lock)
-		return;
-
-	obj_read_use_lock = 1;
-	init_recursive_mutex(&obj_read_mutex);
-}
-
-void disable_obj_read_lock(void)
-{
-	if (!obj_read_use_lock)
-		return;
-
-	obj_read_use_lock = 0;
-	pthread_mutex_destroy(&obj_read_mutex);
-}
-
-int fetch_if_missing = 1;
-
-static int do_oid_object_info_extended(struct repository *r,
-				       const struct object_id *oid,
-				       struct object_info *oi, unsigned flags)
-{
-	static struct object_info blank_oi = OBJECT_INFO_INIT;
-	const struct cached_object *co;
-	struct pack_entry e;
-	int rtype;
-	const struct object_id *real = oid;
-	int already_retried = 0;
-
-
-	if (flags & OBJECT_INFO_LOOKUP_REPLACE)
-		real = lookup_replace_object(r, oid);
-
-	if (is_null_oid(real))
-		return -1;
-
-	if (!oi)
-		oi = &blank_oi;
-
-	co = find_cached_object(real);
-	if (co) {
-		if (oi->typep)
-			*(oi->typep) = co->type;
-		if (oi->sizep)
-			*(oi->sizep) = co->size;
-		if (oi->disk_sizep)
-			*(oi->disk_sizep) = 0;
-		if (oi->delta_base_oid)
-			oidclr(oi->delta_base_oid, the_repository->hash_algo);
-		if (oi->type_name)
-			strbuf_addstr(oi->type_name, type_name(co->type));
-		if (oi->contentp)
-			*oi->contentp = xmemdupz(co->buf, co->size);
-		oi->whence = OI_CACHED;
-		return 0;
-	}
-
-	while (1) {
-		if (find_pack_entry(r, real, &e))
-			break;
-
-		/* Most likely it's a loose object. */
-		if (!loose_object_info(r, real, oi, flags))
-			return 0;
-
-		/* Not a loose object; someone else may have just packed it. */
-		if (!(flags & OBJECT_INFO_QUICK)) {
-			reprepare_packed_git(r);
-			if (find_pack_entry(r, real, &e))
-				break;
-		}
-
-		/*
-		 * If r is the_repository, this might be an attempt at
-		 * accessing a submodule object as if it were in the_repository
-		 * (having called add_submodule_odb() on that submodule's ODB).
-		 * If any such ODBs exist, register them and try again.
-		 */
-		if (r == the_repository &&
-		    register_all_submodule_odb_as_alternates())
-			/* We added some alternates; retry */
-			continue;
-
-		/* Check if it is a missing object */
-		if (fetch_if_missing && repo_has_promisor_remote(r) &&
-		    !already_retried &&
-		    !(flags & OBJECT_INFO_SKIP_FETCH_OBJECT)) {
-			promisor_remote_get_direct(r, real, 1);
-			already_retried = 1;
-			continue;
-		}
-
-		if (flags & OBJECT_INFO_DIE_IF_CORRUPT) {
-			const struct packed_git *p;
-			if ((flags & OBJECT_INFO_LOOKUP_REPLACE) && !oideq(real, oid))
-				die(_("replacement %s not found for %s"),
-				    oid_to_hex(real), oid_to_hex(oid));
-			if ((p = has_packed_and_bad(r, real)))
-				die(_("packed object %s (stored in %s) is corrupt"),
-				    oid_to_hex(real), p->pack_name);
-		}
-		return -1;
-	}
-
-	if (oi == &blank_oi)
-		/*
-		 * We know that the caller doesn't actually need the
-		 * information below, so return early.
-		 */
-		return 0;
-	rtype = packed_object_info(r, e.p, e.offset, oi);
-	if (rtype < 0) {
-		mark_bad_packed_object(e.p, real);
-		return do_oid_object_info_extended(r, real, oi, 0);
-	} else if (oi->whence == OI_PACKED) {
-		oi->u.packed.offset = e.offset;
-		oi->u.packed.pack = e.p;
-		oi->u.packed.is_delta = (rtype == OBJ_REF_DELTA ||
-					 rtype == OBJ_OFS_DELTA);
-	}
-
-	return 0;
-}
-
-static int oid_object_info_convert(struct repository *r,
-				   const struct object_id *input_oid,
-				   struct object_info *input_oi, unsigned flags)
-{
-	const struct git_hash_algo *input_algo = &hash_algos[input_oid->algo];
-	int do_die = flags & OBJECT_INFO_DIE_IF_CORRUPT;
-	struct strbuf type_name = STRBUF_INIT;
-	struct object_id oid, delta_base_oid;
-	struct object_info new_oi, *oi;
-	unsigned long size;
-	void *content;
-	int ret;
-
-	if (repo_oid_to_algop(r, input_oid, the_hash_algo, &oid)) {
-		if (do_die)
-			die(_("missing mapping of %s to %s"),
-			    oid_to_hex(input_oid), the_hash_algo->name);
-		return -1;
-	}
-
-	/* Is new_oi needed? */
-	oi = input_oi;
-	if (input_oi && (input_oi->delta_base_oid || input_oi->sizep ||
-			 input_oi->contentp)) {
-		new_oi = *input_oi;
-		/* Does delta_base_oid need to be converted? */
-		if (input_oi->delta_base_oid)
-			new_oi.delta_base_oid = &delta_base_oid;
-		/* Will the attributes differ when converted? */
-		if (input_oi->sizep || input_oi->contentp) {
-			new_oi.contentp = &content;
-			new_oi.sizep = &size;
-			new_oi.type_name = &type_name;
-		}
-		oi = &new_oi;
-	}
-
-	ret = oid_object_info_extended(r, &oid, oi, flags);
-	if (ret)
-		return -1;
-	if (oi == input_oi)
-		return ret;
-
-	if (new_oi.contentp) {
-		struct strbuf outbuf = STRBUF_INIT;
-		enum object_type type;
-
-		type = type_from_string_gently(type_name.buf, type_name.len,
-					       !do_die);
-		if (type == -1)
-			return -1;
-		if (type != OBJ_BLOB) {
-			ret = convert_object_file(the_repository, &outbuf,
-						  the_hash_algo, input_algo,
-						  content, size, type, !do_die);
-			free(content);
-			if (ret == -1)
-				return -1;
-			size = outbuf.len;
-			content = strbuf_detach(&outbuf, NULL);
-		}
-		if (input_oi->sizep)
-			*input_oi->sizep = size;
-		if (input_oi->contentp)
-			*input_oi->contentp = content;
-		else
-			free(content);
-		if (input_oi->type_name)
-			*input_oi->type_name = type_name;
-		else
-			strbuf_release(&type_name);
-	}
-	if (new_oi.delta_base_oid == &delta_base_oid) {
-		if (repo_oid_to_algop(r, &delta_base_oid, input_algo,
-				 input_oi->delta_base_oid)) {
-			if (do_die)
-				die(_("missing mapping of %s to %s"),
-				    oid_to_hex(&delta_base_oid),
-				    input_algo->name);
-			return -1;
-		}
-	}
-	input_oi->whence = new_oi.whence;
-	input_oi->u = new_oi.u;
-	return ret;
-}
-
-int oid_object_info_extended(struct repository *r, const struct object_id *oid,
-			     struct object_info *oi, unsigned flags)
-{
-	int ret;
-
-	if (oid->algo && (hash_algo_by_ptr(r->hash_algo) != oid->algo))
-		return oid_object_info_convert(r, oid, oi, flags);
-
-	obj_read_lock();
-	ret = do_oid_object_info_extended(r, oid, oi, flags);
-	obj_read_unlock();
-	return ret;
-}
-
-
-/* returns enum object_type or negative */
-int oid_object_info(struct repository *r,
-		    const struct object_id *oid,
-		    unsigned long *sizep)
-{
-	enum object_type type;
-	struct object_info oi = OBJECT_INFO_INIT;
-
-	oi.typep = &type;
-	oi.sizep = sizep;
-	if (oid_object_info_extended(r, oid, &oi,
-				      OBJECT_INFO_LOOKUP_REPLACE) < 0)
-		return -1;
-	return type;
-}
-
-int pretend_object_file(void *buf, unsigned long len, enum object_type type,
-			struct object_id *oid)
-{
-	struct cached_object_entry *co;
-	char *co_buf;
-
-	hash_object_file(the_hash_algo, buf, len, type, oid);
-	if (repo_has_object_file_with_flags(the_repository, oid, OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT) ||
-	    find_cached_object(oid))
-		return 0;
-	ALLOC_GROW(cached_objects, cached_object_nr + 1, cached_object_alloc);
-	co = &cached_objects[cached_object_nr++];
-	co->value.size = len;
-	co->value.type = type;
-	co_buf = xmalloc(len);
-	memcpy(co_buf, buf, len);
-	co->value.buf = co_buf;
-	oidcpy(&co->oid, oid);
-	return 0;
-}
-
-/*
- * This function dies on corrupt objects; the callers who want to
- * deal with them should arrange to call oid_object_info_extended() and give
- * error messages themselves.
- */
-void *repo_read_object_file(struct repository *r,
-			    const struct object_id *oid,
-			    enum object_type *type,
-			    unsigned long *size)
-{
-	struct object_info oi = OBJECT_INFO_INIT;
-	unsigned flags = OBJECT_INFO_DIE_IF_CORRUPT | OBJECT_INFO_LOOKUP_REPLACE;
-	void *data;
-
-	oi.typep = type;
-	oi.sizep = size;
-	oi.contentp = &data;
-	if (oid_object_info_extended(r, oid, &oi, flags))
-		return NULL;
-
-	return data;
-}
-
-void *read_object_with_reference(struct repository *r,
-				 const struct object_id *oid,
-				 enum object_type required_type,
-				 unsigned long *size,
-				 struct object_id *actual_oid_return)
-{
-	enum object_type type;
-	void *buffer;
-	unsigned long isize;
-	struct object_id actual_oid;
-
-	oidcpy(&actual_oid, oid);
-	while (1) {
-		int ref_length = -1;
-		const char *ref_type = NULL;
-
-		buffer = repo_read_object_file(r, &actual_oid, &type, &isize);
-		if (!buffer)
-			return NULL;
-		if (type == required_type) {
-			*size = isize;
-			if (actual_oid_return)
-				oidcpy(actual_oid_return, &actual_oid);
-			return buffer;
-		}
-		/* Handle references */
-		else if (type == OBJ_COMMIT)
-			ref_type = "tree ";
-		else if (type == OBJ_TAG)
-			ref_type = "object ";
-		else {
-			free(buffer);
-			return NULL;
-		}
-		ref_length = strlen(ref_type);
-
-		if (ref_length + the_hash_algo->hexsz > isize ||
-		    memcmp(buffer, ref_type, ref_length) ||
-		    get_oid_hex((char *) buffer + ref_length, &actual_oid)) {
-			free(buffer);
-			return NULL;
-		}
-		free(buffer);
-		/* Now we have the ID of the referred-to object in
-		 * actual_oid.  Check again. */
-	}
-}
-
 static void hash_object_body(const struct git_hash_algo *algo, struct git_hash_ctx *c,
 			     const void *buf, unsigned long len,
 			     struct object_id *oid,
@@ -2154,32 +1225,6 @@ int force_object_loose(const struct object_id *oid, time_t mtime)
 	return ret;
 }
 
-int has_object(struct repository *r, const struct object_id *oid,
-	       unsigned flags)
-{
-	int quick = !(flags & HAS_OBJECT_RECHECK_PACKED);
-	unsigned object_info_flags = OBJECT_INFO_SKIP_FETCH_OBJECT |
-		(quick ? OBJECT_INFO_QUICK : 0);
-
-	if (!startup_info->have_repository)
-		return 0;
-	return oid_object_info_extended(r, oid, NULL, object_info_flags) >= 0;
-}
-
-int repo_has_object_file_with_flags(struct repository *r,
-				    const struct object_id *oid, int flags)
-{
-	if (!startup_info->have_repository)
-		return 0;
-	return oid_object_info_extended(r, oid, NULL, flags) >= 0;
-}
-
-int repo_has_object_file(struct repository *r,
-			 const struct object_id *oid)
-{
-	return repo_has_object_file_with_flags(r, oid, 0);
-}
-
 /*
  * We can't use the normal fsck_error_function() for index_mem(),
  * because we don't yet have a valid oid for it to report. Instead,
@@ -2407,16 +1452,6 @@ int read_pack_header(int fd, struct pack_header *header)
 	return 0;
 }
 
-void assert_oid_type(const struct object_id *oid, enum object_type expect)
-{
-	enum object_type type = oid_object_info(the_repository, oid, NULL);
-	if (type < 0)
-		die(_("%s is not a valid object"), oid_to_hex(oid));
-	if (type != expect)
-		die(_("%s is not a valid '%s' object"), oid_to_hex(oid),
-		    type_name(expect));
-}
-
 int for_each_file_in_obj_subdir(unsigned int subdir_nr,
 				struct strbuf *path,
 				each_loose_object_fn obj_cb,
diff --git a/object-file.h b/object-file.h
index 353d8a85c33..78c84d970a9 100644
--- a/object-file.h
+++ b/object-file.h
@@ -21,6 +21,29 @@ extern int fetch_if_missing;
 int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);
 int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags);
 
+struct object_directory;
+
+const char *odb_loose_path(struct object_directory *odb,
+			   struct strbuf *buf,
+			   const struct object_id *oid);
+
+/*
+ * Return true iff an alternate object database has a loose object
+ * with the specified name.  This function does not respect replace
+ * references.
+ */
+int has_loose_object_nonlocal(const struct object_id *);
+
+int has_loose_object(const struct object_id *);
+
+/**
+ * format_object_header() is a thin wrapper around s xsnprintf() that
+ * writes the initial "<type> <obj-len>" part of the loose object
+ * header. It returns the size that snprintf() returns + 1.
+ */
+int format_object_header(char *str, size_t size, enum object_type type,
+			 size_t objsize);
+
 /**
  * unpack_loose_header() initializes the data stream needed to unpack
  * a loose object header.
@@ -61,6 +84,29 @@ enum unpack_loose_header_result unpack_loose_header(git_zstream *stream,
 struct object_info;
 int parse_loose_header(const char *hdr, struct object_info *oi);
 
+int write_object_file_flags(const void *buf, unsigned long len,
+			    enum object_type type, struct object_id *oid,
+			    struct object_id *comapt_oid_in, unsigned flags);
+static inline int write_object_file(const void *buf, unsigned long len,
+				    enum object_type type, struct object_id *oid)
+{
+	return write_object_file_flags(buf, len, type, oid, NULL, 0);
+}
+
+struct input_stream {
+	const void *(*read)(struct input_stream *, unsigned long *len);
+	void *data;
+	int is_finished;
+};
+
+int write_object_file_literally(const void *buf, unsigned long len,
+				const char *type, struct object_id *oid,
+				unsigned flags);
+int stream_loose_object(struct input_stream *in_stream, size_t len,
+			struct object_id *oid);
+
+int force_object_loose(const struct object_id *oid, time_t mtime);
+
 /**
  * With in-core object data in "buf", rehash it to make sure the
  * object name actually matches "oid" to detect object corruption.
@@ -79,6 +125,10 @@ int check_object_signature(struct repository *r, const struct object_id *oid,
  */
 int stream_object_signature(struct repository *r, const struct object_id *oid);
 
+int loose_object_info(struct repository *r,
+		      const struct object_id *oid,
+		      struct object_info *oi, int flags);
+
 enum finalize_object_file_flags {
 	FOF_SKIP_COLLISION_CHECK = 1,
 };
@@ -90,10 +140,18 @@ int finalize_object_file_flags(const char *tmpfile, const char *filename,
 /* Helper to check and "touch" a file */
 int check_and_freshen_file(const char *fn, int freshen);
 
-void *read_object_with_reference(struct repository *r,
-				 const struct object_id *oid,
-				 enum object_type required_type,
-				 unsigned long *size,
-				 struct object_id *oid_ret);
+/*
+ * Open the loose object at path, check its hash, and return the contents,
+ * use the "oi" argument to assert things about the object, or e.g. populate its
+ * type, and size. If the object is a blob, then "contents" may return NULL,
+ * to allow streaming of large blobs.
+ *
+ * Returns 0 on success, negative on error (details may be written to stderr).
+ */
+int read_loose_object(const char *path,
+		      const struct object_id *expected_oid,
+		      struct object_id *real_oid,
+		      void **contents,
+		      struct object_info *oi);
 
 #endif /* OBJECT_FILE_H */
diff --git a/object-store-ll.h b/object-store-ll.h
index cd3bd5bd99f..8ae80b8a5fa 100644
--- a/object-store-ll.h
+++ b/object-store-ll.h
@@ -49,12 +49,6 @@ struct object_directory {
 	char *path;
 };
 
-struct input_stream {
-	const void *(*read)(struct input_stream *, unsigned long *len);
-	void *data;
-	int is_finished;
-};
-
 void prepare_alt_odb(struct repository *r);
 int has_alt_odb(struct repository *r);
 char *compute_alternate_path(const char *path, struct strbuf *err);
@@ -273,21 +267,6 @@ void hash_object_file(const struct git_hash_algo *algo, const void *buf,
 		      unsigned long len, enum object_type type,
 		      struct object_id *oid);
 
-int write_object_file_flags(const void *buf, unsigned long len,
-			    enum object_type type, struct object_id *oid,
-			    struct object_id *comapt_oid_in, unsigned flags);
-static inline int write_object_file(const void *buf, unsigned long len,
-				    enum object_type type, struct object_id *oid)
-{
-	return write_object_file_flags(buf, len, type, oid, NULL, 0);
-}
-
-int write_object_file_literally(const void *buf, unsigned long len,
-				const char *type, struct object_id *oid,
-				unsigned flags);
-int stream_loose_object(struct input_stream *in_stream, size_t len,
-			struct object_id *oid);
-
 /*
  * Add an object file to the in-memory object store, without writing it
  * to disk.
@@ -299,8 +278,6 @@ int stream_loose_object(struct input_stream *in_stream, size_t len,
 int pretend_object_file(void *, unsigned long, enum object_type,
 			struct object_id *oid);
 
-int force_object_loose(const struct object_id *oid, time_t mtime);
-
 struct object_info {
 	/* Request */
 	enum object_type *typep;
@@ -364,20 +341,6 @@ int oid_object_info_extended(struct repository *r,
 			     const struct object_id *,
 			     struct object_info *, unsigned flags);
 
-/*
- * Open the loose object at path, check its hash, and return the contents,
- * use the "oi" argument to assert things about the object, or e.g. populate its
- * type, and size. If the object is a blob, then "contents" may return NULL,
- * to allow streaming of large blobs.
- *
- * Returns 0 on success, negative on error (details may be written to stderr).
- */
-int read_loose_object(const char *path,
-		      const struct object_id *expected_oid,
-		      struct object_id *real_oid,
-		      void **contents,
-		      struct object_info *oi);
-
 /* Retry packed storage after checking packed and loose storage */
 #define HAS_OBJECT_RECHECK_PACKED 1
 
@@ -405,23 +368,6 @@ int repo_has_object_file(struct repository *r, const struct object_id *oid);
 int repo_has_object_file_with_flags(struct repository *r,
 				    const struct object_id *oid, int flags);
 
-/*
- * Return true iff an alternate object database has a loose object
- * with the specified name.  This function does not respect replace
- * references.
- */
-int has_loose_object_nonlocal(const struct object_id *);
-
-int has_loose_object(const struct object_id *);
-
-/**
- * format_object_header() is a thin wrapper around s xsnprintf() that
- * writes the initial "<type> <obj-len>" part of the loose object
- * header. It returns the size that snprintf() returns + 1.
- */
-int format_object_header(char *str, size_t size, enum object_type type,
-			 size_t objsize);
-
 void assert_oid_type(const struct object_id *oid, enum object_type expect);
 
 /*
@@ -553,4 +499,10 @@ int for_each_object_in_pack(struct packed_git *p,
 int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 			   void *data, enum for_each_object_flags flags);
 
+void *read_object_with_reference(struct repository *r,
+				 const struct object_id *oid,
+				 enum object_type required_type,
+				 unsigned long *size,
+				 struct object_id *oid_ret);
+
 #endif /* OBJECT_STORE_LL_H */
diff --git a/object-store.c b/object-store.c
new file mode 100644
index 00000000000..e5f1f00cdde
--- /dev/null
+++ b/object-store.c
@@ -0,0 +1,972 @@
+#define USE_THE_REPOSITORY_VARIABLE
+
+#include "git-compat-util.h"
+#include "abspath.h"
+#include "config.h"
+#include "environment.h"
+#include "gettext.h"
+#include "hex.h"
+#include "lockfile.h"
+#include "object-file-convert.h"
+#include "object-file.h"
+#include "object-store.h"
+#include "packfile.h"
+#include "path.h"
+#include "promisor-remote.h"
+#include "quote.h"
+#include "replace-object.h"
+#include "run-command.h"
+#include "setup.h"
+#include "strbuf.h"
+#include "strvec.h"
+#include "submodule.h"
+#include "write-or-die.h"
+
+/*
+ * This is meant to hold a *small* number of objects that you would
+ * want repo_read_object_file() to be able to return, but yet you do not want
+ * to write them into the object store (e.g. a browse-only
+ * application).
+ */
+static struct cached_object_entry {
+	struct object_id oid;
+	struct cached_object {
+		enum object_type type;
+		const void *buf;
+		unsigned long size;
+	} value;
+} *cached_objects;
+static int cached_object_nr, cached_object_alloc;
+
+static const struct cached_object *find_cached_object(const struct object_id *oid)
+{
+	static const struct cached_object empty_tree = {
+		.type = OBJ_TREE,
+		.buf = "",
+	};
+	int i;
+	const struct cached_object_entry *co = cached_objects;
+
+	for (i = 0; i < cached_object_nr; i++, co++) {
+		if (oideq(&co->oid, oid))
+			return &co->value;
+	}
+	if (oideq(oid, the_hash_algo->empty_tree))
+		return &empty_tree;
+	return NULL;
+}
+
+int odb_mkstemp(struct strbuf *temp_filename, const char *pattern)
+{
+	int fd;
+	/*
+	 * we let the umask do its job, don't try to be more
+	 * restrictive except to remove write permission.
+	 */
+	int mode = 0444;
+	repo_git_path_replace(the_repository, temp_filename, "objects/%s", pattern);
+	fd = git_mkstemp_mode(temp_filename->buf, mode);
+	if (0 <= fd)
+		return fd;
+
+	/* slow path */
+	/* some mkstemp implementations erase temp_filename on failure */
+	repo_git_path_replace(the_repository, temp_filename, "objects/%s", pattern);
+	safe_create_leading_directories(temp_filename->buf);
+	return xmkstemp_mode(temp_filename->buf, mode);
+}
+
+int odb_pack_keep(const char *name)
+{
+	int fd;
+
+	fd = open(name, O_RDWR|O_CREAT|O_EXCL, 0600);
+	if (0 <= fd)
+		return fd;
+
+	/* slow path */
+	safe_create_leading_directories_const(name);
+	return open(name, O_RDWR|O_CREAT|O_EXCL, 0600);
+}
+
+const char *loose_object_path(struct repository *r, struct strbuf *buf,
+			      const struct object_id *oid)
+{
+	return odb_loose_path(r->objects->odb, buf, oid);
+}
+
+/*
+ * Return non-zero iff the path is usable as an alternate object database.
+ */
+static int alt_odb_usable(struct raw_object_store *o,
+			  struct strbuf *path,
+			  const char *normalized_objdir, khiter_t *pos)
+{
+	int r;
+
+	/* Detect cases where alternate disappeared */
+	if (!is_directory(path->buf)) {
+		error(_("object directory %s does not exist; "
+			"check .git/objects/info/alternates"),
+		      path->buf);
+		return 0;
+	}
+
+	/*
+	 * Prevent the common mistake of listing the same
+	 * thing twice, or object directory itself.
+	 */
+	if (!o->odb_by_path) {
+		khiter_t p;
+
+		o->odb_by_path = kh_init_odb_path_map();
+		assert(!o->odb->next);
+		p = kh_put_odb_path_map(o->odb_by_path, o->odb->path, &r);
+		assert(r == 1); /* never used */
+		kh_value(o->odb_by_path, p) = o->odb;
+	}
+	if (fspatheq(path->buf, normalized_objdir))
+		return 0;
+	*pos = kh_put_odb_path_map(o->odb_by_path, path->buf, &r);
+	/* r: 0 = exists, 1 = never used, 2 = deleted */
+	return r == 0 ? 0 : 1;
+}
+
+/*
+ * Prepare alternate object database registry.
+ *
+ * The variable alt_odb_list points at the list of struct
+ * object_directory.  The elements on this list come from
+ * non-empty elements from colon separated ALTERNATE_DB_ENVIRONMENT
+ * environment variable, and $GIT_OBJECT_DIRECTORY/info/alternates,
+ * whose contents is similar to that environment variable but can be
+ * LF separated.  Its base points at a statically allocated buffer that
+ * contains "/the/directory/corresponding/to/.git/objects/...", while
+ * its name points just after the slash at the end of ".git/objects/"
+ * in the example above, and has enough space to hold all hex characters
+ * of the object ID, an extra slash for the first level indirection, and
+ * the terminating NUL.
+ */
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth);
+static int link_alt_odb_entry(struct repository *r, const struct strbuf *entry,
+	const char *relative_base, int depth, const char *normalized_objdir)
+{
+	struct object_directory *ent;
+	struct strbuf pathbuf = STRBUF_INIT;
+	struct strbuf tmp = STRBUF_INIT;
+	khiter_t pos;
+	int ret = -1;
+
+	if (!is_absolute_path(entry->buf) && relative_base) {
+		strbuf_realpath(&pathbuf, relative_base, 1);
+		strbuf_addch(&pathbuf, '/');
+	}
+	strbuf_addbuf(&pathbuf, entry);
+
+	if (!strbuf_realpath(&tmp, pathbuf.buf, 0)) {
+		error(_("unable to normalize alternate object path: %s"),
+		      pathbuf.buf);
+		goto error;
+	}
+	strbuf_swap(&pathbuf, &tmp);
+
+	/*
+	 * The trailing slash after the directory name is given by
+	 * this function at the end. Remove duplicates.
+	 */
+	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
+		strbuf_setlen(&pathbuf, pathbuf.len - 1);
+
+	if (!alt_odb_usable(r->objects, &pathbuf, normalized_objdir, &pos))
+		goto error;
+
+	CALLOC_ARRAY(ent, 1);
+	/* pathbuf.buf is already in r->objects->odb_by_path */
+	ent->path = strbuf_detach(&pathbuf, NULL);
+
+	/* add the alternate entry */
+	*r->objects->odb_tail = ent;
+	r->objects->odb_tail = &(ent->next);
+	ent->next = NULL;
+	assert(r->objects->odb_by_path);
+	kh_value(r->objects->odb_by_path, pos) = ent;
+
+	/* recursively add alternates */
+	read_info_alternates(r, ent->path, depth + 1);
+	ret = 0;
+ error:
+	strbuf_release(&tmp);
+	strbuf_release(&pathbuf);
+	return ret;
+}
+
+static const char *parse_alt_odb_entry(const char *string,
+				       int sep,
+				       struct strbuf *out)
+{
+	const char *end;
+
+	strbuf_reset(out);
+
+	if (*string == '#') {
+		/* comment; consume up to next separator */
+		end = strchrnul(string, sep);
+	} else if (*string == '"' && !unquote_c_style(out, string, &end)) {
+		/*
+		 * quoted path; unquote_c_style has copied the
+		 * data for us and set "end". Broken quoting (e.g.,
+		 * an entry that doesn't end with a quote) falls
+		 * back to the unquoted case below.
+		 */
+	} else {
+		/* normal, unquoted path */
+		end = strchrnul(string, sep);
+		strbuf_add(out, string, end - string);
+	}
+
+	if (*end)
+		end++;
+	return end;
+}
+
+static void link_alt_odb_entries(struct repository *r, const char *alt,
+				 int sep, const char *relative_base, int depth)
+{
+	struct strbuf objdirbuf = STRBUF_INIT;
+	struct strbuf entry = STRBUF_INIT;
+
+	if (!alt || !*alt)
+		return;
+
+	if (depth > 5) {
+		error(_("%s: ignoring alternate object stores, nesting too deep"),
+				relative_base);
+		return;
+	}
+
+	strbuf_realpath(&objdirbuf, r->objects->odb->path, 1);
+
+	while (*alt) {
+		alt = parse_alt_odb_entry(alt, sep, &entry);
+		if (!entry.len)
+			continue;
+		link_alt_odb_entry(r, &entry,
+				   relative_base, depth, objdirbuf.buf);
+	}
+	strbuf_release(&entry);
+	strbuf_release(&objdirbuf);
+}
+
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth)
+{
+	char *path;
+	struct strbuf buf = STRBUF_INIT;
+
+	path = xstrfmt("%s/info/alternates", relative_base);
+	if (strbuf_read_file(&buf, path, 1024) < 0) {
+		warn_on_fopen_errors(path);
+		free(path);
+		return;
+	}
+
+	link_alt_odb_entries(r, buf.buf, '\n', relative_base, depth);
+	strbuf_release(&buf);
+	free(path);
+}
+
+void add_to_alternates_file(const char *reference)
+{
+	struct lock_file lock = LOCK_INIT;
+	char *alts = repo_git_path(the_repository, "objects/info/alternates");
+	FILE *in, *out;
+	int found = 0;
+
+	hold_lock_file_for_update(&lock, alts, LOCK_DIE_ON_ERROR);
+	out = fdopen_lock_file(&lock, "w");
+	if (!out)
+		die_errno(_("unable to fdopen alternates lockfile"));
+
+	in = fopen(alts, "r");
+	if (in) {
+		struct strbuf line = STRBUF_INIT;
+
+		while (strbuf_getline(&line, in) != EOF) {
+			if (!strcmp(reference, line.buf)) {
+				found = 1;
+				break;
+			}
+			fprintf_or_die(out, "%s\n", line.buf);
+		}
+
+		strbuf_release(&line);
+		fclose(in);
+	}
+	else if (errno != ENOENT)
+		die_errno(_("unable to read alternates file"));
+
+	if (found) {
+		rollback_lock_file(&lock);
+	} else {
+		fprintf_or_die(out, "%s\n", reference);
+		if (commit_lock_file(&lock))
+			die_errno(_("unable to move new alternates file into place"));
+		if (the_repository->objects->loaded_alternates)
+			link_alt_odb_entries(the_repository, reference,
+					     '\n', NULL, 0);
+	}
+	free(alts);
+}
+
+void add_to_alternates_memory(const char *reference)
+{
+	/*
+	 * Make sure alternates are initialized, or else our entry may be
+	 * overwritten when they are.
+	 */
+	prepare_alt_odb(the_repository);
+
+	link_alt_odb_entries(the_repository, reference,
+			     '\n', NULL, 0);
+}
+
+struct object_directory *set_temporary_primary_odb(const char *dir, int will_destroy)
+{
+	struct object_directory *new_odb;
+
+	/*
+	 * Make sure alternates are initialized, or else our entry may be
+	 * overwritten when they are.
+	 */
+	prepare_alt_odb(the_repository);
+
+	/*
+	 * Make a new primary odb and link the old primary ODB in as an
+	 * alternate
+	 */
+	new_odb = xcalloc(1, sizeof(*new_odb));
+	new_odb->path = xstrdup(dir);
+
+	/*
+	 * Disable ref updates while a temporary odb is active, since
+	 * the objects in the database may roll back.
+	 */
+	new_odb->disable_ref_updates = 1;
+	new_odb->will_destroy = will_destroy;
+	new_odb->next = the_repository->objects->odb;
+	the_repository->objects->odb = new_odb;
+	return new_odb->next;
+}
+
+void restore_primary_odb(struct object_directory *restore_odb, const char *old_path)
+{
+	struct object_directory *cur_odb = the_repository->objects->odb;
+
+	if (strcmp(old_path, cur_odb->path))
+		BUG("expected %s as primary object store; found %s",
+		    old_path, cur_odb->path);
+
+	if (cur_odb->next != restore_odb)
+		BUG("we expect the old primary object store to be the first alternate");
+
+	the_repository->objects->odb = restore_odb;
+	free_object_directory(cur_odb);
+}
+
+/*
+ * Compute the exact path an alternate is at and returns it. In case of
+ * error NULL is returned and the human readable error is added to `err`
+ * `path` may be relative and should point to $GIT_DIR.
+ * `err` must not be null.
+ */
+char *compute_alternate_path(const char *path, struct strbuf *err)
+{
+	char *ref_git = NULL;
+	const char *repo;
+	int seen_error = 0;
+
+	ref_git = real_pathdup(path, 0);
+	if (!ref_git) {
+		seen_error = 1;
+		strbuf_addf(err, _("path '%s' does not exist"), path);
+		goto out;
+	}
+
+	repo = read_gitfile(ref_git);
+	if (!repo)
+		repo = read_gitfile(mkpath("%s/.git", ref_git));
+	if (repo) {
+		free(ref_git);
+		ref_git = xstrdup(repo);
+	}
+
+	if (!repo && is_directory(mkpath("%s/.git/objects", ref_git))) {
+		char *ref_git_git = mkpathdup("%s/.git", ref_git);
+		free(ref_git);
+		ref_git = ref_git_git;
+	} else if (!is_directory(mkpath("%s/objects", ref_git))) {
+		struct strbuf sb = STRBUF_INIT;
+		seen_error = 1;
+		if (get_common_dir(&sb, ref_git)) {
+			strbuf_addf(err,
+				    _("reference repository '%s' as a linked "
+				      "checkout is not supported yet."),
+				    path);
+			goto out;
+		}
+
+		strbuf_addf(err, _("reference repository '%s' is not a "
+					"local repository."), path);
+		goto out;
+	}
+
+	if (!access(mkpath("%s/shallow", ref_git), F_OK)) {
+		strbuf_addf(err, _("reference repository '%s' is shallow"),
+			    path);
+		seen_error = 1;
+		goto out;
+	}
+
+	if (!access(mkpath("%s/info/grafts", ref_git), F_OK)) {
+		strbuf_addf(err,
+			    _("reference repository '%s' is grafted"),
+			    path);
+		seen_error = 1;
+		goto out;
+	}
+
+out:
+	if (seen_error) {
+		FREE_AND_NULL(ref_git);
+	}
+
+	return ref_git;
+}
+
+struct object_directory *find_odb(struct repository *r, const char *obj_dir)
+{
+	struct object_directory *odb;
+	char *obj_dir_real = real_pathdup(obj_dir, 1);
+	struct strbuf odb_path_real = STRBUF_INIT;
+
+	prepare_alt_odb(r);
+	for (odb = r->objects->odb; odb; odb = odb->next) {
+		strbuf_realpath(&odb_path_real, odb->path, 1);
+		if (!strcmp(obj_dir_real, odb_path_real.buf))
+			break;
+	}
+
+	free(obj_dir_real);
+	strbuf_release(&odb_path_real);
+
+	if (!odb)
+		die(_("could not find object directory matching %s"), obj_dir);
+	return odb;
+}
+
+static void fill_alternate_refs_command(struct child_process *cmd,
+					const char *repo_path)
+{
+	const char *value;
+
+	if (!git_config_get_value("core.alternateRefsCommand", &value)) {
+		cmd->use_shell = 1;
+
+		strvec_push(&cmd->args, value);
+		strvec_push(&cmd->args, repo_path);
+	} else {
+		cmd->git_cmd = 1;
+
+		strvec_pushf(&cmd->args, "--git-dir=%s", repo_path);
+		strvec_push(&cmd->args, "for-each-ref");
+		strvec_push(&cmd->args, "--format=%(objectname)");
+
+		if (!git_config_get_value("core.alternateRefsPrefixes", &value)) {
+			strvec_push(&cmd->args, "--");
+			strvec_split(&cmd->args, value);
+		}
+	}
+
+	strvec_pushv(&cmd->env, (const char **)local_repo_env);
+	cmd->out = -1;
+}
+
+static void read_alternate_refs(const char *path,
+				alternate_ref_fn *cb,
+				void *data)
+{
+	struct child_process cmd = CHILD_PROCESS_INIT;
+	struct strbuf line = STRBUF_INIT;
+	FILE *fh;
+
+	fill_alternate_refs_command(&cmd, path);
+
+	if (start_command(&cmd))
+		return;
+
+	fh = xfdopen(cmd.out, "r");
+	while (strbuf_getline_lf(&line, fh) != EOF) {
+		struct object_id oid;
+		const char *p;
+
+		if (parse_oid_hex(line.buf, &oid, &p) || *p) {
+			warning(_("invalid line while parsing alternate refs: %s"),
+				line.buf);
+			break;
+		}
+
+		cb(&oid, data);
+	}
+
+	fclose(fh);
+	finish_command(&cmd);
+	strbuf_release(&line);
+}
+
+struct alternate_refs_data {
+	alternate_ref_fn *fn;
+	void *data;
+};
+
+static int refs_from_alternate_cb(struct object_directory *e,
+				  void *data)
+{
+	struct strbuf path = STRBUF_INIT;
+	size_t base_len;
+	struct alternate_refs_data *cb = data;
+
+	if (!strbuf_realpath(&path, e->path, 0))
+		goto out;
+	if (!strbuf_strip_suffix(&path, "/objects"))
+		goto out;
+	base_len = path.len;
+
+	/* Is this a git repository with refs? */
+	strbuf_addstr(&path, "/refs");
+	if (!is_directory(path.buf))
+		goto out;
+	strbuf_setlen(&path, base_len);
+
+	read_alternate_refs(path.buf, cb->fn, cb->data);
+
+out:
+	strbuf_release(&path);
+	return 0;
+}
+
+void for_each_alternate_ref(alternate_ref_fn fn, void *data)
+{
+	struct alternate_refs_data cb;
+	cb.fn = fn;
+	cb.data = data;
+	foreach_alt_odb(refs_from_alternate_cb, &cb);
+}
+
+int foreach_alt_odb(alt_odb_fn fn, void *cb)
+{
+	struct object_directory *ent;
+	int r = 0;
+
+	prepare_alt_odb(the_repository);
+	for (ent = the_repository->objects->odb->next; ent; ent = ent->next) {
+		r = fn(ent, cb);
+		if (r)
+			break;
+	}
+	return r;
+}
+
+void prepare_alt_odb(struct repository *r)
+{
+	if (r->objects->loaded_alternates)
+		return;
+
+	link_alt_odb_entries(r, r->objects->alternate_db, PATH_SEP, NULL, 0);
+
+	read_info_alternates(r, r->objects->odb->path, 0);
+	r->objects->loaded_alternates = 1;
+}
+
+int has_alt_odb(struct repository *r)
+{
+	prepare_alt_odb(r);
+	return !!r->objects->odb->next;
+}
+
+int obj_read_use_lock = 0;
+pthread_mutex_t obj_read_mutex;
+
+void enable_obj_read_lock(void)
+{
+	if (obj_read_use_lock)
+		return;
+
+	obj_read_use_lock = 1;
+	init_recursive_mutex(&obj_read_mutex);
+}
+
+void disable_obj_read_lock(void)
+{
+	if (!obj_read_use_lock)
+		return;
+
+	obj_read_use_lock = 0;
+	pthread_mutex_destroy(&obj_read_mutex);
+}
+
+int fetch_if_missing = 1;
+
+static int do_oid_object_info_extended(struct repository *r,
+				       const struct object_id *oid,
+				       struct object_info *oi, unsigned flags)
+{
+	static struct object_info blank_oi = OBJECT_INFO_INIT;
+	const struct cached_object *co;
+	struct pack_entry e;
+	int rtype;
+	const struct object_id *real = oid;
+	int already_retried = 0;
+
+
+	if (flags & OBJECT_INFO_LOOKUP_REPLACE)
+		real = lookup_replace_object(r, oid);
+
+	if (is_null_oid(real))
+		return -1;
+
+	if (!oi)
+		oi = &blank_oi;
+
+	co = find_cached_object(real);
+	if (co) {
+		if (oi->typep)
+			*(oi->typep) = co->type;
+		if (oi->sizep)
+			*(oi->sizep) = co->size;
+		if (oi->disk_sizep)
+			*(oi->disk_sizep) = 0;
+		if (oi->delta_base_oid)
+			oidclr(oi->delta_base_oid, the_repository->hash_algo);
+		if (oi->type_name)
+			strbuf_addstr(oi->type_name, type_name(co->type));
+		if (oi->contentp)
+			*oi->contentp = xmemdupz(co->buf, co->size);
+		oi->whence = OI_CACHED;
+		return 0;
+	}
+
+	while (1) {
+		if (find_pack_entry(r, real, &e))
+			break;
+
+		/* Most likely it's a loose object. */
+		if (!loose_object_info(r, real, oi, flags))
+			return 0;
+
+		/* Not a loose object; someone else may have just packed it. */
+		if (!(flags & OBJECT_INFO_QUICK)) {
+			reprepare_packed_git(r);
+			if (find_pack_entry(r, real, &e))
+				break;
+		}
+
+		/*
+		 * If r is the_repository, this might be an attempt at
+		 * accessing a submodule object as if it were in the_repository
+		 * (having called add_submodule_odb() on that submodule's ODB).
+		 * If any such ODBs exist, register them and try again.
+		 */
+		if (r == the_repository &&
+		    register_all_submodule_odb_as_alternates())
+			/* We added some alternates; retry */
+			continue;
+
+		/* Check if it is a missing object */
+		if (fetch_if_missing && repo_has_promisor_remote(r) &&
+		    !already_retried &&
+		    !(flags & OBJECT_INFO_SKIP_FETCH_OBJECT)) {
+			promisor_remote_get_direct(r, real, 1);
+			already_retried = 1;
+			continue;
+		}
+
+		if (flags & OBJECT_INFO_DIE_IF_CORRUPT) {
+			const struct packed_git *p;
+			if ((flags & OBJECT_INFO_LOOKUP_REPLACE) && !oideq(real, oid))
+				die(_("replacement %s not found for %s"),
+				    oid_to_hex(real), oid_to_hex(oid));
+			if ((p = has_packed_and_bad(r, real)))
+				die(_("packed object %s (stored in %s) is corrupt"),
+				    oid_to_hex(real), p->pack_name);
+		}
+		return -1;
+	}
+
+	if (oi == &blank_oi)
+		/*
+		 * We know that the caller doesn't actually need the
+		 * information below, so return early.
+		 */
+		return 0;
+	rtype = packed_object_info(r, e.p, e.offset, oi);
+	if (rtype < 0) {
+		mark_bad_packed_object(e.p, real);
+		return do_oid_object_info_extended(r, real, oi, 0);
+	} else if (oi->whence == OI_PACKED) {
+		oi->u.packed.offset = e.offset;
+		oi->u.packed.pack = e.p;
+		oi->u.packed.is_delta = (rtype == OBJ_REF_DELTA ||
+					 rtype == OBJ_OFS_DELTA);
+	}
+
+	return 0;
+}
+
+static int oid_object_info_convert(struct repository *r,
+				   const struct object_id *input_oid,
+				   struct object_info *input_oi, unsigned flags)
+{
+	const struct git_hash_algo *input_algo = &hash_algos[input_oid->algo];
+	int do_die = flags & OBJECT_INFO_DIE_IF_CORRUPT;
+	struct strbuf type_name = STRBUF_INIT;
+	struct object_id oid, delta_base_oid;
+	struct object_info new_oi, *oi;
+	unsigned long size;
+	void *content;
+	int ret;
+
+	if (repo_oid_to_algop(r, input_oid, the_hash_algo, &oid)) {
+		if (do_die)
+			die(_("missing mapping of %s to %s"),
+			    oid_to_hex(input_oid), the_hash_algo->name);
+		return -1;
+	}
+
+	/* Is new_oi needed? */
+	oi = input_oi;
+	if (input_oi && (input_oi->delta_base_oid || input_oi->sizep ||
+			 input_oi->contentp)) {
+		new_oi = *input_oi;
+		/* Does delta_base_oid need to be converted? */
+		if (input_oi->delta_base_oid)
+			new_oi.delta_base_oid = &delta_base_oid;
+		/* Will the attributes differ when converted? */
+		if (input_oi->sizep || input_oi->contentp) {
+			new_oi.contentp = &content;
+			new_oi.sizep = &size;
+			new_oi.type_name = &type_name;
+		}
+		oi = &new_oi;
+	}
+
+	ret = oid_object_info_extended(r, &oid, oi, flags);
+	if (ret)
+		return -1;
+	if (oi == input_oi)
+		return ret;
+
+	if (new_oi.contentp) {
+		struct strbuf outbuf = STRBUF_INIT;
+		enum object_type type;
+
+		type = type_from_string_gently(type_name.buf, type_name.len,
+					       !do_die);
+		if (type == -1)
+			return -1;
+		if (type != OBJ_BLOB) {
+			ret = convert_object_file(the_repository, &outbuf,
+						  the_hash_algo, input_algo,
+						  content, size, type, !do_die);
+			free(content);
+			if (ret == -1)
+				return -1;
+			size = outbuf.len;
+			content = strbuf_detach(&outbuf, NULL);
+		}
+		if (input_oi->sizep)
+			*input_oi->sizep = size;
+		if (input_oi->contentp)
+			*input_oi->contentp = content;
+		else
+			free(content);
+		if (input_oi->type_name)
+			*input_oi->type_name = type_name;
+		else
+			strbuf_release(&type_name);
+	}
+	if (new_oi.delta_base_oid == &delta_base_oid) {
+		if (repo_oid_to_algop(r, &delta_base_oid, input_algo,
+				 input_oi->delta_base_oid)) {
+			if (do_die)
+				die(_("missing mapping of %s to %s"),
+				    oid_to_hex(&delta_base_oid),
+				    input_algo->name);
+			return -1;
+		}
+	}
+	input_oi->whence = new_oi.whence;
+	input_oi->u = new_oi.u;
+	return ret;
+}
+
+int oid_object_info_extended(struct repository *r, const struct object_id *oid,
+			     struct object_info *oi, unsigned flags)
+{
+	int ret;
+
+	if (oid->algo && (hash_algo_by_ptr(r->hash_algo) != oid->algo))
+		return oid_object_info_convert(r, oid, oi, flags);
+
+	obj_read_lock();
+	ret = do_oid_object_info_extended(r, oid, oi, flags);
+	obj_read_unlock();
+	return ret;
+}
+
+
+/* returns enum object_type or negative */
+int oid_object_info(struct repository *r,
+		    const struct object_id *oid,
+		    unsigned long *sizep)
+{
+	enum object_type type;
+	struct object_info oi = OBJECT_INFO_INIT;
+
+	oi.typep = &type;
+	oi.sizep = sizep;
+	if (oid_object_info_extended(r, oid, &oi,
+				      OBJECT_INFO_LOOKUP_REPLACE) < 0)
+		return -1;
+	return type;
+}
+
+int pretend_object_file(void *buf, unsigned long len, enum object_type type,
+			struct object_id *oid)
+{
+	struct cached_object_entry *co;
+	char *co_buf;
+
+	hash_object_file(the_hash_algo, buf, len, type, oid);
+	if (repo_has_object_file_with_flags(the_repository, oid, OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT) ||
+	    find_cached_object(oid))
+		return 0;
+	ALLOC_GROW(cached_objects, cached_object_nr + 1, cached_object_alloc);
+	co = &cached_objects[cached_object_nr++];
+	co->value.size = len;
+	co->value.type = type;
+	co_buf = xmalloc(len);
+	memcpy(co_buf, buf, len);
+	co->value.buf = co_buf;
+	oidcpy(&co->oid, oid);
+	return 0;
+}
+
+/*
+ * This function dies on corrupt objects; the callers who want to
+ * deal with them should arrange to call oid_object_info_extended() and give
+ * error messages themselves.
+ */
+void *repo_read_object_file(struct repository *r,
+			    const struct object_id *oid,
+			    enum object_type *type,
+			    unsigned long *size)
+{
+	struct object_info oi = OBJECT_INFO_INIT;
+	unsigned flags = OBJECT_INFO_DIE_IF_CORRUPT | OBJECT_INFO_LOOKUP_REPLACE;
+	void *data;
+
+	oi.typep = type;
+	oi.sizep = size;
+	oi.contentp = &data;
+	if (oid_object_info_extended(r, oid, &oi, flags))
+		return NULL;
+
+	return data;
+}
+
+void *read_object_with_reference(struct repository *r,
+				 const struct object_id *oid,
+				 enum object_type required_type,
+				 unsigned long *size,
+				 struct object_id *actual_oid_return)
+{
+	enum object_type type;
+	void *buffer;
+	unsigned long isize;
+	struct object_id actual_oid;
+
+	oidcpy(&actual_oid, oid);
+	while (1) {
+		int ref_length = -1;
+		const char *ref_type = NULL;
+
+		buffer = repo_read_object_file(r, &actual_oid, &type, &isize);
+		if (!buffer)
+			return NULL;
+		if (type == required_type) {
+			*size = isize;
+			if (actual_oid_return)
+				oidcpy(actual_oid_return, &actual_oid);
+			return buffer;
+		}
+		/* Handle references */
+		else if (type == OBJ_COMMIT)
+			ref_type = "tree ";
+		else if (type == OBJ_TAG)
+			ref_type = "object ";
+		else {
+			free(buffer);
+			return NULL;
+		}
+		ref_length = strlen(ref_type);
+
+		if (ref_length + the_hash_algo->hexsz > isize ||
+		    memcmp(buffer, ref_type, ref_length) ||
+		    get_oid_hex((char *) buffer + ref_length, &actual_oid)) {
+			free(buffer);
+			return NULL;
+		}
+		free(buffer);
+		/* Now we have the ID of the referred-to object in
+		 * actual_oid.  Check again. */
+	}
+}
+
+int has_object(struct repository *r, const struct object_id *oid,
+	       unsigned flags)
+{
+	int quick = !(flags & HAS_OBJECT_RECHECK_PACKED);
+	unsigned object_info_flags = OBJECT_INFO_SKIP_FETCH_OBJECT |
+		(quick ? OBJECT_INFO_QUICK : 0);
+
+	if (!startup_info->have_repository)
+		return 0;
+	return oid_object_info_extended(r, oid, NULL, object_info_flags) >= 0;
+}
+
+int repo_has_object_file_with_flags(struct repository *r,
+				    const struct object_id *oid, int flags)
+{
+	if (!startup_info->have_repository)
+		return 0;
+	return oid_object_info_extended(r, oid, NULL, flags) >= 0;
+}
+
+int repo_has_object_file(struct repository *r,
+			 const struct object_id *oid)
+{
+	return repo_has_object_file_with_flags(r, oid, 0);
+}
+
+void assert_oid_type(const struct object_id *oid, enum object_type expect)
+{
+	enum object_type type = oid_object_info(the_repository, oid, NULL);
+	if (type < 0)
+		die(_("%s is not a valid object"), oid_to_hex(oid));
+	if (type != expect)
+		die(_("%s is not a valid '%s' object"), oid_to_hex(oid),
+		    type_name(expect));
+}

-- 
2.49.0.777.g153de2bbd5.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH v2 5/9] object-file: split up concerns of `HASH_*` flags
  2025-04-11  9:29 ` [PATCH v2 " Patrick Steinhardt
                     ` (3 preceding siblings ...)
  2025-04-11  9:29   ` [PATCH v2 4/9] object-file: split out functions relating to object store subsystem Patrick Steinhardt
@ 2025-04-11  9:29   ` Patrick Steinhardt
  2025-04-11  9:29   ` [PATCH v2 6/9] object-file: split out functions relating to index subsystem Patrick Steinhardt
                     ` (3 subsequent siblings)
  8 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-11  9:29 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Junio C Hamano

The functions `hash_object_file()`, `write_object_file()` and
`index_fd()` reuse the same set of flags to alter their behaviour. This
not only adds confusion, but given that every function only supports a
subset of the flags it becomes very hard to see which flags can be
passed to what function. Last but not least, this entangles the
implementation of all three function families.

Split up concerns by creating separate flags for each of the function
families.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/hash-object.c  | 23 +++++++++++++++++------
 builtin/replace.c      |  2 +-
 builtin/update-index.c |  2 +-
 bulk-checkin.c         |  6 +++---
 cache-tree.c           |  2 +-
 notes-merge.c          |  2 +-
 object-file.c          | 18 +++++++++---------
 object-file.h          | 25 +++++++++++++++++++++----
 read-cache.c           |  4 ++--
 9 files changed, 56 insertions(+), 28 deletions(-)

diff --git a/builtin/hash-object.c b/builtin/hash-object.c
index a25f0403f44..e7c0d6afdef 100644
--- a/builtin/hash-object.c
+++ b/builtin/hash-object.c
@@ -19,6 +19,11 @@
 #include "strbuf.h"
 #include "write-or-die.h"
 
+enum {
+	HASH_OBJECT_CHECK = (1 << 0),
+	HASH_OBJECT_WRITE = (1 << 1),
+};
+
 /*
  * This is to create corrupt objects for debugging and as such it
  * needs to bypass the data conversion performed by, and the type
@@ -33,7 +38,7 @@ static int hash_literally(struct object_id *oid, int fd, const char *type, unsig
 		ret = -1;
 	else
 		ret = write_object_file_literally(buf.buf, buf.len, type, oid,
-						 flags);
+						  (flags & HASH_OBJECT_WRITE) ? WRITE_OBJECT_FILE_PERSIST : 0);
 	close(fd);
 	strbuf_release(&buf);
 	return ret;
@@ -42,15 +47,21 @@ static int hash_literally(struct object_id *oid, int fd, const char *type, unsig
 static void hash_fd(int fd, const char *type, const char *path, unsigned flags,
 		    int literally)
 {
+	unsigned int index_flags = 0;
 	struct stat st;
 	struct object_id oid;
 
+	if (flags & HASH_OBJECT_WRITE)
+		index_flags |= INDEX_WRITE_OBJECT;
+	if (flags & HASH_OBJECT_CHECK)
+		index_flags |= INDEX_FORMAT_CHECK;
+
 	if (fstat(fd, &st) < 0 ||
 	    (literally
 	     ? hash_literally(&oid, fd, type, flags)
 	     : index_fd(the_repository->index, &oid, fd, &st,
-			type_from_string(type), path, flags)))
-		die((flags & HASH_WRITE_OBJECT)
+			type_from_string(type), path, index_flags)))
+		die((flags & HASH_OBJECT_WRITE)
 		    ? "Unable to add %s to database"
 		    : "Unable to hash %s", path);
 	printf("%s\n", oid_to_hex(&oid));
@@ -102,13 +113,13 @@ int cmd_hash_object(int argc,
 	int no_filters = 0;
 	int literally = 0;
 	int nongit = 0;
-	unsigned flags = HASH_FORMAT_CHECK;
+	unsigned flags = HASH_OBJECT_CHECK;
 	const char *vpath = NULL;
 	char *vpath_free = NULL;
 	const struct option hash_object_options[] = {
 		OPT_STRING('t', NULL, &type, N_("type"), N_("object type")),
 		OPT_BIT('w', NULL, &flags, N_("write the object into the object database"),
-			HASH_WRITE_OBJECT),
+			HASH_OBJECT_WRITE),
 		OPT_COUNTUP( 0 , "stdin", &hashstdin, N_("read the object from stdin")),
 		OPT_BOOL( 0 , "stdin-paths", &stdin_paths, N_("read file names from stdin")),
 		OPT_BOOL( 0 , "no-filters", &no_filters, N_("store file as is without filters")),
@@ -122,7 +133,7 @@ int cmd_hash_object(int argc,
 	argc = parse_options(argc, argv, prefix, hash_object_options,
 			     hash_object_usage, 0);
 
-	if (flags & HASH_WRITE_OBJECT)
+	if (flags & HASH_OBJECT_WRITE)
 		prefix = setup_git_directory();
 	else
 		prefix = setup_git_directory_gently(&nongit);
diff --git a/builtin/replace.c b/builtin/replace.c
index 15ec0922ce1..2b4fc9a68b3 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -305,7 +305,7 @@ static int import_object(struct object_id *oid, enum object_type type,
 		strbuf_release(&result);
 	} else {
 		struct stat st;
-		int flags = HASH_FORMAT_CHECK | HASH_WRITE_OBJECT;
+		int flags = INDEX_FORMAT_CHECK | INDEX_WRITE_OBJECT;
 
 		if (fstat(fd, &st) < 0) {
 			error_errno(_("unable to fstat %s"), filename);
diff --git a/builtin/update-index.c b/builtin/update-index.c
index b2f6b1a3fbb..f0cf964294d 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -304,7 +304,7 @@ static int add_one_path(const struct cache_entry *old, const char *path, int len
 	ce->ce_mode = ce_mode_from_stat(old, st->st_mode);
 
 	if (index_path(the_repository->index, &ce->oid, path, st,
-		       info_only ? 0 : HASH_WRITE_OBJECT)) {
+		       info_only ? 0 : INDEX_WRITE_OBJECT)) {
 		discard_cache_entry(ce);
 		return -1;
 	}
diff --git a/bulk-checkin.c b/bulk-checkin.c
index 23ac00ea0a6..309201a76a6 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -171,7 +171,7 @@ static int stream_blob_to_pack(struct bulk_checkin_packfile *state,
 	unsigned char obuf[16384];
 	unsigned hdrlen;
 	int status = Z_OK;
-	int write_object = (flags & HASH_WRITE_OBJECT);
+	int write_object = (flags & INDEX_WRITE_OBJECT);
 	off_t offset = 0;
 
 	git_deflate_init(&s, pack_compression_level);
@@ -241,7 +241,7 @@ static int stream_blob_to_pack(struct bulk_checkin_packfile *state,
 static void prepare_to_stream(struct bulk_checkin_packfile *state,
 			      unsigned flags)
 {
-	if (!(flags & HASH_WRITE_OBJECT) || state->f)
+	if (!(flags & INDEX_WRITE_OBJECT) || state->f)
 		return;
 
 	state->f = create_tmp_packfile(the_repository, &state->pack_tmp_name);
@@ -275,7 +275,7 @@ static int deflate_blob_to_pack(struct bulk_checkin_packfile *state,
 	git_hash_update(&ctx, obuf, header_len);
 
 	/* Note: idx is non-NULL when we are writing */
-	if ((flags & HASH_WRITE_OBJECT) != 0) {
+	if ((flags & INDEX_WRITE_OBJECT) != 0) {
 		CALLOC_ARRAY(idx, 1);
 
 		prepare_to_stream(state, flags);
diff --git a/cache-tree.c b/cache-tree.c
index bcbcad3d61a..4c8167ea927 100644
--- a/cache-tree.c
+++ b/cache-tree.c
@@ -452,7 +452,7 @@ static int update_one(struct cache_tree *it,
 				 OBJ_TREE, &it->oid);
 	} else if (write_object_file_flags(buffer.buf, buffer.len, OBJ_TREE,
 					   &it->oid, NULL, flags & WRITE_TREE_SILENT
-					   ? HASH_SILENT : 0)) {
+					   ? WRITE_OBJECT_FILE_SILENT : 0)) {
 		strbuf_release(&buffer);
 		return -1;
 	}
diff --git a/notes-merge.c b/notes-merge.c
index 5008faef450..d9f2914c819 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -729,7 +729,7 @@ int notes_merge_commit(struct notes_merge_options *o,
 		/* write file as blob, and add to partial_tree */
 		if (stat(path.buf, &st))
 			die_errno("Failed to stat '%s'", path.buf);
-		if (index_path(o->repo->index, &blob_oid, path.buf, &st, HASH_WRITE_OBJECT))
+		if (index_path(o->repo->index, &blob_oid, path.buf, &st, INDEX_WRITE_OBJECT))
 			die("Failed to write blob object from '%s'", path.buf);
 		if (add_note(partial_tree, &obj_oid, &blob_oid, NULL))
 			die("Failed to add resolved note '%s' to notes tree",
diff --git a/object-file.c b/object-file.c
index 1c74aa239fb..dea5a70a331 100644
--- a/object-file.c
+++ b/object-file.c
@@ -33,9 +33,9 @@
 
 static int get_conv_flags(unsigned flags)
 {
-	if (flags & HASH_RENORMALIZE)
+	if (flags & INDEX_RENORMALIZE)
 		return CONV_EOL_RENORMALIZE;
-	else if (flags & HASH_WRITE_OBJECT)
+	else if (flags & INDEX_WRITE_OBJECT)
 		return global_conv_flags_eol | CONV_WRITE_OBJECT;
 	else
 		return 0;
@@ -835,7 +835,7 @@ static int start_loose_object_common(struct strbuf *tmp_file,
 
 	fd = create_tmpfile(tmp_file, filename);
 	if (fd < 0) {
-		if (flags & HASH_SILENT)
+		if (flags & WRITE_OBJECT_FILE_SILENT)
 			return -1;
 		else if (errno == EACCES)
 			return error(_("insufficient permission for adding "
@@ -967,7 +967,7 @@ static int write_loose_object(const struct object_id *oid, char *hdr,
 		utb.actime = mtime;
 		utb.modtime = mtime;
 		if (utime(tmp_file.buf, &utb) < 0 &&
-		    !(flags & HASH_SILENT))
+		    !(flags & WRITE_OBJECT_FILE_SILENT))
 			warning_errno(_("failed utime() on %s"), tmp_file.buf);
 	}
 
@@ -1178,7 +1178,7 @@ int write_object_file_literally(const void *buf, unsigned long len,
 	write_object_file_prepare_literally(the_hash_algo, buf, len, type,
 					    oid, header, &hdrlen);
 
-	if (!(flags & HASH_WRITE_OBJECT))
+	if (!(flags & WRITE_OBJECT_FILE_PERSIST))
 		goto cleanup;
 	if (freshen_packed_object(oid) || freshen_loose_object(oid))
 		goto cleanup;
@@ -1249,7 +1249,7 @@ static int index_mem(struct index_state *istate,
 {
 	struct strbuf nbuf = STRBUF_INIT;
 	int ret = 0;
-	int write_object = flags & HASH_WRITE_OBJECT;
+	int write_object = flags & INDEX_WRITE_OBJECT;
 
 	if (!type)
 		type = OBJ_BLOB;
@@ -1264,7 +1264,7 @@ static int index_mem(struct index_state *istate,
 			size = nbuf.len;
 		}
 	}
-	if (flags & HASH_FORMAT_CHECK) {
+	if (flags & INDEX_FORMAT_CHECK) {
 		struct fsck_options opts = FSCK_OPTIONS_DEFAULT;
 
 		opts.strict = 1;
@@ -1290,7 +1290,7 @@ static int index_stream_convert_blob(struct index_state *istate,
 				     unsigned flags)
 {
 	int ret = 0;
-	const int write_object = flags & HASH_WRITE_OBJECT;
+	const int write_object = flags & INDEX_WRITE_OBJECT;
 	struct strbuf sbuf = STRBUF_INIT;
 
 	assert(path);
@@ -1422,7 +1422,7 @@ int index_path(struct index_state *istate, struct object_id *oid,
 	case S_IFLNK:
 		if (strbuf_readlink(&sb, path, st->st_size))
 			return error_errno("readlink(\"%s\")", path);
-		if (!(flags & HASH_WRITE_OBJECT))
+		if (!(flags & INDEX_WRITE_OBJECT))
 			hash_object_file(the_hash_algo, sb.buf, sb.len,
 					 OBJ_BLOB, oid);
 		else if (write_object_file(sb.buf, sb.len, OBJ_BLOB, oid))
diff --git a/object-file.h b/object-file.h
index 78c84d970a9..c002fbe2345 100644
--- a/object-file.h
+++ b/object-file.h
@@ -14,10 +14,12 @@ struct index_state;
  */
 extern int fetch_if_missing;
 
-#define HASH_WRITE_OBJECT 1
-#define HASH_FORMAT_CHECK 2
-#define HASH_RENORMALIZE  4
-#define HASH_SILENT 8
+enum {
+	INDEX_WRITE_OBJECT = (1 << 0),
+	INDEX_FORMAT_CHECK = (1 << 1),
+	INDEX_RENORMALIZE  = (1 << 2),
+};
+
 int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);
 int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags);
 
@@ -84,6 +86,21 @@ enum unpack_loose_header_result unpack_loose_header(git_zstream *stream,
 struct object_info;
 int parse_loose_header(const char *hdr, struct object_info *oi);
 
+enum {
+	/*
+	 * By default, `write_object_file_literally()` does not actually write
+	 * anything into the object store, but only computes the object ID.
+	 * This flag changes that so that the object will be written as a loose
+	 * object and persisted.
+	 */
+	WRITE_OBJECT_FILE_PERSIST = (1 << 0),
+
+	/*
+	 * Do not print an error in case something gose wrong.
+	 */
+	WRITE_OBJECT_FILE_SILENT = (1 << 1),
+};
+
 int write_object_file_flags(const void *buf, unsigned long len,
 			    enum object_type type, struct object_id *oid,
 			    struct object_id *comapt_oid_in, unsigned flags);
diff --git a/read-cache.c b/read-cache.c
index 2f9e21c897d..23028f43a11 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -706,11 +706,11 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st,
 	int intent_only = flags & ADD_CACHE_INTENT;
 	int add_option = (ADD_CACHE_OK_TO_ADD|ADD_CACHE_OK_TO_REPLACE|
 			  (intent_only ? ADD_CACHE_NEW_ONLY : 0));
-	unsigned hash_flags = pretend ? 0 : HASH_WRITE_OBJECT;
+	unsigned hash_flags = pretend ? 0 : INDEX_WRITE_OBJECT;
 	struct object_id oid;
 
 	if (flags & ADD_CACHE_RENORMALIZE)
-		hash_flags |= HASH_RENORMALIZE;
+		hash_flags |= INDEX_RENORMALIZE;
 
 	if (!S_ISREG(st_mode) && !S_ISLNK(st_mode) && !S_ISDIR(st_mode))
 		return error(_("%s: can only add regular files, symbolic links or git-directories"), path);

-- 
2.49.0.777.g153de2bbd5.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH v2 6/9] object-file: split out functions relating to index subsystem
  2025-04-11  9:29 ` [PATCH v2 " Patrick Steinhardt
                     ` (4 preceding siblings ...)
  2025-04-11  9:29   ` [PATCH v2 5/9] object-file: split up concerns of `HASH_*` flags Patrick Steinhardt
@ 2025-04-11  9:29   ` Patrick Steinhardt
  2025-04-12  8:17     ` Jeff King
  2025-04-11  9:29   ` [PATCH v2 7/9] object: split out functions relating to object store subsystem Patrick Steinhardt
                     ` (2 subsequent siblings)
  8 siblings, 1 reply; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-11  9:29 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Junio C Hamano

Split out functions relating to the index subsystem from "object-file.c"
to help us separate concerns.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/difftool.c    |   2 +-
 builtin/hash-object.c |   1 +
 builtin/replace.c     |   1 +
 bulk-checkin.c        |   1 +
 diff.c                |   2 +-
 notes-merge.c         |   1 +
 object-file.c         | 225 --------------------------------------------------
 object-file.h         |  11 ---
 read-cache.c          | 224 +++++++++++++++++++++++++++++++++++++++++++++++++
 read-cache.h          |   9 ++
 10 files changed, 239 insertions(+), 238 deletions(-)

diff --git a/builtin/difftool.c b/builtin/difftool.c
index 41cd00066cc..45d6ea8a801 100644
--- a/builtin/difftool.c
+++ b/builtin/difftool.c
@@ -22,7 +22,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "parse-options.h"
-#include "read-cache-ll.h"
+#include "read-cache.h"
 #include "repository.h"
 #include "sparse-index.h"
 #include "strvec.h"
diff --git a/builtin/hash-object.c b/builtin/hash-object.c
index e7c0d6afdef..7aa889b1b4a 100644
--- a/builtin/hash-object.c
+++ b/builtin/hash-object.c
@@ -15,6 +15,7 @@
 #include "blob.h"
 #include "quote.h"
 #include "parse-options.h"
+#include "read-cache.h"
 #include "setup.h"
 #include "strbuf.h"
 #include "write-or-die.h"
diff --git a/builtin/replace.c b/builtin/replace.c
index 2b4fc9a68b3..ce2948a9b2a 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -20,6 +20,7 @@
 #include "object-file.h"
 #include "object-name.h"
 #include "object-store-ll.h"
+#include "read-cache.h"
 #include "replace-object.h"
 #include "tag.h"
 #include "wildmatch.h"
diff --git a/bulk-checkin.c b/bulk-checkin.c
index 309201a76a6..0b9bad92868 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -11,6 +11,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "lockfile.h"
+#include "read-cache.h"
 #include "repository.h"
 #include "csum-file.h"
 #include "pack.h"
diff --git a/diff.c b/diff.c
index 3bcf5028831..c361199ab1b 100644
--- a/diff.c
+++ b/diff.c
@@ -42,7 +42,7 @@
 #include "dir.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "read-cache-ll.h"
+#include "read-cache.h"
 #include "setup.h"
 #include "strmap.h"
 #include "ws.h"
diff --git a/notes-merge.c b/notes-merge.c
index d9f2914c819..13750490eff 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -10,6 +10,7 @@
 #include "object-name.h"
 #include "object-store-ll.h"
 #include "path.h"
+#include "read-cache.h"
 #include "repository.h"
 #include "diff.h"
 #include "diffcore.h"
diff --git a/object-file.c b/object-file.c
index dea5a70a331..235d29858ff 100644
--- a/object-file.c
+++ b/object-file.c
@@ -12,9 +12,7 @@
 
 #include "git-compat-util.h"
 #include "bulk-checkin.h"
-#include "convert.h"
 #include "environment.h"
-#include "fsck.h"
 #include "gettext.h"
 #include "hex.h"
 #include "loose.h"
@@ -25,22 +23,11 @@
 #include "pack.h"
 #include "packfile.h"
 #include "path.h"
-#include "setup.h"
 #include "streaming.h"
 
 /* The maximum size for an object header. */
 #define MAX_HEADER_LEN 32
 
-static int get_conv_flags(unsigned flags)
-{
-	if (flags & INDEX_RENORMALIZE)
-		return CONV_EOL_RENORMALIZE;
-	else if (flags & INDEX_WRITE_OBJECT)
-		return global_conv_flags_eol | CONV_WRITE_OBJECT;
-	else
-		return 0;
-}
-
 static void fill_loose_path(struct strbuf *buf, const struct object_id *oid)
 {
 	int i;
@@ -1225,218 +1212,6 @@ int force_object_loose(const struct object_id *oid, time_t mtime)
 	return ret;
 }
 
-/*
- * We can't use the normal fsck_error_function() for index_mem(),
- * because we don't yet have a valid oid for it to report. Instead,
- * report the minimal fsck error here, and rely on the caller to
- * give more context.
- */
-static int hash_format_check_report(struct fsck_options *opts UNUSED,
-				    void *fsck_report UNUSED,
-				    enum fsck_msg_type msg_type UNUSED,
-				    enum fsck_msg_id msg_id UNUSED,
-				    const char *message)
-{
-	error(_("object fails fsck: %s"), message);
-	return 1;
-}
-
-static int index_mem(struct index_state *istate,
-		     struct object_id *oid,
-		     const void *buf, size_t size,
-		     enum object_type type,
-		     const char *path, unsigned flags)
-{
-	struct strbuf nbuf = STRBUF_INIT;
-	int ret = 0;
-	int write_object = flags & INDEX_WRITE_OBJECT;
-
-	if (!type)
-		type = OBJ_BLOB;
-
-	/*
-	 * Convert blobs to git internal format
-	 */
-	if ((type == OBJ_BLOB) && path) {
-		if (convert_to_git(istate, path, buf, size, &nbuf,
-				   get_conv_flags(flags))) {
-			buf = nbuf.buf;
-			size = nbuf.len;
-		}
-	}
-	if (flags & INDEX_FORMAT_CHECK) {
-		struct fsck_options opts = FSCK_OPTIONS_DEFAULT;
-
-		opts.strict = 1;
-		opts.error_func = hash_format_check_report;
-		if (fsck_buffer(null_oid(the_hash_algo), type, buf, size, &opts))
-			die(_("refusing to create malformed object"));
-		fsck_finish(&opts);
-	}
-
-	if (write_object)
-		ret = write_object_file(buf, size, type, oid);
-	else
-		hash_object_file(the_hash_algo, buf, size, type, oid);
-
-	strbuf_release(&nbuf);
-	return ret;
-}
-
-static int index_stream_convert_blob(struct index_state *istate,
-				     struct object_id *oid,
-				     int fd,
-				     const char *path,
-				     unsigned flags)
-{
-	int ret = 0;
-	const int write_object = flags & INDEX_WRITE_OBJECT;
-	struct strbuf sbuf = STRBUF_INIT;
-
-	assert(path);
-	assert(would_convert_to_git_filter_fd(istate, path));
-
-	convert_to_git_filter_fd(istate, path, fd, &sbuf,
-				 get_conv_flags(flags));
-
-	if (write_object)
-		ret = write_object_file(sbuf.buf, sbuf.len, OBJ_BLOB,
-					oid);
-	else
-		hash_object_file(the_hash_algo, sbuf.buf, sbuf.len, OBJ_BLOB,
-				 oid);
-	strbuf_release(&sbuf);
-	return ret;
-}
-
-static int index_pipe(struct index_state *istate, struct object_id *oid,
-		      int fd, enum object_type type,
-		      const char *path, unsigned flags)
-{
-	struct strbuf sbuf = STRBUF_INIT;
-	int ret;
-
-	if (strbuf_read(&sbuf, fd, 4096) >= 0)
-		ret = index_mem(istate, oid, sbuf.buf, sbuf.len, type, path, flags);
-	else
-		ret = -1;
-	strbuf_release(&sbuf);
-	return ret;
-}
-
-#define SMALL_FILE_SIZE (32*1024)
-
-static int index_core(struct index_state *istate,
-		      struct object_id *oid, int fd, size_t size,
-		      enum object_type type, const char *path,
-		      unsigned flags)
-{
-	int ret;
-
-	if (!size) {
-		ret = index_mem(istate, oid, "", size, type, path, flags);
-	} else if (size <= SMALL_FILE_SIZE) {
-		char *buf = xmalloc(size);
-		ssize_t read_result = read_in_full(fd, buf, size);
-		if (read_result < 0)
-			ret = error_errno(_("read error while indexing %s"),
-					  path ? path : "<unknown>");
-		else if (read_result != size)
-			ret = error(_("short read while indexing %s"),
-				    path ? path : "<unknown>");
-		else
-			ret = index_mem(istate, oid, buf, size, type, path, flags);
-		free(buf);
-	} else {
-		void *buf = xmmap(NULL, size, PROT_READ, MAP_PRIVATE, fd, 0);
-		ret = index_mem(istate, oid, buf, size, type, path, flags);
-		munmap(buf, size);
-	}
-	return ret;
-}
-
-/*
- * This creates one packfile per large blob unless bulk-checkin
- * machinery is "plugged".
- *
- * This also bypasses the usual "convert-to-git" dance, and that is on
- * purpose. We could write a streaming version of the converting
- * functions and insert that before feeding the data to fast-import
- * (or equivalent in-core API described above). However, that is
- * somewhat complicated, as we do not know the size of the filter
- * result, which we need to know beforehand when writing a git object.
- * Since the primary motivation for trying to stream from the working
- * tree file and to avoid mmaping it in core is to deal with large
- * binary blobs, they generally do not want to get any conversion, and
- * callers should avoid this code path when filters are requested.
- */
-static int index_blob_stream(struct object_id *oid, int fd, size_t size,
-			     const char *path,
-			     unsigned flags)
-{
-	return index_blob_bulk_checkin(oid, fd, size, path, flags);
-}
-
-int index_fd(struct index_state *istate, struct object_id *oid,
-	     int fd, struct stat *st,
-	     enum object_type type, const char *path, unsigned flags)
-{
-	int ret;
-
-	/*
-	 * Call xsize_t() only when needed to avoid potentially unnecessary
-	 * die() for large files.
-	 */
-	if (type == OBJ_BLOB && path && would_convert_to_git_filter_fd(istate, path))
-		ret = index_stream_convert_blob(istate, oid, fd, path, flags);
-	else if (!S_ISREG(st->st_mode))
-		ret = index_pipe(istate, oid, fd, type, path, flags);
-	else if (st->st_size <= repo_settings_get_big_file_threshold(the_repository) ||
-		 type != OBJ_BLOB ||
-		 (path && would_convert_to_git(istate, path)))
-		ret = index_core(istate, oid, fd, xsize_t(st->st_size),
-				 type, path, flags);
-	else
-		ret = index_blob_stream(oid, fd, xsize_t(st->st_size), path,
-					flags);
-	close(fd);
-	return ret;
-}
-
-int index_path(struct index_state *istate, struct object_id *oid,
-	       const char *path, struct stat *st, unsigned flags)
-{
-	int fd;
-	struct strbuf sb = STRBUF_INIT;
-	int rc = 0;
-
-	switch (st->st_mode & S_IFMT) {
-	case S_IFREG:
-		fd = open(path, O_RDONLY);
-		if (fd < 0)
-			return error_errno("open(\"%s\")", path);
-		if (index_fd(istate, oid, fd, st, OBJ_BLOB, path, flags) < 0)
-			return error(_("%s: failed to insert into database"),
-				     path);
-		break;
-	case S_IFLNK:
-		if (strbuf_readlink(&sb, path, st->st_size))
-			return error_errno("readlink(\"%s\")", path);
-		if (!(flags & INDEX_WRITE_OBJECT))
-			hash_object_file(the_hash_algo, sb.buf, sb.len,
-					 OBJ_BLOB, oid);
-		else if (write_object_file(sb.buf, sb.len, OBJ_BLOB, oid))
-			rc = error(_("%s: failed to insert into database"), path);
-		strbuf_release(&sb);
-		break;
-	case S_IFDIR:
-		return repo_resolve_gitlink_ref(the_repository, path, "HEAD", oid);
-	default:
-		return error(_("%s: unsupported file type"), path);
-	}
-	return rc;
-}
-
 int read_pack_header(int fd, struct pack_header *header)
 {
 	if (read_in_full(fd, header, sizeof(*header)) != sizeof(*header))
diff --git a/object-file.h b/object-file.h
index c002fbe2345..7d48994f1b6 100644
--- a/object-file.h
+++ b/object-file.h
@@ -4,8 +4,6 @@
 #include "git-zlib.h"
 #include "object.h"
 
-struct index_state;
-
 /*
  * Set this to 0 to prevent oid_object_info_extended() from fetching missing
  * blobs. This has a difference only if extensions.partialClone is set.
@@ -14,15 +12,6 @@ struct index_state;
  */
 extern int fetch_if_missing;
 
-enum {
-	INDEX_WRITE_OBJECT = (1 << 0),
-	INDEX_FORMAT_CHECK = (1 << 1),
-	INDEX_RENORMALIZE  = (1 << 2),
-};
-
-int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);
-int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags);
-
 struct object_directory;
 
 const char *odb_loose_path(struct object_directory *odb,
diff --git a/read-cache.c b/read-cache.c
index 23028f43a11..9909b56902f 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -10,9 +10,11 @@
 #include "git-compat-util.h"
 #include "bulk-checkin.h"
 #include "config.h"
+#include "convert.h"
 #include "date.h"
 #include "diff.h"
 #include "diffcore.h"
+#include "fsck.h"
 #include "hex.h"
 #include "tempfile.h"
 #include "lockfile.h"
@@ -4007,3 +4009,225 @@ int add_files_to_cache(struct repository *repo, const char *prefix,
 	release_revisions(&rev);
 	return !!data.add_errors;
 }
+
+static int get_conv_flags(unsigned flags)
+{
+	if (flags & INDEX_RENORMALIZE)
+		return CONV_EOL_RENORMALIZE;
+	else if (flags & INDEX_WRITE_OBJECT)
+		return global_conv_flags_eol | CONV_WRITE_OBJECT;
+	else
+		return 0;
+}
+
+/*
+ * We can't use the normal fsck_error_function() for index_mem(),
+ * because we don't yet have a valid oid for it to report. Instead,
+ * report the minimal fsck error here, and rely on the caller to
+ * give more context.
+ */
+static int hash_format_check_report(struct fsck_options *opts UNUSED,
+				    void *fsck_report UNUSED,
+				    enum fsck_msg_type msg_type UNUSED,
+				    enum fsck_msg_id msg_id UNUSED,
+				    const char *message)
+{
+	error(_("object fails fsck: %s"), message);
+	return 1;
+}
+
+static int index_mem(struct index_state *istate,
+		     struct object_id *oid,
+		     const void *buf, size_t size,
+		     enum object_type type,
+		     const char *path, unsigned flags)
+{
+	struct strbuf nbuf = STRBUF_INIT;
+	int ret = 0;
+	int write_object = flags & INDEX_WRITE_OBJECT;
+
+	if (!type)
+		type = OBJ_BLOB;
+
+	/*
+	 * Convert blobs to git internal format
+	 */
+	if ((type == OBJ_BLOB) && path) {
+		if (convert_to_git(istate, path, buf, size, &nbuf,
+				   get_conv_flags(flags))) {
+			buf = nbuf.buf;
+			size = nbuf.len;
+		}
+	}
+	if (flags & INDEX_FORMAT_CHECK) {
+		struct fsck_options opts = FSCK_OPTIONS_DEFAULT;
+
+		opts.strict = 1;
+		opts.error_func = hash_format_check_report;
+		if (fsck_buffer(null_oid(the_hash_algo), type, buf, size, &opts))
+			die(_("refusing to create malformed object"));
+		fsck_finish(&opts);
+	}
+
+	if (write_object)
+		ret = write_object_file(buf, size, type, oid);
+	else
+		hash_object_file(the_hash_algo, buf, size, type, oid);
+
+	strbuf_release(&nbuf);
+	return ret;
+}
+
+static int index_stream_convert_blob(struct index_state *istate,
+				     struct object_id *oid,
+				     int fd,
+				     const char *path,
+				     unsigned flags)
+{
+	int ret = 0;
+	const int write_object = flags & INDEX_WRITE_OBJECT;
+	struct strbuf sbuf = STRBUF_INIT;
+
+	assert(path);
+	assert(would_convert_to_git_filter_fd(istate, path));
+
+	convert_to_git_filter_fd(istate, path, fd, &sbuf,
+				 get_conv_flags(flags));
+
+	if (write_object)
+		ret = write_object_file(sbuf.buf, sbuf.len, OBJ_BLOB,
+					oid);
+	else
+		hash_object_file(the_hash_algo, sbuf.buf, sbuf.len, OBJ_BLOB,
+				 oid);
+	strbuf_release(&sbuf);
+	return ret;
+}
+
+static int index_pipe(struct index_state *istate, struct object_id *oid,
+		      int fd, enum object_type type,
+		      const char *path, unsigned flags)
+{
+	struct strbuf sbuf = STRBUF_INIT;
+	int ret;
+
+	if (strbuf_read(&sbuf, fd, 4096) >= 0)
+		ret = index_mem(istate, oid, sbuf.buf, sbuf.len, type, path, flags);
+	else
+		ret = -1;
+	strbuf_release(&sbuf);
+	return ret;
+}
+
+#define SMALL_FILE_SIZE (32*1024)
+
+static int index_core(struct index_state *istate,
+		      struct object_id *oid, int fd, size_t size,
+		      enum object_type type, const char *path,
+		      unsigned flags)
+{
+	int ret;
+
+	if (!size) {
+		ret = index_mem(istate, oid, "", size, type, path, flags);
+	} else if (size <= SMALL_FILE_SIZE) {
+		char *buf = xmalloc(size);
+		ssize_t read_result = read_in_full(fd, buf, size);
+		if (read_result < 0)
+			ret = error_errno(_("read error while indexing %s"),
+					  path ? path : "<unknown>");
+		else if (read_result != size)
+			ret = error(_("short read while indexing %s"),
+				    path ? path : "<unknown>");
+		else
+			ret = index_mem(istate, oid, buf, size, type, path, flags);
+		free(buf);
+	} else {
+		void *buf = xmmap(NULL, size, PROT_READ, MAP_PRIVATE, fd, 0);
+		ret = index_mem(istate, oid, buf, size, type, path, flags);
+		munmap(buf, size);
+	}
+	return ret;
+}
+
+/*
+ * This creates one packfile per large blob unless bulk-checkin
+ * machinery is "plugged".
+ *
+ * This also bypasses the usual "convert-to-git" dance, and that is on
+ * purpose. We could write a streaming version of the converting
+ * functions and insert that before feeding the data to fast-import
+ * (or equivalent in-core API described above). However, that is
+ * somewhat complicated, as we do not know the size of the filter
+ * result, which we need to know beforehand when writing a git object.
+ * Since the primary motivation for trying to stream from the working
+ * tree file and to avoid mmaping it in core is to deal with large
+ * binary blobs, they generally do not want to get any conversion, and
+ * callers should avoid this code path when filters are requested.
+ */
+static int index_blob_stream(struct object_id *oid, int fd, size_t size,
+			     const char *path,
+			     unsigned flags)
+{
+	return index_blob_bulk_checkin(oid, fd, size, path, flags);
+}
+
+int index_fd(struct index_state *istate, struct object_id *oid,
+	     int fd, struct stat *st,
+	     enum object_type type, const char *path, unsigned flags)
+{
+	int ret;
+
+	/*
+	 * Call xsize_t() only when needed to avoid potentially unnecessary
+	 * die() for large files.
+	 */
+	if (type == OBJ_BLOB && path && would_convert_to_git_filter_fd(istate, path))
+		ret = index_stream_convert_blob(istate, oid, fd, path, flags);
+	else if (!S_ISREG(st->st_mode))
+		ret = index_pipe(istate, oid, fd, type, path, flags);
+	else if (st->st_size <= repo_settings_get_big_file_threshold(the_repository) ||
+		 type != OBJ_BLOB ||
+		 (path && would_convert_to_git(istate, path)))
+		ret = index_core(istate, oid, fd, xsize_t(st->st_size),
+				 type, path, flags);
+	else
+		ret = index_blob_stream(oid, fd, xsize_t(st->st_size), path,
+					flags);
+	close(fd);
+	return ret;
+}
+
+int index_path(struct index_state *istate, struct object_id *oid,
+	       const char *path, struct stat *st, unsigned flags)
+{
+	int fd;
+	struct strbuf sb = STRBUF_INIT;
+	int rc = 0;
+
+	switch (st->st_mode & S_IFMT) {
+	case S_IFREG:
+		fd = open(path, O_RDONLY);
+		if (fd < 0)
+			return error_errno("open(\"%s\")", path);
+		if (index_fd(istate, oid, fd, st, OBJ_BLOB, path, flags) < 0)
+			return error(_("%s: failed to insert into database"),
+				     path);
+		break;
+	case S_IFLNK:
+		if (strbuf_readlink(&sb, path, st->st_size))
+			return error_errno("readlink(\"%s\")", path);
+		if (!(flags & INDEX_WRITE_OBJECT))
+			hash_object_file(the_hash_algo, sb.buf, sb.len,
+					 OBJ_BLOB, oid);
+		else if (write_object_file(sb.buf, sb.len, OBJ_BLOB, oid))
+			rc = error(_("%s: failed to insert into database"), path);
+		strbuf_release(&sb);
+		break;
+	case S_IFDIR:
+		return repo_resolve_gitlink_ref(the_repository, path, "HEAD", oid);
+	default:
+		return error(_("%s: unsupported file type"), path);
+	}
+	return rc;
+}
diff --git a/read-cache.h b/read-cache.h
index 043da1f1aae..d655eb64bb7 100644
--- a/read-cache.h
+++ b/read-cache.h
@@ -42,4 +42,13 @@ static inline int ce_path_match(struct index_state *istate,
 			      S_ISDIR(ce->ce_mode) || S_ISGITLINK(ce->ce_mode));
 }
 
+enum {
+	INDEX_WRITE_OBJECT = (1 << 0),
+	INDEX_FORMAT_CHECK = (1 << 1),
+	INDEX_RENORMALIZE  = (1 << 2),
+};
+
+int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);
+int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags);
+
 #endif /* READ_CACHE_H */

-- 
2.49.0.777.g153de2bbd5.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH v2 7/9] object: split out functions relating to object store subsystem
  2025-04-11  9:29 ` [PATCH v2 " Patrick Steinhardt
                     ` (5 preceding siblings ...)
  2025-04-11  9:29   ` [PATCH v2 6/9] object-file: split out functions relating to index subsystem Patrick Steinhardt
@ 2025-04-11  9:29   ` Patrick Steinhardt
  2025-04-11  9:29   ` [PATCH v2 8/9] object-store: remove global array of cached objects Patrick Steinhardt
  2025-04-11  9:29   ` [PATCH v2 9/9] object-store: merge "object-store-ll.h" and "object-store.h" Patrick Steinhardt
  8 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-11  9:29 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Junio C Hamano

Split out functions relating to the object store subsystem from
"object.c". This helps us to separate concerns.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 object-store-ll.h |  3 ---
 object-store.c    | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 object.c          | 67 -------------------------------------------------------
 3 files changed, 66 insertions(+), 70 deletions(-)

diff --git a/object-store-ll.h b/object-store-ll.h
index 8ae80b8a5fa..8bb0f33f9a8 100644
--- a/object-store-ll.h
+++ b/object-store-ll.h
@@ -92,9 +92,6 @@ struct oidtree *odb_loose_cache(struct object_directory *odb,
 /* Empty the loose object cache for the specified object directory. */
 void odb_clear_loose_cache(struct object_directory *odb);
 
-/* Clear and free the specified object directory */
-void free_object_directory(struct object_directory *odb);
-
 struct packed_git {
 	struct hashmap_entry packmap_ent;
 	struct packed_git *next;
diff --git a/object-store.c b/object-store.c
index e5f1f00cdde..ea2d86c429b 100644
--- a/object-store.c
+++ b/object-store.c
@@ -2,11 +2,13 @@
 
 #include "git-compat-util.h"
 #include "abspath.h"
+#include "commit-graph.h"
 #include "config.h"
 #include "environment.h"
 #include "gettext.h"
 #include "hex.h"
 #include "lockfile.h"
+#include "loose.h"
 #include "object-file-convert.h"
 #include "object-file.h"
 #include "object-store.h"
@@ -361,6 +363,14 @@ struct object_directory *set_temporary_primary_odb(const char *dir, int will_des
 	return new_odb->next;
 }
 
+static void free_object_directory(struct object_directory *odb)
+{
+	free(odb->path);
+	odb_clear_loose_cache(odb);
+	loose_object_map_clear(&odb->loose_map);
+	free(odb);
+}
+
 void restore_primary_odb(struct object_directory *restore_odb, const char *old_path)
 {
 	struct object_directory *cur_odb = the_repository->objects->odb;
@@ -970,3 +980,59 @@ void assert_oid_type(const struct object_id *oid, enum object_type expect)
 		die(_("%s is not a valid '%s' object"), oid_to_hex(oid),
 		    type_name(expect));
 }
+
+struct raw_object_store *raw_object_store_new(void)
+{
+	struct raw_object_store *o = xmalloc(sizeof(*o));
+
+	memset(o, 0, sizeof(*o));
+	INIT_LIST_HEAD(&o->packed_git_mru);
+	hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0);
+	pthread_mutex_init(&o->replace_mutex, NULL);
+	return o;
+}
+
+static void free_object_directories(struct raw_object_store *o)
+{
+	while (o->odb) {
+		struct object_directory *next;
+
+		next = o->odb->next;
+		free_object_directory(o->odb);
+		o->odb = next;
+	}
+	kh_destroy_odb_path_map(o->odb_by_path);
+	o->odb_by_path = NULL;
+}
+
+void raw_object_store_clear(struct raw_object_store *o)
+{
+	FREE_AND_NULL(o->alternate_db);
+
+	oidmap_free(o->replace_map, 1);
+	FREE_AND_NULL(o->replace_map);
+	pthread_mutex_destroy(&o->replace_mutex);
+
+	free_commit_graph(o->commit_graph);
+	o->commit_graph = NULL;
+	o->commit_graph_attempted = 0;
+
+	free_object_directories(o);
+	o->odb_tail = NULL;
+	o->loaded_alternates = 0;
+
+	INIT_LIST_HEAD(&o->packed_git_mru);
+	close_object_store(o);
+
+	/*
+	 * `close_object_store()` only closes the packfiles, but doesn't free
+	 * them. We thus have to do this manually.
+	 */
+	for (struct packed_git *p = o->packed_git, *next; p; p = next) {
+		next = p->next;
+		free(p);
+	}
+	o->packed_git = NULL;
+
+	hashmap_clear(&o->pack_map);
+}
diff --git a/object.c b/object.c
index 154525a4972..ccda798b75f 100644
--- a/object.c
+++ b/object.c
@@ -6,16 +6,13 @@
 #include "object.h"
 #include "replace-object.h"
 #include "object-file.h"
-#include "object-store.h"
 #include "blob.h"
 #include "statinfo.h"
 #include "tree.h"
 #include "commit.h"
 #include "tag.h"
 #include "alloc.h"
-#include "packfile.h"
 #include "commit-graph.h"
-#include "loose.h"
 
 unsigned int get_max_object_index(const struct repository *repo)
 {
@@ -567,70 +564,6 @@ struct parsed_object_pool *parsed_object_pool_new(struct repository *repo)
 	return o;
 }
 
-struct raw_object_store *raw_object_store_new(void)
-{
-	struct raw_object_store *o = xmalloc(sizeof(*o));
-
-	memset(o, 0, sizeof(*o));
-	INIT_LIST_HEAD(&o->packed_git_mru);
-	hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0);
-	pthread_mutex_init(&o->replace_mutex, NULL);
-	return o;
-}
-
-void free_object_directory(struct object_directory *odb)
-{
-	free(odb->path);
-	odb_clear_loose_cache(odb);
-	loose_object_map_clear(&odb->loose_map);
-	free(odb);
-}
-
-static void free_object_directories(struct raw_object_store *o)
-{
-	while (o->odb) {
-		struct object_directory *next;
-
-		next = o->odb->next;
-		free_object_directory(o->odb);
-		o->odb = next;
-	}
-	kh_destroy_odb_path_map(o->odb_by_path);
-	o->odb_by_path = NULL;
-}
-
-void raw_object_store_clear(struct raw_object_store *o)
-{
-	FREE_AND_NULL(o->alternate_db);
-
-	oidmap_free(o->replace_map, 1);
-	FREE_AND_NULL(o->replace_map);
-	pthread_mutex_destroy(&o->replace_mutex);
-
-	free_commit_graph(o->commit_graph);
-	o->commit_graph = NULL;
-	o->commit_graph_attempted = 0;
-
-	free_object_directories(o);
-	o->odb_tail = NULL;
-	o->loaded_alternates = 0;
-
-	INIT_LIST_HEAD(&o->packed_git_mru);
-	close_object_store(o);
-
-	/*
-	 * `close_object_store()` only closes the packfiles, but doesn't free
-	 * them. We thus have to do this manually.
-	 */
-	for (struct packed_git *p = o->packed_git, *next; p; p = next) {
-		next = p->next;
-		free(p);
-	}
-	o->packed_git = NULL;
-
-	hashmap_clear(&o->pack_map);
-}
-
 void parsed_object_pool_reset_commit_grafts(struct parsed_object_pool *o)
 {
 	for (int i = 0; i < o->grafts_nr; i++) {

-- 
2.49.0.777.g153de2bbd5.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH v2 8/9] object-store: remove global array of cached objects
  2025-04-11  9:29 ` [PATCH v2 " Patrick Steinhardt
                     ` (6 preceding siblings ...)
  2025-04-11  9:29   ` [PATCH v2 7/9] object: split out functions relating to object store subsystem Patrick Steinhardt
@ 2025-04-11  9:29   ` Patrick Steinhardt
  2025-04-11 22:58     ` Junio C Hamano
  2025-04-11  9:29   ` [PATCH v2 9/9] object-store: merge "object-store-ll.h" and "object-store.h" Patrick Steinhardt
  8 siblings, 1 reply; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-11  9:29 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Junio C Hamano

Cached objects are virtual objects that can be set up without writing
anything into the object store directly. This mechanism for example
allows us to create fake commits in git-blame(1).

The cached objects are stored in a global variable. Refactor the code so
that we instead store the array as part of the raw object store. This is
another step into the direction of libifying our object database.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 blame.c           |  2 +-
 object-store-ll.h | 14 +++++++++++++-
 object-store.c    | 39 +++++++++++++++++++++++----------------
 3 files changed, 37 insertions(+), 18 deletions(-)

diff --git a/blame.c b/blame.c
index 703dab43e78..b7c5bd692e6 100644
--- a/blame.c
+++ b/blame.c
@@ -277,7 +277,7 @@ static struct commit *fake_working_tree_commit(struct repository *r,
 	convert_to_git(r->index, path, buf.buf, buf.len, &buf, 0);
 	origin->file.ptr = buf.buf;
 	origin->file.size = buf.len;
-	pretend_object_file(buf.buf, buf.len, OBJ_BLOB, &origin->blob_oid);
+	pretend_object_file(the_repository, buf.buf, buf.len, OBJ_BLOB, &origin->blob_oid);
 
 	/*
 	 * Read the current index, replace the path entry with
diff --git a/object-store-ll.h b/object-store-ll.h
index 8bb0f33f9a8..bb5e8798a1b 100644
--- a/object-store-ll.h
+++ b/object-store-ll.h
@@ -151,6 +151,8 @@ static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
 	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
 }
 
+struct cached_object_entry;
+
 struct raw_object_store {
 	/*
 	 * Set of all object directories; the main directory is first (and
@@ -203,6 +205,15 @@ struct raw_object_store {
 		unsigned flags;
 	} kept_pack_cache;
 
+	/*
+	 * This is meant to hold a *small* number of objects that you would
+	 * want repo_read_object_file() to be able to return, but yet you do not want
+	 * to write them into the object store (e.g. a browse-only
+	 * application).
+	 */
+	struct cached_object_entry *cached_objects;
+	size_t cached_object_nr, cached_object_alloc;
+
 	/*
 	 * A map of packfiles to packed_git structs for tracking which
 	 * packs have been loaded already.
@@ -272,7 +283,8 @@ void hash_object_file(const struct git_hash_algo *algo, const void *buf,
  * object in persistent storage before writing any other new objects
  * that reference it.
  */
-int pretend_object_file(void *, unsigned long, enum object_type,
+int pretend_object_file(struct repository *repo,
+			void *buf, unsigned long len, enum object_type type,
 			struct object_id *oid);
 
 struct object_info {
diff --git a/object-store.c b/object-store.c
index ea2d86c429b..17fa06a86fa 100644
--- a/object-store.c
+++ b/object-store.c
@@ -30,31 +30,31 @@
  * to write them into the object store (e.g. a browse-only
  * application).
  */
-static struct cached_object_entry {
+struct cached_object_entry {
 	struct object_id oid;
 	struct cached_object {
 		enum object_type type;
 		const void *buf;
 		unsigned long size;
 	} value;
-} *cached_objects;
-static int cached_object_nr, cached_object_alloc;
+};
 
-static const struct cached_object *find_cached_object(const struct object_id *oid)
+static const struct cached_object *find_cached_object(struct raw_object_store *object_store,
+						      const struct object_id *oid)
 {
 	static const struct cached_object empty_tree = {
 		.type = OBJ_TREE,
 		.buf = "",
 	};
-	int i;
-	const struct cached_object_entry *co = cached_objects;
+	const struct cached_object_entry *co = object_store->cached_objects;
 
-	for (i = 0; i < cached_object_nr; i++, co++) {
+	for (size_t i = 0; i < object_store->cached_object_nr; i++, co++)
 		if (oideq(&co->oid, oid))
 			return &co->value;
-	}
-	if (oideq(oid, the_hash_algo->empty_tree))
+
+	if (oid->algo && oideq(oid, hash_algos[oid->algo].empty_tree))
 		return &empty_tree;
+
 	return NULL;
 }
 
@@ -650,7 +650,7 @@ static int do_oid_object_info_extended(struct repository *r,
 	if (!oi)
 		oi = &blank_oi;
 
-	co = find_cached_object(real);
+	co = find_cached_object(r->objects, real);
 	if (co) {
 		if (oi->typep)
 			*(oi->typep) = co->type;
@@ -853,18 +853,21 @@ int oid_object_info(struct repository *r,
 	return type;
 }
 
-int pretend_object_file(void *buf, unsigned long len, enum object_type type,
+int pretend_object_file(struct repository *repo,
+			void *buf, unsigned long len, enum object_type type,
 			struct object_id *oid)
 {
 	struct cached_object_entry *co;
 	char *co_buf;
 
-	hash_object_file(the_hash_algo, buf, len, type, oid);
-	if (repo_has_object_file_with_flags(the_repository, oid, OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT) ||
-	    find_cached_object(oid))
+	hash_object_file(repo->hash_algo, buf, len, type, oid);
+	if (repo_has_object_file_with_flags(repo, oid, OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT) ||
+	    find_cached_object(repo->objects, oid))
 		return 0;
-	ALLOC_GROW(cached_objects, cached_object_nr + 1, cached_object_alloc);
-	co = &cached_objects[cached_object_nr++];
+
+	ALLOC_GROW(repo->objects->cached_objects,
+		   repo->objects->cached_object_nr + 1, repo->objects->cached_object_alloc);
+	co = &repo->objects->cached_objects[repo->objects->cached_object_nr++];
 	co->value.size = len;
 	co->value.type = type;
 	co_buf = xmalloc(len);
@@ -1021,6 +1024,10 @@ void raw_object_store_clear(struct raw_object_store *o)
 	o->odb_tail = NULL;
 	o->loaded_alternates = 0;
 
+	for (size_t i = 0; i < o->cached_object_nr; i++)
+		free((char *) o->cached_objects[i].value.buf);
+	FREE_AND_NULL(o->cached_objects);
+
 	INIT_LIST_HEAD(&o->packed_git_mru);
 	close_object_store(o);
 

-- 
2.49.0.777.g153de2bbd5.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH v2 9/9] object-store: merge "object-store-ll.h" and "object-store.h"
  2025-04-11  9:29 ` [PATCH v2 " Patrick Steinhardt
                     ` (7 preceding siblings ...)
  2025-04-11  9:29   ` [PATCH v2 8/9] object-store: remove global array of cached objects Patrick Steinhardt
@ 2025-04-11  9:29   ` Patrick Steinhardt
  8 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-11  9:29 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Junio C Hamano

The "object-store-ll.h" header has been introduced to keep transitive
header dependendcies and compile times at bay. Now that we have created
a new "object-store.c" file though we can easily move the last remaining
additional bit of "object-store.h", the `odb_path_map`, out of the
header.

Do so. As the "object-store.h" header is now equivalent to its low-level
alternative we drop the latter and inline it into the former.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 apply.c                       |   2 +-
 archive-tar.c                 |   2 +-
 archive-zip.c                 |   2 +-
 archive.c                     |   2 +-
 attr.c                        |   2 +-
 bisect.c                      |   2 +-
 blame.c                       |   2 +-
 builtin/backfill.c            |   2 +-
 builtin/blame.c               |   2 +-
 builtin/cat-file.c            |   2 +-
 builtin/checkout.c            |   2 +-
 builtin/clone.c               |   2 +-
 builtin/commit-graph.c        |   2 +-
 builtin/commit-tree.c         |   2 +-
 builtin/count-objects.c       |   2 +-
 builtin/describe.c            |   2 +-
 builtin/difftool.c            |   2 +-
 builtin/fast-export.c         |   2 +-
 builtin/fast-import.c         |   2 +-
 builtin/fetch.c               |   2 +-
 builtin/fsck.c                |   2 +-
 builtin/gc.c                  |   2 +-
 builtin/grep.c                |   2 +-
 builtin/hash-object.c         |   2 +-
 builtin/index-pack.c          |   2 +-
 builtin/log.c                 |   2 +-
 builtin/ls-tree.c             |   2 +-
 builtin/merge-tree.c          |   2 +-
 builtin/mktag.c               |   2 +-
 builtin/mktree.c              |   2 +-
 builtin/multi-pack-index.c    |   2 +-
 builtin/notes.c               |   2 +-
 builtin/pack-objects.c        |   2 +-
 builtin/pack-redundant.c      |   2 +-
 builtin/prune.c               |   2 +-
 builtin/receive-pack.c        |   2 +-
 builtin/remote.c              |   2 +-
 builtin/repack.c              |   2 +-
 builtin/replace.c             |   2 +-
 builtin/rev-list.c            |   2 +-
 builtin/show-ref.c            |   2 +-
 builtin/submodule--helper.c   |   2 +-
 builtin/tag.c                 |   2 +-
 builtin/unpack-file.c         |   2 +-
 builtin/unpack-objects.c      |   2 +-
 bulk-checkin.c                |   2 +-
 bundle-uri.c                  |   2 +-
 bundle.c                      |   2 +-
 cache-tree.c                  |   2 +-
 combine-diff.c                |   2 +-
 commit-graph.c                |   2 +-
 commit-graph.h                |   2 +-
 commit.c                      |   2 +-
 config.c                      |   2 +-
 connected.c                   |   2 +-
 convert.c                     |   2 +-
 diagnose.c                    |   2 +-
 diff.c                        |   2 +-
 diffcore-rename.c             |   2 +-
 dir.c                         |   2 +-
 entry.c                       |   2 +-
 fetch-pack.c                  |   2 +-
 fmt-merge-msg.c               |   2 +-
 fsck.c                        |   2 +-
 grep.c                        |   2 +-
 http-backend.c                |   2 +-
 http-push.c                   |   2 +-
 http-walker.c                 |   2 +-
 http.c                        |   2 +-
 list-objects-filter.c         |   2 +-
 list-objects.c                |   2 +-
 log-tree.c                    |   2 +-
 mailmap.c                     |   2 +-
 merge-blobs.c                 |   2 +-
 merge-recursive.c             |   2 +-
 notes-merge.c                 |   2 +-
 object-file.c                 |   1 +
 object-name.c                 |   2 +-
 object-store-ll.h             | 517 ------------------------------------------
 object-store.c                |   5 +
 object-store.h                | 516 ++++++++++++++++++++++++++++++++++++++++-
 oss-fuzz/fuzz-pack-idx.c      |   2 +-
 pack-bitmap-write.c           |   2 +-
 pack-bitmap.c                 |   2 +-
 pack-check.c                  |   2 +-
 pack-mtimes.c                 |   2 +-
 pack-objects.h                |   2 +-
 pack-revindex.c               |   2 +-
 packfile.c                    |   2 +-
 path.c                        |   2 +-
 promisor-remote.c             |   2 +-
 protocol-caps.c               |   2 +-
 prune-packed.c                |   2 +-
 reachable.c                   |   2 +-
 read-cache.c                  |   2 +-
 ref-filter.c                  |   2 +-
 reflog.c                      |   2 +-
 refs.c                        |   2 +-
 remote.c                      |   2 +-
 replace-object.c              |   2 +-
 replace-object.h              |   2 +-
 repository.c                  |   2 +-
 rerere.c                      |   2 +-
 revision.c                    |   2 +-
 send-pack.c                   |   2 +-
 sequencer.c                   |   2 +-
 server-info.c                 |   2 +-
 shallow.c                     |   2 +-
 streaming.c                   |   2 +-
 submodule-config.c            |   2 +-
 submodule.c                   |   2 +-
 t/helper/test-pack-mtimes.c   |   2 +-
 t/helper/test-partial-clone.c |   2 +-
 t/helper/test-read-graph.c    |   2 +-
 t/helper/test-read-midx.c     |   2 +-
 t/helper/test-ref-store.c     |   2 +-
 tag.c                         |   2 +-
 tmp-objdir.c                  |   2 +-
 tree-walk.c                   |   2 +-
 tree.c                        |   2 +-
 unpack-trees.c                |   2 +-
 upload-pack.c                 |   2 +-
 walker.c                      |   2 +-
 xdiff-interface.c             |   2 +-
 124 files changed, 637 insertions(+), 642 deletions(-)

diff --git a/apply.c b/apply.c
index f274a379487..2b6f4d0af87 100644
--- a/apply.c
+++ b/apply.c
@@ -14,7 +14,7 @@
 #include "abspath.h"
 #include "base85.h"
 #include "config.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "delta.h"
 #include "diff.h"
 #include "dir.h"
diff --git a/archive-tar.c b/archive-tar.c
index 0edf13fba75..282b48196f9 100644
--- a/archive-tar.c
+++ b/archive-tar.c
@@ -11,7 +11,7 @@
 #include "hex.h"
 #include "tar.h"
 #include "archive.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "strbuf.h"
 #include "streaming.h"
 #include "run-command.h"
diff --git a/archive-zip.c b/archive-zip.c
index 9f32730181b..405da6f3d83 100644
--- a/archive-zip.c
+++ b/archive-zip.c
@@ -12,7 +12,7 @@
 #include "hex.h"
 #include "streaming.h"
 #include "utf8.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "strbuf.h"
 #include "userdiff.h"
 #include "write-or-die.h"
diff --git a/archive.c b/archive.c
index c95e3981524..014c312178c 100644
--- a/archive.c
+++ b/archive.c
@@ -14,7 +14,7 @@
 #include "pretty.h"
 #include "setup.h"
 #include "refs.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tree.h"
 #include "tree-walk.h"
diff --git a/attr.c b/attr.c
index 0bd2750528f..86b6109fc4e 100644
--- a/attr.c
+++ b/attr.c
@@ -22,7 +22,7 @@
 #include "read-cache-ll.h"
 #include "refs.h"
 #include "revision.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "setup.h"
 #include "thread-utils.h"
 #include "tree-walk.h"
diff --git a/bisect.c b/bisect.c
index 269a98bf978..a327468c75b 100644
--- a/bisect.c
+++ b/bisect.c
@@ -20,7 +20,7 @@
 #include "commit-slab.h"
 #include "commit-reach.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "dir.h"
 
diff --git a/blame.c b/blame.c
index b7c5bd692e6..57daa45e899 100644
--- a/blame.c
+++ b/blame.c
@@ -3,7 +3,7 @@
 
 #include "git-compat-util.h"
 #include "refs.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "cache-tree.h"
 #include "mergesort.h"
 #include "commit.h"
diff --git a/builtin/backfill.c b/builtin/backfill.c
index 33e1ea2f84f..aaa104bc91d 100644
--- a/builtin/backfill.c
+++ b/builtin/backfill.c
@@ -13,7 +13,7 @@
 #include "tree.h"
 #include "tree-walk.h"
 #include "object.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "oidset.h"
 #include "promisor-remote.h"
diff --git a/builtin/blame.c b/builtin/blame.c
index c470654c7ec..4e156bfd19d 100644
--- a/builtin/blame.c
+++ b/builtin/blame.c
@@ -28,7 +28,7 @@
 #include "line-log.h"
 #include "progress.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pager.h"
 #include "blame.h"
 #include "refs.h"
diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index b13561cf73b..c870fde260a 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -22,7 +22,7 @@
 #include "packfile.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "replace-object.h"
 #include "promisor-remote.h"
 #include "mailmap.h"
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 3e68623838a..8136962e2b3 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -20,7 +20,7 @@
 #include "merge-recursive.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "parse-options.h"
 #include "path.h"
 #include "preload-index.h"
diff --git a/builtin/clone.c b/builtin/clone.c
index 2993acb630e..d0423a2198b 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -25,7 +25,7 @@
 #include "refs.h"
 #include "refspec.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "tree.h"
 #include "tree-walk.h"
 #include "unpack-trees.h"
diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c
index 8ca75262c59..be06d0a811b 100644
--- a/builtin/commit-graph.c
+++ b/builtin/commit-graph.c
@@ -6,7 +6,7 @@
 #include "hex.h"
 #include "parse-options.h"
 #include "commit-graph.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "progress.h"
 #include "replace-object.h"
 #include "strbuf.h"
diff --git a/builtin/commit-tree.c b/builtin/commit-tree.c
index 38457600a4e..6f9975e7a88 100644
--- a/builtin/commit-tree.c
+++ b/builtin/commit-tree.c
@@ -9,7 +9,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 #include "commit.h"
 #include "parse-options.h"
diff --git a/builtin/count-objects.c b/builtin/count-objects.c
index 1e89148ed74..0bb5360b2f2 100644
--- a/builtin/count-objects.c
+++ b/builtin/count-objects.c
@@ -12,7 +12,7 @@
 #include "parse-options.h"
 #include "quote.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 static unsigned long garbage;
 static off_t size_garbage;
diff --git a/builtin/describe.c b/builtin/describe.c
index 23df333fd04..0f87fbceef3 100644
--- a/builtin/describe.c
+++ b/builtin/describe.c
@@ -19,7 +19,7 @@
 #include "setup.h"
 #include "strvec.h"
 #include "run-command.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "list-objects.h"
 #include "commit-slab.h"
 #include "wildmatch.h"
diff --git a/builtin/difftool.c b/builtin/difftool.c
index 45d6ea8a801..e6e45436531 100644
--- a/builtin/difftool.c
+++ b/builtin/difftool.c
@@ -29,7 +29,7 @@
 #include "strbuf.h"
 #include "lockfile.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "dir.h"
 #include "entry.h"
 #include "setup.h"
diff --git a/builtin/fast-export.c b/builtin/fast-export.c
index 170126d41ac..afacd228b5d 100644
--- a/builtin/fast-export.c
+++ b/builtin/fast-export.c
@@ -14,7 +14,7 @@
 #include "refs.h"
 #include "refspec.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "object.h"
 #include "tag.h"
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index 63880b595cc..0357211ea6c 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -24,7 +24,7 @@
 #include "packfile.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "mem-pool.h"
 #include "commit-reach.h"
 #include "khash.h"
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 02af5054690..b52a32a5e0d 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -14,7 +14,7 @@
 #include "refs.h"
 #include "refspec.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oidset.h"
 #include "oid-array.h"
 #include "commit.h"
diff --git a/builtin/fsck.c b/builtin/fsck.c
index 32d40d8f9fc..47055ebaf9a 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -18,7 +18,7 @@
 #include "packfile.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "read-cache-ll.h"
 #include "replace-object.h"
diff --git a/builtin/gc.c b/builtin/gc.c
index b069629676c..def922e7145 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -29,7 +29,7 @@
 #include "commit.h"
 #include "commit-graph.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pack.h"
 #include "pack-objects.h"
 #include "path.h"
diff --git a/builtin/grep.c b/builtin/grep.c
index 283d64cab80..bcfbe5be5ba 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -26,7 +26,7 @@
 #include "submodule-config.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 #include "pager.h"
 #include "path.h"
diff --git a/builtin/hash-object.c b/builtin/hash-object.c
index 7aa889b1b4a..e4eddd63bf4 100644
--- a/builtin/hash-object.c
+++ b/builtin/hash-object.c
@@ -11,7 +11,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "blob.h"
 #include "quote.h"
 #include "parse-options.h"
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index de127c0ff13..60a8ee05dbc 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -21,7 +21,7 @@
 #include "packfile.h"
 #include "pack-revindex.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "oidset.h"
 #include "path.h"
diff --git a/builtin/log.c b/builtin/log.c
index 06ffaa93e86..72ac45e9433 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -16,7 +16,7 @@
 #include "hex.h"
 #include "refs.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pager.h"
 #include "color.h"
 #include "commit.h"
diff --git a/builtin/ls-tree.c b/builtin/ls-tree.c
index 8542b5d53e4..8aafc30ca48 100644
--- a/builtin/ls-tree.c
+++ b/builtin/ls-tree.c
@@ -10,7 +10,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "tree.h"
 #include "path.h"
 #include "quote.h"
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 3ec7127b3a6..4aafa73c615 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -10,7 +10,7 @@
 #include "commit-reach.h"
 #include "merge-ort.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "parse-options.h"
 #include "blob.h"
 #include "merge-blobs.h"
diff --git a/builtin/mktag.c b/builtin/mktag.c
index 6e188dce50a..7ac11c46d53 100644
--- a/builtin/mktag.c
+++ b/builtin/mktag.c
@@ -6,7 +6,7 @@
 #include "strbuf.h"
 #include "replace-object.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "fsck.h"
 #include "config.h"
 
diff --git a/builtin/mktree.c b/builtin/mktree.c
index 0644f951161..7ffe6eefd8a 100644
--- a/builtin/mktree.c
+++ b/builtin/mktree.c
@@ -12,7 +12,7 @@
 #include "tree.h"
 #include "parse-options.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 static struct treeent {
 	unsigned mode;
diff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c
index 2a938466f53..d98410ca6c6 100644
--- a/builtin/multi-pack-index.c
+++ b/builtin/multi-pack-index.c
@@ -7,7 +7,7 @@
 #include "midx.h"
 #include "strbuf.h"
 #include "trace2.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "replace-object.h"
 #include "repository.h"
 
diff --git a/builtin/notes.c b/builtin/notes.c
index 0dbc233752d..a3f433ca4c0 100644
--- a/builtin/notes.c
+++ b/builtin/notes.c
@@ -16,7 +16,7 @@
 #include "notes.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 
 #include "pretty.h"
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 163aab547fe..488c80f2cf3 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -32,7 +32,7 @@
 #include "list.h"
 #include "packfile.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "replace-object.h"
 #include "dir.h"
 #include "midx.h"
diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c
index 3febe732f8e..5d1fc781761 100644
--- a/builtin/pack-redundant.c
+++ b/builtin/pack-redundant.c
@@ -13,7 +13,7 @@
 #include "hex.h"
 
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "strbuf.h"
 
 #define BLKSIZE 512
diff --git a/builtin/prune.c b/builtin/prune.c
index 8f52da8bd66..e930caa0c0a 100644
--- a/builtin/prune.c
+++ b/builtin/prune.c
@@ -17,7 +17,7 @@
 #include "replace-object.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "shallow.h"
 
 static const char * const prune_usage[] = {
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index ee51bd76f60..be314879e82 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -33,7 +33,7 @@
 #include "packfile.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "protocol.h"
 #include "commit-reach.h"
diff --git a/builtin/remote.c b/builtin/remote.c
index 1b7aad88380..59481b3a82c 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -14,7 +14,7 @@
 #include "rebase.h"
 #include "refs.h"
 #include "refspec.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "strvec.h"
 #include "commit-reach.h"
 #include "progress.h"
diff --git a/builtin/repack.c b/builtin/repack.c
index f3330ade7b8..1fd2874324a 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -17,7 +17,7 @@
 #include "midx.h"
 #include "packfile.h"
 #include "prune-packed.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "promisor-remote.h"
 #include "shallow.h"
 #include "pack.h"
diff --git a/builtin/replace.c b/builtin/replace.c
index ce2948a9b2a..4204ca3605b 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -19,7 +19,7 @@
 #include "run-command.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "read-cache.h"
 #include "replace-object.h"
 #include "tag.h"
diff --git a/builtin/rev-list.c b/builtin/rev-list.c
index bb26bee0d45..0170d79b62f 100644
--- a/builtin/rev-list.c
+++ b/builtin/rev-list.c
@@ -14,7 +14,7 @@
 #include "object.h"
 #include "object-name.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pack-bitmap.h"
 #include "log-tree.h"
 #include "graph.h"
diff --git a/builtin/show-ref.c b/builtin/show-ref.c
index 285cd3e4338..f81209f23c3 100644
--- a/builtin/show-ref.c
+++ b/builtin/show-ref.c
@@ -5,7 +5,7 @@
 #include "hex.h"
 #include "refs/refs-internal.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "object.h"
 #include "string-list.h"
 #include "parse-options.h"
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 570226ea166..a8d1200b7cb 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -28,7 +28,7 @@
 #include "diff.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "advice.h"
 #include "branch.h"
 #include "list-objects-filter-options.h"
diff --git a/builtin/tag.c b/builtin/tag.c
index 536a01ff3ae..e6b372cebf5 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -19,7 +19,7 @@
 #include "refs.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "tag.h"
 #include "parse-options.h"
diff --git a/builtin/unpack-file.c b/builtin/unpack-file.c
index b19e5cabd03..e33acfc4ee4 100644
--- a/builtin/unpack-file.c
+++ b/builtin/unpack-file.c
@@ -4,7 +4,7 @@
 #include "hex.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 static char *create_temp_file(struct object_id *oid)
 {
diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c
index 4078eab9252..661be789f13 100644
--- a/builtin/unpack-objects.c
+++ b/builtin/unpack-objects.c
@@ -9,7 +9,7 @@
 #include "git-zlib.h"
 #include "hex.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "object.h"
 #include "delta.h"
 #include "pack.h"
diff --git a/bulk-checkin.c b/bulk-checkin.c
index 0b9bad92868..b38937d739b 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -19,7 +19,7 @@
 #include "tmp-objdir.h"
 #include "packfile.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 static int odb_transaction_nesting;
 
diff --git a/bundle-uri.c b/bundle-uri.c
index 744257c49c1..96d2ba726d9 100644
--- a/bundle-uri.c
+++ b/bundle-uri.c
@@ -14,7 +14,7 @@
 #include "fetch-pack.h"
 #include "remote.h"
 #include "trace2.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 static struct {
 	enum bundle_list_heuristic heuristic;
diff --git a/bundle.c b/bundle.c
index d7ad6908433..d661c4ec214 100644
--- a/bundle.c
+++ b/bundle.c
@@ -7,7 +7,7 @@
 #include "environment.h"
 #include "gettext.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "repository.h"
 #include "object.h"
 #include "commit.h"
diff --git a/cache-tree.c b/cache-tree.c
index 4c8167ea927..c0e1e9ee1d4 100644
--- a/cache-tree.c
+++ b/cache-tree.c
@@ -10,7 +10,7 @@
 #include "cache-tree.h"
 #include "bulk-checkin.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "read-cache-ll.h"
 #include "replace-object.h"
 #include "repository.h"
diff --git a/combine-diff.c b/combine-diff.c
index 553bf59fed6..dfae9f7995d 100644
--- a/combine-diff.c
+++ b/combine-diff.c
@@ -2,7 +2,7 @@
 #define DISABLE_SIGN_COMPARE_WARNINGS
 
 #include "git-compat-util.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "convert.h"
 #include "diff.h"
diff --git a/commit-graph.c b/commit-graph.c
index 8060c358b84..a36c2f1e303 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -14,7 +14,7 @@
 #include "refs.h"
 #include "hash-lookup.h"
 #include "commit-graph.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "path.h"
 #include "alloc.h"
diff --git a/commit-graph.h b/commit-graph.h
index 67819401954..13f662827d4 100644
--- a/commit-graph.h
+++ b/commit-graph.h
@@ -1,7 +1,7 @@
 #ifndef COMMIT_GRAPH_H
 #define COMMIT_GRAPH_H
 
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oidset.h"
 
 #define GIT_TEST_COMMIT_GRAPH "GIT_TEST_COMMIT_GRAPH"
diff --git a/commit.c b/commit.c
index fbf4f8e87fd..00842678bd2 100644
--- a/commit.c
+++ b/commit.c
@@ -9,7 +9,7 @@
 #include "hex.h"
 #include "repository.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "utf8.h"
 #include "diff.h"
 #include "revision.h"
diff --git a/config.c b/config.c
index accb47e2d18..b18b5617fcd 100644
--- a/config.c
+++ b/config.c
@@ -31,7 +31,7 @@
 #include "hashmap.h"
 #include "string-list.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pager.h"
 #include "path.h"
 #include "utf8.h"
diff --git a/connected.c b/connected.c
index 3099da84f33..4415388beba 100644
--- a/connected.c
+++ b/connected.c
@@ -3,7 +3,7 @@
 #include "git-compat-util.h"
 #include "gettext.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "run-command.h"
 #include "sigchain.h"
 #include "connected.h"
diff --git a/convert.c b/convert.c
index 9cc0ca20ca0..8783e17941f 100644
--- a/convert.c
+++ b/convert.c
@@ -8,7 +8,7 @@
 #include "copy.h"
 #include "gettext.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "attr.h"
 #include "run-command.h"
 #include "quote.h"
diff --git a/diagnose.c b/diagnose.c
index bd485effea2..b1be74be983 100644
--- a/diagnose.c
+++ b/diagnose.c
@@ -7,7 +7,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "strvec.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 #include "parse-options.h"
 #include "repository.h"
diff --git a/diff.c b/diff.c
index c361199ab1b..9f2042fd1a5 100644
--- a/diff.c
+++ b/diff.c
@@ -23,7 +23,7 @@
 #include "color.h"
 #include "run-command.h"
 #include "utf8.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "userdiff.h"
 #include "submodule.h"
 #include "hashmap.h"
diff --git a/diffcore-rename.c b/diffcore-rename.c
index 5002e896aad..787a2cef5f4 100644
--- a/diffcore-rename.c
+++ b/diffcore-rename.c
@@ -8,7 +8,7 @@
 #include "git-compat-util.h"
 #include "diff.h"
 #include "diffcore.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "hashmap.h"
 #include "mem-pool.h"
 #include "oid-array.h"
diff --git a/dir.c b/dir.c
index 16ae3b5169d..1f8b6e2aa67 100644
--- a/dir.c
+++ b/dir.c
@@ -17,7 +17,7 @@
 #include "environment.h"
 #include "gettext.h"
 #include "name-hash.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "refs.h"
 #include "repository.h"
diff --git a/entry.c b/entry.c
index 81b321e53d1..f36ec5ad242 100644
--- a/entry.c
+++ b/entry.c
@@ -1,7 +1,7 @@
 #define USE_THE_REPOSITORY_VARIABLE
 
 #include "git-compat-util.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "dir.h"
 #include "environment.h"
 #include "gettext.h"
diff --git a/fetch-pack.c b/fetch-pack.c
index 1ed5e11dd56..210dc30d50f 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -24,7 +24,7 @@
 #include "oid-array.h"
 #include "oidset.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "connected.h"
 #include "fetch-negotiator.h"
diff --git a/fmt-merge-msg.c b/fmt-merge-msg.c
index 5b63c3b088a..501b5acdd44 100644
--- a/fmt-merge-msg.c
+++ b/fmt-merge-msg.c
@@ -6,7 +6,7 @@
 #include "environment.h"
 #include "refs.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "diff.h"
 #include "diff-merges.h"
 #include "hex.h"
diff --git a/fsck.c b/fsck.c
index 9fc4c25ffd5..8dc8472ceb3 100644
--- a/fsck.c
+++ b/fsck.c
@@ -4,7 +4,7 @@
 #include "date.h"
 #include "dir.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "repository.h"
 #include "object.h"
diff --git a/grep.c b/grep.c
index 9284b5741f5..f8d535182c3 100644
--- a/grep.c
+++ b/grep.c
@@ -5,7 +5,7 @@
 #include "gettext.h"
 #include "grep.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pretty.h"
 #include "userdiff.h"
 #include "xdiff-interface.h"
diff --git a/http-backend.c b/http-backend.c
index 50b2858fad6..0c575aa88aa 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -18,7 +18,7 @@
 #include "url.h"
 #include "strvec.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "protocol.h"
 #include "date.h"
 #include "write-or-die.h"
diff --git a/http-push.c b/http-push.c
index 806eb67cf1b..32e37565f4e 100644
--- a/http-push.c
+++ b/http-push.c
@@ -20,7 +20,7 @@
 #include "url.h"
 #include "packfile.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit-reach.h"
 
 #ifdef EXPAT_NEEDS_XMLPARSE_H
diff --git a/http-walker.c b/http-walker.c
index 7918ddc0968..882cae19c24 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -9,7 +9,7 @@
 #include "list.h"
 #include "transport.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 struct alt_base {
 	char *base;
diff --git a/http.c b/http.c
index 0c9a872809f..bc18ff83c4b 100644
--- a/http.c
+++ b/http.c
@@ -19,7 +19,7 @@
 #include "packfile.h"
 #include "string-list.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "tempfile.h"
 
 static struct trace_key trace_curl = TRACE_KEY_INIT(CURL);
diff --git a/list-objects-filter.c b/list-objects-filter.c
index dc598a081bb..7765761b3c6 100644
--- a/list-objects-filter.c
+++ b/list-objects-filter.c
@@ -12,7 +12,7 @@
 #include "oidmap.h"
 #include "oidset.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 /* Remember to update object flag allocation in object.h */
 /*
diff --git a/list-objects.c b/list-objects.c
index 943e62e868f..1e5512e1318 100644
--- a/list-objects.c
+++ b/list-objects.c
@@ -14,7 +14,7 @@
 #include "list-objects-filter.h"
 #include "list-objects-filter-options.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "trace.h"
 #include "environment.h"
 
diff --git a/log-tree.c b/log-tree.c
index 5dd1b63076f..a4d4ab59ca0 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -9,7 +9,7 @@
 #include "environment.h"
 #include "hex.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "repository.h"
 #include "tmp-objdir.h"
 #include "commit.h"
diff --git a/mailmap.c b/mailmap.c
index f35d20ed7fd..9e2642a043b 100644
--- a/mailmap.c
+++ b/mailmap.c
@@ -6,7 +6,7 @@
 #include "string-list.h"
 #include "mailmap.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "setup.h"
 
 char *git_mailmap_file;
diff --git a/merge-blobs.c b/merge-blobs.c
index 0ad0390fea5..53f36dbc175 100644
--- a/merge-blobs.c
+++ b/merge-blobs.c
@@ -4,7 +4,7 @@
 #include "merge-ll.h"
 #include "blob.h"
 #include "merge-blobs.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 static int fill_mmfile_blob(mmfile_t *f, struct blob *obj)
 {
diff --git a/merge-recursive.c b/merge-recursive.c
index 9aedffc546b..981e57698ed 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -27,7 +27,7 @@
 #include "name-hash.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "repository.h"
 #include "revision.h"
diff --git a/notes-merge.c b/notes-merge.c
index 13750490eff..96e59322a8b 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -8,7 +8,7 @@
 #include "refs.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "read-cache.h"
 #include "repository.h"
diff --git a/object-file.c b/object-file.c
index 235d29858ff..4abca6cb613 100644
--- a/object-file.c
+++ b/object-file.c
@@ -12,6 +12,7 @@
 
 #include "git-compat-util.h"
 #include "bulk-checkin.h"
+#include "dir.h"
 #include "environment.h"
 #include "gettext.h"
 #include "hex.h"
diff --git a/object-name.c b/object-name.c
index 91f731373a1..2c751a5352a 100644
--- a/object-name.c
+++ b/object-name.c
@@ -19,7 +19,7 @@
 #include "oidtree.h"
 #include "packfile.h"
 #include "pretty.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "read-cache-ll.h"
 #include "repo-settings.h"
 #include "repository.h"
diff --git a/object-store-ll.h b/object-store-ll.h
deleted file mode 100644
index bb5e8798a1b..00000000000
--- a/object-store-ll.h
+++ /dev/null
@@ -1,517 +0,0 @@
-#ifndef OBJECT_STORE_LL_H
-#define OBJECT_STORE_LL_H
-
-#include "hashmap.h"
-#include "object.h"
-#include "list.h"
-#include "thread-utils.h"
-#include "oidset.h"
-
-struct oidmap;
-struct oidtree;
-struct strbuf;
-struct repository;
-
-struct object_directory {
-	struct object_directory *next;
-
-	/*
-	 * Used to store the results of readdir(3) calls when we are OK
-	 * sacrificing accuracy due to races for speed. That includes
-	 * object existence with OBJECT_INFO_QUICK, as well as
-	 * our search for unique abbreviated hashes. Don't use it for tasks
-	 * requiring greater accuracy!
-	 *
-	 * Be sure to call odb_load_loose_cache() before using.
-	 */
-	uint32_t loose_objects_subdir_seen[8]; /* 256 bits */
-	struct oidtree *loose_objects_cache;
-
-	/* Map between object IDs for loose objects. */
-	struct loose_object_map *loose_map;
-
-	/*
-	 * This is a temporary object store created by the tmp_objdir
-	 * facility. Disable ref updates since the objects in the store
-	 * might be discarded on rollback.
-	 */
-	int disable_ref_updates;
-
-	/*
-	 * This object store is ephemeral, so there is no need to fsync.
-	 */
-	int will_destroy;
-
-	/*
-	 * Path to the alternative object store. If this is a relative path,
-	 * it is relative to the current working directory.
-	 */
-	char *path;
-};
-
-void prepare_alt_odb(struct repository *r);
-int has_alt_odb(struct repository *r);
-char *compute_alternate_path(const char *path, struct strbuf *err);
-struct object_directory *find_odb(struct repository *r, const char *obj_dir);
-typedef int alt_odb_fn(struct object_directory *, void *);
-int foreach_alt_odb(alt_odb_fn, void*);
-typedef void alternate_ref_fn(const struct object_id *oid, void *);
-void for_each_alternate_ref(alternate_ref_fn, void *);
-
-/*
- * Add the directory to the on-disk alternates file; the new entry will also
- * take effect in the current process.
- */
-void add_to_alternates_file(const char *dir);
-
-/*
- * Add the directory to the in-memory list of alternates (along with any
- * recursive alternates it points to), but do not modify the on-disk alternates
- * file.
- */
-void add_to_alternates_memory(const char *dir);
-
-/*
- * Replace the current writable object directory with the specified temporary
- * object directory; returns the former primary object directory.
- */
-struct object_directory *set_temporary_primary_odb(const char *dir, int will_destroy);
-
-/*
- * Restore a previous ODB replaced by set_temporary_main_odb.
- */
-void restore_primary_odb(struct object_directory *restore_odb, const char *old_path);
-
-/*
- * Populate and return the loose object cache array corresponding to the
- * given object ID.
- */
-struct oidtree *odb_loose_cache(struct object_directory *odb,
-				  const struct object_id *oid);
-
-/* Empty the loose object cache for the specified object directory. */
-void odb_clear_loose_cache(struct object_directory *odb);
-
-struct packed_git {
-	struct hashmap_entry packmap_ent;
-	struct packed_git *next;
-	struct list_head mru;
-	struct pack_window *windows;
-	off_t pack_size;
-	const void *index_data;
-	size_t index_size;
-	uint32_t num_objects;
-	size_t crc_offset;
-	struct oidset bad_objects;
-	int index_version;
-	time_t mtime;
-	int pack_fd;
-	int index;              /* for builtin/pack-objects.c */
-	unsigned pack_local:1,
-		 pack_keep:1,
-		 pack_keep_in_core:1,
-		 freshened:1,
-		 do_not_close:1,
-		 pack_promisor:1,
-		 multi_pack_index:1,
-		 is_cruft:1;
-	unsigned char hash[GIT_MAX_RAWSZ];
-	struct revindex_entry *revindex;
-	const uint32_t *revindex_data;
-	const uint32_t *revindex_map;
-	size_t revindex_size;
-	/*
-	 * mtimes_map points at the beginning of the memory mapped region of
-	 * this pack's corresponding .mtimes file, and mtimes_size is the size
-	 * of that .mtimes file
-	 */
-	const uint32_t *mtimes_map;
-	size_t mtimes_size;
-
-	/* repo denotes the repository this packfile belongs to */
-	struct repository *repo;
-
-	/* something like ".git/objects/pack/xxxxx.pack" */
-	char pack_name[FLEX_ARRAY]; /* more */
-};
-
-struct multi_pack_index;
-
-static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
-				     const struct hashmap_entry *entry,
-				     const struct hashmap_entry *entry2,
-				     const void *keydata)
-{
-	const char *key = keydata;
-	const struct packed_git *pg1, *pg2;
-
-	pg1 = container_of(entry, const struct packed_git, packmap_ent);
-	pg2 = container_of(entry2, const struct packed_git, packmap_ent);
-
-	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
-}
-
-struct cached_object_entry;
-
-struct raw_object_store {
-	/*
-	 * Set of all object directories; the main directory is first (and
-	 * cannot be NULL after initialization). Subsequent directories are
-	 * alternates.
-	 */
-	struct object_directory *odb;
-	struct object_directory **odb_tail;
-	struct kh_odb_path_map *odb_by_path;
-
-	int loaded_alternates;
-
-	/*
-	 * A list of alternate object directories loaded from the environment;
-	 * this should not generally need to be accessed directly, but will
-	 * populate the "odb" list when prepare_alt_odb() is run.
-	 */
-	char *alternate_db;
-
-	/*
-	 * Objects that should be substituted by other objects
-	 * (see git-replace(1)).
-	 */
-	struct oidmap *replace_map;
-	unsigned replace_map_initialized : 1;
-	pthread_mutex_t replace_mutex; /* protect object replace functions */
-
-	struct commit_graph *commit_graph;
-	unsigned commit_graph_attempted : 1; /* if loading has been attempted */
-
-	/*
-	 * private data
-	 *
-	 * should only be accessed directly by packfile.c and midx.c
-	 */
-	struct multi_pack_index *multi_pack_index;
-
-	/*
-	 * private data
-	 *
-	 * should only be accessed directly by packfile.c
-	 */
-
-	struct packed_git *packed_git;
-	/* A most-recently-used ordered version of the packed_git list. */
-	struct list_head packed_git_mru;
-
-	struct {
-		struct packed_git **packs;
-		unsigned flags;
-	} kept_pack_cache;
-
-	/*
-	 * This is meant to hold a *small* number of objects that you would
-	 * want repo_read_object_file() to be able to return, but yet you do not want
-	 * to write them into the object store (e.g. a browse-only
-	 * application).
-	 */
-	struct cached_object_entry *cached_objects;
-	size_t cached_object_nr, cached_object_alloc;
-
-	/*
-	 * A map of packfiles to packed_git structs for tracking which
-	 * packs have been loaded already.
-	 */
-	struct hashmap pack_map;
-
-	/*
-	 * A fast, rough count of the number of objects in the repository.
-	 * These two fields are not meant for direct access. Use
-	 * repo_approximate_object_count() instead.
-	 */
-	unsigned long approximate_object_count;
-	unsigned approximate_object_count_valid : 1;
-
-	/*
-	 * Whether packed_git has already been populated with this repository's
-	 * packs.
-	 */
-	unsigned packed_git_initialized : 1;
-};
-
-struct raw_object_store *raw_object_store_new(void);
-void raw_object_store_clear(struct raw_object_store *o);
-
-/*
- * Create a temporary file rooted in the object database directory, or
- * die on failure. The filename is taken from "pattern", which should have the
- * usual "XXXXXX" trailer, and the resulting filename is written into the
- * "template" buffer. Returns the open descriptor.
- */
-int odb_mkstemp(struct strbuf *temp_filename, const char *pattern);
-
-/*
- * Create a pack .keep file named "name" (which should generally be the output
- * of odb_pack_name). Returns a file descriptor opened for writing, or -1 on
- * error.
- */
-int odb_pack_keep(const char *name);
-
-/*
- * Put in `buf` the name of the file in the local object database that
- * would be used to store a loose object with the specified oid.
- */
-const char *loose_object_path(struct repository *r, struct strbuf *buf,
-			      const struct object_id *oid);
-
-void *map_loose_object(struct repository *r, const struct object_id *oid,
-		       unsigned long *size);
-
-void *repo_read_object_file(struct repository *r,
-			    const struct object_id *oid,
-			    enum object_type *type,
-			    unsigned long *size);
-
-/* Read and unpack an object file into memory, write memory to an object file */
-int oid_object_info(struct repository *r, const struct object_id *, unsigned long *);
-
-void hash_object_file(const struct git_hash_algo *algo, const void *buf,
-		      unsigned long len, enum object_type type,
-		      struct object_id *oid);
-
-/*
- * Add an object file to the in-memory object store, without writing it
- * to disk.
- *
- * Callers are responsible for calling write_object_file to record the
- * object in persistent storage before writing any other new objects
- * that reference it.
- */
-int pretend_object_file(struct repository *repo,
-			void *buf, unsigned long len, enum object_type type,
-			struct object_id *oid);
-
-struct object_info {
-	/* Request */
-	enum object_type *typep;
-	unsigned long *sizep;
-	off_t *disk_sizep;
-	struct object_id *delta_base_oid;
-	struct strbuf *type_name;
-	void **contentp;
-
-	/* Response */
-	enum {
-		OI_CACHED,
-		OI_LOOSE,
-		OI_PACKED,
-		OI_DBCACHED
-	} whence;
-	union {
-		/*
-		 * struct {
-		 * 	... Nothing to expose in this case
-		 * } cached;
-		 * struct {
-		 * 	... Nothing to expose in this case
-		 * } loose;
-		 */
-		struct {
-			struct packed_git *pack;
-			off_t offset;
-			unsigned int is_delta;
-		} packed;
-	} u;
-};
-
-/*
- * Initializer for a "struct object_info" that wants no items. You may
- * also memset() the memory to all-zeroes.
- */
-#define OBJECT_INFO_INIT { 0 }
-
-/* Invoke lookup_replace_object() on the given hash */
-#define OBJECT_INFO_LOOKUP_REPLACE 1
-/* Allow reading from a loose object file of unknown/bogus type */
-#define OBJECT_INFO_ALLOW_UNKNOWN_TYPE 2
-/* Do not retry packed storage after checking packed and loose storage */
-#define OBJECT_INFO_QUICK 8
-/*
- * Do not attempt to fetch the object if missing (even if fetch_is_missing is
- * nonzero).
- */
-#define OBJECT_INFO_SKIP_FETCH_OBJECT 16
-/*
- * This is meant for bulk prefetching of missing blobs in a partial
- * clone. Implies OBJECT_INFO_SKIP_FETCH_OBJECT and OBJECT_INFO_QUICK
- */
-#define OBJECT_INFO_FOR_PREFETCH (OBJECT_INFO_SKIP_FETCH_OBJECT | OBJECT_INFO_QUICK)
-
-/* Die if object corruption (not just an object being missing) was detected. */
-#define OBJECT_INFO_DIE_IF_CORRUPT 32
-
-int oid_object_info_extended(struct repository *r,
-			     const struct object_id *,
-			     struct object_info *, unsigned flags);
-
-/* Retry packed storage after checking packed and loose storage */
-#define HAS_OBJECT_RECHECK_PACKED 1
-
-/*
- * Returns 1 if the object exists. This function will not lazily fetch objects
- * in a partial clone.
- */
-int has_object(struct repository *r, const struct object_id *oid,
-	       unsigned flags);
-
-/*
- * These macros and functions are deprecated. If checking existence for an
- * object that is likely to be missing and/or whose absence is relatively
- * inconsequential (or is consequential but the caller is prepared to handle
- * it), use has_object(), which has better defaults (no lazy fetch in a partial
- * clone and no rechecking of packed storage). In the unlikely event that a
- * caller needs to assert existence of an object that it fully expects to
- * exist, and wants to trigger a lazy fetch in a partial clone, use
- * oid_object_info_extended() with a NULL struct object_info.
- *
- * These functions can be removed once all callers have migrated to
- * has_object() and/or oid_object_info_extended().
- */
-int repo_has_object_file(struct repository *r, const struct object_id *oid);
-int repo_has_object_file_with_flags(struct repository *r,
-				    const struct object_id *oid, int flags);
-
-void assert_oid_type(const struct object_id *oid, enum object_type expect);
-
-/*
- * Enabling the object read lock allows multiple threads to safely call the
- * following functions in parallel: repo_read_object_file(),
- * read_object_with_reference(), oid_object_info() and oid_object_info_extended().
- *
- * obj_read_lock() and obj_read_unlock() may also be used to protect other
- * section which cannot execute in parallel with object reading. Since the used
- * lock is a recursive mutex, these sections can even contain calls to object
- * reading functions. However, beware that in these cases zlib inflation won't
- * be performed in parallel, losing performance.
- *
- * TODO: oid_object_info_extended()'s call stack has a recursive behavior. If
- * any of its callees end up calling it, this recursive call won't benefit from
- * parallel inflation.
- */
-void enable_obj_read_lock(void);
-void disable_obj_read_lock(void);
-
-extern int obj_read_use_lock;
-extern pthread_mutex_t obj_read_mutex;
-
-static inline void obj_read_lock(void)
-{
-	if(obj_read_use_lock)
-		pthread_mutex_lock(&obj_read_mutex);
-}
-
-static inline void obj_read_unlock(void)
-{
-	if(obj_read_use_lock)
-		pthread_mutex_unlock(&obj_read_mutex);
-}
-
-/*
- * Iterate over the files in the loose-object parts of the object
- * directory "path", triggering the following callbacks:
- *
- *  - loose_object is called for each loose object we find.
- *
- *  - loose_cruft is called for any files that do not appear to be
- *    loose objects. Note that we only look in the loose object
- *    directories "objects/[0-9a-f]{2}/", so we will not report
- *    "objects/foobar" as cruft.
- *
- *  - loose_subdir is called for each top-level hashed subdirectory
- *    of the object directory (e.g., "$OBJDIR/f0"). It is called
- *    after the objects in the directory are processed.
- *
- * Any callback that is NULL will be ignored. Callbacks returning non-zero
- * will end the iteration.
- *
- * In the "buf" variant, "path" is a strbuf which will also be used as a
- * scratch buffer, but restored to its original contents before
- * the function returns.
- */
-typedef int each_loose_object_fn(const struct object_id *oid,
-				 const char *path,
-				 void *data);
-typedef int each_loose_cruft_fn(const char *basename,
-				const char *path,
-				void *data);
-typedef int each_loose_subdir_fn(unsigned int nr,
-				 const char *path,
-				 void *data);
-int for_each_file_in_obj_subdir(unsigned int subdir_nr,
-				struct strbuf *path,
-				each_loose_object_fn obj_cb,
-				each_loose_cruft_fn cruft_cb,
-				each_loose_subdir_fn subdir_cb,
-				void *data);
-int for_each_loose_file_in_objdir(const char *path,
-				  each_loose_object_fn obj_cb,
-				  each_loose_cruft_fn cruft_cb,
-				  each_loose_subdir_fn subdir_cb,
-				  void *data);
-int for_each_loose_file_in_objdir_buf(struct strbuf *path,
-				      each_loose_object_fn obj_cb,
-				      each_loose_cruft_fn cruft_cb,
-				      each_loose_subdir_fn subdir_cb,
-				      void *data);
-
-/* Flags for for_each_*_object() below. */
-enum for_each_object_flags {
-	/* Iterate only over local objects, not alternates. */
-	FOR_EACH_OBJECT_LOCAL_ONLY = (1<<0),
-
-	/* Only iterate over packs obtained from the promisor remote. */
-	FOR_EACH_OBJECT_PROMISOR_ONLY = (1<<1),
-
-	/*
-	 * Visit objects within a pack in packfile order rather than .idx order
-	 */
-	FOR_EACH_OBJECT_PACK_ORDER = (1<<2),
-
-	/* Only iterate over packs that are not marked as kept in-core. */
-	FOR_EACH_OBJECT_SKIP_IN_CORE_KEPT_PACKS = (1<<3),
-
-	/* Only iterate over packs that do not have .keep files. */
-	FOR_EACH_OBJECT_SKIP_ON_DISK_KEPT_PACKS = (1<<4),
-};
-
-/*
- * Iterate over all accessible loose objects without respect to
- * reachability. By default, this includes both local and alternate objects.
- * The order in which objects are visited is unspecified.
- *
- * Any flags specific to packs are ignored.
- */
-int for_each_loose_object(each_loose_object_fn, void *,
-			  enum for_each_object_flags flags);
-
-/*
- * Iterate over all accessible packed objects without respect to reachability.
- * By default, this includes both local and alternate packs.
- *
- * Note that some objects may appear twice if they are found in multiple packs.
- * Each pack is visited in an unspecified order. By default, objects within a
- * pack are visited in pack-idx order (i.e., sorted by oid).
- */
-typedef int each_packed_object_fn(const struct object_id *oid,
-				  struct packed_git *pack,
-				  uint32_t pos,
-				  void *data);
-int for_each_object_in_pack(struct packed_git *p,
-			    each_packed_object_fn, void *data,
-			    enum for_each_object_flags flags);
-int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
-			   void *data, enum for_each_object_flags flags);
-
-void *read_object_with_reference(struct repository *r,
-				 const struct object_id *oid,
-				 enum object_type required_type,
-				 unsigned long *size,
-				 struct object_id *oid_ret);
-
-#endif /* OBJECT_STORE_LL_H */
diff --git a/object-store.c b/object-store.c
index 17fa06a86fa..fc48925d230 100644
--- a/object-store.c
+++ b/object-store.c
@@ -4,9 +4,11 @@
 #include "abspath.h"
 #include "commit-graph.h"
 #include "config.h"
+#include "dir.h"
 #include "environment.h"
 #include "gettext.h"
 #include "hex.h"
+#include "khash.h"
 #include "lockfile.h"
 #include "loose.h"
 #include "object-file-convert.h"
@@ -24,6 +26,9 @@
 #include "submodule.h"
 #include "write-or-die.h"
 
+KHASH_INIT(odb_path_map, const char * /* key: odb_path */,
+	struct object_directory *, 1, fspathhash, fspatheq)
+
 /*
  * This is meant to hold a *small* number of objects that you would
  * want repo_read_object_file() to be able to return, but yet you do not want
diff --git a/object-store.h b/object-store.h
index 1b3e3d7d014..46961dc9542 100644
--- a/object-store.h
+++ b/object-store.h
@@ -1,11 +1,517 @@
 #ifndef OBJECT_STORE_H
 #define OBJECT_STORE_H
 
-#include "khash.h"
-#include "dir.h"
-#include "object-store-ll.h"
+#include "hashmap.h"
+#include "object.h"
+#include "list.h"
+#include "oidset.h"
+#include "thread-utils.h"
 
-KHASH_INIT(odb_path_map, const char * /* key: odb_path */,
-	struct object_directory *, 1, fspathhash, fspatheq)
+struct oidmap;
+struct oidtree;
+struct strbuf;
+struct repository;
+
+struct object_directory {
+	struct object_directory *next;
+
+	/*
+	 * Used to store the results of readdir(3) calls when we are OK
+	 * sacrificing accuracy due to races for speed. That includes
+	 * object existence with OBJECT_INFO_QUICK, as well as
+	 * our search for unique abbreviated hashes. Don't use it for tasks
+	 * requiring greater accuracy!
+	 *
+	 * Be sure to call odb_load_loose_cache() before using.
+	 */
+	uint32_t loose_objects_subdir_seen[8]; /* 256 bits */
+	struct oidtree *loose_objects_cache;
+
+	/* Map between object IDs for loose objects. */
+	struct loose_object_map *loose_map;
+
+	/*
+	 * This is a temporary object store created by the tmp_objdir
+	 * facility. Disable ref updates since the objects in the store
+	 * might be discarded on rollback.
+	 */
+	int disable_ref_updates;
+
+	/*
+	 * This object store is ephemeral, so there is no need to fsync.
+	 */
+	int will_destroy;
+
+	/*
+	 * Path to the alternative object store. If this is a relative path,
+	 * it is relative to the current working directory.
+	 */
+	char *path;
+};
+
+void prepare_alt_odb(struct repository *r);
+int has_alt_odb(struct repository *r);
+char *compute_alternate_path(const char *path, struct strbuf *err);
+struct object_directory *find_odb(struct repository *r, const char *obj_dir);
+typedef int alt_odb_fn(struct object_directory *, void *);
+int foreach_alt_odb(alt_odb_fn, void*);
+typedef void alternate_ref_fn(const struct object_id *oid, void *);
+void for_each_alternate_ref(alternate_ref_fn, void *);
+
+/*
+ * Add the directory to the on-disk alternates file; the new entry will also
+ * take effect in the current process.
+ */
+void add_to_alternates_file(const char *dir);
+
+/*
+ * Add the directory to the in-memory list of alternates (along with any
+ * recursive alternates it points to), but do not modify the on-disk alternates
+ * file.
+ */
+void add_to_alternates_memory(const char *dir);
+
+/*
+ * Replace the current writable object directory with the specified temporary
+ * object directory; returns the former primary object directory.
+ */
+struct object_directory *set_temporary_primary_odb(const char *dir, int will_destroy);
+
+/*
+ * Restore a previous ODB replaced by set_temporary_main_odb.
+ */
+void restore_primary_odb(struct object_directory *restore_odb, const char *old_path);
+
+/*
+ * Populate and return the loose object cache array corresponding to the
+ * given object ID.
+ */
+struct oidtree *odb_loose_cache(struct object_directory *odb,
+				  const struct object_id *oid);
+
+/* Empty the loose object cache for the specified object directory. */
+void odb_clear_loose_cache(struct object_directory *odb);
+
+struct packed_git {
+	struct hashmap_entry packmap_ent;
+	struct packed_git *next;
+	struct list_head mru;
+	struct pack_window *windows;
+	off_t pack_size;
+	const void *index_data;
+	size_t index_size;
+	uint32_t num_objects;
+	size_t crc_offset;
+	struct oidset bad_objects;
+	int index_version;
+	time_t mtime;
+	int pack_fd;
+	int index;              /* for builtin/pack-objects.c */
+	unsigned pack_local:1,
+		 pack_keep:1,
+		 pack_keep_in_core:1,
+		 freshened:1,
+		 do_not_close:1,
+		 pack_promisor:1,
+		 multi_pack_index:1,
+		 is_cruft:1;
+	unsigned char hash[GIT_MAX_RAWSZ];
+	struct revindex_entry *revindex;
+	const uint32_t *revindex_data;
+	const uint32_t *revindex_map;
+	size_t revindex_size;
+	/*
+	 * mtimes_map points at the beginning of the memory mapped region of
+	 * this pack's corresponding .mtimes file, and mtimes_size is the size
+	 * of that .mtimes file
+	 */
+	const uint32_t *mtimes_map;
+	size_t mtimes_size;
+
+	/* repo denotes the repository this packfile belongs to */
+	struct repository *repo;
+
+	/* something like ".git/objects/pack/xxxxx.pack" */
+	char pack_name[FLEX_ARRAY]; /* more */
+};
+
+struct multi_pack_index;
+
+static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
+				     const struct hashmap_entry *entry,
+				     const struct hashmap_entry *entry2,
+				     const void *keydata)
+{
+	const char *key = keydata;
+	const struct packed_git *pg1, *pg2;
+
+	pg1 = container_of(entry, const struct packed_git, packmap_ent);
+	pg2 = container_of(entry2, const struct packed_git, packmap_ent);
+
+	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
+}
+
+struct cached_object_entry;
+
+struct raw_object_store {
+	/*
+	 * Set of all object directories; the main directory is first (and
+	 * cannot be NULL after initialization). Subsequent directories are
+	 * alternates.
+	 */
+	struct object_directory *odb;
+	struct object_directory **odb_tail;
+	struct kh_odb_path_map *odb_by_path;
+
+	int loaded_alternates;
+
+	/*
+	 * A list of alternate object directories loaded from the environment;
+	 * this should not generally need to be accessed directly, but will
+	 * populate the "odb" list when prepare_alt_odb() is run.
+	 */
+	char *alternate_db;
+
+	/*
+	 * Objects that should be substituted by other objects
+	 * (see git-replace(1)).
+	 */
+	struct oidmap *replace_map;
+	unsigned replace_map_initialized : 1;
+	pthread_mutex_t replace_mutex; /* protect object replace functions */
+
+	struct commit_graph *commit_graph;
+	unsigned commit_graph_attempted : 1; /* if loading has been attempted */
+
+	/*
+	 * private data
+	 *
+	 * should only be accessed directly by packfile.c and midx.c
+	 */
+	struct multi_pack_index *multi_pack_index;
+
+	/*
+	 * private data
+	 *
+	 * should only be accessed directly by packfile.c
+	 */
+
+	struct packed_git *packed_git;
+	/* A most-recently-used ordered version of the packed_git list. */
+	struct list_head packed_git_mru;
+
+	struct {
+		struct packed_git **packs;
+		unsigned flags;
+	} kept_pack_cache;
+
+	/*
+	 * This is meant to hold a *small* number of objects that you would
+	 * want repo_read_object_file() to be able to return, but yet you do not want
+	 * to write them into the object store (e.g. a browse-only
+	 * application).
+	 */
+	struct cached_object_entry *cached_objects;
+	size_t cached_object_nr, cached_object_alloc;
+
+	/*
+	 * A map of packfiles to packed_git structs for tracking which
+	 * packs have been loaded already.
+	 */
+	struct hashmap pack_map;
+
+	/*
+	 * A fast, rough count of the number of objects in the repository.
+	 * These two fields are not meant for direct access. Use
+	 * repo_approximate_object_count() instead.
+	 */
+	unsigned long approximate_object_count;
+	unsigned approximate_object_count_valid : 1;
+
+	/*
+	 * Whether packed_git has already been populated with this repository's
+	 * packs.
+	 */
+	unsigned packed_git_initialized : 1;
+};
+
+struct raw_object_store *raw_object_store_new(void);
+void raw_object_store_clear(struct raw_object_store *o);
+
+/*
+ * Create a temporary file rooted in the object database directory, or
+ * die on failure. The filename is taken from "pattern", which should have the
+ * usual "XXXXXX" trailer, and the resulting filename is written into the
+ * "template" buffer. Returns the open descriptor.
+ */
+int odb_mkstemp(struct strbuf *temp_filename, const char *pattern);
+
+/*
+ * Create a pack .keep file named "name" (which should generally be the output
+ * of odb_pack_name). Returns a file descriptor opened for writing, or -1 on
+ * error.
+ */
+int odb_pack_keep(const char *name);
+
+/*
+ * Put in `buf` the name of the file in the local object database that
+ * would be used to store a loose object with the specified oid.
+ */
+const char *loose_object_path(struct repository *r, struct strbuf *buf,
+			      const struct object_id *oid);
+
+void *map_loose_object(struct repository *r, const struct object_id *oid,
+		       unsigned long *size);
+
+void *repo_read_object_file(struct repository *r,
+			    const struct object_id *oid,
+			    enum object_type *type,
+			    unsigned long *size);
+
+/* Read and unpack an object file into memory, write memory to an object file */
+int oid_object_info(struct repository *r, const struct object_id *, unsigned long *);
+
+void hash_object_file(const struct git_hash_algo *algo, const void *buf,
+		      unsigned long len, enum object_type type,
+		      struct object_id *oid);
+
+/*
+ * Add an object file to the in-memory object store, without writing it
+ * to disk.
+ *
+ * Callers are responsible for calling write_object_file to record the
+ * object in persistent storage before writing any other new objects
+ * that reference it.
+ */
+int pretend_object_file(struct repository *repo,
+			void *buf, unsigned long len, enum object_type type,
+			struct object_id *oid);
+
+struct object_info {
+	/* Request */
+	enum object_type *typep;
+	unsigned long *sizep;
+	off_t *disk_sizep;
+	struct object_id *delta_base_oid;
+	struct strbuf *type_name;
+	void **contentp;
+
+	/* Response */
+	enum {
+		OI_CACHED,
+		OI_LOOSE,
+		OI_PACKED,
+		OI_DBCACHED
+	} whence;
+	union {
+		/*
+		 * struct {
+		 * 	... Nothing to expose in this case
+		 * } cached;
+		 * struct {
+		 * 	... Nothing to expose in this case
+		 * } loose;
+		 */
+		struct {
+			struct packed_git *pack;
+			off_t offset;
+			unsigned int is_delta;
+		} packed;
+	} u;
+};
+
+/*
+ * Initializer for a "struct object_info" that wants no items. You may
+ * also memset() the memory to all-zeroes.
+ */
+#define OBJECT_INFO_INIT { 0 }
+
+/* Invoke lookup_replace_object() on the given hash */
+#define OBJECT_INFO_LOOKUP_REPLACE 1
+/* Allow reading from a loose object file of unknown/bogus type */
+#define OBJECT_INFO_ALLOW_UNKNOWN_TYPE 2
+/* Do not retry packed storage after checking packed and loose storage */
+#define OBJECT_INFO_QUICK 8
+/*
+ * Do not attempt to fetch the object if missing (even if fetch_is_missing is
+ * nonzero).
+ */
+#define OBJECT_INFO_SKIP_FETCH_OBJECT 16
+/*
+ * This is meant for bulk prefetching of missing blobs in a partial
+ * clone. Implies OBJECT_INFO_SKIP_FETCH_OBJECT and OBJECT_INFO_QUICK
+ */
+#define OBJECT_INFO_FOR_PREFETCH (OBJECT_INFO_SKIP_FETCH_OBJECT | OBJECT_INFO_QUICK)
+
+/* Die if object corruption (not just an object being missing) was detected. */
+#define OBJECT_INFO_DIE_IF_CORRUPT 32
+
+int oid_object_info_extended(struct repository *r,
+			     const struct object_id *,
+			     struct object_info *, unsigned flags);
+
+/* Retry packed storage after checking packed and loose storage */
+#define HAS_OBJECT_RECHECK_PACKED 1
+
+/*
+ * Returns 1 if the object exists. This function will not lazily fetch objects
+ * in a partial clone.
+ */
+int has_object(struct repository *r, const struct object_id *oid,
+	       unsigned flags);
+
+/*
+ * These macros and functions are deprecated. If checking existence for an
+ * object that is likely to be missing and/or whose absence is relatively
+ * inconsequential (or is consequential but the caller is prepared to handle
+ * it), use has_object(), which has better defaults (no lazy fetch in a partial
+ * clone and no rechecking of packed storage). In the unlikely event that a
+ * caller needs to assert existence of an object that it fully expects to
+ * exist, and wants to trigger a lazy fetch in a partial clone, use
+ * oid_object_info_extended() with a NULL struct object_info.
+ *
+ * These functions can be removed once all callers have migrated to
+ * has_object() and/or oid_object_info_extended().
+ */
+int repo_has_object_file(struct repository *r, const struct object_id *oid);
+int repo_has_object_file_with_flags(struct repository *r,
+				    const struct object_id *oid, int flags);
+
+void assert_oid_type(const struct object_id *oid, enum object_type expect);
+
+/*
+ * Enabling the object read lock allows multiple threads to safely call the
+ * following functions in parallel: repo_read_object_file(),
+ * read_object_with_reference(), oid_object_info() and oid_object_info_extended().
+ *
+ * obj_read_lock() and obj_read_unlock() may also be used to protect other
+ * section which cannot execute in parallel with object reading. Since the used
+ * lock is a recursive mutex, these sections can even contain calls to object
+ * reading functions. However, beware that in these cases zlib inflation won't
+ * be performed in parallel, losing performance.
+ *
+ * TODO: oid_object_info_extended()'s call stack has a recursive behavior. If
+ * any of its callees end up calling it, this recursive call won't benefit from
+ * parallel inflation.
+ */
+void enable_obj_read_lock(void);
+void disable_obj_read_lock(void);
+
+extern int obj_read_use_lock;
+extern pthread_mutex_t obj_read_mutex;
+
+static inline void obj_read_lock(void)
+{
+	if(obj_read_use_lock)
+		pthread_mutex_lock(&obj_read_mutex);
+}
+
+static inline void obj_read_unlock(void)
+{
+	if(obj_read_use_lock)
+		pthread_mutex_unlock(&obj_read_mutex);
+}
+
+/*
+ * Iterate over the files in the loose-object parts of the object
+ * directory "path", triggering the following callbacks:
+ *
+ *  - loose_object is called for each loose object we find.
+ *
+ *  - loose_cruft is called for any files that do not appear to be
+ *    loose objects. Note that we only look in the loose object
+ *    directories "objects/[0-9a-f]{2}/", so we will not report
+ *    "objects/foobar" as cruft.
+ *
+ *  - loose_subdir is called for each top-level hashed subdirectory
+ *    of the object directory (e.g., "$OBJDIR/f0"). It is called
+ *    after the objects in the directory are processed.
+ *
+ * Any callback that is NULL will be ignored. Callbacks returning non-zero
+ * will end the iteration.
+ *
+ * In the "buf" variant, "path" is a strbuf which will also be used as a
+ * scratch buffer, but restored to its original contents before
+ * the function returns.
+ */
+typedef int each_loose_object_fn(const struct object_id *oid,
+				 const char *path,
+				 void *data);
+typedef int each_loose_cruft_fn(const char *basename,
+				const char *path,
+				void *data);
+typedef int each_loose_subdir_fn(unsigned int nr,
+				 const char *path,
+				 void *data);
+int for_each_file_in_obj_subdir(unsigned int subdir_nr,
+				struct strbuf *path,
+				each_loose_object_fn obj_cb,
+				each_loose_cruft_fn cruft_cb,
+				each_loose_subdir_fn subdir_cb,
+				void *data);
+int for_each_loose_file_in_objdir(const char *path,
+				  each_loose_object_fn obj_cb,
+				  each_loose_cruft_fn cruft_cb,
+				  each_loose_subdir_fn subdir_cb,
+				  void *data);
+int for_each_loose_file_in_objdir_buf(struct strbuf *path,
+				      each_loose_object_fn obj_cb,
+				      each_loose_cruft_fn cruft_cb,
+				      each_loose_subdir_fn subdir_cb,
+				      void *data);
+
+/* Flags for for_each_*_object() below. */
+enum for_each_object_flags {
+	/* Iterate only over local objects, not alternates. */
+	FOR_EACH_OBJECT_LOCAL_ONLY = (1<<0),
+
+	/* Only iterate over packs obtained from the promisor remote. */
+	FOR_EACH_OBJECT_PROMISOR_ONLY = (1<<1),
+
+	/*
+	 * Visit objects within a pack in packfile order rather than .idx order
+	 */
+	FOR_EACH_OBJECT_PACK_ORDER = (1<<2),
+
+	/* Only iterate over packs that are not marked as kept in-core. */
+	FOR_EACH_OBJECT_SKIP_IN_CORE_KEPT_PACKS = (1<<3),
+
+	/* Only iterate over packs that do not have .keep files. */
+	FOR_EACH_OBJECT_SKIP_ON_DISK_KEPT_PACKS = (1<<4),
+};
+
+/*
+ * Iterate over all accessible loose objects without respect to
+ * reachability. By default, this includes both local and alternate objects.
+ * The order in which objects are visited is unspecified.
+ *
+ * Any flags specific to packs are ignored.
+ */
+int for_each_loose_object(each_loose_object_fn, void *,
+			  enum for_each_object_flags flags);
+
+/*
+ * Iterate over all accessible packed objects without respect to reachability.
+ * By default, this includes both local and alternate packs.
+ *
+ * Note that some objects may appear twice if they are found in multiple packs.
+ * Each pack is visited in an unspecified order. By default, objects within a
+ * pack are visited in pack-idx order (i.e., sorted by oid).
+ */
+typedef int each_packed_object_fn(const struct object_id *oid,
+				  struct packed_git *pack,
+				  uint32_t pos,
+				  void *data);
+int for_each_object_in_pack(struct packed_git *p,
+			    each_packed_object_fn, void *data,
+			    enum for_each_object_flags flags);
+int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
+			   void *data, enum for_each_object_flags flags);
+
+void *read_object_with_reference(struct repository *r,
+				 const struct object_id *oid,
+				 enum object_type required_type,
+				 unsigned long *size,
+				 struct object_id *oid_ret);
 
 #endif /* OBJECT_STORE_H */
diff --git a/oss-fuzz/fuzz-pack-idx.c b/oss-fuzz/fuzz-pack-idx.c
index 3e190214d14..609a343ee3e 100644
--- a/oss-fuzz/fuzz-pack-idx.c
+++ b/oss-fuzz/fuzz-pack-idx.c
@@ -1,5 +1,5 @@
 #include "git-compat-util.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 
 int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size);
diff --git a/pack-bitmap-write.c b/pack-bitmap-write.c
index 6a97b52b36d..4c7f7985e78 100644
--- a/pack-bitmap-write.c
+++ b/pack-bitmap-write.c
@@ -4,7 +4,7 @@
 #include "environment.h"
 #include "gettext.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "diff.h"
 #include "revision.h"
diff --git a/pack-bitmap.c b/pack-bitmap.c
index 0dbd7c4ffe1..fb68d2ae637 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -17,7 +17,7 @@
 #include "packfile.h"
 #include "repository.h"
 #include "trace2.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "list-objects-filter-options.h"
 #include "midx.h"
 #include "config.h"
diff --git a/pack-check.c b/pack-check.c
index 95dcbbe9852..874897d6cba 100644
--- a/pack-check.c
+++ b/pack-check.c
@@ -8,7 +8,7 @@
 #include "progress.h"
 #include "packfile.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 struct idx_entry {
 	off_t                offset;
diff --git a/pack-mtimes.c b/pack-mtimes.c
index bcea28e521d..20900ca88d3 100644
--- a/pack-mtimes.c
+++ b/pack-mtimes.c
@@ -1,7 +1,7 @@
 #include "git-compat-util.h"
 #include "gettext.h"
 #include "pack-mtimes.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 #include "strbuf.h"
 
diff --git a/pack-objects.h b/pack-objects.h
index d73e3843c92..d1c4ae7f9b6 100644
--- a/pack-objects.h
+++ b/pack-objects.h
@@ -1,7 +1,7 @@
 #ifndef PACK_OBJECTS_H
 #define PACK_OBJECTS_H
 
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "thread-utils.h"
 #include "pack.h"
 
diff --git a/pack-revindex.c b/pack-revindex.c
index 1ee7b49e206..37ad9c79268 100644
--- a/pack-revindex.c
+++ b/pack-revindex.c
@@ -1,7 +1,7 @@
 #include "git-compat-util.h"
 #include "gettext.h"
 #include "pack-revindex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 #include "strbuf.h"
 #include "trace2.h"
diff --git a/packfile.c b/packfile.c
index 9d09f8bc726..d91016f1c7f 100644
--- a/packfile.c
+++ b/packfile.c
@@ -19,7 +19,7 @@
 #include "tree-walk.h"
 #include "tree.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "midx.h"
 #include "commit-graph.h"
 #include "pack-revindex.h"
diff --git a/path.c b/path.c
index 910756c8b32..1f7a014322c 100644
--- a/path.c
+++ b/path.c
@@ -15,7 +15,7 @@
 #include "submodule-config.h"
 #include "path.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "lockfile.h"
 #include "exec-cmd.h"
 
diff --git a/promisor-remote.c b/promisor-remote.c
index 5801ebfd9b2..9d058586dfa 100644
--- a/promisor-remote.c
+++ b/promisor-remote.c
@@ -3,7 +3,7 @@
 #include "git-compat-util.h"
 #include "gettext.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "promisor-remote.h"
 #include "config.h"
 #include "trace2.h"
diff --git a/protocol-caps.c b/protocol-caps.c
index 855f279c2f7..9b8db37a210 100644
--- a/protocol-caps.c
+++ b/protocol-caps.c
@@ -6,7 +6,7 @@
 #include "hash.h"
 #include "hex.h"
 #include "object.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "repository.h"
 #include "string-list.h"
 #include "strbuf.h"
diff --git a/prune-packed.c b/prune-packed.c
index 7dad2fc0c16..c1d95a519d7 100644
--- a/prune-packed.c
+++ b/prune-packed.c
@@ -2,7 +2,7 @@
 
 #include "git-compat-util.h"
 #include "gettext.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 #include "progress.h"
 #include "prune-packed.h"
diff --git a/reachable.c b/reachable.c
index 1b26b9b1d76..16e23a38037 100644
--- a/reachable.c
+++ b/reachable.c
@@ -14,7 +14,7 @@
 #include "list-objects.h"
 #include "packfile.h"
 #include "worktree.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pack-bitmap.h"
 #include "pack-mtimes.h"
 #include "config.h"
diff --git a/read-cache.c b/read-cache.c
index 9909b56902f..9aef59cab4c 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -22,7 +22,7 @@
 #include "refs.h"
 #include "dir.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "tree.h"
 #include "commit.h"
diff --git a/ref-filter.c b/ref-filter.c
index 6da8d4c03b6..7a274633cfc 100644
--- a/ref-filter.c
+++ b/ref-filter.c
@@ -12,7 +12,7 @@
 #include "refs.h"
 #include "wildmatch.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "repo-settings.h"
 #include "repository.h"
diff --git a/reflog.c b/reflog.c
index 1b5f031f6d7..60834a124d9 100644
--- a/reflog.c
+++ b/reflog.c
@@ -3,7 +3,7 @@
 
 #include "git-compat-util.h"
 #include "gettext.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "reflog.h"
 #include "refs.h"
 #include "revision.h"
diff --git a/refs.c b/refs.c
index 1208f86629b..14f25c499cd 100644
--- a/refs.c
+++ b/refs.c
@@ -19,7 +19,7 @@
 #include "run-command.h"
 #include "hook.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "object.h"
 #include "path.h"
 #include "submodule.h"
diff --git a/remote.c b/remote.c
index e609cf5c56a..7fc657d98b8 100644
--- a/remote.c
+++ b/remote.c
@@ -12,7 +12,7 @@
 #include "refs.h"
 #include "refspec.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "commit.h"
 #include "diff.h"
diff --git a/replace-object.c b/replace-object.c
index 9a3cdd809a9..7b8a09b5cb4 100644
--- a/replace-object.c
+++ b/replace-object.c
@@ -2,7 +2,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "oidmap.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "replace-object.h"
 #include "refs.h"
 #include "repository.h"
diff --git a/replace-object.h b/replace-object.h
index 66c41b938b4..ba478eb30c4 100644
--- a/replace-object.h
+++ b/replace-object.h
@@ -3,7 +3,7 @@
 
 #include "oidmap.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 struct replace_object {
 	struct oidmap_entry original;
diff --git a/repository.c b/repository.c
index 6cbaf2e3daa..9b3d6665fc6 100644
--- a/repository.c
+++ b/repository.c
@@ -1,7 +1,7 @@
 #include "git-compat-util.h"
 #include "abspath.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "config.h"
 #include "object.h"
 #include "lockfile.h"
diff --git a/rerere.c b/rerere.c
index 740e8ad1a0b..28e59c42380 100644
--- a/rerere.c
+++ b/rerere.c
@@ -18,7 +18,7 @@
 #include "path.h"
 #include "pathspec.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "strmap.h"
 
 #define RESOLVED 0
diff --git a/revision.c b/revision.c
index b536c4a29ad..352e18b1f97 100644
--- a/revision.c
+++ b/revision.c
@@ -8,7 +8,7 @@
 #include "hex.h"
 #include "object-name.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oidset.h"
 #include "tag.h"
 #include "blob.h"
diff --git a/send-pack.c b/send-pack.c
index 856a65d5f5a..5005689cb55 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -4,7 +4,7 @@
 #include "date.h"
 #include "gettext.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pkt-line.h"
 #include "sideband.h"
 #include "run-command.h"
diff --git a/sequencer.c b/sequencer.c
index c112d2e1c43..925cfa1af68 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -13,7 +13,7 @@
 #include "dir.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "object.h"
 #include "pager.h"
 #include "commit.h"
diff --git a/server-info.c b/server-info.c
index 1ca0e00d51e..3048e5ee407 100644
--- a/server-info.c
+++ b/server-info.c
@@ -11,7 +11,7 @@
 #include "packfile.h"
 #include "path.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "server-info.h"
 #include "strbuf.h"
 #include "tempfile.h"
diff --git a/shallow.c b/shallow.c
index 06c3266a3e0..2f82ebd6e3f 100644
--- a/shallow.c
+++ b/shallow.c
@@ -5,7 +5,7 @@
 #include "repository.h"
 #include "tempfile.h"
 #include "lockfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tag.h"
 #include "pkt-line.h"
diff --git a/streaming.c b/streaming.c
index 018b794d252..127d6b5d6ac 100644
--- a/streaming.c
+++ b/streaming.c
@@ -10,7 +10,7 @@
 #include "streaming.h"
 #include "repository.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "replace-object.h"
 #include "packfile.h"
 
diff --git a/submodule-config.c b/submodule-config.c
index d82b404b73e..8630e27947d 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -13,7 +13,7 @@
 #include "submodule.h"
 #include "strbuf.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "parse-options.h"
 #include "thread-utils.h"
 #include "tree-walk.h"
diff --git a/submodule.c b/submodule.c
index 0821507ecaa..3e52ef3b036 100644
--- a/submodule.c
+++ b/submodule.c
@@ -27,7 +27,7 @@
 #include "parse-options.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit-reach.h"
 #include "read-cache-ll.h"
 #include "setup.h"
diff --git a/t/helper/test-pack-mtimes.c b/t/helper/test-pack-mtimes.c
index f8f9afbb5b1..50f5941bff7 100644
--- a/t/helper/test-pack-mtimes.c
+++ b/t/helper/test-pack-mtimes.c
@@ -3,7 +3,7 @@
 #include "test-tool.h"
 #include "hex.h"
 #include "strbuf.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 #include "pack-mtimes.h"
 #include "setup.h"
diff --git a/t/helper/test-partial-clone.c b/t/helper/test-partial-clone.c
index a1af9710c31..34f1aee5581 100644
--- a/t/helper/test-partial-clone.c
+++ b/t/helper/test-partial-clone.c
@@ -1,7 +1,7 @@
 #include "test-tool.h"
 #include "hex.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "setup.h"
 
 /*
diff --git a/t/helper/test-read-graph.c b/t/helper/test-read-graph.c
index 811dde1cb3c..8b413b644be 100644
--- a/t/helper/test-read-graph.c
+++ b/t/helper/test-read-graph.c
@@ -3,7 +3,7 @@
 #include "test-tool.h"
 #include "commit-graph.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "bloom.h"
 #include "setup.h"
 
diff --git a/t/helper/test-read-midx.c b/t/helper/test-read-midx.c
index fc632369618..ac81390899a 100644
--- a/t/helper/test-read-midx.c
+++ b/t/helper/test-read-midx.c
@@ -4,7 +4,7 @@
 #include "hex.h"
 #include "midx.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pack-bitmap.h"
 #include "packfile.h"
 #include "setup.h"
diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c
index 2ff67c067ac..4cfc7c90b59 100644
--- a/t/helper/test-ref-store.c
+++ b/t/helper/test-ref-store.c
@@ -5,7 +5,7 @@
 #include "refs.h"
 #include "setup.h"
 #include "worktree.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "repository.h"
 #include "strbuf.h"
diff --git a/tag.c b/tag.c
index 8d9e9e29304..05be39067cf 100644
--- a/tag.c
+++ b/tag.c
@@ -5,7 +5,7 @@
 #include "environment.h"
 #include "tag.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tree.h"
 #include "blob.h"
diff --git a/tmp-objdir.c b/tmp-objdir.c
index 31d16a4c2c5..c38fbeb5e8a 100644
--- a/tmp-objdir.c
+++ b/tmp-objdir.c
@@ -10,7 +10,7 @@
 #include "strbuf.h"
 #include "strvec.h"
 #include "quote.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "repository.h"
 
 struct tmp_objdir {
diff --git a/tree-walk.c b/tree-walk.c
index a0333979656..90655d52378 100644
--- a/tree-walk.c
+++ b/tree-walk.c
@@ -6,7 +6,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "trace2.h"
 #include "tree.h"
 #include "pathspec.h"
diff --git a/tree.c b/tree.c
index ad86ad1ba99..b85f56267fb 100644
--- a/tree.c
+++ b/tree.c
@@ -4,7 +4,7 @@
 #include "hex.h"
 #include "tree.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "alloc.h"
 #include "tree-walk.h"
diff --git a/unpack-trees.c b/unpack-trees.c
index cf5b73c84be..471837f0329 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -26,7 +26,7 @@
 #include "symlinks.h"
 #include "trace2.h"
 #include "fsmonitor.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "promisor-remote.h"
 #include "entry.h"
 #include "parallel-checkout.h"
diff --git a/upload-pack.c b/upload-pack.c
index 02ce6336028..30e4630f3a1 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -10,7 +10,7 @@
 #include "pkt-line.h"
 #include "sideband.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "object.h"
 #include "commit.h"
diff --git a/walker.c b/walker.c
index 1cf3da02193..4fedc19f346 100644
--- a/walker.c
+++ b/walker.c
@@ -5,7 +5,7 @@
 #include "hex.h"
 #include "walker.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "strbuf.h"
 #include "tree.h"
diff --git a/xdiff-interface.c b/xdiff-interface.c
index 77712811ff1..1edcd319e6e 100644
--- a/xdiff-interface.c
+++ b/xdiff-interface.c
@@ -5,7 +5,7 @@
 #include "gettext.h"
 #include "config.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "strbuf.h"
 #include "xdiff-interface.h"
 #include "xdiff/xtypes.h"

-- 
2.49.0.777.g153de2bbd5.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* Re: [PATCH 1/9] object-file: move `safe_create_leading_directories()` into "dir.c"
  2025-04-11  9:27     ` Patrick Steinhardt
@ 2025-04-11 17:11       ` Elijah Newren
  2025-04-15  9:19         ` Patrick Steinhardt
  0 siblings, 1 reply; 53+ messages in thread
From: Elijah Newren @ 2025-04-11 17:11 UTC (permalink / raw)
  To: Patrick Steinhardt; +Cc: git

On Fri, Apr 11, 2025 at 2:27 AM Patrick Steinhardt <ps@pks.im> wrote:
>
> On Wed, Apr 09, 2025 at 07:36:47AM -0700, Elijah Newren wrote:
> > On Tue, Apr 8, 2025 at 3:37 AM Patrick Steinhardt <ps@pks.im> wrote:
> > >
> > > The `safe_create_leading_directories()` function and its relatives
> >
> > How is mkdir_in_gitdir() a relative of safe_create_leading_directories()?
> >
> > I assumed the relation was "called by", but there is no such
> > relationship.  The rest of the patch looked fine, but I was puzzled
> > for a while trying to figure out what this relationship is.
>
> It's more of a sibling than a child/parent in this case, true. I still
> think it makes sense to move it around as it is rather generic in the
> functionality it provides and doesn't have anything to do with objects.
>
> Patrick

I fully agree it makes sense to move it and that dir.c is a good place
for it, I just think it also makes sense to fix the commit message to
avoid the misleading/confusing text by calling out mkdir_in_gitdir()
separately since it isn't related to
safe_create_leading_directories().  For example, highlighting the text
I added between asterisks, you could make it read:

The `safe_create_leading_directories()` function and its relatives*,
as well as mkdir_in_gitdir()*, are
located in "object-file.c", which is not a good fit as they provide
generic functionality not related to objects at all. Move them into
"dir.c".

However, this is a nitpick and probably not worth another re-roll;
especially since everything else in your v2 looks great to me.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v2 1/9] object-file: move `safe_create_leading_directories()` into "dir.c"
  2025-04-11  9:29   ` [PATCH v2 1/9] object-file: move `safe_create_leading_directories()` into "dir.c" Patrick Steinhardt
@ 2025-04-11 20:09     ` Junio C Hamano
  2025-04-11 21:29       ` Eric Sunshine
  0 siblings, 1 reply; 53+ messages in thread
From: Junio C Hamano @ 2025-04-11 20:09 UTC (permalink / raw)
  To: Patrick Steinhardt; +Cc: git, Elijah Newren

Patrick Steinhardt <ps@pks.im> writes:

> The `safe_create_leading_directories()` function and its relatives are
> located in "object-file.c", which is not a good fit as they provide
> generic functionality not related to objects at all. Move them into
> "dir.c".

It may be debatable that <dir.c>, which has traditionally been a
collection of read-only operations (mostly for exclude/ignore
processing), is a good place to host "mkdir -p", but it certainly is
better than having it in <object-file.c>

Looking good.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v2 1/9] object-file: move `safe_create_leading_directories()` into "dir.c"
  2025-04-11 20:09     ` Junio C Hamano
@ 2025-04-11 21:29       ` Eric Sunshine
  2025-04-15  9:19         ` Patrick Steinhardt
  0 siblings, 1 reply; 53+ messages in thread
From: Eric Sunshine @ 2025-04-11 21:29 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Patrick Steinhardt, git, Elijah Newren

On Fri, Apr 11, 2025 at 4:10 PM Junio C Hamano <gitster@pobox.com> wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> > The `safe_create_leading_directories()` function and its relatives are
> > located in "object-file.c", which is not a good fit as they provide
> > generic functionality not related to objects at all. Move them into
> > "dir.c".
>
> It may be debatable that <dir.c>, which has traditionally been a
> collection of read-only operations (mostly for exclude/ignore
> processing), is a good place to host "mkdir -p", but it certainly is
> better than having it in <object-file.c>

I probably would have expected safe_create_leading_directories() to be
moved to "path.[hc]" which already houses functions such as
safe_create_dir(), normalize_path_copy(), ends_with_path_components(),
longest_ancestor_length(), etc.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v2 8/9] object-store: remove global array of cached objects
  2025-04-11  9:29   ` [PATCH v2 8/9] object-store: remove global array of cached objects Patrick Steinhardt
@ 2025-04-11 22:58     ` Junio C Hamano
  2025-04-15  9:19       ` Patrick Steinhardt
  0 siblings, 1 reply; 53+ messages in thread
From: Junio C Hamano @ 2025-04-11 22:58 UTC (permalink / raw)
  To: Patrick Steinhardt; +Cc: git, Elijah Newren

Patrick Steinhardt <ps@pks.im> writes:

> Cached objects are virtual objects that can be set up without writing
> anything into the object store directly. This mechanism for example
> allows us to create fake commits in git-blame(1).
>
> The cached objects are stored in a global variable. Refactor the code so
> that we instead store the array as part of the raw object store. This is
> another step into the direction of libifying our object database.

While we do need some execution context object to hang these virtual
objects, once we decide that it cannot be global, I am not sure if
epository objects are good home for them.  If your application
running in a repository needs to give one object name to a virtual
object, and then that same application wants to access a submodule
of that repository in the same process image, wouldn't you have one
in-core repository object for the top-level superproject, and one
for each submodule?  If a submodule commit bound to a path in the
superproject's tree is a viertual "pretend" commit object or if it
has a virtual "pretend" tree object, don't you need to expose these
to both submodule and superproject repositories, if your application
wants to seamlessly cross the module boundary (think "git grep
--recurse-submodules" or something)?

For now, as long as the_repository is being used as that "execution
context object", and not a repository instance passed along the call
chain, then the globalness of these virtual objects is maintained,
so this change will not cause breakage (e.g., such an application
may want to pick up the virtual object from the repository instance
for the superproject and it may find it, but when traversing down to
a submdoule, the same virtual object may not be found in the
repository instance for the submodule it descended into and working
in, if you make it per repository and pass repository instance
around along the call chain).  But eventually somebody will start
saying "let's remove USE_THE_REPOSITORY_VARIABLE", at which point I
am not sure how subtle such a bug would become.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v2 6/9] object-file: split out functions relating to index subsystem
  2025-04-11  9:29   ` [PATCH v2 6/9] object-file: split out functions relating to index subsystem Patrick Steinhardt
@ 2025-04-12  8:17     ` Jeff King
  2025-04-14 11:49       ` Junio C Hamano
  2025-04-15  9:19       ` Patrick Steinhardt
  0 siblings, 2 replies; 53+ messages in thread
From: Jeff King @ 2025-04-12  8:17 UTC (permalink / raw)
  To: Patrick Steinhardt; +Cc: git, Elijah Newren, Junio C Hamano

On Fri, Apr 11, 2025 at 11:29:55AM +0200, Patrick Steinhardt wrote:

> Split out functions relating to the index subsystem from "object-file.c"
> to help us separate concerns.

I know these functions all start with "index_", and they do take an
index_state variable, but I'm not sure they are really about Git's index
subsystem at all.

The term "index" here is more about "compute the sha1 index of the
content". E.g., the function index_path() goes all the way back to
ec1fcc16af (Show original and resulting blob object info in diff
output., 2005-10-07)!

Back then it did not take an index struct, or even care about having an
index at all. Later, they learned to call convert_to_git() in 6c510bee20
(Lazy man's auto-CRLF, 2007-02-13). And that function may check the
index for .gitattributes files.

It originally just used the global the_index variable for that, but
later commits like 58bf2a4cc7 (sha1-file.c: remove implicit dependency
on the_index, 2018-09-21) passed the istate around the call stack.

So having access to an index struct is mostly incidental to these
functions. Which makes sense looking at the callers: there are many
pure-object operations that would work without an index (or even a repo
in some cases!) like hash-object, git-replace, diff.

  Side note: I'm actually not even sure we would read attributes from
  the index, since we don't set GIT_ATTR_INDEX. So I wondered if we
  could simply pass NULL to convert_to_git() here. But I think these
  days some of the "auto" CRLF modes also have heuristics based on
  what's the content we find in the index for that path. See
  has_crlf_in_index() and its callers.

So it seems to me that these really are more about creating objects than
they are about the index. I don't mind splitting them out, but it seems
like they're equally weird in read-cache.[ch].

-Peff

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v2 6/9] object-file: split out functions relating to index subsystem
  2025-04-12  8:17     ` Jeff King
@ 2025-04-14 11:49       ` Junio C Hamano
  2025-04-15  9:19       ` Patrick Steinhardt
  1 sibling, 0 replies; 53+ messages in thread
From: Junio C Hamano @ 2025-04-14 11:49 UTC (permalink / raw)
  To: Jeff King; +Cc: Patrick Steinhardt, git, Elijah Newren

Jeff King <peff@peff.net> writes:

> So it seems to me that these really are more about creating objects than
> they are about the index. I don't mind splitting them out, but it seems
> like they're equally weird in read-cache.[ch].

Thanks for a good observation based on history of the code involved.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v2 8/9] object-store: remove global array of cached objects
  2025-04-11 22:58     ` Junio C Hamano
@ 2025-04-15  9:19       ` Patrick Steinhardt
  0 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-15  9:19 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Elijah Newren

On Fri, Apr 11, 2025 at 03:58:03PM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > Cached objects are virtual objects that can be set up without writing
> > anything into the object store directly. This mechanism for example
> > allows us to create fake commits in git-blame(1).
> >
> > The cached objects are stored in a global variable. Refactor the code so
> > that we instead store the array as part of the raw object store. This is
> > another step into the direction of libifying our object database.
> 
> While we do need some execution context object to hang these virtual
> objects, once we decide that it cannot be global, I am not sure if
> epository objects are good home for them.  If your application
> running in a repository needs to give one object name to a virtual
> object, and then that same application wants to access a submodule
> of that repository in the same process image, wouldn't you have one
> in-core repository object for the top-level superproject, and one
> for each submodule?  If a submodule commit bound to a path in the
> superproject's tree is a viertual "pretend" commit object or if it
> has a virtual "pretend" tree object, don't you need to expose these
> to both submodule and superproject repositories, if your application
> wants to seamlessly cross the module boundary (think "git grep
> --recurse-submodules" or something)?
> 
> For now, as long as the_repository is being used as that "execution
> context object", and not a repository instance passed along the call
> chain, then the globalness of these virtual objects is maintained,
> so this change will not cause breakage (e.g., such an application
> may want to pick up the virtual object from the repository instance
> for the superproject and it may find it, but when traversing down to
> a submdoule, the same virtual object may not be found in the
> repository instance for the submodule it descended into and working
> in, if you make it per repository and pass repository instance
> around along the call chain).  But eventually somebody will start
> saying "let's remove USE_THE_REPOSITORY_VARIABLE", at which point I
> am not sure how subtle such a bug would become.

I think the answer is very much "it depends". I can think of usecases
where it might be the right to pretend objects to exist globally, but
there's also usecases where I think it makes sense to treat them as
repository-specific. The thing is: we can do the former if the virtual
objects are specific to a repository, but we can't do the latter if the
virtual objects are global.

As far as I can see we only use this mechanism in git-blame(1) right now
to create a fake working tree commit. This mechanism does not cross into
submodules at all, and if it would I think we would want to create two
separate fake working tree commits anyway: one for the parent
repository, and one for each submodule. So converting this mechanism to
be local to the repository (or rather local to an object store) feels
like the right thing to do to me.

But I agree with you in principle: we will have to be a lot more mindful
going forward as it comes to handling multiple repositories in-memory.
We don't do this well right now, but as we convert more and more code so
that it doesn't use `the_repository` anymore we'll have to become better
at this indeed. From my perspective that isn't only true for these fake
working tree commits, but it's a general thing that we'll have to sort
out over time. It's inherent to the whole libifcation process.

I think for the most part we're fine right now, as we don't make use of
any of the new capabilities that libifcation brings with it in theory.
But once usecases start to come up that _do_ make use of this we will
have to think about those issues a whole lot more carefully.

Patrick

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v2 1/9] object-file: move `safe_create_leading_directories()` into "dir.c"
  2025-04-11 21:29       ` Eric Sunshine
@ 2025-04-15  9:19         ` Patrick Steinhardt
  2025-04-15 15:09           ` Junio C Hamano
  0 siblings, 1 reply; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-15  9:19 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: Junio C Hamano, git, Elijah Newren

On Fri, Apr 11, 2025 at 05:29:13PM -0400, Eric Sunshine wrote:
> On Fri, Apr 11, 2025 at 4:10 PM Junio C Hamano <gitster@pobox.com> wrote:
> > Patrick Steinhardt <ps@pks.im> writes:
> > > The `safe_create_leading_directories()` function and its relatives are
> > > located in "object-file.c", which is not a good fit as they provide
> > > generic functionality not related to objects at all. Move them into
> > > "dir.c".
> >
> > It may be debatable that <dir.c>, which has traditionally been a
> > collection of read-only operations (mostly for exclude/ignore
> > processing), is a good place to host "mkdir -p", but it certainly is
> > better than having it in <object-file.c>
> 
> I probably would have expected safe_create_leading_directories() to be
> moved to "path.[hc]" which already houses functions such as
> safe_create_dir(), normalize_path_copy(), ends_with_path_components(),
> longest_ancestor_length(), etc.

Ah, good catch! Will adapt.

Patrick

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH 1/9] object-file: move `safe_create_leading_directories()` into "dir.c"
  2025-04-11 17:11       ` Elijah Newren
@ 2025-04-15  9:19         ` Patrick Steinhardt
  2025-04-15 15:11           ` Junio C Hamano
  0 siblings, 1 reply; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-15  9:19 UTC (permalink / raw)
  To: Elijah Newren; +Cc: git

On Fri, Apr 11, 2025 at 10:11:52AM -0700, Elijah Newren wrote:
> On Fri, Apr 11, 2025 at 2:27 AM Patrick Steinhardt <ps@pks.im> wrote:
> >
> > On Wed, Apr 09, 2025 at 07:36:47AM -0700, Elijah Newren wrote:
> > > On Tue, Apr 8, 2025 at 3:37 AM Patrick Steinhardt <ps@pks.im> wrote:
> > > >
> > > > The `safe_create_leading_directories()` function and its relatives
> > >
> > > How is mkdir_in_gitdir() a relative of safe_create_leading_directories()?
> > >
> > > I assumed the relation was "called by", but there is no such
> > > relationship.  The rest of the patch looked fine, but I was puzzled
> > > for a while trying to figure out what this relationship is.
> >
> > It's more of a sibling than a child/parent in this case, true. I still
> > think it makes sense to move it around as it is rather generic in the
> > functionality it provides and doesn't have anything to do with objects.
> >
> > Patrick
> 
> I fully agree it makes sense to move it and that dir.c is a good place
> for it, I just think it also makes sense to fix the commit message to
> avoid the misleading/confusing text by calling out mkdir_in_gitdir()
> separately since it isn't related to
> safe_create_leading_directories().  For example, highlighting the text
> I added between asterisks, you could make it read:
> 
> The `safe_create_leading_directories()` function and its relatives*,
> as well as mkdir_in_gitdir()*, are
> located in "object-file.c", which is not a good fit as they provide
> generic functionality not related to objects at all. Move them into
> "dir.c".
> 
> However, this is a nitpick and probably not worth another re-roll;
> especially since everything else in your v2 looks great to me.

Eric has suggested moving it into "path.c", which I think is indeed a
better fix. I'm using that as an opportunity to rename the function to
`safe_create_dir_in_gitdir()` so that it matches `safe_create_dir()`,
which is functionally similar. And because "path.c" does not depend on
`the_repository` anymore I'll also inject a repository via a parameter.

All to say: there's a bunch of additional changes now, so I'll split
this out into a separate commit.

Patrick

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v2 6/9] object-file: split out functions relating to index subsystem
  2025-04-12  8:17     ` Jeff King
  2025-04-14 11:49       ` Junio C Hamano
@ 2025-04-15  9:19       ` Patrick Steinhardt
  1 sibling, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-15  9:19 UTC (permalink / raw)
  To: Jeff King; +Cc: git, Elijah Newren, Junio C Hamano

On Sat, Apr 12, 2025 at 04:17:24AM -0400, Jeff King wrote:
> On Fri, Apr 11, 2025 at 11:29:55AM +0200, Patrick Steinhardt wrote:
> 
> > Split out functions relating to the index subsystem from "object-file.c"
> > to help us separate concerns.
> 
> I know these functions all start with "index_", and they do take an
> index_state variable, but I'm not sure they are really about Git's index
> subsystem at all.
> 
> The term "index" here is more about "compute the sha1 index of the
> content". E.g., the function index_path() goes all the way back to
> ec1fcc16af (Show original and resulting blob object info in diff
> output., 2005-10-07)!
> 
> Back then it did not take an index struct, or even care about having an
> index at all. Later, they learned to call convert_to_git() in 6c510bee20
> (Lazy man's auto-CRLF, 2007-02-13). And that function may check the
> index for .gitattributes files.
> 
> It originally just used the global the_index variable for that, but
> later commits like 58bf2a4cc7 (sha1-file.c: remove implicit dependency
> on the_index, 2018-09-21) passed the istate around the call stack.
> 
> So having access to an index struct is mostly incidental to these
> functions. Which makes sense looking at the callers: there are many
> pure-object operations that would work without an index (or even a repo
> in some cases!) like hash-object, git-replace, diff.
> 
>   Side note: I'm actually not even sure we would read attributes from
>   the index, since we don't set GIT_ATTR_INDEX. So I wondered if we
>   could simply pass NULL to convert_to_git() here. But I think these
>   days some of the "auto" CRLF modes also have heuristics based on
>   what's the content we find in the index for that path. See
>   has_crlf_in_index() and its callers.
> 
> 
> So it seems to me that these really are more about creating objects than
> they are about the index. I don't mind splitting them out, but it seems
> like they're equally weird in read-cache.[ch].

Yeah, I was too trigger-happy in this case indeed. I'll retain the code
as-is for now, thanks!

Patrick

^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH v3 00/10] Split up "object-file.c"
  2025-04-08 10:24 [PATCH 0/9] Split up "object-file.c" Patrick Steinhardt
                   ` (11 preceding siblings ...)
  2025-04-11  9:29 ` [PATCH v2 " Patrick Steinhardt
@ 2025-04-15  9:38 ` Patrick Steinhardt
  2025-04-15  9:38   ` [PATCH v3 01/10] object-file: move `mkdir_in_gitdir()` into "path.c" Patrick Steinhardt
                     ` (10 more replies)
  12 siblings, 11 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-15  9:38 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Eric Sunshine, Jeff King, Junio C Hamano

Hi,

"object-file.c" is quite a grab-bag of all kinds of different functions.
Many of these functions aren't really a good fit though and should be
owned by a different subsystem. This patch series tries to split up
concerns a bit better by splitting out this functionality into other
files:

  - `safe_create_leading_directories()` is moved into "dir.c".
  - `xmmap()` is moved into "wrapper.c".
  - `git_open_cloexec()` is moved into "compat/open.c".
  - Several functions attached to `struct index_state` are moved into
    "read-cache.c".
  - Several functions related to `struct object_store` are moved into a
    new file "object-store.c".

"object-file.c" now mostly contains logic to read and write loose object
files, whereas "object-store.c" contains the higher-level logic to
manage different object directories for a repository. Eventually, these
will become the loose object backend as well as the `struct ref_store`
equivalent for objects, respectively.

The series is built on top of 9d22ac51228 (The third batch, 2025-04-07)
with ps/object-wo-the-repository at 9442b1c919a (Merge remote-tracking
branch 'junio/ps/object-wo-the-repository' into HEAD, 2025-04-08) merged
into it.

Changes in v2:
  - Fix a grammar issue in one of the commit messages.
  - Link to v1: https://lore.kernel.org/r/20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im

Changes in v3:
  - Rename `mkdir_in_gitdir()` to `safe_create_dir_in_gitdir()` to match
    naming of similar functions.
  - Move `safe_create_leading_directories()` et al into "path.c" instead
    of into "dir.c". This also requires us to start injecting a repo via
    parameters as "path.c" doesn't have `the_repository` available
    anymore.
  - Drop the commit that moves `index_blob_stream()` and related
    functions.
  - Expand the reasoning why we want to have cached objects per object
    store instead of globally.
  - Drop `index_blob_stream()`, which is a trivial wrapper around
    `index_blob_bulk_checkin()`.
  - Link to v2: https://lore.kernel.org/r/20250411-pks-split-object-file-v2-0-2bea0c9033ae@pks.im

Thanks!

Patrick

---
Patrick Steinhardt (10):
      object-file: move `mkdir_in_gitdir()` into "path.c"
      object-file: move `safe_create_leading_directories()` into "path.c"
      object-file: move `git_open_cloexec()` to "compat/open.c"
      object-file: move `xmmap()` into "wrapper.c"
      object-file: split out functions relating to object store subsystem
      object-file: split up concerns of `HASH_*` flags
      object-file: drop `index_blob_stream()`
      object: split out functions relating to object store subsystem
      object-store: remove global array of cached objects
      object-store: merge "object-store-ll.h" and "object-store.h"

 Makefile                           |    3 +-
 apply.c                            |    2 +-
 archive-tar.c                      |    2 +-
 archive-zip.c                      |    2 +-
 archive.c                          |    2 +-
 attr.c                             |    2 +-
 bisect.c                           |    2 +-
 blame.c                            |    4 +-
 builtin/backfill.c                 |    2 +-
 builtin/blame.c                    |    2 +-
 builtin/bugreport.c                |    4 +-
 builtin/cat-file.c                 |    2 +-
 builtin/checkout.c                 |    3 +-
 builtin/clone.c                    |    6 +-
 builtin/commit-graph.c             |    2 +-
 builtin/commit-tree.c              |    2 +-
 builtin/count-objects.c            |    2 +-
 builtin/credential-cache--daemon.c |    4 +-
 builtin/describe.c                 |    2 +-
 builtin/diagnose.c                 |    4 +-
 builtin/difftool.c                 |   31 +-
 builtin/fast-export.c              |    2 +-
 builtin/fast-import.c              |    4 +-
 builtin/fetch.c                    |    2 +-
 builtin/fsck.c                     |    4 +-
 builtin/gc.c                       |    9 +-
 builtin/grep.c                     |    2 +-
 builtin/hash-object.c              |   25 +-
 builtin/index-pack.c               |    2 +-
 builtin/init-db.c                  |    3 +-
 builtin/log.c                      |    6 +-
 builtin/ls-tree.c                  |    2 +-
 builtin/merge-file.c               |    1 +
 builtin/merge-tree.c               |    2 +-
 builtin/mktag.c                    |    2 +-
 builtin/mktree.c                   |    3 +-
 builtin/multi-pack-index.c         |    2 +-
 builtin/mv.c                       |    3 +-
 builtin/notes.c                    |    3 +-
 builtin/pack-objects.c             |    2 +-
 builtin/pack-redundant.c           |    2 +-
 builtin/prune.c                    |    2 +-
 builtin/rebase.c                   |    3 +-
 builtin/receive-pack.c             |    3 +-
 builtin/remote.c                   |    2 +-
 builtin/repack.c                   |    2 +-
 builtin/replace.c                  |    4 +-
 builtin/rev-list.c                 |    2 +-
 builtin/show-ref.c                 |    2 +-
 builtin/sparse-checkout.c          |    5 +-
 builtin/submodule--helper.c        |    6 +-
 builtin/tag.c                      |    3 +-
 builtin/unpack-file.c              |    3 +-
 builtin/unpack-objects.c           |    3 +-
 builtin/update-index.c             |    2 +-
 builtin/worktree.c                 |    8 +-
 bulk-checkin.c                     |    8 +-
 bulk-checkin.h                     |   15 +
 bundle-uri.c                       |    2 +-
 bundle.c                           |    2 +-
 cache-tree.c                       |    4 +-
 combine-diff.c                     |    2 +-
 commit-graph.c                     |    5 +-
 commit-graph.h                     |    2 +-
 commit.c                           |    3 +-
 compat/open.c                      |   29 +
 config.c                           |    2 +-
 connected.c                        |    2 +-
 convert.c                          |    2 +-
 diagnose.c                         |    2 +-
 diff.c                             |    2 +-
 diffcore-rename.c                  |    2 +-
 dir.c                              |    7 +-
 entry.c                            |    2 +-
 fetch-pack.c                       |    2 +-
 fmt-merge-msg.c                    |    2 +-
 fsck.c                             |    2 +-
 git-compat-util.h                  |    3 +
 grep.c                             |    2 +-
 http-backend.c                     |    2 +-
 http-push.c                        |    3 +-
 http-walker.c                      |    2 +-
 http.c                             |    2 +-
 list-objects-filter.c              |    2 +-
 list-objects.c                     |    2 +-
 log-tree.c                         |    2 +-
 mailmap.c                          |    2 +-
 match-trees.c                      |    3 +-
 merge-blobs.c                      |    2 +-
 merge-ort.c                        |    3 +-
 merge-recursive.c                  |    6 +-
 meson.build                        |    2 +
 midx-write.c                       |    2 +-
 midx.c                             |    1 -
 notes-cache.c                      |    3 +-
 notes-merge.c                      |    8 +-
 notes.c                            |    3 +-
 object-file.c                      | 1220 +-----------------------------------
 object-file.h                      |  121 ++--
 object-name.c                      |    2 +-
 object-store-ll.h                  |  556 ----------------
 object-store.c                     | 1050 +++++++++++++++++++++++++++++++
 object-store.h                     |  516 ++++++++++++++-
 object.c                           |   67 --
 oss-fuzz/fuzz-pack-idx.c           |    2 +-
 pack-bitmap-write.c                |    2 +-
 pack-bitmap.c                      |    3 +-
 pack-check.c                       |    2 +-
 pack-mtimes.c                      |    3 +-
 pack-objects.h                     |    2 +-
 pack-revindex.c                    |    3 +-
 packfile.c                         |    2 +-
 path.c                             |  111 +++-
 path.h                             |   45 ++
 promisor-remote.c                  |    2 +-
 protocol-caps.c                    |    2 +-
 prune-packed.c                     |    2 +-
 reachable.c                        |    2 +-
 read-cache.c                       |    6 +-
 ref-filter.c                       |    2 +-
 reflog.c                           |    2 +-
 refs.c                             |    2 +-
 refs/files-backend.c               |    4 +-
 remote.c                           |    2 +-
 replace-object.c                   |    2 +-
 replace-object.h                   |    2 +-
 repository.c                       |    2 +-
 rerere.c                           |    7 +-
 revision.c                         |    2 +-
 send-pack.c                        |    2 +-
 sequencer.c                        |    6 +-
 server-info.c                      |    4 +-
 shallow.c                          |    2 +-
 streaming.c                        |    2 +-
 submodule-config.c                 |    2 +-
 submodule.c                        |    4 +-
 t/helper/test-pack-mtimes.c        |    2 +-
 t/helper/test-partial-clone.c      |    2 +-
 t/helper/test-read-graph.c         |    2 +-
 t/helper/test-read-midx.c          |    2 +-
 t/helper/test-ref-store.c          |    2 +-
 tag.c                              |    2 +-
 tmp-objdir.c                       |    2 +-
 tree-walk.c                        |    2 +-
 tree.c                             |    2 +-
 unpack-trees.c                     |    2 +-
 upload-pack.c                      |    2 +-
 walker.c                           |    2 +-
 wrapper.c                          |   48 ++
 xdiff-interface.c                  |    2 +-
 150 files changed, 2147 insertions(+), 2067 deletions(-)

Range-diff versus v2:

 1:  ed337338970 <  -:  ----------- object-file: move `safe_create_leading_directories()` into "dir.c"
 -:  ----------- >  1:  8d838f92936 object-file: move `mkdir_in_gitdir()` into "path.c"
 -:  ----------- >  2:  c7c723db86c object-file: move `safe_create_leading_directories()` into "path.c"
 2:  48730ada01e =  3:  2b4db7b2090 object-file: move `git_open_cloexec()` to "compat/open.c"
 3:  a9436d9e4a6 =  4:  7e8b750d652 object-file: move `xmmap()` into "wrapper.c"
 4:  628d0bcce6c !  5:  3b90e8a5841 object-file: split out functions relating to object store subsystem
    @@ object-file.c: static int get_conv_flags(unsigned flags)
      		return 0;
      }
      
    --
     -int odb_mkstemp(struct strbuf *temp_filename, const char *pattern)
     -{
     -	int fd;
    @@ object-file.c: static int get_conv_flags(unsigned flags)
     -	/* slow path */
     -	/* some mkstemp implementations erase temp_filename on failure */
     -	repo_git_path_replace(the_repository, temp_filename, "objects/%s", pattern);
    --	safe_create_leading_directories(temp_filename->buf);
    +-	safe_create_leading_directories(the_repository, temp_filename->buf);
     -	return xmkstemp_mode(temp_filename->buf, mode);
     -}
     -
    @@ object-file.c: static int get_conv_flags(unsigned flags)
     -		return fd;
     -
     -	/* slow path */
    --	safe_create_leading_directories_const(name);
    +-	safe_create_leading_directories_const(the_repository, name);
     -	return open(name, O_RDWR|O_CREAT|O_EXCL, 0600);
     -}
     -
    @@ object-store.c (new)
     +	/* slow path */
     +	/* some mkstemp implementations erase temp_filename on failure */
     +	repo_git_path_replace(the_repository, temp_filename, "objects/%s", pattern);
    -+	safe_create_leading_directories(temp_filename->buf);
    ++	safe_create_leading_directories(the_repository, temp_filename->buf);
     +	return xmkstemp_mode(temp_filename->buf, mode);
     +}
     +
    @@ object-store.c (new)
     +		return fd;
     +
     +	/* slow path */
    -+	safe_create_leading_directories_const(name);
    ++	safe_create_leading_directories_const(the_repository, name);
     +	return open(name, O_RDWR|O_CREAT|O_EXCL, 0600);
     +}
     +
 5:  fd102d9e50a =  6:  6bc5d4811d3 object-file: split up concerns of `HASH_*` flags
 6:  298a867897d <  -:  ----------- object-file: split out functions relating to index subsystem
 -:  ----------- >  7:  c1e323f1673 object-file: drop `index_blob_stream()`
 7:  53e2bd9d117 =  8:  6b46541fe2a object: split out functions relating to object store subsystem
 8:  00b4a52b8ce !  9:  59df5f22b9d object-store: remove global array of cached objects
    @@ Commit message
         object-store: remove global array of cached objects
     
         Cached objects are virtual objects that can be set up without writing
    -    anything into the object store directly. This mechanism for example
    -    allows us to create fake commits in git-blame(1).
    +    anything into the object store directly, which is used by git-blame(1)
    +    to create fake commits for the working tree.
     
    -    The cached objects are stored in a global variable. Refactor the code so
    -    that we instead store the array as part of the raw object store. This is
    -    another step into the direction of libifying our object database.
    +    These cached objects are stored in a global variable, which is another
    +    roadblock for libification of the object subsystem. Refactor the code so
    +    that we instead store the array as part of the raw object store.
    +
    +    This refactoring raises the question whether virtual objects should
    +    really be specific to a single repository (or rather a single object
    +    store). Hypothetical usecases might for example span across submodules,
    +    and here it may or may not be the right thing to provide virtual objects
    +    across submodule boundaries.
    +
    +    The only existing usecase is git-blame(1) though, which does not know to
    +    blame across submodule boundaries in the first place. As such, storing
    +    these objects both globally and per-repository would achieve the same
    +    result right now. But arguably, if we learned to blame across submodule
    +    boundaries, we would likely want to create separate fare working tree
    +    commits for each of the submodules so that the user can learn which
    +    worktree a specific uncommitted change belongs to. And even if we would
    +    want to create the same fake commit for each of the submodules we could
    +    do that when storing separate virtual objects per object store.
    +
    +    While this is all rather hypothetical, the takeaway is that handling
    +    virtual objects per-object store gives us more flexibility compared to
    +    storing them globally. In a hypothetical future where we have achieved
    +    full libification one might be able to handle unrelated repositories in
    +    a single process, where the state of one repository should not have an
    +    impact on the state of another repository. As such, storing these cached
    +    objects per object store will enable more usecases and should lead to
    +    less surprising outcomes overall.
     
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
 9:  6365dc7cd16 ! 10:  df6a749245f object-store: merge "object-store-ll.h" and "object-store.h"
    @@ builtin/replace.c
      #include "object-name.h"
     -#include "object-store-ll.h"
     +#include "object-store.h"
    - #include "read-cache.h"
      #include "replace-object.h"
      #include "tag.h"
    + #include "wildmatch.h"
     
      ## builtin/rev-list.c ##
     @@
    @@ notes-merge.c
     -#include "object-store-ll.h"
     +#include "object-store.h"
      #include "path.h"
    - #include "read-cache.h"
      #include "repository.h"
    + #include "diff.h"
     
      ## object-file.c ##
     @@
    - 
      #include "git-compat-util.h"
      #include "bulk-checkin.h"
    + #include "convert.h"
     +#include "dir.h"
      #include "environment.h"
    + #include "fsck.h"
      #include "gettext.h"
    - #include "hex.h"
     
      ## object-name.c ##
     @@

---
base-commit: 9442b1c919af9aed513eb0a484fe96358a500cf5
change-id: 20250408-pks-split-object-file-c61d7cd2a21f


^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH v3 01/10] object-file: move `mkdir_in_gitdir()` into "path.c"
  2025-04-15  9:38 ` [PATCH v3 00/10] Split up "object-file.c" Patrick Steinhardt
@ 2025-04-15  9:38   ` Patrick Steinhardt
  2025-04-15  9:38   ` [PATCH v3 02/10] object-file: move `safe_create_leading_directories()` " Patrick Steinhardt
                     ` (9 subsequent siblings)
  10 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-15  9:38 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Eric Sunshine, Jeff King, Junio C Hamano

The `mkdir_in_gitdir()` function is similar to `safe_create_dir()`, but
the former is hosted in "object-file.c" whereas the latter is hosted in
"path.c". The latter code unit makes way more sense though as the logic
has nothing to do with object files in particular.

Move the file into "path.c". While at it, we:

  - Rename the function to `safe_create_dir_in_gitdir()` so that the
    function names are similar to one another.

  - Remove the dependency on `the_repository` by making the callers pass
    the repository instead.

Adjust callers accordingly.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/rebase.c |  3 ++-
 object-file.c    | 33 ++-------------------------------
 object-file.h    |  2 --
 path.c           | 29 +++++++++++++++++++++++++++++
 path.h           | 11 +++++++++++
 rerere.c         |  5 +++--
 6 files changed, 47 insertions(+), 36 deletions(-)

diff --git a/builtin/rebase.c b/builtin/rebase.c
index 1cd802dd92a..cd9371536f8 100644
--- a/builtin/rebase.c
+++ b/builtin/rebase.c
@@ -267,7 +267,8 @@ static int init_basic_state(struct replay_opts *opts, const char *head_name,
 {
 	FILE *interactive;
 
-	if (!is_directory(merge_dir()) && mkdir_in_gitdir(merge_dir()))
+	if (!is_directory(merge_dir()) &&
+	    safe_create_dir_in_gitdir(the_repository, merge_dir()))
 		return error_errno(_("could not create temporary %s"), merge_dir());
 
 	refs_delete_reflog(get_main_ref_store(the_repository), "REBASE_HEAD");
diff --git a/object-file.c b/object-file.c
index 772c311f188..00451876bd0 100644
--- a/object-file.c
+++ b/object-file.c
@@ -90,36 +90,6 @@ static int get_conv_flags(unsigned flags)
 		return 0;
 }
 
-
-int mkdir_in_gitdir(const char *path)
-{
-	if (mkdir(path, 0777)) {
-		int saved_errno = errno;
-		struct stat st;
-		struct strbuf sb = STRBUF_INIT;
-
-		if (errno != EEXIST)
-			return -1;
-		/*
-		 * Are we looking at a path in a symlinked worktree
-		 * whose original repository does not yet have it?
-		 * e.g. .git/rr-cache pointing at its original
-		 * repository in which the user hasn't performed any
-		 * conflict resolution yet?
-		 */
-		if (lstat(path, &st) || !S_ISLNK(st.st_mode) ||
-		    strbuf_readlink(&sb, path, st.st_size) ||
-		    !is_absolute_path(sb.buf) ||
-		    mkdir(sb.buf, 0777)) {
-			strbuf_release(&sb);
-			errno = saved_errno;
-			return -1;
-		}
-		strbuf_release(&sb);
-	}
-	return adjust_shared_perm(the_repository, path);
-}
-
 static enum scld_error safe_create_leading_directories_1(char *path, int share)
 {
 	char *next_component = path + offset_1st_component(path);
@@ -2196,7 +2166,8 @@ int stream_loose_object(struct input_stream *in_stream, size_t len,
 		struct strbuf dir = STRBUF_INIT;
 		strbuf_add(&dir, filename.buf, dirlen);
 
-		if (mkdir_in_gitdir(dir.buf) && errno != EEXIST) {
+		if (safe_create_dir_in_gitdir(the_repository, dir.buf) &&
+		    errno != EEXIST) {
 			err = error_errno(_("unable to create directory %s"), dir.buf);
 			strbuf_release(&dir);
 			goto cleanup;
diff --git a/object-file.h b/object-file.h
index 81b30d269c8..4649a3f37d4 100644
--- a/object-file.h
+++ b/object-file.h
@@ -54,8 +54,6 @@ enum scld_error safe_create_leading_directories(char *path);
 enum scld_error safe_create_leading_directories_const(const char *path);
 enum scld_error safe_create_leading_directories_no_share(char *path);
 
-int mkdir_in_gitdir(const char *path);
-
 int git_open_cloexec(const char *name, int flags);
 #define git_open(name) git_open_cloexec(name, O_RDONLY)
 
diff --git a/path.c b/path.c
index 910756c8b32..c688f874580 100644
--- a/path.c
+++ b/path.c
@@ -902,6 +902,35 @@ void safe_create_dir(struct repository *repo, const char *dir, int share)
 		die(_("Could not make %s writable by group"), dir);
 }
 
+int safe_create_dir_in_gitdir(struct repository *repo, const char *path)
+{
+	if (mkdir(path, 0777)) {
+		int saved_errno = errno;
+		struct stat st;
+		struct strbuf sb = STRBUF_INIT;
+
+		if (errno != EEXIST)
+			return -1;
+		/*
+		 * Are we looking at a path in a symlinked worktree
+		 * whose original repository does not yet have it?
+		 * e.g. .git/rr-cache pointing at its original
+		 * repository in which the user hasn't performed any
+		 * conflict resolution yet?
+		 */
+		if (lstat(path, &st) || !S_ISLNK(st.st_mode) ||
+		    strbuf_readlink(&sb, path, st.st_size) ||
+		    !is_absolute_path(sb.buf) ||
+		    mkdir(sb.buf, 0777)) {
+			strbuf_release(&sb);
+			errno = saved_errno;
+			return -1;
+		}
+		strbuf_release(&sb);
+	}
+	return adjust_shared_perm(repo, path);
+}
+
 static int have_same_root(const char *path1, const char *path2)
 {
 	int is_abs1, is_abs2;
diff --git a/path.h b/path.h
index 65fe968a13a..a427516d818 100644
--- a/path.h
+++ b/path.h
@@ -221,6 +221,17 @@ char *xdg_cache_home(const char *filename);
  */
 void safe_create_dir(struct repository *repo, const char *dir, int share);
 
+/*
+ * Similar to `safe_create_dir()`, but with two differences:
+ *
+ *   - It knows to resolve gitlink files for symlinked worktrees.
+ *
+ *   - It always adjusts shared permissions.
+ *
+ * Returns a negative erorr code on error, 0 on success.
+ */
+int safe_create_dir_in_gitdir(struct repository *repo, const char *path);
+
 # ifdef USE_THE_REPOSITORY_VARIABLE
 #  include "strbuf.h"
 #  include "repository.h"
diff --git a/rerere.c b/rerere.c
index 740e8ad1a0b..0832cc54840 100644
--- a/rerere.c
+++ b/rerere.c
@@ -860,7 +860,7 @@ static int do_plain_rerere(struct repository *r,
 		string_list_insert(rr, path)->util = id;
 
 		/* Ensure that the directory exists. */
-		mkdir_in_gitdir(rerere_path(&buf, id, NULL));
+		safe_create_dir_in_gitdir(the_repository, rerere_path(&buf, id, NULL));
 	}
 
 	for (i = 0; i < rr->nr; i++)
@@ -895,7 +895,8 @@ static int is_rerere_enabled(void)
 	if (rerere_enabled < 0)
 		return rr_cache_exists;
 
-	if (!rr_cache_exists && mkdir_in_gitdir(git_path_rr_cache()))
+	if (!rr_cache_exists &&
+	    safe_create_dir_in_gitdir(the_repository, git_path_rr_cache()))
 		die(_("could not create directory '%s'"), git_path_rr_cache());
 	return 1;
 }

-- 
2.49.0.805.g082f7c87e0.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH v3 02/10] object-file: move `safe_create_leading_directories()` into "path.c"
  2025-04-15  9:38 ` [PATCH v3 00/10] Split up "object-file.c" Patrick Steinhardt
  2025-04-15  9:38   ` [PATCH v3 01/10] object-file: move `mkdir_in_gitdir()` into "path.c" Patrick Steinhardt
@ 2025-04-15  9:38   ` Patrick Steinhardt
  2025-04-15  9:38   ` [PATCH v3 03/10] object-file: move `git_open_cloexec()` to "compat/open.c" Patrick Steinhardt
                     ` (8 subsequent siblings)
  10 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-15  9:38 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Eric Sunshine, Jeff King, Junio C Hamano

The `safe_create_leading_directories()` function and its relatives are
located in "object-file.c", which is not a good fit as they provide
generic functionality not related to objects at all. Move them into
"path.c", which already hosts `safe_create_dir()` and its relative
`safe_create_dir_in_gitdir()`.

"path.c" is free of `the_repository`, but the moved functions depend on
`the_repository` to read the "core.sharedRepository" config. Adapt the
function signature to accept a repository as argument to fix the issue
and adjust callers accordingly.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/bugreport.c                |  4 +-
 builtin/clone.c                    |  4 +-
 builtin/credential-cache--daemon.c |  4 +-
 builtin/diagnose.c                 |  4 +-
 builtin/difftool.c                 | 29 ++++++++------
 builtin/fast-import.c              |  2 +-
 builtin/fsck.c                     |  2 +-
 builtin/gc.c                       |  7 ++--
 builtin/init-db.c                  |  3 +-
 builtin/log.c                      |  4 +-
 builtin/mv.c                       |  3 +-
 builtin/sparse-checkout.c          |  5 ++-
 builtin/submodule--helper.c        |  4 +-
 builtin/worktree.c                 |  8 ++--
 commit-graph.c                     |  2 +-
 dir.c                              |  5 +--
 merge-recursive.c                  |  4 +-
 midx-write.c                       |  2 +-
 notes-merge.c                      |  4 +-
 object-file.c                      | 81 +-------------------------------------
 object-file.h                      | 33 ----------------
 path.c                             | 80 +++++++++++++++++++++++++++++++++++++
 path.h                             | 34 ++++++++++++++++
 refs/files-backend.c               |  4 +-
 sequencer.c                        |  4 +-
 server-info.c                      |  2 +-
 submodule.c                        |  2 +-
 27 files changed, 173 insertions(+), 167 deletions(-)

diff --git a/builtin/bugreport.c b/builtin/bugreport.c
index 66d64bfd5ae..f78c3f2aed6 100644
--- a/builtin/bugreport.c
+++ b/builtin/bugreport.c
@@ -4,13 +4,13 @@
 #include "editor.h"
 #include "gettext.h"
 #include "parse-options.h"
+#include "path.h"
 #include "strbuf.h"
 #include "help.h"
 #include "compat/compiler.h"
 #include "hook.h"
 #include "hook-list.h"
 #include "diagnose.h"
-#include "object-file.h"
 #include "setup.h"
 #include "version.h"
 
@@ -141,7 +141,7 @@ int cmd_bugreport(int argc,
 	}
 	strbuf_addstr(&report_path, ".txt");
 
-	switch (safe_create_leading_directories(report_path.buf)) {
+	switch (safe_create_leading_directories(the_repository, report_path.buf)) {
 	case SCLD_OK:
 	case SCLD_EXISTS:
 		break;
diff --git a/builtin/clone.c b/builtin/clone.c
index 2993acb630e..31f2198c1b3 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -1090,7 +1090,7 @@ int cmd_clone(int argc,
 	sigchain_push_common(remove_junk_on_signal);
 
 	if (!option_bare) {
-		if (safe_create_leading_directories_const(work_tree) < 0)
+		if (safe_create_leading_directories_const(the_repository, work_tree) < 0)
 			die_errno(_("could not create leading directories of '%s'"),
 				  work_tree);
 		if (dest_exists)
@@ -1111,7 +1111,7 @@ int cmd_clone(int argc,
 			junk_git_dir_flags |= REMOVE_DIR_KEEP_TOPLEVEL;
 		junk_git_dir = git_dir;
 	}
-	if (safe_create_leading_directories_const(git_dir) < 0)
+	if (safe_create_leading_directories_const(the_repository, git_dir) < 0)
 		die(_("could not create leading directories of '%s'"), git_dir);
 
 	if (0 <= option_verbosity) {
diff --git a/builtin/credential-cache--daemon.c b/builtin/credential-cache--daemon.c
index e707618e743..5065ff4660b 100644
--- a/builtin/credential-cache--daemon.c
+++ b/builtin/credential-cache--daemon.c
@@ -2,8 +2,8 @@
 #include "builtin.h"
 #include "abspath.h"
 #include "gettext.h"
-#include "object-file.h"
 #include "parse-options.h"
+#include "path.h"
 
 #ifndef NO_UNIX_SOCKETS
 
@@ -271,7 +271,7 @@ static void init_socket_directory(const char *path)
 		 * condition in which somebody can chdir to it, sleep, then try to open
 		 * our protected socket.
 		 */
-		if (safe_create_leading_directories_const(dir) < 0)
+		if (safe_create_leading_directories_const(the_repository, dir) < 0)
 			die_errno("unable to create directories for '%s'", dir);
 		if (mkdir(dir, 0700) < 0)
 			die_errno("unable to mkdir '%s'", dir);
diff --git a/builtin/diagnose.c b/builtin/diagnose.c
index 33c39bd5981..ec86d66389e 100644
--- a/builtin/diagnose.c
+++ b/builtin/diagnose.c
@@ -3,8 +3,8 @@
 #include "builtin.h"
 #include "abspath.h"
 #include "gettext.h"
-#include "object-file.h"
 #include "parse-options.h"
+#include "path.h"
 #include "diagnose.h"
 
 static const char * const diagnose_usage[] = {
@@ -50,7 +50,7 @@ int cmd_diagnose(int argc,
 	strbuf_addftime(&zip_path, option_suffix, localtime_r(&now, &tm), 0, 0);
 	strbuf_addstr(&zip_path, ".zip");
 
-	switch (safe_create_leading_directories(zip_path.buf)) {
+	switch (safe_create_leading_directories(the_repository, zip_path.buf)) {
 	case SCLD_OK:
 	case SCLD_EXISTS:
 		break;
diff --git a/builtin/difftool.c b/builtin/difftool.c
index 41cd00066cc..8292aedaaf0 100644
--- a/builtin/difftool.c
+++ b/builtin/difftool.c
@@ -22,6 +22,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "parse-options.h"
+#include "path.h"
 #include "read-cache-ll.h"
 #include "repository.h"
 #include "sparse-index.h"
@@ -271,9 +272,9 @@ static void changed_files(struct repository *repo,
 	strbuf_release(&buf);
 }
 
-static int ensure_leading_directories(char *path)
+static int ensure_leading_directories(struct repository *repo, char *path)
 {
-	switch (safe_create_leading_directories(path)) {
+	switch (safe_create_leading_directories(repo, path)) {
 		case SCLD_OK:
 		case SCLD_EXISTS:
 			return 0;
@@ -341,11 +342,12 @@ static int checkout_path(unsigned mode, struct object_id *oid,
 	return ret;
 }
 
-static void write_file_in_directory(struct strbuf *dir, size_t dir_len,
-			const char *path, const char *content)
+static void write_file_in_directory(struct repository *repo,
+				    struct strbuf *dir, size_t dir_len,
+				    const char *path, const char *content)
 {
 	add_path(dir, dir_len, path);
-	ensure_leading_directories(dir->buf);
+	ensure_leading_directories(repo, dir->buf);
 	unlink(dir->buf);
 	write_file(dir->buf, "%s", content);
 }
@@ -356,14 +358,15 @@ static void write_file_in_directory(struct strbuf *dir, size_t dir_len,
  * as text files, resulting in behavior that is analogous to what "git diff"
  * displays for symlink and submodule diffs.
  */
-static void write_standin_files(struct pair_entry *entry,
-			struct strbuf *ldir, size_t ldir_len,
-			struct strbuf *rdir, size_t rdir_len)
+static void write_standin_files(struct repository *repo,
+				struct pair_entry *entry,
+				struct strbuf *ldir, size_t ldir_len,
+				struct strbuf *rdir, size_t rdir_len)
 {
 	if (*entry->left)
-		write_file_in_directory(ldir, ldir_len, entry->path, entry->left);
+		write_file_in_directory(repo, ldir, ldir_len, entry->path, entry->left);
 	if (*entry->right)
-		write_file_in_directory(rdir, rdir_len, entry->path, entry->right);
+		write_file_in_directory(repo, rdir, rdir_len, entry->path, entry->right);
 }
 
 static int run_dir_diff(struct repository *repo,
@@ -533,7 +536,7 @@ static int run_dir_diff(struct repository *repo,
 						ADD_CACHE_JUST_APPEND);
 
 				add_path(&rdir, rdir_len, dst_path);
-				if (ensure_leading_directories(rdir.buf)) {
+				if (ensure_leading_directories(repo, rdir.buf)) {
 					ret = error("could not create "
 						    "directory for '%s'",
 						    dst_path);
@@ -576,7 +579,7 @@ static int run_dir_diff(struct repository *repo,
 	 */
 	hashmap_for_each_entry(&submodules, &iter, entry,
 				entry /* member name */) {
-		write_standin_files(entry, &ldir, ldir_len, &rdir, rdir_len);
+		write_standin_files(repo, entry, &ldir, ldir_len, &rdir, rdir_len);
 	}
 
 	/*
@@ -587,7 +590,7 @@ static int run_dir_diff(struct repository *repo,
 	hashmap_for_each_entry(&symlinks2, &iter, entry,
 				entry /* member name */) {
 
-		write_standin_files(entry, &ldir, ldir_len, &rdir, rdir_len);
+		write_standin_files(repo, entry, &ldir, ldir_len, &rdir, rdir_len);
 	}
 
 	strbuf_setlen(&ldir, ldir_len);
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index 63880b595cc..e4523cc6f1b 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -1720,7 +1720,7 @@ static void dump_marks(void)
 	if (!export_marks_file || (import_marks_file && !import_marks_file_done))
 		return;
 
-	if (safe_create_leading_directories_const(export_marks_file)) {
+	if (safe_create_leading_directories_const(the_repository, export_marks_file)) {
 		failure |= error_errno("unable to create leading directories of %s",
 				       export_marks_file);
 		return;
diff --git a/builtin/fsck.c b/builtin/fsck.c
index 9c8a6d6a8df..92312b59591 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -332,7 +332,7 @@ static void check_unreachable_object(struct object *obj)
 				describe_object(&obj->oid));
 			FILE *f;
 
-			if (safe_create_leading_directories_const(filename)) {
+			if (safe_create_leading_directories_const(the_repository, filename)) {
 				error(_("could not create lost-found"));
 				free(filename);
 				return;
diff --git a/builtin/gc.c b/builtin/gc.c
index 99431fd4674..dae1e545514 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -28,7 +28,6 @@
 #include "commit.h"
 #include "commit-graph.h"
 #include "packfile.h"
-#include "object-file.h"
 #include "object-store-ll.h"
 #include "pack.h"
 #include "pack-objects.h"
@@ -2099,7 +2098,7 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
 	}
 	strbuf_addstr(&plist, "</array>\n</dict>\n</plist>\n");
 
-	if (safe_create_leading_directories(filename))
+	if (safe_create_leading_directories(the_repository, filename))
 		die(_("failed to create directories for '%s'"), filename);
 
 	if ((long)lock_file_timeout_ms < 0 &&
@@ -2565,7 +2564,7 @@ static int systemd_timer_write_timer_file(enum schedule_priority schedule,
 
 	filename = xdg_config_home_systemd(local_timer_name);
 
-	if (safe_create_leading_directories(filename)) {
+	if (safe_create_leading_directories(the_repository, filename)) {
 		error(_("failed to create directories for '%s'"), filename);
 		goto error;
 	}
@@ -2638,7 +2637,7 @@ static int systemd_timer_write_service_template(const char *exec_path)
 	char *local_service_name = xstrfmt(SYSTEMD_UNIT_FORMAT, "", "service");
 
 	filename = xdg_config_home_systemd(local_service_name);
-	if (safe_create_leading_directories(filename)) {
+	if (safe_create_leading_directories(the_repository, filename)) {
 		error(_("failed to create directories for '%s'"), filename);
 		goto error;
 	}
diff --git a/builtin/init-db.c b/builtin/init-db.c
index 196dccdd77a..91c2563e341 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -8,7 +8,6 @@
 #include "abspath.h"
 #include "environment.h"
 #include "gettext.h"
-#include "object-file.h"
 #include "parse-options.h"
 #include "path.h"
 #include "refs.h"
@@ -134,7 +133,7 @@ int cmd_init_db(int argc,
 				 */
 				saved = repo_settings_get_shared_repository(the_repository);
 				repo_settings_set_shared_repository(the_repository, 0);
-				switch (safe_create_leading_directories_const(argv[0])) {
+				switch (safe_create_leading_directories_const(the_repository, argv[0])) {
 				case SCLD_OK:
 				case SCLD_PERMS:
 					break;
diff --git a/builtin/log.c b/builtin/log.c
index 0d4c579dad7..516c9ec8400 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -14,7 +14,6 @@
 #include "gettext.h"
 #include "hex.h"
 #include "refs.h"
-#include "object-file.h"
 #include "object-name.h"
 #include "object-store-ll.h"
 #include "pager.h"
@@ -29,6 +28,7 @@
 #include "tag.h"
 #include "reflog-walk.h"
 #include "patch-ids.h"
+#include "path.h"
 #include "shortlog.h"
 #include "remote.h"
 #include "string-list.h"
@@ -2311,7 +2311,7 @@ int cmd_format_patch(int argc,
 		 */
 		saved = repo_settings_get_shared_repository(the_repository);
 		repo_settings_set_shared_repository(the_repository, 0);
-		switch (safe_create_leading_directories_const(output_directory)) {
+		switch (safe_create_leading_directories_const(the_repository, output_directory)) {
 		case SCLD_OK:
 		case SCLD_EXISTS:
 			break;
diff --git a/builtin/mv.c b/builtin/mv.c
index 55a7d471dca..99fe7a0c561 100644
--- a/builtin/mv.c
+++ b/builtin/mv.c
@@ -15,6 +15,7 @@
 #include "gettext.h"
 #include "name-hash.h"
 #include "object-file.h"
+#include "path.h"
 #include "pathspec.h"
 #include "lockfile.h"
 #include "dir.h"
@@ -555,7 +556,7 @@ int cmd_mv(int argc,
 					 */
 					char *dst_dup = xstrdup(dst);
 					string_list_append(&dirty_paths, dst);
-					safe_create_leading_directories(dst_dup);
+					safe_create_leading_directories(the_repository, dst_dup);
 					FREE_AND_NULL(dst_dup);
 					rename(src, dst);
 				}
diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c
index 14dcace5f8f..1bf01591b27 100644
--- a/builtin/sparse-checkout.c
+++ b/builtin/sparse-checkout.c
@@ -9,6 +9,7 @@
 #include "object-file.h"
 #include "object-name.h"
 #include "parse-options.h"
+#include "path.h"
 #include "pathspec.h"
 #include "strbuf.h"
 #include "string-list.h"
@@ -335,7 +336,7 @@ static int write_patterns_and_update(struct pattern_list *pl)
 
 	sparse_filename = get_sparse_checkout_filename();
 
-	if (safe_create_leading_directories(sparse_filename))
+	if (safe_create_leading_directories(the_repository, sparse_filename))
 		die(_("failed to create directory for sparse-checkout file"));
 
 	hold_lock_file_for_update(&lk, sparse_filename, LOCK_DIE_ON_ERROR);
@@ -491,7 +492,7 @@ static int sparse_checkout_init(int argc, const char **argv, const char *prefix,
 		FILE *fp;
 
 		/* assume we are in a fresh repo, but update the sparse-checkout file */
-		if (safe_create_leading_directories(sparse_filename))
+		if (safe_create_leading_directories(the_repository, sparse_filename))
 			die(_("unable to create leading directories of %s"),
 			    sparse_filename);
 		fp = xfopen(sparse_filename, "w");
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 570226ea166..cc001d0b4cd 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1739,7 +1739,7 @@ static int clone_submodule(const struct module_clone_data *clone_data,
 		    !is_empty_dir(clone_data_path))
 			die(_("directory not empty: '%s'"), clone_data_path);
 
-		if (safe_create_leading_directories_const(sm_gitdir) < 0)
+		if (safe_create_leading_directories_const(the_repository, sm_gitdir) < 0)
 			die(_("could not create directory '%s'"), sm_gitdir);
 
 		prepare_possible_alternates(clone_data->name, reference);
@@ -1800,7 +1800,7 @@ static int clone_submodule(const struct module_clone_data *clone_data,
 		if (clone_data->require_init && !stat(clone_data_path, &st) &&
 		    !is_empty_dir(clone_data_path))
 			die(_("directory not empty: '%s'"), clone_data_path);
-		if (safe_create_leading_directories_const(clone_data_path) < 0)
+		if (safe_create_leading_directories_const(the_repository, clone_data_path) < 0)
 			die(_("could not create directory '%s'"), clone_data_path);
 		path = xstrfmt("%s/index", sm_gitdir);
 		unlink_or_warn(path);
diff --git a/builtin/worktree.c b/builtin/worktree.c
index 87ccd47794c..88a36ea9f86 100644
--- a/builtin/worktree.c
+++ b/builtin/worktree.c
@@ -348,7 +348,7 @@ static void copy_sparse_checkout(const char *worktree_git_dir)
 	char *to_file = xstrfmt("%s/info/sparse-checkout", worktree_git_dir);
 
 	if (file_exists(from_file)) {
-		if (safe_create_leading_directories(to_file) ||
+		if (safe_create_leading_directories(the_repository, to_file) ||
 			copy_file(to_file, from_file, 0666))
 			error(_("failed to copy '%s' to '%s'; sparse-checkout may not work correctly"),
 				from_file, to_file);
@@ -367,7 +367,7 @@ static void copy_filtered_worktree_config(const char *worktree_git_dir)
 		struct config_set cs = { { 0 } };
 		int bare;
 
-		if (safe_create_leading_directories(to_file) ||
+		if (safe_create_leading_directories(the_repository, to_file) ||
 			copy_file(to_file, from_file, 0666)) {
 			error(_("failed to copy worktree config from '%s' to '%s'"),
 				from_file, to_file);
@@ -466,7 +466,7 @@ static int add_worktree(const char *path, const char *refname,
 	name = sb_name.buf;
 	repo_git_path_replace(the_repository, &sb_repo, "worktrees/%s", name);
 	len = sb_repo.len;
-	if (safe_create_leading_directories_const(sb_repo.buf))
+	if (safe_create_leading_directories_const(the_repository, sb_repo.buf))
 		die_errno(_("could not create leading directories of '%s'"),
 			  sb_repo.buf);
 
@@ -498,7 +498,7 @@ static int add_worktree(const char *path, const char *refname,
 		write_file(sb.buf, _("initializing"));
 
 	strbuf_addf(&sb_git, "%s/.git", path);
-	if (safe_create_leading_directories_const(sb_git.buf))
+	if (safe_create_leading_directories_const(the_repository, sb_git.buf))
 		die_errno(_("could not create leading directories of '%s'"),
 			  sb_git.buf);
 	junk_work_tree = xstrdup(path);
diff --git a/commit-graph.c b/commit-graph.c
index 8286d5dda24..3b5bae00af9 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -2065,7 +2065,7 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)
 		ctx->graph_name = get_commit_graph_filename(ctx->odb);
 	}
 
-	if (safe_create_leading_directories(ctx->graph_name)) {
+	if (safe_create_leading_directories(the_repository, ctx->graph_name)) {
 		error(_("unable to create leading directories of %s"),
 			ctx->graph_name);
 		return -1;
diff --git a/dir.c b/dir.c
index 28b0e03feb4..49008739b9b 100644
--- a/dir.c
+++ b/dir.c
@@ -17,7 +17,6 @@
 #include "environment.h"
 #include "gettext.h"
 #include "name-hash.h"
-#include "object-file.h"
 #include "object-store-ll.h"
 #include "path.h"
 #include "refs.h"
@@ -4063,12 +4062,12 @@ void connect_work_tree_and_git_dir(const char *work_tree_,
 
 	/* Prepare .git file */
 	strbuf_addf(&gitfile_sb, "%s/.git", work_tree_);
-	if (safe_create_leading_directories_const(gitfile_sb.buf))
+	if (safe_create_leading_directories_const(the_repository, gitfile_sb.buf))
 		die(_("could not create directories for %s"), gitfile_sb.buf);
 
 	/* Prepare config file */
 	strbuf_addf(&cfg_sb, "%s/config", git_dir_);
-	if (safe_create_leading_directories_const(cfg_sb.buf))
+	if (safe_create_leading_directories_const(the_repository, cfg_sb.buf))
 		die(_("could not create directories for %s"), cfg_sb.buf);
 
 	git_dir = real_pathdup(git_dir_, 1);
diff --git a/merge-recursive.c b/merge-recursive.c
index 9aedffc546b..f71490517e1 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -910,7 +910,7 @@ static int make_room_for_path(struct merge_options *opt, const char *path)
 	}
 
 	/* Make sure leading directories are created */
-	status = safe_create_leading_directories_const(path);
+	status = safe_create_leading_directories_const(the_repository, path);
 	if (status) {
 		if (status == SCLD_EXISTS)
 			/* something else exists */
@@ -1003,7 +1003,7 @@ static int update_file_flags(struct merge_options *opt,
 			close(fd);
 		} else if (S_ISLNK(contents->mode)) {
 			char *lnk = xmemdupz(buf, size);
-			safe_create_leading_directories_const(path);
+			safe_create_leading_directories_const(the_repository, path);
 			unlink(path);
 			if (symlink(lnk, path))
 				ret = err(opt, _("failed to symlink '%s': %s"),
diff --git a/midx-write.c b/midx-write.c
index a628ac24dcb..fbba55f9d92 100644
--- a/midx-write.c
+++ b/midx-write.c
@@ -1086,7 +1086,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,
 			    object_dir);
 	else
 		get_midx_filename(r->hash_algo, &midx_name, object_dir);
-	if (safe_create_leading_directories(midx_name.buf))
+	if (safe_create_leading_directories(r, midx_name.buf))
 		die_errno(_("unable to create leading directories of %s"),
 			  midx_name.buf);
 
diff --git a/notes-merge.c b/notes-merge.c
index 5008faef450..fce45043655 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -296,7 +296,7 @@ static void check_notes_merge_worktree(struct notes_merge_options *o)
 				    "(%s exists)."), repo_git_path_replace(the_repository, &buf, "NOTES_MERGE_*"));
 		}
 
-		if (safe_create_leading_directories_const(repo_git_path_replace(the_repository, &buf,
+		if (safe_create_leading_directories_const(the_repository, repo_git_path_replace(the_repository, &buf,
 				NOTES_MERGE_WORKTREE "/.test")))
 			die_errno("unable to create directory %s",
 				  repo_git_path_replace(the_repository, &buf, NOTES_MERGE_WORKTREE));
@@ -314,7 +314,7 @@ static void write_buf_to_worktree(const struct object_id *obj,
 {
 	int fd;
 	char *path = repo_git_path(the_repository, NOTES_MERGE_WORKTREE "/%s", oid_to_hex(obj));
-	if (safe_create_leading_directories_const(path))
+	if (safe_create_leading_directories_const(the_repository, path))
 		die_errno("unable to create directory for '%s'", path);
 
 	fd = xopen(path, O_WRONLY | O_EXCL | O_CREAT, 0666);
diff --git a/object-file.c b/object-file.c
index 00451876bd0..6228e1c40f8 100644
--- a/object-file.c
+++ b/object-file.c
@@ -90,83 +90,6 @@ static int get_conv_flags(unsigned flags)
 		return 0;
 }
 
-static enum scld_error safe_create_leading_directories_1(char *path, int share)
-{
-	char *next_component = path + offset_1st_component(path);
-	enum scld_error ret = SCLD_OK;
-
-	while (ret == SCLD_OK && next_component) {
-		struct stat st;
-		char *slash = next_component, slash_character;
-
-		while (*slash && !is_dir_sep(*slash))
-			slash++;
-
-		if (!*slash)
-			break;
-
-		next_component = slash + 1;
-		while (is_dir_sep(*next_component))
-			next_component++;
-		if (!*next_component)
-			break;
-
-		slash_character = *slash;
-		*slash = '\0';
-		if (!stat(path, &st)) {
-			/* path exists */
-			if (!S_ISDIR(st.st_mode)) {
-				errno = ENOTDIR;
-				ret = SCLD_EXISTS;
-			}
-		} else if (mkdir(path, 0777)) {
-			if (errno == EEXIST &&
-			    !stat(path, &st) && S_ISDIR(st.st_mode))
-				; /* somebody created it since we checked */
-			else if (errno == ENOENT)
-				/*
-				 * Either mkdir() failed because
-				 * somebody just pruned the containing
-				 * directory, or stat() failed because
-				 * the file that was in our way was
-				 * just removed.  Either way, inform
-				 * the caller that it might be worth
-				 * trying again:
-				 */
-				ret = SCLD_VANISHED;
-			else
-				ret = SCLD_FAILED;
-		} else if (share && adjust_shared_perm(the_repository, path)) {
-			ret = SCLD_PERMS;
-		}
-		*slash = slash_character;
-	}
-	return ret;
-}
-
-enum scld_error safe_create_leading_directories(char *path)
-{
-	return safe_create_leading_directories_1(path, 1);
-}
-
-enum scld_error safe_create_leading_directories_no_share(char *path)
-{
-	return safe_create_leading_directories_1(path, 0);
-}
-
-enum scld_error safe_create_leading_directories_const(const char *path)
-{
-	int save_errno;
-	/* path points to cache entries, so xstrdup before messing with it */
-	char *buf = xstrdup(path);
-	enum scld_error result = safe_create_leading_directories(buf);
-
-	save_errno = errno;
-	free(buf);
-	errno = save_errno;
-	return result;
-}
-
 int odb_mkstemp(struct strbuf *temp_filename, const char *pattern)
 {
 	int fd;
@@ -183,7 +106,7 @@ int odb_mkstemp(struct strbuf *temp_filename, const char *pattern)
 	/* slow path */
 	/* some mkstemp implementations erase temp_filename on failure */
 	repo_git_path_replace(the_repository, temp_filename, "objects/%s", pattern);
-	safe_create_leading_directories(temp_filename->buf);
+	safe_create_leading_directories(the_repository, temp_filename->buf);
 	return xmkstemp_mode(temp_filename->buf, mode);
 }
 
@@ -196,7 +119,7 @@ int odb_pack_keep(const char *name)
 		return fd;
 
 	/* slow path */
-	safe_create_leading_directories_const(name);
+	safe_create_leading_directories_const(the_repository, name);
 	return open(name, O_RDWR|O_CREAT|O_EXCL, 0600);
 }
 
diff --git a/object-file.h b/object-file.h
index 4649a3f37d4..922f2bba8c9 100644
--- a/object-file.h
+++ b/object-file.h
@@ -21,39 +21,6 @@ extern int fetch_if_missing;
 int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);
 int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags);
 
-/*
- * Create the directory containing the named path, using care to be
- * somewhat safe against races. Return one of the scld_error values to
- * indicate success/failure. On error, set errno to describe the
- * problem.
- *
- * SCLD_VANISHED indicates that one of the ancestor directories of the
- * path existed at one point during the function call and then
- * suddenly vanished, probably because another process pruned the
- * directory while we were working.  To be robust against this kind of
- * race, callers might want to try invoking the function again when it
- * returns SCLD_VANISHED.
- *
- * safe_create_leading_directories() temporarily changes path while it
- * is working but restores it before returning.
- * safe_create_leading_directories_const() doesn't modify path, even
- * temporarily. Both these variants adjust the permissions of the
- * created directories to honor core.sharedRepository, so they are best
- * suited for files inside the git dir. For working tree files, use
- * safe_create_leading_directories_no_share() instead, as it ignores
- * the core.sharedRepository setting.
- */
-enum scld_error {
-	SCLD_OK = 0,
-	SCLD_FAILED = -1,
-	SCLD_PERMS = -2,
-	SCLD_EXISTS = -3,
-	SCLD_VANISHED = -4
-};
-enum scld_error safe_create_leading_directories(char *path);
-enum scld_error safe_create_leading_directories_const(const char *path);
-enum scld_error safe_create_leading_directories_no_share(char *path);
-
 int git_open_cloexec(const char *name, int flags);
 #define git_open(name) git_open_cloexec(name, O_RDONLY)
 
diff --git a/path.c b/path.c
index c688f874580..62d67166dff 100644
--- a/path.c
+++ b/path.c
@@ -931,6 +931,86 @@ int safe_create_dir_in_gitdir(struct repository *repo, const char *path)
 	return adjust_shared_perm(repo, path);
 }
 
+static enum scld_error safe_create_leading_directories_1(struct repository *repo,
+							 char *path)
+{
+	char *next_component = path + offset_1st_component(path);
+	enum scld_error ret = SCLD_OK;
+
+	while (ret == SCLD_OK && next_component) {
+		struct stat st;
+		char *slash = next_component, slash_character;
+
+		while (*slash && !is_dir_sep(*slash))
+			slash++;
+
+		if (!*slash)
+			break;
+
+		next_component = slash + 1;
+		while (is_dir_sep(*next_component))
+			next_component++;
+		if (!*next_component)
+			break;
+
+		slash_character = *slash;
+		*slash = '\0';
+		if (!stat(path, &st)) {
+			/* path exists */
+			if (!S_ISDIR(st.st_mode)) {
+				errno = ENOTDIR;
+				ret = SCLD_EXISTS;
+			}
+		} else if (mkdir(path, 0777)) {
+			if (errno == EEXIST &&
+			    !stat(path, &st) && S_ISDIR(st.st_mode))
+				; /* somebody created it since we checked */
+			else if (errno == ENOENT)
+				/*
+				 * Either mkdir() failed because
+				 * somebody just pruned the containing
+				 * directory, or stat() failed because
+				 * the file that was in our way was
+				 * just removed.  Either way, inform
+				 * the caller that it might be worth
+				 * trying again:
+				 */
+				ret = SCLD_VANISHED;
+			else
+				ret = SCLD_FAILED;
+		} else if (repo && adjust_shared_perm(repo, path)) {
+			ret = SCLD_PERMS;
+		}
+		*slash = slash_character;
+	}
+	return ret;
+}
+
+enum scld_error safe_create_leading_directories(struct repository *repo,
+						char *path)
+{
+	return safe_create_leading_directories_1(repo, path);
+}
+
+enum scld_error safe_create_leading_directories_no_share(char *path)
+{
+	return safe_create_leading_directories_1(NULL, path);
+}
+
+enum scld_error safe_create_leading_directories_const(struct repository *repo,
+						      const char *path)
+{
+	int save_errno;
+	/* path points to cache entries, so xstrdup before messing with it */
+	char *buf = xstrdup(path);
+	enum scld_error result = safe_create_leading_directories(repo, buf);
+
+	save_errno = errno;
+	free(buf);
+	errno = save_errno;
+	return result;
+}
+
 static int have_same_root(const char *path1, const char *path2)
 {
 	int is_abs1, is_abs2;
diff --git a/path.h b/path.h
index a427516d818..fd1a194b060 100644
--- a/path.h
+++ b/path.h
@@ -232,6 +232,40 @@ void safe_create_dir(struct repository *repo, const char *dir, int share);
  */
 int safe_create_dir_in_gitdir(struct repository *repo, const char *path);
 
+/*
+ * Create the directory containing the named path, using care to be
+ * somewhat safe against races. Return one of the scld_error values to
+ * indicate success/failure. On error, set errno to describe the
+ * problem.
+ *
+ * SCLD_VANISHED indicates that one of the ancestor directories of the
+ * path existed at one point during the function call and then
+ * suddenly vanished, probably because another process pruned the
+ * directory while we were working.  To be robust against this kind of
+ * race, callers might want to try invoking the function again when it
+ * returns SCLD_VANISHED.
+ *
+ * safe_create_leading_directories() temporarily changes path while it
+ * is working but restores it before returning.
+ * safe_create_leading_directories_const() doesn't modify path, even
+ * temporarily. Both these variants adjust the permissions of the
+ * created directories to honor core.sharedRepository, so they are best
+ * suited for files inside the git dir. For working tree files, use
+ * safe_create_leading_directories_no_share() instead, as it ignores
+ * the core.sharedRepository setting.
+ */
+enum scld_error {
+	SCLD_OK = 0,
+	SCLD_FAILED = -1,
+	SCLD_PERMS = -2,
+	SCLD_EXISTS = -3,
+	SCLD_VANISHED = -4
+};
+enum scld_error safe_create_leading_directories(struct repository *repo, char *path);
+enum scld_error safe_create_leading_directories_const(struct repository *repo,
+						      const char *path);
+enum scld_error safe_create_leading_directories_no_share(char *path);
+
 # ifdef USE_THE_REPOSITORY_VARIABLE
 #  include "strbuf.h"
 #  include "repository.h"
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 91d3aca70a7..10c439a56f8 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -705,7 +705,7 @@ static int lock_raw_ref(struct files_ref_store *refs,
 	files_ref_path(refs, &ref_file, refname);
 
 retry:
-	switch (safe_create_leading_directories(ref_file.buf)) {
+	switch (safe_create_leading_directories(the_repository, ref_file.buf)) {
 	case SCLD_OK:
 		break; /* success */
 	case SCLD_EXISTS:
@@ -1109,7 +1109,7 @@ static int raceproof_create_file(const char *path, create_file_fn fn, void *cb)
 			strbuf_addstr(&path_copy, path);
 
 		do {
-			scld_result = safe_create_leading_directories(path_copy.buf);
+			scld_result = safe_create_leading_directories(the_repository, path_copy.buf);
 			if (scld_result == SCLD_OK)
 				goto retry_fn;
 		} while (scld_result == SCLD_VANISHED && create_directories_remaining-- > 0);
diff --git a/sequencer.c b/sequencer.c
index c112d2e1c43..9fda9be9266 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -4411,7 +4411,7 @@ static int write_update_refs_state(struct string_list *refs_to_oids)
 		goto cleanup;
 	}
 
-	if (safe_create_leading_directories(path)) {
+	if (safe_create_leading_directories(the_repository, path)) {
 		result = error(_("unable to create leading directories of %s"),
 			       path);
 		goto cleanup;
@@ -4677,7 +4677,7 @@ static void create_autostash_internal(struct repository *r,
 		strbuf_add_unique_abbrev(&buf, &oid, DEFAULT_ABBREV);
 
 		if (path) {
-			if (safe_create_leading_directories_const(path))
+			if (safe_create_leading_directories_const(the_repository, path))
 				die(_("Could not create directory for '%s'"),
 				    path);
 			write_file(path, "%s", oid_to_hex(&oid));
diff --git a/server-info.c b/server-info.c
index 1ca0e00d51e..f0646ac92a9 100644
--- a/server-info.c
+++ b/server-info.c
@@ -88,7 +88,7 @@ static int update_info_file(struct repository *r, char *path,
 		.old_sb = STRBUF_INIT
 	};
 
-	safe_create_leading_directories(path);
+	safe_create_leading_directories(r, path);
 	f = mks_tempfile_m(tmp, 0666);
 	if (!f)
 		goto out;
diff --git a/submodule.c b/submodule.c
index 0821507ecaa..218c8c17603 100644
--- a/submodule.c
+++ b/submodule.c
@@ -2384,7 +2384,7 @@ static void relocate_single_git_dir_into_superproject(const char *path,
 	if (validate_submodule_git_dir(new_gitdir.buf, sub->name) < 0)
 		die(_("refusing to move '%s' into an existing git dir"),
 		    real_old_git_dir);
-	if (safe_create_leading_directories_const(new_gitdir.buf) < 0)
+	if (safe_create_leading_directories_const(the_repository, new_gitdir.buf) < 0)
 		die(_("could not create directory '%s'"), new_gitdir.buf);
 	real_new_git_dir = real_pathdup(new_gitdir.buf, 1);
 

-- 
2.49.0.805.g082f7c87e0.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH v3 03/10] object-file: move `git_open_cloexec()` to "compat/open.c"
  2025-04-15  9:38 ` [PATCH v3 00/10] Split up "object-file.c" Patrick Steinhardt
  2025-04-15  9:38   ` [PATCH v3 01/10] object-file: move `mkdir_in_gitdir()` into "path.c" Patrick Steinhardt
  2025-04-15  9:38   ` [PATCH v3 02/10] object-file: move `safe_create_leading_directories()` " Patrick Steinhardt
@ 2025-04-15  9:38   ` Patrick Steinhardt
  2025-04-15  9:38   ` [PATCH v3 04/10] object-file: move `xmmap()` into "wrapper.c" Patrick Steinhardt
                     ` (7 subsequent siblings)
  10 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-15  9:38 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Eric Sunshine, Jeff King, Junio C Hamano

The `git_open_cloexec()` wrapper function provides the ability to open a
file with `O_CLOEXEC` in a platform-agnostic way. This function is
provided by "object-file.c" even though it is not specific to the object
subsystem at all.

Move the file into "compat/open.c". This file already exists before this
commit, but has only been compiled conditionally depending on whether or
not open(3p) may return EINTR. With this change we now unconditionally
compile the object, but wrap `git_open_with_retry()` in an ifdef.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Makefile          |  2 +-
 commit-graph.c    |  1 -
 compat/open.c     | 29 +++++++++++++++++++++++++++++
 git-compat-util.h |  3 +++
 meson.build       |  1 +
 midx.c            |  1 -
 object-file.c     | 27 ---------------------------
 object-file.h     |  3 ---
 pack-bitmap.c     |  1 -
 pack-mtimes.c     |  1 -
 pack-revindex.c   |  1 -
 11 files changed, 34 insertions(+), 36 deletions(-)

diff --git a/Makefile b/Makefile
index c41fc41ef0e..bb5407b4703 100644
--- a/Makefile
+++ b/Makefile
@@ -994,6 +994,7 @@ LIB_OBJS += common-exit.o
 LIB_OBJS += common-init.o
 LIB_OBJS += compat/nonblock.o
 LIB_OBJS += compat/obstack.o
+LIB_OBJS += compat/open.o
 LIB_OBJS += compat/terminal.o
 LIB_OBJS += compiler-tricks/not-constant.o
 LIB_OBJS += config.o
@@ -1812,7 +1813,6 @@ ifdef FREAD_READS_DIRECTORIES
 endif
 ifdef OPEN_RETURNS_EINTR
 	COMPAT_CFLAGS += -DOPEN_RETURNS_EINTR
-	COMPAT_OBJS += compat/open.o
 endif
 ifdef NO_SYMLINK_HEAD
 	BASIC_CFLAGS += -DNO_SYMLINK_HEAD
diff --git a/commit-graph.c b/commit-graph.c
index 3b5bae00af9..9621c454972 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -13,7 +13,6 @@
 #include "refs.h"
 #include "hash-lookup.h"
 #include "commit-graph.h"
-#include "object-file.h"
 #include "object-store-ll.h"
 #include "oid-array.h"
 #include "path.h"
diff --git a/compat/open.c b/compat/open.c
index eb3754a23b8..37ae2b1aeb9 100644
--- a/compat/open.c
+++ b/compat/open.c
@@ -1,5 +1,6 @@
 #include "git-compat-util.h"
 
+#ifdef OPEN_RETURNS_EINTR
 #undef open
 int git_open_with_retry(const char *path, int flags, ...)
 {
@@ -23,3 +24,31 @@ int git_open_with_retry(const char *path, int flags, ...)
 
 	return ret;
 }
+#endif
+
+int git_open_cloexec(const char *name, int flags)
+{
+	int fd;
+	static int o_cloexec = O_CLOEXEC;
+
+	fd = open(name, flags | o_cloexec);
+	if ((o_cloexec & O_CLOEXEC) && fd < 0 && errno == EINVAL) {
+		/* Try again w/o O_CLOEXEC: the kernel might not support it */
+		o_cloexec &= ~O_CLOEXEC;
+		fd = open(name, flags | o_cloexec);
+	}
+
+#if defined(F_GETFD) && defined(F_SETFD) && defined(FD_CLOEXEC)
+	{
+		static int fd_cloexec = FD_CLOEXEC;
+
+		if (!o_cloexec && 0 <= fd && fd_cloexec) {
+			/* Opened w/o O_CLOEXEC?  try with fcntl(2) to add it */
+			int flags = fcntl(fd, F_GETFD);
+			if (fcntl(fd, F_SETFD, flags | fd_cloexec))
+				fd_cloexec = 0;
+		}
+	}
+#endif
+	return fd;
+}
diff --git a/git-compat-util.h b/git-compat-util.h
index cf733b38acd..9273a8ee087 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -1000,6 +1000,9 @@ int git_vsnprintf(char *str, size_t maxsize,
 int git_open_with_retry(const char *path, int flag, ...);
 #endif
 
+int git_open_cloexec(const char *name, int flags);
+#define git_open(name) git_open_cloexec(name, O_RDONLY)
+
 #ifdef __GLIBC_PREREQ
 #if __GLIBC_PREREQ(2, 1)
 #define HAVE_STRCHRNUL
diff --git a/meson.build b/meson.build
index 145d2f7ff9e..a55e800b85b 100644
--- a/meson.build
+++ b/meson.build
@@ -263,6 +263,7 @@ libgit_sources = [
   'common-init.c',
   'compat/nonblock.c',
   'compat/obstack.c',
+  'compat/open.c',
   'compat/terminal.c',
   'compiler-tricks/not-constant.c',
   'config.c',
diff --git a/midx.c b/midx.c
index 807fdf72f7b..3d0015f7828 100644
--- a/midx.c
+++ b/midx.c
@@ -5,7 +5,6 @@
 #include "dir.h"
 #include "hex.h"
 #include "packfile.h"
-#include "object-file.h"
 #include "hash-lookup.h"
 #include "midx.h"
 #include "progress.h"
diff --git a/object-file.c b/object-file.c
index 6228e1c40f8..c3e20417f3f 100644
--- a/object-file.c
+++ b/object-file.c
@@ -833,33 +833,6 @@ int stream_object_signature(struct repository *r, const struct object_id *oid)
 	return !oideq(oid, &real_oid) ? -1 : 0;
 }
 
-int git_open_cloexec(const char *name, int flags)
-{
-	int fd;
-	static int o_cloexec = O_CLOEXEC;
-
-	fd = open(name, flags | o_cloexec);
-	if ((o_cloexec & O_CLOEXEC) && fd < 0 && errno == EINVAL) {
-		/* Try again w/o O_CLOEXEC: the kernel might not support it */
-		o_cloexec &= ~O_CLOEXEC;
-		fd = open(name, flags | o_cloexec);
-	}
-
-#if defined(F_GETFD) && defined(F_SETFD) && defined(FD_CLOEXEC)
-	{
-		static int fd_cloexec = FD_CLOEXEC;
-
-		if (!o_cloexec && 0 <= fd && fd_cloexec) {
-			/* Opened w/o O_CLOEXEC?  try with fcntl(2) to add it */
-			int flags = fcntl(fd, F_GETFD);
-			if (fcntl(fd, F_SETFD, flags | fd_cloexec))
-				fd_cloexec = 0;
-		}
-	}
-#endif
-	return fd;
-}
-
 /*
  * Find "oid" as a loose object in the local repository or in an alternate.
  * Returns 0 on success, negative on failure.
diff --git a/object-file.h b/object-file.h
index 922f2bba8c9..353d8a85c33 100644
--- a/object-file.h
+++ b/object-file.h
@@ -21,9 +21,6 @@ extern int fetch_if_missing;
 int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);
 int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags);
 
-int git_open_cloexec(const char *name, int flags);
-#define git_open(name) git_open_cloexec(name, O_RDONLY)
-
 /**
  * unpack_loose_header() initializes the data stream needed to unpack
  * a loose object header.
diff --git a/pack-bitmap.c b/pack-bitmap.c
index 7fd78c634ef..0dbd7c4ffe1 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -17,7 +17,6 @@
 #include "packfile.h"
 #include "repository.h"
 #include "trace2.h"
-#include "object-file.h"
 #include "object-store-ll.h"
 #include "list-objects-filter-options.h"
 #include "midx.h"
diff --git a/pack-mtimes.c b/pack-mtimes.c
index cdf30b8d2b0..bcea28e521d 100644
--- a/pack-mtimes.c
+++ b/pack-mtimes.c
@@ -1,7 +1,6 @@
 #include "git-compat-util.h"
 #include "gettext.h"
 #include "pack-mtimes.h"
-#include "object-file.h"
 #include "object-store-ll.h"
 #include "packfile.h"
 #include "strbuf.h"
diff --git a/pack-revindex.c b/pack-revindex.c
index 038e0c96b1c..1ee7b49e206 100644
--- a/pack-revindex.c
+++ b/pack-revindex.c
@@ -1,7 +1,6 @@
 #include "git-compat-util.h"
 #include "gettext.h"
 #include "pack-revindex.h"
-#include "object-file.h"
 #include "object-store-ll.h"
 #include "packfile.h"
 #include "strbuf.h"

-- 
2.49.0.805.g082f7c87e0.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH v3 04/10] object-file: move `xmmap()` into "wrapper.c"
  2025-04-15  9:38 ` [PATCH v3 00/10] Split up "object-file.c" Patrick Steinhardt
                     ` (2 preceding siblings ...)
  2025-04-15  9:38   ` [PATCH v3 03/10] object-file: move `git_open_cloexec()` to "compat/open.c" Patrick Steinhardt
@ 2025-04-15  9:38   ` Patrick Steinhardt
  2025-04-15  9:38   ` [PATCH v3 05/10] object-file: split out functions relating to object store subsystem Patrick Steinhardt
                     ` (6 subsequent siblings)
  10 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-15  9:38 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Eric Sunshine, Jeff King, Junio C Hamano

The `xmmap()` function is provided by "object-file.c" even though its
functionality has nothing to do with the object file subsystem. Move it
into "wrapper.c", whose header already declares those functions.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 object-file.c | 48 ------------------------------------------------
 wrapper.c     | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 48 insertions(+), 48 deletions(-)

diff --git a/object-file.c b/object-file.c
index c3e20417f3f..a7868201d09 100644
--- a/object-file.c
+++ b/object-file.c
@@ -718,54 +718,6 @@ int has_loose_object(const struct object_id *oid)
 	return check_and_freshen(oid, 0);
 }
 
-static void mmap_limit_check(size_t length)
-{
-	static size_t limit = 0;
-	if (!limit) {
-		limit = git_env_ulong("GIT_MMAP_LIMIT", 0);
-		if (!limit)
-			limit = SIZE_MAX;
-	}
-	if (length > limit)
-		die(_("attempting to mmap %"PRIuMAX" over limit %"PRIuMAX),
-		    (uintmax_t)length, (uintmax_t)limit);
-}
-
-void *xmmap_gently(void *start, size_t length,
-		  int prot, int flags, int fd, off_t offset)
-{
-	void *ret;
-
-	mmap_limit_check(length);
-	ret = mmap(start, length, prot, flags, fd, offset);
-	if (ret == MAP_FAILED && !length)
-		ret = NULL;
-	return ret;
-}
-
-const char *mmap_os_err(void)
-{
-	static const char blank[] = "";
-#if defined(__linux__)
-	if (errno == ENOMEM) {
-		/* this continues an existing error message: */
-		static const char enomem[] =
-", check sys.vm.max_map_count and/or RLIMIT_DATA";
-		return enomem;
-	}
-#endif /* OS-specific bits */
-	return blank;
-}
-
-void *xmmap(void *start, size_t length,
-	int prot, int flags, int fd, off_t offset)
-{
-	void *ret = xmmap_gently(start, length, prot, flags, fd, offset);
-	if (ret == MAP_FAILED)
-		die_errno(_("mmap failed%s"), mmap_os_err());
-	return ret;
-}
-
 static int format_object_header_literally(char *str, size_t size,
 					  const char *type, size_t objsize)
 {
diff --git a/wrapper.c b/wrapper.c
index 8b985931490..3c79778055e 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -829,3 +829,51 @@ uint32_t git_rand(unsigned flags)
 
 	return result;
 }
+
+static void mmap_limit_check(size_t length)
+{
+	static size_t limit = 0;
+	if (!limit) {
+		limit = git_env_ulong("GIT_MMAP_LIMIT", 0);
+		if (!limit)
+			limit = SIZE_MAX;
+	}
+	if (length > limit)
+		die(_("attempting to mmap %"PRIuMAX" over limit %"PRIuMAX),
+		    (uintmax_t)length, (uintmax_t)limit);
+}
+
+void *xmmap_gently(void *start, size_t length,
+		  int prot, int flags, int fd, off_t offset)
+{
+	void *ret;
+
+	mmap_limit_check(length);
+	ret = mmap(start, length, prot, flags, fd, offset);
+	if (ret == MAP_FAILED && !length)
+		ret = NULL;
+	return ret;
+}
+
+const char *mmap_os_err(void)
+{
+	static const char blank[] = "";
+#if defined(__linux__)
+	if (errno == ENOMEM) {
+		/* this continues an existing error message: */
+		static const char enomem[] =
+", check sys.vm.max_map_count and/or RLIMIT_DATA";
+		return enomem;
+	}
+#endif /* OS-specific bits */
+	return blank;
+}
+
+void *xmmap(void *start, size_t length,
+	int prot, int flags, int fd, off_t offset)
+{
+	void *ret = xmmap_gently(start, length, prot, flags, fd, offset);
+	if (ret == MAP_FAILED)
+		die_errno(_("mmap failed%s"), mmap_os_err());
+	return ret;
+}

-- 
2.49.0.805.g082f7c87e0.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH v3 05/10] object-file: split out functions relating to object store subsystem
  2025-04-15  9:38 ` [PATCH v3 00/10] Split up "object-file.c" Patrick Steinhardt
                     ` (3 preceding siblings ...)
  2025-04-15  9:38   ` [PATCH v3 04/10] object-file: move `xmmap()` into "wrapper.c" Patrick Steinhardt
@ 2025-04-15  9:38   ` Patrick Steinhardt
  2025-04-15  9:38   ` [PATCH v3 06/10] object-file: split up concerns of `HASH_*` flags Patrick Steinhardt
                     ` (5 subsequent siblings)
  10 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-15  9:38 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Eric Sunshine, Jeff King, Junio C Hamano

While we have the "object-store.h" header, most of the functionality for
object stores is actually hosted in "object-file.c". This makes it hard
to find relevant functions and causes us to mix up concerns.

Split out functions relating to the object store subsystem into a new
"object-store.c" file.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 Makefile                 |   1 +
 builtin/checkout.c       |   1 +
 builtin/merge-file.c     |   1 +
 builtin/mktree.c         |   1 +
 builtin/notes.c          |   1 +
 builtin/receive-pack.c   |   1 +
 builtin/tag.c            |   1 +
 builtin/unpack-file.c    |   1 +
 builtin/unpack-objects.c |   1 +
 commit.c                 |   1 +
 http-push.c              |   1 +
 match-trees.c            |   3 +-
 merge-ort.c              |   3 +-
 meson.build              |   1 +
 notes-cache.c            |   3 +-
 notes.c                  |   3 +-
 object-file.c            | 990 +----------------------------------------------
 object-file.h            |  68 +++-
 object-store-ll.h        |  60 +--
 object-store.c           | 972 ++++++++++++++++++++++++++++++++++++++++++++++
 20 files changed, 1074 insertions(+), 1040 deletions(-)

diff --git a/Makefile b/Makefile
index bb5407b4703..d8ad76fb791 100644
--- a/Makefile
+++ b/Makefile
@@ -1086,6 +1086,7 @@ LIB_OBJS += notes.o
 LIB_OBJS += object-file-convert.o
 LIB_OBJS += object-file.o
 LIB_OBJS += object-name.o
+LIB_OBJS += object-store.o
 LIB_OBJS += object.o
 LIB_OBJS += oid-array.o
 LIB_OBJS += oidmap.o
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 2e7486cf658..3e68623838a 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -18,6 +18,7 @@
 #include "lockfile.h"
 #include "mem-pool.h"
 #include "merge-recursive.h"
+#include "object-file.h"
 #include "object-name.h"
 #include "object-store-ll.h"
 #include "parse-options.h"
diff --git a/builtin/merge-file.c b/builtin/merge-file.c
index 7e315f374b2..2b16b10d2ca 100644
--- a/builtin/merge-file.c
+++ b/builtin/merge-file.c
@@ -5,6 +5,7 @@
 #include "abspath.h"
 #include "diff.h"
 #include "hex.h"
+#include "object-file.h"
 #include "object-name.h"
 #include "object-store.h"
 #include "config.h"
diff --git a/builtin/mktree.c b/builtin/mktree.c
index 3c16faa40e3..0644f951161 100644
--- a/builtin/mktree.c
+++ b/builtin/mktree.c
@@ -11,6 +11,7 @@
 #include "strbuf.h"
 #include "tree.h"
 #include "parse-options.h"
+#include "object-file.h"
 #include "object-store-ll.h"
 
 static struct treeent {
diff --git a/builtin/notes.c b/builtin/notes.c
index ff61ec5f2da..0dbc233752d 100644
--- a/builtin/notes.c
+++ b/builtin/notes.c
@@ -14,6 +14,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "notes.h"
+#include "object-file.h"
 #include "object-name.h"
 #include "object-store-ll.h"
 #include "path.h"
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index b3e2a9d0c60..ee51bd76f60 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -31,6 +31,7 @@
 #include "tmp-objdir.h"
 #include "oidset.h"
 #include "packfile.h"
+#include "object-file.h"
 #include "object-name.h"
 #include "object-store-ll.h"
 #include "path.h"
diff --git a/builtin/tag.c b/builtin/tag.c
index 7c173535cb3..536a01ff3ae 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -17,6 +17,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "refs.h"
+#include "object-file.h"
 #include "object-name.h"
 #include "object-store-ll.h"
 #include "path.h"
diff --git a/builtin/unpack-file.c b/builtin/unpack-file.c
index fb5fcbc40a8..b19e5cabd03 100644
--- a/builtin/unpack-file.c
+++ b/builtin/unpack-file.c
@@ -2,6 +2,7 @@
 #include "builtin.h"
 #include "config.h"
 #include "hex.h"
+#include "object-file.h"
 #include "object-name.h"
 #include "object-store-ll.h"
 
diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c
index 3bbcaf2de9b..4078eab9252 100644
--- a/builtin/unpack-objects.c
+++ b/builtin/unpack-objects.c
@@ -8,6 +8,7 @@
 #include "gettext.h"
 #include "git-zlib.h"
 #include "hex.h"
+#include "object-file.h"
 #include "object-store-ll.h"
 #include "object.h"
 #include "delta.h"
diff --git a/commit.c b/commit.c
index 48aeefaad31..fbf4f8e87fd 100644
--- a/commit.c
+++ b/commit.c
@@ -29,6 +29,7 @@
 #include "tree.h"
 #include "hook.h"
 #include "parse.h"
+#include "object-file.h"
 #include "object-file-convert.h"
 
 static struct commit_extra_header *read_commit_extra_header_lines(const char *buf, size_t len, const char **);
diff --git a/http-push.c b/http-push.c
index 1b030d96f48..806eb67cf1b 100644
--- a/http-push.c
+++ b/http-push.c
@@ -19,6 +19,7 @@
 #include "tree-walk.h"
 #include "url.h"
 #include "packfile.h"
+#include "object-file.h"
 #include "object-store-ll.h"
 #include "commit-reach.h"
 
diff --git a/match-trees.c b/match-trees.c
index ef14ceb594c..72922d5d64e 100644
--- a/match-trees.c
+++ b/match-trees.c
@@ -6,7 +6,8 @@
 #include "strbuf.h"
 #include "tree.h"
 #include "tree-walk.h"
-#include "object-store-ll.h"
+#include "object-file.h"
+#include "object-store.h"
 #include "repository.h"
 
 static int score_missing(unsigned mode)
diff --git a/merge-ort.c b/merge-ort.c
index 1554900d984..5de3e2fc159 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -36,8 +36,9 @@
 #include "merge-ll.h"
 #include "match-trees.h"
 #include "mem-pool.h"
+#include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "path.h"
 #include "promisor-remote.h"
diff --git a/meson.build b/meson.build
index a55e800b85b..f656f978020 100644
--- a/meson.build
+++ b/meson.build
@@ -355,6 +355,7 @@ libgit_sources = [
   'object-file-convert.c',
   'object-file.c',
   'object-name.c',
+  'object-store.c',
   'object.c',
   'oid-array.c',
   'oidmap.c',
diff --git a/notes-cache.c b/notes-cache.c
index ecfdf6e43b5..150241b15e0 100644
--- a/notes-cache.c
+++ b/notes-cache.c
@@ -2,7 +2,8 @@
 
 #include "git-compat-util.h"
 #include "notes-cache.h"
-#include "object-store-ll.h"
+#include "object-file.h"
+#include "object-store.h"
 #include "pretty.h"
 #include "repository.h"
 #include "commit.h"
diff --git a/notes.c b/notes.c
index ce5a1006a83..d9645c4b5dc 100644
--- a/notes.c
+++ b/notes.c
@@ -6,8 +6,9 @@
 #include "environment.h"
 #include "hex.h"
 #include "notes.h"
+#include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "utf8.h"
 #include "strbuf.h"
 #include "tree-walk.h"
diff --git a/object-file.c b/object-file.c
index a7868201d09..baa828822ea 100644
--- a/object-file.c
+++ b/object-file.c
@@ -11,75 +11,26 @@
 #define DISABLE_SIGN_COMPARE_WARNINGS
 
 #include "git-compat-util.h"
-#include "abspath.h"
-#include "config.h"
+#include "bulk-checkin.h"
 #include "convert.h"
 #include "environment.h"
+#include "fsck.h"
 #include "gettext.h"
 #include "hex.h"
-#include "string-list.h"
-#include "lockfile.h"
-#include "pack.h"
-#include "commit.h"
-#include "run-command.h"
-#include "refs.h"
-#include "bulk-checkin.h"
-#include "repository.h"
-#include "replace-object.h"
-#include "streaming.h"
-#include "dir.h"
-#include "list.h"
-#include "quote.h"
-#include "packfile.h"
+#include "loose.h"
+#include "object-file-convert.h"
 #include "object-file.h"
 #include "object-store.h"
 #include "oidtree.h"
+#include "pack.h"
+#include "packfile.h"
 #include "path.h"
-#include "promisor-remote.h"
 #include "setup.h"
-#include "submodule.h"
-#include "fsck.h"
-#include "loose.h"
-#include "object-file-convert.h"
+#include "streaming.h"
 
 /* The maximum size for an object header. */
 #define MAX_HEADER_LEN 32
 
-/*
- * This is meant to hold a *small* number of objects that you would
- * want repo_read_object_file() to be able to return, but yet you do not want
- * to write them into the object store (e.g. a browse-only
- * application).
- */
-static struct cached_object_entry {
-	struct object_id oid;
-	struct cached_object {
-		enum object_type type;
-		const void *buf;
-		unsigned long size;
-	} value;
-} *cached_objects;
-static int cached_object_nr, cached_object_alloc;
-
-static const struct cached_object *find_cached_object(const struct object_id *oid)
-{
-	static const struct cached_object empty_tree = {
-		.type = OBJ_TREE,
-		.buf = "",
-	};
-	int i;
-	const struct cached_object_entry *co = cached_objects;
-
-	for (i = 0; i < cached_object_nr; i++, co++) {
-		if (oideq(&co->oid, oid))
-			return &co->value;
-	}
-	if (oideq(oid, the_hash_algo->empty_tree))
-		return &empty_tree;
-	return NULL;
-}
-
-
 static int get_conv_flags(unsigned flags)
 {
 	if (flags & HASH_RENORMALIZE)
@@ -90,39 +41,6 @@ static int get_conv_flags(unsigned flags)
 		return 0;
 }
 
-int odb_mkstemp(struct strbuf *temp_filename, const char *pattern)
-{
-	int fd;
-	/*
-	 * we let the umask do its job, don't try to be more
-	 * restrictive except to remove write permission.
-	 */
-	int mode = 0444;
-	repo_git_path_replace(the_repository, temp_filename, "objects/%s", pattern);
-	fd = git_mkstemp_mode(temp_filename->buf, mode);
-	if (0 <= fd)
-		return fd;
-
-	/* slow path */
-	/* some mkstemp implementations erase temp_filename on failure */
-	repo_git_path_replace(the_repository, temp_filename, "objects/%s", pattern);
-	safe_create_leading_directories(the_repository, temp_filename->buf);
-	return xmkstemp_mode(temp_filename->buf, mode);
-}
-
-int odb_pack_keep(const char *name)
-{
-	int fd;
-
-	fd = open(name, O_RDWR|O_CREAT|O_EXCL, 0600);
-	if (0 <= fd)
-		return fd;
-
-	/* slow path */
-	safe_create_leading_directories_const(the_repository, name);
-	return open(name, O_RDWR|O_CREAT|O_EXCL, 0600);
-}
-
 static void fill_loose_path(struct strbuf *buf, const struct object_id *oid)
 {
 	int i;
@@ -136,9 +54,9 @@ static void fill_loose_path(struct strbuf *buf, const struct object_id *oid)
 	}
 }
 
-static const char *odb_loose_path(struct object_directory *odb,
-				  struct strbuf *buf,
-				  const struct object_id *oid)
+const char *odb_loose_path(struct object_directory *odb,
+			   struct strbuf *buf,
+			   const struct object_id *oid)
 {
 	strbuf_reset(buf);
 	strbuf_addstr(buf, odb->path);
@@ -147,513 +65,6 @@ static const char *odb_loose_path(struct object_directory *odb,
 	return buf->buf;
 }
 
-const char *loose_object_path(struct repository *r, struct strbuf *buf,
-			      const struct object_id *oid)
-{
-	return odb_loose_path(r->objects->odb, buf, oid);
-}
-
-/*
- * Return non-zero iff the path is usable as an alternate object database.
- */
-static int alt_odb_usable(struct raw_object_store *o,
-			  struct strbuf *path,
-			  const char *normalized_objdir, khiter_t *pos)
-{
-	int r;
-
-	/* Detect cases where alternate disappeared */
-	if (!is_directory(path->buf)) {
-		error(_("object directory %s does not exist; "
-			"check .git/objects/info/alternates"),
-		      path->buf);
-		return 0;
-	}
-
-	/*
-	 * Prevent the common mistake of listing the same
-	 * thing twice, or object directory itself.
-	 */
-	if (!o->odb_by_path) {
-		khiter_t p;
-
-		o->odb_by_path = kh_init_odb_path_map();
-		assert(!o->odb->next);
-		p = kh_put_odb_path_map(o->odb_by_path, o->odb->path, &r);
-		assert(r == 1); /* never used */
-		kh_value(o->odb_by_path, p) = o->odb;
-	}
-	if (fspatheq(path->buf, normalized_objdir))
-		return 0;
-	*pos = kh_put_odb_path_map(o->odb_by_path, path->buf, &r);
-	/* r: 0 = exists, 1 = never used, 2 = deleted */
-	return r == 0 ? 0 : 1;
-}
-
-/*
- * Prepare alternate object database registry.
- *
- * The variable alt_odb_list points at the list of struct
- * object_directory.  The elements on this list come from
- * non-empty elements from colon separated ALTERNATE_DB_ENVIRONMENT
- * environment variable, and $GIT_OBJECT_DIRECTORY/info/alternates,
- * whose contents is similar to that environment variable but can be
- * LF separated.  Its base points at a statically allocated buffer that
- * contains "/the/directory/corresponding/to/.git/objects/...", while
- * its name points just after the slash at the end of ".git/objects/"
- * in the example above, and has enough space to hold all hex characters
- * of the object ID, an extra slash for the first level indirection, and
- * the terminating NUL.
- */
-static void read_info_alternates(struct repository *r,
-				 const char *relative_base,
-				 int depth);
-static int link_alt_odb_entry(struct repository *r, const struct strbuf *entry,
-	const char *relative_base, int depth, const char *normalized_objdir)
-{
-	struct object_directory *ent;
-	struct strbuf pathbuf = STRBUF_INIT;
-	struct strbuf tmp = STRBUF_INIT;
-	khiter_t pos;
-	int ret = -1;
-
-	if (!is_absolute_path(entry->buf) && relative_base) {
-		strbuf_realpath(&pathbuf, relative_base, 1);
-		strbuf_addch(&pathbuf, '/');
-	}
-	strbuf_addbuf(&pathbuf, entry);
-
-	if (!strbuf_realpath(&tmp, pathbuf.buf, 0)) {
-		error(_("unable to normalize alternate object path: %s"),
-		      pathbuf.buf);
-		goto error;
-	}
-	strbuf_swap(&pathbuf, &tmp);
-
-	/*
-	 * The trailing slash after the directory name is given by
-	 * this function at the end. Remove duplicates.
-	 */
-	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
-		strbuf_setlen(&pathbuf, pathbuf.len - 1);
-
-	if (!alt_odb_usable(r->objects, &pathbuf, normalized_objdir, &pos))
-		goto error;
-
-	CALLOC_ARRAY(ent, 1);
-	/* pathbuf.buf is already in r->objects->odb_by_path */
-	ent->path = strbuf_detach(&pathbuf, NULL);
-
-	/* add the alternate entry */
-	*r->objects->odb_tail = ent;
-	r->objects->odb_tail = &(ent->next);
-	ent->next = NULL;
-	assert(r->objects->odb_by_path);
-	kh_value(r->objects->odb_by_path, pos) = ent;
-
-	/* recursively add alternates */
-	read_info_alternates(r, ent->path, depth + 1);
-	ret = 0;
- error:
-	strbuf_release(&tmp);
-	strbuf_release(&pathbuf);
-	return ret;
-}
-
-static const char *parse_alt_odb_entry(const char *string,
-				       int sep,
-				       struct strbuf *out)
-{
-	const char *end;
-
-	strbuf_reset(out);
-
-	if (*string == '#') {
-		/* comment; consume up to next separator */
-		end = strchrnul(string, sep);
-	} else if (*string == '"' && !unquote_c_style(out, string, &end)) {
-		/*
-		 * quoted path; unquote_c_style has copied the
-		 * data for us and set "end". Broken quoting (e.g.,
-		 * an entry that doesn't end with a quote) falls
-		 * back to the unquoted case below.
-		 */
-	} else {
-		/* normal, unquoted path */
-		end = strchrnul(string, sep);
-		strbuf_add(out, string, end - string);
-	}
-
-	if (*end)
-		end++;
-	return end;
-}
-
-static void link_alt_odb_entries(struct repository *r, const char *alt,
-				 int sep, const char *relative_base, int depth)
-{
-	struct strbuf objdirbuf = STRBUF_INIT;
-	struct strbuf entry = STRBUF_INIT;
-
-	if (!alt || !*alt)
-		return;
-
-	if (depth > 5) {
-		error(_("%s: ignoring alternate object stores, nesting too deep"),
-				relative_base);
-		return;
-	}
-
-	strbuf_realpath(&objdirbuf, r->objects->odb->path, 1);
-
-	while (*alt) {
-		alt = parse_alt_odb_entry(alt, sep, &entry);
-		if (!entry.len)
-			continue;
-		link_alt_odb_entry(r, &entry,
-				   relative_base, depth, objdirbuf.buf);
-	}
-	strbuf_release(&entry);
-	strbuf_release(&objdirbuf);
-}
-
-static void read_info_alternates(struct repository *r,
-				 const char *relative_base,
-				 int depth)
-{
-	char *path;
-	struct strbuf buf = STRBUF_INIT;
-
-	path = xstrfmt("%s/info/alternates", relative_base);
-	if (strbuf_read_file(&buf, path, 1024) < 0) {
-		warn_on_fopen_errors(path);
-		free(path);
-		return;
-	}
-
-	link_alt_odb_entries(r, buf.buf, '\n', relative_base, depth);
-	strbuf_release(&buf);
-	free(path);
-}
-
-void add_to_alternates_file(const char *reference)
-{
-	struct lock_file lock = LOCK_INIT;
-	char *alts = repo_git_path(the_repository, "objects/info/alternates");
-	FILE *in, *out;
-	int found = 0;
-
-	hold_lock_file_for_update(&lock, alts, LOCK_DIE_ON_ERROR);
-	out = fdopen_lock_file(&lock, "w");
-	if (!out)
-		die_errno(_("unable to fdopen alternates lockfile"));
-
-	in = fopen(alts, "r");
-	if (in) {
-		struct strbuf line = STRBUF_INIT;
-
-		while (strbuf_getline(&line, in) != EOF) {
-			if (!strcmp(reference, line.buf)) {
-				found = 1;
-				break;
-			}
-			fprintf_or_die(out, "%s\n", line.buf);
-		}
-
-		strbuf_release(&line);
-		fclose(in);
-	}
-	else if (errno != ENOENT)
-		die_errno(_("unable to read alternates file"));
-
-	if (found) {
-		rollback_lock_file(&lock);
-	} else {
-		fprintf_or_die(out, "%s\n", reference);
-		if (commit_lock_file(&lock))
-			die_errno(_("unable to move new alternates file into place"));
-		if (the_repository->objects->loaded_alternates)
-			link_alt_odb_entries(the_repository, reference,
-					     '\n', NULL, 0);
-	}
-	free(alts);
-}
-
-void add_to_alternates_memory(const char *reference)
-{
-	/*
-	 * Make sure alternates are initialized, or else our entry may be
-	 * overwritten when they are.
-	 */
-	prepare_alt_odb(the_repository);
-
-	link_alt_odb_entries(the_repository, reference,
-			     '\n', NULL, 0);
-}
-
-struct object_directory *set_temporary_primary_odb(const char *dir, int will_destroy)
-{
-	struct object_directory *new_odb;
-
-	/*
-	 * Make sure alternates are initialized, or else our entry may be
-	 * overwritten when they are.
-	 */
-	prepare_alt_odb(the_repository);
-
-	/*
-	 * Make a new primary odb and link the old primary ODB in as an
-	 * alternate
-	 */
-	new_odb = xcalloc(1, sizeof(*new_odb));
-	new_odb->path = xstrdup(dir);
-
-	/*
-	 * Disable ref updates while a temporary odb is active, since
-	 * the objects in the database may roll back.
-	 */
-	new_odb->disable_ref_updates = 1;
-	new_odb->will_destroy = will_destroy;
-	new_odb->next = the_repository->objects->odb;
-	the_repository->objects->odb = new_odb;
-	return new_odb->next;
-}
-
-void restore_primary_odb(struct object_directory *restore_odb, const char *old_path)
-{
-	struct object_directory *cur_odb = the_repository->objects->odb;
-
-	if (strcmp(old_path, cur_odb->path))
-		BUG("expected %s as primary object store; found %s",
-		    old_path, cur_odb->path);
-
-	if (cur_odb->next != restore_odb)
-		BUG("we expect the old primary object store to be the first alternate");
-
-	the_repository->objects->odb = restore_odb;
-	free_object_directory(cur_odb);
-}
-
-/*
- * Compute the exact path an alternate is at and returns it. In case of
- * error NULL is returned and the human readable error is added to `err`
- * `path` may be relative and should point to $GIT_DIR.
- * `err` must not be null.
- */
-char *compute_alternate_path(const char *path, struct strbuf *err)
-{
-	char *ref_git = NULL;
-	const char *repo;
-	int seen_error = 0;
-
-	ref_git = real_pathdup(path, 0);
-	if (!ref_git) {
-		seen_error = 1;
-		strbuf_addf(err, _("path '%s' does not exist"), path);
-		goto out;
-	}
-
-	repo = read_gitfile(ref_git);
-	if (!repo)
-		repo = read_gitfile(mkpath("%s/.git", ref_git));
-	if (repo) {
-		free(ref_git);
-		ref_git = xstrdup(repo);
-	}
-
-	if (!repo && is_directory(mkpath("%s/.git/objects", ref_git))) {
-		char *ref_git_git = mkpathdup("%s/.git", ref_git);
-		free(ref_git);
-		ref_git = ref_git_git;
-	} else if (!is_directory(mkpath("%s/objects", ref_git))) {
-		struct strbuf sb = STRBUF_INIT;
-		seen_error = 1;
-		if (get_common_dir(&sb, ref_git)) {
-			strbuf_addf(err,
-				    _("reference repository '%s' as a linked "
-				      "checkout is not supported yet."),
-				    path);
-			goto out;
-		}
-
-		strbuf_addf(err, _("reference repository '%s' is not a "
-					"local repository."), path);
-		goto out;
-	}
-
-	if (!access(mkpath("%s/shallow", ref_git), F_OK)) {
-		strbuf_addf(err, _("reference repository '%s' is shallow"),
-			    path);
-		seen_error = 1;
-		goto out;
-	}
-
-	if (!access(mkpath("%s/info/grafts", ref_git), F_OK)) {
-		strbuf_addf(err,
-			    _("reference repository '%s' is grafted"),
-			    path);
-		seen_error = 1;
-		goto out;
-	}
-
-out:
-	if (seen_error) {
-		FREE_AND_NULL(ref_git);
-	}
-
-	return ref_git;
-}
-
-struct object_directory *find_odb(struct repository *r, const char *obj_dir)
-{
-	struct object_directory *odb;
-	char *obj_dir_real = real_pathdup(obj_dir, 1);
-	struct strbuf odb_path_real = STRBUF_INIT;
-
-	prepare_alt_odb(r);
-	for (odb = r->objects->odb; odb; odb = odb->next) {
-		strbuf_realpath(&odb_path_real, odb->path, 1);
-		if (!strcmp(obj_dir_real, odb_path_real.buf))
-			break;
-	}
-
-	free(obj_dir_real);
-	strbuf_release(&odb_path_real);
-
-	if (!odb)
-		die(_("could not find object directory matching %s"), obj_dir);
-	return odb;
-}
-
-static void fill_alternate_refs_command(struct child_process *cmd,
-					const char *repo_path)
-{
-	const char *value;
-
-	if (!git_config_get_value("core.alternateRefsCommand", &value)) {
-		cmd->use_shell = 1;
-
-		strvec_push(&cmd->args, value);
-		strvec_push(&cmd->args, repo_path);
-	} else {
-		cmd->git_cmd = 1;
-
-		strvec_pushf(&cmd->args, "--git-dir=%s", repo_path);
-		strvec_push(&cmd->args, "for-each-ref");
-		strvec_push(&cmd->args, "--format=%(objectname)");
-
-		if (!git_config_get_value("core.alternateRefsPrefixes", &value)) {
-			strvec_push(&cmd->args, "--");
-			strvec_split(&cmd->args, value);
-		}
-	}
-
-	strvec_pushv(&cmd->env, (const char **)local_repo_env);
-	cmd->out = -1;
-}
-
-static void read_alternate_refs(const char *path,
-				alternate_ref_fn *cb,
-				void *data)
-{
-	struct child_process cmd = CHILD_PROCESS_INIT;
-	struct strbuf line = STRBUF_INIT;
-	FILE *fh;
-
-	fill_alternate_refs_command(&cmd, path);
-
-	if (start_command(&cmd))
-		return;
-
-	fh = xfdopen(cmd.out, "r");
-	while (strbuf_getline_lf(&line, fh) != EOF) {
-		struct object_id oid;
-		const char *p;
-
-		if (parse_oid_hex(line.buf, &oid, &p) || *p) {
-			warning(_("invalid line while parsing alternate refs: %s"),
-				line.buf);
-			break;
-		}
-
-		cb(&oid, data);
-	}
-
-	fclose(fh);
-	finish_command(&cmd);
-	strbuf_release(&line);
-}
-
-struct alternate_refs_data {
-	alternate_ref_fn *fn;
-	void *data;
-};
-
-static int refs_from_alternate_cb(struct object_directory *e,
-				  void *data)
-{
-	struct strbuf path = STRBUF_INIT;
-	size_t base_len;
-	struct alternate_refs_data *cb = data;
-
-	if (!strbuf_realpath(&path, e->path, 0))
-		goto out;
-	if (!strbuf_strip_suffix(&path, "/objects"))
-		goto out;
-	base_len = path.len;
-
-	/* Is this a git repository with refs? */
-	strbuf_addstr(&path, "/refs");
-	if (!is_directory(path.buf))
-		goto out;
-	strbuf_setlen(&path, base_len);
-
-	read_alternate_refs(path.buf, cb->fn, cb->data);
-
-out:
-	strbuf_release(&path);
-	return 0;
-}
-
-void for_each_alternate_ref(alternate_ref_fn fn, void *data)
-{
-	struct alternate_refs_data cb;
-	cb.fn = fn;
-	cb.data = data;
-	foreach_alt_odb(refs_from_alternate_cb, &cb);
-}
-
-int foreach_alt_odb(alt_odb_fn fn, void *cb)
-{
-	struct object_directory *ent;
-	int r = 0;
-
-	prepare_alt_odb(the_repository);
-	for (ent = the_repository->objects->odb->next; ent; ent = ent->next) {
-		r = fn(ent, cb);
-		if (r)
-			break;
-	}
-	return r;
-}
-
-void prepare_alt_odb(struct repository *r)
-{
-	if (r->objects->loaded_alternates)
-		return;
-
-	link_alt_odb_entries(r, r->objects->alternate_db, PATH_SEP, NULL, 0);
-
-	read_info_alternates(r, r->objects->odb->path, 0);
-	r->objects->loaded_alternates = 1;
-}
-
-int has_alt_odb(struct repository *r)
-{
-	prepare_alt_odb(r);
-	return !!r->objects->odb->next;
-}
-
 /* Returns 1 if we have successfully freshened the file, 0 otherwise. */
 static int freshen_file(const char *fn)
 {
@@ -1055,9 +466,9 @@ int parse_loose_header(const char *hdr, struct object_info *oi)
 	return 0;
 }
 
-static int loose_object_info(struct repository *r,
-			     const struct object_id *oid,
-			     struct object_info *oi, int flags)
+int loose_object_info(struct repository *r,
+		      const struct object_id *oid,
+		      struct object_info *oi, int flags)
 {
 	int status = 0;
 	int fd;
@@ -1153,345 +564,6 @@ static int loose_object_info(struct repository *r,
 	return status;
 }
 
-int obj_read_use_lock = 0;
-pthread_mutex_t obj_read_mutex;
-
-void enable_obj_read_lock(void)
-{
-	if (obj_read_use_lock)
-		return;
-
-	obj_read_use_lock = 1;
-	init_recursive_mutex(&obj_read_mutex);
-}
-
-void disable_obj_read_lock(void)
-{
-	if (!obj_read_use_lock)
-		return;
-
-	obj_read_use_lock = 0;
-	pthread_mutex_destroy(&obj_read_mutex);
-}
-
-int fetch_if_missing = 1;
-
-static int do_oid_object_info_extended(struct repository *r,
-				       const struct object_id *oid,
-				       struct object_info *oi, unsigned flags)
-{
-	static struct object_info blank_oi = OBJECT_INFO_INIT;
-	const struct cached_object *co;
-	struct pack_entry e;
-	int rtype;
-	const struct object_id *real = oid;
-	int already_retried = 0;
-
-
-	if (flags & OBJECT_INFO_LOOKUP_REPLACE)
-		real = lookup_replace_object(r, oid);
-
-	if (is_null_oid(real))
-		return -1;
-
-	if (!oi)
-		oi = &blank_oi;
-
-	co = find_cached_object(real);
-	if (co) {
-		if (oi->typep)
-			*(oi->typep) = co->type;
-		if (oi->sizep)
-			*(oi->sizep) = co->size;
-		if (oi->disk_sizep)
-			*(oi->disk_sizep) = 0;
-		if (oi->delta_base_oid)
-			oidclr(oi->delta_base_oid, the_repository->hash_algo);
-		if (oi->type_name)
-			strbuf_addstr(oi->type_name, type_name(co->type));
-		if (oi->contentp)
-			*oi->contentp = xmemdupz(co->buf, co->size);
-		oi->whence = OI_CACHED;
-		return 0;
-	}
-
-	while (1) {
-		if (find_pack_entry(r, real, &e))
-			break;
-
-		/* Most likely it's a loose object. */
-		if (!loose_object_info(r, real, oi, flags))
-			return 0;
-
-		/* Not a loose object; someone else may have just packed it. */
-		if (!(flags & OBJECT_INFO_QUICK)) {
-			reprepare_packed_git(r);
-			if (find_pack_entry(r, real, &e))
-				break;
-		}
-
-		/*
-		 * If r is the_repository, this might be an attempt at
-		 * accessing a submodule object as if it were in the_repository
-		 * (having called add_submodule_odb() on that submodule's ODB).
-		 * If any such ODBs exist, register them and try again.
-		 */
-		if (r == the_repository &&
-		    register_all_submodule_odb_as_alternates())
-			/* We added some alternates; retry */
-			continue;
-
-		/* Check if it is a missing object */
-		if (fetch_if_missing && repo_has_promisor_remote(r) &&
-		    !already_retried &&
-		    !(flags & OBJECT_INFO_SKIP_FETCH_OBJECT)) {
-			promisor_remote_get_direct(r, real, 1);
-			already_retried = 1;
-			continue;
-		}
-
-		if (flags & OBJECT_INFO_DIE_IF_CORRUPT) {
-			const struct packed_git *p;
-			if ((flags & OBJECT_INFO_LOOKUP_REPLACE) && !oideq(real, oid))
-				die(_("replacement %s not found for %s"),
-				    oid_to_hex(real), oid_to_hex(oid));
-			if ((p = has_packed_and_bad(r, real)))
-				die(_("packed object %s (stored in %s) is corrupt"),
-				    oid_to_hex(real), p->pack_name);
-		}
-		return -1;
-	}
-
-	if (oi == &blank_oi)
-		/*
-		 * We know that the caller doesn't actually need the
-		 * information below, so return early.
-		 */
-		return 0;
-	rtype = packed_object_info(r, e.p, e.offset, oi);
-	if (rtype < 0) {
-		mark_bad_packed_object(e.p, real);
-		return do_oid_object_info_extended(r, real, oi, 0);
-	} else if (oi->whence == OI_PACKED) {
-		oi->u.packed.offset = e.offset;
-		oi->u.packed.pack = e.p;
-		oi->u.packed.is_delta = (rtype == OBJ_REF_DELTA ||
-					 rtype == OBJ_OFS_DELTA);
-	}
-
-	return 0;
-}
-
-static int oid_object_info_convert(struct repository *r,
-				   const struct object_id *input_oid,
-				   struct object_info *input_oi, unsigned flags)
-{
-	const struct git_hash_algo *input_algo = &hash_algos[input_oid->algo];
-	int do_die = flags & OBJECT_INFO_DIE_IF_CORRUPT;
-	struct strbuf type_name = STRBUF_INIT;
-	struct object_id oid, delta_base_oid;
-	struct object_info new_oi, *oi;
-	unsigned long size;
-	void *content;
-	int ret;
-
-	if (repo_oid_to_algop(r, input_oid, the_hash_algo, &oid)) {
-		if (do_die)
-			die(_("missing mapping of %s to %s"),
-			    oid_to_hex(input_oid), the_hash_algo->name);
-		return -1;
-	}
-
-	/* Is new_oi needed? */
-	oi = input_oi;
-	if (input_oi && (input_oi->delta_base_oid || input_oi->sizep ||
-			 input_oi->contentp)) {
-		new_oi = *input_oi;
-		/* Does delta_base_oid need to be converted? */
-		if (input_oi->delta_base_oid)
-			new_oi.delta_base_oid = &delta_base_oid;
-		/* Will the attributes differ when converted? */
-		if (input_oi->sizep || input_oi->contentp) {
-			new_oi.contentp = &content;
-			new_oi.sizep = &size;
-			new_oi.type_name = &type_name;
-		}
-		oi = &new_oi;
-	}
-
-	ret = oid_object_info_extended(r, &oid, oi, flags);
-	if (ret)
-		return -1;
-	if (oi == input_oi)
-		return ret;
-
-	if (new_oi.contentp) {
-		struct strbuf outbuf = STRBUF_INIT;
-		enum object_type type;
-
-		type = type_from_string_gently(type_name.buf, type_name.len,
-					       !do_die);
-		if (type == -1)
-			return -1;
-		if (type != OBJ_BLOB) {
-			ret = convert_object_file(the_repository, &outbuf,
-						  the_hash_algo, input_algo,
-						  content, size, type, !do_die);
-			free(content);
-			if (ret == -1)
-				return -1;
-			size = outbuf.len;
-			content = strbuf_detach(&outbuf, NULL);
-		}
-		if (input_oi->sizep)
-			*input_oi->sizep = size;
-		if (input_oi->contentp)
-			*input_oi->contentp = content;
-		else
-			free(content);
-		if (input_oi->type_name)
-			*input_oi->type_name = type_name;
-		else
-			strbuf_release(&type_name);
-	}
-	if (new_oi.delta_base_oid == &delta_base_oid) {
-		if (repo_oid_to_algop(r, &delta_base_oid, input_algo,
-				 input_oi->delta_base_oid)) {
-			if (do_die)
-				die(_("missing mapping of %s to %s"),
-				    oid_to_hex(&delta_base_oid),
-				    input_algo->name);
-			return -1;
-		}
-	}
-	input_oi->whence = new_oi.whence;
-	input_oi->u = new_oi.u;
-	return ret;
-}
-
-int oid_object_info_extended(struct repository *r, const struct object_id *oid,
-			     struct object_info *oi, unsigned flags)
-{
-	int ret;
-
-	if (oid->algo && (hash_algo_by_ptr(r->hash_algo) != oid->algo))
-		return oid_object_info_convert(r, oid, oi, flags);
-
-	obj_read_lock();
-	ret = do_oid_object_info_extended(r, oid, oi, flags);
-	obj_read_unlock();
-	return ret;
-}
-
-
-/* returns enum object_type or negative */
-int oid_object_info(struct repository *r,
-		    const struct object_id *oid,
-		    unsigned long *sizep)
-{
-	enum object_type type;
-	struct object_info oi = OBJECT_INFO_INIT;
-
-	oi.typep = &type;
-	oi.sizep = sizep;
-	if (oid_object_info_extended(r, oid, &oi,
-				      OBJECT_INFO_LOOKUP_REPLACE) < 0)
-		return -1;
-	return type;
-}
-
-int pretend_object_file(void *buf, unsigned long len, enum object_type type,
-			struct object_id *oid)
-{
-	struct cached_object_entry *co;
-	char *co_buf;
-
-	hash_object_file(the_hash_algo, buf, len, type, oid);
-	if (repo_has_object_file_with_flags(the_repository, oid, OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT) ||
-	    find_cached_object(oid))
-		return 0;
-	ALLOC_GROW(cached_objects, cached_object_nr + 1, cached_object_alloc);
-	co = &cached_objects[cached_object_nr++];
-	co->value.size = len;
-	co->value.type = type;
-	co_buf = xmalloc(len);
-	memcpy(co_buf, buf, len);
-	co->value.buf = co_buf;
-	oidcpy(&co->oid, oid);
-	return 0;
-}
-
-/*
- * This function dies on corrupt objects; the callers who want to
- * deal with them should arrange to call oid_object_info_extended() and give
- * error messages themselves.
- */
-void *repo_read_object_file(struct repository *r,
-			    const struct object_id *oid,
-			    enum object_type *type,
-			    unsigned long *size)
-{
-	struct object_info oi = OBJECT_INFO_INIT;
-	unsigned flags = OBJECT_INFO_DIE_IF_CORRUPT | OBJECT_INFO_LOOKUP_REPLACE;
-	void *data;
-
-	oi.typep = type;
-	oi.sizep = size;
-	oi.contentp = &data;
-	if (oid_object_info_extended(r, oid, &oi, flags))
-		return NULL;
-
-	return data;
-}
-
-void *read_object_with_reference(struct repository *r,
-				 const struct object_id *oid,
-				 enum object_type required_type,
-				 unsigned long *size,
-				 struct object_id *actual_oid_return)
-{
-	enum object_type type;
-	void *buffer;
-	unsigned long isize;
-	struct object_id actual_oid;
-
-	oidcpy(&actual_oid, oid);
-	while (1) {
-		int ref_length = -1;
-		const char *ref_type = NULL;
-
-		buffer = repo_read_object_file(r, &actual_oid, &type, &isize);
-		if (!buffer)
-			return NULL;
-		if (type == required_type) {
-			*size = isize;
-			if (actual_oid_return)
-				oidcpy(actual_oid_return, &actual_oid);
-			return buffer;
-		}
-		/* Handle references */
-		else if (type == OBJ_COMMIT)
-			ref_type = "tree ";
-		else if (type == OBJ_TAG)
-			ref_type = "object ";
-		else {
-			free(buffer);
-			return NULL;
-		}
-		ref_length = strlen(ref_type);
-
-		if (ref_length + the_hash_algo->hexsz > isize ||
-		    memcmp(buffer, ref_type, ref_length) ||
-		    get_oid_hex((char *) buffer + ref_length, &actual_oid)) {
-			free(buffer);
-			return NULL;
-		}
-		free(buffer);
-		/* Now we have the ID of the referred-to object in
-		 * actual_oid.  Check again. */
-	}
-}
-
 static void hash_object_body(const struct git_hash_algo *algo, struct git_hash_ctx *c,
 			     const void *buf, unsigned long len,
 			     struct object_id *oid,
@@ -2154,32 +1226,6 @@ int force_object_loose(const struct object_id *oid, time_t mtime)
 	return ret;
 }
 
-int has_object(struct repository *r, const struct object_id *oid,
-	       unsigned flags)
-{
-	int quick = !(flags & HAS_OBJECT_RECHECK_PACKED);
-	unsigned object_info_flags = OBJECT_INFO_SKIP_FETCH_OBJECT |
-		(quick ? OBJECT_INFO_QUICK : 0);
-
-	if (!startup_info->have_repository)
-		return 0;
-	return oid_object_info_extended(r, oid, NULL, object_info_flags) >= 0;
-}
-
-int repo_has_object_file_with_flags(struct repository *r,
-				    const struct object_id *oid, int flags)
-{
-	if (!startup_info->have_repository)
-		return 0;
-	return oid_object_info_extended(r, oid, NULL, flags) >= 0;
-}
-
-int repo_has_object_file(struct repository *r,
-			 const struct object_id *oid)
-{
-	return repo_has_object_file_with_flags(r, oid, 0);
-}
-
 /*
  * We can't use the normal fsck_error_function() for index_mem(),
  * because we don't yet have a valid oid for it to report. Instead,
@@ -2407,16 +1453,6 @@ int read_pack_header(int fd, struct pack_header *header)
 	return 0;
 }
 
-void assert_oid_type(const struct object_id *oid, enum object_type expect)
-{
-	enum object_type type = oid_object_info(the_repository, oid, NULL);
-	if (type < 0)
-		die(_("%s is not a valid object"), oid_to_hex(oid));
-	if (type != expect)
-		die(_("%s is not a valid '%s' object"), oid_to_hex(oid),
-		    type_name(expect));
-}
-
 int for_each_file_in_obj_subdir(unsigned int subdir_nr,
 				struct strbuf *path,
 				each_loose_object_fn obj_cb,
diff --git a/object-file.h b/object-file.h
index 353d8a85c33..78c84d970a9 100644
--- a/object-file.h
+++ b/object-file.h
@@ -21,6 +21,29 @@ extern int fetch_if_missing;
 int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);
 int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags);
 
+struct object_directory;
+
+const char *odb_loose_path(struct object_directory *odb,
+			   struct strbuf *buf,
+			   const struct object_id *oid);
+
+/*
+ * Return true iff an alternate object database has a loose object
+ * with the specified name.  This function does not respect replace
+ * references.
+ */
+int has_loose_object_nonlocal(const struct object_id *);
+
+int has_loose_object(const struct object_id *);
+
+/**
+ * format_object_header() is a thin wrapper around s xsnprintf() that
+ * writes the initial "<type> <obj-len>" part of the loose object
+ * header. It returns the size that snprintf() returns + 1.
+ */
+int format_object_header(char *str, size_t size, enum object_type type,
+			 size_t objsize);
+
 /**
  * unpack_loose_header() initializes the data stream needed to unpack
  * a loose object header.
@@ -61,6 +84,29 @@ enum unpack_loose_header_result unpack_loose_header(git_zstream *stream,
 struct object_info;
 int parse_loose_header(const char *hdr, struct object_info *oi);
 
+int write_object_file_flags(const void *buf, unsigned long len,
+			    enum object_type type, struct object_id *oid,
+			    struct object_id *comapt_oid_in, unsigned flags);
+static inline int write_object_file(const void *buf, unsigned long len,
+				    enum object_type type, struct object_id *oid)
+{
+	return write_object_file_flags(buf, len, type, oid, NULL, 0);
+}
+
+struct input_stream {
+	const void *(*read)(struct input_stream *, unsigned long *len);
+	void *data;
+	int is_finished;
+};
+
+int write_object_file_literally(const void *buf, unsigned long len,
+				const char *type, struct object_id *oid,
+				unsigned flags);
+int stream_loose_object(struct input_stream *in_stream, size_t len,
+			struct object_id *oid);
+
+int force_object_loose(const struct object_id *oid, time_t mtime);
+
 /**
  * With in-core object data in "buf", rehash it to make sure the
  * object name actually matches "oid" to detect object corruption.
@@ -79,6 +125,10 @@ int check_object_signature(struct repository *r, const struct object_id *oid,
  */
 int stream_object_signature(struct repository *r, const struct object_id *oid);
 
+int loose_object_info(struct repository *r,
+		      const struct object_id *oid,
+		      struct object_info *oi, int flags);
+
 enum finalize_object_file_flags {
 	FOF_SKIP_COLLISION_CHECK = 1,
 };
@@ -90,10 +140,18 @@ int finalize_object_file_flags(const char *tmpfile, const char *filename,
 /* Helper to check and "touch" a file */
 int check_and_freshen_file(const char *fn, int freshen);
 
-void *read_object_with_reference(struct repository *r,
-				 const struct object_id *oid,
-				 enum object_type required_type,
-				 unsigned long *size,
-				 struct object_id *oid_ret);
+/*
+ * Open the loose object at path, check its hash, and return the contents,
+ * use the "oi" argument to assert things about the object, or e.g. populate its
+ * type, and size. If the object is a blob, then "contents" may return NULL,
+ * to allow streaming of large blobs.
+ *
+ * Returns 0 on success, negative on error (details may be written to stderr).
+ */
+int read_loose_object(const char *path,
+		      const struct object_id *expected_oid,
+		      struct object_id *real_oid,
+		      void **contents,
+		      struct object_info *oi);
 
 #endif /* OBJECT_FILE_H */
diff --git a/object-store-ll.h b/object-store-ll.h
index cd3bd5bd99f..8ae80b8a5fa 100644
--- a/object-store-ll.h
+++ b/object-store-ll.h
@@ -49,12 +49,6 @@ struct object_directory {
 	char *path;
 };
 
-struct input_stream {
-	const void *(*read)(struct input_stream *, unsigned long *len);
-	void *data;
-	int is_finished;
-};
-
 void prepare_alt_odb(struct repository *r);
 int has_alt_odb(struct repository *r);
 char *compute_alternate_path(const char *path, struct strbuf *err);
@@ -273,21 +267,6 @@ void hash_object_file(const struct git_hash_algo *algo, const void *buf,
 		      unsigned long len, enum object_type type,
 		      struct object_id *oid);
 
-int write_object_file_flags(const void *buf, unsigned long len,
-			    enum object_type type, struct object_id *oid,
-			    struct object_id *comapt_oid_in, unsigned flags);
-static inline int write_object_file(const void *buf, unsigned long len,
-				    enum object_type type, struct object_id *oid)
-{
-	return write_object_file_flags(buf, len, type, oid, NULL, 0);
-}
-
-int write_object_file_literally(const void *buf, unsigned long len,
-				const char *type, struct object_id *oid,
-				unsigned flags);
-int stream_loose_object(struct input_stream *in_stream, size_t len,
-			struct object_id *oid);
-
 /*
  * Add an object file to the in-memory object store, without writing it
  * to disk.
@@ -299,8 +278,6 @@ int stream_loose_object(struct input_stream *in_stream, size_t len,
 int pretend_object_file(void *, unsigned long, enum object_type,
 			struct object_id *oid);
 
-int force_object_loose(const struct object_id *oid, time_t mtime);
-
 struct object_info {
 	/* Request */
 	enum object_type *typep;
@@ -364,20 +341,6 @@ int oid_object_info_extended(struct repository *r,
 			     const struct object_id *,
 			     struct object_info *, unsigned flags);
 
-/*
- * Open the loose object at path, check its hash, and return the contents,
- * use the "oi" argument to assert things about the object, or e.g. populate its
- * type, and size. If the object is a blob, then "contents" may return NULL,
- * to allow streaming of large blobs.
- *
- * Returns 0 on success, negative on error (details may be written to stderr).
- */
-int read_loose_object(const char *path,
-		      const struct object_id *expected_oid,
-		      struct object_id *real_oid,
-		      void **contents,
-		      struct object_info *oi);
-
 /* Retry packed storage after checking packed and loose storage */
 #define HAS_OBJECT_RECHECK_PACKED 1
 
@@ -405,23 +368,6 @@ int repo_has_object_file(struct repository *r, const struct object_id *oid);
 int repo_has_object_file_with_flags(struct repository *r,
 				    const struct object_id *oid, int flags);
 
-/*
- * Return true iff an alternate object database has a loose object
- * with the specified name.  This function does not respect replace
- * references.
- */
-int has_loose_object_nonlocal(const struct object_id *);
-
-int has_loose_object(const struct object_id *);
-
-/**
- * format_object_header() is a thin wrapper around s xsnprintf() that
- * writes the initial "<type> <obj-len>" part of the loose object
- * header. It returns the size that snprintf() returns + 1.
- */
-int format_object_header(char *str, size_t size, enum object_type type,
-			 size_t objsize);
-
 void assert_oid_type(const struct object_id *oid, enum object_type expect);
 
 /*
@@ -553,4 +499,10 @@ int for_each_object_in_pack(struct packed_git *p,
 int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
 			   void *data, enum for_each_object_flags flags);
 
+void *read_object_with_reference(struct repository *r,
+				 const struct object_id *oid,
+				 enum object_type required_type,
+				 unsigned long *size,
+				 struct object_id *oid_ret);
+
 #endif /* OBJECT_STORE_LL_H */
diff --git a/object-store.c b/object-store.c
new file mode 100644
index 00000000000..a2004dca15a
--- /dev/null
+++ b/object-store.c
@@ -0,0 +1,972 @@
+#define USE_THE_REPOSITORY_VARIABLE
+
+#include "git-compat-util.h"
+#include "abspath.h"
+#include "config.h"
+#include "environment.h"
+#include "gettext.h"
+#include "hex.h"
+#include "lockfile.h"
+#include "object-file-convert.h"
+#include "object-file.h"
+#include "object-store.h"
+#include "packfile.h"
+#include "path.h"
+#include "promisor-remote.h"
+#include "quote.h"
+#include "replace-object.h"
+#include "run-command.h"
+#include "setup.h"
+#include "strbuf.h"
+#include "strvec.h"
+#include "submodule.h"
+#include "write-or-die.h"
+
+/*
+ * This is meant to hold a *small* number of objects that you would
+ * want repo_read_object_file() to be able to return, but yet you do not want
+ * to write them into the object store (e.g. a browse-only
+ * application).
+ */
+static struct cached_object_entry {
+	struct object_id oid;
+	struct cached_object {
+		enum object_type type;
+		const void *buf;
+		unsigned long size;
+	} value;
+} *cached_objects;
+static int cached_object_nr, cached_object_alloc;
+
+static const struct cached_object *find_cached_object(const struct object_id *oid)
+{
+	static const struct cached_object empty_tree = {
+		.type = OBJ_TREE,
+		.buf = "",
+	};
+	int i;
+	const struct cached_object_entry *co = cached_objects;
+
+	for (i = 0; i < cached_object_nr; i++, co++) {
+		if (oideq(&co->oid, oid))
+			return &co->value;
+	}
+	if (oideq(oid, the_hash_algo->empty_tree))
+		return &empty_tree;
+	return NULL;
+}
+
+int odb_mkstemp(struct strbuf *temp_filename, const char *pattern)
+{
+	int fd;
+	/*
+	 * we let the umask do its job, don't try to be more
+	 * restrictive except to remove write permission.
+	 */
+	int mode = 0444;
+	repo_git_path_replace(the_repository, temp_filename, "objects/%s", pattern);
+	fd = git_mkstemp_mode(temp_filename->buf, mode);
+	if (0 <= fd)
+		return fd;
+
+	/* slow path */
+	/* some mkstemp implementations erase temp_filename on failure */
+	repo_git_path_replace(the_repository, temp_filename, "objects/%s", pattern);
+	safe_create_leading_directories(the_repository, temp_filename->buf);
+	return xmkstemp_mode(temp_filename->buf, mode);
+}
+
+int odb_pack_keep(const char *name)
+{
+	int fd;
+
+	fd = open(name, O_RDWR|O_CREAT|O_EXCL, 0600);
+	if (0 <= fd)
+		return fd;
+
+	/* slow path */
+	safe_create_leading_directories_const(the_repository, name);
+	return open(name, O_RDWR|O_CREAT|O_EXCL, 0600);
+}
+
+const char *loose_object_path(struct repository *r, struct strbuf *buf,
+			      const struct object_id *oid)
+{
+	return odb_loose_path(r->objects->odb, buf, oid);
+}
+
+/*
+ * Return non-zero iff the path is usable as an alternate object database.
+ */
+static int alt_odb_usable(struct raw_object_store *o,
+			  struct strbuf *path,
+			  const char *normalized_objdir, khiter_t *pos)
+{
+	int r;
+
+	/* Detect cases where alternate disappeared */
+	if (!is_directory(path->buf)) {
+		error(_("object directory %s does not exist; "
+			"check .git/objects/info/alternates"),
+		      path->buf);
+		return 0;
+	}
+
+	/*
+	 * Prevent the common mistake of listing the same
+	 * thing twice, or object directory itself.
+	 */
+	if (!o->odb_by_path) {
+		khiter_t p;
+
+		o->odb_by_path = kh_init_odb_path_map();
+		assert(!o->odb->next);
+		p = kh_put_odb_path_map(o->odb_by_path, o->odb->path, &r);
+		assert(r == 1); /* never used */
+		kh_value(o->odb_by_path, p) = o->odb;
+	}
+	if (fspatheq(path->buf, normalized_objdir))
+		return 0;
+	*pos = kh_put_odb_path_map(o->odb_by_path, path->buf, &r);
+	/* r: 0 = exists, 1 = never used, 2 = deleted */
+	return r == 0 ? 0 : 1;
+}
+
+/*
+ * Prepare alternate object database registry.
+ *
+ * The variable alt_odb_list points at the list of struct
+ * object_directory.  The elements on this list come from
+ * non-empty elements from colon separated ALTERNATE_DB_ENVIRONMENT
+ * environment variable, and $GIT_OBJECT_DIRECTORY/info/alternates,
+ * whose contents is similar to that environment variable but can be
+ * LF separated.  Its base points at a statically allocated buffer that
+ * contains "/the/directory/corresponding/to/.git/objects/...", while
+ * its name points just after the slash at the end of ".git/objects/"
+ * in the example above, and has enough space to hold all hex characters
+ * of the object ID, an extra slash for the first level indirection, and
+ * the terminating NUL.
+ */
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth);
+static int link_alt_odb_entry(struct repository *r, const struct strbuf *entry,
+	const char *relative_base, int depth, const char *normalized_objdir)
+{
+	struct object_directory *ent;
+	struct strbuf pathbuf = STRBUF_INIT;
+	struct strbuf tmp = STRBUF_INIT;
+	khiter_t pos;
+	int ret = -1;
+
+	if (!is_absolute_path(entry->buf) && relative_base) {
+		strbuf_realpath(&pathbuf, relative_base, 1);
+		strbuf_addch(&pathbuf, '/');
+	}
+	strbuf_addbuf(&pathbuf, entry);
+
+	if (!strbuf_realpath(&tmp, pathbuf.buf, 0)) {
+		error(_("unable to normalize alternate object path: %s"),
+		      pathbuf.buf);
+		goto error;
+	}
+	strbuf_swap(&pathbuf, &tmp);
+
+	/*
+	 * The trailing slash after the directory name is given by
+	 * this function at the end. Remove duplicates.
+	 */
+	while (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')
+		strbuf_setlen(&pathbuf, pathbuf.len - 1);
+
+	if (!alt_odb_usable(r->objects, &pathbuf, normalized_objdir, &pos))
+		goto error;
+
+	CALLOC_ARRAY(ent, 1);
+	/* pathbuf.buf is already in r->objects->odb_by_path */
+	ent->path = strbuf_detach(&pathbuf, NULL);
+
+	/* add the alternate entry */
+	*r->objects->odb_tail = ent;
+	r->objects->odb_tail = &(ent->next);
+	ent->next = NULL;
+	assert(r->objects->odb_by_path);
+	kh_value(r->objects->odb_by_path, pos) = ent;
+
+	/* recursively add alternates */
+	read_info_alternates(r, ent->path, depth + 1);
+	ret = 0;
+ error:
+	strbuf_release(&tmp);
+	strbuf_release(&pathbuf);
+	return ret;
+}
+
+static const char *parse_alt_odb_entry(const char *string,
+				       int sep,
+				       struct strbuf *out)
+{
+	const char *end;
+
+	strbuf_reset(out);
+
+	if (*string == '#') {
+		/* comment; consume up to next separator */
+		end = strchrnul(string, sep);
+	} else if (*string == '"' && !unquote_c_style(out, string, &end)) {
+		/*
+		 * quoted path; unquote_c_style has copied the
+		 * data for us and set "end". Broken quoting (e.g.,
+		 * an entry that doesn't end with a quote) falls
+		 * back to the unquoted case below.
+		 */
+	} else {
+		/* normal, unquoted path */
+		end = strchrnul(string, sep);
+		strbuf_add(out, string, end - string);
+	}
+
+	if (*end)
+		end++;
+	return end;
+}
+
+static void link_alt_odb_entries(struct repository *r, const char *alt,
+				 int sep, const char *relative_base, int depth)
+{
+	struct strbuf objdirbuf = STRBUF_INIT;
+	struct strbuf entry = STRBUF_INIT;
+
+	if (!alt || !*alt)
+		return;
+
+	if (depth > 5) {
+		error(_("%s: ignoring alternate object stores, nesting too deep"),
+				relative_base);
+		return;
+	}
+
+	strbuf_realpath(&objdirbuf, r->objects->odb->path, 1);
+
+	while (*alt) {
+		alt = parse_alt_odb_entry(alt, sep, &entry);
+		if (!entry.len)
+			continue;
+		link_alt_odb_entry(r, &entry,
+				   relative_base, depth, objdirbuf.buf);
+	}
+	strbuf_release(&entry);
+	strbuf_release(&objdirbuf);
+}
+
+static void read_info_alternates(struct repository *r,
+				 const char *relative_base,
+				 int depth)
+{
+	char *path;
+	struct strbuf buf = STRBUF_INIT;
+
+	path = xstrfmt("%s/info/alternates", relative_base);
+	if (strbuf_read_file(&buf, path, 1024) < 0) {
+		warn_on_fopen_errors(path);
+		free(path);
+		return;
+	}
+
+	link_alt_odb_entries(r, buf.buf, '\n', relative_base, depth);
+	strbuf_release(&buf);
+	free(path);
+}
+
+void add_to_alternates_file(const char *reference)
+{
+	struct lock_file lock = LOCK_INIT;
+	char *alts = repo_git_path(the_repository, "objects/info/alternates");
+	FILE *in, *out;
+	int found = 0;
+
+	hold_lock_file_for_update(&lock, alts, LOCK_DIE_ON_ERROR);
+	out = fdopen_lock_file(&lock, "w");
+	if (!out)
+		die_errno(_("unable to fdopen alternates lockfile"));
+
+	in = fopen(alts, "r");
+	if (in) {
+		struct strbuf line = STRBUF_INIT;
+
+		while (strbuf_getline(&line, in) != EOF) {
+			if (!strcmp(reference, line.buf)) {
+				found = 1;
+				break;
+			}
+			fprintf_or_die(out, "%s\n", line.buf);
+		}
+
+		strbuf_release(&line);
+		fclose(in);
+	}
+	else if (errno != ENOENT)
+		die_errno(_("unable to read alternates file"));
+
+	if (found) {
+		rollback_lock_file(&lock);
+	} else {
+		fprintf_or_die(out, "%s\n", reference);
+		if (commit_lock_file(&lock))
+			die_errno(_("unable to move new alternates file into place"));
+		if (the_repository->objects->loaded_alternates)
+			link_alt_odb_entries(the_repository, reference,
+					     '\n', NULL, 0);
+	}
+	free(alts);
+}
+
+void add_to_alternates_memory(const char *reference)
+{
+	/*
+	 * Make sure alternates are initialized, or else our entry may be
+	 * overwritten when they are.
+	 */
+	prepare_alt_odb(the_repository);
+
+	link_alt_odb_entries(the_repository, reference,
+			     '\n', NULL, 0);
+}
+
+struct object_directory *set_temporary_primary_odb(const char *dir, int will_destroy)
+{
+	struct object_directory *new_odb;
+
+	/*
+	 * Make sure alternates are initialized, or else our entry may be
+	 * overwritten when they are.
+	 */
+	prepare_alt_odb(the_repository);
+
+	/*
+	 * Make a new primary odb and link the old primary ODB in as an
+	 * alternate
+	 */
+	new_odb = xcalloc(1, sizeof(*new_odb));
+	new_odb->path = xstrdup(dir);
+
+	/*
+	 * Disable ref updates while a temporary odb is active, since
+	 * the objects in the database may roll back.
+	 */
+	new_odb->disable_ref_updates = 1;
+	new_odb->will_destroy = will_destroy;
+	new_odb->next = the_repository->objects->odb;
+	the_repository->objects->odb = new_odb;
+	return new_odb->next;
+}
+
+void restore_primary_odb(struct object_directory *restore_odb, const char *old_path)
+{
+	struct object_directory *cur_odb = the_repository->objects->odb;
+
+	if (strcmp(old_path, cur_odb->path))
+		BUG("expected %s as primary object store; found %s",
+		    old_path, cur_odb->path);
+
+	if (cur_odb->next != restore_odb)
+		BUG("we expect the old primary object store to be the first alternate");
+
+	the_repository->objects->odb = restore_odb;
+	free_object_directory(cur_odb);
+}
+
+/*
+ * Compute the exact path an alternate is at and returns it. In case of
+ * error NULL is returned and the human readable error is added to `err`
+ * `path` may be relative and should point to $GIT_DIR.
+ * `err` must not be null.
+ */
+char *compute_alternate_path(const char *path, struct strbuf *err)
+{
+	char *ref_git = NULL;
+	const char *repo;
+	int seen_error = 0;
+
+	ref_git = real_pathdup(path, 0);
+	if (!ref_git) {
+		seen_error = 1;
+		strbuf_addf(err, _("path '%s' does not exist"), path);
+		goto out;
+	}
+
+	repo = read_gitfile(ref_git);
+	if (!repo)
+		repo = read_gitfile(mkpath("%s/.git", ref_git));
+	if (repo) {
+		free(ref_git);
+		ref_git = xstrdup(repo);
+	}
+
+	if (!repo && is_directory(mkpath("%s/.git/objects", ref_git))) {
+		char *ref_git_git = mkpathdup("%s/.git", ref_git);
+		free(ref_git);
+		ref_git = ref_git_git;
+	} else if (!is_directory(mkpath("%s/objects", ref_git))) {
+		struct strbuf sb = STRBUF_INIT;
+		seen_error = 1;
+		if (get_common_dir(&sb, ref_git)) {
+			strbuf_addf(err,
+				    _("reference repository '%s' as a linked "
+				      "checkout is not supported yet."),
+				    path);
+			goto out;
+		}
+
+		strbuf_addf(err, _("reference repository '%s' is not a "
+					"local repository."), path);
+		goto out;
+	}
+
+	if (!access(mkpath("%s/shallow", ref_git), F_OK)) {
+		strbuf_addf(err, _("reference repository '%s' is shallow"),
+			    path);
+		seen_error = 1;
+		goto out;
+	}
+
+	if (!access(mkpath("%s/info/grafts", ref_git), F_OK)) {
+		strbuf_addf(err,
+			    _("reference repository '%s' is grafted"),
+			    path);
+		seen_error = 1;
+		goto out;
+	}
+
+out:
+	if (seen_error) {
+		FREE_AND_NULL(ref_git);
+	}
+
+	return ref_git;
+}
+
+struct object_directory *find_odb(struct repository *r, const char *obj_dir)
+{
+	struct object_directory *odb;
+	char *obj_dir_real = real_pathdup(obj_dir, 1);
+	struct strbuf odb_path_real = STRBUF_INIT;
+
+	prepare_alt_odb(r);
+	for (odb = r->objects->odb; odb; odb = odb->next) {
+		strbuf_realpath(&odb_path_real, odb->path, 1);
+		if (!strcmp(obj_dir_real, odb_path_real.buf))
+			break;
+	}
+
+	free(obj_dir_real);
+	strbuf_release(&odb_path_real);
+
+	if (!odb)
+		die(_("could not find object directory matching %s"), obj_dir);
+	return odb;
+}
+
+static void fill_alternate_refs_command(struct child_process *cmd,
+					const char *repo_path)
+{
+	const char *value;
+
+	if (!git_config_get_value("core.alternateRefsCommand", &value)) {
+		cmd->use_shell = 1;
+
+		strvec_push(&cmd->args, value);
+		strvec_push(&cmd->args, repo_path);
+	} else {
+		cmd->git_cmd = 1;
+
+		strvec_pushf(&cmd->args, "--git-dir=%s", repo_path);
+		strvec_push(&cmd->args, "for-each-ref");
+		strvec_push(&cmd->args, "--format=%(objectname)");
+
+		if (!git_config_get_value("core.alternateRefsPrefixes", &value)) {
+			strvec_push(&cmd->args, "--");
+			strvec_split(&cmd->args, value);
+		}
+	}
+
+	strvec_pushv(&cmd->env, (const char **)local_repo_env);
+	cmd->out = -1;
+}
+
+static void read_alternate_refs(const char *path,
+				alternate_ref_fn *cb,
+				void *data)
+{
+	struct child_process cmd = CHILD_PROCESS_INIT;
+	struct strbuf line = STRBUF_INIT;
+	FILE *fh;
+
+	fill_alternate_refs_command(&cmd, path);
+
+	if (start_command(&cmd))
+		return;
+
+	fh = xfdopen(cmd.out, "r");
+	while (strbuf_getline_lf(&line, fh) != EOF) {
+		struct object_id oid;
+		const char *p;
+
+		if (parse_oid_hex(line.buf, &oid, &p) || *p) {
+			warning(_("invalid line while parsing alternate refs: %s"),
+				line.buf);
+			break;
+		}
+
+		cb(&oid, data);
+	}
+
+	fclose(fh);
+	finish_command(&cmd);
+	strbuf_release(&line);
+}
+
+struct alternate_refs_data {
+	alternate_ref_fn *fn;
+	void *data;
+};
+
+static int refs_from_alternate_cb(struct object_directory *e,
+				  void *data)
+{
+	struct strbuf path = STRBUF_INIT;
+	size_t base_len;
+	struct alternate_refs_data *cb = data;
+
+	if (!strbuf_realpath(&path, e->path, 0))
+		goto out;
+	if (!strbuf_strip_suffix(&path, "/objects"))
+		goto out;
+	base_len = path.len;
+
+	/* Is this a git repository with refs? */
+	strbuf_addstr(&path, "/refs");
+	if (!is_directory(path.buf))
+		goto out;
+	strbuf_setlen(&path, base_len);
+
+	read_alternate_refs(path.buf, cb->fn, cb->data);
+
+out:
+	strbuf_release(&path);
+	return 0;
+}
+
+void for_each_alternate_ref(alternate_ref_fn fn, void *data)
+{
+	struct alternate_refs_data cb;
+	cb.fn = fn;
+	cb.data = data;
+	foreach_alt_odb(refs_from_alternate_cb, &cb);
+}
+
+int foreach_alt_odb(alt_odb_fn fn, void *cb)
+{
+	struct object_directory *ent;
+	int r = 0;
+
+	prepare_alt_odb(the_repository);
+	for (ent = the_repository->objects->odb->next; ent; ent = ent->next) {
+		r = fn(ent, cb);
+		if (r)
+			break;
+	}
+	return r;
+}
+
+void prepare_alt_odb(struct repository *r)
+{
+	if (r->objects->loaded_alternates)
+		return;
+
+	link_alt_odb_entries(r, r->objects->alternate_db, PATH_SEP, NULL, 0);
+
+	read_info_alternates(r, r->objects->odb->path, 0);
+	r->objects->loaded_alternates = 1;
+}
+
+int has_alt_odb(struct repository *r)
+{
+	prepare_alt_odb(r);
+	return !!r->objects->odb->next;
+}
+
+int obj_read_use_lock = 0;
+pthread_mutex_t obj_read_mutex;
+
+void enable_obj_read_lock(void)
+{
+	if (obj_read_use_lock)
+		return;
+
+	obj_read_use_lock = 1;
+	init_recursive_mutex(&obj_read_mutex);
+}
+
+void disable_obj_read_lock(void)
+{
+	if (!obj_read_use_lock)
+		return;
+
+	obj_read_use_lock = 0;
+	pthread_mutex_destroy(&obj_read_mutex);
+}
+
+int fetch_if_missing = 1;
+
+static int do_oid_object_info_extended(struct repository *r,
+				       const struct object_id *oid,
+				       struct object_info *oi, unsigned flags)
+{
+	static struct object_info blank_oi = OBJECT_INFO_INIT;
+	const struct cached_object *co;
+	struct pack_entry e;
+	int rtype;
+	const struct object_id *real = oid;
+	int already_retried = 0;
+
+
+	if (flags & OBJECT_INFO_LOOKUP_REPLACE)
+		real = lookup_replace_object(r, oid);
+
+	if (is_null_oid(real))
+		return -1;
+
+	if (!oi)
+		oi = &blank_oi;
+
+	co = find_cached_object(real);
+	if (co) {
+		if (oi->typep)
+			*(oi->typep) = co->type;
+		if (oi->sizep)
+			*(oi->sizep) = co->size;
+		if (oi->disk_sizep)
+			*(oi->disk_sizep) = 0;
+		if (oi->delta_base_oid)
+			oidclr(oi->delta_base_oid, the_repository->hash_algo);
+		if (oi->type_name)
+			strbuf_addstr(oi->type_name, type_name(co->type));
+		if (oi->contentp)
+			*oi->contentp = xmemdupz(co->buf, co->size);
+		oi->whence = OI_CACHED;
+		return 0;
+	}
+
+	while (1) {
+		if (find_pack_entry(r, real, &e))
+			break;
+
+		/* Most likely it's a loose object. */
+		if (!loose_object_info(r, real, oi, flags))
+			return 0;
+
+		/* Not a loose object; someone else may have just packed it. */
+		if (!(flags & OBJECT_INFO_QUICK)) {
+			reprepare_packed_git(r);
+			if (find_pack_entry(r, real, &e))
+				break;
+		}
+
+		/*
+		 * If r is the_repository, this might be an attempt at
+		 * accessing a submodule object as if it were in the_repository
+		 * (having called add_submodule_odb() on that submodule's ODB).
+		 * If any such ODBs exist, register them and try again.
+		 */
+		if (r == the_repository &&
+		    register_all_submodule_odb_as_alternates())
+			/* We added some alternates; retry */
+			continue;
+
+		/* Check if it is a missing object */
+		if (fetch_if_missing && repo_has_promisor_remote(r) &&
+		    !already_retried &&
+		    !(flags & OBJECT_INFO_SKIP_FETCH_OBJECT)) {
+			promisor_remote_get_direct(r, real, 1);
+			already_retried = 1;
+			continue;
+		}
+
+		if (flags & OBJECT_INFO_DIE_IF_CORRUPT) {
+			const struct packed_git *p;
+			if ((flags & OBJECT_INFO_LOOKUP_REPLACE) && !oideq(real, oid))
+				die(_("replacement %s not found for %s"),
+				    oid_to_hex(real), oid_to_hex(oid));
+			if ((p = has_packed_and_bad(r, real)))
+				die(_("packed object %s (stored in %s) is corrupt"),
+				    oid_to_hex(real), p->pack_name);
+		}
+		return -1;
+	}
+
+	if (oi == &blank_oi)
+		/*
+		 * We know that the caller doesn't actually need the
+		 * information below, so return early.
+		 */
+		return 0;
+	rtype = packed_object_info(r, e.p, e.offset, oi);
+	if (rtype < 0) {
+		mark_bad_packed_object(e.p, real);
+		return do_oid_object_info_extended(r, real, oi, 0);
+	} else if (oi->whence == OI_PACKED) {
+		oi->u.packed.offset = e.offset;
+		oi->u.packed.pack = e.p;
+		oi->u.packed.is_delta = (rtype == OBJ_REF_DELTA ||
+					 rtype == OBJ_OFS_DELTA);
+	}
+
+	return 0;
+}
+
+static int oid_object_info_convert(struct repository *r,
+				   const struct object_id *input_oid,
+				   struct object_info *input_oi, unsigned flags)
+{
+	const struct git_hash_algo *input_algo = &hash_algos[input_oid->algo];
+	int do_die = flags & OBJECT_INFO_DIE_IF_CORRUPT;
+	struct strbuf type_name = STRBUF_INIT;
+	struct object_id oid, delta_base_oid;
+	struct object_info new_oi, *oi;
+	unsigned long size;
+	void *content;
+	int ret;
+
+	if (repo_oid_to_algop(r, input_oid, the_hash_algo, &oid)) {
+		if (do_die)
+			die(_("missing mapping of %s to %s"),
+			    oid_to_hex(input_oid), the_hash_algo->name);
+		return -1;
+	}
+
+	/* Is new_oi needed? */
+	oi = input_oi;
+	if (input_oi && (input_oi->delta_base_oid || input_oi->sizep ||
+			 input_oi->contentp)) {
+		new_oi = *input_oi;
+		/* Does delta_base_oid need to be converted? */
+		if (input_oi->delta_base_oid)
+			new_oi.delta_base_oid = &delta_base_oid;
+		/* Will the attributes differ when converted? */
+		if (input_oi->sizep || input_oi->contentp) {
+			new_oi.contentp = &content;
+			new_oi.sizep = &size;
+			new_oi.type_name = &type_name;
+		}
+		oi = &new_oi;
+	}
+
+	ret = oid_object_info_extended(r, &oid, oi, flags);
+	if (ret)
+		return -1;
+	if (oi == input_oi)
+		return ret;
+
+	if (new_oi.contentp) {
+		struct strbuf outbuf = STRBUF_INIT;
+		enum object_type type;
+
+		type = type_from_string_gently(type_name.buf, type_name.len,
+					       !do_die);
+		if (type == -1)
+			return -1;
+		if (type != OBJ_BLOB) {
+			ret = convert_object_file(the_repository, &outbuf,
+						  the_hash_algo, input_algo,
+						  content, size, type, !do_die);
+			free(content);
+			if (ret == -1)
+				return -1;
+			size = outbuf.len;
+			content = strbuf_detach(&outbuf, NULL);
+		}
+		if (input_oi->sizep)
+			*input_oi->sizep = size;
+		if (input_oi->contentp)
+			*input_oi->contentp = content;
+		else
+			free(content);
+		if (input_oi->type_name)
+			*input_oi->type_name = type_name;
+		else
+			strbuf_release(&type_name);
+	}
+	if (new_oi.delta_base_oid == &delta_base_oid) {
+		if (repo_oid_to_algop(r, &delta_base_oid, input_algo,
+				 input_oi->delta_base_oid)) {
+			if (do_die)
+				die(_("missing mapping of %s to %s"),
+				    oid_to_hex(&delta_base_oid),
+				    input_algo->name);
+			return -1;
+		}
+	}
+	input_oi->whence = new_oi.whence;
+	input_oi->u = new_oi.u;
+	return ret;
+}
+
+int oid_object_info_extended(struct repository *r, const struct object_id *oid,
+			     struct object_info *oi, unsigned flags)
+{
+	int ret;
+
+	if (oid->algo && (hash_algo_by_ptr(r->hash_algo) != oid->algo))
+		return oid_object_info_convert(r, oid, oi, flags);
+
+	obj_read_lock();
+	ret = do_oid_object_info_extended(r, oid, oi, flags);
+	obj_read_unlock();
+	return ret;
+}
+
+
+/* returns enum object_type or negative */
+int oid_object_info(struct repository *r,
+		    const struct object_id *oid,
+		    unsigned long *sizep)
+{
+	enum object_type type;
+	struct object_info oi = OBJECT_INFO_INIT;
+
+	oi.typep = &type;
+	oi.sizep = sizep;
+	if (oid_object_info_extended(r, oid, &oi,
+				      OBJECT_INFO_LOOKUP_REPLACE) < 0)
+		return -1;
+	return type;
+}
+
+int pretend_object_file(void *buf, unsigned long len, enum object_type type,
+			struct object_id *oid)
+{
+	struct cached_object_entry *co;
+	char *co_buf;
+
+	hash_object_file(the_hash_algo, buf, len, type, oid);
+	if (repo_has_object_file_with_flags(the_repository, oid, OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT) ||
+	    find_cached_object(oid))
+		return 0;
+	ALLOC_GROW(cached_objects, cached_object_nr + 1, cached_object_alloc);
+	co = &cached_objects[cached_object_nr++];
+	co->value.size = len;
+	co->value.type = type;
+	co_buf = xmalloc(len);
+	memcpy(co_buf, buf, len);
+	co->value.buf = co_buf;
+	oidcpy(&co->oid, oid);
+	return 0;
+}
+
+/*
+ * This function dies on corrupt objects; the callers who want to
+ * deal with them should arrange to call oid_object_info_extended() and give
+ * error messages themselves.
+ */
+void *repo_read_object_file(struct repository *r,
+			    const struct object_id *oid,
+			    enum object_type *type,
+			    unsigned long *size)
+{
+	struct object_info oi = OBJECT_INFO_INIT;
+	unsigned flags = OBJECT_INFO_DIE_IF_CORRUPT | OBJECT_INFO_LOOKUP_REPLACE;
+	void *data;
+
+	oi.typep = type;
+	oi.sizep = size;
+	oi.contentp = &data;
+	if (oid_object_info_extended(r, oid, &oi, flags))
+		return NULL;
+
+	return data;
+}
+
+void *read_object_with_reference(struct repository *r,
+				 const struct object_id *oid,
+				 enum object_type required_type,
+				 unsigned long *size,
+				 struct object_id *actual_oid_return)
+{
+	enum object_type type;
+	void *buffer;
+	unsigned long isize;
+	struct object_id actual_oid;
+
+	oidcpy(&actual_oid, oid);
+	while (1) {
+		int ref_length = -1;
+		const char *ref_type = NULL;
+
+		buffer = repo_read_object_file(r, &actual_oid, &type, &isize);
+		if (!buffer)
+			return NULL;
+		if (type == required_type) {
+			*size = isize;
+			if (actual_oid_return)
+				oidcpy(actual_oid_return, &actual_oid);
+			return buffer;
+		}
+		/* Handle references */
+		else if (type == OBJ_COMMIT)
+			ref_type = "tree ";
+		else if (type == OBJ_TAG)
+			ref_type = "object ";
+		else {
+			free(buffer);
+			return NULL;
+		}
+		ref_length = strlen(ref_type);
+
+		if (ref_length + the_hash_algo->hexsz > isize ||
+		    memcmp(buffer, ref_type, ref_length) ||
+		    get_oid_hex((char *) buffer + ref_length, &actual_oid)) {
+			free(buffer);
+			return NULL;
+		}
+		free(buffer);
+		/* Now we have the ID of the referred-to object in
+		 * actual_oid.  Check again. */
+	}
+}
+
+int has_object(struct repository *r, const struct object_id *oid,
+	       unsigned flags)
+{
+	int quick = !(flags & HAS_OBJECT_RECHECK_PACKED);
+	unsigned object_info_flags = OBJECT_INFO_SKIP_FETCH_OBJECT |
+		(quick ? OBJECT_INFO_QUICK : 0);
+
+	if (!startup_info->have_repository)
+		return 0;
+	return oid_object_info_extended(r, oid, NULL, object_info_flags) >= 0;
+}
+
+int repo_has_object_file_with_flags(struct repository *r,
+				    const struct object_id *oid, int flags)
+{
+	if (!startup_info->have_repository)
+		return 0;
+	return oid_object_info_extended(r, oid, NULL, flags) >= 0;
+}
+
+int repo_has_object_file(struct repository *r,
+			 const struct object_id *oid)
+{
+	return repo_has_object_file_with_flags(r, oid, 0);
+}
+
+void assert_oid_type(const struct object_id *oid, enum object_type expect)
+{
+	enum object_type type = oid_object_info(the_repository, oid, NULL);
+	if (type < 0)
+		die(_("%s is not a valid object"), oid_to_hex(oid));
+	if (type != expect)
+		die(_("%s is not a valid '%s' object"), oid_to_hex(oid),
+		    type_name(expect));
+}

-- 
2.49.0.805.g082f7c87e0.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH v3 06/10] object-file: split up concerns of `HASH_*` flags
  2025-04-15  9:38 ` [PATCH v3 00/10] Split up "object-file.c" Patrick Steinhardt
                     ` (4 preceding siblings ...)
  2025-04-15  9:38   ` [PATCH v3 05/10] object-file: split out functions relating to object store subsystem Patrick Steinhardt
@ 2025-04-15  9:38   ` Patrick Steinhardt
  2025-04-15  9:38   ` [PATCH v3 07/10] object-file: drop `index_blob_stream()` Patrick Steinhardt
                     ` (4 subsequent siblings)
  10 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-15  9:38 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Eric Sunshine, Jeff King, Junio C Hamano

The functions `hash_object_file()`, `write_object_file()` and
`index_fd()` reuse the same set of flags to alter their behaviour. This
not only adds confusion, but given that every function only supports a
subset of the flags it becomes very hard to see which flags can be
passed to what function. Last but not least, this entangles the
implementation of all three function families.

Split up concerns by creating separate flags for each of the function
families.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/hash-object.c  | 23 +++++++++++++++++------
 builtin/replace.c      |  2 +-
 builtin/update-index.c |  2 +-
 bulk-checkin.c         |  6 +++---
 cache-tree.c           |  2 +-
 notes-merge.c          |  2 +-
 object-file.c          | 18 +++++++++---------
 object-file.h          | 25 +++++++++++++++++++++----
 read-cache.c           |  4 ++--
 9 files changed, 56 insertions(+), 28 deletions(-)

diff --git a/builtin/hash-object.c b/builtin/hash-object.c
index a25f0403f44..e7c0d6afdef 100644
--- a/builtin/hash-object.c
+++ b/builtin/hash-object.c
@@ -19,6 +19,11 @@
 #include "strbuf.h"
 #include "write-or-die.h"
 
+enum {
+	HASH_OBJECT_CHECK = (1 << 0),
+	HASH_OBJECT_WRITE = (1 << 1),
+};
+
 /*
  * This is to create corrupt objects for debugging and as such it
  * needs to bypass the data conversion performed by, and the type
@@ -33,7 +38,7 @@ static int hash_literally(struct object_id *oid, int fd, const char *type, unsig
 		ret = -1;
 	else
 		ret = write_object_file_literally(buf.buf, buf.len, type, oid,
-						 flags);
+						  (flags & HASH_OBJECT_WRITE) ? WRITE_OBJECT_FILE_PERSIST : 0);
 	close(fd);
 	strbuf_release(&buf);
 	return ret;
@@ -42,15 +47,21 @@ static int hash_literally(struct object_id *oid, int fd, const char *type, unsig
 static void hash_fd(int fd, const char *type, const char *path, unsigned flags,
 		    int literally)
 {
+	unsigned int index_flags = 0;
 	struct stat st;
 	struct object_id oid;
 
+	if (flags & HASH_OBJECT_WRITE)
+		index_flags |= INDEX_WRITE_OBJECT;
+	if (flags & HASH_OBJECT_CHECK)
+		index_flags |= INDEX_FORMAT_CHECK;
+
 	if (fstat(fd, &st) < 0 ||
 	    (literally
 	     ? hash_literally(&oid, fd, type, flags)
 	     : index_fd(the_repository->index, &oid, fd, &st,
-			type_from_string(type), path, flags)))
-		die((flags & HASH_WRITE_OBJECT)
+			type_from_string(type), path, index_flags)))
+		die((flags & HASH_OBJECT_WRITE)
 		    ? "Unable to add %s to database"
 		    : "Unable to hash %s", path);
 	printf("%s\n", oid_to_hex(&oid));
@@ -102,13 +113,13 @@ int cmd_hash_object(int argc,
 	int no_filters = 0;
 	int literally = 0;
 	int nongit = 0;
-	unsigned flags = HASH_FORMAT_CHECK;
+	unsigned flags = HASH_OBJECT_CHECK;
 	const char *vpath = NULL;
 	char *vpath_free = NULL;
 	const struct option hash_object_options[] = {
 		OPT_STRING('t', NULL, &type, N_("type"), N_("object type")),
 		OPT_BIT('w', NULL, &flags, N_("write the object into the object database"),
-			HASH_WRITE_OBJECT),
+			HASH_OBJECT_WRITE),
 		OPT_COUNTUP( 0 , "stdin", &hashstdin, N_("read the object from stdin")),
 		OPT_BOOL( 0 , "stdin-paths", &stdin_paths, N_("read file names from stdin")),
 		OPT_BOOL( 0 , "no-filters", &no_filters, N_("store file as is without filters")),
@@ -122,7 +133,7 @@ int cmd_hash_object(int argc,
 	argc = parse_options(argc, argv, prefix, hash_object_options,
 			     hash_object_usage, 0);
 
-	if (flags & HASH_WRITE_OBJECT)
+	if (flags & HASH_OBJECT_WRITE)
 		prefix = setup_git_directory();
 	else
 		prefix = setup_git_directory_gently(&nongit);
diff --git a/builtin/replace.c b/builtin/replace.c
index 15ec0922ce1..2b4fc9a68b3 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -305,7 +305,7 @@ static int import_object(struct object_id *oid, enum object_type type,
 		strbuf_release(&result);
 	} else {
 		struct stat st;
-		int flags = HASH_FORMAT_CHECK | HASH_WRITE_OBJECT;
+		int flags = INDEX_FORMAT_CHECK | INDEX_WRITE_OBJECT;
 
 		if (fstat(fd, &st) < 0) {
 			error_errno(_("unable to fstat %s"), filename);
diff --git a/builtin/update-index.c b/builtin/update-index.c
index b2f6b1a3fbb..f0cf964294d 100644
--- a/builtin/update-index.c
+++ b/builtin/update-index.c
@@ -304,7 +304,7 @@ static int add_one_path(const struct cache_entry *old, const char *path, int len
 	ce->ce_mode = ce_mode_from_stat(old, st->st_mode);
 
 	if (index_path(the_repository->index, &ce->oid, path, st,
-		       info_only ? 0 : HASH_WRITE_OBJECT)) {
+		       info_only ? 0 : INDEX_WRITE_OBJECT)) {
 		discard_cache_entry(ce);
 		return -1;
 	}
diff --git a/bulk-checkin.c b/bulk-checkin.c
index 23ac00ea0a6..309201a76a6 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -171,7 +171,7 @@ static int stream_blob_to_pack(struct bulk_checkin_packfile *state,
 	unsigned char obuf[16384];
 	unsigned hdrlen;
 	int status = Z_OK;
-	int write_object = (flags & HASH_WRITE_OBJECT);
+	int write_object = (flags & INDEX_WRITE_OBJECT);
 	off_t offset = 0;
 
 	git_deflate_init(&s, pack_compression_level);
@@ -241,7 +241,7 @@ static int stream_blob_to_pack(struct bulk_checkin_packfile *state,
 static void prepare_to_stream(struct bulk_checkin_packfile *state,
 			      unsigned flags)
 {
-	if (!(flags & HASH_WRITE_OBJECT) || state->f)
+	if (!(flags & INDEX_WRITE_OBJECT) || state->f)
 		return;
 
 	state->f = create_tmp_packfile(the_repository, &state->pack_tmp_name);
@@ -275,7 +275,7 @@ static int deflate_blob_to_pack(struct bulk_checkin_packfile *state,
 	git_hash_update(&ctx, obuf, header_len);
 
 	/* Note: idx is non-NULL when we are writing */
-	if ((flags & HASH_WRITE_OBJECT) != 0) {
+	if ((flags & INDEX_WRITE_OBJECT) != 0) {
 		CALLOC_ARRAY(idx, 1);
 
 		prepare_to_stream(state, flags);
diff --git a/cache-tree.c b/cache-tree.c
index bcbcad3d61a..4c8167ea927 100644
--- a/cache-tree.c
+++ b/cache-tree.c
@@ -452,7 +452,7 @@ static int update_one(struct cache_tree *it,
 				 OBJ_TREE, &it->oid);
 	} else if (write_object_file_flags(buffer.buf, buffer.len, OBJ_TREE,
 					   &it->oid, NULL, flags & WRITE_TREE_SILENT
-					   ? HASH_SILENT : 0)) {
+					   ? WRITE_OBJECT_FILE_SILENT : 0)) {
 		strbuf_release(&buffer);
 		return -1;
 	}
diff --git a/notes-merge.c b/notes-merge.c
index fce45043655..520b92942cd 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -729,7 +729,7 @@ int notes_merge_commit(struct notes_merge_options *o,
 		/* write file as blob, and add to partial_tree */
 		if (stat(path.buf, &st))
 			die_errno("Failed to stat '%s'", path.buf);
-		if (index_path(o->repo->index, &blob_oid, path.buf, &st, HASH_WRITE_OBJECT))
+		if (index_path(o->repo->index, &blob_oid, path.buf, &st, INDEX_WRITE_OBJECT))
 			die("Failed to write blob object from '%s'", path.buf);
 		if (add_note(partial_tree, &obj_oid, &blob_oid, NULL))
 			die("Failed to add resolved note '%s' to notes tree",
diff --git a/object-file.c b/object-file.c
index baa828822ea..2051991f4de 100644
--- a/object-file.c
+++ b/object-file.c
@@ -33,9 +33,9 @@
 
 static int get_conv_flags(unsigned flags)
 {
-	if (flags & HASH_RENORMALIZE)
+	if (flags & INDEX_RENORMALIZE)
 		return CONV_EOL_RENORMALIZE;
-	else if (flags & HASH_WRITE_OBJECT)
+	else if (flags & INDEX_WRITE_OBJECT)
 		return global_conv_flags_eol | CONV_WRITE_OBJECT;
 	else
 		return 0;
@@ -835,7 +835,7 @@ static int start_loose_object_common(struct strbuf *tmp_file,
 
 	fd = create_tmpfile(tmp_file, filename);
 	if (fd < 0) {
-		if (flags & HASH_SILENT)
+		if (flags & WRITE_OBJECT_FILE_SILENT)
 			return -1;
 		else if (errno == EACCES)
 			return error(_("insufficient permission for adding "
@@ -967,7 +967,7 @@ static int write_loose_object(const struct object_id *oid, char *hdr,
 		utb.actime = mtime;
 		utb.modtime = mtime;
 		if (utime(tmp_file.buf, &utb) < 0 &&
-		    !(flags & HASH_SILENT))
+		    !(flags & WRITE_OBJECT_FILE_SILENT))
 			warning_errno(_("failed utime() on %s"), tmp_file.buf);
 	}
 
@@ -1179,7 +1179,7 @@ int write_object_file_literally(const void *buf, unsigned long len,
 	write_object_file_prepare_literally(the_hash_algo, buf, len, type,
 					    oid, header, &hdrlen);
 
-	if (!(flags & HASH_WRITE_OBJECT))
+	if (!(flags & WRITE_OBJECT_FILE_PERSIST))
 		goto cleanup;
 	if (freshen_packed_object(oid) || freshen_loose_object(oid))
 		goto cleanup;
@@ -1250,7 +1250,7 @@ static int index_mem(struct index_state *istate,
 {
 	struct strbuf nbuf = STRBUF_INIT;
 	int ret = 0;
-	int write_object = flags & HASH_WRITE_OBJECT;
+	int write_object = flags & INDEX_WRITE_OBJECT;
 
 	if (!type)
 		type = OBJ_BLOB;
@@ -1265,7 +1265,7 @@ static int index_mem(struct index_state *istate,
 			size = nbuf.len;
 		}
 	}
-	if (flags & HASH_FORMAT_CHECK) {
+	if (flags & INDEX_FORMAT_CHECK) {
 		struct fsck_options opts = FSCK_OPTIONS_DEFAULT;
 
 		opts.strict = 1;
@@ -1291,7 +1291,7 @@ static int index_stream_convert_blob(struct index_state *istate,
 				     unsigned flags)
 {
 	int ret = 0;
-	const int write_object = flags & HASH_WRITE_OBJECT;
+	const int write_object = flags & INDEX_WRITE_OBJECT;
 	struct strbuf sbuf = STRBUF_INIT;
 
 	assert(path);
@@ -1423,7 +1423,7 @@ int index_path(struct index_state *istate, struct object_id *oid,
 	case S_IFLNK:
 		if (strbuf_readlink(&sb, path, st->st_size))
 			return error_errno("readlink(\"%s\")", path);
-		if (!(flags & HASH_WRITE_OBJECT))
+		if (!(flags & INDEX_WRITE_OBJECT))
 			hash_object_file(the_hash_algo, sb.buf, sb.len,
 					 OBJ_BLOB, oid);
 		else if (write_object_file(sb.buf, sb.len, OBJ_BLOB, oid))
diff --git a/object-file.h b/object-file.h
index 78c84d970a9..c002fbe2345 100644
--- a/object-file.h
+++ b/object-file.h
@@ -14,10 +14,12 @@ struct index_state;
  */
 extern int fetch_if_missing;
 
-#define HASH_WRITE_OBJECT 1
-#define HASH_FORMAT_CHECK 2
-#define HASH_RENORMALIZE  4
-#define HASH_SILENT 8
+enum {
+	INDEX_WRITE_OBJECT = (1 << 0),
+	INDEX_FORMAT_CHECK = (1 << 1),
+	INDEX_RENORMALIZE  = (1 << 2),
+};
+
 int index_fd(struct index_state *istate, struct object_id *oid, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);
 int index_path(struct index_state *istate, struct object_id *oid, const char *path, struct stat *st, unsigned flags);
 
@@ -84,6 +86,21 @@ enum unpack_loose_header_result unpack_loose_header(git_zstream *stream,
 struct object_info;
 int parse_loose_header(const char *hdr, struct object_info *oi);
 
+enum {
+	/*
+	 * By default, `write_object_file_literally()` does not actually write
+	 * anything into the object store, but only computes the object ID.
+	 * This flag changes that so that the object will be written as a loose
+	 * object and persisted.
+	 */
+	WRITE_OBJECT_FILE_PERSIST = (1 << 0),
+
+	/*
+	 * Do not print an error in case something gose wrong.
+	 */
+	WRITE_OBJECT_FILE_SILENT = (1 << 1),
+};
+
 int write_object_file_flags(const void *buf, unsigned long len,
 			    enum object_type type, struct object_id *oid,
 			    struct object_id *comapt_oid_in, unsigned flags);
diff --git a/read-cache.c b/read-cache.c
index 2f9e21c897d..23028f43a11 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -706,11 +706,11 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st,
 	int intent_only = flags & ADD_CACHE_INTENT;
 	int add_option = (ADD_CACHE_OK_TO_ADD|ADD_CACHE_OK_TO_REPLACE|
 			  (intent_only ? ADD_CACHE_NEW_ONLY : 0));
-	unsigned hash_flags = pretend ? 0 : HASH_WRITE_OBJECT;
+	unsigned hash_flags = pretend ? 0 : INDEX_WRITE_OBJECT;
 	struct object_id oid;
 
 	if (flags & ADD_CACHE_RENORMALIZE)
-		hash_flags |= HASH_RENORMALIZE;
+		hash_flags |= INDEX_RENORMALIZE;
 
 	if (!S_ISREG(st_mode) && !S_ISLNK(st_mode) && !S_ISDIR(st_mode))
 		return error(_("%s: can only add regular files, symbolic links or git-directories"), path);

-- 
2.49.0.805.g082f7c87e0.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH v3 07/10] object-file: drop `index_blob_stream()`
  2025-04-15  9:38 ` [PATCH v3 00/10] Split up "object-file.c" Patrick Steinhardt
                     ` (5 preceding siblings ...)
  2025-04-15  9:38   ` [PATCH v3 06/10] object-file: split up concerns of `HASH_*` flags Patrick Steinhardt
@ 2025-04-15  9:38   ` Patrick Steinhardt
  2025-04-15  9:38   ` [PATCH v3 08/10] object: split out functions relating to object store subsystem Patrick Steinhardt
                     ` (3 subsequent siblings)
  10 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-15  9:38 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Eric Sunshine, Jeff King, Junio C Hamano

The `index_blob_stream()` function is a mere wrapper around
`index_blob_bulk_checkin()`. This has been the case since 568508e7657
(bulk-checkin: replace fast-import based implementation, 2011-10-28),
which has moved the implementation from `index_blob_stream()` (which was
still called `index_stream()`) into `index_bulk_checkin()` (which has
since been renamed to `index_blob_bulk_checkin()`).

Remove the redirection by dropping the wrapper. Move the comment to
`index_blob_bulk_checkin()` to retain its context.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 bulk-checkin.h | 15 +++++++++++++++
 object-file.c  | 26 ++------------------------
 2 files changed, 17 insertions(+), 24 deletions(-)

diff --git a/bulk-checkin.h b/bulk-checkin.h
index aa7286a7b3e..7246ea58dcf 100644
--- a/bulk-checkin.h
+++ b/bulk-checkin.h
@@ -9,6 +9,21 @@
 void prepare_loose_object_bulk_checkin(void);
 void fsync_loose_object_bulk_checkin(int fd, const char *filename);
 
+/*
+ * This creates one packfile per large blob unless bulk-checkin
+ * machinery is "plugged".
+ *
+ * This also bypasses the usual "convert-to-git" dance, and that is on
+ * purpose. We could write a streaming version of the converting
+ * functions and insert that before feeding the data to fast-import
+ * (or equivalent in-core API described above). However, that is
+ * somewhat complicated, as we do not know the size of the filter
+ * result, which we need to know beforehand when writing a git object.
+ * Since the primary motivation for trying to stream from the working
+ * tree file and to avoid mmaping it in core is to deal with large
+ * binary blobs, they generally do not want to get any conversion, and
+ * callers should avoid this code path when filters are requested.
+ */
 int index_blob_bulk_checkin(struct object_id *oid,
 			    int fd, size_t size,
 			    const char *path, unsigned flags);
diff --git a/object-file.c b/object-file.c
index 2051991f4de..6084d603136 100644
--- a/object-file.c
+++ b/object-file.c
@@ -1356,28 +1356,6 @@ static int index_core(struct index_state *istate,
 	return ret;
 }
 
-/*
- * This creates one packfile per large blob unless bulk-checkin
- * machinery is "plugged".
- *
- * This also bypasses the usual "convert-to-git" dance, and that is on
- * purpose. We could write a streaming version of the converting
- * functions and insert that before feeding the data to fast-import
- * (or equivalent in-core API described above). However, that is
- * somewhat complicated, as we do not know the size of the filter
- * result, which we need to know beforehand when writing a git object.
- * Since the primary motivation for trying to stream from the working
- * tree file and to avoid mmaping it in core is to deal with large
- * binary blobs, they generally do not want to get any conversion, and
- * callers should avoid this code path when filters are requested.
- */
-static int index_blob_stream(struct object_id *oid, int fd, size_t size,
-			     const char *path,
-			     unsigned flags)
-{
-	return index_blob_bulk_checkin(oid, fd, size, path, flags);
-}
-
 int index_fd(struct index_state *istate, struct object_id *oid,
 	     int fd, struct stat *st,
 	     enum object_type type, const char *path, unsigned flags)
@@ -1398,8 +1376,8 @@ int index_fd(struct index_state *istate, struct object_id *oid,
 		ret = index_core(istate, oid, fd, xsize_t(st->st_size),
 				 type, path, flags);
 	else
-		ret = index_blob_stream(oid, fd, xsize_t(st->st_size), path,
-					flags);
+		ret = index_blob_bulk_checkin(oid, fd, xsize_t(st->st_size), path,
+					     flags);
 	close(fd);
 	return ret;
 }

-- 
2.49.0.805.g082f7c87e0.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH v3 08/10] object: split out functions relating to object store subsystem
  2025-04-15  9:38 ` [PATCH v3 00/10] Split up "object-file.c" Patrick Steinhardt
                     ` (6 preceding siblings ...)
  2025-04-15  9:38   ` [PATCH v3 07/10] object-file: drop `index_blob_stream()` Patrick Steinhardt
@ 2025-04-15  9:38   ` Patrick Steinhardt
  2025-04-15  9:38   ` [PATCH v3 09/10] object-store: remove global array of cached objects Patrick Steinhardt
                     ` (2 subsequent siblings)
  10 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-15  9:38 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Eric Sunshine, Jeff King, Junio C Hamano

Split out functions relating to the object store subsystem from
"object.c". This helps us to separate concerns.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 object-store-ll.h |  3 ---
 object-store.c    | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 object.c          | 67 -------------------------------------------------------
 3 files changed, 66 insertions(+), 70 deletions(-)

diff --git a/object-store-ll.h b/object-store-ll.h
index 8ae80b8a5fa..8bb0f33f9a8 100644
--- a/object-store-ll.h
+++ b/object-store-ll.h
@@ -92,9 +92,6 @@ struct oidtree *odb_loose_cache(struct object_directory *odb,
 /* Empty the loose object cache for the specified object directory. */
 void odb_clear_loose_cache(struct object_directory *odb);
 
-/* Clear and free the specified object directory */
-void free_object_directory(struct object_directory *odb);
-
 struct packed_git {
 	struct hashmap_entry packmap_ent;
 	struct packed_git *next;
diff --git a/object-store.c b/object-store.c
index a2004dca15a..896d9ac3509 100644
--- a/object-store.c
+++ b/object-store.c
@@ -2,11 +2,13 @@
 
 #include "git-compat-util.h"
 #include "abspath.h"
+#include "commit-graph.h"
 #include "config.h"
 #include "environment.h"
 #include "gettext.h"
 #include "hex.h"
 #include "lockfile.h"
+#include "loose.h"
 #include "object-file-convert.h"
 #include "object-file.h"
 #include "object-store.h"
@@ -361,6 +363,14 @@ struct object_directory *set_temporary_primary_odb(const char *dir, int will_des
 	return new_odb->next;
 }
 
+static void free_object_directory(struct object_directory *odb)
+{
+	free(odb->path);
+	odb_clear_loose_cache(odb);
+	loose_object_map_clear(&odb->loose_map);
+	free(odb);
+}
+
 void restore_primary_odb(struct object_directory *restore_odb, const char *old_path)
 {
 	struct object_directory *cur_odb = the_repository->objects->odb;
@@ -970,3 +980,59 @@ void assert_oid_type(const struct object_id *oid, enum object_type expect)
 		die(_("%s is not a valid '%s' object"), oid_to_hex(oid),
 		    type_name(expect));
 }
+
+struct raw_object_store *raw_object_store_new(void)
+{
+	struct raw_object_store *o = xmalloc(sizeof(*o));
+
+	memset(o, 0, sizeof(*o));
+	INIT_LIST_HEAD(&o->packed_git_mru);
+	hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0);
+	pthread_mutex_init(&o->replace_mutex, NULL);
+	return o;
+}
+
+static void free_object_directories(struct raw_object_store *o)
+{
+	while (o->odb) {
+		struct object_directory *next;
+
+		next = o->odb->next;
+		free_object_directory(o->odb);
+		o->odb = next;
+	}
+	kh_destroy_odb_path_map(o->odb_by_path);
+	o->odb_by_path = NULL;
+}
+
+void raw_object_store_clear(struct raw_object_store *o)
+{
+	FREE_AND_NULL(o->alternate_db);
+
+	oidmap_free(o->replace_map, 1);
+	FREE_AND_NULL(o->replace_map);
+	pthread_mutex_destroy(&o->replace_mutex);
+
+	free_commit_graph(o->commit_graph);
+	o->commit_graph = NULL;
+	o->commit_graph_attempted = 0;
+
+	free_object_directories(o);
+	o->odb_tail = NULL;
+	o->loaded_alternates = 0;
+
+	INIT_LIST_HEAD(&o->packed_git_mru);
+	close_object_store(o);
+
+	/*
+	 * `close_object_store()` only closes the packfiles, but doesn't free
+	 * them. We thus have to do this manually.
+	 */
+	for (struct packed_git *p = o->packed_git, *next; p; p = next) {
+		next = p->next;
+		free(p);
+	}
+	o->packed_git = NULL;
+
+	hashmap_clear(&o->pack_map);
+}
diff --git a/object.c b/object.c
index 154525a4972..ccda798b75f 100644
--- a/object.c
+++ b/object.c
@@ -6,16 +6,13 @@
 #include "object.h"
 #include "replace-object.h"
 #include "object-file.h"
-#include "object-store.h"
 #include "blob.h"
 #include "statinfo.h"
 #include "tree.h"
 #include "commit.h"
 #include "tag.h"
 #include "alloc.h"
-#include "packfile.h"
 #include "commit-graph.h"
-#include "loose.h"
 
 unsigned int get_max_object_index(const struct repository *repo)
 {
@@ -567,70 +564,6 @@ struct parsed_object_pool *parsed_object_pool_new(struct repository *repo)
 	return o;
 }
 
-struct raw_object_store *raw_object_store_new(void)
-{
-	struct raw_object_store *o = xmalloc(sizeof(*o));
-
-	memset(o, 0, sizeof(*o));
-	INIT_LIST_HEAD(&o->packed_git_mru);
-	hashmap_init(&o->pack_map, pack_map_entry_cmp, NULL, 0);
-	pthread_mutex_init(&o->replace_mutex, NULL);
-	return o;
-}
-
-void free_object_directory(struct object_directory *odb)
-{
-	free(odb->path);
-	odb_clear_loose_cache(odb);
-	loose_object_map_clear(&odb->loose_map);
-	free(odb);
-}
-
-static void free_object_directories(struct raw_object_store *o)
-{
-	while (o->odb) {
-		struct object_directory *next;
-
-		next = o->odb->next;
-		free_object_directory(o->odb);
-		o->odb = next;
-	}
-	kh_destroy_odb_path_map(o->odb_by_path);
-	o->odb_by_path = NULL;
-}
-
-void raw_object_store_clear(struct raw_object_store *o)
-{
-	FREE_AND_NULL(o->alternate_db);
-
-	oidmap_free(o->replace_map, 1);
-	FREE_AND_NULL(o->replace_map);
-	pthread_mutex_destroy(&o->replace_mutex);
-
-	free_commit_graph(o->commit_graph);
-	o->commit_graph = NULL;
-	o->commit_graph_attempted = 0;
-
-	free_object_directories(o);
-	o->odb_tail = NULL;
-	o->loaded_alternates = 0;
-
-	INIT_LIST_HEAD(&o->packed_git_mru);
-	close_object_store(o);
-
-	/*
-	 * `close_object_store()` only closes the packfiles, but doesn't free
-	 * them. We thus have to do this manually.
-	 */
-	for (struct packed_git *p = o->packed_git, *next; p; p = next) {
-		next = p->next;
-		free(p);
-	}
-	o->packed_git = NULL;
-
-	hashmap_clear(&o->pack_map);
-}
-
 void parsed_object_pool_reset_commit_grafts(struct parsed_object_pool *o)
 {
 	for (int i = 0; i < o->grafts_nr; i++) {

-- 
2.49.0.805.g082f7c87e0.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH v3 09/10] object-store: remove global array of cached objects
  2025-04-15  9:38 ` [PATCH v3 00/10] Split up "object-file.c" Patrick Steinhardt
                     ` (7 preceding siblings ...)
  2025-04-15  9:38   ` [PATCH v3 08/10] object: split out functions relating to object store subsystem Patrick Steinhardt
@ 2025-04-15  9:38   ` Patrick Steinhardt
  2025-04-15  9:38   ` [PATCH v3 10/10] object-store: merge "object-store-ll.h" and "object-store.h" Patrick Steinhardt
  2025-04-16  6:41   ` [PATCH v3 00/10] Split up "object-file.c" Elijah Newren
  10 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-15  9:38 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Eric Sunshine, Jeff King, Junio C Hamano

Cached objects are virtual objects that can be set up without writing
anything into the object store directly, which is used by git-blame(1)
to create fake commits for the working tree.

These cached objects are stored in a global variable, which is another
roadblock for libification of the object subsystem. Refactor the code so
that we instead store the array as part of the raw object store.

This refactoring raises the question whether virtual objects should
really be specific to a single repository (or rather a single object
store). Hypothetical usecases might for example span across submodules,
and here it may or may not be the right thing to provide virtual objects
across submodule boundaries.

The only existing usecase is git-blame(1) though, which does not know to
blame across submodule boundaries in the first place. As such, storing
these objects both globally and per-repository would achieve the same
result right now. But arguably, if we learned to blame across submodule
boundaries, we would likely want to create separate fare working tree
commits for each of the submodules so that the user can learn which
worktree a specific uncommitted change belongs to. And even if we would
want to create the same fake commit for each of the submodules we could
do that when storing separate virtual objects per object store.

While this is all rather hypothetical, the takeaway is that handling
virtual objects per-object store gives us more flexibility compared to
storing them globally. In a hypothetical future where we have achieved
full libification one might be able to handle unrelated repositories in
a single process, where the state of one repository should not have an
impact on the state of another repository. As such, storing these cached
objects per object store will enable more usecases and should lead to
less surprising outcomes overall.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 blame.c           |  2 +-
 object-store-ll.h | 14 +++++++++++++-
 object-store.c    | 39 +++++++++++++++++++++++----------------
 3 files changed, 37 insertions(+), 18 deletions(-)

diff --git a/blame.c b/blame.c
index 703dab43e78..b7c5bd692e6 100644
--- a/blame.c
+++ b/blame.c
@@ -277,7 +277,7 @@ static struct commit *fake_working_tree_commit(struct repository *r,
 	convert_to_git(r->index, path, buf.buf, buf.len, &buf, 0);
 	origin->file.ptr = buf.buf;
 	origin->file.size = buf.len;
-	pretend_object_file(buf.buf, buf.len, OBJ_BLOB, &origin->blob_oid);
+	pretend_object_file(the_repository, buf.buf, buf.len, OBJ_BLOB, &origin->blob_oid);
 
 	/*
 	 * Read the current index, replace the path entry with
diff --git a/object-store-ll.h b/object-store-ll.h
index 8bb0f33f9a8..bb5e8798a1b 100644
--- a/object-store-ll.h
+++ b/object-store-ll.h
@@ -151,6 +151,8 @@ static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
 	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
 }
 
+struct cached_object_entry;
+
 struct raw_object_store {
 	/*
 	 * Set of all object directories; the main directory is first (and
@@ -203,6 +205,15 @@ struct raw_object_store {
 		unsigned flags;
 	} kept_pack_cache;
 
+	/*
+	 * This is meant to hold a *small* number of objects that you would
+	 * want repo_read_object_file() to be able to return, but yet you do not want
+	 * to write them into the object store (e.g. a browse-only
+	 * application).
+	 */
+	struct cached_object_entry *cached_objects;
+	size_t cached_object_nr, cached_object_alloc;
+
 	/*
 	 * A map of packfiles to packed_git structs for tracking which
 	 * packs have been loaded already.
@@ -272,7 +283,8 @@ void hash_object_file(const struct git_hash_algo *algo, const void *buf,
  * object in persistent storage before writing any other new objects
  * that reference it.
  */
-int pretend_object_file(void *, unsigned long, enum object_type,
+int pretend_object_file(struct repository *repo,
+			void *buf, unsigned long len, enum object_type type,
 			struct object_id *oid);
 
 struct object_info {
diff --git a/object-store.c b/object-store.c
index 896d9ac3509..0f1dcc113ed 100644
--- a/object-store.c
+++ b/object-store.c
@@ -30,31 +30,31 @@
  * to write them into the object store (e.g. a browse-only
  * application).
  */
-static struct cached_object_entry {
+struct cached_object_entry {
 	struct object_id oid;
 	struct cached_object {
 		enum object_type type;
 		const void *buf;
 		unsigned long size;
 	} value;
-} *cached_objects;
-static int cached_object_nr, cached_object_alloc;
+};
 
-static const struct cached_object *find_cached_object(const struct object_id *oid)
+static const struct cached_object *find_cached_object(struct raw_object_store *object_store,
+						      const struct object_id *oid)
 {
 	static const struct cached_object empty_tree = {
 		.type = OBJ_TREE,
 		.buf = "",
 	};
-	int i;
-	const struct cached_object_entry *co = cached_objects;
+	const struct cached_object_entry *co = object_store->cached_objects;
 
-	for (i = 0; i < cached_object_nr; i++, co++) {
+	for (size_t i = 0; i < object_store->cached_object_nr; i++, co++)
 		if (oideq(&co->oid, oid))
 			return &co->value;
-	}
-	if (oideq(oid, the_hash_algo->empty_tree))
+
+	if (oid->algo && oideq(oid, hash_algos[oid->algo].empty_tree))
 		return &empty_tree;
+
 	return NULL;
 }
 
@@ -650,7 +650,7 @@ static int do_oid_object_info_extended(struct repository *r,
 	if (!oi)
 		oi = &blank_oi;
 
-	co = find_cached_object(real);
+	co = find_cached_object(r->objects, real);
 	if (co) {
 		if (oi->typep)
 			*(oi->typep) = co->type;
@@ -853,18 +853,21 @@ int oid_object_info(struct repository *r,
 	return type;
 }
 
-int pretend_object_file(void *buf, unsigned long len, enum object_type type,
+int pretend_object_file(struct repository *repo,
+			void *buf, unsigned long len, enum object_type type,
 			struct object_id *oid)
 {
 	struct cached_object_entry *co;
 	char *co_buf;
 
-	hash_object_file(the_hash_algo, buf, len, type, oid);
-	if (repo_has_object_file_with_flags(the_repository, oid, OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT) ||
-	    find_cached_object(oid))
+	hash_object_file(repo->hash_algo, buf, len, type, oid);
+	if (repo_has_object_file_with_flags(repo, oid, OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT) ||
+	    find_cached_object(repo->objects, oid))
 		return 0;
-	ALLOC_GROW(cached_objects, cached_object_nr + 1, cached_object_alloc);
-	co = &cached_objects[cached_object_nr++];
+
+	ALLOC_GROW(repo->objects->cached_objects,
+		   repo->objects->cached_object_nr + 1, repo->objects->cached_object_alloc);
+	co = &repo->objects->cached_objects[repo->objects->cached_object_nr++];
 	co->value.size = len;
 	co->value.type = type;
 	co_buf = xmalloc(len);
@@ -1021,6 +1024,10 @@ void raw_object_store_clear(struct raw_object_store *o)
 	o->odb_tail = NULL;
 	o->loaded_alternates = 0;
 
+	for (size_t i = 0; i < o->cached_object_nr; i++)
+		free((char *) o->cached_objects[i].value.buf);
+	FREE_AND_NULL(o->cached_objects);
+
 	INIT_LIST_HEAD(&o->packed_git_mru);
 	close_object_store(o);
 

-- 
2.49.0.805.g082f7c87e0.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH v3 10/10] object-store: merge "object-store-ll.h" and "object-store.h"
  2025-04-15  9:38 ` [PATCH v3 00/10] Split up "object-file.c" Patrick Steinhardt
                     ` (8 preceding siblings ...)
  2025-04-15  9:38   ` [PATCH v3 09/10] object-store: remove global array of cached objects Patrick Steinhardt
@ 2025-04-15  9:38   ` Patrick Steinhardt
  2025-04-16  6:41   ` [PATCH v3 00/10] Split up "object-file.c" Elijah Newren
  10 siblings, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-15  9:38 UTC (permalink / raw)
  To: git; +Cc: Elijah Newren, Eric Sunshine, Jeff King, Junio C Hamano

The "object-store-ll.h" header has been introduced to keep transitive
header dependendcies and compile times at bay. Now that we have created
a new "object-store.c" file though we can easily move the last remaining
additional bit of "object-store.h", the `odb_path_map`, out of the
header.

Do so. As the "object-store.h" header is now equivalent to its low-level
alternative we drop the latter and inline it into the former.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 apply.c                       |   2 +-
 archive-tar.c                 |   2 +-
 archive-zip.c                 |   2 +-
 archive.c                     |   2 +-
 attr.c                        |   2 +-
 bisect.c                      |   2 +-
 blame.c                       |   2 +-
 builtin/backfill.c            |   2 +-
 builtin/blame.c               |   2 +-
 builtin/cat-file.c            |   2 +-
 builtin/checkout.c            |   2 +-
 builtin/clone.c               |   2 +-
 builtin/commit-graph.c        |   2 +-
 builtin/commit-tree.c         |   2 +-
 builtin/count-objects.c       |   2 +-
 builtin/describe.c            |   2 +-
 builtin/difftool.c            |   2 +-
 builtin/fast-export.c         |   2 +-
 builtin/fast-import.c         |   2 +-
 builtin/fetch.c               |   2 +-
 builtin/fsck.c                |   2 +-
 builtin/gc.c                  |   2 +-
 builtin/grep.c                |   2 +-
 builtin/hash-object.c         |   2 +-
 builtin/index-pack.c          |   2 +-
 builtin/log.c                 |   2 +-
 builtin/ls-tree.c             |   2 +-
 builtin/merge-tree.c          |   2 +-
 builtin/mktag.c               |   2 +-
 builtin/mktree.c              |   2 +-
 builtin/multi-pack-index.c    |   2 +-
 builtin/notes.c               |   2 +-
 builtin/pack-objects.c        |   2 +-
 builtin/pack-redundant.c      |   2 +-
 builtin/prune.c               |   2 +-
 builtin/receive-pack.c        |   2 +-
 builtin/remote.c              |   2 +-
 builtin/repack.c              |   2 +-
 builtin/replace.c             |   2 +-
 builtin/rev-list.c            |   2 +-
 builtin/show-ref.c            |   2 +-
 builtin/submodule--helper.c   |   2 +-
 builtin/tag.c                 |   2 +-
 builtin/unpack-file.c         |   2 +-
 builtin/unpack-objects.c      |   2 +-
 bulk-checkin.c                |   2 +-
 bundle-uri.c                  |   2 +-
 bundle.c                      |   2 +-
 cache-tree.c                  |   2 +-
 combine-diff.c                |   2 +-
 commit-graph.c                |   2 +-
 commit-graph.h                |   2 +-
 commit.c                      |   2 +-
 config.c                      |   2 +-
 connected.c                   |   2 +-
 convert.c                     |   2 +-
 diagnose.c                    |   2 +-
 diff.c                        |   2 +-
 diffcore-rename.c             |   2 +-
 dir.c                         |   2 +-
 entry.c                       |   2 +-
 fetch-pack.c                  |   2 +-
 fmt-merge-msg.c               |   2 +-
 fsck.c                        |   2 +-
 grep.c                        |   2 +-
 http-backend.c                |   2 +-
 http-push.c                   |   2 +-
 http-walker.c                 |   2 +-
 http.c                        |   2 +-
 list-objects-filter.c         |   2 +-
 list-objects.c                |   2 +-
 log-tree.c                    |   2 +-
 mailmap.c                     |   2 +-
 merge-blobs.c                 |   2 +-
 merge-recursive.c             |   2 +-
 notes-merge.c                 |   2 +-
 object-file.c                 |   1 +
 object-name.c                 |   2 +-
 object-store-ll.h             | 517 ------------------------------------------
 object-store.c                |   5 +
 object-store.h                | 516 ++++++++++++++++++++++++++++++++++++++++-
 oss-fuzz/fuzz-pack-idx.c      |   2 +-
 pack-bitmap-write.c           |   2 +-
 pack-bitmap.c                 |   2 +-
 pack-check.c                  |   2 +-
 pack-mtimes.c                 |   2 +-
 pack-objects.h                |   2 +-
 pack-revindex.c               |   2 +-
 packfile.c                    |   2 +-
 path.c                        |   2 +-
 promisor-remote.c             |   2 +-
 protocol-caps.c               |   2 +-
 prune-packed.c                |   2 +-
 reachable.c                   |   2 +-
 read-cache.c                  |   2 +-
 ref-filter.c                  |   2 +-
 reflog.c                      |   2 +-
 refs.c                        |   2 +-
 remote.c                      |   2 +-
 replace-object.c              |   2 +-
 replace-object.h              |   2 +-
 repository.c                  |   2 +-
 rerere.c                      |   2 +-
 revision.c                    |   2 +-
 send-pack.c                   |   2 +-
 sequencer.c                   |   2 +-
 server-info.c                 |   2 +-
 shallow.c                     |   2 +-
 streaming.c                   |   2 +-
 submodule-config.c            |   2 +-
 submodule.c                   |   2 +-
 t/helper/test-pack-mtimes.c   |   2 +-
 t/helper/test-partial-clone.c |   2 +-
 t/helper/test-read-graph.c    |   2 +-
 t/helper/test-read-midx.c     |   2 +-
 t/helper/test-ref-store.c     |   2 +-
 tag.c                         |   2 +-
 tmp-objdir.c                  |   2 +-
 tree-walk.c                   |   2 +-
 tree.c                        |   2 +-
 unpack-trees.c                |   2 +-
 upload-pack.c                 |   2 +-
 walker.c                      |   2 +-
 xdiff-interface.c             |   2 +-
 124 files changed, 637 insertions(+), 642 deletions(-)

diff --git a/apply.c b/apply.c
index f274a379487..2b6f4d0af87 100644
--- a/apply.c
+++ b/apply.c
@@ -14,7 +14,7 @@
 #include "abspath.h"
 #include "base85.h"
 #include "config.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "delta.h"
 #include "diff.h"
 #include "dir.h"
diff --git a/archive-tar.c b/archive-tar.c
index 0edf13fba75..282b48196f9 100644
--- a/archive-tar.c
+++ b/archive-tar.c
@@ -11,7 +11,7 @@
 #include "hex.h"
 #include "tar.h"
 #include "archive.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "strbuf.h"
 #include "streaming.h"
 #include "run-command.h"
diff --git a/archive-zip.c b/archive-zip.c
index 9f32730181b..405da6f3d83 100644
--- a/archive-zip.c
+++ b/archive-zip.c
@@ -12,7 +12,7 @@
 #include "hex.h"
 #include "streaming.h"
 #include "utf8.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "strbuf.h"
 #include "userdiff.h"
 #include "write-or-die.h"
diff --git a/archive.c b/archive.c
index c95e3981524..014c312178c 100644
--- a/archive.c
+++ b/archive.c
@@ -14,7 +14,7 @@
 #include "pretty.h"
 #include "setup.h"
 #include "refs.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tree.h"
 #include "tree-walk.h"
diff --git a/attr.c b/attr.c
index 0bd2750528f..86b6109fc4e 100644
--- a/attr.c
+++ b/attr.c
@@ -22,7 +22,7 @@
 #include "read-cache-ll.h"
 #include "refs.h"
 #include "revision.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "setup.h"
 #include "thread-utils.h"
 #include "tree-walk.h"
diff --git a/bisect.c b/bisect.c
index 269a98bf978..a327468c75b 100644
--- a/bisect.c
+++ b/bisect.c
@@ -20,7 +20,7 @@
 #include "commit-slab.h"
 #include "commit-reach.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "dir.h"
 
diff --git a/blame.c b/blame.c
index b7c5bd692e6..57daa45e899 100644
--- a/blame.c
+++ b/blame.c
@@ -3,7 +3,7 @@
 
 #include "git-compat-util.h"
 #include "refs.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "cache-tree.h"
 #include "mergesort.h"
 #include "commit.h"
diff --git a/builtin/backfill.c b/builtin/backfill.c
index 33e1ea2f84f..aaa104bc91d 100644
--- a/builtin/backfill.c
+++ b/builtin/backfill.c
@@ -13,7 +13,7 @@
 #include "tree.h"
 #include "tree-walk.h"
 #include "object.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "oidset.h"
 #include "promisor-remote.h"
diff --git a/builtin/blame.c b/builtin/blame.c
index c470654c7ec..4e156bfd19d 100644
--- a/builtin/blame.c
+++ b/builtin/blame.c
@@ -28,7 +28,7 @@
 #include "line-log.h"
 #include "progress.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pager.h"
 #include "blame.h"
 #include "refs.h"
diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index b13561cf73b..c870fde260a 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -22,7 +22,7 @@
 #include "packfile.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "replace-object.h"
 #include "promisor-remote.h"
 #include "mailmap.h"
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 3e68623838a..8136962e2b3 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -20,7 +20,7 @@
 #include "merge-recursive.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "parse-options.h"
 #include "path.h"
 #include "preload-index.h"
diff --git a/builtin/clone.c b/builtin/clone.c
index 31f2198c1b3..f64be7e1e15 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -25,7 +25,7 @@
 #include "refs.h"
 #include "refspec.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "tree.h"
 #include "tree-walk.h"
 #include "unpack-trees.h"
diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c
index 8ca75262c59..be06d0a811b 100644
--- a/builtin/commit-graph.c
+++ b/builtin/commit-graph.c
@@ -6,7 +6,7 @@
 #include "hex.h"
 #include "parse-options.h"
 #include "commit-graph.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "progress.h"
 #include "replace-object.h"
 #include "strbuf.h"
diff --git a/builtin/commit-tree.c b/builtin/commit-tree.c
index 38457600a4e..6f9975e7a88 100644
--- a/builtin/commit-tree.c
+++ b/builtin/commit-tree.c
@@ -9,7 +9,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 #include "commit.h"
 #include "parse-options.h"
diff --git a/builtin/count-objects.c b/builtin/count-objects.c
index 1e89148ed74..0bb5360b2f2 100644
--- a/builtin/count-objects.c
+++ b/builtin/count-objects.c
@@ -12,7 +12,7 @@
 #include "parse-options.h"
 #include "quote.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 static unsigned long garbage;
 static off_t size_garbage;
diff --git a/builtin/describe.c b/builtin/describe.c
index 23df333fd04..0f87fbceef3 100644
--- a/builtin/describe.c
+++ b/builtin/describe.c
@@ -19,7 +19,7 @@
 #include "setup.h"
 #include "strvec.h"
 #include "run-command.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "list-objects.h"
 #include "commit-slab.h"
 #include "wildmatch.h"
diff --git a/builtin/difftool.c b/builtin/difftool.c
index 8292aedaaf0..f17a55b3cf9 100644
--- a/builtin/difftool.c
+++ b/builtin/difftool.c
@@ -30,7 +30,7 @@
 #include "strbuf.h"
 #include "lockfile.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "dir.h"
 #include "entry.h"
 #include "setup.h"
diff --git a/builtin/fast-export.c b/builtin/fast-export.c
index 170126d41ac..afacd228b5d 100644
--- a/builtin/fast-export.c
+++ b/builtin/fast-export.c
@@ -14,7 +14,7 @@
 #include "refs.h"
 #include "refspec.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "object.h"
 #include "tag.h"
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index e4523cc6f1b..c1e198f4e34 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -24,7 +24,7 @@
 #include "packfile.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "mem-pool.h"
 #include "commit-reach.h"
 #include "khash.h"
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 02af5054690..b52a32a5e0d 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -14,7 +14,7 @@
 #include "refs.h"
 #include "refspec.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oidset.h"
 #include "oid-array.h"
 #include "commit.h"
diff --git a/builtin/fsck.c b/builtin/fsck.c
index 92312b59591..6cac28356ce 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -17,7 +17,7 @@
 #include "packfile.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "read-cache-ll.h"
 #include "replace-object.h"
diff --git a/builtin/gc.c b/builtin/gc.c
index dae1e545514..4247e0e7f7c 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -28,7 +28,7 @@
 #include "commit.h"
 #include "commit-graph.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pack.h"
 #include "pack-objects.h"
 #include "path.h"
diff --git a/builtin/grep.c b/builtin/grep.c
index 283d64cab80..bcfbe5be5ba 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -26,7 +26,7 @@
 #include "submodule-config.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 #include "pager.h"
 #include "path.h"
diff --git a/builtin/hash-object.c b/builtin/hash-object.c
index e7c0d6afdef..cd53fa3bde8 100644
--- a/builtin/hash-object.c
+++ b/builtin/hash-object.c
@@ -11,7 +11,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "blob.h"
 #include "quote.h"
 #include "parse-options.h"
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index de127c0ff13..60a8ee05dbc 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -21,7 +21,7 @@
 #include "packfile.h"
 #include "pack-revindex.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "oidset.h"
 #include "path.h"
diff --git a/builtin/log.c b/builtin/log.c
index 516c9ec8400..b450cd3bde8 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -15,7 +15,7 @@
 #include "hex.h"
 #include "refs.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pager.h"
 #include "color.h"
 #include "commit.h"
diff --git a/builtin/ls-tree.c b/builtin/ls-tree.c
index 8542b5d53e4..8aafc30ca48 100644
--- a/builtin/ls-tree.c
+++ b/builtin/ls-tree.c
@@ -10,7 +10,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "tree.h"
 #include "path.h"
 #include "quote.h"
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 3ec7127b3a6..4aafa73c615 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -10,7 +10,7 @@
 #include "commit-reach.h"
 #include "merge-ort.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "parse-options.h"
 #include "blob.h"
 #include "merge-blobs.h"
diff --git a/builtin/mktag.c b/builtin/mktag.c
index 6e188dce50a..7ac11c46d53 100644
--- a/builtin/mktag.c
+++ b/builtin/mktag.c
@@ -6,7 +6,7 @@
 #include "strbuf.h"
 #include "replace-object.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "fsck.h"
 #include "config.h"
 
diff --git a/builtin/mktree.c b/builtin/mktree.c
index 0644f951161..7ffe6eefd8a 100644
--- a/builtin/mktree.c
+++ b/builtin/mktree.c
@@ -12,7 +12,7 @@
 #include "tree.h"
 #include "parse-options.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 static struct treeent {
 	unsigned mode;
diff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c
index 2a938466f53..d98410ca6c6 100644
--- a/builtin/multi-pack-index.c
+++ b/builtin/multi-pack-index.c
@@ -7,7 +7,7 @@
 #include "midx.h"
 #include "strbuf.h"
 #include "trace2.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "replace-object.h"
 #include "repository.h"
 
diff --git a/builtin/notes.c b/builtin/notes.c
index 0dbc233752d..a3f433ca4c0 100644
--- a/builtin/notes.c
+++ b/builtin/notes.c
@@ -16,7 +16,7 @@
 #include "notes.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 
 #include "pretty.h"
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 163aab547fe..488c80f2cf3 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -32,7 +32,7 @@
 #include "list.h"
 #include "packfile.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "replace-object.h"
 #include "dir.h"
 #include "midx.h"
diff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c
index 3febe732f8e..5d1fc781761 100644
--- a/builtin/pack-redundant.c
+++ b/builtin/pack-redundant.c
@@ -13,7 +13,7 @@
 #include "hex.h"
 
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "strbuf.h"
 
 #define BLKSIZE 512
diff --git a/builtin/prune.c b/builtin/prune.c
index 8f52da8bd66..e930caa0c0a 100644
--- a/builtin/prune.c
+++ b/builtin/prune.c
@@ -17,7 +17,7 @@
 #include "replace-object.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "shallow.h"
 
 static const char * const prune_usage[] = {
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index ee51bd76f60..be314879e82 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -33,7 +33,7 @@
 #include "packfile.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "protocol.h"
 #include "commit-reach.h"
diff --git a/builtin/remote.c b/builtin/remote.c
index 1b7aad88380..59481b3a82c 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -14,7 +14,7 @@
 #include "rebase.h"
 #include "refs.h"
 #include "refspec.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "strvec.h"
 #include "commit-reach.h"
 #include "progress.h"
diff --git a/builtin/repack.c b/builtin/repack.c
index f3330ade7b8..1fd2874324a 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -17,7 +17,7 @@
 #include "midx.h"
 #include "packfile.h"
 #include "prune-packed.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "promisor-remote.h"
 #include "shallow.h"
 #include "pack.h"
diff --git a/builtin/replace.c b/builtin/replace.c
index 2b4fc9a68b3..48c7c6a2d56 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -19,7 +19,7 @@
 #include "run-command.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "replace-object.h"
 #include "tag.h"
 #include "wildmatch.h"
diff --git a/builtin/rev-list.c b/builtin/rev-list.c
index bb26bee0d45..0170d79b62f 100644
--- a/builtin/rev-list.c
+++ b/builtin/rev-list.c
@@ -14,7 +14,7 @@
 #include "object.h"
 #include "object-name.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pack-bitmap.h"
 #include "log-tree.h"
 #include "graph.h"
diff --git a/builtin/show-ref.c b/builtin/show-ref.c
index 285cd3e4338..f81209f23c3 100644
--- a/builtin/show-ref.c
+++ b/builtin/show-ref.c
@@ -5,7 +5,7 @@
 #include "hex.h"
 #include "refs/refs-internal.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "object.h"
 #include "string-list.h"
 #include "parse-options.h"
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index cc001d0b4cd..53da2116ddf 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -28,7 +28,7 @@
 #include "diff.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "advice.h"
 #include "branch.h"
 #include "list-objects-filter-options.h"
diff --git a/builtin/tag.c b/builtin/tag.c
index 536a01ff3ae..e6b372cebf5 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -19,7 +19,7 @@
 #include "refs.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "tag.h"
 #include "parse-options.h"
diff --git a/builtin/unpack-file.c b/builtin/unpack-file.c
index b19e5cabd03..e33acfc4ee4 100644
--- a/builtin/unpack-file.c
+++ b/builtin/unpack-file.c
@@ -4,7 +4,7 @@
 #include "hex.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 static char *create_temp_file(struct object_id *oid)
 {
diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c
index 4078eab9252..661be789f13 100644
--- a/builtin/unpack-objects.c
+++ b/builtin/unpack-objects.c
@@ -9,7 +9,7 @@
 #include "git-zlib.h"
 #include "hex.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "object.h"
 #include "delta.h"
 #include "pack.h"
diff --git a/bulk-checkin.c b/bulk-checkin.c
index 309201a76a6..684d49023af 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -18,7 +18,7 @@
 #include "tmp-objdir.h"
 #include "packfile.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 static int odb_transaction_nesting;
 
diff --git a/bundle-uri.c b/bundle-uri.c
index 744257c49c1..96d2ba726d9 100644
--- a/bundle-uri.c
+++ b/bundle-uri.c
@@ -14,7 +14,7 @@
 #include "fetch-pack.h"
 #include "remote.h"
 #include "trace2.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 static struct {
 	enum bundle_list_heuristic heuristic;
diff --git a/bundle.c b/bundle.c
index d7ad6908433..d661c4ec214 100644
--- a/bundle.c
+++ b/bundle.c
@@ -7,7 +7,7 @@
 #include "environment.h"
 #include "gettext.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "repository.h"
 #include "object.h"
 #include "commit.h"
diff --git a/cache-tree.c b/cache-tree.c
index 4c8167ea927..c0e1e9ee1d4 100644
--- a/cache-tree.c
+++ b/cache-tree.c
@@ -10,7 +10,7 @@
 #include "cache-tree.h"
 #include "bulk-checkin.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "read-cache-ll.h"
 #include "replace-object.h"
 #include "repository.h"
diff --git a/combine-diff.c b/combine-diff.c
index 553bf59fed6..dfae9f7995d 100644
--- a/combine-diff.c
+++ b/combine-diff.c
@@ -2,7 +2,7 @@
 #define DISABLE_SIGN_COMPARE_WARNINGS
 
 #include "git-compat-util.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "convert.h"
 #include "diff.h"
diff --git a/commit-graph.c b/commit-graph.c
index 9621c454972..6394752b0b0 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -13,7 +13,7 @@
 #include "refs.h"
 #include "hash-lookup.h"
 #include "commit-graph.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "path.h"
 #include "alloc.h"
diff --git a/commit-graph.h b/commit-graph.h
index 67819401954..13f662827d4 100644
--- a/commit-graph.h
+++ b/commit-graph.h
@@ -1,7 +1,7 @@
 #ifndef COMMIT_GRAPH_H
 #define COMMIT_GRAPH_H
 
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oidset.h"
 
 #define GIT_TEST_COMMIT_GRAPH "GIT_TEST_COMMIT_GRAPH"
diff --git a/commit.c b/commit.c
index fbf4f8e87fd..00842678bd2 100644
--- a/commit.c
+++ b/commit.c
@@ -9,7 +9,7 @@
 #include "hex.h"
 #include "repository.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "utf8.h"
 #include "diff.h"
 #include "revision.h"
diff --git a/config.c b/config.c
index accb47e2d18..b18b5617fcd 100644
--- a/config.c
+++ b/config.c
@@ -31,7 +31,7 @@
 #include "hashmap.h"
 #include "string-list.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pager.h"
 #include "path.h"
 #include "utf8.h"
diff --git a/connected.c b/connected.c
index 3099da84f33..4415388beba 100644
--- a/connected.c
+++ b/connected.c
@@ -3,7 +3,7 @@
 #include "git-compat-util.h"
 #include "gettext.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "run-command.h"
 #include "sigchain.h"
 #include "connected.h"
diff --git a/convert.c b/convert.c
index 9cc0ca20ca0..8783e17941f 100644
--- a/convert.c
+++ b/convert.c
@@ -8,7 +8,7 @@
 #include "copy.h"
 #include "gettext.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "attr.h"
 #include "run-command.h"
 #include "quote.h"
diff --git a/diagnose.c b/diagnose.c
index bd485effea2..b1be74be983 100644
--- a/diagnose.c
+++ b/diagnose.c
@@ -7,7 +7,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "strvec.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 #include "parse-options.h"
 #include "repository.h"
diff --git a/diff.c b/diff.c
index 3bcf5028831..91d5b962b57 100644
--- a/diff.c
+++ b/diff.c
@@ -23,7 +23,7 @@
 #include "color.h"
 #include "run-command.h"
 #include "utf8.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "userdiff.h"
 #include "submodule.h"
 #include "hashmap.h"
diff --git a/diffcore-rename.c b/diffcore-rename.c
index 5002e896aad..787a2cef5f4 100644
--- a/diffcore-rename.c
+++ b/diffcore-rename.c
@@ -8,7 +8,7 @@
 #include "git-compat-util.h"
 #include "diff.h"
 #include "diffcore.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "hashmap.h"
 #include "mem-pool.h"
 #include "oid-array.h"
diff --git a/dir.c b/dir.c
index 49008739b9b..5c4675b4ac4 100644
--- a/dir.c
+++ b/dir.c
@@ -17,7 +17,7 @@
 #include "environment.h"
 #include "gettext.h"
 #include "name-hash.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "refs.h"
 #include "repository.h"
diff --git a/entry.c b/entry.c
index 81b321e53d1..f36ec5ad242 100644
--- a/entry.c
+++ b/entry.c
@@ -1,7 +1,7 @@
 #define USE_THE_REPOSITORY_VARIABLE
 
 #include "git-compat-util.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "dir.h"
 #include "environment.h"
 #include "gettext.h"
diff --git a/fetch-pack.c b/fetch-pack.c
index 1ed5e11dd56..210dc30d50f 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -24,7 +24,7 @@
 #include "oid-array.h"
 #include "oidset.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "connected.h"
 #include "fetch-negotiator.h"
diff --git a/fmt-merge-msg.c b/fmt-merge-msg.c
index 5b63c3b088a..501b5acdd44 100644
--- a/fmt-merge-msg.c
+++ b/fmt-merge-msg.c
@@ -6,7 +6,7 @@
 #include "environment.h"
 #include "refs.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "diff.h"
 #include "diff-merges.h"
 #include "hex.h"
diff --git a/fsck.c b/fsck.c
index 9fc4c25ffd5..8dc8472ceb3 100644
--- a/fsck.c
+++ b/fsck.c
@@ -4,7 +4,7 @@
 #include "date.h"
 #include "dir.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "repository.h"
 #include "object.h"
diff --git a/grep.c b/grep.c
index 9284b5741f5..f8d535182c3 100644
--- a/grep.c
+++ b/grep.c
@@ -5,7 +5,7 @@
 #include "gettext.h"
 #include "grep.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pretty.h"
 #include "userdiff.h"
 #include "xdiff-interface.h"
diff --git a/http-backend.c b/http-backend.c
index 50b2858fad6..0c575aa88aa 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -18,7 +18,7 @@
 #include "url.h"
 #include "strvec.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "protocol.h"
 #include "date.h"
 #include "write-or-die.h"
diff --git a/http-push.c b/http-push.c
index 806eb67cf1b..32e37565f4e 100644
--- a/http-push.c
+++ b/http-push.c
@@ -20,7 +20,7 @@
 #include "url.h"
 #include "packfile.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit-reach.h"
 
 #ifdef EXPAT_NEEDS_XMLPARSE_H
diff --git a/http-walker.c b/http-walker.c
index 7918ddc0968..882cae19c24 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -9,7 +9,7 @@
 #include "list.h"
 #include "transport.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 struct alt_base {
 	char *base;
diff --git a/http.c b/http.c
index 0c9a872809f..bc18ff83c4b 100644
--- a/http.c
+++ b/http.c
@@ -19,7 +19,7 @@
 #include "packfile.h"
 #include "string-list.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "tempfile.h"
 
 static struct trace_key trace_curl = TRACE_KEY_INIT(CURL);
diff --git a/list-objects-filter.c b/list-objects-filter.c
index dc598a081bb..7765761b3c6 100644
--- a/list-objects-filter.c
+++ b/list-objects-filter.c
@@ -12,7 +12,7 @@
 #include "oidmap.h"
 #include "oidset.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 /* Remember to update object flag allocation in object.h */
 /*
diff --git a/list-objects.c b/list-objects.c
index 943e62e868f..1e5512e1318 100644
--- a/list-objects.c
+++ b/list-objects.c
@@ -14,7 +14,7 @@
 #include "list-objects-filter.h"
 #include "list-objects-filter-options.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "trace.h"
 #include "environment.h"
 
diff --git a/log-tree.c b/log-tree.c
index 5dd1b63076f..a4d4ab59ca0 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -9,7 +9,7 @@
 #include "environment.h"
 #include "hex.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "repository.h"
 #include "tmp-objdir.h"
 #include "commit.h"
diff --git a/mailmap.c b/mailmap.c
index f35d20ed7fd..9e2642a043b 100644
--- a/mailmap.c
+++ b/mailmap.c
@@ -6,7 +6,7 @@
 #include "string-list.h"
 #include "mailmap.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "setup.h"
 
 char *git_mailmap_file;
diff --git a/merge-blobs.c b/merge-blobs.c
index 0ad0390fea5..53f36dbc175 100644
--- a/merge-blobs.c
+++ b/merge-blobs.c
@@ -4,7 +4,7 @@
 #include "merge-ll.h"
 #include "blob.h"
 #include "merge-blobs.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 static int fill_mmfile_blob(mmfile_t *f, struct blob *obj)
 {
diff --git a/merge-recursive.c b/merge-recursive.c
index f71490517e1..b852f467674 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -27,7 +27,7 @@
 #include "name-hash.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "repository.h"
 #include "revision.h"
diff --git a/notes-merge.c b/notes-merge.c
index 520b92942cd..dae8e6a281a 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -8,7 +8,7 @@
 #include "refs.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "repository.h"
 #include "diff.h"
diff --git a/object-file.c b/object-file.c
index 6084d603136..450e1cae346 100644
--- a/object-file.c
+++ b/object-file.c
@@ -13,6 +13,7 @@
 #include "git-compat-util.h"
 #include "bulk-checkin.h"
 #include "convert.h"
+#include "dir.h"
 #include "environment.h"
 #include "fsck.h"
 #include "gettext.h"
diff --git a/object-name.c b/object-name.c
index 91f731373a1..2c751a5352a 100644
--- a/object-name.c
+++ b/object-name.c
@@ -19,7 +19,7 @@
 #include "oidtree.h"
 #include "packfile.h"
 #include "pretty.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "read-cache-ll.h"
 #include "repo-settings.h"
 #include "repository.h"
diff --git a/object-store-ll.h b/object-store-ll.h
deleted file mode 100644
index bb5e8798a1b..00000000000
--- a/object-store-ll.h
+++ /dev/null
@@ -1,517 +0,0 @@
-#ifndef OBJECT_STORE_LL_H
-#define OBJECT_STORE_LL_H
-
-#include "hashmap.h"
-#include "object.h"
-#include "list.h"
-#include "thread-utils.h"
-#include "oidset.h"
-
-struct oidmap;
-struct oidtree;
-struct strbuf;
-struct repository;
-
-struct object_directory {
-	struct object_directory *next;
-
-	/*
-	 * Used to store the results of readdir(3) calls when we are OK
-	 * sacrificing accuracy due to races for speed. That includes
-	 * object existence with OBJECT_INFO_QUICK, as well as
-	 * our search for unique abbreviated hashes. Don't use it for tasks
-	 * requiring greater accuracy!
-	 *
-	 * Be sure to call odb_load_loose_cache() before using.
-	 */
-	uint32_t loose_objects_subdir_seen[8]; /* 256 bits */
-	struct oidtree *loose_objects_cache;
-
-	/* Map between object IDs for loose objects. */
-	struct loose_object_map *loose_map;
-
-	/*
-	 * This is a temporary object store created by the tmp_objdir
-	 * facility. Disable ref updates since the objects in the store
-	 * might be discarded on rollback.
-	 */
-	int disable_ref_updates;
-
-	/*
-	 * This object store is ephemeral, so there is no need to fsync.
-	 */
-	int will_destroy;
-
-	/*
-	 * Path to the alternative object store. If this is a relative path,
-	 * it is relative to the current working directory.
-	 */
-	char *path;
-};
-
-void prepare_alt_odb(struct repository *r);
-int has_alt_odb(struct repository *r);
-char *compute_alternate_path(const char *path, struct strbuf *err);
-struct object_directory *find_odb(struct repository *r, const char *obj_dir);
-typedef int alt_odb_fn(struct object_directory *, void *);
-int foreach_alt_odb(alt_odb_fn, void*);
-typedef void alternate_ref_fn(const struct object_id *oid, void *);
-void for_each_alternate_ref(alternate_ref_fn, void *);
-
-/*
- * Add the directory to the on-disk alternates file; the new entry will also
- * take effect in the current process.
- */
-void add_to_alternates_file(const char *dir);
-
-/*
- * Add the directory to the in-memory list of alternates (along with any
- * recursive alternates it points to), but do not modify the on-disk alternates
- * file.
- */
-void add_to_alternates_memory(const char *dir);
-
-/*
- * Replace the current writable object directory with the specified temporary
- * object directory; returns the former primary object directory.
- */
-struct object_directory *set_temporary_primary_odb(const char *dir, int will_destroy);
-
-/*
- * Restore a previous ODB replaced by set_temporary_main_odb.
- */
-void restore_primary_odb(struct object_directory *restore_odb, const char *old_path);
-
-/*
- * Populate and return the loose object cache array corresponding to the
- * given object ID.
- */
-struct oidtree *odb_loose_cache(struct object_directory *odb,
-				  const struct object_id *oid);
-
-/* Empty the loose object cache for the specified object directory. */
-void odb_clear_loose_cache(struct object_directory *odb);
-
-struct packed_git {
-	struct hashmap_entry packmap_ent;
-	struct packed_git *next;
-	struct list_head mru;
-	struct pack_window *windows;
-	off_t pack_size;
-	const void *index_data;
-	size_t index_size;
-	uint32_t num_objects;
-	size_t crc_offset;
-	struct oidset bad_objects;
-	int index_version;
-	time_t mtime;
-	int pack_fd;
-	int index;              /* for builtin/pack-objects.c */
-	unsigned pack_local:1,
-		 pack_keep:1,
-		 pack_keep_in_core:1,
-		 freshened:1,
-		 do_not_close:1,
-		 pack_promisor:1,
-		 multi_pack_index:1,
-		 is_cruft:1;
-	unsigned char hash[GIT_MAX_RAWSZ];
-	struct revindex_entry *revindex;
-	const uint32_t *revindex_data;
-	const uint32_t *revindex_map;
-	size_t revindex_size;
-	/*
-	 * mtimes_map points at the beginning of the memory mapped region of
-	 * this pack's corresponding .mtimes file, and mtimes_size is the size
-	 * of that .mtimes file
-	 */
-	const uint32_t *mtimes_map;
-	size_t mtimes_size;
-
-	/* repo denotes the repository this packfile belongs to */
-	struct repository *repo;
-
-	/* something like ".git/objects/pack/xxxxx.pack" */
-	char pack_name[FLEX_ARRAY]; /* more */
-};
-
-struct multi_pack_index;
-
-static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
-				     const struct hashmap_entry *entry,
-				     const struct hashmap_entry *entry2,
-				     const void *keydata)
-{
-	const char *key = keydata;
-	const struct packed_git *pg1, *pg2;
-
-	pg1 = container_of(entry, const struct packed_git, packmap_ent);
-	pg2 = container_of(entry2, const struct packed_git, packmap_ent);
-
-	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
-}
-
-struct cached_object_entry;
-
-struct raw_object_store {
-	/*
-	 * Set of all object directories; the main directory is first (and
-	 * cannot be NULL after initialization). Subsequent directories are
-	 * alternates.
-	 */
-	struct object_directory *odb;
-	struct object_directory **odb_tail;
-	struct kh_odb_path_map *odb_by_path;
-
-	int loaded_alternates;
-
-	/*
-	 * A list of alternate object directories loaded from the environment;
-	 * this should not generally need to be accessed directly, but will
-	 * populate the "odb" list when prepare_alt_odb() is run.
-	 */
-	char *alternate_db;
-
-	/*
-	 * Objects that should be substituted by other objects
-	 * (see git-replace(1)).
-	 */
-	struct oidmap *replace_map;
-	unsigned replace_map_initialized : 1;
-	pthread_mutex_t replace_mutex; /* protect object replace functions */
-
-	struct commit_graph *commit_graph;
-	unsigned commit_graph_attempted : 1; /* if loading has been attempted */
-
-	/*
-	 * private data
-	 *
-	 * should only be accessed directly by packfile.c and midx.c
-	 */
-	struct multi_pack_index *multi_pack_index;
-
-	/*
-	 * private data
-	 *
-	 * should only be accessed directly by packfile.c
-	 */
-
-	struct packed_git *packed_git;
-	/* A most-recently-used ordered version of the packed_git list. */
-	struct list_head packed_git_mru;
-
-	struct {
-		struct packed_git **packs;
-		unsigned flags;
-	} kept_pack_cache;
-
-	/*
-	 * This is meant to hold a *small* number of objects that you would
-	 * want repo_read_object_file() to be able to return, but yet you do not want
-	 * to write them into the object store (e.g. a browse-only
-	 * application).
-	 */
-	struct cached_object_entry *cached_objects;
-	size_t cached_object_nr, cached_object_alloc;
-
-	/*
-	 * A map of packfiles to packed_git structs for tracking which
-	 * packs have been loaded already.
-	 */
-	struct hashmap pack_map;
-
-	/*
-	 * A fast, rough count of the number of objects in the repository.
-	 * These two fields are not meant for direct access. Use
-	 * repo_approximate_object_count() instead.
-	 */
-	unsigned long approximate_object_count;
-	unsigned approximate_object_count_valid : 1;
-
-	/*
-	 * Whether packed_git has already been populated with this repository's
-	 * packs.
-	 */
-	unsigned packed_git_initialized : 1;
-};
-
-struct raw_object_store *raw_object_store_new(void);
-void raw_object_store_clear(struct raw_object_store *o);
-
-/*
- * Create a temporary file rooted in the object database directory, or
- * die on failure. The filename is taken from "pattern", which should have the
- * usual "XXXXXX" trailer, and the resulting filename is written into the
- * "template" buffer. Returns the open descriptor.
- */
-int odb_mkstemp(struct strbuf *temp_filename, const char *pattern);
-
-/*
- * Create a pack .keep file named "name" (which should generally be the output
- * of odb_pack_name). Returns a file descriptor opened for writing, or -1 on
- * error.
- */
-int odb_pack_keep(const char *name);
-
-/*
- * Put in `buf` the name of the file in the local object database that
- * would be used to store a loose object with the specified oid.
- */
-const char *loose_object_path(struct repository *r, struct strbuf *buf,
-			      const struct object_id *oid);
-
-void *map_loose_object(struct repository *r, const struct object_id *oid,
-		       unsigned long *size);
-
-void *repo_read_object_file(struct repository *r,
-			    const struct object_id *oid,
-			    enum object_type *type,
-			    unsigned long *size);
-
-/* Read and unpack an object file into memory, write memory to an object file */
-int oid_object_info(struct repository *r, const struct object_id *, unsigned long *);
-
-void hash_object_file(const struct git_hash_algo *algo, const void *buf,
-		      unsigned long len, enum object_type type,
-		      struct object_id *oid);
-
-/*
- * Add an object file to the in-memory object store, without writing it
- * to disk.
- *
- * Callers are responsible for calling write_object_file to record the
- * object in persistent storage before writing any other new objects
- * that reference it.
- */
-int pretend_object_file(struct repository *repo,
-			void *buf, unsigned long len, enum object_type type,
-			struct object_id *oid);
-
-struct object_info {
-	/* Request */
-	enum object_type *typep;
-	unsigned long *sizep;
-	off_t *disk_sizep;
-	struct object_id *delta_base_oid;
-	struct strbuf *type_name;
-	void **contentp;
-
-	/* Response */
-	enum {
-		OI_CACHED,
-		OI_LOOSE,
-		OI_PACKED,
-		OI_DBCACHED
-	} whence;
-	union {
-		/*
-		 * struct {
-		 * 	... Nothing to expose in this case
-		 * } cached;
-		 * struct {
-		 * 	... Nothing to expose in this case
-		 * } loose;
-		 */
-		struct {
-			struct packed_git *pack;
-			off_t offset;
-			unsigned int is_delta;
-		} packed;
-	} u;
-};
-
-/*
- * Initializer for a "struct object_info" that wants no items. You may
- * also memset() the memory to all-zeroes.
- */
-#define OBJECT_INFO_INIT { 0 }
-
-/* Invoke lookup_replace_object() on the given hash */
-#define OBJECT_INFO_LOOKUP_REPLACE 1
-/* Allow reading from a loose object file of unknown/bogus type */
-#define OBJECT_INFO_ALLOW_UNKNOWN_TYPE 2
-/* Do not retry packed storage after checking packed and loose storage */
-#define OBJECT_INFO_QUICK 8
-/*
- * Do not attempt to fetch the object if missing (even if fetch_is_missing is
- * nonzero).
- */
-#define OBJECT_INFO_SKIP_FETCH_OBJECT 16
-/*
- * This is meant for bulk prefetching of missing blobs in a partial
- * clone. Implies OBJECT_INFO_SKIP_FETCH_OBJECT and OBJECT_INFO_QUICK
- */
-#define OBJECT_INFO_FOR_PREFETCH (OBJECT_INFO_SKIP_FETCH_OBJECT | OBJECT_INFO_QUICK)
-
-/* Die if object corruption (not just an object being missing) was detected. */
-#define OBJECT_INFO_DIE_IF_CORRUPT 32
-
-int oid_object_info_extended(struct repository *r,
-			     const struct object_id *,
-			     struct object_info *, unsigned flags);
-
-/* Retry packed storage after checking packed and loose storage */
-#define HAS_OBJECT_RECHECK_PACKED 1
-
-/*
- * Returns 1 if the object exists. This function will not lazily fetch objects
- * in a partial clone.
- */
-int has_object(struct repository *r, const struct object_id *oid,
-	       unsigned flags);
-
-/*
- * These macros and functions are deprecated. If checking existence for an
- * object that is likely to be missing and/or whose absence is relatively
- * inconsequential (or is consequential but the caller is prepared to handle
- * it), use has_object(), which has better defaults (no lazy fetch in a partial
- * clone and no rechecking of packed storage). In the unlikely event that a
- * caller needs to assert existence of an object that it fully expects to
- * exist, and wants to trigger a lazy fetch in a partial clone, use
- * oid_object_info_extended() with a NULL struct object_info.
- *
- * These functions can be removed once all callers have migrated to
- * has_object() and/or oid_object_info_extended().
- */
-int repo_has_object_file(struct repository *r, const struct object_id *oid);
-int repo_has_object_file_with_flags(struct repository *r,
-				    const struct object_id *oid, int flags);
-
-void assert_oid_type(const struct object_id *oid, enum object_type expect);
-
-/*
- * Enabling the object read lock allows multiple threads to safely call the
- * following functions in parallel: repo_read_object_file(),
- * read_object_with_reference(), oid_object_info() and oid_object_info_extended().
- *
- * obj_read_lock() and obj_read_unlock() may also be used to protect other
- * section which cannot execute in parallel with object reading. Since the used
- * lock is a recursive mutex, these sections can even contain calls to object
- * reading functions. However, beware that in these cases zlib inflation won't
- * be performed in parallel, losing performance.
- *
- * TODO: oid_object_info_extended()'s call stack has a recursive behavior. If
- * any of its callees end up calling it, this recursive call won't benefit from
- * parallel inflation.
- */
-void enable_obj_read_lock(void);
-void disable_obj_read_lock(void);
-
-extern int obj_read_use_lock;
-extern pthread_mutex_t obj_read_mutex;
-
-static inline void obj_read_lock(void)
-{
-	if(obj_read_use_lock)
-		pthread_mutex_lock(&obj_read_mutex);
-}
-
-static inline void obj_read_unlock(void)
-{
-	if(obj_read_use_lock)
-		pthread_mutex_unlock(&obj_read_mutex);
-}
-
-/*
- * Iterate over the files in the loose-object parts of the object
- * directory "path", triggering the following callbacks:
- *
- *  - loose_object is called for each loose object we find.
- *
- *  - loose_cruft is called for any files that do not appear to be
- *    loose objects. Note that we only look in the loose object
- *    directories "objects/[0-9a-f]{2}/", so we will not report
- *    "objects/foobar" as cruft.
- *
- *  - loose_subdir is called for each top-level hashed subdirectory
- *    of the object directory (e.g., "$OBJDIR/f0"). It is called
- *    after the objects in the directory are processed.
- *
- * Any callback that is NULL will be ignored. Callbacks returning non-zero
- * will end the iteration.
- *
- * In the "buf" variant, "path" is a strbuf which will also be used as a
- * scratch buffer, but restored to its original contents before
- * the function returns.
- */
-typedef int each_loose_object_fn(const struct object_id *oid,
-				 const char *path,
-				 void *data);
-typedef int each_loose_cruft_fn(const char *basename,
-				const char *path,
-				void *data);
-typedef int each_loose_subdir_fn(unsigned int nr,
-				 const char *path,
-				 void *data);
-int for_each_file_in_obj_subdir(unsigned int subdir_nr,
-				struct strbuf *path,
-				each_loose_object_fn obj_cb,
-				each_loose_cruft_fn cruft_cb,
-				each_loose_subdir_fn subdir_cb,
-				void *data);
-int for_each_loose_file_in_objdir(const char *path,
-				  each_loose_object_fn obj_cb,
-				  each_loose_cruft_fn cruft_cb,
-				  each_loose_subdir_fn subdir_cb,
-				  void *data);
-int for_each_loose_file_in_objdir_buf(struct strbuf *path,
-				      each_loose_object_fn obj_cb,
-				      each_loose_cruft_fn cruft_cb,
-				      each_loose_subdir_fn subdir_cb,
-				      void *data);
-
-/* Flags for for_each_*_object() below. */
-enum for_each_object_flags {
-	/* Iterate only over local objects, not alternates. */
-	FOR_EACH_OBJECT_LOCAL_ONLY = (1<<0),
-
-	/* Only iterate over packs obtained from the promisor remote. */
-	FOR_EACH_OBJECT_PROMISOR_ONLY = (1<<1),
-
-	/*
-	 * Visit objects within a pack in packfile order rather than .idx order
-	 */
-	FOR_EACH_OBJECT_PACK_ORDER = (1<<2),
-
-	/* Only iterate over packs that are not marked as kept in-core. */
-	FOR_EACH_OBJECT_SKIP_IN_CORE_KEPT_PACKS = (1<<3),
-
-	/* Only iterate over packs that do not have .keep files. */
-	FOR_EACH_OBJECT_SKIP_ON_DISK_KEPT_PACKS = (1<<4),
-};
-
-/*
- * Iterate over all accessible loose objects without respect to
- * reachability. By default, this includes both local and alternate objects.
- * The order in which objects are visited is unspecified.
- *
- * Any flags specific to packs are ignored.
- */
-int for_each_loose_object(each_loose_object_fn, void *,
-			  enum for_each_object_flags flags);
-
-/*
- * Iterate over all accessible packed objects without respect to reachability.
- * By default, this includes both local and alternate packs.
- *
- * Note that some objects may appear twice if they are found in multiple packs.
- * Each pack is visited in an unspecified order. By default, objects within a
- * pack are visited in pack-idx order (i.e., sorted by oid).
- */
-typedef int each_packed_object_fn(const struct object_id *oid,
-				  struct packed_git *pack,
-				  uint32_t pos,
-				  void *data);
-int for_each_object_in_pack(struct packed_git *p,
-			    each_packed_object_fn, void *data,
-			    enum for_each_object_flags flags);
-int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
-			   void *data, enum for_each_object_flags flags);
-
-void *read_object_with_reference(struct repository *r,
-				 const struct object_id *oid,
-				 enum object_type required_type,
-				 unsigned long *size,
-				 struct object_id *oid_ret);
-
-#endif /* OBJECT_STORE_LL_H */
diff --git a/object-store.c b/object-store.c
index 0f1dcc113ed..6ab50d25d3e 100644
--- a/object-store.c
+++ b/object-store.c
@@ -4,9 +4,11 @@
 #include "abspath.h"
 #include "commit-graph.h"
 #include "config.h"
+#include "dir.h"
 #include "environment.h"
 #include "gettext.h"
 #include "hex.h"
+#include "khash.h"
 #include "lockfile.h"
 #include "loose.h"
 #include "object-file-convert.h"
@@ -24,6 +26,9 @@
 #include "submodule.h"
 #include "write-or-die.h"
 
+KHASH_INIT(odb_path_map, const char * /* key: odb_path */,
+	struct object_directory *, 1, fspathhash, fspatheq)
+
 /*
  * This is meant to hold a *small* number of objects that you would
  * want repo_read_object_file() to be able to return, but yet you do not want
diff --git a/object-store.h b/object-store.h
index 1b3e3d7d014..46961dc9542 100644
--- a/object-store.h
+++ b/object-store.h
@@ -1,11 +1,517 @@
 #ifndef OBJECT_STORE_H
 #define OBJECT_STORE_H
 
-#include "khash.h"
-#include "dir.h"
-#include "object-store-ll.h"
+#include "hashmap.h"
+#include "object.h"
+#include "list.h"
+#include "oidset.h"
+#include "thread-utils.h"
 
-KHASH_INIT(odb_path_map, const char * /* key: odb_path */,
-	struct object_directory *, 1, fspathhash, fspatheq)
+struct oidmap;
+struct oidtree;
+struct strbuf;
+struct repository;
+
+struct object_directory {
+	struct object_directory *next;
+
+	/*
+	 * Used to store the results of readdir(3) calls when we are OK
+	 * sacrificing accuracy due to races for speed. That includes
+	 * object existence with OBJECT_INFO_QUICK, as well as
+	 * our search for unique abbreviated hashes. Don't use it for tasks
+	 * requiring greater accuracy!
+	 *
+	 * Be sure to call odb_load_loose_cache() before using.
+	 */
+	uint32_t loose_objects_subdir_seen[8]; /* 256 bits */
+	struct oidtree *loose_objects_cache;
+
+	/* Map between object IDs for loose objects. */
+	struct loose_object_map *loose_map;
+
+	/*
+	 * This is a temporary object store created by the tmp_objdir
+	 * facility. Disable ref updates since the objects in the store
+	 * might be discarded on rollback.
+	 */
+	int disable_ref_updates;
+
+	/*
+	 * This object store is ephemeral, so there is no need to fsync.
+	 */
+	int will_destroy;
+
+	/*
+	 * Path to the alternative object store. If this is a relative path,
+	 * it is relative to the current working directory.
+	 */
+	char *path;
+};
+
+void prepare_alt_odb(struct repository *r);
+int has_alt_odb(struct repository *r);
+char *compute_alternate_path(const char *path, struct strbuf *err);
+struct object_directory *find_odb(struct repository *r, const char *obj_dir);
+typedef int alt_odb_fn(struct object_directory *, void *);
+int foreach_alt_odb(alt_odb_fn, void*);
+typedef void alternate_ref_fn(const struct object_id *oid, void *);
+void for_each_alternate_ref(alternate_ref_fn, void *);
+
+/*
+ * Add the directory to the on-disk alternates file; the new entry will also
+ * take effect in the current process.
+ */
+void add_to_alternates_file(const char *dir);
+
+/*
+ * Add the directory to the in-memory list of alternates (along with any
+ * recursive alternates it points to), but do not modify the on-disk alternates
+ * file.
+ */
+void add_to_alternates_memory(const char *dir);
+
+/*
+ * Replace the current writable object directory with the specified temporary
+ * object directory; returns the former primary object directory.
+ */
+struct object_directory *set_temporary_primary_odb(const char *dir, int will_destroy);
+
+/*
+ * Restore a previous ODB replaced by set_temporary_main_odb.
+ */
+void restore_primary_odb(struct object_directory *restore_odb, const char *old_path);
+
+/*
+ * Populate and return the loose object cache array corresponding to the
+ * given object ID.
+ */
+struct oidtree *odb_loose_cache(struct object_directory *odb,
+				  const struct object_id *oid);
+
+/* Empty the loose object cache for the specified object directory. */
+void odb_clear_loose_cache(struct object_directory *odb);
+
+struct packed_git {
+	struct hashmap_entry packmap_ent;
+	struct packed_git *next;
+	struct list_head mru;
+	struct pack_window *windows;
+	off_t pack_size;
+	const void *index_data;
+	size_t index_size;
+	uint32_t num_objects;
+	size_t crc_offset;
+	struct oidset bad_objects;
+	int index_version;
+	time_t mtime;
+	int pack_fd;
+	int index;              /* for builtin/pack-objects.c */
+	unsigned pack_local:1,
+		 pack_keep:1,
+		 pack_keep_in_core:1,
+		 freshened:1,
+		 do_not_close:1,
+		 pack_promisor:1,
+		 multi_pack_index:1,
+		 is_cruft:1;
+	unsigned char hash[GIT_MAX_RAWSZ];
+	struct revindex_entry *revindex;
+	const uint32_t *revindex_data;
+	const uint32_t *revindex_map;
+	size_t revindex_size;
+	/*
+	 * mtimes_map points at the beginning of the memory mapped region of
+	 * this pack's corresponding .mtimes file, and mtimes_size is the size
+	 * of that .mtimes file
+	 */
+	const uint32_t *mtimes_map;
+	size_t mtimes_size;
+
+	/* repo denotes the repository this packfile belongs to */
+	struct repository *repo;
+
+	/* something like ".git/objects/pack/xxxxx.pack" */
+	char pack_name[FLEX_ARRAY]; /* more */
+};
+
+struct multi_pack_index;
+
+static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,
+				     const struct hashmap_entry *entry,
+				     const struct hashmap_entry *entry2,
+				     const void *keydata)
+{
+	const char *key = keydata;
+	const struct packed_git *pg1, *pg2;
+
+	pg1 = container_of(entry, const struct packed_git, packmap_ent);
+	pg2 = container_of(entry2, const struct packed_git, packmap_ent);
+
+	return strcmp(pg1->pack_name, key ? key : pg2->pack_name);
+}
+
+struct cached_object_entry;
+
+struct raw_object_store {
+	/*
+	 * Set of all object directories; the main directory is first (and
+	 * cannot be NULL after initialization). Subsequent directories are
+	 * alternates.
+	 */
+	struct object_directory *odb;
+	struct object_directory **odb_tail;
+	struct kh_odb_path_map *odb_by_path;
+
+	int loaded_alternates;
+
+	/*
+	 * A list of alternate object directories loaded from the environment;
+	 * this should not generally need to be accessed directly, but will
+	 * populate the "odb" list when prepare_alt_odb() is run.
+	 */
+	char *alternate_db;
+
+	/*
+	 * Objects that should be substituted by other objects
+	 * (see git-replace(1)).
+	 */
+	struct oidmap *replace_map;
+	unsigned replace_map_initialized : 1;
+	pthread_mutex_t replace_mutex; /* protect object replace functions */
+
+	struct commit_graph *commit_graph;
+	unsigned commit_graph_attempted : 1; /* if loading has been attempted */
+
+	/*
+	 * private data
+	 *
+	 * should only be accessed directly by packfile.c and midx.c
+	 */
+	struct multi_pack_index *multi_pack_index;
+
+	/*
+	 * private data
+	 *
+	 * should only be accessed directly by packfile.c
+	 */
+
+	struct packed_git *packed_git;
+	/* A most-recently-used ordered version of the packed_git list. */
+	struct list_head packed_git_mru;
+
+	struct {
+		struct packed_git **packs;
+		unsigned flags;
+	} kept_pack_cache;
+
+	/*
+	 * This is meant to hold a *small* number of objects that you would
+	 * want repo_read_object_file() to be able to return, but yet you do not want
+	 * to write them into the object store (e.g. a browse-only
+	 * application).
+	 */
+	struct cached_object_entry *cached_objects;
+	size_t cached_object_nr, cached_object_alloc;
+
+	/*
+	 * A map of packfiles to packed_git structs for tracking which
+	 * packs have been loaded already.
+	 */
+	struct hashmap pack_map;
+
+	/*
+	 * A fast, rough count of the number of objects in the repository.
+	 * These two fields are not meant for direct access. Use
+	 * repo_approximate_object_count() instead.
+	 */
+	unsigned long approximate_object_count;
+	unsigned approximate_object_count_valid : 1;
+
+	/*
+	 * Whether packed_git has already been populated with this repository's
+	 * packs.
+	 */
+	unsigned packed_git_initialized : 1;
+};
+
+struct raw_object_store *raw_object_store_new(void);
+void raw_object_store_clear(struct raw_object_store *o);
+
+/*
+ * Create a temporary file rooted in the object database directory, or
+ * die on failure. The filename is taken from "pattern", which should have the
+ * usual "XXXXXX" trailer, and the resulting filename is written into the
+ * "template" buffer. Returns the open descriptor.
+ */
+int odb_mkstemp(struct strbuf *temp_filename, const char *pattern);
+
+/*
+ * Create a pack .keep file named "name" (which should generally be the output
+ * of odb_pack_name). Returns a file descriptor opened for writing, or -1 on
+ * error.
+ */
+int odb_pack_keep(const char *name);
+
+/*
+ * Put in `buf` the name of the file in the local object database that
+ * would be used to store a loose object with the specified oid.
+ */
+const char *loose_object_path(struct repository *r, struct strbuf *buf,
+			      const struct object_id *oid);
+
+void *map_loose_object(struct repository *r, const struct object_id *oid,
+		       unsigned long *size);
+
+void *repo_read_object_file(struct repository *r,
+			    const struct object_id *oid,
+			    enum object_type *type,
+			    unsigned long *size);
+
+/* Read and unpack an object file into memory, write memory to an object file */
+int oid_object_info(struct repository *r, const struct object_id *, unsigned long *);
+
+void hash_object_file(const struct git_hash_algo *algo, const void *buf,
+		      unsigned long len, enum object_type type,
+		      struct object_id *oid);
+
+/*
+ * Add an object file to the in-memory object store, without writing it
+ * to disk.
+ *
+ * Callers are responsible for calling write_object_file to record the
+ * object in persistent storage before writing any other new objects
+ * that reference it.
+ */
+int pretend_object_file(struct repository *repo,
+			void *buf, unsigned long len, enum object_type type,
+			struct object_id *oid);
+
+struct object_info {
+	/* Request */
+	enum object_type *typep;
+	unsigned long *sizep;
+	off_t *disk_sizep;
+	struct object_id *delta_base_oid;
+	struct strbuf *type_name;
+	void **contentp;
+
+	/* Response */
+	enum {
+		OI_CACHED,
+		OI_LOOSE,
+		OI_PACKED,
+		OI_DBCACHED
+	} whence;
+	union {
+		/*
+		 * struct {
+		 * 	... Nothing to expose in this case
+		 * } cached;
+		 * struct {
+		 * 	... Nothing to expose in this case
+		 * } loose;
+		 */
+		struct {
+			struct packed_git *pack;
+			off_t offset;
+			unsigned int is_delta;
+		} packed;
+	} u;
+};
+
+/*
+ * Initializer for a "struct object_info" that wants no items. You may
+ * also memset() the memory to all-zeroes.
+ */
+#define OBJECT_INFO_INIT { 0 }
+
+/* Invoke lookup_replace_object() on the given hash */
+#define OBJECT_INFO_LOOKUP_REPLACE 1
+/* Allow reading from a loose object file of unknown/bogus type */
+#define OBJECT_INFO_ALLOW_UNKNOWN_TYPE 2
+/* Do not retry packed storage after checking packed and loose storage */
+#define OBJECT_INFO_QUICK 8
+/*
+ * Do not attempt to fetch the object if missing (even if fetch_is_missing is
+ * nonzero).
+ */
+#define OBJECT_INFO_SKIP_FETCH_OBJECT 16
+/*
+ * This is meant for bulk prefetching of missing blobs in a partial
+ * clone. Implies OBJECT_INFO_SKIP_FETCH_OBJECT and OBJECT_INFO_QUICK
+ */
+#define OBJECT_INFO_FOR_PREFETCH (OBJECT_INFO_SKIP_FETCH_OBJECT | OBJECT_INFO_QUICK)
+
+/* Die if object corruption (not just an object being missing) was detected. */
+#define OBJECT_INFO_DIE_IF_CORRUPT 32
+
+int oid_object_info_extended(struct repository *r,
+			     const struct object_id *,
+			     struct object_info *, unsigned flags);
+
+/* Retry packed storage after checking packed and loose storage */
+#define HAS_OBJECT_RECHECK_PACKED 1
+
+/*
+ * Returns 1 if the object exists. This function will not lazily fetch objects
+ * in a partial clone.
+ */
+int has_object(struct repository *r, const struct object_id *oid,
+	       unsigned flags);
+
+/*
+ * These macros and functions are deprecated. If checking existence for an
+ * object that is likely to be missing and/or whose absence is relatively
+ * inconsequential (or is consequential but the caller is prepared to handle
+ * it), use has_object(), which has better defaults (no lazy fetch in a partial
+ * clone and no rechecking of packed storage). In the unlikely event that a
+ * caller needs to assert existence of an object that it fully expects to
+ * exist, and wants to trigger a lazy fetch in a partial clone, use
+ * oid_object_info_extended() with a NULL struct object_info.
+ *
+ * These functions can be removed once all callers have migrated to
+ * has_object() and/or oid_object_info_extended().
+ */
+int repo_has_object_file(struct repository *r, const struct object_id *oid);
+int repo_has_object_file_with_flags(struct repository *r,
+				    const struct object_id *oid, int flags);
+
+void assert_oid_type(const struct object_id *oid, enum object_type expect);
+
+/*
+ * Enabling the object read lock allows multiple threads to safely call the
+ * following functions in parallel: repo_read_object_file(),
+ * read_object_with_reference(), oid_object_info() and oid_object_info_extended().
+ *
+ * obj_read_lock() and obj_read_unlock() may also be used to protect other
+ * section which cannot execute in parallel with object reading. Since the used
+ * lock is a recursive mutex, these sections can even contain calls to object
+ * reading functions. However, beware that in these cases zlib inflation won't
+ * be performed in parallel, losing performance.
+ *
+ * TODO: oid_object_info_extended()'s call stack has a recursive behavior. If
+ * any of its callees end up calling it, this recursive call won't benefit from
+ * parallel inflation.
+ */
+void enable_obj_read_lock(void);
+void disable_obj_read_lock(void);
+
+extern int obj_read_use_lock;
+extern pthread_mutex_t obj_read_mutex;
+
+static inline void obj_read_lock(void)
+{
+	if(obj_read_use_lock)
+		pthread_mutex_lock(&obj_read_mutex);
+}
+
+static inline void obj_read_unlock(void)
+{
+	if(obj_read_use_lock)
+		pthread_mutex_unlock(&obj_read_mutex);
+}
+
+/*
+ * Iterate over the files in the loose-object parts of the object
+ * directory "path", triggering the following callbacks:
+ *
+ *  - loose_object is called for each loose object we find.
+ *
+ *  - loose_cruft is called for any files that do not appear to be
+ *    loose objects. Note that we only look in the loose object
+ *    directories "objects/[0-9a-f]{2}/", so we will not report
+ *    "objects/foobar" as cruft.
+ *
+ *  - loose_subdir is called for each top-level hashed subdirectory
+ *    of the object directory (e.g., "$OBJDIR/f0"). It is called
+ *    after the objects in the directory are processed.
+ *
+ * Any callback that is NULL will be ignored. Callbacks returning non-zero
+ * will end the iteration.
+ *
+ * In the "buf" variant, "path" is a strbuf which will also be used as a
+ * scratch buffer, but restored to its original contents before
+ * the function returns.
+ */
+typedef int each_loose_object_fn(const struct object_id *oid,
+				 const char *path,
+				 void *data);
+typedef int each_loose_cruft_fn(const char *basename,
+				const char *path,
+				void *data);
+typedef int each_loose_subdir_fn(unsigned int nr,
+				 const char *path,
+				 void *data);
+int for_each_file_in_obj_subdir(unsigned int subdir_nr,
+				struct strbuf *path,
+				each_loose_object_fn obj_cb,
+				each_loose_cruft_fn cruft_cb,
+				each_loose_subdir_fn subdir_cb,
+				void *data);
+int for_each_loose_file_in_objdir(const char *path,
+				  each_loose_object_fn obj_cb,
+				  each_loose_cruft_fn cruft_cb,
+				  each_loose_subdir_fn subdir_cb,
+				  void *data);
+int for_each_loose_file_in_objdir_buf(struct strbuf *path,
+				      each_loose_object_fn obj_cb,
+				      each_loose_cruft_fn cruft_cb,
+				      each_loose_subdir_fn subdir_cb,
+				      void *data);
+
+/* Flags for for_each_*_object() below. */
+enum for_each_object_flags {
+	/* Iterate only over local objects, not alternates. */
+	FOR_EACH_OBJECT_LOCAL_ONLY = (1<<0),
+
+	/* Only iterate over packs obtained from the promisor remote. */
+	FOR_EACH_OBJECT_PROMISOR_ONLY = (1<<1),
+
+	/*
+	 * Visit objects within a pack in packfile order rather than .idx order
+	 */
+	FOR_EACH_OBJECT_PACK_ORDER = (1<<2),
+
+	/* Only iterate over packs that are not marked as kept in-core. */
+	FOR_EACH_OBJECT_SKIP_IN_CORE_KEPT_PACKS = (1<<3),
+
+	/* Only iterate over packs that do not have .keep files. */
+	FOR_EACH_OBJECT_SKIP_ON_DISK_KEPT_PACKS = (1<<4),
+};
+
+/*
+ * Iterate over all accessible loose objects without respect to
+ * reachability. By default, this includes both local and alternate objects.
+ * The order in which objects are visited is unspecified.
+ *
+ * Any flags specific to packs are ignored.
+ */
+int for_each_loose_object(each_loose_object_fn, void *,
+			  enum for_each_object_flags flags);
+
+/*
+ * Iterate over all accessible packed objects without respect to reachability.
+ * By default, this includes both local and alternate packs.
+ *
+ * Note that some objects may appear twice if they are found in multiple packs.
+ * Each pack is visited in an unspecified order. By default, objects within a
+ * pack are visited in pack-idx order (i.e., sorted by oid).
+ */
+typedef int each_packed_object_fn(const struct object_id *oid,
+				  struct packed_git *pack,
+				  uint32_t pos,
+				  void *data);
+int for_each_object_in_pack(struct packed_git *p,
+			    each_packed_object_fn, void *data,
+			    enum for_each_object_flags flags);
+int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,
+			   void *data, enum for_each_object_flags flags);
+
+void *read_object_with_reference(struct repository *r,
+				 const struct object_id *oid,
+				 enum object_type required_type,
+				 unsigned long *size,
+				 struct object_id *oid_ret);
 
 #endif /* OBJECT_STORE_H */
diff --git a/oss-fuzz/fuzz-pack-idx.c b/oss-fuzz/fuzz-pack-idx.c
index 3e190214d14..609a343ee3e 100644
--- a/oss-fuzz/fuzz-pack-idx.c
+++ b/oss-fuzz/fuzz-pack-idx.c
@@ -1,5 +1,5 @@
 #include "git-compat-util.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 
 int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size);
diff --git a/pack-bitmap-write.c b/pack-bitmap-write.c
index 6a97b52b36d..4c7f7985e78 100644
--- a/pack-bitmap-write.c
+++ b/pack-bitmap-write.c
@@ -4,7 +4,7 @@
 #include "environment.h"
 #include "gettext.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "diff.h"
 #include "revision.h"
diff --git a/pack-bitmap.c b/pack-bitmap.c
index 0dbd7c4ffe1..fb68d2ae637 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -17,7 +17,7 @@
 #include "packfile.h"
 #include "repository.h"
 #include "trace2.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "list-objects-filter-options.h"
 #include "midx.h"
 #include "config.h"
diff --git a/pack-check.c b/pack-check.c
index 95dcbbe9852..874897d6cba 100644
--- a/pack-check.c
+++ b/pack-check.c
@@ -8,7 +8,7 @@
 #include "progress.h"
 #include "packfile.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 struct idx_entry {
 	off_t                offset;
diff --git a/pack-mtimes.c b/pack-mtimes.c
index bcea28e521d..20900ca88d3 100644
--- a/pack-mtimes.c
+++ b/pack-mtimes.c
@@ -1,7 +1,7 @@
 #include "git-compat-util.h"
 #include "gettext.h"
 #include "pack-mtimes.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 #include "strbuf.h"
 
diff --git a/pack-objects.h b/pack-objects.h
index d73e3843c92..d1c4ae7f9b6 100644
--- a/pack-objects.h
+++ b/pack-objects.h
@@ -1,7 +1,7 @@
 #ifndef PACK_OBJECTS_H
 #define PACK_OBJECTS_H
 
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "thread-utils.h"
 #include "pack.h"
 
diff --git a/pack-revindex.c b/pack-revindex.c
index 1ee7b49e206..37ad9c79268 100644
--- a/pack-revindex.c
+++ b/pack-revindex.c
@@ -1,7 +1,7 @@
 #include "git-compat-util.h"
 #include "gettext.h"
 #include "pack-revindex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 #include "strbuf.h"
 #include "trace2.h"
diff --git a/packfile.c b/packfile.c
index 9d09f8bc726..d91016f1c7f 100644
--- a/packfile.c
+++ b/packfile.c
@@ -19,7 +19,7 @@
 #include "tree-walk.h"
 #include "tree.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "midx.h"
 #include "commit-graph.h"
 #include "pack-revindex.h"
diff --git a/path.c b/path.c
index 62d67166dff..4505bb78e8b 100644
--- a/path.c
+++ b/path.c
@@ -15,7 +15,7 @@
 #include "submodule-config.h"
 #include "path.h"
 #include "packfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "lockfile.h"
 #include "exec-cmd.h"
 
diff --git a/promisor-remote.c b/promisor-remote.c
index 5801ebfd9b2..9d058586dfa 100644
--- a/promisor-remote.c
+++ b/promisor-remote.c
@@ -3,7 +3,7 @@
 #include "git-compat-util.h"
 #include "gettext.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "promisor-remote.h"
 #include "config.h"
 #include "trace2.h"
diff --git a/protocol-caps.c b/protocol-caps.c
index 855f279c2f7..9b8db37a210 100644
--- a/protocol-caps.c
+++ b/protocol-caps.c
@@ -6,7 +6,7 @@
 #include "hash.h"
 #include "hex.h"
 #include "object.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "repository.h"
 #include "string-list.h"
 #include "strbuf.h"
diff --git a/prune-packed.c b/prune-packed.c
index 7dad2fc0c16..c1d95a519d7 100644
--- a/prune-packed.c
+++ b/prune-packed.c
@@ -2,7 +2,7 @@
 
 #include "git-compat-util.h"
 #include "gettext.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 #include "progress.h"
 #include "prune-packed.h"
diff --git a/reachable.c b/reachable.c
index 1b26b9b1d76..16e23a38037 100644
--- a/reachable.c
+++ b/reachable.c
@@ -14,7 +14,7 @@
 #include "list-objects.h"
 #include "packfile.h"
 #include "worktree.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pack-bitmap.h"
 #include "pack-mtimes.h"
 #include "config.h"
diff --git a/read-cache.c b/read-cache.c
index 23028f43a11..570744bb566 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -20,7 +20,7 @@
 #include "refs.h"
 #include "dir.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "tree.h"
 #include "commit.h"
diff --git a/ref-filter.c b/ref-filter.c
index 6da8d4c03b6..7a274633cfc 100644
--- a/ref-filter.c
+++ b/ref-filter.c
@@ -12,7 +12,7 @@
 #include "refs.h"
 #include "wildmatch.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "repo-settings.h"
 #include "repository.h"
diff --git a/reflog.c b/reflog.c
index 1b5f031f6d7..60834a124d9 100644
--- a/reflog.c
+++ b/reflog.c
@@ -3,7 +3,7 @@
 
 #include "git-compat-util.h"
 #include "gettext.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "reflog.h"
 #include "refs.h"
 #include "revision.h"
diff --git a/refs.c b/refs.c
index 1208f86629b..14f25c499cd 100644
--- a/refs.c
+++ b/refs.c
@@ -19,7 +19,7 @@
 #include "run-command.h"
 #include "hook.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "object.h"
 #include "path.h"
 #include "submodule.h"
diff --git a/remote.c b/remote.c
index e609cf5c56a..7fc657d98b8 100644
--- a/remote.c
+++ b/remote.c
@@ -12,7 +12,7 @@
 #include "refs.h"
 #include "refspec.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "commit.h"
 #include "diff.h"
diff --git a/replace-object.c b/replace-object.c
index 9a3cdd809a9..7b8a09b5cb4 100644
--- a/replace-object.c
+++ b/replace-object.c
@@ -2,7 +2,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "oidmap.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "replace-object.h"
 #include "refs.h"
 #include "repository.h"
diff --git a/replace-object.h b/replace-object.h
index 66c41b938b4..ba478eb30c4 100644
--- a/replace-object.h
+++ b/replace-object.h
@@ -3,7 +3,7 @@
 
 #include "oidmap.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 
 struct replace_object {
 	struct oidmap_entry original;
diff --git a/repository.c b/repository.c
index 6cbaf2e3daa..9b3d6665fc6 100644
--- a/repository.c
+++ b/repository.c
@@ -1,7 +1,7 @@
 #include "git-compat-util.h"
 #include "abspath.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "config.h"
 #include "object.h"
 #include "lockfile.h"
diff --git a/rerere.c b/rerere.c
index 0832cc54840..3cd37c5f0ae 100644
--- a/rerere.c
+++ b/rerere.c
@@ -18,7 +18,7 @@
 #include "path.h"
 #include "pathspec.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "strmap.h"
 
 #define RESOLVED 0
diff --git a/revision.c b/revision.c
index b536c4a29ad..352e18b1f97 100644
--- a/revision.c
+++ b/revision.c
@@ -8,7 +8,7 @@
 #include "hex.h"
 #include "object-name.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oidset.h"
 #include "tag.h"
 #include "blob.h"
diff --git a/send-pack.c b/send-pack.c
index 856a65d5f5a..5005689cb55 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -4,7 +4,7 @@
 #include "date.h"
 #include "gettext.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pkt-line.h"
 #include "sideband.h"
 #include "run-command.h"
diff --git a/sequencer.c b/sequencer.c
index 9fda9be9266..552051e275c 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -13,7 +13,7 @@
 #include "dir.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "object.h"
 #include "pager.h"
 #include "commit.h"
diff --git a/server-info.c b/server-info.c
index f0646ac92a9..d6cd20a39d7 100644
--- a/server-info.c
+++ b/server-info.c
@@ -11,7 +11,7 @@
 #include "packfile.h"
 #include "path.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "server-info.h"
 #include "strbuf.h"
 #include "tempfile.h"
diff --git a/shallow.c b/shallow.c
index 06c3266a3e0..2f82ebd6e3f 100644
--- a/shallow.c
+++ b/shallow.c
@@ -5,7 +5,7 @@
 #include "repository.h"
 #include "tempfile.h"
 #include "lockfile.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tag.h"
 #include "pkt-line.h"
diff --git a/streaming.c b/streaming.c
index 018b794d252..127d6b5d6ac 100644
--- a/streaming.c
+++ b/streaming.c
@@ -10,7 +10,7 @@
 #include "streaming.h"
 #include "repository.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "replace-object.h"
 #include "packfile.h"
 
diff --git a/submodule-config.c b/submodule-config.c
index d82b404b73e..8630e27947d 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -13,7 +13,7 @@
 #include "submodule.h"
 #include "strbuf.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "parse-options.h"
 #include "thread-utils.h"
 #include "tree-walk.h"
diff --git a/submodule.c b/submodule.c
index 218c8c17603..ead3fb5dadc 100644
--- a/submodule.c
+++ b/submodule.c
@@ -27,7 +27,7 @@
 #include "parse-options.h"
 #include "object-file.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit-reach.h"
 #include "read-cache-ll.h"
 #include "setup.h"
diff --git a/t/helper/test-pack-mtimes.c b/t/helper/test-pack-mtimes.c
index f8f9afbb5b1..50f5941bff7 100644
--- a/t/helper/test-pack-mtimes.c
+++ b/t/helper/test-pack-mtimes.c
@@ -3,7 +3,7 @@
 #include "test-tool.h"
 #include "hex.h"
 #include "strbuf.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "packfile.h"
 #include "pack-mtimes.h"
 #include "setup.h"
diff --git a/t/helper/test-partial-clone.c b/t/helper/test-partial-clone.c
index a1af9710c31..34f1aee5581 100644
--- a/t/helper/test-partial-clone.c
+++ b/t/helper/test-partial-clone.c
@@ -1,7 +1,7 @@
 #include "test-tool.h"
 #include "hex.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "setup.h"
 
 /*
diff --git a/t/helper/test-read-graph.c b/t/helper/test-read-graph.c
index 811dde1cb3c..8b413b644be 100644
--- a/t/helper/test-read-graph.c
+++ b/t/helper/test-read-graph.c
@@ -3,7 +3,7 @@
 #include "test-tool.h"
 #include "commit-graph.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "bloom.h"
 #include "setup.h"
 
diff --git a/t/helper/test-read-midx.c b/t/helper/test-read-midx.c
index fc632369618..ac81390899a 100644
--- a/t/helper/test-read-midx.c
+++ b/t/helper/test-read-midx.c
@@ -4,7 +4,7 @@
 #include "hex.h"
 #include "midx.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "pack-bitmap.h"
 #include "packfile.h"
 #include "setup.h"
diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c
index 2ff67c067ac..4cfc7c90b59 100644
--- a/t/helper/test-ref-store.c
+++ b/t/helper/test-ref-store.c
@@ -5,7 +5,7 @@
 #include "refs.h"
 #include "setup.h"
 #include "worktree.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "path.h"
 #include "repository.h"
 #include "strbuf.h"
diff --git a/tag.c b/tag.c
index 8d9e9e29304..05be39067cf 100644
--- a/tag.c
+++ b/tag.c
@@ -5,7 +5,7 @@
 #include "environment.h"
 #include "tag.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tree.h"
 #include "blob.h"
diff --git a/tmp-objdir.c b/tmp-objdir.c
index 31d16a4c2c5..c38fbeb5e8a 100644
--- a/tmp-objdir.c
+++ b/tmp-objdir.c
@@ -10,7 +10,7 @@
 #include "strbuf.h"
 #include "strvec.h"
 #include "quote.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "repository.h"
 
 struct tmp_objdir {
diff --git a/tree-walk.c b/tree-walk.c
index a0333979656..90655d52378 100644
--- a/tree-walk.c
+++ b/tree-walk.c
@@ -6,7 +6,7 @@
 #include "gettext.h"
 #include "hex.h"
 #include "object-file.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "trace2.h"
 #include "tree.h"
 #include "pathspec.h"
diff --git a/tree.c b/tree.c
index ad86ad1ba99..b85f56267fb 100644
--- a/tree.c
+++ b/tree.c
@@ -4,7 +4,7 @@
 #include "hex.h"
 #include "tree.h"
 #include "object-name.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "alloc.h"
 #include "tree-walk.h"
diff --git a/unpack-trees.c b/unpack-trees.c
index cf5b73c84be..471837f0329 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -26,7 +26,7 @@
 #include "symlinks.h"
 #include "trace2.h"
 #include "fsmonitor.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "promisor-remote.h"
 #include "entry.h"
 #include "parallel-checkout.h"
diff --git a/upload-pack.c b/upload-pack.c
index 02ce6336028..30e4630f3a1 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -10,7 +10,7 @@
 #include "pkt-line.h"
 #include "sideband.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "oid-array.h"
 #include "object.h"
 #include "commit.h"
diff --git a/walker.c b/walker.c
index 1cf3da02193..4fedc19f346 100644
--- a/walker.c
+++ b/walker.c
@@ -5,7 +5,7 @@
 #include "hex.h"
 #include "walker.h"
 #include "repository.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "commit.h"
 #include "strbuf.h"
 #include "tree.h"
diff --git a/xdiff-interface.c b/xdiff-interface.c
index 77712811ff1..1edcd319e6e 100644
--- a/xdiff-interface.c
+++ b/xdiff-interface.c
@@ -5,7 +5,7 @@
 #include "gettext.h"
 #include "config.h"
 #include "hex.h"
-#include "object-store-ll.h"
+#include "object-store.h"
 #include "strbuf.h"
 #include "xdiff-interface.h"
 #include "xdiff/xtypes.h"

-- 
2.49.0.805.g082f7c87e0.dirty


^ permalink raw reply related	[flat|nested] 53+ messages in thread

* Re: [PATCH v2 1/9] object-file: move `safe_create_leading_directories()` into "dir.c"
  2025-04-15  9:19         ` Patrick Steinhardt
@ 2025-04-15 15:09           ` Junio C Hamano
  0 siblings, 0 replies; 53+ messages in thread
From: Junio C Hamano @ 2025-04-15 15:09 UTC (permalink / raw)
  To: Patrick Steinhardt; +Cc: Eric Sunshine, git, Elijah Newren

Patrick Steinhardt <ps@pks.im> writes:

> On Fri, Apr 11, 2025 at 05:29:13PM -0400, Eric Sunshine wrote:
>> On Fri, Apr 11, 2025 at 4:10 PM Junio C Hamano <gitster@pobox.com> wrote:
>> > Patrick Steinhardt <ps@pks.im> writes:
>> > > The `safe_create_leading_directories()` function and its relatives are
>> > > located in "object-file.c", which is not a good fit as they provide
>> > > generic functionality not related to objects at all. Move them into
>> > > "dir.c".
>> >
>> > It may be debatable that <dir.c>, which has traditionally been a
>> > collection of read-only operations (mostly for exclude/ignore
>> > processing), is a good place to host "mkdir -p", but it certainly is
>> > better than having it in <object-file.c>
>> 
>> I probably would have expected safe_create_leading_directories() to be
>> moved to "path.[hc]" which already houses functions such as
>> safe_create_dir(), normalize_path_copy(), ends_with_path_components(),
>> longest_ancestor_length(), etc.
>
> Ah, good catch! Will adapt.

Yeah, path.* API sounds like a better fit, indeed.

Thanks, both.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH 1/9] object-file: move `safe_create_leading_directories()` into "dir.c"
  2025-04-15  9:19         ` Patrick Steinhardt
@ 2025-04-15 15:11           ` Junio C Hamano
  0 siblings, 0 replies; 53+ messages in thread
From: Junio C Hamano @ 2025-04-15 15:11 UTC (permalink / raw)
  To: Patrick Steinhardt; +Cc: Elijah Newren, git

Patrick Steinhardt <ps@pks.im> writes:

> better fix. I'm using that as an opportunity to rename the function to
> `safe_create_dir_in_gitdir()` so that it matches `safe_create_dir()`,
> which is functionally similar.

Excellent.

Thanks.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v3 00/10] Split up "object-file.c"
  2025-04-15  9:38 ` [PATCH v3 00/10] Split up "object-file.c" Patrick Steinhardt
                     ` (9 preceding siblings ...)
  2025-04-15  9:38   ` [PATCH v3 10/10] object-store: merge "object-store-ll.h" and "object-store.h" Patrick Steinhardt
@ 2025-04-16  6:41   ` Elijah Newren
  2025-04-16  7:44     ` Patrick Steinhardt
  2025-04-16 16:21     ` Junio C Hamano
  10 siblings, 2 replies; 53+ messages in thread
From: Elijah Newren @ 2025-04-16  6:41 UTC (permalink / raw)
  To: Patrick Steinhardt; +Cc: git, Eric Sunshine, Jeff King, Junio C Hamano

On Tue, Apr 15, 2025 at 2:38 AM Patrick Steinhardt <ps@pks.im> wrote:
>
> Hi,
>
> "object-file.c" is quite a grab-bag of all kinds of different functions.
> Many of these functions aren't really a good fit though and should be
> owned by a different subsystem. This patch series tries to split up
> concerns a bit better by splitting out this functionality into other
> files:
>
>   - `safe_create_leading_directories()` is moved into "dir.c".
>   - `xmmap()` is moved into "wrapper.c".
>   - `git_open_cloexec()` is moved into "compat/open.c".
>   - Several functions attached to `struct index_state` are moved into
>     "read-cache.c".
>   - Several functions related to `struct object_store` are moved into a
>     new file "object-store.c".
>
> "object-file.c" now mostly contains logic to read and write loose object
> files, whereas "object-store.c" contains the higher-level logic to
> manage different object directories for a repository. Eventually, these
> will become the loose object backend as well as the `struct ref_store`
> equivalent for objects, respectively.
>
> The series is built on top of 9d22ac51228 (The third batch, 2025-04-07)
> with ps/object-wo-the-repository at 9442b1c919a (Merge remote-tracking
> branch 'junio/ps/object-wo-the-repository' into HEAD, 2025-04-08) merged
> into it.
>
> Changes in v2:
>   - Fix a grammar issue in one of the commit messages.
>   - Link to v1: https://lore.kernel.org/r/20250408-pks-split-object-file-v1-0-f1fd50191143@pks.im
>
> Changes in v3:
>   - Rename `mkdir_in_gitdir()` to `safe_create_dir_in_gitdir()` to match
>     naming of similar functions.
>   - Move `safe_create_leading_directories()` et al into "path.c" instead
>     of into "dir.c". This also requires us to start injecting a repo via
>     parameters as "path.c" doesn't have `the_repository` available
>     anymore.
>   - Drop the commit that moves `index_blob_stream()` and related
>     functions.
>   - Expand the reasoning why we want to have cached objects per object
>     store instead of globally.
>   - Drop `index_blob_stream()`, which is a trivial wrapper around
>     `index_blob_bulk_checkin()`.
>   - Link to v2: https://lore.kernel.org/r/20250411-pks-split-object-file-v2-0-2bea0c9033ae@pks.im

v3 also addressed my feedback on v1 & v2.

[...]
> Range-diff versus v2:

I read over the range-diff and the three new patches (1, 2, & 7); this
round looks good to me.  I particularly like the extended rationale in
the commit message for what is now patch 9.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v3 00/10] Split up "object-file.c"
  2025-04-16  6:41   ` [PATCH v3 00/10] Split up "object-file.c" Elijah Newren
@ 2025-04-16  7:44     ` Patrick Steinhardt
  2025-04-16 16:21     ` Junio C Hamano
  1 sibling, 0 replies; 53+ messages in thread
From: Patrick Steinhardt @ 2025-04-16  7:44 UTC (permalink / raw)
  To: Elijah Newren; +Cc: git, Eric Sunshine, Jeff King, Junio C Hamano

On Tue, Apr 15, 2025 at 11:41:34PM -0700, Elijah Newren wrote:
> On Tue, Apr 15, 2025 at 2:38 AM Patrick Steinhardt <ps@pks.im> wrote:
> > Changes in v3:
> >   - Rename `mkdir_in_gitdir()` to `safe_create_dir_in_gitdir()` to match
> >     naming of similar functions.
> >   - Move `safe_create_leading_directories()` et al into "path.c" instead
> >     of into "dir.c". This also requires us to start injecting a repo via
> >     parameters as "path.c" doesn't have `the_repository` available
> >     anymore.
> >   - Drop the commit that moves `index_blob_stream()` and related
> >     functions.
> >   - Expand the reasoning why we want to have cached objects per object
> >     store instead of globally.
> >   - Drop `index_blob_stream()`, which is a trivial wrapper around
> >     `index_blob_bulk_checkin()`.
> >   - Link to v2: https://lore.kernel.org/r/20250411-pks-split-object-file-v2-0-2bea0c9033ae@pks.im
> 
> v3 also addressed my feedback on v1 & v2.
> 
> [...]
> > Range-diff versus v2:
> 
> I read over the range-diff and the three new patches (1, 2, & 7); this
> round looks good to me.  I particularly like the extended rationale in
> the commit message for what is now patch 9.

Thanks for your review!

Patrick

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v3 00/10] Split up "object-file.c"
  2025-04-16  6:41   ` [PATCH v3 00/10] Split up "object-file.c" Elijah Newren
  2025-04-16  7:44     ` Patrick Steinhardt
@ 2025-04-16 16:21     ` Junio C Hamano
  1 sibling, 0 replies; 53+ messages in thread
From: Junio C Hamano @ 2025-04-16 16:21 UTC (permalink / raw)
  To: Elijah Newren; +Cc: Patrick Steinhardt, git, Eric Sunshine, Jeff King

Elijah Newren <newren@gmail.com> writes:

> I read over the range-diff and the three new patches (1, 2, & 7); this
> round looks good to me.  I particularly like the extended rationale in
> the commit message for what is now patch 9.

Thanks, both, this round looked good to me, too.


^ permalink raw reply	[flat|nested] 53+ messages in thread

end of thread, other threads:[~2025-04-16 16:21 UTC | newest]

Thread overview: 53+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-04-08 10:24 [PATCH 0/9] Split up "object-file.c" Patrick Steinhardt
2025-04-08 10:24 ` [PATCH 1/9] object-file: move `safe_create_leading_directories()` into "dir.c" Patrick Steinhardt
2025-04-09 14:36   ` Elijah Newren
2025-04-11  9:27     ` Patrick Steinhardt
2025-04-11 17:11       ` Elijah Newren
2025-04-15  9:19         ` Patrick Steinhardt
2025-04-15 15:11           ` Junio C Hamano
2025-04-08 10:24 ` [PATCH 2/9] object-file: move `git_open_cloexec()` to "compat/open.c" Patrick Steinhardt
2025-04-08 10:24 ` [PATCH 3/9] object-file: move `xmmap()` into "wrapper.c" Patrick Steinhardt
2025-04-09 14:36   ` Elijah Newren
2025-04-08 10:24 ` [PATCH 4/9] object-file: split out functions relating to object store subsystem Patrick Steinhardt
2025-04-08 10:24 ` [PATCH 5/9] object-file: split up concerns of `HASH_*` flags Patrick Steinhardt
2025-04-08 10:24 ` [PATCH 6/9] object-file: split out functions relating to index subsystem Patrick Steinhardt
2025-04-08 10:24 ` [PATCH 7/9] object: split out functions relating to object store subsystem Patrick Steinhardt
2025-04-08 10:24 ` [PATCH 8/9] object-store: remove global array of cached objects Patrick Steinhardt
2025-04-08 10:24 ` [PATCH 9/9] object-store: merge "object-store-ll.h" and "object-store.h" Patrick Steinhardt
2025-04-08 23:29 ` [PATCH 0/9] Split up "object-file.c" Junio C Hamano
2025-04-11  9:26   ` Patrick Steinhardt
2025-04-09 14:42 ` Elijah Newren
2025-04-11  9:27   ` Patrick Steinhardt
2025-04-11  9:29 ` [PATCH v2 " Patrick Steinhardt
2025-04-11  9:29   ` [PATCH v2 1/9] object-file: move `safe_create_leading_directories()` into "dir.c" Patrick Steinhardt
2025-04-11 20:09     ` Junio C Hamano
2025-04-11 21:29       ` Eric Sunshine
2025-04-15  9:19         ` Patrick Steinhardt
2025-04-15 15:09           ` Junio C Hamano
2025-04-11  9:29   ` [PATCH v2 2/9] object-file: move `git_open_cloexec()` to "compat/open.c" Patrick Steinhardt
2025-04-11  9:29   ` [PATCH v2 3/9] object-file: move `xmmap()` into "wrapper.c" Patrick Steinhardt
2025-04-11  9:29   ` [PATCH v2 4/9] object-file: split out functions relating to object store subsystem Patrick Steinhardt
2025-04-11  9:29   ` [PATCH v2 5/9] object-file: split up concerns of `HASH_*` flags Patrick Steinhardt
2025-04-11  9:29   ` [PATCH v2 6/9] object-file: split out functions relating to index subsystem Patrick Steinhardt
2025-04-12  8:17     ` Jeff King
2025-04-14 11:49       ` Junio C Hamano
2025-04-15  9:19       ` Patrick Steinhardt
2025-04-11  9:29   ` [PATCH v2 7/9] object: split out functions relating to object store subsystem Patrick Steinhardt
2025-04-11  9:29   ` [PATCH v2 8/9] object-store: remove global array of cached objects Patrick Steinhardt
2025-04-11 22:58     ` Junio C Hamano
2025-04-15  9:19       ` Patrick Steinhardt
2025-04-11  9:29   ` [PATCH v2 9/9] object-store: merge "object-store-ll.h" and "object-store.h" Patrick Steinhardt
2025-04-15  9:38 ` [PATCH v3 00/10] Split up "object-file.c" Patrick Steinhardt
2025-04-15  9:38   ` [PATCH v3 01/10] object-file: move `mkdir_in_gitdir()` into "path.c" Patrick Steinhardt
2025-04-15  9:38   ` [PATCH v3 02/10] object-file: move `safe_create_leading_directories()` " Patrick Steinhardt
2025-04-15  9:38   ` [PATCH v3 03/10] object-file: move `git_open_cloexec()` to "compat/open.c" Patrick Steinhardt
2025-04-15  9:38   ` [PATCH v3 04/10] object-file: move `xmmap()` into "wrapper.c" Patrick Steinhardt
2025-04-15  9:38   ` [PATCH v3 05/10] object-file: split out functions relating to object store subsystem Patrick Steinhardt
2025-04-15  9:38   ` [PATCH v3 06/10] object-file: split up concerns of `HASH_*` flags Patrick Steinhardt
2025-04-15  9:38   ` [PATCH v3 07/10] object-file: drop `index_blob_stream()` Patrick Steinhardt
2025-04-15  9:38   ` [PATCH v3 08/10] object: split out functions relating to object store subsystem Patrick Steinhardt
2025-04-15  9:38   ` [PATCH v3 09/10] object-store: remove global array of cached objects Patrick Steinhardt
2025-04-15  9:38   ` [PATCH v3 10/10] object-store: merge "object-store-ll.h" and "object-store.h" Patrick Steinhardt
2025-04-16  6:41   ` [PATCH v3 00/10] Split up "object-file.c" Elijah Newren
2025-04-16  7:44     ` Patrick Steinhardt
2025-04-16 16:21     ` Junio C Hamano

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).