* [PATCH 01/13] commit-graph: require resolved packfile paths for `stdin_packs`
2026-10-02 10:08 [PATCH 00/13] odb/source-files: move alternates into the backend Patrick Steinhardt
@ 2026-10-02 10:08 ` Patrick Steinhardt
2026-10-05 19:27 ` Karthik Nayak
2026-10-02 10:08 ` [PATCH 02/13] commit-graph: stop depending on `struct odb_source` Patrick Steinhardt
` (13 subsequent siblings)
14 siblings, 1 reply; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-02 10:08 UTC (permalink / raw)
To: git
Users can ask git-commit-graph(1) to write a commit graph specifically
for a set of packfiles via the "--stdin-packs" option. Those users are
expected to pass in relative paths, and those eventually get resolved in
`fill_oids_from_packs()`. This ties the logic in "commit-graph.c" to the
specific object database source.
Refactor the logic to instead require the caller to pass in resolved
packfiles to untangle that dependency.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/commit-graph.c | 17 ++++++++++++++---
commit-graph.c | 15 +++++----------
2 files changed, 19 insertions(+), 13 deletions(-)
diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c
index d62005edc0..b5784ad3c7 100644
--- a/builtin/commit-graph.c
+++ b/builtin/commit-graph.c
@@ -8,6 +8,7 @@
#include "parse-options.h"
#include "commit-graph.h"
#include "odb.h"
+#include "odb/source.h"
#include "progress.h"
#include "replace-object.h"
#include "strbuf.h"
@@ -302,9 +303,19 @@ static int graph_write(int argc, const char **argv, const char *prefix,
}
if (opts.stdin_packs) {
- while (strbuf_getline(&buf, stdin) != EOF)
- string_list_append_nodup(&pack_indexes,
- strbuf_detach(&buf, NULL));
+ struct strbuf packname = STRBUF_INIT;
+ size_t dirlen;
+
+ strbuf_addf(&packname, "%s/pack/", source->path);
+ dirlen = packname.len;
+
+ while (strbuf_getline(&buf, stdin) != EOF) {
+ strbuf_setlen(&packname, dirlen);
+ strbuf_addbuf(&packname, &buf);
+ string_list_append(&pack_indexes, packname.buf);
+ }
+
+ strbuf_release(&packname);
} else if (opts.stdin_commits) {
oidset_init(&commits, 0);
if (opts.progress)
diff --git a/commit-graph.c b/commit-graph.c
index 983c11ce85..73814c1622 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -1936,12 +1936,8 @@ static int fill_oids_from_packs(struct write_commit_graph_context *ctx,
{
uint32_t i;
struct strbuf progress_title = STRBUF_INIT;
- struct strbuf packname = STRBUF_INIT;
- int dirlen;
int ret = 0;
- strbuf_addf(&packname, "%s/pack/", ctx->odb_source->path);
- dirlen = packname.len;
if (ctx->report_progress) {
strbuf_addf(&progress_title,
Q_("Finding commits for commit graph in %"PRIuMAX" pack",
@@ -1954,15 +1950,15 @@ static int fill_oids_from_packs(struct write_commit_graph_context *ctx,
}
for (i = 0; i < pack_indexes->nr; i++) {
struct packed_git *p;
- strbuf_setlen(&packname, dirlen);
- strbuf_addstr(&packname, pack_indexes->items[i].string);
- p = add_packed_git(ctx->r, packname.buf, packname.len, 1);
+
+ p = add_packed_git(ctx->r, pack_indexes->items[i].string,
+ strlen(pack_indexes->items[i].string), 1);
if (!p) {
- ret = error(_("error adding pack %s"), packname.buf);
+ ret = error(_("error adding pack %s"), pack_indexes->items[i].string);
goto cleanup;
}
if (open_pack_index(p)) {
- ret = error(_("error opening index for %s"), packname.buf);
+ ret = error(_("error opening index for %s"), pack_indexes->items[i].string);
close_pack(p);
free(p);
goto cleanup;
@@ -1976,7 +1972,6 @@ static int fill_oids_from_packs(struct write_commit_graph_context *ctx,
cleanup:
stop_progress(&ctx->progress);
strbuf_release(&progress_title);
- strbuf_release(&packname);
return ret;
}
--
2.56.0.379.gc618271300.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* Re: [PATCH 01/13] commit-graph: require resolved packfile paths for `stdin_packs`
2026-10-02 10:08 ` [PATCH 01/13] commit-graph: require resolved packfile paths for `stdin_packs` Patrick Steinhardt
@ 2026-10-05 19:27 ` Karthik Nayak
2026-10-06 12:18 ` Patrick Steinhardt
0 siblings, 1 reply; 41+ messages in thread
From: Karthik Nayak @ 2026-10-05 19:27 UTC (permalink / raw)
To: Patrick Steinhardt, git
[-- Attachment #1: Type: text/plain, Size: 576 bytes --]
Patrick Steinhardt <ps@pks.im> writes:
> Users can ask git-commit-graph(1) to write a commit graph specifically
> for a set of packfiles via the "--stdin-packs" option. Those users are
> expected to pass in relative paths, and those eventually get resolved in
> `fill_oids_from_packs()`. This ties the logic in "commit-graph.c" to the
> specific object database source.
>
> Refactor the logic to instead require the caller to pass in resolved
> packfiles to untangle that dependency.
>
Nit: the changes look good, what I'm missing is 'why' are we doing this
change.
[snip]
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH 01/13] commit-graph: require resolved packfile paths for `stdin_packs`
2026-10-05 19:27 ` Karthik Nayak
@ 2026-10-06 12:18 ` Patrick Steinhardt
0 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-06 12:18 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git
On Mon, Oct 05, 2026 at 03:27:27PM -0400, Karthik Nayak wrote:
> Patrick Steinhardt <ps@pks.im> writes:
>
> > Users can ask git-commit-graph(1) to write a commit graph specifically
> > for a set of packfiles via the "--stdin-packs" option. Those users are
> > expected to pass in relative paths, and those eventually get resolved in
> > `fill_oids_from_packs()`. This ties the logic in "commit-graph.c" to the
> > specific object database source.
> >
> > Refactor the logic to instead require the caller to pass in resolved
> > packfiles to untangle that dependency.
> >
>
> Nit: the changes look good, what I'm missing is 'why' are we doing this
> change.
It's basically this sentence:
This ties the logic in "commit-graph.c" to the specific object
database source.
With the current logic we have assumptions in "commit-graph.c" about
where a specific packfile lives relative to an object database source.
But with the next commit that becomes a bit harder to realize So
resolving packfile paths early on gets rid of parts of these
assumptions, so that we don't have to resolve those paths in
"commit-graph.c" anymore.
I'll rephrase this a bit.
Patrick
^ permalink raw reply [flat|nested] 41+ messages in thread
* [PATCH 02/13] commit-graph: stop depending on `struct odb_source`
2026-10-02 10:08 [PATCH 00/13] odb/source-files: move alternates into the backend Patrick Steinhardt
2026-10-02 10:08 ` [PATCH 01/13] commit-graph: require resolved packfile paths for `stdin_packs` Patrick Steinhardt
@ 2026-10-02 10:08 ` Patrick Steinhardt
2026-10-05 19:43 ` Karthik Nayak
2026-10-02 10:08 ` [PATCH 03/13] odb/source-files: introduce `struct odb_files_dir` Patrick Steinhardt
` (12 subsequent siblings)
14 siblings, 1 reply; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-02 10:08 UTC (permalink / raw)
To: git
To read or write a commit graph we require access to the repository that
the graph should be read from or written for as well as the object
directory to derive its location. Instead of passing in these two bits
of information explicitly though, we pass in a `struct odb_source`,
which carries with it both of these pieces of information.
But ultimately, this is somewhat flawed as we now assume that the source
even has an on-disk representation, and furthermore we assume that the
commit graphs would live in ".git/objects/info". This is true for the
"files" backend, but it's not necessarily true for any other backend
that we may eventually want to introduce. So eventually, we'll want to
evolve the commit-graph subsystem to become agnostic of the backend's
layout and let the backend itself decide where to read a commit graph
from or where to write it to.
We're not there yet to do that switch, but the current design is already
causing issues for the intent of this patch series where we want to move
alternates into the "files" backend.
Convert the subsystem to take a repository plus an object directory path
instead. This unblocks moving around alternates, and it's also a step
in the right direction for moving commit graphs into the source in a
later series.
Note that we previously compared the `struct odb_source` pointers of two
commit graphs to figure out whether they were located in the same object
directory, whereas we now have to compare their paths. Callers may pass
in those paths in different forms though, for example relative to the
current working directory or as absolute paths. Convert the object
directory into an absolute path both when loading and when writing
commit graphs so that the comparisons become more robust.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/commit-graph.c | 10 ++--
builtin/commit.c | 2 +-
builtin/fetch.c | 4 +-
builtin/gc.c | 3 +-
builtin/merge.c | 2 +-
commit-graph.c | 115 +++++++++++++++++++++++++--------------------
commit-graph.h | 31 ++++++++----
t/helper/test-read-graph.c | 3 +-
t/t4216-log-bloom.sh | 4 +-
9 files changed, 102 insertions(+), 72 deletions(-)
diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c
index b5784ad3c7..a986f08a94 100644
--- a/builtin/commit-graph.c
+++ b/builtin/commit-graph.c
@@ -104,8 +104,8 @@ static int graph_verify(int argc, const char **argv, const char *prefix,
flags |= COMMIT_GRAPH_WRITE_PROGRESS;
source = odb_find_source_or_die(the_repository->objects, opts.obj_dir);
- graph_name = get_commit_graph_filename(source);
- chain_name = get_commit_graph_chain_filename(source);
+ graph_name = get_commit_graph_filename(source->path);
+ chain_name = get_commit_graph_chain_filename(source->path);
if (open_commit_graph(graph_name, &fd, &st))
opened = OPENED_GRAPH;
else if (errno != ENOENT)
@@ -123,7 +123,7 @@ static int graph_verify(int argc, const char **argv, const char *prefix,
if (opened == OPENED_NONE)
return 0;
else if (opened == OPENED_GRAPH)
- graph = load_commit_graph_one_fd_st(source, fd, &st);
+ graph = load_commit_graph_one_fd_st(the_repository, source->path, fd, &st);
else
graph = load_commit_graph_chain_fd_st(the_repository->objects, fd, &st,
&incomplete_chain);
@@ -297,7 +297,7 @@ static int graph_write(int argc, const char **argv, const char *prefix,
source = odb_find_source_or_die(the_repository->objects, opts.obj_dir);
if (opts.reachable) {
- if (write_commit_graph_reachable(source, flags, &write_opts))
+ if (write_commit_graph_reachable(the_repository, source->path, flags, &write_opts))
result = 1;
goto cleanup;
}
@@ -334,7 +334,7 @@ static int graph_write(int argc, const char **argv, const char *prefix,
stop_progress(&progress);
}
- if (write_commit_graph(source,
+ if (write_commit_graph(the_repository, source->path,
opts.stdin_packs ? &pack_indexes : NULL,
opts.stdin_commits ? &commits : NULL,
flags,
diff --git a/builtin/commit.c b/builtin/commit.c
index 840b6b4083..0e750f6157 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1959,7 +1959,7 @@ int cmd_commit(int argc,
"new index file. Check that disk is not full and quota is\n"
"not exceeded, and then \"git restore --staged :/\" to recover."));
- git_test_write_commit_graph_or_die(the_repository->objects->sources);
+ git_test_write_commit_graph_or_die(the_repository);
repo_rerere(the_repository, 0);
run_auto_maintenance(the_repository, quiet);
diff --git a/builtin/fetch.c b/builtin/fetch.c
index b2decc6cfd..a4b21d2651 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -15,6 +15,7 @@
#include "refspec.h"
#include "object-name.h"
#include "odb.h"
+#include "odb/source.h"
#include "oidset.h"
#include "oid-array.h"
#include "commit.h"
@@ -2900,7 +2901,8 @@ int cmd_fetch(int argc,
commit_graph_flags |= COMMIT_GRAPH_WRITE_PROGRESS;
trace2_region_enter("fetch", "write-commit-graph", the_repository);
- write_commit_graph_reachable(the_repository->objects->sources,
+ write_commit_graph_reachable(the_repository,
+ the_repository->objects->sources->path,
commit_graph_flags,
NULL);
trace2_region_leave("fetch", "write-commit-graph", the_repository);
diff --git a/builtin/gc.c b/builtin/gc.c
index 57a3520263..7acd4f3215 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -733,7 +733,8 @@ int cmd_gc(int argc,
}
if (the_repository->settings.gc_write_commit_graph == 1)
- write_commit_graph_reachable(the_repository->objects->sources,
+ write_commit_graph_reachable(the_repository,
+ the_repository->objects->sources->path,
!opts.quiet && !daemonized ? COMMIT_GRAPH_WRITE_PROGRESS : 0,
NULL);
diff --git a/builtin/merge.c b/builtin/merge.c
index 5b4eb23a83..d7bf209114 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -1863,7 +1863,7 @@ int cmd_merge(int argc,
if (squash) {
finish(head_commit, remoteheads, NULL, NULL);
- git_test_write_commit_graph_or_die(the_repository->objects->sources);
+ git_test_write_commit_graph_or_die(the_repository);
} else
write_merge_state(remoteheads);
diff --git a/commit-graph.c b/commit-graph.c
index 73814c1622..80ebc6a542 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -1,6 +1,7 @@
#define DISABLE_SIGN_COMPARE_WARNINGS
#include "git-compat-util.h"
+#include "abspath.h"
#include "config.h"
#include "csum-file.h"
#include "environment.h"
@@ -28,7 +29,7 @@
#include "tree.h"
#include "chunk-format.h"
-void git_test_write_commit_graph_or_die(struct odb_source *source)
+void git_test_write_commit_graph_or_die(struct repository *repo)
{
int flags = 0;
if (!git_env_bool(GIT_TEST_COMMIT_GRAPH, 0))
@@ -37,7 +38,7 @@ void git_test_write_commit_graph_or_die(struct odb_source *source)
if (git_env_bool(GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS, 0))
flags = COMMIT_GRAPH_WRITE_BLOOM_FILTERS;
- if (write_commit_graph_reachable(source, flags, NULL))
+ if (write_commit_graph_reachable(repo, repo->objects->sources->path, flags, NULL))
die("failed to write commit-graph under GIT_TEST_COMMIT_GRAPH");
}
@@ -196,21 +197,21 @@ static int commit_gen_cmp(const void *va, const void *vb)
return 0;
}
-char *get_commit_graph_filename(struct odb_source *source)
+char *get_commit_graph_filename(const char *dir)
{
- return xstrfmt("%s/info/commit-graph", source->path);
+ return xstrfmt("%s/info/commit-graph", dir);
}
-static char *get_split_graph_filename(struct odb_source *source,
+static char *get_split_graph_filename(const char *dir,
const char *oid_hex)
{
- return xstrfmt("%s/info/commit-graphs/graph-%s.graph", source->path,
+ return xstrfmt("%s/info/commit-graphs/graph-%s.graph", dir,
oid_hex);
}
-char *get_commit_graph_chain_filename(struct odb_source *source)
+char *get_commit_graph_chain_filename(const char *dir)
{
- return xstrfmt("%s/info/commit-graphs/commit-graph-chain", source->path);
+ return xstrfmt("%s/info/commit-graphs/commit-graph-chain", dir);
}
static struct commit_graph *alloc_commit_graph(void)
@@ -253,7 +254,8 @@ int open_commit_graph(const char *graph_file, int *fd, struct stat *st)
return 1;
}
-struct commit_graph *load_commit_graph_one_fd_st(struct odb_source *source,
+struct commit_graph *load_commit_graph_one_fd_st(struct repository *repo,
+ const char *dir,
int fd, struct stat *st)
{
void *graph_map;
@@ -262,7 +264,7 @@ struct commit_graph *load_commit_graph_one_fd_st(struct odb_source *source,
graph_size = xsize_t(st->st_size);
- if (graph_size < graph_min_size(source->odb->repo->hash_algo)) {
+ if (graph_size < graph_min_size(repo->hash_algo)) {
close(fd);
error(_("commit-graph file is too small"));
return NULL;
@@ -270,9 +272,9 @@ struct commit_graph *load_commit_graph_one_fd_st(struct odb_source *source,
graph_map = xmmap(NULL, graph_size, PROT_READ, MAP_PRIVATE, fd, 0);
close(fd);
- ret = parse_commit_graph(source->odb->repo, graph_map, graph_size);
+ ret = parse_commit_graph(repo, graph_map, graph_size);
if (ret)
- ret->odb_source = source;
+ ret->dir = absolute_pathdup(dir);
else
munmap(graph_map, graph_size);
@@ -410,6 +412,7 @@ struct commit_graph *parse_commit_graph(struct repository *r,
graph = alloc_commit_graph();
+ graph->repo = r;
graph->hash_algo = r->hash_algo;
graph->num_chunks = *(unsigned char*)(data + 6);
graph->data = graph_map;
@@ -490,7 +493,8 @@ struct commit_graph *parse_commit_graph(struct repository *r,
return NULL;
}
-static struct commit_graph *load_commit_graph_one(struct odb_source *source,
+static struct commit_graph *load_commit_graph_one(struct repository *repo,
+ const char *dir,
const char *graph_file)
{
struct stat st;
@@ -501,17 +505,18 @@ static struct commit_graph *load_commit_graph_one(struct odb_source *source,
if (!open_ok)
return NULL;
- g = load_commit_graph_one_fd_st(source, fd, &st);
+ g = load_commit_graph_one_fd_st(repo, dir, fd, &st);
if (g)
- g->filename = xstrdup(graph_file);
+ g->filename = absolute_pathdup(graph_file);
return g;
}
-static struct commit_graph *load_commit_graph_v1(struct odb_source *source)
+static struct commit_graph *load_commit_graph_v1(struct repository *repo,
+ const char *dir)
{
- char *graph_name = get_commit_graph_filename(source);
- struct commit_graph *g = load_commit_graph_one(source, graph_name);
+ char *graph_name = get_commit_graph_filename(dir);
+ struct commit_graph *g = load_commit_graph_one(repo, dir, graph_name);
free(graph_name);
return g;
@@ -666,8 +671,8 @@ struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,
valid = 0;
for (source = odb->sources; source; source = source->next) {
- char *graph_name = get_split_graph_filename(source, line.buf);
- struct commit_graph *g = load_commit_graph_one(source, graph_name);
+ char *graph_name = get_split_graph_filename(source->path, line.buf);
+ struct commit_graph *g = load_commit_graph_one(odb->repo, source->path, graph_name);
free(graph_name);
@@ -700,29 +705,31 @@ struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,
return graph_chain;
}
-static struct commit_graph *load_commit_graph_chain(struct odb_source *source)
+static struct commit_graph *load_commit_graph_chain(struct repository *repo,
+ const char *dir)
{
- char *chain_file = get_commit_graph_chain_filename(source);
+ char *chain_file = get_commit_graph_chain_filename(dir);
struct stat st;
int fd;
struct commit_graph *g = NULL;
- if (open_commit_graph_chain(chain_file, &fd, &st, source->odb->repo->hash_algo)) {
+ if (open_commit_graph_chain(chain_file, &fd, &st, repo->hash_algo)) {
int incomplete;
/* ownership of fd is taken over by load function */
- g = load_commit_graph_chain_fd_st(source->odb, fd, &st, &incomplete);
+ g = load_commit_graph_chain_fd_st(repo->objects, fd, &st, &incomplete);
}
free(chain_file);
return g;
}
-struct commit_graph *read_commit_graph_one(struct odb_source *source)
+struct commit_graph *read_commit_graph_one(struct repository *repo,
+ const char *dir)
{
- struct commit_graph *g = load_commit_graph_v1(source);
+ struct commit_graph *g = load_commit_graph_v1(repo, dir);
if (!g)
- g = load_commit_graph_chain(source);
+ g = load_commit_graph_chain(repo, dir);
return g;
}
@@ -767,7 +774,7 @@ static struct commit_graph *prepare_commit_graph(struct repository *r)
return NULL;
for (source = r->objects->sources; source; source = source->next) {
- r->objects->commit_graph = read_commit_graph_one(source);
+ r->objects->commit_graph = read_commit_graph_one(r, source->path);
if (r->objects->commit_graph)
break;
}
@@ -866,7 +873,7 @@ static struct commit_list **insert_parent_or_die(struct commit_graph *g,
die("invalid parent position %"PRIu32, pos);
load_oid_from_graph(g, pos, &oid);
- c = lookup_commit(g->odb_source->odb->repo, &oid);
+ c = lookup_commit(g->repo, &oid);
if (!c)
die(_("could not find commit %s"), oid_to_hex(&oid));
commit_graph_data_at(c)->graph_pos = pos;
@@ -1103,7 +1110,7 @@ static struct tree *load_tree_for_commit(struct commit_graph *g,
graph_pos - g->num_commits_in_base);
oidread(&oid, commit_data, g->hash_algo);
- set_commit_tree(c, lookup_tree(g->odb_source->odb->repo, &oid));
+ set_commit_tree(c, lookup_tree(g->repo, &oid));
return c->maybe_tree;
}
@@ -1126,7 +1133,7 @@ struct tree *get_commit_tree_in_graph(struct repository *r, const struct commit
struct write_commit_graph_context {
struct repository *r;
- struct odb_source *odb_source;
+ char *dir;
char *graph_name;
struct oid_array oids;
struct commit_stack commits;
@@ -1902,7 +1909,8 @@ static int add_ref_to_set(const struct reference *ref, void *cb_data)
return 0;
}
-int write_commit_graph_reachable(struct odb_source *source,
+int write_commit_graph_reachable(struct repository *repo,
+ const char *dir,
enum commit_graph_write_flags flags,
const struct commit_graph_opts *opts)
{
@@ -1911,20 +1919,20 @@ int write_commit_graph_reachable(struct odb_source *source,
int result;
memset(&data, 0, sizeof(data));
- data.repo = source->odb->repo;
+ data.repo = repo;
data.commits = &commits;
if (flags & COMMIT_GRAPH_WRITE_PROGRESS)
data.progress = start_delayed_progress(
- source->odb->repo,
+ repo,
_("Collecting referenced commits"), 0);
- refs_for_each_ref(get_main_ref_store(source->odb->repo), add_ref_to_set,
+ refs_for_each_ref(get_main_ref_store(repo), add_ref_to_set,
&data);
stop_progress(&data.progress);
- result = write_commit_graph(source, NULL, &commits,
+ result = write_commit_graph(repo, dir, NULL, &commits,
flags, opts);
oidset_clear(&commits);
@@ -2103,10 +2111,10 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)
strbuf_addf(&tmp_file,
"%s/info/commit-graphs/tmp_graph_XXXXXX",
- ctx->odb_source->path);
+ ctx->dir);
ctx->graph_name = strbuf_detach(&tmp_file, NULL);
} else {
- ctx->graph_name = get_commit_graph_filename(ctx->odb_source);
+ ctx->graph_name = get_commit_graph_filename(ctx->dir);
}
if (safe_create_leading_directories(ctx->r, ctx->graph_name)) {
@@ -2116,7 +2124,7 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)
}
if (ctx->split) {
- char *lock_name = get_commit_graph_chain_filename(ctx->odb_source);
+ char *lock_name = get_commit_graph_chain_filename(ctx->dir);
repo_hold_lock_file_for_update_mode(ctx->r, &lk, lock_name,
LOCK_DIE_ON_ERROR, 0444);
@@ -2205,7 +2213,7 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)
if (ctx->split && ctx->base_graph_name && ctx->num_commit_graphs_after > 1) {
char *new_base_hash = xstrdup(oid_to_hex(&ctx->new_base_graph->oid));
- char *new_base_name = get_split_graph_filename(ctx->new_base_graph->odb_source, new_base_hash);
+ char *new_base_name = get_split_graph_filename(ctx->new_base_graph->dir, new_base_hash);
free(ctx->commit_graph_filenames_after[ctx->num_commit_graphs_after - 2]);
free(ctx->commit_graph_hash_after[ctx->num_commit_graphs_after - 2]);
@@ -2245,7 +2253,7 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)
}
}
} else {
- char *graph_name = get_commit_graph_filename(ctx->odb_source);
+ char *graph_name = get_commit_graph_filename(ctx->dir);
unlink(graph_name);
free(graph_name);
}
@@ -2253,7 +2261,7 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)
free(ctx->commit_graph_hash_after[ctx->num_commit_graphs_after - 1]);
ctx->commit_graph_hash_after[ctx->num_commit_graphs_after - 1] =
xstrdup(hash_to_hex_algop(file_hash, ctx->r->hash_algo));
- final_graph_name = get_split_graph_filename(ctx->odb_source,
+ final_graph_name = get_split_graph_filename(ctx->dir,
ctx->commit_graph_hash_after[ctx->num_commit_graphs_after - 1]);
free(ctx->commit_graph_filenames_after[ctx->num_commit_graphs_after - 1]);
ctx->commit_graph_filenames_after[ctx->num_commit_graphs_after - 1] = final_graph_name;
@@ -2305,7 +2313,7 @@ static void split_graph_merge_strategy(struct write_commit_graph_context *ctx,
flags != COMMIT_GRAPH_SPLIT_REPLACE) {
while (g && (g->num_commits <= st_mult(size_mult, num_commits) ||
(max_commits && num_commits > max_commits))) {
- if (g->odb_source != ctx->odb_source)
+ if (strcmp(g->dir, ctx->dir))
break;
if (unsigned_add_overflows(num_commits, g->num_commits))
@@ -2327,10 +2335,10 @@ static void split_graph_merge_strategy(struct write_commit_graph_context *ctx,
"should be 1 with --split=replace");
if (ctx->num_commit_graphs_after == 2) {
- char *old_graph_name = get_commit_graph_filename(g->odb_source);
+ char *old_graph_name = get_commit_graph_filename(g->dir);
if (!strcmp(g->filename, old_graph_name) &&
- g->odb_source != ctx->odb_source) {
+ strcmp(g->dir, ctx->dir)) {
ctx->num_commit_graphs_after = 1;
ctx->new_base_graph = NULL;
}
@@ -2500,13 +2508,13 @@ static void expire_commit_graphs(struct write_commit_graph_context *ctx)
if (ctx->opts && ctx->opts->expire_time)
expire_time = ctx->opts->expire_time;
if (!ctx->split) {
- char *chain_file_name = get_commit_graph_chain_filename(ctx->odb_source);
+ char *chain_file_name = get_commit_graph_chain_filename(ctx->dir);
unlink(chain_file_name);
free(chain_file_name);
ctx->num_commit_graphs_after = 0;
}
- strbuf_addstr(&path, ctx->odb_source->path);
+ strbuf_addstr(&path, ctx->dir);
strbuf_addstr(&path, "/info/commit-graphs");
dir = opendir(path.buf);
@@ -2548,16 +2556,15 @@ static void expire_commit_graphs(struct write_commit_graph_context *ctx)
strbuf_release(&path);
}
-int write_commit_graph(struct odb_source *source,
+int write_commit_graph(struct repository *r,
+ const char *dir,
const struct string_list *const pack_indexes,
struct oidset *commits,
enum commit_graph_write_flags flags,
const struct commit_graph_opts *opts)
{
- struct repository *r = source->odb->repo;
struct write_commit_graph_context ctx = {
.r = r,
- .odb_source = source,
.append = flags & COMMIT_GRAPH_WRITE_APPEND ? 1 : 0,
.report_progress = flags & COMMIT_GRAPH_WRITE_PROGRESS ? 1 : 0,
.split = flags & COMMIT_GRAPH_WRITE_SPLIT ? 1 : 0,
@@ -2588,6 +2595,8 @@ int write_commit_graph(struct odb_source *source,
return 0;
}
+ ctx.dir = absolute_pathdup(dir);
+
bloom_settings.hash_version = r->settings.commit_graph_changed_paths_version;
bloom_settings.bits_per_entry = git_env_ulong("GIT_TEST_BLOOM_SETTINGS_BITS_PER_ENTRY",
bloom_settings.bits_per_entry);
@@ -2710,6 +2719,7 @@ int write_commit_graph(struct odb_source *source,
cleanup:
free(ctx.graph_name);
free(ctx.base_graph_name);
+ free(ctx.dir);
commit_stack_clear(&ctx.commits);
oid_array_clear(&ctx.oids);
clear_topo_level_slab(&topo_levels);
@@ -2762,7 +2772,7 @@ static int verify_one_commit_graph(struct commit_graph *g,
struct progress *progress,
uint64_t *seen)
{
- struct repository *r = g->odb_source->odb->repo;
+ struct repository *r = g->repo;
uint32_t i, cur_fanout_pos = 0;
struct object_id prev_oid, cur_oid;
struct commit *seen_gen_zero = NULL;
@@ -2926,7 +2936,7 @@ int verify_commit_graph(struct commit_graph *g, int flags)
if (!(flags & COMMIT_GRAPH_VERIFY_SHALLOW))
total += g->num_commits_in_base;
- progress = start_progress(g->odb_source->odb->repo,
+ progress = start_progress(g->repo,
_("Verifying commits in commit graph"),
total);
}
@@ -2949,6 +2959,7 @@ void free_commit_graph(struct commit_graph *g)
if (g->data)
munmap((void *)g->data, g->data_len);
+ free(g->dir);
free(g->filename);
free(g->bloom_filter_settings);
free(g);
diff --git a/commit-graph.h b/commit-graph.h
index 13ca4ff010..bccf9c5c84 100644
--- a/commit-graph.h
+++ b/commit-graph.h
@@ -21,7 +21,7 @@
* call this method outside of a builtin, and only if you know what
* you are doing!
*/
-void git_test_write_commit_graph_or_die(struct odb_source *source);
+void git_test_write_commit_graph_or_die(struct repository *repo);
struct commit;
struct bloom_filter_settings;
@@ -29,8 +29,8 @@ struct repository;
struct object_database;
struct string_list;
-char *get_commit_graph_filename(struct odb_source *source);
-char *get_commit_graph_chain_filename(struct odb_source *source);
+char *get_commit_graph_filename(const char *dir);
+char *get_commit_graph_chain_filename(const char *dir);
int open_commit_graph(const char *graph_file, int *fd, struct stat *st);
int open_commit_graph_chain(const char *chain_file, int *fd, struct stat *st,
const struct git_hash_algo *hash_algo);
@@ -85,12 +85,13 @@ struct commit_graph {
const unsigned char *data;
size_t data_len;
+ struct repository *repo;
const struct git_hash_algo *hash_algo;
unsigned char num_chunks;
uint32_t num_commits;
struct object_id oid;
+ char *dir;
char *filename;
- struct odb_source *odb_source;
uint32_t num_commits_in_base;
unsigned int read_generation_data;
@@ -114,12 +115,20 @@ struct commit_graph {
struct bloom_filter_settings *bloom_filter_settings;
};
-struct commit_graph *load_commit_graph_one_fd_st(struct odb_source *source,
+/*
+ * Load commit graphs from the given object directory `dir`. The directory may
+ * be given as a relative path; it is canonicalized internally so that graphs
+ * loaded from the same directory compare equal regardless of how the caller
+ * spelled the path.
+ */
+struct commit_graph *load_commit_graph_one_fd_st(struct repository *repo,
+ const char *dir,
int fd, struct stat *st);
struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,
int fd, struct stat *st,
int *incomplete_chain);
-struct commit_graph *read_commit_graph_one(struct odb_source *source);
+struct commit_graph *read_commit_graph_one(struct repository *repo,
+ const char *dir);
struct repo_settings;
@@ -171,11 +180,17 @@ struct commit_graph_opts {
* and a negative value on failure. Note that if the repository
* is not compatible with the commit-graph feature, then the
* methods will return 0 without writing a commit-graph.
+ *
+ * The object directory `dir` may be given as a relative path; it is
+ * canonicalized internally so that it compares equal to the directory of
+ * graphs that have already been loaded.
*/
-int write_commit_graph_reachable(struct odb_source *source,
+int write_commit_graph_reachable(struct repository *repo,
+ const char *dir,
enum commit_graph_write_flags flags,
const struct commit_graph_opts *opts);
-int write_commit_graph(struct odb_source *source,
+int write_commit_graph(struct repository *r,
+ const char *dir,
const struct string_list *pack_indexes,
struct oidset *commits,
enum commit_graph_write_flags flags,
diff --git a/t/helper/test-read-graph.c b/t/helper/test-read-graph.c
index 9f07b9c25a..a75c817e47 100644
--- a/t/helper/test-read-graph.c
+++ b/t/helper/test-read-graph.c
@@ -4,6 +4,7 @@
#include "commit-graph.h"
#include "repository.h"
#include "odb.h"
+#include "odb/source.h"
#include "bloom.h"
#include "setup.h"
@@ -81,7 +82,7 @@ int cmd__read_graph(int argc, const char **argv)
prepare_repo_settings(the_repository);
- graph = read_commit_graph_one(source);
+ graph = read_commit_graph_one(the_repository, source->path);
if (!graph) {
ret = 1;
goto done;
diff --git a/t/t4216-log-bloom.sh b/t/t4216-log-bloom.sh
index ad2686669d..f57a3d6621 100755
--- a/t/t4216-log-bloom.sh
+++ b/t/t4216-log-bloom.sh
@@ -755,7 +755,7 @@ test_expect_success PERL_TEST_HELPERS 'Bloom reader notices too-small data chunk
test_expect_success PERL_TEST_HELPERS 'Bloom reader notices out-of-bounds filter offsets' '
check_corrupt_graph BIDX 12 FFFFFFFF &&
# use grep to avoid depending on exact chunk size
- test_grep "warning: ignoring out-of-range offset (4294967295) for changed-path filter at pos 3 of .git/objects/info/commit-graph" err
+ test_grep "warning: ignoring out-of-range offset (4294967295) for changed-path filter at pos 3 of $(pwd)/.git/objects/info/commit-graph" err
'
test_expect_success PERL_TEST_HELPERS 'Bloom reader notices too-small index chunk' '
@@ -773,7 +773,7 @@ test_expect_success PERL_TEST_HELPERS 'Bloom reader notices out-of-order index o
# actually reading from the bogus offsets anyway.
corrupt_graph BIDX 4 0000000c00000005 &&
echo "warning: ignoring decreasing changed-path index offsets" \
- "(12 > 5) for positions 1 and 2 of .git/objects/info/commit-graph" >expect.err &&
+ "(12 > 5) for positions 1 and 2 of $(pwd)/.git/objects/info/commit-graph" >expect.err &&
git -c core.commitGraph=false log -- A/B/file2 >expect.out &&
git -c core.commitGraph=true log -- A/B/file2 >out 2>err &&
test_cmp expect.out out &&
--
2.56.0.379.gc618271300.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* Re: [PATCH 02/13] commit-graph: stop depending on `struct odb_source`
2026-10-02 10:08 ` [PATCH 02/13] commit-graph: stop depending on `struct odb_source` Patrick Steinhardt
@ 2026-10-05 19:43 ` Karthik Nayak
2026-10-06 12:18 ` Patrick Steinhardt
0 siblings, 1 reply; 41+ messages in thread
From: Karthik Nayak @ 2026-10-05 19:43 UTC (permalink / raw)
To: Patrick Steinhardt, git
[-- Attachment #1: Type: text/plain, Size: 926 bytes --]
Patrick Steinhardt <ps@pks.im> writes:
[snip]
> @@ -28,7 +29,7 @@
> #include "tree.h"
> #include "chunk-format.h"
>
> -void git_test_write_commit_graph_or_die(struct odb_source *source)
> +void git_test_write_commit_graph_or_die(struct repository *repo)
> {
> int flags = 0;
> if (!git_env_bool(GIT_TEST_COMMIT_GRAPH, 0))
> @@ -37,7 +38,7 @@ void git_test_write_commit_graph_or_die(struct odb_source *source)
> if (git_env_bool(GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS, 0))
> flags = COMMIT_GRAPH_WRITE_BLOOM_FILTERS;
>
> - if (write_commit_graph_reachable(source, flags, NULL))
> + if (write_commit_graph_reachable(repo, repo->objects->sources->path, flags, NULL))
> die("failed to write commit-graph under GIT_TEST_COMMIT_GRAPH");
> }
>
Shouldn't the caller of `git_test_write_commit_graph_or_die()` send in
(repo, path) and we forward that path, instead of using the path from
`repo->objects->sources->path`?
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 41+ messages in thread* Re: [PATCH 02/13] commit-graph: stop depending on `struct odb_source`
2026-10-05 19:43 ` Karthik Nayak
@ 2026-10-06 12:18 ` Patrick Steinhardt
2026-10-06 20:11 ` Karthik Nayak
0 siblings, 1 reply; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-06 12:18 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git
On Mon, Oct 05, 2026 at 03:43:44PM -0400, Karthik Nayak wrote:
> Patrick Steinhardt <ps@pks.im> writes:
>
> [snip]
Thanks for trimming! One more ask though: it's helpful to retain the
diff header itself so that one knows which file this is that you are
commenting on :)
> > @@ -28,7 +29,7 @@
> > #include "tree.h"
> > #include "chunk-format.h"
> >
> > -void git_test_write_commit_graph_or_die(struct odb_source *source)
> > +void git_test_write_commit_graph_or_die(struct repository *repo)
> > {
> > int flags = 0;
> > if (!git_env_bool(GIT_TEST_COMMIT_GRAPH, 0))
> > @@ -37,7 +38,7 @@ void git_test_write_commit_graph_or_die(struct odb_source *source)
> > if (git_env_bool(GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS, 0))
> > flags = COMMIT_GRAPH_WRITE_BLOOM_FILTERS;
> >
> > - if (write_commit_graph_reachable(source, flags, NULL))
> > + if (write_commit_graph_reachable(repo, repo->objects->sources->path, flags, NULL))
> > die("failed to write commit-graph under GIT_TEST_COMMIT_GRAPH");
> > }
> >
>
> Shouldn't the caller of `git_test_write_commit_graph_or_die()` send in
> (repo, path) and we forward that path, instead of using the path from
> `repo->objects->sources->path`?
I'd agree if this were a properly designed function. But it's basically
just a hack for our test suite, so I was aiming for the easiest fix
possible to make this work.
Patrick
^ permalink raw reply [flat|nested] 41+ messages in thread* Re: [PATCH 02/13] commit-graph: stop depending on `struct odb_source`
2026-10-06 12:18 ` Patrick Steinhardt
@ 2026-10-06 20:11 ` Karthik Nayak
0 siblings, 0 replies; 41+ messages in thread
From: Karthik Nayak @ 2026-10-06 20:11 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: git
[-- Attachment #1: Type: text/plain, Size: 1486 bytes --]
Patrick Steinhardt <ps@pks.im> writes:
> On Mon, Oct 05, 2026 at 03:43:44PM -0400, Karthik Nayak wrote:
>> Patrick Steinhardt <ps@pks.im> writes:
>>
>> [snip]
>
> Thanks for trimming! One more ask though: it's helpful to retain the
> diff header itself so that one knows which file this is that you are
> commenting on :)
>
Sure! Next time!
>> > @@ -28,7 +29,7 @@
>> > #include "tree.h"
>> > #include "chunk-format.h"
>> >
>> > -void git_test_write_commit_graph_or_die(struct odb_source *source)
>> > +void git_test_write_commit_graph_or_die(struct repository *repo)
>> > {
>> > int flags = 0;
>> > if (!git_env_bool(GIT_TEST_COMMIT_GRAPH, 0))
>> > @@ -37,7 +38,7 @@ void git_test_write_commit_graph_or_die(struct odb_source *source)
>> > if (git_env_bool(GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS, 0))
>> > flags = COMMIT_GRAPH_WRITE_BLOOM_FILTERS;
>> >
>> > - if (write_commit_graph_reachable(source, flags, NULL))
>> > + if (write_commit_graph_reachable(repo, repo->objects->sources->path, flags, NULL))
>> > die("failed to write commit-graph under GIT_TEST_COMMIT_GRAPH");
>> > }
>> >
>>
>> Shouldn't the caller of `git_test_write_commit_graph_or_die()` send in
>> (repo, path) and we forward that path, instead of using the path from
>> `repo->objects->sources->path`?
>
> I'd agree if this were a properly designed function. But it's basically
> just a hack for our test suite, so I was aiming for the easiest fix
> possible to make this work.
>
> Patrick
All good!
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 41+ messages in thread
* [PATCH 03/13] odb/source-files: introduce `struct odb_files_dir`
2026-10-02 10:08 [PATCH 00/13] odb/source-files: move alternates into the backend Patrick Steinhardt
2026-10-02 10:08 ` [PATCH 01/13] commit-graph: require resolved packfile paths for `stdin_packs` Patrick Steinhardt
2026-10-02 10:08 ` [PATCH 02/13] commit-graph: stop depending on `struct odb_source` Patrick Steinhardt
@ 2026-10-02 10:08 ` Patrick Steinhardt
2026-10-02 10:08 ` [PATCH 04/13] odb: refactor `odb_for_each_alternate()` to yield dirs Patrick Steinhardt
` (11 subsequent siblings)
14 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-02 10:08 UTC (permalink / raw)
To: git
The "files" object database source consists of two separate backends for
storing loose and packed objects. These are managed as somewhat separate
entities even though they derive from the same object directory, whether
it's the primary object directory or that one from an alternate.
In a subsequent commit we'll move the handling of alternates into the
"files" backend completely so that it becomes another implementation
detail thereof. As part of this, we'll want to keep track of both of
these sub-sources as a single entity derived from their respective
object directory.
Prepare for this change by introducing a new `struct odb_files_dir` that
encapsulates them. For now, every "files" source has exactly one such
directory. In a subsequent commit though, we'll make it a linked list of
directories so that we can manage multiple such directories in a single
"files" source.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fast-import.c | 6 ++--
builtin/index-pack.c | 2 +-
builtin/multi-pack-index.c | 14 ++++-----
builtin/pack-objects.c | 18 +++++------
builtin/repack.c | 4 +--
commit-graph.c | 2 +-
http-walker.c | 2 +-
http.c | 6 ++--
loose.c | 6 ++--
midx.c | 26 ++++++++--------
odb/source-files.c | 77 ++++++++++++++++++++++++++++++----------------
odb/source-files.h | 27 ++++++++++++++--
pack-bitmap.c | 2 +-
packfile.c | 8 ++---
packfile.h | 4 +--
repack-geometry.c | 2 +-
repack-midx.c | 6 ++--
repack.c | 2 +-
t/helper/test-read-midx.c | 2 +-
19 files changed, 131 insertions(+), 85 deletions(-)
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index fbd919982c..0bf76b028b 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -921,7 +921,7 @@ static void end_packfile(void)
idx_name = keep_pack(create_index());
/* Register the packfile with core git's machinery. */
- new_p = packfile_store_load_pack(files->packed, idx_name, 1);
+ new_p = packfile_store_load_pack(files->dirs->packed, idx_name, 1);
if (!new_p)
die(_("core Git rejected index %s"), idx_name);
all_packs[pack_id] = new_p;
@@ -1005,7 +1005,7 @@ static int store_object(
for (source = the_repository->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- if (!packfile_list_find_oid(packfile_store_get_packs(files->packed), &oid))
+ if (!packfile_list_find_oid(packfile_store_get_packs(files->dirs->packed), &oid))
continue;
e->type = type;
e->pack_id = MAX_PACK_ID;
@@ -1215,7 +1215,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
for (source = the_repository->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- if (!packfile_list_find_oid(packfile_store_get_packs(files->packed), &oid))
+ if (!packfile_list_find_oid(packfile_store_get_packs(files->dirs->packed), &oid))
continue;
e->type = OBJ_BLOB;
e->pack_id = MAX_PACK_ID;
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 6b2a87e2d3..70860b8f27 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1641,7 +1641,7 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
if (do_fsck_object && startup_info->have_repository) {
struct odb_source_files *files =
odb_source_files_downcast(the_repository->objects->sources);
- packfile_store_load_pack(files->packed, final_index_name, 0);
+ packfile_store_load_pack(files->dirs->packed, final_index_name, 0);
}
if (!from_stdin) {
diff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c
index 753bd53a70..a170ec80b9 100644
--- a/builtin/multi-pack-index.c
+++ b/builtin/multi-pack-index.c
@@ -213,7 +213,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,
read_packs_from_stdin(&packs);
- ret = write_midx_file_only(source->packed, &packs,
+ ret = write_midx_file_only(source->dirs->packed, &packs,
opts.preferred_pack,
opts.refs_snapshot,
opts.incremental_base, opts.flags);
@@ -225,7 +225,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,
}
- ret = write_midx_file(source->packed, opts.preferred_pack,
+ ret = write_midx_file(source->dirs->packed, opts.preferred_pack,
opts.refs_snapshot, opts.flags);
free(opts.refs_snapshot);
@@ -284,7 +284,7 @@ static int cmd_multi_pack_index_compact(int argc, const char **argv,
FREE_AND_NULL(options);
- m = get_multi_pack_index(source->packed);
+ m = get_multi_pack_index(source->dirs->packed);
for (cur = m; cur && !(from_midx && to_midx); cur = cur->base_midx) {
const char *midx_csum = midx_get_checksum_hex(cur);
@@ -307,7 +307,7 @@ static int cmd_multi_pack_index_compact(int argc, const char **argv,
die(_("MIDX %s must be an ancestor of %s"), argv[0], argv[1]);
}
- ret = write_midx_file_compact(source->packed, from_midx, to_midx,
+ ret = write_midx_file_compact(source->dirs->packed, from_midx, to_midx,
opts.incremental_base, opts.flags);
return ret;
@@ -339,7 +339,7 @@ static int cmd_multi_pack_index_verify(int argc, const char **argv,
FREE_AND_NULL(options);
- return verify_midx_file(source->packed, opts.flags);
+ return verify_midx_file(source->dirs->packed, opts.flags);
}
static int cmd_multi_pack_index_expire(int argc, const char **argv,
@@ -368,7 +368,7 @@ static int cmd_multi_pack_index_expire(int argc, const char **argv,
FREE_AND_NULL(options);
- return expire_midx_packs(source->packed, opts.flags);
+ return expire_midx_packs(source->dirs->packed, opts.flags);
}
static int cmd_multi_pack_index_repack(int argc, const char **argv,
@@ -400,7 +400,7 @@ static int cmd_multi_pack_index_repack(int argc, const char **argv,
FREE_AND_NULL(options);
- return midx_repack(source->packed, (size_t)opts.batch_size, opts.flags);
+ return midx_repack(source->dirs->packed, (size_t)opts.batch_size, opts.flags);
}
int cmd_multi_pack_index(int argc,
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index af9390a46b..070659b6ed 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1570,7 +1570,7 @@ static int want_cruft_object_mtime(struct repository *r,
for (source = r->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- struct packed_git **cache = packfile_store_get_kept_pack_cache(files->packed, flags);
+ struct packed_git **cache = packfile_store_get_kept_pack_cache(files->dirs->packed, flags);
for (; *cache; cache++) {
struct packed_git *p = *cache;
@@ -1765,7 +1765,7 @@ static int want_object_in_pack_mtime(const struct object_id *oid,
struct odb_source *source = the_repository->objects->sources->next;
for (; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- if (!odb_source_read_object_info(&files->loose->base, oid, NULL, 0, NULL))
+ if (!odb_source_read_object_info(&files->dirs->loose->base, oid, NULL, 0, NULL))
return 0;
}
}
@@ -1787,7 +1787,7 @@ static int want_object_in_pack_mtime(const struct object_id *oid,
for (source = the_repository->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- struct multi_pack_index *m = get_multi_pack_index(files->packed);
+ struct multi_pack_index *m = get_multi_pack_index(files->dirs->packed);
struct pack_entry e;
if (m && midx_fill_entry(m, oid, &e, NULL) == MIDX_FILL_HIT) {
@@ -1800,11 +1800,11 @@ static int want_object_in_pack_mtime(const struct object_id *oid,
for (source = the_repository->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- for (e = files->packed->packs.head; e; e = e->next) {
+ for (e = files->dirs->packed->packs.head; e; e = e->next) {
struct packed_git *p = e->pack;
want = want_object_in_pack_one(p, oid, exclude, found_pack, found_offset, found_mtime);
if (!exclude && want > 0)
- packfile_list_prepend(&files->packed->packs, p);
+ packfile_list_prepend(&files->dirs->packed->packs, p);
if (want != -1)
return want;
}
@@ -4175,7 +4175,7 @@ static void add_cruft_object_entry(const struct object_id *oid, enum object_type
for (; !found && source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- if (!odb_source_read_object_info(&files->loose->base, oid, NULL, 0, NULL))
+ if (!odb_source_read_object_info(&files->dirs->loose->base, oid, NULL, 0, NULL))
found = 1;
}
@@ -4532,7 +4532,7 @@ static void add_objects_in_unpacked_packs(void)
if (!source->local)
continue;
- if (odb_source_for_each_object(&files->packed->base, &oi,
+ if (odb_source_for_each_object(&files->dirs->packed->base, &oi,
add_object_in_unpacked_pack, NULL, &opts))
die(_("cannot open pack index"));
}
@@ -4642,7 +4642,7 @@ static int force_object_loose(struct odb_source *source,
for (struct odb_source *s = source->odb->sources; s; s = s->next) {
struct odb_source_files *files = odb_source_files_downcast(s);
- if (!odb_source_read_object_info(&files->loose->base, oid, NULL, 0, NULL))
+ if (!odb_source_read_object_info(&files->dirs->loose->base, oid, NULL, 0, NULL))
return 0;
}
@@ -4664,7 +4664,7 @@ static int force_object_loose(struct odb_source *source,
compat_oid_p = &compat_oid;
}
- ret = odb_source_write_object(&files->loose->base, buf, len, type, oid,
+ ret = odb_source_write_object(&files->dirs->loose->base, buf, len, type, oid,
compat_oid_p, mtime, 0);
out:
diff --git a/builtin/repack.c b/builtin/repack.c
index c4360382c1..5d06872d77 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -617,7 +617,7 @@ int cmd_repack(int argc,
* midx_has_unknown_packs() will make the decision for
* us.
*/
- if (!get_multi_pack_index(files->packed))
+ if (!get_multi_pack_index(files->dirs->packed))
midx_must_contain_cruft = 1;
}
@@ -775,7 +775,7 @@ int cmd_repack(int argc,
if (git_env_bool(GIT_TEST_MULTI_PACK_INDEX_WRITE_INCREMENTAL, 0))
flags |= MIDX_WRITE_INCREMENTAL;
- write_midx_file(files->packed, NULL, NULL, flags);
+ write_midx_file(files->dirs->packed, NULL, NULL, flags);
}
cleanup:
diff --git a/commit-graph.c b/commit-graph.c
index 80ebc6a542..7cc486d140 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -2020,7 +2020,7 @@ static void fill_oids_from_all_packs(struct write_commit_graph_context *ctx)
for (source = ctx->r->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- odb_source_for_each_object(&files->packed->base, &oi, add_packed_commits_oi,
+ odb_source_for_each_object(&files->dirs->packed->base, &oi, add_packed_commits_oi,
ctx, &opts);
}
diff --git a/http-walker.c b/http-walker.c
index abafca84d6..0a6c99f471 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -542,7 +542,7 @@ static int fetch_object(struct walker *walker, const struct object_id *oid)
} else if (req->rename < 0) {
struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->sources);
struct strbuf buf = STRBUF_INIT;
- odb_loose_path(files->loose, &buf, &req->oid);
+ odb_loose_path(files->dirs->loose, &buf, &req->oid);
ret = error("unable to write sha1 filename %s", buf.buf);
strbuf_release(&buf);
}
diff --git a/http.c b/http.c
index c8fcfd7693..fe6ec88a21 100644
--- a/http.c
+++ b/http.c
@@ -2720,7 +2720,7 @@ void http_install_packfile(struct packed_git *p,
{
struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->sources);
packfile_list_remove(list_to_remove_from, p);
- packfile_store_add_pack(files->packed, p);
+ packfile_store_add_pack(files->dirs->packed, p);
}
struct http_pack_request *new_http_pack_request(
@@ -2861,7 +2861,7 @@ struct http_object_request *new_http_object_request(const char *base_url,
oidcpy(&freq->oid, oid);
freq->localfile = -1;
- odb_loose_path(files->loose, &filename, oid);
+ odb_loose_path(files->dirs->loose, &filename, oid);
strbuf_addf(&freq->tmpfile, "%s.temp", filename.buf);
strbuf_addf(&prevfile, "%s.prev", filename.buf);
@@ -3014,7 +3014,7 @@ int finish_http_object_request(struct http_object_request *freq)
unlink_or_warn(freq->tmpfile.buf);
return -1;
}
- odb_loose_path(files->loose, &filename, &freq->oid);
+ odb_loose_path(files->dirs->loose, &filename, &freq->oid);
freq->rename = finalize_object_file(the_repository, freq->tmpfile.buf, filename.buf);
strbuf_release(&filename);
diff --git a/loose.c b/loose.c
index c159d29d2d..957bf83e6b 100644
--- a/loose.c
+++ b/loose.c
@@ -117,7 +117,7 @@ int repo_read_loose_object_map(struct repository *repo)
for (source = repo->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- if (loose_object_map_load(files->loose) < 0)
+ if (loose_object_map_load(files->dirs->loose) < 0)
return -1;
}
@@ -127,7 +127,7 @@ int repo_read_loose_object_map(struct repository *repo)
int repo_write_loose_object_map(struct repository *repo)
{
struct odb_source_files *files = odb_source_files_downcast(repo->objects->sources);
- kh_oid_map_t *map = files->loose->map->to_compat;
+ kh_oid_map_t *map = files->dirs->loose->map->to_compat;
struct lock_file lock;
int fd;
khiter_t iter;
@@ -237,7 +237,7 @@ int repo_loose_object_map_oid(struct repository *repo,
for (source = repo->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- struct loose_object_map *loose_map = files->loose->map;
+ struct loose_object_map *loose_map = files->dirs->loose->map;
if (!loose_map)
continue;
map = (to == repo->compat_hash_algo) ?
diff --git a/midx.c b/midx.c
index 6d1c548e3d..c0f82c4163 100644
--- a/midx.c
+++ b/midx.c
@@ -837,20 +837,20 @@ void clear_midx_file(struct repository *r)
for (source = r->objects->sources; source; source = source->next) {
files = odb_source_files_downcast(source);
- if (files->packed->midx)
- close_midx(files->packed->midx);
- files->packed->midx = NULL;
+ if (files->dirs->packed->midx)
+ close_midx(files->dirs->packed->midx);
+ files->dirs->packed->midx = NULL;
}
}
files = odb_source_files_downcast(r->objects->sources);
- get_midx_filename(files->packed, &midx);
+ get_midx_filename(files->dirs->packed, &midx);
if (remove_path(midx.buf))
die(_("failed to clear multi-pack-index at %s"), midx.buf);
- clear_midx_files_ext(files->packed, MIDX_EXT_BITMAP, NULL);
- clear_midx_files_ext(files->packed, MIDX_EXT_REV, NULL);
+ clear_midx_files_ext(files->dirs->packed, MIDX_EXT_BITMAP, NULL);
+ clear_midx_files_ext(files->dirs->packed, MIDX_EXT_REV, NULL);
strbuf_release(&midx);
}
@@ -864,21 +864,21 @@ void clear_incremental_midx_files(struct repository *r,
for (source = r->objects->sources; source; source = source->next) {
files = odb_source_files_downcast(source);
- if (files->packed->midx)
- close_midx(files->packed->midx);
- files->packed->midx = NULL;
+ if (files->dirs->packed->midx)
+ close_midx(files->dirs->packed->midx);
+ files->dirs->packed->midx = NULL;
}
files = odb_source_files_downcast(r->objects->sources);
- get_midx_chain_filename(files->packed, &chain);
+ get_midx_chain_filename(files->dirs->packed, &chain);
if (!keep_hashes && remove_path(chain.buf))
die(_("failed to clear multi-pack-index chain at %s"),
chain.buf);
- clear_incremental_midx_files_ext(files->packed, MIDX_EXT_BITMAP, keep_hashes);
- clear_incremental_midx_files_ext(files->packed, MIDX_EXT_REV, keep_hashes);
- clear_incremental_midx_files_ext(files->packed, MIDX_EXT_MIDX, keep_hashes);
+ clear_incremental_midx_files_ext(files->dirs->packed, MIDX_EXT_BITMAP, keep_hashes);
+ clear_incremental_midx_files_ext(files->dirs->packed, MIDX_EXT_REV, keep_hashes);
+ clear_incremental_midx_files_ext(files->dirs->packed, MIDX_EXT_MIDX, keep_hashes);
strbuf_release(&chain);
}
diff --git a/odb/source-files.c b/odb/source-files.c
index f2fc4cd9ab..1f4cedaffa 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -24,6 +24,30 @@
#include "tree.h"
#include "write-or-die.h"
+struct odb_files_dir *odb_files_dir_new(struct object_database *odb,
+ const char *path, bool local)
+{
+ struct odb_files_dir *dir;
+
+ CALLOC_ARRAY(dir, 1);
+ dir->abspath = absolute_pathdup(path);
+ dir->local = local;
+ dir->loose = odb_source_loose_new(odb, path, local);
+ dir->packed = odb_source_packed_new(odb, path, local);
+
+ return dir;
+}
+
+void odb_files_dir_free(struct odb_files_dir *dir)
+{
+ if (!dir)
+ return;
+ odb_source_free(&dir->loose->base);
+ odb_source_free(&dir->packed->base);
+ free(dir->abspath);
+ free(dir);
+}
+
static void odb_source_files_reparent(const char *old_cwd,
const char *new_cwd,
void *cb_data)
@@ -31,6 +55,7 @@ static void odb_source_files_reparent(const char *old_cwd,
struct odb_source_files *files = cb_data;
char *path = reparent_relative_path(old_cwd, new_cwd,
files->base.path);
+
free(files->base.path);
files->base.path = path;
}
@@ -39,8 +64,7 @@ static void odb_source_files_free(struct odb_source *source)
{
struct odb_source_files *files = odb_source_files_downcast(source);
chdir_notify_unregister(odb_source_files_reparent, files);
- odb_source_free(&files->loose->base);
- odb_source_free(&files->packed->base);
+ odb_files_dir_free(files->dirs);
odb_source_release(&files->base);
free(files);
}
@@ -48,8 +72,8 @@ static void odb_source_files_free(struct odb_source *source)
static void odb_source_files_close(struct odb_source *source)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- odb_source_close(&files->loose->base);
- odb_source_close(&files->packed->base);
+ odb_source_close(&files->dirs->loose->base);
+ odb_source_close(&files->dirs->packed->base);
}
static int odb_source_files_create_on_disk(struct odb_source *source,
@@ -144,8 +168,8 @@ static void odb_source_files_prepare(struct odb_source *source,
enum odb_prepare_flags flags)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- odb_source_prepare(&files->loose->base, flags);
- odb_source_prepare(&files->packed->base, flags);
+ odb_source_prepare(&files->dirs->loose->base, flags);
+ odb_source_prepare(&files->dirs->packed->base, flags);
}
static enum odb_read_status odb_source_files_read_object_info(struct odb_source *source,
@@ -157,12 +181,12 @@ static enum odb_read_status odb_source_files_read_object_info(struct odb_source
struct odb_source_files *files = odb_source_files_downcast(source);
enum odb_read_status ret_packed, ret_loose;
- ret_packed = odb_source_read_object_info(&files->packed->base, oid, oi,
+ ret_packed = odb_source_read_object_info(&files->dirs->packed->base, oid, oi,
flags, errmsg);
if (!ret_packed)
return 0;
- ret_loose = odb_source_read_object_info(&files->loose->base, oid, oi, flags,
+ ret_loose = odb_source_read_object_info(&files->dirs->loose->base, oid, oi, flags,
ret_packed == ODB_READ_NOT_FOUND ? errmsg : NULL);
if (!ret_loose)
return 0;
@@ -184,8 +208,8 @@ static int odb_source_files_read_object_stream(struct odb_stream **out,
const struct object_id *oid)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- if (!odb_source_read_object_stream(out, &files->packed->base, oid) ||
- !odb_source_read_object_stream(out, &files->loose->base, oid))
+ if (!odb_source_read_object_stream(out, &files->dirs->packed->base, oid) ||
+ !odb_source_read_object_stream(out, &files->dirs->loose->base, oid))
return 0;
return -1;
}
@@ -200,12 +224,12 @@ static int odb_source_files_for_each_object(struct odb_source *source,
int ret;
if (!(opts->flags & ODB_FOR_EACH_OBJECT_PROMISOR_ONLY)) {
- ret = odb_source_for_each_object(&files->loose->base, request, cb, cb_data, opts);
+ ret = odb_source_for_each_object(&files->dirs->loose->base, request, cb, cb_data, opts);
if (ret)
return ret;
}
- ret = odb_source_for_each_object(&files->packed->base, request, cb, cb_data, opts);
+ ret = odb_source_for_each_object(&files->dirs->packed->base, request, cb, cb_data, opts);
if (ret)
return ret;
@@ -220,14 +244,14 @@ static int odb_source_files_count_objects(struct odb_source *source,
unsigned long count;
int ret;
- ret = odb_source_count_objects(&files->packed->base, flags, &count);
+ ret = odb_source_count_objects(&files->dirs->packed->base, flags, &count);
if (ret < 0)
goto out;
if (!(flags & ODB_COUNT_OBJECTS_APPROXIMATE)) {
unsigned long loose_count;
- ret = odb_source_count_objects(&files->loose->base, flags, &loose_count);
+ ret = odb_source_count_objects(&files->dirs->loose->base, flags, &loose_count);
if (ret < 0)
goto out;
@@ -250,11 +274,11 @@ static int odb_source_files_find_abbrev_len(struct odb_source *source,
unsigned len = min_len;
int ret;
- ret = odb_source_find_abbrev_len(&files->packed->base, oid, len, &len);
+ ret = odb_source_find_abbrev_len(&files->dirs->packed->base, oid, len, &len);
if (ret < 0)
goto out;
- ret = odb_source_find_abbrev_len(&files->loose->base, oid, len, &len);
+ ret = odb_source_find_abbrev_len(&files->dirs->loose->base, oid, len, &len);
if (ret < 0)
goto out;
@@ -270,8 +294,8 @@ static int odb_source_files_freshen_object(struct odb_source *source,
const time_t *mtime)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- if (odb_source_freshen_object(&files->packed->base, oid, mtime) ||
- odb_source_freshen_object(&files->loose->base, oid, mtime))
+ if (odb_source_freshen_object(&files->dirs->packed->base, oid, mtime) ||
+ odb_source_freshen_object(&files->dirs->loose->base, oid, mtime))
return 1;
return 0;
}
@@ -285,7 +309,7 @@ static int odb_source_files_write_object(struct odb_source *source,
enum odb_write_object_flags flags)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- return odb_source_write_object(&files->loose->base, buf, len, type,
+ return odb_source_write_object(&files->dirs->loose->base, buf, len, type,
oid, compat_oid, mtime, flags);
}
@@ -294,7 +318,7 @@ static int odb_source_files_write_object_stream(struct odb_source *source,
struct object_id *oid)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- return odb_source_write_object_stream(&files->loose->base, stream, oid);
+ return odb_source_write_object_stream(&files->dirs->loose->base, stream, oid);
}
static int odb_source_files_begin_transaction(struct odb_source *source,
@@ -330,7 +354,7 @@ static int too_many_loose_objects(struct odb_source_files *files, int limit)
if (limit <= 0)
return 0;
- if (odb_source_count_objects(&files->loose->base, ODB_COUNT_OBJECTS_APPROXIMATE,
+ if (odb_source_count_objects(&files->dirs->loose->base, ODB_COUNT_OBJECTS_APPROXIMATE,
&loose_count) < 0)
return 0;
@@ -349,7 +373,7 @@ static struct packed_git *find_base_packs(struct odb_source_files *files,
struct packfile_list_entry *e;
struct packed_git *base = NULL;
- for (e = packfile_store_get_packs(files->packed); e; e = e->next) {
+ for (e = packfile_store_get_packs(files->dirs->packed); e; e = e->next) {
if (e->pack->is_cruft)
continue;
if (limit) {
@@ -374,7 +398,7 @@ static int too_many_packs(struct odb_source_files *files, int gc_auto_pack_limit
if (gc_auto_pack_limit <= 0)
return 0;
- for (e = packfile_store_get_packs(files->packed); e; e = e->next) {
+ for (e = packfile_store_get_packs(files->dirs->packed); e; e = e->next) {
if (e->pack->pack_keep)
continue;
/*
@@ -937,8 +961,8 @@ static int odb_source_files_fsck(struct odb_source *source,
if (!(opts->flags & ODB_FSCK_FULL) && !source->local)
return 0;
- ret |= odb_source_fsck(&files->loose->base, opts);
- ret |= odb_source_fsck(&files->packed->base, opts);
+ ret |= odb_source_fsck(&files->dirs->loose->base, opts);
+ ret |= odb_source_fsck(&files->dirs->packed->base, opts);
return ret;
}
@@ -951,8 +975,7 @@ struct odb_source_files *odb_source_files_new(struct object_database *odb,
CALLOC_ARRAY(files, 1);
odb_source_init(&files->base, odb, ODB_SOURCE_FILES, path, local);
- files->loose = odb_source_loose_new(odb, path, local);
- files->packed = odb_source_packed_new(odb, path, local);
+ files->dirs = odb_files_dir_new(odb, path, local);
files->base.free = odb_source_files_free;
files->base.close = odb_source_files_close;
diff --git a/odb/source-files.h b/odb/source-files.h
index 9630b5f962..7f465853b1 100644
--- a/odb/source-files.h
+++ b/odb/source-files.h
@@ -6,14 +6,37 @@
struct odb_source_loose;
struct odb_source_packed;
+/*
+ * A single object directory that encapsulates access to both the loose and
+ * packed backend. This can either be the primary or an alternate object
+ * directory.
+ */
+struct odb_files_dir {
+ /* Absolute path to the object directory. */
+ char *abspath;
+
+ /* The two sources derived from this object directory. */
+ struct odb_source_loose *loose;
+ struct odb_source_packed *packed;
+
+ /*
+ * Whether this is the local object directory of the owning
+ * repository. Directories added via alternates are not local.
+ */
+ bool local;
+};
+
+struct odb_files_dir *odb_files_dir_new(struct object_database *odb,
+ const char *path, bool local);
+void odb_files_dir_free(struct odb_files_dir *dir);
+
/*
* The files object database source uses a combination of loose objects and
* packfiles. It is the default backend used by Git to store objects.
*/
struct odb_source_files {
struct odb_source base;
- struct odb_source_loose *loose;
- struct odb_source_packed *packed;
+ struct odb_files_dir *dirs;
};
/* Allocate and initialize a new object source. */
diff --git a/pack-bitmap.c b/pack-bitmap.c
index 3de8e9590c..52556b4543 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -720,7 +720,7 @@ static int open_bitmap(struct repository *r,
for (source = r->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- if (!open_bitmap_for_source(files->packed, bitmap_git))
+ if (!open_bitmap_for_source(files->dirs->packed, bitmap_git))
found = true;
/*
diff --git a/packfile.c b/packfile.c
index 4fa5fd67c8..93b69d7f50 100644
--- a/packfile.c
+++ b/packfile.c
@@ -280,7 +280,7 @@ static int unuse_one_window(struct object_database *odb)
for (source = odb->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- for (e = files->packed->packs.head; e; e = e->next)
+ for (e = files->dirs->packed->packs.head; e; e = e->next)
scan_windows(e->pack, &lru_p, &lru_w, &lru_l);
}
@@ -458,7 +458,7 @@ static int close_one_pack(struct repository *r)
for (source = r->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- for (e = files->packed->packs.head; e; e = e->next) {
+ for (e = files->dirs->packed->packs.head; e; e = e->next) {
if (e->pack->pack_fd == -1)
continue;
find_lru_pack(e->pack, &lru_p, &mru_w, &accept_windows_inuse);
@@ -1925,7 +1925,7 @@ int has_object_pack(struct repository *r, const struct object_id *oid)
for (source = r->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- if (!odb_source_read_object_info(&files->packed->base, oid, NULL, 0, NULL))
+ if (!odb_source_read_object_info(&files->dirs->packed->base, oid, NULL, 0, NULL))
return 1;
}
@@ -1942,7 +1942,7 @@ int has_object_kept_pack(struct repository *r, const struct object_id *oid,
struct odb_source_files *files = odb_source_files_downcast(source);
struct packed_git **cache;
- cache = packfile_store_get_kept_pack_cache(files->packed, flags);
+ cache = packfile_store_get_kept_pack_cache(files->dirs->packed, flags);
for (; *cache; cache++) {
struct packed_git *p = *cache;
diff --git a/packfile.h b/packfile.h
index 6d30d15a00..fd1cf0ad6c 100644
--- a/packfile.h
+++ b/packfile.h
@@ -79,7 +79,7 @@ static inline struct repo_for_each_pack_data repo_for_eack_pack_data_init(struct
for (struct odb_source *source = repo->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- struct packfile_list_entry *entry = packfile_store_get_packs(files->packed);
+ struct packfile_list_entry *entry = packfile_store_get_packs(files->dirs->packed);
if (!entry)
continue;
data.source = source;
@@ -100,7 +100,7 @@ static inline void repo_for_each_pack_data_next(struct repo_for_each_pack_data *
for (source = data->source->next; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- struct packfile_list_entry *entry = packfile_store_get_packs(files->packed);
+ struct packfile_list_entry *entry = packfile_store_get_packs(files->dirs->packed);
if (!entry)
continue;
data->source = source;
diff --git a/repack-geometry.c b/repack-geometry.c
index 15b3412950..b541c34af5 100644
--- a/repack-geometry.c
+++ b/repack-geometry.c
@@ -33,7 +33,7 @@ void pack_geometry_init(struct pack_geometry *geometry,
struct packed_git *p;
struct strbuf buf = STRBUF_INIT;
struct odb_source_files *files = odb_source_files_downcast(existing->source);
- struct multi_pack_index *m = get_multi_pack_index(files->packed);
+ struct multi_pack_index *m = get_multi_pack_index(files->dirs->packed);
repo_for_each_pack(existing->repo, p) {
if (geometry->midx_layer_threshold_set && m &&
diff --git a/repack-midx.c b/repack-midx.c
index 64c7f8d0f4..25140569bd 100644
--- a/repack-midx.c
+++ b/repack-midx.c
@@ -564,7 +564,7 @@ static void repack_make_midx_append_plan(struct repack_write_midx_opts *opts,
size_t steps_nr = 0, steps_alloc = 0;
odb_reprepare(opts->existing->repo->objects);
- m = get_multi_pack_index(files->packed);
+ m = get_multi_pack_index(files->dirs->packed);
if (opts->names->nr) {
struct strbuf buf = STRBUF_INIT;
@@ -620,7 +620,7 @@ static int repack_make_midx_compaction_plan(struct repack_write_midx_opts *opts,
opts->existing->repo);
odb_reprepare(opts->existing->repo->objects);
- m = get_multi_pack_index(files->packed);
+ m = get_multi_pack_index(files->dirs->packed);
for (i = 0; m && i < m->num_packs + m->num_packs_in_base; i++) {
if (prepare_midx_pack(m, i)) {
@@ -949,7 +949,7 @@ static int write_midx_incremental(struct repack_write_midx_opts *opts)
size_t i;
int ret = 0;
- get_midx_chain_filename(files->packed, &lock_name);
+ get_midx_chain_filename(files->dirs->packed, &lock_name);
if (safe_create_leading_directories(opts->existing->repo,
lock_name.buf))
die_errno(_("unable to create leading directories of %s"),
diff --git a/repack.c b/repack.c
index d2aa58e134..e20431690c 100644
--- a/repack.c
+++ b/repack.c
@@ -60,7 +60,7 @@ void repack_remove_redundant_pack(struct repository *repo, const char *dir_name,
{
struct strbuf buf = STRBUF_INIT;
struct odb_source_files *files = odb_source_files_downcast(repo->objects->sources);
- struct multi_pack_index *m = get_multi_pack_index(files->packed);
+ struct multi_pack_index *m = get_multi_pack_index(files->dirs->packed);
strbuf_addf(&buf, "%s.pack", base_name);
if (m && files->base.local && midx_contains_pack(m, buf.buf)) {
clear_midx_file(repo);
diff --git a/t/helper/test-read-midx.c b/t/helper/test-read-midx.c
index 83b07c6236..3f4bafff61 100644
--- a/t/helper/test-read-midx.c
+++ b/t/helper/test-read-midx.c
@@ -22,7 +22,7 @@ static struct multi_pack_index *setup_midx(const char *object_dir,
source = odb_find_source(the_repository->objects, object_dir);
if (source) {
- packed = odb_source_files_downcast(source)->packed;
+ packed = odb_source_files_downcast(source)->dirs->packed;
} else {
packed = odb_source_packed_new(the_repository->objects,
object_dir, false);
--
2.56.0.379.gc618271300.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* [PATCH 04/13] odb: refactor `odb_for_each_alternate()` to yield dirs
2026-10-02 10:08 [PATCH 00/13] odb/source-files: move alternates into the backend Patrick Steinhardt
` (2 preceding siblings ...)
2026-10-02 10:08 ` [PATCH 03/13] odb/source-files: introduce `struct odb_files_dir` Patrick Steinhardt
@ 2026-10-02 10:08 ` Patrick Steinhardt
2026-10-06 8:42 ` Karthik Nayak
2026-10-02 10:08 ` [PATCH 05/13] odb: refactor `odb_find_source()` " Patrick Steinhardt
` (10 subsequent siblings)
14 siblings, 1 reply; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-02 10:08 UTC (permalink / raw)
To: git
The function `odb_for_each_alternate()` iterates through all alternates
of an object database. This is becoming an implementation detail of the
"files" backend though, where each alternate will be represented by one
`struct odb_files_dir`.
Adapt `odb_for_each_alternate()` to already iterate through these
structs instead of iterating through sources.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/count-objects.c | 4 ++--
builtin/submodule--helper.c | 7 ++++---
diagnose.c | 8 ++++----
odb.c | 18 +++++++++++-------
odb.h | 3 ++-
5 files changed, 23 insertions(+), 17 deletions(-)
diff --git a/builtin/count-objects.c b/builtin/count-objects.c
index 18f6e33b6f..f2abfaccec 100644
--- a/builtin/count-objects.c
+++ b/builtin/count-objects.c
@@ -81,10 +81,10 @@ static int count_cruft(const char *basename UNUSED, const char *path,
return 0;
}
-static int print_alternate(struct odb_source *alternate, void *data UNUSED)
+static int print_alternate(struct odb_files_dir *alternate, void *data UNUSED)
{
printf("alternate: ");
- quote_c_style(alternate->path, NULL, stdout, 0);
+ quote_c_style(alternate->abspath, NULL, stdout, 0);
putchar('\n');
return 0;
}
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 40a052d674..64412adf9f 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -30,6 +30,7 @@
#include "object-name.h"
#include "odb.h"
#include "odb/source.h"
+#include "odb/source-files.h"
#include "advice.h"
#include "branch.h"
#include "list-objects-filter-options.h"
@@ -1770,7 +1771,7 @@ static const char alternate_error_advice[] = N_(
);
static int add_possible_reference_from_superproject(
- struct odb_source *alt_odb, void *sas_cb)
+ struct odb_files_dir *alt_odb, void *sas_cb)
{
struct submodule_alternate_setup *sas = sas_cb;
size_t len;
@@ -1779,12 +1780,12 @@ static int add_possible_reference_from_superproject(
* If the alternate object store is another repository, try the
* standard layout with .git/(modules/<name>)+/objects
*/
- if (strip_suffix(alt_odb->path, "/objects", &len)) {
+ if (strip_suffix(alt_odb->abspath, "/objects", &len)) {
struct repository alternate;
char *sm_alternate;
struct strbuf sb = STRBUF_INIT;
struct strbuf err = STRBUF_INIT;
- strbuf_add(&sb, alt_odb->path, len);
+ strbuf_add(&sb, alt_odb->abspath, len);
if (repo_init(&alternate, sb.buf, NULL) < 0)
die(_("could not get a repository handle for gitdir '%s'"),
diff --git a/diagnose.c b/diagnose.c
index 5092bf80d3..89240e47d6 100644
--- a/diagnose.c
+++ b/diagnose.c
@@ -59,13 +59,13 @@ static void dir_file_stats_objects(const char *full_path,
(uintmax_t)st.st_size);
}
-static int dir_file_stats(struct odb_source *source, void *data)
+static int dir_file_stats(struct odb_files_dir *source, void *data)
{
struct strbuf *buf = data;
- strbuf_addf(buf, "Contents of %s:\n", source->path);
+ strbuf_addf(buf, "Contents of %s:\n", source->abspath);
- for_each_file_in_pack_dir(source->path, dir_file_stats_objects,
+ for_each_file_in_pack_dir(source->abspath, dir_file_stats_objects,
data);
return 0;
@@ -228,7 +228,7 @@ int create_diagnostics_archive(struct repository *r,
strbuf_reset(&buf);
strbuf_addstr(&buf, "--add-virtual-file=packs-local.txt:");
- dir_file_stats(r->objects->sources, &buf);
+ dir_file_stats(odb_source_files_downcast(r->objects->sources)->dirs, &buf);
odb_for_each_alternate(r->objects, dir_file_stats, &buf);
strvec_push(&archiver_args, buf.buf);
diff --git a/odb.c b/odb.c
index 0200e26f21..9b70859c23 100644
--- a/odb.c
+++ b/odb.c
@@ -14,6 +14,7 @@
#include "object-name.h"
#include "odb.h"
#include "odb/source-inmemory.h"
+#include "odb/source-files.h"
#include "path.h"
#include "promisor-remote.h"
#include "quote.h"
@@ -435,18 +436,19 @@ static void read_alternate_refs(struct repository *repo,
}
struct alternate_refs_data {
+ struct repository *repo;
odb_for_each_alternate_ref_fn *fn;
void *payload;
};
-static int refs_from_alternate_cb(struct odb_source *alternate,
+static int refs_from_alternate_cb(struct odb_files_dir *alternate,
void *payload)
{
struct strbuf path = STRBUF_INIT;
size_t base_len;
struct alternate_refs_data *cb = payload;
- if (!strbuf_realpath(&path, alternate->path, 0))
+ if (!strbuf_realpath(&path, alternate->abspath, 0))
goto out;
if (!strbuf_strip_suffix(&path, "/objects"))
goto out;
@@ -458,7 +460,7 @@ static int refs_from_alternate_cb(struct odb_source *alternate,
goto out;
strbuf_setlen(&path, base_len);
- read_alternate_refs(alternate->odb->repo, path.buf, cb->fn, cb->payload);
+ read_alternate_refs(cb->repo, path.buf, cb->fn, cb->payload);
out:
strbuf_release(&path);
@@ -468,9 +470,11 @@ static int refs_from_alternate_cb(struct odb_source *alternate,
void odb_for_each_alternate_ref(struct object_database *odb,
odb_for_each_alternate_ref_fn cb, void *payload)
{
- struct alternate_refs_data data;
- data.fn = cb;
- data.payload = payload;
+ struct alternate_refs_data data = {
+ .fn = cb,
+ .payload = payload,
+ .repo = odb->repo,
+ };
odb_for_each_alternate(odb, refs_from_alternate_cb, &data);
}
@@ -481,7 +485,7 @@ int odb_for_each_alternate(struct object_database *odb,
int r = 0;
for (alternate = odb->sources->next; alternate; alternate = alternate->next) {
- r = cb(alternate, payload);
+ r = cb(odb_source_files_downcast(alternate)->dirs, payload);
if (r)
break;
}
diff --git a/odb.h b/odb.h
index 797eecbb94..3715351bb3 100644
--- a/odb.h
+++ b/odb.h
@@ -12,6 +12,7 @@
struct cached_object_entry;
struct list_objects_filter_options;
struct odb_source_inmemory;
+struct odb_files_dir;
struct packed_git;
struct repository;
struct strbuf;
@@ -257,7 +258,7 @@ void odb_restore_primary_source(struct object_database *odb,
* function returns a non-zero value, in which case the value is bubbled up
* from the callback.
*/
-typedef int odb_for_each_alternate_fn(struct odb_source *, void *);
+typedef int odb_for_each_alternate_fn(struct odb_files_dir *, void *);
int odb_for_each_alternate(struct object_database *odb,
odb_for_each_alternate_fn cb, void *payload);
--
2.56.0.379.gc618271300.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* Re: [PATCH 04/13] odb: refactor `odb_for_each_alternate()` to yield dirs
2026-10-02 10:08 ` [PATCH 04/13] odb: refactor `odb_for_each_alternate()` to yield dirs Patrick Steinhardt
@ 2026-10-06 8:42 ` Karthik Nayak
0 siblings, 0 replies; 41+ messages in thread
From: Karthik Nayak @ 2026-10-06 8:42 UTC (permalink / raw)
To: Patrick Steinhardt, git
[-- Attachment #1: Type: text/plain, Size: 977 bytes --]
Patrick Steinhardt <ps@pks.im> writes:
[snip]
> @@ -468,9 +470,11 @@ static int refs_from_alternate_cb(struct odb_source *alternate,
> void odb_for_each_alternate_ref(struct object_database *odb,
> odb_for_each_alternate_ref_fn cb, void *payload)
> {
> - struct alternate_refs_data data;
> - data.fn = cb;
> - data.payload = payload;
> + struct alternate_refs_data data = {
> + .fn = cb,
> + .payload = payload,
> + .repo = odb->repo,
> + };
> odb_for_each_alternate(odb, refs_from_alternate_cb, &data);
> }
>
> @@ -481,7 +485,7 @@ int odb_for_each_alternate(struct object_database *odb,
> int r = 0;
>
> for (alternate = odb->sources->next; alternate; alternate = alternate->next) {
> - r = cb(alternate, payload);
> + r = cb(odb_source_files_downcast(alternate)->dirs, payload);
> if (r)
> break;
> }
>
Okay, so here we call the callbacks with the `dirs` now and this
corresponds with the changes in the rest of the patch. Makes sense.
[snip]
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 41+ messages in thread
* [PATCH 05/13] odb: refactor `odb_find_source()` to yield dirs
2026-10-02 10:08 [PATCH 00/13] odb/source-files: move alternates into the backend Patrick Steinhardt
` (3 preceding siblings ...)
2026-10-02 10:08 ` [PATCH 04/13] odb: refactor `odb_for_each_alternate()` to yield dirs Patrick Steinhardt
@ 2026-10-02 10:08 ` Patrick Steinhardt
2026-10-02 10:08 ` [PATCH 06/13] odb/source-files: add the ability to have multiple object dirs Patrick Steinhardt
` (9 subsequent siblings)
14 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-02 10:08 UTC (permalink / raw)
To: git
Similar as in the preceding commit, `odb_find_source()` can be used to
yield a single source by its object directory. This function is also
specific to the "files" backend once alternates are an implementation
detail thereof.
Refactor it to be specific to the "files" backend and return an `struct
odb_files_dir` to prepare for moving alternates into the "files" source.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/commit-graph.c | 26 ++++++++++++++++----------
builtin/multi-pack-index.c | 42 +++++++++++++++++++++---------------------
odb.c | 26 --------------------------
odb.h | 9 ---------
odb/source-files.c | 26 ++++++++++++++++++++++++++
odb/source-files.h | 6 ++++++
t/helper/test-read-midx.c | 8 ++++----
7 files changed, 73 insertions(+), 70 deletions(-)
diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c
index a986f08a94..ff244c866f 100644
--- a/builtin/commit-graph.c
+++ b/builtin/commit-graph.c
@@ -9,6 +9,7 @@
#include "commit-graph.h"
#include "odb.h"
#include "odb/source.h"
+#include "odb/source-files.h"
#include "progress.h"
#include "replace-object.h"
#include "strbuf.h"
@@ -68,7 +69,7 @@ static int graph_verify(int argc, const char **argv, const char *prefix,
struct repository *repo UNUSED)
{
struct commit_graph *graph = NULL;
- struct odb_source *source = NULL;
+ struct odb_files_dir *dir = NULL;
char *graph_name;
char *chain_name;
enum { OPENED_NONE, OPENED_GRAPH, OPENED_CHAIN } opened = OPENED_NONE;
@@ -103,9 +104,12 @@ static int graph_verify(int argc, const char **argv, const char *prefix,
if (opts.progress)
flags |= COMMIT_GRAPH_WRITE_PROGRESS;
- source = odb_find_source_or_die(the_repository->objects, opts.obj_dir);
- graph_name = get_commit_graph_filename(source->path);
- chain_name = get_commit_graph_chain_filename(source->path);
+ dir = odb_source_files_find_dir(the_repository->objects, opts.obj_dir);
+ if (!dir)
+ die(_("could not find object directory matching %s"), opts.obj_dir);
+
+ graph_name = get_commit_graph_filename(dir->abspath);
+ chain_name = get_commit_graph_chain_filename(dir->abspath);
if (open_commit_graph(graph_name, &fd, &st))
opened = OPENED_GRAPH;
else if (errno != ENOENT)
@@ -123,7 +127,7 @@ static int graph_verify(int argc, const char **argv, const char *prefix,
if (opened == OPENED_NONE)
return 0;
else if (opened == OPENED_GRAPH)
- graph = load_commit_graph_one_fd_st(the_repository, source->path, fd, &st);
+ graph = load_commit_graph_one_fd_st(the_repository, dir->abspath, fd, &st);
else
graph = load_commit_graph_chain_fd_st(the_repository->objects, fd, &st,
&incomplete_chain);
@@ -226,7 +230,7 @@ static int graph_write(int argc, const char **argv, const char *prefix,
struct string_list pack_indexes = STRING_LIST_INIT_DUP;
struct strbuf buf = STRBUF_INIT;
struct oidset commits = OIDSET_INIT;
- struct odb_source *source = NULL;
+ struct odb_files_dir *dir = NULL;
int result = 0;
enum commit_graph_write_flags flags = 0;
struct progress *progress = NULL;
@@ -294,10 +298,12 @@ static int graph_write(int argc, const char **argv, const char *prefix,
git_env_bool(GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS, 0))
flags |= COMMIT_GRAPH_WRITE_BLOOM_FILTERS;
- source = odb_find_source_or_die(the_repository->objects, opts.obj_dir);
+ dir = odb_source_files_find_dir(the_repository->objects, opts.obj_dir);
+ if (!dir)
+ die(_("could not find object directory matching %s"), opts.obj_dir);
if (opts.reachable) {
- if (write_commit_graph_reachable(the_repository, source->path, flags, &write_opts))
+ if (write_commit_graph_reachable(the_repository, dir->abspath, flags, &write_opts))
result = 1;
goto cleanup;
}
@@ -306,7 +312,7 @@ static int graph_write(int argc, const char **argv, const char *prefix,
struct strbuf packname = STRBUF_INIT;
size_t dirlen;
- strbuf_addf(&packname, "%s/pack/", source->path);
+ strbuf_addf(&packname, "%s/pack/", dir->abspath);
dirlen = packname.len;
while (strbuf_getline(&buf, stdin) != EOF) {
@@ -334,7 +340,7 @@ static int graph_write(int argc, const char **argv, const char *prefix,
stop_progress(&progress);
}
- if (write_commit_graph(the_repository, source->path,
+ if (write_commit_graph(the_repository, dir->abspath,
opts.stdin_packs ? &pack_indexes : NULL,
opts.stdin_commits ? &commits : NULL,
flags,
diff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c
index a170ec80b9..fc8b494996 100644
--- a/builtin/multi-pack-index.c
+++ b/builtin/multi-pack-index.c
@@ -86,13 +86,13 @@ static int parse_object_dir(const struct option *opt, const char *arg,
return 0;
}
-static struct odb_source_files *handle_object_dir_option(struct repository *repo)
+static struct odb_source_packed *handle_object_dir_option(struct repository *repo)
{
- struct odb_source *source = odb_find_source(repo->objects, opts.object_dir);
- if (!source)
+ struct odb_files_dir *dir = odb_source_files_find_dir(repo->objects, opts.object_dir);
+ if (!dir)
die(_("object directory is not an alternate of the current repository: '%s'"),
opts.object_dir);
- return odb_source_files_downcast(source);
+ return dir->packed;
}
static struct option common_opts[] = {
@@ -169,7 +169,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,
N_("refs snapshot for selecting bitmap commits")),
OPT_END(),
};
- struct odb_source_files *source;
+ struct odb_source_packed *packed_source;
int ret;
opts.flags |= MIDX_WRITE_BITMAP_HASH_CACHE;
@@ -204,7 +204,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,
options);
}
- source = handle_object_dir_option(repo);
+ packed_source = handle_object_dir_option(repo);
FREE_AND_NULL(options);
@@ -213,7 +213,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,
read_packs_from_stdin(&packs);
- ret = write_midx_file_only(source->dirs->packed, &packs,
+ ret = write_midx_file_only(packed_source, &packs,
opts.preferred_pack,
opts.refs_snapshot,
opts.incremental_base, opts.flags);
@@ -225,7 +225,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,
}
- ret = write_midx_file(source->dirs->packed, opts.preferred_pack,
+ ret = write_midx_file(packed_source, opts.preferred_pack,
opts.refs_snapshot, opts.flags);
free(opts.refs_snapshot);
@@ -239,7 +239,7 @@ static int cmd_multi_pack_index_compact(int argc, const char **argv,
struct multi_pack_index *m, *cur;
struct multi_pack_index *from_midx = NULL;
struct multi_pack_index *to_midx = NULL;
- struct odb_source_files *source;
+ struct odb_source_packed *packed_source;
int ret;
struct option *options;
@@ -280,11 +280,11 @@ static int cmd_multi_pack_index_compact(int argc, const char **argv,
options);
}
- source = handle_object_dir_option(the_repository);
+ packed_source = handle_object_dir_option(the_repository);
FREE_AND_NULL(options);
- m = get_multi_pack_index(source->dirs->packed);
+ m = get_multi_pack_index(packed_source);
for (cur = m; cur && !(from_midx && to_midx); cur = cur->base_midx) {
const char *midx_csum = midx_get_checksum_hex(cur);
@@ -307,7 +307,7 @@ static int cmd_multi_pack_index_compact(int argc, const char **argv,
die(_("MIDX %s must be an ancestor of %s"), argv[0], argv[1]);
}
- ret = write_midx_file_compact(source->dirs->packed, from_midx, to_midx,
+ ret = write_midx_file_compact(packed_source, from_midx, to_midx,
opts.incremental_base, opts.flags);
return ret;
@@ -321,7 +321,7 @@ static int cmd_multi_pack_index_verify(int argc, const char **argv,
static struct option builtin_multi_pack_index_verify_options[] = {
OPT_END(),
};
- struct odb_source_files *source;
+ struct odb_source_packed *packed_source;
options = add_common_options(builtin_multi_pack_index_verify_options);
@@ -335,11 +335,11 @@ static int cmd_multi_pack_index_verify(int argc, const char **argv,
if (argc)
usage_with_options(builtin_multi_pack_index_verify_usage,
options);
- source = handle_object_dir_option(the_repository);
+ packed_source = handle_object_dir_option(the_repository);
FREE_AND_NULL(options);
- return verify_midx_file(source->dirs->packed, opts.flags);
+ return verify_midx_file(packed_source, opts.flags);
}
static int cmd_multi_pack_index_expire(int argc, const char **argv,
@@ -350,7 +350,7 @@ static int cmd_multi_pack_index_expire(int argc, const char **argv,
static struct option builtin_multi_pack_index_expire_options[] = {
OPT_END(),
};
- struct odb_source_files *source;
+ struct odb_source_packed *packed_source;
options = add_common_options(builtin_multi_pack_index_expire_options);
@@ -364,11 +364,11 @@ static int cmd_multi_pack_index_expire(int argc, const char **argv,
if (argc)
usage_with_options(builtin_multi_pack_index_expire_usage,
options);
- source = handle_object_dir_option(the_repository);
+ packed_source = handle_object_dir_option(the_repository);
FREE_AND_NULL(options);
- return expire_midx_packs(source->dirs->packed, opts.flags);
+ return expire_midx_packs(packed_source, opts.flags);
}
static int cmd_multi_pack_index_repack(int argc, const char **argv,
@@ -381,7 +381,7 @@ static int cmd_multi_pack_index_repack(int argc, const char **argv,
N_("during repack, collect pack-files of smaller size into a batch that is larger than this size")),
OPT_END(),
};
- struct odb_source_files *source;
+ struct odb_source_packed *packed_source;
options = add_common_options(builtin_multi_pack_index_repack_options);
@@ -396,11 +396,11 @@ static int cmd_multi_pack_index_repack(int argc, const char **argv,
if (argc)
usage_with_options(builtin_multi_pack_index_repack_usage,
options);
- source = handle_object_dir_option(the_repository);
+ packed_source = handle_object_dir_option(the_repository);
FREE_AND_NULL(options);
- return midx_repack(source->dirs->packed, (size_t)opts.batch_size, opts.flags);
+ return midx_repack(packed_source, (size_t)opts.batch_size, opts.flags);
}
int cmd_multi_pack_index(int argc,
diff --git a/odb.c b/odb.c
index 9b70859c23..1dc8647159 100644
--- a/odb.c
+++ b/odb.c
@@ -348,32 +348,6 @@ char *compute_alternate_path(const char *path, struct strbuf *err)
return ref_git;
}
-struct odb_source *odb_find_source(struct object_database *odb, const char *obj_dir)
-{
- struct odb_source *source;
- char *obj_dir_real = real_pathdup(obj_dir, 1);
- struct strbuf odb_path_real = STRBUF_INIT;
-
- for (source = odb->sources; source; source = source->next) {
- strbuf_realpath(&odb_path_real, source->path, 1);
- if (!strcmp(obj_dir_real, odb_path_real.buf))
- break;
- }
-
- free(obj_dir_real);
- strbuf_release(&odb_path_real);
-
- return source;
-}
-
-struct odb_source *odb_find_source_or_die(struct object_database *odb, const char *obj_dir)
-{
- struct odb_source *source = odb_find_source(odb, obj_dir);
- if (!source)
- die(_("could not find object directory matching %s"), obj_dir);
- return source;
-}
-
static void fill_alternate_refs_command(struct repository *repo,
struct child_process *cmd,
const char *repo_path)
diff --git a/odb.h b/odb.h
index 3715351bb3..5c86572b5d 100644
--- a/odb.h
+++ b/odb.h
@@ -226,15 +226,6 @@ struct odb_fsck_options {
*/
int odb_fsck(struct object_database *odb, struct odb_fsck_options *opts);
-/*
- * Find source by its object directory path. Returns a `NULL` pointer in case
- * the source could not be found.
- */
-struct odb_source *odb_find_source(struct object_database *odb, const char *obj_dir);
-
-/* Same as `odb_find_source()`, but dies in case the source doesn't exist. */
-struct odb_source *odb_find_source_or_die(struct object_database *odb, const char *obj_dir);
-
/*
* Replace the current writable object directory with the specified temporary
* object directory and return the newly installed primary source. The former
diff --git a/odb/source-files.c b/odb/source-files.c
index 1f4cedaffa..e5e43b1543 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -967,6 +967,32 @@ static int odb_source_files_fsck(struct odb_source *source,
return ret;
}
+struct odb_files_dir *odb_source_files_find_dir(struct object_database *odb, const char *obj_dir)
+{
+ char *obj_dir_real = real_pathdup(obj_dir, 1);
+ struct strbuf odb_path_real = STRBUF_INIT;
+ struct odb_files_dir *dir = NULL;
+ struct odb_source *source;
+
+ for (source = odb->sources; source; source = source->next) {
+ struct odb_source_files *files;
+
+ if (source->type != ODB_SOURCE_FILES)
+ continue;
+ files = odb_source_files_downcast(source);
+
+ strbuf_realpath(&odb_path_real, files->dirs->abspath, 1);
+ if (!strcmp(obj_dir_real, odb_path_real.buf)) {
+ dir = files->dirs;
+ break;
+ }
+ }
+
+ free(obj_dir_real);
+ strbuf_release(&odb_path_real);
+ return dir;
+}
+
struct odb_source_files *odb_source_files_new(struct object_database *odb,
const char *path,
bool local)
diff --git a/odb/source-files.h b/odb/source-files.h
index 7f465853b1..77f4d842e0 100644
--- a/odb/source-files.h
+++ b/odb/source-files.h
@@ -72,4 +72,10 @@ static inline struct odb_source_files *odb_source_files_downcast(struct odb_sour
return container_of(source, struct odb_source_files, base);
}
+/*
+ * Find "files" directory by its object directory path. Returns a `NULL`
+ * pointer in case the object directory could not be found.
+ */
+struct odb_files_dir *odb_source_files_find_dir(struct object_database *odb, const char *obj_dir);
+
#endif
diff --git a/t/helper/test-read-midx.c b/t/helper/test-read-midx.c
index 3f4bafff61..88a79fdfc8 100644
--- a/t/helper/test-read-midx.c
+++ b/t/helper/test-read-midx.c
@@ -16,13 +16,13 @@ static struct multi_pack_index *setup_midx(const char *object_dir,
struct odb_source_packed **out)
{
struct odb_source_packed *packed;
- struct odb_source *source;
+ struct odb_files_dir *dir;
setup_git_directory(the_repository);
- source = odb_find_source(the_repository->objects, object_dir);
- if (source) {
- packed = odb_source_files_downcast(source)->dirs->packed;
+ dir = odb_source_files_find_dir(the_repository->objects, object_dir);
+ if (dir) {
+ packed = dir->packed;
} else {
packed = odb_source_packed_new(the_repository->objects,
object_dir, false);
--
2.56.0.379.gc618271300.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* [PATCH 06/13] odb/source-files: add the ability to have multiple object dirs
2026-10-02 10:08 [PATCH 00/13] odb/source-files: move alternates into the backend Patrick Steinhardt
` (4 preceding siblings ...)
2026-10-02 10:08 ` [PATCH 05/13] odb: refactor `odb_find_source()` " Patrick Steinhardt
@ 2026-10-02 10:08 ` Patrick Steinhardt
2026-10-02 10:08 ` [PATCH 07/13] tmp-objdir: absorb logic to set and restore primary sources Patrick Steinhardt
` (8 subsequent siblings)
14 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-02 10:08 UTC (permalink / raw)
To: git
In a subsequent commit we'll migrate handling of alternates into the
"files" backend. As part of that, the backend will need to grow the
ability to handle multiple object directories for alternates and for
transactions.
Introduce a `struct odb_files_dir::next` pointer so that we can have
multiple directories. Adapt the backend to loop through this list as
necessary. In general:
- For reading paths we will loop through all object directories until
we have found the object.
- For writing paths we will typically write to the first of our object
directories.
This mechanism isn't used yet as we still only ever have a single object
directory at the current point in time. But that will change over
subsequent commits.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
odb/source-files.c | 147 +++++++++++++++++++++++++++++++++--------------------
odb/source-files.h | 9 ++++
2 files changed, 100 insertions(+), 56 deletions(-)
diff --git a/odb/source-files.c b/odb/source-files.c
index e5e43b1543..9389546b3e 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -63,8 +63,15 @@ static void odb_source_files_reparent(const char *old_cwd,
static void odb_source_files_free(struct odb_source *source)
{
struct odb_source_files *files = odb_source_files_downcast(source);
+
chdir_notify_unregister(odb_source_files_reparent, files);
- odb_files_dir_free(files->dirs);
+
+ while (files->dirs) {
+ struct odb_files_dir *next = files->dirs->next;
+ odb_files_dir_free(files->dirs);
+ files->dirs = next;
+ }
+
odb_source_release(&files->base);
free(files);
}
@@ -72,8 +79,11 @@ static void odb_source_files_free(struct odb_source *source)
static void odb_source_files_close(struct odb_source *source)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- odb_source_close(&files->dirs->loose->base);
- odb_source_close(&files->dirs->packed->base);
+
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ odb_source_close(&dir->loose->base);
+ odb_source_close(&dir->packed->base);
+ }
}
static int odb_source_files_create_on_disk(struct odb_source *source,
@@ -168,8 +178,11 @@ static void odb_source_files_prepare(struct odb_source *source,
enum odb_prepare_flags flags)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- odb_source_prepare(&files->dirs->loose->base, flags);
- odb_source_prepare(&files->dirs->packed->base, flags);
+
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ odb_source_prepare(&dir->loose->base, flags);
+ odb_source_prepare(&dir->packed->base, flags);
+ }
}
static enum odb_read_status odb_source_files_read_object_info(struct odb_source *source,
@@ -179,28 +192,33 @@ static enum odb_read_status odb_source_files_read_object_info(struct odb_source
struct strbuf *errmsg)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- enum odb_read_status ret_packed, ret_loose;
-
- ret_packed = odb_source_read_object_info(&files->dirs->packed->base, oid, oi,
- flags, errmsg);
- if (!ret_packed)
- return 0;
-
- ret_loose = odb_source_read_object_info(&files->dirs->loose->base, oid, oi, flags,
- ret_packed == ODB_READ_NOT_FOUND ? errmsg : NULL);
- if (!ret_loose)
- return 0;
+ enum odb_read_status status = ODB_READ_NOT_FOUND;
/*
- * Reading the packed object may have failed even though the object
- * exists, for example because it is corrupt. Report this failure to
- * the caller in case neither of the sources was able to read the
- * object, and prefer the error of the packed source in case both
- * reads have failed.
+ * Reading an object may fail even though the object exists, for
+ * example because it is corrupt. Report this failure to the caller in
+ * case none of the directories was able to read the object, and
+ * prefer the first such error in case multiple reads have failed.
*/
- if (ret_packed != ODB_READ_NOT_FOUND)
- return ret_packed;
- return ret_loose;
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ enum odb_read_status ret;
+
+ ret = odb_source_read_object_info(&dir->packed->base, oid, oi, flags,
+ status == ODB_READ_NOT_FOUND ? errmsg : NULL);
+ if (!ret)
+ return 0;
+ if (ret != ODB_READ_NOT_FOUND && status == ODB_READ_NOT_FOUND)
+ status = ret;
+
+ ret = odb_source_read_object_info(&dir->loose->base, oid, oi, flags,
+ status == ODB_READ_NOT_FOUND ? errmsg : NULL);
+ if (!ret)
+ return 0;
+ if (ret != ODB_READ_NOT_FOUND && status == ODB_READ_NOT_FOUND)
+ status = ret;
+ }
+
+ return status;
}
static int odb_source_files_read_object_stream(struct odb_stream **out,
@@ -208,9 +226,12 @@ static int odb_source_files_read_object_stream(struct odb_stream **out,
const struct object_id *oid)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- if (!odb_source_read_object_stream(out, &files->dirs->packed->base, oid) ||
- !odb_source_read_object_stream(out, &files->dirs->loose->base, oid))
- return 0;
+
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next)
+ if (!odb_source_read_object_stream(out, &dir->packed->base, oid) ||
+ !odb_source_read_object_stream(out, &dir->loose->base, oid))
+ return 0;
+
return -1;
}
@@ -223,16 +244,21 @@ static int odb_source_files_for_each_object(struct odb_source *source,
struct odb_source_files *files = odb_source_files_downcast(source);
int ret;
- if (!(opts->flags & ODB_FOR_EACH_OBJECT_PROMISOR_ONLY)) {
- ret = odb_source_for_each_object(&files->dirs->loose->base, request, cb, cb_data, opts);
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ if (opts->flags & ODB_FOR_EACH_OBJECT_LOCAL_ONLY && !dir->local)
+ continue;
+
+ if (!(opts->flags & ODB_FOR_EACH_OBJECT_PROMISOR_ONLY)) {
+ ret = odb_source_for_each_object(&dir->loose->base, request, cb, cb_data, opts);
+ if (ret)
+ return ret;
+ }
+
+ ret = odb_source_for_each_object(&dir->packed->base, request, cb, cb_data, opts);
if (ret)
return ret;
}
- ret = odb_source_for_each_object(&files->dirs->packed->base, request, cb, cb_data, opts);
- if (ret)
- return ret;
-
return 0;
}
@@ -241,21 +267,23 @@ static int odb_source_files_count_objects(struct odb_source *source,
unsigned long *out)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- unsigned long count;
+ unsigned long count = 0;
int ret;
- ret = odb_source_count_objects(&files->dirs->packed->base, flags, &count);
- if (ret < 0)
- goto out;
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ unsigned long dir_count;
- if (!(flags & ODB_COUNT_OBJECTS_APPROXIMATE)) {
- unsigned long loose_count;
-
- ret = odb_source_count_objects(&files->dirs->loose->base, flags, &loose_count);
+ ret = odb_source_count_objects(&dir->packed->base, flags, &dir_count);
if (ret < 0)
goto out;
+ count += dir_count;
- count += loose_count;
+ if (!(flags & ODB_COUNT_OBJECTS_APPROXIMATE)) {
+ ret = odb_source_count_objects(&dir->loose->base, flags, &dir_count);
+ if (ret < 0)
+ goto out;
+ count += dir_count;
+ }
}
*out = count;
@@ -272,15 +300,17 @@ static int odb_source_files_find_abbrev_len(struct odb_source *source,
{
struct odb_source_files *files = odb_source_files_downcast(source);
unsigned len = min_len;
- int ret;
+ int ret = 0;
- ret = odb_source_find_abbrev_len(&files->dirs->packed->base, oid, len, &len);
- if (ret < 0)
- goto out;
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ ret = odb_source_find_abbrev_len(&dir->packed->base, oid, len, &len);
+ if (ret < 0)
+ goto out;
- ret = odb_source_find_abbrev_len(&files->dirs->loose->base, oid, len, &len);
- if (ret < 0)
- goto out;
+ ret = odb_source_find_abbrev_len(&dir->loose->base, oid, len, &len);
+ if (ret < 0)
+ goto out;
+ }
*out = len;
ret = 0;
@@ -294,9 +324,12 @@ static int odb_source_files_freshen_object(struct odb_source *source,
const time_t *mtime)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- if (odb_source_freshen_object(&files->dirs->packed->base, oid, mtime) ||
- odb_source_freshen_object(&files->dirs->loose->base, oid, mtime))
- return 1;
+
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next)
+ if (odb_source_freshen_object(&dir->packed->base, oid, mtime) ||
+ odb_source_freshen_object(&dir->loose->base, oid, mtime))
+ return 1;
+
return 0;
}
@@ -958,11 +991,13 @@ static int odb_source_files_fsck(struct odb_source *source,
struct odb_source_files *files = odb_source_files_downcast(source);
int ret = 0;
- if (!(opts->flags & ODB_FSCK_FULL) && !source->local)
- return 0;
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ if (!(opts->flags & ODB_FSCK_FULL) && !dir->local)
+ continue;
- ret |= odb_source_fsck(&files->dirs->loose->base, opts);
- ret |= odb_source_fsck(&files->dirs->packed->base, opts);
+ ret |= odb_source_fsck(&dir->loose->base, opts);
+ ret |= odb_source_fsck(&dir->packed->base, opts);
+ }
return ret;
}
diff --git a/odb/source-files.h b/odb/source-files.h
index 77f4d842e0..36af0c1b8b 100644
--- a/odb/source-files.h
+++ b/odb/source-files.h
@@ -15,6 +15,9 @@ struct odb_files_dir {
/* Absolute path to the object directory. */
char *abspath;
+ /* List of alternate object directories. */
+ struct odb_files_dir *next;
+
/* The two sources derived from this object directory. */
struct odb_source_loose *loose;
struct odb_source_packed *packed;
@@ -36,6 +39,12 @@ void odb_files_dir_free(struct odb_files_dir *dir);
*/
struct odb_source_files {
struct odb_source base;
+
+ /*
+ * List of all object directories; the main directory is first (and
+ * cannot be NULL after initialization). Subsequent directories are
+ * alternates.
+ */
struct odb_files_dir *dirs;
};
--
2.56.0.379.gc618271300.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* [PATCH 07/13] tmp-objdir: absorb logic to set and restore primary sources
2026-10-02 10:08 [PATCH 00/13] odb/source-files: move alternates into the backend Patrick Steinhardt
` (5 preceding siblings ...)
2026-10-02 10:08 ` [PATCH 06/13] odb/source-files: add the ability to have multiple object dirs Patrick Steinhardt
@ 2026-10-02 10:08 ` Patrick Steinhardt
2026-10-02 10:08 ` [PATCH 08/13] tmp-objdir: manage quarantine as an object directory Patrick Steinhardt
` (7 subsequent siblings)
14 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-02 10:08 UTC (permalink / raw)
To: git
The functions `odb_set_temporary_primary_source()` and
`odb_restore_primary_source()` can be used to adapt the primary object
database source. Nowadays though we only have a single user of this
subsystem left, which is the "tmp-objdir" subsystem.
Despite that, this functionality is also becoming less useful overall as
alternates are becoming an implementation detail of the "files" backend.
And with that change, there will only ever be a single source attached
to the object database anyway.
Move the logic into the "tmp-objdir" subsystem accordingly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
odb.c | 45 ---------------------------------------------
odb.h | 17 -----------------
tmp-objdir.c | 38 ++++++++++++++++++++++++++++++++++----
3 files changed, 34 insertions(+), 66 deletions(-)
diff --git a/odb.c b/odb.c
index 1dc8647159..8b54271c27 100644
--- a/odb.c
+++ b/odb.c
@@ -239,51 +239,6 @@ static struct odb_source *odb_add_alternate_recursively(struct object_database *
return alternate;
}
-struct odb_source *odb_set_temporary_primary_source(struct object_database *odb,
- const char *dir, int will_destroy,
- struct odb_source **prev_source)
-{
- struct odb_source *source;
-
- /*
- * Make a new primary odb and link the old primary ODB in as an
- * alternate
- */
- source = odb_source_new(odb, dir, false);
-
- /*
- * Disable ref updates while a temporary odb is active, since
- * the objects in the database may roll back.
- */
- odb->repo->disable_ref_updates = true;
- source->will_destroy = will_destroy;
- source->next = odb->sources;
- odb->sources = source;
-
- if (prev_source)
- *prev_source = source->next;
-
- return source;
-}
-
-void odb_restore_primary_source(struct object_database *odb,
- struct odb_source *restore_source,
- const char *old_path)
-{
- struct odb_source *cur_source = odb->sources;
-
- if (strcmp(old_path, cur_source->path))
- BUG("expected %s as primary object store; found %s",
- old_path, cur_source->path);
-
- if (cur_source->next != restore_source)
- BUG("we expect the old primary object store to be the first alternate");
-
- odb->repo->disable_ref_updates = false;
- odb->sources = restore_source;
- odb_source_free(cur_source);
-}
-
char *compute_alternate_path(const char *path, struct strbuf *err)
{
char *ref_git = NULL;
diff --git a/odb.h b/odb.h
index 5c86572b5d..4143812f55 100644
--- a/odb.h
+++ b/odb.h
@@ -226,23 +226,6 @@ struct odb_fsck_options {
*/
int odb_fsck(struct object_database *odb, struct odb_fsck_options *opts);
-/*
- * Replace the current writable object directory with the specified temporary
- * object directory and return the newly installed primary source. The former
- * primary source is reported via `prev_source` when non-NULL.
- */
-struct odb_source *odb_set_temporary_primary_source(struct object_database *odb,
- const char *dir, int will_destroy,
- struct odb_source **prev_source);
-
-/*
- * Restore the primary source that was previously replaced by
- * `odb_set_temporary_primary_source()`.
- */
-void odb_restore_primary_source(struct object_database *odb,
- struct odb_source *restore_source,
- const char *old_path);
-
/*
* Iterate through all alternates of the database and execute the provided
* callback function for each of them. Stop iterating once the callback
diff --git a/tmp-objdir.c b/tmp-objdir.c
index deaaf6ba2e..31a7920be7 100644
--- a/tmp-objdir.c
+++ b/tmp-objdir.c
@@ -51,6 +51,26 @@ static void tmp_objdir_reparent(const char *old_cwd,
free(path);
}
+/*
+ * Restore the primary source that was previously replaced by
+ * `tmp_objdir_replace_primary_odb()`.
+ */
+static void tmp_objdir_restore_source(struct tmp_objdir *t)
+{
+ struct odb_source *cur_source = t->repo->objects->sources;
+
+ if (strcmp(t->path.buf, cur_source->path))
+ BUG("expected %s as primary object store; found %s",
+ t->path.buf, cur_source->path);
+
+ if (cur_source->next != t->prev_source)
+ BUG("we expect the old primary object store to be the first alternate");
+
+ t->repo->disable_ref_updates = false;
+ t->repo->objects->sources = t->prev_source;
+ odb_source_free(cur_source);
+}
+
int tmp_objdir_destroy(struct tmp_objdir *t)
{
int err;
@@ -62,7 +82,7 @@ int tmp_objdir_destroy(struct tmp_objdir *t)
the_tmp_objdir = NULL;
if (t->prev_source)
- odb_restore_primary_source(t->repo->objects, t->prev_source, t->path.buf);
+ tmp_objdir_restore_source(t);
err = remove_dir_recursively(&t->path, 0);
@@ -298,7 +318,7 @@ int tmp_objdir_migrate(struct tmp_objdir *t)
if (t->prev_source) {
if (t->repo->objects->sources->will_destroy)
BUG("migrating an ODB that was marked for destruction");
- odb_restore_primary_source(t->repo->objects, t->prev_source, t->path.buf);
+ tmp_objdir_restore_source(t);
t->prev_source = NULL;
}
@@ -328,6 +348,16 @@ struct odb_source *tmp_objdir_replace_primary_odb(struct tmp_objdir *t,
BUG("the primary object database is already replaced");
t->will_destroy = will_destroy;
- return odb_set_temporary_primary_source(t->repo->objects, t->path.buf,
- will_destroy, &t->prev_source);
+ /*
+ * Make a new primary source and link the old primary source in as an
+ * alternate. Disable ref updates while a temporary source is active,
+ * since the objects in the database may roll back.
+ */
+ t->prev_source = t->repo->objects->sources;
+ t->repo->objects->sources = odb_source_new(t->repo->objects, t->path.buf, false);
+ t->repo->objects->sources->next = t->prev_source;
+ t->repo->objects->sources->will_destroy = will_destroy;
+ t->repo->disable_ref_updates = true;
+
+ return t->repo->objects->sources;
}
--
2.56.0.379.gc618271300.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* [PATCH 08/13] tmp-objdir: manage quarantine as an object directory
2026-10-02 10:08 [PATCH 00/13] odb/source-files: move alternates into the backend Patrick Steinhardt
` (6 preceding siblings ...)
2026-10-02 10:08 ` [PATCH 07/13] tmp-objdir: absorb logic to set and restore primary sources Patrick Steinhardt
@ 2026-10-02 10:08 ` Patrick Steinhardt
2026-10-06 20:18 ` Karthik Nayak
2026-10-02 10:08 ` [PATCH 09/13] tmp-objdir: replace primary source at creation time Patrick Steinhardt
` (6 subsequent siblings)
14 siblings, 1 reply; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-02 10:08 UTC (permalink / raw)
To: git
When creating a quarantine directory via the "tmp-objdir" subsystem we
create a new "files" backend that new objects part of the transaction
can be written to. In a future commit though we'll move handling of
alternates into the "files" backend, and as part of that it will no
longer be possible for us to have multiple sources attached to a single
object database.
In a preceding commit, we have prepared the "files" backend to be able
to handle multiple object directories. We don't use that mechanism for
alternates yet, but will start doing so in a subsequent commit. But with
that infrastructure ready we can already migrate tmp-objdirs over to use
this new mechanism.
Adapt the subsystem so we create a `struct odb_files_dir` instead of a
new "files" source.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
object-file.c | 4 ++--
tmp-objdir.c | 48 +++++++++++++++++++++++++++++-------------------
tmp-objdir.h | 4 ++--
3 files changed, 33 insertions(+), 23 deletions(-)
diff --git a/object-file.c b/object-file.c
index a4cbf8b081..fe9139c96d 100644
--- a/object-file.c
+++ b/object-file.c
@@ -488,7 +488,7 @@ struct odb_transaction_files {
enum odb_transaction_flags flags;
struct tmp_objdir *objdir;
- struct odb_source *quarantine;
+ struct odb_files_dir *quarantine;
struct transaction_packfile packfile;
const char *prefix;
@@ -1443,7 +1443,7 @@ static int odb_transaction_files_write_pack(struct odb_transaction *base,
return -1;
}
- odb_source_prepare(transaction->quarantine,
+ odb_source_prepare(&transaction->quarantine->packed->base,
ODB_PREPARE_FLUSH_CACHES);
}
diff --git a/tmp-objdir.c b/tmp-objdir.c
index 31a7920be7..a3903b6e6d 100644
--- a/tmp-objdir.c
+++ b/tmp-objdir.c
@@ -12,13 +12,17 @@
#include "quote.h"
#include "odb.h"
#include "odb/source.h"
+#include "odb/source-files.h"
+#include "odb/source-loose.h"
+#include "odb/source-packed.h"
#include "repository.h"
struct tmp_objdir {
struct repository *repo;
struct strbuf path;
struct strvec env;
- struct odb_source *prev_source;
+ struct odb_files_dir *temp_dir;
+ struct odb_files_dir *orig_dir;
int will_destroy;
};
@@ -57,18 +61,19 @@ static void tmp_objdir_reparent(const char *old_cwd,
*/
static void tmp_objdir_restore_source(struct tmp_objdir *t)
{
- struct odb_source *cur_source = t->repo->objects->sources;
+ struct odb_source_files *files = odb_source_files_downcast(t->repo->objects->sources);
+ struct odb_files_dir *cur_dir = files->dirs;
- if (strcmp(t->path.buf, cur_source->path))
+ if (t->temp_dir != files->dirs)
BUG("expected %s as primary object store; found %s",
- t->path.buf, cur_source->path);
+ t->temp_dir->abspath, cur_dir->abspath);
- if (cur_source->next != t->prev_source)
+ if (t->temp_dir->next != t->orig_dir)
BUG("we expect the old primary object store to be the first alternate");
t->repo->disable_ref_updates = false;
- t->repo->objects->sources = t->prev_source;
- odb_source_free(cur_source);
+ files->dirs = t->orig_dir;
+ odb_files_dir_free(cur_dir);
}
int tmp_objdir_destroy(struct tmp_objdir *t)
@@ -81,7 +86,7 @@ int tmp_objdir_destroy(struct tmp_objdir *t)
if (t == the_tmp_objdir)
the_tmp_objdir = NULL;
- if (t->prev_source)
+ if (t->orig_dir)
tmp_objdir_restore_source(t);
err = remove_dir_recursively(&t->path, 0);
@@ -315,11 +320,11 @@ int tmp_objdir_migrate(struct tmp_objdir *t)
if (!t)
return 0;
- if (t->prev_source) {
- if (t->repo->objects->sources->will_destroy)
+ if (t->orig_dir) {
+ if (t->will_destroy)
BUG("migrating an ODB that was marked for destruction");
tmp_objdir_restore_source(t);
- t->prev_source = NULL;
+ t->orig_dir = NULL;
}
strbuf_addbuf(&src, &t->path);
@@ -341,10 +346,12 @@ const char **tmp_objdir_env(const struct tmp_objdir *t)
return t->env.v;
}
-struct odb_source *tmp_objdir_replace_primary_odb(struct tmp_objdir *t,
- int will_destroy)
+struct odb_files_dir *tmp_objdir_replace_primary_odb(struct tmp_objdir *t,
+ int will_destroy)
{
- if (t->prev_source)
+ struct odb_source_files *files = odb_source_files_downcast(t->repo->objects->sources);
+
+ if (t->temp_dir)
BUG("the primary object database is already replaced");
t->will_destroy = will_destroy;
@@ -353,11 +360,14 @@ struct odb_source *tmp_objdir_replace_primary_odb(struct tmp_objdir *t,
* alternate. Disable ref updates while a temporary source is active,
* since the objects in the database may roll back.
*/
- t->prev_source = t->repo->objects->sources;
- t->repo->objects->sources = odb_source_new(t->repo->objects, t->path.buf, false);
- t->repo->objects->sources->next = t->prev_source;
- t->repo->objects->sources->will_destroy = will_destroy;
+ t->temp_dir = odb_files_dir_new(t->repo->objects, t->path.buf, false);
+ t->temp_dir->loose->base.will_destroy = will_destroy;
+ t->temp_dir->packed->base.will_destroy = will_destroy;
+ t->temp_dir->next = files->dirs;
+
+ t->orig_dir = files->dirs;
+ files->dirs = t->temp_dir;
t->repo->disable_ref_updates = true;
- return t->repo->objects->sources;
+ return t->temp_dir;
}
diff --git a/tmp-objdir.h b/tmp-objdir.h
index 05f0d08d10..5ed0db1ee0 100644
--- a/tmp-objdir.h
+++ b/tmp-objdir.h
@@ -61,7 +61,7 @@ void tmp_objdir_discard_objects(struct tmp_objdir *);
* If will_destroy is nonzero, the object directory may not be migrated. Returns
* the newly installed primary source.
*/
-struct odb_source *tmp_objdir_replace_primary_odb(struct tmp_objdir *,
- int will_destroy);
+struct odb_files_dir *tmp_objdir_replace_primary_odb(struct tmp_objdir *,
+ int will_destroy);
#endif /* TMP_OBJDIR_H */
--
2.56.0.379.gc618271300.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* Re: [PATCH 08/13] tmp-objdir: manage quarantine as an object directory
2026-10-02 10:08 ` [PATCH 08/13] tmp-objdir: manage quarantine as an object directory Patrick Steinhardt
@ 2026-10-06 20:18 ` Karthik Nayak
2026-10-06 20:27 ` Karthik Nayak
0 siblings, 1 reply; 41+ messages in thread
From: Karthik Nayak @ 2026-10-06 20:18 UTC (permalink / raw)
To: Patrick Steinhardt, git
[-- Attachment #1: Type: text/plain, Size: 1205 bytes --]
Patrick Steinhardt <ps@pks.im> writes:
> When creating a quarantine directory via the "tmp-objdir" subsystem we
> create a new "files" backend that new objects part of the transaction
> can be written to. In a future commit though we'll move handling of
> alternates into the "files" backend, and as part of that it will no
> longer be possible for us to have multiple sources attached to a single
> object database.
I understand the moving of the alternates to the files source, I didn't
grasp the need for removal of multiples sources from the object
database. Wouldn't it hypothetically make sense to have different
sources which use different strategies based on the type of objects?
> In a preceding commit, we have prepared the "files" backend to be able
> to handle multiple object directories. We don't use that mechanism for
> alternates yet, but will start doing so in a subsequent commit. But with
> that infrastructure ready we can already migrate tmp-objdirs over to use
> this new mechanism.
>
> Adapt the subsystem so we create a `struct odb_files_dir` instead of a
> new "files" source.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
[snip]
The changes themselves look good to me!
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH 08/13] tmp-objdir: manage quarantine as an object directory
2026-10-06 20:18 ` Karthik Nayak
@ 2026-10-06 20:27 ` Karthik Nayak
0 siblings, 0 replies; 41+ messages in thread
From: Karthik Nayak @ 2026-10-06 20:27 UTC (permalink / raw)
To: Patrick Steinhardt, git
[-- Attachment #1: Type: text/plain, Size: 1352 bytes --]
Karthik Nayak <karthik.188@gmail.com> writes:
> Patrick Steinhardt <ps@pks.im> writes:
>
>> When creating a quarantine directory via the "tmp-objdir" subsystem we
>> create a new "files" backend that new objects part of the transaction
>> can be written to. In a future commit though we'll move handling of
>> alternates into the "files" backend, and as part of that it will no
>> longer be possible for us to have multiple sources attached to a single
>> object database.
>
> I understand the moving of the alternates to the files source, I didn't
> grasp the need for removal of multiples sources from the object
> database. Wouldn't it hypothetically make sense to have different
> sources which use different strategies based on the type of objects?
>
I see this is explained further down in 12/13, so ignore this!
>> In a preceding commit, we have prepared the "files" backend to be able
>> to handle multiple object directories. We don't use that mechanism for
>> alternates yet, but will start doing so in a subsequent commit. But with
>> that infrastructure ready we can already migrate tmp-objdirs over to use
>> this new mechanism.
>>
>> Adapt the subsystem so we create a `struct odb_files_dir` instead of a
>> new "files" source.
>>
>> Signed-off-by: Patrick Steinhardt <ps@pks.im>
>
> [snip]
>
> The changes themselves look good to me!
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 41+ messages in thread
* [PATCH 09/13] tmp-objdir: replace primary source at creation time
2026-10-02 10:08 [PATCH 00/13] odb/source-files: move alternates into the backend Patrick Steinhardt
` (7 preceding siblings ...)
2026-10-02 10:08 ` [PATCH 08/13] tmp-objdir: manage quarantine as an object directory Patrick Steinhardt
@ 2026-10-02 10:08 ` Patrick Steinhardt
2026-10-02 10:08 ` [PATCH 10/13] odb/source: make `will_destroy` an implementation detail Patrick Steinhardt
` (5 subsequent siblings)
14 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-02 10:08 UTC (permalink / raw)
To: git
When creating transactions via the "tmp-objdir" subsystem callers are
expected to first call `tmp_objdir_create()` to create the quarantine
directory and then `tmp_objdir_replace_primary_odb()` to activate that
quarantine directory as the primary object database source so that all
newly written objects are written into it.
This dance is performed by all users of temporary object directories, so
that makes the setup unnecessarily involved. Merge the logic to replace
the primary object database source into `tmp_objdir_create()` to
simplify the dance.
Note that with the quarantine directory being installed by
`tmp_objdir_create()`, users of this function no longer get a pointer to
the underlying quarantine directory. So instead of repreparing only that
directory, we now reprepare the whole source instead, and as we know
that the source has wired up the quarantine directory already, this will
cause us to reprepare the quarantine directory, too.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
log-tree.c | 3 +--
object-file.c | 8 ++------
tmp-objdir.c | 47 +++++++++++++++++++----------------------------
tmp-objdir.h | 18 +++++-------------
4 files changed, 27 insertions(+), 49 deletions(-)
diff --git a/log-tree.c b/log-tree.c
index 83a3c4bf9b..361dfadbb3 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -1043,10 +1043,9 @@ static int do_remerge_diff(struct rev_info *opt,
* into the alternative object store list as the primary.
*/
if (opt->remerge_diff && !opt->remerge_objdir) {
- opt->remerge_objdir = tmp_objdir_create(the_repository, "remerge-diff");
+ opt->remerge_objdir = tmp_objdir_create(the_repository, "remerge-diff", 1);
if (!opt->remerge_objdir)
return error(_("unable to create temporary object directory"));
- tmp_objdir_replace_primary_odb(opt->remerge_objdir, 1);
}
/* Setup merge options */
diff --git a/object-file.c b/object-file.c
index fe9139c96d..02521afcaf 100644
--- a/object-file.c
+++ b/object-file.c
@@ -488,7 +488,6 @@ struct odb_transaction_files {
enum odb_transaction_flags flags;
struct tmp_objdir *objdir;
- struct odb_files_dir *quarantine;
struct transaction_packfile packfile;
const char *prefix;
@@ -511,12 +510,10 @@ int odb_transaction_files_prepare(struct odb_transaction *base)
if (!transaction || transaction->objdir)
return 0;
- transaction->objdir = tmp_objdir_create(base->source->odb->repo, transaction->prefix);
+ transaction->objdir = tmp_objdir_create(base->source->odb->repo, transaction->prefix, 0);
if (!transaction->objdir)
return error(_("unable to create temporary object directory"));
- transaction->quarantine = tmp_objdir_replace_primary_odb(transaction->objdir, 0);
-
return 0;
}
@@ -1443,8 +1440,7 @@ static int odb_transaction_files_write_pack(struct odb_transaction *base,
return -1;
}
- odb_source_prepare(&transaction->quarantine->packed->base,
- ODB_PREPARE_FLUSH_CACHES);
+ odb_source_prepare(base->source, ODB_PREPARE_FLUSH_CACHES);
}
return 0;
diff --git a/tmp-objdir.c b/tmp-objdir.c
index a3903b6e6d..719b55e580 100644
--- a/tmp-objdir.c
+++ b/tmp-objdir.c
@@ -57,7 +57,7 @@ static void tmp_objdir_reparent(const char *old_cwd,
/*
* Restore the primary source that was previously replaced by
- * `tmp_objdir_replace_primary_odb()`.
+ * `tmp_objdir_create()`.
*/
static void tmp_objdir_restore_source(struct tmp_objdir *t)
{
@@ -157,8 +157,10 @@ static int setup_tmp_objdir(const char *root)
}
struct tmp_objdir *tmp_objdir_create(struct repository *r,
- const char *prefix)
+ const char *prefix,
+ int will_destroy)
{
+ struct odb_source_files *files = odb_source_files_downcast(r->objects->sources);
static int installed_handlers;
struct tmp_objdir *t;
@@ -167,6 +169,7 @@ struct tmp_objdir *tmp_objdir_create(struct repository *r,
t = xcalloc(1, sizeof(*t));
t->repo = r;
+ t->will_destroy = will_destroy;
strbuf_init(&t->path, 0);
strvec_init(&t->env);
@@ -204,6 +207,20 @@ struct tmp_objdir *tmp_objdir_create(struct repository *r,
env_replace(&t->env, GIT_QUARANTINE_ENVIRONMENT,
absolute_path(t->path.buf));
+ /*
+ * Make a new primary source and link the old primary source in as an
+ * alternate. Disable ref updates while a temporary source is active,
+ * since the objects in the database may roll back.
+ */
+ t->temp_dir = odb_files_dir_new(t->repo->objects, t->path.buf, false);
+ t->temp_dir->loose->base.will_destroy = will_destroy;
+ t->temp_dir->packed->base.will_destroy = will_destroy;
+ t->temp_dir->next = files->dirs;
+
+ t->orig_dir = files->dirs;
+ files->dirs = t->temp_dir;
+ t->repo->disable_ref_updates = true;
+
return t;
}
@@ -345,29 +362,3 @@ const char **tmp_objdir_env(const struct tmp_objdir *t)
return NULL;
return t->env.v;
}
-
-struct odb_files_dir *tmp_objdir_replace_primary_odb(struct tmp_objdir *t,
- int will_destroy)
-{
- struct odb_source_files *files = odb_source_files_downcast(t->repo->objects->sources);
-
- if (t->temp_dir)
- BUG("the primary object database is already replaced");
- t->will_destroy = will_destroy;
-
- /*
- * Make a new primary source and link the old primary source in as an
- * alternate. Disable ref updates while a temporary source is active,
- * since the objects in the database may roll back.
- */
- t->temp_dir = odb_files_dir_new(t->repo->objects, t->path.buf, false);
- t->temp_dir->loose->base.will_destroy = will_destroy;
- t->temp_dir->packed->base.will_destroy = will_destroy;
- t->temp_dir->next = files->dirs;
-
- t->orig_dir = files->dirs;
- files->dirs = t->temp_dir;
- t->repo->disable_ref_updates = true;
-
- return t->temp_dir;
-}
diff --git a/tmp-objdir.h b/tmp-objdir.h
index 5ed0db1ee0..a301b01d84 100644
--- a/tmp-objdir.h
+++ b/tmp-objdir.h
@@ -11,7 +11,7 @@
* Example:
*
* struct child_process child = CHILD_PROCESS_INIT;
- * struct tmp_objdir *t = tmp_objdir_create(repo, "incoming");
+ * struct tmp_objdir *t = tmp_objdir_create(repo, "incoming", 0);
* strvec_push(&child.args, cmd);
* strvec_pushv(&child.env, tmp_objdir_env(t));
* if (!run_command(&child)) && !tmp_objdir_migrate(t))
@@ -25,10 +25,11 @@ struct repository;
struct tmp_objdir;
/*
- * Create a new temporary object directory with the specified prefix;
- * returns NULL on failure.
+ * Create a new temporary object directory with the specified prefix and
+ * install the directory as the primary write target; returns NULL on failure.
*/
-struct tmp_objdir *tmp_objdir_create(struct repository *r, const char *prefix);
+struct tmp_objdir *tmp_objdir_create(struct repository *r, const char *prefix,
+ int will_destroy);
/*
* Return a list of environment strings, suitable for use with
@@ -55,13 +56,4 @@ int tmp_objdir_destroy(struct tmp_objdir *);
*/
void tmp_objdir_discard_objects(struct tmp_objdir *);
-/*
- * Replaces the writable object store in the current process with the temporary
- * object directory and makes the former main object store an alternate.
- * If will_destroy is nonzero, the object directory may not be migrated. Returns
- * the newly installed primary source.
- */
-struct odb_files_dir *tmp_objdir_replace_primary_odb(struct tmp_objdir *,
- int will_destroy);
-
#endif /* TMP_OBJDIR_H */
--
2.56.0.379.gc618271300.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* [PATCH 10/13] odb/source: make `will_destroy` an implementation detail
2026-10-02 10:08 [PATCH 00/13] odb/source-files: move alternates into the backend Patrick Steinhardt
` (8 preceding siblings ...)
2026-10-02 10:08 ` [PATCH 09/13] tmp-objdir: replace primary source at creation time Patrick Steinhardt
@ 2026-10-02 10:08 ` Patrick Steinhardt
2026-10-02 10:08 ` [PATCH 11/13] odb/source-files: extract reading alternates Patrick Steinhardt
` (4 subsequent siblings)
14 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-02 10:08 UTC (permalink / raw)
To: git
The `struct odb_source::will_destroy` flag tracks whether a source is
part of a transaction that we know we'll destruct anyway. If so, the
backend can optimize for that particular case, for example by not
flushing any data to disk.
While the intent is sensible, it assumes that transactions are backed by
a separate source that's being linked into the object database. But that
may or may not be true, as backends may have significantly better ways
to achieve the same. So the assumption doesn't make much sense in the
first place, as we're now tracking backend-specific details on the
generic `struct odb_source` level.
Move the field from the generic source into the "loose" source, as this
is the only source that ever makes use of it anyway.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
odb/source-loose.c | 2 +-
odb/source-loose.h | 3 +++
odb/source.h | 5 -----
tmp-objdir.c | 4 +---
4 files changed, 5 insertions(+), 9 deletions(-)
diff --git a/odb/source-loose.c b/odb/source-loose.c
index 3c9edba46a..b2fbccd3c0 100644
--- a/odb/source-loose.c
+++ b/odb/source-loose.c
@@ -599,7 +599,7 @@ static int odb_source_loose_freshen_object(struct odb_source *source,
static void close_loose_object(struct odb_source_loose *loose,
int fd, const char *filename)
{
- if (loose->base.will_destroy)
+ if (loose->will_destroy)
goto out;
if (batch_fsync_enabled(FSYNC_COMPONENT_LOOSE_OBJECT))
diff --git a/odb/source-loose.h b/odb/source-loose.h
index 3cf2e1f8f1..7c7e845fe3 100644
--- a/odb/source-loose.h
+++ b/odb/source-loose.h
@@ -28,6 +28,9 @@ struct odb_source_loose {
/* Map between object IDs for loose objects. */
struct loose_object_map *map;
+
+ /* Whether this is a source that will never be committed to disk. */
+ int will_destroy;
};
struct odb_source_loose *odb_source_loose_new(struct object_database *odb,
diff --git a/odb/source.h b/odb/source.h
index ea00873763..9fd2b2e5b5 100644
--- a/odb/source.h
+++ b/odb/source.h
@@ -81,11 +81,6 @@ struct odb_source {
*/
bool local;
- /*
- * This object store is ephemeral, so there is no need to fsync.
- */
- int will_destroy;
-
/*
* Path to the source. If this is a relative path, it is relative to
* the current working directory.
diff --git a/tmp-objdir.c b/tmp-objdir.c
index 719b55e580..2f2ffbbc7d 100644
--- a/tmp-objdir.c
+++ b/tmp-objdir.c
@@ -14,7 +14,6 @@
#include "odb/source.h"
#include "odb/source-files.h"
#include "odb/source-loose.h"
-#include "odb/source-packed.h"
#include "repository.h"
struct tmp_objdir {
@@ -213,8 +212,7 @@ struct tmp_objdir *tmp_objdir_create(struct repository *r,
* since the objects in the database may roll back.
*/
t->temp_dir = odb_files_dir_new(t->repo->objects, t->path.buf, false);
- t->temp_dir->loose->base.will_destroy = will_destroy;
- t->temp_dir->packed->base.will_destroy = will_destroy;
+ t->temp_dir->loose->will_destroy = will_destroy;
t->temp_dir->next = files->dirs;
t->orig_dir = files->dirs;
--
2.56.0.379.gc618271300.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* [PATCH 11/13] odb/source-files: extract reading alternates
2026-10-02 10:08 [PATCH 00/13] odb/source-files: move alternates into the backend Patrick Steinhardt
` (9 preceding siblings ...)
2026-10-02 10:08 ` [PATCH 10/13] odb/source: make `will_destroy` an implementation detail Patrick Steinhardt
@ 2026-10-02 10:08 ` Patrick Steinhardt
2026-10-02 10:08 ` [PATCH 12/13] odb/source-files: move alternates into the backend Patrick Steinhardt
` (3 subsequent siblings)
14 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-02 10:08 UTC (permalink / raw)
To: git
In the next commit we'll add a new callsite that wants to read
alternates without having a proper object database source for a given
alternate available to it. Prepare for this by extracting the logic into
a separate function that only requires an object directory path as
input.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
odb/source-files.c | 33 +++++++++++++++++++--------------
1 file changed, 19 insertions(+), 14 deletions(-)
diff --git a/odb/source-files.c b/odb/source-files.c
index 9389546b3e..6aaf625352 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -174,6 +174,24 @@ static int odb_source_files_create_on_disk(struct odb_source *source,
return ret;
}
+static int read_alternates(const char *object_dir, struct strvec *out)
+{
+ struct strbuf buf = STRBUF_INIT;
+ char *path;
+
+ path = xstrfmt("%s/info/alternates", object_dir);
+ if (strbuf_read_file(&buf, path, 1024) < 0) {
+ warn_on_fopen_errors(path);
+ free(path);
+ return 0;
+ }
+ parse_alternates(buf.buf, '\n', object_dir, out);
+
+ strbuf_release(&buf);
+ free(path);
+ return 0;
+}
+
static void odb_source_files_prepare(struct odb_source *source,
enum odb_prepare_flags flags)
{
@@ -364,20 +382,7 @@ static int odb_source_files_begin_transaction(struct odb_source *source,
static int odb_source_files_read_alternates(struct odb_source *source,
struct strvec *out)
{
- struct strbuf buf = STRBUF_INIT;
- char *path;
-
- path = xstrfmt("%s/info/alternates", source->path);
- if (strbuf_read_file(&buf, path, 1024) < 0) {
- warn_on_fopen_errors(path);
- free(path);
- return 0;
- }
- parse_alternates(buf.buf, '\n', source->path, out);
-
- strbuf_release(&buf);
- free(path);
- return 0;
+ return read_alternates(source->path, out);
}
static int too_many_loose_objects(struct odb_source_files *files, int limit)
--
2.56.0.379.gc618271300.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* [PATCH 12/13] odb/source-files: move alternates into the backend
2026-10-02 10:08 [PATCH 00/13] odb/source-files: move alternates into the backend Patrick Steinhardt
` (10 preceding siblings ...)
2026-10-02 10:08 ` [PATCH 11/13] odb/source-files: extract reading alternates Patrick Steinhardt
@ 2026-10-02 10:08 ` Patrick Steinhardt
2026-10-06 20:51 ` Karthik Nayak
2026-10-02 10:08 ` [PATCH 13/13] odb/source: drop `read_alternates` callback Patrick Steinhardt
` (2 subsequent siblings)
14 siblings, 1 reply; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-02 10:08 UTC (permalink / raw)
To: git
Originally, when designing pluggable object databases the goal was that
the object database can have multiple sources, and every source attached
to it could use a different backend. This would have allowed for quite a
lot of flexibility, as you could trivially mix and match different kinds
of object storages in whatever way you like.
But while well-intentioned, this design led to a bunch of conceptual
problems:
- We're now trying to read objects in source order, whereas we
previously tried to read objects via packfiles before trying to read
them via loose objects. This led to a performance regression when
using alternates or when using a quarantine directory.
- Some data structures are supposed to only ever exist once, like for
example bitmaps and commit graphs. At the same time, those data
structures also span across the union of all objects, so they may
cross sources.
- It is unclear how we can extend GIT_OBJECT_DIRECTORY or
GIT_ALTERNATE_OBJECT_DIRECTORIES to become backend-agnostic in a
backwards-compatible way. In general, introducing an object storage
extension into the current status quo where alternates may have to
be extended to become generic was proving to be painful.
- Some mechanisms of alternates assume way too much about how exactly
their backends work. Alternate refs for example assume that the
alternate is backed by a filesystem path, and that this filesystem
path may also allow us to read references. This is not a given
though, as backends may not even have local data at all.
In short, there are a bunch of conceptual mismatches when we have
alternates and pluggable object databases coexist. So while the original
idea was nice, it does not result in a system that is easy to reason
about.
Correct course by moving alternates into the "files" source itself so
that it becomes an implementation detail thereof so that we can avoid
all of these shortcomings. While it's unfortunate that we cannot easily
mix and match sources now, that ability doesn't go away. It's still very
much feasible to introduce a new backend that allows for exactly that
use case, and such a backend may also be a lot more flexible as we can
now add new logic to determine which objects should be stored where. So
the original motivation for having per-source backends can still be
realized with the new architecture.
Note that as part of this move, we also handle the GIT_OBJECT_DIRECTORY
and GIT_ALTERNATE_OBJECT_DIRECTORIES environment variables in the
"files" backend. This may be surprising at first, but object directories
are very much a concept of that backend, too. So these variables would
have bad interactions with other backends, and they create a bit of a
mismatch with the eventual object storage extension that we plan to
introduce.
Note that this commit is way larger than I'd like it to be. I'm sorry, I
couldn't find a way to split it up further. That being said, most of the
changes are straight-forward conversions that go from iterating over
sources to iterating over object directories. The more involved changes
are moving the infrastructure to track individual sources from "odb.c"
and moving them into "odb/source-files.c".
This also serves to show a bit of a who's-who of commands that don't
work properly with pluggable object databases. Almost all of these
commands are related to housekeeping though -- some of them will be
converted eventually, like for example commit graphs or MIDXs. But many
of them will stay incompatible going forward as they are simply too
specific to the "files" backend.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/count-objects.c | 2 +-
builtin/fast-import.c | 18 +--
builtin/fetch.c | 2 +-
builtin/fsck.c | 6 +-
builtin/gc.c | 14 +-
builtin/index-pack.c | 2 +-
builtin/multi-pack-index.c | 6 +-
builtin/pack-objects.c | 64 ++++----
builtin/prune.c | 2 +-
bundle.c | 2 +-
commit-graph.c | 28 ++--
diagnose.c | 2 +-
fetch-pack.c | 2 +-
http-walker.c | 2 +-
http.c | 6 +-
loose.c | 16 +-
midx.c | 29 ++--
odb.c | 354 +++++----------------------------------------
odb.h | 33 +----
odb/source-files.c | 260 ++++++++++++++++++++++++++++++---
odb/source-files.h | 23 ++-
odb/source.c | 5 +-
odb/source.h | 25 +---
odb/streaming.c | 8 +-
odb/transaction.c | 2 +-
pack-bitmap.c | 8 +-
packfile.c | 28 ++--
packfile.h | 21 ++-
path.c | 2 +-
prune-packed.c | 2 +-
repack.c | 4 +-
repository.c | 4 +-
setup.c | 2 +-
t/helper/test-read-graph.c | 2 +-
tmp-objdir.c | 4 +-
35 files changed, 441 insertions(+), 549 deletions(-)
diff --git a/builtin/count-objects.c b/builtin/count-objects.c
index f2abfaccec..3f7b20fde5 100644
--- a/builtin/count-objects.c
+++ b/builtin/count-objects.c
@@ -118,7 +118,7 @@ int cmd_count_objects(int argc,
report_linked_checkout_garbage(the_repository);
}
- for_each_loose_file_in_source(the_repository->objects->sources,
+ for_each_loose_file_in_source(the_repository->objects->source,
count_loose, count_cruft, NULL, NULL);
if (verbose) {
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index 0bf76b028b..5fc082bcc7 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -895,7 +895,7 @@ static void end_packfile(void)
running = 1;
clear_delta_base_cache();
if (object_count) {
- struct odb_source_files *files = odb_source_files_downcast(pack_data->repo->objects->sources);
+ struct odb_source_files *files = odb_source_files_downcast(pack_data->repo->objects->source);
struct packed_git *new_p;
struct object_id cur_pack_oid;
char *idx_name;
@@ -975,7 +975,7 @@ static int store_object(
struct object_id *oidout,
uintmax_t mark)
{
- struct odb_source *source;
+ struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->source);
void *out, *delta;
struct object_entry *e;
unsigned char hdr[96];
@@ -1002,10 +1002,8 @@ static int store_object(
return 1;
}
- for (source = the_repository->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
-
- if (!packfile_list_find_oid(packfile_store_get_packs(files->dirs->packed), &oid))
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ if (!packfile_list_find_oid(packfile_store_get_packs(dir->packed), &oid))
continue;
e->type = type;
e->pack_id = MAX_PACK_ID;
@@ -1125,10 +1123,10 @@ static void truncate_pack(struct hashfile_checkpoint *checkpoint)
static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
{
+ struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->source);
size_t in_sz = 64 * 1024, out_sz = 64 * 1024;
unsigned char *in_buf = xmalloc(in_sz);
unsigned char *out_buf = xmalloc(out_sz);
- struct odb_source *source;
struct object_entry *e;
struct object_id oid;
unsigned long hdrlen;
@@ -1212,10 +1210,8 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
goto out;
}
- for (source = the_repository->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
-
- if (!packfile_list_find_oid(packfile_store_get_packs(files->dirs->packed), &oid))
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ if (!packfile_list_find_oid(packfile_store_get_packs(dir->packed), &oid))
continue;
e->type = OBJ_BLOB;
e->pack_id = MAX_PACK_ID;
diff --git a/builtin/fetch.c b/builtin/fetch.c
index a4b21d2651..2e9c14d4ed 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -2902,7 +2902,7 @@ int cmd_fetch(int argc,
trace2_region_enter("fetch", "write-commit-graph", the_repository);
write_commit_graph_reachable(the_repository,
- the_repository->objects->sources->path,
+ the_repository->objects->source->path,
commit_graph_flags,
NULL);
trace2_region_leave("fetch", "write-commit-graph", the_repository);
diff --git a/builtin/fsck.c b/builtin/fsck.c
index 9af4cc085b..da90cf84bf 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -868,7 +868,6 @@ int cmd_fsck(int argc,
OPT_BOOL(0, "references", &check_references, N_("check reference database consistency")),
OPT_END(),
};
- struct odb_source *source;
struct snapshot snap = {
.nr = 0,
.alloc = 0,
@@ -983,13 +982,14 @@ int cmd_fsck(int argc,
check_connectivity(repo);
if (repo->settings.core_commit_graph) {
+ struct odb_source_files *files = odb_source_files_downcast(repo->objects->source);
struct child_process commit_graph_verify = CHILD_PROCESS_INIT;
- for (source = repo->objects->sources; source; source = source->next) {
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
child_process_init(&commit_graph_verify);
commit_graph_verify.git_cmd = 1;
strvec_pushl(&commit_graph_verify.args, "commit-graph",
- "verify", "--object-dir", source->path, NULL);
+ "verify", "--object-dir", dir->abspath, NULL);
if (show_progress)
strvec_push(&commit_graph_verify.args, "--progress");
else
diff --git a/builtin/gc.c b/builtin/gc.c
index 7acd4f3215..28ada9f566 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -734,7 +734,7 @@ int cmd_gc(int argc,
if (the_repository->settings.gc_write_commit_graph == 1)
write_commit_graph_reachable(the_repository,
- the_repository->objects->sources->path,
+ the_repository->objects->source->path,
!opts.quiet && !daemonized ? COMMIT_GRAPH_WRITE_PROGRESS : 0,
NULL);
@@ -998,7 +998,7 @@ static int loose_object_auto_condition(struct gc_config *cfg UNUSED)
if (loose_object_auto_limit < 0)
return 1;
- return for_each_loose_file_in_source(the_repository->objects->sources,
+ return for_each_loose_file_in_source(the_repository->objects->source,
loose_object_count,
NULL, NULL, &count);
}
@@ -1033,7 +1033,7 @@ static int pack_loose(struct maintenance_run_opts *opts)
* Do not start pack-objects process
* if there are no loose objects.
*/
- if (!for_each_loose_file_in_source(r->objects->sources,
+ if (!for_each_loose_file_in_source(r->objects->source,
bail_on_loose,
NULL, NULL, NULL))
return 0;
@@ -1045,7 +1045,7 @@ static int pack_loose(struct maintenance_run_opts *opts)
strvec_push(&pack_proc.args, "--quiet");
else
strvec_push(&pack_proc.args, "--no-quiet");
- strvec_pushf(&pack_proc.args, "%s/pack/loose", r->objects->sources->path);
+ strvec_pushf(&pack_proc.args, "%s/pack/loose", r->objects->source->path);
pack_proc.in = -1;
@@ -1073,7 +1073,7 @@ static int pack_loose(struct maintenance_run_opts *opts)
else if (data.batch_size > 0)
data.batch_size--; /* Decrease for equality on limit. */
- for_each_loose_file_in_source(r->objects->sources,
+ for_each_loose_file_in_source(r->objects->source,
write_loose_object_to_stdin,
NULL, NULL, &data);
@@ -1396,7 +1396,7 @@ static int maintenance_run_tasks(struct maintenance_run_opts *opts,
int result = 0;
struct lock_file lk;
struct repository *r = the_repository;
- char *lock_path = xstrfmt("%s/maintenance", r->objects->sources->path);
+ char *lock_path = xstrfmt("%s/maintenance", r->objects->source->path);
enum auto_gc_hook_result auto_gc_hook_result = AUTO_GC_HOOK_UNDECIDED;
if (repo_hold_lock_file_for_update(r, &lk, lock_path, LOCK_NO_DEREF) < 0) {
@@ -2972,7 +2972,7 @@ static int update_background_schedule(const struct maintenance_start_opts *opts,
unsigned int i;
int result = 0;
struct lock_file lk;
- char *lock_path = xstrfmt("%s/schedule", the_repository->objects->sources->path);
+ char *lock_path = xstrfmt("%s/schedule", the_repository->objects->source->path);
if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) {
if (errno == EEXIST)
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 70860b8f27..0264c18c5f 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1640,7 +1640,7 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
if (do_fsck_object && startup_info->have_repository) {
struct odb_source_files *files =
- odb_source_files_downcast(the_repository->objects->sources);
+ odb_source_files_downcast(the_repository->objects->source);
packfile_store_load_pack(files->dirs->packed, final_index_name, 0);
}
diff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c
index fc8b494996..c48212290c 100644
--- a/builtin/multi-pack-index.c
+++ b/builtin/multi-pack-index.c
@@ -80,7 +80,7 @@ static int parse_object_dir(const struct option *opt, const char *arg,
char **value = opt->value;
free(*value);
if (unset)
- *value = xstrdup(the_repository->objects->sources->path);
+ *value = xstrdup(the_repository->objects->source->path);
else
*value = real_pathdup(arg, 1);
return 0;
@@ -426,8 +426,8 @@ int cmd_multi_pack_index(int argc,
if (the_repository &&
the_repository->objects &&
- the_repository->objects->sources)
- opts.object_dir = xstrdup(the_repository->objects->sources->path);
+ the_repository->objects->source)
+ opts.object_dir = xstrdup(the_repository->objects->source->path);
argc = parse_options(argc, argv, prefix, options,
builtin_multi_pack_index_usage, 0);
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 070659b6ed..ca3a891dfb 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1566,11 +1566,10 @@ static int want_cruft_object_mtime(struct repository *r,
const struct object_id *oid,
unsigned flags, uint32_t mtime)
{
- struct odb_source *source;
+ struct odb_source_files *files = odb_source_files_downcast(r->objects->source);
- for (source = r->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- struct packed_git **cache = packfile_store_get_kept_pack_cache(files->dirs->packed, flags);
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ struct packed_git **cache = packfile_store_get_kept_pack_cache(dir->packed, flags);
for (; *cache; cache++) {
struct packed_git *p = *cache;
@@ -1753,21 +1752,19 @@ static int want_object_in_pack_mtime(const struct object_id *oid,
off_t *found_offset,
uint32_t found_mtime)
{
+ struct odb_source_files *files =
+ odb_source_files_downcast(the_repository->objects->source);
int want;
struct packfile_list_entry *e;
- struct odb_source *source;
if (!exclude && local) {
/*
- * Note that we start iterating at `sources->next` so that we
- * skip the local object source.
+ * Note that we start iterating at `dirs->next` so that we
+ * skip the local object directory.
*/
- struct odb_source *source = the_repository->objects->sources->next;
- for (; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- if (!odb_source_read_object_info(&files->dirs->loose->base, oid, NULL, 0, NULL))
+ for (struct odb_files_dir *dir = files->dirs->next; dir; dir = dir->next)
+ if (!odb_source_read_object_info(&dir->loose->base, oid, NULL, 0, NULL))
return 0;
- }
}
/*
@@ -1785,9 +1782,8 @@ static int want_object_in_pack_mtime(const struct object_id *oid,
*found_offset = 0;
}
- for (source = the_repository->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- struct multi_pack_index *m = get_multi_pack_index(files->dirs->packed);
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ struct multi_pack_index *m = get_multi_pack_index(dir->packed);
struct pack_entry e;
if (m && midx_fill_entry(m, oid, &e, NULL) == MIDX_FILL_HIT) {
@@ -1797,14 +1793,12 @@ static int want_object_in_pack_mtime(const struct object_id *oid,
}
}
- for (source = the_repository->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
-
- for (e = files->dirs->packed->packs.head; e; e = e->next) {
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ for (e = dir->packed->packs.head; e; e = e->next) {
struct packed_git *p = e->pack;
want = want_object_in_pack_one(p, oid, exclude, found_pack, found_offset, found_mtime);
if (!exclude && want > 0)
- packfile_list_prepend(&files->dirs->packed->packs, p);
+ packfile_list_prepend(&dir->packed->packs, p);
if (want != -1)
return want;
}
@@ -4170,14 +4164,13 @@ static void add_cruft_object_entry(const struct object_id *oid, enum object_type
if (!want_object_in_pack_mtime(oid, 0, &pack, &offset, mtime))
return;
if (!pack && type == OBJ_BLOB) {
- struct odb_source *source = the_repository->objects->sources;
+ struct odb_source_files *files =
+ odb_source_files_downcast(the_repository->objects->source);
int found = 0;
- for (; !found && source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- if (!odb_source_read_object_info(&files->dirs->loose->base, oid, NULL, 0, NULL))
+ for (struct odb_files_dir *dir = files->dirs; !found && dir; dir = dir->next)
+ if (!odb_source_read_object_info(&dir->loose->base, oid, NULL, 0, NULL))
found = 1;
- }
/*
* If a traversed tree has a missing blob then we want
@@ -4512,7 +4505,8 @@ static int add_object_in_unpacked_pack(const struct object_id *oid,
static void add_objects_in_unpacked_packs(void)
{
- struct odb_source *source;
+ struct odb_source_files *files =
+ odb_source_files_downcast(to_pack.repo->objects->source);
time_t mtime;
struct odb_for_each_object_options opts = {
.flags = ODB_FOR_EACH_OBJECT_PACK_ORDER |
@@ -4526,13 +4520,11 @@ static void add_objects_in_unpacked_packs(void)
.source_infop = &source_info,
};
- for (source = to_pack.repo->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
-
- if (!source->local)
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ if (!dir->local)
continue;
- if (odb_source_for_each_object(&files->dirs->packed->base, &oi,
+ if (odb_source_for_each_object(&dir->packed->base, &oi,
add_object_in_unpacked_pack, NULL, &opts))
die(_("cannot open pack index"));
}
@@ -4576,7 +4568,7 @@ static int add_loose_object(const struct object_id *oid, const char *path,
*/
static void add_unreachable_loose_objects(struct rev_info *revs)
{
- for_each_loose_file_in_source(the_repository->objects->sources,
+ for_each_loose_file_in_source(the_repository->objects->source,
add_loose_object, NULL, NULL, revs);
}
@@ -4640,11 +4632,9 @@ static int force_object_loose(struct odb_source *source,
size_t len;
int ret;
- for (struct odb_source *s = source->odb->sources; s; s = s->next) {
- struct odb_source_files *files = odb_source_files_downcast(s);
- if (!odb_source_read_object_info(&files->dirs->loose->base, oid, NULL, 0, NULL))
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next)
+ if (!odb_source_read_object_info(&dir->loose->base, oid, NULL, 0, NULL))
return 0;
- }
oi.typep = &type;
oi.sizep = &len;
@@ -4691,7 +4681,7 @@ static void loosen_unused_packed_objects(void)
if (!packlist_find(&to_pack, &oid) &&
!has_sha1_pack_kept_or_nonlocal(&oid) &&
!loosened_object_can_be_discarded(&oid, p->mtime)) {
- if (force_object_loose(the_repository->objects->sources,
+ if (force_object_loose(the_repository->objects->source,
&oid, &p->mtime))
die(_("unable to force loose object"));
loosened_objects_nr++;
diff --git a/builtin/prune.c b/builtin/prune.c
index a7e4678d11..97f4c1418b 100644
--- a/builtin/prune.c
+++ b/builtin/prune.c
@@ -198,7 +198,7 @@ int cmd_prune(int argc,
revs.exclude_promisor_objects = 1;
}
- for_each_loose_file_in_source(repo->objects->sources,
+ for_each_loose_file_in_source(repo->objects->source,
prune_object, prune_cruft, prune_subdir, &revs);
prune_packed_objects(show_only ? PRUNE_PACKED_DRY_RUN : 0);
diff --git a/bundle.c b/bundle.c
index f55a521b2a..a81fd29505 100644
--- a/bundle.c
+++ b/bundle.c
@@ -239,7 +239,7 @@ int verify_bundle(struct repository *r,
.quiet = 1,
};
- if (!r || !r->objects || !r->objects->sources)
+ if (!r || !r->objects || !r->objects->source)
return error(_("need a repository to verify a bundle"));
for (i = 0; i < p->nr; i++) {
diff --git a/commit-graph.c b/commit-graph.c
index 7cc486d140..673d54db29 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -15,6 +15,7 @@
#include "hash-lookup.h"
#include "commit-graph.h"
#include "odb.h"
+#include "odb/source-files.h"
#include "oid-array.h"
#include "path.h"
#include "alloc.h"
@@ -38,7 +39,7 @@ void git_test_write_commit_graph_or_die(struct repository *repo)
if (git_env_bool(GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS, 0))
flags = COMMIT_GRAPH_WRITE_BLOOM_FILTERS;
- if (write_commit_graph_reachable(repo, repo->objects->sources->path, flags, NULL))
+ if (write_commit_graph_reachable(repo, repo->objects->source->path, flags, NULL))
die("failed to write commit-graph under GIT_TEST_COMMIT_GRAPH");
}
@@ -657,7 +658,7 @@ struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,
CALLOC_ARRAY(oids, count);
for (i = 0; i < count; i++) {
- struct odb_source *source;
+ struct odb_source_files *files;
if (strbuf_getline_lf(&line, fp) == EOF)
break;
@@ -670,9 +671,11 @@ struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,
}
valid = 0;
- for (source = odb->sources; source; source = source->next) {
- char *graph_name = get_split_graph_filename(source->path, line.buf);
- struct commit_graph *g = load_commit_graph_one(odb->repo, source->path, graph_name);
+
+ files = odb_source_files_downcast(odb->source);
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ char *graph_name = get_split_graph_filename(dir->abspath, line.buf);
+ struct commit_graph *g = load_commit_graph_one(odb->repo, dir->abspath, graph_name);
free(graph_name);
@@ -742,7 +745,7 @@ struct commit_graph *read_commit_graph_one(struct repository *repo,
*/
static struct commit_graph *prepare_commit_graph(struct repository *r)
{
- struct odb_source *source;
+ struct odb_source_files *files;
/*
* Early return if there is no object database or if the commit graph is
@@ -773,8 +776,9 @@ static struct commit_graph *prepare_commit_graph(struct repository *r)
if (!commit_graph_compatible(r))
return NULL;
- for (source = r->objects->sources; source; source = source->next) {
- r->objects->commit_graph = read_commit_graph_one(r, source->path);
+ files = odb_source_files_downcast(r->objects->source);
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ r->objects->commit_graph = read_commit_graph_one(r, dir->abspath);
if (r->objects->commit_graph)
break;
}
@@ -2003,7 +2007,7 @@ static int fill_oids_from_commits(struct write_commit_graph_context *ctx,
static void fill_oids_from_all_packs(struct write_commit_graph_context *ctx)
{
- struct odb_source *source;
+ struct odb_source_files *files;
enum object_type type;
struct odb_for_each_object_options opts = {
.flags = ODB_FOR_EACH_OBJECT_PACK_ORDER,
@@ -2018,9 +2022,9 @@ static void fill_oids_from_all_packs(struct write_commit_graph_context *ctx)
_("Finding commits for commit graph among packed objects"),
ctx->approx_nr_objects);
- for (source = ctx->r->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- odb_source_for_each_object(&files->dirs->packed->base, &oi, add_packed_commits_oi,
+ files = odb_source_files_downcast(ctx->r->objects->source);
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ odb_source_for_each_object(&dir->packed->base, &oi, add_packed_commits_oi,
ctx, &opts);
}
diff --git a/diagnose.c b/diagnose.c
index 89240e47d6..2c1bcfe431 100644
--- a/diagnose.c
+++ b/diagnose.c
@@ -228,7 +228,7 @@ int create_diagnostics_archive(struct repository *r,
strbuf_reset(&buf);
strbuf_addstr(&buf, "--add-virtual-file=packs-local.txt:");
- dir_file_stats(odb_source_files_downcast(r->objects->sources)->dirs, &buf);
+ dir_file_stats(odb_source_files_downcast(r->objects->source)->dirs, &buf);
odb_for_each_alternate(r->objects, dir_file_stats, &buf);
strvec_push(&archiver_args, buf.buf);
diff --git a/fetch-pack.c b/fetch-pack.c
index aad07b1153..b14e3ba71b 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -1075,7 +1075,7 @@ static int get_pack(struct fetch_pack_args *args,
die(_("fetch-pack: unable to fork off %s"), cmd_name);
if (do_keep && (pack_lockfiles || fsck_objects)) {
int is_well_formed;
- char *pack_lockfile = index_pack_lockfile(the_repository->objects->sources,
+ char *pack_lockfile = index_pack_lockfile(the_repository->objects->source,
cmd.out,
&is_well_formed);
diff --git a/http-walker.c b/http-walker.c
index 0a6c99f471..4f50dd9b8d 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -540,7 +540,7 @@ static int fetch_object(struct walker *walker, const struct object_id *oid)
} else if (!oideq(&obj_req->oid, &req->real_oid)) {
ret = error("File %s has bad hash", hex);
} else if (req->rename < 0) {
- struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->sources);
+ struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->source);
struct strbuf buf = STRBUF_INIT;
odb_loose_path(files->dirs->loose, &buf, &req->oid);
ret = error("unable to write sha1 filename %s", buf.buf);
diff --git a/http.c b/http.c
index fe6ec88a21..230c4de494 100644
--- a/http.c
+++ b/http.c
@@ -2718,7 +2718,7 @@ int finish_http_pack_request(struct http_pack_request *preq)
void http_install_packfile(struct packed_git *p,
struct packfile_list *list_to_remove_from)
{
- struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->sources);
+ struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->source);
packfile_list_remove(list_to_remove_from, p);
packfile_store_add_pack(files->dirs->packed, p);
}
@@ -2846,7 +2846,7 @@ static size_t fwrite_sha1_file(char *ptr, size_t eltsize, size_t nmemb,
struct http_object_request *new_http_object_request(const char *base_url,
const struct object_id *oid)
{
- struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->sources);
+ struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->source);
char *hex = oid_to_hex(oid);
struct strbuf filename = STRBUF_INIT;
struct strbuf prevfile = STRBUF_INIT;
@@ -2987,7 +2987,7 @@ void process_http_object_request(struct http_object_request *freq)
int finish_http_object_request(struct http_object_request *freq)
{
- struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->sources);
+ struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->source);
struct stat st;
struct strbuf filename = STRBUF_INIT;
diff --git a/loose.c b/loose.c
index 957bf83e6b..d6a2bb5ba3 100644
--- a/loose.c
+++ b/loose.c
@@ -113,11 +113,10 @@ int loose_object_map_load(struct odb_source_loose *loose)
int repo_read_loose_object_map(struct repository *repo)
{
- struct odb_source *source;
+ struct odb_source_files *files = odb_source_files_downcast(repo->objects->source);
- for (source = repo->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- if (loose_object_map_load(files->dirs->loose) < 0)
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ if (loose_object_map_load(dir->loose) < 0)
return -1;
}
@@ -126,7 +125,7 @@ int repo_read_loose_object_map(struct repository *repo)
int repo_write_loose_object_map(struct repository *repo)
{
- struct odb_source_files *files = odb_source_files_downcast(repo->objects->sources);
+ struct odb_source_files *files = odb_source_files_downcast(repo->objects->source);
kh_oid_map_t *map = files->dirs->loose->map->to_compat;
struct lock_file lock;
int fd;
@@ -231,13 +230,12 @@ int repo_loose_object_map_oid(struct repository *repo,
const struct git_hash_algo *to,
struct object_id *dest)
{
- struct odb_source *source;
+ struct odb_source_files *files = odb_source_files_downcast(repo->objects->source);
kh_oid_map_t *map;
khiter_t pos;
- for (source = repo->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- struct loose_object_map *loose_map = files->dirs->loose->map;
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ struct loose_object_map *loose_map = dir->loose->map;
if (!loose_map)
continue;
map = (to == repo->compat_hash_algo) ?
diff --git a/midx.c b/midx.c
index c0f82c4163..8638ddf0be 100644
--- a/midx.c
+++ b/midx.c
@@ -829,21 +829,15 @@ void clear_incremental_midx_files_ext(struct odb_source_packed *source, const ch
void clear_midx_file(struct repository *r)
{
- struct odb_source_files *files;
+ struct odb_source_files *files = odb_source_files_downcast(r->objects->source);
struct strbuf midx = STRBUF_INIT;
- if (r->objects) {
- struct odb_source *source;
-
- for (source = r->objects->sources; source; source = source->next) {
- files = odb_source_files_downcast(source);
- if (files->dirs->packed->midx)
- close_midx(files->dirs->packed->midx);
- files->dirs->packed->midx = NULL;
- }
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ if (dir->packed->midx)
+ close_midx(dir->packed->midx);
+ dir->packed->midx = NULL;
}
- files = odb_source_files_downcast(r->objects->sources);
get_midx_filename(files->dirs->packed, &midx);
if (remove_path(midx.buf))
@@ -858,18 +852,15 @@ void clear_midx_file(struct repository *r)
void clear_incremental_midx_files(struct repository *r,
const struct strvec *keep_hashes)
{
- struct odb_source_files *files;
- struct odb_source *source;
+ struct odb_source_files *files = odb_source_files_downcast(r->objects->source);
struct strbuf chain = STRBUF_INIT;
- for (source = r->objects->sources; source; source = source->next) {
- files = odb_source_files_downcast(source);
- if (files->dirs->packed->midx)
- close_midx(files->dirs->packed->midx);
- files->dirs->packed->midx = NULL;
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ if (dir->packed->midx)
+ close_midx(dir->packed->midx);
+ dir->packed->midx = NULL;
}
- files = odb_source_files_downcast(r->objects->sources);
get_midx_chain_filename(files->dirs->packed, &chain);
if (!keep_hashes && remove_path(chain.buf))
diff --git a/odb.c b/odb.c
index 8b54271c27..98c5c862f5 100644
--- a/odb.c
+++ b/odb.c
@@ -13,11 +13,10 @@
#include "object-file.h"
#include "object-name.h"
#include "odb.h"
-#include "odb/source-inmemory.h"
#include "odb/source-files.h"
+#include "odb/source-inmemory.h"
#include "path.h"
#include "promisor-remote.h"
-#include "quote.h"
#include "replace-object.h"
#include "run-command.h"
#include "setup.h"
@@ -28,48 +27,6 @@
#include "trace2.h"
#include "write-or-die.h"
-/*
- * NEEDSWORK: we're using "core.ignoreCase" to deduplicate alternates that
- * _may_ be the same. This requires quite a bit of boilerplate for dubious
- * benefit:
- *
- * - Duplicating alternates should really only lead to regressed performance.
- *
- * - We don't properly resolve symlinks or mointpoints, so we may still end
- * up duplicating alternates.
- *
- * - The value may be lying, in which case we might deduplicate alternates
- * that are in fact not mapping to the same directory.
- *
- * We should investigate whether we can remove this whole mechanism outright.
- */
-static int odb_source_paths_cmp(struct object_database *o,
- const char *a, const char *b)
-{
- if (o->source_paths_icase < 0) {
- int icase = 0;
- repo_config_get_bool(o->repo, "core.ignorecase", &icase);
- o->source_paths_icase = icase;
- }
-
- return o->source_paths_icase ? strcasecmp(a, b) : strcmp(a, b);
-}
-
-static int odb_source_by_path_cmp(const void *cb_data,
- const struct hashmap_entry *entry,
- const struct hashmap_entry *entry_or_key,
- const void *keydata)
-{
- struct object_database *o = (struct object_database *)cb_data;
- const struct odb_source *source = container_of(entry, const struct odb_source, by_path_entry);
- const char *path = keydata;
-
- if (!path)
- path = container_of(entry_or_key, const struct odb_source, by_path_entry)->path;
-
- return odb_source_paths_cmp(o, source->path, path);
-}
-
int odb_mkstemp(struct object_database *odb,
struct strbuf *temp_filename, const char *pattern)
{
@@ -91,154 +48,6 @@ int odb_mkstemp(struct object_database *odb,
return xmkstemp_mode(temp_filename->buf, mode);
}
-/*
- * Return non-zero iff the path is usable as an alternate object database.
- */
-static bool odb_is_source_usable(struct object_database *o, const char *path)
-{
- struct strbuf normalized_objdir = STRBUF_INIT;
- struct hashmap_entry key;
- bool usable = false;
-
- strbuf_realpath(&normalized_objdir, o->sources->path, 1);
-
- /* Detect cases where alternate disappeared */
- if (!is_directory(path)) {
- error(_("object directory %s does not exist; "
- "check .git/objects/info/alternates"),
- path);
- goto out;
- }
-
- /*
- * Prevent the common mistake of listing the same
- * thing twice, or object directory itself.
- */
- if (!hashmap_get_size(&o->source_by_path)) {
- assert(!o->sources->next);
- hashmap_entry_init(&o->sources->by_path_entry,
- strihash(o->sources->path));
- hashmap_add(&o->source_by_path, &o->sources->by_path_entry);
- }
-
- if (!odb_source_paths_cmp(o, path, normalized_objdir.buf))
- goto out;
-
- hashmap_entry_init(&key, strihash(path));
- if (hashmap_get(&o->source_by_path, &key, path))
- goto out;
-
- usable = true;
-
-out:
- strbuf_release(&normalized_objdir);
- return usable;
-}
-
-void parse_alternates(const char *string,
- int sep,
- const char *relative_base,
- struct strvec *out)
-{
- struct strbuf pathbuf = STRBUF_INIT;
- struct strbuf buf = STRBUF_INIT;
-
- if (!string || !*string)
- return;
-
- while (*string) {
- const char *end;
-
- strbuf_reset(&buf);
- strbuf_reset(&pathbuf);
-
- if (*string == '#') {
- /* comment; consume up to next separator */
- end = strchrnul(string, sep);
- } else if (*string == '"' && !unquote_c_style(&buf, string, &end)) {
- /*
- * quoted path; unquote_c_style has copied the
- * data for us and set "end". Broken quoting (e.g.,
- * an entry that doesn't end with a quote) falls
- * back to the unquoted case below.
- */
- } else {
- /* normal, unquoted path */
- end = strchrnul(string, sep);
- strbuf_add(&buf, string, end - string);
- }
-
- if (*end)
- end++;
- string = end;
-
- if (!buf.len)
- continue;
-
- if (!is_absolute_path(buf.buf) && relative_base) {
- strbuf_realpath(&pathbuf, relative_base, 1);
- strbuf_addch(&pathbuf, '/');
- }
- strbuf_addbuf(&pathbuf, &buf);
-
- strbuf_reset(&buf);
- if (!strbuf_realpath(&buf, pathbuf.buf, 0)) {
- error(_("unable to normalize alternate object path: %s"),
- pathbuf.buf);
- continue;
- }
-
- /*
- * The trailing slash after the directory name is given by
- * this function at the end. Remove duplicates.
- */
- while (buf.len && buf.buf[buf.len - 1] == '/')
- strbuf_setlen(&buf, buf.len - 1);
-
- strvec_push(out, buf.buf);
- }
-
- strbuf_release(&pathbuf);
- strbuf_release(&buf);
-}
-
-static struct odb_source *odb_add_alternate_recursively(struct object_database *odb,
- const char *source,
- int depth)
-{
- struct odb_source *alternate = NULL;
- struct strvec sources = STRVEC_INIT;
-
- if (!odb_is_source_usable(odb, source))
- goto error;
-
- alternate = odb_source_new(odb, source, false);
-
- /* add the alternate entry */
- *odb->sources_tail = alternate;
- odb->sources_tail = &(alternate->next);
-
- hashmap_entry_init(&alternate->by_path_entry, strihash(alternate->path));
- if (hashmap_get(&odb->source_by_path, &alternate->by_path_entry,
- alternate->path))
- BUG("source must not yet exist");
- hashmap_add(&odb->source_by_path, &alternate->by_path_entry);
-
- /* recursively add alternates */
- odb_source_read_alternates(alternate, &sources);
- if (sources.nr && depth + 1 > 5) {
- error(_("%s: ignoring alternate object stores, nesting too deep"),
- source);
- } else {
- for (size_t i = 0; i < sources.nr; i++)
- odb_add_alternate_recursively(odb, sources.v[i], depth + 1);
- }
-
- error:
- strvec_clear(&sources);
- return alternate;
-}
-
char *compute_alternate_path(const char *path, struct strbuf *err)
{
char *ref_git = NULL;
@@ -410,34 +219,22 @@ void odb_for_each_alternate_ref(struct object_database *odb,
int odb_for_each_alternate(struct object_database *odb,
odb_for_each_alternate_fn cb, void *payload)
{
- struct odb_source *alternate;
+ struct odb_source_files *files = odb_source_files_downcast(odb->source);
int r = 0;
- for (alternate = odb->sources->next; alternate; alternate = alternate->next) {
- r = cb(odb_source_files_downcast(alternate)->dirs, payload);
+ for (struct odb_files_dir *dir = files->dirs->next; dir; dir = dir->next) {
+ r = cb(dir, payload);
if (r)
break;
}
return r;
}
-static void odb_prepare_alternates(struct object_database *odb,
- const char *alternate_db)
-{
- struct strvec sources = STRVEC_INIT;
-
- parse_alternates(alternate_db, PATH_SEP, NULL, &sources);
- odb_source_read_alternates(odb->sources, &sources);
-
- for (size_t i = 0; i < sources.nr; i++)
- odb_add_alternate_recursively(odb, sources.v[i], 0);
-
- strvec_clear(&sources);
-}
-
int odb_has_alternates(struct object_database *odb)
{
- return !!odb->sources->next;
+ if (odb->source->type != ODB_SOURCE_FILES)
+ return 0;
+ return !!odb_source_files_downcast(odb->source)->dirs->next;
}
int obj_read_use_lock = 0;
@@ -481,16 +278,12 @@ static enum odb_read_status do_oid_object_info_extended(struct object_database *
return 0;
while (1) {
- struct odb_source *source;
-
- for (source = odb->sources; source; source = source->next) {
- ret = odb_source_read_object_info(source, real, oi, flags,
- corrupt_err.len ? NULL : &corrupt_err);
- if (!ret)
- goto out;
- if (ret != ODB_READ_NOT_FOUND)
- corrupt = true;
- }
+ ret = odb_source_read_object_info(odb->source, real, oi, flags,
+ corrupt_err.len ? NULL : &corrupt_err);
+ if (!ret)
+ goto out;
+ if (ret != ODB_READ_NOT_FOUND)
+ corrupt = true;
/*
* When the object hasn't been found we try a second read and
@@ -498,15 +291,13 @@ static enum odb_read_status do_oid_object_info_extended(struct object_database *
* caches or reload on-disk state.
*/
if (!(flags & OBJECT_INFO_QUICK)) {
- for (source = odb->sources; source; source = source->next) {
- ret = odb_source_read_object_info(source, real, oi,
- flags | OBJECT_INFO_SECOND_READ,
- corrupt_err.len ? NULL : &corrupt_err);
- if (!ret)
- goto out;
- if (ret != ODB_READ_NOT_FOUND)
- corrupt = true;
- }
+ ret = odb_source_read_object_info(odb->source, real, oi,
+ flags | OBJECT_INFO_SECOND_READ,
+ corrupt_err.len ? NULL : &corrupt_err);
+ if (!ret)
+ goto out;
+ if (ret != ODB_READ_NOT_FOUND)
+ corrupt = true;
}
/* Check if it is a missing object */
@@ -748,11 +539,7 @@ int odb_has_object(struct object_database *odb, const struct object_id *oid,
int odb_freshen_object(struct object_database *odb,
const struct object_id *oid)
{
- struct odb_source *source;
- for (source = odb->sources; source; source = source->next)
- if (odb_source_freshen_object(source, oid, NULL))
- return 1;
- return 0;
+ return odb_source_freshen_object(odb->source, oid, NULL);
}
int odb_for_each_object_ext(struct object_database *odb,
@@ -761,18 +548,7 @@ int odb_for_each_object_ext(struct object_database *odb,
void *cb_data,
const struct odb_for_each_object_options *opts)
{
- int ret;
-
- for (struct odb_source *source = odb->sources; source; source = source->next) {
- if (opts->flags & ODB_FOR_EACH_OBJECT_LOCAL_ONLY && !source->local)
- continue;
-
- ret = odb_source_for_each_object(source, request, cb, cb_data, opts);
- if (ret)
- return ret;
- }
-
- return 0;
+ return odb_source_for_each_object(odb->source, request, cb, cb_data, opts);
}
int odb_for_each_object(struct object_database *odb,
@@ -791,7 +567,6 @@ int odb_count_objects(struct object_database *odb,
enum odb_count_objects_flags flags,
unsigned long *out)
{
- struct odb_source *source;
unsigned long count = 0;
int ret;
@@ -800,15 +575,9 @@ int odb_count_objects(struct object_database *odb,
return 0;
}
- for (source = odb->sources; source; source = source->next) {
- unsigned long c;
-
- ret = odb_source_count_objects(source, flags, &c);
- if (ret < 0)
- goto out;
-
- count += c;
- }
+ ret = odb_source_count_objects(odb->source, flags, &count);
+ if (ret < 0)
+ goto out;
odb->object_count = count;
odb->object_count_valid = 1;
@@ -879,13 +648,7 @@ int odb_find_abbrev_len(struct object_database *odb,
goto out;
}
- for (struct odb_source *source = odb->sources; source; source = source->next) {
- ret = odb_source_find_abbrev_len(source, oid, len, &len);
- if (ret)
- goto out;
- }
-
- ret = 0;
+ ret = odb_source_find_abbrev_len(odb->source, oid, len, &len);
*out = len;
out:
@@ -941,7 +704,7 @@ int odb_write_object_ext(struct object_database *odb,
compat_oid_p = &compat_oid;
}
- return odb_source_write_object(odb->sources, buf, len, type,
+ return odb_source_write_object(odb->source, buf, len, type,
oid, compat_oid_p, NULL, flags);
}
@@ -949,19 +712,19 @@ int odb_write_object_stream(struct object_database *odb,
struct odb_stream *stream,
struct object_id *oid)
{
- return odb_source_write_object_stream(odb->sources, stream, oid);
+ return odb_source_write_object_stream(odb->source, stream, oid);
}
int odb_optimize(struct object_database *odb,
const struct odb_optimize_options *opts)
{
- return odb_source_optimize(odb->sources, opts);
+ return odb_source_optimize(odb->source, opts);
}
bool odb_optimize_required(struct object_database *odb,
const struct odb_optimize_options *opts)
{
- return odb_source_optimize_required(odb->sources, opts);
+ return odb_source_optimize_required(odb->source, opts);
}
void odb_generate_pack_options_release(struct odb_generate_pack_options *opts)
@@ -975,9 +738,9 @@ int odb_generate_pack(struct object_database *odb,
struct odb_pack_generator **out,
const struct odb_generate_pack_options *opts)
{
- if (!odb->sources->generate_pack)
+ if (!odb->source->generate_pack)
return error(_("primary object source does not support generating packfiles"));
- return odb_source_generate_pack(odb->sources, out, opts);
+ return odb_source_generate_pack(odb->source, out, opts);
}
int odb_pack_generator_finish(struct odb_pack_generator *generator)
@@ -988,55 +751,29 @@ int odb_pack_generator_finish(struct odb_pack_generator *generator)
struct object_database *odb_new(struct repository *repo,
enum odb_new_flags flags)
{
- char *primary_source = NULL, *secondary_sources = NULL;
struct object_database *o;
CALLOC_ARRAY(o, 1);
o->repo = repo;
pthread_mutex_init(&o->replace_mutex, NULL);
- hashmap_init(&o->source_by_path, odb_source_by_path_cmp, o, 0);
- o->source_paths_icase = -1;
-
- if (flags & ODB_NEW_HONOR_ENV) {
- primary_source = xstrdup_or_null(getenv(DB_ENVIRONMENT));
- secondary_sources = xstrdup_or_null(getenv(ALTERNATE_DB_ENVIRONMENT));
- }
- if (!primary_source)
- primary_source = xstrfmt("%s/objects", repo->commondir);
- o->sources = odb_source_new(o, primary_source, true);
- o->sources_tail = &o->sources->next;
+ o->source = odb_source_new(o, flags);
o->inmemory_objects = &odb_source_inmemory_new(o)->base;
- odb_prepare_alternates(o, secondary_sources);
-
- free(secondary_sources);
- free(primary_source);
return o;
}
void odb_close(struct object_database *o)
{
- struct odb_source *source;
- for (source = o->sources; source; source = source->next)
- odb_source_close(source);
+ odb_source_close(o->source);
close_commit_graph(o);
}
static void odb_free_sources(struct object_database *o)
{
- while (o->sources) {
- struct odb_source *next;
-
- next = o->sources->next;
- odb_source_free(o->sources);
- o->sources = next;
- }
-
+ odb_source_free(o->source);
odb_source_free(o->inmemory_objects);
o->inmemory_objects = NULL;
-
- hashmap_clear(&o->source_by_path);
}
void odb_free(struct object_database *o)
@@ -1055,24 +792,12 @@ void odb_free(struct object_database *o)
void odb_prepare(struct object_database *o, enum odb_prepare_flags flags)
{
- struct odb_source *source;
-
obj_read_lock();
- /*
- * Reprepare alt odbs, in case the alternates file was modified
- * during the course of this process. This only _adds_ odbs to
- * the linked list, so existing odbs will continue to exist for
- * the lifetime of the process. Consequently, we don't have to
- * reprocess GIT_ALTERNATE_OBJECT_DIRECTORIES here.
- */
- if (flags & ODB_PREPARE_FLUSH_CACHES) {
- odb_prepare_alternates(o, NULL);
+ if (flags & ODB_PREPARE_FLUSH_CACHES)
o->object_count_valid = 0;
- }
- for (source = o->sources; source; source = source->next)
- odb_source_prepare(source, flags);
+ odb_source_prepare(o->source, flags);
obj_read_unlock();
}
@@ -1084,8 +809,5 @@ void odb_reprepare(struct object_database *o)
int odb_fsck(struct object_database *odb, struct odb_fsck_options *options)
{
- int ret = 0;
- for (struct odb_source *source = odb->sources; source; source = source->next)
- ret |= odb_source_fsck(source, options);
- return ret;
+ return odb_source_fsck(odb->source, options);
}
diff --git a/odb.h b/odb.h
index 4143812f55..0ccc47ee73 100644
--- a/odb.h
+++ b/odb.h
@@ -28,13 +28,16 @@ char *compute_alternate_path(const char *path, struct strbuf *err);
/*
* The object database encapsulates access to objects in a repository. It
- * manages one or more sources that store the actual objects which are
- * configured via alternates.
+ * manages the object source as well as auxiliary data structures required to
+ * manage objects.
*/
struct object_database {
/* Repository that owns this database. */
struct repository *repo;
+ /* The source backing this object database. */
+ struct odb_source *source;
+
/*
* State of current object database transaction. Only one
* transaction may be pending at a time. Is NULL when no transaction is
@@ -42,27 +45,6 @@ struct object_database {
*/
struct odb_transaction *transaction;
- /*
- * Set of all object directories; the main directory is first (and
- * cannot be NULL after initialization). Subsequent directories are
- * alternates.
- */
- struct odb_source *sources;
- struct odb_source **sources_tail;
-
- /*
- * Map of object database sources, keyed by their respective paths.
- * This map is used to detect the case where the same source is
- * registered multiple times.
- */
- struct hashmap source_by_path;
-
- /*
- * Whether source paths shall be compared case-insensitively, as
- * determined by "core.ignoreCase".
- */
- int source_paths_icase;
-
/*
* Objects that should be substituted by other objects
* (see git-replace(1)).
@@ -822,9 +804,4 @@ int odb_generate_pack(struct object_database *odb,
*/
int odb_pack_generator_finish(struct odb_pack_generator *generator);
-void parse_alternates(const char *string,
- int sep,
- const char *relative_base,
- struct strvec *out);
-
#endif /* ODB_H */
diff --git a/odb/source-files.c b/odb/source-files.c
index 6aaf625352..072f515b36 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -15,6 +15,7 @@
#include "packfile.h"
#include "path.h"
#include "promisor-remote.h"
+#include "quote.h"
#include "repack.h"
#include "run-command.h"
#include "strbuf.h"
@@ -71,6 +72,7 @@ static void odb_source_files_free(struct odb_source *source)
odb_files_dir_free(files->dirs);
files->dirs = next;
}
+ hashmap_clear(&files->dirs_by_path);
odb_source_release(&files->base);
free(files);
@@ -174,6 +176,160 @@ static int odb_source_files_create_on_disk(struct odb_source *source,
return ret;
}
+/*
+ * NEEDSWORK: we're using "core.ignoreCase" to deduplicate alternates that
+ * _may_ be the same. This requires quite a bit of boilerplate for dubious
+ * benefit:
+ *
+ * - Duplicating alternates should really only lead to regressed performance.
+ *
+ * - We don't properly resolve symlinks or mointpoints, so we may still end
+ * up duplicating alternates.
+ *
+ * - The value may be lying, in which case we might deduplicate alternates
+ * that are in fact not mapping to the same directory.
+ *
+ * We should investigate whether we can remove this whole mechanism outright.
+ */
+static int odb_files_dir_paths_cmp(struct odb_source_files *files,
+ const char *a, const char *b)
+{
+ if (files->dirs_paths_icase < 0) {
+ int icase = 0;
+ repo_config_get_bool(files->base.odb->repo, "core.ignorecase", &icase);
+ files->dirs_paths_icase = icase;
+ }
+
+ return files->dirs_paths_icase ? strcasecmp(a, b) : strcmp(a, b);
+}
+
+static int odb_files_dir_by_path_cmp(const void *cb_data,
+ const struct hashmap_entry *entry,
+ const struct hashmap_entry *entry_or_key,
+ const void *keydata)
+{
+ struct odb_source_files *files = (struct odb_source_files *)cb_data;
+ const struct odb_files_dir *dir = container_of(entry, const struct odb_files_dir, by_path_entry);
+ const char *path = keydata;
+
+ if (!path)
+ path = container_of(entry_or_key, const struct odb_files_dir, by_path_entry)->abspath;
+
+ return odb_files_dir_paths_cmp(files, dir->abspath, path);
+}
+
+/*
+ * Return non-zero iff the path is usable as an alternate object directory.
+ */
+static bool odb_files_dir_is_usable(struct odb_source_files *files,
+ const char *path)
+{
+ struct strbuf normalized_objdir = STRBUF_INIT;
+ struct hashmap_entry key;
+ bool usable = false;
+
+ strbuf_realpath(&normalized_objdir, files->dirs->abspath, 1);
+
+ /* Detect cases where alternate disappeared */
+ if (!is_directory(path)) {
+ error(_("object directory %s does not exist; "
+ "check .git/objects/info/alternates"),
+ path);
+ goto out;
+ }
+
+ /*
+ * Prevent the common mistake of listing the same
+ * thing twice, or object directory itself.
+ */
+ if (!hashmap_get_size(&files->dirs_by_path)) {
+ assert(!files->dirs->next);
+ hashmap_entry_init(&files->dirs->by_path_entry,
+ strihash(files->dirs->abspath));
+ hashmap_add(&files->dirs_by_path, &files->dirs->by_path_entry);
+ }
+
+ if (!odb_files_dir_paths_cmp(files, path, normalized_objdir.buf))
+ goto out;
+
+ hashmap_entry_init(&key, strihash(path));
+ if (hashmap_get(&files->dirs_by_path, &key, path))
+ goto out;
+
+ usable = true;
+
+out:
+ strbuf_release(&normalized_objdir);
+ return usable;
+}
+
+static void parse_alternates(const char *string,
+ int sep,
+ const char *relative_base,
+ struct strvec *out)
+{
+ struct strbuf pathbuf = STRBUF_INIT;
+ struct strbuf buf = STRBUF_INIT;
+
+ if (!string || !*string)
+ return;
+
+ while (*string) {
+ const char *end;
+
+ strbuf_reset(&buf);
+ strbuf_reset(&pathbuf);
+
+ if (*string == '#') {
+ /* comment; consume up to next separator */
+ end = strchrnul(string, sep);
+ } else if (*string == '"' && !unquote_c_style(&buf, string, &end)) {
+ /*
+ * quoted path; unquote_c_style has copied the
+ * data for us and set "end". Broken quoting (e.g.,
+ * an entry that doesn't end with a quote) falls
+ * back to the unquoted case below.
+ */
+ } else {
+ /* normal, unquoted path */
+ end = strchrnul(string, sep);
+ strbuf_add(&buf, string, end - string);
+ }
+
+ if (*end)
+ end++;
+ string = end;
+
+ if (!buf.len)
+ continue;
+
+ if (!is_absolute_path(buf.buf) && relative_base) {
+ strbuf_realpath(&pathbuf, relative_base, 1);
+ strbuf_addch(&pathbuf, '/');
+ }
+ strbuf_addbuf(&pathbuf, &buf);
+
+ strbuf_reset(&buf);
+ if (!strbuf_realpath(&buf, pathbuf.buf, 0)) {
+ error(_("unable to normalize alternate object path: %s"),
+ pathbuf.buf);
+ continue;
+ }
+
+ /*
+ * The trailing slash after the directory name is given by
+ * this function at the end. Remove duplicates.
+ */
+ while (buf.len && buf.buf[buf.len - 1] == '/')
+ strbuf_setlen(&buf, buf.len - 1);
+
+ strvec_push(out, buf.buf);
+ }
+
+ strbuf_release(&pathbuf);
+ strbuf_release(&buf);
+}
+
static int read_alternates(const char *object_dir, struct strvec *out)
{
struct strbuf buf = STRBUF_INIT;
@@ -192,11 +348,70 @@ static int read_alternates(const char *object_dir, struct strvec *out)
return 0;
}
+static void odb_add_alternate_recursively(struct odb_source_files *files,
+ const char *path,
+ int depth)
+{
+ struct odb_files_dir *alternate;
+ struct strvec alternates = STRVEC_INIT;
+
+ if (!odb_files_dir_is_usable(files, path))
+ goto out;
+
+ alternate = odb_files_dir_new(files->base.odb, path, false);
+
+ /* add the alternate entry */
+ *files->dirs_tail = alternate;
+ files->dirs_tail = &(alternate->next);
+
+ hashmap_entry_init(&alternate->by_path_entry, strihash(alternate->abspath));
+ if (hashmap_get(&files->dirs_by_path, &alternate->by_path_entry,
+ alternate->abspath))
+ BUG("object directory must not yet exist");
+ hashmap_add(&files->dirs_by_path, &alternate->by_path_entry);
+
+ /* recursively add alternates */
+ read_alternates(alternate->abspath, &alternates);
+ if (alternates.nr && depth + 1 > 5) {
+ error(_("%s: ignoring alternate object stores, nesting too deep"),
+ path);
+ } else {
+ for (size_t i = 0; i < alternates.nr; i++)
+ odb_add_alternate_recursively(files, alternates.v[i], depth + 1);
+ }
+
+ out:
+ strvec_clear(&alternates);
+}
+
+static void odb_prepare_alternates(struct odb_source_files *files,
+ const char *alternate_db)
+{
+ struct strvec alternates = STRVEC_INIT;
+
+ parse_alternates(alternate_db, PATH_SEP, NULL, &alternates);
+ read_alternates(files->dirs->abspath, &alternates);
+
+ for (size_t i = 0; i < alternates.nr; i++)
+ odb_add_alternate_recursively(files, alternates.v[i], 0);
+
+ strvec_clear(&alternates);
+}
+
static void odb_source_files_prepare(struct odb_source *source,
enum odb_prepare_flags flags)
{
struct odb_source_files *files = odb_source_files_downcast(source);
+ /*
+ * Reprepare alternates, in case the alternates file was modified
+ * during the course of this process. This only _adds_ directories to
+ * the linked list, so existing directories will continue to exist
+ * for the lifetime of the process.
+ */
+ if (flags & ODB_PREPARE_FLUSH_CACHES)
+ odb_prepare_alternates(files, NULL);
+
for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
odb_source_prepare(&dir->loose->base, flags);
odb_source_prepare(&dir->packed->base, flags);
@@ -1009,23 +1224,15 @@ static int odb_source_files_fsck(struct odb_source *source,
struct odb_files_dir *odb_source_files_find_dir(struct object_database *odb, const char *obj_dir)
{
+ struct odb_source_files *files = odb_source_files_downcast(odb->source);
char *obj_dir_real = real_pathdup(obj_dir, 1);
struct strbuf odb_path_real = STRBUF_INIT;
- struct odb_files_dir *dir = NULL;
- struct odb_source *source;
-
- for (source = odb->sources; source; source = source->next) {
- struct odb_source_files *files;
-
- if (source->type != ODB_SOURCE_FILES)
- continue;
- files = odb_source_files_downcast(source);
+ struct odb_files_dir *dir;
- strbuf_realpath(&odb_path_real, files->dirs->abspath, 1);
- if (!strcmp(obj_dir_real, odb_path_real.buf)) {
- dir = files->dirs;
+ for (dir = files->dirs; dir; dir = dir->next) {
+ strbuf_realpath(&odb_path_real, dir->abspath, 1);
+ if (!strcmp(obj_dir_real, odb_path_real.buf))
break;
- }
}
free(obj_dir_real);
@@ -1034,14 +1241,27 @@ struct odb_files_dir *odb_source_files_find_dir(struct object_database *odb, con
}
struct odb_source_files *odb_source_files_new(struct object_database *odb,
- const char *path,
- bool local)
+ enum odb_new_flags flags)
{
struct odb_source_files *files;
+ char *object_dir = NULL;
+ char *alternates = NULL;
+
+ if (flags & ODB_NEW_HONOR_ENV) {
+ object_dir = xstrdup_or_null(getenv(DB_ENVIRONMENT));
+ alternates = xstrdup_or_null(getenv(ALTERNATE_DB_ENVIRONMENT));
+ }
+ if (!object_dir)
+ object_dir = xstrfmt("%s/objects", odb->repo->commondir);
CALLOC_ARRAY(files, 1);
- odb_source_init(&files->base, odb, ODB_SOURCE_FILES, path, local);
- files->dirs = odb_files_dir_new(odb, path, local);
+ odb_source_init(&files->base, odb, ODB_SOURCE_FILES, object_dir, true);
+
+ hashmap_init(&files->dirs_by_path, odb_files_dir_by_path_cmp, files, 0);
+ files->dirs_paths_icase = -1;
+
+ files->dirs = odb_files_dir_new(odb, object_dir, true);
+ files->dirs_tail = &files->dirs->next;
files->base.free = odb_source_files_free;
files->base.close = odb_source_files_close;
@@ -1067,8 +1287,12 @@ struct odb_source_files *odb_source_files_new(struct object_database *odb,
* is not (yet) possible though because we access and assume relative
* paths in the primary ODB source in some user-facing functionality.
*/
- if (!is_absolute_path(path))
+ if (!is_absolute_path(object_dir))
chdir_notify_register(odb_source_files_reparent, files);
+ odb_prepare_alternates(files, alternates);
+
+ free(object_dir);
+ free(alternates);
return files;
}
diff --git a/odb/source-files.h b/odb/source-files.h
index 36af0c1b8b..837e890304 100644
--- a/odb/source-files.h
+++ b/odb/source-files.h
@@ -18,6 +18,12 @@ struct odb_files_dir {
/* List of alternate object directories. */
struct odb_files_dir *next;
+ /*
+ * Entry in the files source's map of directories, keyed by this
+ * directory's path.
+ */
+ struct hashmap_entry by_path_entry;
+
/* The two sources derived from this object directory. */
struct odb_source_loose *loose;
struct odb_source_packed *packed;
@@ -46,12 +52,25 @@ struct odb_source_files {
* alternates.
*/
struct odb_files_dir *dirs;
+ struct odb_files_dir **dirs_tail;
+
+ /*
+ * Map of object directories, keyed by their respective paths. This
+ * map is used to detect the case where the same directory is
+ * registered multiple times.
+ */
+ struct hashmap dirs_by_path;
+
+ /*
+ * Whether directory paths shall be compared case-insensitively, as
+ * determined by "core.ignoreCase".
+ */
+ int dirs_paths_icase;
};
/* Allocate and initialize a new object source. */
struct odb_source_files *odb_source_files_new(struct object_database *odb,
- const char *path,
- bool local);
+ enum odb_new_flags flags);
/*
* Optimize the files object database source by repacking loose objects and
diff --git a/odb/source.c b/odb/source.c
index 30188b806d..b25ef14df8 100644
--- a/odb/source.c
+++ b/odb/source.c
@@ -24,10 +24,9 @@ const char *odb_source_type_to_name(enum odb_source_type type)
}
struct odb_source *odb_source_new(struct object_database *odb,
- const char *path,
- bool local)
+ enum odb_new_flags flags)
{
- return &odb_source_files_new(odb, path, local)->base;
+ return &odb_source_files_new(odb, flags)->base;
}
void odb_source_init(struct odb_source *source,
diff --git a/odb/source.h b/odb/source.h
index 9fd2b2e5b5..6718aced6a 100644
--- a/odb/source.h
+++ b/odb/source.h
@@ -49,24 +49,9 @@ struct odb_create_on_disk_options {
/*
* The source is the part of the object database that stores the actual
* objects. It thus encapsulates the logic to read and write the specific
- * on-disk format. An object database can have multiple sources:
- *
- * - The primary source, which is typically located in "$GIT_DIR/objects".
- * This is where new objects are usually written to.
- *
- * - Alternate sources, which are configured via "objects/info/alternates" or
- * via the GIT_ALTERNATE_OBJECT_DIRECTORIES environment variable. These
- * alternate sources are only used to read objects.
+ * on-disk format.
*/
struct odb_source {
- struct odb_source *next;
-
- /*
- * Entry in the object database's map of sources, keyed by this
- * source's path.
- */
- struct hashmap_entry by_path_entry;
-
/* Object database that owns this object source. */
struct object_database *odb;
@@ -331,13 +316,11 @@ struct odb_source {
};
/*
- * Allocate and initialize a new source for the given object database located
- * at `path`. `local` indicates whether or not the source is the local and thus
- * primary object source of the object database.
+ * Allocate and initialize a new source for the given object database. The path
+ * of the source is derived from repository paths.
*/
struct odb_source *odb_source_new(struct object_database *odb,
- const char *path,
- bool local);
+ enum odb_new_flags flags);
/*
* Initialize the source for the given object database located at `path`.
diff --git a/odb/streaming.c b/odb/streaming.c
index 8f2143cab5..5ac172beb6 100644
--- a/odb/streaming.c
+++ b/odb/streaming.c
@@ -182,12 +182,8 @@ static int istream_source(struct odb_stream **out,
struct object_database *odb,
const struct object_id *oid)
{
- struct odb_source *source;
-
- for (source = odb->sources; source; source = source->next)
- if (!odb_source_read_object_stream(out, source, oid))
- return 0;
-
+ if (!odb_source_read_object_stream(out, odb->source, oid))
+ return 0;
return open_istream_incore(out, odb, oid);
}
diff --git a/odb/transaction.c b/odb/transaction.c
index f6f20088ec..69824a3921 100644
--- a/odb/transaction.c
+++ b/odb/transaction.c
@@ -12,7 +12,7 @@ int odb_transaction_begin(struct object_database *odb,
if (odb->transaction)
return error(_("object database transaction already pending"));
- ret = odb_source_begin_transaction(odb->sources, out, flags);
+ ret = odb_source_begin_transaction(odb->source, out, flags);
if (!ret)
odb->transaction = *out;
diff --git a/pack-bitmap.c b/pack-bitmap.c
index 52556b4543..d3c7997fb3 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -712,15 +712,13 @@ static int open_bitmap_for_source(struct odb_source_packed *source,
static int open_bitmap(struct repository *r,
struct bitmap_index *bitmap_git)
{
- struct odb_source *source;
+ struct odb_source_files *files = odb_source_files_downcast(r->objects->source);
bool found = false;
assert(!bitmap_git->map);
- for (source = r->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
-
- if (!open_bitmap_for_source(files->dirs->packed, bitmap_git))
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ if (!open_bitmap_for_source(dir->packed, bitmap_git))
found = true;
/*
diff --git a/packfile.c b/packfile.c
index 93b69d7f50..ebe430662e 100644
--- a/packfile.c
+++ b/packfile.c
@@ -273,14 +273,13 @@ static void scan_windows(struct packed_git *p,
static int unuse_one_window(struct object_database *odb)
{
- struct odb_source *source;
+ struct odb_source_files *files = odb_source_files_downcast(odb->source);
struct packfile_list_entry *e;
struct packed_git *lru_p = NULL;
struct pack_window *lru_w = NULL, *lru_l = NULL;
- for (source = odb->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- for (e = files->dirs->packed->packs.head; e; e = e->next)
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ for (e = dir->packed->packs.head; e; e = e->next)
scan_windows(e->pack, &lru_p, &lru_w, &lru_l);
}
@@ -450,15 +449,14 @@ static void find_lru_pack(struct packed_git *p, struct packed_git **lru_p, struc
static int close_one_pack(struct repository *r)
{
- struct odb_source *source;
+ struct odb_source_files *files = odb_source_files_downcast(r->objects->source);
struct packfile_list_entry *e;
struct packed_git *lru_p = NULL;
struct pack_window *mru_w = NULL;
int accept_windows_inuse = 1;
- for (source = r->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- for (e = files->dirs->packed->packs.head; e; e = e->next) {
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ for (e = dir->packed->packs.head; e; e = e->next) {
if (e->pack->pack_fd == -1)
continue;
find_lru_pack(e->pack, &lru_p, &mru_w, &accept_windows_inuse);
@@ -1921,11 +1919,10 @@ struct packed_git **packfile_store_get_kept_pack_cache(struct odb_source_packed
int has_object_pack(struct repository *r, const struct object_id *oid)
{
- struct odb_source *source;
+ struct odb_source_files *files = odb_source_files_downcast(r->objects->source);
- for (source = r->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- if (!odb_source_read_object_info(&files->dirs->packed->base, oid, NULL, 0, NULL))
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ if (!odb_source_read_object_info(&dir->packed->base, oid, NULL, 0, NULL))
return 1;
}
@@ -1935,14 +1932,13 @@ int has_object_pack(struct repository *r, const struct object_id *oid)
int has_object_kept_pack(struct repository *r, const struct object_id *oid,
unsigned flags)
{
- struct odb_source *source;
+ struct odb_source_files *files = odb_source_files_downcast(r->objects->source);
struct pack_entry e;
- for (source = r->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
struct packed_git **cache;
- cache = packfile_store_get_kept_pack_cache(files->dirs->packed, flags);
+ cache = packfile_store_get_kept_pack_cache(dir->packed, flags);
for (; *cache; cache++) {
struct packed_git *p = *cache;
diff --git a/packfile.h b/packfile.h
index fd1cf0ad6c..ed3378c457 100644
--- a/packfile.h
+++ b/packfile.h
@@ -69,20 +69,20 @@ void packfile_store_add_pack(struct odb_source_packed *store,
struct packfile_list_entry *packfile_store_get_packs(struct odb_source_packed *store);
struct repo_for_each_pack_data {
- struct odb_source *source;
+ struct odb_files_dir *dir;
struct packfile_list_entry *entry;
};
static inline struct repo_for_each_pack_data repo_for_eack_pack_data_init(struct repository *repo)
{
struct repo_for_each_pack_data data = { 0 };
+ struct odb_source_files *files = odb_source_files_downcast(repo->objects->source);
- for (struct odb_source *source = repo->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- struct packfile_list_entry *entry = packfile_store_get_packs(files->dirs->packed);
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ struct packfile_list_entry *entry = packfile_store_get_packs(dir->packed);
if (!entry)
continue;
- data.source = source;
+ data.dir = dir;
data.entry = entry;
break;
}
@@ -92,23 +92,22 @@ static inline struct repo_for_each_pack_data repo_for_eack_pack_data_init(struct
static inline void repo_for_each_pack_data_next(struct repo_for_each_pack_data *data)
{
- struct odb_source *source;
+ struct odb_files_dir *dir;
data->entry = data->entry->next;
if (data->entry)
return;
- for (source = data->source->next; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- struct packfile_list_entry *entry = packfile_store_get_packs(files->dirs->packed);
+ for (dir = data->dir->next; dir; dir = dir->next) {
+ struct packfile_list_entry *entry = packfile_store_get_packs(dir->packed);
if (!entry)
continue;
- data->source = source;
+ data->dir = dir;
data->entry = entry;
return;
}
- data->source = NULL;
+ data->dir = NULL;
data->entry = NULL;
}
diff --git a/path.c b/path.c
index c3a709a928..4608a56927 100644
--- a/path.c
+++ b/path.c
@@ -396,7 +396,7 @@ static void adjust_git_path(struct repository *repo,
strbuf_splice(buf, 0, buf->len,
repo->index_file, strlen(repo->index_file));
else if (dir_prefix(base, "objects"))
- replace_dir(buf, git_dir_len + 7, repo->objects->sources->path);
+ replace_dir(buf, git_dir_len + 7, repo->objects->source->path);
else if (repo_settings_get_hooks_path(repo) && dir_prefix(base, "hooks"))
replace_dir(buf, git_dir_len + 5, repo_settings_get_hooks_path(repo));
else if (repo->different_commondir)
diff --git a/prune-packed.c b/prune-packed.c
index d49dc11957..b6749b8d13 100644
--- a/prune-packed.c
+++ b/prune-packed.c
@@ -40,7 +40,7 @@ void prune_packed_objects(int opts)
progress = start_delayed_progress(the_repository,
_("Removing duplicate objects"), 256);
- for_each_loose_file_in_source(the_repository->objects->sources,
+ for_each_loose_file_in_source(the_repository->objects->source,
prune_object, NULL, prune_subdir, &opts);
/* Ensure we show 100% before finishing progress */
diff --git a/repack.c b/repack.c
index e20431690c..53c34197f3 100644
--- a/repack.c
+++ b/repack.c
@@ -59,7 +59,7 @@ void repack_remove_redundant_pack(struct repository *repo, const char *dir_name,
bool wrote_incremental_midx)
{
struct strbuf buf = STRBUF_INIT;
- struct odb_source_files *files = odb_source_files_downcast(repo->objects->sources);
+ struct odb_source_files *files = odb_source_files_downcast(repo->objects->source);
struct multi_pack_index *m = get_multi_pack_index(files->dirs->packed);
strbuf_addf(&buf, "%s.pack", base_name);
if (m && files->base.local && midx_contains_pack(m, buf.buf)) {
@@ -158,7 +158,7 @@ void existing_packs_collect(struct existing_packs *existing,
string_list_append(&existing->non_kept_packs, buf.buf);
}
- existing->source = existing->repo->objects->sources;
+ existing->source = existing->repo->objects->source;
string_list_sort(&existing->kept_packs);
string_list_sort(&existing->non_kept_packs);
diff --git a/repository.c b/repository.c
index b857e1c580..fa4214ff41 100644
--- a/repository.c
+++ b/repository.c
@@ -126,9 +126,9 @@ const char *repo_get_common_dir(struct repository *repo)
const char *repo_get_object_directory(struct repository *repo)
{
- if (!repo->objects->sources)
+ if (!repo->objects->source)
BUG("repository hasn't been set up");
- return repo->objects->sources->path;
+ return repo->objects->source->path;
}
const char *repo_get_index_file(struct repository *repo)
diff --git a/setup.c b/setup.c
index 29474fc292..bc7d451917 100644
--- a/setup.c
+++ b/setup.c
@@ -2669,7 +2669,7 @@ void create_object_database(struct repository *repo,
repo->objects = odb_new(repo, ODB_NEW_HONOR_ENV);
- if (odb_source_create_on_disk(repo->objects->sources, &opts) < 0)
+ if (odb_source_create_on_disk(repo->objects->source, &opts) < 0)
die(_("failed creating object database"));
}
diff --git a/t/helper/test-read-graph.c b/t/helper/test-read-graph.c
index a75c817e47..62e2384052 100644
--- a/t/helper/test-read-graph.c
+++ b/t/helper/test-read-graph.c
@@ -78,7 +78,7 @@ int cmd__read_graph(int argc, const char **argv)
int ret = 0;
setup_git_directory(the_repository);
- source = the_repository->objects->sources;
+ source = the_repository->objects->source;
prepare_repo_settings(the_repository);
diff --git a/tmp-objdir.c b/tmp-objdir.c
index 2f2ffbbc7d..3debb60270 100644
--- a/tmp-objdir.c
+++ b/tmp-objdir.c
@@ -60,7 +60,7 @@ static void tmp_objdir_reparent(const char *old_cwd,
*/
static void tmp_objdir_restore_source(struct tmp_objdir *t)
{
- struct odb_source_files *files = odb_source_files_downcast(t->repo->objects->sources);
+ struct odb_source_files *files = odb_source_files_downcast(t->repo->objects->source);
struct odb_files_dir *cur_dir = files->dirs;
if (t->temp_dir != files->dirs)
@@ -159,7 +159,7 @@ struct tmp_objdir *tmp_objdir_create(struct repository *r,
const char *prefix,
int will_destroy)
{
- struct odb_source_files *files = odb_source_files_downcast(r->objects->sources);
+ struct odb_source_files *files = odb_source_files_downcast(r->objects->source);
static int installed_handlers;
struct tmp_objdir *t;
--
2.56.0.379.gc618271300.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* Re: [PATCH 12/13] odb/source-files: move alternates into the backend
2026-10-02 10:08 ` [PATCH 12/13] odb/source-files: move alternates into the backend Patrick Steinhardt
@ 2026-10-06 20:51 ` Karthik Nayak
2026-10-07 5:50 ` Patrick Steinhardt
0 siblings, 1 reply; 41+ messages in thread
From: Karthik Nayak @ 2026-10-06 20:51 UTC (permalink / raw)
To: Patrick Steinhardt, git
[-- Attachment #1: Type: text/plain, Size: 4796 bytes --]
Patrick Steinhardt <ps@pks.im> writes:
> Originally, when designing pluggable object databases the goal was that
> the object database can have multiple sources, and every source attached
> to it could use a different backend. This would have allowed for quite a
> lot of flexibility, as you could trivially mix and match different kinds
> of object storages in whatever way you like.
>
> But while well-intentioned, this design led to a bunch of conceptual
> problems:
>
> - We're now trying to read objects in source order, whereas we
> previously tried to read objects via packfiles before trying to read
> them via loose objects. This led to a performance regression when
> using alternates or when using a quarantine directory.
>
Could the design be instead to use a mapping function which allows us to
map objects to sources, based on some characteristics of the object?
> - Some data structures are supposed to only ever exist once, like for
> example bitmaps and commit graphs. At the same time, those data
> structures also span across the union of all objects, so they may
> cross sources.
>
This is not really a problem for having multiple sources though.
> - It is unclear how we can extend GIT_OBJECT_DIRECTORY or
> GIT_ALTERNATE_OBJECT_DIRECTORIES to become backend-agnostic in a
> backwards-compatible way. In general, introducing an object storage
> extension into the current status quo where alternates may have to
> be extended to become generic was proving to be painful.
>
> - Some mechanisms of alternates assume way too much about how exactly
> their backends work. Alternate refs for example assume that the
> alternate is backed by a filesystem path, and that this filesystem
> path may also allow us to read references. This is not a given
> though, as backends may not even have local data at all.
>
These two points do make sense around moving alternates into the files
source.
> In short, there are a bunch of conceptual mismatches when we have
> alternates and pluggable object databases coexist. So while the original
> idea was nice, it does not result in a system that is easy to reason
> about.
>
> Correct course by moving alternates into the "files" source itself so
> that it becomes an implementation detail thereof so that we can avoid
> all of these shortcomings. While it's unfortunate that we cannot easily
> mix and match sources now, that ability doesn't go away. It's still very
> much feasible to introduce a new backend that allows for exactly that
> use case, and such a backend may also be a lot more flexible as we can
> now add new logic to determine which objects should be stored where. So
> the original motivation for having per-source backends can still be
> realized with the new architecture.
>
Okay, this makes sense, so the new source could be merged source of some
sorts, with internal logic which it uses to map to different sources.
Nice.
> Note that as part of this move, we also handle the GIT_OBJECT_DIRECTORY
> and GIT_ALTERNATE_OBJECT_DIRECTORIES environment variables in the
> "files" backend. This may be surprising at first, but object directories
> are very much a concept of that backend, too. So these variables would
> have bad interactions with other backends, and they create a bit of a
> mismatch with the eventual object storage extension that we plan to
> introduce.
>
Yeah, this makes sense. The term 'DIRECTORY' itself is not always
extensible to other sources.
[snip]
> diff --git a/midx.c b/midx.c
> index c0f82c4163..8638ddf0be 100644
> --- a/midx.c
> +++ b/midx.c
> @@ -829,21 +829,15 @@ void clear_incremental_midx_files_ext(struct odb_source_packed *source, const ch
>
> void clear_midx_file(struct repository *r)
> {
> - struct odb_source_files *files;
> + struct odb_source_files *files = odb_source_files_downcast(r->objects->source);
We remove the the previous `if(r->objects)` check here, is that okay?
> struct strbuf midx = STRBUF_INIT;
>
> - if (r->objects) {
> - struct odb_source *source;
> -
> - for (source = r->objects->sources; source; source = source->next) {
> - files = odb_source_files_downcast(source);
> - if (files->dirs->packed->midx)
> - close_midx(files->dirs->packed->midx);
> - files->dirs->packed->midx = NULL;
> - }
> + for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
> + if (dir->packed->midx)
> + close_midx(dir->packed->midx);
> + dir->packed->midx = NULL;
> }
>
> - files = odb_source_files_downcast(r->objects->sources);
> get_midx_filename(files->dirs->packed, &midx);
>
> if (remove_path(midx.buf))
[snip]
This is the only question I have in this big patch, I did a full read
and tried to compare the moves visually and they looked okay to me.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 41+ messages in thread* Re: [PATCH 12/13] odb/source-files: move alternates into the backend
2026-10-06 20:51 ` Karthik Nayak
@ 2026-10-07 5:50 ` Patrick Steinhardt
0 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-07 5:50 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git
On Tue, Oct 06, 2026 at 01:51:33PM -0700, Karthik Nayak wrote:
> Patrick Steinhardt <ps@pks.im> writes:
>
> > Originally, when designing pluggable object databases the goal was that
> > the object database can have multiple sources, and every source attached
> > to it could use a different backend. This would have allowed for quite a
> > lot of flexibility, as you could trivially mix and match different kinds
> > of object storages in whatever way you like.
> >
> > But while well-intentioned, this design led to a bunch of conceptual
> > problems:
> >
> > - We're now trying to read objects in source order, whereas we
> > previously tried to read objects via packfiles before trying to read
> > them via loose objects. This led to a performance regression when
> > using alternates or when using a quarantine directory.
>
> Could the design be instead to use a mapping function which allows us to
> map objects to sources, based on some characteristics of the object?
You could, but it adds complexity that only needs to exist because of
the needs of the "files" backend. Ideally though, we'd not be leaking
internal implementation details of specific backends into callers and
have the interfaces be as agnoics as possible.
> > - Some data structures are supposed to only ever exist once, like for
> > example bitmaps and commit graphs. At the same time, those data
> > structures also span across the union of all objects, so they may
> > cross sources.
>
> This is not really a problem for having multiple sources though.
Not necessarily, but it makes it extremely awkward. The sources now need
to reach into the other sources and be aware of them, and that is a huge
design smell. I've tried multiple times to squeeze these data structures
into the design, but everything single time the result was atrocious.
[snip]
> > In short, there are a bunch of conceptual mismatches when we have
> > alternates and pluggable object databases coexist. So while the original
> > idea was nice, it does not result in a system that is easy to reason
> > about.
> >
> > Correct course by moving alternates into the "files" source itself so
> > that it becomes an implementation detail thereof so that we can avoid
> > all of these shortcomings. While it's unfortunate that we cannot easily
> > mix and match sources now, that ability doesn't go away. It's still very
> > much feasible to introduce a new backend that allows for exactly that
> > use case, and such a backend may also be a lot more flexible as we can
> > now add new logic to determine which objects should be stored where. So
> > the original motivation for having per-source backends can still be
> > realized with the new architecture.
> >
>
> Okay, this makes sense, so the new source could be merged source of some
> sorts, with internal logic which it uses to map to different sources.
> Nice.
Yes, exactly. And such a design would also have three important benefits:
- We can start from scratch and be sure that such a filtering system
is well defined instead of trying to shoehorn this into the object
database somehow.
- The design can be a lot more flexible because we start from scratch,
and it can easily have configuration to fine-tune things.
- The logic to handle this would be entirely self-contained in such a
backend, and its design details would not have to leak into callers.
The only downside is that we'd have to have another backend specific to
such a thing. But I'd rather have a backend specifically designed for
this that is entirely self-contained compared to having to support such
a feature with code cluttered around our object subsystems.
It took me a while to realize this myself though.
> > diff --git a/midx.c b/midx.c
> > index c0f82c4163..8638ddf0be 100644
> > --- a/midx.c
> > +++ b/midx.c
> > @@ -829,21 +829,15 @@ void clear_incremental_midx_files_ext(struct odb_source_packed *source, const ch
> >
> > void clear_midx_file(struct repository *r)
> > {
> > - struct odb_source_files *files;
> > + struct odb_source_files *files = odb_source_files_downcast(r->objects->source);
>
> We remove the the previous `if(r->objects)` check here, is that okay?
Yes, it is. There's only a single caller, and that caller
unconditionally dereferences `r->objects` already. And we also
dereference that pointer a bit further down in this same function here.
So the check was giving a false sense of security anyway, and we're
basically just moving up the unconditional dereference of the pointer
now.
Thanks!
Patrick
^ permalink raw reply [flat|nested] 41+ messages in thread
* [PATCH 13/13] odb/source: drop `read_alternates` callback
2026-10-02 10:08 [PATCH 00/13] odb/source-files: move alternates into the backend Patrick Steinhardt
` (11 preceding siblings ...)
2026-10-02 10:08 ` [PATCH 12/13] odb/source-files: move alternates into the backend Patrick Steinhardt
@ 2026-10-02 10:08 ` Patrick Steinhardt
2026-10-06 20:53 ` [PATCH 00/13] odb/source-files: move alternates into the backend Karthik Nayak
2026-10-08 8:35 ` [PATCH v2 " Patrick Steinhardt
14 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-02 10:08 UTC (permalink / raw)
To: git
Now that alternates are an implementation detail of the "files" backend
there is no reason anymore to expose them anywhere else. The callback to
read alternates is thus no longer required. Remove it.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
odb/source-files.c | 7 -------
odb/source-inmemory.c | 7 -------
odb/source-loose.c | 7 -------
odb/source-packed.c | 7 -------
odb/source.h | 27 ---------------------------
5 files changed, 55 deletions(-)
diff --git a/odb/source-files.c b/odb/source-files.c
index 072f515b36..455f409db0 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -594,12 +594,6 @@ static int odb_source_files_begin_transaction(struct odb_source *source,
return odb_transaction_files_begin(source, out, flags);
}
-static int odb_source_files_read_alternates(struct odb_source *source,
- struct strvec *out)
-{
- return read_alternates(source->path, out);
-}
-
static int too_many_loose_objects(struct odb_source_files *files, int limit)
{
unsigned long loose_count;
@@ -1277,7 +1271,6 @@ struct odb_source_files *odb_source_files_new(struct object_database *odb,
files->base.write_object = odb_source_files_write_object;
files->base.write_object_stream = odb_source_files_write_object_stream;
files->base.begin_transaction = odb_source_files_begin_transaction;
- files->base.read_alternates = odb_source_files_read_alternates;
files->base.optimize = odb_source_files_optimize;
files->base.optimize_required = odb_source_files_optimize_required;
files->base.generate_pack = odb_source_files_generate_pack;
diff --git a/odb/source-inmemory.c b/odb/source-inmemory.c
index f67c74a724..868e723c81 100644
--- a/odb/source-inmemory.c
+++ b/odb/source-inmemory.c
@@ -322,12 +322,6 @@ static int odb_source_inmemory_begin_transaction(struct odb_source *source UNUSE
return error("in-memory source does not support transactions");
}
-static int odb_source_inmemory_read_alternates(struct odb_source *source UNUSED,
- struct strvec *out UNUSED)
-{
- return 0;
-}
-
static void odb_source_inmemory_close(struct odb_source *source UNUSED)
{
}
@@ -390,7 +384,6 @@ struct odb_source_inmemory *odb_source_inmemory_new(struct object_database *odb)
source->base.write_object_stream = odb_source_inmemory_write_object_stream;
source->base.freshen_object = odb_source_inmemory_freshen_object;
source->base.begin_transaction = odb_source_inmemory_begin_transaction;
- source->base.read_alternates = odb_source_inmemory_read_alternates;
return source;
}
diff --git a/odb/source-loose.c b/odb/source-loose.c
index b2fbccd3c0..0664634938 100644
--- a/odb/source-loose.c
+++ b/odb/source-loose.c
@@ -985,12 +985,6 @@ static int odb_source_loose_begin_transaction(struct odb_source *source UNUSED,
return error("loose source does not support transactions");
}
-static int odb_source_loose_read_alternates(struct odb_source *source UNUSED,
- struct strvec *out UNUSED)
-{
- return 0;
-}
-
static void odb_source_loose_clear_cache(struct odb_source_loose *loose)
{
oidtree_clear(loose->cache);
@@ -1145,7 +1139,6 @@ struct odb_source_loose *odb_source_loose_new(struct object_database *odb,
loose->base.write_object = odb_source_loose_write_object;
loose->base.write_object_stream = odb_source_loose_write_object_stream;
loose->base.begin_transaction = odb_source_loose_begin_transaction;
- loose->base.read_alternates = odb_source_loose_read_alternates;
if (!is_absolute_path(loose->base.path))
chdir_notify_register(odb_source_loose_reparent, loose);
diff --git a/odb/source-packed.c b/odb/source-packed.c
index 831501f397..2226fbd313 100644
--- a/odb/source-packed.c
+++ b/odb/source-packed.c
@@ -687,12 +687,6 @@ static int odb_source_packed_begin_transaction(struct odb_source *source UNUSED,
return error("packed backend cannot begin transactions");
}
-static int odb_source_packed_read_alternates(struct odb_source *source UNUSED,
- struct strvec *out UNUSED)
-{
- return 0;
-}
-
void (*report_garbage)(unsigned seen_bits, const char *path);
static void report_helper(const struct string_list *list,
@@ -1019,7 +1013,6 @@ struct odb_source_packed *odb_source_packed_new(struct object_database *odb,
packed->base.write_object = odb_source_packed_write_object;
packed->base.write_object_stream = odb_source_packed_write_object_stream;
packed->base.begin_transaction = odb_source_packed_begin_transaction;
- packed->base.read_alternates = odb_source_packed_read_alternates;
if (!is_absolute_path(path))
chdir_notify_register(odb_source_packed_reparent, packed);
diff --git a/odb/source.h b/odb/source.h
index 6718aced6a..1867347dfc 100644
--- a/odb/source.h
+++ b/odb/source.h
@@ -258,19 +258,6 @@ struct odb_source {
struct odb_transaction **out,
enum odb_transaction_flags flags);
- /*
- * This callback is expected to read the list of alternate object
- * database sources connected to it and write them into the `strvec`.
- *
- * The result is expected to be paths to the alternates. All paths must
- * be resolved to absolute paths.
- *
- * The callback is expected to return 0 on success, a negative error
- * code otherwise.
- */
- int (*read_alternates)(struct odb_source *source,
- struct strvec *out);
-
/*
* This callback is expected to optimize the object database source.
* Returns 0 on success, a negative error code otherwise.
@@ -507,20 +494,6 @@ static inline int odb_source_write_object_stream(struct odb_source *source,
return source->write_object_stream(source, stream, oid);
}
-/*
- * Read the list of alternative object database sources from the given backend
- * and populate the `strvec` with them. The listing is not recursive -- that
- * is, if any of the yielded alternate sources has alternates itself, those
- * will not be yielded as part of this function call.
- *
- * Return 0 on success, a negative error code otherwise.
- */
-static inline int odb_source_read_alternates(struct odb_source *source,
- struct strvec *out)
-{
- return source->read_alternates(source, out);
-}
-
/*
* Create a new transaction that can be used to write objects into a temporary
* staging area. The objects will only be persisted when the transaction is
--
2.56.0.379.gc618271300.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* Re: [PATCH 00/13] odb/source-files: move alternates into the backend
2026-10-02 10:08 [PATCH 00/13] odb/source-files: move alternates into the backend Patrick Steinhardt
` (12 preceding siblings ...)
2026-10-02 10:08 ` [PATCH 13/13] odb/source: drop `read_alternates` callback Patrick Steinhardt
@ 2026-10-06 20:53 ` Karthik Nayak
2026-10-07 5:50 ` Patrick Steinhardt
2026-10-08 8:35 ` [PATCH v2 " Patrick Steinhardt
14 siblings, 1 reply; 41+ messages in thread
From: Karthik Nayak @ 2026-10-06 20:53 UTC (permalink / raw)
To: Patrick Steinhardt, git
[-- Attachment #1: Type: text/plain, Size: 2356 bytes --]
Patrick Steinhardt <ps@pks.im> writes:
> Hi,
>
> Originally, when designing pluggable object databases the goal was that
> the object database can have multiple sources, and every source attached
> to it could use a different backend. This would have allowed for quite a
> lot of flexibility, as you could trivially mix and match different kinds
> of object storages in whatever way you like.
>
> But while well-intentioned, this design led to a bunch of conceptual
> problems:
>
> - We're now trying to read objects in source order, whereas we
> previously tried to read objects via packfiles before trying to read
> them via loose objects. This led to a performance regression when
> using alternates or when using a quarantine directory.
>
> - Some data structures are supposed to only ever exist once, like for
> example bitmaps and commit graphs. At the same time, those data
> structures also span across the union of all objects, so they may
> cross sources.
>
> - It is unclear how we can extend GIT_OBJECT_DIRECTORY or
> GIT_ALTERNATE_OBJECT_DIRECTORIES to become backend-agnostic in a
> backwards-compatible way. In general, introducing an object storage
> extension into the current status quo where alternates may have to
> be extended to become generic was proving to be painful.
>
> - Some mechanisms of alternates assume way too much about how exactly
> their backends work. Alternate refs for example assume that the
> alternate is backed by a filesystem path, and that this filesystem
> path may also allow us to read references. This is not a given
> though, as backends may not even have local data at all.
>
> In short, there are a bunch of conceptual mismatches when we have
> alternates and pluggable object databases coexist. So while the original
> idea was nice, it does not result in a system that is easy to reason
> about.
>
> This patch series corrects course by moving alternates into the "files"
> backend itself so that they become another implementation detail. It's
> unfortunately on the bigger side, and I'm sorry about that, but I
> couldn't really find a way to split it up further in a sensible way.
I went through the series, took attention split over two days. The
changes look good to me, but would definitely like to see another review :)
[snip]
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 41+ messages in thread* Re: [PATCH 00/13] odb/source-files: move alternates into the backend
2026-10-06 20:53 ` [PATCH 00/13] odb/source-files: move alternates into the backend Karthik Nayak
@ 2026-10-07 5:50 ` Patrick Steinhardt
0 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-07 5:50 UTC (permalink / raw)
To: Karthik Nayak; +Cc: git
On Tue, Oct 06, 2026 at 01:53:24PM -0700, Karthik Nayak wrote:
> Patrick Steinhardt <ps@pks.im> writes:
>
> > Hi,
> >
> > Originally, when designing pluggable object databases the goal was that
> > the object database can have multiple sources, and every source attached
> > to it could use a different backend. This would have allowed for quite a
> > lot of flexibility, as you could trivially mix and match different kinds
> > of object storages in whatever way you like.
> >
> > But while well-intentioned, this design led to a bunch of conceptual
> > problems:
> >
> > - We're now trying to read objects in source order, whereas we
> > previously tried to read objects via packfiles before trying to read
> > them via loose objects. This led to a performance regression when
> > using alternates or when using a quarantine directory.
> >
> > - Some data structures are supposed to only ever exist once, like for
> > example bitmaps and commit graphs. At the same time, those data
> > structures also span across the union of all objects, so they may
> > cross sources.
> >
> > - It is unclear how we can extend GIT_OBJECT_DIRECTORY or
> > GIT_ALTERNATE_OBJECT_DIRECTORIES to become backend-agnostic in a
> > backwards-compatible way. In general, introducing an object storage
> > extension into the current status quo where alternates may have to
> > be extended to become generic was proving to be painful.
> >
> > - Some mechanisms of alternates assume way too much about how exactly
> > their backends work. Alternate refs for example assume that the
> > alternate is backed by a filesystem path, and that this filesystem
> > path may also allow us to read references. This is not a given
> > though, as backends may not even have local data at all.
> >
> > In short, there are a bunch of conceptual mismatches when we have
> > alternates and pluggable object databases coexist. So while the original
> > idea was nice, it does not result in a system that is easy to reason
> > about.
> >
> > This patch series corrects course by moving alternates into the "files"
> > backend itself so that they become another implementation detail. It's
> > unfortunately on the bigger side, and I'm sorry about that, but I
> > couldn't really find a way to split it up further in a sensible way.
>
> I went through the series, took attention split over two days. The
> changes look good to me, but would definitely like to see another review :)
Thanks for your review!
Patrick
^ permalink raw reply [flat|nested] 41+ messages in thread
* [PATCH v2 00/13] odb/source-files: move alternates into the backend
2026-10-02 10:08 [PATCH 00/13] odb/source-files: move alternates into the backend Patrick Steinhardt
` (13 preceding siblings ...)
2026-10-06 20:53 ` [PATCH 00/13] odb/source-files: move alternates into the backend Karthik Nayak
@ 2026-10-08 8:35 ` Patrick Steinhardt
2026-10-08 8:35 ` [PATCH v2 01/13] commit-graph: require resolved packfile paths for `stdin_packs` Patrick Steinhardt
` (13 more replies)
14 siblings, 14 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-08 8:35 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak
Hi,
Originally, when designing pluggable object databases the goal was that
the object database can have multiple sources, and every source attached
to it could use a different backend. This would have allowed for quite a
lot of flexibility, as you could trivially mix and match different kinds
of object storages in whatever way you like.
But while well-intentioned, this design led to a bunch of conceptual
problems:
- We're now trying to read objects in source order, whereas we
previously tried to read objects via packfiles before trying to read
them via loose objects. This led to a performance regression when
using alternates or when using a quarantine directory.
- Some data structures are supposed to only ever exist once, like for
example bitmaps and commit graphs. At the same time, those data
structures also span across the union of all objects, so they may
cross sources.
- It is unclear how we can extend GIT_OBJECT_DIRECTORY or
GIT_ALTERNATE_OBJECT_DIRECTORIES to become backend-agnostic in a
backwards-compatible way. In general, introducing an object storage
extension into the current status quo where alternates may have to
be extended to become generic was proving to be painful.
- Some mechanisms of alternates assume way too much about how exactly
their backends work. Alternate refs for example assume that the
alternate is backed by a filesystem path, and that this filesystem
path may also allow us to read references. This is not a given
though, as backends may not even have local data at all.
In short, there are a bunch of conceptual mismatches when we have
alternates and pluggable object databases coexist. So while the original
idea was nice, it does not result in a system that is easy to reason
about.
This patch series corrects course by moving alternates into the "files"
backend itself so that they become another implementation detail. It's
unfortunately on the bigger side, and I'm sorry about that, but I
couldn't really find a way to split it up further in a sensible way.
Note that the above problems aren't fixed by this series yet, but it is
the prerequisite to fix them in subsequent patch series.
The series is built on top of c46c1e3772 (Start Git 2.98 cycle,
2026-09-30). See the first version of this patch series for the conflict
resolution that's required for this merge base.
Changes in v2:
- Adapt the first commit message to better motivate the refactoring.
- Link to v1: https://patch.msgid.link/20261002-pks-odb-move-alternates-v1-0-8a63507b88c4@pks.im
Thanks!
Patrick
---
Patrick Steinhardt (13):
commit-graph: require resolved packfile paths for `stdin_packs`
commit-graph: stop depending on `struct odb_source`
odb/source-files: introduce `struct odb_files_dir`
odb: refactor `odb_for_each_alternate()` to yield dirs
odb: refactor `odb_find_source()` to yield dirs
odb/source-files: add the ability to have multiple object dirs
tmp-objdir: absorb logic to set and restore primary sources
tmp-objdir: manage quarantine as an object directory
tmp-objdir: replace primary source at creation time
odb/source: make `will_destroy` an implementation detail
odb/source-files: extract reading alternates
odb/source-files: move alternates into the backend
odb/source: drop `read_alternates` callback
builtin/commit-graph.c | 41 ++--
builtin/commit.c | 2 +-
builtin/count-objects.c | 6 +-
builtin/fast-import.c | 20 +-
builtin/fetch.c | 4 +-
builtin/fsck.c | 6 +-
builtin/gc.c | 15 +-
builtin/index-pack.c | 4 +-
builtin/merge.c | 2 +-
builtin/multi-pack-index.c | 48 ++---
builtin/pack-objects.c | 66 +++---
builtin/prune.c | 2 +-
builtin/repack.c | 4 +-
builtin/submodule--helper.c | 7 +-
bundle.c | 2 +-
commit-graph.c | 150 +++++++-------
commit-graph.h | 31 ++-
diagnose.c | 8 +-
fetch-pack.c | 2 +-
http-walker.c | 4 +-
http.c | 12 +-
log-tree.c | 3 +-
loose.c | 18 +-
midx.c | 43 ++--
object-file.c | 8 +-
odb.c | 439 +++++-----------------------------------
odb.h | 62 +-----
odb/source-files.c | 482 ++++++++++++++++++++++++++++++++++++--------
odb/source-files.h | 65 +++++-
odb/source-inmemory.c | 7 -
odb/source-loose.c | 9 +-
odb/source-loose.h | 3 +
odb/source-packed.c | 7 -
odb/source.c | 5 +-
odb/source.h | 57 +-----
odb/streaming.c | 8 +-
odb/transaction.c | 2 +-
pack-bitmap.c | 8 +-
packfile.c | 28 ++-
packfile.h | 21 +-
path.c | 2 +-
prune-packed.c | 2 +-
repack-geometry.c | 2 +-
repack-midx.c | 6 +-
repack.c | 6 +-
repository.c | 4 +-
setup.c | 2 +-
t/helper/test-read-graph.c | 5 +-
t/helper/test-read-midx.c | 8 +-
t/t4216-log-bloom.sh | 4 +-
tmp-objdir.c | 67 ++++--
tmp-objdir.h | 18 +-
52 files changed, 883 insertions(+), 954 deletions(-)
Range-diff versus v1:
1: 985b068271 ! 1: 43a288604d commit-graph: require resolved packfile paths for `stdin_packs`
@@ Commit message
for a set of packfiles via the "--stdin-packs" option. Those users are
expected to pass in relative paths, and those eventually get resolved in
`fill_oids_from_packs()`. This ties the logic in "commit-graph.c" to the
- specific object database source.
+ specific object database source, as the subsystem now needs to assume
+ where a specific packfile is located relative to the source itself.
Refactor the logic to instead require the caller to pass in resolved
- packfiles to untangle that dependency.
+ packfiles to untangle that dependency. This also makes the next change
+ easier to implement, where we'll get rid of passing the source to the
+ commit-graph subsystem.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
2: 537c2895a9 = 2: f455f1986c commit-graph: stop depending on `struct odb_source`
3: ce5ade106a = 3: 062f8b02d6 odb/source-files: introduce `struct odb_files_dir`
4: 4336f3d53d = 4: 68e73bbdc6 odb: refactor `odb_for_each_alternate()` to yield dirs
5: 77c5749db8 = 5: 078d51c82d odb: refactor `odb_find_source()` to yield dirs
6: 3d59418c91 = 6: b203565c43 odb/source-files: add the ability to have multiple object dirs
7: 38b3658234 = 7: c53ba11900 tmp-objdir: absorb logic to set and restore primary sources
8: 928edd062b = 8: 92c0a38193 tmp-objdir: manage quarantine as an object directory
9: f51db68dea = 9: e1fc5be9a7 tmp-objdir: replace primary source at creation time
10: a71a69e2c3 = 10: 98605aa991 odb/source: make `will_destroy` an implementation detail
11: fd434eaa74 = 11: 2ce6282162 odb/source-files: extract reading alternates
12: a4a8e53216 = 12: bc2a33d360 odb/source-files: move alternates into the backend
13: 39a3e9e1dd = 13: 41834f9155 odb/source: drop `read_alternates` callback
---
base-commit: 2f92b2890ddaf3d7ea29470c02418271c1a4cd79
change-id: 20260924-pks-odb-move-alternates-4a0babe4b0f3
^ permalink raw reply [flat|nested] 41+ messages in thread* [PATCH v2 01/13] commit-graph: require resolved packfile paths for `stdin_packs`
2026-10-08 8:35 ` [PATCH v2 " Patrick Steinhardt
@ 2026-10-08 8:35 ` Patrick Steinhardt
2026-10-08 8:35 ` [PATCH v2 02/13] commit-graph: stop depending on `struct odb_source` Patrick Steinhardt
` (12 subsequent siblings)
13 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-08 8:35 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak
Users can ask git-commit-graph(1) to write a commit graph specifically
for a set of packfiles via the "--stdin-packs" option. Those users are
expected to pass in relative paths, and those eventually get resolved in
`fill_oids_from_packs()`. This ties the logic in "commit-graph.c" to the
specific object database source, as the subsystem now needs to assume
where a specific packfile is located relative to the source itself.
Refactor the logic to instead require the caller to pass in resolved
packfiles to untangle that dependency. This also makes the next change
easier to implement, where we'll get rid of passing the source to the
commit-graph subsystem.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/commit-graph.c | 17 ++++++++++++++---
commit-graph.c | 15 +++++----------
2 files changed, 19 insertions(+), 13 deletions(-)
diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c
index d62005edc0..b5784ad3c7 100644
--- a/builtin/commit-graph.c
+++ b/builtin/commit-graph.c
@@ -8,6 +8,7 @@
#include "parse-options.h"
#include "commit-graph.h"
#include "odb.h"
+#include "odb/source.h"
#include "progress.h"
#include "replace-object.h"
#include "strbuf.h"
@@ -302,9 +303,19 @@ static int graph_write(int argc, const char **argv, const char *prefix,
}
if (opts.stdin_packs) {
- while (strbuf_getline(&buf, stdin) != EOF)
- string_list_append_nodup(&pack_indexes,
- strbuf_detach(&buf, NULL));
+ struct strbuf packname = STRBUF_INIT;
+ size_t dirlen;
+
+ strbuf_addf(&packname, "%s/pack/", source->path);
+ dirlen = packname.len;
+
+ while (strbuf_getline(&buf, stdin) != EOF) {
+ strbuf_setlen(&packname, dirlen);
+ strbuf_addbuf(&packname, &buf);
+ string_list_append(&pack_indexes, packname.buf);
+ }
+
+ strbuf_release(&packname);
} else if (opts.stdin_commits) {
oidset_init(&commits, 0);
if (opts.progress)
diff --git a/commit-graph.c b/commit-graph.c
index 983c11ce85..73814c1622 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -1936,12 +1936,8 @@ static int fill_oids_from_packs(struct write_commit_graph_context *ctx,
{
uint32_t i;
struct strbuf progress_title = STRBUF_INIT;
- struct strbuf packname = STRBUF_INIT;
- int dirlen;
int ret = 0;
- strbuf_addf(&packname, "%s/pack/", ctx->odb_source->path);
- dirlen = packname.len;
if (ctx->report_progress) {
strbuf_addf(&progress_title,
Q_("Finding commits for commit graph in %"PRIuMAX" pack",
@@ -1954,15 +1950,15 @@ static int fill_oids_from_packs(struct write_commit_graph_context *ctx,
}
for (i = 0; i < pack_indexes->nr; i++) {
struct packed_git *p;
- strbuf_setlen(&packname, dirlen);
- strbuf_addstr(&packname, pack_indexes->items[i].string);
- p = add_packed_git(ctx->r, packname.buf, packname.len, 1);
+
+ p = add_packed_git(ctx->r, pack_indexes->items[i].string,
+ strlen(pack_indexes->items[i].string), 1);
if (!p) {
- ret = error(_("error adding pack %s"), packname.buf);
+ ret = error(_("error adding pack %s"), pack_indexes->items[i].string);
goto cleanup;
}
if (open_pack_index(p)) {
- ret = error(_("error opening index for %s"), packname.buf);
+ ret = error(_("error opening index for %s"), pack_indexes->items[i].string);
close_pack(p);
free(p);
goto cleanup;
@@ -1976,7 +1972,6 @@ static int fill_oids_from_packs(struct write_commit_graph_context *ctx,
cleanup:
stop_progress(&ctx->progress);
strbuf_release(&progress_title);
- strbuf_release(&packname);
return ret;
}
--
2.56.0.406.ga2d225a756.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* [PATCH v2 02/13] commit-graph: stop depending on `struct odb_source`
2026-10-08 8:35 ` [PATCH v2 " Patrick Steinhardt
2026-10-08 8:35 ` [PATCH v2 01/13] commit-graph: require resolved packfile paths for `stdin_packs` Patrick Steinhardt
@ 2026-10-08 8:35 ` Patrick Steinhardt
2026-10-08 8:35 ` [PATCH v2 03/13] odb/source-files: introduce `struct odb_files_dir` Patrick Steinhardt
` (11 subsequent siblings)
13 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-08 8:35 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak
To read or write a commit graph we require access to the repository that
the graph should be read from or written for as well as the object
directory to derive its location. Instead of passing in these two bits
of information explicitly though, we pass in a `struct odb_source`,
which carries with it both of these pieces of information.
But ultimately, this is somewhat flawed as we now assume that the source
even has an on-disk representation, and furthermore we assume that the
commit graphs would live in ".git/objects/info". This is true for the
"files" backend, but it's not necessarily true for any other backend
that we may eventually want to introduce. So eventually, we'll want to
evolve the commit-graph subsystem to become agnostic of the backend's
layout and let the backend itself decide where to read a commit graph
from or where to write it to.
We're not there yet to do that switch, but the current design is already
causing issues for the intent of this patch series where we want to move
alternates into the "files" backend.
Convert the subsystem to take a repository plus an object directory path
instead. This unblocks moving around alternates, and it's also a step
in the right direction for moving commit graphs into the source in a
later series.
Note that we previously compared the `struct odb_source` pointers of two
commit graphs to figure out whether they were located in the same object
directory, whereas we now have to compare their paths. Callers may pass
in those paths in different forms though, for example relative to the
current working directory or as absolute paths. Convert the object
directory into an absolute path both when loading and when writing
commit graphs so that the comparisons become more robust.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/commit-graph.c | 10 ++--
builtin/commit.c | 2 +-
builtin/fetch.c | 4 +-
builtin/gc.c | 3 +-
builtin/merge.c | 2 +-
commit-graph.c | 115 +++++++++++++++++++++++++--------------------
commit-graph.h | 31 ++++++++----
t/helper/test-read-graph.c | 3 +-
t/t4216-log-bloom.sh | 4 +-
9 files changed, 102 insertions(+), 72 deletions(-)
diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c
index b5784ad3c7..a986f08a94 100644
--- a/builtin/commit-graph.c
+++ b/builtin/commit-graph.c
@@ -104,8 +104,8 @@ static int graph_verify(int argc, const char **argv, const char *prefix,
flags |= COMMIT_GRAPH_WRITE_PROGRESS;
source = odb_find_source_or_die(the_repository->objects, opts.obj_dir);
- graph_name = get_commit_graph_filename(source);
- chain_name = get_commit_graph_chain_filename(source);
+ graph_name = get_commit_graph_filename(source->path);
+ chain_name = get_commit_graph_chain_filename(source->path);
if (open_commit_graph(graph_name, &fd, &st))
opened = OPENED_GRAPH;
else if (errno != ENOENT)
@@ -123,7 +123,7 @@ static int graph_verify(int argc, const char **argv, const char *prefix,
if (opened == OPENED_NONE)
return 0;
else if (opened == OPENED_GRAPH)
- graph = load_commit_graph_one_fd_st(source, fd, &st);
+ graph = load_commit_graph_one_fd_st(the_repository, source->path, fd, &st);
else
graph = load_commit_graph_chain_fd_st(the_repository->objects, fd, &st,
&incomplete_chain);
@@ -297,7 +297,7 @@ static int graph_write(int argc, const char **argv, const char *prefix,
source = odb_find_source_or_die(the_repository->objects, opts.obj_dir);
if (opts.reachable) {
- if (write_commit_graph_reachable(source, flags, &write_opts))
+ if (write_commit_graph_reachable(the_repository, source->path, flags, &write_opts))
result = 1;
goto cleanup;
}
@@ -334,7 +334,7 @@ static int graph_write(int argc, const char **argv, const char *prefix,
stop_progress(&progress);
}
- if (write_commit_graph(source,
+ if (write_commit_graph(the_repository, source->path,
opts.stdin_packs ? &pack_indexes : NULL,
opts.stdin_commits ? &commits : NULL,
flags,
diff --git a/builtin/commit.c b/builtin/commit.c
index 840b6b4083..0e750f6157 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1959,7 +1959,7 @@ int cmd_commit(int argc,
"new index file. Check that disk is not full and quota is\n"
"not exceeded, and then \"git restore --staged :/\" to recover."));
- git_test_write_commit_graph_or_die(the_repository->objects->sources);
+ git_test_write_commit_graph_or_die(the_repository);
repo_rerere(the_repository, 0);
run_auto_maintenance(the_repository, quiet);
diff --git a/builtin/fetch.c b/builtin/fetch.c
index b2decc6cfd..a4b21d2651 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -15,6 +15,7 @@
#include "refspec.h"
#include "object-name.h"
#include "odb.h"
+#include "odb/source.h"
#include "oidset.h"
#include "oid-array.h"
#include "commit.h"
@@ -2900,7 +2901,8 @@ int cmd_fetch(int argc,
commit_graph_flags |= COMMIT_GRAPH_WRITE_PROGRESS;
trace2_region_enter("fetch", "write-commit-graph", the_repository);
- write_commit_graph_reachable(the_repository->objects->sources,
+ write_commit_graph_reachable(the_repository,
+ the_repository->objects->sources->path,
commit_graph_flags,
NULL);
trace2_region_leave("fetch", "write-commit-graph", the_repository);
diff --git a/builtin/gc.c b/builtin/gc.c
index 57a3520263..7acd4f3215 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -733,7 +733,8 @@ int cmd_gc(int argc,
}
if (the_repository->settings.gc_write_commit_graph == 1)
- write_commit_graph_reachable(the_repository->objects->sources,
+ write_commit_graph_reachable(the_repository,
+ the_repository->objects->sources->path,
!opts.quiet && !daemonized ? COMMIT_GRAPH_WRITE_PROGRESS : 0,
NULL);
diff --git a/builtin/merge.c b/builtin/merge.c
index 5b4eb23a83..d7bf209114 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -1863,7 +1863,7 @@ int cmd_merge(int argc,
if (squash) {
finish(head_commit, remoteheads, NULL, NULL);
- git_test_write_commit_graph_or_die(the_repository->objects->sources);
+ git_test_write_commit_graph_or_die(the_repository);
} else
write_merge_state(remoteheads);
diff --git a/commit-graph.c b/commit-graph.c
index 73814c1622..80ebc6a542 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -1,6 +1,7 @@
#define DISABLE_SIGN_COMPARE_WARNINGS
#include "git-compat-util.h"
+#include "abspath.h"
#include "config.h"
#include "csum-file.h"
#include "environment.h"
@@ -28,7 +29,7 @@
#include "tree.h"
#include "chunk-format.h"
-void git_test_write_commit_graph_or_die(struct odb_source *source)
+void git_test_write_commit_graph_or_die(struct repository *repo)
{
int flags = 0;
if (!git_env_bool(GIT_TEST_COMMIT_GRAPH, 0))
@@ -37,7 +38,7 @@ void git_test_write_commit_graph_or_die(struct odb_source *source)
if (git_env_bool(GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS, 0))
flags = COMMIT_GRAPH_WRITE_BLOOM_FILTERS;
- if (write_commit_graph_reachable(source, flags, NULL))
+ if (write_commit_graph_reachable(repo, repo->objects->sources->path, flags, NULL))
die("failed to write commit-graph under GIT_TEST_COMMIT_GRAPH");
}
@@ -196,21 +197,21 @@ static int commit_gen_cmp(const void *va, const void *vb)
return 0;
}
-char *get_commit_graph_filename(struct odb_source *source)
+char *get_commit_graph_filename(const char *dir)
{
- return xstrfmt("%s/info/commit-graph", source->path);
+ return xstrfmt("%s/info/commit-graph", dir);
}
-static char *get_split_graph_filename(struct odb_source *source,
+static char *get_split_graph_filename(const char *dir,
const char *oid_hex)
{
- return xstrfmt("%s/info/commit-graphs/graph-%s.graph", source->path,
+ return xstrfmt("%s/info/commit-graphs/graph-%s.graph", dir,
oid_hex);
}
-char *get_commit_graph_chain_filename(struct odb_source *source)
+char *get_commit_graph_chain_filename(const char *dir)
{
- return xstrfmt("%s/info/commit-graphs/commit-graph-chain", source->path);
+ return xstrfmt("%s/info/commit-graphs/commit-graph-chain", dir);
}
static struct commit_graph *alloc_commit_graph(void)
@@ -253,7 +254,8 @@ int open_commit_graph(const char *graph_file, int *fd, struct stat *st)
return 1;
}
-struct commit_graph *load_commit_graph_one_fd_st(struct odb_source *source,
+struct commit_graph *load_commit_graph_one_fd_st(struct repository *repo,
+ const char *dir,
int fd, struct stat *st)
{
void *graph_map;
@@ -262,7 +264,7 @@ struct commit_graph *load_commit_graph_one_fd_st(struct odb_source *source,
graph_size = xsize_t(st->st_size);
- if (graph_size < graph_min_size(source->odb->repo->hash_algo)) {
+ if (graph_size < graph_min_size(repo->hash_algo)) {
close(fd);
error(_("commit-graph file is too small"));
return NULL;
@@ -270,9 +272,9 @@ struct commit_graph *load_commit_graph_one_fd_st(struct odb_source *source,
graph_map = xmmap(NULL, graph_size, PROT_READ, MAP_PRIVATE, fd, 0);
close(fd);
- ret = parse_commit_graph(source->odb->repo, graph_map, graph_size);
+ ret = parse_commit_graph(repo, graph_map, graph_size);
if (ret)
- ret->odb_source = source;
+ ret->dir = absolute_pathdup(dir);
else
munmap(graph_map, graph_size);
@@ -410,6 +412,7 @@ struct commit_graph *parse_commit_graph(struct repository *r,
graph = alloc_commit_graph();
+ graph->repo = r;
graph->hash_algo = r->hash_algo;
graph->num_chunks = *(unsigned char*)(data + 6);
graph->data = graph_map;
@@ -490,7 +493,8 @@ struct commit_graph *parse_commit_graph(struct repository *r,
return NULL;
}
-static struct commit_graph *load_commit_graph_one(struct odb_source *source,
+static struct commit_graph *load_commit_graph_one(struct repository *repo,
+ const char *dir,
const char *graph_file)
{
struct stat st;
@@ -501,17 +505,18 @@ static struct commit_graph *load_commit_graph_one(struct odb_source *source,
if (!open_ok)
return NULL;
- g = load_commit_graph_one_fd_st(source, fd, &st);
+ g = load_commit_graph_one_fd_st(repo, dir, fd, &st);
if (g)
- g->filename = xstrdup(graph_file);
+ g->filename = absolute_pathdup(graph_file);
return g;
}
-static struct commit_graph *load_commit_graph_v1(struct odb_source *source)
+static struct commit_graph *load_commit_graph_v1(struct repository *repo,
+ const char *dir)
{
- char *graph_name = get_commit_graph_filename(source);
- struct commit_graph *g = load_commit_graph_one(source, graph_name);
+ char *graph_name = get_commit_graph_filename(dir);
+ struct commit_graph *g = load_commit_graph_one(repo, dir, graph_name);
free(graph_name);
return g;
@@ -666,8 +671,8 @@ struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,
valid = 0;
for (source = odb->sources; source; source = source->next) {
- char *graph_name = get_split_graph_filename(source, line.buf);
- struct commit_graph *g = load_commit_graph_one(source, graph_name);
+ char *graph_name = get_split_graph_filename(source->path, line.buf);
+ struct commit_graph *g = load_commit_graph_one(odb->repo, source->path, graph_name);
free(graph_name);
@@ -700,29 +705,31 @@ struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,
return graph_chain;
}
-static struct commit_graph *load_commit_graph_chain(struct odb_source *source)
+static struct commit_graph *load_commit_graph_chain(struct repository *repo,
+ const char *dir)
{
- char *chain_file = get_commit_graph_chain_filename(source);
+ char *chain_file = get_commit_graph_chain_filename(dir);
struct stat st;
int fd;
struct commit_graph *g = NULL;
- if (open_commit_graph_chain(chain_file, &fd, &st, source->odb->repo->hash_algo)) {
+ if (open_commit_graph_chain(chain_file, &fd, &st, repo->hash_algo)) {
int incomplete;
/* ownership of fd is taken over by load function */
- g = load_commit_graph_chain_fd_st(source->odb, fd, &st, &incomplete);
+ g = load_commit_graph_chain_fd_st(repo->objects, fd, &st, &incomplete);
}
free(chain_file);
return g;
}
-struct commit_graph *read_commit_graph_one(struct odb_source *source)
+struct commit_graph *read_commit_graph_one(struct repository *repo,
+ const char *dir)
{
- struct commit_graph *g = load_commit_graph_v1(source);
+ struct commit_graph *g = load_commit_graph_v1(repo, dir);
if (!g)
- g = load_commit_graph_chain(source);
+ g = load_commit_graph_chain(repo, dir);
return g;
}
@@ -767,7 +774,7 @@ static struct commit_graph *prepare_commit_graph(struct repository *r)
return NULL;
for (source = r->objects->sources; source; source = source->next) {
- r->objects->commit_graph = read_commit_graph_one(source);
+ r->objects->commit_graph = read_commit_graph_one(r, source->path);
if (r->objects->commit_graph)
break;
}
@@ -866,7 +873,7 @@ static struct commit_list **insert_parent_or_die(struct commit_graph *g,
die("invalid parent position %"PRIu32, pos);
load_oid_from_graph(g, pos, &oid);
- c = lookup_commit(g->odb_source->odb->repo, &oid);
+ c = lookup_commit(g->repo, &oid);
if (!c)
die(_("could not find commit %s"), oid_to_hex(&oid));
commit_graph_data_at(c)->graph_pos = pos;
@@ -1103,7 +1110,7 @@ static struct tree *load_tree_for_commit(struct commit_graph *g,
graph_pos - g->num_commits_in_base);
oidread(&oid, commit_data, g->hash_algo);
- set_commit_tree(c, lookup_tree(g->odb_source->odb->repo, &oid));
+ set_commit_tree(c, lookup_tree(g->repo, &oid));
return c->maybe_tree;
}
@@ -1126,7 +1133,7 @@ struct tree *get_commit_tree_in_graph(struct repository *r, const struct commit
struct write_commit_graph_context {
struct repository *r;
- struct odb_source *odb_source;
+ char *dir;
char *graph_name;
struct oid_array oids;
struct commit_stack commits;
@@ -1902,7 +1909,8 @@ static int add_ref_to_set(const struct reference *ref, void *cb_data)
return 0;
}
-int write_commit_graph_reachable(struct odb_source *source,
+int write_commit_graph_reachable(struct repository *repo,
+ const char *dir,
enum commit_graph_write_flags flags,
const struct commit_graph_opts *opts)
{
@@ -1911,20 +1919,20 @@ int write_commit_graph_reachable(struct odb_source *source,
int result;
memset(&data, 0, sizeof(data));
- data.repo = source->odb->repo;
+ data.repo = repo;
data.commits = &commits;
if (flags & COMMIT_GRAPH_WRITE_PROGRESS)
data.progress = start_delayed_progress(
- source->odb->repo,
+ repo,
_("Collecting referenced commits"), 0);
- refs_for_each_ref(get_main_ref_store(source->odb->repo), add_ref_to_set,
+ refs_for_each_ref(get_main_ref_store(repo), add_ref_to_set,
&data);
stop_progress(&data.progress);
- result = write_commit_graph(source, NULL, &commits,
+ result = write_commit_graph(repo, dir, NULL, &commits,
flags, opts);
oidset_clear(&commits);
@@ -2103,10 +2111,10 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)
strbuf_addf(&tmp_file,
"%s/info/commit-graphs/tmp_graph_XXXXXX",
- ctx->odb_source->path);
+ ctx->dir);
ctx->graph_name = strbuf_detach(&tmp_file, NULL);
} else {
- ctx->graph_name = get_commit_graph_filename(ctx->odb_source);
+ ctx->graph_name = get_commit_graph_filename(ctx->dir);
}
if (safe_create_leading_directories(ctx->r, ctx->graph_name)) {
@@ -2116,7 +2124,7 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)
}
if (ctx->split) {
- char *lock_name = get_commit_graph_chain_filename(ctx->odb_source);
+ char *lock_name = get_commit_graph_chain_filename(ctx->dir);
repo_hold_lock_file_for_update_mode(ctx->r, &lk, lock_name,
LOCK_DIE_ON_ERROR, 0444);
@@ -2205,7 +2213,7 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)
if (ctx->split && ctx->base_graph_name && ctx->num_commit_graphs_after > 1) {
char *new_base_hash = xstrdup(oid_to_hex(&ctx->new_base_graph->oid));
- char *new_base_name = get_split_graph_filename(ctx->new_base_graph->odb_source, new_base_hash);
+ char *new_base_name = get_split_graph_filename(ctx->new_base_graph->dir, new_base_hash);
free(ctx->commit_graph_filenames_after[ctx->num_commit_graphs_after - 2]);
free(ctx->commit_graph_hash_after[ctx->num_commit_graphs_after - 2]);
@@ -2245,7 +2253,7 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)
}
}
} else {
- char *graph_name = get_commit_graph_filename(ctx->odb_source);
+ char *graph_name = get_commit_graph_filename(ctx->dir);
unlink(graph_name);
free(graph_name);
}
@@ -2253,7 +2261,7 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)
free(ctx->commit_graph_hash_after[ctx->num_commit_graphs_after - 1]);
ctx->commit_graph_hash_after[ctx->num_commit_graphs_after - 1] =
xstrdup(hash_to_hex_algop(file_hash, ctx->r->hash_algo));
- final_graph_name = get_split_graph_filename(ctx->odb_source,
+ final_graph_name = get_split_graph_filename(ctx->dir,
ctx->commit_graph_hash_after[ctx->num_commit_graphs_after - 1]);
free(ctx->commit_graph_filenames_after[ctx->num_commit_graphs_after - 1]);
ctx->commit_graph_filenames_after[ctx->num_commit_graphs_after - 1] = final_graph_name;
@@ -2305,7 +2313,7 @@ static void split_graph_merge_strategy(struct write_commit_graph_context *ctx,
flags != COMMIT_GRAPH_SPLIT_REPLACE) {
while (g && (g->num_commits <= st_mult(size_mult, num_commits) ||
(max_commits && num_commits > max_commits))) {
- if (g->odb_source != ctx->odb_source)
+ if (strcmp(g->dir, ctx->dir))
break;
if (unsigned_add_overflows(num_commits, g->num_commits))
@@ -2327,10 +2335,10 @@ static void split_graph_merge_strategy(struct write_commit_graph_context *ctx,
"should be 1 with --split=replace");
if (ctx->num_commit_graphs_after == 2) {
- char *old_graph_name = get_commit_graph_filename(g->odb_source);
+ char *old_graph_name = get_commit_graph_filename(g->dir);
if (!strcmp(g->filename, old_graph_name) &&
- g->odb_source != ctx->odb_source) {
+ strcmp(g->dir, ctx->dir)) {
ctx->num_commit_graphs_after = 1;
ctx->new_base_graph = NULL;
}
@@ -2500,13 +2508,13 @@ static void expire_commit_graphs(struct write_commit_graph_context *ctx)
if (ctx->opts && ctx->opts->expire_time)
expire_time = ctx->opts->expire_time;
if (!ctx->split) {
- char *chain_file_name = get_commit_graph_chain_filename(ctx->odb_source);
+ char *chain_file_name = get_commit_graph_chain_filename(ctx->dir);
unlink(chain_file_name);
free(chain_file_name);
ctx->num_commit_graphs_after = 0;
}
- strbuf_addstr(&path, ctx->odb_source->path);
+ strbuf_addstr(&path, ctx->dir);
strbuf_addstr(&path, "/info/commit-graphs");
dir = opendir(path.buf);
@@ -2548,16 +2556,15 @@ static void expire_commit_graphs(struct write_commit_graph_context *ctx)
strbuf_release(&path);
}
-int write_commit_graph(struct odb_source *source,
+int write_commit_graph(struct repository *r,
+ const char *dir,
const struct string_list *const pack_indexes,
struct oidset *commits,
enum commit_graph_write_flags flags,
const struct commit_graph_opts *opts)
{
- struct repository *r = source->odb->repo;
struct write_commit_graph_context ctx = {
.r = r,
- .odb_source = source,
.append = flags & COMMIT_GRAPH_WRITE_APPEND ? 1 : 0,
.report_progress = flags & COMMIT_GRAPH_WRITE_PROGRESS ? 1 : 0,
.split = flags & COMMIT_GRAPH_WRITE_SPLIT ? 1 : 0,
@@ -2588,6 +2595,8 @@ int write_commit_graph(struct odb_source *source,
return 0;
}
+ ctx.dir = absolute_pathdup(dir);
+
bloom_settings.hash_version = r->settings.commit_graph_changed_paths_version;
bloom_settings.bits_per_entry = git_env_ulong("GIT_TEST_BLOOM_SETTINGS_BITS_PER_ENTRY",
bloom_settings.bits_per_entry);
@@ -2710,6 +2719,7 @@ int write_commit_graph(struct odb_source *source,
cleanup:
free(ctx.graph_name);
free(ctx.base_graph_name);
+ free(ctx.dir);
commit_stack_clear(&ctx.commits);
oid_array_clear(&ctx.oids);
clear_topo_level_slab(&topo_levels);
@@ -2762,7 +2772,7 @@ static int verify_one_commit_graph(struct commit_graph *g,
struct progress *progress,
uint64_t *seen)
{
- struct repository *r = g->odb_source->odb->repo;
+ struct repository *r = g->repo;
uint32_t i, cur_fanout_pos = 0;
struct object_id prev_oid, cur_oid;
struct commit *seen_gen_zero = NULL;
@@ -2926,7 +2936,7 @@ int verify_commit_graph(struct commit_graph *g, int flags)
if (!(flags & COMMIT_GRAPH_VERIFY_SHALLOW))
total += g->num_commits_in_base;
- progress = start_progress(g->odb_source->odb->repo,
+ progress = start_progress(g->repo,
_("Verifying commits in commit graph"),
total);
}
@@ -2949,6 +2959,7 @@ void free_commit_graph(struct commit_graph *g)
if (g->data)
munmap((void *)g->data, g->data_len);
+ free(g->dir);
free(g->filename);
free(g->bloom_filter_settings);
free(g);
diff --git a/commit-graph.h b/commit-graph.h
index 13ca4ff010..bccf9c5c84 100644
--- a/commit-graph.h
+++ b/commit-graph.h
@@ -21,7 +21,7 @@
* call this method outside of a builtin, and only if you know what
* you are doing!
*/
-void git_test_write_commit_graph_or_die(struct odb_source *source);
+void git_test_write_commit_graph_or_die(struct repository *repo);
struct commit;
struct bloom_filter_settings;
@@ -29,8 +29,8 @@ struct repository;
struct object_database;
struct string_list;
-char *get_commit_graph_filename(struct odb_source *source);
-char *get_commit_graph_chain_filename(struct odb_source *source);
+char *get_commit_graph_filename(const char *dir);
+char *get_commit_graph_chain_filename(const char *dir);
int open_commit_graph(const char *graph_file, int *fd, struct stat *st);
int open_commit_graph_chain(const char *chain_file, int *fd, struct stat *st,
const struct git_hash_algo *hash_algo);
@@ -85,12 +85,13 @@ struct commit_graph {
const unsigned char *data;
size_t data_len;
+ struct repository *repo;
const struct git_hash_algo *hash_algo;
unsigned char num_chunks;
uint32_t num_commits;
struct object_id oid;
+ char *dir;
char *filename;
- struct odb_source *odb_source;
uint32_t num_commits_in_base;
unsigned int read_generation_data;
@@ -114,12 +115,20 @@ struct commit_graph {
struct bloom_filter_settings *bloom_filter_settings;
};
-struct commit_graph *load_commit_graph_one_fd_st(struct odb_source *source,
+/*
+ * Load commit graphs from the given object directory `dir`. The directory may
+ * be given as a relative path; it is canonicalized internally so that graphs
+ * loaded from the same directory compare equal regardless of how the caller
+ * spelled the path.
+ */
+struct commit_graph *load_commit_graph_one_fd_st(struct repository *repo,
+ const char *dir,
int fd, struct stat *st);
struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,
int fd, struct stat *st,
int *incomplete_chain);
-struct commit_graph *read_commit_graph_one(struct odb_source *source);
+struct commit_graph *read_commit_graph_one(struct repository *repo,
+ const char *dir);
struct repo_settings;
@@ -171,11 +180,17 @@ struct commit_graph_opts {
* and a negative value on failure. Note that if the repository
* is not compatible with the commit-graph feature, then the
* methods will return 0 without writing a commit-graph.
+ *
+ * The object directory `dir` may be given as a relative path; it is
+ * canonicalized internally so that it compares equal to the directory of
+ * graphs that have already been loaded.
*/
-int write_commit_graph_reachable(struct odb_source *source,
+int write_commit_graph_reachable(struct repository *repo,
+ const char *dir,
enum commit_graph_write_flags flags,
const struct commit_graph_opts *opts);
-int write_commit_graph(struct odb_source *source,
+int write_commit_graph(struct repository *r,
+ const char *dir,
const struct string_list *pack_indexes,
struct oidset *commits,
enum commit_graph_write_flags flags,
diff --git a/t/helper/test-read-graph.c b/t/helper/test-read-graph.c
index 9f07b9c25a..a75c817e47 100644
--- a/t/helper/test-read-graph.c
+++ b/t/helper/test-read-graph.c
@@ -4,6 +4,7 @@
#include "commit-graph.h"
#include "repository.h"
#include "odb.h"
+#include "odb/source.h"
#include "bloom.h"
#include "setup.h"
@@ -81,7 +82,7 @@ int cmd__read_graph(int argc, const char **argv)
prepare_repo_settings(the_repository);
- graph = read_commit_graph_one(source);
+ graph = read_commit_graph_one(the_repository, source->path);
if (!graph) {
ret = 1;
goto done;
diff --git a/t/t4216-log-bloom.sh b/t/t4216-log-bloom.sh
index ad2686669d..f57a3d6621 100755
--- a/t/t4216-log-bloom.sh
+++ b/t/t4216-log-bloom.sh
@@ -755,7 +755,7 @@ test_expect_success PERL_TEST_HELPERS 'Bloom reader notices too-small data chunk
test_expect_success PERL_TEST_HELPERS 'Bloom reader notices out-of-bounds filter offsets' '
check_corrupt_graph BIDX 12 FFFFFFFF &&
# use grep to avoid depending on exact chunk size
- test_grep "warning: ignoring out-of-range offset (4294967295) for changed-path filter at pos 3 of .git/objects/info/commit-graph" err
+ test_grep "warning: ignoring out-of-range offset (4294967295) for changed-path filter at pos 3 of $(pwd)/.git/objects/info/commit-graph" err
'
test_expect_success PERL_TEST_HELPERS 'Bloom reader notices too-small index chunk' '
@@ -773,7 +773,7 @@ test_expect_success PERL_TEST_HELPERS 'Bloom reader notices out-of-order index o
# actually reading from the bogus offsets anyway.
corrupt_graph BIDX 4 0000000c00000005 &&
echo "warning: ignoring decreasing changed-path index offsets" \
- "(12 > 5) for positions 1 and 2 of .git/objects/info/commit-graph" >expect.err &&
+ "(12 > 5) for positions 1 and 2 of $(pwd)/.git/objects/info/commit-graph" >expect.err &&
git -c core.commitGraph=false log -- A/B/file2 >expect.out &&
git -c core.commitGraph=true log -- A/B/file2 >out 2>err &&
test_cmp expect.out out &&
--
2.56.0.406.ga2d225a756.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* [PATCH v2 03/13] odb/source-files: introduce `struct odb_files_dir`
2026-10-08 8:35 ` [PATCH v2 " Patrick Steinhardt
2026-10-08 8:35 ` [PATCH v2 01/13] commit-graph: require resolved packfile paths for `stdin_packs` Patrick Steinhardt
2026-10-08 8:35 ` [PATCH v2 02/13] commit-graph: stop depending on `struct odb_source` Patrick Steinhardt
@ 2026-10-08 8:35 ` Patrick Steinhardt
2026-10-08 8:35 ` [PATCH v2 04/13] odb: refactor `odb_for_each_alternate()` to yield dirs Patrick Steinhardt
` (10 subsequent siblings)
13 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-08 8:35 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak
The "files" object database source consists of two separate backends for
storing loose and packed objects. These are managed as somewhat separate
entities even though they derive from the same object directory, whether
it's the primary object directory or that one from an alternate.
In a subsequent commit we'll move the handling of alternates into the
"files" backend completely so that it becomes another implementation
detail thereof. As part of this, we'll want to keep track of both of
these sub-sources as a single entity derived from their respective
object directory.
Prepare for this change by introducing a new `struct odb_files_dir` that
encapsulates them. For now, every "files" source has exactly one such
directory. In a subsequent commit though, we'll make it a linked list of
directories so that we can manage multiple such directories in a single
"files" source.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/fast-import.c | 6 ++--
builtin/index-pack.c | 2 +-
builtin/multi-pack-index.c | 14 ++++-----
builtin/pack-objects.c | 18 +++++------
builtin/repack.c | 4 +--
commit-graph.c | 2 +-
http-walker.c | 2 +-
http.c | 6 ++--
loose.c | 6 ++--
midx.c | 26 ++++++++--------
odb/source-files.c | 77 ++++++++++++++++++++++++++++++----------------
odb/source-files.h | 27 ++++++++++++++--
pack-bitmap.c | 2 +-
packfile.c | 8 ++---
packfile.h | 4 +--
repack-geometry.c | 2 +-
repack-midx.c | 6 ++--
repack.c | 2 +-
t/helper/test-read-midx.c | 2 +-
19 files changed, 131 insertions(+), 85 deletions(-)
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index fbd919982c..0bf76b028b 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -921,7 +921,7 @@ static void end_packfile(void)
idx_name = keep_pack(create_index());
/* Register the packfile with core git's machinery. */
- new_p = packfile_store_load_pack(files->packed, idx_name, 1);
+ new_p = packfile_store_load_pack(files->dirs->packed, idx_name, 1);
if (!new_p)
die(_("core Git rejected index %s"), idx_name);
all_packs[pack_id] = new_p;
@@ -1005,7 +1005,7 @@ static int store_object(
for (source = the_repository->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- if (!packfile_list_find_oid(packfile_store_get_packs(files->packed), &oid))
+ if (!packfile_list_find_oid(packfile_store_get_packs(files->dirs->packed), &oid))
continue;
e->type = type;
e->pack_id = MAX_PACK_ID;
@@ -1215,7 +1215,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
for (source = the_repository->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- if (!packfile_list_find_oid(packfile_store_get_packs(files->packed), &oid))
+ if (!packfile_list_find_oid(packfile_store_get_packs(files->dirs->packed), &oid))
continue;
e->type = OBJ_BLOB;
e->pack_id = MAX_PACK_ID;
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 6b2a87e2d3..70860b8f27 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1641,7 +1641,7 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
if (do_fsck_object && startup_info->have_repository) {
struct odb_source_files *files =
odb_source_files_downcast(the_repository->objects->sources);
- packfile_store_load_pack(files->packed, final_index_name, 0);
+ packfile_store_load_pack(files->dirs->packed, final_index_name, 0);
}
if (!from_stdin) {
diff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c
index 753bd53a70..a170ec80b9 100644
--- a/builtin/multi-pack-index.c
+++ b/builtin/multi-pack-index.c
@@ -213,7 +213,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,
read_packs_from_stdin(&packs);
- ret = write_midx_file_only(source->packed, &packs,
+ ret = write_midx_file_only(source->dirs->packed, &packs,
opts.preferred_pack,
opts.refs_snapshot,
opts.incremental_base, opts.flags);
@@ -225,7 +225,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,
}
- ret = write_midx_file(source->packed, opts.preferred_pack,
+ ret = write_midx_file(source->dirs->packed, opts.preferred_pack,
opts.refs_snapshot, opts.flags);
free(opts.refs_snapshot);
@@ -284,7 +284,7 @@ static int cmd_multi_pack_index_compact(int argc, const char **argv,
FREE_AND_NULL(options);
- m = get_multi_pack_index(source->packed);
+ m = get_multi_pack_index(source->dirs->packed);
for (cur = m; cur && !(from_midx && to_midx); cur = cur->base_midx) {
const char *midx_csum = midx_get_checksum_hex(cur);
@@ -307,7 +307,7 @@ static int cmd_multi_pack_index_compact(int argc, const char **argv,
die(_("MIDX %s must be an ancestor of %s"), argv[0], argv[1]);
}
- ret = write_midx_file_compact(source->packed, from_midx, to_midx,
+ ret = write_midx_file_compact(source->dirs->packed, from_midx, to_midx,
opts.incremental_base, opts.flags);
return ret;
@@ -339,7 +339,7 @@ static int cmd_multi_pack_index_verify(int argc, const char **argv,
FREE_AND_NULL(options);
- return verify_midx_file(source->packed, opts.flags);
+ return verify_midx_file(source->dirs->packed, opts.flags);
}
static int cmd_multi_pack_index_expire(int argc, const char **argv,
@@ -368,7 +368,7 @@ static int cmd_multi_pack_index_expire(int argc, const char **argv,
FREE_AND_NULL(options);
- return expire_midx_packs(source->packed, opts.flags);
+ return expire_midx_packs(source->dirs->packed, opts.flags);
}
static int cmd_multi_pack_index_repack(int argc, const char **argv,
@@ -400,7 +400,7 @@ static int cmd_multi_pack_index_repack(int argc, const char **argv,
FREE_AND_NULL(options);
- return midx_repack(source->packed, (size_t)opts.batch_size, opts.flags);
+ return midx_repack(source->dirs->packed, (size_t)opts.batch_size, opts.flags);
}
int cmd_multi_pack_index(int argc,
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index af9390a46b..070659b6ed 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1570,7 +1570,7 @@ static int want_cruft_object_mtime(struct repository *r,
for (source = r->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- struct packed_git **cache = packfile_store_get_kept_pack_cache(files->packed, flags);
+ struct packed_git **cache = packfile_store_get_kept_pack_cache(files->dirs->packed, flags);
for (; *cache; cache++) {
struct packed_git *p = *cache;
@@ -1765,7 +1765,7 @@ static int want_object_in_pack_mtime(const struct object_id *oid,
struct odb_source *source = the_repository->objects->sources->next;
for (; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- if (!odb_source_read_object_info(&files->loose->base, oid, NULL, 0, NULL))
+ if (!odb_source_read_object_info(&files->dirs->loose->base, oid, NULL, 0, NULL))
return 0;
}
}
@@ -1787,7 +1787,7 @@ static int want_object_in_pack_mtime(const struct object_id *oid,
for (source = the_repository->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- struct multi_pack_index *m = get_multi_pack_index(files->packed);
+ struct multi_pack_index *m = get_multi_pack_index(files->dirs->packed);
struct pack_entry e;
if (m && midx_fill_entry(m, oid, &e, NULL) == MIDX_FILL_HIT) {
@@ -1800,11 +1800,11 @@ static int want_object_in_pack_mtime(const struct object_id *oid,
for (source = the_repository->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- for (e = files->packed->packs.head; e; e = e->next) {
+ for (e = files->dirs->packed->packs.head; e; e = e->next) {
struct packed_git *p = e->pack;
want = want_object_in_pack_one(p, oid, exclude, found_pack, found_offset, found_mtime);
if (!exclude && want > 0)
- packfile_list_prepend(&files->packed->packs, p);
+ packfile_list_prepend(&files->dirs->packed->packs, p);
if (want != -1)
return want;
}
@@ -4175,7 +4175,7 @@ static void add_cruft_object_entry(const struct object_id *oid, enum object_type
for (; !found && source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- if (!odb_source_read_object_info(&files->loose->base, oid, NULL, 0, NULL))
+ if (!odb_source_read_object_info(&files->dirs->loose->base, oid, NULL, 0, NULL))
found = 1;
}
@@ -4532,7 +4532,7 @@ static void add_objects_in_unpacked_packs(void)
if (!source->local)
continue;
- if (odb_source_for_each_object(&files->packed->base, &oi,
+ if (odb_source_for_each_object(&files->dirs->packed->base, &oi,
add_object_in_unpacked_pack, NULL, &opts))
die(_("cannot open pack index"));
}
@@ -4642,7 +4642,7 @@ static int force_object_loose(struct odb_source *source,
for (struct odb_source *s = source->odb->sources; s; s = s->next) {
struct odb_source_files *files = odb_source_files_downcast(s);
- if (!odb_source_read_object_info(&files->loose->base, oid, NULL, 0, NULL))
+ if (!odb_source_read_object_info(&files->dirs->loose->base, oid, NULL, 0, NULL))
return 0;
}
@@ -4664,7 +4664,7 @@ static int force_object_loose(struct odb_source *source,
compat_oid_p = &compat_oid;
}
- ret = odb_source_write_object(&files->loose->base, buf, len, type, oid,
+ ret = odb_source_write_object(&files->dirs->loose->base, buf, len, type, oid,
compat_oid_p, mtime, 0);
out:
diff --git a/builtin/repack.c b/builtin/repack.c
index c4360382c1..5d06872d77 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -617,7 +617,7 @@ int cmd_repack(int argc,
* midx_has_unknown_packs() will make the decision for
* us.
*/
- if (!get_multi_pack_index(files->packed))
+ if (!get_multi_pack_index(files->dirs->packed))
midx_must_contain_cruft = 1;
}
@@ -775,7 +775,7 @@ int cmd_repack(int argc,
if (git_env_bool(GIT_TEST_MULTI_PACK_INDEX_WRITE_INCREMENTAL, 0))
flags |= MIDX_WRITE_INCREMENTAL;
- write_midx_file(files->packed, NULL, NULL, flags);
+ write_midx_file(files->dirs->packed, NULL, NULL, flags);
}
cleanup:
diff --git a/commit-graph.c b/commit-graph.c
index 80ebc6a542..7cc486d140 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -2020,7 +2020,7 @@ static void fill_oids_from_all_packs(struct write_commit_graph_context *ctx)
for (source = ctx->r->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- odb_source_for_each_object(&files->packed->base, &oi, add_packed_commits_oi,
+ odb_source_for_each_object(&files->dirs->packed->base, &oi, add_packed_commits_oi,
ctx, &opts);
}
diff --git a/http-walker.c b/http-walker.c
index abafca84d6..0a6c99f471 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -542,7 +542,7 @@ static int fetch_object(struct walker *walker, const struct object_id *oid)
} else if (req->rename < 0) {
struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->sources);
struct strbuf buf = STRBUF_INIT;
- odb_loose_path(files->loose, &buf, &req->oid);
+ odb_loose_path(files->dirs->loose, &buf, &req->oid);
ret = error("unable to write sha1 filename %s", buf.buf);
strbuf_release(&buf);
}
diff --git a/http.c b/http.c
index c8fcfd7693..fe6ec88a21 100644
--- a/http.c
+++ b/http.c
@@ -2720,7 +2720,7 @@ void http_install_packfile(struct packed_git *p,
{
struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->sources);
packfile_list_remove(list_to_remove_from, p);
- packfile_store_add_pack(files->packed, p);
+ packfile_store_add_pack(files->dirs->packed, p);
}
struct http_pack_request *new_http_pack_request(
@@ -2861,7 +2861,7 @@ struct http_object_request *new_http_object_request(const char *base_url,
oidcpy(&freq->oid, oid);
freq->localfile = -1;
- odb_loose_path(files->loose, &filename, oid);
+ odb_loose_path(files->dirs->loose, &filename, oid);
strbuf_addf(&freq->tmpfile, "%s.temp", filename.buf);
strbuf_addf(&prevfile, "%s.prev", filename.buf);
@@ -3014,7 +3014,7 @@ int finish_http_object_request(struct http_object_request *freq)
unlink_or_warn(freq->tmpfile.buf);
return -1;
}
- odb_loose_path(files->loose, &filename, &freq->oid);
+ odb_loose_path(files->dirs->loose, &filename, &freq->oid);
freq->rename = finalize_object_file(the_repository, freq->tmpfile.buf, filename.buf);
strbuf_release(&filename);
diff --git a/loose.c b/loose.c
index c159d29d2d..957bf83e6b 100644
--- a/loose.c
+++ b/loose.c
@@ -117,7 +117,7 @@ int repo_read_loose_object_map(struct repository *repo)
for (source = repo->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- if (loose_object_map_load(files->loose) < 0)
+ if (loose_object_map_load(files->dirs->loose) < 0)
return -1;
}
@@ -127,7 +127,7 @@ int repo_read_loose_object_map(struct repository *repo)
int repo_write_loose_object_map(struct repository *repo)
{
struct odb_source_files *files = odb_source_files_downcast(repo->objects->sources);
- kh_oid_map_t *map = files->loose->map->to_compat;
+ kh_oid_map_t *map = files->dirs->loose->map->to_compat;
struct lock_file lock;
int fd;
khiter_t iter;
@@ -237,7 +237,7 @@ int repo_loose_object_map_oid(struct repository *repo,
for (source = repo->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- struct loose_object_map *loose_map = files->loose->map;
+ struct loose_object_map *loose_map = files->dirs->loose->map;
if (!loose_map)
continue;
map = (to == repo->compat_hash_algo) ?
diff --git a/midx.c b/midx.c
index 6d1c548e3d..c0f82c4163 100644
--- a/midx.c
+++ b/midx.c
@@ -837,20 +837,20 @@ void clear_midx_file(struct repository *r)
for (source = r->objects->sources; source; source = source->next) {
files = odb_source_files_downcast(source);
- if (files->packed->midx)
- close_midx(files->packed->midx);
- files->packed->midx = NULL;
+ if (files->dirs->packed->midx)
+ close_midx(files->dirs->packed->midx);
+ files->dirs->packed->midx = NULL;
}
}
files = odb_source_files_downcast(r->objects->sources);
- get_midx_filename(files->packed, &midx);
+ get_midx_filename(files->dirs->packed, &midx);
if (remove_path(midx.buf))
die(_("failed to clear multi-pack-index at %s"), midx.buf);
- clear_midx_files_ext(files->packed, MIDX_EXT_BITMAP, NULL);
- clear_midx_files_ext(files->packed, MIDX_EXT_REV, NULL);
+ clear_midx_files_ext(files->dirs->packed, MIDX_EXT_BITMAP, NULL);
+ clear_midx_files_ext(files->dirs->packed, MIDX_EXT_REV, NULL);
strbuf_release(&midx);
}
@@ -864,21 +864,21 @@ void clear_incremental_midx_files(struct repository *r,
for (source = r->objects->sources; source; source = source->next) {
files = odb_source_files_downcast(source);
- if (files->packed->midx)
- close_midx(files->packed->midx);
- files->packed->midx = NULL;
+ if (files->dirs->packed->midx)
+ close_midx(files->dirs->packed->midx);
+ files->dirs->packed->midx = NULL;
}
files = odb_source_files_downcast(r->objects->sources);
- get_midx_chain_filename(files->packed, &chain);
+ get_midx_chain_filename(files->dirs->packed, &chain);
if (!keep_hashes && remove_path(chain.buf))
die(_("failed to clear multi-pack-index chain at %s"),
chain.buf);
- clear_incremental_midx_files_ext(files->packed, MIDX_EXT_BITMAP, keep_hashes);
- clear_incremental_midx_files_ext(files->packed, MIDX_EXT_REV, keep_hashes);
- clear_incremental_midx_files_ext(files->packed, MIDX_EXT_MIDX, keep_hashes);
+ clear_incremental_midx_files_ext(files->dirs->packed, MIDX_EXT_BITMAP, keep_hashes);
+ clear_incremental_midx_files_ext(files->dirs->packed, MIDX_EXT_REV, keep_hashes);
+ clear_incremental_midx_files_ext(files->dirs->packed, MIDX_EXT_MIDX, keep_hashes);
strbuf_release(&chain);
}
diff --git a/odb/source-files.c b/odb/source-files.c
index f2fc4cd9ab..1f4cedaffa 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -24,6 +24,30 @@
#include "tree.h"
#include "write-or-die.h"
+struct odb_files_dir *odb_files_dir_new(struct object_database *odb,
+ const char *path, bool local)
+{
+ struct odb_files_dir *dir;
+
+ CALLOC_ARRAY(dir, 1);
+ dir->abspath = absolute_pathdup(path);
+ dir->local = local;
+ dir->loose = odb_source_loose_new(odb, path, local);
+ dir->packed = odb_source_packed_new(odb, path, local);
+
+ return dir;
+}
+
+void odb_files_dir_free(struct odb_files_dir *dir)
+{
+ if (!dir)
+ return;
+ odb_source_free(&dir->loose->base);
+ odb_source_free(&dir->packed->base);
+ free(dir->abspath);
+ free(dir);
+}
+
static void odb_source_files_reparent(const char *old_cwd,
const char *new_cwd,
void *cb_data)
@@ -31,6 +55,7 @@ static void odb_source_files_reparent(const char *old_cwd,
struct odb_source_files *files = cb_data;
char *path = reparent_relative_path(old_cwd, new_cwd,
files->base.path);
+
free(files->base.path);
files->base.path = path;
}
@@ -39,8 +64,7 @@ static void odb_source_files_free(struct odb_source *source)
{
struct odb_source_files *files = odb_source_files_downcast(source);
chdir_notify_unregister(odb_source_files_reparent, files);
- odb_source_free(&files->loose->base);
- odb_source_free(&files->packed->base);
+ odb_files_dir_free(files->dirs);
odb_source_release(&files->base);
free(files);
}
@@ -48,8 +72,8 @@ static void odb_source_files_free(struct odb_source *source)
static void odb_source_files_close(struct odb_source *source)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- odb_source_close(&files->loose->base);
- odb_source_close(&files->packed->base);
+ odb_source_close(&files->dirs->loose->base);
+ odb_source_close(&files->dirs->packed->base);
}
static int odb_source_files_create_on_disk(struct odb_source *source,
@@ -144,8 +168,8 @@ static void odb_source_files_prepare(struct odb_source *source,
enum odb_prepare_flags flags)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- odb_source_prepare(&files->loose->base, flags);
- odb_source_prepare(&files->packed->base, flags);
+ odb_source_prepare(&files->dirs->loose->base, flags);
+ odb_source_prepare(&files->dirs->packed->base, flags);
}
static enum odb_read_status odb_source_files_read_object_info(struct odb_source *source,
@@ -157,12 +181,12 @@ static enum odb_read_status odb_source_files_read_object_info(struct odb_source
struct odb_source_files *files = odb_source_files_downcast(source);
enum odb_read_status ret_packed, ret_loose;
- ret_packed = odb_source_read_object_info(&files->packed->base, oid, oi,
+ ret_packed = odb_source_read_object_info(&files->dirs->packed->base, oid, oi,
flags, errmsg);
if (!ret_packed)
return 0;
- ret_loose = odb_source_read_object_info(&files->loose->base, oid, oi, flags,
+ ret_loose = odb_source_read_object_info(&files->dirs->loose->base, oid, oi, flags,
ret_packed == ODB_READ_NOT_FOUND ? errmsg : NULL);
if (!ret_loose)
return 0;
@@ -184,8 +208,8 @@ static int odb_source_files_read_object_stream(struct odb_stream **out,
const struct object_id *oid)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- if (!odb_source_read_object_stream(out, &files->packed->base, oid) ||
- !odb_source_read_object_stream(out, &files->loose->base, oid))
+ if (!odb_source_read_object_stream(out, &files->dirs->packed->base, oid) ||
+ !odb_source_read_object_stream(out, &files->dirs->loose->base, oid))
return 0;
return -1;
}
@@ -200,12 +224,12 @@ static int odb_source_files_for_each_object(struct odb_source *source,
int ret;
if (!(opts->flags & ODB_FOR_EACH_OBJECT_PROMISOR_ONLY)) {
- ret = odb_source_for_each_object(&files->loose->base, request, cb, cb_data, opts);
+ ret = odb_source_for_each_object(&files->dirs->loose->base, request, cb, cb_data, opts);
if (ret)
return ret;
}
- ret = odb_source_for_each_object(&files->packed->base, request, cb, cb_data, opts);
+ ret = odb_source_for_each_object(&files->dirs->packed->base, request, cb, cb_data, opts);
if (ret)
return ret;
@@ -220,14 +244,14 @@ static int odb_source_files_count_objects(struct odb_source *source,
unsigned long count;
int ret;
- ret = odb_source_count_objects(&files->packed->base, flags, &count);
+ ret = odb_source_count_objects(&files->dirs->packed->base, flags, &count);
if (ret < 0)
goto out;
if (!(flags & ODB_COUNT_OBJECTS_APPROXIMATE)) {
unsigned long loose_count;
- ret = odb_source_count_objects(&files->loose->base, flags, &loose_count);
+ ret = odb_source_count_objects(&files->dirs->loose->base, flags, &loose_count);
if (ret < 0)
goto out;
@@ -250,11 +274,11 @@ static int odb_source_files_find_abbrev_len(struct odb_source *source,
unsigned len = min_len;
int ret;
- ret = odb_source_find_abbrev_len(&files->packed->base, oid, len, &len);
+ ret = odb_source_find_abbrev_len(&files->dirs->packed->base, oid, len, &len);
if (ret < 0)
goto out;
- ret = odb_source_find_abbrev_len(&files->loose->base, oid, len, &len);
+ ret = odb_source_find_abbrev_len(&files->dirs->loose->base, oid, len, &len);
if (ret < 0)
goto out;
@@ -270,8 +294,8 @@ static int odb_source_files_freshen_object(struct odb_source *source,
const time_t *mtime)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- if (odb_source_freshen_object(&files->packed->base, oid, mtime) ||
- odb_source_freshen_object(&files->loose->base, oid, mtime))
+ if (odb_source_freshen_object(&files->dirs->packed->base, oid, mtime) ||
+ odb_source_freshen_object(&files->dirs->loose->base, oid, mtime))
return 1;
return 0;
}
@@ -285,7 +309,7 @@ static int odb_source_files_write_object(struct odb_source *source,
enum odb_write_object_flags flags)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- return odb_source_write_object(&files->loose->base, buf, len, type,
+ return odb_source_write_object(&files->dirs->loose->base, buf, len, type,
oid, compat_oid, mtime, flags);
}
@@ -294,7 +318,7 @@ static int odb_source_files_write_object_stream(struct odb_source *source,
struct object_id *oid)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- return odb_source_write_object_stream(&files->loose->base, stream, oid);
+ return odb_source_write_object_stream(&files->dirs->loose->base, stream, oid);
}
static int odb_source_files_begin_transaction(struct odb_source *source,
@@ -330,7 +354,7 @@ static int too_many_loose_objects(struct odb_source_files *files, int limit)
if (limit <= 0)
return 0;
- if (odb_source_count_objects(&files->loose->base, ODB_COUNT_OBJECTS_APPROXIMATE,
+ if (odb_source_count_objects(&files->dirs->loose->base, ODB_COUNT_OBJECTS_APPROXIMATE,
&loose_count) < 0)
return 0;
@@ -349,7 +373,7 @@ static struct packed_git *find_base_packs(struct odb_source_files *files,
struct packfile_list_entry *e;
struct packed_git *base = NULL;
- for (e = packfile_store_get_packs(files->packed); e; e = e->next) {
+ for (e = packfile_store_get_packs(files->dirs->packed); e; e = e->next) {
if (e->pack->is_cruft)
continue;
if (limit) {
@@ -374,7 +398,7 @@ static int too_many_packs(struct odb_source_files *files, int gc_auto_pack_limit
if (gc_auto_pack_limit <= 0)
return 0;
- for (e = packfile_store_get_packs(files->packed); e; e = e->next) {
+ for (e = packfile_store_get_packs(files->dirs->packed); e; e = e->next) {
if (e->pack->pack_keep)
continue;
/*
@@ -937,8 +961,8 @@ static int odb_source_files_fsck(struct odb_source *source,
if (!(opts->flags & ODB_FSCK_FULL) && !source->local)
return 0;
- ret |= odb_source_fsck(&files->loose->base, opts);
- ret |= odb_source_fsck(&files->packed->base, opts);
+ ret |= odb_source_fsck(&files->dirs->loose->base, opts);
+ ret |= odb_source_fsck(&files->dirs->packed->base, opts);
return ret;
}
@@ -951,8 +975,7 @@ struct odb_source_files *odb_source_files_new(struct object_database *odb,
CALLOC_ARRAY(files, 1);
odb_source_init(&files->base, odb, ODB_SOURCE_FILES, path, local);
- files->loose = odb_source_loose_new(odb, path, local);
- files->packed = odb_source_packed_new(odb, path, local);
+ files->dirs = odb_files_dir_new(odb, path, local);
files->base.free = odb_source_files_free;
files->base.close = odb_source_files_close;
diff --git a/odb/source-files.h b/odb/source-files.h
index 9630b5f962..7f465853b1 100644
--- a/odb/source-files.h
+++ b/odb/source-files.h
@@ -6,14 +6,37 @@
struct odb_source_loose;
struct odb_source_packed;
+/*
+ * A single object directory that encapsulates access to both the loose and
+ * packed backend. This can either be the primary or an alternate object
+ * directory.
+ */
+struct odb_files_dir {
+ /* Absolute path to the object directory. */
+ char *abspath;
+
+ /* The two sources derived from this object directory. */
+ struct odb_source_loose *loose;
+ struct odb_source_packed *packed;
+
+ /*
+ * Whether this is the local object directory of the owning
+ * repository. Directories added via alternates are not local.
+ */
+ bool local;
+};
+
+struct odb_files_dir *odb_files_dir_new(struct object_database *odb,
+ const char *path, bool local);
+void odb_files_dir_free(struct odb_files_dir *dir);
+
/*
* The files object database source uses a combination of loose objects and
* packfiles. It is the default backend used by Git to store objects.
*/
struct odb_source_files {
struct odb_source base;
- struct odb_source_loose *loose;
- struct odb_source_packed *packed;
+ struct odb_files_dir *dirs;
};
/* Allocate and initialize a new object source. */
diff --git a/pack-bitmap.c b/pack-bitmap.c
index 3de8e9590c..52556b4543 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -720,7 +720,7 @@ static int open_bitmap(struct repository *r,
for (source = r->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- if (!open_bitmap_for_source(files->packed, bitmap_git))
+ if (!open_bitmap_for_source(files->dirs->packed, bitmap_git))
found = true;
/*
diff --git a/packfile.c b/packfile.c
index 4fa5fd67c8..93b69d7f50 100644
--- a/packfile.c
+++ b/packfile.c
@@ -280,7 +280,7 @@ static int unuse_one_window(struct object_database *odb)
for (source = odb->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- for (e = files->packed->packs.head; e; e = e->next)
+ for (e = files->dirs->packed->packs.head; e; e = e->next)
scan_windows(e->pack, &lru_p, &lru_w, &lru_l);
}
@@ -458,7 +458,7 @@ static int close_one_pack(struct repository *r)
for (source = r->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- for (e = files->packed->packs.head; e; e = e->next) {
+ for (e = files->dirs->packed->packs.head; e; e = e->next) {
if (e->pack->pack_fd == -1)
continue;
find_lru_pack(e->pack, &lru_p, &mru_w, &accept_windows_inuse);
@@ -1925,7 +1925,7 @@ int has_object_pack(struct repository *r, const struct object_id *oid)
for (source = r->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- if (!odb_source_read_object_info(&files->packed->base, oid, NULL, 0, NULL))
+ if (!odb_source_read_object_info(&files->dirs->packed->base, oid, NULL, 0, NULL))
return 1;
}
@@ -1942,7 +1942,7 @@ int has_object_kept_pack(struct repository *r, const struct object_id *oid,
struct odb_source_files *files = odb_source_files_downcast(source);
struct packed_git **cache;
- cache = packfile_store_get_kept_pack_cache(files->packed, flags);
+ cache = packfile_store_get_kept_pack_cache(files->dirs->packed, flags);
for (; *cache; cache++) {
struct packed_git *p = *cache;
diff --git a/packfile.h b/packfile.h
index 6d30d15a00..fd1cf0ad6c 100644
--- a/packfile.h
+++ b/packfile.h
@@ -79,7 +79,7 @@ static inline struct repo_for_each_pack_data repo_for_eack_pack_data_init(struct
for (struct odb_source *source = repo->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- struct packfile_list_entry *entry = packfile_store_get_packs(files->packed);
+ struct packfile_list_entry *entry = packfile_store_get_packs(files->dirs->packed);
if (!entry)
continue;
data.source = source;
@@ -100,7 +100,7 @@ static inline void repo_for_each_pack_data_next(struct repo_for_each_pack_data *
for (source = data->source->next; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- struct packfile_list_entry *entry = packfile_store_get_packs(files->packed);
+ struct packfile_list_entry *entry = packfile_store_get_packs(files->dirs->packed);
if (!entry)
continue;
data->source = source;
diff --git a/repack-geometry.c b/repack-geometry.c
index 15b3412950..b541c34af5 100644
--- a/repack-geometry.c
+++ b/repack-geometry.c
@@ -33,7 +33,7 @@ void pack_geometry_init(struct pack_geometry *geometry,
struct packed_git *p;
struct strbuf buf = STRBUF_INIT;
struct odb_source_files *files = odb_source_files_downcast(existing->source);
- struct multi_pack_index *m = get_multi_pack_index(files->packed);
+ struct multi_pack_index *m = get_multi_pack_index(files->dirs->packed);
repo_for_each_pack(existing->repo, p) {
if (geometry->midx_layer_threshold_set && m &&
diff --git a/repack-midx.c b/repack-midx.c
index 64c7f8d0f4..25140569bd 100644
--- a/repack-midx.c
+++ b/repack-midx.c
@@ -564,7 +564,7 @@ static void repack_make_midx_append_plan(struct repack_write_midx_opts *opts,
size_t steps_nr = 0, steps_alloc = 0;
odb_reprepare(opts->existing->repo->objects);
- m = get_multi_pack_index(files->packed);
+ m = get_multi_pack_index(files->dirs->packed);
if (opts->names->nr) {
struct strbuf buf = STRBUF_INIT;
@@ -620,7 +620,7 @@ static int repack_make_midx_compaction_plan(struct repack_write_midx_opts *opts,
opts->existing->repo);
odb_reprepare(opts->existing->repo->objects);
- m = get_multi_pack_index(files->packed);
+ m = get_multi_pack_index(files->dirs->packed);
for (i = 0; m && i < m->num_packs + m->num_packs_in_base; i++) {
if (prepare_midx_pack(m, i)) {
@@ -949,7 +949,7 @@ static int write_midx_incremental(struct repack_write_midx_opts *opts)
size_t i;
int ret = 0;
- get_midx_chain_filename(files->packed, &lock_name);
+ get_midx_chain_filename(files->dirs->packed, &lock_name);
if (safe_create_leading_directories(opts->existing->repo,
lock_name.buf))
die_errno(_("unable to create leading directories of %s"),
diff --git a/repack.c b/repack.c
index d2aa58e134..e20431690c 100644
--- a/repack.c
+++ b/repack.c
@@ -60,7 +60,7 @@ void repack_remove_redundant_pack(struct repository *repo, const char *dir_name,
{
struct strbuf buf = STRBUF_INIT;
struct odb_source_files *files = odb_source_files_downcast(repo->objects->sources);
- struct multi_pack_index *m = get_multi_pack_index(files->packed);
+ struct multi_pack_index *m = get_multi_pack_index(files->dirs->packed);
strbuf_addf(&buf, "%s.pack", base_name);
if (m && files->base.local && midx_contains_pack(m, buf.buf)) {
clear_midx_file(repo);
diff --git a/t/helper/test-read-midx.c b/t/helper/test-read-midx.c
index 83b07c6236..3f4bafff61 100644
--- a/t/helper/test-read-midx.c
+++ b/t/helper/test-read-midx.c
@@ -22,7 +22,7 @@ static struct multi_pack_index *setup_midx(const char *object_dir,
source = odb_find_source(the_repository->objects, object_dir);
if (source) {
- packed = odb_source_files_downcast(source)->packed;
+ packed = odb_source_files_downcast(source)->dirs->packed;
} else {
packed = odb_source_packed_new(the_repository->objects,
object_dir, false);
--
2.56.0.406.ga2d225a756.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* [PATCH v2 04/13] odb: refactor `odb_for_each_alternate()` to yield dirs
2026-10-08 8:35 ` [PATCH v2 " Patrick Steinhardt
` (2 preceding siblings ...)
2026-10-08 8:35 ` [PATCH v2 03/13] odb/source-files: introduce `struct odb_files_dir` Patrick Steinhardt
@ 2026-10-08 8:35 ` Patrick Steinhardt
2026-10-08 8:35 ` [PATCH v2 05/13] odb: refactor `odb_find_source()` " Patrick Steinhardt
` (9 subsequent siblings)
13 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-08 8:35 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak
The function `odb_for_each_alternate()` iterates through all alternates
of an object database. This is becoming an implementation detail of the
"files" backend though, where each alternate will be represented by one
`struct odb_files_dir`.
Adapt `odb_for_each_alternate()` to already iterate through these
structs instead of iterating through sources.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/count-objects.c | 4 ++--
builtin/submodule--helper.c | 7 ++++---
diagnose.c | 8 ++++----
odb.c | 18 +++++++++++-------
odb.h | 3 ++-
5 files changed, 23 insertions(+), 17 deletions(-)
diff --git a/builtin/count-objects.c b/builtin/count-objects.c
index 18f6e33b6f..f2abfaccec 100644
--- a/builtin/count-objects.c
+++ b/builtin/count-objects.c
@@ -81,10 +81,10 @@ static int count_cruft(const char *basename UNUSED, const char *path,
return 0;
}
-static int print_alternate(struct odb_source *alternate, void *data UNUSED)
+static int print_alternate(struct odb_files_dir *alternate, void *data UNUSED)
{
printf("alternate: ");
- quote_c_style(alternate->path, NULL, stdout, 0);
+ quote_c_style(alternate->abspath, NULL, stdout, 0);
putchar('\n');
return 0;
}
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 40a052d674..64412adf9f 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -30,6 +30,7 @@
#include "object-name.h"
#include "odb.h"
#include "odb/source.h"
+#include "odb/source-files.h"
#include "advice.h"
#include "branch.h"
#include "list-objects-filter-options.h"
@@ -1770,7 +1771,7 @@ static const char alternate_error_advice[] = N_(
);
static int add_possible_reference_from_superproject(
- struct odb_source *alt_odb, void *sas_cb)
+ struct odb_files_dir *alt_odb, void *sas_cb)
{
struct submodule_alternate_setup *sas = sas_cb;
size_t len;
@@ -1779,12 +1780,12 @@ static int add_possible_reference_from_superproject(
* If the alternate object store is another repository, try the
* standard layout with .git/(modules/<name>)+/objects
*/
- if (strip_suffix(alt_odb->path, "/objects", &len)) {
+ if (strip_suffix(alt_odb->abspath, "/objects", &len)) {
struct repository alternate;
char *sm_alternate;
struct strbuf sb = STRBUF_INIT;
struct strbuf err = STRBUF_INIT;
- strbuf_add(&sb, alt_odb->path, len);
+ strbuf_add(&sb, alt_odb->abspath, len);
if (repo_init(&alternate, sb.buf, NULL) < 0)
die(_("could not get a repository handle for gitdir '%s'"),
diff --git a/diagnose.c b/diagnose.c
index 5092bf80d3..89240e47d6 100644
--- a/diagnose.c
+++ b/diagnose.c
@@ -59,13 +59,13 @@ static void dir_file_stats_objects(const char *full_path,
(uintmax_t)st.st_size);
}
-static int dir_file_stats(struct odb_source *source, void *data)
+static int dir_file_stats(struct odb_files_dir *source, void *data)
{
struct strbuf *buf = data;
- strbuf_addf(buf, "Contents of %s:\n", source->path);
+ strbuf_addf(buf, "Contents of %s:\n", source->abspath);
- for_each_file_in_pack_dir(source->path, dir_file_stats_objects,
+ for_each_file_in_pack_dir(source->abspath, dir_file_stats_objects,
data);
return 0;
@@ -228,7 +228,7 @@ int create_diagnostics_archive(struct repository *r,
strbuf_reset(&buf);
strbuf_addstr(&buf, "--add-virtual-file=packs-local.txt:");
- dir_file_stats(r->objects->sources, &buf);
+ dir_file_stats(odb_source_files_downcast(r->objects->sources)->dirs, &buf);
odb_for_each_alternate(r->objects, dir_file_stats, &buf);
strvec_push(&archiver_args, buf.buf);
diff --git a/odb.c b/odb.c
index 0200e26f21..9b70859c23 100644
--- a/odb.c
+++ b/odb.c
@@ -14,6 +14,7 @@
#include "object-name.h"
#include "odb.h"
#include "odb/source-inmemory.h"
+#include "odb/source-files.h"
#include "path.h"
#include "promisor-remote.h"
#include "quote.h"
@@ -435,18 +436,19 @@ static void read_alternate_refs(struct repository *repo,
}
struct alternate_refs_data {
+ struct repository *repo;
odb_for_each_alternate_ref_fn *fn;
void *payload;
};
-static int refs_from_alternate_cb(struct odb_source *alternate,
+static int refs_from_alternate_cb(struct odb_files_dir *alternate,
void *payload)
{
struct strbuf path = STRBUF_INIT;
size_t base_len;
struct alternate_refs_data *cb = payload;
- if (!strbuf_realpath(&path, alternate->path, 0))
+ if (!strbuf_realpath(&path, alternate->abspath, 0))
goto out;
if (!strbuf_strip_suffix(&path, "/objects"))
goto out;
@@ -458,7 +460,7 @@ static int refs_from_alternate_cb(struct odb_source *alternate,
goto out;
strbuf_setlen(&path, base_len);
- read_alternate_refs(alternate->odb->repo, path.buf, cb->fn, cb->payload);
+ read_alternate_refs(cb->repo, path.buf, cb->fn, cb->payload);
out:
strbuf_release(&path);
@@ -468,9 +470,11 @@ static int refs_from_alternate_cb(struct odb_source *alternate,
void odb_for_each_alternate_ref(struct object_database *odb,
odb_for_each_alternate_ref_fn cb, void *payload)
{
- struct alternate_refs_data data;
- data.fn = cb;
- data.payload = payload;
+ struct alternate_refs_data data = {
+ .fn = cb,
+ .payload = payload,
+ .repo = odb->repo,
+ };
odb_for_each_alternate(odb, refs_from_alternate_cb, &data);
}
@@ -481,7 +485,7 @@ int odb_for_each_alternate(struct object_database *odb,
int r = 0;
for (alternate = odb->sources->next; alternate; alternate = alternate->next) {
- r = cb(alternate, payload);
+ r = cb(odb_source_files_downcast(alternate)->dirs, payload);
if (r)
break;
}
diff --git a/odb.h b/odb.h
index 797eecbb94..3715351bb3 100644
--- a/odb.h
+++ b/odb.h
@@ -12,6 +12,7 @@
struct cached_object_entry;
struct list_objects_filter_options;
struct odb_source_inmemory;
+struct odb_files_dir;
struct packed_git;
struct repository;
struct strbuf;
@@ -257,7 +258,7 @@ void odb_restore_primary_source(struct object_database *odb,
* function returns a non-zero value, in which case the value is bubbled up
* from the callback.
*/
-typedef int odb_for_each_alternate_fn(struct odb_source *, void *);
+typedef int odb_for_each_alternate_fn(struct odb_files_dir *, void *);
int odb_for_each_alternate(struct object_database *odb,
odb_for_each_alternate_fn cb, void *payload);
--
2.56.0.406.ga2d225a756.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* [PATCH v2 05/13] odb: refactor `odb_find_source()` to yield dirs
2026-10-08 8:35 ` [PATCH v2 " Patrick Steinhardt
` (3 preceding siblings ...)
2026-10-08 8:35 ` [PATCH v2 04/13] odb: refactor `odb_for_each_alternate()` to yield dirs Patrick Steinhardt
@ 2026-10-08 8:35 ` Patrick Steinhardt
2026-10-08 8:35 ` [PATCH v2 06/13] odb/source-files: add the ability to have multiple object dirs Patrick Steinhardt
` (8 subsequent siblings)
13 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-08 8:35 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak
Similar as in the preceding commit, `odb_find_source()` can be used to
yield a single source by its object directory. This function is also
specific to the "files" backend once alternates are an implementation
detail thereof.
Refactor it to be specific to the "files" backend and return an `struct
odb_files_dir` to prepare for moving alternates into the "files" source.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/commit-graph.c | 26 ++++++++++++++++----------
builtin/multi-pack-index.c | 42 +++++++++++++++++++++---------------------
odb.c | 26 --------------------------
odb.h | 9 ---------
odb/source-files.c | 26 ++++++++++++++++++++++++++
odb/source-files.h | 6 ++++++
t/helper/test-read-midx.c | 8 ++++----
7 files changed, 73 insertions(+), 70 deletions(-)
diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c
index a986f08a94..ff244c866f 100644
--- a/builtin/commit-graph.c
+++ b/builtin/commit-graph.c
@@ -9,6 +9,7 @@
#include "commit-graph.h"
#include "odb.h"
#include "odb/source.h"
+#include "odb/source-files.h"
#include "progress.h"
#include "replace-object.h"
#include "strbuf.h"
@@ -68,7 +69,7 @@ static int graph_verify(int argc, const char **argv, const char *prefix,
struct repository *repo UNUSED)
{
struct commit_graph *graph = NULL;
- struct odb_source *source = NULL;
+ struct odb_files_dir *dir = NULL;
char *graph_name;
char *chain_name;
enum { OPENED_NONE, OPENED_GRAPH, OPENED_CHAIN } opened = OPENED_NONE;
@@ -103,9 +104,12 @@ static int graph_verify(int argc, const char **argv, const char *prefix,
if (opts.progress)
flags |= COMMIT_GRAPH_WRITE_PROGRESS;
- source = odb_find_source_or_die(the_repository->objects, opts.obj_dir);
- graph_name = get_commit_graph_filename(source->path);
- chain_name = get_commit_graph_chain_filename(source->path);
+ dir = odb_source_files_find_dir(the_repository->objects, opts.obj_dir);
+ if (!dir)
+ die(_("could not find object directory matching %s"), opts.obj_dir);
+
+ graph_name = get_commit_graph_filename(dir->abspath);
+ chain_name = get_commit_graph_chain_filename(dir->abspath);
if (open_commit_graph(graph_name, &fd, &st))
opened = OPENED_GRAPH;
else if (errno != ENOENT)
@@ -123,7 +127,7 @@ static int graph_verify(int argc, const char **argv, const char *prefix,
if (opened == OPENED_NONE)
return 0;
else if (opened == OPENED_GRAPH)
- graph = load_commit_graph_one_fd_st(the_repository, source->path, fd, &st);
+ graph = load_commit_graph_one_fd_st(the_repository, dir->abspath, fd, &st);
else
graph = load_commit_graph_chain_fd_st(the_repository->objects, fd, &st,
&incomplete_chain);
@@ -226,7 +230,7 @@ static int graph_write(int argc, const char **argv, const char *prefix,
struct string_list pack_indexes = STRING_LIST_INIT_DUP;
struct strbuf buf = STRBUF_INIT;
struct oidset commits = OIDSET_INIT;
- struct odb_source *source = NULL;
+ struct odb_files_dir *dir = NULL;
int result = 0;
enum commit_graph_write_flags flags = 0;
struct progress *progress = NULL;
@@ -294,10 +298,12 @@ static int graph_write(int argc, const char **argv, const char *prefix,
git_env_bool(GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS, 0))
flags |= COMMIT_GRAPH_WRITE_BLOOM_FILTERS;
- source = odb_find_source_or_die(the_repository->objects, opts.obj_dir);
+ dir = odb_source_files_find_dir(the_repository->objects, opts.obj_dir);
+ if (!dir)
+ die(_("could not find object directory matching %s"), opts.obj_dir);
if (opts.reachable) {
- if (write_commit_graph_reachable(the_repository, source->path, flags, &write_opts))
+ if (write_commit_graph_reachable(the_repository, dir->abspath, flags, &write_opts))
result = 1;
goto cleanup;
}
@@ -306,7 +312,7 @@ static int graph_write(int argc, const char **argv, const char *prefix,
struct strbuf packname = STRBUF_INIT;
size_t dirlen;
- strbuf_addf(&packname, "%s/pack/", source->path);
+ strbuf_addf(&packname, "%s/pack/", dir->abspath);
dirlen = packname.len;
while (strbuf_getline(&buf, stdin) != EOF) {
@@ -334,7 +340,7 @@ static int graph_write(int argc, const char **argv, const char *prefix,
stop_progress(&progress);
}
- if (write_commit_graph(the_repository, source->path,
+ if (write_commit_graph(the_repository, dir->abspath,
opts.stdin_packs ? &pack_indexes : NULL,
opts.stdin_commits ? &commits : NULL,
flags,
diff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c
index a170ec80b9..fc8b494996 100644
--- a/builtin/multi-pack-index.c
+++ b/builtin/multi-pack-index.c
@@ -86,13 +86,13 @@ static int parse_object_dir(const struct option *opt, const char *arg,
return 0;
}
-static struct odb_source_files *handle_object_dir_option(struct repository *repo)
+static struct odb_source_packed *handle_object_dir_option(struct repository *repo)
{
- struct odb_source *source = odb_find_source(repo->objects, opts.object_dir);
- if (!source)
+ struct odb_files_dir *dir = odb_source_files_find_dir(repo->objects, opts.object_dir);
+ if (!dir)
die(_("object directory is not an alternate of the current repository: '%s'"),
opts.object_dir);
- return odb_source_files_downcast(source);
+ return dir->packed;
}
static struct option common_opts[] = {
@@ -169,7 +169,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,
N_("refs snapshot for selecting bitmap commits")),
OPT_END(),
};
- struct odb_source_files *source;
+ struct odb_source_packed *packed_source;
int ret;
opts.flags |= MIDX_WRITE_BITMAP_HASH_CACHE;
@@ -204,7 +204,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,
options);
}
- source = handle_object_dir_option(repo);
+ packed_source = handle_object_dir_option(repo);
FREE_AND_NULL(options);
@@ -213,7 +213,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,
read_packs_from_stdin(&packs);
- ret = write_midx_file_only(source->dirs->packed, &packs,
+ ret = write_midx_file_only(packed_source, &packs,
opts.preferred_pack,
opts.refs_snapshot,
opts.incremental_base, opts.flags);
@@ -225,7 +225,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,
}
- ret = write_midx_file(source->dirs->packed, opts.preferred_pack,
+ ret = write_midx_file(packed_source, opts.preferred_pack,
opts.refs_snapshot, opts.flags);
free(opts.refs_snapshot);
@@ -239,7 +239,7 @@ static int cmd_multi_pack_index_compact(int argc, const char **argv,
struct multi_pack_index *m, *cur;
struct multi_pack_index *from_midx = NULL;
struct multi_pack_index *to_midx = NULL;
- struct odb_source_files *source;
+ struct odb_source_packed *packed_source;
int ret;
struct option *options;
@@ -280,11 +280,11 @@ static int cmd_multi_pack_index_compact(int argc, const char **argv,
options);
}
- source = handle_object_dir_option(the_repository);
+ packed_source = handle_object_dir_option(the_repository);
FREE_AND_NULL(options);
- m = get_multi_pack_index(source->dirs->packed);
+ m = get_multi_pack_index(packed_source);
for (cur = m; cur && !(from_midx && to_midx); cur = cur->base_midx) {
const char *midx_csum = midx_get_checksum_hex(cur);
@@ -307,7 +307,7 @@ static int cmd_multi_pack_index_compact(int argc, const char **argv,
die(_("MIDX %s must be an ancestor of %s"), argv[0], argv[1]);
}
- ret = write_midx_file_compact(source->dirs->packed, from_midx, to_midx,
+ ret = write_midx_file_compact(packed_source, from_midx, to_midx,
opts.incremental_base, opts.flags);
return ret;
@@ -321,7 +321,7 @@ static int cmd_multi_pack_index_verify(int argc, const char **argv,
static struct option builtin_multi_pack_index_verify_options[] = {
OPT_END(),
};
- struct odb_source_files *source;
+ struct odb_source_packed *packed_source;
options = add_common_options(builtin_multi_pack_index_verify_options);
@@ -335,11 +335,11 @@ static int cmd_multi_pack_index_verify(int argc, const char **argv,
if (argc)
usage_with_options(builtin_multi_pack_index_verify_usage,
options);
- source = handle_object_dir_option(the_repository);
+ packed_source = handle_object_dir_option(the_repository);
FREE_AND_NULL(options);
- return verify_midx_file(source->dirs->packed, opts.flags);
+ return verify_midx_file(packed_source, opts.flags);
}
static int cmd_multi_pack_index_expire(int argc, const char **argv,
@@ -350,7 +350,7 @@ static int cmd_multi_pack_index_expire(int argc, const char **argv,
static struct option builtin_multi_pack_index_expire_options[] = {
OPT_END(),
};
- struct odb_source_files *source;
+ struct odb_source_packed *packed_source;
options = add_common_options(builtin_multi_pack_index_expire_options);
@@ -364,11 +364,11 @@ static int cmd_multi_pack_index_expire(int argc, const char **argv,
if (argc)
usage_with_options(builtin_multi_pack_index_expire_usage,
options);
- source = handle_object_dir_option(the_repository);
+ packed_source = handle_object_dir_option(the_repository);
FREE_AND_NULL(options);
- return expire_midx_packs(source->dirs->packed, opts.flags);
+ return expire_midx_packs(packed_source, opts.flags);
}
static int cmd_multi_pack_index_repack(int argc, const char **argv,
@@ -381,7 +381,7 @@ static int cmd_multi_pack_index_repack(int argc, const char **argv,
N_("during repack, collect pack-files of smaller size into a batch that is larger than this size")),
OPT_END(),
};
- struct odb_source_files *source;
+ struct odb_source_packed *packed_source;
options = add_common_options(builtin_multi_pack_index_repack_options);
@@ -396,11 +396,11 @@ static int cmd_multi_pack_index_repack(int argc, const char **argv,
if (argc)
usage_with_options(builtin_multi_pack_index_repack_usage,
options);
- source = handle_object_dir_option(the_repository);
+ packed_source = handle_object_dir_option(the_repository);
FREE_AND_NULL(options);
- return midx_repack(source->dirs->packed, (size_t)opts.batch_size, opts.flags);
+ return midx_repack(packed_source, (size_t)opts.batch_size, opts.flags);
}
int cmd_multi_pack_index(int argc,
diff --git a/odb.c b/odb.c
index 9b70859c23..1dc8647159 100644
--- a/odb.c
+++ b/odb.c
@@ -348,32 +348,6 @@ char *compute_alternate_path(const char *path, struct strbuf *err)
return ref_git;
}
-struct odb_source *odb_find_source(struct object_database *odb, const char *obj_dir)
-{
- struct odb_source *source;
- char *obj_dir_real = real_pathdup(obj_dir, 1);
- struct strbuf odb_path_real = STRBUF_INIT;
-
- for (source = odb->sources; source; source = source->next) {
- strbuf_realpath(&odb_path_real, source->path, 1);
- if (!strcmp(obj_dir_real, odb_path_real.buf))
- break;
- }
-
- free(obj_dir_real);
- strbuf_release(&odb_path_real);
-
- return source;
-}
-
-struct odb_source *odb_find_source_or_die(struct object_database *odb, const char *obj_dir)
-{
- struct odb_source *source = odb_find_source(odb, obj_dir);
- if (!source)
- die(_("could not find object directory matching %s"), obj_dir);
- return source;
-}
-
static void fill_alternate_refs_command(struct repository *repo,
struct child_process *cmd,
const char *repo_path)
diff --git a/odb.h b/odb.h
index 3715351bb3..5c86572b5d 100644
--- a/odb.h
+++ b/odb.h
@@ -226,15 +226,6 @@ struct odb_fsck_options {
*/
int odb_fsck(struct object_database *odb, struct odb_fsck_options *opts);
-/*
- * Find source by its object directory path. Returns a `NULL` pointer in case
- * the source could not be found.
- */
-struct odb_source *odb_find_source(struct object_database *odb, const char *obj_dir);
-
-/* Same as `odb_find_source()`, but dies in case the source doesn't exist. */
-struct odb_source *odb_find_source_or_die(struct object_database *odb, const char *obj_dir);
-
/*
* Replace the current writable object directory with the specified temporary
* object directory and return the newly installed primary source. The former
diff --git a/odb/source-files.c b/odb/source-files.c
index 1f4cedaffa..e5e43b1543 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -967,6 +967,32 @@ static int odb_source_files_fsck(struct odb_source *source,
return ret;
}
+struct odb_files_dir *odb_source_files_find_dir(struct object_database *odb, const char *obj_dir)
+{
+ char *obj_dir_real = real_pathdup(obj_dir, 1);
+ struct strbuf odb_path_real = STRBUF_INIT;
+ struct odb_files_dir *dir = NULL;
+ struct odb_source *source;
+
+ for (source = odb->sources; source; source = source->next) {
+ struct odb_source_files *files;
+
+ if (source->type != ODB_SOURCE_FILES)
+ continue;
+ files = odb_source_files_downcast(source);
+
+ strbuf_realpath(&odb_path_real, files->dirs->abspath, 1);
+ if (!strcmp(obj_dir_real, odb_path_real.buf)) {
+ dir = files->dirs;
+ break;
+ }
+ }
+
+ free(obj_dir_real);
+ strbuf_release(&odb_path_real);
+ return dir;
+}
+
struct odb_source_files *odb_source_files_new(struct object_database *odb,
const char *path,
bool local)
diff --git a/odb/source-files.h b/odb/source-files.h
index 7f465853b1..77f4d842e0 100644
--- a/odb/source-files.h
+++ b/odb/source-files.h
@@ -72,4 +72,10 @@ static inline struct odb_source_files *odb_source_files_downcast(struct odb_sour
return container_of(source, struct odb_source_files, base);
}
+/*
+ * Find "files" directory by its object directory path. Returns a `NULL`
+ * pointer in case the object directory could not be found.
+ */
+struct odb_files_dir *odb_source_files_find_dir(struct object_database *odb, const char *obj_dir);
+
#endif
diff --git a/t/helper/test-read-midx.c b/t/helper/test-read-midx.c
index 3f4bafff61..88a79fdfc8 100644
--- a/t/helper/test-read-midx.c
+++ b/t/helper/test-read-midx.c
@@ -16,13 +16,13 @@ static struct multi_pack_index *setup_midx(const char *object_dir,
struct odb_source_packed **out)
{
struct odb_source_packed *packed;
- struct odb_source *source;
+ struct odb_files_dir *dir;
setup_git_directory(the_repository);
- source = odb_find_source(the_repository->objects, object_dir);
- if (source) {
- packed = odb_source_files_downcast(source)->dirs->packed;
+ dir = odb_source_files_find_dir(the_repository->objects, object_dir);
+ if (dir) {
+ packed = dir->packed;
} else {
packed = odb_source_packed_new(the_repository->objects,
object_dir, false);
--
2.56.0.406.ga2d225a756.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* [PATCH v2 06/13] odb/source-files: add the ability to have multiple object dirs
2026-10-08 8:35 ` [PATCH v2 " Patrick Steinhardt
` (4 preceding siblings ...)
2026-10-08 8:35 ` [PATCH v2 05/13] odb: refactor `odb_find_source()` " Patrick Steinhardt
@ 2026-10-08 8:35 ` Patrick Steinhardt
2026-10-08 8:35 ` [PATCH v2 07/13] tmp-objdir: absorb logic to set and restore primary sources Patrick Steinhardt
` (7 subsequent siblings)
13 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-08 8:35 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak
In a subsequent commit we'll migrate handling of alternates into the
"files" backend. As part of that, the backend will need to grow the
ability to handle multiple object directories for alternates and for
transactions.
Introduce a `struct odb_files_dir::next` pointer so that we can have
multiple directories. Adapt the backend to loop through this list as
necessary. In general:
- For reading paths we will loop through all object directories until
we have found the object.
- For writing paths we will typically write to the first of our object
directories.
This mechanism isn't used yet as we still only ever have a single object
directory at the current point in time. But that will change over
subsequent commits.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
odb/source-files.c | 147 +++++++++++++++++++++++++++++++++--------------------
odb/source-files.h | 9 ++++
2 files changed, 100 insertions(+), 56 deletions(-)
diff --git a/odb/source-files.c b/odb/source-files.c
index e5e43b1543..9389546b3e 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -63,8 +63,15 @@ static void odb_source_files_reparent(const char *old_cwd,
static void odb_source_files_free(struct odb_source *source)
{
struct odb_source_files *files = odb_source_files_downcast(source);
+
chdir_notify_unregister(odb_source_files_reparent, files);
- odb_files_dir_free(files->dirs);
+
+ while (files->dirs) {
+ struct odb_files_dir *next = files->dirs->next;
+ odb_files_dir_free(files->dirs);
+ files->dirs = next;
+ }
+
odb_source_release(&files->base);
free(files);
}
@@ -72,8 +79,11 @@ static void odb_source_files_free(struct odb_source *source)
static void odb_source_files_close(struct odb_source *source)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- odb_source_close(&files->dirs->loose->base);
- odb_source_close(&files->dirs->packed->base);
+
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ odb_source_close(&dir->loose->base);
+ odb_source_close(&dir->packed->base);
+ }
}
static int odb_source_files_create_on_disk(struct odb_source *source,
@@ -168,8 +178,11 @@ static void odb_source_files_prepare(struct odb_source *source,
enum odb_prepare_flags flags)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- odb_source_prepare(&files->dirs->loose->base, flags);
- odb_source_prepare(&files->dirs->packed->base, flags);
+
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ odb_source_prepare(&dir->loose->base, flags);
+ odb_source_prepare(&dir->packed->base, flags);
+ }
}
static enum odb_read_status odb_source_files_read_object_info(struct odb_source *source,
@@ -179,28 +192,33 @@ static enum odb_read_status odb_source_files_read_object_info(struct odb_source
struct strbuf *errmsg)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- enum odb_read_status ret_packed, ret_loose;
-
- ret_packed = odb_source_read_object_info(&files->dirs->packed->base, oid, oi,
- flags, errmsg);
- if (!ret_packed)
- return 0;
-
- ret_loose = odb_source_read_object_info(&files->dirs->loose->base, oid, oi, flags,
- ret_packed == ODB_READ_NOT_FOUND ? errmsg : NULL);
- if (!ret_loose)
- return 0;
+ enum odb_read_status status = ODB_READ_NOT_FOUND;
/*
- * Reading the packed object may have failed even though the object
- * exists, for example because it is corrupt. Report this failure to
- * the caller in case neither of the sources was able to read the
- * object, and prefer the error of the packed source in case both
- * reads have failed.
+ * Reading an object may fail even though the object exists, for
+ * example because it is corrupt. Report this failure to the caller in
+ * case none of the directories was able to read the object, and
+ * prefer the first such error in case multiple reads have failed.
*/
- if (ret_packed != ODB_READ_NOT_FOUND)
- return ret_packed;
- return ret_loose;
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ enum odb_read_status ret;
+
+ ret = odb_source_read_object_info(&dir->packed->base, oid, oi, flags,
+ status == ODB_READ_NOT_FOUND ? errmsg : NULL);
+ if (!ret)
+ return 0;
+ if (ret != ODB_READ_NOT_FOUND && status == ODB_READ_NOT_FOUND)
+ status = ret;
+
+ ret = odb_source_read_object_info(&dir->loose->base, oid, oi, flags,
+ status == ODB_READ_NOT_FOUND ? errmsg : NULL);
+ if (!ret)
+ return 0;
+ if (ret != ODB_READ_NOT_FOUND && status == ODB_READ_NOT_FOUND)
+ status = ret;
+ }
+
+ return status;
}
static int odb_source_files_read_object_stream(struct odb_stream **out,
@@ -208,9 +226,12 @@ static int odb_source_files_read_object_stream(struct odb_stream **out,
const struct object_id *oid)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- if (!odb_source_read_object_stream(out, &files->dirs->packed->base, oid) ||
- !odb_source_read_object_stream(out, &files->dirs->loose->base, oid))
- return 0;
+
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next)
+ if (!odb_source_read_object_stream(out, &dir->packed->base, oid) ||
+ !odb_source_read_object_stream(out, &dir->loose->base, oid))
+ return 0;
+
return -1;
}
@@ -223,16 +244,21 @@ static int odb_source_files_for_each_object(struct odb_source *source,
struct odb_source_files *files = odb_source_files_downcast(source);
int ret;
- if (!(opts->flags & ODB_FOR_EACH_OBJECT_PROMISOR_ONLY)) {
- ret = odb_source_for_each_object(&files->dirs->loose->base, request, cb, cb_data, opts);
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ if (opts->flags & ODB_FOR_EACH_OBJECT_LOCAL_ONLY && !dir->local)
+ continue;
+
+ if (!(opts->flags & ODB_FOR_EACH_OBJECT_PROMISOR_ONLY)) {
+ ret = odb_source_for_each_object(&dir->loose->base, request, cb, cb_data, opts);
+ if (ret)
+ return ret;
+ }
+
+ ret = odb_source_for_each_object(&dir->packed->base, request, cb, cb_data, opts);
if (ret)
return ret;
}
- ret = odb_source_for_each_object(&files->dirs->packed->base, request, cb, cb_data, opts);
- if (ret)
- return ret;
-
return 0;
}
@@ -241,21 +267,23 @@ static int odb_source_files_count_objects(struct odb_source *source,
unsigned long *out)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- unsigned long count;
+ unsigned long count = 0;
int ret;
- ret = odb_source_count_objects(&files->dirs->packed->base, flags, &count);
- if (ret < 0)
- goto out;
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ unsigned long dir_count;
- if (!(flags & ODB_COUNT_OBJECTS_APPROXIMATE)) {
- unsigned long loose_count;
-
- ret = odb_source_count_objects(&files->dirs->loose->base, flags, &loose_count);
+ ret = odb_source_count_objects(&dir->packed->base, flags, &dir_count);
if (ret < 0)
goto out;
+ count += dir_count;
- count += loose_count;
+ if (!(flags & ODB_COUNT_OBJECTS_APPROXIMATE)) {
+ ret = odb_source_count_objects(&dir->loose->base, flags, &dir_count);
+ if (ret < 0)
+ goto out;
+ count += dir_count;
+ }
}
*out = count;
@@ -272,15 +300,17 @@ static int odb_source_files_find_abbrev_len(struct odb_source *source,
{
struct odb_source_files *files = odb_source_files_downcast(source);
unsigned len = min_len;
- int ret;
+ int ret = 0;
- ret = odb_source_find_abbrev_len(&files->dirs->packed->base, oid, len, &len);
- if (ret < 0)
- goto out;
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ ret = odb_source_find_abbrev_len(&dir->packed->base, oid, len, &len);
+ if (ret < 0)
+ goto out;
- ret = odb_source_find_abbrev_len(&files->dirs->loose->base, oid, len, &len);
- if (ret < 0)
- goto out;
+ ret = odb_source_find_abbrev_len(&dir->loose->base, oid, len, &len);
+ if (ret < 0)
+ goto out;
+ }
*out = len;
ret = 0;
@@ -294,9 +324,12 @@ static int odb_source_files_freshen_object(struct odb_source *source,
const time_t *mtime)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- if (odb_source_freshen_object(&files->dirs->packed->base, oid, mtime) ||
- odb_source_freshen_object(&files->dirs->loose->base, oid, mtime))
- return 1;
+
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next)
+ if (odb_source_freshen_object(&dir->packed->base, oid, mtime) ||
+ odb_source_freshen_object(&dir->loose->base, oid, mtime))
+ return 1;
+
return 0;
}
@@ -958,11 +991,13 @@ static int odb_source_files_fsck(struct odb_source *source,
struct odb_source_files *files = odb_source_files_downcast(source);
int ret = 0;
- if (!(opts->flags & ODB_FSCK_FULL) && !source->local)
- return 0;
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ if (!(opts->flags & ODB_FSCK_FULL) && !dir->local)
+ continue;
- ret |= odb_source_fsck(&files->dirs->loose->base, opts);
- ret |= odb_source_fsck(&files->dirs->packed->base, opts);
+ ret |= odb_source_fsck(&dir->loose->base, opts);
+ ret |= odb_source_fsck(&dir->packed->base, opts);
+ }
return ret;
}
diff --git a/odb/source-files.h b/odb/source-files.h
index 77f4d842e0..36af0c1b8b 100644
--- a/odb/source-files.h
+++ b/odb/source-files.h
@@ -15,6 +15,9 @@ struct odb_files_dir {
/* Absolute path to the object directory. */
char *abspath;
+ /* List of alternate object directories. */
+ struct odb_files_dir *next;
+
/* The two sources derived from this object directory. */
struct odb_source_loose *loose;
struct odb_source_packed *packed;
@@ -36,6 +39,12 @@ void odb_files_dir_free(struct odb_files_dir *dir);
*/
struct odb_source_files {
struct odb_source base;
+
+ /*
+ * List of all object directories; the main directory is first (and
+ * cannot be NULL after initialization). Subsequent directories are
+ * alternates.
+ */
struct odb_files_dir *dirs;
};
--
2.56.0.406.ga2d225a756.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* [PATCH v2 07/13] tmp-objdir: absorb logic to set and restore primary sources
2026-10-08 8:35 ` [PATCH v2 " Patrick Steinhardt
` (5 preceding siblings ...)
2026-10-08 8:35 ` [PATCH v2 06/13] odb/source-files: add the ability to have multiple object dirs Patrick Steinhardt
@ 2026-10-08 8:35 ` Patrick Steinhardt
2026-10-08 8:35 ` [PATCH v2 08/13] tmp-objdir: manage quarantine as an object directory Patrick Steinhardt
` (6 subsequent siblings)
13 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-08 8:35 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak
The functions `odb_set_temporary_primary_source()` and
`odb_restore_primary_source()` can be used to adapt the primary object
database source. Nowadays though we only have a single user of this
subsystem left, which is the "tmp-objdir" subsystem.
Despite that, this functionality is also becoming less useful overall as
alternates are becoming an implementation detail of the "files" backend.
And with that change, there will only ever be a single source attached
to the object database anyway.
Move the logic into the "tmp-objdir" subsystem accordingly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
odb.c | 45 ---------------------------------------------
odb.h | 17 -----------------
tmp-objdir.c | 38 ++++++++++++++++++++++++++++++++++----
3 files changed, 34 insertions(+), 66 deletions(-)
diff --git a/odb.c b/odb.c
index 1dc8647159..8b54271c27 100644
--- a/odb.c
+++ b/odb.c
@@ -239,51 +239,6 @@ static struct odb_source *odb_add_alternate_recursively(struct object_database *
return alternate;
}
-struct odb_source *odb_set_temporary_primary_source(struct object_database *odb,
- const char *dir, int will_destroy,
- struct odb_source **prev_source)
-{
- struct odb_source *source;
-
- /*
- * Make a new primary odb and link the old primary ODB in as an
- * alternate
- */
- source = odb_source_new(odb, dir, false);
-
- /*
- * Disable ref updates while a temporary odb is active, since
- * the objects in the database may roll back.
- */
- odb->repo->disable_ref_updates = true;
- source->will_destroy = will_destroy;
- source->next = odb->sources;
- odb->sources = source;
-
- if (prev_source)
- *prev_source = source->next;
-
- return source;
-}
-
-void odb_restore_primary_source(struct object_database *odb,
- struct odb_source *restore_source,
- const char *old_path)
-{
- struct odb_source *cur_source = odb->sources;
-
- if (strcmp(old_path, cur_source->path))
- BUG("expected %s as primary object store; found %s",
- old_path, cur_source->path);
-
- if (cur_source->next != restore_source)
- BUG("we expect the old primary object store to be the first alternate");
-
- odb->repo->disable_ref_updates = false;
- odb->sources = restore_source;
- odb_source_free(cur_source);
-}
-
char *compute_alternate_path(const char *path, struct strbuf *err)
{
char *ref_git = NULL;
diff --git a/odb.h b/odb.h
index 5c86572b5d..4143812f55 100644
--- a/odb.h
+++ b/odb.h
@@ -226,23 +226,6 @@ struct odb_fsck_options {
*/
int odb_fsck(struct object_database *odb, struct odb_fsck_options *opts);
-/*
- * Replace the current writable object directory with the specified temporary
- * object directory and return the newly installed primary source. The former
- * primary source is reported via `prev_source` when non-NULL.
- */
-struct odb_source *odb_set_temporary_primary_source(struct object_database *odb,
- const char *dir, int will_destroy,
- struct odb_source **prev_source);
-
-/*
- * Restore the primary source that was previously replaced by
- * `odb_set_temporary_primary_source()`.
- */
-void odb_restore_primary_source(struct object_database *odb,
- struct odb_source *restore_source,
- const char *old_path);
-
/*
* Iterate through all alternates of the database and execute the provided
* callback function for each of them. Stop iterating once the callback
diff --git a/tmp-objdir.c b/tmp-objdir.c
index deaaf6ba2e..31a7920be7 100644
--- a/tmp-objdir.c
+++ b/tmp-objdir.c
@@ -51,6 +51,26 @@ static void tmp_objdir_reparent(const char *old_cwd,
free(path);
}
+/*
+ * Restore the primary source that was previously replaced by
+ * `tmp_objdir_replace_primary_odb()`.
+ */
+static void tmp_objdir_restore_source(struct tmp_objdir *t)
+{
+ struct odb_source *cur_source = t->repo->objects->sources;
+
+ if (strcmp(t->path.buf, cur_source->path))
+ BUG("expected %s as primary object store; found %s",
+ t->path.buf, cur_source->path);
+
+ if (cur_source->next != t->prev_source)
+ BUG("we expect the old primary object store to be the first alternate");
+
+ t->repo->disable_ref_updates = false;
+ t->repo->objects->sources = t->prev_source;
+ odb_source_free(cur_source);
+}
+
int tmp_objdir_destroy(struct tmp_objdir *t)
{
int err;
@@ -62,7 +82,7 @@ int tmp_objdir_destroy(struct tmp_objdir *t)
the_tmp_objdir = NULL;
if (t->prev_source)
- odb_restore_primary_source(t->repo->objects, t->prev_source, t->path.buf);
+ tmp_objdir_restore_source(t);
err = remove_dir_recursively(&t->path, 0);
@@ -298,7 +318,7 @@ int tmp_objdir_migrate(struct tmp_objdir *t)
if (t->prev_source) {
if (t->repo->objects->sources->will_destroy)
BUG("migrating an ODB that was marked for destruction");
- odb_restore_primary_source(t->repo->objects, t->prev_source, t->path.buf);
+ tmp_objdir_restore_source(t);
t->prev_source = NULL;
}
@@ -328,6 +348,16 @@ struct odb_source *tmp_objdir_replace_primary_odb(struct tmp_objdir *t,
BUG("the primary object database is already replaced");
t->will_destroy = will_destroy;
- return odb_set_temporary_primary_source(t->repo->objects, t->path.buf,
- will_destroy, &t->prev_source);
+ /*
+ * Make a new primary source and link the old primary source in as an
+ * alternate. Disable ref updates while a temporary source is active,
+ * since the objects in the database may roll back.
+ */
+ t->prev_source = t->repo->objects->sources;
+ t->repo->objects->sources = odb_source_new(t->repo->objects, t->path.buf, false);
+ t->repo->objects->sources->next = t->prev_source;
+ t->repo->objects->sources->will_destroy = will_destroy;
+ t->repo->disable_ref_updates = true;
+
+ return t->repo->objects->sources;
}
--
2.56.0.406.ga2d225a756.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* [PATCH v2 08/13] tmp-objdir: manage quarantine as an object directory
2026-10-08 8:35 ` [PATCH v2 " Patrick Steinhardt
` (6 preceding siblings ...)
2026-10-08 8:35 ` [PATCH v2 07/13] tmp-objdir: absorb logic to set and restore primary sources Patrick Steinhardt
@ 2026-10-08 8:35 ` Patrick Steinhardt
2026-10-08 8:35 ` [PATCH v2 09/13] tmp-objdir: replace primary source at creation time Patrick Steinhardt
` (5 subsequent siblings)
13 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-08 8:35 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak
When creating a quarantine directory via the "tmp-objdir" subsystem we
create a new "files" backend that new objects part of the transaction
can be written to. In a future commit though we'll move handling of
alternates into the "files" backend, and as part of that it will no
longer be possible for us to have multiple sources attached to a single
object database.
In a preceding commit, we have prepared the "files" backend to be able
to handle multiple object directories. We don't use that mechanism for
alternates yet, but will start doing so in a subsequent commit. But with
that infrastructure ready we can already migrate tmp-objdirs over to use
this new mechanism.
Adapt the subsystem so we create a `struct odb_files_dir` instead of a
new "files" source.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
object-file.c | 4 ++--
tmp-objdir.c | 48 +++++++++++++++++++++++++++++-------------------
tmp-objdir.h | 4 ++--
3 files changed, 33 insertions(+), 23 deletions(-)
diff --git a/object-file.c b/object-file.c
index a4cbf8b081..fe9139c96d 100644
--- a/object-file.c
+++ b/object-file.c
@@ -488,7 +488,7 @@ struct odb_transaction_files {
enum odb_transaction_flags flags;
struct tmp_objdir *objdir;
- struct odb_source *quarantine;
+ struct odb_files_dir *quarantine;
struct transaction_packfile packfile;
const char *prefix;
@@ -1443,7 +1443,7 @@ static int odb_transaction_files_write_pack(struct odb_transaction *base,
return -1;
}
- odb_source_prepare(transaction->quarantine,
+ odb_source_prepare(&transaction->quarantine->packed->base,
ODB_PREPARE_FLUSH_CACHES);
}
diff --git a/tmp-objdir.c b/tmp-objdir.c
index 31a7920be7..a3903b6e6d 100644
--- a/tmp-objdir.c
+++ b/tmp-objdir.c
@@ -12,13 +12,17 @@
#include "quote.h"
#include "odb.h"
#include "odb/source.h"
+#include "odb/source-files.h"
+#include "odb/source-loose.h"
+#include "odb/source-packed.h"
#include "repository.h"
struct tmp_objdir {
struct repository *repo;
struct strbuf path;
struct strvec env;
- struct odb_source *prev_source;
+ struct odb_files_dir *temp_dir;
+ struct odb_files_dir *orig_dir;
int will_destroy;
};
@@ -57,18 +61,19 @@ static void tmp_objdir_reparent(const char *old_cwd,
*/
static void tmp_objdir_restore_source(struct tmp_objdir *t)
{
- struct odb_source *cur_source = t->repo->objects->sources;
+ struct odb_source_files *files = odb_source_files_downcast(t->repo->objects->sources);
+ struct odb_files_dir *cur_dir = files->dirs;
- if (strcmp(t->path.buf, cur_source->path))
+ if (t->temp_dir != files->dirs)
BUG("expected %s as primary object store; found %s",
- t->path.buf, cur_source->path);
+ t->temp_dir->abspath, cur_dir->abspath);
- if (cur_source->next != t->prev_source)
+ if (t->temp_dir->next != t->orig_dir)
BUG("we expect the old primary object store to be the first alternate");
t->repo->disable_ref_updates = false;
- t->repo->objects->sources = t->prev_source;
- odb_source_free(cur_source);
+ files->dirs = t->orig_dir;
+ odb_files_dir_free(cur_dir);
}
int tmp_objdir_destroy(struct tmp_objdir *t)
@@ -81,7 +86,7 @@ int tmp_objdir_destroy(struct tmp_objdir *t)
if (t == the_tmp_objdir)
the_tmp_objdir = NULL;
- if (t->prev_source)
+ if (t->orig_dir)
tmp_objdir_restore_source(t);
err = remove_dir_recursively(&t->path, 0);
@@ -315,11 +320,11 @@ int tmp_objdir_migrate(struct tmp_objdir *t)
if (!t)
return 0;
- if (t->prev_source) {
- if (t->repo->objects->sources->will_destroy)
+ if (t->orig_dir) {
+ if (t->will_destroy)
BUG("migrating an ODB that was marked for destruction");
tmp_objdir_restore_source(t);
- t->prev_source = NULL;
+ t->orig_dir = NULL;
}
strbuf_addbuf(&src, &t->path);
@@ -341,10 +346,12 @@ const char **tmp_objdir_env(const struct tmp_objdir *t)
return t->env.v;
}
-struct odb_source *tmp_objdir_replace_primary_odb(struct tmp_objdir *t,
- int will_destroy)
+struct odb_files_dir *tmp_objdir_replace_primary_odb(struct tmp_objdir *t,
+ int will_destroy)
{
- if (t->prev_source)
+ struct odb_source_files *files = odb_source_files_downcast(t->repo->objects->sources);
+
+ if (t->temp_dir)
BUG("the primary object database is already replaced");
t->will_destroy = will_destroy;
@@ -353,11 +360,14 @@ struct odb_source *tmp_objdir_replace_primary_odb(struct tmp_objdir *t,
* alternate. Disable ref updates while a temporary source is active,
* since the objects in the database may roll back.
*/
- t->prev_source = t->repo->objects->sources;
- t->repo->objects->sources = odb_source_new(t->repo->objects, t->path.buf, false);
- t->repo->objects->sources->next = t->prev_source;
- t->repo->objects->sources->will_destroy = will_destroy;
+ t->temp_dir = odb_files_dir_new(t->repo->objects, t->path.buf, false);
+ t->temp_dir->loose->base.will_destroy = will_destroy;
+ t->temp_dir->packed->base.will_destroy = will_destroy;
+ t->temp_dir->next = files->dirs;
+
+ t->orig_dir = files->dirs;
+ files->dirs = t->temp_dir;
t->repo->disable_ref_updates = true;
- return t->repo->objects->sources;
+ return t->temp_dir;
}
diff --git a/tmp-objdir.h b/tmp-objdir.h
index 05f0d08d10..5ed0db1ee0 100644
--- a/tmp-objdir.h
+++ b/tmp-objdir.h
@@ -61,7 +61,7 @@ void tmp_objdir_discard_objects(struct tmp_objdir *);
* If will_destroy is nonzero, the object directory may not be migrated. Returns
* the newly installed primary source.
*/
-struct odb_source *tmp_objdir_replace_primary_odb(struct tmp_objdir *,
- int will_destroy);
+struct odb_files_dir *tmp_objdir_replace_primary_odb(struct tmp_objdir *,
+ int will_destroy);
#endif /* TMP_OBJDIR_H */
--
2.56.0.406.ga2d225a756.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* [PATCH v2 09/13] tmp-objdir: replace primary source at creation time
2026-10-08 8:35 ` [PATCH v2 " Patrick Steinhardt
` (7 preceding siblings ...)
2026-10-08 8:35 ` [PATCH v2 08/13] tmp-objdir: manage quarantine as an object directory Patrick Steinhardt
@ 2026-10-08 8:35 ` Patrick Steinhardt
2026-10-08 8:36 ` [PATCH v2 10/13] odb/source: make `will_destroy` an implementation detail Patrick Steinhardt
` (4 subsequent siblings)
13 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-08 8:35 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak
When creating transactions via the "tmp-objdir" subsystem callers are
expected to first call `tmp_objdir_create()` to create the quarantine
directory and then `tmp_objdir_replace_primary_odb()` to activate that
quarantine directory as the primary object database source so that all
newly written objects are written into it.
This dance is performed by all users of temporary object directories, so
that makes the setup unnecessarily involved. Merge the logic to replace
the primary object database source into `tmp_objdir_create()` to
simplify the dance.
Note that with the quarantine directory being installed by
`tmp_objdir_create()`, users of this function no longer get a pointer to
the underlying quarantine directory. So instead of repreparing only that
directory, we now reprepare the whole source instead, and as we know
that the source has wired up the quarantine directory already, this will
cause us to reprepare the quarantine directory, too.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
log-tree.c | 3 +--
object-file.c | 8 ++------
tmp-objdir.c | 47 +++++++++++++++++++----------------------------
tmp-objdir.h | 18 +++++-------------
4 files changed, 27 insertions(+), 49 deletions(-)
diff --git a/log-tree.c b/log-tree.c
index 83a3c4bf9b..361dfadbb3 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -1043,10 +1043,9 @@ static int do_remerge_diff(struct rev_info *opt,
* into the alternative object store list as the primary.
*/
if (opt->remerge_diff && !opt->remerge_objdir) {
- opt->remerge_objdir = tmp_objdir_create(the_repository, "remerge-diff");
+ opt->remerge_objdir = tmp_objdir_create(the_repository, "remerge-diff", 1);
if (!opt->remerge_objdir)
return error(_("unable to create temporary object directory"));
- tmp_objdir_replace_primary_odb(opt->remerge_objdir, 1);
}
/* Setup merge options */
diff --git a/object-file.c b/object-file.c
index fe9139c96d..02521afcaf 100644
--- a/object-file.c
+++ b/object-file.c
@@ -488,7 +488,6 @@ struct odb_transaction_files {
enum odb_transaction_flags flags;
struct tmp_objdir *objdir;
- struct odb_files_dir *quarantine;
struct transaction_packfile packfile;
const char *prefix;
@@ -511,12 +510,10 @@ int odb_transaction_files_prepare(struct odb_transaction *base)
if (!transaction || transaction->objdir)
return 0;
- transaction->objdir = tmp_objdir_create(base->source->odb->repo, transaction->prefix);
+ transaction->objdir = tmp_objdir_create(base->source->odb->repo, transaction->prefix, 0);
if (!transaction->objdir)
return error(_("unable to create temporary object directory"));
- transaction->quarantine = tmp_objdir_replace_primary_odb(transaction->objdir, 0);
-
return 0;
}
@@ -1443,8 +1440,7 @@ static int odb_transaction_files_write_pack(struct odb_transaction *base,
return -1;
}
- odb_source_prepare(&transaction->quarantine->packed->base,
- ODB_PREPARE_FLUSH_CACHES);
+ odb_source_prepare(base->source, ODB_PREPARE_FLUSH_CACHES);
}
return 0;
diff --git a/tmp-objdir.c b/tmp-objdir.c
index a3903b6e6d..719b55e580 100644
--- a/tmp-objdir.c
+++ b/tmp-objdir.c
@@ -57,7 +57,7 @@ static void tmp_objdir_reparent(const char *old_cwd,
/*
* Restore the primary source that was previously replaced by
- * `tmp_objdir_replace_primary_odb()`.
+ * `tmp_objdir_create()`.
*/
static void tmp_objdir_restore_source(struct tmp_objdir *t)
{
@@ -157,8 +157,10 @@ static int setup_tmp_objdir(const char *root)
}
struct tmp_objdir *tmp_objdir_create(struct repository *r,
- const char *prefix)
+ const char *prefix,
+ int will_destroy)
{
+ struct odb_source_files *files = odb_source_files_downcast(r->objects->sources);
static int installed_handlers;
struct tmp_objdir *t;
@@ -167,6 +169,7 @@ struct tmp_objdir *tmp_objdir_create(struct repository *r,
t = xcalloc(1, sizeof(*t));
t->repo = r;
+ t->will_destroy = will_destroy;
strbuf_init(&t->path, 0);
strvec_init(&t->env);
@@ -204,6 +207,20 @@ struct tmp_objdir *tmp_objdir_create(struct repository *r,
env_replace(&t->env, GIT_QUARANTINE_ENVIRONMENT,
absolute_path(t->path.buf));
+ /*
+ * Make a new primary source and link the old primary source in as an
+ * alternate. Disable ref updates while a temporary source is active,
+ * since the objects in the database may roll back.
+ */
+ t->temp_dir = odb_files_dir_new(t->repo->objects, t->path.buf, false);
+ t->temp_dir->loose->base.will_destroy = will_destroy;
+ t->temp_dir->packed->base.will_destroy = will_destroy;
+ t->temp_dir->next = files->dirs;
+
+ t->orig_dir = files->dirs;
+ files->dirs = t->temp_dir;
+ t->repo->disable_ref_updates = true;
+
return t;
}
@@ -345,29 +362,3 @@ const char **tmp_objdir_env(const struct tmp_objdir *t)
return NULL;
return t->env.v;
}
-
-struct odb_files_dir *tmp_objdir_replace_primary_odb(struct tmp_objdir *t,
- int will_destroy)
-{
- struct odb_source_files *files = odb_source_files_downcast(t->repo->objects->sources);
-
- if (t->temp_dir)
- BUG("the primary object database is already replaced");
- t->will_destroy = will_destroy;
-
- /*
- * Make a new primary source and link the old primary source in as an
- * alternate. Disable ref updates while a temporary source is active,
- * since the objects in the database may roll back.
- */
- t->temp_dir = odb_files_dir_new(t->repo->objects, t->path.buf, false);
- t->temp_dir->loose->base.will_destroy = will_destroy;
- t->temp_dir->packed->base.will_destroy = will_destroy;
- t->temp_dir->next = files->dirs;
-
- t->orig_dir = files->dirs;
- files->dirs = t->temp_dir;
- t->repo->disable_ref_updates = true;
-
- return t->temp_dir;
-}
diff --git a/tmp-objdir.h b/tmp-objdir.h
index 5ed0db1ee0..a301b01d84 100644
--- a/tmp-objdir.h
+++ b/tmp-objdir.h
@@ -11,7 +11,7 @@
* Example:
*
* struct child_process child = CHILD_PROCESS_INIT;
- * struct tmp_objdir *t = tmp_objdir_create(repo, "incoming");
+ * struct tmp_objdir *t = tmp_objdir_create(repo, "incoming", 0);
* strvec_push(&child.args, cmd);
* strvec_pushv(&child.env, tmp_objdir_env(t));
* if (!run_command(&child)) && !tmp_objdir_migrate(t))
@@ -25,10 +25,11 @@ struct repository;
struct tmp_objdir;
/*
- * Create a new temporary object directory with the specified prefix;
- * returns NULL on failure.
+ * Create a new temporary object directory with the specified prefix and
+ * install the directory as the primary write target; returns NULL on failure.
*/
-struct tmp_objdir *tmp_objdir_create(struct repository *r, const char *prefix);
+struct tmp_objdir *tmp_objdir_create(struct repository *r, const char *prefix,
+ int will_destroy);
/*
* Return a list of environment strings, suitable for use with
@@ -55,13 +56,4 @@ int tmp_objdir_destroy(struct tmp_objdir *);
*/
void tmp_objdir_discard_objects(struct tmp_objdir *);
-/*
- * Replaces the writable object store in the current process with the temporary
- * object directory and makes the former main object store an alternate.
- * If will_destroy is nonzero, the object directory may not be migrated. Returns
- * the newly installed primary source.
- */
-struct odb_files_dir *tmp_objdir_replace_primary_odb(struct tmp_objdir *,
- int will_destroy);
-
#endif /* TMP_OBJDIR_H */
--
2.56.0.406.ga2d225a756.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* [PATCH v2 10/13] odb/source: make `will_destroy` an implementation detail
2026-10-08 8:35 ` [PATCH v2 " Patrick Steinhardt
` (8 preceding siblings ...)
2026-10-08 8:35 ` [PATCH v2 09/13] tmp-objdir: replace primary source at creation time Patrick Steinhardt
@ 2026-10-08 8:36 ` Patrick Steinhardt
2026-10-08 8:36 ` [PATCH v2 11/13] odb/source-files: extract reading alternates Patrick Steinhardt
` (3 subsequent siblings)
13 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-08 8:36 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak
The `struct odb_source::will_destroy` flag tracks whether a source is
part of a transaction that we know we'll destruct anyway. If so, the
backend can optimize for that particular case, for example by not
flushing any data to disk.
While the intent is sensible, it assumes that transactions are backed by
a separate source that's being linked into the object database. But that
may or may not be true, as backends may have significantly better ways
to achieve the same. So the assumption doesn't make much sense in the
first place, as we're now tracking backend-specific details on the
generic `struct odb_source` level.
Move the field from the generic source into the "loose" source, as this
is the only source that ever makes use of it anyway.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
odb/source-loose.c | 2 +-
odb/source-loose.h | 3 +++
odb/source.h | 5 -----
tmp-objdir.c | 4 +---
4 files changed, 5 insertions(+), 9 deletions(-)
diff --git a/odb/source-loose.c b/odb/source-loose.c
index 3c9edba46a..b2fbccd3c0 100644
--- a/odb/source-loose.c
+++ b/odb/source-loose.c
@@ -599,7 +599,7 @@ static int odb_source_loose_freshen_object(struct odb_source *source,
static void close_loose_object(struct odb_source_loose *loose,
int fd, const char *filename)
{
- if (loose->base.will_destroy)
+ if (loose->will_destroy)
goto out;
if (batch_fsync_enabled(FSYNC_COMPONENT_LOOSE_OBJECT))
diff --git a/odb/source-loose.h b/odb/source-loose.h
index 3cf2e1f8f1..7c7e845fe3 100644
--- a/odb/source-loose.h
+++ b/odb/source-loose.h
@@ -28,6 +28,9 @@ struct odb_source_loose {
/* Map between object IDs for loose objects. */
struct loose_object_map *map;
+
+ /* Whether this is a source that will never be committed to disk. */
+ int will_destroy;
};
struct odb_source_loose *odb_source_loose_new(struct object_database *odb,
diff --git a/odb/source.h b/odb/source.h
index ea00873763..9fd2b2e5b5 100644
--- a/odb/source.h
+++ b/odb/source.h
@@ -81,11 +81,6 @@ struct odb_source {
*/
bool local;
- /*
- * This object store is ephemeral, so there is no need to fsync.
- */
- int will_destroy;
-
/*
* Path to the source. If this is a relative path, it is relative to
* the current working directory.
diff --git a/tmp-objdir.c b/tmp-objdir.c
index 719b55e580..2f2ffbbc7d 100644
--- a/tmp-objdir.c
+++ b/tmp-objdir.c
@@ -14,7 +14,6 @@
#include "odb/source.h"
#include "odb/source-files.h"
#include "odb/source-loose.h"
-#include "odb/source-packed.h"
#include "repository.h"
struct tmp_objdir {
@@ -213,8 +212,7 @@ struct tmp_objdir *tmp_objdir_create(struct repository *r,
* since the objects in the database may roll back.
*/
t->temp_dir = odb_files_dir_new(t->repo->objects, t->path.buf, false);
- t->temp_dir->loose->base.will_destroy = will_destroy;
- t->temp_dir->packed->base.will_destroy = will_destroy;
+ t->temp_dir->loose->will_destroy = will_destroy;
t->temp_dir->next = files->dirs;
t->orig_dir = files->dirs;
--
2.56.0.406.ga2d225a756.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* [PATCH v2 11/13] odb/source-files: extract reading alternates
2026-10-08 8:35 ` [PATCH v2 " Patrick Steinhardt
` (9 preceding siblings ...)
2026-10-08 8:36 ` [PATCH v2 10/13] odb/source: make `will_destroy` an implementation detail Patrick Steinhardt
@ 2026-10-08 8:36 ` Patrick Steinhardt
2026-10-08 8:36 ` [PATCH v2 12/13] odb/source-files: move alternates into the backend Patrick Steinhardt
` (2 subsequent siblings)
13 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-08 8:36 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak
In the next commit we'll add a new callsite that wants to read
alternates without having a proper object database source for a given
alternate available to it. Prepare for this by extracting the logic into
a separate function that only requires an object directory path as
input.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
odb/source-files.c | 33 +++++++++++++++++++--------------
1 file changed, 19 insertions(+), 14 deletions(-)
diff --git a/odb/source-files.c b/odb/source-files.c
index 9389546b3e..6aaf625352 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -174,6 +174,24 @@ static int odb_source_files_create_on_disk(struct odb_source *source,
return ret;
}
+static int read_alternates(const char *object_dir, struct strvec *out)
+{
+ struct strbuf buf = STRBUF_INIT;
+ char *path;
+
+ path = xstrfmt("%s/info/alternates", object_dir);
+ if (strbuf_read_file(&buf, path, 1024) < 0) {
+ warn_on_fopen_errors(path);
+ free(path);
+ return 0;
+ }
+ parse_alternates(buf.buf, '\n', object_dir, out);
+
+ strbuf_release(&buf);
+ free(path);
+ return 0;
+}
+
static void odb_source_files_prepare(struct odb_source *source,
enum odb_prepare_flags flags)
{
@@ -364,20 +382,7 @@ static int odb_source_files_begin_transaction(struct odb_source *source,
static int odb_source_files_read_alternates(struct odb_source *source,
struct strvec *out)
{
- struct strbuf buf = STRBUF_INIT;
- char *path;
-
- path = xstrfmt("%s/info/alternates", source->path);
- if (strbuf_read_file(&buf, path, 1024) < 0) {
- warn_on_fopen_errors(path);
- free(path);
- return 0;
- }
- parse_alternates(buf.buf, '\n', source->path, out);
-
- strbuf_release(&buf);
- free(path);
- return 0;
+ return read_alternates(source->path, out);
}
static int too_many_loose_objects(struct odb_source_files *files, int limit)
--
2.56.0.406.ga2d225a756.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* [PATCH v2 12/13] odb/source-files: move alternates into the backend
2026-10-08 8:35 ` [PATCH v2 " Patrick Steinhardt
` (10 preceding siblings ...)
2026-10-08 8:36 ` [PATCH v2 11/13] odb/source-files: extract reading alternates Patrick Steinhardt
@ 2026-10-08 8:36 ` Patrick Steinhardt
2026-10-08 8:36 ` [PATCH v2 13/13] odb/source: drop `read_alternates` callback Patrick Steinhardt
2026-10-08 9:24 ` [PATCH v2 00/13] odb/source-files: move alternates into the backend Karthik Nayak
13 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-08 8:36 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak
Originally, when designing pluggable object databases the goal was that
the object database can have multiple sources, and every source attached
to it could use a different backend. This would have allowed for quite a
lot of flexibility, as you could trivially mix and match different kinds
of object storages in whatever way you like.
But while well-intentioned, this design led to a bunch of conceptual
problems:
- We're now trying to read objects in source order, whereas we
previously tried to read objects via packfiles before trying to read
them via loose objects. This led to a performance regression when
using alternates or when using a quarantine directory.
- Some data structures are supposed to only ever exist once, like for
example bitmaps and commit graphs. At the same time, those data
structures also span across the union of all objects, so they may
cross sources.
- It is unclear how we can extend GIT_OBJECT_DIRECTORY or
GIT_ALTERNATE_OBJECT_DIRECTORIES to become backend-agnostic in a
backwards-compatible way. In general, introducing an object storage
extension into the current status quo where alternates may have to
be extended to become generic was proving to be painful.
- Some mechanisms of alternates assume way too much about how exactly
their backends work. Alternate refs for example assume that the
alternate is backed by a filesystem path, and that this filesystem
path may also allow us to read references. This is not a given
though, as backends may not even have local data at all.
In short, there are a bunch of conceptual mismatches when we have
alternates and pluggable object databases coexist. So while the original
idea was nice, it does not result in a system that is easy to reason
about.
Correct course by moving alternates into the "files" source itself so
that it becomes an implementation detail thereof so that we can avoid
all of these shortcomings. While it's unfortunate that we cannot easily
mix and match sources now, that ability doesn't go away. It's still very
much feasible to introduce a new backend that allows for exactly that
use case, and such a backend may also be a lot more flexible as we can
now add new logic to determine which objects should be stored where. So
the original motivation for having per-source backends can still be
realized with the new architecture.
Note that as part of this move, we also handle the GIT_OBJECT_DIRECTORY
and GIT_ALTERNATE_OBJECT_DIRECTORIES environment variables in the
"files" backend. This may be surprising at first, but object directories
are very much a concept of that backend, too. So these variables would
have bad interactions with other backends, and they create a bit of a
mismatch with the eventual object storage extension that we plan to
introduce.
Note that this commit is way larger than I'd like it to be. I'm sorry, I
couldn't find a way to split it up further. That being said, most of the
changes are straight-forward conversions that go from iterating over
sources to iterating over object directories. The more involved changes
are moving the infrastructure to track individual sources from "odb.c"
and moving them into "odb/source-files.c".
This also serves to show a bit of a who's-who of commands that don't
work properly with pluggable object databases. Almost all of these
commands are related to housekeeping though -- some of them will be
converted eventually, like for example commit graphs or MIDXs. But many
of them will stay incompatible going forward as they are simply too
specific to the "files" backend.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/count-objects.c | 2 +-
builtin/fast-import.c | 18 +--
builtin/fetch.c | 2 +-
builtin/fsck.c | 6 +-
builtin/gc.c | 14 +-
builtin/index-pack.c | 2 +-
builtin/multi-pack-index.c | 6 +-
builtin/pack-objects.c | 64 ++++----
builtin/prune.c | 2 +-
bundle.c | 2 +-
commit-graph.c | 28 ++--
diagnose.c | 2 +-
fetch-pack.c | 2 +-
http-walker.c | 2 +-
http.c | 6 +-
loose.c | 16 +-
midx.c | 29 ++--
odb.c | 354 +++++----------------------------------------
odb.h | 33 +----
odb/source-files.c | 260 ++++++++++++++++++++++++++++++---
odb/source-files.h | 23 ++-
odb/source.c | 5 +-
odb/source.h | 25 +---
odb/streaming.c | 8 +-
odb/transaction.c | 2 +-
pack-bitmap.c | 8 +-
packfile.c | 28 ++--
packfile.h | 21 ++-
path.c | 2 +-
prune-packed.c | 2 +-
repack.c | 4 +-
repository.c | 4 +-
setup.c | 2 +-
t/helper/test-read-graph.c | 2 +-
tmp-objdir.c | 4 +-
35 files changed, 441 insertions(+), 549 deletions(-)
diff --git a/builtin/count-objects.c b/builtin/count-objects.c
index f2abfaccec..3f7b20fde5 100644
--- a/builtin/count-objects.c
+++ b/builtin/count-objects.c
@@ -118,7 +118,7 @@ int cmd_count_objects(int argc,
report_linked_checkout_garbage(the_repository);
}
- for_each_loose_file_in_source(the_repository->objects->sources,
+ for_each_loose_file_in_source(the_repository->objects->source,
count_loose, count_cruft, NULL, NULL);
if (verbose) {
diff --git a/builtin/fast-import.c b/builtin/fast-import.c
index 0bf76b028b..5fc082bcc7 100644
--- a/builtin/fast-import.c
+++ b/builtin/fast-import.c
@@ -895,7 +895,7 @@ static void end_packfile(void)
running = 1;
clear_delta_base_cache();
if (object_count) {
- struct odb_source_files *files = odb_source_files_downcast(pack_data->repo->objects->sources);
+ struct odb_source_files *files = odb_source_files_downcast(pack_data->repo->objects->source);
struct packed_git *new_p;
struct object_id cur_pack_oid;
char *idx_name;
@@ -975,7 +975,7 @@ static int store_object(
struct object_id *oidout,
uintmax_t mark)
{
- struct odb_source *source;
+ struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->source);
void *out, *delta;
struct object_entry *e;
unsigned char hdr[96];
@@ -1002,10 +1002,8 @@ static int store_object(
return 1;
}
- for (source = the_repository->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
-
- if (!packfile_list_find_oid(packfile_store_get_packs(files->dirs->packed), &oid))
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ if (!packfile_list_find_oid(packfile_store_get_packs(dir->packed), &oid))
continue;
e->type = type;
e->pack_id = MAX_PACK_ID;
@@ -1125,10 +1123,10 @@ static void truncate_pack(struct hashfile_checkpoint *checkpoint)
static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
{
+ struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->source);
size_t in_sz = 64 * 1024, out_sz = 64 * 1024;
unsigned char *in_buf = xmalloc(in_sz);
unsigned char *out_buf = xmalloc(out_sz);
- struct odb_source *source;
struct object_entry *e;
struct object_id oid;
unsigned long hdrlen;
@@ -1212,10 +1210,8 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
goto out;
}
- for (source = the_repository->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
-
- if (!packfile_list_find_oid(packfile_store_get_packs(files->dirs->packed), &oid))
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ if (!packfile_list_find_oid(packfile_store_get_packs(dir->packed), &oid))
continue;
e->type = OBJ_BLOB;
e->pack_id = MAX_PACK_ID;
diff --git a/builtin/fetch.c b/builtin/fetch.c
index a4b21d2651..2e9c14d4ed 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -2902,7 +2902,7 @@ int cmd_fetch(int argc,
trace2_region_enter("fetch", "write-commit-graph", the_repository);
write_commit_graph_reachable(the_repository,
- the_repository->objects->sources->path,
+ the_repository->objects->source->path,
commit_graph_flags,
NULL);
trace2_region_leave("fetch", "write-commit-graph", the_repository);
diff --git a/builtin/fsck.c b/builtin/fsck.c
index 9af4cc085b..da90cf84bf 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -868,7 +868,6 @@ int cmd_fsck(int argc,
OPT_BOOL(0, "references", &check_references, N_("check reference database consistency")),
OPT_END(),
};
- struct odb_source *source;
struct snapshot snap = {
.nr = 0,
.alloc = 0,
@@ -983,13 +982,14 @@ int cmd_fsck(int argc,
check_connectivity(repo);
if (repo->settings.core_commit_graph) {
+ struct odb_source_files *files = odb_source_files_downcast(repo->objects->source);
struct child_process commit_graph_verify = CHILD_PROCESS_INIT;
- for (source = repo->objects->sources; source; source = source->next) {
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
child_process_init(&commit_graph_verify);
commit_graph_verify.git_cmd = 1;
strvec_pushl(&commit_graph_verify.args, "commit-graph",
- "verify", "--object-dir", source->path, NULL);
+ "verify", "--object-dir", dir->abspath, NULL);
if (show_progress)
strvec_push(&commit_graph_verify.args, "--progress");
else
diff --git a/builtin/gc.c b/builtin/gc.c
index 7acd4f3215..28ada9f566 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -734,7 +734,7 @@ int cmd_gc(int argc,
if (the_repository->settings.gc_write_commit_graph == 1)
write_commit_graph_reachable(the_repository,
- the_repository->objects->sources->path,
+ the_repository->objects->source->path,
!opts.quiet && !daemonized ? COMMIT_GRAPH_WRITE_PROGRESS : 0,
NULL);
@@ -998,7 +998,7 @@ static int loose_object_auto_condition(struct gc_config *cfg UNUSED)
if (loose_object_auto_limit < 0)
return 1;
- return for_each_loose_file_in_source(the_repository->objects->sources,
+ return for_each_loose_file_in_source(the_repository->objects->source,
loose_object_count,
NULL, NULL, &count);
}
@@ -1033,7 +1033,7 @@ static int pack_loose(struct maintenance_run_opts *opts)
* Do not start pack-objects process
* if there are no loose objects.
*/
- if (!for_each_loose_file_in_source(r->objects->sources,
+ if (!for_each_loose_file_in_source(r->objects->source,
bail_on_loose,
NULL, NULL, NULL))
return 0;
@@ -1045,7 +1045,7 @@ static int pack_loose(struct maintenance_run_opts *opts)
strvec_push(&pack_proc.args, "--quiet");
else
strvec_push(&pack_proc.args, "--no-quiet");
- strvec_pushf(&pack_proc.args, "%s/pack/loose", r->objects->sources->path);
+ strvec_pushf(&pack_proc.args, "%s/pack/loose", r->objects->source->path);
pack_proc.in = -1;
@@ -1073,7 +1073,7 @@ static int pack_loose(struct maintenance_run_opts *opts)
else if (data.batch_size > 0)
data.batch_size--; /* Decrease for equality on limit. */
- for_each_loose_file_in_source(r->objects->sources,
+ for_each_loose_file_in_source(r->objects->source,
write_loose_object_to_stdin,
NULL, NULL, &data);
@@ -1396,7 +1396,7 @@ static int maintenance_run_tasks(struct maintenance_run_opts *opts,
int result = 0;
struct lock_file lk;
struct repository *r = the_repository;
- char *lock_path = xstrfmt("%s/maintenance", r->objects->sources->path);
+ char *lock_path = xstrfmt("%s/maintenance", r->objects->source->path);
enum auto_gc_hook_result auto_gc_hook_result = AUTO_GC_HOOK_UNDECIDED;
if (repo_hold_lock_file_for_update(r, &lk, lock_path, LOCK_NO_DEREF) < 0) {
@@ -2972,7 +2972,7 @@ static int update_background_schedule(const struct maintenance_start_opts *opts,
unsigned int i;
int result = 0;
struct lock_file lk;
- char *lock_path = xstrfmt("%s/schedule", the_repository->objects->sources->path);
+ char *lock_path = xstrfmt("%s/schedule", the_repository->objects->source->path);
if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) {
if (errno == EEXIST)
diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 70860b8f27..0264c18c5f 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1640,7 +1640,7 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
if (do_fsck_object && startup_info->have_repository) {
struct odb_source_files *files =
- odb_source_files_downcast(the_repository->objects->sources);
+ odb_source_files_downcast(the_repository->objects->source);
packfile_store_load_pack(files->dirs->packed, final_index_name, 0);
}
diff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c
index fc8b494996..c48212290c 100644
--- a/builtin/multi-pack-index.c
+++ b/builtin/multi-pack-index.c
@@ -80,7 +80,7 @@ static int parse_object_dir(const struct option *opt, const char *arg,
char **value = opt->value;
free(*value);
if (unset)
- *value = xstrdup(the_repository->objects->sources->path);
+ *value = xstrdup(the_repository->objects->source->path);
else
*value = real_pathdup(arg, 1);
return 0;
@@ -426,8 +426,8 @@ int cmd_multi_pack_index(int argc,
if (the_repository &&
the_repository->objects &&
- the_repository->objects->sources)
- opts.object_dir = xstrdup(the_repository->objects->sources->path);
+ the_repository->objects->source)
+ opts.object_dir = xstrdup(the_repository->objects->source->path);
argc = parse_options(argc, argv, prefix, options,
builtin_multi_pack_index_usage, 0);
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 070659b6ed..ca3a891dfb 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1566,11 +1566,10 @@ static int want_cruft_object_mtime(struct repository *r,
const struct object_id *oid,
unsigned flags, uint32_t mtime)
{
- struct odb_source *source;
+ struct odb_source_files *files = odb_source_files_downcast(r->objects->source);
- for (source = r->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- struct packed_git **cache = packfile_store_get_kept_pack_cache(files->dirs->packed, flags);
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ struct packed_git **cache = packfile_store_get_kept_pack_cache(dir->packed, flags);
for (; *cache; cache++) {
struct packed_git *p = *cache;
@@ -1753,21 +1752,19 @@ static int want_object_in_pack_mtime(const struct object_id *oid,
off_t *found_offset,
uint32_t found_mtime)
{
+ struct odb_source_files *files =
+ odb_source_files_downcast(the_repository->objects->source);
int want;
struct packfile_list_entry *e;
- struct odb_source *source;
if (!exclude && local) {
/*
- * Note that we start iterating at `sources->next` so that we
- * skip the local object source.
+ * Note that we start iterating at `dirs->next` so that we
+ * skip the local object directory.
*/
- struct odb_source *source = the_repository->objects->sources->next;
- for (; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- if (!odb_source_read_object_info(&files->dirs->loose->base, oid, NULL, 0, NULL))
+ for (struct odb_files_dir *dir = files->dirs->next; dir; dir = dir->next)
+ if (!odb_source_read_object_info(&dir->loose->base, oid, NULL, 0, NULL))
return 0;
- }
}
/*
@@ -1785,9 +1782,8 @@ static int want_object_in_pack_mtime(const struct object_id *oid,
*found_offset = 0;
}
- for (source = the_repository->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- struct multi_pack_index *m = get_multi_pack_index(files->dirs->packed);
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ struct multi_pack_index *m = get_multi_pack_index(dir->packed);
struct pack_entry e;
if (m && midx_fill_entry(m, oid, &e, NULL) == MIDX_FILL_HIT) {
@@ -1797,14 +1793,12 @@ static int want_object_in_pack_mtime(const struct object_id *oid,
}
}
- for (source = the_repository->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
-
- for (e = files->dirs->packed->packs.head; e; e = e->next) {
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ for (e = dir->packed->packs.head; e; e = e->next) {
struct packed_git *p = e->pack;
want = want_object_in_pack_one(p, oid, exclude, found_pack, found_offset, found_mtime);
if (!exclude && want > 0)
- packfile_list_prepend(&files->dirs->packed->packs, p);
+ packfile_list_prepend(&dir->packed->packs, p);
if (want != -1)
return want;
}
@@ -4170,14 +4164,13 @@ static void add_cruft_object_entry(const struct object_id *oid, enum object_type
if (!want_object_in_pack_mtime(oid, 0, &pack, &offset, mtime))
return;
if (!pack && type == OBJ_BLOB) {
- struct odb_source *source = the_repository->objects->sources;
+ struct odb_source_files *files =
+ odb_source_files_downcast(the_repository->objects->source);
int found = 0;
- for (; !found && source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- if (!odb_source_read_object_info(&files->dirs->loose->base, oid, NULL, 0, NULL))
+ for (struct odb_files_dir *dir = files->dirs; !found && dir; dir = dir->next)
+ if (!odb_source_read_object_info(&dir->loose->base, oid, NULL, 0, NULL))
found = 1;
- }
/*
* If a traversed tree has a missing blob then we want
@@ -4512,7 +4505,8 @@ static int add_object_in_unpacked_pack(const struct object_id *oid,
static void add_objects_in_unpacked_packs(void)
{
- struct odb_source *source;
+ struct odb_source_files *files =
+ odb_source_files_downcast(to_pack.repo->objects->source);
time_t mtime;
struct odb_for_each_object_options opts = {
.flags = ODB_FOR_EACH_OBJECT_PACK_ORDER |
@@ -4526,13 +4520,11 @@ static void add_objects_in_unpacked_packs(void)
.source_infop = &source_info,
};
- for (source = to_pack.repo->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
-
- if (!source->local)
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ if (!dir->local)
continue;
- if (odb_source_for_each_object(&files->dirs->packed->base, &oi,
+ if (odb_source_for_each_object(&dir->packed->base, &oi,
add_object_in_unpacked_pack, NULL, &opts))
die(_("cannot open pack index"));
}
@@ -4576,7 +4568,7 @@ static int add_loose_object(const struct object_id *oid, const char *path,
*/
static void add_unreachable_loose_objects(struct rev_info *revs)
{
- for_each_loose_file_in_source(the_repository->objects->sources,
+ for_each_loose_file_in_source(the_repository->objects->source,
add_loose_object, NULL, NULL, revs);
}
@@ -4640,11 +4632,9 @@ static int force_object_loose(struct odb_source *source,
size_t len;
int ret;
- for (struct odb_source *s = source->odb->sources; s; s = s->next) {
- struct odb_source_files *files = odb_source_files_downcast(s);
- if (!odb_source_read_object_info(&files->dirs->loose->base, oid, NULL, 0, NULL))
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next)
+ if (!odb_source_read_object_info(&dir->loose->base, oid, NULL, 0, NULL))
return 0;
- }
oi.typep = &type;
oi.sizep = &len;
@@ -4691,7 +4681,7 @@ static void loosen_unused_packed_objects(void)
if (!packlist_find(&to_pack, &oid) &&
!has_sha1_pack_kept_or_nonlocal(&oid) &&
!loosened_object_can_be_discarded(&oid, p->mtime)) {
- if (force_object_loose(the_repository->objects->sources,
+ if (force_object_loose(the_repository->objects->source,
&oid, &p->mtime))
die(_("unable to force loose object"));
loosened_objects_nr++;
diff --git a/builtin/prune.c b/builtin/prune.c
index a7e4678d11..97f4c1418b 100644
--- a/builtin/prune.c
+++ b/builtin/prune.c
@@ -198,7 +198,7 @@ int cmd_prune(int argc,
revs.exclude_promisor_objects = 1;
}
- for_each_loose_file_in_source(repo->objects->sources,
+ for_each_loose_file_in_source(repo->objects->source,
prune_object, prune_cruft, prune_subdir, &revs);
prune_packed_objects(show_only ? PRUNE_PACKED_DRY_RUN : 0);
diff --git a/bundle.c b/bundle.c
index f55a521b2a..a81fd29505 100644
--- a/bundle.c
+++ b/bundle.c
@@ -239,7 +239,7 @@ int verify_bundle(struct repository *r,
.quiet = 1,
};
- if (!r || !r->objects || !r->objects->sources)
+ if (!r || !r->objects || !r->objects->source)
return error(_("need a repository to verify a bundle"));
for (i = 0; i < p->nr; i++) {
diff --git a/commit-graph.c b/commit-graph.c
index 7cc486d140..673d54db29 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -15,6 +15,7 @@
#include "hash-lookup.h"
#include "commit-graph.h"
#include "odb.h"
+#include "odb/source-files.h"
#include "oid-array.h"
#include "path.h"
#include "alloc.h"
@@ -38,7 +39,7 @@ void git_test_write_commit_graph_or_die(struct repository *repo)
if (git_env_bool(GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS, 0))
flags = COMMIT_GRAPH_WRITE_BLOOM_FILTERS;
- if (write_commit_graph_reachable(repo, repo->objects->sources->path, flags, NULL))
+ if (write_commit_graph_reachable(repo, repo->objects->source->path, flags, NULL))
die("failed to write commit-graph under GIT_TEST_COMMIT_GRAPH");
}
@@ -657,7 +658,7 @@ struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,
CALLOC_ARRAY(oids, count);
for (i = 0; i < count; i++) {
- struct odb_source *source;
+ struct odb_source_files *files;
if (strbuf_getline_lf(&line, fp) == EOF)
break;
@@ -670,9 +671,11 @@ struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,
}
valid = 0;
- for (source = odb->sources; source; source = source->next) {
- char *graph_name = get_split_graph_filename(source->path, line.buf);
- struct commit_graph *g = load_commit_graph_one(odb->repo, source->path, graph_name);
+
+ files = odb_source_files_downcast(odb->source);
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ char *graph_name = get_split_graph_filename(dir->abspath, line.buf);
+ struct commit_graph *g = load_commit_graph_one(odb->repo, dir->abspath, graph_name);
free(graph_name);
@@ -742,7 +745,7 @@ struct commit_graph *read_commit_graph_one(struct repository *repo,
*/
static struct commit_graph *prepare_commit_graph(struct repository *r)
{
- struct odb_source *source;
+ struct odb_source_files *files;
/*
* Early return if there is no object database or if the commit graph is
@@ -773,8 +776,9 @@ static struct commit_graph *prepare_commit_graph(struct repository *r)
if (!commit_graph_compatible(r))
return NULL;
- for (source = r->objects->sources; source; source = source->next) {
- r->objects->commit_graph = read_commit_graph_one(r, source->path);
+ files = odb_source_files_downcast(r->objects->source);
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ r->objects->commit_graph = read_commit_graph_one(r, dir->abspath);
if (r->objects->commit_graph)
break;
}
@@ -2003,7 +2007,7 @@ static int fill_oids_from_commits(struct write_commit_graph_context *ctx,
static void fill_oids_from_all_packs(struct write_commit_graph_context *ctx)
{
- struct odb_source *source;
+ struct odb_source_files *files;
enum object_type type;
struct odb_for_each_object_options opts = {
.flags = ODB_FOR_EACH_OBJECT_PACK_ORDER,
@@ -2018,9 +2022,9 @@ static void fill_oids_from_all_packs(struct write_commit_graph_context *ctx)
_("Finding commits for commit graph among packed objects"),
ctx->approx_nr_objects);
- for (source = ctx->r->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- odb_source_for_each_object(&files->dirs->packed->base, &oi, add_packed_commits_oi,
+ files = odb_source_files_downcast(ctx->r->objects->source);
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ odb_source_for_each_object(&dir->packed->base, &oi, add_packed_commits_oi,
ctx, &opts);
}
diff --git a/diagnose.c b/diagnose.c
index 89240e47d6..2c1bcfe431 100644
--- a/diagnose.c
+++ b/diagnose.c
@@ -228,7 +228,7 @@ int create_diagnostics_archive(struct repository *r,
strbuf_reset(&buf);
strbuf_addstr(&buf, "--add-virtual-file=packs-local.txt:");
- dir_file_stats(odb_source_files_downcast(r->objects->sources)->dirs, &buf);
+ dir_file_stats(odb_source_files_downcast(r->objects->source)->dirs, &buf);
odb_for_each_alternate(r->objects, dir_file_stats, &buf);
strvec_push(&archiver_args, buf.buf);
diff --git a/fetch-pack.c b/fetch-pack.c
index aad07b1153..b14e3ba71b 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -1075,7 +1075,7 @@ static int get_pack(struct fetch_pack_args *args,
die(_("fetch-pack: unable to fork off %s"), cmd_name);
if (do_keep && (pack_lockfiles || fsck_objects)) {
int is_well_formed;
- char *pack_lockfile = index_pack_lockfile(the_repository->objects->sources,
+ char *pack_lockfile = index_pack_lockfile(the_repository->objects->source,
cmd.out,
&is_well_formed);
diff --git a/http-walker.c b/http-walker.c
index 0a6c99f471..4f50dd9b8d 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -540,7 +540,7 @@ static int fetch_object(struct walker *walker, const struct object_id *oid)
} else if (!oideq(&obj_req->oid, &req->real_oid)) {
ret = error("File %s has bad hash", hex);
} else if (req->rename < 0) {
- struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->sources);
+ struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->source);
struct strbuf buf = STRBUF_INIT;
odb_loose_path(files->dirs->loose, &buf, &req->oid);
ret = error("unable to write sha1 filename %s", buf.buf);
diff --git a/http.c b/http.c
index fe6ec88a21..230c4de494 100644
--- a/http.c
+++ b/http.c
@@ -2718,7 +2718,7 @@ int finish_http_pack_request(struct http_pack_request *preq)
void http_install_packfile(struct packed_git *p,
struct packfile_list *list_to_remove_from)
{
- struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->sources);
+ struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->source);
packfile_list_remove(list_to_remove_from, p);
packfile_store_add_pack(files->dirs->packed, p);
}
@@ -2846,7 +2846,7 @@ static size_t fwrite_sha1_file(char *ptr, size_t eltsize, size_t nmemb,
struct http_object_request *new_http_object_request(const char *base_url,
const struct object_id *oid)
{
- struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->sources);
+ struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->source);
char *hex = oid_to_hex(oid);
struct strbuf filename = STRBUF_INIT;
struct strbuf prevfile = STRBUF_INIT;
@@ -2987,7 +2987,7 @@ void process_http_object_request(struct http_object_request *freq)
int finish_http_object_request(struct http_object_request *freq)
{
- struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->sources);
+ struct odb_source_files *files = odb_source_files_downcast(the_repository->objects->source);
struct stat st;
struct strbuf filename = STRBUF_INIT;
diff --git a/loose.c b/loose.c
index 957bf83e6b..d6a2bb5ba3 100644
--- a/loose.c
+++ b/loose.c
@@ -113,11 +113,10 @@ int loose_object_map_load(struct odb_source_loose *loose)
int repo_read_loose_object_map(struct repository *repo)
{
- struct odb_source *source;
+ struct odb_source_files *files = odb_source_files_downcast(repo->objects->source);
- for (source = repo->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- if (loose_object_map_load(files->dirs->loose) < 0)
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ if (loose_object_map_load(dir->loose) < 0)
return -1;
}
@@ -126,7 +125,7 @@ int repo_read_loose_object_map(struct repository *repo)
int repo_write_loose_object_map(struct repository *repo)
{
- struct odb_source_files *files = odb_source_files_downcast(repo->objects->sources);
+ struct odb_source_files *files = odb_source_files_downcast(repo->objects->source);
kh_oid_map_t *map = files->dirs->loose->map->to_compat;
struct lock_file lock;
int fd;
@@ -231,13 +230,12 @@ int repo_loose_object_map_oid(struct repository *repo,
const struct git_hash_algo *to,
struct object_id *dest)
{
- struct odb_source *source;
+ struct odb_source_files *files = odb_source_files_downcast(repo->objects->source);
kh_oid_map_t *map;
khiter_t pos;
- for (source = repo->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- struct loose_object_map *loose_map = files->dirs->loose->map;
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ struct loose_object_map *loose_map = dir->loose->map;
if (!loose_map)
continue;
map = (to == repo->compat_hash_algo) ?
diff --git a/midx.c b/midx.c
index c0f82c4163..8638ddf0be 100644
--- a/midx.c
+++ b/midx.c
@@ -829,21 +829,15 @@ void clear_incremental_midx_files_ext(struct odb_source_packed *source, const ch
void clear_midx_file(struct repository *r)
{
- struct odb_source_files *files;
+ struct odb_source_files *files = odb_source_files_downcast(r->objects->source);
struct strbuf midx = STRBUF_INIT;
- if (r->objects) {
- struct odb_source *source;
-
- for (source = r->objects->sources; source; source = source->next) {
- files = odb_source_files_downcast(source);
- if (files->dirs->packed->midx)
- close_midx(files->dirs->packed->midx);
- files->dirs->packed->midx = NULL;
- }
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ if (dir->packed->midx)
+ close_midx(dir->packed->midx);
+ dir->packed->midx = NULL;
}
- files = odb_source_files_downcast(r->objects->sources);
get_midx_filename(files->dirs->packed, &midx);
if (remove_path(midx.buf))
@@ -858,18 +852,15 @@ void clear_midx_file(struct repository *r)
void clear_incremental_midx_files(struct repository *r,
const struct strvec *keep_hashes)
{
- struct odb_source_files *files;
- struct odb_source *source;
+ struct odb_source_files *files = odb_source_files_downcast(r->objects->source);
struct strbuf chain = STRBUF_INIT;
- for (source = r->objects->sources; source; source = source->next) {
- files = odb_source_files_downcast(source);
- if (files->dirs->packed->midx)
- close_midx(files->dirs->packed->midx);
- files->dirs->packed->midx = NULL;
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ if (dir->packed->midx)
+ close_midx(dir->packed->midx);
+ dir->packed->midx = NULL;
}
- files = odb_source_files_downcast(r->objects->sources);
get_midx_chain_filename(files->dirs->packed, &chain);
if (!keep_hashes && remove_path(chain.buf))
diff --git a/odb.c b/odb.c
index 8b54271c27..98c5c862f5 100644
--- a/odb.c
+++ b/odb.c
@@ -13,11 +13,10 @@
#include "object-file.h"
#include "object-name.h"
#include "odb.h"
-#include "odb/source-inmemory.h"
#include "odb/source-files.h"
+#include "odb/source-inmemory.h"
#include "path.h"
#include "promisor-remote.h"
-#include "quote.h"
#include "replace-object.h"
#include "run-command.h"
#include "setup.h"
@@ -28,48 +27,6 @@
#include "trace2.h"
#include "write-or-die.h"
-/*
- * NEEDSWORK: we're using "core.ignoreCase" to deduplicate alternates that
- * _may_ be the same. This requires quite a bit of boilerplate for dubious
- * benefit:
- *
- * - Duplicating alternates should really only lead to regressed performance.
- *
- * - We don't properly resolve symlinks or mointpoints, so we may still end
- * up duplicating alternates.
- *
- * - The value may be lying, in which case we might deduplicate alternates
- * that are in fact not mapping to the same directory.
- *
- * We should investigate whether we can remove this whole mechanism outright.
- */
-static int odb_source_paths_cmp(struct object_database *o,
- const char *a, const char *b)
-{
- if (o->source_paths_icase < 0) {
- int icase = 0;
- repo_config_get_bool(o->repo, "core.ignorecase", &icase);
- o->source_paths_icase = icase;
- }
-
- return o->source_paths_icase ? strcasecmp(a, b) : strcmp(a, b);
-}
-
-static int odb_source_by_path_cmp(const void *cb_data,
- const struct hashmap_entry *entry,
- const struct hashmap_entry *entry_or_key,
- const void *keydata)
-{
- struct object_database *o = (struct object_database *)cb_data;
- const struct odb_source *source = container_of(entry, const struct odb_source, by_path_entry);
- const char *path = keydata;
-
- if (!path)
- path = container_of(entry_or_key, const struct odb_source, by_path_entry)->path;
-
- return odb_source_paths_cmp(o, source->path, path);
-}
-
int odb_mkstemp(struct object_database *odb,
struct strbuf *temp_filename, const char *pattern)
{
@@ -91,154 +48,6 @@ int odb_mkstemp(struct object_database *odb,
return xmkstemp_mode(temp_filename->buf, mode);
}
-/*
- * Return non-zero iff the path is usable as an alternate object database.
- */
-static bool odb_is_source_usable(struct object_database *o, const char *path)
-{
- struct strbuf normalized_objdir = STRBUF_INIT;
- struct hashmap_entry key;
- bool usable = false;
-
- strbuf_realpath(&normalized_objdir, o->sources->path, 1);
-
- /* Detect cases where alternate disappeared */
- if (!is_directory(path)) {
- error(_("object directory %s does not exist; "
- "check .git/objects/info/alternates"),
- path);
- goto out;
- }
-
- /*
- * Prevent the common mistake of listing the same
- * thing twice, or object directory itself.
- */
- if (!hashmap_get_size(&o->source_by_path)) {
- assert(!o->sources->next);
- hashmap_entry_init(&o->sources->by_path_entry,
- strihash(o->sources->path));
- hashmap_add(&o->source_by_path, &o->sources->by_path_entry);
- }
-
- if (!odb_source_paths_cmp(o, path, normalized_objdir.buf))
- goto out;
-
- hashmap_entry_init(&key, strihash(path));
- if (hashmap_get(&o->source_by_path, &key, path))
- goto out;
-
- usable = true;
-
-out:
- strbuf_release(&normalized_objdir);
- return usable;
-}
-
-void parse_alternates(const char *string,
- int sep,
- const char *relative_base,
- struct strvec *out)
-{
- struct strbuf pathbuf = STRBUF_INIT;
- struct strbuf buf = STRBUF_INIT;
-
- if (!string || !*string)
- return;
-
- while (*string) {
- const char *end;
-
- strbuf_reset(&buf);
- strbuf_reset(&pathbuf);
-
- if (*string == '#') {
- /* comment; consume up to next separator */
- end = strchrnul(string, sep);
- } else if (*string == '"' && !unquote_c_style(&buf, string, &end)) {
- /*
- * quoted path; unquote_c_style has copied the
- * data for us and set "end". Broken quoting (e.g.,
- * an entry that doesn't end with a quote) falls
- * back to the unquoted case below.
- */
- } else {
- /* normal, unquoted path */
- end = strchrnul(string, sep);
- strbuf_add(&buf, string, end - string);
- }
-
- if (*end)
- end++;
- string = end;
-
- if (!buf.len)
- continue;
-
- if (!is_absolute_path(buf.buf) && relative_base) {
- strbuf_realpath(&pathbuf, relative_base, 1);
- strbuf_addch(&pathbuf, '/');
- }
- strbuf_addbuf(&pathbuf, &buf);
-
- strbuf_reset(&buf);
- if (!strbuf_realpath(&buf, pathbuf.buf, 0)) {
- error(_("unable to normalize alternate object path: %s"),
- pathbuf.buf);
- continue;
- }
-
- /*
- * The trailing slash after the directory name is given by
- * this function at the end. Remove duplicates.
- */
- while (buf.len && buf.buf[buf.len - 1] == '/')
- strbuf_setlen(&buf, buf.len - 1);
-
- strvec_push(out, buf.buf);
- }
-
- strbuf_release(&pathbuf);
- strbuf_release(&buf);
-}
-
-static struct odb_source *odb_add_alternate_recursively(struct object_database *odb,
- const char *source,
- int depth)
-{
- struct odb_source *alternate = NULL;
- struct strvec sources = STRVEC_INIT;
-
- if (!odb_is_source_usable(odb, source))
- goto error;
-
- alternate = odb_source_new(odb, source, false);
-
- /* add the alternate entry */
- *odb->sources_tail = alternate;
- odb->sources_tail = &(alternate->next);
-
- hashmap_entry_init(&alternate->by_path_entry, strihash(alternate->path));
- if (hashmap_get(&odb->source_by_path, &alternate->by_path_entry,
- alternate->path))
- BUG("source must not yet exist");
- hashmap_add(&odb->source_by_path, &alternate->by_path_entry);
-
- /* recursively add alternates */
- odb_source_read_alternates(alternate, &sources);
- if (sources.nr && depth + 1 > 5) {
- error(_("%s: ignoring alternate object stores, nesting too deep"),
- source);
- } else {
- for (size_t i = 0; i < sources.nr; i++)
- odb_add_alternate_recursively(odb, sources.v[i], depth + 1);
- }
-
- error:
- strvec_clear(&sources);
- return alternate;
-}
-
char *compute_alternate_path(const char *path, struct strbuf *err)
{
char *ref_git = NULL;
@@ -410,34 +219,22 @@ void odb_for_each_alternate_ref(struct object_database *odb,
int odb_for_each_alternate(struct object_database *odb,
odb_for_each_alternate_fn cb, void *payload)
{
- struct odb_source *alternate;
+ struct odb_source_files *files = odb_source_files_downcast(odb->source);
int r = 0;
- for (alternate = odb->sources->next; alternate; alternate = alternate->next) {
- r = cb(odb_source_files_downcast(alternate)->dirs, payload);
+ for (struct odb_files_dir *dir = files->dirs->next; dir; dir = dir->next) {
+ r = cb(dir, payload);
if (r)
break;
}
return r;
}
-static void odb_prepare_alternates(struct object_database *odb,
- const char *alternate_db)
-{
- struct strvec sources = STRVEC_INIT;
-
- parse_alternates(alternate_db, PATH_SEP, NULL, &sources);
- odb_source_read_alternates(odb->sources, &sources);
-
- for (size_t i = 0; i < sources.nr; i++)
- odb_add_alternate_recursively(odb, sources.v[i], 0);
-
- strvec_clear(&sources);
-}
-
int odb_has_alternates(struct object_database *odb)
{
- return !!odb->sources->next;
+ if (odb->source->type != ODB_SOURCE_FILES)
+ return 0;
+ return !!odb_source_files_downcast(odb->source)->dirs->next;
}
int obj_read_use_lock = 0;
@@ -481,16 +278,12 @@ static enum odb_read_status do_oid_object_info_extended(struct object_database *
return 0;
while (1) {
- struct odb_source *source;
-
- for (source = odb->sources; source; source = source->next) {
- ret = odb_source_read_object_info(source, real, oi, flags,
- corrupt_err.len ? NULL : &corrupt_err);
- if (!ret)
- goto out;
- if (ret != ODB_READ_NOT_FOUND)
- corrupt = true;
- }
+ ret = odb_source_read_object_info(odb->source, real, oi, flags,
+ corrupt_err.len ? NULL : &corrupt_err);
+ if (!ret)
+ goto out;
+ if (ret != ODB_READ_NOT_FOUND)
+ corrupt = true;
/*
* When the object hasn't been found we try a second read and
@@ -498,15 +291,13 @@ static enum odb_read_status do_oid_object_info_extended(struct object_database *
* caches or reload on-disk state.
*/
if (!(flags & OBJECT_INFO_QUICK)) {
- for (source = odb->sources; source; source = source->next) {
- ret = odb_source_read_object_info(source, real, oi,
- flags | OBJECT_INFO_SECOND_READ,
- corrupt_err.len ? NULL : &corrupt_err);
- if (!ret)
- goto out;
- if (ret != ODB_READ_NOT_FOUND)
- corrupt = true;
- }
+ ret = odb_source_read_object_info(odb->source, real, oi,
+ flags | OBJECT_INFO_SECOND_READ,
+ corrupt_err.len ? NULL : &corrupt_err);
+ if (!ret)
+ goto out;
+ if (ret != ODB_READ_NOT_FOUND)
+ corrupt = true;
}
/* Check if it is a missing object */
@@ -748,11 +539,7 @@ int odb_has_object(struct object_database *odb, const struct object_id *oid,
int odb_freshen_object(struct object_database *odb,
const struct object_id *oid)
{
- struct odb_source *source;
- for (source = odb->sources; source; source = source->next)
- if (odb_source_freshen_object(source, oid, NULL))
- return 1;
- return 0;
+ return odb_source_freshen_object(odb->source, oid, NULL);
}
int odb_for_each_object_ext(struct object_database *odb,
@@ -761,18 +548,7 @@ int odb_for_each_object_ext(struct object_database *odb,
void *cb_data,
const struct odb_for_each_object_options *opts)
{
- int ret;
-
- for (struct odb_source *source = odb->sources; source; source = source->next) {
- if (opts->flags & ODB_FOR_EACH_OBJECT_LOCAL_ONLY && !source->local)
- continue;
-
- ret = odb_source_for_each_object(source, request, cb, cb_data, opts);
- if (ret)
- return ret;
- }
-
- return 0;
+ return odb_source_for_each_object(odb->source, request, cb, cb_data, opts);
}
int odb_for_each_object(struct object_database *odb,
@@ -791,7 +567,6 @@ int odb_count_objects(struct object_database *odb,
enum odb_count_objects_flags flags,
unsigned long *out)
{
- struct odb_source *source;
unsigned long count = 0;
int ret;
@@ -800,15 +575,9 @@ int odb_count_objects(struct object_database *odb,
return 0;
}
- for (source = odb->sources; source; source = source->next) {
- unsigned long c;
-
- ret = odb_source_count_objects(source, flags, &c);
- if (ret < 0)
- goto out;
-
- count += c;
- }
+ ret = odb_source_count_objects(odb->source, flags, &count);
+ if (ret < 0)
+ goto out;
odb->object_count = count;
odb->object_count_valid = 1;
@@ -879,13 +648,7 @@ int odb_find_abbrev_len(struct object_database *odb,
goto out;
}
- for (struct odb_source *source = odb->sources; source; source = source->next) {
- ret = odb_source_find_abbrev_len(source, oid, len, &len);
- if (ret)
- goto out;
- }
-
- ret = 0;
+ ret = odb_source_find_abbrev_len(odb->source, oid, len, &len);
*out = len;
out:
@@ -941,7 +704,7 @@ int odb_write_object_ext(struct object_database *odb,
compat_oid_p = &compat_oid;
}
- return odb_source_write_object(odb->sources, buf, len, type,
+ return odb_source_write_object(odb->source, buf, len, type,
oid, compat_oid_p, NULL, flags);
}
@@ -949,19 +712,19 @@ int odb_write_object_stream(struct object_database *odb,
struct odb_stream *stream,
struct object_id *oid)
{
- return odb_source_write_object_stream(odb->sources, stream, oid);
+ return odb_source_write_object_stream(odb->source, stream, oid);
}
int odb_optimize(struct object_database *odb,
const struct odb_optimize_options *opts)
{
- return odb_source_optimize(odb->sources, opts);
+ return odb_source_optimize(odb->source, opts);
}
bool odb_optimize_required(struct object_database *odb,
const struct odb_optimize_options *opts)
{
- return odb_source_optimize_required(odb->sources, opts);
+ return odb_source_optimize_required(odb->source, opts);
}
void odb_generate_pack_options_release(struct odb_generate_pack_options *opts)
@@ -975,9 +738,9 @@ int odb_generate_pack(struct object_database *odb,
struct odb_pack_generator **out,
const struct odb_generate_pack_options *opts)
{
- if (!odb->sources->generate_pack)
+ if (!odb->source->generate_pack)
return error(_("primary object source does not support generating packfiles"));
- return odb_source_generate_pack(odb->sources, out, opts);
+ return odb_source_generate_pack(odb->source, out, opts);
}
int odb_pack_generator_finish(struct odb_pack_generator *generator)
@@ -988,55 +751,29 @@ int odb_pack_generator_finish(struct odb_pack_generator *generator)
struct object_database *odb_new(struct repository *repo,
enum odb_new_flags flags)
{
- char *primary_source = NULL, *secondary_sources = NULL;
struct object_database *o;
CALLOC_ARRAY(o, 1);
o->repo = repo;
pthread_mutex_init(&o->replace_mutex, NULL);
- hashmap_init(&o->source_by_path, odb_source_by_path_cmp, o, 0);
- o->source_paths_icase = -1;
-
- if (flags & ODB_NEW_HONOR_ENV) {
- primary_source = xstrdup_or_null(getenv(DB_ENVIRONMENT));
- secondary_sources = xstrdup_or_null(getenv(ALTERNATE_DB_ENVIRONMENT));
- }
- if (!primary_source)
- primary_source = xstrfmt("%s/objects", repo->commondir);
- o->sources = odb_source_new(o, primary_source, true);
- o->sources_tail = &o->sources->next;
+ o->source = odb_source_new(o, flags);
o->inmemory_objects = &odb_source_inmemory_new(o)->base;
- odb_prepare_alternates(o, secondary_sources);
-
- free(secondary_sources);
- free(primary_source);
return o;
}
void odb_close(struct object_database *o)
{
- struct odb_source *source;
- for (source = o->sources; source; source = source->next)
- odb_source_close(source);
+ odb_source_close(o->source);
close_commit_graph(o);
}
static void odb_free_sources(struct object_database *o)
{
- while (o->sources) {
- struct odb_source *next;
-
- next = o->sources->next;
- odb_source_free(o->sources);
- o->sources = next;
- }
-
+ odb_source_free(o->source);
odb_source_free(o->inmemory_objects);
o->inmemory_objects = NULL;
-
- hashmap_clear(&o->source_by_path);
}
void odb_free(struct object_database *o)
@@ -1055,24 +792,12 @@ void odb_free(struct object_database *o)
void odb_prepare(struct object_database *o, enum odb_prepare_flags flags)
{
- struct odb_source *source;
-
obj_read_lock();
- /*
- * Reprepare alt odbs, in case the alternates file was modified
- * during the course of this process. This only _adds_ odbs to
- * the linked list, so existing odbs will continue to exist for
- * the lifetime of the process. Consequently, we don't have to
- * reprocess GIT_ALTERNATE_OBJECT_DIRECTORIES here.
- */
- if (flags & ODB_PREPARE_FLUSH_CACHES) {
- odb_prepare_alternates(o, NULL);
+ if (flags & ODB_PREPARE_FLUSH_CACHES)
o->object_count_valid = 0;
- }
- for (source = o->sources; source; source = source->next)
- odb_source_prepare(source, flags);
+ odb_source_prepare(o->source, flags);
obj_read_unlock();
}
@@ -1084,8 +809,5 @@ void odb_reprepare(struct object_database *o)
int odb_fsck(struct object_database *odb, struct odb_fsck_options *options)
{
- int ret = 0;
- for (struct odb_source *source = odb->sources; source; source = source->next)
- ret |= odb_source_fsck(source, options);
- return ret;
+ return odb_source_fsck(odb->source, options);
}
diff --git a/odb.h b/odb.h
index 4143812f55..0ccc47ee73 100644
--- a/odb.h
+++ b/odb.h
@@ -28,13 +28,16 @@ char *compute_alternate_path(const char *path, struct strbuf *err);
/*
* The object database encapsulates access to objects in a repository. It
- * manages one or more sources that store the actual objects which are
- * configured via alternates.
+ * manages the object source as well as auxiliary data structures required to
+ * manage objects.
*/
struct object_database {
/* Repository that owns this database. */
struct repository *repo;
+ /* The source backing this object database. */
+ struct odb_source *source;
+
/*
* State of current object database transaction. Only one
* transaction may be pending at a time. Is NULL when no transaction is
@@ -42,27 +45,6 @@ struct object_database {
*/
struct odb_transaction *transaction;
- /*
- * Set of all object directories; the main directory is first (and
- * cannot be NULL after initialization). Subsequent directories are
- * alternates.
- */
- struct odb_source *sources;
- struct odb_source **sources_tail;
-
- /*
- * Map of object database sources, keyed by their respective paths.
- * This map is used to detect the case where the same source is
- * registered multiple times.
- */
- struct hashmap source_by_path;
-
- /*
- * Whether source paths shall be compared case-insensitively, as
- * determined by "core.ignoreCase".
- */
- int source_paths_icase;
-
/*
* Objects that should be substituted by other objects
* (see git-replace(1)).
@@ -822,9 +804,4 @@ int odb_generate_pack(struct object_database *odb,
*/
int odb_pack_generator_finish(struct odb_pack_generator *generator);
-void parse_alternates(const char *string,
- int sep,
- const char *relative_base,
- struct strvec *out);
-
#endif /* ODB_H */
diff --git a/odb/source-files.c b/odb/source-files.c
index 6aaf625352..072f515b36 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -15,6 +15,7 @@
#include "packfile.h"
#include "path.h"
#include "promisor-remote.h"
+#include "quote.h"
#include "repack.h"
#include "run-command.h"
#include "strbuf.h"
@@ -71,6 +72,7 @@ static void odb_source_files_free(struct odb_source *source)
odb_files_dir_free(files->dirs);
files->dirs = next;
}
+ hashmap_clear(&files->dirs_by_path);
odb_source_release(&files->base);
free(files);
@@ -174,6 +176,160 @@ static int odb_source_files_create_on_disk(struct odb_source *source,
return ret;
}
+/*
+ * NEEDSWORK: we're using "core.ignoreCase" to deduplicate alternates that
+ * _may_ be the same. This requires quite a bit of boilerplate for dubious
+ * benefit:
+ *
+ * - Duplicating alternates should really only lead to regressed performance.
+ *
+ * - We don't properly resolve symlinks or mointpoints, so we may still end
+ * up duplicating alternates.
+ *
+ * - The value may be lying, in which case we might deduplicate alternates
+ * that are in fact not mapping to the same directory.
+ *
+ * We should investigate whether we can remove this whole mechanism outright.
+ */
+static int odb_files_dir_paths_cmp(struct odb_source_files *files,
+ const char *a, const char *b)
+{
+ if (files->dirs_paths_icase < 0) {
+ int icase = 0;
+ repo_config_get_bool(files->base.odb->repo, "core.ignorecase", &icase);
+ files->dirs_paths_icase = icase;
+ }
+
+ return files->dirs_paths_icase ? strcasecmp(a, b) : strcmp(a, b);
+}
+
+static int odb_files_dir_by_path_cmp(const void *cb_data,
+ const struct hashmap_entry *entry,
+ const struct hashmap_entry *entry_or_key,
+ const void *keydata)
+{
+ struct odb_source_files *files = (struct odb_source_files *)cb_data;
+ const struct odb_files_dir *dir = container_of(entry, const struct odb_files_dir, by_path_entry);
+ const char *path = keydata;
+
+ if (!path)
+ path = container_of(entry_or_key, const struct odb_files_dir, by_path_entry)->abspath;
+
+ return odb_files_dir_paths_cmp(files, dir->abspath, path);
+}
+
+/*
+ * Return non-zero iff the path is usable as an alternate object directory.
+ */
+static bool odb_files_dir_is_usable(struct odb_source_files *files,
+ const char *path)
+{
+ struct strbuf normalized_objdir = STRBUF_INIT;
+ struct hashmap_entry key;
+ bool usable = false;
+
+ strbuf_realpath(&normalized_objdir, files->dirs->abspath, 1);
+
+ /* Detect cases where alternate disappeared */
+ if (!is_directory(path)) {
+ error(_("object directory %s does not exist; "
+ "check .git/objects/info/alternates"),
+ path);
+ goto out;
+ }
+
+ /*
+ * Prevent the common mistake of listing the same
+ * thing twice, or object directory itself.
+ */
+ if (!hashmap_get_size(&files->dirs_by_path)) {
+ assert(!files->dirs->next);
+ hashmap_entry_init(&files->dirs->by_path_entry,
+ strihash(files->dirs->abspath));
+ hashmap_add(&files->dirs_by_path, &files->dirs->by_path_entry);
+ }
+
+ if (!odb_files_dir_paths_cmp(files, path, normalized_objdir.buf))
+ goto out;
+
+ hashmap_entry_init(&key, strihash(path));
+ if (hashmap_get(&files->dirs_by_path, &key, path))
+ goto out;
+
+ usable = true;
+
+out:
+ strbuf_release(&normalized_objdir);
+ return usable;
+}
+
+static void parse_alternates(const char *string,
+ int sep,
+ const char *relative_base,
+ struct strvec *out)
+{
+ struct strbuf pathbuf = STRBUF_INIT;
+ struct strbuf buf = STRBUF_INIT;
+
+ if (!string || !*string)
+ return;
+
+ while (*string) {
+ const char *end;
+
+ strbuf_reset(&buf);
+ strbuf_reset(&pathbuf);
+
+ if (*string == '#') {
+ /* comment; consume up to next separator */
+ end = strchrnul(string, sep);
+ } else if (*string == '"' && !unquote_c_style(&buf, string, &end)) {
+ /*
+ * quoted path; unquote_c_style has copied the
+ * data for us and set "end". Broken quoting (e.g.,
+ * an entry that doesn't end with a quote) falls
+ * back to the unquoted case below.
+ */
+ } else {
+ /* normal, unquoted path */
+ end = strchrnul(string, sep);
+ strbuf_add(&buf, string, end - string);
+ }
+
+ if (*end)
+ end++;
+ string = end;
+
+ if (!buf.len)
+ continue;
+
+ if (!is_absolute_path(buf.buf) && relative_base) {
+ strbuf_realpath(&pathbuf, relative_base, 1);
+ strbuf_addch(&pathbuf, '/');
+ }
+ strbuf_addbuf(&pathbuf, &buf);
+
+ strbuf_reset(&buf);
+ if (!strbuf_realpath(&buf, pathbuf.buf, 0)) {
+ error(_("unable to normalize alternate object path: %s"),
+ pathbuf.buf);
+ continue;
+ }
+
+ /*
+ * The trailing slash after the directory name is given by
+ * this function at the end. Remove duplicates.
+ */
+ while (buf.len && buf.buf[buf.len - 1] == '/')
+ strbuf_setlen(&buf, buf.len - 1);
+
+ strvec_push(out, buf.buf);
+ }
+
+ strbuf_release(&pathbuf);
+ strbuf_release(&buf);
+}
+
static int read_alternates(const char *object_dir, struct strvec *out)
{
struct strbuf buf = STRBUF_INIT;
@@ -192,11 +348,70 @@ static int read_alternates(const char *object_dir, struct strvec *out)
return 0;
}
+static void odb_add_alternate_recursively(struct odb_source_files *files,
+ const char *path,
+ int depth)
+{
+ struct odb_files_dir *alternate;
+ struct strvec alternates = STRVEC_INIT;
+
+ if (!odb_files_dir_is_usable(files, path))
+ goto out;
+
+ alternate = odb_files_dir_new(files->base.odb, path, false);
+
+ /* add the alternate entry */
+ *files->dirs_tail = alternate;
+ files->dirs_tail = &(alternate->next);
+
+ hashmap_entry_init(&alternate->by_path_entry, strihash(alternate->abspath));
+ if (hashmap_get(&files->dirs_by_path, &alternate->by_path_entry,
+ alternate->abspath))
+ BUG("object directory must not yet exist");
+ hashmap_add(&files->dirs_by_path, &alternate->by_path_entry);
+
+ /* recursively add alternates */
+ read_alternates(alternate->abspath, &alternates);
+ if (alternates.nr && depth + 1 > 5) {
+ error(_("%s: ignoring alternate object stores, nesting too deep"),
+ path);
+ } else {
+ for (size_t i = 0; i < alternates.nr; i++)
+ odb_add_alternate_recursively(files, alternates.v[i], depth + 1);
+ }
+
+ out:
+ strvec_clear(&alternates);
+}
+
+static void odb_prepare_alternates(struct odb_source_files *files,
+ const char *alternate_db)
+{
+ struct strvec alternates = STRVEC_INIT;
+
+ parse_alternates(alternate_db, PATH_SEP, NULL, &alternates);
+ read_alternates(files->dirs->abspath, &alternates);
+
+ for (size_t i = 0; i < alternates.nr; i++)
+ odb_add_alternate_recursively(files, alternates.v[i], 0);
+
+ strvec_clear(&alternates);
+}
+
static void odb_source_files_prepare(struct odb_source *source,
enum odb_prepare_flags flags)
{
struct odb_source_files *files = odb_source_files_downcast(source);
+ /*
+ * Reprepare alternates, in case the alternates file was modified
+ * during the course of this process. This only _adds_ directories to
+ * the linked list, so existing directories will continue to exist
+ * for the lifetime of the process.
+ */
+ if (flags & ODB_PREPARE_FLUSH_CACHES)
+ odb_prepare_alternates(files, NULL);
+
for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
odb_source_prepare(&dir->loose->base, flags);
odb_source_prepare(&dir->packed->base, flags);
@@ -1009,23 +1224,15 @@ static int odb_source_files_fsck(struct odb_source *source,
struct odb_files_dir *odb_source_files_find_dir(struct object_database *odb, const char *obj_dir)
{
+ struct odb_source_files *files = odb_source_files_downcast(odb->source);
char *obj_dir_real = real_pathdup(obj_dir, 1);
struct strbuf odb_path_real = STRBUF_INIT;
- struct odb_files_dir *dir = NULL;
- struct odb_source *source;
-
- for (source = odb->sources; source; source = source->next) {
- struct odb_source_files *files;
-
- if (source->type != ODB_SOURCE_FILES)
- continue;
- files = odb_source_files_downcast(source);
+ struct odb_files_dir *dir;
- strbuf_realpath(&odb_path_real, files->dirs->abspath, 1);
- if (!strcmp(obj_dir_real, odb_path_real.buf)) {
- dir = files->dirs;
+ for (dir = files->dirs; dir; dir = dir->next) {
+ strbuf_realpath(&odb_path_real, dir->abspath, 1);
+ if (!strcmp(obj_dir_real, odb_path_real.buf))
break;
- }
}
free(obj_dir_real);
@@ -1034,14 +1241,27 @@ struct odb_files_dir *odb_source_files_find_dir(struct object_database *odb, con
}
struct odb_source_files *odb_source_files_new(struct object_database *odb,
- const char *path,
- bool local)
+ enum odb_new_flags flags)
{
struct odb_source_files *files;
+ char *object_dir = NULL;
+ char *alternates = NULL;
+
+ if (flags & ODB_NEW_HONOR_ENV) {
+ object_dir = xstrdup_or_null(getenv(DB_ENVIRONMENT));
+ alternates = xstrdup_or_null(getenv(ALTERNATE_DB_ENVIRONMENT));
+ }
+ if (!object_dir)
+ object_dir = xstrfmt("%s/objects", odb->repo->commondir);
CALLOC_ARRAY(files, 1);
- odb_source_init(&files->base, odb, ODB_SOURCE_FILES, path, local);
- files->dirs = odb_files_dir_new(odb, path, local);
+ odb_source_init(&files->base, odb, ODB_SOURCE_FILES, object_dir, true);
+
+ hashmap_init(&files->dirs_by_path, odb_files_dir_by_path_cmp, files, 0);
+ files->dirs_paths_icase = -1;
+
+ files->dirs = odb_files_dir_new(odb, object_dir, true);
+ files->dirs_tail = &files->dirs->next;
files->base.free = odb_source_files_free;
files->base.close = odb_source_files_close;
@@ -1067,8 +1287,12 @@ struct odb_source_files *odb_source_files_new(struct object_database *odb,
* is not (yet) possible though because we access and assume relative
* paths in the primary ODB source in some user-facing functionality.
*/
- if (!is_absolute_path(path))
+ if (!is_absolute_path(object_dir))
chdir_notify_register(odb_source_files_reparent, files);
+ odb_prepare_alternates(files, alternates);
+
+ free(object_dir);
+ free(alternates);
return files;
}
diff --git a/odb/source-files.h b/odb/source-files.h
index 36af0c1b8b..837e890304 100644
--- a/odb/source-files.h
+++ b/odb/source-files.h
@@ -18,6 +18,12 @@ struct odb_files_dir {
/* List of alternate object directories. */
struct odb_files_dir *next;
+ /*
+ * Entry in the files source's map of directories, keyed by this
+ * directory's path.
+ */
+ struct hashmap_entry by_path_entry;
+
/* The two sources derived from this object directory. */
struct odb_source_loose *loose;
struct odb_source_packed *packed;
@@ -46,12 +52,25 @@ struct odb_source_files {
* alternates.
*/
struct odb_files_dir *dirs;
+ struct odb_files_dir **dirs_tail;
+
+ /*
+ * Map of object directories, keyed by their respective paths. This
+ * map is used to detect the case where the same directory is
+ * registered multiple times.
+ */
+ struct hashmap dirs_by_path;
+
+ /*
+ * Whether directory paths shall be compared case-insensitively, as
+ * determined by "core.ignoreCase".
+ */
+ int dirs_paths_icase;
};
/* Allocate and initialize a new object source. */
struct odb_source_files *odb_source_files_new(struct object_database *odb,
- const char *path,
- bool local);
+ enum odb_new_flags flags);
/*
* Optimize the files object database source by repacking loose objects and
diff --git a/odb/source.c b/odb/source.c
index 30188b806d..b25ef14df8 100644
--- a/odb/source.c
+++ b/odb/source.c
@@ -24,10 +24,9 @@ const char *odb_source_type_to_name(enum odb_source_type type)
}
struct odb_source *odb_source_new(struct object_database *odb,
- const char *path,
- bool local)
+ enum odb_new_flags flags)
{
- return &odb_source_files_new(odb, path, local)->base;
+ return &odb_source_files_new(odb, flags)->base;
}
void odb_source_init(struct odb_source *source,
diff --git a/odb/source.h b/odb/source.h
index 9fd2b2e5b5..6718aced6a 100644
--- a/odb/source.h
+++ b/odb/source.h
@@ -49,24 +49,9 @@ struct odb_create_on_disk_options {
/*
* The source is the part of the object database that stores the actual
* objects. It thus encapsulates the logic to read and write the specific
- * on-disk format. An object database can have multiple sources:
- *
- * - The primary source, which is typically located in "$GIT_DIR/objects".
- * This is where new objects are usually written to.
- *
- * - Alternate sources, which are configured via "objects/info/alternates" or
- * via the GIT_ALTERNATE_OBJECT_DIRECTORIES environment variable. These
- * alternate sources are only used to read objects.
+ * on-disk format.
*/
struct odb_source {
- struct odb_source *next;
-
- /*
- * Entry in the object database's map of sources, keyed by this
- * source's path.
- */
- struct hashmap_entry by_path_entry;
-
/* Object database that owns this object source. */
struct object_database *odb;
@@ -331,13 +316,11 @@ struct odb_source {
};
/*
- * Allocate and initialize a new source for the given object database located
- * at `path`. `local` indicates whether or not the source is the local and thus
- * primary object source of the object database.
+ * Allocate and initialize a new source for the given object database. The path
+ * of the source is derived from repository paths.
*/
struct odb_source *odb_source_new(struct object_database *odb,
- const char *path,
- bool local);
+ enum odb_new_flags flags);
/*
* Initialize the source for the given object database located at `path`.
diff --git a/odb/streaming.c b/odb/streaming.c
index 8f2143cab5..5ac172beb6 100644
--- a/odb/streaming.c
+++ b/odb/streaming.c
@@ -182,12 +182,8 @@ static int istream_source(struct odb_stream **out,
struct object_database *odb,
const struct object_id *oid)
{
- struct odb_source *source;
-
- for (source = odb->sources; source; source = source->next)
- if (!odb_source_read_object_stream(out, source, oid))
- return 0;
-
+ if (!odb_source_read_object_stream(out, odb->source, oid))
+ return 0;
return open_istream_incore(out, odb, oid);
}
diff --git a/odb/transaction.c b/odb/transaction.c
index f6f20088ec..69824a3921 100644
--- a/odb/transaction.c
+++ b/odb/transaction.c
@@ -12,7 +12,7 @@ int odb_transaction_begin(struct object_database *odb,
if (odb->transaction)
return error(_("object database transaction already pending"));
- ret = odb_source_begin_transaction(odb->sources, out, flags);
+ ret = odb_source_begin_transaction(odb->source, out, flags);
if (!ret)
odb->transaction = *out;
diff --git a/pack-bitmap.c b/pack-bitmap.c
index 52556b4543..d3c7997fb3 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -712,15 +712,13 @@ static int open_bitmap_for_source(struct odb_source_packed *source,
static int open_bitmap(struct repository *r,
struct bitmap_index *bitmap_git)
{
- struct odb_source *source;
+ struct odb_source_files *files = odb_source_files_downcast(r->objects->source);
bool found = false;
assert(!bitmap_git->map);
- for (source = r->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
-
- if (!open_bitmap_for_source(files->dirs->packed, bitmap_git))
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ if (!open_bitmap_for_source(dir->packed, bitmap_git))
found = true;
/*
diff --git a/packfile.c b/packfile.c
index 93b69d7f50..ebe430662e 100644
--- a/packfile.c
+++ b/packfile.c
@@ -273,14 +273,13 @@ static void scan_windows(struct packed_git *p,
static int unuse_one_window(struct object_database *odb)
{
- struct odb_source *source;
+ struct odb_source_files *files = odb_source_files_downcast(odb->source);
struct packfile_list_entry *e;
struct packed_git *lru_p = NULL;
struct pack_window *lru_w = NULL, *lru_l = NULL;
- for (source = odb->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- for (e = files->dirs->packed->packs.head; e; e = e->next)
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ for (e = dir->packed->packs.head; e; e = e->next)
scan_windows(e->pack, &lru_p, &lru_w, &lru_l);
}
@@ -450,15 +449,14 @@ static void find_lru_pack(struct packed_git *p, struct packed_git **lru_p, struc
static int close_one_pack(struct repository *r)
{
- struct odb_source *source;
+ struct odb_source_files *files = odb_source_files_downcast(r->objects->source);
struct packfile_list_entry *e;
struct packed_git *lru_p = NULL;
struct pack_window *mru_w = NULL;
int accept_windows_inuse = 1;
- for (source = r->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- for (e = files->dirs->packed->packs.head; e; e = e->next) {
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ for (e = dir->packed->packs.head; e; e = e->next) {
if (e->pack->pack_fd == -1)
continue;
find_lru_pack(e->pack, &lru_p, &mru_w, &accept_windows_inuse);
@@ -1921,11 +1919,10 @@ struct packed_git **packfile_store_get_kept_pack_cache(struct odb_source_packed
int has_object_pack(struct repository *r, const struct object_id *oid)
{
- struct odb_source *source;
+ struct odb_source_files *files = odb_source_files_downcast(r->objects->source);
- for (source = r->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- if (!odb_source_read_object_info(&files->dirs->packed->base, oid, NULL, 0, NULL))
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ if (!odb_source_read_object_info(&dir->packed->base, oid, NULL, 0, NULL))
return 1;
}
@@ -1935,14 +1932,13 @@ int has_object_pack(struct repository *r, const struct object_id *oid)
int has_object_kept_pack(struct repository *r, const struct object_id *oid,
unsigned flags)
{
- struct odb_source *source;
+ struct odb_source_files *files = odb_source_files_downcast(r->objects->source);
struct pack_entry e;
- for (source = r->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
struct packed_git **cache;
- cache = packfile_store_get_kept_pack_cache(files->dirs->packed, flags);
+ cache = packfile_store_get_kept_pack_cache(dir->packed, flags);
for (; *cache; cache++) {
struct packed_git *p = *cache;
diff --git a/packfile.h b/packfile.h
index fd1cf0ad6c..ed3378c457 100644
--- a/packfile.h
+++ b/packfile.h
@@ -69,20 +69,20 @@ void packfile_store_add_pack(struct odb_source_packed *store,
struct packfile_list_entry *packfile_store_get_packs(struct odb_source_packed *store);
struct repo_for_each_pack_data {
- struct odb_source *source;
+ struct odb_files_dir *dir;
struct packfile_list_entry *entry;
};
static inline struct repo_for_each_pack_data repo_for_eack_pack_data_init(struct repository *repo)
{
struct repo_for_each_pack_data data = { 0 };
+ struct odb_source_files *files = odb_source_files_downcast(repo->objects->source);
- for (struct odb_source *source = repo->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- struct packfile_list_entry *entry = packfile_store_get_packs(files->dirs->packed);
+ for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {
+ struct packfile_list_entry *entry = packfile_store_get_packs(dir->packed);
if (!entry)
continue;
- data.source = source;
+ data.dir = dir;
data.entry = entry;
break;
}
@@ -92,23 +92,22 @@ static inline struct repo_for_each_pack_data repo_for_eack_pack_data_init(struct
static inline void repo_for_each_pack_data_next(struct repo_for_each_pack_data *data)
{
- struct odb_source *source;
+ struct odb_files_dir *dir;
data->entry = data->entry->next;
if (data->entry)
return;
- for (source = data->source->next; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
- struct packfile_list_entry *entry = packfile_store_get_packs(files->dirs->packed);
+ for (dir = data->dir->next; dir; dir = dir->next) {
+ struct packfile_list_entry *entry = packfile_store_get_packs(dir->packed);
if (!entry)
continue;
- data->source = source;
+ data->dir = dir;
data->entry = entry;
return;
}
- data->source = NULL;
+ data->dir = NULL;
data->entry = NULL;
}
diff --git a/path.c b/path.c
index c3a709a928..4608a56927 100644
--- a/path.c
+++ b/path.c
@@ -396,7 +396,7 @@ static void adjust_git_path(struct repository *repo,
strbuf_splice(buf, 0, buf->len,
repo->index_file, strlen(repo->index_file));
else if (dir_prefix(base, "objects"))
- replace_dir(buf, git_dir_len + 7, repo->objects->sources->path);
+ replace_dir(buf, git_dir_len + 7, repo->objects->source->path);
else if (repo_settings_get_hooks_path(repo) && dir_prefix(base, "hooks"))
replace_dir(buf, git_dir_len + 5, repo_settings_get_hooks_path(repo));
else if (repo->different_commondir)
diff --git a/prune-packed.c b/prune-packed.c
index d49dc11957..b6749b8d13 100644
--- a/prune-packed.c
+++ b/prune-packed.c
@@ -40,7 +40,7 @@ void prune_packed_objects(int opts)
progress = start_delayed_progress(the_repository,
_("Removing duplicate objects"), 256);
- for_each_loose_file_in_source(the_repository->objects->sources,
+ for_each_loose_file_in_source(the_repository->objects->source,
prune_object, NULL, prune_subdir, &opts);
/* Ensure we show 100% before finishing progress */
diff --git a/repack.c b/repack.c
index e20431690c..53c34197f3 100644
--- a/repack.c
+++ b/repack.c
@@ -59,7 +59,7 @@ void repack_remove_redundant_pack(struct repository *repo, const char *dir_name,
bool wrote_incremental_midx)
{
struct strbuf buf = STRBUF_INIT;
- struct odb_source_files *files = odb_source_files_downcast(repo->objects->sources);
+ struct odb_source_files *files = odb_source_files_downcast(repo->objects->source);
struct multi_pack_index *m = get_multi_pack_index(files->dirs->packed);
strbuf_addf(&buf, "%s.pack", base_name);
if (m && files->base.local && midx_contains_pack(m, buf.buf)) {
@@ -158,7 +158,7 @@ void existing_packs_collect(struct existing_packs *existing,
string_list_append(&existing->non_kept_packs, buf.buf);
}
- existing->source = existing->repo->objects->sources;
+ existing->source = existing->repo->objects->source;
string_list_sort(&existing->kept_packs);
string_list_sort(&existing->non_kept_packs);
diff --git a/repository.c b/repository.c
index b857e1c580..fa4214ff41 100644
--- a/repository.c
+++ b/repository.c
@@ -126,9 +126,9 @@ const char *repo_get_common_dir(struct repository *repo)
const char *repo_get_object_directory(struct repository *repo)
{
- if (!repo->objects->sources)
+ if (!repo->objects->source)
BUG("repository hasn't been set up");
- return repo->objects->sources->path;
+ return repo->objects->source->path;
}
const char *repo_get_index_file(struct repository *repo)
diff --git a/setup.c b/setup.c
index 29474fc292..bc7d451917 100644
--- a/setup.c
+++ b/setup.c
@@ -2669,7 +2669,7 @@ void create_object_database(struct repository *repo,
repo->objects = odb_new(repo, ODB_NEW_HONOR_ENV);
- if (odb_source_create_on_disk(repo->objects->sources, &opts) < 0)
+ if (odb_source_create_on_disk(repo->objects->source, &opts) < 0)
die(_("failed creating object database"));
}
diff --git a/t/helper/test-read-graph.c b/t/helper/test-read-graph.c
index a75c817e47..62e2384052 100644
--- a/t/helper/test-read-graph.c
+++ b/t/helper/test-read-graph.c
@@ -78,7 +78,7 @@ int cmd__read_graph(int argc, const char **argv)
int ret = 0;
setup_git_directory(the_repository);
- source = the_repository->objects->sources;
+ source = the_repository->objects->source;
prepare_repo_settings(the_repository);
diff --git a/tmp-objdir.c b/tmp-objdir.c
index 2f2ffbbc7d..3debb60270 100644
--- a/tmp-objdir.c
+++ b/tmp-objdir.c
@@ -60,7 +60,7 @@ static void tmp_objdir_reparent(const char *old_cwd,
*/
static void tmp_objdir_restore_source(struct tmp_objdir *t)
{
- struct odb_source_files *files = odb_source_files_downcast(t->repo->objects->sources);
+ struct odb_source_files *files = odb_source_files_downcast(t->repo->objects->source);
struct odb_files_dir *cur_dir = files->dirs;
if (t->temp_dir != files->dirs)
@@ -159,7 +159,7 @@ struct tmp_objdir *tmp_objdir_create(struct repository *r,
const char *prefix,
int will_destroy)
{
- struct odb_source_files *files = odb_source_files_downcast(r->objects->sources);
+ struct odb_source_files *files = odb_source_files_downcast(r->objects->source);
static int installed_handlers;
struct tmp_objdir *t;
--
2.56.0.406.ga2d225a756.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* [PATCH v2 13/13] odb/source: drop `read_alternates` callback
2026-10-08 8:35 ` [PATCH v2 " Patrick Steinhardt
` (11 preceding siblings ...)
2026-10-08 8:36 ` [PATCH v2 12/13] odb/source-files: move alternates into the backend Patrick Steinhardt
@ 2026-10-08 8:36 ` Patrick Steinhardt
2026-10-08 9:24 ` [PATCH v2 00/13] odb/source-files: move alternates into the backend Karthik Nayak
13 siblings, 0 replies; 41+ messages in thread
From: Patrick Steinhardt @ 2026-10-08 8:36 UTC (permalink / raw)
To: git; +Cc: Karthik Nayak
Now that alternates are an implementation detail of the "files" backend
there is no reason anymore to expose them anywhere else. The callback to
read alternates is thus no longer required. Remove it.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
odb/source-files.c | 7 -------
odb/source-inmemory.c | 7 -------
odb/source-loose.c | 7 -------
odb/source-packed.c | 7 -------
odb/source.h | 27 ---------------------------
5 files changed, 55 deletions(-)
diff --git a/odb/source-files.c b/odb/source-files.c
index 072f515b36..455f409db0 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -594,12 +594,6 @@ static int odb_source_files_begin_transaction(struct odb_source *source,
return odb_transaction_files_begin(source, out, flags);
}
-static int odb_source_files_read_alternates(struct odb_source *source,
- struct strvec *out)
-{
- return read_alternates(source->path, out);
-}
-
static int too_many_loose_objects(struct odb_source_files *files, int limit)
{
unsigned long loose_count;
@@ -1277,7 +1271,6 @@ struct odb_source_files *odb_source_files_new(struct object_database *odb,
files->base.write_object = odb_source_files_write_object;
files->base.write_object_stream = odb_source_files_write_object_stream;
files->base.begin_transaction = odb_source_files_begin_transaction;
- files->base.read_alternates = odb_source_files_read_alternates;
files->base.optimize = odb_source_files_optimize;
files->base.optimize_required = odb_source_files_optimize_required;
files->base.generate_pack = odb_source_files_generate_pack;
diff --git a/odb/source-inmemory.c b/odb/source-inmemory.c
index f67c74a724..868e723c81 100644
--- a/odb/source-inmemory.c
+++ b/odb/source-inmemory.c
@@ -322,12 +322,6 @@ static int odb_source_inmemory_begin_transaction(struct odb_source *source UNUSE
return error("in-memory source does not support transactions");
}
-static int odb_source_inmemory_read_alternates(struct odb_source *source UNUSED,
- struct strvec *out UNUSED)
-{
- return 0;
-}
-
static void odb_source_inmemory_close(struct odb_source *source UNUSED)
{
}
@@ -390,7 +384,6 @@ struct odb_source_inmemory *odb_source_inmemory_new(struct object_database *odb)
source->base.write_object_stream = odb_source_inmemory_write_object_stream;
source->base.freshen_object = odb_source_inmemory_freshen_object;
source->base.begin_transaction = odb_source_inmemory_begin_transaction;
- source->base.read_alternates = odb_source_inmemory_read_alternates;
return source;
}
diff --git a/odb/source-loose.c b/odb/source-loose.c
index b2fbccd3c0..0664634938 100644
--- a/odb/source-loose.c
+++ b/odb/source-loose.c
@@ -985,12 +985,6 @@ static int odb_source_loose_begin_transaction(struct odb_source *source UNUSED,
return error("loose source does not support transactions");
}
-static int odb_source_loose_read_alternates(struct odb_source *source UNUSED,
- struct strvec *out UNUSED)
-{
- return 0;
-}
-
static void odb_source_loose_clear_cache(struct odb_source_loose *loose)
{
oidtree_clear(loose->cache);
@@ -1145,7 +1139,6 @@ struct odb_source_loose *odb_source_loose_new(struct object_database *odb,
loose->base.write_object = odb_source_loose_write_object;
loose->base.write_object_stream = odb_source_loose_write_object_stream;
loose->base.begin_transaction = odb_source_loose_begin_transaction;
- loose->base.read_alternates = odb_source_loose_read_alternates;
if (!is_absolute_path(loose->base.path))
chdir_notify_register(odb_source_loose_reparent, loose);
diff --git a/odb/source-packed.c b/odb/source-packed.c
index 831501f397..2226fbd313 100644
--- a/odb/source-packed.c
+++ b/odb/source-packed.c
@@ -687,12 +687,6 @@ static int odb_source_packed_begin_transaction(struct odb_source *source UNUSED,
return error("packed backend cannot begin transactions");
}
-static int odb_source_packed_read_alternates(struct odb_source *source UNUSED,
- struct strvec *out UNUSED)
-{
- return 0;
-}
-
void (*report_garbage)(unsigned seen_bits, const char *path);
static void report_helper(const struct string_list *list,
@@ -1019,7 +1013,6 @@ struct odb_source_packed *odb_source_packed_new(struct object_database *odb,
packed->base.write_object = odb_source_packed_write_object;
packed->base.write_object_stream = odb_source_packed_write_object_stream;
packed->base.begin_transaction = odb_source_packed_begin_transaction;
- packed->base.read_alternates = odb_source_packed_read_alternates;
if (!is_absolute_path(path))
chdir_notify_register(odb_source_packed_reparent, packed);
diff --git a/odb/source.h b/odb/source.h
index 6718aced6a..1867347dfc 100644
--- a/odb/source.h
+++ b/odb/source.h
@@ -258,19 +258,6 @@ struct odb_source {
struct odb_transaction **out,
enum odb_transaction_flags flags);
- /*
- * This callback is expected to read the list of alternate object
- * database sources connected to it and write them into the `strvec`.
- *
- * The result is expected to be paths to the alternates. All paths must
- * be resolved to absolute paths.
- *
- * The callback is expected to return 0 on success, a negative error
- * code otherwise.
- */
- int (*read_alternates)(struct odb_source *source,
- struct strvec *out);
-
/*
* This callback is expected to optimize the object database source.
* Returns 0 on success, a negative error code otherwise.
@@ -507,20 +494,6 @@ static inline int odb_source_write_object_stream(struct odb_source *source,
return source->write_object_stream(source, stream, oid);
}
-/*
- * Read the list of alternative object database sources from the given backend
- * and populate the `strvec` with them. The listing is not recursive -- that
- * is, if any of the yielded alternate sources has alternates itself, those
- * will not be yielded as part of this function call.
- *
- * Return 0 on success, a negative error code otherwise.
- */
-static inline int odb_source_read_alternates(struct odb_source *source,
- struct strvec *out)
-{
- return source->read_alternates(source, out);
-}
-
/*
* Create a new transaction that can be used to write objects into a temporary
* staging area. The objects will only be persisted when the transaction is
--
2.56.0.406.ga2d225a756.dirty
^ permalink raw reply related [flat|nested] 41+ messages in thread* Re: [PATCH v2 00/13] odb/source-files: move alternates into the backend
2026-10-08 8:35 ` [PATCH v2 " Patrick Steinhardt
` (12 preceding siblings ...)
2026-10-08 8:36 ` [PATCH v2 13/13] odb/source: drop `read_alternates` callback Patrick Steinhardt
@ 2026-10-08 9:24 ` Karthik Nayak
13 siblings, 0 replies; 41+ messages in thread
From: Karthik Nayak @ 2026-10-08 9:24 UTC (permalink / raw)
To: Patrick Steinhardt, git
[-- Attachment #1: Type: text/plain, Size: 1144 bytes --]
Patrick Steinhardt <ps@pks.im> writes:
[snip]
> Range-diff versus v1:
>
> 1: 985b068271 ! 1: 43a288604d commit-graph: require resolved packfile paths for `stdin_packs`
> @@ Commit message
> for a set of packfiles via the "--stdin-packs" option. Those users are
> expected to pass in relative paths, and those eventually get resolved in
> `fill_oids_from_packs()`. This ties the logic in "commit-graph.c" to the
> - specific object database source.
> + specific object database source, as the subsystem now needs to assume
> + where a specific packfile is located relative to the source itself.
>
> Refactor the logic to instead require the caller to pass in resolved
> - packfiles to untangle that dependency.
> + packfiles to untangle that dependency. This also makes the next change
> + easier to implement, where we'll get rid of passing the source to the
> + commit-graph subsystem.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
This looks good, the previous version itself was in great shape. I have
nothing more to add :)
[snip]
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]
^ permalink raw reply [flat|nested] 41+ messages in thread