[PATCH 0/5] Improvements for storage format handling

[PATCH 0/5] Improvements for storage format handling

[Patrick Steinhardt]

Hi,

this patch series includes a couple of improvements for how we handle
ref and object storage formats.

Most importantly, this series allows us to run tests with a different
object storage backend via `GIT_TEST_DEFAULT_OBJECT_STORAGE_FORMAT=`. As
we don't have an alternative backend, I also introduce the ability to
specify a default object (and ref) storage format that carries a payload
so that you can run all tests with "files://objdir", for example. That
surfaces a bunch of tests that assume objects exist in ".git/objects".

Overall it's a surprisingly small number of test failures, only 1105 out
of 35000 tests fail. Fixing those is not part of this patch series, but
we'll start upstreaming fixes once this lands. Such tests will either
have to be adapted to use plumbing commands, or they'll need to be
tagged with a prerequisite. It's going to be a bunch of work, but it's
basically preparing for the introduction of alternative backends
already.

The series is built on top of 0ffa3e35c7c (The 15th batch, 2026-11-04)
with ps/odb-object-storage-extension at 87a55d790ca (builtin/submodule:
propagate "--object-storage-format=", 2026-10-27) merged into it.

Thanks!

Patrick

---
Patrick Steinhardt (5):
      refs: create leading directories
      odb/files: create leading directories
      setup: allow speciyfing payloads with default storage formats
      builtin/rev-parse: print payload for the ref storage format
      t: allow testing with a non-default object storage foramt

 builtin/rev-parse.c     |  6 +++++-
 odb/source-files.c      |  1 +
 refs/files-backend.c    |  1 +
 refs/reftable-backend.c |  1 +
 setup.c                 | 24 ++++++++++++++++-----
 t/t0001-init.sh         | 57 ++++++++++++++++++++++++++++++++++++++++++++++---
 t/t1423-ref-backend.sh  | 24 +++++++++++++++++++++
 t/t1500-rev-parse.sh    |  9 ++++++++
 t/test-lib.sh           |  4 ++++
 9 files changed, 118 insertions(+), 9 deletions(-)


---
base-commit: df904e7114cda6ca4ebf514115835a2dcd2f2b64
change-id: 20261105-pks-storage-format-improvements-7b716599bada

[PATCH 1/5] refs: create leading directories

[Patrick Steinhardt]

Both the "files" and "reftable" backends accept a payload that allows
the user to use a non-default location for their files. So e.g. if the
user specifies "files://refdir" as the ref storage format, then we would
put files into ".git/refdir/refs" instead of ".git/refs".

Using this is somewhat unwieldy though, as the directory must exist
before we create the ref storage as we'd otherwise die. This isn't that
bad in the case where the user specifies an absolute path that is
outside of the repository. But it becomes quite weird when asked to
create the directory relative to commondir, as the user would first have
to "mkdir -p repo/.git/refdir".

Adapt the reference backends to create leading directories. Note that we
use the `noshare()` variant for this because it's very much possible
that it will exist outside of the repository, and if so we shouldn't
adapt permissions on such leading directories.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 refs/files-backend.c    |  1 +
 refs/reftable-backend.c |  1 +
 t/t1423-ref-backend.sh  | 24 ++++++++++++++++++++++++
 3 files changed, 26 insertions(+)

diff --git a/refs/files-backend.c b/refs/files-backend.c
index 87d04557a53..1bdb16c0e7b 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -3699,6 +3699,7 @@ static int files_ref_store_create_on_disk(struct ref_store *ref_store,
 	 *   they do not understand the reference format extension.
 	 */
 	strbuf_addf(&sb, "%s/refs", ref_store->gitdir);
+	safe_create_leading_directories_no_share(sb.buf);
 	safe_create_dir(refs->base.repo, sb.buf, 1);
 	adjust_shared_perm(refs->base.repo, sb.buf);
 
diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c
index d0de0663552..3ba6a9ac0db 100644
--- a/refs/reftable-backend.c
+++ b/refs/reftable-backend.c
@@ -504,6 +504,7 @@ static int reftable_be_create_on_disk(struct ref_store *ref_store,
 	struct strbuf sb = STRBUF_INIT;
 
 	strbuf_addf(&sb, "%s/reftable", refs->base.gitdir);
+	safe_create_leading_directories_no_share(sb.buf);
 	safe_create_dir(ref_store->repo, sb.buf, 1);
 	strbuf_reset(&sb);
 
diff --git a/t/t1423-ref-backend.sh b/t/t1423-ref-backend.sh
index ee2bb66b99c..85d1e876b82 100755
--- a/t/t1423-ref-backend.sh
+++ b/t/t1423-ref-backend.sh
@@ -284,6 +284,30 @@ test_expect_success 'initializing repository with --ref-storage-format and paylo
 	test_cmp expect actual
 '
 
+for format in files reftable
+do
+	test_expect_success "default ref storage format with $format backend and payload" '
+		test_when_finished "rm -rf repo refdir" &&
+		BACKEND="$format://$(pwd)/refdir" &&
+		echo "$BACKEND" >expect &&
+		git init --ref-storage-format="$BACKEND" repo &&
+		test_path_is_dir refdir &&
+		case "$format" in
+		files)
+			test_path_is_file refdir/HEAD &&
+			test_path_is_dir refdir/refs
+			;;
+		reftable)
+			test_path_is_dir refdir/reftable &&
+			test_path_is_file refdir/reftable/tables.list
+			;;
+		*)
+			BUG "unknown ref storage format: $format"
+			;;
+		esac
+	'
+done
+
 test_expect_success 'cloning repository with alt ref directory' '
 	test_when_finished "rm -rf source repo refdir" &&
 	mkdir refdir &&

-- 
2.56.0.500.g5d06dc02cd.dirty

[PATCH 2/5] odb/files: create leading directories

[Patrick Steinhardt]

With the same reasoning as in the preceding commit, create leading
directories when creating the "files" object backend on disk.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 odb/source-files.c | 1 +
 t/t0001-init.sh    | 6 +++---
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/odb/source-files.c b/odb/source-files.c
index 6b423499fe5..e7503548e38 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -97,6 +97,7 @@ static int odb_source_files_create_on_disk(struct odb_source *source,
 	struct strbuf line = STRBUF_INIT;
 	int ret;
 
+	safe_create_leading_directories_no_share(source->path);
 	safe_create_dir(source->odb->repo, source->path, 1);
 
 	strbuf_addf(&path, "%s/pack", source->path);
diff --git a/t/t0001-init.sh b/t/t0001-init.sh
index 23317e0733a..735ab9d8475 100755
--- a/t/t0001-init.sh
+++ b/t/t0001-init.sh
@@ -944,13 +944,13 @@ test_expect_success 'init with --object-storage-format=files://<relative-path>'
 
 test_expect_success 'init with --object-storage-format=files://<absolute-path>' '
 	test_when_finished "rm -rf objformat objdir" &&
-	git init --object-storage-format=files://"$(pwd)/objdir" objformat &&
-	echo "files://$(pwd)/objdir" >expect &&
+	git init --object-storage-format=files://"$(pwd)/absolute/objdir" objformat &&
+	echo "files://$(pwd)/absolute/objdir" >expect &&
 	git -C objformat config get extensions.objectstorage >actual &&
 	test_cmp expect actual &&
 	test_commit -C objformat A &&
 	git -C objformat rev-parse --verify HEAD &&
-	test_path_is_dir objdir/pack
+	test_path_is_dir absolute/objdir/pack
 '
 
 test_expect_success 'GIT_OBJECT_DIRECTORY takes precedence over extensions.objectStorage payload' '

-- 
2.56.0.500.g5d06dc02cd.dirty

[PATCH 3/5] setup: allow speciyfing payloads with default storage formats

[Patrick Steinhardt]

While the "git init --ref-storage-format=" switch allows the user to
specify a payload that gets passed to the backend, this isn't possible
when specifying a default via "init.defaultRefStorageFormat" or via
`GIT_DEFAULT_REF_STORAGE_FORMAT`. The same restriction also applies to
the object storage format, which needlessly restricts the flexibility
and usefulness of these mechanism.

Extend the logic to allow specifying a payload.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 setup.c         | 24 +++++++++++++++++++-----
 t/t0001-init.sh | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 70 insertions(+), 5 deletions(-)

diff --git a/setup.c b/setup.c
index 186815c7801..cedc031c54f 100644
--- a/setup.c
+++ b/setup.c
@@ -2797,7 +2797,9 @@ static void separate_git_dir(struct repository *repo,
 struct default_format_config {
 	int hash;
 	enum ref_storage_format ref_storage_format;
+	char *ref_storage_payload;
 	enum object_storage_format object_storage_format;
+	char *object_storage_payload;
 };
 
 static int read_default_format_config(const char *key, const char *value,
@@ -2823,9 +2825,12 @@ static int read_default_format_config(const char *key, const char *value,
 		ret = git_config_string(&str, key, value);
 		if (ret)
 			goto out;
-		cfg->ref_storage_format = ref_storage_format_by_name(str);
+
+		FREE_AND_NULL(cfg->ref_storage_payload);
+		cfg->ref_storage_format = ref_storage_format_by_uri(str, &cfg->ref_storage_payload);
 		if (cfg->ref_storage_format == REF_STORAGE_FORMAT_UNKNOWN)
 			warning(_("unknown ref storage format '%s'"), str);
+
 		goto out;
 	}
 
@@ -2833,9 +2838,12 @@ static int read_default_format_config(const char *key, const char *value,
 		ret = git_config_string(&str, key, value);
 		if (ret)
 			goto out;
-		cfg->object_storage_format = object_storage_format_by_name(str);
+
+		FREE_AND_NULL(cfg->object_storage_payload);
+		cfg->object_storage_format = object_storage_format_by_uri(str, &cfg->object_storage_payload);
 		if (cfg->object_storage_format == OBJECT_STORAGE_FORMAT_UNKNOWN)
 			warning(_("unknown object storage format '%s'"), str);
+
 		goto out;
 	}
 
@@ -2947,17 +2955,19 @@ static void repository_format_configure(struct repository_format *repo_fmt,
 		ref_storage_format = repo_fmt->ref_storage_format;
 		ref_storage_payload = xstrdup_or_null(repo_fmt->ref_storage_payload);
 	} else if ((env = getenv("GIT_DEFAULT_REF_STORAGE_FORMAT"))) {
-		ref_storage_format = ref_storage_format_by_name(env);
+		ref_storage_format = ref_storage_format_by_uri(env, &ref_storage_payload);
 		if (ref_storage_format == REF_STORAGE_FORMAT_UNKNOWN)
 			die(_("unknown ref storage format specified via %s: '%s'"),
 			    "GIT_DEFAULT_REF_STORAGE_FORMAT", env);
 	} else if ((env = getenv("GIT_DEFAULT_REF_FORMAT"))) {
-		ref_storage_format = ref_storage_format_by_name(env);
+		ref_storage_format = ref_storage_format_by_uri(env, &ref_storage_payload);
 		if (ref_storage_format == REF_STORAGE_FORMAT_UNKNOWN)
 			die(_("unknown ref storage format specified via %s: '%s'"),
 			    "GIT_DEFAULT_REF_FORMAT", env);
 	} else if (cfg.ref_storage_format != REF_STORAGE_FORMAT_UNKNOWN) {
 		ref_storage_format = cfg.ref_storage_format;
+		ref_storage_payload = cfg.ref_storage_payload;
+		cfg.ref_storage_payload = NULL;
 	} else {
 		ref_storage_format = REF_STORAGE_FORMAT_DEFAULT;
 	}
@@ -2995,12 +3005,14 @@ static void repository_format_configure(struct repository_format *repo_fmt,
 		object_storage_format = repo_fmt->object_storage_format;
 		object_storage_payload = xstrdup_or_null(repo_fmt->object_storage_payload);
 	} else if ((env = getenv("GIT_DEFAULT_OBJECT_STORAGE_FORMAT"))) {
-		object_storage_format = object_storage_format_by_name(env);
+		object_storage_format = object_storage_format_by_uri(env, &object_storage_payload);
 		if (object_storage_format == OBJECT_STORAGE_FORMAT_UNKNOWN)
 			die(_("unknown object storage format specified via %s: '%s'"),
 			    "GIT_DEFAULT_OBJECT_STORAGE_FORMAT", env);
 	} else if (cfg.object_storage_format != OBJECT_STORAGE_FORMAT_UNKNOWN) {
 		object_storage_format = cfg.object_storage_format;
+		object_storage_payload = cfg.object_storage_payload;
+		cfg.object_storage_payload = NULL;
 	} else {
 		object_storage_format = OBJECT_STORAGE_FORMAT_FILES;
 	}
@@ -3024,10 +3036,12 @@ static void repository_format_configure(struct repository_format *repo_fmt,
 			die(_("attempt to reinitialize repository with different object storage payload"));
 	}
 
+	free(cfg.ref_storage_payload);
 	free(repo_fmt->ref_storage_payload);
 	repo_fmt->ref_storage_format = ref_storage_format;
 	repo_fmt->ref_storage_payload = ref_storage_payload;
 
+	free(cfg.object_storage_payload);
 	free(repo_fmt->object_storage_payload);
 	repo_fmt->object_storage_format = object_storage_format;
 	repo_fmt->object_storage_payload = object_storage_payload;
diff --git a/t/t0001-init.sh b/t/t0001-init.sh
index 735ab9d8475..09b7258bf10 100755
--- a/t/t0001-init.sh
+++ b/t/t0001-init.sh
@@ -745,6 +745,33 @@ do
 	'
 done
 
+test_expect_success 'initializing repository with init.defaultRefStorageFormat and payload' '
+	test_when_finished "rm -rf repo refdir" &&
+
+	BACKEND="files://$(pwd)/refdir" &&
+	test_config_global init.defaultRefStorageFormat "$BACKEND" &&
+	(
+		sane_unset GIT_DEFAULT_REF_STORAGE_FORMAT &&
+		git init repo
+	) &&
+	test_path_is_file refdir/HEAD &&
+	test_path_is_dir refdir/refs &&
+	git -C repo config get extensions.refstorage >actual &&
+	echo "$BACKEND" >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'initializing repository with GIT_DEFAULT_REF_STORAGE_FORMAT and payload' '
+	test_when_finished "rm -rf repo refdir" &&
+	BACKEND="files://$(pwd)/refdir" &&
+	GIT_DEFAULT_REF_STORAGE_FORMAT="$BACKEND" git init repo &&
+	test_path_is_file refdir/HEAD &&
+	test_path_is_dir refdir/refs &&
+	git -C repo config get extensions.refstorage >actual &&
+	echo "$BACKEND" >expect &&
+	test_cmp expect actual
+'
+
 test_expect_success "--ref-storage-format= overrides GIT_DEFAULT_REF_STORAGE_FOMAT" '
 	test_when_finished "rm -rf refformat" &&
 	GIT_DEFAULT_REF_STORAGE_FORMAT=files git init --ref-storage-format=reftable refformat &&
@@ -1064,6 +1091,30 @@ test_expect_success 'init with init.defaultObjectStorageFormat' '
 	test_cmp expect actual
 '
 
+test_expect_success 'init with init.defaultObjectStorageFormat and payload' '
+	test_when_finished "rm -rf repo objdir" &&
+	BACKEND="files://$(pwd)/objdir" &&
+	test_config_global init.defaultObjectStorageFormat "$BACKEND" &&
+	(
+		sane_unset GIT_DEFAULT_OBJECT_STORAGE_FORMAT &&
+		git init repo
+	) &&
+	test_path_is_dir objdir/pack &&
+	git -C repo config get extensions.objectstorage >actual &&
+	echo "$BACKEND" >expect &&
+	test_cmp expect actual
+'
+
+test_expect_success 'init with GIT_DEFAULT_OBJECT_STORAGE_FORMAT and payload' '
+	test_when_finished "rm -rf repo objdir" &&
+	BACKEND="files://$(pwd)/objdir" &&
+	GIT_DEFAULT_OBJECT_STORAGE_FORMAT="$BACKEND" git init repo &&
+	test_path_is_dir objdir/pack &&
+	git -C repo config get extensions.objectstorage >actual &&
+	echo "$BACKEND" >expect &&
+	test_cmp expect actual
+'
+
 test_expect_success MINGW 'core.hidedotfiles = false' '
 	git config --global core.hidedotfiles false &&
 	rm -rf newdir &&

-- 
2.56.0.500.g5d06dc02cd.dirty

[PATCH 4/5] builtin/rev-parse: print payload for the ref storage format

[Patrick Steinhardt]

While we print both the object storage format and its payload for the
"--show-object-storage-format" switch, we only print the ref storage
format with "--show-ref-storage-format". Adapt the latter to also show
the payload.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 builtin/rev-parse.c  | 6 +++++-
 t/t1500-rev-parse.sh | 9 +++++++++
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/builtin/rev-parse.c b/builtin/rev-parse.c
index d2154eac8d2..fbc5ef87276 100644
--- a/builtin/rev-parse.c
+++ b/builtin/rev-parse.c
@@ -1136,7 +1136,11 @@ int cmd_rev_parse(int argc,
 				continue;
 			}
 			if (!strcmp(arg, "--show-ref-format") || !strcmp(arg, "--show-ref-storage-format")) {
-				puts(ref_storage_format_to_name(the_repository->ref_storage_format));
+				const char *format = ref_storage_format_to_name(the_repository->ref_storage_format);
+				if (the_repository->ref_storage_payload)
+					printf("%s://%s\n", format, the_repository->ref_storage_payload);
+				else
+					puts(format);
 				continue;
 			}
 			if (!strcmp(arg, "--show-object-storage-format")) {
diff --git a/t/t1500-rev-parse.sh b/t/t1500-rev-parse.sh
index 30bf8a4d0a0..c4e6bc1dd89 100755
--- a/t/t1500-rev-parse.sh
+++ b/t/t1500-rev-parse.sh
@@ -209,6 +209,7 @@ test_expect_success 'rev-parse --show-object-format in repo' '
 
 
 test_expect_success RUST 'rev-parse --show-object-format in repo with compat mode' '
+	test_when_finished "rm -rf repo" &&
 	mkdir repo &&
 	(
 		sane_unset GIT_DEFAULT_HASH &&
@@ -247,6 +248,14 @@ test_expect_success 'rev-parse --show-ref-storage-format' '
 	test_cmp expect actual
 '
 
+test_expect_success 'rev-parse --show-ref-storage-format with payload' '
+	test_when_finished "rm -rf repo" &&
+	git init repo --ref-storage-format="files://$(pwd)/refdir" &&
+	git -C repo rev-parse --show-ref-storage-format >actual &&
+	echo "files://$(pwd)/refdir" >expect &&
+	test_cmp expect actual
+'
+
 test_expect_success 'rev-parse --show-ref-storage-format with invalid storage' '
 	test_when_finished "rm -rf repo" &&
 	git init repo &&

-- 
2.56.0.500.g5d06dc02cd.dirty

[PATCH 5/5] t: allow testing with a non-default object storage foramt

[Patrick Steinhardt]

It is already possible to test Git with a non-default ref storage format
by specifying `GIT_TEST_DEFAULT_REF_STORAGE_FORMAT`. We don't provide
the same mechanism yet for testing the object storage format, and will
instead use the default ("files"). Even worse though, we may inherit
whatever the user has configured in `GIT_DEFAULT_OBJECT_STORAGE_FORMAT`.

Introduce `GIT_TEST_DEFAULT_OBJECT_STORAGE_FORMAT` and have it default
to the "files" backend.

Note that this environment variable isn't all that useful, as we do not
have an alternative backend. But what it does allow is to figure out
which tests will definitely break once we do have an alternative backend
because they assume that objects exist in ".git/objects", as we can now
move objects into a different location by specifying a payload.

Sure enough, running with "files://objdir" shows that around 1105 or our
tests are failing. This is a surprisingly small number of failures, but
we'll have to fix them if we ever want to introduce an alternative
backend.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/test-lib.sh | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/t/test-lib.sh b/t/test-lib.sh
index a893963f86c..7fdd8272162 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -576,9 +576,13 @@ export EDITOR
 GIT_TEST_BUILTIN_HASH=$("$GIT_BINARY" version --build-options | sed -ne 's/^default-hash: //p')
 GIT_DEFAULT_HASH="${GIT_TEST_DEFAULT_HASH:-$GIT_TEST_BUILTIN_HASH}"
 export GIT_DEFAULT_HASH
+
 GIT_DEFAULT_REF_STORAGE_FORMAT="${GIT_TEST_DEFAULT_REF_STORAGE_FORMAT:-files}"
 export GIT_DEFAULT_REF_STORAGE_FORMAT
 
+GIT_DEFAULT_OBJECT_STORAGE_FORMAT="${GIT_TEST_DEFAULT_OBJECT_STORAGE_FORMAT:-files}"
+export GIT_DEFAULT_OBJECT_STORAGE_FORMAT
+
 # Tests using GIT_TRACE typically don't want <timestamp> <file>:<line> output
 GIT_TRACE_BARE=1
 export GIT_TRACE_BARE

-- 
2.56.0.500.g5d06dc02cd.dirty

Re: [PATCH 1/5] refs: create leading directories

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 1978 bytes --]

Patrick Steinhardt <ps@pks.im> writes:

The patch looks good.

> diff --git a/t/t1423-ref-backend.sh b/t/t1423-ref-backend.sh
> index ee2bb66b99c..85d1e876b82 100755
> --- a/t/t1423-ref-backend.sh
> +++ b/t/t1423-ref-backend.sh
> @@ -284,6 +284,30 @@ test_expect_success 'initializing repository with --ref-storage-format and paylo
>  	test_cmp expect actual
>  '
>
> +for format in files reftable
> +do

This would run the loop for both the files and reftable CI jobs.

> +	test_expect_success "default ref storage format with $format backend and payload" '
> +		test_when_finished "rm -rf repo refdir" &&
> +		BACKEND="$format://$(pwd)/refdir" &&
> +		echo "$BACKEND" >expect &&
> +		git init --ref-storage-format="$BACKEND" repo &&
> +		test_path_is_dir refdir &&
> +		case "$format" in
> +		files)
> +			test_path_is_file refdir/HEAD &&
> +			test_path_is_dir refdir/refs
> +			;;
> +		reftable)
> +			test_path_is_dir refdir/reftable &&
> +			test_path_is_file refdir/reftable/tables.list
> +			;;
> +		*)
> +			BUG "unknown ref storage format: $format"
> +			;;
> +		esac
> +	'
> +done
> +

Should we instead do (untested):

test_expect_success "default ref storage format with $format backend
and payload" '
	test_when_finished "rm -rf repo refdir" &&
    format="$(test_detect_ref_format)" &&
	BACKEND="$format://$(pwd)/refdir" &&
	echo "$BACKEND" >expect &&
	git init --ref-storage-format="$BACKEND" repo &&
	test_path_is_dir refdir &&
	case "$format" in
	files)
		test_path_is_file refdir/HEAD &&
		test_path_is_dir refdir/refs
		;;
	reftable)
		test_path_is_dir refdir/reftable &&
		test_path_is_file refdir/reftable/tables.list
		;;
	*)
		BUG "unknown ref storage format: $format"
		;;
	esac
'

This eliminates the loop, while still testing both backends on their
respective CI jobs.

>  test_expect_success 'cloning repository with alt ref directory' '
>  	test_when_finished "rm -rf source repo refdir" &&
>  	mkdir refdir &&
>
> --
> 2.56.0.500.g5d06dc02cd.dirty

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH 4/5] builtin/rev-parse: print payload for the ref storage format

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 348 bytes --]

Patrick Steinhardt <ps@pks.im> writes:

> While we print both the object storage format and its payload for the
> "--show-object-storage-format" switch, we only print the ref storage
> format with "--show-ref-storage-format". Adapt the latter to also show
> the payload.
>

Nice, this makes sense and was clearly a miss. Thanks for fixing.

[snip]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH 0/5] Improvements for storage format handling

[Karthik Nayak]

[-- Attachment #1: Type: text/plain, Size: 1404 bytes --]

Patrick Steinhardt <ps@pks.im> writes:

> Hi,
>
> this patch series includes a couple of improvements for how we handle
> ref and object storage formats.
>
> Most importantly, this series allows us to run tests with a different
> object storage backend via `GIT_TEST_DEFAULT_OBJECT_STORAGE_FORMAT=`. As
> we don't have an alternative backend, I also introduce the ability to
> specify a default object (and ref) storage format that carries a payload
> so that you can run all tests with "files://objdir", for example. That
> surfaces a bunch of tests that assume objects exist in ".git/objects".
>
> Overall it's a surprisingly small number of test failures, only 1105 out
> of 35000 tests fail. Fixing those is not part of this patch series, but
> we'll start upstreaming fixes once this lands. Such tests will either
> have to be adapted to use plumbing commands, or they'll need to be
> tagged with a prerequisite. It's going to be a bunch of work, but it's
> basically preparing for the introduction of alternative backends
> already.
>
> The series is built on top of 0ffa3e35c7c (The 15th batch, 2026-11-04)
> with ps/odb-object-storage-extension at 87a55d790ca (builtin/submodule:
> propagate "--object-storage-format=", 2026-10-27) merged into it.
>
> Thanks!
>
> Patrick
>

This was a very easy read. I don't have much to add apart from a comment
on 1/5. But you can also leave it as is! Thanks

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 690 bytes --]

Re: [PATCH 1/5] refs: create leading directories

[Patrick Steinhardt]

On Fri, Nov 06, 2026 at 02:37:23AM -0800, Karthik Nayak wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> > diff --git a/t/t1423-ref-backend.sh b/t/t1423-ref-backend.sh
> > index ee2bb66b99c..85d1e876b82 100755
> > --- a/t/t1423-ref-backend.sh
> > +++ b/t/t1423-ref-backend.sh
> > @@ -284,6 +284,30 @@ test_expect_success 'initializing repository with --ref-storage-format and paylo
> >  	test_cmp expect actual
> >  '
> >
> > +for format in files reftable
> > +do
> 
> This would run the loop for both the files and reftable CI jobs.
> 
> > +	test_expect_success "default ref storage format with $format backend and payload" '
> > +		test_when_finished "rm -rf repo refdir" &&
> > +		BACKEND="$format://$(pwd)/refdir" &&
> > +		echo "$BACKEND" >expect &&
> > +		git init --ref-storage-format="$BACKEND" repo &&
> > +		test_path_is_dir refdir &&
> > +		case "$format" in
> > +		files)
> > +			test_path_is_file refdir/HEAD &&
> > +			test_path_is_dir refdir/refs
> > +			;;
> > +		reftable)
> > +			test_path_is_dir refdir/reftable &&
> > +			test_path_is_file refdir/reftable/tables.list
> > +			;;
> > +		*)
> > +			BUG "unknown ref storage format: $format"
> > +			;;
> > +		esac
> > +	'
> > +done
> > +
> 
> Should we instead do (untested):
> 
> test_expect_success "default ref storage format with $format backend
> and payload" '
> 	test_when_finished "rm -rf repo refdir" &&
>     format="$(test_detect_ref_format)" &&
> 	BACKEND="$format://$(pwd)/refdir" &&
> 	echo "$BACKEND" >expect &&
> 	git init --ref-storage-format="$BACKEND" repo &&
> 	test_path_is_dir refdir &&
> 	case "$format" in
> 	files)
> 		test_path_is_file refdir/HEAD &&
> 		test_path_is_dir refdir/refs
> 		;;
> 	reftable)
> 		test_path_is_dir refdir/reftable &&
> 		test_path_is_file refdir/reftable/tables.list
> 		;;
> 	*)
> 		BUG "unknown ref storage format: $format"
> 		;;
> 	esac
> '
> 
> This eliminates the loop, while still testing both backends on their
> respective CI jobs.

Well, in this case the system under test is explicitly the ref storage
format itself and how it behaves together with the extension. The
GIT_TEST_DEFAULT_REF_STORAGE format on the other hand is design to
explicitly test that everything else works as-is with an alternate ref
store.

So I think it makes sense to keep this explicit test.

Patrick

Re: [PATCH 0/5] Improvements for storage format handling

[Patrick Steinhardt]

On Fri, Nov 06, 2026 at 02:41:41AM -0800, Karthik Nayak wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > Hi,
> >
> > this patch series includes a couple of improvements for how we handle
> > ref and object storage formats.
> >
> > Most importantly, this series allows us to run tests with a different
> > object storage backend via `GIT_TEST_DEFAULT_OBJECT_STORAGE_FORMAT=`. As
> > we don't have an alternative backend, I also introduce the ability to
> > specify a default object (and ref) storage format that carries a payload
> > so that you can run all tests with "files://objdir", for example. That
> > surfaces a bunch of tests that assume objects exist in ".git/objects".
> >
> > Overall it's a surprisingly small number of test failures, only 1105 out
> > of 35000 tests fail. Fixing those is not part of this patch series, but
> > we'll start upstreaming fixes once this lands. Such tests will either
> > have to be adapted to use plumbing commands, or they'll need to be
> > tagged with a prerequisite. It's going to be a bunch of work, but it's
> > basically preparing for the introduction of alternative backends
> > already.
> >
> > The series is built on top of 0ffa3e35c7c (The 15th batch, 2026-11-04)
> > with ps/odb-object-storage-extension at 87a55d790ca (builtin/submodule:
> > propagate "--object-storage-format=", 2026-10-27) merged into it.
> >
> > Thanks!
> >
> > Patrick
> >
> 
> This was a very easy read. I don't have much to add apart from a comment
> on 1/5. But you can also leave it as is! Thanks

Thanks for your review! I'd prefer to leave 1/5 as-is as explained in my
reply there.

Patrick

Re: [PATCH 5/5] t: allow testing with a non-default object storage foramt

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> Subject: Re: [PATCH 5/5] t: allow testing with a non-default object storage foramt

"foramt" -> "format".

> diff --git a/t/test-lib.sh b/t/test-lib.sh
> index a893963f86c..7fdd8272162 100644
> --- a/t/test-lib.sh
> +++ b/t/test-lib.sh
> @@ -576,9 +576,13 @@ export EDITOR
>  GIT_TEST_BUILTIN_HASH=$("$GIT_BINARY" version --build-options | sed -ne 's/^default-hash: //p')
>  GIT_DEFAULT_HASH="${GIT_TEST_DEFAULT_HASH:-$GIT_TEST_BUILTIN_HASH}"
>  export GIT_DEFAULT_HASH
> +
>  GIT_DEFAULT_REF_STORAGE_FORMAT="${GIT_TEST_DEFAULT_REF_STORAGE_FORMAT:-files}"
>  export GIT_DEFAULT_REF_STORAGE_FORMAT

This unrelated change created unnecessary conflicts.

No need to resend.

>  
> +GIT_DEFAULT_OBJECT_STORAGE_FORMAT="${GIT_TEST_DEFAULT_OBJECT_STORAGE_FORMAT:-files}"
> +export GIT_DEFAULT_OBJECT_STORAGE_FORMAT
> +
>  # Tests using GIT_TRACE typically don't want <timestamp> <file>:<line> output
>  GIT_TRACE_BARE=1
>  export GIT_TRACE_BARE