From: Patrick Steinhardt <ps@pks.im>
To: git@vger.kernel.org
Cc: Karthik Nayak <karthik.188@gmail.com>,
Junio C Hamano <gitster@pobox.com>,
Eric Sunshine <sunshine@sunshineco.com>
Subject: [PATCH v3 00/12] Introduce `refStorage` extension
Date: Fri, 29 Dec 2023 08:26:22 +0100 [thread overview]
Message-ID: <cover.1703833818.git.ps@pks.im> (raw)
In-Reply-To: <cover.1703067989.git.ps@pks.im>
[-- Attachment #1: Type: text/plain, Size: 15642 bytes --]
Hi,
this is the third version of my patch series that introduces the new
`refStorage` extension. This extension will be used for the upcoming
reftable backend.
Changes compared to v3:
- The `ref_storage_format` is now tracked as an `unsigned int`,
proposed by Junio.
- Reworded the commit message in patch 2, proposed by Eric.
- Added a NEEDSWORK comment to `get_worktrees_internal()`, propose by
Eric.
Thanks for your reviews!
Patrick
Patrick Steinhardt (12):
t: introduce DEFAULT_REPO_FORMAT prereq
worktree: skip reading HEAD when repairing worktrees
refs: refactor logic to look up storage backends
setup: start tracking ref storage format
setup: set repository's formats on init
setup: introduce "extensions.refStorage" extension
setup: introduce GIT_DEFAULT_REF_FORMAT envvar
t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvar
builtin/rev-parse: introduce `--show-ref-format` flag
builtin/init: introduce `--ref-format=` value flag
builtin/clone: introduce `--ref-format=` value flag
t9500: write "extensions.refstorage" into config
Documentation/config/extensions.txt | 11 +++
Documentation/git-clone.txt | 6 ++
Documentation/git-init.txt | 7 ++
Documentation/git-rev-parse.txt | 3 +
Documentation/git.txt | 5 ++
Documentation/ref-storage-format.txt | 1 +
.../technical/repository-version.txt | 5 ++
builtin/clone.c | 17 ++++-
builtin/init-db.c | 15 +++-
builtin/rev-parse.c | 4 ++
refs.c | 34 ++++++---
refs.h | 3 +
refs/debug.c | 1 -
refs/files-backend.c | 1 -
refs/packed-backend.c | 1 -
refs/refs-internal.h | 1 -
repository.c | 6 ++
repository.h | 7 ++
setup.c | 66 +++++++++++++++--
setup.h | 10 ++-
t/README | 3 +
t/t0001-init.sh | 70 +++++++++++++++++++
t/t1500-rev-parse.sh | 17 +++++
t/t3200-branch.sh | 2 +-
t/t5601-clone.sh | 17 +++++
t/t9500-gitweb-standalone-no-errors.sh | 5 ++
t/test-lib-functions.sh | 5 ++
t/test-lib.sh | 15 +++-
worktree.c | 31 +++++---
29 files changed, 334 insertions(+), 35 deletions(-)
create mode 100644 Documentation/ref-storage-format.txt
Range-diff against v2:
1: 3613439cb7 = 1: 578deaabcf t: introduce DEFAULT_REPO_FORMAT prereq
2: ecf4f1ddee ! 2: 77c7213c66 worktree: skip reading HEAD when repairing worktrees
@@ Commit message
is logic that we resolve their respective worktree HEADs, even though
that information may not actually be needed in the end by all callers.
- In the context of git-init(1) this is about to become a problem, because
- we do not have a repository that was set up via `setup_git_directory()`
- or friends. Consequentially, it is not yet fully initialized at the time
- of calling `repair_worktrees()`, and properly setting up all parts of
- the repository in `init_db()` before we repair worktrees is not an easy
- thing to do. While this is okay right now where we only have a single
- reference backend in Git, once we gain a second one we would be trying
- to look up the worktree HEADs before we have figured out the reference
- format, which does not work.
+ Although not a problem presently with the file-based reference backend,
+ it will become a problem with the upcoming reftable backend. In the
+ context of git-init(1) we do not have a fully-initialized repository set
+ up via `setup_git_directory()` or friends. Consequently, we do not know
+ about the repository format when `repair_worktrees()` is called, and
+ properly setting up all parts of the repositroy in `init_db()` before we
+ try to repair worktrees is not an easy task. With the introduction of
+ the reftable backend, we would ultimately try to look up the worktree
+ HEADs before we have figured out the reference format, which does not
+ work.
We do not require the worktree HEADs at all to repair worktrees. So
let's fix this issue by skipping over the step that reads them.
@@ worktree.c: static void mark_current_worktree(struct worktree **worktrees)
}
-struct worktree **get_worktrees(void)
++/*
++ * NEEDSWORK: This function exists so that we can look up metadata of a
++ * worktree without trying to access any of its internals like the refdb. It
++ * would be preferable to instead have a corruption-tolerant function for
++ * retrieving worktree metadata that could be used when the worktree is known
++ * to not be in a healthy state, e.g. when creating or repairing it.
++ */
+static struct worktree **get_worktrees_internal(int skip_reading_head)
{
struct worktree **list = NULL;
3: 12329b99b7 ! 3: 47649570bf refs: refactor logic to look up storage backends
@@ refs.c
+};
-static struct ref_storage_be *find_ref_storage_backend(const char *name)
-+static const struct ref_storage_be *find_ref_storage_backend(int ref_storage_format)
++static const struct ref_storage_be *find_ref_storage_backend(unsigned int ref_storage_format)
{
- struct ref_storage_be *be;
- for (be = refs_backends; be; be = be->next)
@@ refs.c
return NULL;
}
-+int ref_storage_format_by_name(const char *name)
++unsigned int ref_storage_format_by_name(const char *name)
+{
-+ for (int i = 0; i < ARRAY_SIZE(refs_backends); i++)
++ for (unsigned int i = 0; i < ARRAY_SIZE(refs_backends); i++)
+ if (refs_backends[i] && !strcmp(refs_backends[i]->name, name))
+ return i;
+ return REF_STORAGE_FORMAT_UNKNOWN;
+}
+
-+const char *ref_storage_format_to_name(int ref_storage_format)
++const char *ref_storage_format_to_name(unsigned int ref_storage_format)
+{
+ const struct ref_storage_be *be = find_ref_storage_backend(ref_storage_format);
+ if (!be)
@@ refs.c: static struct ref_store *ref_store_init(struct repository *repo,
{
- const char *be_name = "files";
- struct ref_storage_be *be = find_ref_storage_backend(be_name);
-+ int format = REF_STORAGE_FORMAT_FILES;
++ unsigned int format = REF_STORAGE_FORMAT_FILES;
+ const struct ref_storage_be *be = find_ref_storage_backend(format);
struct ref_store *refs;
@@ refs.h: struct string_list;
struct string_list_item;
struct worktree;
-+int ref_storage_format_by_name(const char *name);
-+const char *ref_storage_format_to_name(int ref_storage_format);
++unsigned int ref_storage_format_by_name(const char *name);
++const char *ref_storage_format_to_name(unsigned int ref_storage_format);
+
/*
* Resolve a reference, recursively following symbolic refererences.
4: ddd099fbaf ! 4: 837764d0b5 setup: start tracking ref storage format
@@ refs.c: static struct ref_store *ref_store_init(struct repository *repo,
const char *gitdir,
unsigned int flags)
{
-- int format = REF_STORAGE_FORMAT_FILES;
+- unsigned int format = REF_STORAGE_FORMAT_FILES;
- const struct ref_storage_be *be = find_ref_storage_backend(format);
+ const struct ref_storage_be *be;
struct ref_store *refs;
@@ repository.c: void repo_set_hash_algo(struct repository *repo, int hash_algo)
repo->hash_algo = &hash_algos[hash_algo];
}
-+void repo_set_ref_storage_format(struct repository *repo, int format)
++void repo_set_ref_storage_format(struct repository *repo, unsigned int format)
+{
+ repo->ref_storage_format = format;
+}
@@ repository.h: struct repository {
const struct git_hash_algo *hash_algo;
+ /* Repository's reference storage format, as serialized on disk. */
-+ int ref_storage_format;
++ unsigned int ref_storage_format;
+
/* A unique-id for tracing purposes. */
int trace2_repo_id;
@@ repository.h: void repo_set_gitdir(struct repository *repo, const char *root,
const struct set_gitdir_args *extra_args);
void repo_set_worktree(struct repository *repo, const char *path);
void repo_set_hash_algo(struct repository *repo, int algo);
-+void repo_set_ref_storage_format(struct repository *repo, int format);
++void repo_set_ref_storage_format(struct repository *repo, unsigned int format);
void initialize_the_repository(void);
RESULT_MUST_BE_USED
int repo_init(struct repository *r, const char *gitdir, const char *worktree);
@@ setup.c: static int is_reinit(void)
}
-void create_reference_database(const char *initial_branch, int quiet)
-+void create_reference_database(int ref_storage_format,
++void create_reference_database(unsigned int ref_storage_format,
+ const char *initial_branch, int quiet)
{
struct strbuf err = STRBUF_INIT;
@@ setup.c: static void validate_hash_algorithm(struct repository_format *repo_fmt,
}
}
-+static void validate_ref_storage_format(struct repository_format *repo_fmt, int format)
++static void validate_ref_storage_format(struct repository_format *repo_fmt,
++ unsigned int format)
+{
+ if (repo_fmt->version >= 0 &&
+ format != REF_STORAGE_FORMAT_UNKNOWN &&
@@ setup.c: static void validate_hash_algorithm(struct repository_format *repo_fmt,
int init_db(const char *git_dir, const char *real_git_dir,
- const char *template_dir, int hash, const char *initial_branch,
+ const char *template_dir, int hash,
-+ int ref_storage_format, const char *initial_branch,
++ unsigned int ref_storage_format,
++ const char *initial_branch,
int init_shared_repository, unsigned int flags)
{
int reinit;
@@ setup.h: struct repository_format {
int worktree_config;
int is_bare;
int hash_algo;
-+ int ref_storage_format;
++ unsigned int ref_storage_format;
int sparse_index;
char *work_tree;
struct string_list unknown_extensions;
@@ setup.h: void check_repository_format(struct repository_format *fmt);
int init_db(const char *git_dir, const char *real_git_dir,
const char *template_dir, int hash_algo,
-+ int ref_storage_format,
++ unsigned int ref_storage_format,
const char *initial_branch, int init_shared_repository,
unsigned int flags);
void initialize_repository_version(int hash_algo, int reinit);
-void create_reference_database(const char *initial_branch, int quiet);
-+void create_reference_database(int ref_storage_format,
++void create_reference_database(unsigned int ref_storage_format,
+ const char *initial_branch, int quiet);
/*
5: 01a1e58a97 = 5: a51da56d9b setup: set repository's formats on init
6: 0a586fa648 ! 6: a1e03e4392 setup: introduce "extensions.refStorage" extension
@@ setup.c: static enum extension_result handle_extension(const char *var,
data->hash_algo = format;
return EXTENSION_OK;
+ } else if (!strcmp(ext, "refstorage")) {
-+ int format;
++ unsigned int format;
+
+ if (!value)
+ return config_error_nonbool(var);
@@ setup.c: static int needs_work_tree_config(const char *git_dir, const char *work
}
-void initialize_repository_version(int hash_algo, int reinit)
-+void initialize_repository_version(int hash_algo, int ref_storage_format,
++void initialize_repository_version(int hash_algo,
++ unsigned int ref_storage_format,
+ int reinit)
{
char repo_version_string[10];
@@ setup.c: static int create_default_files(const char *template_path,
## setup.h ##
@@ setup.h: int init_db(const char *git_dir, const char *real_git_dir,
- int ref_storage_format,
+ unsigned int ref_storage_format,
const char *initial_branch, int init_shared_repository,
unsigned int flags);
-void initialize_repository_version(int hash_algo, int reinit);
-+void initialize_repository_version(int hash_algo, int ref_storage_format,
++void initialize_repository_version(int hash_algo,
++ unsigned int ref_storage_format,
+ int reinit);
- void create_reference_database(int ref_storage_format,
+ void create_reference_database(unsigned int ref_storage_format,
const char *initial_branch, int quiet);
7: 6d8754f73a ! 7: 5ffc70e9be setup: introduce GIT_DEFAULT_REF_FORMAT envvar
@@ Documentation/git.txt: double-quotes and respecting backslash escapes. E.g., the
## setup.c ##
@@ setup.c: static void validate_hash_algorithm(struct repository_format *repo_fmt, int hash
-
- static void validate_ref_storage_format(struct repository_format *repo_fmt, int format)
+ static void validate_ref_storage_format(struct repository_format *repo_fmt,
+ unsigned int format)
{
+ const char *name = getenv("GIT_DEFAULT_REF_FORMAT");
+
8: c645932f3d = 8: 13c074acdf t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvar
9: 761d647770 = 9: 4ee3c9a2d1 builtin/rev-parse: introduce `--show-ref-format` flag
10: e382b5bf08 ! 10: 25773e3560 builtin/init: introduce `--ref-format=` value flag
@@ builtin/init-db.c: int cmd_init_db(int argc, const char **argv, const char *pref
+ const char *ref_format = NULL;
const char *initial_branch = NULL;
int hash_algo = GIT_HASH_UNKNOWN;
-+ int ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN;
++ unsigned int ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN;
int init_shared_repository = -1;
const struct option init_db_options[] = {
OPT_STRING(0, "template", &template_dir, N_("template-directory"),
11: 257233658d ! 11: 3f1cb6b9e5 builtin/clone: introduce `--ref-format=` value flag
@@ builtin/clone.c: int cmd_clone(int argc, const char **argv, const char *prefix)
int submodule_progress;
int filter_submodules = 0;
int hash_algo;
-+ int ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN;
++ unsigned int ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN;
const int do_not_override_repo_unix_permissions = -1;
struct transport_ls_refs_options transport_ls_refs_options =
12: b8cd06ec53 = 12: 2e7682b2f3 t9500: write "extensions.refstorage" into config
base-commit: e79552d19784ee7f4bbce278fe25f93fbda196fa
--
2.43.GIT
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
next prev parent reply other threads:[~2023-12-29 7:26 UTC|newest]
Thread overview: 60+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-12-20 10:54 [PATCH 00/12] Introduce `refStorage` extension Patrick Steinhardt
2023-12-20 10:54 ` [PATCH 01/12] t: introduce DEFAULT_REPO_FORMAT prereq Patrick Steinhardt
2023-12-22 11:41 ` Karthik Nayak
2023-12-28 8:55 ` Patrick Steinhardt
2023-12-20 10:54 ` [PATCH 02/12] worktree: skip reading HEAD when repairing worktrees Patrick Steinhardt
2023-12-22 12:23 ` Karthik Nayak
2023-12-20 10:55 ` [PATCH 03/12] refs: refactor logic to look up storage backends Patrick Steinhardt
2023-12-22 12:38 ` Karthik Nayak
2023-12-28 8:56 ` Patrick Steinhardt
2023-12-20 10:55 ` [PATCH 04/12] setup: start tracking ref storage format when Patrick Steinhardt
2023-12-20 18:30 ` Junio C Hamano
2023-12-28 8:56 ` Patrick Steinhardt
2023-12-28 17:15 ` Junio C Hamano
2023-12-28 20:01 ` Patrick Steinhardt
2023-12-22 13:09 ` Karthik Nayak
2023-12-28 8:56 ` Patrick Steinhardt
2023-12-20 10:55 ` [PATCH 05/12] setup: set repository's formats on init Patrick Steinhardt
2023-12-20 10:55 ` [PATCH 06/12] setup: introduce "extensions.refStorage" extension Patrick Steinhardt
2023-12-20 10:55 ` [PATCH 07/12] setup: introduce GIT_DEFAULT_REF_FORMAT envvar Patrick Steinhardt
2023-12-20 10:55 ` [PATCH 08/12] t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvar Patrick Steinhardt
2023-12-20 10:55 ` [PATCH 09/12] builtin/rev-parse: introduce `--show-ref-format` flag Patrick Steinhardt
2023-12-20 10:55 ` [PATCH 10/12] builtin/init: introduce `--ref-format=` value flag Patrick Steinhardt
2023-12-20 10:55 ` [PATCH 11/12] builtin/clone: " Patrick Steinhardt
2023-12-20 10:55 ` [PATCH 12/12] t9500: write "extensions.refstorage" into config Patrick Steinhardt
2023-12-22 13:43 ` Karthik Nayak
2023-12-22 13:43 ` [PATCH 00/12] Introduce `refStorage` extension Karthik Nayak
2023-12-26 20:56 ` Junio C Hamano
2023-12-28 9:57 ` [PATCH v2 " Patrick Steinhardt
2023-12-28 9:57 ` [PATCH v2 01/12] t: introduce DEFAULT_REPO_FORMAT prereq Patrick Steinhardt
2023-12-28 9:57 ` [PATCH v2 02/12] worktree: skip reading HEAD when repairing worktrees Patrick Steinhardt
2023-12-28 18:08 ` Eric Sunshine
2023-12-28 18:13 ` Eric Sunshine
2023-12-28 20:18 ` Patrick Steinhardt
2023-12-28 9:57 ` [PATCH v2 03/12] refs: refactor logic to look up storage backends Patrick Steinhardt
2023-12-28 17:25 ` Junio C Hamano
2023-12-28 20:11 ` Patrick Steinhardt
2023-12-28 20:42 ` Junio C Hamano
2023-12-28 9:57 ` [PATCH v2 04/12] setup: start tracking ref storage format Patrick Steinhardt
2023-12-28 9:57 ` [PATCH v2 05/12] setup: set repository's formats on init Patrick Steinhardt
2023-12-28 9:57 ` [PATCH v2 06/12] setup: introduce "extensions.refStorage" extension Patrick Steinhardt
2023-12-28 9:57 ` [PATCH v2 07/12] setup: introduce GIT_DEFAULT_REF_FORMAT envvar Patrick Steinhardt
2023-12-28 9:57 ` [PATCH v2 08/12] t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvar Patrick Steinhardt
2023-12-28 9:58 ` [PATCH v2 09/12] builtin/rev-parse: introduce `--show-ref-format` flag Patrick Steinhardt
2023-12-28 9:58 ` [PATCH v2 10/12] builtin/init: introduce `--ref-format=` value flag Patrick Steinhardt
2023-12-28 9:58 ` [PATCH v2 11/12] builtin/clone: " Patrick Steinhardt
2023-12-28 9:58 ` [PATCH v2 12/12] t9500: write "extensions.refstorage" into config Patrick Steinhardt
2023-12-29 7:26 ` Patrick Steinhardt [this message]
2023-12-29 7:26 ` [PATCH v3 01/12] t: introduce DEFAULT_REPO_FORMAT prereq Patrick Steinhardt
2023-12-29 7:26 ` [PATCH v3 02/12] worktree: skip reading HEAD when repairing worktrees Patrick Steinhardt
2023-12-29 7:26 ` [PATCH v3 03/12] refs: refactor logic to look up storage backends Patrick Steinhardt
2023-12-29 7:26 ` [PATCH v3 04/12] setup: start tracking ref storage format Patrick Steinhardt
2023-12-29 7:26 ` [PATCH v3 05/12] setup: set repository's formats on init Patrick Steinhardt
2023-12-29 7:26 ` [PATCH v3 06/12] setup: introduce "extensions.refStorage" extension Patrick Steinhardt
2023-12-29 7:26 ` [PATCH v3 07/12] setup: introduce GIT_DEFAULT_REF_FORMAT envvar Patrick Steinhardt
2023-12-29 7:26 ` [PATCH v3 08/12] t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvar Patrick Steinhardt
2023-12-29 7:27 ` [PATCH v3 09/12] builtin/rev-parse: introduce `--show-ref-format` flag Patrick Steinhardt
2023-12-29 7:27 ` [PATCH v3 10/12] builtin/init: introduce `--ref-format=` value flag Patrick Steinhardt
2023-12-29 7:27 ` [PATCH v3 11/12] builtin/clone: " Patrick Steinhardt
2023-12-29 7:27 ` [PATCH v3 12/12] t9500: write "extensions.refstorage" into config Patrick Steinhardt
2023-12-29 8:44 ` [PATCH v3 00/12] Introduce `refStorage` extension Eric Sunshine
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1703833818.git.ps@pks.im \
--to=ps@pks.im \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=karthik.188@gmail.com \
--cc=sunshine@sunshineco.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.