* [PATCH 0/6] Support hashing objects larger than 4GB on Windows
From: Johannes Schindelin via GitGitGadget @ 2026-06-04 17:15 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin
Philip Oakley has contributed these patches ~4.5 years ago, and they have
been carried in Git for Windows ever since.
Now that there are already other patch series flying around that try to
address various aspects about >4GB objects (which aren't handled well by Git
until it stops forcing unsigned long to do size_t's job), it seems a good
time to upstream these patches, too, at long last.
Philip Oakley (6):
hash-object: demonstrate a >4GB/LLP64 problem
object-file.c: use size_t for header lengths
hash algorithms: use size_t for section lengths
hash-object --stdin: verify that it works with >4GB/LLP64
hash-object: add another >4GB/LLP64 test case
hash-object: add a >4GB/LLP64 test case using filtered input
object-file.c | 18 +++++++++---------
object-file.h | 4 ++--
sha1dc_git.c | 3 +--
sha1dc_git.h | 2 +-
t/t1007-hash-object.sh | 39 +++++++++++++++++++++++++++++++++++++++
5 files changed, 52 insertions(+), 14 deletions(-)
base-commit: 9ac3f193c05c2237e2b14ebaa1149e9fc8a1abe0
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2138%2Fdscho%2FPhilipOakley%2Fhashliteral_t-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2138/dscho/PhilipOakley/hashliteral_t-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/2138
--
gitgitgadget
^ permalink raw reply
* Re: [PATCH] docs: fix typos
From: Tuomas Ahola @ 2026-06-04 16:55 UTC (permalink / raw)
To: Kristoffer Haugsbakk; +Cc: git
In-Reply-To: <92fe3db2-83bd-4aa9-a1f4-bec01dfaf8ca@app.fastmail.com>
"Kristoffer Haugsbakk" <kristofferhaugsbakk@fastmail.com> wrote:
> On Thu, Jun 4, 2026, at 15:14, Tuomas Ahola wrote:
> > [PATCH] docs: fix typos
>
> The area `docs` isn’t correct since you are also changing comments in
> source files.
>
> `*` could be used (as in a wildcard). Other people have used other
> things for “treewide” changes.
>
Hmm, I took that from the other typofix patches we have currently in `seen`.
> > * The tm->tm_mday field has an additional logic of using negative values
> > * for date adjustments: -2 means yesterday and -3 the day before that,
> > - * and so on. The idea is to deref such adjustments until we are sure
> > + * and so on. The idea is to defer such adjustments until we are sure
>
> “deref” could have been “dereference” but this must indeed mean
> “defer”. We are putting off a decision until later.
>
Yes, that was my own typo. I always meant "defer".
> >
> > base-commit: 94f057755b7941b321fd11fec1b2e3ca5313a4e0
> > prerequisite-patch-id: f827362e061e199150f149dd36c67664c77406bc
> > prerequisite-patch-id: e5b32f0b916ec86eab6631b9bd9bafd639191765
> > [...]
> > prerequisite-patch-id: 083f554bc5e09ae54c6b545628196e11a9e90cea
>
> Okay, these must be all the non-merge commits from the topics you
> merged in.
>
That's true. Quite a list though.
Thanks for review!
^ permalink raw reply
* [PATCH] Documentation: remove redundant 'instead' in --subject-prefix
From: Lucas Seiki Oshiro @ 2026-06-04 16:34 UTC (permalink / raw)
To: git; +Cc: Lucas Seiki Oshiro
The documentation for --subject-prefix has two words "instead" in
the same sentence, making it a little bit confusing to read.
Change the order of the phrase to a more natural "Use [...]
instead of [...]" structure.
Signed-off-by: Lucas Seiki Oshiro <lucasseikioshiro@gmail.com>
---
Documentation/git-format-patch.adoc | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/Documentation/git-format-patch.adoc b/Documentation/git-format-patch.adoc
index 5662382450..f7905c0f7c 100644
--- a/Documentation/git-format-patch.adoc
+++ b/Documentation/git-format-patch.adoc
@@ -221,10 +221,9 @@ populated with placeholder text.
for generating the cover letter.
--subject-prefix=<subject-prefix>::
- Instead of the standard '[PATCH]' prefix in the subject
- line, instead use '[<subject-prefix>]'. This can be used
- to name a patch series, and can be combined with the
- `--numbered` option.
+ Use '[<subject-prefix>]' instead of the standard '[PATCH]'
+ prefix in the subject line. This can be used to name a patch
+ series, and can be combined with the `--numbered` option.
+
The configuration variable `format.subjectPrefix` may also be used
to configure a subject prefix to apply to a given repository for
--
2.50.1 (Apple Git-155)
^ permalink raw reply related
* [PATCH 2/2] mingw: really handle SIGINT
From: Johannes Schindelin via GitGitGadget @ 2026-06-04 16:24 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Johannes Schindelin
In-Reply-To: <pull.2130.git.1780590261.gitgitgadget@gmail.com>
From: Johannes Schindelin <johannes.schindelin@gmx.de>
Previously, we did not install any handler for Ctrl+C, but now we really
want to because the MSYS2 runtime learned the trick to call the
ConsoleCtrlHandler when Ctrl+C was pressed.
With this, hitting Ctrl+C while `git log` is running will only terminate
the Git process, but not the pager. This finally matches the behavior on
Linux and on macOS.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
compat/mingw.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/compat/mingw.c b/compat/mingw.c
index 973049ffe3..f2b6c51f98 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -3580,7 +3580,14 @@ static void adjust_symlink_flags(void)
symlink_file_flags |= 2;
symlink_directory_flags |= 2;
}
+}
+static BOOL WINAPI handle_ctrl_c(DWORD ctrl_type)
+{
+ if (ctrl_type != CTRL_C_EVENT)
+ return FALSE; /* we did not handle this */
+ mingw_raise(SIGINT);
+ return TRUE; /* we did handle this */
}
#ifdef _MSC_VER
@@ -3617,6 +3624,8 @@ int wmain(int argc, const wchar_t **wargv)
#endif
#endif
+ SetConsoleCtrlHandler(handle_ctrl_c, TRUE);
+
maybe_redirect_std_handles();
adjust_symlink_flags();
--
gitgitgadget
^ permalink raw reply related
* [PATCH 1/2] mingw: kill child processes in a gentler way
From: Johannes Schindelin via GitGitGadget @ 2026-06-04 16:24 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Johannes Schindelin
In-Reply-To: <pull.2130.git.1780590261.gitgitgadget@gmail.com>
From: Johannes Schindelin <johannes.schindelin@gmx.de>
The TerminateProcess() function does not actually leave the child
processes any chance to perform any cleanup operations. This is bad
insofar as Git itself expects its signal handlers to run.
A symptom is e.g. a left-behind .lock file that would not be left behind
if the same operation was run, say, on Linux.
To remedy this situation, we use an obscure trick: we inject a thread
into the process that needs to be killed and to let that thread run the
ExitProcess() function with the desired exit status. Thanks J Wyman for
describing this trick.
The advantage is that the ExitProcess() function lets the atexit
handlers run. While this is still different from what Git expects (i.e.
running a signal handler), in practice Git sets up signal handlers and
atexit handlers that call the same code to clean up after itself.
In case that the gentle method to terminate the process failed, we still
fall back to calling TerminateProcess(), but in that case we now also
make sure that processes spawned by the spawned process are terminated;
TerminateProcess() does not give the spawned process a chance to do so
itself.
Please note that this change only affects how Git for Windows tries to
terminate processes spawned by Git's own executables. Third-party
software that *calls* Git and wants to terminate it *still* need to make
sure to imitate this gentle method, otherwise this patch will not have
any effect.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
compat/mingw.c | 29 +++++--
compat/win32/exit-process.h | 165 ++++++++++++++++++++++++++++++++++++
2 files changed, 186 insertions(+), 8 deletions(-)
create mode 100644 compat/win32/exit-process.h
diff --git a/compat/mingw.c b/compat/mingw.c
index 2023c16db6..973049ffe3 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -13,6 +13,7 @@
#include "symlinks.h"
#include "trace2.h"
#include "win32.h"
+#include "win32/exit-process.h"
#include "win32/lazyload.h"
#include "wrapper.h"
#include <aclapi.h>
@@ -2208,16 +2209,28 @@ int mingw_execvp(const char *cmd, char *const *argv)
int mingw_kill(pid_t pid, int sig)
{
if (pid > 0 && sig == SIGTERM) {
- HANDLE h = OpenProcess(PROCESS_TERMINATE, FALSE, pid);
-
- if (TerminateProcess(h, -1)) {
+ HANDLE h = OpenProcess(PROCESS_CREATE_THREAD |
+ PROCESS_QUERY_INFORMATION |
+ PROCESS_VM_OPERATION | PROCESS_VM_WRITE |
+ PROCESS_VM_READ | PROCESS_TERMINATE,
+ FALSE, pid);
+ int ret;
+
+ if (h)
+ ret = exit_process(h, 128 + sig);
+ else {
+ h = OpenProcess(PROCESS_TERMINATE, FALSE, pid);
+ if (!h) {
+ errno = err_win_to_posix(GetLastError());
+ return -1;
+ }
+ ret = terminate_process_tree(h, 128 + sig);
+ }
+ if (ret) {
+ errno = err_win_to_posix(GetLastError());
CloseHandle(h);
- return 0;
}
-
- errno = err_win_to_posix(GetLastError());
- CloseHandle(h);
- return -1;
+ return ret;
} else if (pid > 0 && sig == 0) {
HANDLE h = OpenProcess(PROCESS_QUERY_INFORMATION, FALSE, pid);
if (h) {
diff --git a/compat/win32/exit-process.h b/compat/win32/exit-process.h
new file mode 100644
index 0000000000..d53989884c
--- /dev/null
+++ b/compat/win32/exit-process.h
@@ -0,0 +1,165 @@
+#ifndef EXIT_PROCESS_H
+#define EXIT_PROCESS_H
+
+/*
+ * This file contains functions to terminate a Win32 process, as gently as
+ * possible.
+ *
+ * At first, we will attempt to inject a thread that calls ExitProcess(). If
+ * that fails, we will fall back to terminating the entire process tree.
+ *
+ * For simplicity, these functions are marked as file-local.
+ */
+
+#include <tlhelp32.h>
+
+/*
+ * Terminates the process corresponding to the process ID and all of its
+ * directly and indirectly spawned subprocesses.
+ *
+ * This way of terminating the processes is not gentle: the processes get
+ * no chance of cleaning up after themselves (closing file handles, removing
+ * .lock files, terminating spawned processes (if any), etc).
+ */
+static int terminate_process_tree(HANDLE main_process, int exit_status)
+{
+ HANDLE snapshot = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0);
+ PROCESSENTRY32 entry;
+ DWORD pids[16384];
+ int max_len = sizeof(pids) / sizeof(*pids), i, len, ret = 0;
+ pid_t pid = GetProcessId(main_process);
+
+ pids[0] = (DWORD)pid;
+ len = 1;
+
+ /*
+ * Even if Process32First()/Process32Next() seem to traverse the
+ * processes in topological order (i.e. parent processes before
+ * child processes), there is nothing in the Win32 API documentation
+ * suggesting that this is guaranteed.
+ *
+ * Therefore, run through them at least twice and stop when no more
+ * process IDs were added to the list.
+ */
+ for (;;) {
+ int orig_len = len;
+
+ memset(&entry, 0, sizeof(entry));
+ entry.dwSize = sizeof(entry);
+
+ if (!Process32First(snapshot, &entry))
+ break;
+
+ do {
+ for (i = len - 1; i >= 0; i--) {
+ if (pids[i] == entry.th32ProcessID)
+ break;
+ if (pids[i] == entry.th32ParentProcessID)
+ pids[len++] = entry.th32ProcessID;
+ }
+ } while (len < max_len && Process32Next(snapshot, &entry));
+
+ if (orig_len == len || len >= max_len)
+ break;
+ }
+
+ for (i = len - 1; i > 0; i--) {
+ HANDLE process = OpenProcess(PROCESS_TERMINATE, FALSE, pids[i]);
+
+ if (process) {
+ if (!TerminateProcess(process, exit_status))
+ ret = -1;
+ CloseHandle(process);
+ }
+ }
+ if (!TerminateProcess(main_process, exit_status))
+ ret = -1;
+ CloseHandle(main_process);
+
+ return ret;
+}
+
+/**
+ * Determine whether a process runs in the same architecture as the current
+ * one. That test is required before we assume that GetProcAddress() returns
+ * a valid address *for the target process*.
+ */
+static inline int process_architecture_matches_current(HANDLE process)
+{
+ static BOOL current_is_wow = -1;
+ BOOL is_wow;
+
+ if (current_is_wow == -1 &&
+ !IsWow64Process (GetCurrentProcess(), ¤t_is_wow))
+ current_is_wow = -2;
+ if (current_is_wow == -2)
+ return 0; /* could not determine current process' WoW-ness */
+ if (!IsWow64Process (process, &is_wow))
+ return 0; /* cannot determine */
+ return is_wow == current_is_wow;
+}
+
+/**
+ * Inject a thread into the given process that runs ExitProcess().
+ *
+ * Note: as kernel32.dll is loaded before any process, the other process and
+ * this process will have ExitProcess() at the same address.
+ *
+ * This function expects the process handle to have the access rights for
+ * CreateRemoteThread(): PROCESS_CREATE_THREAD, PROCESS_QUERY_INFORMATION,
+ * PROCESS_VM_OPERATION, PROCESS_VM_WRITE, and PROCESS_VM_READ.
+ *
+ * The idea comes from the Dr Dobb's article "A Safer Alternative to
+ * TerminateProcess()" by Andrew Tucker (July 1, 1999),
+ * http://www.drdobbs.com/a-safer-alternative-to-terminateprocess/184416547
+ *
+ * If this method fails, we fall back to running terminate_process_tree().
+ */
+static int exit_process(HANDLE process, int exit_code)
+{
+ DWORD code;
+
+ if (GetExitCodeProcess(process, &code) && code == STILL_ACTIVE) {
+ static int initialized;
+ static LPTHREAD_START_ROUTINE exit_process_address;
+ PVOID arg = (PVOID)(intptr_t)exit_code;
+ DWORD thread_id;
+ HANDLE thread = NULL;
+
+ if (!initialized) {
+ HINSTANCE kernel32 = GetModuleHandleA("kernel32");
+ if (!kernel32)
+ die("BUG: cannot find kernel32");
+ exit_process_address =
+ (LPTHREAD_START_ROUTINE)(void (*)(void))
+ GetProcAddress(kernel32, "ExitProcess");
+ initialized = 1;
+ }
+ if (!exit_process_address ||
+ !process_architecture_matches_current(process))
+ return terminate_process_tree(process, exit_code);
+
+ thread = CreateRemoteThread(process, NULL, 0,
+ exit_process_address,
+ arg, 0, &thread_id);
+ if (thread) {
+ CloseHandle(thread);
+ /*
+ * If the process survives for 10 seconds (a completely
+ * arbitrary value picked from thin air), fall back to
+ * killing the process tree via TerminateProcess().
+ */
+ if (WaitForSingleObject(process, 10000) ==
+ WAIT_OBJECT_0) {
+ CloseHandle(process);
+ return 0;
+ }
+ }
+
+ return terminate_process_tree(process, exit_code);
+ }
+
+ return 0;
+}
+
+#endif
--
gitgitgadget
^ permalink raw reply related
* [PATCH 0/2] mingw: terminate child processes in a gentler way
From: Johannes Schindelin via GitGitGadget @ 2026-06-04 16:24 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin
This patch series consists of two patches that have been carried in Git for
Windows since 2017 an 2018, respectively.
The problem they work around is a fundamental mismatch between Git's
understanding how processes can be terminated and Windows' multi-threading
centric world view (where multi-process architectures are quite, quite
rare), where processes do not tell other processes to terminate gently
(meaning: giving them a chance to run their atexit() handlers).
As such, Git thinks that it can send processes signals to terminate or
force-stop ("kill") them. There are no signals in the Unix sense on Windows,
though. So we try to emulate them. At present, in vanilla Git that means
that we use the TerminateProcess() Win32 API functions, which is most
similar to Unix' SIGKILL and is typically frowned upon because it does not
allow an orderly shutdown of multi-threaded applications. That's definitely
not what Git wants to do: If it wants to terminate a child process, it wants
that child process to clean up any .lock files, for example. And therefore
it wants to send a SIGTERM.
But the SIGTERM signal does not really have any equivalent on Windows. The
closest is to somehow get the target process to call the ExitProcess() Win32
API function. There is a trick that we employ here to do precisely that: we
create a remote thread in the target process, and specify the ExitProcess()
function as the callee. This works because that function matches the
function signature of thread functions enough that we can get away with it,
and because the address of that function is identical between processes
matching the same CPU architecture. Read: This approach does not work when
trying to terminate i686 processes from an x86_64 git.exe. But since it is
rare to mix and match processes of different CPU architectures on Windows
(certainly in Git scenarios), we kind of resort to this best effort that
works often enough to make it worthwhile.
It's a different story for SIGINT: That signal matches most closely what
Windows calls a ConsoleCtrlEvent. It is different, though, in that a
ConsoleCtrlEvent is not sent to a process, but to a Console, and is handled
by all processes that are attached to said Console. In the MSYS2 runtime
that provides the POSIX emulation layer required by the Bash distributed
with Git for Windows, we work around that by using a similar trick as the
SIGTERM/ExitProcess() injection: a thread is injected into the remote
process, passing the address of the (undocumented) kernel32!CtrlRoutine.
This is quite hacky and requires spawning a separate process to just to
figure out the address of said function, which only works in the MSYS2
runtime because it acquires that address once, and then remembers it for the
rest of its lifetime. Git also simply has no business emulating a Ctrl+C and
instead sends child processes SIGTERM. Therefore, there is no support for
sending SIGINT in this patch series. But patch number 2 implements reacting
to the emulated SIGINT "sent" by the MSYS2 runtime.
Johannes Schindelin (2):
mingw: kill child processes in a gentler way
mingw: really handle SIGINT
compat/mingw.c | 38 +++++++--
compat/win32/exit-process.h | 165 ++++++++++++++++++++++++++++++++++++
2 files changed, 195 insertions(+), 8 deletions(-)
create mode 100644 compat/win32/exit-process.h
base-commit: 94f057755b7941b321fd11fec1b2e3ca5313a4e0
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2130%2Fdscho%2Fmingw-kill-gentle-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2130/dscho/mingw-kill-gentle-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/2130
--
gitgitgadget
^ permalink raw reply
* Re: [PATCH v2] rebase: skip branch symref aliases
From: Phillip Wood @ 2026-06-04 15:37 UTC (permalink / raw)
To: Son Luong Ngoc via GitGitGadget, git; +Cc: Kristoffer Haugsbakk, Son Luong Ngoc
In-Reply-To: <pull.2126.v2.git.1780482436865.gitgitgadget@gmail.com>
On 03/06/2026 11:27, Son Luong Ngoc via GitGitGadget wrote:
> From: Son Luong Ngoc <sluongng@gmail.com>
>
> git rebase --update-refs can fail after the normal rebase path has
> updated the current branch when another local branch is a symref to it.
> This can happen during a default-branch rename where refs/heads/main
> points at refs/heads/master while users migrate.
>
> The sequencer queues update-ref commands from local branch decorations.
> Commit 106b6885c7 (rebase: ignore non-branch update-refs) filters out
> decorations that are not local branches, such as HEAD and tags. A branch
> symref is different: it is still a local branch decoration, but if it
> resolves to another branch then that target branch is itself present in
> the decoration list and will be updated as a concrete branch.
>
> Skip branch decorations whose symrefs resolve to refs/heads/*, because
> those targets are already represented by concrete branch decorations.
> This prevents aliases from scheduling a second update for the same
> branch. Keep symrefs to non-branch targets on the existing path.
Makes sense
> Preserve the existing checked-out branch handling before applying these
> skips. Such refs still need a todo-list comment instead of an update-ref
> command, even when the checked-out ref is the branch being rebased or a
> branch symref alias. Use a copy of the resolved HEAD ref so later ref
> resolution does not overwrite it.
I don't quite understand this. A symref that points to another branch
should always be skipped. When we look up which branches are checked out
(see worktree.c:add_head_info()) we use
refs_resolve_ref_unsafe(get_worktree_ref_store(wt),
"HEAD",
0,
&wt->head_oid, &flags);
so it will never report a symref as being checked out - it always
resolves any symrefs first.
If we have a symref pointing somewhere outside of "refs/heads" then we
need to check whether the target is checked out, not the symref itself.
I'm not sure how likely that is to happen in practice.
> diff --git a/sequencer.c b/sequencer.c
> index 1ee4b2875b..6ab8b47108 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -6445,28 +6445,46 @@ static int add_decorations_to_list(const struct commit *commit,
> struct todo_add_branch_context *ctx)
> {
> const struct name_decoration *decoration = get_name_decoration(&commit->object);
> - const char *head_ref = refs_resolve_ref_unsafe(get_main_ref_store(the_repository),
> - "HEAD",
> - RESOLVE_REF_READING,
> - NULL,
> - NULL);
> + struct ref_store *refs = get_main_ref_store(the_repository);
> + char *head_ref = refs_resolve_refdup(refs, "HEAD",
> + RESOLVE_REF_READING,
> + NULL, NULL);
This part and the test look good now
> while (decoration) {
> struct todo_item *item;
> const char *path;
> + const char *resolved_ref;
> + int flags = 0;
> size_t base_offset = ctx->buf->len;
>
> /*
> - * If the branch is the current HEAD, then it will be
> - * updated by the default rebase behavior.
> - * Exclude it from the list of refs to update,
> - * as well as any non-branch decorations.
> * Non-branch decorations may be present if the pretty format
> * includes "%d", which would have loaded all refs
> * into the global decoration table.
> */
> - if ((head_ref && !strcmp(head_ref, decoration->name)) ||
> - (decoration->type != DECORATION_REF_LOCAL)) {
> + if (decoration->type != DECORATION_REF_LOCAL) {
> + decoration = decoration->next;
> + continue;
> + }
If a decoration matches the current branch why don't we just skip it
like we used to? (As an aside the existing code in wrong because if the
user runs "git rebase --update-refs <upstream> <branch>" HEAD does not
point to "<branch>" but lets not worry about that now)
> + path = branch_checked_out(decoration->name);
As I said above if the symref target is anther branch we should skip it
and if the target is not a branch then we need to check if the target is
checked out so we need to resolve the ref before calling
branch_checked_out().
Thanks
Phillip
^ permalink raw reply
* Re: [PATCH] docs: fix typos
From: Kristoffer Haugsbakk @ 2026-06-04 14:45 UTC (permalink / raw)
To: Tuomas Ahola, git
In-Reply-To: <20260604131457.19215-1-taahol@utu.fi>
On Thu, Jun 4, 2026, at 15:14, Tuomas Ahola wrote:
> [PATCH] docs: fix typos
The area `docs` isn’t correct since you are also changing comments in
source files.
`*` could be used (as in a wildcard). Other people have used other
things for “treewide” changes.
> Fix some typos and grammar errors in comments and documentation files.
>
> Signed-off-by: Tuomas Ahola <taahol@utu.fi>
> ---
>
> Notes:
> Written mostly as an exercise on how to submit patches that depend
> on other topics.
>
> $ git log --oneline --first-parent v2.54.0..
> d19e9182ab (HEAD -> ta/typofixes) docs: fix typos
> 5a7e9cc03d Merge branch 'ta/approxidate-noon-fix'
> f03649d802 Merge branch 'kh/name-rev-custom-format'
> 023a226b4b Merge branch 'jc/neuter-sideband-fixup'
>
> As can be seen, these topics have already graduated to master:
>
> $ git cherry master
> + d19e9182ab097a722e32d459a9a58c8985831e3b
Okay, so you ran this from your branch and git-cherry(1) only found one
non-merge commit that was not already in `master`. Makes sense.
>
> Documentation/config/sideband.adoc | 2 +-
> Documentation/git-format-rev.adoc | 2 +-
> date.c | 2 +-
> replay.h | 2 +-
> t/t9902-completion.sh | 2 +-
> 5 files changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/Documentation/config/sideband.adoc
> b/Documentation/config/sideband.adoc
> index 96fade7f5f..ff007aeb73 100644
> --- a/Documentation/config/sideband.adoc
> +++ b/Documentation/config/sideband.adoc
> @@ -13,7 +13,7 @@ sideband.allowControlCharacters::
> Allow control sequences that move the cursor. This is
> disabled by default.
> `erase`::
> - Allow control sequences that erase charactrs. This is
> + Allow control sequences that erase characters. This is
Correction is correct.
> disabled by default.
> `false`::
> Mask all control characters other than line feeds and
> diff --git a/Documentation/git-format-rev.adoc
> b/Documentation/git-format-rev.adoc
> index c40d52e9f6..505a52fecc 100644
> --- a/Documentation/git-format-rev.adoc
> +++ b/Documentation/git-format-rev.adoc
> @@ -33,7 +33,7 @@ OPTIONS
> The argument `rev` is also accepted.
>
> `text`;; Formats all commit object names found in freeform text. These
> - must the full object names, i.e. abbreviated hexidecimal object
> + must be full object names, i.e. abbreviated hexadecimal object
Correct. It should have been “hexadecimal”.
This also corrects a bewildering “the” where “be” should have been.
> names will not be interpreted.
> +
> Anything that is parsed as an object name but that is not found to be a
> diff --git a/date.c b/date.c
> index 05b78d852f..014065b419 100644
> --- a/date.c
> +++ b/date.c
> @@ -1074,7 +1074,7 @@ void datestamp(struct strbuf *out)
> *
> * The tm->tm_mday field has an additional logic of using negative values
> * for date adjustments: -2 means yesterday and -3 the day before that,
> - * and so on. The idea is to deref such adjustments until we are sure
> + * and so on. The idea is to defer such adjustments until we are sure
“deref” could have been “dereference” but this must indeed mean
“defer”. We are putting off a decision until later.
> * there's no explicit mday specification in the approxidate string.
> */
> static time_t update_tm(struct tm *tm, struct tm *now, time_t sec)
> diff --git a/replay.h b/replay.h
> index 0ab74b9805..90ed299ff0 100644
> --- a/replay.h
> +++ b/replay.h
> @@ -19,7 +19,7 @@ struct replay_revisions_options {
>
> /*
> * Starting point at which to create the new commits; must be a
> - * committish. References pointing at decendants of `onto` will be
> + * committish. References pointing at descendants of `onto` will be
Correct.
> * updated to point to the new commits.
> */
> const char *onto;
> diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
> index 2f9a597ec7..7c6db76c9d 100755
> --- a/t/t9902-completion.sh
> +++ b/t/t9902-completion.sh
> @@ -2446,7 +2446,7 @@ test_expect_success FUNNYNAMES \
> >repeated-quoted/2-file &&
> >repeated-quoted/3\"file && # ... and here, too.
>
> - # Still, we shold only list the directory name only once.
> + # Still, we should list the directory name only once.
Correct, that’s a “shold” typo.
Second time looking it over I see that you also drop the doubled “only”.
> test_path_completion repeated repeated-quoted
> '
>
>
> base-commit: 94f057755b7941b321fd11fec1b2e3ca5313a4e0
> prerequisite-patch-id: f827362e061e199150f149dd36c67664c77406bc
> prerequisite-patch-id: e5b32f0b916ec86eab6631b9bd9bafd639191765
> prerequisite-patch-id: 567a1832a220b2dbf095796cc8093b526d6a076c
> prerequisite-patch-id: aafa4bd4ceb7836a92d28d4c89b57032f74332e9
> prerequisite-patch-id: 2e073762fc9dceafcc6f16711bba425384a24305
> prerequisite-patch-id: 0aa605f0acdb71aa2eb173fdf3c57713c9561fe2
> prerequisite-patch-id: 5163040262c89eed4bcb04228b445d76497c9d58
> prerequisite-patch-id: c06c0461bf75ed638214ce98a54bba6578941c10
> prerequisite-patch-id: 571fdf3570f30fd41f6d681e99acc37df94d09a3
> prerequisite-patch-id: 54e7102e880d24a6b2d22bef9aa90a3078086d4d
> prerequisite-patch-id: d829fff1fcc8b6d086fcb6a40c62f835226ae32f
> prerequisite-patch-id: d1d8e2f2e274565e1d7437aa5ccfe44c3f3d8355
> prerequisite-patch-id: c79ebac6894b9a206f5699e9811e0348e111753d
> prerequisite-patch-id: a7750d7d2ec637d906f975f27ba3d03b33a4a34f
> prerequisite-patch-id: 083f554bc5e09ae54c6b545628196e11a9e90cea
Okay, these must be all the non-merge commits from the topics you
merged in.
> --
> 2.30.2
^ permalink raw reply
* Re: Is it intended behaviour that 'git gc' ignores the 'commitGraph.changedPaths' setting?
From: Theodore Tso @ 2026-06-04 14:27 UTC (permalink / raw)
To: Patrick Steinhardt; +Cc: Tomasz Konojacki, git
In-Reply-To: <aiF0aN9BwBvQffGL@pks.im>
On Thu, Jun 04, 2026 at 02:49:44PM +0200, Patrick Steinhardt wrote:
> So my recommendation would be to stop using git-gc(1) altogether -- I am
> biased as I have helped implementing the new maintenance strategy, but I
> would say that git-gc(1) is nowadays a legacy tool that inches closer
> towards the end of its life. Git's default maintenance nowadays uses
> git-maintenance(1) without using git-gc(1) at all anymore.
I wonder if we should add a tunable that makes "git gc" redirect to
"git maintenance run" or some such. For those of us who like to
launch maintenance tasks at specific times (for example, before I
disconnect from the AC mains and get on a plane), "git gc" has a major
advantage --- it has fewer characters to type, and I'm lazy. :-)
The other things to perhaps suggest is ways that developers can set up
rules like disabling git maintenace running while on battery, etc.
This might require OS-specific mechanisms for determining whether the
laptop is running on battery --- but I note that git-maintenance is
already hooked into systemd and launchctl for Linux and MacOS,
respectively, so there's precedent for that sort of thing.
Cheers,
- Ted
^ permalink raw reply
* [PATCH v2] transport-helper: fix TSAN race in transfer_debug()
From: Pushkar Singh @ 2026-06-04 13:23 UTC (permalink / raw)
To: pushkarkumarsingh1970; +Cc: git, gitster, peff
In-Reply-To: <20260602201309.38434-2-pushkarkumarsingh1970@gmail.com>
Currently, transfer_debug() lazily initializes a static variable based
on GIT_TRANSLOOP_DEBUG. Since the function may be called from multiple
worker threads, this initialization is racy and is therefore suppressed
in .tsan-suppressions.
Initialize the variable in bidirectional_transfer_loop() before any
worker threads or processes are created. This patch removes the race and
allows dropping the corresponding TSAN suppression.
Signed-off-by: Pushkar Singh <pushkarkumarsingh1970@gmail.com>
---
Changes since v1:
- Treat negative values as disabled by using transfer_debug_enabled <= 0
.tsan-suppressions | 1 -
transport-helper.c | 17 ++++++-----------
2 files changed, 6 insertions(+), 12 deletions(-)
diff --git a/.tsan-suppressions b/.tsan-suppressions
index 5ba86d6845..d84883bd90 100644
--- a/.tsan-suppressions
+++ b/.tsan-suppressions
@@ -7,7 +7,6 @@
# A static variable is written to racily, but we always write the same value, so
# in practice it (hopefully!) doesn't matter.
race:^want_color$
-race:^transfer_debug$
# A boolean value, which tells whether the replace_map has been initialized or
# not, is read racily with an update. As this variable is written to only once,
diff --git a/transport-helper.c b/transport-helper.c
index 04d55572a9..9e69c67cde 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -1361,24 +1361,16 @@ int transport_helper_init(struct transport *transport, const char *name)
/* This should be enough to hold debugging message. */
#define PBUFFERSIZE 8192
+static int transfer_debug_enabled = -1;
+
/* Print bidirectional transfer loop debug message. */
__attribute__((format (printf, 1, 2)))
static void transfer_debug(const char *fmt, ...)
{
- /*
- * NEEDSWORK: This function is sometimes used from multiple threads, and
- * we end up using debug_enabled racily. That "should not matter" since
- * we always write the same value, but it's still wrong. This function
- * is listed in .tsan-suppressions for the time being.
- */
-
va_list args;
char msgbuf[PBUFFERSIZE];
- static int debug_enabled = -1;
- if (debug_enabled < 0)
- debug_enabled = getenv("GIT_TRANSLOOP_DEBUG") ? 1 : 0;
- if (!debug_enabled)
+ if (transfer_debug_enabled <= 0)
return;
va_start(args, fmt);
@@ -1648,6 +1640,9 @@ int bidirectional_transfer_loop(int input, int output)
{
struct bidirectional_transfer_state state;
+ if (transfer_debug_enabled < 0)
+ transfer_debug_enabled = getenv("GIT_TRANSLOOP_DEBUG") ? 1 : 0;
+
/* Fill the state fields. */
state.ptg.src = input;
state.ptg.dest = 1;
--
2.53.0.582.gca1db8a0f7
^ permalink raw reply related
* [PATCH] docs: fix typos
From: Tuomas Ahola @ 2026-06-04 13:14 UTC (permalink / raw)
To: git; +Cc: Tuomas Ahola
Fix some typos and grammar errors in comments and documentation files.
Signed-off-by: Tuomas Ahola <taahol@utu.fi>
---
Notes:
Written mostly as an exercise on how to submit patches that depend
on other topics.
$ git log --oneline --first-parent v2.54.0..
d19e9182ab (HEAD -> ta/typofixes) docs: fix typos
5a7e9cc03d Merge branch 'ta/approxidate-noon-fix'
f03649d802 Merge branch 'kh/name-rev-custom-format'
023a226b4b Merge branch 'jc/neuter-sideband-fixup'
As can be seen, these topics have already graduated to master:
$ git cherry master
+ d19e9182ab097a722e32d459a9a58c8985831e3b
Documentation/config/sideband.adoc | 2 +-
Documentation/git-format-rev.adoc | 2 +-
date.c | 2 +-
replay.h | 2 +-
t/t9902-completion.sh | 2 +-
5 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/Documentation/config/sideband.adoc b/Documentation/config/sideband.adoc
index 96fade7f5f..ff007aeb73 100644
--- a/Documentation/config/sideband.adoc
+++ b/Documentation/config/sideband.adoc
@@ -13,7 +13,7 @@ sideband.allowControlCharacters::
Allow control sequences that move the cursor. This is
disabled by default.
`erase`::
- Allow control sequences that erase charactrs. This is
+ Allow control sequences that erase characters. This is
disabled by default.
`false`::
Mask all control characters other than line feeds and
diff --git a/Documentation/git-format-rev.adoc b/Documentation/git-format-rev.adoc
index c40d52e9f6..505a52fecc 100644
--- a/Documentation/git-format-rev.adoc
+++ b/Documentation/git-format-rev.adoc
@@ -33,7 +33,7 @@ OPTIONS
The argument `rev` is also accepted.
`text`;; Formats all commit object names found in freeform text. These
- must the full object names, i.e. abbreviated hexidecimal object
+ must be full object names, i.e. abbreviated hexadecimal object
names will not be interpreted.
+
Anything that is parsed as an object name but that is not found to be a
diff --git a/date.c b/date.c
index 05b78d852f..014065b419 100644
--- a/date.c
+++ b/date.c
@@ -1074,7 +1074,7 @@ void datestamp(struct strbuf *out)
*
* The tm->tm_mday field has an additional logic of using negative values
* for date adjustments: -2 means yesterday and -3 the day before that,
- * and so on. The idea is to deref such adjustments until we are sure
+ * and so on. The idea is to defer such adjustments until we are sure
* there's no explicit mday specification in the approxidate string.
*/
static time_t update_tm(struct tm *tm, struct tm *now, time_t sec)
diff --git a/replay.h b/replay.h
index 0ab74b9805..90ed299ff0 100644
--- a/replay.h
+++ b/replay.h
@@ -19,7 +19,7 @@ struct replay_revisions_options {
/*
* Starting point at which to create the new commits; must be a
- * committish. References pointing at decendants of `onto` will be
+ * committish. References pointing at descendants of `onto` will be
* updated to point to the new commits.
*/
const char *onto;
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 2f9a597ec7..7c6db76c9d 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -2446,7 +2446,7 @@ test_expect_success FUNNYNAMES \
>repeated-quoted/2-file &&
>repeated-quoted/3\"file && # ... and here, too.
- # Still, we shold only list the directory name only once.
+ # Still, we should list the directory name only once.
test_path_completion repeated repeated-quoted
'
base-commit: 94f057755b7941b321fd11fec1b2e3ca5313a4e0
prerequisite-patch-id: f827362e061e199150f149dd36c67664c77406bc
prerequisite-patch-id: e5b32f0b916ec86eab6631b9bd9bafd639191765
prerequisite-patch-id: 567a1832a220b2dbf095796cc8093b526d6a076c
prerequisite-patch-id: aafa4bd4ceb7836a92d28d4c89b57032f74332e9
prerequisite-patch-id: 2e073762fc9dceafcc6f16711bba425384a24305
prerequisite-patch-id: 0aa605f0acdb71aa2eb173fdf3c57713c9561fe2
prerequisite-patch-id: 5163040262c89eed4bcb04228b445d76497c9d58
prerequisite-patch-id: c06c0461bf75ed638214ce98a54bba6578941c10
prerequisite-patch-id: 571fdf3570f30fd41f6d681e99acc37df94d09a3
prerequisite-patch-id: 54e7102e880d24a6b2d22bef9aa90a3078086d4d
prerequisite-patch-id: d829fff1fcc8b6d086fcb6a40c62f835226ae32f
prerequisite-patch-id: d1d8e2f2e274565e1d7437aa5ccfe44c3f3d8355
prerequisite-patch-id: c79ebac6894b9a206f5699e9811e0348e111753d
prerequisite-patch-id: a7750d7d2ec637d906f975f27ba3d03b33a4a34f
prerequisite-patch-id: 083f554bc5e09ae54c6b545628196e11a9e90cea
--
2.30.2
^ permalink raw reply related
* Re: Is it intended behaviour that 'git gc' ignores the 'commitGraph.changedPaths' setting?
From: Patrick Steinhardt @ 2026-06-04 12:49 UTC (permalink / raw)
To: Tomasz Konojacki; +Cc: git
In-Reply-To: <20260604132419.F2FA.5C4F47F8@xenu.pl>
Hi,
On Thu, Jun 04, 2026 at 01:24:20PM +0200, Tomasz Konojacki wrote:
> It seems that 'git gc' (and also 'fetch' with 'fetch.writeCommitGraph'
> enabled) ignore the 'commitGraph.changedPaths' setting.
I think this is a bug -- it really should write the bloom filters when
you've enabled the above config option.
The root cause of this seems to be that we call
`write_commit_graph_reachable()` directly, and that basically means that
it becomes git-gc(1)'s responsibility to set all the correct flags. And
as we don't pay attention to "commitGraph.changedPaths" at all we simply
ignore it.
In Git 2.54, the default housekeeping strategy used by git-maintenance(1)
has changed from using git-gc(1) to use individual tasks for more
flexibility. One of these tasks is responsible for writing commit
graphs, and that task doesn't call `write_commit_graph_reachable()` but
instead executes git-commit-graph(1) directly. And because we do this
infrastructure works correctly.
So my recommendation would be to stop using git-gc(1) altogether -- I am
biased as I have helped implementing the new maintenance strategy, but I
would say that git-gc(1) is nowadays a legacy tool that inches closer
towards the end of its life. Git's default maintenance nowadays uses
git-maintenance(1) without using git-gc(1) at all anymore.
We could of course fix this, and it shouldn't be all that hard to do.
We'd either start using `run_write_commit_graph()` directly in git-gc(1)
or we'd start respecting the config option to pass the additional flag.
I myself probably don't care enough about git-gc(1) to do that anymore,
but if anybody else feels like it I'm happy to review.
Thanks!
Patrick
^ permalink raw reply
* Re: [PATCH] log: improve --follow following renames in merge commits
From: Miklos Vajna @ 2026-06-04 12:20 UTC (permalink / raw)
To: Jeff King; +Cc: Elijah Newren, git
In-Reply-To: <ahqDqSH7yfYVOOyE@collabora.com>
Hi Jeff,
On Sat, May 30, 2026 at 08:29:03AM +0200, Miklos Vajna <vmiklos@collabora.com> wrote:
> Could you please comment on this, if this tweaked rule and its
> implementation in the patch looks OK to you? Let me know if I should
> just wait some more.
Just to come back to this, the idea was to make the --follow behavior
slightly more useful by not always assuming we should follow a first
parent in merge commits, but see if only one parent has effective
changes to the followed file, and if so, follow that one.
I did this by doing a diff on the followed path in each parent, then
mark the parent as "interesting" if DIFF_FILE_VALID() says so. This is
true if the file is touched or the rename happens inside the merge
commit (vs that parent), but it's not true if the file is really not
touched or the file only shows up as an addition. And if we have only
have one interesting parent, then switch to this, even if it's not the
first parent. With this rule, I think we address your worry case about
"making some other cases" worse and this still works for the subtree
case, and this is relatively easy to do.
What do you think?
Thanks,
Miklos
^ permalink raw reply
* [PATCH 16/16] odb/source-packed: drop pointer to "files" parent source
From: Patrick Steinhardt @ 2026-06-04 11:25 UTC (permalink / raw)
To: git
In-Reply-To: <20260604-pks-odb-source-packed-v1-0-2e7ab31b4b5c@pks.im>
Over the last commits we have turned the packfile store into a proper
object database source that can be used as a standalone backend. As
such, it is no longer necessary to have it coupled to the "files" parent
source.
Remove the pointer to the owning "files" source so that the "packed"
source can be used as a standalone entity.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
odb/source-files.c | 2 +-
odb/source-packed.c | 27 +++++++++++++--------------
odb/source-packed.h | 7 ++++---
packfile.c | 2 +-
4 files changed, 19 insertions(+), 19 deletions(-)
diff --git a/odb/source-files.c b/odb/source-files.c
index fa2e18e71b..3bc6419dd7 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -269,7 +269,7 @@ struct odb_source_files *odb_source_files_new(struct object_database *odb,
CALLOC_ARRAY(files, 1);
odb_source_init(&files->base, odb, ODB_SOURCE_FILES, path, local);
files->loose = odb_source_loose_new(odb, path, local);
- files->packed = odb_source_packed_new(files);
+ files->packed = odb_source_packed_new(odb, path, local);
files->base.free = odb_source_files_free;
files->base.close = odb_source_files_close;
diff --git a/odb/source-packed.c b/odb/source-packed.c
index d513b3efc3..42c28fba0e 100644
--- a/odb/source-packed.c
+++ b/odb/source-packed.c
@@ -585,7 +585,7 @@ static void report_pack_garbage(struct string_list *list)
}
struct prepare_pack_data {
- struct odb_source *source;
+ struct odb_source_packed *source;
struct string_list *garbage;
};
@@ -593,15 +593,14 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
const char *file_name, void *_data)
{
struct prepare_pack_data *data = (struct prepare_pack_data *)_data;
- struct odb_source_files *files = odb_source_files_downcast(data->source);
size_t base_len = full_name_len;
if (strip_suffix_mem(full_name, &base_len, ".idx") &&
- !(files->packed->midx &&
- midx_contains_pack(files->packed->midx, file_name))) {
+ !(data->source->midx &&
+ midx_contains_pack(data->source->midx, file_name))) {
char *trimmed_path = xstrndup(full_name, full_name_len);
- packfile_store_load_pack(files->packed,
- trimmed_path, data->source->local);
+ packfile_store_load_pack(data->source,
+ trimmed_path, data->source->base.local);
free(trimmed_path);
}
@@ -626,7 +625,7 @@ static void prepare_pack(const char *full_name, size_t full_name_len,
report_garbage(PACKDIR_FILE_GARBAGE, full_name);
}
-static void prepare_packed_git_one(struct odb_source *source)
+static void prepare_packed_git_one(struct odb_source_packed *source)
{
struct string_list garbage = STRING_LIST_INIT_DUP;
struct prepare_pack_data data = {
@@ -634,7 +633,7 @@ static void prepare_packed_git_one(struct odb_source *source)
.garbage = &garbage,
};
- for_each_file_in_pack_dir(source->path, prepare_pack, &data);
+ for_each_file_in_pack_dir(source->base.path, prepare_pack, &data);
report_pack_garbage(data.garbage);
string_list_clear(data.garbage, 0);
@@ -675,7 +674,7 @@ void odb_source_packed_prepare(struct odb_source_packed *source)
return;
prepare_multi_pack_index_one(source);
- prepare_packed_git_one(&source->files->base);
+ prepare_packed_git_one(source);
sort_packs(&source->packs.head, sort_pack);
for (struct packfile_list_entry *e = source->packs.head; e; e = e->next)
@@ -733,14 +732,14 @@ static void odb_source_packed_free(struct odb_source *source)
free(packed);
}
-struct odb_source_packed *odb_source_packed_new(struct odb_source_files *parent)
+struct odb_source_packed *odb_source_packed_new(struct object_database *odb,
+ const char *path,
+ bool local)
{
struct odb_source_packed *packed;
CALLOC_ARRAY(packed, 1);
- odb_source_init(&packed->base, parent->base.odb, ODB_SOURCE_PACKED,
- parent->base.path, parent->base.local);
- packed->files = parent;
+ odb_source_init(&packed->base, odb, ODB_SOURCE_PACKED, path, local);
strmap_init(&packed->packs_by_path);
packed->base.free = odb_source_packed_free;
@@ -758,7 +757,7 @@ struct odb_source_packed *odb_source_packed_new(struct odb_source_files *parent)
packed->base.read_alternates = odb_source_packed_read_alternates;
packed->base.write_alternate = odb_source_packed_write_alternate;
- if (!is_absolute_path(parent->base.path))
+ if (!is_absolute_path(path))
chdir_notify_register(NULL, odb_source_packed_reparent, packed);
return packed;
diff --git a/odb/source-packed.h b/odb/source-packed.h
index 6645f4f943..ef5a10b224 100644
--- a/odb/source-packed.h
+++ b/odb/source-packed.h
@@ -18,7 +18,6 @@ struct packfile_list_entry {
*/
struct odb_source_packed {
struct odb_source base;
- struct odb_source_files *files;
/*
* The list of packfiles in the order in which they have been most
@@ -74,9 +73,11 @@ struct odb_source_packed {
/*
* Allocate and initialize a new empty packfile store for the given object
- * database source.
+ * database.
*/
-struct odb_source_packed *odb_source_packed_new(struct odb_source_files *parent);
+struct odb_source_packed *odb_source_packed_new(struct object_database *odb,
+ const char *path,
+ bool local);
/*
* Cast the given object database source to the packed backend. This will cause
diff --git a/packfile.c b/packfile.c
index d7de0412ff..478f00ce02 100644
--- a/packfile.c
+++ b/packfile.c
@@ -884,7 +884,7 @@ struct packed_git *packfile_store_load_pack(struct odb_source_packed *store,
p = strmap_get(&store->packs_by_path, key.buf);
if (!p) {
- p = add_packed_git(store->files->base.odb->repo, idx_path,
+ p = add_packed_git(store->base.odb->repo, idx_path,
strlen(idx_path), local);
if (p)
packfile_store_add_pack(store, p);
--
2.54.0.1064.gd145956f57.dirty
^ permalink raw reply related
* [PATCH 15/16] midx: refactor interfaces to work on "packed" source
From: Patrick Steinhardt @ 2026-06-04 11:25 UTC (permalink / raw)
To: git
In-Reply-To: <20260604-pks-odb-source-packed-v1-0-2e7ab31b4b5c@pks.im>
Our interfaces used to interact with MIDXs all work on top of the
generic `struct odb_source`. This doesn't make much sense though: a MIDX
is strictly tied to the "packed" source, so passing in a generic source
gives the false sense that it may also work with a different type of
source.
Fix this conceptual weirdness and instead require the caller to pass in
a "packed" source explicitly. This also makes the next commit easier to
implement, where we drop the pointer to the "files" source in the
"packed" source.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/multi-pack-index.c | 29 +++++------
builtin/pack-objects.c | 3 +-
builtin/repack.c | 8 ++-
midx-write.c | 34 ++++++-------
midx.c | 118 ++++++++++++++++++++++-----------------------
midx.h | 30 ++++++------
odb/source-packed.c | 12 ++---
pack-bitmap.c | 8 +--
pack-revindex.c | 6 +--
repack-geometry.c | 3 +-
repack-midx.c | 9 ++--
repack.c | 6 +--
t/helper/test-read-midx.c | 7 ++-
13 files changed, 144 insertions(+), 129 deletions(-)
diff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c
index 00ffb36394..6e73c85cde 100644
--- a/builtin/multi-pack-index.c
+++ b/builtin/multi-pack-index.c
@@ -10,6 +10,7 @@
#include "trace2.h"
#include "odb.h"
#include "odb/source.h"
+#include "odb/source-files.h"
#include "replace-object.h"
#include "repository.h"
@@ -85,12 +86,12 @@ static int parse_object_dir(const struct option *opt, const char *arg,
return 0;
}
-static struct odb_source *handle_object_dir_option(struct repository *repo)
+static struct odb_source_files *handle_object_dir_option(struct repository *repo)
{
struct odb_source *source = odb_find_source(repo->objects, opts.object_dir);
if (!source)
source = odb_add_to_alternates_memory(repo->objects, opts.object_dir);
- return source;
+ return odb_source_files_downcast(source);
}
static struct option common_opts[] = {
@@ -167,7 +168,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,
N_("refs snapshot for selecting bitmap commits")),
OPT_END(),
};
- struct odb_source *source;
+ struct odb_source_files *source;
int ret;
opts.flags |= MIDX_WRITE_BITMAP_HASH_CACHE;
@@ -211,7 +212,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,
read_packs_from_stdin(&packs);
- ret = write_midx_file_only(source, &packs,
+ ret = write_midx_file_only(source->packed, &packs,
opts.preferred_pack,
opts.refs_snapshot,
opts.incremental_base, opts.flags);
@@ -223,7 +224,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,
}
- ret = write_midx_file(source, opts.preferred_pack,
+ ret = write_midx_file(source->packed, opts.preferred_pack,
opts.refs_snapshot, opts.flags);
free(opts.refs_snapshot);
@@ -237,7 +238,7 @@ static int cmd_multi_pack_index_compact(int argc, const char **argv,
struct multi_pack_index *m, *cur;
struct multi_pack_index *from_midx = NULL;
struct multi_pack_index *to_midx = NULL;
- struct odb_source *source;
+ struct odb_source_files *source;
int ret;
struct option *options;
@@ -282,7 +283,7 @@ static int cmd_multi_pack_index_compact(int argc, const char **argv,
FREE_AND_NULL(options);
- m = get_multi_pack_index(source);
+ m = get_multi_pack_index(source->packed);
for (cur = m; cur && !(from_midx && to_midx); cur = cur->base_midx) {
const char *midx_csum = midx_get_checksum_hex(cur);
@@ -305,7 +306,7 @@ static int cmd_multi_pack_index_compact(int argc, const char **argv,
die(_("MIDX %s must be an ancestor of %s"), argv[0], argv[1]);
}
- ret = write_midx_file_compact(source, from_midx, to_midx,
+ ret = write_midx_file_compact(source->packed, from_midx, to_midx,
opts.incremental_base, opts.flags);
return ret;
@@ -319,7 +320,7 @@ static int cmd_multi_pack_index_verify(int argc, const char **argv,
static struct option builtin_multi_pack_index_verify_options[] = {
OPT_END(),
};
- struct odb_source *source;
+ struct odb_source_files *source;
options = add_common_options(builtin_multi_pack_index_verify_options);
@@ -337,7 +338,7 @@ static int cmd_multi_pack_index_verify(int argc, const char **argv,
FREE_AND_NULL(options);
- return verify_midx_file(source, opts.flags);
+ return verify_midx_file(source->packed, opts.flags);
}
static int cmd_multi_pack_index_expire(int argc, const char **argv,
@@ -348,7 +349,7 @@ static int cmd_multi_pack_index_expire(int argc, const char **argv,
static struct option builtin_multi_pack_index_expire_options[] = {
OPT_END(),
};
- struct odb_source *source;
+ struct odb_source_files *source;
options = add_common_options(builtin_multi_pack_index_expire_options);
@@ -366,7 +367,7 @@ static int cmd_multi_pack_index_expire(int argc, const char **argv,
FREE_AND_NULL(options);
- return expire_midx_packs(source, opts.flags);
+ return expire_midx_packs(source->packed, opts.flags);
}
static int cmd_multi_pack_index_repack(int argc, const char **argv,
@@ -379,7 +380,7 @@ static int cmd_multi_pack_index_repack(int argc, const char **argv,
N_("during repack, collect pack-files of smaller size into a batch that is larger than this size")),
OPT_END(),
};
- struct odb_source *source;
+ struct odb_source_files *source;
options = add_common_options(builtin_multi_pack_index_repack_options);
@@ -398,7 +399,7 @@ static int cmd_multi_pack_index_repack(int argc, const char **argv,
FREE_AND_NULL(options);
- return midx_repack(source, (size_t)opts.batch_size, opts.flags);
+ return midx_repack(source->packed, (size_t)opts.batch_size, opts.flags);
}
int cmd_multi_pack_index(int argc,
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 5e94805478..424c92cc29 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -1775,7 +1775,8 @@ static int want_object_in_pack_mtime(const struct object_id *oid,
odb_prepare_alternates(the_repository->objects);
for (source = the_repository->objects->sources; source; source = source->next) {
- struct multi_pack_index *m = get_multi_pack_index(source);
+ struct odb_source_files *files = odb_source_files_downcast(source);
+ struct multi_pack_index *m = get_multi_pack_index(files->packed);
struct pack_entry e;
if (m && fill_midx_entry(m, oid, &e)) {
diff --git a/builtin/repack.c b/builtin/repack.c
index 1524a9c13a..47966a686b 100644
--- a/builtin/repack.c
+++ b/builtin/repack.c
@@ -458,6 +458,8 @@ int cmd_repack(int argc,
}
if (!names.nr) {
+ struct odb_source_files *files = odb_source_files_downcast(existing.source);
+
if (!po_args.quiet)
printf_ln(_("Nothing new to pack."));
/*
@@ -473,7 +475,7 @@ int cmd_repack(int argc,
* midx_has_unknown_packs() will make the decision for
* us.
*/
- if (!get_multi_pack_index(existing.source))
+ if (!get_multi_pack_index(files->packed))
midx_must_contain_cruft = 1;
}
@@ -626,10 +628,12 @@ int cmd_repack(int argc,
update_server_info(repo, 0);
if (git_env_bool(GIT_TEST_MULTI_PACK_INDEX, 0)) {
+ struct odb_source_files *files = odb_source_files_downcast(existing.source);
unsigned flags = 0;
+
if (git_env_bool(GIT_TEST_MULTI_PACK_INDEX_WRITE_INCREMENTAL, 0))
flags |= MIDX_WRITE_INCREMENTAL;
- write_midx_file(existing.source, NULL, NULL, flags);
+ write_midx_file(files->packed, NULL, NULL, flags);
}
cleanup:
diff --git a/midx-write.c b/midx-write.c
index 561e9eedc0..7cafc49fb8 100644
--- a/midx-write.c
+++ b/midx-write.c
@@ -25,9 +25,9 @@
#define NO_PREFERRED_PACK (~((uint32_t)0))
extern int midx_checksum_valid(struct multi_pack_index *m);
-extern void clear_midx_files_ext(struct odb_source *source, const char *ext,
+extern void clear_midx_files_ext(struct odb_source_packed *source, const char *ext,
const char *keep_hash);
-extern void clear_incremental_midx_files_ext(struct odb_source *source,
+extern void clear_incremental_midx_files_ext(struct odb_source_packed *source,
const char *ext,
const struct strvec *keep_hashes);
extern int cmp_idx_or_pack_name(const char *idx_or_pack_name,
@@ -119,7 +119,7 @@ struct write_midx_context {
struct string_list *to_include;
struct repository *repo;
- struct odb_source *source;
+ struct odb_source_packed *source;
};
static uint32_t midx_pack_perm(struct write_midx_context *ctx,
@@ -1107,7 +1107,7 @@ static int link_midx_to_chain(struct multi_pack_index *m)
return ret;
}
-static void clear_midx_files(struct odb_source *source,
+static void clear_midx_files(struct odb_source_packed *source,
const struct strvec *hashes, unsigned incremental)
{
/*
@@ -1237,7 +1237,7 @@ static int midx_hashcmp(const struct multi_pack_index *a,
}
struct write_midx_opts {
- struct odb_source *source; /* non-optional */
+ struct odb_source_packed *source; /* non-optional */
struct string_list *packs_to_include;
struct string_list *packs_to_drop;
@@ -1253,7 +1253,7 @@ struct write_midx_opts {
static int write_midx_internal(struct write_midx_opts *opts)
{
- struct repository *r = opts->source->odb->repo;
+ struct repository *r = opts->source->base.odb->repo;
struct strbuf midx_name = STRBUF_INIT;
unsigned char midx_hash[GIT_MAX_RAWSZ];
uint32_t start_pack;
@@ -1301,7 +1301,7 @@ static int write_midx_internal(struct write_midx_opts *opts)
if (ctx.incremental)
strbuf_addf(&midx_name,
"%s/pack/multi-pack-index.d/tmp_midx_XXXXXX",
- opts->source->path);
+ opts->source->base.path);
else
get_midx_filename(opts->source, &midx_name);
if (safe_create_leading_directories(r, midx_name.buf))
@@ -1396,7 +1396,7 @@ static int write_midx_internal(struct write_midx_opts *opts)
fill_packs_from_midx_range(&ctx, bitmap_order);
} else {
ctx.to_include = opts->packs_to_include;
- for_each_file_in_pack_dir(opts->source->path, add_pack_to_midx, &ctx);
+ for_each_file_in_pack_dir(opts->source->base.path, add_pack_to_midx, &ctx);
}
stop_progress(&ctx.progress);
@@ -1847,7 +1847,7 @@ static int write_midx_internal(struct write_midx_opts *opts)
return result;
}
-int write_midx_file(struct odb_source *source,
+int write_midx_file(struct odb_source_packed *source,
const char *preferred_pack_name,
const char *refs_snapshot,
unsigned flags)
@@ -1862,7 +1862,7 @@ int write_midx_file(struct odb_source *source,
return write_midx_internal(&opts);
}
-int write_midx_file_only(struct odb_source *source,
+int write_midx_file_only(struct odb_source_packed *source,
struct string_list *packs_to_include,
const char *preferred_pack_name,
const char *refs_snapshot,
@@ -1881,7 +1881,7 @@ int write_midx_file_only(struct odb_source *source,
return write_midx_internal(&opts);
}
-int write_midx_file_compact(struct odb_source *source,
+int write_midx_file_compact(struct odb_source_packed *source,
struct multi_pack_index *from,
struct multi_pack_index *to,
const char *incremental_base,
@@ -1898,7 +1898,7 @@ int write_midx_file_compact(struct odb_source *source,
return write_midx_internal(&opts);
}
-int expire_midx_packs(struct odb_source *source, unsigned flags)
+int expire_midx_packs(struct odb_source_packed *source, unsigned flags)
{
uint32_t i, *count, result = 0;
struct string_list packs_to_drop = STRING_LIST_INIT_DUP;
@@ -1915,7 +1915,7 @@ int expire_midx_packs(struct odb_source *source, unsigned flags)
if (flags & MIDX_PROGRESS)
progress = start_delayed_progress(
- source->odb->repo,
+ source->base.odb->repo,
_("Counting referenced objects"),
m->num_objects);
for (i = 0; i < m->num_objects; i++) {
@@ -1927,7 +1927,7 @@ int expire_midx_packs(struct odb_source *source, unsigned flags)
if (flags & MIDX_PROGRESS)
progress = start_delayed_progress(
- source->odb->repo,
+ source->base.odb->repo,
_("Finding and deleting unreferenced packfiles"),
m->num_packs);
for (i = 0; i < m->num_packs; i++) {
@@ -2085,9 +2085,9 @@ static void fill_included_packs_batch(struct repository *r,
free(pack_info);
}
-int midx_repack(struct odb_source *source, size_t batch_size, unsigned flags)
+int midx_repack(struct odb_source_packed *source, size_t batch_size, unsigned flags)
{
- struct repository *r = source->odb->repo;
+ struct repository *r = source->base.odb->repo;
int result = 0;
uint32_t i, packs_to_repack = 0;
unsigned char *include_pack;
@@ -2131,7 +2131,7 @@ int midx_repack(struct odb_source *source, size_t batch_size, unsigned flags)
strvec_push(&cmd.args, "pack-objects");
- strvec_pushf(&cmd.args, "%s/pack/pack", source->path);
+ strvec_pushf(&cmd.args, "%s/pack/pack", source->base.path);
if (delta_base_offset)
strvec_push(&cmd.args, "--delta-base-offset");
diff --git a/midx.c b/midx.c
index 00bbd137b2..cc6b94f9dd 100644
--- a/midx.c
+++ b/midx.c
@@ -17,9 +17,9 @@
#define MIDX_PACK_ERROR ((void *)(intptr_t)-1)
int midx_checksum_valid(struct multi_pack_index *m);
-void clear_midx_files_ext(struct odb_source *source, const char *ext,
+void clear_midx_files_ext(struct odb_source_packed *source, const char *ext,
const char *keep_hash);
-void clear_incremental_midx_files_ext(struct odb_source *source, const char *ext,
+void clear_incremental_midx_files_ext(struct odb_source_packed *source, const char *ext,
const struct strvec *keep_hashes);
int cmp_idx_or_pack_name(const char *idx_or_pack_name,
const char *idx_name);
@@ -27,25 +27,25 @@ int cmp_idx_or_pack_name(const char *idx_or_pack_name,
const char *midx_get_checksum_hex(const struct multi_pack_index *m)
{
return hash_to_hex_algop(midx_get_checksum_hash(m),
- m->source->odb->repo->hash_algo);
+ m->source->base.odb->repo->hash_algo);
}
const unsigned char *midx_get_checksum_hash(const struct multi_pack_index *m)
{
- return m->data + m->data_len - m->source->odb->repo->hash_algo->rawsz;
+ return m->data + m->data_len - m->source->base.odb->repo->hash_algo->rawsz;
}
-void get_midx_filename(struct odb_source *source, struct strbuf *out)
+void get_midx_filename(struct odb_source_packed *source, struct strbuf *out)
{
get_midx_filename_ext(source, out, NULL, NULL);
}
-void get_midx_filename_ext(struct odb_source *source, struct strbuf *out,
+void get_midx_filename_ext(struct odb_source_packed *source, struct strbuf *out,
const unsigned char *hash, const char *ext)
{
- strbuf_addf(out, "%s/pack/multi-pack-index", source->path);
+ strbuf_addf(out, "%s/pack/multi-pack-index", source->base.path);
if (ext)
- strbuf_addf(out, "-%s.%s", hash_to_hex_algop(hash, source->odb->repo->hash_algo), ext);
+ strbuf_addf(out, "-%s.%s", hash_to_hex_algop(hash, source->base.odb->repo->hash_algo), ext);
}
static int midx_read_oid_fanout(const unsigned char *chunk_start,
@@ -99,17 +99,16 @@ static int midx_read_object_offsets(const unsigned char *chunk_start,
return 0;
}
-struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
+struct multi_pack_index *get_multi_pack_index(struct odb_source_packed *source)
{
- struct odb_source_files *files = odb_source_files_downcast(source);
- odb_source_packed_prepare(files->packed);
- return files->packed->midx;
+ odb_source_packed_prepare(source);
+ return source->midx;
}
-static struct multi_pack_index *load_multi_pack_index_one(struct odb_source *source,
+static struct multi_pack_index *load_multi_pack_index_one(struct odb_source_packed *source,
const char *midx_name)
{
- struct repository *r = source->odb->repo;
+ struct repository *r = source->base.odb->repo;
struct multi_pack_index *m = NULL;
int fd;
struct stat st;
@@ -234,23 +233,23 @@ static struct multi_pack_index *load_multi_pack_index_one(struct odb_source *sou
return NULL;
}
-void get_midx_chain_dirname(struct odb_source *source, struct strbuf *buf)
+void get_midx_chain_dirname(struct odb_source_packed *source, struct strbuf *buf)
{
- strbuf_addf(buf, "%s/pack/multi-pack-index.d", source->path);
+ strbuf_addf(buf, "%s/pack/multi-pack-index.d", source->base.path);
}
-void get_midx_chain_filename(struct odb_source *source, struct strbuf *buf)
+void get_midx_chain_filename(struct odb_source_packed *source, struct strbuf *buf)
{
get_midx_chain_dirname(source, buf);
strbuf_addstr(buf, "/multi-pack-index-chain");
}
-void get_split_midx_filename_ext(struct odb_source *source, struct strbuf *buf,
+void get_split_midx_filename_ext(struct odb_source_packed *source, struct strbuf *buf,
const unsigned char *hash, const char *ext)
{
get_midx_chain_dirname(source, buf);
strbuf_addf(buf, "/multi-pack-index-%s.%s",
- hash_to_hex_algop(hash, source->odb->repo->hash_algo), ext);
+ hash_to_hex_algop(hash, source->base.odb->repo->hash_algo), ext);
}
static int open_multi_pack_index_chain(const struct git_hash_algo *hash_algo,
@@ -306,11 +305,11 @@ static int add_midx_to_chain(struct multi_pack_index *midx,
return 1;
}
-static struct multi_pack_index *load_midx_chain_fd_st(struct odb_source *source,
+static struct multi_pack_index *load_midx_chain_fd_st(struct odb_source_packed *source,
int fd, struct stat *st,
int *incomplete_chain)
{
- const struct git_hash_algo *hash_algo = source->odb->repo->hash_algo;
+ const struct git_hash_algo *hash_algo = source->base.odb->repo->hash_algo;
struct multi_pack_index *midx_chain = NULL;
struct strbuf buf = STRBUF_INIT;
int valid = 1;
@@ -362,7 +361,7 @@ static struct multi_pack_index *load_midx_chain_fd_st(struct odb_source *source,
return midx_chain;
}
-static struct multi_pack_index *load_multi_pack_index_chain(struct odb_source *source)
+static struct multi_pack_index *load_multi_pack_index_chain(struct odb_source_packed *source)
{
struct strbuf chain_file = STRBUF_INIT;
struct stat st;
@@ -370,7 +369,8 @@ static struct multi_pack_index *load_multi_pack_index_chain(struct odb_source *s
struct multi_pack_index *m = NULL;
get_midx_chain_filename(source, &chain_file);
- if (open_multi_pack_index_chain(source->odb->repo->hash_algo, chain_file.buf, &fd, &st)) {
+ if (open_multi_pack_index_chain(source->base.odb->repo->hash_algo,
+ chain_file.buf, &fd, &st)) {
int incomplete;
/* ownership of fd is taken over by load function */
m = load_midx_chain_fd_st(source, fd, &st, &incomplete);
@@ -380,7 +380,7 @@ static struct multi_pack_index *load_multi_pack_index_chain(struct odb_source *s
return m;
}
-struct multi_pack_index *load_multi_pack_index(struct odb_source *source)
+struct multi_pack_index *load_multi_pack_index(struct odb_source_packed *source)
{
struct strbuf midx_name = STRBUF_INIT;
struct multi_pack_index *m;
@@ -456,7 +456,7 @@ static uint32_t midx_for_pack(struct multi_pack_index **_m,
int prepare_midx_pack(struct multi_pack_index *m,
uint32_t pack_int_id)
{
- struct odb_source_files *files = odb_source_files_downcast(m->source);
+ struct odb_source_packed *packed = m->source;
struct strbuf pack_name = STRBUF_INIT;
struct packed_git *p;
@@ -467,10 +467,10 @@ int prepare_midx_pack(struct multi_pack_index *m,
if (m->packs[pack_int_id])
return 0;
- strbuf_addf(&pack_name, "%s/pack/%s", files->base.path,
+ strbuf_addf(&pack_name, "%s/pack/%s", packed->base.path,
m->pack_names[pack_int_id]);
- p = packfile_store_load_pack(files->packed,
- pack_name.buf, files->base.local);
+ p = packfile_store_load_pack(packed,
+ pack_name.buf, packed->base.local);
strbuf_release(&pack_name);
if (!p) {
@@ -523,7 +523,7 @@ int bsearch_one_midx(const struct object_id *oid, struct multi_pack_index *m,
{
int ret = bsearch_hash(oid->hash, m->chunk_oid_fanout,
m->chunk_oid_lookup,
- m->source->odb->repo->hash_algo->rawsz,
+ m->source->base.odb->repo->hash_algo->rawsz,
result);
if (result)
*result += m->num_objects_in_base;
@@ -554,7 +554,7 @@ struct object_id *nth_midxed_object_oid(struct object_id *oid,
n = midx_for_object(&m, n);
oidread(oid, m->chunk_oid_lookup + st_mult(m->hash_len, n),
- m->source->odb->repo->hash_algo);
+ m->source->base.odb->repo->hash_algo);
return oid;
}
@@ -734,26 +734,25 @@ int midx_preferred_pack(struct multi_pack_index *m, uint32_t *pack_int_id)
return 0;
}
-int prepare_multi_pack_index_one(struct odb_source *source)
+int prepare_multi_pack_index_one(struct odb_source_packed *source)
{
- struct odb_source_files *files = odb_source_files_downcast(source);
- struct repository *r = source->odb->repo;
+ struct repository *r = source->base.odb->repo;
prepare_repo_settings(r);
if (!r->settings.core_multi_pack_index)
return 0;
- if (files->packed->midx)
+ if (source->midx)
return 1;
- files->packed->midx = load_multi_pack_index(source);
+ source->midx = load_multi_pack_index(source);
- return !!files->packed->midx;
+ return !!source->midx;
}
int midx_checksum_valid(struct multi_pack_index *m)
{
- return hashfile_checksum_valid(m->source->odb->repo->hash_algo,
+ return hashfile_checksum_valid(m->source->base.odb->repo->hash_algo,
m->data, m->data_len);
}
@@ -776,7 +775,7 @@ static void clear_midx_file_ext(const char *full_path, size_t full_path_len UNUS
die_errno(_("failed to remove %s"), full_path);
}
-void clear_midx_files_ext(struct odb_source *source, const char *ext,
+void clear_midx_files_ext(struct odb_source_packed *source, const char *ext,
const char *keep_hash)
{
struct clear_midx_data data = {
@@ -793,12 +792,12 @@ void clear_midx_files_ext(struct odb_source *source, const char *ext,
strbuf_release(&buf);
}
- for_each_file_in_pack_dir(source->path, clear_midx_file_ext, &data);
+ for_each_file_in_pack_dir(source->base.path, clear_midx_file_ext, &data);
strset_clear(&data.keep);
}
-void clear_incremental_midx_files_ext(struct odb_source *source, const char *ext,
+void clear_incremental_midx_files_ext(struct odb_source_packed *source, const char *ext,
const struct strvec *keep_hashes)
{
struct clear_midx_data data = {
@@ -817,7 +816,7 @@ void clear_incremental_midx_files_ext(struct odb_source *source, const char *ext
}
}
- for_each_file_in_pack_subdir(source->path, "multi-pack-index.d",
+ for_each_file_in_pack_subdir(source->base.path, "multi-pack-index.d",
clear_midx_file_ext, &data);
strbuf_release(&buf);
@@ -826,26 +825,28 @@ void clear_incremental_midx_files_ext(struct odb_source *source, const char *ext
void clear_midx_file(struct repository *r)
{
+ struct odb_source_files *files;
struct strbuf midx = STRBUF_INIT;
- get_midx_filename(r->objects->sources, &midx);
-
if (r->objects) {
struct odb_source *source;
for (source = r->objects->sources; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
+ files = odb_source_files_downcast(source);
if (files->packed->midx)
close_midx(files->packed->midx);
files->packed->midx = NULL;
}
}
+ files = odb_source_files_downcast(r->objects->sources);
+ get_midx_filename(files->packed, &midx);
+
if (remove_path(midx.buf))
die(_("failed to clear multi-pack-index at %s"), midx.buf);
- clear_midx_files_ext(r->objects->sources, MIDX_EXT_BITMAP, NULL);
- clear_midx_files_ext(r->objects->sources, MIDX_EXT_REV, NULL);
+ clear_midx_files_ext(files->packed, MIDX_EXT_BITMAP, NULL);
+ clear_midx_files_ext(files->packed, MIDX_EXT_REV, NULL);
strbuf_release(&midx);
}
@@ -853,28 +854,27 @@ void clear_midx_file(struct repository *r)
void clear_incremental_midx_files(struct repository *r,
const struct strvec *keep_hashes)
{
- struct odb_source *source = r->objects->sources;
+ struct odb_source_files *files;
+ struct odb_source *source;
struct strbuf chain = STRBUF_INIT;
- get_midx_chain_filename(source, &chain);
-
- for (; source; source = source->next) {
- struct odb_source_files *files = odb_source_files_downcast(source);
+ for (source = r->objects->sources; source; source = source->next) {
+ files = odb_source_files_downcast(source);
if (files->packed->midx)
close_midx(files->packed->midx);
files->packed->midx = NULL;
}
+ files = odb_source_files_downcast(r->objects->sources);
+ get_midx_chain_filename(files->packed, &chain);
+
if (!keep_hashes && remove_path(chain.buf))
die(_("failed to clear multi-pack-index chain at %s"),
chain.buf);
- clear_incremental_midx_files_ext(r->objects->sources, MIDX_EXT_BITMAP,
- keep_hashes);
- clear_incremental_midx_files_ext(r->objects->sources, MIDX_EXT_REV,
- keep_hashes);
- clear_incremental_midx_files_ext(r->objects->sources, MIDX_EXT_MIDX,
- keep_hashes);
+ clear_incremental_midx_files_ext(files->packed, MIDX_EXT_BITMAP, keep_hashes);
+ clear_incremental_midx_files_ext(files->packed, MIDX_EXT_REV, keep_hashes);
+ clear_incremental_midx_files_ext(files->packed, MIDX_EXT_MIDX, keep_hashes);
strbuf_release(&chain);
}
@@ -918,9 +918,9 @@ static int compare_pair_pos_vs_id(const void *_a, const void *_b)
display_progress(progress, _n); \
} while (0)
-int verify_midx_file(struct odb_source *source, unsigned flags)
+int verify_midx_file(struct odb_source_packed *source, unsigned flags)
{
- struct repository *r = source->odb->repo;
+ struct repository *r = source->base.odb->repo;
struct pair_pos_vs_id *pairs = NULL;
uint32_t i;
struct progress *progress = NULL;
diff --git a/midx.h b/midx.h
index 63853a03a4..939c18e588 100644
--- a/midx.h
+++ b/midx.h
@@ -37,7 +37,7 @@ struct strvec;
"GIT_TEST_MULTI_PACK_INDEX_WRITE_INCREMENTAL"
struct multi_pack_index {
- struct odb_source *source;
+ struct odb_source_packed *source;
const unsigned char *data;
size_t data_len;
@@ -92,16 +92,16 @@ struct multi_pack_index {
const char *midx_get_checksum_hex(const struct multi_pack_index *m) /* static buffer */;
const unsigned char *midx_get_checksum_hash(const struct multi_pack_index *m);
-void get_midx_filename(struct odb_source *source, struct strbuf *out);
-void get_midx_filename_ext(struct odb_source *source, struct strbuf *out,
+void get_midx_filename(struct odb_source_packed *source, struct strbuf *out);
+void get_midx_filename_ext(struct odb_source_packed *source, struct strbuf *out,
const unsigned char *hash, const char *ext);
-void get_midx_chain_dirname(struct odb_source *source, struct strbuf *out);
-void get_midx_chain_filename(struct odb_source *source, struct strbuf *out);
-void get_split_midx_filename_ext(struct odb_source *source, struct strbuf *buf,
+void get_midx_chain_dirname(struct odb_source_packed *source, struct strbuf *out);
+void get_midx_chain_filename(struct odb_source_packed *source, struct strbuf *out);
+void get_split_midx_filename_ext(struct odb_source_packed *source, struct strbuf *buf,
const unsigned char *hash, const char *ext);
-struct multi_pack_index *get_multi_pack_index(struct odb_source *source);
-struct multi_pack_index *load_multi_pack_index(struct odb_source *source);
+struct multi_pack_index *get_multi_pack_index(struct odb_source_packed *source);
+struct multi_pack_index *load_multi_pack_index(struct odb_source_packed *source);
int prepare_midx_pack(struct multi_pack_index *m, uint32_t pack_int_id);
struct packed_git *nth_midxed_pack(struct multi_pack_index *m,
uint32_t pack_int_id);
@@ -123,22 +123,22 @@ int midx_contains_pack(struct multi_pack_index *m,
int midx_layer_contains_pack(struct multi_pack_index *m,
const char *idx_or_pack_name);
int midx_preferred_pack(struct multi_pack_index *m, uint32_t *pack_int_id);
-int prepare_multi_pack_index_one(struct odb_source *source);
+int prepare_multi_pack_index_one(struct odb_source_packed *source);
/*
* Variant of write_midx_file which writes a MIDX containing only the packs
* specified in packs_to_include.
*/
-int write_midx_file(struct odb_source *source,
+int write_midx_file(struct odb_source_packed *source,
const char *preferred_pack_name, const char *refs_snapshot,
unsigned flags);
-int write_midx_file_only(struct odb_source *source,
+int write_midx_file_only(struct odb_source_packed *source,
struct string_list *packs_to_include,
const char *preferred_pack_name,
const char *refs_snapshot,
const char *incremental_base,
unsigned flags);
-int write_midx_file_compact(struct odb_source *source,
+int write_midx_file_compact(struct odb_source_packed *source,
struct multi_pack_index *from,
struct multi_pack_index *to,
const char *incremental_base,
@@ -146,9 +146,9 @@ int write_midx_file_compact(struct odb_source *source,
void clear_midx_file(struct repository *r);
void clear_incremental_midx_files(struct repository *r,
const struct strvec *keep_hashes);
-int verify_midx_file(struct odb_source *source, unsigned flags);
-int expire_midx_packs(struct odb_source *source, unsigned flags);
-int midx_repack(struct odb_source *source, size_t batch_size, unsigned flags);
+int verify_midx_file(struct odb_source_packed *source, unsigned flags);
+int expire_midx_packs(struct odb_source_packed *source, unsigned flags);
+int midx_repack(struct odb_source_packed *source, size_t batch_size, unsigned flags);
void close_midx(struct multi_pack_index *m);
diff --git a/odb/source-packed.c b/odb/source-packed.c
index 08a2de9bc5..d513b3efc3 100644
--- a/odb/source-packed.c
+++ b/odb/source-packed.c
@@ -136,8 +136,8 @@ static int for_each_prefixed_object_in_midx(
for (; m; m = m->base_midx) {
uint32_t num, i, first = 0;
- int len = opts->prefix_hex_len > m->source->odb->repo->hash_algo->hexsz ?
- m->source->odb->repo->hash_algo->hexsz : opts->prefix_hex_len;
+ int len = opts->prefix_hex_len > m->source->base.odb->repo->hash_algo->hexsz ?
+ m->source->base.odb->repo->hash_algo->hexsz : opts->prefix_hex_len;
if (!m->num_objects)
continue;
@@ -249,7 +249,7 @@ static int odb_source_packed_for_each_prefixed_object(
store->skip_mru_updates = true;
- m = get_multi_pack_index(&store->files->base);
+ m = get_multi_pack_index(store);
if (m) {
ret = for_each_prefixed_object_in_midx(store, m, opts, data);
if (ret)
@@ -348,7 +348,7 @@ static int odb_source_packed_count_objects(struct odb_source *source,
unsigned long count = 0;
int ret;
- m = get_multi_pack_index(&packed->files->base);
+ m = get_multi_pack_index(packed);
if (m)
count += m->num_objects + m->num_objects_in_base;
@@ -465,7 +465,7 @@ static int odb_source_packed_find_abbrev_len(struct odb_source *source,
struct packfile_list_entry *e;
struct multi_pack_index *m;
- m = get_multi_pack_index(&packed->files->base);
+ m = get_multi_pack_index(packed);
if (m)
find_abbrev_len_for_midx(m, oid, min_len, &min_len);
@@ -674,7 +674,7 @@ void odb_source_packed_prepare(struct odb_source_packed *source)
if (source->initialized)
return;
- prepare_multi_pack_index_one(&source->files->base);
+ prepare_multi_pack_index_one(source);
prepare_packed_git_one(&source->files->base);
sort_packs(&source->packs.head, sort_pack);
diff --git a/pack-bitmap.c b/pack-bitmap.c
index f9af8a96bd..6bfcbc8ce6 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -238,7 +238,7 @@ static uint32_t bitmap_name_hash(struct bitmap_index *index, uint32_t pos)
static struct repository *bitmap_repo(struct bitmap_index *bitmap_git)
{
if (bitmap_is_midx(bitmap_git))
- return bitmap_git->midx->source->odb->repo;
+ return bitmap_git->midx->source->base.odb->repo;
return bitmap_git->pack->repo;
}
@@ -711,7 +711,8 @@ static int open_midx_bitmap(struct repository *r,
odb_prepare_alternates(r->objects);
for (source = r->objects->sources; source; source = source->next) {
- struct multi_pack_index *midx = get_multi_pack_index(source);
+ struct odb_source_files *files = odb_source_files_downcast(source);
+ struct multi_pack_index *midx = get_multi_pack_index(files->packed);
if (midx && !open_midx_bitmap_1(bitmap_git, midx))
ret = 0;
}
@@ -3399,7 +3400,8 @@ int verify_bitmap_files(struct repository *r)
odb_prepare_alternates(r->objects);
for (source = r->objects->sources; source; source = source->next) {
- struct multi_pack_index *m = get_multi_pack_index(source);
+ struct odb_source_files *files = odb_source_files_downcast(source);
+ struct multi_pack_index *m = get_multi_pack_index(files->packed);
char *midx_bitmap_name;
if (!m)
diff --git a/pack-revindex.c b/pack-revindex.c
index 1b67863606..62387ae632 100644
--- a/pack-revindex.c
+++ b/pack-revindex.c
@@ -383,13 +383,13 @@ int load_midx_revindex(struct multi_pack_index *m)
* not want to accidentally call munmap() in the middle of the
* MIDX.
*/
- trace2_data_string("load_midx_revindex", m->source->odb->repo,
+ trace2_data_string("load_midx_revindex", m->source->base.odb->repo,
"source", "midx");
m->revindex_data = (const uint32_t *)m->chunk_revindex;
return 0;
}
- trace2_data_string("load_midx_revindex", m->source->odb->repo,
+ trace2_data_string("load_midx_revindex", m->source->base.odb->repo,
"source", "rev");
if (m->has_chain)
@@ -401,7 +401,7 @@ int load_midx_revindex(struct multi_pack_index *m)
midx_get_checksum_hash(m),
MIDX_EXT_REV);
- ret = load_revindex_from_disk(m->source->odb->repo->hash_algo,
+ ret = load_revindex_from_disk(m->source->base.odb->repo->hash_algo,
revindex_name.buf,
m->num_objects,
&m->revindex_map,
diff --git a/repack-geometry.c b/repack-geometry.c
index 2064683dcf..15b3412950 100644
--- a/repack-geometry.c
+++ b/repack-geometry.c
@@ -32,7 +32,8 @@ void pack_geometry_init(struct pack_geometry *geometry,
{
struct packed_git *p;
struct strbuf buf = STRBUF_INIT;
- struct multi_pack_index *m = get_multi_pack_index(existing->source);
+ struct odb_source_files *files = odb_source_files_downcast(existing->source);
+ struct multi_pack_index *m = get_multi_pack_index(files->packed);
repo_for_each_pack(existing->repo, p) {
if (geometry->midx_layer_threshold_set && m &&
diff --git a/repack-midx.c b/repack-midx.c
index b6b1de7180..7c7c3620e5 100644
--- a/repack-midx.c
+++ b/repack-midx.c
@@ -557,13 +557,14 @@ static void repack_make_midx_append_plan(struct repack_write_midx_opts *opts,
struct midx_compaction_step **steps_p,
size_t *steps_nr_p)
{
+ struct odb_source_files *files = odb_source_files_downcast(opts->existing->source);
struct multi_pack_index *m;
struct midx_compaction_step *steps = NULL;
struct midx_compaction_step *step;
size_t steps_nr = 0, steps_alloc = 0;
odb_reprepare(opts->existing->repo->objects);
- m = get_multi_pack_index(opts->existing->source);
+ m = get_multi_pack_index(files->packed);
if (opts->names->nr) {
struct strbuf buf = STRBUF_INIT;
@@ -606,6 +607,7 @@ static int repack_make_midx_compaction_plan(struct repack_write_midx_opts *opts,
struct midx_compaction_step **steps_p,
size_t *steps_nr_p)
{
+ struct odb_source_files *files = odb_source_files_downcast(opts->existing->source);
struct multi_pack_index *m;
struct midx_compaction_step *steps = NULL;
struct midx_compaction_step step = { 0 };
@@ -618,7 +620,7 @@ static int repack_make_midx_compaction_plan(struct repack_write_midx_opts *opts,
opts->existing->repo);
odb_reprepare(opts->existing->repo->objects);
- m = get_multi_pack_index(opts->existing->source);
+ m = get_multi_pack_index(files->packed);
for (i = 0; m && i < m->num_packs + m->num_packs_in_base; i++) {
if (prepare_midx_pack(m, i)) {
@@ -938,6 +940,7 @@ static int repack_make_midx_compaction_plan(struct repack_write_midx_opts *opts,
static int write_midx_incremental(struct repack_write_midx_opts *opts)
{
+ struct odb_source_files *files = odb_source_files_downcast(opts->existing->source);
struct midx_compaction_step *steps = NULL;
struct strbuf lock_name = STRBUF_INIT;
struct lock_file lf;
@@ -946,7 +949,7 @@ static int write_midx_incremental(struct repack_write_midx_opts *opts)
size_t i;
int ret = 0;
- get_midx_chain_filename(opts->existing->source, &lock_name);
+ get_midx_chain_filename(files->packed, &lock_name);
if (safe_create_leading_directories(opts->existing->repo,
lock_name.buf))
die_errno(_("unable to create leading directories of %s"),
diff --git a/repack.c b/repack.c
index 571dabb665..d2aa58e134 100644
--- a/repack.c
+++ b/repack.c
@@ -59,10 +59,10 @@ void repack_remove_redundant_pack(struct repository *repo, const char *dir_name,
bool wrote_incremental_midx)
{
struct strbuf buf = STRBUF_INIT;
- struct odb_source *source = repo->objects->sources;
- struct multi_pack_index *m = get_multi_pack_index(source);
+ struct odb_source_files *files = odb_source_files_downcast(repo->objects->sources);
+ struct multi_pack_index *m = get_multi_pack_index(files->packed);
strbuf_addf(&buf, "%s.pack", base_name);
- if (m && source->local && midx_contains_pack(m, buf.buf)) {
+ if (m && files->base.local && midx_contains_pack(m, buf.buf)) {
clear_midx_file(repo);
if (!wrote_incremental_midx)
clear_incremental_midx_files(repo, NULL);
diff --git a/t/helper/test-read-midx.c b/t/helper/test-read-midx.c
index 790000fb26..fb16ec0176 100644
--- a/t/helper/test-read-midx.c
+++ b/t/helper/test-read-midx.c
@@ -13,13 +13,16 @@
static struct multi_pack_index *setup_midx(const char *object_dir)
{
+ struct odb_source_files *files;
struct odb_source *source;
setup_git_directory(the_repository);
source = odb_find_source(the_repository->objects, object_dir);
if (!source)
source = odb_add_to_alternates_memory(the_repository->objects,
object_dir);
- return load_multi_pack_index(source);
+ files = odb_source_files_downcast(source);
+
+ return load_multi_pack_index(files->packed);
}
static int read_midx_file(const char *object_dir, const char *checksum,
@@ -70,7 +73,7 @@ static int read_midx_file(const char *object_dir, const char *checksum,
for (i = 0; i < m->num_packs; i++)
printf("%s\n", m->pack_names[i]);
- printf("object-dir: %s\n", m->source->path);
+ printf("object-dir: %s\n", m->source->base.path);
if (show_objects) {
struct object_id oid;
--
2.54.0.1064.gd145956f57.dirty
^ permalink raw reply related
* [PATCH 14/16] odb/source-packed: stub out remaining functions
From: Patrick Steinhardt @ 2026-06-04 11:25 UTC (permalink / raw)
To: git
In-Reply-To: <20260604-pks-odb-source-packed-v1-0-2e7ab31b4b5c@pks.im>
Stub out remaining functions that we either don't need or that are
basically no-ops.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
odb/source-packed.c | 42 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 42 insertions(+)
diff --git a/odb/source-packed.c b/odb/source-packed.c
index e40b52e445..08a2de9bc5 100644
--- a/odb/source-packed.c
+++ b/odb/source-packed.c
@@ -501,6 +501,43 @@ static int odb_source_packed_freshen_object(struct odb_source *source,
return 1;
}
+static int odb_source_packed_write_object(struct odb_source *source UNUSED,
+ const void *buf UNUSED,
+ unsigned long len UNUSED,
+ enum object_type type UNUSED,
+ struct object_id *oid UNUSED,
+ struct object_id *compat_oid UNUSED,
+ unsigned flags UNUSED)
+{
+ return error("packed backend cannot write objects");
+}
+
+static int odb_source_packed_write_object_stream(struct odb_source *source UNUSED,
+ struct odb_write_stream *stream UNUSED,
+ size_t len UNUSED,
+ struct object_id *oid UNUSED)
+{
+ return error("packed backend cannot write object streams");
+}
+
+static int odb_source_packed_begin_transaction(struct odb_source *source UNUSED,
+ struct odb_transaction **out UNUSED)
+{
+ return error("packed backend cannot begin transactions");
+}
+
+static int odb_source_packed_read_alternates(struct odb_source *source UNUSED,
+ struct strvec *out UNUSED)
+{
+ return 0;
+}
+
+static int odb_source_packed_write_alternate(struct odb_source *source UNUSED,
+ const char *alternate UNUSED)
+{
+ return error("packed backend cannot write alternates");
+}
+
void (*report_garbage)(unsigned seen_bits, const char *path);
static void report_helper(const struct string_list *list,
@@ -715,6 +752,11 @@ struct odb_source_packed *odb_source_packed_new(struct odb_source_files *parent)
packed->base.count_objects = odb_source_packed_count_objects;
packed->base.find_abbrev_len = odb_source_packed_find_abbrev_len;
packed->base.freshen_object = odb_source_packed_freshen_object;
+ packed->base.write_object = odb_source_packed_write_object;
+ packed->base.write_object_stream = odb_source_packed_write_object_stream;
+ packed->base.begin_transaction = odb_source_packed_begin_transaction;
+ packed->base.read_alternates = odb_source_packed_read_alternates;
+ packed->base.write_alternate = odb_source_packed_write_alternate;
if (!is_absolute_path(parent->base.path))
chdir_notify_register(NULL, odb_source_packed_reparent, packed);
--
2.54.0.1064.gd145956f57.dirty
^ permalink raw reply related
* [PATCH 13/16] odb/source-packed: wire up `freshen_object()` callback
From: Patrick Steinhardt @ 2026-06-04 11:25 UTC (permalink / raw)
To: git
In-Reply-To: <20260604-pks-odb-source-packed-v1-0-2e7ab31b4b5c@pks.im>
Move `packfile_store_freshen_object()` and from "packfile.c" into
"odb/source-packed.c" and wire it up as the `freshen_object()` callback
of the "packed" source.
Note that this removes the last external caller of `find_pack_entry()`
from "packfile.c", which means that we can now make this function
static.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
odb/source-files.c | 2 +-
odb/source-packed.c | 26 +++++++++++++++++++++++---
odb/source-packed.h | 6 ------
packfile.c | 16 ----------------
packfile.h | 3 ---
5 files changed, 24 insertions(+), 29 deletions(-)
diff --git a/odb/source-files.c b/odb/source-files.c
index 8ad782dc7b..fa2e18e71b 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -152,7 +152,7 @@ static int odb_source_files_freshen_object(struct odb_source *source,
const struct object_id *oid)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- if (packfile_store_freshen_object(files->packed, oid) ||
+ if (odb_source_freshen_object(&files->packed->base, oid) ||
odb_source_freshen_object(&files->loose->base, oid))
return 1;
return 0;
diff --git a/odb/source-packed.c b/odb/source-packed.c
index b801b62023..e40b52e445 100644
--- a/odb/source-packed.c
+++ b/odb/source-packed.c
@@ -9,9 +9,9 @@
#include "odb/streaming.h"
#include "packfile.h"
-int find_pack_entry(struct odb_source_packed *store,
- const struct object_id *oid,
- struct pack_entry *e)
+static int find_pack_entry(struct odb_source_packed *store,
+ const struct object_id *oid,
+ struct pack_entry *e)
{
struct packfile_list_entry *l;
@@ -482,6 +482,25 @@ static int odb_source_packed_find_abbrev_len(struct odb_source *source,
return 0;
}
+static int odb_source_packed_freshen_object(struct odb_source *source,
+ const struct object_id *oid)
+{
+ struct odb_source_packed *packed = odb_source_packed_downcast(source);
+ struct pack_entry e;
+
+ if (!find_pack_entry(packed, oid, &e))
+ return 0;
+ if (e.p->is_cruft)
+ return 0;
+ if (e.p->freshened)
+ return 1;
+ if (utime(e.p->pack_name, NULL))
+ return 0;
+ e.p->freshened = 1;
+
+ return 1;
+}
+
void (*report_garbage)(unsigned seen_bits, const char *path);
static void report_helper(const struct string_list *list,
@@ -695,6 +714,7 @@ struct odb_source_packed *odb_source_packed_new(struct odb_source_files *parent)
packed->base.for_each_object = odb_source_packed_for_each_object;
packed->base.count_objects = odb_source_packed_count_objects;
packed->base.find_abbrev_len = odb_source_packed_find_abbrev_len;
+ packed->base.freshen_object = odb_source_packed_freshen_object;
if (!is_absolute_path(parent->base.path))
chdir_notify_register(NULL, odb_source_packed_reparent, packed);
diff --git a/odb/source-packed.h b/odb/source-packed.h
index aa2e80281b..6645f4f943 100644
--- a/odb/source-packed.h
+++ b/odb/source-packed.h
@@ -98,10 +98,4 @@ static inline struct odb_source_packed *odb_source_packed_downcast(struct odb_so
*/
void odb_source_packed_prepare(struct odb_source_packed *source);
-struct pack_entry;
-
-int find_pack_entry(struct odb_source_packed *store,
- const struct object_id *oid,
- struct pack_entry *e);
-
#endif
diff --git a/packfile.c b/packfile.c
index 69f6354ed0..d7de0412ff 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1975,22 +1975,6 @@ int packfile_fill_entry(struct packed_git *p,
return 1;
}
-int packfile_store_freshen_object(struct odb_source_packed *store,
- const struct object_id *oid)
-{
- struct pack_entry e;
- if (!find_pack_entry(store, oid, &e))
- return 0;
- if (e.p->is_cruft)
- return 0;
- if (e.p->freshened)
- return 1;
- if (utime(e.p->pack_name, NULL))
- return 0;
- e.p->freshened = 1;
- return 1;
-}
-
static void maybe_invalidate_kept_pack_cache(struct odb_source_packed *store,
unsigned flags)
{
diff --git a/packfile.h b/packfile.h
index 8d23bec644..ed49ab7f21 100644
--- a/packfile.h
+++ b/packfile.h
@@ -144,9 +144,6 @@ static inline void repo_for_each_pack_data_next(struct repo_for_each_pack_data *
struct packed_git *packfile_store_load_pack(struct odb_source_packed *store,
const char *idx_path, int local);
-int packfile_store_freshen_object(struct odb_source_packed *store,
- const struct object_id *oid);
-
enum kept_pack_type {
KEPT_PACK_ON_DISK = (1 << 0),
KEPT_PACK_IN_CORE = (1 << 1),
--
2.54.0.1064.gd145956f57.dirty
^ permalink raw reply related
* [PATCH 12/16] odb/source-packed: wire up `find_abbrev_len()` callback
From: Patrick Steinhardt @ 2026-06-04 11:25 UTC (permalink / raw)
To: git
In-Reply-To: <20260604-pks-odb-source-packed-v1-0-2e7ab31b4b5c@pks.im>
Move `packfile_store_find_abbrev_len()` and its associated helpers from
"packfile.c" into "odb/source-packed.c" and wire it up as the
`find_abbrev_len()` callback of the "packed" source.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
odb/source-files.c | 2 +-
odb/source-packed.c | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++
packfile.c | 111 ---------------------------------------------------
packfile.h | 5 ---
4 files changed, 113 insertions(+), 117 deletions(-)
diff --git a/odb/source-files.c b/odb/source-files.c
index 274923e0ba..8ad782dc7b 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -133,7 +133,7 @@ static int odb_source_files_find_abbrev_len(struct odb_source *source,
unsigned len = min_len;
int ret;
- ret = packfile_store_find_abbrev_len(files->packed, oid, len, &len);
+ ret = odb_source_find_abbrev_len(&files->packed->base, oid, len, &len);
if (ret < 0)
goto out;
diff --git a/odb/source-packed.c b/odb/source-packed.c
index 013d8a50f8..b801b62023 100644
--- a/odb/source-packed.c
+++ b/odb/source-packed.c
@@ -370,6 +370,117 @@ static int odb_source_packed_count_objects(struct odb_source *source,
return ret;
}
+static int extend_abbrev_len(const struct object_id *a,
+ const struct object_id *b,
+ unsigned *out)
+{
+ unsigned len = oid_common_prefix_hexlen(a, b);
+ if (len != hash_algos[a->algo].hexsz && len >= *out)
+ *out = len + 1;
+ return 0;
+}
+
+static void find_abbrev_len_for_midx(struct multi_pack_index *m,
+ const struct object_id *oid,
+ unsigned min_len,
+ unsigned *out)
+{
+ unsigned len = min_len;
+
+ for (; m; m = m->base_midx) {
+ int match = 0;
+ uint32_t num, first = 0;
+ struct object_id found_oid;
+
+ if (!m->num_objects)
+ continue;
+
+ num = m->num_objects + m->num_objects_in_base;
+ match = bsearch_one_midx(oid, m, &first);
+
+ /*
+ * first is now the position in the packfile where we
+ * would insert the object ID if it does not exist (or the
+ * position of the object ID if it does exist). Hence, we
+ * consider a maximum of two objects nearby for the
+ * abbreviation length.
+ */
+
+ if (!match) {
+ if (nth_midxed_object_oid(&found_oid, m, first))
+ extend_abbrev_len(&found_oid, oid, &len);
+ } else if (first < num - 1) {
+ if (nth_midxed_object_oid(&found_oid, m, first + 1))
+ extend_abbrev_len(&found_oid, oid, &len);
+ }
+ if (first > 0) {
+ if (nth_midxed_object_oid(&found_oid, m, first - 1))
+ extend_abbrev_len(&found_oid, oid, &len);
+ }
+ }
+
+ *out = len;
+}
+
+static void find_abbrev_len_for_pack(struct packed_git *p,
+ const struct object_id *oid,
+ unsigned min_len,
+ unsigned *out)
+{
+ int match;
+ uint32_t num, first = 0;
+ struct object_id found_oid;
+ unsigned len = min_len;
+
+ num = p->num_objects;
+ match = bsearch_pack(oid, p, &first);
+
+ /*
+ * first is now the position in the packfile where we would insert
+ * the object ID if it does not exist (or the position of mad->hash if
+ * it does exist). Hence, we consider a maximum of two objects
+ * nearby for the abbreviation length.
+ */
+ if (!match) {
+ if (!nth_packed_object_id(&found_oid, p, first))
+ extend_abbrev_len(&found_oid, oid, &len);
+ } else if (first < num - 1) {
+ if (!nth_packed_object_id(&found_oid, p, first + 1))
+ extend_abbrev_len(&found_oid, oid, &len);
+ }
+ if (first > 0) {
+ if (!nth_packed_object_id(&found_oid, p, first - 1))
+ extend_abbrev_len(&found_oid, oid, &len);
+ }
+
+ *out = len;
+}
+
+static int odb_source_packed_find_abbrev_len(struct odb_source *source,
+ const struct object_id *oid,
+ unsigned min_len,
+ unsigned *out)
+{
+ struct odb_source_packed *packed = odb_source_packed_downcast(source);
+ struct packfile_list_entry *e;
+ struct multi_pack_index *m;
+
+ m = get_multi_pack_index(&packed->files->base);
+ if (m)
+ find_abbrev_len_for_midx(m, oid, min_len, &min_len);
+
+ for (e = packfile_store_get_packs(packed); e; e = e->next) {
+ if (e->pack->multi_pack_index)
+ continue;
+ if (open_pack_index(e->pack) || !e->pack->num_objects)
+ continue;
+
+ find_abbrev_len_for_pack(e->pack, oid, min_len, &min_len);
+ }
+
+ *out = min_len;
+ return 0;
+}
void (*report_garbage)(unsigned seen_bits, const char *path);
@@ -583,6 +694,7 @@ struct odb_source_packed *odb_source_packed_new(struct odb_source_files *parent)
packed->base.read_object_stream = odb_source_packed_read_object_stream;
packed->base.for_each_object = odb_source_packed_for_each_object;
packed->base.count_objects = odb_source_packed_count_objects;
+ packed->base.find_abbrev_len = odb_source_packed_find_abbrev_len;
if (!is_absolute_path(parent->base.path))
chdir_notify_register(NULL, odb_source_packed_reparent, packed);
diff --git a/packfile.c b/packfile.c
index 5cb14b28de..69f6354ed0 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2120,117 +2120,6 @@ int for_each_object_in_pack(struct packed_git *p,
return r;
}
-static int extend_abbrev_len(const struct object_id *a,
- const struct object_id *b,
- unsigned *out)
-{
- unsigned len = oid_common_prefix_hexlen(a, b);
- if (len != hash_algos[a->algo].hexsz && len >= *out)
- *out = len + 1;
- return 0;
-}
-
-static void find_abbrev_len_for_midx(struct multi_pack_index *m,
- const struct object_id *oid,
- unsigned min_len,
- unsigned *out)
-{
- unsigned len = min_len;
-
- for (; m; m = m->base_midx) {
- int match = 0;
- uint32_t num, first = 0;
- struct object_id found_oid;
-
- if (!m->num_objects)
- continue;
-
- num = m->num_objects + m->num_objects_in_base;
- match = bsearch_one_midx(oid, m, &first);
-
- /*
- * first is now the position in the packfile where we
- * would insert the object ID if it does not exist (or the
- * position of the object ID if it does exist). Hence, we
- * consider a maximum of two objects nearby for the
- * abbreviation length.
- */
-
- if (!match) {
- if (nth_midxed_object_oid(&found_oid, m, first))
- extend_abbrev_len(&found_oid, oid, &len);
- } else if (first < num - 1) {
- if (nth_midxed_object_oid(&found_oid, m, first + 1))
- extend_abbrev_len(&found_oid, oid, &len);
- }
- if (first > 0) {
- if (nth_midxed_object_oid(&found_oid, m, first - 1))
- extend_abbrev_len(&found_oid, oid, &len);
- }
- }
-
- *out = len;
-}
-
-static void find_abbrev_len_for_pack(struct packed_git *p,
- const struct object_id *oid,
- unsigned min_len,
- unsigned *out)
-{
- int match;
- uint32_t num, first = 0;
- struct object_id found_oid;
- unsigned len = min_len;
-
- num = p->num_objects;
- match = bsearch_pack(oid, p, &first);
-
- /*
- * first is now the position in the packfile where we would insert
- * the object ID if it does not exist (or the position of mad->hash if
- * it does exist). Hence, we consider a maximum of two objects
- * nearby for the abbreviation length.
- */
- if (!match) {
- if (!nth_packed_object_id(&found_oid, p, first))
- extend_abbrev_len(&found_oid, oid, &len);
- } else if (first < num - 1) {
- if (!nth_packed_object_id(&found_oid, p, first + 1))
- extend_abbrev_len(&found_oid, oid, &len);
- }
- if (first > 0) {
- if (!nth_packed_object_id(&found_oid, p, first - 1))
- extend_abbrev_len(&found_oid, oid, &len);
- }
-
- *out = len;
-}
-
-int packfile_store_find_abbrev_len(struct odb_source_packed *store,
- const struct object_id *oid,
- unsigned min_len,
- unsigned *out)
-{
- struct packfile_list_entry *e;
- struct multi_pack_index *m;
-
- m = get_multi_pack_index(&store->files->base);
- if (m)
- find_abbrev_len_for_midx(m, oid, min_len, &min_len);
-
- for (e = packfile_store_get_packs(store); e; e = e->next) {
- if (e->pack->multi_pack_index)
- continue;
- if (open_pack_index(e->pack) || !e->pack->num_objects)
- continue;
-
- find_abbrev_len_for_pack(e->pack, oid, min_len, &min_len);
- }
-
- *out = min_len;
- return 0;
-}
-
struct add_promisor_object_data {
struct repository *repo;
struct oidset *set;
diff --git a/packfile.h b/packfile.h
index 9f5b5c145a..8d23bec644 100644
--- a/packfile.h
+++ b/packfile.h
@@ -229,11 +229,6 @@ int for_each_object_in_pack(struct packed_git *p,
each_packed_object_fn, void *data,
enum odb_for_each_object_flags flags);
-int packfile_store_find_abbrev_len(struct odb_source_packed *store,
- const struct object_id *oid,
- unsigned min_len,
- unsigned *out);
-
/* A hook to report invalid files in pack directory */
#define PACKDIR_FILE_PACK 1
#define PACKDIR_FILE_IDX 2
--
2.54.0.1064.gd145956f57.dirty
^ permalink raw reply related
* [PATCH 11/16] odb/source-packed: wire up `count_objects()` callback
From: Patrick Steinhardt @ 2026-06-04 11:25 UTC (permalink / raw)
To: git
In-Reply-To: <20260604-pks-odb-source-packed-v1-0-2e7ab31b4b5c@pks.im>
Move `packfile_store_count_objects()` from "packfile.c" into
"odb/source-packed.c" and wire it up as the `count_objects()` callback
of the "packed" source.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
odb/source-files.c | 2 +-
odb/source-packed.c | 34 ++++++++++++++++++++++++++++++++++
packfile.c | 31 -------------------------------
packfile.h | 10 ----------
4 files changed, 35 insertions(+), 42 deletions(-)
diff --git a/odb/source-files.c b/odb/source-files.c
index c73a7e5f90..274923e0ba 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -103,7 +103,7 @@ static int odb_source_files_count_objects(struct odb_source *source,
unsigned long count;
int ret;
- ret = packfile_store_count_objects(files->packed, flags, &count);
+ ret = odb_source_count_objects(&files->packed->base, flags, &count);
if (ret < 0)
goto out;
diff --git a/odb/source-packed.c b/odb/source-packed.c
index a61c809c8c..013d8a50f8 100644
--- a/odb/source-packed.c
+++ b/odb/source-packed.c
@@ -338,6 +338,39 @@ static int odb_source_packed_for_each_object(struct odb_source *source,
return ret;
}
+static int odb_source_packed_count_objects(struct odb_source *source,
+ enum odb_count_objects_flags flags UNUSED,
+ unsigned long *out)
+{
+ struct odb_source_packed *packed = odb_source_packed_downcast(source);
+ struct packfile_list_entry *e;
+ struct multi_pack_index *m;
+ unsigned long count = 0;
+ int ret;
+
+ m = get_multi_pack_index(&packed->files->base);
+ if (m)
+ count += m->num_objects + m->num_objects_in_base;
+
+ for (e = packfile_store_get_packs(packed); e; e = e->next) {
+ if (e->pack->multi_pack_index)
+ continue;
+ if (open_pack_index(e->pack)) {
+ ret = -1;
+ goto out;
+ }
+
+ count += e->pack->num_objects;
+ }
+
+ *out = count;
+ ret = 0;
+
+out:
+ return ret;
+}
+
+
void (*report_garbage)(unsigned seen_bits, const char *path);
static void report_helper(const struct string_list *list,
@@ -549,6 +582,7 @@ struct odb_source_packed *odb_source_packed_new(struct odb_source_files *parent)
packed->base.read_object_info = odb_source_packed_read_object_info;
packed->base.read_object_stream = odb_source_packed_read_object_stream;
packed->base.for_each_object = odb_source_packed_for_each_object;
+ packed->base.count_objects = odb_source_packed_count_objects;
if (!is_absolute_path(parent->base.path))
chdir_notify_register(NULL, odb_source_packed_reparent, packed);
diff --git a/packfile.c b/packfile.c
index 95136b050b..5cb14b28de 100644
--- a/packfile.c
+++ b/packfile.c
@@ -949,37 +949,6 @@ struct packfile_list_entry *packfile_store_get_packs(struct odb_source_packed *s
return store->packs.head;
}
-int packfile_store_count_objects(struct odb_source_packed *store,
- enum odb_count_objects_flags flags UNUSED,
- unsigned long *out)
-{
- struct packfile_list_entry *e;
- struct multi_pack_index *m;
- unsigned long count = 0;
- int ret;
-
- m = get_multi_pack_index(&store->files->base);
- if (m)
- count += m->num_objects + m->num_objects_in_base;
-
- for (e = packfile_store_get_packs(store); e; e = e->next) {
- if (e->pack->multi_pack_index)
- continue;
- if (open_pack_index(e->pack)) {
- ret = -1;
- goto out;
- }
-
- count += e->pack->num_objects;
- }
-
- *out = count;
- ret = 0;
-
-out:
- return ret;
-}
-
unsigned long unpack_object_header_buffer(const unsigned char *buf,
unsigned long len, enum object_type *type, size_t *sizep)
{
diff --git a/packfile.h b/packfile.h
index 210650923f..9f5b5c145a 100644
--- a/packfile.h
+++ b/packfile.h
@@ -153,16 +153,6 @@ enum kept_pack_type {
KEPT_PACK_IN_CORE_OPEN = (1 << 2),
};
-/*
- * Count the number objects contained in the given packfile store. If
- * successful, the number of objects will be written to the `out` pointer.
- *
- * Return 0 on success, a negative error code otherwise.
- */
-int packfile_store_count_objects(struct odb_source_packed *store,
- enum odb_count_objects_flags flags,
- unsigned long *out);
-
/*
* Retrieve the cache of kept packs from the given packfile store. Accepts a
* combination of `kept_pack_type` flags. The cache is computed on demand and
--
2.54.0.1064.gd145956f57.dirty
^ permalink raw reply related
* [PATCH 10/16] odb/source-packed: wire up `for_each_object()` callback
From: Patrick Steinhardt @ 2026-06-04 11:25 UTC (permalink / raw)
To: git
In-Reply-To: <20260604-pks-odb-source-packed-v1-0-2e7ab31b4b5c@pks.im>
Move `packfile_store_for_each_object()` and its associated helpers from
"packfile.c" into "odb/source-packed.c" and wire it up as the
`for_each_object()` callback of the "packed" source.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/cat-file.c | 4 +-
builtin/pack-objects.c | 4 +-
commit-graph.c | 4 +-
odb/source-files.c | 2 +-
odb/source-packed.c | 258 ++++++++++++++++++++++++++++++++++++++++++++++++
packfile.c | 260 +------------------------------------------------
packfile.h | 17 +---
7 files changed, 269 insertions(+), 280 deletions(-)
diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index 04b64006a5..d997011531 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -916,8 +916,8 @@ static void batch_each_object(struct batch_options *opt,
for (source = the_repository->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- int ret = packfile_store_for_each_object(files->packed, &oi,
- batch_one_object_oi, &payload, &opts);
+ int ret = odb_source_for_each_object(&files->packed->base, &oi,
+ batch_one_object_oi, &payload, &opts);
if (ret)
break;
}
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 50675481e1..5e94805478 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -4503,8 +4503,8 @@ static void add_objects_in_unpacked_packs(void)
if (!source->local)
continue;
- if (packfile_store_for_each_object(files->packed, &oi,
- add_object_in_unpacked_pack, NULL, &opts))
+ if (odb_source_for_each_object(&files->packed->base, &oi,
+ add_object_in_unpacked_pack, NULL, &opts))
die(_("cannot open pack index"));
}
}
diff --git a/commit-graph.c b/commit-graph.c
index 9abe62bd5a..1e4038baf3 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -2016,8 +2016,8 @@ static void fill_oids_from_all_packs(struct write_commit_graph_context *ctx)
odb_prepare_alternates(ctx->r->objects);
for (source = ctx->r->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- packfile_store_for_each_object(files->packed, &oi, add_packed_commits_oi,
- ctx, &opts);
+ odb_source_for_each_object(&files->packed->base, &oi, add_packed_commits_oi,
+ ctx, &opts);
}
if (ctx->progress_done < ctx->approx_nr_objects)
diff --git a/odb/source-files.c b/odb/source-files.c
index dff69d0e4e..c73a7e5f90 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -88,7 +88,7 @@ static int odb_source_files_for_each_object(struct odb_source *source,
return ret;
}
- ret = packfile_store_for_each_object(files->packed, request, cb, cb_data, opts);
+ ret = odb_source_for_each_object(&files->packed->base, request, cb, cb_data, opts);
if (ret)
return ret;
diff --git a/odb/source-packed.c b/odb/source-packed.c
index 23d7149fe3..a61c809c8c 100644
--- a/odb/source-packed.c
+++ b/odb/source-packed.c
@@ -81,6 +81,263 @@ static int odb_source_packed_read_object_stream(struct odb_read_stream **out,
return packfile_read_object_stream(out, oid, e.p, e.offset);
}
+struct odb_source_packed_for_each_object_wrapper_data {
+ struct odb_source_packed *store;
+ const struct object_info *request;
+ odb_for_each_object_cb cb;
+ void *cb_data;
+};
+
+static int odb_source_packed_for_each_object_wrapper(const struct object_id *oid,
+ struct packed_git *pack,
+ uint32_t index_pos,
+ void *cb_data)
+{
+ struct odb_source_packed_for_each_object_wrapper_data *data = cb_data;
+
+ if (data->request) {
+ off_t offset = nth_packed_object_offset(pack, index_pos);
+ struct object_info oi = *data->request;
+
+ if (packed_object_info_with_index_pos(pack, offset,
+ &index_pos, &oi) < 0) {
+ mark_bad_packed_object(pack, oid);
+ return -1;
+ }
+
+ return data->cb(oid, &oi, data->cb_data);
+ } else {
+ return data->cb(oid, NULL, data->cb_data);
+ }
+}
+
+static int match_hash(unsigned len, const unsigned char *a, const unsigned char *b)
+{
+ do {
+ if (*a != *b)
+ return 0;
+ a++;
+ b++;
+ len -= 2;
+ } while (len > 1);
+ if (len)
+ if ((*a ^ *b) & 0xf0)
+ return 0;
+ return 1;
+}
+
+static int for_each_prefixed_object_in_midx(
+ struct odb_source_packed *store,
+ struct multi_pack_index *m,
+ const struct odb_for_each_object_options *opts,
+ struct odb_source_packed_for_each_object_wrapper_data *data)
+{
+ int ret;
+
+ for (; m; m = m->base_midx) {
+ uint32_t num, i, first = 0;
+ int len = opts->prefix_hex_len > m->source->odb->repo->hash_algo->hexsz ?
+ m->source->odb->repo->hash_algo->hexsz : opts->prefix_hex_len;
+
+ if (!m->num_objects)
+ continue;
+
+ num = m->num_objects + m->num_objects_in_base;
+
+ bsearch_one_midx(opts->prefix, m, &first);
+
+ /*
+ * At this point, "first" is the location of the lowest
+ * object with an object name that could match "opts->prefix".
+ * See if we have 0, 1 or more objects that actually match(es).
+ */
+ for (i = first; i < num; i++) {
+ const struct object_id *current = NULL;
+ struct object_id oid;
+
+ current = nth_midxed_object_oid(&oid, m, i);
+
+ if (!match_hash(len, opts->prefix->hash, current->hash))
+ break;
+
+ if (data->request) {
+ struct object_info oi = *data->request;
+
+ ret = odb_source_read_object_info(&store->base, current,
+ &oi, 0);
+ if (ret)
+ goto out;
+
+ ret = data->cb(&oid, &oi, data->cb_data);
+ if (ret)
+ goto out;
+ } else {
+ ret = data->cb(&oid, NULL, data->cb_data);
+ if (ret)
+ goto out;
+ }
+ }
+ }
+
+ ret = 0;
+
+out:
+ return ret;
+}
+
+static int for_each_prefixed_object_in_pack(
+ struct odb_source_packed *store,
+ struct packed_git *p,
+ const struct odb_for_each_object_options *opts,
+ struct odb_source_packed_for_each_object_wrapper_data *data)
+{
+ uint32_t num, i, first = 0;
+ int len = opts->prefix_hex_len > p->repo->hash_algo->hexsz ?
+ p->repo->hash_algo->hexsz : opts->prefix_hex_len;
+ int ret;
+
+ num = p->num_objects;
+ bsearch_pack(opts->prefix, p, &first);
+
+ /*
+ * At this point, "first" is the location of the lowest object
+ * with an object name that could match "bin_pfx". See if we have
+ * 0, 1 or more objects that actually match(es).
+ */
+ for (i = first; i < num; i++) {
+ struct object_id oid;
+
+ nth_packed_object_id(&oid, p, i);
+ if (!match_hash(len, opts->prefix->hash, oid.hash))
+ break;
+
+ if (data->request) {
+ struct object_info oi = *data->request;
+
+ ret = odb_source_read_object_info(&store->base, &oid, &oi, 0);
+ if (ret)
+ goto out;
+
+ ret = data->cb(&oid, &oi, data->cb_data);
+ if (ret)
+ goto out;
+ } else {
+ ret = data->cb(&oid, NULL, data->cb_data);
+ if (ret)
+ goto out;
+ }
+ }
+
+ ret = 0;
+
+out:
+ return ret;
+}
+
+static int odb_source_packed_for_each_prefixed_object(
+ struct odb_source_packed *store,
+ const struct odb_for_each_object_options *opts,
+ struct odb_source_packed_for_each_object_wrapper_data *data)
+{
+ struct packfile_list_entry *e;
+ struct multi_pack_index *m;
+ bool pack_errors = false;
+ int ret;
+
+ if (opts->flags)
+ BUG("flags unsupported");
+
+ store->skip_mru_updates = true;
+
+ m = get_multi_pack_index(&store->files->base);
+ if (m) {
+ ret = for_each_prefixed_object_in_midx(store, m, opts, data);
+ if (ret)
+ goto out;
+ }
+
+ for (e = packfile_store_get_packs(store); e; e = e->next) {
+ if (e->pack->multi_pack_index)
+ continue;
+
+ if (open_pack_index(e->pack)) {
+ pack_errors = true;
+ continue;
+ }
+
+ if (!e->pack->num_objects)
+ continue;
+
+ ret = for_each_prefixed_object_in_pack(store, e->pack, opts, data);
+ if (ret)
+ goto out;
+ }
+
+ ret = 0;
+
+out:
+ store->skip_mru_updates = false;
+ if (!ret && pack_errors)
+ ret = -1;
+ return ret;
+}
+
+static int odb_source_packed_for_each_object(struct odb_source *source,
+ const struct object_info *request,
+ odb_for_each_object_cb cb,
+ void *cb_data,
+ const struct odb_for_each_object_options *opts)
+{
+ struct odb_source_packed *packed = odb_source_packed_downcast(source);
+ struct odb_source_packed_for_each_object_wrapper_data data = {
+ .store = packed,
+ .request = request,
+ .cb = cb,
+ .cb_data = cb_data,
+ };
+ struct packfile_list_entry *e;
+ int pack_errors = 0, ret;
+
+ if (opts->prefix)
+ return odb_source_packed_for_each_prefixed_object(packed, opts, &data);
+
+ packed->skip_mru_updates = true;
+
+ for (e = packfile_store_get_packs(packed); e; e = e->next) {
+ struct packed_git *p = e->pack;
+
+ if ((opts->flags & ODB_FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
+ continue;
+ if ((opts->flags & ODB_FOR_EACH_OBJECT_PROMISOR_ONLY) &&
+ !p->pack_promisor)
+ continue;
+ if ((opts->flags & ODB_FOR_EACH_OBJECT_SKIP_IN_CORE_KEPT_PACKS) &&
+ p->pack_keep_in_core)
+ continue;
+ if ((opts->flags & ODB_FOR_EACH_OBJECT_SKIP_ON_DISK_KEPT_PACKS) &&
+ p->pack_keep)
+ continue;
+ if (open_pack_index(p)) {
+ pack_errors = 1;
+ continue;
+ }
+
+ ret = for_each_object_in_pack(p, odb_source_packed_for_each_object_wrapper,
+ &data, opts->flags);
+ if (ret)
+ goto out;
+ }
+
+ ret = 0;
+
+out:
+ packed->skip_mru_updates = false;
+
+ if (!ret && pack_errors)
+ ret = -1;
+ return ret;
+}
+
void (*report_garbage)(unsigned seen_bits, const char *path);
static void report_helper(const struct string_list *list,
@@ -291,6 +548,7 @@ struct odb_source_packed *odb_source_packed_new(struct odb_source_files *parent)
packed->base.reprepare = odb_source_packed_reprepare;
packed->base.read_object_info = odb_source_packed_read_object_info;
packed->base.read_object_stream = odb_source_packed_read_object_stream;
+ packed->base.for_each_object = odb_source_packed_for_each_object;
if (!is_absolute_path(parent->base.path))
chdir_notify_register(NULL, odb_source_packed_reparent, packed);
diff --git a/packfile.c b/packfile.c
index bcb4b61608..95136b050b 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1445,8 +1445,8 @@ static void add_delta_base_cache(struct packed_git *p, off_t base_offset,
hashmap_add(&delta_base_cache, &ent->ent);
}
-static int packed_object_info_with_index_pos(struct packed_git *p, off_t obj_offset,
- uint32_t *maybe_index_pos, struct object_info *oi)
+int packed_object_info_with_index_pos(struct packed_git *p, off_t obj_offset,
+ uint32_t *maybe_index_pos, struct object_info *oi)
{
struct pack_window *w_curs = NULL;
size_t size;
@@ -2151,262 +2151,6 @@ int for_each_object_in_pack(struct packed_git *p,
return r;
}
-struct odb_source_packed_for_each_object_wrapper_data {
- struct odb_source_packed *store;
- const struct object_info *request;
- odb_for_each_object_cb cb;
- void *cb_data;
-};
-
-static int packfile_store_for_each_object_wrapper(const struct object_id *oid,
- struct packed_git *pack,
- uint32_t index_pos,
- void *cb_data)
-{
- struct odb_source_packed_for_each_object_wrapper_data *data = cb_data;
-
- if (data->request) {
- off_t offset = nth_packed_object_offset(pack, index_pos);
- struct object_info oi = *data->request;
-
- if (packed_object_info_with_index_pos(pack, offset,
- &index_pos, &oi) < 0) {
- mark_bad_packed_object(pack, oid);
- return -1;
- }
-
- return data->cb(oid, &oi, data->cb_data);
- } else {
- return data->cb(oid, NULL, data->cb_data);
- }
-}
-
-static int match_hash(unsigned len, const unsigned char *a, const unsigned char *b)
-{
- do {
- if (*a != *b)
- return 0;
- a++;
- b++;
- len -= 2;
- } while (len > 1);
- if (len)
- if ((*a ^ *b) & 0xf0)
- return 0;
- return 1;
-}
-
-static int for_each_prefixed_object_in_midx(
- struct odb_source_packed *store,
- struct multi_pack_index *m,
- const struct odb_for_each_object_options *opts,
- struct odb_source_packed_for_each_object_wrapper_data *data)
-{
- int ret;
-
- for (; m; m = m->base_midx) {
- uint32_t num, i, first = 0;
- int len = opts->prefix_hex_len > m->source->odb->repo->hash_algo->hexsz ?
- m->source->odb->repo->hash_algo->hexsz : opts->prefix_hex_len;
-
- if (!m->num_objects)
- continue;
-
- num = m->num_objects + m->num_objects_in_base;
-
- bsearch_one_midx(opts->prefix, m, &first);
-
- /*
- * At this point, "first" is the location of the lowest
- * object with an object name that could match "opts->prefix".
- * See if we have 0, 1 or more objects that actually match(es).
- */
- for (i = first; i < num; i++) {
- const struct object_id *current = NULL;
- struct object_id oid;
-
- current = nth_midxed_object_oid(&oid, m, i);
-
- if (!match_hash(len, opts->prefix->hash, current->hash))
- break;
-
- if (data->request) {
- struct object_info oi = *data->request;
-
- ret = odb_source_read_object_info(&store->base, current,
- &oi, 0);
- if (ret)
- goto out;
-
- ret = data->cb(&oid, &oi, data->cb_data);
- if (ret)
- goto out;
- } else {
- ret = data->cb(&oid, NULL, data->cb_data);
- if (ret)
- goto out;
- }
- }
- }
-
- ret = 0;
-
-out:
- return ret;
-}
-
-static int for_each_prefixed_object_in_pack(
- struct odb_source_packed *store,
- struct packed_git *p,
- const struct odb_for_each_object_options *opts,
- struct odb_source_packed_for_each_object_wrapper_data *data)
-{
- uint32_t num, i, first = 0;
- int len = opts->prefix_hex_len > p->repo->hash_algo->hexsz ?
- p->repo->hash_algo->hexsz : opts->prefix_hex_len;
- int ret;
-
- num = p->num_objects;
- bsearch_pack(opts->prefix, p, &first);
-
- /*
- * At this point, "first" is the location of the lowest object
- * with an object name that could match "bin_pfx". See if we have
- * 0, 1 or more objects that actually match(es).
- */
- for (i = first; i < num; i++) {
- struct object_id oid;
-
- nth_packed_object_id(&oid, p, i);
- if (!match_hash(len, opts->prefix->hash, oid.hash))
- break;
-
- if (data->request) {
- struct object_info oi = *data->request;
-
- ret = odb_source_read_object_info(&store->base, &oid, &oi, 0);
- if (ret)
- goto out;
-
- ret = data->cb(&oid, &oi, data->cb_data);
- if (ret)
- goto out;
- } else {
- ret = data->cb(&oid, NULL, data->cb_data);
- if (ret)
- goto out;
- }
- }
-
- ret = 0;
-
-out:
- return ret;
-}
-
-static int packfile_store_for_each_prefixed_object(
- struct odb_source_packed *store,
- const struct odb_for_each_object_options *opts,
- struct odb_source_packed_for_each_object_wrapper_data *data)
-{
- struct packfile_list_entry *e;
- struct multi_pack_index *m;
- bool pack_errors = false;
- int ret;
-
- if (opts->flags)
- BUG("flags unsupported");
-
- store->skip_mru_updates = true;
-
- m = get_multi_pack_index(&store->files->base);
- if (m) {
- ret = for_each_prefixed_object_in_midx(store, m, opts, data);
- if (ret)
- goto out;
- }
-
- for (e = packfile_store_get_packs(store); e; e = e->next) {
- if (e->pack->multi_pack_index)
- continue;
-
- if (open_pack_index(e->pack)) {
- pack_errors = true;
- continue;
- }
-
- if (!e->pack->num_objects)
- continue;
-
- ret = for_each_prefixed_object_in_pack(store, e->pack, opts, data);
- if (ret)
- goto out;
- }
-
- ret = 0;
-
-out:
- store->skip_mru_updates = false;
- if (!ret && pack_errors)
- ret = -1;
- return ret;
-}
-
-int packfile_store_for_each_object(struct odb_source_packed *store,
- const struct object_info *request,
- odb_for_each_object_cb cb,
- void *cb_data,
- const struct odb_for_each_object_options *opts)
-{
- struct odb_source_packed_for_each_object_wrapper_data data = {
- .store = store,
- .request = request,
- .cb = cb,
- .cb_data = cb_data,
- };
- struct packfile_list_entry *e;
- int pack_errors = 0, ret;
-
- if (opts->prefix)
- return packfile_store_for_each_prefixed_object(store, opts, &data);
-
- store->skip_mru_updates = true;
-
- for (e = packfile_store_get_packs(store); e; e = e->next) {
- struct packed_git *p = e->pack;
-
- if ((opts->flags & ODB_FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
- continue;
- if ((opts->flags & ODB_FOR_EACH_OBJECT_PROMISOR_ONLY) &&
- !p->pack_promisor)
- continue;
- if ((opts->flags & ODB_FOR_EACH_OBJECT_SKIP_IN_CORE_KEPT_PACKS) &&
- p->pack_keep_in_core)
- continue;
- if ((opts->flags & ODB_FOR_EACH_OBJECT_SKIP_ON_DISK_KEPT_PACKS) &&
- p->pack_keep)
- continue;
- if (open_pack_index(p)) {
- pack_errors = 1;
- continue;
- }
-
- ret = for_each_object_in_pack(p, packfile_store_for_each_object_wrapper,
- &data, opts->flags);
- if (ret)
- goto out;
- }
-
- ret = 0;
-
-out:
- store->skip_mru_updates = false;
-
- if (!ret && pack_errors)
- ret = -1;
- return ret;
-}
-
static int extend_abbrev_len(const struct object_id *a,
const struct object_id *b,
unsigned *out)
diff --git a/packfile.h b/packfile.h
index e5875e1215..210650923f 100644
--- a/packfile.h
+++ b/packfile.h
@@ -239,21 +239,6 @@ int for_each_object_in_pack(struct packed_git *p,
each_packed_object_fn, void *data,
enum odb_for_each_object_flags flags);
-/*
- * Iterate through all packed objects in the given packfile store and invoke
- * the callback function for each of them. If an object info request is given,
- * then the object info will be read for every individual object and passed to
- * the callback as if `packfile_store_read_object_info()` was called for the
- * object.
- *
- * The flags parameter is a combination of `odb_for_each_object_flags`.
- */
-int packfile_store_for_each_object(struct odb_source_packed *store,
- const struct object_info *request,
- odb_for_each_object_cb cb,
- void *cb_data,
- const struct odb_for_each_object_options *opts);
-
int packfile_store_find_abbrev_len(struct odb_source_packed *store,
const struct object_id *oid,
unsigned min_len,
@@ -366,6 +351,8 @@ extern int do_check_packed_object_crc;
*/
int packed_object_info(struct packed_git *pack,
off_t offset, struct object_info *);
+int packed_object_info_with_index_pos(struct packed_git *p, off_t obj_offset,
+ uint32_t *maybe_index_pos, struct object_info *oi);
void mark_bad_packed_object(struct packed_git *, const struct object_id *);
const struct packed_git *has_packed_and_bad(struct repository *, const struct object_id *);
--
2.54.0.1064.gd145956f57.dirty
^ permalink raw reply related
* [PATCH 09/16] odb/source-packed: wire up `read_object_stream()` callback
From: Patrick Steinhardt @ 2026-06-04 11:25 UTC (permalink / raw)
To: git
In-Reply-To: <20260604-pks-odb-source-packed-v1-0-2e7ab31b4b5c@pks.im>
Wire up the `read_object_stream()` callback for the packed source and
call it in the "files" source via the `odb_source_read_object_stream()`
interface.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
odb/source-files.c | 2 +-
odb/source-packed.c | 16 ++++++++++++++++
packfile.c | 12 ------------
packfile.h | 4 ----
4 files changed, 17 insertions(+), 17 deletions(-)
diff --git a/odb/source-files.c b/odb/source-files.c
index 8cae35d25e..dff69d0e4e 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -67,7 +67,7 @@ static int odb_source_files_read_object_stream(struct odb_read_stream **out,
const struct object_id *oid)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- if (!packfile_store_read_object_stream(out, files->packed, oid) ||
+ if (!odb_source_read_object_stream(out, &files->packed->base, oid) ||
!odb_source_read_object_stream(out, &files->loose->base, oid))
return 0;
return -1;
diff --git a/odb/source-packed.c b/odb/source-packed.c
index f71a194739..23d7149fe3 100644
--- a/odb/source-packed.c
+++ b/odb/source-packed.c
@@ -2,9 +2,11 @@
#include "abspath.h"
#include "chdir-notify.h"
#include "dir.h"
+#include "git-zlib.h"
#include "mergesort.h"
#include "midx.h"
#include "odb/source-packed.h"
+#include "odb/streaming.h"
#include "packfile.h"
int find_pack_entry(struct odb_source_packed *store,
@@ -66,6 +68,19 @@ static int odb_source_packed_read_object_info(struct odb_source *source,
return 0;
}
+static int odb_source_packed_read_object_stream(struct odb_read_stream **out,
+ struct odb_source *source,
+ const struct object_id *oid)
+{
+ struct odb_source_packed *packed = odb_source_packed_downcast(source);
+ struct pack_entry e;
+
+ if (!find_pack_entry(packed, oid, &e))
+ return -1;
+
+ return packfile_read_object_stream(out, oid, e.p, e.offset);
+}
+
void (*report_garbage)(unsigned seen_bits, const char *path);
static void report_helper(const struct string_list *list,
@@ -275,6 +290,7 @@ struct odb_source_packed *odb_source_packed_new(struct odb_source_files *parent)
packed->base.close = odb_source_packed_close;
packed->base.reprepare = odb_source_packed_reprepare;
packed->base.read_object_info = odb_source_packed_read_object_info;
+ packed->base.read_object_stream = odb_source_packed_read_object_stream;
if (!is_absolute_path(parent->base.path))
chdir_notify_register(NULL, odb_source_packed_reparent, packed);
diff --git a/packfile.c b/packfile.c
index 565186e9c7..bcb4b61608 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2741,15 +2741,3 @@ int packfile_read_object_stream(struct odb_read_stream **out,
return 0;
}
-
-int packfile_store_read_object_stream(struct odb_read_stream **out,
- struct odb_source_packed *store,
- const struct object_id *oid)
-{
- struct pack_entry e;
-
- if (!find_pack_entry(store, oid, &e))
- return -1;
-
- return packfile_read_object_stream(out, oid, e.p, e.offset);
-}
diff --git a/packfile.h b/packfile.h
index 2bb6656c3b..e5875e1215 100644
--- a/packfile.h
+++ b/packfile.h
@@ -136,10 +136,6 @@ static inline void repo_for_each_pack_data_next(struct repo_for_each_pack_data *
((p) = (eack_pack_data.entry ? eack_pack_data.entry->pack : NULL)); \
repo_for_each_pack_data_next(&eack_pack_data))
-int packfile_store_read_object_stream(struct odb_read_stream **out,
- struct odb_source_packed *store,
- const struct object_id *oid);
-
/*
* Open the packfile and add it to the store if it isn't yet known. Returns
* either the newly opened packfile or the preexisting packfile. Returns a
--
2.54.0.1064.gd145956f57.dirty
^ permalink raw reply related
* [PATCH 08/16] odb/source-packed: wire up `read_object_info()` callback
From: Patrick Steinhardt @ 2026-06-04 11:25 UTC (permalink / raw)
To: git
In-Reply-To: <20260604-pks-odb-source-packed-v1-0-2e7ab31b4b5c@pks.im>
Move the logic to read object info from a "packed" source into
"odb/source-packed.c" and wire it up as the `read_object_info()`
callback.
Note that we also move around the supporting `find_pack_entry()`, but we
still have to expose it to other callers that exist in "packfile.c".
This will be fixed in subsequent commits though, where all callers in
"packfile.c" will have been moved into "odb/source-packed.c", and at
that point we'll be able to make `find_pack_entry()` file-local again.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
odb/source-files.c | 2 +-
odb/source-packed.c | 60 +++++++++++++++++++++++++++++++++++++++++++
odb/source-packed.h | 6 +++++
packfile.c | 74 ++++++-----------------------------------------------
packfile.h | 15 +++--------
5 files changed, 79 insertions(+), 78 deletions(-)
diff --git a/odb/source-files.c b/odb/source-files.c
index 7b1e0ac565..8cae35d25e 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -55,7 +55,7 @@ static int odb_source_files_read_object_info(struct odb_source *source,
{
struct odb_source_files *files = odb_source_files_downcast(source);
- if (!packfile_store_read_object_info(files->packed, oid, oi, flags) ||
+ if (!odb_source_read_object_info(&files->packed->base, oid, oi, flags) ||
!odb_source_read_object_info(&files->loose->base, oid, oi, flags))
return 0;
diff --git a/odb/source-packed.c b/odb/source-packed.c
index e8e2e5bb48..f71a194739 100644
--- a/odb/source-packed.c
+++ b/odb/source-packed.c
@@ -7,6 +7,65 @@
#include "odb/source-packed.h"
#include "packfile.h"
+int find_pack_entry(struct odb_source_packed *store,
+ const struct object_id *oid,
+ struct pack_entry *e)
+{
+ struct packfile_list_entry *l;
+
+ odb_source_packed_prepare(store);
+ if (store->midx && fill_midx_entry(store->midx, oid, e))
+ return 1;
+
+ for (l = store->packs.head; l; l = l->next) {
+ struct packed_git *p = l->pack;
+
+ if (!p->multi_pack_index && packfile_fill_entry(p, oid, e)) {
+ if (!store->skip_mru_updates)
+ packfile_list_prepend(&store->packs, p);
+ return 1;
+ }
+ }
+
+ return 0;
+}
+
+static int odb_source_packed_read_object_info(struct odb_source *source,
+ const struct object_id *oid,
+ struct object_info *oi,
+ enum object_info_flags flags)
+{
+ struct odb_source_packed *packed = odb_source_packed_downcast(source);
+ struct pack_entry e;
+ int ret;
+
+ /*
+ * In case the first read didn't surface the object, we have to reload
+ * packfiles. This may cause us to discover new packfiles that have
+ * been added since the last time we have prepared the packfile store.
+ */
+ if (flags & OBJECT_INFO_SECOND_READ)
+ odb_source_reprepare(source);
+
+ if (!find_pack_entry(packed, oid, &e))
+ return 1;
+
+ /*
+ * We know that the caller doesn't actually need the
+ * information below, so return early.
+ */
+ if (!oi)
+ return 0;
+
+ ret = packed_object_info(e.p, e.offset, oi);
+ if (ret < 0) {
+ mark_bad_packed_object(e.p, oid);
+ return -1;
+ }
+
+ return 0;
+}
+
void (*report_garbage)(unsigned seen_bits, const char *path);
static void report_helper(const struct string_list *list,
@@ -215,6 +274,7 @@ struct odb_source_packed *odb_source_packed_new(struct odb_source_files *parent)
packed->base.free = odb_source_packed_free;
packed->base.close = odb_source_packed_close;
packed->base.reprepare = odb_source_packed_reprepare;
+ packed->base.read_object_info = odb_source_packed_read_object_info;
if (!is_absolute_path(parent->base.path))
chdir_notify_register(NULL, odb_source_packed_reparent, packed);
diff --git a/odb/source-packed.h b/odb/source-packed.h
index 6645f4f943..aa2e80281b 100644
--- a/odb/source-packed.h
+++ b/odb/source-packed.h
@@ -98,4 +98,10 @@ static inline struct odb_source_packed *odb_source_packed_downcast(struct odb_so
*/
void odb_source_packed_prepare(struct odb_source_packed *source);
+struct pack_entry;
+
+int find_pack_entry(struct odb_source_packed *store,
+ const struct object_id *oid,
+ struct pack_entry *e);
+
#endif
diff --git a/packfile.c b/packfile.c
index 3ee71d7f71..565186e9c7 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1978,9 +1978,9 @@ int is_pack_valid(struct packed_git *p)
return !open_packed_git(p);
}
-static int fill_pack_entry(const struct object_id *oid,
- struct pack_entry *e,
- struct packed_git *p)
+int packfile_fill_entry(struct packed_git *p,
+ const struct object_id *oid,
+ struct pack_entry *e)
{
off_t offset;
@@ -2006,29 +2006,6 @@ static int fill_pack_entry(const struct object_id *oid,
return 1;
}
-static int find_pack_entry(struct odb_source_packed *store,
- const struct object_id *oid,
- struct pack_entry *e)
-{
- struct packfile_list_entry *l;
-
- odb_source_packed_prepare(store);
- if (store->midx && fill_midx_entry(store->midx, oid, e))
- return 1;
-
- for (l = store->packs.head; l; l = l->next) {
- struct packed_git *p = l->pack;
-
- if (!p->multi_pack_index && fill_pack_entry(oid, e, p)) {
- if (!store->skip_mru_updates)
- packfile_list_prepend(&store->packs, p);
- return 1;
- }
- }
-
- return 0;
-}
-
int packfile_store_freshen_object(struct odb_source_packed *store,
const struct object_id *oid)
{
@@ -2045,41 +2022,6 @@ int packfile_store_freshen_object(struct odb_source_packed *store,
return 1;
}
-int packfile_store_read_object_info(struct odb_source_packed *store,
- const struct object_id *oid,
- struct object_info *oi,
- enum object_info_flags flags)
-{
- struct pack_entry e;
- int ret;
-
- /*
- * In case the first read didn't surface the object, we have to reload
- * packfiles. This may cause us to discover new packfiles that have
- * been added since the last time we have prepared the packfile store.
- */
- if (flags & OBJECT_INFO_SECOND_READ)
- odb_source_reprepare(&store->base);
-
- if (!find_pack_entry(store, oid, &e))
- return 1;
-
- /*
- * We know that the caller doesn't actually need the
- * information below, so return early.
- */
- if (!oi)
- return 0;
-
- ret = packed_object_info(e.p, e.offset, oi);
- if (ret < 0) {
- mark_bad_packed_object(e.p, oid);
- return -1;
- }
-
- return 0;
-}
-
static void maybe_invalidate_kept_pack_cache(struct odb_source_packed *store,
unsigned flags)
{
@@ -2136,7 +2078,7 @@ int has_object_pack(struct repository *r, const struct object_id *oid)
odb_prepare_alternates(r->objects);
for (source = r->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- if (!packfile_store_read_object_info(files->packed, oid, NULL, 0))
+ if (!odb_source_read_object_info(&files->packed->base, oid, NULL, 0))
return 1;
}
@@ -2157,7 +2099,7 @@ int has_object_kept_pack(struct repository *r, const struct object_id *oid,
for (; *cache; cache++) {
struct packed_git *p = *cache;
- if (fill_pack_entry(oid, &e, p))
+ if (packfile_fill_entry(p, oid, &e))
return 1;
}
}
@@ -2291,8 +2233,8 @@ static int for_each_prefixed_object_in_midx(
if (data->request) {
struct object_info oi = *data->request;
- ret = packfile_store_read_object_info(store, current,
- &oi, 0);
+ ret = odb_source_read_object_info(&store->base, current,
+ &oi, 0);
if (ret)
goto out;
@@ -2342,7 +2284,7 @@ static int for_each_prefixed_object_in_pack(
if (data->request) {
struct object_info oi = *data->request;
- ret = packfile_store_read_object_info(store, &oid, &oi, 0);
+ ret = odb_source_read_object_info(&store->base, &oid, &oi, 0);
if (ret)
goto out;
diff --git a/packfile.h b/packfile.h
index 1bce805f74..2bb6656c3b 100644
--- a/packfile.h
+++ b/packfile.h
@@ -140,17 +140,6 @@ int packfile_store_read_object_stream(struct odb_read_stream **out,
struct odb_source_packed *store,
const struct object_id *oid);
-/*
- * Try to read the object identified by its ID from the object store and
- * populate the object info with its data. Returns 1 in case the object was
- * not found, 0 if it was and read successfully, and a negative error code in
- * case the object was corrupted.
- */
-int packfile_store_read_object_info(struct odb_source_packed *store,
- const struct object_id *oid,
- struct object_info *oi,
- enum object_info_flags flags);
-
/*
* Open the packfile and add it to the store if it isn't yet known. Returns
* either the newly opened packfile or the preexisting packfile. Returns a
@@ -352,6 +341,10 @@ off_t nth_packed_object_offset(const struct packed_git *, uint32_t n);
*/
off_t find_pack_entry_one(const struct object_id *oid, struct packed_git *);
+int packfile_fill_entry(struct packed_git *p,
+ const struct object_id *oid,
+ struct pack_entry *e);
+
int is_pack_valid(struct packed_git *);
void *unpack_entry(struct repository *r, struct packed_git *, off_t, enum object_type *, unsigned long *);
unsigned long unpack_object_header_buffer(const unsigned char *buf, unsigned long len, enum object_type *type, size_t *sizep);
--
2.54.0.1064.gd145956f57.dirty
^ permalink raw reply related
* [PATCH 07/16] packfile: use higher-level interface to implement `has_object_pack()`
From: Patrick Steinhardt @ 2026-06-04 11:25 UTC (permalink / raw)
To: git
In-Reply-To: <20260604-pks-odb-source-packed-v1-0-2e7ab31b4b5c@pks.im>
In `has_object_pack()` we're checking whether a specific object exists
as part of a packfile. This is done by calling the low-level function
`find_pack_entry()`, but this function will eventually be moved into
"odb/source-packed.c" and made file-local.
Refactor the code to use `packfile_store_read_object_info()` instead.
This refactoring is functionally equivalent as that function will call
`find_pack_entry()` itself and then return immediately when it ain't got
no object info pointer as parameter.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
packfile.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/packfile.c b/packfile.c
index 902b7f70f2..3ee71d7f71 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2132,14 +2132,12 @@ struct packed_git **packfile_store_get_kept_pack_cache(struct odb_source_packed
int has_object_pack(struct repository *r, const struct object_id *oid)
{
struct odb_source *source;
- struct pack_entry e;
odb_prepare_alternates(r->objects);
for (source = r->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- int ret = find_pack_entry(files->packed, oid, &e);
- if (ret)
- return ret;
+ if (!packfile_store_read_object_info(files->packed, oid, NULL, 0))
+ return 1;
}
return 0;
--
2.54.0.1064.gd145956f57.dirty
^ permalink raw reply related
* [PATCH 06/16] odb/source-packed: wire up `reprepare()` callback
From: Patrick Steinhardt @ 2026-06-04 11:25 UTC (permalink / raw)
To: git
In-Reply-To: <20260604-pks-odb-source-packed-v1-0-2e7ab31b4b5c@pks.im>
Move the logic to prepare and reprepare the "packed" source into
"odb/source-packed.c" and wire it up as the `reprepare()` callback.
Note that "preparing" a source is not yet generic. Eventually, it would
probably make sense to turn the existing `reprepare()` callback into a
`prepare()` callback with an optional flag to force re-preparing. But
this step will be handled in a separate patch series.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
builtin/grep.c | 2 +-
midx.c | 2 +-
odb/source-files.c | 2 +-
odb/source-packed.c | 157 +++++++++++++++++++++++++++++++++++++++++++++++++++
odb/source-packed.h | 9 +++
packfile.c | 160 +---------------------------------------------------
packfile.h | 17 ------
7 files changed, 172 insertions(+), 177 deletions(-)
diff --git a/builtin/grep.c b/builtin/grep.c
index 6a09571903..8080d1bf5e 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -1363,7 +1363,7 @@ int cmd_grep(int argc,
odb_prepare_alternates(the_repository->objects);
for (source = the_repository->objects->sources; source; source = source->next) {
struct odb_source_files *files = odb_source_files_downcast(source);
- packfile_store_prepare(files->packed);
+ odb_source_packed_prepare(files->packed);
}
}
diff --git a/midx.c b/midx.c
index efbfbb13f4..00bbd137b2 100644
--- a/midx.c
+++ b/midx.c
@@ -102,7 +102,7 @@ static int midx_read_object_offsets(const unsigned char *chunk_start,
struct multi_pack_index *get_multi_pack_index(struct odb_source *source)
{
struct odb_source_files *files = odb_source_files_downcast(source);
- packfile_store_prepare(files->packed);
+ odb_source_packed_prepare(files->packed);
return files->packed->midx;
}
diff --git a/odb/source-files.c b/odb/source-files.c
index 9b0fa9ccdc..7b1e0ac565 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -45,7 +45,7 @@ static void odb_source_files_reprepare(struct odb_source *source)
{
struct odb_source_files *files = odb_source_files_downcast(source);
odb_source_reprepare(&files->loose->base);
- packfile_store_reprepare(files->packed);
+ odb_source_reprepare(&files->packed->base);
}
static int odb_source_files_read_object_info(struct odb_source *source,
diff --git a/odb/source-packed.c b/odb/source-packed.c
index 74805be1dd..e8e2e5bb48 100644
--- a/odb/source-packed.c
+++ b/odb/source-packed.c
@@ -1,10 +1,166 @@
#include "git-compat-util.h"
#include "abspath.h"
#include "chdir-notify.h"
+#include "dir.h"
+#include "mergesort.h"
#include "midx.h"
#include "odb/source-packed.h"
#include "packfile.h"
+void (*report_garbage)(unsigned seen_bits, const char *path);
+
+static void report_helper(const struct string_list *list,
+ int seen_bits, int first, int last)
+{
+ if (seen_bits == (PACKDIR_FILE_PACK|PACKDIR_FILE_IDX))
+ return;
+
+ for (; first < last; first++)
+ report_garbage(seen_bits, list->items[first].string);
+}
+
+static void report_pack_garbage(struct string_list *list)
+{
+ int baselen = -1, first = 0, seen_bits = 0;
+
+ if (!report_garbage)
+ return;
+
+ string_list_sort(list);
+
+ for (size_t i = 0; i < list->nr; i++) {
+ const char *path = list->items[i].string;
+ if (baselen != -1 &&
+ strncmp(path, list->items[first].string, baselen)) {
+ report_helper(list, seen_bits, first, i);
+ baselen = -1;
+ seen_bits = 0;
+ }
+ if (baselen == -1) {
+ const char *dot = strrchr(path, '.');
+ if (!dot) {
+ report_garbage(PACKDIR_FILE_GARBAGE, path);
+ continue;
+ }
+ baselen = dot - path + 1;
+ first = i;
+ }
+ if (!strcmp(path + baselen, "pack"))
+ seen_bits |= 1;
+ else if (!strcmp(path + baselen, "idx"))
+ seen_bits |= 2;
+ }
+ report_helper(list, seen_bits, first, list->nr);
+}
+
+struct prepare_pack_data {
+ struct odb_source *source;
+ struct string_list *garbage;
+};
+
+static void prepare_pack(const char *full_name, size_t full_name_len,
+ const char *file_name, void *_data)
+{
+ struct prepare_pack_data *data = (struct prepare_pack_data *)_data;
+ struct odb_source_files *files = odb_source_files_downcast(data->source);
+ size_t base_len = full_name_len;
+
+ if (strip_suffix_mem(full_name, &base_len, ".idx") &&
+ !(files->packed->midx &&
+ midx_contains_pack(files->packed->midx, file_name))) {
+ char *trimmed_path = xstrndup(full_name, full_name_len);
+ packfile_store_load_pack(files->packed,
+ trimmed_path, data->source->local);
+ free(trimmed_path);
+ }
+
+ if (!report_garbage)
+ return;
+
+ if (!strcmp(file_name, "multi-pack-index") ||
+ !strcmp(file_name, "multi-pack-index.d"))
+ return;
+ if (starts_with(file_name, "multi-pack-index") &&
+ (ends_with(file_name, ".bitmap") || ends_with(file_name, ".rev")))
+ return;
+ if (ends_with(file_name, ".idx") ||
+ ends_with(file_name, ".rev") ||
+ ends_with(file_name, ".pack") ||
+ ends_with(file_name, ".bitmap") ||
+ ends_with(file_name, ".keep") ||
+ ends_with(file_name, ".promisor") ||
+ ends_with(file_name, ".mtimes"))
+ string_list_append(data->garbage, full_name);
+ else
+ report_garbage(PACKDIR_FILE_GARBAGE, full_name);
+}
+
+static void prepare_packed_git_one(struct odb_source *source)
+{
+ struct string_list garbage = STRING_LIST_INIT_DUP;
+ struct prepare_pack_data data = {
+ .source = source,
+ .garbage = &garbage,
+ };
+
+ for_each_file_in_pack_dir(source->path, prepare_pack, &data);
+
+ report_pack_garbage(data.garbage);
+ string_list_clear(data.garbage, 0);
+}
+
+DEFINE_LIST_SORT(static, sort_packs, struct packfile_list_entry, next);
+
+static int sort_pack(const struct packfile_list_entry *a,
+ const struct packfile_list_entry *b)
+{
+ int st;
+
+ /*
+ * Local packs tend to contain objects specific to our
+ * variant of the project than remote ones. In addition,
+ * remote ones could be on a network mounted filesystem.
+ * Favor local ones for these reasons.
+ */
+ st = a->pack->pack_local - b->pack->pack_local;
+ if (st)
+ return -st;
+
+ /*
+ * Younger packs tend to contain more recent objects,
+ * and more recent objects tend to get accessed more
+ * often.
+ */
+ if (a->pack->mtime < b->pack->mtime)
+ return 1;
+ else if (a->pack->mtime == b->pack->mtime)
+ return 0;
+ return -1;
+}
+
+void odb_source_packed_prepare(struct odb_source_packed *source)
+{
+ if (source->initialized)
+ return;
+
+ prepare_multi_pack_index_one(&source->files->base);
+ prepare_packed_git_one(&source->files->base);
+
+ sort_packs(&source->packs.head, sort_pack);
+ for (struct packfile_list_entry *e = source->packs.head; e; e = e->next)
+ if (!e->next)
+ source->packs.tail = e;
+
+ source->initialized = true;
+}
+
+static void odb_source_packed_reprepare(struct odb_source *source)
+{
+ struct odb_source_packed *packed = odb_source_packed_downcast(source);
+ packed->initialized = false;
+ odb_source_packed_prepare(packed);
+}
+
static void odb_source_packed_reparent(const char *name UNUSED,
const char *old_cwd,
const char *new_cwd,
@@ -58,6 +214,7 @@ struct odb_source_packed *odb_source_packed_new(struct odb_source_files *parent)
packed->base.free = odb_source_packed_free;
packed->base.close = odb_source_packed_close;
+ packed->base.reprepare = odb_source_packed_reprepare;
if (!is_absolute_path(parent->base.path))
chdir_notify_register(NULL, odb_source_packed_reparent, packed);
diff --git a/odb/source-packed.h b/odb/source-packed.h
index abe3310069..6645f4f943 100644
--- a/odb/source-packed.h
+++ b/odb/source-packed.h
@@ -89,4 +89,13 @@ static inline struct odb_source_packed *odb_source_packed_downcast(struct odb_so
return container_of(source, struct odb_source_packed, base);
}
+/*
+ * Prepare the source by loading packfiles and multi-pack indices for
+ * all alternates. This becomes a no-op if the source is already prepared.
+ *
+ * It shouldn't typically be necessary to call this function directly, as
+ * functions that access the source know to prepare it.
+ */
+void odb_source_packed_prepare(struct odb_source_packed *source);
+
#endif
diff --git a/packfile.c b/packfile.c
index 3cff3b8ffa..902b7f70f2 100644
--- a/packfile.c
+++ b/packfile.c
@@ -8,7 +8,6 @@
#include "pack.h"
#include "repository.h"
#include "dir.h"
-#include "mergesort.h"
#include "packfile.h"
#include "delta.h"
#include "hash-lookup.h"
@@ -895,52 +894,6 @@ struct packed_git *packfile_store_load_pack(struct odb_source_packed *store,
return p;
}
-void (*report_garbage)(unsigned seen_bits, const char *path);
-
-static void report_helper(const struct string_list *list,
- int seen_bits, int first, int last)
-{
- if (seen_bits == (PACKDIR_FILE_PACK|PACKDIR_FILE_IDX))
- return;
-
- for (; first < last; first++)
- report_garbage(seen_bits, list->items[first].string);
-}
-
-static void report_pack_garbage(struct string_list *list)
-{
- int i, baselen = -1, first = 0, seen_bits = 0;
-
- if (!report_garbage)
- return;
-
- string_list_sort(list);
-
- for (i = 0; i < list->nr; i++) {
- const char *path = list->items[i].string;
- if (baselen != -1 &&
- strncmp(path, list->items[first].string, baselen)) {
- report_helper(list, seen_bits, first, i);
- baselen = -1;
- seen_bits = 0;
- }
- if (baselen == -1) {
- const char *dot = strrchr(path, '.');
- if (!dot) {
- report_garbage(PACKDIR_FILE_GARBAGE, path);
- continue;
- }
- baselen = dot - path + 1;
- first = i;
- }
- if (!strcmp(path + baselen, "pack"))
- seen_bits |= 1;
- else if (!strcmp(path + baselen, "idx"))
- seen_bits |= 2;
- }
- report_helper(list, seen_bits, first, list->nr);
-}
-
void for_each_file_in_pack_subdir(const char *objdir,
const char *subdir,
each_file_in_pack_dir_fn fn,
@@ -983,116 +936,9 @@ void for_each_file_in_pack_dir(const char *objdir,
for_each_file_in_pack_subdir(objdir, NULL, fn, data);
}
-struct prepare_pack_data {
- struct odb_source *source;
- struct string_list *garbage;
-};
-
-static void prepare_pack(const char *full_name, size_t full_name_len,
- const char *file_name, void *_data)
-{
- struct prepare_pack_data *data = (struct prepare_pack_data *)_data;
- struct odb_source_files *files = odb_source_files_downcast(data->source);
- size_t base_len = full_name_len;
-
- if (strip_suffix_mem(full_name, &base_len, ".idx") &&
- !(files->packed->midx &&
- midx_contains_pack(files->packed->midx, file_name))) {
- char *trimmed_path = xstrndup(full_name, full_name_len);
- packfile_store_load_pack(files->packed,
- trimmed_path, data->source->local);
- free(trimmed_path);
- }
-
- if (!report_garbage)
- return;
-
- if (!strcmp(file_name, "multi-pack-index") ||
- !strcmp(file_name, "multi-pack-index.d"))
- return;
- if (starts_with(file_name, "multi-pack-index") &&
- (ends_with(file_name, ".bitmap") || ends_with(file_name, ".rev")))
- return;
- if (ends_with(file_name, ".idx") ||
- ends_with(file_name, ".rev") ||
- ends_with(file_name, ".pack") ||
- ends_with(file_name, ".bitmap") ||
- ends_with(file_name, ".keep") ||
- ends_with(file_name, ".promisor") ||
- ends_with(file_name, ".mtimes"))
- string_list_append(data->garbage, full_name);
- else
- report_garbage(PACKDIR_FILE_GARBAGE, full_name);
-}
-
-static void prepare_packed_git_one(struct odb_source *source)
-{
- struct string_list garbage = STRING_LIST_INIT_DUP;
- struct prepare_pack_data data = {
- .source = source,
- .garbage = &garbage,
- };
-
- for_each_file_in_pack_dir(source->path, prepare_pack, &data);
-
- report_pack_garbage(data.garbage);
- string_list_clear(data.garbage, 0);
-}
-
-DEFINE_LIST_SORT(static, sort_packs, struct packfile_list_entry, next);
-
-static int sort_pack(const struct packfile_list_entry *a,
- const struct packfile_list_entry *b)
-{
- int st;
-
- /*
- * Local packs tend to contain objects specific to our
- * variant of the project than remote ones. In addition,
- * remote ones could be on a network mounted filesystem.
- * Favor local ones for these reasons.
- */
- st = a->pack->pack_local - b->pack->pack_local;
- if (st)
- return -st;
-
- /*
- * Younger packs tend to contain more recent objects,
- * and more recent objects tend to get accessed more
- * often.
- */
- if (a->pack->mtime < b->pack->mtime)
- return 1;
- else if (a->pack->mtime == b->pack->mtime)
- return 0;
- return -1;
-}
-
-void packfile_store_prepare(struct odb_source_packed *store)
-{
- if (store->initialized)
- return;
-
- prepare_multi_pack_index_one(&store->files->base);
- prepare_packed_git_one(&store->files->base);
-
- sort_packs(&store->packs.head, sort_pack);
- for (struct packfile_list_entry *e = store->packs.head; e; e = e->next)
- if (!e->next)
- store->packs.tail = e;
-
- store->initialized = true;
-}
-
-void packfile_store_reprepare(struct odb_source_packed *store)
-{
- store->initialized = false;
- packfile_store_prepare(store);
-}
-
struct packfile_list_entry *packfile_store_get_packs(struct odb_source_packed *store)
{
- packfile_store_prepare(store);
+ odb_source_packed_prepare(store);
if (store->midx) {
struct multi_pack_index *m = store->midx;
@@ -2166,7 +2012,7 @@ static int find_pack_entry(struct odb_source_packed *store,
{
struct packfile_list_entry *l;
- packfile_store_prepare(store);
+ odb_source_packed_prepare(store);
if (store->midx && fill_midx_entry(store->midx, oid, e))
return 1;
@@ -2213,7 +2059,7 @@ int packfile_store_read_object_info(struct odb_source_packed *store,
* been added since the last time we have prepared the packfile store.
*/
if (flags & OBJECT_INFO_SECOND_READ)
- packfile_store_reprepare(store);
+ odb_source_reprepare(&store->base);
if (!find_pack_entry(store, oid, &e))
return 1;
diff --git a/packfile.h b/packfile.h
index 23af47f0c1..1bce805f74 100644
--- a/packfile.h
+++ b/packfile.h
@@ -67,23 +67,6 @@ void packfile_list_append(struct packfile_list *list, struct packed_git *pack);
struct packed_git *packfile_list_find_oid(struct packfile_list_entry *packs,
const struct object_id *oid);
-/*
- * Prepare the packfile store by loading packfiles and multi-pack indices for
- * all alternates. This becomes a no-op if the store is already prepared.
- *
- * It shouldn't typically be necessary to call this function directly, as
- * functions that access the store know to prepare it.
- */
-void packfile_store_prepare(struct odb_source_packed *store);
-
-/*
- * Clear the packfile caches and try to look up any new packfiles that have
- * appeared since last preparing the packfiles store.
- *
- * This function must be called under the `odb_read_lock()`.
- */
-void packfile_store_reprepare(struct odb_source_packed *store);
-
/*
* Add the pack to the store so that contained objects become accessible via
* the store. This moves ownership into the store.
--
2.54.0.1064.gd145956f57.dirty
^ permalink raw reply related
* [PATCH 05/16] odb/source-packed: wire up `close()` callback
From: Patrick Steinhardt @ 2026-06-04 11:25 UTC (permalink / raw)
To: git
In-Reply-To: <20260604-pks-odb-source-packed-v1-0-2e7ab31b4b5c@pks.im>
Wire up a new `close()` callback for the packed source and call it from
the "files" source via the generic `odb_source_close()` interface.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
odb/source-files.c | 2 +-
odb/source-packed.c | 16 ++++++++++++++++
packfile.c | 12 ------------
packfile.h | 6 ------
4 files changed, 17 insertions(+), 19 deletions(-)
diff --git a/odb/source-files.c b/odb/source-files.c
index 3608808e7c..9b0fa9ccdc 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -38,7 +38,7 @@ static void odb_source_files_close(struct odb_source *source)
{
struct odb_source_files *files = odb_source_files_downcast(source);
odb_source_close(&files->loose->base);
- packfile_store_close(files->packed);
+ odb_source_close(&files->packed->base);
}
static void odb_source_files_reprepare(struct odb_source *source)
diff --git a/odb/source-packed.c b/odb/source-packed.c
index f81a990cbd..74805be1dd 100644
--- a/odb/source-packed.c
+++ b/odb/source-packed.c
@@ -1,6 +1,7 @@
#include "git-compat-util.h"
#include "abspath.h"
#include "chdir-notify.h"
+#include "midx.h"
#include "odb/source-packed.h"
#include "packfile.h"
@@ -16,6 +17,20 @@ static void odb_source_packed_reparent(const char *name UNUSED,
packed->base.path = path;
}
+static void odb_source_packed_close(struct odb_source *source)
+{
+ struct odb_source_packed *packed = odb_source_packed_downcast(source);
+
+ for (struct packfile_list_entry *e = packed->packs.head; e; e = e->next) {
+ if (e->pack->do_not_close)
+ BUG("want to close pack marked 'do-not-close'");
+ close_pack(e->pack);
+ }
+ if (packed->midx)
+ close_midx(packed->midx);
+ packed->midx = NULL;
+}
+
static void odb_source_packed_free(struct odb_source *source)
{
struct odb_source_packed *packed = odb_source_packed_downcast(source);
@@ -42,6 +57,7 @@ struct odb_source_packed *odb_source_packed_new(struct odb_source_files *parent)
strmap_init(&packed->packs_by_path);
packed->base.free = odb_source_packed_free;
+ packed->base.close = odb_source_packed_close;
if (!is_absolute_path(parent->base.path))
chdir_notify_register(NULL, odb_source_packed_reparent, packed);
diff --git a/packfile.c b/packfile.c
index 8fefc4358e..3cff3b8ffa 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2832,18 +2832,6 @@ int parse_pack_header_option(const char *in, unsigned char *out, unsigned int *l
return 0;
}
-void packfile_store_close(struct odb_source_packed *store)
-{
- for (struct packfile_list_entry *e = store->packs.head; e; e = e->next) {
- if (e->pack->do_not_close)
- BUG("want to close pack marked 'do-not-close'");
- close_pack(e->pack);
- }
- if (store->midx)
- close_midx(store->midx);
- store->midx = NULL;
-}
-
struct odb_packed_read_stream {
struct odb_read_stream base;
struct packed_git *pack;
diff --git a/packfile.h b/packfile.h
index d6814b564e..23af47f0c1 100644
--- a/packfile.h
+++ b/packfile.h
@@ -67,12 +67,6 @@ void packfile_list_append(struct packfile_list *list, struct packed_git *pack);
struct packed_git *packfile_list_find_oid(struct packfile_list_entry *packs,
const struct object_id *oid);
-/*
- * Close all packfiles associated with this store. The packfiles won't be
- * free'd, so they can be re-opened at a later point in time.
- */
-void packfile_store_close(struct odb_source_packed *store);
-
/*
* Prepare the packfile store by loading packfiles and multi-pack indices for
* all alternates. This becomes a no-op if the store is already prepared.
--
2.54.0.1064.gd145956f57.dirty
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox